com.netflix.genie.client

Class JobClient

java.lang.Object

com.netflix.genie.client.JobClient

public class JobClient
extends java.lang.Object

Client library for the Job Service.

Since:

3.0.0

Constructor Summary

Constructors

Constructor and Description
JobClient(@NotNull retrofit2.Retrofit retrofit, int maxStatusRetries) Constructor.
JobClient(@NotEmpty java.lang.String url, java.util.List<okhttp3.Interceptor> interceptors, GenieNetworkConfiguration genieNetworkConfiguration) Deprecated. Use JobClient(Retrofit, int)

Method Summary

Modifier and Type	Method and Description
Job	getJob(java.lang.String jobId) Method to get a job from Genie.
java.util.List<Application>	getJobApplications(java.lang.String jobId) Method to get the Applications for the job.
Cluster	getJobCluster(java.lang.String jobId) Method to get the cluster on which the job executes.
Command	getJobCommand(java.lang.String jobId) Method to get the command on which the job executes.
JobExecution	getJobExecution(java.lang.String jobId) Method to get the Job Execution information for the job.
JobMetadata	getJobMetadata(java.lang.String jobId) Method to get the metadata information for the job.
java.io.InputStream	getJobOutputFile(java.lang.String jobId, java.lang.String outputFilePath) Method to fetch an output file for a job from Genie.
java.io.InputStream	getJobOutputFile(java.lang.String jobId, java.lang.String outputFilePath, java.lang.Long rangeStart, java.lang.Long rangeEnd) Method to fetch an output file for a job from Genie and accepting an range of bytes to return.
JobRequest	getJobRequest(java.lang.String jobId) Method to get the Job Request for the job.
java.util.List<JobSearchResult>	getJobs() Method to get a list of all the jobs.
java.util.List<JobSearchResult>	getJobs(java.lang.String id, java.lang.String name, java.lang.String user, java.util.Set<java.lang.String> statuses, java.util.Set<java.lang.String> tags, java.lang.String clusterName, java.lang.String clusterId, java.lang.String commandName, java.lang.String commandId, java.lang.Long minStarted, java.lang.Long maxStarted, java.lang.Long minFinished, java.lang.Long maxFinished) Deprecated.
java.util.List<JobSearchResult>	getJobs(java.lang.String id, java.lang.String name, java.lang.String user, java.util.Set<java.lang.String> statuses, java.util.Set<java.lang.String> tags, java.lang.String clusterName, java.lang.String clusterId, java.lang.String commandName, java.lang.String commandId, java.lang.Long minStarted, java.lang.Long maxStarted, java.lang.Long minFinished, java.lang.Long maxFinished, java.lang.String grouping, java.lang.String groupingInstance) Deprecated. Use getJobs(String, String, String, Set, Set, String, String, String, String, Long, Long, Long, Long, String, String, Integer, SortAttribute, SortDirection, Integer)
java.util.List<JobSearchResult>	getJobs(java.lang.String id, java.lang.String name, java.lang.String user, java.util.Set<java.lang.String> statuses, java.util.Set<java.lang.String> tags, java.lang.String clusterName, java.lang.String clusterId, java.lang.String commandName, java.lang.String commandId, java.lang.Long minStarted, java.lang.Long maxStarted, java.lang.Long minFinished, java.lang.Long maxFinished, java.lang.String grouping, java.lang.String groupingInstance, @Min(value=1L) java.lang.Integer pageSize, SortAttribute sortAttribute, SortDirection sortDirection, @Min(value=0L) java.lang.Integer pageIndex) Method to get a list of all the jobs from Genie for the query parameters specified.
JobStatus	getJobStatus(java.lang.String jobId) Method to fetch the status of a job.
java.io.InputStream	getJobStderr(java.lang.String jobId) Method to fetch the stderr of a job from Genie.
java.io.InputStream	getJobStderr(java.lang.String jobId, java.lang.Long rangeStart, java.lang.Long rangeEnd) Method to fetch the stderr of a job from Genie.
java.io.InputStream	getJobStdout(java.lang.String jobId) Method to fetch the stdout of a job from Genie.
java.io.InputStream	getJobStdout(java.lang.String jobId, java.lang.Long rangeStart, java.lang.Long rangeEnd) Method to fetch the stdout of a job from Genie.
void	killJob(java.lang.String jobId) Method to send a kill job request to Genie.
java.lang.String	submitJob(JobRequest jobRequest) Submit a job to genie using the jobRequest provided.
java.lang.String	submitJobWithAttachments(JobRequest jobRequest, java.util.Map<java.lang.String,java.io.InputStream> attachments) Submit a job to genie using the jobRequest and attachments provided.
JobStatus	waitForCompletion(java.lang.String jobId, long blockTimeout) Wait for job to complete, until the given timeout.
JobStatus	waitForCompletion(java.lang.String jobId, long blockTimeout, long pollTime) Wait for job to complete, until the given timeout.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobClient

public JobClient(@NotNull
                 @NotNull retrofit2.Retrofit retrofit,
                 int maxStatusRetries)

Constructor.

Parameters:

retrofit - The configured Retrofit client to a Genie server

maxStatusRetries - The maximum number of retries to check for job status

JobClient

@Deprecated
public JobClient(@NotEmpty
                             @NotEmpty java.lang.String url,
                             @Nullable
                             java.util.List<okhttp3.Interceptor> interceptors,
                             @Nullable
                             GenieNetworkConfiguration genieNetworkConfiguration)
                      throws GenieClientException

Deprecated. Use JobClient(Retrofit, int)

Constructor.

Parameters:

url - The endpoint URL of the Genie API. Not null or empty

interceptors - Any interceptors to configure the client with, can include security ones

genieNetworkConfiguration - The network configuration parameters. Could be null

Throws:

GenieClientException - On error

Method Detail

submitJob

public java.lang.String submitJob(JobRequest jobRequest)
                           throws java.io.IOException,
                                  GenieClientException

Submit a job to genie using the jobRequest provided.

Parameters:

jobRequest - A job request containing all the details for running a job.

Returns:

jobId The id of the job submitted.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

submitJobWithAttachments

public java.lang.String submitJobWithAttachments(JobRequest jobRequest,
                                                 java.util.Map<java.lang.String,java.io.InputStream> attachments)
                                          throws java.io.IOException,
                                                 GenieClientException

Submit a job to genie using the jobRequest and attachments provided.

Parameters:

jobRequest - A job request containing all the details for running a job.

attachments - A map of filenames/input-streams needed to be sent to the server as attachments.

Returns:

jobId The id of the job submitted.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobs

public java.util.List<JobSearchResult> getJobs()
                                        throws java.io.IOException,
                                               GenieClientException

Method to get a list of all the jobs.

Returns:

A list of jobs.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobs

@Deprecated
public java.util.List<JobSearchResult> getJobs(java.lang.String id,
                                                           java.lang.String name,
                                                           java.lang.String user,
                                                           java.util.Set<java.lang.String> statuses,
                                                           java.util.Set<java.lang.String> tags,
                                                           java.lang.String clusterName,
                                                           java.lang.String clusterId,
                                                           java.lang.String commandName,
                                                           java.lang.String commandId,
                                                           java.lang.Long minStarted,
                                                           java.lang.Long maxStarted,
                                                           java.lang.Long minFinished,
                                                           java.lang.Long maxFinished)
                                                    throws java.io.IOException,
                                                           GenieClientException

Deprecated.

Method to get a list of all the jobs from Genie for the query parameters specified.

Deprecated: For new search fields

Parameters:

id - id for job

name - name of job (can be a SQL-style pattern such as HIVE%)

user - user who submitted job

statuses - statuses of jobs to find

tags - tags for the job

clusterName - the name of the cluster

clusterId - the id of the cluster

commandName - the name of the command run by the job

commandId - the id of the command run by the job

minStarted - The time which the job had to start after in order to be return (inclusive)

maxStarted - The time which the job had to start before in order to be returned (exclusive)

minFinished - The time which the job had to finish after in order to be return (inclusive)

maxFinished - The time which the job had to finish before in order to be returned (exclusive)

Returns:

A list of jobs.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

See Also:

getJobs( String, String, String, Set, Set, String, String, String, String, Long, Long, Long, Long, String, String )

getJobs

public java.util.List<JobSearchResult> getJobs(@Nullable
                                               java.lang.String id,
                                               @Nullable
                                               java.lang.String name,
                                               @Nullable
                                               java.lang.String user,
                                               @Nullable
                                               java.util.Set<java.lang.String> statuses,
                                               @Nullable
                                               java.util.Set<java.lang.String> tags,
                                               @Nullable
                                               java.lang.String clusterName,
                                               @Nullable
                                               java.lang.String clusterId,
                                               @Nullable
                                               java.lang.String commandName,
                                               @Nullable
                                               java.lang.String commandId,
                                               @Nullable
                                               java.lang.Long minStarted,
                                               @Nullable
                                               java.lang.Long maxStarted,
                                               @Nullable
                                               java.lang.Long minFinished,
                                               @Nullable
                                               java.lang.Long maxFinished,
                                               @Nullable
                                               java.lang.String grouping,
                                               @Nullable
                                               java.lang.String groupingInstance)
                                        throws java.io.IOException,
                                               GenieClientException

Deprecated. Use getJobs(String, String, String, Set, Set, String, String, String, String, Long, Long, Long, Long, String, String, Integer, SortAttribute, SortDirection, Integer)

Method to get a list of all the jobs from Genie for the query parameters specified.

Parameters:

id - id for job

name - name of job (can be a SQL-style pattern such as HIVE%)

user - user who submitted job

statuses - statuses of jobs to find

tags - tags for the job

clusterName - the name of the cluster

clusterId - the id of the cluster

commandName - the name of the command run by the job

commandId - the id of the command run by the job

minStarted - The time which the job had to start after in order to be return (inclusive)

maxStarted - The time which the job had to start before in order to be returned (exclusive)

minFinished - The time which the job had to finish after in order to be return (inclusive)

maxFinished - The time which the job had to finish before in order to be returned (exclusive)

grouping - The grouping the job should be a member of

groupingInstance - The grouping instance the job should be a member of

Returns:

A list of jobs.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobs

public java.util.List<JobSearchResult> getJobs(@Nullable
                                               java.lang.String id,
                                               @Nullable
                                               java.lang.String name,
                                               @Nullable
                                               java.lang.String user,
                                               @Nullable
                                               java.util.Set<java.lang.String> statuses,
                                               @Nullable
                                               java.util.Set<java.lang.String> tags,
                                               @Nullable
                                               java.lang.String clusterName,
                                               @Nullable
                                               java.lang.String clusterId,
                                               @Nullable
                                               java.lang.String commandName,
                                               @Nullable
                                               java.lang.String commandId,
                                               @Nullable
                                               java.lang.Long minStarted,
                                               @Nullable
                                               java.lang.Long maxStarted,
                                               @Nullable
                                               java.lang.Long minFinished,
                                               @Nullable
                                               java.lang.Long maxFinished,
                                               @Nullable
                                               java.lang.String grouping,
                                               @Nullable
                                               java.lang.String groupingInstance,
                                               @Nullable @Min(value=1L)
                                               @Min(value=1L) java.lang.Integer pageSize,
                                               @Nullable
                                               SortAttribute sortAttribute,
                                               @Nullable
                                               SortDirection sortDirection,
                                               @Nullable @Min(value=0L)
                                               @Min(value=0L) java.lang.Integer pageIndex)
                                        throws java.io.IOException,
                                               GenieClientException

Method to get a list of all the jobs from Genie for the query parameters specified.

Parameters:

id - id for job

name - name of job (can be a SQL-style pattern such as HIVE%)

user - user who submitted job

statuses - statuses of jobs to find

tags - tags for the job

clusterName - the name of the cluster

clusterId - the id of the cluster

commandName - the name of the command run by the job

commandId - the id of the command run by the job

minStarted - The time which the job had to start after in order to be return (inclusive)

maxStarted - The time which the job had to start before in order to be returned (exclusive)

minFinished - The time which the job had to finish after in order to be return (inclusive)

maxFinished - The time which the job had to finish before in order to be returned (exclusive)

grouping - The grouping the job should be a member of

groupingInstance - The grouping instance the job should be a member of

pageSize - The maximum number of results returned

sortAttribute - The entity attribute used to sort

sortDirection - The sort direction

pageIndex - The page index

Returns:

A list of jobs.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJob

public Job getJob(java.lang.String jobId)
           throws java.io.IOException,
                  GenieClientException

Method to get a job from Genie.

Parameters:

jobId - The id of the job to get.

Returns:

The job details.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobCluster

public Cluster getJobCluster(java.lang.String jobId)
                      throws java.io.IOException,
                             GenieClientException

Method to get the cluster on which the job executes.

Parameters:

jobId - The id of the job.

Returns:

The cluster object.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobCommand

public Command getJobCommand(java.lang.String jobId)
                      throws java.io.IOException,
                             GenieClientException

Method to get the command on which the job executes.

Parameters:

jobId - The id of the job.

Returns:

The command object.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobRequest

public JobRequest getJobRequest(java.lang.String jobId)
                         throws java.io.IOException,
                                GenieClientException

Method to get the Job Request for the job.

Parameters:

jobId - The id of the job.

Returns:

The job requests object.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobExecution

public JobExecution getJobExecution(java.lang.String jobId)
                             throws java.io.IOException,
                                    GenieClientException

Method to get the Job Execution information for the job.

Parameters:

jobId - The id of the job.

Returns:

The job execution object.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobMetadata

public JobMetadata getJobMetadata(java.lang.String jobId)
                           throws java.io.IOException,
                                  GenieClientException

Method to get the metadata information for the job.

Parameters:

jobId - The id of the job.

Returns:

The metadata object.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobApplications

public java.util.List<Application> getJobApplications(java.lang.String jobId)
                                               throws java.io.IOException,
                                                      GenieClientException

Method to get the Applications for the job.

Parameters:

jobId - The id of the job.

Returns:

The list of Applications.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobStdout

public java.io.InputStream getJobStdout(java.lang.String jobId)
                                 throws java.io.IOException,
                                        GenieClientException

Method to fetch the stdout of a job from Genie.

Parameters:

jobId - The id of the job whose output is desired.

Returns:

An input stream to the output contents.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobStdout

public java.io.InputStream getJobStdout(java.lang.String jobId,
                                        @Nullable
                                        java.lang.Long rangeStart,
                                        @Nullable
                                        java.lang.Long rangeEnd)
                                 throws java.io.IOException,
                                        GenieClientException

Method to fetch the stdout of a job from Genie.

Range Logic:

rangeStart but no rangeEnd then go from the start byte to the end of available content

rangeStart and rangeEnd return that range of bytes from the file if they exist

If only rangeEnd then return the last number of those bytes from the file if they exist

Parameters:

jobId - The id of the job whose output is desired.

rangeStart - The start byte of the file to retrieve. Optional. Greater than or equal to 0.

rangeEnd - The end byte of the file to retrieve. Optional. Greater than or equal to 0. Must be greater than rangeStart.

Returns:

An input stream to the output contents.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobStderr

public java.io.InputStream getJobStderr(java.lang.String jobId)
                                 throws java.io.IOException,
                                        GenieClientException

Method to fetch the stderr of a job from Genie.

Parameters:

jobId - The id of the job whose stderr is desired.

Returns:

An input stream to the stderr contents.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobStderr

public java.io.InputStream getJobStderr(java.lang.String jobId,
                                        @Nullable
                                        java.lang.Long rangeStart,
                                        @Nullable
                                        java.lang.Long rangeEnd)
                                 throws java.io.IOException,
                                        GenieClientException

Method to fetch the stderr of a job from Genie.

Range Logic:

rangeStart but no rangeEnd then go from the start byte to the end of available content

rangeStart and rangeEnd return that range of bytes from the file if they exist

If only rangeEnd then return the last number of those bytes from the file if they exist

Parameters:

jobId - The id of the job whose stderr is desired.

rangeStart - The start byte of the file to retrieve. Optional. Greater than or equal to 0.

rangeEnd - The end byte of the file to retrieve. Optional. Greater than or equal to 0. Must be greater than rangeStart.

Returns:

An input stream to the stderr contents.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobOutputFile

public java.io.InputStream getJobOutputFile(java.lang.String jobId,
                                            java.lang.String outputFilePath)
                                     throws java.io.IOException,
                                            GenieClientException

Method to fetch an output file for a job from Genie.

NOTE: If the specified outputFilePath is a directory, then the directory manifest is returned.

Parameters:

jobId - The id of the job whose output file is desired.

outputFilePath - The path to the file within output directory.

Returns:

An input stream to the output file contents.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

getJobOutputFile

public java.io.InputStream getJobOutputFile(java.lang.String jobId,
                                            java.lang.String outputFilePath,
                                            @Nullable
                                            java.lang.Long rangeStart,
                                            @Nullable
                                            java.lang.Long rangeEnd)
                                     throws java.io.IOException,
                                            GenieClientException

Method to fetch an output file for a job from Genie and accepting an range of bytes to return.

NOTE: If the specified outputFilePath is a directory, then the directory manifest is returned.

Range Logic:

rangeStart but no rangeEnd then go from the start byte to the end of available content

rangeStart and rangeEnd return that range of bytes from the file if they exist

If only rangeEnd then return the last number of those bytes from the file if they exist

Parameters:

jobId - The id of the job whose output file is desired.

outputFilePath - The path to the file within output directory.

rangeStart - The start byte of the file to retrieve. Optional. Greater than or equal to 0.

rangeEnd - The end byte of the file to retrieve. Optional. Greater than or equal to 0. Must be greater than rangeStart.

Returns:

An input stream to the output file contents.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

See Also:

Range Header Documentation

getJobStatus

public JobStatus getJobStatus(java.lang.String jobId)
                       throws java.io.IOException,
                              GenieClientException

Method to fetch the status of a job.

Parameters:

jobId - The id of the job.

Returns:

The status of the Job.

Throws:

GenieClientException - If the response recieved is not 2xx.

java.io.IOException - For Network and other IO issues.

killJob

public void killJob(java.lang.String jobId)
             throws java.io.IOException,
                    GenieClientException

Method to send a kill job request to Genie.

Parameters:

jobId - The id of the job.

Throws:

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

waitForCompletion

public JobStatus waitForCompletion(java.lang.String jobId,
                                   long blockTimeout,
                                   long pollTime)
                            throws GenieClientException,
                                   java.lang.InterruptedException,
                                   java.io.IOException,
                                   GenieTimeoutException

Wait for job to complete, until the given timeout.

Parameters:

jobId - the Genie job ID to wait for completion

blockTimeout - the time to block for (in ms), after which a GenieClientException will be thrown

pollTime - the time to sleep between polling for job status

Returns:

The job status for the job after completion

Throws:

java.lang.InterruptedException - on thread errors.

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

GenieTimeoutException - If the job times out.

waitForCompletion

public JobStatus waitForCompletion(java.lang.String jobId,
                                   long blockTimeout)
                            throws GenieClientException,
                                   java.lang.InterruptedException,
                                   java.io.IOException,
                                   GenieTimeoutException

Wait for job to complete, until the given timeout.

Parameters:

jobId - the Genie job ID to wait for completion.

blockTimeout - the time to block for (in ms), after which a GenieClientException will be thrown.

Returns:

The job status for the job after completion.

Throws:

java.lang.InterruptedException - on thread errors.

GenieClientException - If the response received is not 2xx.

java.io.IOException - For Network and other IO issues.

GenieTimeoutException - If the job times out.