com.netflix.genie.web.data.services

Interface JobPersistenceService

All Known Implementing Classes:

JpaJobPersistenceServiceImpl

@Validated
public interface JobPersistenceService

Interfaces for providing persistence functions for jobs other than search.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
void	claimJob(@NotBlank(message="Job id is missing and is required") java.lang.String id, @Valid AgentClientMetadata agentClientMetadata) Set a job identified by id to be owned by the agent identified by agentClientMetadata.
void	createJob(JobRequest jobRequest, JobMetadata jobMetadata, @NotNull Job job, @NotNull JobExecution jobExecution) Save all the initial job fields in the data store.
long	deleteBatchOfJobsCreatedBeforeDate(@NotNull java.time.Instant date, @Min(value=1L) int maxDeleted, @Min(value=1L) int pageSize) This method will delete a chunk of jobs whose creation time is earlier than the given date.
FinishedJob	getFinishedJob(@NotBlank(message="Job id is missing and is required") java.lang.String id) Get a DTO representing a finished job.
java.util.Optional<java.lang.String>	getJobArchiveLocation(@NotBlank(message="Job id is missing and is required") java.lang.String id) Get the location a job directory was archived to if at all.
java.util.Optional<JobRequest>	getJobRequest(@NotBlank(message="Id is missing and is required") java.lang.String id) Get the original request for a job.
java.util.Optional<JobSpecification>	getJobSpecification(@NotBlank(message="Id is missing and is required") java.lang.String id) Get the saved job specification for the given job.
JobStatus	getJobStatus(@NotBlank(message="Job id is missing and is required") java.lang.String id) Get the status for a job with the given id.
boolean	isApiJob(@NotBlank(message="Job id is missing and is required") java.lang.String id) Get whether the job with the given ID was submitted via the REST API or other mechanism.
boolean	isV4(@NotBlank(message="Id is missing and is required") java.lang.String id) Get whether the job is a v4 job.
java.lang.String	saveJobSubmission(@Valid JobSubmission jobSubmission) Save the given job submission information in the underlying data store.
void	saveResolvedJob(@NotBlank(message="Id is missing and is required") java.lang.String id, @Valid ResolvedJob resolvedJob) Save the given resolved details for a job.
void	setJobCompletionInformation(@NotBlank(message="No job id entered. Unable to update.") java.lang.String id, int exitCode, @NotNull(message="No job status entered. Unable to update") JobStatus status, @NotBlank(message="Status message can\'t be blank. Unable to update") java.lang.String statusMessage, java.lang.Long stdOutSize, java.lang.Long stdErrSize) Method to set all job completion information for a job execution.
void	setJobRunningInformation(@NotBlank java.lang.String id, @Min(value=0L,message="Must be no lower than zero") int processId, @Min(value=1L,message="Must be at least 1 millisecond, preferably much more") long checkDelay, @NotNull java.time.Instant timeout) Update the job with information for the running job process.
void	updateJobStatus(@NotBlank(message="Id is missing and is required") java.lang.String id, @NotNull JobStatus currentStatus, @NotNull JobStatus newStatus, java.lang.String newStatusMessage) Update the status of the job identified with id to be newStatus provided that the current status of the job matches newStatus.
void	updateJobStatus(@NotBlank(message="No job id entered. Unable to update.") java.lang.String id, @NotNull(message="Status cannot be null.") JobStatus jobStatus, @NotBlank(message="Status message cannot be empty.") java.lang.String statusMsg) Update the status and status message of the job.
void	updateJobWithRuntimeEnvironment(@NotBlank java.lang.String jobId, @NotBlank java.lang.String clusterId, @NotBlank java.lang.String commandId, @NotNull java.util.List<java.lang.String> applicationIds, @Min(value=1L) int memory) Update the job with the various resources used to run the job including the cluster, command and applications.

Method Detail

createJob

void createJob(@NotNull
               JobRequest jobRequest,
               @NotNull
               JobMetadata jobMetadata,
               @NotNull
               @NotNull Job job,
               @NotNull
               @NotNull JobExecution jobExecution)
        throws GenieException

Save all the initial job fields in the data store.

Parameters:

jobRequest - the Job request object to save. Not null

jobMetadata - metadata about the job request. Not null

job - The Job object to create

jobExecution - The job execution object to create

Throws:

GenieException - if there is an error

updateJobWithRuntimeEnvironment

void updateJobWithRuntimeEnvironment(@NotBlank
                                     @NotBlank java.lang.String jobId,
                                     @NotBlank
                                     @NotBlank java.lang.String clusterId,
                                     @NotBlank
                                     @NotBlank java.lang.String commandId,
                                     @NotNull
                                     @NotNull java.util.List<java.lang.String> applicationIds,
                                     @Min(value=1L)
                                     @Min(value=1L) int memory)
                              throws GenieException

Update the job with the various resources used to run the job including the cluster, command and applications.

Parameters:

jobId - The id of the job to update

clusterId - The id of the cluster the job runs on

commandId - The id of the command the job runs with

applicationIds - The ids of the applications used to run the job

memory - The amount of memory (in MB) to run the job with

Throws:

GenieException - For any problems while updating

updateJobStatus

void updateJobStatus(@NotBlank(message="No job id entered. Unable to update.")
                     @NotBlank(message="No job id entered. Unable to update.") java.lang.String id,
                     @NotNull(message="Status cannot be null.")
                     @NotNull(message="Status cannot be null.") JobStatus jobStatus,
                     @NotBlank(message="Status message cannot be empty.")
                     @NotBlank(message="Status message cannot be empty.") java.lang.String statusMsg)
              throws GenieException

Update the status and status message of the job.

Parameters:

id - The id of the job to update the status for.

jobStatus - The updated status of the job.

statusMsg - The updated status message of the job.

Throws:

GenieException - if there is an error

setJobRunningInformation

void setJobRunningInformation(@NotBlank
                              @NotBlank java.lang.String id,
                              @Min(value=0L,message="Must be no lower than zero")
                              @Min(value=0L,message="Must be no lower than zero") int processId,
                              @Min(value=1L,message="Must be at least 1 millisecond, preferably much more")
                              @Min(value=1L,message="Must be at least 1 millisecond, preferably much more") long checkDelay,
                              @NotNull
                              @NotNull java.time.Instant timeout)
                       throws GenieException

Update the job with information for the running job process.

Parameters:

id - the id of the job to update the process id for

processId - The id of the process on the box for this job

checkDelay - The delay to check the process with

timeout - The date at which this job should timeout

Throws:

GenieException - if there is an error

setJobCompletionInformation

void setJobCompletionInformation(@NotBlank(message="No job id entered. Unable to update.")
                                 @NotBlank(message="No job id entered. Unable to update.") java.lang.String id,
                                 int exitCode,
                                 @NotNull(message="No job status entered. Unable to update")
                                 @NotNull(message="No job status entered. Unable to update") JobStatus status,
                                 @NotBlank(message="Status message can\'t be blank. Unable to update")
                                 @NotBlank(message="Status message can\'t be blank. Unable to update") java.lang.String statusMessage,
                                 @Nullable
                                 java.lang.Long stdOutSize,
                                 @Nullable
                                 java.lang.Long stdErrSize)
                          throws GenieException

Method to set all job completion information for a job execution.

Parameters:

id - the id of the job to update the exit code

exitCode - The exit code of the process

status - The job status for the job

statusMessage - The job status message

stdOutSize - The size (in bytes) of the standard out file or null if there isn't one

stdErrSize - The size (in bytes) of the standard error file or null if there isn't one

Throws:

GenieException - if there is an error

deleteBatchOfJobsCreatedBeforeDate

long deleteBatchOfJobsCreatedBeforeDate(@NotNull
                                        @NotNull java.time.Instant date,
                                        @Min(value=1L)
                                        @Min(value=1L) int maxDeleted,
                                        @Min(value=1L)
                                        @Min(value=1L) int pageSize)

This method will delete a chunk of jobs whose creation time is earlier than the given date.

Parameters:

date - The date before which all jobs should be deleted

maxDeleted - The maximum number of jobs that should be deleted (soft limit, can be rounded up to multiple of page size)

pageSize - Page size used to iterate through jobs

Returns:

the number of deleted jobs

saveJobSubmission

@Nonnull
java.lang.String saveJobSubmission(@Valid
                                            @Valid JobSubmission jobSubmission)
                                     throws IdAlreadyExistsException,
                                            SaveAttachmentException

Save the given job submission information in the underlying data store.

The job record will be created with initial state of JobStatus.RESERVED meaning that the unique id returned by this API has been reserved within Genie and no other job can use it. If JobSubmission contains some attachments these attachments will be persisted to a configured storage system (i.e. local disk, S3, etc) and added to the set of dependencies for the job. The underlying attachment storage system must be accessible by the agent process configured by the system. For example if the server is set up to write attachments to local disk but the agent is not running locally but instead on the remote system it will not be able to access those attachments (as dependencies) and fail. See AttachmentService for more information.

Parameters:

jobSubmission - All the information the system has gathered regarding the job submission from the user either via the API or via the agent CLI

Returns:

The unique id of the job within the Genie ecosystem

Throws:

IdAlreadyExistsException - If the id the user requested already exists in the system for another job

SaveAttachmentException - If attachments were sent in with the job submission and they were unable to be persisted to an underlying storage mechanism meant to share the data with an agent process

getJobRequest

java.util.Optional<JobRequest> getJobRequest(@NotBlank(message="Id is missing and is required")
                                             @NotBlank(message="Id is missing and is required") java.lang.String id)

Get the original request for a job.

Parameters:

id - The unique id of the job to get

Returns:

The job request if one was found. Wrapped in Optional so empty optional returned if no job found

Throws:

GenieRuntimeException - On error converting entity values to DTO values

saveResolvedJob

void saveResolvedJob(@NotBlank(message="Id is missing and is required")
                     @NotBlank(message="Id is missing and is required") java.lang.String id,
                     @Valid
                     @Valid ResolvedJob resolvedJob)

Save the given resolved details for a job. Sets the job status to JobStatus.RESOLVED.

Parameters:

id - The id of the job

resolvedJob - The resolved information for the job

Throws:

GenieJobNotFoundException - When the job identified by id can't be found and the specification can't be saved

GenieClusterNotFoundException - When the cluster specified in the job specification doesn't actually exist

GenieCommandNotFoundException - When the command specified in the job specification doesn't actually exist

GenieApplicationNotFoundException - When an application specified in the job specification doesn't actually exist

getJobSpecification

java.util.Optional<JobSpecification> getJobSpecification(@NotBlank(message="Id is missing and is required")
                                                         @NotBlank(message="Id is missing and is required") java.lang.String id)

Get the saved job specification for the given job. If the job hasn't had a job specification resolved an empty Optional will be returned.

Parameters:

id - The id of the job

Returns:

The job specification if one is present else an empty Optional

Throws:

GenieJobNotFoundException - If no job with id exists

GenieClusterNotFoundException - When the cluster isn't found in the database which it should be at this point given the input to the db was valid at the time of persistence

GenieCommandNotFoundException - When the command isn't found in the database which it should be at this point given the input to the db was valid at the time of persistence

GenieRuntimeException - on unexpected error

claimJob

void claimJob(@NotBlank(message="Job id is missing and is required")
              @NotBlank(message="Job id is missing and is required") java.lang.String id,
              @Valid
              @Valid AgentClientMetadata agentClientMetadata)

Set a job identified by id to be owned by the agent identified by agentClientMetadata. The job status in the system will be set to JobStatus.CLAIMED

Parameters:

id - The id of the job to claim. Must exist in the system.

agentClientMetadata - The metadata about the client claiming the job

Throws:

GenieJobNotFoundException - if no job with the given id exists

GenieJobAlreadyClaimedException - if the job with the given id already has been claimed

GenieInvalidStatusException - if the current job status is not JobStatus.RESOLVED

updateJobStatus

void updateJobStatus(@NotBlank(message="Id is missing and is required")
                     @NotBlank(message="Id is missing and is required") java.lang.String id,
                     @NotNull
                     @NotNull JobStatus currentStatus,
                     @NotNull
                     @NotNull JobStatus newStatus,
                     @Nullable
                     java.lang.String newStatusMessage)

Update the status of the job identified with id to be newStatus provided that the current status of the job matches newStatus. Optionally a status message can be provided to provide more details to users. If the newStatus is JobStatus.RUNNING the start time will be set. If the newStatus is a member of JobStatus.getFinishedStatuses() and the job had a started time set the finished time of the job will be set.

Parameters:

id - The id of the job to update status for. Must exist in the system.

currentStatus - The status the caller to this API thinks the job currently has

newStatus - The new status the caller would like to update the status to

newStatusMessage - An optional status message to associate with this change

Throws:

GenieJobNotFoundException - if no job with the given id exists

GenieInvalidStatusException - if the current status of the job identified by id in the system doesn't match the supplied currentStatus. Also if the currentStatus equals the newStatus.

isV4

boolean isV4(@NotBlank(message="Id is missing and is required")
             @NotBlank(message="Id is missing and is required") java.lang.String id)

Get whether the job is a v4 job.

Parameters:

id - The id of the job

Returns:

true if its a v4 job

Throws:

GenieJobNotFoundException - If no job with id exists

getJobStatus

JobStatus getJobStatus(@NotBlank(message="Job id is missing and is required")
                       @NotBlank(message="Job id is missing and is required") java.lang.String id)
                throws GenieNotFoundException

Get the status for a job with the given id.

Parameters:

id - The id of the job to get status for

Returns:

The job status

Throws:

GenieNotFoundException - If no job with the given id exists

getJobArchiveLocation

java.util.Optional<java.lang.String> getJobArchiveLocation(@NotBlank(message="Job id is missing and is required")
                                                           @NotBlank(message="Job id is missing and is required") java.lang.String id)
                                                    throws GenieNotFoundException

Get the location a job directory was archived to if at all.

Parameters:

id - The id of the job to get the location for

Returns:

The archive location or Optional.empty()

Throws:

GenieNotFoundException - When there is no job with id id

getFinishedJob

FinishedJob getFinishedJob(@NotBlank(message="Job id is missing and is required")
                           @NotBlank(message="Job id is missing and is required") java.lang.String id)
                    throws GenieNotFoundException,
                           GenieInvalidStatusException

Get a DTO representing a finished job.

Parameters:

id - The id of the job to retrieve

Returns:

the finished job

Throws:

GenieNotFoundException - if no job with the given id exists

GenieInvalidStatusException - if the current status of the job is not final

isApiJob

boolean isApiJob(@NotBlank(message="Job id is missing and is required")
                 @NotBlank(message="Job id is missing and is required") java.lang.String id)
          throws GenieNotFoundException

Get whether the job with the given ID was submitted via the REST API or other mechanism.

Parameters:

id - The id of the job. Not blank.

Returns:

true if the job was submitted via the API. false otherwise

Throws:

GenieNotFoundException - If no job with id exists