com.netflix.genie.web.agent.apis.rpc.v4.endpoints

Class GRpcAgentFileStreamServiceImpl

java.lang.Object

com.netflix.genie.proto.FileStreamServiceGrpc.FileStreamServiceImplBase

com.netflix.genie.web.agent.apis.rpc.v4.endpoints.GRpcAgentFileStreamServiceImpl

All Implemented Interfaces:

AgentFileStreamService, io.grpc.BindableService

public class GRpcAgentFileStreamServiceImpl
extends com.netflix.genie.proto.FileStreamServiceGrpc.FileStreamServiceImplBase
implements AgentFileStreamService

AgentFileStreamService gRPC implementation. Receives and caches manifests from connected agents. Allows requesting a file, which is returned in the form of a AgentFileStreamService.AgentFileResource.

Implementation overview: Each agent maintains a single 'sync' channel, through which manifests are pushed to the server. On top of the same channel, the server can request a file. When a file is requested, the agent opens a separate 'transmit' stream and sends the file in chunks. The server acknowledges a chunk in order to request the next one.

This service returns a resource immediately, but maintains a handle on a buffer where data is written as it is received.

Since:

4.0.0

Nested Class Summary

Nested classes/interfaces inherited from interface com.netflix.genie.web.agent.services.AgentFileStreamService

AgentFileStreamService.AgentFileResource

Constructor Summary

Constructors

Constructor and Description
GRpcAgentFileStreamServiceImpl(JobDirectoryManifestProtoConverter converter, org.springframework.scheduling.TaskScheduler taskScheduler) Constructor.

Method Summary

Modifier and Type	Method and Description
java.util.Optional<DirectoryManifest>	getManifest(java.lang.String jobId) Returns the manifest for a given job, boxed in an Optional.
java.util.Optional<AgentFileStreamService.AgentFileResource>	getResource(java.lang.String jobId, java.nio.file.Path relativePath, java.net.URI uri) Returns a Resource for the given job file boxed in an Optional.
io.grpc.stub.StreamObserver<com.netflix.genie.proto.AgentManifestMessage>	sync(io.grpc.stub.StreamObserver<com.netflix.genie.proto.ServerControlMessage> responseObserver)
io.grpc.stub.StreamObserver<com.netflix.genie.proto.AgentFileMessage>	transmit(io.grpc.stub.StreamObserver<com.netflix.genie.proto.ServerAckMessage> responseObserver)

Methods inherited from class com.netflix.genie.proto.FileStreamServiceGrpc.FileStreamServiceImplBase

bindService

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GRpcAgentFileStreamServiceImpl

public GRpcAgentFileStreamServiceImpl(JobDirectoryManifestProtoConverter converter,
                                      org.springframework.scheduling.TaskScheduler taskScheduler)

Constructor.

Parameters:

converter - The JobDirectoryManifestProtoConverter instance to use

taskScheduler - A TaskScheduler instance to use

Method Detail

getResource

public java.util.Optional<AgentFileStreamService.AgentFileResource> getResource(java.lang.String jobId,
                                                                                java.nio.file.Path relativePath,
                                                                                java.net.URI uri)

Returns a Resource for the given job file boxed in an Optional. If the service is unable to determine whether the file exists, the optional is empty. In all other cases, the optional is not empty. However the resource may return false to exist() calls (if the file is not believed to exist on the agent) or false to isReadable() if the file cannot be streamed for other reasons.

Specified by:

getResource in interface AgentFileStreamService

Parameters:

jobId - the job id

relativePath - the relative path in the job directory

uri - the file uri //TODO redundant

Returns:

an optional Resource

getManifest

public java.util.Optional<DirectoryManifest> getManifest(java.lang.String jobId)

Returns the manifest for a given job, boxed in an Optional. The manifest may not be present if the agent is not connected to this node (for example because execution has completed, or because the agent is connected to a different node).

Specified by:

getManifest in interface AgentFileStreamService

Parameters:

jobId - the job id

Returns:

an optional DirectoryManifest

sync

public io.grpc.stub.StreamObserver<com.netflix.genie.proto.AgentManifestMessage> sync(io.grpc.stub.StreamObserver<com.netflix.genie.proto.ServerControlMessage> responseObserver)

Overrides:

sync in class com.netflix.genie.proto.FileStreamServiceGrpc.FileStreamServiceImplBase

transmit

public io.grpc.stub.StreamObserver<com.netflix.genie.proto.AgentFileMessage> transmit(io.grpc.stub.StreamObserver<com.netflix.genie.proto.ServerAckMessage> responseObserver)

Overrides:

transmit in class com.netflix.genie.proto.FileStreamServiceGrpc.FileStreamServiceImplBase