com.netflix.fenzo

Class TaskSchedulingService

java.lang.Object

com.netflix.fenzo.TaskSchedulingService

public class TaskSchedulingService
extends java.lang.Object

A task scheduling service that maintains a scheduling loop to continuously assign resources to tasks pending in the queue. This service maintains a scheduling loop to assign resources to tasks in the queue created when constructing this service. It calls TaskScheduler.scheduleOnce(TaskIterator, List) from within its scheduling loop. Users of this service add tasks into this service's queue, which are held until they are assigned. Here's a typical use of this service:

• Build a TaskScheduler using its builder, TaskScheduler.Builder.
• Build this service using its builder, TaskSchedulingService.Builder, providing a queue implementation from TaskQueues. Specify scheduling interval and other callbacks.
• Start the scheduling loop by calling start().
• Receive callbacks for scheduling result that provide a SchedulingResult object. Note that it is not allowed to call TaskScheduler.getTaskAssigner() for tasks assigned in the result, they are assigned from within this scheduling service. This service assigns the tasks before making the result available to you via the callback. To mark tasks as running for those tasks that were running from before this service was created, use initializeRunningTask(QueuableTask, String). Later, call removeTask(String, QAttributes, String) when tasks complete or they no longer need resource assignments.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TaskSchedulingService.Builder

Method Summary

Modifier and Type	Method and Description
void	addLeases(java.util.List<? extends VirtualMachineLease> leases) Add new leases to be used for next scheduling iteration.
void	initializeRunningTask(QueuableTask task, java.lang.String hostname) Mark the given tasks as running.
boolean	isShutdown()
void	removeTask(java.lang.String taskId, QAttributes qAttributes, java.lang.String hostname) Mark the task to be removed.
void	requestAllTasks(Action1<java.util.Map<TaskQueue.TaskState,java.util.Collection<QueuableTask>>> action) Get all of the tasks in the scheduling service's queue and call the given action when it is available.
void	requestResourceStatus(Action1<java.util.Map<java.lang.String,java.util.Map<VMResource,java.lang.Double[]>>> action) Get resource status information and call the given action when available.
void	requestVmCurrentStates(Action1<java.util.List<VirtualMachineCurrentState>> action) Get the current states of all known VMs and call the given action when available.
void	setTaskToClusterAutoScalerMapGetter(Func1<QueuableTask,java.util.List<java.lang.String>> getter) Set the getter function that maps a given queuable task object to a list of names of VM groups for which cluster autoscaling rules have been set.
void	shutdown() Mark this scheduler as shutdown and prevent any further scheduling iterations from starting.
void	start() Start this scheduling service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

start

public void start()

Start this scheduling service. Tasks are scheduled continuously in a loop with a delay between two consecutive iterations of at least the value specified via TaskSchedulingService.Builder.withLoopIntervalMillis(long), and at most delay specified via TaskSchedulingService.Builder.withMaxDelayMillis(long). The delay between consecutive iterations is longer if the service notices no change since the previous iteration. Changes include additions of new tasks and additions of new leases.

shutdown

public void shutdown()

Mark this scheduler as shutdown and prevent any further scheduling iterations from starting. This may let an already running scheduling iteration to complete.

isShutdown

public boolean isShutdown()

addLeases

public void addLeases(java.util.List<? extends VirtualMachineLease> leases)

Add new leases to be used for next scheduling iteration. Leases with IDs previously added cannot be added again. If duplicates are found, the scheduling iteration throws an exception and is available via the scheduling result callback. See TaskScheduler.scheduleOnce(List, List) for details on behavior upon encountering an exception. This method can be called anytime without impacting any currently running scheduling iterations. The leases will be used in the next scheduling iteration.

Parameters:

leases - New leases to use for scheduling.

requestAllTasks

public void requestAllTasks(Action1<java.util.Map<TaskQueue.TaskState,java.util.Collection<QueuableTask>>> action)
                     throws TaskQueueException

Get all of the tasks in the scheduling service's queue and call the given action when it is available. The list of queues returned is in a consistent state, that is, transitionary actions from ongoing scheduling iterations do not affect the returned collection of tasks. Although an ongoing scheduling iteration is unaffected by this call, onset of the next scheduling iteration may be delayed until the call to the given action returns. Therefore, it is expected that the action callback return quickly.

Parameters:

action - The action to call with task collection.

Throws:

TaskQueueException - if too many actions are pending to get tasks collection.

requestResourceStatus

public void requestResourceStatus(Action1<java.util.Map<java.lang.String,java.util.Map<VMResource,java.lang.Double[]>>> action)
                           throws TaskQueueException

Get resource status information and call the given action when available. Although an ongoing scheduling iteration is unaffected by this call, onset of the next scheduling iteration may be delayed until the call to the given action returns. Therefore, it is expected that the action callback return quickly.

Parameters:

action - The action to call with resource status.

Throws:

TaskQueueException - if too many actions are pending to get resource status.

requestVmCurrentStates

public void requestVmCurrentStates(Action1<java.util.List<VirtualMachineCurrentState>> action)
                            throws TaskQueueException

Get the current states of all known VMs and call the given action when available. Although an ongoing scheduling iteration is unaffected by this call, onset of the next scheduling iteration may be delayed until the call to the given action returns. Therefore, it is expected that the action callback return quickly.

Parameters:

action - The action to call with VM states.

Throws:

TaskQueueException - if too many actions are pending to get VM states.

initializeRunningTask

public void initializeRunningTask(QueuableTask task,
                                  java.lang.String hostname)

Mark the given tasks as running. This is expected to be called for all tasks that were already running from before TaskSchedulingService started running. For example, when the scheduling service is being started after a restart of the system and there were some tasks launched in the previous run of the system. Any tasks assigned resources during scheduling invoked by this service will be automatically marked as running.

Parameters:

task - The task to mark as running

hostname - The name of the VM that the task is running on.

removeTask

public void removeTask(java.lang.String taskId,
                       QAttributes qAttributes,
                       java.lang.String hostname)

Mark the task to be removed. This is expected to be called for all tasks that were added to the queue, whether or not the task is already running. If the task is running, the hostname parameter must be set, otherwise, it can be null. The actual remove operation is performed before the start of the next scheduling iteration.

Parameters:

taskId - The Id of the task to be removed.

qAttributes - The queue attributes of the queue that the task belongs to

hostname - The name of the VM where the task was assigned resources from, or, null if it was not assigned any resources.

setTaskToClusterAutoScalerMapGetter

public void setTaskToClusterAutoScalerMapGetter(Func1<QueuableTask,java.util.List<java.lang.String>> getter)

Set the getter function that maps a given queuable task object to a list of names of VM groups for which cluster autoscaling rules have been set. This function will be called by autoscaler, if it was setup for the TaskScheduler using TaskScheduler.Builder.withAutoScaleRule(AutoScaleRule), to determine if the autoscaling rule should be triggered for aggressive scale up. The function call is expected to return a list of autoscale group names to which the task can be launched, if there are resources available. If either this function is not set, or if the function returns no entries when called, the task is assumed to be able to run on any autoscale group.

Parameters:

getter - The function that takes a queuable task object and returns a list of autoscale group names