com.netflix.fenzo.queues.tiered

Class TieredQueue

java.lang.Object

com.netflix.fenzo.queues.tiered.TieredQueue

All Implemented Interfaces:

InternalTaskQueue, TaskQueue, TaskIterator

public class TieredQueue
extends java.lang.Object
implements InternalTaskQueue

A tiered queuing system where queues are arranged in multiple tiers and then among multiple buckets within each tier. Tiers represent coarse grain priority, in which higher tier's queues are considered for resource assignment before any of the lower tiers' queues are considered. Within a tier, multiple queues are considered for resource assignment such that their dominant resource usage shares are similar. For example, a queue bucket using 60% of the total memory in use is said to be similar in usage to another bucket using 60% of the total CPUs in use, even if the latter's memory usage is, say, only 10%. The tiers are numbered 0 to N-1 for N tiers, with 0 being the highest priority level.

Nested Class Summary

Nested classes/interfaces inherited from interface com.netflix.fenzo.queues.TaskQueue

TaskQueue.TaskState

Constructor Summary

Constructors

Constructor and Description
TieredQueue(int numTiers) Construct a tiered queue system with the given number of tiers.

Method Summary

Modifier and Type	Method and Description
java.util.Map<TaskQueue.TaskState,java.util.Collection<QueuableTask>>	getAllTasks() Get all of the tasks in the queue.
UsageTrackedQueue	getUsageTracker() This method provides a bridge to the usage tracked queues contained within the tiered queues implementation.
Assignable<QueuableTask>	next() This implementation dynamically picks the next task to consider for resource assignment based on tiers and then based on current dominant resource usage.
void	queueTask(QueuableTask task) Add a task to the queue.
boolean	reset() Reset the queue and make it ready for next scheduling iteration.
void	setSla(TaskQueueSla sla) Set SLA for the queue.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TieredQueue

public TieredQueue(int numTiers)

Construct a tiered queue system with the given number of tiers.

Parameters:

numTiers - The number of tiers.

Method Detail

queueTask

public void queueTask(QueuableTask task)

Description copied from interface: TaskQueue

Add a task to the queue. Duplicates are not allowed, as in, a task request that has the same Id as another existing element will be rejected. The added task will be assigned resources by a scheduler. To add a task into Fenzo that is already running from before, use TaskSchedulingService.initializeRunningTask(QueuableTask, String).

This operation is designed to be performed asynchronously, when it is safe to modify the queue. The queue implementations generally do not modify the queue while a scheduling iteration is in progress.

Specified by:

queueTask in interface TaskQueue

Parameters:

task - A task to add to the queue.

setSla

public void setSla(TaskQueueSla sla)
            throws java.lang.IllegalArgumentException

Description copied from interface: TaskQueue

Set SLA for the queue. The queue implementation determines the implementation of TaskQueueSla that is accepted.

Specified by:

setSla in interface TaskQueue

Parameters:

sla - The SLA to set for the queue.

Throws:

java.lang.IllegalArgumentException - if the implementation of the TaskQueueSla is incompatible with the queue implementation.

next

public Assignable<QueuableTask> next()
                              throws TaskQueueException

This implementation dynamically picks the next task to consider for resource assignment based on tiers and then based on current dominant resource usage. The usage is updated with each resource assignment during the scheduling iteration, in addition to updating with all running jobs from before.

Specified by:

next in interface TaskIterator

Returns:

The next task to assign resources to, a task with assignment failure if the task cannot be scheduled due to some internal constraints (for example exceeds allowed resource usage for a queue). Returns null if none remain for consideration.

Throws:

TaskQueueException - if there is an unknown error getting the next task to launch from any of the tiers or queue buckets.

reset

public boolean reset()
              throws TaskQueueMultiException

Description copied from interface: InternalTaskQueue

Reset the queue and make it ready for next scheduling iteration. Any operations requested that were not safe to carry out during a scheduling iteration can be carried out during this method, before the next scheduling iteration begins.

Specified by:

reset in interface InternalTaskQueue

Returns:

true if the queue was changed as part of this operation, false otherwise. The queue is deemed changed if any queue modifications that were held for safety are carried out during this method, such as adding to or removing from the queue.

Throws:

TaskQueueMultiException - If any exceptions that may have occurred during resetting the pointer to the head of the queue. Or, this may include exceptions that arose when applying any deferred operations from TaskQueue.queueTask(QueuableTask) method.

getUsageTracker

public UsageTrackedQueue getUsageTracker()

This method provides a bridge to the usage tracked queues contained within the tiered queues implementation.

Specified by:

getUsageTracker in interface InternalTaskQueue

Returns:

Implementation for UsageTrackedQueue to account for all the queues within this tiered queue implementation. This implementation focuses on usage tracking only and therefore does not allow invoking UsageTrackedQueue.nextTaskToLaunch() and UsageTrackedQueue.getAllTasks().

getAllTasks

public java.util.Map<TaskQueue.TaskState,java.util.Collection<QueuableTask>> getAllTasks()

Description copied from interface: InternalTaskQueue

Get all of the tasks in the queue. Consistent state of the queue is returned when this is called outside of scheduling iteration runs. Calling this concurrently with a scheduling iteration results in an exception.

Specified by:

getAllTasks in interface InternalTaskQueue

Returns:

List of all tasks in queue as a Map with TaskState as key and Collection of QueuableTask as values.