artifact file
Each Mantis Job has an associated artifact file that contains its source code and JSON configuration.
You can establish an autoscaling policy for each component of your Mantis Job that governs how Mantis adjusts the number of workers assigned to that component as its workload changes.
⇒ See Mantis Job Autoscaling
Backpressure refers to a set of possible strategies for coping with ReactiveX Observables that produce items more rapidly than their observers consume them.
⇒ See backpressure operators
Binary compression
has a particular meaning in the Mantis context... see (Connecting to a Source Job)[/writingjobs/source#connecting-to-a-source-job]
Broadcast mode
In broadcast mode, each worker of your Mantis Job gets all the data from all workers of the Source Job rather than having that data distributed equally among the workers of your Job.
⇒ See Source Job Sources: Broadcast Mode
Apache Cassandra is an open source, distributed database management system.
⇒ See Apache Cassandra Documentation.
A Job Cluster is a containing entity for Mantis Jobs. It defines metadata and certain service-level agreements. Job Clusters ease job lifecycle management and job revisioning.
A Mantis Cluster is a group of cloud container instances that hold your Mantis resources.
cold Observables
A cold ReactiveX Observable waits until an observer subscribes to it before it begins to emit items. This means the observer is guaranteed to see the whole Observable sequence from the beginning. This is in contrast to a hot Observable, which may begin emitting items as soon as it is created, even before observers have subscribed to it.
A Mantis Job is composed of three types of component: a Source, one or more Processing Stages, and a Sink.
Custom source
In contrast to a Source Job, which is a built-in variety of Source component designed to pull data from a common sort of data source, a custom source typically accesses data from less-common sources or has unusual delivery guarantee semantics.
The stage executor is responsible for loading the bytecode for a Mantis Job and then executing its stages and workers in a coordinated fashion. In the Mesos UI, workers are also referred to as executors.
Fenzo is a Java library that implements a generic task scheduler for Mesos frameworks.
⇒ See the Fenzo documentation.
grouped data
Grouped (or keyed) data is distinguished from scalar data in that each datum is accompanied by a key that indicates what group it belongs to. Grouped data can be processed by a RxJava GroupedObservable or by a MantisGroup.
GRPC is an open-source RPC framework using Protocol Buffers.
⇒ See
hot Observables
A hot ReactiveX Observable may begin emitting items as soon as it is created, even before observers have subscribed to it. This means the observer may miss items that were emitted before the observer subscribed. This is in contrast to a cold Observable, which waits until an observer subscribes to it before it begins to emit items.
Java Mission Control is a tool from Oracle with which developers can monitor and manage Java applications.
⇒ See Java Components
A Mantis Job takes in a stream of data, transforms it by using RxJava operators, and then outputs the results as another stream. It is composed of a Source, one or more Processing Stages, and a Sink.
⇒ See Writing Mantis Jobs
Job Cluster
see Cluster
Job Master
If a job is configured with autoscaling, Mantis will add a Job Master component to it as its initial component. This component will send metrics back to Mantis to help it govern the autoscaling process.
Apache Kafka is a large-scale, distributed streaming platform.
⇒ See Apache Kafka.
keyed data
see grouped data
Keystone is Netflix’s data backbone, a stream processing platform that focuses on data analytics.
⇒ See Keystone Real-time Stream Processing Platform
A label is a text key/value pair that you can add to a Job Cluster or to an individual Job to make it easier to search for or group.
Log4j is a Java-based logging framework.
⇒ See Apache Log4j
Mantis Master
The Mantis Master coordinates the execution of Mantis Jobs and starts the services on each Worker.
Apache Mesos is an open-source technique for balancing resources across frameworks in clusters.
⇒ See Apache Mesos
Mantis inserts metadata into its Job payload. This may include information about where the data came from, for instance. You can define additional metadata to include in the payload when you establish the Job Cluster.
meta message
A Source Job may occasionally inject meta messages into its data stream that indicate things like data drops.
migration strategy
Mantis Publish
Mantis Publish (internally at Netflix known as Mantis Realtime Events or MRE) is a library that your application can use to stream events into Mantis while respecting MQL filters.
⇒ See MQL.
You use Mantis Query Language to define filters and other data processing that Mantis applies to a Source data stream at its point of origin, so as to reduce the amount of data going over the wire.
In ReactiveX an Observable is the method of processing a stream of data in a way that facilitates its transformation and consumption by observers. Observables come in hot and cold varieties. There is also a GroupedObservable that is specialized to grouped data.
⇒ See Observable.
A Mantis Job may accept parameters that modify its behavior. You can define these in your Job Cluster definition, and set their values on a per-Job basis.
Processing Stage
A Processing Stage component of a Mantis Job transforms the RxJava Observables it obtains from the Source component. A Job with only one Processing Stage is called a single-stage Job.
⇒ See The Processing Stage Component
A property is a particular named data value found within events in an event stream.
Reactive Streams
Reactive Streams is the latest advance of the ReactiveX project. It is an API for manipulating streams of asynchronous data in a non-blocking fashion, with backpressure.
⇒ See Reactive Streams.
ReactiveX is a software technique for transforming, combining, reacting to, and managing streams of data. RxJava is an example of a library that implements this technique.
⇒ See
RxJava is the Java implementation of ReactiveX, a software technique for transforming, combining, reacting to, and managing streams of data.
⇒ See
Sampling is an MQL strategy for mitigating data volume issues. There are two sampling strategies: Random and Sticky. Random sampling uniformly downsamples the source stream to a percentage of its original volume. Sticky sampling selectively samples data from the source stream based on key values.
⇒ See MQL: Sampling
scalar data
Scalar data is distinguished from keyed or grouped data in that it is not categorized into groups by key. Scalar data can be processed by an ordinary ReactiveX Observable.
The Sink is the final component of a Mantis Job. It takes the Observables that has been transformed by the Processing Stage and outputs it in the form of a new data stream.
⇒ See The Sink Component
A service-level agreement, in the Mantis context, is defined on a per-Cluster basis. You use it to configure how many Jobs in the cluster will be in operation at any time, among other things.
The Source component of a Mantis Job fetches data from a source outside of Mantis and makes it available to the Processing Stage component in the form of an RxJava Observable. There are two varieties of Source: a Source Job and a custom source.
⇒ See The Source Component
Source Job
A Source Job is a Mantis Job that you can use as a Source, which wraps a data source external to Mantis and makes it easier for you to create a job that observes its data.
⇒ See Mantis Source Jobs
server-sent events
Server-sent events (SSE) are a way for a browser to receive automatic updates from a server through an HTTP connection. Mantis includes an SSE Sink.
⇒ See Wikipedia: Server-sent events
A transformation acts on each datum from a stream or Observables of data, changing it in some manner before passing it along as a new stream or Observable. Transformations may change data between scalar and grouped forms.
transient jobs
A transient (or ephemeral) Mantis Job is automatically killed by Mantis after a certain amount of time has passed since the last subscriber to the job disconnects.
WebSocket is a two-way, interactive communication channel that works over HTTP. In the Mantis context, it is an alternative to SSE.
⇒ See
A worker is the smallest unit of work that is scheduled within a Mantis component. You can configure how many resources Mantis allocates to each worker, and Mantis will adjust the number of workers your Mantis component needs based on its autoscaling policy.
Apache Zookeeper is an open-source server that maintains configuration information and other services required by distributed applications.
⇒ See Apache ZooKeeper