1. Introduction
This is the REST API documentation for Genie version 3.3.8-SNAPSHOT. Here you will find specific details of how to invoke the Genie REST API and what you can expect as return values.
For more Genie information see the website. |
For high level information and installation instructions please see the reference guide. |
For a demo of this version of Genie please see demo guide. |
2. Assumptions
This documentation assumes a basic knowledge of how REST APIs work. If you don’t know things like HTTP verbs see websites like this one.
3. HATEOAS
A lot of the API’s return application/hal+json
as a content type. This is a superset of JSON built to support
HATEOAS (Hypermedia as the Engine of Application State). Rather than re-writing what
HATEOAS is see this great write up on the Spring Website about it.
4. Security
All of this documentation assumes security is disabled for Genie. If your installation has security enabled the REST APIs will be secured via OAuth2 or x509 certs or both.
These are the currently supported API security mechanisms but others could be plugged in. We’ll only talk about OAuth2 here for now until another implementation is done. |
4.1. OAuth2
If OAuth2 is enabled you’ll need to send your JWT you got back from your provider along with your
API request. This is accomplished by setting your Authorization
header to a value of Bearer {your token here}
.
5. Resources
This section contains the documentation for various resources sent to and from the various Genie APIs. Note their structure and fields, particularly what is marked optional. Optional means when you submit one of these resources to Genie you don’t need to set that field. Genie either doesn’t require it or if it does, as is the case with ID’s, it will create one if you don’t provide it. When you get a resource all fields Genie has stored will be returned.
5.1. Application
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
5.2. Cluster
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the cluster |
true |
5.3. Command
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Number |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
dependencies |
Array |
The dependencies for the command |
true |
5.4. Job Request
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
true |
|
clusterCriterias |
Array |
List of cluster criteria’s for which a match will be attempted with register cluster tags. Each set of tags within a given cluster criteria must have at least one non-blank (e.g. ' ', ' ', null) tag. |
Must not be empty |
false |
commandCriteria |
Array |
Set of tags which will attempt to match against the commands linked to selected cluster. There must be at least one non-blank (e.g. ' ', ' ', null) criteria within the set |
Must not be empty |
false |
group |
String |
A group that the job should be run under on the linux system |
Size must be between 0 and 255 inclusive |
true |
disableLogArchival |
Boolean |
If you want to disable backing up job output files set this to true. Default: false |
true |
|
String |
If you want e-mail notification on job completion enter address here |
Must be a well-formed email address. Size must be between 0 and 255 inclusive |
true |
|
cpu |
Number |
For future use. Currently has no impact. |
Must be at least 1 |
true |
memory |
Number |
The amount of memory (in MB) desired for job client. Cannot exceed configured max. |
Must be at least 1 |
true |
timeout |
Number |
The timeout (in seconds) after which job will be killed by system, system setting used if not set |
Must be at least 1 |
true |
configs |
Array |
URI’s of configuration files which will be downloaded into job working directory at runtime |
true |
|
dependencies |
Array |
URI’s of dependency files which will be downloaded into job working directory at runtime |
true |
|
applications |
Array |
Complete list of application ids if power user wishes to override selected command defaults |
true |
|
grouping |
String |
The grouping of the job relative to other jobs. e.g. scheduler job name |
true |
|
groupingInstance |
String |
The grouping instance of the job relative to other jobs. e.g. scheduler job run |
true |
5.5. Job
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
true |
|
status |
String |
The status of the job. Options: [INIT, RUNNING, SUCCEEDED, KILLED, FAILED, INVALID] |
Must not be null |
false |
statusMsg |
String |
The status message of the job |
Size must be between 0 and 255 inclusive |
true |
started |
String |
The time (UTC ISO8601 with millis) the job was started |
true |
|
finished |
String |
The time (UTC ISO8601 with millis) the job finished |
true |
|
archiveLocation |
String |
The URI where the working directory zip was stored |
Size must be between 0 and 1024 inclusive |
true |
clusterName |
String |
The name of the cluster the job was run on if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
commandName |
String |
The name of the command the job was run with if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
runtime |
String |
Runtime of the job in ISO8601 duration format |
Must not be null |
true |
grouping |
String |
The grouping of the job relative to other jobs. e.g. scheduler job name |
true |
|
groupingInstance |
String |
The grouping instance of the job relative to other jobs. e.g. scheduler job run |
true |
5.6. Job Execution
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
hostName |
String |
The host name of the Genie node responsible for the job |
Size must be between 1 and 1024 inclusive |
false |
processId |
Number |
The id of the job client process on the Genie node |
true |
|
checkDelay |
Number |
The amount of time in milliseconds between checks of the job status by Genie |
Must be at least 1 |
true |
timeout |
String |
The date (UTC ISO8601 with millis) when the job will be killed by Genie due to timeout |
true |
|
exitCode |
Number |
The job client process exit code after the job is done |
true |
|
memory |
Number |
The amount of memory (in MB) allocated to the job client |
Must be at least 1 |
true |
5.7. Job Metadata
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
clientHost |
String |
The host name of the client that submitted the job to Genie |
true |
|
userAgent |
String |
The user agent string that was passed to Genie on job request |
true |
|
numAttachments |
Number |
The number of attachments sent to Genie with the job request |
true |
|
totalSizeOfAttachments |
Number |
The total size of all attachments sent to Genie with the job request. In bytes. |
true |
|
stdOutSize |
Number |
The final size of the stdout file after a job is completed. In bytes. |
true |
|
stdErrSize |
Number |
The final size of the stderr file after a job is completed. In bytes. |
true |
6. Jobs API
6.1. Submit a Job
6.1.1. Description
The main API in Genie. Submitting a job to be run.
There are two different ways to submit a job to Genie. Both go to the same endpoint but you can simply send a JSON
payload if you have no attachments. This means setting your content type to application/json
. If you do have
attachments you want to send with your job request you need to set the content type to multipart/form-data
and
post the request that way. Examples are below.
6.1.2. Endpoint
POST /api/v3/jobs
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
This Example is without attachments |
6.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
true |
|
clusterCriterias |
Array |
List of cluster criteria’s for which a match will be attempted with register cluster tags. Each set of tags within a given cluster criteria must have at least one non-blank (e.g. ' ', ' ', null) tag. |
Must not be empty |
false |
commandCriteria |
Array |
Set of tags which will attempt to match against the commands linked to selected cluster. There must be at least one non-blank (e.g. ' ', ' ', null) criteria within the set |
Must not be empty |
false |
group |
String |
A group that the job should be run under on the linux system |
Size must be between 0 and 255 inclusive |
true |
disableLogArchival |
Boolean |
If you want to disable backing up job output files set this to true. Default: false |
true |
|
String |
If you want e-mail notification on job completion enter address here |
Must be a well-formed email address. Size must be between 0 and 255 inclusive |
true |
|
cpu |
Number |
For future use. Currently has no impact. |
Must be at least 1 |
true |
memory |
Number |
The amount of memory (in MB) desired for job client. Cannot exceed configured max. |
Must be at least 1 |
true |
timeout |
Number |
The timeout (in seconds) after which job will be killed by system, system setting used if not set |
Must be at least 1 |
true |
configs |
Array |
URI’s of configuration files which will be downloaded into job working directory at runtime |
true |
|
dependencies |
Array |
URI’s of dependency files which will be downloaded into job working directory at runtime |
true |
|
applications |
Array |
Complete list of application ids if power user wishes to override selected command defaults |
true |
|
grouping |
String |
The grouping of the job relative to other jobs. e.g. scheduler job name |
true |
|
groupingInstance |
String |
The grouping instance of the job relative to other jobs. e.g. scheduler job run |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/jobs' -i -X POST -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"metadata" : {
"schedulerJobName" : "04befc7d-85bb-45d7-b786-2d368f80dcd8",
"schedulerRunId" : "c64ccbeb-d67d-4c99-95b2-9109e1c448af"
},
"tags" : [ "6f9a1aab-1b32-4f17-a2d5-30b1197ac8aa", "33ecdb92-124a-4f52-a11f-1b6cb832f295" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/config1" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"applications" : [ ],
"grouping" : null,
"groupingInstance" : null
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"metadata" : {
"schedulerJobName" : "04befc7d-85bb-45d7-b786-2d368f80dcd8",
"schedulerRunId" : "c64ccbeb-d67d-4c99-95b2-9109e1c448af"
},
"tags" : [ "6f9a1aab-1b32-4f17-a2d5-30b1197ac8aa", "33ecdb92-124a-4f52-a11f-1b6cb832f295" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/config1" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"applications" : [ ],
"grouping" : null,
"groupingInstance" : null
}' | http POST 'https://genie.example.com/api/v3/jobs' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
POST /api/v3/jobs HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 1262
{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"metadata" : {
"schedulerJobName" : "04befc7d-85bb-45d7-b786-2d368f80dcd8",
"schedulerRunId" : "c64ccbeb-d67d-4c99-95b2-9109e1c448af"
},
"tags" : [ "6f9a1aab-1b32-4f17-a2d5-30b1197ac8aa", "33ecdb92-124a-4f52-a11f-1b6cb832f295" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/config1" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"applications" : [ ],
"grouping" : null,
"groupingInstance" : null
}
6.1.4. API Docs
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/jobs' -i -X POST -H 'Content-Type: multipart/form-data;charset=UTF-8' -F 'request={"id":null,"created":null,"updated":null,"version":"1.0","user":"genie","name":"List * ... Directories bash job","description":"Genie 3 Test Job","metadata":null,"tags":[],"configs":[],"dependencies":[],"setupFile":"/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile","commandArgs":"-c 'echo hello world'","clusterCriterias":[{"tags":["localhost"]}],"commandCriteria":["bash"],"group":null,"disableLogArchival":true,"email":null,"cpu":null,"memory":null,"timeout":null,"applications":[],"grouping":null,"groupingInstance":null};type=application/json' -F 'attachment=@query.sql;type=application/octet-stream' -F 'attachment=@query2.sql;type=application/octet-stream'
HTTPie
$ http --form POST 'https://genie.example.com/api/v3/jobs' \
'request'@<(echo '{"id":null,"created":null,"updated":null,"version":"1.0","user":"genie","name":"List * ... Directories bash job","description":"Genie 3 Test Job","metadata":null,"tags":[],"configs":[],"dependencies":[],"setupFile":"/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile","commandArgs":"-c 'echo hello world'","clusterCriterias":[{"tags":["localhost"]}],"commandCriteria":["bash"],"group":null,"disableLogArchival":true,"email":null,"cpu":null,"memory":null,"timeout":null,"applications":[],"grouping":null,"groupingInstance":null}') \
'attachment'@'query.sql' \
'attachment'@'query2.sql'
HTTP Messages
Request
POST /api/v3/jobs HTTP/1.1
Content-Type: multipart/form-data;charset=UTF-8; boundary=6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Host: genie.example.com
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=request
Content-Type: application/json
{"id":null,"created":null,"updated":null,"version":"1.0","user":"genie","name":"List * ... Directories bash job","description":"Genie 3 Test Job","metadata":null,"tags":[],"configs":[],"dependencies":[],"setupFile":"/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile","commandArgs":"-c 'echo hello world'","clusterCriterias":[{"tags":["localhost"]}],"commandCriteria":["bash"],"group":null,"disableLogArchival":true,"email":null,"cpu":null,"memory":null,"timeout":null,"applications":[],"grouping":null,"groupingInstance":null}
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=attachment
Content-Type: application/octet-stream
select * from myTable;
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=attachment
Content-Type: application/octet-stream
SELECT * FROM your_table JOIN my_table ON your_table.id = my_table.id WHERE x > 50;
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm--
6.2. Find Jobs
6.2.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The id of the jobs to find. Use % symbol for regex like search. |
|
The name of the jobs to find. Use % symbol for regex like search. |
|
The user of the jobs to find. Use % symbol for regex like search. |
|
The status(es) of the jobs to find. Can have multiple. Options: [INIT, RUNNING, SUCCEEDED, KILLED, FAILED, INVALID] |
|
The tag(s) of the jobs to find. Can have multiple. |
|
The name of the cluster on which the jobs ran. Use % symbol for regex like search. |
|
The id of the cluster on which the jobs ran. |
|
The name of the command which was executed by the job. Use % symbol for regex like search. |
|
The id of the command which was executed by the job. |
|
The minimum started time of the job in milliseconds since epoch. (inclusive) |
|
The maximum started time of the job in milliseconds since epoch. (exclusive) |
|
The minimum finished time of the job in milliseconds since epoch. (inclusive) |
|
The maximum finished time of the job in milliseconds since epoch. (exclusive) |
|
The grouping the job should be a member of. Use % symbol for regex like search. |
|
The grouping instance the job should be a member of. Use % symbol for regex like search. |
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.jobSearchResultList |
Array |
The found jobs. |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 947
{
"_embedded" : {
"jobSearchResultList" : [ {
"id" : "cb0b2c45-e828-418a-9719-b8ae345c82cf",
"name" : "List * ... Directories bash job",
"user" : "genie",
"status" : "SUCCEEDED",
"started" : "2018-04-11T18:03:06.461Z",
"finished" : "2018-04-11T18:03:06.583Z",
"clusterName" : "Local laptop",
"commandName" : "Unix Bash command",
"runtime" : "PT0.122S",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs?user=genie{&id,name,status,tag,clusterName,clusterId,commandName,commandId,minStarted,maxStarted,minFinished,maxFinished,grouping,groupingInstance}",
"templated" : true
}
},
"page" : {
"size" : 10,
"totalElements" : 1,
"totalPages" : 1,
"number" : 0
}
}
6.3. Get a Job
6.3.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
true |
|
status |
String |
The status of the job. Options: [INIT, RUNNING, SUCCEEDED, KILLED, FAILED, INVALID] |
Must not be null |
false |
statusMsg |
String |
The status message of the job |
Size must be between 0 and 255 inclusive |
true |
started |
String |
The time (UTC ISO8601 with millis) the job was started |
true |
|
finished |
String |
The time (UTC ISO8601 with millis) the job finished |
true |
|
archiveLocation |
String |
The URI where the working directory zip was stored |
Size must be between 0 and 1024 inclusive |
true |
clusterName |
String |
The name of the cluster the job was run on if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
commandName |
String |
The name of the command the job was run with if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
runtime |
String |
Runtime of the job in ISO8601 duration format |
Must not be null |
true |
grouping |
String |
The grouping of the job relative to other jobs. e.g. scheduler job name |
true |
|
groupingInstance |
String |
The grouping instance of the job relative to other jobs. e.g. scheduler job run |
true |
HATEOAS Links
Relation | Description |
---|---|
|
URI for this job |
|
The request that kicked off this job |
|
The job execution information for this job |
|
The job metadata information for this job |
|
The output URI for the job |
|
The current status of the job |
|
The cluster this job ran on |
|
The command this job executed |
|
The applications this job used |
HTTP Messages
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 2026
{
"id" : "cb0b2c45-e828-418a-9719-b8ae345c82cf",
"created" : "2018-04-11T18:03:06.236Z",
"updated" : "2018-04-11T18:03:06.583Z",
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"metadata" : {
"schedulerJobName" : "04befc7d-85bb-45d7-b786-2d368f80dcd8",
"schedulerRunId" : "c64ccbeb-d67d-4c99-95b2-9109e1c448af"
},
"tags" : [ "6f9a1aab-1b32-4f17-a2d5-30b1197ac8aa", "33ecdb92-124a-4f52-a11f-1b6cb832f295" ],
"status" : "SUCCEEDED",
"statusMsg" : "Job finished successfully.",
"started" : "2018-04-11T18:03:06.461Z",
"finished" : "2018-04-11T18:03:06.583Z",
"archiveLocation" : null,
"clusterName" : "Local laptop",
"commandName" : "Unix Bash command",
"runtime" : "PT0.122S",
"commandArgs" : "-c 'echo hello world'",
"grouping" : null,
"groupingInstance" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output"
},
"request" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/request"
},
"execution" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/execution"
},
"metadata" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/metadata"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/status"
},
"cluster" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/cluster"
},
"command" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/command"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/applications"
}
}
}
6.4. Kill a Job
6.5. Get Job Status
6.6. Get Contents From the Job Working Directory
6.6.1. Description
This API acts as a file browser for the remote working directory of your job. You can request endpoints and if it is a directory you will be shown the contents of the directory and be able to navigate. If the endpoint points to a file it will return the file.
6.6.3. Getting a directory as JSON
This is the default behavior for directories if no Accept header is present
|
6.6.4. API Docs
Request Documentation
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
parent |
Object |
Information about the parent of this directory |
true |
|
parent.name |
String |
The name of the parent directory |
true |
|
parent.url |
String |
The url to get the parent |
true |
|
parent.size |
Number |
The size of the parent in bytes |
true |
|
parent.lastModified |
String |
The last time the parent was modified in ISO8601 UTC with milliseconds included |
true |
|
directories |
Array |
All the subdirectories of this directory |
false |
|
directories[].name |
String |
The name of the directory |
false |
|
directories[].url |
String |
The url to get the directory |
false |
|
directories[].size |
Number |
The size of the directory in bytes |
false |
|
directories[].lastModified |
String |
The last time the directory was modified in ISO8601 UTC with milliseconds included |
false |
|
files |
Array |
All the files in this directory |
false |
|
files[].name |
String |
The name of the file |
false |
|
files[].url |
String |
The url to get the file |
false |
|
files[].size |
Number |
The size of the file in bytes |
false |
|
files[].lastModified |
String |
The last time the file was modified in ISO8601 UTC with milliseconds included |
false |
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/ HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/json;charset=UTF-8
Content-Length: 1453
{
"parent" : null,
"directories" : [ {
"name" : "genie/",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/genie/",
"size" : 0,
"lastModified" : "2018-04-11T18:03:06.000Z"
} ],
"files" : [ {
"name" : "config1",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/config1",
"size" : 0,
"lastModified" : "2018-04-11T18:03:06.000Z"
}, {
"name" : "dep1",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/dep1",
"size" : 0,
"lastModified" : "2018-04-11T18:03:06.000Z"
}, {
"name" : "jobsetupfile",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/jobsetupfile",
"size" : 0,
"lastModified" : "2018-04-11T18:03:06.000Z"
}, {
"name" : "run",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/run",
"size" : 3850,
"lastModified" : "2018-04-11T18:03:06.000Z"
}, {
"name" : "stderr",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/stderr",
"size" : 0,
"lastModified" : "2018-04-11T18:03:06.000Z"
}, {
"name" : "stdout",
"url" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/stdout",
"size" : 12,
"lastModified" : "2018-04-11T18:03:06.000Z"
} ]
}
6.6.5. Getting a directory as HTML
This would happen automatically if you called from browser due to browser default accept header |
6.6.6. API Docs
Request Documentation
Request Examples
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/ HTTP/1.1
Accept: text/html
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: text/html;charset=UTF-8
Content-Length: 3097
<!DOCTYPE html><html><head><title>cb0b2c45-e828-418a-9719-b8ae345c82cf</title><style type="text/css"><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}--></style> </head><body><h1>cb0b2c45-e828-418a-9719-b8ae345c82cf</h1><HR size="1" noshade="noshade"><table width="100%" cellspacing="0" cellpadding="5" align="center"><tr><td align="left"><font size="+1"><strong>Filename</strong></font></td><td align="right"><font size="+1"><strong>Size</strong></font></td><td align="right"><font size="+1"><strong>Last Modified</strong></font></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/genie/"><tt>genie/</tt></a></td><td align="right"><tt>-</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr><tr><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/config1"><tt>config1</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/dep1"><tt>dep1</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr><tr><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/jobsetupfile"><tt>jobsetupfile</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/run"><tt>run</tt></a></td><td align="right"><tt>3 KB</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr><tr><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/stderr"><tt>stderr</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/stdout"><tt>stdout</tt></a></td><td align="right"><tt>12 bytes</tt></td><td align="right"><tt>Wed, 11 Apr 2018 18:03:06 GMT</tt></td></tr></table><HR size="1" noshade="noshade"><h3>Apache Tomcat/8.5.27</h3></body></html>
6.6.8. API Docs
Request Documentation
Request Examples
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output/run HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Last-Modified: Wed, 11 Apr 2018 18:03:06 GMT
Content-Length: 3850
Content-Type: text/plain;charset=UTF-8
Accept-Ranges: bytes
#!/usr/bin/env bash
set -o nounset -o pipefail
# Set function in case any of the exports or source commands cause an error
trap "handle_failure" ERR EXIT
function handle_failure {
ERROR_CODE=$?
echo "Handling exit signal (code: ${ERROR_CODE})"
# Good exit
if [[ ${ERROR_CODE} -eq 0 ]]; then
exit 0
fi
# Bad exit
printf '{"exitCode": "%s"}\n' "${ERROR_CODE}" > ${GENIE_JOB_DIR}/genie/genie.done
exit "${ERROR_CODE}"
}
# Set function for handling kill signal from the job kill service
trap "handle_kill_request" SIGTERM
function handle_kill_request {
# Disable traps
trap "" SIGTERM ERR EXIT
echo "Kill signal received"
KILL_EXIT_CODE=999
### Write the kill exit code to genie.done file as exit code before doing anything else
echo "Generate done file with exit code ${KILL_EXIT_CODE}"
printf '{"exitCode": "%s"}\n' "${KILL_EXIT_CODE}" > ${GENIE_JOB_DIR}/genie/genie.done
### Send a kill signal the entire process group
echo "Sending a kill signal to the process group"
pkill -P $$
for ((iteration=1; iteration < 30; iteration++))
{
if kill -0 ${CHILDREN_PID} &> /dev/null;
then
echo "Waiting for children of ${SELF_PID} to terminate"
pgrep -P ${SELF_PID}
sleep 1
else
echo "Children process no longer running, exiting"
exit 0
fi
}
### Reaching at this point means the children did not die. If so send kill -9 to the entire process group
# this is a hard kill and will this process itself as well
echo "Sending a kill -9 to children"
pkill -9 -P $$
echo "Done"
}
SELF_PID=$$
echo Start: `date '+%Y-%m-%d %H:%M:%S'`
export GENIE_JOB_DIR="/tmp/1523469774245-0/cb0b2c45-e828-418a-9719-b8ae345c82cf"
export GENIE_APPLICATION_DIR="${GENIE_JOB_DIR}/genie/applications"
export GENIE_COMMAND_DIR="${GENIE_JOB_DIR}/genie/command/cmd1"
export GENIE_COMMAND_ID="cmd1"
export GENIE_COMMAND_NAME="Unix Bash command"
export GENIE_COMMAND_TAGS="bash,genie.id:cmd1,genie.name:Unix Bash command"
export GENIE_CLUSTER_DIR="${GENIE_JOB_DIR}/genie/cluster/cluster1"
export GENIE_CLUSTER_ID="cluster1"
export GENIE_CLUSTER_NAME="Local laptop"
export GENIE_CLUSTER_TAGS="genie.id:cluster1,genie.name:Local laptop,localhost"
export GENIE_JOB_ID="cb0b2c45-e828-418a-9719-b8ae345c82cf"
export GENIE_JOB_NAME="List * ... Directories bash job"
export GENIE_JOB_MEMORY=1024
export GENIE_JOB_TAGS="33ecdb92-124a-4f52-a11f-1b6cb832f295,6f9a1aab-1b32-4f17-a2d5-30b1197ac8aa"
export GENIE_REQUESTED_COMMAND_TAGS="bash"
export GENIE_REQUESTED_CLUSTER_TAGS="[[localhost]]"
export GENIE_REQUESTED_CLUSTER_TAGS_0="localhost"
export GENIE_VERSION=3
# Sourcing setup file from Cluster: cluster1
source ${GENIE_JOB_DIR}/genie/cluster/cluster1/setupfile
# Sourcing setup file from Application: app1
source ${GENIE_JOB_DIR}/genie/applications/app1/setupfile
# Sourcing setup file from Application: app2
source ${GENIE_JOB_DIR}/genie/applications/app2/setupfile
# Sourcing setup file from Command: cmd1
source ${GENIE_JOB_DIR}/genie/command/cmd1/setupfile
# Sourcing setup file specified in job request
source ${GENIE_JOB_DIR}/jobsetupfile
# Dump the environment to a env.log file
env | sort > ${GENIE_JOB_DIR}/genie/logs/env.log
# Kick off the command in background mode and wait for it using its pid
/bin/bash -c 'echo hello world' > ${GENIE_JOB_DIR}/stdout 2> ${GENIE_JOB_DIR}/stderr &
export CHILDREN_PID=$!
wait ${CHILDREN_PID}
# Write the return code from the command in the done file.
printf '{"exitCode": "%s"}\n' "$?" > ${GENIE_JOB_DIR}/genie/genie.done.temp
# Swapping done file, unless one exist created by trap handler.
mv -n ${GENIE_JOB_DIR}/genie/genie.done.temp ${GENIE_JOB_DIR}/genie/genie.done
echo End: `date '+%Y-%m-%d %H:%M:%S'`
6.7. Get a Job Request
6.7.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
true |
|
clusterCriterias |
Array |
List of cluster criteria’s for which a match will be attempted with register cluster tags. Each set of tags within a given cluster criteria must have at least one non-blank (e.g. ' ', ' ', null) tag. |
Must not be empty |
false |
commandCriteria |
Array |
Set of tags which will attempt to match against the commands linked to selected cluster. There must be at least one non-blank (e.g. ' ', ' ', null) criteria within the set |
Must not be empty |
false |
group |
String |
A group that the job should be run under on the linux system |
Size must be between 0 and 255 inclusive |
true |
disableLogArchival |
Boolean |
If you want to disable backing up job output files set this to true. Default: false |
true |
|
String |
If you want e-mail notification on job completion enter address here |
Must be a well-formed email address. Size must be between 0 and 255 inclusive |
true |
|
cpu |
Number |
For future use. Currently has no impact. |
Must be at least 1 |
true |
memory |
Number |
The amount of memory (in MB) desired for job client. Cannot exceed configured max. |
Must be at least 1 |
true |
timeout |
Number |
The timeout (in seconds) after which job will be killed by system, system setting used if not set |
Must be at least 1 |
true |
configs |
Array |
URI’s of configuration files which will be downloaded into job working directory at runtime |
true |
|
dependencies |
Array |
URI’s of dependency files which will be downloaded into job working directory at runtime |
true |
|
applications |
Array |
Complete list of application ids if power user wishes to override selected command defaults |
true |
|
grouping |
String |
The grouping of the job relative to other jobs. e.g. scheduler job name |
true |
|
groupingInstance |
String |
The grouping instance of the job relative to other jobs. e.g. scheduler job run |
true |
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/request HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 2096
{
"id" : "cb0b2c45-e828-418a-9719-b8ae345c82cf",
"created" : "2018-04-11T18:03:06.236Z",
"updated" : "2018-04-11T18:03:06.583Z",
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"metadata" : {
"schedulerJobName" : "04befc7d-85bb-45d7-b786-2d368f80dcd8",
"schedulerRunId" : "c64ccbeb-d67d-4c99-95b2-9109e1c448af"
},
"tags" : [ "6f9a1aab-1b32-4f17-a2d5-30b1197ac8aa", "33ecdb92-124a-4f52-a11f-1b6cb832f295" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/config1" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"applications" : [ ],
"grouping" : null,
"groupingInstance" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/request"
},
"job" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf"
},
"execution" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/execution"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/status"
},
"metadata" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/metadata"
}
}
}
6.8. Get a Job Execution
6.8.1. Description
Get metadata about a job execution. Information like where a job was run, its process exit code, etc.
6.8.3. API Docs
Request Examples
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
hostName |
String |
The host name of the Genie node responsible for the job |
Size must be between 1 and 1024 inclusive |
false |
processId |
Number |
The id of the job client process on the Genie node |
true |
|
checkDelay |
Number |
The amount of time in milliseconds between checks of the job status by Genie |
Must be at least 1 |
true |
timeout |
String |
The date (UTC ISO8601 with millis) when the job will be killed by Genie due to timeout |
true |
|
exitCode |
Number |
The job client process exit code after the job is done |
true |
|
memory |
Number |
The amount of memory (in MB) allocated to the job client |
Must be at least 1 |
true |
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/execution HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1076
{
"id" : "cb0b2c45-e828-418a-9719-b8ae345c82cf",
"created" : "2018-04-11T18:03:06.236Z",
"updated" : "2018-04-11T18:03:06.583Z",
"hostName" : "travis-job-a61444d2-bf45-4eb3-a261-1b41b7eb2f18",
"processId" : 7511,
"checkDelay" : 1,
"timeout" : "2018-04-18T18:03:06.443Z",
"exitCode" : 0,
"memory" : 1024,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/execution"
},
"job" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf"
},
"request" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/request"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/status"
},
"metadata" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/metadata"
}
}
}
6.9. Get Job Metadata
6.9.1. Description
Get metadata about a job beyond core information. Information like where a job was submitted from, how large its output files were, etc.
6.9.3. API Docs
Request Examples
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
clientHost |
String |
The host name of the client that submitted the job to Genie |
true |
|
userAgent |
String |
The user agent string that was passed to Genie on job request |
true |
|
numAttachments |
Number |
The number of attachments sent to Genie with the job request |
true |
|
totalSizeOfAttachments |
Number |
The total size of all attachments sent to Genie with the job request. In bytes. |
true |
|
stdOutSize |
Number |
The final size of the stdout file after a job is completed. In bytes. |
true |
|
stdErrSize |
Number |
The final size of the stderr file after a job is completed. In bytes. |
true |
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/metadata HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1039
{
"id" : "cb0b2c45-e828-418a-9719-b8ae345c82cf",
"created" : "2018-04-11T18:03:06.236Z",
"updated" : "2018-04-11T18:03:06.583Z",
"clientHost" : "127.0.0.1",
"userAgent" : null,
"numAttachments" : 0,
"totalSizeOfAttachments" : 0,
"stdOutSize" : 12,
"stdErrSize" : 0,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/metadata"
},
"job" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf"
},
"request" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/request"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/output"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/status"
},
"execution" : {
"href" : "https://genie.example.com/api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/execution"
}
}
}
6.10. Get the Cluster a Job Ran On
6.10.1. Description
Get the metadata about the cluster which was selected to run the job based on the cluster and command criteria in the job request.
6.10.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the cluster |
true |
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/cluster HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1375
{
"id" : "cluster1",
"created" : "2018-04-11T18:03:06.038Z",
"updated" : "2018-04-11T18:03:06.179Z",
"version" : "1.0",
"user" : "genie",
"name" : "Local laptop",
"description" : null,
"metadata" : null,
"tags" : [ "localhost", "genie.id:cluster1", "genie.name:Local laptop" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/config2" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/dep1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/dep2" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/setupfile",
"status" : "UP",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters/cluster1"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/clusters/cluster1/commands{?status}",
"templated" : true
}
}
}
6.11. Get the Command a Job Executed
6.11.1. Description
Get the metadata about the command which was selected to be executed by the job based on the cluster and command criteria in the job request.
6.11.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Number |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
dependencies |
Array |
The dependencies for the command |
true |
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/command HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1684
{
"id" : "cmd1",
"created" : "2018-04-11T18:03:06.139Z",
"updated" : "2018-04-11T18:03:06.162Z",
"version" : "1.0",
"user" : "genie",
"name" : "Unix Bash command",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:Unix Bash command", "bash", "genie.id:cmd1" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/config2", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/GB18030.txt" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/dep1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/dep2" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/setupfile",
"status" : "ACTIVE",
"executable" : "/bin/bash",
"checkDelay" : 1,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/cmd1"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/cmd1/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/cmd1/clusters{?status}",
"templated" : true
}
}
}
6.12. Get the Applications a Job Command Used
6.12.1. Description
Get the list of applications that were used by the command selected by job to execute.
6.12.3. API Docs
Request Examples
Response Documentation
HTTP Messages
Request
GET /api/v3/jobs/cb0b2c45-e828-418a-9719-b8ae345c82cf/applications HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 2674
[ {
"id" : "app1",
"created" : "2018-04-11T18:03:05.835Z",
"updated" : "2018-04-11T18:03:05.835Z",
"version" : "1.0",
"user" : "genie",
"name" : "Application 1",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:app1", "genie.name:Application 1" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/config2" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/dep1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/dep2" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/setupfile",
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/app1"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/app1/commands{?status}"
} ]
}, {
"id" : "app2",
"created" : "2018-04-11T18:03:05.935Z",
"updated" : "2018-04-11T18:03:05.935Z",
"version" : "1.0",
"user" : "genie",
"name" : "Application 2",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:app2", "genie.name:Application 2" ],
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/config2" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/dep2", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/dep1" ],
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/setupfile",
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/app2"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/app2/commands{?status}"
} ]
} ]
7. Applications API
7.1. Create an Application
7.1.2. Endpoint
POST /api/v3/applications
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
7.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/applications' -i -X POST -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"metadata" : null,
"tags" : [ "type:spark", "ver:1.5.1" ],
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"status" : "ACTIVE",
"type" : "spark"
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"metadata" : null,
"tags" : [ "type:spark", "ver:1.5.1" ],
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"status" : "ACTIVE",
"type" : "spark"
}' | http POST 'https://genie.example.com/api/v3/applications' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
POST /api/v3/applications HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 448
{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"metadata" : null,
"tags" : [ "type:spark", "ver:1.5.1" ],
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"status" : "ACTIVE",
"type" : "spark"
}
7.2. Get an Application
7.2.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
HTTP Messages
Request
GET /api/v3/applications/50a169e1-85fa-4b74-93cc-fc14e0f4ec4d HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 906
{
"id" : "50a169e1-85fa-4b74-93cc-fc14e0f4ec4d",
"created" : "2018-04-11T18:03:26.369Z",
"updated" : "2018-04-11T18:03:26.369Z",
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"metadata" : null,
"tags" : [ "type:spark", "ver:1.5.1", "genie.name:spark", "genie.id:50a169e1-85fa-4b74-93cc-fc14e0f4ec4d" ],
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"status" : "ACTIVE",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/50a169e1-85fa-4b74-93cc-fc14e0f4ec4d"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/50a169e1-85fa-4b74-93cc-fc14e0f4ec4d/commands{?status}",
"templated" : true
}
}
}
7.3. Find Applications
7.3.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The name of the applications to find. Use % to perform a regex like query |
|
The user of the applications to find. Use % to perform a regex like query |
|
The status(es) of the applications to find. Can have multiple. Options: [ACTIVE, DEPRECATED, INACTIVE] |
|
The tag(s) of the applications to find. Can have multiple. |
|
The type of the applications to find. Use % to perform a regex like query |
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.applicationList |
Array |
The found applications. |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 6382
{
"_embedded" : {
"applicationList" : [ {
"id" : "30cb0397-5f94-4e92-8540-f2aad34b3098",
"created" : "2018-04-11T18:03:25.530Z",
"updated" : "2018-04-11T18:03:25.530Z",
"version" : "1.0.0",
"user" : "genieUser7",
"name" : "hive",
"description" : "Hive 1.0.0 for Genie",
"metadata" : null,
"tags" : [ "genie.id:30cb0397-5f94-4e92-8540-f2aad34b3098", "genie.name:hive", "type:hive", "ver:1.0.0" ],
"configs" : [ "s3://mybucket/hive/hive-env.sh", "s3://mybucket/hive/hive-log4j.properties" ],
"dependencies" : [ "s3://mybucket/hive/hive-1.0.0.tar.gz" ],
"setupFile" : "s3://mybucket/hive/setup-hive.sh",
"status" : "ACTIVE",
"type" : "hive",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/30cb0397-5f94-4e92-8540-f2aad34b3098"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/30cb0397-5f94-4e92-8540-f2aad34b3098/commands{?status}",
"templated" : true
}
}
}, {
"id" : "d6f56b41-3e63-462d-a001-1f311236b2a8",
"created" : "2018-04-11T18:03:25.501Z",
"updated" : "2018-04-11T18:03:25.501Z",
"version" : "0.4.0",
"user" : "genieUser6",
"name" : "spark",
"description" : "Pig 0.15.0 for Genie",
"metadata" : null,
"tags" : [ "ver:0.15.0", "type:pig", "genie.id:d6f56b41-3e63-462d-a001-1f311236b2a8", "genie.name:spark" ],
"configs" : [ "s3://mybucket/pig/pig.properties" ],
"dependencies" : [ "s3://mybucket/pig/pig-0.15.0.tar.gz" ],
"setupFile" : "s3://mybucket/pig/setup-pig.sh",
"status" : "ACTIVE",
"type" : "pig",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/d6f56b41-3e63-462d-a001-1f311236b2a8"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/d6f56b41-3e63-462d-a001-1f311236b2a8/commands{?status}",
"templated" : true
}
}
}, {
"id" : "e7130d86-e45d-4c76-86e8-325ce56bdcb0",
"created" : "2018-04-11T18:03:25.478Z",
"updated" : "2018-04-11T18:03:25.478Z",
"version" : "1.3.1",
"user" : "genieUser5",
"name" : "spark",
"description" : "Spark 1.3.1 for Genie",
"metadata" : null,
"tags" : [ "type:spark", "genie.id:e7130d86-e45d-4c76-86e8-325ce56bdcb0", "ver:1.3.1", "genie.name:spark" ],
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.3.1.tar.gz" ],
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"status" : "DEPRECATED",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/e7130d86-e45d-4c76-86e8-325ce56bdcb0"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/e7130d86-e45d-4c76-86e8-325ce56bdcb0/commands{?status}",
"templated" : true
}
}
}, {
"id" : "d96d068b-0ecf-44e8-b423-2b34756b0675",
"created" : "2018-04-11T18:03:25.459Z",
"updated" : "2018-04-11T18:03:25.459Z",
"version" : "1.4.0",
"user" : "genieUser4",
"name" : "spark",
"description" : "Spark 1.4.0 for Genie",
"metadata" : null,
"tags" : [ "genie.id:d96d068b-0ecf-44e8-b423-2b34756b0675", "type:spark", "ver:1.4.0", "genie.name:spark" ],
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.4.0.tar.gz" ],
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"status" : "DEPRECATED",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/d96d068b-0ecf-44e8-b423-2b34756b0675"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/d96d068b-0ecf-44e8-b423-2b34756b0675/commands{?status}",
"templated" : true
}
}
}, {
"id" : "7b62ed02-2f02-4a85-93d9-1bfb2cbbe743",
"created" : "2018-04-11T18:03:25.418Z",
"updated" : "2018-04-11T18:03:25.418Z",
"version" : "1.5.0",
"user" : "genieUser2",
"name" : "spark",
"description" : "Spark 1.5.0 for Genie",
"metadata" : null,
"tags" : [ "type:spark", "ver:1.5.0", "genie.id:7b62ed02-2f02-4a85-93d9-1bfb2cbbe743", "genie.name:spark" ],
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.5.0.tar.gz" ],
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"status" : "ACTIVE",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/7b62ed02-2f02-4a85-93d9-1bfb2cbbe743"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/7b62ed02-2f02-4a85-93d9-1bfb2cbbe743/commands{?status}",
"templated" : true
}
}
}, {
"id" : "21edd56a-a422-48ec-8a3a-fc73b012ae59",
"created" : "2018-04-11T18:03:25.398Z",
"updated" : "2018-04-11T18:03:25.398Z",
"version" : "1.5.1",
"user" : "genieUser1",
"name" : "spark",
"description" : "Spark 1.5.1 for Genie",
"metadata" : null,
"tags" : [ "genie.id:21edd56a-a422-48ec-8a3a-fc73b012ae59", "type:spark", "ver:1.5.1", "genie.name:spark" ],
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.5.1.tar.gz" ],
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"status" : "ACTIVE",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/21edd56a-a422-48ec-8a3a-fc73b012ae59"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/21edd56a-a422-48ec-8a3a-fc73b012ae59/commands{?status}",
"templated" : true
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications?status=ACTIVE&status=DEPRECATED{&name,user,tag,type}",
"templated" : true
}
},
"page" : {
"size" : 10,
"totalElements" : 6,
"totalPages" : 1,
"number" : 0
}
}
7.4. Update an Entire Application
7.4.1. Description
Update the metadata about an application. This method does a complete replace of the resource. Usual flow is to call to get the most recent state then update what you want and call this API with the result.
For more fine grained control call the PATCH API. |
7.4.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/applications/418c2c0e-4bb2-4b57-855a-d946e1fa686a' -i -X PUT -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : "418c2c0e-4bb2-4b57-855a-d946e1fa686a",
"created" : "2018-04-11T18:03:26.867Z",
"updated" : "2018-04-11T18:03:26.867Z",
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:418c2c0e-4bb2-4b57-855a-d946e1fa686a", "genie.name:spark" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"type" : null
}'
HTTPie
$ echo '{
"id" : "418c2c0e-4bb2-4b57-855a-d946e1fa686a",
"created" : "2018-04-11T18:03:26.867Z",
"updated" : "2018-04-11T18:03:26.867Z",
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:418c2c0e-4bb2-4b57-855a-d946e1fa686a", "genie.name:spark" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"type" : null
}' | http PUT 'https://genie.example.com/api/v3/applications/418c2c0e-4bb2-4b57-855a-d946e1fa686a' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
PUT /api/v3/applications/418c2c0e-4bb2-4b57-855a-d946e1fa686a HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 434
{
"id" : "418c2c0e-4bb2-4b57-855a-d946e1fa686a",
"created" : "2018-04-11T18:03:26.867Z",
"updated" : "2018-04-11T18:03:26.867Z",
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:418c2c0e-4bb2-4b57-855a-d946e1fa686a", "genie.name:spark" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"type" : null
}
7.5. Update Parts of an Application
7.5.1. Description
Update the metadata about an application using JSON Patch. This will ONLY update the fields you request be changed.
For more information about JSON Patch RFC see the website. |
7.5.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
[] |
Array |
Array of patches to apply |
false |
|
[].op |
String |
Patch operation to perform. |
add, remove, replace, copy, move, test |
false |
[].path |
String |
The json path to operate on. e.g. /user |
false |
|
[].from |
String |
The json path to move or copy from. e.g. /user |
true |
|
[].value |
String |
The json value to put at the path for an add, replace or test |
true |
Request Examples
7.7. Delete an Application
7.8. Get Configs for Application
7.8.3. API Docs
Request Documentation
Request Examples
Response Documentation
7.9. Add Configs to Application
7.9.1. Description
Add configuration file locations to an existing application. Configurations are stored as a set so any duplicates will be ignored.
7.9.3. API Docs
Request Documentation
Request Examples
7.10. Update Configs for an Application
7.10.1. Description
Update the configuration file locations for an existing application. Stored as a set so duplicates are ignored.
7.10.3. API Docs
Request Documentation
Request Examples
7.11. Remove All Configs From an Application
7.12. Get Dependencies For an Application
7.12.3. API Docs
Request Documentation
Request Examples
Response Documentation
7.13. Add Dependencies to Application
7.13.1. Description
Add dependency file locations to an existing application. Stored as a set so all duplicates are ignored.
7.13.3. API Docs
Request Documentation
Request Examples
7.14. Update Dependencies for an Application
7.14.3. API Docs
Request Documentation
Request Examples
7.15. Remove All Dependencies From an Application
7.16. Get Tags For an Application
7.16.3. API Docs
Request Examples
7.17. Add Tags to Application
7.17.1. Description
Add tags to an existing application. Stored as a set so all duplicates are ignored.
genie.id:{id} and genie.name:{name} tags are automatically added by the service.
|
7.17.3. API Docs
Request Examples
7.18. Update Tags for an Application
7.18.1. Description
Update the tags for an existing application.
The genie.id:{id} and genie.name:{name} tags can’t be removed. They will automatically be added back
by the system.
|
7.18.3. API Docs
Request Examples
7.19. Remove All Tags From Application
7.19.1. Description
Remove all the tags for an existing application
The genie.id:{id} and genie.name:{name} tags will NOT be removed by this operation
|
7.20. Remove Tag From Application
7.20.1. Description
Remove a tag from an existing application.
You can’t remove the genie.id:{id} and genie.name:{name} tags. They will just be added back by the system
|
7.20.3. API Docs
Request Documentation
7.21. Get the Commands that Use an Application
7.21.3. API Docs
Request Documentation
Request Examples
HTTP Messages
Request
GET /api/v3/applications/418c2c0e-4bb2-4b57-855a-d946e1fa686a/commands?status=ACTIVE&status=INACTIVE HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1064
[ {
"id" : "c172de49-da40-403a-b7b6-504ea2a73bc7",
"created" : "2018-04-11T18:03:27.079Z",
"updated" : "2018-04-11T18:03:27.128Z",
"version" : "355a96eb-cde6-4081-9320-c15ad45cbc43",
"user" : "355a96eb-cde6-4081-9320-c15ad45cbc43",
"name" : "355a96eb-cde6-4081-9320-c15ad45cbc43",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:355a96eb-cde6-4081-9320-c15ad45cbc43", "genie.id:c172de49-da40-403a-b7b6-504ea2a73bc7" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "ACTIVE",
"executable" : "355a96eb-cde6-4081-9320-c15ad45cbc43",
"checkDelay" : 1000,
"memory" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/commands/c172de49-da40-403a-b7b6-504ea2a73bc7"
}, {
"rel" : "applications",
"href" : "https://genie.example.com/api/v3/commands/c172de49-da40-403a-b7b6-504ea2a73bc7/applications"
}, {
"rel" : "clusters",
"href" : "https://genie.example.com/api/v3/commands/c172de49-da40-403a-b7b6-504ea2a73bc7/clusters{?status}"
} ]
} ]
8. Clusters API
8.1. Create a Cluster
8.1.2. Endpoint
POST /api/v3/clusters
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
8.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the cluster |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/clusters' -i -X POST -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "UP"
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "UP"
}' | http POST 'https://genie.example.com/api/v3/clusters' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
POST /api/v3/clusters HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 266
{
"id" : null,
"created" : null,
"updated" : null,
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "UP"
}
8.2. Get a Cluster
8.2.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the cluster |
true |
HTTP Messages
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 716
{
"id" : "ac58dd80-c9d4-488e-bc9b-49c049d3403a",
"created" : "2018-04-11T18:03:28.166Z",
"updated" : "2018-04-11T18:03:28.166Z",
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:h2prod", "genie.id:ac58dd80-c9d4-488e-bc9b-49c049d3403a" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "UP",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters/ac58dd80-c9d4-488e-bc9b-49c049d3403a"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/clusters/ac58dd80-c9d4-488e-bc9b-49c049d3403a/commands{?status}",
"templated" : true
}
}
}
8.3. Find Clusters
8.3.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The name of the clusters to find. Use % to perform a regex like query |
|
The status(es) of the clusters to find. Can have multiple. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
|
The tag(s) of the clusters to find. Can have multiple. |
|
The minimum time (in milliseconds from epoch UTC) that the cluster(s) were updated at. |
|
The maximum time (in milliseconds from epoch UTC) that the cluster(s) were updated at. |
Request Examples
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.clusterList |
Array |
The found clusters. |
false |
HTTP Messages
Request
GET /api/v3/clusters?tag=genie.id%3A02481c79-836a-4566-a9b2-78ee1ea466bf HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1297
{
"_embedded" : {
"clusterList" : [ {
"id" : "02481c79-836a-4566-a9b2-78ee1ea466bf",
"created" : "2018-04-11T18:03:31.665Z",
"updated" : "2018-04-11T18:03:31.665Z",
"version" : "d818b126-dae8-4d39-a9a9-fc81dee6228b",
"user" : "e0c7b577-c746-46c8-8f20-954754494983",
"name" : "3dae7342-18c9-4046-bc42-b6e8bc2f45de",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:3dae7342-18c9-4046-bc42-b6e8bc2f45de", "genie.id:02481c79-836a-4566-a9b2-78ee1ea466bf" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "UP",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters/02481c79-836a-4566-a9b2-78ee1ea466bf"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/clusters/02481c79-836a-4566-a9b2-78ee1ea466bf/commands{?status}",
"templated" : true
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters?tag=genie.id:02481c79-836a-4566-a9b2-78ee1ea466bf{&name,status,minUpdateTime,maxUpdateTime}",
"templated" : true
}
},
"page" : {
"size" : 64,
"totalElements" : 1,
"totalPages" : 1,
"number" : 0
}
}
8.4. Update an Entire Cluster
8.4.1. Description
Update the metadata about a cluster. This method does a complete replace of the resource. Usual flow is to call to get the most recent state then update what you want and call this API with the result.
For more fine grained control call the PATCH API. |
8.4.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the cluster |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/clusters/275df087-c610-4051-9c5f-3a187c1beb8f' -i -X PUT -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : "275df087-c610-4051-9c5f-3a187c1beb8f",
"created" : "2018-04-11T18:03:34.115Z",
"updated" : "2018-04-11T18:03:34.115Z",
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:275df087-c610-4051-9c5f-3a187c1beb8f", "genie.name:h2prod" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "OUT_OF_SERVICE"
}'
HTTPie
$ echo '{
"id" : "275df087-c610-4051-9c5f-3a187c1beb8f",
"created" : "2018-04-11T18:03:34.115Z",
"updated" : "2018-04-11T18:03:34.115Z",
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:275df087-c610-4051-9c5f-3a187c1beb8f", "genie.name:h2prod" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "OUT_OF_SERVICE"
}' | http PUT 'https://genie.example.com/api/v3/clusters/275df087-c610-4051-9c5f-3a187c1beb8f' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
PUT /api/v3/clusters/275df087-c610-4051-9c5f-3a187c1beb8f HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 425
{
"id" : "275df087-c610-4051-9c5f-3a187c1beb8f",
"created" : "2018-04-11T18:03:34.115Z",
"updated" : "2018-04-11T18:03:34.115Z",
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:275df087-c610-4051-9c5f-3a187c1beb8f", "genie.name:h2prod" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "OUT_OF_SERVICE"
}
8.5. Update Parts of a Cluster
8.5.1. Description
Update the metadata about a cluster using JSON Patch. This will ONLY update the fields you request be changed.
For more information about JSON Patch RFC see the website. |
8.5.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
[] |
Array |
Array of patches to apply |
false |
|
[].op |
String |
Patch operation to perform. |
add, remove, replace, copy, move, test |
false |
[].path |
String |
The json path to operate on. e.g. /user |
false |
|
[].from |
String |
The json path to move or copy from. e.g. /user |
true |
|
[].value |
String |
The json value to put at the path for an add, replace or test |
true |
Request Examples
8.6. Delete All Clusters
8.6.1. Description
Delete all the clusters currently stored in the system.
You won’t be able to delete any cluster that has run any job that is still in the system to maintain the ability for users to look up information about their job (where it ran, etc) |
8.7. Delete a Cluster
8.7.1. Description
Delete a cluster currently configured in the system.
You won’t be able to delete a cluster that has run any job that is still in the system to maintain the ability for users to look up information about their job (where it ran, etc) |
8.8. Get Configs for Cluster
8.8.3. API Docs
Request Examples
Response Documentation
8.9. Add Configs to Cluster
8.9.1. Description
Add configuration file locations to an existing cluster.
Configurations are stored as a set so any duplicates will be ignored. |
8.9.3. API Docs
Request Examples
8.10. Update Configs for a Cluster
8.10.1. Description
Update the configuration file locations for an existing cluster. Stored as a set so duplicates are ignored.
8.10.3. API Docs
Request Examples
8.11. Remove All Configs From a Cluster
8.12. Get Dependencies For an Cluster
8.12.3. API Docs
Request Documentation
Request Examples
Response Documentation
8.13. Add Dependencies to Cluster
8.13.1. Description
Add dependency file locations to an existing cluster. Stored as a set so all duplicates are ignored.
8.13.3. API Docs
Request Documentation
Request Examples
8.14. Update Dependencies for an Cluster
8.14.3. API Docs
Request Examples
8.15. Remove All Dependencies From an Cluster
8.16. Get Tags For a Cluster
8.17. Add Tags to Cluster
8.17.1. Description
Add tags to an existing cluster. Stored as a set so all duplicates are ignored.
genie.id:{id} and genie.name:{name} tags are automatically added by the service.
|
8.17.3. API Docs
Request Examples
8.18. Update Tags for a Cluster
8.18.1. Description
Update the tags for an existing cluster.
The genie.id:{id} and genie.name:{name} tags can’t be removed. They will automatically be added back
by the system.
|
8.18.3. API Docs
Request Examples
8.19. Remove All Tags From Cluster
8.19.1. Description
Remove all the tags for an existing cluster
The genie.id:{id} and genie.name:{name} tags will NOT be removed by this operation
|
8.20. Remove Tag From Cluster
8.20.1. Description
Remove a tag from an existing cluster.
You can’t remove the genie.id:{id} and genie.name:{name} tags. They will just be added back by the system
|
8.20.3. API Docs
Request Documentation
8.21. Add Commands to a Cluster
8.21.1. Description
Append new commands (as array of their ID’s) to the existing list of commands the cluster can run.
The order of the commands is important. This represents priority order if two commands with same tags match for the given cluster the one earlier in the list will be selected. |
The commands for the ID’s must already exist in the system |
8.21.3. API Docs
Request Examples
8.22. Get the Commands for a Cluster
8.22.1. Description
Get the commands currently linked to the cluster. This indicates the command is able to be run on that cluster.
The order of the commands is important. This represents priority order if two commands with same tags match for the given cluster the one earlier in the list will be selected. |
8.22.3. API Docs
Request Examples
HTTP Messages
Request
GET /api/v3/clusters/275df087-c610-4051-9c5f-3a187c1beb8f/commands?status=INACTIVE HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1066
[ {
"id" : "490d56bb-8059-4b37-96b3-4c9aebb55488",
"created" : "2018-04-11T18:03:29.182Z",
"updated" : "2018-04-11T18:03:29.182Z",
"version" : "286f220a-53ae-4abc-af4b-9ef611b8c5ba",
"user" : "286f220a-53ae-4abc-af4b-9ef611b8c5ba",
"name" : "286f220a-53ae-4abc-af4b-9ef611b8c5ba",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:490d56bb-8059-4b37-96b3-4c9aebb55488", "genie.name:286f220a-53ae-4abc-af4b-9ef611b8c5ba" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"executable" : "286f220a-53ae-4abc-af4b-9ef611b8c5ba",
"checkDelay" : 1000,
"memory" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/commands/490d56bb-8059-4b37-96b3-4c9aebb55488"
}, {
"rel" : "applications",
"href" : "https://genie.example.com/api/v3/commands/490d56bb-8059-4b37-96b3-4c9aebb55488/applications"
}, {
"rel" : "clusters",
"href" : "https://genie.example.com/api/v3/commands/490d56bb-8059-4b37-96b3-4c9aebb55488/clusters{?status}"
} ]
} ]
8.23. Set Commands for a Cluster
8.23.1. Description
Set the commands (as array of their ID’s) that can run on the cluster.
The order of the commands is important. This represents priority order if two commands with same tags match for the given cluster the one earlier in the list will be selected. |
The commands for the ID’s must already exist in the system |
8.23.3. API Docs
Request Examples
8.24. Remove All Commands From a Cluster
8.24.1. Description
Remove all the commands currently associated with the given cluster.
This does NOT delete the command(s) from the system just unlinks them |
8.25. Remove A Single Command From a Cluster
8.25.1. Description
Remove the identified command from the cluster.
This does NOT delete the command from the system just unlinks it |
8.25.3. API Docs
Request Documentation
9. Commands API
9.1. Create a Command
9.1.2. Endpoint
POST /api/v3/commands
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
9.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Number |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
dependencies |
Array |
The dependencies for the command |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/commands' -i -X POST -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : "Hive command v1.0.0",
"metadata" : null,
"tags" : [ "tag:bar", "tag:foo" ],
"configs" : [ "s3:///path/to/config-foo", "s3:///path/to/config-bar" ],
"dependencies" : [ "/path/to/file/foo", "/path/to/file/bar" ],
"setupFile" : null,
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : 1024
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : "Hive command v1.0.0",
"metadata" : null,
"tags" : [ "tag:bar", "tag:foo" ],
"configs" : [ "s3:///path/to/config-foo", "s3:///path/to/config-bar" ],
"dependencies" : [ "/path/to/file/foo", "/path/to/file/bar" ],
"setupFile" : null,
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : 1024
}' | http POST 'https://genie.example.com/api/v3/commands' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
POST /api/v3/commands HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 485
{
"id" : null,
"created" : null,
"updated" : null,
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : "Hive command v1.0.0",
"metadata" : null,
"tags" : [ "tag:bar", "tag:foo" ],
"configs" : [ "s3:///path/to/config-foo", "s3:///path/to/config-bar" ],
"dependencies" : [ "/path/to/file/foo", "/path/to/file/bar" ],
"setupFile" : null,
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : 1024
}
9.2. Get a Command
9.2.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Number |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
dependencies |
Array |
The dependencies for the command |
true |
HTTP Messages
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 1073
{
"id" : "2daa9b02-5ff0-429c-8906-afe9601294e9",
"created" : "2018-04-11T18:03:37.925Z",
"updated" : "2018-04-11T18:03:37.925Z",
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : "Hive command v1.0.0",
"metadata" : null,
"tags" : [ "genie.id:2daa9b02-5ff0-429c-8906-afe9601294e9", "tag:bar", "tag:foo", "genie.name:hive" ],
"configs" : [ "s3:///path/to/config-foo", "s3:///path/to/config-bar" ],
"dependencies" : [ "/path/to/file/foo", "/path/to/file/bar" ],
"setupFile" : null,
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : 1024,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/2daa9b02-5ff0-429c-8906-afe9601294e9"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/2daa9b02-5ff0-429c-8906-afe9601294e9/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/2daa9b02-5ff0-429c-8906-afe9601294e9/clusters{?status}",
"templated" : true
}
}
}
9.3. Find Commands
9.3.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The name of the commands to find. Use % to perform a regex like query |
|
The user of the commands to find. Use % to perform a regex like query |
|
The status(es) of the commands to find. Can have multiple. Options: [ACTIVE, DEPRECATED, INACTIVE] |
|
The tag(s) of the commands to find. Can have multiple. |
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.commandList |
Array |
The found commands. |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 2719
{
"_embedded" : {
"commandList" : [ {
"id" : "7f4f8b54-2cc3-4564-8a50-a86bbf5f5fca",
"created" : "2018-04-11T18:03:36.822Z",
"updated" : "2018-04-11T18:03:36.822Z",
"version" : "4a95f93c-176a-486b-94ca-93fdcd4c9fb2",
"user" : "3a1c4266-796c-48f6-98d1-d442180c5c9b",
"name" : "fe764819-6260-4ad6-9817-bffff9f2af4a",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:7f4f8b54-2cc3-4564-8a50-a86bbf5f5fca", "genie.name:fe764819-6260-4ad6-9817-bffff9f2af4a" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"executable" : "deb352a3-9540-4a78-9123-41859966126a",
"checkDelay" : 10000,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/7f4f8b54-2cc3-4564-8a50-a86bbf5f5fca"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/7f4f8b54-2cc3-4564-8a50-a86bbf5f5fca/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/7f4f8b54-2cc3-4564-8a50-a86bbf5f5fca/clusters{?status}",
"templated" : true
}
}
}, {
"id" : "7ecbfd7d-bd3a-4c08-a8ba-ea82e39e149b",
"created" : "2018-04-11T18:03:34.792Z",
"updated" : "2018-04-11T18:03:34.792Z",
"version" : "b87395ce-f894-482b-a7d9-1b146b2b9448",
"user" : "f3a0bb35-1f73-402d-8f9f-e38a6348f28e",
"name" : "0c484913-94b0-4b99-98c2-2383505789c1",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:7ecbfd7d-bd3a-4c08-a8ba-ea82e39e149b", "genie.name:0c484913-94b0-4b99-98c2-2383505789c1" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "ACTIVE",
"executable" : "1e7aea0a-b132-4837-ae30-cb59b24e90e7",
"checkDelay" : 10000,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/7ecbfd7d-bd3a-4c08-a8ba-ea82e39e149b"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/7ecbfd7d-bd3a-4c08-a8ba-ea82e39e149b/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/7ecbfd7d-bd3a-4c08-a8ba-ea82e39e149b/clusters{?status}",
"templated" : true
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands?status=ACTIVE&status=INACTIVE{&name,user,tag}",
"templated" : true
}
},
"page" : {
"size" : 64,
"totalElements" : 2,
"totalPages" : 1,
"number" : 0
}
}
9.4. Update an Entire Command
9.4.1. Description
Update the metadata about a command. This method does a complete replace of the resource. Usual flow is to call to get the most recent state then update what you want and call this API with the result.
For more fine grained control call the PATCH API. |
9.4.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 1000 inclusive |
true |
metadata |
Object |
Any semi-structured metadata. Must be valid JSON |
true |
|
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Number |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
dependencies |
Array |
The dependencies for the command |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/commands/8a83f464-0081-4e9b-a12d-5c108319ef60' -i -X PUT -H 'Content-Type: application/json;charset=UTF-8' -d '{
"id" : "8a83f464-0081-4e9b-a12d-5c108319ef60",
"created" : "2018-04-11T18:03:38.271Z",
"updated" : "2018-04-11T18:03:38.271Z",
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:hive", "genie.id:8a83f464-0081-4e9b-a12d-5c108319ef60" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}'
HTTPie
$ echo '{
"id" : "8a83f464-0081-4e9b-a12d-5c108319ef60",
"created" : "2018-04-11T18:03:38.271Z",
"updated" : "2018-04-11T18:03:38.271Z",
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:hive", "genie.id:8a83f464-0081-4e9b-a12d-5c108319ef60" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}' | http PUT 'https://genie.example.com/api/v3/commands/8a83f464-0081-4e9b-a12d-5c108319ef60' 'Content-Type:application/json;charset=UTF-8'
HTTP Messages
Request
PUT /api/v3/commands/8a83f464-0081-4e9b-a12d-5c108319ef60 HTTP/1.1
Content-Type: application/json;charset=UTF-8
Host: genie.example.com
Content-Length: 498
{
"id" : "8a83f464-0081-4e9b-a12d-5c108319ef60",
"created" : "2018-04-11T18:03:38.271Z",
"updated" : "2018-04-11T18:03:38.271Z",
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:hive", "genie.id:8a83f464-0081-4e9b-a12d-5c108319ef60" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "INACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}
9.5. Update Parts of a Command
9.5.1. Description
Update the metadata about a command using JSON Patch. This will ONLY update the fields you request be changed.
For more information about JSON Patch RFC see the website. |
9.5.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
[] |
Array |
Array of patches to apply |
false |
|
[].op |
String |
Patch operation to perform. |
add, remove, replace, copy, move, test |
false |
[].path |
String |
The json path to operate on. e.g. /user |
false |
|
[].from |
String |
The json path to move or copy from. e.g. /user |
true |
|
[].value |
String |
The json value to put at the path for an add, replace or test |
true |
Request Examples
9.6. Delete All Commands
9.6.1. Description
Delete all the commands currently stored in the system.
You won’t be able to delete any command that has been used by any job that is still in the system to maintain the ability for users to look up information about their job (where it ran, what it ran, etc) |
9.7. Delete a Command
9.7.1. Description
Delete a command currently configured in the system.
You won’t be able to delete a command that has been used by any job that is still in the system to maintain the ability for users to look up information about their job (where it ran, what it ran, etc) |
9.8. Get Configs for Command
9.8.3. API Docs
Request Examples
Response Documentation
9.9. Add Configs to Command
9.9.1. Description
Add configuration file locations to an existing command.
Configurations are stored as a set so any duplicates will be ignored. |
9.9.3. API Docs
Request Examples
9.10. Update Configs for a Command
9.10.1. Description
Update the configuration file locations for an existing command. Stored as a set so duplicates are ignored.
9.10.3. API Docs
Request Examples
9.11. Remove All Configs From a Command
9.12. Get Dependencies For an Command
9.12.3. API Docs
Request Documentation
Request Examples
Response Documentation
9.13. Add Dependencies to Command
9.13.1. Description
Add dependency file locations to an existing command. Stored as a set so all duplicates are ignored.
9.13.3. API Docs
Request Documentation
Request Examples
9.14. Update Dependencies for an Command
9.14.3. API Docs
Request Documentation
Request Examples
9.15. Remove All Dependencies From an Command
9.16. Get Tags For a Command
9.17. Add Tags to Command
9.17.1. Description
Add tags to an existing command. Stored as a set so all duplicates are ignored.
genie.id:{id} and genie.name:{name} tags are automatically added by the service.
|
9.17.3. API Docs
Request Examples
9.18. Update Tags for a Command
9.18.1. Description
Update the tags for an existing command.
The genie.id:{id} and genie.name:{name} tags can’t be removed. They will automatically be added back
by the system.
|
9.18.3. API Docs
Request Examples
9.19. Remove All Tags From Command
9.19.1. Description
Remove all the tags for an existing command
The genie.id:{id} and genie.name:{name} tags will NOT be removed by this operation
|
9.20. Remove Tag From Command
9.20.1. Description
Remove a tag from an existing command.
You can’t remove the genie.id:{id} and genie.name:{name} tags. They will just be added back by the system
|
9.20.3. API Docs
Request Documentation
9.21. Add Applications to a Command
9.21.1. Description
Add new applications (as array of their ID’s) to the existing set of applications the command depends on
The applications for the ID’s must already exist in the system |
9.21.3. API Docs
Request Documentation
Request Examples
9.22. Get the Applications for a Command
9.22.1. Description
Get the applications the command currently depends on. These are the applications that will be installed at runtime when a job is run with the given command
9.22.3. API Docs
Request Documentation
Request Examples
Response Documentation
HTTP Messages
Request
GET /api/v3/commands/8a83f464-0081-4e9b-a12d-5c108319ef60/applications HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 2540
[ {
"id" : "1ad8189c-29e4-45c6-ba01-c4686a779bc9",
"created" : "2018-04-11T18:03:37.260Z",
"updated" : "2018-04-11T18:03:37.260Z",
"version" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"user" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"name" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:1ad8189c-29e4-45c6-ba01-c4686a779bc9", "genie.name:e7d0f690-1dde-414c-bc67-ada5e3722ef3" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/1ad8189c-29e4-45c6-ba01-c4686a779bc9"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/1ad8189c-29e4-45c6-ba01-c4686a779bc9/commands{?status}"
} ]
}, {
"id" : "0d8dee6c-996e-497f-8f78-0f18cf85bf38",
"created" : "2018-04-11T18:03:37.274Z",
"updated" : "2018-04-11T18:03:37.274Z",
"version" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"user" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"name" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:0d8dee6c-996e-497f-8f78-0f18cf85bf38", "genie.name:e7d0f690-1dde-414c-bc67-ada5e3722ef3" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/0d8dee6c-996e-497f-8f78-0f18cf85bf38"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/0d8dee6c-996e-497f-8f78-0f18cf85bf38/commands{?status}"
} ]
}, {
"id" : "d61e241d-9370-4686-ac47-234cf2d18372",
"created" : "2018-04-11T18:03:37.330Z",
"updated" : "2018-04-11T18:03:37.330Z",
"version" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"user" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"name" : "e7d0f690-1dde-414c-bc67-ada5e3722ef3",
"description" : null,
"metadata" : null,
"tags" : [ "genie.id:d61e241d-9370-4686-ac47-234cf2d18372", "genie.name:e7d0f690-1dde-414c-bc67-ada5e3722ef3" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/d61e241d-9370-4686-ac47-234cf2d18372"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/d61e241d-9370-4686-ac47-234cf2d18372/commands{?status}"
} ]
} ]
9.23. Set Applications for a Command
9.23.1. Description
Set the applications (as array of their ID’s) that are dependencies of the command.
The applications with the ID’s must already exist in the system |
9.23.3. API Docs
Request Documentation
Request Examples
9.24. Remove All Applications From a Command
9.24.1. Description
Remove all the applications currently set as dependencies of the command
This does NOT delete the application(s) from the system just unlinks them |
9.25. Remove A Single Application From a Command
9.25.1. Description
Remove the identified application as a dependency of the command.
This does NOT delete the application from the system just un-links it |
9.25.3. API Docs
Request Documentation
9.26. Get the Clusters that Have Command Available
9.26.3. API Docs
Request Examples
HTTP Messages
Request
GET /api/v3/commands/8a83f464-0081-4e9b-a12d-5c108319ef60/clusters?status=UP HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
X-Application-Context: genie:integration,db,db-h2:0
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 819
[ {
"id" : "7b8da2ec-1045-423d-8932-18b37d44cc14",
"created" : "2018-04-11T18:03:37.710Z",
"updated" : "2018-04-11T18:03:37.736Z",
"version" : "78e16c2e-7c8f-4201-823e-34f43931237d",
"user" : "78e16c2e-7c8f-4201-823e-34f43931237d",
"name" : "78e16c2e-7c8f-4201-823e-34f43931237d",
"description" : null,
"metadata" : null,
"tags" : [ "genie.name:78e16c2e-7c8f-4201-823e-34f43931237d", "genie.id:7b8da2ec-1045-423d-8932-18b37d44cc14" ],
"configs" : [ ],
"dependencies" : [ ],
"setupFile" : null,
"status" : "UP",
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/clusters/7b8da2ec-1045-423d-8932-18b37d44cc14"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/clusters/7b8da2ec-1045-423d-8932-18b37d44cc14/commands{?status}"
} ]
} ]