1. Introduction
This is the REST API documentation for Genie version 3.0.0. Here you will find specific details of how to invoke the Genie REST API and what you can expect as return values.
For more Genie information see the website. |
For high level information and installation instructions please see the reference guide. |
For a demo of this version of Genie please see demo guide. |
2. Assumptions
This documentation assumes a basic knowledge of how REST APIs work. If you don’t know things like HTTP verbs see websites like this one.
3. HATEOAS
A lot of the API’s return application/hal+json
as a content type. This is a superset of JSON built to support
HATEOAS (Hypermedia as the Engine of Application State). Rather than re-writing what
HATEOAS is see this great write up on the Spring Website about it.
4. Security
All of this documentation assumes security is disabled for Genie. If your installation has security enabled the REST APIs will be secured via OAuth2 or x509 certs or both.
These are the currently supported API security mechanisms but others could be plugged in. We’ll only talk about OAuth2 here for now until another implementation is done. |
4.1. OAuth2
If OAuth2 is enabled you’ll need to send your JWT you got back from your provider along with your
API request. This is accomplished by setting your Authorization
header to a value of Bearer {your token here}
.
5. Resources
This section contains the documentation for various resources sent to and from the various Genie APIs. Note their structure and fields, particularly what is marked optional. Optional means when you submit one of these resources to Genie you don’t need to set that field. Genie either doesn’t require it or if it does, as is the case with ID’s, it will create one if you don’t provide it. When you get a resource all fields Genie has stored will be returned.
5.1. Application
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
5.2. Cluster
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
5.3. Command
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Null |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
5.4. Job Request
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
Must not be null. Size must be between 1 and 10000 inclusive |
false |
clusterCriterias |
Array |
List of cluster criteria’s for which a match will be attempted with register cluster tags |
Must not be empty |
false |
commandCriteria |
Array |
List of tags which will attempt to match against the commands linked to selected cluster |
Must not be empty |
false |
group |
Null |
A group that the job should be run under on the linux system |
Size must be between 0 and 255 inclusive |
true |
disableLogArchival |
Boolean |
If you want to disable backing up job output files set this to true. Default: false |
true |
|
Null |
If you want e-mail notification on job completion enter address here |
Must be a well-formed email address. Size must be between 0 and 255 inclusive |
true |
|
cpu |
Null |
For future use. Currently has no impact. |
Must be at least 1 |
true |
memory |
Null |
The amount of memory (in MB) desired for job client. Cannot exceed configured max. |
Must be at least 1 |
true |
timeout |
Null |
The timeout (in seconds) after which job will be killed by system, system setting used if not set |
Must be at least 1 |
true |
dependencies |
Array |
URI’s of dependency files which will be downloaded into job working directory at runtime |
true |
|
applications |
Array |
Complete list of application ids if power user wishes to override selected command defaults |
true |
5.5. Job
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
Size must be between 1 and 10000 inclusive |
false |
status |
String |
The status of the job. Options: [INIT, RUNNING, SUCCEEDED, KILLED, FAILED, INVALID] |
Must not be null |
false |
statusMsg |
String |
The status message of the job |
Size must be between 0 and 255 inclusive |
true |
started |
String |
The time (UTC ISO8601 with millis) the job was started |
true |
|
finished |
String |
The time (UTC ISO8601 with millis) the job finished |
true |
|
archiveLocation |
Null |
The URI where the working directory zip was stored |
Size must be between 0 and 1024 inclusive |
true |
clusterName |
String |
The name of the cluster the job was run on if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
commandName |
String |
The name of the command the job was run with if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
runtime |
String |
Runtime of the job in ISO8601 duration format |
Must not be null |
true |
5.6. Job Execution
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
hostName |
String |
The host name of the Genie node responsible for the job |
false |
|
processId |
Number |
The id of the job client process on the Genie node |
true |
|
checkDelay |
Number |
The amount of time in milliseconds between checks of the job status by Genie |
true |
|
timeout |
String |
The date (UTC ISO8601 with millis) when the job will be killed by Genie due to timeout |
true |
|
exitCode |
Number |
The job client process exit code after the job is done |
true |
|
memory |
Number |
The amount of memory (in MB) allocated to the job client |
true |
6. Jobs API
6.1. Submit a Job
6.1.1. Description
The main API in Genie. Submitting a job to be run.
There are two different ways to submit a job to Genie. Both go to the same endpoint but you can simply send a JSON
payload if you have no attachments. This means setting your content type to application/json
. If you do have
attachments you want to send with your job request you need to set the content type to multipart/form-data
and
post the request that way. Examples are below.
6.1.2. Endpoint
POST /api/v3/jobs
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
This Example is without attachments |
6.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
Null |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
Null |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
Null |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
Must not be null. Size must be between 1 and 10000 inclusive |
false |
clusterCriterias |
Array |
List of cluster criteria’s for which a match will be attempted with register cluster tags |
Must not be empty |
false |
commandCriteria |
Array |
List of tags which will attempt to match against the commands linked to selected cluster |
Must not be empty |
false |
group |
Null |
A group that the job should be run under on the linux system |
Size must be between 0 and 255 inclusive |
true |
disableLogArchival |
Boolean |
If you want to disable backing up job output files set this to true. Default: false |
true |
|
Null |
If you want e-mail notification on job completion enter address here |
Must be a well-formed email address. Size must be between 0 and 255 inclusive |
true |
|
cpu |
Null |
For future use. Currently has no impact. |
Must be at least 1 |
true |
memory |
Null |
The amount of memory (in MB) desired for job client. Cannot exceed configured max. |
Must be at least 1 |
true |
timeout |
Null |
The timeout (in seconds) after which job will be killed by system, system setting used if not set |
Must be at least 1 |
true |
dependencies |
Array |
URI’s of dependency files which will be downloaded into job working directory at runtime |
true |
|
applications |
Array |
Complete list of application ids if power user wishes to override selected command defaults |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/jobs' -i -X POST -H 'Content-Type: application/json' -d '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"applications" : [ ]
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"applications" : [ ]
}' | http POST 'https://genie.example.com/api/v3/jobs' 'Content-Type:application/json'
HTTP Messages
Request
POST /api/v3/jobs HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 819
{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"applications" : [ ]
}
6.1.4. API Docs
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/jobs' -i -X POST -H 'Content-Type: multipart/form-data' -F 'request={"id":null,"created":null,"updated":null,"tags":[],"version":"1.0","user":"genie","name":"List * ... Directories bash job","description":"Genie 3 Test Job","setupFile":"/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile","commandArgs":"-c 'echo hello world'","clusterCriterias":[{"tags":["localhost"]}],"commandCriteria":["bash"],"group":null,"disableLogArchival":true,"email":null,"cpu":null,"memory":null,"timeout":null,"dependencies":[],"applications":[]};type=application/json' -F 'attachment=@query.sql;type=application/octet-stream' -F 'attachment=@query2.sql;type=application/octet-stream'
HTTPie
$ http --form POST 'https://genie.example.com/api/v3/jobs' \
'request'@<(echo '{"id":null,"created":null,"updated":null,"tags":[],"version":"1.0","user":"genie","name":"List * ... Directories bash job","description":"Genie 3 Test Job","setupFile":"/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile","commandArgs":"-c 'echo hello world'","clusterCriterias":[{"tags":["localhost"]}],"commandCriteria":["bash"],"group":null,"disableLogArchival":true,"email":null,"cpu":null,"memory":null,"timeout":null,"dependencies":[],"applications":[]}') \
'attachment'@'query.sql' \
'attachment'@'query2.sql'
HTTP Messages
Request
POST /api/v3/jobs HTTP/1.1
Content-Type: multipart/form-data; boundary=6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Host: genie.example.com
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=request
Content-Type: application/json
{"id":null,"created":null,"updated":null,"tags":[],"version":"1.0","user":"genie","name":"List * ... Directories bash job","description":"Genie 3 Test Job","setupFile":"/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile","commandArgs":"-c 'echo hello world'","clusterCriterias":[{"tags":["localhost"]}],"commandCriteria":["bash"],"group":null,"disableLogArchival":true,"email":null,"cpu":null,"memory":null,"timeout":null,"dependencies":[],"applications":[]}
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=attachment
Content-Type: application/octet-stream
select * from myTable;
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm
Content-Disposition: form-data; name=attachment
Content-Type: application/octet-stream
SELECT * FROM your_table JOIN my_table ON your_table.id = my_table.id WHERE x > 50;
--6o2knFse3p53ty9dmcQvWAIx1zInP11uCfbm--
6.2. Find Jobs
6.2.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The id of the jobs to find. Use % symbol for regex like search. |
|
The name of the jobs to find. Use % symbol for regex like search. |
|
The user of the jobs to find. |
|
The status(es) of the jobs to find. Can have multiple. Options: [INIT, RUNNING, SUCCEEDED, KILLED, FAILED, INVALID] |
|
The tag(s) of the jobs to find. Can have multiple. |
|
The name of the cluster on which the jobs ran. |
|
The id of the cluster on which the jobs ran. |
|
The name of the command which was executed by the job. |
|
The id of the command which was executed by the job. |
|
The minimum started time of the job in milliseconds since epoch. |
|
The maximum started time of the job in milliseconds since epoch. |
|
The minimum finished time of the job in milliseconds since epoch. |
|
The maximum finished time of the job in milliseconds since epoch. |
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.jobSearchResultList |
Array |
The found jobs. |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 784
{
"_embedded" : {
"jobSearchResultList" : [ {
"id" : "425c4a6a-a069-4849-a66e-d08d6e8d1912",
"name" : "List * ... Directories bash job",
"user" : "genie",
"status" : "SUCCEEDED",
"started" : "2017-01-12T18:43:42.566Z",
"finished" : "2017-01-12T18:43:42.597Z",
"clusterName" : "Local laptop",
"commandName" : "Unix Bash command",
"runtime" : "PT0.031S",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs?user=genie"
}
},
"page" : {
"size" : 10,
"totalElements" : 1,
"totalPages" : 1,
"number" : 0
}
}
6.3. Get a Job
6.3.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
Size must be between 1 and 10000 inclusive |
false |
status |
String |
The status of the job. Options: [INIT, RUNNING, SUCCEEDED, KILLED, FAILED, INVALID] |
Must not be null |
false |
statusMsg |
String |
The status message of the job |
Size must be between 0 and 255 inclusive |
true |
started |
String |
The time (UTC ISO8601 with millis) the job was started |
true |
|
finished |
String |
The time (UTC ISO8601 with millis) the job finished |
true |
|
archiveLocation |
Null |
The URI where the working directory zip was stored |
Size must be between 0 and 1024 inclusive |
true |
clusterName |
String |
The name of the cluster the job was run on if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
commandName |
String |
The name of the command the job was run with if it’s been determined |
Size must be between 0 and 255 inclusive |
true |
runtime |
String |
Runtime of the job in ISO8601 duration format |
Must not be null |
true |
HATEOAS Links
Relation | Description |
---|---|
|
URI for this job |
|
The request that kicked off this job |
|
The job execution information for this job |
|
The output URI for the job |
|
The current status of the job |
|
The cluster this job ran on |
|
The command this job executed |
|
The applications this job used |
HTTP Messages
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 1621
{
"id" : "425c4a6a-a069-4849-a66e-d08d6e8d1912",
"created" : "2017-01-12T18:43:42.461Z",
"updated" : "2017-01-12T18:43:42.601Z",
"tags" : [ ],
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"status" : "SUCCEEDED",
"statusMsg" : "Job finished successfully.",
"started" : "2017-01-12T18:43:42.566Z",
"finished" : "2017-01-12T18:43:42.597Z",
"archiveLocation" : null,
"clusterName" : "Local laptop",
"commandName" : "Unix Bash command",
"runtime" : "PT0.031S",
"commandArgs" : "-c 'echo hello world'",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output"
},
"request" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/request"
},
"execution" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/execution"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/status"
},
"cluster" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/cluster"
},
"command" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/command"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/applications"
}
}
}
6.4. Kill a Job
6.5. Get Job Status
6.6. Get Contents From the Job Working Directory
6.6.1. Description
This API acts as a file browser for the remote working directory of your job. You can request endpoints and if it is a directory you will be shown the contents of the directory and be able to navigate. If the endpoint points to a file it will return the file.
6.6.3. Getting a directory as JSON
This is the default behavior for directories if no Accept header is present
|
6.6.4. API Docs
Request Documentation
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
parent |
Null |
Information about the parent of this directory |
true |
|
parent.name |
String |
The name of the parent directory |
true |
|
parent.url |
String |
The url to get the parent |
true |
|
parent.size |
Number |
The size of the parent in bytes |
true |
|
parent.lastModified |
String |
The last time the parent was modified in ISO8601 UTC with milliseconds included |
true |
|
directories |
Array |
All the subdirectories of this directory |
false |
|
directories[].name |
String |
The name of the directory |
false |
|
directories[].url |
String |
The url to get the directory |
false |
|
directories[].size |
Number |
The size of the directory in bytes |
false |
|
directories[].lastModified |
String |
The last time the directory was modified in ISO8601 UTC with milliseconds included |
false |
|
files |
Array |
All the files in this directory |
false |
|
files[].name |
String |
The name of the file |
false |
|
files[].url |
String |
The url to get the file |
false |
|
files[].size |
Number |
The size of the file in bytes |
false |
|
files[].lastModified |
String |
The last time the file was modified in ISO8601 UTC with milliseconds included |
false |
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/ HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 1253
{
"parent" : null,
"directories" : [ {
"name" : "genie/",
"url" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/genie/",
"size" : 0,
"lastModified" : "2017-01-12T18:43:42.000Z"
} ],
"files" : [ {
"name" : "dep1",
"url" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/dep1",
"size" : 0,
"lastModified" : "2017-01-12T18:43:42.000Z"
}, {
"name" : "jobsetupfile",
"url" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/jobsetupfile",
"size" : 0,
"lastModified" : "2017-01-12T18:43:42.000Z"
}, {
"name" : "run",
"url" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/run",
"size" : 3346,
"lastModified" : "2017-01-12T18:43:42.000Z"
}, {
"name" : "stderr",
"url" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/stderr",
"size" : 0,
"lastModified" : "2017-01-12T18:43:42.000Z"
}, {
"name" : "stdout",
"url" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/stdout",
"size" : 12,
"lastModified" : "2017-01-12T18:43:42.000Z"
} ]
}
6.6.5. Getting a directory as HTML
This would happen automatically if you called from browser due to browser default accept header |
6.6.6. API Docs
Request Documentation
Request Examples
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/ HTTP/1.1
Accept: text/html
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 2816
<!DOCTYPE html><html><head><title>425c4a6a-a069-4849-a66e-d08d6e8d1912</title><style type="text/css"><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}.line {height: 1px; background-color: #525D76; border: none;}--></style> </head><body><h1>425c4a6a-a069-4849-a66e-d08d6e8d1912</h1><HR size="1" noshade="noshade"><table width="100%" cellspacing="0" cellpadding="5" align="center"><tr><td align="left"><font size="+1"><strong>Filename</strong></font></td><td align="right"><font size="+1"><strong>Size</strong></font></td><td align="right"><font size="+1"><strong>Last Modified</strong></font></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/genie/"><tt>genie/</tt></a></td><td align="right"><tt>-</tt></td><td align="right"><tt>Thu, 12 Jan 2017 18:43:42 GMT</tt></td></tr><tr><td align="left"> <a href="https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/dep1"><tt>dep1</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Thu, 12 Jan 2017 18:43:42 GMT</tt></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/jobsetupfile"><tt>jobsetupfile</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Thu, 12 Jan 2017 18:43:42 GMT</tt></td></tr><tr><td align="left"> <a href="https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/run"><tt>run</tt></a></td><td align="right"><tt>3 KB</tt></td><td align="right"><tt>Thu, 12 Jan 2017 18:43:42 GMT</tt></td></tr><tr bgcolor="#eeeeee"><td align="left"> <a href="https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/stderr"><tt>stderr</tt></a></td><td align="right"><tt>0 bytes</tt></td><td align="right"><tt>Thu, 12 Jan 2017 18:43:42 GMT</tt></td></tr><tr><td align="left"> <a href="https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/stdout"><tt>stdout</tt></a></td><td align="right"><tt>12 bytes</tt></td><td align="right"><tt>Thu, 12 Jan 2017 18:43:42 GMT</tt></td></tr></table><HR size="1" noshade="noshade"><h3>Apache Tomcat/8.0.37</h3></body></html>
6.6.8. API Docs
Request Documentation
Request Examples
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output/run HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Last-Modified: Thu, 12 Jan 2017 18:43:42 GMT
Content-Length: 3346
Accept-Ranges: bytes
#!/usr/bin/env bash
set -o nounset -o pipefail
# Set function in case any of the exports or source commands cause an error
trap "handle_failure" ERR EXIT
function handle_failure {
ERROR_CODE=$?
# Good exit
if [[ ${ERROR_CODE} -eq 0 ]]; then
exit 0
fi
# Bad exit
printf '{"exitCode": "%s"}\n' "${ERROR_CODE}" > ./genie/genie.done
exit "${ERROR_CODE}"
}
# Set function for handling kill signal from the job kill service
trap "handle_kill_request" SIGTERM
function handle_kill_request {
KILL_EXIT_CODE=999
# Disable SIGTERM signal for the script itself
trap "" SIGTERM
echo "Kill signal received"
### Write the kill exit code to genie.done file as exit code before doing anything else
echo "Generate done file with exit code ${KILL_EXIT_CODE}"
printf '{"exitCode": "%s"}\n' "${KILL_EXIT_CODE}" > ./genie/genie.done
### Send a kill signal the entire process group
echo "Sending a kill signal to the process group"
pkill -g $$
COUNTER=0
NUM_CHILD_PROCESSES=`pgrep -g ${SELF_PID} | wc -w`
# Waiting for 30 seconds for the child processes to die
while [[ $COUNTER -lt 30 ]] && [[ "$NUM_CHILD_PROCESSES" -gt 3 ]]; do
echo The counter is $COUNTER
let COUNTER=COUNTER+1
echo "Sleeping now for 1 seconds"
sleep 1
NUM_CHILD_PROCESSES=`pgrep -g ${SELF_PID} | wc -w`
done
# check if any children are still running. If not just exit.
if [ "$NUM_CHILD_PROCESSES" -eq 3 ]
then
echo "Done"
exit
fi
### Reaching at this point means the children did not die. If so send kill -9 to the entire process group
# this is a hard kill and will this process itself as well
echo "Sending a kill -9 to children"
pkill -9 -g $$
echo "Done"
}
SELF_PID=$$
echo Start: `date '+%Y-%m-%d %H:%M:%S'`
export GENIE_JOB_DIR="/tmp/1484246599678-0/425c4a6a-a069-4849-a66e-d08d6e8d1912"
export GENIE_APPLICATION_DIR="${GENIE_JOB_DIR}/genie/applications"
export GENIE_COMMAND_DIR="${GENIE_JOB_DIR}/genie/command/cmd1"
export GENIE_COMMAND_ID="cmd1"
export GENIE_COMMAND_NAME="Unix Bash command"
export GENIE_CLUSTER_DIR="${GENIE_JOB_DIR}/genie/cluster/cluster1"
export GENIE_CLUSTER_ID="cluster1"
export GENIE_CLUSTER_NAME="Local laptop"
export GENIE_JOB_ID="425c4a6a-a069-4849-a66e-d08d6e8d1912"
export GENIE_JOB_NAME="List * ... Directories bash job"
export GENIE_JOB_MEMORY=1024
export GENIE_VERSION=3
# Sourcing setup file from Cluster: cluster1
source ${GENIE_JOB_DIR}/genie/cluster/cluster1/setupfile
# Sourcing setup file from Application: app1
source ${GENIE_JOB_DIR}/genie/applications/app1/setupfile
# Sourcing setup file from Application: app2
source ${GENIE_JOB_DIR}/genie/applications/app2/setupfile
# Sourcing setup file from Command: cmd1
source ${GENIE_JOB_DIR}/genie/command/cmd1/setupfile
# Sourcing setup file specified in job request
source ${GENIE_JOB_DIR}/jobsetupfile
# Dump the environment to a env.log file
env | sort > ${GENIE_JOB_DIR}/genie/logs/env.log
# Kick off the command in background mode and wait for it using its pid
/bin/bash -c 'echo hello world' > stdout 2> stderr &
wait $!
# Write the return code from the command in the done file.
printf '{"exitCode": "%s"}\n' "$?" > ./genie/genie.done
echo End: `date '+%Y-%m-%d %H:%M:%S'`
6.7. Get a Job Request
6.7.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
commandArgs |
String |
Any arguments to append to the command executable when the job is run |
Must not be null. Size must be between 1 and 10000 inclusive |
false |
clusterCriterias |
Array |
List of cluster criteria’s for which a match will be attempted with register cluster tags |
Must not be empty |
false |
commandCriteria |
Array |
List of tags which will attempt to match against the commands linked to selected cluster |
Must not be empty |
false |
group |
Null |
A group that the job should be run under on the linux system |
Size must be between 0 and 255 inclusive |
true |
disableLogArchival |
Boolean |
If you want to disable backing up job output files set this to true. Default: false |
true |
|
Null |
If you want e-mail notification on job completion enter address here |
Must be a well-formed email address. Size must be between 0 and 255 inclusive |
true |
|
cpu |
Null |
For future use. Currently has no impact. |
Must be at least 1 |
true |
memory |
Null |
The amount of memory (in MB) desired for job client. Cannot exceed configured max. |
Must be at least 1 |
true |
timeout |
Null |
The timeout (in seconds) after which job will be killed by system, system setting used if not set |
Must be at least 1 |
true |
dependencies |
Array |
URI’s of dependency files which will be downloaded into job working directory at runtime |
true |
|
applications |
Array |
Complete list of application ids if power user wishes to override selected command defaults |
true |
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/request HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 1526
{
"id" : "425c4a6a-a069-4849-a66e-d08d6e8d1912",
"created" : "2017-01-12T18:43:42.460Z",
"updated" : "2017-01-12T18:43:42.460Z",
"tags" : [ ],
"version" : "1.0",
"user" : "genie",
"name" : "List * ... Directories bash job",
"description" : "Genie 3 Test Job",
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/jobsetupfile",
"commandArgs" : "-c 'echo hello world'",
"clusterCriterias" : [ {
"tags" : [ "localhost" ]
} ],
"commandCriteria" : [ "bash" ],
"group" : null,
"disableLogArchival" : true,
"email" : null,
"cpu" : null,
"memory" : null,
"timeout" : null,
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/job/dep1" ],
"applications" : [ ],
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/request"
},
"job" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912"
},
"execution" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/execution"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/status"
}
}
}
6.8. Get a Job Execution
6.8.1. Description
Get metadata about a job execution. Information like where a job was run, its process exit code, etc.
6.8.3. API Docs
Request Examples
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
hostName |
String |
The host name of the Genie node responsible for the job |
false |
|
processId |
Number |
The id of the job client process on the Genie node |
true |
|
checkDelay |
Number |
The amount of time in milliseconds between checks of the job status by Genie |
true |
|
timeout |
String |
The date (UTC ISO8601 with millis) when the job will be killed by Genie due to timeout |
true |
|
exitCode |
Number |
The job client process exit code after the job is done |
true |
|
memory |
Number |
The amount of memory (in MB) allocated to the job client |
true |
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/execution HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 950
{
"id" : "425c4a6a-a069-4849-a66e-d08d6e8d1912",
"created" : "2017-01-12T18:43:42.461Z",
"updated" : "2017-01-12T18:43:42.601Z",
"hostName" : "testing-gce-ba31b5ee-ead7-45d2-bc60-c5cc18ef6037",
"processId" : 6403,
"checkDelay" : 1,
"timeout" : "2017-01-19T18:43:42.559Z",
"exitCode" : 0,
"memory" : 1024,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/execution"
},
"job" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912"
},
"request" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/request"
},
"output" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/output"
},
"status" : {
"href" : "https://genie.example.com/api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/status"
}
}
}
6.9. Get the Cluster a Job Ran On
6.9.1. Description
Get the metadata about the cluster which was selected to run the job based on the cluster and command criteria in the job request.
6.9.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/cluster HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 998
{
"id" : "cluster1",
"created" : "2017-01-12T18:43:42.359Z",
"updated" : "2017-01-12T18:43:42.427Z",
"tags" : [ "localhost", "genie.id:cluster1", "genie.name:Local laptop" ],
"version" : "1.0",
"user" : "genie",
"name" : "Local laptop",
"description" : null,
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/setupfile",
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cluster1/config2" ],
"status" : "UP",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters/cluster1"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/clusters/cluster1/commands"
}
}
}
6.10. Get the Command a Job Executed
6.10.1. Description
Get the metadata about the command which was selected to be executed by the job based on the cluster and command criteria in the job request.
6.10.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Null |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/command HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 1155
{
"id" : "cmd1",
"created" : "2017-01-12T18:43:42.387Z",
"updated" : "2017-01-12T18:43:42.409Z",
"tags" : [ "genie.name:Unix Bash command", "bash", "genie.id:cmd1" ],
"version" : "1.0",
"user" : "genie",
"name" : "Unix Bash command",
"description" : null,
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/setupfile",
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/cmd1/config2" ],
"status" : "ACTIVE",
"executable" : "/bin/bash",
"checkDelay" : 1,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/cmd1"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/cmd1/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/cmd1/clusters"
}
}
}
6.11. Get the Applications a Job Command Used
6.11.1. Description
Get the list of applications that were used by the command selected by job to execute.
6.11.3. API Docs
Request Examples
Response Documentation
HTTP Messages
Request
GET /api/v3/jobs/425c4a6a-a069-4849-a66e-d08d6e8d1912/applications HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 2614
[ {
"id" : "app1",
"created" : "2017-01-12T18:43:42.317Z",
"updated" : "2017-01-12T18:43:42.317Z",
"tags" : [ "genie.id:app1", "genie.name:Application 1" ],
"version" : "1.0",
"user" : "genie",
"name" : "Application 1",
"description" : null,
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/setupfile",
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/config2" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/dep1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app1/dep2" ],
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/app1"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/app1/commands"
} ]
}, {
"id" : "app2",
"created" : "2017-01-12T18:43:42.338Z",
"updated" : "2017-01-12T18:43:42.338Z",
"tags" : [ "genie.id:app2", "genie.name:Application 2" ],
"version" : "1.0",
"user" : "genie",
"name" : "Application 2",
"description" : null,
"setupFile" : "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/setupfile",
"configs" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/config1", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/config2" ],
"dependencies" : [ "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/dep2", "/home/travis/build/Netflix/genie/genie-web/build/resources/test/com/netflix/genie/web/controllers/JobRestControllerIntegrationTests/app2/dep1" ],
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/app2"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/app2/commands"
} ]
} ]
7. Applications API
7.1. Create an Application
7.1.2. Endpoint
POST /api/v3/applications
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
7.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
Null |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
Null |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
Null |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/applications' -i -X POST -H 'Content-Type: application/json' -d '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ "type:spark", "ver:1.5.1" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"status" : "ACTIVE",
"type" : "spark"
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ "type:spark", "ver:1.5.1" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"status" : "ACTIVE",
"type" : "spark"
}' | http POST 'https://genie.example.com/api/v3/applications' 'Content-Type:application/json'
HTTP Messages
Request
POST /api/v3/applications HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 427
{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ "type:spark", "ver:1.5.1" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"status" : "ACTIVE",
"type" : "spark"
}
7.2. Get an Application
7.2.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
String |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
String |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
String |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
HTTP Messages
Request
GET /api/v3/applications/07d38b08-91f1-47e9-8629-29e42a6e2b8a HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 850
{
"id" : "07d38b08-91f1-47e9-8629-29e42a6e2b8a",
"created" : "2017-01-12T18:43:57.857Z",
"updated" : "2017-01-12T18:43:57.857Z",
"tags" : [ "type:spark", "genie.id:07d38b08-91f1-47e9-8629-29e42a6e2b8a", "ver:1.5.1", "genie.name:spark" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : "Spark for Genie",
"setupFile" : "s3://mybucket/spark/1.5.1/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/1.5.1/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/1.5.1/spark.tar.gz" ],
"status" : "ACTIVE",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/07d38b08-91f1-47e9-8629-29e42a6e2b8a"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/07d38b08-91f1-47e9-8629-29e42a6e2b8a/commands"
}
}
}
7.3. Find Applications
7.3.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The name of the applications to find. |
|
The user of the applications to find. |
|
The status(es) of the applications to find. Can have multiple. Options: [ACTIVE, DEPRECATED, INACTIVE] |
|
The tag(s) of the applications to find. Can have multiple. |
|
The type of the applications to find. |
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.applicationList |
Array |
The found applications. |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 5951
{
"_embedded" : {
"applicationList" : [ {
"id" : "dc7cb100-a471-4edc-8d9e-285b35c8b86a",
"created" : "2017-01-12T18:43:57.343Z",
"updated" : "2017-01-12T18:43:57.343Z",
"tags" : [ "genie.id:dc7cb100-a471-4edc-8d9e-285b35c8b86a", "genie.name:hive", "type:hive", "ver:1.0.0" ],
"version" : "1.0.0",
"user" : "genieUser7",
"name" : "hive",
"description" : "Hive 1.0.0 for Genie",
"setupFile" : "s3://mybucket/hive/setup-hive.sh",
"configs" : [ "s3://mybucket/hive/hive-env.sh", "s3://mybucket/hive/hive-log4j.properties" ],
"dependencies" : [ "s3://mybucket/hive/hive-1.0.0.tar.gz" ],
"status" : "ACTIVE",
"type" : "hive",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/dc7cb100-a471-4edc-8d9e-285b35c8b86a"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/dc7cb100-a471-4edc-8d9e-285b35c8b86a/commands"
}
}
}, {
"id" : "d0ff6ca7-f721-46bc-ba14-c3e43a4c8f37",
"created" : "2017-01-12T18:43:57.328Z",
"updated" : "2017-01-12T18:43:57.328Z",
"tags" : [ "genie.id:d0ff6ca7-f721-46bc-ba14-c3e43a4c8f37", "type:pig", "ver:0.15.0", "genie.name:spark" ],
"version" : "0.4.0",
"user" : "genieUser6",
"name" : "spark",
"description" : "Pig 0.15.0 for Genie",
"setupFile" : "s3://mybucket/pig/setup-pig.sh",
"configs" : [ "s3://mybucket/pig/pig.properties" ],
"dependencies" : [ "s3://mybucket/pig/pig-0.15.0.tar.gz" ],
"status" : "ACTIVE",
"type" : "pig",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/d0ff6ca7-f721-46bc-ba14-c3e43a4c8f37"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/d0ff6ca7-f721-46bc-ba14-c3e43a4c8f37/commands"
}
}
}, {
"id" : "f2b8ba9b-31d6-45e7-814e-d6440c4630d3",
"created" : "2017-01-12T18:43:57.315Z",
"updated" : "2017-01-12T18:43:57.315Z",
"tags" : [ "genie.id:f2b8ba9b-31d6-45e7-814e-d6440c4630d3", "type:spark", "ver:1.3.1", "genie.name:spark" ],
"version" : "1.3.1",
"user" : "genieUser5",
"name" : "spark",
"description" : "Spark 1.3.1 for Genie",
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.3.1.tar.gz" ],
"status" : "DEPRECATED",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/f2b8ba9b-31d6-45e7-814e-d6440c4630d3"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/f2b8ba9b-31d6-45e7-814e-d6440c4630d3/commands"
}
}
}, {
"id" : "3f94fb99-b35e-48e2-a9b5-5a8d879bf7c5",
"created" : "2017-01-12T18:43:57.300Z",
"updated" : "2017-01-12T18:43:57.300Z",
"tags" : [ "type:spark", "genie.id:3f94fb99-b35e-48e2-a9b5-5a8d879bf7c5", "ver:1.4.0", "genie.name:spark" ],
"version" : "1.4.0",
"user" : "genieUser4",
"name" : "spark",
"description" : "Spark 1.4.0 for Genie",
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.4.0.tar.gz" ],
"status" : "DEPRECATED",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/3f94fb99-b35e-48e2-a9b5-5a8d879bf7c5"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/3f94fb99-b35e-48e2-a9b5-5a8d879bf7c5/commands"
}
}
}, {
"id" : "5ab93b34-2368-452f-b29d-33c376b1cfd9",
"created" : "2017-01-12T18:43:57.272Z",
"updated" : "2017-01-12T18:43:57.272Z",
"tags" : [ "type:spark", "genie.id:5ab93b34-2368-452f-b29d-33c376b1cfd9", "ver:1.5.0", "genie.name:spark" ],
"version" : "1.5.0",
"user" : "genieUser2",
"name" : "spark",
"description" : "Spark 1.5.0 for Genie",
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.5.0.tar.gz" ],
"status" : "ACTIVE",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/5ab93b34-2368-452f-b29d-33c376b1cfd9"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/5ab93b34-2368-452f-b29d-33c376b1cfd9/commands"
}
}
}, {
"id" : "11f93346-2464-42ef-be07-bab6af8d964b",
"created" : "2017-01-12T18:43:57.259Z",
"updated" : "2017-01-12T18:43:57.259Z",
"tags" : [ "type:spark", "ver:1.5.1", "genie.name:spark", "genie.id:11f93346-2464-42ef-be07-bab6af8d964b" ],
"version" : "1.5.1",
"user" : "genieUser1",
"name" : "spark",
"description" : "Spark 1.5.1 for Genie",
"setupFile" : "s3://mybucket/spark/setup-spark.sh",
"configs" : [ "s3://mybucket/spark/spark-env.sh" ],
"dependencies" : [ "s3://mybucket/spark/spark-1.5.1.tar.gz" ],
"status" : "ACTIVE",
"type" : "spark",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications/11f93346-2464-42ef-be07-bab6af8d964b"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/applications/11f93346-2464-42ef-be07-bab6af8d964b/commands"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/applications?status=ACTIVE&status=DEPRECATED"
}
},
"page" : {
"size" : 10,
"totalElements" : 6,
"totalPages" : 1,
"number" : 0
}
}
7.4. Update an Entire Application
7.4.1. Description
Update the metadata about an application. This method does a complete replace of the resource. Usual flow is to call to get the most recent state then update what you want and call this API with the result.
For more fine grained control call the PATCH API. |
7.4.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
type |
Null |
The type of application this is (e.g. hadoop, presto, spark). Can be used to group. |
true |
|
status |
String |
The status of the application. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
dependencies |
Array |
The dependencies for the application |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/applications/cfe21485-80be-4bbf-8437-440aab1ae70c' -i -X PUT -H 'Content-Type: application/json' -d '{
"id" : "cfe21485-80be-4bbf-8437-440aab1ae70c",
"created" : "2017-01-12T18:43:58.330Z",
"updated" : "2017-01-12T18:43:58.330Z",
"tags" : [ "genie.id:cfe21485-80be-4bbf-8437-440aab1ae70c", "genie.name:spark" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"dependencies" : [ ],
"status" : "INACTIVE",
"type" : null
}'
HTTPie
$ echo '{
"id" : "cfe21485-80be-4bbf-8437-440aab1ae70c",
"created" : "2017-01-12T18:43:58.330Z",
"updated" : "2017-01-12T18:43:58.330Z",
"tags" : [ "genie.id:cfe21485-80be-4bbf-8437-440aab1ae70c", "genie.name:spark" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"dependencies" : [ ],
"status" : "INACTIVE",
"type" : null
}' | http PUT 'https://genie.example.com/api/v3/applications/cfe21485-80be-4bbf-8437-440aab1ae70c' 'Content-Type:application/json'
HTTP Messages
Request
PUT /api/v3/applications/cfe21485-80be-4bbf-8437-440aab1ae70c HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 413
{
"id" : "cfe21485-80be-4bbf-8437-440aab1ae70c",
"created" : "2017-01-12T18:43:58.330Z",
"updated" : "2017-01-12T18:43:58.330Z",
"tags" : [ "genie.id:cfe21485-80be-4bbf-8437-440aab1ae70c", "genie.name:spark" ],
"version" : "1.5.1",
"user" : "genie",
"name" : "spark",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"dependencies" : [ ],
"status" : "INACTIVE",
"type" : null
}
7.5. Update Parts of an Application
7.5.1. Description
Update the metadata about an application using JSON Patch. This will ONLY update the fields you request be changed.
For more information about JSON Patch RFC see the website. |
7.5.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
[] |
Array |
Array of patches to apply |
false |
|
[].op |
String |
Patch operation to perform. |
add, remove, replace, copy, move, test |
false |
[].path |
String |
The json path to operate on. e.g. /user |
false |
|
[].from |
String |
The json path to move or copy from. e.g. /user |
true |
|
[].value |
String |
The json value to put at the path for an add, replace or test |
true |
7.7. Delete an Application
7.8. Get Configs for Application
7.8.3. API Docs
Request Documentation
Request Examples
7.9. Add Configs to Application
7.9.1. Description
Add configuration file locations to an existing application. Configurations are stored as a set so any duplicates will be ignored.
7.9.3. API Docs
Request Documentation
7.10. Update Configs for an Application
7.10.1. Description
Update the configuration file locations for an existing application. Stored as a set so duplicates are ignored.
7.10.3. API Docs
Request Documentation
7.11. Remove All Configs From an Application
7.12. Get Dependencies For an Application
7.12.3. API Docs
Request Documentation
Request Examples
7.13. Add Dependencies to Application
7.13.1. Description
Add dependency file locations to an existing application. Stored as a set so all duplicates are ignored.
7.13.3. API Docs
Request Documentation
7.14. Update Dependencies for an Application
7.14.3. API Docs
Request Documentation
7.15. Remove All Dependencies From an Application
7.16. Get Tags For an Application
7.16.3. API Docs
Request Examples
7.17. Add Tags to Application
7.17.1. Description
Add tags to an existing application. Stored as a set so all duplicates are ignored.
genie.id:{id} and genie.name:{name} tags are automatically added by the service.
|
7.18. Update Tags for an Application
7.18.1. Description
Update the tags for an existing application.
The genie.id:{id} and genie.name:{name} tags can’t be removed. They will automatically be added back
by the system.
|
7.19. Remove All Tags From Application
7.19.1. Description
Remove all the tags for an existing application
The genie.id:{id} and genie.name:{name} tags will NOT be removed by this operation
|
7.20. Remove Tag From Application
7.20.1. Description
Remove a tag from an existing application.
You can’t remove the genie.id:{id} and genie.name:{name} tags. They will just be added back by the system
|
7.20.3. API Docs
Request Documentation
7.21. Get the Commands that Use an Application
7.21.3. API Docs
Request Documentation
Request Examples
HTTP Messages
Request
GET /api/v3/applications/cfe21485-80be-4bbf-8437-440aab1ae70c/commands?status=ACTIVE&status=INACTIVE HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 1010
[ {
"id" : "149ba6bd-feac-46d5-9970-dfae3cb0d1c7",
"created" : "2017-01-12T18:43:58.496Z",
"updated" : "2017-01-12T18:43:58.533Z",
"tags" : [ "genie.name:16329dfc-a7e3-4640-bc34-a1ee6249198f", "genie.id:149ba6bd-feac-46d5-9970-dfae3cb0d1c7" ],
"version" : "16329dfc-a7e3-4640-bc34-a1ee6249198f",
"user" : "16329dfc-a7e3-4640-bc34-a1ee6249198f",
"name" : "16329dfc-a7e3-4640-bc34-a1ee6249198f",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "ACTIVE",
"executable" : "16329dfc-a7e3-4640-bc34-a1ee6249198f",
"checkDelay" : 1000,
"memory" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/commands/149ba6bd-feac-46d5-9970-dfae3cb0d1c7"
}, {
"rel" : "applications",
"href" : "https://genie.example.com/api/v3/commands/149ba6bd-feac-46d5-9970-dfae3cb0d1c7/applications"
}, {
"rel" : "clusters",
"href" : "https://genie.example.com/api/v3/commands/149ba6bd-feac-46d5-9970-dfae3cb0d1c7/clusters"
} ]
} ]
8. Clusters API
8.1. Create a Cluster
8.1.2. Endpoint
POST /api/v3/clusters
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
8.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
Null |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
Null |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
Null |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/clusters' -i -X POST -H 'Content-Type: application/json' -d '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "UP"
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "UP"
}' | http POST 'https://genie.example.com/api/v3/clusters' 'Content-Type:application/json'
HTTP Messages
Request
POST /api/v3/clusters HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 221
{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "UP"
}
8.2. Get a Cluster
8.2.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 636
{
"id" : "35c9df87-3883-48b1-a3f4-92a7f5d7bdfe",
"created" : "2017-01-12T18:43:28.441Z",
"updated" : "2017-01-12T18:43:28.441Z",
"tags" : [ "genie.id:35c9df87-3883-48b1-a3f4-92a7f5d7bdfe", "genie.name:h2prod" ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "UP",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters/35c9df87-3883-48b1-a3f4-92a7f5d7bdfe"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/clusters/35c9df87-3883-48b1-a3f4-92a7f5d7bdfe/commands"
}
}
}
8.3. Find Clusters
8.3.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The name of the clusters to find. Use % to perform a regex like query |
|
The status(es) of the clusters to find. Can have multiple. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
|
The tag(s) of the clusters to find. Can have multiple. |
|
The minimum time (in milliseconds from epoch UTC) that the cluster(s) were updated at. |
|
The maximum time (in milliseconds from epoch UTC) that the cluster(s) were updated at. |
Request Examples
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.clusterList |
Array |
The found clusters. |
false |
HTTP Messages
Request
GET /api/v3/clusters?tag=genie.id%3Adaa1cb5d-a0f9-4e95-a95c-60192eec85c1 HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 1137
{
"_embedded" : {
"clusterList" : [ {
"id" : "daa1cb5d-a0f9-4e95-a95c-60192eec85c1",
"created" : "2017-01-12T18:43:34.778Z",
"updated" : "2017-01-12T18:43:34.778Z",
"tags" : [ "genie.name:49199e1d-61ac-451b-b00c-0c678af58be0", "genie.id:daa1cb5d-a0f9-4e95-a95c-60192eec85c1" ],
"version" : "fbb1308a-c698-491b-accf-dceb6fe20809",
"user" : "dd5c83d6-8d36-471b-9887-5f7db89bab84",
"name" : "49199e1d-61ac-451b-b00c-0c678af58be0",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "UP",
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters/daa1cb5d-a0f9-4e95-a95c-60192eec85c1"
},
"commands" : {
"href" : "https://genie.example.com/api/v3/clusters/daa1cb5d-a0f9-4e95-a95c-60192eec85c1/commands"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/clusters?tag=genie.id:daa1cb5d-a0f9-4e95-a95c-60192eec85c1"
}
},
"page" : {
"size" : 64,
"totalElements" : 1,
"totalPages" : 1,
"number" : 0
}
}
8.4. Update an Entire Cluster
8.4.1. Description
Update the metadata about a cluster. This method does a complete replace of the resource. Usual flow is to call to get the most recent state then update what you want and call this API with the result.
For more fine grained control call the PATCH API. |
8.4.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the cluster. Options: [UP, OUT_OF_SERVICE, TERMINATED] |
Must not be null |
false |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/clusters/daf937ea-503a-4d1a-95e6-2a79ddab9298' -i -X PUT -H 'Content-Type: application/json' -d '{
"id" : "daf937ea-503a-4d1a-95e6-2a79ddab9298",
"created" : "2017-01-12T18:43:37.213Z",
"updated" : "2017-01-12T18:43:37.213Z",
"tags" : [ "genie.name:h2prod", "genie.id:daf937ea-503a-4d1a-95e6-2a79ddab9298" ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "OUT_OF_SERVICE"
}'
HTTPie
$ echo '{
"id" : "daf937ea-503a-4d1a-95e6-2a79ddab9298",
"created" : "2017-01-12T18:43:37.213Z",
"updated" : "2017-01-12T18:43:37.213Z",
"tags" : [ "genie.name:h2prod", "genie.id:daf937ea-503a-4d1a-95e6-2a79ddab9298" ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "OUT_OF_SERVICE"
}' | http PUT 'https://genie.example.com/api/v3/clusters/daf937ea-503a-4d1a-95e6-2a79ddab9298' 'Content-Type:application/json'
HTTP Messages
Request
PUT /api/v3/clusters/daf937ea-503a-4d1a-95e6-2a79ddab9298 HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 380
{
"id" : "daf937ea-503a-4d1a-95e6-2a79ddab9298",
"created" : "2017-01-12T18:43:37.213Z",
"updated" : "2017-01-12T18:43:37.213Z",
"tags" : [ "genie.name:h2prod", "genie.id:daf937ea-503a-4d1a-95e6-2a79ddab9298" ],
"version" : "2.7.1",
"user" : "genie",
"name" : "h2prod",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "OUT_OF_SERVICE"
}
8.5. Update Parts of a Cluster
8.5.1. Description
Update the metadata about a cluster using JSON Patch. This will ONLY update the fields you request be changed.
For more information about JSON Patch RFC see the website. |
8.5.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
[] |
Array |
Array of patches to apply |
false |
|
[].op |
String |
Patch operation to perform. |
add, remove, replace, copy, move, test |
false |
[].path |
String |
The json path to operate on. e.g. /user |
false |
|
[].from |
String |
The json path to move or copy from. e.g. /user |
true |
|
[].value |
String |
The json value to put at the path for an add, replace or test |
true |
8.7. Delete a Cluster
8.8. Get Configs for Cluster
8.9. Add Configs to Cluster
8.9.1. Description
Add configuration file locations to an existing cluster.
Configurations are stored as a set so any duplicates will be ignored. |
8.10. Update Configs for a Cluster
8.10.1. Description
Update the configuration file locations for an existing cluster. Stored as a set so duplicates are ignored.
8.11. Remove All Configs From a Cluster
8.12. Get Tags For a Cluster
8.13. Add Tags to Cluster
8.13.1. Description
Add tags to an existing cluster. Stored as a set so all duplicates are ignored.
genie.id:{id} and genie.name:{name} tags are automatically added by the service.
|
8.14. Update Tags for a Cluster
8.14.1. Description
Update the tags for an existing cluster.
The genie.id:{id} and genie.name:{name} tags can’t be removed. They will automatically be added back
by the system.
|
8.15. Remove All Tags From Cluster
8.15.1. Description
Remove all the tags for an existing cluster
The genie.id:{id} and genie.name:{name} tags will NOT be removed by this operation
|
8.16. Remove Tag From Cluster
8.16.1. Description
Remove a tag from an existing cluster.
You can’t remove the genie.id:{id} and genie.name:{name} tags. They will just be added back by the system
|
8.16.3. API Docs
Request Documentation
8.17. Add Commands to a Cluster
8.17.1. Description
Append new commands (as array of their ID’s) to the existing list of commands the cluster can run.
The order of the commands is important. This represents priority order if two commands with same tags match for the given cluster the one earlier in the list will be selected. |
The commands for the ID’s must already exist in the system |
8.18. Get the Commands for a Cluster
8.18.1. Description
Get the commands currently linked to the cluster. This indicates the command is able to be run on that cluster.
The order of the commands is important. This represents priority order if two commands with same tags match for the given cluster the one earlier in the list will be selected. |
8.18.3. API Docs
Request Examples
HTTP Messages
Request
GET /api/v3/clusters/daf937ea-503a-4d1a-95e6-2a79ddab9298/commands?status=INACTIVE HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 1012
[ {
"id" : "fdf72b2c-4d65-4b7b-a905-f24f422e1c6c",
"created" : "2017-01-12T18:43:29.428Z",
"updated" : "2017-01-12T18:43:29.428Z",
"tags" : [ "genie.id:fdf72b2c-4d65-4b7b-a905-f24f422e1c6c", "genie.name:1723af46-be56-4345-b933-4760d56051f9" ],
"version" : "1723af46-be56-4345-b933-4760d56051f9",
"user" : "1723af46-be56-4345-b933-4760d56051f9",
"name" : "1723af46-be56-4345-b933-4760d56051f9",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "INACTIVE",
"executable" : "1723af46-be56-4345-b933-4760d56051f9",
"checkDelay" : 1000,
"memory" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/commands/fdf72b2c-4d65-4b7b-a905-f24f422e1c6c"
}, {
"rel" : "applications",
"href" : "https://genie.example.com/api/v3/commands/fdf72b2c-4d65-4b7b-a905-f24f422e1c6c/applications"
}, {
"rel" : "clusters",
"href" : "https://genie.example.com/api/v3/commands/fdf72b2c-4d65-4b7b-a905-f24f422e1c6c/clusters"
} ]
} ]
8.19. Set Commands for a Cluster
8.19.1. Description
Set the commands (as array of their ID’s) that can run on the cluster.
The order of the commands is important. This represents priority order if two commands with same tags match for the given cluster the one earlier in the list will be selected. |
The commands for the ID’s must already exist in the system |
8.20. Remove All Commands From a Cluster
8.20.1. Description
Remove all the commands currently associated with the given cluster.
This does NOT delete the command(s) from the system just unlinks them |
8.21. Remove A Single Command From a Cluster
8.21.1. Description
Remove the identified command from the cluster.
This does NOT delete the command from the system just unlinks it |
8.21.3. API Docs
Request Documentation
9. Commands API
9.1. Create a Command
9.1.2. Endpoint
POST /api/v3/commands
The id in this example is optional. If you include one Genie will use it. If not it will provide one. If you provide one it must be unique. |
9.1.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
Null |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
Null |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
Null |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Null |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/commands' -i -X POST -H 'Content-Type: application/json' -d '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}'
HTTPie
$ echo '{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}' | http POST 'https://genie.example.com/api/v3/commands' 'Content-Type:application/json'
HTTP Messages
Request
POST /api/v3/commands HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 306
{
"id" : null,
"created" : null,
"updated" : null,
"tags" : [ ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}
9.2. Get a Command
9.2.3. API Docs
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Null |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
HTTP Messages
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 858
{
"id" : "34d41ab2-7e6f-4f7d-8d1d-3405ff851583",
"created" : "2017-01-12T18:43:26.077Z",
"updated" : "2017-01-12T18:43:26.077Z",
"tags" : [ "genie.id:34d41ab2-7e6f-4f7d-8d1d-3405ff851583", "genie.name:hive" ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "ACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/34d41ab2-7e6f-4f7d-8d1d-3405ff851583"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/34d41ab2-7e6f-4f7d-8d1d-3405ff851583/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/34d41ab2-7e6f-4f7d-8d1d-3405ff851583/clusters"
}
}
}
9.3. Find Commands
9.3.3. API Docs
Request Documentation
Query Parameters
Parameter | Description |
---|---|
|
The page number to get. Default to 0. |
|
The size of the page to get. Default to 64. |
|
The fields to sort the results by. Defaults to 'updated,desc'. |
|
The name of the commands to find. |
|
The user of the commands to find. |
|
The status(es) of the commands to find. Can have multiple. Options: [ACTIVE, DEPRECATED, INACTIVE] |
|
The tag(s) of the commands to find. Can have multiple. |
Response Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
_links |
Object |
Links to other resources. |
false |
|
page |
Object |
The result page information. |
false |
|
page.size |
Number |
The number of elements in this page result. |
false |
|
page.totalElements |
Number |
The total number of elements this search result could return. |
false |
|
page.totalPages |
Number |
The total number of pages there could be at the current page size. |
false |
|
page.number |
Number |
The current page number. |
false |
|
_embedded.commandList |
Array |
The found commands. |
false |
HTTP Messages
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 2493
{
"_embedded" : {
"commandList" : [ {
"id" : "de5acb64-1c0f-4d9a-9c31-22bf7d26e156",
"created" : "2017-01-12T18:43:24.529Z",
"updated" : "2017-01-12T18:43:24.529Z",
"tags" : [ "genie.name:c9ef0ec9-1405-4ffb-84bf-f0b1420271e8", "genie.id:de5acb64-1c0f-4d9a-9c31-22bf7d26e156" ],
"version" : "6d22809a-c04c-4bef-9fa0-b405abcccb68",
"user" : "55768cac-53cd-4c9e-b845-82d2ddacc8b0",
"name" : "c9ef0ec9-1405-4ffb-84bf-f0b1420271e8",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "INACTIVE",
"executable" : "e17625fa-4d1f-45ae-aa06-03afa8137eb9",
"checkDelay" : 10000,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/de5acb64-1c0f-4d9a-9c31-22bf7d26e156"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/de5acb64-1c0f-4d9a-9c31-22bf7d26e156/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/de5acb64-1c0f-4d9a-9c31-22bf7d26e156/clusters"
}
}
}, {
"id" : "83be84b6-fffa-4b2d-8111-05a9ed295b32",
"created" : "2017-01-12T18:43:22.485Z",
"updated" : "2017-01-12T18:43:22.485Z",
"tags" : [ "genie.name:5d07edeb-0a2a-45f9-8cc2-a9c9ba109558", "genie.id:83be84b6-fffa-4b2d-8111-05a9ed295b32" ],
"version" : "29cbc8eb-6cd1-49e9-bc26-91b4567a6b0d",
"user" : "376aac21-643c-46d2-9bf7-ee680a3a0278",
"name" : "5d07edeb-0a2a-45f9-8cc2-a9c9ba109558",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "ACTIVE",
"executable" : "9edc4d2f-910a-4fb4-86f1-fab8ab65a838",
"checkDelay" : 10000,
"memory" : null,
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands/83be84b6-fffa-4b2d-8111-05a9ed295b32"
},
"applications" : {
"href" : "https://genie.example.com/api/v3/commands/83be84b6-fffa-4b2d-8111-05a9ed295b32/applications"
},
"clusters" : {
"href" : "https://genie.example.com/api/v3/commands/83be84b6-fffa-4b2d-8111-05a9ed295b32/clusters"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://genie.example.com/api/v3/commands?status=ACTIVE&status=INACTIVE"
}
},
"page" : {
"size" : 64,
"totalElements" : 2,
"totalPages" : 1,
"number" : 0
}
}
9.4. Update an Entire Command
9.4.1. Description
Update the metadata about a command. This method does a complete replace of the resource. Usual flow is to call to get the most recent state then update what you want and call this API with the result.
For more fine grained control call the PATCH API. |
9.4.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
id |
String |
The id. If not set the system will set one. |
Size must be between 0 and 255 inclusive |
true |
created |
String |
The UTC time of creation. Set by system. ISO8601 format including milliseconds. |
true |
|
updated |
String |
The UTC time of last update. Set by system. ISO8601 format including milliseconds. |
true |
|
name |
String |
The name |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
user |
String |
The user |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
version |
String |
The version |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
description |
Null |
Any description |
Size must be between 0 and 10000 inclusive |
true |
tags |
Array |
The tags |
true |
|
setupFile |
Null |
A location for any setup that needs to be done when installing |
Size must be between 0 and 1024 inclusive |
true |
configs |
Array |
Any configuration files needed for the resource |
true |
|
status |
String |
The status of the command. Options: [ACTIVE, DEPRECATED, INACTIVE] |
Must not be null |
false |
executable |
String |
The executable to run on the Genie node when this command is used. e.g. /usr/bin/hadoop |
Must not be empty. Size must be between 0 and 255 inclusive |
false |
checkDelay |
Number |
The amount of time (in milliseconds) to delay between checks of the jobs using this command |
Must be at least 1 |
false |
memory |
Null |
The default amount of memory (in MB) that should be allocated for instances of this command client |
Must be at least 1 |
true |
Request Examples
cURL
$ curl 'https://genie.example.com/api/v3/commands/b62fc9a2-b502-47a3-a4d5-3d76ab525d9c' -i -X PUT -H 'Content-Type: application/json' -d '{
"id" : "b62fc9a2-b502-47a3-a4d5-3d76ab525d9c",
"created" : "2017-01-12T18:43:27.255Z",
"updated" : "2017-01-12T18:43:27.255Z",
"tags" : [ "genie.name:hive", "genie.id:b62fc9a2-b502-47a3-a4d5-3d76ab525d9c" ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "INACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}'
HTTPie
$ echo '{
"id" : "b62fc9a2-b502-47a3-a4d5-3d76ab525d9c",
"created" : "2017-01-12T18:43:27.255Z",
"updated" : "2017-01-12T18:43:27.255Z",
"tags" : [ "genie.name:hive", "genie.id:b62fc9a2-b502-47a3-a4d5-3d76ab525d9c" ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "INACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}' | http PUT 'https://genie.example.com/api/v3/commands/b62fc9a2-b502-47a3-a4d5-3d76ab525d9c' 'Content-Type:application/json'
HTTP Messages
Request
PUT /api/v3/commands/b62fc9a2-b502-47a3-a4d5-3d76ab525d9c HTTP/1.1
Content-Type: application/json
Host: genie.example.com
Content-Length: 453
{
"id" : "b62fc9a2-b502-47a3-a4d5-3d76ab525d9c",
"created" : "2017-01-12T18:43:27.255Z",
"updated" : "2017-01-12T18:43:27.255Z",
"tags" : [ "genie.name:hive", "genie.id:b62fc9a2-b502-47a3-a4d5-3d76ab525d9c" ],
"version" : "1.0.0",
"user" : "genie",
"name" : "hive",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "INACTIVE",
"executable" : "/apps/hive/bin/hive",
"checkDelay" : 10000,
"memory" : null
}
9.5. Update Parts of a Command
9.5.1. Description
Update the metadata about a command using JSON Patch. This will ONLY update the fields you request be changed.
For more information about JSON Patch RFC see the website. |
9.5.3. API Docs
Request Documentation
Payload Fields
Path | Type | Description | Constraints | Optional |
---|---|---|---|---|
[] |
Array |
Array of patches to apply |
false |
|
[].op |
String |
Patch operation to perform. |
add, remove, replace, copy, move, test |
false |
[].path |
String |
The json path to operate on. e.g. /user |
false |
|
[].from |
String |
The json path to move or copy from. e.g. /user |
true |
|
[].value |
String |
The json value to put at the path for an add, replace or test |
true |
9.7. Delete a Command
9.8. Get Configs for Command
9.9. Add Configs to Command
9.9.1. Description
Add configuration file locations to an existing command.
Configurations are stored as a set so any duplicates will be ignored. |
9.10. Update Configs for a Command
9.10.1. Description
Update the configuration file locations for an existing command. Stored as a set so duplicates are ignored.
9.11. Remove All Configs From a Command
9.12. Get Tags For a Command
9.13. Add Tags to Command
9.13.1. Description
Add tags to an existing command. Stored as a set so all duplicates are ignored.
genie.id:{id} and genie.name:{name} tags are automatically added by the service.
|
9.14. Update Tags for a Command
9.14.1. Description
Update the tags for an existing command.
The genie.id:{id} and genie.name:{name} tags can’t be removed. They will automatically be added back
by the system.
|
9.15. Remove All Tags From Command
9.15.1. Description
Remove all the tags for an existing command
The genie.id:{id} and genie.name:{name} tags will NOT be removed by this operation
|
9.16. Remove Tag From Command
9.16.1. Description
Remove a tag from an existing command.
You can’t remove the genie.id:{id} and genie.name:{name} tags. They will just be added back by the system
|
9.16.3. API Docs
Request Documentation
9.17. Add Applications to a Command
9.17.1. Description
Add new applications (as array of their ID’s) to the existing set of applications the command depends on
The applications for the ID’s must already exist in the system |
9.17.3. API Docs
Request Documentation
9.18. Get the Applications for a Command
9.18.1. Description
Get the applications the command currently depends on. These are the applications that will be installed at runtime when a job is run with the given command
9.18.3. API Docs
Request Documentation
Request Examples
Response Documentation
HTTP Messages
Request
GET /api/v3/commands/b62fc9a2-b502-47a3-a4d5-3d76ab525d9c/applications HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 2450
[ {
"id" : "f6762922-4d6e-4194-aba8-0b759b751e51",
"created" : "2017-01-12T18:43:25.313Z",
"updated" : "2017-01-12T18:43:25.313Z",
"tags" : [ "genie.name:cfc4ecbd-934d-4a8d-8d68-74a13609b2f4", "genie.id:f6762922-4d6e-4194-aba8-0b759b751e51" ],
"version" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"user" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"name" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"dependencies" : [ ],
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/f6762922-4d6e-4194-aba8-0b759b751e51"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/f6762922-4d6e-4194-aba8-0b759b751e51/commands"
} ]
}, {
"id" : "a2754a79-4233-4428-b69b-24975baa6166",
"created" : "2017-01-12T18:43:25.331Z",
"updated" : "2017-01-12T18:43:25.331Z",
"tags" : [ "genie.name:cfc4ecbd-934d-4a8d-8d68-74a13609b2f4", "genie.id:a2754a79-4233-4428-b69b-24975baa6166" ],
"version" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"user" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"name" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"dependencies" : [ ],
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/a2754a79-4233-4428-b69b-24975baa6166"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/a2754a79-4233-4428-b69b-24975baa6166/commands"
} ]
}, {
"id" : "ac58cb4f-0b29-4c61-800d-edaede1d33bb",
"created" : "2017-01-12T18:43:25.455Z",
"updated" : "2017-01-12T18:43:25.455Z",
"tags" : [ "genie.id:ac58cb4f-0b29-4c61-800d-edaede1d33bb", "genie.name:cfc4ecbd-934d-4a8d-8d68-74a13609b2f4" ],
"version" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"user" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"name" : "cfc4ecbd-934d-4a8d-8d68-74a13609b2f4",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"dependencies" : [ ],
"status" : "ACTIVE",
"type" : null,
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/applications/ac58cb4f-0b29-4c61-800d-edaede1d33bb"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/applications/ac58cb4f-0b29-4c61-800d-edaede1d33bb/commands"
} ]
} ]
9.19. Set Applications for a Command
9.19.1. Description
Set the applications (as array of their ID’s) that are dependencies of the command.
The applications with the ID’s must already exist in the system |
9.19.3. API Docs
Request Documentation
9.20. Remove All Applications From a Command
9.20.1. Description
Remove all the applications currently set as dependencies of the command
This does NOT delete the application(s) from the system just unlinks them |
9.21. Remove A Single Application From a Command
9.21.1. Description
Remove the identified application as a dependency of the command.
This does NOT delete the application from the system just un-links it |
9.21.3. API Docs
Request Documentation
9.22. Get the Clusters that Have Command Available
9.22.3. API Docs
Request Examples
HTTP Messages
Request
GET /api/v3/commands/b62fc9a2-b502-47a3-a4d5-3d76ab525d9c/clusters?status=UP HTTP/1.1
Host: genie.example.com
Response
HTTP/1.1 200 OK
Content-Type: application/hal+json
Content-Length: 765
[ {
"id" : "73f78a4c-4b7f-4f2a-8402-027b418a7993",
"created" : "2017-01-12T18:43:25.789Z",
"updated" : "2017-01-12T18:43:25.851Z",
"tags" : [ "genie.name:10455e27-5aa9-4874-9718-d2283459eae5", "genie.id:73f78a4c-4b7f-4f2a-8402-027b418a7993" ],
"version" : "10455e27-5aa9-4874-9718-d2283459eae5",
"user" : "10455e27-5aa9-4874-9718-d2283459eae5",
"name" : "10455e27-5aa9-4874-9718-d2283459eae5",
"description" : null,
"setupFile" : null,
"configs" : [ ],
"status" : "UP",
"links" : [ {
"rel" : "self",
"href" : "https://genie.example.com/api/v3/clusters/73f78a4c-4b7f-4f2a-8402-027b418a7993"
}, {
"rel" : "commands",
"href" : "https://genie.example.com/api/v3/clusters/73f78a4c-4b7f-4f2a-8402-027b418a7993/commands"
} ]
} ]