Project

Namespaces

Public Vars

pigpen.pig

Functions to convert a PigPen query into a Pig script.

generate-script

added in 0.1.0

(generate-script query)(generate-script opts query)
Generates a Pig script from the relation specified and returns it as a string.
You can pass any relation to this and it will generate a Pig script - it doesn't
have to be an output. However, if there are no store commands, the script won't
do much. If you have more than one store command, use pigpen.core/store-many to
combine them. Optionally takes a map of options.

  Example:

    (generate-script (pig/store-clj "output.clj" foo))
    (generate-script {:debug "/temp/"} (pig/store-clj "output.clj" foo))

  Options:

    :debug - Enables debugging, which writes the output of every step to a file.
             The value is a path to place the debug output.

    :dedupe - Set to false to disable command deduping.

    :pigpen-jar-location - The location where your uberjar resides.
                           Defaults to 'pigpen.jar'.

  See also: pigpen.pig/write-script, pigpen.core/store-many
view source

set-options

(set-options opts relation)
Used to set options in a pig script. `opts` is a map of string/keyword/symbol
to anything. str is called on values.

  Example:

    (set-options {:pig.maxCombinedSplitSize 1000000} relation)

  Note: Pig options are global and apply to the entire script.
view source

write-script

added in 0.1.0

(write-script location query)(write-script location opts query)
Generates a Pig script from the relation specified and writes it to location.
You can pass any relation to this and it will generate a Pig script - it doesn't
have to be an output. However, if there are no store commands, the script won't
do much. If you have more than one store command, use pigpen.core/store-many to
combine them. Optionally takes a map of options.

  Example:

    (write-script "my-script.pig" (pig/store-clj "output.clj" foo))
    (write-script "my-script.pig" {:debug "/temp/"} (pig/store-clj "output.clj" foo))

  Options:

    :debug - Enables debugging, which writes the output of every step to a file.
             The value is a path to place the debug output.

    :dedupe - Set to false to disable command deduping.

    :pigpen-jar-location - The location where your uberjar resides.
                           Defaults to 'pigpen.jar'.

  See also: pigpen.pig/generate-script, pigpen.core/store-many
view source