pigpen.cascading

Functions to convert a PigPen query into a Cascading flow.

generate-flow

added in 0.3.0

(generate-flow query)(generate-flow connector query)
Transforms the relation specified into a Cascading flow that is ready to be
executed.

Optionally takes a Cascading FlowConnector (defaults to HadoopFlowConnector)

  Example:

    (generate-flow (pig/store-clj "output.clj" foo))

load-tap

added in 0.3.0

(load-tap tap)(load-tap tap f)
A thin wrapper around a tap. By default a vector of the tap's source fields
is created and returned as a single field. A custom function can be provided to
map the tap's source fields onto a single value.

  Example:

    (load-tap tap)
    (load-tap tap (partial zipmap [:a :b :c]))

store-tap

added in 0.3.0

(store-tap tap relation)
A thin wrapper around a sink tap. The tap must accept a single sink field
which is the value to store.