pigpen.fold
avg
added in 0.2.0
(avg)(avg fold)
Average the values. All values must be numeric. Optionally takes another
fold operation to compose.
Example:
(fold/avg)
(->> (fold/map :foo) (fold/avg)) ; average the foo's
(->> (fold/keep identity) (fold/avg)) ; avg non-nils
(->> (fold/filter #(< 0 %)) (fold/avg)) ; avg positive numbers
(->>
(fold/map :foo)
(fold/keep identity)
(fold/avg))
See also: pigpen.fold/count, pigpen.fold/sumcount
added in 0.2.0
(count)(count fold)
Counts the values, including nils. Optionally takes another fold operation
to compose.
Example:
(fold/count)
(->> (fold/keep identity) (fold/count)) ; count non-nils
(->> (fold/filter #(< 0 %)) (fold/count)) ; count positive numbers
(->>
(fold/map :foo)
(fold/keep identity)
(fold/count))
See also: pigpen.fold/sum, pigpen.fold/avgdistinct
added in 0.2.0
(distinct)(distinct fold)
Returns the distinct set of values.
Example:
(fold/distinct)
(->> (fold/map :foo)
(fold/keep identity)
(fold/distinct))filter
added in 0.2.0
(filter f)(filter f fold)
Pre-processes data for a fold operation. Same as clojure.core/filter.
first
added in 0.2.0
(first)(first fold)
Returns the first output value. This is a post-reduce operation, meaning that
it can only be applied after a fold operation that produces a sequence.
Example:
(fold/first)
(->> (fold/map :foo)
(fold/sort)
(fold/first))
See also: pigpen.fold/last, pigpen.fold/min, pigpen.fold/maxfold-fn
added in 0.2.0
(fold-fn reducef)(fold-fn combinef reducef)(fold-fn combinef reducef post)(fold-fn pre combinef reducef post)
Creates a pre-defined fold operation. Can be used with cogroup and group-by
to aggregate large groupings in parallel. See pigpen.core/fold for usage of
reducef and combinef.
Example:
(def count
(pig/fold-fn + (fn [acc _] (inc acc))))
(def sum
(pig/fold-fn +))
(defn sum-by [f]
(pig/fold-fn + (fn [acc value] (+ acc (f value)))))juxt
added in 0.2.0
(juxt & folds)
Applies multiple fold fns to the same data. Produces a vector of results.
Example:
(fold/juxt (fold/count) (fold/sum) (fold/avg))
keep
added in 0.2.0
(keep f)(keep f fold)
Pre-processes data for a fold operation. Same as clojure.core/keep.
last
added in 0.2.0
(last)(last fold)
Returns the last output value. This is a post-reduce operation, meaning that
it can only be applied after a fold operation that produces a sequence.
Example:
(fold/last)
(->> (fold/map :foo)
(fold/sort)
(fold/last))
See also: pigpen.fold/first, pigpen.fold/min, pigpen.fold/maxmap
added in 0.2.0
(map f)(map f fold)
Pre-processes data for a fold operation. Same as clojure.core/map.
mapcat
added in 0.2.0
(mapcat f)(mapcat f fold)
Pre-processes data for a fold operation. Same as clojure.core/mapcat.
max
added in 0.2.0
(max)(max fold)(max comp)(max comp fold)
Return the maximum (last) value of the collection. If a comparator is not
specified, clojure.core/compare is used. Optionally takes another fold
operation to compose.
Example:
(fold/max)
(fold/max >)
(->>
(fold/map :foo)
(fold/max >))
See also: pigpen.fold/max-key, pigpen.fold/min, pigpen.fold/topmax-key
added in 0.2.0
(max-key keyfn)(max-key keyfn fold)(max-key keyfn comp)(max-key keyfn comp fold)
Return the maximum (last) value of the collection based on (keyfn value).
If a comparator is not specified, clojure.core/compare is used. Optionally takes
another fold operation to compose.
Example:
(fold/max-key :foo)
(fold/max-key :foo >)
See also: pigpen.fold/max, pigpen.fold/min-key, pigpen.fold/top-bymin
added in 0.2.0
(min)(min fold)(min comp)(min comp fold)
Return the minimum (first) value of the collection. If a comparator is not
specified, clojure.core/compare is used. Optionally takes another fold
operation to compose.
Example:
(fold/min)
(fold/min >)
(->>
(fold/map :foo)
(fold/min >))
See also: pigpen.fold/min-key, pigpen.fold/max, pigpen.fold/topmin-key
added in 0.2.0
(min-key keyfn)(min-key keyfn fold)(min-key keyfn comp)(min-key keyfn comp fold)
Return the minimum (first) value of the collection based on (keyfn value).
If a comparator is not specified, clojure.core/compare is used. Optionally takes
another fold operation to compose.
Example:
(fold/min-key :foo)
(fold/min-key :foo >)
See also: pigpen.fold/min, pigpen.fold/max-key, pigpen.fold/top-bypreprocess
added in 0.2.0
(preprocess f')
Takes a a clojure seq function, like map or filter, and returns a fold
preprocess function. The function must take two params: a function and a seq.
Example:
(def map (preprocess clojure.core/map))
(pig/fold (map :foo))remove
added in 0.2.0
(remove f)(remove f fold)
Pre-processes data for a fold operation. Same as clojure.core/remove.
sort
added in 0.2.0
(sort)(sort fold)(sort c fold)
Sorts the data. This sorts the data after every element, so it's best to use
with take, which also limits the data after every value. If a comparator is not
specified, clojure.core/compare is used.
Example:
(fold/sort)
(->>
(fold/sort)
(fold/take 40))
(->>
(fold/sort >)
(fold/take 40))
See also: pigpen.fold/sort-by, pigpen.fold/topsort-by
added in 0.2.0
(sort-by keyfn)(sort-by keyfn fold)(sort-by keyfn c fold)
Sorts the data by (keyfn value). This sorts the data after every element, so
it's best to use with take, which also limits the data after every value. If a
comparator is not specified, clojure.core/compare is used.
Example:
(fold/sort-by :foo)
(->> (vec)
(fold/sort-by :foo)
(fold/take 40))
(->> (vec)
(fold/sort-by :foo >)
(fold/take 40))
See also: pigpen.fold/sort, pigpen.fold/top-bysum
added in 0.2.0
(sum)(sum fold)
Sums the values. All values must be numeric. Optionally takes another
fold operation to compose.
Example:
(fold/sum)
(->> (fold/map :foo) (fold/sum)) ; sum the foo's
(->> (fold/keep identity) (fold/sum)) ; sum non-nils
(->> (fold/filter #(< 0 %)) (fold/sum)) ; sum positive numbers
(->>
(fold/map :foo)
(fold/keep identity)
(fold/sum))
See also: pigpen.fold/count, pigpen.fold/avgtake
added in 0.2.0
(take n)(take n fold)
Returns a sequence of the first n items in coll. This is a post-reduce
operation, meaning that it can only be applied after a fold operation that
produces a sequence.
Example:
(->>
(fold/sort)
(fold/take 40))top
added in 0.2.0
(top n)(top comp n)
Returns the top n items in the collection. If a comparator is not specified,
clojure.core/compare is used.
Example:
(fold/top 40)
(fold/top > 40)
See also: pigpen.fold/top-bytop-by
added in 0.2.0
(top-by keyfn n)(top-by keyfn comp n)
Returns the top n items in the collection based on (keyfn value). If a
comparator is not specified, clojure.core/compare is used.
Example:
(fold/top-by :foo 40)
(fold/top-by :foo > 40)
See also: pigpen.fold/topvec
added in 0.2.0
(vec)
Returns all values as a vector. This is the default fold operation if none
other is specified.
Example:
(fold/vec)
(->> (fold/vec)
(fold/take 5))