Transducers

A pipeline like (->> xs (filter odd?) (map inc) (take 3)) creates an intermediate lazy sequence at each step. A transducer describes the same steps without tying them to sequences, so they run in one pass and can be reused with collections, streams, and channels.

Creating and using transducers

(def xf (comp (filter odd?)
              (map inc)
              (take 3)))

(println (into [] xf (range 20)))           ; build a collection
(println (transduce xf + (range 20)))        ; reduce with a transducer
(println (sequence xf (range 20)))           ; lazy sequence
(println (into #{} (map keyword) ["a" "b" "a"]))

[2 4 6]
12
(2 4 6)
#{:b :a}

• map, filter, take, remove, mapcat, partition-all, dedupe, and many more return a transducer when called without a collection.
• comp chains them — note they run left to right, like ->>, even though it's comp.
• into, transduce, and sequence apply a transducer to a source.

Why bother?

(def data (vec (range 1000000)))

(defn with-seqs [xs]
  (->> xs (filter even?) (map #(* % %)) (reduce +)))

(defn with-xf [xs]
  (transduce (comp (filter even?) (map #(* % %))) + xs))

(println (with-seqs data))
(println (with-xf data))
(println (= (with-seqs data) (with-xf data)))

166666166667000000
166666166667000000
true

Same result, but the transducer version allocates no intermediate sequences and is typically noticeably faster on large inputs.

Reusing one pipeline

(require '[clojure.string :as str])

(def clean-words
  (comp (mapcat #(str/split % #"\s+"))
        (map str/lower-case)
        (remove str/blank?)
        (distinct)))

(def lines ["The quick  brown fox" "the lazy dog" "  Quick thinking "])

(println (into [] clean-words lines))
(println (transduce clean-words (completing (fn [n _] (inc n))) 0 lines))
(println (vec (eduction clean-words (take 2 lines))))

[the quick brown fox lazy dog thinking]
7
[the quick brown fox lazy dog]

The same clean-words works with into, transduce, and eduction — and, in Level 3, with core.async channels. completing turns a two-argument reducing function into a full one.

What's next

Try the Intermediate Projects, then continue to Level 3 — Advanced: macros, core.async, spec, project tooling, testing, web services, ClojureScript, and Babashka.