Iterators and Generators

Many Julia functions return lazy iterators that produce values on demand instead of building arrays. The Iterators module has the building blocks:

evens = (n for n in 1:typemax(Int) if iseven(n))   # infinite generator
println(collect(Iterators.take(evens, 5)))

println(collect(Iterators.take(Iterators.countfrom(10, 5), 4)))
println(collect(Iterators.take(Iterators.cycle([:a, :b]), 5)))
println(collect(Iterators.flatten([[1, 2], [3], [4, 5]])))
println(collect(Iterators.partition(1:7, 3)))
println(collect(Iterators.product(1:2, "ab")))

[2, 4, 6, 8, 10]
[10, 15, 20, 25]
[:a, :b, :a, :b, :a]
[1, 2, 3, 4, 5]
UnitRange{Int64}[1:3, 4:6, 7:7]
[(1, 'a') (1, 'b'); (2, 'a') (2, 'b')]

• A generator (expr for x in itr) is a lazy comprehension.
• take, drop, takewhile, dropwhile limit an iterator.
• countfrom, cycle, repeated make infinite ones.
• flatten, partition, product, zip, and enumerate combine them.

Why laziness helps

# sum of squares of the first 1000 odd numbers, without building arrays
total = sum(x^2 for x in Iterators.take(Iterators.filter(isodd, Iterators.countfrom(1)), 1000))
println(total)

first_big = first(n for n in Iterators.countfrom(1) if n^2 > 5000)
println(first_big)

1333333000
71

Lazy pipelines work on infinite or very large sequences and use almost no memory. Use collect only when you need an actual array.

Stateful generation with Channels

For a generator with complex internal state, a Channel lets a function put! values as it goes:

function fibonacci(n)
    Channel{Int}() do ch
        a, b = 0, 1
        for _ in 1:n
            put!(ch, a)
            a, b = b, a + b
        end
    end
end

println(collect(fibonacci(10)))
for f in fibonacci(5)
    print(f, " ")
end
println()

[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
0 1 1 2 3

The function runs as a task alongside the loop that consumes the values. Channels are also how tasks communicate in concurrent code (Level 3).