Streams

As an alternative to Enum, Elixir provides the Stream module which supports lazy operations:

iex> 1..100_000 |> Stream.map(&(&1 * 3)) |> Stream.filter(odd?) |> Enum.sum
7500000000

Streams are lazy, composable enumerables.

In the example above, 1..100_000 |> Stream.map(&(&1 * 3)) returns a data type, an actual stream, that represents the map computation over the range 1..100_000:

iex> 1..100_000 |> Stream.map(&(&1 * 3))
#Stream<[enum: 1..100000, funs: [#Function<34.16982430/1 in Stream.map/2>]]>

Furthermore, they are composable because we can pipe many stream operations:

iex> 1..100_000 |> Stream.map(&(&1 * 3)) |> Stream.filter(odd?)
#Stream<[enum: 1..100000, funs: [...]]>

Instead of generating intermediate lists, streams build a series of computations that are invoked only when we pass the underlying stream to the Enum module. Streams are useful when working with large, possibly infinite, collections.

Many functions in the Stream module accept any enumerable as an argument and return a stream as a result. It also provides functions for creating streams. For example, Stream.cycle/1 can be used to create a stream that cycles a given enumerable infinitely. Be careful to not call a function like Enum.map/2 on such streams, as they would cycle forever:

iex> stream = Stream.cycle([1, 2, 3])
#Function<15.16982430/2 in Stream.cycle/1>
iex> Enum.take(stream, 10)
[1, 2, 3, 1, 2, 3, 1, 2, 3, 1]

On the other hand, Stream.unfold/2 can be used to generate values from a given initial value:

iex> stream = Stream.unfold("hełło", &String.next_codepoint/1)
#Function<39.75994740/2 in Stream.unfold/2>
iex> Enum.take(stream, 3)
["h", "e", "ł"]

Another interesting function is Stream.resource/3 which can be used to wrap around resources, guaranteeing they are opened right before enumeration and closed afterwards, even in the case of failures. For example, we can use it to stream a file:

iex> stream = File.stream!("path/to/file")
#Function<18.16982430/2 in Stream.resource/3>
iex> Enum.take(stream, 10)

The example above will fetch the first 10 lines of the file you have selected. This means streams can be very useful for handling large files or even slow resources like network resources.

The amount of functions and functionality in the Enum and Stream modules can be daunting at first, but you will get familiar with them case by case. In particular, focus on the Enum module first and only move to Stream for the particular scenarios where laziness is required, to either deal with slow resources or large, possibly infinite, collections.

Next we’ll look at a feature central to Elixir, Processes, which allows us to write concurrent, parallel and distributed programs in an easy and understandable way.