elixir-lang/elixir — pull request #1338 by josevalim (2013-06-29 20:42:13+00:00)
Title: Introducing Streams

Since the addition of Enumerable in Elixir v0.9.0, we have had some dangling definitions regarding our data structures and how they are enumerated. Also, since 0.9.0, the Enumerable module allows lazy enumeration and composition, which we haven't exploited so far. This is proposal introduces streams as an attempt to tie up the loose ends by providing a consistent names and a more complete API.

## The Stream module

Streams are composable, lazy enumerables. Any enumerable that generates
items one by one during enumeration are called streams. For example,
Elixir's `Range` is a stream:

iex> range = 1..5
1..5
iex> Enum.map range, &1 * 2
[2,4,6,8,10]

In the example above, as we mapped over the range, the elements being
enumerated were created one by one, during enumeration. The `Stream`
module allows us to map the range, without triggering its enumeration:

iex> range = 1..3
iex> stream = Stream.map(range, &1 * 2)
iex> Enum.map(stream, &1 + 1)
[3,5,7]

Notice we started with a range and then we created a stream that is
meant to multiply each item in the range by 2. At this point, no
computation was done yet. Just when `Enum.map/2` is called that we
enumerate each item in the range, multiplying it per 2 and adding 1.
We say the functions in `Stream` are lazy and the functions in `Enum`
are eager.

Due to its laziness, streams are useful when working with large collections
or even infinite collections. When chaining many operations with `Enum`,
intermediary lists are created, while `Stream` creates a recipe of
computations that are executed just at a later moment. Let's see another
example:

1..3 |>
Enum.map(IO.inspect(&1)) |>
Enum.map(&1 * 2) |>
Enum.map(IO.inspect(&1))
1
2
3
2
4
6
#=> [2,4,6]

Notice that we first printed each item in the list, then multiplied each
element by 2 and finally printed each new value. In this example, the list
was iterated three times. Let's see an example with streams:

stream = 1..3 |>
Stream.map(IO.inspect(&1)) |>
Stream.map(&1 * 2) |>
Stream.map(IO.inspect(&1))
Enum.to_list(stream)
1
2
2
4
3
6
#=> [2,4,6]

Although the end result is the same, the order in which the iterms were
printed changed! With streams, we print the first item and then print
its double. In this example, the list was iterated just once!

That's what we meant when we first said that streams are composable,
lazy enumerables. Notice we could call `Stream.map/2` multiple times,
effectively composing the streams and they are lazy. The computations
are performed only when you call a function from the `Enum` module.

## Creating Streams

There are many functions in Elixir's standard library that returns
streams, some examples are:

* `IO.lines_stream/1` - It streams input lines, one by one;
* `URI.query_decoder/1` - Decodes a query string, pair by pair;

The `Stream` module also allows us to create streams from any enumerable:

iex> stream = Stream.map([1,2,3], &1 * 2)
iex> Enum.map(stream, &1 + 1)
[3,5,7]

By simply passing a list (which is an enumerable) as first argument
to `Stream.map/2`, we have automatically created a stream that will
multiply the items in the list by 2 on enumeration.

The `Stream` module also provides other functions for creating streams,
like `Stream.cycle/1`.

## This pull request...

1. Adds `Stream.map/2`, `Stream.filter/2` as basic examples, we can extend the API once we get this merged;

2. Adds `Stream.cycle/1` as an example of infinite stream;

3. Adds `Stream.with_index/1` for enumeration with indexes;

4. Deprecates `File.iterator/1` in favor of `File.lines_stream/1`;

## Feedback

The most important feedback for now is: are Streams correct conceptually? Does it feel similar to what other languages have been calling streams? Suggestions in the code are welcome, but please don't worry about typos in the docs for now.