Generators in Typescript
Tags: javascript, typescript, node, webdev
Published: 2021-02-15 19:33:03

In this post we will understand what ES2015 generators are in Javascript/Typescript. Generators rely heavily on iterators, so if you don't know or would like to refresh your memory, take a look at my last [post](https://dev.to/gsarciotto/iterators-in-typescript-1d78).

# Introduction

As we know, iterators allows us to have total control of iterating through some structure, we get to decide if and when we get the next element of our iteration sequence, while hiding from our iterator's consumers implementation details of how we get these elements. However, everything has a cost, iterators can be quite tricky to implement since we have to keep track of the states that will control the flow of execution so that we can, for example, mark the iterator as complete.

Generators allows us to easily create iterators, making possible to implement some really cool stuff like stopping execution of functions to resume them later (sounds familiar to `async/await`?), pass values to the generator between these pauses and more.

# The basics

Generators can be quite complicated and somewhat different from what we are used to, so pay close attention to the details. A generator declaration is very similar to a function declaration:
```typescript
function* fooGen() {
    console.log("Hello from fooGen");
}
function foo() {
    console.log("Hello from foo")
}
```
You define a generator by using `function* fooGen` (you can actually do `function * fooGen` or `function *fooGen`). This is the only difference between our generator declaration and  the declaration of our `foo` function but they actually behave very differently. Consider the following:

```typescript
foo(); // Hello from foo
fooGen(); //
```
Our invocation of `foo` is as expected, however the invocation of `fooGen` didn't log anything. That seems odd, but this is the first big difference between functions and generators. **Functions** are **eager**, meaning whenever invoked, they will immediately begin execution while **generators** are **lazy**, meaning they will only execute our code whenever you explicitly tell them to execute. You may argue "but I ordered it to execute", however calling the generator doesn't execute its code, it only does some internal initialization.

So how do I tell a generator to execute *our* code? First let's see what `fooGen()` returns us. If we look at the type of `fooGen`, we will see the following: `function fooGen(): Generator<never, void, unknown>`, so let's look at what this `Generator` type is:

```typescript
interface Generator<T = unknown, TReturn = any, TNext = unknown> extends Iterator<T, TReturn, TNext> {
    // NOTE: 'next' is defined using a tuple to ensure we report the correct assignability errors in all places.
    next(...args: [] | [TNext]): IteratorResult<T, TReturn>;
    return(value: TReturn): IteratorResult<T, TReturn>;
    throw(e: any): IteratorResult<T, TReturn>;
    [Symbol.iterator](): Generator<T, TReturn, TNext>;
}
```
Wait, this interface has a `next`, `return` and `throw` methods isn't this an **iterator**? The answer is yes, but also notice that it is an **iterable**. So this interface is actually somewhat similar to the `IterableIterator` interface. If you want to know why they aren't the same, take a look at this [question](https://stackoverflow.com/questions/58568399/why-are-typescripts-iterableiterator-and-generator-generics-slightly-differ).

To order the generator to execute our code, we only need to call `next`:
```typescript
foo(); // Hello from foo
const it = fooGen();
it.next() // Hello from fooGen
```

Let's return some value from our generator:
```typescript
function* fooGen() {
    console.log("Hello from fGen");
    return "Bye from fGen";
}

const it = fooGen();
const result = it.next(); // Hello from fGen
console.log(result); // { value: 'Bye from fGen', done: true }
console.log(it.next()); // { value: undefined, done: true }
```
Notice that when you **return** something from a generator, it automatically completes the iterator, no need to manage state. Also notice that the value of the `return` expression is returned only once, subsequent calls to `it.next` return `undefined` in the value. Keep in mind that if there is no explicit `return` statement on your function or if the execution didn't reach a logical branch with the return, then `undefined` is assumed to be the return value.

## The `yield` keyword

So far we didn't do anything exciting with generators, we just used them as some more complicated functions. As said in the introduction, we can pause the execution of generators. We achieve this using the `yield` keyword.

The `yield` keyword pauses the execution of our iterator.
Whenever we call `next`, the generator will synchronously execute our code until a `yield` or a `return` statement is reached (assuming no errors happened, which we will see later). If the generator was in a paused state and we call `next` again it will resume the execution from wherever it was paused from.

```typescript
function*  fooGen() {
    console.log("Begin execution");
    yield;
    console.log("End execution");
}

const it = fooGen();
it.next();
console.log("The generator is paused");
it.next();

// Begin execution
// The generator is paused
// End execution
```

We can use `yield` to allow our generator to "return" multiple values (we say the generator yields these). We do this as follows:

```typescript
function*  fooGen() {
    console.log("Begin execution");
    yield "This value was yielded";
    console.log("End execution");
}

const it = fooGen();
console.log(it.next());
console.log("The generator is paused");
it.next();
// Begin execution
// { value: 'This value was yielded', done: false }
// The generator is paused
// End execution
```
Notice that using `yield` doesn't complete the generator iterator. This is very powerful. One example of where this behavior is useful is for producing (infinite) sequences in a memory efficient way, for example, let's look how we can implement [Fibonacci sequence](https://en.wikipedia.org/wiki/Fibonacci_number) using generators.

```typescript
function* fibonacciGenerator() {
    const f0 = 0;
    yield f0;
    const f1 = 1;
    yield f1;
    let previousValue = f0, currentValue = f1, nextValue;
    while(true) {
        nextValue = previousValue + currentValue;
        previousValue = currentValue;
        currentValue = nextValue;
        yield nextValue;
    }
}

const it = fibonacciGenerator();
console.log(it.next().value); // 0
console.log(it.next().value); // 1
console.log(it.next().value); // 1
console.log(it.next().value); // 2
console.log(it.next().value); // 3
```
Notice how the lazy nature of generators is very useful and how the ability to pause execution allows us to generate infinite elements of the sequence (let's ignore possible integer overflows) whenever we want while only needing to save the previous and the current values. Quite nice isn't it? Notice that we don't actually need to complete a generator, we may only take some values and never call `next` again, although I wouldn't recommend that.

## Passing values to the generator

There are two ways we can pass values to our generator. One is just as we would to a function, when creating the generator iterator. Let's expand the Fibonacci example to allow us to choose where to start the sequence:

```typescript
function* fibonacciGenerator(startingPosition = 1) {
    const f0 = 0;
    if(startingPosition === 1) {
        yield f0;
    }
    const f1 = 1;
    if(startingPosition <= 2) {
        yield f1;
    }
    let previousValue = f0, currentValue = f1, nextValue;
    let currentPosition = 3;
    while(true) {
        nextValue = previousValue + currentValue;
        previousValue = currentValue;
        currentValue = nextValue;
        if(currentPosition >= startingPosition){
            yield nextValue;
        } else {
            currentPosition += 1;
        }
    }
}

const it = fibonacciGenerator();
console.log(it.next().value); // 0
console.log(it.next().value); // 1
console.log(it.next().value); // 1
console.log(it.next().value); // 2
console.log(it.next().value); // 3

console.log();

const it2 = fibonacciGenerator(4);
console.log(it2.next().value); // 2
console.log(it2.next().value); // 3
console.log(it2.next().value); // 5
console.log(it2.next().value); // 8
console.log(it2.next().value); // 13
```
The other way to pass values to a generator is through `yield`. You may be confused, since until now we have been using `yield` to, well, yield values from the generator. The truth is that `yield` is an expression, meaning it evaluates to some value. To clarify, let's look at this example:

```typescript
function* fooGen() {
    while(true) {
        console.log(yield);
    }
}

const it = fooGen();
it.next();
it.next(1); // 1
it.next(2); // 2
it.next("heey"); // heey
```
The first call of `it.next()` will simply initiate execution of our generator iterator. Whenever it finds the `yield` expression, it will simply stop execution. Whenever we do `it.next(1)`, the `yield` will evaluate to the value `1` and thus we have `console.log(1)` and so on.

The following is allowed:

```typescript
function* accumulator(startingValue = 0): Generator<number, any, number> {
    let value = startingValue;
    while(true) {
        const input = yield value;
        value += input;
    }
}

const it = accumulator();
it.next();
console.log(it.next(3).value); // 3
console.log(it.next(10).value); // 13
console.log(it.next(-3).value); // 10
```
First the code is executed until the `yield` is found, yielding `value` (`startingValue`) . Whenever we call `next(3)`, the expression `yield value` evaluates to `3`, so now `input === 3` and then `value === 3`. The cycle then repeats.

A comment above about types. I had to explicitly type the generator above so that Typescript could automatically detect the type of `input`. The type inference of yield expressions is an [ongoing struggle](https://github.com/microsoft/TypeScript/issues/32523).

**Attention**: Whatever you pass to the **first** invocation of `next` will be ignored, so watch out.

## Error Handling

The code of our generator is just like any other function code, meaning we can put `try...catch` blocks inside it:
```typescript
function* fooGen() {
    try {
        throw "Hi";
    } catch(err) {
        console.log("Err caught in fooGen:", err);
    }
    return "End of execution";
}

const it = fooGen();
it.next();
console.log(it.next())

// Err caught in fooGen: Hi
// { value: "End of execution", done: true }
// { value: undefined, done: true }
```
Notice that after the exception was handled, the generator continued its execution. If we didn't have a `try...catch` inside the generator, the exception would bubble as it would normally:

```typescript
function* fooGen() {
    throw "Hi";
    return "End of execution";
}

const it = fooGen();
try {
    it.next();
} catch(err) {
    console.log("Exception caught outside of generator: ", err);
}
console.log(it.next());

// Exception caught outside of generator:  Hi
// { value: undefined, done: true }
```
Notice that our generator was completed because of the uncaught exception and didn't reach our return statement.

We can also throw errors from outside our generator to the inside:

```typescript
function* fooGen() {
    console.log("Beginning of execution");
    try {
        yield;
    } catch(err) {
        console.log("Error caught inside fooGen: ", err);
    }
    return "End of execution";
}

const it = fooGen();
it.next();
console.log(it.throw("Hi from outside"));
console.log(it.next());

// Beginning of execution
// Error caught inside fooGen:  Hi from outside
// { value: 'End of execution', done: true }
// { value: undefined, done: true }
```

Notice that the error was thrown at the point that the generator execution paused. If there was no `try...catch` at that point, then it would have bubbled as normal.

An example of where we would like to use `Generator.throw` is with our Fibonacci example. As it is implemented, eventually we will run into an [overflow](https://en.wikipedia.org/wiki/Integer_overflow#:~:text=In%20computer%20programming,%20an%20integer,than%20the%20minimum%20representable%20value). We can avoid this by using [bigInt](https://github.com/tc39/proposal-bigint/blob/master/README.md). In our case, we just want to complete the iterator when overflow happens.

```typescript
function* fibonacciGenerator() {
    const f0 = 0;
    yield f0;
    const f1 = 1;
    yield f1;
    let previousValue = f0, currentValue = f1, nextValue;
    try {
        while(true) {
            nextValue = previousValue + currentValue;
            previousValue = currentValue;
            currentValue = nextValue;
            yield nextValue;
        }
    } catch(err) {
        return;
    }
}
let flag = true;
let value: number | void;
const it = fibonacciGenerator();
while(flag) {
    value = it.next().value;
    if(value === Number.MAX_SAFE_INTEGER || !Number.isFinite(value)) {
        it.throw("overflow");
        console.log("overflow detected");
        console.log(it.next());
        flag = false;
    } else {
        console.log(value);
    }
}
```
Whenever we detect an overflow from outside our generator, we simply call `it.throw` to complete it so that no other garbage value gets generated from it.

# Generator Delegation

We may compose two or more generator using generator delegation `yield*` syntax:

```typescript
function* g1() {
    yield 2;
    yield 3;
    yield 4;
  }

function* g2() {
    yield 1;
    yield* g1();
    yield 5;
  }

const iterator = g2();
console.log(iterator.next()); // {value: 1, done: false}
console.log(iterator.next()); // {value: 2, done: false}
console.log(iterator.next()); // {value: 3, done: false}
console.log(iterator.next()); // {value: 4, done: false}
console.log(iterator.next()); // {value: 5, done: false}
console.log(iterator.next()); // {value: undefined, done: true}
```

What happens is that whenever a `yield*` is encountered, every subsequent `next` or `throw` will go to the delegated generator, `g2` in this case. This happens until `g2` completes and the completion value of `g2` is the value of `yield* g2()`. The subsequent call to `next` on `g1` after `g2` completes will continue from where `g1` was paused as normal. This is how you may write [coroutines](https://en.wikipedia.org/wiki/Coroutine) in Javascript.

You can actually use `yield*` with any iterable, such as arrays.

# Conclusion

Generators are a somewhat obscure but very interesting structure in Javascript. You probably won't find a generator in the wild, however it is good to know of their existence.

You can build very cool stuff with generators, *Async/Await* is implemented with generators and promises. If you want to learn more, see my next [post](https://dev.to/gsarciotto/implementing-async-await-55f).

Any doubts or sugestions, feel free to add a comment. Stay safe and until next time :)