Generator Functions & Iterators in JavaScript
Tags: javascript, webdev, programming, beginners
Published: 2024-04-29 10:54:23

More often than not, interviewers ask about Generator Functions, the Yield Keyword, and Iterators. Most developers have no idea what they mean. 

Through this article, I will explain those complicated concepts using the fundamentals. Even I didn’t know they existed. I read about them in a QNA where developers were supposed to submit the names of intricate concepts.

However, Generators have the same reputation as Closures. They are complicated because the crowd says so. Let us understand what Generators mean, the Yield keyword, and Iterators.

## What are Iterators 

**By definition, iterators represent recurrence**. In JavaScript, as per MDN, Iterators and Generators bring the concept of iteration, i.e., repetition, directly into the core language. 

If you remember, we have loops in JavaScript to perform the repetition of code statements. We have methods, like `map()`, too.

However, the concept of Iterators in this context alongside Generators is different. Iterators that are custom and Generators use loops to perform their function. Generators and custom Iterators provide a mechanism for modifying the behaviour of the default iterators in the language, like loops.

```
const exampleArr = [1, 2, 3, 4, 5];

for(const val of exampleArr) {
	console.log(val);
}
```

You are looking at a standard for-of loop. It doesn’t keep track of the current index or iteration number. It doesn’t know whether it should begin from the left or right-hand side. Unlike the traditional for-loop, we don’t mention a starting or ending index in the for-of loop.

The loop moves from left to right sequentially because the array contains an iterator function instructing the for-of loop to iterate through a specific array in that sequence. It is the default behaviour.

The for-of loop is an iterator because it executes the same statements on each array element repetitively. However, the for-of loop is a default iterator.

Sometimes, you require a custom iterator that repeats specific code statements or traverses through a datatype in a specified manner. In massive applications, developers create custom data types. Those datatypes require custom iterators to access that data and manipulate them. Iterators are crucial when writing custom libraries.

In the new context, an iterator is an object that defines the sequence of the iteration and potentially the values that will get returned when the code statements terminate. We can create custom iterators using the Iterator Protocol.

Every iterator implements the Iterator Protocol with a `next()` method that gets returned by it. The `next()` method seems logical because the job of iterators is to traverse through elements from one index to another in a specific sequence. The method returns an object with two properties —

1. `value` — It holds the next value in the iteration sequence.

2. `done` — It holds a boolean value. It represents whether the last value in the sequence has been traversed or consumed. If the `.value` and `.done` properties are present in an object, they represent the iterator’s return value.

The most common iterators in JavaScript are `forEach()`, `filter()`, `map()`, etc., which return each value in a sequence. Those are default iterators. Now, let us create a custom iterator.

### Custom Iterator

Let’s reciprocate the for-loop and create a sequence that executes code statements for each iteration of an imaginary datatype. Remember, this is not how we make custom iterators. We use Generators, and the following example is the complicated approach. However, it is required to understand Generator Functions.

```
function customIterate(start = 0, end = Infinity, step = 1) {
  let nextIndex = start;
  let iterationCount = 0;

  return {
    // Returns a method to iterate over the iterator
    next() {
      // Stores an object throughout the iteration to keep track of the value and progress
      let result;
      // Indicates that iteration isn't finished
      if (nextIndex < end) {
        // Assigning those traditional value and done properties
        result = {
          value: nextIndex,
          done: false,
        };

        // Incrementing the index based on the steps
        nextIndex += step;
        // Incrementing the iteration count
        iterationCount++;

        // Returning the object
        return result;
      } else {
        return { value: iterationCount, done: true };
      }
    },
  };
}

// Storing the instance of the Iterator
const instanceOfIterator = customIterate(0, 20, 5);

// Invoking the next() method that iterates through the iterator
let result = instanceOfIterator.next();

// Till the .done property returns true
while (!result.done) {
  console.log(result.value);

  // invoking the next() method to update the result value for the while-loop and the iterator
  result = instanceOfIterator.next();
}
```

I will start with a function `customIterate()`. It accepts your desired starting index, ending index, and step size as inputs. We will use default values if the user fails to provide them.

Now, imagine an invisible runner flashing along numbers. Initially, they stand at the starting index. Inside the function, we prepare two trackers: `nextIndex` tells us where the runner is now, and `iterationCount` keeps tabs on how many laps they completed.

The real magic happens in the `next()` method. It is the engine that drives the iteration. If our runner has not crossed the finish line, it hands you the current position and prompts the running to continue with a `.done` flag set to `false`. Then, it hops forward based on the step size and keeps track of one more lap.

But when the runner reaches the end, things change. Instead of another position, I offer the total number of laps as a farewell and wave a `.done` flag of `true`. The `.done` property lets you know the exercise is over.

To put this runner to work, you create an iterator object using `customIterate()`. Then, you kick things off with a manual call to the `next()`. The manual call gives you the initial position and gets the runner moving.

The next step is a familiar friend: the `while` loop. As long as the runner keeps going, you can access their current location through the `result.value` property, do whatever cool stuff you want with it, and then hand them a new `next()` call to keep the journey going.

This custom iterator approach works, but it is like building a bicycle from scratch when a sleek, ready-made option exists. Now, this is where generator functions come in. They offer a much simpler and safer way to handle iterations, and I will dive into their world next.

## What are Generator Functions

We cannot create iterators that use the mechanism of the for-loop with standard Iterator Protocol. The solution is to use Generator Functions, but that's not the only reason anyone would want to use them.

Standard Iterators store explicit state values inside their iterator functions. For example, the `nextStart` and `iterationCount` variables store the current progress or state of the iterator.

Maintaining these states manually with variables is complicated without careful programming principles. Generator Functions are a powerful alternative that can maintain the states.

Generator functions allow you to write the same iterative algorithm by creating a single function whose execution is not continuous. We can pause and resume at random intervals. These functions have a separate syntax.

To define generators, we use the `function*` keyword. There is an extra asterisk to the standard syntax. We're still trying to iterate through custom datatype but without explicit states. In addition to the strange syntax, JavaScript introduces a new keyword called `yield`.

When we invoke generator functions, they don't initially execute the code. Instead, they return a specific object called a `Generator`. We store that object inside another variable and invoke the `next()` method or access the `.done` and `.value` properties through that object. Remember, Generators are beyond Iterators.

Generator functions can run infinite loops because they can pause and resume their execution. We cannot have infinite loops without them because that would freeze our respective machines. The loop will continuously execute without any pauses.

Generators allow us to insert those pauses and run infinite loops without worrying about freezing our computers. Until the `next()` method gets called, the loop stays on hold after discovering the `yield` keyword.

You will understand this in a bit. Depending on your application, you can use Generators to mimic the `async`/`await` functionality and deal with asynchronous code. Not to mention, they are also used to implement infinite scrolls on the front end.

### What is the Yield Keyword

By definition, the `yield` keyword pauses and resumes the generator execution. The `yield` keyword pauses the execution of the generator till it encounters another `next()` method. Or till the next iteration comes around while using a loop. It is a distinctive sort of `return` statement keyword used inside a generator.

The `yield` keyword is similar to `async` / `await`. Without the `async` keyword, we cannot utilize `await` and vice-versa. Similarly, we cannot use the `yield` keyword without a generator function.

```
// Generator Function
function* iterable() {
  yield 1;
  yield 2;
  yield 3;
  yield 4;
}

// Storing the returned object of the Generator
const iterableVal = iterable();

// Iterating
for (const val of iterableVal) {
  console.log(val);
}
```

If you run this code, you will get a sequence of values printed to the console from 1 to 4. It prints all the values provided inside the generator function with the `yield` keyword. Why? Let me explain.

Using the for-of loop, JavaScript prints the results sequentially. The JS Engine detects a generator function that represents iterable and repetitive elements. The `yield` keyword works in harmony with the function.

Generators accept values and repeatedly return a new output. It returns the output using the `yield` keyword sequentially with pauses at different intervals. The `yield` keyword is like the `return` keyword. Hence, those numbers show up in the console.

Generators inherit the idea of an iterator that iterates and executes the same code on each element to get an output. Whenever JS encounters a `yield` keyword, it prints the value followed by the keyword and resumes the execution. When it encounters another `yield` keyword, it repeats the same process.

Let us expand on that example. I will implement the same logic we had implemented earlier when understanding Iterators.

```
// Example 2.0

// Generator Function
function* genIterator(start, end, step = 1) {
  for (let i = start; i <= end; i += step) {
    yield i;
  }
}

// Creates a Generator Object using the Generator Function
const generatorObject = genIterator(1, 20);

// Loops through the values of the Generator Object
for (const val of generatorObject) {
  console.log(val);
}
```

I defined a function with a for-loop, passing in the same parameters. We do not need if-statements or manual step-by-step increments; the for-loop does the heavy lifting, returning an integer for each iteration.

I stored the outcome of invoking the generator function inside another variable because generator functions return a `Generator` object. In our case, we stored the object inside the `generatorObject` variable after invoking `genIterator()`.

The `yield` statement returns the value stored in `i` after each iteration. This lightweight approach lets you iterate through custom data types without crafting logic for each step. 

You can fetch information from the database and call the `yield` keyword to pause and return the values instead of iterating continuously.

Think of it this way: instead of gulping down a whole database at once, you can use `yield` to savour each piece of information, pausing between bites. It ensures efficient handling of large datasets and provides ultimate control over the iteration process.

If we do not use the `yield` keyword, we can remove the for-of loop and invoke the `next()` method repeatedly and manually. Since the `yield` keyword pauses the generator, we must execute the `next()` method to resume the execution. We use the for-of loop for the same reason.

The for-loop replaces the massive logic inside the loop, and the for-of loop remains the same to continue the iterator. You can access the same `.done` and `.value` properties too.

```
function* genIterator(start, end, step = 1) {
  for (let i = start; i <= end; i += step) {
    yield i;
  }
}

const generatorObject = genIterator(1, 5);

console.log(generatorObject.next().value);
console.log(generatorObject.next().done);

console.log(generatorObject.next().value);
console.log(generatorObject.next().done);

console.log(generatorObject.next().value);
console.log(generatorObject.next().done);
```

![Output Image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i2719t91mdo0a5nfwjht.png)

A few iterations are skipped in this approach because the `next()` method gets executed. The paused iterations are resumed and then skipped while printing them. Instead, you can do the following.

```
function* genIterator(start, end, step = 1) {
  for (let i = start; i <= end; i += step) {
    yield i;
  }
}

const generatorObject = genIterator(1, 5);

console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
console.log(generatorObject.next());
```

![Output 2 Image](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ap2oyswiil9apgcta24s.png)

The simplest way to access the values directly without skipping iterations and manually executing the `next()` method is to use a for-of loop, as shown in Example 2.0.

If you still cannot understand the difference, let me explain. We all know that loops do not halt at any point to execute code statements. Once they start, they only stop at the end. 

We can use the `break` keyword to stop the loop entirely or the `continue` keyword to skip iterations. However, we can never pause the execution of a loop.

It is where the `yield` keyword becomes a convenient mechanism. We are using the for-loop inside the generator function. Instead of letting the for-loop complete its execution immediately, we use the `yield` keyword to pause and resume its execution once we attain the desired data. 

It helps us manage database transactions, manipulation or handling data. Let me do DOM manipulation using a Generator Function.

```
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Generator Functions</title>
  </head>
  <body>
    <button class="yield-result">Click Here</button>

    <script src="script.js"></script>
    <script src="example.js"></script>
  </body>
</html>
```

```
function* genIterator(start, end, step = 1) {
  for (let i = start; i < end; i += step) {
    yield i;
  }
}

const generatorObject = genIterator(1, 10);
const btn = document.querySelector(".yield-result");

btn.addEventListener("click", () => {
  btn.textContent = generatorObject.next().value;
});
```

![Output 3 GIF](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/irhxjb2b9rydxfdux370.gif)

I created a button element inside the HTML document. I attached an event listener, used the values fetched from the generator function after each pause and displayed it through the button.

The last returned object from the `next()` with the `.done` property contained `true` and indicated its competition after the entire generator completes its execution. The `.value` property remained `undefined` after resuming the last yield statement because there are no more values to return.

When you invoke the function, it returns nothing except a `Generator` object. Your code remains unexecuted. Previously, I had four yield statements inside a generator function and configured a for-of loop to iterate through them. Here’s how they worked.

{% embed https://youtu.be/6hzZwWeNTuQ?si=9ZRhW9cPpzUXKYvg %}

The last `yield` statement doesn't return `true` for the `.done` property because the generator function gets paused and doesn't finish the execution.

We can have multiple generators at the same time. You can create another instance or object of the same or another generator and use a loop or manually invoke the `next()` method.

```
// First Generator
function* firstGen() {
  yield 1;
}

// Second Generator
function* secondGen() {
  yield 1;
}

const firstObj = firstGen();
const secondObj = firstGen();

// First Object
console.log(firstObj.next());
// Second Object
console.log(secondObj.next());
```

Each instance of the same generator is separate and does not share values. You can create many of them. Let’s create an infinite loop and control it using a generator.

```
function* infiniteLoop() {
  const username = "@whyafan";
	
	// Infinite Loop
  while (true) {
    yield username;
    console.log("Yielded!");
  }

	// Ignored because infinite loop never ends
  console.log("Can JS reach this statement?");
}

const infiniteExample = infiniteLoop();

console.log(infiniteExample.next());
console.log(infiniteExample.next());
console.log(infiniteExample.next());
console.log(infiniteExample.next());
```

I inserted a while-loop and conditioned it to execute the code statements inside it till the condition represented a truthy value. Of course, true will infinitely return a truthy value, so the loop will never stop. 

I yielded my username on all social media platforms through the while-loop. Whenever the returned object gets printed, the value will be @whyafan. 

![Output Image 4](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bprw5j0epsnjc2647rqx.png)

Remember, the `.done` property will never return true because this is an infinite loop. I invoked the `next()` method four times and received the returned object in the same limit because the `yield` keyword pauses the generator till an approaching `next()` method gets discovered in the wild.

Let us see how we can use the generator function for an array. We will access the array elements sequentially.

```
function* arrayExample(start, end = Infinity, step = 1, array) {
  for (let i = start; i <= end; i += step) {
    yield array[i];
  }
}

const array = [1, 2, 3, 4, 5, 6, 7, 8, 9];
const arrayObj = arrayExample(0, 10, 2, array);

console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
console.log(arrayObj.next());
```

![Output Image 5](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x3ubyt8vs24l5ixbwykn.png)

Lastly, let us see how we can use the `yield` keyword to accept values passed as a parameter to the `next()` method.

```
function* passingToNext() {
  let id = 1;

  while (true) {
		// This is where JS received the params
    const increment = yield id;

    if (increment !== null || increment !== undefined) {
      id += increment;
    } else {
      id++;
    }
  }
}

const yieldObj = passingToNext();
console.log(yieldObj.next());
console.log(yieldObj.next(2));
console.log(yieldObj.next(3));
console.log(yieldObj.next(4));
```

I created an infinite while loop like the previous examples. I purposely stored the result of the `yield` keyword inside another variable because when you pass a value as a parameter to the `next()` method, it takes that value and gives it to the `yield` keyword.

When I passed `2` to the `next()` method, it received that integer through the `id` variable with the `yield` keyword and stored that inside the `increment` variable. Then, I checked whether the received value is `null` or `undefined`. If yes, I incremented the `id` variable without the value received from the user.

Otherwise, I incremented the `id` variable with the integer received by the user when they passed it as a value to the `next()` method. 

You cannot pass a value to the first `next()` method execution because `yield` doesn’t exist during that iteration. It returns a result at the second iteration onwards.

Remember, Generator functions can be a property of an object or the method of a class.