Behind The Scenes Of For Loop
Tags: python, programming, codenewbie
Published: 2023-01-11 15:52:33

---
title: Behind The Scenes Of For Loop
published: true
description: Debunking the internal workings of for loop and understanding an Iterator and a Generator in Python
tags: Python, Programming, codenewbie
series: Learn Python with AV
# cover_image:
# Use a ratio of 100:42 for best results.
#published_at: 2023-01-11 15:28 +0000
---

![Photo by mahdis mousavi on Unsplash](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dggsgantrngdj0ue8jsr.jpg)

A **For Loop** is a control flow statement for specifying iteration in computer science. In simpler terms, a for loop is a section or block of code which runs **repeatedly** until a certain condition has been satisfied. It is the most basic concept, without which every programming language will be incomplete.

While exploring the ever-surprising python language, I uncovered that it is an **Iterator** that plays a huge part in the For Loop and many other higher-order functions such as the range function. This article is written to understand what is an iterator, what is a generator, and how it helps us to understand the crux behind the For Loop.

**What is an Iterator in python?**

An **iterator is** any python object which can be **iterated upon** and returns data one at a time. They are implemented in comprehension, loops, and generators but are not readily seen in plain sight. They are the pillars behind most of the beforementioned functionality of python.

An iterator object must support two methods namely __iter__() and __next__(). This is collectively called an **Iterator protocol**.

* iter() is a function used to return an iterator by calling the __iter__() method on an Iterable object.

* next() is used to iterate through each element. StopIteration is an exception raised whenever the end is reached.

Wait, iter() method is called on an **Iterable** object? What is the difference between an **Iterator** and an **Iterable**?

Well, an **Iterable** is an object, that one can **iterate over**. It generates an Iterator when passed to iter() method. Example of iterable objects includes string, list, tuple, dictionary, etc.

> Remember, every iterator is also an iterable, but not every iterable is an iterator.

Now that we have some theoretical knowledge about an iterator let’s see them in action.

Let us first see how the iterable object can be converted into an iterator.

```python
    # We know string can be iterated over.
    s = "Loreum Ipsum"
    for word in s:
        print(word)
```

Output:
```
    L
    o
    r
    e
    u
    m
     
    I
    p
    s
    u
    m
```

What will happen if we call, next() on the string

```python
    s = "Loreum Ipsum"
    next(s)
```
Output:
```
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    ~\AppData\Local\Temp/ipykernel_41560/3883344095.py in <module>
          1 # Calling next() on string
    ----> 2 next(s)
    
    TypeError: 'str' object is not an iterator
```

As we mentioned earlier we first need to call the iter method to create an iterator from the iterable object then only we can use this next method to access the element one by one. See the code snippet below which corrects that,

```python
    s = "Loreum Ipsum"
    myIter = iter(s)
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
    print(next(myIter))
```
Output:
```
    L
    o
    r
    e
```

Now we will try to create an iterator that will print numbers from 1 to n.

```python
    class PrintNumber:
        def __init__(self, max):
            self.max = max
    
        def __iter__(self):
            self.num = 0
            return self
    
        def __next__(self):
            if(self.num >= self.max):
                raise StopIteration
            self.num += 1
            return self.num
    
    print_num = PrintNumber(3)
    
    print_num_iter = iter(print_num)
    print(next(print_num_iter))  
    print(next(print_num_iter))  
    print(next(print_num_iter)) 
```
Output:
```
    1
    2
    3
```

Notice, we have called next only 3 times, since we explicitly provided max or n as 3. If we give another call to next it will throw **StopIteration.**

**How does For loop work?**

Equipped with our knowledge of iterator and how it needs to be implemented it is very easy now to understand how exactly the for loop works. For Loop calls an **iter** method to get an iterator and then calls method **next** over and over until a **StopIteration** exception is raised, which it handles gracefully to end the loop without breaking the code. That’s all, we have debunked the behind the scene of the For loop.

Still, something feels incomplete, inefficient. To create an iterator we have to implement an **Iterator Protocol**, a class with **iter()** and **next()** methods. We also need to keep track of the internal states and handle **StopIteration** exception so that the code is not broken. It seems like lots of work. This is where **Generator** comes in.

**Generator in Python,**

Simply speaking, a **generator** is a function that returns an iterator. It is very easy to create a generator in python. We simply need to define a normal function with one change being, using the keyword **yield** instead of **return**. If a function contains at least one **yield** statement then it is a generator.

The difference between return and yield is, while return terminates a function entirely, yield temporarily halts a function while retaining its local values and later continues from there on successive calls.

Below is the generator class which iterates the number from 0 to n and yields a number that is divisible by 7

```python
    class divisible_by_7_generator:
        def __init__(self, num):
            self.num = num
        #Generator
        def get_nums_divisible_by_7(self):
            for i in range(0, self.num):
                if (i % 7 == 0):
                    yield i
       
    n = 100
    result = divisible_by_7_generator(n)
    print(f"Numbers which are divisible between 0 to {n} are:")
    for num in result.get_nums_divisible_by_7():
        print(num, end = ",")
```

Output:
```
    Numbers which are divisible between 0 to 100 are:
    0,7,14,21,28,35,42,49,56,63,70,77,84,91,98,
```

**Python Generator Expression**

A simple generator can be easily created on the fly using generator expressions. This is very similar to lambda functions with a syntax resembling to the list comprehension in python, with square brackets replaced by round parentheses. Unlike list comprehensions, generator expressions follow lazy execution, meaning producing items only when asked for and thus hugely beneficial for memory management.

Check out the below example, finding a cube of each element in a list.

```python
    myList = [1,2,3,4,5,6]
    #list Comprehension
    cubeList = [x**3 for x in myList]
    # generator expressions are surrounded by parenthesis ()
    cubeGenerator = (x**3 for x in myList)
    
    print(cubeList)
    print(cubeGenerator)
```

Output:
```
    [1, 8, 27, 64, 125, 216]
    <generator object <genexpr> at 0x0000022F5D3EBD60>
```

We can use for loop to access the values from the generator as follow,

```python
    for i in cubeGenerator: 
        print(i)
```

Output:
```
    1
    8
    27
    64
    125
    216
```

Notice, that list comprehension returns a list while the generator returns elements one at a time.

**Conclusion,**

In this article, we dug deep into the For Loop and in the process learned about very interesting and useful features such as Iterators, Generators. We also get to understand what is yield and how a generator can be implemented.

Finally, I would like to list a few frequently asked questions related to this topic in interviews:

 1. What is the difference between an Iterator and an Iterable?

 2. Can next() can be called on a string or on an iterable object?

 3. What is an Iterator Protocol?

 4. What is a Generator? What is the difference between an Iterator and a Generator?

 5. What is the Difference between Generator and a Function?

 6. What is Yield and what is the difference between Yield and Return?

If you like this article please follow me and don’t forget to like and give feedback. Every feedback will motivate me to explore more and come up with many more interesting topics in the future. All the code snippets from the article are available on my [GitHub ](https://github.com/ashutosh-vaidya/My-Writings/tree/main/Behind%20the%20Sconces%20of%20For%20Loop)repo if anyone wants to play around. You can get in touch with me on my [LinkedIn](https://www.linkedin.com/in/ashutoshvaidya11/).