5 Steps To Understanding Generators In Python
By: Kurt F.
Tags: Programming, Python Programming, Python, Software Development, Software Engineering

1. What are generators?

Generators are a special type of function to temporarily store all codes that generate information in memory.

Generally functional; Instead of a huge area stored in memory, it is being able to use it in a more rational way. It does this by making it easier to create iterators. And as a result, a series of results can be produced instead of a single value, thanks to the generator function.

It can be said that generators produce a stream of values. To better understand this, we can think of a simple machine that sequentially reads the code and executes the work process with a command that gives us the requested information.

Image created by the author

2. When is it used?

It should be noted that generators are also a repetitive. I find this important to reduce more processing time when we want to do multiple operations. Generators can achieve this when we want two significant different program structures to be expressed in a common language in multi-variable or list readings.

Basically, speeding up the function is directly proportional to code simplification. And generators are a good alternative for potential problems like cumbersome and slow. In projects related to big data where it will be important to generate a single value from a defined array, it creates operational comfort by reducing memory usage for detailed analysis, and such projects are where its actual usage purpose is well observed.

3. What are the advantages and disadvantages?

Advantages

Generators are experts at splitting multiple processes at once. It is effective in analyzing big data for quick print out, performing multiple scans and resolving the appropriate changes quickly. It makes many data analysis more flexible by easing the memory load.

Disadvantages

In fact, the things that seem disadvantageous are the issues that need to be considered while using it. Besides, we cannot perform indexing and slicing operations with it as in the lists. Also you can use a generator once. This can typically be an important factor when iterators are needed. And yes that’s why those can be called lazy: they only yield values when explicitly asked.

When operations continue forward in a generator, the previous operation is discarded, and when the entire generator is completed, all is discarded from memory. For those who are not used to this type of operation, using it in a wrong project will not mean taking advantage of it.

4. How is it applied?

First of all, it is useful to underline two new concepts:

next(): It is the command we specified to call a generator after we create it. Also the call to .next() is necessary to start execution of our generator.

yield: yield is statement in Python for generators that is used to return from a function without destroying. In fact, yield is our term that processes the generator to function.

Unlike a normal function, the yield statement replaces the generation return in definition(def). And it returns the result different than return. For example, check out the code blog below:

And there are two different ways to create a generator:

Generator “function”

Generator “expression”

Generator “function”

We know that we have a query inside a function we defined in the ‘for’ loop. With next (), the next element in the loop is queried piece by piece. If we continue by adding to the above statement, a multi-defined function will be formed as follows:

But there is a way to freeze the iterate in the first query. And in this way, an action can be done after StopIteration. For this we use multi_generate() directly in next(). So calling a bracketed format inside the parentheses will only show the result as the first digit output:

Let’s also try to understand how we use generators to read files.

For this, we will first designate our file as our generator function that will distribute line by line. We set it as “FileGenerator”.

We used one of the Covid-19 data sets for the file we wanted to read line by line. “covid19_Confirmed_dataset.csv”

We will use the open() function to open a file. Here we will select the one that matches the Latin alphabet for the encoding ISO-8859–1 data in brackets. In that case:

So we were able to read the first three lines of our file separately with the next() command.

Generator “expression”

Let’s try to get the sum of the exponential products of the numbers 1 through 10 for example to create a generator object. It is possible to do this with the following statement.

I should mention that the difference of the variable from the lists is the use of square brackets.

If we activate the sum () function with this variable we created, we get the total result as follows:

5. Conclusion

The performance of generators in terms of big data efficiency is still valid today. It is possible to work in architectures where more than one generator is built by connecting, in terms of the advantages it provides. I expect more python developers to be included in the toolbox now and beyond. However, I also know that it will be difficult to learn how to create piplines with generators if establishing and managing a more cumbersome system is defined as an indispensable kind of ritual. Sometimes it may be necessary to dive. Especially if the data tell you this!