Understanding Generator Expressions In Python | by Richmond Alake | Towards Data Science

This article is an introduction to generator expressions(Genexps) within the Python programming language.
This article is aimed at developers of all levels. If you’re a beginner, you can pick up new concepts such as generator expressions, list comprehensions(listcomps) and sequence type generations. Intermediate developers can learn a thing or two on scalability and memory efficiency, or simply use this article as a refresher.
Within this article, you will find the following:
Description of generator expression
How to utilize generator expressions (Code)
Advantages of generator expressions to other similar solutions
Memory and time measurements of genexps and listcomps
Genexps are elegant and memory-efficient solutions to generating sequence types such as arrays, tuples, collections within python.
Generator expressions are comparable to list comprehensions(listcomps) — another means of constructing the list sequence type within python. Genexps and listcomps share similarities in the way they are implemented.
The shortcode snippet below depicts their syntactic similarity:
In the code below, generator expression used to calculate the sum of a range of values that are incremented by an integer.
#Generator Expressionaccumulated_gexp = sum((1 + x for x in range(2000000)))print(accumulated_gexp)>> 2000001000000#List Comprehensionaccumulated_listcomp = sum([1 + x for x in range(2000000)])print(accumulated_listcomp)>>2000001000000
Although subtle, the main difference between the genexp and listcomp examples above is that the genexp start and end with brackets, while the listcomp starts and ends with square brackets — a better term would be ‘parenthesis’.
Here is another example of using generator expressions to create a tuple.
beginning_topic = ['Machine', 'Deep', 'Reinforcement']ending_topic = 'Learning'tuple(print(beginning + " " + ending_topic) for beginning in beginning_topic)>> Machine Learning>> Deep Learning>> Reinforcement Learning
Generation expressions are memory-efficient in comparison to listcomps. Genexps memory efficiency is a result of the utilization of the python iterator protocol to ‘yield’ or return the items within the iterator. In contrast, list comprehensions utilize memory for a generated list, and it’s content.
A generator will only yield items within its iterator when required, hence giving genexps memory-efficient characteristics.
If what is written above is not clear enough, the code snippet below will make it all more apparent. The snippet below shows the memory size requirement (in bytes) of genexps and listcomps examples.
from sys import getsizeofaccumulated_gexp = (1 + x for x in range(2000000))print(type(accumulated_gexp))print(getsizeof(accumulated_gexp))>> <class 'generator'>>> 112accumulated_listcomp = [1 + x for x in range(2000000)]print(type(accumulated_listcomp))print(getsizeof(accumulated_listcomp))>> <class 'list'>>> 17632624
In the code snippet above, we can observe that the list comprehension uses 17632624 bytes; whereas the generator expression utilized a measly 112 bytes in memory.
It’s also possible to access the content of the sequences by using the next function of the iterator for the generator expression and list indexing for the list comprehension.
print(next(accumulated_gexp))>> 1print(accumulated_listcomp[0])>> 1
Another advantage of generator expressions is their time efficiency characteristics when compared to list comprehensions.
For many developers out there, especially beginners, you are more familiar and exposed to the utilization of list comprehension.
But in most cases, the benefits of using generator expressions are not so easy to ignore, especially when the speed of program execution is of great importance.
In practical scenarios, when you are iterating over a sequence once, you’ll most likely be better off utilizing a genexps. For more flexibility and multiple iterations of sequences, you are likely to utilize listcomps.
The code snippet below demonstrates the execution time differences between listcomps and genexps.
import timeitgenerator_exp_time = timeit.timeit('''accumulated_gexp = (1 + x for x in range(200))''', number=1000000)print(generator_exp_time)>> 1.5132575110037578list_comp_time = timeit.timeit('''accumulated_listcomp = [1 + x for x in range(200)]''', number=1000000)print(list_comp_time)>> 29.604462443996454
Using the timeit python module, we can measure the execution time of lines of code, just as seen above.
As we can observe, the generator expression takes less than two seconds (generator_exp_time: 1.51...), and the list comprehension execution time is almost 20x more.
Here is a summary of the advantages of generation expressions within python:
Memory efficient method of generating sequence types in python.
Adds further brevity and readability to written code. Generator expressions are generator functions shortened.
Time-efficient when compared to list comparisons.
Generator expressions aren’t complicated at all, and they make python written code efficient and scalable.
For beginners, learning when to use list comprehensions and generator expressions is an excellent concept to grasp early on in your career.
To connect with me or find more content similar to this article, do the following:
Subscribe to my YouTube channel for video contents coming soon hereFollow me on MediumConnect and reach me on LinkedIn
Subscribe to my YouTube channel for video contents coming soon here
Follow me on Medium
Connect and reach me on LinkedIn