Comprehending the ‘Comprehensions’ in Python | by Parul Pandey | Towards Data Science

“Good code is its own best documentation” — Steve McConnell
In my last article, I explained the concept of Lambda, Map, Filter, and Reduce functions in Python, which are based on the Functional Programming Paradigm. I also touched upon the idea of List Comprehensions, which are considered as a substitute for the lambda function. This entire article covers the essentials of comprehensions and their various forms, in detail.
Comprehensions are constructs that allow sequences to be built from other sequences. Python 2.0 introduced us to the concept of list comprehensions, and Python 3.0 took it further by including the dictionary and set comprehensions.
Why are the comprehensions so powerful? We shall try and understand this with an example. We all know Python offers various ways to express a list. For instance:
One can either explicitly write the whole things as:
squares = [0, 1, 4, 9,16,25]
Or, write a for loop to create a list:
squares = []for num in range(6):    squares.append(num*num)
Another way to create a list is by using a single line of code.
squares = [num*num for num in range(6)]
This one-liner above is called a List Comprehension and is a convenient way to create a list. It removes the dependence on loops and makes the code compact. The next section further goes deeper into the concept of the list and other types comprehensions provided in Python 3.
List comprehension is a way to define and create lists in Python in a concise way. In most cases, list comprehensions let us create lists in a single line of code without worrying about initializing lists or setting up loops.
A list comprehension consists of the following parts:
Say, we need to find the squares of the first five even numbers. As seen in the previous section, there are two ways to do this: with an explicit for-loop or with a list comprehension. Let’s try with both.
Using For Loop
even_squares = []>>> for num in range(11):...    if num%2 == 0:...        even_squares.append(num * num)>>> even_squares[0, 4, 16, 36, 64, 100]
Using List Comprehensions
even_squares = [num * num for num in range(11) if num%2 == 0]even_squares[0, 4, 16, 36, 64, 100]
If we look closely, it can be seen that a list comprehension can be created by merely rearranging a For loop.
List comprehensions are Python’s way of implementing the notation for sets as used in mathematics.
Let’s try and see some more examples of creating lists with the help of List Comprehensions.
A Pythagorean triple consists of three positive integers a, b, and c, such that a2 + b2 = c2. We usually write such a triplet as (a,b,c), for instance (3,4,5).
[(a,b,c) for a in range(1,30) for b in range(1,30) for c in range(1,30)if a**2 + b**2 == c**2][(3, 4, 5), (4, 3, 5), (5, 12, 13), (6, 8, 10), (7, 24, 25), (8, 6, 10), (8, 15, 17), (9, 12, 15), (10, 24, 26), (12, 5, 13), (12, 9, 15), (12, 16, 20), (15, 8, 17), (15, 20, 25),(16, 12, 20), (20, 15, 25),(20, 21, 29), (21, 20, 29), (24, 7, 25), (24, 10, 26)]
Converting lowercase letters in a string to uppercase.
colors = ["pink", "white", "blue", "black", purple"][color.upper() for color in colors]['RED', 'GREEN', 'BLUE', 'PURPLE']
Swapping the first and the last name in a given list.
presidents_usa = ["George Washington", "John Adams","Thomas Jefferson","James Madison","James Monroe","John Adams","Andrew Jackson"]split_names = [name.split(" ") for name in presidents_usa]swapped_list = [split_name[1] + " " + split_name[0] for split_name in split_names]swapped_list['Washington George', 'Adams John', 'Jefferson Thomas', 'Madison James', 'Monroe James', 'Adams John', 'Jackson Andrew']
If the expression contains a tuple (e.g. the (x, y) , it must be parenthesized.
# Convert height from cms to feet using List Comprehension : 1 cm = 0.0328 feetheight_in_cms = [('Tom',183),('Daisy',171),('Margaret',179),('Michael',190),('Nick',165)]height_in_feet = [(height[0],round(height[1]*0.0328,1)) for height in height_in_cms]height_in_feet[('Tom', 6.0), ('Daisy', 5.6), ('Margaret', 5.9), ('Michael', 6.2), ('Nick', 5.4)]
List comprehensions can also be nested to create complicated lists. For instance, we can create a matrix using only list comprehensions.
Creating a 3X3 Matrix
matrix = [[j * j+i for j in range(3)] for i in range(3)]matrix[[0, 1, 4], [1, 2, 5], [2, 3, 6]]
A set comprehension is similar to a list comprehension but returns a set instead of a list. The syntax is slightly different in the sense that we use curly brackets instead of square brackets to create a set.
Consider the following list that consists of names of people:
names = [ 'Arnold', 'BILL', 'alice', 'arnold', 'MARY', 'J', 'BIll' ,'maRy']
The list includes a lot of duplicates, and there are names with only a single letter. What we want is a list that consists of names that are longer than one letter and have only the first letter capitalized. To accomplish such a task, we turn to set comprehensions.
{name.capitalize() for name in names if len(name) > 1}{'Alice', 'Arnold', 'Bill', 'Mary'}
Dictionary comprehensions are used when the input is in the form of a dictionary or a key: value pair. For instance, consider a dictionary where the keys represent characters, and the values denote the number of times these characters appear in a corpus.
char_dict = {'A' : 4,'z': 2, 'D' : 8, 'a': 5, 'Z' : 10 }
The dictionary char_dict consists of a mix of upper and lowercase letters. We want to count the total number of occurrence of the letters irrespective of their case. Let’s use a dictionary comprehension to achieve this:
{ k.lower() : char_dict.get(k.lower(), 0) + char_dict.get(k.upper(), 0) for k in char_dict.keys()}{'a': 9, 'z': 12, 'd': 8}
List comprehensions are to lists, as generator expressions are to generators. Generator functions output values one-at-a-time from a given sequence instead of giving them all at once. Here is a nice article which explains the nitty-gritty of Generators in Python.
www.dataquest.io
The syntax and the way of working of generator expressions are precisely like a list comprehension except that they use round brackets instead of square ones. Let’s say we want to calculate the sum of squares of the first ten natural numbers.
# Sum of first ten natural numbers using List Comprehensionssum([num**2 for num in range(11)])385
The result would have been the same had we used any other iterable and not necessarily a list.
sum({num**2 for num in range(11)})385
Now, if we use a generator expression to calculate the squares of the first ten natural numbers, it would be something like this:
squares = (num**2 for num in range(11))squaressquares<generator object <genexpr> at 0x1159536d8>
Unlike list comprehensions, a generator expression doesn’t return a list but a generator object. To get the result, we can use the above expression with the sum function.
sum(n ** 2 for n in numbers)385
See how we got rid of the redundant parenthesis in the expression above, making the code more efficient.
List comprehensions are an effective way to reduce the length of the code. They also make the code more readable. But there are instances when we can comfortably do without them.
It is not advisable to use comprehensions when the logic of your program is too long. The main idea of using comprehensions is to shorten the code. However, when we start packing too much of code into a single statement, we tend to compromise with the code’s readability. A for loop is a better idea in such situations.
List Comprehensions Overview