List Comprehensions
Tags: python, datascience, machinelearning
Published: 2020-10-30 17:18:02

**Edit**: 1.11.2020 - Add scenarios where list comprehensions replace `map` and `filter` functions. 

[Previously](https://dev.to/paulapivat/control-flow-truthiness-17gg), we saw **if-statements** expressed in one-line, for example:

```python
y = []

# Falsy
print("Truthy") if y else print("Falsy")
```

We can also write **for-loops** in one-line. And thats a way to think about `list comprehensions`. 

```python

# traditional for-loop; [0, 2, 4]
num = []
for x in range(5):
    if x % 2 == 0:
        num.append(x)

num # call num

# list comprehension, provides the same thing
# [0, 2, 4]
[x for x in range(5) if x % 2 == 0]
```

Here are some examples from Data Science from Scratch:

```python
# [0, 2, 4]
even_numbers = [x for x in range(5) if x % 2 == 0] 

# [0, 1, 4, 9, 16]
squares = [x * x for x in range(5)]

# [0, 4, 16]
even_squares = [x * x for x in even_numbers]
```
[Dan Bader provides](https://dbader.org/blog/list-dict-set-comprehensions-in-python) a helpful way to conceptualizing `list comprehensions`:

```python
(values) = [ (expression) for (item) in (collections) ]
```
A good way to understand `list comprehensions` is to de-construct it back to a regular for-loop:

```python
# recreation of even_numbers; [0, 2, 4]
even_bracket = []
for x in range(5):
    if x % 2 == 0:
       even_bracket.append(x)
       
# recreation of squares; [0, 1, 4, 9, 16]
square_bracket = []
for x in range(5):
    square_bracket.append(x * x)

# recreate even_squares; [0, 4, 16]
square_even_bracket = []
for x in even_bracket:
    square_even_bracket.append(x * x)

```

Moreover, list comprehensions also allow for **filtering with conditions**. Again, we can understand this with a brief comparison with the for-loop.

```python
# traditional for-loop
filtered_bracket = []

for x in range(10):
    if x > 5:
        filtered_bracket.append(x * x)
        
# list comprehension
filtered_comprehension = [x * x
                          for x in range(10)
                          if x > 5]

```
The key take-away here is that `list comprehensions` follow a pattern. Knowing this allows us to better understand how they work.

```python
values = [expression
          for item in collection
          if condition]
```

Python also supports dictionaries or sets comprehension, although we'll have to revisit this post as to **why** we would want to do this in a data wrangling, transformation or analysis context.

```python
# {0: 0, 1: 1, 2: 4, 3: 9, 4: 16}
square_dict = {x: x * x for x in range(5)}

# {1}
square_set = {x * x for x in [1,-1]}
```

Finally, comprehensions can include nested for-loops:

```python
pairs = [(x,y)
         for x in range(10)
         for y in range(10)]
```

We will expect to use `list comprehensions` often, so we'll revisit this section as we see more applications in context. 

#### Map, Filter, Reduce, Partial

In the first edition of this book the author introduced these functions, but has since reached enlightenment 🧘, he states:

"On my journey toward enlightenment I have realized that these functions (i.e., map, filter, reduce, partial) are best avoided, and their uses in the book have been replaced with list comprehensions, for loops and other, more Pythonic constructs." (p.36)

He's being facetious, but I was intrigued anyways. So here's an example replacing **map** with **list comprehensions**.

```python
# create list of names
names = ['Russel', 'Kareem', 'Jordan', 'James']

# use map function to loop over names and apply an anonymous function
greeted = map(lambda x: 'Hi ' + x, names)

# map returns an iterator (see also lazy evaluation)
print(greeted) # <map object at 0x7fc667c81f40>

# because lazy evaluation, won't do anything unless iterate over it
for name in greeted:
     print(name)

#Hi Russel
#Hi Kareem
#Hi Jordan
#Hi James

## List Comprehension way to do this operation
greeted2 = ['Hi ' + name for name in names]

# non-lazy evaluation (or eager)
print(greeted2) # ['Hi Russel', 'Hi Kareem', 'Hi Jordan', 'Hi James']
```
Here's another example replacing **filter** with **list comprehensions**:

```python
# create list of integers
numbers = [13, 4, 18, 35]

# filter creates an interator
div_by_5 = filter(lambda num: num % 5 == 0, numbers)

print(div_by_5) # <filter object at 0x7fc667c9ad30>
print(list(div_by_5)) # must convert iterator into a list - [35]

# using list comprehension to achieve the same thing
another_div_by_5 = [num for num in numbers if num % 5 == 0]

# lists do not use lazy evaluation, so it will print out immediately
print(another_div_by_5) # [35]

```
In both cases, it seems `list comprehensions` is more pythonic and efficient.