How to use map, filter and reduce in Python
Tags: python, functional, tutorial, beginners
Published: 2020-11-07 14:23:11

This is part of my Functional Programming post series.

Demonstration of how to use `map`, `filter`, `reduce`.

These functions all built into Python. There are actually not included originally but a contributor with a background in a Functional Programming (Lisp, I think) missed them and added them.

## Map

Using a named function.

```python
def square(value):
    return value**2


a = [1, 2, 3]
b = map(square, a)
list(b)
# [1, 4, 9]
```

Skip to [Consuming a generator](#consuming-a-generator) section to understand by `list` is needed.


Using a `lambda`, i.e. an anonymous function. This typically uses `x` as the variable.

```python  
a = [1, 2, 3]
b = map(lambda x: x**2, a)
list(b)
# [1, 4, 9]
```

For the next two sections, I'll just use the `lambda` style.

## Filter

```python
a = [1, 2, 3]
b = filter(lambda x: x > 2, a)
list(b)
# [3]
```

Combining map and filter. Note the _innermost_ call is evaluated first - in this case, the `map` call.

```python
a = [1, 2, 3]
b = filter(lambda x: x > 1, map(lambda x: x**2, a))
b
# [4, 9]
```

## Reduce

Use `reduce` to do calculations on an list which accumulate along the way and output as a **single** value like a `str` or `int`, not a `list`.

Here we add all the values together (pretending that `sum` doesn't exist).

In Python 3, you have to **import** `reduce`. But `map` and `filter` don't need an import.

```python
from functools import reduce


a = [1, 2, 3]
b = reduce(lambda x, y: x+y, a, 0)
b
# 6
```

The `lambda` has two variables - we use `x` for the value from the _previous_ iteration and `y` for the _current_ item. We use `0` as previous value on the first iteration.

Breaking down what happens in each iteration:

```python
# prev  current
0     + 1
# result: 1

# prev  current
1     + 2
# result: 3

# prev  current
2     + 3 
# result: 6
```

We already have the `sum` function in Python, so what we did above is not novel.

Next, we multiply all values together in a list i.e. `1*5*6*7`, using `1` as the starting value.

```python
a = [5, 6, 7]
b = reduce(lambda x, y: x*y, a, 1)
b
# 210
```

## Consuming a generator

Note that in Python 3 that `map` and `filter` are generators now. This is more efficient from a memory perspective but requires you to use `list` or a `for` loop to consume the values in the generator.

```python
b = map(square, a)

# Show a reference to the object which has computed nothing.
b
# <map object at 0x1019aa310>

# Consume all of the values.
c = list(b)
c
# [1, 4, 9]

# The generator is now empty.
list(b)
# []

# Calculate it over.
b = map(square, a)

# Demonstration with a for loop instead of list(b).
for i in b:
    print(i)
# 1
# 4
# 9
```

## Scoping and safety

One more thing to note about passing a value, to `map`, `reduce` or `filter` is that an actual value is in to the call, not just a reference. This is like closures in JavaScript, where you would use the `let` keyword.

Here is a standard flow.

```python
a = [1, 2, 3]
b = map(lambda x: x + 1, a)
list(b)
# [2, 3, 4]
```

But here, we change `a` in between defining `b` and consuming the `map`. Yet we get the same result for `b`.

```python
a = [1, 2, 3]
b = map(lambda x: x + 1, a)
a = [9, 10]
list(b)
# [2, 3, 4]
```

What happened is that we only reassigned `a` to point to `[9, 10]`. The old value `[1, 2, 3]` still exists in the scope of the uncalled `b` call, just without a variable name assigned to it.

This is good for Functional Programming principle. The unconsumed `b` call has **state** in the sense that is has a value passed into it that it knows about. But the uncalled `b` does **not** depend on an external value of `a` which could change.

Think how unpredictable it would be if in the middle of the `b` map computing values in `a`, then `a` pointed somewhere else or had a value changed or even **changed size**.


Python doesn't protect you from changing the size of a list when iterating over it... this `for` will **never** get to the last element as each time it iterates it adds one to the end.

```python
a = [1, 2, 3]
for x in a:
    a.append(x+1)
```
I had to cancel the command after a few seconds and saw that the list had ballooned in size.

```python
a[:20]
# [1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6, 5, 6, 7, 6, 7, 8, 7, 8]
# len(a)
14968136
```

Fortunately, Python does protect you when it comes to updating a dictionary when iterating over it:

```python
c = {'A': 1}
for k in c:
    c['B'] = k
```
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: dictionary changed size during iteration
```


## Links

Interested in more of my writing on Python? I have 3 main places where I collect Python material that is useful to me.

- [Python topic on Learn to Code](https://github.com/MichaelCurrin/learn-to-code/tree/master/en/topics/scripting_languages/Python) - good for beginners but the resources list will useful for all levels.
- [Python Cheatsheets](https://michaelcurrin.github.io/dev-cheatsheets/cheatsheets/python/)
- [Python Recipes](https://michaelcurrin.github.io/code-cookbook/recipes/python/)

Here are all my [Python repos on GitHub](https://github.com/MichaelCurrin?tab=repositories&q=&type=&language=python) if you want to see what I like to build.