Way to BETTER explain Python lambda in for loop.
Published: 2023-11-06 04:35:23

This [Python FAQ](https://docs.python.org/3.11/faq/programming.html#why-do-lambdas-defined-in-a-loop-with-different-values-all-return-the-same-result), addresses a problem new Python programmers always run into: lambda in list comprehension (in essence, in a `for` loop). However, it is poorly explained.

I tried to figure out a way to explain it by breaking down and refactoring the code just like what you would do in a math proof problem. Tell me whether this "proof" makes sense or whether "proof" makes better explanation.

Credit: Thanks to YouTube channel `mCoding` for showing the internals of Python list comprehensions!

------

`increment_by_i = [lambda x: x + i for i in range(5)]` is equalivent to:  
`increment_by_i = list(lambda x: x + i for i in range(5))`, where   
`lambda(x: x + i for i in range(10))` is a generator expression, which is equalivent to:

```python
def lambda_generator():			# Define a generator function
	for i in range(5):
		yield lambda x: x + i
generator = lambda_generator()	# Then call it
```

By the property of `lambda` in Python, this is eqivalent to:
```python
def lambda_generator():
	for i in range(5):
		def lambda_function(x):
			return x + i
		yield lambda_function
generator = lambda_generator()
```

If you run this code with the `inspcect.getclosurevars` function, you can already see from the printed output that during the definition of `lambda_generator()`, the `i` variable have already been iterated from 0 to 4. So `lambda_generator()` function becomes a function that adds 4 to its inputs.

```python
from inspect import getclosurevars

def lambda_generator():
	for i in range(5):
		def lambda_function(x):
			return x + i
		yield lambda_function
		print(getclosurevars(lambda_function))
generator = lambda_generator()

list(generator)[3](4)

# Output:
# ClosureVars(nonlocals={'i': 0}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 1}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 2}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 3}, globals={}, builtins={}, unbound=set())
# ClosureVars(nonlocals={'i': 4}, globals={}, builtins={}, unbound=set())
# 8
```

(Lemma: A small example demonstrating the underlying mechanism of `for` loops in Python)  

```python
"""
for i in range(5):
	print(i)
is equivalent to
"""

itr = iter(range(5))	# Apply `iter()` to range object to get iterator `itr`
while True:
	try:
		print(next(itr))
	except StopIteration:
		break
```

By the underlying mechanism of `for` loops (create an iterator with `iter()` function, then repeatedly apply `next()` to the iterator until `StopIteration`) in Python, `lambda_generator()` is equalivent to:

```python
def lambda_generator():
	itr = iter(range(5))	# Apply `iter()` to range object to get iterator `itr`
	while True:
		try:
			def lambda_function(x):
				return x + next(itr)	# `next(iter)` is the `i`
			print(lambda_function(0))	# Set `x` to be 0, so that the printed result = next(itr)
		except StopIteration:
			yield lambda_function
			break						# This break may not be necessary though
```

**In every `while` loop (before `StopIteration`), `next(itr)` is called, modifying the `itr` iterator outside the `while` loop (inside the `lambda_generator()` function). Therefore, it is OK to refactor the `itr` iterator inside the `while` loop:**  

```python
def lambda_generator():
	def lambda_function(x):
		itr = iter(range(4))
		result = None
		while True:
			try:
				result = x + next(itr)
			except StopIteration:
				break
		return result
	yield lambda_function
```

Now we can easily see that by the time `lambda_function()` is yielded to `lambda_generator()`, `lambda_function()` have already become a function that adds 4 to its input.

```python
def lambda_function(x):
	itr = iter(range(5))
	result = None
	while True:
		try:
			result = x + next(itr)
		except StopIteration:
			break
	return result

print(lambda_function(0))	# Set x = 0, so that the printed result = the final next(iter) value

# Output: 4
```