Python Tricks: Flattening Lists. Are you still iterating through for... | by Louis Chan | Towards Data Science

Welcome to a series of short posts each with handy Python tricks that can help you become a better Python programmer. In this blog, we will look into how to flatten lists.
We have all dealt with lists of lists or even worse: lists of nested lists.
Theoretically, we can peel the list layer by layer like an onion:
l = [0, 1, 2, [3, 4, 5, [6, 7, 8]]]from collections import Iterabledef unwrap(l):    flat_list = []    for item in l:        if isinstance(item, Iterable):            flat_list.extend(item)        else:            flat_list.append(item)    return flat_listl = unwrap(l)# [0, 1, 2, 3, 4, 5, [6, 7, 8]]l = unwrap(l)# [0, 1, 2, 3, 4, 5, 6, 7, 8]
That would also mean that we need to run unwrap for however many nested layers there is.
Is there a more efficient way to flatten nested lists?
itertools is one of Python’s standard libraries that provide handy functions that create iterators for efficient looping. Among them is a function called chain which creates a chain object that concatenates a list of iterables. A chain object can be understood as an iterator that iterates through each of the iterables. Hence, you would need to cast it into a list explicitly if you would need more than just an iterator.
from itertools import chainl = [[0, 1], [2, 3]]list(chain(*l))# [0, 1, 2, 3]list(chain.from_iterable(l))# [0, 1, 2, 3]
However, chain also comes with a couple of limitations: it concatenates a list of iterables but does not unwrap further, and it also requires all elements to be iterables:
from itertools import chainl = [[0, 1], [2, 3, [4, 5]]]list(chain(*l))# [0, 1, 2, 3, [4, 5]]l = [0, 1, [2, 3, [4, 5]]]list(chain(*l))# TypeError: 'int' object is not iterable
As an alternative to itertools.chain, pandas has a flatten function under pandas.core.common. Similar to itertools.chain, flatten returns a generator instead of a list. There are some differences in the implementation and the perks of generators vs iterators. While we will cover that in the future, a simple one-liner for their differences is that generators are iterators that use a yield statement and can only be read once.
from pandas.core.common import flattenl = [0, 1, [2, 3, [4, 5]]]list(flatten(l))# [0, 1, 2, 3, 4, 5]
From the example above, we can see that flatten also takes care of nested lists unlike chain!
So which one should we use?
It depends.
If you have a list of nested lists, either refactor the code to avoid creating that beast or using flatten to completely flatten it in one go.
If you have a list of lists, then chain would suffice. After all, chain is also faster than flatten
l = [[0, 1], [2, 3, 4], [5, 6, 7, 8]]%timeit list(chain(l))# 236 ns ± 7.68 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)%timeit list(flatten(l))# 5.13 μs ± 164 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
That’s about it for this blog post! I hope you have found this useful. If you are interested in other Python tricks, I have put together a list of these short blogs for you:
Python Tricks: How to Check Table Merging with Pandas
Python Tricks: Simplifying If Statements & Boolean Evaluation
Python Tricks: Check Multiple Variables against Single Value
If you want to read more about Python, Data Science, or Machine Learning, you may want to check out these posts:
7 Easy Ways for Improving Your Data Science Workflow
Efficient Conditional Logic on Pandas DataFrames
Memory Efficiency of Common Python Data Structures
Parallelism with Python
Essential Jupyter Extension for Data Science Set Up
Efficient Root Searching Algorithms in Python
If you would like to read more about how to apply machine learning to trading & investing, here are some other posts that may be of interest:
Genetic Algorithm for Trading Strategy Optimization in Python
Genetic Algorithm — Stop Overfitting Trading Strategies
ANN Recommendation System for Stock Selection