Filtering Lists in Python. Filtering Methods in Python | by Sadrach Pierre, Ph.D. | Towards Data Science

In this post, we will discuss three methods for list filtering in Python. Specifically, we will walk through how to use list comprehension, generator expressions and the built-in ‘filter()’ method to filter lists in python.
Let’s get started!
Suppose we have data in a list and we want to extract values or reduce the list based on some criteria. Specifically, let’s consider the following list which contains a list on medical charges with some missing values:
medical_charges = ["500", "1000", None, "450", "230", None]print(medical_charges)
To start, we can use list comprehension to filter out the ‘None’ values:
medical_charges = [n for n in medical_charges if n != None]print(medical_charges)
We can also convert the elements of the list to integers with a slight change to the list comprehension:
medical_charges = [int(n) for n in medical_charges if n != None]print(medical_charges)
Upon converting each element to an integer, we can also filter based off of the magnitude of each integer. Suppose we want to keep charges greater than or equal to $500, we can write the following list comprehension:
medical_charges = [n for n in medical_charges if n >= 500]print(medical_charges)
If we are dealing with a significant amount of data, which is often the case with medical records, we can filter iteratively using a generator expression. If we want to convert our original list using a generator expression we do the following :
charges = (int(n) for n in medical_charges if n != None)print(charges)
The main difference in syntax, from list comprehension, being the use of parentheses instead of square brackets. We can now iterate over the generator:
for charge in charges:    print(charge)
There are times when filtering criteria can’t be easily expressed using list comprehension or generator expressions. Let’s consider the example of converting our original list of string valued medical charges into integers and removing missing values:
medical_charges = ["500", "1000", None , "450", "230", None]
We can define a function that takes a list and tries to convert each element into an integer. When the conversion throws no errors, we return true. When the conversion throws a value error, we use an except statement to catch the error and return false:
def convert_and_filter(input_list):    try:        int(input_list)        return True    except ValueError:        return False
We can then use the built-in ‘filter()’ and ‘list()’ methods. The ‘filter()’function creates an iterator and the ‘list()’ method allows us to create a list of results:
charges = list(filter(convert_and_filter, medical_charges))print(charges)
I’ll stop here but feel free to play around with the examples above. For example, you can try changing the filtering conditions in the list comprehension example to only include charges less than or equal to $500.
To summarize, in this post we discussed four methods for list filtering in Python. We discussed list comprehension which is useful for its readability. We also discussed generator expressions which we use if we want to avoid generating large results from large data sets. Finally, we discussed using the built-in ‘filter()’ and ‘list()’ methods, along with a custom function, to filter lists. I hope you found this post useful/interesting. The code from this post is available on GitHub. Thank you for reading!