How to Fix ValueError: The truth value of a Series is ambiguous in Pandas | Towards Data Science

One of the most commonly reported error in pandas is
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()
and it may sometimes be quite tricky to deal with, especially if you are new to pandas library (or even Python).
In today’s article, we are going to understand why and when this error is being raised in the first place and additionally showcase how to get rid of it. Specifically, we will discuss how to deal with this ValueError by using
Python’s bitwise operators, or
NumPy’s logical operators methods
Before getting into the details, let’s reproduce the error using an example that we’ll also reference throughout this article in order to demonstrate a few concepts that will eventually help us understand the actual error and how to get rid of it.
Let’s get started and create an example DataFrame in pandas.
import pandas as pddf = pd.DataFrame(    [        (1, 531, True, 12.8),        (2, 126, False, 74.2),        (3, 175, False, 1.1),        (4, 127, True, 45.8),        (5, 543, True, 21.1),        (6, 254, False, 98.1),    ],    columns=['colA', 'colB', 'colC', 'colD'])print(df)   colA  colB   colC  colD0     1   531   True  12.81     2   126  False  74.22     3   175  False   1.13     4   127   True  45.84     5   543   True  21.15     6   254  False  98.1
Now let’s assume that we want to filter our pandas DataFrame using a couple of logical conditions. Say we want to keep only the rows whose values in column colB are greater than 200 and values in column colD are less or equal to 50.
df = df[(df['colB'] > 200) and (df['colD'] <= 50)]
The above expression will fail with the following error:
File "/usr/local/lib/python3.7/site-packages/pandas/core/generic.py", line 1555, in __nonzero__  self.__class__.__name__ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
The error is raised because you chain multiple conditions using logical operators (such as and, or, not) resulting in ambiguous logic since the returned results are column-based for each individual condition specified.
In other words, the error is telling you that you are attempting to fetch the boolean value of a pandas Series object. To put this into a more simple context, consider the expression below, that once again will raise this particular error:
>>> import pandas as pd>>> s = pd.Series([1, 2, 3])>>> s0    11    22    3dtype: int64>>> bool(s)Traceback (most recent call last):  File "/usr/local/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code    exec(code_obj, self.user_global_ns, self.user_ns)  File "<ipython-input-5-68e48e81da14>", line 1, in <module>    bool(s)  File "/usr/local/lib/python3.7/site-packages/pandas/core/generic.py", line 1555, in __nonzero__    self.__class__.__name__ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().
When multiple conditions are specified and chained together using logical operators, each individual operand is implicitly turned into a bool object, resulting into the error in question.
Now in order to fix this error, the first option you have is to use Python bitwise operators.
Operator |   Meaning----------------------   &     | Bitwise AND   |     | Bitwise OR   ~     | Bitwise NOT   ^     | Bitwise XOR
Now the expression should work as expected and no ValueError will be raised:
df = df[(df['colB'] > 200) & (df['colD'] <= 50)]print(df)   colA  colB  colC  colD0     1   531  True  12.84     5   543  True  21.1
Alternatively, you can use NumPy’s logical operator methods that compute the truth values element-wise and thus the truth values won’t be ambiguous.
Operator | Method-----------------AND      | numpy.logical_andOR       | numpy.logical_orNOT      | numpy.logical_not XOR      | numpy.logical_xor
In our example, numpy.logical_and method should do the trick:
import numpy as npdf = df[np.logical_and(df['colB'] > 200, df['colD'] <= 50)]print(df)   colA  colB  colC  colD0     1   531  True  12.84     5   543  True  21.1
In today’s guide we discussed about one of the most commonly reported errors in pandas and Python, namely ValueError: The truth value of a Series is ambiguous.
We reproduced the error in an attempt to better understand why the error is raised in the first place and additionally, we discussed how to deal with it using Python’s bitwise operators or NumPy’s logical operators methods.
Become a member and read every story on Medium. Your membership fee directly supports me and other writers you read. You’ll also get full access to every story on Medium.
You may also like