Conditional Selection and Assignment With .loc in Pandas | by Byron Dolon | Towards Data Science

There are many different ways to select data in Pandas, but some methods work better than others. In this piece, we’ll go over how to edit your DataFrames based on conditional statements using the .loc method.
If you’ve been working with Pandas for a while now, you may already have come across the dreaded “SettingwithCopyWarning” message when you run your code. It may seem scary at first (trust me, I’ve been there), but all it’s telling you is that you are probably trying to assign a value to a copy of a Pandas object when in reality you want to be editing the actual value.
For some operations, you can get around this warning simply by adding the inplace=True parameter to whatever function you’re running. When you’re working with conditional selection, however, it’s worth going over a few examples to understand how to make your changes stick properly.
Examples in this piece will use some old Tesla stock price data from Yahoo Finance. Feel free to run the code below if you want to follow along. For more information on pd.read_html and df.sort_values, check out the links at the end of this piece.
import pandas as pddf = pd.read_html("https://finance.yahoo.com/quote/TSLA/history?period1=1546300800&period2=1550275200&interval=1d&filter=history&frequency=1d")[0]df = df.head(11).sort_values(by='Date')df = df.astype({"Open":'float',                "High":'float',                "Low":'float',                "Close*":'float',                "Adj Close**":'float',                "Volume":'float'})df['Gain'] = df['Close*'] - df['Open']
If you haven’t worked with .loc in the past at all, check out this piece for some simple examples. Otherwise, let’s dive straight in!
First, let’s just try to grab all rows in our DataFrame that match one condition. In this example, I’d just like to get all the rows that occur after a certain date, so we’ll run the following code below:
df1 = df.loc[df['Date'] > 'Feb 06, 2019']
And that’s all!
.loc allows you to set a condition and the result will be a DataFrame that contains only the rows that match that condition. Now that we understand the basic syntax, let’s move on to a slightly more interesting example.
Now, we’ll introduce the syntax that allows you to specify which columns you want .loc to return. In this case, we’ll use the same conditional statement as before to filter out specific dates. However, our goal this time is to only select two columns (Date and Open) from the original DataFrame. To do so, we run the following code:
df2 = df.loc[df['Date'] > 'Feb 06, 2019', ['Date','Open']]
As you can see, after the conditional statement .loc, we simply pass a list of the columns we would like to find in the original DataFrame. The resulting DataFrame gives us only the Date and Open columns for rows with a Date value greater than February 6, 2019.
But what if we wanted to filter by multiple conditions? Let’s keep going.
If you have two or more conditions you would like to use to get a very specific subset of your data, .loc allows you to do that very easily. In our case, let’s take the rows that not only occur after a specific date but also have an Open value greater than a specific value. To do so, we run the following:
df3 = df.loc[(df['Date'] > 'Feb 06, 2019') & (df['Open'] > 62), ['Date', 'Open']]
As you can see, we’ve simply wrapped added another conditional by including the & sign to indicate that we want both conditions to be fulfilled (note that the | sign will also work for “or”). We’ve also wrapped each conditional statement in parentheses for a clean look. The result is a very small subset of the original DataFrame with only the rows that meet our two conditions.
Now that we’ve gone over all the components, we’re ready to make changes to our DataFrame!
As mentioned before, there may be other ways to do this, but you might end up with a “SettingwithCopyWarning” if you’re not careful. Using .loc to assign values will take care of this issue for you!
We’ll put together all the previous steps and edit our DataFrame so that rows that meet a condition that we set will be assigned a specific value. In this case, we’ll create a new “Remarkable” column, which will include rows that either has a very high Volume or a positive Gain. To do so, we run the following code:
remarkable_filter = (df['Volume'] > 30000000) | (df['Gain'] > 0)df4 = df.copy()df4['Remarkable'] = ''df4.loc[remarkable_filter, ['Remarkable']] = Truedf4.loc[~remarkable_filter, ['Remarkable']] = False
For clarity, we put our conditional statements in a separate variable, which is used later in .loc. Then, we assign either True to the Remarkable column for all the rows that meet our conditional statements. We use the ~ symbol to find all the rows that don’t meet our conditional statement and then assign False to the Remarkable column for those rows.
And that’s all!
I hope you found this useful in further understanding .loc and how you can use it to filter and edit your DataFrames in Pandas! This way, you’ll also be safe from the “SettingwithCopyWarning”, because all we’re doing is following the warning’s instructions:
Try using .loc[row_indexer, col_indexer] = value instead
Best of luck on your Pandas adventures!
More by me:- Don’t Miss Out on Rolling Window Functions in Pandas- 4 Different Ways to Efficiently Sort a Pandas DataFrame- Top 4 Repositories on GitHub to Learn Pandas- How to Quickly Create and Unpack Lists with Pandas- Learning to Forecast With Tableau in 5 Minutes Or Less