Easily Select Data Python Pandas with loc & iloc | Towards Data Science

Pandas library of Python is very powerful library for the manipulation of tabular data. In Python, tabular data is stored in Pandas DataFrame. Every now and then, we need to select certain part of the data for our use.
Feel free to join my email list for Free to get notified whenever I write a new story!! ⚡
The loc and iloc are essential Pandas methods used for filtering, selecting, and manipulating data. They are quick, fast, easy to read, and sometimes interchangeable.
In this article, we’ll explore how to perform data selection with loc and iloc . We will go over the following topics:
· Difference between loc() and iloc() in Pandas DataFrame· Select Multiple Rows and Columns· Slicing Rows· Extract Subset of Pandas DataFrame based on Conditions· Create a New Column based on 1 condition· Create a New Column based on Multiple Conditions
Let’s use the solar power plants data available on data.world and start with reading the data in Pandas DataFrame with read_excel(). For convenience, I changed a column name as well.
import pandas as pdurl = 'https://query.data.world/s/xgoscc46yofwpqz6qzquyf6kd7bo33'df = pd.read_excel(url)df.rename(columns=          {'MWp (Megawatts Peak Capacity)': 'MWp_Capacity'}          ,inplace=True)df.head()
This is a small 20 rows x 7 columns dataset. This is an open source data from makeovermonday.co.uk which is free to share and use for project. Source for the data is Wikipedia. https://www.makeovermonday.co.uk/data/. The data was uploaded to data.world by Andy Kriebel.
Both are used in slicing of data i.e. extracting subset of data from the Pandas DataFrame. But there is a key difference in which they access the data as shown in below picture!
Lets get the first 5 rows from the Name column of the DataFrame.
Missed something?? 🚩
Another key difference is, .iloc() method does not include the last element of the range passed in it unlike .loc()
To select multiple rows and column, we can pass the list of their indices to .iloc() or list of their labels to .loc()
for example, selecting 3rd, 7th,10th row and columns Country and Year from the dataset.
df.loc[[3,7,10], ["Country", "Year"]]
The same output can be obtained with df.iloc() by replacing column names Country and Year by their indices as 1 and 6 respectively.
Another example, to get rows 10 till 15 and columns 3 to 5. This can be solved using .iloc(), as we are directly interested in the rows and columns at specific indices.
df.iloc[9:15, 2:5]
The most robust and consistent way of slicing rows is using .iloc() ⚡
for example, first three rows can be selected by simply writing
df.iloc[:3]
As we did not pass any indices for columns, all the columns from the DataFrame are selected.
Certainly, this can be achieved by writing only df[:3].
But how about selecting 1st, 7th and 8th row ❗❗
In that case df[0,6,7] will throw an KeyError but df.iloc[[0,6,7]] will do the job, as seen below.
🚩 Remember, in Python indexing starts at 0 i.e. the first row or the first column will always have index 0 unless it is specifically changed.
Also, it is possible to select and extract the alternate rows such as 1st, 3rd, 5th and so on.
df.iloc[1::2]
Ultimately you need to pass the row indices to .iloc() method.
Often, we need to extract the subset of DataFrame based on one or more conditions. Using .loc() this task can be easily done. ⚡
If it is one condition, we can pass the condition to .loc() method as shown below.
df.loc[df["Acres"]>5000]
If there are multiple conditions from a single column or even from multiple columns, simply specify them within .loc() separated by & for logical and and by | for logical or.
🚩 Don’t forget to enclose each condition within parenthesis ( )
For instance, selecting subset of the data where either values in Acres are greater than 5000 or values in MWp_Capacity column are more than 2000.
df.loc[(df["Acres"]>5000) | (df["MWp_Capacity"]>2000)]
This was an example of logical or . similarly subset can be extracted using logical and.
Sometimes, you need to create a new column based on values in one column. This is very quickly and efficiently done using .loc() method.
The syntax is quite simple and straightforward.
Dataframe_name.loc[condition, new_column_name] = new_column_value
The new_column_value is the value assigned in the new column if the condition in .loc() is True.
for example, creating a column Size based on the Acres column in the our Pandas DataFrame.
df.loc[df["Acres"]>5000, "Size"] = "Large"df.head()
As we can see in the above result, for the rows where the condition, i.e. the value in Acres column is less than 5000, the NaN is added in the Size column.
The story doesn’t end here.
We can add multiple conditions in the .loc() method and create a new column based on multiple conditions.
For example, creating a column named Powerful based on values in Acres and MWp_Capacity
All the missing values i.e. NaN can be handled separately using fillna().
Fyi, missing values can be filled as below. ♻️
The NaN in Size column are filled by “Small” and NaN in Powerful column are filled with “No”.
Summing up,
The most time consuming task of data scientist is to clean up and manipulate the real world data. Hence, it is important to have a flexibility in accessing data from a DataFrame.
The loc() and iloc() methods provide all the tools we need to extract the rows and columns of Pandas DataFrame in Python.
With these five examples, I tried to cover all the possible ways to Master the Pandas DataFrame slicing in Python. Let me know, if I missed anything.
If you enjoy reading such articles, be a Medium member today. With just $5/month you will get unlimited access to Medium & I’ll get a small portion of it. Nonetheless, you can always subscribe to my email list for Free.