11 Examples to Master Pandas Groupby Function | by Soner Yıldırım | Towards Data Science

Pandas Groupby function is a versatile and easy-to-use function that helps to get an overview of the data. It makes it easier to explore the dataset and unveil the underlying relationships among variables.
In this post, we will go through 11 different examples to have a comprehensive understanding of the groupby function and see how it can be useful in exploring the data.
I will use a customer churn dataset for the examples. The first step is to read the dataset into a pandas dataframe.
import pandas as pdimport numpy as npdf = pd.read_csv("/content/Churn_Modelling.csv")df.head()
We have some features about the customers and their products at a bank. The goal here is to predict whether a customer will churn (i.e. exited = 1) using the provided features.
In terms of machine earning, this is a classification problem. Groupby function can be used to explore how features are related or have an effect on the target variable (“Exited”).
The following figure represents the logic and working principle of groupby function.
Let’s start with the examples.
The first one is to check if gender makes any difference in customer churn.
#example 1df[['Gender','Exited']].groupby('Gender').mean()
We take a subset of the dataframe which consists of gender and exited columns. We then group the rows based on the values in the gender column which are male and female. Finally, an aggregate function is applied. The result is the average churn rate for females and males.
In addition to the mean, you may also want to see how many males and females exist in the dataset. If there is an extreme imbalance, checking the mean may cause false assumptions.
The solution is to apply both mean and count as aggregate functions.
#example 2df[['Gender','Exited']].groupby('Gender').agg(['mean','count'])
The number of females and males are close so there is not a considerable imbalance. On average, the churn rate for females is higher than that of males.
You may think that it is too general to make a comparison based on gender only. In this case, we can group by multiple columns by passing a list of columns to groupby function.
#example 3df[['Gender','Geography','Exited']].groupby(['Gender','Geography']).mean()
Churn rate is higher for females in the three countries in our dataset.
We can also sort the results.
#example 4df[['Gender','Geography','Exited']].groupby(['Gender','Geography']).mean().sort_values(by='Exited')
The results are in ascending order but we can change it using the ascending parameter.
#example 5df[['Gender','Geography','Exited']].groupby(['Gender','Geography']).mean().sort_values(by='Exited', ascending=False)
We can also check multiple features based on groups in another feature. Let’s check how age and tenure changes in different countries.
#example 6df[['Geography','Age','Tenure']].groupby(['Geography']).agg(['mean','max'])
The average and maximum values of age and tenure columns are pretty close in the three countries.
Let’s also add the “Exited” column to example 6.
#example 7df[['Exited','Geography','Age','Tenure']].groupby(['Exited','Geography']).agg(['mean','count'])
We can sort the results based on any column. However, since it is a multi-index, we need to pass a tuple to the sort_values function.
#example 8df[['Exited','Geography','Age','Tenure']].groupby(['Exited','Geography']).agg(['mean','count']).sort_values(by=[('Age','mean')])
The results are sorted by (‘Age’, ‘mean’) column.
The variables in the groupby function are returned as the index of the resulting dataframe. We can change it by setting the as_index parameter as false.
#example 9df[['Exited','IsActiveMember','NumOfProducts','Balance']].groupby(['Exited','IsActiveMember'], as_index=False).mean()
Each column in the groupby function is represented with a column. There is a row for each combination of categories.
The groupby function drops the missing values by default. Our dataset does not have any missing values. Let’s add some missing values and see how the dropna parameter is used.
#example 10df['Geography'][30:50] = np.nandf[['Geography','Exited']].groupby('Geography').mean()
Although we have missing values in the geography column, they are ignored. We can change it by setting the dropna parameter as false.
#example 11df[['Geography','Exited']].groupby('Geography', dropna=False).agg(['mean','count'])
As you can see, there is another category for missing values.
Note: In order to use the dropna parameter of the groupby function, you need to have pandas version 1.1.0 or higher.
We can diversify the examples but the underlying logic is the same. Groupby function can be used as a first step in the exploratory data analysis process because it gives us an idea about the relationships between variables in the dataset.
Thanks for reading. Please let me know if you have any feedback.