9 Pandas value_counts() tricks to improve your data analysis | by B. Chen | Towards Data Science

Data Scientists often spend most of their time exploring and preprocessing data. When it comes to data profiling and understand the data structure, Pandas value_counts() is one of the top favorites. The function returns a Series containing counts of unique values. The resulting Series can be sorted in descending or ascending order, including or excluding NA through parameter control.
In this article, we will explore the different use cases of Pandas value_counts(). You’ll learn how to use it to deal with the following common tasks.
Default parametersSort results in ascending orderSort results alphabeticallyInclude NA from the resultsShow result in a percentage countBin continuous data into discrete intervalsGroup by and call value_counts()Convert resulting Series into a DataFrameApply to a DataFrame
Default parameters
Sort results in ascending order
Sort results alphabetically
Include NA from the results
Show result in a percentage count
Bin continuous data into discrete intervals
Group by and call value_counts()
Convert resulting Series into a DataFrame
Apply to a DataFrame
Please check out Notebook for source code.
Visit Github Repo for other tutorials
Pandas value_counts() function returns a Series containing counts of unique values. By default, the resulting Series is in descending order without any NA values. For example, let’s get counts for the column “Embarked” from the Titanic dataset.
>>> df['Embarked'].value_counts()S    644C    168Q     77Name: Embarked, dtype: int64
The series returned by value_count() is in descending order by default. We can set the argument ascending to True for an ascending results.
>>> df['Embarked'].value_counts(ascending=True)Q     77C    168S    644Name: Embarked, dtype: int64
We have learned the argument ascending to get result sorted by the value counts ASC or DESC. In some cases, it’s more preferable to display our results in alphabetical order. This can be done by calling sort_index(ascending=True) after the value_counts(), for example
>>> df['Embarked'].value_counts(ascending=True).sort_index(ascending=True)C    168Q     77S    644Name: Embarked, dtype: int64
By default, rows that contain any NA values are omitted from the results. There is an argument dropna to configure it. We can set the value to False to include counts of rows with NA.
df['Embarked'].value_counts(dropna=False)S      644C      168Q       77NaN      2Name: Embarked, dtype: int64
When doing Exploratory Data Analysis, sometimes it can be more useful to see a percentage count of the unique values. This can be done by setting the argument normalize to True , for example:
df['Embarked'].value_counts(normalize=True)S    0.724409C    0.188976Q    0.086614Name: Embarked, dtype: float64
If we prefer the result to be formatted with a percent sign (%), we can set the Pandas display option as follows:
>>> pd.set_option('display.float_format', '{:.2%}'.format)>>> df['Embarked'].value_counts(normalize = True)S   72.44%C   18.90%Q    8.66%Name: Embarked, dtype: float64
By calling pd.set_option('display.float_format', '{:.2%}'.format), it updates Pandas default display settings and applies to all float values. To reset it, you can call pd.reset_option('display.float_format').
In addition, thanks for David B Rosen (PhD)’s advice. Instead of the Pandas display option, we can simply convert the result into a DataFrame and call style.format('{:.2%}') as follows:
df['Embarked'].value_counts(normalize = True).to_frame().style.format('{:.2%}')
If you want to learn more about Pandas Display options, you can check out:
towardsdatascience.com
Pandas value_counts() can be used to bin continuous data into discrete intervals with the bin argument. Similar to the Pandas cut() function, we can pass an integer or a list to the bin argument.
When an integer is passed to bin, the function will discretize continuous values into equal-sized bins, for example:
>>> df['Fare'].value_counts(bins=3)(-0.513, 170.776]     871(170.776, 341.553]     17(341.553, 512.329]      3Name: Fare, dtype: int64
When a list is passed to bin, the function will divide continuous values into custom groups, for example:
>>> df['Fare'].value_counts(bins=[-1, 20, 100, 550])(-1.001, 20.0]    515(20.0, 100.0]     323(100.0, 550.0]     53Name: Fare, dtype: int64
If you want to learn more about binning with Pandas cut(), you can check out:
towardsdatascience.com
Pandas groupby() allows us to split data into separate groups to perform computations for better analysis. One of the common use cases is to group by a certain column and then get the count of unique values of another column. For example, let’s group by “Embarked” column and get the count of different “Sex” values.
>>> df.groupby('Embarked')['Sex'].value_counts()Embarked  Sex   C         male       95          female     73Q         male       41          female     36S         male      441          female    203Name: Sex, dtype: int64
If you want to learn more about groupby(), you can check out:
towardsdatascience.com
Pandas value_counts() returns a Series, including the previous example with MultiIndex. If we would like our result to be displayed as a DataFrame, we can call to_frame() after the value_count().
>>> df.groupby('Embarked')['Sex'].value_counts().to_frame()
So far, we have been applying value_counts() to a Pandas Series, there is an equivalent method in Pandas DataFrame. Pandas DataFrame.value_counts() returns a Series containing counts of unique rows in the DataFrame.
Let’s take a look at an example to better understand it:
df = pd.DataFrame({    'num_legs': [2, 4, 4, 6],    'num_wings': [2, 0, 0, 0]},    index=['falcon', 'dog', 'cat', 'ant'])>>> df.value_counts()num_legs  num_wings4         0            26         0            12         2            1dtype: int64
By calling value_counts() on df, it returns a MultiIndex Series with num_legs and num_wings as the index. From the result, we can find that 2 records have num_legs=4 and num_wing=0.
Similarly, we can call to_frame() to convert the result into a DataFrame
>>> df.value_counts().to_frame()
In this article, we have explored the different use cases of Pandas value_counts(). It’s very handy and one of the top favorite methods during Exploratory Data Analysis and Data Preprocessing.
I hope this article will help you to save time in learning Pandas. I recommend you to check out the documentation for the value_counts() API and to know about other things you can do.
Thanks for reading. Please check out the notebook for the source code and stay tuned if you are interested in the practical aspect of machine learning.
All Pandas json_normalize() you should know for flattening JSON
Using Pandas method chaining to improve code readability
How to do a Custom Sort on Pandas DataFrame
All the Pandas shift() you should know for data analysis
When to use Pandas transform() function
Pandas concat() tricks you should know
Difference between apply() and transform() in Pandas
All the Pandas merge() you should know
Working with datetime in Pandas DataFrame
Pandas read_csv() tricks you should know
4 tricks you should know to parse date columns with Pandas read_csv()
More tutorials can be found on Github