4 Different Ways to Efficiently Sort a Pandas DataFrame | by Byron Dolon | Towards Data Science

Sorting data properly can make it easy for you to understand it.
Many people turn to advanced indexing and aggregate functions in Pandas to answer questions at every stage of analysis. These features can be incredibly useful when you need to manipulate data. However, Pandas also offers different ways of sorting a DataFrame, which could be more suited to analyzing data than .loc[] and other vectorized solutions.
If you have multiple columns you want to sort by, or if you just need to sort a series, Pandas has built-in functionality that can help with that.
By the end of this piece, using different methods of sorting in Pandas, we’ll have answered the following questions:
Which five video games have the most sales in the European market?solved with: .sort_values()
What game was published the earliest and has the lowest global sales in its year?solved with: multiple arguments .sort_values()
What are the top four highest and lowest video game sales values for the North American market?solved with: nsmallest() and nlargest()
Which publishers had the highest Xbox One sales in a year, and what did its sales for PC and PS4 look like in the same year?solved with: MultiIndex .sort_values()
We’ll be using this video game sales data, so download the csv file if you want to follow along. As always, don’t forget to import pandas before you get going.
import pandas as pd# load datadf = pd.read_csv('vgsales.csv').dropna()df['Year'] = df['Year'].astype(int)df.head()
All of the sorting methods available in Pandas fall under the following three categories:
Sorting by index labels;
Sorting by column values;
Sorting by a combination of index labels and column values.
Pandas automatically generates an index for every DataFrame you create. The index label starts at 0 and increments by 1 for every row. You can see on the screenshot above that the index labels are present in the leftmost column (the one with no column name).
You can see that there wouldn’t really be a point in sorting the DataFrame based on its auto-generated index. However, if we set the “Name” column as the DataFrame’s index, then we could sort the table by alphabetical order. We can change and sort by the index with a simple line of code:
df.set_index('Name').sort_index()
Here, we can see that the .sort_index() function re-organized that data so the games with a punctuation mark as the first character of its name are at the top of the list.
If you look further down, it will show the games with a number as the first character of its name next, and then after that those that start with letters. Last in the sorting hierarchy are special symbols, like the final entry on the table above.
That was a really quick example of sorting by a DataFrame’s index. Now let’s get into some more advanced sorting!
To answer this question, we’re going to use the .sort_values() function. The code is as follows:
df.sort_values(by='EU_Sales', ascending=False).head(5)
Here, we input the column to sort by (EU_Sales) and organize our data so that the highest values come first (set ascending to “False”). You can see the output below.
The sorted DataFrame has been reorganized, so we can now see that the games with the most sales in the EU are not exactly the same as the games with the most sales globally.
To answer this question, we are interested in both the “Year” column and the “Global_Sales” column. We’re going to use the .sort_values() function again, but with a slight change:
df.sort_values(by=['Year','Global_Sales']).head(1)
This is a case where we wanted to sort our data by more than one column. You can sort by as many columns as you want with this function, simply by passing a list of column names. In this case, we have written “Year” first and “Global_Sales” second, which means the function will sort the DataFrame by its publishing year first and then its sales.
By default, the function will sort the DataFrame with the lowest numerical value first, so we don’t need to pass ascending=True (although you could if you wanted to). You can see the result below.
Here, we have identified the game “Checkers” published by “Atari” as the earliest game published (1980) in the dataset with the lowest global sales.
With this question, we only care about the sales values, which means we don’t need the video game metadata like “Name” and “Platform”. We only care about one column, which is “NA_Sales” that gives us the values for the North American market. This means we are working with a “series”, which is a one-dimensional array in Pandas.
Because we’re working with a series, we can use the nsmallest() and nlargest() methods that give us the smallest or largest “n” values in the series. To use this function, you simply pass a value “n” which refers to the number of results you want to see. To answer this question, we write the following:
df['NA_Sales'].nlargest(4)
And:
df['NA_Sales'].nsmallest(4)
According to the Pandas documentation, you could use these methods because they might be faster than using the .sort_values() and head() method that we’ve been doing so far. However, it seems that this applies more to very large data sets, as even for one with 16,291 rows there was no difference. For example:
df['NA_Sales'].sort_values(ascending=False).head(4)
You can see that we get the same values with a different line of code, and it also took the same time (11 milliseconds). However, I think using nlargest() and nsmallest() make for cleaner code. The function names are self-explanatory and easy to understand, so you don’t have to sacrifice readability for a nifty function.
To answer this question, we’re going to create a pivot table. These are great when you want to aggregate and present data in an easy-to-read table.
The code to create a pivot table looks like this:
df_pivot = df.loc[df['Platform'].isin(['PC','XOne','PS4'])]df_pivot = df_pivot[['Platform','Year','Publisher','Global_Sales']]df_pivot = df_pivot.pivot_table(index = ['Publisher','Year'],                                columns = 'Platform',                                aggfunc = 'sum',                                fill_value = 0)
If you’re unfamiliar with pivot tables and using a MultiIndex, I’d suggest checking out my previous piece on the topic.
towardsdatascience.com
To answer our question, we want to be able to compare the PC, PS4 and Xbox One sales, so this pivot table makes it easy for us to look at the data. The initial pivot table looks like this:
Now, when sorting this data, we’re interested in the highest “Global_Sales” for “XOne”. If we look at the columns in this DataFrame, we’ll see that there is a MultiIndex in place.
df_pivot.columns
When sorting by a MultiIndex column, you need to make sure to specify all levels of the MultiIndex in question. To sort our newly created pivot table, we use the following code:
df_pivot.sort_values(by=('Global_Sales','XOne'), ascending=False)
Here, you can see we pass a tuple into the .sort_values() function. This is because the column name we want to sort by is (‘Global_Sales’, ‘XOne’). You cannot only pass “XOne”, as that is not a valid column name in this case. So make sure to specify every level of the MultiIndex when sorting this kind of DataFrame. The resulting table looks like this:
Based on this, we can see that Electronic Arts (EA) has the highest Xbox One sales across all years in 2015. We can also see that its sales in the PC and PS4 segments in the same year. The sorted pivot table makes it easy to do a side-by-side comparison of different columns, so keep this method in mind, as you can use it for many different situations.
And we’re done!
This was just an introduction to sorting with Pandas. There are many different ways to find answers to the questions we went through, but I find that sorting is a quick and easy way to conduct preliminary analysis.
Remember that there’s more than one way to sort a DataFrame. Some methods may be more effective than others, and that Pandas offers in-built methods to help you write cleaner looking code.
Good luck with your sorting adventures!