Ultimate Pandas Guide — Window Functions | by Skyler Dale | Towards Data Science

Window functions are an efficient way to understand more about the relationship between each of the records within our data and the groups they belong to.
They also happen to be a common data science interview question so they’re a good thing to understand well.
In this post, I’ll walk through an intuitive explanation of Window functions and how to implement them in Pandas.
In a standard “groupby,” we split our data up into groups, apply an aggregation, and then combine each of the results in a new table.
This helps us answer questions about the characteristics of the groups within our data.
For example, if we have monthly customer-level sales data, we can use a groupby to understand what our total sales are for each month:
If you’re a little rusty on how this works, I recommend you check out my post on the groupby function:
towardsdatascience.com
The output above is useful if we only care about monthly sales.
But the drawback of this result is that our information is now spread across two tables — the first gives us the sales details and the second gives us aggregated information about each month.
Window functions help us bridge this gap, by bringing our group-level aggregations back to the initial table:
If it’s not immediately clear why that’s useful, let’s add a percent column to our output:
Now we can see what percentage of the total monthly sales each of our records comprises.
The window function is useful because it helps us run these types of calculations without having to perform any separate joins.
The syntax for a window function in Pandas is pleasantly simple, and very similar to the syntax we would use for a groupby aggregation.
The key difference is that to perform a window function we use the “transform” method rather than the “agg” method:
And the transform method returns a series instead of a DataFrame, so we need to add it as a column if we want it to be a part of our original DataFrame:
sales_data[“monthly_sales”] = sales_data.groupby(“month”).transform(sum)[“purchase_amount”]
And that’s it! Not too bad.
Let’s try one more example. Below we rank each transaction by state:
sales_data['state_rank'] = sales_data.sort_values(by =                   'purchase_amount', ascending = False).groupby("state").transform('rank', method = 'min')["purchase_amount"]
Note a couple things here:
We use the “sort_values” function to make sure we are ranking in order of the purchase amount.We pass an argument (“min”) to the “method” parameter within our transform. This is a parameter of the rank method and gets passed to it when executed.This type of analysis tends to be useful when we want to return the “top” values within groups. For example, we could filter the result above for the top 2 transactions within each state and then pass those to sales reps that cover each state.
We use the “sort_values” function to make sure we are ranking in order of the purchase amount.
We pass an argument (“min”) to the “method” parameter within our transform. This is a parameter of the rank method and gets passed to it when executed.
This type of analysis tends to be useful when we want to return the “top” values within groups. For example, we could filter the result above for the top 2 transactions within each state and then pass those to sales reps that cover each state.
For those with a strong SQL background, this syntax might feel a bit strange.
In SQL we execute a window function by starting with an aggregation and then applying it “over” an optional “partition by” and “order by” :
select rank() over (partition by state order by purchase_amount desc)
In order to help reconcile the two approaches, I’ve converted the elements of the Pandas code from above into their SQL equivalent:
Window functions are powerful, efficient, and fairly straightforward to implement in Pandas.
It’s worth noting, however, that the “transform” method is not the correct approach for performing position-based aggregations, like summing a column’s values for the previous 5 records. In order to address this, we’ll have to look deeper at the “shift” and “expanding” functions in Pandas in a future post.