How to compute a growth rate in BigQuery using SQL | by Romain Granger | Towards Data Science

For some use cases, it’s important to monitor the change of a specific variable over a period of time. A common example would be to monitor revenue month-over-month to understand the growth of a business and measure the impact of, for example, special marketing efforts, a limited time offer, giveaways, changes in pricing, etc...
Besides, it does not only apply to revenue. A long list of potential variables exists:
Stock price
Revenue/Transactions
Users/Active or Churn Users
From experience, it’s not easy to find a good and easy method in SQL. Lots of them are using exhaustive grouping and joining clauses, complex date filtering or indexing, and even sometimes, methods or functions that are not present in all SQL dialects.
So if you are interested in understanding an efficient and easy method to achieve this calculation, you are definitely in the right place! 😎
In this article, we will be computing a month-over-month revenue growth rate, based on the Google Merchandise shop data. It’s one of the most common examples we pick here, but the method can work for any variable other than revenue. It can be used for understanding user growth or price changes and more...
It is the same for the time period, we are executing our example over a monthly period but it can be days, weeks, or years.
Our solution will be using navigation functions. They are functions that allow us to compute or get a value over a group of rows and return a single result for each row. It’s like traveling or browsing over a single table without performing joins or grouping over dimensions.
To know more about navigation functions, you can have a look here 👇🏼
https://cloud.google.com/bigquery/docs/reference/standard-sql/navigation_functions
For this experiment, we are using Google Analytics sample data from the public BigQuery data set. You can get more information and access it by following this link 👇🏼
https://support.google.com/analytics/answer/7586738?hl=en
You can try the following queries assuming that you have already:
A project and dataset configured in BigQuery (You’ll need your project ID)
Access to the BigQuery public data set and the google analytics sample table (google_analytics_sample)
Here is the full code below on gist, that we will explain step by step:
First, we would like to get our specific variable (in this case revenue) aggregated by our desire time period (in this case on a monthly basis).
The above query returns a table with the revenue generated by the online shop every month.
By using the “EXPLORE DATA” button between the query editor and the result table, we can display nicely on Data Studio the monthly revenue as follow:
That’s only the first step, but it’s a good start, hang on 😅
To compute the growth rates, we will use navigation functions and more precisely the LAG() function. If you know it already you can jump directly to step #3, if you are not familiar with these functions, we will explain its mechanism in the next step to fully understand our process.
Hints and explanations about the query
We are using PARSE_DATE() to change the format of the date because the original Google Analytics format is YYYYMMDD. In this case, the function will return a DATE type with a standard YYYY-MM-DD format.
We then apply a DATE_TRUNC(), to extract the monthly part of the date, like a %Y-%m format, and get the revenue per month.
We are converting the revenue because this field is expressed as the value passed to Analytics multiplied by 106 (e.g., 2.40 would be given as 2400000), we need then to divide it by 1000000. Note that this format is really specific to google analytics data in BigQuery.
Grouping by 1 means that we actually perform the grouping by the first column regardless of its name.
To compute the growth rate, we are using the relative change formula which is defined by the following:
((final value — initial value) / initial value) * 100
But what does it mean for us?
In our case, the final value is therevenue from our current month, and the initial value is therevenue from the previous month (or previous row), and that’s why windowing functions will be really helpful in this case.
The LAG() function is not that hard, I promise 😎
It is a function, that returns the value of the wanted variable from a preceding row.
It is a function that requires an ORDER BY clause in the OVER() statement. It is usually used as LAG(variable,offset) where:
variable can be any data type. (in our case a FLOAT)
offset must be a non-negative integer.
Hints and explanations about the query
For readability, we write the previous query inside a WITH clause, we also remove the first ORDER BY which was just to nicely display the previous table.
After running this query, we obtain the following output:
As you can see on the output, on the same row, we have the revenue for the current month and the last_month_revenue which is the revenue value from the previous month (which is our preceding row here).
As said, the LAG() function use as input the column containing our desire variable (revenue), and an OFFSET that you can define (by default 1, meaning the first previous row)
LAG(revenue,1) OVER(ORDER BY months ASC)
Changing the offset value changes which preceding row is returned. The default value is 1, indicating that it will pick the previous row. This is why we write it without filling an offset in our example.
BigQuery LAG() documentation 👉🏼 here
Now that we have our relative change formula, and understood the use of the LAG() function, we need to write our final query expressing the formula in SQL:
The first WITH clause doesn’t change. Under the “ — Main Query” comment, we only replace our previous last_month_revenue by the relative change formula. This gives us pretty much the same output:
That’s how we get the month-over-month growth rate! 🍻 It is then ready to be displayed on Data studio.
Hints and explanations about the query
As a reminder, we use ((final value — initial value) / initial value) * 100
revenue = Final value, which is the value of our current month.
LAG(revenue)OVER(ORDER BY months ASC) = Initial value, the value of our previous month.
Here, we just convert the formula using our SQL syntax and fields.
To have a cleaner result, we round the output applying ROUND(value,2) to limit the number to two decimals.
We can then use Datastudio to display our results, which gives us a nice overview of our revenue changes month-over-month:
This method can be used on any kind of time series and variable. The interesting part is the use of navigation functions. They make this exercise a lot easier even if they are not that easy to understand at first.
I hope you’ll like this method and the article will help you save some time and maybe even understand navigation functions better if you are new to it!
But more importantly, I hope this will help you get more at ease using SQL and BigQuery! It’s a good use case to practice your skills!
Time to dive into some data! 🤓