PIVOT in BigQuery | by Lak Lakshmanan | Towards Data Science

Sometimes, you might want to reformat a table result so you have separate columns for each unique value. This is called a Pivot table — normally, it’s only a display function supported by BI tools. However, it can occasionally be helpful to create the pivot table in SQL. Here’s how to use the PIVOT operator in Google BigQuery to create a pivot table.
Pivot changes rows to columns. For example, suppose we have a table of flights and flight delays like this:
And we want output where we obtain the average departure delay for each airport-airline pair, where the airline codes are columns:
We’d do a PIVOT. Note that if we simply want the average departure delay and are okay with the averages being on different rows, we could simply do:
SELECT   airline,  departure_airport,  AVG(departure_delay)FROM `bigquery-samples.airline_ontime_data.flights`GROUP BY 1, 2
and obtain:
It’s the fact that we need it laid out as a 2D table that requires a PIVOT.
The Pivot operator in BigQuery needs you to specify three things:
from_item that functions as the input. The three columns (airline, departure_airport, departure_delay) from the flights table is our from_item.
aggregate since each cell of the output table consists of multiple values. Here, that’s the AVG of the departure_delay
pivot_column, the column whose values form the columns in the output table. Unfortunately, this needs to be a constant — you can’t put a query in there.
Let’s look at a couple of examples.
We want departure delays by airport and airline. The pivot query is:
SELECT * FROM(  -- #1 from_item  SELECT     airline,    departure_airport,    departure_delay  FROM `bigquery-samples.airline_ontime_data.flights`)PIVOT(  -- #2 aggregate  AVG(departure_delay) AS avgdelay  -- #3 pivot_column  FOR airline in ('AA', 'KH', 'DL', '9E'))
Note the three parts to pivot the departure delays into a table organized by departure-airport and airline code. Because the pivot_column needs to be a constant, we have to list them explicitly.
Result:
The reason for the prefix avgdelay is that we called the aggregate avgdelay.
Let’s create a pivot of the number of questions with a given tag by year
 SELECT    tag,    EXTRACT(YEAR from creation_date) AS year,    COUNT(*) AS n  FROM     `bigquery-public-data.stackoverflow.posts_questions`,    UNNEST(SPLIT(tags, '|')) AS tag  WHERE tags IS NOT null  GROUP BY 1, 2
where the rows are the tags and the columns are the years:
SELECT * FROM (  -- #1 from_item  SELECT    tag,    EXTRACT(YEAR from creation_date) AS year  FROM     `bigquery-public-data.stackoverflow.posts_questions`,    UNNEST(SPLIT(tags, '|')) AS tag  WHERE tags IS NOT null)PIVOT(  -- #2 aggregate  COUNT(*) AS n  -- #3 pivot_column  FOR year in (2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020))
What if you can’t list all the possible values or the pivot column beforehand? One solution is to use scripting. Here, I create a variable called airlines with that prints out the string in the right format (try it out):
DECLARE airlines STRING;SET airlines = (  SELECT     CONCAT('("', STRING_AGG(DISTINCT airline, '", "'), '")'),  FROM `bigquery-samples.airline_ontime_data.flights`);
Then, I can use EXECUTE IMMEDIATE to run the actual query:
EXECUTE IMMEDIATE format("""SELECT * FROM(  SELECT     airline,    departure_airport,    departure_delay  FROM `bigquery-samples.airline_ontime_data.flights`)PIVOT(  AVG(departure_delay) AS avgdelay  FOR airline in %s)ORDER BY departure_airport ASC""", airlines);
This results in: