How to group and aggregate data using SQL | by Kate Marie Lewis | Towards Data Science

I have been so enjoying engaging with my friends to share my love of data science. We are powering through learning SQL, with three lessons under our belt already. If you would like to start at the beginning here is a link to the first lesson. All of the lessons can also be found here. Otherwise, get stuck into this lesson on grouping and aggregating data using SQL.
Last week, we covered filtering data using SQL. Using one of my favourite shows, Charmed for example data, we went through using WHERE clauses in queries. In addition, we explored the use of the keywords IN, AND, OR, LIKE, BETWEEN and NOT.
Now that we know how to filter data, we will move onto aggregation. We will learn how to use the keywords MIN and MAX to find the minimum and maximum of our data respectively. We will also practice using the COUNT, AVG and SUM keywords in a similar manner.
This will also be the first lesson that we have encountered NULL values. So we will learn how to deal with them in our datasets.
It just so happens that all of the friends I am teaching data science to, whilst self-isolating are women. So what better example to use in this lesson than the Spice Girls to celebrate a bit of girl power. We will use data on the first Spice Girls album titled Spice. This is one that was on high rotation in my house growing up.
use the keyword MIN to find the minimum value in a column
use the keyword MAX to find the maximum value in a column
use the keyword COUNT to count the number of rows in a column or table
use the keyword AVG to find the mean of a numerical column
use the keyword SUM to find the total of a numerical column when all the values are added together
use the keyword GROUP BY to group by a column in a table
know how NULL values will be handled in each of the above methods
understand how aliases work and how to use the AS keyword to create them
I think that the first Spice Girls album is the best one. Just in case people don’t believe me, I want to find some data to back it up. Using Australian charts data for the singles released from each Spice Girls album, I think I can prove the first one rules!
This dataset contains information on all the tracks in each album that the Spice Girls released. The table contains the length of each track, when each single was released, what position the song peaked at in the Australian charts and how many weeks the song was in the Australian charts. I didn’t include the greatest hits album because I thought that would confuse matters.
I got the data for this table from the Spice Girls discography Wikipedia page, the Spice(album) Wikipedia page, Spice World(album) Wikipedia page, Forever(Spice Girls Album) and on an Australian charts website.
In this lesson I will teach you how to use the aggregating keywords MIN, MAX, COUNT, SUM and AVG in the SELECT statement. These aggregators may also be used elsewhere in queries. For example, they can be used in a HAVING clause, but that is beyond the scope of this lesson. I have written another story comparing WHERE and HAVING in SQL in case you want to learn more.
The MAX keyword can be used to find the maximum value within a column. It can be used on many different datatypes including integers, floats, strings and dates.
As you can see below, the syntax to use MAX involves putting the MAX keyword inside the select statement. The MAX keyword has a set of brackets after it, which is where you put the column name you want to find the maximum value of.
SELECT     MAX(name_column_one) AS 'largest_col_one'FROM    name_of_table;
Another useful keyword that can be used in conjunction with aggregators is AS. AS is used to create an alias or temporary name for the column created by the aggregator in the returned table. Since the aggregator has been used on column 1, it is useful to give the result a new name so that it is easily identifiable in the resulting table. The keyword AS may also be used to give an alias to a table, but we will cover that in our lesson on joining tables.
The MIN keyword is used in much the same way as the MAX keyword. As you would expect, it is used to return the minimum value of the selected column.
SELECT     MIN(name_column_two) AS 'smallest_col_two'FROM    name_of_table;
COUNT is a keyword used to count the number of rows in the selected column. So in the example below it would count the total number of rows in the table.
SELECT     COUNT(name_column_three) AS 'col_three_counts'FROM    name_of_table;
However, COUNT is more useful when there is some filter or grouping applied to the table. In that case you would be counting the number of rows in the table that meet the filter or grouping conditions.
SELECT     COUNT(name_column_three) AS 'col_three_counts'FROM    name_of_tableWHERE    name_column_one = value;
Unlike MIN, MAX and COUNT, the keywords SUM and AVG are only able to be used on columns that contain numeric datatypes like integers or floats. AVG is used to return the average value of a column.
SELECT     AVG(name_column_four) AS 'mean_col_four'FROM    name_of_table;
SUM adds together the values in a numeric column. The syntax to use SUM is the same as the other aggregators.
SELECT     SUM(name_column_five) AS 'summed_col_five'FROM    name_of_table;
Grouping is very useful for summarising data that has repeated values. It is often used in conjunction with the aggregators MIN, MAX, COUNT, SUM and AVG. GROUP BY collects together identical values in the column it is applied to. Then an aggregator can be used on another column based on those groupings. The GROUP BY clause will often be used on a column that contains category values.
For example, if you use the AVG aggregator with GROUP BY you can find the average value for one column for each group of values in another column. The syntax for this example is shown below. In this case the table returned would contain the average value from column one for each value in column two.
SELECT     AVG(name_column_one) AS mean_col_one, name_column_twoFROM    name_of_tableGROUP BY     name_column_two;
A NULL value is what happens when a field has no value entered. So it is like a placeholder to let you know that there is an empty space in the table. It is different from a 0 value or an empty string ‘’ as they are still values. NULL has its own keywords.
You can check for NULL values in a column using the IS NULL keyword. This is done in the WHERE statement and will return only the records that contain NULL values in the column specified in the WHERE statement.
SELECT     *FROM    name_of_tableWHERE    name_column_one IS NULL;
However, the more common scenario would be where you want to exclude all records that contain NULL values. In that case you can add the NOT keyword to your query. In the example below only the records that have no NULL values in column two will be included in the returned table.
SELECT     *FROM    name_of_tableWHERE    name_column_two IS NOT NULL;
NULL values are ignored by aggregate keywords MIN, MAX, AVG, SUM and COUNT. For MIN and MAX it is fairly straight forward, you will get the minimum or maximum value in the column ignoring all NULL values. Similarly for SUM, all the values in the column that are not NULL will be added together. For AVG you get the average value of the column when all NULL values are removed.
COUNT is only slightly more tricky, all the rows in the selected column will be counted, except for the rows containing NULL values. The only exception to that rule is shown below. In this case all of the rows of the table will be counted, regardless of whether there are NULL values.
SELECT     COUNT(*)FROM     name_of_table;
In contrast to the aggregators, GROUP BY does not ignore NULL values. Instead they are grouped together as a single grouping category.
go to http://sqlfiddle.com/ (or you can use https://www.db-fiddle.com/ as I have found SQL fiddle has had some downtime recently)In the left hand box put the CREATE TABLE and INSERT INTO queries below
go to http://sqlfiddle.com/ (or you can use https://www.db-fiddle.com/ as I have found SQL fiddle has had some downtime recently)
In the left hand box put the CREATE TABLE and INSERT INTO queries below
CREATE TABLE spice_girls(    album varchar(255),    title varchar(255),    length TIME(0),    track_number int,    single_released date,    peak_chart_position_au int,    chart_weeks_au int);INSERT INTO spice_girls(    album,    title,    length,    track_number,    single_released,    peak_chart_position_au,    chart_weeks_au)VALUES    ("Spice", "Wannabe", '000:02:53', 1, '1996-07-08', 1, 29),    ("Spice", "Say You'll Be There", '000:03:55', 2, '1996-10-10', 12, 22),    ("Spice", "2 Become 1", '000:04:01', 3, '1996-12-16', 2, 18),    ("Spice", "Love Thing", '000:03:38', 4, NULL, NULL, NULL),    ("Spice", "Last Time Lover", '000:04:11', 5, NULL, NULL, NULL),    ("Spice", "Mama", '000:05:04', 6, '1997-02-27', 13, 14),        ("Spice", "Who Do You Think You Are", '000:04:00', 7, '1997-02-27', 13, 14),    ("Spice", "Something Kinda Funny", '000:04:05', 8, NULL, NULL, NULL),    ("Spice", "Naked", '000:04:25', 9, NULL, NULL, NULL),    ("Spice", "If U Can't Dance", '000:03:48', 10, NULL, NULL, NULL),    ("Spice World", "Spice Up Your Life", '000:02:53', 1, '1997-10-06', 8, 20),    ("Spice World", "Stop", '000:03:24', 2, '1997-01-01', 5, 21),    ("Spice World", "Too Much", '000:04:31', 3, '1997-12-08', 9, 15),    ("Spice World", "Saturday Night Divas", '000:04:25', 4, NULL, NULL, NULL),    ("Spice World", "Never Give Up on the Good Times", '000:04:30', 5, NULL, NULL, NULL),    ("Spice World", "Move Over", '000:02:46', 6, NULL, NULL, NULL),        ("Spice World", "Do It", '000:04:04', 7, NULL, NULL, NULL),    ("Spice World", "Denying", '000:03:46', 8, NULL, NULL, NULL),    ("Spice World", "Viva Forever", '000:05:09', 9, '1998-07-20', 2, 21),    ("Spice World", "The Lady Is a Vamp", '000:03:09', 10, NULL, NULL, NULL),    ("Forever", "Holler", '000:04:15', 1, '2000-10-23', 2, 15),    ("Forever", "Tell Me Why", '000:04:13', 2, NULL, NULL, NULL),    ("Forever", "Let Love Lead the Way", '000:04:57', 3, '2000-10-23', 2, NULL),    ("Forever", "Right Back at Ya", '000:04:09', 4, NULL, NULL, NULL),    ("Forever", "Get Down with Me", '000:03:45', 5, NULL, NULL, NULL),    ("Forever", "Wasting My Time", '000:04:13', 6, NULL, NULL, NULL),        ("Forever", "Weekend Love", '000:04:04', 7, NULL, NULL, NULL),    ("Forever", "Time Goes By", '000:04:51', 8, NULL, NULL, NULL),    ("Forever", "If You Wanna Have Some Fun", '000:05:25', 9, NULL, NULL, NULL),    ("Forever", "Oxygen", '000:04:55', 10, NULL, NULL, NULL),    ("Forever", "Goodbye", '000:04:35', 11, '1998-12-07', 3, 16);
Note about special characters: In creating this table we come across a new syntactical nuance that we have not seen in the previous lessons. Some of the song names contain apostrophes. So if we surrounded the song name strings with single quotation marks ‘’ like we normally would, there would be an error. This is because the apostrophe would be confused for the closing quotation by the program. So to include a string containing an apostrophe in our table we need to use double quotations around the strings.
The reverse can also be done if you need to have a string that contains double quotations. If you would like to learn more about special characters like quotations, this is a great resource.
Note about TIME datatype: There is also a datatype in this table that we have not seen in any of the previous lessons. It is the TIME datatype. Time has the format ‘hours:minutes:seconds’.
3. Click the build schema button
4. In the right hand box put your queries
5. Run the query below and see if it returns what you would expect it to:
SELECT    SUM(length) AS 'total_length'FROM     spice_girls;
6. Run the query below and see if it returns what you would expect it to:
SELECT    SUM(length) AS 'album_length',     albumFROM     spice_girlsGROUP BY     album;
7. Run the query below and see if it returns what you would expect it to:
SELECT    album,    SUM(length) AS 'album_length',     AVG(length) AS 'average_song_length',    COUNT(length) AS 'number_of_songs',    MIN(length) AS 'shortest_song_length',    MAX(length) AS 'longest_song_length'FROM     spice_girlsGROUP BY     albumORDER BY     album;
Exercise 1: Write a query to show which album is the best, if we define the best album as the one with the highest peak chart position of one of its singles.
Exercise 2: Write a query to show which album is the best, if we define the best album as the one with the highest average peak chart position of all of the singles in the album.
Exercise 3: Write a query to show which album is the best, if we define the best album as the one with the highest number of singles in the Australian charts for at least 5 weeks.
After completing this lesson you should know:
how to use the keyword MIN to find the minimum value in a column
how to use the keyword MAX to find the maximum value in a column
how to use the keyword COUNT to count the number of rows in a column or table
how to use the keyword AVG to find the mean of a numerical column
how to use the keyword SUM to find the total of a numerical column when all the values are added together
how to use the keyword GROUP BY to group by a column in a table
understand how NULL values are handled
be able to write your own queries using one or more of the above methods
be able to use the AS keyword to create an alias for an aggregated column
Next week will be our final SQL lesson. We will learn how to join tables together. There are several different ways that we can combine datasets, using the UNION, UNION ALL and JOIN keywords. If you are eager to get into looking at joins before next week I have previously written an article about the difference between inner and outer joins in SQL.
In addition to data, my other passion is painting. You can find my wildlife art at www.katemarielewis.com