Track COVID-19 Data Yourself Using R | by Chris Ross | Towards Data Science

When the global pandemic was first gaining momentum earlier this year, I relied on the media for updates, as most people do. I soon found that the media reports were not only inconsistent but often presented incomplete information. The fundamental issue with these limitations is that they frequently lead to misunderstandings and misinterpretations of the data, as evidenced by the widely divergent views on the severity of this global crisis at any given point in time.
Media organizations often cherry-pick the stats and graphs they report with a preference for sensational headlines that align with the story they are trying to tell. You can’t blame them too much for this tendency given their primary goal is to engage their audience. But if you’re hoping to get an accurate and reliable picture of the current impact of COVID-19 in your country, including tracking and charting trends, you can’t beat analyzing the source data yourself.
In this article, we’re going to cover how to write a script in R to pull and analyze current coronavirus data. The best part is that once you’ve written the script, you can easily save and rerun it at will. You no longer need to rely on incomplete snapshots of the data based on what others decided to analyze and report. Now you’ll quickly and easily have the most recent COVID-19 data at your fingertips and can track current stats and trends yourself!
I use R Studio and have this script saved in R markdown (Rmd). If you prefer to use base R, no worries, the code will work in a standard R script as well. I’m not going to cover how to install R [1] or R Studio [2], but both are open source (and free) and you can check the references below to find detailed documentation and installation instructions.
First, open a new Rmd (or R) file. If you’re using Rmd, use Ctrl + Alt + I (Mac: Cmd + Option + I) to insert a code chunk. In this new chunk, we’ll load the packages we’re going to use. Note that you’ll first need to install these two packages if you haven’t already. I included the code below, to both install and load them, though you’ll need to uncomment the install lines (remove the hashtags) first if you need to install them. You only need to install packages once, so feel free to delete those commented lines once you’ve installed them.
# install.packages('lubridate')# install.packages('tidyverse')library(lubridate)library(tidyverse)
We won’t go into too much detail about what each package does because it’s beyond the scope of this article. I will, however, provide a brief description of them below and include links to their documentation in the references if you want to learn more. I highly recommend you take a closer look at the docs for these packages if you plan to use R regularly. They are awesome and worth learning!
The first package is lubridate, which provides a lot of useful functions for working with date variables [3]. For our purposes, we’re just using lubridate to convert the date format provided in the dataset into a date variable recognized by R. The other package, tidyverse, is actually a group of packages that make up the “Tidyverse” [4]. When you install (and load) tidyverse, the entire group of included packages is automatically installed (and loaded) for you. Two of these included packages are worth mentioning because we’ll use them a few times, dplyr and ggplot2.
The dplyr package “is a grammar of data manipulation, providing a consistent set of verbs that help you solve the most common data manipulation challenges” [5]. There could be a full book just covering how to use dplr, so that description will have to suffice for now. Ggplot2 is another amazing package used for plotting graphs and charts [6]. I took a full semester data visualization class in grad school that exclusively used ggplot2, and we only scratched the surface of available functionality. After installing and loading these two packages, we’re ready to dig in and start exploring the data!
When first looking for available coronavirus datasets earlier this year, I checked out several different options. After a few weeks of testing these options out, I developed a preference for the European Center for Disease Control (ECDC) data [7]. It’s consistently updated (nightly) and offers most of the info I was looking for, including daily case counts, deaths, and population info, broken down by country/territory. Over the past 4–5 months, they’ve updated the dataset a few times changing the formatting and adding columns. But overall they’ve been remarkably consistent in maintaining and updating the data. Thus, the ECDC dataset has been my go-to source for COVID-19 tracking, and it’s what we’ll use here.
# FROM: https://www.ecdc.europa.eudata <- read.csv(“https://opendata.ecdc.europa.eu/covid19/casedistribution/csv", na.strings = “”, fileEncoding = “UTF-8-BOM”, stringsAsFactors = F)data# convert date formatdata$date_reported <- mdy(paste0(data$month,”-”,data$day,”-”,data$year))
The ECDC provides a handy URL for their dataset in CSV format which we can easily pull into R using the built-in “read.csv” function. The nice thing about this link is that they update the dataset daily using the same URL so you never need to change the code to import the dataset. After running the read.csv line (Ctrl + Enter to run a single line or highlighted selection), the dataset is saved in the “data” variable. Run the next line that just says “data” to take a peek at the raw dataset.
As you can see, the “dateRep” column formats the date in a rather unique manner using DD/MM/YYYY. The last line of code in the above chunk essentially just converts that date string into a format that R (and ggplot2) can read. There are numerous ways to accomplish this task but for simplicity, we’ll using the mdy() function included in the lubridate package [3].
Let’s start by taking a look at the cumulative total cases and deaths worldwide. The first line will sum the total number of COVID-19 cases globally to date. Next, we’ll break it down by country and calculate the total number of cases in that country, as well as the maximum number of cases reported in a single day in each country. We then sort the results by total cases in descending order. Below that, we will calculate the same thing but for coronavirus deaths this time (instead of new cases).
# total cases worldwide to datesum(data$cases)# total cases and max single day by countrydata %>%  group_by(countriesAndTerritories) %>%  summarise(cases_sum = sum(cases), cases_max = max(cases)) %>%  arrange(desc(cases_sum))# total deaths worldwide to datesum(data$deaths)# total deaths and max single day by countrydata %>%  group_by(countriesAndTerritories) %>%  summarise(deaths_sum = sum(deaths), deaths_max = max(deaths)) %>%  arrange(desc(deaths_sum))
There will be four outputs from the above chunk of code. Below is the last output, showing the total COVID-19 deaths by country, as well as the max number of deaths in a single day. Currently (Aug. 2020), the US has the most coronavirus deaths by a fairly wide margin.
Now we’ll start plotting the data to identify trends. Since I live in the US, I’m going to plot US cases. You can easily modify the code using other countries, which we’ll cover shortly.
us <- data[data$countriesAndTerritories == ‘United_States_of_America’,]usUS_cases <- ggplot(us,  aes(date_reported, as.numeric(cases))) + geom_col(fill = ‘blue’, alpha = 0.6) +  theme_minimal(base_size = 14) + xlab(NULL) + ylab(NULL) +  scale_x_date(date_labels = “%Y/%m/%d”)US_cases + labs(title=”Daily COVID-19 Cases in US”)
First, I filter the dataset to just look at US cases and store that to a variable. I then use ggplot2 to plot the daily new cases of COVID-19. For more info on how to use ggplot2, check out their documentation [6]. After running the above chunk, you should see the below plot.
Awesome, right? Of course, the picture painted by the data is not awesome, but the fact that you can track it yourself certainly is!
Let’s move on and do the same thing for coronavirus deaths. The code is virtually the same except we are now tracking “deaths” instead of “cases.”
US_deaths <- ggplot(us,  aes(date_reported, as.numeric(deaths))) + geom_col(fill = ‘purple’, alpha = 0.6) +  theme_minimal(base_size = 14) + xlab(NULL) + ylab(NULL) +  scale_x_date(date_labels = “%Y/%m/%d”)US_deaths + labs(title=”Daily COVID-19 Deaths in US”)
As you can see, the death rate paints a different picture than the case counts. While the 2nd wave of cases was twice the size of the 1st, the 2nd wave of deaths hasn’t eclipsed the 1st (yet). These are interesting trends to watch unfold and are definitely worth tracking.
Hold onto your hats, because the last plot we’re going to cover will allow us to compare different countries! I chose the US, China, Italy, and Spain, but you can mix it up and choose whatever countries/territories you’re interested in.
# Now lets add in a few more countrieschina <- data[data$countriesAndTerritories == ‘China’,]spain <- data[data$countriesAndTerritories == ‘Spain’,]italy <- data[data$countriesAndTerritories == ‘Italy’,]USplot <- ggplot(us,  aes(date_reported, as.numeric(Cumulative_number_for_14_days_of_COVID.19_cases_per_100000))) + geom_col(fill = ‘blue’, alpha = 0.6) +  theme_minimal(base_size = 14) + xlab(NULL) + ylab(NULL) +  scale_x_date(date_labels = “%Y/%m/%d”)China_US <- USplot + geom_col(data=china,  aes(date_reported, as.numeric(Cumulative_number_for_14_days_of_COVID.19_cases_per_100000)), fill=”red”, alpha = 0.5)Ch_US_Sp <- China_US + geom_col(data=spain,  aes(date_reported, as.numeric(Cumulative_number_for_14_days_of_COVID.19_cases_per_100000)), fill=”#E69F00", alpha = 0.4)Chn_US_Sp_It <- Ch_US_Sp + geom_col(data=italy,  aes(date_reported, as.numeric(Cumulative_number_for_14_days_of_COVID.19_cases_per_100000)), fill=”#009E73", alpha = 0.9)Chn_US_Sp_It + labs(title=”China, US, Italy, & Spain”)
This code chunk looks a bit more intimidating, but it’s actually pretty straightforward. At the top we filter the dataset for the countries we’re interested in and save each in its own variable. The next series of code blocks create the plots that we are going to stack, one for each country we add.
Considering we want to compare countries, we will use a new column this time, one that lists the cumulative number of new cases over the past 14 days per 100,000 people in that country. Since the population of each country varies, using the number of cases per 100,000 people allows us to standardize the case counts based on the population. Let’s see how that looks.
The little red hump (in the beginning) is China, with Italy in green, Spain in yellow, and the US in blue. As you can see, China was hit first although the overall impact was significantly lower than the other three countries (based on the numbers they reported at least). Sadly, the US still looks in pretty bad shape comparatively. Of these four countries, Spain was hit hardest by the first wave while the US and Italy appear similarly impacted.
Unfortunately, the 2nd wave hit the US like a truck (in terms of cumulative cases), although we appear to be starting a downtrend (fingers crossed). Spain’s 2nd wave appears to still be ramping up. We’ll have to continue monitoring their data to see when they start flattening the curve. Italy, on the other hand, appears to have done an excellent job curbing the spread of COVID-19 after their first wave. There may be lessons we can learn from their success.
This is just the beginning. We’ve only scratched the surface of what you can do with this data. I hope you take what we covered here and run with it! Let it be a springboard for your own discovery. You no longer need to rely on snippets from the media, or other data scientists, to stay on top of the COVID-19 data and track trends. Let me know in the comments what you do with the data, and share with us what you discover!
✍️ Subscribe to get my newest articles, featured in publications like The Startup & Towards Data Science ➡️
[1] R, “R: The R Project for Statistical Computing” r-project.org, 2020. [Online]. Available: https://www.r-project.org. [Accessed: Aug. 10, 2020].
[2] R Studio, “R Studio IDE Desktop” rstudio.com, 2020. [Online]. Available: https://rstudio.com/products/rstudio. [Accessed: Aug. 10, 2020].
[3] lubridate package | R Documentation, “Make Dealing with Dates a Little Easier” rdocumentation.org, 2020. [Online]. Available: https://www.rdocumentation.org/packages/lubridate/versions/1.7.9. [Accessed: Aug. 10, 2020].
[4] tidyverse package | R Documentation, “Easily Install and Load the ‘Tidyverse’” rdocumentation.org, 2020. [Online]. Available: https://www.rdocumentation.org/packages/tidyverse/versions/1.3.0. [Accessed: Aug. 10, 2020].
[5] dplyr package | R Documentation, “A Grammar of Data Manipulation” rdocumentation.org, 2020. [Online]. Available: https://www.rdocumentation.org/packages/dplyr/versions/0.7.8. [Accessed: Aug. 10, 2020].
[6] ggplot2 package | R Documentation, “Create Elegant Data Visualisations Using the Grammar of Graphics” rdocumentation.org, 2020. [Online]. Available: https://www.rdocumentation.org/packages/ggplot2/versions/3.3.2. [Accessed: Aug. 10, 2020].
[7] European CDC, “ ECDC COVID-19 pandemic” ecdc.europa.eu, 2020. [Online]. Available: https://www.ecdc.europa.eu/en/covid-19-pandemic. [Accessed: Aug. 10, 2020].