Practical ML Part 1: COVID-19 in Brazil
By: Mariana Almeida
Tags: Data Science, Linear Regression, Covid 19, Machine Learning, Data Analysis

Implementing a machine learning model to predict COVID-19 deaths in Brazil

Photo by James Yarema on Unsplash

This is the first post from a series beginner’s projects to implement Machine Learning models. The motivation is to gain some practical experience, after taking Standford’s Machine Learning course.

COVID-19 data is a good source of data to put into practice data analysis and machine learning concepts, as we have lots of available data. This is good for the implementation of a machine learning model.

The data used in this study is available on kaggle. The code for the solution I will describe in this post can be found here.

The main dataset is the covid_19_data.csv file on the folder. This file is being updated constantly and will be considered in the data analysis part of this study. In this study , the dataset had data points updated until 11/15/2020.

1 — Reading the data and checking the status for world cases

Getting to know the dataset:

This is a dataset with 8 columns and 156,292 rows:

The dataset has originally 8 columns:

Sno — Serial number

ObservationDate — Date of the observation in MM/DD/YYYY

Province/State — Province or state of the observation

Country/Region — Country of observation

Last Update — Time in UTC at which the row is updated for the given province or country. (Not standardised and so please clean before using it)

Confirmed — Cumulative number of confirmed cases till that date

Deaths — Cumulative number of of deaths till that date

Recovered — Cumulative number of recovered cases till that date

One additional information that is important at this moment is the active cases. This is simply calculated by subtracting the deaths and recovered cases from the confirmed cases on the date:

Visualizing the status on the last date registered on the dataset:

So, by 11/15/2020 there were 54,370,186 cases of COVID-19 in the world, from which : 64,3% were recovered, 33,3% were active and 2,42% resulted in deaths.

2 — Creating a new data frame for the cases from Brazil

The active cases column is already on this new data frame as we saved the original file with this new information. Now we have a dataset with 9 columns and 4964 rows.

The status of COVID-19 cases in Brazil on the last date registered on the dataset was:

Comparing with the world cases the mortality rate of COVID-19 was slightly higher: 2.83%.

3 — EDA for Brazil data

Before implementing the machine learning model, exploratory data analysis was carried out to clean the data.

Variable types

The ObservationDate column must be converted to datetime so that we can visualize evolution over time:

Duplicated values

Checking for duplicated rows:

Null values

There are 85 rows with null values in the Province/State column. Checking all states present in the dataset:

This is a type of data that it is not possible to replace easily, as this is related to the location where the data was collected. Since there is an ‘Unknown’ category, the NAN values will be replaced by ‘Unknown’:

Evolution of confirmed cases by month

To do that, I created a new column in the data frame with the information gathered by month (originally it was gathered by observation date).

For plotting the information correctly, the data frame was sorted by date. The plot is shown below:

Evolution by observation date

There are various data points for the same observation date, so, in order to create a visualization by observation date, first we need to group the data by observation date:

The plot below shows the evolution of confirmed cases, deaths, active cases and recovered cases, by observation date:

4 — Model for predicting deaths by COVID-19 in Brazil

For the implementation of the machine learning model to predict death cases in Brazil a new dataset was considered: time_series_covid_19_deaths.csv. This is a useful dataset for this case, as it has only the data needed for the model implementation.

In this file, the death cases are informed in the columns, each column corresponding to an observation date.

Preparing the data for the model

Columns from 0 to 4 have unnecessary data for our machine learning model, so only columns from 5 until the last one will be selected:

Next step is to create matrices X (input variable) and y (what I want to predict). In this case, y will be the matrix containing the death numbers and X will store the dates.

These matrices are generated by transforming the original data into numpy arrays, as shown below:

Lastly, a forecast matrix was created to predict death cases for the next 15 days. In this case, this means the predictions for the second half of the month of November, as the dataset considered for this study had information until 11/15/2020.

Separating the model into train and test

Using scikit-learn , 15% of the dataset will be separated as test data and the rest will be the train data which will be used to train the model.

Linear Regression

The simplest way of performing regression of a model with continuous variables is linear regression. It will fit the data into a straight line (equation shown below).

Linear regression is a special case of polynomial regression, which will be shown in the next section.

Implementing the model:

Checking the predictions on a plot:

Polynomial Regression

When you have a complex relationship between the data, maybe a polynomial regression model will be a best fit. One should note that this is still a linear regression but allows a non linear relationship between X (dependent variable) and Y (independent variable). For this study, a fourth-order polynomial was considered, as shown in the equation below:

Implementing the model:

Checking the results of the model:

Metrics evaluation

Errors for the linear regression model:

Error for the polynomial linear regression model:

MAE (mean absolute error) measures the average magnitude of errors in a set of predictions.

In RMSE (root mean squared error) the errors are squared means. In this case, a much larger weight is assigned to larger errors. This means that an error of 50 units is penalized more than an error of 5 units, for example.

Polynomial linear regression demonstrated to be a better fit, as MAE and RMSE were lower than the values for linear regression. We should note that linear regression underestimated the mortality of COVID-19 in Brazil. On the other hand polynomial linear regression overestimated the predictions. This type of analysis should used very carefully as we are dealing with a major health issue.

Plotting forecast using polynomial regression model

As the polynomial regression resulted in a better fit, let’s see a plot for the predictions of death cases :