Beating Datadriven’s DengAI competition Benchmark in 5 minutes
By: Steven Van Dorpe
Tags: Facebook Prophet, Time Series Forecasting, Python, Data Science, Dengai

One of the longest running and most intriguing data science and machine learning competitions is without doubt the DengAi one. The competitors are challenged to predict the number of dengue cases for two South American cities based on environmental information.

In this article we’re trying to beat the competition’s benchmark score by using only one feature and time series forecasting. With the help of Facebook’s Prophet library, a Jupyter notebook and all in all twenty lines of code, we learn how to beat the benchmark score within minutes.

DengaAI

The competition challenges you to predict the number of dengue fever cases reported each week in San Juan (Puerto Rico) and Iquitos (Peru) using environmental data collected by various U.S. Federal Government agencies. The data provided has features such as maximum temperature, humidity and precipitation on a weekly base.

The mean absolute error (MAE) is used for scoring. Taking a look at the leader board top scores, it becomes clear the challenge isn’t an easy one: the top achievers only get to a MAE of 13 and a bit, while the people ranked around position 100 only make it to 17. If you know the average number of cases per week in Iquitos of the provided data was only around 8, this means even the top models have little predictive power.

However, the competition organisators provide a benchmark score, including a complete notebook to help you getting started. In this example they only use a few features in a negative binomial generalized linear model, resulting in a score of 25.81. The goal of this article is to beat this score as fast as possible only by using time series forecasting techniques. To help us with this task we’ll use Facebook’s Prophet library for Python.

What is Facebook Prophet

In early 2017 Facebook open sourced Prophet, a forecasting tool available in Python and R. Prophet is particularly good for forecasting time series with a strong seasonality, historical trend changes and trends that follow non-linear growth curves. Other good reasons to use Prophet are the excellent documentation provided and the overall ease of use.

Prophet for Python is available as any other library and can be installed or updated dropping one line the command prompt. Since I use the Anaconda distribution of Python, I used a simple conda install.

conda install -c conda-forge fbprophet

If you have any trouble or errors installing Prophet, take a look at the Github issues or just throw your error in a Google search.

Preparing the data

Before can start forecasting, we need to get our hands on the data. Go to the DengAI competition page and download the train features, train labels and test features. Then, open a Jupyter Notebook and import pandas, numpy, matplotlib and fbprophet (whole library or only Prophet class).

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

from fbprophet import Prophet

Load the data into your Notebook.

train = pd.read_csv('dengue_features_train.csv')

test = pd.read_csv('dengue_features_test.csv')

labels = pd.read_csv('dengue_labels_train.csv')

Since we’re going to try to make prediction purely based on time data, we can slice city, week_start_date and total_cases off our dataframe. Note that we pick week_start_date as time data, because this feature is already represented as a standard datetime string and does not need any conversion.

train['total_cases'] = labels['total_cases'] prophet_data = train[['city', 'week_start_date', 'total_cases']]

Next, split your data for Iquitos and San Juan and drop the city column as we won’t need this for our forecast.

prophet_data_sj = prophet_data[prophet_data['city'] == 'sj']\

.drop('city', axis=1)

prophet_data_iq = prophet_data[prophet_data['city'] == 'iq']\

.drop('city', axis=1)

Then, do the same for your test data, but only keep the week_start_date column.

test_sj = test[test['city'] == 'sj']['week_start_date']

test_iq = test[test['city'] == 'iq']['week_start_date']

To make the Prophet library function properly, it’s mandatory to name the time feature ds and the values to predict y. Do this for both the dataframes.

prophet_data_sj.columns, prophet_data_iq.columns =

['ds', 'y'], ['ds', 'y']

Now our data is ready for some Prophet magic!

Prophet magic

We can start using the fbprophet library to forecast values for our test data. For convenience, we’ll create a function that takes train data, test data and two forecasting parameters as input.

The first step of setting up a Prophet prediction is creating a new instance of the Prophet class. You can set a whole bunch of parameters (e.g. weekly_seasonality or daily_seasonality) during the initiation of this class, but we will only set the changepoint_prior_scale (default 0.05) and yearly_seasonality (default 10).

To understand the meaning of the first parameter you need to know that time series frequently have abrupt changes in their trend. These changes are called changepoints. Prophet will automatically detect these changepoints and will allow the trend to adapt appropriately.

Prophet detects changepoints by first specifying a large number of potential changepoints and then puts a sparse prior on the magnitudes of the rate changes. This essentially means that Prophet has a large number of possible places where the rate can change, but will use as few of them.

Adjusting the strength of the sparse prior will change the flexibility of the model and can be be tuned using the changepoint_prior_scale argument. By adjusting this argument upwards, more changepoints will be used and thus the model will fit the train data better.

Since this argument is used to change the flexibility of the model and to counter over- or underfitting, we call this argument flex in our function.

The second argument, yearly_seasonality, is estimated using a partial Fourier sum. Take a look at this paper for more information about partial Fourier sums and periodic signals. In essence, the seasonality argument determines how quickly the seasonality can change.

In our case, we would most likely be better of with a lower seasonality because our data is closely knit to yearly weather patterns that don’t seem to change (take a look at the climate train and test data).

The next step in our function is to fit our train data to our Prophet by invoking the fit() function on it. Then we want to create a dataframe representing the dates in the future where we want to make a prediction for. This can easily be done by invoking the make_future_dataframe() function on our Prophet.

The function takes two arguments: periods and frequency. In our case, the number of periods should be exactly the length of our test data, and the frequency should be set to weekly (‘W’), since we have data points on a weekly base.

Then, it’s finally time to fill in our future dataframe with predicted values. We do this by invoking predict() on the Prophet with the future dataframe as input. We also run plot() on the Prophet to get a visualization of our predictions.

def create_forecast(train_data, test_data, flex, seas):



prophet = Prophet(changepoint_prior_scale=flex,

yearly_seasonality=seas)



prophet.fit(train_data)



future_frame = prophet.make_future_dataframe(

periods=len(test_data),

freq='W')



forecast = prophet.predict(future_frame)



prophet.plot(forecast)



forecast = forecast[['ds', 'yhat']].yhat\

.apply(lambda x : int(x)) return forecast[len(train_data):]

If you take a look at the forecast dataframe, you should see it has a whole bunch of variables including a timestamp (ds) and the predicted value (yhat). Slice off these two values and convert the prediction to an integer (the competition only accept integers as prediction).

The dataframe also includes all the dates both from our train and test data. We only need the predictions for the test data. Slice off the test data and return this dataframe as output of our function. Now our function is complete and we can start using it to make predictions for our dengue data.

Make two forecasts, one for Iquitos and one for San Jose, and store them in a variable. Here, we use the default value for seasonality (10) and flexibility (0.05).

forecast_sj = create_forecast(prophet_data_sj, test_sj, 0.05, 10)

forecast_iq = create_forecast(prophet_data_iq, test_iq, 0.05, 10)

Looking at the forecasts, we can see that we indeed created two pandas series with predictions for our test data. Along with the series, there should also pop up a graph with the original data scattered and the predictions as a continuous line. Play around with the seasonality and flexibility parameters to see how they influence predictions (blue line), confidence (light blue), trend (red line) and changepoints (vertical dotted red lines).