Simplified Forecasting with Facebook Prophet | by Josh Johnson | Towards Data Science

Timeseries forecasting is a complex art form. Many models are very sensitive to trends, cycles (called ‘seasons’) and changing magnitudes of fluctuations, and instead require stationary data, which lack these features.
This devastating disease has killed hundreds of thousands in the US and millions around the world and at this time continues to spread ever faster. Forecasting the rate of future infections can help hospitals and aid organizations plan and prepare for future needs.
A good example of data that not-stationary is the cumulative spread of COVID-19 in the United States over 2020. For this project I will be using data from the Coronavirus Government Response Tracker, which provides daily updated data on every country, including government interventions. If you want to take your forecasting further by including exogenous variables (variables outside of the data being modeled that may have an effect on it) you can find many relevant ones in this dataset. However, for this example we will only be using cumulative reported infections in the United States.
You can see above that not only is there an upward trend, but the trend keeps changing. The seasonality is not apparent from the cumulative data, but when we do a first order differencing to transform cumulative cases to daily new cases, we see the seasonality appear.
The weekly seasonality is probably because:
People tend to get tested on weekends and their results tend to come back on Wednesdays, creating a peak of reported cases on Wednesdays and valleys on other days. Or,Labs report cases in bulk on certain days.
People tend to get tested on weekends and their results tend to come back on Wednesdays, creating a peak of reported cases on Wednesdays and valleys on other days. Or,
Labs report cases in bulk on certain days.
Notice the dips on 12–07, 12–14, 12–21, etc. There are also a large dips on 12–25 and 01–01. I assume not many lab workers were working on Christmas and New Year’s day, so reports dipped.
We must keep in mind that data is not reality, and cases charted are actually cases reported, not actual infections. The patterns of reporting create the seasonality in the data.
An ARMA model is a combination of an autoregressive and a moving average model, and a crucial tool in predicting stationary time series data. I won’t explain it fully here, since this post is about a different technique, but you can read about it here.
Extensions to the ARMA model, such as ARIMA can account for trends and SARIMA can take into account the seasonality of data. However, the accuracy of these models require carefully tuning several hyperparameters to determine how and to what degree past data effects the future. data scientists often make a series of transforms on the data, such as subtracting moving averages or standard deviations, or log scaling it to get good predictions. With enough effort and expertise these are powerful models.
With effort and thorough data exploration I tuned mine to predict cases in the first 18 days of January at 99% accuracy. You can experience my adventures in SARIMA modeling through my Github repo predicting COVID case rates.
The Facebook Prophet library for R and Python does a lot of that work for you. It allows for quick and easy time series forecasts, but also provides options for more seasoned forecasters to tune the internal settings. It can detect adjust to seasonality, trends and shifting standard deviations in the residuals. However, it is particular about how you implement it. Let’s see how to get started.
We will use Pandas to manipulate the data and download the fbprophet library. It installs fine into Google Colab notebooks with !pip install fbprophet.
On my local Windows machine running the Anaconda package manager, I had to install using in my bash terminal with conda install -c conda-forge fbprophet. Facebook has more guidance on installation here.
import pandas as pd
#!pip install fbprophet
from fbprophet import Prophet

division = 'country'  #regional data is available for some countries
region = 'United States'
prediction = 'ConfirmedCases' #ConfirmedDeaths is also available for forecasting.

#get the latest data from OxCGRT
DATA_URL = 'https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/OxCGRT_latest.csv'
full_df = pd.read_csv(DATA_URL,
                usecols=['Date','CountryName','RegionName','Jurisdiction',
                           'ConfirmedCases','ConfirmedDeaths'],
                parse_dates=['Date'],
                encoding="ISO-8859-1",
                dtype={"RegionName": str,
                        "CountryName":str})

#Filter the region we want to predict
if division == 'country':
    df = full_df[(full_df['Jurisdiction'] == 'NAT_TOTAL') & (full_df['CountryName'] == region)][:-1]
elif division == 'state':
    df = full_df[(full_df['Jurisdiction'] == 'STATE_TOTAL') & (full_df['RegionName'] == region)][:-1]

#Since we are not using exogenous variables, we just keep the dates and endogenous data
df = df[['Date',prediction]].rename(columns = {'Date':'ds', prediction:'y'})

You may notice that the last line renames the ‘Date’ columns to ‘ds’ and the ‘CumulativeCases’ column to ‘y’. This is required by Facebook in order to use the dataframe for modeling. Many experienced Pandas users may be tempted to make the datetime column the index, but for Prophet, don’t do that.
Our dataframe is ready for modeling. Creating and training the out-of-the-box model is very simple:
m = Prophet()m.fit(df)
But, here is the tricky part. You have to prepare a special dataframe to store the actual prediction. Luckily the model comes with a handy method to do it for you if you want. You can make it yourself if you want, you just need the values of the periods you want to predict in the ‘ds’ column and an empty ‘y’ column
future = m.make_future_dataframe(periods=length_of_desired_forecast)
Finally, to make the prediction, simply feed the future dataframe as an argument to the the model’s .predict() method, like so:
forecast = m.predict(future)
I love the variable name conventions that the folks at Facebook have created, where the model predicts the future to make forecast. It’s satisfying and just the right amount of on the nose for me.
import pandas as pd
#!pip install fbprophet
from fbprophet import Prophet
import matplotlib.pyplot as plt

division = 'country'  #regional data is available for some countries
region = 'United States'
prediction = 'ConfirmedCases' #ConfirmedDeaths is also available for forecasting.

#get the latest data from OxCGRT
DATA_URL = 'https://raw.githubusercontent.com/OxCGRT/covid-policy-tracker/master/data/OxCGRT_latest.csv'
full_df = pd.read_csv(DATA_URL,
                usecols=['Date','CountryName','RegionName','Jurisdiction',
                           'ConfirmedCases','ConfirmedDeaths'],
                parse_dates=['Date'],
                encoding="ISO-8859-1",
                dtype={"RegionName": str,
                        "CountryName":str})

#Filter the region we want to predict
if division == 'country':
    df = full_df[(full_df['Jurisdiction'] == 'NAT_TOTAL') 
    & (full_df['CountryName'] == region)][:-1]
elif division == 'state':
    df = full_df[(full_df['Jurisdiction'] == 'STATE_TOTAL') 
    & (full_df['RegionName'] == region)][:-1]

#Since we are not using exogenous variables, we just keep the dates and endogenous data
df = df[['Date',prediction]].rename(columns = {'Date':'ds', prediction:'y'})

# set how many days to forecast
forecast_length = 30
# instantiate and fit the model
m = Prophet()
m.fit(df)
# create the prediction dataframe 'forecast_length' days past the fit data
future = m.make_future_dataframe(periods=forecast_length)
# make the forecast to the end of the 'future' dataframe
forecast = m.predict(future)

to_plot = forecast[forecast.ds > '2020-12-01'].merge(df, how='left')

plt.figure(figsize = (10,7))
plt.plot(to_plot['ds'], to_plot['yhat'], label='Forecasted Cases')
plt.plot(to_plot['ds'], to_plot['y'], label='True Cases')
plt.fill_between(to_plot['ds'], to_plot['yhat_upper'], to_plot['yhat_lower'],
                 alpha=.2, label='Confidence')
plt.title('Facebook Prophet Forecasted COVID-19 cases, 1-22-2021 to 2-20-2021')
plt.legend()
plt.savefig('prophet_forecast.png')
plt.show()
print('\n The "forecast" DataFrame \n')
forecast

forecast will be a dataframe that contains your prediction, out to the end of the ‘ds’ column in your future dataframe, as well as much more. It includes confidence intervals, and all the parameters the model used for each step of the prediction. This allows some lovely inspection the internal workings of the model and opportunity to do some fun graphs with confidence bands.
As you can see above, this out of the box Prophet has made a very basic linear regression as a prediction. Further tuning of the model will improve the accuracy. If you run the code you will see warnings that indicate the Prophet has automatically identified weekly seasonality in the data, but that you can manually tell it to look for monthly or yearly seasonality. There are a number of other tweaks and refinements you can try including adding extra (exogenous) variables, such as holidays or the government interventions provided in the dataset we used.
A much more comprehensive guide can by found in this awesome article by Greg Rafferty. But, the above should get you off the ground and give you something to get started with.
If you want to see my full work on trying to predict COVID-19 cases for an Xprize competition, you can check it out here:(Spoiler, I ended up going with a more accurate SARIMA model)
Facebook Prophet Quickstart Guide
Forecasting in Python with Facebook Prophet by Greg Rafferty
Data:
Coronavirus Government Response TrackerThanks to the Blavatnic School of Government and the University of Oxford