Forecasting Business KPIs With Python Using Prophet - Part 2 | by AnBento | Towards Data Science

Update: Many of you contacted me asking for valuable resources to learn more about time series forecasting with Python. Below I share 2 courses that I personally took and that would strongly recommend to expand your knowledge on the topic:
Python For Time Series Analysis (using ARIMA, SARIMAX, VARMA, PROPHET) 15+ hours of content
Predictive Analytics For Business (ETS, ARIMA) →Very high quality course!
Hope you’ll find them useful too! Now enjoy the article :D
In the first part of this tutorial (Forecasting Business KPIs With Python Using Prophet-Part I), we built a simple predictive model using Facebook’s Prophet library to forecast two daily KPIs (Sales Number & Sales Value GBP) 426 days in the future. The final forecasts were assigned to the empty future dates in the original dataframe to obtain the following result (GitHub):
df.loc[(df['observation_date'] > cutoff_date )].head()df.loc[(df[‘observation_date’] > cutoff_date )].tail()
We also tried to assess the model performance, obtaining a MAPE of 6.73%. Despite this was a fair result for our first model, by overlapping predicted and actual values, we noticed the our forecast was still not responding well to sudden peaks and dips in the observed sales volume:
Because the model is not fitting well many of the past observed values, it is reasonable to claim that the same is true for future values. For this reason in this article we are going to improve the original fit introducing holidays and customized seasonal components and by tweaking three more parameters (Fourier Order, Changepoint Prior Scale and Changepoint Range).
towardsdatascience.com
In Prophet, the term “holidays” should not be taken literally, as it more broadly denotes unusual events that generate a visible trend deviation from its pattern, that then returns on track once the events are over. From this descriptions it may seem that adding holidays to the model is equivalent to include additional regressors ( achieved with the Prophet.add_regressor() method), however the main difference between the two categories is that:
additional regressors should be used when we wish to model an unexpected event confined on a specific date;
holidays should be used when trend deviations are cyclical and they could start before the event date and continue after it. This means that when adding an holiday, we will be allowed to specify a lower window and an upper window to model potential anomalies around the event itself.
Using our domain knowledge, we identified six events we would like to pass as holidays to the model:
The first working day of the month: represented by the binary field fl_first_working_day;
The last working day of the month: represented by the binary field fl_last_working_day;
The last Friday of the month: represented by the binary field fl_last_friday_day;
New Year Day: represented by the binary field fl_new_year;
Easter Day in a window [-4, +2]: these dates will be imputed manually;
Christmas Day in in a window [-3, +3]: also these dates will be imputed manually;
In order to use these events in the model, we first need to create a holidays dataframe as follows:
What is happening in the code above?
Firstly we created four lists (one for each flag in the original df ) including dates were flag = 1 . These lists were then used to build as many individual dataframes as the number of the events we wished to model. Each dataframe should include four columns, namely holiday , ds , lower_window and upper_window.
In particular, note that, in the case of flags, the ds column was created by automatically passing the list of dates from step one, whereas for Easter and Christmas the date was imputed manually. In both cases, Prophet requires for each holiday, to include data not only for the past, but also for the range of dates to forecast. Eventually, the holidays DF should look like the one below:
By including the holidays DF in our original model, we obtain:
If we now rerun the forecast for the same period and compute the MAPE for the sales_num KPI, the result is 5.28%. This means that by taking into account cyclic events affecting the business, we were able to reduce the mean absolute percentage error by almost 22% :
Output: 5.2806026397
The improvements in the way the model fits the df_train dataset become apparent when actuals and predicted values are overlapped like in Chart 1:
Before ending this section, it is worthwhile highlighting that, if we are forecasting KPIs for a specific Country, we might also want to add national festivities for that market (full list of allowed Country codes available here):
model.add_country_holidays(country_name = 'GB')
Also if we wished to display how the holidays DF looked like, we could easily achieve that using the components plot:
model.plot_components(forecast)
The performance of our model has improved, but there is still some work to be done on seasonalities. For example, our domain knowledge also suggests us that the business undergoes a marked monthly seasonality, but the current model is not taking this information into account.
Fortunately, Prophet allows us to turn off the standardweekly_seasonality and yearly_seasonality and add our customized seasons. This approach gives us much more control as we can set an exact period for each season as well as the fourier_order:
In effect, to model multi-period seasonalities, fbprophet relies on a standard Fourier Series (as described in this paper):
Where P is the regular period of a season (e.g. P = 365.25 for yearly data and P = 7 for weekly data, when the dataset granularity is in days). In the formula the N parameter represents the fourier_order that need to be chosen in order to fit a seasonality. In general, by increasing N, increases our ability of fitting seasonal patterns that change quickly and with it, the risk of overfitting the training dataset.
For this reason, it is usually good practice to select a fourier_order in the interval [5, 20] and this selection process could be automated employing a model selection procedure like AIC (a topic widely addressed in PART II). For our model, I found [5, 14, 10] to be the the optimal values for weekly, monthly and yearly seasonality. This means that the model will be more reactive to fit recurring changes at the monthly and yearly level and less reactive at the weekly level.
Running the model with these parameters, leads to a MAPE of 3.6%, a reduction of 46.5% compared to the initial model presented in PART I. All we had to do achieve that, was to add customized seasonalities and find the right combination parameters.
At these point, we should be reasonably content with our model, but before closing this tutorial, I would like to present three more parameters that can help us reducing MAPE even more:
Seasonality Prior Scale (defaults to 10): this parameter is used to modulate the strength of the seasonal components, meaning that higher values allow the model to fit wider seasonal fluctuations, smaller values dampen the seasonality. It can be added as general parameter, or specified for individual season as part of add_seasonality() (that is what has been shown below). In our case a value of 15 applied to all the seasons, reduces the MAPE slightly to 3.59%.
Holidays Prior Scale (defaults to 10): this parameter modulates the strength of the holidays components in the model, determining the impact holidays should have on the forecasts. For our model, the default value is the one leading to the lowest MAPE.
Changepoint Prior Scale (default = 0.05): this parameter modulates the flexibility of the automatic changepoint selection, meaning that higher values will allow more changepoints, lower values will reduce the number of allowed changepoints. I found that, in some occasions doubling the default value (from 0.05 to 0.1), can lead to an increased accuracy of the model and then a lower MAPE. This was not the case for our model though, where the default value led to the best results.
In this second part of the tutorial, we learnt how to use holidays, customized seasonalities and more advanced parameters to improve the predictive accuracy of our model. These adjustments led to a final MAPE of 3.59% a 46.5% reduction from the initial 6.73%. The final model used to get to this result is presented below:
In the last part (PART III) of this tutorial on forecasting business KPIs, we will learn how to implement more advanced techniques to perform model evaluation and how to automatically select the best possible parameters combination using AIC as a benchmark.
I hope you found this article useful & keep learning!
A note for my readers: This post includes affiliate links for which I may make a small commission at no extra cost to you should, you make a purchase.