Automatic vs. Manual ARIMA Configuration | by Michael Grogan | Towards Data Science

When implementing an ARIMA model, it is particularly common to automate the selection of the p, d, q coordinates using a library such as auto.arima in R and pmdarima in Python.
For context, p is the autoregressive term which indicates the extent to which the model relies on past values to predict future ones. In other words, p stands for the number of lags that should be included in the model. For instance, an AR(1) model relies on the value at t-1 to predict future values in the series.
The q parameter, on the other hand, is the moving average component of the model. In other words, it represents the lagged value of the forecast errors in the model and allows for better forecasting of sudden shifts in the trajectory of the time series.
A model with just p and q parameters is known as an ARMA model, or an Autoregressive Moving Average Model. Such a model does not contain any trend as a model with an autoregressive component assumes that the time series is stationary, i.e. has a constant mean, variance, and autocorrelation.
An ARIMA model stands for Autoregressive Integrated Moving Average Model, and the key difference is that the model is designed to work with non-stationary data. It does this by specifying a value for the d parameter, or the number of differences that are necessary to make the model stationary.
As mentioned, one can either elect to manually select these parameters by inspecting the data. Alternatively, numerous packages have been designed to automate the selection of these parameters.
Let’s take a look at the latter approach first in forecasting electricity consumption patterns for the Dublin City Council Civic Offices. Then, the coordinates will be manually selected, and the performance for each forecast compared.
Here is a plot of the daily electricity consumption time series:
To get a better idea of the overall trend in the series, here is a plot of the 7-day moving average:
Using the first 80% of values (the training set), the model is trained using pmdarima and the forecasted values compared to the test set using RMSE (the root mean squared error).
>>> Arima_model=pm.auto_arima(train_df, start_p=0, start_q=0, max_p=10, max_q=10, start_P=0, start_Q=0, max_P=10, max_Q=10, m=7, stepwise=True, seasonal=True, information_criterion='aic', trace=True, d=1, D=1, error_action='warn', suppress_warnings=True, random_state = 20, n_fits=30)Performing stepwise search to minimize aic ARIMA(0,1,0)(0,1,0)[7]             : AIC=8483.714, Time=0.04 sec ARIMA(1,1,0)(1,1,0)[7]             : AIC=8421.989, Time=0.12 sec ARIMA(0,1,1)(0,1,1)[7]             : AIC=8319.248, Time=0.34 sec ARIMA(0,1,1)(0,1,0)[7]             : AIC=8360.091, Time=0.10 sec ARIMA(0,1,1)(1,1,1)[7]             : AIC=inf, Time=0.78 sec ARIMA(0,1,1)(0,1,2)[7]             : AIC=inf, Time=1.36 sec ARIMA(0,1,1)(1,1,0)[7]             : AIC=8328.690, Time=0.27 sec ARIMA(0,1,1)(1,1,2)[7]             : AIC=inf, Time=2.29 sec ARIMA(0,1,0)(0,1,1)[7]             : AIC=8451.400, Time=0.18 sec ARIMA(1,1,1)(0,1,1)[7]             : AIC=inf, Time=0.81 sec ARIMA(0,1,2)(0,1,1)[7]             : AIC=inf, Time=0.75 sec ARIMA(1,1,0)(0,1,1)[7]             : AIC=8415.537, Time=0.09 sec ARIMA(1,1,2)(0,1,1)[7]             : AIC=inf, Time=1.16 sec ARIMA(0,1,1)(0,1,1)[7] intercept   : AIC=8321.247, Time=1.30 secBest model:  ARIMA(0,1,1)(0,1,1)[7]          Total fit time: 9.588 seconds
This model has a p value of 0, a d value of 1, and a q value of 1. The seasonal components for P, D, Q remain the same.
When generating an autocorrelation function for the series, it was found that there was a spike in correlations at the 7th lag. In this regard, a seasonal factor of 7 was set for the model.
The forecast is generated:
The root mean squared error is calculated:
>>> mse = mean_squared_error(test_df, prediction)>>> rmse = math.sqrt(mse)>>> print('RMSE: %f' % rmse)RMSE: 1288.000396
Given that the mean kilowatt consumption was 3,862 across the test set, the RMSE accounts for roughly 33% of the size of the mean. This indicates that there is room for improvement. Let’s make an attempt to manually select the ARIMA coordinates and see if the forecast improves accordingly.
To determine the appropriate coordinates for the ARIMA model, let us take a look at 1) the partial autocorrelation function for the series, and 2) the autocorrelation function for the differenced series.
The partial autocorrelation shows the effect of direct correlations with between an observation and its previous values, while removing indirect correlations that would be picked up by using the autocorrelation function. Here, we can see that the partial autocorrelation shows a sudden drop at lag 2. In this regard, p = 2 is specified for the model.
Here is the autocorrelation function for the differenced series — or the series with the trend and seasonality components removed. We observe that there is no significant positive correlation for the first few lags. Therefore, the correlation does not “drop off” as was the case for the partial autocorrelation function.
Therefore, a value of q =0 is specified, i.e. the model does not contain a moving average term.
Given that the autocorrelation function showed evidence of stationarity by first-differencing the series, d is set to 1.
As in the previous ARIMA model, a seasonal factor of 7 is used. The seasonal P, D, Q parameters are also assumed to be the same as the ones just identified.
Now, the ARIMA model forecast is implemented:
model=sm.tsa.statespace.sarimax.SARIMAX(endog=train_df,order=(2,1,0),seasonal_order=(2,1,0,7),trend='c',enforce_invertibility=False)results=model.fit()print(results.summary())                           SARIMAX Results                                     ====================================================================Dep. Variable:    y           No. Observations:                  554Model: SARIMAX(2, 1, 0)x(2, 1, 0, 7)   Log Likelihood      -4149.500Date:  Wed, 16 Dec 2020       AIC                           8311.000Time:  15:46:44               BIC                           8336.816Sample:  0                    HQIC                          8321.091                              - 554                                         Covariance Type:              opg                                         ==================================================================== coef    std err          z      P>|z|      [0.025      0.975]------------------------------------------------------------------------------intercept      1.2503  25.029    0.050  0.960  -47.805      50.306ar.L1         -0.3564  0.028   -12.564  0.000  -0.412      -0.301ar.L2         -0.4235  0.032   -13.178  0.000  -0.486      -0.360ar.S.L7       -0.2847  0.034   -8.319   0.000   -0.352      -0.218ar.S.L14      -0.1649  0.038   -4.305   0.000    -0.240      -0.090sigma2      2.429e+05 8541.926  28.434  0.000    2.26e+05    2.6e+05====================================================================Ljung-Box (Q):     137.95   Jarque-Bera (JB):               855.11Prob(Q):             0.00   Prob(JB):                         0.00Heteroskedasticity (H): 0.23   Skew:                          0.02Prob(H) (two-sided):    0.00   Kurtosis:                      9.13====================================================================Warnings:[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Predictions are generated:
predictions=results.predict(555, 680, typ='levels')
Here is the RMSE as compared to the test set:
>>> mse = mean_squared_error(test_df, predictions)>>> rmse = math.sqrt(mse)>>> print('RMSE: %f' % rmse)RMSE: 965.263754
The RMSE of 965 is significantly lower than the previous RMSE value of 1,288 obtained when automatically configuring the ARIMA model.
This error still represents about 25% of the size of the mean of 3,862.
That said, we can see that a significant reason for this is due to the period between time periods 70–80 in the test set, where electricity consumption was unusually low. As a result, RMSE is penalising the error on this anomaly more heavily.
Aside from this, the model does appear to show significant accuracy in predicting the highs and lows of the series.
As we have seen, automatically selecting the ARIMA coordinates did not necessarily deliver the best model accuracy. While automating model selection can be a good starting point, this example demonstrates the importance of understanding the components of a time series and how these components relate to the proper configuration of an ARIMA model.
My advice: automate as a starting point, but don’t stop there. Manually analysing the data and configuring accordingly can often lead to better forecasts.
Many thanks for your time, and the relevant GitHub repository with the dataset and code examples can be found here.
Disclaimer: This article is written on an “as is” basis and without warranty. It was written with the intention of providing an overview of data science concepts, and should not be interpreted as professional advice. The findings and interpretations in this article are those of the author and are not endorsed by or affiliated with any third parties mentioned in this article.
data.gov.ie: Energy Consumption (Gas and Electricity) Civic Offices 2009–2012
Machine Learning Mastery: A Gentle Introduction to Autocorrelation and Partial Autocorrelation
Machine Learning Mastery: Time Series Forecasting Performance Measures With Python
people.duke.edu: Identifying the numbers of AR or MA terms in an ARIMA model
Towards Data Science - ARIMA vs. LSTM: Forecasting Electricity Consumption