Time Series Analysis with Auto.Arima in R
By: Luis Losada
Tags: R, Time Series Forecasting, Auto Arima, Arima, Regression

Regression Models with ARIMA errors:

Now comes the real deal. ARIMA models are great for forecasting blindly into the future using historical data. However, sometimes we don’t need to forecast blindly, sometimes we have variables that can help us predict future behavior. Regression models are ideal in this scenario.

Let’s put this into perspective then. Let’s says you have time series data with a seasonal pattern that has an attribute that can be used as a regressor. Here’s an example: Taking the Quarterly Data of the Total E-Commerce Sales and US Total Retail Sales as recorded by FRED conforming the periods from 1999 Q4 to 2019 Q1, we see that the E-Commerce data is aggregated and reported first than US Total retail sales, so let’s use E-commerce data as a regressor and US Retail sales as the dependent variable to predict the sales US Retail Sales of 2019 Q2.

Now let’s compare both datasets. By calculating the growth percentage quarter over quarter we can normalize the data and compare their behavior. Here’s how this looks like:

Source: Data extracted from the Federal Reserve Economic Data website corresponding periods between 1999 Q4 and 2019 Q2 ( https://fred.stlouisfed.org/tags/series?t=quarterly%3Bretail+trade).

As you can see there’s a clear lag between the both sets of data, but the lag seems to be consistent. This is a perfect scenario for using a Regression Model with ARIMA errors because it encapsulates two of the fortes of this method: a heavily seasonal data that also includes a trend and a regressor that offers a consistent correlation to the dependent variable. In this case, auto.arima from the forecast package in R allows us to implement a model of this type with relative ease. Here is where our guide truly begins.

First we’ll let auto.arima do its job and choose the best model for our data. We then fit an auto.arima model using the default parameters. The model will aim to forecast the values of the US Total Sales between the 2018Q2 and 2019Q2 periods using the Retail Sales variable as the regressor.

#Fitting an auto.arima model in R using the Forecast package fit_basic1<- auto.arima(trainUS,xreg=trainREG_TS)

forecast_1<-forecast(fit_basic1,xreg = testREG_TS)

Results of the Regression Model with ARIMA Errors fitted using the Retail Sales data as a regressor to predict US Total Sales.

Now let’s look at the residuals of the model. The residuals will tell us if the model was able to capture all of the information provided by the data.

Three graphs one story: 1) Time Series graph of the residuals, 2) ACF Graph for the first 20 lags, 3) Distribution of the residuals.

It seems that the model left information in the residuals. The first and last graph shows us that the residuals don’t appear to be white noise but to have some correlation amongst them and the second graph confirms it. The first 12 lags go above the threshold established by the auto correlation function. Let’s fix this by tuning our model using these 4 steps:

1.STL Decomposition:

Auto.arima has the ability to decide whether or not the data used to train the model needs a seasonal differencing. However, sometimes the data might not be able to clearly express its behavior and auto.arima needs a nudge in the right direction to understand it. This is where STL decomposition comes in handy. STL works as an additive process where the data is decomposed into trend, seasonality and remainder and each component of the data can be taken apart for analysis.

By decomposing the data into these three components we are able to measure how much weight each component has on the overall data by measuring the strength of it. So, for example we could measure the strength of seasonality on the data and get a score for it using this formula:

A seasonal strength of above 0.64 has been regarded as strong and is the threshold I use to decide if the data has enough seasonal strength. More on this can be found here.

Using this threshold we can then tell auto.arima when to start looking into a seasonal model. Here’s an example using our Retail data:

Using STL decomposition we can measure the strength of the seasonal and trend parts of the data.

As you can see in the table provided, the data has a high seasonal pattern. So, if we know that ARIMA has to be seasonal then let’s tell auto.arima where to start.

The auto.arima function has arguments for every order of the ARIMA function represented by their values in their (p,d,q) (P,D,Q) representations. So, let’s force auto.arima into iterating over ARIMA models with a differencing of the first order on the seasonal pattern. We can do this by specifying the argument D=1 as one of the parameters of the auto.arima function.

Note: In most cases auto.arima is able to do this without any specification as it uses an u-test to decide whether to use or not a SARIMA model, but in some cases, especially if there’s limited data available and seasonality is not so obvious, the function could use our help. This is useful when using auto.arima to forecast over several datasets with different characteristics, by measuring the seasonality strength we can specify the auto.arima function where to start looking.

2. Approximation:

Next parameter to look at is the approximation parameter. This one is very straightforward but makes a huge difference. The way auto.arima picks the best model is by fitting several models and calculating its AICc score. The model with the lowest score wins. However, so that the function can find a solution faster, the algorithm skips some steps and approximates the results so that less models are fitted. This is useful on datasets with lots of data but compromises performance in favor of speed. Changing this parameter to FALSE makes auto.arima work harder to find the right solution.

3. Lambda:

A way to better understand your data and deal with outliers or a big range is to transform it. For Time Series the go to transformation technique is a Box Cox transformation. Box Cox transformations help determine what is the best way to transform your data based on the lambda.

Lambda here is used to represent the number that will be used to select the optimal transformation for your data. The optimal transformation of the data is that transformation that makes the data approximate the most to a normal distribution.

Formula for a Box Cox transformation. A power transformation is used if the value is not 0 and a log transformation is used if the value is 0.

In R this simple step can help you chose the ideal lambda for your data:

Lambda<- BoxCox.lambda(trainUS)

Then our auto.arima function lets us specify the value for lambda as one of its parameters.

auto.arima(trainUS, xreg = trainREG_TS,D=1, approximation = FALSE, lambda = Lambda)

4. Mean and Drift:

These two other methods allow for constants to be added to the model and for more complex models to be considered. Here’s a brief explanation of both of them.

Drift: Only available when the differencing is above 0 and allows models with a changing average to be fit.

Mean: Allows models with a non-zero mean to be considered.

By default, R sets them as FALSE, again opting for speed over performance. Setting these parameters to TRUE allows the model to work harder, but watch out for overfitting.

Final Results

Ok, so now that we know that:

The original auto.arima model left a lot of information in the residuals. auto.arima can work harder by having a couple of parameters tweaked. And the data can be transformed to avoid the trends in the data to affect the final results.

Let's try refitting our data with all the of the information learned and see what we get:

#Fitting the model with the new parameters fit_complex1<- auto.arima(trainUS,xreg=trainREG_TS,D=1,approximation = F,allowdrift = T,allowmean = T,lambda = Lambda) forecast_complex1<-forecast(fit_complex1,xreg = testREG_TS) checkresiduals(fit_complex1)

It is now obvious that this model was able to capture much more information from the data and thus produce a much better result.

accuracy(f = forecast_complex1,x = testUS)

Results of the Regression Model with ARIMA Errors fitted using the Retail Sales data as a regressor to predict US Total Sales and the new parameters.

Things to Remember:

A couple of things to remember.

Time Series data have a continuity and a dependence and having any missing values will affect your model severely.

SARIMA models work wonders with highly seasonal data, however there are much better methods for linear or highly unstructured time series data. Methods like FB Prophet, Tree Models, or Neural Networks could perform much better than SARIMA in these cases.

The output of your models is only as good as your input. Adding regressors to an ARIMA model only makes sense if there is some clear correlation between the variables.

The residuals in ARIMA models tell a story about the performance of your model and should be taken into consideration when evaluating them. The functions checkresiduals, ACF and PACF make it easy to keep track of the information left behind in the residuals by your model.

** The code used for this article is available here.