Time Series Analysis with Auto.Arima in R | by Luis Losada | Towards Data Science

Forecasting, predicting, clustering, modeling has all become too easy with the recent wave of tools for data analysis and data science. Using complex methods that forcefully iterate over different models to pick the best model is now common practice and something extremely easy to do. Just take algorithms like xgboost or randomforest, algorithms that iterate over hundreds of possibilities to pick out the best result in minutes. They always plateau at a point and we sometimes take it for granted that that is the best the algorithm can do and that it has found the best solution for our prediction. But, for all these automated algorithms to perform at its maximum capacity there are several steps that need to be taken into account before feeding the data to the algorithm. This post will explore how to take advantage of the auto selection algorithm for ARIMA from the forecast package in R and will provide a suggestive guide to manipulating time series data and getting it ready for modeling and forecasting.
ARIMA is an acronym for Auto Regressive (AR) Integrated (I) Moving Average (MA) which indicates that an ARIMA model has three components to it. I will go over the components very briefly in this post and if you need a more in-depth understanding of the concepts, I suggest using these resources which are available for free.
ARIMA models can be expressed in two forms: Non-seasonal models where the model exhibits an order in the form of (p,d,q) where:
p = The order of the Auto Regressive Model
d = The order of differencing
q = The order of the Moving Average
Auto Regressive Models (AR):
Auto regressive models are similar to a regression model but the regressor in this case is the same dependent variable with a specific lag.
Differencing (I):
For ARIMA to perform at its best it needs the data to be stationary. That means that the mean and variance are constant over the entire set. Differencing is used to transform the data so that it is stationary.
Moving Average (MA):
Moving averages are widely used in time series analysis and is an already well-known concept. It involves getting the average of the points in a series for a specific lag and can be expressed like:
Where m = 2k +1,
k = the number of values to use in the average before and after the point used and
t = is the time at which the trend cycle is calculated.
In ARIMA, a Moving Average Model is used
Seasonal ARIMA models (SARIMA):
These models take into account the seasonality in the data and does the same ARIMA steps but on the seasonal pattern. So, if the data has a seasonal pattern every quarter then the SARIMA will get an order for (p,d,q) for all the points and a (P,D,Q) for each quarter.
Regression Models with ARIMA errors:
Now comes the real deal. ARIMA models are great for forecasting blindly into the future using historical data. However, sometimes we don’t need to forecast blindly, sometimes we have variables that can help us predict future behavior. Regression models are ideal in this scenario.
Let’s put this into perspective then. Let’s says you have time series data with a seasonal pattern that has an attribute that can be used as a regressor. Here’s an example: Taking the Quarterly Data of the Total E-Commerce Sales and US Total Retail Sales as recorded by FRED conforming the periods from 1999 Q4 to 2019 Q1, we see that the E-Commerce data is aggregated and reported first than US Total retail sales, so let’s use E-commerce data as a regressor and US Retail sales as the dependent variable to predict the sales US Retail Sales of 2019 Q2.
Now let’s compare both datasets. By calculating the growth percentage quarter over quarter we can normalize the data and compare their behavior. Here’s how this looks like:
As you can see there’s a clear lag between the both sets of data, but the lag seems to be consistent. This is a perfect scenario for using a Regression Model with ARIMA errors because it encapsulates two of the fortes of this method: a heavily seasonal data that also includes a trend and a regressor that offers a consistent correlation to the dependent variable. In this case, auto.arima from the forecast package in R allows us to implement a model of this type with relative ease. Here is where our guide truly begins.
First we’ll let auto.arima do its job and choose the best model for our data. We then fit an auto.arima model using the default parameters. The model will aim to forecast the values of the US Total Sales between the 2018Q2 and 2019Q2 periods using the Retail Sales variable as the regressor.
#Fitting an auto.arima model in R using the Forecast packagefit_basic1<- auto.arima(trainUS,xreg=trainREG_TS)forecast_1<-forecast(fit_basic1,xreg = testREG_TS)
Now let’s look at the residuals of the model. The residuals will tell us if the model was able to capture all of the information provided by the data.
It seems that the model left information in the residuals. The first and last graph shows us that the residuals don’t appear to be white noise but to have some correlation amongst them and the second graph confirms it. The first 12 lags go above the threshold established by the auto correlation function. Let’s fix this by tuning our model using these 4 steps:
1.STL Decomposition:
Auto.arima has the ability to decide whether or not the data used to train the model needs a seasonal differencing. However, sometimes the data might not be able to clearly express its behavior and auto.arima needs a nudge in the right direction to understand it. This is where STL decomposition comes in handy. STL works as an additive process where the data is decomposed into trend, seasonality and remainder and each component of the data can be taken apart for analysis.
By decomposing the data into these three components we are able to measure how much weight each component has on the overall data by measuring the strength of it. So, for example we could measure the strength of seasonality on the data and get a score for it using this formula:
A seasonal strength of above 0.64 has been regarded as strong and is the threshold I use to decide if the data has enough seasonal strength. More on this can be found here.
Using this threshold we can then tell auto.arima when to start looking into a seasonal model. Here’s an example using our Retail data:
As you can see in the table provided, the data has a high seasonal pattern. So, if we know that ARIMA has to be seasonal then let’s tell auto.arima where to start.
The auto.arima function has arguments for every order of the ARIMA function represented by their values in their (p,d,q) (P,D,Q) representations. So, let’s force auto.arima into iterating over ARIMA models with a differencing of the first order on the seasonal pattern. We can do this by specifying the argument D=1 as one of the parameters of the auto.arima function.
Note: In most cases auto.arima is able to do this without any specification as it uses an u-test to decide whether to use or not a SARIMA model, but in some cases, especially if there’s limited data available and seasonality is not so obvious, the function could use our help. This is useful when using auto.arima to forecast over several datasets with different characteristics, by measuring the seasonality strength we can specify the auto.arima function where to start looking.
2. Approximation:
Next parameter to look at is the approximation parameter. This one is very straightforward but makes a huge difference. The way auto.arima picks the best model is by fitting several models and calculating its AICc score. The model with the lowest score wins. However, so that the function can find a solution faster, the algorithm skips some steps and approximates the results so that less models are fitted. This is useful on datasets with lots of data but compromises performance in favor of speed. Changing this parameter to FALSE makes auto.arima work harder to find the right solution.
3. Lambda:
A way to better understand your data and deal with outliers or a big range is to transform it. For Time Series the go to transformation technique is a Box Cox transformation. Box Cox transformations help determine what is the best way to transform your data based on the lambda.
Lambda here is used to represent the number that will be used to select the optimal transformation for your data. The optimal transformation of the data is that transformation that makes the data approximate the most to a normal distribution.
In R this simple step can help you chose the ideal lambda for your data:
Lambda<- BoxCox.lambda(trainUS)
Then our auto.arima function lets us specify the value for lambda as one of its parameters.
auto.arima(trainUS, xreg = trainREG_TS,D=1, approximation = FALSE, lambda = Lambda)
4. Mean and Drift:
These two other methods allow for constants to be added to the model and for more complex models to be considered. Here’s a brief explanation of both of them.
Drift: Only available when the differencing is above 0 and allows models with a changing average to be fit.
Mean: Allows models with a non-zero mean to be considered.
By default, R sets them as FALSE, again opting for speed over performance. Setting these parameters to TRUE allows the model to work harder, but watch out for overfitting.
Ok, so now that we know that:
The original auto.arima model left a lot of information in the residuals.auto.arima can work harder by having a couple of parameters tweaked.And the data can be transformed to avoid the trends in the data to affect the final results.
The original auto.arima model left a lot of information in the residuals.
auto.arima can work harder by having a couple of parameters tweaked.
And the data can be transformed to avoid the trends in the data to affect the final results.
Let's try refitting our data with all the of the information learned and see what we get:
#Fitting the model with the new parametersfit_complex1<- auto.arima(trainUS,xreg=trainREG_TS,D=1,approximation = F,allowdrift = T,allowmean = T,lambda = Lambda)forecast_complex1<-forecast(fit_complex1,xreg = testREG_TS)checkresiduals(fit_complex1)
It is now obvious that this model was able to capture much more information from the data and thus produce a much better result.
accuracy(f = forecast_complex1,x = testUS)
A couple of things to remember.
Time Series data have a continuity and a dependence and having any missing values will affect your model severely.
SARIMA models work wonders with highly seasonal data, however there are much better methods for linear or highly unstructured time series data. Methods like FB Prophet, Tree Models, or Neural Networks could perform much better than SARIMA in these cases.
The output of your models is only as good as your input. Adding regressors to an ARIMA model only makes sense if there is some clear correlation between the variables.
The residuals in ARIMA models tell a story about the performance of your model and should be taken into consideration when evaluating them. The functions checkresiduals, ACF and PACF make it easy to keep track of the information left behind in the residuals by your model.
** The code used for this article is available here.