Introduction to ARIMA for Time Series Forecasting | by Juan Nathaniel | Towards Data Science

Auto Regressive Integrated Moving Average (ARIMA) model is among one of the more popular and widely used statistical methods for time-series forecasting. It is a class of statistical algorithms that captures the standard temporal dependencies that is unique to a time series data. In this post, I will introduce you to the basic principles of ARIMA and present a hands-on tutorial to develop ARIMA for time-series forecasting in Python.
Table of Content:1. What is ARIMA?2. Introduction to the Dataset3. Implementing ARIMA model in Python4. Forecasting using ARIMANotes before we begin:1. Download the rainfall CSV dataset here2. Install dependencies:   - pip install statsmodels OR conda install statsmodels  - pip install patsy OR conda install patsy
Before we dig right into ARIMA’s formal mathematical definition, let me introduce you to the concept of stationarity. Stationarity simply means observations that do not depend on time. For data that depends on time (eg. seasonal rainfall), the stationarity condition may not hold as different timing will yield different values for these observations.
Another important concept to understanding ARIMA is autocorrelation. How does it different from the typical correlation? First of all, correlation relates two different sets of observations (eg. between housing prices and the number of available public amenities) while autocorrelation relates the same set of observation but across different timing (eg. between rainfall in the summer versus that in the fall).
Now let’s break down the different components of ARIMA: Auto Regressive (AR), Integrated (I), and the Moving Average (MA)
As you may have guessed it, Auto Regressive (AR) regression model is built on top of the autocorrelation concept, where the dependent variable depends on the past values of itself (eg. rainfall today may depend on rainfall yesterday, and so on). The general equation is:
As illustrated, an observation Y at time t, Yt, depends on Yt-1, Yt-2, ..., Yt-p . Why Yt-p and not Y0(ie. the initial value)? The p here is called the lag order which indicates the number of prior lag observations we include in the model (eg. Maybe we exclude observations beyond 5 days prior to the present time because these earlier rainfall observations might not be correlated) .
Recall our explanation on stationarity. The integrated part of ARIMA attempts to convert the non-stationarity nature of the time-series data to something a little bit more stationary. How can we do that? By performing prediction on the difference between any two pair of observation rather than directly on the data itself.
Notice how we could perform the differencing operations many times levels (Y →Z and Z →Q), depending on the hyperparameter d that we set when training the ARIMA model.
Now the final piece of ARIMA is the MA or Moving Average. It attempts to reduce the noise in ourtime series data by performing some sort of aggregation operation to your past observations in terms of residual error ε (read: epsilon).
The ε terms represent the residual errors from the aggregation function and q here is another hyperparameter that is identical to p. But instead of identifying the time window (p) to the time series data itself, q specifies the time window for the moving average’s residual error.
Now let’s get our hands dirty by implementing ARIMA in Python.
If you have not already downloaded the dataset, please do so here.
Let’s perform some basic exploration to the dataset.
import pandas as pddf = pd.read_csv('rainfall.csv')df
Our dataset is a daily rainfall time series data (in mm) from January, 1st 2010 to May, 1st 2020.
Let’s plot how the rainfall data varies with time (we take one month for illustration purposes).
import matplotlib.pyplot as plt# First 30 daysfirst_30 = df[:30]first_30.plot.line(x='date', y='rainfall')
We can also check for any autocorrelation that may occur in the dataset.
from pandas.plotting import autocorrelation_plotautocorrelation_plot(first_30['rainfall'])
There seems to be slight correlation when the lag time is short (0–5 days) and when it is sufficiently long (20–25 days), but not in between the intermediate values.
Valuable information that we can pick up for our ARIMA implementation next!
First, we would need to import the statsmodels library.
from statsmodels.tsa.arima_model import ARIMA
Then, we define the model with these initial hyperparameters for p, d, q (as defined earlier in the What is ARIMA? section).
# fit modelp,d,q = 5,1,0model = ARIMA(first_30['rainfall'], order=(p,d,q))model_fit = model.fit()
Let’s get the result summaries.
print(model_fit.summary())
Notice the AIC, BIC, and HQIC metrics at the top? The lower these values are, the better the fit of the model is. So you can perform further hyperparameter tuning or data preprocessing to achieve better results!
We have our first working model of ARIMA. The final section will walk you through how to make future prediction given our trained model.
Let’s first expand our dataset to include 365 days instead of 30 .
data = df[:365]['rainfall'].values
We then split the data into train (66%) and test set (34%).
We then split the data into train (66%) and test set (34%).
train_size = int(len(data) * 0.66)train, test = data[0:train_size], data[train_size:len(data)]
2. And initialize the historical and prediction values for comparison purposes
history = [x for x in train]predictions = list()
3. Now we train the model and make future forecast as stored in the test data
for t in range(len(test)):    model = ARIMA(history, order=(5,1,0))    model_fit = model.fit()    pred = model_fit.forecast()    yhat = pred[0]    predictions.append(yhat)    # Append test observation into overall record    obs = test[t]    history.append(obs)
4. Lets evaluate our performance
from sklearn.metrics import mean_squared_errorfrom math import sqrtrmse = sqrt(mean_squared_error(test, predictions))print('Test RMSE: %.3f' % rmse)>>> Test RMSE: 20.664
Pretty huge RMSE! Definitely there’s room for improvement here.
We can also plot the difference between observations and predictions, and compare how both are similar (or diverging!)
Not too bad! Our prediction (red) closely resembles the observations (blue) for some days (except in extreme rainfall cases with noticeably high peaks).
That’s it! We have so far discussed the basic principles of ARIMA and how to implement one in Python. One key characteristic of ARIMA, just like any other ML models, is its dependency on hyperparameter tuning. So be sure to perform those steps! In addition to fine-tuning, a good working domain knowledge is also key to produce good-performing ARIMA model. So what do you think of ARIMA?
I regularly summarize AI research papers in plain English and beautiful visualization via my Email Newsletter, do subscribe at https://tinyurl.com/2npw2fnz.