An Introduction to Time Series Analysis with ARIMA
By: Taha Binhuraib
Tags: Statistics, Mathematics, Forecasting, Machine Learning, Arima

Image by Author

Time series forecasting focuses on analyzing data changes across equally spaced time intervals. Time series analysis is used in a wide variety of domains, ranging from econometrics to geology and earthquake prediction; it’s also used in almost all applied sciences and engineering. Examples of time series data include S&P 500 Index, disease rates, mortality rates, blood pressure tracking, global temperatures. This post will be looking at how the autoregressive integrated moving average(ARIMA) models work and are fitted to time series data. The first point to consider before moving forward is the difference between Multi and Univariate forecasting. The former uses only the previous values in time to forecast future values. The latter makes use of different predictors other than the series itself. Please refer to my GitHub repo for the Jupyter notebooks and relevant data.

White noise and why it matters

White noise, by definition, is random data. A time series is white noise(random) if the variables are independent and identically distributed(i.i.d) with a mean of zero. In other words, the time series has a mean μ = 0 and a constant standard deviation σ = c. You can use the df.describe() function to check your mean:

Image by Author

Since our mean is well above zero, we can rest assured that our data is not white noise. Even though we do not need to do the following, let’s just divide our data into 2 parts and take a look at their respective standard deviations:

df_1 = df[:100]

df_2 = df[100:] print(df_1.describe())

print(df_2.describe())

Image by Author

Here we can clearly see that our σ₁ ≠σ₂. The next image will illustrate an example of white noise. As seen below, the signal oscillates around a mean of zero, and it is not that hard to notice that the standard deviation is constant across time.

Image courtesy of Wikipedia

White noise

y = noise(t) = ϵₜ

Since the definition of white noise implies randomness, you cannot model it and make predictions. Once you see white noise, do not continue with any predictive model!

Introduction to ARIMA

Arima is short for Auto-Regressive Integrated Moving Average, which is a forecasting algorithm based on the assumption that previous values carry inherent information and can be used to predict future values. We can develop a predictive model to predict xₜ given past values., formally denoted as the following:

p(xₜ | xₜ₋₁, … ,x₁)

In order to understand ARIMA, we first have to separate it into its foundational constituents:

AR I MA

The ARIMA model takes in three parameters:

p is the order of the AR term q is the order of the MA term d is the number of differencing

Autoregressive AR and Moving average MA

The AR model only depends on past values (lags) to estimate future values. Let’s take a look at the generalized form of the AR model:

The value “p” determines the number of past values p will be taken into account for the prediction. The higher the order of the model, the more past values will be taken into account. For the sake of simplicity, let’s look at an AR(1) model.

The AR model can simply be thought of as the linear combination of p past values.

The moving-average MA model, on the other hand, depends on past forecast errors to make predictions. Let’s look at the generalized form of the MA model:

The MA model can simply be thought of as the linear combination of q past forecast errors.

Now before going any further, we have to discuss the difference between stationary and non-stationary data.

stationary

Stationary data is data in which the time series does not depend on the time variable; this assumption is needed in order to develop a predictive model that only depends on past values only. Not having stationary data would render your predictions unsatisfactory as your model would take the time variable into account. Intuitively, parameters such as the mean and variance do not change over time. The two graphs below illustrate the difference between stationary and non-stationary data:

Image by Author

As seen above, the non-stationary data has a clear upwards trend.

Now let’s look at a concrete example of non-stationary data and use Pandas’ diff() function to achieve stationarity:

Non-stationary data

#Calculates the difference of a Dataframe element compared with another element in the Dataframe (default is element in previous row = 1). df_1.diff(1).plot()

Stationary data after using Pandas Diff function

As seen above, we have achieved stationarity by taking the difference between successive rows.

If d=0: yₜ = Yₜ

If d=1: yₜ = Yₜ — Yₜ₋₁

If d=2: yₜ = (Yₜ — Yₜ-₁) -(Yₜ-₁ — Yₜ-₂)

Now let’s consider ARIMA(1,1,1) for the time series x. For the sake of brevity, constant terms have been omitted.

yₜ = yₜ — y_t₋₁

yₜ = ϕ₁yₜ₋₁ + ϵₜ — θ₁ ϵₜ₋₁

How do we find the parameters (p,d,q)

We can simply use Auto.Arima and cross-validate in order to find the best parameters for the model. First, let’s load the data and plot it.

df = pd.read_csv('passengers.csv', usecols=[1], engine='python')

plt.plot(df)

Image by Author

As seen above, the data has a clear upwards trend with seasonality. Now we shall simply make use of Auto.Arima to fit the data and make predictions

y = df.values

train, test = train_test_split(y, train_size=100) # Fit model

model = pm.auto_arima(train, seasonal=True, m=12) forecasts = model.predict(test.shape[0]) x = np.arange(y.shape[0])

plt.plot(x, y, c='blue' , label = 'expected')

plt.plot(x[100:], forecasts, c='green' , label = 'predicted')

plt.legend()

plt.show()



Image by Author

Great fit on the test data!

Summary

Arima is a great tool for time series analysis, and Auto Arima packages make the process of fine-tuning a lot easier Always plot your data and perform Explanatory Data analysis EDA in order to get a better understanding of the data. Learning the technicalities behind different prediction models can help you choose the correct one.

References

https://machinelearningmastery.com/arima-for-time-series-forecasting-with-python/

https://people.duke.edu/~rnau/411arim.htm

https://towardsdatascience.com/understanding-arima-time-series-modeling-d99cd11be3f8