An Introduction to Time Series Analysis with ARIMA | by Taha Binhuraib | Towards Data Science

Time series forecasting focuses on analyzing data changes across equally spaced time intervals. Time series analysis is used in a wide variety of domains, ranging from econometrics to geology and earthquake prediction; it’s also used in almost all applied sciences and engineering. Examples of time series data include S&P 500 Index, disease rates, mortality rates, blood pressure tracking, global temperatures. This post will be looking at how the autoregressive integrated moving average(ARIMA) models work and are fitted to time series data. The first point to consider before moving forward is the difference between Multi and Univariate forecasting. The former uses only the previous values in time to forecast future values. The latter makes use of different predictors other than the series itself. Please refer to my GitHub repo for the Jupyter notebooks and relevant data.
White noise, by definition, is random data. A time series is white noise(random) if the variables are independent and identically distributed(i.i.d) with a mean of zero. In other words, the time series has a mean μ = 0 and a constant standard deviation σ = c. You can use the df.describe() function to check your mean:
Since our mean is well above zero, we can rest assured that our data is not white noise. Even though we do not need to do the following, let’s just divide our data into 2 parts and take a look at their respective standard deviations:
df_1 = df[:100]df_2 = df[100:]print(df_1.describe())print(df_2.describe())
Here we can clearly see that our σ1 ≠σ2. The next image will illustrate an example of white noise. As seen below, the signal oscillates around a mean of zero, and it is not that hard to notice that the standard deviation is constant across time.
White noisey = noise(t) = εt
Since the definition of white noise implies randomness, you cannot model it and make predictions. Once you see white noise, do not continue with any predictive model!
Arima is short for Auto-Regressive Integrated Moving Average, which is a forecasting algorithm based on the assumption that previous values carry inherent information and can be used to predict future values. We can develop a predictive model to predict xt given past values., formally denoted as the following:p(xt | xt−1, ... ,x1)
In order to understand ARIMA, we first have to separate it into its foundational constituents:
ARIMA
AR
I
MA
The ARIMA model takes in three parameters:
p is the order of the AR termq is the order of the MA termd is the number of differencing
p is the order of the AR term
q is the order of the MA term
d is the number of differencing
The AR model only depends on past values (lags) to estimate future values. Let’s take a look at the generalized form of the AR model:
The value “p” determines the number of past values p will be taken into account for the prediction. The higher the order of the model, the more past values will be taken into account. For the sake of simplicity, let’s look at an AR(1) model.
The AR model can simply be thought of as the linear combination of p past values.
The moving-average MA model, on the other hand, depends on past forecast errors to make predictions. Let’s look at the generalized form of the MA model:
The MA model can simply be thought of as the linear combination of q past forecast errors.
Now before going any further, we have to discuss the difference between stationary and non-stationary data.
Stationary data is data in which the time series does not depend on the time variable; this assumption is needed in order to develop a predictive model that only depends on past values only. Not having stationary data would render your predictions unsatisfactory as your model would take the time variable into account. Intuitively, parameters such as the mean and variance do not change over time. The two graphs below illustrate the difference between stationary and non-stationary data:
As seen above, the non-stationary data has a clear upwards trend.
Now let’s look at a concrete example of non-stationary data and use Pandas’ diff() function to achieve stationarity:
#Calculates the difference of a Dataframe element compared with another element in the Dataframe (default is element in previous row = 1).df_1.diff(1).plot()
As seen above, we have achieved stationarity by taking the difference between successive rows.
If d=0: yt = Yt
If d=1: yt = Yt — Yt−1
If d=2: yt = (Yt — Yt-1) -(Yt-1 — Yt-2)
Now let’s consider ARIMA(1,1,1) for the time series x. For the sake of brevity, constant terms have been omitted.
yt = yt — y_t−1
yt = φ1yt−1 + εt — θ1 εt−1
We can simply use Auto.Arima and cross-validate in order to find the best parameters for the model. First, let’s load the data and plot it.
df = pd.read_csv('passengers.csv', usecols=[1], engine='python')plt.plot(df)
As seen above, the data has a clear upwards trend with seasonality. Now we shall simply make use of Auto.Arima to fit the data and make predictions
y = df.valuestrain, test = train_test_split(y, train_size=100)# Fit modelmodel = pm.auto_arima(train, seasonal=True, m=12)forecasts = model.predict(test.shape[0])x = np.arange(y.shape[0])plt.plot(x, y, c='blue' , label = 'expected')plt.plot(x[100:], forecasts, c='green' , label = 'predicted')plt.legend()plt.show()
Great fit on the test data!
Arima is a great tool for time series analysis, and Auto Arima packages make the process of fine-tuning a lot easierAlways plot your data and perform Explanatory Data analysis EDA in order to get a better understanding of the data.Learning the technicalities behind different prediction models can help you choose the correct one.
Arima is a great tool for time series analysis, and Auto Arima packages make the process of fine-tuning a lot easier
Always plot your data and perform Explanatory Data analysis EDA in order to get a better understanding of the data.
Learning the technicalities behind different prediction models can help you choose the correct one.