Your Comprehensive Guide to the Basics of Time Series Modeling
By: Konstantin Rink
Tags: Time Series Modeling, Time Series Analysis, Forecasting, Data Science, Time Series Forecasting

The concept of stationarity

Stationarity is an essential characteristic of time series since (classical) time series models like ARMA assume it and can lead to incorrect results if the underlying data is not stationary.

For the sake of comprehensiveness, we have to distinguish between strict stationarity and weak stationarity.

Strict stationarity

We say a time series process is strictly stationary if its properties are unaffected by a change of time origin.

if its properties are by a change of time origin. The joint distribution of the observations yₜ, yₜ₊₁, …, yₜ₊ₙ is exactly the same as the joint probability distribution of the observations yₜ+h, yₜ₊ₕ₊₁, …, yₜ₊ₕ₊ₙ.

as the joint probability distribution of the observations yₜ+h, yₜ₊ₕ₊₁, …, yₜ₊ₕ₊ₙ. Therefore it is unaffected by forward or backward shifts in time.

However, many real-life processes are not strictly stationary. But even if a process is strictly stationary, we usually have no explicit knowledge of the latent time series process. As strict stationarity is defined as a property of the process’s joint distributions, it is impossible to prove from an observed time series.

Weak stationarity

We define a time series process as weak stationary if

The expectation E(yₜ) is constant over time

The variance Var(yₜ) is constant over time

The covariance Cov(Yₜ,Yₜ₊ₕ) depends only on the lag h, Cov(Yₜ₁,Yₜ₂) = Cov(Yₜ₁₊ₕ,Yₜ₂₊ₕ)

Broadly speaking, a time series said to be weakly stationary if there is no systematic change in mean μ (i.e. trends) if there is no systematic change in variance σ² and if strictly periodic variations (i.e. seasonality) have been removed.

How are these definitions or types of stationarity used in practice?

In practice, people usually mean by the term stationarity weak stationarity since strict stationarity is only a theoretical concept. To be here consistent, we will use from now on stationarity when we mean weak stationarity.

Alright, so now that we know what the concept of stationarity means, let’s test our knowledge by observing the following graphs in figure 5 and check if they are stationary or not.

Figure 5. Examples for stationary and non-stationary time series.

Plot (a) shows a clear trend (no constant mean over time) in the data so the series can not be stationary.

Plot (b) on the opposite does not show a trend or any seasonality in the data. That indicates that plot (b) is stationary.

Plot (c ) has like plot (a) a clear trend in the data and therefore is non-stationary as well.

Last but not least a confusing case by Hyndman and Athanasopoulos (2018). Plot (d) appears non-stationary due to its strong cycles. However, these cycles are aperiodic. When the lynx population becomes too large for the available feed, they stop breeding, and the population falls too low numbers. Then the regeneration of their food sources allows the population to grow again and so on. That means in the long-term, the timing of these aperiodic cycles is not predictable. Because of this cyclic behavior with no trend nor seasonality the series is stationary.

I bet you are surprised that plot (d) turned out to be stationary. Sometimes it can be hard to tell if a time series is stationary visually. Thus we need a more robust way to check stationarity.

Luckily there are several statistical tests (also called unit-root tests) to check if a series is stationary or not.

Augmented Dickey Fuller (ADF) test

One most widely used is the Augmented Dickey Fuller (ADF) test.

There are many more unit root tests available like the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test or the Elliott–Rothenberg–Stock (ADF-GLS) test.

The ADF comes up with a null and an alternative hypothesis.

H₀: The series has a unit root so it is non-stationary .

. Hₐ: The series has no unit root so it is stationary.

Broadly speaking, a unit root is a random walk with drift time series. It contains a stochastic trend that shows an unpredictable pattern. If you are interested in a more deeper explanation please see Makridakis (1998).

Statsmodels provides us for this purpose the adfuller function .

For this example, I used the airpassengers data set from above, which is not stationary. Since the data set shows clearly a trend, I set the function’s regression parameter to ct (constant and trend).

The function returns several values. The second one represents the calculated p-value. Since 0.55 is above the threshold of 0.05, we can not reject H₀, so the data is not stationary.

Now that we know our data is not stationary, how can we make them stationary?

How to make data stationary

As already mentioned, a lot of (classical) models assume stationary time series. So what if we know that our data are not stationary? Two very common approaches to make data stationary are:

Transforming the data (e.g. log and/or using square roots transformations)

the data (e.g. log and/or using square roots transformations) Differencing

Transforming the data is a very basic approach that is also used in other statistical fields like regressions. One applies a log or square roots transformation to the data to make it stationary.

Another very common approach is differencing. It can be applied alternatively or in addition to the transformation approach. We usually take first differences of the series. That is also called first-order differencing. We are creating a new time series of successive differences Yₜ -Yₜ₋₁. For example, the original time series is Y₁, Y₂, Y₃, …, Yₙ, then we apply first-order differencing, which leads to a new time series Y₂-Y₁, Y₃-Y₂, Y₄-Y₃, …, Yₙ - Yₙ₋₁.

If this does not work and we still have non-stationary data, then we can also consider a second-order differencing by taking the differences of the created differences (not to be confused with taking the second difference Yₜ -Yₜ₋₂).

If your time series is in data frame format, you can make use of pandas function .diff(). This function takes the number of periods you want to difference over as a parameter.

If we want to apply first-order differencing we just append the function .diff() to our data frame. In our example we apply the .diff() method to the ausbeer data set (table 1).

Table 1. Differenced ausbeer data set.

Since we take first differences of the series (Y₂-Y₁) we will lose one (the first) data point. That means if we apply second-order differencing, we lose the first two data points.

💡Key takeaways