Descriptive statistics in Time Series Modelling
By: Snigdha Cheekoty
Tags: Time Series Forecasting, Data Science, Time Series Data, Time Series Analysis, Machine Learning

There are various statistical tests that can be performed to describe the time series data. Time series modelling requires the data to be in a certain way, and these requirements vary from model-to-model. These models, once fitted to the data, need some kind of validation which can be done through statistical tests.

But most commonly, what we look for in time series are properties like stationarity, causality, correlations, seasonality, etc. Models like ARMA, ARIMA, SARIMA, Holt Winters address different types of time series data. The data needs to be checked for its underlying attributes and statsmodels has a variety of tests to explore and play around with these attributes. Following is a brief overview of these properties.

Stationarity

A time series is said to be stationary if it doesn’t increase or decrease with time linearly or exponentially(no trends), and if it doesn’t show any kind of repeating patterns(no seasonality). Mathematically, this is described as having constant mean and constant variance over time. Along, with variance, the autocovariance should also not be a function of time. If you have forgotten what mean and variance are: mean is the average of the data and variance is the average squared distance from the mean.

Stationary data: Total number of daily female births

Non- stationary data: Number of airlines passengers over the years

Some models make a general assumption of stationarity of time series, but most of the models like ARMA, ARIMA, SARIMA , VARMA, etc., need the data to be stationary. But, how do we determine whether the given data is stationary or not?

The most basic approach for understanding this is to plot the data and check if there’s any hint at the presence of underlying trends or seasonality. This visual practice rarely helps and often its difficult for the human eye to tell this. So we can add rolling statistics (like moving average) to the data with a fixed window size to check for stationarity.

Rolling mean with a window size of 12

Sometimes, it’s even difficult to interpret the rolling mean visually so we take the help of statistical tests to identify this, one such being Augmented Dickey Fuller Test. ADCF Test is implemented using statsmodels in python which performs a classic null hypothesis test and returns a p-value.

Interpretation of null hypothesis test: If p-value is less than 0.05 (p-value: low), we reject the null hypothesis and assume that the data is stationary. But if the p-value is more than 0.05 (p-value: high), then we fail to reject the null hypothesis and determine the data to be non-stationary.

Once the stationarity of the dataset is determined, we perform the transformation based on the requirement. If the data is found to be stationary, we proceed to do some modelling on it, but if its found to be not stationary, then we must transform it before fitting the model. “Differencing” is a common way to transform the data to stationary, which is nothing but finding the differences between consecutive data terms and shift them by 1. First, we try the first -order differencing to the data and then the second , third, and so on, until the data becomes stationary. But each differencing step comes at the cost of losing a row of data. (since we shift the rows/data points by 1 for every step) If the data shows seasonality, then the differencing is done by season. For example, if we had monthly data which has yearly seasonality, then we difference the data by a time unit of 12.

Causality

Causality is related to the phenomenon of causation: if one time series can be used to predict another time series, then those two are said to have some causal relation. How to test for causality? Granger Causality test is undoubtedly the best way to check for it. It can be again implemented using statsmodels in python.

Visually, to determine whether there is an causality in the data becomes difficult. It’s extremely unlikely for us to understand the causality, as we see in the below example.

The two time series ‘a’ and ‘d’ have causality

In the above graph, the two time series have some sort of causality; if you shift ‘a’ by two units ahead , it gives you the ‘d’ values.

Interpretation of Granger causality test: when p-value is more than 0.05, there’s no causality and when the p-value is low, i.e., less than 0.05, there is a sign of causality in the data.

Evaluating forecasts

Time series modelling involves seeing correlations in the data by plotting the data with a lagged version of itself by some time units. When we compare the time series against a lagged version of itself, it gets difficult to see any correlations with a considerable increase in time. To evaluate the model-fit and the quality of the forecasts, we use metrics like MAE, MSE, RMSE, AIC and BIC.

Mean absolute error, mean squared error and root mean squared error are few such errors that we use for regression problems, thereby also used for regression- based time series models.

What we would like to understand are the AIC and BIC metrics.

AIC: Akaike Information Criterion, developed in 1971, evaluates a collection of models- it evaluates the quality of each model relative to the others and compares the models against each other.

AIC is specifically used for ARIMA based models to determine the p,d,q parameters. So, we you’re going to run the ARIMA model with different orders of p, d and q, you would need a metric that can compare the models against one another. AIC is a perfect metric to do this job. Th advantage of this metric is that it provides the penalties for number of parameters used in an effort to thwart overfitting. Let’s say you have a simple model that is performing relatively well and a complex model that is performing slightly better than the previous one. If there’s a minor performance improvement between these two models, then AIC is going to take this into account and will tell you to choose the simpler model rather than going for the complex one. In this scenario, the complex model will have a lower information criterion value than the simpler model. In a way, AIC penalizes the models that use too many parameters.

Thanks for reading the article!