Multivariate time series forecasting
By: Mahbubul Alam
Tags: Time Series Forecasting, Data Science, Forecasting, Machine Learning, Vector Autoregression

The long version

The short version was short, but the long version can be really long, depending on where you want to stop. You can start with converting the time series data to a ts object, doing all sorts of time series EDA (exploratory data analysis) to tuning and evaluating model performance as many different ways you want, based on project objectives.

1 ) Import libraries

The initial set of libraries needed remains the same as in the “short” version, but we are adding a plotting library matplotlib to visualize the time series object.

# loading essential libraries first

import pandas as pd

import statsmodels.api as sm

from statsmodels.tsa.api import VAR

import matplotlib.pyplot as plt

2) Import, inspect & wrangle data

After importing data you should be going through your usual data wrangling ritual (selecting columns of interest, renaming, summary statistics etc.). But one essential step is to find out if there are NA values and if so, you need to deal with them (see here). As part of data wrangling, you might also want to slice/transform data in different ways for visualization purposes.

# data

mdata = sm.datasets.macrodata.load_pandas().data

df = mdata.iloc[:, 2:4]

df.head()

3) Visualize

If you want to do EDA of time series data you have some additional work to do such as transforming the data into a time series object. But at a minimum, you may want to visualize data to see how the trend-lines look like and how they compare with each other. It gives you the necessary intuition needed for model evaluation.

plt.plot(df)

3) Test for causality

You would want to see if there’s a correlation between the variables. For that you can run Granger’s causality test. Although the name suggests, it’s really not a test of “causality”, you cannot say if one is causing the other, all you can say is if there is an association between the variables.

# import for Granger's Causality Test

from statsmodels.tsa.stattools import grangercausalitytests granger_test = sm.tsa.stattools.grangercausalitytests(df, maxlag=2, verbose=True)

granger_test

4) Split data

As in most machine learning algorithms, it’s a good idea to split data into training and testing set.

nobs = 4

df_train, df_test = df[0:-nobs], df[-nobs:]

5a) Check for stationarity

For time series modeling, data needs to be stationary — meaning if there is a trend in the data you need to get rid of it. To check whether data is stationary there is a test called Augmented Dickey-Fuller (ADF) Test.

# Augmented Dickey-Fuller Test (ADF Test)/unit root test

from statsmodels.tsa.stattools import adfuller def adf_test(ts, signif=0.05):

dftest = adfuller(ts, autolag='AIC')

adf = pd.Series(dftest[0:4], index=['Test Statistic','p-value','# Lags','# Observations'])

for key,value in dftest[4].items():

adf['Critical Value (%s)'%key] = value

print (adf)

p = adf['p-value'] if p <= signif:

print(f" Series is Stationary")

else:

print(f" Series is Non-Stationary") #apply adf test on the series

adf_test(df_train["realgdp"])

adf_test(df_train["realcons"])

5b) Make data stationery

If the data is not stationary you can make it so in several ways, but the simplest one is taking a first difference. After taking first difference you need to go back to the previous step to test again if the data is now stationary. If not, a second difference my be necessary.

# 1st difference

df_differenced = df_train.diff().dropna() # stationarity test again with differenced data

adf_test(df_differenced["realgdp"])

6) Modeling

You can now instantiate the model with VAR() and then fit the model to first differenced data. After running the model you can check the summary results below.

# model fitting

model = VAR(df_differenced)

results = model.fit(maxlags=15, ic='aic')

results.summary()

7) Forecasting

Now that you have your model set up, it’s time to play with it and do actual forecast. Here I am asking the model to forecast 5 steps ahead. The model returns an array of 5 forecast values for both the variables.

# forecasting

lag_order = results.k_ar

results.forecast(df.values[-lag_order:], 5)

8) Plotting

It is now possible to plot the forecast values along with associated standard errors.

# plotting

results.plot_forecast(20)

9) Evaluating

This is an extra step to evaluate forecasting model using Forecast Error Variance Decomposition (FEVD) method using fevd() function.

# Evaluation

fevd = results.fevd(5)

fevd.summary()

10) Inverting

One final step remains. You didn’t fit the model to original data, because you had to transform (first difference) it to make data stationary in step 5b. So the forecast results need to be inverted to the original form.

# forecasting

pred = results.forecast(results.y, steps=nobs)

df_forecast = pd.DataFrame(pred, index=df.index[-nobs:], columns=df.columns + '_1d')

df_forecast.tail() # inverting transformation

def invert_transformation(df_train, df_forecast, second_diff=False):

"""Revert back the differencing to get the forecast to original scale."""

df_fc = df_forecast.copy()

columns = df_train.columns

for col in columns:

# Roll back 2nd Diff

if second_diff:

df_fc[str(col)+'_1d'] = (df_train[col].iloc[-1]-df_train[col].iloc[-2]) + df_fc[str(col)+'_1d'].cumsum()

# Roll back 1st Diff

df_fc[str(col)+'_forecast'] = df_train[col].iloc[-1] + df_fc[str(col)+'_1d'].cumsum()

return df_fc # show inverted results in a dataframe

df_results = invert_transformation(df_train, df_forecast, second_diff=True)

df_results.loc[:, ['realgdp_forecast', 'realcons_forecast']]

Parting notes

Time series data analysis is a fundamental part of business decision-making, therefore decision-makers and data scientists/analysts can benefit from having some degree of familiarity with the mechanics of forecasting models. The article first introduced the concept of multivariate time series and how it is used in different industries. Then I provided a short python implementation as a way to provide intuition for a more complex implementation using a machine learning approach.

For any related questions I can be reached via Twitter.