Time Series Prediction using SARIMAX
By: Renu Khandelwal
Tags: Sarima, Data Science, Python, Time Series Forecasting, Arima

This article will be using time series predictive model SARIMAX for Time series prediction using Python. I have used stock price data set for AAPL to demonstrate the implementation, which will use multiple input features for prediction.

For understanding Step by Step Time Series prediction, read this article.

The data set was downloaded from yahoo finance for Apple (AAPL) for a time period from Oct 1, 2008, till Sep 30, 2019.

Importing the required libraries

import pandas as pd

import numpy as np

import matplotlib.pyplot as plt

import seaborn as sns

from pandas.plotting import register_matplotlib_converters

register_matplotlib_converters()

%matplotlib inline

Loading the data set

dataset= pd.read_csv(“C:\\Data\\aapl.csv”)

Displaying the columns and data from the data set

dataset.head(2)

Adding a field to the data set for calculating the mean of High and Low

dataset[‘Mean’] = (dataset[‘Low’] + dataset[‘High’])/2

dataset.head(2)

Data set after adding a column for an average of low and High stock prices for the day

Printing information about the Data frame

dataset.info()

We need to predict the stock prices today based on the data from yesterday. Predicting stock prices for Oct 2, 2008, should be based on Oct 1, 2008 data.

We need to restructure the data set by adding a target column ‘Actual’, which will the stock prices for the next day.

If we need to take data from 2 days previous for prediction, then adjust steps to -2

steps=-1

dataset_for_prediction= dataset.copy()

dataset_for_prediction[‘Actual’]=dataset_for_prediction[‘Mean’].shift(steps)

dataset_for_prediction.head(3)

Actual for Oct 1, 2008, is stock price for Oct 2, 2008

Dropping columns with null values

dataset_for_prediction=dataset_for_prediction.dropna()

Creating Date as the index of the DataFrame

For the Time series data set, it is imperative to set the date time as the index for the dataset.

We know that a date column is an object, and we need to convert it to datetime64 using pandas to_datetime()

I am not setting the freq of the index and leaving it None but set the index to the right frequency if you have the frequency.

dataset_for_prediction[‘Date’] =pd.to_datetime(dataset_for_prediction[‘Date’])

dataset_for_prediction.index= dataset_for_prediction[‘Date’]

Plotting the data

Let’s plot the mean stock prices for the current day and look at the trends.

dataset_for_prediction[‘Mean’].plot(color=’green’, figsize=(15,2))

plt.legend([‘Next day value’, ‘Mean’])

plt.title(“ Tyson Opening Stock Value”)

Apple Stock prices over 2008 till 2019

Plotting volume of Apple stocks sold daily

dataset_for_prediction[‘Volume’].plot(color=’blue’, figsize=(15,2))

plt.title(“ Apple Stock Volume”)

The volume of Apple Stocks sold daily

Normalizing the input and target features

Since the stock prices and volume are on a different scale, we need to normalize the data. We use MinMaxScaler; it will scale the data to a fixed range between 0 to 1

Scaling the input features- Low, High, Open, Close, Volume, Adjusted Close, and Mean

from sklearn.preprocessing import MinMaxScaler

sc_in = MinMaxScaler(feature_range=(0, 1)) scaled_input = sc_in.fit_transform(dataset_for_prediction[[‘Low’, ‘High’,’Open’, ‘Close’, ‘Volume’, ‘Adj Close’, ‘Mean’]]) scaled_input =pd.DataFrame(scaled_input)

X= scaled_input

Scaling the output features -Actual. We are using a different instance of MinMaxScaler here. This will allow us to perform an inverse transform of the predicted stock prices later easily.

sc_out = MinMaxScaler(feature_range=(0, 1)) scaler_output = sc_out.fit_transform(dataset_for_prediction[[‘Actual’]]) scaler_output =pd.DataFrame(scaler_output)

y=scaler_output

Renaming the column names for input features and target variable for better understanding as the column names are currently numbers.

X.rename(columns={0:’Low’, 1:’High’, 2:’Open’, 3:’Close’, 4:’Volume’, 5:’Adj Close’, 6:’Mean’}, inplace=True)

X.head(2)

Renaming target variable

y.rename(columns={0:’Stock Price next day’}, inplace= True)

y.index=dataset_for_prediction.index

y.head(2)

Splitting the data into training and test set

Our training set will be 70%, and the test set will be 30% of the entire data set.

train_size=int(len(dataset) *0.7)

test_size = int(len(dataset)) — train_size train_X, train_y = X[:train_size].dropna(), y[:train_size].dropna()

test_X, test_y = X[train_size:].dropna(), y[train_size:].dropna()

Understanding the Time series data

Decomposing Time Series into several components-Trend, Seasonality, and Random noise

We know that Time series data is composed of Level, Trend, Seasonality, and Random noise. Let’s decompose the data and plot the trend, seasonality, and randomness in the data.

We use statsmodel for seasonal decompose as an additive model and the frequency of the time series, which is the periodicity of the data, which is 365 days for yearly data.

Additive model = Trend + Seasonality + Random Noise

import statsmodels.api as sm

seas_d=sm.tsa.seasonal_decompose(X[‘Mean’],model=’add’,freq=365); fig=seas_d.plot()

fig.set_figheight(4)

plt.show()

Seasonal decomposition of the Time series data

Check for Data Stationarity using Augmented Dickey-Fuller(ADF) test.

If we make the data stationary, then the model can make predictions based on the fact that mean and variance will remain the same in the future. A stationarized series is easier to predict

To check if the data is stationary, we will use the Augmented Dickey-Fuller test. It is the most popular statistical method to find if the series is stationary or not. It is also called as Unit Root Test.

We have written a function that will check the p-value from the ADF test results. If the value of p <0.05, then the data is stationary, the data is not stationary. We can make the data stationary by methods like differencing

from statsmodels.tsa.stattools import adfuller def test_adf(series, title=''):

dfout={}

dftest=sm.tsa.adfuller(series.dropna(), autolag='AIC', regression='ct')

for key,val in dftest[4].items():

dfout[f'critical value ({key})']=val

if dftest[1]<=0.05:

print("Strong evidence against Null Hypothesis")

print("Reject Null Hypothesis - Data is Stationary")

print("Data is Stationary", title)

else:

print("Strong evidence for Null Hypothesis")

print("Accept Null Hypothesis - Data is not Stationary")

print("Data is NOT Stationary for", title)

we create a series for the target variable to check for stationarity

y_test=y[‘Stock Price next day’][:train_size].dropna()

test_adf(y_test, " Stock Price")

Let’s check if we can make the data stationary by applying one difference using diff()

test_adf(y_test.diff(), “ Stock Price”)

We see that the data is now stationary.

Building the Model

Let’s apply ARIMA and SARIMAX to our data and see which one works better. For both ARIMA and SARIMA or SARIMAX, we need to know the AR and MA terms to correct any autocorrelation in the differenced series.

The systematic way to do this is to use autocorrelation function (ACF) and partial autocorrelation (PACF) plots of the differenced series.

ACF plot: is a bar chart of the coefficients of correlation between a time series and its lags. It helps determine the value of p or the AR term.

PACF plot: a plot of the partial correlation coefficients between the series and lags of itself. Helps determine the value of q or the MA term

fig,ax= plt.subplots(2,1, figsize=(10,5))

fig=sm.tsa.graphics.plot_acf(y_test, lags=50, ax=ax[0])

fig=sm.tsa.graphics.plot_pacf(y_test, lags=50, ax=ax[1])

plt.show()

We see that the PACF plot has a significant spike at lag 1 and lag 2, meaning that all the higher-order autocorrelations are effectively explained by the lag-1 and lag 2 autocorrelations.

We will use pyramid auto Arima to perform a stepwise search for AR and MA term that gives the lowest value of AIC.

from pmdarima.arima import auto_arima step_wise=auto_arima(train_y,

exogenous= train_X,

start_p=1, start_q=1,

max_p=7, max_q=7,

d=1, max_d=7,

trace=True,

error_action=’ignore’,

suppress_warnings=True,

stepwise=True)

The auto_arima function fits the best ARIMA model to the time series data. This is done based on the parameters that you provide based on information criterion like AIC. The function performs a stepwise search over possible model & seasonal orders within the constraints provided. It selects the parameters that minimize the given metric like AIC(Akaike Information Criterion).

Explaining the parameters for auto_arima

exogenous: An optional 2-d array of exogenous variables. If provided, these variables are used as additional features in the regression operation. If an ARIMA is fit on exogenous features, it must be provided exogenous features for making predictions.

Seasonal: Whether to fit a seasonal ARIMA. By default, it is set to True. If we enable the seasonal parameter, we need to provide the P,D, and Q parameters.

m: Refers to the period for seasonal differencing, number of periods in each season.

d: auto_arima works by conducting differencing tests, and this is a parameter used for determining the order of differencing

start_p, max_p, start_q, max_q: We fit the model based on these defined ranges

trend: Trend of the time series. “c” for constant trend and “t” for linear trend and when we have both we can specify “ct”

error_action: default behavior is to “warn”, In our case, we are ignoring the error

trace: will print status on the fits

stepwise: specifies if we want to use a stepwise algorithm. The stepwise algorithm is significantly faster than fitting all hyper-parameter combinations and is less likely to over-fit the model.

step_wise.summary()

Summary of the best model and parameters for the model based on stepwise execution of auto_arima

The model suggested by auto_arima is SARIMAX, and the value for p,d,q is 0,1,1, respectively.

Train the model

As suggested by auto_arima, we will use SARIMAX to train our data. SARIMAX has the ability to work on datasets with missing values.

from statsmodels.tsa.statespace.sarimax import SARIMAX model= SARIMAX(train_y,

exog=train_X,

order=(0,1,1),

enforce_invertibility=False, enforce_stationarity=False)

We now train the model by giving the target variable and the exog variable, the input features. We specify the order for p,d,q

Fit the model

we now fit the model

results= model.fit()

Predict or forecast the results

To predict, we can predict() or forecast() methods of SARIMAX on the object returned by fitting the data.

Below we use predict() and provide the start and end, along with the exog variable based on which the predictions will be made.

predictions= results.predict(start =train_size, end=train_size+test_size+(steps)-1,exog=test_X)

We can also use forecast() and provide steps and exog parameters. Steps is an integer value that specifies the number of steps to forecast from the end of the sample.

forecast_1= results.forecast(steps=test_size-1, exog=test_X)

Plot the predictions

we now plot the predictions and the forecast

We will create a data frame for string the Actual scaled stock prices

act= pd.DataFrame(scaler_output.iloc[train_size:, 0])

We now create the data frame to store the Date, Actual and Predicted data

predictions=pd.DataFrame(predictions)

predictions.reset_index(drop=True, inplace=True)

predictions.index=test_X.index

predictions['Actual'] = act['Stock Price next day']

predictions.rename(columns={0:'Pred'}, inplace=True)

we now plot the Actual and the predicted data

predictions[‘Actual’].plot(figsize=(20,8), legend=True, color=’blue’)

predictions[‘Pred’].plot(legend=True, color=’red’, figsize=(20,8))

We now create the data frame to store the Date, Actual and Forecasted data.

forecast_apple= pd.DataFrame(forecast_1)

forecast_apple.reset_index(drop=True, inplace=True)

forecast_apple.index=test_X.index

forecast_apple[‘Actual’] =scaler_output.iloc[train_size:, 0]

forecast_apple.rename(columns={0:’Forecast’}, inplace=True)



Plotting the forecast

forecast_apple[‘Forecast’].plot(legend=True)

forecast_apple[‘Actual’].plot(legend=True)

Evaluating the model

We will use root mean square error for evaluating the model’s performance

from statsmodels.tools.eval_measures import rmse

error=rmse(predictions[‘Pred’], predictions[‘Actual’])

error

Root Mean Square Error

Scaling back to original values

We will use inverse transform to scale back to original stock prices

trainPredict = sc_out.inverse_transform(predictions[[‘Pred’]])

testPredict = sc_out.inverse_transform(predictions[[‘Actual’]])

Conclusion:

We predicted the future stock prices by identifying the best ARIMA model for the dataset using multivariate input features

code available at -GitHub

References:

https://medium.com/r/?url=http%3A%2F%2Fwww.alkaline-ml.com%2Fpmdarima%2F0.9.0%2Fmodules%2Fgenerated%2Fpyramid.arima.auto_arima.html