Monthly Spot Price Prediction - Time Series Analysis by State Space Method | by Sarit Maitra | Towards Data Science

https://sarit-maitra.medium.com/membership
SARIMAX application to original series have the advantages e.g. interpret-ability of model parameters. The objective here is to develop a Seasonal Autoregressive Integrated Moving Average with external variables (SARIMAX) model which tries to account all the effects due to the price influencing factors, to forecast the monthly spot price at Henry Hub. Natural gas price is a random variable which follows a stochastic process with a random trend.
The ability to predict trends is more useful rather than point forecasts in a complex system like stock market. We can predict trends with greater confidence and reliability, rather than the values or levels of these quantities.
Here, our time series forecasting model will use the past price and other variables e.g. climate, storage etc. to predict their future values Natural Gas. Our time series are influenced by lot of external factors. SARIMAX has the capability to incorporate both seasonal and non-seasonal factors in a multiplicative model. Therefore, when the time series exhibits a seasonal variation, the SARIMAX model is usually applied.
SARIMAX model is a SARIMA model with external variables, called SARIMAX (p,d,q) (P,D,Q)S (X), where X is the vector of external variables.
Data related to prices, consumption, reserves, storage, productions, pipelines etc., and climate data including cooling/heating degree days, extreme maximum/minimum temperature, mean temperature, and mean maximum/minimum temperature for New Orleans, LA. All these data were combined together in a single file as an input data file.
We can see that our time-series has more than one time-dependent variable. Each variable depends on its past values and also has some dependency on other variables. We will use this dependency here to predict future values.
SARIMAX is an example of a fully fledged model created using the state space back-end for estimation. SARIMAX model consists of five iterative steps-
Model identification where we identify order of differencing (d), order of seasonal differencing (D), seasonal length (S), non-seasonal autoregressive order (p), seasonal autoregressive order (P), non-seasonal moving average order (q), and seasonal moving average order (Q).Parameter estimation where parameters of the identified model from step 1 are estimated.The fitness of model is diagnosed using Ljung-Box Q statistic to check the adequacy. If the residuals are not normally distributed, we need go to step 4. Otherwise, proceed to step 5.Inclusion of external variables: The relevant external variables are included into the SARIMA model using linear regression.The diagnosed model is validated using out-sample. The validated model is used for forecasting the future values.
Model identification where we identify order of differencing (d), order of seasonal differencing (D), seasonal length (S), non-seasonal autoregressive order (p), seasonal autoregressive order (P), non-seasonal moving average order (q), and seasonal moving average order (Q).
Parameter estimation where parameters of the identified model from step 1 are estimated.
The fitness of model is diagnosed using Ljung-Box Q statistic to check the adequacy. If the residuals are not normally distributed, we need go to step 4. Otherwise, proceed to step 5.
Inclusion of external variables: The relevant external variables are included into the SARIMA model using linear regression.
The diagnosed model is validated using out-sample. The validated model is used for forecasting the future values.
plt.figure()plt.subplot(211)plot_acf(df['hh_sp'], ax=plt.gca())plt.subplot(212)plot_pacf(df['hh_sp'], ax=plt.gca())plt.xlabel('Lag')plt.tight_layout()plt.show()
df = data.copy()df.set_index('Date', inplace=True)print(df.info())df = df.astype(float)res = sm.tsa.seasonal_decompose(df['hh_sp'],freq=12)fig = res.plot()fig.set_figheight(8)fig.set_figwidth(15)plt.show()
From the trend plot, we do not observe any clear trend which follows somewhat stochastic movement here (random walk with drift).
We can see here that, just differencing is not helping to normalize the data set.
df.isnull().sum()def fill_missing(df):for row in range(df.shape[0]):for col in range(df.shape[1]):if np.isnan(df[row,col]):df[row,col]= df[row-1, col]fill_missing(df.values)np.isnan(df).sum()fill_missing(df.values)np.isnan(df).sum()
We have used feature ranking with recursive feature elimination and cross- validated selection of the best number of features using linear regression as the model.
The feature ranking, such that ranking_[i] corresponds to the ranking position of the i-th feature. Selected (i.e., estimated best) features are assigned rank 1.
y = df['hh_sp']X = df.drop(['hh_sp'], axis=1)names=pd.DataFrame(X.columns)lin_reg = LinearRegression()mod =  RFECV(lin_reg, step=1, cv=20) mod_fit = mod.fit(X,y) rankings=pd.DataFrame(mod_fit.ranking_, index=names) #Make it into data framerankings.rename(columns ={0: 'Rank'}, inplace=True)rankings.transpose()
14 features are selected for inclusion in the model
Let us define our target; we will project next 5 months spot price.
df['target'] = df['hh_sp'].shift(-5)df.dropna(inplace=True)n = df.drop(['target'], axis=1) # storing in a dataframe for future usedataset = df[['im_pr','ex_pr','prod_mp','prod_liq','prod_dng','imp_tot','imp_pip','ImpLiq', 'mean_max_txt', 'mean_min_txt', 'mean_txt', 'mean_max_lat', 'mean_min_lat','mean_lat', 'target' ]]dataset.describe()
X = np.array(dataset.drop(['target'], 1))y = np.array(dataset['target'])tscv = TimeSeriesSplit(max_train_size=None, n_splits=5)for train_samples, test_samples in tscv.split(X):#print("TRAIN:", train_samples, "TEST:", test_samples)X_train, X_test = X[train_samples], X[test_samples]y_train, y_test = y[train_samples], y[test_samples]print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)feature_names = ['im_pr','ex_pr','prod_mp','prod_liq','prod_dng','imp_tot','imp_pip','ImpLiq','mean_max_tx', 'mean_min_txt', 'mean_txt', 'mean_max_lat', 'mean_min_lat','mean_lat' ]X_train = pd.DataFrame(data=X_train, columns=feature_names)X_train.index = dataset[:186].indexy_train = pd.DataFrame(y_train, columns = ['target'])y_train.index = X_train.indexX_test = pd.DataFrame(data=X_test, columns=feature_names)X_test.index = dataset[186:].indexy_test = pd.DataFrame(y_test, columns = ['target'])y_test.index = X_test.index
Knowing that, stock price in general follow random walk; the first stage is to generate a set of changes or differences for each of the variables, since spot price can be explained by reference to the unexpected changes in the set of chosen variables rather than their levels. The unexpected value of a variable can be defined as the difference between the actual value of the variable and its expected value. The entire change in the variables from one period to the next is the unexpected change
The advantage of series being stationary is that, the effect of a shock will ease out gradually compared to non-stationary system where effect of shock is permanent.
Whether or not to use differencing partially conditional maximum likelihood estimation. If True, differencing is performed prior to estimation, which discards the first sD+d initial rows but results in a smaller state-space formulation. See the Notes section for important details about interpreting results when this option is used. If False, the full SARIMAX model is put in state-space form so that all data points can be used in estimation. Default is False.
As we know that, SARIMAX combines the ARIMA model with the ability to perform the same auto-regression, differencing, and moving average modeling at the seasonal level. Let us estimate the parameters using maximum likelihood method.
# Fit auto_arima functionstepwise_fit = auto_arima(y_train_tran, X_train_tran, start_p = 1,                          start_q = 1, max_p = 7, max_q = 7, m = 12,                          start_P = 0, seasonal = True,d = None,                           D = 1, trace = True,                           error_action ='ignore',                             suppress_warnings = True,                            stepwise = True)           # To print the summarystepwise_fit.summary()
The p-values of the parameters of the model shows that the seasonal and non-seasonal AR and MA values are not significant at 5% level .
Here, exogenous variables are given as parallel input sequences that have observations at the same time steps as the endogenous series.
# fit modelmodel = sm.tsa.statespace.SARIMAX(                     endog = y_train_tran,                     exog = X_train_tran, order = (0, 0, 0),                       seasonal_order = (2, 1, 1, 12),                      enforce_invertibility = False,                     enforce_stationarity = False)model_fit = model.fit(disp = False)model_fit.summary()
This tests the null hypothesis of all the slope parameters being jointly zero. The p-value of zero attached to the test statistic shows that this null hypothesis should be rejected. Here, all our parameters are having p> 0.05 and therefore null hypothesis cannot be rejected.
The model is diagnosed using the Ljung-Box Q statistic which indicates that the residuals are not white noise. The null hypothesis of the Box Ljung Test, H0, is that our model does not show lack of fit.
Diagnostic plots help us determine visually how our model is fitting the data and if any of the basic assumptions of an SARIMAX model are being violated.
We can see that, the underlying error distribution is Gaussian with constant mean and standard deviation of parametric assumptions.
# Fit a local level modelendog = ytest_tranlocal_model = sm.tsa.UnobservedComponents(endog, 'local level')# Fit the model via maximum likelihoodresult = local_model.fit()# result is an instance of the Unobserved Components Results classprint(result.summary())# Show a plot of the estimated level and trend component seriesfig = result.plot_components()plt.tight_layout()
X_test_tran = np.log(X_test).diff()Xtest_tran = X_test_tran.copy()Xtest_tran = Xtest_tran[~Xtest_tran.isin([np.nan, np.inf, -np.inf]).any(1)]Xtest_tran.describe()y_test_log = np.log(y_test['target']).dropna()y_test_tran = y_test_log.diff().dropna()ytest_tran = y_test_tran.copy()#ytest_tran = ytest_tran[~ytest_tran.isin([np.nan, np.inf, -np.inf]).any(1)]ytest_tran.describe()
rebuilt = pred.cumsum().fillna(0)print('\nCumsum:\n', rebuilt.tail())rebuilt_further = rebuilt + y_test_log[-1:] # last value of test setprint('\nDe-difference:\n',rebuilt_further.tail())original = np.exp(rebuilt_further)print('\nOriginal form:\n',original.tail())
We had shifted our target variables to 5 periods (months); therefore, above should be future 5 months forecast. Let us do some data manipulation to get future dates as shown below.
It can be noticed that, setting up monthly frequency by fault shows future all dates are end of month instead 15 as per original data set. Well, this can also be fixed to synchronize with original data.
# Graphfig, ax = plt.subplots(figsize=(10,5))ax.set(title='Henry Hub Spot price', xlabel='Date', ylabel='USD / Mbtu')# Plot data pointsy_test.plot(ax=ax, label='Observed')# Plot predictionsoriginal.plot(ax=ax, style='r--', label='One-step-ahead forecast')legend = ax.legend(loc='best')
Moreover, it is also possible to get the the confidence interval between upper and lower level of predicted price and draw confidence interval plot too.
It is always a good idea to start with a simple model and test whether it is good enough for its purpose. Applying iterative modeling approach we can add features gradually. Comparing results of successive models provides a form of internal validation, so we can catch conceptual, mathematical, and software errors. By adding and removing features, we can tell which ones have the biggest effect on the results, and which can be ignored.
The important assumption of time series forecasting is the current price is a function of the demand and past price. However, the relation is not just straight forward; demand also depends on climatic condition, import/export price etc. A set of 14 variables are found through recursive feature elimination methods. These variables are found to be important to determine monthly Natural Gas spot price. SARIMAX model is used here for price forecasting, which has a distinct advantage in its explanation of the outlying data. Based on prediction, we could see that SARIMAX prediction can follow the pattern of actual data smoothly.
I can be reached here.