Forecasting Of Electricity Generation In US Using Vector Autoregression (VAR)
By: Azamat Kulzhanov
Tags: Vector Auto Regression, Electricity, Renewable Energy, Var

Forecasting

Despite the fact that the US owns second place in electricity generation by renewables, its’ major power source is based on fossil fuel. Hence, it would be interesting to figure out how the US power industry might be reshaped in the future in terms of alternative energy. Moreover, the availability of the data related to the industry makes the US a perfect candidate for our analysis.

#electricity generation by power source time series edata = pd.read_excel('/content/data/MyDrive/python/Python project/Table_7.2a_Electricity_Net_Generation__Total_(All_Sectors).xlsx', skiprows=10) edata.drop(0, axis = 0, inplace=True)

edata.replace('Not Available', 0, inplace=True)

edata_old = [] for col in edata.columns:

edata_old.append(col) edata_new = ['date', 'coal', 'petroleum','natural gas', 'other gases','nuclear', 'hydropump', 'hydroelectric','wood', 'waste', 'geothermal', 'solar', 'wind', 'total'] edata.rename(columns=dict(zip(edata_old, edata_new)), inplace=True)

edata.set_index('date', drop=True, inplace=True) edata.head()

The monthly time series of national power generation by source from 1973 to 2020

The dataset above represents the time series of the national monthly power generation from 1973 to 2020. The dataset is split by power source as seen from the dataframe. For the simplicity of forecasting the dataset was grouped by renewables and non-renewables.

#Grouping the dataset by renewables and non-renewables edata['nonren'] = edata['coal'] + edata['petroleum'] + edata['natural gas'] + edata['other gases'] + edata['nuclear'] edata['ren'] = edata['hydroelectric'] + edata['waste'] + edata['geothermal'] + edata['solar'] + edata['wind'] df = edata[['nonren','ren']]

df.head()

The visualization of the dataset that we will be working on might help us to gain a general sense of the data and the recent state of the US power industry.

# Plot fig, axes = plt.subplots(nrows=1, ncols=2, dpi=120, figsize=(10,2))

for i, ax in enumerate(axes.flatten()):

data = df[df.columns[i]]

ax.plot(data, color='red', linewidth=1)

# Decorations

ax.set_title(df.columns[i])

ax.xaxis.set_ticks_position('none')

ax.yaxis.set_ticks_position('none')

ax.spines["top"].set_alpha(0)

ax.tick_params(labelsize=6) plt.tight_layout();

Overview of the national power industry through time

According to the lineplot above, there is a clear positive trend in conventional power sources and a more-or-less stable state in renewables until 2010. However, the situation in the industry changes after 2010 — the electricity generation by non-renewables is no longer rising, whereas, renewables show an almost two-fold increase from 30000 to 60000 units during one decade. Nevertheless, fossil fuels are still dominating in the power industry despite the observed stagnation — there is a 10 times difference between the two types of power sources.

We are proceeding with our analysis with the following libraries:

from statsmodels.tsa.api import VAR

from statsmodels.tsa.stattools import adfuller

from statsmodels.tools.eval_measures import rmse, aic

from statsmodels.tsa.vector_ar.vecm import coint_johansen

from statsmodels.tsa.stattools import grangercausalitytests

from statsmodels.stats.stattools import durbin_watson

The vector autoregression (VAR) model extends the idea of univariate autoregression to k time series regressions, where the lagged values of all k series appear as regressors. Put differently, in a VAR model we regress a vector of time series variables on lagged vectors of these variables. As for AR(p) models, the lag order is denoted by p so the VAR(p) model of two variables Xt and Yt (k=2) is given by the equations:

We split the dataset by two parts — one for training the model and another one for testing it. But before we start training our model it is important to check the stationarity of the dataset since the VAR requires the data to be stationary. There are numerous methods to check the stationarity but we would proceed with the Augmented Dickey-Fuller test provided by statsmodels.tsa.stattools . If the result of the test shows a p-value less than significance level then the null-hypothesis, that time series is non-stationary, is rejected.

#Splitting data into train and test portions nobs = 48

df_train, df_test = df[0:-nobs], df[-nobs:] #Test the stationarity def adfuller_test(series, signif=0.05, name='', verbose=False):

r = adfuller(series, autolag='AIC')

output = {'test_statistic':round(r[0], 4), 'pvalue':round(r[1], 4), 'n_lags':round(r[2], 4), 'n_obs':r[3]}

p_value = output['pvalue']

def adjust(val, length= 6): return str(val).ljust(length) # Print Summary print(f' Augmented Dickey-Fuller Test on "{name}"', "

", '-'*47)

print(f' Null Hypothesis: Data has unit root. Non-Stationary.')

print(f' Significance Level = {signif}')

print(f' Test Statistic = {output["test_statistic"]}')

print(f' No. Lags Chosen = {output["n_lags"]}') for key,val in r[4].items():

print(f' Critical value {adjust(key)} = {round(val, 3)}') if p_value <= signif:

print(f" => P-Value = {p_value}. Rejecting Null Hypothesis.")

print(f" => Series is Stationary.")

else:

print(f" => P-Value = {p_value}. Weak evidence to reject the Null Hypothesis.")

print(f" => Series is Non-Stationary.") # ADF Test on each column for name, column in df_train.iteritems():

adfuller_test(column, name=column.name)

print('

')

Results of the first ADFuller test

As results suggest, the null-hypothesis is failed to be rejected for both variables. As in the case with the stationarity test, there are plenty of techniques to deal with non-stationarity of the data. The simplest one is the differencing.

The differencing implies subtracting of every value in a time series by the preceding value. Hence, every differencing step reduces the whole dataset by one row. After we complete the differencing we need to check the stationarity again. In case the null-hypothesis is still failed to be rejected, the differincing step and ADFuller test are required to be conducted again.

df_differenced = df_train.diff().dropna() # ADF Test on each column

for name, column in df_differenced.iteritems():

adfuller_test(column, name=column.name)

print('

')

Results of the second ADFuller test

The dataset required only one differencing step to fix non-stationarity but data preparation is not ended. statsmodels.tsa.stattools provides another helpful test called Grangers Causality test. The aim of this test is to make sure that past values of time series affect both variables. It is important test since VAR’s method relies on this premise. Values in the matrix below 5% significance level reject the null-hypothesis that variable_x does not cause variable_y .

maxlag=12 test = 'ssr_chi2test' def grangers_causation_matrix(data, variables, test='ssr_chi2test', verbose=False):

df = pd.DataFrame(np.zeros((len(variables), len(variables))), columns=variables, index=variables)

for c in df.columns:

for r in df.index:

test_result = grangercausalitytests(data[[r, c]], maxlag=maxlag, verbose=False)

p_values = [round(test_result[i+1][0][test][1],4) for i in range(maxlag)]

if verbose: print(f'Y = {r}, X = {c}, P Values = {p_values}')

min_p_value = np.min(p_values)

df.loc[r, c] = min_p_value

df.columns = [var + '_x' for var in variables]

df.index = [var + '_y' for var in variables]

return df grangers_causation_matrix(df_differenced, variables = df_differenced.columns)

Grangers Causation Matrix

The causation test showed good results. The last step of data preparation is Johanson’s Cointegration test that examines statistically significant relationship between all variables present in the dataset (in our case, two).

def cointegration_test(df, alpha=0.05):

out = coint_johansen(df,-1,5)

d = {'0.90':0, '0.95':1, '0.99':2}

traces = out.lr1

cvts = out.cvt[:, d[str(1-alpha)]]

def adjust(val, length= 6): return str(val).ljust(length)

# Summary

print('Name :: Test Stat > C(95%) => Signif

', '--'*20)

for col, trace, cvt in zip(df.columns, traces, cvts):

print(adjust(col), ':: ', adjust(round(trace,2), 9), ">", adjust(cvt, 8), ' => ' , trace > cvt) cointegration_test(df_differenced)

Results of Cointegration Test

The results of the Cointegration Test gave enough evidence to reject the null-hypothesis that there is no statistically significant relationship between variables in the dataset.

Our dataset is ready for the model and the first step in training a model is the identification of the important parameter p that stands for the order or lag order. The VAR provides a helpful method select_order which treats the dataset with different lag values and highlights the lowest AIC (Akaike Information Criterion) together with other important statistics.

model = VAR(df_differenced)

x = model.select_order(maxlags=12)

x.summary()

Results of select_order method with a highlighted preferrable order value

model_fitted = model.fit(12)

model_fitted.summary()

Among different information of the results of the fitted model, we find a correlation matrix of residuals. Since VAR is a type of regression technique it is vital to show that residuals of the fitted model are not correlated between variables. It means that errors are random and up to this stage the model can be considered good.

To support our observation related to the model residuals we can also perform Durbin-Watson statistical test. The closer the value to 2.0 the more random and uncorrelated the error is.

out = durbin_watson(model_fitted.resid) for col, val in zip(df.columns, out):

print(col, ':', round(val, 2))

Results of DW test support the observation made on the correlation matrix. The next step is to test the fitted model by forecasting and compare results with the actual values via visualization.

But before comparing with actual values we need to return forecasted values back to their original state, i.e. invert differencing.

# Get the lag order

lag_order = model_fitted.k_ar # Input data for forecasting

forecast_input = df_differenced.values[-lag_order:] #Creating a function for inverting differencing

def invert_transformation(df_train, df_forecast, second_diff=False):

df_fc = df_forecast.copy()

columns = df_train.columns

for col in columns:

# Roll back 2nd Diff

if second_diff:

df_fc[str(col)+'_1d'] = (df_train[col].iloc[-1]-df_train[col].iloc[-2]) + df_fc[str(col)+'_2d'].cumsum()

# Roll back 1st Diff

df_fc[str(col)+'_forecast'] = df_train[col].iloc[-1] + df_fc[str(col)+'_1d'].cumsum()

return df_fc #Forecasting the fitted model

fc = model_fitted.forecast(y=forecast_input, steps=nobs) df_forecast = pd.DataFrame(fc, index=df.index[-nobs:], columns=df.columns + '_1d') #Inverting differencing

df_results = invert_transformation(df_train, df_forecast, second_diff=False) #Visualization of forecasted values vs actual

fig, axes = plt.subplots(nrows=int(len(df.columns)/2), ncols=2, dpi=150, figsize=(10,6)) for i, (col,ax) in enumerate(zip(df.columns, axes.flatten())):

df_results[col+'_forecast'].plot(legend=True,

ax=ax).autoscale(axis='x',tight=True)

df_test[col][-nobs:].plot(legend=True, ax=ax);

ax.set_title(col + ": Forecast vs Actuals")

ax.xaxis.set_ticks_position('none')

ax.yaxis.set_ticks_position('none')

ax.spines["top"].set_alpha(0)

ax.tick_params(labelsize=6) plt.tight_layout();

VIsualization of forecasted and actual values

#Assessment of predicted values over actuals from sklearn.metrics import mean_squared_error

from statsmodels.tools.eval_measures import rmse print(mean_squared_error(df_test['nonrenewables'], df_results['nonrenewables_forecast']))

print(rmse(df_test['nonrenewables'], df_results['nonrenewables_forecast'])) print(mean_squared_error(df_test['renewables'], df_results['renewables_forecast']))

print(rmse(df_test['renewables'], df_results['renewables_forecast'])) #Output

539972550.4891773

23237.30945030378

107447935.89275086

10365.709618388451

Despite the fact that visually predicted values coincide with actual ones RMSE and MSE of these values are way above preferred. The graph below shows a significant discrepancy between forecasted and actual values thereby confirming the poor quality of the model. The possible reason for such an observation may be that the data wasn’t prepared enough for the model.

# Forecast extended fc = model_fitted.forecast(y=forecast_input, steps=264)

df_forecast = pd.DataFrame(fc, columns=df.columns + '_1d')

df_results = invert_transformation(df_train, df_forecast, second_diff=False) #adding date column to forecasted values date_column = pd.date_range('2016-06-01', periods= 264, freq='MS')

df_results['date'] = date_column

df_results.set_index('date', inplace=True) #Visualization of extended forecast

fig, axes = plt.subplots(nrows=int(len(df.columns)/2), ncols=2, dpi=150, figsize=(10,5)) for i, (col,ax) in enumerate(zip(df.columns, axes.flatten())):

df_results[col+'_forecast'].plot(legend=True, ax=ax).autoscale(axis='x',tight=True)

df[col].plot(legend=True, ax=ax);

ax.set_title(col + ": Forecast vs Actuals")

ax.xaxis.set_ticks_position('none')

ax.yaxis.set_ticks_position('none')

ax.spines["top"].set_alpha(0)

ax.tick_params(labelsize=6) plt.tight_layout();

I also tried modeling every type of power source separately, i.e. treating ‘petroleum’, ‘coal’, ‘natural gas’, etc., separately, rather than combining them into one variable. Such a model also resulted in high values of error.

Final Thoughts

Ideally, I anticipated the forecasted decline in power generation by non-renewables, since the US has huge potential in terms of alternative energy.

Nevertheless, I consider this attempt as one of the initial steps in Python, and further researches and studies on this topic are definitely required. I would also appreciate any comments on possible ways to improve my model.

Link to full code on GitHub.