Time Series Analysis for Energy Generation
By: Chidi Ndego
Tags: Energy, Time Series Analysis, Data Science, Forecasting, Data Visualization

source: research snipers

Time series forecasting is an analysis that can be done on any data that is dependent/collected over time. Its main aim is to carefully study the past observations of a time series and develop a model which appositely describes the intrinsic pattern of the series. This model is then used to predict the future values. For this reason, its application is extensively used in various fields such as finance, science, health, engineering and many others. Data used for time series analysis could be recorded on yearly basis, monthly, hour, minutes or even seconds, all being time units.

This tutorial is focused on predicting energy generation. Data for this project is from a power station in Nigeria and is recorded on a monthly basis. Analysis would be carried out using pandas library on python. Models would be built on AR, MA, ARIMA, SARIMA, and Holt Winter’s models. Without wasting much of your time, let’s begin!!

Import necessary libraries

# Import the libraries

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

%matplotlib inline

import seaborn as sns # Prevent warning messages

import warnings

warnings.filterwarnings('ignore')

Lets load and see how our data is set

# Import data

df = pd.read_csv('Energy Generated.csv')

print(df.head())

print('

')

print(df.info())

The data contains energy generated in Megawatt-Hour (MWH) over 65 months period, starting from May of 2013. No Null values are reported. You’d also notice the data types are ‘object’ and ‘float’ — not read as time series. As such, the MONTH column would have to be converted to datetime data type.

# Convert month column to datetime data type

df['MONTH'] = pd.to_datetime(df['MONTH'])

df.columns = ['month','energy_gen'] #rename column names

df.head()

Exploratory Data Analysis

It is good practice to always have an idea of your data set before modelling. By this, you’re getting more familiar with the data set and early inferences could be made. What better way to do this by exploring and visualizing with plots.

# Plot of energy generated over the time period

plt.figure(figsize=(16,7))

plt.plot(df[‘month’],df[‘energy_gen’])

plt.xlabel(‘Time’)

plt.ylabel(‘Energy Generated (MWH)’)

plt.title(‘Energy Generation’)

Oops! Not too much to pick from here. By sight, data seems randomly distributed over the years. Lets plot a stacked line chart for each month to get insight in possible monthly pattern.

# Create new columns for month and year

import datetime

df['year'] = df.month.dt.year

df['month_new'] = df['month'].dt.strftime('%b') # Stacked line plot

plt.figure(figsize=(16,7))

sns.pointplot(x='month_new',y='energy_gen',hue='year',data=df)

It looks like September-October-November always have a decrease-increase-decrease trend respectively in energy generation over the years except in 2015 where it was otherwise. Unfortunately, generally, it appears like the data is a victim of high irregularities caused by unpredictable influences that may include strike, equipment breakdown amongst other possible causes.

Lets observe the yearly and monthly pattern in energy generated.

plt.figure(figsize=(16,7))

plt.subplot(121)

df.groupby(‘year’)[‘energy_gen’].mean().plot.bar()

plt.subplot(122)

df.groupby(‘month_new’)[‘energy_gen’].mean().plot.bar()

Ughh..Yet again, we are not getting much insights from year wise and month wise plots. Although, it shows that October and September cumulatively represent months with the highest and lowest energy generation respectively over the time series.

Generally, time series is usually affected by three main components, which can be separated from the observed data. These components are: Trend, Seasonal and Irregular/Residual components.

Trend: This describes the general tendency of a time series to change (increase or/and decrease) or remain stagnate over a period of time.

Seasonality: This describes any predictable change or pattern in a time series that recurs or repeats over a specific time period. Such can occur due to climatic and weather condition, seasonal festive celebrations amongst others.

Residuals: This describes irregular variations in a time series caused by unpredictable influences, which are neither regular nor repeat in a particular pattern.

Lets decompose our time series data to check for trend, seasonality, and residuals.

# Indexing the datetime data type

decompose = df[["month", "energy_gen"]]

decompose.index = df["month"]

decompose = decompose[["energy_gen"]] from statsmodels.tsa.seasonal import seasonal_decompose

decomposition = seasonal_decompose(decompose)

trend = decomposition.trend

seasonal = decomposition.seasonal

residual = decomposition.resid

plt.figure(figsize=(16, 7))

plt.subplot(411)

plt.plot(df["energy_gen"], label='Original')

plt.legend(loc='best')

plt.subplot(412)

plt.plot(trend, label='Trend')

plt.legend(loc='best')

plt.subplot(413)

plt.plot(seasonal,label='Seasonality')

plt.legend(loc='best')

plt.subplot(414)

plt.plot(residual, label='Residuals')

plt.legend(loc='best')

plt.tight_layout()

Finally, we get insights!! From the above plot, our data obviously doesn’t follow a defined trend, which is a good feature for data stationarity. But, also, it shows a seasonal pattern over the years.

Splitting data into training and test data

We will divide our data into train and test data. Model would be built on the train while prediction/evaluation of model on the test data. Training and test take 70% and 30% respectively of the data set.

df1 = decompose

x_train = df1[df1.index < datetime.datetime(2017,1,1,0,0,0)]

x_test = df1[df1.index >= datetime.datetime(2017,1,1,0,0,0)]

print(x_train.shape,x_test.shape)

Modeling

We’d consider model parsimony in choosing our forecasting models. Similar to the famous Occam’s razor principle, simple models which still describe the inherent properties of the time series should be considered before complex ones. For this reason, model is built on just the ‘ARIMA family’ models, and the famous Holt winter’s model.

Test for Stationarity

One major assumption of most time series models including the ARIMA models, is data stationarity. You’d agree that if a time series has a particular behavior over time, there is a very high probability that it will follow the same in the future. By this, the series is independent of time and its statistical properties such as mean, variance, and covariance remain constant over time.

Dickey-Fuller Test: This is one of the statistical tests for checking stationarity. Here the null hypothesis assumes the time series to be non-stationary. The test results comprise of a Test Statistic and some Critical Values for different confidence levels. 95% confidence level is the generally acceptable interval. If the ‘Test Statistic’ is less than the ‘Critical Value’, we reject the null hypothesis and say that the series is stationary. Check this article for more.

from statsmodels.tsa.stattools import adfuller

def stationary_test(data):

dftest = adfuller(data.energy_gen,autolag=’AIC’)

dfoutput = pd.Series(dftest[0:4], index=[‘Test Statistic’,’p-value’,’#Lags Used’,’Number of Observations Used’])

for key,value in dftest[4].items():

dfoutput[‘Critical Value (%s)’%key] = value

print(dfoutput)

plt.figure(figsize=(16, 7))

plt.plot(data.index, data.energy_gen)

stationary_test(x_train)

Data is seen to be stationary. In cases where it is otherwise, some techniques such as data transformation (trend removal), seasonality removal, and differencing could be applied.

To further reduce the trend of the data, data was transformed using box cox transformation.

With lambda = 0, it assumes a logarithmic transformation. Check this article for more.

from scipy import stats

import statsmodels.api as sm

# Transformation of data using Boxcox

box_cox_train = stats.boxcox(x_train,lmbda=0)

box_cox_train = pd.DataFrame(data=box_cox_train)

box_cox_train.index = x_train.index

box_cox_train.columns = [‘energy_gen’]

stationary_test(box_cox_train)

Forecasting the time series using ARIMA

To fit the ARIMA model on our time series, there are certain parameters that need to be optimized.

Number of AR (Auto-Regressive) terms (p): This assumes that the future value of a variable is dependent on p past observations. For instance if p is 3, the future prediction for x at a time t, (x(t)), will be dependent on x(t-1), x(t-2), and x(t-3). Number of MA (Moving Average) terms (q): Rather than directly using past observations, the Moving Average term, q makes use of the errors of these past observations in predicting future values. For instance if q is 3, the future predction for x(t) will be dependent on e(t-1), e(t-2), and e(t-3) where e(t-i) is the difference between the moving average at ith instant and actual value. Number of Differences (d): These are the number of nonseasonal differences to be performed on the data to achieve stationarity.

To find the optimized values of p and q, we will use the AutoCorrelation Function (ACF) and Partial AutoCorrelation Function (PACF) plots.

ACF is a measure of the correlation between the time series with a lagged version of itself. As the ACF values attenuate rapidly for increasing lags, we can assume the series is stationary. This is observed to estimate q.

PACF measures the correlation between the time series with a lagged version of itself at k period, but after eliminating the variations already explained by the intervening comparisons, that is at lags less than k. This is observed to estimate p.

ACF and PACF Plots

Number of lags is usually estimated from N/4. Where N = length of series.

from statsmodels.tsa.stattools import acf,pacf

#ACF and PACF plots to estimate p,q values

lag_acf = acf(x_train,nlags=16)

lag_pacf = pacf(x_train,nlags=16,method=’ols’) plt.figure(figsize=(16, 7))

#Plot ACF:

plt.subplot(121)

plt.stem(lag_acf, marker=”o”)

plt.axhline(y=0,linestyle=’ — ‘,color=’gray’) #Plot of confidence interval

plt.axhline(y=-1.96/np.sqrt(len(x_train)),linestyle=’ — ‘,color=’gray’)

plt.axhline(y=1.96/np.sqrt(len(x_train)),linestyle=’ — ‘,color=’gray’)

plt.title(‘Autocorrelation Function’) #Plot PACF:

plt.subplot(122)

plt.stem(lag_pacf, marker=”o”)

plt.axhline(y=0,linestyle=’ — ‘,color=’gray’) #Plot of confidence interval

plt.axhline(y=-1.96/np.sqrt(len(x_train)),linestyle=’ — ‘,color=’gray’)

plt.axhline(y=1.96/np.sqrt(len(x_train)),linestyle=’ — ‘,color=’gray’)

plt.title(‘Partial Autocorrelation Function’)

plt.tight_layout()

p — The lag value on the PACF plot that crosses the upper confidence interval first. In this case, p=1. q — The lag value on the ACF plot that crosses the upper confidence interval first. In this case q=1. d — No differencing was done to achieve stationarity, thus, d = 0.

AutoRegressive (AR) Model

This model specifies that the forecast variable of interest depends linearly on its own previous values.

from statsmodels.tsa.arima_model import ARIMA

# AR Model p=1, d=0, q=0

model = ARIMA(box_cox_train, order=(1, 0, 0))

results_AR = model.fit(disp=-1) plt.figure(figsize=(16, 7))

plt.plot(x_train.index, x_train.values, color="lightblue")

plt.plot(x_test.index, x_test.values, color="green",label='Original') # Predicting

pred = pd.DataFrame(results_AR.forecast(len(x_test))[0])

pred.columns = ["energy_gen"]

pred.index = x_test.index # Converting from log to normal value

import math

pred["energy_gen"] = pred["energy_gen"].apply(lambda x: math.exp(x)-1) # Measuring error

from sklearn.metrics import mean_squared_error,mean_absolute_error

RMSE = math.pow(mean_squared_error(x_test.values, pred.values), 0.5)

MSE = mean_squared_error(x_test.values,pred.values)

MAE = mean_absolute_error(x_test.values,pred.values)

MAPE = float(sum(abs((x_test.values - pred.values)/(x_test.values)) * 100)/len(x_test))

plt.plot(pred.index, pred.fillna(0).values, color="red",label='Predicted') plt.xlabel('Date')

plt.ylabel('Energy Generated')

plt.legend()

Moving Average (MA) Model

This technique is very useful for forecasting short-term trends. It is simply the average of a select set of time periods. It’s called ‘moving’ because as a new energy generated is predicted for an upcoming time period, the oldest number in the set falls off, keeping the time period locked.

# MA Model p=0, d=0, q=1

model = ARIMA(box_cox_train, order=(0, 0, 1))

results_AR = model.fit() plt.figure(figsize=(16, 7))

plt.plot(x_train.index, x_train.values, color="lightblue")

plt.plot(x_test.index, x_test.values, color="green",label='Original') # Predicting

pred = pd.DataFrame(results_AR.forecast(len(x_test))[0])

pred.columns = ["energy_gen"]

pred.index = x_test.index # Converting from log to normal value

pred["energy_gen"] = pred["energy_gen"].apply(lambda x: math.exp(x)-1) # Measuring error

RMSE = math.pow(mean_squared_error(x_test.values, pred.values), 0.5)

MSE = mean_squared_error(x_test.values,pred.values)

MAE = mean_absolute_error(x_test.values,pred.values)

MAPE = float(sum(abs((x_test.values - pred.values)/(x_test.values)) * 100)/len(x_test))

plt.plot(pred.index, pred.fillna(0).values, color="red",label='Predicted') plt.xlabel('Date')

plt.ylabel('Energy Generated')

plt.legend()

ARIMA Model

This model integrates the AR and MA models. ARIMA — AutoRegressive Integrated Moving Average.

# ARIMA Model p=1, d=0, q=1

model = ARIMA(box_cox_train, order=(1, 0, 1))

results_AR = model.fit() plt.figure(figsize=(16, 7))

plt.plot(x_train.index, x_train.values, color=”lightblue”)

plt.plot(x_test.index, x_test.values, color=”green”,label=’Original’) # Predicting

pred = pd.DataFrame(results_AR.forecast(len(x_test))[0])

pred.columns = [“energy_gen”]

pred.index = x_test.index # Converting from log to normal value

pred[“energy_gen”] = pred[“energy_gen”].apply(lambda x: math.exp(x)-1) # Measuring error

RMSE = math.pow(mean_squared_error(x_test.values, pred.values), 0.5)

MSE = mean_squared_error(x_test.values,pred.values)

MAE = mean_absolute_error(x_test.values,pred.values)

MAPE = float(sum(abs((x_test.values — pred.values)/(x_test.values)) * 100)/len(x_test))

plt.plot(pred.index, pred.fillna(0).values, color=”red”,label=’Predicted’) plt.xlabel(‘Date’)

plt.ylabel(‘Energy Generated’)

plt.title(‘ARIMA Model’)

plt.legend()

SARIMA Model

While the traditional ARIMA model is best applicable to non-seasonal data, the SARIMA model is capable of modeling a wide range of seasonal data.

Non Seasonal part of the model: (p,d,q). Already found estimated above to be (1,0,1)

Seasonal part of the model: (p,d,q,m). Where m is the number of observations per year. In this case, assuming a weekly basis, m = 7. Iteration would be run to obtain possible seasonal order.

import itertools

pdq = [1,0,1]

print('Possible Orders:')

print(pdq)

print('Possible Seasonal Orders:')

i = j = k = range(0, 2)

a = [7]

for w in list(a):

seasonal_pdq = [(x[0], x[1], x[2],w) for x in list(itertools.product(i, j, k))]

print(seasonal_pdq)

Possible combinations for seasonal order

All possible combinations are tested to minimize loss.

from statsmodels.tsa.statespace import sarimax

for param_seasonal in seasonal_pdq:

mod = sm.tsa.statespace.SARIMAX(box_cox_train.energy_gen,order=(1,0,1),seasonal_order=param_seasonal,enforce_stationarity=False, enforce_invertibility=False)

results = mod.fit()

print('{} x {} - AIC:{}'.format(param, param_seasonal, results.aic))

Combination (1,0,1) x (0,1,1,7) has the least AIC error, and would thus be subjected to the SARIMA model.



mod = sarimax.SARIMAX(box_cox_train.energy_gen,order=(1,0,1),seasonal_order=(0,1,1,7),enforce_stationarity=False, enforce_invertibility=False)

results = mod.fit(disp=-1) plt.figure(figsize=(16, 7))

plt.plot(x_train.index,x_train.values,color='lightblue')

plt.plot(x_test.index,x_test.values,color='green',label='Original') pred = pd.DataFrame(results.forecast(len(x_test)))

pred.columns = ['energy_gen']

pred.index = x_test.index # Converting from log to normal value

pred["energy_gen"] = pred["energy_gen"].apply(lambda x: math.exp(x)-1) # Measuring error.

RMSE = math.pow(mean_squared_error(x_test.values, pred.values), 0.5)

MSE = mean_squared_error(x_test.values,pred.values)

MAE = mean_absolute_error(x_test.values,pred.values)

MAPE = float(sum(abs((x_test.values - pred.values)/(x_test.values)) * 100)/len(x_test))



plt.plot(pred.index, pred.fillna(0).values, color="red",label='Predicted') plt.xlabel('Date')

plt.ylabel('Energy Generated')

plt.title('SARIMA Model')

plt.legend()

Holt Winter’s Model

The idea behind Holt’s Winter is to apply exponential smoothing to the seasonal components in addition to level and trend. Check the model documentation for more.

from statsmodels.tsa.holtwinters import ExponentialSmoothing

fit = ExponentialSmoothing(np.asarray(x_train['energy_gen']) ,seasonal_periods=7 ,damped=True,

trend='add', seasonal='add',).fit()

plt.figure(figsize=(16,7))

plt.plot(x_train.index, x_train.values, color="lightblue")

plt.plot(x_test.index, x_test.values, color="green",label='Original') # Predicting

pred = pd.DataFrame(fit.forecast(len(x_test)))

pred.columns = ["energy_gen"]

pred.index = x_test.index # Measuring error

RMSE = math.pow(mean_squared_error(x_test.values, pred.values), 0.5)

MSE = mean_squared_error(x_test.values,pred.values)

MAE = mean_absolute_error(x_test.values,pred.values)

MAPE = float(sum(abs((x_test.values - pred.values)/(x_test.values)) * 100)/len(x_test))



plt.plot(pred.index, pred.fillna(0).values, color="red",label='Predicted') plt.xlabel('Date')

plt.ylabel('Energy Generated')

plt.title('Holt Winters Method')

plt.legend()

Model Performance Evaluation

It can be seen that the best forecasting model by Mean Absolute Percentage Error (MAPE) is the Seasonal ARIMA model, although, Root Mean Square Error (RMSE) and Mean Square Error (MSE) proves otherwise, favouring MA. But, it should also be stated that the ‘ARIMA family’ models are stochastic — linear model and would not best define a time series with seasonal changes. In addition, time series forecasting becomes better with increasing historical data. Similarly, our model had just 65 observations — pretty small. It is expected that with more data at hand, models would perform better.

In conclusion, we have been able to apply simple forecasting models to energy generation in Nigeria. There are other models that have proven to be efficient in time series analysis and could do better than these used here, including: Facebook’s Prophet, Neural Net, and Kera’s LSTM.

I really do hope you have enjoyed this as much as I would have wanted. Github link to project is here. I’m on a learning path, so, I’m open to constructive criticism and advice. Thanks for your time.

email: chidindego@gmail.com

twitter: @chidindego

LinkedIn Profile: Chidi Ndego

References: Analytics Vidhya’s Intro Course to time series forecasting. An Introductory Study on Time Series Modeling and Forecasting by Ratnadip Adhikari, R. K. Agrawal