Predicting Electricity Consumption with XGBRegressor
By: Michael Grogan
Tags: Machine Learning, Data Science, Time Series Forecasting, Time Series Analysis

Predicting Electricity Consumption with XGBRegressor

Source: Photo by 3938030 from Pixabay

In this example, XGBRegressor is used to predict kilowatt consumption patterns for the Dublin City Council Civic Offices, Ireland. The dataset in question is available from data.gov.ie.

What is XGBRegressor?

Have you used XGBoost (Extreme Gradient Boosting) for classification tasks before? If so, you will be familiar with the workings of this model.

Essentially, a gradient boosting model works by adding predictors to an ensemble in a sequential fashion, with the new predictor being fit to the residual errors made by the previous predictor. The Hands-On Machine Learning with Scikit-Learn & TensorFlow guide by Aurelien Geron gives quite a good overview of the theory behind this model, and I would recommend referring to this guide for further information.

XGBRegressor seeks to accomplish the same thing — the only difference being that we are using this model to solve a regression problem, i.e. where the outcome variable is numerical.

In the context of time series specifically, XGBRegressor uses the lags of the time series as features in predicting the outcome variable.

Data Manipulation

Before we dive into the XGBRegressor model, let’s take a look at the dataset itself.

The original dataset displays the electricity consumption patterns for each day across 15-minute intervals.

df = pd.read_csv('dccelectricitycivicsblocks34p20130221-1840.csv', engine='python', skipfooter=3)

df

Source: Jupyter Notebook Output

For this purpose, we would prefer to analyse data on a daily basis — analysing consumption patterns every 15 minutes would likely induce too much volatility into the time series to discern any meaningful pattern in the data.

The data is sorted into a daily format through summing the consumption patterns every 15 minutes as follows:

df2=df.rename(columns=df.iloc[0])

df3=df2.drop(df.index[0])

df3

df3.drop(df3.index[0])

df4=df3.drop('Date', axis=1)

df5=df4.drop('Values', axis=1)

df5

df6=df5.dropna()

df7=df6.values

df7

dataset=np.sum(df7, axis=1, dtype=float)

dataset

The array is now displayed in numpy format as follows:

Source: Jupyter Notebook Output

Here is a line graph of the daily consumption patterns:

Source: Jupyter Notebook Output

Now that the daily time series is formed, the data is split into training and test partitions.

train_size = int(len(df) * 0.8)

test_size = len(df) - train_size

train, test = df[0:train_size,:], df[train_size:len(df),:]

A dataset matrix is then created to store the lags of the time series as features:

def create_dataset(df, previous=1):

dataX, dataY = [], []

for i in range(len(df)-previous-1):

a = df[i:(i+previous), 0]

dataX.append(a)

dataY.append(df[i + previous, 0])

return np.array(dataX), np.array(dataY)

Analysis

The below analysis uses the tutorial from Machine Learning Mastery as a template.

When configuring the XGBRegressor model, the first step is to decide the lookback period, i.e. how many prior periods should the model take into account in predicting the consumption value for time t?

Source: Jupyter Notebook Output

When analysing the autocorrelation function, there appears to be weekly seasonality present in the data, i.e. the peak in consumption correlations over time appear every 7 lags.

In this regard, a lookback period of 7 is chosen for the model.

# Lookback period

lookback = 7

X_train, Y_train = create_dataset(train, lookback)

X_test, Y_test = create_dataset(test, lookback)

The XGBRegressor is now fit on the training data.

from xgboost import XGBRegressor model = XGBRegressor(objective='reg:squarederror', n_estimators=1000)

model.fit(X_train, Y_train)

1,000 trees are used in the ensemble initially to ensure sufficient learning of the data. However, the number of n_estimators will be modified to determine whether the same level of accuracy can be obtained with a lower value.

The objective is set to ‘reg:squarederror’, i.e. a regression with squared loss which penalizes errors on extreme values more heavily.

The model is then used to make predictions on the test data:

testpred = model.predict(X_test)

Both Y_test (the actual test values) and testpred (the predicted values) are reshaped in order to compare the model accuracy using the root mean squared error.

Y_test=Y_test.reshape(-1,1)

testpred=testpred.reshape(-1,1)

The RMSE is now calculated:

>>> import math

>>> from math import sqrt

>>> test_mse = mean_squared_error(Y_test, testpred)

>>> rmse = sqrt(test_mse)

>>> print('RMSE: %f' % rmse) RMSE: 437.935136

Let’s compare this to the mean value across the test data:

>>> np.mean(Y_test) 3895.140625

The size of the error is approximately 11% of the size of the mean kilowatt consumption value across the test set. This indicates that the model has done reasonably well in predicting the relevant consumption trends.

Modifying number of trees

Now, remember that 1,000 n_estimators were used in obtaining this value? What if we decide to lower this value? Let’s try 300 n_estimators.

>>> import math

>>> from math import sqrt

>>> test_mse = mean_squared_error(Y_test, testpred)

>>> rmse = sqrt(test_mse)

>>> print('RMSE: %f' % rmse) RMSE: 437.930710

We can see that the obtained RMSE was virtually identical — meaning the model did just as well in forecasting kilowatt consumption across the test set.

When experimenting with 20 trees, the RMSE rose slightly to 440, and at 10 trees the RMSE rose to 471.

This would suggest that a minimum of 20 trees should be used in formulating this model. However, the XGBRegressor does not appear to need much training to learn the pattern of the data — the fact that weekly seasonality is present and a lookback period of 7 is being used suggests that including the appropriate number of lags as features in the model outweighs extra training by simply adding more trees to the model.

With this problem, as any other — data is king.

Conclusion

In this example, you have seen:

How to properly manipulate time series data for proper analysis

Use of the XGBRegressor and how to modify model parameters appropriately

Calculation of RMSE in determining model accuracy across the test set

How to determine the appropriate number of trees to use in the model

Many thanks for your time, and you can find more of my data science content at michael-grogan.com.

Disclaimer: This article is written on an “as is” basis and without warranty. It was written with the intention of providing an overview of data science concepts, and should not be interpreted as any sort of professional advice. The author has no relationship with any third-parties mentioned in this article.