Multivariate Multi-Step Short-term load forecasting using Deep Neural Networks (DNN) — feature engineering
By: Shreya Pamecha
Tags: Deep Neural Networks, Feature Engineering, Multivariate, Load Forecasting, Forecasting

Photo credit: stockup

Forecasting signifies predicting future values (energy consumption) using historical data and other factors/features/regressors (like temperature, dew point, day of the week) affecting the output. Short-term implies the time between each prediction is short, hourly, or half-hourly. Medium-term can imply daily or weekly forecasts, and long-term can mean monthly or yearly predictions. Multi-step suggests predicting multiple future values in one go whereas one-step is predicting only one future value at a time. For instance, considering a short-term problem, it requires 24 hourly load forecasting from 11:00 of the 1st day (13) to 11:00 the next day (11). Multivariate tells us that multiple variables (or features) are being used as inputs to model a prediction problem.

Note: For architectures and models: you can directly go to the second article.

Data preparation

You need to at least have one-year data for training to incorporate seasonality trends. The average of past hourly data can extrapolate all the missing data (or use exponential smoothening). The dataset should be divided into two parts:

Training (at least one year)

Testing or Validation

Either ignore the data recorded during a lockdown, believing it would have erratic and deviating consumption patterns or add another feature, ‘lockdown flag’ for training the model appropriately.

Feature design and feature selection

It is advisable to visualize your data before choosing features for the training phase. You can visualize your data using a Dashboard (can be made using Dash & Plotly libraries) or using basic Python libraries for data processing — Pandas, Numpy.

The regressors/features/factors affecting the consumption value can be chosen from the following:

Lag features: Historical load sequence — 168 (7 days 24 hours each) or 24 (the previous 24 hours)

Suppose Sundays or weekends come out to be different in terms of consumption level (less) (check it via visualization). In that case, it’s advisable to use the past seven days’ consumption as lag features. However, it is computationally expensive; the execution time per epoch gets 7–8 times. So instead, we can use 48 lag features (only one previous day).

2. Time features — Day Of the Week (DOW), Hour Of the Day (HOD), and Season. These can be represented in the following ways:

a. One-hot encoded features (1D): HOD (24), DOW(7), Months(12)

Mapping the integer values to categorical variables that are illustrated in binary forms. The disadvantage is that feature space can blow up (high dimensionality), increasing the modeling process’s complexity. These shortcomings can be overcome by grouping some months, some days of the week using clustering or observing the trend. For example, seasons can be used instead of months, weekends, and weekdays (2 features) can be used instead of seven days of the week. (Only if the consumption pattern trend matches!)

b. Cyclic features (2D): hour_x, hour_y, day_x, day_y, month_x, month_y (Total 6 features)

hourmin = hour + minute/60

hour_x = sin((2*pi/24)*hourmin)

hour_y = cos((2*pi/24)*hourmin)

EoM — number of days in a month

day_x = sin((2*pi/EoM)*day)

day_y = cos((2*pi/EoM)*day)

month_x = sin((2*pi/12)*month)

month_y = cos((2*pi/12)*month)

Using this, we transform the time data in 2D form using sine and cosine transformations. This aids the model in realizing that these features are occurring in cycles (recurring time intervals). The disadvantage is that the 1st hour will be 0, and the last hour will be 1; it seems we render weight to the ‘hour’ feature, which should not be the case unless we find that the consumption value monotonously increases or decreases with the hour of the day.

c. 1D time — MinMax Scaled to [0, 1]

One column for each feature — DOW (1 to 7), HOD (1 to 24), Seasons/Months (1 to 4 or 1 to 12) and each column minmax scaled to [0, 1]. This kind has the drawback (same as of cyclic features) of unintendedly giving weightage to these three features.

3. Temperature features

This regressor can also be exploited in several ways:

a. Temperature and Temperature_square

By seeing the consumption pattern trend, one can notice if consumption seems to be a little higher when the weather is hot or when the temperature is high. In that case, more energy would be consumed in the summer season than in any other season.

Since we know squaring a higher temperature is significantly greater than squaring a lower temperature, this feature pushes the model to learn that a higher value of temperature_square would lead to higher consumption.

b. Temperature_weekend & Temperature_weekday

Temperature_weekend = Temperature (if it is a weekend)

= 0 (if it is a weekday)

Temperature _weekday = Temperature (if it is a weekday)

= 0 (if it is a weekend)

This feature will ensure consumption difference between weekends and weekdays. After feature engineering, if you find that the consumption patterns are different for weekends and weekdays, then only, you should try this feature.

c. HDD (Heating Degree Days) & CDD (Cooling Degree Days)

This feature gives us value for the whole day. It quantifies the demand for energy needed in the entire day. For instance, the average temperature can be the same for so many days, but HDD varies according to how many times we would require to consume the energy in turning on the A.C (HDD) or the heater (CDD).

In the hourly forecasting (short-term), we must not use HDD or CDD as the temperature itself gives us the notion about energy consumption for each hour.

4. Dew point and Relative Humidity

The relative humidity is the ratio of the amount of water vapor present in the air to the maximum it can withhold at a particular temperature. The more the temperature, the more the atmosphere can withhold. So, we cannot compare two relative humidities at two different temperatures. Thus, we can choose ‘dew point’ instead as a feature in the forecasting model.

5. Wind & Rainfall

If you can get scraped features such as wind and rainfall, it is prudent to use them for better results.

6. Dummy variables: Data Analysis should assuredly inspire other features.

For examples: is_public_holiday, is_working_hour, is_there_power_cut, is_lockdown, is_school_holiday

Go to the second article which consists of the architectures and models that can be used for your forecasting problem.

I hope you enjoyed this article. Happy Learning!!