# Deep Learning for Short-Term Load Forecasting--Industrial Consumer Case Study
## 3 Materials and Methods
### Proposed Methodology  
This work uses DRNN for hourly variations of the electricity consumption prediction using weather, type of day, day of week and an autoregressive variable (AR) as input in the training and testing dataset. The historical data used as input in the networks is selected based on the observed results. The best-obtained results use the past two weeks of hourly consumption as input in the neural networks. Shorter lag periods increased the MAPE. This aspect means that daily patterns exist and repeat weekly. For the AR method, the lag was selected based on the \(p\)-value analysis.  
Autoregressive method (AR) is a regression implementation of time series data to predict future values based on past correlations. Two weeks of hourly data was analysed and based on the \(p\)-value the relevant past data was used in the forecast Equation (7). To forecast hours \((h_{1.24})\) in day \((d+1)\) we consider hours \((h_{1.24})\) in days \((d-1,d-2\ldots d-14)\)  
\[\hat{Y}_{t}=\beta_{0}+\beta_{1}y_{t-1}+\beta_{2}y_{t-2}+\ldots+\beta_{14}y_{t-1 4}. \tag{7}\]  
Based on the regression \(p\)-value scoring presented in Table 2, we keep the past hours that are relevant for the regression Equation (7).  
All the previous steps with a \(p\)-value score greater than 0.05 are removed from the regression equation. The forecast \(\hat{Y}_{t}\) generated by AR(9) is given as input in the training of the GRU network. Recurrent neural networks consisting of more than two hidden layers are called deep recurrent neural networks (DRNN). The main feature of DRNN is that each hidden state is continuously passed to the next time step of the current layer and the next layer of the current time step. The hidden state of the hidden layer \(H^{\ell}\in\mathbb{R}^{n\times h}\), the output layer variable \(Y_{t}\in\mathbb{R}^{n\times o}\), and the hidden layer activation. The hidden state of the hidden layer \(\ell\) is computed with Equation (8) and the output of the DRNN with Equation (9).  
\[H^{\ell}_{t}=\tanh(H^{\ell-1}_{t}W^{\ell}_{xh}+H^{\ell}_{t-1}W^{\ell}_{hh}+b^{ \ell}_{h}) \tag{8}\]  
\[Y_{t}=H^{(L)}_{t}W_{ho}+b_{o} \tag{9}\]  
\(o\): number of outputs  
\(W^{\ell}_{xh}\in\mathbb{R}^{h\times h}\); \(W^{\ell}_{hh}\in\mathbb{R}^{h\times h}\) and \(W_{ho}\in\mathbb{R}^{h\times o}\): weight parameters  
\(b^{\ell}_{h}\in\mathbb{R}^{1\times h}\) and \(b_{o}\in\mathbb{R}^{1\times o}\): bias parameter  
Considering the mentioned equations, we present in Figure 5 the implementation framework for the deep learning algorithms. All the training variables are used to learn long-term dependencies. In the testing phase, the DRNN model takes as inputs the past three days of hourly consumption, AR(9) hourly forecast for the day (d + 1), and the exogenous variables for the day (d + 1). The output represents the hourly forecast for the next day (24 h).  
Forecasting based on the test dataset is implemented on a day ahead approach with the variables presented in Figure 5. We forecast each day once for 24 h and compare it with the actual consumption. This process is repeated for the entire test data without tunning the DRNN algorithms or other ML algorithms used in this work.  
The historical data used as input in the networks is selected based on the observed results. The best-obtained results use the past two weeks of hourly consumption as input in the neural networks. Shorter lag periods increased the MAPE. This aspect means that daily patterns exist and repeat weekly. For the AR method, the lag was selected based on the \(p\)-value analysis. The exogenous variable considered in forecasting have direct influence in electricity consumption.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & **Coeff.** & **Std. Error** & **t Stat** & \(p\)**-Value** & **Lower 95\%** & **Upper 95\%** & **Lower 95.0\%** & **Upper 95.0\%** \\ \hline  & \(\beta_{0}\) & 0.41 & 0.039 & 6.823 & 0.000000 & 0.157 & 0.31 & 0.17 & 0.401 \\ T-1 & \(\beta_{1}\) & 0.311 & 0.0121 & 31.86 & 0.000000 & 0.46 & 0.402 & 0