# Deep Learning for Short-Term Load Forecasting--Industrial Consumer Case Study
## 4 Results
In other scenarios, compared with LSTM, the performance loss of GRU decreases. GRU can forget and choose memory with one gate, and fewer parameters, while LSTM needs to use more gates and more parameters to complete the same task. For our scenario, with short-term industrial load forecasting in Figure 8, the GRU network offers better result than the LSTM and combined GRU and LSTM networks Figure 9. Because in the training stage the same architecture for the RNN networks were used, the LSTM is underfitted in  
Figure 6: Comparison between the actual and the forecasted values with GRU, GRU+LSTM and AR methods. Figure 7: Comparison between the actual and the forecasted values with AR, LSTM encoder-decoder, LSTM, MLP and simple RNN methods. the training stage. Building larger and training for more epochs did not improve the errors and the network failed due to overfitting on the training data. LSTM-GRU network improves the overall errors, because of the GRU layer, but there is a clear pattern of high variations for peak values and off-peak period. The authors in [47] could find a clear difference between LSTM and GRU, and suggest that the selection of the type of gated recurrent unit depend on the dataset and corresponding task. In our case, with industrial load forecasting, the results are clearly in favor of GRU. For a better understanding of the evolution of the GRU results for the testing period in Figure 10 the actual and forecasted load curve can be correlated with the daily MAPE. The best daily MAPE is 1.58%, but the worst error value is 25.38%. The high error was caused by two legal holiday periods from 30 November to 1 December 2019. The GRU fails to correctly identify the evolution of the actual consumption because in the training dataset, this period is not seen by the network, and the GRU can not generalize on such a situation. On top of this, the previous weekend, an uncommon event occurred due to probable losses in the compress air system. Figure 8: Comparison between the actual and the forecasted values with GRU. Figure 9: Comparison between the actual and the forecasted values with LSTM-GRU and LSTM. The process of using DL algorithms is stochastic, and with different MAPE values scored by using the same architecture, is difficult to fine-tune the hyperparameters of the neural networks to generalize efficiently on the training data. For this reason, in Figure 11 various implemented architectures are analyzed for the best method implemented in this article. The results indicate that using a complex architecture can lead to higher errors due to overfitting the training data. The lowest forecasting error scored by the GRU model (MAPE 4.82%) was reached by using a network with three hidden layers (GRU 241311001100148124). It can be observed that the training MAPE error keeps decreasing, the test error increasing, indicating that the network is overfitting and fails to generalize on the new data. This situation needs to be quantified to find the structure that offers the lowest errors. A simple indicator was defined to quantify the complexity of the DL algorithm in Equation (10) and to compare the results of the GRU algorithm. \[DL_{index(i)}=\frac{E_{i}\times N_{pi}}{E_{max}\times N_{pmax}} \tag{10}\]  
where \(i\) is the number of analyzed architectures, \(E\) is the number of epochs, and \(N_{p}\) is the number of total parameters used in the \(DL\) architecture (weight and biases). \(E_{max}\) is the maximum number of epoch used in all the simulations, and \(N_{pmax}\) is the maximum number of parameters in all simulations. For each simulation, the (\(DL_{index}\)) is compared with the MAPE for both datasets (train and test) and training time to make sure the \(DL\) does not overfit on the training data. In Figure 12, the evolution of error concerning epochs, time of training, and the complexity of the DNN (\(DL_{index}\)) are highlighted. Training complex architectures with a high number of epochs, besides time and resources, leads to higher errors because of overfitting.