## 4 Results and Discussion
### Day-Ahead Hourly STLF
#### 4.1.2 Deep Learning Model's Training and Optimization  
The Deep Neural Network architecture has been selected based on sensitivity analysis, where the four-year hourly demand data was split into 90% training data to train the model, 5% validation-data to tune the hyperparameters and select the best architecture, and 5% testing-data coupled with the last month of 2019 used as a final test the model's generalization performance. The data were then normalized using the Z-scoring method relative to the training data's mean and standard deviation. A sensitivity analysis was used to select the final model's hidden layers' (referred to as HL) configuration shown in Table 7. \begin{table}
\begin{tabular}{c c c} \hline
**No.** & **Endogenous Input Features** & **Range** \\ \hline
**1** & Lagged Demand (\(-24\) h) & \(1195\)–\(3380\) \\
**2** & Lagged Demand (\(-25\) h) & \(1195\)–\(3380\) \\
**3** & Lagged Demand (\(-26\) h) & \(1195\)–\(3380\) \\
**4** & Lagged Demand (\(-48\) h) & \(1195\)–\(3380\) \\
**5** & Lagged Demand (\(-49\) h) & \(1195\)–\(3380\) \\
**6** & Lagged Demand (\(-50\) h) & \(1195\)–\(3380\) \\
**7** & Lagged Demand (\(-168\) h) & \(1195\)–\(3380\) \\
**8** & Lagged Demand (\(-169\) h) & \(1195\)–\(3380\) \\
**9** & Lagged Demand (\(-170\) h) & \(1195\)–\(3380\) \\
**10** & Lagged Demand (\(-192\) h) & \(1195\)–\(3380\) \\
**11** & Lagged Demand (\(-193\) h) & \(1195\)–\(3380\) \\
**12** & Lagged Demand (\(-194\) h) & \(1195\)–\(3380\) \\ \hline \end{tabular}
\end{table} 
Table 6: Endogenous input features related to the demand at previous hours of the week. \begin{table}
\begin{tabular}{c c c} \hline
**No.** & **Exogenous Input Features** & **Range** \\ \hline
**1** & Morning Peak-Load Time Temperature \({}^{\circ}\)C & 4–42 \\
**2** & Evening Peak-Load Time Temperature \({}^{\circ}\)C & 2–37 \\
**3** & Minimum Load-Time Temperature \({}^{\circ}\)C & \(-1\)–34 \\
**4** & Hour of the Day & 1–24 \\
**5** & Day of the Year & 1–366 \\
**6** & Week of the Year & 1–53 \\
**7** & Normal Day & \([0,1]\) \\
**8** & National Holiday & \([0,1]\) \\
**9** & Ramadan & \([0,1]\) \\
**10** & Sunday & \([0,1]\) \\
**16** & Saturday & \([0,1]\) \\ \hline \end{tabular}
\end{table} 
Table 5: Exogenous related to demand at the hour to be predicted. The architecture starts with a high number of neurons, then descends to lower numbers, which aligns with recent works such as [48], and the elu activation function showed slightly higher performance than the RELU activation function. Dropouts were only applied to the first two layers, which showed better performance in combination with _l2_ regularization. Table 8 shows the final performance results for the training, validation, and testing data. The MAPE error for the test data achieved a 1.411% error, just above the validation and training error; hence, the model achieves good generalization and high accuracy. Finally, the predictions of the last month of 2019, which is the final test of the model as it was not used to train the model, similar to the validation and testing data, are shown in Figure 13. The final model prediction results achieved a MAPE error just above the testing data at 2.03% since it is the hardest to predict at the end of the training period. This shows the importance of the continuous training of the model every day when new data is acquired to sustain high accuracy and generalization. To better investigate the occurrence of high errors in the proposed prediction in Figures 13 and 14 depicts the hourly Absolute Percentage Error for each day in December 2019, where each day has been labeled according to its weather description as observed in NEPCO's data. First, the relatively high errors can occur between 8:00 and 17:00, with