# Load Forecasting with Machine Learning and Deep Learning Methods
## 5 Results
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Split**} & \multirow{2}{*}{**Metric**} & \multirow{2}{*}{**Quartile**} & \multirow{2}{*}{**RF**} & \multirow{2}{*}{**SVR**} & \multirow{2}{*}{**XGBoost**} & \multirow{2}{*}{**MLP**} & \multirow{2}{*}{**LSTM**} & \multirow{2}{*}{**Conv-1D**} \\ \hline \multirow{4}{*}{**Validation**} & \multirow{2}{*}{**nMBE**} & **Median** & 0.56\% & 4.11\% & 0.64\% & \(-\)0.08\% & \(-\)0.02\% & 0.48\% \\  & & **IQR** & 1.73\% & 3.03\% & 1.98\% & 1.49\% & 0.44\% & 0.97\% \\ \cline{2-9}  & \multirow{2}{*}{**nRMSE**} & **Median** & 9.44\% & 13.80\% & 9.33\% & 14.32\% & 2.76\% & 9.64\% \\  & & **IQR** & 2.39\% & 0.57\% & 1.59\% & 0.43\% & 1.71\% & 1.76\% \\ \hline \multirow{4}{*}{**Test**} & \multirow{2}{*}{**nMBE**} & **Median** & \(-\)0.19\% & 4.63\% & \(-\)0.07\% & 0.53\% & \(-\)0.54\% & 1.59\% \\  & & **IQR** & 0.25\% & 0.12\% & 0.16\% & 1.04\% & 0.52\% & 1.93\% \\ \cline{2-9}  & \multirow{2}{*}{**nRMSE**} & **Median** & 7.75\% & 14.62\% & 7.59\% & 15.59\% & 4.74\% & 14.00\% \\ \cline{1-1}  & & **IQR** & 0.52\% & 0.09\% & 0.76\% & 0.06\% & 0.58\% & 0.36\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of error metrics in validation and test sets. Figure 6: Relationship intensity in the adjustment of the models in the test set. of the models are not able to adjust so well to the variable to be predicted. Evaluating the test set, i.e., independently of the training process set, the variance is 3% higher in the case of RF and XGBoost, and around 10% higher in the rest of the models. The bias, i.e., the tendency shown by the \(nMBE\), is stable, showing slight variations in the adjustment of the models. The predictions show that possible variation in new datasets is reduced since the IQR is less than 1% with most techniques. The comparison of the adjustments in the different models built is graphically represented in Figure 7 for the validation sample, where a random cross-validation split was selected. As can be seen, the LSTM is the model that best fit the training sample. This, as expected, can adequately meet fluctuations in electrical loads in spite of it showing a slight underestimation in some periods when maximum demand occurs. The Conv-1D also has a good fit, although it seems to exhibit some anomalous behavior, for instance, in the fluctuations that appear on Wednesdays or not reaching peak levels during the day. As expected with the analysis of the metrics, RF and XGBoost have similar predictions, overestimating residual demands and not achieving a suitable adjustment to the variations in electrical loads. The worst settings are SVR and MLP. Nevertheless, SVR, is capable of reproducing part of the information to poorly meet the electrical loads pattern. The accumulation of inaccuracies observed in the SVR can be seen in the poor values in the metrics. On the other, because of the variability in the adjustment of MLP, its poor calibration does not stand out in the calculated metrics. Figure 8 confirms the above comments about the superior performance of LSTM. This is the only model capable of faithfully reproducing the electrical loads on the testing sample. The main problem in RF and XGBoost seems to be the inability to accurately reproduce the residual demand values. Thus, these models underestimate these periods. This also occurs with Conv-1D, which still scarcely matches the real data fluctuations. As observed with the training sample, the SVR and MLP models performed worst in forecasting the electrical loads. Figure 7: Adjustment in validation sample in a random cross-validation split.