## 6 Results
### Statistical Analysis of the Forecasting Errors  
Once MIRD evaluates all models, it is possible to apply descriptive statistics and determine some basic characteristics of the error distributions. Table 4 shows the descriptive statistics of the forecasting errors. Table 4 shows that the second and third quartiles of the errors are 7.49% and 14.02%, respectively. The quartile values indicate that at least 75% of the models are 85.98% accurate, and half of the models have an accuracy of 92.51%. To represent this information visually, Figure 13 shows the relative frequency and cumulative frequency histograms of the SMAPE scores. The cumulative histogram of Figure 13 shows that inside the first bin (models with SMAPE error from 0 to 8.33%) lie around 63% of the models; a total of 82% of the models lie between the first and second bins (SMAPE \(\leq\) 16.66%). This information reflects that the great majority of the models were performing their respective forecasts with reasonable accuracy. In many cases, the bad performance of models (e.g., those producing an error larger than 25%) was due to the low quality of the source data; that is, missing data, outliers, and noise. A question that arises by examining the histogram is: what forecasting models predict their respective variables the best? To answer this question, the models were divided in three categories: models with an SMAPE \(\leq\) 10% (Figure 14), models with an SMAPE in the range >10% and \(\leq\)20% (Figure 15), and models with an SMAPE > 20% (Figure 16). \begin{table}
\begin{tabular}{l c} \hline \hline
**Statistic** & **Value** \\ \hline Mean & 14.41 \\ Standard Deviation & 19.92 \\ Minimum & 0.05 \\
25\% & 4.69 \\
50\% & 7.49 \\
75\% & 14.02 \\ Maximum & 187.48 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Description of Forecasting Errors. Figure 13: Relative frequency and cumulative relative frequency histograms of errors. Figure 16: Variables distribution for SMAPE \(\geq\) 20%. Figure 14: Variables distribution for SMAPE \(<\) 10%. Figure 15: Variables distribution for 10% \(\leq\) SMAPE \(<\) 20%. These figures show that the voltage variables are the easiest to predict, given their low variation range; their SMAPE was \(<\)10%. On the other extreme, the kVAR variable represents 66% of the total models in the category of SMAPE \(\geq\) 20%. These percentages indicate that kVAR is the most challenging variable to predict. The rest of the variables are almost uniformly present in each set. Except for kVAR, there is no evidence of bias towards any variable; i.e., the accuracy of any given model depends on the quality of the data provided. To further prove that the model design was adequate, one model from each category was randomly selected to conduct a brief analysis of its forecasting errors. Figure 17 shows the forecasts made by a model with SMAPE \(<\) 10%. The forecast correctly follows the actual data with small differences during parts of the day, except for the first day when the model could not predict a sudden change. Note that the model accurately reaches the low and highs of the day for most days. Figure 18 shows the forecasts made by a model with 10% \(\leq\) SMAPE \(<\) 20%. The model follows the general behavior of the curve, although the actual values contain abrupt changes. In addition, the actual values show a large unexpected increment during the first readings. These readings showcase an important disadvantage of the system; it cannot forecast sudden changes. Figure 19 shows the forecasts of a model with SMAPE \(\geq\) 20%. This case is similar to the previous model, where the forecasts followed the curve's general shape. However, since the forecasts are made seven days in advance, it is impossible to anticipate such behavioral changes in the time series. The trained models were capable of following the overall shape of the respective curves with varying degrees of success. The reasonable-error indices indicate that the models could generalize the time-series' behavior during training without overfitting.