# Novel Data-Driven Models Applied to Short-Term Electric Load Forecasting
## 4 Results and Discussion  
The objective of this work was to assess the suitability of the different models for STLF. In this section, we present in detail the forecast performance metrics obtained by the different models considered for this research. An additional aim was to present together (and under homogeneous evaluation criteria) the results obtained by classic ML models and new time-series forecasting methods, with an emphasis on novel techniques, which originated in other fields and which have not been applied, or have rarely been applied, to STLF, i.e., models that integrate ideas from deep learning with gradient boosting and ensemble architectures (gaNet) [11], and models based on dynamical systems analysis techniques (DMD) [3; 4; 5]. The analyzed models are presented in detail in Section 3.2. Several forecast performance metrics were considered: mean square error (MSE), mean absolute error (MAE), median absolute error (MAD), coefficient of determination (\(R^{2}\)), relative root mean squared error (RRMSE), and symmetric mean absolute percentage error (sMAPE). The definition of the performance metrics were based on the following definitions (Equations (9)-(12)), where: \(Y\) corresponds to the ground-truth values, \(\hat{Y}\) is the predicted values, \(\overline{Y}\) is the mean values of \(Y\), \(Y_{i}\) represents each particular ground-truth value and \(\hat{Y}_{i}\) represents each particular predicted value, and N is the number of samples:  
\[MSE=Mean\Big{(}\big{(}Y-\hat{Y}\big{)}^{2}\Big{)}\text{: }MAE=Mean\big{(}|Y-\hat{Y}|\big{)} \text{: }MAD=Median\big{(}|Y-\hat{Y}|\big{)} \tag{9}\]  
\[RRMSE=\frac{\sqrt{\sum_{i=0}^{N}\big{(}Y_{i}-\hat{Y}_{i}\big{)}^{2}}}{\sqrt{ \sum_{i=0}^{N}\big{(}Y_{i}\big{)}^{2}}} \tag{10}\]  
\[R^{2}=1-\frac{\sum_{i=0}^{N}\big{(}Y_{i}-\hat{Y}_{i}\big{)}^{2}}{\sum_{i=0}^{ N}\big{(}Y_{i}-\overline{Y}\big{)}^{2}} \tag{11}\]  
\[sMAPE=\frac{100}{N}\sum_{i=0}^{N}\frac{2}{|Y_{i}|+|\hat{Y}_{i}|}\,\%_{0} \tag{12}\]  
All metrics have values greater than zero with no upper limit, except \(R^{2}\) that has an upper limit of 1 with no lower limit, and sMAPE which has an upper limit of 200%. In all cases, the smaller the value, the better the result, except the \(R^{2}\) metric, where the relationship is the opposite. \(R^{2}\) provides an indication of the variance explained by the model. A value of 1 corresponds to a perfect fit of the model to the real data, a value of zero indicates a dummy prediction always using the mean value, and a negative value a worse prediction than always choosing the mean. The other metrics (MSE, MAE, MAD, sMAPE, and RRMSE) are error metrics, and they are always positive, with the best result for a value of zero. We considered especially important the metrics \(R^{2}\), RRMSE and SMAPE, since they are metrics representing a ratio between the error and the actual values. All results provided in this section were obtained with the test set presented in Section 3.1. The results are given after inverting the scale (in the range [0,1]) performed for training. Therefore, the results are based on unscaled load values, which can be especially important for metrics based on actual values and not on ratios (e.g., MSE, MAE, MAD). We provide an extensive analysis of results for different values of the parameters: \(k\) (number of forecast values), \(p\) (number of predictor time-slots), and \(f\) (length of features used as predictors). Figures 6-8 show the forecast metrics for the different models (Section 3.2), and for different values of the parameters \(f\), \(p\), and \(k\). They present the data in two sections, with the upper section presenting the raw data with a color-code format, and the lower section showing a subset of the data in a bar chart format. The bar charts below the tables present the sMAPE metrics (average, T-0 and T-23) for some of the best performing models in the corresponding table. These tables present the metrics using a color code to easily show where the best values are.