# Short-Term Power Forecasting Framework for Microgrids Using Combined Baseline and Regression Models
## 3 Methodology
### Forecasting Performance Metric  
Which indicator to use for comparing the obtained forecasting errors is a very common topic of discussion in the literature. In [64], a summary of possible indicators can be found.  
In classical approaches, the mean average error (_MAE_), root mean square error (_RMSE_) and coefficient of determination (R\({}^{2}\)) are considered useful for the evaluation of model performance. Furthermore, there are other widely used indicators, such as the mean average percentage error (_MAPE_) [34]. However, the _MAPE_ is not convenient for predicting certain variables that contain positive and negative values, which may exist in a distribution network that includes generation resources. In these situations, the _MAPE_ tends to yield very large values due to the existence of zero or low values in the data to be predicted, and this could lead to misunderstanding in their interpretation. Finally, the main indicator used for the comparison of model performances is the coefficient of variation of the root mean square error (_CV(RMSE)_) [34; 64], also called the relative root mean square error (_RRMSE_) by some authors (i.e., [65]). This indicator has the disadvantage of yielding higher forecasting error values than those of other indicators when the variable to be predicted has a low average value (as highlighted in [33]), but it avoids the appearance of very high (or even infinite) values in near-zero samples.  
Another metric that is defined here due to its simplicity is the normalized _RMSE_ (_nRMSE_). While its definition can vary from one author to another [66], here, it is considered equal to the _RMSE_ divided by the capacity of generation (in cases with generation resources) or by the capacity of consumption (in cases with loads).  
The expressions of the abovementioned metrics are shown in (8)-(12):  
\[MAE=\frac{1}{n}\ \sum_{t=1}^{n}\lvert y_{t}-\hat{y}_{t}\rvert \tag{8}\]  
\[RMSE=\sqrt{\frac{1}{n}\ \sum_{t=1}^{n}(y_{t}-\hat{y}_{t})^{2}} \tag{9}\]  
\[MAPE=\frac{1}{n}\ \sum_{t=1}^{n}\Bigl{\lvert}\frac{y_{t}-\hat{y}_{t}}{y_{t}} \Bigr{\rvert} \tag{10}\]  
\[CV(RMSE)(\%)=\frac{\sqrt{\frac{1}{n}\ \sum_{t=1}^{n}(y_{t}-\hat{y}_{t})^{2}}}{ \overline{y}}\cdot 100 \tag{11}\]  
\[nRMSE\ (\%)=\frac{\sqrt{\frac{1}{n}\ \sum_{t=1}^{n}(y_{t}-\hat{y}_{t})^{2}}}{ max(\lvert y\rvert)}\cdot 100 \tag{12}\]  
Furthermore, many other indicators can be found in the literature. The authors in [35] point out the importance of using well-defined indicators as error metrics. One of the main problems faced when comparing the forecasting results of different studies is precisely the absence of error metric standardization. This ambiguity is particularly noticeable in normalized indicators such as the normalized _MAE_ (_nMAE_) or normalized _RMSE_ (_nRMSE_).  
For other applications, some authors propose more specific metrics to emphasize certain aspects of the resulting forecast, such as consumption peaks. For example, the authors in [49] propose a \"semi-metric\" approach to avoid the double penalty effect when a forecasted peak deviates in time from the real peak. This approach is useful for applications specifically centered on peak reductions at the network level; as for these applications, it is more important that peaks be predicted at approximately the correct times rather than not at all. However, for general applications in energy management, it is preferable to follow some of the common indicators that were previously described, as most of the authorsdo in the reviewed bibliography (for example, [44; 45; 47]). Therefore, this is the approach adopted in the present study.  
After analyzing the advantages of each indicator, the \(CV(RMSE)\) was finally chosen, as it is appropriate for the evaluation of a series with both positive and negative values.