# Comparison of Training Approaches for Photovoltaic Forecasts by Means of Machine Learning
## 3 Evaluation Indexes  
The effect of the different methods of training is investigated by means of some evaluation indexes. These indexes aim at assessing the accuracy of the forecasts and the related error, and it is therefore necessary to define the indexes. There is a wide variety of existing definitions of the forecasting performance, and technical papers present many of these indexes; hence, we will report some of the most commonly used definitions in the literature ([32; 33; 34]). The hourly error \(e_{h}\) is the starting definition given as the difference between the hourly mean values of the power measured in the \(h\)-th hour \(P_{m,h}\) and the forecast \(P_{p,h}\) provided by the adopted model [32; 35]:  
\[e_{h}=P_{m,h}-P_{p,h}\ \ \ \left(\mathrm{W}\right). \tag{1}\]  
From the hourly error expression and its absolute value \(|e_{h}|\), other definitions can be inferred; i.e., the well-known mean absolute percentage error (\(MAPE\)):  
\[MAPE=\frac{1}{N}\sum_{h=1}^{N}\left|\frac{e_{h}}{P_{m,h}}\right| \cdot 100\,, \tag{2}\]  
where \(N\) represents the number of samples (hours) considered: usually it is calculated for a single day, month, or year. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Data-Set** & **Trials** & **Samples** \\ \hline A & Incremental & Dependent & Consecutive (10\%_vs_ 90\%_ts_) \\ A\({}^{\star}\) & Incremental & Dependent & Consecutive (90\%_ts_ 10\%_vs_) \\ B1 & Incremental & Independent & Random \({}^{\dagger}\) \\ B2 & Incremental & Dependent & Random \({}^{\ddagger}\) \\ C1 & Complete & Independent & Random \({}^{\ddagger}\) \\ C2 & Complete & Dependent & Random \({}^{\ddagger}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Different methods for the composition of the ANN training data-sets which have been analysed. \({}^{\ddagger}\) (90%_ts_ 10%_vs_) \(ts=\) training set; _vs_ = validation set. Figure 6: Hourly samples belonging to an extended period of time in a complete training database are randomly mixed. Since the hourly measured power \(P_{m,h}\) significantly changes during the same day (i.e., sunrise, noon, and sunset), for the sake of a fair comparison, in this paper the authors preferred to consider the _normalized mean absolute error_\(NMAE_{\%}\):  
\[NMAE_{\%}=\frac{1}{N}\sum_{h=1}^{N}\left|\frac{e_{h}}{C}\right|\cdot 100\,, \tag{3}\]  
where the percentage of the absolute error is referred to the rated power \(C\) of the plant, in place of the hourly measured power \(P_{m,h}\). In this paper we also adopted the mean value of all the \(NMAE_{\%,d}\), which refers to the \(d\)-th day, calculated over the whole period. Therefore, we introduce \(\overline{NMAE_{\%}}\), which is the mean of all the daily \(NMAE_{\%,d}\) obtained with a given data-set:  
\[\overline{NMAE_{\%}}=\frac{1}{D}\sum_{d=1}^{D}NMAE_{\%,d}\,. \tag{4}\]  
The _weighted mean absolute error_\(WMAE_{\%}\) is based on total energy production:  
\[WMAE_{\%}=\frac{\sum_{h=1}^{N}|e_{h}|}{\sum_{h=1}^{N}P_{m,h}}\cdot 100\,. \tag{5}\]  
The _normalized root mean square error_\(nRMSE\) is based on the maximum hourly power output \(P_{m,h}\):  
\[nRMSE_{\%}=\frac{\sqrt{\frac{\sum_{h=1}^{N}|e_{h}|^{2}}{N}}}{max(P_{m,h})} \cdot 100\,. \tag{6}\]  
This error definition is the well-known root mean square error (\(RMSE\)) which has been normalized over the maximum hourly power output \(P_{m,h}\) measured in the considered time range, for the sake of a fair comparison. \(NMAE_{\%}\) is largely used to evaluate the accuracy of predictions and trend estimations. In fact, often relative errors are large because they are divided by small power values (for instance the low values associated to sunset and sunrise): in such cases, \(WMAE_{\%}\) could result very large and biased, while \(NMAE_{\%}\), by weighting these values with the capacity of the plant \(C\), is more useful. The \(nRMSE_{\%}\) measures the mean magnitude of the absolute hourly errors \(e_{h,abs}\).