# Comparison of Training Approaches for Photovoltaic Forecasts by Means of Machine Learning
## 3 Evaluation Indexes
In fact, it gives a relatively higher weight to larger errors, thus allowing particularly undesirable results to be emphasized. In fact, if we consider the daily trends of the aforementioned indexes (which are shown in Figure 7), it can be seen how they are correlated, while in the same Figure 8, the scatterplot of their normalized values with the relative maxima clearly shows these correlations between the three error indexes. Furthermore, the Pearson-Bravais correlation index \(\rho_{xy}\)[36] has been calculated to underline the direct relationship among the error indexes:  
\[\rho_{xy}=\frac{\sum_{h=1}^{N}{(x_{i}-\mu_{x})(y_{i}-\mu_{y})}}{\sqrt{\sum_{ h=1}^{N}{(x_{i}-\mu_{x})^{2}}}\sqrt{\sum_{h=1}^{N}{(y_{i}-\mu_{y})^{2}}}}\,. \tag{7}\]  
However, as it is shown in Figure 7, the daily evaluation indexes expressed in Equations (3), (5), and (6) could vary a great deal, being unable to give complete information \"at a glimpse\" of the accuracy of the prediction. For example, consider Figures 9 and 10, where the forecasts and the relevant evaluation indexes for 1 April and 4 November 2014, respectively, are depicted. In both cases, daily \(NMAE_{\%}\) values are quite low (around 2-3%) and a forecast assessment solely based on this basis could be misleading. Actually, the 1 April was quite a sunny day and the bell-shaped hourly power curve which has been forecast--the red starred line--was accurately following the measured one--the blue circled line. The cloudy winter day 4 November 2014 was a different story; in fact, the forecast red curve is biased on the noon hours, while the actual blue curve in the morning. However, in the second day, the daily \(NMAE_{\%}\) value is lower. This is owing to the normalisation of the mean absolute error with the net capacity of the plant. Regarding the other evaluation indexes, even if they are correlated, they can exceed the 100% cap, as happens for example to \(WMAE_{\%}\) in Figure 7 on day 72. Figure 8: Normalized daily errors correlated in a scatterplot. Figure 7: Example of the daily errors trend. \(NMAE\): normalized mean absolute error; \(nRMSE\): normalized root mean square error; \(WMAE\): weighted mean absolute error. Starting from these assumptions, and in view of a more useful summary evaluation, an additional performance index is proposed, aiming to provide a value between 0% and 100% of the forecast accuracy. Therefore the _envelope-weighted mean absolute error_, \(EMAE_{\%}\) is defined as:  
\[EMAE_{\%}=\frac{\sum_{h=1}^{N}|e_{h}|}{\sum_{h=1}^{N}max(P_{m,h},P_{p,h})}\cdot 1 00\,, \tag{8}\]  
where the numerator is the sum of the absolute hourly errors, as in \(WMAE_{\%}\), while the denominator is the sum of the maximum between the forecast and the measured hourly power. In particular, this definition is consistent with a graphical representation of the error, where the numerator corresponds to the yellow area shown in Figures 9 and 10 and the denominator is the sum of the gray and yellow areas highlighted in the same figures. With reference to the above-mentioned days, while the two \(NMAE_{\%}\) values are nearly the same, the \(EMAE_{\%}\) is 11% in the first case and 40% in the second case, and it never exceeds 100%. Figure 6:0 Example of a cloudy day forecast—4 November 2014—with the relevant evaluation indexes. Figure 6:1 Example of a sunny day forecast—1 April 2014—with the relevant evaluation indexes. \(EMAE\%\): envelope-weighted mean absolute error. As with the daily \(NMAE_{\%,d}\), in this study we also introduced the mean value of all the \(EMAE_{\%,d}\), which are referred to the \(d\)-th day, calculated over the whole period. Therefore, \(\overline{EMAE_{\%}}\) is the mean of all the daily \(EMAE_{\%,d}\) for a given data-set:  
Figure 6:2