## 7 Model Validation
However, it can be observed that the accuracy decreases in the test set, especially in 2015, where the simulated installed PV is significantly lower than the observed one; the proposed ABM overestimates w.r.t. the historical data. The statistical results obtained considering both the train and the test set were the following: \(R^{2}\) = 0.991, MAE = 0.524, RMSE = 0.693, and MAPE = 7.71. Conversely, if we computed the average values only over the test period, these were the results: MAE = 0.896, RMSE = 1.16, and MAPE = 16.36. The precision was skewed especially by the large error made in 2015 (MAPE equal to 32.95). Probably this was due to the fact that the ABM underestimates the impact of the reduced national and regional incentives on the households decision process; as a reminder, national incentives by the Italian government ceased at the end of 2013. In order to contextualize these values, the comparison with the validation results proposed by a recent related work could be considered, namely [PERSON] et al. [39] (see Section 2 for details); this work was chosen because it provided easily comparable metrics and has a similar approach (tuning the parameters of a ABM using historical data). The comparison might not be completely fair since [PERSON] et al. used different sets for parameter tuning and validation, respectively, building-based data and area-wide (summing the power of all PV panels installed in a neighbourhood) data while our ABM was trained and validated using region-wide yearly adoption data. However, the comparison is legitimate because we include the test set not used for the parameters calibration, thus increasing the difficulty of the task tackled by the proposed approach. [PERSON] et al. proposed three different ABMs, with MAPE equal to 14.86, 13.57, and 62.52 (average value computed over all years). Our ABM instead has an average MAPE (considering both training set and test set) equal to 7.71, a significantly lower value. Moreover, it can be worth to notice that even the results on the test set alone (MAPE = 16.36) are similar to those obtained by [PERSON] et al., although these numbers are not really comparable since they  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{_Year_} & \multicolumn{2}{c}{_Yearly Installed Power_} & \multicolumn{3}{c}{_Cumulative Power_} \\ \cline{2-7}  & **MAE** & **RMSE** & **MAPE** & **MAE** & **RMSE** & **MAPE** \\ \hline
2007 & 0.02 & 0.04 & 0.001 & 0.0003 & 0.0004 & 3.33 \\
2008 & 0.53 & 0.62 & 15.41 & 0.006 & 0.008 & 12.93 \\
2009 & 0.24 & 0.35 & 4.76 & 0.004 & 0.004 & 3.83 \\
2010 & 0  
refer to entirely different tasks, pure validation ([PERSON] et al.) and prediction (the approach proposed in this paper). Figure 2 shows the PV installation growth rate in the considered time period (2007-2017). The solid blue line corresponds to the PV growth rate predicted by the agent-based model while the black dashed lines depicts the real installation trend in Emilia-Romagna. The \(x\)-axis displays the year and the \(y\)-axis reports the PV power growth in percentage. The figure contains the results for both the training set (period 2007-2013) and the test set (2014-2017). In this way it is possible to observe the quality of the proposed fitting mechanism, by observing the lines discrepancies in the training set. At the same time, the figure reveals that the ABM does not suffer from overfitting and the proposed methodology can be used to create models that generalize well and, consequently, that can be used for predictive purposes--by looking at the differences between the lines in the test set. The visual analysis revealed results similar to those presented with the quantitative analysis. In fact, it can be noted that the parameter tuning was very effective (in practice the two lines overlap in the training set) and that the predictive capability are good as well, albeit slightly less accurate. Focusing on the PV case study, there were also useful insights that can be gained by looking at the installation trends, both real and simulated.