## 2 Materials and Methods
### Model Validation  
Model validation is a critical point in the development of UBEMs to ensure they meet their intended objectives. Currently, there is no consensus in the literature on which indicators or acceptable values to use when evaluating an UBEM, resulting in widely varying acceptable accuracy values. [PERSON] et al. [50] study the validation of energy models, highlighting NMBE, CV(RMSE), and R2 as the most used indicators in literature, referencing the ASHRAE guideline as the primary standard in the field. [PERSON] et al. [23] evaluate the accuracy of different UBEMs, both for single buildings and aggregated buildings, based on NMBE, CV(RSME) and R2, finding UBEM accuracies ranging from NMBE \(-\)15 to +4% and CV(RMSE) between 3-50%, with aggregated CV(RMSE) values of 10-20%. The ASHRAE Guideline 14-2014 Measurement Of Energy, Demand, And Water Savings [51] uses NMBE and CV(RMSE) to validate models, considering acceptable results at a monthly scale with NMBE \(\pm\) 5% and CV(RMSE) \(<\) 15%. However, the ASHRAE Guideline applies this validation to district-scale models, categorized as 'UBEM for building-level recommendations' according to the [PERSON] et al. [19] categorization, where each model type requires different precision levels. As noted in [23] aggregated building precision of ten exceeds ASHRAE's recommended values, while single-building precision, especially CV(RMSE), tends to be higher. At a national scale, [PERSON] et al. [10] study simulation accuracy using AutoBEM software to create a national-scale UBEM in the US with OpenStudio and EnergyPlus, validating the model using median error (MdE) and R\({}^{2}\). They achieved an R\({}^{2}\) of 0.069 due to high model dispersion and median errors of \(-3\%\) in residential use, indicating a balance between overestimation and underestimation. Based on the review of existing scientific literature, we identified the most pertinent indicators for model evaluation as those outlined in Equations (2)-(4). These indicators are the most frequently employed in UBEM studies and have established reference values. They are as follows:  
Normalized mean bias error (NMBE (%)):  
\[NMBE(\%)=\frac{\sum_{i=1}^{n}(m_{i}-s_{i})}{\sum_{i=1}^{n}(m_{i})}*100 \tag{2}\]  
Coefficient of variation of root mean square error (CV(RMSE) (%)):  
\[CV(RSME)(\%)=\frac{\sqrt{\frac{1}{n}\sum_{i=1}^{n}(m_{i}-s_{i})^{2}}}{\overline {m}}*100 \tag{3}\]  
Coefficient of determination (R\({}^{2}\)):  
\[R^{2}=1-\frac{\sum_{i=1}^{n}(m_{i}-s_{i})^{2}}{\sum_{i=1}^{n}(m_{i}-\overline {m})^{2}} \tag{4}\]  
where  
\(m_{i}=\textit{measured values}\)  
\(s_{i}=\textit{simulated values}\)  
\(n=\textit{number of measured data points}\)  
\(\overline{m}=\textit{mean of measured values}\)  
The selected metrics, NMBE, CV(RMSE), and R\({}^{2}\), are widely used to evaluate predictive models in this field, but each has inherent limitations that must be considered, particularly when applied to energy models with large datasets, such as those at the national scale. NMBE quantifies the average bias of predictions normalized by the observed mean. This metric is particularly useful at large scales to assess whether the model adequately represents the'mass' of buildings, capturing the values of the most frequent building types. However, when positive and negative errors offset each other, it may falsely indicate a good model fit. For this reason, it is important to include scatter plots of measured versus predicted values in the results to better understand the model's behavior. CV(RMSE) measures the variability of normalized errors but does not distinguish between systematic (bias) and random errors. Due to the quadratic nature of RMSE calculations, this metric is highly sensitive to outliers, disproportionately penalizing cases with large errors. Models dealing with large datasets at national scales often contain outliers, including mislabeled or erroneous data, as well as correct data from statistically infrequent buildings that the model struggles to predict.