# Optimizing Energy Forecasting Using ANN and RF Models for HVAC and Heating Predictions
## 2 Methodology
### Evaluation Metrics  
There are a variety of metrics used in predictive modeling and regression analysis that show how well a model fits the analyzed systems. Commonly reported metrics include the Root Mean Square Percentage Error (RMSPE), Root Mean Square Relative Percentage Error (RMSRPE), Mean Absolute Percentage Error (MAPE), Mean Absolute Relative Percentage Error (MARPE), Kling-Gupta Efficiency (KGE), and timing Nash-Sutcliffe Efficiency (NSE). Each metric emphasizes different components of model generalization and predictive ability, allowing researchers and practitioners to evaluate how well their models achieve the outcomes they intend. Overall, developing a robust model relies significantly on understanding these metrics to provide reliable forecasts [48; 49; 50].  
RMSPE and MAPE are commonly used to assess the error between predicted and actual values. RMSPE employs a quadratic scoring rule, squaring the differences and, thus, placing greater emphasis on larger errors, making it sensitive to outliers. It is particularly useful in situations where avoiding significant errors is crucial, as it effectively captures substantial discrepancies. In contrast, MAPE is a linear measure of absolute error, which tends to be less influenced by outliers since it averages the absolute errors.  
KGE (Kling-Gupta Efficiency) and NSE (Nash-Sutcliffe Efficiency) are key metrics for performance analysis, especially in water and environmental modeling. KGE is a composite index that evaluates correlation, bias, and differences between predicted and observed data. It allows for a comprehensive assessment, with values close to 1 indicating better performance. NSE further compares the model's predictions against actual measurements, revealing the proportion of variance explained by the model, with coefficients ranging from negative to positive indicating varying degrees of prediction accuracy. Lastly, R\({}^{2}\) measures how well the predicted values account for the variance in the actual data, with values approaching 1 signifying strong explanatory power. Together, these metrics create a solid framework for evaluating and enhancing model performance, promoting better understanding and more effective decision making in the modeling process [51; 52]. These metrics provide a sound basis on which model performance evaluation and improvement can be based, as show in Table 5, therefore offering enhanced insight and decision making in modeling.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Component** & **Equation** & **NO. Equation** \\ \hline The split criterion for a node n can be expressed as follows & Impurity\((n)\ =\ \frac{1}{N}\sum\limits_{i\ =\ 1}^{k}p_{i}(1-p_{i})\) & (8) \\ Prediction & \(\widehat{y}\ =\ \frac{1}{T}\sum\limits_{t\ =\ 1}^{T}\widehat{y}_{t}\) & (9) \\ The predicted class is given by the majority vote among all trees & \(\widehat{y}\ =\ \text{mode}(\widehat{y}_{1},\widehat{y}_{2},\ldots, \widehat{y}_{T})\) & (10) \\ Out-of-bag error & OOB Error & \(=\ \frac{1}{N}\sum\limits_{i\ =\ 1}^{N}\text{I}(\widehat{y}_{i}\
eq y_{i})\) & (11) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Mathematical model for RF.