## 3 Results  
### Model Selection Results  
This section presents a detailed analysis of the performance of various machine learning models during the model development phase. The models are evaluated using the \(R^{2}\) metric, which measures the proportion of variance explained by the model. Performance stability is further characterised by standard deviations (SDs) and 95% confidence intervals (CIs) across repeated runs. Table 4 presents the model selection results based on the validation dataset. In more details, the best K-Nearest Neighbours model (five neighbours, \(p=1.0\) for Manhattan distance, distance weighting) achieved a mean RMSE of 1723.41291 and a mean \(R^{2}\) of 0.8081, indicating that the model explains about 80.8% of the variance in the target variable, while the standard deviation of 0.0041 across multiple runs reflects consistent performance. Additionally, the confidence interval (95%) ranged from 0.8074 to 0.8089, further validating the model's stability and reliability. Overall, this result demonstrates moderate predictive accuracy for this model. Regarding the Random Forest models, they were evaluated using two criteria: _friedman_mse_ and _squared_error_. Both configurations used 200 estimators and achieved a mean RMSE of 1405.59474 and nearly identical mean \(R^{2}\) values of approximately 0.8728. This indicates that Random Forest models captured 87.3% of the variance, showcasing strong predictive capabilities. The standard deviation of these models was exceptionally low at around 0.0024, highlighting their stability. Confidence intervals for both configurations were narrow, with lower and upper bounds closely centred around the mean, demonstrating highly reliable performance across evaluations. Neural networks were tested with different hidden layer configurations. A model with 50 hidden layers, combined with the _adam_ optimiser, a learning rate of 0.01 and ReLU activation function, achieved a mean RMSE of 1671.11765 and the highest mean \(R^{2}\) of 0.8193. This configuration outperformed the smaller network with 20 hidden layers with the same hyperparameters, which yielded a mean \(R^{2}\) of 0.8143. Despite their accuracy, Neural Networks showed slightly higher variability, with standard deviations of 0.0153 and \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Model** & **Mean \(R^{2}\)** & **Std. Dev.** & **95\% CI** \\ \hline k-NN & 0.8081 & 0.0041 & [0.8074, 0.8089] \\ Random Forest & 0.8728 & 0.0024 & [0.87275, 0.87285] \\ Neural Network & 0.8193 & 0.0153 & [0.8190, 0.8196] \\ Decision Tree & 0.8909 & 0.0031 & [0.8905, 0.8912] \\ XGBoost & **0.9490** & 0.00093 & [0.9488, 0.9492] \\ LightGBM & 0.9474 & 0.0011 & [0.9469, 0.9478] \\ SVM & 0.7544 & 0.0017 & [0.7541, 0.7547] \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Model selection results on validation set. Figure 2: The methodological workflow. 0.0161 for the 50- and 20-layer configurations, respectively. Confidence intervals were wider than those observed for ensemble methods, indicating that while neural networks perform well, they exhibit less stability compared to models like Random Forest and XGBoost. Despite its simplicity, the best Decision Tree model, using a minimum sample split of 20 and a mean RMSE of 1299.57351, achieved a mean \(R^{2}\) of 0.8909. This result is among the highest single-model performances, demonstrating its capability to explain 89.1% of the variance in the target variable. The standard deviation was low (0.0031), indicating consistent results across runs, while the narrow confidence interval, ranging from 0.8905 to 0.8912, further confirms the reliability of the Decision Tree model. XGBoost emerged as the best-performing model, achieving a maximum mean \(R^{2}\) of 0.9490 with \(\lambda=10\), 500 estimators, a learning rate of 0.05, and a maximum depth of 20. Reducing the number of estimators to 300 slightly lowered the mean RMSE to 888.29617 and the mean \(R^{2}\) to 0.9482, indicating diminishing returns with fewer trees.