## 2 Materials and Methods
### Hyperparamters Tuning  
Hyperparameters of a machine learning model are configuration settings made by the user before training and remain fixed throughout the training process. These settings are the controlling parameters that regulate the behavior of the model and influence its performance and learning process. Optimizing hyperparameters leads to improved generalization and better performance on unseen data, helps in the efficient utilization of  
Figure 10: Fundamental structure of an XGBR model. computational resources, and saves time by finding the right balance between overfitting and underfitting [89]. The process of hyperparameter optimization aims to find the best set of hyperparameters, 'x\", which minimizes an objective score (usually the root-mean-squared error) 'f(x)' evaluated using the validation set. The mathematical representation of the process is shown in Equation (24), where the set of hyperparameters 'x' can be any value chosen from a wide range of values in the hyperparameter space '\(\chi\)'. \[x^{*}=\underset{x\in\mathbb{X}^{\prime}}{\operatorname{argmin}}f(x), \tag{24}\]  
However, the challenge lies in the elevated computational cost associated with evaluating the objective function. Each attempt with different hyperparameters necessitates the training of the model on the training dataset and predictions on the validation data. This process becomes very time-consuming, especially when dealing with numerous hyperparameters or complex models such as ensembles and deep neural networks. In such situations where manual optimization becomes infeasible and extremely complex, automated hyperparameter tuning provides a better alternative solution. Different methods of automated tuning include grid search, randomized search, evolutionary algorithms, and stochastic algorithms such as Bayesian optimization (BO). Of these, grid and randomized searches normally consume a lot of time analyzing the bad hyperparameters, since the processes are completely uninformed about the prior evaluations. On the other hand, evolutionary algorithms require a large number of iterations to converge and can be computationally expensive. Therefore, in this study, BO was used as the technique for tuning the hyperparameters of the regression models. Bayesian Optimization (BO)  
In BO, firstly, a probabilistic surrogate model that represents the objective function is created by employing one of three main types: the Gaussian process, sequential model-based algorithm configuration (SMAC), and tree Parzen estimator (TPE) [60]. Then, by obtaining the projected distribution of this probabilistic surrogate model, an acquisition function accomplishes the allocation of evaluation points by balancing the trade-off between the sampling instances from unexplored areas and the sample data points from the regions likely to contain the global optimum based on the posterior distribution. This enables continuous refinement of the search space and guides the selection of hyperparameter configurations. The selected hyperparameter values are then evaluated on the actual objective function, and the iterations are performed until the maximum threshold is reached. For tuning the regression models in this research work, the Gaussian process surrogate model [90] was used, which is predominantly employed for optimizing continuous variables. Assuming that 'f(x)', defined earlier, is a realization of the Gaussian process and has a mean 'u' and covariance '\(\sigma^{2}\)', and the predictions follow a normal distribution as shown in Equation (25). \[p(y|x,\chi)=\mathcal{N}\left(y\middle|\hat{\mu},\sigma^{2}\right), \tag{25}\]  
Furthermore, an acquisition function based on expected improvement (EI) (Equation (26)) was used, which is the expected value of an improvement function defined in Equation (27), where, '\(\varphi\)(z)' is the probability density function of the normal distribution. N (0, 1).