## 3 Model Results and Discussion
### Hyperparameter Optimization  
The hyperparameters of each model were optimized using a grid search process and a five-fold cross-validation was performed in the training set. For all methods, an exhaustive search over specified parameter values was performed using the function _GridSearchCV_ from the module _sklearn.model_selection_. For the BME model, the parameters that control whether samples and features are drawn with or without replacement were set according to their default stage (samples drawn with replacement and features drawn without replacement). Since averaging methods work best with strong and complex models, the trees for these methods were not given a maximum depth. For the AB and the GTB models, the learning was set to 0.01. For each model, the tuned values for each hyperparameter are shown in Table 4. The coefficient of determination (\(R^{2}\)) was used as the statistical error to obtain hyperparameters with maximum accuracy while minimizing overfitting. Since there are only three inputs, the maximum number of features when splitting a node was considered to be the default value for all the models, which means all of them. The maximum depth for the DT, AB and GTB models was searched in the range between 3 and 15. The number of estimators was searched in the range between 10 and 100 for the BME and the FRT models, and in the range between 100 and 1500 for the AB and GTB models, since for these ones a low value for the learning rate was chosen. The fractions of the maximum number of samples, the alpha and the subsample were searched in the range 0.1-1.0. A total of 161 data samples was used to train all the models, and 41 samples were used to test them. The five-fold cross-validation results and statistical breakdown are shown in Table 5, where the coefficient of variation (COV) based on the average \(R^{2}\) and the standard deviation (STD) are presented for each model. There is no noticeable fluctuation in the results of the five-folds, and the overall accuracy remains very good for all the models. In particular, the FRT model showed to be excellent with the smallest value for the COV of 0.6857%. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Criterion** & **Max Depth** & \(n\) **Estimators** &
\begin{tabular}{c} **Max** \\ **Samples** \\ \end{tabular} & **Loss** & \(\alpha\) & **Subsample** \\ \hline DT & Poisson & 8 & - & - & - & - & - \\ BME & MAE & - & 66 & 0.97 & - & - & - \\ FRT & MAE & - & 56 & - & - & - & - \\ AB & Friedman MSE & 6 & 1250 & - & Square & - & - \\ GTB & Friedman MSE & 3 & 1500 & - & Huber & 0.94 & 0.67 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Hyperparameters for each model. The validation curve for the coefficient of determination as a function of the maximum depth of the tree for the chosen hyperparameters is represented in Figure 6 for the DT, AB and GTB models. For this, _validation_curve_ from the module _sklearn.model_selection_ was used. As referred earlier, for the BME and FRT model the trees were not given a maximum depth, since they are averaging methods. Figure 6 shows that, for all models, the \(R^{2}\) quickly reaches a plateau, and the results stop significantly improving beyond a critical number of maximum depths. It can also be observed that the models are not overfitting, since the training and validation scores are both high. This way, the used values of _max_depth_ were deemed as satisfactory.