## 3 Model Results and Discussion  
### Parameters Description of the Regression Functions  
For the Decision Tree (DT), the module _sklearn.tree_ provides the function _DecisionTreeRegressor_. This function supports four different criteria to measure the quality of a split via the parameter _criterion_: the MSE, the MSE with improvement score by [PERSON], the MAE and the Poisson deviance. The size of the regression tree can be controlled by specifying the parameter _max_depth_, which limits the number of nodes in the tree. The maximum number of features to consider when looking for the best split is given by the parameter _max_features_.  
For the Bagging Meta-Estimator (BME), the module _sklearn.ensemble_ provides the function _BaggingRegressor_, taking as input an estimator along with parameters specifying the strategy to draw random subsets. The base estimator used was a DT regressor. The parameter _n_estimators_ is the number of base estimators in the ensemble. The parameters _max_samples_ and _max_features_ control the size of the subsets in terms of samples and features to train each base estimator, while _bootstrap_ and _bootstrap_features_ control whether samples and features are drawn with or without replacement.  
For the Forests of Randomized Trees (FRT), the module _sklearn.ensemble_ provides the function _RandomForestRegressor_. The parameter _n_estimators_ is the number of trees in the forest. This function also supports the same four criteria to measure the quality of a split via the parameter _criterion_ as the _DecisionTreeRegressor_. When building trees, the best node split can be found either from all input features or a random subset of size _max_features_. Bootstrap samples are used by default and the fraction of samples to train each base estimator is given by the parameter _max_samples_.  
For the AdaBoost (AB), the module _sklearn.ensemble_ provides the function _AdaBoostRegressor_, taking as input a user-specified estimator from which the ensemble is built along with other parameters. The base estimator used was a DT regressor. The learning rate can be set via the parameter _learning_rate_, which controls the contribution of each tree in the final combination. This strongly interacts with the parameter _n_estimators_. Empirical evidence suggests that small values of _learning_rate_ favor better test error and lead to better model generalization [71]. Smaller values of _learning_rate_ require larger numbers of weak learners to maintain a constant training error, and this comes at the cost of greater computing time. It is recommended to set the learning rate to a small constant and choose \(n\_estimators\) by early stopping [71]. This function also supports four different loss functions when updating the weights after each boosting iteration via the parameter _loss_: the linear, the square and the exponential.  
Finally, for the Gradient Tree Boosting (GTB) the module _sklearn.ensemble_ provides the function _GradientBoostingRegressor_, which uses gradient boosted trees. It supports four different loss functions to be optimized via the parameter _loss_: the squared error, the absolute error, the huber and the quantile. The Huber loss function is a combination of the first two loss functions. It applies the squared error loss for small deviations from the actual value and the absolute error loss for large deviations. There is also a parameter _alpha_ that dictates the threshold between these losses. This function also supports two different criteria to measure the quality of a split via the parameter _criterion_: the MSE and the MSE with improvement score by Friedman. The size of the individual trees can be controlled by the parameter _max_depth_. The _subsample_ parameter represents the fraction of samples to be used for fitting the individual base learners. The learning rate can be set via _learning_rate_ and the number of estimators can be set via _n_estimators_. The number of features to consider when looking for the best split can be given by _max_features_.