## 2 Materials and Methods
### Machine Learning Methods
#### 2.1.4 Regression Trees and Tree-Based Ensemble Methods  
An RT is a variant of the decision tree model [54] in which the target variable is continuous. Using a set of decision rules, the interdependence between the predictor variables and the target variable is graphically represented in the form of a tree structure. This structure is defined by the topology of the tree (configuration of nodes and arcs) and the splitting rules applied to build the tree. Given a training set \(\{(x^{(1)},y_{1}),(x^{(2)},y_{2}),\ldots,(x^{(m)},y_{m})\}\) with \(x^{(i)}=(x_{i1},\ldots,x_{id}),\ i=1,2,\ldots,m\), and with the aim of predicting the values of the quantitative variable \(Y\) from the values obtained for the predictor variables \(X_{1},X_{2},\ldots X_{d}\), a recursive process is used to build the tree, starting from the root node representing the entire current dataset. The dataset is then partitioned into smaller and smaller subsets based on the input variables considered in an attempt to obtain similar or homogeneous behaviour within nodes with respect to the target variable \(Y\). In an RT, the space of all joint predictor variable values is partitioned into disjoint regions \(R_{j},\ j=1,2,\ldots,J\), which are represented by the terminal nodes of the tree. A constant value \(\gamma_{j}\) is assigned to each region, and the predictive rule is obtained as \(x\in R_{j}\to f(x)=\gamma_{j}\). Hence, a tree can be expressed as \(T(x;\Theta)=\sum_{j=1}^{J}\gamma_{j}I(x\in R_{j})\) with parameters \(\Theta=\left\{R_{j},\gamma_{j}\right\}_{1}^{J}\), where \(I(\cdot)\) denotes the indicator function of a set. The parameter values are found with the aim of minimising some specified loss function \(L\), that is, \(\tilde{\Theta}=\arg\min\limits_{\Theta}\sum\limits_{j=1}^{J}\sum\limits_{x_{i }\in R_{j}}L(y_{i},\gamma_{j})\)[55]. In this context, some well-known RT algorithms are CHAID, exhaustive CHAID, and CRT. We note that when they are used for prediction in regression problems, these algorithms use the mean of the corresponding nodes. In addition, CHAID and exhaustive CHAID use the \(F\)-test as a splitting criterion [56]. The CRT algorithm builds binary trees to maximise within-node homogeneity. The node impurity is a measure of the homogeneity between the values of the node, and in regression problems, the impurity is typically measured using the least-squared deviation [19]. However, overfitting is a common problem in decision trees, and very different trees can be obtained with small changes in the training dataset. To overcome this limitation, RF can be used [57]. RF is an ensemble machine learning method that combines several decision trees based on various random subsamples (see Figure 4). The construction of the RF model is usually done using bagging (bootstrap aggregation). The bagging technique uses several training datasets obtained from the full dataset by random sampling with replacement. Each training dataset is then used to build a tree model. That is, in RF, each tree is trained independently and at the same time. An ensemble of these strong learners, using for example the average of all of them, produces a more robust model than a single decision tree [58]. The important hyperparameters in RF include the number of trees used to build the model, the function chosen to measure the quality of each split (MSE, MAE, etc. ), and several useful parameters for controlling the growth of the trees. Increasing the number of trees when building an RF model does not cause overfitting. Figure 4: Flowchart of the RF algorithm. In contrast, boosting is an ensemble iterative technique in which each new tree is built in an attempt to minimise the errors of previous trees, and thus each tree depends on the previous one. This technique minimises the loss function by adding, at each step, a new tree that best reduces the loss function. A well-known method of this type is GBR [59; 60]. Algorithm 1 describes this technique.