## 4 Methodology
### Boosting Approaches  
The first version of the boosting algorithm was Adaptive Boosting (AdaBoost), which builds a prediction model based on an ensemble of weak models [27]. In fact, a strong classifier \(G(x)\) is constructed as a linear combination of weak classifiers in the following form:  
\[G_{N}(x)=\sum_{i=1}^{N}\alpha_{i}g_{i}(x)=G_{N-1}(x)+\alpha_{N}g_{N}(x)\]  
where \(x\) represents the input data, \(g_{i}(x)\) is the prediction of the \(i\)-th weak classifiers and \(N\) denotes the number of iterations performed by the algorithm. In the generic \(i\)-th iteration, the corresponding weak learner is selected and the weight \(\alpha_{i}\) is assigned in order to minimize the error [28], so finally, the output \(\alpha_{i}g_{i}(x)\) is computed. The N parameter is established in the training step. Unfortunately, the AdaBoost is afflicted by noise and, consequently, by overfitting. The Gradient Boosting (GB) algorithm is an evolution of the AdaBoost and is based on the calculation of the so-called residuals [29]. The residual is the difference between the real value \(y\) and the approximate value \(\hat{y}=F(x)\) that is predicted [30]. The goal of the algorithm is to minimize the loss function \(L(y,F(x))\), which can be defined through MSE in the following form:  
\[L(y,F(x))=\frac{1}{I}\sum_{i=1}^{I}(y_{i}-\hat{y}_{i})^{2}\]  
where \(I\) indicates the instance number. The model parameters are \(I\) and the number \(M\) of trees. These parameters are varied in order to minimize the following pseudo-residual:  
\[r_{im}=\frac{\partial L(y_{i},F(x_{i}))}{\partial F(x_{i})}\]  
where \(i=1,\ \ldots,\ I\ \text{e}\ m=1,\ \ldots,\ M\). The number \(M\) of trees, which coincides with the number of gradient boosting iterations in the model, is a regularization parameter. In fact, by increasing \(M\) it is possible to reduce the error during the training phase. On the other hand, a too-high \(M\) value causes overfitting, which worsens the model's ability to generalize [31; 32]. XGBoost is a scalable technology that optimizes the boosting concept underlying the GB algorithm [33]. This efficient algorithm allows the implementation of a predictor with  
Figure 1: Functional scheme of the proposed intelligent system. excellent mathematical ability and with reduced computational costs [21; 23]. XGBoost is effective and flexible due to the various hyperparameters [34]. As in the case of the GB algorithm, XGBoost is based on the iteration of \(N\) steps that allow the writing of the function \(F_{i}^{(N)}\), which represents the prediction, in the following form:  
\[F_{i}^{(N)}=\sum_{k=1}^{N}f_{k}(x_{i})=F_{i}^{(N-1)}+f_{N}(x_{i})\]  
where \(x_{i}\) denotes the i-th input feature, \(F_{i}^{(N-1)}\) denotes the prediction at the step \((N-1)\) and \(f_{N}(x_{i})\) denotes the learner at the step \(N\). The objective function, which provides an estimate of the model ability, is the sum of two contributions, that is, the training loss and the regularization term:  
\[Obj^{(N)}=\sum_{i=1}^{M}L(y_{i}-\hat{y}_{i})+\sum_{k=1}^{N}\sigma(f_{k})\]  
where \(L\) is the loss function and \(M\) is the number of the implemented observations. Here, \(\sigma(f_{i})\) expresses the complexity of the tree \(f_{i}\). The regularization term controls the complexity of the model and takes the following form:  
\[\sigma(f)=\gamma T+\frac{\lambda||\omega||^{2}}{2}\]  
where \(T\) is the number of the leaves of the tree and \(\eta\omega\) is the output scores of the leaves, so that \(\left|\left|\omega\right|\right|^{2}=\sum_{j=1}^{T}\omega_{j}^{2}\). The value \(\gamma\) controls the minimum loss reduction necessary to further divide the leaf node and \(\lambda\) represents the regularization parameters. This regularization mechanism, which is not present in the standard GB algorithm, allows XGBoost to significantly reduce overfitting. The generic unregularized XGBoost algorithm can be expressed through the pseudo code that is shown in Algorithm 1.