## 3 Methods
### ML Algorithms
#### 3.2.3 Gradient Boosting Decision Tree (GBDT)  
The GBDT is a powerful boosting algorithm, which builds a series of decision trees as weak learners. The main idea of gradient boosting is to construct each decision tree model sequentially in the gradient descent direction of the loss function based on the residuals of the previous model so that the loss of each iteration decreases along the negative gradient, and the prediction is determined based on the sum of the results of all decision trees [46]. The principle of the GBDT is shown in Figure 6. The framework of the GBDT is described as follows [47].  
Let \(D=\{(x_{i},y_{i})\mid i\in\{1,\ldots,n\},x_{i}\in R^{m},y_{i}\in R\}\) denote the dataset. The basic learner is \(h(x)\), \(x_{i}=(x_{1i},x_{2i},...,x_{pi})\) is the eigenvector, and \(p\) is the number of features. \(y_{i}\) is the predicted value, and \(n\) denotes the number of samples for training. The steps of the GBDT [47,48] are presented as follows:  
1. For regression tasks, the initial predicted value is obtained by calculating the average of all samples, which serves as the basis for subsequent iterations. The initial constant value of the model \(F_{0}(x)\) is given: \[F_{0}(x)=\frac{1}{n}\sum_{i=1}^{n}y_{i}\] (4)
2. For the number of iterations \(m\in\{1,\ldots,M\}\), \(M\) is the times of iteration. The residuals of the \((m-1)\)th iteration of the model with respect to \(y_{i}\) are calculated. Residuals reflect the parts of the current model that have not yet been fitted, and subsequent models are fitted by constructing decision trees for these residuals. \[r_{i}^{(m)}=y_{i}-F_{n-1}(x_{i})\] (5)
3. The basic learner is used to fit a regression tree \(h_{m}(x_{i})\) with \((x_{i},r_{i}^{(m)})\) as the new training data. The construction of a regression tree is achieved through the split of the feature space, ensuring that the data within each leaf node exhibit comparable objective values. 4. The \(m\)th iteration model \(F_{m}(x)\) is updated. \[F_{m}(x)=F_{m-1}(x)+\alpha_{m}h_{m}(x_{i})\] (6) where \(\alpha_{m}\) denotes the learning rate, which is defined as the extent to which each newly generated decision tree influences the final predicted value. 5. Following \(M\) iterations, the final model \(F_{M}(x)\) is obtained. \[F_{M}(x)=F_{0}(x)+\sum_{m=1}^{M}\alpha_{m}h_{m}(x_{i})\] (7)  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Meaning of Parameter** \\ \hline n\_estimators & The number of decision trees in the forest. \\ max\_depth & The maximum depth of each decision tree, controlling model complexity. \\ min\_samples\_split & The minimum number of samples that can be divided into leaf nodes; the default value is 2. \\ min\_samples\_leaf & The minimum sample size that a leaf node must contain, with a default value of 1. \\ max\_features & The maximum number of features used for each split. \\ bootstrap & A Boolean indicating whether bootstrap sampling is used when building trees, and the default is true. \\ \hline \hline \end{tabular}
\end{table}
Table 4: RF model parameters.  
The performance of the GBDT is affected by hyperparameters, and reasonable adjustment of these parameters can improve the predictive accuracy of the model. The meaning of the parameters that need to be tuned in the GBDT model is shown in Table 5.