## 3 XGBoost  
The integrated machine learning algorithm combines many \"weak learners\" into one \"strong learner\" and has two integrated methods. One of the methods is \"bagging\" [28]. Each weak learner will randomly select some samples for independent training. The final classification result is to calculate the category that all weak learners discriminate the most times (majority voting). The most representative algorithm is random forest [29]. Another method is \"boosting\" [30]. The weak learner has a sequence relationship. The next weak learner will learn the information that the previous weak learner has not learned. After repeating \(N\) times, the \(N\) weak learners are weighted and combined into a strong learner. The most representative algorithm is the adaptive boost (AdaBoost) algorithm.  
[PERSON] and [PERSON] [22] proposed the XGBoost algorithm. It combines the advantages of bagging and boosting and introduces a regularization function to improve the boosting method, and only optimizes the loss function. The regularization function is mainly used to limit the complexity of the model. With the regularization function, the model will be less complicated and is less likely to overfit. XGBoost uses a classification and regression tree (CART) [31] to classify weak learners. CART can be applied to classification tasks because CART uses binary segmentation, and features can be reused to generate trees. Further, it can also be applied to regression tasks. CART uses the maximum Gini index (Gini) as a method to select features to reduce the number of calculations.  
For a given training set \(S\), its Gini index is  
\[Gini(S)=1-\sum_{k=1}^{K}\left(\frac{C_{k}}{|S|}\right)^{2} \tag{23}\]  
where \(C_{k}\) denotes a subset of samples belonging to the \(k\)-th category in \(S\), and \(K\) represents the number of categories. The greater the Gini index, the greater the uncertainty of the data.  
CART uses a binary tree as a decision tree and only classifies node features as \"yes\" or \"no.\" Therefore, the decision tree is equivalent to recursively dicing each feature: divide the feature space into a finite number of units, and determine the prediction probability distribution on these units. Thus, the overall process comprises two steps: decision tree generation and pruning. The generating calculation step is to start from the root node and divide the root node recursively until the stopping conditions are met. The stopping conditions are as follows: (1) the number of samples in the node is less than the preset threshold; (2) the Gini index of the sample set is less than the preset threshold; (3) the depth of the decision tree meets the specified conditions; (4) after the feature is used, it cannot be divided. The pruning step is to start pruning from the bottom of the decision tree \(T_{0}\) generated using the decision tree generation algorithm and pruning one node at a time until the root node of \(T_{0}\) forms a subtree sequence \(\{T_{0},T_{1},\cdots T_{n}\}\). Subsequently, the cross-validation method predicts the subtree sequence in the verification data set, and the best subtree \(T_{a}\) is selected from it.  
Suppose that when a new tree \(f_{n}\) is to be constructed in the \(n\)-th iteration, the objective function is  
\[L^{(n)}=\sum_{i=1}^{\widetilde{T}}l\big{(}y_{i},\hat{y}_{i}^{(n-1)}+f_{n}(x_{ i})\big{)}+\Omega(f_{n}) \tag{24}\]  
\[\Omega(f_{n})=Y\widetilde{T}+\frac{1}{2}\lambda\|w\|^{2} \tag{25}\]  
\[\hat{y}_{i}^{(0)}=0,\;\hat{y}_{i}^{(1)}=\hat{y}_{i}^{(0)}+f_{1}(x_{i}),\ldots,\hat{y}_{i}^{(N)}=\hat{y}_{i}^{(N-1)}+f_{N}(x_{i}) \tag{26}\]  
where \(l(\cdot)\) denotes a loss function that is a convex function;