## 3 Methods
### XGBoost Ensemble Learning Method  
In addition to deep learning for experiments, the XGBoost ensemble learning algorithm was also used. XGBoost is an ensemble learning method based on GBDT [21] (Gradient Boosting Decision Tree). It improves the model's prediction accuracy by constructing multiple decision trees and adding their prediction results. XGBoost optimizes the original GBDT, including more precise loss functions (using the second-order Taylor expansion) and regularization terms to avoid overfitting and parallel computing. The objective function of XGBoost consists of the loss function and the regularization term [22]. The loss function measures the difference between the model's predicted value and the actual value, while the regularization term controls the complexity of the model to prevent overfitting. XGBoost uses Taylor expansion to approximate the loss function [23] and takes the structural complexity of the tree as the regularization term. Specifically, the Taylor expansion of the objective function [24] can be expressed as:  
\[L(\Theta)\approx L(\Theta^{(t-1)})+g_{i}f_{t}(x_{i})+\frac{1}{2}h_{i}f_{t}^{2} (x_{i}) \tag{5}\]  
Equation (5) represents the second-order Taylor expansion of the loss function used in XGBoost to approximate the objective function. In this equation, \(L(\Theta)\) denotes the loss function of the model with parameters \(\Theta\), while \(L(\Theta^{(t-1)})\) represents the loss [25,26] from the previous iteration (\(t-1\)). The term \(g_{i}\) is the first-order derivative of the loss function [27] concerning the prediction [28, 29, 30] (\(\frac{\partial{L_{i}}}{\partial{g_{i}}}\)), known as the gradient [31]. The function \(f_{t}(x_{i})\) indicates the prediction made by the current decision tree [32] for the input sample \(x_{i}\). Additionally, \(h_{i}\) represents the second-order derivative of the loss function concerning the prediction (\(\frac{\partial^{2}{L_{i}}}{\partial{g_{i}^{2}}}\)), commonly referred to as the Hessian [33]. The coefficient \(\frac{1}{2}\) is used to balance the second-order term, which captures the curvature of the loss function and helps improve the optimization process. This second-order Taylor expansion effectively combines gradient and curvature information to enhance model [34, 35, 36, 37] performance and convergence. During training, XGBoost iteratively adds trees, and in each iteration, it selects the best split point to split the nodes, thereby minimizing the objective function. This process involves complex mathematical derivations and optimization algorithms, but the core idea is to gradually use the gradient boosting method [38] to build a robust predictive model. The general flowchart of the XGBoost model algorithm for constructing trees is shown in Figure 5.  
In this paper, the XGBoost parameter settings are shown in Table 5, with the maximum number of trees generated being 80. These parameter values were obtained through extensive hyperparameter tuning, using cross-validation to determine the optimal configuration. Additionally, to address potential class imbalance, we employed an oversampling method to balance the class distribution in the training data. Figure 5: The Process of Generating Trees in XGBoost. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Max\_depth & 5 \\ Tree & 80 \\ Eta & 0.1 \\ Gamma & 0.3 \\ Lambda & 0.2 \\ Alpha & 0.2 \\ Objective & binary:logistic \\ Eval\_metri & Logloss \\ \hline \hline \end{tabular}
\end{table} 
Table 5: XGBoost Parameter Settings. Algorithm 3 illustrates the training of an XGBoost binary classification model and the performance of feature importance analysis. The algorithm begins by loading the dataset from a CSV file, separating the features and labels. The dataset is then split into training and testing sets, and the data are converted into the DMatrix format, which is optimized for XGBoost. Model parameters are defined, and then the XGBoost model is used on the training data.