## 3 Materials and Methods
### XGBoost  
XGBoost (extreme gradient boosting) is a high-performance extension of the gradient boosting algorithm. It is designed for fast and efficient learning on large-scale datasets while delivering high predictive accuracy. This algorithm sequentially trains weak learners, primarily decision trees, to form a powerful ensemble model. XGBoost improves model performance at each stage using gradient-based optimization and incorporates enhancements that outperform traditional gradient boosting, as shown in Figure 4. In XGBoost, the predicted value \(\hat{y}_{i}\) is computed by summing the outputs of multiple weak learners. This process can be mathematically expressed as follows:  
\[\hat{y}_{i}=\sum_{i=1}^{T}f_{i}(x_{i}) \tag{9}\]where \(T\) represents the total number of generated trees and \(f_{t}(x_{i})\) denotes the output of the \(t\)--tree for the \(i\)--data point xix_ixi. Each tree \(f_{t}(x)\) is sequentially added during training based on the gradient of the loss function. During the training process, XGBoost optimizes the model by minimizing the loss function \(L^{(t)}\) at each iteration. The loss function is defined as follows:  
\[L^{(t)}=\sum_{i=1}^{n}\ell\left(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i})\right) +\Omega(f_{t}) \tag{10}\]  
where \(\ell\left(y_{i},\hat{y}_{i}\right)\) represents the loss between the predicted value \(\hat{y}_{i}\) and the actual value \(y_{i}\), while \(\Omega(f_{t})\) is a regularization term that controls model complexity. The regularization term \(\Omega(f_{t})\) helps reduce model complexity and prevents overfitting, and it is expressed as follows:  
\[\Omega(f_{t})=\gamma T+\frac{1}{2}\lambda\sum_{j=1}^{T}w_{j}^{2} \tag{11}\]  
Here, \(T\) represents the number of leaf nodes in the tree, \(w_{j}\) denotes the weight of each leaf node, and \(\gamma\) and \(\lambda\) are hyperparameters that control the regularization strength. XGBoost supports efficient distributed and parallel processing during training, enabling rapid learning even with large-scale datasets. It demonstrates strong performance in both classification and regression tasks, with key hyperparameters (e.g., learning rate, maximum tree depth, and number of trees) that can be fine-tuned for optimal results. Through this structural design, XGBoost effectively learns complex data patterns, delivers high predictive performance, and is widely utilized across various data analysis and machine learning applications. XGBoost Model and Methodology in This Paper  
This study used the XGBoost algorithm to design a model to solve the multi-class classification problem. As an extension of the gradient boosting algorithm, XGBoost sequentially trains weak learners (decision trees), correcting the errors of previous models to form a strong learner. XGBoost supports optimized parallel processing and regularization techniques, enabling faster and more stable training compared to traditional gradient boosting models. The model was configured with estimators = 100, learning rate = 0.1, and random state = 42, making it well suited for the multi-class classification task. Training and validation datasets were randomly resampled in each epoch to evaluate the model's generalization performance. This random resampling method assessed model performance  
Figure 4: XGBoost model structure. across various data distributions. The model undergoes a fitting process at each epoch using training data and generates predictions and class probability values for the validation set. The model's performance was monitored using multiple evaluation metrics, including F1 score, accuracy, log loss, and area under the curve (AUC). The F1 score was computed using a macro-averaging approach to ensure balanced performance across multiple classes while accurately quantifying the model's overall predictive performance. Log loss measured the difference between predicted probability distributions and actual labels, and AUC was computed using the one vs. rest (OvR) method to assess the model's classification capability.