## 2 Data Preprocessing
### The XGB Algorithm
#### 2.2.2 Ten-Fold Cross-Validation Parameter Tuning  
The XGB algorithm does not rely on predefined relationships between input and output variables The specificity of hyperparameters lies in the fact that they are not learned from data but require experience and experimentation to determine the optimal values. Their importance is evident in the fact that selecting appropriate hyperparameters can significantly enhance model performance, influencing both the model's generalization ability and training effectiveness. Hyperparameter tuning is a crucial task in machine learning and is essential for constructing effective models. Therefore, it is essential to tune these hyperparameters [45; 46; 47]. The hyperparameters associated with the XGB algorithm are presented in Table 2.  
For hyperparameter tuning, 10-fold cross-validation is commonly employed as a statistical method to assess the model's generalization ability [48; 49]. According to the theory of 10-fold cross-validation, the training set is divided into ten parts, with nine parts used as the test data and one part as the validation data in each iteration. The results of 10-fold cross-validation are more robust compared with the randomness of data partitioning. Since each data point appears in different rounds of model selection, the final performance evaluation is more statistically meaningful. By adjusting hyperparameters in each iteration of cross-validation and selecting the best-performing ones, it is more efficient to find the parameter configuration that optimizes the model's performance. To evaluate the XGB algorithm's performance in predicting the unconfined compressive strength of the training set under different parameter combinations, various hyperparameters were introduced, and the 10-fold cross-validation method was applied. The optimal parameter was selected based on the performance observed during cross-validation. Figure 1 illustrates the 10-fold cross-validation method, and Table 3 presents the results of hyperparameter tuning.  
\begin{table}
\begin{tabular}{c c c} \hline
**Hyperparameterization** & **Meaning of Parameter** & **Parameter Tuning Range** \\ \hline learning\_rate & Learning rate & 0.01, 0.05, 0.07, 0.1, 0.2 \\ \hline n\_estimators & Optimal number of iterations & [100,1000] \\ \hline max\_depth & Maximum depth of the tree & 3, 4, 5, 6, 7, 8, 9, 10 \\ \hline min\_child\_weight & The minimum sum of weights of the child nodes generated from the sample & 1, 2, 3, 4, 5, 6 \\ \hline subsample & Sample proportion for each tree & 0.5, 0.6, 0.7, 0.8, 0.9, 1 \\ \hline colsample\_bytes & The ratio of features randomly selected for each tree & 0.5, 0.6, 0.7, 0.8, 0.9, 1 \\ \hline gamma & The reduction ratio of the loss function & 0.1, 0.2, 0.3, 0.4, 0.5, 0.6 \\ \hline reg\_alpha & Weights for L1 regularization & 0.05, 0.1, 1, 2, 3 \\ \hline reg\_lambda & Weights for L2 regularization & 0.05, 0.1, 1, 2, 3 \\ \hline \end{tabular}
\end{table}
Table 2: Hyperparameters of XGB algorithm.  
\begin{table}
\begin{tabular}{c c} \hline
**Hyperparameterization** & **Optimal Results** \\ \hline learning\_rate & 0.1 \\ \hline n\_estimators & 225 \\ \hline max\_depth & 10 \\ \hline min\_child\_weight & 4 \\ \hline Subsample & 0.5 \\ \hline colsample\_bytes & 0.9 \\ \hline Gamma & 0.3 \\ \hline reg\_alpha & 0.02 \\ \hline reg\_lambda & 0.9 \\ \hline \end{tabular}
\end{table}
Table 3: Hyperparameter tuning results.