# Prediction of Soil Compaction Parameters Using Machine Learning Models
## 2 Materials and Methods
### Machine Learning Algorithms
#### 2.3.4 Extreme Gradient Boosting  
XGBoost is improved from the Gradient Boosting Decision Tree (GBDT) and belongs to the boosting algorithm in ensemble learning. As shown in Figure 5, the boosting algorithm is used to integrate many weak classifiers together to form a strong classifier, and its core idea is to correct the error of the previous model each time a new tree model is trained, so as to improve the overall performance of the model [37].  
XGBoost is an additive model consisting of k tree models. The tree model to be trained in the \(t\)th iteration is \(f_{t}(x)\), and then there is  
\[\hat{y}_{i}^{(t)}=\sum_{k=1}^{t}f_{k}(x_{i})=\hat{y}_{i}^{(t-1)}+f_{t}(x_{i}) \tag{7}\]  
where \(\hat{y}_{i}^{(t)}\) is the prediction result of sample \(i\) after the \(t\)th iteration, \(\hat{y}_{i}^{(t-1)}\) is the prediction result of the first \(t-1\) tree, and \(f_{t}(x_{i})\) is the model of the \(t\)th tree.  
Figure 4: Schematic diagram of RF.  
The loss function can be represented by the predicted value \(\hat{y}_{i}\) and the true value \(\hat{y}_{i}\):  
\[L=\sum\limits_{i=1}^{n}l(y_{i},\hat{y}_{i}) \tag{8}\]  
where \(n\) is the number of samples.  
The objective function is composed of the loss function L of the model and the canonical term, and so the objective function is  
\[Obj=\sum\limits_{i=1}^{n}l(y_{i},\hat{y}_{i})+\sum\limits_{i=1}^{t}\Omega(f_{ i}) \tag{9}\]  
where \(\sum\limits_{i=1}^{t}\Omega(f_{i})\) is the canonical term.  
Bringing Equation (7) into the objective function, the objective function is transformed into  
\[Obj^{(t)}=\sum\limits_{i=1}^{n}l\left(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i}) \right)+\Omega(f_{i}) \tag{10}\]  
The second-order [PERSON] expansion formula is  
\[f(x+\Delta x)=f(x)+f^{\prime}(x)\Delta x+\frac{1}{2}f^{\prime\prime}(x)\Delta x ^{2} \tag{11}\]  
Definition:  
\[\begin{array}{l}g_{i}=\partial_{\hat{y}^{(t-1)}}l\left(y_{i},\hat{y}^{(t-1) }\right)\\ h_{i}=\partial_{\hat{y}^{(t-1)}}^{2}l\left(y_{i},\hat{y}^{(t-1)}\right)\end{array} \tag{12}\]  
A second-order Taylor expansion of the loss function gives the objective function as  
\[Obj^{(t)}=\sum\limits_{i=1}^{n}\left[l\left(y_{i},\hat{y}_{i}^{(t-1)}\right) +g_{i}f_{t}(x_{i})+\frac{1}{2}h_{i}f_{i}^{2}(x_{i})\right]+\Omega(f_{i}) \tag{13}\]  
On the one hand, XGBoost improves the overfitting resistance and generalization ability of the models by adding regular terms to the loss function. On the other hand, the introduction of second-order Taylor expansion makes the model more accurate and efficient in the training and optimization process. The advantages of XGBoost include a more accurate approximation of the loss function, faster convergence of the model, more stable training of the model, more optimized splitting strategy, and a better handling of  
Figure 5: Schematic diagram of XGBoost.  
nonlinear data, etc. [38; 39; 40]. In addition, XGBoost supports parallel processing to increase the speed of model training and provides feature significance to study the impact of features on model results. XGBoost has become one of the most respected algorithms today due to its excellence in most aspects. The risk of overfitting can be reduced by limiting the depth of the tree or setting the minimum number of samples for leaf nodes, which can prevent the model from generating overly complex trees.