# Prediction of Soil Compaction Parameters Using Machine Learning Models
## 4 Discussion  
In this paper, different prediction models are trained for different machine learning algorithms, different kernel functions, and different numbers of neurons, and all models are evaluated for model performance by 5-fold cross validation to obtain the best performing prediction model. The k-fold cross-validation is a commonly used method for model evaluation, in which the database is first divided into k subsets, then each subset is made a test set once, and the remaining k-1 subsets are used as the training set. Finally, the results of multiple evaluations are averaged, thus eliminating the effect of unbalanced database partitioning. The results of the best prediction models in SVM, RF, XGBoost, and ANN are shown in Table 6.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**OMC**} & \multicolumn{2}{c}{**MDD**} \\ \cline{2-5}  & **RMSE** & \(\mathbf{R^{2}}\) & **RMSE** & \(\mathbf{R^{2}}\) \\ \hline SVM & 1.6308 & 0.8847 & 0.5039 & 0.8831 \\ RF & 1.3605 & 0.9198 & 0.5664 & 0.8523 \\ XGBoost & 1.3290 & 0.9234 & 0.4447 & 0.9098 \\ ANN & 1.6302 & 0.8848 & 0.5727 & 0.8490 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Comparison of best model performance for the test set.  
Figure 11: Feature importance of OMC; (**a**) RF; (**b**) XGBoost.  
Figure 12: Feature importance of MDD; (**a**) RF; (**b**) XGBoost.  
As shown in Figure 13, XGBoost has excellent performance in predicting both OMC and MDD, with its \(R^{2}\) reaching above 0.90, and RF also has good performance in predicting OMC (RMSE = 1.3605, \(R^{2}\) = 0.9198), which suggests that the ensemble algorithm possesses superior model performance and generalization ability. However, RF performs poorly in predicting MDD, which may be attributed to the failure to optimize the best combination of hyperparameters. SVM has moderate performance in predicting both OMC and MDD, which is attributed to the fact that the core idea and fundamentals of SVM have not been changed, and it is still considered as a single machine learning algorithm. For ANN, whether it is predicting OMC or MDD, it shows the worst model performance. On the one hand, it may be related to the sample size being too small. On the other hand, it is not paired with an optimization algorithm to optimize the global optimal solution. The optimization algorithm is a kind of algorithm with global optimization performance, generality, and it is suitable for parallel processing. Optimization algorithms can theoretically find the optimal solution or near-optimal solution in a certain time. Commonly used optimization algorithms are Genetic Algorithm (GA), Simulated Annealing (SA), Particle Swarm Optimization (PSO), Ant Colony Algorithm (ACA), and so on.  
Figures 14 and 15 show the actual and predicted values of OMC and MDD for the test set in XGBoost, respectively. From the figures, it can be seen that the overlap between the real and predicted values is very high when the model has excellent performance. Even though some of the data are poorly predicted, the error with the true values is around 4%, which is an acceptable range. This shows that machine learning algorithms can be applied to predict OMC and MDD with great efficiency and advantage.  
Figure 13: Comparison of RMSE for the test set in different models.