# Prediction of Unconfined Compressive Strength in Cement-Treated Soils: A Machine Learning Approach
## 3 Results and Discussion
### Model Performance Comparison
#### 3.2.2 Nested Cross-Validation Results  
Based on the initial screening results, the three top-performing algorithms (Random Forest, Gradient Boosting, and XGBoost) were selected for comprehensive hyperparameter optimization and rigorous evaluation through nested cross-validation with Bayesian hyperparameter optimization [84]. This rigorous evaluation approach ensured unbiased performance estimates while simultaneously generating the prediction error data essential for the subsequent uncertainty quantification framework.  
Table 4 presents the cross-validated performance metrics for all three optimized algorithms, revealing the subtle but important differences in their behavior across the complete range of cement stabilization conditions.  
The performance metrics reveal that all three ensemble methods achieved excellent predictive capability, with R\({}^{2}\) values exceeding 0.94--a threshold generally considered indicative of strong predictive performance in geotechnical engineering applications [88; 104]. However, the subtle differences in their error patterns provide important insights into their suitability for different aspects of the modeling framework.  
Gradient Boosting achieved the highest mean R\({}^{2}\) (0.9488 \(\pm\) 0.0131) and the lowest RMSE (286.23 \(\pm\) 43.45 kPa), demonstrating superior central prediction accuracy. This performance advantage reflects the algorithm's sequential learning approach, which effectively captures complex non-linear relationships by iteratively correcting prediction errors. The RMSE values across all models represent approximately 13% of the mean UCS (2169.93 kPa), indicating strong practical accuracy for engineering applications.  
Random Forest demonstrated particularly balanced performance characteristics that proved crucial for the uncertainty quantification framework. While achieving a marginally lower R\({}^{2}\) (0.9471 \(\pm\) 0.0120) than Gradient Boosting, Random Forest exhibited the most stable performance across validation folds, evidenced by the lowest standard deviation in R\({}^{2}\) scores. More significantly, Random Forest achieved both the lowest MAE (186.17 \(\pm\) 19.82 kPa) and the most consistent MAPE (8.67% \(\pm\) 0.87%), indicating superior average accuracy and minimal relative error variability across the diverse range of UCS values in the dataset.  
XGBoost provided intermediate performance with an R\({}^{2}\) of 0.9467 \(\pm\) 0.0171, but showed the highest variability across folds (largest standard deviations across most metrics), suggesting greater sensitivity to specific data characteristics. This variability, while not necessarily detrimental for point predictions, can complicate the development of well-calibrated uncertainty estimates.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **R\({}^{2}\) (Mean \(\pm\) Std)** & **RMSE (kPa)** & **MAE (kPa)** & **MAPE (\%)** \\ \hline Random Forest & 0.9471 \(\pm\) 0.0120 & 291.23 \(\pm\) 39.00 & 186.17 \(\pm\) 19.82 & 8.67 \(\pm\) 0.87 \\ Gradient Boosting & 0.9488 \(\pm\) 0.0131 & 286.23 \(\pm\) 43.45 & 191.08 \(\pm\) 21.16 & 11.57 \(\pm\) 1.86 \\ XGBoost & 0.9467 \(\pm\) 0.0171 & 289.89 \(\pm\) 48.21 & 195.04 \(\pm\) 29.48 & 9.73 \(\pm\) 1.45 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Cross-validated performance metrics of optimized models.  
The cross-validation results confirmed that all three tree-based ensemble methods could effectively capture the complex relationships governing UCS development in cement-stabilized soil. The consistency across different validation folds suggested that these models would maintain reliable performance when applied to new, unseen data from the same experimental domain.