# Ensemble learning for predicting cement-stabilized soil strength by comparing bagging and boosting techniques

Muhammad Hasnain Ayub Khan, Olivier Cuisinier, Adel Abdallah

► To cite this version:

Muhammad Hasnain Ayub Khan, Olivier Cuisinier, Adel Abdallah. Ensemble learning for predicting cement-stabilized soil strength by comparing bagging and boosting techniques. 9th International Symposium on Geotechnical Safety and Risk (ISGSR), Norwegian Geotechnical Institute, Aug 2025, Oslo (Norway), Norway. 10.3850/GRF-25280825_isgsr-086-P108-cd. hal-05400932

HAL Id: hal-05400932

https://hal.science/hal-05400932v1

Submitted on 5 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# ENSEMBLE LEARNING FOR PREDICTING CEMENT-STABILIZED SOIL STRENGTH BY COMPARING BAGGING AND BOOSTING TECHNIQUES

Muhammad Hasnain Ayub Khan*, Olivier Cuisinier, Adel Abdallah

Université de Lorraine, CNRS, LEMTA, F-54000 Nancy, France

Email address*: muhammad-hasnain-ayu.khan@univ-lorraine.fr

## Abstract

In construction projects, the limited availability of suitable virgin soil often necessitates working with weak or problematic soils, which can pose significant risks to infrastructure if not properly stabilized. Cement is widely recognized as one of the most effective binders for stabilizing weak soils. Experimental studies in the literature highlight the key parameters influencing strength development in cement-stabilized soils (CSS). Despite this, the complexity of the underlying mechanisms remains a scientific challenge, hindering the effective use of cement for soil stabilization. This necessitates conducting a study that dives deep into accessing quantitatively the relative significance of strength-controlling factors in CSS. The objective of this research is to investigate the impact of soil type, initial compaction conditions, cement dosage, cement type, curing time, and porosity/volumetric cement content ratio (·/C<sub>iv</sub>) on the strength development in cement-stabilized compacted soils using ensemble learning approaches. Following an extensive literature review, a dataset was constructed by compiling 260 datapoints from different sources in the literature. The dataset covers a wide range of soil and cement types. The compiled dataset was complemented with 48 additional datapoints generated through laboratory testing. Two ensemble learning approaches i.e. bagging (random forest) and boosting (extreme gradient boost), were utilized to develop predictive models. Both ensemble models were optimized using Bayesian optimization. The extreme gradient boost (XGB) model performed significantly better than the random forest (RF) model. For the XGB model, further insights into the impact of input features on the unconfined compressive strength (UCS) were gained using Shapley additive explanations (SHAP). SHAP showed that initial dry density(Δ̂<sub>i</sub>) has the highest individual influence on the UCS with an average impact of 544.24 kPa. Additionally, SHAP confirmed that while soil and cement types do affect the UCS of CSS, their influence is not as pronounced as that of other parameters. Initial compaction conditions were found to have a more significant impact on strength development than cement dosage. This insight confirms that the initial compaction conditions are more crucial for strength development in CSS as compared to cement dosage.

**Keywords:** Cement stabilized soils, Random Forest, SHAP.

## 1. Introduction

In construction projects, the availability of suitable virgin soil is limited forcing the engineers to deal with weak soils. Such soils are a threat to the stability of infrastructure if not properly dealt with. Cement is widely regarded as one of the most effective binders for stabilizing weak soils but it is affected by different strength-controlling factors i.e., physiochemical properties of soil (e.g., particle size distribution, density, porosity, and Atterberg limits), compaction parameters (moisture content and dry density), the type and quantity of the binder, as well as the curing period and curing conditions (Consoli et al., 2000; Lepakshi and Reddy, 2020). Attempts have been made by the researchers to investigate the controlling factor concerning the UCS in cement-stabilized soils (CSS) through laboratory testing and empirical equations have been proposed to predict the UCS of CSS. (Lorenzo and Bergado, 2004) identified the post-curing void ratio to cement content ratio as a critical factor influencing the strength and compressibility of CSS with high moisture content. They proposed an empirical function that relates the void ratio after curing and atmospheric pressure to the UCS. (Consoli et al., 2011) studied the effect of molding conditions, such as initial dry density and water content, on the UCS of CSS and observed that in unsaturated soils, unfilled voids alter strength by affecting the number of contact points in the soil matrix. To address this, they introduced the porosity/volumetric cement content ratio (·/C<sub>iv</sub>) to capture the combined impact of porosity and cement dosage on UCS and proposed an empirical equation linking molding water content and ·/C<sub>iv</sub> to the UCS. Several other studies have examined the relationship between UCS and strength-controlling factors, presenting these correlations through empirical functions (Kang et al., 2015; Tsuchida and Tang, 2015; Do et al., 2021). While the influence of individual factors on strength can be conveniently evaluated in laboratory settings using empirical functions, understanding their combined effects remains a challenge.

In recent years, artificial intelligence (AI)-based techniques have been increasingly utilized to investigate the combined effects of various factors on UCS (Abdallah et al., 2023; Kardani et al., 2021). However, the AI-based models presented in these studies are tailored to specific soil and cement types, limiting their generalizability to diverse conditions. The first objective of this study is to explore bagging (RF) and boosting (XGB) ensemble learning approaches to predict the UCS of CSS with a focus on diversifying the applicability of the model for different soil and cement types through data compilation from diverse sources in the literature. The second
objective of this study is to get insights into the impact of different strength-controlling factors used in model development using SHapley Additive exPlanations (SHAP). By providing a clear understanding of how each factor influences the model's predictions, SHAP enhances the interpretability of the ensemble models.

## 2. Data compilation and statistical analysis

The datasetto evaluate the impact of initial compaction conditions (initial dry density and initial water content), cement dosage and type, soil type, and curing time on the strength development of CSS was compiled from five different sources in the literature(Abdallah et al., 2023; Baldovino et al., 2020; Chamling et al., 2021; Dong et al., 2022; Mengue et al., 2017). A total of 260 datapoints were collected for different soils stabilized using different types of cement.The compiled dataset was complemented with 48 additional datapoints generated through laboratory testing and the total number of datapoints added up to 308. The descriptive statistics of the data arerepresented in Table 1. The compiled database includes silty sands stabilized with CEM-II and OPC grade 53 cements. The database also covers fine-grained soils stabilized with Type-III and CEM-II cements, as illustrated in the plasticity chart in Fig. 1(a). For the statistical analysis of the data, Pearson correlation analysis was performed as shown in Fig. 1(b). The analysis revealed considerable linear relationships between input features and UCS. Tocheck the multicollinearity among input features, the Variance Inflation Factor (VIF) analysis was performed as shown in Fig. 1(c). The analysis confirmed that there is no multicollinearity among features in the dataset as all the VIF values were less than 4. The categorical feature, representing cement types, was processed using one-hot encoding.

<table><caption>Table 1. Descriptive statistics of the data</caption><thead><tr><th>Feature</th><th>Unit</th><th>Mean</th><th>Standard Deviation</th><th>Minimum</th><th>Median</th><th>Maximum</th></tr></thead><tbody><tr><td>Plasticity index (PI)</td><td>%</td><td>12.6</td><td>7.4</td><td>0</td><td>8</td><td>24.6</td></tr><tr><td>Fine contents (FC)</td><td>%</td><td>74.6</td><td>29.8</td><td>1</td><td>99.2</td><td>99.2</td></tr><tr><td>Initial dry density (Ȧ̇<sub>i</sub>)</td><td>kN/m<sup>3</sup></td><td>17.4</td><td>1.5</td><td>13.8</td><td>17</td><td>20.7</td></tr><tr><td>Initial water content (w<sub>i</sub>)</td><td>%</td><td>15.0</td><td>3.5</td><td>8.6</td><td>14.4</td><td>26.5</td></tr><tr><td>Cement (C)</td><td>%</td><td>4.8</td><td>1.8</td><td>1</td><td>5</td><td>9</td></tr><tr><td>Curing time (T)</td><td>Days</td><td>28.4</td><td>30.5</td><td>0</td><td>21</td><td>90</td></tr><tr><td>· /C<sub>iv</sub></td><td>-</td><td>16.8</td><td>10.1</td><td>5.76</td><td>13.13</td><td>78.2</td></tr><tr><td>UCS</td><td>kPa</td><td>1721.9</td><td>1276.1</td><td>189</td><td>1301.5</td><td>5610</td></tr></tbody></table>

<figure><img src="image_2.png" /><figcaption>Fig. 1. (a) Plasticity chart of the studied soils (b) Pearson correlation analysis (c) VIF analysis</figcaption></figure>

## 3. Ensemble learning: Bagging and Boosting

Tree-based models are known for their fast training process and high interpretability, making them a popular choice in various applications. However, they are particularly susceptible to overfitting, which can limit their standalone performance on complex datasets. In general, the overfitting can potentially be reduced by different strategies i.e., increasing the dataset, early-stopping, network-reduction and regularization (Ying, 2019). For tree-based models, the models are frequently employed as weak learners in ensemble techniques, such as boosting or bagging, which effectively mitigate overfitting and enhance their predictive power. In the context of tree-based algorithms, ensemble learning is broadly categorized into bagging (e.g., random forest) and boosting (e.g., extreme gradient boost). Bagging techniques work by randomly sampling subsets of the training dataset, training multiple independent base estimators in parallel on these subsets, and then aggregating their outputs to
create a single, unified model. This approach helps to reduce variance and improve overall predictive performance by leveraging the combined strengths of the individual estimators. Random forest(RF) is a powerful tree-based method introduced by (Breiman, 2001) in which each decision tree decides independently on a specific problem, and then the results are combined through averaging to make the final prediction as shown in Fig. 2(a). In contrast, boosting methods construct base estimators sequentially, with each new estimator specifically designed to address and minimize the residual errors made by the previous ones. This iterative process allows the model to progressively improve its performance by concentrating on the areas where earlier estimators performed poorly. Extreme gradient boost (XGB), introduced by (Chen and Guestrin, 2016), is a boosting algorithm where decision trees (or models) are created sequentially, and each tree tries to fit the residual of the previously trained tree as shown in Fig. 2(b).

To develop the RF and XGB model, the dataset was randomly shuffled and split into 70% training and 30% testing sets. The models were developed using the Scikit-learn library in Python and optimized using Bayesian optimization to obtain the optimal values of hyperparameters. Bayesian optimization finds the value that minimizes the objective function by creating a probabilistic model that learns from past evaluations and improves over time to make better predictions (Zhang et al., 2021).

<figure><img src="image_3.png" /><figcaption>Fig. 2. Illustration of the working of (a) RF (Zhou et al., 2023) (b) XGB (Potdar et al., 2017)</figcaption></figure>

The predictive performance of RF and XGB models is shown in Fig. 3 via regression plots. The coefficient of determination (R²), root mean squared error (RMSE) and mean absolute error (MAE) values were estimated to be 0.98, 158.07 kPa, and 115.8 kPa, respectively for the training of the RF model, and 0.99, 55.8 kPa, and 38.7 kPa, respectively for the training of XGB model. The R², RMSE, and MAE for testing were observed to be 0.88, 403.3 kPa, and 288.4 kPa, respectively, for the RF model, and 0.97, 196.9 kPa, and 137.1 kPa, for the XGB model.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Predictive performance of ensemble models for (a) Training and (b) Testing</figcaption></figure>

The comparison between the RF and XGB models, shown in Fig. 4, reveals a significant performance advantage for the XGB model over the RF model during both the training and testing phases. This notable difference underscores the superior generalization ability of the XGB model. Additionally, the substantial gap observed between the training and testing accuracy of the RF model strongly suggests that it suffers from overfitting, where the model learns the training data too well at the expense of its performance on unseen data. XGB, on the other hand, performed significantly well on the training as well as testing sets compared to RF model.
<figure><img src="image_5.png" /><figcaption>Fig.4.Comparison of RF and XGB model</figcaption></figure>

4. SHAP model interpretations

Shapley additive explanations (SHAP) were introduced by (Lundberg, 2017) and are used to explain the predictions of black-box machine learning models.SHAP analysis was performed for the XGB model. SHAP feature importance plot in Fig.5(a)revealed that $\hat{A}$has the highest individual impact on UCS with a mean absolute SHAP value of 544.24 kPa followed by $T, C/_{iv},w_i,C$, and $PI$ with mean absolute SHAP values of 364.99 kPa, 285.96 kPa, 216.36 kPa,158.59 kPa and 78.75 kPa, respectively.While the cement quality does have some impact on the UCS but not as significant as the impact of other features.Initial compaction conditions ($\hat{A}_i, w_i$) were found to have a more significant impact on strength development than cement dosage. This insight confirms that the initial compaction conditions are more crucial for strength development in CSS as compared to cement dosage.Fig.5(b) illustrates the SHAP summary plot of the XGB model and goes beyond giving just the feature importance as it explains the positive and negative impact of the feature on the UCS. The colour bar shift from blue, indicating low feature values, to red indicating high feature values. Each dot on the plot shows the SHAP value against a specific value of the feature. It can be seen that as the value of $\hat{A}$increases, the SHAP values shift from negative to positive side indicating a positive impact on the UCS. While for $C/_{iv}$, the SHAP values shift from positive to negative as the feature value increases indicating that $C/_{iv}$ has a negative impact on the UCS. Similarly, $T$ and $C$ have a positive impact on the strength while $w_i$ and $F$ have a negative impact on the strength. Readers are recommended to study (Khan et al., 2025) for more in-depth SHAP analysis of feature affecting the strength development in CSS.In summary,this research presented a comprehensive analysis of the factors that influence the strength of the CSS and also explored the impact of cement type on strength, an aspect frequently overlooked in previous studies.

While the SHAP analysis offered valuable and detailed insights into the influence of individual factors on UCS, an even deeper understanding could be gained by further exploring and visualizing the interactions and dependencies between input features. By examining how different features work together, a more comprehensive view of their collective impact on UCS can be developed.

<figure><img src="image_6.png" /><figcaption>Fig.5. (a) SHAP feature importance plot (b) SHAP summary plot</figcaption></figure>
**5. Conclusions**

Weak soil is a threat to structural stability. While cement stabilization is one of the widely used methods to deal with weak soil, it is affected by different strength-controlling factors. This research aimed at utilizing ensemble machine learning approaches to predict the UCS of CSS while incorporating data pertaining to diverse soil and cement types, and consequently employing SHAP to quantify the relative significance of different factors concerning the UCS. Random forest (RF) and extreme gradient boost (XGB)-based models were developed using datasets compiled from literature and laboratory-generated dataset. Both ensemble models were optimized using Bayesian optimization. The large gap between the training and testing accuracy of the RF model indicates that the model suffered from overfitting as it became overly tailored to the training data, reducing its effectiveness on unseen data. The XGB model was observed to perform significantly better than the RF model during training as well as testing phases. SHAP algorithm was used to explain XGB model predictions which revealed that Ã has the highest individual impact on UCS of CSS. Initial compaction conditions were observed to affect the strength more than cement dosage which confirms that the initial compaction conditions are more crucial for strength development as compared to cement dosage in CSS.

Based on the findings of this study, future research should focus on further exploring the interactions and dependencies among different factors influencing the UCS of CSS to develop a more comprehensive understanding of their combined effects. Additionally, expanding the dataset to include a wider range of soil and cement types, as well as varying environmental conditions, could enhance the generalizability of the models.

**References**

Abdallah, A., Russo, G., Cuisinier, O., 2023. Statistical and Predictive Analyses of the Strength Development of a Cement-Treated Clayey Soil. Geotechnics 3, 465-479.

Baldovino, J. de J.A., Izzo, R.L. dos S., Pereira, M.D., Rocha, E.V. de G., Rose, J.L., Bordignon, V.R., 2020. Equations controlling tensile and compressive strength ratio of sedimentary soil-cement mixtures under optimal compaction conditions. J. Mater. Civ. Eng. 32, 4019320.

Breiman, L., 2001. Random forests. Mach. Learn. 45, 5-32.

Chamling, P.K., Biswal, D.R., Sahoo, U.C., 2021. Effect of moulding water content on strength characteristics of a cement-stabilized granular lateritic soil. Innov. Infrastruct. Solut. 6, 1-10.

Chen, T., Guestrin, C., 2016. Xgboost: A scalable tree boosting system. Presented at the Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794.

Consoli, N., Rotta, G., Prietto, P., 2000. Influence of curing under stress on the triaxial response of cemented soils. Geotechnique 50, 99-105.

Consoli, N.C., Rosa, D.A., Cruz, R.C., Dalla Rosa, A., 2011. Water content, porosity and cement content as parameters controlling strength of artificially cemented silty soil. Eng. Geol. 122, 328-333.

Do, H.-D., Pham, V.-N., Nguyen, H.-H., Huynh, P.-N., Han, J., 2021. Prediction of unconfined compressive strength and flexural strength of cement-stabilized sandy soils: a case study in Vietnam. Geotech. Geol. Eng. 39, 4947-4962.

Dong, X., Bao, X., Cui, H., Xu, C., Chen, X., 2022. Effects of cement treatment on mechanical properties and microstructure of a granite residual soil. Appl. Sci. 12, 12549.

Kang, G., Tsuchida, T., Athapaththu, A., 2015. Strength mobilization of cement-treated dredged clay during the early stages of curing. Soils Found. 55, 375-392.

Kardani, N., Zhou, A., Shen, S.-L., Nazem, M., 2021. Estimating unconfined compressive strength of unsaturated cemented soils using alternative evolutionary approaches. Transp. Geotech. 29, 100591.

Khan, M.H.A., Abdallah, A., Cuisinier, O., 2025. Insights into the strength development in cement-treated soils: An explainable AI-based approach for optimized mix design. Comput. Geotech. 180, 107103.

Lepakshi, R., Reddy, B.V., 2020. Shear strength parameters and Mohr-Coulomb failure envelopes for cement stabilised rammed earth. Constr. Build. Mater. 249, 118708.

Lorenzo, G.A., Bergado, D.T., 2004. Fundamental parameters of cement-admixed clay—New approach. J. Geotech. Geoenvironmental Eng. 130, 1042-1050.

Lundberg, S., 2017. A unified approach to interpreting model predictions. ArXiv Prepr. ArXiv170507874.

Mengue, E., Mroueh, H., Lancelot, L., Eko, R.M., 2017. Mechanical improvement of a fine-grained lateritic soil treated with cement for use in road construction. J. Mater. Civ. Eng. 29, 4017206.
Potdar, K., Pardawala, T.S., Pai, C.D., 2017. A comparative study of categorical variable encoding techniques for neural network classifiers. Int. J. Comput. Appl. 175, 7–9.

Tsuchida, T., Tang, Y.X., 2015. Estimation of compressive strength of cement-treated marine clays with different initial water contents. Soils Found. 55, 359–374.

Ying, X., 2019. An overview of overfitting and its solutions. Presented at the Journal of physics: Conference series, IOP Publishing, p. 022022.

Zhang, W., Wu, C., Zhong, H., Li, Y., Wang, L., 2021. Prediction of undrained shear strength using extreme gradient boosting and random forest based on Bayesian optimization. Geosci. Front. 12, 469–477.

Zhou, J., Wang, Z., Li, C., Wei, W., Wang, S., Armaghani, D.J., Peng, K., 2023. Hybridized random forest with population-based optimization for predicting shear properties of rock fractures. J. Comput. Sci. 72, 102097.