## 3 Materials and Methods
### Modelling Process
#### 3.2.2 Model Training  
The data were divided into training and test subsets that represented 80% and 20% of the data, respectively. The training dataset is used to fit the predictors, whereas test dataset is applied to evaluate the predictive performance of the fitted or developed model. In ensemble method, one way to achieve differences between base models is to train each model on a different subset of the available training data. Models are trained on different subsets of the training data naturally through the use of resampling methods such as cross-validation and the bootstrap, which are designed to estimate the average performance of the model generally on unseen data. Though bagging regression tree forms the training and validation set based on the embedded sampling procedure, crossvalidation resampling technique is applied for both methods. There are diverse types of cross-validation procedures. In the case of a limited dataset, \(K\)-fold crossvalidation technique is the most desirable choice to achieve an unbiased prediction, which in turn enhances the generalization ability of the model without overfitting [21]. In \(K\)-fold cross-validation, the training data are arbitrarily partitioned into \(K\) subsets with approximately the same sizes. Each of the \(K\) subsets is employed as a validation dataset for assessing the performance of the model and the remaining \((K-1)\) subsets as a training dataset. In total, \(K\) models are fit, and \(K\) validation statistics are obtained. The performance evaluations from the \(K\)-folds are averaged to measure the overall performance of the model. In this work, 10-fold crossvalidation was applied.  
Regression trees are fitted on bootstrap samples and aggregated to create bagging regression trees. Boosting regression trees are also formed by fitting multiple regression trees iteratively in such a way that the model training at a given step depends on the models fitted at the previous steps. Every new model focuses its efforts on the most problematic instances, ultimately forming a strong learner. Using optimizable ensemble method, either bagging or boosting regression trees, four models to predict compaction and strength of stabilized soils are developed. Seven features under the category of amended soils, Atterberg limits, and soil classification are employed to predict OMC and MDD of soils. Based on the input variables, two UCS prediction models (UCS-EM\({}^{+}\) and UCS-EM\({}^{-}\)) are developed. UCS-EM\({}^{+}\), besides the seven features, employed two more features describing the compaction properties (OMC and MDD), whereas UCS-EM\({}^{-}\) employed the same features considered in the OMC and MDD models. Classification of the models is presented in Table 3.  
Tuning the hyperparameters of any AI-based models is essential to optimize their performance. In the present work, with the combination of 10-fold crossvalidation, hyperparameter tuning is carried out using Bayesian optimization to improve the performance of the adopted models by finding the optimal combination of hyperparameter values that minimizes the loss function (mean square error). The considered hyperparameters and the search range of each hyperparameter are: (i) Ensemble method--Bag and LSBoost; (ii) Minimum leaf size--integers log-scaled in the range [1, max (2, floor (n/2))], where n is the number of observations; (iii) Number of learners--integers log-scaled in the range [10, 500]; (iv) Learning rate--real values log-scaled in the range [0.001, 1]; and (v) Number of predictors to sample--integers in the range [1, max (2, p)], where p is the number of predictor variables.