# Random Forests and Cubist Algorithms for Predicting Shear Strengths of Rockfill Materials
## 3 Results and Discussion
### Results of Hyper-Parameters Tuning  
In the process of the regression of the random forests and Cubist models, an approximate function is determined to predict the value of the output. First, the original datasets of rockfill shear strength are randomly separated to a training set and a test set. In this paper, approximately four-fifths of the available data were randomly chosen to establish the RFM training dataset. One-fifth of the available data were used for testing purpose. The test set was used to evaluate the accuracy of the function and estimate the performance of the regression models. The previously mentioned methods were adopted to predict the shear strength of the RFM in this section. Figure 3 depicts the total procedure for establishing the predictive models and the application of the predictive models to the testing sets. The input parameters (i.e., D\({}_{10}\), D\({}_{30}\), D\({}_{60}\), D\({}_{90}\), Cc, Cu, Gm, Fm, R, UCSmin, UCSmax, \(\gamma\) and Ns) in the random forests and Cubist models are the variables that affect the prediction target (shear strength of RFM). Before implementing the experiments, an appropriate division of the RFM datasets is necessary. In supervised regression problems, the RFM datasets are divided into training sets and testing sets. The training set is used for developing RFM estimators and the testing set is used for testing the performance of RFM estimators. The number of training sets determines the performance of RFM estimators. Especially, too few training datasets cannot represent the characteristic of the whole dataset and thus the under-fitting may occur. On the contrast, too large training datasets reduce the capability of generalization of estimators. To conduct the experiments, 165 groups of data on rockfill shear strength are randomly split into a training dataset (133) and a test dataset (32), as shown in Figure 3. To guarantee the same outcome distribution of the whole dataset, the training set is selected from the original datasets with a stratified sampling method [32]. The training set is applied to fit a regression model that is employed to evaluate the parameters of random forests and Cubist regression methods. As previously mentioned, a five-fold CV procedure was implemented to determine the final optimum hyperparameters of random forests and Cubist models. Thus, the original training dataset was randomly divided into five groups, where one-fifth of the data was selected as the testing data, and the remaining four-fifths of the data comprises the  
Figure 2: Correlation matrix plot of the original database. training data. For each fold CV procedure, the regression function is established, and a predictive accuracy is obtained, the total five CV procedures are conducted, and the final accuracy is the mean of the previously mentioned accuracy values of the five CV procedures, as demonstrated in Figure 3.  
In the application of random forests experiments, the caret package in R was utilized to build a regression model using a Cubist model tree. To investigate the behavior of the random forests hyper-parameters (_ntree_ and _nttry_), the model was tuned using a relatively fine search grid with CV methods, and the tuning parameters are set as follows: the optimum hyper-parameter _mtry_ is selected in an integer scope of {1,2,3,...,13}, and the optimum hyper-parameter _ntree_ is selected in a scope of [50:1000], incrementing 50 at a time. The smallest value of the RMSE usually indicates the best optimization model. The simplified diagram of the RMSE cross-validated for the random forests is depicted in Figure 4a; the RMSE value of the random forests model is more sensitive to _mtry_ than to _ntree_. The best optimization values for the random forests model with a 5-fold CV procedure (Figure 4a) were _ntree_ = 150 and _mtry_ = 13. The RMSE, R\({}^{2}\) and MAE of the random forests model for 133 sets of training data are 0.1858, 0.9316 and 0.1062, respectively.