# Random Forest-Based Stability Prediction Modeling of Closed Wall for Goaf
## 5 Results and Discussion
### Performance of Machine Learning Techniques Results  
After the hyperparameter optimization, the optimal prediction accuracy and other metrics for the random forest-based closed wall damage prediction model are as shown in Table 4. The trained RF algorithm achieved an accuracy of 96%, with a prediction accuracy of 0.99 for the no damage class, a recall of 0.97, and an F1 value of 0.98. For the minor damage class, the prediction accuracy is 0.93, the recall is 0.95, and the F1-score is 0.94. In the severe damage category, the prediction accuracy is 0.94, the recall is 0.94, and the F1-score is also 0.94. The weighted averages for the accuracy, recall, and F1-score are 0.95, 0.96, and 0.95, respectively. The model identified six misjudgment groups in the test set, including one for no damage, two for minor damage, and three for serious damage. The confusion matrix for each model is presented in Figure 7.  
The experimental results show that, by using grid search to optimize the hyperparameters, a high prediction accuracy can be achieved with the RF algorithm. Compared to traditional prediction methods, this approach offers comparable predictive performance but is simpler and easier to deploy. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Accuracy** & **Recall** & **F1** & **Average** \\ \hline No damage & 0.99 & 0.97 & 0.98 & \\ Minor damage & 0.93 & 0.95 & 0.94 & 0.959 \\ Serious damage & 0.94 & 0.94 & 0.94 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Prediction performance table of the random forest model on the training set. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Hyperparameters** & **Number** & **Accuracy** \\ \hline RF & n\_estimators & 250 & \\  & max\_depth & 4 & \\  & min\_samples\_leaf & 4 & \\  & min\_samples\_split & 10 & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Best hyperparameter combination with the random forest algorithm. Figure 7: Confusion matrix of the random forest model on the training set. This figure displays a confusion matrix for a classification model, used to evaluate the model’s predictive performance across different classes. The rows of the confusion matrix represent the predicted classes by the model (Prediction class), while the columns represent the true classes (True class). The value in each cell of the matrix indicates the number of samples where the model predicted a particular class, given the true class of the sample. To test the robustness of the algorithm, its performance was validated on the test set. The confusion matrix of the RF model on the test set is shown in Figure 8. The performance results are shown in Table 5.  
The experimental results on the test set show that the model can still maintain a 95% accuracy on the test set. The performance degradation is not significant, indicating strong robustness. In this study, a machine learning modeling algorithm for predicting the stability of closed walls was proposed by combining the recent development of the RF model and ML methods. The results obtained in this study will help to encourage the use of machine learning in the prediction of closed wall stability, which can be useful for engineering geology-related fields. However, the method presented in this paper does not account for the influence of temperature and humidity on the stability of confined walls. Experimental results may vary under different environmental conditions. Future work could explore hybrid modeling incorporating physical models to consider more influencing factors, evaluate the algorithm's feasibility, and address potential issues accordingly. Furthermore, the sample size used in this study was limited to 167 sets. A larger sample size is needed, and future collaborations across different regions could enhance the model's generalizability. Additional improvements could include expanding the dataset to include multiple mineral types, integrating time-series data to predict the long-term stability, and developing a real-time monitoring system embedded with the model.