# Predicting the Pillar Stability of Underground Mines with Random Trees and C4.5 Decision Trees
## 5 Results and Discussion  
Several performance measures based on a confusion matrix were made using training and testing datasets for underground mines in order to quantify the performance measures of the proposed models. The performance results of the random tree and C4.5 DT were obtained and compared with FDA and SVM models available in the literature based on the same training and testing datasets. The confusion matrix of each model was then determined, which is shown in Table 5. Table 5 also shows the pillar stability prediction results from a couple of previous studies. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{3}{*}{**Dataset**} & \multicolumn{6}{c}{**Model**} \\ \cline{3-10}  & \multicolumn{2}{c}{**RT**} & \multicolumn{2}{c}{**C4.5 DT**} & \multicolumn{2}{c}{**FDA [18]**} & \multicolumn{2}{c}{**SVM [18]**} \\ \cline{2-10}  & \multirow{2}{*}{**True**} & \multicolumn{6}{c}{**Predicted**} \\ \cline{3-10}  & & **Failed** & **Stable** & **Failed** & **Stable** & **Failed** & **Stable** & **Failed** & **Stable** \\ \hline \multirow{2}{*}{Training} & Failed & 28 & 0 & 27 & 1 & 28 & 0 & 28 & 0 \\  & Stable & 0 & 12 & 0 & 12 & 2 & 10 & 0 & 12 \\ \hline \multirow{2}{*}{Testing} & Failed & 4 & 0 & 4 & 0 & 4 & 0 & 4 & 0 \\  & Stable & 0 & 2 & 1 & 1 & 1 & 1 & 0 & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Confusion matrices results for training and testing datasets. The values along the main diagonal show the number of correctly predicted cases. It may be noted from the results of Table 5 that most of the cases were accurately classified using the random tree and C4.5 DT. Based on Table 6, the overall accuracy, recall, precision, F-measure were calculated using Equations (3)-(7), which are given in Table 6. All models have good performance in training and testing the pillar stability of underground mines datasets, according to the result obtained. Furthermore, the overall accuracy, recall, precision, F-measure and MCC for stable and failed classes of the random tree and SVM models on training and testing datasets were found at par. Furthermore, a major drawback of SVM is the black-box nature of the model and the fact that correlation between input and output variables of the system are given in terms of a matrix of weight and biases which make it very challenging for users to access and understand the prediction process [19]. Decision tree algorithms i.e., random tree and C4.5, are quite transparent, and model and internal parameters need not be configured either. The SVM model is a black-box model that can show the input data and the results of the subsequent classification. However, because of its own characteristics, the black-box model cannot demonstrate the internal data mining mechanism, so the useful information relating to pillar stability is difficult to obtain for system operators [35; 42]. Random tree and C4.5 DT are white-box models that are more intuitive and interpretable than ones with the black-box model i.e., the SVM model. Due to tree-like structures, the proposed models can not only obtain accurate classification results, but can also show the internal mechanism for classification results. Although, C4.5 DT and FDA possessed the same accuracy, MCC, precision, recall and F-measure for failed and stable pillars on the testing dataset, the training performances were different. Comparing their values of MCC, recall, precision, F-measure for failed and stable pillars, the C4.5 DT performed better than FDA in the training dataset. Therefore, based on their training and testing performances and bearing in mind the black-box nature of the SVM model, the random tree model is relatively better SVM and RBF models available in literature due to their easier application and of a clear graphical outcome. Although the proposed method produces desirable results for predictions, certain shortcomings should be discussed in the future. The dataset is relatively unbalanced and small. Generally, if the dataset is limited it will affect model generalization and reliability.