# Research on Hard Rock Pillar Stability Prediction Based on SABO-LSSVM Model
## 4 Results and Discussion
### Performance of Different Balancing Methods
the latter two in terms of predictive performance. Additionally, after optimizing the LSSVM model with the Tomek algorithm, the gap in accuracy between the LSSVM and SABO-LSSVM models was slightly reduced. Furthermore, the SABO-LSSVM model had higher Micro F1 and Macro F1 values, compared to the LSSVM and SVM models, respectively. Among the three categories of pillar samples, the \"unstable\" category had a significantly better performance in terms of the F1 score when predicted using the SABO-LSSVM model, with improvements of 22.74% and 80.55% compared to the LSSVM and SVM models, respectively. Therefore, the overall performance of the SABO-optimized model was superior to that of the unoptimized models under the same conditions. To compare the effects of different sampling methods on the prediction results of the model, a horizontal comparison between balancing algorithms is required. In terms of overall prediction accuracy, the SABO-LSSVM model without data balancing achieved an accuracy of 80.95%. After balancing the dataset using the Smote and Adasyn oversampling algorithms, the accuracy of the model was improved by 7.15% and 9.53%, respectively. The Tomek-Smote group achieved an overall accuracy of 92.86%, which was 11.91% higher than that of the Default group, showing the most significant improvement. Similar to the occurrence of sample misclassifications in the previous prediction results, the F1 scores of the \"stable\" and \"failed\" pillar sample categories in the oversampling groups showed a decrease compared to the Default group. The Adasyn group showed no significant change, whereas the Smote group experienced a decrease of 3.03% and 4.21% in the F1 scores of the second and third categories, respectively. This suggests that the introduction of new samples through oversampling introduced noise that can affect the predictive performance of certain categories. The Tomek-Smote group showed improvements of 11.7%, 25.62%, and 2.99% in the F1 scores of the three categories, compared to the Default group. Therefore, when using the Tomek-Smote balancing algorithm, not only can the overall model prediction efficiency be effectively improved, but it can also mitigate the negative impact of noise information on certain category samples, comprehensively enhancing the predictive capability of the model. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Accuracy** & **Micro F1** & **Macro F1** & **Class** & **Precision** & **Recall** & **F1 Score** \\ \hline SABO-LSSVM(Tomek–Smote) & 0.9286 & 0.9286 & 0.9284 & 1 & 0.9333 & 1.0000 & 0.9655 \\  & 0.9286 & 0.9286 & 0.9284 & 2 & 1.0000 & 0.8571 & 0.9231 \\  & 0.9286 & 0.9286 & 3 & 0.8667 & 0.9286 & 0.8966 \\ \hline SABO-LSSVM(Smote) & 0.8810 & 0.7857 & 0.7708 & 1 & 0.7368 & 1.0000 & 0.8485 \\  & 0.8810 & 0.7857 & 0.7708 & 2 & 0.8750 & 0.5000 & 0.6364 \\  & 0.8800 & 0.7857 & 0.7939 & 3 & 0.8000 & 0.8571 & 0.8276 \\ \hline SABO-LSSVM(Adasyn) & 0.9048 & 0.8095 & 0.7939 & 1 & 0.7368 & 1.0000 & 0.8485 \\  & 0.8095 & 0.8095 & 0.7939 & 2 & 1.0000 & 0.5000 & 0.6667 \\  & 0.8095 & 0.8095 & 0.7939 & 2 & 1.0000 & 0.5000 & 0.6667 \\  & 0.8125 & 0.9286 & 0.8667 & 3 & 0.8125 & 0.9286 & 0.8667 \\ \hline LSSVM(Tomek–Smote) & 0.7857 & 0.8095 & 0.8040 & 1 & 0.9286 & 0.9286 & 0.9286 \\  & 0.8040 & 2 & 0.8889 & 0.5714 & 0.6957 \\  & 0.8042 & 0.9286 & 0.7879 & 3 & 0.6842 & 0.9286 & 0.7879 \\ \hline SVM(Tomek–Smote) & 0.6190 & 0.6190 & 0.5583 & 1 & 1.0000 & 0.8571 & 0.9231 \\  & 0.8042 & 0.9286 & 0.8667 & 3 & 0.3333 & 0.0714 & 0.1176 \\  & 0.8042 & 0.9286 & 0.8667 & 3 & 0.4815 & 0.9286 & 0.6341 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Calculation results of model performance evaluation metrics.