# Research on Hard Rock Pillar Stability Prediction Based on SABO-LSSVM Model
## 4 Results and Discussion
### Performance of Different Balancing Methods
Figure 11: Prediction results of the SABO-LSSVM model (M-3 dataset) under different sampling methods: (**a**) Tomek–Smote, (**b**) Smote, (**c**) Adasyn, and (**d**) Default. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & \multicolumn{3}{c}{**Predicted Value**} \\  & & **Positive** & **Negative** & \\ \hline \multirow{2}{*}{True value} & Positive & TP & FN & TPR \\  & Negative & FP & TN & TNR \\ \hline \multirow{2}{*}{True value} & PPV & FPR & ACC \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Confusion matrix indicator meaning. The F1 score is commonly used for binary classification problems. For the multi-class problem in this study, the concepts of micro-average (Micro F1) and macro-average (Macro F1) must be introduced [45]. When calculating the F1 score, a particular class was considered the positive class, while the other samples were treated as negative samples. When calculating Micro F1, all class samples were combined, and the formula is as follows (where N represents the total number of samples):  
\[Precision_{micro}=\frac{\sum_{i=1}^{N}TP}{\sum_{i=1}^{N}TP+\sum_{i=1}^{N}FP} \tag{16}\]  
\[Recall_{micro}=\frac{\sum_{i=1}^{N}TP}{\sum_{i=1}^{N}TP+\sum_{i=1}^{N}FN} \tag{17}\]  
\[Micro~{}F1=\frac{2 Precision_{micro}*Recall_{micro}}{Precision_{micro}+Recall _{micro}} \tag{18}\]  
When calculating Macro F1, it is necessary to first calculate the precision and recall for each class and then take the arithmetic average. The formula is as follows (where L represents the number of classes):  
\[Precision_{macro}=\frac{\sum_{i=1}^{N}Precision_{i}}{|L|} \tag{19}\]  
\[Recall_{macro}=\frac{\sum_{i=1}^{N}Recall_{i}}{|L|} \tag{20}\]  
\[Macro~{}F1=\frac{2 Precision_{macro}*Recall_{macro}}{Precision_{macro}+Recall _{macro}} \tag{21}\]  
Micro F1 and Macro F1 have slightly different applications. Micro F1 tends to favor datasets with balanced distributions, whereas Macro F1 tends to emphasize small-sample classes. Generally, both evaluation metrics can be applied when the sample class distribution is relatively balanced. If the importance of small-sample events is higher in practical applications, Micro F1 can be considered. On the other hand, if the importance of large-sample events is higher, the Macro F1 metric should be considered. From Table 7, it can be observed that the SABO-LSSVM model based on the M-3 parameter combination, when using the Tomek-Smote balancing algorithm, had a higher accuracy than the LSSVM and SVM models under the same conditions, with improvements of 14.29% and 30.96%, respectively. This indicates that even after removing some noise from the oversampled samples using the Tomek Links, the SABO-LSSVM model outperformed \begin{table}
\begin{tabular}{c c c} \hline \hline
**Index** & **Formula** & **Significance** \\ \hline Accuracy (ACC) & \(ACC=\frac{TP+TN}{TP+TN+FP+TN}\) & Proportion of correctly predicted samples among all predictions \\ \hline Precision (PPV) & \(PPV=\frac{TP}{TP+TP}\) & Proportion of correctly predicted positive samples among positive predictions \\ \hline False-positive rate (FPR) & \(FRP=\frac{FP}{TP+TN}\) & Proportion of incorrectly predicted negative samples among negative predictions \\ \hline Recall (TPR) & \(TPR=\frac{TP}{TP+TN}\) & The ratio that is accurately predicted in an actual positive example \\ \hline Specificity (INR) & \(SNR=\frac{TN}{TN+FP}\) & The ratio that is accurately predicted in an actual negative example \\ \hline F1 score & \(F1~{}score=\frac{2 PPV+TPR}{PPV+TP}\) & The harmonic mean of precision and recall \\ \hline Micro F1 & \(Micro~{}F1=\frac{2 PPV_{micro}*TPR_{micro}}{\frac{TP+TP+TN}{TP+TN}}\) & Calculate F1 score using precision and recall based on the overall sample set without distinguishing between classes \\ \hline Macro F1 & \(Macro~{}F1=\frac{2 PPV_{micro}*TPR_{micro}}{PPV_{micro}+TPR_{macro}}\) & For class distinction, calculate average precision and recall for each class separately to compute F1 score \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Formulas and significance of model performance evaluation metrics.