## 5 Evaluation
F-measure results: As shown in Table 2, our proposed SCM feature selection framework, using a DT classifier, achieves the best F-measure on five of the seven projects and the best average F-measure across all projects, compared with SCM using the other four classifiers. More specifically, SCM with DT improves the average F-measure by 0.171, 0.226, 0.447, and 0.132 compared to SCM with SVM, LR, RF, and \(k\)NN, respectively. g-mean results: As shown in Table 3, our proposed SCM feature selection framework, using a DT classifier, achieves the best g-mean on six of the seven projects and the best average g-mean across all projects, compared with SCM using the other four classifiers. More specifically, SCM with DT improves the average F-measure by 0.206, 0.230, 0.508, and 0.145 compared to SCM with SVM, LR, RF, and \(k\) NN, respectively. MCC results: As shown in Table 4, our proposed SCM feature selection framework, using a DT classifier, achieves the best MCC on five of the seven projects and the best average MCC across all projects, compared with SCM using other four classifiers. More specifically, SCM with DT improves the average F-measure by 0.123, 0.197, 0.342, and 0.118 compared to SCM with SVM, LR, RF, and \(k\) NN, respectively. AUC results: As shown in Table 5, our proposed SCM feature selection framework, using a DT classifier, achieves the best AUC on all seven projects and the best average AUC across all projects, compared to SCM with the other four classifiers. More specifically, SCM with DT improves the average F-measure by 0.111, 0.135, 0.212, 0.093 compared to SCM with SVM, LR, RF, and \(k\) NN, respectively. **Answer**: Our proposed SCM feature selection framework using a DT classifier performs markedly better than the comparative classifiers for predicting the residences of crashing faults in terms of all four performance indicators. Thus, it is reasonable to choose DT as our classification model to predict crashing fault residence for better performance. RQ2: Does the MIC-Based Feature Ranking in Our Proposed Feature Selection Framework Perform Better than Other Feature Ranking Strategies? **Methods**: The essence of the feature ranking strategy is to calculate the correlation between each feature and the class label using different criteria. For this question, we carefully selected three classic feature correlation calculation methods to combine with the same SC-based feature clustering for comparison, i.e., chi-square (CS), information gain (IG), and ReliefF (Ref). More specifically, CS is a statistics-based method, IG is a probability-based method, Ref is an instance-based method, and MIC is an information-theory-based method. For a fair comparison, we just replaced MIC with the baseline method in our hybrid feature selection framework. And the DT classifier was used as the classification model for all methods, including the three baseline methods. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Projects** & **SCM-S** & **SCM-L** & **SCM-R** & **SCM-\(k\)N** & **SCM-D** \\ \hline Codec & 0.607 & 0.564 & 0.504 & 0.661 & **0.764** \\ Collections & 0.664 & 0.609 & 0.503 & 0.689 & **0.815** \\ IO & 0.698 & 0.691 & 0.587 & 0.731 & **0.814** \\ Jsoup & 0.506 & 0.513 & 0.500 & 0.529 & **0.612** \\ JSqlParser & 0.797 & 0.813 & 0.785 & 0.786 & **0.817** \\ Mango & 0.704 & 0.636 & 0.528 & 0.671 & **0.751** \\ Ornlite-Core & 0.707 & 0.686 & 0.567 & 0.743 & **0.888** \\ \hline Average & 0.669 & 0.645 & 0.568 & 0.687 & **0.780** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The AUC values of our proposed SCM feature selection framework, using five different classifiers, on each software project. F-measure results: As shown in Table 6, our proposed SCM feature selection framework, combined with the MIC-based feature ranking strategy, achieves the best F-measure on five of seven projects and the best average F-measure across all projects, compared with SCM combined with the other three feature ranking strategies.