## 5 Evaluation  
In this section, we aim to address the aforementioned two research questions. In RQ1, we examine whether our proposed SCM feature selection framework, combined with a decision tree classifier, outperforms four other widely employed classifiers. In RQ2, we investigate whether incorporating the MIC-based feature ranking strategy within the SCM framework yields better performance than three other commonly used ranking strategies. RQ1: Does Our Proposed SMC Feature Selection Framework Using a Decision Tree Classifier Achieve Better Prediction Performance than It Does with Other Classifiers? **Methods**: We carefully selected four classifiers that are widely employed in the tasks of SE for comparison, including the support vector machine (SVM) classifier [30], logistic regression (LR) classifier [31], random forest (RF) classifier [32], and k-nearest neighbor (\(k\)NN) classifier [33]. More specifically, SVM is a generalized linear classification model, LR is a statistic-based classification model, RF is an ensemble-based classification model, \(k\)NN is a sample-based classification model. In our study, DT is classified as an information theory-based classification model. In other words, these classifiers come from different families. In addition, these classifiers have been widely used in previous studies [9; 34]. Before applying our SCM framework to different classifiers, the data was standardized using the widely adopted z-score technique [5]. **Results**: Tables 2-5 report the results of F-measure, g-mean, MCC, and AUC values, respectively, for our proposed SCM feature selection framework using five different classifiers. These five classifiers are abbreviated as S, L, R, \(k\)N, and D in the tables. In the four tables, we discuss the following findings: \begin{table}
\begin{tabular}{c c c c c c} \hline
**Projects** & **SCM-S** & **SCM-L** & **SCM-R** & **SCM-\(k\)N** & **SCM-D** \\ \hline Codec & 0.292 & 0.179 & 0.029 & 0.342 & **0.529** \\ Collections & 0.461 & 0.318 & 0.040 & 0.459 & **0.638** \\ IO & 0.490 & 0.468 & 0.353 & 0.527 & **0.651** \\ Jsoup & 0.047 & 0.048 & 0.002 & 0.112 & **0.231** \\ JSqlParser & 0.728 & 0.706 & **0.733** & 0.691 & 0.663 \\ Mango & **0.595** & 0.420 & 0.142 & 0.506 & 0.479 \\ Ormlite-Core & 0.512 & 0.462 & 0.290 & 0.519 & **0.789** \\ \hline Average & 0.446 & 0.372 & 0.227 & 0.451 & **0.569** \\ \hline \end{tabular}
\end{table} 
Table 4: The MCC values of our proposed SCM feature selection framework, using five different classifiers, for each software project. \begin{table}
\begin{tabular}{c c c c c} \hline
**Projects** & **SCM-S** & **SCM-L** & **SCM-R** & **SCM-\(k\)N** & **SCM-D** \\ \hline Codec & 0.380 & 0.287 & 0.018 & 0.511 & **0.663** \\ Collections & 0.485 & 0.362 & 0.013 & 0.526 & **0.707** \\ IO & 0.547 & 0.535 & 0.294 & 0.602 & **0.721** \\ Jsoup & 0.027 & 0.087 & 0.001 & 0.142 & **0.382** \\ JSqlParser & **0.728** & 0.722 & 0.720 & 0.696 & 0.687 \\ Mango & **0.560** & 0.393 & 0.098 & 0.478 & 0.503 \\ Ormlite-Core & 0.575 & 0.536 & 0.230 & 0.624 & **0.840** \\ \hline Average & 0.472 & 0.417 & 0.196 & 0.511 & **0.643** \\ \hline \end{tabular}
\end{table} 
Table 2: The F-measure values of our proposed SCM feature selection framework, using five different classifiers, for each software project. \begin{table}
\begin{tabular}{c c c c c} \hline
**Projects** & **SCM-S** & **SCM-L** & **SCM-R** & **SCM-\(k\)N** & **SCM-D** \\ \hline Codec & 0.503 & 0.426 & 0.050 & 0.631 & **0.755** \\ Collections & 0.585 & 0.493 & 0.045 & 0.636 & **0.803** \\ IO & 0.644 & 0.638 & 0.413 & 0.698 & **0.803** \\ Jsoup & 0.078 & 0.205 & 0.004 & 0.274 & **0.563** \\ JSqlParser & 0.771 & **0.793** & 0.753 & 0.757 & 0.786 \\ Mango & 0.634 & 0.518 & 0.147 & 0.582 & **0.716** \\ Ormlite-Core & 0.659 & 0.632 & 0.348 & 0.722 & **0.886** \\ \hline Average & 0.553 & 0.529 & 0.251 & 0.614 & **0.759** \\ \hline \end{tabular}
\end{table} 
Table 3: The g-mean values of our proposed SCM feature selection framework, using five different classifiers, for each software project.