# Credibility Based Imbalance Boosting Method for Software Defect Proneness Prediction
## 5 Experimental Results
### RQ2: How Is the Generalization Ability of CIB?  
According to the experimental results on NASA datasets, we find that CIB is more promising to deal with the class-imbalance problem in SDP when compared with existing methods. Here, we want to investigate that whether the findings can be generalized to other scenarios, e.g., on other defect datasets. To this end, we compared CIB with the existing methods (see Section 4.2) on PROMISE datasets [43]. The PROMISE repository includes many defect datasets collected from open-source JAVA projects, such as camel, ant, etc. The details of nine PROMISE datasets are shown in Table 12. We take the same settings as Section 4.5. Due to the space limitation, we just report the experimental results of \(F_{1}\), AGF, AUC, and MCC with box plots as shown in the Figure 3.  
From the figures, we can notice that CIB still outperforms all baselines in terms of \(F_{1}\), AGF, AUC, or MCC. Therefore, we can conclude that the findings have a good generalization.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Dataset** & **\# of Metrics** & **\# of Instances** & **\# of Defective Instances** & **Defect Ratio** \\ \hline ant-1.7 & 20 & 745 & 166 & 0.2228 \\ camel-1.6 & 20 & 965 & 188 & 0.1948 \\ ivy-2.0 & 20 & 352 & 40 & 0.1136 \\ jedit-4.1 & 20 & 312 & 79 & 0.2532 \\ log4j-1.0 & 20 & 135 & 34 & 0.2519 \\ prop-6 & 20 & 660 & 66 & 0.1 \\ tomcat & 20 & 858 & 77 & 0.0897 \\ xalan-2.6 & 20 & 885 & 411 & 0.4644 \\ xerces-1.3 & 20 & 453 & 69 & 0.1523 \\ \hline \hline \end{tabular}
\end{table}
Table 12: Statistics of PROMISE datasets used in this study.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Data** & **Our CIB** & \
\begin{tabular}{c} **MAHAKIL** \\ [44] \\ \end{tabular} & **AdaC2**[45] &
\begin{tabular}{c} **AdaBoost** \\ [46] \\ \end{tabular} & **SMOTE** & **RUS**[35] & **None** \\ \hline CM1 & [71.9, 73.5] & [58.0, 61.2] & [72.3, 74.5] & [73.1, 75.5] & [59.4, 63.2] & [61.4, 64.4] & [53.1, 56.9] \\ KC1 & [67.4, 68.2] & [58.5, 59.7] & [67.8, 68.4] & [67.0, 68.0] & [59.7, 60.7] & [59.5, 60.7] & [57.9, 59.3] \\ KC3 & [70.8, 73.6] & [59.1, 63.3] & [68.2, 70.8] & [68.1, 70.5] & [59.9, 63.5] & [61.2, 65.6] & [57.0, 60.8] \\ MC1 & [82.4, 84.2] & [68.0, 71.2] & [84.3, 87.1] & [84.4, 86.8] & [64.0, 67.6] & [68.9, 71.9] & [58.2, 62.0] \\ MC2 & [74.5, 76.5] & [64.8, 68.0] & [72.5, 75.3] & [73.4, 76.2] & [60.1, 62.9] & [59.7, 63.9] & [59.4, 63.4] \\ MW1 & [68.0, 70.2] & [55.7, 60.5] & [68.1, 70.9] & [68.8, 70.6] & [60.9, 63.9] & [58.6, 62.2] & [52.1, 56.9] \\ PC1 & [85.3, 86.5] & [68.0, 71.6] & [83.3, 84.7] & [83.5, 85.1] & [67.4, 70.2] & [72.2, 74.8] & [64.1, 68.1] \\ PC2 & [77.4, 81.4] & [60.1, 67.3] & [70.6, 74.2] & [70.6, 74.2] & [52.4, 59.6] & [70.8, 74.2] & [49.9, 51.5] \\ PC3 & [80.2, 81.0] & [60.4, 63.4] & [79.1, 80.1] & [78.5, 79.5] & [60.7, 62.9] & [68.6, 70.8] & [62.0, 64.6] \\ PC4 & [92.1, 92.5] & [71.9, 73.9] & [92.4, 92.8] & [92.2, 92.6] & [74.4, 76.0] & [81.3, 82.5] & [73.3, 76.1] \\ JM1 & [67.1, 67.5] & [60.7, 61.5] & [66.4, 66.8] & [66.2, 66.8] & [61.9, 62.7] & [61.5, 62.1] & [60.8, 62.0] \\ \hline \hline \end{tabular}
\end{table}
Table 11: 95% confidence interval of AUC for CIB and the baselines on 11 NASA datasets.