# Causally Remove Negative Confound Effects of Size Metric for Software Defect Prediction
## 4 Experiments and Results
### Presence of Confounding Effects  
For each data set, we applied the proposed method introduced in Section 3.2. Then we count the number of metrics affected by the positive effects and the number of metrics affected by the positive effect. As shown in Table 2, we can know that negatively affected metrics are significantly less than the positively affected metrics in all data sets. This is the reason why removing all the confounding effects without distinguishing the direction does not necessarily contribute to defect prediction. The numbers in the number row of Table 2 correspond to the numbers of metric names listed in Table 3. These names are the metric names that are negatively confounded by the size metric in each data set. We verified our method's ability to remove confounding effects on the metrics listed in Table 3. The odds-ratio-based method was applied to analyze the extent of confounding effects, and the greater value obtained meant the metric suffered a greater extent of confounding effect. Table 4 lists the results before and after applying CRNCEM. The indexes in Table 4 correspond to that of Table 3, and the bold value means that the metric suffers a lesser extent of confounding effect. The results in Table 4 show that our method effectively removes the confounding effects on most metrics (47 from 50). Figure 2 shows this result more intuitively. Our method performed badly only on the numbers 7, 16, and 17 metrics. In the machine learning field, the p-value represents the significance of the variable. A smaller p-value means more statistical evidence for the higher significance of the variable and a stronger correlation between the independent and dependent variables. Usually, the p-value threshold is set to 0.05. The Table 5 lists the p-values (metrics listed in Table 3 against defect-proneness by LR) before and after applying CRNCEM. Our method effectively reduces the metrics' p-value, even under the threshold of 0.05. This indicates that the CRNCEM enhances the ability of metrics to predict defect-proneness. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Dataset** & **Instances** & **Metrics** & **Defect** & **Nondefect** & **Defect Rate** \\ \hline CM1 & 344 & 37 & 42 & 302 & 12\% \\ KC1 & 2096 & 21 & 325 & 1771 & 16\% \\ KC3 & 200 & 39 & 36 & 164 & 18\% \\ MC1 & 9277 & 38 & 68 & 9209 & 1\% \\ MC2 & 127 & 39 & 44 & 83 & 35\% \\ MW1 & 264 & 37 & 27 & 237 & 10\% \\ PC3 & 1125 & 37 & 140 & 985 & 12\% \\ PC4 & 1399 & 37 & 178 & 1221 & 13\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Data sets from metric data program (MDP). Figure 2: Results of analysis of confounding effects. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**No.** & **Before** & **After** & **No.** & **Before** & **After** \\ \hline
1 & 1.10484079 & **0.28503274** & 26 & 0.39090061 & **0.29508718** \\
2 & 0.14999267 & **0.10896084** & 27 & 1.25104075 & **0.57190734** \\
3 & 1.21986822 & **0.17888081** & 28 & 1.37456997 & **0.75384506** \\
4 & 1.52170872 & **0.26126476** & 29 & 1.22532643 & **0.51697049** \\
5 & 1.417992 & **0.04117915** & 30 & 1.34782366 & **0.74316556** \\
6 & 1.30598938 & **0.18745917** & 31 & 0.21131936 & **0.0981697** \\
7 & **1.02329347** & 1.39972728 & 32 & 0.54911514 & **0.09985847** \\
8 & 1.82762755 & **0.06118248** & 33 & 1.20388571 & **0.51786812** \\
9 & 1.88311737 & **0.12962644** & 34 & 1.45755614 & **0.74346193** \\
10 & 1.01205966 & **0.06075898** & 35 & 0.75490994 & **0.18276533** \\
11 & 0.26350997 & **0.15513292** & 36 & 0.16204335 & **0.046405** \\
12 & 1.2228172 & **0.63065639** & 37 & 1.26829314 & **0.52093941** \\
13 & 1.09302779 & **0.49116623** & 38 & 1.41444823 & **0.78327196** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results of analyzing the extent of confounding effects.