# The MADAG Strategy for Fault Location Techniques
## 4 Empirical Study
### Experimental Results and Analysis
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Program**} & \multirow{2}{*}{**Total TS**} & \multicolumn{2}{c}{**Ochiai**} & \multicolumn{2}{c}{**Tarantula**} & \multicolumn{2}{c}{**Jaccard**} \\ \cline{3-8}  & & **Test Cases** & **Ratio** & **Test Cases** & **Ratio** & **Test Cases** & **Ratio** \\ \hline printtokens & 4130 & 8.40 & 99.797\% & 6.60 & 99.840\% & 6.20 & 99.850\% \\ printtokens2 & 4115 & 11.78 & 99.714\% & 13.11 & 99.681\% & 10.89 & 99.735\% \\ schedule & 2650 & 10.80 & 99.592\% & 9.60 & 99.638\% & 9.00 & 99.660\% \\ schedule2 & 2710 & 9.22 & 99.660\% & 9.00 & 99.668\% & 3.89 & 99.856\% \\ tcas & 1608 & 4.78 & 99.703\% & 4.08 & 99.747\% & 3.38 & 99.790\% \\ totinfo & 1052 & 7.13 & 99.322\% & 9.35 & 99.111\% & 5.17 & 99.508\% \\ replace & 5542 & 17.62 & 99.682\% & 18.17 & 99.672\% & 10.55 & 99.810\% \\
**Siemens Avg.** & & **9.96** & **99.639\%** & **9.99** & **99.622\%** & **7.01** & **99.744\%** \\ \hline grep & 470 & 9.69 & 97.938\% & 10.85 & 97.692\% & 10.38 & 97.791\% \\ gzip & 214 & 12.47 & 94.174\% & 12.07 & 94.361\% & 12.20 & 94.299\% \\ flex & 525 & 13.09 & 97.508\% & 10.57 & 97.986\% & 13.85 & 97.362\% \\ sed & 360 & 13.56 & 96.235\% & 18.44 & 94.877\% & 13.56 & 97.235\% \\ space & 13,585 & 22.93 & 99.831\% & 17.62 & 99.870\% & 22.76 & 99.832\% \\
**UNIX Avg.** & & **14.35** & **97.137\%** & **13.91** & **96.957\%** & **14.55** & **97.104\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Test cases and reduction ratio on subject programs. _RQ2. Is it still efficient to apply the SBFL technique with the whole test cases?_  
In addition to considering the reduction ratio, we must also determine whether the test cases selected by the MADAG strategy are suitable for the SBFL technique. To measure the effectiveness of the selected test cases by the MADAG strategy for the SBFL technique, we apply whole test cases (called original test) to Ochiai, Tarantula, and Jaccard to calculate the cost of fault localization for comparison with the test cases selected by the MADAG strategy (called MADAG test). The experimental results shown in Figures 6a, 7a and 8a are the results of the Ochiai, Tarantula, and Jaccard techniques respectively, for the Siemens suite, and Figures 6b, 7b and 8b are the same for the UNIX suite. In this figure, the x-axis represents the percentage of statements examined, and the y-axis represents the number of faulty versions where the fault is located by the examination of a number of statements less than or equal to the corresponding value on the x-axis. In Figure 6a, we observe that the MADAG test and original test can find 38.3% and 44.1% faulty versions, respectively, in the Siemens suite by the Ochiai when the x-axis is 5%. Obviously, the gap between the MADAG test and the original test is 5.8%. However, examining more statements until the x-axis reaches 20%; both the MADAG test and the original test have quite similar effectiveness in fault localization. The curve of the MADAG test is equal to that of the original test when examining 30% of the statements, and they overlap when examining 40% of the statements. Here, we merely apply approximately 10 test cases on average of the Siemens suite with 7 programs, whereas the effectiveness of fault localization can achieve or equal that of the original test. This proves that the test cases selected by the MADAG strategy are fruitful for spectrum-based fault localization techniques. Figure 6: (**a**) Effectiveness comparison on the Siemens suite for Ochiai; and (**b**) on the UNIX suite for Ochiai.