# Model-Based Test Suite Generation Using Mutation Analysis for Fault Localization
## 4 Experimental Results
### Results
Figure 10 shows the changes in the suspiciousness of each state chart for the non-mutant cases. When the Jaccard distance was used, the suspiciousness tended to increase while the rest decreased. Overall, there were no significant differences between the performances of the Jaccard distance and lookahead entropy methods when the faults had low difficulty levels, but the performance of the Jaccard distance method improved as the fault difficulty increased. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**State Chart** & **Difficulty (Killed)** & \(\frac{\tau_{\text{I}}}{\tau_{\text{R}}}\) & \(\frac{\tau_{\text{L}}}{\tau_{\text{R}}}\) & \(\frac{\tau_{\text{I}}}{\tau_{\text{L}}}\) \\ \hline \multirow{4}{*}{Mesh} & W (1750) & 25.64 & 26.92 & \(-\)1.01 \\ \cline{2-5}  & R (878) & 24 & 21.33 & 2.19 \\ \cline{2-5}  & H (509) & 20.58 & 13.23 & 6.49 \\ \cline{2-5}  & N & 26.31 & 18.42 & 6.67 \\ \hline \multirow{4}{*}{Ring} & W (1706) & 17.64 & 17.64 & 0 \\ \cline{2-5}  & R (818) & 28.98 & 5.79 & 21.91 \\ \cline{2-5}  & H (426) & 15.94 & \(-\)5.79 & 23.07 \\ \cline{2-5}  & N & 16.67 & \(-\)8.33 & 27.27 \\ \hline \multirow{4}{*}{Star} & W (1719) & 15.29 & 14.11 & 1.03 \\ \cline{2-5}  & R (856) & 22.97 & 21.62 & 1.11 \\ \cline{2-5}  & H (461) & 33.33 & 21.21 & 10 \\ \cline{2-5}  & N & 27.39 & 20.55 & 5.68 \\ \hline \multirow{4}{*}{Tree} & W (1671) & 13.79 & 6.89 & 6.45 \\ \cline{2-5}  & R (896) & 23.28 & \(-\)2.73 & 26.76 \\ \cline{2-5}  & H (428) & 19.44 & \(-\)1.38 & 21.12 \\ \cline{2-5}  & N & 18.91 & \(-\)6.76 & 27.53 \\ \hline \multirow{4}{*}{The} & Average (W) & 18.09 & 16.39 & 1.61 \\ \cline{2-5}  & Average (R) & 24.81 & 11.50 & 12.99 \\ \hline \multirow{4}{*}{Average (H)} & Average (H) & 22.32 & 6.81 & 15.17 \\ \cline{2-5}  & Average (N) & 22.32 & 5.96 & 16.79 \\ \hline \multirow{4}{*}{Average (total)} & 21.90 & 10.17 & 11.64 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of performance between methods based on mean suspiciousness value. Figure 8: Average suspiciousness of faulty statements (resistant case). Figure 7: Average suspiciousness of faulty statements (weak case). Figures 11-14 show the suspiciousness of the transitions after completing the execution of the 10 test suites generated using each method in 16 cases. The graph shows the difference between the faulty statement and the other statements. The clearer the difference between the suspiciousness of a faulty statement and the suspiciousness of other statements is, the better the performance was. The transition numbers of transitions that contain faults are circled. In the results from the random test suites, the suspiciousness was sometimes higher in non-faulty transitions than in faulty transitions. Ten test suites were considered insufficient to perform fault localization through random test suites alone. When the Jaccard distance method was used, the suspiciousness values of non-faulty transitions were not considered. The results from the random test suites were not shown in the results from the random test suites. The results from the random test suites were not shown in the results from the random test suites. were distributed evenly. However, when the lookahead entropy method was used, the suspiciousness values of the non-faulty transitions were not even. The most extreme case of this phenomenon is shown in Figure 12a, where weak faults were inserted into the ring state chart. The suspiciousness of the faulty transition tr1 was satisfactory; however, in other transitions, there were several non-faulty transitions with suspiciousness values of 1. Furthermore, most other cases exhibited similar suspiciousness values for faulty and non-faulty transitions, or non-faulty transitions had higher suspiciousness levels. The mean values of results obtained using the Jaccard distance and the lookahead entropy did not differ significantly.