# Automatic Test Data Generation Using the Activity Diagram and Search-Based Technique
## 5 Experimental Results  
This section presents the results of the experiments by answering the RQs. Table 4 summarizes the overall _SC_, _BC_, and _FD_ results achieved by the three techniques across the subject systems. **RQ1**: How do the tests generated by the proposed AutoTDGen perform, compared with alternative approaches, in terms of statement and branch coverage? The results presented in Table 3 for SC indicate that AutoTDGen performed slightly better than DFAAD and EvoSuite on two subjects, although EvoSuite had the best SC coverage (one hundred percent) in one subject (Coffee Maker). In terms of BC, AutoTDGen achieved greater BC than EvoSuite in only a single subject (Elevator), but it outperformed DFAAD in all subjects. On the other hand, except for the Elevator system, EvoSuite performed very well in terms of BC. The average coverage achieved by EvoSuite was higher than that of DFAAD, but AutoTDGen performed better on average, with an improvement in covered branches of 3.67 percent. Thus, on average, AutoTDGen achieved higher or nearly equal branch and statement coverage compared with both DFAAD and EvoSuite. However, as reported in Table 3, the magnitude of the differences is small. **RQ2**: What is the difference in fault detection effectiveness between the tests generated by the proposed AutoTDGen and alternative approaches? The overall results for fault detection effectiveness \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{_Subject Systems_} & \multicolumn{3}{c}{_DFAAD_} & \multicolumn{3}{c}{_EvoSuite_} & \multicolumn{3}{c}{_AutoTDGen_} \\ \cline{2-10}  & **SC** & **BC** & **FD** & **SC** & **BC** & **FD** & **SC** & **BC** & **FD** \\ \hline Cruise Control & 97.6 & 77.3 & 67.9 & 97 & **95.5** & 44 & **99** & 83.3 & **73** \\ Elevator & 96.2 & 86.4 & 69.6 & 77 & 63 & 24.3 & **96.5** & 87.4 & **85.82** \\ Coffee Maker & 98 & 90 & 84.6 & **100** & **100** & 83.8 & 99.4 & 98.7 & **88** \\ Mean & 97.3 & 84.6 & 74 & 91.25 & **86.13** & 50.7 & **98.3** & 89.8 & **82.33** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The overall statement coverage (SC), branch coverage (BC), and fault detection (FD) ratio achieved across subjects and applied techniques. are also reported in Table 3. AutoTDGen outperformed both EvoSuite and DFAAD in all experimental subjects, with a mean effectiveness difference of 31.53 percent and 8.23 percent, respectively. To provide a better view of fault detection effectiveness, Figure 5 highlights the number of detected and failed to detect faults as well as the fault detection ratio across subject systems and applied techniques. As shown in Figure 5, the best fault-detection result achieved by AutoTDGen was for Elevator, which is large and complex with 247 seeded mutants, of which AutoTDGen detected 212 mutants, compared with 60 and 172 mutants detected by EvoSuite and DFAAD, respectively. The reason for that difference is further discussed below. **RQ3**: _Is there any variation in the types of faults detected by the proposed AutoTDGen and alternative approaches_? The types of fault detected across the experimental subjects and techniques are reported in Tables 5-7. The tables list the total number of faults (mutants) seeded per mutation operator, the number of faults detected, the number of failed to detect faults, and the percentage of faults detected per mutation operator by each technique. The best results  
RQ3 investigates whether AutoTDGen is more likely to detect certain types of faults compared with the alternative approaches. The results indicate a great variation in the number and types of faults detected by the applied techniques. For instance, out of the 30 MM seeded faults for the Elevator system, 26 faults were detected by AutoTDGen and 14 faults were detected by DFAAD, but none of the MM faults seeded in the Elevator system were detected by EvoSuite. In summary, the results indicate that AutoTDGen better detected certain type of faults (e.g., MM, INM, CBM) than the two alternative approaches.