## 4 The Experiments and Results
### Experimental Results
#### 4.5.3 Failure Detection Efficiency  
A set of faults is seeded into each tested program to investigate the \"ProbBetter\" [16] relationship between the two testing criteria. Each program is executed using the adequate test suites of the second-order mutation and all du-pairs criteria. Then, the failure detection efficiency of each test suite is estimated. The number of faults seeded into each program is given in the second column of Table 9, while the third and fourth columns give the failure \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Tested Program** & **Suite1** & **Suite2** & **Suite3** & **Average** \\ \hline Triangle & 98\% & 100\% & 98\% & 98.7\% \\ \hline Sort & 96\% & 96\% & 96\% & 95.6\% \\ \hline Stack & 89\% & 100\% & 89\% & 92.6\% \\ \hline Reversed & 100\% & 100\% & 100\% & 100\% \\ \hline Clock & 100\% & 100\% & 100\% & 100\% \\ \hline Quotient & 100\% & 100\% & 100\% & 100\% \\ \hline Product & 100\% & 100\% & 100\% & 100\% \\ \hline Area & 100\% & 100\% & 100\% & 100\% \\ \hline Dist & 100\% & 100\% & 100\% & 100\% \\ \hline Middle & 100\% & 100\% & 100\% & 100\% \\ \hline Average & 98.3\% & 99.6\% & 98.3\% & 98.7\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Second-order mutant coverage ratio using all du-pairs cover. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Tested Program** & **Suite1** & **Suite2** & **Suite3** & **Average** \\ \hline Triangle & 80\% & 82\% & 80\% & 81\% \\ \hline Sort & 85\% & 85\% & 85\% & 85\% \\ \hline Stack & 89\% & 85\% & 85\% & 86\% \\ \hline Reversed & 75\% & 80\% & 75\% & 77\% \\ \hline Clock & 80\% & 80\% & 80\% & 80\% \\ \hline Quotient & 70\% & 73\% & 70\% & 71\% \\ \hline Product & 78\% & 80\% & 75\% & 78\% \\ \hline Area & 80\% & 83\% & 88\% & 84\% \\ \hline Dist & 79\% & 90\% & 79\% & 83\% \\ \hline Middle & 85\% & 85\% & 85\% & 85\% \\ \hline Average & 80.1\% & 82.3\% & 80.2\% & 80.9\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: All du-pairs coverage ratio using second-order mutant cover. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Second-Order Mutant** & **Du-Pairs** \\ \hline Mean & 98.7\% & 80.9\% \\ \hline Variance & 0.1\% & 0.2\% \\ \hline P(T \(\leq\) t) two-tail & \(1.11\times 10^{-5}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 8: \(t\)-Test: paired two sample for means of coverage ratio. detection ratio for the test suite of the two criteria. The total number of faults seeded into the tested programs is 10,565 faults (1056 faults on average, 2459 faults on maximum, and 109 on minimum). The failure detection efficiency of the second-order mutation is 98% (on average), while the failure detection efficiency of the all du-pairs is 86% (on average). In addition, the second-order mutation criterion detects all the seeded faults in 7 out of 10 tested programs, while the all du-pairs criterion failed to detect 100% of faults in any tested program. Figure 4 shows the failure detection efficiency of the two criteria. Table 10 shows the statistical comparison between the obtained failure detection efficiency and their significant difference. According to the results of the \(t\)-Test, the \(p\)-value (9.195 \(\times\) 10\({}^{-5}\)) is less than 0.05; therefore, the failure detection efficiency of the mutation (98%) is significantly better than the failure detection efficiency of data flow (86%). Therefore, one can accept H3 and reject H4. Consequently, second-order mutation testing is \"ProbBetter\" than the all du-pairs data flow testing. Figure 4: Failure detection efficiency.