# Analyzing the Performance of the Multiple-Searching Genetic Algorithm to Generate Test Cases
## 7 Performance Comparison  
To evaluate the performance and robustness of the MSGA, the solution of the MSGA was compared with those of a traditional GA and an HGA in terms of execution time, number of test cases, and percentage coverage. To the stability of the MSGA compared to that of the GA and HGA, the Vargha-Delaney \(\hat{A}_{12}\) effect size is calculated. Furthermore, each algorithm was reported with a 95% confidence interval of number of test cases achieved. The execution time includes the whole process for generating test cases, running test cases, and coverage acquisition. The percentage coverage was obtained from the number of executed branch statements. The parameters were defined according to the findings of the parametric analysis (Tables 3-8). All three algorithms were assigned the same parameters (Table 9). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Mutation Probability**} & \multicolumn{2}{c}{**Best Value**} & \multicolumn{2}{c}{**Average Value**} & \multicolumn{2}{c}{**Worst Value**} \\ \cline{2-5}  & **Time (s)** & **No. of Generations** & **Time (s)** & **No. of Generations** & **Time (s)** & **No. of Generations** \\ \hline
1/\(n\) & 50.5 & 2 & 97.25 & 5.71 & 136.40 & 9 \\
0.1 & 50.8 & 2 & 94.99 & 5.60 & 157.40 & 11 \\
0.2 & 64.9 & 3 & 100.36 & 6.43 & 141.50 & 11 \\
0.3 & 62.0 & 3 & 107.96 & 6.91 & 152.60 & 11 \\
0.4 & 62.9 & 6 & 111.17 & 7.15 & 173.50 & 15 \\
0.5 & 50.3 & 2 & 110.94 & 7.17 & 157.90 & 13 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Effect of the mutation operator for test case generation using the MSGA. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Pair of Values** & **\(t\)-Test** & **Mann-Whitney (\(p\)-Value)** \\ \hline
0.1 vs. 1/\(n\) & 0.316314 & 0.2776 \\
0.1 vs. 0.2 & \textless{}0.00001 & \textless{}0.00001 \\
0.1 vs. 0.3 & \textless{}0.00001 & \textless{}0.00001 \\
0.1 vs. 0.4 & \textless{}0.00001 & \textless{}0.00001 \\
0.1 vs. 0.5 & \textless{}0.00001 & \textless{}0.00001 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Statistical comparison between the 0.1 mutation probability and the others. \begin{table}
\begin{tabular}{c} After 100 generations or \\ coverage \(>85\%\) \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Parameter settings for performance comparison. Note: \(M_{p}\) and \(M_{e}\) indicate mutation probability of conservative and explorer chromosomes, respectively. The mutation operation of the GA and the HGA were the same as the mutation of conservative chromosomes. The processes of all three algorithms stopped after the maximum number of generations (100) was reached or the percentage coverage of the branch statements exceeded 85%. It is impossible to achieve 100% coverage. The maximum coverage level possible is 5/6 or 83% [42]. Thus, 85% coverage is enough to the stopping criterion. All seven programs in the SUT were independently executed 100 times for each algorithm. The MSGA outperformed the GA and the HGA in the case of test case generation, except for schedule.c and schedule2.c (Table 10). The percentage of coverage is the most important performance indicator because it indicates that the test cases generated with each algorithm can execute an equal number of branch statements. The test cases of the MSGA can execute more branch statements than all seven programs with a coverage of over 90%. For four out of seven programs, the MSGA yielded fewer generations, fewer test cases, and a higher percentage of coverage than both the GA and the HGA (Table 10). For the program tcas.c, the MSGA and the HGA had an equal percentage coverage, but the MSGA had a lower number of generations and number of test cases. The programs schedule.c and schedule2.c demonstrated similar performance among the three algorithm types, generating the same number of generations and test cases and very similar percentages of coverage (Table 10). The MSGA took less time to achieve coverage than the GA and the HGA for all seven programs (Table 11).