## 4 Simulation Results
### Results on the Benchmark Test-Suite
Note that even though 30 runs were executed in each case, the sample sizes are not always 30 (because not all runs were successful in all cases); for instance, for the Goldstein-Price function (executed with parameters PopSize = 15 and Gens = 5000), \(n_{1}=n_{2}=30\) for the mean best-of-run fitness calculation, whereas \(n_{1}=5\) and \(n_{2}=6\) for the mean FirstHitEvals computation. (To avoid division by zero, we cannot use the above formulas when both \(s_{1}\) and \(s_{2}\) are zeros or when any one of \(n_{1}\) and \(n_{2}\) is unity.) Using \(\alpha=0.05\) as the level of significance, we see from the results in Table 4 that on the best-of-run metric, out of a total of 19 cases, ten cases produce a positive \(t\) statistic that corresponds to a one-tailed \(p\)-value less than \(\alpha\) (the \(p\)-values were obtained with \(t\)-tests from scipy.stats). Thus the null hypothesis \(\bar{x}_{1}=\bar{x}_{2}\) must be rejected in favor of \(\bar{x}_{1}>\bar{x}_{2}\) for those ten cases. The 19 cases include a lone negative \(t\) score, but the corresponding \(p\)-value is greater than 0.05. On the FirstHitEvals metric, we have a total of 19 cases (the two occurrences of 19 between best-of-run and FirstHitEvals is a coincidence), of which fourteen have a positive \(t\) with a \(p\)-value less than 0.05, and a single case has a negative \(t\)-score with a less-than-0.05 \(p\)-value. The statistical tests in Table 4 provide performance comparison separately on each of the twelve functions (using two different algorithm parameter settings for each function). A measure of the combined performance on the 12 functions taken together can be obtained using a paired-sample [PERSON] signed rank test on the 12-function suite. The results of this test for each of the two metrics are presented in Table 5 where the null hypothesis is that the Jaya mean and the SJaya mean are identical and the alternate hypothesis is that the former is larger than the latter. The second column in Table 5 shows the number of zero differences between SJaya and Jaya; \(n\) represents the effective number of samples obtained by ignoring the samples, if any, corresponding to zero differences (e.g., \(n\) is \(24-5=19\) for the mean of best-of-run fitness metric); \(W\) is the test statistic obtained as the minimum of \(W+\) and \(W-;\alpha\) represents the level of significance (a value of 0.05 is used here); and the critical \(W\) for a given \(n\) and for \(\alpha=0.05\) is obtained from standard statistical tables. The \(W\) statistic is seen to be less than the critical \(W\). The mean of \(W\) is  
\[\text{mean}=\frac{n(n+1)}{4},\]  
and its standard deviation is given by  
\[\text{std dev}=\sqrt{\frac{n(n+1)(2n+1)}{24}},\]and, arguing that the sample size \(n\) is large enough for the discrete distribution of the \(W\) statistic to be approximated by a normal distribution, we obtain the \(z\)-statistic as  
\[z=\frac{W-\text{mean}}{\text{std dev}}.\]  
The one-tailed \(p\)-value corresponding to the above \(z\)-statistic is obtained from standard tables of the normal distribution. From the results in Tables 4 and 5 we conclude that at the 5% significance level, Slyaya is better than Jaya on the benchmark test-set.