## 3 Numerical Experiment
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Function** & **Metric** & **FSS** & **FSSN** & **FSSGD** \\ \hline \multirow{3}{*}{\(f_{1}\)} & Mean & **0.85** & 0.86 & **0.85** \\  & Min & **0.83** & 0.85 & 0.84 \\  & Std. dev & 0.01 & 0.01 & 0.01 \\ \hline \multirow{3}{*}{\(f_{2}\)} & Mean & **1.14** & 1.16 & 1.15 \\  & Min & **1.13** & 1.14 & 1.14 \\  & Std. dev & 0.01 & 0.01 & 0.01 \\ \hline \multirow{3}{*}{\(f_{3}\)} & Mean & **1.16** & 1.23 & 1.17 \\  & Min & 1.10 & 1.19 & **1.02** \\  & Std. dev & 0.04 & 0.03 & 0.05 \\ \hline \multirow{3}{*}{\(f_{4}\)} & Mean & **0.93** & 0.95 & **0.93** \\  & Min & **0.90** & 0.94 & 0.91 \\  & Std. dev & 0.01 & 0.01 & 0.01 \\ \hline \multirow{3}{*}{\(f_{9}\)} & Mean & **1.59** & 1.66 & 1.62 \\  & Min & **0.97** & 0.98 & 0.99 \\ \cline{1-1}  & Std. dev & 0.31 & 0.32 & 0.30 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Time benchmarks of 5-dimensional test functions listed in Table 1.  
Figure 2: Hybrid algorithms optimizing test functions from Table 1.  
From Tables 4 and 5 we can conclude that the hybrid algorithms usually run a bit longer than the evolutionary-inspired algorithm without modifications. However, according to Tables 2 and 3, in most cases the hybrids find solutions that are more accurate. The FSSGD algorithm is generally faster than the FSSN algorithm. The cause of this is that the hybrid algorithm based on [PERSON]'s method computes the Hessian matrix on each iteration, using finite difference approximation, and such computations can be expensive, especially in high-dimensional space. To verify, that the hybrid algorithms outperform the original algorithm, we used the Wilcoxon signed-ranks statistical test, often used to determine if significant statistical differences exist between two algorithms in computational intelligence. The results are shown in Table 6.  
In our case, we ensured that the significant differences exist between FSS and FSSN, and between FSS and FSSGD. We assumed the level of significance \(\alpha=5\times 10^{-2}\), which means the 95% confidence level. The null hypothesis \(H_{0}\) is that the two related paired samples come from the same distribution. For each test function \(f_{i}\) listed in Table 1, we ran the original algorithm and hybrids 20 times. Then, we applied the Wilcoxon signed-ranks statistical test to the obtained distributions of solutions. Based on the results shown in Table 6, we observed, that both of the hybrid algorithms can improve the accuracy of the original population-based algorithm, and in most cases, a statistically significant difference exists between the distributions the considered algorithms produce. The FSSN hybrid, which is based on [PERSON]'s optimization method, outperforms the FSSGD hybrid based on gradient ascent, however in some cases [PERSON]'s method converges to a stationary point or is not applicable to a function due to the singularity of its Hessian matrix in the given point. The FSSGD \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Function** & \(\mathbf{W_{1}}\) & \(\mathbf{W_{2}}\) & \(\mathbf{W_{3}}\) \\ \hline \(f_{1}\) & \(+\) & \(+\) & \(+\) \\ \(f_{2}\) & \(+\) & \(+\) & \(=\) \\ \(f_{3}\) & \(+\) & \(+\) & \(+\) \\ \(f_{4}\) & \(+\) & \(+\) & \(+\) \\ \(f_{5}\) & \(+\) & \(+\) & \(+\) \\ \(f_{6}\) & \(+\) & \(+\) & \(+\) \\ \(f_{7}\) & \(=\) & \(+\) & \(-\) \\ \(f_{8}\) & \(+\) & \(+\) & \(+\) \\ \(f_{9}\) & \(=\) & \(+\) & \(-\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Wilcoxon signed-rank test applied to the original and hybrid algorithms accuracy measurements. The sign of \(W_{1}\) denotes whether FSSN algorithm outperforms Fish School Search (FSS), the sign of \(W_{2}\) denotes whether FSSGD outperforms FSS, \(W_{3}\) sign denotes whether FSSN outperforms FSSGD. The equality sign indicates no significant difference.