## 3 Results
### Results of Computational Experiments
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{2}{c}{**Setting 1**} & \multicolumn{2}{c}{**Setting 2**} \\  & \(E_{BBC}\) & \(E_{O}\) & \(E_{BBC}\) & \(E_{O}\) \\ \hline mQSO\({}_{29}\) & 9.00 \(\pm\) 1.64 & 12.99 \(\pm\) 1.76 & 12.66 \(\pm\) 1.90 & 18.85 \(\pm\) 2.19 \\ mQSODE\({}_{21,0.1,p}\) & 8.50 \(\pm\) 1.26 & 12.18 \(\pm\) 1.39 & 12.35 \(\pm\) 1.21 & 18.05 \(\pm\) 1.28 \\ FTmPSO & 7.11 \(\pm\) 0.26 & 12.70 \(\pm\) 0.34 & 8.70 \(\pm\) 0.26 & 16.06 \(\pm\) 0.35 \\ mCMA-ES & **5.11 \(\pm\) 0.22** & **7.59 \(\pm\) 0.24** & **7.39 \(\pm\) 0.26** & **11.65 \(\pm\) 0.28** \\ mPSO & 8.51 \(\pm\) 0.22 & 12.88 \(\pm\) 0.27 & 11.63 \(\pm\) 0.19 & 18.52 \(\pm\) 0.26 \\ \hline \multirow{2}{*}{Algorithm} & \multicolumn{2}{c}{Setting 3} & \multicolumn{2}{c}{Setting 4} \\  & \(E_{BBC}\) & \(E_{O}\) & \(E_{BBC}\) & \(E_{O}\) \\ \hline mQSO\({}_{29}\) & 9.26 \(\pm\) 1.13 & 12.89 \(\pm\) 1.29 & 13.20 \(\pm\) 1.45 & 18.41 \(\pm\) 2.06 \\ mQSODE\({}_{21,0.1,p}\) & 9.09 \(\pm\) 1.06 & 12.48 \(\pm\) 1.10 & 13.48 \(\pm\) 1.39 & 17.75 \(\pm\) 1.65 \\ FTmPSO & 7.99 \(\pm\) 0.16 & 13.48 \(\pm\) 0.21 & 11.57 \(\pm\) 0.32 & 18.04 \(\pm\) 0.39 \\ mCMA-ES & **6.14 \(\pm\) 0.15** & **8.49 \(\pm\) 0.16** & **8.56 \(\pm\) 0.31** & **11.06 \(\pm\) 0.33** \\ mPSO & 9.58 \(\pm\) 0.17 & 13.86 \(\pm\) 0.21 & 12.68 \(\pm\) 0.27 & 17.67 \(\pm\) 0.34 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Comparison of mQSO and mQSODE with alternative approaches from [10]. Figure 3: Offline error graphs of three tested algorithms with two benchmark cases: 2000 and 5000 function evaluations between changes. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Measure** & **Setting 1** & **Setting 2** & **Setting 3** & **Setting 4** \\ \hline \multicolumn{5}{c}{mQSO\({}_{21}\) vs. mQSODE\({}_{21,0.1,p}\)} \\ \hline \(E_{O}\) & 0 (1.485) & 0 (0.993) & 1 (3.062) & 0 (\(-\)0.021) \\ \(E_{BBC}\) & 0 (1.696) & 0 (1.528) & 1 (3.118) & 0 (1.035) \\ \hline \multicolumn{5}{c}{mQSO\({}_{29}\) vs. mQSODE\({}_{21,0.1,p}\)} \\ \hline \(E_{O}\) & 0 (1.077) & 0 (0.598) & 0 (0.683) & 0 (\(-\)0.978) \\ \(E_{BBC}\) & 0 (1.936) & 0 (1.668) & 0 (1.457) & 0 (1.316) \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Pairwise Mann-Whitney tests of mQSO and mQSODE, all settings, test results, and standard scores. Analyzing Figures 2 and 3 it can be noted that mQSODE\({}_{21,0.1,p}\) is more conservative at the beginning, i.e., during the first several environmental changes, while mQSO\({}_{29}\) achieves better values. However, at the final environmental changes both current and offline errors of the modified algorithm are better, which indicates that it makes a better job tracking the local optima. To analyze the reasons for increased performance in the long term, the diversity measure was used for three algorithms, namely mQSO\({}_{21}\), mQSO\({}_{29}\), and mQSODE\({}_{21,0.1,p}\). The diversity was measured at every iteration as follows:  
\[DM=\frac{1}{N_{s}}\sum_{i=1}^{N_{p}}\sum_{j=i}^{N_{p}}\sum_{k=1}^{D}\left(x_{ nik}-x_{njk}\right)^{2}\]  
In other words, the \(DM\) was calculated as the sum of pairwise distances between all particles in each swarm and averaged over the number of swarms. Figure 4 shows the diversity measures for one of the runs with setting 1 of the three mentioned algorithms. The graphs in Figure 4 show that given the same population size, the algorithm without a DE search operator has significantly smaller diversity, up to several times smaller. When comparing mQSODE\({}_{21,0.1,p}\) with mQSO\({}_{29}\), which has a larger population, the diversity of the mQSO after an environmental change is usually larger, however, before the change when the swarms are converged, the diversity measures of these algorithms are comparable, which explains their similar performance. To compare the efficiency of all the tested modifications, the ranking procedure from the Friedman statistical test was applied, which will further be referred to as the Friedman ranking.