# Combining Parallel Stochastic Methods and Mixed Termination Rules in Optimization
## 2 Adopted Algorithms
### The Method Multistart
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Problems** & **1 \(\times\) 500** & **2 \(\times\) 250** & **5 \(\times\) 100** & **10 \(\times\) 50** \\ \hline RASTRIGIN & 13,657 & 6636 & 6129 & 5399 \\ ROSENBROCK8 & 9456 & 6979 & 7165 & 7049 \\ ROSENBROCK16 & 9853 & 8620 & 8110 & 8379 \\ SHEKEL5 & 10,959 & 5437 & 5684 & 5616 \\ SHEKEL7 & 10,651 & 5715 & 5739 & 5486 \\ SHEKEL10 & 1533 & 6064 & 5772 & 5516 \\ SINU4 & 10,291 & 6249 & 5426 & 5188 \\ SINU8 & 9993 & 6847 & 6080 & 5582 \\ TEST2N4 & 10,242 & 6131 & 5837 & 5480 \\ TEST2N9 & 12,998 & 7402 & 6455 & 5702 \\ TEST30N3 & 6228 & 5712 & 5723 & 5321 \\ TEST30N4 & 5540 & 5437 & 5490 & 5484 \\
**Total** & **335,768** & **217,369** & **207,518** & **199,637** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: _Cont._ \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Problems** & **1 \(\times\) 500** & **2 \(\times\) 250** & **5 \(\times\) 100** & **10 \(\times\) 50** \\ \hline BF1 & 10,839 & 6543 & 6375 & 5933 \\ BF2 & 9838 & 6317 & 5978 & 5716 \\ BRANIN & 5787 & 5003 & 5076 & 4936 \\ CAMEL & 12,204 & 5889 & 5703 & 5434 \\ CIGAR10 & 7631 & 6575 & 6582 & 6500 \\ CM4 & 16,222 & 8935 & 6837 & 6392 \\ DISCUS10 & 5997 & 5039 & 5001 & 5252 \\ EASOM & 4701 & 4791 & 4772 & 4751 \\ ELP10 & 6230 & 5363 & 5471 & 5205 \\ EXP4 & 10,277 & 5542 & 5620 & 5533 \\ EXP16 & 9577 & 5518 & 5544 & 5532 \\ GKLS250 & 13,328 & 5993 & 5205 & 5011 \\ GKLS350 & 10,131 & 7373 & 5094 & 4869 \\ GRIEWANK2 & 8504 & 8284 & 6371 & 5761 \\ GRIEWANK10 & 8535 & 7585 & 7229 & 6922 \\ POTENTIAL3 & 11,338 & 5666 & 5830 & 5695 \\ POTENTIAL5 & 12,414 & 7310 & 7126 & 6996 \\ POTENTIAL6 & 14,760 & 8450 & 7477 & 7561 \\ POTENTIAL10 & 17,729 & 11,928 & 10,431 & 9338 \\ HANSEN & 8379 & 6862 & 4883 & 4970 \\ HARTMAN3 & 10,674 & 5395 & 5219 & 5078 \\ HARTMAN6 & 10,841 & 5486 & 5649 & 5348 \\ RASTRIGIN & 13,657 & 8283 & 5850 & 5292 \\ ROSENBROCK8 & 9456 & 6751 & 7232 & 6979 \\ ROSENBROCK16 & 9853 & 7993 & 8377 & 8086 \\ SHEKEL5 & 10,959 & 5749 & 5810 & 5884 \\ SHEKEL7 & 10,651 & 5828 & 5843 & 5698 \\ SHEKEL10 & 1533 & 5863 & 6112 & 5588 \\ SINU4 & 10,291 & 6427 & 5371 & 5251 \\ SINU8 & 9993 & 7139 & 6274 & 5583 \\ TEST2N4 & 10,242 & 7516 & 5573 & 5129 \\ TEST2N9 & 12,998 & 10,137 & 6112 & 5566 \\ TEST30N3 & 6228 & 5512 & 5402 & 5485 \\ TEST30N4 & 5540 & 5749 & 5750 & 5350 \\
**Total** & **337,337** & **228,794** & **207,179** & **198,624** \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Experiments using the proposed optimization technique and proposed stopping rule. Numbers in cells denote average function calls. And, in this case, it is observed that the increase in parallel processing units significantly reduces the required number of function calls, as in the two previous termination rules. However, the proposed termination rule dramatically reduces the number of required function calls and consequently the computation time compared to previous termination rules. This reduction in fact intensifies as the number of computing nodes increases. Also, for the same set of experiments, a statistical test (Wilcoxon test) was conducted and it is graphically presented in Figure 6.

A \(p\)-value of less than 0.05 (two-tailed) was used to determine statistical significance and is indicated in bold. An additional experiment was performed to assess the capabilities of the parallel technique using the CM function for various values of the dimension. This function is defined as:

\[f(x)=\sum_{i=1}^{n}x_{i}^{2}-\frac{1}{10}\sum_{i=1}^{n}\cos(5\tau x_{i})\]

The proposed method was tested on this function for different values of the dimension \(n\) and for a variety of processing units and the results are graphically illustrated in Figure 7.

From the experimental results, it is obvious that the required number of function calls is drastically reduced with the increase in computing units from 1 to 2 or to 5. Also, the proposed method was applied on the GKLS function for different versions of the function. The dimension of the function was in the range \([2\dots 5]\).