# Combining Parallel Stochastic Methods and Mixed Termination Rules in Optimization
## 2 Adopted Algorithms
### The Method Multistart
The first algorithm will be called _best--fitness_ and in this termination technique, at each iteration \(k\), the difference between the current best value \(f_{min}^{(k)}\) and the previous best value \(f_{min}^{(k+1)}\) is calculated [77], i.e., the absolute difference:

\[\left|f_{\text{min}}^{(k)}-f_{\text{min}}^{(k-1)}\right| \tag{2}\]

If this difference is zero for a series of predefined consecutive iterations \(N_{k}\), then the method terminates. In the second termination rule, which will be called _mean--fitness_, the average function value for each iteration is computed. If this value remains relatively unchanged over a certain number of consecutive iterations, it indicates that the method may not be making significant progress towards discovering a new global minimum. Therefore, termination is warranted [19]. Hence, in every iteration \(k\), we compute:

\[\delta^{(k)}=\left|\sum_{i=1}^{\text{NP}}\left|f_{i}^{(k)}\right|-\sum_{i=1}^{ \text{NP}}\left|f_{i}^{(k-1)}\right|\right| \tag{3}\]

The value \(NP\) denotes the number of particles or chromosomes that participate in the algorithm. The termination rule is defined as follows: terminate if \(\delta^{(k)}\leq\epsilon\) for a predefined number \(N_{k}\) of iterations. The third stopping rule is the so-called DoubleBox stopping rule, which was initially proposed in the work of [PERSON] [78]. In this criterion, the search process is terminated once sufficient coverage of the search space has been achieved. The estimation of the coverage is based on the asymptotic approximation of the relative proportion of points that lead to local minima. Since the exact coverage cannot be directly calculated, sampling is conducted over a larger region. The search is stopped when the variance of the sample distribution falls below a predefined threshold, which is adjusted based on the most recent discovery of a new local minimum. According to this criterion, the algorithm terminates when one of the following conditions is met:

* The iteration \(k\) count exceeds a predefined limit of iterations \(k_{max}\);
* The relative variance \(\sigma^{(k)}\) falls below half of the variance \(\sigma^{(k_{last})}\) of the last iteration \(k_{(last)}\) where a new optimal functional value was found. \[k\geq N_{k}\,or\,\sigma^{(k)}\leq\frac{\sigma^{(klast)}}{2}\] (4) This proposed termination criterion is the combination of the aforementioned similarity types. Optimization termination is achieved when any homogeneity condition is satisfied, resulting in improved speed. Hence, by using Equations (2)-(4), the following termination rule is derived: \[\left|f_{\text{min}}^{(k)}-f_{\text{min}}^{(k-1)}\right|or\] \[\delta^{(k)}=\left|\sum_{i=1}^{\text{NP}}\left|f_{i}^{(k)}\right|-\sum_{i =1}^{\text{NP}}\left|f_{i}^{(k-1)}\right|\right|or\] \[\sigma^{(iteration)}\leq\frac{\sigma^{(klast)}}{2}\,or\] \[k\geq N_{k}\]

### The Proposed Algorithm

In the scientific literature, parallelization typically involves distributing the population among parallel computing units to save computational power and time [77; 79]. In the present study, we concurrently compute the optimal solutions of three different stochastic optimization methods originating from different classification categories. The proposed algorithm is shown in Algorithm 4. ```
1. Set as \(N_{I}\) the total number of parallel processing units. 2. Set as \(N_{k}\) the total number of allowed iterations. 3. Set \(k=0\) the iteration number. 4. For\(j=1,\ldots,N_{I}\) do in parallel (a) Execute an iteration of any stochastic algorithm mentioned in Section 2. (b) Find the best element from all optimization methods and propagate it to the rest of processing units. 5. End For
6. Update\(k=k+1\)
7. Check the proposed termination rule. If the termination rule is valid, then goto step 7a else goto step 4. (a) Terminate and report the best value from all processing units.