# Combining Parallel Stochastic Methods and Mixed Termination Rules in Optimization
## 2 Adopted Algorithms
### The Method Multistart  
In contrast, the Multistart method follows a different approach than previous methods. Instead of focusing on a single initial point, it repeats the search from various initial points in the search space. This allows for a broader exploration of the search space and increases the chances of finding the optimal solution. The Multistart method is particularly useful when optimization algorithms may get trapped in local minima [73; 74; 75]. The approach of multiple starts belongs to the simplest techniques for global optimization. At the outset of the process, an initial distribution of points is made in Euclidean space. Subsequently, local optimization begins simultaneously from these points. The discovered minima are compared, and the best one is retained as the global minimum. Local optimizations rely on the Broyden Fletcher Goldfarb Shanno (BFGS) method [76]. The main steps of the multistart method are presented in Algorithm 3. ```
1.Initialization. (a) Set \(k=0\) (iteration counter). (b) Set the number of particles \(N_{p}\). (c) Set the maximum number of iterations allowed \(k\)max (d) Set the local search rate \(p_{l}\in[0,1]\). (e) Initialize randomly the positions of the



```
1. INPUT: (a) The population size \(N_{d}\geq 4\). The members of this population are also called agents. (b) The crossover probability \(\text{CR}\in[0,1]\). (c) The differential weight \(F\in[0,2]\). 2. OUTPUT: (a) The agent \(x_{\text{best}}\) with the lowest function value \(f\big{(}x_{\text{best}}\big{)}\). 3. Initialize all agents in \(S\). 4. While termination criteria are not met do (a) For\(i=1\ldots N_{d}\)do i. Select as \(x\) the agent \(i\). ii. Select randomly three agents \(a,b,c\) with the property \(a\
eq b,\ b\
eq c,\ c\
eq a\). iii. Select a random position \(R\in\{1,\ldots,n\}\) iv. Create the vector \(y=[y_{1},y_{2},\ldots,y_{n}]\) with the following procedure v. For\(j=1,\ldots,n\)do A. Set \(r_{i}\in[0,1]\) a random number. B. If\(r_{j}<\text{CR}\)or\(j=R\)then\(y_{j}=a_{j}+F\times\big{(}b_{j}-c_{j}\big{)}\)else\(y_{j}=x_{j}\). vi. If \(y\in S\) AND \(f(y)\leq f(x)\) then \(x=y\). vi. EndFor (b) EndFor
```

**Algorithm 2** The main steps of the DE method

```
1. Initialization step. (a) Set \(N_{m}\) as the total number of samples. (b) Set \((x^{*},y^{*})\) as the global minimum. Initialize \(y^{*}\) to a very large value. 2. Sampling step. (a) For\(i=1\ldots N_{m}\)Do i. Sample a point \(x_{i}\in S\) ii. \(y_{i}=\text{LS}(x_{i})\). Where LS(x) is a local search procedure. iii. If\(y_{i}\leq y^{*}\) then \(x^{*}=x_{i},y^{*}=y_{i}\) EndFor
```

**Algorithm 3** The main steps of the Multistart method

## 3 The Proposed Method

In the present work, a mixture of termination rules is proposed as a novel technique, which can be used without partitioning in any stochastic global optimization technique. Also, a new parallel global optimization technique is proposed that utilizes the new termination rule. The existing method that was presented in Section 2, employs Particle Swarm Optimization (PSO), Differential Evolution (DE), and Multistart techniques, each with its own independent termination criteria. In contrast, the proposed method that will be presented in this section integrates an innovative combination of termination rules, including Best-Fitness, Mean-Fitness, and DoubleBox, enhancing the termination process of the search. By incorporating these advanced termination strategies into a parallel algorithm, the new method achieves faster convergence and improved solution quality. This approach combines the efficiency of parallel execution with the flexibility of advanced termination criteria, representing a significant advancement over traditional methods. Inthe following subsections, the new termination rule is fully described, followed by the new optimization method. ### The New Termination Rule

The proposed termination rule is a mixture of several stochastic termination rules proposed in the recent literature.