# Combining Parallel Stochastic Methods and Mixed Termination Rules in Optimization
## 2 Adopted Algorithms
### The Method Multistart
```

**Algorithm 4** The proposed overall algorithm

The proposed algorithm is graphically outlined in Figure 1. The methods that are used in each processing unit may include Particle Swarm Optimization, the Multistart method, and Differential Evolution (DE). These techniques have been adopted since they have been widely used in the relevant literature and provide the possibility to be parallelized relatively easily. Initially the algorithm distributes the three methodologies across \(N_{I}\) threads, where \(N_{I}>3\). The first thread executes the DE method, the next executes PSO, and the third runs the Multistart method. As shown in Figure 1, at each iteration, every computational unit that uses this specific optimization method calculates the best solution and replaces the worst solutions of the other computational units. This process is repeated to ensure an equivalent distribution of the methods across all threads, optimizing the utilization of available computational resources. When the number of threads is equal to or a multiple of the number of methods, the distribution is 100% equivalent. In other cases, the degree of equivalence increases with the number of threads. As presented in this subsection, the proposed algorithm exploits the parallel execution of three distinct optimization methods, with its efficiency in termination procedures incorporated through the criteria described in Section 3.1. The algorithm uses three termination criteria: the difference between the best values in successive iterations, the stability of the mean fitness value, and the relative variance of the cost function values. Each criterion contributes to the evaluation of the algorithm's progress, dynamically switching between these criteria based on the current state of the search. Specifically, when any of the criteria meets its predefined conditions, the algorithm terminates the search, thus adapting the process to the continuous search conditions. This strategy ensures the flexibility and efficiency of the algorithm, minimizing computational resources and time wastage. ## 4 Experiments

All experiments conducted were repeated 30 times to ensure the reliability of the algorithm producing the results. In experimental research, the number of repetitions required to ensure reliable results depends on the nature of the experiment, the variability of the data, and the precision requirements. There is no universal answer that applies to all experiments, but general guidelines and best practices have been established in the literature. In many scientific studies, at least 30 repetitions are recommended to achieve satisfactory statistical power. This is based on statistical theories such as the Law of Large Numbers and theorems for estimating sample distributions. The initial sample distribution is the same across all optimization methods, and the other parameters have been kept constant, as shown in Table 1. Specifically, in the parallel algorithm, the total number of initial samples remains as constant as possible, regardless of the number of threads used. The parallelization was achieved using the OpenMP library [80], while the implementation of the method was done in ANSI C++ within the optimization package OPTIMUS, available at [[https://github.com/itsoulos/OPTIMUS](https://github.com/itsoulos/OPTIMUS)]([https://github.com/itsoulos/OPTIMUS](https://github.com/itsoulos/OPTIMUS)) (accessed on 24 July 2024). The experiments were executed on a Debian Linux system that runs on an AMD Ryzen 5950X processor with 128 GB of RAM. The parameters used here are either suggested in the original publications of the methods (e.g., Differential Evolution, Particle Swarm Optimization) or have been chosen so that there is a compromise between the execution time of the method and its efficiency. Figure 1: Flowchart of the overall process. The DE acronym stands for the Differential Evolution method, the PU acronym represents the Parallel Unit that executes the algorithm.