## 3 Materials and Methods
### Strategies for Combining Optimization Algorithms
#### 3.3.3 Hybrid Single Elite Solution Strategy  
In the hybrid single elite solution strategy (Figure 5), we propose that an algorithm does not depend on the results of other algorithms at the same ensemble iteration. However, at the end of an ensemble iteration, we take the best solution from each algorithm in the ensemble to define an improved population. Thus, a new ensemble iteration will use the previous improved population as the initial population. The suggestive name of the \"single elite solution\" strategy is derived from the fact that we take only one solution (i.e., the elite) from each algorithm to define an improved population.  
In the hybrid single elite solution strategy, an ensemble iteration always starts from a population of three individuals. Therefore, the population generation procedure is suggestively called _GeneratePopulation_3-to-N_. If a different number of algorithms are used in the ensemble, e.g., _NoAlg_\(\geq 2\), the population generation procedure is adapted for the general use case, i.e., _GeneratePopulation_NoAlg-to-N_. In this procedure, we propose that the cloning operation of the initial solutions is influenced by their objective function value. Let the initial solutions be \(s_{1}\), \(s_{2}\),..., \(s_{n}\) and their objective function values \(f_{1}\),\(f_{2}\),...,\(f_{n}\). We use the objective function values \(f_{1}\),\(f_{2}\),..., \(f_{n}\) to determine the weights \(w_{1}\), \(w_{2}\),..., \(w_{n}\) associated with the initial solutions using the softmax function:  
\[w_{i}=\frac{e^{-f_{i}}}{\sum_{k=1}^{n}e^{-f_{k}}} \tag{6}\]  
Since the goal is to minimize the RMSE value of an MLP, in (6) we use the objective function value with a negative sign to obtain higher weights for solutions with a lower objective function value. The obtained weights determine the number of cloning operations for each initial solution. In other words, we clone a larger number of fitter solutions and a smaller number of less fitted solutions. For example, if the initial solutions \(s_{1}\),\(s_{2}\),\(s_{3}\) have the objective function values \(f_{1}=0.01\),\(f_{2}=0.5\),\(f_{3}=0.9\), then their weights are approximatively \(w_{1}=0.5\), \(w_{2}=0.3\), \(w_{3}=0.2\). Therefore, for a population size of \(N=100\) individuals, a population of \(50\)\(s_{1}\) clones, \(30\)\(s_{2}\) clones, and \(20\)\(s_{3}\) clones will be generated.  
Figure 4: Hybrid cascade strategy. For each iteration of the ensemble, the initial population is generated from the improved solution of the previous iteration. Starting from this solution, a new population sol_i*, i = 1... N, is generated by applying a mutation operator to clones sol_i, i = 1... N of the previously-improved solution.  
In the last step of the _GeneratePopulation_NoAlg-to-N_ procedure, we apply the mutation operator to each cloned solution to obtain the final population.  
One can see in Figure 5 that the population generation procedure is used independently for each algorithm. The main reason we generate different populations is that each algorithm requires a population of a different size. Additionally, generating different populations leads to better diversity that could improve ensemble performance.