## 4 Experiments and Results
The number of evaluations of the hybrid multiple elite solutions ensemble (39,245) is competitive with the number of evaluations of the individual optimizers ICA (37,159), TLBO (40,424), and SHLO (40,033). Therefore, the faster convergence and smaller errors provided by the hybrid multiple elite solutions ensemble make this strategy the preferred choice. We can see in Table 2 that each ensemble strategy has roughly the same number of evaluations, i.e., 39,200. This is an expected result because we use the same configuration of weak optimizers in all ensemble strategies. However, we emphasize that the value of 39,200 is the approximate average of the number of evaluations from the individual optimizers ICA (37,159), TLBO (40,424), and SHLO (40,033). Since we use an equal number of iterations (i.e., five) for each weak optimizer in all ensemble strategies, each ensemble inherits the average computational effort of the individual optimizers. However, choosing an unbalanced number of iterations for weak optimizers of an ensemble is an easy way to control the trade-off between inherited computational effort and inherited performance. To investigate the impact of an unbalanced number of iterations for weak optimizers, we present in Figure 9 the convergence curve graphs of each ensemble strategy for the following scenarios:  
* _Scenario_ 1: five iterations are used for each weak optimizer in all ensemble strategies. This is the original experiment setup, and we use it as a baseline comparison. The convergence curve graphs for this scenario are shown in Figure 9 with continuous lines and are labeled Cascade-balanced, SingleElite-balanced, and MultipleElite-balanced;
* _Scenario_ 2: nine iterations for ICA weak optimizer and three iterations for TLBO and SHLO are used. In this scenario, we want the ensembles to inherit, to a greater extent, the characteristics of the best optimizer, i.e., ICA. The convergence curve graphs for this scenario are shown in Figure 9 with dashed red lines and are labeled Cascade-ICA, SingleElite-ICA, and MultipleElite-ICA; \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{3}{c}{**Average Results for 30 Independent Runs**} & \multicolumn{3}{c}{**Best Results**} \\  & **RMSE Training** & **RMSE Testing** & **No. Evals** & **RMSE Training** & **RMSE Testing** & **MLP Structure** \\ \hline ICA & 0.00609 & 0.01079 & 37,159 & 0.00553 & 0.00982 & [3-8-2-3] \\ TLBO & 0.00693 & 0.01210 & 40,424 & 0.00598 & 0.01053 & [3-7-2-3] \\ SHLO & 0.00719 & 0.01265 & 40,033 & 0.00653 & 0.01194 & [3-11-2-3] \\ Hybrid cascade & 0.00655 & 0.01159 & 39,136 & 0.00555 & 0.00983 & [3-8-3-3] \\ Hybrid single elite solution & 0.00466 & 0.01142 & 39,225 & 0.00548 & 0.00973 & [3-11-2-3] \\ Hybrid multiple elite solutions & 0.00582 & 0.01029 & 39,245 & 0.00517 & 0.00932 & [3-8-4-3] \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Simulation results obtained on the MMA data set with the individual optimizers and the ensemble strategies. * _Scenario 3_: nine iterations for TLBO weak optimizer and three iterations for ICA and SHLO are used. In this scenario, we want the ensembles to inherit, to a greater extent, the characteristics of the second best optimizer, i.e., TLBO. The convergence curve graphs for this scenario are shown in Figure 9 with dashed blue lines and are labeled Cascade-TLBO, SingleElite-TLBO, and MultipleElite-TLBO;
* _Scenario 4_: nine iterations for SHLO weak optimizer and three iterations for ICA and TLBO are used. In this scenario, we want the ensembles to inherit, to a greater extent, the characteristics of the worst optimizer, i.e., SHLO. The convergence curve graphs for this scenario are shown in Figure 9 with dashed gray lines and are labeled Cascade-SHLO, SingleElite-SHLO, and MultipleElite-SHLO. Note that we use a cumulative number of 15 iterations for the weak optimizers in all scenarios. We choose this number of iterations for convenience because the results for _Scenario 1_ are already available, and new simulations are unnecessary.