## 4 CNN optimization simulation environment, the algorithms' adaptations, and parameter setup
### Parameter setup  
In the following few paragraphs, we briefly describe the EE-TGA and \(E^{3}\)-FA control parameter values that were utilized in the simulation with CNN hyperparameter optimization. The same environment was utilized in [71; 72]. In EE-TGA, the population size \(N\) was set to eight CNN structures, and subpopulations \(N_{1}\) and \(N_{2}\) were set to three. The algorithm was iterated in 20 iterations (\(t_{max}=20\)). In each iteration, two worst solutions were removed from the population, and instead of them, two new random solutions were generated \((N3=N-(N1+N2))\). In the fourth group \((N_{4})\), in each iteration, one new solution was generated and modified with the max operator. The values of the control parameters \(\lambda\) and \(\theta\) were set to 0.5 and 0.2, respectively. The initial step size \(\sigma_{0}\) was set to three. In the second enhanced metaheuristic, \(E^{3}\)-FA, the initial population size, maximum iteration, and the initial step size had the same values as in EE-TGA. The \(\alpha\) randomization parameter was set to 0.5, the light absorption coefficient \(\gamma\) to 1.0, and the attractiveness at a distance of zero to 0.2. The metaheuristic control parameter setup of both enhanced algorithms is summarized in Table 1. The control parameters for the original TGA and FA were the same as for the enhanced versions, except the step size, which was not used in the basic TGA and FA implementations. The search space of the number of convolutional layers was set within the range between zero and six; the lower and upper bounds of the number of fully-connected layers were set within the interval \([1,4]\); and the lower and upper boundaries of the filter size were adjusted between one and eight, respectively. The number of filters was set between one and 128, and the hidden unit number in the FC layer was bounded between values of 16 and 2048. We restricted the possible set of CNNs hyperparameters' values to reduce the search space and to exclude the most infeasible CNN architectures. Lower and upper bounds for each hyperparameter were determined empirically and in this way, the proposed framework was much more efficient. Moreover, we wanted to start the search process by generating less complex and, thus, less expensive CNN configurations. For this reason, in the initialization phase, we set an even narrower range of possible values for the number of convolutional layers (\(h_{conv}\)) and the number of FC layers (\(h_{fc}\)). Please refer to Table 2. \begin{table}
\begin{tabular}{l l c} \hline \hline
**Algorithm** & **Parameter** & **Value(s)** \\ \hline EE-TGA & Population size (\(N\)) & 8 \\  & Sub-population size (\(N_{1}\)) & 3 \\  & Sub-population size (\(N_{2}\)) & 3 \\  & Number of discarded solutions (\(N_{3}\)) & 2 \\  & Number of novel solutions (\(N_{4}\)) & 1 \\  & Reproduction rate (\(\theta\)) & 0.2 \\  & Control parameter (\(\lambda\)) & 0.5 \\  & Maximum iteration number (\(t_{max}\)) & 20 \\  & Initial step size (\(\sigma_{0}\)) & 3 \\ \hline \(E^{3}\)-FA & Population size (\(N\)) & 8 \\  & Initial value for randomization parameter (\(\alpha\)) & 0.5 \\  & Light absorption coefficient (\(\gamma\)) & 1.0 \\  & Attractiveness at \(r=0\) (\(\beta_{0}\)) & 0.2 \\  & Maximum iteration number (\(t_{max}\)) & 20 \\  & Initial step size (\(\sigma_{0}\)) & 3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Metaheuristic control parameters. EE-TGA, exploitation enhanced tree growth algorithm; \(E^{3}\)-FA, exploitation and exploration enhanced firefly algorithm. With the objectives of reducing the amount of parameters (weights), the computational burden, and to control overfitting, for each convolutional layer, we used a pooling (downsampling) layer of a fixed size of 2 \(\times\) 2 along with the max pooling option. Further, we utilized the ReLU (rectified linear unit) activation function and employed the Adam optimizer with a fixed learning rate of \(\theta=0.0001\) and the mean squared error (MSE) loss function.