## 4 CNN optimization simulation environment, the algorithms' adaptations, and parameter setup
### Solutions' encoding and algorithms' adaptations  
As was already mentioned in Section 1, CNN hyperparameters that were subject to optimization in the proposed model included the number of convolution layers along with the number of kernels per layer and the kernel size and number of fully-connected (dense) layers along with the size of each fully-connected layer. We did not include the type of pooling layer, activation function, regularization technique, and other CNN hyperparameters in the automatic search because of the limitations of available computing power. A detailed description of the utilized mathematical model is given in Section 2. The possible hyperparameter values that were subject to the optimization in this study were defined within the lower and upper bounds (these bounds are given in Subsection 4.2), with the aim to exclude infeasible solutions of CNN architectures and reduce the utilization of computer resources. The population at iteration \(t\) consisted of the set of \(N\) CNN configuration solutions, and the set is defined as follows:  
\[H^{t}=\{h_{0},h_{1},\dots,h_{N}\} \tag{25}\]  
where each configuration \(h\) consists of \(h_{conv}\) and \(h_{fc}\). According to the mathematical formulation of the utilized model, given in Section 2, each solution in the population (network configuration \(h\)) consisted of variable length parameters. Since we did not include continuous parameters like the learning rate, all parameters took only discrete values. The number of solution parameters was different for each solution in the population and depended on the number of convolutional layers (where each convolutional layer was further determined by two parameters: the number of kernels and the kernel size) and the number of FC network layers. For example, if a generated solution (network configuration) consisted of three convolutional layers and four FC layers, then such a solution would be encoding with 10 discrete parameters (\(3\cdot 2+4\)). A visual representation of the solutions' encoding scheme is given in Figure 1.  
To decrease the number of possible network configurations and to reduce the search domain, the convolutional layers were sorted in descending order, first concerning the filter size (\(k_{size}\)) and then based on the number of filters (\(k_{count}\)). The value of \(s\), which denotes the FC layer size, was utilized for sorting dense layers. Figure 1: Solutions’ encoding scheme. In the beginning, the initial population of solutions (convolutional neural network structures) was randomly generated within the defined lower and upper bounds by using the following equation:  
\[h_{i,j}=min_{j}+rand\cdot(max_{j}-min_{j}) \tag{26}\]  
where \(h_{i,j}\) represents the \(j\)th parameter of the \(i\)th solution in the population, \(rand\) is a random number drawn from a uniform distribution, and \(min_{j}\) is the lower bound and \(max_{j}\) the upper bound of the \(j\)th parameter. Due to the relatively large range of parameter values, we did not adapt the proposed approaches for discrete optimization, and we just rounded the results to the nearest integer. After generating the initial population, we evaluated the fitness value of each solution, and the current best solution in the population was determined. The fitness function that was utilized was simply inversely proportional to the classification error rate (\(err\)) of the respective CNN architecture \(h\). For calculating the fitness of solution \(i\), the following expression is used:  
\[F(h_{i})=\frac{1}{1+|err_{i}|} \tag{27}\]  
where the fitness function of CNN structure \(h_{i}\) is denoted by \(F(h_{i})\) and \(err_{i}\) represents the objective function of the concerned structure. The fitness calculation process utilized much computational resource because each fitness evaluation required a CNN to be developed, trained, and validated against the dataset.