# Metamaterial Design with Nested-CNN and Prediction Improvement with Imputation
## 3 The Nested-CNN Architecture and Hyperparameter Optimization  
The nested-CNN consisted of two models. Model-2 was trained to detect the reflection coefficient of the given input, which was the image of MM. Model-1 was trained to detect the image of MM of the desired reflection coefficient that was given as the input. In CNN and DNN models, weight, bias and kernels were updated with gradient-based optimization methods [41]. In gradient-based optimization methods, the model's update  
Figure 2: Metamaterial dimensions. process depends on the derivative of the loss value with respect to weights. The update process is given below:  
\[W_{t}=W_{t-1}-\alpha\frac{dL}{dW_{t-1}} \tag{1}\]  
where \(W_{t}\) is new weights, \(W_{t-1}\) is old weights, \(L\) is loss of the model, \(\alpha\) is the learning rate. In nested-CNN, Model-2 that was used in Model-1's loss function was trained first and used in the training process of Model-1. Loss value has been created by comparing the desired reflection coefficient, which was the input of Model-1 and the reflection coefficient, which was the output of Model-2. The schematic of the nested-CNN is shown in Figure 3. Model-1's input and Model-2's output were used in the _MAE_; the desired reflection coefficient and reflection coefficient of the produced image was compared and Model-1 was updated and improved using the defined loss method. \(S11_{D}\) is the desired reflection coefficient, \(S11_{P}\) is the generated image's reflection coefficient, and _MAE_ is calculated as given below:  
\[MAE=\frac{1}{N}\sum\abs{S11_{D}-S11_{P}} \tag{2}\]  
Hyperparameters are the variables that determine the artificial intelligence (AI) model structure, and the success of the CNN model depends on hyperparameters. Keras Tuner is a hyperparameter optimizer [42] that searches the parameters by using the random search algorithm [43], hyperband [44], or Bayesian optimization [45]. The random search algorithm requires more processing time than hyperband and Bayesian optimization but guarantees optimal results. In our experiment, hyperparameter optimization was provided by using Keras Tuner with the random search algorithm for both models. Parameters are given in Table 1, which were used for optimization. Figure 3: Schematic of the nested-CNN. Some popular momentum-like optimization methods, such as the root mean square error probability (RMSProp), adaptive moment estimation (Adam), and Adadelta were used to estimate the optimum direction and velocity for the cost to move towards the global minimum [46]. Dropout randomly drops layers with a specified rate to avoid overfitting; dropout was chosen--None or 0.5 for our case. The learning rate controls how quickly the model was adapted to the problem. Learning rates, large or small, have different advantages and disadvantages; smaller learning rates require a high number of epochs and there is a possibility of getting stuck before reaching the optimum result, while with larger learning rates, the result can be reached much faster, but it may converge to non-optimal results other than optimum. Four learning rates were used in hyperparameters optimizations: 1, 0.1, 0.01, 0.001. The batch size was the number of data used per iteration for training, and the batch size was investigated with values of 1, 2, 4, 8, 16, 32. CNN filters extract the feature from the portions of the image, and the kernel's size was investigated between 3 by 3 to 5 by 5. The activation function is used in deep learning models to define the output by using an input or set of inputs of the corresponding layer. The activation function mimics the stimulation of a biological neuron. In hyperparameter optimization, the used activation functions were binary, linear, sigmoid, tanh, rectified linear unit (ReLU), leaky ReLU (LReLU), parametric ReLU (PReLU), exponential linear unit (eLU), ReLU-6, scaled exponential linear unit (Selu), Softplus, Softsign and Softmax, Swish [47]. Validation and test data have been reserved to test the success of both models and to prevent overfitting.