## 4 ANNs Modeling the Random Reflection Coefficient of an Ordinary Patch Antenna  
In this section, the author presents the experiments and their results corresponding to the derivation of ANNs that model the mean and standard deviation of \(\,|S_{11}|\,\) of an ordinary patch antenna shown in Figure 10. He conducted numerical experiments with the structure of ANN\({}_{\text{S}}\) shown in Figure 13.  
As a result, two separate ANNs were derived. The first one models the mean of \(\,|S_{11}|\,\). The standard deviation of \(\,|S_{11}|\,\) is modeled by the second ANN. Each ANN contains fully connected layers of neurons. The number of ANN layers is denoted by N, while the number of neurons in each ANN layer is denoted by M. Each neuron is associated with the ReLu activation function. The bias is used before each of the neurons and directly before ANN output. The initial values of weights are chosen using uniform random distribution in the range from 0 to 1. Input data are not normalized. The nominal values of \(\mathbf{f}_{0}\), \(\mathbf{y}_{0}\), and \(\mathbf{w}_{0}\) are put at the input of ANN using the number of GHz, mm, and mm, respectively. In the case of the training data described in the last paragraph of the previous section, the order of the input data can be illustrated as follows:  
\[X_{train}=\left[ \begin{array}{ccccccccc}24.8&\dots&24.8&24.8&\dots&24.8& \dots&24.8&25.4\\ 4.1&\dots&4.1&4.2&\dots&4.2&\dots&4.5&4.0&\dots\\ 1.2&\dots&0.3&1.2&\dots&0.3&\dots&0.3&1.2&\end{array} \right] \tag{1}\]  
Figure 13: The structure of each ANN used to model the mean and standard deviation of the random reflection coefficient of an ordinary Patch antenna in the 24.5–28.7 GHz frequency band. The learning rate of the values of weights and biases of ANNs is controlled using the ADAM algorithm [22], in which values of weights and biases (denoted by \(\theta\) in the ADAM algorithm) are updated in each time step \(t\) according to the formula:  
\[\theta_{t}=\theta_{t-1}-\alpha\cdot\hat{m}_{t}/\left(\sqrt{\hat{\theta}_{t}}+ \epsilon\right), \tag{2}\]  
where:  
\[\hat{\theta}_{t}=\hat{\theta}_{t}/\left(1-\beta_{2}{}^{t}\right), \tag{3}\]  
\[\hat{m}_{t}=m_{t}/\left(1-\beta_{1}{}^{t}\right), \tag{4}\]  
\[\hat{\theta}_{t}=\beta_{2}\cdot\hat{\theta}_{t-1}+(1-\beta_{2})\cdot\hat{g_{t} }^{2}, \tag{5}\]  
\[m_{t}=\beta_{1}\cdot m_{t-1}+(1-\beta_{1})\cdot\hat{g_{t}}, \tag{6}\]  
\[g_{t}=\
abla\theta_{t}(\theta_{t-1}), \tag{7}\]  
where \(g_{t}\) is the random nature gradient of the cost function at time step t, while \(\alpha\), \(\beta_{1}\), and \(\beta_{2}\) are the parameters whose values should be adjusted. The following cost function is used:  
\[MSE_{(y,y^{\prime})}=\frac{\sum_{i=1}^{W}(y_{i}-y_{i}^{\prime})^{2}}{W}, \tag{8}\]  
where \(y_{i}\) is the expected (training or testing data) value, while \(y_{i}^{\prime}\) is the estimation given by ANN. The function of the reflection coefficient of an ordinary patch antenna is characterized by deep local minima. These minima are deeper than those experienced, e.g., in [19]. The author performed many numerical experiments to find the best ANN candidates for modeling the mean and standard deviation of \(\left|\,S_{11}\,\right|\). Consequently, the following conclusions were made. The best results of the approximation of the mean of \(\left|\,S_{11}\,\right|\) are obtained for the values of \(\beta_{1}\), \(\beta_{2}\), and \(\alpha\) equal to approximately 0.99, 0.999, and 0.001, respectively. The optimal batch size is 20-40. For the case of the approximation of the standard deviation of \(\left|\,S_{11}\,\right|\), \(\alpha=0.005\) is a better choice. The optimal number of the ANN layers for the mean of \(\left|\,S_{11}\,\right|\) is 9. The corresponding number for the standard deviation of \(\left|\,S_{11}\,\right|\) is 7. In both cases, the optimal number of neurons in each of ANN layers is about 20; however, for the case of the standard deviation of \(\left|\,S_{11}\,\right|\), it should be slightly less, e.g., 18.