# Machine-Learning-Based Optimization for Multiple-IRS-Aided Communication System
## 5 Simulation Results  
In this section, we present the numerical results for both the static UE scenario and the dynamic UE scenario. The considered system's 2D deployment is depicted in Figure 4, where the BS is positioned at \((x_{B},y_{B},z_{B})=(-200,0)\), and three IRSs are deployed at position coordinates denoted by \((x_{l},y_{l},z_{l})\) at \((0,-100,10)\), (100,0,10), and (0,100,10), respectively, with the same BS height. We consider that each IRS surface deployed on a 2D rectangular grid has \(N=N_{H}N_{V}\) reflecting elements, where \(N_{H}\) and \(N_{V}\) denote the number of elements per row and column, respectively. There are \(K=4\) UEs randomly placed in a circle with a radius of R = 80 m centered at the origin. As for the channel model, the BS-UE channels are assumed to follow Rayleigh fading, i.e., \(\mathbf{h}_{dk}^{H}=\beta_{0,k}\widetilde{\mathbf{h}}_{dk}^{H}\), where \(\widetilde{h}_{dk}^{H}\) is modeled as \(\mathcal{CN}(0,1)\), and \(\beta_{0,k}\) is the corresponding path loss between the BS and \(k\)-th UE modeled as \(32.6+36.7 log10(d^{direct})\) [dB] [10], where \(d^{direct}\) is the distance between the BS and \(k\)-th UE of the direct link. The IRSs are deployed to maintain the line-of-sight (LOS) links with the BS and with the UEs. Thus, the channels of the BS-IRSs and IRSs-UEs are modeled as Rician fading, i.e., \(\mathbf{G}_{l}=\beta_{1,l}(\sqrt{\frac{\epsilon}{1+\epsilon}}\widetilde{\mathbf{G}}_ {l}^{(\text{LOS})}+\sqrt{\frac{1}{1+\epsilon}}\widetilde{\mathbf{G}}_{l}^{(\text{ NLOS})})\) and \(\mathbf{h}_{rl}^{H}=\beta_{2,kl}(\sqrt{\frac{\epsilon}{1+\epsilon}}\widetilde{\mathbf{h}}_ {rl}^{H(\text{LOS})}+\sqrt{\frac{1}{1+\epsilon}}\widetilde{\mathbf{h}}_{rl}^{H( \text{NLOS})})\), respectively, where \(\beta_{1,l}\) and \(\beta_{2,kl}\) are the path loss between the BS and \(l\)-th IRS, and between the \(l\)-th IRS and \(k\)-th UE, respectively. The path loss in indirect channels is modeled as \(35.6+22 log10(d^{indirect})\) [dB] [10], where \(d^{indirect}\) is the corresponding distance between nodes in indirect links, i.e., BS-\(l\)-th IRS or \(l\)-th-\(k\)-th UE. Here, \(\epsilon\) is the Rician factor, set as 10 in the simulations. The elements of NLOS components can be constructed as [37], i.e., \(\widetilde{\mathbf{G}}_{l}^{\text{LOS}}=\mathbf{a}_{IRS}(\theta_{1,l},\psi_{l,l})\mathbf{a }_{BS}(\theta_{1},\psi_{1})^{H}\) and \(\widetilde{\mathbf{h}}_{rl}^{H(\text{LOS})}=\mathbf{a}_{IRS}(\theta_{l,k},\psi_{l,k})^ {H}\), where \(\mathbf{a}_{IRS}\) and \(\mathbf{a}_{BS}\) are the IRS and BS steering vectors, respectively, with corresponding azimuth and elevation angles, i.e., \(\theta\) and \(\psi\). The \(n\)-th element of the \(l\)-th IRS steering vector is \(\mathbf{a}_{IRS}(\vartheta_{l,k},\psi_{l,k})\)\({}_{n}=e^{2\pi d^{l}/\lambda\lambda_{c}}\{i_{1}(n)\sin\left(\vartheta_{l,k}\right)cos(\psi_{l,k})+i_{2 }(n)sin(\psi_{l,k})\}\), where \(d^{l}\) is the distance separating the \(l\)-th IRS adjacent elements, \(\lambda_{c}\) is the wavelength of carrier, \(i_{1}(n)=mod(n-1,N_{H})\), \(i_{2}(n)=\lfloor(n-1)/N_{H}\rfloor\). In the simulations, we set \(2d^{l}=\lambda_{c}\), fix \(N_{H}=2\), and increase \(N_{V}\) in our deployed IRSs. Additionally, \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyper-Parameter** & **Setting** \\ \hline Actor learning rate \(\mu_{a}\) & 0.001 \\ Critic learning rate \(\mu_{c}\) & 0.001 \\ Target smooth factor \(\tau\) & 0.001 \\ Discount factor \(\gamma\) & 0.95 \\ Mini-batch size \(N_{\text{BS}}\) & 16 \\ Experience replay buffer size C & 200,000 \\ Target update frequency \(U\) & 1 \\ Sampling time \(T_{s}\) & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Proposed DDPG hyper-parameter settings. \(\sin\left(\vartheta_{l,k}\right)\cos(\psi_{l,k})=(y_{k}-y_{l})/d_{k}^{l}\) and \(\sin\left(\psi_{l,k}\right)=(z_{k}-z_{l})/d_{k}^{l}\), where \(d_{k}^{l}\) is the distance between the \(l\)-th IRS and \(k\)-th UE.