# Deep Reinforcement Learning for RIS-Aided Multiuser MISO System with Hardware Impairments
## 4 Simulation Results  
In this section, the performance of the DDPG algorithm-based scheme is evaluated. Firstly, The locations of A and B are set at (0, 0, 30 m) and (100 m, 20 m, 10 m), respectively. Besides, the users are limited to a circle centered at (150 m, 0, 1.5 m) with a radius of 20 m. Other parameters are shown in Table 2.  
Based on the table above, we adopt the DDPG algorithm based on statistical CSI. The number of reflecting elements is set to \(N=20\), 30, 40, and 50, respectively. For each time step, we use the beamforming matrix \(\mathbf{W}\) and the phase shift matrix \(\mathbf{\Phi}\) at time step \(t\) as the input of the DDPG neural network, and the output will be \(\mathbf{W}\) and \(\mathbf{\Phi}\) at time \(t+1\). In Figures 3-6, we illustrate the minimum average user date versus the time steps for different \(N\). \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter Name** & **Sign** & **Parameter Value** \\ \hline Noise power density & \(\rho_{n}\) & \(-174\) dBm/Hz \\ \hline Channel bandwidth & \(B\) & \(1\) MHz \\ \hline Reference path loss & \(PL_{0}\) & \(0\)–\(30\) dB \\ \hline Reference distance & \(d_{0}\) & \(1\) m \\ \hline \multirow{3}{*}{Path loss coefficients} & \(\beta\) & \(2.2\) \\ \cline{2-3}  & \(\alpha_{k}\) & \(2.2\) \\ \cline{2-3}  & \(\gamma_{k}\) & \(3.75\) \\ \hline \multirow{3}{*}{Rician factors} & \(\delta\) & \(3\) \\ \cline{2-3}  & \(\varepsilon_{k}\) & \(3\) \\ \cline{2-3}  & \(\rho_{k}\) & \(3\) \\ \hline Correlation coefficients & \(\rho\) & \(0.1\) \\ \hline Normalized variance & \(k_{B}\) & \(0.01\) \\ \hline Normalized variance & \(k_{S}\) & \(0.01\) \\ \hline Numbers of antennas & \(M\) & \(8\) \\ \hline Numbers of users & \(K\) & \(4\) \\ \hline \multicolumn{3}{c}{Numbers of reflecting elements} & \(N\) & \(20\)–\(50\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Simulation Parameters. Figure 4: RIS = 30. Figure 3: RIS = 20. As shown in Figures 3-6, when the number of RIS reflecting elements is set to \(N=20\), the MAUR (minimum average user date) converges to 0.75 when the number is set to \(N=30\), MAUR converges to about 0.8. When the number of RIS reflecting elements increases to 40, MAUR increases to 0.9, and when \(N\) is 50, the rate is about 1.1. Hence, the MAUR increases with the number of reflecting elements. Meanwhile, the simulation results show that, under different conditions, the increase in the number of reflecting elements does not affect the converging speed of the proposed DDPG algorithm. In Figures 7-10, we respectively set \(k_{B}=k_{s}\) as 0.01-0.15 to explore the convergence of MUAR under different conditions. It indicates that the MAUR decreases with \(k_{B}\) and \(k_{s}\). This conclusion is also consistent with the SINR formula in (13). Furthermore, when \(k_{B},k_{s}\) are set to 0, the result is identical to the situation when hardware impairments are not considered. In addition, considering the influence of wind and rain in nature, we have added a random variable on the channel angles (angle of departure and angle of arrival), where the random variable is assumed to follow the uniform distribution. In general, the uniform distribution can be regarded as the worst case since the variables are uniformly distributed rather than peaking at one point for the Gaussian distribution. Then, we use the trained solution obtained from our DDPG networks for the realistic channels with angle variance to demonstrate the effectiveness of our algorithm. As observed from Figure 11, we can find that the performance degradation due to the channel variations is not too much, which confirms the robustness of our proposed algorithm. Figure 5: RIS = 40. Figure 6: RIS = 50. Figure 8: Convergence Speed when \(k_{B}=k_{S}=0.05\). Figure 7: Convergence Speed when \(k_{B}=k_{S}=0.01\). Figure 9: Convergence Speed when \(k_{B}=k_{S}=0.1\). Finally, in Figure 12, we compare the performance of the proposed algorithm with the existing non-optimized algorithm to evaluate the effectiveness of the optimization operations.