# Deep Reinforcement Learning for RIS-Aided Multiuser MISO System with Hardware Impairments
## 2 System Model  
We consider a RIS-aided downlink multi-user system where the base station (BS) is equipped with \(M\) antennas and the user has a single antenna. The system architecture is shown in Figure 1. In this system, we assume that the RIS has \(N\) reflecting elements. Considering the hardware impairment, the transmit signal at the BS can be expressed as  
\[\mathbf{x}=\sum_{k=1}^{K}(\mathbf{w}_{k}s_{k}+\mathbf{\eta}_{s}), \tag{1}\]  
where \(\mathbf{w}_{k}\in\mathbb{C}^{M\times 1}\) represents the beamforming vector from the BS to the \(k\)-th user and \(s_{k}\sim\mathcal{CN}(0,1)\) represents the data signal symbol transmitted to the \(k\)-th user which satisfies \(\mathbb{E}\Big{\{}\abs{s_{k}}^{2}\Big{\}}=1\). Furthermore, \(\mathbf{\eta}_{s}\) denotes the independent Gaussian distortion noise, which satisfies the Gaussian distribution of zero mean, and its distortion noise power is proportional to the transmit power of the antenna. Then, \(\mathbf{\eta}_{s}\) can be represented as \(\mathbf{\eta}_{s}\sim\mathcal{CN}(\mathbf{0},k_{s}\text{diag}\Big{\{}\sum_{k=1}^{K} \mathbf{w}_{k}\mathbf{w}_{k}^{H}\Big{\}})\), where \(k_{s}\in(0,1)\) denotes the normalized variance of the emission distortion noise. For \(K\) users, the beamforming matrix at the BS can be expressed as  
\[\mathbf{W}=[\mathbf{w}_{1},\cdots,\mathbf{w}_{K}]. \tag{2}\]  
The beamforming matrix \(\mathbf{W}\) has to satisfy the power constraints, which can be formulated as  
\[\text{tr}\big{\{}\mathbf{W}\mathbf{W}^{H}\big{\}}\leq P_{\text{max}}. \tag{3}\]  
Furthermore, the channel between the base station and the RIS is denoted as \(\mathbf{H}_{SI}\in\mathbb{C}^{N\times M}\), the channel between the \(k\)-th user and the BS is denoted as \(\mathbf{h}_{SD,k}\in\mathbb{C}^{M\times 1}\) and the channel matrix from RIS to the \(k\)-th user is denoted as \(\mathbf{h}_{ID,k}\in\mathbb{C}^{N\times 1}\). In this paper, we consider the Rician fading model, the channel \(\mathbf{H}_{SI}\), \(\mathbf{h}_{SD,k}\), and \(\mathbf{h}_{ID,k}\) can be formulated as  
\[\mathbf{H}_{SI}=\sqrt{\beta}\left(\sqrt{\frac{\delta}{\delta+1}}\overline{ \mathbf{H}}_{SI}+\sqrt{\frac{1}{\delta+1}}\overline{\mathbf{H}}_{SI}\right), \tag{4}\]  
Figure 1: System Model. \[\mathbf{h}_{SD,k} =\sqrt{\gamma_{k}}\Bigg{(}\sqrt{\frac{\rho_{k}}{\rho_{k}+1}}\overline{ \mathbf{h}}_{SD,k}+\sqrt{\frac{1}{\rho_{k}+1}}\widetilde{\mathbf{h}}_{SD,k} \Bigg{)}, \tag{5}\] \[\mathbf{h}_{ID,k} =\sqrt{\alpha_{k}}\Bigg{(}\sqrt{\frac{\varepsilon_{k}}{\varepsilon _{k}+1}}\overline{\mathbf{h}}_{ID,k}+\sqrt{\frac{1}{\varepsilon_{k}+1}} \widetilde{\mathbf{h}}_{ID,k}\Bigg{)}, \tag{6}\]  
where \(\beta,\gamma_{k}\) and \(\alpha_{k}\) are the large-scale path loss coefficients; \(\delta,\rho_{k}\) and \(\varepsilon_{k}\) are the Rician factors; \(\overline{\mathbf{H}}_{SI}\), \(\overline{\mathbf{h}}_{SD,k}\) and \(\overline{\mathbf{h}}_{ID,k}\) represent the line-of-sight components, which are statistical CSI and remain unchanged over long time. When using the uniform area array model, the line-of-sight components \(\overline{\mathbf{H}}_{SI}\), \(\overline{\mathbf{h}}_{SD,k}\) and \(\overline{\mathbf{h}}_{ID,k}\) can be formulated as  
\[\overline{\mathbf{H}}_{SI} =\mathbf{a}_{N}(\theta_{A}^{a},\theta_{A}^{e})\mathbf{a}_{M}^{H} (\varphi_{D}^{a},\varphi_{D}^{e}), \tag{7}\] \[\overline{\mathbf{h}}_{ID,k} =\mathbf{a}_{M}(\theta_{D,k}^{a},\theta_{D,k}^{e}),\] (8) \[\mathbf{x}=\sum_{k=1}^{K}(\mathbf{w}_{k}s_{k}+\mathbf{\eta}_{s}), \tag{1}\]0  
where \(\theta_{A}^{a}\) and \(\theta_{A}^{e}\) denote the azimuth and elevation angles of arrival at the RIS from the BS, respectively; \(\varphi_{D}^{a}\) and \(\varphi_{D}^{e}\) denote the azimuth and elevation angles of departure from the BS to the RIS, respectively; \(\theta_{D,k}^{a}\) and \(\theta_{D,k}^{e}(\varphi_{D,k}^{a},\varphi_{D,k}^{e})\) are the the azimuth and elevation angles of departure from the RIS to the \(k\)-th user (from the BS to the \(k\)-th user). These angles are randomly generated.