# Deep Reinforcement Learning for RIS-Aided Multiuser MISO System with Hardware Impairments
###### Abstract  
In this paper, we study a reconfigurable intelligent surface (RIS)-aided multiuser MISO system with imperfect hardware, where the transceiver design is based on the statistical channel state information (CSI). Considering the transceiver hardware impairments (HWI), we aim to maximize the minimum average user data rate, where the precoding matrices at the base station (BS) and the reflecting phase shifts at the RIS are jointly optimized. Since the problem is nonconvex and the objective function cannot be derived in closed form, we adopt the deep deterministic policy gradient (DDPG) algorithm to deal with this challenging optimization problem, where we generate a set of CSI vectors in an offline way, and then these data sets are used to train the neural networks. The simulation results demonstrate the rapid convergence speed of the adopted DDPG algorithm and also emphasize that it is crucial to consider the HWI when optimizing the transceiver. intelligent reflecting surface (IRS); reconfigurable intelligent surface (RIS); hardware impairment (HWI); deep deterministic policy gradient (DDPG) +
Footnote †: journal: Applied  
for RIS-aided communication systems was studied in [15], where both imperfect CSI and HWI were taken into account. The semidefinite programming was proposed to solve the robust problem. However, all the above papers were based on the assumption that the BS can acquire the instantaneous CSI, which is challenging in practice due to the limited channel coherence time. Recently, the researchers have focused on the phase shift design based on the statistical CSI such as location/angle information or channel distribution information such as channel covariance matrices, which varies in a much lower time scale than the instantaneous CSI. There are several advantages to using statistical CSI for transceiver design [16]. Firstly, the channel estimation overhead can be reduced as only statistical CSI is needed, which changes very slowly. Secondly, the computational complexity is significantly reduced as the phase shifts at the RIS are only needed to be recomputed when the statistical CSI has changed. Thirdly, the feedback overhead is decreased since the phase shift values of the RIS are only fed back to the RIS controller only when its values are updated, which changes with statistical CSI. Due to the above appealing advantages, the transceiver design based on statistical CSI for RIS-aided systems has attracted extensive research attention [17; 18]. Specifically, the authors of [17] derived the closed-form date rate expression for a RIS-aided multiuser system. Then, a genetic algorithm was first proposed to optimize the phase shifts, which only depend on the statistical CSI. As a step further, the authors extended the work in [17] to the practical case when there are imperfect hardware, and a robust transmission design was proposed to optimize the phase shift by considering HWI. However, the contributions in [17; 18] considered the two-time scale design, where the BS designed its precoding matrices based on the instantaneous effective CSI, while only the phase shifts were designed based on statistical CSI. This means that the BS still needs to estimate the instantaneous effective CSI, which will incur sizable channel estimation overhead for highly-mobile scenarios. Against the above background, the authors of [19] studied the transceiver design for RIS-aided communication systems based on fully statistical CSI, where both the precoding matrices at the BS and the reflecting phase shifts at the RIS were designed based on statistical CSI. However, this work was based on the ideal assumption of perfect hardware, which is difficult to hold in practice. As a result, the contributions of this work are summarized as follows:  
* We consider optimizing the precoding matrices at the BS and the reflecting phase shifts at the RIS based on statistical CSI to maximize the minimum user data rate to ensure fairness among the users, where the imperfect hardware is taken into account.