# Machine-Learning-Based Optimization for Multiple-IRS-Aided Communication System
## 1 Introduction
Additionally, the authors in [14] introduced a variety of system scenarios, including a single- and multi-IRS-assisted system, and used alternating optimization to solve the designed problems for joint beamforming under the presented traffic patterns. Although the above-presented approaches and algorithms, which adopt traditional optimization-based methods, provide reasonable performance, they are generally constrained in practice by system complexity [7]. Thus, the attention of researchers was drawn to machine learning (ML) methods because of their exceptional abilities in handling complex practical communication systems [15]. In the same perspective, machine learning (ML) is recognized as a vital enabler in implementing IRS-aided systems because of its critical role in decreasing optimization complexity [2; 16]. Generally, ML learns from fresh data and refines its operations to increase the total performance, gaining \"intelligence\" over time to anticipate output values within an acceptable range. A comprehensive survey of recent studies on ML-based IRS-assisted systems can be found in [16]. Among the ML approaches, deep learning (DL) has shown tremendous achievements in current communication systems. Due to its exceptional learning abilities, it has been primarily applied in IRS-aided communication systems to study channel estimation (e.g., [17; 18]) and beamforming optimization (e.g., [19; 20; 21; 22]). In terms of channel estimation, the work in [17] presented a DL framework for direct and cascaded channel estimation, where a convolutional neural network-based architecture is modeled and fed by the received pilots. Meanwhile, the authors in [18] proposed a deep denoising neural network for uplink channel estimation in mmWave IRS-assisted system. Note that channel estimation approaches are strongly related to the channel model of the constructed system model,which further influences the prediction result; some recent works in the channel model can be found in [23; 24], which are suitable to be applied in IRS-aided communication system. In terms of beamforming optimization, the work in [19] designed a deep neural network for the online setting and optimized the beamforming in an indoor environment. In this work, the training model was developed to maximize the final received signal strength by mapping between the UE's position data and the optimal IRS phases. The used database was collected offline before training the designed model. A model based on unsupervised learning was developed for the IRS passive beamforming configuration [20], which had lower computational complexity than the SDR-based model. The authors of [21] used supervised learning to develop an interaction mechanism between the IRS and incident signals by leveraging the information of sampling channel vectors at the IRS's active units. A deep learning model was presented in [22] for an IRS-assisted system. The reflection angles of the IRS and channels were estimated based on the received signal at the IRS, aiding in detecting and estimating symbols through the system. This model can enhance the bit error rate in the system compared to standard detectors. Besides DL, through combining DL and reinforcement learning, deep reinforcement learning (DRL) is widely adopted in IRS-aided systems for optimization design. It learns by a trial-and-error mechanism and does not need the current data for outcome prediction, as in DL. In the context of the optimization of IRS reflection phases, the authors in [25] proposed a DRL algorithm for predicting the IRS reflection phases with minimal beam training overhead compared to the supervised-based approach proposed in [21]. In [11], a DRL approach for IRS reflection was presented to optimize the throughput in the imperfect CSI scenario, where the learning model was trained to predict the reflection beamforming based on the quantile regression approach for efficient convergence.