# Collaborative Beamforming with DQN for Interference Mitigation in 5G and Beyond Networks
## 2 Related Work
NHB leverages a machine learning-based design applicable to various beamforming/combining architectures, offering flexibility and compatibility not found in prior methods. Furthermore, the proposed iterative training strategy ensures fast convergence, making NHB a promising solution for practical mmWave MIMO systems. The work in [13] focused on leveraging machine learning (ML) techniques to optimize beamforming configurations and overcome challenges like complex channels, hardware constraints, and high complexity. The paper reviews existing ML methods, discusses their applications and potential benefits, explores future research directions, and presents a case study comparing ML-based and conventional approaches. In essence, it highlights how ML can be used to unlock efficient and intelligent beamforming for improved mmWave communication. Research conducted in [4] developed a recurrent neural network model for predicting optimal beam patterns in mmWave systems. The research in [5] applied Q-learning with received signal strength (RSS) feedback to adaptively adjust beams in indoor 60 GHz networks. These studies demonstrate the potential of machine learning to enable intelligent beamforming by modeling high-dimensional spaces and optimizing based on data. In addition, the study in [12] developed a neural hybrid beamforming approach compatible with different system configurations. The method trains an autoencoder model in an unsupervised manner to obtain the analog and digital beamforming parameters. This demonstrates the potential of leveraging artificial intelligence for flexible hybrid beamforming optimization in mmWave MIMO systems. Furthermore, the approach in [16] proposed a novel machine learning-based beam scheduling approach specifically designed for wireless virtualization with large-scale MIMO and subleased radio frequency slices. This approach leverages ML for optimized scheduling, aiming to outperform existing methods. Another approach, in [17], proposed an adaptive BeamMaP model for user localization in distributed massive MIMO systems. The approach leverages training data across various channel conditions and adaptively steers beams based on estimated user locations. This demonstrates the potential of adaptive beamforming techniques to enable robust localization in challenging outdoor settings with multipath and dynamic weather conditions. Aspects of secure communications in Internet of Things networks have been examined in earlier research, such as the use of intelligent reflecting surfaces and cooperative jamming. Nevertheless, there is still a lot to learn about how to combine edge computing with deep reinforcement learning, namely deep deterministic policy gradient (DDPG), to maximize energy economy and confidentiality in IRS-assisted cooperative networks [18]. To improve wireless communication security, The work discussed in [19] investigated the combination of physical layer security methods with intelligent reflecting surfaces. Nevertheless, there is more work to be done in the area of jointly optimizing transmit beamforming and intelligent reflecting surfaces (IRS) phase shifts for energy efficiency maximization under poor channel state information (CSI), especially when using deep reinforcement learning in an anti-jamming setting. On the other hand, the investigation in [20] proposed that the beam codebook design for multiuser MIMO systems has traditionally depended on either reference signal received power (RSRP)-based user clustering or complete CSI feedback, which often leads to resource-intensive operations or restricted accuracy. Though mostly limited to offline training and fixed beam topologies, recent machine learning-based techniques have shown promise and opened the door for more flexible and effective solutions in dynamic contexts. Conventional coordinated beamforming techniques in large MIMO systems have mostly depended on centralized optimization methods, which have a high computing cost and need global CSI, making them unsuitable for use in dynamic mobile situations by [21].