# Collaborative Beamforming with DQN for Interference Mitigation in 5G and Beyond Networks
###### Abstract  
This paper addresses the problem of side lobe interference in 5G networks by proposing a unique collaborative beamforming strategy based on Deep Q-Network (DQN) reinforcement learning. Our method, which operates in the sub-6 GHz band, maximizes beam steering and power management by using a two-antenna system with DQN-controlled phase shifters. We provide an OFDM cellular network environment where inter-cell interference is managed while many base stations serve randomly dispersed customers. In order to reduce interference strength and improve signal-to-interference-plus-noise ratio (SINR), the DQN agent learns to modify the interference angle. Our model integrates experience replay memory with a long short-term memory (LSTM) recurrent neural network for time series prediction to enhance learning stability. The outcomes of our simulations show that our suggested DQN approach works noticeably better than current DQN and Q-learning methods. In particular, our technique reaches a maximum of 29.18 dB and a minimum of 5.15 dB, whereas the other approaches only manage 0.77-27.04 dB. Additionally, we significantly decreased the average interference level to 5.42 dB compared to competing approaches of 38.84 dB and 34.12 dB. The average sum-rate capacity is also increased to 3.90 by the suggested strategy, outperforming previous approaches. These findings demonstrate how well our cooperative beamforming method reduces interference and improves overall network performance in 5G systems. collaborative beamforming; Deep Q-Network (DQN); 5G networks; interference mitigation; LSTM neural networks; OFDM; sub-6 GHz 20  
Here, we can observe that the beam width is wider because of the small antenna distance (d = 43 mm). Moreover, the directivity of the main lobe is narrower after increasing the antenna spacing (d = 300 mm), but the higher side lobe arrives in Figure 1c, which can create interference to another user where multiple \(BS\) exist near to one another. In 5G beamforming, we need to ensure good data transmission quality for the user, so we need to control the side lobe to reduce interference. Collaborative beamforming is one of the promising techniques to control the side lobe [5]. Collaborative beamforming creates a virtual antenna array where multiple independent base stations are located to cooperate with one another [6]. The collaborative beamforming enhances the voice quality during transmission, such as the signal interference noise ratio (SINR), with a minimum amount of power. There are many collaborative approaches, such as power control and beam-steer control, available to deal with the higher side lobe problem [1, 6]. The authors in [6] studied the excitation current control using hybrid discrete and continuous optimization to reduce the side lobe for enhancing the energy quality. The authors in [1] proposed a Deep Q-Network (DQN) based joint power control technique to reduce the interference power for better sum-rate capacity in the uniform linear array (ULA) based beamforming sector. However, the authors improved the voice and data bearer's signal quality by increasing the number of antennas up to 32, which basically increases the cost of the production. Therefore, we need an optimized solution to overcome cost by minimizing the number of antennas. In this work, we propose a two-antenna collaborative beamforming technique using deep reinforcement learning to improve the SINR and sum-rate capacity by controlling beam-steering and the power of the interfering signal [7]. The following are the detailed contributions to this paper:  
1. We introduce the cost-effective DQN solution to control the beam-steering to reduce the slide lobe interference. 2. We mainly focus on reducing the interference power controlling beam-steering using the proposed DQN. 3. In this work, our proposed solution applies to a two-antenna collaborative environment to reduce the cost of operation. 4.