# Collaborative Beamforming with DQN for Interference Mitigation in 5G and Beyond Networks
## 4 Performance Evaluation  
In our simulation, we develop the environment using Python language. Python is an open-source language that helps to build large environments in a very fast way [26]. We also use the TensorFlow framework [27] to train our DQN model under the Anaconda environment [28]. The layer structure is easily made from Keras library [29]. To make our environment as well as the DQN model, we use some parameters that are given in Table 2. The parameter of Table 3 helps to fine tune our DQN model appropriately by a trial and error method. Signal to interference plus noise ratio (SINR) is an essential metric that represents how well the signal quality is received by a receiver. It indicates how strong the desired signal is compared to the combined effect of interference and noise. A higher SINR typically indicates better signal quality and consequently better communication performance. In our proposed DQN-based method, the SINR at each successful step is higher than in other  
Figure 4: Flow chart of proposed DQN model. Figure 5: Model summary of proposed LSTM + Dense. existing work [1]. Higher SINR indicates better quality of service and communication. The proposed DQN (blue line) in Figure 6 is constantly higher than the other two existing methods [1]. This is also an indication that our SINR achieved through the proposed method provides better quality of service (QoS). In Figure 6, we can see that our proposed technique minimum SINR is 5 dB, whereas the existing DQN and Q-Table give the SINR near one and two, respectively. Moreover, our proposed DQN holds a very good SINR, which is greater than 20 dB, whereas others cannot cross 20 dB SINR. Therefore, the proposed DQN improves significantly compared to the existing solution. Furthermore, we derived a comparison in Table 4 according to serving power, interference power, SINR, and sum-rate capacity. The sum-rate capacity is calculated (similar to [1]) as in (7). \[\text{Sum-Rate Capacity}=\sum_{i=1}^{T}\log_{2}(1+\gamma(t)). \tag{7}\]  
In Table 4, we can observe that our proposed DQN uses minimum serve power of 8.51 dB whereas the maximum served does not cross 20 dB. However, the served power of the existing technique [1] is greater than our proposed technique. Moreover, the existing DQN and Q-Table techniques used 38.84 and 34.12 as average interference power, which are very high compared to our proposed DQN, which is only 5.42. Due to having very low average interference power by controlling beam steering intelligently via our proposed method, the served power is higher than interference power; as a result, our SINR achievement is much better than the existing DQN and existing Q-Table method. Finally, our proposed DQN technique shows an average sum-rate capacity of 3.90, whereas existing DQN and existing Q-Table are 3.07 and 3.45, respectively. \begin{table}
\begin{tabular}{l l} \hline \hline
**Parameters** & **Values** \\ \hline Cellular environment type & Circular \\ Maximum transmit power of \(BS\) & 40 \\ Bearer & Voice \\ Minimum SINR & 3 dB \\ Number of antennas, \(N\) & 2 \\ Transmission type & downlink \\ Frequency band & 2.1 GHz \\ Cell radius, \(r\) & 350 m \\ Inter-site Ddistance & 525 m \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Environmental parameters. \begin{table}
\begin{tabular}{l l} \hline \hline
**Parameters** & **Values** \\ \hline Discount factor, Size of state & 8 \\ Size of action & 2 \\ Minimum exploration rate & 0.001 \\ Learning rate & 0.001 \\ Maximum size of memory & 100 \\ Loss function & Categorical\_Crossentropy \\ Activation function in hidden layer & Relu \\ Activation function in output layer & Sigmoid \\ LSTM model type & Sequential \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Training parameters. Interference power refers to the unwanted signals or noise that degrade the quality of the received signal. Interference can arise from various sources and can have a significant impact on the performance of the communication signal.