# Intelligent Interference Management in UAV-Based HetNets +
## 6 Simulation Results  
The ICIC and UAV placement algorithms were evaluated in a UAV HetNet, and the parameters are as defined in Table 5. The DDQN-based AI algorithm was implemented using Intel RL Coach [29]. Intel RL Coach is a python framework, and it implements many state-of-the-art algorithms. These algorithms can be used through a set of application programming interfaces (APIs). The developer defines the environment and the optimization problem and calls the Intel RL Coach APIs to solve the problem. The UAV HetNet environment was created using python scripts, while Matlab scripts were used to simulate the path-loss model, associate UEs to BSs, and calculate the SE for a particular realization. The agent was trained over 300 realizations and then evaluated over 250 realizations, where the location of UEs and BSs in each realization were generated using a random Poisson point process (PPP) with UE and MBS densities as indicated in Table 5. The CDF of the median and mean SE achieved by the algorithms, calculated over multiple realizations, are shown in Figure 6a,b, respectively, when the density of MBSs is 4 MBSs per km\({}^{2}\), resulting in one MBS in the simulation area. Blue curves denote the CDF obtained using DDQN, red when using the greedy algorithm, and black when using the optimal exhaustive search. \begin{table}
\begin{tabular}{c c} \hline \hline
**Layer** & **Number of Neurons** \\ \hline Input layer & Size of state space \\ Middle layer & 512, densely connected \\ Output Layer & Size of action space \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Parameters relevant to the DDQN RL algorithm. \begin{table}
\begin{tabular}{l l} \hline \hline
**Parameter** & **Value** \\ \hline MBS and UE density & 8 or 4 per km\({}^{2}\) and 100 per km\({}^{2}\) \\ MBS and UABS transmit powers & 46 dBm and 30 dBm \\ Path-loss exponent & 4 \\ Altitude of UABSs & 121.92 m (400 feet) \\ Simulation area & 0.5 \(\times\) 0.5 km\({}^{2}\) \\ Range expansion bias in dB & 0 to 15 \\ Power reduction factor for MBS during (\(\alpha\)) & 0 to 1 \\ Duty cycle for the transmission of USF (\(\beta\)) & 0 to 1 \\ Scheduling threshold for UEs served by MBSs (\(\rho\)) & 35 dB or 45 dB \\ Scheduling threshold for UEs served by UABSs (\(\rho\)) & –20 dB to –5 dB \\ Downlink frequency & 763 MHz \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Simulation parameters. It can be seen that the DDQN algorithm performs better when the value of the exploration-exploitation trade-off (\(\epsilon\)) during evaluation is 0.5 (achieving 96.56% of the optimal median SE and 98.31% of the optimal mean SE), compared to when \(\epsilon\) is 0.05 (achieving 89.1% of the optimal mean SE and 82.85% of the optimal median SE). Using a higher \(\epsilon\) implies that the agent takes more random actions, compensating for under-training and helping it to come out of local minima. For 8 MBSs per km\({}^{2}\) (two MBSs in the simulation area), the CDF of median and mean SE achieved by the algorithms are shown in Figure 7a,b, respectively. Again, the DDQN performance is better when using \(\epsilon=0.5\) (achieving 93.46% of the optimal median SE and 95.83% of the optimal mean SE), than when using an \(\epsilon=0.05\) (achieving 79.73% of the optimal median SE and 82.89% of the optimal mean SE). The greedy algorithm achieves  
Figure 6: SE results for an MBS density of 4 MBS per km\({}^{2}\), which corresponds to one MBS in the simulation area. (**a**) CDF of median SE. DDQN achieves 96.56% of the optimal when \(\epsilon=0.5\) and 82.85% of the optimal when \(\epsilon=0.05\). (**b**) CDF of mean SE. DDQN achieves 98.31% of the optimal when \(\epsilon=0.5\) and 89.1% of the optimal when \(\epsilon=0.05\). the optimal results in the scenarios that we consider.