# NOMA-Based Rate Optimization for Multi-UAV-Assisted D2D Communication Networks
## 5 Simulation Experiment and Result Analysis
### Analysis of Simulation Results  
In order to comprehensively evaluate the performance of the DH-MDQN algorithm proposed in this paper, the following four different benchmark algorithms are selected for comparison experiments:  
(1) Joint dynamic hypergraph Deep Q Network algorithm (DH-DQN): this algorithm uses a dynamic hypergraph to solve the subchannel assignment problem and a DQN algorithm to solve the trajectory planning and power control problem. \begin{table}
\begin{tabular}{l l} \hline \hline
**Simulation Parameters** & **Value** \\ \hline Plane area boundaries & \(x_{\min}=y_{\min}=0\) m, \(x_{\max}=y_{\max}=500\) m \\ UAV altitude range & \(h_{\min}=20m,\ h_{\max}=150\) m \\ Number of UAVs & \(U=3\) \\ Maximum UAV flight speed & \(V=5\) m/s \\ UAV maximum transmit power & \(P=29\) dBm \\ Maximum cellular user transmit power & \(P_{i}^{u}\)\({}_{\max}=23\) dBm \\ Number of cellular users & \(M=15\) \\ D2D cluster maximum transmit power & \(P_{i,D}^{u}\)\({}_{max}=20\) dBm \\ \hline Maximum spacing of D2D clusters & \(d_{i,DT_{i},D}^{u}\)\({}_{B_{i,\max}}=50\) m \\ Number of D2D clusters & \(N=30\) \\ Maximum number of D2D clusters & \(\xi_{u}=10\) \\ associated with a UAV & \(f_{c}=2\) GHz \\ Carrier frequency & \(B=15\) kHz \\ Bandwidth & \(\sigma=-100\) dBm/Hz \\ AWGN power & \(\xi=2\) \\ Path loss coefficient & \(\lambda_{c}=\lambda_{d}=18\) dBm \\ Threshold & 0.001 \\ Learning rate & 1 \\ Discount factor & 1 \\ Experience replay pool & 10,000 samples \\ Batch size & 128 samples \\ Optimizer & Adam \\ Greed coefficient & 0–0.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation parameter setting. (2) Joint graph-theoretic Multi-Agent Deep Q Network algorithm (G-MDQN): this algorithm employs graph theory to solve the subchannel assignment problem and the MDQN algorithm to solve the trajectory planning and power control problem. (3) Multi-Agent Deep Q Network algorithm (MDQN): this algorithm optimizes trajectory planning and power control using only the MDQN, while the subchannel allocation is based on a random assignment strategy. (4) Dynamic hypergraph algorithm (DH): the algorithm is based on the dynamic hypergraph to realize the optimization of subchannel allocation, and under the constraint of guaranteeing the basic performance of users, the power control is completed by using a fixed trajectory with random generation. Next in this study, the proposed DH-MDQN algorithm is simulated and validated in several dimensions. (1) Validation of the effectiveness of the DH-MDQN algorithm. In order to verify the effectiveness of the proposed algorithm, Figure 6 shows the effect of different learning rates on the DH-MDQN and DH-DQN algorithms. It is ensured that the algorithms achieve fast convergence and maintain high stability in the proposed NOMA-based multi-UAV-assisted D2D communication network environment. From the figure, it can be seen that different learning rates affect the learning performance and convergence efficiency of the two algorithms. The cumulative reward values of the algorithms at different learning rates grow slowly at the beginning of training due to the random initialization of the neural network parameters, a phenomenon that can be attributed to the high percentage of random actions and the fact that the replay memory buffer is not completely filled, which results in a model that has not yet entered into the effective training phase. Both DH-MDQN and DH-DQN show the best learning performance when the learning rate is 0.001. The DH-MDQN algorithm converges faster and stabilizes at about 360 episodes, while the DH-DQN algorithm converges with a relative lag and does not stabilize until close to 430 episodes. The main reason for this difference is that the improved mechanism of the DH-MDQN algorithm enables multiple UAVs to share the same neural network, which can adjust the network parameters faster and reach the optimal strategy in a shorter time.