# NOMA-Based Rate Optimization for Multi-UAV-Assisted D2D Communication Networks
## 5 Simulation Experiment and Result Analysis
### Analysis of Simulation Results
In contrast, when the learning rate is 0.1, the algorithm  
Figure 6: Convergence of DH-MDQN and DH-DQN algorithms with different learning rates. exhibits significant oscillations and instability, resulting in the reward value always remaining at a low level. This is because too high a learning rate will lead to too large a parameter adjustment step, and the neural network will be prone to deviate from the optimal solution or even fail to converge during the optimization process. In contrast, when the learning rate is 0.01, although the convergence stability of the model is improved, the too-large step size still leads to slower convergence, and the model performance fails to reach the optimal performance. Taken together, a learning rate of 0.001 can ensure the convergence speed of the model as well as stabilize the parameter adjustment process to maximize the reward value. Among them, the DH-MDQN algorithm outperforms DH-DQN, mainly due to its efficient utilization of empirical samples and stronger exploration ability, which enables it to achieve a higher cumulative reward value in a shorter training cycle. (2) Comparative performance analysis of DH-MDQN algorithm. Figure 7 shows the variation of system throughput of different algorithms (DH-MDQN, DH-DQN, G-MDQN, MDQN, and DH) as the number of training sets increases. From the results, it can be seen that the DH-MDQN algorithm performs the best, the throughput increases rapidly with the increase in the number of training sets and stabilizes after about 360 training sessions, and the final system throughput is about 42,000 Kb. Compared with the DH-DQN algorithm, the DH-MDQN algorithm significantly improves the system throughput. Under training stabilization, the DH-MDQN algorithm improves the system throughput by about 14% compared to the DH-DQN algorithm. In contrast, the system throughput of G-MDQN is lower than that of DH-MDQN, mainly due to the fact that the dynamic hypergraph is able to comprehensively model the cumulative disturbances of the users, which allows for a more fine-grained and flexible resource allocation. In addition, its dynamism permits real-time updating of the network structure to adapt to environmental changes and improve the efficiency and fairness of subchannel allocation. In contrast, traditional graph theory can only establish point-to-point binary relationships, which makes it difficult to capture the complex resource competition and collaboration among multiple users, and thus slightly lacks in allocation flexibility and efficiency. MDQN performs even lower, and its throughput, although improved at the beginning of the training period, has a low final convergence value. This is due to the fact  
Figure 7: Comparison of system throughput with different algorithms. that MDQN only considers power allocation and trajectory planning and does not cover the critical factor of subchannel allocation. Neglecting subchannel allocation leads to less efficient utilization of spectrum resources, which significantly limits the improvement of system throughput. The DH algorithm performs the worst, with its throughput barely improving throughout the training process. This is because DH only focuses on subchannel allocation without involving power control and trajectory planning optimization. Although the subchannel allocation can improve the spectrum utilization efficiency, in the absence of power control and trajectory planning, the overall resource scheduling capability of the system is severely limited and cannot adapt to the demands of the complex dynamic environment, which leads to the throughput always remaining at a low level. Figure 8 shows the trend of system throughput with the number of D2D clusters under different algorithms. It can be seen that the system throughput of all algorithms shows a growing trend as the number of D2D clusters gradually increases from 10 to 30, but the growth rate and the final performance vary depending on the algorithms.