## 4 Solutions
### Algorithm Structure
The convergence conditions of training for different numbers of UAVs are depicted in Figure 5. It is evident from the figure that, at the beginning of training, the initial cumulative reward is relatively low because the drone has not yet learned the appropriate trajectory in the dynamic environment to cover the UEs. However, over time, the cumulative reward rapidly increases as the drone continues to learn. In addition, due to the nonstationary nature of

Figure 4: Impact of the number of hidden layers. the environment, the rewards fluctuate around the average value. However, as training proceeds, the trend of cumulative rewards continues to increase until convergence. ### Analysis of the Communication QoS Threshold \(SR_{th}\)

We want to balance the relationship between communication coverage and communication QoS. To this end, we select two signal to interference and noise ratio thresholds, \(SR_{th}^{low}=0\) and \(SR_{th}^{high}=5\). The numerical results of these two thresholds in different environments are depicted in Figure 6. As the number of drones increases, the number of UEs that can meet the high SR threshold decreases. When the number of drones reaches five or six, the number of UEs that meet the high QoS service reduces to zero. However, for low SR threshold requirements, each drone can serve the maximum number of users it is capable of serving. In the scenarios tested with different numbers of drones, all the drones are able to meet the low QoS threshold requirements of the UEs. Therefore, in the subsequent experiments, we select a low SR threshold parameter by default. Figure 5: Convergence conditions over time. ### Analysis of Different Drone Numbers

We test the performance of different numbers of drones in dynamic scenarios. As evident from Figure 6(a), when the number of drones reaches five, the drone network can achieve full coverage of the UEs in the scene. However, when the number of drones is six, the system coverage is reduced due to communication interference between the drones, thereby resulting in a corresponding reduction in reward, as depicted in Figure 6(b). It is also worth noting that, as the number of drones increases, the average energy consumption of the entire scene rises sharply. Figure 6(c) takes the energy consumption of the system when the number of drones is three as the benchmark and compares it to the energy consumption when testing other numbers of drones. Obviously, when the number of drones doubles to six, the energy consumed by the drone system also doubles. In conclusion, the number of drones that achieve the maximum cost-effectiveness for our system is five. Figure 6: Impact of signal to interference and noise ratio thresholds. Figure 7: _Cont_. ### Performance Comparisons with Other Methods

General planning or optimization methods are typically designed to address static problems and are not inherently suited for dynamic scenarios. Our problem, however, is dynamic and represents a multi-agent Markov decision process (MDP) that requires continuous real-time decision-making. This complexity is further compounded by the extensive state and action spaces involved. The traditional optimization algorithms generally fail to manage this complexity effectively. Consequently, we cannot demonstrate the efficacy of our method by comparing it with traditional optimization algorithms. To evaluate the advantage of drone mobility in communication and the effectiveness of our mobility method, we conduct comparisons with three other drone configuration methods: (1) suspended in a fixed position (see Figure 7(a))--drones hover at a fixed distance in the middle of the area; (2) move randomly (see Figure 7(b))--the drones move randomly within the area, with the maximum moving distance in each movement interval not exceeding 300 m; and (3) move at a constant speed (see Figure 7(c))--throughout the entire period, the drone follows the same search route as the rescuers, moving at a constant speed from one side of the area to the other.