## 6 Simulation Results
### Comparison with Other Methods  
This subsection compares MAPPO with A-CMOMT, ACO and Random algorithms to examine the effectiveness of our method. The results of the comparison were obtained from scenarios of different scales: 20 \(\times\) 20 cells with 10 targets, 30 \(\times\) 30 cells with 15 targets and 40 \(\times\) 40 cells with 20 targets. Each data point is an average of 10 tests. Moreover, the performance results of MAPPO are obtained using the trained model with 5 UAVs, 7 targets and 20 \(\times\) 20 cells. An introduction to the comparison algorithm is as follows:  
* **A-CMOMIT**: A-CMOMT [16] is a traditional method to solve the target searching and tracking problem. In this method, control of multiple agents is based on a combination of force vectors that are attractive for nearby targets and repulsive for nearby agents;
* **ACO:** As a swarm intelligence algorithm, ant colony optimization (ACO) is also applied in the target searching and tracking problem [21; 43]. The pheromone used to initialize all the cells in this approach is identical. The pheromone of cells encountered by UAVs will undergo vaporization at every time step. The pheromone map, target existence probability and UAVs' locations will be included in the heuristic information to guide UAVs' decisions;
* **Random:** At each timestep, the agents randomly select an action from the potential candidates. The performance comparison of MAPPO, A-CMOMT, ACO and Random algorithms under different number of UAVs from scenarios of different scales is depicted in Figure 7. Figure 7a-c show the results of the average observation rate \(\alpha\) versus the UAVs' quantity for different scenarios. We can see that with the increase in the UAVs' quantity, the average  
Figure 6: Comparison of results with and without communication when the sensing range is increasing. (**a**) Average episode rewards with and without communication. (**b**) Average observation rate \(\alpha\) with and without communication. (**c**) Average exploration rate \(\beta\) with and without communication. observation rate \(\alpha\) increases across all algorithms and the performance of MAPPO, A-CMOMMT and ACO far surpass that of Random. We can see that our MAPPO algorithm has the highest average observation rate \(\alpha\) in most cases. A-CMOMMT outperforms ACO by a small margin in the environment with 10 targets and 20 \(\times\) 20 cells and 15 targets and 30 \(\times\) 30 cells. But ACO performs better in the environment with 20 targets and 40 \(\times\) 40 cells. ACO has better performance when the scale of the problem is large, but the difference between A-CMOMMT and ACO is not much overall. Figure 7d-f display the correlation between the average exploration rate \(\beta\) and the quantity of UAVs. in different scenarios. We can see that the Random algorithm has the highest average exploration rate \(\beta\) and the performance of MAPPO is second only to Random. And we can find that the average exploration rate \(\beta\) of A-CMOMMT and ACO is far inferior to that of Random and MAPPO. The average exploration rate \(\beta\) in our paper is used to examine the effectiveness of algorithms in exploring the mission region. And a higher value of the indicator also means that the observation of all targets is done in a relatively uniform manner. The standard deviation of observation rate across all targets in the environment with 10 targets and 20 \(\times\) 20 cells is listed in Table 4. As shown in Table 4, with better exploring performance, MAPPO can observe the targets relatively uniformly. In summary, the performance comparison illustrates that the proposed method MAPPO performs better in target searching and tracking problem than the traditional algorithm A-CMOMMT and swarm intelligence algorithm ACO. Figure 7: Performance comparison of MAPPO, A-CMOMMT, ACO and Random algorithms under different number of UAVs. (**a**) Average observation rate \(\alpha\) in the environment with 10 targets and 20 \(\times\) 20 cells.