## 6 Simulation Results
### Performance Analysis  
Figure 4 indicates that the convergence performance of basic network architecture is better than recurrent network architecture. On one hand, RNN is too complex to the problem, which will lead to overfitting. On the other hand, the information update and fusion mechanism proposed in our method already preserves historical information, making the role of recurrent neural network redundant. We can also see that the convergence of MAPPO training algorithm is faster and more stable compared to IPPO. The MAPPO algorithm using CTDE architecture can effectively utilize global information to achieve better and more stable convergence in the training process. Different from MAPPO, as a fully decentralized method, IPPO algorithm directly applies a single-agent RL algorithm for  
Figure 4: Comparison of convergence performance of MAPPO and IPPO with different network architectures. (**a**) Average episode rewards. (**b**) Average observation rate \(\alpha\). (**c**) Average exploration rate \(\beta\). \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Number of steps to execute (\(E\)) & \(10^{5}\) \\ Batch size (\(\beta\)) & \(16\) \\ Learning rate (\(L_{r}\)) & \(5\times 10^{-4}\) \\ Discount factor (\(\gamma\)) & 0.99 \\ Clip factor (\(\epsilon\)) & 0.2 \\ Optimizer & Adam \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Hyperparameters used in training process. each individual agent without considering changes of other agents. Thus, the environment is non-stationary to the agents and convergence performance cannot be guaranteed. In this subsection, we use heatmap to visualize the cognitive information maps that incorporate all the detection information at different timesteps and analyze the effectiveness of our method. Furthermore, the performance of MAPPO with different sensing and communication capabilities is analyzed. The testing environment contains 5 UAVs and 7 targets in an area of 20 \(\times\) 20 cells. Figure 5 shows the target probability map and environment search status map at different timesteps. In target probability map, the degree of yellow in the cell is directly linked to the likelihood of the target's presence. In environment search status map, the proximity of the cell's color to dark red directly correlates with a decrease in the duration since the previous detection on the cell. Here we do not present the UAV location map but directly mark the locations of UAVs and targets on the other two maps. From Figure 5a,c, it can be seen that only two targets are under observation at timestep 30 while five targets have already been tracked at timestep 180. In target probability map, target existence probability of the cells under observation where targets are located is 1. Target existence probability of other cells that have been detected but do not contain targets approaches 0. And probability of undetected cells is 0.5, which means that no information is available about the presence of targets. The target probability map proposed in our paper contains information about distribution of targets, which can effectively guide UAVs to track and monitor the moving targets. In Figure 5b,d, we can see that multi-UAV system has only detected part of the mission region at timestep 30 while almost the entire area has been explored at timestep 180. The environment search status map contains search status information of the mission region. It can not only guide the UAVs to explore undetected areas but also promote revisiting of area that has not been explored for a while. Figure 5: Target probability map and environment search status map at different time steps, where pink pentacles represent targets and blue airplane icons represent UAVs. (**a**) Target probability map at timestep 30. (**b**) Environment search status map at timestep 30. (**c**) Target probability map at timestep 180. (**d**) Environment search status map at timestep 180.