## 6 Simulation Results
### Experimental Results and Analysis  
In this section, we present the performance of the previously proposed MURPPO algorithm based on branch training. All results were recorded via Tensorboard during the training process. The reward curves for three red team UAVs are illustrated in Figure 6, with the x-axis representing the number of training episodes and the y-axis showing the corresponding reward values. Each episode consists of 100 time steps, and the reward values represent the cumulative rewards obtained by UAVs through multiple mechanisms including perception rewards, first detection rewards and penalties, as well as additional localization rewards when three-point localization is achieved. Figure 5(a)-c individually illustrate the reward progression for UAVs 1, 2, and 3, reflecting each UAV's independent learning performance. The learning process shows two distinct phases, a rapid improvement phase in the first 200 episodes where the agents quickly learn effective strategies, followed by a stabilization phase where the rewards fluctuate around converged values. UAVs 1 and 2 converge to approximately 70 and 80, respectively, while UAV 3 exhibits superior performance with more rapid convergence. Figure 5(d) displays the aggregate rewards for all three UAVs, which ultimately stabilize around 250. The overall trend indicates that, notwithstanding intermittent fluctuations, the system effectively enhances task execution capabilities through multi-UAV collaboration, resulting in successful policy convergence. Overall, the red team's cooperative mechanism is effective, enabling them to gradually gain a competitive edge in the complex electromagnetic environment. This validates the effectiveness of multi-UAV cooperation and the algorithmic improvements. The system demonstrates its ability to learn and adapt in a challenging, adversarial setting. Figure 7 demonstrates the superior performance of our proposed MURPPO algorithm compared to baseline methods (MAA2C and MAPPO) over 1000 training episodes. The horizontal axis represents the number of training episodes, while the vertical axis indicates the localization rate. From an overall performance perspective, MURPPO demonstrated the highest localization rate throughout most of the training phases, followed by  
Figure 6: Rewards obtained by red team UAVs over 1000 training episodes: (**a**) rewards obtained by UAV 1; (**b**) rewards obtained by UAV 2; (**c**) rewards obtained by UAV 3; (**d**) total reward. MAPPO, while MAA2C's performance was comparatively subpar. During the initial training period (approximately episodes 0-200), all three algorithms exhibited an upward trend, with MURPPO showing the fastest convergence rate, surpassing both MAPPO and MAA2C. As training progressed, the performance disparities among the algorithms became increasingly apparent. In the later stages of training (after approximately 700 episodes), MURPPO's advantages became more pronounced, not only maintaining a high success rate but also exhibiting exceptional stability. Although MAPPO's performance experienced some decline, it generally remained superior to MAA2C. Notably, MURPPO's learning curve was the most consistent, with minimal fluctuations. In contrast, MAPPO's performance fluctuated considerably, especially during the middle and later stages. While MAA2C exhibited relatively small fluctuations, its overall performance consistently remained lower. In the final 300 episodes of training, MURPPO achieved an average localization rate of 56.5%. This was approximately 38.5% higher than MAPPO's and 41.6% higher than MAA2C's. In summary, the MURPPO algorithm surpasses the benchmark algorithms in key metrics such as localization rate, stability, and convergence speed. Figure 8 presents a novel analysis of policy convergence stability through KL divergence tracking for each UAV independently, offering unique insights into the training dynamics of multi-agent systems in electromagnetic environments.