## 2 Related Work
### Research Gaps and Our Approach  
Compared to heuristic algorithms, reinforcement learning offers significant advantages such as strong adaptability, consideration of long-term rewards, and model-free characteristics. Reinforcement learning can automatically adjust its strategy through continuous interaction with the environment, adapting to dynamic changes. It focuses on long-term rewards, enabling it to find optimal solutions in UAV perception and detection problems. Additionally, this method does not rely on an explicit environmental model, making it suitable for learning in unknown or complex situations.  
A detailed analysis of existing reinforcement learning approaches reveals several specific limitations in current research. The work in [24] proposed adaptive transition speed Q-learning, but it is constrained by Q-table size, which limits its scalability in complex environments. While [25; 28] introduced the multi-UAV control framework and multi-agent DRL framework, respectively, both rely solely on single-point position estimation or single UAV readings rather than utilizing multi-UAV triangulation for improved accuracy. The approach in [26] handles high-dimensional value functions for energy-efficient area coverage but is restricted to coverage tasks without closing the perception-localization loop. Although [27] implemented centralized training with distributed execution, it primarily focuses on reducing search overlap without leveraging multi-UAV cooperation for precise localization. The three-body cooperative defense guidance proposed in [29] lacks the incorporation of advanced intelligent optimization algorithms. These methods are summarized in Table 1. Our proposed MURPPO algorithm addresses these identified limitations through the following:  
* It establishes a distributed multi-UAV collaborative perception and localization architecture to facilitate efficient cooperative operations.
* It integrates a branched actor network architecture for enhanced computational efficiency while utilizing trilateration methods to exploit the spatial distribution of UAVs for precise localization.
* it uses sophisticated reward mechanisms to ensure robust closed-loop perception and detection.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Reference** & **Main Contributions** & **Key Limitations** & **Proposed Solutions** \\ \hline \multirow{4}{*}{[24]} & \(\bullet\) & Adaptive transition & \(\bullet\) & Limited by Q-table size & \(\bullet\) \\  & speed Q-learning & \(\bullet\) & Only for single UAV scenarios & \(\bullet\) \\  & \(\bullet\) & Two-stage strategy: rapid search and optimal planning & \(\bullet\) & Limited state-action space representation & \(\bullet\) \\ \hline \multirow{4}{*}{[25]} & \(\bullet\) & Multi-UAV control framework & \(\bullet\) & Relies solely on single-point position estimation & \(\bullet\) \\  & & & & \\ \hline \multirow{4}{*}{[26]} & \(\bullet\) & Handles high dimensional value functions & \(\bullet\) & The perception and localization loop has not been closed & \(\bullet\) \\ \cline{1-1}  & & & & \\ \cline{1-1}  & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of different reinforcement learning approaches.