###### Abstract  
In this paper, we propose a distributed multi-agent reinforcement learning (MARL) method to learn cooperative searching and tracking policies for multiple unmanned aerial vehicles (UAVs) with limited sensing range and communication ability. Firstly, we describe the system model for multi-UAV cooperative searching and tracking for moving targets and consider average observation rate and average exploration rate as the metrics. Moreover, we propose the information update and fusion mechanisms to enhance environment perception ability of the multi-UAV system. Then, the details of our method are demonstrated, including observation and action space representation, reward function design and training framework based on multi-agent proximal policy optimization (MAPPO). The simulation results have shown that our method has well convergence performance and outperforms other baseline algorithms in terms of average observation rate and average exploration rate.  
unmanned aerial vehicle (UAV); cooperative search and track; moving targets; information fusion; multi-agent reinforcement learning +
Footnote †: journal: Applied Sciences  
_Applied_  
_Acticle_  
Multi-UAV Cooperative Searching and Tracking for Moving Targets Based on Multi-Agent Reinforcement Learning  
_Applied_  
+++ ==WARNING: Truncated because of repetitions==
_  
++  
reinforcement learning (MARL) has also devoloped repaidly recently as an extension of reinforcement learning. It focuses on learning in environments with multiple agents that interact with each other. In MARL, the agents can have different objectives and may either collaborate or compete with each other. It is anticipated that MARL will be a key approach in resolving the problem of multi-UAV collaboration.  
This paper proposes a distributed MARL method to learn target searching and tracking policies of multi-UAV system. Firstly, we discribe the system model of multi-UAV cooperative searching and tracking for moving targets. Moreover, the information update and fusion mechanism is proposed to enhance environment perception ability of the UAVs. Then, the details of our method are demonstrated, including observation and action space representation, reward function design and training framework based on MAPPO. The main contributions of our paper can be summarized as follows:  
* The system model of multi-UAV cooperative searching and tracking for moving targets is constructed, which is extended from decentralized partially observable markov decision process. By optimizing both the average observation rate of moving targets and the average exploration rate of mission area, UAVs can maintain a constant observation over perceived targets and exploration for unknown environment;
* A novel information update and fusion mechanism is proposed to enhance environment perception ability of the multi-UAV system. In our model, each UAV keeps its individual cognitive information about the mission region to guide its action in a fully distributed decision-making approach. UAVs can achieve better understanding of the environment and better cooperatioin via information update and fusion;
* A distributed MARL method is proposed to learn cooperative searching and tracking policies for multi-UAV system.The reward function and observation space are newly designed, considering both target searching and region exploration. The method has also been proven effective through simulation analysis.  
The remainder of the paper is organized as follows. Section 2 introduces related works about target searching and MARL. Section 3 describes the mission model of multi-UAV cooperative searching for moving targets. Section 4 details the information update and fusion mechanisms for multi-UAV system. In Section 5, the proposed method based on MARL is demonstrated. In Section 6, simulation results and discussion are presented. Finally, Section 7 concludes the paper.