# UAV-Cooperative Penetration Dynamic-Tracking Interceptor Method Based on DDPG  
[PERSON]  
1 School of Astronautics, Beihang University (BUAA), Beijing 100191, China; [EMAIL_ADDRESS] (Y.L. ); [EMAIL_ADDRESS] (K.Z.) [PERSON]  
1 School of Astronautics, Beihang University (BUAA), Beijing 100191, China; [EMAIL_ADDRESS] (Y.L. ); [EMAIL_ADDRESS] (K.Z.) [PERSON]  
1 School of Astronautics, Beihang University (BUAA), Beijing 100191, China; [EMAIL_ADDRESS] 10000-0001  
Multi-UAV-cooperative penetration generally requires that multiple UAVs achieve time coordination to penetrate defenses according to different trajectories and finally reach the target area at the same time or according to a certain time sequence. When the UAVs depart from different locations, it will greatly increase the difficulty of collaboration. The multi-UAV-coordination algorithm can be improved based on the traditional single-UAV-penetration algorithm adapted to the multi-UAV environment. [PERSON] uses the optimal-control method to improve the artificial potential-field method to achieve multi-UAV coordination [11; 12], and [PERSON] improves the RRT algorithm to achieve multi-UAV coordination [13]. At the same time, there are a large number of methods for cooperative control of UAVs based on the graph theory [3]. [PERSON] proposed a multi-UAV-collaboration method based on the graph theory [14]. [PERSON] proposed a multi-UAV-coordination method based on multi objective optimization [15]. The above methods all realize the coordination of multiple UAVs, but their algorithms lack the research on dynamic environments and cannot adapt to the complex and dynamic battlefield environment. Aimed at the environment with dynamic threats, this paper proposes a method based on deep reinforcement learning to achieve multi-UAV-cooperative penetration. Reinforcement learning is an important branch of machine learning. Its main feature is to evaluate the action policy of the agent based on the final rewards and through the interaction and trial and error between the agent and the environment. Reinforcement learning is a far-sighted machine-learning method that considers long-term rewards [16; 17]. It is often used to solve sequential decision-making problems. Reinforcement learning can not only be applied in a static environment but also when the parameters of the environment are constantly changing, and the agent can also be applied in a dynamic environment [16; 17; 18]. The research of reinforcement learning is mostly concentrated in the field of a single agent, but there is also a large body of research on reinforcement-learning algorithms for multiagent systems. There is also much research on applying reinforcement learning to UAVs. [PERSON] successfully applied deep reinforcement learning to UAVs to realize autonomous navigation of UAVs in large, unknown, and complex environments based on deep reinforcement learning [20]. [PERSON] applied reinforcement learning to the target search and tracking of UAVs [21]. Based on deep reinforcement learning, [PERSON] studied the task scheduling problem of UAV clusters and solved the application problem of reinforcement learning in multi-UAV systems [22]. Through the investigation of relevant literature, it can be found that the application of deep reinforcement learning to multi-UAV systems is a feasible method, which can be used to achieve complex multi-UAV-system tasks and has great research potential. The main work of this paper uses the multi-UAV-cooperative penetration dynamic-tracking interceptor as the scenario. Based on the deep reinforcement-learning DDPG algorithm, we establish the intelligent UAV model and realize the multi-UAV-cooperative penetration dynamic-tracking interceptor by designing the reward function related to coordination and penetration.