###### Abstract  
Task planning involving multiple unmanned aerial vehicles (UAVs) is one of the main research topics in the field of cooperative unmanned aerial vehicle control systems. This is a complex optimization problem where task allocation and path planning are dealt with separately. However, the recalculation of optimal results is too slow for real-time operations in dynamic environments due to a large amount of computation required, and traditional algorithms are difficult to handle scenarios of varying scales. Meanwhile, the traditional approach confines task planning to a 2D environment, which deviates from the real world. In this paper, we design a 3D dynamic environment and propose a method for task planning based on sequence-to-sequence multi-agent deep deterministic policy gradient (SMADDPG) algorithm. First, we construct the task-planning problem as a multi-agent system based on the Markov decision process. Then, the DDPG is combined sequence-to-sequence to learn the system to solve task assignment and path planning simultaneously according to the corresponding reward function. We compare our approach with the traditional reinforcement learning algorithm in this system. The simulation results show that our approach satisfies the task-planning requirements and can accomplish tasks more efficiently in competitive as well as cooperative scenarios with dynamic or constant scales. s +
Footnote †: journal: Applied sciences  
There are many solutions to the path-planning problem, but common solutions include clustering-based algorithm [22], Voronoi diagrams with the Dijkstra algorithm [23], adaptive random search algorithm [24], model predictive control [25], mixed-integer linear programming [26; 27], etc. These algorithms cannot effectively deal with dynamic environments because they need global information about the environment to calculate the optimal result. However, real environments change rapidly. For example, the target's position may change randomly. Once the environment changes, the original results fail. In addition, these algorithms need many calculations to achieve optimal results after the environment changes, and the calculation process is too slow for real-time operation [28]. Meanwhile, these methods only simulate in 2D environments and do not take into account the fact that this simulation environment differs significantly from the real world. These shortcomings limit the application of such algorithms in UAV clusters in real, dynamic environments. With the rise of reinforcement learning technology, reinforcement learning technology has attracted the attention of researchers and also provides a new method for task planning. [PERSON] et al. [29] applied the deep deterministic policy gradient (DDPG) to single UAV path planning and achieved good results. However, there are frequent interactions between UAVs in the multi-UAV collaboration scene, so it is impossible to obtain better results using this completely independent algorithm. At the same time, considering the dynamic change of scale that may exist in the process of multi-UAV flight, and the insensitivity of the sequence-to-sequence (Seq2 Seq) network [30; 31] to the input and output length, this paper combines DDPG [32] with Seq2 Seq, uses DDPG to build a local network, inputs local observations of all agents, and outputs common actions of agents unrelated to the dynamic change of agent scale. Seq2 Seq is used to build a global network, inputs the state containing complex interactions between agents, and outputs the personalized actions of agents related to the scale of agents. Based on the above ideas, the contribution of this study is as follows. First of all, a 3D dynamic environment was created especially for task planning using UAV movement. Using a reward function, the environment evaluates the UAV's motion for task planning in the unknown space. In the same way, it denounces the UAV for negative occasions, such as obstacle collisions.