## 7 Summary and Conclusions  
This paper proposes a deep multi-agent reinforcement-learning-based task-planning algorithm using the Seq2 Seq for multi-UAV when there are fixed and dynamic scales. Since the POMDP under consideration is a high-dimensional problem with a continuous action space, DDPG is employed for reinforcement learning. In order to use DDPG for the task-planning problem under consideration, POMDP is defined appropriately first. Then, for the dynamic change of scale, we transplant Seq2 Seq, which is commonly used in the field of natural language processing, to the multi-agent reinforcement learning field to learning the mapping relationship between global observation and personality action that changes with the number of agents. Later, in view of the successful application experience  
Figure 11: Two indicators are used to evaluate algorithms in the cooperative communication environment with N agents: (**a**) reward, (**b**) success rate.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & \multicolumn{2}{c}{\(N=6\)} & \multicolumn{2}{c}{\(N=12\)} \\
**Agent \(\pi\)** & **Adversary \(\pi\)** & **AG succ\%** & **ADV succ\%** & **AG succ\%** & **ADV succ\%** \\ \hline SMADDPG & DDPG & **85.2** & **23.4** & **81.3** & **19.9** \\ SMADDPG & SMADDPG & 70.1 & 14.8 & 68.4 & 11.6 \\ DDPG & DDPG & 15.6 & 34.7 & 12.9 & 30.2 \\ DDPG & SMADDPG & 30.7 & 27.5 & 25.8 & 22.7 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results on the physical deception task, with \(N=6\), 12 cooperative agents.  
of DDPG in single-UAV path planning, we use DDPG to build a local network, input local observations of agents, and output common actions that determine the flight direction and speed of agents. Finally, we add the personality action and common action to the simulation environment for the next update. Later, we verify our algorithm in the competition and cooperation scenarios, and the results are shown in Figures 10 and 11, all of which achieved good results. This also proves that our algorithm can achieve better task-planning results than other reinforcement learning algorithms in the environment of dynamic scale change.  
During the experiment, we noticed that the dimension of the global observation in the 3D environment increased significantly, so the training time was increased. We also observed that when there were too many agents, the update of the agent strategy was more relevant to the neighboring agents. To reduce the training time of multi-agent reinforcement learning, we will later introduce the concept of the graph in reinforcement learning. Each agent updates the policy according to the state of its adjacent agent, which can reduce the input dimension of the central network and the training time of the multi-agent.  
At the same time, in the collaborative confrontation environment we studied, we noticed that the dynamics were not only reflected in the dynamic changes of the agent scale but also included the behavior changes of unknown agents. Therefore, our subsequent research hopes to introduce long short-term memory (LSTM) to predict unknown agents, which can help us make correct task-planning methods in advance.  
The authors jointly proposed the idea and contributed equally to the writing of manuscript. Writing: Z.L.; Coding: Z.Z. and C.Q.; supervised the research and revised the manuscript C.Q. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Not applicable.  
The authors declare no conflict of interest.