## 6 Simulation Experience and Verification
### Experimental Result  
This paper verifies SMADDPG, MADDPG and DDPG in the scenarios described in Section 6.1. In the above two scenarios with the fixed agent scale, the experiment process of SMADDPG and DDPG is shown in Figure 9. SMADDPG can better complete the tasks specified in the two task scenarios. Although the DDPG algorithm has an excellent learning effect in the field of a single agent, it fails to meet the task requirements in the scenario of multi-agent cooperation and competition. In the cooperative communication scenario, DDPG cannot deal with the communication between agents, so it cannot know which landmark is the target, and it can only move to the center of each landmark to maximize the reward. In the physical deception scenario, DDPG does not consider the overall reward of the agent cluster macroscopically, but greedily maximizes the individual reward, so agents will move to the target landmark, instead of achieving the task of decentralized coverage  
Figure 8: Task scenario adopted in the experience, (**a**) Cooperative Communication (**b**) Physical Deception.  
to deceive the other party. It can be seen that the SMADDPG algorithm can learn better task-planning methods than the DDPG algorithm in the cooperative competition scenario.  
Then, the quantitative analysis results of reward and success rate are shown in Figures 10 and 11 for the two situations of scale change and scale unchanged in the cooperative communication scenario. In the scenario of scale change, the number of UAVs changes with the following formula, where \(E\) is the number of current episodes; that is, the agent scale changes once every 300 episodes, and the agent scale is 6 and 12 in turn.  
\[N=6*\left[\frac{E\bmod 600}{300}\right]+6. \tag{21}\]  
It can be seen from Figure 10 that the SMADDPG algorithm can obtain a higher reward than MADDPG and DDPG when the agent scale is unchanged, and the fluctuation of the SMADDPG algorithm performance is relatively stable in the success rate. It can be seen from Figure 11 that SMADDPG can cope with this scenario. Although the reward and success rate fluctuate to a certain extent due to the change of scale, it can still maintain at a high level and learn better task-planning methods. To sum up, SMADDPG can achieve the best results in the fixed and changed scale cooperation scenarios.  
Figure 10: Two indicators are used to evaluate algorithms in the cooperative communication environment with six agents: (**a**) reward, (**b**) success rate.  
Figure 9: Comparison between SMADDPG (a) and DDPG (b) in the environment of cooperative communication (first row) and physical deception (second row) environment at start, middle and end.  
Then, in the physical deception scenario, SMADDPG and DDPG are used as the training algorithms of our side and the other side respectively. The success rates of both are shown in Table 3.  
The experimental results shown in Table 3 can explain the running effect of the SMADDPG and DDPG algorithms in Figure 9. When we use SMADDPG and the other uses DDPG, the difference between the two sides is the largest. This is because SMADDPG can cover each landmark in a decentralized way, making the other DDPG unable to know which landmark to go to, so the deception effect is the best at this time. To sum up, we can use the SMADDPG algorithm to achieve better decentralized coverage in the physical deception scenario, so as to cheat the other party.  
Therefore, through experiments, it can be concluded that the SMADDPG algorithm, compared with other reinforcement learning algorithms, is more able to obtain a better task-planning method from the overall reward value of the cluster, so as to better deal with cooperation and competition scenarios and improve the task execution efficiency of the UAV cluster.