# Multiagent Hierarchical Cognition Difference Policy for Multiagent Cooperation
## 5 Simulation Results and Analysis
### Group Containment  
For the group containment, there are \(n\) agents and \(m\) landmarks in the environment as shown in Figure 6b. The relationship between \(n\) and \(m\) is defined as \(\frac{n}{m}=k,\ k\in\mathbb{Z}\) where \(\mathbb{Z}\) represents a set of positive integers. Based on the above restriction, two scenarios including 6 agents with 2 landmarks and 10 agents with 2 landmarks are designed to evaluate the performance of our scheme. In these scenarios, all the agents must be divided into two groups and distributed evenly around the two landmarks without collision. The compound reward function for this task is defined as follows:  
\[ \begin{array}{lcl}R^{i}_{total}&=&R^{i}_{dist}+R^{i}_{collision}+R_{cross}\\ R^{i}_{dist}&=&-Hungarian\left(p^{t}_{i},p^{g}_{i}\right)\\ R^{i}_{collision}&=&\left\{\begin{array}{l}0\ \ \ if\ dist(p^{t}_{i},p^{t}_{j})>r_{i}+r_{j}\\ -1\ \ else\end{array}\right.\\ \end{array}  \tag{27}\]  
Figure 7: Simulation result in cooperative navigation with six agents. where \(p_{i}^{t}\) is the position of agent \(i\) and \(p_{i}^{S}\) is the position of goal. _Hungarian_ represents the Hungarian algorithm, which is a combination optimization algorithm. The Hungarian algorithm is adopted to calculate the mean distance between the agents and the goal positions. With the reward setting, the distance reward for each agent is related to the other agents. In this paper, there are two different scenarios with different number of agents for the group containment tasks. Specifically, these scenarios include scenario (a) with 6 agents and scenario (b) with 10 agents. MA-HCDP is compared with MADDPG and TRANSFER in these scenarios. The simulation results of MA-HCDP and TRANSFER are presented in Table 3. MA-HCDP obtain 12 optimal measurements during four scenarios and outperforms than TRANSFER and MADDPG in terms of steps and rewards. In this kind of competitive scenario, where there is a high demand for communication effectiveness, CDN in MA-HCDP can filter out unrelated agents, and SCN can process the information of filtered agents with attention weights to promote cooperation. To further demonstrate the effectiveness of our method, we present the attention value distribution of agents in Figure 8. Take agent 2 and agent 5 in Figure 8a as an example. As shown in Figure 8, we can obtain the attention value distribution for agent 2 (Figure 8c) and agent 5 (Figure 8d). Note that agent 2 only communicates with agent 1 and agent 3, and agents 5 communicates with agent 6. The reason why agents 2 and 5 do not communicate is that agent 2 needs to go around landmark 1, while agent 5 needs to go around landmark 2. They have different perceptions of the environment and goals, so there is no need for communication. Based on the communication status, the attention coefficients are only assigned to those agents communicating with agent 2 and agent 5. Moreover, since different agents have different influences, they are assigned different attention coefficients. It demonstrates that MA-HCDP can filter out unrelated agents and process the information of different agents to promote cooperation.