## 5 Experiments and Results
### Validating Tests  
#### 5.5.1 Validating Tests in the MPE Environment  
First of all, we must point out two important differences between the MPE and the SMAC environments. (1) _The action spaces of different agent groups consist of the **same** action semantics._ As shown in Figure 4 and Section 5.1, agents are partitioned into different groups only because of their distinct goals. Therefore, the heterogeneity of the MPE scenario differs from the _physical_ heterogeneity that we have focused on in this paper. Such a difference strongly affects the effectiveness of the GAS and CGP methods. (2) _The MPE scenario is not a **strongly cooperative** environment._ Although the global optimal _return_ is achieved only when all agents reduce their distances to goals synchronously, \begin{table}
\begin{tabular}{c c c c c c} \hline
**Map Name** & **G-QMIX (0.02)** & **G-QMIX (0.04)** & **G-QMIX (0.06)** & **G-QMIX (0.08)** & **G-QMIX (0.1)** \\ \hline MMM2 & \(0.78_{(0.10)}\) & \(0.88_{(0.08)}\) & \(\mathbf{0.99}_{(0.03)}\) & \(0.89_{(0.05)}\) & \(0.80_{(0.35)}\) \\
6m2m\_15m & \(0.72_{(0.11)}\) & \(0.78_{(0.15)}\) & \(\mathbf{0.88}_{(0.13)}\) & \(0.83_{(0.14)}\) & \(0.79_{(0.12)}\) \\
6m2m\_16m & \(0.49_{(0.35)}\) & \(0.52_{(0.25)}\) & \(\mathbf{0.58}_{(0.14)}\) & \(0.55_{(0.27)}\) & \(0.37_{(0.28)}\) \\ \hline \end{tabular}
\end{table} 
Table 10: Results of ablation study on the hyper-parameter \(\lambda_{I}\) in G-QMIX. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Map Name** & **G-MAPPO (0.1)** & **G-MAPPO (0.2)** & **G-MAPPO (0.3)** & **G-MAPPO (0.4)** & **G-MAPPO (0.5)** \\ \hline MMM2 & \(0.79_{(0.08)}\) & \(\mathbf{0.85}_{(0.09)}\) & \(0.83_{(0.14)}\) & \(0.80_{(0.17)}\) & \(0.78_{(0.20)}\) \\
6m2m\_15m & \(0.77_{(0.10)}\) & \(\mathbf{0.85}_{(0.09)}\) & \(0.82_{(0.14)}\) & \(0.78_{(0.20)}\) & \(0.73_{(0.29)}\) \\
6m2m\_16m & \(0.74_{(0.33)}\) & \(\mathbf{0.85}_{(0.09)}\) & \(0.78_{(0.18)}\) & \(0.70_{(0.25)}\) & \(0.64_{(0.43)}\) \\ \hline \end{tabular}
\end{table} 
Table 11: Results of ablation study on the hyper-parameter \(\epsilon_{p}\) in G-MAPPO. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Map Name** & **G-MAPPO (0.1)** & **G-MAPPO (0.2)** & **G-MAPPO (0.3)** & **G-MAPPO (0.4)** & **G-MAPPO (0.5)** \\ \hline MMM2 & \(0.75_{(0.08)}\) & \(\mathbf{0.85}_{(0.09)}\) & \(0.84_{(0.11)}\) & \(0.80_{(0.15)}\) & \(0.80_{(0.14)}\) \\
6m2m\_15m & \(0.72_{(0.06)}\) & \(\mathbf{0.85}_{(0.09)}\) & \(0.83_{(0.12)}\) & \(0.80_{(0.17)}\) & \(0.77_{(0.24)}\) \\
6m2m\_16m & \(0.70_{(0.06)}\) & \(\mathbf{0.85}_{(0.09)}\) & \(0.80_{(0.14)}\) & \(0.73_{(0.23)}\) & \(0.60_{(0.50)}\) \\ \hline \end{tabular}
\end{table} 
Table 12: Results of ablation study on the hyper-parameter \(\epsilon_{v}\) in G-MAPPO. the interactions between different agents or groups are missing. In other words, all agents are able to perform the optimal policy without any help from other agents. In SMAC, heterogeneous agents must fully utilize their different characteristics and help each other to win the game. This property of the MPE scenario strongly affects the necessity of CGP loss and further affects the overall performance of our methods. As a result, we hereby present the test result as an additional experiment only to show the generalizability and limitations of our methods. In Figures 10a and 11a, we see the outstanding performance of G-QMIX and G-MAPPO, which not only acquire the optimal _return_ but also very small variance. When the scale of the scenario is small, the CGP loss is capable of enhancing the mutual information of different groups and thus brings better cooperation. In Figures 10b and 11b, as the scale of the scenario doubles, the aforementioned problems show their impacts. The usage of CGP loss restricts the diversity of agent policies, which dramatically influences the learning process in a stochastic environment and thus leads to the \"wave\" of the red curve in Figure 10b. The policy-based methods could not perform well in large-scale discrete environments, as shown in Figure 11b.