## 5 Experiments and Results
### Ablation Study  
#### 5.4.1 Ablation Tests About GAS and CGP  
In order to determine the effectiveness of the GAS and CGP loss, we conduct ablation tests of the two components in (a) 6m2m_15m and (b) 8m3m_21m. For value-based methods, QMIX is the baseline algorithm, while \"QMIX + GAS\" and \"QMIX + CGP\" are the ablation groups. Similarly, for policy-based methods, MAPPO is the baseline algorithm, while \"MAPPO + GAS\" and \"MAPPO + CGP\" are the ablation groups. The results are shown in Figures 8 and 9, respectively. In general, the introduction of GAS can improve the performance and reduce the variance, and the performance of CGP loss is unstable. In Figure 8a, QMIX + GAS significantly outperforms QMIX with higher _WR_ and lower variance, indicating the effectiveness of distinguishing different action semantics with GAS. QMIX + CGP can still outperform QMIX in the last 1 M training steps; however, it suffers from high variance. In Figure 8b, however, QMIX + CGP acquires higher _WR_ than QMIX + GAS. In Figure 9a, the performances of MAPPO + GAS and MAPPO + CGP are similar to the performances in Figure 8a. In Figure 9b, the performances of MAPPO + GAS and MAPPO + CGP are similar in the first 5M training steps, but MAPPO + GAS then outperforms MAPPO + CGP and reaches a similar _WR_ to the G-MAPPO. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Map Name** & **QMIX** & **GHQ** & **CDS** & **ASN** & **MAPPO** & **HAPPO** & **HetGPPO** \\ \hline MMM2 & 1.3\({}_{(0.254)}\) & 1.3\({}_{(0.262)}\) & 18.4\({}_{(0.000)}\) & 3.6\({}_{(0.071)}\) & 10.2\({}_{(0.003)}\) & 311.4\({}_{(0.000)}\) & 6.3\({}_{(0.017)}\) \\
6m2m\_15m & 11.9\({}_{(0.002)}\) & 6.3\({}_{(0.019)}\) & 29.1\({}_{(0.000)}\) & 7.1\({}_{(0.014)}\) & 25.3\({}_{(0.000)}\) & 88.6\({}_{(0.000)}\) & 46.6\({}_{(0.000)}\) \\
6m2m\_16m & 9.1\({}_{(0.006)}\) & 5.3\({}_{(0.030)}\) & 103.1\({}_{(0.000)}\) & 28.4\({}_{(0.000)}\) & 32.1\({}_{(0.000)}\) & 801.4\({}_{(0.000)}\) & 178.7\({}_{(0.000)}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Results of One-way ANOVA tests of G-QMIX and G-MAPPO against other algorithms. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Items** & **G-MAPPO** & **MAPPO** & **G-QMIX** & **QMIX** \\ \hline Computing Time (h) & 20 & 15 & 14 & 8 \\ Converged Time-step (\%) & 0.7 & 0.9 & 0.6 & 0.4 \\ Converged Time (h) & 14 & 13.5 & 8.4 & 3.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Analysis of computational cost of G-QMIX and G-MAPPO in MMM2. One possible reason for the instability introduced by the usage of CGP loss only is that the two diverse optimizing objects of the CGP loss against the original MARL loss weaken the stability of the learning process. In addition, the effectiveness of cross-group prediction requires the high cooperation of the scenario, and thus may be unstable across different maps. Using GAS makes the backward propagation more accurate and therefore improves the _WR_. The combination of the GAS and CGP loss brings a larger network capacity and thus leads to the best performance of G-QMIX and G-MAPPO. In addition, for map (b) with larger scale and heterogeneity, the usage of CGP loss brings a greater increase in the _WR_ in contrast with the \"GAS-only\" method, indicating the necessity of improving the correlation between different groups in complex heterogeneous MARL problems. 5.4.2 Ablation Study on the Hyper-Parameters \(\lambda_{I}\), \(\epsilon_{p}\), and \(\epsilon_{v}\)  
In order to determine the optimal values of \(\lambda_{I}\), \(\epsilon_{p}\), and \(\epsilon_{v}\), we conduct several groups of pre-experiments. The results of \(\lambda_{I}\) in G-MAPPO tests are summarized in Table 9, the results of \(\lambda_{I}\) in G-QMIX tests are summarized in Table 10, the results of \(\epsilon_{p}\) in G-MAPPO tests are summarized in Table 11, and the results of \(\epsilon_{v}\) in G-MAPPO tests are summarized in Table 12. The values of hyper-parameters are in the brackets after the name of the methods and the optimal groups are in bold font.