## 5 Experiments and Results
### Comparison Results
#### 5.3.1 Results of Comparison of Value-Based Algorithms  
(1) _Easy: We test all algorithms on the original asymmetric heterogeneous map MMM2_. The results are shown in Figure 5a. G-QMIX and RODE converge to 1.0 at about 3M steps. The _WR_ curve of RODE increases faster than G-QMIX before 2 M steps but suffers from high variance. QMIX and GHQ converge to about 0.8 at 2 M steps. CDS also converges to 0.8 but requires 5M training steps. The _WR_ curve of ASN and UNMAS can only converge to 0.5 with high variance. ROMA is only capable of acquiring 0.1 at 5M training steps. (2) _Medium: We scale up ally units and increase enemy units for balancing_. In Figure 5b-e, the number of \"Marines + Medivacs\" increases from \"6 + 2\" to \"8 + 3\" and \"8 + 4\", while the number of enemy Marines increases from 15 to 23. As concluded in [27], the power of 1 Medivac is roughly equal to 3.5 Marines. Therefore, the duels in these four maps are basically fair and the total amount of entities in the maps is also not too high. Comparing the results in Figure 5a,b, it is easy to figure out that the difficulty of these two maps is similar, since all algorithms can acquire similar performance on these two maps. In Figure 5b, G-QMIX reaches the best _WR_ of 0.9, and RODE reaches the second best _WR_ of 0.8, with a lower rate of increase QMIX, ASN, UNMAS, and CDS end with about 0.6, 0.45, 0.4, and 0.3, respectively. In the first 2M steps, GHQ grows faster than all other \begin{table}
\begin{tabular}{c c c c c c c c c} \hline
**Map Name** & **G-QMIX** & **QMIX** & **GHQ** & **ROMA** & **RODE** & **CDS** & **ASN** & **UNMAS** \\ \hline MMM2 & \(\mathbf{0.99}_{(0.03)}\) & \(0.85_{(0.35)}\) & \(0.85_{(0.35)}\) & \(0.09_{(0.18)}\) & \(\mathbf{1.00}_{(0.01)}\) & \(0.85_{(0.06)}\) & \(0.61_{(0.44)}\) & \(0.62_{(0.44)}\) \\
6m2m\_15m & \(\mathbf{0.88}_{(0.13)}\) & \(0.68_{(0.31)}\) & \(0.74_{(0.27)}\) & \(0.18_{(0.21)}\) & \(0.86_{(0.09)}\) & \(0.30_{(0.12)}\) & \(0.46_{(0.48)}\) & \(0.41_{(0.40)}\) \\
6m2m\_16m & \(\mathbf{0.58}_{(0.14)}\) & \(0.33_{(0.23)}\) & \(0.55_{(0.22)}\) & \(0.02_{(0.03)}\) & \(0.18_{(0.20)}\) & \(0.08_{(0.08)}\) & \(0.23_{(0.20)}\) & \(0.14_{(0.16)}\) \\
8m3m\_21m & \(\mathbf{0.84}_{(0.13)}\) & \(0.30_{(0.24)}\) & \(0.66_{(0.28)}\) & \(0.04_{(0.06)}\) & \(0.16_{(0.25)}\) & \(0.21_{(0.37)}\) & \(0.23_{(0.21)}\) & \(0.14_{(0.13)}\) \\
8m4m\_23m & \(\mathbf{0.81}_{(0.12)}\) & \(0.13_{(0.20)}\) & \(0.54_{(0.25)}\) & \(0.00_{(0.00)}\) & \(0.02_{(0.04)}\) & \(0.07_{(0.11)}\) & \(0.02_{(0.03)}\) & \(0.02_{(0.02)}\) \\
12m4m\_30m & \(\mathbf{0.60}_{(0.15)}\) & \(0.02_{(0.04)}\) & \(0.40_{(0.27)}\) & \(0.00_{(0.00)}\) & \(0.01_{(0.01)}\) & \(0.00_{(0.00)}\) & \(0.02_{(0.02)}\) & \(0.00_{(0.00)}\) \\ \hline \end{tabular} The bold fonts indicate the best results.  
\end{table} 
Table 4: Results of comparison of value-based algorithms. Figure 6: Results of the training curves of comparison of policy-based algorithms. algorithms, but converges at _WR_ 0.7 at about 3M steps. ROMA fails to learn effective policy in Figure 5b. In Figure 5c, the increased 1 enemy Marine dramatically decreases the _WR_ of all algorithms. Only G-QMIX and GHQ are capable of reaching the _WR_ 0.5. QMIX, RODE, ASN, and UNMAS end with a _WR_ of about 0.3, 0.14, 0.15, and 0.13 separately. ROMA and CDS are both unable to learn any effective policy. In Figure 5d, we scale up allies with 1 Medivac and 2 Marines, and increase to 5 enemy Marines for balancing. After 5 M training steps, G-QMIX ends with _WR_ 0.8 with relatively low variance. GHQ attains _WR_ 0.6 with high variance. None of the other algorithms can reach a _WR_ higher than 0.25. Similarly, in Figure 5e, G-QMIX ends with _WR_ 0.8 with low variance, while GHQ can only reach _WR_ 0.5 with high variance. Still, none of the other algorithms can reach a _WR_ higher than 0.1. The above results significantly prove the effectiveness of G-QMIX against other baseline value-based methods in small-scale and medium-scale asymmetric heterogeneous SMAC maps.