## 5 Experiments and Results  
### Experimental Details  
#### 5.1.1 Main Experiments in the SMAC Environment  
The SMAC environment is a multi-agent real-time tactical micromanagement game. The basic mission scenario is that two adversarial MASs fight against each other. The goal is to train a MARL algorithm to defeat enemy agents controlled by the internal script of the SMAC environment. The algorithm requires learning tactics and skills for choosing the best action response while utilizing the properties of different agents.  
Our experimental environments are shown in Figure 4. We use five maps mentioned in [27] and the MMM2 map from the original map set to conduct our experiments. Detailed map information can be seen in Table 3. As we have introduced in Sections 3.3 and 4.1, the Marine is an attacking unit and its action space consists of \(A_{s}\) and \(A_{e}\), while the Medivac is a supporting unit and its action space consists of \(A_{s}\) and \(A_{a}\).  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Hyper\_Parameters** & **Value** & **Hyper\_Parameters** & **Value** \\ \hline learning rate & \(5\times 10^{-4}\) & training step & 10 M \\ \(\epsilon_{p}\) & 0.2 & \(\epsilon_{v}\) & 0.2 \\ \(\lambda_{E}\) & 0.01 & \(\lambda_{V}\) & 1 \\ \(\lambda_{I}\) & 0.8 & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: The hyper-parameters of G-MAPPO.  
Figure 4: The example figure of experimental environments. (**A**) is the SMAC map 12m4m_30m. (**B**) is the MPE scenario large_spread_v1, red arrows point out the target and agents for every group.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Hyper\_Parameters** & **Value** & **Hyper\_Parameters** & **Value** \\ \hline learning rate & \(3\times 10^{-4}\) & training step & 5 M \\ lr\_decay factor & 0.5 & \(\epsilon\) & 1.0 \(\rightarrow\) 0.05 \\ \(\lambda_{I}\) & 0.06 & & \\ \hline \hline \end{tabular}
\end{table}
Table 2: The hyper-parameters of G-QMIX.  
For all agents, the dim of _self_ action \(|A_{s}|\) is 6. Action ID 0 is the _null_ action, only available for dead agents. Action ID 1 is the _stop_ action, and IDs \(\{2,3,4,5\}\) are _moving_ actions, indicating moving up, down, left, and right. The _self_ actions \(A_{s}\) are available for all agents. The dim of _ally_ action \(|A_{a}|\) equals the number of _allies_, while the dim of _enemy_ action \(|A_{e}|\) equals the number of _enemies_.  
We use the default global reward function of the SMAC environment. Agents are rewarded when dealing _damage_ to enemies, _killing_ enemies, and _winning_ the game. The _damage_ reward equals the absolute damage value dealt to the enemies. The _killing_ reward is a constant 10 for every enemy kill. And the _winning_ reward is a constant 200 given at the terminal time-step if the agents defeat the enemies.  
We use the traditional winning rate (_WR_) as the measuring criterion. _WR_ is the probability of MARL agents eliminating all enemies and winning the game, and we use the average value of _WR_ over 32 testing episodes for every 10,000 training steps (about 1000 training episodes). We take five rounds of individual experiments with different random seeds. Then, the curves of _WR_ changing over time are plotted with the \(p\)-value being 0.05 and the confidence intervals being 95%.