## 2 Materials and Methods
### Proposed Method
#### 2.3.4 Implementations
QMIX-GNN and QMIX achieve relatively high winning rate, with QMIX-GNN showing varying degrees of improvement over QMIX, suggesting that in more complex scenarios the agents need to focus more on information from other agents in order to achieve better cooperation. \begin{table}
\begin{tabular}{c c} \hline
**Parameter** & **Value** \\ \hline SMAC Difficulty & 7 (Hard) \\ Episode & 1,000,000 \\ Discount factor \(\gamma\) & 0.99 \\ Parameter update step & 200 \\ Attention head & 6 \\ \(K\)-nearest neighbor & 5 \\ Replay buffer size & 3000 \\ \hline \end{tabular}
\end{table} 
Table 4: The experimental parameters. The training processes for the four scenarios are shown in Figure 8, while the training steps required for each model to reach an 80% winning rate are shown in Table 6:

* In the 3m scenario, all four models converge relatively quickly and maintain a stable winning rate, although COMA only manages to maintain a relatively low winning rate. * In the 2s3z scenario, both QMIX and QMIX-GNN converge quickly and maintain a high winning rate, but QMIX has a more stable convergence curve with smaller fluctuations, while COMA exhibits an unstable convergence curve and performs poorly. In comparison, QTRAN converges slowly. * In the 1c3s5z, 10m_vs_11m, mmm, and mmm2 scenarios, QTRAN converges slowly and COMA fails to converge. Both QMIX and QMIX-GNN show good convergence performance, with QMIX-GNN converging significantly faster than QMIX and demonstrating better overall training results. ### Ablation Study

To evaluate the impact of the proposed modules in QMIX-GNN, we designed an ablation study in the 1c3s5z scenario to analyze the contributions of each component to overall model performance. Specifically, we compared three variants: the original QMIX model, our proposed QMIX-GNN, and QMIX-HT, i.e., QMIX with our heterogeneous information processing structures but without the GNN-based information fusion module. The training curves are shown in Figure 9 and a performance comparison is provided in Table 7. \begin{table}
\begin{tabular}{c c c c c} \hline
**Episodes** & **3m** & **2s3z** & **1c3s5z** & **10m_vs_11m** & **mmm** \\ \hline COMA & 820k & \(\infty\) & \(\infty\) & \(\infty\) & \(\infty\) \\ QTRAN & 150k & \(\infty\) & \(\infty\) & \(\infty\) & \(\infty\) \\ QMIX & 140k & 300k & 730k & 680k & 800k \\ QMIX-GNN & 120k & 410k & 380k & 420k & 640k \\ \hline \end{tabular}
\end{table} 
Table 6: Number of training episodes to reach an 80% winning rate. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Winning Rate** & **3m** & **2s3z** & **1c3s5z** & **10m_vs_11m** & **mmm** & **mmm2** \\ \hline COMA & 0.778 & 0.203 & 0.285 & 0.252 & 0.015 & 0.008 \\ QTRAN & **0.991*** & 0.720 & 0.713 & 0.717 & 0.316 & 0.167 \\ QMIX & 0.956 & **0.969** & 0.846 & 0.855 & 0.877 & 0.542 \\ QMIX-GNN & 0.967 & 0.957 & **0.900** & **0.892** & **0.928** & **0.655** \\ \hline Comparison with & & & & & & \\ QMIX & & & & & & \\ \hline \end{tabular} \({}^{*}\)The best results are highlighted in bold.

\end{table} 
Table 7: Performance comparison from the ablation study. \begin{table}
\begin{tabular}{c c c c c c c} \hline
**Winning Rate** & **3m** & **2s3z** & **1c3s5z** & **10m_vs_11m** & **mmm** & **mmm2** \\ \hline COMA & 0.778 & 0.203 & 0.285 & 0.252 & 0.015 & 0.008 \\ QTRAN & **0.991*** & 0.720 & 0.713 & 0.717 & 0.316 & 0.167 \\ QMIX & 0.956 & **0.969** & 0.846 & 0.855 & 0.877 & 0.542 \\ QMIX-GNN & 0.967 & 0.957 & **0.900** & **0.892** & **0.928** & **0.655** \\ \hline Comparison with & & & & & & \\ QMIX & & & & & & \\ \hline \end{tabular} \({}^{*}\)The best results are highlighted in bold.

\end{table} 
Table 5: Comparison of winning rates. Figure 8: Training results for the six scenarios: (**a**) 3m, (**b**) 2s3z, (**c**) 1c3s5z, (**d**) 10m_vs_11m, (**e**) mmm, and (**f**) mmm2. The experimental results of the ablation study indicate that QMIX-HT improves the winning rate by 0.027 over the baseline QMIX.