## 4 Method
### Generalized Action Space  
As analyzed in Section 3.2, adopting global parameter sharing in physically heterogeneous MARL problems without proper structure can lead to harmful sub-optimality. Therefore, we introduce the _Generalized Action Space_ as a solution.  
**Definition 1**.: _Generalized Action Space (GAS): For all agents \(i\in N\), divide their local available action set \(A_{i}\) into three sets with different action semantics: \(A_{s}\), \(A_{a}\) and \(A_{e}\). Then, the union set of the three sets of actions forms the Generalized Action Space \(A^{U}=A_{s}\cup A_{a}\cup A_{e}\)._  
It is obvious that the GAS is a superset to the original action space \(A^{U}\supseteq A\), as the original set is not forced to discriminate different _action semantics_. If some action semantics in the environment vary between the aforementioned three types over time-steps or states, it is required to define proper \"mate-actions\" for clearly distinguishing the different action semantics. Because the \(IO\) is a certain type at any certain time-step for a specific agent, such definitions of meta-actions are executable and can construct a similar GAS that fulfills our requirements.  
As analyzed in Section 3.3, in physically heterogeneous MARL problems, different action semantics determine physical heterogeneity and can be used to divide agents into different groups following the _IOG_ requirement [27]. Furthermore, different action semantics can be identified through different available action masks (_AMs_) defined on the GAS. For example, in SMAC, _Marine_ is an attacking unit whose action semantics include _moving_ and _attacking enemies_. As a result, the \(AM_{mar}\) only includes two subsets, \(A^{U}\odot AM_{mar}=A_{s}\cup A_{e}\). On the other hand, _Medivac_ is a supporting unit whose action semantics include _moving_ and _healing allies_. Therefore, the \(AM_{med}\) is only available for \(A^{U}\odot AM_{med}=A_{s}\cup A_{a}\). We point out that the \(\odot\) operator indicates the _masking-policy_ operation. Figure 1 illustrates the usage of GAS and the different action semantics.  
Introducing the GAS allows us to apply global parameter sharing while also avoiding the limitation of action semantic conflict. Considering the example in Section 3.2, let us suppose that the GAS is \(A^{U}=\{a_{0},\ldots,a_{n},a^{\prime}_{0},\ldots,a^{\prime}_{m}\}\). Then, the GAS policy distribution becomes \(\theta^{U}=\{\theta_{0},\ldots,\theta_{n},\theta_{0}^{\prime},\ldots,\theta_{m }^{\prime}\}\) for all agents. For generating group-specific policy, we need to mask the GAS distribution into group policy distributions: \(\theta_{\mathbb{G}_{0}}=\theta^{U}\odot AM_{\mathbb{G}_{0}}=\{\hat{\theta}_{0} ^{\prime},\ldots,\hat{\theta}_{m}^{\prime}\}\) and \(\theta_{\mathbb{G}_{1}}=\theta^{U}\odot AM_{\mathbb{G}_{1}}=\{\hat{\theta}_{0},\ldots,\hat{\theta}_{n}\}\). Because the mask operation renormalizes group policies, we have \(\sum_{m=0}^{|\mathbb{G}_{0}|}\hat{\theta_{m}}^{\prime}=1\) and \(\sum_{m=0}^{|\mathbb{G}_{1}|}\hat{\theta_{m}}^{\prime}=1\). Consequently, the policies of the two groups become independent and we can implement the same deterministic global optimal joint policy as \(J^{*}\) requires without restriction. As a result, the \(J^{*}_{\theta}\) with the usage of the GAS and \(AM\) becomes 1, which is equivalent to the global maximum \(J^{*}\).