## 6 Conclusions  
This paper focuses on solving the action semantic conflict in physically heterogeneous MARL problems. The contributions of the paper are pointing out the effect of action semantic conflict under a global parameter-sharing constraint and proposing an available method to solve the problem. Our core motivation is to design a proper global parameter-sharing method that can not only cooperate well but also avoid bad local optimality led by the action semantic conflict. Inspired by previous studies on action semantics, we introduce the Generalized Action-Prediction Optimization (GAPO) method, which includes the _Generalized Action Space_ (GAS) and the _Cross-Group Prediction_ (CGP) loss.  
As a solution for distinguishing the heterogeneous action semantics of agents, the GAS is capable of acquiring better performance than the baseline method since the accurate action semantics improve the optimization process. To further improve the cooperation between heterogeneous groups, the CGP loss uses the trajectory of one group to predict the policy or Q value of other groups. This method avoids the calculation of mutual information, which must be taken between two groups, and therefore dramatically reduces the overall computation cost. We combine the GAPO algorithm with standard value-based and policy-based methods to generate the G-QMIX and G-MAPPO methods.  
Comparison and ablation experiments are conducted in physically heterogeneous SMAC maps to prove the effectiveness of G-QMIX and G-MAPPO. The results show that G-QMIX and G-MAPPO outperform other state-of-the-art algorithms with higher winning rate and lower variance. The validating tests in the MPE, MAMuJoCo and RPE environments show the effectiveness of G-QMIX and G-MAPPO in small-scale problems, and also reveal the limitations of our method in large-scale MARL problems and weakly cooperative heterogeneous MARL problems. In order to acquire good performance, when conducting the GAPO method in different cooperative heterogeneous MARL scenarios, it is necessary to carefully analyze the properties of the scenario and conduct pre-tests to determine the optimal value range of \(\lambda_{I}\).  
In the future, we will try to focus on large-scale MARL problems and real-world scenarios in MPE, MAMuJoCo, RPE, or other simulating environments. For example, we can further enlarge the number of agents in our SMAC maps or autonomous transport networks or distributed resource management scenarios. We hope our work could be helpful for further study in heterogeneous MARL problems. In addition, we are open to discovering the complex combination of communication or scalability with heterogeneity in open-world MARL problems.  
Conceptualization: X.Y. and S.W.; Methodology: X.Y. and S.W.; Software, Formal analysis and investigation: X.Y.; Writing--original draft preparation: X.Y.; Writing--review and editing: X.Y., Y.L., S.W. and S.H.; Funding acquisition: Y.L. and S.H.; Resources: Y.L. and S.H.; Supervision: Y.L. and S.H. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the Fundamental Research Funds for the Central Universities [grant number 2024 XXRCR080].  
Figure 15: Results for RPE validating tests using value-based algorithms.