## 1 Introduction  
Multi-Agent Reinforcement Learning (MARL) has emerged as a powerful framework for developing autonomous agents that can interact, collaborate, and compete in dynamic environments. As a subset of reinforcement learning (RL), MARL involves multiple agents learning simultaneously, sharing information, and making decisions based on their local observations and interactions with others [1, 2, 3, 4, 5]. The significance of MARL lies in its ability to model and solve complex real-world problems that require cooperation, coordination, and competition among agents across various domains, including information security [6, 7], resource management [8, 9, 10], building information modeling [11], and autonomous systems. In these fields, agents must adapt to continuously changing conditions, which makes MARL a promising approach for developing intelligent systems capable of navigating uncertain and dynamic environments. One of the key challenges in MARL is balancing the tradeoff between centralized and decentralized approaches [12]. In a fully centralized approach, agents share global information, which can alleviate coordination problems but may result in the explosion of joint state and action spaces, leading to inefficiency and scalability issues as the environment's complexity increases. On the other hand, in fully decentralized approaches, agents make independent decisions based solely on their local observations, focusing on maximizing their individual rewards. While this simplifies decision-making, it can hinder collaboration and coordination, particularly in environments where joint actions are essential for optimal outcomes. The Centralized Training and Decentralized Execution (CTDE) framework seeks to address this tradeoff by allowing agents to be trained with global information while making independent decisions during execution. However, this approach still faces the limitation of agents not having access to comprehensive information about other agents during execution, which limits their ability to make fully informed decisions and diminishes overall system performance. Graph Neural Networks (GNNs) have emerged as a powerful tool for capturing the complex relationships and dependencies between agents in multi-agent systems [13; 14]. Recent advances in GNN-based MARL have demonstrated the potential of graph structures to model agent interactions. For instance, ref. [15] proposed GNNComm-MARL by utilizing the graph attention network (GAT) [16] to enhance observability, mitigate non-stationarity, and improve scalability in multi-agent systems, while ref. [17] introduced MARGIN by extending the idea of mutual information optimization from the graph domain to multi-agent systems. In addition, ref. [18] leveraged GNNs to process both local and global information, further enhancing model performance. Building upon these developments, we propose QMIX-GNN, a heterogeneous MARL model that integrates GNNs to improve collaboration and coordination among agents. The key novelties of our approach compared to existing methods are as follows:  
* Adaptive Information Fusion: QMIX-GNN introduces a GNN-based information infusion mechanism that aggregates data from multiple agents, allowing for enhanced team perception and decision-making. * Heterogeneous Information Processing: Our model incorporates a dedicated module that handles the diverse observation and action spaces of heterogeneous agents, facilitating seamless communication and collaboration. * Scalability on Complex Tasks: By integrating these modules within the CTDE framework, QMIX-GNN achieves significant improvements in both performance and convergence speed on complex multi-agent coordination tasks.