## 2 Related Work
[53] introduced blockchain technology to establish explainability, and investigated their method using a swarm of unmanned aerial vehicles. [PERSON] et al. [54] proposed the real-time beliefs-desires-intentions (RT-BDI) framework, and highlighted the need of XMAS in a real-time process. In this paper, we particularly pursue an approach to establish interpretability of agents in MADRL, instead of introducing an interpretable neural network architecture in MAS. We focus on incorporating the attention mechanism in an actor neural network to highlight which information is correlated to the unknown decision-making process in MADRL. Attention mechanism in MADRL: Various MADRL models utilizing the attention mechanism for XRL have also been proposed. Previous studies [15; 55; 56; 57; 58; 59; 60; 61] often used the attention mechanism as a centralized communication processor that efficiently handles encoded messages among agents in MADRL. Incorporation of the attention mechanism in MAS is also beneficial for constrained problems [62], such as the approximation of underlying behaviors of agents [63] and trajectory prediction [64; 65]. In particular, [PERSON] et al. [55] introduced the multi-focus attention network, which helps agents attend to important sensory-input information using multiple, parallel attention mechanisms in a grid-like environment. [PERSON] et al. [66] investigated the enhancement of the agents' ability to efficiently adapt to complicated environments (Box-World and StarCraft II) that require relational reasoning over structured representations by the attention mechanism. [PERSON] et al. [67] introduced joint attention, which aggregates every other agent's attention map and demonstrated its cost-effectiveness in multi-agent coordination environments. The initial investigations into MADRL concentrated on improving agents' performance by using centralized attentions in the centralized training with decentralized execution (CTDE) [68] methodology. Nonetheless, the fully decentralized approach is generally more dependable because of reduced policy update variability, and is more realistic in real-world environments [69]. Furthermore, the analysis of coordination through attention heatmaps has not been fully explored. We aim to conduct a specific behavioral analysis of collaborative agents using decentralized attention heatmaps to determine how to employ alternate strategies and establish coordination based on their observations, to improve interpretability. DA3-X: DA3-X is a neural network model comprising the attention mechanism [18]. The key aspect of DA3-X lies in its ability to provide interpretability for agents in distributed multi-agent systems. In such systems, agents must establish coordination with one another, making it essential to understand how their cooperative behaviors arise from their black-box decision-making processes. While previous studies have explored transparency in coordination within centralized multi-agent systems, they have often overlooked the more practical distributed systems commonly found in real-world applications. By employing DA3-X as a baseline method, we present an analysis of its interpretability, specifically its capability to selectively identify crucial segments of observation by examining the attention weights within DA3-X [18]. Additionally, the flexible network structure of DA3-X allows for the application of various reinforcement learning methods. Furthermore, we demonstrate that the scalability of DA3-X remains unaffected by the number of agents, as it assumes a distributed system and does not rely on models from other agents. In this paper, we further extended the attention mechanism of DA3-X for further interpretability of agents in MADRL. While only standard attention is available in DA3-X, our proposed method, eDA3-X, is capable of serving the positional attention and class attention in addition to the standard attention.