## 1. Introduction  
Explainable reinforcement learning (XRL) has gained considerable attention from both academia and industry due to the vast potential of deep reinforcement learning (DRL) in various applications. However, the black-box problem of DRL, which refers to its unaccountable decision-making process, remains a significant limitation. XRL seeks to overcome this challenge by providing transparency and interpretability in the decision-making process of DRL agents. This is particularly important in critical domains such as autonomous vehicles, where even a small error in the decision can lead to undesirable real-world consequences. Providing explanations for DRL algorithm decisions not only facilitates error identification and correction but also builds trust and acceptance of DRL across various industries. According to recent research [1; 2; 3], multiple categories of XRL provide the interpretability of agents. For example, the decision-making process of agents has been interpreted by decomposing reward functions [4; 5; 6; 7] or visualizing their saliency maps based on the integrated gradients method [8; 9; 10; 11]. The attention mechanism and transformer developed by [PERSON] et al. [12] also play a critical role in providing transparency to the decision-making process. As a result, various neural network models based on the attention mechanism and transformer, such as vision transformer (ViT) [13] and decision transformer [14], have facilitated the successful visualization of input information that is instrumental in achieving state-of-the-art performance for explainability in computer vision and DRL. Despite the importance of clarifying the black-box coordination/cooperation mechanism for enhancing the productivity and robustness of the entire system, limited research has been conducted on XRL in multi-agent systems (MAS). The multi-actor-attention-critic (MAAC) [15], which integrates the attention mechanism in the style of MADDPG [16],illustrates how agents selectively focus on cooperative agents using the central attention mechanism, thereby improving the overall efficiency of the system. [PERSON] and [PERSON] [PERSON] and [PERSON] (2017) have proposed a multi-agent transformer deep Q-network (MAT-DQN) to establish the interpretability of distributed agents' coordinated behaviors by analyzing individual agents' attention mechanisms. Their findings are expected to enhance the explainability of learned behaviors of individuals as well as the efficiency of the entire system. However, their investigation on attention analysis remains inadequate; for instance, they have not addressed how the agents' attention patterns change throughout their training. To address this shortcoming, we previously proposed a distributed attentional actor architecture (DA3-X) [PERSON] and [PERSON] (2017), an extension of MAT-DQN [PERSON] and [PERSON] (2017), to enhance the interpretability of coordinated behaviors in multi-agent DRL (MADRL). The proposed model architecture relies on trainable parameters known as saliency vectors. As implied by X in DA3-X, the network architecture can accommodate various reinforcement learning algorithms, such as DA3-RAINBOW [PERSON] and [PERSON] (2017), DA3-DDPG [PERSON] and [PERSON] (2017), and DA3-IQN [PERSON] and [PERSON] (2017) to support agents' adaptation flexibility to a variety of environments. DA3-X is a sequential network architecture comprising three main modules: state embedder, transformer encoder, and DRL head. Agents utilizing DA3-X (DA3-X agents) can visualize information that is relevant to their learned actions via attention heatmaps. Similar to those of the ViT [PERSON] and [PERSON] (2017), attention heatmaps of DA3-X indicating the intensity of interest in the image-like observation can be generated by extracting the attention weight from the attention mechanism in the transformer encoder.