## 4 Proposed Method: eDA3-X
### Standard, Positional, and Class Attentions  
Based on our previous study [18], we further extend DA3-X to improve its interpretability. While only standard attention is available in DA3-X, we introduce hyperparameters to derive positional attention and class attention in eDA3-X. The following equation is the mathematical procedure of decision-making process in DA3-X:  
\[\begin{split}& h_{0}=[\mathbf{v}^{\text{sal}};x^{1}E;x^{2}E;\ldots;x^{ 1}E]+p^{\text{pos}}\\ & h_{l}=\text{TEL}(h_{l-1}),\qquad l=1,\ldots,L\\ & Q=\text{DRLHead}(h_{L}^{0})\end{split} \tag{4}\]  
where \(x\in\mathbb{R}^{I\times(p^{2}N_{\text{C}})}\) are patched observation matrices, \(E\in\mathbb{R}^{(p^{2}N_{\text{C}})\times\mathbb{C}}\) represents embedding parameters, \(I=\lfloor\frac{K_{\text{P}}}{P}\rfloor\cdot\lfloor\frac{K_{\text{V}}}{P}\rfloor\) is the number of the matrices after the patch operation. Each line in Equation (4) corresponds to the procedure in the state embedder, transformer encoder layer (TEL), and DRL head in Figure 2, respectively.  
We introduce a new state embedder based on the previous state embedder in Equation (4) to derive the positional attention and class attention in addition to the standard attention in eDA3-X. The state embedder in eDA3-X becomes  
\[h_{0}=[\mathbf{v}^{\text{sal}};\alpha x^{1}E;\alpha x^{2}E;\ldots;\alpha x^{1}E]+ \beta p^{\text{pos}} \tag{5}\]  
where \(\alpha\in\{0,1\}\) and \(\beta\in\{0,1\}\) are hyperparameters depending on the purpose of attention analysis. While the eDA3-X agent is trained, those parameters are set as \(\alpha=1\) and \(\beta=1\), such that Equation (5) becomes identical to that in Equation (4). We call it standard attention to distinguish it from other attentions. Suppose \(\alpha=0\) and \(\beta=1\), we derive  
\[h_{0}=[\mathbf{v}^{\text{sal}};0;0;\ldots;0]+p^{\text{pos}}. \tag{6}\]  
during attention analysis. When \(\alpha=0\), all information about the observation is masked to zero, meaning the observation information is omitted. Hence, similarities between the saliency vector and sequences of position embedding are obtained in MHA. We call this attention positional attention. Similarly, when we set \(\alpha=1\) and \(\beta=0\), the state embedder in Equation (5) becomes  
\[h_{0}=[\mathbf{v}^{\text{sal}};x^{1}E;x^{2}E;\ldots;x^{1}E] \tag{7}\]  
where the similarities of the saliency vector only depends on embedded observation matrices. In this case, we call this attention class attention. Each combination of parameters (\(\{\alpha,\beta\}=\{1,1\},\{0,1\},\{1,0\}\)) allows us to examine attention analysis in different aspects.  
The DA3-X model utilized only the standard attention (\(\{\alpha,\beta\}=\{1,1\}\)), which highlighted influential segments in observation, such as cooperative agents with a high attention weight. However, it was unclear whether DA3-X assigned high attention to these segments to identify cooperative/coordinated actions or simply due to their proximity as nearby locations/objects that require constant awareness. This lack of interpretability hindered a thorough understanding of the model's decision-making process.  
Conversely, eDA3-X incorporates positional attention (\(\{\alpha,\beta\}=\{0,1\}\)) and class attentions (\(\{\alpha,\beta\}=\{1,0\}\)) to improve the interpretability of agents' decision making. The positional attention highlights the segments that eDA3-X always pays attention to, regardless of the situation-specific observations. For example, eDA3-X's focus on its surroundings in standard attention can be explained by the positional attention of eDA3-X. The class attention interprets the significance of the contribution of the attribute of segments to an agent's decision-making process, regardless of the observation's distance. For instance, eDA3-X assigns the same intensity of class attention to the same class of segments (other agents, objects, or walls), thereby elucidating how eDA3-X considers other agents' cooperativeness or irrationality in constructing coordination.