## 5 Experiments and Results
### Attention Analysis from Coordination  
It was confirmed that eDA3-DQN and eDA3-IQN agents learn to adapt to the object collection game in the environment. In addition to the performance improvement of eDA3-X agents (Figure 3 and Table 1), eDA3-X agents can provide the information to interpret their decisions unlike agents with the baseline methods. Therefore, we first analyzed \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Observation** & **Model** & **Episode Reward** & **Objects Collected** & **Agents Collision** & **Walls Collision** \\ \hline local & dqn & \(62.95\pm 26.42\) & \(68.41\pm 26.56\) & \(2.63\pm 2.36\) & \(2.83\pm 2.81\) \\  & iqn & \(64.95\pm 33.87\) & \(70.04\pm 33.27\) & \(2.27\pm 1.99\) & \(2.82\pm 3.36\) \\  & eda3-dqn & \(82.36\pm 33.86\) & \(86.58\pm 34.22\) & \(2.03\pm 1.09\) & \(2.19\pm 2.25\) \\  & eda3-iqn & \(93.17\pm 38.17\) & \(99.23\pm 37.83\) & \(3.04\pm 5.09\) & \(3.02\pm 5.64\) \\ \hline relative & dqn & \(200.49\pm 48.18\) & \(210.36\pm 28.15\) & \(3.60\pm 3.74\) & \(6.27\pm 23.47\) \\  & iqn & \(234.19\pm 16.14\) & \(239.99\pm 14.62\) & \(3.13\pm 3.11\) & \(2.67\pm 4.76\) \\  & eda3-dqn & \(243.73\pm 18.84\) & \(249.11\pm 15.10\) & \(3.16\pm 5.08\) & \(2.22\pm 4.39\) \\  & eda3-iqn & \(250.26\pm 13.17\) & \(255.67\pm 12.09\) & \(3.19\pm 3.76\) & \(2.22\pm 2.81\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Quantitative performance comparison. Figure 3: Learning performance comparison between local and relative view methods. (**a**) Episode reward by DQN-based algorithms. (**b**) Episode reward by IQN-based algorithms. the interpretability of eDA3-X agents with the local view method. Note that eDA3-DQN agents provided similar interpretability as eDA3-IQN agents; hence, we only analyzed the attention of the eDA3-IQN agents. Figure 4 shows the attention heatmaps of a trained eDA3-IQN agent named _agent A_ in three different cases when it behaves cooperatively. The left figure in Figure 4 is an observation of _agent A_. The color and marks in the observation are identical to those in Figure 0(a), and the blue arrow next to _agent A_ expresses the direction of the next movement. Three attention heatmaps depict the standard, class, and positional attention heatmaps of _agent A_ from left to right, respectively. Each value in the attention heatmaps corresponds to the same segment in the observation, and each value in the attention heatmap is the attention weight. Note that positional attention heatmaps are identical throughout the three cases because positional attention shows the underlying focus strategy and is not affected by individual observations (see Equation (6)). In Case 1, _agent A_ finds two objects at different distances and approaches the closer object, as shown in Figure 3(a). This action is explained as _agent A_ assigns higher attention weight to the closer object than to the farther one (0.221 and 0.139, respectively), according to the standard attention heatmap in Figure 3(a). The standard attention heatmap also demonstrates that _agent A_ assigns attention weight to its surrounding cells as its right, upside, left, and downside cells show attention weights of 0.028, 0.034, 0.031, and 0.055, respectively. This phenomenon has already been reported in previous studies [17, 18] but with an insufficient explanation of this feature; for example, we cannot explain using only standard attention why the lower cell has a larger weight of 0.055 than that of the left cell (0.031), which is the position after the next move. On the other hand, with eDA3-X, we can explain that the lower cell is assigned a larger attention weight in the standard attention because the lower cell is usually more focused (0.125) than left cell (0.041) regardless of observation situation, according to the positional attention heatmap. Figure 4: Attention analysis with local view. Blue cell labeled \(A\) at center is observing agent. Blue arrow represents the direction of next movement. \(\bigstar\) marks represent objects.