## 5 Experiments and Results  
### Experimental Setup  
For our experiment, we utilized a grid-like environment \(\mathcal{E}\), as illustrated in Figure 1a, where \(G_{\mathrm{X}}\times G_{\mathrm{Y}}=25\times 25\), including the walls. Each cell in \(\mathcal{E}\) is represented by different colors to indicate entities, where black represents walls, blue and red represent the initial position of agents, and white represents empty space. The object spawn regions are represented by green, red, and beige segments, respectively. The agents aim to collect the \(\bigstar\)-marked objects, with a total of 40 objects in the environment. Once an object is collected by an agent, a new object will spawn at a random location within the corresponding spawn region.  
We consider a scenario where six agents (\(N=6\)) are placed in the environment \(\mathcal{E}\), consisting of four intelligent agents and two roaming agents. At the beginning of each episode, the intelligent agents spawn at specific blue cells (\(\{0,1,2,3\}\)) in Figure 1a and start exploring. Each agent learns its policy \(\pi_{i}\) to maximize the collection of objects in its designated area without collisions. Specifically, _agent_ 0 and _agent_ 1 can collect objects only in the green and beige regions, while _agent_ 2 and _agent_ 3 can collect objects in the red and beige regions. If agents try to pick up objects outside their designated area, the objects remain in the same location, and the agents receive zero reward \(r_{i}(s,a)=0\). The agents are not aware of this region's limitations and need to learn it through their experience. The roaming agents spawn at the red cells (\(\{4,5\}\)) and randomly move around in the environment without collecting objects, potentially confusing the intelligent agents. Since the roaming agents are not cooperative, the intelligent agents must determine which agents in \(\mathcal{E}\) are worth building coordination and exclude the non-cooperative agents. We set \(N_{\mathrm{C}}=N+2=8\), where \(N=6\) matrices are used to distinguish the agents' location and those of other nearby agents, and two matrices are used for objects and the invisible area, as shown in Figure 1b,c.  
In our experiment, we investigated the performance of two types of agents, namely _eDA3-DQN_ agents and _eDA3-lQN_ agents. The former had a multi-layer perceptron (MLP) as a _deep Q-network_ (DQN) [72] installed in their DRL head, while the latter had an _implicit quantile network_ (IQN) [21] installed. The eDA3-X agents were equipped with four attentional heads (\(h=4\)), and the transformer encoder was looped only once (\(L=1\)), which are the same hyperparameters as previous study [18]. The patch size was set to \(P=1\) and \(P=5\) for the local and relative views, respectively. The length of the saliency vector \(\mathbf{v}^{\mathrm{sal}}\) was set to \(C=64\) (same as [18]). As for the baseline algorithms, we trained standard DQN agents and IQN agents using the _double Q-learning_[73] and _dueling network_[74] algorithms.  
We trained agents for 5000 episodes, each consisting of \(H=200\) steps. The reward functions used were \(r_{\mathrm{obs}}=1\) for agents moving to a collectible object and \(r_{\mathrm{col}}=-1\) for colliding with other objects such as agents and walls. Agents were provided with an image-like partial observation of size \(R_{\mathrm{X}},R_{\mathrm{Y}}=7,7\) using either local view or a relative observation of size \(R_{\mathrm{X}},R_{\mathrm{Y}}=G_{\mathrm{X}}\times G_{\mathrm{Y}}=25\times 25\) using the relative view method, with a visible range of \(7\times 7\).