L EARNING E FFICIENT M ULTI -A GENT C OOPERATIVE V ISUAL E XPLORATION
A BSTRACT
We consider the task of visual indoor exploration with multiple agents, where the agents need to cooperatively explore the entire indoor region using as few steps as possible.
Classical planning-based methods often suffer from particularly expensive computation at each inference step and a limited expressiveness of cooperation strategy.
By contrast, reinforcement learning (RL) has become a trending paradigm for tackling this challenge due to its modeling capability of arbitrarily complex strategies and minimal inference overhead.
We extend the state-of-the-art singleagent RL solution, Active Neural SLAM (ANS), to the multi-agent setting by introducing a novel RL-based global-goal planner, Spatial Coordination Planner (SCP), which leverages spatial information from each individual agent in an end-toend manner and effectively guides the agents to navigate towards different spatial goals with high exploration efﬁciency.
SCP consists of a transformer-based relation encoder to capture intra-agent interactions and a spatial action decoder to produce accurate goals.
In addition, we also implement a few multi-agent enhancements to process local information from each agent for an aligned spatial representation and more precise planning.
Our ﬁnal solution, Multi-Agent Active Neural SLAM (MAANS), combines all these techniques and substantially outperforms 4 different planning-based methods and various RL baselines in the photo-realistic physical testbed, Habitat.
I NTRODUCTION
Navigation is an important problem for building intelligent robot systems, which has been extensively studied in enormous domains, including rescue (Kleiner et al., 2006), autonomous driving (Bresson et al., 2017), drone (von Stumberg et al., 2017) and mobile robots (Rubio et al., 2019).
In this paper, we consider a multi-agent exploration problem, where multiple homogeneous robots simultaneously explore an unknown spatial region via visual and sensory signals in a cooperative fashion.
The existence of multiple agents enables complex cooperation strategies to effectively distribute the workload among different agents, which could lead to remarkably higher exploration efﬁciency than the single-agent counterparts.
However, planning over multiple agents becomes substantially harder as well thanks to an exponentially large action space.
Planning-based solutions have been widely adopted for robotic navigation problems for both singleagent and multi-agent scenarios (Chaplot et al., 2020a; Burgard et al., 2005; Umari & Mukhopadhyay, 2017).
Planning-based methods require little training and can be directly applied to different scenarios.
However, these methods often suffer from limited expressiveness capability on coordination strategies, require non-trivial hyper-parameter tuning for each test scenario, and are particularly time-consuming due to repeated re-planning at each decision step.
By contrast, reinforcement learning (RL) has been promising solution for a wide range of decision-making problems (Lillicrap et al., 2015; Mnih et al., 2013), including various visual navigation tasks (Chen et al., 2019a; Chaplot et al., 2020a;
Savinov et al., 2019).
An RL-based agent is often parameterized as a deep neural network and directly produces actions based on raw sensory signals.
Once a policy is well trained by an RL algorithm, the robot can capture arbitrarily complex strategies and produce real-time decisions with efﬁcient inference computation (i.e., a single forward-pass of neural network).
However, training effective RL policies can be particularly challenging.
As a result, most existing RL methods for robot exploration problems focus on the single-agent setting while most multi-agent RL methods are only evaluated on simple scenarios like maze or grid world (Wakilpoor et al., 2020).
We develop Multi-Agent Active Neural SLAM (MAANS), the ﬁrst RL-based solution for cooperative multi-agent exploration that substantially outperforms classical planning-based methods in a photorealistic physical simulator, Habitat (Savva et al., 2019).
MAANS extends the single-agent Active
Neural SLAM method (Chaplot et al., 2020a) to the multi-agent setting and consists of 4 components, a neural SLAM module, a planning-based local planner, a local policy for control, and the most critical one, an RL-based global planner, which performs multi-agent planning and generates a navigation target for each agent.
We propose a novel multi-agent global planner, Spatial Coordination Planner (SCP), which is trained by the multi-agent PPO algorithm (Yu et al., 2021a) and integrates all the local spatial information for effective planning in an end-to-end manner.
SCP leverages a transformer-based relation encoder to effectively model the complex intra-agent interactions, and a spatial action decoder to generate accurate and effective goal assignments towards accelerated exploration.
In addition to the SCP module, we also implement a map reﬁner to align the spatial representation of each agent’s local map, and a map merger, which enables the local planner to perform more precise sub-goal generation over a manually combined approximate 2D map.
Experiment results show that MAANS achieves substantially higher exploration efﬁciency than a collection of planning-based competitors as well as various RL baselines on a set of carefully selected Habitat maps with sufﬁcient difﬁculty-level.
Our contributions are summarized as follows:
• We introduce a multi-agent cooperative navigation framework, Multi-Agent Active Neural SLAM (MAANS).
• We develop a novel RL-based multi-agent planner, Spatial Coordination Planner (SCP), which applies a transformer-based relation encoder to effectively fuse each agent’s local information and adopts a spatial action decoder to generate accurate global goals.
• We implement a map reﬁner, which uniﬁes the map representation in MAANS, and a map merger, which improves local trajectory planning.
• MAANS produces substantially higher exploration efﬁciency than 4 planning-based methods in a photo-realistic physical environment, Habitat, with a 16.4% ACS improvement on 2agent training maps, 8.2% on 2-agent unseen maps and 12.1% on 3-agent maps than the best of planning-based baselines.
Ablation studies also show that MAANS outperforms various RL-based methods.
R ELATED W ORK
Visual Navigation.
Navigation with visual sensory is a critical task for building mobile robots.
Classical methods typically apply a SLAM algorithm (Fuentes-Pacheco et al., 2015; Thrun, 2002) to reconstruct a spatial map from visual signals, and then derive a trajectory towards the target via path planning (Kavraki et al., 1996;
LaValle et al., 2001; Sethian, 1996).
For effective exploration, frontierbased sampling (Holz et al., 2010; Yamauchi, 1997; Umari & Mukhopadhyay, 2017; Dornhege &
Kleiner, 2013) is the most widely used heuristic, which adaptively selects navigation goals located on the boundary between the explored and the unexplored region.
Recently, with the advances in deep RL, many RL-based solutions have been developed (Mikolov et al., 2010; Mirowski et al., 2017; Zhu et al., 2017; Fang et al., 2019).
Many works also leverage the spatial inductive bias to effective reconstruct the 2D map via an end-to-end spatial memory layer (Chen et al., 2019b;
Mousavian et al., 2019; Henriques & Vedaldi, 2018; Parisotto & Salakhutdinov, 2018; Zhang et al., 2017) or an explicitly constructed graph structure to reﬂect the semantic topological structure of the environment (Bhatti et al., 2016;
Wu et al., 2019;
Yang et al., 2018; Chaplot et al., 2020c;
Savinov et al., 2018; 2019).
The Active Neural SLAM (ANS) method (Chaplot et al., 2020a) is the state-of-the-art framework for single-agent visual exploration, which takes advantage of both planning-based and RL-based techniques via a modular design (details in Sec. 3.2).
There are also follow-up enhancements based on the ANS framework, such as improving map reconstruction with occupancy anticipation (Ramakrishnan et al., 2020) and incorporating semantic signals into the reconstructed map for semantic exploration (Chaplot et al., 2020b).
Our MAANS can be viewed as a multi-agent extension of ANS with a few multi-agent-speciﬁc components.
Multi-agent Exploration.
Cooperation among multiple robots is another important challenge in robotics.
Many works have extended classical planning-based navigation methods from the singleagent setting to the multi-agent setting by sharing the reconstructed map (Dornhege & Kleiner, 2013; Holz et al., 2010; Umari & Mukhopadhyay, 2017; Yamauchi, 1997) or by applying some explicit cooperation heuristics (Wurm et al., 2008; Cohen, 1996; Burgard et al., 2005; Cohen, 1996).
These simple techniques require little training but may have very limited expressiveness power to represent complex multi-agent cooperative strategies.
Meanwhile, similar multi-agent extensions have been also developed for RL-based methods, such as introducing communication channels between individual agents (Wang et al., 2021; Zhu et al., 2021), utilizing intrinsic reward (Wang* et al., 2020), or cooperative training with curriculum learning (Long et al., 2020;
Yang et al., 2020; Wang et al., 2020).
However, jointly optimizing multiple policies makes multi-agent RL training remarkably more challenging than its single-agent counterpart.
Hence, these end-to-end RL methods either focus on much simpliﬁed domains, like grid world or particle world, or still produce poor exploration efﬁciency compared with classical planning-based solutions.
Our MAANS framework adopts a modular design and is the ﬁrst RL-based solution that signiﬁcantly outperforms classical planning-based baselines in a photo-realistic physical environment.
Finally, we remark that MAANS utilizes a centralized global planner SCP, which assumes perfect communication between agents.
There are also works on multi-agent cooperation with limited or constrained communication (Sukhbaatar et al., 2016; Peng et al., 2017; Jiang & Lu, 2018), which are parallel to our focus.
P RELIMINARY 3. T ASK S ETUP
We use the Habitat environment (Savva et al., 2019), which provides photo-realistic visual signals and physics dynamics.
At each step, each agent observes an ﬁrst-person view RGB image in the shape of 128 × 128 as well as its pose sensory signals.
The available actions include moving forward and rotation.
We remark that Habitat also introduces random noise to pose signals and actions to simulate the real-world physics.
We consider a multi-agent cooperative exploration task, where a team of robots needs to explore a given house to maximize the accumulative amount of explored region as fast as possible within a limited horizon.
In this task, we assume centralized decision setting, i.e., each agent can be fully aware of all the information from its teammates.
Note that in this cooperative exploration setting, the birthplace of agents matters a lot, i.e., if all the agents are uniformly spread over the room, even random exploration could result in satisfactory room coverage rate.
Hence, we focus on a more challenging setting where all the agents are always spawn together, i.e., within a randomly positioned circle of 2 meters in diameter per episode.
3. S INGLE - AGENT A CTIVE N EURAL SLAM
The ANS framework (Chaplot et al., 2020a) consists of 4 parts: a neural SLAM module, a RL-based global planner, a planning-based local planner and a local policy.
The neural SLAM module, which is trained by supervised learning, takes an RGB image, the pose sensory signals, and its past outputs as inputs, and outputs an updated 2D reconstructed map and a current pose estimation.
Note that in ANS, the output 2D map only covers a neighboring region of the agent location and always keeps the agent at the egocentric position.
For clariﬁcation, we call this raw output map from the SLAM module a agent-centric local map .
The global planner in ANS takes in an augmented agent-centric local map , which includes channels indicating explored regions, unexplored regions and obstacles and the history trajectory, as its input, and outputs two real numbers from two Gaussian distributions denoting the coordinate of the long-term goal.
This global planner is parameterized as a CNN policy and trained by the PPO algorithm (Schulman et al., 2017).
Note that the global planner is actually trained over a much simpler 2D state-space with a shorter episode length compared to the original task since the agent needs to take a few environment steps to reach each global goal.
The local planner performs classical planning, i.e., Fast Marching Method (FMM)
(Sethian, 1996), over the agent-centric local map towards a given long-term goal, and outputs a trajectory of short-term sub-goals.
Finally, the local policy produces actions given an RGB image and a sub-goal and is trained by imitation learning.
M ETHODOLOGY
We illustrate the overall framework of Multi-Agent Active Neural SLAM (MAANS) in Fig.
Each agent ﬁrst passes its pose sensory signals and RGB image to the neural SLAM module to obtain [Figure] the agent-centric local map and the pose estimation.
Each local map is normalized by the map reﬁner and combined with additional agent-speciﬁc information as a input global map to the Spatial
Coordination Planner (SCP).
For each agent with ID k , SCP takes in the ID information, applies a transformer-based relation encoder over the extracted features of all the input maps, and generates a global goal via the spatial action decoder for agent k .
The local planner performs trajectory planning on the merged global map towards the global goal.
Finally, an action is generated by the local policy.
Note that the neural SLAM module and the local policy do not involve multi-agent interactions, so we directly reuse these two modules from ANS (Chaplot et al., 2020a).
4. S PATIAL C OORDINATION P LANNER
The SCP module has 3 parts, i.e., CNN-based feature extractors, a transformer-based relation encoder and a spatial action decoder.
A illustration of SCP is shown in Fig.
4.1. CNN- BASED F EATURE E XTRACTOR
We apply a separate 5-layer CNN tower to process each agent’s input map and extract a 8 × 8 feature map with 32 channels.
Note that we simply use N CNN-towers in the setting of N agents without parameter sharing.
Then, a channel-wise concatenation is performed over all the feature maps to produce the input to the relation encoder.
In order to make sure the network is fully aware of the current decision agent k , we always use the ﬁrst CNN-tower to process the input map for that agent.
4.1. T RANSFORMER - BASED R ELATION E NCODER
We apply a simpliﬁed transformer block as a multi-agent relation encoder over the 8 × 8 spatial feature map to better capture the intra-agent interactions.
The relation encoder consists of two parts, including position encoding and spatial self-attention.
Agent-Speciﬁc Position Encoding:
We adopt an agent-speciﬁc variant of relative position encoding (Shaw et al., 2018).
When computing attention from grid ( x 1 , y 1 ) to grid ( x 2 , y 2 ) from the 8 × 8 feature map, conventional position encoding computes the embedding of the relative distances between the two attending grids, i.e., [ embed ( x 1 − x 2 ) , embed ( y 1 − y 2 )] .
In our task, the agent positions, i.e., the grids containing agents , are more important than those grids without agents.
Hence, for each grid position ( x, y ) and each agent i at grid ( x i , y i ) , we compute [ embed ( x − x i ) , embed ( y − y i )] , which yields a total of N embeddings at each spatial position as additional channels to the feature map.
Finally, to ensure the network is fully aware of the decision agent k , we similarly make the embedding for agent k as the leading channels and include an embedding of ID k at each grid.
Spatial Self-Attention: Inspired by the vision transformer model (Dosovitskiy et al., 2020), we convert the 8 × 8 feature map with position encoding into a sequence of 64 tokens and apply multihead self-attention (Vaswani et al., 2017).
The output sequence is then reshaped back to an 8 ×feature map with 16 channels.
4.1. S PATIAL A CTION D ECODER
We utilize a hierarchical action space to better capture the spatial structure in our exploration problem.
We design 3 action heads, a discrete region head denoting which grid to choose from the 8 × 8 feature map, and two continuous point heads denoting the x and y coordinate within the grid of choice by the region head .
In particular, we ﬁrst apply a CNN layer with 1 kernel size to project the 8 × 8 ×feature map to a 8 × 8 logit map, and apply a spatial softmax operator over these logits to derive
[Figure]
the categorical distribution over the 64 grid.
For each of point head, we parameterize the action distribution as a Gaussion distribution and directly project the entire feature map to the Gaussian mean and standard deviation.
A sigmoid function is applied to ensure the Gaussian output is bounded within 0 and 1.
When producing an action, we sample a discrete grid position from the region head and a relative coordinate from the point head, which suggests the global goal is located at a speciﬁc position within the selected grid region.
4. M AP R EFINER FOR A LIGNED 2D M APS
We develop a map reﬁner to ensure all the maps from the neural SLAM module are within the same coordinate system.
The workﬂow is shown as the blue and green part in Fig.
The map reﬁner ﬁrst composes all the past agent-centric local maps to recover the agent-centric global map.
Then, we transform the coordinate system based on the pose estimates to normalize the global maps from all the agents w.r.t.
the same coordinate system.
Note that when an agent explores the border of the house, the agent-centric local map often covers a large portion of invisible region.
As a result, the normalized global map will accordingly contain a large unexplorable boundary surrounding the actual explorable house region.
To ensure the feature extractor in SCP concentrates only on the viable part and also induce a more focused spatial action space, we crop the unexplorable boundary of the normalized map and enlarge the house region as our ﬁnal reﬁned map.
4. M AP M ERGER FOR I MPROVED L OCAL P LANNING
The local planner from ANS plans sub-goals on the agent-centric local map, while in our setting, we can also leverage the information from other agents to plan over a more accurate map.
The diagram of map merger is shown in Fig.
After obtaining N enlarged global maps via the map reﬁner, the map merger simply integrates all these maps by applying a max-pooling operator for each pixel location.
That is, for each pixel in the merged global map, the probability of it being an obstacle is the maximum value at that pixel over all the individual enlarged global maps.
We remark that the artiﬁcial merged global map is only utilized in the local planner, but not in the global planner SCP.
We empirically observe that having a coarse merged map produces better short-term local goal while such an artiﬁcial map is not sufﬁcient for accurate global planning (more details can be found in Sec.
4. RL T RAINING FOR SCP
We adopt the multi-agent PPO framework (Yu et al., 2021a) with parameter sharing.
Some of the most important training factors are presented below while full details can be found at https: //sites.google.com/view/maans .
4.4. R EWARD
The team-based reward includes 4 terms, i.e., (1) a coverage reward, which is the increment of the explored region, (2) a success reward, which will be given when 90% and 95% coverage is achieved,
[Figure]
4.4. I NPUT R EPRESENTATION
Each agent input map for SCP is a 240 × 240 map with 7 channels, including (1) an obstacle channel, where each pixel value denotes the probability of being an obstacle, (2) an explored region channel, which is a probability map for each pixel being explored, (3) a one-hot position channel, which denotes the position of that agent, (4) a trajectory channel, which covers all the historical position of that agent with an exponentially decaying weight, (5) a one-hot global goal channel denoting the position of previous global goal, (6) a goal history channel with all the past global goals, and (7) an agent ID channel.
We remark that SCP takes N such reﬁned input maps as its input state.
4.4. M ULTI -T ASK T RAINING AND D ISTILLATION
We empirically notice that in our multi-agent exploration task, the training progress varies a lot on different maps.
Therefore, when SCP is directly trained over a uniform distribution of all training maps, SCP may easily overﬁt to some particularly easy maps and stop learning on those harder ones.
In order to obtain a single policy that can work on multiple maps and eventually generalize to unseen maps, we propose a simple training-and-distillation solution, i.e., MAANS-TD : at the ﬁrst stage, we train a specialized policy using RL for every training map while at the distillation stage, we train a general policy over all the maps by running imitation learning over the expert policies.
We adopt a
KL-divergence loss for the region head and a MSE loss for point heads as our imitation objective.
E XPERIMENT R ESULTS
We consider the 3D houses from the Gibson dataset (Xia et al., 2018) and eliminate those houses with disconnected viable regions or multiple ﬂoors.
Based on the size of available area, we adopt 7 middle maps and 2 large maps for training, and 3 middle maps for evaluation.
We assume the two-agent scenario ( N = 2 ) in all the following experiments unless otherwise mentioned (i.e., Sec. 5.3.3).
Every RL training is performed with 10 4 training episodes over 3 random seeds.
Each evaluation score is expressed in the format of “mean (standard deviation)”, which is averaged over a total oftesting episodes, i.e., 100 episodes per random seed.
5. E VALUATION M ETRIC
We proposed Accumulative Coverage Score (ACS) as our performance metric, which captures the overall exploration progress throughout an episode.
In particular, let C t denote the coverage rate, i.e., the ratio of explored region to the total explorable area, at timestep t for an episode.
The ACS number at timestep k , A k , is computed by A k = (cid:80) kt =0 C t .
A higher ACS number implies faster exploration.
We take the ACS number at timestep 200 in all our experiments.
5. B EHAVIOR S TATISTICS
In addition to ACS
[Table]
we also consider 3 additional behavior statistics measurement to capture different characteristics of a particular exploration strategy.
We remark that these metrics is only for analysis purpose
[Table]
and we primarily focus on ACS as our performance criteria.
• Coverage Ratio : the ﬁnal ratio of explored area to total area when an episode terminates.
Higher coverage implies more exhaustive exploration.
• Steps : the timesteps that the agents use to reach a 90% coverage.
Fewer steps implies more efﬁcient exploration.
• Overlap Ratio : the ratio of the overlapped region explored by multiple agents to the current explored area when 90% coverage is reached.
Lower overlap ratio implies better credit assignment.
[Table] [Table]
[Table]
[Figure]
5. C OMPARISON WITH P LANNING - BASED M ETHODS
We consider 4 planning-based competitors, including a utility-maximizing method ( Utility ) (Juliá et al., 2012), a search-based nearest-frontier method ( Nearest ) (Yamauchi, 1997), the artiﬁcial potential ﬁeld method ( APF ) (Yu et al., 2021b), which applies resistance forces among agents as a cooperation mechanism, and a rapid-exploring-random-tree-based method ( RRT )
(Umari &
Mukhopadhyay, 2017).
Since these planning-based methods also require a SLAM module and a controller, we simply substitute SCP in our framework with a planning algorithm while the neural
SLAM module, the local planner and the local policy are all preserved.
5.3. T RAINING P ERFORMANCE
Tab. 1 summarizes the average performance of MAANS and planning-based baselines over the training maps (7 middle maps and 2 large maps).
We notice that RRT consistently produces the best result among all the planning-based methods under every evaluation metric.
However, it is still outperformed by MAANS with a clear margin.
MAANS produces the same ﬁnal coverage ratio as
RRT while achieves a 15% higher ACS, more than 30 fewer steps for 90% coverage (horizon is 300), and about 20% lower overlap ratio.
These metrics suggest that MAANS is able to explore more space faster and with better multi-agent coordination.
To better measure the overall exploration process, Fig.
4 visualizes how the coverage ratio as well as the overlap ratio varies throughout each testing episode.
In particular, we plot the average coverage rate at each episode step, and the overlap ratio w.r.t.
different coverage ratio.
Regarding coverage rate, we can observe that MAANS explores substantially more area than other methods particularly in the early stage of exploration, leading to a higher ACS value consequently.
For overlap ratio, MAANS maintains a signiﬁcantly lower curve particularly in the later period where more than 30% of the house is explored.
This suggests that MAANS is able to well control the agents to avoid repetitive exploration.
5.3. G ENERALIZATION P ERFORMANCE
Tab. 2 shows the evaluation performance of our distillation policy ( MAANS-TD ) and planning-based methods on 3 unseen test maps.
MAANS-TD achieves the best ﬁnal coverage ratio, the fewest steps for 90% coverage and a comparable overlap ratio.
More importantly, MAANS-TD yields the highest
ACS score with a clear margin (i.e., 9+ ACS number) compared to the best planning-based competitor, which suggests that our training-and-imitation solution leads to a generalizable policy.
5.3. P ERFORMANCE WITH M ORE A GENTS
Here we apply MAANS and other baselines to the setting of N = 3 agents.
Tab. 3 summarizes the average training performance w.r.t.
different evaluation metrics on 7 middle maps and 2 large maps.
With N = 3 agents, MAANS still produces the best ﬁnal coverage ratio, the fewest steps towards 90% coverage, the least overlap ratio, as well as the highest ACS number with a clear margin (15+ in ACS value) over other baselines.
The rightmost ﬁgure of Fig.
4 shows the comparison of ACS performance between 2 agents and 3 agents.
This surprising advantage of MAANS over planning-based methods shows the potential to further improve collaborative exploration efﬁciency with more agents.
5. C OMPARISON WITH O THER ANS V ARIANTS
We consider 2 additional ANS variants other than MAANS.
For simplicity, we report the training performances on 3 selected maps, Colebrook , Dryville and Sodaville .
ANS-blind:
We train N ANS agents to explore blindly, i.e., without any communication, in the environment.
ANS-stack: We directly stack all the agent-centric local maps from the neural SLAM module as the input representation to the global planner, and retrain the ANS global planner under our multi-agent task setting.
We measure the ACS and the Overlap Ratio metric over these 3 maps and demonstrate the training curves in Fig.
Regarding the ACS, both ANS variants perform consistently worse than MAANS on each of the map.
Regarding the overlap ratio, the blind variant fails to cooperate completely while the stack variant produces comparable overlap ratio to MAANS despite its low exploration efﬁciency.
We hypothesis that this is due to the fact that ANS-stack performs global and local planning completely on the agent-centric local map.
The local map is a narrow sub-region over the entire house, which naturally leads to a much conservative exploration strategy and accordingly helps produces a low overlap ratio.
5. A BLATION S TUDY ON SCP
We consider 3 SCP variants and report the training performances on map Quantico .
SCP w.o. RE:
We consider the SCP without the relation encoder.
The output feature maps from the
CNN-based feature extractors are channel-wise concatenated and directly fed into the spatial action decoder.
SCP w.o. AE:
We remove the region head from the spatial action decoder, so that the global goal is directly generated over the entire reﬁned global map via two Gaussian action distributions.
We remark that such an action space design follows the original ANS paper (Chaplot et al., 2020a).
SCP-merge: We consider another SCP variant that applies a single CNN feature extractor over the manually merged global map from the map merger, instead of forcing the network to learn to fuse each agent’s information.
We measure the ACS and the Overlap metric and visualize the training curves in Fig.
In summary, the full MAANS module produces both the highest ACS and the lowest overlap ratio.
Among all the SCP variants, SCP w.o. AE produces the lowest ACS.
This suggests that a simple Gaussian representation of actions may not be able to fully capture the distribution of good long-term goals, which can be highly multi-modal in the early exploration stage.
In addition, SCP-merge produces a much higher overlap ratio than all the other methods.
We hypothesis that this is due to the fact that many agent-speciﬁc information are lost in the manually merged maps while SCP can learn to utilize these features implicitly.
C ONCLUSION
We propose the ﬁrst multi-agent cooperative navigation framework, Multi-Agent Active Neural SLAM (MAANS) that outperforms planning-based competitors in a photo-realist physical environment.
MAANS consists of a novel RL-based multi-agent planner, Spatial Coordination Planner, and a collection of multi-agent-speciﬁc enhancements.
Experiments in the Habitat testbed shows that
MAANS achieves better training and generalization performances than all the baseline methods.
We also show that our method can be applied to a system with various number of agents.
We hope that our method can be a starting point for more efﬁcient multi-agent exploration systems in even more challenging settings, such as a large number of agents or constrained communication, in the future research.