NextBestPath: Efficient 3D Mapping of Unseen Environments
Authors: Shiyao Li
Antoine Guedon
Clémentin Boittiaux
Shizhe Chen
Vincent Lepetit
Venue: ICLR 2025 Poster (2025)
Keywords: 3D reconstruction
active mapping
Primary area: applications to robotics, autonomy, planning

Abstract: This work addresses the problem of active 3D mapping, where an agent must find an efficient trajectory to exhaustively reconstruct a new scene.
Previous approaches mainly predict the next best view near the agent's location, which is prone to getting stuck in local areas. Additionally, existing indoor datasets are insufficient due to limited geometric complexity and inaccurate ground truth meshes.
To overcome these limitations, we introduce a novel dataset AiMDoom with a map generator for the Doom video game, enabling to better benchmark active 3D mapping in diverse indoor environments.
Moreover, we propose a new method we call next-best-path (NBP), which predicts long-term goals rather than focusing solely on short-sighted views.
The model jointly predicts accumulated surface coverage gains for long-term goals and obstacle maps, allowing it to efficiently plan optimal paths with a unified model.
By leveraging online data collection, data augmentation and curriculum learning, NBP significantly outperforms state-of-the-art methods on both the existing MP3D dataset and our AiMDoom dataset, achieving more efficient mapping in indoor environments of varying complexity.

Published as a conference paper at ICLR 2025
NEXTBESTPATH: EFFICIENT 3D MAPPING OF UNSEEN ENVIRONMENTS
Shiyao Li', Antoine Guédon', Clémentin Boittiaux', Shiizie Shizhe Chen', Vincent Lepetit'
LIGM, Acoll Nationale des Ponts Chaussees, IP Paris, Univ Gustave Eiffel, CNRS, France (firstname. aastname @enpc. fr Iniia, École normale supérieure, CNRS, PSL Research University, France (firstname, Lastrame) iinria.fr fr
ABSTRACT
This work addresses the problem of active 3D mapping, where an agent must find an efficient trajectory exhaustively reconstruct new scene. Previous approaches mainly predict the next best view near the agent's location, which is prone to getting stuck in local areas. Additionally, existing indoor datasets are insufficient due to limited geometric complexity and inaccurate ground truth meshes. To overcome these limitations, we introduce a novel dataset AiMDoom with map generator for the Doom video game, enabling better benchmark active 3D mapping in diverse indoor environments. Moreover, we propose a new method call next-best-patt (NBP), which predicts long-term goals rather than focusing solely on short-sighted views. The model jointly predicts accumulated surface coverage gains for long-term goals and obstacle maps, allowing it to effi- ciently plan optimal paths with a unified model. By leveraging online data collec- tion, data augmentation and curriculum learning, NBP significantly outperforms state-of-the methods on both the existing MP3D dataset and our AiMDoom dataset, achieving more efficient mapping in indoor environments of varying com- plexity. Project page: https: //sh:: yao- 1i. thub. io/nbp/
INTRODUCTION
Autonomous 3D mapping of new scenes holds substantial importance for vision, robotics, and graphics communities, with applications including digital twins. In this paper, we focus on the problem of active 3D mapping, where the goal is for an agent to find the shortest possible trajectory to scan the entire surface of new scene using depth sensor.
This task is extremely challenging as the agent has identify efficient trajectory without knowing the scene in advance. Existing works can be broadly categorized into rulc-based and learning-based based approaches. Rule-based approaches, such as frontier-based exploration (FBE) (Yamauchi) 1997), utilize heuristic rules to select optimal frontiers at the boundaries of the already-known space for the next movement. Though being simple and generalizable, they f'all leverage data priors to develop more efficient planning strategies. To address this, learning-based methods, often referred as next-best-view planning (NBV), train parametric policies for action prediction. Although NBV approaches have demonstrated promising results, most of them only are evaluated on single-object datasets or outdoor scenes (Guédon et al.) 2022 Chang al. 2015 Peralta al.) (020), ignoring a critical but more difficult setting of indoor environments for active 3D mapping applications.
Existing indoor datasets (Xia al., 2018 Chang al. 2017). however, offer limited geometry com- plexity and often include imperfect ground truth meshes, making them inadequate to fully evaluate model performance in complex indoor environments. indoor dataset called AiMDoom for active 3D mapping. AiMDoom is built upon a map generator for the Doom video game, and features a wide range of indoor settings of four difficulty levels: Sim- ple, Normal, Hard and Insane. As illustrated in Figure Ial even in relatively simple indoor settings of our dataset, the state-of-the -re NBV approach MACARONS (Guédon et al., 2023) is frequently trapped in limited area and misses substantial portions of the scene. This limitation arises because
this work, we automatically construct a new

---

Published as a conference paper at ICLR 2025
(a) MACARONS (simple scene).
(b) Our NBP (simple scene).
(c) Our NBP (hard scene).
Reconstruction results and trajectories of MACARONS (Guédon et al. 2023 and our NBP model. Guédon et al. (2023) fails to fully map the environment in simple scenes (a), while our NBP model manages capture the full scene (b), even much more complex geometry (c).
Figure 1:
most NBV methods only look one step ahead to identify the next best view in neighbouring regions, making it difficult explore under-reconstrructed areas at far distances.
Some recent works (Chen et al. 2024 Feng et al. 2024 Zhan et al. [022] Georgakis et al. 2022) at-
tempt to overcome this limitation by searching for the next optimal view across a broader range. For example. [Goorgaiis et al. (2022) utilizes a strategy that relies on averaging predicted uncertainties each point along every sampled path, and uses a trained point-goal navigation model. However, training separate uncertainty map prediction and navigation models is less efficient, and the scene uncertainty does not directly align with the ultimate objective of 3D mapping.
Therefore, we further propose novel approach called next-hest-path (NBP) planning, which shifts from NBV approaches that predict a single nearby view, to predicting an optimal path in a unified model. Our model is composed of three key components: mapping progress encoder, coverage gain decoder and an obstacle map decoder. The mapping progress encoder efficiently encodes the currently reconstructed point cloud along with the agent's past trajectory. Based on the encoded representation, the coverage gain decoder predicts a value map over a large spatial range centred on the agent's current location. Each cell in the map represents the surface coverage gain accumulated along the optimal trajectory from the agent's location to the cell, which corresponds to the final metric for active mapping. The cell with the highest value score is viewed as long-term goal. The obstacle map decoder predicts obstacles in both seen and unseen regions by leveraging the agent's s current knowledge of the scene. This allows us to compute the shortest path to the long-term goal while avoiding obstacles. To train the model, we collect data online and iteratively improve the model. We also propose a data augmentation method that exploits property of shortest paths and combined curriculum and multitask learning strategy to enhance training efficiency.
We evaluate our methods on the existing indoor benchmark MP3D (Chang et al.] 2017) and our dataset AiMDoom. The proposed NBP model significantly outperforms state-of-the- art methods on both datasets from simple (Figure Tb) to more complex indoor environments (Figure Tch.
Our key contributions can be summarized as follows:
We introduce AiMDoom, the first benchmark systematically evaluate active mapping in indoor scenes of different levels of difficulties.
We propose novel next-best-path approach that jointly predicts long-term goals with optimal reconstruction coverage gains, and obstacle maps for trajectory planning.
Our approach achieved state-of-the-art results on both the AiMDoom and MP3D datasets.
RELATED WORK
Active Mapping. Active mapping aims to exhaustively reconstruct 3D scene in the shortest pos- sible time with a moving agent. Unlike SLAM (Chaplot et al.) 2020 Placed et al., 2023 Matsuki et al., 2024), which addresses both localization and mapping, active mapping focuses struction, continuously selecting viewpoints to cover the entire scene, assuming the pose is known. Early methods often relied frontier- exploration (FBE) approaches (Yamauchi) 1997). The key idea is to move the agent toward a heuristically selected frontier along the boundary between reconstructed and unknown regions of the scene. Among different strategies (Bircher et al. 2016
recon-
2

---

Published as a conference paper at ICLR 2025
Table 1: Comparison between AiMDoom and prior indoor 3D datasets. Navigation complexity is the maximum ratio of geodesic to euclidean distances between any two navigable locations in the scene. Universal accessibility means whether windows and doors are accessible.
Datasct
Replica
RobeTHOR
MP3D
Gibson (4+ only)
ScanNet
HMBD
AiMDuom (Ours)
Simple
Normal
Hard
Insane
Number of scenes
75
571 (106)
1613
1000
100
100
Flour space (m²)
2.19k
3.17k
101.82
217. .9%9 (17. (7.74k)
39.98k
368.42k
63.33k
134.84k
321.38k
548,85k
Navigution complexity
5.99
2.06
17.09
14.25 (11.9)
3.78
13.31
11.31
18.38
36.05
45,25
Universal accessibility
x
X
Easy expansion
x
X
X
X
X
X
/
Cieslewski et al. 2017 Zhou et al. 2021; Tao al., 2023 for frontier selection, moving to the nearest frontier serves as strong baseline. Additionally, there are efforts (Cao et al. 2021 Xu al.] (024) that combine global FBE and local planning strategies within hierarchical optimiza- tion framework to enhance exploration. However, these FBE-based approaches are heuristic- based and cannot exploit prior learned from data to explore more efficiently, restricting their performance in complex environments.
To address this limitation, saaming-basede approaches have been explored select the next-best views (NBV) for efficient 3D mapping. The NB V-based methods train models to select the optimal
pose from nearby camera poses (Guédon et al. 2022 2023 Lee et al. 2023) or from a limited
predefined view space such as hemisphere (Zhan et al. 2022 Lee et al. 2022] Peralta et al.. 2020
Zeng et al. 2020; Mendoza et al.) 2020). While these methods show promising results to reconstruct single objects, their performance remains limited in large environments. Due to the narrow search space for the next pose, NBV methods hehave like a greedy policy and thus can easily get stuck in local regions. To mitigate this, some works Ramrakhya al. [2022][Chen et al. (2023) use imitation learning to learn from human demonstrates which prioritize unseen exploration but with the cost of heavy labelling. More recently, efforts have been made enlarge the search range for the next best view (Chen et al. [2024] Ran et al., [0233] Pan et al. [0022] Georgakis et al., 2022). However, these methods are still primarily evaluated on single-object datasets with small moving steps, and often rely on optimizing indirect metrics like reconstruction uncertainty (Georgakis et al., (022), which are not directly aligned with the goal of exhaustive 3D reconstruction. In this work, we extend the evaluation to more complex indoor environments and also introduce new surface coverage gain criterion that optimizes the coverage gain along the best trajectory towards a long-term goal.
3D mapping datasets. Existing datasets for 3D mapping mainly focus on single isolated objects such as those in ShapeNet (Chang et al. 2015) and OmniObject3D (Wu et al. 2023). or outdoor scenes (Lu et al., 2023; Hardouin al., 2020 where the agent only needs to move around the scene to achieve full reconstruction. These datasets are comparatively less complex than indoor en- vironments where the agent must enter into the scene. The indoor scenes contain unique challenges such as dead ends and tight corners, which often force the agent to backtrack without significantly improving its objective.
While some works (Yan et al. 2023 Georgakis et al. 2022 Ramakrishnan et al. 2020 incorporate indoor scene datasets such as Gibson (Xia et al. (0118) and MP3D (Chang al. 2017). these often
exhibit significant limitations. Existing synthetic datasets (Strab et al. 2019 Deitke et al., 2020 often lack scene complexity, whereas real-world scans (Dai et al.] [017]] Ramakrishnan al. 2021), despite offering greater representational fidelity, are constrained by limited structural and map diver- sity and often suffer from substantial noise artifacts. This lack of reliable datasets prevents compre- hensive evaluation in active 3D) mapping tasks. In this work, we propose new dataset - AiMDoom, designed for benchmarking active mapping in indoor environments of different complexities.
3 THE AIMDOOM DATASET
In this section, we introduce AiMDoom, a new dataset for Active 3D Mapping in complex indoor environments based on the Doom video game|] | As Doom features wide variety of indoor settings, we use its map generator to create four sets of maps of increasing geometric complexity: Simple,
http///oooo_frrchhftttaa (franchi se)
3

---

Published as a conference paper at ICLR 2025
(a) Bird-eye views of samples from the Simple, Normal, Hard, and Insane levels (from left right).
(b) Representative images showing the internal structural composition of the scene.
Figure 2: Maps from our AiMDoom dataset. The AiMDoom dataset includes four levels of geometric complexity with various textures.
Normal, Hard, and Insane. the following, we first detail how built these maps and then discuss the key challenges presented in our AiMDoom dataset.
Dataset construction. We used the open-source software Obsidian to automatically generate Doom maps as our indoor environments. Four sets of hyperparameters are proposed to control architectural complexity and texture styles in Obsidian. By varying these hyperparameters, we pro- duced maps categorized into Simple, Normal, Hard and Insane difficulty levels. Each difficulty level made of 100 maps with 70 for training and 30 for evaluation.
The maps include doors and windows, all of which are configured to be open. This allows the agent to see and pass through the doors and windows. We converted the maps to the widely used OBJ format, and used Blender (Community (018)) to consolidate the texture images of each map into a single texture image. This makes the maps compatible with Pytorch3D (Ravi et al. 2020) and Open31) (Zhou et al. 2018). Further details presented in the supplementary material.
Key challenges. The AiMDoom dataset presents three key challenges for active 3D mapping. Firstly, the dataset features environments with intricate geometries and layouts shown in Fig- ure 2 making it challenging to determine the optimal exploration direction for effective mapping. Secondly, the maps have small doors and narrow corridors, requiring careful path planning igate. Finally, the map diversity requires the reconstruction system to generalize across different scenes. Table [ compares AiMDoom with existing indoor 3D datasets (Straub al. 2019 Deitke et al. 2020 (Cangg et al., 2017 Dai et al. 2017 Xia et al.) 2018 Ramakrishnan et al. 2021) high- lighting our dataset's strengths in scene area and navigation complexity.
We will release the dataset along with comprehensive toolkit to generate the data, which enables easy expansion of the dataset for future research.
4 LEARNING ACTIVE 3D MAPPING
4.1 OVERVIEW
Problem definition. Active 3D mapping aims to control agent, such as an unmanned aerial vehicle (UAV) or wheeled robot, to efficiently and exhaustively reconstruct a 3D scene. The agent starts at a random location within the scene, and at each time step t, it receives an RGB-D image Ie and must predict the next one G = (c. coa)) in the immediate surrounding of the agent. Here, c denotes the position coordinates, and c.ah represents the orientation angles. The agent continually
https::/cbssidaan -ennnn-eeveee level maakrrmaee... weaker.githan wig.t..t.iii.. io/
4

---

Published as a conference paper at ICLR 2025
Coverage gain and obstacle map prediction model
Hoc
Decision making
Coverage Gain Dacoder
Me
Mapping Process Encoder
Value Map Ma
long-lem goal cp
mapping
Obstacle Decoder
Ho
Pe: reconstructed point representation
progress
next-best path (Cp
We Obstacle Map Oze
Figure 3: Overview of the proposed next-best-patt (NBP) framework. The model (left, see Section 4.2) predicts value map of coverage gain and obstacle map, which are used for decision making (right, see Section 4.3) to obtain a next-best path.
predicts successive e, until a predefined time limit 7 is reached. The final output is the reconstructed 3D point cloud of the explored environment.
Overview of our approach. Existing approaches for active mapping (Guédon et al. 2022 2023) typically predict the next camera pose C a greedy manner, which often suffers from getting stuck in limited areas. To address this limitation, goal camera pose and uses to guide the next camera pose selection. Given all past observations and camera poses,
propose a novel approach that predicts a long-term
model predicts two key components centred on the agent's current pose c₁ (1) value map Mc., which estimates the surface coverage gain of candidate poses c in the surrounding of Ct, and (2) an obstacle map Oct- which accounts for both visible and predicted unseen obstacles in the environment. From the value map Ma.- we derive the long-term goal pose Cg and combine with the obstacle map Oc, compute an optimal path T₁ (Ce, Ct+1. Cg) that navigates the agent from its current pose to the goal pose Cy. This long-term goal-driven strategy helps the model avoid the pitfalls of short-sighted decisions and enhances coverage efficiency.
In the following, we first describe the model for Mc, and Oc, prediction in Section then fol- lowed by the decision-making making process to determine the next best path T Section Finally, in Section 4.4 we introduce the training algorithm for our model.
4.2 COVERAGE GAIN AND OBSTACLE PREDICTION MODEL
Figure 3 depicts the deep model we use to predict the coverage gains and the obstacle map. We detail this model below.
Mapping Progress Encoder. Let's denote P, the reconstructed point cloud at each time step t, obtained by adding the back-projected depth image to the previously accumulated point cloud Pt-1. Directly encoding the point cloud via 3D neural networks can be complex and inefficient. Therefore, we convert the 3D point cloud into multiple 2D images as inputs to 2D-based encoder.
To be specific, we first centre and crop the point cloud based on the agent's current position C Centering the input on the agent makes the model invariant the agent's position and thus improves generalization. Then, the point cloud is divided into K horizontal layers along the gravity axis. For each layer, we average the occupancy value along the gravity axis to transform each 3D data into a 2D image. In this image, each pixel encodes the density of 3D points within specific height range. The stack of point cloud projected images provides a simplified yet informative representation of the 3D structure.
Similarly, we project the 3D trajectory of the agent's 's past camera poses onto a 2D plane where each pixel denotes the frequency of visits to that location. This plane serves to mitigate the exploratory value of previously traversed regions. We define &s, to include the point cloud projected images and single historical trajectory image.
Given the stacked 2D images of E we employ an Attention UNet (Oktay et al. 2018) encoder with 4 downsampling convolutional blocks extract mapping progress features Cc.-
5

---

Published as a conference paper at ICLR 2025
Coverage Gain Decoder. This decoder predicts from Ce. 3D value map Ma. E R centered on the agent. is composed two upsampling convolutional blocks with an attention mechanism. The first two dimensions of the predicted value map, He and W.. correspond to the camera's s 2D position in the environment, while the third dimension N₂ represents different camera orientations. Each value in Ma, quantifies the estimated coverage gain achievable by moving the camera along the shortest trajectory from its current pose to the specific camera pose. The value map Mc, guides the selection of both long-term goal poses Cy and intermediate poses along the trajectory, enabling a two-stage optimization for efficient exploration, which will be discussed in Section
Obstacle Map Decoder. This decoder predicts the geometric layout O₂, IR. TR., of the current moving plane, also from the encoder output Cc.- Oc, is binary map representing potential obstacles around the current agent location, which used for path planning. To be noted, Oa2 includes not only visible obstacles but also anticipated unseen obstacles based on the structure of the partially reconstructed point cloud, providing useful priors for navigation. This decoder is implemented using Attention U-Net with 4 upsampling convolutional blocks, and the output is passed through sigmoid activation function to generate the binary obstacle map.
4.3 DECISION MAKING FOR NEXT-BEST-PATH PREDICTION
We derive both a long-term goal Cg and next-best-path T. (ct, C++1...., Cg) from the predicted Mc. and Oc employing different decision making strategies for training and inference. During training, we balance exploitation and exploration, while we prioritize exploitation during inference.
Training phase. We rely on the Boltzmann exploration strategy (lesa-Bianchi et al. (017) to sample camera pose the goal Cg based the value map Me.- The probability of selecting camera pose e as the goal is given by:
(xp(Me,lcl/3)
P(c,=c) - Exre (c]/B))
where C' represents all possible camera poses within Mc., f is the temperature parameter that bal- ances exploration and exploitation, and Mc, denotes the value of the cell for candidate c.
Once the long-tern goal cg is sampled, we use the Dijkstra algorithm to find the shortest obstacle- free path from the current position e to goal position ch with ground truth obstacle map. To select camera orientation along the path, we also leverage Ma, to sample one orientation from No potential orientations at each position. This strategy enhances data diversity and alleviates the risk of converging to local optima.
Inference phase. At inference, take Co as the pose with the maximum value in Ma. and the path planning is based on the predicted obstacle map Oc, instead of ground truth. Each position in the jectory is assigned the optimal orientation from the heatmap Mc, for its location. In practice, the predicted obstacle map may not be entirely accurate. Encountering an unexpected obstacle requires halting the trajectory and initiating a new decision- making phase.
Algorithm 1 Training procedure.
N: number training iterations
Ne: number of iterations using casy data
S, the number of trajectories per scene Initialize memory M and model parameters 0 for <- N do Initialize training set T - for each scene in training set do for <- 1 to S., do Collect training data {de =
if S N. then T + T {de 10
else < T U (d) endif
end for end for M MUT T +- T RandonSammle(MMM... for + to do
4.4 MODEL TRAINING
Algorithm [ outlines the training proce- dure for our model. We first gather train- ing data from all training scenes using the current model, and then update the model with the new data. This process is repeated iteratively until the model achieves con- vergence. We detail below the data col- lection, training objectives update the model, and the training strategy.
Update @ with loss in Eq. B over T end for end for return

---

Published as a conference paper at ICLR 2025
Training data collection. After sampling the goal pose Cg and the trajectory Tr, we generate ground truth labels to train the value map Mc. and obstacle map Oc..
For Mc., compute the coverage gain for the cell that corresponds cg the ground truth label. Let Pt and P, denote the reconstructed point clouds at pose &₁ and cy respectively, where Pg is the result of accumulating depth information into Pa the agent moves along the trajectory rę. By comparing the reconstructed point clouds with the ground truth point cloud pot, we can obtain the coverage gain ACov -..
Nor
-
G ( min MEP
<c)
Co
1
| yll <<) ) vCPs
min
yl
<
(2)
NGT
where Ner is the number of points in PGT ||| denotes the Euclidean distance, and is predefined distance threshold. Consequently, we set CCovo.... c. as the ground truth value for Mo, [cyl-
For Oc., we use the 3D mesh of the scene to derive the ground truth obstacle map Out Specifically, intersect the 3D mesh with plane at the agent's height, and project this intersection onto 2D grid. This 2D grid is binarized to distinguish between obstacles and free space. Finally, we centre the 2D grid around the agent's current position as OCT
To enhance the efficiency of data generation, we further perform a data augmentation by leveraging the property Dijkstra's algorithm, where every sub-path of a shortest path is also shortest path. From a given path each segment of the path (c,,e) where S < S m. More specifically, we update the ground truth values along the Dijkstra path MC [c]] = ACove,+ey- We also collect the input & and the ground truth of surrounding obstacles OCT for each E T2. This significantly increases the number of training samples derived from a single trajectory.
(co .t......e Cm Cg). we compute the coverage gain ACove. e for
We store all augmented pairs {di)= di = (E,,, M GT OCT) in memory for training, where L is the length of the trajectory.
Multi-task training. We jointly train the coverage gain and obstacle map prediction using data stored in memory. We use the mean squared error (MSE) loss for training the coverage gain predic- tion, and the binary cross-entropy (BCH) loss for training the obstacle map prediction. To balance these tasks effectively, apply learnable uncertainty weights for each task, following Kendall et al. (2018). Our multi-task loss function for sample di is formulated as follows:
1 [(0; di) = CMSE (M " Ma.)) LBC a. (O.))..))))))... + log σ1 + +og 02
1
(3)
207
02
where 9 represents the model parameters, 0i and are learnable uncertainty weights, Ma, and Ó.. are the model's predictions for the coverage gain and obstacle maps respectively.
Training strategy. We adopt curriculum training strategy (Liu et al. 2017] Yuan et al, 2022
Yan et al.] 2021; De Lange et al.) 2021) to train our model, starting with easier-to-predict samples and gradually incorporating the entire dataset. In particular, we consider that the initial steps of trajectory are more challenging since the agent has limited observations. Therefore, during the first Ne training iterations, exclude samples from the first 10 steps in a trajectory. After N, iterations, all samples in a trajectory are used in training.
During each training iteration, we use balanced combination of previously stored data from the memory and newly collected data generated by the current model (Wulfmeier et al., 2018 Mnih 2013 Rolnick et al. 2019 Aljundi et al.) 2019), which helps prevent catastrophic forgetting. Each training phase is limited to epochs to balance between enhancing performance and preventing overfitting on sub-optimal data.
EXPERIMENTS
5.1
EXPERIMENTAL SETUP
Dataset and simulation setup. We evaluate our model on the Matterport3D (MP3D) dataset (Chang al. 2017) and our own AiMDoom dataset.
7

---

Published as a conference paper at ICLR 2025
Table 2: Evaluation results on AiMDoom Dataset. For each difficulty level, all baseline models,
including ours, are trained from scratch on the corresponding training set to ensure a fair comparison.
Simple
Normal
Hard
Insane
Final Cov.
AUCs
Final Cov.
AUCs
Final Cov.
AUCs
Final Cov.
AUCs
Random
0.323
0.270aa.335
0.190am:2s
0.15220.108
0.124
0.088
0.074
0.050an.ass
FBE
0.605 0.760
0.565
0,415
0.425
0.31 100000
0.330
0.239
SCONE
0,577
0.483
0.412
0.313
0.290sa.095
0.19640.079
0.1 40
MACARONS
0,59940.200
0.479
0,418
0.314.0.0s
0.302
0.21 8
0. 0.19220.078
0.139ss.05s
NBP (Ours)
0.879ta.42
0.69220.156
0.734.0.122
0.52620.112
0.618
0.432
0.47220.095
0.312
For MP3D, we use the same setting as prior work (Yan et al. posed depth images have a resolution of 256 x 256 with horizontal field of view (hFOV) of 90" The mobile agent starts in the traversable space at height of 1.25m and chooses its next camera pose by moving forward by 6.5cm or turning left/right by 10 Depending on the size of each scene, the agent can take maximum of 1000 2000 steps. We focus only on single-floor scenes following Yan et al. (2023) with 10 and 5 scenes in training and evaluation respectively.
2023) for fair comparison. The input
For AiMDoom, utilize 70/30 train/test split for scenes in each difficulty level. The input RGB- D images are rendered at the resolution of 456 x 256 with a vFOV of 60%. The agent navigates in a traversable space of height 1.65m. The moving step includes 4 position movements (move forward, backward, left, or right by 1.5m) and 8 rotation movements (turn left or right by increments of 45,, covering the full 360%). For dense reconstruction, all methods capture three additional images between adjacent poses using linear interpolation. The maximum steps for Simple, Normal, Hard, and Insane levels are set to 100, 200, 400, and 500 respectively, to adapt to their different complexity.
Evaluation metrics. We follow prior works (Chen et al., 2024 Guédon et al. 2023) and adopt two key metries to evaluate the performance of active 3D reconstruction: (1) Final Coverage measures the scene coverage at the end of the trajectory, and (2) AUCs evaluates the efficiency of the recon- struction process by caleulating the area under the curve of coverage over time. The surface coverage is computed using ground truth meshes, consistent with prior work (Guédon et al. al.] (202)). We evalu- ate five trajectories per scene using identical random initial camera poses for different methods. We report the mean and standard deviation for each metric across all testing trajectories.
For a fair comparison with prior work in MP3D, we employ another set of metrics to evaluate coverage: (1) Comp. (%), the proportion of ground truth vertices within 5cm of any observation, and (2) Comp. (cm), the average minimum distance between ground truth vertices and observations.
Implementation details. Our model takes a stack of K 4 projected 2D images and one previous trajectory projected image as inputs, each with a resolution of 256 x 256 covering a 40m x 40m exploration area centred on the camera's current position. The extracted feature ec, from the encoder is of size 16 x 16 x 1024. The output value map Mc, is of size 64 x 64 x 8 and an obstacle map of 256 x 256 x 1, both representing the same 40m 40m area. The model is trained for at most N = 15 iterations, with the first Ne = 1 iterations using easier samples and Sn = 2 trajectories per scene. For subsequent iterations, we use all samples and reduce the trajectory count to S,, per scene. Fach trajectory has length of 100 steps and starts at random location. During the first data collection iteration, we randomly sample 1, 1,000 validation examples from memory and exclude them from training. Gradient accumulation is used in training which results in an effective batch size of 448. The learning rate is set to 0.001 and is decayed by a factor of 0. if the validation loss plateaus. We apply early stopping to terminate training when validation loss no longer decreases. The training is performed on a single NVIDIA RTX A6000 GPU, with an average completion time of 25 hours.
5.2 COMPARISON WITH STATE OF THE ART METHODS
MP3D. We compare our method with five baselines on the MP3D dataset, including: 1) Random, which randomly selects a camera pose among all candidates for the next step; 2) Frontier- Exploration (FBE) (Yamauchi 1997). which heuristically moves the agent to the nearest frontier; 3) OccAnt Ramakrishnan et al. (0200). which predicts the occupancy status of unexplored areas and rewards the agent for accurate predictions; 4) UPEN (Georgakis al. 2022), which utilizes
8

---

Published as a conference paper at ICLR 2025
(a) Results MACARONS. It generates complicated trajectories and often gets trapped in local areas.
(b) Results of our NBP method. It efficiently travels the scene and reconstructs the scene well.
Figure Comparison of our NBP method with the state-of-the-art MACARONS method. Both methods start from the same initial pose, marked in deep blue. We also include a demonstration video of active mapping using our method in the supplementary materials.
an ensemble of occupancy prediction models to guide the agent towards paths with the highest
uncertainty: 5) ANM (Yan et al. 2023), which guides exploration through a continually- neural scene representation. Results in Table 3 show our NBP performs best, with 6.23 absolute gain for the completion ratio compared to the state-of-the-art ANM (Yan et al.
2023) model.
AiMDoom. The proposed AiMDoom dataset is more challenging than MP3D dataset for ac- tive 3D mapping. We benchmark our approach against state-of-the-art Next-Best-View (NBV) approaches, including: SCONE (Guédon et al. 2022) which employs volumetric integration to sum the potential visibility points for each can- didate camera pose in the subsequent step and trained using supervised learning: and MAC- ARONS (Guédon et al.) [023)) which quanti- fies the coverage gains of potential next cam- poses to select the best one and utilizes self-supervised online learning paradigm. Both approaches select the next camera pose in a greedy manner. Unfortunately,
Table 3: Comparison on the MP3D dataset.
Method
Comp. (%) f
Comp. (cm) +
Random
45.67
26.53
FBE
71.18
9.78
UPEN
69.06
10.60
OccAnt
71.72
9.40
ANM
73.15
9.11
NBP (ours)
79.38
6.78
unable include UPEN (Georgakis et al. 2022) and ANM (Yan) jet al. 2023) our comparison. These methods rely on the navigation policy DD-PPO (Wijmans al. 2019 trained on their environments (Savva al. 2019),, which requires extensive GPU hours and thus is infeasible to retrain it on our dataset. However, we implemented FBE (Yamauchi 1997)
on our dataset, recognized strong baseline in reconstruction and exploration tasks.
As shown in Table 2, our method significantly outperforms the baselines across all metrics on four levels of AiMDoom. While NBV approaches such as SCONE and MACARONS excel in outdoor or single-object scenarios, their performance deteriorates in complex indoor environments. As illus- trated in Figure 4 MACARONS struggles to escape local areas due its short-term focus. It only selects the next best pose in nearby regions, and once these areas such as the interior of a single room are fully reconstructed, it has difficulty moving out of the room to explore under-explored, distant regions. In contrast, our approach overcomes this limitation by incorporating long-term goal guidance to determine the next- best path. In addition, our method surpasses the strong baseline FBE. Although FBE enables better exploration compared state-of-the-an NBV methods our dataset, its simple heuristic of moving to the nearest frontier leads to sub-optimal scene reconstruction as lacks strategic planning for efficient coverage.

---

Published as a conference paper at ICLR 2025
Despite the superior performance of our model, the results in hard and insane environments still unsatisfactory, highlighting the significant challenges posed in our dataset.
5.3 ABLATION STUDY
In this section, we perform ablation experi- ments to demonstrate the effectiveness of dif- ferent components in our model. All the ex- periments below are conducted on the Normal level of AiMDoom.
0.55
Coverage
AUCS
0.34
Coverage (%)
0.72
Spatial range of long-term goal. We com-
0.70
pare the impact of different spatial ranges for
the prediction of the value map Mc, and ob- stacle map Oet, which in turn determines the maximum distance of the long-term goals Cg- Specifically, we experiment with map sizes of 20m x 20m 50m 50m. The results pre- sented in Figure [ When the value map covers smaller area, the goal Cg is close the agent's 's current position, leading to behaviour similar to existing NBV methods that struggle with explo- ration. On the other hand, if the map size is too large, predicting Mc, and Oz, becomes much more challenging. Our findings demonstrate that selecting an appropriate spatial range for the value map is crucial for balancing exploration efficiency and prediction accuracy.
0.68
0.00
30
40
Range of value map prediction (m)
Figure 5: Comparisons of different spatial ranges for value map prediction.
Oracle obstacle map. Table 4 we replace the predicted obstacle map with the ground truth obstacle map for path planning during inference, while maintaining to use the predicted value map for long-term goals. Using the oracle obstacle map improves the performance by 0.074 final coverage and 0.054 on AUCs, but is far from per- fect. This suggests that the major bottleneck is the value map prediction.
Table 4: Ablation study on using the oracle map for obstacle avoidance at inference.
Obstacle Map
Final Cov.
AUC'
Predicted Oracle
0.734 20.122
0.526 10.112
0.808 s0.115
0.580 10.
Multi-task training. We also explore the influence of multi-tast learning in predicting the value map Mc, and the obstacle map Oc,- For comparison, we train two separate models that use the same input to predict M and O., respectively. The results show that multi-task learning improved the precision of obstacle prediction to 0.805, exceeding the 0.754 achieved by single-task learn- ing. Table 5 further demonstrates that multi-task learning achieves better performance, indicating that the two tasks
Table 5: Comparison of single-task and multi-task learning for the value map and obstacle map prediction.
Strategy Final Cov.
AUCs
Single-task 0.712 0.501
Multi-task 0.734 +0.122 0.526 +0.12
complement each other enhance learning.
6 CONCLUSION
In this paper, we tackle the challenging problem of active 3D mapping of unknown environments. We introduce new dataset, AiMDoom, designed to benchmark active mapping in indoor scenes with four difficulty levels. Our evaluations of existing methods on the AiMDoom dataset reveal shortcomings of short-sighted next-best-view prediction in complex large indoor environments. Hence, we propose the next-best-path (NBP) method, which integrates mapping progress encoder, coverage gain decoder and an obstacle map decoder. The NBP model can efficiently reconstruct unseen environments guided by predicted long-term goals, achieving state-of tftte-of-theatt the-art performance on both the MP3D and AiMDoom datasets. However, observe considerable room for improve- ment in more difficult levels of our dataset, and the major limitation lies in long-term goal prediction rather than obstacle map prediction.
10

---

Published as a conference paper at ICLR 2025
7 ACKNOWLEDGEMENTS
This project was funded in part by the European Union (ERC Advanced Grant explorer Funding ID #101097259). This work was granted access to the HPC resources of IDRIS under the allocation 2024-AD01 1014703RI made by GENCI. We thank Ruijie Wu for his valuable contribution to mesh editing throughout the AiMDoom dataset generation.
REFERENCES
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. Advances in neural information processing systems, 32, 2019.
Andreas Bircher, Mina Kamel, Kostas Alexis, Helen Oleynikova, and Roland Siegwart. Receding horizon" next-best-view" planner for 3d exploration. In 2016 IEEE international conference on robotics and automation (ICRA), pp. 1662-166. IEEE, 2016.
Chao Cao, Hongbiao Zhu, Howie Choset, and Ji Zhang. Tare: A hicrarchical framework for effi- ciently exploring complex 3d environments. In Robotics: Science and Systems, volume 5, pp. 2, 2021.
Nicolò Cesa-Biancis Bianchi, Claudio Gentile, Gábor Lugosi, and Gergely Neu. Boltzmann Exploration Done Right. In Advances in Neural Information Processing Systems, 2017.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3D: Learning from RGB-D Data in Indoor Environments. In International Conference on 3D Vision, 2017.
Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, and Others. Shapenet: An Information- Rich 3D Model Repository. In arXiv Preprint, 2015.
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning explore using active neural slam. arXiv preprint wxXiv:2004.05155 2020.
Shizhe Chen, Thomas Chabal, Ivan Laptev, and Cordelia Schmid. Object Goal Navigation with Recursive Implicit Maps. In International Conference Intelligent Robots and Systems, 2023.
Xiao Chen, Quanyi Li, Tai Wang, Tianfan Xue, and Jiangmiao Pang. GenNBV: Generalizable Next- Best-View Policy for Active 3D Reconstruction. In Conference Computer Vision and Pattern Recognition, 2024.
Titus Cieslewski, Elia Kaufmann, and Davide Scaramuzza. Rapid exploration with multi-rotors: A frontier selection method for high speed flight. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2135-2142. IEEE, 2017.
Blender Online Community, Blender - A 3D Modelling and Rendering Package, 2018. URL URLhttp:
w/www.blender.o org
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias
Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5828-5839 2017.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366-333 3388336666385,,, 2021.
Matt Deitke, Winson Han, Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mot- taghi, Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, Luca Weihs, Mark Yatskar, and Ali Farhadi. Robothor: An open simulation-to-reat embodied ai platform. In IEFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2020.

---

Published as a conference paper at ICLR 2025
Ziyue Feng, Huangying Zhan, Zheng Chen, Qingan Yan, Xiangyu Xu, Changjiang Cai, Bing Li, Qilun Zhu, and Yi Xu. Naruto: Neural active reconstruction from uncertain target observations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 21572-21583, 2024.
Georgios Georgakis, Bernadette Bucher, Anton Arapin, Karl Schmeckpeper, Nikolai Matni, and Kostas Daniilidss. Uncertainty-Driver Planner for Exploration and Navigation. In International Conference on Robotics and Automation, 2022.
Antoine Guédon, Pascal Monasse, and Vincent Lepetit. Scone: Surface Coverage Optimization in Unknown Environments by Volumetric Integration. In Advances in Neural Information Process- ing Systems, 2022.
Antoine Guédon, Tom Monnier, Pascal Monasse, and Vincent Lepetit. Macarons: Mapping and Coverage Anticipation with RGB Online Self-Supervision. In Conference on Computer Vision and Puttern Recognition, 2023.
Guillaume Hardouin, Julien Moras, Fabio Morbidi, Julien Marzat, and EI Mustapha Mouaddib. Next-best-view planning for surface reconstruction of large-scale 3d environments with multiple uavs. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp- 1567-1574. IEEE, 2020.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. In Conference Computer Vision and Pattern Recog- nition, 2018.
Keifer Lee, Shubham Gupta, Sunglyoung Kim, Bhargav Makwana, Chao Chen, and Chen Feng. So- nerf: Active view planning for nerf using surrogate objectives. arXiv preprini arXiv:23/2.03266. 2023.
Soomin Lee, Le Chen, Jiahao Wang, Alexander Liniger, Suryansh Kumar, and Fisher Yu. Uncer- tainty Guided Policy for Active Robotic 3D Reconstruction Using Neural Radiance Fields. IEEE Robotics and Automation Letters, 7(4), 2022.
Weiwei Liu, Ivor W. Tsang, and Klaus-Robert Müller. An Easy-To-Hard Learning Paradigm for Multiple Classes and Multiple Labels. Journal of Machine Learning Research, 18(94), 2017.
Chongshan Lu, Fukun Yin, Xin Chen, Wen Liu, Tao Chen, Gang Yu, and Jiayuan Fan. A large- scale outdoor multi-modal dataset and benchmark for novel view synthesis and implicit scene reconstruction. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 7557-7561 2023.
Hidenobu Matsuki, Riku Murai, Paul HJ Kelly, and Andrew Davison. Gaussian splatting slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 18039- 18048, 2024.
Miguel Mendoza, J Irving Vasquez-Gomez, Hind Taud, L. Enrique Sucar, and Carolina Reta. Su- pervised learning of the next-best-view for 3d object reconstruction. Pattern Recognition Letters, 133:224-231 2020.
Volodymyr Mnih. Playing atari with deep reinforcement learning. arXiv preprint arXiv: 1312,5600 2013.
Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven Mcdonagh, Nils Y. Hammerla, and Bernhard Kainz. Attention U-Net: Learning Where to Look for the Pancreas. In Medical Imaging with Deep Learning, 2018.
Xuran Pan, Zihang Lai, Shiji Song, and Gao Huang. ActiveNeRF: Learning Where See with Uncertainty Estimation. In European Conference Computer Vision, 2022.
Daryl Peralta, Joel Casimiro, Aldrin Michael Nilles, Justine Aletta Aguilar, Rowel Atienza, and Rhandley Cajote. Next-Best View Policy for 3D Reconstruction. Workshop at European Conference Computer Vision, 2020.
12

---

Published as a conference paper at ICLR 2025
Julio A Placed, Jared Strader, Henry Carrillo, Nikolay Atanasov, Vadim Indelman, Luca Carlone, and José A Castellanos. A survey on active simultaneous localization and mapping: State of the art and new frontiers. IEEE Transactions on Robotics, 39(3):668-11755, 2023.
Santhosh K. Ramakrishnan, Ziad Al-Halah, and Kristen Grauman. Occupancy Anticipation for Efficient Exploration and Navigation. In European Conference on Computer Vision, 2020.
Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexan- der Clegg, John M. Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X. Chang. Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-Matterpor 3D Dataset (HM3D): 1000 Large-Scall 3D Environments for Embodied AI. In Thirty-fifth Conference on Neural In- formattion Processing Systems Datasets and Benchmarks Track, 2021.
Ram Ramrakhya, Eric Undersander, Dhruv Batra, and Abhishek Das. Habitat-web: Learning em- bodied object-search strategies from human demonstrations at scale. In International Conference on Computer Vision, pp. 1173-5183, 2022.
Yunlong Ran, Jing Zeng, Shibo He, Jiming Chen, Lincheng Li, Yingfeng Chen, Gimhee Lee, and Qi Ye. Neurar: Neural Uncertainty for Autonomous 3D Reconstruction with Implicit Neural Representations. IEEE Robotics and Automation Letters, 8(2), 2023.
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Ye Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3D Deep Learning with PyTorch3D. In arXiv Preprint, 2020.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in neural information processing systems, 32, 2019.
Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. Habitat: A Platform for Embodied AI Research. In International Conference Computer Vision, 2019.
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Brik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gillingham, Elias Mueggler, Luis Pesqueira. Manolis Savva, Dhruv Batra, Hauke M. Strasdat, Renzo De Nardi, Michael Goesele, Steven Lovegrove, and Richard Newcombe. The Replica Dataset: A Digital Replica of Indoor Spaces. In arXiv Preprint, 2019.
Yuezhan Tao, Yuwei Wu, Beiming Li, Fernando Cladera, Alex Zhou, Dinesh Thakur, and Vijay Kumar. Seer: Safe efficient exploration for acrial robots using learning to predict information gain. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 1235- 1241. IEEE, 2023.
Erik Wijmans, Abhishek Kadian, Ari Morcos, Stefan Lee, Irfan Essa, Devi Parikh, Manolis Savva, and Dhruv Batra. Dd-ppo: Learning near-perfect pointgoal navigators from 2.5 billion frames. arXiv preprint arXiv:1911.00357 2019.
Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Liang Pan Jiawei Ren, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu. Omniobject3d: aace-looyy 3d object dataset for realistic perception, reconstruction and generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Markus Wulfmeier, Alex Bewley, and Ingmar Posner. Incremental adversarial domain adaptation for continually changing environments. In 2018 IEEE International conference robotics and automation (ICRA), pp. 4489-4495. IEEE, 2018.
Fei Xia, R. Zamir, Amir, Zhi- Yang He, Alexander Sax, Jitendra Malik, and Silvio Savarese. Gib- son Env: Real-World Perception for Embodied Agents. Conference on Computer Vision and Pattern Recognition, 2018
Zhefan Xu, Christopher Suzuki, Xiaoyang Zhan, and Kenji Shimada. Heuristic- based incremen- tal probabilistic roadmap for efficient uav exploration in dynamic environments. In 2024 IEEE International Conference Robotics and Automation (ICRA), pp. 11832-11838 IEFF, 2024.
13

---

Published as a conference paper at ICLR 2025
Brian Yamauchi. A Frontier-Basee Approach for Autonomous Exploration. In IEEE International Symposium on Computational Intelligence Robotics and Automation, 1997.
Zike Yan, Yuxin Tian, Xuesong Shi, Ping Guo, Peng Wang, and Hongbin Zha. Continual Neural
Mapping: Learning an Implicit Scene Representation from Sequential Observations. Interna- tional Conference on Computer Vision, 2021.
Zike Yan, Haoxiang Yang. and Hongbin Zha. Active Neural Mapping. In International Conference on Computer Vision, 2023.
Zhenghang Yuan, Lichao Mou, Qi Wang, and Xiao Xiang Zhu. From easy to hard: Learning language-guided curriculum for visual question answering remote sensing data. IEEE trans- actions on geoscience and remote sensing, 60:1-11, 2022.
Rui Zeng, Wang Zhao, and Yong-Jin Liu. Pc-Nbv: A Point Cloud Based Deep Network for Efficient Next Best View Planning. In International Conference on Intelligent Robots and Systems, 2020.
Huangying Zhan, Jiyang Zheng, Yi Xu, Ian Reid, and Hamid Rezatofighi. Activermap: Radiance Field for Active Mapping and Planning. In arXiv Preprint, 2022.
Boyu Zhou, Yichen Zhang, Xinyi Chen, and Shaojie Shen. Fuel: Fast exploration using incre- mental frontier structure and hierarchical planning. IEEE Robotics and Automation Letters, 6(2): 779-786, 2021.
Qian-Yr Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A Modern Library for 3D Data Process- ing. In arXiv Preprint, 2018.
14

---

Published as a conference paper at ICLR 2025
APPENDIX
A DATASET
Dataset construction. To ensure that each map offers full accessibility for various robotic platforms such unmanned aerial vehicles (UAVs) and wheeled robots, configure all doors and windows to remain open during map generation. However, we observe that Obsidian does not consistently guarantee accessibility all areas. To resolve this, we manually edited each scene ensure the traversability of windows, doors, and hidden passages.
Dataset statistic. To calculate navigation complexity, we sampled location pairs from four envi- ronments ranging in difficulty from simple to insane, at rates of 1%, 0.1%, 0.05%, and 0.02%, constrained by computational resource limitations. Consequently, our data effectively represents lower bound of navigation complexity. Despite this conservative sampling approach, our dataset remains the most complex one, offering a significant challenge for future research.
Figure 6 presents more examples of maps from our dataset, showcasing various levels and diverse scenarios.
B DETAILS OF MAPPING PROCESS ENCODER
We provide more details of the Mapping Process Encoder of our proposed approach in this section.
The mapping encoding is predicted from both the current reconstruction progress and historical tra- jectory data. At each time step 2 we construct and refine a surface point cloud Pa by integrating information from newly captured depth map D, S2 + R and merging it with our existing recon- structed point cloud. For each camera pose e (ce (or), we transform the corresponding depth
map Dj into a set of 3D points. This transformation makes use of the camera's intrinsic matrix K 33x3 and the pose matrix T; E (3), derived from the 6D pose c;:
Pacr (u, v) T. (D:(u..) K-1 [ 11
(u, EΩ,
(4)
where Q2 C IR2 represents the domain of the depth map. We accumulate points over time:
P. = P {Peurface (u, v) (u, Ω2 De(u, 0}.
(5)
To enhance scalability and generalization, we introduce a slice mapping approach that transforms the point cloud into set of K images. We begin by filtering the point cloud based the camera's position:
pf = {p- (p.. Pu. E Pe lp, - ze, Sr and Pp= - Ze. rr.
(6)
where r is the radius of observation window and (Ie,) Vee Zea) is the current camera position. We then divide Pf. into n equal vertical slices along the Y-axis, ynni and Yuamar come from a defined exploration bounding box, as Guédon et al. (2023) did:
San. {p (Pz, Py. Py.P.) Pl/ Ymin + (j 1)hatice S Py < Ymin + j.haice
(7)
where halioe (Vmax Mmin)/n /n and 1,...,n. n.. Each slice Sou. mapped to image la.s of
size H x W using a projection function & : R RR2
@(p) = (1 (Px- Iz. + +r) W 2r
(p= 2c, +r) H 1)
(8)
2r
This projection centres the camera in the middle of the image. Finally, we calculate the point density for each pixel (u, v)) the image Ior.s
Ie.s(u,))
== Sc..
6(0(p) - (u, u))dp.
(9)
Here, 8(-) is the Dirac delta function and p represents points from the slice Sc... This process yields n density images le.. for each time step t, effectively transforming 3D point cloud data into 2D representations.
15

---

Published as a conference paper at ICLR 2025
Figure 6: More maps from our dataset. Rows from top to bottom represent increasing scene complexity, categorized into four levels: Simple, Normal, Hard, Insane.
In addition, we apply a similar approach to project the camera's 's historical trajectory, resulting in single 2D image. We filter the camera's historical positions based on their proximity to the current camera position in the XZ.pllan, using the same threshold Tzz:
cf
{ct pos
(æk, Vk: 24) k < t, Iz Fzl S and 2 21 S Taz}
(10)
We then map these filtered positions onto single image He, of the same size H x W:
Ha, (u, = Σ 8(0(ch - (u, v))
(11)
pos EC!
This results in a single-density image He, representing the camera's historical trajectory near its current position. To synthesize the information obtained, define set Ec, encapsulating the entirety the current exploration embedding: E., = (I... Lo H..)
C EXPERIMENTS
Detailed quantitative results. Table and Table show our superior performance on both the AiMDoom training set and the test set. Furthermore, offer detailed results for each test scene in MP3D, as illustrated in T
Additional ablation study. We study the impact of different spatial range information used to predict the next best path by training four different models on the AiMDoom Normal level training split. These models processed input crop sizes ranging from 20m 20m to 50m 50m, with each
model tasked with predicting a value map and an obstacle map within a 40m x 40m area. The Tabbe shows the results.
16

---

Published as a conference paper at ICLR 2025
Table 6: Evaluation results for each test scene on MP3D dataset.
Comp, (%)
Comp, (em) J
Scene
Rooms
Random
FBE
UPEN
OccAnt
ANM
NBP (ours)
Random
FBE
UPEN
OceAnt
ANM
NBP (ours)
Gdvggr+
81.78
82.39
5.14
g2677
82.96
6.19
HxpKQ*
52.70
pl.c4w
66.09
66.76
71.34
11.82
11.51
YmJkq
68.32
60.47
79.35
24.61
11.85
15.77
mean
45.67
71.18
69.06
71.72
73.15
79.38
26.53
9.78
10.60
9.40
9.11
6.78
Table Evaluation results on AiMDoom dataset (Simple and Normal).
AiMDoom Simple
AiMDoom Normal
Seen
Unseen
Seen
Unseen
Final Cov.
AUC
Final Cov.
AUC
Final Cov.
AUC
Final Cov.
AUC
Random Walk
0.362
0.306
0.323
0,270
0.198
0.159
0.190
0.152
#0.175
20.156
20.156
20.135
a0.125
20.104
a0.124
10.103
FBE
0.770
0.628
0.760
0,605
0.564
0,423
0.565
0,415
#0.163
20.147
#0.174
#0.171
#0.171
#0.127
#0.139
10.109
SCONE
0.597
0.482
0.577
0.483
0,421
0.315
0,412
0.313
20.177
20.138
#0.173
20.138
+0.138
#0,102
#0.114
+0.087
MACARONS
0.600
0,483
0.599
0,479
0.442
0,332
0.418
0.314
#0.176
#0.145
#0,200
#0.172
#0.135
40.104
#0.120
#0.088
NBP (Ours)
0.870
0.697
0.879
0.692
0.746
0.538
0.734
0.526
20.121
20.134
20.142
20.156
20.152
20.142
10.142
20.112
Table 8: Evaluation results on AiMDoom dataset (Hard and Insane).
AiMDoom Hard
AiMDoom Insane
Seen
Unseen
Seen
Unseen
Final Cov.
AUC
Final Cov.
AUC
Final Cov.
AUC
Final Cov.
AUC
Random Walk
0.121
0,086
0.124
0.088
0.070
0.048
0.074
0.050
#0.062
20.035
FBE
0.426
0.310
0.425
0.311
0.313
0.226
0.330
0.239
20.119
20.091
20.114
20.080
#0.082
20.066
#0.097
10.079
SCONE
0.271
0.199
0.290
0.210
0.204
0.146
0.196
0.140
#0.100
#0.172
#0.093
40,072
#0.069
+0.052
#0.079
+0.060
MACARONS
0.316
0.202
0.302
0.218
0.201
0.143
0.192
0.139
#0.106
+0.074
#0.097
+0,070
#0.068
#0.051
+0.078
+0.058
NBP (Ours)
0.627
0.430
0.618
0.432
0.486
0,315
0.472
0,312
20.144
20.111
20.153
20.115
20.106
10.047
10.095
10.073
The results indicate that optimal performance is achieved when the input crop size corresponds to the crop size of the area being predicted. This is due the fact that when the input crop size is the
either smaller or larger than that of the output maps, predictive errors arise. Specifically, input crop size is too small, it limits the model's ability formulate effective long-term objectives. Conversely, when the input crop size is too large, the predictions for obstacles near the camera become less accurate, negatively impacting both exploration and reconstruction efficiency.
We also investigate the different strategies in inference. We conducted this experiment on the AiM- Doom Normal level, extending our previous ablation studies. Table sOssows the results, the Original Strategy adheres to the original approach of updating long-term goals upon completing path, while the New strategy updates goals each step.
The results indicate that the New Strategy, which frequently updates long-term goals, performs worse than the Original Strategy. This inferior performance mainly due to the lack decision
17

---

Published as a conference paper at ICLR 2025
Table 9: Comparison of different spatial ranges of information used predict the next best path.
Range
20m
20m
30m
30m
40m
40m
50m
50m
Final Cov.
0.630 +0.151
0.691 -0.10
0.734 +0.142
0.647 +0.144
AUCs
0.469 10.107
0.501 #0.106
0.526 +0.112
0.457 +0.106
continuity in the New Strategy, where the agent frequently changes its long-term goals. Such fre- quent shifts can cause the agent to oscillate be- tween paths, wasting movement steps, particu- larly as our experiments were conducted with a limited number of steps. Additionally, the pre- dictive accuracy of the value map is not perfect,
Table 10: Comparison of Different Strategies
Strategy
Final Cov.
AUCs
Original strategy
0.734 20.142
0.526 20.112
New strategy
0.432 #0.168
0.367 =0.135
and forecasting over long distances naturally tails uncertainty. New Strategy accumulates more predictive errors by recalculating predictions at every step, and frequent updates in decision-making exacerbate these errors.
Despite these challenges, our results still surpassed the performance previous state-of-the-as next-best-vies (NBV) methods, detailed in Table. This suggests that predicting coverage gains over long distances can indeed benefit efficient active mapping, even when the goal is updated at each step.
(a) Ground truth mesh
(b) NBP (Ours)
Figure 7: Failure case 1: Our method initially prioritizes the exploration of higu-value areas, inad- vertently neglecting regions of secondary importance. Thus, it results in incomplete reconstruction in the initial area of the beginning trajectory.
(a) Ground truth mesh
(b) NBP (Ours)
Figure 8: Failure case This scene contains multiple narrow areas, prompting our method to depend more heavily on our precise prediction of obstacles. Under these challenging conditions, our approach may overlook exploring this area.
Failure cases. As Figure. 7 and Figure. illustrated,, we also show that in very complex environ- ments, we could only achieve about 65% coverage. This is because, in complex environments, our method prioritizes the exploration of areas with multiple valuable goals, ignoring places of lesser
18

---

Published as a conference paper at ICLR 2025
current value. After the initial exploration is complete, it is likely to explore other regions, overlook- ing previously encountered areas with higher value. Consequently, developing methods that aim to achieve global optimum is a promising and valuable direction for future research.
19


Reviews:
[{"review_id": "ixkz265S2l", "reviewer": "Reviewer_1qFw", "summary": "This paper proposes a method to predict paths during mapping that optimize accumulated coverage. The goal is to cover the environment in a minimal time using only a depth sensor.\n\nPoint clouds and the robot trajectory are input to an encoder which yields a latent that is decoded into a value map representing the coverage gain and an occupancy map. The cell in the coverage gain map with the highest value is set as the long term goal.\n\nThe authors proposed a new dataset AiMDoom played out on a game (simulation) environment, since active methods unlike passive SLAM have to be evaluated on the same environment with different actions/trajectories.", "questions": "Q1. L068: You write \"scene uncertainty does not directly align with the ultimate objective of 3D mapping\". Is the uncertainty of predicted occupancy not the uncertainty of the 3D map? What do you mean here?\n\nQ2: The computed information gain during training is using the ground-truth ( eq 2) while inference uses the ground-truth instead. It is not clear whether the use of the ground-truth \n\nQ3: The authors should clarify the first term in eq. 3. What does it mean ground-truth coverage gain?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1: the new dataset has complex challenging lay-outs.\n\nS2:  the new dataset has diversity with a lot of opportunities to evaluate generalization\n\nS3: the decoded occupancy map includes unseen places, behind walls, for example.", "weaknesses": "W1: The main weakness of the paper is the scope of active mapping addressed is only coverage rather than the map itself. While coverage is indeed significant it assumes that  3D reconstructions are error-free (the M in SLAM). Moreover, poses are assumed accurate, an assumption far from reality.\n\nW2: The paper is set in a very narrow context by ignoring the literature on Active SLAM. In particular, active mapping has been based on first principles of information theory. See the excellent exposition here Julio A Placed, Jared Strader, Henry Carrillo, Nikolay Atanasov, Vadim Indelman, Luca Carlone,\nand José A Castellanos. A survey on active simultaneous localization and mapping: State of the art and new frontiers. IEEE Transactions on Robotics, 2023\n\nI think the authors would benefit a lot in rethinking their approach and rewriting their paper by reading this article.\n\nW3: The approach is very similar to (Georgakis, 2022). While Georgakis et al. predict occupancy probability and model uncertainty, here the authors predict occupancy and a value map that should have the interpretation of information gain/uncertainty. While Georgakis' objective is point-goal navigation one can use its exploration policy as a pure mapper. Georgakis' value map is based on explicit computation of covariance from ensembles without the use of any ground-truth.\nFinally, Georgakis choose long-term goal and then estimate paths based on occupancy maps, similar to the approach here.\n\nW4: The main idea of exploration is trying to choose paths where the measurements are not predictable by the occupancy paths. The expression in (2), however, defines the gain as minimal error to the ground-truth. This will not encourage the agent to go to new unvisited directions but rather to directions where the prediction error will be very small.\n\nW5: There is considerable literature that has been ignored in related work and experimental comparisons. In particular, we would like to see comparisons with \n\na.  D. S. Chaplot, D. Gandhi, S. Gupta, A. Gupta, and R. Salakhutdinov, “Learning to explore using active neural SLAM,” in Proc. Int. Conf. Learn. Representations, 2020.\n\nb. A. Bircher, M. Kamel, K. Alexis, H. Oleynikova, and R. Siegwart, “Recedinghorizon“next-best-view”plannerfor3Dexploration,”inProc. IEEE Int. Conf. Robot. Autom., 2016, pp. 1462–1468.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4nx28qY6xO", "reviewer": "Reviewer_E6rF", "summary": "In this paper, the authors focus on improving the reconstruction efficiency of active mapping in a new environment. Previous methods mainly predict the next best viewpoint near the current location and are prone to getting stuck in local areas. \n\nInstead, the authors propose the leverage accumulated history information to find a long-term goal which could bring the largest gain. At the same time, an obstacle map is predicted to find the long-term goal efficiently. \n\nIn addition to the proposed method, the authors also introduce a new synthetic dataset which has more complicated structures and larger diversities of map than previous datasets. \n\nExperiments on one public dataset and the new dataset demonstrate that the proposed method outperforms prior methods significantly.", "questions": "See the weaknesses section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The strengths of this paper are as follows:\n\n1.\tA more complicated dataset (AiMDoom) for active mapping. Compared with other either synthetic or real datasets, such as Replica, RoboTHOR, MP3D, and ScanNet, HM3D, the new dataset AiMDoom has more scenes, larger area size and different levels of difficulty. Intricate geometries and layouts, small doors and narrow corridors, the high diversity of scenes bring new challenges to the active mapping task. This benefits the whole community. \n\n2.\tA novel approach for active mapping. The authors propose to predict the next best path (NBP) to find the next optimal location instead of directly predicting the one close to current position of the agent. The value map in NBP provides the best location and the obstacle map allows to use Dijkstra algorithm to find the shortest path from current location to the goal. This is a useful combination and brings inspiration to following works. \n\n3.\tImpressive performance. The proposed approach obtains state-of-the-art performance on MP3D dataset and gives significantly better results than previous methods like MACARONS on the new dataset. Ablation studies show the effectiveness of the obstacle map.\n\n4.\tThe paper is well-organized and easy to read.", "weaknesses": "I don’t see obvious weaknesses of this paper. Some of my concerns about the method and the dataset are as follows.\n\n1. In L243, the point clouds are cropped at the current location of the agent. I am curious how does it work and what kind of parameters are used? My understanding is that the crop size may influence how much history information is used for the next path prediction.\n\n2. In L245, the 3D point clouds are projected onto 2D image to simplify the processing. This strategy works for scenes with a single layer but may lose generalization ability in scenes with multiple layers as part of the depth information is discarded.\n\n3. In the ablation study, the efficacy to the final reconstruction results of both the obstacle map and multi-task training are tested, however, it would be good to see the accuracy of the obstacle map itself and the value map itself instead of final reconstruction accuracy. \n\n4. The new dataset is a synthetic dataset, so the domain gap may exist. As far as I know, a new version of Scannet dataset has been released. It has more scenes with more complicated geometry and structures. In the future, it might be possible to reorganize this new Scannet dataset and make a real complicated dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RYQQBGDu6p", "reviewer": "Reviewer_aPVh", "summary": "This paper proposes a learning-based method to the problem of active 3D mapping of unknown environments. The method is hinged on next-best-path (NBP). It integrates a mapping progress encoder, a coverage gain decoder and an obstacle map decoder. The coverage gain and the obstacle map are used to compute the NBP. The NBP can direct the robot to reconstruct unseen environments with predicted long-term goals, achieving state-of-the-art performance on both the MP3D and AiMDoom datasets. The paper also contributes a dataset, AiMDoom, designed to benchmark active mapping in indoor scenes.", "questions": "No.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The idea of estimating NBP has good merit and shows promising results. The method is overall reasonably designed and the results are good. The evaluation demonstrates the strength of the proposed method.", "weaknesses": "The paper claims that the main novelty is the idea of next best path planning. It shows that NBP performs better than NBV which is reasonable and convincing. However, the method how NBP is computed is rather simplistic and the major technical components are actually the reconstructed map encoder and the two map decoders. With the estimated value map and obstacle map, NBP is computed in a straightforward way. On the other hand, training a network to predict value maps for scene coverage has good merit.\n\nSome of the paths computed by the proposed method are too close to the walls, see Fig 4(b) left, making them collision-prone. The visualization of paths does not have to show shadows; showing the path on the ground could be clearer.\n\nThe experiment of spatial range of long-term goal is interesting. However, it would be more useful if more test can be conducted to find out how to choose the size of range for a given scene.\n\nThe proposed method is trained on the train split of AiMDoom. It is unclear whether the other alternative methods being compared were trained on AiMDoom or pretrained on other datasets. If so, the comparison would be unfair. More explanation is needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B0FCQP4YVP", "reviewer": "Reviewer_Ayjg", "summary": "This paper proposes a path-planning algorithm for efficient 3D mapping. The core components of the NBP are the coverage gain decoder and obstacle map decoder, which are leveraged for long-term goal selection. The experiments showcase significant improvements against baselines. Besides, this paper also proposes a  new dataset called AiMDoom, which includes 400 scenes with different level of diffculty.", "questions": "- Do the obstacle map and value map encompass the entire scene? This could result in significant computational costs in large-scale environments.\n\n- If the long-term goal is updated at each step, does this strategy enhance performance?\n\n- In Table 2, `comp.` represents the average minimum distance between ground truth vertices and observations. However, the HM3D depth data should be noise-free. If I haven't overlooked any details, why is the `comp.` metric considered valid?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Developing methods for highly efficient exploration is an intriguing topic with potential applications across various navigation tasks. This paper provides comprehensive details on NPV techniques, and the dataset is expected to benefit the community by supporting further investigation into exploration strategies.", "weaknesses": "- The scenes in AiMDoom contain minimal furniture or objects, resulting in mostly open space. This does not align with real-world environments, making these scenes suboptimal for training and evaluation purposes.\n\n- The states described in papers L199 and L373 indicate that the proposed method operates within a 3-DoF domain. However, NBV tasks often involve planning in a 6-DoF camera pose space. Moreover, baseline methods, such as MACARONS and SCONE, support 6-DoF camera pose planning. A discussion is needed to explain why this paper considers only a 3-DoF setting.\n\n- For 3-DoF trajectory planning, several well-known works exist, such as [1]. The authors should discuss why this paper’s approach offers advantages over previous works.\n\n[1] TARE: A Hierarchical Framework for Efficiently ExploringComplex 3D Environments", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)