Understanding while Exploring: Semantics-driven Active Mapping
Authors: Liyan Chen
Huangying Zhan
Hairong Yin
Yi Xu
Philippos Mordohai
Venue: NeurIPS 2025 poster ()
Keywords: Active Semantic Mapping
Active Mapping
Gaussian Splatting
Exploration
Primary area: applications
TL;DR: This paper presents the first dense active semantic mapping framework using 3D Gaussian Splatting, enabling efficient exploration and robust mapping with sparse semantic representation.

Abstract: Effective robotic autonomy in unknown environments demands proactive exploration and precise understanding of both geometry and semantics. In this paper, we propose ActiveSGM, an active semantic mapping framework designed to predict the informativeness of potential observations before execution. Built upon a 3D Gaussian Splatting (3DGS) mapping backbone, our approach employs semantic and geometric uncertainty quantification, coupled with a sparse semantic representation, to guide exploration. By enabling robots to strategically select the most beneficial viewpoints, ActiveSGM efficiently enhances mapping completeness, accuracy, and robustness to noisy semantic data, ultimately supporting more adaptive scene exploration. Our experiments on the Replica and Matterport3D datasets highlight the effectiveness of ActiveSGM in active semantic mapping tasks.

Understanding while Exploring: Semantics-driven Active Mapping
Liyan Chen +1 Huangying Zhan 2 Hairong Yin 3, Yi Xu Philippos Mordohai
Stevens Institute of Technology 2 Goertek Alpha Labs 3 Purdue University
Abstract
Effective robotic autonomy in unknown environments demands proactive explo- ration and precise understanding of both geometry and semantics. In this paper, we propose ActiveSGM, an active semantic mapping framework designed to predict the informativeness of potential observations before execution. Built upon a 3D Gaussian Splatting (3DGS) mapping backbone, our approach employs semantic and geometric uncertainty quantification, coupled with a sparse semantic repre- sentation, to guide exploration. By enabling robots to strategically select the most beneficial viewpoints, ActiveSGM efficiently enhances mapping completeness, accuracy, and robustness to noisy semantic data, ultimately supporting more adap- tive scene exploration. Our experiments the Replica and Matterport3D datasets highlight the effectiveness of ActiveSGM in active semantic mapping tasks.
Introduction
Mobile robots are expected to play a significant role in human-centered environments, such as warchouses, factories, hospitals, and homes, well as dangerous settings, such mines and nuclear facilities. Rich and accurate geometric and semantic representations are necessary in these scenarios so that robots can understand, interpret, and interact meaningfully with their surroundings. For instance, in automated warehouses, robots are required to recognize various items and place them in the correct sorting zones accordingly. Scene understanding enabled by a semantic map that is linked to the geometric map []] which represents the spatial layout of an environment, and enriches it with high-level information such as object categories, surface labels, and functional affordances (2(5) Such maps are critical for range of tasks including navigation, inspection, object manipulation, human-robot interaction, and long-term autonomy.
Despite substantial advances in semantic mapping, most current approaches are unable to determine the most informative path for the robot. Instead, they passively rely on externally determined trajectories or predefined exploration strategies 66 leading to incomplete or suboptimal scene understanding. In this paper, we present an approach of active semantic mapping that seeks close the loop between perception and action. This allows agents to plan their next moves and observations improve the quality, completeness, and efficiency of the semantic map. Our approach, named ActiveSGM (Active Semantic Gaussian Mapping), is the first active semantic mapping system based on radiance fields, enabling rapid exploration, efficient environment understanding, and high-fidelin real-time rendering, ultimately leading to more intelligent and efficient robotic behaviors. ActiveSGM aims to infer the semantic labels of all visible surfaces, without favoring any particular label.
To select the most informative views for the robot, we seek to quantify both geometric and semantic uncertainty. At the geometric level, uncertainty typically measured by the expected error in the estimated 3D coordinates 10 [2] At the semantic level, uncertainty estimation primarily captures ambiguity among semantic classes. Recent surveys on semantic uncertainty quantification [5] found that it is inherently dependent upon the choice of semantic representation.
"Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

The choice of semantic representation plays a critical role in semantic mapping systems, which commonly adopt two forms: probability distributions or embeddings. For distribution- basedd repre- sentations, existing methods (e.g.. [8)188 employ either hard or soft assignment strategies. Hard assignments, such as one-hot encoding, strictly assign a single label to each pixel. In contrast, soft assignments allocate a complete categorical probability distribution to each 3D primitíve, naturally capturing uncertainty but also incurring higher memory requirements as the number of categories grows. Alternatively, embedding-
representations can also be viewed as a form of soft as- signment. Methods like [9] [9] utilize features, such as those from DINO 20 or CLIP 2I1, to encode semantic embeddings. However, these embeddings high-dimensional, posing challenges for storage and real-time rendering in large scenes. Consequently, some approaches compress the spaces, such as the dirre-dimensional RGB color space. The dimen-
features into sion of the embedding feature space directly determines the effectiveness of class discrimination. High-dimensional embeddings, like those from DINO CLIP, provide sufficiently expressive feature space to effectively distinguish categories. However, as embeddings become compressed, for instance into RGB space, color blending during multi-view reconstruction inevitably occurs, producing blended colors that may correspond to unrelated categories instead of the original ones.
In this paper, address semantic representation under the closed-vocabulary assumption, adopting probability distribution approach that we argue offers better categorical discrimination. In Section 3 discuss how to store high-dimensional probability distributions within proposed sparse semantic representation.
To summarize, we propose the first dense active semantic mapping system built upon 3D Gaussian Splatting (3DGS) backbone, which integrates semantic-aware mapping and planning for active reconstruction. This enables the robot to construct a more accurate geometric map and a richer semantic map with fewer observations. Our method addresses several key challenges:
Semantics-aware exploration: We design a novel semantic exploration criterion that enhances semantic coverage and facilitates disambiguation across observations during exploration.
High-dimensional semantic representations and memory footprint: We adopt a closed- vocabulary setting and introduce a sparse semantic representation that retains the top-k most probable categories, reducing memory overhead without sacrificing semantic richness.
Robustness to noisy semantic observations: Unlike prior works that rely on ground-truth labels, real- world deployment requires handling noisy semantic predictions. We use a pre-trained segmen- tation model to generate these inputs and design our pipeline tolerate and progressively refine them, achieving high segmentation quality.
2 Related Work
In this section, review prior work, starting from dense SLAM, active mapping, semantic mapping, and concluding with active semantic mapping. We focus on methods utilizing either Neural Radiance Fields (NeRF) 24 Gaussian Splatting (GS) 27 the representation.
Dense SLAM. Autonomous robotics relies foundational capabilities such localization, map- ping, planning, and motion control (88) The need to realize these capabilities has spurred advance- structure- fronee---oo-mmmmoe mottoontiion (SfM) [31], and
ments various areas, including visual odometry 22 30
Simultaneous Localization and Mapping (SLAM) 34 10 35). For surveys of the impact of radiance fields SLAM and robotics in general, we refer readers to (36-33] 27]. Progress in radiance fields has given rise to multitude of dense SLAM methods, that estimate depth for almost every pixel of the input images, using NeRF (or other implicit representations, such as TSDF) 39 48
or GS [4] to represent scene geometry and appearance. We use SplaTAM [5] as the SLAM backbone of our algorithm.
Active Mapping. The goal of SLAM to estimate the camera/vehicle trajectory from sensor data. Active mapping, or exploration, is a related problem in the domain of active perception [55] (56). where the goal guiding the sensor to acquire images beneficial to downstream task. The most common objectives are to reduce uncertainty, equivalently to increase information gain, [57] or to detect and visit frontiers [58]. Prior approaches such as (58)66) adopt a long-term planning paradigm, which the information gain is estimated along multiple candidate trajectories before execution. Subsequent

---

studies 63], on the other hand, reformulate the problem within next-best-view framework, where the exploration process guided by sequential decisions that progressively construct the overall trajectory. Early work demonstrated the effectiveness of active mapping 644 68 while
overviews of the state of the art can be found in 10
Active Mapping using Radiance Fields. Recently, NeRF-based approaches have been applied to
path planning [69] and next-best-view selection (70 though they are often limited by their high computational cost [73]. To overcome these limitations, hybrid models such as ActiveRMAP [74] integrate implicit and explicit representations. NARUTO [6]] introduces an active neural mapping system with 6DoF movement in unrestricted spaces, while Kuang et al. 773 integrate Voronoi planning to scale exploration to larger environments. 3DGS offers faster alternative, making
real-time mapping and exploration more feasible. Recent works like ActiveSplat [75] utilize hybrid
map with topological abstractions for efficient planning, ActiveGS [76] also uses hybrid map and associates a confidence with each Gaussian to guide exploration, and AG-SLAM [77] incorporates 3DGS with Fisher Information to balance exploration and localization in complex environments. ActiveGAMER (63] introduces rendering-based information gain criterion that selects the next-best view for enhancing geometric and photometric reconstruction accuracy in complex environments. RT-GuIDE (78 uses a simple uncertainty measure to achieve real-time planning and exploration robot. Recently, NextBestPath [79] considers longer horizons than just the single next view. Like all the methods in this paragraph, however, it does not consider semantics.
Semantic Mapping. The goal of semantic mapping is to infer scene descriptions that go beyond geometry [.]. In general, methods in this category endow their 3D representation with semantic labels, which are inferred via semantic segmentation of the input RGB or RGB-D images. Early work includes approaches such SemanticFusion [80], +usion++ 4. PanopticFusion [81] and Kimera (6], which have adopted different representations exploring tradeoffs between precision and efficiency. Radiance field-based methods surveyed by Nguyen et al. [82]. Among them, GSNeRF (83) introduces the Semantic Reasonnnn and Depth-Guided Visual modules to train a NeRF that encodes semantics along with appearance. Wilson et al. [1] use the variance of the semantic representation at each Gaussian as a proxy for semantic uncertainty. HUGS (84) jointly optimizes geometry, appearance, semanties, and motion using a combination of static and dynamic 3D Gaussians. Logits for all classes are stored with the Gaussians, but the number of classes is small. All of these approaches operate on all frames in batch mode, however.
Semantic SLAM. Gaussian splats are well suited for semantic mapping because they can encode additional attributes and are amenable to continual learning, unlike NeRF [85). All methods below operate on RGB-D video inputs. We point out the important representation choices made by their authors. SGS-SLAM 7 augments a GS-based SLAM system with additional test-time supervision via 2D semantic maps. The authors argue that any off-the-self semantic segmentation algorithm can be integrated in SGS-SLAM and use ground truth labels to supervise the splats for simplicity. High-dimensional semantic labels are converted to "semantic colors" to save space. NIDS-SLAM (86) uses 2D transformer (8) to estimate keyframe semantics, also converting the semantic labels into "semantic colors". NEDS -SSMM g reduces the memory footprint of the high- diggesinnnsi semantic features obtained by DINO 20 to three values per splat via a lightweight encoder. OpenGS-SLAM
[9 infers consistent labels via the consensus of 2D foundational models across multiple views. It can
handle an open vocabulary, but stores only one label per splat.
To overcome the limited dimensionality of colormaps, researchers have endowed the splats with embeddings of the gigh-dimensional vectors of logits. SNI-SLAM (88) model the correlations among appearance, geometry and semantic features through a cross-attention mechanism and use feature planes 4 to save memory. DNS-SLAM (89) relies on multi-resolution hash-based feature grid. Optimization is performed in latent space, while ground truth 2D semantic maps are used as inputs. ManGGauss-SLAN [90] augments the splats with 16-channel semantic embedding and presents semantic-informel bundle adjustment. The paper includes results using ground truth 2D labels for supervision, as well as labels inferred by a classifier operating on DINOv2 [91] features. Hier-SLAM [ addresses the increased storage requirements via a hierarchical tree representation, generated by a large language model. can handle over 500 semantic classes, but is also provided the ground truth semantic maps of the images during optimization.
3

---

Keyframe Database
Mapping
Color Loss
Depth Loss
P
Semantic Loss
4 vurrent
keyframe
occupied
keyframe
unexplored
feature categorical prob Semantic 3DGS
Mapping
Exploration Map
Observation
Exploration
Sample candidate views
Geometric E.C
Path Planner
Next-Best View
Semantic
Figure 1: Overview of the ActiveSGM System. Our framework integrates observation, mapping. and planning into a unified active semantic mapping system. At each time step, posed RGB-D frames along with semantic predictions from OneFormer [17] are stored in keyframe database. Selected frames are used to update a Semantic Gaussian Map that encodes geometric, photometric, and semantic properties and is optimized through differentiable rendering. An occupaney- based Exploration Map is updated using the current view and used to sample candidate viewpoints in free space. Next-best views selected by jointly evaluating geometric and semantic exploration criteria (E.C.), and a path planner navigates toward the selected pose. This closed-loop system enables efficient, semantics-aware reconstruction and exploration in complex 3D environments.
Active Semantic Mapping. The above semantic mapping and SLAM approaches are "passive" in the sense that the camera is not actively controlled but follows predetermined trajectory. In contrast, methods such as 660 92] actively plan trajectories that maximize the mutual info rmation between past and future semantic observations. Other relevant works have focused object search using semantic contextual priors e.g., leveraging knowledge that cups are typically found in kitchens wrrchhns-wwihhutt without explicitly predicting semantic labels for every point in the map 93.25). Among these approaches, more relevant to ours the work of Zhang et al. (96] that relies on semantic mutual information and properties of the SLAM pose graph for metric-semantic active mapping. An octree is used to maintain the map, but the current implementation is limited to 2D motion and 8 classes, while ground truth labels are used semantic observations. Marza al. 997 added a semantic head to Nerfacto 198] and used it for active mapping of appearance, geometry and semantics. They compared using ground truth semantic labels and Mask-R R-CNN 99 to detect object categories, and observed large differences in the metrics. Exploration policies are trained using reinforcement learning and consider the 15 object categories. Unlike our approach, the trajectory restricted to the ground plane. It is not clear how this approach would have to be modified if all semantic classes in the scene would have be considered.
3 Method
In this section, we present Active Semantic Gaussian Mapping (ActiveSGM), 3D Semantic Gaussian Splatting framework for active reconstruction that tightly integrates semantic- mapping and planning. Section [3.1 introduces Semantic Gaussian Mapping, an efficient representation that enables high-fidelin geometrie, photometric, and semantic reconstruction. To reduce the computational and memory overhead of semantic mapping, we propose a sparse semantic representation that supports efficient storage and fast rendering. Building on this, Section 3.2 describes our exploration strategy for next-hest-vien selection, which jointly leverages geometric and semantic cues guide the reconstruction of high-quality semantic maps. We outline the ActiveSGM framework in Figure

---

3.1 Semantic Gaussian Mapping
Gaussian Mapping. Gaussian Mapping leverages 3DGS to represent scenes as collections of 3D Gaussians, encoding both appearance and geometry for real-time rendering of high-fidelity color and depth images. Building upon the work of Kerbl et al. [25], we adopt the streamlined approach proposed in SplaTAM [51]. This method employs isotropic Gaussians with fiew-independent color, optimizing parameters such as color (c), center position (14), radius (r), and opacity (o).
notable advantage of 3DGS is its capability for real-time rendering, enabling the synthesis of high-fidelity color and depth images from arbitrary camera poses. This is achieved by transforming 3D Gaussians into camera space, sorting them front-to-back, projecting them onto the 2D image plane, and employing alpha-blending for compositing. The color, depth, and silhouette at pixel rendered from the Gaussian map, where the silhouette indicates whether p receives significant projection from any Gaussian. The general rendering process is formulated as
R(p) = Σ 1=1
(P) II (1 fs(D)) J=1
(1)
where 2; {c,, 1,, and R(p)
depending whether color, depth, silhouette is being rendered. (i) is derived from the Gaussian's position and size in 2D pixel space. The differentiable nature of this rendering process allows for end-to-end optimization, where gradients are computed based on discrepancies between rendered images and RGB-D inputs, and the optimization objective is formulated as:
D(p),
L= -=E, (S(p) > 0.99) (L((D()) + (CLL)
where only pixels inside the silhouette are considered.
Semantic Prediction. We incorporate semantics into the 3DGS map by using OneFormer 17 state-of-the-art model for unified segmentation, to perform semantic segmentation. Its predictions serve as our primary source of semantic observations.
Sparse Semantic Representation. Given the semantic predictions from OneFormer, represented as a probability distribution p (Pi, P2,..., over M semantic categories, straightforward approach to constructing a Semantic Gaussian Map is to incorporate P an additional attribute in each 3D Gaussian. However, storing and optimizing such high-dimensional semantic properties can lead to significant memory overhead.
To mitigate this issue, we introduce a sparse semantic representation, where only the top-k categories with the highest probabilities from the initial observation are retained per Gaussian. Specifically, for each Gaussian G, we define the sparse semantic vector as P. (P:, Pia, Pi....., Piz) ). This compact form preserves most of the semantic information while significantly reducing storage and computation costs. As new observations arrive, the probabilities are updated while keeping the original top-k indices fixed, allowing semantic refinement over time without restoring the full distribution.
Semantic Rendering. Similar to color and depth, semantic rendering projects 3D Gaussians into 2D and composites their semantic properties at each pixel. To preserve efficiency, we render only using Gaussians within the current view and aggregate their sparse top-k semantic distributions into full semantic probability map. Given each Gaussian's 's sparse vector P., we compute the class-m probability at pixel p as:
Pm) Σ pr (p) (1-1((() 1=1
j=1
where Pim is the probability of class for Gaussian G., and f+(p) denotes its projected influence at pixel p. This approach enables smooth, class-wise semantic rendering while avoiding the overhead of fully dense representations, striking balance between accuracy and memory efficiency.
5

---

Semantic Loss. To optimize the semantic 3DGS, we employ combination of the Hellinger distance and cosine similarity losses. Predictions from OneFormer serve as the pseudo-groumd truth Par. while the rendered semantic outputs are treated as predictions P pred- To filter out uncertain supervision, we apply an entropy-based mask Mи (p) < т) on PGT, where T(-) is the indicator function and r is the entropy of uniform distribution over l categories, i.e. = log The entropy at each pixel is computed as:
M H(p) = Σ Pm log Pm (p).
(4)
m=1
The Hellinger distance encourages the predicted semantic distribution closely match the pseudo ground truth while providing smooth and bounded gradients. To further regularize the optimization, we incorporate the cosine similarity loss, which promotes angular alignment between the predicted and target distributions. This combination ensures both probabilistic accuracy and structural consistency,
leading to more stable and robust training for semantic 3DGS. The final semantic loss is defined as:
Leemmn
MH (AHD
(Pred) + Acae (1 - cos Ppea))),
(5)
where DHD(- |I. denotes the Hellinger distance and cos(:;.) is the cosine similarity, We set AHD = 0.8 and Acos 0.2 to balance their contributions.
To prevent noisy semantic predictions from affecting the entire 3DGS representation, we restrict backpropagation of this loss only the semantic attributes of each Gaussian, leaving geometric and photometric components untouched.
Keyframe Selection Strategy. Following SplaTAM [51]. our Gaussian Mapping backbone opti- mizes the map using a subset of keyframes instead of all input frames. Every fifth frame is considered keyframe candidate, and the map is updated using local keyframes with the highest 3D overlap, computed by backprojecting depth maps and evaluating visibility within keyframe frustums. This provides efficient multiview supervision but may overfit occluded regions, reducing opacity for valid Gaussians behind surfaces.
To address this, we introduce global-local keyframe strategy. In addition to local keyframes, we select global keyframes based on: (1) low rendering quality, and (2) low semantic entropy and fewer unknown labels to ensure confident supervision. These global keyframes help cover under-observed and ambiguous regions. In practice, we maintain a 50-50 mix of local and global keyframes to balance local detail with global coverage.
3.2 Exploration Planning
To enable efficient semantic reconstruction, we design an exploration plamning module that actively selects informative viewpoints. Each candidate pose is evaluated using two criteria: geometric coverage, measured by silhouette completeness, and semantic uncertainty, quantified by entropy. These criteria approximate information gain [TO. 12), which measures the expected reduction in uncertainty from new observations. While computing true information gain is intractable in high- dimensional semantic maps [100,, our entropy- and coverage-based approximations allow efficient real-time scoring of candidate viewpoints. To keep computation efficient, we maintain dynamic candidate pool and adopt coarse-to to-fiee sampling strategy that first explores broadly, then refines with denser sampling. We now detail the geometric and semantic exploration criteria, the overall scoring formulation, and the implementation of candidate management.
Geometric Exploration Criterion. We adopt ActiveGAMER's s 63 exploration criterion formu- lation to evaluate the geometric coverage of candidate viewpoints. Given candidate viewpoint we compute its exploration criterion Ty based on the rendered silhouette 5" with respect to the up-to-date Semantic Gaussian Map. The number of missing pixels in the rendered silhouette, denoted as Ns., quantifies the exploration criterion for the candidate viewpoint, which is formulated as:
Iyeo = olog(Nss)), Ns- = Σ I(S = 0)
66
where a(-) is the softmax function, which normalizes the scores across all candidate viewpoints, and I(()) is the indicator function, counting pixels with zero values in the silhouette.

---

GT RGB
GT - Semantic
SGS-SLAM
Ours
Figure 2: Qualitative Results for Replica. Our method generates denser and more accurate semantic maps than SGS-SLAM, with fewer exploration steps. Yellow boxes highlight improved boundaries and semantic consistency. Black regions denote unknown labels.
Semantic Exploration Criterion. In addition to geometric coverage, we assess semantic uncer- tainty by rendering the semantic probability map of each candidate viewpoint from the current Semantic Gaussian Map. To ensure numerical stability, we clip the probabilities to [0.001, 1] and normalize them to form valid probability distributions. Given a candidate pose v, the semantic exploration score is defined as:
Temaan= = (E H'
where (p) is the entropy at pixel p. computed as in Eqn. This encourages selecting views that reduce semantic uncertainty and improve coverage in ambiguous regions.
Overall Exploration Criterion. To guide efficient scene coverage and reduce redundant motion, we define the overall exploration criterion by combining geometric and semantic objectives with motion cost that penalizes distant candidate viewpoints. This encourages the system to prioritize informative poses that are also close to the current camera location.
Given candidate camera pose U, we first compute the exploration criteria Ioo and Tseman described above. To encourage travel efficiency, we define motion cost based on the Lg distance between the candidate pose location Te and the current camera location T denoted as !|T T ll2- We apply softmax function to the motion cost to normalize the cost across all candidates. The final distance-an aware exploration criterion defined as:
I" = (1 - o(()) (Ig Tyeman)
(8)
This formulation balances information gain and travel efficiency, favoring views that improve map quality while minimizing unnecessary motion.
Exploration Strategy. To efficiently evaluate candidate viewpoints, maintain Exploration Map, a voxel-based occupancy grid that tracks free space. Newly observed voxels are identified by comparing the updated grid its previous state, and candidate viewpoints new voxels. Candidate positions are spaced every i1 units of length, with Vi viewing directions uniformly distributed using the Fibonacci lattice. Each pose T is scored using the overall exploration criterion (Eqn. R). and vaw-value candidates (Ns, < 0.5% image pixels) are pruned from the pool.
sampled from these
To balance speed and coverage, we use coarse-to-fine strategy: the coarse stage samples on single height plane with larger steps (w) = 1) and fewer directions (12 = 5); the fine stage increases density with smaller steps (v) 0.5), multiple heights, and more directions (U2 15), removing redundant views to maintain exploration efficiency and completeness.
Experiments and Results
4.1 Experimental Setup
In this paper, we focus on evaluating the semantie segmentation accuracy of the proposed pipeline, particularly under noisy observation settings, which better reflect real-world conditions. Deploying
7

---

and evaluating on physical robots is challenging due to the lack of well-annotated data especially ground-truth semantic labels- for reliable evaluation. Consequently, prior methods are typically evaluated in simulation, making photorealistic environments the most practical and fair testbed for benchmarking. Following the common evaluation protocol, all of the following experiments are conducted in simulated environment.
Simulator and Datasets. We use Habitat 101 to generate RGB-D frames and OneFormer 17 for semantic segmentation. Frames are captured 680 x 1200 resolution with 60 vertical and 90 horizontal FOV. The Exploration Map uses voxel size of 5 cm.
We evaluate three photorealistic datasets: Replica [02], ReplicaSLAM, and MP3D [03]
Replica includes high-fidelity meshes and 101 semantic classes; we use scenes from [44]. Repli- caSLAM provides predefined camera trajectories for the same 8 scenes. MP3D includes 40 semantic
classes; we use 5 scenes for evaluation. Each experiment runs for 2,000 steps on Replica and 5,000 on MP3D, with early termination if the exploration candidate pool is exhausted.
Semantic Model Fine-tuning. To improve semantic prediction accuracy, we collect 500 RGB- Semantic frames from each scene and fine-tune OneFormer separately on Replica and MP3D. The fine-tuned models are used to generate per-pixel semantic class probability maps, which are then converted into sparse semantic representations for each 3D Gaussian.
Semantic Evaluation Metrics. We follow SGS-SLAM's 7 evaluation protocol and compute the Average Mean Intersection over Union (mloU) by mapping the rendered semantic predictions to ground-truth categories within each test view. In addition, we evaluate per-pixel semantic classification using Top-I and Top-3 Accuracy, and assess the complete category distribution (not limited to the categories present in given image) using Mean Average Precision (mAP) and FI-score. Owing to space constraints, report only subset of the results in Table T please refer to Section S.2 of the supplementary material for the full results.
Geometric and Photometric Metrics. We geometric reconstruction using three metries: Accuracy (cm), Completeness (cm), and Completeness ratio (%) with a 5 cm threshold. These are computed by uniformly sampling 3D points from both the ground-truth mesh and the reconstructed Gaussian Map. For measuring rendering quality, use PSNR, SSIM, LPIPS and Depth LI (D-LI),
Baselines. To the best of our knowledge, this is the first work to tackle dense active semantic mapping with 3D Gaussian Splatting (3DGS); direct, one-to-one baselines are therefore difficult to establish. We evaluate our system against three categories of baselines: (1) semantic SLAM methods based on NeRF or 3DGS, which primarily target segmentation and rendering quality; (2) a passive semantic mapping pipeline reconfigured from the strongest representative in (1), to highlight the benefits of our semantic representation and active exploration policy; and (3) geometry-based active mapping methods, which focus on maximizing 3I) reconstruction accuracy.
All experiments were conducted on two NVIDIA RTX A6000 GPUs. Additional implementation details and extended results are provided in the supplementary material.
4.2 Semantic Segmentation Evaluation
Comparison with NeRF/GS-based Semantic SLAM on ReplicaSLAM. Most existing NeRF/3DGS-based semantic mapping approaches- such NIDS-SLAMÍSA DNS- (NS-SLAM(89) SNI-SLAM(8S), and SGS-SLAMM] formulated as semantic SLAM systems, i.c., they jointly solve for localization and semantic mapping. We report results from SGS-SLAM [7] only for refer- ence and completeness since we assume perfect localization. We follow the protocol, which measures the agreement between rendered semantic masks and ground -ruth labels visible in each view (Table yellow Our setup differs from prior baselines in three key aspects: (I) we use pseudo labels generated by OneFormer [17] instead ground truth; (2) we evaluate on views not seen during training; and (3) we train with only one-third of the available images. Despite these constraints, our method achieves performance comparable to fully supervised baselines by effectively fusing noisy predictions across views into coherent semantic map.
evaluation

---

Table 1: Semantic Segmentation Results. We evaluate ActiveSGM on Replica and MP3D without access to ground truth semantic labels, requiring fewer mapping steps, and testing on novel views not during training. We report average mloU Replica and average IoU MP3D. "GT" means ground-truth labels. while "Pred." means predicted labels from OneFormer.
Method
Dataset
Labels
Evaluation View
Stepe
Arg. [%)) (-(
ReplicaSI
2000
DNS-SLAM 50
AM
GT
Train
2000
84.77
SNLSLAM
33
ReplicaSLAM
GT
Train
2000
87.41
SGS-SLAM
ReplicaSLAM
GT
Train
2000
92.72
OneFormer
ReplicaSLAM
Ours
ReplicaSLAM
Prod.
Novel
713
$5.13
SGS- 7
Replica
Pied.
Novel
2000
80142
18.70 30
Ours (Parsive)
Replica
Pred.
Novel
2000
80.14
67.81
Replica
777
SSMI 02
MP3)
GT
Tran
36.14
TARE [04
MP3D
GT
Train
31.70
Zhang
96
MP3D
GT
Train
42.92
GT Semantic
SGS-SLAM
SGS-SLAM
SGS-SLAM label conversion)
(rasterized)
(local conversin)
Figure Color-Coding Ambiguities. SGS-SLAM blend colors leading to label confusion, especially under global conversion, and the introduction of irrelevant categories.
Comparison with Passive Semantic Mapping on Replica (Novel Views). To enable fairer comparison with semantic SLAM methods, we select SGS-SLAM, as the strongest representative baseline and reconfigure it to function as a semantic mapping-one system: (I) We disable its tracking module and use ground truth poses. (2) Replace ground truth semantics with semantic predictions. This allows us to directly evaluate the semantic representation quality, which is key contribution our method. We then disable our own active exploration module to ensure passive, mapping-onle setting, enabling a fair apples-to-applee comparison. The results in Table [] blue show that full pipeline with active exploration achieves better segmentation with fewer observations/stess, demonstrating the effectiveness of our exploration strategy guided by semantic and geometric uncertainty.
Color-Coding Limitations. shown in Figure 3 SGS-SLAM and similar methods use color encoding to represent semantic labels, which often blend during multi-view fusion and introduce arbitrary labels, leading to misclassification and ambiguity (yellow boxes). To recover labels, they apply nearest-color matching using either local label conversion, which maps to the nearest color among ground-truth classes the current view, or globul label conversion, which considers all ground-truth classes in the scene. However, assuming access to view-specifi ground-trute truth labels is unrealistic. Inconsistencies between local and global conversion are shown in cyan boxes.
Comparison with Active Semantic Mapping MP3D. We also include comparisons to recent active semantic mapping baselines, which are more aligned with our task definition. These further validate the advantage of our sparse semantic representation and active policy. We evaluate on large indoor scenes from MP3D (Table 1 red ). Table shows active semantic mapping baselines [92 [04]
from (96). do not know which scenes were used by the basclines, but we evaluate on a common set
Semantic
Entropy
Figure 4: Qualitative Results for MP3D. Top-down visualizations of reconstructed scene, semantic labels and semantic entropy heatmap (low, high). Notably, our results show no high-entropy regions, and produce coherent and dense semantic reconstructions even in large scale MP3D scenes.

---

Table 2: Ablation of Semantic Components. Experiments on officeO and roomt from Replica to
evaluate the impact of the number of retained categories (Top-k) and the use of Hellinger distance (H.D.), KL- KL-Divergence (KI.) and cosine similarity (Cos.) in the semantic loss.
Top-
H.D. KL.
Cos.
Avg. mlou (%)
Top-1 Acc (%)
Top-3 (%)
mAP (%)
F-1 (%)
Top-5
83,06
95.66
99.68
94,79
74.24
Top-8
/
83.34
95.70
99.66
95.05
74.23
Top-16
/
84.08
95.68
99.73
94,92
74.73
Top-16
V
82.26
95.57
99.61
94,21
74.10)
Top-16
" 82.70
95.62
99.73
94,40
72.43
Top-16
82.22
95.63
99.70
94.66
73.93
16p-16
/
84.08
95.68
99.73
94,92
74.73
of labels, and report Average IoU. All baselines use ground truth labels during optimization. Despite relying predicted labels and novel views, our method significantly outperforms all baselines. F shows that our system produces clean and consistent semantic maps across complex indoor scenes.
Comparison on Reconstruction and Novel View Synthesis. evaluate ActiveSGM reconstruction and novel view synthesis on MP3D and Replica. On MP3D, our method achieves 1.56 em accuracy and 97.35% completeness, surpassing ActiveGAMER [63] (1.66 cm, 95.32%). In novel view synthesis on Replica, ActiveSGM achieves SSIM of 0.96, closely matching ActiveGAMER's 0.97 despite not using a photometric refinement stage. This highlights ActiveSGM's ability to main- tain balance between photometric quality and geometric fidelity. Full quantitative and qualitative results are provided in the supplement.
4.3 Ablation Studies
We perform ablation studies two key components of our proposed method that influence semantic mapping performance: (1) the number of categories used in the sparse semantic representation, and (2) the effect of individual loss terms in optimizing semantic features. Experiments are conducted on the office0 and roomO scenes from the Replica dataset. As shown in Table 2 (I) Using more categories improves accuracy, we retain the top-16 categories for the best overall performance. (2) Removing either the Hellinger distance or cosine similarity reduces the Average mloU. Using both terms together, accuracy reaches 84.08%, confirming their effectiveness. (3) We also compare KL divergence and Hellinger distance, finding that the Hellinger distance is a more effective choice. Notably, KI. divergence can lead gradient vanishing due the instability of the logarithmic function, necessitating gradient norm clipping during training.
5 Limitations and Conclusion
Despite the strong performance of ActiveSGM we show above, several limitations remain: Perfect Localization: We assume known robot poses throughout the process; real deployments, separate localization or tracking module would be required. Perfect Execution: The robot is assumed to precisely follow planned trajectories; navigation errors should be considered for deployment. Semantic Segmentation Model: Our system relies on an external segmentation model (OneFormer) for semantic predictions, which may introduce domain--spon specifie errors. Stronger or fine-tuned models can improve results, while weaker ones may degrade semantic quality. Limited Joint Optimization: To stabilize training, we block gradients from the semantic loss to geometric properties, decoupling the optimization of geometry and semantic features.
In conclusion, We presented ActiveSGM, the first dense active semantic mapping system built 3D Gaussian Splatting backbone. By unifying geometry, appearance, and semantics, ActiveSGM enables efficient exploration and high-quality mapping with fewer observations. It improves semantic coverage through uncertainty-guie exploration, reduces memory via top-k sparse representations, and handles noisy predictions without ground -rrth labels. We hope ActiveSGM will inspire future research in active mapping, semantic understanding, and autonomous robotic exploration. Our code avaiable at withttt::/// ggt..tt.....t.. comm/lyy0412//cttieeeee.
git
Acknowledgement. under award 2024653.
This research has been supported in part by the National Science Foundation
10

---

References
[1] S. Raychaudhuri and A. X. Chang, "Semantic Mapping in Indoor Embodicd Al-A Comprehensive Survey and Future Directions," arXiv preprint (XXi::5501.05555 2025.
[2] J.-R. Ruiz-Sarmiento, C. Galindo, and J. Gonzalez-Jimenez, "Building multiversal semantic maps for mobile rohot operation," " Knowledge-Rasee Systems, vol. 119, pp. 257-272, 2017.
[3] R. Mascaro, L. Teixeira, and M. Chli, "Volumetric Instance-Leved Semantic Mapping Via Multi-View 2D-to-3D Label Diftusion." IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3531 3531-3538, 2022.
[4] J. McCormac. R. Clark, M. Bloesch. A. Davison, and S. Leutenegger. "Fusion++: Volumetric Object-level SLAM," in International Conference on 3D Vision (3DV), 2018, pp. 32-41.
[5] B. Li, Z. Cai, Y-F. Li, I. Reid, and H. Rezatofighi, "Hier-SIAM: Scaling-up Semantics in SLAAM with a Hierarchically Categorical Gaussian Splatting," in ICRA, 2025.
[6] A. Rosinol, M. Abate, Y. Chang, and L. Carlone, "Kimera: an Open-source Library for Real-time Metric-Semantic Localization and Mapping," in ICRA, 2020, pp. 1689-1696.
[7] M. Li, S. Liu. H. Zhou, G. Zhu, N. Cheng, T. Deng. and H. Wang, "SGS-SLAM: Semantic Gaussian Splatting for Neural Dense SLAM," in ECCV, 2024, pp. 163-179.
[8] Ji, Liu, G. Xie, B. Ma, Xie, and H. Liu, "NEDS-SLAM: A Neural Explicit Dense Semantic
SLAM Framework using 3D Gaussian Splatting," IEEE Robotics and Automation Letters, 2024.
[9] D. Yang. Y. Gao, X. Wang, Yue, Y. Yang, and M. Fu, "OpenGS-SLAM: Open-Set Dense Semantic SLAM with 3D Gaussian Splatting for Object-Level Scene Understanding." in ICRA, 2025.
[10] Cadena, L. Carlone, H. Carrillo, Latif, D. Scaramuzza, J. Neira, Reid, and J. Leonard, "Past. present, and future of simultaneous localization and mapping: Toward the robust-perception age." JEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309-1333 1332, 2016.
[11] I. Lluvia, F. Lazkano, and A. Ansuategi, "Active mapping and robot exploration: survey," Sensors, vol. 21, no. 7, p. 2445, 2021.
[12] J. Placed, J. Strader, Carrillo, N. Atanasov, V. Indelman, L. Carlone, and J. Castellanos, "A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers," IEEE Transactions on Robotics, vol. 39, no. 3, pp. 1686- 1006-177,2220 2023.
[13] J.-L. Matez-Bandera, P. Ojeda, J. Monroy, J. Gonzalez-Jimened and J.-R. Saiiz-Srmiinto,, "Voxeland: Probabilistic Instance- Aware Semantic Mapping with Evidence-based Uncertainty Quantification, arXiv preprint
2024.
[14] J. Wilson, Y. Fu, J. Friesen, P. Ewen, A. Capodieci, P. JJyakuuar,, K. Barton, and M. Ghaffari, "ConvBKI: Real-Time Probabilistic Semantic Mapping Network with Quantifiable Uncertainty," IEEE Transactions Robotics, 2024.
[15] J. Wilson, M. Almeida, M. Sun, S. Mahajan, M. Ghaffari, P. Ewen, O. Ghasemalizadeh, C.-H. Kuo, and A. Sen, "Modeling Uncertainty in 3D Gaussian Splatting through Continuous Semantic Splatting." arXiv preprint 1.02547, 2024.
[16] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmenta- tion," MICCAI, 2015, pp. 234-241.
[17] J. Jain, J. Li. M. Chiu, A. Hassani, N. Orlov, and H. Shi, "OneFormer: One Transformer to Rule Universal Image Segmentation," in CVPR, 2023.
[18] Y.TT. Hu, J.-B. Huang, and A. Schwing, "MaskRNN: Instance Level Video Object Segmentation." NewrIPS, vol. 30, 2017.
[19] M. Qin, W. J. Zhou, H. Wang, and H. Pfister, "LangSplat: 3D Language Gaussian Splatting," in CVPR, 2024.
[20] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, "Emerging Properties in Self-Supervieed Vision Transformers." in ICCV, 2021, pp. 9650-9660.
[21] M. Cherti, R. Beaumont, R. Wightman, M. Wortsman, G. Ilharco, C. Gordon, C. Schuhmann, L. Schmidt, and J. Jitsev, "Reproducible scaling laws for contrastive language-image learning." in CVPR, 2023, pp. 2818 2829.
11

---

[22] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. "NeRF: Representing scenes neural radiance fields for view synthesis," in ECCV. Springer, 2020, pp. 405-421.
[23] A. Tewari, J. Thies, B. Mildenhall, P. Srinivasan, E. Tretschk, W. Yifan, C. Lassner, Vitzmann,, R. Martin- Brualla, S. Lombardi, T. Simon, C. Theohalt, M. NieBner, J. T. Barron, G. Wetzstein, M. Zollhöfer, and Golyanik, "Advances Neural Rendering." " Computer Graphics Forum, vol. 41, no. pp. 703-735, 2022.
[24] Xie, Takikawa, S. Saito, O. Litany, Yan, N. Khan, Tombari, J. Tompkin, Sitzmann, and S. Sridhar, "Neural fields in visual computing and beyond," Computer Graphics Forum, vol. 41, no. 2, pp. 641 676, 2022.
[25] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, "3D Gaussian Splatting for Rcal- Sime-Time Radiance Field Rendering." "" ACM Trans. Gruph., vol. 42, no. 4, pp. 1023.
[26] B. Fei, J. Xu, R. Zhang, Q Zhou, W. Yang, and Y. "e,"D IEEF Transactions Visualization and Computer Graphics, 2024.
"3D Gaussian Splatting as New Era: A Survey,"
[27] S. Zhu, G. Wang, D. Kong, and H. Wang, "3D Gaussian Splatting in Robotics: A Survey," arXiv preprint arXiv:2410. 1T222440..1262,, 2024.
[28] R. Siegwart, I. R. Nourbakhsh. and D. Scaramuzza, Introduction to Autonomous Mobile Robots. press, 2011.
MIT
[29] D. Scaramuzza and F. Fraundorfer, "Visual odometry [tutorial]," IEEE Robotics & Automation Magazine, vol. 18, 4, pp. 80-92, 2011.
[30] H. Zhan, C. S. Weerasekera, J.-W. Bian, and I. Reid, "Visual Odometry Revisited: What Should Be Learnt?" in ICRA, 2020. pp. 4203-4210.
[31] J. L.. Schonberger and J.-M. Frahm, "Stmacture-from-Mee Motion Revisited," in CVPR, 2016, pp. 4104-4113.
[32] S. Thrun, "Probabilistic Robotics," Communications of the ACM, vol. 45, no. 3, pp. 52-57, 2002.
[33] H. Durrant-Whyte and T. Bailey, "Simultaneous Localization and Mapping: Part I," IEEE Robotics & Automation Magazine, vol. 13, no. 2. pp. 99-119 2006.
[34] A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse, "MonoSLAM: Real-time single camera SLAM," IEEE Transactions on Puttern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1052-1067, 2007.
[35] C. Campos, R. Elvira, J. Rodríguez, J. M. Montiel, and J. D. Tardós, "ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial, and Multimap SLAM,"" IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1874-1890, 2021.
[36] Y. Bao, T. Ding. J. Huo, Y. Liu, Y. Li, W. Li, Y. Gao, and J. Luo, "3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities," IEEE Transactions on Circuits and Systems for Video Technology, 2025.
[37] Tosi, Y. Zhang, Gong, E. Sandström, Mattoccia, M. R. Oswald, and M. Poggi, "How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: Survey," arXiv preprint arXiv:2402.1331 13255, 2024.
[38] G. Wang, L. Pan, Peng. S. Liu, C. Xu, Y. Miao, W. Zhan, M. Tomizuka, M. Pollefeys, and H. Wang. "NeRF in Robotics: A survey," arXiv preprint arXiv:2405.01333 2024.
[39] C.-M. Chung, Y.-C Tseng, Y.-. Hsu, X.-Q. Shi, Y.H. Hua, Yeh, W.-C. Chen, Y.-T. Chen, and W. Hsu, "Orbeez-SLAM: A Real-time Monocular Visual SLAM with ORB Features and NeRF-realized Mapping," in ICRA, 2023, pp. 9400-9406 94069
[40] T. Deng, G. Shen, T. Qin, J. Wang, W. Zhao, J. Wang. D. Wang, and W. Chen, "PLGSLAM: Progressive neural scene represenation with local to global bundle adjustment," in CVPR, 2024, pp. 157--199 666.
[41] M. M. Johari, C. Carta, and F. Fleuret, "ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields," in CVPR, 2023, pp. 17 174917
[42] M. Li, J. Hc, Y. Wang, and H. Wang, "End-to-Ent End RGB-D SLAM With Multi-MLPs Dense Neural Implicit Representations," IEEE Robotics and Automation Letters, vol. 8, no. 11, pp. 7138-7145, 2023.
[43] N. Stier, A. Ranjan, A. Colburn, Y. Yan. L. Yang, F. Ma. and B. Angles, "FineRecon: Depth aware Feed-forward Network for Detailed 3D Reconstruction in ICCV, 2023, pp. 18 18422118433 4322..
12

---

[44] E. Sucar, S. Liu, J. Ortiz, and A. J. Davison, "iMAP: Implicit mapping and positioning in real-time." in ICCV, 2021, pp. 0229-6238.
[45] H. Wang, J. Wang, and L.. Agapito, "Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SL.AM," in CVPR, 2023, pp. 13 13 302.
[46] Y. Zhang, F. Tosi, S. Mattoccia, and M. Poggi. "GO-SLAM, Global Optimization for Consistent 3D Instant Reconstruction," in ICCV, 2023, pp. 3727-3737.
[47] Z. Zhu, S. Peng, V. Larsson, W. Xu, H. Bao, Z. Cui, M. R. Oswald, and M. Pollefeys, "NICE-SLAN Neural Implicit Scalable Encoding for SLAM," in CVPR, 2022. pp. 12 12787-1221111 796.
[48] Z. Zhu, S. Peng, V. Larsson, Z. Cui, M. R. Oswald, A. Geiger, and M. Pollefeys, "NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM," in International Conference on Vision (3DV), 2024.
[49] T. Deng, Chen, L. Zhang, Yang, S. Yuan, J. Liu, D. Wang, H. Wang, and W. Chen, "Compact 3D Gaussian Splatting for Dense Visual SLAM," arXiv preprint ArX:v:2403. 11.400.//227. 1247, 2024.
[50] H. Huang. L. Li, H. Cheng, and S.-K. Yeung, "Photo-SLA Real-time Simultancous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D D Cameras," in CVPR, 2024, pp. 21 584 21 593.
[51] Keetha, J. Karhade, K. M. Jatavallabhula, G. Yang, S. Scherer. D. Ramanan, and J. Luiten, "SplaTAM: Splat Track & Map 3D Gaussians for Dense RGB-D SLAM," in CVPR, 2024, pp. 21 357-21 366.
[52] H. Matsuki, R. Murai, P. H. Kelly, and A. J. Davison, "Gaussian Splatting SLAM," in CVPR, 2024, pp. 18 039-11 048.
[53] C. Yan, D. Qu. D. Xu, B. Zhao, Z. Wang, D. Wang, and X. Li, "GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 19559-199 604.
[54] V. Yugay. Y. Li, T. Gevers, and M. R. Oswald, "Caussian-SLAM, Photo-realistic Dense SLAM with Gaussian Splatting." arXiv preprint arXiv:2312.100700 2023.
[55] J. Aloimonos, I. Weiss, and A. Bandyopadhyay, "Active Vision," International Journal of Computer Vision, vol. I, no. 4, pp. 333-356, 1988.
[56] R. Bajesy, "Active Perception," Proceedings of the IEEE, vol. 76, no. 8, pp. 966-1005, 1988.
[57] C. Stachniss, Robotic Mapping and Exploration. Springer, 2009, vol. 55.
[58] B. Yamauchi, "A Frontier-basse Approach for Autonomous Exploration." in IEEE International Sympo- sium on Computational Intelligence in Robotics and Automation CIRA 97.'Towards New Computational Principles for Robotics and Automation', 1997, pp. 146-151.
[59] G. Georgakis. B. Bucher, A. Arapin, K. Schmeckpeper, N. Matni, and K. Danillidis, "Uncertainty-driven planner for exploration and navigation," ICRA, 2022. pp. 11 295-11 302.
[60] H. Siming, C. D. Hsu, D. Ong, Y. S. Shao, and P. Chaudhari. "Active perception using neural radiance fields." in American Control Conference (ACC), 2024, pp. 4353-4358.
[61] 7. Feng, H. Zhan, Z. Chen, Q. Yan, X. Xu, C. Cai, B. Li, Q. 7hu, and Y. Xu, "NARUTO: Neural Active Reconstruction from Uncertain Target Observations," " in CVPR, 2024, pp- 572-21 583.
[62] K. Ramakrishnan, Z. Al-Halah, and K. Grauman, "Oecupancy anticipation for efficient exploration and navigation," in Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part V 16. Springer, 2020, pp. 400-418.
[63] L. Chen, H. Zhan, K. Chen, X. Xu, Q. Yan, C. Cai, and Y. Xu, "ActiveGAMER: Active GAussian Mapping through Efficient Rendering," in CVPR, 2025.
[64] F. Bourgault, A. A. Makarenko, S. B. Williams, B. Grocholsky, and H. F. Durrant-Whyte, "Information based Adaptive Robotic Exploration," in IROS, vol. 1, 2002, pp. 545.
[65] A. J. Davison and D. W. Murray, "Simultancous Localization and Map-Building Using Active Vision," " IEEE TPAMI. vol. 24, no. 7, pp. 865-880, 2002.
[66] H. J. S. Feder, J. J. Leonard, and C. M. Smith, "Adaptive Mobile Robot Navigation and Mapping." The International Journal of Robotics Research, vol. 18, no. 7, pp. 650-668. 1999.
13

---

[67] C. Stachniss, D. Hahnel. and W. Burgard, "Exploration with active loop-closing for FastSLAM." in IROS,
vol. 2, 2004, pp. 1505-1510.
[68] S. B. Thrun and K. Möller, "Active Exploration in Dynamic Environments," NewrIPS, vol. 4, 1991.
[69] M. Adamkiewicz, I. Chen, A. Caccavale, R. Gardner, P. Culbertson. J. Bohg, and M. Schwager, "Vision- only robot navigation in a neural radiance world," IEEE Robotics and Automation Letters, vol. 7, no. pp. 4606-4613, 2022.
[70] Y. Ran, J. Zeng. S. He, J. Chen, L. Li, Y. Chen, G. Lee, and Q. Ye, "NeurAR: Neural Uncertainty for Autonomous 3D Reconstruction With Implicit Neural Representations," IEEE Robotics and Automation Letters, vol. 8, no. 2, pp. 1125-1 132, 2023.
[71] S. Lee,, L. Chen, J. Wang, A. Liniger, S. Kumar, and F. Yu, "Uncertainty Guided Policy for Active Robotic 3D Reconstruction Using Neural Radiance Fields," IEEE Robotics and Automation Letters, 2022.
[72] X. Pan, Z. Lai, Song, and G. Huang. "ActiveNeRF: Learning where See with Uncertainty Estimation," ECCV. Springer, 2022, pp. 230-246.
[73] Z. Kuang, Z. Yan, H. Zhao, G. Zhou, and H. Zha, "Active neural mapping at scale," in IROS, 2024,
[74] Zhan, J. Zheng, Xu, I. Reid, and Rezatofighi, "ActiveRMAP: Radiance Field for Active Mapping And Planning," arXiv preprint 1xXx::2211.2222 12656, 2022.
[75] Y. Li, 7. Kuang, T. Li, G. Zhou, S. Zhang, and Z. Yan, "Apai High-Fidelity Scene Reconstruction through Active Gaussian Splatting," arXiv preprint urXi::2400221955, 2024.
[76] L. Jin, X. Zhong, Y. Pan, J. Behley, Stachniss, and M. Popović, "ActiveGS: Active Scene Reconstruction using Gaussian Splatting," " IEEE Robotics and Automation Letters, 2025.
[77] W. Jiang, B. Lei. K. Ashton, and K. Daniilidis, "AG-SLAM: Active Gaussian Splatting SLAM," arXiv preprint arXiv:2410.17422 2024.
[78] Tao, Ong, Murali, Spasojevic, P. Chaudhari, and V. Kumar, "RT-GuIDE: Real-Time Gaussian splatting for Information Driven Exploration," arXiv preprint (XXx::4409.1822 2024.
[79] S. Li, A. Guédon, C. Boittiaux, S. Chen, and V. Lepetit, "NextBestPath: Efficient 3D Mapping of Unseen Enviromments," in ICLR, 2025.
[80] J. McCormac, A. Handa, A. Davison, and S. Leutenegger, "SemanticFusion. Dense 3D Semantic Mapping with Convolutional Neural Networks," in ICRA, 2017, pp. 4628 4635.
[81] G. Narita, T. Seno, T. Ishikawa, and Y. Kaji, "PanopticFusion: Online Volumetric Semantic Mapping the Level of Stuff and Things," in IROS, 2019, pp. 4205-4212.
[82] T.-A-Q. Nguyen, A. Bourki, M. Macudzinski, Brunel, and M. Bennamoun, "Semantically-awe aware Neural Radiance Fields for Visual Scene Understanding: Comprehensive Review," " arXiv preprint arXiv:2402.1113 11141, 2024.
[83] Z.-T. Chou, S.-Y. Huang, I. Liu, and Y--C. F. Wang, "GSNeRF: Generalizable Semantic Neural Radiance Fields with Enhanced 3D Scene Understanding," in CVPR, 2024, pp. 20 806-20815.
[84] H. Zhou, J. Shao, L. Xu, D. Bai, W. Qiu, B. Liu, Y. Wang, A. Geiger, and Y. Liao, "HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting," in CVPR, 2024, pp. 21 336-21 345.
[85] Z. Cai and M. Müller, "CLNeRF: Continual Learning Meets NeRF," in ICCV, 2023, pp. 23 185-23 194.
[86] Y. Haghighi, S. Kumar, J.PP. Thiran, and L. Van Gool, "Neural Implicit Dense Semantic SLAM," arXiv preprint arXiv:2304. 20l.2300...4400 2023.
[87] Cheng, Misra. A. Schwing. A. Kirillov, and R. Girdhar, "Masked-attenne attention Mask Transformer for Universal Image Segmentation." in CVPR, 2022, pp. 1290-1299.
[88] S. Zhu, G. Wang, H. Blum, Liu, L. Song. M. Pollefeys, and H. Wang, "SNI-SLAM: Semantic Neural Implicit Slam," in CVPR, 2024, pp. 21 167-21 177.
[89] K. Li. M. Niemeyer, N. Navab, and F. Tombari, "DNS-SLAM: SLAM: Dense Neural Semantic-Informed SLAM," in IROS, 2024, pp.
14

---

[90] S. Zhu, R. Qin, G. Wang. J. Liu, and H. Wang. "SemGauss-SLAM: Dense Semantic Gaussian Splatting Slam," arXiv preprint urXiv:2403.07494, 2024.
[91] M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. HAZ- IZA, F. Massa, A. El-Nouby et al., "DINOv2: Learning Robust Visual Features without Supervision." Transactions on Machine Learning Research, 2023.
[92] A. Asgharivaskasi and N. Atanasov, "Semantic OcTree Mapping and Shannon Mutual Information Computation for Robot Exploration." " IEEE Transactions on Robotics, vol. 39, no. 3, pp. 1910-1922 2023.
[93] Gervet, Chintala, D. Batra, J. Malik, and Chaplot, "Navigating to Objects Real World," Science Robotics, vol. 8, no. 79, 2023.
[94] Georgakis, B. Bucher, K. Schmeckpeper, Singh, and K. Daniilidis, "Learning Map for Active Semantic Goal Navigation," in ICLR, 2022.
[95] K. Ramakrishnan, Chaplot, Z. Al-Halah, J. Malik, and Grauman, "PONI: Potential Functions for ObjectGoal Navigation With Interaction-Free Learning," in CVPR, 2022, pp. 18 890-18 900.
[96] Zhang, H. M. Bong, and G. Beltrame, "Active Semantic Mapping and Pose Graph Spectral Analysis for Robot Exploration," in IROS, 2024, pp. 13 787-13 794.
[97] Marza, L. Matignon, O. Simonin, Batra, Wolf, and D. S. Chaplot, "AutoNeRF: Training Implicit Scene Representations with Autonomous Agents," in IROS, 2024, pp. 13 13444-114499 449.
[98] M. Tancik, Weber, Ng. R. Li, B. Yi, Wang, A. Kristoffersen, J. Austin, Salahi, Ahuja et al., "Nerfstudio: A Modular Framework for Neural Radiance Field Development," ACM SIGGRAPH 2023 conference proceedings, 2023, pp. 1-12.
[99] K. Hc, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN," in ICCV, 2017, pp. 2961-2969.
[100] Charrow, S. Liu, V. Kumar, and Michael, "Information- thhoretic mapping using Cauchy-Schwart Quadratic Mutual Information," in ICRA, 2015, pp. 4791-4798.
[101] M. Savva, A. Kadian, O. Maksymets, Zhao, F. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik et al., "Habitat: A Platform for Embodied AT Research," in ICCV, 2019, pp. 9339-9347.
[102] J. Straub, T. Whelan, Ma, Y. Chen, Wijmans, S. Green, J. J. Engel, R. Mur-Artal, C. Ren, S. Verma et al., "The Replica Dataset: A Digital Replica of Indoor Spaces," arXiv preprint arXiv: 1:Xiv:9906.05777, 2019.
[103] A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner M. Savva, S. Song, A. Zeng, and Y. Zhang, "Matterport3D: Learning from RGB-D Data in Indoor Environments," in International Conference on 3D Vision (3DV), 2017.
[104] C. Cao, H. 7hu, 7. Ren, H. Choset, and J. Zhang, "Representation granularity enables time-efficient autonomous exploration in large, complex worlds." Science Robotics, vol. 8. no. 80, 2023.
[105] Z. Yan, H. Yang, and H. Zha, "Active neural mapping," in Proceedings of the IEEE/CVF International Conference Computer Vision 2023, pp. 1098--0099.
15

---

Supplement
In this supplement, provide detailed outline structured follows: Section S.1 offers additional implementation details of ActiveSGM, Section SS2 includes extended quantitative and qualitative results, along with runtime analysis. Section S.3 provides justifications for the checklist items.
S.1 Implementation Details
Hardware and Software. We conducted the experiments on a server with NVIDIA RTX A6000 GPUs and Intel 19-10900X CPU with 20 cores. Our ActiveSGM is implemented with python 3.8 and CUDA 11.7. Please refer to Section S.3.4 for more information about baselines and other packages we used. Our code can be found
httpas///ggithub..omm/////////// htt.1///cciivtttt.. git
OneFormer Finetuning Details. Following the approach ActiveGAMER [63, we implemented
the geometry- exploration criterion to construct fine-tuning dataset. Beginning from random position, the agent performs 500 exploration steps, collecting 500
frame pairs per scene. We then fine-tuned OneFormer [17] separately on the collected data from Replica and MP3D, training for 3,000 steps per scene. The Replica dataset has 101 classes, while MP3D has 40. The fine-tuning process follows the official OneFormer tutorial provided Hugging Face (https //huggingt angeeeegf...... co docs Slocs/trantt
oneformer: The novel trajec- tories described in Table I of the main paper are used the set. These trajectories are distinct from those used for fine-tuning. The train/test Top-1 accuracy is reported in Table S.I
mettmmerrmmminn/ eeeeee/mooaee htt//mode__dooe
Sparse Rendering. We illustrate the semantic rendering process using proposed sparse semantic representation (with fewer classes) in F.g. S.I The overall rendering process proceeds as follows:
For each tile:
For each pixel in the tile:
For each batch of Gaussians in the frustum:
Load batch shared memory fewer classes decreases loading time For each Gaussian the batch:
If pixel is affected: Compute contribution (semantic, alpha) Composite with alpha blending sparse mode needs fever iters. Early exit if opacity is sufficient
Write final semantic
our sparse representation is not used, each Gaussian stores a full probability distribution over all classes, and alpha blending of semantic probabilities is performed by iterating over all classes:
For each Gaussian G_i in the batch: for 1dx in range (ataaummmlssssss)) [[ix]] prob[idx] alpha[idx] transmittance [idx]
where is the rendered probability distribution of each pixel. This becomes increasingly inefficient when the number of classes is large and many probabilities are near zero. For instance, in the Replica dataset with 101 classes plus unknown class, this results in 102 iterations per Gaussian.
In contrast, our sparse rendering strategy stores only the Top-k most probable classes per Gaussian (k number of classes). During rasterization, alpha blending is performed only these sparse indices:
For each Gaussian G_i in the batch: indices topk_indices in G_i for idx in indices:
Table S.I: OneFormer Fintuneing Accuracy
Dataset
Splits Train
Ávg.
Dε0
0f1
0f2
0f3
0f4
RO
97.31
98.75
98.67
99.17
97.35
96.45
98.13
98.83
91.15
Replica 102
Test
89.12
89.07
71.84
92.76
93.18
91.54
87.37
92.25
94.96
GdvgF
g76f7
HxpXQ
pLe4w
YmJkq
Train
93.87
94.37
94,99
93.84
95.22
90.94
MP3D
103
Test
89.77
93.68
92.58
91.21
89,52
81.86
16

---

Table S.2: Semantic Segmentation on ReplicaSLAM. Rendered semantics are evaluated on 4
scenes using the SGS-SLAM 7 protocol, which compares predictions ground-truth categories visible per view. Our method uses semantic predictions from OneFormer and is evaluated on novel views.
Methods
Semantic
View
Avg. Steps +
Avg. mloU (%)
(%)
R1 (%)
(%)
0f0 (%)
NIDS- SLAM
86
GT
Train
2000
82.37
82.45
84,08
76.99
85.94
DNS- UNS-SLAM
89
GT
Train
2000
84.77
88.32
84,90
81.20
84.66
SNI-SLAM
GT
Train
2000
87.41
88.42
87.43
86.16
87.63
OGS-SLAM
Train
92.72
92.95
92.91
92.10
92.90
OneFormer
GT
Novel
3000
65.41
69.06
65.71
67.01
59.85
Ours
Pred.
Novel
713
85.
84.54
85.98
85.40
84.60
[[ixx]
problidxt
alpha[idxx
transmittance [idx]
This reduces the number of memory accesses and blending operations without sacrificing semantic fidelity. By reducing the number stored logits and accessed channels, our sparse representation speeds up both the memory workload, as more Gaussians can be loaded into shared memory, and the Gaussian processing loop, leading to faster semantic rendering. Please refer to Section S.2.2 for quantitative runtime comparison.
idx
prob
prob
idx
prob
idx
prob
idx
prob
Figure S.1: Visualization of Rendering Semantic Map with Sparse Semantic Vector. Each Gaussian only stores indexes and probabilities of the top-k most probable categories, the semantic distribution of the given pixel is rendered following Eqn. (3) in the main paper.
Local Path Planner. We employed the Efficient Rapid-exploration Random Tree (RRT) proposed by NARUTO [61] for local path planing. Once the goal location is determined, we use an efficient RRT-based planner to find path from the current state s, to the goal 8g. using the Exploration Map to measure collision and reachability. (Specifically, the agent should only move within the free voxels defined by the Exploration Map. Additionally, agent avoids regions that are too close to surrounding surfaces.) To speed up planning in large-scale 3D environments, enhance standard RRT by also attempting direct connections between samples and the goal. This greatly improves efficiency.
enforce collision buffer of 20 cm, ensuring the
S.2
Additional Results
S.2.1
Quantitative Results
Semantic Segmentation on ReplicaSLAM We evaluate on scenes following the SGS-SLAM protocol 77 which compares rendered semantic masks to ground-truth labels visible in each view. The full results are shown in the Table S.2 and have been summarized in Table 1 yellow
17

---

Table S.3: Semantic Segmentation on Replica (Novel Views). We present four settings: (1) Fine- tuning results of Oneformer as reference; (2) SGS- retrained using OneFormer predictions, instead of ground-truth labels as used in Table 1 of the main paper-leads to a noticeable drop in performance: (3) Our method without active exploration, which demonstrates the advantage of the sparse semantic representation alone; (4) Our full pipeline with active exploration, which achieves better segmentation performance with fewer steps.
Methods
Metries
Avg.
oro
0f1
Df2
0f3
D14
RO
R1
R2
Steps J
mloU (%)
66.05
62.73
55.67
66.38
70.03
69.81
62.16
74.19
67.43
MAP (%)
84.59
83.29
72.47
87.39
88.36
85.83
81.56
90.68
87.12
OneFormer
17
F-1 (%)
57.96
57.78
40.45
59.51
66.33
47.89
59.20
69.70
62.81
Top-1 1 Ace (%)
f
89.12
89.07
71.84
92.76
93.18
91.54
87.37
92.25
94.96
Top-3 Ace (%)
96.18
96.76
89.10
96.53
97.70
97.29
96.40
96.92
98.76
Steps 1
2000
2000
2000
2000
2000
2000
2000
2000
2000
mloU ($)f
80.42
77.60
75.68
78.70
78.10
89.96
83.23
83,97
76.12
mAP ($a)
89.94
86.37
84.67
88.63
90,98
96.22
92.10
93.80
86.73
SGS-SLAM
F-1 (%)t
18.70
18.35
15.06
19.03
17.68
18.02
25.28
18.47
17.69
Top-1 Acc (%)
94,42
92.68
90.06
93.52
93,42
98.14
97.16
96.71
93.64
Top-3 Acc (%) f
95.53
93.39
90.90
94.54
96,64
98.70
98.00
97.35
94.68
Steps 1
2000
2000
2000
2000
2000
2000
2000
2000
2000
ml)
80.14
74.15
74.88
76.97
79.60
88.29
84.50
84.50
78.23
mAP (8))
90.09
88.91
84.86
86.27
89.12
94.86
93.78
93.78
89.13
Ours (Pussive)
F-1 (%)f
67.81
64.54
53.49
72.42
64.26
66.48
70.18
80.09
71.03
Top-1 Ace (%)
94.05
89.80
89.69
94.99
93.83
97
95.65
95.65
95.16
Top-3 Acc (%) f
96.82
95.00
91.71
96.58
98.71
99.45
98.14
98.14
96.85
Steps 1
777
664
501
749
1175
941
1082
514
591
mloU (%)
84.89
82.58
83,99
83.57
83.40
89.36
84.08
85.28
86.83
mAP (%)
94.39
94.66
91.93
92.86
93.65
96.35
95.19
94.93
95.55
Ours (Active)
(%)f
77.56
73,81
72.53
79.57
75.95
76.80
75.65
83.85
82.33
Top-I Ace (%) f
96.62
94.55
96.07
98.39
94.82
97.75
96.80
96.18
98.40
Top-3 Ace (%) f
99.52
99.76
99.01
99.77
99.51
99.58
99.69
99.05
99.81
Table S.4: Semantic Segmentation MP3D.
Methods
Semantic
View
Avg.
ceilling
appliances
sink
plant
counter
table
mpcat40
SSMI
S2
GT
Train
36.14
46.02
41.01
25.13
39.30
36.12
29.25
TARE [04
GT
Train
31.70
42.01
36.86
23.86
32.51
31.70
23.27
Zhang et
96
GT
Train
42.92
50.73
45.26
43.91
40.42
39.18
37.99
Ours
Pred.
Novel
65.58
70.31
76.95
69.36
73.60
14.03
69.89
55.77
Semantic Segmentation on Replica (Novel Views) To assess generalization, we generate new trajectories near the SLAM trajectories, following the instructions of SplaTAM (51). We present the complete results Table S.3 as supplement to Table blue in the main paper.
Semantic Segmentation on MP3D We also evaluate the average IoU on five large indoor scenes from MP3D (see Table 1 red in the main paper). Table re4 the IoU scores for six common categories, as well the mean IoU across all 40 categories of our method (denoted as 'mpcat40"). The semantic ground-truth meshes provided by MP3D are noisier than the texture meshes, often containing floaters and missing regions. To ensure a fair comparison, we computed the L.I distance from each point in the semantic mesh to its nearest neighbor in the texture mesh, and filtered out all points with distances greater than cm. Points in the texture meshes inherit the semantic label the nearest neighboring point in the semantic mesh if it is within 5 cm, otherwise their labels are set to unknown, and then they are used as ground truth in the evaluation. We show an example of the filtered mesh in Figure S.2
3D Reconstruction and Novel View Synthesis. We evaluate the 3D reconstruction and novel view synthesis (NVS) performance of ActiveSGM on MP3D and Replica. The 3D reconstruction results are reported Table while the NVS results are presented in T.6le Please refer to Section 4.2 for details on how the novel trajectories are generated. Overall, ActiveSGM achieves the best 3D reconstruction and NVS performance on MP3D and performs on par with the state-of-the-e method ActiveGAMER on Replica.
18

---

Original
Filtered
Figure S.2: Filtered Semantic Mesh for MP3D. We present both the original and the filtered semantic mesh from MP3D scene. After filtering, most of the floaters-such as those highlighted in the yellow box-ar are successfully removed. The cleaned meshes are then used for semantic segmentation evaluation on MP3D.
Table S.5: 3D Reconstruction Results Replica and MP3D. Overall, method achieves the best performance on MP3D and ranks second on Replica, delivering higher reconstruction accuracy and improved scene completeness compared prior approaches. Notably, ours is the only method that incorporates semantic information into the exploration criterion, whereas all other baselines rely on geometry- strategies.
Methods
Datasel
Ace. (cm) +
Comp. (cm)
Comp. Ratio (%)
NARUTO [61
Replica
1.61
1.66
97.20
ActiveGAMER X
Replica
1.16
1.56
96.50
Ours
Replica
1.19
1.59
96.68
FBE [58
MP3D
9.78
71.18
UPEN (59
MP3D
10.60
69.06
OccAnt 62
MP3D
9.40
71.72
ANM LOS
MP3D
7.80
9.11
73.15
NARUTO 61
MP3D
6.31
3.00
90.18
ActiveGAMER
63
MP3D
1.66
2.30
95.32
Ours
MP3D
1.56
1.77
97.35
S.2.2 Runtime Analysis
We conduct a runtime analysis using the room0 scene from the Replica dataset to highlight the efficiency of our sparse semantic representation and rendering strategy. The scene, measuring 8m x 4.8 m x 3 m, is explored and mapped by ActiveSGM in 1082 steps over 48 minutes. During the rendering of a semantic map with resolution (340 x 600 x 102), approximately 204k Gaussians are involved in the rasterization process. Using dense semantic representation where each Gaussian carries a full 102-class probability distribution- -tee rendering takes 61 ms. In contrast, our sparse semantic representation significantly reduces computation, requiring only 3.1 ms to render the same map. This improvement stems from the reduced number of active channels during rendering and more
Table S.6: Novel View Rendering Performance Replica and MP3D. We report the average dering metrics across scenes for each method. Our approach delivers consistently strong performance in terms of PSNR, SSIM, LPIPS, and L1 depth error, achieving comparable better results than baselines, ranking as the second-best on Replica and the best on MP3D. Notably, our method the only one that also addresses semantic segmentation.
Method
Dataset
PSNR
SSIM
LPIPS
L1-D 1
SplaTAM (11]
Replica
29.08
0.95
0.14
1.38
SGS-SLAM 7
Replica
27.14
0.94
0.16
7.09
NARUTO 61
Replica
26.01
0.89
0.41
9.54
ActiveGAAMER
63
Replica
32.02
0.97
0.11
1.12
Ours
Replica
30.61
0.96
0.14
1.36
NARUTO 61
MP3D
20.52
0.72
0.58
7.95
ActiveGAMER
63
MP3D
24.76
0.90
0.25
4.83
Ours
MP3D
26.15
0.92
0.26
3.76
19

---

office0
officel
office2
office3
office4
roomo
reom1
room2
Figure S.3: RGB and Semantic Reconstruction for Replica.
importantly from the reduced amount of data transfers the GPU, showcasing the effectiveness of our sparse approach for real-time semantic mapping.
S.2.3
Qualitative Results
We also preset the top-down view visualization of the 8 scenes from Replica in Figure S.3 and scenes from MP3D Figure 5.4 please zoom to see more details.
S.3
Assets Used and Reproducibility
S.3.1
Experimental result reproducibility
We provide sufficient implementation details in Section S.T in the supplement to make results reproducible. We build upon open source software, and our code can be found at htttps://github com//1y00412/Activees git
S.3.2
Experimental setting/details
Please refer to Section 4.1 in the main paper.
S.3.3
Experiments compute resources
Please see the Hardware paragraph in Section S.I in the supplement, and also refer See S.2.2 for runtime analysis.
S.3.4 Licenses for existing assets
Datasets. In this paper, conduct experiments the following publicly available datasets. We list the URLs, license information, and citation for each dataset below.
Replica Dataset 102
URL: https: t/tps://github.b
License: Research or Education httppp/e/gtaatt..cccacccceeeeeee
20

---

GdvgF
g26f7
YmJkq
HxpkQ
pLe4w
Figure S.4: RGB and Semantic Reconstruction for MP3D.
2. Matterport3D Dataset
103
URL: http:///lesseee..iiiiiiiio/a/aaa License: Non-commereial ltttps::/kallir...... in tum. mwt.d///ttterrrr///P (TOS.pdF
Software. We use Habitat-Sin our simulation environment and develop a custom sparse raster- ization CUDA toolkit based on 3D Gaussian Splatting. For mapping, we adopt SplaTAM as the backbone and fine-tune OneFormer to serve as our semantic camera. During evaluation, we also implement SGS-SLAM for comparative analysis. The source code for these components is available at:
1. Habitat-Sim 101
URL:
https///hhtt..oooocccooderertttt git
Liicens: MIT
2. 3D Gaussian Splatting (3DGS) 25
URL: htttttttt:/:/ttbbbbccmm ott.aaaaee///gaaaaaacaareaeeeeee
http:h/wwww..eeaeeeeeeeeiaaaaaaa git
License: gauesaaispaaatiagaararreeeeeeeee www..//ggrpaccoooiiiroaaaal
Custom (https://github.co http:/hwww.oooggoaaaaaaaaaa
3.
SplaTAM [51]
URL:
License: BSD-3-Clause
4. SGS-SLAM 7
URR:
License: BSD-3-Clause
21

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims presented in the abstract and introduction are consistent with the content of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
22

---

This is not theoretical paper, however, it clearly states all relevant assumptions regarding the imputs, outputs, and model.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide sufficient implementation details in the supplement to make our results reproducible. We build upon open source software, and our code can be found on GitHub.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
23

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The code can be found at a public repository. All experiments use publicly available data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccppbliccguddeeeeeeeeeeee Cooee///ddeeemmmmcii ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The section in the main paper and supplementary material provide sufficient details for readers familiar with the field to understand and reproduce the results.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Error bars reported because it would be computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Section and the supplement describe the computer resources used in our experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The datasets we use do not depict people and do not contain any personal, private or health information.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [NA]
This paper presents fundamental research. While the methods could potentially be applicd robotic perception systems that might be misused, such risks are common to many forms of basic scientific research.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
25

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We did not create new datasets and our code does not generate synthetic data. The potential for misuse without major modifications is low.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The supplement contains a list of all assets used with relevant information.
Guidelines:
The answer NA means that the paper does
existing assets. The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
26

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
27

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttpps////eurpss.... /ctp////eeeeeeecc///22/////// for what should or should be described.
28


Reviews:
[{"review_id": "hbWNc33mhS", "reviewer": "Reviewer_fEfZ", "summary": "The authors present an active mapping system based on 3D Gaussian splatting called Active Semantic Gaussian Mapping (ActiveSGM).\nActiveSGM represents the scene using 3D Gaussians (as opposed to NERFs) which is a new contribution. ActiveSGM infers the semantics of the visible surfaces and combines a geometric and semantic uncertainty criterion for planning exploratory actions to visit frontiers. ActiveSGM operates under a closed vocabulary assumption and shows robustness to noisy semantic predictions by leveraging predictions from a pre-trained model (relevant in real world systems) and balances memory constraints when dealing with many semantic classes in a scene by using the top-k semantic classes instead of all of them.\nAnother key part of the paper is using 3D Gaussians as a representation of the scene instead of NERFs. This is relatively new and the addition of semantics is a novel contribution to such systems.", "questions": "4.2 ReplicaSLAM\nWhat part of the SGS protocol are we talking about here? Is there an active mapping component in this experiment?\nIf we’re just considering a pre-defined trajectory, then only point 1 (using OneFormer instead of ground truth) seems to be different from SGS-SLAM. The other points are only related to training evaluation and training data. In that case, what component of ActiveSGM are the authors trying to highlight with this experiment?\nIf we are considering an active mapping, then how are the number of steps coming into play - because SLAM systems would work with a pre-defined trajectory. So I am unable to understand the purpose of “steps”\nWhat does the training refer to here? Does it refer to fine-tuning the OneFormer model? Based on my understanding, all the components mentioned in section 3 talk about the deployment phase of ActiveSGM, so what does training on 1/3 rd of the data mean for ActiveSGM. Additionally, while giving ActiveSGM a more difficult evaluation in terms of less training data and evaluation on novel views seems interesting, not providing the results while considering the same amount of data raises questions on the ceiling of the ActiveSGM performance - how much better is it compared to the baselines if given all the training data and evaluated on training scenarios or does the performance saturate. And what are the benefits compared to the baselines when the baselines are evaluated on the novel views and ActiveSGM is evaluated in novel views. The current set of of experiments shows different conditions for baselines and ActiveSGM and warrants more controlled evaluation conditions - the current experiment reads like the following - “We gave ActiveSGM less training data and also evaluated on novel views and it performs similar to some baselines and worse than the best” - it is important to have the apples-to-apples in the testing conditions to properly appreciate the system. Alternatively, if there are some holes in my understanding of the experimental setup then please point it out.\n4.2 Replica (Novel Views)\nActiveSGM without exploration performs better than SGS-SLAM, thereby showing the benefits of the sparse representation.\nIt is hard to appreciate the benefits of the sparse representation because the paragraph does not properly emphasize the differences between ActiveSGM and SGS-SLAM. Is the following elaboration appropriate - “ActiveSGM without exploration is the same as using a predefined trajectory and doing SLAM. This is very similar to SGS-SLAM but SGS-SLAM compresses semantic information to r,g,b while ActiveSGM uses the sparse representation. ActiveSGM outperforms SGS-SLAM and hence shows the benefits of the sparse representation over compressing semantics into r, g, b?”. If so, then making this more clear will improve the clarity of the paper. If not, please explain how the authors arrive at that conclusion.\n4.2 MP3D\nThe authors mention that compared to the baselines, ActiveSGM does not rely on ground truth semantic labels and still performs better. Based on the content in the paragraph, it is hard to understand what choices of ActiveSGM compared to the baselines lead to the improved performance. Is it the 3D Gaussian representation? Is it the exploration criterion?\nWhile the individual components of ActiveSGM are explained in Section 3, not emphasizing the differences between ActiveSGM and the baselines in the experiments section makes it hard to understand “why” or “what components” might be contributing to better performance (especially for reader who aren’t extremely well versed with all the baselines)\nIt can be hard to provide an exact answer but a proper discussion on what changes and how that affected the results would provide clarity to readers.\nMP3D experiment (NOT CRITICAL FOR REBUTTAL) - Did the authors consider using ground truth labels for ActiveSGM - would the performance get better or", "limitations": "See above.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\nThe paper describes a method for active dense mapping using 3D Gaussians as the representation. In addition, ActiveSGM is demonstrated with model predictions (without assuming ground truth semantic labels) and hence describes a good proof-of-concept for utilizing 3D Gaussians and semantic information for active mapping\nThe details of ActiveSGM are easy to follow\nWeaknesses\nMy main criticism is related to the writing in the experiments section of the paper - as a reader I found it hard to grasp what components of ActiveSGM led to better performance over baselines in the individual experiments. I will elaborate on these in the Questions section.\n\nQuality\nThe paper explains the components of the ActiveSGM pipeline well and is easy to follow.\nThe explanation of the exploration heuristics is also fairly intuitive and well explained.\nThe evaluation metrics and qualitative examples highlight capabilities of the proposed method\nClarity\nThe description of the ActiveSGM algorithm is clear. However, when I read the paper, it seems like I could not fully grasp the similarities and differences to previous methods which led to the improvements - a way to mitigate it might be to explicitly talk about the similarities and differences in the components (semantic representation, 3DGS vs NERF, ground truth vs semantic predictions, etc) while explaining the results from the experiments. In some cases the information is there but currently does seem to be emphasized enough. For example, the section on 3D Reconstruction and Novel View Synthesis shows how AvticeSGM is better than ActiveGAMER. But through the paper, I only understood the similarities to ActiveGAMER in terms of objectives and the fact that ActiveSGM does not have photometric refinement stage. So what are some reasons ActiveSGM performed better than ActiveGAMER?\nSignificance\nThe paper presents a method for leveraging 3D Gaussians as the scene representation for active mapping. ActiveSGM also incorporates a sparse representation of semantics and demonstrates active mapping with semantic predictions instead of utilizing ground truth semantic labels.\nOriginality\nThe method is original.\nThis paper on using NERFs for active perception seems relevant to the problem at hand (Active Perception using Neural Radiance Fields https://arxiv.org/abs/2310.09892). They try to maximize a predictive information objective. While they assume ground truth semantic labels, this paper could be an important related work. It would be nice to understand how this fits into the current work and an explanation of the benefits / drawbacks of this compared to ActiveSGM would be helpful to provide more clarity in the paper", "was_revised": true, "final_justification": "I have read the rebuttal and the comments with the other reviewers. Overall, while I appreciate that there are differences in the proposed to prior work - in particular, ActiveGAMER - those are not particularly large deviations from current SOTA. The results also show relatively minor performance improvement. Furthermore, the work is limited to simulated datasets, though I appreciate the difficulty of running real world experiments. In sum, I do not feel that for this top venue, the work, at this time, makes a sufficient contribution.", "rebuttal": "We thank the reviewer for the thoughtful review and encouraging feedback. We appreciate your recognition of our **practical setup** using predicted semantics, the **potential of combining semantic understanding with 3DGS**, and the **clarity of our pipeline and exploration heuristics**. We're glad the **evaluation effectively highlighted our method’s capabilities**. \nHere, we respond to the concerns and suggestionsraised by the reviewer.\n\n```\nQ1.1 Experiments section lacks clarity on what drives performance\n```\n\nWe will substantially rewrite the experimental results section to better delineate concepts we inherited, such as benchmarks, metrics and baselines, and the contributions of our work. \n\n```\nQ1.2 Advantage over ActiveGAMER is not clearly justified\n```\n\nWhile ActiveGAMER focuses on actively acquiring geometric and photometric information for 3D reconstruction, ActiveSGM is specifically designed for semantic-aware active mapping. There are three key differences that contribute to ActiveSGM’s improved performance:\n\n1. **Semantic-Aware Exploration**: ActiveSGM maintains a semantic map with sparse representation during exploration, enabling the system to reason explicitly about semantic coverage and uncertainty—something ActiveGAMER does not address.\n\n2. **Joint Semantic and Geometric Criteria**: Our exploration strategy leverages both geometric completeness and semantic uncertainty to select informative viewpoints. This leads to more targeted exploration, particularly in semantically rich or ambiguous regions.\n\n3. **No Post-Refinement Required**: Unlike ActiveGAMER, which relies on a post-exploration photometric refinement stage, ActiveSGM’s semantic feedback loop encourages natural revisiting of uncertain areas, making the mapping process more efficient and integrated.\n\nTogether, these design choices allow ActiveSGM to produce higher-quality reconstructions with stronger semantic fidelity, even without relying on additional refinement stages.\n\n```\nQ1.3 Component-wise differences to prior work are underexplained\n```\n\nTo address this, we highlight that our three core experiments are designed to isolate and evaluate specific aspects of our semantic mapping pipeline. The bullets below refer to Table 1.\n\n1. **Replica-SLAM (yellow)**: We compare against NeRF/3DGS-based methods using dense RGB-encoded semantics and ground-truth labels, while our method uses only predicted semantics and still shows robustness by correcting noise over time. \n2. **Replica-NVS (blue)**: We compare fairly against SGS-SLAM by disabling its tracking module and providing GT poses, isolating mapping performance and showing that our sparse semantic representation is both more memory-efficient and more accurate than dense alternatives. \n3. **MP3D (red)**: We evaluate generalization in large-scale scenes and show that ActiveSGM, with full 3D path planning and no reliance on GT semantics, outperforms both geometry-only and semantic mapping baselines under the same RGB-D input. \n\nThese comparisons collectively demonstrate the contribution of each design choice in our pipeline. We also refer the reviewer to the response Q2.6 to Reviewer 7urF for a more comprehensive comparison.\n\n```\nQ1.4 He et al. on active perception is missing\n```\n\nWe will cite this paper. Compared to their method:\n- They require GT semantic masks; we use pseudo labels,which is more practical in real-world settings.\n- Their planning involves long-horizon, dense sampling; ours uses adaptive step-wise planning.\n- They retrain NeRF after each planning iteration; we optimize keyframes online without storing the full history.\n- They report no semantic metrics, limiting direct comparison.\n\nIn contrast, ActiveSGM emphasizes semantic-guided, memory-efficient, online refinement.\n\n\n```\nQ1.5 Clarification on the Active Mapping task\n```\n\nUnlike passive mapping or SLAM, active mapping introduces **autonomous decision-making** into the mapping process. SLAM and passive mapping systems may operate on pre-recorded trajectories or under externally controlled by a human operator or predefined script. These systems passively consume data without influencing where the sensor moves or what information is acquired. \nIn contrast, active mapping empowers the agent to plan its own trajectory in real time, actively selecting the most informative viewpoints and deciding when to stop exploration. The motivation behind active mapping is to achieve more efficient and targeted scene understanding by coupling perception with action, rather than relying on exhaustive or unguided data collection.\n\n\n```\nQ1.6 [ReplicaSLAM]: Clarification on comparison protocol\n```\n\nWe appreciate the reviewer’s detailed questions and would like to clarify the setup and intent behind the ReplicaSLAM experiment (Yellow Band in Table 1).\n\n**Baselines**: The results are directly copied from SGS-SLAM [7]. These baselines operate under passive SLAM settings, taking RGB-D inputs with ground-truth semantic labels, and evaluate performance on the observed (training) frames. They rely on manually captured sequences and do not include any active exploration components.\n\n**Our Setup**: In contrast, our method operates under an active mapping setting, where the system autonomously selects which views to observe. We use posed RGB-D as input, and semantics are predicted on-the-fly using a finetuned OneFormer model. As our system plans its own trajectory, it does not observe the same frames used by the baselines—those frames are considered novel views to ActiveSGM.\n\n**Evaluation Protocol**: To ensure fairness and use the same evaluation testbed, we evaluate ActiveSGM on the same frames used by the baselines, even though our system has not seen these views during exploration. This allows a direct comparison in reconstruction quality while keeping the test set consistent.\n\n**Purpose of This Experiment**: This experiment is designed to highlight the effectiveness of our semantic mapping pipeline, particularly:\n\n(1) Our sparse semantic representation achieves accurate reconstruction even without access to ground-truth semantic labels.\n\n(2) Our semantic-guided exploration criterion helps select informative views, which improves both geometric and semantic reconstruction—despite relying on noisy pseudo labels.\n\nWe hope this clarifies that the goal of the ReplicaSLAM experiment is not to compare exploration strategies directly, but rather to showcase the strength of our semantic representation and active observation selection under a more realistic, challenging setting.\n\n```\nQ1.7 [ReplicaSLAM]: Clarification on \"step\", \"training\", and \"data usage\"\n```\n\nWe appreciate the reviewer’s questions regarding the meaning of “steps” and “training” in our experimental setup. The term “steps” is used in two contexts:\n\n1.  For OneFormer, it refers to the number of fine-tuning iterations.\n\n2. In SLAM and active mapping pipelines, “steps” refer to the number of observation frames used for optimizing the map representation. Typically, each observation frame results in one optimization step—thus, the number of steps is approximately equal to the number of observed frames.\n\nThe term “training” also has two meanings:\n\n1. For OneFormer, it refers to the offline fine-tuning process before deployment.\n\n2. For SLAM/mapping, it refers to optimizing the scene representation from observations, which is commonly called “training” in radiance field literature.\n\nWhen we mention “training on 1/3 of the data,” we mean that ActiveSGM only observes and optimizes its scene representation using one-third the number of RGB-D frames, which may not necessarily correspond to the same observations used by the baselines. We will revise the text to clarify the terminology.\n\n\n```\nQ1.8 [ReplicaSLAM]: Performance of ActiveSGM Trained on All Views vs. Active Pipeline \n```\n\nDue to the limitation of pages, we cannot show all the experiments in the main paper. Please also see experiments in the supplement Table S.4, in which we also train ActiveSGM in a passive maner. As clarified in Q1.6, SGS-SLAM is trained and evaluated on the same trajectory. For a fair comparison, we construct a novel trajectory and evaluate both ActiveSGM and SGS-SLAM on it. \nThe results show that even training on all views of the SLAM trajectory yields lower performance compared to our active pipeline. This supports the core motivation of ActiveSGM — actively selecting informative trajectories leads to better reconstruction and semantic understanding than passively using all available observations. \n\n```\nQ1.9 [Replica (Novel Views)]: Experimental result interpretation (Sparse Semantic Representation)\n```\n\nYes, the interpretation is mostly correct. ActiveSGM without exploration is the same as using a predefined trajectory and doing *Semantic* SLAM *(w/o tracking)*.\n\nAs we shown in Figure 3, the color-coding ambiguity issue is inherent in RGB-encoding used by SGS-SLAM. ActiveSGM maintains a global semantic distribution rather than relying on RGB features, which avoids this problem and leads to a better semantic understanding.\nWe will clarify this in the revised version.\n\n```\nQ1.10 [MP3D] Reasons for improved performance \n```\n\nThe improved performance comes from both the 3D Gaussian representation and the semantic exploration criterion — unlike baselines in Table 1 (blue) that use hard semantic assignments (each voxel can only have one class label), our soft semantic representation (distribution of most likely classes) combined with active view selection leads to better semantic and geometric reconstruction, as shown in Tables S.6 and S.7.\n\n```\nQ1.11 Differences between baselines and ActiveSGM\n```\n\nWe refer the reviewer to Q1.2..\n\n```\nQ1.12 Usage of ground-truth semantic labels\n```\n\nWe did not use ground-truth semantic labels in ActiveSGM, as our focus is on the more realistic scenario of handling noisy observations; instead, we aim to improve reconstruction performance using only imperfect pseudo labels."}, {"review_id": "ytm22wNakF", "reviewer": "Reviewer_7urF", "summary": "This paper proposes an active semantic mapping technique based on the ActiveGAMER active mapping technique.  The paper proposes a way to extend the Gaussian splatting map representation to include a sparse representation of the semantic information associated with a Gaussian component in the map, a way to update the semantic information using an uncertain measurement, as wel as incorporating the semantic information in the map into an exploration criterion used to determine the next best view used for active mapping.  The proposed approach is tested using a number of simulated scenarios, and evaluated against a number of different competing methods; it achieves good results that compares well with that of competing methods.", "questions": "- Are there any aspects of this review that are mischaracterisations of the proposed approach or the presentation thereof?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Quality:\nThe proposed approach can be viewed as a fairly simple addition of semantic representation to an existing active mapping approach.  This is done fairly well and is supported by good results; however, this aspect of the paper has its weaknesses:\n- One of the main contributions of the paper is the augmentation of a Gaussian splatting map representation with a distribution over semantic labels.  This distribution is approximated to reduce the storage complexity of the representation.  The approximation is a very simple one: The $k$ most likely labels as seen *from the initial observation* (line 191) are stored and updated.  This will most likely cause the proposed approach to be brittle, since it cannot recover from a case where the correct semantic label is incorrectly discarded based on the initial observation (it conceivable that this could regularly happen in certain types of scenarios).  An obvious alternative representation is to retain a distribution over all the semantic labels, but to group all the labels with a low likelihood into a \"super category\"; this would allow a similar reduction in storage requirement as that of the proposed approach, but would allow the mapping technique to recover from inaccurate semantic observations in the initial view.  \n- Another contribution is the incorporation of the semantic map representation into the exploration criterion used for active mapping.  However, this is done very simply (Eq. 8) and is unmotivated; this aspect of the paper could have benefitted from a more principled development.  \n- The proposed approach is a simple adaption of the active mapping technique ActiveGAMER (ref. 74 in paper; see \"Originality\" for a discussion).  As a result, the proposed approach inherits weaknesses inherent to ActiveGAMER.  For example, the proposed approach seems to perform active mapping by calculating the next-best view, and then navigating to this view (Fig. 1).  This is quite a rudimentary form of active mapping, and it would have been better to plan a trajectory based on the expected information gain *along the trajectory*.  It is also not clear how the proposed approach could be extended to function in real-world scenarios, where it might not be possible to reach the calculated next-best view, and even if it is reachable, the very simple motion cost model used would probably be inappropriate.\n\nThe proposed approach was tested in a number of different ways and evaluated against several other competing methods.  However, this aspect of the paper also has some minor weaknesses:\n- The proposed approach is only tested in simulated scenarios.  Although the simulation environments are widely used and photorealistic, the lack of testing on real-world data causes uncertainty about the applicability of the proposed approach to real-world scenarios.  Testing active mapping in the real world would have been a significant undertaking though.  \n- The paper evaluates the proposed approach against a range of different techniques.  However, some of the competing techniques address a different problem (SLAM vs. mapping, and active *semantic* mapping vs. active mapping), and some of the competing techniques are not evaluated in exactly the same way (see discussions under \"Clarity\" below).  This makes it difficult to reliably draw conclusions from the results of the experiments.  The paper could have benefitted from a more systematic evaluation, isolating the causes for the differences in results seen by the evaluated methods. For example, since the semantic representation chosen for the proposed approach is one of the main contributions of the paper (see \"Originality\" below), an experiment that directly compares the proposed representation with alternatives would have provided a much better way to evaluate this design choice.  \n\n### Clarity:\nThe paper is mostly well written, clear and neat.  The ideas presented in this paper and how they are related are quickly clear on an overview level after a quick read-through.  However, when reading carefully through the paper, a number of issues with the presentation become clear, including the following:\n- The reasons for reviewing SLAM techniques (lines 76-84 and 118-140) and evaluating against SLAM techniques (Sec. 4) are not clear, since the paper proposes a semantic *mapping* technique.  If the goal is to review and evaluate against the *map representation* of existing SLAM techniques, then it has to be done in a much more motivated and nuanced manner than is done in the paper, since the contexts are different.  The statement in line 84: \"We use SplaTAM [51] as the SLAM backbone of our algorithm,\" which is obviously not accurate, compounds this issue.  For readers not already familiar with the field, it would be beneficial to mention the difference between mapping and SLAM.  \n- Although the related work is discussed extensively and clearly, the relationship between the proposed approach and the related work could have been presented better.  For example, two active semantic mapping techniques, SSMI and TARE (refs. 100 and 101 in the paper) are evaluated against, but they are not included in the literature review of Sec. 2, and they are therefore not characterised at all.  \n- The proposed method uses local and global keyframes (see Fig. 1 and lines 226-230), but their meanings and the distinction between them are never given (the discussion in lines 226-230 provides no help).  \n- The system overview in Fig. 1 shows a path planner that forms part of the exploration module; however, this path planner is not described anywhere in the paper.  \n- The Gaussian splatting representation used in the proposed approach, as well as the loss function used for training are discussed in Sec. 3.1, but the training procedure is not discussed at all.  For example, it is unclear how Gaussian components are generated and how their parameters are subsequently determined.   \n- There are several aspects of the map representation that is unclear or ambiguous, including:\n  - According to the rendering procedure in Eq. 1, *every* Gaussian component in the map will infuence *every* pixel in the image.  There is presumably some cut-off where the effect of a Gaussian component below a certain threshold is excluded, but this is not mentioned at all. \n  - The map representation seems to require RGB-D inputs (see lines 176 and 179), but this requirement is never explicitly stated, and it is not clear whether the proposed approach can be applied to RGB images.  \n  - The meaning of $f_i(\\mathbf{p})$ is not given (line 177), and consequently Eq. 1 cannot be interpreted with certainty.  \n  - For the definition of the semantic loss (lines 204-219), the OneFormer observations are used as ground truth and the projected label distributions as predictions.  To remove uncertain \"ground truth\" labels, a mask is defined based on the entropy of the distribution over the labels (line 207), but this entropy is then defined in terms of the *predicted* label distribution (Eq. 4), which does not make sense to me.  \n- In the definition of the geometric exploration criterion (Eq. 6), the pixels that have a nonzero silhouette value are counted.  However, according to the definition of the silhouette in Eq. 1, no pixel will have a silhouette value that is exactly zero (except at initialisation), and the geometric exploration criterion in Eq. 6 therefore simply counts the number of pixels in the image (which does not make sense).  There is presumably some cut-off where the silhouette projection to a pixel that falls below a certain threshold is considered to be zero, but this is not discussed at all.  The motivation for the way in which the geometric and semantic exploration criteria are combined (Eq. 8) is also not given.  \n- The results initially appear to be quite good, but at closer inspection, there are some issues with the experimental setup and results that are not properly discussed, and which create some uncertainty.  These include:\n  - It is unclear why the F-1 scores in Table 1 are only give for some entries.\n  - It appears that results of the first 4 rows of Table 1 have been lifted from the SGS-SLAM paper (ref. 7 in the paper), but this is not mentioned, and there is therefore questions about whether the experimental setups are the same and consequently whether the comparison is fair.  \n  - The first 4 rows of Table 1 are all results of SLAM methods, which addresses a different (although related) problem than that of the proposed approach, which addesses the mapping problem.  An assumption of the mapping problem is that the camera poses are known with certainty, whereas for the SLAM problem, the poses are estimated.  The availability of exact knowledge of the camera poses could possibly be a significantly advantage for the proposed approach when comparing the quality of the maps; however, this fact is not discussed in the interpretation of the results.  \n  - Line 328 seems to say that results for the first 3 rows in the red band in Table 1 have been lifted from the paper by Zhang *et al.* (ref. 92 in the paper); however, it was not clear to me whether this was the case (I could not find the results quoted in Table 1 in the paper by Zhang *et al.*).  In addition, lines 326-328 state that the proposed approach is not evaluated on exactly the same data as the competing methods, and it is consequently uncertain whether any reliable conclusion can be made from a comparison between the results.\n  - Fig. 3 illustrates issues with the predicted semantic labels for SGS-SLAM in one scene and compares it with the ground-truth labels.  However, the predicted labels for the proposed approach for the same scene is not shown, and the relevance of this figure is therefore not clear.  In addition, it is unclear whether the issues are influenced by the pose uncertainty inherent to the SLAM problem, which makes the relevance of the figure even less clear.  \n  - Line 294 states that top-1 and top-3 accuracy results are given, but I could not find it anywhere in the paper.  \n\n### Significance:\nDespite the limited nature of the new ideas presented in this paper (see \"Originality\" below), the good results obtained in the experiments and simplicity of the new ideas give an indication that the ideas will most probably be adopted by the research community.  \n\n### Originality:\nThe technique proposed in this paper is heavily based on the active mapping technique ActiveGAMER (ref. 74 in the paper).  Specifically, the geometric and colour map representation and approach to learning the map, the exploration map and approach to select the next-best view, the path planner, and the keyframe scheme are identical.  Essentially, the proposed approach only adds:\n- a representation for incorporating semantic information into the map,\n- a way to project the semantic information stored in the map to an image and use it to train the map, and \n- the incorporation of the semantic information in the map into an exploration criterion.  \n\nNevertheless, the proposed does indeed seem to be the first active semantic mapping technique based on the Gaussian splatting map representation (lines 349-350).  Good results also support the contribution.  \n\nA note about contextualisation: Although ActiveGAMER is appropriately mentioned and referenced, the extend to which the proposed approach is similar to ActiveGAMER is only clear after a careful read of the paper.  A clearer contrast between the two methods would have been better.  \n\n### Other:\nThere are a number of minor issues in the paper, including the following:\n- Line 82: \"TSDF\" is undefined.  \n- Lines 124, 126 and 141: Quotation marks should be formatted better.\n- Line 141 states that \"All the above approaches are \"passive\"\", but this is not true since active mapping is discussed in lines 85-105.  \n- Line 176: $c_i$ should be $\\mathbf{c}_i$, and $d_i$ is undefined.  \n- Eq. 7: A bracket is missing.  \n- Table 1: The meaning of \"GT\" is undefined; the reader has to guess that this refers to the ground truth.  \n- Lines 320-321: \"and introduce arbitrary\" -- sentence incomplete", "was_revised": true, "final_justification": "The authors' responses mostly confirmed my initial view of the paper:\n- It is the first active semantic mapping technique based on the Gaussian splatting map representation.  \n- It is heavily based on the ActiveGAMER active mapping technique, which is simply extended to incorporate semantic information.  \n- It achieves good results, but the experiments could have been designed better to be more informative.", "rebuttal": "We sincerely thank the reviewer for recognizing the strengths of our work. We appreciate the positive remarks on the **clarity and simplicity of the method**, as well as the acknowledgment of its **strong empirical performance**. We're especially grateful for the recognition that the approach shows **potential for impact and adoption by the research community**.\nHere, we respond to the suggestions and concerns raised by the reviewer.\n\n\n```\nQ2.1 [Quality] \"Super category\" for semantic representation\n```\n\nThanks for the suggestion. We will explore integrating \"super categories\" for other classes in future work.\n\n```\nQ2.2 [Quality] Motivation of the exploration criterion\n```\n\nOur semantic exploration criterion is designed to favor observations that \n\n(1) cover previously unseen regions (areas without existing Gaussians), \n\n(2) reduce semantic uncertainty (entropy of the predicted class distribution), and\n\n(3) low travel cost to encourage efficient exploration. \n\nThese components are combined to form a practical and lightweight criterion that balances coverage, semantic gain, and efficiency. A similar heuristic is used in Ref [71], Eq. 9.\n\n```\nQ2.3 [Quality] Next-best-view vs. entire trajectory planning\n```\nWe use step-wise next-best-view planning with a shorter horizon (0.5 m) to enable more efficient and adaptive exploration. Long-horizon trajectory planning, like in He et al. (mentioned by Reviewer fEfZ), requires dense sampling and costly post-optimization. In practice, planning is cheaper than moving the robot. Our method integrates new observations on-the-fly and avoids continual learning issues in NeRF.\n\n```\nQ2.4 [Quality] Real-World Applicability\n```\nWe account for real-world constraints by maintaining an occupancy grid with a 0.2 m collision buffer, restricting the agent to move only within free voxels. These ensure that planned viewpoints are both reachable and obstacle-free.\n\n```\nQ2.5 [Quality] Simulation vs. Real World\n```\nReal-world evaluation is challenging due to the lack of well-annotated RGB-D-semantic datasets and the difficulty of deploying all baselines on physical robots. For this reason, prior semantic mapping methods are also evaluated only in simulation, making photorealistic environments the most practical and fair testbed for benchmarking.\n\n\n```\nQ2.6 [Quality] Comparisons with Competing Methods\n```\n\nWe thank the reviewer for highlighting the concern regarding comparing our method with a range of techniques addressing slightly different problems. We agree that such comparisons need to be handled carefully, and we clarify our intentions and methodology as follows:\n\n**Semantic-SLAM vs. Semantic Mapping:**\n    To the best of our knowledge, most NeRF/3DGS-based semantic mapping methods—such as NIDS-SLAM, DNS-SLAM, SNI-SLAM, and SGS-SLAM are formulated as semantic SLAM systems that jointly solve for localization and semantic mapping. In contrast, our method assumes known camera poses, focuses solely on semantic mapping. Therefore, we agree that direct performance comparisons may not be entirely fair, as SLAM inherently involves the harder task of localization. \n    Furthermore, many SLAM methods use ground truth semantic labels, while our method uses noisy semantic predictions from a pre-trained segmentation model, adding further difficulty to our task. \n    To account for this, we include the yellow band results in Table 1 solely for reference and completeness. We explicitly do not claim superiority over SLAM systems in this context.\n    \n**Fair Comparison via Controlled Setup (Table 1 – SGS-SLAM variant):**\n      \nTo enable a fairer comparison with semantic SLAM methods, we select SGS-SLAM as the strongest representative baseline and reconfigure it to function as a semantic mapping-only system: \n(1) Disable its tracking module and use ground truth poses. \n(2) Replace ground truth semantics with semantic predictions.\nThis allows us to directly evaluate the semantic representation quality, which is a key contribution of our method. We then disable our own active exploration module to ensure a passive, mapping-only setting, enabling a fair apples-to-apples comparison between semantic representations.\n\n**Active vs. Passive Mapping (Table 1 – Blue):**\nBy comparing ActiveSGM (with exploration enabled) against its passive variant and against the above baselines, we highlight our method achieves better segmentation with fewer observations/steps, demonstrating the effectiveness of our active exploration strategy guided by semantic and geometric uncertainty.\n\n**Active 3D Mapping and View Synthesis (Tables S.6, S.7):**\nWhile some baselines focus on active mapping tasks without semantics (e.g., for 3D reconstruction or novel view synthesis), we ensure fair comparisons by using posed RGB-D input across methods. Our semantics are predicted from RGB data, not an extra input. This allows us to show that even with semantic reasoning added, our system remains competitive in standard active mapping tasks.\n\n**Active Semantic Mapping Baselines (Table 1 – Red):**\nWe also include comparisons to recent active semantic mapping baselines, which are more aligned with our task definition. These further validate the advantage of our sparse semantic representation and active policy.\n\nWe hope this clarifies the rationale behind our evaluation setup and demonstrates that we took significant care to ensure fair and interpretable comparisons across different problem settings. We will make these clarifications more explicit in the final version of the paper.\n\n```\nQ2.7 [Quality] Evaluation of Semantic Representation\n```\n\nPlease also see supplement Table S.4, in which we also train ActiveSGM in a passive maner using the same frames of SGS-SLAM. The result shows that our sparse semantic representation maintains more accurate semantic understanding than RGB-encoding representation.\n\n```\nQ2.8 [Clarity] SLAM or Mapping\n```\nWe will clarify the related work. Here we mainly focus on the mapping moudle of previous methods, specificlly for those using 3DGS or NeRF as mapping backbones.\n\n```\nQ2.9 [Clarity] SplaTAM Misstatement\n```\nApologies for the misstatement — we use the mapping backbone of SplaTAM, not the full SLAM system. We will correct this in the revised version.\n\n```\nQ2.10 [Clarity] Missing related work discussion (SSMI and TARE)\n```\nThese were missed due to oversight and will be included in the final version. They are used as baselines in Ref [92].\n\n```\nQ2.11 [Clarity] Keyframe Definitions\n```\n\nPlease see L220-230. Local keyframes are the those have the highest overlap within all observations, as designed in SplaTAM. The global keyframes are described in L226-230, and are designed for maintaining global semantic information.\n\n```\nQ2.12 [Clarity] Path Planner\n```\n\nDue to the limited pages, we described the path planner in the supplement, L 51-58. The complete path planner is: we use the exploration criterion to decide the destination, and use the local path planner to decide the path from the current position to the destination. \n\n```\nQ2.13 [Clarity] Questions about 3DGS\n```\nWe follow the standard training scheme for the Gaussian components as in SplaTAM (Ref [51]) and 3DGS (Ref [25]); please see Supplement L146–L150 (all training configurations are available online). Equation 1 follows the standard volumetric rendering process using alpha blending. $f_i(p)$ in Eq. 1 denotes the probability that a ray stops at sampling position $p$ (see also Eq. 3 in Ref [22]). In practice, we apply an early stopping criterion when the accumulated transmittance drops below 0.01, effectively ignoring negligible Gaussian contributions.\n\n```\nQ2.14 [Clarity] RGB-D Requirement\n```\nOur method requires RGB-D input, as we focus on semantic mapping with accurate geometry. If only RGB images are available, an additional depth estimation module would be needed to generate the necessary 3D structure.\n\n```\nQ2.15 [Clarity] Entropy-Based Filtered Mask\n```\nApologies for the confusion. The entropy filter at L207 is applied on the OneFormer prediction, not on the rendered semantic mask.\n\n```\nQ2.16 [Clarity] Geometric Exploration Criterion\n```\nThe unobserved regions do not contain any Gaussians, so the silhouette mask value in those pixels is 0 by definition.\n\n```\nQ2.17 [Clarity] F-1 / Top-1 / Top-3 Accuracy\n```\nOther baseline methods report only mIoU. To provide a more comprehensive evaluation, we select the strongest baseline (SGS-SLAM) and additionally compute F-1, mAP, Top-1, and Top-3 accuracy, as shown in Table S.4 of the supplement.\n\n```\nQ2.18 [Clarity] Yellow Results in Table 1\n```\nYes, the yellow-band results in Table 1 were copied from the original publications. We include them for reference because they are among the few methods that use 3DGS or NeRF as a semantic representation.\n\n```\nQ2.19 [Clarity] SLAM vs. Mapping Results\n```\nFor the experiments in Table 1 (blue) and Tables S.4–S.7, all baselines are run with RGB-D inputs. To ensure fairness, we disable the tracking modules in SLAM baselines and use ground-truth camera poses.\n\n```\nQ2.20 [Clarity] Results from Zhang et al.\n```\nDue to space constraints in the main paper, we report averaged scores from Table 2 of Zhang et al. Full per-class results are provided in Table S.5 of the supplement.\n\n```\nQ2.21 [Clarity] Purpose of Figure 3\n```\nThe purpose of Figure 3 is to illustrate the color-coding ambiguity in RGB-encoded semantics used by SGS-SLAM. In contrast, ActiveSGM maintains a global semantic distribution, avoiding such ambiguity. To eliminate pose-related confounds, we provide ground-truth poses to SGS-SLAM in this comparison.\n\n```\nQ2.22 [Originality] Significance of ActiveSGM, v.s. ActiveGAMER\n```\nPlease refer to our comparison between ActiveGAMER and ActiveSGM (Q1.2), where we discuss the significance of our contributions in terms of accuracy, efficiency, and semantic fidelity.\n\n\n```\nQ2.23 Other minor issues\n```\nThanks for pointing out the issues. We promise to fix them in the final version."}, {"review_id": "CcMA5VFTpj", "reviewer": "Reviewer_MRuN", "summary": "The authors present the first method for active semantic mapping with neural radiance fields. They propose a representation in which cropped semantic probability vectors are stored in a 3DGS backbone. The objective for active mapping simultaneously incentivizes regions of high semantic entropy computed from the semantic probability vectors and low geometric coverage computed from missing pixel counts. This approach is validated on the Replica and Matterport3D datasets and shows additional capability over alternate approaches while simultaneously achieving strong (often state of the art) performance on accurate and efficient mapping.", "questions": "I am confused about the notation around when and how the top k probability class values are cropped in the semantic vector. It seems to me that \\tilde{P} is the cropped semantic vector and P is the uncropped version. However, \\tilde{P} is only used once in the method section, but it seems that the vector would necessarily be cropped prior to entropy calculations during . Could you clarify?\n\nDid you perform hyperparameter tuning to find the chosen values of each lambda in equation 5? Is the method very sensitive to the choice of hyperparameters?\n\nDo you think that your method could be extended in future work to incorporate open vocabulary semantic segmentation models? Or would it require rethinking the entire approach?", "limitations": "Yes, particularly in the supplement.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Overall, this is a strong paper which I enjoyed reading. The method fills a clear gap in current literature and is a novel and well presented approach. The literature review is thorough and educational across the different axes of research that come together to make the contribution in this work. The experiments are thorough and show a clear advantage over the baselines. The baselines are well selected and the differences between them and the new approach is clearly explained. The tables and figures are easy to understand.\n\nAs a suggestion for improvement, the results in the supplement regarding runtime performance are significant and should be included in the main paper with extra space on publication. In addition, the limitation regarding localization and execution should be ultimately included in the main paper. The immediate next question regarding the impact and significance of this work is the ability to run it effectively on a physical robot in real time since no hardware experiments were provided. Evidence in this benchmarking is sufficient to convince me that it would be possible, and further discussion regarding the execution and localization limitations would help inform extensions of this work.\n\nThere is a missing parenthesis in equation 7.", "was_revised": true, "final_justification": "This method fills a clear gap in current literature as the first method for active semantic mapping with neural radiance fields. The authors present a novel and clear approach. The literature review is thorough and educational across the different axes of research that come together to make the contribution in this work. The experiments are thorough and show a clear advantage over the baselines. The baselines are well selected and the differences between them and the new approach is clearly explained. The tables and figures are easy to understand.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and encouraging review. We truly appreciate your recognition of the **novelty and clarity of our method**, the **thoroughness of our literature review and experiments**, and the **clear explanation and design of our tables and figures**. We are also grateful that you found the paper **enjoyable to read** and acknowledged its **potential impact and relevance** to the research community.\nWe respond to the suggestions and concerns as following:\n\n```\nQ3.1 Semantic vetor and Entropy \n```\nEach Gaussian stores only a cropped semantic vector for memory efficiency, but entropy is computed using the full vector during rendering (see supplement Line 20, Sparse Rendering). In practice, low-probability categories contribute negligibly to the entropy since $plog(p)$ approaches zero, so the approximation remains accurate.\n\n```\nQ3.2 No real-world hardware experiments \n```\nDeploying and evaluating based on physical robots is challenging due to the lack of well-annotated data, especially to get the grounth-truth semantic label for evaluation. For this reason, prior methods are evaluated only in simulation, making photorealistic environments the most practical and fair testbed for benchmarking. We will consider to extend our work into real-world application in the future.\n\n```\nQ3.3 lambda values \n```\nYes, the λ values in Equation 5 were selected through empirical tuning, but the method is not highly sensitive to these choices. As long as the two semantic loss terms are balanced in terms of numerical magnitude, the pipeline remains stable and performs well.\n\n```\nQ3.4 Support Open-Vocabulary Model\n```\n\nWe consider this an exciting direction for future work. Incorporating open-vocabulary semantic features would require reliable uncertainty quantification to ensure robust integration into our framework."}, {"review_id": "800AjEMl65", "reviewer": "Reviewer_f7pY", "summary": "The paper introduces ActiveSGM (Active Semantic Gaussian Mapping), a novel framework for active semantic mapping. This system combines semantic and geometric uncertainty quantification, guiding robots to explore their environments efficiently by strategically selecting the most informative viewpoints. It utilizes a 3D Gaussian Splatting (3DGS) backbone to generate dense, high-fidelity semantic maps, enhancing exploration and scene understanding. The framework has been tested on the Replica and Matterport3D datasets, showcasing superior performance in improving mapping completeness, accuracy, and robustness to noisy semantic data.", "questions": "See Weakness", "limitations": "See Weakness", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. The combination of active exploration with semantic and geometric uncertainty provides a meaningful advancement in robotic mapping. \n2. The paper demonstrates its effectiveness through comprehensive experiments on popular datasets like Replica and MP3D. \n3. \nWeakness\n1. The use of pseudo labels from the pre-trained OneFormer model for semantic predictions introduces a dependency on external, potentially error-prone models. This could lead to inaccuracies, especially in complex or occluded scenes where the pretrained model might perform suboptimally.\n2. The method may be limited by the initial semantic map’s accuracy, as semantic initialization from first observations can be unreliable, especially in occluded or unseen regions.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the feedback. We’re grateful for your recognition of the **novelty and value** of combining active exploration with semantic and geometric uncertainty, and we appreciate your acknowledgment of our **strong performance** across comprehensive experiments on the Replica and MP3D datasets.\n\n```\nQ4.1 Dependency on semantic predictions\n```\n\nYes, relying on pseudo labels from a pre-trained OneFormer model can introduce errors, particularly in complex or occluded scenes. However, this reflects **a more realistic setting**, as ground-truth semantic masks are never available in real-world applications. ActiveSGM is explicitly designed to handle such noisy inputs. \nFirst, our use of **a soft semantic representation** allows the model to *retain uncertainty rather than commit to potentially incorrect labels*. \nSecond, the **semantic exploration criterion** actively guides the agent to *revisit uncertain or ambiguous regions*, effectively reducing semantic noise through repeated observations. \nAs shown in our experiments, this approach enables ActiveSGM to **refine the initial pseudo labels over time**, resulting in robust semantic reconstruction despite the inherent limitations of pre-trained segmentation models."}]

Author rebuttal:

Meta-review: The recommendations from the reviewers are: BR (fEfZ), BA (7urF), A (MRuN), A (f7pY)

Despite reminders, f7pY provided a superficial review and did not participate to the discussion. We had thus to discard this review.

The remaining concerns by fEfZ and 7urF is the similarity with ActiveGAMER.  However, the committee believes the authors answered correctly when discussing the introduction of the semantic information.
fEfZ is also (slightly) concerned by the lack of real world experiments.

MRuN is much more convinced about the contribution made by the paper and has only minor concerns.  Since the authors answered correctly about the concern on the similarity with ActiveGAMER in the view of the committee, the committee decided to propose to accept this submission.
Decision: Accept (poster)