DG-SLAM: Robust Dynamic Gaussian Splatting SLAM with Hybrid Pose Optimization
Authors: Yueming Xu
Haochen Jiang
Zhongyang Xiao
Jianfeng Feng
Li Zhang
Venue: NeurIPS 2024 poster (2024)
Keywords: SLAM
Deep learning method
Gaussian splatting
Hybrid pose estimation
Dynamic environment
Primary area: robotics

Abstract: Achieving robust and precise pose estimation in dynamic scenes is a significant research challenge in Visual Simultaneous Localization and Mapping (SLAM). Recent advancements integrating Gaussian Splatting into SLAM systems have proven effective in creating high-quality renderings using explicit 3D Gaussian models, significantly improving environmental reconstruction fidelity. However, these approaches depend on a static environment assumption and face challenges in dynamic environments due to inconsistent observations of geometry and photometry. To address this problem, we propose DG-SLAM, the first robust dynamic visual SLAM system grounded in 3D Gaussians, which provides precise camera pose estimation alongside high-fidelity reconstructions. Specifically, we propose effective strategies, including motion mask generation, adaptive Gaussian point management, and a hybrid camera tracking algorithm to improve the accuracy and robustness of pose estimation. Extensive experiments demonstrate that DG-SLAM delivers state-of-the-art performance in camera pose estimation, map reconstruction, and novel-view synthesis in dynamic scenes, outperforming existing methods meanwhile preserving real-time rendering ability.

DG-SLAM: Robust Dynamic Gaussian Splatting SLAM with Hybrid Pose Optimization
Yueming Xul
Haochen Jiang'
Zhongyang Xiao²
Jianfeng Feng'
Li Zhang
Fudan University
Autonomous Driving Division, NIO
https://githul. com www.foaanoooooooooooooop..aa.a www.ddaanngg/-SLAAAAAA
Abstract
Achieving robust and precise pose estimation in dynamic scenes is significant research challenge in Visual Simultancous Localization and Mapping (SLAM). Recent advancements integrating Gaussian Splatting into SLAM systems have proven effective in creating high-quality renderings using explicit 3D Gaussian models, significantly improving environmental reconstruction fidelity. However, these approaches depend static environment assumption and face challenges in dynamic environments due to inconsistent observations of geometry and pho- tometry. To address this problem, we propose DG-SLAM, the first robust dynamic visual SL.AM system grounded in 3D Gaussians, which provides precise camera pose estimation alongside high-fidelity reconstructions, Specifically, we propose effective strategies, including motion mask generation. adaptive Gaussian point management, and a hybrid camera tracking algorithm to improve the accuracy and robustness of pose estimation. Extensive experiments demonstrate that DG-SLAM delivers sfate-of-the-a performance in camera pose estimation, map reconstrue- tion, and novel-view synthesis in dynamic scenes, outperforming existing methods meanwhile preserving real-time rendering ability.
Introduction
Visual simultaneous localization and mapping (SLAM), the task reconstructing 3D map within an unknown environment while simultaneously estimating the camera pose, is recognized as a critical component achieving autonomous navigation in novel 3D environments for mobile robots [1]. It has been widely used in various forms in fields such as robotics, autonomous driving, and aug- mented/virtual reality (AR/VR). However, the majority previous research (2, 4, 6, 7, 8, 9, 10] typically relies on the assumption of static environments, limiting the practical applicability of this technology in daily life. Consequently, how to achieve accurate and robust pose estimation dynamic scenes remains urgent problem to be addressed for mobile robots.
Recently, many researchers [6, 7, 8, 10] have endeavored replace the conventional explicit representations used in visual SLAM, such as Signed Distance Function (SDF), voxel grids [11], meshes [12], and surfel clouds [13]. with the neural radiance field (NeRF) [14] approach for recon- structing the neural implicit map. This novel map representation is more continuous, efficient, and able to be optimized with differentiable rendering, which has the potential benefit applications like navigation and reconstruction. However, these methods exhibit two primary issues: the scene's 's bounds are required be predefined initialize the neural voxel grid; and the implicit representation proves challenging for information fusion and editing. To address these problems, recent works
YYueming Xu and Haochen Jiang contribute equally to this work. 'Li Zhang (lizhangfd fudan.edu.cn) is the corresponding author with School of Data Science, Fudan University.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

like GS-SLAM [15], SplaTam [16]. and Gaussian splatting SLAM [17] leverage the 3D-GS [18] to explicit represent the scene's map. This explicit geometric representation is also smooth, continuous, and differentiable. Moreover, a substantial array of Gaussians can be rendered with high efficiency through splatting rasterization techniques, achieving up to 300 frames per second (FPS) at a resolution of 1080p. However, all these above-mentione neural SLAM methods do not perform well in dynamic scenes. The robustness of these systems significantly decreases, even leading to tracking failures, when dynamic objects appear in the environment.
To tackle these problems, we propose a novel 3D Gaussian- based visual SLAM that can reliably track camera motion in dynamic indoor environments. Due to the capability of 3D-GS to accomplish high-quality rendering in real- teml-time, the SLAM system more readily converges during pose optimization, thereby achieving better and more stable pose optimization results. A cornerstone of our approach to achieving robust pose estimation lies in the innovative motion mask generation algorithm. This algorithm filters out sampled pixels situated within invalid zones, thereby refining the estimation process. addition to the constraint of depth residual, we employ spatio- temporal consistency strategy within an observation window generate depth warp masks. By incrementally fusing the depth warp mask and semantic mask, the motion mask will become more precise to reflect the true motion state of objects. To improve the accuracy and stability of pose estimation, we leverage DROID -LAM [19] odometry (DROID- VO) to provide an initial pose estimate and devise a coarse-to fine optimization algorithm built upon the initially estimated camera pose. This aims to minimize the disparity between pose estimation and the reconstructed map, employing photorealistic alignment optimization through Gaussian Splatting, Moreover, this hybrid pose optimization approach effectively ensures the accuracy and quality of the generated depth warp mask, thereby facilitating better performance in the next camera tracking stage. To obtain high-quality rendering results, we propose a novel adaptive Gaussian point addition and pruning method to keep the geometry clean and enable accurate and robust camera tracking. Capitalizing on the factor graph structure inherent in DROID-SLAN our system is capable of executing dense Bundle Adjustment (DBA) upon completion of tracking eliminate accumulated errors.
global optimum
(i) To the best of our knowledge, this is the first robust dynamic Gaussian splatting SLAM with hybrid pose optimization, capable of achieving both real-time rendering and high-fidelity reconstruction performance. (ii) To mitigate the impact of dynamic objects during pose estimation, we propose advanced motion mask generation strategy that integrates spatio-temporal consistent depth masks with semantic priors, thereby significantly enhancing the precision of motion object segmentation. (iii) We design hybrid camera tracking strategy utilizing the coarse-to-fine pose optimization algorithm to improve the consistency and accuracy between the estimated pose and the reconstructed map. (iv) To better manage and expand the Gaussian map, we propose an adaptive Gaussian point addition and pruning strategy. It ensures geometric integrity and facilitates accurate camera tracking. (v) Extensive evaluations on two challenging dynamic datasets and one common static dataset demonstrate the the performance of our proposed SLAM system, particularly in real-world scenarios.
In summary, our contributions are summarized as follows:
2
Related work
Visual SLAM with dynamic objects filter. Dynamic object filtering is pivotal for reconstructing static scenes and bolstering the robustness of pose estimation. Existing approaches fall into two main categories: the first relies on re-sampling and residual optimization strategies to remove outliers, as seen in works such as ORB-SLAM2 [2], ORB-SLAM3 [3], and Refusion [5]. These methods, however, are generally limited to addressing small-scale motions and often falter in the face of extensive, continuous object movements. The second group employs the additional prior knowledge, for example, semantic segmentation or object detection prior [20, 21, 22, 23, 24, 25, 26], to remove the dynamic objects. However, all these methods often exhibit a domain gap when applied in real- world environments, leading to the introduction of prediction errors. More recently, deep neural networks have been employed to build end-to-end visual odometry, which performs better in specific environments such as RRAID-SLAM [19], Dytan VO [27], and DeFMowSLAM [28]. However, these methods still require a substantial amount of training data and are unable to reconstruct the high-fidelity static map.
RGB-D SLAM with neural implicit representation. Neural implicit scene representations, also known neural fields [29], have attracted considerable attention in the field of RGB-D SLAM for

---

Gunrraan
Mapplng
Reshare Idociin
Adptin paba
Inikallssiaa cslematie (Coone stag))
EGG.0
Tracking eptinization
DOX Oper
Firal
MRAI-m-R mi
Figure Overview of DG-SLAM. Given series of RGB-D frames, we reconstruct the static high-fidelity 3D Gaussian map and optimize the camera pose represented with lie algebra &
their impressive expressiveness and low memory footprint. Initial studies, including iMap [6] and DI-Fusion [30], explored the utilization of a single MI.P and feature grid to encode scene geometries within a latent space. However, they both share critical issue: the problem of network forgetting, which is catastrophic for long-term localization and mapping. In response to this limitation, NICE- SLAM [7] introduces a hierarchical feature grid approach to enhance scene representation fidelity and implements a localized feature updating strategy to address the issue of network forgetting. While these advancements contribute to improved accuracy, they necessitate greater memory consumption and may impact the system's 's ability to operate in real-time. More recently, existing methods like Vox-Fusion [10], Co-SLAM [8], and ESLAM [9] explore sparse encoding tri-plane representation strategy to improve the quality of scene reconstruction and the system's 's execution efficiency. Point- SLAM [31] draws inspiration from the concept of Point-NeRF [32], utilizing neural points to encode spatial geometry and color features. It employs an explicit method to represent spatial maps, effectively improving the accuracy of localization and mapping. All these methods have demonstrated impressive results based on the strong assumptions of static scene conditions. The robustness of these systems significantly decreases when dynamic objects appear in the environment. Recently, Rodyn-SLAM [33] proposed utilizing optical flow and semantic segmentation prior to filter out dynamic objects, and employing neural rendering method as the frontend. However, this approach is computationally intensive and limits the maximum accuracy achievable in pose estimation.
3D Gaussian splatting SLAM methods. Compared to the aforementioned NeRF-based SI.AM methods, 3D Gaussian splatting (3D-GS) [18] has garnered widespread interest among researchers due to its advantages in high-fidelity and real-time rendering. Unlike previous implicit map represen- tation methods, 3D-GS explicitly models scene maps by independent Gaussian spheres, naturally endowing it with geometric structure property. Some researchers [17, 15, 34, 16, 35] are exploring the replacement of implicit representations (NeRF) with 3D-GS in the mapping thread. However, these methods are currently constrained by the assumption of static environments, rendering them ineffective in dynamic scenes. It significantly restricts the practical application of Gaussian SLAM systems in rife-life scenarios. Under the premise of ensuring time rendering, our system is designed to improve the accuracy and robustness of pose estimation under dynamic environments.
reconstruction and real-
3 Approach
Given a sequence of RGB-D frames (4,, D.), E RR, E R, our method (Fig. 1) aims to simultaneously recover camera poses ( & E SE(3) and reconstruct the static 3D scene map represented by 3D Gaussian sphere in dynamic environments. Similar to most modern SLAM systems [36, 37], our system comprises two distinct processes: the tracking process as the frontend and the mapping process the backend.
3

---

3.1 3D Gaussian map representation
To obtain real-time rendering and high- fideliiylity reconstruction mapping, we represent the scene as set of 3d Gaussian ellipsoid G, which simultaneously possesses geometric and appearance properties.
g {9: : (1i, Σ,, 0z, h;) Vg, E E
(I)
Each 3D Gaussian ellipsoid G; is composed of its center position (4; E IR, covariance matrix Σ, E RR333, opacity og E IR, and spherical harmonics coefficients h, E RR16.
Color and depth splatting rendering. To obtain the rendering image of color and depth, we project the 3D Gaussian (mlh Ew, in world coordinate to 2D Gaussian (μz, Σir on the image plane:
(2) where R is the viewing transformation and J is the Jacobian of the affine approximation of the projective transformation. Following the alpha blending method in 3DGS [18], we accumulate the splatting Gaussian ellipsoid along the observation image pixel at the current estimation pose & to render the color and depth as:
F1 = π (T pw)) Σ, = "'R
C= EiEc C,f,(uy. Ex) I1;) fy E,)), D= Eieo d.fily, I (1- E,)), (3)
where s&(-) is the Gaussian distribution function weighted by opacity a; c; represents the color of the projected Gaussian point computed by learnable spherical harmonics coefficients h, Similarly, d denotes the depth of Gaussian point 9i, obtained by projecting to 7-axis in the camera coordinate.
Accumulated opacity. We use accumulated opacity Ô represent the rendering reliability for each pixel and judge whether the Gaussian map is well-optimized
Ô Eieg Silps, Ex) I1 fz(p1, En)),
(4)
3.2 Motion mask generation
For each input keyframe, we select its associated keyframe set within a sliding window. To reduce the computation and improve the accuracy of generating motion mask, we employ the depth warping operation solely on keyframes. To ensure the overlap between adjacent keyframes is not too small, we employ optical-flow distance to determine keyframe insertion. In regions with more intense motion, our goal is to insert as many keyframes as possible.
For the pixel in keyframe i, we reproject it onto keyframe j as follows:
Pi-jj Swarp (Sii, Pt, P.(p.)) = KT;; (K-) D. (pp))))
(5)
where K and T3 represent the intrinsic matrix and the transformation matrix between frame and frame j. respectively. phomo = (u,, 1) is the homogeneous coordinate of p;.
Given any associate keyframes D,, Dj E D, we utilize warp function fwarp to compute the residual of reprojection depth value. By setting a suitable threshold eth, we derive the depth warp mask My corresponding to dynamic objects
Mud {NPED,, I(D)(P.,)) Di(p)) < en) @ Imxn
(6)
where Ixxn represents a matrix of the same size as the image, filled with ones. The @ operation
signifies that for each element in the matrix Imx,, we assess whether its warp depth meets a specified threshold and subsequently modify the corresponding value at that position. As illustrated in Fig. 1, to derive more precise warp mask, we consider the spatial coherence of object motion within sliding window of length N and combine the multiple observation warp masks. Unlike ReFusion [5], our method can mitigate the potential impact of observation noise from single warp mask. When object motion becomes significant, we only mask the foreground pixels where the depth residual positive to avoid a large portion of pixels being masked as dynamic regions. Subsequently, we integrate the warp mask and semantic mask to derive the final motion mask M, as:
M, Mind n Mand Mydd 2 .. Mae UM,
(7)
Thanks to our innovative approach to generating motion masks, the omission of dynamic objects by semantic priors can be effectively compensated. Furthermore, by leveraging a spatial consistency strategy, the inaccuracy of edge region recognition during depth warping can be significantly reduced.

---

3.3 Coarse-to-fine camera tracking.
The constant speed motion model struggles to infer a reasonable initial optimized pose value in dynamic scenes, An inaccurate initial pose will lead to optimization getting trapped in local optima more easily and can affect the generation quality of the depth warp mask. To achieve more precise pose estimation in dynamic environments, we utilize the visual odometry (VO) component from BROID-SLAM [19] as the coarse pose estimation results camera tracking. We also conduct a dense bundle adjustment in every interaction for set of keyframes to optimize the corresponding pose G and depth d:
E d) = Eugyee Ip Te (Gú o TT (p., d.)) I.,
where Eij diag (wiy): wij represents the confidence weights. G.y denotes the relative trans- formation between the poses G; and Gj- P: means a grid of pixel coordinates. Moreover, Pi, is corrected correspondence as predicted by updated optical flow estimation. To overcome the influence of dynamic objects on bundle adjustment, we introduce suppression through the motion mask M, applied to the weighted covariance matrix. Consequently, the weighted confidence associated with dynamic objects is reduced zero.
To further improve the accuracy of pose estimation and minimize inconsistencies between the estimated pose and the reconstructed map, we implement fine camera tracking by leveraging Gaussian splatting, based on the initial obtained pose. Due to obstructions by dynamic objects and the inadequate optimization of Gaussian points in the map, the rendered image may exhibit blurring black floaters. Therefore, we employ the accumulated opacity, as outlined in 4, to denote the pixel rendering reliability. The reliable mask O for camera tracking is generated as follows:
6. ::Nuussoaee (0[pj, vy] > Ttrank) Imxa
where Rj. represents the pixel location. When the accumulated opacity at the pixel exceeds a given threshold Trrack, we consider the Gaussian point associated with that pixel to be well-optimized. Consequently, using the rendered image at these pixels for pose estimation is deemed reliable. The overall loss function finally formulated as the following minimization:
M Σl (cIG.s) MM..))MMMM..
M. M.....
1 No
Σ (DIC..))D) - 0.l.: DEN,
C: arg min
(10)
Where & denotes the camera pose requiring optimization. C' and D denote the ground truth color and depth map, respectively. M represents the number of sampled pixels in the current image. Note that only pixels with valid depth value N₂ are considered in optimization.
This hybrid pose optimization approach effectively ensures the accuracy and quality of the warp mask, thereby facilitating better performance in the next camera tracking stage. short, this hybrid pose optimization strategy enables us to achieve more precise and robust pose estimation whether in dynamic or static environments.
3.4 SLAM system
Map initialization. For the first frame, we do not conduct the tracking step and the camera pose is set to the identity matrix. To better initialization, so that can add more Gaussian points. For pixels located outside the motion mask, we sample and reproject them to the world coordinates. We initialize the Gaussian point color and center position fa; with the RGB value and reprojection coordinate of the pixel, respectively. The opacity a, is set as 0. and the scale vector S, is initialized based the mean distance of the three nearest neighbor points. The rotation matrix R; is initialized as the identity matrix.
reduce the gradient-bused dynamic radius half
Map optimization. To optimize the scene Gaussian representation, we render depth and color in independent view as seen Eq. 3, comparing with the ground truth map:
Ergs
M (c (C- -.. Läepth - - Epen. ( (D-D). M
(11)
In contrast to existing methods, we introduce the motion mask M, to remove sampled pixels within the dynamic region effectively. The final Gaussian map optimization is performed by minimizing the
5

---

Input RGB
Input Depth
WepWarp Mask
Semantic Mask
Final Mask
balloon
mov- box2
Figure 2: Qualitative results of the motion mask generation. By fusing the semantic mask and depth warp mask, the final mask will be more precise.
geometric depth loss and photometric color loss:
Emapping = Lrgb + A2Csaim + >3
(12)
where Lasim denotes the structural similarity loss between two images. Moreover, A1. Ag, Às weight factors for balance in the optimization process.
Adapative Gaussian point adding strategy. To guarantee the superior scene representation capability of 3D Gaussian, we also employ dynamic Gaussian point density, which is inspired by Point- SLAM [31]. The insertion radius is determined based on color gradient, allowing for a denser allocation of points areas with high texture while minimizing point addition in regions of low texture. This method efficiently manages memory consumption by adapting the point density according to the textural complexity of the scene.
To ensure the points we add are both necessary and effective, we adopt two wto-stage adaptive point-add strategy. For new viewpoints without previous observation, we initially perform uniform sampling across the entire image to ensure that new observed areas can be covered by Gaussian points. Moreover, if the accumulated opacity Ô calculated by Eq. 4, falls below the threshold Oth, or the depth residual between the rendered pixels and the ground truth depth is excessively large, we then add 3D Gaussian points based on these under-fitting pixels. At last, these new Gaussian points will be initialized based on the point density.
Map point deleting strategy. Given that the added 3D Gaussian points have not been subjected to geometric consistency verification and may exhibit unreasonable representation values during optimization, this could lead to the generation of a low-quality dense map or the introduction of numerous artifacts the rendering image. we implement the pruning operation as part of the Gaussian map optimization. To ensure the points we delete are both reasonable and accurate, we also adopt a two-stage point delete strategy. For all Gaussian points observed from the current viewpoint, we delete points based on three criteria: the opacity value, the maximum scale, and the ratio between the ellipsoid's 's major and minor axes, described as follow:
04 < Ta or max(S) > Tsl OR
max min
Ts2
(13)
Moreover, due potential inaccuracies along the edges of the current motion mask, adding these points could result in artifacts within the scene map. Thus, we project these points on keyframes in a small sliding window to recheck whether they can be observed by these keyframes. If the number of observations of a point is too low, we consider its addition to be insufficiently robust, and therefore, it will be removed from the current Gaussian map.
4 Experiments
Datasets. Our methodology is evaluated using three publicly available challenging datasets: TUM RGB-D dataset [38], BONN RGB-D Dynamic dataset [5] and ScanNel [39]. These three datasets contain both challenging dynamic scenes and real static scenes. This selection facilitates com- prehensive assessment of our approach under varied conditions, demonstrating its applicability and robustness in real-world indoor environments.

---

Co-SLAM [8]
ESLAM [9]
SplaTAM [16]
GS-SLAM [17]
Ours
Input CT
walk- walk-xyy xyz
walk rpy
mov- mov-box2 br-box2 ps-track
Figure 3: Visual comparison of the rendering image on the TUM and BONN datasets. Our results more complete and accurate without the dynamic object floaters.
ball ball2 ps_trk ps_trk2 mv_ mox2 Avg.
Acc./em|4
X
24.30
43.11
74.92
17.56
39.97
NICE-SLAM[7]
Comp.[cm]]
X
16.65
117.95
172.20
18.19
81.25
Comp. Ratio[s 5cm%]t
X
29.68
15.89
13.96
32.18
22.93
Acc./em|t
10.61
14.49
26.46
26.00
12.73
18.06
Co-SLAM[8]
Comp.[cm]
10.65
40.23
124.86
118.35
10,22
60.86
Comp. Ratio 5cm%]]
34.10
3.21
2.05
2.90
39.10
16.27
Acc,[cm]]
17.17
26.82
59.18
89.22
12.32
40.94
ESLAM[9]
Comp.[cm]]
9.11
13.58
145.78
186.65
10.03
73.03
Comp. Ratiols 5cm%]]
47.44
47.94
20.53
17.33
41.41
34.93
Acc.[em]
7.00
5.80
9.14
11.78
6.56
8.06
DG-SLAM((Urs))
Comp.[cm]]
9.80
8.05
17.99
20.10
7.61
15.46
Comp. Ratiol 3cm%|1
49.46
52.41
34.62
32.81
49.02
43.67
Table 1:
Reconstruction results on several dynamic scene sequences in the BONN dataset. Instances of tracking failures are denoted by "X". The most superior outcomes within the domain of RGB-D SLAMs are highlighted bold for emphasis.
Metrics. For the evaluation of pose estimation, we utilize the Root Mean Square Error (RMSE) and Standard Deviation (STD) of Absolute Trajectory Error (ATE) [38]. Prior to assessment, the estimated trajectory is aligned with the ground truth trajectory via Horn's Procrustes method. [40], ensuring coherent basis for evaluation. To evaluate the reconstruction quality of static maps in dynamic scenes, we employ three metrics(i) Accuracy (cm), (ii) Completion (cm), and (iii) Completion Ratio (percentage of points within a 5cm threshold), following NICE-SLAM [7]. Since the BONN dataset provides only the ground truth point cloud, we randomly sampled 200,000 points from the GT point cloud and the reconstructed mesh surface to calculate these metrics.
Implementation details. We run our DG-SLAM on an RTX 3090 Ti GPU FPS on BONN datasets, which takes roughly 9GB of memory. We set the loss weight X = 0.9 X2 = 0.2 and As 0.1 train our model. The number of iterations for the tracking and mapping processes has been set to 20 and 40, respectively. For the Gaussian points deleting, we set Ta 0.005, TsI 0.4 and 7s2 36 to avoid the generation of abnormal Gaussian points. What's more, we utilize Oneformer [41] to generate prior semantic segmentation. For the depth wrap mask, we set the window size to and the depth threshold to 0.6. We also adopt the keyframe selection strategy from DROID-VO [19] based on optical flow.
7

---

Method
f3/w_r f3/u_x f3/w_s
f3/s_x f2/d_p 13/1_o Avg.
ORB-SLAM3 [3]
68.7
28.1
2.0
1.0
1.5
17.1
ReFusion [5]
9.9
1.7
5.2
Co-fusion [42]
69.6
55.1
2.7
42.5
MID-fusion [4]]
6.8
2.3
6.2
5.1
EM-fusion [44]
6.6
1.4
3.7
3.9
iMAP*[6]
139.5
111.5
137.3
23.6
119.0
5.8
89.5
NICF-SIAMM[]
X
113.8
88.2
7.9
X
6.9
54.2
Vox-Fusion [0]
146.6
109.9
3.8
26.1
71.6
Co-SLAM[8]
52.1
51.8
49.5
6.0
7.6
2.4
28.3
ESLAM[9]
90.4
45.7
93.6
7.6
X
2.5
48.0
Rodyn-SLAM[33]
7.8
8.3
1.7
5.1
5.6
2.8
5.3
SplaTAM[16]
100.4
218.3
115.2
1.7
5.4
5.1
74.4
GS-SLAM[17]
33.5
37.7
8.4
2.7
8.6
15.5
DROID- VO[19]
10.0
1.7
0.7
1.1
3.7
2.3
3.3
DG-SLAM(Ours) 4.3
1.6
0.6
3.2
2.3
2.2
Table 2: Camera tracking results on several dynamic scene sequences in the TUM dataset. "x"
denotes the version reproduced by NICE-SLAM. "X" and "-" denote the tracking failures and absence of mention, respectively. The metric is Absolute Trajectory Error (ATE) and the unit is [cm].
Method
ball bal12
ps_tk
pa_tk2 tk2
ball_th 1_tk mv boxx Avg.
ORB-SLAM3 [3]
5.8
17.7
70.7
77.9
3.1
3,5
29.8
ReFusion [5]
17.5
25.4
28.9
46.3
30.2
17.9
27.7
[MAP*[6]
14.9
67.0
28.3
52.8
24.8
28.3
36.1
NICE-SLAM[?]
X
66.8
54.9
45.3
21.2
31.9
44.1
Vox- Fox-Fusion[10) 10]
65.7
82.1
128.6
162.2
43.9
47.5
88.4
Co-SLAM[S]
28.8
20.6
61.0
59.1
38.3
70.0
ESLAM[9]
22.6
36.2
48.0
51.4
12.4
17.7
31.4
Rodyn-SLAM[33]
7.9
11.5
14.5
13.8
13.3
12.6
12.3
SplaTAM[16]
35.5
36.1
149.7
91.2
12.5
19.0
57.4
GS-SLAM[17]
37.5
26.8
46.8
50.4
31.9
4.8
33.1
DROID- VOL1
5.4
4.6
21.4
46.0
8.9
5.9
15.4
DG-SLAM(OOrs)
3.7
4.1
4.5
6.9
10.0
3.5
5.5
Table 3: Camera tracking results on several dynamic scene sequences in the BONN dataset. "*" denotes the version reproduced by NICE-SLAM. "X" denotes the tracking failures. The metric is ATE and the unit is [cm].
4.1 Evaluation of generating motion mask
We evaluated our method the balloon and move_ne box2 sequences of the BONN dataset to show the qualitative results of the generated motion mask. In these two sequences, in addition to typical motion of pedestrians, there are movements of atypical objects accompanying the human, such as balloons and boxes. It might be missed if we rely solely on semantic prior. As shown in Fig. 2, generated methods notably improve the precision of motion mask segmentation, effectively reducing the inconsistencies along edge regions and detecting the true dynamic objects.
4.2 Evaluation of mapping performance
To more effectively showcase the performance of our system in dynamic environments, we evaluate reconstruction results from both qualitative and quantitative perspectives. Given dynamic scene datasets seldom offer static GT mesh point cloud, we utilize the BONN dataset for our quantitative analytical experiments. We compare our DG-SLAM method against current state-of-the-an neural- based SLAM methods, all of which are available open-source projects.

---

As shown in Tab. 1, our method significantly surpasses contemporary approaches in terms of accuracy. completion, and completion ratio metrics, achieving state-of-the-art performance. Meanwhile, the reconstructed static map can be rendered with high fidelity, as shown in Fig. 3. This also indirectly demonstrates that our methods can generate more accurate static map compared with other mainstream SLAM systems.
4.3 Evaluation of tracking performance
To comprehensively assess the tracking performance of our DG-SLAM approach, we conduct comparative analyses within highly dynamic, slightly dynamic, and static environments. The compar- ison methods encompass classical SLAM methods such as ORB-SLAM3 [3], ReFusion [S].MID- fusion [43], and EM-fusion [44], and widely recognized NeRF-based SLAM systems such as
NICE-SLAM [7]. iMap [6], Vox-Fusion [10], ESLAM [9]. Co-SLAM [8]. We also incorporate
comparison with the newly proposed dynamic neural RGB-D SLAM system, Rodyn-SLAM [33]. Fur-
thermore, our evaluation extends to the latest advancements 3D Faussian-based SLAM, including
SplaTAM [16] and GS- GS-SLAM [17].
Dynamic scenes. As shown in Tab. 2, we report the results three highly dynamic sequences, two slightly dynamic sequences, and one static sequence from the TUM RGB-D dataset. Our system exhibits exceptional tracking performance, attributed the implementation of the map point deleting strategy and the powerful coarse-to-fine to-fine camera tracking algorithm. Furthermore, the tracking ca- pabilities of our system have also been rigorously evaluated on the intricate and demanding BONN RGB-D dataset, with outcomes presented in Tab. 3. In dynamic scenarios characterized by height- ened complexity and challenge, our method has consistently demonstrated superior performance, underscoring its effectiveness and reliability in real-worle navigation applications. Meanwhile, we also showcased the number of iterations of the tracking and mapping process, taking TUM as an example, as shown in Tab. 5. Compared to other methods, our method achieved superior results while maintaining competitive iterations and efficiency.
Static scenes. To better illustrate the robustness of our system, we also evaluate our methods with existing SL.AM common real-world static sequences from ScanNet [39]. As shown in Tab. our DG-SLAM still achieves competitive performance within static scenes with fewer tracking and mapping iterations, despite our method being designed for dynamic scenes. Notably, the motion mask will become irrelevant in static scenes. Thus, it can sufficiently demonstrate the effectiveness of our proposed hybrid camera tracking strategy and adaptive Gaussian point management strategy.
Iterations
tracking mapping
Method
00
59
106
169
181
207
Avg.
NICE-SLAM [7]
200
60
NICE-SLAM [7]
12.0
14.0
7.9
10.9
13.4
6.2
10.7
Co-SLAM [8]
20
Co-SLAM [8]
7.1
11.1
9.4
5.9
11.8
7.1
8.8
ESLAM [9]
200
60
Point-SLAM [31]
10.2
7.8
8.7
22.2
14.8
9.5
12.2
Point-SLAM [31]
200
150
SplaTAM [16]
12.8
10.1
17.7
12.1
11.1
7.5
11.9
SplaTAM [16]
200
30
GS-SLAM [17]
13.6
7.6
8.1
13.7
34.6
12.5
15.1
GS-SLAM [17]
100
DG-SLAM(OUrs)
7.9
11.5
8.0
8.3
7.3
8.2
8.6
DG-SLAM(OUrs)
20
40
Table 4: Camera tracking results on ScanNet. The
Table 5: Running iterations on TUM.
metric is ATE and the unit is [cm].
4.4 Ablation study
To evaluate the efficacy of the proposed algorithm in our system, we conducted ablation studies across seven representative sequences from the BONN dataset. Given that the semantic priors within the TUM dataset have covered major motion categories (humans) while motion categories in BONN contained undefined dynamic objects such as balloons and boxes, thus we opted to perform ablation studies on the BONN dataset. We calculated the average ATE and STD metrics illustrate the impact of various components on the overall system performance. As the results presented in Tab. 6, the findings affirm the effectiveness of all proposed methods improving camera tracking capabilities. Specifically, the strategies for adding and pruning points are equally crucial, significantly impacting the quality of the Gaussian map reconstruction and, in turn, affecting the tracking performance. The depth warp operation effectively removes floaters from multiple viewpoints, thereby noticcably

---

enhancing the quality of the rendered images. One of the main contributions comes from the hybrid camera tracking strategy, which indirectly underscores the importance of eliminating inconsistencies between pose estimation and map reconstruction.
4.5 Time consumption analysis
As shown in Tab. 7, we report time consumption (per frame) of the tracking and mapping without computing semantic segmentation. These results were achieved through an identical experimental setup that involved conducting 20 iterations for tracking and 40 iterations for mapping. all processed on an RTX 3090Ti GPU. Benefiting from the rapid execution speed of Droid VO and fast rendering of 3D Gaussian Splatting, our method exhibits a leading edge in terms of time consumption during the tracking process. Although our approach does not match the mapping speed of Co-SLAM, it achieves high-quality mapping and high-fidelity rendering with competitive mapping running time. We also evaluate the inference time of our used semantic segmentation network, which required 163ms for every frame. It should be noted that our approach does not focus on the specific semantic segmentation network used, but rather on the fusion method itself.
ATE/em]] STD[em]
Tracking Mapping Avg. Running [ms]]
[ms]]]
[ms] +
w/o Add
6.63
3,08
w/o Prune
6.89
3.16
NICE-SLAM [7]
3186.2
1705.1
4892.3
w/o Dep. Warp
6.40
3.22
ESLAM [9]
2045.9
1641.4
3688.5
w/o Seg
15.27
7.47
Point-SLAN [31]
2279.5
1544.4
3823.9
w/o Fine Tracking
7.36
3.62
Co-SLAM [8]
101.4
140.1
241.6
DG-SLAM(OUrs)
5.51
2.79
DG-SLAMMOOrs)
89.2
549.3
645.9
Table 6: Ablation study on BONN dataset.
Table 7: Run-time comparison on TUM f3/w_s,
5
Conclusion
In this paper, we have presented DG-SLAM, robust dynamic Gaussian splatting SLAM with hybrid pose optimization under dynamic environments. Via motion mask filter strategy and coarse-to-fine camera tracking algorithm, our system significantly advances the accuracy and robustness of pose estimation within dynamic scenes. The proposed adaptive 3D Gaussians adding and pruning strategy effectively improves the quality of reconstructed maps and rendering images. We demonstrate its effectiveness achieving state-of-the-ar results in camera pose estimation, scene reconstruction, and novel-view synthesis in dynamic environments. While the tracking and reconstruction of large- scale scenes is currently the biggest limitation of our system, we believe will be addressed by a more flexible loop-closure optimization algorithm in future work. Moreover, the accuracy of pose estimation of our system is still influenced by the segmentation precision of the semantic prior. Therefore, efficiently perceiving moving objects within the dynamic scenes remains unresolved issue that warrants further exploration.
Acknowledgement
This work is supported by National Key R&D Program of China (No.2019YFA07095002 No.2018YFCC111294 National Natural Science Foundation of China (Grant No.621 06050 and 62376060), Natural Science Foundation of Shanghai (Grant 00.227R14075001 ZJ Lab, Shanghai Center for Brain Science and Brain-Inspired Technology, and NIO University Programme (NIO UP).
References
[1] John J Leonard and Hugh F Durrant-Whyte. Simultancous map building and localization for an autonomous mobile robot. In IROS, 1991.
[2] Raul Mur-Artal and Juan D Tardós. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE TRO, 2017. 1.2
10

---

[3] Carlos Campos, Richard Elvira, Juan J Gómez Rodriguez, José MM Montiel, and Juan D Tardós. Orb-slam3: An accurate open-source library for visual, IEEE TRO, 2021. 2, 8,
and multimap slam.
[4] Haochen Jiang, Rui Qian, Liang Du, Jian Pu, and Jianfeng Feng. Ul-slam: A universal monocular base slam via unifying structural and non-structural constraints. IEEE TASE, 2024.
[5] Emanuele Palazzolo, Jens Behley, Philipp Lottes, Philippe Giguere, and Cyrill Stachniss. Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In IROS, 2019. 1, 2, 4, 8,
[6] Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew Davison. iMAP: Implicit mapping and positioning in real-time. In ICCV, 2021. 1, 8,
[7] Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. In CVPR,
2022. 1, 3, 7, 8, 9, 10
[8] Hengyi Wang, Jingwen Wang, and Lourdes Agapito. Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam. In CVPR, 2023. 1, 7, 8, 10
[9] Mohammad Mahdi Johari, Camilla Carta, and François Fleuret. Eslam: Efficient dense slam system based on hybrid representation of signed distance fields. In CVPR, 2023. 1, 7, 8, 9, 10
[10] Xingrui Yang, Hai Li, Hongjaa Zhai, Yuhang Ming, Yuqian Liu, and Guofeng Zhang. Vox- fusion: Dense tracking and mapping with voxel-based neural implicit representation. In ISMAR. 2022. 3, 8,
[11] Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. Kinectfusion: Real-time dense surface mapping and tracking. In ISMAR, 2011. 1
[12] Thomas Schöps, Torsten Sattler, and Mare Pollefeys. Surfelmeshing: Online surfel-based mesh reconstruction. JEEE TPAMI, 2019.
[13] Thomas Schops, Torsten Sattler, and Mare Pollefeys. Bad slam: Bundle adjusted direct rgb-d slam. In CVPR, 2019.
[14] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun Acm, 2021.
[15] Chi Yan, Delin Qu, Dong Wang, Dan Xu, Zhigang Wang, Bin Zhao, and Xuelong Li. Gs-slam: Dense visual slam with 3d gaussian splatting. CVPR, 2024. 2.3
[16] Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam. In CVPR, 2024. 2, 3, 7, 8,
[17] Hidenobu Matsuki, Riku Murai, Paul H. J. Kelly, and Andrew J. Davison. Gaussian Splatting SLAM. In CVPR, 2024. 2, 3, 8,
[18] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023. 2, 3, 4
[19] Zachary Teed and Jia Deng. RGB-D Cameras. In NeurlPS, 2021. 2, 5. 7,8
Deep Visual SLAM for Monocular, Stereo, and
[20] Chao Yu, Zuxin Liu, Xin-Jun Liu, Fugui Xie, Yi Yang, Qi Wei, and Qiao Fei. Ds-slam: A semantic visual slam towards dynamic environments, IROS, 2018. 2
[21] Berta Bescos, José Fácil, Javier Civera, and José Neira. Dynaslam: Tracking, mapping, and inpainting in dynamic scenes. IEEE RAL, 2018. 2

---

[22] Linhui Xiao, Jinge Wang, Xiaosong Qiu, Zheng Rong, and Xudong Zou. Dynamic-slami Semantic monocular visual localization and mapping based on deep learning dynamic environment. IEFE RAS, 2019. 2
[23] Berta Bescos, Carlos Campos, Juan D Tardós, and José Neira. Dynaslam ii: Tightly-coupled multi-object tracking and slam. IEEE RAL, 2021. 2
[24] Jun Zhang, Mina Henein, Robert Mahony, and Viorela lla. Vdo-slam: visual dynamic object-aware slam system. arXiv preprint, 2020. 2
[25] João Carlos Virgolino Soares, Marcelo Gattass, and Marco Antonio Meggiolaro. Crowd-slam: visual slam towards crowded environments using object detection. Journal of Intelligent & Robotic Systems, 2021. 2
[26] Mingrui Li, Jiaming He, Guangan Jiang, and Hongyu Wang. Ddn-slam: Real-time dense dynamic neural implicit slam with joint semantic encoding. arXiv preprin, 2024. 2
[27] Shihao Shen, Yilin Cai, Wenshan Wang, and Sebastian Scherer. Dytanvo: Joint refinement of visual odometry and motion segmentation in dynamic environments. In ICRA, 2023. 2
[28] Weicai Ye, Xingyuan Yu, Xinyue Lan, Yuhang Ming, Jinyu Li, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. Deflowslam: Self-supervised scene motion decomposition for dynamic dense slam, arXiv preprint, 2022.
[29] Ben Mildenhall, Pratul P Srinivasan. Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun ACM, 2021. 2
[30] Jiahui Huang, Shi-Sheng Huang, Haoxuan Song, and Shi-Min Hu. Di-fusion: Online implicit 3d reconstruction with deep priors. In CVPR, 2021.
[31] Erik Sandström, Yue Li, Luc Van Gool, and Martin R. Oswald, Point-slam: Dense neural point cloud-based slam. In ICCV, 2023. 3, 6, 9, 10
[32] Qiangeng Xu, Zexiang Xu, Julien Philip, Sai Bi, Zhixin Shu, Kalyan Sunkavalli, and Ulrich Neumann. Point-nerf: Point-based neural radiance fields. In CVPR, 2022. 3
[33] Haochen Jiang, Yueming Xu, Kejie Li, Jianfeng Feng, and Li Zhang. Rodyn-slam: Robust dynamic dense rgb-d slam with neural radiance fields. RAL, 2024. 3, 8,9
[34] Vladimir Yugay, Yue Li, Theo Gevers, and Martin R Oswald. Gaussian-slam: Photo-realistic dense slam with gaussian splatting. arXiv preprint, 2023. 3
[35] Tianchen Deng, Yaohui Chen, Leyan Zhang, Jianfei Yang, Shenghai Yuan, Danwei Wang, and Weidong Chen. Compact 3d gaussian splatting for dense visual slam. arXiv preprint, 2024. 3
[36] Georg Klein and David Murray. Parallel tracking and mapping for small ar workspaces. In ISMAR, 2007.
[37] Richard A. Newcombe, Steven J. Lovegrove, and Andrew J. Davison. Dtam: Dense tracking and mapping in real-time. In ICCV, 2011.
[38] Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation of rgb-d slam systems. In IROS, 2012. 6, 7
[39] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 6,9
[40] Berthold KP Horn. Closed-form solution of absolute orientation using unit quaternions. Josa a.
1987. 7
[41] Jitesh Jain, Jiachen Li, Mang Chiu, Ali Hassani, Nikita Orloy, and Humphrey Shi. One- Former: One Transformer to Rule Universal Image Segmentation. In CVPR, 2023. 7
12

---

[42] Martin Rünz and Lourdes Agapito. Co-fusion: Real-time segmentation, tracking and fusion of multiple objects. In ICRA, 2017.
[43] Binbin Xu, Wenbin Li, Dimos Tzoumanikas, Michael Bloesch, Andrew Davison, and Stefan Leutenegger. Mid-fusion: Octree-based object-level multi-instance dynamic slam. In ICRA, 2019. 8,
[44] Michael Strecke and Jorg Stuckler. Em-fusion: Dynamic object-level slam with probabilistic data association. In ICCV, 2019. 8,
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have clearly stated the contribution and scope of the paper in the abstract.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of the work the conclusion part.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We have provided the theoretical assumption and proof of this paper in the method part.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have disclosed all the information needed to reproduce the main experi- mental results of the paper in the experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We have not released code this paper in the submission.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. ca/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ttpp.cccppbblll/ http:ccccpppccccccccceeccccccccc htppuuliccguuieeesseeeeppp guides/ /CoeeSubmdbibiii ssi ssisssoolooooooooo oonooiiyyyylolooooiomoooomiiiilp for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have specified all the training and test details of this paper in the experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have reported appropriate information about the statistical significance of the experiments in the experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided sufficient information on the computer resources of this method in the experiment part.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: / //t:////nnee /tt.riss....peeee//eee/ee cc cc/ppblicc/Bchhccsuulldeeee
Answer: [Yes]
Justification:
We have conducted research under the NeurIPS Code Of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the societal impacts of this paper in the introduction and method parts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper has not released data models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All the assets used in the paper are properly licensed. Our paper cites the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets, including data and models. We also did not publish or submit the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research topic of this paper is robotic SLAM, not involving crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research topic of this paper is robotic SLAM, not involving crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "2GUIt2KsCB", "reviewer": "Reviewer_UDnM", "summary": "The paper combines existing methods for deep SLAM (DROID-SLAM) and 3D Gaussian splatting for 3D mapping into a system and adds an approach for dynamics filtering by depth warping to make tracking more robust in dynamic scenes. The approach is evaluated on several dynamic SLAM benchmarks and compared with recent deep learning based RGB-D SLAM approaches.", "questions": "See questions in paper weaknesses.", "limitations": "The conclusion provides an obvious limitation of the approach due to not considering loop closing. Further limitations and assumptions are not discussed.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The proposed approach of dynamics filtering for SLAM with 3D Gaussian splatting for map representation seems novel. \n- Experiments demonstrate improvements over previous methods for neural RGB-D SLAM which have been designed for static scenes.", "weaknesses": "- Unfortunately, this paper has significant shortcomings in writing quality and clarity. Many variables and concepts are not defined properly (for instance, J, R in eq 2., \\otimes operator, I_{m\\times n}, \\Pi in eq 8, S_i in l. 192, the symbols in eq. 13). \n- Some phrases in the introduction are not accurate or proper English. What does \"finishing pose estimation robustness\" mean? What are invalid zones, what does furnishing an initial pose estimate mean?\n- Sec. 2, the first paragraph starts with the heading \"traditional visual SLAM with dynamic objects filter\", but contains recent deep SLAM approaches for static scenes. Please restructure.\n- The raycasted color and depth in eq. 3 requires a specific ordering of the gaussian splats, as indicated in the products ranging from j=1 to i-1. Please explain how it is established.\n- What does the intersection operation over pixels achieve in eq 6 and 9? What does it mean to apply an indicator function and take a \\otimes product with the I_mxn ? What is I_mxn? \n- The variable \\mu is used for coordinates and means of 3D Gaussians ambiguously, please change notations.\n- l. 179, how is the camera pose parametrized and optimized ?\n- l. 188, what is the dynamic radius?\n- l. 192, what is the scale vector S_i and why should it be computed from the mean distance of three nearest neighbor points? Where do these points come from?\n- eq 12, how are the lambda_1-3 chosen?\n- l. 235, \"unit quaternion algorithm\" is not the right name for the method but rather Horn's procrustes method.\n- l. 257, what is an \"analytical experiment\" ?\n- Table 2, 3, what are the metrics for the numbers ? ATE, RPE ? how is it computed?\n- Tables 2, and 3 should also compare with classical dynamic SLAM methods like Co-Fusion [*1], MID-Fusion [*2], EM-Fusion [*3] to name a few. Also these methods are missing from the discussion of related work.\n- what is an \"invalid Gaussian point filter\" in l. 272 ?\n- Table 5 seems to indicate that different settings of the iteration count hyperparameter are needed for the various datasets. This indicates an issue with generalization for the method. Are other hyperparameters chosen consistently across datasets?\n- l. 290 what does to \"encompass major motion categories\" mean ?\n- l. 291, what is the definition of the STD metric ?\n- Table 7 / l. 302, please include run-time for semantic segmentation as it seems essential according to Table 6.\n- How is the semantic segmentation mask obtained ? What is the runtime of this processing step?\n- The paper does not discuss limitations and assumptions of the method beyond not performing loop closing.\n\n[*1] Martin Ruenz, Lourdes Agapito. Co-fusion: Real-time segmentation, tracking and fusion of multiple objects. ICRA 2017\n\n[*2] Binbin Xu, Wenbin Li, Dimos Tzoumanikas, Michael Bloesch, Andrew J. Davison, Stefan Leutenegger. MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM. ICRA 2019\n\n[*3] Michael Strecke, Joerg Stueckler. EM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association. ICCV 2019", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer's meticulous and detailed review of our work, which helped us improve the quality of this paper. It should be noted that some of the issues you raised are typically treated as foundational definitions in the 3D Gaussian Splatting paper and traditional SLAM literature. To avoid unnecessary repetition, we have omitted some of the common expressions and explanations of symbols in our previous papers.\n\nA1: (1)Eq.2 comes from the 3DGS[1] paper. $R$ is the viewing transformation and $J$ is the Jacobian of the affine approximation of the projective transformation. (2) $I_{m\\times n}$ represents a matrix of the same size as the image, filled with ones. (3) Eq.8 is derived from DROID[2], where $p_i$ means a grid of pixel coordinates. (4) $S_i$ in l.192 and $R_i$ in l.193 are scaling vector and rotation matrix respectively. (5) In Eq.13, $\\alpha_{i}$ means the opacity, which has been defined in l.125, and $S$ means the scaling vector. Other threshold parameters can be find more details in l.244.\n\nA2: (1)\"finishing pose estimation robustness\" means achieving robust pose estimation. (2)“invalid zones” means dynamic object regions, illegal depth value regions, and regions with less rendering opacity. (3)\"furnishing an initial pose estimate\" represents that the coarse stage provides a better initial camera pose for the fine stage.\n\nA3: We will modify it to \"visual SLAM with dynamic objects filter\".\n\nA4: This has been explained in 3DGS[1]. The raycasting will sort all the Gaussian points based on their depth value. \n\nA5: For the defination of $I_{m\\times n}$, please refer to A1. The intersection operation signifies that for each element in the matrix $I_{m\\times n}$, we assess whether its warp depth meets a specified threshold and subsequently modify the corresponding value at that position.\n\nA6: The mean of  3D Gaussian points in space means the center coordinates of the Gaussian points. Thus, we use $\\mu$ to represent the coordinates and mean. Their meanings are consistent.\n\nA7: We parametrize the camera pose using quaternion and translation. To efficiently execute the pose optimization process, we have computed the Jacobian of the designed loss function to these two optimization variables and have implemented modifications in CUDA.\n\nA8: We employed the dynamic point density management strategy, which determines whether to insert new Gaussian points according to the dynamic radius. The dynamic radius is determined based on color gradient, as explained in l.202 - l.204.\n\nA9: $S_i$ refers to the scale of each Gaussian point, initialized by the mean distance of three nearest neighbor points. This operation originated from the original 3DGS[1] paper.\n\nA10: $\\lambda_1$-$\\lambda_3$ are the loss weight, determined through grid search. We show the details of these parameters in l.242.\n\nA11: Thank you for your advice. We will update this expression in the revised paper.\n\nA12: We utilized the GT point cloud provided by the Bonn to qualitatively and quantitatively evaluate the reconstruction results, where \"quantitative analytical experiments\" refer to the results in Tab.1. \n\nA13: The values presented in Tab. 2 and 3 correspond to the Absolute Trajectory Error (ATE) metrics, as mentioned in the Metrics section of the experiments (l.233). ATE quantifies the average absolute trajectory error between the estimated and ground truth poses across all measurement locations. Further details can be found in the Evo evaluation tool.\n\nA14: Thank you for your suggestion. In the experimental section, we have added and compared some classic dynamic SLAM methods: Co-Fusion[4], MID-Fusion[5], and EM-Fusion[6].  The results are shown in Tab. 1(one-page PDF). Our method has showcased superior pose estimation results in comparison to classic dynamic SLAM methods. We will also discuss these methods in related work.\n\nA15: It means “Map point deleting strategy” mentioned in l.214, Sec 3.4.\n\nA16: In Table 5, we present the number of iterations of the tracking and mapping process for different SLAM methods. The experimental results show that our method can achieve superior results with fewer iterations. What's more, together with Table 7, Table 5 also demonstrates the competitiveness of our method in time consumption.\n\nA17: We divided some default dynamic object classes based on semantic prior, such as humans. However, the semantic mask requires prior knowledge of motion categories and faces generalization challenges. Thus we use the depth warp mask to identify undefined dynamic objects such as balloons and boxes. \n\n\"encompass major motion categories\" means the dynamic object category contained in Tum is human. Therefore, we opted to perform ablation studies on BONN. It contains moving balloons and boxes, which can show the effectiveness of our depth warp mask.\n\nA18: STD means Standard Deviation of Absolute Trajectory Error. This concept has been mentioned in l.234. Further details can be found in the Evo evaluation tool.\n\nA19: The semantic segmentation process can be considered as a part of data preprocessing. Thus, it has not been included in the system runtime calculations. To address your concern, we have tested the inference time of the Oneformer[7] method we adopted on an A6000 GPU, which is 163ms for every frame. It should be noted that our approach does not focus on the specific semantic segmentation network used, but rather on the fusion method itself.\n\nA20: Thank you for your helpful advice. We will enhance the description of the experimental details in the implementation section. We utilize OneFormer[7] to generate prior semantic segmentation. We reported the run-time in A19.\n\nA21: Extending dynamic 3DGS to large-scale scenarios has always been a challenging problem. As mentioned in GS-SLAM[8], the problem of loop closure for handling large-scale scenes is an interesting direction for future research. Thus, we believe it is still a challenge worth solving in dynamic SLAM.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "begrlEKUCL", "reviewer": "Reviewer_DPxX", "summary": "The paper present DG_SLAM to address the inconsistent observation of geometry and photometry in dynamic environments among the current slam-related field.  Specifically, the authors develop several technologies such as the motion mask generation, adaptive gaussian point management, and a hybrid camera tracking algorithm to improve the performance of slam system. And the experiments on TUM RGB_D and BONN RGB_D dataset showcases the state-of-the-art performance both on static and dynamic environments.", "questions": "see weakness part", "limitations": "The method fails to be conducted on several large-scale real scenes.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tFor the task of motion object segmentation, the authors propose an advanced motion task generation method which integrates spatio-temporal consistent depth masks combined with semantic priors, and several rigorous mathematic formulations are illustrated to demonstrate the effectiveness.\n2.\tA coarse-to-fine camera pose optimization is proposed to optimize the whole slam system, and improve the constancy between pose and reconstructed 3d map.\n3.\tThe authors also present an adaptive gaussian point addition and pruning strategy to manage the generated gaussian map.", "weaknesses": "1.\tA motion mask generation strategy is used to mitigate the potential impact of observation noise from a single warp mask, can the authors give more details about this part? Are there any theoretically analysis for that?\n\n2.\tA hand-crafted rule is adapted in the map point deleting stage. For example, the authors delete points based on three baselines: the opacity value, the maximum scale, the ration between ellipsoid’s major and minor axes, can authors give more explanation about these criteria? Moreover, the authors showcase the removement of number of observations of the point that are too low, but fail to show its quantitative criteria.\n3.\tthe running time on BONN dataset during the mapping phase seems a bit large in table 7, can authors give several details about this result? Are there any connections with the proposed map point deleting strategy?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1: The details of the motion mask generation strategy.\nA1: During our experiments, we observed that inaccuracies in pose estimation, coupled with noise in the captured ground-truth depth values, can result in unstable motion segmentation outcomes when relying solely on a single warp mask. However, by integrating observations from depth warps across multiple frames, a more accurate motion mask can be constructed. This approach significantly enhances the reliability and precision of the motion segmentation process, as shown in Fig. 1 (one-page global PDF).\n\n### Q2: The design criteria of the three gaussian point deletion methods.\nA2: Thank you for your suggestions. We aim to clarify your concerns from the following perspectives: \n\n(1) **The opacity value** denotes the importance of each gaussian point along each ray during the rendering process. Our opacity verification operation is similar in spirit to the original 3D GS. Since we implement an adaptive dense control algorithm to regulate the density of the Gaussian map, we contend that the deletion of non-essential Gaussian points can facilitate the incorporation of new Gaussian points and enhance the convergence of pose optimization.\n\n(2) Through the visualization of reconstructed Gaussian maps, we have observed that the Gaussian spheres with **large scale do not effectively represent the local details of objects**; instead, they tend to cause blurring or shadowing during the rendering process. This results in highly unstable pose optimization, which in turn leads to the failure of camera tracking.\n\n(3) The third criterion for evaluation arises because we have found that the rasterization of 3DGS imposes **no constraints on the Gaussians along the viewing ray direction**, even when a depth observation is present. This does not pose a problem when a sufficient number of carefully selected viewpoints are available. However, in continuous SLAM, this lack of constraint leads to numerous artifacts, which complicates tracking. Consequently, it becomes necessary to remove these Gaussian points.\n\nFor each added Gaussian point, only a sufficient number of observations can confirm that the point is not an artifact. Based on this design principle, we select half the window size as the empirical threshold for point deletion operations. If a point is observed too infrequently, we consider it not robust enough and, therefore, it needs to be removed promptly.\n\n### Q3: The details of the mapping processing running time on BONN dataset.\nA3: Your understanding is correct. The extended execution time of the mapping process on the Bonn dataset is closely linked to the point management strategy. More specifically, it is intricately associated with the extent of camera movement within the dataset.\n\nDue to the relatively large movements of the camera in the Bonn dataset, an increased number of Gaussian points are inserted during the mapping process to maintain tracking stability. Consequently, this leads to a noticeable augmentation in processing time during the mapping phase, as shown in Tab.2 (one-page global PDF). Even when addressing complex pose estimation issues in dynamic scenes, the runtime of our mapping process is approximately the same as that of other GS-based SLAM methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7h6t0UAuCx", "reviewer": "Reviewer_MuLC", "summary": "The paper introduces the first robust dynamic visual SLAM system based on 3D Gaussian Splatting. It provides precise camera pose estimation and high-fidelity reconstructions. Strategies such as motion mask generation, adaptive Gaussian point management, and hybrid camera tracking are proposed to improve the accuracy and robustness of pose estimation.", "questions": "1. According to Table 6, the segmentation mask seems to play a much more significant role than other components, such as the depth warp mask. Please discuss the utility of the other components in detail. A concern here is that a more accurate segmentation mask (e.g. produced by SAM) might facilitate even better dynamic removal without the need for other components.\n\n2. In the implementation detail, the author should elaborate more on how to perform semantic segmentation, the parameter of depth warping, and the keyframe selection strategy they have integrated in the proposed system.\n\n3. The dynamic features are removed in the tracking system; however, a large portion of the removed features might reduce the tracking accuracy. In other words, to what extent can the method proposed in the paper tolerate motion? Is there any failure case/example of tracking, e.g. when dynamic features dominate the current view?\n\n4. The semantic mask is retrieved by utilizing a pretrained segmentation model. There is a concern regarding the generalizability of this semantic mask. How is the dynamic object defined with respect to the semantic mask? For instance, the human might be considered a dynamic object in the BONN dataset, but might be stable in other scenes (e.g. wax figure). In addition, how does this integration of the semantic model affect the real-time processing capability of the SLAM system?", "limitations": "There is no societal impact of this work, and limitations are discussed in the paper.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The writing is clear and easy to understand.\nThe paper proposed the first GS-based SLAM systems for dynamic scenes.\nCompared with previous NeRF-based and GS-based SLAM systems, it achieves optimal performance on dynamic scenes.", "weaknesses": "1. The evaluation conducted in the paper primarily compares the proposed method against NeRF or Gaussian-splatting-based SLAM systems, which is not sufficient. It would be beneficial to include traditional SLAM systems as baselines, which present a more robust tracking capability, such as ReFusion [1] and ORB-SLAM3 [2].\n\n2. Lacks the comparison with MonoGS [3] as the baseline of GS-based SLAM. MonoGS demonstrates more advanced map reconstruction ability compared with other GS-based alternatives, and presents robust camera tracking utilizing Jacobian optimization (e.g. compared with trivial pose optimization via photometric loss implemented in SplaTAM). It should be included as a baseline method for more exhausted comparisons.\n\n3. Lacks survey on the literature of related work, such as Crowd-SLAM [4] and DDN-SLAM [5], which were also proposed for the same motivation of reconstruction on the dynamic environment.\n\n\n4. The experiments on two datasets have shown the effectiveness (see point 5) of the proposed method in dynamic indoor settings. However, for a SLAM system, it would be preferable to conduct experiments on real-world scenes, especially in outdoor environments.\n\n5. The proposed system largely relies on dynamic object removal in the tracking system; however, it lacks certain optimization for the mapping system. This results in the reconstructed scenes still containing artifacts.\n\n\n[1] Palazzolo, J. Behley, P. Lottes, P. Giguere, and C. Stachniss, “Refusion: 3d reconstruction\nin dynamic environments for rgb-d cameras exploiting residuals,” in IROS, 2019.\n\n[2] C. Campos, R. Elvira, J. J. G. Rodr ́ıguez, J. M. Montiel, and J. D. Tard ́os, “Orb-slam3: An\naccurate open-source library for visual, visual–inertial, and multimap slam,” IEEE TRO, 2021.\n\n[3] Matsuki, Hidenobu, et al. \"Gaussian splatting slam.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[4] Soares, João Carlos Virgolino, Marcelo Gattass, and Marco Antonio Meggiolaro. \"Crowd-SLAM: visual SLAM towards crowded environments using object detection.\" Journal of Intelligent & Robotic Systems 102.2 (2021): 50.\n\n[5] Li, Mingrui, et al. \"Ddn-slam: Real-time dense dynamic neural implicit slam with joint semantic encoding.\" arXiv preprint arXiv:2401.01545 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### W1: More experements results.\n\nA1: Thank you for your suggestion. We have added experiments comparing ORB-SLAM3[2] and Refusion[1]. As shown in Table 4 (one-page global PDF), our method presents a more robust tracking capability.\n\n### W2: Comparing with MonoGS.\n\nA2: Thank you for your suggestion. Actually, we have already compared our method with MonoGS[3] in the initial draft. MonoGS is the name of the open-source framework and we used the name of its paper, namely GS-SLAM, as shown in Table2,3,4.  We will make modifications to this issue to prevent misunderstandings.\n\n### W3: Related work.\n\nA3: Thank you for your suggestion. We will discuss and cite Crowd-SLAM[4] and DDN-SLAM[5] in the revised version.\n\n### W4: Real-world scenes experiments.\nA4: Extending dynamic 3DGS to large-scale outdoor scenarios has always been a challenging problem. As mentioned in the conclusion part, the problem of loop closure for handling large-scale scenes is an interesting direction for future research. Therefore, we did not provide experimental results for outdoor environments.\n\nHowever, we provided experimental results for real-world scenes, as shown in Fig. 2 and Tab. 3(one-page global PDF). We conduct experiments on 3 real-world sequences and all results indicate the superiority of our method. \n\n### W5: Lack certain optimization for the mapping system.\nA5: We have designed an adaptive Gaussian point addition and pruning strategy for the mapping system. We will perform geometric consistency verification on map points through depth warp. Consequently, artifacts can be eliminated to a certain extent. Of course, the constructed Gaussian map may still contain a small number of artifacts, and we are considering how to better resolve this issue.\n\n### Q1: The role of the semantic mask. According to Table 6, the segmentation mask is much more important than other components.\nA1: In response to your observation that the segmentation mask seems to play a much more significant role than other components, we believe it relates to the dynamic object classes in the validation datasets, such as TUM and BONN. The semantic prior predominantly features dynamic object classes, such as humans, enhancing semantic segmentation performance. For non-rigid objects like humans, the results of depth warp fusion are less accurate compared to semantic segmentation, which is reasonable.\n\nFollowing your suggestion, we consider that SAM methods can indeed generate more accurate segmentation masks, provided that they receive inputs that are precise and unambiguous, such as points, boxes, or masks.\nWhile SAM methods could serve as an alternative approach to semantic segmentation, they still do not capture the true motion of objects. Therefore, a geometric consistency module is necessary for additional verification. Moreover, the inference speed of SAM methods is slower compared to current semantic segmentation approaches. Your suggestion has opened up new avenues for our research, and we are considering how best to integrate the SAM method into our SLAM system to improve the accuracy of our motion segmentation.\n\n### Q2: More implementation detail.\nA2: Thank you for your helpful advice. We will enhance the description of the experimental details in the implementation section. We utilize OneFormer to generate prior semantic segmentation. For the depth wrap mask, we set the window size to 4 and the depth threshold to 0.6. For keyframe selection, we adopt the keyframe selection strategy from DROID-VO based on optical flow.\n\n### Q3: To what extent can the method proposed in the paper tolerate motion?\nA3: As you mentioned, removing a large number of features could compromise tracking accuracy. Therefore, instead of employing a feature-based SLAM method as the front end, we utilize dense optical flow tracking from droid-SLAM for motion estimation. This choice enables our method to better tolerate dynamic environments. It can operate robustly and accurately in both low and high dynamic sequences of public datasets (TUM and BONN), as well as in real-world settings (Self-collected dataset). However, if occlusions are extensive, covering more than two-thirds of the entire image, the system is still prone to track loss. This issue remains a significant challenge for current SLAM systems and is yet to be resolved.\n\n### Q4: The definition of the dynamic object with respect to the semantic mask and the running time of the semantic model.\nA4: Thank you for your suggestions. It is important to note that semantic segmentation is just one of the methods we use to generate motion masks; we also employ geometric consistency checks with depth warping operation. For different real-world applications, we typically predefine certain dynamic categories and initially obtain motion masks through semantic segmentation. In the vast majority of daily scenarios, humans are generally considered dynamic objects. Of course, the special scenario you mentioned, involving wax figures, may indeed struggle to balance predetermined motion priors with actual movements. However, such scenarios are relatively rare compared to typical applications, and we have not yet considered these cases in our approach. Regarding the real-time aspect of semantic segmentation you mentioned, it is essential to emphasize that our method does not focus on the specific architecture of the semantic segmentation network used, but rather on the design of information fusion methods. With ongoing advancements in these research fields and improvements in computing power, the processing speeds for semantic segmentation are expected to increase, ensuring they do not become bottlenecks for our method.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UBvtWwQE6d", "reviewer": "Reviewer_fGfv", "summary": "This paper proposed a dynamic RGB-D SLAM system based on 3D Gaussian Splatting (3D-GS) and DROID-SLAM. Semantic segmentation masks and depth warping residuals are used to generate motion masks to remove the dynamic part of the scene. Experiments on three real-world datasets show the proposed approach achieves superior results on both dynamic and static scenes.", "questions": "Please see Weakness.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written, technically sound and easy to follow.\n2. The idea of using depth warping residuals as a complement to semantic priors for motion mask generation is simple but works effectively well.\n3. The proposed method is very well engineered and achieved promising results on real-world dynamic sequences.\n4. The proposed method is the first dynamic SLAM method based on 3D Gaussian Splatting (3DGS).", "weaknesses": "1. The technical novelty is a bit limited. The paper focuses on adding new components to previous SLAM methods in order to achieve dynamic SLAM.\n2. The hybrid coarse-to-fine camera tracking seems like a trick: The coarse stage directly comes from DROID-SLAM while the fine stage simply adds masks to 3DGS-based tracking.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1: Technical novelty.\nA1: Thank you for your advice. We aim to clarify your concerns from the following perspectives: (1) As dynamic 3D Gaussian-based SLAM remains in its nascent stages of research and diverges markedly from traditional approaches, this paper presents preliminary explorations in this domain. We are the first work aimed at addressing the challenge of robust pose estimation for 3D Gaussian-based SLAM in dynamic environments. We hope to design an innovative dynamic SLAM system to better utilize 3D Gaussian explicit representation. (2) We have discovered that directly employing a 3D Gaussian representation for pose optimization in dynamic environments tends to result in convergence to local optima, and the solution process is notably unstable. Thus, we need to build a relevant robust tracking front end. (3) We design the motion mask generation method to filter out the invalid optical flow correspondence in Droid-VO to improve the robustness of the original tracking process. Furthermore, we have observed that Gaussian representations, when initialized with stable values, demonstrate improved convergence behavior. (4) Based on these observations and analyses, we propose a hybrid optimization approach incorporating a coarse-to-fine pose optimization strategy. Specifically, we employ a rectified confidence matrix within the droid-slam framework to provide initial pose estimates, which are subsequently refined using a global Gaussian map. This suite of designs significantly enhances the accuracy and robustness of pose estimation for GS-based SLAM in dynamic environments.\n\n### Q2: The meaning and motivation of designing a hybrid coarse-to-fine strategy.\nA2: We sincerely appreciate the reviewer's meticulous and detailed review of our work, which helped us improve the quality of this paper. It should be noted that the coarse-to-fine approach is an optimization thought that has proven effective in enhancing the accuracy and robustness of pose estimation for complex tasks. However, how to achieve precise and robust pose optimization during the coarse and fine stages continues to be an open and challenging problem. \n\nWe have introduced a hybrid pose optimization strategy specifically tailored for Gaussian-based SLAM systems to effectively tackle pose estimation challenges in dynamic environments, which also represents a central innovative contribution of our paper. Specifically, **in the coarse-optimized stage**, we have implemented a strategy that filters out unreliable and inaccurate optical flow estimations within the existing confidence matrix component of the original droid-slam system. This strategy removes inaccurate matching correspondences, thereby providing a relatively precise initial pose estimate. **During the fine-optimized stage**, in addition to utilizing motion mask generation methods to filter out invalid sampled points, we have designed a novel point management technique to maintain the Gaussian map. This encompasses adaptive point addition and effective outlier removal strategies. These elements are critical in ensuring that our SLAM system maintains high levels of accuracy and robustness in dynamic environments.\n\nAdditionally, the coarse-to-fine pose optimization strategy **forms a compact and integral feedback mechanism within the entire SLAM system**. Improved pose optimization precision facilitates the generation of more accurate motion masks, which in turn enhance the accuracy of the initial pose estimates obtained during the coarse optimized stage and lead to better pose optimization results in the fine optimized stage. \n\nFurthermore, the keyframe selection strategy, which is predicated on optical flow motion during the coarse-optimized stage, further enhances the efficiency and accuracy of pose optimization in the fine-optimized stage. This integrated approach ensures a compact and effective optimization process throughout the whole SLAM system.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We are immensely grateful for the time and effort expended by all reviewers in reviewing our manuscript. The technical evaluations and detailed comments provided have been invaluable and have substantially enhanced the quality of our work. In this response, we have meticulously addressed each question posed by the reviewers on a point-by-point basis. Additionally, we have included the figures and tables from the supplementary experiments in the one-page PDF attachment.

The citation numbers in response to the reviewer UDnM are as follows:

[1] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023.

[2] Zachary Teed and Jia Deng. DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and
362 RGB-D Cameras. In NeurIPS, 2021.

[3] Erik Sandström, Yue Li, Luc Van Gool, and Martin R. Oswald. Point-slam: Dense neural point cloud-based slam. In ICCV, 2023.

[4] Martin Ruenz, Lourdes Agapito. Co-fusion: Real-time segmentation, tracking and fusion of multiple objects. In ICRA, 2017.

[5] Binbin Xu, Wenbin Li, Dimos Tzoumanikas, Michael Bloesch, Andrew J. Davison, Stefan Leutenegger. MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM. In ICRA, 2019.

[6] Michael Strecke, Joerg Stueckler. EM-Fusion: Dynamic Object-Level SLAM With Probabilistic Data Association. In ICCV, 2019.

[7] J. Jain, J. Li, M. Chiu, A. Hassani, N. Orlov, and H. Shi, “OneFormer: One Transformer to Rule Universal Image Segmentation,” in CVPR,
2023.

[8] Hidenobu Matsuki, Riku Murai, Paul H. J. Kelly, and Andrew J. Davison. Gaussian Splatting SLAM. In CVPR, 2024.

Meta-review: This paper proposes a DG-SLAM system in dynamic environments grounded in 3D Gaussians with fidelity reconstructions. The key parts include motion mask generation, adaptive Gaussian point management, and a hybrid camera tracking algorithm to improve the accuracy and robustness of pose estimation. Experimental results show the effectiveness of the method. 

The contribution of the paper is to tackle dynamic objects when appearing in the environment. The existing neural SLAM methods do not perform well in dynamic scenes. The robustness of these systems significantly decreases, even leading to tracking failures. This paper proposes a 3D Gaussian-based visual SLAM that can reliably track camera motion in dynamic indoor environments.

The paper received one “Reject” rating, one “Weak Accept” rating, and two “Borderline Accept” ratings. 

Reviewer UDnM giving “Reject” proposed a series of questions. The questions were addressed or discussed by the authors. In particular, the experiments were performed with classical methods Co-Fusion, MID-Fusion, EM-Fusion on accuracy in Tables 2 and 3, and performed with Co-Fusion, MID-Fusion on runtime. The authors discussed both NeRF and GS-based SLAM methods exhibit a certain gap in runtime when compared to traditional approaches. Addressing this discrepancy is a challenge that the research community continues to actively pursue.

Reviewer DPxX thinks the authors have processed all his/her concerns. Reviewer DPxX keeps the “Weak Accept”.

Reviewer MuLC thinks his/her concerns are either reasonably addressed or considered as future works. Reviewer MuLC raised the score to Borderline Accept.

Reviewer fGfv didn’t give replies to the authors’ rebuttals. Reviewer fGfv’s score is “Borderline Accept”.

In summary, the paper has new contributions and the questions of reviewers are addressed in the authors’ replies. 

An expected revision is to see the experiments compared with ORBSLAM3 on accuracy. Comparisons on runtime are not required. ORBSLAM3 is also a dense RGBD SLAM and can deal with dynamic objects. Generally, many SLAM systems can deal with dynamic objects by removing the matchings on the dynamic objects. When the occupancies of the dynamic objects on images are larger, the camera pose tracking will fail. How about the proposed method at the time? 

Another expected revision is to see the discussions on why the method can’t be used in outdoor environments. Although RGBD cameras fail to capture depths, Lidar-cameras still can be used.
Decision: Accept (poster)