ProDyG: Progressive Dynamic Scene Reconstruction via Gaussian Splatting from Monocular Videos
Authors: Shi Chen
Erik Sandström
Sandro Lombardi
Siyuan Li
Martin R. Oswald
Venue: NeurIPS 2025 poster ()
Keywords: Dynamic
scene reconstruction
dynamic scene reconstruction
4D reconstruction
Gaussian Splatting
SLAM
monocular
dynamic SLAM
pose estimation
mapping
tracking
Primary area: applications

Abstract: Achieving truly practical dynamic 3D reconstruction requires online operation, global pose and map consistency, detailed appearance modeling, and the flexibility to handle both RGB and RGB-D inputs. However, existing SLAM methods typically merely remove the dynamic parts or require RGB-D input, while offline methods are not scalable to long video sequences, and current transformer-based feedforward methods lack global consistency and appearance details. To this end, we achieve online dynamic scene reconstruction by disentangling the static and dynamic parts within a SLAM system. The poses are tracked robustly with a novel motion masking strategy, and dynamic parts are reconstructed leveraging a progressive adaptation of a Motion Scaffolds graph. Our method yields novel view renderings competitive to offline methods and achieves on-par tracking with state-of-the-art dynamic SLAM methods.

ProDyG: Progressive Dynamic Scene Reconstruction via Gaussian Splatting from Monocular Videos
Shi Chen ETH Zürich
Erik Sandström
Google
Sandro Lombardi Independent Researcher
Siyuan Li ETH Zürich
Martin R. Oswald University of Amsterdam
Abstract
Achieving truly practical dynamic 3D reconstruction requires online operation, global pose and map consistency, detailed appearance modeling, and the flexibil- ity to handle both RGB and RGB-D inputs. However, existing SLAM methods typically merely remove the dynamic parts or require RGB-D B input, while offline methods are not scalable to long video sequences, and current transformer- based feedforward methods lack global consistency and appearance details. To this end, achieve online dynamic namic parts within SLAM system. The poses are tracked robustly with a novel mo- tion masking strategy, and dynamic parts are reconstructed leveraging a progressive adaptation of a Motion Scaffolds graph. Our method yields novel view renderings competitive offline methods and achieves on-par tracking with state-of-the-art dynamic SLAM methods. Project page: cs-vision, muisionngithub.io/ooo hitp:ggbbbbbboooooogggiiiuo
reconstruction by disentangling the static and dy-
Introduction
Dynamic scene reconstruction is fundamental to problems like action recognition, scene under- standing, autonomous driving, robotics and augmented reality, because it provides temporally consistent spatial understanding of how objects and agents move and interact their environment an essential prerequisite for any system to perceive, predict, and act in the world. This problem has been tackled in various ways e.g. as an online SLAM task
89
an offline reconstruction task
69] However, most dynamic SLAM works ignore the dynamic parts [89 4. only reconstructing the static world, only track rigid restricted to object-centric reconstruction 45 188. Offline methods typically separate the reconstruction task into pose estimation followed by reconstruction [55] [56], or are not scalable to long input videos due to their reliance global optimization over all past frames [2] 73] Feed-forward methods train large transformers for online dynamic scene reconstruction, but are yet 86).
(29 73
79
39
56 (55, and lately as feedforward task [86]
87
objects 85 4
to achieve global pose consistency and only produce point clouds 669
Despite recent progress, existing methods fall short of least one of the requirements for practical dynamic scene reconstruction: (1) online operation, tightly coupling pose estimation and dense map reconstruction for scalability; (2) global pose and map consistency; (3) expressive representations like 3DGS [2]] for detailed appearance and geometry: and (4) flexibility to handle both RGB and RGB-D input. SLAM systems often ignore dynamics or lack detail; feedforward methods trade consistency and accuracy for speed; and offline methods challenging scale.
We propose P method for Progressive, Dynamic scene reconstruction with Gaussians from monocular input, that meets all four criteria. Our contributions are:
Prooouncedd "Prodigy".
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Online RGB Stream
Camera Tracking & Dynamic Scene Reconstruction
ProDyG
Legend
Comera Trajectory
Trajertor
t
Figure 1: ProDyG: Given an online RGB stream, ProDyG robustly tracks the camera and progres- sively reconstructs the static background and the dynamic foreground using 3D Gaussian Splat- ting (23] (3DGS). ProDyG reconstructs dynamic scenes with high-quality novel view synthesis, by fusing observations from different timestamps into a consistent dynamic 3DGS [23] representation with Motion Scaffolds [29]. On the right, we show rendering from a novel view overlaid with the estimated camera trajectory and dynamic Gaussian trajectories.
A motion mask prediction strategy, using dynamic flow and refined with semantic point prompting (50). This is integrated into the SLAM backend for robust pose estimation.
An online dynamic reconstruction pipeline that uses Motion Scaffolds 29 to propagate 3DGS in space and time.
Support for both RGB and RGB-D, with ProDyG as the first online RGB-only method.
Competitive novel view synthesis results against state-of-the-ar offline methods.
2 Related Work
Dynamic Dense SLAM. Dynamic dense SLAM methods mainly address two scenarios; filtering
dynamic content to reconstruct only static environments 4 133 90
89
77
19
[5
8
5
74 59 48
57
82
37
32
88
24
7
64
26
72
51),
explicitly tracking and reconstructing dynamic objects
alongside static structures 85
3
43
52
53
78
45
58
65
34
35
31. Basic strategies for
handling dynamic elements include outlier filtering, robust loss functions 90
44
5],
covariance
scaling [1]. but these fail with extensive dynamic content. Instead, a motion mask is estimated with
optical flow 88 (2), semantic segmentation (26 14
35
57
4
33 51
8T
74
85
59
72
53
combination of the two 34 19 [5] 52], with cetetaintt-guuider loss functions 89
with motion
segmentation networks [57] point tracking [65]. unsupervised clustering [7 or via conditional
random fields (32). In many real-world scenarios, reconstructing dynamic objects is critical. Some traditional works track rigid dynamic content during SLAM [85]
3 43
17
52
53
78
[4 or
perform object-centric non-rigid reconstruction [45] 58
8
While the above mentineet methods
can reconstruct dynamic point clouds
4
88 surfels
52
53
oct-trees [8 or
signed distance functions 48
8, they struggle to model photometric details and lighting effects needed for scene understanding and photorealistic rendering. In response, neural implicit methods (33 77 [11 19 have been proposed, but they are too slow for real-time SLAM. Dynamic
45
58
32 72 26 with concurrent works such as DynaGSLAM 34 using dynamic scene flow from optical flow to update the Gaussian means, (65] adapts 779 to the online setting and (35) uses an MLP to deform the dynamic parts. These works cannot handle pure RGB input. For recent survey on dynamic SLAM, we refer to 771..
3DGS (3) SLAM has emerged as solution [15 34 35 31 89
Dynamic 3D Reconstruction. Dynamic 3D reconstruction shares many similarities with dynamic 28
SLAM, but videos are processed offline 29
73 79 9 21
80
27
36
40
41 49 60,
84 2 16
39 25
18
38
68
These works typically implement strategy for deforming dynamic content
over time. Some optimize a deformation field via an MLP (2] 60 40 80
25
73). use motion basis
functions 16
49
36
28 27 21
39 18 or extend 3D Gaussian Splatting (3 with a time
attribute [79 9 Due to the ill-posed nature of dynamic reconstruction, strong priors are commonly
USed, such as point trackers, optical flow, and regularizers like se-rigdd-as-possibe (ARAP), constant
velocity, and acceleration (29
56 68 39]. Inductive biases also help, such MLP smoothness
2

---

Input
Tracking
(30 Fracke Fridis Lifiing
BBA
Colrucees)
Moneslar Mack
Blacks
Depth
Poses
Mask
optiaiind aiech
Monn-eeyfaaess based Keyframes based
Mudian Masch Lase
Musce
Goomarry
Dynamic
salection
RAFT
OBA
Static
Deformation
Photamatiic
Optiaa Fen
Deparity
Lanera
Morralltn Nerballbe Neon
Coarse Batie
Reconstruction
apeinee Static Galariea
modes
Aparnn
Figure 2: ProDyG Architecture. We achieve motion-agnostic online tracking by leveraging to first create keyframe-basee bassed coarse motion masks, from which we seed prompts for SAM2 to distill per-frame fine-gr grained masks. ProDyG processes batches of frames incrementally, employing a keyframe selection similar to [54]. Static background is reconstructed by optimizing the static of Gaussians with proxy depth maps [54]. For dynamic reconstruction,
attach Gaussians to Motion Scaffolds [29], which are initialized by lifting 2D tracks to 3D, to encode a dense motion field. Subsequent final geometric and photometric optimization, the Motion Scaffolds and dynamic Gaussians are extended temporally when new batch of images arrives.
and continuity, or selecting a sparse set of motion bases. While some of these works use SLAM for pose estimation, reconstruction a post-processing step [55] [66..
Feed-forward Methods. Feed-forward methods inspired by DUS:3R [70] and MAS(3R (30 have been proposed to solve dynamic 3D reconstruction. These works rely large-scale training of transformer networks and can infer pointmaps from dynamic input image pairs [86 87
13 69].
However, they typically entangle static and dynamic points 669 86), meaning that the motion of dynamic points cannot be tracked over time. In concurrent work,
87 propose to enable 3I correspondence estimation of dynamic 3D points. DAS3R 775 learns to predict motion masks building on MonST3R [86), and only predicts the static world, using 3DGS (33 Common to all
feed-forward methods that they process the video either in image pairs or in a sliding window,
without guarantecing global pose and map consistency, contrary SLAM. In contrast, we perform globally consistent and online dynamic reconstruction using SLAM.
Method
ProDyG is online dynamic dense mapping and tracking system that robustly tracks monocular camera (sec. 3.1] while also reconstructing and disentangling the static and dynamic parts (sec. 3.3 with 3D Gaussian Splatting representation (sec. 3.2) For overview, see fig.
3.1 Motion-Agnostic Online Camera Tracking
Flow-Based Robust Camera Tracking. We employ Splat-SLAM 54 as our tracking backend, i.e. we maintain factor graph storing camera extrinsics w;, disparity estimates d, R ( per keyframe (node) i, and optical flow pij p: x Xxxjxx per edge (i.j), where pt is the flattened pixel grid from keyframe i, and Pig represents the flattened predicted pixel coordinates when is projected into keyframe j using optical flow. We also store the confidence Wij E IR (H W)x1 associated with the optical flow. Tracking is achieved following Dense Bundle Adjustment (DBA) 666,, where the keyframe poses and disparities d are optimized with reprojection error:
arg min w.d
Σ 1iig - Pis 1 Psj = KujY(w (w,(1/d) '(6,, 117), Σ/ = diag(w, C.). (1)
(E,))E
3

---

(b) Mean Mag. f; of Residual Flow
(a) Input Image
(c) Coarse Motion Mask (d) Fine Motion Mask
Figure 3: Semantic-guided Motion Mask Refinement. The flow magnitude F; (b) is thresholded to
yield (c). Point prompts from (c) are used as input to SAM2 [50] to yield the fine motion masks (d).
Here, & are the edges of a local factor graph, applied in a sliding window manner, K is the camera intrinsics, || l!z denotes the Mahalanobis distance with confidence weights suppressed by coarse binary motion mask C, defined as where motion is detected and C₁ stands for negation of C,.
We generate the coarse motion masks {Ci} as follows. After each DBA iteration, compute the residual flow fız= = (P.y p.) (Ptj Pt) = Pij Pij by subtracting the camera-induced flow Pij p3 from the estimated optical flow Piy pt:- This residual is zero in static regions and larger in dynamic ones. Dynamic areas are identified by evaluating the normalized mean magnitude (1, y) of Piş over the connected target keyframes N, for each keyframe
[n, v)ll Ee'sg/ o/ 11 (4', .2' - p; ,/)
(x.
Σ JEN.
N
HW
For each keyframe, C. is computed by thresholding (a y) the top 20%, assigning zero weight to potentially dynamic regions during DBA. As F, y) is updated each DBA iteration, is progressively refined, enabling robust pose estimation in dynamic environments.
Semantic-Guided Motion Mask Refinement. To improve the separation of dynamic and static regions for reconstruction (sec. 3.3). we use SAM2 [50] to generate fine- mne-grained motion masks for every frame. We show an example visualization of the input image, the mean magnitude F, of the residual flow, the coarse motion mask. and the fine motion mask in fig. 3
Our approach has two phases: initialization and incremental prediction. During initialization, we generate fine motion masks by prompting SAM2 the centroids of connected regions in the median- filtered coarse masks C, During incremental prediction, we extend existing object-wise masks into new frames and add new segmentations based the coarse masks. We validate each segmentation by counting prompt point candidates to filter false positives. Before this phase, we adjust the threshold (top 20% at initialization) based on the dynamic pixel ratio in the latest fine motion mask, adapting to sequence-spece speccfc motion. Detailed algorithms are in the supplemental material.
With the fine motion masks [M.}. further enhance mask C; with the fine during DBA (eq. (T)) as Ei = diag way (1 - M,)). This semantic- semantic-guided guided approach provides more accurate exclusion of dynamic distractors for subsequent iterations of DBA, including local BA, global BA and loop BA (54.
tracking robustness by replacing the coarse
3.2 Static and Dynamic Map Representation
We represent the scene map G = (G., Ca()) as a set of 3D Gaussians 233. split into static G. {g. N, and time-varying dynamic components Ga(t) (t) Each Gaussian
static or dynamic-is parameterized by a mean μæ R rotation matrix R, E g3x3 scale s; E R3, opacity o E [0, 1], and RGB color c; R3 Rendering is performed by first projecting the 3D Gaussians onto the image plane, approximated as 2D Gaussians. The 2D opacity at a pixel p is then
ap(p) og exp (-1(r- (i)--()= TE (p-11)) -1))
where ni: 1w-1 μ4 with the camera- world pose, K the camera intrinsics, and Σ PÊ,PT- P. R )TT with S; diag(s:) and P the affine projection (92). Given the pose

---

w, we render a pixel following the 3DGS (23 pipeline for RGB images C, RaDe-GS 83 for depth maps D', and additionally compute opacity maps O
i-1
-(p)= c,a caa REK. 2-1
a1,), D'()) - - a D1()==
a))) EE j=1
o = Σ as II (1) (4)
EK
Here, K. is the set of Gaussians projected to p. We use the unbiased RaDe-GS rasterizer, which replaces the z-buffer depth with di the ray-Gaussian intersection depth along the viewing ray.
3.3 Progressive Dynamic Scene Reconstruction
We propose a progressive approach dynamic scene reconstruction, which enables online processing and allows the system to adapt to the continuously evolving scene geometry and camera motion. ProDyG uses the same keyframe selection strategy as Splat-SLAM [54], and the addition of new keyframes triggers progressive reconstruction of both static and dynamic regions. To achieve global map consistency, we apply 3DGS map deformations for static regions followed by photometric optimization as in [54]. For dynamic Gaussians, extend the Motion Scaffold representation 29 to handle incremental updates. Next, we review the necessary steps.
Non-Keyframe Camera Poses and Depths. Dynamic scene reconstruction benefits from tempo- rally dense supervision to capture smooth motion, yet the SLAM backend [54] operates on sparse keyframes for efficiency. To bridge this gap. we derive the camera poses and depth maps for non-keyframes that exist between keyframes.
To obtain non-keyframe camera poses, we follow the practice of DROID-SLAM 66
For each non- keyframe F. positioned between neighboring keyframes Fs, and Ftg, construct temporary local graph with edges connecting both keyframes to Fr. We then execute the recurrent update operator and the Dense Bundle Adjustment (DBA) layer this graph, which optimizes the non-keyframe poses based on the already optimized keyframe poses.
For non-keyframe depth estimation, leverage reprojection from neighboring keyframes. Given non-keyframe Fú with its derived camera-to comneer-tt-woon world pose wy and neighboring keyframes Fi, and Fi, with poses Wt, and Wey, first back-project the pixels of both keyframes into 3D:
3D (p+,) = wi, (I/d,))K''I 1)T Pr
PeP (Ptt)) = WE, (1/d1)) [p;,, 1)"
(5)
where K is the camera intrinsic matrix, and de, and de, are the disparities of keyframes Fr, and Peg. respectively. We then reproject all back back-projected 3D points 3D = -rt U U14 from both keyframes to the camera of the non-keyframe F, and record their distance to the camera along the z-axis:
p - Pi..o P:
Kpi
Di (p.) = (pi
(6)
We filter out points that are back- from dynamic pixels or using invalid disparity estimates (determined by a multi-view consistency check in [54] Using the remaining reprojected 21) coordinates and their corresponding depth values, we bilinearly interpolate to create reprojection depth map D Finally, to get dense depth, we align a monocular depth estimate Mo with D repo by estimating scale O, and shift ?t parameters through least squares fitting:
0,, 7/ arg min 0,y
Σ ((OD (u, v) + +) D. (M, (u, x))² (21,2)
D Daligned D.Don
Y
Motion Scaffolds. We adopt Motion Scaffolds 29 (MoSca) as our dynamic representation. MoSca a
structured graph (Va, Ea) representing the underlying motion the scene, where each node (.(m) e Va encodes the motion trajectory of a specific region. Each MoSea node (((m) is defined as
(m) (QQ)
(m)
((()))
(8)
where Q (T2) E SE(3) represents the per- rigid transformation at time and (mm) is an RBF
radius parameterizing the node's influence. These nodes are initially anchored by lifting 2D pixel trajectories using the estimated camera poses W, and depth maps Dilignedd computed in eq.
5

---

Similar to the static map deformation [54], we also deform the dynamic Gaussians to reflect the continuously updated pose and depth estimates from the SLAM backend. Each dynamic Gaussian is defined relative to MoSca node (not in world coordinates) and by re-anchoring the MoSca nodes, the dynamic Gaussians are updated accordingly. The MoSea nodes are re-anchored in 3D when new pose and depth updates are available. Thus, ProDyG achieves global dynamic map consistency prior to optimization. This ensures better convergence and consistency between the static and dynamic components of the scene.
Following MoSca [29], the set of dynamic Gaussians Gu(t) at any query timestamp is formed by warping each individual Gaussian 9i (t,²*) (Hg, R₁, St, Os. C;; t Aw;) from its reference timestamp gg (the timestamp where it is initialized) to t
Ga(t) = (t)))))) s,, o, T((t) W(p, w(x) + Await ())
(9)
Here, w(-) is the base RBF skinning weight parametrized by Jmesttm)) where the neighbor- hood (mm)) consists of the nearest MoSca node u(T²") and all nodes connected to u(m"") and Aw; are learnable per- ner-Gaussian skinning weight corrections. The warping function W(-) is computed using Dual Quaternion Blending (DQB) [2]:
W(x, W; lares last) DQB (wwms ((wm; ImEEs(mr)
(10)
where AQ(m) (m)) = (m)) (m)) ( is the relative transformation between tyre and tast for node m.
Progressive Construction of Motion Scaffolds. After the initial bootstrapping phase of the SLAM backend, we initialize the Motion Scaffolds and dynamic Gaussians. First, we employ 3oTracker3
(20 to generate dense long-term 2D pixel trajectories within the fine motion masks {M, (sec. 3.1 and corresponding per-timestamp visibility labels. Similar to [29]. these 2D trajectories are first lifted into 3D space at visible timestamps using the camera poses and depth maps estimated by the backend, while we linearly interpolate between nearby observations at invisible timestamps. Finally, we sample subset of the lifted 3D tracks to serve as the initial positions for the MoSca nodes.
Subsequently, we carry out geometry optimization on the initialized MoSca nodes as described in 29 to infer rotations and positions of invisible nodes, minimizing the as-rigid-as-pel assssssblee (ARAP) loss, velocity consistency loss and acceleration consistency loss.
After the geometry optimization of the Motion Scaffolds, we initialize the dynamic Gaussians at 3D positions obtained by back-projecting pixels within the fine motion masks. Finally, we perform photometric optimization of both the Motion Scaffolds and the dynamic Gaussians using a combination losses following [29]: an RGB loss Ligh that enforces color consistency, depth (eq. (7)). track
loss Edepth that aligns rendered depth D) with the aligned monocular depth aaligned loss Lunck that ensures consistent motion with the 2D trajectories, and the aforementioned ARAP, velocity, and acceleration losses for geometric regularization. In addition to [29), we introduce a novel motion mask loss that penalizes the rendered opacity of dynamic Gaussians at pixels identified as static by the motion masks
Lmask
Σ Putans
O(p)
(11)
Patatic
where Putatic is the set of static pixels at timestamp t given by negating the motion mask M,, and O(p) is the accumulated opacity of dynamic Gaussians pixel (eq. (4)). This loss effectively prevents dynamic Gaussians from overflowing into static regions, maintaining a clean separation between static and dynamic components of the scene.
As more frames become available, we extend our dynamic reconstruction. For each new frame batch, we run CoTracker3 (20 in temporal window of the new frame batch and an 8-frame overlap with previously processed frames to ensure continuity in the reconstruction. First, we identify "recently visible" tracks as those marked as visible for at least 4 frames within the 8 overlapping frames and extend them into the new frame batch. These extended 2D tracks are then lifted into 3D using the latest camera pose and depth estimates. To identify newly visible dynamic regions, we back-projecr

---

all pixels within the fine motion masks of the new frames into 3D and perform a spherical search to determine whether each baojeproee point has least one lifted 3D track within a predefined radius Teerchh Pixels without nearby 3D tracks are marked as "newly-seen", representing previously unobserved portions of dynamic objects. To capture these newly-seen regions, we execute a second run of the point tracker, specifically querying newly-seen pixels. Finally, we run third tracking pass within the entire dynamic regions of the new frames to replenish the density of visible 2D tracks. This multi-stage tracking strategy ensures dense coverage of all dynamic elements in the scene.
Within the new temporal window, all extended and newly added 2D tracks are lifted into 3D using the same procedure as during initialization. For temporal consistency, we warp new tracks to past timestamps with DQB (Eqn [TO,, using Motion Scaffolds from the previous update. This leverages the latest photometric optimization to guide newly-seen 3D tracks through past frames despite invisibility. We initialize dynamic Gaussians only at newly-seen pixels, then jointly optimize geometry and appearance over the expanded MoSca and dynamic Gaussians using the same losses as during initialization.
4 Experiments
4.1 Experimental Setup
Implementation Details. All experiments were conducted on a cluster with an AMD EPYC 7H12 or 7742 CPU and an NVIDIA A6000 GPU. The kernel size of the median filter used to denoise the coarse motion masks is
The spherical search radius for "newly-seen" pixel identification is Psearch = 0.00m. For geometry and photometric optimization, we keep our loss weights identical with those applied in MoSca (29. and set Amask = 1 as the weight of Lmask. For more implementation details, we refer to the supplemental material.
Datasets.
We evaluate our camera tracking on the Bonn RGB-D Dynamic Dataset 47 and the TUM RGB-D Dataset [62] (dynamic scenes). Since existing works report tracking results on different sets of sequences, we select four mostly used sequences from each dataset to evaluate our method. For rendering, we report novel view synthesis (NVS) results both qualitatively and quantitatively on the iPhone dataset [1]]. To align with Shape of Motion 688, we evaluate our method and all baselines on the 5 sequences used in 68 with the 2x downsampled image resolution. For a fair comparison, we use the preprocessed motion masks given by 68 for all our experiments on the iPhone Dataset.
Baselines. For tracking, we compare with various works on RGB and RGB-D SLAM. The main baseline is Splat-SLAM (54) since base our tracking pipeline on it. For rendering, the baseline methods are NVS-capable monocular Gaussian-based dynamic reconstruction methods including Shape of Motion (68) DynOMo [56]. MoSca 29] and Gaussian Marbles [6].
Metrics. For tracking, we evaluate ATE RMSE [cm] [63 after aligning the estimated camera trajectory with the ground truth via Umeyama alignment [67] For NVS, we report PSNR, SSIM and LPIPS evaluated within the covisibility masks provided by 11 and averaged over all novel views.
4.2 Tracking
In tab. [ we evaluate tracking performance on the Bonn RGB-D Dynamic Dataset 47 and the TUM RGB-D Dataset (62). For both datasets, ProDyG performs competitively among all the RGB-D and RGB SLAM works and shows significant advantage over the main baseline Splat-SLAM 54 This improvement validates the effectiveness of our motion-agnostic camera tracking method introduced
in sec. 3.1 WildGS-SLAM [89] is the only baseline method to outperform ProDyG on both datasets.
Since both methods build upon the tracking framework of Splat-SLAM [54), we attribute this performance gap to two key factors: (1) WildGS-SLAN employs a test-time- time-optimized MLP that produces soft uncertainty masks to suppress confidence weights. As it reconstructs only the static background, it can aggressively suppress regions beyond actual dynamic object boundaries (e.g. moving shadows) without degrading mapping quality. In contrast, ProDyG prioritizes accurate dynamic reconstruction and therefore requires motion masks with precise boundaries, which may miss some effective distractors. (2) WildGS-SLAN benefits from additional DINOv2 [46] features and on-the-fly training of the uncertainty MIP at the cost of higher computational complexity, while the primary computational overhead of our tracking approach comes from SAM2 (50) inference.
7

---

Bonn RGB-D Dynamic Dataset
47
TUM RGB-D Dataset
D9
Method
Type
Ball
Bal12
Pars
Pers2
Avg.
+3/vs
f3/ux
+3/vF
f3/uhn
Inpuf
ORB-SLAM2
S
6.5
23.0
6.9
7.9
11.1
40.8
72.2
80.5
72.3
66.45
NICE-SLAM 90
S
24.4
20.2
24.5
53.6
30.7
79.8
86.5
244.0
152.0
140.57
ReFusion 48
R
17.5
25.4
28.9
46.3
29.5
9.9
40.6
10.4
15.7
DynaSI [N+G]
R
3.0
2.9
6.1
7.8
5.0
0.6
1.5
3.5
2.5
2.03
3.7
RoDyn-SI.AND
19
R
7.9
11.5
14.5
13.8
11.9
1.7
8.3
5.6
DDN-SLAM (RGB-D)
R
1.8
4.1
3.5
1.4
3.9
2.3
2.15
RGB Input
10
DROID-SLAM 60
S
7.5
4.1
4.3
5.4
5.3
1.2
1.6
4.0
2.2
2.25
MonoGS
15.3
17.3
26.4
23.6
21.5
44.2
21.05
Splar-SLAM
S
8.8
3.0
4,9
25.8
10.6
2.3
1.3
3.9
2.2
2.43
DDN-SLAM (RGB) 33
R
2.5
2,8
8.9
4.1
4.58
MryaSaM x
R
3.7
2.6
3.6
1.5
2.6
1.8
1.63
WildGS-SLAM
89
R
2.7
2.4
3.6
3.1
2.94
1.3
3.3
1.6
1.63
DynaMoN
DynaMoN (MS&SS)
651
D
2.8
2.7
14.8
2.2
5.6
0.7
1.4
3.9
1.9
1.98
D4DGS-SLAM
65
D
3.6
3.9
4.3
4D-GS SLAM
35
2.4
3.7
8.9
6.1
2.1
2.6
ProDyG (Ours)
D
2.7
2.6
4.9
3.29
1.2
3.0
Table 1: Tracking Performance on Bonn RGB-D Dynamic Dataset 44
and TUM RGB-D
Dataset [62]. (ATE RMSE | [em]). Best results highlighted as first second third and
concurrent works with". We take the numbers from [89] except [89] 65
35 We categorize each
method into static reconstruction (S), robust against dynamics (R) and producing globally consistent dynamic model (D). ProDyG is competitive with WildGS-SL... -LMM
[9] and MegaSaM [37 while
explicitly reconstructing consistent dynamic model, contrary to
g9 37..
4.3 Rendering
In tab. 2 we evaluate NVS performance quantitatively on the iPhone Dataset [1]] Notably, we differentiate the experiment setups of each methods using the checkboxes, whether the reconstruction or tracking is executed online, and whether the input modalities are RGB only or RGB-D, -DD, As shown in tab. ProDyG outperforms the offline method Shape of Motion [68] in both PSNR and SSIM, while running both the tracking and dynamic scene reconstruction online. This is significantly more difficult task. Compared MoSca (29]. which is an offline method that shares the same representation of the motion field with our framework, ProDyG shows only minimal disadvantages effectiveness of our progressive dynamic reconstruction. DynOMo [56] is the only method capable of online reconstruction with precomputed camera poses, and ProDyG shows a significant advantage over DynOMo when tested under the same constraint. Furthermore, ProDyG still maintains reasonably good performance when only having access to RGB images, while none of the baseline methods is capable of operating under the same setting. Finally, evaluate NVS when estimated poses from the SLAM backend are loaded offline to the online mapper. The performance is very similar in both RGB-D and RGB-only modes, showing that method can find the offline solution,
when optimized with RGB-D input. This demonstrates
when optimized in pure online mode.
In fig. we show qualitative comparisons of novel view renderings on the iPhone Dataset [2] Compared to other methods, DynOMo (56) exhibits worse quality in novel view renderings due to lack of motion constraints (e.g. 2D point tracking). Therefore, ProDyG essentially the first method to support high-quality novel view synthesis of dynamic scenes through online reconstruction from monocular videos. Due to the motion mask loss (eq. (II).. dynamic objects reconstructed by our method tend show more accurate silhouettes than Shape of Motion (68 and MoSca [29].
Limitations. Our dynamic representation struggles with objects that move predominantly outside the viewing frustum and later reappear, due to insufficient photometric constraints for Motion Scaffolds. This causes optimization be dominated by regularization terms, often leading undesired deforma- tions. This issue is inherent limitation also observed in MoSca 29) Additionally, similar to most monocular dynamic view synthesis methods, our system requires per-scene test-time optimization, thereby limiting its applicability in real-time scenarios. Furthermore, ProDyG cannot effectively handle large changes in novel viewpoints, which would necessitate generative models or data-driven priors to hallucinate unseen regions. These challenges highlight key directions for future work.

---

Shape of
Gaussian
ProDyG ProDyG ProDyG ProDyG
DynOMo
[66
MoSca 29
Motion
68
Marbles
61
(Ours)
(Ours)
(Ours)
(Ours)
Online Reconstr.
v
Online Tracking
X
X
X
X
X
v
x
/
RGB-only
X
x
X
X
X
X
v
v
PSNRT SSIM f LPIPS
17.43
11.98
18.44
16.00
17.65
17.87
15.41
15,40
0.591
0.436
0.666
0.634
0.643
0.603
0.582
0.303
0.748
0.311
0.437
0.390
0.377
0.462
0.492
Table 2: Novel View Synthesis Evaluation on iPhone Dataset [12]. All results are averaged over the 5 sequences evaluated [68], with the standard downsampling.
methods except [68] are evaluated without ground truth camera poses. Best results are highlighted as first second third Our method shows superior PSNR and SSIM over the offline Shape of Motion and falls short of the state-of thtt-offt offline method MoSca [29] by a small margin under the extra constraints of online reconstruction and tracking. When tested with precomputed camera poses, ProDyG outperforms the only online competitor DynOMo (56) by significant advantage. Notably, our method still works reasonably well with RGB-only input while being online.
wrindmii 111
Papeer -or-wwirinn
ap2..
DynOMo [56
SoM 68
MoSca ProDyG (Ours) Ground Truth
All methods are trained with access ground truth depth maps. Dynamic objects reconstructed by our method tend to show more accurate silhouettes than (68) and 29]. Note that ProDyG the only one among the methods to perform both tracking and reconstruction online.
Figure 4: Qualitative Novel View Synthesis Results on iPhone 12
5 Conclusion
We proposed ProDyG, progressive dynamic 3D reconstruction framework that meets four key criteria for practical deployment: online operation, global pose and map consistency, detailed appearance and geometry modeling through 3D Gaussian Splatting, and flexibility to operate with either RGB or RGB-D input. Our novel flow-based notion-mask prediction integrated with the SLAM backend enables robust camera tracking dynamic environments, while our online dynamic reconstruction pipeline updates and optimizes Motion Scaffolds and dynamic Gaussians in a progressive manner. Our experiments demonstrate that ProDyG achieves competitive performance both tracking and novel view synthesis.

---

Acknowledgments
We thank the Computer Vision Lab at ETH Zurich for providing the GPU compute resources used in this research, with approval from Ender Konukoglu. Erik Sandström conducted this work as part of his employment at Google Zurich, which supported his participation in this research; Google had no direct role in the project's s design, execution, or analysis. Shi Chen acknowledges support from the NeurlPS 2025 Scholar Award and thanks Stefan Leutenegger for additional travel support.
References
[1] Agarwal, P. Tipaldi, G.D., Spinello, L., Stachniss, C., Burgard, W.: Robust map optimization using dynamic covariance scaling. In: 2013 TEFE international conference on robotics and automation, pp. 62-69. leee (2013)
[2] Bac, Kim, S., Yun, Y., Lec, H., Bang, G., Uh, Y.: Per-gaussian embedding- baaedd deformation for deformable 3d gaussian splatting. European Conference on Computer Vision, pp. 321-335. Springer (2024)
[3] Bescos, B., Campos, C., Tardós, J.D., Neira, J.: Dynaslam ii: Tightly-couped multi-object tracking and slam. IEEE robotics and automation letters 6(3), 5191-5198 (2021)
[4] Bescos, B., Fácil, J.M., Civera, J., Neira, J.: Dynaslam: Tracking. mapping, and inpainting in dynamic scenes. IEEE roboties and automation letters 3(4), 4076-4083 (2018)
Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M., Tardós, J.D.: Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE transactions on robotics 37(6). 1874 1890 (2021)
[6] Chen, X., Chen, Y.. Xiu, Y., Geiger. A., Chen, A.: Easi3r: Estimating disentangled motion from dust3r without training. arXiv preprint arXiv:2503. 24391 (2025)
[7] Dai, W., Zhang, Y., Li, P.. Fang. Z. Scherer, S.: Rgb-d slam in dynamic environments using point
correlations. IEEE transactions on pattern analysis and machine intelligence 44(1), 373-389 (2020)
[8] Dou, M., Khamis, S., Degtyarev, Y., Davidson, P., Fanello, S.R., Kowdle, A., Escolano, S.O., Rhemann, C., Kim, D., Taylor, J., al.: Fusion4d: Real-time performance capture of challenging scenes. ACM Transactions on Graphics (ToG) 35(4), 1-13 (2016)
[9] Duan, Y., Wei, F., Dai, Q., He, Y., Chen, W., Chen, B.: 4d-rotor gaussian splatting: towards efficient novel view synthesis for dynamic scenes. In: ACM SIGGRAPH 2024 Conference Papers. pp- 1-11 (2024)
[10] Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE transactions on pattern analysis and machine intelligence 40(3), 61 611-225 (2017)
[11] Feng, H., Zhang. J., Wang, Q. Ye, Y., Yu, P., Black, M.J., Darrell, T., Kanazawa, A.: St4rtrack: Simultane- ous 4d reconstruction and tracking in the world. arXiv preprint 2534: (2025)
[12] Gao, H., Li, R., Tulsiani, S., Russell, B., Kanazawa, A.: Dynamic novel-view synthesis: reality check. NeurIPS (2022)
[13] Han, J., An, H., Jung. J., Narihira, T., Seo, J., Fukuda, K., Kim, C., Hong, S., Mitsufuji, Y., Kim, S.: Duuut: Enhancing 3d reconstruction with 4d pointmaps for dynamic scenes. arXiv preprint arXiv:2504.0626: (2025)
[14] Henein. M., Zhang, J.. Mahony, R., Ila, V.: Dynamic slam: The need for speed. In: 2020 IEEE International Conference Robotics and Automation (ICRA). pp. 2123-2129. IEEE (2020)
[15] Hou, C., Yeo, Q.X.. Guo, M., Su, Y., Li. Y., Lee, G.H.: Mvgsr: Multi view consistency gaussian splatting for robust surface reconstruction. arXiv preprint arXiv:2503.0800 (2025)
[16] Hu, B., Li. Y., Xie, R., Xu. B., Dong. H. Yao, J., Lee, G.H.: Learnable infinite taylor gaussian for dynamic view rendering. Xivi preprint 2rXiv:2412.04282 (2024)
[17] Huang, J., Yang. S., Zhao, Z. Lai, Y.K., Hu, S.M.: Clusterslam: A slam backend for simultaneous rigid body clustering and motion estimation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 5875-5884 (2019)
10

---

[18] Jeong. Y., Lee, J., Choi, H. Cho. M.: Rodygs: Robust dynamic gaussian splatting for casual videos. arXiv preprint arXiv:2412.03077 (2024)
[19] Jiang, H., Xu, Y., Li, K., Feng. J., Zhang, L.: Rodyn-slam: Robust dynamic dense rgb-d slam with neural radiance fields. IEEE Robotics and Automation Letters (2024)
[20] Karaev, N., Makarov, I., Wang, J., Neverova, N., Vedaldi, A., Rupprecht, C.: Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410. 11831 (2024)
[21] Katsumata, K., Vo, D.M., Nakayama, H.: A compact dynamic 3d gaussian representation for real-time dynamic view synthesis. In: European Conference on Computer Vision. pp. 394 412. Springer (2024)
[22] Kavan, L... Collins, S.. Zára, J., O'Sullivan. C.: Skinning with dual quaternions. In: Proceedings of the 2007 symposium on Interactive 3D graphics and games. pp. 39-46 (2007)
[23] Kerbl, B., Kopanas, G., Leimkühhler T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering, ACM Trans. Graph. 42(4), 139- 1 (2023)
[24] Kim, D.H., Kim, J.H.: Effective background model-based rgb-d dense visual odometry in dynamic environment. IEEE Transactions on Robotics 32(6). 1565-1573 (2016)
[25] Kong, H., Yang, X., Wang, X.: Efficient gaussian splatting for monocular dynamic scene rendering via sparse vime-variant attribute modeling. In: Proceedings of the AAAI Conference on Artificial Intelligence. pp. 4374 4382 (2025)
[26] Kong, M., Lee, J., Loe, S., Kim, E.: Dgs-slam: Gaussian splatting slam in dynamic environment. arXiv preprint wrXiv:2411.10722 (2024)
[27] Kratimenos, A., Lei, J., Daniilidss, K.: Dynmf: Neural motion factorization for real-time dynamic view
synthesis with 3d gaussian splatting. In: European Conference on Computer Vision. pp. 252-269. Springer (2024)
[28] Lee, J., Won, C., Jung, H., Bac, Jcon, H.G.: Fully explicit dynamic gaussian splatting. Advances in Neural Information Processing Systems 37, 3384-5409 (2024)
[29] Lei, J., Weng, Y., Harley, A., Guibas, I.., Danillidis, K.: Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv preprint arXiv:2405. 17421 (2024)
[30] Leroy, V., Cabon, Y., Revaud, J.: Grounding image matching in 3d with mast3r. In: European Conference on Computer Vision. pp. 71-91. Springer (2024)
[31] Li. H., Meng, X., Zuo. X., Liu. Z., Wang, H., Cremers, D.: Pg-slam: Photo-realistic and geometry-aware rgh-d slam in dynamic environments. arXiv preprint 1rXiv:2411. 11.122411.18800 (2024)
[32] Li, M., Chen, W., Cheng, N., J., Li. D., Wang, H.: Garad-slam: 3d gaussian splatting for real-time anti dynamic slam, arXiv preprint arXiv:2502.03228 (2025)
[33] Li, M., Zhou, Y., Jiang, G., Deng. T., Wang, Y., Wang. H.: Ddn-slam: Real-time dense dynamic neural implicit slam. arXiv preprint arXiv:2401.0155 (2024)
[34] R.B., Shaghaghi, M., Suzuki, K., Liu, X., Moparthi, V., Du, B., Curtis, W., Renschler, M., Lee, K.M.B., Atanasov, N., et al.: Dynagslam: Real-time gaussian-splatiin slam for online rendering, tracking, motion predictions of moving objects in dynamic scenes. arXiv preprint arXiv:2503.1 11979 (2025)
[35] Li. Y. Fang, Y., Zhu, Z., Li, K., Ding, Y., Tombari, F: 4d gaussian splatting slam. arXiv preprint MXXv:2503.16711 (2025)
[36] Li. Z., Chen, Z., Li, Z., Xu, Y.: Spacetime gaussian feature splatting for real-time dynamic view synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8508 8520 (2024)
[37] Li, Z., Tucker, R., Cole, F., Wang, Q. Jin, L., Ye, V., Kanazawa, A., Holynski, A., Snavely, N.: Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos. arXiv preprint arXiv:2412.04463 (2024)
[38] Liang. H., Ren, J., Mirzaei, A.. Torralba, A.. Liu, Z. Gilitschenski, I., Fidler, S., Oztireli, C., Ling. H., Gojcic, Z., et al.: Feed-forwar bullet-time reconstruction of dynamic scenes from monocular videos. arXiv preprint wxXiv:2412.0352 (2024)

---

[39] Liang, Y., Xu, T., Kikuchi, Y.: Himor: Monocular deformable gaussian reconstruction with hierarchical motion representation. arXiv preprint mrXivv2504.06210 (2025)
[40] Liang, Y., Khan, N., Ii, 7., Nguyen- Sguyen-Phuoc, T., Lanman, D., Tompkin, J. Xiao, I.:: Gaufre: Gaussian deformation fields for real-time dynamic novel view synthesis. In: 2025 TEFF/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 2642-6222222 22222. IEEE (2025)
[41] Luiten, J., Kopanas, G., Leibe, B., Ramanan, D.: Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In: 2024 International Conference on 3D Vision (3DV). pp. 800-809. IEEE (2024)
[42] Matsuki, H., Murai, R., Kelly, P.H., Davison, A.J.: Gaussian splatting slam. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18039-18000 19944-8048000 (2024)
[43] Morris, J.. Wang. Y.. Kliniewski, M., Ila, V.: Dynosam: Open-source smoothing and mapping framework for dynamic slam. arXiv preprint arXiv:2501. 1189 (2025)
[44] Mur-Arta,, R., Tardós, J.D.: Orb-slam2: An open-source slam system for monocular, steren, and rgh-d cameras, IERE transactions on robotics 33(5), 1255-1262 (2017)
[45] Newcombe, R.A., Fox, D., Seitz, S.M.: Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In: 2015 Proceedings of the IEEE conference computer vision and pattern recognition. IEFE (2015)
[46] Oquab, M.. Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Mairal, J., Labatut, P., Joulin, A., Bojanowski, P.:
Dinov2: Learning robust visual features without supervision (2024), https: / //txxv//xiiv.////a org/ aba/ 2304
07193
[47] Palazzolo, E., Behley, J., Lottes, P., Giguère, P., Stachniss, C.: ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals (2019), https: ipb. uni bonn. de/
paftspaalazzooo2199irco .ddf
[48] Palazzolo, E.,, Behley, J., Lottes, P., Giguere, P., Stachniss, C.: Refusion: 3d reconstruction in dynamic
environments for rgb-d cameras exploiting residuals. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 7855-7862. IEEE (2019)
[49] Park, J.. Bui. M.Q. V. Bello, J.L.G., Moon, J., Oh, J., Kim, M.: Splinegs: Robust motion-adaptin spline for real-time dynamic 3d gaussians from monocular video. arXiv preprint arXiv:2412.09982 (2024)
[50] Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L.,, et al.: Sam 2: Segment anything in images and videos. arXiv preprint axXiv:2408.007 (2024)
[51] Ruan, C., Zang, Q. Zhang, K., Huang, K.: Dn-slam: visual slam with orb features and nerf mapping dynamic environments. IEEE Sensors Journal 24(4), 5279-5287 (2023)
[52] Rünz, M., Agapito, L.: Co-fusion: Real-time segmentation, tracking and fusion of multiple objects. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). pp. 4471-4478. IEEE (2017)
[53] Runz, M., Buffier, M., Agapito, L.: Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects. In: 2018 IEEE international symposium mixed and augmented reality (ISMAR). pp. 10-20. IEEE (2018)
[54] Sandström, E., Tateno, K., Oechsle, M., Niemeyer, M., Van Gool, L., Oswald, M.R., Tombari, F:: Splat- slam: Globally optimized rgb-only slam with 3d gaussians. arXiv preprint arXiv:2405. (2024)
[55] Schischka, N., Schieber, H., Karaoglu, M.A., Gorgulu, M., Grötzner, F,, Ladikos, A., Navab, N., Roth, D., Busam, B.; Dynamon: Motion-aware fast and robust camera localization for dynamic neural radiance fields. IEEE Robotics and Automation Letters (2024)
[56] Seidenschwarz, J., Zhou, Q. Duisterhof, B., Ramanan, D.,, Leal-Taixé, L.: Dynomo: Online point tracking by dynamic online monocular gaussian reconstruction. arXiv preprint arXiv:2409.02 (2024)
[57] Shen, S., Cai, Y., Wang, W., Scherer, S.: Dytanvo: Joint refinement of visual odometry and motion segmen- tation in dynamic environments. In: 2023 IEFE International Conference on Robotics and Automation (ICRA). pp. 4048-4055. IEEE (2023)
[58] Slavcheva, M., Baust, M., Cremers, D., Ilie, S.: Killingfusion: Non-rigid 3d reconstruction without correspondences. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1386-1395 (2017)
12

---

[59] Soares, J.C.V., Gattass, M., Meggiolaro, M.A.: Crowd-slam: visual slam towards crowded environments using object detection. Journal of Intelligent & Robotic Systems 102(2), 50 (2021)
[60] Somraj, N., Choudhary, K., Mupparaju, S.H., Soundararajan, R.: Factorized motion fields for fast sparse input dynamic view synthesis. In: ACM SIGGRAPH 2024 Conference Papers. pp. 1-12 (2024)
[61] Stearns, C., Harley, A., Uy. M., Dubost, F., Tombari, F., Wetzstein, G.. Guibas, L.: Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In: SIGGRAPH Asia 2024 Conference
Papers. pp. 1-11 (2024)
[62]
Sturm, J., Engelhard. N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of rgb-d slam systems. In: Proc. of International Conference on Intelligent Robot Systems (IROS) (Oct 2012)
[63] Sturm, Engelhard, N., Endres, Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. International Conference on Intelligent Rohots and Systems (IROS). IEEE/RSI (2012). http:///di.org/10.111 TTW.///O...00...33885113.
leee. org/document/ 6385773/
[64] Sun, Y., Liu, M., Meng, M.Q.H..: Improving rgb-d slam dynamic environments: motion removal
approach. Robotics and Autonomous Systems 89, 110-122 (2017)
[65] Sun, Z., Lo, J., Hu, J.: Embracing dynamics: Dynamics-aware gaussian splatting slam, arXiv preprint arXiv:2504.0484 (2025)
[66] Teed, Z., Deng. J.: Droid-slam: Deep visual slam monocular, stereo, and rgb-d cameras. Advances in neural information processing systems 34, 16558-16569 (2021)
[67] Umeyama, S.: Least-squares estimation transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence 13(04), 376-380 (1991)
[68] Wang, Q. Ye, V., Gao, H., Austin, Li, 7.,, Kanazawa, A.: Shape of mation: 4d reconstruction from a single video. arXiv preprint arXiv:2407. 13764 (2024)
[69] Wang, Q. Zhang, Y., Holynski, A., Efros, A.A., Kanazawa, A.: Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 12337 (2025)
[70] Wang, S., Leroy, V., Cabon, Y., Chidlovskii, B., Revaud, J.: Dust3r: Geometric 3d vision made easy. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20697-20709 (2024)
[71] Wang, Y., Tian, Y., Chen, J., Xu, K., Ding, X.: A survey of visual slam in dynamic environment: The evolution from geometric semantic approaches. IEEE Transactions on Instrumentation and Measurement (2024)
[72] Wen, L., Li, S., Zhang, Y., Huang. Y., Lin, J., Pan, F, Bing, Z., Knoll, A.: Gassidy: Gaussian splatting slam in dynamic environments. arXiv preprint arXiv:2411.15470 (2024)
[73] Wu, G., Yi, T., Fang. Xie, L.. Zhang, X., Wei, W., Liu, W., Tian, Q. Wang, X.: 4d gaussian splatting for real-time dynamic scene rendering. In: Proceedings of the IEEE/CVF conference computer vision and pattern recognition. pp. 20310-20322 (2024)
[74] Xiao, L., Wang, J., Qiu. X., Rong, Z., Zou, X.: Dynamic slam: Semantic monocular visual localization and mapping based on deep learning in dynamic environment. Robotics and Autonomous Systems 117, 1-16 (2019)
[75] Xu, K., Tse, T.H.E., Peng, J., Yao, A.: Dasăr: Dynamics- aware gaussian splatting for static seene reconstruction. arXiv preprint arXiv:2412. 19584 (2024)
[76] Xu, Y.. Jiang, H., Xiao, Z., Feng. J., Zhang, L.: Dg-slam: Robust dynamic gaussian splatting slam with hybrid pose optimization. arXiv preprint arXiv:241 1.08373 (2024)
[77] Xu, Z., Niu, J., Li, Q., Ren, T., Chen, C.: Nid-slam: Neural implicit representation based rgb-d slam in dynamic euvironments, In: 2024 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6. IEEE (2024)
[78] Yang, S., Scherer, S.: Cubeslam: Monocular 3-d object slam. IEEE Transactions on Robotics 35(4), 925-938 (2019)
[79] Yang, Z., Yang, H., Pan. Z., Zhang, L.: Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting, arXiv preprint arXiv:23 10.14 (2023)
13

---

[80] Yang. Z., Cao. X., Zhou, W., Jiao, S., Zhang, Y.. Jin, X.: Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In: Proceedings of the IEEB/CVF conference on computer vision and pattern recognition. pp. 20331-20341 (2024)
[81] Yu, C.,, Liu, Z., Liu, X.J., Xie, F., Yang. Y., Wei, Q. Fei, Q. Ds-slam: A semantic visual slam towards dynamic environments. In: 2018 TEEF/RSJ international conference intelligent robots and systems (IROS). pp. 1168-1174. IEEE (2018)
[82] Yu, X., Ye. W. Guo, X.. Ming. Y., Li, J., Bao, H., Cui, Z., Zhang, G.:
D Dllowslmm: Self supervisd dynamic slam with flow motion decomposition and dino guidance. arXiv preprint arXiv:2207.0877 (2022)
[83] Zhang, B., Fang. C., Shrestha, R., Liang, Y., Long, X., Tan, P.: Rade-gs: Rasterizing depth in gaussian splatting. arXiv preprint arXiv:2406.014 01667 (2024)
[84] Zhang, D., Li, G., Li, J., Bressicux, M., Hilliges, O., Pollcfeys, M., Van Gool, L., Wang, X.: Egogaus- sian: Dynamic scene understanding from egocentric video with 3d gaussian splatting, arXiv preprint arXiv:2406. 19811 (2024)
[85] Zhang, J.. Henein. M., Mahony, R., Ila, V.: Vdo-slam: A visual dynamic object-aware slam system. arXiv preprint arXiv: 1052 (2020)
[86] Zhang, J., Hermann, C., Hur, J., Jampani, V., Darrell, T., Cole, F,, Sun, D., Yang, M.H.: Monstr:: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825 (2024)
187] Zhang, S., Ge, Y., Tian, J., Xu, G., Chen, H., Lv, C., Shen, C.: Pomato: Marrying pointmap matching with temporal motion for dynamic 3d reconstruction. arXiv preprint arXiv:2504.05697 (2025)
[88] Zhang, T., Zhang, H., Li. Y.. Nakamura, Y., Zhang, L.: Flowfusion: Dynamic dense rgb-d slam based on optical flow. In: 2020 IEEE international conference on robotics and automation (ICRA). pp. 7322-7328. IEEE (2020)
[89] Zheng, J., Zhu, 7., Bieri, V. Pollefeys, M., Peng, S., Armeni, T.: Wildgs-slam: Monocular gaussian splatting slam in dynamic environments. arXiv preprint arXiv:2504.03886 (2025)
[90] Zhu, Z., Peng, S., Larsson, V. Xu, W., Bao, H., Cui, Z., Oswald, M.R., Pollefeys, M.: Nice-slam: Neural implicit scalable encoding for slam. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12786-12796 (2022)
[91] Zou, Z., Qu Z., Peng. X., Boominathan, V., Pediredla, A., Chakravarthula, P:: High-speed dynamic 3d imaging with sensor fusion splatting. arXiv preprint arXiv:2502.0466 (2025)
[92] Zwicker, M., Pfister, H., Van Baar, J., Gross, M.: Surface splatting. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques. pp. 371-378 (2001)
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We cover contributions in the abstract and bullet points in the introduc-
tion.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the
contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
14

---

2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
We discuss limitations in separate paragraph towards the end of the experi- ment section.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specinf well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer:
[NA]
Justification:
We do not derive new theoretical results in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
15

---

Answer: [No]
Justification: There are details related to hyperparameters that are not covered in the paper, but disclosed in the works which reference. All information will, however, be made available once the source code is published.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis-
sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We wished to have provided the code at submission time, but to improve readability of the code, we choose to release this after submission time, but as soon as possible.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
public/ public/guides/ httii//uuudiee//eeubimieeeeee
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https //nips https//w.ww.....ee//////////eaaa
for more details.
16

---

The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [No]
Justification: We provide all information necessary to understand the results, but as men- tioned before, some hyperparamters that are the same as in Splat-SLAAM [54 and MoSca [29] are not disclosed.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
17

---

Answer: [Yes]
We include the CPU and GPU specs used to benchmark the runtime of our system, but believe that small differences in the setup of that hardware may still result in differences in runtime across devices.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We provide a framework for dynamic dense SLAM whose primary objective is AR/VR applications and household robotics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We target dense dynamic scene reconstruction with the primary objective of household AR/VR and roboties.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
18

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks. mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [No]
Justification:
We will not release data or models that have a high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We credit the authors where we are inspired by their works, and we respect their licenses, but we do not describe their licenses etc. This will be properly done when releasing the code publicly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the sesrrswithhcdee package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: We do not release assets (code) yet, but hope to do so as soon as possible. Then we will document this.
Guidelines:
19

---

The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: All data we use is publicly available covering scenes where appropriate licenses and compensation have been taken care of for potential participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [No]
Justification: All data we use is publicly available covering scenes where appropriate licenses and compensation have been taken care of for potential participants.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [No]
Justification: We do LLMMs for our methodology in any way.
Guidelines:
20

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy (https://neurip:: /ttp://pp.ccc//offfeeeee//////// for what should or should not be described.
21


Reviews:
[{"review_id": "UUdK9VEFOI", "reviewer": "Reviewer_mjPB", "summary": "The paper proposes a method for online dynamic reconstruction from an unposed monocular video stream. The method works by computing masks for moving scene regions in two steps, and using dense bundle adjustment to compute keyframe poses based on the unmasked static scene regions. Interpolated poses for non-keyframes are then used along with scaled depth to generate the static regions of the scene as 3D Gaussians. The dynamic regions are reconstructed and updated using a previous method.", "questions": "1. Given that the overhead of optimizing dynamic Gaussians is likely to prevent real-time or even interactive execution, what is the particular use case the authors imagine for an offline method? And doesn't this limit the practicality of the method on the same lines as the previous approach mentioned in the introduction (L26-27)?\n\n2. Since the dynamic reconstruction happens progressively, how does the temporal consistency of the reconstruction compare to the baseline methods? If space allows, it would be insightful to provide some evaluation (either qualitative or quantitative) of the temporal consistency of the proposed approach.", "limitations": "None", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The task of online 4D reconstruction is novel and has huge potential in practical applications.\n2. The paper is well-written, and provides a thorough and detailed overview of the method.\n3. Given the constraints of online reconstruction, the results of the paper in comparison to past work (such as SoM) are impressive.\n\nWeaknesses:\n1. The method section is quite heavy on details and can become overwhelming. To avoid losing sight of the forest among the trees, I would suggest providing a more intuitive overview of each section, and better motivating the design decisions.", "was_revised": true, "final_justification": "The authors answered my questions in the rebuttal.", "rebuttal": "Thank you for your input! We respond to the weaknesses and questions you raised as follows:\n\n## Weaknesses\n\n**1.** Agreed. Thank you for the suggestion! We restructured the method section in the camera ready version to improve readability.\n\n## Questions\n\n**1.** **We believe that an online method, though not real-time, can be more scalable than a purely offline one.** This is because online systems like ProDyG often work in a sliding window manner rather than processing the entire sequence at once such that the compute demands remain managable even for longer sequences. An online approach also has advantages if more image data is added to improve the quality of a scanned scene without the need to recompute everything from scratch. Our online system is able to reuse previous computations. Also, **our online pipeline shows a paradigm upon which a faster system can be built**. For example, the dynamic Gaussian optimization can potentially be replaced by recent feed-forward methods to approach or mapping optimizations can be made spatially adaptive and sped up with second order solvers to achieve real-time or interactive performance in the future.\nWe believe our unoptimized code base can serve as a starting point and as proof-of-concept baseline for future dynamic SLAM methods which may benefit from the many recently proposed speed-ups for both mapping and tracking, but they are out of scope/focus for our work.\n\n**2.** Thank you for the advice! With the Motion Scaffolds being able to warp dynamic Gaussians over time, temporal consistency is considered to be a strength of ProDyG over existing methods. We will add qualitative and quantitative evaluations of point tracking on the iPhone dataset in the camera ready version to demonstrate the temporal consistency of our method. Additionally, we would like to stress that even without the point tracking evaluation, our NVS results shown in Table 2, Figure 4 and the supplementary video are already able to demonstrate how well ProDyG achieves temporal consistency in dynamic regions to a great extent."}, {"review_id": "oZAxpTUyl8", "reviewer": "Reviewer_JEry", "summary": "This is a new online SLAM system that performs dense reconstruction of both static and dynamic scene elements, using gaussian splats. The emphasis is not so much on novel components as combining them in an intelligent and novel way into an end-to-end system with good performance.", "questions": "Must field-of-view / intrinsics be supplied or does ProDyG optimize them (sorry if I missed this in the paper)? A column for this could be a nice addition to Table S1 (\"Method Classification\").\n\nl.101,113 describe threshold motion masks at 20%. Does this tell us something about how much dynamic content can be present before the method stops working successfully?\n\nThe MegaSaM system is claimed to also work well on scenes with low parallax (i.e. very little camera motion or motion with primarily rotation) that traditional SLAM systems tend to fail on. How does ProDyG fare on such cases?", "limitations": "Yes, but it would also be informative (in the supplementary video) to see some qualitative examples of failure modes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "[Strengths] The writing is excellent throughout, with clear and concise descriptions and a minimum of tedious puffery. This is a complex system with many moving parts and yet the paper manages both to describe it at a high level and to provide many specific details (with many more in the supplementary material). Reimplementing such a system is a major task but with the promise to release code, it should be possible to use it and base future research on it.\n\n[Strength] Both the overall approach and the many individual design decisions are well-motivated. To call out a couple in particular: I like the method of using flow to generate coarse masks and a segmentation model to refine them, and I was impressed to see that tracking and motion scaffolds have been added without breaking the online nature of the system.\n\n[Mixed] Evaluation is performed against a wide selection of baselines, and covers both tracking and novel view synthesis. However, the choice of datasets is somewhat limited, as are the metrics (no rotation error metrics are reported). Runtime numbers are also given (in the supplementary materials), though here there is not much comparison to other methods. (I was also curious to know whether the runtime might be competitive with other SLAM systems when used in a tracking-only manner.)\n\n[Weakness] There are not many qualitative examples, and no qualitative comparison of tracks. Are the top methods all similar enough that there is no interesting qualitative difference?", "was_revised": true, "final_justification": "My questions were answered very satisfactorily. After reading the response and the other authors' reviews, I retain my rating of Accept.", "rebuttal": "Thank you for your input! We respond as follows:\n\n## Strength and Weaknesses\n\n> “[Mixed] Evaluation is performed against a wide selection of baselines... the choice of datasets is somewhat limited, as are the metrics (no rotation error metrics are reported)... whether the runtime might be competitive with other SLAM systems when used in a tracking-only manner.”\n\nPossible options of datasets are limited, when requiring dynamic objects and known camera motion. Regarding the metrics for tracking, we could have reported rotation error for our method, but as this is not reported in the vast majority of baseline methods we compare our method to, we ended up only reporting ATE RMSE. \nWhen used in a tracking-only manner, our runtime is almost the same as Splat-SLAM [54] (tracking-only, ~10 FPS). To make this clearer, we show below a **component-wise runtime breakdown** when running ProDyG on the paper-windmill sequence in the iPhone dataset:\n\n| Component                                        | Time (s) |\n| :----------------------------------------------- | -------: |\n| **Total**                                       | 8052     |\n| ├─ Tracking                                      | 22       |\n| └─ Mapping                                       | 8030     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ Geometry optimization         | 1230     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ Photometric optimization      | 5424     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ CoTracker3 running time      | 785      |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;└─ Rest                                      | 591      |\n\nHere, we split the total running time into time for tracking / mapping, and the time for mapping is further split into time for the geometry optimization, the photometric optimization, the CoTracker3 [20] running time, and time for the rest of the operations (such as data loading).\n\n> “[Weakness] There are not many qualitative examples, and no qualitative comparison of tracks. Are the top methods all similar enough that there is no interesting qualitative difference?”\n\nYes. For example, our predicted camera trajectories tend to look very similar to what WildGS-SLAM predicts (and also to GT trajectories) when plotted in 2D. Therefore, we prioritized showing qualitative results of NVS given the limited space. We generated additional qualitative videos which will be published on our project website.\n\n## Questions\n\n> “Must field-of-view / intrinsics be supplied…”\n\nUnder the current design, the intrinsics need to be supplied beforehand. However, since MoSca, an important component of our method, supports the joint optimization of camera intrinsics, it should be possible to extend ProDyG to handle it.\nThank you also for your suggestion to add a column about this in Table S1! We will do so in the camera ready version.\n\n> “l.101,113 describe threshold motion masks at 20%. Does this tell us something about how much dynamic content can be present...”\n\nYes, this is a heuristic hyperparameter setting and it does provide a prior about the general proportion of dynamic content present in the frames. However, please note that this is a global hyperparameter applied to all sequences we report experimental results on (i.e. we did not tune it sequence-by-sequence).\n\n> “The MegaSaM system is claimed to also work well on scenes with low parallax… How does ProDyG fare on such cases?”\n\nThank you for bringing this up! Our observation is that ProDyG’s tracking performance can degrade in such cases, as triangulation becomes degenerate and we do not employ any special designs to address this issue as MegaSaM does. That said, it should be easy to address this issue in a similar fashion since both ProDyG and MegaSaM are based on DROID-SLAM.\n\n## Limitations\n\n> “Yes, but it would also be informative (in the supplementary video) to see some qualitative examples of failure modes.”\n\nThank you for the suggestion! Unfortunately, we are not allowed to show them here in the rebuttal. We will instead show some qualitative examples of failure modes on our upcoming project page.\n\n## Paper Formatting Concerns\n\n> “I didn't exactly understand the claim at line 38 ('first online RGB-only method')…”\n\nThank you for pointing out the ambiguity! The line 38 was meant for \"the first online RGB-only method to achieve dynamic reconstruction from monocular input\". We will make this clearer in the camera ready version. In Table S1, DynaMoN is the only existing method to satisfy these conditions. However, as we point out starting from line 9 in the supplementary material, DynaMoN's novel view synthesis performance under larger viewpoint changes remains unvalidated. In contrast, we demonstrate through NVS evaluations that ProDyG achieves dynamic reconstruction with 3D consistency.\n\n> “There are lots of capitalization errors in the References (\"3d\", \"slam\", etc).”\n\nThank you for pointing this out! We will go through the references and make sure to fix these errors in the camera ready version.\n\n## References\n\n[20] Karaev, N., Makarov, I., Wang, J., Neverova, N., Vedaldi, A., Rupprecht, C.: CoTracker3: Simpler and Better Point Tracking by Pseudo-labelling Real Videos. arXiv preprint arXiv:2410.11831 (2024)\n[54] Sandström, E., Tateno, K., Oechsle, M., Niemeyer, M., Van Gool, L., Oswald, M.R., Tombari, F.: Splat-SLAM: Globally optimized RGB-only SLAM with 3D Gaussians. arXiv preprint arXiv:2405.16544 (2024)"}, {"review_id": "fZJvpSNE9k", "reviewer": "Reviewer_KrLA", "summary": "This paper presents a complete online system that simultaneously tracks the camera and reconstructs both static backgrounds and dynamic foregrounds using 3D Gaussian Splatting (3DGS). The proposed pipeline leverages motion scaffolds to propagate 3DGS representations in real time and handles dynamic environments by integrating coarse and fine-grained motion cues.", "questions": "It would be helpful if the authors could make the method section clearer and more organized.\nIt would also be great to include results on long video sequences.", "limitations": "Yes.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nEffective Dynamic Masks:\nThe system combines DROID-SLAM for coarse dynamic masking and SAM2 for fine-grained motion segmentation. This combination is both practical and efficient, enabling robust foreground-background separation without requiring heavy supervision or extensive engineering.\n\nPracticality and Real-World Application:\nWhile the system is somewhat high-level and system-oriented, it targets real-world challenges in dynamic scene reconstruction. The integration of robust tracking and dynamic object modeling makes it highly applicable for robotics and AR/VR use cases. The effort to build a usable system in this underexplored direction is commendable.\n\n\nWeaknesses and Questions:\n\n1. Unclear Initialization Strategy:\nThe initialization phase of the system is not clearly described. It would be helpful to elaborate on how camera poses are recovered at the start and how the initial 3D reconstruction is built.\n2. Pose Estimation in Dynamic Scenes:\nIt's not fully explained how the system estimates accurate poses in the presence of dynamic elements. What is the robustness mechanism during scenes with large dynamic foreground motion?\n3. Optical Flow Redundancy:\nThe system uses DROID-SLAM for motion masking and CoTracker-3 for generating long-term dense 2D trajectories. A natural question arises: why not rely solely on CoTracker to distinguish dynamic and static regions? It could simplify the pipeline and reduce dependencies.\n4. System Redundancy and Efficiency:\nThe current design involves multiple overlapping components—DROID-SLAM for masks, SAM2 for refinement, and CoTracker for 2D tracking. While each tool contributes to robustness, the pipeline appears somewhat redundant. A discussion on potential simplification or trade-offs in performance vs. efficiency would strengthen the work.\n5. Real-Time Performance and Memory Profiling:\nThe paper claims real-time capability, but there is no detailed evaluation of runtime or memory usage across modules. A table or analysis comparing the runtime breakdown and GPU memory requirements of each stage would make the real-time claim more convincing.\n6. Pose and Motion Optimization Details:\nThe pose optimization process is not well explained. How does the system jointly recover camera poses and dynamic object trajectories? Is the optimization based on sparse or dense correspondences, and is it integrated into the 3DGS update?", "was_revised": true, "final_justification": "After reviewing the other reviewers’ feedback, I believe this paper is on the borderline between acceptance and rejection, and I am comfortable with either outcome.", "rebuttal": "Thank you for your insightful input! Since your questions mainly concern specific details of the method section and the motivation behind our design choices, we believe these can be effectively addressed by our responses below (including a module-wise runtime breakdown):\n\n## Weaknesses and Questions\n\n> **1.** “Unclear Initialization Strategy”\n\nSorry for not explaining clearly enough how the initialization phase works. **We chose to focus on our contributions which pertains to updating the camera poses and the scene representation progressively.** Roughly speaking, the way we initialize the camera poses and static Gaussians is similar to that of Splat-SLAM [54]. Specifically, the system first builds a factor graph by adding keyframes (a new keyframe is added when the average optical flow magnitude exceeds a preset threshold compared to the last keyframe), and as soon as the number of keyframes reaches a certain number (set to 12 in all our experiments), the system bootstraps the collected keyframes to get their camera poses. After the bootstrapping, the static Gaussians are initialized and optimized using the latest keyframe (RGB image and depth map estimated by the SLAM backend). As for the initialization of dynamic reconstruction, we do it similarly to MoSca [29] using all frames (RGB images, estimated camera poses, depth maps and motion masks) up to the last keyframe used for initial bootstrapping. Specifically, we run CoTracker3 [20] within the identified dynamic regions to get dense 2D tracks and sample a subset of them to lift into 3D and initialize motion scaffolds. Then, geometry and photometric optimizations are carried out to get the initial dynamic reconstruction. We will clarify this part further in the camera ready version.\n\n> **2.** “Pose Estimation in Dynamic Scenes”\n\nGenerally, ProDyG is robust against dynamic foreground motion by predicting motion masks and suppressing the confidence weight of identified dynamic regions to zero in subsequent DBA iterations. After each DBA iteration, the system computes (see Eqn. 2) and thresholds the “residual flow” at the top 20% to get a coarse motion mask. Then, the coarse motion mask filters out potentially dynamic regions in the next DBA iteration for more robust egomotion estimation. As the DBA iterations go on, the coarse motion masks are gradually updated, and the pose estimation becomes increasingly robust against foreground dynamics because of the improvement of the masks. When the mapping module is triggered, we further refine the coarse motion masks by prompting SAM2 [50] (see “Semantic-Guided Motion Mask Refinement” in 3.1 and in Supplementary Material A), leveraging the power of semantic segmentation. Afterwards, we replace the coarse masks using the refined ones for more accurate, confidence weighted suppression, in subsequent iterations of DBA, which includes local BA, global BA and loop BA (please refer to Splat-SLAM [54] regarding these different types of Bundle Adjustment).\n\n> **3.** “Optical Flow Redundancy” \n\nOur observation is that the internal flow representation of DROID-SLAM [66] is rather low-resolution, but more robust for pose estimation. We use CoTracker3 [20] to predict dense 2D tracks which we use to initialize motion scaffolds, and to provide extra supervision for dynamic Gaussians. We choose not to use CoTracker3 also for dynamic-static separation for mainly two reasons: a) **CoTracker point tracking results can be noisy**, especially in textureless regions, which can lead to suboptimal motion masking (the low-resolution optical flow estimated by DROID-SLAM is more robust in this sense); b) Currently, we only use Cotracker to track points within the motion masks. **If we tracked all points densely, it would add a considerable computation overhead.**\n\n> **4.** “System Redundancy and Efficiency” \n\nThank you for the insightful question! Indeed, the optical flow estimation in DROID-SLAM [66], SAM2 [50] for semantic segmentation of dynamic objects and CoTracker3 [20] for dense 2D point tracking have some functional overlap in terms of finding spatial correspondences, so in theory there is room for simplification. In practice, however, considering the main runtime bottleneck is geometry and photometric optimization of Motion Scaffolds and dynamic Gaussians (see the runtime breakdown we provide in response to 5), **the efficiency gain of removing this redundancy would be marginal.**\n\n> **5.** “Real-Time Performance and Memory Profiling” \n\nWe need to first clarify that **we don’t claim real-time capability**. In addition to Table S2 where we report the overall runtime and GPU memory consumption, here we provide an extra **module-wise runtime breakdown** when running ProDyG on the paper-windmill sequence in the iPhone dataset (we split the total runtime into time for tracking / mapping, and the time for mapping is further split into time for the geometry optimization, the photometric optimization, the CoTracker3 [20] runtime, and time for the rest of the operations such as data loading):\n\n| Component                                        | Time (s) |\n| :----------------------------------------------- | -------: |\n| **Total**                                       | 8052     |\n| ├─ Tracking                                      | 22       |\n| └─ Mapping                                       | 8030     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ Geometry optimization         | 1230     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ Photometric optimization      | 5424     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ CoTracker3 running time      | 785      |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;└─ Rest                                      | 591      |\n\n> **6.** “Pose and Motion Optimization Details” \n\nIn the ProDyG pipeline, camera pose estimation and dynamic reconstruction are separated. At a high level, camera poses are estimated as follows: low-resolution optical flow is computed between keyframes and keyframe poses and depth maps are optimized to explain the flow in static regions indicated in the estimated motion masks (see our response to 2) as much as possible. The dynamic object trajectories are mainly recovered from the CoTracker3-estimated [20] dense tracks and photometric clues. Therefore, both pose estimation and dynamic reconstruction are based on dense correspondences, but the optical flow used for pose estimation is at a lower resolution, and yes, it is integrated into the 3DGS update since we use a photometric loss and a 2D track loss during the photometric optimization phase.\n\n> “It would be helpful if the authors could make the method section clearer and more organized. It would also be great to include results on long video sequences.” \n\nThank you for the advice. We will improve the structure of the method section to make it clearer in the camera ready version. In terms of video lengths, the longest video among those we conduct experiments on is “freiburg3_walking_halfsphere” in the TUM RGB-D dataset, which is 35.81 seconds long. We believe this is a sufficiently long sequence to show the capabilities of our method.\n\n## References\n\n[20] Karaev, N., Makarov, I., Wang, J., Neverova, N., Vedaldi, A., Rupprecht, C.: Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.11831 (2024)  \n[29] Lei, J., Weng, Y., Harley, A., Guibas, L., Daniilidis, K.: Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. arXiv preprint arXiv:2405.17421 (2024)  \n[50] Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., et al.: Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 (2024)  \n[54] Sandström, E., Tateno, K., Oechsle, M., Niemeyer, M., Van Gool, L., Oswald, M.R., Tombari, F.: Splat-slam: Globally optimized rgb-only slam with 3d gaussians. arXiv preprint arXiv:2405.16544 (2024)  \n[66] Teed, Z., Deng, J.: Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems 34, 16558–16569 (2021)"}, {"review_id": "4ib1S7TWRX", "reviewer": "Reviewer_KJPL", "summary": "The submission tackles online monocular dense dynamic scene reconstruction. The key differentiator to prior work is that depth input is optional. Similar to Motion Scaffolds, the method uses a number of pretrained models to extract depth, segmentation masks, and optical flow. It combines all this information with a motion scaffold to obtain a dynamic reconstruction. Unlike Motion Scaffolds, the proposed method works in an online setting. It borrows ideas like its keyframe handling and static reconstruction from Splat-SLAM to estimate camera poses online, which originally only handles static scenes.", "questions": "See the major weaknesses above since they are intertwined. All major weaknesses are experimental. Although I enjoyed this submission, I'd like to see video results, ablations, and better tracking evaluations to recommend acceptance. Since the rebuttal format does not allow for videos, I am not sure that my concerns can be addressed in a rebuttal.", "limitations": "The method is global for the static part of the scene but the dynamic part is only global \"accidentally\" by stitching together local dynamics, i.e. dynamic loop closure / track re-identification after prolonged invisibility is not tackled beyond accidental re-identification due to simple regularizers like ARAP. Please make this clearer in the Limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe problem setting is very interesting and challenging. It is also timely, with two concurrent works [8,18], see Table 1 in the supplement and Table 1 in the main paper.\n\nThe Related Work section is very thorough and to the point.\n\nThe Method section is written well.\n\nThe method is designed quite well, and extends prior work in a sensible manner.\n\nThe authors promise a code release.\n\n\n\nWeaknesses:\n\nThe evaluation has very few qualitative results. There are no video comparisons to other methods.\n\nQualitative video results for Table 2 would be good to get a sense of how important depth input is and how well the method estimates poses when they are not given.\n\nOther than Table 2, there appear to be no ablations despite the rather involved design of the method. For example, does the motion mask algorithm need to be this complicated since it ultimately only seeds queries to SAM2 (and removes dynamic components for pose estimation but being over-eager there doesn't seem like a problem)?\n\nSomething seems off about the qualitative comparisons in Fig. 4: Why are the results of DynOMo so bad? The results on their project page look much better, including on the same scene.\n\nPresumably the tracking performance is evaluated on static and dynamic components? If so, the static components would heavily dominate the results, as seen by WildGS-SLAM (which apparently is only robust against dynamics and does not reconstruct them) dominating in Table 1. Please provide masked results where only the dynamic points are evaluated (even using coarse motion masks should give enough signal but preferably use the fine motion masks).\n\n*** Minor ***\n\nHow are poses in between keyframes estimated? Presumably, the method runs until it hits the next keyframe and then goes back and processes all non-keyframes in between in an online fashion? This could be clearer in the text.\n\nWhy is the method so slow, at 10-30 sec per frame? How is this split among the components? Presumably the pretrained models make up the bulk of that because I don't see how the presented method takes that much time otherwise?\n\nWhy is static Splat-SLAM treated as the main tracking baseline? It obviously does not handle dynamics.\n\nThe trajectories are hard to see in the video results.", "was_revised": true, "final_justification": "The rebuttal addresses all of my concerns well enough, given the limitations of the rebuttal setting. I read the other discussions as well and I didn't find anything objectionable. I trust the authors to follow up on their promises in a revised version. I'll raise my rating to accept.", "rebuttal": "Thank you for the input! We respond to the weaknesses, questions and limitations you raised as follows:\n\n## Weaknesses\n\n> “The evaluation has very few qualitative results. There are no video comparisons to other methods.”\n\n**Due to the volume limit of only 9 one-columned pages**, we were only able to show a few qualitative results in the paper. **We have many more which we plan to release on our upcoming project page**, including predicted camera trajectories, training-view rerenderings, novel-view renderings on the iPhone dataset (all sequences including those not providing GT test views, e.g. handwavy, haru-sit…), the Bonn Dynamic dataset and the TUM RGB-D Dynamic dataset. And thank you for pointing out the lack of video comparisons. **We will also put video comparisons** with some important baseline methods (offline: Shape of Motion, MoSca, etc., online: DynOMo, DynaMoN, etc.) **on our project page**.\n\n> “Qualitative video results for Table 2 would be good to get a sense of how important depth input is and how well the method estimates poses…”\n\nThank you for the suggestion. **We will also show these video comparisons on our project page** to give readers a better idea of the influence of depth and pose inputs.\n\n> “Other than Table 2, there appear to be no ablations… does the motion mask algorithm need to be this complicated... being over-eager there doesn't seem like a problem)?”\n\nThank you for pointing this out! Yes, for tracking/pose estimation it is okay for the motion mask algorithm to be over-eager to some extent as long as it leaves enough static regions for the DBA to function well. Of course, if the algorithm misses some dynamic objects in the scenes, the tracking performance degrades. The comparison of ProDyG and Splat-SLAM in Table 1 supports this statement. **We believe the importance of having accurate fine motion masks lies more in the reconstruction/NVS side.** To demonstrate the importance of our algorithm design for SAM2 prompting, **we conducted an ablation study focused on NVS quality**: We compared the NVS performance on the paper-windmill sequence from the iPhone dataset under three different setups:\n**1.** ProDyG (using our residual-flow-based prompting)\n**2.** ProDyG, but replacing the residual-flow-based prompting with a similar approach based on optical flow applied in DynaGSLAM [34]. Specifically, we compute and threshold the magnitude gradient of optical flow estimated by the SLAM backend to detect object edges, and close the shapes using dilations and erosions. After that, we prompt SAM2 at the centroids of connected regions in the resulting binary masks. We will include more technical details of this algorithm in the camera ready version of our supplementary material.\n**3.** Splat-SLAM (no motion masking at all)\nWe show the results in the table below:\n\n| Setup | PSNR  | SSIM | LPIPS |\n| :--: | :--: | :--: | :--: |\n| **1 (Ours)** | **20.05** | **0.62** | **0.26** |\n| 2 | 19.18 | 0.51 | 0.31 |\n| 3 | 17.49 | 0.54 | 0.31 |\n\n**Our SAM2 prompting algorithm clearly outperforms the substitute method and the case where no motion masking is applied.** The substitute method (Setup 2) tends to over-segment, mis-identifying some static foreground objects (e.g. the flower pot) as dynamic ones. As a result, the static objects are reconstructed with some faulty motion, causing poor NVS quality in those regions. In comparison, **our residual-flow-based method manages to decouple true scene motion from camera-induced motion and thus more accurate motion masks, resulting in superior NVS results.**\n\n> “Something seems off about the qualitative comparisons in Fig. 4: Why are the results of DynOMo so bad? …”\n\nThank you for the question. We need to point out that **the renderings on DynOMo’s project page are all from training views, while what we show in Fig. 4 are novel-view renderings**. According to our observations running DynOMo, their NVS is constantly poor, especially later in sequences (will be shown in the video comparisons on our project page). They don’t seem to employ any effective constraints on the Gaussian primitives along the ray directions to ensure 3D consistency. After all, DynOMo only guarantees good 2D tracking.\n\n> “Presumably the tracking performance is evaluated on static and dynamic components? ...”\n\nWe believe **there has been a misunderstanding between point tracking**, which measures how  dynamic objects move, **and camera tracking**, which estimates the camera extrinsics. **What we show in Table 1 are the camera tracking results**, which we measure using ATE RMSE. We will add point tracking results evaluated on the iPhone dataset (only on dynamic points) in the camera ready version. However, we would like to stress that even without point tracking evaluation, our NVS results shown in Table 2, Figure 4 and the supplementary video are already able to demonstrate how well ProDyG models scene dynamics to a great extent. Moreover, NVS is usually a more difficult task than 2D point tracking due to the need to maintain 3D consistency. DynOMo, as we show in Table 2 and Figure 4, is a typical example of performing very well in 2D point tracking while being extremely poor at NVS. This is why we prioritized NVS evaluation over point tracking given the limited space of the main paper.\n\n### Minor\n\n> “How are poses in between keyframes estimated? ...”\n\nApologies for not explaining this clearly enough. Your interpretation is absolutely correct. \n\n> “Why is the method so slow… How is this split among the components? ...”\n\n**The geometry optimization of the Motion Scaffolds, the photometric optimization of dynamic Gaussians, and CoTracker tracking are the three main contributors of the runtime.** We show in Table S2 that we can cut down a major part of the runtime by reducing the number of iterations for the geometry and photometric optimization and number of initial tracks for CoTracker, with minimal performance loss. Here, we also show a clearer **component-wise runtime breakdown** when running ProDyG on the paper-windmill sequence in the iPhone dataset below (Here, we split the total runtime into time for tracking / mapping, and the time for mapping is further split into time for the geometry optimization, the photometric optimization, the CoTracker3 runtime, and time for the rest of the operations, e.g. data loading.).\n\n| Component                                        | Time (s) |\n| :----------------------------------------------- | -------: |\n| **Total**                                       | 8052     |\n| ├─ Tracking                                      | 22       |\n| └─ Mapping                                       | 8030     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ Geometry optimization         | 1230     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ Photometric optimization      | 5424     |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├─ CoTracker3 running time      | 785      |\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;└─ Rest                                      | 591      |\n\n> “Why is static Splat-SLAM treated as the main tracking baseline? ...”\n\nThank you for pointing this out! Indeed, “main baseline” is confusing. This is only meant to show that our modification based on Splat-SLAM manages to make it handle dynamics. We will improve the wording here in the camera ready version.\n\n> “The trajectories are hard to see in the video results.”\n\nApologies. Currently both the camera and Gaussian trajectories are visualized by inserting extra Gaussians and rendering them along with other Gaussians. We will improve the visualization code before putting the videos on our project page. \n\n## Questions\n\n> “See the major weaknesses above since they are intertwined…”\n\nThank you very much for the insightful review! We tried our best within the short rebuttal period to respond to your questions, and to address some of the issues you raised, we added **an ablation of the motion mask generation algorithm** to demonstrate its importance to reconstruction/NVS. We also provide **an extra component-wise runtime breakdown**. Unfortunately, **we are not allowed to show more qualitative results in the rebuttal, but we will instead put them on our upcoming project page**. We hope our response and additional experimental results address your concerns more or less.\n\n## Limitations\n\n> “The method is global for the static part of the scene but the dynamic part is only global \"accidentally\" by stitching together local dynamics...”\n\nAgreed. ProDyG is yet to handle re-identification of dynamic objects after long invisibility, and this is indeed an interesting and important direction for future work. We will clarify this in the camera ready version.\n\n## References\n\n[34] Li, R.B., Shaghaghi, M., Suzuki, K., Liu, X., Moparthi, V., Du, B., Curtis, W., Renschler, M., Lee, K.M.B., Atanasov, N., et al.: Dynagslam: Real-time gaussian-splatting slam for online rendering, tracking, motion predictions of moving objects in dynamic scenes. arXiv preprint arXiv:2503.11979 (2025)"}]

Author rebuttal:

Meta-review: This paper received in average very positive reviews. Three reviewers gave an accept, while one reviewer gave a borderline reject, however stating that the rebuttal has addressed his concerns. Final review is as of now still outstanding. After having read all the discussions and justifications, the conclusion is that this is a positively received work with only minor issues in the wording as well as an incomplete runtime evaluation or a missing evaluation of temporal consistency. Overall, this AC aligns with the reviewers' view that these problems should be easy to address in the final version, and recommends acceptance for this work.
Decision: Accept (poster)