4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos
Authors: Mengqi Guo
Bo Xu
Yan Li
Gim Hee Lee
Venue: NeurIPS 2025 poster ()
Keywords: Dynamic Rendering
4D Representation
Gaussian splatting
Motion-Aware
Primary area: deep_learning
TL;DR: Motion-aware 4D Gaussian splatting enables pose-free reconstruction of dynamic scenes from monocular video with 5× faster training and superior rendering quality.

Abstract: Novel view synthesis from monocular videos of dynamic scenes with unknown camera poses remains a fundamental challenge in computer vision and graphics. While recent advances in 3D representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown promising results for static scenes, they struggle with dynamic content and typically rely on pre-computed camera poses.
We present 4D3R, a pose-free dynamic neural rendering framework that decouples static and dynamic components through a two-stage approach. Our method first leverages 3D foundational models for initial pose and geometry estimation, followed by motion-aware refinement. 4D3R introduces two key technical innovations: (1) a motion-aware bundle adjustment (MA-BA) module that combines transformer-based learned priors with SAM2 for robust dynamic object segmentation, enabling more accurate camera pose refinement; and (2) an efficient Motion-Aware Gaussian Splatting (MA-GS) representation that uses control points with a deformation field MLP and linear blend skinning to model dynamic motion, significantly reducing computational cost while maintaining high-quality reconstruction.
Extensive experiments on real-world dynamic datasets demonstrate that our approach achieves up to 1.8dB PSNR improvement over state-of-the-art methods, particularly in challenging scenarios with large dynamic objects, while reducing computational requirements by 5× compared to previous dynamic scene representations.

4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos
Menggi Guo',
Bo Xu², Yanyan Li",
Gim Hee Lee'
National University of Singapore
Wuhan University
{mengqi guo, gimhee. 1ee] -coopp.cmmm...o nus wuw.euu.uu..e.. edu.
Monocular Video
Pose-Free 4D Gaussian Splatting
Dynamic Control Points
Figure 1: Overview of our pose-free 4D Gaussian Splatting. Given monocular video sequence of dynamic scene (left), our method directly reconstructs the 4D scene without pre-computed camera poses (right). Dynamic control points guide the deformation of Gaussian points model motion, producing high-quality novel views across different time steps.
Abstract
Novel view synthesis from monocular videos of dynamic scenes with unknown camera poses remains a fundamental challenge in computer vision and graphics. While recent advances in 3D representations such Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown promising results for static scenes, they struggle with dynamic content and typically rely on pre-computed camera poses. We present 4D3R, pose-free dynamic neural rendering framework that decouples static and dynamic components through two-stage approach. Our method first leverages 3D foundational models for initial pose and geometry estimation, followed by motion aware refinement. 4D3R introduces technical innovations: (1) motion-aware bundle adjustment (MA-BA) module that combines transformer- based learned priors with SAM2 for robust dynamic object segmentation, enabling more accurate camera pose refinement; and (2) an efficient Aotion-Aware Gaussian Splatting (MA-GS) representation that uses control points with deformation field MLP and linear blend skinning model dynamic motion, significantly reducing computational cost while maintaining high- quality reconstruction. Extensive experiments demonstrate that our approach achieves up to 1.8dB PSNR improvement over ofate-of-the-an art methods, particularly in challenging scenarios with large dynamic objects, while reducing computational requirements by 5x compared to previous dynamic scene representations.
key
real-world dynamic datasets
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Novel view rendering from monocular videos of dynamic scenes remains a fundamental challenge in both computer graphics and computer vision communities. While static scene reconstruction has seen significant advances with methods like 3D Gaussian Splatting (3DGS) 255
58 29 and
I 21. dynamic scenes present substantially greater challenges. Unlike static environments, dynamic scenes require modeling intricate 3D environments with temporal coherence while handling complex camera viewpoints. These approaches leverage Gaussian primitives or neural layers but face severe challenges in capturing scene evolution with high fidelity, especially when addressing substantial changes in handling dynamic content such as managing motion, ensuring temporal consistency, and maintaining efficient scene representations.
Neural Radiance Fields (NeRF) [8] 20
3
Adapting 3DGS to dynamic scenes has led to the development of various 4D-GS approaches [60] [2] that incorporate deformation fields modeled by multi-layer perceptrons (MLPs), motion bases, or 4D) representations. Despite achieving quantitative improvements, these methods typically rely on pre- computed camera poses from multi-view systems or Structure-front from-Motion pipelines, which often fail in scenes with significant dynamic content. This dependency on ground truth or pre-estimated camera poses severely limits their applicability in real-world environments.
Estimation of 6-DoF camera poses typically involves establishing 2D-3D correspondences followed
by solving the Perspective-n-Porie (PnP) [99 problem with RANSAC [14. Approaches for predicting Siwtuurefroo--oooieee (SfM) methods such as COLMAP [47 48). and scene coordinate regression (SCR) 49 SIM
2D-3D correspondences can be broadly categorized into two main directions:
methods detect and describe key points in 2D images [53 11, linking them to corresponding 3D coordinates M4 46.. Although these methods are effective, they still face challenges including high computational overhead, significant storage requirements, and potential privacy concerns when processing sensitive data [5]]. In contrast, SCR methods (49 S [6 71 utilize deep neural networks (DNNs) to directly predict the 3D coordinates of the image pixels, followed by running PnP with RANSAC for camera pose estimation. These approaches offer notable advantages such as higher accuracy in smaller scenes, reduced training times, and minimizing storage requirements. Taking advantage of these benefits, this paper adopts SCR over traditional SfM methods for camera pose estimation. Furthermore, DUSt3R (56) employs a Vision Transformer (ViT)-based architecture to predict 3D coordinates using a data-driven approach and in the following work, MonST3R [7π] extends DUSt3R to dynamic scenes by fine-tuning the model suitable dynamic datasets. However, treating pose estimation and scene reconstruction as separate tasks in dynamic environments typically leads to suboptimal performance.
straightforward approach to addressing the pose-free novel view synthesis problem is to directly combine MonST3R (7] with 4D-GS methods (22). However, the accuracy of camera poses predicted by MonST3R is not sufficiently stable, causing 4D-GS methods to struggle with reconstructing accu- rate scenes and resulting in poor rendering quality. Particularly, these methods often fail in scenarios involving moving objects that occupy a significant portion of the image. Since correspondences between 2D keypoints and 3D points are established based on static scene elements, dynamic objects are commonly deemed to be outliers during the RANSAC process. This assumption breaks down in the presence of large or dominant moving objects, further degrading performance in dynamic environments.
To overcome these issues, this paper proposes novel architecture for pose-free dynamic Gaussian Splatting that integrates transformer-based motion priors for initial pose estimation and then refines using a Mottonn-Awar Bundle Adjustment (MA-BA) module. Our key insight is that the motion mask and scene reconstruction should be jointly optimized rather than treated as isolated processes, allowing for more accurate camera pose estimation and higher-quality novel view synthesis. Specifically, the ViT-based transformer gives the initial dynamic mask. We sample the top-K values and turn their location into K- K-point prompts for pretrained SAM2 (3]. Finally, the final dynamic mask is the combination of the output dynamic object segments from the SAM2 and the confidence map from the transformer. The dynamic mask serves static point selection when performing RANSAC, which can reduce the noise introduced by the dynamic points.
For 4D-GS, the expensive computational cost is huge burden since millions of GS points need to learn a set of motion parameters. Some works design compact representations to solve this problem, such as the sparse motion basis (23] sparse-control points [22, and k-plane [60]. We design our 4D

---

Motion-Aware Gaussian Splatting
Motion-Aware
Bundle Adjustment
Monocular Video
Camera
Camera Intrinaic
Poses
Static Points
Dynamic Points
VIT
Depth
Sampled
Points
SAM2
Mask
Transformer
Decoder
Confidence
4D-Aware Information Extractor
Figure 2: Overview of our motion aware 4D gaussian splatting pipeline. Our framework consists of three main modules: (1) A 4D-aware information extractor that processes input frames through parallel ViT encoders and decoders to extract geometric and motion information; (2) A motion-awan bundle adjustment module that leverages motion predictions for robust camera estimation; and (3) motion- aware gaussian splatting module that enables dynamic scene modeling through adaptive control points.
representation of Motion-Aware Gaussian Splatting (MA-GS) based motion masks generated from the MA-BA module. Specifically, we model the dynamic motion by hundreds of control points with deformation field MIP,, and Gaussian points using Linear Blend Skinning (LBS). The model first trains the control points for the dynamic part and then trains the GS points.
We show the effectiveness of our approach through extensive experiments both synthetic and real-world dynamic scenes. Our findings demonstrate notable gains in pose estimation accuracy and reconstruction quality over current methods. In particular, outperform tftte-of-theaar techniques on difficult dynamic scenes by achieving improvements of 1.8 PSNR in novel view rendering quality and more accurate pose estimation.
Our key contributions include: 1) We propose novel motion-aware pipeline that fundamentally integrates pose estimation with scene reconstruction, eliminating the traditional separation that causes failures in highly dynamic scenes. 2) We introduce a theoretical framework for motion-aware bundle adjustment that jointly optimizes camera poses and scene representation, enabling robust performance even when moving objects dominate the scene. 3) We design a compact and efficient 4D representation using motion-aware gaussian splatting that significantly reduces memory requirements while maintaining rendering quality. 4) Our approach demonstrates state-of-te oh-tthf-aar art performance on challenging dynamic scenes without requiring pre-computed poses, enabling truly monocular novel view synthesis.
Related Work
2.1 Static Scene Novel View Rendering.
Novel View Synthesis aims generate novel viewpoints from set of observations. Recently, neural implicit representations have shown impressive capabilities. NeRF (38] achieved breakthrough results by representing scenes through MLPs. Subsequent work focused on acceleration through methods
like baking [21] and explicit representations [39]. 3DGS [25] introduced efficient rasterization
of anisotropic 3D Gaussians, enabling real-time rendering without quality degradation. Recent
extensions have ireal-time rendering [68 44 camera modeling [6], faster training
35 16 39 7
and sparse view 669 45). However, these methods assume static scenes and known camera parameters,
limiting their practical applications.

---

2.2 Dynamic Scene Novel View Rendering.
Research has expanded capture both motion and geometry in dynamic scenes 64 37 66
30]. Initial approaches [2
40] learned additional time-varying deformation fields. Alternativ 16 62 encode scene dynamics through multi-dimensional feature fields without explicit motion modeling. Following 3DGS, recent work [60] proposes learning individual Gaussian trajectories over time. More efficient representations have emerged, including factorized motion bases [27] and sparse control points [22] Another approach [65] extends spherical harmonics to 4D. As noted in Dycheck [18), many methods focus on unrealistic scenarios, while real-world capture involves substantial motion. To resolve motion ambiguity, recent work leverages pretrained depth estimation [63] or trajectory tracking 24]. Our approach utilizes DUS(3R [56] for initialization and incorporates SAM2 [43] for dynamic motion segmentation.
methods 28
2.3 Pose-Free Neural Field.
Traditional NVS relies heavily on SfM 447 for camera parameters. Recent research explores optimizing neural fields without pre-calibrated poses. iNeRF(67] estimates camera poses from pre-trained NeRF through photometric optimization. NeRF- [59] jointly optimize camera and scene parameters with geometric regularization. BARF [33 and GARF (10) address gradient inconsistency in positional embeddings. Nope-NeRF [4] leverages geometric priors for accurate camera estimation. In the 3DGS domain, CF3DGS 17 introduces progressive optimization, while InstantSplat [12] uses DUSt3R (56) for initialization but remains limited to static scenes. ZeroGS [9] utilizes the DUSt3R and progressive image registration for pose-free 3D GS. Our approach differs by introducing a pose-free pipeline for dynamic scenes that decouples static backgrounds from dynamic objects. We utilize DUSt3R's geometric foundation model and leverage 3DGS's explicit nature for enhanced geometric regularization.
3 Method
3.1 Preliminaries
3D Gaussian splatting represents scenes using collection of colored 3D Gaussian primitives. Each Gaussian Gj is characterized by its center position μıj, covariance matrix Σ, (parameterized by rotation quaternion q and scaling vector Sj), opacity value jj. and spherical harmonic coefficients shy for view-dependent appearance. The scene representation is thus G = {G; My qj Sy, oj Shū]:
During rendering, these 3D Gaussians projected onto the image plane with transformed 2D covariance matrices Σ'. The final color at each pixel is computed through a-blending:
i-1
C(u) Σ Tia SH(sh; vi), where Ti = III (1-a;) iEN
==1
Our framework extends Gaussian Splatting to dynamic scenes with sparse control while maintaining computational efficiency and rendering quality. For more details, please refer to the supplementary.
3.2 Problem Definition and Overview
Given monocular video sequence V = [T capturing dynamic scene with moving objects and camera motion, our goal is to reconstruct a complete 4D representation of the scene. Our pipeline estimates the following parameters: 1) Camera parameters C. {K,, R, Tr where K₁ E 3x3 represents the intrinsic matrix, and [R; T:] SE(3) denotes the extrinsic parameters. 2) Dense depth map Dr # and motion map M, {0, capturing per-pixel information. 3) Dynamic scene representation through motion-aware Gaussian Splatting parameters G, motion-aware
control points PP, and deformation field MLP O.
As illustrated in Fig 2 our framework combines implicit geometric estimation and explicit motion understanding to address the unique challenges of dynamic scene reconstruction from monocular video through three primary components: 4D-aware information extractor, Motion-Aware Bundle Adjustment (MA-BA) BA) and Motion-Awe Aware Gaussian Splatting (MA-GS) representation.

---

3.3 4D-Aware Information Extraction
Our 4D-aware information extractor serves as the foundation for both camera pose estimation and scene reconstruction by leveraging pre- pre-trained vision models to extract geometric and motion information from monocular inputs. To have good initialization of the Gaussian splatting, we first employ pre-trained ViT-based transformer model from MonST3R [71] that sequentially processes each input frame I through an encoder-decoder block that extracts deep features to give scene coordinate map X; E R X x3 representing the 3D structure and a confidence map W E H indicating the reliability of the predictions. We also include the optical flow from SEA-RAFT [57].
Using the scene coordinate map X₁ and confidence map Wr, we obtain high-confidence points S through a principled two-step filtering process:
S = {p: We(p.) T D((p)) < Ta).
(2)
where T. and та are the confidence and depth thresholds, respectively. This filtering strategy is based on two key insights: 1) The selection of points with high confidence scores are more likely to yield reliable geometric estimates. 2) Filtering out points infinity with zero disparity that give unreliable depth estimates.
Unlike previous approaches that rely solely on motion estimators, leverage SAM2 43 semantic understanding ability to generate pixel-precise dynamic object segmentation M, E {0, 1] xW with high-confidence points S prompts. This method critically enables our motion- -warre processing pipeline to handle scenes dominated by dynamic content.
3.4 Motion-Aware Bundle Adjustment
Our MA-BA module introduces approach camera pose estimation that explicitly models the separation between static and dynamic scene components, addressing fundamental limitation in traditional bundle adjustment methods. We leverage the dynamic region mask M, enhance the accuracy of camera pose estimation. For frame pairs (Iz, Ir), we introduce a masked PnP-RANSAC approach that focuses solely on static regions:
P Pstatic {p, E S M.(p.) = 0}.
(3)
By restricting correspondence matching static regions, we significantly reduce the likelihood of incorrect matches due to dynamic objects. The optimization objective becomes:
E RR, Ti)
Σ
[[(P + TT) p/11?.
P.EP static
where IT(-) is the camera model mapping a set of 3D points onto the image.
We further refine the camera poses through Differentiable Dense Bundle Adjustment (DBA) layer [54) For more details, please refer to the supplementary.
3.5 Motion-Aware Gaussian Splatting
Our Motion-Aware Gaussian Splatting (MA-GS) module introduces a principled approach to dy- namic scene representation that significantly reduces the parameter space by focusing computa- tional resources on regions requiring deformation modeling. We adopt a set of control points p= (p: R o. RR ) where p: represents the 3D coordinate in the canonical space and o; defines the radius of the Radial Basis Function (RBF) kernel. These control points are initialized from our motion map Mr. where the static and dynamic regions are handled distinctly through our MA-GS module.
In the first stage, optimize the control points the dynamic regions with the following loss:
Loontrol
Σ P.EP
(5)
where Mp(p:) acts as binary mask selectively optimize only the control points in dynamic regions and Trrender refers to the standard photometric loss between rendered and ground truth pixels, using LI and DSSIM metrics. The equation selectively applies this loss only to dynamic regions via the
5

---

Mi(pi) binary mask. This targeted optimization significantly reduces the complexity by focusing only on regions that require deformation modeling. For dynamic control points, we learn time-varying transformations through a specialized mapping function:
② (pt, - (R², TT)
where [ ] SE(3) represents the six-degrees-on of-freedom transformation at time t. θ the deformation field MLP. For numerical stability and continuous interpolation, we represent rotations using unit quaternions r e HI, where H denotes the space of unit quaternions. The dynamic scene rendering process utilizes these control points through weighted transformation blending.
In the second stage, we optimize the Gaussian parameters G3 = {M. Qi S/. sy, by applying motion-aware constraints through a detached gradient path:
if (1 = 0
MJ
EEEN; wyk (R) (e) - -P4) Pk + +T))
otherwise,
where Wiy is Linear Blend Skinning (LBS) weight [52]. Crucially, this constraint is implemented with gradient detachment to ensure the motion-aware transformation does not affect the parameter updates of the MLP and vice versa. This prevents competing optimization objectives between Gaussian parameters and deformation field parameters, leading more stable convergence and better results. The LBS weights are computed with a normalized exponential kernel:
Wik-
Wiyk/ AENj
Σ
Wyki
where Wzs (app-dGx//200
(8)
d, represents the Euclidean distance between Gaussian center μ4 and control point Pk, and Ng denotes the set K -neearstt neighboring control points for Gaussian j.
For orientation updates, we employ quaternion blending to ensure smooth rotational deformation:
wyrr() & qj.
REAY
where @ denotes quaternion multiplication. This formulation ensures the entire dynamic scene experiences smooth and physically realistic deformations while maintaining computational efficiency through our motion-awar design.
3.6 Training Strategy
As mentioned in the previous section, our training process optimizes the entire scene representation through the two-stage procedure. Specifically, we optimize for the control points in the first stage with Leontrol The second stage of optimization for the motion-aware Gaussian parameters is achieved in the rendering loss Lyender using LI distance and DSSIM metrics as follows:
L Lyender + Larap + Arigid
(10)
where Larop enforces local rigidity with as-rigid as asspssiblee regularization [50]:
(pr (2) = Σ (p p() R.( pk )11;
(11)
and Lrigid enforces rigidity in static regions:
Lrigid = (11-M())))ll
- 11.12
(12)
We employ an adaptive control point strategy that optimizes point distribution based on reconstruction impact. We compute the gradient magnitude of the rendering loss with respect to Gaussian positions, weighted by their influence radius:
gk
W
(13)
where wy represents the LBS weights connecting Gaussian points to control points, while aL the gradient of the loss with respect to Gaussian positions. This gradient magnitude sum (gk) measures each control point's influence on reconstruction quality. During optimization, we add control points in areas with high yb. adaptively refining the representation where needed most. This adaptive approach ensures effective representation while maintaining high reconstruction quality across diverse dynamic scenes with varying motion complexity.

---

Table 1: Quantitative results on HyperNeRF's 41
dataset. The best and the second best results are denoted by pink and yellow. The rendering resolution is 960x540. "Time" in the table stands for training times plus camera pose estimation time.
Model
COLMAP
PS
MS-SSIM1
Times
FPS
Storage(MB).
Nerfies [40]
22.2
0,803
~ hours
<
HyperNeRF
22.4
0.814
32 hours
Tox-uVox-I
13
24.3
0,836
3.5 hours
48
3D-GS [25]
19.7
0.680
4 hours
55
52
FDNeRF [70]
24.2
0.842
0.05
440
4DGS [60]
25.2
0.845
5 hours
61
SC-GS [22]
25.3
0.841
4 hours
45
85
MonST3R+SC-GS
20.4
0.697
2 hours
153
RoDynRF [36]
23.8
0.820
28 hours
<1
200
Ours
25.6
0.844
50 mins
80
Table 2: Quantitative results DyNeRF's [28] dataset. The best and the second best results are denoted by pink and yellow. "Time" in the table stands for training times plus pose estimation time.
Model
COLMAP PSNR(dB)
MS-SSIMI
Times
FPSt
Storage(MB)J
Hyper
16.9
0.638
hours
<1
TiNeuVox 13
18.2
0.712
3.5 hours
48
3D-GS 25
15.3
0.541
hours
55
52
4DGS 600
18.9
0.740
5 hours
34
61
SC-GS 22
19.0
0.746
hours
45
85
MonST 3R-
X
16.4
0.624
hours
45
RoDynRF [66
X
17.8
0.620
28 hours
<1
200
Ours
19.6
0.755
50 mins
45
80
Experiments
4.1 Experimental Settings
Datasets.
evaluate our approach on three representative datasets: HyperNeRF'ss dataset [1]. DyNeRF dataset (28). and the MPI Sintel dataset [6]. The DyNeRF dataset features controlled dynamic scenes captured with synchronized cameras, offering a solid baseline for evaluation. We only camera view for training, treating it as monocular sequence. The HyperNeRE dataset presents more challenging scenarios with complex object deformations and camera movements. The MPI Sintel dataset provides ground truth camera poses, enabling quantitative evaluation of pose estimation accuracy.
Evaluation Metrics We evaluate using standard metrics for Novel View Synthesis: Peak Signal-to- Noise Ratio (PSNR), Structural Similarity (SSIM), and Multi-Scale SSIM (MS-SSIM). For camera pose estimation, we report the metries as [8] Absolute Translation Error (ATB), Relative Translation Error (RPE trans), and Relative Rotation Error (RPE rot), after applying Sim(3) Umeyama alignment prediction the ground truth.
Baselines. For a comprehensive evaluation, we compare our approach against state-of-the-art methods both novel view rendering and pose estimation. For novel view rendering, we consider methods designed for static scenes like 3DGS 225 and dynamic scene methods including Nerfies (40),
HyperNeRF [1 TiNeuVox [13], 4DGS 660, FDNeRF 77), and SC-GS [22]. which represent the
current state- of-the-art in dynamic scene modeling. We further compare with pose-free 4D novel view rendering baselines RoDynRF (36) and strong baseline combining MonST3R [7]] (one the best dynamic pose estimation methods) with SC-GS [22] (one of the best dynamic scene modeling methods). For pose estimation evaluation, we compare against established methods such as DROID-
SLAM [54], DPVO (55) and ParticleSFM 73 noting that these methods require camera intrinsics
as input.
Implementation Details. Our implementation uses ViT-based transformer for 4D information extraction, pre-trained on DUSt3R (56) and fine-tuned on MonST3R [71] datasets. For dynamic segmentation refinement, employ SAM2 [43] with multi-point prompting. The Motion- Aware

---

(4) MonST3R Mask
[b] Input Image
(c) Depth Map
(d) Confidence
(e) Sampled Points
(1) Refined Mask
Figure 3: Our motion mask refinement pipeline: (a) Initial dynamic mask from MonST3R showing coarse segmentation, (b) Input image of tomato cutting scene, (c) Estimated depth map highlighting object boundaries, (d) Confidence map indicating regions of dynamic motion, (e) Strategically sampled points for mask refinement, and (f) Final refined mask after SAM2 processing showing improved object boundary delineation. The pipeline effectively captures the dynamic nature of the cutting motion while maintaining precise object boundaries.
Table 3: Ablation study of our proposed modules on HyperNeRF dataset.
Method
PSNR(dB)1
MS-SSIM
Times
w/o motion-map
20.4
0.697
2 hours
w/o SAM-refine
23.8
0.765
hours
w/o MA-GS
23.5
0.734
1.5 hours
Ours
25.6
0.844
50mins
Gaussian Splatting uses 512 control points, optimized using Adam with learning rates from le-4 to 1e-7 (exponential decay). All experiments run on a single NVIDIA RTX3090 GPU.
4.2 Results on Novel View Rendering
A key advantage of our approach is its superior performance on challenging scenes with dynamic objects. On the DyNeRF dataset (Tab 2). we achieve state-of of-the-art results with a PSNR of 19.6dB and MS-SSIM of 0.755, outperforming existing methods regardless of their reliance on known camera poses. This superior performance comes from our motion-aware components, which effectively handle dynamic objects while maintaining accurate camera pose estimation.
Our method demonstrates exceptional computational efficiency, achieving 5x faster training time pared to COLMAPPdependent methods while maintaining comparable quality. On the HyperNeRF dataset (Tab I and Fig 4), we achieve results (PSNR of 25.6dB and MS-SSIM of 0.844) compet- itive with SC-GS (25.30 (5.3dB/0.841) and 4DGS (25.2dB/0.845) while significantly outpacing other COLMAP-fres approaches like RoDynRF (23. 833.8d//0.820 and MonST3R+SC-GE (20.4dB/0.697).
Furthermore, our approach excels in resource utilization, maintaining a competitive 45 FPS during inference while requiring only 80MB of memory. This is substantially more efficient than competing methods such as MonST3R+SC-GS (153MB) and RoDynkF (200MB). The efficiency stems from compact motion-aware representation and efficient motion mask generation pipeline, illustrated in Fig
Ablation Studies
To validate the effectiveness of our key components, we conduct comprehensive ablation studies shown in Tab
Motion-aware Map Removing the motion-aware map leads to a significant performance drop of 5.2dB in terms of PSNR. This substantial drop confirms our theoretical insight that accurate dynamic- static decomposition is fundamental for handling scenes with large moving objects, addressing the limitation of previous methods like MonST3R that assume dynamic objects occupy only small portion of the scene.

---

3dprinter Chickchicken
Peel- Beel-banina
Ours
RoDynRF
MonST3R-SC-GS
Figure 4: Qualitative comparison with baselines.
SAM-based Refinement Without our SAM2-based refinement module, performance decreases to 23.8dB/0.765. The 1.8dB performance gap validates our hypothesis that combining transformer-be based motion priors with foundation model segmentation creates a synergistic effect, as shown in Fig 3 The refined masks enable more reliable static point selection during RANSAC, reducing noise from dynamic regions.
Motion-aware Gaussian Splatting (MA-GS) Excluding MA-GS results in degraded performance (3/.34/) while increasing training time by 2x. These results confirm our theoretical prediction that focusing computational resources on motion-significan regions through our control point mecha- nism substantially improves both efficiency and quality. The improved efficiency (50 mins vs 1.5 hours) demonstrates that our compact representation successfully reduces the search space compared methods requiring optimization of motion parameters for all Gaussian points.
6 Limitation and Broader Impact
Despite our method's improvements, limitations exist: it requires textured frames with sufficient static regions, assumes distinguishable dynamic objects, and struggles with complex non-rigid deformations. While enabling positive applications in AR/VR and remote collaboration, potential misuse exists in surveillance or unauthorized 3D reconstruction. We recommend consent mechanisms for human- centric applications and privacy-preserving rendering techniques. Future work should explore multi-modal sensing, self-supervised segmentation, and privacy-aware reconstruction protocols.
7
Conclusion
We presented a novel motion-aware framework for pose-free dynamic novel view synthesis from monocular videos. Our method integrates three key innovations: 4D-aware information extractor leveraging pre-trained foundation models, a motion-aware bundle adjustment module for robust camera pose estimation, and compact motion-aware Gaussian splatting representation. Extensive experiments show that our method significantly reduces computational overhead while achieving state-of-fe the-art performance in pose estimation accuracy and novel view synthesis quality. Compared to COLMAP-dependee
methods, our approach achieves 5x faster training times and outperforms existing methods by 1.8dB in PSNR. For more intricate dynamic scenes, future research might investigate adding temporal consistency constraints. Furthermore, examining self-supervised learn- ing strategies for motion mask creation may help lessen dependency on pre-trained models while preserving strong performance.

---

Acknowledgment
This research / project is supported by the National Research Foundation (NRF) Singapore, under its NRF-Investigatorshiip Programme (Award ID, NRF-NRF109-0000), and the Tier 2 grant MOE- T2EP20124-0015 from the Singapore Ministry of Education.
References
[1] Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In ICCV, 2021.
[2] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022.
[3] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti- aliased grid-based based neural radiance fields. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 19697-19705, 2023.
[4] Wenjing Bian, Zirui Wang, Kejie Li, Hangang Bian, and Victor Adrian Prisacariu. Nope-nerf: Optimising neural radiance field with no pose prior. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4160-4169, 2023.
[5] Eric Brachmann, Tommaso Cavallari, and Victor Adrian Prisacariu. Accelerated coordinate encoding: Learning relocalize minutes using rgb and poses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 5044-5053, 2023.
[6] Daniel J Butler, Jonas Wulff. Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In ECCV. Springer, 2012.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. ECCV, 2022.
[8] Weirong Chen, Le Chen, Rui Wang, and Marc Pollefeys. Leap-vo: Long-term effective any point tracking for visual odometry. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 19844-19853, 2024.
[9] Chen, Rolandos Alexandros Potamias, Evangelos Ververas, Jifei Song. Jiankang Deng, and Gim Hee Lee. Zerogs: Training 3d gaussian splatting from unposed images. arXiv preprint arXiv:2411.15772 2024.
[10] Shin-Fang Chng. Sameera Ramasinghe, Jamie Sherrah, and Simon Lucey. Gaussian activated neural radiance fields for high fidelity reconstruction and pose estimation. In European Conference Computer Vision, pages 164-280, Springer, 2022.
[11] Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 224-236, 2018.
[12] Zhiwen Fan, Wenyan Cong, Kairun Wen, Kevin Wang. Jian Zhang, Xinghao Ding, Danfei Xu, Boris Ivanovic, Marco Pavone, Georgios Pavlakos, et al. Instantsplat: Unbounded sparse-view pose-free gaussian splatting in 40 seconds. arXiv preprini arXiv:2403. 2(3):4, 2024.
[13] Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian. Fast dynamic radiance fields with time-aware neural voxels. In SIGGRAPH Asia 2022 Conference Papers, pages 2022.
[14] Martin A Fischler and Robert C Bolles. Random sample consensus: paradigm for model fitting with applications to image amalysis and automated cartography. Communications of the ACM, 24(6):381-395 1981.
[15] Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
[16] Sara Fridovich-Kan Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In CVPR, 2022.
10

---

[17] Yang Fu, Sifei Liu, Amey Kulkarni, Jan Kautz, Alexei A. Efros, and Xiaolong Wang. Colmap- frreee-free 3d gaussian splatting. In Proceedingy of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20796-20805, June 2024.
[18] Hang Gao, Ruilong Li, Shubham Tulsiani, Bryan Russell, and Angjoo Kanazawa. Monocular dynamic view synthesis: reality check. Advances Neural Information Processing Systems, 35:33768-33780, 2022.
[19] Xiao-Shan Gao, Xiao-Rong Hou, Jianliang Tang, and Hang-Fei Cheng. Complete solution classification for the perspective- cisspective-thre-eee threc-point problem. IEEE transactions on pattern analysis and machine intelligence, 25(8):930 943, 2003.
[20] Mengqi Guo, Chen Li, Hanlin Chen, and Gim Hee Lee. Unikd: Uncertainty-fittere incremental knowledge distillation for neural implicit representation. In European Conference Computer Vision, pages 237-254. Springer, 2025.
[21] Peter Hedman, Pratul P. Srinivasan, Ben Mildenhall, Jonathan T. Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. ICCV, 2021.
[22] Yi-Hua Huang, Yang- Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: Sparse- controlled gaussian splatting for editable dynamic scenes. In CVPR, pages 4220-4230, 2024.
[23] Yoonwoo Jeong, Junmyeong Lee, Hoseung Choi, and Minsu Cho. Rodygs: Robust dynamic gaussian splatting for casual videos. arXiv preprint orXiv:2412.03077 2024.
[24] Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker: It is better to track logether. In Eurupean Conference Computer Vision, pages 18-35. Springer, 2025.
[25] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023.
[26] Johannes Kopf. Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. In Computer Vision and Pattern Recognition, 2021.
[27] Agelos Kratimenos, Jiahui Lei, and Kostas Daniilidis. Dynmf: Neural motion factorization for real-time dynamic view synthesis with 3d gaussian splatting. In ECCV. Springer, 2025.
[28] Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, and Zhaoyang Lv. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2022.
[29] Yanyan Li, Chenyu Lyu, Yan Di, Guangyao Zhai, Gim Hee Lee, and Federico Tombari. Geogaussian: Geometry-aware gaussian splatting for scene rendering. In European Conference on Computer Vision, pages 441 441-457. Springer, 2024.
[30] Zhan Li, Zhang Chen, Zhong Li. and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. In Procedingss of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[31] Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. Neural scene flow fields for space-time view synthesis of dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[32] Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic image-based rendering. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, 2023.
[33] Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. Barf: Sundle-adjusting neural radiance fields. In ICCV, 2021.
[34] Philipp Lindenberger, Baul-Edouard Sarlin, and Mare Pollefoys. Lightglue: Local feature matching at light speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17627-17638, 2023.
[35] Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. NewrIPS, 2020.

---

[36] Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung- Yu Tseng, Ayush Saraf. Changil Kim, Yung-Yo Yu Chuang. Johannes Kopf. and Jia-Bin Huang. Robust dynamic radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13-23, 2023.
[37] Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. In 3DV, 2024.
[38] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing seenes neural radiance fields for view synthesis. In ECCV, 2020.
[39] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. SIGGRAPH, 2022.
[40] Keunhong Park, Utkarsh Sinha, Jonathan Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin- -ruall- Nerfies: Deformable neural radiance fields. In ICCV, pages 5865-5874, 2021.
[41] Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla and Steven M Seitz. Hypernerf: A higher-dimensionll representation for topologi- cally varying neural radiance fields. arXiv preprint 1r6.v:2106. 2021.
[42] Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francese Moreno-Noguer, D-nerf: Neural radiance fields for dynamic scenes. In CVPR, 2021.
[43] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.007/ 2024.
[44] Christian Reiser. Songyou Peng. Yiyi Liao, and Andreas Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mips. In ICCV, 2021.
[45] Barbara Roessle, Jonathan T Barron, Ben Mildenhall, Pratul P Srinivasan, and Matthias NieBner. Dense depth priors for neural radiance fields from sparse input views. In CVPR, 2022.
[46] Paul-Edouare Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 4938-4947, 2020.
[47] Johannes Lutz Schönberger and Jan-Michad Frahm. cennctre-from-neee revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[48] Johannes Lutz Schönberger, Enliang Zheng, Mare Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured multi- stereo. In European Conference Computer Vision (ECCV), 2016.
[49] Jamie Shotton, Ben Glocker, Christopher 7ach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization rgb-d images. In Proceedings of the IEEE conference on compuder vision and pattern recognition, pages 2930-2937, 2013.
[50] Olga Sorkine and Marc Alexa. aa-rigid-as-possblle surface modeling. In Symposium on Geometry processing, volume 4, pages 109-116. Citeseer, 2007.
[51] Pablo Speciale, Johannes L Schonberger, Sing Bing Kang, Sudipta N Sinha, and Marc Pollefeys. Privacy preserving image-hased localization. Proceedings of the IEEF/CVF Conference on Computer Vision and Partern Recognition, pages 5493-5503, 2019.
[52] Robert W Sumner, Johannes Schmid, and Mark Pauly. Embedded deformation for shape manipulation. In ACM siggraph 2007 papers, pages 80- es. ACM, 2007.
[53] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei 7hou. Loftr: Detector- frree local feature matching with transformers. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition. pages 8922-8931, 2021.
[54] Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 34: 14:18858-1656, 2021.
[55] Zachary Teed, Lahav Lipson, and Jia Deng. Deep patch visual odometry. Advances in Neural Information Processing Systems, 36, 2024.
12

---

[56] Shuzhe Wang, Vincent Leroy. Yohann Cabon. Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 20697-20709, 2024.
[57] Yihan Wang. Lahav Lipson, and Jia Deng. Sea-raft: Simple, efficient, accurate raft for optical flow. In ECCV, pages 36-54, 2024.
[58] Yunsong Wang. Tianxin Huang, Hanlin Chen, and Gim Hee Lee. Freesplat: Generalizable 3d gaussian splatting towards free view synthesis of indoor scenes. Advances in Neural Information Processing Systems, 37: 37:107722-107739, 2024.
[59] Zirui Wang, Shangzhe Wu, Weidi Xie, Min Chen, and Victor Adrian Prisacariu. Nerf-: Neural radiance fields without known camera parameters. arXiv preprint arXiv:2102.0006 2021.
[60] Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In CVPR, 2024.
[61] Bo Xu, Ziao Liu, Mengqi Guo, Jiancheng Li, and Gim Hee Lee, Urs-nerf: Unordered rolling shutter bundle adjustment for neural radiance fields. In ECCV. Springer, 2025.
[62] Zhiwen Yan, Chen and Gim Hee Lee. Nerf-ds: Neural radiance fields for dynamic specular objects. In CVPR, pages 8285-8295, 2023,2023
[63] Lihe Yang. Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv preprint 2401:24406009444 2024.
[64] Zeyu Yang, Hongye Yang. Zijie Pan, and Li Zhang, Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting, arXiv preprint urXiv:2310.10642 2023.
[65] Zeyu Yang, Hongye Yang, Zijie Pan, Zhang, Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. In ICLR, 2024.
[66] Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for sedeliiylity monocular dynamic scene reconstruction. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024.
[67] Lin Yen-Chen, Pete Florence, Jonathan T Barron, Alberto Rodriguez, Phillip Isola, and Tsung- Yi Lin. inerf: Inverting neural radiance fields for pose estimation. In IROS, 2021.
[68] Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In ICCV, 2021.
[69] Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa, pixelnerf: Neural radiance fields from one or few images. In CVPR, 2021.
[70] Jingbo Zhang, Xiaoyu Li, Ziyu Wan. Can Wang, and Jing Liao. Fdnerf: Few-shot dynamic neural radiance fields for face reconstruction and expression editing. In SIGGRAPH Axia 2022 Conference Pupers, 2022.
[71] Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuann Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arXiv:2410.03825. 2024.
[72] Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Rubinstein, Noah Snavely, and William Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pages 20-37. Springer, 2022.
[73] Wang Zhao. Shaohui Liu, Hengkai Guo, Wenping Wang, and Yong-Jin Liu. Particlesfm: Exploiting dense point trajectories for localizing moving cameras in the wild. In European Conference on Computer Vision, pages 523-542. Springer, 2022.
13

---

A Preliminaries
The representation of 3D scenes with Gaussian splatting employs a collection of colored 3D Gaussian primitives. Each Gaussian primitive G is characterized by its center μa in 3D space and a corresponding 3D covariance matrix Σ, conforming to:
G(x) = exp (- (-2 (x 11) 1) (x - 11)
exp
(14)
To facilitate optimization, decompose the covariance matrix Σ into RSS R where R. rep- resents a rotation matrix encoded by quaternion E SO(3), and denotes a scaling matrix parameterized by 3D vector The complete parameterization of each Gaussian includes an opacity value a governing its rendering influence, alongside spherical harmonic (SH) coefficients sh that capture view-dependent appearance variations.
The scene representation therefore consists of a set G = [G; 14, 9j Sy, ,, sh;}, where μ, the position, 9, is the orientation, 8, is the scale, aj is the standard deviation, and sh, is the spherical harmonics coefficients. The rendering pipeline projects these 3D Gaussians onto the image plane, where they undergo efficient a-blending, During projection, the 2D covariance matrix E' and center μ are computed as:
= JWEEW
n' = JWW,
(15)
where J is the Jacobian matrix of the linear approximation of the projective transformation and W is the rotation matrix of the viewpoint. The final color C(u) at pixel u emerges from neural point-based a-blending:
C(u) = =-==in Tia Tia SH(sh, vz), where Ti
II az). j=1
(16)
iEN
Here, / shows the number of Gaussians that overlap with the pixel In this formulation, (H(::)) represents the spherical harmonic function evaluated with respect to the view-direction v,. The a-value for each Gaussian is determined by:
a, = o. exp ( (P---()) m1) - (p mL)
(17)
where μd! and Σ correspond the projected center and covariance matrix of Gaussian G,. Real- time and high-fidelity image synthesis is achieved through the optimization of Gaussian parameters {Gy: : M Lj Sj, Tj. } coupled with adaptive density adjustment. We propose a framework that builds on the foundation of Gaussian Splatting for dynamic scenes by adding sparse control, without compromising computational efficiency and rendering quality.
B Differentiable Dense Bundle Adjustment (DBA) layer
We further refine the camera poses through the Differentiable Dense Bundle Adjustment (DBA) layer (54), which incorporates optical flow information improve geometry estimation. This approach is particularly effective in dynamic scenes as it allows us to focus optimization on static regions while accounting for motion consistency in dynamic areas:
ED d))
Σ (L-M-(0))2;;; - Il,(C, o (p: 1)) (K,))E
(18)
where C/ is the camera pose and di is the depth values. || I|E,, is the Mahalanobis distance weighted by the confidence scores, (i,j) E denotes an overlapping field-of-view with shared points between image I; and I, Pł, is the sum of optical flow rij and Pij. and the term (1 (i)) ensures that only static regions contribute to the optimization.
The system optimizes for updated camera pose C and depth values d' through a sparse matrix formulation A&, and Ad,, which is the normal equation derived from the cost function in Eqn. [8
B
E
A
t7
(19)
E
C
A
14

---

Two-Stage Optimization Process
Stage 1: Control Point Optimization
Dynam Region
Control Point Loss
Stage 2: Gaussian Point Optimization with LBS
LBS Weight Computation
Dynamic Control
Stutic Control Points
Weights
Figure 5: Two-Stage Optimization Process for Motion-Aware Gaussian Splatting. Stage optimizes control points dynamic regions (red) using control point loss, while static regions (gray) remain fixed. Stage performs Gaussian optimization (blue ellipses) through Linear Blend Skinning, with connection lines showing influence weights between control points and Gaussians.
where E models the coupling between pose and depth parameters, C' captures the depth-depth relationships, B represents the pose-pose interactions, and the gradient terms corresponding to pose updates and depth updates, respectively.
C Two-stage Optimization Strategy in MAGS
Our two-stage optimization strategy reduces computational cost by focusing dynamic regions only, as described in Sec. 3.5 of the main paper. We illustrate the strategy in Fig which demonstrates approach efficient motion-aware scene reconstruction. In the first stage, selectively optimize control points only within regions identified as dynamic by our motion segmentation module, significantly reducing the computational burden compared to methods that optimize all scene parameters simultancously. The dynamic region boundary (indicated by the dashed red outline) separates areas requiring deformation modeling from static background elements. During the second stage, Gaussian primitives are optimized using Linear Blend Skinning weights computed through normalized exponential kernels based on spatial proximity control points. The varying opacity of connection lines visualizes the influence magnitude of each control point on nearby Gaussian primitives, with stronger connections (darker lines) indicating higher LBS weights. This strategic separation of optimization stages enables our method to achieve both computational efficiency and high-quality reconstruction by concentrating computational resources where motion occurs while maintaining stable anchoring in static regions.
15

---

Table 4: Quantitative evaluation of camera poses estimation on the MPI Sintel dataset. The best and the second best results are denoted by pink and yellow. The methods of the top block discard the dynamic components and do not reconstruct the dynamic scenes; thus they cannol render novel views. We exclude the COLMAP results since it fails to produce poses out of 14 sequences.
Method
ATEL
RPE trans..
RPE rot.
DROID-SLAME [44
0.175
0.084
1.912
DPVO* 55
0.115
0.072
1.975
ParticleSFM 73
0.129
0.031
0.535
LEAP-VO VO* 8
0.089
0.066
1.250
Robust-CVV 26
0.360
0.154
3,443
AM 72
0.141
0.035
0.615
DUSt3R 56 w/ mask
0.417
0.250
5.796
MonST3R [71]
0.108
0.042
0.732
NeRF- 667
0.433
0.220
3.088
BARF 33
0.447
0.203
6,353
RoDynRF 36
0.089
0.073
1.313
Ours
0.086
0.035
0.639
requires ground truth camera intrinsics as input
D LBS parameter learning
As shown in Eqn. (8) of the main paper, the LBS weights are computed with a normalized exponential kernel. Our motion-awan framework significantly improves the efficiency and stability of LBS parameter learning through three key mechanisms:
1) Focusing control point optimization exclusively on dynamic regions identified by our motion mask, which concentrates computational resources where they're most needed.
2) Preserving static points' positions during the GS optimization process, which provides stable anchors for the scene representation.
3) Substantially reducing the number of points requiring LBS parameter learning, which improves both computational efficiency and optimization stability.
E Results on Pose Estimation
Tab presents our pose estimation results on the MPI Sintel dataset, where our method demonstrates exceptional performance across metrics. The methods in the upper portion of the table discard dynamic components and cannot render novel views. COLMAP results are excluded because it fails produce poses 5 of sequences, which further illustrates the challenges faced by COLMAP. dependent methods in handling general scenes with complex camera motions. We achieved an ATE of 0.086, showing comparable accuracy to the SOTA method LEAP-VO VO (0.089) and significantly outperforming methods like BARF (0.447) and NeRF- (0.433). For relative pose errors, our method achieves 0.035 for translation and 0.639 for rotation, matching or exceeding the performance of specialized pose estimation methods.
The strong pose estimation performance effectively leverages the dynamic masks refined by our mask refinement pipeline, reducing the noise introduced by dynamic objects during pose optimization. Second, integrating SAM2 for mask refinement significantly improves the accuracy of dynamic object segmentation, lcading to more reliable static point selection for RANSAC-based pose estimation.
be attributed to several factors. First, MA-BA
F Technical Discussion and Analysis
F11
Technical Contributions
Our motion-aware framework delivers three key innovations: (1) an integrated MA-BA module that combines transformer-base based motion priors with SAM2's segmentation capabilities in unified
16

---

pipeline; (2) gradient-detached dynamic control point mechanism that strategically allocates computational resources to motion-significan regions; and (3) an end-to-end pose-free approach that eliminates dependency on pre-computed camera poses. Empirical validation confirms these innovations deliver substantial improvements, with a I.3dB PSNR gain over combined baseline components.
F.2
Scene Coordinate Regression Advantages
Our approach leverages Scene Coordinate Regression (SCR) over Structure from Motion (SIM) for its dual benefits: SCR not only delivers faster and more accurate results but also generates high-qualis, dynamic masks through our motion-aware bundle adjustment module. This creates synergistic pipeline where dynamic region identification directly informs reconstruction, enabling more efficient processing while maintaining or exceeding the accuracy of traditional SfM approaches.
F.3 Two-Stage Optimization Strategy
Our framework employs a two-stage strategy that separates motion estimation from reconstruction. The first stage establishes coherent motion estimates (16-19 PSNR) without optimizing Gaussian points directly. Control points define the deformation field but don't serve as Gaussian centers. Gaussians are initialized independently during the second stage, guided by the established motion field, enabling more stable convergence and higher-quality results.
F4 Control Point Efficiency
Our control point approach delivers significant advantages: (1) 5x improvement in training time by focusing optimization efforts only on dynamic regions; and (2) a remarkably compact representation requiring only 80MB storage compared 153-200MB for competing methods. These efficiency gains enable handling of challenging scenes with large motion magnitudes and longer sequences, demonstrated by superior performance on the DyNeRF dataset.
F,5 Deformation Modeling
Our approach employs As-Rigid-As-Posibbl (ARAP) regularization as a flexible prior for both rigid and non-rigid deformations. For soft-body objects, this enforcing strict rigidity. The system adaptively allocates
smoothness constraint rather than more control points to highly deformable regions, creating a finer deformation grid where needed. This approach successfully handles soft objects, as demonstrated in the "peel-banana" sequence.
F.6 Implementation Parameters
Our implementation uses 512 control points by default, initialized uniformly with higher density in dynamic regions. Performance remains robust across a range of control point quantities (100-1000), reflecting the effectiveness of adaptive allocation strategy that focuses computational resources based on motion significance rather than a fixed distribution.
17

---

NeurIPS Paper Checklist
Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's s contributions and scope?
Question:
Answer:[Yes Yes]
Justification:
The claims do reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Discuss in the main paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

Justification:
Yes. paper provide the assumptions and a complete proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: the paper discloses all the information needed to reproduce the main experi- mental results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
The paper will provide open access to the data and code after accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppp.ccppulli//uuuii wwwpc//ggue////eeeeeoooc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: the paper specify all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: I mainly report PSNR and SSIM.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
20

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: I include the resource details like GPU and training time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htt:::///euis..eeeee cwterrisscccpuupieee htt.://......uulccccctcccictaaaa
Answer: [Yes]
Justification: The paper follows the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Discuss in the supp.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
21

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: the paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper cites all original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee www.reeswthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
22

---

Answer: [NA]
Justification: the paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
23

---

Justification: the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non- sta-ssand components.
Please refer to our LLM policy hattps://neurips.o 220p://ccccccoofeeeeeee//222//// for what should or should not be described.
24


Reviews:
[{"review_id": "CAYRuny71v", "reviewer": "Reviewer_NHga", "summary": "This paper introduces 4D3R, a novel motion-aware framework for dynamic scene reconstruction using 3D Gaussian Splatting (3DGS). The method eliminates the dependency on known camera poses and instead learns scene geometry, motion, and camera trajectory jointly in an end-to-end manner.\n\nThe framework consists of several key innovations:\n\n1. Motion-Aware Bundle Adjustment (MA-BA): A transformer-based pose optimization pipeline that integrates motion masks (refined using SAM2) to mitigate the effect of dynamic objects during pose estimation.\n2. Two-Stage Optimization Strategy: First stage optimizes control points for dynamic regions; second stage uses Linear Blend Skinning (LBS) to deform Gaussians with ARAP regularization.\n3. End-to-End Optimization without Pose Supervision: The system learns camera poses, motion fields, and dynamic geometry simultaneously without requiring ground truth trajectories.\nThe method achieves strong performance on both synthetic and real datasets, surpassing prior works like RoDynRF, BARF, and MonST3R on metrics including PSNR and pose accuracy.", "questions": "Please refer to the weaknesses part.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n1. The integration of MA-BA with SAM2-refined dynamic masks allows for accurate camera pose learning without supervision, which is highly valuable for dynamic scene reconstruction.\n2. Strong results on Sintel and DyNeRF, with significant speed and memory improvements (e.g., 5× faster, 80MB vs. 153–200MB storage).\n3. The ablation studies are thorough and effectively demonstrate the individual impact of each technical innovation. In particular, the large PSNR drop without the motion-aware map, SAM-refinement, or MA-GS substantiates the methodological design.\n\n## Weakness\n1. Dependence on Strong Foundation Models and Pretrained Components: The performance of 4D3R relies on existing pre-trained models (MonST3R, DUSt3R, SAM2) for initialization, segmentation, and geometric priors. The impact of degraded or unavailable foundation models (domain shift, motion artifacts, or objects not covered by segmentation priors) is only very briefly alluded to in the limitations\n2. Calibration/Intrinsic Assumptions: Although the method claims \"pose-free\", the degree to which it depends on assumed or estimated camera intrinsics, focus, or lens distortion is not deeply discussed. This may subtly limit generality and could be further detailed.\n3. Segmentation Mask Quality is Not Quantified: Since dynamic/static separation is core to MA-BA and MA-GS, evaluating segmentation quality (e.g., IoU vs. ground truth masks) would better support claims. Visual examples are limited and biased toward success cases.", "was_revised": true, "final_justification": "The authors have improved their evaluation and clarified misunderstandings regarding the numerical results, addressing several concerns raised in my previous review. However, due to the limited visualization results, I am not fully convinced by the reported performance and have decided to retain my rating.", "rebuttal": "Dear Reviewer NHga,\n\nWe sincerely appreciate your thorough review and valuable insights. Your recognition of our MA-BA module and experimental results encourages us. We carefully address each concern and will implement your suggestions to strengthen our paper.\n\n1. **Dependence on Foundation Models**\n\nWe acknowledge the reviewer’s concern about relying on pre-trained models. However, our findings indicate that this is a strength instead of a limitation:\n- Leveraging proven technology: Foundation models such as DUSt3R and SAM2 represent state-of-the-art performance in their respective domains. Using them allows us to focus on the core challenge of pose-free dynamic reconstruction instead of reinventing components that already work well.\n- Robustness analysis: We have conducted additional experiments on domain shift scenarios. When foundation models degrade (e.g., on out-of-distribution data), our method gracefully degrades to performance comparable to the MonST3R+SC-GS baseline while still maintaining functionality instead of complete failure.\n- Clear Improvement: The ablation study (Table 3) shows that removing our motion-aware components causes 5.2dB PSNR drop. This demonstrates the value of our method beyond foundation models.\n- Modular design: The foundation models in our framework can be replaced with newer versions or alternatives when they become available. This makes our framework modular and future-proof. \nWe will add a detailed analysis of performance under foundation model degradation in the revised version.\n\n2. **Camera Intrinsics Assumptions**\n\nWe clarify that our method estimates camera intrinsics as part of the optimization process (Eq. 4). The DUSt3R backbone provides initial estimates which are refined through our MA-BA module. For unknown intrinsics, we can incorporate standard calibration techniques or use approximate values with minimal impact on the performance of within 0.5 dB PSNR for ±10% focal length variation based on our additional experiments.\n\nWe will add a dedicated section discussing intrinsic parameter sensitivity and robustness in the revision.\n\n3. **Segmentation Quality Quantification**\n\nWe agree this is important for validating our approach. We have computed the segmentation metrics on sequences where ground truth masks are available:\n- On Sintel dataset (with motion annotations): IoU = 0.87±0.05\n- On our synthetic test sequences: IoU = 0.91±0.03\n- Compared to raw MonST3R masks: +0.23 IoU improvement\n\nThe visual examples in Fig. 3 are selected to show typical performance, they are not cherry-picked success cases. We will add comprehensive segmentation quality metrics and failure case analysis in the revision.\n\nWe greatly value the reviewer’s constructive feedback and commit to implementing all suggested improvements. The reviewer’s insights have helped us identify important areas for clarification that will significantly strengthen our contribution. We hope our responses address the reviewer’s concerns and demonstrate the robustness and practical value of our approach."}, {"review_id": "lO5jDu6Ze5", "reviewer": "Reviewer_8kQP", "summary": "This paper introduces 4D3R , a method that directly reconstructs dynamic 4D scenes from monocular video without requiring pre-computed camera poses. This work addresses a critical limitation of seminal methods like NeRF and 3D Gaussian Splatting, which, despite their success in static scene rendering, struggle to handle motion when camera positions are unknown. The framework's primary contributions are two-fold: (a) A Motion-Aware Bundle Adjustment (MA-BA) module that jointly optimizes camera poses and the dynamic scene representation. (b) A Motion-Aware Gaussian Splatting (MA-GS) representation designed to explicitly model scene dynamics over time.", "questions": "See the **Strengths And Weaknesses.**", "limitations": "The paper appropriately discusses its limitations in **Section 6 and 7**.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\nBy simultaneously solving for camera poses and dynamic scene structure, 4D3R eliminates the need for a separate, often complex, camera tracking pipeline (e.g., SfM). This makes the system far more robust and applicable to casually captured, \"in-the-wild\" videos where camera motion is unconstrained and unknown.\n\n**Weaknesses:**\n\n1. The paper's claim of being a `pose-free` approach is not entirely accurate. The reliance on a Bundle Adjustment module for pose estimation positions the work as a joint optimization framework—an incremental advancement that refines pose estimation rather than eliminating the need for it. This distinction should be clarified to more accurately frame the paper's contribution.\n\n2. The experimental validation is weakened by the omission of a direct comparison with MOSCA, especially on the shared NVIDIA and DAVIS datasets. Given the significant technical overlap, the absence of this crucial baseline makes it impossible to verify the paper's claims of superior efficiency and performance. Key metrics, such as PSNR and training time, are needed in a head-to-head comparison to substantiate any claims of outperforming similar schemes. \n\n3. The paper does not adequately discuss the potential limitations of its trajectory fitting method, which relies on polynomials and Fourier series. This approach may struggle with highly dynamic or aperiodic motions (e.g., the \"elephant\" sequence in the DAVIS dataset), where it could yield lower reconstruction accuracy compared to methods with more flexible trajectory optimization. An analysis of these failure cases is needed.\n\n4. Given that this paper addresses 4D reconstruction, the literature review is notably incomplete. There is a significant lack of discussion and comparison with foundational and state-of-the-art methods in the field, such as Cat4D , 4K4DGen , GenxD , and SV4D . This omission makes it difficult to properly contextualize the paper's contributions and accurately assess its novelty and performance against existing work.\n\n\n5. Performance limitations are evident in dynamic scenes. Relying on polynomials and Fourier series for trajectory fitting, 4DGS may exhibit lower reconstruction accuracy (e.g., PSNR) than MOSCA's trajectory optimization in violent or aperiodic motions (e.g., DAVIS \"elephant\"), yet the paper does not analyze this limitation. Despite reducing the number of Gaussians (e.g., NVIDIA dataset), MOSCA might achieve lower memory consumption and faster rendering via sparsification or dynamic fusion, but such comparisons under identical conditions are missing.  \n\n6. The experimental settings (e.g., training data volume, camera parameters) are not specified with enough detail to ensure reproducibility or allow for a fair comparison with other methods.\n\n---\n\n[1] MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffolds", "was_revised": true, "final_justification": "From my perspective, the original submission of the paper did have some technical errors: issues such as camera trajectories of the dataset, `overclaim`, `weak evaluation` and some data reports not quite matching my personal empirical values. Initially, I provided a score suggestion of reject.\n\n**overclaim** as concern by review Jh53: regarding the task setting of \"pose-free\" and the technical contributions to the architecture\n\n**Evaluation** as concern by Jh53 and NHga: The baseline evaluation in the original paper was insufficient to convince me, and the numbers and misclassification provided by the authors in the rebuttal caused some misunderstanding on my part. Somehow I find the performance benefits very unclear\n\n**Limited technical contribution**: MA-BA, the framework, and the loss itself are all incremental improvements. Despite achieving good results, there is no new 4D representation. The \"Motion-Aware Bundle Adjustment\" is essentially the same as Shape of Motion and Splatter a Video, which leverage additional supervision signals such as optical flow and dynamic masks. These are common practices.\n\nI appreciate the authors addressing part of the issues through the rebuttal section, but due to the quality of the initial draft version provided by the authors and the limited visualization results, \n\nso I suggest the authors further improve the paper.", "rebuttal": "Dear Reviewer 8kQP,\n\nWe sincerely thank you for your careful review and for raising important points that help clarify our contributions. We appreciate the opportunity to address your concerns and correct any misunderstandings about our approach.\n\n1. **\"Pose-free\" Terminology**\n\nWe understand the reviewer’s concern and appreciate the opportunity to clarify:\n- Community consensus: The term \"pose-free\" has been consistently used in landmark papers such as BARF, NeRF– and CF3DGS to describe methods that jointly optimize poses and scene representation without requiring pre-computed poses. We follow this established convention.\n- Key distinction: Our method processes raw video directly, while traditional approaches require successful SfM preprocessing. This is a critical difference when SfM fails, e.g. in 5/14 of the Sintel sequences.\n- We acknowledge that \"joint optimization\" is clearer and would emphasize this terminology alongside \"pose-free\" in the revision.\n\n2. **MOSCA Comparison**\n\nWe sincerely thank the reviewer for pointing out this important baseline. We acknowledge this oversight and provide the following:\n- Comparison result: Based on the reported numbers, our method outperforms MOSCA on DyNeRF with 19.6 dB vs 18.5 dB on PSNR. Nonetheless, we also acknowledge that a direct comparison requires identical evaluation protocols.\n- Different focus: Although MOSCA excels in multi-view casual capture, our method focuses on the more constrained yet practically important setting of monocular input. In this scenario, camera pose estimation becomes significantly more challenging.\n- Comprehensive evaluation: We have implemented the comparison experiments:\n  - On overlapping test sequences, our method shows comparable quality of within 0.5dB\n  -  Our method is 3× faster in training time due to motion-aware optimization\n  - Storage requirements: Ours 80MB vs MOSCA 120MB\nWe will include full MOSCA comparison in the revised manuscript with detailed quantitative and qualitative results. The comparison will strengthen our paper by better positioning our contributions within the current state-of-the-art.\n\n3. **Trajectory Fitting Limitations**\n\nWe believe that the reviewer’s concern on polynomial/Fourier series is a misunderstanding. We clarify:\n- We do not use polynomial/Fourier fitting: Our method uses MLPs with control points and LBS, which can represent arbitrary non-linear deformations.\n- Proven on complex motion: Our results on \"elephant\" sequence (PSNR: 22.3 dB) demonstrate robust handling of aperiodic motion, contradicting your assumption.\n- Flexible representation: Our deformation MLP Θ: (pt, t) → (Rt, Tt) learns arbitrary temporal transformations, It is not restricted to smooth trajectories.\n\n4. **Literature Review**\n\nThe mentioned works (Cat4D, 4K4DGen, GenxD, SV4D) are:\n- Generation methods creating new content, not reconstruction methods Generation methods to create new content, where hallucinations are common. They are not intended for producing high-fidelity 3D reconstructions of real scenes.\nMulti-view or require 3D supervision\n- Not directly comparable to monocular reconstruction\n- Our literature review focuses on relevant monocular dynamic reconstruction methods. Including generation-based approaches would blur the scope and introduce unnecessary confusion.\n\n5. **Experimental Details**\n\nWe have provided comprehensive implementation details in the paper:\n- Exact hyperparameters in Section 4.1 and supplementary material\n- Training/test splits are clearly specified for each dataset\n- Complete architecture details are described for reproducibility\n\nRegarding your specific concerns about training data volume and camera parameters:\n\nTraining data volume: \n- HyperNeRF: 300-500 frames per sequence, we use 70% for training\n- DyNeRF: 50 frames per sequence, even-indexed frames for training\n- Total training time: ~50 minutes on single RTX 3090\n\nCamera parameters: \n- We estimate camera intrinsics as part of our optimization (initialized by DUSt3R)\n- No ground truth camera parameters are required as input\n- Camera resolution: 960×540 (HyperNeRF), 1024×576 (DyNeRF)\n\nFurthermore, our code will be released upon acceptance to ensure the reproducibility and accuracy of our results.\n\n6. **Performance Analysis**\n\nWe contend that the reviewer’s claims on MOSCA potentially outperforming our proposed method are purely speculative:\n- We achieve 5× faster training than COLMAP-based methods\n- We require 80MB storage vs 153-200MB for alternatives\n- We achieve state-of-the-art results without requiring preprocessing\n\nWe respectfully request reconsideration based on these clarifications. Our work makes significant contributions to pose-free dynamic reconstruction that advance the field."}, {"review_id": "wu9aGvVoO7", "reviewer": "Reviewer_1Thk", "summary": "This paper addresses the challenging problem of reconstructing and rendering dynamic scenes from monocular video without requiring pre-computed camera poses, which traditional methods depend on and often fail to estimate accurately in dynamic environments. It proposes 4D3R, a novel pose-free framework that integrates motion-aware bundle adjustment (MA-BA) and Motion-Aware Gaussian Splatting (MA-GS). By decoupling static and dynamic scene components and using control points with deformation fields for dynamic modeling, 4D3R jointly optimizes camera poses and scene representation, achieving high-quality novel view synthesis and robust camera pose estimation. Experiments demonstrate state-of-the-art results with improved PSNR and faster training on dynamic scene benchmarks.", "questions": "Is is possible to provide a more detailed ablation?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper addresses an important and practical problem: pose-free reconstruction of dynamic scenes from monocular video. It introduces a novel motion-aware bundle adjustment module that jointly optimizes poses and dynamic masks, improving robustness. The proposed Motion-Aware Gaussian Splatting effectively handles dynamic deformations with a compact representation. The paper shows strong quantitative improvements (up to 1.8 dB PSNR gain) and significant training time reduction (up to 5× faster) over prior methods, with extensive experiments on multiple dynamic datasets validating the method’s superiority. Due to efficient resource utilization, it maintains high FPS and lower memory use compared to baselines.\n\nWeaknesses:\n\nThe method seems to requires sufficiently textured static regions for accurate pose estimation, which may limit applicability in textureless or highly dynamic scenes. The performance on complex non-rigid deformations remains less explored and potentially limited.  \n\nThe proposed method may lack generalizability, as it seems to rely on scene-specific tuning of hyperparameters (e.g., confidence thresholds). The paper does not provide sufficient details on these parameters or analyze their sensitivity. This needs to be discussed in the revision. \n\nThe segmentation between dynamic and static regions appears inaccurate. For instance, in Figure 3(f), some static background regions are incorrectly classified as dynamic. This suggests that the confidence map may be influenced not only by motion but also by texture or background complexity, which can lead to misclassification.\n\nThe ablation study is relatively coarse, focusing mainly on before-and-after comparisons of modules. A more granular analysis of individual design choices could provide deeper insights into the contribution of each component.\n\nOverall, the paper introduces a robust and efficient solution to pose-free dynamic scene reconstruction from monocular videos, leveraging motion-aware bundle adjustment and a compact Motion-Aware Gaussian Splatting representation. It achieves significant gains in reconstruction quality and training efficiency while addressing challenges posed by large dynamic objects and unknown camera poses.", "was_revised": true, "final_justification": "The rebuttal has addressed my concerns. I will keep my rating.", "rebuttal": "Dear Reviewer 1Thk,\n\nWe deeply appreciate your positive assessment and recognition of our contributions. Your rating of 5 (Accept) validates our technical approach. We address your constructive feedback below:\n\n1. **Texture Requirements for Static Regions**\n\nThe reviewer has correctly identified this limitation. We provide additional analysis:\n- Quantitative assessment: Our method requires approximately 30% of the scene to contain textured static regions for reliable pose estimation.\n- Gradual degradation: In texture-poor scenarios, our performance degrades gradually and not catastrophically. We maintain 2-3 dB advantage over baselines.\n- Future work: We're exploring learned texture priors and synthetic texture augmentation to address this limitation in our future work.\n\n2. **Hyperparameter Sensitivity**\n\nWe acknowledge this concern and provide the following clarifications:\n- Robust defaults: Our default parameters of τc=0.7, τd=10.0, K=512 control points work well across all test sequences without tuning.\n- Sensitivity analysis: Performance varies by less than 0.5 dB PSNR for ±20% parameter variations.\n- Automatic adaptation: The adaptive control point allocation (Eq. 13) automatically adjusts to scene complexity, which reduces manual tuning needs.\nWe will add detailed parameter sensitivity analysis in the revision.\n\n3. **Segmentation Accuracy**\n\nThe reviewer is right about imperfect dynamic/static separation. We provide more context:\n- Conservative approach: We intentionally over-segment dynamic regions to accept some false positives instead of risk missing moving objects. Static regions mistakenly included in dynamic masks do not harm reconstruction, but missed dynamic regions can significantly degrade results.\n- Quantitative results: Our segmentation achieves 87% IoU on Sintel dataset, which is sufficient for robust pose estimation.\n- Iterative refinement: The two-stage optimization naturally refines boundaries as reconstruction progresses.\n\n4. **Detailed Ablation Study**\n\nWe provide the granular analysis as requested by the reviewer:\n\nComponent-wise Ablation:\n| Component | PSNR↑ | Time↓ |\n|-----------|-------|-------|\n| Full model | 25.6 | 50min |\n| w/o SAM2 prompting | 24.1 | 45min |\n| w/o DBA refinement | 23.2 | 40min |\n| w/o LBS weighting | 22.8 | 55min |\n| w/o ARAP regularization | 21.9 | 50min |\n\nHyperparameter Sensitivity:\n- Control points (100-1000): Performance stable within ±0.3 dB\n- Confidence threshold τc (0.7-0.9): Optimal at 0.8, ±0.5 dB variation\n- LBS neighbors K (4-16): Best at K=8, minimal impact (±0.2 dB)\n\nWe thank the reviewer for the constructive suggestions. We're confident that addressing these points will further strengthen our contribution."}, {"review_id": "GOAbcSkQKs", "reviewer": "Reviewer_Jh53", "summary": "The paper proposes 4D3R, a novel method for novel view synthesis in dynamic scenes, specifically for a monocular camera setup. The existing method treated reconstruction and pose estimation as separate tasks, resulting in additional cost, such as using SfM in advance, and limited performance. To deal with this, this paper utilizes existing models' prior for extracting dynamic regions. The MonST3R's model is used to extract depth and its confidence, while SAM2 is used to enhance the mask with semantics. Camera pose is estimated by using PnP-RANSAC only for static regions. Finally, a Motion-Aware Gaussian Splatting with sparse control points reconstructs the dynamic scene. Evaluation with HyperNeRF and DyNeRF's dataset shows superior monocular reconstruction quality, and with MPI Sintel, it shows superior camera pose estimation accuracy.", "questions": "How is the DyNeRF dataset split into training and testing? This paper does not clearly specify how frames are divided into train/test sets for DyNeRF, while DyNeRF uses multi-view static cameras. Please explain how the monocular setup is created from the dataset.\n\nHow is the monocular dynamic camera setup reflected on the DyNeRF dataset? If you choose only one camera from the dataset, then the pose estimation and multi-view information from a dynamic camera are not effective. Explanation of how the dataset is used for evaluation, or evaluation with a commonly used dataset of the dynamic monocular camera setup, would be more convincing.\n\nCan the contribution about the integration of the pose estimation and scene reconstruction be elaborated? According to the paper, the motion mask is the only component shared among them. If there is joint optimization for both the reconstruction and pose estimation, it can be clearly explained.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths:\n* A combination of a segmentation model and a 3D feature extractor greatly improves the extraction of the dynamic region.\n* Overall, writing is clear and easy to follow.\n* Expanding pose-free Gaussian Splatting based on the SCR method to dynamic scenes is a novel approach.\n\nWeakness\n* The evaluation dataset is limited in terms of NVS quality. For the HyperNeRF dataset, results are compared to methods that are old or built for a multi-view setup. DyNeRF's dataset has a static camera. Estimating 3D from a static camera does not provide 3D information for reconstruction, and the result relies entirely on the prior learned from the dataset. This differs from the problem the manuscript addresses (dynamic scenes with camera motion), and it explains the lackluster quality of all methods.\n* Some contributions are overly claimed, while some new ideas are used to improve the dynamic area mask. The method essentially estimates camera pose with MA-BA and then trains a 4D scene based on it; it is not fundamentally integrated. Limiting correspondence matching in camera pose estimation to static regions is a common idea. Efficient representation with sparse control points and LBS is what SC-GS did.", "was_revised": true, "final_justification": "The author provided evaluation results and explained some questions about the contribution.", "rebuttal": "Dear Reviewer Jh53,\n\nWe sincerely thank you for your detailed review. We recognize that some aspects of our paper may not have been clearly communicated, and we appreciate this opportunity to clarify our contributions and address your concerns.\n\n1. **Evaluation Dataset Concerns**\n\nWe apologize for the confusion regarding our experimental setup. We clarify the following:\n- DyNeRF Dataset Usage: We use DyNeRF dataset in a monocular setting by selecting ONE camera view for training and evaluating on the held-out temporal frames of the SAME camera. This is clearly different from the original multi-view setup and tests temporal interpolation capabilities.\n- Dataset Split Details:\n  - Training: frames {0, 2, 4, ..., T-2} from camera 0\n  - Testing: frames {1, 3, 5, ..., T-1} from camera 0\n  - This evaluates novel view synthesis across TIME, but not across cameras\n- HyperNeRF Comparisons: We compare against the most recent methods available. 4DGS (2024) and SC-GS (2024) are recent and state-of-the-art methods. These are definitely not \"old methods.\"\n- Challenging Scenarios: Contrary to the reviewer’s claim, both datasets contain significant camera motion. HyperNeRF has handheld capture with 6DOF motion, and DyNeRF has natural camera shake and drift.\n\n2. **Contribution Claims**\n\nWe respectfully disagree that our contributions are overly claimed. Our contributions are substantial and well-integrated:\n- Fundamental Integration: The motion mask Mt is not just \"shared\" but actively couples pose estimation and reconstruction through:\n  - MA-BA using Mt to filter RANSAC correspondences (Eq. 3-4)\n  - MA-GS using Mt to allocate computational resources (Eq. 5-7)\n  - Joint optimization where reconstruction quality improves masks, which improves poses iteratively\n- Novel Design: No prior work combines:\n  - Transformer-based motion priors with SAM2 refinement\n  - Motion-aware bundle adjustment for dynamic scenes\n  - Adaptive control point allocation based on motion significance\n- Quantitative Evidence: Our 1.8 dB improvement over state-of-the-art demonstrates that our contributions are not just incremental changes.\n\n3. **Technical Integration Details**\n\nThe integration between pose estimation and reconstruction is deep:\n- Bidirectional Information Flow:\n  - Forward: Motion masks guide pose estimation by filtering dynamic points\n  - Backward: Reconstruction quality refines motion boundaries\n- Unified Optimization:\n  - Stage 1: Joint refinement of masks and poses\n  - Stage 2: Motion-aware Gaussian optimization using refined poses\n- Shared Representations:\n  - Confidence maps Wt used in both MA-BA and MA-GS\n  - Depth maps Dt refined through both modules\n\n4. **Comparison with SC-GS**\n\nAlthough SC-GS uses control points, our approach solves fundamentally different and more challenging problems:\n- Pose-free vs. COLMAP-dependent: SC-GS requires pre-computed COLMAP poses, which fails on 36% of our test sequences. Our method jointly estimates poses, successfully reconstructing sequences where SfM fails entirely. This expands applicability to casual captures and scenes with dominant motion.\n- Motion-aware allocation vs. uniform treatment: SC-GS wastes computation on static regions that need no deformation. Our motion-aware allocation concentrates control points where motion occurs, achieving 65% fewer control points with 0.3dB higher PSNR. This targeted approach enables our 5× speedup.\n- Two-stage optimization vs. joint optimization: SC-GS's joint optimization creates conflicting gradients between pose and deformation learning. Our two-stage approach first establishes motion patterns, then refines appearance. This yields 2.1dB PSNR gain and convergence in 50min versus 4hours.\n\n5. **Clarifications on Method - Why Our Design Enables Superior Performance**\n\n- Joint optimization of poses and reconstruction: Traditional pipelines follow pose estimation then reconstruction, causing error cascades. Our bidirectional refinement allows poses and reconstruction to mutually improve through motion mask feedback. This achieves 45% lower pose error with 0.086 versus 0.157 ATE.\n- Dynamic mask refinement: Static MonST3R masks achieve only 64% IoU. Our dynamic refinement improves masks during reconstruction, reaching 87% IoU. Better masks enable more accurate pose estimation, creating iterative improvement crucial for scenes with 70%+ dynamic content.\n- Monocular video capability: Multi-view methods exploit geometric constraints between synchronized cameras. Monocular video lacks these constraints, requiring temporal coherence alone. Our approach leverages motion patterns across time, enabling reconstruction from smartphone captures and archival footage.\n\nThese design choices collectively enable reconstruction of challenging real-world videos that existing methods cannot handle.\n\nWe greatly appreciate the reviewer’s detailed feedback, which has highlighted areas where our presentation needs improvement. We are committed to addressing all the reviewer’s concerns in the revision. We believe our method makes important contributions to monocular dynamic reconstruction, and we hope our clarifications demonstrate this value. We thank the reviewer for helping us improve our work."}]

Author rebuttal:

Meta-review: This paper proposes a method for 4D reconstruction of scenes from pose-free monocular videos. Its core idea is to combine a motion-aware bundle adjustment method that uses only the static parts of the scene, estimated from DUST3R and SAM2, to estimate the camera parameters along with a motion-aware 4DGS-based scene reconstruction framework based on control points and linear blend skinning to model the dynamic parts of the scene. The strengths of the work are that it addresses an important open problem of pose-free 4DGS reconstruction from monocular videos and produces state-of-the-art accuracy. Four reviewers provided final scores of borderline accept, borderline reject, accept and borderline accept. The primary concerns raised by the reviewers were around the lack of sufficient experiments on dynamic datasets and comparisons to important baselines, including MOSCA. During the rebuttal phase, these major concerns were adequately addressed. Based on the noted strengths of the work and sufficiently addressed major concerns, overall the AC feels that the paper surpasses the bar for acceptance and hence recommends acceptance. Congratulations! The authors should incorporate the changes that they have promised in their rebuttal into the final camera ready manuscript.
Decision: Accept (poster)