Enhancing 3D Reconstruction for Dynamic Scenes
Authors: Jisang Han
Honggyu An
Jaewoo Jung
Takuya Narihira
Junyoung Seo
Kazumi Fukuda
Chaehyun Kim
Sunghwan Hong
Yuki Mitsufuji
Seungryong Kim
Venue: NeurIPS 2025 poster ()
Keywords: 3D Reconstruction
4D Reconstruction
Primary area: applications

Abstract: In this work, we address the task of 3D reconstruction in dynamic scenes, where object motions frequently degrade the quality of previous 3D pointmap regression methods, such as DUSt3R, that are originally designed for static 3D scene reconstruction. Although these methods provide an elegant and powerful solution in static settings, they struggle in the presence of dynamic motions that disrupt alignment based solely on camera poses. To overcome this, we propose D$^2$USt3R that directly regresses Static-Dynamic Aligned Pointmaps (SDAP) that simultaneiously capture both static and dynamic 3D scene geometry. By explicitly incorporating both spatial and temporal aspects, our approach successfully encapsulates 3D dense correspondence to the proposed pointmaps, enhancing downstream tasks. Extensive experimental evaluations demonstrate that our proposed approach consistently achieves superior 3D reconstruction performance across various datasets featuring complex motions.

Enhancing 3D Reconstruction for Dynamic Scenes
Jisang Han
Honggyu An
Jaewoo Jung'
Takuya Narihira²
Junyoung Seo
Kazumi 2ukuda²
Chachyun Kim
Sunghwan Hong* Yuki Mitsufuji2.41 Seungryong Kim
KAIST AI
Sony AI
ETH Zürich AI Center, CVG, PRS
Sony Group Corporation
httpa://cvlab kaist. github io/DDSS
Stalic-anly Aligned 5D Pointrop
Pontrey
Takerap
Dynamic imput views and comreponning points
DUSI3R
MonST3R
Maate-Dynamic Aligned 3D Pontmop
D'US (Ours)
Figure 1: Teaser. Given pair of input views, D accurately establishes dense correspondence not only in static regions but also in dynamic regions, and enables 3D reconstruction of dynamic scene via our proposed static-dynamic aligned pointmap. The colored pointmaps highlight that DUSt3R [44] and MonST3R [5I] align pointmaps solely based on camera motion, causing corre- sponding 2D pixels within dynamic object misaligned. We also compare the coss-attention maps. established correspondence fields, and estimated depth maps produced by D2US13R against baseline methods
51
where our method shows higher precision.
Abstract
In this work, we address the task of 3D reconstruction in dynamic scenes, where object motions frequently degrade the quality of previous 3D pointmap regression methods, such as DUS(3R, that are originally designed for static 3D scene recon- struction. Although these methods provide an elegant and powerful solution static settings, they struggle in the presence of dynamic motions that disrupt align- ment based solely on camera poses. To overcome this, we propose US$3R that directly regresses Static- Dantm- Aligned Pointmaps (SDAP) that simultaneiously
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

capture both static and dynamic 3D scene geometry. By explicitly incorporating both spatial and temporal aspects, our approach successfully encapsulates 3D dense correspondence to the proposed pointmaps, enhancing downstream tasks. Exten- sive experimental evaluations demonstrate that our proposed approach consistently achieves superior 3D reconstruction performance across various datasets featuring complex motions.
1 Introduction
Recovering 3D scene geometry from images remains a core problem in computer vision. Traditional
approaches, such as crntture-foo--Motionn (SIM) [33 and Multi-View Stereo (MVS) [35], have achieved impressive results in this context. While these methods are originally designed for recovering precise 3D scene geometry, they often struggle with scenes that include objects, symmetries, imagery
with minimal overlapping and textureless regions [26
34
3 9
To overcome these limitations, recent approaches
44
21 41 leverage learning framework to
streamline the 3D reconstruction pipeline and enhance performance. DUSt3R [44], as a pioneer-
ing method, introduced unified learning-based framework for dense stereo 3D reconstruction. Specifically, DUS(3R directly regresses 3D pointmaps that encode scene geometry, pixel-to-scene correspondences, and inter-view relationships, mitigating error accumulation typical in multi-stage pipelines. Despite its strengths in static scenarios, DUS(3R significantly struggles with dynamic scenes due its rigidity assumption, as exemplified in Figure
Dynamic scenes, prevalent real-world scenarios, pose significant challenges 3D scene recon- struction task, object motions disrupts the camera pose- baseb alignment [44), causing misaligned correspondences and inaccurate depth estimates in dynamic objects, which further degrades recon- struction accuracy in static regions. While recent methods such as MonST3R [5]] extends training to dynamic-scene video collections to account for dynamic objects, it still models all pointmaps as if generated by single global rigid transformation. Consequently, these approaches suffer from compromised correspondence learning for dynamic objects, in turn impairing depth accuracy and robust geometry recovery.
In this paper, we propose Dynamic Dense Unconstrained Stereo 3D Reconstruction (D*US13R), a novel feed-forward framework that directly regresses Static- Aligned Pointmaps (SDAP), simultaneously accounting for both spatial structures and temporal motions to enable more reliable 3D reconstruction of both static and dynamic regions. Unlike MonST3R [51], which overlooks correspondences on moving objects, our model captures dense inter-framm matches by treating unified problem in dynamic scenes. We achieve this with
correspondence and reconstruction novel training scheme that applies separate supervisory signals to static and dynamic regions, signals that are further stabilized and localized to regions of interests by occlusion and dynamic masks. Experimental results and visual comparisons confirm that our approach delivers a significant boost in reconstruction accuracy.
Our contributions are summarized as follows:
Our approach, DRUR, captures dynamic motion by leveraging static dyniii- aligned pointmaps, allowing for the comprehensive 3D reconstruction of all scene elements in any environment.
To compensate for the missing direct 3D correspondences between dynamic objects, we propose 3D alignment loss that effectively accounts for the occlusions and object motions.
D²USt3R achieves state-of-the-art performance across several downstream tasks, including multi-frame depth estimation as well as camera pose estimation, demonstrating superior results in dense 3D reconstruction of dynamic scenes.
2 Related Work
Per-scene 3D reconstruction. Classical 3D reconstruction methods typically follow a multi- stage pipeline to recover scene geometry and camera parameters from a set of uncalibrated images. Prominent examples include Structure-from-MMttion (SIM) [52), and Simultancous Localization and

---

Dynecrio
Target
Source
Layer O
Latyrr 4
Layer 6
Layer 8
Average
Figure 2: Cross-attention visualization DUS13R [44] on static vs. dynamic scenes. We show cource-image attention maps for highlighted query point (red) at each layer and averaged across layers. While DUSt3R captures geometric correspondences well in static regions, it fails in dynamic areas due its rigid-motion,
assumption.
MonS1
DIM
DM
M
0M
9M9
00
9M
Oux
Layer
4
Layer
Layer 8
Average
Figure 3: Cross-attention visualization on dynamic scenes: MonST3R 51 vS. Ours. Using
the same setup Figure 2 MonST3R inherits DUSt3R's static-scene supervision and fails to align moving regions, limiting reconstruction. In contrast, our method consistently matches dynamic frames and produces sharply localized attention.
Mapping (SLAM) 8 SfM incrementally reconstructs sparse 3D points through feature matching and bundle adjustment where SLAM simultaneously estimates camera trajectories and builds sparse or semi-dense maps in real-time. Building upon SíM, ParticleSIM [53] incorporate particle-based trajectory modeling track object motions, paving the way for reconstruction in dynamic scenes. Despite all of these approaches showing remarkable performance, they typically require long process- ing time and resources for scene-specifie optimization and often struggle with error accumulations from multi-stage pipelines.
Learning-basee static scene reconstruction. Building top of estabilished correspondence fields [4] 5 reconstruct static 3D scenes by learning strong 3D priors, representing the scene as point clouds [25
17 18 [1], various learning-bast based approaches [41] [5] 12 37 [6] have been proposed to
13). meshes 112 42), voxels [37 6 and 3DGS [15). Recently, DUSt3R [44] notably provides a unified, feed-forward pipeline for dense stereo matching, geometry estimation, and triangulation by directly regressing structured 3D pointmaps. Although DUSt3R significantly improves reconstruction quality and efficiency by reducing cumulative errors, DUSt3R inherently designed for static scenes, limiting its effectiveness in scenarios involving dynamic components.
Learning-based dynamic scene reconstruction. Dynamic scene reconstruction introduces ad- ditional complexities due to non non-rigid transformations occurring across frames. Similarly to static scene reconstruction, several recent approaches [23] 28 24
43 48 have employed learning- based methods tackle dynamic scene reconstruction. Among these, MonST3R [5T] directly fine-tunes DUSt3 using videos consisting of dynamic scenes. Although it has shown competitive performance, MonST3R retains DUSt3R's per-frame training paradigm and lacks explicit mechanisms for linking corresponding points across frames in dynamic scenes, leading inconsistent depth estimations that arises from lacking constraints that fail to capture the intricate motion patterns of objects. Our method addresses this limitation by augmenting the existing feed-forward framework with motion-aware training objectives, explicitly enforcing consistent 3D point correspondences over time.
3 Preliminary
Given a pair input images 12, 12 Wxx DUSt3R 44
predicts a pair 3D pointmaps
X1.1 X E R. R x for both images, each expressed in the camera coordinate system of /t, To supervised manner, ground-truth pointmaps for each image are defined in the
train the network coordinate space of the first camera. Specifically, given the camera intrinsics matrix K g3x3

---

Image/"
Consistncyy
Static region
Edyn
Image
Optical flow
Dynamic mask Maya
Dynamic region
(a) Dynamic mask eslimation
(b) Pointmap alignment
Figure 4: Construction alignment loss defined at static and dynamic regions. We introduce pipeline for constructing static-dynamic aligned pointmap (SDAP) that explicitly handles occlusions in dynamic regions. First, we compute and refine optical flow via cycle consistency checks and derive dynamic mask Mayn To align image 2 with image I then: I) warp static pixels using the known camera poses, and 2) warp dynamic pixels using the optical flow. By combining these two warps, finally obtain SDAP that registers every corresponding 2D pixel into 3D space.
world-to norld-to-camera pose matrices P", pm Pm D D W the ground-trute pointmap is computed X pm (Pm) h((K b(K* KalKK-lpp DD), where
R4x4 for images and and ground -truth truth depth map
h: (2.3,2) + (x, y, 1) represents the transformation to homogeneous coordinates.
Using 3D pointmaps, DUSt3R learns its parameters by minimizing the Fuclidean distance between the
ground-truth pointmaps X1.1 2 and the predicted pointmaps X1.1, X2.1 for two corresponding sets of valid pixels D D2 (1.... (1 H}
which ground-truth defined using regression
loss defined as:
Cierr 1. i) = x
where v E {1,2) is the input views and i D denotes valid pixel positions. The scaling factors
norm(1 x2.1) and v norm(X 1,1 ,22,,1 are computed using the normalization function:
norm X x²)
Σ Σ |xp||
D
2E/1,2)1ED
Additionally, DUSt3R incorporates confidence score learn reject errorneously defined GT pointmaps, and it included the final loss function such that:
Lcont
Σ rE(1,2)
Σ
C (v, i) a log C.
(3)
Methodology
4.1 Motivation and overview
As shown Figure T DUSt3R 44 predicts 3D pointmaps in static regions with high accuracy, leveraging precise stereo correspondences for robust reconstruction. Inspired by ZeroCo's finding that DUSt3R's 's cross-attention inherently encodes geometric correspondences [Л] we visualize attention maps for both static and dynamic scenes in Figure 2 While DUS(3R excels in static areas. it fails to produce localized attention scores in dynamic regions, lcading to noisy pointmap predictions as visualized in Figure To address this, MonST3R [1]] augment DUS(3R'S training data with dynamic sequences, but they still neglect explicit object-to-object correspondences. This omission prevents dynamic objects from serving anchors that could strengthen the spatial structure and improve depth estimates of neighboring static regions, thereby limiting overall reconstruction quality. Figure highlights these shorteomings, showing how noisy missing correspondences degrade accuracy.
To overcome these limitations, we propose Static-Dynamic Aligned Pointmaps (SDAP). Unlike static-only aligned pointmaps, which align 3D points solely with rigid transformations, SDAP

---

Table 1: Training datasets. All datasets consists of synthetic scenes and provide both camera and depth. We excluded scenes containing incomplete dataset annotations or noisy objects (e.g., smoke). Details regarding each scene are documented the supplementary material.
Domain
# frames
# Scenes
Dynamies
Dynamic mask
Optical flow
Blinkvision Outdoor
32
Ouldoors
6k
23
Realistic
38.75%
Blinkvision Indoor 22
Indoors
24
Realistic
23.75% V
PointOdyssev 54
Indoors & Outdoors
200k
131
Realistic
X 12.5%
TartanAir (45
Indoors & Outdoors
1000k
163
None
/
12.5%
Spring
Outdoors
37
Realistic
12.5%
simultaneously aligns both static and dynamic components in the scene, fully encoding each pixel's spatial and temporal information. In the following sections, we describe our proposed SDAP representation and outline our training procedure.
4.2 Static-Dynamic Aligned Pointmap
While 3D pointmaps excel encoding static 3D structure 44 they break down moving objects due to temporal misalignment. To address this, we introduce Static-Dynamic Aligned Pointmaps (SDAP), which jointly enforce spatial consistency and capture pixel motion across time. Our key insight is to represent every pixel in a unified world coordinate system that aligns its position at each timestep. yielding a cohesive reconstruction of dynamic scenes. However, dynamic motion often leads to occlusions, meaning some pixels appear in only one frame. This results in incomplete alignment, which ultimately limits the quality 3D reconstruction.
Occlusion Masks. To eliminate occluded regions in SDAP representation, we first obtain dense 2D correspondences across each image pair using an off-the-shelf optical flow estimator [46]. If ground-truth flows available in the dataset, use those instead. In scenarios with large camera baselines, where occlusions become more frequent, we further apply a to derive precise occlusion masks. The full procedure is detailed below.:
consistency
check 39 50
212 p1 f(P1), pi p/2 b(p2); Mace
IPn' p1 t).
where P; denotes a pixel in image 1t, and represent forward and backward optical flows, respectively, and is occlusion threshold.
Dynamic Masks. Dynamic regions, characterized by moving objects and non-rigid transformations, introduce significant challenges accurately aligning pointmaps due to inconsistencies between camera-induced motion and object-specific motion. Without explicitly accounting for such dynamic behaviors, the network may attempt to incorrectly fit dynamic regions as if they were static, thus impairing reconstruction accuracy. To address this, we introduce dynamic mask Mayn that explicitly highlights moving regions, guiding the network toward a more stable learning process. Specifically, the dynamic mask Mäyn is computed by comparing optical flow f with the expected flow induced purely by camera motion feam- Given the depth map D, intrinsics matrix K, relative rotation and translation R, T, and pixel coordinates p, we define:
f.om TDKKK
KT) p, Mayn fllfaam fl| T],
(5)
where म (x. y, 2) + (x/2,0/z) is the projection operation, and serves as the dynamic threshold.
4.3
Objective Function
Given SDAP representation equipped with occlusion and dynamic masks, formulate training objective as confidence-aware 3D regression [44) leveraging these masks to achieve enhanced stability. To effectively account for the distinct characteristies of static and dynamic regions, introduce two separate objective functions, each explicitly designed to handle the respective scenarios and ensure accurate reconstruction in both cases.
Pointmap alignment in static regions. The regression loss in DUSt3R inherently aligns 3D pointmaps using camera pose alone. To ensure alignment focuses solely on static regions within image 12, we employ the dynamic mask Mayn and restrict the computation of the regression loss
5

---

accordingly. Thus, we modify the regression loss Logg as follows:
(1egr(1,8)
X 1,1
1
1,1
(6)
(2, (1 May kyn.:)
x2
X 2.1
To account for errorneously defined GT pointmaps, we additionally introduce a confidence-aware loss static regions, Saatii to incorporate uncertainties into the alignment process:
Ssattic
EEG
C% (, log C,
vE(1.2)
Pointmap alignment in dynamic regions. To align dynamic region from 1 to 1 we introduce dynamic alignment loss that leverages both the occlusion mask Mace and the dynamic mask Mayn to effectively address occlusions and motion. As illustrated in Figure 4 these masks are computed in dedicated pipeline. This method ensures that when points from the second view 1², are transformed into the coordinate system of / they accurately correspond to the temporal state of the first view. In line with our regression loss, we further incorporate confidence estimates to define a confidence-aware alignment loss. The dynamic alignment loss is formulated follows:
Ldyn Σ IED2
Ma Moc..
)..
1
1,1 b+b(1)
1
x²²
a log C,²¹!
11
21
(8)
Σ iED!
M (1
M Mocc
C.!
1 2.2 (+(())
x!!2 2/
log C.!²?
22
We leverage optical flow f to establish dense correspondences between 1 and 12, and its reverse direction b. The first term in our loss function enforces the alignment of the pointmap in the coordinate space of the camera system associated with /2, while the second term introduces symmetric constraint by aligning the points when the roles of the views are swapped. Final Objective. Our final objective function is defined as following:
Liotal =
Lstatic + Ldyn-
(9)
This combined loss function enables our model to capture precise 3D geometry and robust correspon- dences in dynamic scenes while retaining DUSt3R's 's proven benefits in static regions.
4.4 Additional heads for downstream task
To further enhance the capabilities and interpretability of our SDAP framework, we introduce two additional downstream heads dedicated to explicitly modeling dynamic masks and optical flow.
Dynamic mask head. Since our model implicitly encodes regions corresponding to dynamic motion, we explicitly regress dynamic masks using an additional head. Specifically, we predict single-channel logit map, Mayn- using DPT head [32], structured similarly to our pointmap regression head. We supervise this head using binary cross-entropy loss defined follows:
Lyask
[Danl
Mayn.s log(a (Maa,.)) + (1 Mayn,;) log o (Mayy:)))
(10)
VED
where (.) denotes the sigmoid function and Dai denotes the set of all pixels.
Optical flow head. To accurately estimate optical flow, incorporate additional head based on the RAFT architecture [40]. Inspired by recent findings from ZeroCo (1. our flow head utilizes cross- attention maps instead of traditional 4D correlation volumes. The optical flow head is supervised using the Mixture-of- loss [46].
Experiments
5.1 Experimental setup
Implementation details. Building on top of DUSt3R 444, we freeze the encoder and fine-tune only the decoder and the DPT head [32], as done similarly by MonST3R. For each epoch,

---

DUSER
MonST3R
Ours
McnST3R
Ours
Figure 5: Pointmap reconstruction. We qualitatively compare the 3D pointmap of D' against other pointmap regression models [44 511. All visualizations presents per-pixel pointmaps without applying confidence thresholding. It is notable that both DUS(3R 44 and MonST3R [5]] struggle to accurately reconstruct scenes that include dynamic object. We find that inaccurately established corre- spondence fields between dynamic regions negatively affect the overall reconstruction performance.
Table Multi-frame depth estimation results. We compare multi-frame depth for both the entire scene and dynamic regions separately. The comparison for dynamic regions is conducted only when the dynamic parts are identifiable. * Reproduced with same dataset as Ours.
Pl TUM--yyames
Bon
Smnel
KIVII
Category
Methods
All
Dyzamic
All
Dynamic
All
Dynanic
All
AbsRelj
6,7 AbsReli
4.
AbrRelj
AbsRel
5.1
AbaRel)
AlsRalj
6,
AbsRel, 5
Singlo-iame dopei
Marigelá
Depthskyyhnngd
50.5
0.211
0.122
MASI3R
0.199
Multi frame
MonS133
0.122
MonST3R2
41]
0.159
$1.0
0.151
76.5
0.076
93.9
0.71
91.4
0.349
52.5
01565
36.9
0.100
90.9
D7 P'USSERI(OPP (Cwin)
0,142
83.9
0.148
82.9
0,000
95.8
0.099
95.7
0.324
57.5
0.568
48.0
0.104
90.7
randomly sample 20,000 image pairs and the network trained for 50 epochs. We use the AdamW optimizer (27 with an initial learning rate of 5e-5. We train with 4 NVIDIA RTX 6000 GPUs, with batch size images per GPU and gradient accumulation steps set to 2.
multiple datasets, including Blink Vi- sion Outdoor (22] BlinkVision Indoor 22 Spring (29), PointOdyssey [54], and TartanAir (45). Each
Training datasets. As shown in Table 1 train D
epoch consisted of sampling 7, 7,750, 4,750, 2,500, 2,500, and 2,500 pairs, respectively. Additionally,

---

Table 3: Single-frame depth estimation results. Repro-
Table 5: Pointmap alignment racy in dynamic objects. We report End-Point Error (EPE) on the Sintel and KITTI datasets. Note that Croco-
duced using the same dataset as ours.
Metlods
Boas AbwRell
Stenel
KITTI
NY
65
TUM Dsnoriss
f.
DUSIAR
Ed
0:141
82.5
(1:24
96.7
0:113
86.3
03880
96.7
0.175
76.
MASIAR
0.142
$3.0
08355
17.9
IJ776
98.2
0.126
64.9
0.100
76-7
Flow [47 is shown in gray the Sin-
Me
01.9
13363
0.101
59.3
11.1
Mas53Re
50
tel benchmark because was trained
DPUSIS (chrrs
6.96
05.2
0.380
86.3
01885
0.10
829
on that dataset. *: Reproduced using
Table 4: Camera pose estimation results. *; Reproduced using the same dataset
the same dataset as ours.
Methods
Simeel Clean
Simel Finzl
KITTI
DUSAR
30.96
$5.11
14.19
Medieds
Statcl
Scanhhe
MASHER
39:37
50.50
13.23
Aegl
Mevlj
Ang.
Mad.
Args
Midl]
Aegi
Med]
Ang:
Aegi
MouST3R
MonSTaR
M1
38.47
37.47
41.91
40.55
1491
14:55
DSUSERR (Our))
16.19
25.31
R91
MASHUM 21
1,40
0.23
2.55
1.15
010
038
11.04
50
4MM
2.30
0.36
1422
1.38
1.39
030
010
636
1076
18.00
11.0
CacceFlor Flon
+b
3:31
4.25
13.24
SEA-RAF RAFT
5.21
13.18
4.45
D' US138 (0urs)
5.56
2,67
0.36
130
141
0.2
6.5
0.37
0.68
0.56
P'USER Flow bead
9.55
12.79
Input image
DUS$3
MonST3R
Ours
Figure Depth estimation qualitative results.
DUSI3
MAS33
McnST3R
Ous
Figure 7:
Visualization of correspondences given pair of images. We show that our method can find accurate correspondences between dynamic objects.
perform random sampling with temporal strides varying from to account for large camera motions and dynamic scenarios.
Baselines. Following 44 we evaluate our method depth and camera pose estimation. We
additionally evaluate pointmap alignment accuracy in dynamic regions. We compare our method against existing sftte-of-thh-art pointmap regression models, specifically DUSt3R [4], MASt3R (1]] and MonST3R [51]. Furthermore, to ensure fair comparisons and demonstrate the effectiveness of approach, we trained a variant of MonST3R, termed MonST3R under the same setup ours. Evaluation setup. For multi-frame depth estimation, we evaluate on the TUM-Dynamics (38), Bonn R0 Sintel (2). KITTI [11. and ScanNet 7 datasets using image pairs with source frames offset from the target by strides of 1, 3, 7, and frames. We assess performance over entire scene and exclusively on dynamic regions when dynamic masks are available. For single-

---

Iragc
At Dymanic mak
Our Flav load
Figure 8: Predicted optical flow.
Figure 9: Predicted dynamic mask.
Table 6: Robustness analysis frame intervals. We evaluate the robustness of our method with respect to varying frame intervals At E {1,3, 3, 5, (,9). 9). *: Reproduced using the same dataset as ours.
Af
Af
A1.
At
Methods
AbsRel|
61 1
AbsRel
61 f
AbsRel]
61 f
AbsRel
61
AbsReL,
61 f
MonST3R=
0.078
93.8
0.078
93.5
0.075
93.9
0.074
93.9
0.072
94.3
DSUR (Ours)
0.061
95.6
0.061
95.5
0.060
95.7
0.061
95.9
0.058
96.2
frame depth estimation, we evaluate on the Bonn [30], Sintel 2 KITTI TO, NYU-v2 [6). and TUM-Dynamics [38] datasets. In both settings, we follow the affine-invariant depth evaluation protocol, reporting the Absolute Relative Error (AbsRel) and the percentage inlier points (d), where 5 < 1.25).
5.2 Experimental results
Depth estimation. In this experiment, we evaluate our method and compare with existing methods on single/muuti-frre framme depth estimation. The results are summarized in Table 2 and We also show qualitative comparisons in Figure Figure 6 From the results, we observe that D2USt3R outperforms other methods overall. However, our performance on KITTI is somewhat limited, likely due to the absence of driving scenes in our training data, which may have disadvantaged our model. Nonetheless, we compare with MonST3R", which was trained with the same training datasets as ours, and find that the performance is comparable. Finally, we highlight in Figure that our approach consistently predicts accurate depth for dynamic human subjects.
Camera pose estimation. In Table 4 evaluate camera pose performance of model on the Sintel 22. TUM-dynamics (38] and ScanNet 77 datasets. Note that we use the 2D-3D matching between the prediction 2 and the original coordinates to obtain camera poses via PNP-RANSAC.
Dynamic alignment. In Table we evaluate pointmap alignment accuracy in dynamic objects. For this, we use Sintel ( and KITTI [10]. Since we directly obtain aligned pointmaps even in dynamic objects,
easily derive 2D-2D matching points by computing the difference between X2.1 and X1,1, From the results, we find that our method outperforms other baselines, which further supported in Figure 7 where accurately captured correspondences between objects in different time step frames are observed. This gap is further broadened when we leverage an additional flow head, as shown in Table 5 where ours with the flow head outperforms the state-of-the-an SEA-RAFT [46). We provide qualitative examples in Figure g
Dynamic mask. In this experiment, we show that using dynamic mask head, our method reliably predicts dynamic regions. We show visualizations in Figure 9 where the dynamic mask head effectively segments dynamic objects across diverse in-the-wild scenarios.
5.3 Robustness analysis frame intervals
We evaluate multi-frame depth estimation for the robustness of our method with respect to varying frame intervals on the Bonn dataset [30], as summarized in Table 6 Across all tested intervals (At E (1,3.5 5, 7,9)), the results show that our model maintains stable performance and even exhibits slight improvement as the interval widens, demonstrating strong temporal consistency and resilience to larger motion baselines. These results suggest that the proposed flow-chaining strategy effectively preserves geometric coherence across diverse motion dynamics, enabling reliable depth estimation under varying temporal conditions.

---

5.4 Ablation study
Although we 've already validated our 3D aligmment loss and effectiveness of pro- posed SDAP, we further examine how dif- ferent fine-tuning strategies affect perfor- mance. Since our method, unlike DUSt3R or MonST3R, learns precise maps of cross- attention (Figure3) 3). we ask whether updat- ing the encoder along with the decoder and head yields gains. As shown in Table 7 fine-tuning only
Table 7: Ablation on training strategy.
TUM-Dynamics AbsRel,
Bonn
Methods
61
AbsRell
6g 1
finctune
0.161
77.0
0.081
91.9
Finetune decoder head
0.142
83.9
0.060
95.8
decoder and downstream head actually outperforms full fine-tuning, Therefore, this work we restrict training to the decoder and head.
6 Conclusion
In this paper, we have introduced a novel approach for 3D dynamic scene reconstruction, featuring simple yet effective extension existing pointmap representations motions. Our proposed method significantly enhances the quality of 3D reconstruction in dynamic environments. We evaluated our approach comprehensively across tasks including depth estimation, camera pose estimation, and 3D point alignment. Experimental results demonstrate that our method outperforms existing approaches on real-world, large-scale datasets, achieving new state-of-the- performance.
accommodate dynamic
Acknowledgments and Disclosure of Funding
This research was supported by Institute of Information communications TTehhnooog Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2019-1119007,, RS-2024- 00509279, RS-2025-II2 1206,, RS-2023-00227592, RS-2025-022144799 RS-2024-00057882, RS- 015-254448388
RS-2025-022 14479, RS-2025-02217255) and the Culture,
Sports, and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism (RS-2024-00345023 RS-2024-00333068. RS-2023- 00222280, 2S--203--00266000 and National Research Foundation of Korea RKS-2024-00346597),
This research was supported by the ETH AI Center through an ETH AI Center postdoctoral fellowship Sunghwan Hong.
10

---

References
[1] Honggyu An, Jinhyeon Kim, Seonghoon Park. Jaewoo Jung, Jisang Han, Sunghwan Hong, and Seun- gryong Kim. Cross-view completion models are zero-shot correspondence estimators, arXiv preprint arXiv:2412.09072, 2024,
[2] Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Computer Vision-RCCV 2012: 12th Furopean Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12, pages 611-625. Springer, 2012.
[3] Carlos Campos, Richard Elvira, Juan J Gómez Rodríguez, José MM Montiel, Juan Tardós. Orb- slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. JEEE Transactions Robotics, 1776)11874-180 2021.
[4] Seokju Cho, Sunghwan Hong, Sangryul Jeon, Yunsung Lee, Kwanghoon Sohn, and Seungryong Kim. Cats: Cost aggregation transformers for visual correspondence. Advances in Neural Information Processing Systems, 34:901 1-9023, 2021.
[5] Seokju Cho, Sunghwan Hong, and Seungryong Kim. Cats++: Boosting cost aggregation with convolutions and transformers. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(6):7174-7194, 2022.
[6] Christopher Choy, Danfei Xu, Young Gwak. Kevin Chen, and Silvio Savarese. 3d-r2n2: A unified approach single and multi-vier view object reconstruction. Computer Vision ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VIII 14, pages 628-644. Springer, 2016.
[7] Angela Dai, Angel Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattemn recognition, pages 5828-5839,
2017. 2017.
[8] Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE Irunsactions pattern analysis and machine intelligence, 296)) 2007.
[9] Yasutaka Furukawa, Carlos Hernández, et al. Multi-view stereo: A tutorial. Foundations and Trends in Computer Graphics and Vision, 91-2): 1-88, 2015.
[10] Andreas Geiger, Martin Lauer, Christian Wojek, Christoph Stiller, and Raquel Urtasun. 3d traffic scene understanding from movable platforms. IEEE transactions puttern analysis and machine intelligence, 36(5):1012 1025, 2013.
[11] Andreas Geiger. Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The international journal of robotics research, 123317722, 2013.
[12] Georgia Gkioxari, Jitendra Malik, and Justin Johnson. Mesh r-cnn. In Proceedings of the IEEE/CVF international conference computer vision, pages 9785 9795, 2019.
[13] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE transactions pattern analysis and machine intelligence, 43(1))4338- 4364, 2020.
[14] Sunghwan Hong, Seokju Cho, Jisu Nam, Stephen Lin, and Seungryong Kim. Cost aggregation with 4d convolutional swin transformer for few-shot segmentation. In European Conference on Computer Vision, pages 108-126. Springer, 2022.
[15] Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jisang Han, Jiaolong Yang. Chong Luo, and Seungryong Kim. Pf3plat: Pose-free feed-forward 3d gaussian splatting. arXiv preprint arXiv:2410.22128 2024.
[16] Sunghwan Hong, Jaewoo Jung, Heeseong Shin, Jiaolong Yang, Seungryong Kim, and Chong Luo. Unifying correspondence pose and nerf for generalized pose-free novel view synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 20196-20206, 2024.
[17] Sunghwan Hong and Seungryong Kim. Deep matching prior: Test-time optimization for dense correspon- dence. In Proceedingy of the IEEE/CVF international conference on computer vision, pages 9907-9917, 2021.
[18] Sunghwan Hong, Jisu Nam, Scokju Cho, Susung Hong Sangryul Jeon, Dongbo Min, and Seungryong Kim. Neural matching fields: Implicit representation of matching fields for visual correspondence. Advances in Neural Information Processing Systems, 35:13512- 13526, 2022.

---

[19] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion--ased image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recoognition (CVPR), 2024.
[20] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler. and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. 42(4) 2023.
[21] Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. arXiv preprint arXiv:2406, 09756, 2024.
[22] Yijin Li, Yichen Shen, Zhaoyang Huang, Shuo Chen, Weikang Bian, Xiaoyu Shi, Fu- Yun Wang, Keqiang Sun, Hujun Bao, Zhaopeng Cui, et al. Blinkvision: A benchmark for optical flow, scene flow and point tracking estimation using rgb frames and events. In European Conference on Computer Vision, pages 19-36 36. Springer, 2024.
[23] Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast, and robust structure and motion from casual dynamic vidcos. arXiv preprint arXiv:2412.04461. 2024.
[24] Yiqing Liang. Abhishek Badki, Hang Su, James Tompkin, and Orazio Gallo. Zero-shot monocular scene flow estimation in the wild. arXiv preprint arXiv:2501. 10357, 2025.
[25] Chen-Hsuan Lin, Chen Kong, and Simon Lucey. Learning efficient point cloud generation for dense 3d object reconstruction. In proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[26] Philipp Lindenberger, Paul-Edouard Sarlin, Viktor Larsson, and Mare Pollefeys. Pixel-perfect structure- from-motion with featuremetric refinement. Proceedings of the IEEE/CVF international conference on computer vision, pages 3987-5997 2021.
[27] Ilya Loshchilov and Frank Hutter. arXiv:1711.05111.22 2017.
Decoupled weight decay regularization. arXiv preprint
[28] Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic videos. arXiv preprint arXiv:2412.03077 2024.
[29] Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nalivayko, and Andrés Bruhn. Spring: A high- resolution high-detail dataset and benchmark for scene flow, optical flow and stereo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 498 1981-4991, 2023.
[30] Emanuele Palazzolo, Jens Behley. Philipp Lottes. Philippe Giguere. and Cyrill Stachniss. Refusion: 3d reconstruction in dynamic environments for rgb-d cameras exploiting residuals. In 2019 IEEE/RS] International Conference on Intelligent Robots and Systems (IROS). pages 7855-7862. IEEE, 2019.
[31] Jordi Pont-Tuset, Federico Perazzi. Sergi Caelles, Pablo Arbeláez, Alex Hornungg and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675. 2017.
[32] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179-122 12188, 2021.
[33] Johannes L Schonberger and Jan-Michael Frahm. Structueeffrommmootion revisited. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 4104-4113, 2016
[34] Thomas Schops, Johannes Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Mare Pollefeys, and Andreas Geigen. A multi-view stereo benchmark with high-resolution images and multi- camera videos. In Proceedings of the IEEE conference on compuder vision and pattern recognition, pages 2017.
[35] Steven M Seitz, Brian Curless, James Diebel, Daniel Scharstein, and Richard Szeliski. A comparison and evaluation of multi-view stereo reconstruction algorithms. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR 066) volume 1, pages 519-528. IEEE, 2006.
[36] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 20/2, Proceedings, Part V 12, pages 746-760. Springer, 2012.
[37] Vincent Sitzmann, Justus Thies, Felix Heide, Matthias NieBner, Gordon Wetzstein, and Michael Zollhofer. Deepvoxels: Learning persistent 3d feature embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2437-2446, 2019.
12

---

[38] Jürgen Sturm, Nikolas Engelhard. Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 573-580. IEEE, 2012.
[39] Narayanan Sundaram, Thomas Brox, and Kurt Keutzer. Dense point trajectories by gpu-accelerated large displacement optical flow. In European conference on computer vision, pages 438 45114 Springer. 2010.
[40] Zachary Teed and Jia Deng- Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part 16. pages 402 419. Springer, 2020.
[41] Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novolay. Vggsfi:: Visual geometry grounded deep structure from motion. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 21686-21697, 2024.
[42] Nanyang Wang, Yinda Zhang. Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference on computer vision (ECCV), pages 52-67, 2018.
[43] Qianqian Wang, Yifci Zhang, Aleksander Holynski, Alexci A Efros, and Angjoo Kanazawa, Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 1101..13383 12387, 2025.
[44] Shuzhe Wang, Vincent Leroy. Yohann Cabon. Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made casy. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 20697-20709, 2024.
[45] Wenshan Wang. Delong Zhu, Xiangwei Wang. Yaoyu Hu, Yuheng Qiu, Chen Wang. Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: dataset to push the limits of visual slam. In 2020 JEEE/RSJ International Conference on Intelligent Rohots and Systems (IROS), pages 4909-4916. IEFF, 2020.
[46] Yihan Wang, Lahav Lipson, and Jia Deng. Sca-raft: Simple, efficient, accurate raft for optical flow. In European Conference on Computer Vision, pages 36 54. Springer. 2024.
[47] Philippe Weinzaepfel. Thomas Lucas, Vincent Leroy. Yohann Cabon, Vaibhav Arora. Romain Brégier. Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and Jérôme Revaud. Croco v2: Improved cross-view completion pre-training for stereo matching and optical flow, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17969-17 17980,1710000000 2023.
[48] Jianing Yang, Alexander Sax, Kevin J Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. arXiv preprint arXiv: 13928, 2025.
[49] Lihe Yang. Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arX1v:2406.09414. 2024.
[50] Zhichao Yin and Jianping Shi. Geonet: Unsupervised learning of dense depth, optical flow and camera pose. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1983-1992, 2018
[51] Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry the presence of motion. arXiv preprint arXiv:2410.03e6 2024.
[52] Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Rubinstein, Noah Snavely, and William Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pages 20-37. Springer, 2022.
[53] Wang Zhao, Shaohui Liu, Hengkai Guo, Wenping Wang, and Yong-Jin Liu, Particlesfm: Exploiting dense point trajectories for localizing moving cameras in the wild. In European Conference on Computer Vision, pages 523-542. Springer, 2022.
[54] Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wetzstein, and Leonidas I Guibas. Pointodyssey: A large-scale synthetic dataset for long-ter term point tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19855-19866 2023.
13

---

Enhancing 3D Reconstruction for Dynamic Scenes Supplementary Materials
Jisang Han
Honggyu An
Jaewoo Jung'
Takuya Narihira" Junyoung Seo
Kazumi Fukuda²
Chachyun Kim!
Sunghwan Hong* Yuki Mitsufujii..1 Seungryong Kim-t
KAIST Al
Sony Al
ETH Zürich Al Center, CVG, PRS
Sony Group Corporation
https://cvlab-kaiit -kist.. .ithhb. io/DDUSt.3R
A. Training dataset
Table 8: List of excluded scenes from the datasets.
Dataset
Excluded Scenes
animal_s, animal smoke__ unimall_s__,.. animall [ss animal22s, animall
animal3 animal4 animal6
cab e e_eg2, cab_el__ 33dr, cab__elegol2. cab bench, _3d, cab h. bench, beo22, characterO, f, characterO_ character3 character4_ character5_ churacteró, enh dlah 0215 2215_3rd,, enb dlab 0215_0225_000 aaa___22__egll egol. enb_dlab. enb dlab. 022__0222__000002 3rd, cub dlab. 02250022200000 _e____222________0 dancingroom 3rd. human in scene, kg. r5 new
PointOdyssey
cnb_dlab_
d78, (ec____16_eeeee_ scenel_011 _0000, seminar_h52_ege
3rd, scene, d78 00180188..0 egoll scene dd___dd88eee.01 118__3318eego j716 17716_A40 Ard, scene_j716_ego egol.
outdoor train, autopilot tree 01, outdoor train_ autopilot_toot tree tree__2 02. outdoor train_raaaaaa autppplotttttttecoo free outdoor train_track anickanimml,.. _napmal_ppoplleeeeeee
Blinkvision Outdoor
We provide the scene names that were excluded during our training. These scenes are excluded, either because their annotations include errors. Please refer to Table 8
B. Bullet time reconstruction for dynamic video input
Owing to dynamic alignment and SDAP, we can aggregate and render a highly dynamic video input into single, coherent bullet-time view. As illustrated in Figure [00 video consisting of large input frames can be aligned into an intermediate bullet-time representation. This approach enables static reconstruction even when the input video includes dynamic motion. Consequently, becomes feasible to directly train a 3D Gaussian splatting (00 on scenes containing moving people in landmarks dynamics that are challenging to render using conventional methods.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Input video
Ours
MonST3R
Figure 10: Visualization of bullet-time reconstruction from long sequence of dynamic video inputs. We visualize the reconstruction of bullet-time view (20 frames) from a dynamic video input consisting of 40 frames. Since MonST3R is incapable of dynamic alignment, predicts depth independently at each timestep, similar to monocular depth estimation, resulting in the reconstruction sequence of 3D pointmaps.
15

---

MonST3R
Ours
Figure 11: Additional qualitative results for pointmap reconstruction. We qualitatively compare 3D pointmap of D'UR against MonST3R. All visualizations presents per-pixel pointmaps without applying confidence thresholding.
Input image
DUS13R
MonST3R
Ours
Figure 12: Additional qualitative results for depth estimation on Sintel dataset.
C. Additional results
In Figure II Figure 12 additionally present qualitative results for pointmap reconstruction and depth estimation. Utilizing our SDAP, dynamic elements are well-aligned, thereby enhancing both depth estimation and 3DD reconstruction quality. In Figure 13 and Figure [4 we present qualitative results for optical flow on DAVIS (31 and KITTI [I] datasets. We observed that D DSURR is capable of accurately predicting optical flow without relying on any dedicated optical flow module.
16

---

1
Inpur views
DUS:3
MonST3R
Ours
Ours Flow head
Figure 13: Qualitative results for optical flow estimation DAVIS dataset.
Expaa dittn
Oven
hoad
Figure 14: Qualitative results for optical flow estimation on KITTI dataset.
17

---

Table 9: Multi-frame depth estimation results end-to-end integration analysis.
TUM-Dynamics AbsRel.
Bonn AbsRell
Sintel
KITTI
Methods
61 t
61
AhsRell
61 1
AbsRell
61 f
DOUSE3R
0.142
83.9
0.060
95.8
0.324
57.5
0.104
90.7
D'USt3R Dynamic mask head
0.140
84.4
0.059
95.6
0.313
58.3
0.105
90.3
D. End-to-end dynamic mask head integration
We further explored an end-to-end training strategy in which the entire network, including the dynamic mask head, is jointly optimized. As shown in Table 9 this integrated setup encourages implicit interactions between sub-tasks through shared representations, leading to consistent, albeit moderate, performance improvements. Specifically, we observe gains on two of the four benchmarks, with mean reduction AbsRel 1.7% and mean increase in å of 0.9 percentage points. These results confirm that joint optimization promotes beneficial cross-task interactions, yielding complementary improvements across depth and dynamic mask estimation.
E. Limitations
Despite effectively handling dynamic scenarios, our dynamic alignment process has certain limitations. Since our model relies on an off-the-shelf optical flow model, its performance is dependent on the quality of the predicted optical flow. Additionally, our model is non-generative and can only take two frames as input. Because we aim to achieve alignment across time, extending our approach to video input would require an additional global optimization technique.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction (Section D. we clearly state the assump- tions, claims, and contributions of this paper. Furthermore, the analysis (Section 4.1 and experimental evidence (Section 5) are consistent with these claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
In Section D, we discuss the limitations of our approach.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer: [NA]
Justification:
Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We describe the dataset construction in Sections 4.2 the implementation of the loss functions in Scction 4.3 and 4.4 and the implementation details in Section
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
20

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Due the large size of the pretrained weights, we plan release them publicly at a later time.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.cc
aablic/guudees/Codeetbbm.ss itwpoeeeSuubiissiooooooooooogg for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wwtps/cccppplggguuiieeeeee
Saaeee//CoSSSubisssS.S.. www.ssssiicgg.oooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide training and testing details necessary to understand the results in Section 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Since our model does not involve any randomness during inference when trained weights are used, we do not report error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτor rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details on GPU resources and training time are provided in Section 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https://neuurpg:: wwtpp/ppwcc/tcccccccciiiiiieeeee
Answer: [Yes]
Justification:
We have adhered strictly to the NeurIPS Code of Ethics all our experiments.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper aims to advance the field of Machine Learning, particularly in training 3D reconstruction. We do not believe there are any specific societal consequences of our work that require special attention here.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
22

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We believe our research, which focuses on 3D reconstruction, does not pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All papers associated with the datascts we used have been properly cited.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
23

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.paperssit
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification:
We do not introduce any new assets in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our research does not involve crowdsourcing nor research with human sub- jects. 1080
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our research does not involve crowdsourcing nor research with human sub- jects. 1080
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: We use LLM for editing (e.g., grammar, spelling, word choice).
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
25


Reviews:
[{"review_id": "v5U1ZWxa5l", "reviewer": "Reviewer_YZSu", "summary": "The paper proposes an enhanced 3D reconstruction method for dynamic scenes, addressing limitations in existing approaches like DUS3R and MonST3R, which struggle with dynamic object motion due to their reliance on static scene assumptions. The authors introduce Static-Dynamic Aligned Pointmaps (SDAP), a novel representation that integrates camera pose information for static regions and optical flow for dynamic regions to achieve consistent 3D point correspondences across frames. By incorporating motion-aware training objectives, the method improves depth estimation and pointmap alignment in dynamic scenes. The approach is evaluated on datasets like TUM-Dynamics, Sintel, and KITTI, demonstrating superior performance in multi-frame depth estimation and pointmap alignment compared to baselines. Training details, including datasets and hyperparameters, are provided, though pretrained weights are not released due to their size.", "questions": "Please see the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The introduction of Static-Dynamic Aligned Pointmaps is a significant contribution, effectively addressing the misalignment issues in dynamic scenes by combining camera pose-based alignment for static regions with optical flow-based warping for dynamic regions. This dual approach is a clever solution to the temporal inconsistency problem.\n- The paper conducts thorough evaluations across multiple datasets (TUM-Dynamics, Sintel, KITTI) with varying temporal strides, providing robust evidence of the method’s effectiveness in both static and dynamic regions. The inclusion of qualitative visualizations (Figures 5-7) strengthens the interpretability of results.\n- By augmenting the feed-forward framework with explicit motion-aware constraints, the method tackles the limitations of per-frame training paradigms in prior work, leading to improved consistency in depth estimation and pointmap reconstruction.\n- The paper provides sufficient details on dataset construction, loss functions, and training procedures (Sections 4.2, 4.3, 4.4, 5.1), facilitating potential reproducibility despite the lack of code release.\n\nWeaknesses\n- The paper provides an \"end-to-end\" solution for dynamic scene reconstruction. However, the core component—Static-Dynamic Aligned Pointmap (SDAP)—relies heavily on external optical flow estimators (RAFT or precomputed ground-truth flow). This design introduces additional computational overhead and potential inconsistencies, and ultimately constrains the upper bound of model performance due to its dependence on the quality of the external flow. Moreover, the paper does not provide an in-depth analysis of the robustness of the reconstruction to flow errors, nor does it evaluate system performance under scenarios with missing or inaccurate optical flow.\n- The authors introduce an occlusion mask to filter out unalignable regions. However, the results lack any quantitative evaluation of reconstruction quality specifically within occluded areas. Given that occlusion is often the most challenging aspect in dynamic scene reconstruction, this omission limits the assessment of the method’s robustness and generalization to boundary conditions.\n- The authors design separate losses for static and dynamic regions (L_static and L_dyn), yet the alignment loss for dynamic regions depends on warping via optical flow indices. In practice, this approach is prone to sparsity and misalignment issues, especially in occluded regions, at depth discontinuities, or near object boundaries. The resulting loss becomes unstable and may introduce noise into the overall reconstruction. Furthermore, this split-loss design fails to consider continuity and smoothness across static-dynamic region boundaries, which can lead to “crack-like” artifacts in the reconstruction.\n- The authors introduce additional heads—Dynamic Mask Head and Optical Flow Head—aiming to support dynamic region recognition and auxiliary supervision. However, these modules are trained independently and are not integrated into the SDAP optimization process in a closed-loop manner. This lack of interaction limits their effectiveness and wastes the potential of mutual reinforcement.\n- The loss function in D2USt3R is formulated purely based on local frame-pair consistency, without incorporating longer-term temporal coherence or dense path modeling across multiple frames. This absence of long-range temporal constraints makes the method prone to inconsistency or drift, particularly in fast motion or non-rigid object scenarios.", "was_revised": true, "final_justification": "My concerns have been addressed, maintaining the score.", "rebuttal": "> [W1] SDAP depends heavily on external optical flow, introducing potential overhead, inconsistency, and performance limits. It also lacks analysis of robustness to inaccurate or missing flow.\n> \n\nFirst, we would like to clarify that optical flow is **used only during training** to generate the occlusion and dynamic masks that shape our alignment loss. These are done prior to network training, as a preprocessing step. **At inference time, the network outputs a full pointmap directly from the input images—no RAFT or other flow engine is involved—so latency and memory consumption are identical to DUSt3R/MonST3R, with no additional computational overhead.** \n\nThough, we acknowledge that supervision derived from off‑the‑shelf optical flow models could, in principle, limit performance (Please note that leveraging off-the-shelf flow models has been a common practice in many prior works, e.g., [1, 2]). In our setting, however, this risk is minimized for following reasons:\n\n1. **Quality of optical flow.** Following previous works, our work mostly utilizes **synthetic data** to train with accurate depth, optical flow, and camera pose information, which is typically hard to accurately obtain in real-world videos. As a result, only 12.5% of our training batch utilizes flow from off-the-shelf models (Point Odyssey dataset), where the remaining flow is obtained from ground-truth information.\n2. **Uncertainty‑aware point‑map regression.** In addition to the majority of the data coming from ground-truth optical flow, in our framework, the flow provides soft correspondences that are blended with multiview cues; erroneous matches and potentially noisy flows from off-the-shelf models are down‑weighted through the uncertainty term in our loss, as in DUSt3R. We also observe from our empirical experiments that errors near depth discontinuities and occlusions are suppressed.\n\nWhile we have prepared visualizations that highlight these low‑confidence predictions, as the rebuttal format does not allow external media, we will include these visuals in the final version of the paper.\n\nTo deepen the analysis of our method’s robustness to optical‑flow noise, we conduct a controlled study in which we inject uniformly sampled noise within [-1, 1] range into the flow fields, systematically degrading their quality. For each noise level we retrain and evaluate the model, enabling a direct, quantitative comparison across flow qualities. The “SEA-RAFT” row represents the common case where ground‑truth flow is unavailable, while the last row represents when the same model is driven by noisy flows, emulating the use of less capable off‑the‑shelf estimators.\n\n| Methods (AbsRel / $\\delta$) | TUM-Dynamics | Bonn | Sintel | KITTI |\n| --- | --- | --- | --- | --- |\n| SEA-RAFT | 0.142 / 83.9 | 0.060 / 95.8 | 0.324 / 57.5 | 0.104 / 90.7 |\n| SEA-RAFT+ 1 px noise | 0.143 / 83.8 | 0.063 / 95.5 | 0.346 / 52.8 | 0.106 / 90.5 |\n\n| Methods (mean / medium) | Sintel (rotation) | Sintel (Translation) | Tum (rotation) | Tum (translation) |\n| --- | --- | --- | --- | --- |\n| SEA-RAFT | 6.96 / 2.67 | 0.26 / 0.22 | 1.80 / 1.41 | 0.03 / 0.02 |\n| SEA-RAFT+ 1 px noise | 6.51 / 3.09 | 0.27 / 0.23 | 1.89 / 1.44 | 0.03 / 0.02 |\n\nFrom the results, we find that adding noises leads to comparable or only slightly degraded performance. Despite injecting noise levels twice as high as those observed in SEA-RAFT predictions, the model still learns reasonably robust representations.\n\n \n\nBelow, we conduct an additional robustness check – accuracy of SEA‑RAFT flow versus ground‑truth (GT).\n\n| EPE | BlinkVision Outdoor | BlinkVision Indoor | TartanAir | Sintel |\n| --- | --- | --- | --- | --- |\n| Ours vs GT Flow (EPE) | 1.44 | 1.14 | 1.18 | 0.74 |\n\nAcross the four diverse benchmarks the end‑point error (EPE) stays tightly clustered around a single‑pixel discrepancy (mean ≈ 1.13 px). On 288×512 frames this is < 0.2 % of the image diagonal, implying that the off‑the‑shelf SEA‑RAFT flow is already close to the information content of GT flow, and adopting cycle consistency further reduces the error. Consequently, substituting GT with SEA‑RAFT is unlikely to introduce a training bias attributable to flow quality, corroborating the earlier noise‑injection study. \n\n[1] Wang, et al. \"Tracking everything everywhere all at once.\" ICCV’23.\n\n[2] Wang, et al. \"Shape of motion: 4d reconstruction from a single video.\" ICCV’25\n\n> [W2] The results lack any quantitative evaluation of reconstruction quality specifically within occluded areas.\n> \n\nWe highly appreciate the reviewer for underscoring the importance of occlusion handling in dynamic‑scene reconstruction. However, by definition, **ground‑truth depth is undefined for pixels that are invisible to every sensor**, and public dynamic‑scene benchmarks (TUM‑Dynamics, Sintel, KITTI, Bonn) therefore provide depth **only for first‑surface hits**. Evaluating per‑pixel errors where no reference exists or occluded areas would be ill‑posed and would favour methods that merely copy foreground depth. \n\nAs a result, any method must fall back on learned priors and cross‑view cues when estimating depth in those occluded regions. By training with **explicit, dense cross‑view correspondences—even within dynamic areas—**our approach harnesses those priors more effectively, allowing it to infer robust depths for pixels that are occluded in the reference view but revealed in at least one of the other views.\n\nTo stress‑test occlusion reasoning without this pitfall, our paper already reports complementary metrics. Specifically, Table 2 lists depth accuracy on both *All* pixels (static + dynamic) and *Dynamic* pixels only; in both cases, our method is competitive or achieve better results when compared to our baseline (Monst3r*). \n\nWe hope this clarifies how our evaluation addresses occlusions, and we welcome further suggestions if any aspect remains unclear.\n\n> [W3] The separate losses for static and dynamic regions may suffer from sparsity, misalignment, and instability, and do not account for continuity across region boundaries.\n> \n\nWe note that the dynamic‑alignment loss is sensitive to the quality of its optical‑flow anchors: when ground‑truth flow is unavailable, the supervision it provides can become sparse or misaligned. In practice, obtaining sufficiently sharp and accurate flow fields is notoriously difficult—real‑world capture seldom matches the fidelity achievable with synthetic renderers such as Kubric—so this limitation must be considered when applying the loss in the wild. To counter this, we adopt a **confidence‑aware alignment loss** (Eq. 8). The confidence term automatically down‑weights pixels with uncertain correspondences—those near depth discontinuities or under occlusion—thereby stabilizing training. In ablations without this weighting we observed noisy gradients at object boundaries that occasionally destabilized learning, whereas the confidence term prompts the network to assign low weights to such pixels and delivers clean, reliable supervision. We will include visualizations of this effect in the final version.\n\n> [W4] The additional heads are trained independently and not integrated into the SDAP optimization, limiting their effectiveness and potential for mutual reinforcement.\n> \n\nActing on the reviewer’s suggestion, we trained the entire network—including the dynamic mask head—in a single end-to-end manner. This setup forces the intermediate feature tensor to act as a shared representation for the different heads, enabling complementary supervisory signals to interact implicitly.  The results are summarized below.\n\n| Methods (AbsRel / $\\delta$) | TUM-Dynamics | Bonn | Sintel | KITTI |\n| --- | --- | --- | --- | --- |\n| Ours | 0.142 / 83.9 | 0.060 / 95.8 | 0.324 / 57.5 | 0.104 / 90.7 |\n| Ours + integrated | 0.140 / 84.4 | 0.059 / 95.6 | 0.313 / 58.3 | 0.105 / 90.3 |\n\nEven though only the dynamic mask head was jointly optimized, we still observed performance improvements—end-to-end training delivers consistent, if moderate, gains on two of the four benchmarks (mean AbsRel ↓ 1.7 %, mean δ ↑ 0.9 pp), confirming that joint optimization promotes the tasks to improve one another—an effect also reported in [1, 2,3].\n\nHowever, we regret that due to limited time and resources available for this rebuttal, we were unable to include the optical flow head in this end-to-end training. However, building upon a well established empirical knowledge that correspondence-based supervisory signals for multi-task learning formulations can enhance more reliable 3D reconstruction [1, 2], we expect there would exist potential enhancements if we include additional heads, such as tracking, flows and camera pose estimation\n\n[1] Hong,  et al. \"Unifying correspondence, pose and nerf for pose-free novel view synthesis from stereo pairs.\" CVPR'24.\n\n[2] Wang, et al. \"Vggt: Visual geometry grounded transformer.\" CVPR’25.\n\n[3] Mai, et al. \"Tracknerf: Bundle adjusting nerf from sparse and noisy views via feature tracks.\" ECCV’24.\n\n> [W5] The loss is based only on local frame-pair consistency, lacking long-term temporal constraints, which may lead to drift or inconsistency, especially in fast or non-rigid motion.\n> \n\nWe would like to clarify that our training procedure samples frame pairs with variable strides at every epoch, exposing the network to wide‑baseline correspondences and fast motions. This design helps reduce drift while avoiding the memory overhead associated with dense‑path losses or explicit tracking. As noted in the supplementary material, we currently focus on frame pairs because the main contribution is the **Static‑Dynamic Aligned Pointmap (SDAP)** representation, which jointly models static geometry and dynamic motion. Extending SDAP to true multi‑view or sliding‑window optimization is a natural next step, and we are actively exploring this direction for future work."}, {"review_id": "PZQBfyQMwI", "reviewer": "Reviewer_tcmj", "summary": "The paper proposes $D^2UST3R$, an approach to improve upon previous 3D aware pointmap regression approaches such as Dust3r and Monst3r for dynamic scenes. They propose SDAP: Static-Dynamic Aligned Pointmaps, which capture both static and dynamics elements of the 3d geometry. They also propose leveraging an alignment loss that takes into account missing direct 3d correspondences due to occlusion or motion. Finally, they benchmark across a variety of tasks such as camera pose estimation, multi-frame depth, etc.", "questions": "1. I would like to see comparisons to some more recent methods for dynamic scene reconstruction as stated in the weaknesses.\n2. How does inaccurate flow impact depth or pose estimation, for eg if you were to use a weaker model than the current one your approach is using?\n3. Have you tested on real dynamic-scene datasets with hand-held cameras such as DAVIS or Waymo? How does the model perform in those cases?", "limitations": "1. Please address the point about not using real-world dataset in training. Is that by design or some other reason?", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper is written well and easy to follow\n2. Unified modelling of motion and static elements(SDAP) is an interesting approach to handle dynamic scenes\n\n\nWeaknesses\n1. While the method improves upon 3D aware regression, it still seems imperative to compare to methods such as Align3R[1], MegaSAM[2], etc.\n2. Quality of Flow: How robust is the approach towards noisy flow? Currently the method relies on ground truth flow and off-the-shelf state of the art flow estimators.\n3. All five training datasets are synthetic. No real-capture dynamic-scene sequence seems to be used for training so generalizability to real photometrics, motion-blur, sensor noise is unproven.\n\n[1]Lu, Jiahao, et al. \"Align3r: Aligned monocular depth estimation for dynamic videos.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n\n[2] Li, Zhengqi, et al. \"MegaSaM: Accurate, fast and robust structure and motion from casual dynamic videos.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.", "was_revised": true, "final_justification": "I have read through the responses and thank the authors for providing a detailed rebuttal. The responses seemed to have given more clarity about the positioning and experiments of the paper. Therefore, I'm holding my original score.", "rebuttal": "> [W1, Q1] While the method improves upon 3D aware regression, it still seems imperative to compare to methods such as Align3R[1], MegaSAM[2], etc. I would like to see comparisons to some more recent methods for dynamic scene reconstruction as stated in the weaknesses.\n> \n\nWe thank the reviewer for the constructive suggestion. Due to the long runtime required for MegaSAM’s test-time optimization, we provide comparison results on the Bonn and Sintel datasets. Also we provide pointmap alignment accuracy on the Sintel dataset. The results are shown below:\n\n[Multi-frame depth estimation]\n\n| Methods (AbsRel / $\\delta$) | Bonn | Bonn (dynamic) | Sintel | Sintel (dynamic) |\n| --- | --- | --- | --- | --- |\n| Align3r | 0.067 / 95.4 | 0.075 / 94.5 | 0.353 / 59.8 | 0.543 / 48.1 |\n| MegaSAM | 0.079 / 93.6 | 0.086 / 91.6 | 0.424 / 65.0 | 0.635 / 57.5 |\n| Ours | 0.060 / 95.8 | 0.059 / 95.7 | 0.324 / 57.5 | 0.568 / 48.0 |\n\n[Camera pose estimation]\n\n| Methods (mean / median) | Sintel (rotation) | Sintel (Translation) |\n| --- | --- | --- |\n| Align3r | 8.50 / 2.61 | 0.27 / 0.23 |\n| MegaSAM | 6.67 / 2.46 | 0.32 / 0.23 |\n| Ours | 6.96 / 2.67 | 0.26 / 0.22 |\n\nAdditionally, in contrary to our novel SDAP representation that simultaneously capture both static and dynamic 3D scene geometry, Align3R models pointmaps generated by a single global rigid transformation, leading to compromised correspondence learning, which is supported by the table below.\n\n[Pointmap alignment accuracy]\n\n| Methods (EPE) | Sintel (clean) | Sintel (final) |\n| --- | --- | --- |\n| Align3r | 38.64 | 39.89 |\n| Ours | 16.19 | 25.31 |\n\nMoreover, we additionally include speed and efficiency comparisons for completeness. \n\n[Time complexity and Memories]\n\n| Methods | Time (s) | Memories (MB) |\n| --- | --- | --- |\n| Align3r | 2.166 | 10447 |\n| MegaSAM | 13.33 | 7364 |\n| Ours | 0.226 | 3483 |\n\nTo summarize, our novel approach to adopt SDAP enables simultaneous modeling of both static and dynamic 3D geometry, showing competitive performance in multi-frame depth and pose estimation while significantly improving the performance in dynamic region alignment, with improved efficiency and speed.\n\n> [W2, Q2] Quality of Flow: How robust is the approach towards noisy flow? Currently the method relies on ground truth flow and off-the-shelf state of the art flow estimators. How does inaccurate flow impact depth or pose estimation, for eg if you were to use a weaker model than the current one your approach is using?\n> \n\nThank you for the constructive feedback.\n\nWe acknowledge that supervision derived from off‑the‑shelf optical flow models could, in principle, limit performance (However, please note that leveraging off-the-shelf flow models has been a common practice in many prior works, e.g., [1, 2]). In our setting, however, this risk is minimized for three reasons:\n\n1. **Quality of optical flow.** Following previous works, our work mostly utilizes **synthetic data** to train with accurate depth, optical flow, and camera pose information, which is typically hard to accurately obtain in real-world videos. As a result, only 12.5% of our training batch utilizes flow from off-the-shelf models (Point Odyssey dataset), where the remaining flow is obtained from ground-truth information.\n2. **Uncertainty‑aware point‑map regression.** In addition to the majority of the data coming from ground-truth optical flow, in our framework, the flow provides soft correspondences that are blended with multiview cues; erroneous matches and potentially noisy flows from off-the-shelf models are down‑weighted through the uncertainty term in our loss, as in DUSt3R.\n3. **Empirical evidence.** Our experiments show that the uncertainty‑aware point‑map regression effectively suppresses errors near depth discontinuities and occlusions by assigning low confidence to those regions.\n\nWe have prepared visualizations that highlight these low‑confidence predictions along boundaries and occlusions through our empirical studies. Although the rebuttal format does not allow external media, we will include these visuals in the final version of the paper.\n\nTo deepen the analysis of our method’s robustness to optical‑flow noise, we conduct a controlled study in which we inject uniformly sampled noise within [-1, 1] range into the flow fields, systematically degrading their quality. For each noise level we retrain and evaluate the model, enabling a direct, quantitative comparison across flow qualities. The row (“SEA-RAFT”) corresponds to the default setting that relies on a standard off‑the‑shelf optical‑flow estimator—representing the common case where ground‑truth flow is unavailable. The last row shows performance when the same model is driven by noisy flows, emulating the use of weaker off‑the‑shelf estimators. Below summarizes the resulting metrics.\n\n| Methods (AbsRel / $\\delta$) | TUM-Dynamics | Bonn | Sintel | KITTI |\n| --- | --- | --- | --- | --- |\n| SEA-RAFT | 0.142 / 83.9 | 0.060 / 95.8 | 0.324 / 57.5 | 0.104 / 90.7 |\n| SEA-RAFT+ 1 px noise | 0.143 / 83.8 | 0.063 / 95.5 | 0.346 / 52.8 | 0.106 / 90.5 |\n\n| Methods (mean / medium) | Sintel (rotation) | Sintel (Translation) | Tum (rotation) | Tum (translation) |\n| --- | --- | --- | --- | --- |\n| SEA-RAFT | 6.96 / 2.67 | 0.26 / 0.22 | 1.80 / 1.41 | 0.03 / 0.02 |\n| SEA-RAFT+ 1 px noise | 6.51 / 3.09 | 0.27 / 0.23 | 1.89 / 1.44 | 0.03 / 0.02 |\n\nFrom the results, we find that adding noises leads to comparable or only slightly degraded performance. Despite injecting noise levels twice as high as those observed in Sea-RAFT predictions (Sea-RAFT noise + additional noise; the error of SEA-RAFT is roughly 1px compared to ground-truth shown in the table below), the model still learns reasonably robust representations. Consequently, our empirical evidence and comprehensive robustness evaluations strongly support that our framework effectively minimizes the impact of optical-flow inaccuracies, even when utilizing less accurate, off-the-shelf estimators.\n\n \n\nBelow, we conduct an additional robustness check – accuracy of Sea‑RAFT flow versus ground‑truth (GT).\n\n| EPE | BlinkVision Outdoor | BlinkVision Indoor | TartanAir | Sintel |\n| --- | --- | --- | --- | --- |\n| Ours vs GT Flow (EPE) | 1.44 | 1.14 | 1.18 | 0.74 |\n\nAcross the four diverse benchmarks the end‑point error (EPE) stays tightly clustered around a single‑pixel discrepancy (mean ≈ 1.13 px). On 288×512 frames this is < 0.2 % of the image diagonal, implying that the off‑the‑shelf Sea‑RAFT flow is already close to the information content of GT flow, and adopting cycle consistency further reduces the error. We believe that the advances in the optical-flow field will further refine our pseudo-ground truth optical flow, narrowing the already small gap to perfect accuracy.\n\n> [W3, L1] All five training datasets are synthetic. No real-capture dynamic-scene sequence seems to be used for training so generalizability to real photometrics, motion-blur, sensor noise is unproven. Please address the point about not using real-world dataset in training. Is that by design or some other reason?\n> \n\nAs dynamic datasets with ground-truth depth and camera pose are difficult to obtain, we used only synthetic datasets for training. This has been a **common practice in numerous works** [1,2,3,4,5], as real-world data captured with sensors is typically noisy and sparse, which may be detrimental to network learning rather than helping. Prior works [1,2,3,4,5] also show that this convention is effective in generalizing well to real-world datasets. This is supported by our evaluation in Tab. 2,3,4,5, in which TUM-Dynamics and Bonn contain challenging factors such as camera/motion blur, yet our model was still able to generalize well under these conditions, further validating its robustness.\n\n[1] Li, Zhengqi, et al. \"MegaSaM: Accurate, fast and robust structure and motion from casual dynamic videos.\" CVPR’25\n\n[2] Lu, Jiahao, et al. \"Align3r: Aligned monocular depth estimation for dynamic videos.\" CVPR'25\n\n[3] Wang, Qianqian, et al. \"Tracking everything everywhere all at once.\" ICCV’23.\n\n[4] Wang, Qianqian, et al. \"Shape of motion: 4d reconstruction from a single video.\" arXiv’24\n\n[5] Zhang, Junyi, et al. \"Monst3r: A simple approach for estimating geometry in the presence of motion.\" ICLR’25\n\n> [Q3] Have you tested on real dynamic-scene datasets with hand-held cameras such as DAVIS or Waymo? How does the model perform in those cases?\n> \n\nWe would like to clarify that the TUM-Dynamics and Bonn datasets used in our evaluation are real-world dynamic scene datasets. For the DAVIS dataset, quantitative evaluation is not feasible due to the lack of ground-truth depth. However, as shown in the qualitative results in Figure 5, which includes the pointmap reconstructions from a YouTube real-world video and DAVIS scenes, our method demonstrates strong performance even in real dynamic scenes."}, {"review_id": "8cpHKpHBte", "reviewer": "Reviewer_kbwQ", "summary": "This paper trains a neural network to estimate a point map for each frame from two given video frames that capture dynamic objects. The network learns how to estimate depth for different types of motion. To this end, the static background and the dynamic object regions are separated and are trained with sophisticated objectives that address occlusion and dynamic regions. The qualitative and quantitative comparisons demonstrate its performance across various dynamic scene datasets.", "questions": "I have the following questions:\n\n- Please clarify the notation of a predicted point map X^{n,m} and explain whether there is a conflict between the two objective terms mentioned above.\n\n- In the supplemental, the reconstruction in the background suffers from wavy artifacts and a multiview inconsistency problem. Does the proposed method show degraded performance for the static dataset? Figure 5 also shows that the Dust3r achieves reasonable relative scene scales and a field of view on the static region comparable to Monst3r and the proposed method. It would be great to conduct the evaluation for the static region and the dynamic region individually. \n\n- Do the predicted optical flow and depth agree with each other? \n\n- Does the proposed method produce temporal consistency and multiview consistency? Since the video is not produced, those components cannot be evaluated.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper’s strengths are as follows;\n\n- The performance outperforms other learning-based stereo models, such as Dust3r and Monst3r, for monocular and binocular depth estimation. \n- The occlusion mask and dynamic mask are utilized to train the network without false guidance. For a dynamic region, the objective encourages the network to estimate a point map at the time as its counterpart, shown in Equation 8.\n- Supplemental shows the outperformance of the proposed method against Monst3r.\n\nThere are the following weak points:\n\n- Term X^{1,2} in Equation 8 is used, but there is no explicit definition of it. This makes me confused about understanding the training objectives. Specifically, the first term in Equation 6 and the second term in Equation 8 could conflict with each other if X_1,2 and X_1,1 are from the same estimation. \n- Term X has different meanings at Line 89 and Line 92. It should be consistent throughout the paper context.\n- The optical flow supervision is available for synthetic datasets. This may lead to a domain gap when the trained model is applied to a real-world dataset.\n- The proposed model doesn’t utilize the predicted optical flow for depth estimation. Plus, optical flow and depth estimation are computed independently. This may cause the inconsistency between two different types of predictions: optical flow and 3d points. However, it has not been discussed in the paper.\n- The evaluation doesn't separate the dynamic and the static regions.\n- Figure 7 is too small to see the correspondence matches.\n- There is no video demo that can verify the temporal consistency of the proposed method.\n- The comparison for camera pose estimation is not plausibly discussed.", "was_revised": true, "final_justification": "The reviewers addressed my concerns with thorough experiments. This makes me lean toward accepting this paper. If the authors had submitted the supplemental video in advance, the reviewers could have appreciated the performance of this paper at a glance. In the final version, I highly recommend including the supplemental video of the proposed method.", "rebuttal": "> [W1, Q1] No definition of X^{1,2}. Specifically, the first term in Equation 6 and the second term in Equation 8 could conflict with each other if X_1,2 and X_1,1 are from the same estimation. Please clarify the notation of a predicted point map X^{n,m} and explain whether there is a conflict between the two objective terms mentioned above.\n> \n\nOur definition of the pointmap strictly follows the notation used in DUSt3R. As described in Line 89, X^{n,m} denotes the 3D point map predicted from image I^n, but expressed in the camera coordinate frame of I^m. Therefore, X^{1,2} denotes the 3D pointmap predicted from I^1, expressed in the camera coordinate frame of I^2. \n\nRegarding the potential conflict raised by the reviewer, the term X_{1,2} in Equation 8 refers to the pointmap obtained by swapping the input image pair during inference which is from different estimation, as explained in Line 163. Therefore, X_{1,2} and X_{1,1} in the first term of Equation 6 are not the same estimation in different coordinate frames, but are instead distinct predictions resulting from different input orderings. Additionally, in Equation 8, X_{1,2} refers to the pointmap predicted from I^1, where the dynamic regions are estimated to be aligned from the temporal state of I^2. We apologize for any confusion caused by the lack of precise clarification in our notation and we will revise the manuscript accordingly to improve clarity.\n\n> [W2] Term X has different meanings at Line 89 and Line 92.\n> \n\nWe apologize for the confusion. The term X at Line 89 should be corrected to $\\bar{X}$. We will revise the manuscript accordingly.\n\n> [W3] The optical flow supervision is available for synthetic datasets. This may lead to a domain gap when the trained model is applied to a real-world dataset.\n> \n\nAs dynamic datasets with ground-truth depth and camera pose are difficult to obtain, we used only synthetic datasets for training. This has been **a common practice in numerous works** [1,2,3,4,5], as real-world data captured with sensors is typically noisy and sparse, which may be detrimental to network learning rather than helping. Prior works [1,2,3,4,5] also show that this convention is effective in generalizing well to real-world datasets. This is supported by our evaluation in Tab. 2,3,4,5, in which TUM-Dynamics and Bonn contain challenging factors such as camera/motion blur, yet our model was still able to generalize well under these conditions, further validating its robustness.\n\n[1] Li, Zhengqi, et al. \"MegaSaM: Accurate, fast and robust structure and motion from casual dynamic videos.\" CVPR’25\n\n[2] Lu, Jiahao, et al. \"Align3r: Aligned monocular depth estimation for dynamic videos.\" CVPR'25\n\n[3] Wang, Qianqian, et al. \"Tracking everything everywhere all at once.\" ICCV’23.\n\n[4] Wang, Qianqian, et al. \"Shape of motion: 4d reconstruction from a single video.\" arXiv’24\n\n[5] Zhang, Junyi, et al. \"Monst3r: A simple approach for estimating geometry in the presence of motion.\" ICLR’25\n\n> [W4, Q3] The proposed model doesn’t utilize the predicted optical flow for depth estimation. Plus, optical flow and depth estimation are computed independently. This may cause the inconsistency between two different types of predictions: optical flow and 3d points. However, it has not been discussed in the paper. Do the predicted optical flow and depth agree with each other?\n> \n\nWe wish to highlight that our novel SDAP representation aims to encapsulates depth, camera intrinsic, extrinsic and flow. The separate optical flow head is designed for modeling better smoothness prior, but this still stems from our shared feature representation that also outputs the SDAP. However, we agree that there may exist inconsistency between our SDAP output and the optical flow from the separate head. Nevertheless, **this is also shared by the recent works CoPoNeRF [1] and VGGT [2]**, the radiance field does not align perfectly with the flows and where pointmap does not perfectly align with depth and camera pose outputs. Still, the predictions are within reasonable range, making predictions do not exhibit significant inconsistencies.  We will also include this discussion in the final version.\n\nAdditionally, acting on the reviewer’s suggestion, we could try to train the entire network—including the dynamic mask head ,in a single end-to-end manner, to encourage implicit interactions between the predictions via shared representation. This setup forces the intermediate feature tensor to act as a shared representation for the different heads, enabling complementary supervisory signals to interact implicitly.  The corresponding results are summarized in the table below.\n\n| Methods (AbsRel / $\\delta$) | TUM-Dynamics | Bonn | Sintel | KITTI |\n| --- | --- | --- | --- | --- |\n| Ours | 0.142 / 83.9 | 0.060 / 95.8 | 0.324 / 57.5 | 0.104 / 90.7 |\n| Ours + integrated | 0.140 / 84.4 | 0.059 / 95.6 | 0.313 / 58.3 | 0.105 / 90.3 |\n\nEven though only the dynamic mask head was jointly optimized, we still observed performance improvements—end-to-end training delivers consistent, if moderate, gains on two of the four benchmarks (mean AbsRel ↓ 1.7 %, mean δ ↑ 0.9 pp), confirming that joint optimization promotes the tasks to improve one another—an effect also reported in [1, 2,3].\n\nHowever, we regret that due to limited time and resources available for this rebuttal, we were unable to include the optical flow head in this end-to-end training. However, building upon a well established empirical knowledge that correspondence-based supervisory signals for multi-task learning formulations can enhance more reliable 3D reconstruction [1, 2], we expect there would exist potential enhancements if we include additional heads, such as tracking, flows and camera pose estimation.\n\n[1] Hong, Sunghwan, et al. \"Unifying correspondence, pose and nerf for pose-free novel view synthesis from stereo pairs.\" CVPR'24.\n\n[2] Wang, Jianyuan, et al. \"Vggt: Visual geometry grounded transformer.\" CVPR’25.\n\n> [W5, Q2] The evaluation doesn't separate the dynamic and the static regions.\n> \n\nWe wish to clarify that in **Tab. 2, we have separated the evaluation metric to All and Dynamic, which should already reflect the reviewer’s comment**. Nevertheless, we conduct an additional experiment, where below shows **evaluations solely performed on static regions**.\n\n| Methods (AbsRel / $\\delta$) | TUM-Dynamics | Bonn | Sintel | KITTI |\n| --- | --- | --- | --- | --- |\n| Monst3r | 0.147 / 83.2 | 0.061 / 95.8 | 0.313 / 56.4 | N/A |\n| Ours | 0.137 / 85.3 | 0.059 / 96.1 | 0.323 / 55.3 | N/A |\n\nFrom the results, we find that the results are comparable to Monst3R on static regions, adhering to our SDAP formulation where the static regions are supervised identically to the original DUSt3R. By also supervising dynamic regions so that both static and dynamic regions are aligned to their corresponding counterparts, we achieve similar performance in static regions while largely improving the reconstruction quality in dynamic regions.\n\n> [W6] Fig7 is too small.\n> \n\nWe apologize for the inconvenience. We will revise it accordingly.\n\n> [W7, Q4] No video demo that can verify the temporal consistency of the proposed method. Does the proposed method produce temporal consistency and multiview consistency? Since the video is not produced, those components cannot be evaluated.\n> \n\nWe would like to clarify that, as noted in the **supplementary material**, the current method operates on frame pairs because our primary contribution is the **Static‑Dynamic Aligned Pointmap (SDAP),** a novel representation that jointly models static geometry and dynamic motion. Extending SDAP to true multi‑view or sliding‑window optimization for temporal consistency is an orthogonal direction that we are actively investigating and view as promising future work.\n\n> [W8] Comparison for camera pose estimation is not plausibly discussed.\n> \n\nWe appreciate the reviewer’s concern about camera-pose accuracy. In 3D reconstruction in dynamic scenes, pose estimates obtained with a single PnP-RANSAC pass are often sub-optimal for systems which handle correspondences in dynamic regions. To mitigate this issue, recent systems such as **Shape-of-Motion**, **Spatial Tracker v2**, and **MegaSAM** tackle this by iteratively refining the pose by masking out dynamic regions at each step, or feed-forward frameworks like Dust3R and Monst3R do not address correspondences in dynamic regions in their 3D pointmap representation.\n\nOur goal, however, is to keep the pipeline **fully feed-forward and lightweight while also modeling correspondences in dynamic regions**. We introduce a unified representation (SDAP) that predicts geometry for both static and dynamic content while embedding camera intrinsics and extrinsics in a single pass. To preserve this simplicity we use the **vanilla PnP-RANSAC**, foregoing specialized pose modules or iterative motion segmentation.\n\nAlthough this design sacrifices some of the accuracy gains that iterative schemes can deliver, it yields a streamlined, scalable model that still matches—or surpasses—state-of-the-art results on several benchmarks. We will add this trade-off discussion to the final paper and clarify that incorporating auxiliary heads for flow, tracking, or explicit pose refinement is an exciting direction for future work. We will include this discussion in the final version."}, {"review_id": "OL1JnmTr7Z", "reviewer": "Reviewer_JB4c", "summary": "This paper introduces a framework for robust 3D reconstruction in dynamic scenes. It addresses the limitations of prior pointmap regression methods that struggle with object motion by proposing Static-Dynamic Aligned Pointmaps (SDAP). SDAP simultaneously accounts for both spatial structures and temporal motions, providing a more reliable 3D reconstruction of both static and dynamic regions. The method integrates occlusion and dynamic masks, derived from optical flow, to guide the network and stabilize learning. With a confidence-aware 3D regression loss that combines separate objectives for static and dynamic areas,this paper demonstrates superior 3D reconstruction performance across various complex motion datasets.", "questions": "Refer weaknesses", "limitations": "No. Integrate semantic segmentation models or object detection networks to provide high-level semantic and shape information, can make the dynamic mask generation more robust and accurate, especially for complex or ambiguous motions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strenth\n\n\n1) Distinguishing camera-induced motion from genuine object-specific motion is challenging, as it requires precisely separating the camera’s ego-motion from the independent movements of objects in the scene.\n\n2) It introduces a unique representation called Static-Dynamic Aligned Pointmaps, which can simultaneously account for both static spatial structures and dynamic temporal motions in a unified framework.\n\n3) The paper is clearly written and easy to follow.\n\n\n\nWeakness\n\n\n1)The authors describe their dynamic mask generation as relying primarily on optical flow analysis, comparing the observed optical flow with that induced purely by camera motion. However, this approach does not inherently leverage high-level semantic or shape information. Modern state-of-the-art object tracking methods, such as SAMURAI, can effectively track dynamic objects in videos and could potentially address the challenges of dynamic mask generation; however, the authors did not conduct any study to evaluate this possibility.\n\n\n2)The results in Table 2 show no improvement on the KITTI dataset, highlighting poor generalization. Despite KITTI’s mostly static scenes, the method still degrades, suggesting it fails to adapt when dynamic motion is absent.\n\n\n3) There is a contradiction in the results: while Table 2 shows no improvement on the KITTI dataset, Table 5 reports a significant boost in pointmap alignment accuracy for dynamic objects. This suggests that the new method may actually degrade overall reconstruction quality.\n\n\n4) The camera pose results in Table 4 do not show any improvement over the state of the art, and the authors fail to clearly explain this outcome.\n\n\n5) The state-of-the-art methods considered are insufficient, as they do not include a comparison with 'Shape of Motion: 4D Reconstruction from a Single Video,' which performs reconstruction for long-range 3D tracking", "was_revised": true, "final_justification": "The new experiments with Segment Any Motion shows good results. Also the authors show additional comparison which seems to be convincing compared to earlier.", "rebuttal": "> [W1, L1] The authors describe their dynamic mask generation as relying primarily on optical flow analysis, comparing the observed optical flow with that induced purely by camera motion. However, this approach does not inherently leverage high-level semantic or shape information. Modern state-of-the-art object tracking methods, such as SAMURAI, can effectively track dynamic objects in videos and could potentially address the challenges of dynamic mask generation; however, the authors did not conduct any study to evaluate this possibility.\n> \n\nWe thank the reviewer for the valuable feedback that can further strengthen our method. Below, we adopt Segment Any Motion in Videos [1] that leverages DINOv2 and 2D tracks for dynamic mask prediction, and use it to replace the dynamic masks we acquired from our approach. Below summarizes the results.\n\n| Methods (AbsRel / $\\delta$) | TUM  | Bonn | Sintel | KITTI |\n| --- | --- | --- | --- | --- |\n| Ours | 0.142 / 83.9 | 0.060 / 95.8 | 0.324 / 57.5 | 0.104 / 90.7 |\n| Ours + Segment Any Motion | 0.140 / 84.4 | 0.062 / 95.4 | 0.313 / 58.4 | 0.104 / 90.5 |\n\nFrom the experiment, we confirm that our method can benefit from more advanced dynamic mask methods, leaving a room for further improvement. We will include this discussion in the final version.\n\n[1] Huang, Nan, et al. \"Segment Any Motion in Videos.\" CVPR'25.\n\n> [W2] The results in Table 2 show no improvement on the KITTI dataset, highlighting poor generalization. Despite KITTI’s mostly static scenes, the method still degrades, suggesting it fails to adapt when dynamic motion is absent.\n> \n\nWe would like to clarify that **it is natural and reasonable to expect and observe similar to Monst3R for KITTI dataset, as the reviewer said, it is mostly static, where SDAP is identical to Monst3R’s objective function**. This is denoted as Monst3R*, which shows apparent difference to original Monst3R that was specifically trained with the driving scene dataset. its domain is significantly different from that of the other datasets. As mentioned in Line 208, our model was not trained on any driving scenes, so it is difficult to expect performance improvements on this dataset.\n\n> [W3] There is a contradiction in the results: while Table 2 shows no improvement on the KITTI dataset, Table 5 reports a significant boost in pointmap alignment accuracy for dynamic objects. This suggests that the new method may actually degrade overall reconstruction quality.\n> \n\nWe apologize for the confusion and appreciate the opportunity to clarify the apparent discrepancy. **Table 2 and Table 5 evaluate two different tasks under distinct protocols.** Table 2 follows the standard multi‑frame **depth‑estimation benchmark**, which scores per‑pixel depth on the reference frames. Because ground‑truth depths in this benchmark are provided almost exclusively for static, first‑surface points, every method—ours included—is evaluated mainly on regions where object motion is minimal. Consequently, the numerical differences between our approach and strong baselines are necessarily small.\n\n**Table 5, by contrast, measures pointmap alignment error across time, explicitly including the dynamic regions detected by our masks.** Baseline methods do not enforce correspondences on moving objects, so their alignment accuracy drops sharply in these areas. Our motion‑aware objective, however, keeps the pointmaps coherent even when objects move, leading to the much larger performance gap reported in Table 5. The two tables are therefore complementary: Table 2 reflects conventional static‑depth accuracy, while Table 5 highlights our method’s key advantage—robust temporal alignment in scenes with motions.\n\n> [W4] The camera pose results in Table 4 do not show any improvement over the state of the art, and the authors fail to clearly explain this outcome.\n> \n\nWe appreciate the reviewer’s concern about camera-pose accuracy. In 3D reconstruction in dynamic scenes, pose estimates obtained with a single PnP-RANSAC pass are often sub-optimal for systems which handle correspondences in dynamic regions. To mitigate this issue, recent systems such as **Shape-of-Motion**, **Spatial Tracker v2**, and **MegaSAM** tackle this by iteratively refining the pose by masking out dynamic regions at each step, or feed-forward frameworks like Dust3R and Monst3R do not address correspondences in dynamic regions in their 3D pointmap representation.\n\nOur goal, however, is to keep the pipeline **fully feed-forward and lightweight while also modeling correspondences in dynamic regions**. We introduce a unified representation (SDAP) that predicts geometry for both static and dynamic content while embedding camera intrinsics and extrinsics in a single pass. To preserve this simplicity we use the **vanilla PnP-RANSAC**, foregoing specialized pose modules or iterative motion segmentation.\n\nAlthough this design sacrifices some of the accuracy gains that iterative schemes can deliver, it yields a streamlined, scalable model that still matches—or surpasses—state-of-the-art results on several benchmarks. We will add this trade-off discussion to the final paper and clarify that incorporating auxiliary heads for flow, tracking, or explicit pose refinement is an exciting direction for future work. We will include this discussion in the final version.\n\n> [W5] The state-of-the-art methods considered are insufficient, as they do not include a comparison with 'Shape of Motion: 4D Reconstruction from a Single Video,' which performs reconstruction for long-range 3D tracking\n> \n\nWe would like to emphasize that our primary goal is to demonstrate that our SDAP brings improvements in 3D reconstruction for dynamic scenes when trained on the same dataset. While *Shape of Motion* performs reconstruction and 2D/3D tracking from video, it relies on scene-specific optimization methods and primarily focuses on tracking and novel view synthesis, which is quite different from our approach. As a more reasonable comparison, we additionally include evaluations of multi-frame depth, camera pose, and pointmap alignment accuracy against Align3r[1] and MegaSAM[2], which are more aligned with our method. The results are presented below.\n\n[Multi-frame depth estimation]\n\n| Methods (AbsRel / $\\delta$) | Bonn | Bonn (dynamic) | Sintel | Sintel (dynamic) |\n| --- | --- | --- | --- | --- |\n| Align3r | 0.067 / 95.4 | 0.075 / 94.5 | 0.353 / 59.8 | 0.543 / 48.1 |\n| MegaSAM | 0.079 / 93.6 | 0.086 / 91.6 | 0.424 / 65.0 | 0.635 / 57.5 |\n| Ours | 0.060 / 95.8 | 0.059 / 95.7 | 0.324 / 57.5 | 0.568 / 48.0 |\n\n[Camera pose estimation]\n\n| Methods (mean / median) | Sintel (rotation) | Sintel (Translation) |\n| --- | --- | --- |\n| Align3r | 8.50 / 2.61 | 0.27 / 0.23 |\n| MegaSAM | 6.67 / 2.46 | 0.32 / 0.23 |\n| Ours | 6.96 / 2.67 | 0.26 / 0.22 |\n\nAdditionally, in contrary to our novel SDAP representation that simultaneously capture both static and dynamic 3D scene geometry, Align3R models pointmaps generated by a single global rigid transformation, leading to compromised correspondence learning, which is supported by the table below.\n\n[Pointmap alignment accuracy]\n\n| Methods (EPE) | Sintel (clean) | Sintel (final) |\n| --- | --- | --- |\n| Align3r | 38.64 | 39.89 |\n| Ours | 16.19 | 25.31 |\n\nMoreover, we additionally include speed and efficiency comparisons for completeness. \n\n[Time complexity and Memories]\n\n| Methods | Time (s) | Memories (MB) |\n| --- | --- | --- |\n| Align3r | 2.166 | 10447 |\n| MegaSAM | 13.33 | 7364 |\n| Ours | 0.226 | 3483 |\n\nTo summarize, our novel approach to adopt SDAP enables simultaneous modeling of both static and dynamic 3D geometry, showing competitive performance in multi-frame depth and pose estimation while significantly improving the performance in dynamic region alignment, with improved efficiency and speed.\n\n[1] Lu, Jiahao, et al. \"Align3r: Aligned monocular depth estimation for dynamic videos.\" CVPR'25\n\n[2] Li, Zhengqi, et al. \"MegaSaM: Accurate, fast and robust structure and motion from casual dynamic videos.\" CVPR’25"}]

Author rebuttal:

Meta-review: After the rebuttal and discussion, the final recommendations were: 4x Borderline Accept.  The rebuttal solved most of the concerns raised by the rebuttal.  The reviewers did not change their scores, however the committee agrees that the average score reflects the paper quality.
Decision: Accept (poster)