Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation
Authors: Weining Ren
Hongjun Wang
Xiao Tan
Kai Han
Venue: NeurIPS 2025 poster ()
Keywords: 3D Reconsturction
Knowledge Distillation
Fine-Tuning
Primary area: applications
TL;DR: We propose a unified framework that distills knowledge from monocular geometry models to improve the performance of feed-forward reconstruction networks.

Abstract: We present Fin3R, a simple, effective, and general fine-tuning method for feed-forward 3D reconstruction models. The family of feed-forward reconstruction model regresses pointmap of all input images to a reference frame coordinate system, along with other auxiliary outputs, in a single forward pass. However, we find that current models struggle with fine geometry and robustness due to (\textit{i}) the scarcity of high-fidelity depth and pose supervision and (\textit{ii}) the inherent geometric misalignment from multi-view pointmap regression. Fin3R jointly tackles two issues with an extra lightweight fine-tuning step. We freeze the decoder, which handles view matching, and fine-tune only the image encoder—the component dedicated to feature extraction. The encoder is enriched with fine geometric details distilled from a strong monocular teacher model on large, unlabeled datasets, using a custom, lightweight LoRA adapter. We validate our method on a wide range of models, including DUSt3R, MASt3R, CUT3R, and VGGT. The fine-tuned models consistently deliver sharper boundaries, recover complex structures, and achieve higher geometric accuracy in both single- and multi-view settings, while adding only the tiny LoRA weights, which leave test-time memory and latency virtually unchanged. Project page: \href{http://visual-ai.github.io/fin3r}{https://visual-ai.github.io/fin3r}

Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation
Weining Ren'
Hongjun Wang!
Xiao Tan
Kai Han
Visual AI Lab, The University of Hong Kong Department of Computer Vision Technology (VIS), Baidu Inc. weiningdconnect. hku.hk, hjwang&connect. hku tanxiao0 @baidu. com, kaihanx@hku. hk
Input
Input
DUSt3R
VGGT
DUSt3R+Ours
VGGT+Ours
Input
CUT3R
CUT3R+Ours
VGGT
VGGT+Ours
Input
MASI3R
MASI3R+Ours
Input
MASt3R
MASt3R+Ours
Figure 1: Fin3R consistently improves the reconstructed geometry quality in DUSL3R, MASt3R, CUT3R, and VGGT, recovering finer details and producing sharper boundaries.
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Abstract
We present Fin3R, a simple, effective, and general fine-tuning method for feed- forward 3D reconstruction models. The family of feed-forward reconstruction model regresses pointmap of all input images to a reference frame coordinate system, along with other auxiliary outputs, in a single forward pass. However, we find that current models struggle with fine geometry and robustness due to (i) the searcity of high-fidelity depth and pose supervision and (ii) the inherent geometric misalignment from multi-view pointmap regression. Fin3R jointly tackles two issues with an extra lightweight fine-tuning step. We freeze the decoder, which handles view matching, and fine-tune only the image encoder-the component dedicated to feature extraction. The encoder is enriched with fine geometric details distilled from strong monocular teacher model on large, unlabeled datasets, using a custom, lightweight LoRA adapter. We validate our method on a wide range of models, including DUSt3R, MASt3R, CUT3R, and VGGT. The fine-tuned models consistently deliver sharper boundaries, recover complex structures, and achieve higher geometric accuracy in both single- and multi-view settings, while adding only the tiny LoRA weights, which leave test-time memory and latency virtually unchanged. Project page: https://visual-ar wttsssaalla.iguub...... wwwhtb.io/ffnnnnn. io/fin3r
Introduction
Recently, neural feed-forward 3D reconstruction models [68, 28, 64, 61, 75, 55, 60] have demon- strated advantages in certain aspects compared to the traditional Structure from Motion (SfM) pipeline [48, 41]. These methods can transform single image or even hundreds of images- into pointmaps defined in the reference frame within single forward pass, thereby eliminating the need for hand-crafted features and time-consuming iterative optimization. At their core, these architectures share a common structure: shared encoder extracts features from input images, followed by de- coder correlating these features across views. Subsequent task-specific heads then regress pointmaps while optionally simultancously estimating auxiliary outputs like camera parameters and depth.
Despite their efficiency and fiexibility, these models still lag behind state-of-the-arr monocular geometry estimation approaches [77, 66, 23, 43] in capturing fine geometric detail and robustness. While architectures such as CUT3R [65] leverage large-scale data supervision and VGGT [61] integrates muuieent-based losses to capture fine details, the resulting depth and pointmap outputs remain coarse. Fine structures are frequently over-smoothed, object boundaries become blurred, and transparent or glossy surfaces are reconstructed with significant inaccuracies, yielding point clouds that lack crisp geometry. This persistent gap in performance raises crucial question: why do these feed-forwe weed-forward models consistently struggle to capture high-fidelit geometry? To answer this, we identify two primary factors that limit the geometric fidelity of these models: (1) Data quality constraints: Current real-world datasets providing accurate camera poses and high-fidelity depth remain limited. Existing non-synthetic depth labels are noisy [77] and predominantly biased toward indoor environments. (2) sequeee pointmap degradation: Inherent ambiguities in multi-view pointmap regression impede the network's 's ability to capture fine details over long sequences.
Motivated by these challenges, we investigate whether extensive unlabelled single-view data can be used to fine-tune pre-trained models to improve fine geometry recovery and robustness without sacrificing multi-view performance. This approach relaxes the constraint of high-quality data and long-sequence degradation. Recalling the common structure of recent feed-forward reconstruction models, we distill a oftt--ofthhe-n monocular geometry estimator (MoGe [66]) into the encoder using the diverse SA-1B IB dataset [25], while freezing the decoder to preserve its multi-view performance.
However, we observe that naïve encoder-only distillation, though beneficial for single-frame frame accuracy, leads to an increase in encoder feature norms. This drift pushes the features outside the range expected by the frozen decoder and undermines multi-view capability. To counteract this, we initially combined LoRA [20] with multi-view data replay, but the shift persisted. We therefore embed customized re-normalization layers within each LoRA block to dynamically correct this drift. Our solution achieves crisp depth predictions for single images while maintaining milti-view performance, all without the need for additional decoder fine-tuning.

---

To summarize, we propose a simple, effective, and general fine-tuning approach. By freezing the decoder and integrating a customized re-normalization LoRA adapter into the encoder, we distill the model from high-fidelity monocular teacher using a diverse dataset. Remarkably, the same implementation is applied four baselines-DUUScRR [68] pairwise prediction with relative depth, MASt3R'ss 's [28] pairwise prediction with metric depth, CUT3R's [64] recurrent crisper and more robust single-view depth, while preserving even slightly improving multi-view performance. Our contributions are threefold: (i) a general encoder-only distillation strategy that enhances local geometric detail and overall robustness in feed-forward 3D reconstruction models; (ii) a feature shift mitigation approach combining customized re-normalization LoRA with multi-view data replay to reduce distribution shifts over long sequences; and (iti) comprehensive evaluation on DUSt3R. MAS13R, CUT3R, and VGGT, demonstrating improved depth fidelity and correspondence accuracy while preserving global multi-view performance.
network, and VGGT's [61] parallel transformer-yieddinee
2 Related Work
Optimization-based Multi-view Reconstruction For over two decades, mainstream 3D recon- struction methods [19, 39] treated reconstruction as a large-scale optimization problem. The standard workflow [53, 1, 14] starts with exhaustive matching, triangulation, and bundle adjustment-strrctuuve from-motion (SfM))impleman
in toolkits such COLMAP [48]. SfM yields a sparse, metrically consistent point cloud that is densified by photo-consisten multi-view stereo (MVS). Early MVS relied on hand-crafted heuristics [16, 17]; recent variants adopt learned cost volumes [78, 18, 37] or neural-implic global optimisation [38, 15]. Deep learning has also upgraded SIM components: keypoints [82, 12], matchers [47, 31], even the full loop via differentiable Bundle Adjustment
(BA) [57, 62]. Yet these optimization- heavy pipelines remain iiabbration-ensiitiie and slow.
Feed-forward 3D Reconstruction Models Recent work removes explicit optimisation loops and predicts scene geometry in a single network pass. DUSt3R [68] pioneers this trend: from two uncalibrated images it produces a dense PointMap anchored in the first view, from which pose, depth, and correspondences are recovered through post post-processing. MASt3R [28] retains the same backbone but adds feature heads for matching. To extend beyond pairs, methods diverge into (i) recurrent architectures that process frames sequentially, e.g., CUT3R [65] and Span3R [60], and (ii) fully parallel attention across all views, e.g., MV-DUSS3R++ [55], FLARE [87], Fast3R [76], and VGGT [61]. Despite their strong implicit multi-view correspondence capability, these ford-forward models still struggle to capture sharp local geometry and reconstruct complex surfaces. Recent fine-tuning works [35, 86) rely on test-time optimization, requiring per-scene finetuning for each new instance. In contrast, our method involves a single, universal finetuning phase to create one model that generalizes to new scenes in a zero-shot manner.
Monocular Priors for Multi-view Geometry Leveraging monocular cues to assist multi-view problems has a long history. Dense monocular depth, surface normals and semantics have been used to assist SLAM [56, 9, 33, 32, 89), to fill in gaps dense reconstruction [44, 36, 72, 85), and to guide novel view synthesis [54, 74]. Other works explore monocular priors for relative pose [4, 84), PAP-RANSAC on depth maps [32], and monocular-assistee SfM [42]. More recently, significant
progress in monocular depth [77, 43, 23, 66] and normal estimation [83, 73, 3] has positioned these methods as strong priors for a variety of tasks.
Recently, monocular priors have also been injected into the new trend of feed-forward 3D reconstruc- tion networks. Align3R [34], Pow3R [21], and Mono3R [29] inject single-image depth (or sparse depth hints) to improve the pointmap prediction of DUSt3R-style models. However, they either (i) rely on an external geometric estimator or (ii) assume sparse, high-quality depth inputs. contrast, our approach keeps the heavy inference modules, or runtime overhead. Instead, we focus on training a stronger encoder that yields markedly more robust multi-view geometry without compromising speed.
pipeline fully self-contained:: we do not introduce any extra
3

---

10
View1
View2
0.09
Rep. Err. (pix) 0.08
Viewl
L2 Dist. Err.
View2
Ref View
0.07
Frame Index
(a) Scale Uncertainty Illustration
(b) Error metrics.
Figure Analysis of scale uncertainty and error metries. (a) Two views of red cube are connected by blue epipolar line. Gaussian distributions overlaid on a foreground point (green) and background point (yellow) illustrate their respective scale uncertainties, with the foreground exhibiting notably larger epipolar dispersion after projection. (b) Reprojection error and Euclidean distance loss are computed for 10 inputs processed by VGGT [61], with 1,000 samples drawn from Hypersim.
3 Method
3.1
Observations and Challenges
Our analysis reveals two main challenges in existing datasets and long sequence scenarios that critically affect training of geometry regression heads:
Data Scarcity. Existing datasets suffer from depth [77] and pose noise, and the limited availability of multi-view data further restricts the model's ability generalize. These noisy and insufficient labels hinder the model's capacity to capture fine details and to robustly adapt to diverse scenarios.
Long-Sequence Degradation. Long sequences introduce additional issues for pointmap regression: (1) Coupled Prediction: Although DUSt3R [68]'s multi-view pointmap regression has enabled feed- forward 3D reconstruction, it inherently couples pose and depth estimation in pointmap regression, injecting pose regression error into the geometry heads. (2) Drift: As the views progressively move further away from the initial reference frame, progressive drift becomes inevitable. This drift results in increasing errors on non-reference views and negatively affects the preservation fine structural details. (3) Scale Uncertainty: During training, both predicted and ground-troe -tunt pointmap require normalization to ensure scale consistency. However, this scale uncertainty tends erode fine foreground boundary along the epipolar line in views beyond the first frame. This phenomenon is illustrated Figure 2a and mathematically validated the appendix.
Consequently, pointmap regression introduces substantial errors in non-reference views, as evidenced by the pronounced reprojection error Figure 2b. Although CUT3R [65] leverages extensive depth supervision and VGGT [61] employs gradient-based loss to refine local geometry-with both methods incorporating dedicated self-view pointmap depth estimation heads-the resulting outputs remain relatively coarse. We suspect that the multi-view pointmap regression undermines the performance of these self-view estimation heads, thereby limiting the model's ''s ability to capture fine-grained details.
These observations not only underscore the necessity for high-quality supervision from diverse datasets but also highlight the inherent challenges associated with multi-view pointmap regression.
3.2 Fin3R
AAthoough VGGT [61] circumvents an explicit normalization step by implicitly inferring the prediction scale, it does not entirely resolve the inherent scale uncertainty in SIM.

---

(a) Input Image
(b) VGGT Avg: 9.61
(c) LoRA Only Avg: 10.53
(d) LoRA+Replay Avg: 10.34
(e) Full Avg: 9.73
Figure 3: Heatmaps show spatial variations in Le norms of encoder patch tokens across configurations. "Avg"n is the average norm of the feature map, and (e) Full indicates the full model with re-nommalization LoRA and multi-view data replay.
Labeled Multi-View -10%
Based on these observations, we introduce Fin3R-our solution that integrates lightweight fine-tuning stage simultancously address both challenges by monocular knowledge distillation.
Pointmap
LoRA
Head
Unlabeled Single-
Decoder
Self-View
Head
3.2.1 Encoder-only Distillation
~90%
We aim to enhance our model's capability to cap- ture fine details and complex surface geometries while preserving its multi-view performance. Re- call that feed-forward 3D reconstruction models typically consist a shared encoder, which ex- tracts features from input images, followed by decoder that correlates these features across views. We contend that the limitations detail recovery primarily originate from the encoder. Therefore, we enrich the encoder using robust monocular teacher [66], distilled large and diverse dataset [25]. This strategy is designed to improve local geometric detail recovery without compromising the decoder's proven matching capabilities.
Data
Pipeline of our method. Green dashed lines denote pointmap supervision; pur- ple dashed lines indicate distillation supervision.
Figure 4:
3.2.2 Monocular Finetuning Needs Feature Re-normalization
An initial exploration into naïve encoder Distillation-thrugee full parameter fine-tuning- that while the single-view geometric details were significantly improved, the fine-tuning adversely affected the multi-view matching capability even when we froze the decoder. Our early attempts to alleviate this issue involved leveraging LoRA and multi-view data replay. However, these strategies only partially mitigated the problem, as the degradation in multi-view performance persisted.
A closer examination of the model revealed key culprit: single-view distillation led to continuous increase in feature norms, as shown Figure 3. This norm shift pushed the feature beyond the range expected by the frozen decoder, thereby impairing multi-view matching. To directly address this challenge, we propose refined integration of LoRA with a re-normalization strategy specifically designed constrain feature norm drift. Concretely, given an original weight matrix W and its corresponding LoRA update AW, we re-normalize the combined weight after each update as follows:
w' (W + AW) ||W
I + AW]
Here, || denotes the L2 norm. This operation ensures that the updated weight W' maintains the original norm ||W||a ||,, thereby preserving the distribution of feature activations that the frozen decoder expects. As a result, we retain the crucial multi-view matching capability while still obtaining the benefits of enhanced local geometry recovery from self-view distillation. Although this method necessarily sufficient to address all feature shifts, we found it generally effective in most cases.
3.3 Training
We optimize two loss functions computed over images indexed by i in the training set. For each image, the monocular distillation loss refines single-view details by aligning the predicted depth D; with the high-fidelity pseudo-label D. provided by a monocular teacher, weighted by the aleatoric
(() uncertainty 8P; it is defined as distill 3. Dr D Alog sP. The pointmap regression loss

---

enforces robust multi-view matching while mitigating potential feature shift; to ensure this loss is applied only to multi-view samples, we introduce an indicator function 1 (i) that equals 1 if the
i-th image belongs to the multi-view dataset and 0 otherwise, and define the loss as ((s) pointmap
m (i) (8P1IP: P
1 A Algg (8.). The overall training objective the average loss over all N
images, given by C = # E (Calamn + c poontmup) with the uncertainty terms modeled as in [24].
4 Experiment
Implementation Details. We use MoGe [66] as the teacher model for pseudo-label generation. Since the depth predicted by MoGe is affine-invariant, we subtract the shift in the z-component and then apply the normalization used in DUS13R. For DUSt3R [68], we use 2-view data with distillation supervision applied exclusively
the view-1 pointmap head for distillation loss. In contrast, CUT3R [65] and VGGT [61] utilize 2-8 views, with supervision on either the self-view head or the depth head. During each epoch, we sample 20,000 images from SA- AA-BB [26], 1,000 from Hypersim [46], and 1,000 from TartainAir [69]. Training runs for 10 epochs on four NVIDIA L20 GPUs over a single day. Further implementation details are provided in the appendix.
Evaluation Protocol. We evaluate our approach across three settings: single-view, two-view, and multi-view. In the single-view setting, we focus on monocular depth estimation. The two-view configuration evaluates relative pose estimation, where we extract pairwise correspondences using DUS13 [68]'s matching method for VGGT. the multi-view setting, we perform multi-view depth estimation, pointmap estimation, and pose estimation. Since CUT3R [65] is designed for long sequences and unsuitable for pairwise correspondences, we remove it in the two-view evaluation.
Table I:
Quantitative results for monocular depth estimation. "+Ours" denotes the integration of our fine-tuning, and MoGe is the teacher model. Best results in each session are highlighted in bold.
Scale-invariant relative depth
NYUv2
KITTI
ETH3D
iBims-1
DDAD
DIODE
HAMMER
Average
Method
Rel
6
Rel 67
Rel
Rel
6
Rel
6
Rel.
47
Rel 67
Rel 1 6]
DUSB3 [68]
3.83
97.7
7.64
91.1
5.35
95.9
3.97
96.5
17.34 75.5
6.85
92.4
4.23
96.9
7.03
92.3
D
3.68
97.8
6.02
94.7
4.41
96.8
3.47
97.4
13.11
83.1
4.70
95.3
3.66
98.7
5.58
94.8
CUT3R [65]
3.73
97.9
7.20
91.7
4.69
96.4
4.06
96.4
15.62
76.9
5.93
93.2
4.01
98.2
6.46
92.9
CUT3R+Ours
3.68
97.9
5.93
94.7
4.67
96.6
3.46 97.7
13.12
82.3
5.08
94.8
3.20
99.3
5.59
94.7
VGGT [61]
3.14
98.3
5.83
94.1
3.64
97.5
3.61
96.8
81.3 13.74
5.24
94.5
5.18
95.2
5.77
94.0
VGGT+Ours
3.10
98.3
97.2 4.59
3.07
98.7
2.73 98.2
10.65
88.1
3,59
96.7
2.31
99.5
4.29
96.7
MoGe [66]
3.02 98.5
97.4 4.39
2.96
98.9
2.65 98.2
90.0 9.64
3.23
97.4
3.09
98.2
4.14
96.9
Metric depth
MASB3 [28]
10.79
89.6
55.11
10.9
46.91
21.3] 18.65 61.5 15.5
21.3
62.90
4.3
55.34
18.3
18.3
97.62
5.6
49.62
30.2
MASt3R+Oure
11.71
88.4
10.69
89.1
26.30 56.0
11.29 86.3
26.50
55.5
22.84
50.1
83.89
24.5
27.60
64.3
MoGe-2 [67]
6.92 96.7
16.72
70.1
10.92 88.2
81.1 14.08
15.82
74.1
15.97
71.3
23:30
68.5
14.82
78.6
4.1
Monocular Depth Estimation
We follow the evaluation of MoGe [66] evaluate our method using standard metrics: relative absolute difference (rel) and the ó₁ score. Specifically, rel = + E 14:477 denote the predicted and ground truth depths, respectively, while 8, represents the percentage of predictions satisfying max (#: di τ. 1.25. Table presents quantitative results for affine-invariant
di - where d, and dt
depth evaluation. The table shows that our integrated models consistently achieve lower relative depth error and higher &j scores. Figure 5 shows the qualitative comparison between baselines and the results from our tine-tuning method. After fine-tuning, our method improves the model's ability to capture fine details and complex surfaces such as transparent ones. Fine-tuned VGGT performs almost as well as the sfate-of-the-ar expert model, MoGe. Interestingly, we observe that although DUSt3R's 's depth estimates rank last among the evaluated models, they exhibit the sharpest boundaries compared with the other two baseline models. This likely because CUT3R and VGGT trained on long sequences and are consequently more affected by the long- sequence degradation

---

Input
GT
DUS0R
DUSI3R
CUT3R
CUT3R
VGGT
VGGT
Figure 5: Depth prediction across baseline models. * indicates integration with our method.
discussed Section 3.1. We also present the fine-tuned MASt3R model with metric depth prediction, demonstrating that our method is capable of handling not only relative depth prediction but also metric depth estimation.
4.2 Relative Pose Estimation
Table 2 summarizes evaluation of relative pose estimation the ScanNet dataset [10]. Following
[47], we assess performance using area-under wera--nder-tte-uureeee the-cvve (AUC) metrics computed at thresholds of 5, 10, and 20 degrees. The results indicate that fine-tuning method consistently improves the baseline model correspondence by improving the geometry. In particular, our fine-tuned VGGT model outperforms Reloc3r [11] the 5 threshold, despite Reloc3R being designed only for pose regression and lacking geometric modeling capability.
4.3 Multi-view Depth Estimation
Following CUT3R [65], we evaluate the performance of our method on video depth estimation. Table 4 summarizes the results, demonstrating that our method preserves multi-view consistency and improves single-view accuracy. The fine-tuned versions of CUT3R and VGGT consistently outperform their respective baselines across datasets spanning diverse domains. Note that VGGT is not trained on dynamic datasets, so its performance bottleneck may stem from dataset limitations rather than our fine-tuning method.

---

Table 2: Relative Camera Pose Evaluation the ScanNet1500 [10, 47] datasets. "Ours" in- dicates the integration of our distillation method. Better results are highlighted in bold.
Table 3: Quantitative Results for Multi-
view Pose Estimation on RealEstatelOk [88]. "Ours" is the fine-tuned model using our method. Better results are highlighted in bold.
Methods
ScanNet1500
ACC@10
AUC
Methods
RealEstaae 10k
RRA@5
@TA@5
AUC 30
Efficiett LoFTR
19.20
37.00
53.60
ROMA [13]
28.90
50.40
68.30
DUSI3R [68]
94.01
42.39
62,40
NoPoSplat [80]
31.80
$3.80
71.70
Ourrs
95.41
47.07
64.81
DUS(R [68]
31.61
53.77
70.99
MAS13R [28]
94.89
52.21
73.45
DUSERR+Orr
33.73
55.67
72.66
MuSr3R+Ours
95.02
53.74
73.87
MASI3 [28]
37.60
59.96
76.24
CUT3R [65]
96.66
61.66
78.95
MASI3R+Ours
37.93
60.21
76.68
CUT3
96.99
62.15
79.13
VGGT [6]]
VGGT [61]
95.28
53.14
74.18
VGGT+Ours
35.21
56,70
72.80
Reloc3r [11]
34.79
58.37
75.56
VGGT+Ours
96.27
56.54
75.35
Table 4: Results for Video Depth Estimation. The arrows (1/1) indicate whether lower or higher values are better. Best results are highlighted in bold.
Method
ETH3D [49]
T&T [27]
KITTI [58]
Sintel [6]
Bonn [40]
rel 4
81 f
rel 4
81 f
rel +
61 f
rel +
81 1
rel +
81 f
CUT3R [65]
0.126
83.1
0.209
69.5
0.123
87.4
0.428
47.4
0.077
93.9
Ours
0.130
82.8
0.180
76.2
0.112
89.8
0.406
58.4
0.062
96.8
VGGT [6T]
0.044
97.9
0.137
85.3
0.072
96.5
0.301
68.4
0.052
97.3
VGGT+Ours 0.041
99.2
0.115
88.0
0.069
96.6
0.252
72.7
0.048
97.5
4.4 Multi-view Pointmap Estimation
Table presents the multi-view reconstruction performance on the 7Scenes [52] and NRGBD [2] datasets following Spann3R [60]. Note that DUS(3R employs global alignment, while the other methods operate in a feed-forward manner. Because both DUSt3R and VGGT produce scale-invariant point maps, apply Umeyama alignment [59] to align scale. We report mean and median values for three metrics: accuracy (Acc), completeness (Comp), and normal consistency (NC). The results indicate that models enhanced with our distillation method consistently achieve lower Acc and Comp as well as improved NC scores across most baselines. Qualitative results can be found at Figure
VGGT
VGGT+Ours
VGGT
VGGT
VGGT+
VGGT+Ours
Figure 6: Additional results of pointmap estimation.

---

Table 5: Pointmap Regression 7-Scenes [52] and NRGBD [2]
Datasets. "+Ours" represents
the integration of our distillation method. The best results at each session are bold.
7-Scenes [52]
NRGBD [2]
Acc|
Compl
NC1
Ace!
Compl
NC1
Method
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
D [68]
0.026
0.011
0.033
0.018
0.641
0.725
0.050
0.030
0.036
0.019
0.851
0.983
D
0.024
0.009
0.029
0.015
0.641
0.726
0.043
0.027
0.030
0.017
0.863
0.986
CUT3R [65]
0.024
0.011
0.029
0.010
0.664
0.758
0.075
0.031
0.046
0.019
0.828
0.966
Ours 0,025
0.012
0.026
0.010
0.666
0.762
0.075
0.028
0.043
0.019
0.833
0.968
VGGT [61]
0.017
0.006
0.024
0.011
0.645
0.727
0.019
0.012
0.018
0.009
0.914
0.992
VGGT+Ours
0.012
0.006
0.023
0.011
0.651
0.739
0.021
0.014
0.020
0.011
0.921
0.993
Table 6: Pointmap Regression on the DTU and ETH3D datasets. The arrows (1/)) indicate
whether lower or higher values are better. Best results highlighted in bold.
Method
DTU [22]
ETH3D [50)
Acc. 4
Comp.
N.C. 1
Acc.
Comp. L
N.C. 1
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Pi3 [71]
1.151
0.622
1.793
0.629
0.668
0.754
0.194
0.130
0.220
0.135
0.867
0.965
VGGT [61]
1.187
0.715
2.229
1.309
0.694
0.779
0.290
0.196
0.371
0.230
0.839
0.932
VGGT+Ours
0.948
0,520
1.879
0.905
0.699
0.787
0.209
0.112
0.170
0.085
0.861
0.972
Table 8: Ablation Study on Fine-tuning Strat- egy. Our proposed components consistently im- prove matching performance.
Table Ablation Study for Distillation Mod-
ule. Combining all our strategies yields the high- est accuracy.
Method
AUC@S
AUC@10
AUC@20
Tahel Sups.
Mono. Tiacher
SA-1B
Rel (3)
Ag (T)
Acc (L)
VGGT
28,40
47.36
61.51
94..1
0.017
(1) Dec. Fall
28.42
51.59
67.30
(2) -Fnc. Full
32.06
52.29
68.04
5.00
95.1
0.013
Full
/
4.35
96.3
0.012
(4) -Eac. Lora
32.96
54.21
70.40
(5) -Enn. LoraeRe-
35.21
36,70
72,80
Table 6 also compares our method with the concurrent model Pi3 [71] on the DTU [22] and ETH3D [50] datasets using the pointmap head of VGGT. Our method delivers comparable per- formance while requiring significantly fewer resources for fine-tuning.
4.5 Multi-view Pose Estimation
Table 3 summarizes the performance of baseline models and our fine-tuned methods on the RealEstate 10k [88] dataset. We evaluate performance using three metries: Recall of Relative Angle (RRA @5), Recall of Relative Translation (RTA @55), and AUC@30. These results indicate that our method primarily refines the geometry head without significantly affecting the pose head, We attribute this improvement to the decoder functioning as an implicit feature matcher, which allows it to leverage the enhanced feature details for more accurate pose prediction.
4.6 Ablation Study
Distillation Strategy. Table shows that our distillation pipeline incrementally enhances geometric accuracy on VGGT [61]. The first two columns report mean monocular depth metrics (see Table 1). while the final column details the 7-Scenes [52] accuracy. The top row represents VGGT model without fine-tuning, which can benefit from single-vier distillation (second row) on subset of training datasets (see appendix) with supervision from dataset depth labels. Replacing the depth labels with monocular teacher further improves performance, and changing the dataset to SA-1B yields the best performance. Together, these results highlight that monocular finetuning with high-quality pseudo-labels from the diverse dataset improves both single-vion view and multi-view accuracy.

---

Finetuning Strategy. Table evaluates our fine-tuning strategy on ScanNet relative pose estimation using VGGT. Lines (1), (2), and (3) demonstrate that fine-tuning the decoder with monocular data harms multi-view consistency, highlighting the effectiveness of our encoder-only fine-tuning design. Lines (2), (4), and (5) show that full full-parametric fine-tuning improves the baseline's performance, while integrating the LoRA module further refines the representations. Notably, the re-normalization LoRA mitigates norm drift, leading to progressively improved matching performance. These results confirm that our modifications effectively reduce domain shifts while enhancing both fine detail recovery and multi-view consistency.
4.7 Discussion
During our experiments, observed that models like VGGT often produce blurry geometry accompanied by low confidence scores, as shown in Fig 8. After our fine-tuning, the model becomes more confident in its predictions and is capable of gener- ating sharper geometry with better calibrated confidence. We attribute this improvement primarily to the incorpo- ration of unlabeled datasets, which enhance the model's robustness and overall performance. This underscores the necessity of including in-the-wild data alongside high- quality datasets during training to achieve optimal results.
Confidence and Fine Details:
Input
GT Depth
VGGT Depth
VGGT Depth
Cross-Head Generalization via a Robust Encoder While our training only distills the depth head of VGGT with pseudo-labed our findings indicate that the pointmap head exhibits similar improvements (see Figure 7). This demonstrates that a robustly trained encoder benefits down- stream heads even without direct supervision.
VGGT Pointmap VGGT Pointmap
Figure 7: Depth from depth head and pointmap head of VGGT. * denotes our fine-tuning model.
Position Our Method Our approach is lightweight, resource-efficient fine-tuning strategy for feed-forward reconstruction models. By carefully fine-tuning the encoder, it avoids the resource-intensee decoder tuning, which typically requires long-sequence inputs from diverse datasets with large batch sizes. Although further decoder tuning may yield additional gains, our method minimizes complexity without compromising quality. As 3D vision enters the of large models, hope approach and analysis offer valuable insights into fine-tuning 3D large models with limited resources.
Input Image
VGGT Depth
Fine-tuned Depth
VGGT Conf.
Fine-tuned Conf.
Figure 8: Visualization of depth and confidence predictions. The confidence color ranges from dark purple (low confidence) bright yellow (high confidence).
5 Conclusion
We introduced Fin3R, lightweight fine-tuning approach that leverages monocular distillation and re-nomalization LoRA to enhance fine geometry and robustness in feed-forward 3D reconstruction models. Extensive experiments on DUSt3R, MASt3R, CUT3R, and VGGT validate that our method sharpens local details while preserving robust cross-vies view ability. Our results highlight the effective- ness and efficiency of our fine-tuning strategy, achieving notable performance gains while requiring minimal computational overhead.
10

---

This work is supported by Hong Kong Research Grant Council General Research Fund (Grant No. 17213825). Weining Ren is supported by Hong Kong PhD Fellowship Scheme (HKPFS).
Acknowledgments
References
[1] Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M Seitz, and Richard Szeliski. Building rome in day. Communications of the ACM, 2011.
[2] Dejan Azinović, Ricardo Martin-Brualla, Dan B Goldman, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. In CVPR, 2022. 8,
[3] Gwangbin Bae and Andrew J. Davison. Rethinking inductive biases for surface normal estima- tion. In CVPR, 2024.7
[4] Daniel Barath and Chris Sweeney. Relative pose solvers using monocular depth. In ICPR, 2022.
3
[5] Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARKitscenes a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. In NeurlPS, 2021. 23
[6] D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012. 8
[7] Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2, 2020. 23
[8] Yue Chen, Xingyu Chen, Anpei Chen, Gerard Pons-Moll, and Yuliang Xiu. Feat2gs: Probing visual foundation models with gaussian splatting. In CVPR, 2025. 24
[9] Jan Czanowski, Tristan Laidlow, Ronald Clark, and Andrew J Davison. Deepfactors: Real-time probabilistic dense monocular slam. RAL, 2020.
[10] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 7, 8, 23
[11] Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. arXiv preprint arXiv:2412.08376 2024. 8
[12] Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Mare Pollefeys, Josef Sivic, Akihiko Torii, and Torsten Sattler. D2-net: trainable enn for joint description and detection of local features. In CVPR, 2019. 3
[13] Johan Edstedt, Qiyu Sun, Georg Bökman, Märten Wadenbäck, and Michael Felsberg, RoMa: Robust Dense Feature Matching. CVPR, 2024.
[14] Jan-Michael Frahm, Pierre Fite-Georgel, David Gallup, Tim Johnson, Rahul Raguram, Changehang Wu, Yi-Hung Jen, Enrique Dunn, Brian Clipp, Svetlana Lazebnik, et al. Building rome on a cloudless day. In ECCV, 2010. 3
[15] Qiancheng Fu, Qingshan Xu, Yew Soon Ong, and Wenbing Tao. Geo-neus: Geosetry-consistent neural implicit surfaces learning for multi-view reconstruction. NeurIPS, 2022.
[16] Yasutaka Furukawa, Carlos Hernández, et al. Multi-view stereo: A tutorial. Foundations and Trends@ in Computer Graphics and Vision, 9(1-2):1-148, 2015.
[17] Silvano Galliani, Katrin Lasinger, and Konrad Schindler. Massively parallel multiview stereopsis by surface normal diffusion. In ICCV, 2015.
[18] Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong Tan, and Ping Tan. Cascade cost volume for high-resolution multi-view stereo and stereo matching, In CVPR. 2020. 3
11

---

[19] Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cam- bridge University Press, 2000.
[20] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 2
[21] Wonbong Jang, Philippe Weinzaepfel, Vincent Leroy, Lourdes Agapito, and Jerome Revaud. Pow3r: Empowering unconstrained 3d reconstruction with camera and scene priors. arXiv preprint 1rXXi::5033.7316,, 2025.
[22] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In CVPR, 2014. 9
[23] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion-based image generators for monocular depth estimation. In CVPR, 2024. 2,
[24] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? NeurIPS, 2017. 6
[25] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan- Lo, al. Segment anything. In ICCV, 2023. 2, 5, 23
[26] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Yen Lo, et al. Segment anything. In ICCV, 2023. 23
[27] Arno Knapitsch, Jaesik Park, Qian- Yi Zhou, and Vladlen Koltun. Tanks and temples: Bench- marking lcage-scale scene reconstruction. ACM TOG, 2017. 8
[28] Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In ECCV, 2024. 2, 3, 8
[29] Wenyu Li, Sidun Liu, Peng Qiao, and Yong Dou. Mono3r: Exploiting monocular cues for geometric 3d reconstruction. arXiv preprint 1rX1v22504.13419, 2025. 3
[30] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In CVPR, 2018. 23
[31] Philipp Lindenberger, Paul-Edouard Sarlin, and Mare Pollefeys. Lightglue: Local feature matching at light speed. arXiv preprint arXiv:2306. 13643, 2023. 3
[32] Sheng Liu, Xiaohan Nie, and Raffay Hamid. Depth-guided sparse movies and tv shows. CVPR, 2022.
for [33] Shing Yan Loo, Syamsiah Mashohor, Sai Hong Tang, and Hong Zhang. Deeprelativefusion:: Dense monocular slam using single-image relative depth prediction. IROS, 2021.
[34] Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic videos. arXiv preprint arXiv:2412.0307
2024. 3
[35] Ziqi Lu, Heng Yang, Danfei Xu, Boyi Li, Boris Ivanovic, Marco Pavone, and Yue Wang. Lora3d: Low-rank self-calibration of 3d geometric foundation models. arXiv preprint arXiv:24/2.077746, 2024.
[36] Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. ACM TOG, 2020. 3
[37] Zeyu Ma, Zachary Teed, and Jia Deng. Multiview stereo with cascaded epipolar raft, In ECCV, 2022.
12

---

[38] Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In CVPR. 2020. 3
[39] Onur Özyeşil, Vladislav Voroninski, Ronen Basri, and Amit Singer. A survey of structure from motion*. Acta Numerica, 26:305-364 2017. 3
[40] E. Palazzolo, J. Behley, P. Lottes, P. Giguère, and C. Stachniss. ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals. In IROS, 2019,
[41] Linfei Pan, Daniel Barath, Mare Pollefeys, and Johannes Lutz Schönberger. Global Structure- from-Motion Revisited. In ECCV, 2024.
[42] Zador Pataki, Paul-Fdouard Sarlin, Johannes L. Schöneeger, and Mare Pollefeys. MP-SfM: Monocular Surface Priors for Robust Structure-fromMMotiin. In CVPR, 2025. 3
[43] Luigi Piccinelli, Yung- Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. CVPR, 2024. 2,
[44] Matia Pizzoli, Christian Forster, and Davide Scaramuzza. Remode: Probabilistic, monocular
dense reconstruction in real time. In ICRA, 2014. 3
[45] Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In ICCV, 2021. 23, 24
[46] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. ICCV, 2021. 6, 23
[47] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In CVPR, pages 4938 4947, 2020. 3,
8
[48] Johannes Lutz Schönberger and Jan-Michael Frahm. Stracture-from-motion revisited. In CVPR,
2016. 2,3
[49] Thomas Schöps, Torsten Sattler, and Mare Pollefeys. BAD SLAM: Bundle adjusted direct RGB-D -D SLAM. In CVPR, 2019.
[50] Thomas Schops, Johannes L. Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and Andreas Geiger. multi-view stereo benchmark with high-resolution images and multi-camera videos. CVPR, pages 3260-3269, 2017.
[51] Philipp Schröppel, Jan Bechtold, Artemij Amiranashvili, and Thomas Brox. A benchmark and a baseline for robust multi-view depth estimation. In 3DV, 2022. 23
[52] Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. In CVPR, pages 2930-2937, 2013. 8.
[53] Noah Snavely, Steven M Seitz, and Richard Szeliski. Photo tourism: exploring photo collections in 3d. In ACM siggraph 2006 papers, pages 835-846. 2006. 3
[54] Jiuhn Song, Sconghoon Park, Honggyu An, Scokju Cho, Min-Seop Kwak, Sungjin Cho, and Seungryong Kim. Därf: boosting radiance fields from sparse inputs with monocular depth adaptation. In NeurlPS, 2023. 3
[55] Zhenggang Tang, Yuchen Fan, Dilin Wang, Hongyu Xu, Rakesh Ranjan, Alexander Schwing, and Zhicheng Yan. Mv-dust3r4: Single-stage scene reconstruction from sparse views in seconds. arXiv preprint arXiv:2412.069974 2024. 2,
[56) Keisuke Tateno, Federico Tombari, Iro Laina, and Nassir Navab. Con-slam: Real-time dense monocular slam with learned depth prediction. In CVPR, 2017.
13

---

[57] Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. NeurIPS, 2021. 3
[58] Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, and Andreas Geiger. Sparsity invariant cnns. In 3DV, 2017. 8
[59] Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE TPAMI, 1991.
[60] Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint arXiv:2408. 16061, 2024. 2, 3,
[61] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In CVPR, 2025. 2, 3, 4, 6, 8, 9, 24
[62] Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In CVPR, 2024. 3, 23
[63] Jianyuan Wang, Christian Rupprecht, and David Novotny. Posediffusion: Solving pose estima- tion via diffusion-aided bundle adjustment. In ICCV, pages 9773-9783, 2023. 24
[64] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state. arXiv preprint arXiv:2501. 12387, 2025. 2,3
[65] Qianqian Wang. Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state, 2025. 2, 3, 4, 6, 7, 8, 9, 24
[66] Ruicheng Wang. Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge:
Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. arXiv preprint arXiv:2410.191 /9/1, 2024. 2, 3, 5, 6, 23, 29
[67] Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv. Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. NeurIPS, 2025. 6
[68] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024. 2, 3, 6, 8, 9,
[69] Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. IROS, 2020. 6, 23
[70] Yifan Wang, Xingyi He, Sida Peng, Dongli Tan, and Xiaowei Zhou. Efficient LoFTR: Semi- dense local feature matching with sparse-like speed. In CVPR, 2024. 8
[71] Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. 33: Scalable permutation-eqquiaaian visual geometry learning. arXiv preprint 2507.13344 2025. 9
[72] Yi Wei, Shaohui Liu, Yongming Rao, Wang Zhao, Jiwen Lu, and Jie Zhou. Nerfingmvs: Guided optimization of neural radiance fields for indoor multi-view stereo. In ICCV, 2021. 3
[73] Guangkai Xu, Yongtao Ge, Mingyu Liu, Chengxiang Fan, Kangyang Xie, Zhiyue Zhao Hao Chen, and Chunhua Shen. What matters when repurposing diffusion models for general dense perception tasks? arXiv preprint arXiv:2403.0609, 2024.
[74] Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Mare Pollefeys. Depthsplat: Connecting gaussian splatting and depth. In CVPR, 2025.
[75] Jianing Yang, Alexander Sax, Kevin J. Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. In CVPR, June 2025. 2
14

---

[76] Jianing Yang, Alexander Sax, Kevin Liang. Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images one forward pass. arXiv preprint arXiv:2501.1392. 2025. 3
[77] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv:2406.09414, 2024. 2, 3,4
[78] Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: Depth inference for unstructured multi-view stereo. In ECCV, 2018. 3
[79] Yao Yao. Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. CVPR, 2020. 23
[80] Botao Ye, Sifei Liu, Haofei Xu, Li Xueting, Mare Pollefeys, Ming-Hsuan Yang, and Peng Songyou. No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint arXiv:2410.24207 2024. 8
[81] Chandan Yeshwanth, Yueh- Chen-Cheng Liu, Matthias NieBner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. In ICCV, 2023. 23
[82] Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal Fua. LIFT: Learned Invariant Feature Transform. In ECCV, 2016. 3
[83] Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from single image. In ICCV, 2023.
[84] Yifan Yu, Shaohui Liu, Rémi Pautrat, Marc Pollefeys, and Viktor Larsson. Relative pose estimation through affine corrections of monocular depth priors. CVPR, 2025.
[85] Zehao Yu, Songyou Peng. Michael Niemeyer, Torsten Sattler, and Andreas Geiger. Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction. In NeurIPS, 2022.
[86] Yuheng Yuan, Qiuhong Shen, Shizun Wang. Xingyi Yang, and Xinchao Wang. Test3r: Learning reconstruct 3d at test time. NeurlPS, 2025. 3
[87] Shangzhan Zhang, Jianyuan Wang, Yinghao Xu, Nan Xue, Christian Rupprecht, Xiaowei Zhou, Yujun Shen, and Gordon Wetzstein. Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views. CVPR, 2025. 2,
[88] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnifi- cation: learning view synthesis using multiplane images. ACM TOG, 2018. 9, 23
[89] Zihan Zhu, Songyou Peng. Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Mare Pollefeys. Nicer-slam: Neural implicit scene encoding for rgb slam. In 3DV, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction clearly reflect the main contribution the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Our custom ne-ormalization Lora can not be merged into models like vanilla Lora. Although our method enhances fine geometric details, its boundary accuracy remains inferior that of MoGe, which produces sharper results. A mixed-teacher strategy or more dedicated distillation design may offer further improvements. Additionally, the current model supports only 512/518 resolution, and scaling higher resolutions remains challenge for future work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
16

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our method is simple and easy integrate any baselines. We provide all details need to reproduce our results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers have some path reproducing verifying the results.
17

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification:
We will open-source the code and the model after the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips..
seguiidde//oddduuiissss wwtpemisssssooooiiiooiooooooeeee onPoliccy for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: gwt////i//////e////g
/ //nips.
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All details for training and testing are included in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification:
The error bar is not common in forward 3D reconstruction.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: It's stated in the Experiment Section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https /ttee:///eee...eeeeeeee cetiiiieeeeeiiiiiceeeeeeeeeee
Answer: [Yes]
Justification:
We follow the ethics of NeurlPS in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction..
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This research enhances 3D reconstruction, benefiting robotics, AR/VR, and accessibility by using diverse single-view data. However, improved realism carries risks of misuse in generating synthetic content, and potential biases from teacher models or datasets must be addressed. Responsible development and ethical guidelines are crucial to navigate these impacts. This work contributes to more detailed and robust 3D perception.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We follow the license of VGGT, using Attribution-NonCommmee 4.0 Inter- national license.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: The paper does not release new assets.
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification: Paper does not involve crowdsourcing nor research with human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA],
Justification: LLM is only used for writing and editing.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttp///erspp/////ee cc/Conffeeecces/2255 /LLM) for what should or should be described.
22

---

Appendix
A Experiment Details
A.1 Training Details
In all experiments, we set both the rank and alpha of LoRA to 8.
DUS13R. Since DUSt3R doesn't 't have dedicated self-view head for canonical view estimation, we use DUSt3R's first viewpoint pointmap regression head for distillation. Training is performed at a resolution of 512 width, with aspect ratios (c.g.. 16:9, 4:3) randomly sampled for each batch. During each epoch, we randomly sample 20,000 pairs from the SA-1B [25] dataset, 1,000 pairs from the Hypersim [46] dataset, and 1,000 pairs from the TartanAir [69] dataset. The model is fine-tuned for 10 epochs. The learning rate is initialized at le-4 with one-epoch warm-up phase and gradually decayed to a minimum of le-6. A batch size of 2 per GPU is used, and gradients are accumulated over iterations to achieve an effective batch size of 64.
CUT3R/VGGT. We compute the distillation loss using the self-view pointmap head for CUT3R and the depth head for VGGT, following the same dataset configuration as in DUS:3R fine-tuning. CUT3R is trained at a resolution of 512 width, while VGGT is trained at a resolution of 518 width. The model is fine-tuned for 10 epochs with an initial learning rate of le-4, which is warmed up for one epuch and then gradually decayed to a minimum of 1e-6. Additionally, the sequence length is dynamically selected between 2 and 8, with the product of batch size and sequence length fixed at 8. The accumulation iteration is changed accordingly to ensure an effective total batch size of 64.
A.2 Evaluation Details
Monocular Depth Estimation. We follow the evaluation protocol from MoGe [66] to assess our models. For DUSt3R, we duplicate the input images and use the value from the view-1 pointmap head as the predicted depth. For CUT3R, depth is obtained from the z value of the self-view pointmap head, and for VGGT, we use the output of the depth head. Since these models are trained at resolutions of 512 width (or 518 width for VGGT), the original images are resized accordingly for evaluation. Although this differs from the standard MoGe protocol, which evaluates at higher resolutions, we ensure that both the base model and our fine-tuned models share the same settings. Furthermore, we exclude evaluation datasets such as Sintel and Spring since DUSt3R and VGGT are not designed for dynamic scenes.
Two-view Evaluation. We extract two-view correspondences using the nearest neighbor matching strategy from DUSt3R, which leverages geometric distance and is well-suited for assessing our enhanced geometry. We avoid using VGGT's tracking head for matching for two main reasons. First, the current release of VGGT's tracking head does not perform as well as the version reported in the original paper'. Second, in the Scannet- 1500 relative pose estimation task, our geometry-baa based correspondence method outperforms the tracking- based approach described the original VGGT paper. Furthermore, we plan fine-tune the tracking head using our stronger encoder, which believe can provide more accurate and robust features to further enhance tracking performance.
Multi-View Pose Estimation. We evaluate our method primarily on the RealEstate 10k dataset [88], following the procedure in VGGSIM [62] that involves randomly sampling 10 frames from each sequence for pose evaluation. Since some of the original YouTube links in RealEstate 10k are unavailable, our evaluation is conducted 1,756 out of the original 1,800 scenes.
Ablation Mix Dataset. For the ablation study, we replace the SA-IB dataset [26] with mixed dataset composed of MegaDepth [30], CO3Dv2 [45], ARkitScene [5], Scannet++ [81]. Scannet [10], VirtualKIlTiv2 [7], BlendedMVS [79], and MhaiicTings33D [51]. Each dataset is equally weighted, providing coverage that is comparable to the DUSt3R training set.
23

---

Table S1: Quantitative results for multi-vies pose estimation on the C03Dv2 dataset. "Ours" signifies the integration of our finetuning method. Best results in each session are highlighted bold.
CO3Dv2
Methods
@RA@5
RTA @5
AUC@30
DUSIR [68]
80.49
75.22
81.03
DUSr3R+Ours
85.75
78.02
82.83
CUT3R [65]
70.83
64.39
74.10
+Ur3
70.89
63.76
73.74
VGGT [61l
95.20
84,28
88.35
VGGT+Ours
95.47
84.18
88.77
B Additional Experiments
B.1
Multi-view Pose On CO3Dv2
We also conduct experiments on multi- pose estimation using the CO3Dv2 dataset [45]. Follow- ing the evaluation protocol in PoseDiffusion [63], select the first 10 frames from each sequence for evaluation. The results are presented in Table $1. Our fine-tuning improves DUSt3R by refining the geometry- baased correspondence. However, the performance of CUT3R CO3Dy2 is negatively affected, and the impact on VGGT is marginal. We suspect this is primarily because CO3Dv2 is used train the pose head, causing it to strongly memorize the dataset.
B.2 Multi-view Feature Feat2GS
We further provide empirical evidence to demonstrate that our approach successfully maintains multi-view consistency Fear2GS [8] benchmark, which directly evaluates the quality of multi-view features for novel view synthesis. The results are shown in Table S2. Our method not only preserves but slightly improves multi-view performance, evidenced by the gains in PSNR and LPIPS. It is important to contextualize these numbers: performance on Feat2GS is typically concentrated within very narrow range (e.g., PSNR often between 19.40 and 19.70). This demonstrates method successfully improves single-view geometry while preserving the integrity of multi-view features.
Table S2: Quantitative comparison on Feat2GS [8] benchmark.
Goometry
Texture
All
PSNR"
SSIM
[PIPS]
PSNR1
SSIMI
IPPS
PSNRT
SSIMI
I.PISS]
DUSSR VOET
0.3181
18.05
0.6006
0.3221
0.3700
19.66
0.6588
0.3123
18.07
0.6003
0.3225
19.61
0.6510
0.3788
VGGT e.
19.70
0.6561
0.3115
18.10
0.6008
0.3224
19.66
0.6514
0.3781
C Additional Visualizations
We provide additional visualizations on diverse, in-the-wild data in Figures S1, S2, and S3 to demonstrate how our tine-tuning method robustly enhances the original baseline. More visualizations can be found in the Supplementary Video, which includes fly-through sequences of the multi-view reconstruction results.
24

---

Input
VGGT Depth
VGGT+Ours Depth
Figure S1: Additional Visualization of Depth Estimation Results: Input Images, Baseline (VGGT Depth), and Improved Method (VGGT+Ours Depth)
25

---

Input
DUS:3
DUS:3R Ours
Figure S2: Additional visualization of depth estimation results. From left to right: input image, baseline (DUSt3R), and our improved method (DUS),
26

---

Input
CUT3R
CUT3R Ours
DOTRF
Figure S3: Additional visualization of depth estimation results. From left to right: input image, baseline (CUT3R), and improved method (CUT3
27

---

D Proof for Long-Sequence Scale Uncertainty
Let 3D point in the world coordinate system be
p
y/
with multiplicative scale uncertainty modelled
p == (1+ 6)p,
8~
where å is small perturbation. Under rigid transformation characterized by rotation A and translation T, the unperturbed point the second view is given by
P2
Rp + T.
When the uncertainty is introduced, the perturbed second-vies point becomes
P2 Rp T = R (11+
+ 5)p] + T = P2 + (Rp).
We define the rotated coordinates by writing
Rp
where, due the relationship p2 -Rp + T. the first and third components satisfy:
a X2 Tu 8 Z2 T2.
X2 Y2
with P2
22
Assuming pinhole camera model with focal length f. the unperturbed horizontal image coordinate given by
f X2 Z2
For the perturbed coordinates we express
= X2 + 8a,
23 Zz 53.
Thus, the image coordinate under perturbation is
u(6) fX2 88a Z2 + 5
Our goal is to analyze the induced projection error,
Au u())
without using Taylor expansion. We begin by forming the exact difference:
Au (X2 +d)) f fx2
Za
By combining the terms over common denominator, have:
Au f (X2 + +8) X2 + 8 (2, 8,3)
Expanding the numerator yields:
(8)+ X2(22 + 03) X222 + X222 - 82 00 2z -x20)
Thus, the error simplifies to:
Au of 7, X
Zz [Zg - 8
28

---

Substituting the expressions a X2 - Tx and _ = Z2 T., we obtain:
7/ X (X2 Tz) X2(722 T=) = X2T= 72
Therefore, the error becomes:
X2T; - T,
Au of
Z2 (Z2 + (Z2 -)))
since / = Z/2 - T
To gain further insight into the dependency on depth Zz, let assume that along object boundaries the ratio X2/Z2 Z2 remains approximately constant, i.e.,
X2 R Zz.
for some constant c. Under this assumption, the numerator approximates as
X2T: TT422 Z2 (cT; Tx).
Substituting this back, we get:
Zr (c Tτ)
cT. - To f Zz 8 (Zz T²)
Au f
(2 (Z Tx))
For small 6, the term 8 (Z2 - T2) in the denominator is negligible compared to Zz. That is,
Zz 6 Tg) Zz-
Thus, we arrive at the simplified expression:
Au ~ (cT, -
Zz
This result shows that the projection error Au inversely proportional to Zz, meaning that fore- ground points (with small Zz) experience larger epipolar displacements due to scale uncertainty phenomenon we term foreground erosion. Moreover, our analysis demonstrates that, except for the first view, the normalization process amplifies minor scale errors in the foreground; this amplification results in substantial epipolar displacement and the erosion of fine details these regions.
E
Limitations
Although our method enhances fine geometric details, its boundary accuracy remains inferior to that of MoGe [66], which produces sharper results. mixed-teacher strategy or more dedicated distillation design may offer further improvements. Additionally, the current model supports only 512/518 resolution, and scaling to higher resolutions remains a challenge for future work.
References
[1] Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven Seitz, and Richard Szeliski. Building rome in day. Communications of the ACM, 2011. 3
[2] Dejan Azinović, Ricardo Martin-Brualla, Dan B Goldman, Matthias Nießner, and Justus Thies. Neural rgb-d surface reconstruction. CVPR, 2022. 8,
[3] Gwangbin Bae and Andrew Davison. Rethinking inductive biases for surface normal estima- tion. In CVPR, 2024. 3
[4] Daniel Barath and Chris Sweeney. Relative pose solvers using monocular depth. In ICPR, 2022.
[5] Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARKitscenes diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. In NeurIPS, 2021. 23
29

---

[6] D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation, In ECCV, 2012. 8
[7] Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2, 2020. 23
[8] Yue Chen, Xingyu Chen, Anpei Chen, Gerard Pons-Moll, and Yuliang Xiu. Fear2gs: Probing visual foundation models with gaussian splatting. In CVPR, 2025. 24
[9] Jan Czarnowski, Tristan Laidlow, Ronald Clark, and Andrew Davison. Deepfactors: Real-time probabilistiic dense monocular slam. RAL, 2020.
[10] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 7, 23
[11] Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. arXiv preprint arXiv:2412.08376, 2024.
[12] Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Mare Pollefeys, Josef Sivie, Akihiko Torii, and Torsten Sattler. D2-net: trainable enn for joint description and detection of local features. In CVPR, 2019. 3
[13] Johan Edstedt, Qiyu Sun, Georg Bökman, Marten Wadenbäck, and Michael Felsberg. RoMa: Robust Dense Feature Matching. CVPR, 2024. 8
[14] Jan-Michad Frahm, Pierre Fite-Georgel, David Gallup, Tim Johnson, Rahul Raguram, Changehang Wu, Yi-Hung Jen, Enrique Dunn, Brian Clipp, Svetlana Lazebnik, et al. Building rome on a cloudless day. In ECCV, 2010. 3
[15] Qiancheng Fu, Qingshan Xu, Yew Soon Ong, and Wenbing Tao. Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction. NeurIPS, 2022.
[16] Yasutaka Furukawa, Carlos Hernández, et al. Multi-view stereo: A tutorial. Foundations and Trends@ in Computer Graphics and Vision, 91-2):1-144, 2015. 3
[17] Silvano Galliani, Katrin Lasinger, and Konrad Schindler. Massively parallel multiview stereopsis by surface normal diffusion. In ICCV, 2015. 3
[18] Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong Tan, and Ping Tan. Cascade cost volume for high-resolution multi-view stereo and stereo matching. In CVPR, 2020. 3
[19] Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cam-
bridge University Press, 2000. 3
[20] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022.
[21] Wonbong Jang, Philippe Weinzaepfel, Vincent Leroy, Lourdes Agapito, and Jerome Revaud. Pow3r: Empowering unconstrained 3d reconstruction with camera and scene priors. arXiv preprint arXiv:2503.1731 2025.
[22] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In CVPR, 2014.
[23] Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Kon- rad Schindler. Repurposing diffusion-bann based image generators for monocular depth estimation. In CVPR, 2024. 2,3
[24] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? NeurIPS, 2017. 6
30

---

[25] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitchead, Alexander Berg, Wan- Yen Lo, al. Segment anything. In ICCV, 2023. 2, 23
[26] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan- Yen Lo, et al. Segment anything. In ICCV, 2023. 6, 23
[27] Arno Knapitsch, Jaesik Park, Qian- Yi Zhou, and Vladlen Koltun. Tanks and temples: Bench- marking large-scale scene reconstruction. ACM TOG, 2017. 8
[28] Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3d with mast3r. In ECCV, 2024. 2, 3, 6,
[29] Wenyu Li, Sidun Liu, Peng Qiao, and Yong Dou. Mono3r: Exploiting monocular cues for geometric 3d reconstruction. arXiv preprint arXiv:2504. 13419, 2025. 3
[30] Zhengqi Li and Noah Snavely. Megadepth: Learning single-view depth prediction from internet photos. In CVPR, 2018. 23
[31] Philipp Lindenberger, Paul-Edouard Sarlin, and Mare Pollefeys. Lightglue: Local feature matching at light speed. arXiv preprint arXiv:2306. 13643, 2023. 3
[32] Sheng Liu, Xiaohan Nie, and Raffay Hamid. Depth-guidet sparse ttructureffrom www--fooooooino for movies and tv shows. CVPR, 2022.
[33] Shing Yan Loo, Syamsiah Mashohor, Sai Hong Tang, and Hong Zhang. Deeprelativefusion: Dense monocular slam using single-image relative depth prediction. In IROS, 2021.
[34] Jiahao Lu, Tianyu Huang, Peng Li, Zhiyang Dou, Cheng Lin, Zhiming Cui, Zhen Dong, Sai-Kit Yeung, Wenping Wang, and Yuan Liu. Align3r: Aligned monocular depth estimation for dynamic videos. arXiv preprint arXiv:2412.0307, 2024. 3
[35] Ziqi Lu, Heng Yang, Danfei Xu, Boyi Li, Boris Ivanovic, Marco Pavone, and Yue Wang. Lora3d: Low-rank self-calibration of 3d geometrie foundation models. arXiv preprint arXiv:2412.07746, 2024. 3
[36] Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. ACM TOG, 2020. 3
[37] Zeyu Ma, Zachary Teed, and Jia Deng. Multiview stereo with cascaded epipolar raft. In ECCV, 2022.7
[38] Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In CVPR, 2020.
[39] Onur Özyeşil, Vladislav Voroninski, Ronen Basri, and Amit Singer. A survey of structure from = Acta Numerica, 26:305-364, 2017.
[40] E. Palazzolo, J. Behley, Lottes, P. Giguère, and C. Stachniss. ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals. In IROS, 2019.
[41] Linfei Pan, Daniel Barath, Marc Pollefeys, and Johannes Lutz Schönberger. Global Structure- from-Motion Revisited. In ECCV, 2024. 2
[42] Zador Pataki, Paul-Edouard Sarlin, Johannes L. Schönberger, and Mare Pollefeys. MP-SIM: Monocular Surface Priors for Robust Strccure-from-MMotion In CVPR, 2025. 3
[43] Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Lue Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. CVPR, 2024. 2,
[44] Matia Pizzoli, Christian Forster, and Davide Scaramuzza. Remode: Probabilistic, monocular
dense reconstruction in real time. In ICRA, 2014. 3
31

---

[45] Jeremy Reizenstein, Roman Shapovalov. Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In ICCV, 2021. 23, 24
[46] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In ICCV, 2021. 6,
[47] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In CVPR, pages 1938-4947, 2020. 3,
8
[48] Johannes Lutz Schönberger and Jan- Jan-Michael Frahm. crructreeffrommmoote revisited. In CVPR,
2016.
2,3 3
[49] Thomas Schöps, Torsten Sattler, and Mare Pollefeys. BAD SLAM: Bundle adjusted direct
RGB-D SLAM. CVPR, 2019.
[50] Thomas Schops, Johannes L Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Mare Pollefeys, and Andreas Geiger. multi-view stereo benchmark with high-resolution images and multi-camera videos. In CVPR, pages 3260-3269, 2017. 9
[51] Philipp Schröppel, Jan Bechtold, Artemij Amiranashvili, and Thomas Brox, A benchmark and a baseline for robust multi-view depth estimation. In 3DV, 2022. 23
[52] Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, and Andrew Fitzgibbon. Scene coordinate regression forests for camera relocalization in rgb-d images. In CVPR, pages 2930-2937, 2013. 8,9
[53] Noah Snavely, Steven M Seitz, and Richard Szeliski. Photo tourism: exploring photo collections in 3d. In ACM siggraph 2006 papers, pages 835-846. 2006. 3
[54] Jiuhn Song, Seonghoon Park, Honggyu An, Seokju Cho, Min-Seop Kwak, Sungjin Cho, and Seungryong Kim. Därf: boosting radiance fields from sparse inputs with monocular depth adaptation. In NeurlPS, 2023. 3
[55] Zhenggang Tang, Yuchen Fan, Dilin Wang, Hongyu Xu, Rakesh Ranjan, Alexander Schwing, and Zhicheng Yan. Mv-dust3r+: Singe-stage scene reconstruction from sparse views in seconds. arXiv preprint arXiv:2412.06977, 2024. 2,
[56] Keisuke Tateno, Federico Tombari, Iro Laina, and Nassir Navab, Con-slam: Real-time dense monocular slam with learned depth prediction. In CVPR, 2017. 3
[57] Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. NeurIPS, 2021. 3
[58] Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, and Andreas Geiger. Sparsity invariant cnns. In 3DV, 2017. 8
[59] Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE TPAMI, 1991.
[60] Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory. arXiv preprint a408 16061, 2024. 2, 3, 8
[61] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In CVPR, 2025. 2. 3, 6, 8, 9, 24
[62] Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In CVPR, 2024. 3, 23
[63] Jianyuan Wang, Christian Rupprecht, and David Novotny. Posediffusion: Solving pose estima- tion via diffusion-aided bundle adjustment. In ICCV, pages 9773-9783, 2023. 24
32

---

[64] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state, arXiv preprint arXiv:2501. 2025. 2,
[65] Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state, 2025. 2, 3, 4, 6, 7, 8, 9, 24
[66] Ruicheng Wang. Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. arXiv preprint ar arXhv:2410.19115, 2024. 2, 3, 5, 6, 23, 29
[67] Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. Neur/PS, 2025. 6
[68] Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In CVPR, 2024. 3, 4, 6, 9, 24
[69] Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset push the limits of visual slam. IROS, 2020. 6, 23
[70] Yifan Wang, Xingyi He, Sida Peng, Dongli Tan, and Xiaowei Zhou. Efficient LoFTR: Semi- dense local feature matching with sparse- speed. In CVPR, 2024. 8
[71] Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. T² Scalable permutation-equivariaann visual geometry learning, arXiv preprint 2507.13347 2025. 9
[72] Yi Wei, Shaohui Liu, Yongming Rao, Wang Zhao, Jiwen Lu, and Jie Zhou. Nerfingmvs: Guided optimization of neural radiance fields for indoor multi-view stereo. In ICCV, 2021. 3
[73] Guangkai Xu, Yongtao Ge, Mingyu Liu, Chengxiang Fan, Kangyang Xie, Zhiyue Zhao, Hao Chen, and Chunhua Shen. What matters when repurposing diffusion models for general dense perception tasks? arXiv preprint arXiv:2403.0600 2024.
[74] Haofei Xu, Songyou Peng, Fangjinhua Wang, Hermann Blum, Daniel Barath, Andreas Geiger, and Marc Pollefeys. Depthsplat: Connecting gaussian splatting and depth. In CVPR, 2025,
[75] Jianing Yang, Alexander Sax, Kevin Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. In CVPR, June 2025. 2
[76] Jianing Yang, Alexander Sax, Kevin Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. arXiv preprint arXiv:2501.1392 2025. 3
[77] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv:2406.. 444444 2024. 2, 3,4
[78] Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mysnet: Depth inference for unstructured multi-view stereo. In ECCV, 2018.
[79] Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. CVPR. 2020. 23
[80] Botao Ye, Sifei Liu, Haofei Xu, Li Xueting, Mare Pollefeys, Ming- Hsug- Yang, and Peng Songyou. No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images. arXiv preprint 11xii:24100.4207. 2024. 8
[81] Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: high-fidelity dataset of 3d indoor scenes. In ICCV, 2023. 23
33

---

[82] Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal Fua. LIFT: Learned Invariant Feature Transform. ECCV, 2016.
[83] Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from single image. In ICCV, 2023.
[84] Yifan Yu, Shaohui Liu, Rémi Pautrat, Marc Pollefeys, and Viktor Larsson. Relative pose estimation through affine corrections of monocular depth priors. In CVPR, 2025.
[85] Zehao Yu, Songyou Peng. Michael Niemeyer, Torsten Sattler, and Andreas Geiger. Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction. In NeurIPS, 2022.
[86] Yuheng Yuan, Qiuhong Shen, Shizun Wang, Xingyi Yang, and Xinchao Wang. Test3r: Learning to reconstruct 3d at test time. NeurIPS, 2025. 3
[87] Shangzhan Zhang, Jianyuan Wang, Yinghao Xu, Nan Xue, Christian Rupprecht, Xiaowei Zhou, Yujun Shen, and Gordon Wetzstein. Flare: Feed-forward geometry, appearance and camera estimation from uncalibrated sparse views. CVPR, 2025. 2,
[88] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnifi- cation: learning view synthesis using multiplane images. ACM TOG, 2018. 8, 9, 23
[89] Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Mare Pollefeys. Nicer-slam: Neural implicit scene encoding for rgb slam. In 3DV, 2024.
34


Reviews:
[{"review_id": "3oUzkz7L9Y", "reviewer": "Reviewer_dQ4p", "summary": "This paper introduces Fin3R, a lightweight and effective fine-tuning method aimed at improving feed-forward 3D reconstruction networks such as DUSt3R, CUT3R, and VGGT. These models predict 3D pointmaps of input images in a single pass but frequently struggle with capturing fine geometric details due to challenges like limited high-fidelity supervision and geometric misalignments across views. Fin3R addresses these issues by fine-tuning only the image encoder, while freezing the decoder of the model. By introducing a monocular teacher model and a custom LoRA adapter, Fin3R distills high-quality geometry from large unlabeled data, enhancing the encoder's ability to capture complex structures and sharp boundaries. Experiments on the DUSt3R, CUT3R, and VGGT validate that Fin3R consistently improves geometry prediction in both single-view and multi-view settings.", "questions": "1. What are the criteria for selecting training data for fine-tuning?\n\n2. Please provide more results on large-scale outdoor scenes, such as the Tank and Temples dataset.\n\n3. Can the proposed method be used to fine-tune models for metric depth/pointmap prediction, such as Mast3R?", "limitations": "Although the method proposed in this paper improves the performance of most models, it negatively impacts some models that already perform well, which is often undesirable in practical applications.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.\tThis paper is well-written and easy to follow.\n\n2.\tThe improvement across several tasks with different datasets is promising, especially considering that the proposed method only requires fine-tuning the target model with a small amount of unseen data.\n\nWeaknesses:\n\n1.\tThe paper lacks experiments on dense multi-view stereo (MVS) estimation using the high-quality DTU dataset, which could effectively reflect both pose and depth improvements. Additionally, according to Sec. 3.3, only monocular distillation loss and pointmap loss are used for fine-tuning. It is unclear why the proposed method improves relative pose estimation for models like VGGT and CUT3R, which directly regress camera parameters.\n\n2.\tThere is insufficient discussion and comparison with similar methods that also use LoRA to fine-tune feed-forward 3D reconstruction networks, such as LoRA3D (ICLR 2025).\n\n3.\tCan the proposed method be used to improve Mast3R, which uses an architecture similar to DUSt3R but regresses metric depth?\n\n4.\tConfidence maps are widely used in feed-forward 3D reconstruction methods to filter unreliable predictions. It would be interesting to discuss whether the proposed method can improve the accuracy of pointmaps or depths while increasing confidence in predictions.\n\n6.\tCould the use of larger-scale datasets and more diverse scenes further enhance the fine-tuning results? Additionally, what was the rationale behind selecting the SA-1B dataset for fine-tuning? What specific criteria informed this decision?", "was_revised": true, "final_justification": "Most of my concerns have been addressed. The comparisons on MVS, video depth, metric depth estimation (mast3r), and LoRA-based Fine-tuning further enhance the effectiveness and robustness of the proposed method.", "rebuttal": "We thank Reviewer dQ4p for insightful and helpful suggestions. We are very pleased that the reviewer found the paper to be \"**well-written and easy to follow**\" and recognized that the \"**improvement across several tasks with small amount of datasets**\" We appreciate this positive assessment and will now address the points raised for further clarification.\"\n\n **W1: On MVS Evaluation and the Mechanism for Pose Improvement**\n\n**1. On MVS Evaluation Protocol and Results**\n\n> We followed the established evaluation protocol from recent works (e.g., Spann3R, CUT3R) and the concurrent work Pi3 by assessing performance on **point map estimation**. This approach is standard in the absence of an open-source Multi-View Stereo (MVS) evaluation benchmark for the 3R model series and effectively reflects both geometric accuracy and pose estimation quality.\n>\n> Our results on the DTU and ETH3D datasets, evaluated using the official Pi3 codebase, are presented below. The table shows our fine-tuned model (VGGT+Ours) not only surpasses the VGGT baseline but also **outperforms the concurrent work Pi3 on 8 of 12 metrics**, while being vastly more efficient: our fine-tuning took 1 day on 4 L20 GPUs, versus Pi3's several days on 64 A100 GPUs(they also use VGGT weight initialization).\n\n| Method | | |DTU | | | | | | ETH3D| | | |\n|--------|-----|-----|-----|-----|-----|-----|-------|-----|-----|-----|-----|-----|\n| | Acc. ↓ | | Comp. ↓ | | N.C. ↑ | | Acc. ↓ | | Comp. ↓ | | N.C. ↑ | |\n| | Mean | Med. | Mean | Med. | Mean | Med. | Mean | Med. | Mean | Med. | Mean | Med. |\n| Pi3 | 1.151 | 0.622 | **1.793** | **0.629** | 0.668 | 0.754 | **0.194** | 0.130 | 0.220 | 0.135 | **0.867** | 0.965 |\n| VGGT | 1.187 | 0.715 | 2.229 | 1.309 | 0.694 | 0.779 | 0.290 | 0.196 | 0.371 | 0.230 | 0.839 | 0.932 |\n| VGGT+Ours | **0.948** | **0.520** | 1.879 | 0.905 | **0.699** | **0.787** | 0.209 | **0.112** | **0.170** | **0.085** | 0.861 | **0.972** |\n>\n\n**2. On the Pose Improvement Mechanism:**  \n>While direct pose enhancement was not our primary objective, our empirical results demonstrate this benefit. We hypothesize this stems from the fact that the decoder is a **secret feature-matching module**. The decoder is actually doing matching across images like other matchers (e.g., superglue). With an improved feature with fine details, the matching result can potentially be improved. This hypothesis is also supported by our following Mast3r fine-tuning experiments and our feat2gs benchmark analysis (see response to Reviewer CnZb), which empirically confirms that our method preserves multi-view feature consistency but improves accuracy.\n>\n\n**W2: Comparison with LoRA-based Fine-tuning (e.g., LoRA3D)**\n\n> We thank the reviewer for this point. The key distinction lies in the **fine-tuning paradigm**:\n>\n> *   Our Method: We perform a **one-time, general fine-tuning** to create a single model that offers **zero-shot generalization** at inference.\n> *   LoRA3D: This is a **per-scene (or per-dataset), test-time optimization** that requires fine-tuning for each new scene (or dataset).\n>\n> In essence, our goal is a universal, zero-shot model, whereas LoRA3D is a test-time adaptation technique. We will clarify this distinction in our final manuscript.\n\n**W3 & Q2: Applicability to Metric Depth Models like Mast3r**\n\n> **Yes, our fine-tuning strategy is model-agnostic and effective for metric depth models.** To demonstrate this, we applied our method directly to Mast3r, using the same dataset and loss configuration from our primary experiments and use MoGe-V2 as teacher model.\n>\n> The results, presented below, confirm the significant benefits of our approach.\n>\n> **Fine-tuning Results on Mast3r**\n>\n> | Method | Monocular Depth | |  |Relative Pose Estimation | |\n> | :--- | :---: | :---: | :---: | :---: | :---: |\n> | | rel ↓ | δ1 ↑ | AUC @ 5° ↑ | AUC @ 10° ↑ | AUC @ 20° ↑ |\n> | Mast3r | 49.62 | 30.2 | 37.60 | 59.96 | 76.24 |\n> | Mast3r + Ours | 27.60 | 64.3 | 37.93 | 60.21 | 76.68 |\n> | MoGe-V2 (SOTA) | 14.82 | 78.6 | - | - | - |\n>\n> **Analysis of Results:**\n>\n> 1.  **Solid Improvement in Metric Depth:** Our method improve the mast3r's depth. We **nearly halved the relative error (49.62 → 27.60)** and **more than doubled the δ1 accuracy (30.2 → 64.3)**, significantly closing the gap to the state-of-the-art MoGe-V2 model.\n>\n> 2.  **Consistent Improvement in Pose Estimation:** Notably, relative pose estimation also slightly and consistently improves across all AUC metrics. This is particularly insightful because these gains were achieved **without fine-tuning the decoder or the feature head**. This result provides strong evidence for our hypothesis: enhancing encoder features(with our strategy) alone directly leads to more robust feature matching and, consequently, better pose solver result.\n>\n> These gains were achieved with only 10 training epochs. Metric depth potentionally need more data compared with relative depth, underscoring and potential impact of our fine-tuning approach on metric depth *3R models.\n\n\n**W4: Confidence map improved?**\n\n>Yes, our method significantly enhances the reliability of the predicted confidence maps. This improvement is a direct result of fine-tuning on diverse, pseudo-labeled in-the-wild data, which teaches the model to better assess its own prediction quality.\n>\n>While we cannot include figures in this year nips rebuttal, we provide quantitative evidence below. We measured the average predicted confidence during monocular depth estimation across several standard benchmarks. Our fine-tuned model (VGGT+ours) more than doubles the average confidence score from 4.69 to 9.58. Notably, datasets where our model achieves larger gains in depth accuracy, such as DIODE and iBims-1, also exhibit the more substantial increases in confidence.\n>\n>**Uncertainty Results**\n>\n>\n>|        | ETH3D | NYUv2 | KITTI | DIODE | HAMMER | iBims-1 | **Average** |\n>|--------|--------|--------|--------|--------|--------|--------|--------|\n>| **VGGT** | 3.94 | 8.78 | 2.14 | 6.51 | 4.12 | 4.45 | 4.69 |\n>| **VGGT+ours** | 9.05 | 8.71 | 4.13 | 10.21 | 9.65 | 15.71 | **9.58** |\n>\n>Besides, we assessed the model's reliability on challenging out-of-distribution data, including paintings, sport games, movies, and images with reflective surfaces or low light. On these OOD samples, our model produced an average confidence score of **8.51** compared to the baseline VGGT's score of **1.87**. This demonstrates a markedly improved ability to produce reliable confidence estimates even in novel and difficult scenarios, highlight the robustness of our finetuned model.\n>\n**W5&Q1 Dataset Selection**\n\n>Our choice of the SA1B dataset was inspired by its proven success in state-of-the-art models like Depth Anythingv2 and DistillAnyDepth. SA1b accounts for 1/6 DepthAnything data and DistillAnyDepth demonstrate only use SA1B can achieve sota performance through self-distillation.\n>\n>Our experiments confirm that performance scales directly with data volume; increasing our training set 10x (from 0.2M to 2M images) improved results and further closed the gap to the state-of-the-art MoGe model. While a mixed dataset(BDD100K, SA1B, GoggleLandmarkv2, Places365) at a smaller scale (0.2M) did not show significant gains over using SA-1B alone, we suspect this is a scale-dependent observation. As suggested by related work (e.g., Depth Anything v2 Appendix B.4), data diversity remains critical for performance at very large scales.\n>\n>The key insight from our findings is that recent 3D foundation models(like VGGT) have already gain very strong ability in depth estimation. And they can be rapidly advanced to near SOTA performance with a **relatively small amount of pseudo-labeled in-the-wild data**, highlighting the efficiency and power of our fine-tuning strategy.\n>\n>| Method | Monocular Depth |  |\n>|--------|-----------------|--|\n>|  | rel ↓ | delta1 ↑ |\n>| SA1B(0.2M) | 4.29 | 96.7 | \n>| Mixed(0.2M) | 4.27 | 96.6 |\n>| SA1B(2M) | 4.18 | 96.9 |\n>| MoGe | 4.14 | 96.9 |\n\n**Q2 Large Scale Results, like T&T**\n\n>As suggested by reviewer Cnzb. We add Video Depth Estimation benchmark, which includes T&T dataset. Our method concistent improve depth estimation accuracy while containing multi-view consistency.\n>\n>**Video Depth Estimation**\n>| Method | ETH3D |  | T&T |  | KITTI |  | Sintel |  | Bonn |  |\n>|--------|-----|--|-----|--|-----|--|-----|--|-----|--|\n>|        | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ |\n>| CUT3R | 0.126 | 83.1 | 0.209 | 69.5 | 0.123 | 87.4 | 0.428 | 47.4 | 0.077 | 93.9 |\n>| CUT3R+Ours | 0.130 | 82.8 | 0.180 | 76.2 | 0.112 | 89.8 | 0.406 | 58.4 | 0.062 | 96.8 |\n>| VGGT | 0.044 | 97.9 | 0.137 | 85.3 | 0.072 | 96.5 | 0.301 | 68.4 | 0.052 | 97.3 |\n>| VGGT+Ours | 0.041 | 99.2 | 0.115 | 88.0 | 0.069 | 96.6 | 0.252 | 72.7 | 0.048 | 97.5 |\n\n**Limitation: On the Perceived Negative Impact on Some Models**\n\n> We thank the reviewer for raising this critical point. However, we have provided extensive multi-view experiments, including Video Depth Estimation, Point Map Estimation, Mast3r fine-tuning, and Feat2GS benchmarking(see reviewer Cnzb) show that our method **does not harm multi-view consistency** and provides broad benefits.\n>\n> The specific instances of performance degradation are explainable artifacts:\n>\n> 1.  **CUT3R in Table 3:** The negative result is due to **data contamination in the baseline's protocol**(as we illustrate in L217-L218). The original CUT3R was trained on the re10k, leading to an overfitted baseline. Our model, trained on general data, only fluctuates this overfitted score by a marginal amount (0.12, 0.82, and 0.41 in AUC).\n>\n> 2.  **Table 5 Fluctuation:** As we demonstrate in our response to Reviewer Hi89, this fluctuation is attributable to **inaccurate camera poses** provided by the CUT3R baseline, not a failure of our method.\n>  \nWe hope that the reviewer’s concerns have been properly addressed with our responses above."}, {"review_id": "GERPq31are", "reviewer": "Reviewer_Hi89", "summary": "The authors fine-tune 3D estimators (outputting point clouds) that are trained on videos (using structure from motion as a means to get GT/surrogate 3D) with pre-trained single-view estimators that have higher detail fidelity. First attempts that did not work, due to feature shift, are reported, showing the difficulty of the task. The main contribution is to prevent this shift with only training the encoder and with weight normalization.", "questions": "- Line 168 \" we use 2-view data with distillation supervision applied exclusively to the view1 pointmap head for distillation loss\" - why not also to the second? Is it a good thing to only use one? Would drift be prevented if two are used?\n- Line 218 \" The only exception is CUT3R—likely due to its training includes the RealEstate10k dataset\" What does this dataset bring to the table that prevents improvement?\n\n- Line 171 \"During each epoch, we sample 20,000 images from SA-1B [23], 1,000 from Hypersim [40], and 1,000 from TartainAir [61]\" I wonder how this differs from the training of baselines and whether this could also have an effect on the improvements?\n- Was there a validation set used to determine stopping, would there be overfitting if training for more than 10 epochs?", "limitations": "* Limitations are not clearly stated. In particular, given the mixed results on real images, it would be important to discuss and exemplify where things are getting worse.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength:\n* An effective method to prevent drift by normalizing weights after an update.  \n* Keeping the same network architecture and even pre-trained decoder keeps inference efficient and provides a drop-in replacement\n* Section 4.1 Monocular Depth Estimation shows clear and consistent improvement across several backbones\n\nWeaknesses:\n* Results for MultiView Pose Estimation on RealEstate10k are a bit mixed, fine tuning sometimes makes things worse\n* Results in Table 5 are also a bit mixed. As no std across multiple runs are reported, it is hard to judge how significant the improvements are. Could you explain when the method degrades?\n* Figure 6 is only for VGGT, it would be nice to see side views for all methods. In general, depth maps are hard to read, in particular if the scale is not normalized as in Figure 7. It would be much better to show 3D side views for analyzing depth map quality.\n\n* In Table 7, ablation, I am missing an experiment on fine-tuning just the encoder vs. the whole model (please explicitly state whether any of the existing models are fine-tuning all). Also, some of the initial exploratory experiments were (unless I am mistaken) done on fine-tuning the whole model. Does this also hold when only fine-tuning the encoder?\n\nSummary:\nA simple yet effective method. My main concern is the significance of the improvements on real data for which repeated experiments with std reported would help.", "was_revised": true, "final_justification": "Questions have been answered in the rebuttal", "rebuttal": "We thank the reviewer Hi89 for the positive and constructive feedback. We are pleased they recognized our method as **simple yet effective**, its practical value as an efficient **\"drop-in replacement\"**, and the \"**clear and consistent improvement**\" shown in our monocular depth estimation experiments. We will now address the points raised for further clarification.\n\n**W1 & Q2: The Multi-view pose estimation CUT3R Baseline and the RealEstate10k Dataset**\n>We thank the reviewer for this critical question, which allows us to clarify this specific result. The issue is **data contamination** in the baseline's protocol. The original CUT3R was trained on the RealEstate10k dataset, from which the test set is also drawn. This leads to an overfitted baseline with an artificially inflated score. Therefore, this isolated result is an artifact of the baseline's evaluation, not a limitation of our method. Our fintuned DUSt3R and VGGT demonstrate clear improvement on multi-view pose estimation.\n>\n\n**W2: Table 5 mixed results**\n\n>To assess model robustness, we measured the standard deviation (std) of metrics over 10 runs with randomly permuted input image orders. The std results are presented in following tables.\n>\n>As expected, VGGT and our fine-tuned version (VGGT+Ours) are exceptionally stable (most std < 0.001), a direct result of VGGT's permutation-invariant architecture-only change the first reference image will change the result. In contrast, CUT3R exhibits a higher sensitivity to input order, an inherent characteristic that our fine-tuning does not alter. large variance of CUT3R is also observed by concurrent work Pi3. Dust3R was excluded as its dense pair-wise matching design is inherently order-invariant. \n>\n>**std on pointmap estimation**\n>\n>| Method |  | |7-scenes | | | |  | |NRGBD | | | |\n>|--------|-----|-----|-----|-----|-----|-----|-------|-----|-----|-----|-----|-----|\n>| | Acc. ↓ | | Comp. ↓ | | N.C. ↑ | | Acc. ↓ | | Comp. ↓ | | N.C. ↑ | |\n>| | Mean | Med. | Mean | Med. | Mean | Med. | Mean | Med. | Mean | Med. | Mean | Med. |\n>| CUT3R | 0.005 | 0.002 | 0.006 | 0.002 | 0.001 | 0.001 | 0.014 | 0.009 | 0.005 | 0.004 | 0.001 | < 0.001 |\n>| CUT3R+Ours | 0.006 | 0.002 | 0.006 | 0.002 | 0.001 | <0.001 | 0.012 | 0.010 |0.006 | 0.004 | 0.001 | < 0.001 |\n>| VGGT | 0.001 |  <0.001 |  <0.001 |  <0.001 | < 0.001 | < 0.001 | 0.001 | < 0.001 | < 0.001 | < 0.001 | < 0.001 | < 0.001 |\n>| VGGT+Ours | <0.001 |  <0.001 |  <0.001 | <0.001 | < 0.001 | < 0.001 | 0.001 | < 0.001 | < 0.001 | < 0.001| < 0.001 | < 0.001 |\n\n>Our post-submission analysis finds that CUT3R's poor performance stems from the base model's **inaccurate pose estimation**. The resulting large-scale misalignments(multi-layer wall, misalignment tables) dominate the evaluation metrics, thereby obscuring the geometric benefits of our method. Due to the rebuttal policy change this year I cannot share the visualization results. But it can be easilly confirmed by using the official CUT3R huggingface demo with some images from the 7-Scenes or NRGBD datasets.\n>\n>The VGGT's minor performance dip on NRGBD is due to a **ceiling effect**, as its synthetic scenes offer limited room for improvement over a strong baseline. Conversely, our method's true efficacy is evident on more demanding benchmarks. We direct the reviewers to our response to Reviewer dQ4p, where we present more results showing that our fine-tuned VGGT yields significant gains on the real-world ETH3D and high-quality DTU datasets.\n>\n\n**W3: 3D visualization**\n>We agree that 3D visualizations would strengthen the paper. While NeurIPS policy prevents us from adding new visuals during the rebuttal, we direct the reviewer to the existing depth visualizations in our supplementary material, where we show monoculda depth estimation results from out-of-distribution images. We commit to adding the requested 3D side-view and reconstruction results from all models to the next version to enhance our analysis.\n>\n\n**W4: Decoder Tuning**\n\n>We sincerely thank the reviewer for this insightful comment. We agree that providing an empirical justification for freezing the decoder is essential, as this is a central design choice in our methodology.\n>\n>This was, in fact, a core hypothesis that we tested during our initial exploratory studies. The results from our early experiments confirmed that fine-tuning the decoder with single-view data creates a conflict: it **improves per-image depth** but **degrades the cross-view matching ability** required for robust pose estimation. In preparing the final manuscript, we omitted this specific ablation study in the interest of conciseness, choosing to focus on the results of our final proposed model.\n>\n>We agree with the reviewer Hi89 also reviewer Kjwd that providing these results now significantly strengthens the paper's narrative. We are grateful for the opportunity to share the data that validates our approach. Following is relative pose estimation result on scannet.\n>\n> | Method |  |Relative Pose Estimation | |\n> | :--- | :---: | :---: | :---: |\n> | | AUC @ 5° ↑ | AUC @ 10° ↑ | AUC @ 20° ↑ |\n> | VGGT         | 28.40 | 47.36 | 61.51 |\n> | Encoder-only | 32.06 | 52.29 | 68.04 |\n> | Decoder-only | 28.42 | 51.59 | 67.30 |\n> | Full Finetune| 26.35 | 45.90 | 60.02 |\n> | VGGT+Ours    | **35.21** | **56.70** | **72.80** |\n\n**Q1: why not 2nd view supervision for dust3r**\n> This is due to DUST3R's architecture: it predicts point map for view 2 in the coordinate frame of view 1. This makes direct supervision with a monocular depth teacher on view 2 impossible since it does not predict self-view geometry. Despite this, we found that supervising only the first view is highly effective and works \"out of the box\" for both DUST3R and MAST3R (as noted in our response to Reviewer dQ4p), yielding strong performance.\n>\n\n**Q3&Q4 dataset selection and training epoch**\n>**Dataset Selection**: Our choice of the SA1B dataset was inspired by its proven success in state-of-the-art models like Depth-Anything-v2 and DistillAnyDepth. SA1b accounts for 1/6 DepthAnything data and DistillAnyDepth demonstrates that only using SA1B can achieve sota performance through self-distillation. We selected the synthetic datasets Hypersim and TartanAir specifically for their high-quality, sharp geometric boundaries compared with other real-world dataset.\n>\n>**Training Scale & Duration**: Regarding the training duration, our experiments reveal that the model's performance scales positively with data volume without showing signs of overfitting. We explicitly tested this by increasing our training data tenfold from 0.2M to 2M images. As the results below show, this significantly improved performance and closed the gap with the state-of-the-art MoGe model.\n>\n>| Method | Monocular Depth |  |\n>|--------|-----------------|--|\n>|  | rel ↓ | delta1 ↑ |\n>| SA1B(0.2M) | 4.29 | 96.7 | \n>| SA1B(2M) | 4.18 | 96.9 |\n>| MoGe | 4.14 | 96.9 |\n\n>The key insight from our findings is that recent 3D foundation models(like VGGT) have already gain very strong ability in depth estimation. And they can be rapidly advanced to near SOTA performance with a **relatively small amount of pseudo-labeled in-the-wild data**, highlighting the efficiency and power of our fine-tuning strategy.\n>\n\n**Limitations**\n>Regarding the results in Tables 5 and 6, we refer to our clarification in W1 & Q2 and W2. We also provide experiments on point map estimation (reviewer dQ4p), video depth estimation (reviewer Cnzb), and MASt3R fine-tuning(reviewer dQ4p) that further confirm our method's robust generalization in real-world scenarios. We respectfully hope the reviewer can reconsider their initial assessment of this point.\n>\n>The primary limitation of our work is that our renormalization LoRA weights are not mergeable like standard LoRA. This introduces a minor but measurable overhead (~5% in computational time and <0.1% in parameters).\n\nWe hope that the reviewer’s concerns have been properly addressed with our responses above."}, {"review_id": "I1hwBpWRAC", "reviewer": "Reviewer_Cnzb", "summary": "The authors propose an encoder-only fine-tuning method for the *3R family, where monocular depth information from MoGe (single-view DUST3R) is distilled into the backbone encoder feature. When fine-tuning, they apply re-normalization LoRA on the encoder to address the over-smoothed geometry caused by multi-view pointmap regression.\n\nThe goal of this approach is to produce sharper and more robust single-view depth, while performing comparable multi-view reconstruction.\n\nThey compare their method to the backbones in *3R family, such as DUSt3R, CUT3R, and VGGT, evaluating on benchmarks for monocular depth, pose estimation and 3D reconstruction.", "questions": "Refer to [Major Weaknesses] 1) and [Minor Weaknesses] 1) 2)\nFor now, I am not yet convinced by the performance of the method due to the mentioned points around evaluation. However, I would be willing to raise my score if the authors address the concerns raised in my review and provide clarification on the points mentioned above.", "limitations": "No, there is a lack of discussion on the limitations of monocular depth distillation's impact on the inherent multi-view consistency of the backbone feature.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n\nThis work is insightful, offering valuable insights for fine-tuning the *3R family to adapt them for producing fine-grained details. Instead of fine-tuning the decoder or using noisy depth labels, they chose to apply a re-normalized LoRA to fine-tune the encoder using MoGe. This approach can be generalizable to subsequent *3R methods. \n\n[Major Weaknesses]\n\n1) Missing video (multi-view) depth evaluation: The goal of the paper is improving monocular depth while preserving multi-view consistency. While the method is evaluated on pose estimation and 3D reconstruction, improved pose can lead to better 3D reconstruction. It is very important to evaluate multi-view depth, because it can help better factor out the impact of this fine-tuning method on multi-view consistency. And these datasets (e.g., DTU, ScanNet, KITTI, ETH3D, T&T, Bonn, TUM Dyn) are also widely used in previous work like CUT3R/VGGT.\n\n[Minor Weaknesses]\n\n1) Monocular depth distillation in the encoder may harm inherent multi-view consistency in feature space except increasing feature norms. It could be verified whether the features after adaption still enable fine-grained, multi-view consistent reconstruction using Feat2GS benchmark. There seems to be an overclaim in L132-135.\n\n2) CUT3R is not only a feed-forward 3D reconstruction model but also a 4D foundation model. It would be interesting if the authors could show the proposed adaptation’s performance on 4D fine details reconstruction.\n\n3) L108-111: The motivation for 'Scale Uncertainty' paragraph is unclear. There’s a common issue in *3R models—incorrect depth or scale leads to multi-layered walls in reconstruction. It would be great to demonstrate this improvement by providing some visual examples where the baseline predicts layered walls, but this proposed method predicts a single layer wall, \n\n4) L116-117 lacks evidence: It would be helpful to cite VGGT(in the Table 3 of the original paper), as it supports this point.\n\n5) Lack of discussion on LORA3D and TEST3R. Why not use Visual Prompt Tuning insdead of LORA?", "was_revised": true, "final_justification": "The authors provide a comprehensive rebuttal and reply to all of my questions, which is very much appreciated. And I also appreciate the responsiveness of the authors and verify major weakness 1) and minor weakness 1) completely. I strongly recommend incorporating Video Depth Estimation Result into the main paper. I recommend accepting this paper.", "rebuttal": "We thank the reviewer for their positive feedback and for accurately summarizing the core contributions of our work. We are particularly encouraged that the reviewer found our fine-tuning strategy to be insightful and recognized its generalizability to the broader *3R family of models, as this was a key objective of our research. Next, I will address the concern.\n\n**Major: video-depth estimation**\n>Thanks reviewer for bringing this forward. The primary reason this was not included in our initial submission is that prominent video benchmarks used in cut3r (e.g., Sintel, KITTI, Bonn) primarily feature dynamic scenes. This creates a significant domain mismatch for models like VGGT, which are trained exclusively on static images. To provide a fair and rigorous evaluation, we additionally add ETH3D and Tanks & Temples, two multi-view datasets that are standard for evaluating performance on static scenes. (Note: We excluded DUST3R from this comparison as it requires a post-alignment step for multi-view inputs).\n>\n>As shown in the table below, our method consistently improves both the CUT3R and VGGT backbones on these benchmarks. This demonstrates that our fine-tuning strategy enhances multi-view consistent depth accuracy and generalizes across different model architectures.\n>\n>**Video Depth Estimation Result**\n>| Method | ETH3D |  | T&T |  | KITTI |  | Sintel |  | Bonn |  |\n>|--------|-----|--|-----|--|-----|--|-----|--|-----|--|\n>|        | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ | rel ↓ | delta1 ↑ |\n>| CUT3R | 0.126 | 83.1 | 0.209 | 69.5 | 0.123 | 87.4 | 0.428 | 47.4 | 0.077 | 93.9 |\n>| CUT3R+Ours | 0.130 | 82.8 | 0.180 | 76.2 | 0.112 | 89.8 | 0.406 | 58.4 | 0.062 | 96.8 |\n>| VGGT | 0.044 | 97.9 | 0.137 | 85.3 | 0.072 | 96.5 | 0.301 | 68.4 | 0.052 | 97.3 |\n>| VGGT+Ours | 0.041 | 99.2 | 0.115 | 88.0 | 0.069 | 96.6 | 0.252 | 72.7 | 0.048 | 97.5 |\n\n>Furthermore, to reinforce the point about multi-view performance, we encourage the reviewer to see our **additional point map estimation results on DTU and ETH3D (provided in our response to Reviewer dQ4p)**. These results highlight not only improved multi-view consistent geometry but also pose, confirming that our method enhances single-view detail without compromising multi-view consistency.\n\n**Minor1: Feat2GS Benchmark**\n>We thank the reviewer for this insightful point. We agree that our method may not address all potential sources of feature shift, and we will revise the claims in our manuscript to more precisely reflect our specific contributions.\n>\n>However, we provide extensive empirical evidence to demonstrate that our approach successfully maintains multi-view consistency. To further validate this, we now present results on Feat2GS benchmark, which directly evaluates the quality of multi-view features for novel view synthesis.\n>\n>Due to the rebuttal time constraints and feat2gs haven't included cut3r yet, we focused our evaluation on the DUST3R and VGGT backbones. The results are as follows:\n>|           |              | **Geometry**|             |             | **Texture** |             |           |   **All**   |             |\n>|:----------|:------------:|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:|:---------:|:-----------:|:-----------:|\n>|           | PSNR↑        | SSIM↑       | LPIPS↓      | PSNR↑       | SSIM↑       | LPIPS↓      | PSNR↑     | SSIM↑       | LPIPS↓      |\n>| DUSt3R    | 19.56        | 0.6504      | 0.3181      | 18.06       | 0.6006      | 0.3221      | 19.40     | 0.6477      | 0.3700      |\n>| DUSt3R_ft | 19.60        | 0.6512      | 0.3181      | 18.05       | 0.6015      | 0.3217      | 19.65     | 0.6417      | 0.3669      |\n>| VGGT_e    | 19.66        | 0.6558      | 0.3123      | 18.07       | 0.6003      | 0.3225      | 19.61     | 0.6510      | 0.3788      |\n>| VGGT_e_ft | 19.70        | 0.6561      | 0.3115      | 18.10       | 0.6008      | 0.3224      | 19.66     | 0.6514      | 0.3781      |\n\n>As shown, our method **not only preserves but slightly improves multi-view performance**, evidenced by the gains in PSNR and LPIPS. It is important to contextualize these numbers: performance on Feat2GS is typically concentrated within a very narrow range (e.g., PSNR often between 19.40 and 19.70). This demonstrates our method successfully improves single-view geometry while preserving the integrity of multi-view features.\n>\n\n**Minor2: CUT3R Fine Structure**\n>While the rebuttal format prevents us from including new qualitative figures, we can provide quantitative evidence of the improvement in boundary sharpness. To do this, we use the Boundary F1 score, a metric designed by DepthPro specifically to measure geometric detail at object edges, which has been adopted by recent works like MoGe-v2 for this exact purpose. We evaluated our method on a diverse set of benchmarks, including static real-world scenes (iBims-1, HAMMER) and dynamic synthetic scenes (Sintel, Spring). The results are as follows:\n>\n>**Boundary F1 Score for cut3r**\n>| Method | iBims-1 | HAMMER | Sintel | Spring|\n>|--------|--------|--------|--------|--------|\n>| **CUT3R**      | 8.34 | 4.01 | 23.79 | 28.88 |\n>| **CUT3R+Ours** | 20.63 | 16.66 | 29.78 | 29.35  |\n\n>The data clearly shows that our method (CUT3R + Ours) achieves improvements in boundary sharpness over the original CUT3R across both static and dynamic datasets. This quantitatively confirms that our fine-tuning enhances the model's ability to capture fine-grained geometry. For qualitative evidence, we also invite the reviewer to examine the supplementary material of our submission, which contains several examples of in-the-wild depth estimations from our enhanced CUT3R model.\n>\n\n**Minor3: Scale-Uncertainty ambiguity**\n>We apologize for the lack of clarity. Our central argument is that the point map regression head inhibits the capture of fine details.\n>\n>The  Scale-Uncertainty is one of the reasons. It is primarily related to the normalization process when calculating loss; the normalization is fixed to the reference view. This creates reprojection errors for all non-reference views, a phenomenon we prove mathematically in the supplement and show empirically in Figure 2(b). These errors cause high loss at object boundaries, forcing the model to learn blurred edges.\n>\n>The recent concurrent work, Pi3, independently validates this diagnosis, as they also remove the pointmap head from VGGT to finetune it, albeit with a different focus on equivariance.\n\nRegarding the **multi-layer wall issue**, our method shows improvements, which can be seen in Figure 6. For example, in the top-left kitchen scene, our method produces clean cabinets where the baseline shows multi-layer artifacts, as the zoomed-in region shows. Likewise, in the right image, our method corrects the layered wall at the upper left corner produced by the original VGGT.\n\nWhile we acknowledge the issue is not entirely solved and can still occur occasionally in close-ups, we believe the fly-through videos in our supplementary material offer a more dynamic and compelling demonstration of the overall reduction of these artifacts. We invite the reviewer to view them.\n\n**Minor4: Cite VGGT to support**\n\n> Thanks for the kind suggestion, we will add a citation here for our next version!\n\n**Minor5: Compare with LORA3D/Test3R; why not visual prompt tuning**\n\n> The key distinction between our method lies in the **fine-tuning paradigm**:\n>\n> *   Our Method: We perform a **one-time, general fine-tuning** to create a single model that offers **zero-shot generalization** at inference.\n> *   LoRA3D/Test3R: This is a **per-scene (or per-dataset), test-time optimization** that requires fine-tuning for each new scene (or dataset).\n> In essence, our goal is a universal, zero-shot model, whereas LoRA3D/Test3R is a test-time adaptation technique. We will clarify this distinction in our final manuscript. For the visual prompt tuning comparison. We also implement a visual prompt (following Test3R) tuning for the encoder, following table shows the results.\n\n**Visual Prompt Tuning**\n| Method | Monocular Depth |  | Relative Pose Estimation |  |  |\n|--------|-----------------|--|-------------------------|--|--|\n|  | rel ↓ | delta1 ↑ | AUC @ 5° ↑ | AUC @ 10° ↑ | AUC @ 20° ↑ |\n| VGGT+Ours | **4.29** | **96.7** | **35.21** | **56.70** | **72.80** |\n| VGGT+Visual Prompt Tuning | 4.50 | 96.0 | 33.87 | 54.46 | 70.06 |\n\nOur method outperforms the VPT baseline. Our analysis suggests this is because VPT primarily adapts a model to a new task by tuning input prompts, leaving the core feature extraction unchanged. Conversely, our approach directly refines the feature extractor itself. This allows the model to generate more robust and detailed features, which is essential for high-quality geometric reconstruction and explains our method's performance advantage.\n\n\n**Limitation: Addressing the Concern Regarding Multi-View Consistency**\n\nWe thank the reviewer for their feedback on this crucial point. To directly address the initial concern about the multi-view consistency of our learned features, we have conducted extensive experiments, including:\n\n- **Video Depth Estimation**: Demonstrating strong temporal and view-to-view consistency.\n- **Additional Point Map Estimations**: Showing geometrically coherent reconstructions across multiple viewpoints.\n- **The Feat2GS Benchmark**: Providing quantitative validation on a feature-wise benchmark specifically designed to measure feature quality for 3D reconstruction.\n\nWe hope that the reviewer’s concerns have been properly addressed with our responses above."}, {"review_id": "YHL1EAnZRq", "reviewer": "Reviewer_Kjwd", "summary": "This paper proposed a new fine-tuning strategy to improve the performance of the current feed-forward reconstruction models. The authors involve MoGe as the teacher network and finetune the model on SA-1B dataset. Besides, they also involve the LoRA model, and meanwhile rescale the norm of the feature to satisfy the requirements of the frozen decoders.", "questions": "Please refer to the weaknesses.", "limitations": "Please refer to weaknesses.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n1. The authors made a thorough analysis of the limitations of the current feed-forward reconstruction methods.\n2. The authors proposed a new fine-tuning strategy.\n\n**Weaknesses**\n1. Instead of distilling the knowledge from MoGe, why not directly integrate it into the proposed model? Since in Table 1, none of the finetuned method has superior performance than MoGe.\n2. In Line 132-133, the authors claimed, 'the limitations in detail recovery primarily originate from the encoder'. However, I did not see any theoretical or empirical evidence to support the statement. Finetuning the decoder with a frozen encoder could also improve the performance. \n3. What motivates the authors to integrate LoRA instead of others?\n4. The improvement in Figure 5 and 6 is tiny, in the 6th line of Figure 4, it seems such finetune degrade the original performance.\n5. In Table 3, why the proposed method improved the static scene reconstruction methods, but somehow make the dynamic scene reconstruction method worse?\n6. How about finetuning the encoder and decoder on the new dataset without any proposed so-called strategy?", "was_revised": true, "final_justification": "The answers of authors addressed my concerns.", "rebuttal": "We sincerely thank the reviewer for their positive and encouraging feedback. We are particularly encouraged that the reviewer recognized the two central contributions of our paper: our detailed analysis of limitations in existing feed-forward reconstruction methods, and the value of our fine-tuning strategy for generating more robust geometric features. We appreciate their thorough evaluation and will now address the points raised for further clarification.\n\n**W1: Why not integrate Moge directly**\n> We thank the reviewer for this point, as it allows us to clarify the central research question of our work. Our primary goal is **not** to develop a system that surpasses the top-performing monocular depth estimators like MoGe. Instead, we investigate a different and fundamental question: **\"How can we leverage a powerful monocular depth model to improve the training of feed-forward 3D reconstruction models?\"**\n>\n>Our work focuses on addressing the inherent limitations of these feed-forward models, which often struggle with geometric details and robustness. By using the monocular model as a supervisory signal, we successfully enhance the underlying feature representation. This contribution is validated by the fact that our method achieves promising results even with limited data, a key strength recognized by Reviewer dQ4p.\n>\n>This distinction is important. While one could likely achieve a higher metric by directly integrating a large model like MoGe, this represents more of an engineering solution to boost performance. Such an approach is not relevant to our scientific inquiry and would introduce additional practical drawbacks:\n> - Computational Cost: Integrating MOGE would add 24 ViT layers, drastically increasing the model's parameter count.\n> - Inference Overhead: This added complexity would lead to a substantial increase in inference time.\n\n**W2/W6: encoder / decoder tuning**\n\n>We sincerely thank the reviewer for this insightful comment. We agree that providing an empirical justification for freezing the decoder is essential, as this is a central design choice in our methodology.\n>\n>This was, in fact, a core hypothesis that we tested during our initial exploratory studies. The results from these early experiments confirmed that fine-tuning the decoder with **single-view** data creates a conflict: it **improves per-image depth** but **degrades the cross-view matching ability** required for robust pose estimation. In preparing the final manuscript, we omitted this specific ablation study in the interest of conciseness, choosing to focus on the results of our final proposed model.\n>\n>We agree with the reviewer Hi89 as well as reviewer Kjwd, that providing these results now significantly strengthens the paper's narrative. We are grateful for the opportunity to share the data that validates our approach. The following is the relative pose estimation result on Scannet.\n>\n> | Method | |Relative Pose Estimation  | |\n> | :--- | :---: | :---: | :---: |\n> | | AUC @ 5° ↑ | AUC @ 10° ↑ | AUC @ 20° ↑ |\n> | VGGT         | 28.40 | 47.36 | 61.51 |\n> | Encoder-only | 32.06 | 52.29 | 68.04 |\n> | Decoder-only | 28.42 | 51.59 | 67.30 |\n> | Full Finetune| 26.35 | 45.90 | 60.02 |\n> | VGGT+Ours    | **35.21** | **56.70** | **72.80** |\n\n**W3: Why Lora instead of others?**\n\n>Our primary objective is to enhance the feature extractor's ability to capture fine-grained geometric details and improve its robustness. This requires a fine-tuning method that can directly modify the internal representations of the model. We selected LoRA because it is exceptionally well-suited for this task. By injecting low-rank adaptation matrices directly into the layers of the encoder, LoRA refines the model's weights, fundamentally improving the feature generation process itself.\n>\n>To validate our hypothesis, we contrasted this approach with Visual Prompt Tuning. VPT introduces learnable tokens into the input space, which is highly effective for adapting a fixed feature extractor to a new task but does not alter its core feature representation.\n>\n>The results below confirm that for our goal of improving the features themselves, our LoRA-based approach is superior:\n>\n>### Visual Prompt Tuning\n>| Method | Monocular Depth |  |  | Relative Pose Estimation |  |\n>|--------|-----------------|--|-------------------------|--|--|\n>|  | rel ↓ | delta1 ↑ | AUC @ 5° ↑ | AUC @ 10° ↑ | AUC @ 20° ↑ |\n>| VGGT+Ours | **4.29** | **96.7** | **35.21** | **56.70** | **72.80** |\n>| VGGT+Visual Prompt Tuning | 4.50 | 96.0 | 33.87 | 54.46 | 70.06 |\n\n**W4: Tiny improvement**\n>**Tiny improvements in Figure 5** Indeed, this figure shows consistent and notable improvements in monocular depth estimation—boundary sharpness and robustness—across three different baselines. We will further highlight the improvement in our revision.\n>\n>**Tiny improvements in Figure 6** We appreciate the reviewer's comment. The improvements shown in Figure 6 are indeed targeted and precise, as they are designed to correct the specific failure modes of the baseline model. Figure 6, for instance, is not about large-scale changes but about critical geometric details: the sharp chair boundaries, the coherent reconstruction of a reflective cabin, the planar surface near the bread machine, and the removal of wall artifacts. These examples are direct proof that our method enhances fine geometry and robustness where it matters most. For further evidence of these improvements at a scene-wide scale, we direct the reviewer's attention to our supplementary video, which contains fly-throughs that clearly illustrate the superior geometric quality and stability provided by our method.\n>\n>**6th line of Figure 4 finetuned model degrades** We believe the reviewer may be referring to the 6th row of Figure 5, rather than Figure 4, which contains feature norm heatmaps rather than reconstruction outputs. This example, featuring a banner with complex geometry, was intentionally chosen to showcase how our method enhances model robustness in challenging scenarios. We would like to highlight that the baseline models completely fail to reconstruct this banner. In contrast, our fine-tuned models can successfully capture the banner's overall structure.\n>\n\n**W5: Table 3 results**\n> The issue is **data contamination** in the cut3r's training protocol (as we illustrate in L217-L218). The original CUT3R training dataset includes RealEstate10k training set, which will boost its performance on the re10k test set. This leads to an overfitted baseline; this isolated result is an artifact of the baseline's evaluation, not a limitation of our method. Our fin-tuned DUSt3R and VGGT demonstrate clear improvement on multi-view pose estimation.\n>"}]

Author rebuttal:

Meta-review: After the rebuttal phase, all four reviewers recommend acceptance.
Decision: Accept (poster)