Surf3R: Rapid Surface Reconstruction from Sparse RGB Views in Seconds

## Abstract

Abstract Current multi-view 3D reconstruction methods rely on accurate camera calibration and pose estimation, requiring complex and time-intensive pre-processing that hinders their practical deployment.
To address this challenge, we introduce Surf3R , an end-to-end feedforward approach that reconstructs 3D surfaces from sparse views without
estimating camera poses and completes an entire scene in under 10 seconds .
Our method employs a multi-branch and multi-view decoding architecture in which multiple reference views jointly guide the reconstruction process.
Through the proposed branch-wise processing, cross-view attention, and inter-branch fusion, the model effectively captures complementary geometric cues without requiring camera calibration.
Moreover, we introduce a D-Normal regularizer based on an explicit 3D Gaussian representation for surface reconstruction. It couples surface normals with other geometric parameters to jointly optimize the 3D geometry, significantly improving 3D consistency and surface detail accuracy.
Experimental results demonstrate that Surf3R achieves state-of-the-art performance on multiple surface reconstruction metrics on ScanNet++ and Replica datasets, exhibiting excellent generalization and efficiency.

## 1 Introduction

3D surface reconstruction is a long-standing problem that aims to create 3D surfaces of an object or scene captured from multiple viewpoints (Broadhurst, Drummond, and Cipolla 2001; Kutulakos and Seitz 1999; Seitz and Dyer 1999).
This technique has wide applications in robotics, graphics, virtual reality, and other fields.
Traditional 3D surface reconstruction methods typically include two main approaches: Structure-from-Motion (SfM) combined with Multi-View Stereo (MVS) (Chen, Li, and Lee 2023; Huang et al. 2024; Chen et al. 2024a) or volumetric methods (Zak Murez and Rabinovich 2020; Sun et al. 2021).
The SfM+MVS pipeline involves first estimating camera poses and generating sparse 3D point clouds using SfM (Crandall et al. 2013; Charatan et al. 2024; Schönberger and Frahm 2016). This is followed by computing per-view depth maps and fusing them into the final 3D surface using MVS techniques (Huang et al. 2024; Guédon and Lepetit 2024; Chen et al. 2024a).
On the other hand, volumetric methods, such as Atlas (Zak Murez and Rabinovich 2020) and NeuralRecon (Sun et al. 2021), predict 3D volumes like Truncated Signed Distance Function (TSDF) from multiple views, often avoiding the explicit depth map computation.
Although these two kinds of methods achieve high-quality surface reconstruction, they often rely on prior knowledge or require nontrivial pre-processing steps, such as SfM to estimate camera intrinsics and extrinsics. These pre-processing steps often require heavy GPU computation and are time-consuming (typically taking 1–2 hours per scene on a modern GPU), making real-time inference challenging and reducing their practical usability.

Figure: Figure 1: Comparison Between Traditional Methods and Our Approach. Traditional methods rely on SfM for sparse point clouds and calibrated poses, followed by different 3D Reconstruction Methods (RMs). In contrast, our method directly reconstructs the scene from uncalibrated images in under 10 s, eliminating the need for calibration or iterative refinement.
Refer to caption: x1.png

To address the aforementioned limitations, inspired by DUSt3R (Wang et al. 2024), we propose Surf3R, the first feed-forward network that performs pose-free surface reconstruction from sparse RGB inputs in a single pass.
Specifically, we first encode all input views using a shared encoder to extract multi-scale visual features.
To effectively model cross-view information interactions, we introduce Feature-Refine (FR) blocks that jointly learn not only the pairwise relationships between a selected reference view and all other source views, but also the interactions among source views themselves.
When reconstructing a large scene from sparse multi-view images, the geometric correspondence between a selected reference view and certain source views could be insufficient.
This is because substantial changes in camera poses make it difficult to directly infer the relation between the reference view and those source views.
To mitigate this issue, we further introduce a cross-reference fusion mechanism, implemented via a multi-branch design where multiple reference views are independently selected.
Each branch processes the input views through its own FR blocks and integrates information using dedicated Cross-Reference Fusion (CRF) blocks, enabling effective propagation of long-range and complementary information across views.
Based on the fused multi-view features, we first generate a sparse 3D point cloud for reconstruction.
While directly converting this point cloud into a mesh using NKSR (Huang et al. 2023) is feasible, our experiments (see Sec. [4.2](https://arxiv.org/html/2508.04508v1#S4.SS2)) show that this naive approach yields poor reconstruction quality.
The underlying limitation is that point-cloud supervision is applied in a view-separated manner and thus lacks global 3D consistency.
We therefore adopt a Gaussian representation: each Gaussian resides in a unified 3D space and is projected into every view during rendering, so the per-view loss implicitly regularizes the entire scene and yields smoother, more accurate surfaces.
The final per-pixel Gaussian primitives are derived from specifically designed Gaussian heads.

To further facilitate accurate surface reconstruction from the predicted Gaussian parameters, we introduce a Depth-Normal Regularization strategy (Chen et al. 2024b) designed to enhance the geometric fidelity of the reconstructed surfaces.
Specifically, we first apply a flattening operation to the Gaussian primitives to better align them with the local surface geometry.
Subsequently, we introduce a D-Normal formulation, in which surface normals are not directly blended from 3D Gaussians, but instead derived from the gradient of the rendered depth map.
As a result, it allows the Gaussian parameters to be directly supervised by surface normals, jointly optimizing the geometry and markedly enhancing 3D consistency and surface detail.
Extensive experiments on the ScanNet++ datasets demonstrate that Surf3R achieves state-of-the-art surface reconstruction performance.
It significantly outperforms both optimization-based and feed-forward baselines in terms of accuracy and completeness.
Moreover, when evaluated on the unseen Replica dataset in a zero-shot setting, our model maintains competitive accuracy, demonstrating robust generalization to novel scenes.
And it also delivers strong performance on novel view synthesis tasks.
Notably, Surf3R reconstructs an entire scene in under 10 seconds, making it highly efficient for real-time or interactive applications.
We summarize our contributions as follows:

- •
We present Surf3R, the first feed-forward network for pose-free surface reconstruction from sparse multi-view RGB inputs.
It achieves real-time surface reconstruction in under 10 seconds, offering both high efficiency and scalability.
- •
We employ a multi-branch architecture where multiple reference views are jointly leveraged to capture long-range cross-view interactions.
Furthermore, we introduce a Depth-Normal Regularization strategy to enhance geometric fidelity.
- •
Extensive experiments on ScanNet++ and Replica datasets demonstrate that Surf3R achieves state-of-the-art surface reconstruction, generalizes zero-shot to new scenes, and remains competitive for novel-view synthesis.

Figure: Figure 2: Overall Framework of Surf3R. The reference view branches are shown in orange, while the branches of other source views are shown in blue. Each model path uses a different reference view. For clarity, only one of the stacked FRBlock and CRFBlock is displayed.
Refer to caption: x2.png

## 2 Related Works

Multi-View Surface Reconstruction.
Multi-view surface reconstruction recovers dense geometry from images captured at multiple viewpoints. Classical pipelines fuse depth maps obtained by multi-view stereo (Furukawa, Hernández et al. 2015; Seitz et al. 2006; Schönberger et al. 2016; Zhang et al. 2020; Yao et al. 2018; Bleyer, Rhemann, and Rother 2011) or optimize voxel occupancy fields (Bonet and Viola 1999; Kutulakos and Seitz 2000; Broadhurst, Drummond, and Cipolla 2001; Seitz and Dyer 1999), but these approaches are limited by memory and cross-view noise (Tulsiani et al. 2017; Ummenhofer et al. 2017). Implicit neural representations such as signed distance fields (Park et al. 2019; Liu et al. 2020; Ma et al. 2023; Sitzmann et al. 2020) and neural volume rendering (Yariv et al. 2021; Wang et al. 2021a) alleviate some constraints, yet methods like NeuS (Wang et al. 2021a), MonoSDF (Yu et al. 2022), and Geo-NeuS (Fu et al. 2022) remain optimization intensive and do not scale well. 3D Gaussian Splatting (3DGS) (Kerbl et al. 2023) introduces an explicit alternative that rasterizes anisotropic Gaussians in real time (Yifan et al. 2019), though the vanilla version lacks geometric supervision. Extensions such as VastGaussian (Lin et al. 2024), SuGaR (Guédon and Lepetit 2024) and HRGS (Li et al. 2025) incorporate view-consistent depth and normal constraints, substantially improving accuracy and convergence.
However, these methods often require pre-processing step, limiting their practical deployment.
To bridge this gap, we present a feedforward network that dispenses with costly pre-processing while reconstructs high idelity 3D susrface from sparse views in under 10 seconds.

Novel View Synthesis.
Novel-view synthesis has progressed from geometry-aware volumetric grids (e.g., Soft3D (Penner and Zhang 2017), voxel colouring (Seitz and Dyer 1999)) to neural radiance fields such as NeRF (Mildenhall et al. 2021) and Mip-NeRF (Barron et al. 2021), which deliver high fidelity but are hampered by dense ray sampling.
Hash-encoded feature grids and sparse voxel accelerations (InstantNGP (Müller et al. 2022), Plenoxels (Fridovich-Keil et al. 2022), KiloNeRF (Reiser et al. 2021)) mitigate runtime yet remain limited in sparse-view or large-scale scenarios (Garbin et al. 2021; Chen et al. 2022). Recent work on 3D Gaussian Splatting (3DGS) (Kerbl et al. 2023) achieves real-time rendering by rasterising anisotropic Gaussians, achieving faster and higher-quality rendering without neural components. In this work, we utilize the advantage of Gaussian Splatting to perform surface reconstruction and incorporate normal priors to guide the reconstruction.

Learning-based 3D Reconstruction.
Learning-based 3D reconstruction approaches have recently seen a lot of progress.
Notably, DUSt3R (Wang et al. 2024) generates two point maps for an input image pair within a shared coordinate system, implicitly incorporating both intrinsic and extrinsic camera parameters. Nevertheless, the method is intrinsically pair-wise and lacks native support for multi-view input. Multi-view operation is possible only by appending a separate global-alignment stage that registers the individual reconstructions. This alignment must be performed offline, incurs substantial computational overhead, and cannot adapt on-the-fly when additional views become available.
Several recent studies (Leroy, Cabon, and Revaud 2024; Wang and Agapito 2024; Zhang et al. 2025; Wang et al. 2025) replace DUSt3R’s test-time optimization with feed-forward neural networks to accelerate inference, yet their primary focus lies in view synthesis and related tasks rather than high-fidelity surface reconstruction.
In this work, we close this gap by introducing a feed-forward, geometry-aware framework that delivers state-of-the-art surface reconstruction while retaining real-time efficiency.

## 3 Methodology

Our proposed Surf3R framework enables accurate surface reconstruction from a sparse set of RGB images, without the need for known camera intrinsics or poses. First, we introduce a feedforward network that extracts multi-view features and predicts per-pixel 3D Gaussian parameters (Sec.[3.1](https://arxiv.org/html/2508.04508v1#S3.SS1)). These Gaussians are then flattened into 2D planes to better represent the surface, which helps achieve more accurate depth estimation for reconstruction (Sec.[3.2](https://arxiv.org/html/2508.04508v1#S3.SS2)). Finally, the model is optimized using geometry-aware loss functions (Sec. [3.3](https://arxiv.org/html/2508.04508v1#S3.SS3)).

### 3.1 Feedforward Geometry Reconstruction

As shown in Fig. [2](https://arxiv.org/html/2508.04508v1#S1.F2), Surf3R employs a feedforward network architecture to extract and fuse visual information across multiple input views without camera intrinsics and poses. Unlike traditional methods, which depend on estimating these parameters, Surf3R employs a single feedforward pass to directly reconstruct 3D surfaces through view-specific processing.

Multi-view Feature Encoding and Fusion.
Given $N$ input images $\{I_{i}\}_{i=1}^{N}$, a shared-weight Vision Transformer (ViT) is first employed to encode each image $I_{i}$ into visual tokens $F_{0}^{i}=\operatorname{ViT}(I_{i})$, with a spatial resolution reduced by a factor of 16.
To mitigate the limitation that a single reference view may not provide uniformly accurate geometric cues across the entire scene, we introduce a multi-branch architecture in which multiple reference views collaboratively participate in the reconstruction process.
Specifically, we select $M$ reference views $\{r_{m}\}_{m=1}^{M}$ to construct $M$ decoding branches $\{b_{m}\}_{m=1}^{M}$, each centered around a different reference view.
Within each branch $b_{m}$, a dedicated multi-view decoder, consisting of $D$ stacked Feature-Refine (FR) blocks, is employed to refine features via cross-view attention. These blocks are denoted as $\text{FRBlock}^{\text{ref}}_{d}$ for the reference view $r_{m}$ and $\text{FRBlock}^{\text{src}}_{d}$ for the remaining $N{-}1$ source views, where $d\in\{1,\ldots,D\}$.
At each decoding layer $d$, the token representations are updated in a view-specific manner. For a given view $I_{v}$, the decoder block takes as input the primary tokens $F^{v,m}_{d-1}$ from view $I_{v}$ and the secondary tokens $\mathcal{F}^{-v,m}_{d-1}=\{F^{i,m}_{d-1}\mid i\neq v\}$ from all other views. The update is performed as:

$$ $f^{v,m}_{d}=\begin{cases}\text{FRBlock}^{\text{ref}}_{d}(F^{v,m}_{d-1},\mathcal{F}^{-v,m}_{d-1}),&\text{if }v=r_{m}\\ \text{FRBlock}^{\text{src}}_{d}(F^{v,m}_{d-1},\mathcal{F}^{-v,m}_{d-1}),&\text{otherwise}.\end{cases}$ (1) $$

To further enhance the expressiveness of feature representations, we introduce a Cross-Reference Fusion (CRF) block after each decoder block to fuse and update per-view tokens computed under different reference views. Specifically, the updated feature is computed as:

$$ $\mathcal{F}^{v,m}_{d}=\text{CRFBlock}_{d}(f^{v,m}_{d},f^{v,-m}_{d}),$ (2) $$

where $f^{v,-m}_{d}=\{f^{v,1}_{d},\ldots,f^{v,m-1}_{d},f^{v,m+1}_{d},\ldots,f^{v,M}_{d}\}$ denotes the set of representations for view $v$ at layer $d$ across all other reference branches.

Gaussian Parameterizing.
Based on the fused multi-view features, we derive a sparse 3D point cloud.
Direct meshing with NKSR (Huang et al. 2023) is possible, but Sec. [4.2](https://arxiv.org/html/2508.04508v1#S4.SS2) shows that it yields poor surface fidelity because global 3D consistency is absent.
Accordingly, we adopt a unified 3D Gaussian representation whose projections into all views let per-view losses regularize the entire scene, producing smoother and more accurate surfaces.
To predict the final Gaussian parameters from $F^{v,m}_{D}$, we introduce two types of heads: $\text{Head}^{\text{ref}}$ for the reference view and $\text{Head}^{\text{src}}$ for the remaining source views. Each head comprises two sets of regression branches.
The first set includes a pointmap head and a confidence head, which respectively predict a 3D pointmap $P^{v,m}\in\mathbb{R}^{H\times W\times 3}$ and a confidence map $C^{v,m}\in\mathbb{R}^{H\times W}$ for each view.
The second set consists of Gaussian-specific heads that regress the per-pixel Gaussian parameters, including scaling factors $S^{v,m}\in\mathbb{R}^{H\times W\times 3}$, rotation quaternions $q^{v,m}\in\mathbb{R}^{H\times W\times 4}$, and opacity values $\alpha^{v,m}\in\mathbb{R}^{H\times W}$, which are essential for novel view synthesis.
Notably, the predicted pointmap serves as the center of the Gaussian, the input pixel color $I_{v}$ is used for its color and fix the spherical harmonics degree to be 0.
During inference, A model with M branches is used but the final per-view Gaussian
predictions are computed using the heads in the first branch.

### 3.2 Planar Geometry Formulation

To facilitate surface reconstruction from the predicted Gaussian parameters, we introduce a Depth-Normal Regularization strategy aimed at enhancing the accuracy of depth representation. This strategy leverages two fundamental planar geometric properties: normal and depth from our predicted 3D Gaussian primitives.

Flattening 3D Gaussians.
To enhance the capacity of Gaussians in modeling surface geometry, we first apply a flattening operation to the Gaussian primitives. Inspired by (Chen, Li, and Lee 2023),
we specifically introduce a scale regularization loss $\mathcal{L}_{s}$, which minimizes the smallest of the three scaling factors $\mathbf{S}=\left(s_{1},s_{2},s_{3}\right)^{\top}\in\mathbb{R}^{3}$ for each Gaussian:

$$ $\mathcal{L}_{\mathrm{s}}=\left\|\min\left(s_{1},s_{2},s_{3}\right)\right\|_{1}.$ (3) $$

By minimizing the loss, the Gaussian is driven towards a flat shape, effectively approximating a local surface plane.

Normal Map Rendering.
Once a Gaussian is flattened onto a local plane, the surface normal $\mathbf{n}$ is computed from the predicted rotation quaternion $q$ and scaling factors $S$ predicted by our feedforward network.
We first convert $q$ into a rotation matrix $R\in\mathbb{R}^{3\times 3}$. The normal is then defined as the direction corresponding to the smallest scaling factor: $\mathbf{n}={R}[k,:]\in\mathbb{R}^{3},k=\operatorname{argmin}\left(\left[s_{1},s_{2},s_{3}\right]\right)$.
The normal $\mathbf{n}$ is subsequently transformed into the camera coordinate system.
Finally, a rendered normal map $\hat{\mathbf{N}}$ is generated by a weighted summation of individual Gaussian normals $\mathbf{n}_{i}$ and their opacities $\alpha_{i}$ along each ray:

$$ $\hat{\mathbf{N}}={\sum_{i\in K}\mathbf{n}_{i}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j})}/{\sum_{i\in K}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j})},$ (4) $$

Depth Map Rendering.
To achieve more precise and geometrically consistent depth values than simply using the Gaussian’s center position (Tang et al. 2024), we compute the depth as the intersection point of a viewing ray originating from the camera center with the plane represented by the flattened Gaussian. Formally, the intersection depth $\mathbf{d}(\mathbf{n},\mathbf{p},\mathbf{r})$ is calculated by:

$$ $\mathbf{d}(\mathbf{n},\mathbf{p},\mathbf{r})=\mathbf{r}_{z}*(\mathbf{n}\cdot\mathbf{p})/(\mathbf{n}\cdot\mathbf{r}),$ (5) $$

where $\mathbf{r}_{z}$ is the z-value of the ray direction.
This formulation reveals that the intersection depth of a Gaussian is jointly determined by its position and surface normal, thereby enabling a more geometrically grounded and accurate depth estimation.
Leveraging this property, a view-consistent depth map $\hat{D}$ is rendered through a weighted summation of these intersection depths $d_{i}$, weighted by their opacities $\alpha_{i}$:

$$ $\hat{D}=\frac{\sum_{i\in K}d_{i}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j})}{\sum_{i\in K}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j})},$ (6) $$

where $K$ denotes the set of Gaussians along a ray, sorted by depth. The properties form the foundation for regularization.

**Table 1: Quantitative comparison on ScanNet++ dataset. Bold indicates best result. Our method achieves state-of-the-art performance across all metrics. *Surf3R-P*: point-map heads, trained with $\mathcal{L}_{c}$; *Surf3R-G*: + Gaussian heads, adds $\mathcal{L}_{r}$; *Surf3R-GD* (Full model): + D-Normal regularization, adds $\mathcal{L}_{s}$, $\mathcal{L}_{n}$ and $\mathcal{L}_{dn}$.**
| Method | Per-Scene | Feedforward |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| NeuS | 2DGS | SuGaR | PGSR | DUSt3R | Surf3R-P | Surf3R-G | Surf3R-GD (Ours) |  |
| Precision $\uparrow$ | 29.42 | 23.01 | 38.30 | 35.33 | 4.62 | 63.75 | 78.50 | 80.24 |
| Recall $\uparrow$ | 22.14 | 16.04 | 34.92 | 21.70 | 4.84 | 62.36 | 75.34 | 77.55 |
| F1-score $\uparrow$ | 25.13 | 18.30 | 36.12 | 24.92 | 4.06 | 62.89 | 76.72 | 78.71 |
| Time | $>$ 30 min | $>$ 1 min | $<$ 10 s |  |  |  |  |  |

### 3.3 Geometry-aware Loss Functions

To train our model effectively, we employ a set of loss functions tailored to guide the learning of geometry-aware representations.
We begin with a confidence-aware pointmap regression loss, denoted as $\mathcal{L}_{\text{conf}}$, which supervises the predicted 3D pointmaps $P^{k,m}$ using their associated confidence maps $Q^{k,m}$. The loss is defined as:

$$ $\mathcal{L}_{c}=\sum_{k,m}\sum_{p\in P^{k,m}}Q^{k,m}_{p}\left\|P^{k,m}_{p}-\mathbf{P}_{gt,p}\right\|_{1}-\beta\log Q^{k,m}_{p}$ (7) $$

where $P^{k,m}_{p}$ is the predicted 3D point, $\mathbf{P}_{gt,p}$ is the ground truth, $Q^{k,m}_{p}$ is the confidence score, and $\beta$ is a regularization parameter.
We also employ a standard RGB rendering loss $\mathcal{L}_{\text{r}}$ (Charatan et al. 2024), which supervises the rendered image against the ground-truth RGB image to preserve photometric fidelity.

To improve surface regularity, we introduce a flattening loss $\mathcal{L}_{\text{s}}$, which encourages the predicted Gaussian primitives to lie on locally planar surfaces.
In addition, we employ a rendered normal map loss $\mathcal{L}_{n}$ to align the rendered normal map $\hat{\mathbf{N}}$ with a reference normal map $\mathbf{N}_{\text{gt}}$, which is computed from the ground-truth depth via finite difference-based gradients.
The loss combines an $\ell_{1}$ term and a cosine similarity term, and is defined as:

$$ $\mathcal{L}_{n}=|\hat{\mathbf{N}}-\mathbf{N}_{\text{gt}}|_{1}+\left(1-\hat{\mathbf{N}}\cdot\mathbf{N}_{\text{gt}}\right),$ (8) $$

where the first term enforces per-pixel accuracy, and the second term promotes angular alignment. This supervision encourages the rendered normals to more faithfully reflect the underlying scene geometry.

While explicit normal regularization can effectively refine the orientation of 3D Gaussians, it has less impact on their positions. To address this limitation and ensure robust 3D surface reconstruction, we introduce a Depth-Normal (D-Normal) regularization strategy (Chen et al. 2024b), which enables joint optimization of both the orientation and positional accuracy of the Gaussians.
The D-Normal $\overline{\mathbf{N}}_{d}$ is derived from the rendered depth $\hat{D}$ by computing the cross-product of horizontal and vertical finite differences from neighboring points:

$$ $\overline{\mathbf{N}}_{d}=\frac{\nabla_{v}\mathbf{d}\times\nabla_{h}\mathbf{d}}{\left|\nabla_{v}\mathbf{d}\times\nabla_{h}\mathbf{d}\right|},$ (9) $$

where $\mathbf{d}$ represents the 3D coordinates of a pixel obtained via back-projection from the depth map.
Finally, the D-Normal regularization loss $\mathcal{L}_{dn}$ is defined as:

$$ $\mathcal{L}_{dn}=\left(\|\bar{\mathbf{N}}_{d}-\mathbf{N}_{gt}\|_{1}+(1-\bar{\mathbf{N}}_{d}\cdot\mathbf{N}_{gt})\right),$ (10) $$

Overall Loss. The final total loss $\mathcal{L}_{total}$ combines the geometric regularization losses for Gaussian primitives and the pointmap regression loss:

$$ $\mathcal{L}_{total}=\lambda_{c}\mathcal{L}_{c}+\lambda_{r}\mathcal{L}_{r}+\lambda_{s}\mathcal{L}_{s}+\lambda_{n}\mathcal{L}_{n}+\lambda_{dn}\mathcal{L}_{dn}$ (11) $$

The weighting factors $\lambda_{c}$, $\lambda_{r}$, $\lambda_{s}$, $\lambda_{n}$, and $\lambda_{dm}$ balance the contributions of each loss term, ensuring a holistic optimization of the reconstructed geometry and pointmaps.

## 4 Experiments

We begin by presenting the experimental setup in Sec. [4.1](https://arxiv.org/html/2508.04508v1#S4.SS1).
We assess the effectiveness and generalization capability of our approach for surface reconstruction in Sec. [4.2](https://arxiv.org/html/2508.04508v1#S4.SS2).
We further demonstrates the novel view synthesis capability of our method in Sec. [4.3](https://arxiv.org/html/2508.04508v1#S4.SS3).
Additionally, we validate the effectiveness of the proposed techniques in Sec. [4.4](https://arxiv.org/html/2508.04508v1#S4.SS4).

### 4.1 Implementation Details

We train our model on ScanNet++ dataset (Yeshwanth et al. 2023).
View sequences $\{I_{v}\}_{v=1}^{N}$ are generated with an overlap–based sampler.
Starting from a random keyframe, a candidate view is appended whenever the
overlap between its point cloud and the accumulated scene cloud falls within
$30\%\text{–}70\%$.
For training, each scene provides $100$ trajectories of $10$ views, yielding diverse yet geometrically consistent inputs.
For validation, we construct $1\,000$ trajectories with $30$ views on the ScanNet++ validation split and retain the $50$ widest-baseline views per scene for surface fusion to maximise spatial coverage.
Additionally, we conduct zero-shot generalization experiments on Replica dataset (Straub et al. 2019) to assess the cross-dataset adaptability of our model.

We train using 32 NVIDIA H800 GPUs, processing input views at a resolution of $224\times 224$.
For each training trajectory, the first $N=8$ views are used as input, from which $M=4$ reference views are randomly selected. The model is trained for 50 epochs, resulting in a total training time of approximately 40 hours. Additional training details are provided in the Appendix.

Figure: Figure 3: Qualitative comparison of surface reconstruction results on ScanNet++ dataset.
Refer to caption: x3.png

Figure: Figure 4: Qualitative comparison of zero-shot surface reconstruction results on Replica dataset.
Refer to caption: x4.png

### 4.2 Surface Reconstruction

We evaluate three progressively enhanced variants of our framework.
Surf3R-P employs only the pointmap heads and is trained with the reconstruction loss $\mathcal{L}_{c}$.
Surf3R-G extends this baseline by introducing Gaussian heads together with the associated rendering loss $\mathcal{L}_{r}$.
Surf3R-GD, the full approach, further incorporates the D-Normal regularization strategy and is optimized with the additional terms $\mathcal{L}_{s}$, $\mathcal{L}_{n}$ and $\mathcal{L}_{dn}$.
As shown in Tab. [1](https://arxiv.org/html/2508.04508v1#S3.T1), the results for per-scene methods are computed on the eight ScanNet++ validation scenes, while feed-forward models are evaluated across all 50 scenes, with the table reporting the dataset-wide averages.
Surf3R-GD achieves state-of-the-art surface reconstruction performance, with an F1-score of 78.71.
Compared to traditional per-scene reconstruction approaches such as NeuS (Wang et al. 2021b), 2DGS (Huang et al. 2024), SuGaR (Guédon and Lepetit 2024), and PGSR (Chen et al. 2024a), our method achieves significantly higher surface reconstruction quality.
In particular, compared to the concurrent method SuGaR (Guédon and Lepetit 2024), our approach yields a substantial improvement (78.71 vs. 36.12 in F1-score).
Moreover, our model exhibits exceptional efficiency, offering a reconstruction speed that is approximately $180\times$ faster than per-scene methods.
We further compare our method with feedforward-based approach DUSt3R (Wang et al. 2024).
DUSt3R and Surf3R-P reconstruct surfaces by first back-projecting point clouds to depth maps, which are then fused via TSDF.
In contrast, both Surf3R-G and Surf3R-GD leverage Gaussian rendering to directly estimate high-quality depth maps for mesh reconstruction.
As shown in Tab. [1](https://arxiv.org/html/2508.04508v1#S3.T1), Surf3R-P achieves a significant improvement over DUSt3R, which requires explicit global alignment, with an F1-score increasing from 4.06 to 62.89.
This underscores the advantage of aggregating geometric cues across all input views rather than relying on pairwise stereo matches processed one at a time.
Moreover, enriching Surf3R-P with a 3D Gaussian representation (Surf3R-G) enhances global geometric consistency and raises the F1-score from 62.89 to 76.72.
And the additional introduction of D-Normal regularization (Surf3R-GD) pushes it further to 78.71, yielding the best overall performance.
As shown in Fig [3](https://arxiv.org/html/2508.04508v1#S4.F3), our approach yields more accurate and complete reconstructions, particularly excelling at recovering planar surfaces and capturing fine-grained geometric details.

Moreover, our method demonstrates strong generalization capabilities. As shown in Tab. [2](https://arxiv.org/html/2508.04508v1#S4.T2), under zero-shot inference on the Replica dataset, Surf3R-GD also achieves state-of-the-art performance with an F1-score of 41.92.
Compared to traditional methods such as NeuralRecon (Sun et al. 2021), DUSt3R (Wang et al. 2024), and Surf3R-P+NKSR (Huang et al. 2023), it consistently outperforms all baselines. As shown in Fig. [4](https://arxiv.org/html/2508.04508v1#S4.F4), our method produces more complete and faithful surfaces, highlighting the superior generalization capability of our approach under unseen scenes.

**Table 2: Quantitative comparison on Replica dataset. Bold indicates best result. Our method achieves superior performance across all metrics. *Surf3R-P*: point-map heads, trained with $\mathcal{L}_{c}$; *Surf3R-G*: + Gaussian heads, adds $\mathcal{L}_{r}$; *Surf3R-GD* (Full model): + D-Normal regularization, adds $\mathcal{L}_{s}$, $\mathcal{L}_{n}$ and $\mathcal{L}_{dn}$.**
| Method | NeuralRecon | DUSt3R | Surf3R-P | Surf3R-P + NKSR | Surf3R-G | Surf3R-GD (Ours) |
| --- | --- | --- | --- | --- | --- | --- |
| Precision $\uparrow$ | 14.61 | 20.16 | 22.31 | 24.14 | 24.86 | 36.66 |
| Recall $\uparrow$ | 12.33 | 14.80 | 21.52 | 31.37 | 32.06 | 49.04 |
| F1-score $\uparrow$ | 13.41 | 16.90 | 21.88 | 27.16 | 27.96 | 41.92 |

**Table 3: NVS results on ScanNet++ dataset. Bold indicates best result. Our method achieves NVS rendering quality comparable with other Gaussian-based methods. *Surf3R-G*: + Gaussian heads, adds $\mathcal{L}_{r}$; *Surf3R-GD* (Full model): + D-Normal regularization, adds $\mathcal{L}_{s}$, $\mathcal{L}_{n}$ and $\mathcal{L}_{dn}$.**
|  | 4 Views | 12 Views | 24 Views |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ |
| DUSt3R | 11.66 | 0.47 | 0.63 | 10.72 | 0.46 | 0.67 | 10.81 | 0.40 | 0.68 |
| Surf3R-G | 13.21 | 0.64 | 0.31 | 16.77 | 0.55 | 0.30 | 17.69 | 0.55 | 0.26 |
| Surf3R-GD | 15.06 | 0.66 | 0.26 | 17.72 | 0.61 | 0.24 | 18.08 | 0.58 | 0.23 |

### 4.3 Muliti-view NVS on ScanNet++

As shown in Tab. [3](https://arxiv.org/html/2508.04508v1#S4.T3), Surf3R-GD consistently achieves the best novel view synthesis performance across all multi-view configurations on the ScanNet++ dataset.
With only 4 input views, it outperforms all baselines, achieving a PSNR of 15.06, SSIM of 0.66, and LPIPS of 0.26, demonstrating robust geometry-aware synthesis even under sparse view conditions.
As the number of input views increases to 12 and 24, our method maintains leading performance, particularly in perceptual quality metrics.
Notably, the LPIPS score drops to 0.23 at 24 views, outperforming DUSt3R (0.68), indicating more stable view synthesis and sharper geometric details.
These results underscore the strong generalization capability of our geometry-guided surface reconstruction framework, which not only delivers accurate 3D geometry but also enables high-quality NVS across varying input densities.

### 4.4 Ablation Study

#### View ablations.

To investigate the impact of input view count on reconstruction quality, we conduct an ablation study by varying the number of input views during inference. As shown in Table [4](https://arxiv.org/html/2508.04508v1#S4.T4), our method achieves the best performance when using 50 input views, with an F1-score of 41.92. Interestingly, increasing the number of views beyond this point does not lead to further improvements and may even slightly degrade performance. We attribute this to accumulated pose estimation errors from the input point clouds, which become more pronounced as the number of views increases, ultimately affecting the mesh reconstruction quality.

**Table 4: View Ablation Study on Replica dataset. Bold indicates best result. Performance comparison under different numbers of input views.**
| Views | 10 Views | 30 Views | 50 Views | 70 Views | 100 Views |
| --- | --- | --- | --- | --- | --- |
| Precision $\uparrow$ | 9.84 | 18.41 | 36.66 | 35.42 | 34.87 |
| Recall $\uparrow$ | 17.57 | 30.38 | 49.04 | 47.24 | 45.29 |
| F1-score $\uparrow$ | 12.29 | 22.88 | 41.92 | 40.47 | 39.37 |

#### Branch and Loss ablations.

As shown in Tab. [5](https://arxiv.org/html/2508.04508v1#S4.T5), when restricting the network to a single branch (one reference view) degrades the F1-score from 36.66 to 23.24 (Row A). This indicates that, with sparse views, a single reference cannot establish reliable geometric correspondences across wide baselines, resulting in poor reconstructions.
We also verify the effectiveness of different regularization terms on reconstruction quality.
As shown in Tab. [5](https://arxiv.org/html/2508.04508v1#S4.T5), both components contribute significantly to the reconstruction quality.
Excluding the scale term (Row B) and the normal term (Row C) results in a notable decline in all metrics.
Notabaly, removing the D-Normal term (Row D) leads to a substantial drop in performance, with the F1-score decreasing from 41.92 to 30.96. This suggests that the D-Normal regularization plays a critical role in encouraging the predicted normals to align with the underlying surface geometry.
Our full model (Row E) achieves the best performance across all metrics, demonstrating the benefits of both components in producing surface reconstructions.

**Table 5: Branch and Loss Ablation Study on Replica dataset. Bold indicates best result. Performance with different regularization terms.**
| Ablation Item | Precision $\uparrow$ | Recall $\uparrow$ | F-score $\uparrow$ |
| --- | --- | --- | --- |
| A. w/o Multi-branch | 23.24 | 30.90 | 26.53 |
| B. w/o Scale | 32.9 | 43.20 | 37.35 |
| C. w/o Normal | 34.5 | 45.80 | 39.35 |
| D. w/o D-Normal | 25.38 | 39.69 | 30.96 |
| E. Full | 36.66 | 49.04 | 41.92 |

## 5 Conclusion

In this paper, we propose Surf3R, a novel feed-forward framework for pose-free 3D surface reconstruction from sparse multi-view RGB inputs. Unlike traditional MVS methods that rely heavily on accurate camera calibration and iterative alignment, Surf3R eliminates the need for camera intrinsics or extrinsics by leveraging a cross-view attention mechanism and a multi-branch cross-reference fusion strategy. This enables effective feature propagation across arbitrarily selected views and mitigates the degradation caused by large viewpoint gaps.
Additionally, We introduce a novel Depth-Normal Regularizer grounded in 3D Gaussian representations, which integrates normal estimation into the geometric parameter learning process, yielding more consistent and detailed surfaces.
Extensive experiments on benchmark datasets such as ScanNet++ and Replica demonstrate that Surf3R achieves state-of-the-art performance in surface reconstruction while maintaining strong generalization ability in unseen scenarios.

## References

- Barron et al. (2021)
Barron, J. T.; Mildenhall, B.; Tancik, M.; Hedman, P.; Martin-Brualla, R.; and Srinivasan, P. P. 2021.
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields.
In *ICCV*.
- Bleyer, Rhemann, and Rother (2011)
Bleyer, M.; Rhemann, C.; and Rother, C. 2011.
PatchMatch Stereo-Stereo Matching with Slanted Support Windows.
In *BMVC*.
- Bonet and Viola (1999)
Bonet, J. S. D.; and Viola, P. 1999.
Poxels: Probabilistic Voxelized Volume Reconstruction.
- Broadhurst, Drummond, and Cipolla (2001)
Broadhurst, A.; Drummond, T.; and Cipolla, R. 2001.
A Probabilistic Framework for Space Carving.
In *ICCV*.
- Charatan et al. (2024)
Charatan, D.; Li, S. L.; Tagliasacchi, A.; and Sitzmann, V. 2024.
PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction.
In *CVPR*.
- Chen et al. (2022)
Chen, A.; Xu, Z.; Geiger, A.; Yu, J.; and Su, H. 2022.
Tensorf: Tensorial Radiance Fields.
In *ECCV*.
- Chen et al. (2024a)
Chen, D.; Li, H.; Ye, W.; Wang, Y.; Xie, W.; Zhai, S.; Wang, N.; Liu, H.; Bao, H.; and Zhang, G. 2024a.
PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction.
*CoRR*, abs/2406.06521.
- Chen, Li, and Lee (2023)
Chen, H.; Li, C.; and Lee, G. H. 2023.
NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance.
*CoRR*, abs/2312.00846.
- Chen et al. (2024b)
Chen, H.; Wei, F.; Li, C.; Huang, T.; Wang, Y.; and Lee, G. H. 2024b.
VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstruction.
In *NeuralIPS*.
- Crandall et al. (2013)
Crandall, D. J.; Owens, A.; Snavely, N.; and Huttenlocher, D. P. 2013.
SfM with MRFs: Discrete-Continuous Optimization for Large-Scale Structure from Motion.
*PAMI*, 35(12).
- Fridovich-Keil et al. (2022)
Fridovich-Keil, S.; Yu, A.; Tancik, M.; Chen, Q.; Recht, B.; and Kanazawa, A. 2022.
Plenoxels: Radiance Fields without Neural Networks.
- Fu et al. (2022)
Fu, Q.; Xu, Q.; Ong, Y. S.; and Tao, W. 2022.
Geo-neus: Geometry-consistent neural implicit surfaces learning for multi-view reconstruction.
In *NeurIPS*.
- Furukawa, Hernández et al. (2015)
Furukawa, Y.; Hernández, C.; et al. 2015.
Multi-view stereo: A tutorial.
*Foundations and Trends® in Computer Graphics and Vision*.
- Garbin et al. (2021)
Garbin, S. J.; Kowalski, M.; Johnson, M.; Shotton, J.; and Valentin, J. 2021.
FastNeRF: High-Fidelity Neural Rendering at 200 FPS.
In *ICCV*.
- Guédon and Lepetit (2024)
Guédon, A.; and Lepetit, V. 2024.
SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering.
In *CVPR*.
- Guédon and Lepetit (2024)
Guédon, A.; and Lepetit, V. 2024.
SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction.
In *CVPR*.
- Huang et al. (2024)
Huang, B.; Yu, Z.; Chen, A.; Geiger, A.; and Gao, S. 2024.
2D Gaussian Splatting for Geometrically Accurate Radiance Fields.
In *SIGGRAPH*.
- Huang et al. (2023)
Huang, J.; Gojcic, Z.; Atzmon, M.; Litany, O.; Fidler, S.; and Williams, F. 2023.
Neural Kernel Surface Reconstruction.
In *CVPR*.
- Kerbl et al. (2023)
Kerbl, B.; Kopanas, G.; Leimkühler, T.; and Drettakis, G. 2023.
3D Gaussian Splatting for Real-Time Radiance Field Rendering.
*ACM Transactions on Graphics*.
- Kutulakos and Seitz (1999)
Kutulakos, K. N.; and Seitz, S. M. 1999.
A Theory of Shape by Space Carving.
In *Proceedings of the International Conference on Computer Vision, Kerkyra, Corfu, Greece, September 20-25, 1999*, 307–314. IEEE Computer Society.
- Kutulakos and Seitz (2000)
Kutulakos, K. N.; and Seitz, S. M. 2000.
A Theory of Shape by Space Carving.
*International Journal of Computer Vision*, 38: 199–218.
- Leroy, Cabon, and Revaud (2024)
Leroy, V.; Cabon, Y.; and Revaud, J. 2024.
Grounding image matching in 3d with mast3r.
*arXiv preprint arXiv:2406.09756*.
- Li et al. (2025)
Li, C.; Zhu, H.; Chen, H.; Zhang, J.; Chen, T.; Yang, S.; Shao, S.; Dong, W.; and Zhang, B. 2025.
HRGS: Hierarchical Gaussian Splatting for Memory-Efficient High-Resolution 3D Reconstruction.
*CoRR*, abs/2506.14229.
- Lin et al. (2024)
Lin, J.; Li, Z.; Tang, X.; Liu, J.; Liu, S.; Liu, J.; Lu, Y.; Wu, X.; Xu, S.; Yan, Y.; and Yang, W. 2024.
VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction.
In *CVPR*.
- Liu et al. (2020)
Liu, S.; Zhang, Y.; Peng, S.; Shi, B.; Pollefeys, M.; and Cui, Z. 2020.
Dist: Rendering deep implicit signed distance function with differentiable sphere tracing.
In *CVPR*.
- Ma et al. (2023)
Ma, B.; Zhou, J.; Liu, Y.-S.; and Han, Z. 2023.
Towards better gradient consistency for neural signed distance functions via level set alignment.
In *CVPR*.
- Mildenhall et al. (2021)
Mildenhall, B.; Srinivasan, P. P.; Tancik, M.; Barron, J. T.; Ramamoorthi, R.; and Ng, R. 2021.
Nerf: Representing scenes as neural radiance fields for view synthesis.
*Communications of the ACM*.
- Müller et al. (2022)
Müller, T.; Evans, A.; Schied, C.; and Keller, A. 2022.
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding.
*ACM Transactions on Graphics*.
- Park et al. (2019)
Park, J. J.; Florence, P.; Straub, J.; Newcombe, R.; and Lovegrove, S. 2019.
Deepsdf: Learning continuous signed distance functions for shape representation.
In *CVPR*.
- Penner and Zhang (2017)
Penner, E.; and Zhang, L. 2017.
Soft 3D Reconstruction for View Synthesis.
*ACM Transactions on Graphics*.
- Reiser et al. (2021)
Reiser, C.; Peng, S.; Liao, Y.; and Geiger, A. 2021.
Kilonerf: Speeding Up Neural Radiance Fields with Thousands of Tiny MLPs.
In *ICCV*.
- Schönberger and Frahm (2016)
Schönberger, J. L.; and Frahm, J. 2016.
Structure-from-Motion Revisited.
In *CVPR*.
- Schönberger et al. (2016)
Schönberger, J. L.; Zheng, E.; Frahm, J.-M.; and Pollefeys, M. 2016.
Pixelwise view selection for unstructured multi-view stereo.
In *ECCV*.
- Seitz et al. (2006)
Seitz, S. M.; Curless, B.; Diebel, J.; Scharstein, D.; and Szeliski, R. 2006.
A comparison and evaluation of multi-view stereo reconstruction algorithms.
In *CVPR*.
- Seitz and Dyer (1999)
Seitz, S. M.; and Dyer, C. R. 1999.
Photorealistic Scene Reconstruction by Voxel Coloring.
*Int. J. Comput. Vis.*, 35(2): 151–173.
- Sitzmann et al. (2020)
Sitzmann, V.; Chan, E.; Tucker, R.; Snavely, N.; and Wetzstein, G. 2020.
Metasdf: Meta-learning signed distance functions.
In *NeurIPS*.
- Straub et al. (2019)
Straub, J.; Whelan, T.; Ma, L.; Chen, Y.; Wijmans, E.; Green, S.; Engel, J. J.; Mur-Artal, R.; Ren, C. Y.; Verma, S.; Clarkson, A.; Yan, M.; Budge, B.; Yan, Y.; Pan, X.; Yon, J.; Zou, Y.; Leon, K.; Carter, N.; Briales, J.; Gillingham, T.; Mueggler, E.; Pesqueira, L.; Savva, M.; Batra, D.; Strasdat, H. M.; De Nardi, R.; Goesele, M.; Lovegrove, S.; and Newcombe, R. A. 2019.
The Replica Dataset: A Digital Replica of Indoor Spaces.
*CoRR*.
- Sun et al. (2021)
Sun, J.; Xie, Y.; Chen, L.; Zhou, X.; and Bao, H. 2021.
NeuralRecon: Real-Time Coherent 3D Reconstruction From Monocular Video.
In *CVPR*.
- Tang et al. (2024)
Tang, J.; Ren, J.; Zhou, H.; Liu, Z.; and Zeng, G. 2024.
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation.
In *ICLR*.
- Tulsiani et al. (2017)
Tulsiani, S.; Zhou, T.; Efros, A. A.; and Malik, J. 2017.
Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency.
In *CVPR*.
- Ummenhofer et al. (2017)
Ummenhofer, B.; Zhou, H.; Uhrig, J.; Mayer, N.; Ilg, E.; Dosovitskiy, A.; and Brox, T. 2017.
Demon: Depth and Motion Network for Learning Monocular Stereo.
In *CVPR*.
- Wang and Agapito (2024)
Wang, H.; and Agapito, L. 2024.
3d reconstruction with spatial memory.
*arXiv preprint arXiv:2408.16061*.
- Wang et al. (2025)
Wang, J.; Chen, M.; Karaev, N.; Vedaldi, A.; Rupprecht, C.; and Novotný, D. 2025.
VGGT: Visual Geometry Grounded Transformer.
In *CVPR*.
- Wang et al. (2021a)
Wang, P.; Liu, L.; Liu, Y.; Theobalt, C.; Komura, T.; and Wang, W. 2021a.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-View Reconstruction.
*arXiv preprint arXiv:2106.10689*.
- Wang et al. (2021b)
Wang, P.; Liu, L.; Liu, Y.; Theobalt, C.; Komura, T.; and Wang, W. 2021b.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction.
In *NeurIPS*.
- Wang et al. (2024)
Wang, S.; Leroy, V.; Cabon, Y.; Chidlovskii, B.; and Revaud, J. 2024.
Dust3r: Geometric 3d vision made easy.
In *CVPR*.
- Yao et al. (2018)
Yao, Y.; Luo, Z.; Li, S.; Fang, T.; and Quan, L. 2018.
MVSnet: Depth Inference for Unstructured Multi-View Stereo.
In *ECCV*.
- Yariv et al. (2021)
Yariv, L.; Gu, J.; Kasten, Y.; and Lipman, Y. 2021.
Volume Rendering of Neural Implicit Surfaces.
*NeurIPS*.
- Yeshwanth et al. (2023)
Yeshwanth, C.; Liu, Y.-C.; Nießner, M.; and Dai, A. 2023.
Scannet++: A high-fidelity dataset of 3d indoor scenes.
In *ICCV*.
- Yifan et al. (2019)
Yifan, W.; Serena, F.; Wu, S.; Öztireli, C.; and Sorkine-Hornung, O. 2019.
Differentiable Surface Splatting for Point-Based Geometry Processing.
*ACM Transactions on Graphics*, 38(6): 1–14.
- Yu et al. (2022)
Yu, Z.; Peng, S.; Niemeyer, M.; Sattler, T.; and Geiger, A. 2022.
MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction.
In *NeurIPS*.
- Zak Murez and Rabinovich (2020)
Zak Murez, J. B. A. S. V. B., Tarrence van As; and Rabinovich, A. 2020.
Atlas: Endto-End 3D Scene Reconstruction from Posed Images.
In *ECCV*.
- Zhang et al. (2020)
Zhang, J.; Yao, Y.; Li, S.; Luo, Z.; and Fang, T. 2020.
Visibility-Aware Multi-View Stereo Network.
In *BMVC*.
- Zhang et al. (2025)
Zhang, S.; Wang, J.; Xu, Y.; Xue, N.; Rupprecht, C.; Zhou, X.; Shen, Y.; and Wetzstein, G. 2025.
FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views.
In *CVPR*.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please refer to weakness.

[1] Tang, Shitao, et al. "Mvdiffusion++: A dense high-resolution multi-view diffusion model for single or sparse-view 3d object reconstruction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
\
[2] Deng, Zijun, et al. "MV-Diffusion: Motion-aware video diffusion model." Proceedings of the 31st ACM International Conference on Multimedia. 2023.
\
[3] Charatan, D., S. Li, and A. Tagliasacchi. "Sitzmann, V. pixelSplat: 3D Gaussian splats from image pairs for scalable generalizable 3D reconstruction." arXiv preprint (2023).
\
[4] Tang, Shengji, et al. "Hisplat: Hierarchical 3d gaussian splatting for generalizable sparse-view reconstruction." arXiv preprint arXiv:2410.06245 (2024).
soundness: 3
strengths: * The paper addresses a highly relevant problem. Achieving end-to-end 3D reconstruction without camera pose estimation is significant, as traditional pipelines that rely on Structure-from-Motion (SfM) for pose estimation are computationally expensive and often fragile under sparse inputs. 

* The motivation of the paper is clear, and the paper is well-written and the proposed method is easy to follow and the overall presentation is clear. 

* Experimental results show that this method can achieve high quality surface reconstruction (Table 1, 2 and Fig. 3 & 4) compared to the baselines on both within dataset and cross dataset.
summary: The paper proposes a feed-forward, pose-free surface reconstruction method that recovers 3D surfaces from sparse RGB inputs without requiring camera calibration or pose estimation. Unlike traditional Structure-from-Motion (SfM) or Multi-View Stereo (MVS) pipelines, which rely on heavy preprocessing, the proposed approach enables fast surface reconstruction through a pose-free feed-forward model. The key component is a multi-branch, multi-view decoder that jointly processes multiple reference views using Feature-Refine (FR) and Cross-Reference Fusion (CRF) blocks to facilitate long-range cross-view interaction. Additionally, a depth-normal regularizer based on 3D Gaussian representation is introduced to enhance 3D consistency and surface detail. Experiments on ScanNet++ and Replica datasets demonstrate that the proposed method can achieve good surface reconstruction quality compared to baselines, while also supporting novel view synthesis.
weaknesses: * While the proposed multi-branch design and cross-reference fusion blocks appear effective for large-scale scene-level reconstruction from sparse views, the overall architectural concept feels relatively straightforward. Similar design principles though applied in different contexts have been explored in prior works [1,2].

* The combination of FR and CRF blocks, built on a multi-branch design, appears effective for representing scenes and facilitating feature communication across views. However, it is unclear how scalable this design is, as the multiple attention modules across branches likely make it memory- and computation-intensive. A discussion or experiment analyzing the computational and memory overhead of this design would greatly strengthen the paper.

* Although the method achieves good results over the point map based baselines like DusTr, I feel the method should also compare against other sparse view reconstruction methods based on 3DGS like PixelSplat [3] or HiSplat [4]. A comparison with these methods, will help to further reinforce the claims of the paper.

* A more comprehensive discussion on the failure mode will be helpful; currently it only describes its poor performance for transparent objects which is not interesting.

---

confidence: 4
contribution: 1
initial_rating: 0
presentation: 2
questions: Since my main concern is the copy of existing papers as own contributions, my only question to the authors is whether there is any other explanation for what I described in the weaknesses section and whether I missed anything.
soundness: 2
strengths: - The paper addresses a challenging and interesting problem: Feed-forward 3D surface reconstruction from sparse and unposed RGB images.
  - To the best of my knowledge, it is the first work in the line of DUSt3R follow-ups that focuses on direct watertight surface reconstruction.
- The qualitative and quantitative evaluation shows advantages over the chosen baselines.
- The ablation studies regarding use of 3D Gaussians instead of just point maps and the use of the Depth-Normal regularization strategy from prior work validate the effectiveness of the design choices.
- The paper and appendix provide details about the training recipe, helpful for reproducibility of experiments.
summary: The paper presents Surf3R, a method for feed-forward 3D surface reconstruction from unposed sparse views. Following the previous work of DUSt3R, the authors propose to use a multi-view, transformer-based architecture to 3D Gaussians instead of point maps that are then supervised via previously introduced confidence-aware point map regression, RGB rendering, normal, and 3D Gaussian flattening losses.
Additionally, the paper employs a D-Normal regularization loss, also introduced in prior work (VCR-GauS, NeurIPS 2024).
Experiments on ScanNet++ and zero-shot evaluation on Replica show improvements over feed-forward and optimization-based baselines.
The authors further ablate on the use of 3D Gaussians instead of point maps and TSDF fusion as well as the use of the D-Normal regularization loss.
weaknesses: - The authors blatantly sell ideas of two existing papers as their own contributions, while citing only one of these two papers insufficiently and without clearly stating what is their contribution and what not:
  - The architecture is 1:1 copied from MV-DUSt3R [1] without any citation of this work in this paper.
    - This paper proposes Feature-Refine (FR) blocks that correspond to the DecBlocks in MV-DUSt3R and the paper also shares almost the same notation (see equation 1 this paper vs equation 1 in MV-DUSt3R).
    - This paper proposes Cross-Reference Fusion (CRF) blocks that correspond to the CrossRefViewBlock in MV-DUSt3R.
    - Furthermore, in MV-DUSt3R, they have an additional MV-DUSt3R+ version for novel view synthesis by adding a 3D Gaussian head, which this paper does as well.
    - The method figure 2 of this paper is quite similar to Fig. 3 and Fig. 5 of MV-DUSt3R.
  - The Depth-Normal Regularization is adopted from VCR-GauS [2], but sold as one of this paper's contributions. 
    - The authors insufficiently reference this paper twice in lines 86 and 295: " we introduce a Depth-Normal Regularization strategy Chen et al. (2024b)" " we introduce a Depth-Normal (D-Normal) regularization strategy Chen et al. (2024b)".
    - They never state anything regarding what this prior work has done.
    - This paper includes an entire section of replicated content without any citation to this paper. Section 3.2 (Planar Geometry Formulation) in this paper clearly reformulates parts of Sections 3.2 and Sections 3.3 in VCR-GauS without citing it.
    - The Depth-Normal Regularization strategy that is copied from VCR-GauS is also mentioned without citation in their contributions in the introduction (line 104): "Furthermore, we introduce a Depth-Normal Regularization strategy to enhance geometric fidelity."
- The related works and baselines are outdated.
  - Regarding feed-forward 3D reconstruction from sparse and unposed images, the paper mostly limits itself to DUSt3R as reference, ignoring more recent follow-ups like MV-DUSt3R [1], VGGT [3], MASt3R-SfM [4], except for one sentence in related work, citing some of them (lines 147ff.).
  - Regarding optimization-based surface reconstruction approaches, the paper is also not up-to-date, missing Spurfies [5] and MAtCha Gaussians [6] in both related work and baselines and even missing VCR-GauS [2] as baseline, from which the Depth-Normal Regularization strategy was copied.
- Despite adopted from MV-DUSt3R, the architecture description in the method section is difficult to understand.

References:
- [1] MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds. CVPR 2025
- [2] VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstruction. NeurIPS 2024
- [3] VGGT: Visual Geometry Grounded Transformer. CVPR 2025
- [4] MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion. 3DV 2025
- [5] Spurfies: Sparse Surface Reconstruction using Local Geometry Priors. 3DV 2025
- [6] MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views. CVPR 2025

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: I am curious why the proposed cross-attention and multi-branch cross-reference fusion mechanisms outperform the local and global self-attention used in the VGGT paper. It would be helpful if the authors could provide a more detailed explanation or ablations demonstrating the advantage of their design.

In addition, I suggest that the paper more clearly differentiate the proposed approach from existing baselines such as DUSt3R, VGGT, and their follow-up works, emphasizing what specific innovations or insights this method introduces beyond them.
soundness: 2
strengths: 1. The writing is easy to follow
2. The proposed pipeline is efficient and achieves superior results compared to Dust3R and NeuralRecon
summary: This paper proposes an end-to-end feedforward approach that reconstructs 3D surfaces from sparse views without camera calibration.

D-Normal regularization is proposed for surface reconstruction based on an explicit 3D Gaussian representation.

Experiments demonstrate superior results compared to other baselines like neuralrecon and dust3r, and excellent efficiency.
weaknesses: Major:
1. The paper claims to propose a feed-forward approach for 3D surface reconstruction without camera calibration as one of its main contributions. However, this idea appears conceptually similar to Dust3R [1], VGGT [2], and their subsequent works. Moreover, the proposed Depth–Normal Regularization seems to originate from another existing paper [Chen et al. (2024b)]. Therefore, the novelty of this work is unclear. I encourage the authors to clarify how their approach differs fundamentally from these prior methods in the rebuttal.

2. It is not clearly stated what coordinate system the predicted point clouds are represented in. Are they in the camera coordinates of the reference view? In addition, how does the proposed method estimate camera poses from its predictions? 

3. Several important baselines are missing. In particular, more recent works following DUSt3R, such as Cut3R [3], VGGT [2], and SLAM-based approaches like WildGS-SLAM [4], should be included for a fair comparison. The authors should explain why these methods were not considered as baselines in the current evaluation.

Minor:

1. Lines 147–150 state that the primary focus of VGGT lies in view synthesis rather than high-fidelity surface reconstruction. I do not think this statement is accurate, as VGGT has demonstrated strong performance in reconstruction-related tasks.

2. For the novel view synthesis (NVS) evaluation, it would strengthen the paper to include comparisons with more recent state-of-the-art NVS methods rather than only Dust3R.

[1] Wang, Shuzhe, et al. "Dust3r: Geometric 3d vision made easy." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.
[2] Wang, Jianyuan, et al. "Vggt: Visual geometry grounded transformer." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.
[3] Wang, Qianqian, et al. "Continuous 3d perception model with persistent state." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.
[4] Zheng, Jianhao, et al. "Wildgs-slam: Monocular gaussian splatting slam in dynamic environments." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. **Dust3R Comparison Protocol:** Regarding the Dust3R[1] baseline, please clarify the experimental protocol:
    - How was rendering quality evaluated against Dust3R, given its output is a colored point cloud?
    - How was scale ambiguity handled? Please specify the evaluation domain (e.g., normalized or original reconstructed domain) used for this comparison.
2. **Justification of Novel Components:** Following Weakness #5, please provide a deeper analysis or ablation study to justify the specific design choices for the multi-reference view structure and the partitioned head design.
3. **Generalizability:** To better demonstrate the method's generalizability, we request an evaluation on other standard multi-view datasets, such as DTU[2] or BlendedMVS[3].

---
[1] Wang, Shuzhe, et al. "Dust3r: Geometric 3d vision made easy." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[2] Jensen, Rasmus, et al. "Large scale multi-view stereopsis evaluation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2014.

[3] Yao, Yao, et al. "Blendedmvs: A large-scale dataset for generalized multi-view stereo networks." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020.
soundness: 2
strengths: - Successfully tackles the underexplored problem of pose-free surface reconstruction from uncalibrated cameras, removing the need for prior camera calibration.
- Achieves real-time surface reconstruction in under 10 seconds, making it suitable for practical, interactive applications.
summary: Surf3R proposes a feedforward network for efficient surface reconstruction from uncalibrated RGB images. It uses a multi-branch design to process multiple reference views, capturing complementary geometric information. The method employs 3D Gaussian splatting for surface representation, allowing smooth and continuous reconstructions. To improve accuracy, a Depth-Normal (DN) regularization is introduced, optimizing both surface depth and normal consistency. Feature-Refine (FR) blocks and Cross-Reference Fusion (CRF) blocks are used to refine multi-view features and aggregate information across views. Surf3R is trained with geometry-aware losses, achieving fast real-time reconstructions in under 10 seconds while maintaining strong performance across different datasets.
weaknesses: 1. **Writing and Presentation:** The citation formatting is frequently inconsistent or incorrect, which hinders readability.
2. **Experimental Rigor and Clarity:** The validation of results is not sufficiently clear. Specifically, the comparison to Dust3R lacks detail, raising concerns about the fairness and validity of the evaluation.
3. **Missing Baselines (Reconstruction):** The experimental comparison could be strengthened by including other recent, high-performant baselines for feed-forward, pose-free point cloud reconstruction (e.g., Mast3R[1], VGGT[2]).
4. **Missing Baselines (GS):** For the meshing results (Surf3R-G and Surf3R-GD), the paper would be more comprehensive if it included comparisons against other relevant pose-free Gaussian splatting (e.g., PFSplat[3], NoPoSplat[4]) combined with meshing (e.g. TSDF).
5. **Novelty and Justification:** The method's conceptual novelty appears limited, as several components (e.g., the D-Normal loss) are incremental adaptations of existing work [5]. The core novel components, such as the multi-reference view structure of the network, currently lack sufficient theoretical motivation and experimental validation.

---
[1] Leroy, Vincent, Yohann Cabon, and Jérôme Revaud. "Grounding image matching in 3d with mast3r." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.

[2] Wang, Jianyuan, et al. "Vggt: Visual geometry grounded transformer." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.

[3] Hong, Sunghwan, et al. "PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis." Forty-second International Conference on Machine Learning.

[4] Ye, Botao, et al. "No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images." The Thirteenth International Conference on Learning Representations.

[5] Chen, Hanlin, et al. "Vcr-gaus: View consistent depth-normal regularizer for gaussian surface reconstruction." Advances in Neural Information Processing Systems 37 (2024): 139725-139750.


Decision: Reject