ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction
Authors: Ziyu Tang
Weicai Ye
Yifan Wang
Di Huang
Hujun Bao
Tong He
Guofeng Zhang
Venue: ICLR 2025 Spotlight (2025)
Keywords: Normal Deflection Fields
High-Fidelity Indoor Reconstruction
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction

Abstract: Neural implicit reconstruction via volume rendering has demonstrated its effectiveness in recovering dense 3D surfaces. However, it is non-trivial to simultaneously recover meticulous geometry and preserve smoothness across regions with differing characteristics. To address this issue, previous methods typically employ geometric priors, which are often constrained by the performance of the prior models. In this paper, we propose ND-SDF, which learns a Normal Deflection field to represent the angular deviation between the scene normal and the prior normal. Unlike previous methods that uniformly apply geometric priors on all samples, introducing significant bias in accuracy, our proposed normal deflection field dynamically learns and adapts the utilization of samples based on their specific characteristics, thereby improving both the accuracy and effectiveness of the model. Our method not only obtains smooth weakly textured regions such as walls and floors but also preserves the geometric details of complex structures. In addition, we introduce a novel ray sampling strategy based on the deflection angle to facilitate the unbiased rendering process, which significantly improves the quality and accuracy of intricate surfaces, especially on thin structures. Consistent improvements on various challenging datasets demonstrate the superiority of our method.

Published as a conference paper at ICLR 2025
ND-SDF: LEARNING NORMAL DEFLECTION FIELDS FOR HIGH-FIDELITY INDOOR RECONSTRUCTION
Ziyu Tang', Weicai Ye 12 Yifan Wang?, Di Huang', Hujun Bao', Tong He Guofeng Zhang' State Key Lab of CAD&CG, Zhejiang University, *Shanghai AI Laboratory attps://zju3ee gå titb io/nd-sdf/
Surface normal
3D dense surface
Figure I: We present ND-SDF, a framework for high-fidelity 3D indoor surface reconstruction from multi-views. Shown above is an extracted mesh from ScanNet++.
ABSTRACT
Neural implicit reconstruction via volume rendering has demonstrated its effec- tiveness in recovering dense 3D surfaces. However, non-trivial to simultane- ously recover meticulous geometry and preserve smoothness across regions with differing characteristics. To address this issue, previous methods typically employ geometric priors, which are often constrained by the performance of the prior models. In this paper, we propose ND-SDF, which learns a Normal Deflection field to represent the angular deviation between the scene normal and the prior normal. Unlike previous methods that uniformly apply geometric priors on all samples, introducing significant bias in accuracy, our proposed normal deflection field dynamically learns and adapts the utilization of samples based on their spe- cific characteristics, thereby improving both the accuracy and effectiveness of the model. Our method not only obtains smooth weakly textured regions such as walls and floors but also preserves the geometric details of complex structures. addition, we introduce a novel ray sampling strategy based the deflection an- gle to facilitate the unbiased rendering process, which significantly improves the quality and accuracy of intricate surfaces, especially on thin structures. Consistent improvements on various challenging datasets demonstrate the superiority of our method.

---

Published as a conference paper at ICLR 2025
INTRODUCTION
3D surface reconstruction Chen et al. (2024a); Wang et al. (2024); Ye et al. 2022:
2023b); Liu
al. (202T); Li al. (2020) aims to recover watertight, dense 3D geometry from multi-view im- ages Hartley & Zisserman (2003), representing a significant research area in computer vision and graphics. The recovered surfaces have proven invaluable for many downstream tasks, including robotic navigation, AR/VR, and smart cities.
Recently, coordinate- networks Mildenhall et al. (2021); Barron et al. (2022): ) Müller et al.
(2022): Martin-Brualla et al. (202T): Zhang et al.] (2020); Ye et al. (2023a); Huang et al. (2024); Ming et al. (2022) are drawing increasing attention, due to their remarkable performance in the task of novel view synthesis. Inspired by implicit representations of the scene, many subsequent works have introduced Signed Distance Functions (SDF) Park et al. (2019) or occupancy Li et al. (2022) to parameterize the 3D geometry. Notably, some of these approaches Yariv et al. (2021) Wang al.
(2021) outperform other methods in recovering dense surfaces by representing scene geometry with implicit distance fields and integrating them with differentiable rendering frameworks enhance surface reconstruction.
However, recovering high-fidelity surfaces remains challenging, as relying solely on color images for supervision often results in an underconstrained problem, especially for regions like textureless walls and ceilings. Previous methods Yu al. (2022b); Wang et al. (2022) have attempted to mili- gate this issue by incorporating auxiliary supervision. For instance, MonoS DF employs monocular cues from a pretrained model as ggeudo-goundd truth supervise the reconstruction process, which alleviates the issue in textureless regions. However, the large errors due to domain gaps in monoc- ular cues and the inconsistencies in view-dependent prior guidance often lead to visible detail loss and erroneous surfaces Yu et al. (2022b).
Noting that prior models are highly accurate in simpler regions like floors and walls but struggle with complex geometries. Based this insight, we introduce ND-SDF for high-fidelity indoor reconstruction [.. Our core innovation is the construction of deflection field that adaptively learns the geometric deviations between the actual scene geometry and the prior geometry derived from normal priors. The inherent deviation is defined as the angular difference between the true scene normals and the prior normals. By aligning the deflected scene normals with the normal cues, our model can adaptively learn the deviation, which encourages accurate recovery of intricate structures maintaining fine details without being misled by erroneous priors.
In addition, we propose a novel adaptive deflection angle prior loss that leverages prior normals for differential supervision of high and low-frequency areas, achieving an optimal balance between smoothness and detail. Our observations indicate that large angle deviations are primarily concen- trated in thin and fine-grained structures. Building on this insight, we introduce deflection-angle guided optimization to proactively facilitate the recovery of detailed structures. Furthermore, we address a significant bias issue and propose deflection angle guided unbiased rendering to improve the reconstruction of small or thin structures, shown in Figure [ In summary, we present the following contributions:
We propose a novel scene attribute field, named normal deflection field, which adaptively learns the deviations between the scene and normal priors. This method aids in dis- tinguishing between detailed and textureless areas. Therefore, we can restore more fine structures while ensuring the smoothness and integrity of the scene.
Empirically assuming that prior models incur larger errors in complex areas, we employ the deflection angle to discriminate between high and low-frequency regions. Consequently, we propose a novel adaptive deflection angle prior loss that dynamically adjusts the utiliza- tion of distinct cues, thereby achieving a balance between complex structures and smooth surfaces. Furthermore, we utilize the deflection angle to guide ray sampling and photomet- ric optimization, facilitating the restoration of finer-grained structures.
To address the inherent bias issue of neural surface rendering, we integrate the unbiasing method Zhang et al.] (2023) to implement an adaptive, unbiased rendering strategy. This approach facilitates the recovery of extremely thin structures without compromising scene fidelity. Our method outperforms previous approaches significantly in indoor reconstruc- tion evaluations, and this superiority is validated through extensive ablation experiments.
2

---

Published as a conference paper at ICLR 2025
2
RELATED WORK
Neural Surface Representation of 3D scenes Recently, the representation of 3D scenes using neural fields has gained popularity due to their expressiveness and simplicity. NeRF-type ap- proaches Mildenhall et al. (2021); Barron et al. (2022); Müller et al. (2022); Fridovich-Keil et al. 2022);[Sun et al. (2022) p have proposed encoding coordinate-bas
density and appearance of scenes by utilizing simple multilayer perceptrons (MLPs) and explicit structures such voxel grids, result- ing in photorcalistic novel view synthesis. However, these approaches fail to accurately recover surfaces due to the lack of constraints density. To address this issue, subsequent works, such as VoISDF Yariy et al. 2021) and Neus Wang et al. (202)) implicitly represent signed distance functions (SDFS and employ SDF-densiyy conversion function to encourage precise surface construction. Other methods Li et al. (2023b); Rosu & Behnke (2023); Wang et al. (2023), 3 Yariv al. (023); Fu et al. 20022 Zhang al. (2023) have also been proposed, introducing different representations and optimization techniques to further enhance reconstruction quality and efficiency. Nevertheless, these aforementioned methods struggle handle indoor scenes with a large number of low-frequency areas (e.g., walls and floors), as photometric loss becomes unreliable in such regions.
Neural reconstruction in Indoor Environments Due to the complex layouts of indoor scenes, additional auxiliary data are required for reasonable reconstruction. Manhattan-SDF Guo et al. (022) employs the Manhattan assumption and semantic priors to jointly regularize textureless re- gions such as walls and floors. HelixSurf Liang et al. (2023) achieves intertwined regularization iteratively by combining neural implicit surface learning with PatchMatch-
multi-view stereo (MVS) Barnes et al. (2009) robustly and efficiently. Other works Yu et al. (2022b); Wang et al. 2022 propose utilizing monocular priors from pretrained models to achieve smooth and complete reconstruction. However, directly applying normal or depth priors Yu et al. (2022b) may lead to undesirable reconstruction results due to the unreliable nature of these priors. NeuRIS Wang et al. (022)) filters out unreliable priors based on the assumption that areas with rich 2D visual features more error-prone. Park et al. (2024) simply utilizes uncertainty prior re-weight the normal prior loss. Nevertheless, both of these methods exhibit weak generalization capabilities. For example, the visual feature hypothesis Wang al. (2022) struggles differentiate between rich- textured planar areas. Similarly, prior-guided re- weighting Park et al. (2024) is further constrained by the domain gap between the prior model and the scene. On the other hand, DebSDF
(iaao Xiao et al.) (024) effectively designs an uncertainty field based on probabilistic model to guide the loss func- tion, leading to more robust and accurate reconstructions. eralization capabilities and those relying on probabilistic models, our proposed method, ND-SDF, learns a Normal Deflection field that enables the dynamic adaptation of priors based on their char- acteristics. This field directly models geometrically meaningful SO(3) residuals, resulting in highly detailed surface reconstructions while maintaining smoothness and robustness.
contrast to methods with weaker gen-
3 METHOD
The primary objective of our Normal Deflection Signed Distance Function (ND-SDF) is to recon- struct dense surfaces from calibrated multi-view images, with primary focus on indoor scenes with established priors. To achieve this, introduce novel component known as the normal deflection field. This field is designed to quantify and learn the deviations between actual scene geometries and their corresponding normal priors. By integrating this field, our method can adaptively supervise both high and low- fww-frryenccy regions in the scene. This adaptive supervision is crucial for preserving fine details while ensuring overall surface smoothness.
The operational framework of our approach is depicted in Figure 2 into a comprehensive discussion on the implementation of the normal deflection field. Additionally, we explore various strategies that leverage this field to enhance the fidelity of surface details, thereby promoting more accurate surface reconstruction.
In subsequent sections, we delve
3.1 PRELIMINARIES
Volume Rendering NeRF assumes that a ray r(t) = o ty is emitted from viewpoint o in direction where denotes the distance from the viewpoint. N points sampled along the ray, i.e. x,
3

---

Published as a conference paper at ICLR 2025
Depci Priot
Normal Priot
Dellection angle guided sampling
Adaptive deflection angle peior loss
X.PE
Defiection angle
Depth
ANommal
Deflertion angle guided rendering
RGB
Al
Figure 2: Overview of Our method. We utilize multi-resolution hash grids Yr. as scene represen- tation. The core ND-SDF the normal deflection field, which we represent using quaternions predicted by the deflection network (denoted as fa). We align the deflected normals with the prior normals to learn the deviation between the scene and the priors. To distinctly supervise high and low-frequency areas, employ an adaptive deflection angle prior loss, ensuring both smoothness and detail. Furthermore, we utilize the deflection angle A6 to distinguish complex structures, en- abling angle guided sampling and color loss facilitate the recovery of intricate surface details. Lastly, we combine the unbiased rendering method Zhang et al. (2023) to ensure the generation of extremely thin structures indoors.
o i E (1, J...N.. Given each point's volume density a, and color c; the color the ray C(r) can be synthesized using volume rendering techniques as: i-1
C(r) T,a., a; = exp (
II (1-og). j=1
(1)
i=1
where is the opacity of i-th ray segment, T; denotes the transmittance of light reaching the i-th ray segment, 8, = 4-11 represents the distance between adjacent sample points.
A color loss supervised using ground-tran -tund (GT) color is utilized to optimize the neural fields:
Loolor Σ |l((r) C(r)
FER
where R is the sampled ray set, C(r) is the GT color.
SDF-induced Volume Rendering The rise neural surface reconstruction is attributed to seminal works like VoISDF and NeuS, which propose specific SDF to volume density conversion, allowing optimization via volume rendering. The final surface, denoted as S, is equivalent to the zero level-set
of the implicit distance field, i.e., S {x TR3 [x() 0), where sxx) is the SDF value.
We follow VoLSDF Yariv et al. 22021) employing the Laplace Cumulative Distribution Function (CDF) to model the relationship between SDF and volume density:
(x) 1 (-s(x))
{
exp
(x))
3
sf 2 0
(3)
exp
s(N)
if sf) <0
3
28
where V denotes the Laplace CDF, 3 denotes the variance. Depth (rr) and normal Ñ(r) the surface intersection are synthesized using the same approach as that employed for synthesizing colors:
N
D(T) Tast. N(r) = Σ T.ann
1=1
Here, n, is the analytical gradient the SDF network at point Following MonoSDF Yu et al. b022bb, utilize monocular depth and normal priors generated from pretrained model Eftekhar et al..a (2021) to supervise the rendered depth and normal:
Ldepth ErEr D(r) + + (r)]11
Lnormal = E [[((r) - N(r) + 1 ((r))N())]]
(5)
4

---

Published as a conference paper at ICLR 2025
where the two coefficients (w, 9) obtained by least square algorithms are utilized to align the scale between monocular depth and rendered depth, i.e..u
D.
We encode scene geometry using Instant-NGP Müller et al. (2022) and a shallow MLP fg. that is, (s(x) E I2, z(x) E R226)) = fg (x, (x))), x)), where x(x) denotes the latent geometry feature.
Also, utilize the eikonal term Gropp al. (2020); Yariv et al. 2020) to regularize the shape of SDF in 3D space:
1
Leik
(x:)ll -
(6)
Following Neuralangelo| Li et al. (202b)), we further employ numerical gradients to enhance surface geometric consistency and use curvature loss for smoothness. The detailed definitions of them provided in the appendix.
3.2 NORMAL DEFLECTION FIELD
significant challenge in indoor 3D reconstruction is achieving balance between the smoothness of overall surfaces and the intricacy of complex structures. Traditional approaches relying solely on photometric loss have proven inadequate for accurately capturing smooth areas, particularly in texture-deficient regions. Later methods often leverage auxiliary data, such normal priors, to enhance the reconstruction quality textureless areas. However, the uniform application of normal priors across different scene types can impair the recovery of complex structures. This is primarily due to the variable reliability of these priors in diverse regions, where they may not accurately reflect the underlying geometry. To address this issue, propose the development of a Normal Deflection Field. This field designed to dynamically represent the existing 3D angular deviation between the actual scene normals and the provided normal priors. By doing so, it effectively circumvents potential misguidance arising from inconsistent priors, thereby enabling more reliable and nuanced reconstruction of both smooth and complex indoor structures.
Specifically, we choose quaternion as the deflection form, which is lightweight rotation represen- tation. The quaternion can be parameterized by a single MLP fa:
4.- = fx n;, z:),
where q, = (q q 4 g is the deflection quaternion (normalized default) for the sampled x;- The quaternion at the surface intersection point is synthesized using volume rendering techniques, analogous to the way NeRF synthesizes colors:
N
Q(r) Toaq.
(8)
We deflect the rendered normal using the synthesized deflection quaternion Q(T):
N Ne (r) Q(r) @ Ñ(r) X (r).
where 2d denotes the deflected rendered normal,
denotes the inverse of Q. also known as the conjugate, and @ a quaternion multiplication operation. Since a quaternion can be represented in 2 + sin (u' i, u²j, u²k), this operation is to rotate the rendered (u² u2 n²) by 0 degrees.
trigonometric form, i.e. Q cos #
normal around the quaternion axis
The deviation between the scene and priors is learned by minimizing the difference between the deflected rendered normal and the prior normal. Thus, we define the deflected normal loss:
Cd
I (r)-)(()ll + 11 N (r) N N(r)], (
(10)
PER
3.3 ADAPTIVE DEFLECTION ANGLE PRIOR LOSS
We assume that the learned deviation is encompassed within the actual deflected angle when the scene normal is deflected. This deflected angle (20) is computed follows:
AO- arccos (Ň(r): N (r)),
(11)
5

---

Published as a conference paper at ICLR 2025
where A0 [0, rr. Empirically, the deviation between the scene and priors increases the structure becomes more complex. This intuitively suggests that the deflection angle is small in smooth regions and large in high-frequency zones. Based on this, the adaptive deflection angle normal prior loss is proposed to dynamically adjust the utilization of differing priors based on their characteristics:
Lnormal - Erer 9 ( (Ñ' (r), N(r)) + 9(00)) (r),
(12)
where g and 9 are modulation functions that adjust the weights of the deflected and original normal loss terms based on the deflection angle. We similarly define an adaptive depth prior loss:
Cad depth
Σ g( (dAf) (D(r), D(r)).
(13)
FER
As AO increases, g increases, and y (0) decreases, since we should less apply priors in complex areas indicated by large deflection angles. The combined adaptive normal and depth prior loss is termed the adaptive deflection angle prior loss (as illustrated in Figure (); see appendix for detailed definitions of the proposed modulation functions.
3.4 DEFLECTION ANGLE GUIDED OPTIMIZATION
Through the utilization of the proposed adaptive prior loss, we dynamically adjust the utilization of various priors, significantly enhancing the quality of reconstruction. However, this alone is insuffi- cient for generating more complex structures. Fundamentally. merely learning the deviation does not endow the capability to reconstruct additional details. Recognizing that large deflection angles in- dicate complex areas, we introduce three deflection angle guided optimization methods specifically designed to facilitate the recovery of more thin and fine-grained structures.
Deflection angle guided sampling. Textureless areas. such as walls, constitute significant por- tion of indoor scenes. These regions converge rapidly by leveraging prior knowledge, partly due to their inherent simplicity. To prevent continuous oversampling of well-learned smooth regions, we proactively sample more rays in complex areas to capture finer details. The sampling process is naturally guided by the learned deflection angles (see Appendix A.3.2 for details).
Deflection angle guided photometric loss. To further enhance detail recovery, we impose addi- tional photometric loss complex areas identified by the deflection angles. The original color loss term is re-weighted based on the deflection angle, resulting in re- re-weighted color loss.
Cootor
Weotor (AI(T)))((()) C(r)ll1-
(14)
(r(r) denotes the actual deflected angle and Weotor is the re-weighting function (Appendix A.3.2
Deflection angle guided unbiased rendering. Accurately reconstructing thin structures, such as chair legs, remains challenging. This limitation arises from the inherent bias issues in SDF-induced volume rendering, manifesting in two ways: (1) According to TUVR TUVR[Zhang et al. (2023). the deriva- tive of the rendering weight @W()) is influenced by various angle differences between ray directions 0t and scene normals. This results in non-maximum value of w at the surface intersection point, thereby degrading the surface quality; (2) For rays passing close to the object, the Laplace CDF as- signs a high density to points near the surface, which leads to incorrect rendering weights, and sub- sequently affects the depth and normal compositing. This issue causes thin structures to disappear during training. Therefore, is crucial to apply an unbiased rendering method during reconstruction. We illustrate this biased scenario in Fig. highlighting the significance of ensuring unbiasedness.
Inspired by TUVR Zhang et al. 2023). we transform SDF to density using an unbiased function:
(f((((())) 3 U (F((((())))
1
(r((,))
(15)
where r(t:) + f.v is a sampled point on the ray, and is the geometry network, which predicts the SDF value. This transformation effectively mitigates bias, as shown in Fig 3 However, we observe that simply applying it leads poor convergence of the overall surface. Therefore, we only partially apply it to thin structures indicated by the learned deflection angles. detailed analysis of the convergence issue is presented in Appendix B.I

---

Published as a conference paper at ICLR 2025
Densdy
deaidy
ray
uedentet welgh!
Figure 3: We present scenario in which ray passes close to chair leg. Along the ray, point A is situated near the chair leg, while point located at the ray-ground intersection. In the absence of unbiasing, abnormal density and rendering weight are observed at point A, whereas density should be exclusively assigned to point B.
Method
Acc.
Comp4
Pref
Recallt
Chamfer.
F-scoref
COLMAP Schönberger el al. 2016
0.047
0.235
71.1
44.1
0.141
53.7
VoISDE Yanv et al. 202
0.414
0.120
32.1
39.4
0.267
34.6
Neus Wang al. 2021
0.179
0.208
31.3
27.5
0.194
29.1
NeuRIS Wang el al.
2022
0.050
0.049
71.7
66.9
0.050
69.2
MonoSDF Yu
2022b
0.035
0.048
79.9
68.1
0,042
73.3
HelixSurf Liang
2023
0.038
0.044
78.6
72.7
0.042
75.5
DebSDF Xiao et al.
(0244)
0.036
0.040
80.7
76.5
0.038
78.5
H2OSDF
Park
(2024)
0.032
0.037
83.4
76.9
0.035
79.9
Ours
0.031
0.036
84.0
80.3
0.034
82.0
Table 1: Quantitative results the ScanNet dataset. Our method substantially outperforms
previously best-performing methods across all metrics to date.
3.5 OPTIMIZATION
The overall loss function is defined as:
L Ce + AiCkk + X2heurv + Ag depttpth-
(16)
The weights AI... are utilized to balance the importance of these loss terms.
4 EXPERIMENT
Datasets We conducted experiments on four indoor datasets: ScanNet Dai et al. (201 Replica Straub al. (2019), TanksandTemples Knapitsch et al. (2017). and ScanNet++ Yeshwanth al. (002)). ScanNet contains 1513 indoor scenes captured with an iPad, processed using BundleFusion algorithm to obtain camera poses and surface reconstruction. Replica is synthetic dataset comprising 18 indoor scenes. Each scene features dense geometry and high dynamic range textures. Tenksasd is large-scale 3D reconstruction dataset including high-resolution out- door and indoor environments. We followed the splits from MonoSDF and applied the same eval- uation settings. We also conducted experiments on ScanNet++. which includes 460 indoor scenes captured using laser scanners. These scenes offer high-quality dense reconstructions and images. For testing, we selected six scenes from ScanNet++.
Implementation details Our method was implemented using PyTorch Paszke et al. (2019). The image resolution for all scenes is 384x384. We obtained normal and depth cues using Om- nidata [Eftekhar et al.] (2021). Multi-resolution hash grids were utilized for scene representation. Both the geometry network and color network consists of two layers, each with 256 nodes. Our network was optimized using Adam Loshchilo & Hutter (2017) optimizer with learning rate of le-3. The weights for loss terms were: A₁ 0.05, X 0.0005, X3 = 0.025, X = 0.05. Upon adequate initialization of the deflection field, we initiated deflection angle guided sampling, photometric optimization, and unbiased rendering. All experiments were conducted on an NVIDIA TESLA A A100 PCLe 40GB, with each iteration sampling 4x 1024 rays, totaling 128,000 training steps. The training time was about 14 GPU hours. Our hash encoding resolution spanned from 2 to 211 across 16 levels, with the initial activation level to 8 and activation steps 2000.

---

Published as a conference paper at ICLR 2025
Soli
DehSDF
Figure 4: Qualitative results on ScanNet. Our approach effectively generates thin structures, such as chair legs, and produces more accurate surfaces.
Method
Auditorium
Ballroom
Courtroom
Museum
Mean
NeuRIS Wang et al. 2022 MonoSDF (Gnd) Yu el al A122b
0.79
0.84
2.78
1.34
1.13
6.32
6.43
18.73
8.96
10.11
DebSDF (Grid) Xiao el al 2024
7.45
4.21
19.33
7.64
9.66
Ours
7.07
10.74
23,92
16.11
14.46
Table 2: F-score on the LunnksandTeeplee dataset. 'Grid' indicates using hash encoding as the
feature representation backend. More detailed comparison results can be found in the appendix.
Metrics accordance with prior researches, we employ six standard metrics to evaluate the re- constructed meshes: Accuracy, Completeness, Chamfer Distance, Precision, Recall, and F-score. Additionally, normal consistency is used to evaluate the Replica dataset.
Baselines We compare our approach with the following methods: (1) traditional MVS techniques COLMAP (.chobbrScoonerggerr & Frahm (2016); [Schönberger et al. (2016); (2) basic neural implicit meth- ods, including VoISDF and NeuS; (3) methods incorporating auxiliary data, including MonoSDF and NeuRIS; (4) other indoor reconstruction methods, such as HelixSurf Liang et al. (2023)
4.1 COMPARISONS
ScanNet: In Figure we present qualitative results and in Table quantitative results are displayed. These quantitative results are averaged over the selected four scenes, consistent with MonoSDE. ND- SDF significantly surpasses previous best-performing works, achieving state-of-the-a performance. Specifically, we achieve the highest F-score, reliable metric that reflects reconstruction accuracy by balancing both Accuracy and Completeness. When compared with HelixSurf and DebSDF (as shown in Figure 4 approach also accurately captures thin structures, such as chair legs. The results highlight the effectiveness of our proposed deflection methods, which substantially enhance the surface accuracy and the recovery of thin and fine-grained structures in indoor environments.
Method
Acc|,
Comp..
Pret
Recallf
Chamfer F-scoref
VolSDF Yariv et al. 2021
0.070
0.102
0.405
0.339
0.086
0.368
Baked- Angelo Yu et al 2022a
0.152
0.039
0.543
0.718
0.095
0.614
MonoSDF (Grid) Yu al 202Zb
0.057
0.032
0.651
0.703
0.044
0.675
DebSDF (Grid) Xıao et 2024
0.060
0.031
0.669
0.727
0.045
0.696
Ours
0.056
0.024
0.667
0.785
0.040
0.721
Table 3: Quantitative results on the ScanNet++ dataset. We select scenes and calculate their
average metrics. Baked-Angelo is the best-performing reconstruction method tested without priors.
Replica: Following MonoSDF Yu et al. 2022b)) used their pre-processed masks to filter abnormal priors. Quantitative results are presented Table 4 where our approach achieves the highest F-score and the lowest Chamfer Distance, significantly surpassing previous methods.
Tanks and temples:
We also conducted experiments on the advanced split of T&T, featuring several challenging large-scale indoor scenes. As shown in Table ND-SDF achieves the highest average F.
8

---

Published as a conference paper at ICLR 2025
Method
Normal C.f
Chamfer),
F-scoref
Unisurl Oechsle
2021
90.96
4.93
78.99
MonoSDI (Gnd] Yu 2022b
90.93
3.23
85 35.91
DehSDF (MIP) Yu et al. Ours
2022h
93.23
2.90
88.36
92.85
2.57
91.60
Table 4: Quantitative results of Replica dataset. Note that DebSDF reported results only with
MLP representations.
Method
Omnidata Eftekhar et al. 2021 Wizurd Fu ct al. 2024
Base
0.700
0.661
Ours
0.750
0.730
Table 5: F-score of Different Priors Models ScanNet++.
score compared to previous methods. Additional comparison results can found Appendix C.3 where method recovered substantial detailed structures.
ScanNet++ comprises numerous complex indoor scenes, each offering high high-resolution GT mesh and RGB data. Since public results are available for ScanNet++, trained several baselines methods, including VoISDF, MonoSDF, DebSDF and Baked Annelo Yu et al. (2022a). Among these, MonoSDF and DebSDF utilized monocular priors we did. The quantitative results, summarized in Table demonstrate the state-of-the-an performance our method, particularly in terms of the Recall metric. This highlights the capability of our approach recover significantly finer structures of the GT surface. Additional visualization results are prensented in Appendix C.2
ScanNet++:
ModelA
ModklBB
ModelC
Figure 5: Visualization Results of the Models after Ablation ScanNet++. Detailed definitions of these models and corresponding quantitative results are provided in Table 0
4.2
ABLATION STUDY
4.2.1
ABLATION OF DIFFERENT MODULES
The learned deviations play a crucial role in locating high-frequency regions. Consequently. we employ deflection angles for sampling, photometric optimization, and unbiased rendering to facil- itate the recovery thin and fine-grained structures. We conducted detailed ablation experiments assess the effectiveness of these modules, resulting in five post-ablation models: Base, ModelA,

---

Published as a conference paper at ICLR 2025
Method
Base
ND
AP
DO
DU
Acel
Compl
Prect
Recallf
Chamner)
F-score"
Base
x
x
X
X
0.046
0.046
0.576
0.640
0.046
0.606
ModelA
x
X
0.044
0.042
0.601
0.666
0.043
0.632
ModelB
X
0.038
0.032
0.637
0.726
0.035
0.679
ModelC
4
X 0.040
0.030
0.631
0.740
0.035
0.681
Ours
V 0.038
0.031
0.648
0.728
0.034
0.686
Table 6: Quantitative Results of Ablation for Different Modules on ScanNet++. 'Base' can
be viewed simple combination of MonoSDF and Neuralangelo. 'ND' denotes applying the deflection field and the deflected normal loss term L normall 'AP" refers to the adaptive deflection angle prior loss terms including Lad normal and Lad depth 'DO' denotes the two deflection angle guided optimizations, namely sampling and photometric loss. 'DU" denotes the deflection angle guided unbiased rendering.
ModelB, ModelC, and Ours. For quantitative results and comprehensive definitions, please refer to Table 6 Figure 5 visually illustrates how the proposed components enhance overall performance.
Analysis of Normal Deflection Field The Base model strictly adheres to monocular cues, leading to significant detail loss due to misguided priors in complex areas. In ModelA, we introduced the deflection field and directly applied the deflected normal loss to learn deviations. The improvement in the F-score 0.632 (as shown in Table (6) confirms the superiority of the deviation learning approach. However, we observed wrinkles (as depicted in Figure 5 in textureless regions like walls. This issue arises from the lack of constraints on the deflection, resulting in inefficient utilization of priors in smooth regions. In Model B, the introduction of the adaptive deflection angle prior loss significantly enhances reconstruction quality, striking a balance between smoothness and detail.
Analysis of deflection angle guided optimization While ModelB substantially improves recon- struction quality compared to the Base, still struggles with finer structures (as depicted in Fig- ure 5). In ModelC, we introduced deflection angle guided sampling and color loss to emphasize surface details. Finally, we incorporated an adaptive unbiased rendering method to facilitate the recovery of thin structures. Figure F illlustaates that Ours recovers wide range of complex and fine structures, such as skull models and iron swabs, surpassing the Base model. As indicated in T Ours achieved the highest F-score, quantitatively outperforming other models. This ablation study strongly validates the effectiveness the proposed modules in restoring high-fidelity surfaces.
4.2.2 ABLATION OF DIFFERENT PRIOR MODELS
Our method learns deviations between the scene and normal priors predicted by specific pretrained model. It is essential to evaluate its effectiveness across cues generated by different pretrained mod- els. To achieve this, we employed the statc-of-the-arr method Geowizard Fu et al. (2024) to generate both depth and normal cues. The quantitative results, summarized in Table 5 demonstrate that method significantly enhances surface quality irrespective of the cues utilized. This experiment il- lustrates the superiority of our approach. It invariably improves reconstruction accuracy in scenarios where prior knowledge is used.
5 CONCLUSION
We have presented ND-SDE, a novel approach that leams deviations between the scene and nor- mal priors for high-fidelity indoor surface reconstruction. We introduce adaptive deflection gle prior loss to dynamically supervise areas with varying characteristics. By identifying high- frequency regions based on deflection angles, we employ angle guided optimization to generate thin and fine-grained structures. Our method recovers substantial number of complex structures, as demonstrated by extensive qualitative and quantitative results that confirm its superiority. ND-SDF efficiently adapts to unreliable priors in challenging areas, significantly addressing the detail loss issue caused by large prior errors in regions with fine structures.
L Limitation ND-SDF is confined to reconstruction scenarios involving priors. Our approach exhibits limited capability in handling areas with less coverage. Due to inherent ambiguities observations, the deflection field may learn incorrect structures and converge to local optima. We encourage the reader to refer the supplementary material for more analysis and results. the future, we may explore multi-view consistency constraints enhance reconstruction quality.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was partially supported by Key R&D Program of Zhejiang Province (N..223301039) and NSF of China (No.62425209).
REFERENCES
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. PatchMatch: A ran- domized correspondence algorithm for structural image editing, ACM Trans. Graph., 28(3):24, 2009.
Jonathan Barron, Ben Mildenhall, Dor Verbin, Pratul Srinivasan, and Peter Hedman. Mip-NeRF 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattenn Recognition, pp. 5470-5479, 2022.
Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang, Weijian Xie, Shangjin Zhai, Nan Wang. Haomin Liu, Hujun Bao, and Guofeng Zhang. PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction. arXiv preprint (iix:2240000652.... 2024a
Ziyi Chen, Xiaolong Wu, and Yu Zhang. NC-SDF: Enhancing Indoor Scene Reconstruction Using Neural SDFs with View-Dependent Normal Compensation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5155-5165, 2024b.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. ScanNet: annoty-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5828-5839, 2017.
Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10786-10796, 2021.
Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5501-5510, 2022.
Qiancheng Fu, Qingshan Xu, Yew Soon Ong, and Wenbing Tao. Geo-NeuS: Geometry-consisteen neural implicit surfaces learning for multi-view reconstruction. Advances in Neural Information Processing Systems, 35:3403-3416, 2022.
Xiao Fu, Wei Yin. Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojic Shen, Dahua Lin, and Xiaoxiao Long. Wizaiz Unleashing the Diffusion Priors for 3D Geometry Estimation from Single Image. arXiv preprint 2403 2024.
Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regu- larization for learning shapes. arXiv preprint arXiv:2002. 10099, 2020.
Haoyu Guo, Sida Peng, Haotong Lin, Qianqian Wang, Guofeng Zhang, Hujun Bao, and Xiaowei Zhou. Neural 3d scene reconstruction with the manhattan-world assumption. In Proceedingy of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5511-5520, 2022.
Richard Hartley and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge
university press, 2003.
Chenxi Huang, Yuenan Hou, Weicai Ye, Huang, Xiaoshui Huang, Binbin Lin, Deng Cai, and Wanli Ouyang. NeRF-Det++: Incorporating semantic cues and perspective-aware depth supervi- sion for indoor multi-view 3d detection. arXiv preprint arXiv:2402. 14464, 2024.
Arno Knapitsch, Jacsik Park, Qian Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions Graphics (ToG), 36(4):1-13, 2017.
Hai Li, Weicai Ye, Guofeng Zhang, Sanyuan Zhang, and Hujun Bao. Saliency guided subdivision for single-view mesh reconstruction. In 2020 International Conference on 3D Vision (3DV), pp. 1008-1107. IEEF, 2020.
11

---

Published as a conference paper at ICLR 2025
Hai Li, Xingrui Yang, Hongjia Zhai, Yuqian Liu, Hujun Bao, and Guofeng Zhang. Vox-Surf: Voxel- based implicit surface representation. IEEE Transactions on Visualization and Computer Graph- ics, 2022.
Xinghui Li, Yikang Ding, Jia Guo, Xiansong Lai, Shihao Ren, Wensen Feng, and Long Zeng. Edge- aware Neural Implicit Surface Reconstruction. 2023 JEEE International Conference on Multi- media and Expo (ICME), pp. 1643-1648, 2023a. URL
org/Corpus ID: 261127864
Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H Taylor, Mathias Unberath, Ming- Yu Liu, and Chen-Hsuan Lin. Neuralangelo: High-fidelity neural surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8456-8465, 2023b.
Zhihao Liang, Zhangjin Huang, Changxing Ding, and Kui Jia. HelixSurf: A robust and efficient neural implicit surface learning of indoor scenes with iterative intertwined regularization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13165- 13174, 2023.
Xiangyu Liu. Weicai Ye. Chaoran Tian, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang, Cox- graph: multi-robot collaborative, globally consistent, online dense reconstruction system. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8722- 8728. IEEE, 2021.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17X10:1711.05100 2017.
Ricardo Martin-Brualla Noha Radwan, Mehdi SM Sajjadi. Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth. NeRF in the wild: Neural radiance fields for unconstrained photo collec- tions. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pp. 7210-7219, 2021.
Luca Medeiros et al. Lang Segment Anything. https: htw.:/uuw...mmmmuucmmmmeeee.oood -areennnttattttniinggggg 2024. Accessed: 2024-11-18.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1)999-106, 2021.
Yuhang Ming, Weicai Ye, and Andrew Calway. Idf-Slam: End-to-end rgh-d slam with neural im- plicit mapping and deep feature tracking. arXiv preprint arXiv:2209.07919 2022.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding. ACM transactions on graphics (TOG), (4): 2022.
Michael Oechsle. Songyou Peng, and Andreas Geiger. UniSurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 5589-5599, 2021.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. Proceed- ings of the IEEE/CVF conference computer vision and pattern recognition, pp. 165-174, 2019.
Minyoung Park, Mirac Do. YeonJac Shin, Jaescok Yoo, Jongkwang Hong, Joongrock Kim, and Chul Lee. H2O-SDF: Two-phase Learning for 3D Indoor Reconstruction using Object Surface Fields. arXiv preprint arXiv:2402.08//3 08/308 2024.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Radu Alexandru Rosu and Sven Behnke. PermutoSDF: Fast multi-view reconstruction with implicit surfaces using permutohedral lattices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8466-8475, 2023.
12

---

Published as a conference paper at ICLR 2025
Johannes L Schonberger and Jan-Michael Frahm. Structure- frructue--rom--iee momtmootion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104 2016.
Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. Pixelwise view selection for unstructured multi-view stereo. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings. Part III 14, pp- 501-518. Springer, 2016.
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren. Shobhit Verma, et al. The Replica dataset: A digital replica of indoor spaces. arXiv preprint arXiv:/906.088 1906.05797 2019.
Zhuo Su, Wenzhe Liu, Zitong Yu, Dewen Hu, Qing Liao, Qi Tian, Matti Pictikäinen, and Li Liu. Pixel difference networks for efficient edge detection. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pp- 5117-5127,
Cheng Sun, Min Sun, and Hwann-Taze Tzong Chen. Direct voxel grid optimization: Super-fast con- vergence for radiance fields reconstruction. In Proceedings of the IEEE/CV Conference on Computer Vision and Pattern Recognition, pp.
2022.
Jiepeng Wang, Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. NeuRis: Neural reconstruction of indoor scenes using normal priors. In European Conference on Computer Vision, pp. 139-155. Springer, 2022.
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2 100. 10689, 2021.
Yifan Wang, Di Huang, Weicai Ye, Guofeng Zhang, Wanli Ouyang, and Tong He. NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction. arXiv preprint arXiv:2408. 10178, 2024.
Yiming Wang. Qin Han, Marc Habermann, Kostas Daniilidis, Christian Theobalt, and Lingjie Liu. NeuS2: Fast learning of neural implicit surfaces for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2023.
Yuting Xiao, Jingwei Xu, Zehao Yu, and Shenghua Gao. DebSDF: Delving into the Details and Bias of Neural Indoor Scene Reconstruction. IEEE Transactions Pattern Analysis and Machine Intelligence (TPAMI), 2024.
Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lip- man. Multiview neural surface reconstruction by disentangling geometry and appearance. Ad- vances in Neural Information Processing Systems, 33:2492-2502,, 2020.
Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805-488 2021.
Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P Srinivasan, Richard Szeliski, Jonathan T Barron, and Ben Mildenhall. BakedSDF: Meshing neural sdfs for real-time view synthesis. In ACM SIGGRAPH 2023 Conference Proceedingg, pp. 1-9, 2023.
Weicai Ye, Xingyuan Yu, Xinyue Lan, Yuhang Ming, Jinyu Li, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. Deflowslam: Self-supervised scene motion decomposition for dynamic dense slam. arXiv preprini 2rXiv:2207.0877 2022.
Weicai Ye, Shuo Chen, Chong Bao, Hujun Bao, Mare Pollefeys, Zhaopeng Cui, and Guofeng Zhang. IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis. In Proceedings of the IEEF/CVF International Conference on Computer Vision 2023a.
Weicai Ye, Xinyue Lan, Shuo Chen, Yuhang Ming, Xingyuan Yu, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. PVO: Panoptic Visual Odometry. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), pp. 9579-9589, June 2023b.
13

---

Published as a conference paper at ICLR 2025
Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. ScanNet++: A high- fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 12 12-22, 2023.
Zehao Yu, Anpei Chen, Bozidar Antic, Songyou Peng Peng, Apratim Bhattacharyya, Michael Niemeyer, Siyu Tang, Torsten Sattler, and Andreas Geiger. SDFStudio: A unified framework for surface reconstruction, 2022a.
Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattler, and Andreas Geiger. MonoSDF: Ex- ploring monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems, 01225018855322 2022b.
Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. NeRF++: Analyzing and improving neural radiance fields. arXiv preprint arXi::2010.07492, 2020.
2020.
Yongqiang Zhang, Zhipeng Hu, Haoqian Wu, Minda Zhao, Lincheng Li, Zhengxia Zou, and Changjic Fan. Towards unbiased volume rendering of neural implicit surfaces with geometry priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 4359 4368, 2023.
14

---

Published as a conference paper at ICLR 2025
APPENDIX / SUPPLEMENTARY MATERIAL
Color Network fe
Geometry Network g
SH(V)
PE(N)
256
256
RGB
Inpul
interpolatc
+
(E(N)
71
256
256
256
256
32
Deflection Network
256
2.56
256
quaternion
+ Dirn-wise concalenation
Inputs
Weights of MLP
Ray direction
Analytical gmddient SDF
Geortry feature Dellection quatemion
(E
Figure 6: Architecture of ND-SDE. We utlize multi-resolution hash grids Y/. for scene represen- tation. The geometry network fg. color network fe. and deflection network fa all constructed using simple multilayer perceptrons (MLPs).
A IMPLEMENTATION DETAILS
A. ARCHITECTURE
When considering the inputs to the deflection network, two scenarios are contemplated: (1) No deflection is required when the prior is completely accurate. In this case, the quaternion axis aligns with the scene normals, indicating that the deflection axis is closely related to the scene geometry; (2) Since the prior model is view-dependent, the quaternion for alignment with the prior must also be new-deeendentt Consequently, we define:
fa(x, V. z,).
(17)
where = (4o: 4: 92 93) represents the deflection quaternion. As detailed in Section 3.2 the quaternion can expressed in trigonometric form as q = c Within the deflection network, the rotation axis is initialized along the x-axis, i.e., (1, 0), and the rotation angle #/2 is set to π 4/2. By default, the network normalizes the output quaternions.
(u u²j u k))
We observe that the structure of the deflection network mirrors that of the color network Yariv al. (2020), denoted as e = fx v, z). However, considering that the color network exclusively models scene-related radiance, while the deflection network encapsulates a broader range of prior deviations, we separate these two networks.
A.2 PROGRESSIVE WARM-UP
In the early stages of training, deflection fields may introduce negative effects, such as noise. Before the rough formation of scene geometry, these deflection quaternions random and erroneous. If unconstrained, deflecting the true normals during this initial phase can lead to reconstruction errors, such surface protrusions.
To address this issue, we propose a gradual activation strategy for deflection fields, allowing them to become effective the surface takes shape. Specifically, design a "quat anneal" policy to
15

---

Published as a conference paper at ICLR 2025
Method
w/o warm-up w warm-up
F-score
0.667
0.686
Table 7: Ablation of Warm-Up Strategy on ScanNet++. The warm-up strategy better facilitate the initialization of the deflection field, resulting in improved surface reconstruction quality.
Deflection angle(A0)
$15
>25*
GT RGB
Figure 7: Visualization of the Deflection Angle. We visualize the angle map and the angle mask using thresholds of 5% 15,, and 25"
enable the deflection effect to progress gradually. Given the surface normals synthesized by volume rendering, denoted N(r), and the synthesized quaternion Q(r) (Qo, Q1 Q2. Q3), we first decompose Q(r) into its trigonometric representation:
arccos Qo
2
(18)
Q
Q2 Q₂1
sin
Here, u denotes the rotation axis corresponding to the quaternion Q(r). and represents the rotation angle. We then utilize the scene normals warm up the deflection axis, starting with zero rotation:
Oster
0
prog, 2
(19)
Uste = prog, x + (1 - prog.) x Ñ(r)
Here, the parameter prog, linearly increases from 0 to I during the training process within the process interval [0,
Once the training progress reaches rameal-quat.ee the warm-up phase concludes, and prog, becomes equal to 1.
The warm-up deflection quaternion is computed as:
0 fiter
Aster
Q (r) = cos
sin x Uster-
(20)
2
During the initial training stage, the deflection angles start at zero and gradually increase, allowing the model to effectively leverage prior knowledge to optimize smooth regions and approximate the scene geometry. We consider it to be adequately initialized upon the warm-up phase concludes. Throughout this process, the deflection field gradually adapts to more complex structures and learns the deviations between the true geometry and the normal priors. Quantitative results with and with- out progressive warm-up are presented Table 7
A.3 DETAILED IMPLEMENTATION ASPECTS
A.3.1
ADAPTIVE DEFLECTION ANGLE PRIOR LOSS
In Section 3.3
we introduce two modulation functions. namely g" and g (Eq. 12). which adjust the weights of the deflected and original normal loss terms based on the deflection angle.
Here provide the specific forms for both functions. We employ a shifted logistic function to define them:
1
g (0) = 1 e 0o)
(21)
9(0) = 1 - (0)
16

---

Published as a conference paper at ICLR 2025
Modulation functions
1.05
1,00
0.95
0.90
(29".0.9
0.85
0.80
0.75
0.65
0.60
(d
0.50
15",0.5)
g(40) 940)
0.45
0.35
0.30
0,20
0.15
0.10
(5
0.05
24 Deflection angle!")
Figure 8: Modulation Functions defined in Eq. 21
Here, the parameter controls the steepness of g", and Og denotes the offset term. In our experi- ments, we set 8y = 12.5, 0u - T2 for scenes.
As shown in Figure 7 deviation angle of less than 5" consistently indicates a simple flat region, whereas an angle greater than 15a typically denotes a complex structural region.
Additionally, we visualize the modulation function (~) in Figure g with (so, Oo) set to (12.5, )²) For regions with deviation angle less than 5% weights greater than 0.9 are assigned to the original normal loss term, ensuring the effective incorporation of normal priors in smooth regions.
To further demonstrate the effectiveness of the proposed adaptive deflection angle prior loss, visualize three normal loss heatmaps: the naive normal loss, the deflected normal loss, and the adap- tive deflection angle normal loss (as shown in Figure The black box highlights thin structures and ambiguous regions where monocular priors are largely unreliable. In these areas, the naive normal loss heatmap shows significant errors, while both the deflected and adaptive losses exhibit signifi- cantly lower errors by aligning the deflected scene normals with the normal priors (GT). In summary, the adaptive prior loss can dynamically adjust the weighting between the naive and deflected loss terms, promoting globally correct convergence while fully leveraging the normal prior.
A.3.2 DEFLECTION ANGLE GUIDED OPTIMIZATION
In Section 3.4 we introduce three deflection angle guided optimization methods, including ray sam- pling, photometric optimization, and unbiased rendering.
To implement ray sampling guided by the deflection angle, we dynamically maintain a deflection angle map (AO) for each image during the training process. We update the angle map using the following equation:
(r(r) max (Arrl(() 1,, A0(r)),
(22)
where is the sampled ray and v is the decay coefficient. We initialize the angle map to the beginning.
In the sampling phase, we first calculate the per-pixel sampling probability based on the angle map, then use this probability map for inverse sampling, such that areas with greater deflection magnitude tend to have rays sampled.
We employ a scaled and shifted logistic function to calculate the per-pixel sampling probability:
4
((r:) = 1
(23)
(A0r)) - 00)
17

---

Published as a conference paper at ICLR 2025
GT Norma
Normal
Deflected Normal
Normal loss
Deflected Normal loss
Adaptive loss
narmal
c normal
pad normal
Figure 9: Heatmap comparison of naive, deflected, and adaptive deflection angle normal loss.
where A represents the maintained angle map, and p(r,) E [1], with the steepness controlled
by 81. We then normalize the probability map for inverse sampling using the following equation:
p(r.)
p(r.)
(24)
P(r()))
For the angle guided color loss, similarly employ scaled and shifted logistic function to weight the original color loss term based on the deflection angle (Eq, 14):
Wootor (AO(r)) = 11 (+00))(A(()) 02)"
(25)
Regarding the unbiased rendering guided by the deflection angle, we first define a bias confidence using another variant of the logistic function to compute it:
efd(r) 1 (A0()) 03)
(26)
Note that utilize the maintained angle map to caleulate the confidence instead of the deflection angle Ar(r) computed from current iteration, as unbiased densities of the sampled points along the ray required prior to the volume compositing process. This then leads the deflection angle guided unbiasing:
1 (r((t)) = U
(f(((()))
(77)
cfd(r) (g(((t))) + +1-cfd(r)
In practice, we set the parameters as follows: (81 = 25,01 13.4 = 4), (82 25, 02 2), (83 = 25, 0g )
1/2 l2
A.3.3 NUMERICAL GRADIENT
Following Neuralangelo al. (023b), we employ numerical gradients w.r.t multi-resolution hash grids to promote surface consistency, proved effective in their work. The numerical gradient is computed as follows (considering only the SDF value output of geometry network fy):
Vafg (xx) fg (% (x, + cx)) fg (YL (x; - -c))
(28)
2
18

---

Published as a conference paper at ICLR 2025
w/o partial
partial
Figure 10: Visual comparisons of partial unbiased rendering ScanNet.
Here, 6a = [e, 0, 0] signifies the incremental step size along the x-axis. The method for calculating the numerical gradient along the y or z-axis remains the same.
We also utilize the curvature loss Li et al. (2023b) to further encourage the smoothness of the recon- structed surfaces. This loss term is defined as:
Leurv
Σ f (x:)
(29)
This term decreases exponentially during training, similar to Neuralangelo. fact, our tests indicate that it has minimal impact surface smoothness, resulting in only slight improvement compared to its absence. We attribute this to the use of monocular cues, as the cues are particularly accurate in smooth regions.
B EXTENSIVE ABLATION STUDIES
B. ABLATION OF PARTIAL UNBIASED RENDERING
we introduced an unbiased function. We partially apply this technique to thin struc- tures indicated by the learned deflection angles, without doing may result in inability verge and decrease in reconstruction quality.
In Section 3.5
We attribute this issue to the inconsistency in perspective inherent in the transform function. We compute the derivative of fy (considering only its SDF value output) w.r.t the sampling distance t; follows:
(r((t) f + - V
(30)
where v is the ray direction and n, represents the geometry normal at point x; means that (r((). equivalent to the cosine of the ray direction and the normal.
at,
Consider another ray in space, denoted as r'(t) o' + tv.. Assuming that this ray also passes through X,, sampling the same point such that x, = o' + t,y' = x,. According to Eq. [5] the volumetric densities of x, and x, under the transform function are given by:
-
w/ (fo(r(t))
1
V (o(((()))
a(r(t)))
3 ([g(((t)
,o(r((t)))
3 (PACT)))
(31)
We observe that (r((t)) = fg (1' (t²)) because X; = Xy, but (r(,)) * (r(((tt))) because n; v # n, V' even though n; = The inconsistency in perspective leads to ambiguity in the volume density at the same sampling point, causing complex indoor surfaces to fail to converge.
Therefore, to reduce this ambiguity as much as possible, we choose to apply the transform function only to fine regions indicated by the deflection angle.
19

---

Published as a conference paper at ICLR 2025
Method
w/o partial w partial
F-score
0.707
0.746
Table 8: Ablation of Partial Unbiased Rendering ScanNet. We observed decrease in recon-
struction quality after omitting partial unbiased rendering.
Method w/o d'eephh Caepth F-score
0.664
0.686
Table 9: Ablation of adaptive depth prior loss on ScanNet++. We observed decrease in recon- struction accuracy after omitting the adaptive depth loss.
The quantitative ablation results on ScanNet for partial unbiased rendering are presented in Table where the improved F-score highlights the effectiveness this strategy.
Figure provides qualitative comparisons. As indicated by the blue box, the surfaces reconstructed without the proposed partial unbiased rendering exhibit wrinkles and irregularities, and the fine structural details of the chair legs (highlighted in red) are not well captured. With the application of this strategy, the surfaces become significantly smoother and more precise. These visual results underscore the advantages of the partial unbiased rendering approach.
We further visualize the inverse variance (1/3) of the SDF-density conversion function (Eq. 3 during training in Figure aa)a) and find that, without partial unbiasing. 1/B fails converge
relatively high value. In fact, the larger the inverse variance, the sharper and more accurate the SDF- density field becomes. Conversely, smaller value indicates greater uncertainty in the geometric surface, which leads to lower reconstruction accuracy and failure to converge properly. As shown in Figure b)), an increase in 1/3 results in the rendering weight becoming more concentrated near the surface. Without a partial strategy, the surface is learned inaccurately and appears blurry.
More discussions about unbiasing methods are presented in Appendix
1/ curve
weight curve (8t=0.002)
1200
(b)
partal tunfformty)
0.5
10=100
1,9-200
1000
1,p=500
1/0-1000
800
0.3
600
wogght
400
0.1
1,15 - 1:0
0.100
0.075
0.000
0.02
0.000
-0.025
-0,000
-0.075
-0 10.100
50f
Figure 11: (a) The inverse variance curve of the SDF-density conversion function during training. (b) The rendering weight curve with respect to the inverse variance.
B.2 ABLATION OF ADAPTIVE MONOCULAR DEPTH LOSS
Notably, we apply the same method to adjust depth supervision (Eq. 13 based on the deflection angle. This simplified treatment rests on strong foundation for two reasons. First, both depth and normal cues share common inductive bias, they stem from the same pretrained model Eftekhar ctt al. (2021). Second, due to the inherent lack of scale in monocular depth, it cannot provide pre- cise supervision. In implicit surface reconstruction with priors, depth cues primarily utilized to facilitate convergence and accelerate surface formation. Therefore, automatically masking high- frequency areas based on the deflection angle has adverse impact the recovery of details.
We present the quantitative ablation results in Table 9 The improved performance underscores the importance of utilizing the adaptive deflection angle depth prior loss.
20

---

Published as a conference paper at ICLR 2025
Nawd Por
NCT
Figure 12:
Detailed visualization comparisons of structural region confidence maps obtained using various methods. A comprehensive analysis provided in Appendix [B.3
B.3 CONFIENCE MAP ANALYSIS
As illustrated in Figure 12 we present the confidence maps that highlight structural regions, which obtained using a variety of methods, including the NCC map Wang al. (2022), edge map Li et al. (2023a), uncertainty map Xiao et al. (2024) and our angle map. The NCC map exhibits low performance due its simplistic local cross-correlation computing method, and it fails properly distinguish feeuurryy flat regions, such as textured floors. The edge map, generated by pre- trained network Su et al. (202I), effectively captures object edges but still lacks precise localization of the fine structures. The uncertainty map, which is learned by constructing an uncertainty field, performs better than the first two methods. However, it only provides an approximate indication of high-frequency regions, as it inherently serves as an intensity field to measure the confidence of monocular cues, independent actual geometry. As a result, the uncertainty map fails to precisely evaluate the actual differences between scene geometry and priors, leading to inefficient utilization of various priors. Additionally, it fails to recognize highlighted areas, as shown in the second row of Figure 12 and the first of Figure [6] In contrast, our method, by modeling geometry-relevan SO(3) residual fields, accurately locates specific structural regions and provides precise angular deviations. The angle map clearly reflects the underlying geometry difference. Compared to the methods discussed above, our approach yields more reasonable and accurate results.
In terms of the sampling strategy guided by the confidence maps illustrated in Figure 12 our sam- pling process is fully weighted by geometry-aware angular deviations and is precisely localized to structural regions. In contrast, other methods, such as edge-aware and uncertainty-drive sampling, exhibit lower localization precision and are less sensitive to geometric differences from the priors, failing to provide effective sampling weights for various priors.
This analysis is further validated through extensive experiments. As shown in Figure 16 and Fig- ure 17 our method generates more accurate fine structures compared DebSDF.
B.4 ANALYSIS OF THE ROBUSTNESS OF MONOCULAR CUES
To deeply investigate the effectiveness of our proposed explicit deflection-based method, we con- ducted experiments under the assumption that the priors for smooth regions are inaccurate. Specifi- cally. we manually rotated all monocular normals in the smooth regions by angles ranging from 0º to 60%. A SAM performed by rotating the normal priors by specified degree (e.g., 60%) toward pre-defined axis. We evaluated MonoSDF, DebSDF, and ND-SDF on the 0e75f3c4d scene from ScanNet++. The results are summarized Table 10 and Figure I3 As demonstrated the results, decreases in F- score are observed across all methods. Specifically, MonoSDF produces an entirely abnormal result, as it rigidly relies on the biased priors. DebSDF also fails to maintain robustness in this scenario. Although the uncertainty field in DebSDF can easily mask irregular priors by learning uncertainty variances, leads to inefficient utilization of the consistency information inherent in monocular cues within smooth regions. In contrast, our method explicitly models the actual bias amplitude,
et al. (2024) was employed to detect the flat areas. The rotation was
21

---

Published as a conference paper at ICLR 2025
Rised
Figure 13: Visualization results with monocular normal cues manually biased by 60 degrees. Our method accurately captures the view-dependent biased priors and yields promising reconstruc- tion result.
Methods
0
50
15e
30
45
60%
0-0+60-
Ours
0.750
0.735
0.678
0.654
0.709
0.655
-0.095
MonoSDF
0.633
0.140
-0.493
DebSDF
0.701
0.517
-0.184
Table 10: Quantitative results under manually biased monocular normal cues. The observed
minimal decline in F-score underscores the effectiveness and robustness of our approach.
achieving notably robust performance with the smallest decrease in F-score (Table 10 even when the normal priors are biased by much as 60%. These findings further validate the generalization and robustness of our method in non-trivial assumptions of the monocular cues.
C MORE RESULTS
C.I REPLICA
We present additional visualization results of our method on Replica in Figure
Figure 14: Qualitative results of our method on Replica dataset.
C.2 SCANNET++
We present detailed quantitative results of the selected six scenes from ScanNet++ in Table 11 and provide more visual comparison results with previously state-of-the-art methods in Figure 15 Figure 16 and Figure 17 Compared the baselines, our method yields significantly improved reconstructions both visually and quantitatively. notably excelling at capturing thin, fine- fine-grained structures while maintaining smoothness in flat regions.
C.3 TANKS AND TEMPLES
we extracted the reconstructed meshes resolution of 2048. At this high resolution, the meshes demonstrate that our method significantly enhances the generation of small and thin structures, even in challenging large-scale indoor scenes. For all experiments, the hash grid settings were kept at their defaults, with resolutions ranging from 2 to 211 across 16 levels, and hashmap size of 219
As illustrated in Figure 19 and Figure 20
22

---

Published as a conference paper at ICLR 2025
We present more qualitative comparisons with previous otate-of-the-art baselines, including MonoSDF Yu et al. (2022b) and DebSDF Xiao et al. (2024) at 2048 resolution, as depicted in
Figure 18
It is evident that ND-SDF achieves accurate dense surfaces replete with extensive details.
336h
0c7513o4d9
Method
Aec|
Compl
Pre
Recall
Chumier,
F-score
Acci
Comp.
Pre
Recallf
Chamler]
T-score
VoISDF
0.053 0.098
0.475
0.372
0.0755
0.417
0.053
0.078
0.395
0.363
0.065
0.378
Baked-Angelo
0.168 0.023
0.52
0.78
0.0955
0.624
0.121
0.053
0.525
0.719
0.087
0.607
MonoSDI (Grd)
0.037 0.037
0.606
0.656
0.037
0.630
0.064
0.027
0.574
0.705
0.046
0.633
DetSDF (Grid)
0.033
0.032
0.679
0.744
0.032
0.710
0.067
0.023
0.62
0.794
0.045
0.701
ND-SDF
0.037 0.031
0.648
0.728
0.034
0.686
0.054
0.018
0.662
0.864
0.036
0.750
108ec0b806
714d173c9c
Method
Ace|.
Compl
Pref
Recallf
Chamfer,
F-scoret
Acei.
Compl
Pret
Recallf
Chamfer)
Fuscoret
VoISDF
0.049 0.097
0.475
0.372
0.073
0.417
0.097
0.075
0.457
0.436
0.086
0.446
Baked-Angelo
0.092 0.050
0.498
0.624
0.071
0.554
0.420
0.024
0.457
0.799
0.222
0.582
MonoSDF (Grid)
0.041
0.045
0.575
0.576
0.043
0.576
0.138
0.022
0.708
0.813
0.080
0.757
DebSDF (Grid)
0.036
0.045
0.61
0.595
0.041
0.602
0.154
0.021
0.733
0.844
0.087
0.785
ND-SDF
0.047 0.040
0.591
0.646
0.044
0.617
0.112
0.015
0.721
0.888
0.064
0.796
ab11145646
e050c1558d
Method
Acc|
Compl
Pre
Recallf
Chamier,
[-scure|
Acci
Compl
Pre
Recallf
Chamler]
I-score
VolSDF
0.089 0.164
0.321
0.226
0.126
0.265
0.077
0.101
0.307
0.262
0.089
0,283
Baked-Angelo
0.076 0.038
0.575
0.701
0.057
0.632
0.034
0.043
0.685
0.687
0.038
0.686
MonoSDF (Grid)
0.036 0.035
0.695
0,700
0.036
0.697
0.027
0.024
0.747
0.770
0.026
0.758
DetSDF (Grid)
0.038 0.036
0.685
0.686
0.036
0.685
0.03
0.027
0.684
0.697
0.028
0.69
ND-SDF
0.054 0.024 0.657
0.786
0.039
0.716
0.036
0.020
0.723
0.798
0.028
0.759
Table 11: Quantitative results of the 6 selected scenes from ScanNet++, ND-SDF significantly
outperforms previous of-te-of-thh-ar methods in most metrics, particularly in terms of Recall.
C.4 ADDITIONAL DISCUSSIONS
More information about dataset. The qualitative results ScanNet presented in the main con- tent (as shown in Figure 4 demonstrate that our method captures fine details, such as the chair legs, which are even overlooked by the ground truth (GT). This highlights known issue regarding the reliability of ScanNet GT meshes. Constructed in 2017, the ScanNet dataset is based on RGB-D data captured by an iPad with an additional structure sensor. The images suffer from relatively low resolution, motion blur, and lighting variations. Furthermore, the limitations of the bundle-fusion algorithm result in GT surfaces lacking fine structures, which can adversely affect evaluation perfor- mance in these areas. This underscores our preference for conducting quantitative ablation studies on ScanNet++, which provides more reliable evaluation results.
Concurrent work. We notice that the concurrent work NC-SDF Chen et al. (004b) share anal- ogous idea with us. We clarify that our method is entirely independent of NC- Although we similarly model an SO(3) residual field to enhance surface reconstruction, our design diverges sub- stantially. NC-SDF employs two-stage approach, yet struggles with noise in the second stage's 's plicit deflection. In contrast, our adaptive prior loss automatically assigns weights based on sample characteristics (Eq.. 12). removing the need for staged training. Additionally, NC-SDF requires ex- tra boundary information for optimization, reducing robustness and adding hyperparameters. While NC-SDF was tested datasets with simple layouts like ScanNet and ICL-NUIM, we evaluated our method on more complex indoor scenes, including ScanNet++ and T&T. Extensive experiments confirm our method's superior generalization ability challenging environments, an advantage not observed in NC-SDF.
About unbiasing methods. It's crucial to resolve bias issues in SDF-based surface reconstruc- tions. We show the necessity to employ unbiasing transformation through an actual sampling process in Figure supported by ablation results in Figure Inherently, bias refers to the failure of the ren- dering weight to attain its maximum value at the zero-level set of SDF, thereby leading to incorrect rendering depth. This issue not only damages the surface quality but also severely affects the gen- eration of fine structures. Many works have been proposed to solve this: The unbias assumption of Neus Wang et al. (2021) ensures the local maximum of the weight for a local plane. But the weight is severely reduced when the viewing direction gets closer to the tangent plane. TUVR Zhang et al.) 2023) transforms the input SDF value the actual depth from the tangent plane and extends the
23

---

Published as a conference paper at ICLR 2025
extremal property of weight to any surface distribution. But TUVR still generates abnormal bi- modal shaped weight when passing close to an object if 1/3 is not large enough. To address this, DebSDF[Xiao et al. (2024) proposed using curvature radius to correct the SDF value, tremendously reducing the bias. However, these unbiasing methods, such as TUVR and DebSDF, face serve con- vergence issues, the transformation must be perspective-dependent causing inconsistency. We analyze this issue using TUVR as example in Appendix B.I To mitigate this, we utilize explicit learned angular deviations help apply the unbiasing transformation only to low-proportion fine- grained structures in the scene, which has proved effective in our method. Simpler solutions also exist, such as applying unbiasing only after the surface has sufficiently converged. However, an- other issue arises: the implicit SDF field is difficult to reshape, making it challenging to capture fine structures in later stages. There are still many issues be addressed.
D SOCIETAL IMPACT
Our method contributes to high-fidelity indoor surface reconstruction. On the positive side, pre- cise indoor surface reconstruction can revolutionize fields such as architecture, virtual reality, and robotics. enables architects to visualize and optimize designs, enhances immersive experiences in virtual environments, and aids in navigation for autonomous robots. However, there are also chal- lenges. Misuse of this technology could invade privacy, as detailed indoor reconstructions might reveal sensitive information. Striking a balance between innovation and responsible deployment will be crucial for maximizing the positive impact while minimizing potential harm.
24

---

Published as a conference paper at ICLR 2025
VoISDE
Baked-Angelo
MonoSDDF
Figure 15: Qualitative Comparison on ScanNet++. We compare ND-SDF with several base- line methods, including VolSDE Baked-Angelo, and MonoSDF. Notably, only MonoSDF and Ours leverage monocular priors. Our method effectively balances smoothness and detail, whereas MonoSDF loses significant fine structures, and Baked-Angelo struggles handle textureless smooth regions, such as floors.
25

---

Published as a conference paper at ICLR 2025
MonoSDF (Grid)
DebSDF (Grid)
Ours
GT
Additional Visualization Comparisons on ScanNet++. We further compare our method with state-of-the-art method DebSDF. As shown above, ND-SDF tremendously outperforms previous methods when facing challenging complex indoor scenes. shows superior ability to cap- ture accurate and gewmetry- genmstry-consistent detailed structures.
Figure 16:
26

---

Published as a conference paper at ICLR 2025
MonoSDF (Grid)
DebSDF (Grid)
Ours
Figure Additional Visualization Comparisons on ScanNet++. We further compare our method with fate-of-the-att method DebSDF. As shown above, ND-SDF tremendously outperforms previous methods when facing challenging complex indoor scenes. shows superior ability to cap- ture accurate and geometry-consistent detailed structures.
27

---

Published as a conference paper at ICLR 2025
MonoSDF (Grid)
DebSDF (Grid)
Ours
GT RGB
Figure 18: Qualitative Comparison on Tanks & Temples. We compare ND-SDF with previous
state-of-the-art indoor implicit reconstruction methods.
28

---

Published as a conference paper at ICLR 2025
Ours
GT RGB
Figure 19: Qualitative Comparison of Different extraction resolutions on Tanks & Temples. We additionally extract the recovered mesh from the implicit SDF a resolution of 2048.
29

---

Published as a conference paper at ICLR 2025
(nommal)
Qurs. (mesh)
Figure 20: Qualitative Results of the extracted mesh at a resolution of 2048 on Tanks & Tem- ples.
30


Reviews:
[{"review_id": "Jm2XfDE70J", "reviewer": "Reviewer_EWRb", "summary": "This paper presents a pipeline for multi-view 3D reconstruction of indoor scenes. The pipeline is based on recently popular differentiable volume rendering methods using Signed Distance Functions (SDF), such as VolSDF and NeuS. To enhance reconstruction quality in indoor scenes, particularly in textureless regions, the paper introduces monocular priors for additional supervision, inspired by recent works like MonoSDF and NeuRIS.\n\nA core contribution of this paper is the introduction of a normal deflection field, which helps the model identify and correct inaccuracies in reference normal images estimated from monocular models, such as the Omnidata model. Experimental results show that this deflection field aids in reconstructing fine geometric details, such as thin structures.\n\nAdditionally, the authors introduce techniques to further improve geometry reconstruction quality, including (1) an adaptive normal loss weight and (2) unbiased volume rendering on thin objects.\n\nThe experimental results are promising and include comprehensive ablation studies. Overall, the paper is well-argued and substantiated.", "questions": "- In Line 280, is this a typo on $g(\\theta)$? Looks like this is a missing $\\Delta$.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed normal deflection fields, which correct inaccurately estimated normals from normal maps, seem reasonable to me. \n- Table 6 and Figure 5 effectively showcase the contribution of the method's design through comprehensive ablation studies.\n- The experimental results demonstrate that this proposed pipeline achieves state-of-the-art reconstruction quality in indoor scenarios.", "weaknesses": "- Using unbiased volume rendering for thin structures seems reasonable. However, why not apply this unbiased volume rendering weight uniformly across all regions? The authors mention potential convergence issues with this approach, but it would be helpful if they provided a more in-depth analysis explaining the nature of these convergence issues and offered insights into why they occur.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QmNwNBqgA7", "reviewer": "Reviewer_KRgu", "summary": "The paper proposed a method to improve prior normal-guided neural SDF indoor scene reconstruction. Instead of supervising the normal uniformly across the whole domain, the author proposes to use a network to learn the deflect normal and desired adaptive angle prior to loss according to the deviation of the deflect normal to the prior normal. The intuition is that large deviation happens in complex structure areas where the normal prior in general is bad quality.  The author also proposes to use the re-weighted loss on normal, depth, and color rendering.  The experiments show the proposed method performs well qualitatively and quantitatively.", "questions": "1. would it make sense to consider depth maps together to decide if the normal prior is accurate or not? For example, in most complex geometry areas such as thin structure areas, the depth is not continuous, and one may not be able to hope the normal is accurate there. \n2. what is the reason for the artifacts (which I mentioned in weakness 4), is that because without supervision the model struggles to recover the details?\n3. to deal with 2, maybe add some smooth loss / geometric preserving loss on these areas that the normal prior is categorized as bad? \n4. I don't see a reason why this method only works indoor, has the author also tried it on outdoor datasets?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow. \n2. The normal deflection detection design is under reasonable assumptions and it offers a solution to deal with non-accurate normal prior.  \n3. The angle deviation loss design fits the situation. \n4. The author provides complete experiments to show their method's performance. Including comparison with previous work quantitatively and qualitatively and ablation studies to show each module improves the results.", "weaknesses": "1. The bad quality normal prior problem seems only solved half. The author reduced the loss weight on the part that the normal prior is not accurate, but not the other half, what to do to improve the method itself performance in these areas.\n2. Like the other prior-based methods, the quality of the prior matters. The author assumes smooth area the prior is relatively accurate. \n3. Like the other SDF-based methods, the reconstruction seems to still struggle with the thin structures.  \n4. I noticed that in most cases the method outperforms previous methods, but sometimes it creates noise and artifacts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ghqRq9gxMO", "reviewer": "Reviewer_eHed", "summary": "This paper proposes ND-SDF, which learns a normal deflection field to reduce the significant bias in monocular normal and depth priors. The normal deflection field can be rendered through volume rendering to further guide adaptive prior loss, ray sampling and unbiased rendering. The numerical and visualization results are compelling.", "questions": "See weakness.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The inaccuracy and bias are widely existed in monocular depth and normal priors, therefore, the idea of exploring the uncertainty of monocular clues to improve the reconstruction performance is reasonable and valuable.\n 2. The normal deflection field is effectively incorporated throughout the training process, enabling rendered deflected normal images to be supervised by estimated normals and adaptively applied in ray sampling, photometric loss, and unbiased rendering.\n3. The numerical and visualization results are attractive and impressive, which highly demonstrate the effectiveness of the proposed method.\n4. The ablation studies are thorough, clearly showcasing the effectiveness of each proposed module.", "weaknesses": "1. Lack of citation. A similar approach of learning rotated normals to mitigate the bias between normal priors and ground truth normals was first proposed in NC-SDF[1]. While I know that NC-SDF has not released the source code so it’s difficult to directly compare with it, an appropriate clarification is necessary to distinguish the proposed idea from NC-SDF.\n2. Lack of quantitative comparisons with DebSDF. DebSDF also predicts the uncertainty of monocular priors and is able to reconstruct details such as chair legs. The numerical result of DebSDF is included in the table but visual comparison with DebSDF is absent, which is insufficient.\n\nThe methodology and experiments in this paper are solid. I like this paper, however, the two major concerns above prevent me from giving a higher score. I am willing to raise my score if the concerns are well addressed.\n\n[1]. Chen Z, Wu X, Zhang Y. NC-SDF: Enhancing Indoor Scene Reconstruction Using Neural SDFs with View-Dependent Normal Compensation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 5155-5165.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kbJPqPsq7P", "reviewer": "Reviewer_M33k", "summary": "This paper introduces ND-SDF and proposes to learn a Normal Deflection field to represent the angular deviation between the scene normal and the normal prior.\n\nThe paper also additionally introduces a ray sampling strategy based on the deflection angle to improve the surface quality further. The paper also proposes to use an unbiased function inspired by TUVR.", "questions": "1. For Figure 3, what about the comparison with unbiased density function in NeuS, TUVR, DebSDF, HF-NeuS[3], and NeuRodin(optional, because NeuRodin is the concurrent work)[4]? These works also propose an unbiased density strategy. (visual comparison is better for good understanding)\n\n2. In Figure 7, The deflection angle map appears to have a higher value at complex structural regions. What's the visual comparison between edge mask(EaNeuS), NCC mask(Equation 6 in NeuRIS), and uncertainty-aware mask(DDPNeRF[5], DebSDF, H2OSDF)?\n\n3. For Equation 12 of this paper, it seems that when $\\hat{\\mathbf{N}}^d(\\mathbf{r})$ and $\\hat{\\mathbf{N}}(\\mathbf{r})$ perfectly fits the prior $\\mathbf{N}(\\mathbf{r})$, then this loss will become a perfect zero. The reviewer is concerned that this situation that causes zero loss doesn't seem to be in a good state. Is there a more comprehensive explanation about this training objective and why this training objective will work to help solve the concerns above?\n\n4. Does the proposed normal deflection field have some advantages over the normal uncertainty field, like in DebSDF and H2OSDF?\n\nReference:\n\n[3] HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details. Wang et al. (Neurips 2022)\n\n[4] NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction. Wang et al. (Neurips 2024)\n\n[5] Dense Depth Priors for Neural Radiance Fields from Sparse Input Views. Roessle et al. (CVPR 2022)", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper writing is easy to follow.\n\n2. The experiment covers lots of different dataset for a comprehensive comparison, like Scannet, TanksandTemples, Scannet++, Replica.\n\n3. The ablation study indicates the effectiveness of the deflection field and adaptive angle prior when compared to its base model.", "weaknesses": "Major weakness:\n\n1. Although the dataset included is various, the compared methods are not so much comprehensive(except the Scannet). The quantitative comparison on TanksandTemples, Scannet++, Replica only includes three works in 2022(NeuRIS, MonoSDF, VolSDF) and one work in 2021(Unisurf). This leads to an insufficient experiment. For example, what about the comparison with HelixSurf, TUVR, DebSDF, H2OSDF on TanksandTemples, Scannet++, Replica? The reviewer thinks the comparison without newer baselines is insufficient on TanksandTemples, Scannet++, Replica. Although Table 1 includes most newer baselines, usually only 4 scenes in Scannet are used for indoor scene reconstruction experiments. This is why the reviewer thinks that more quantitative experiments are required here.\n\nDetails: The reviewer understands that it's not necessary to run all previous baselines in all datasets. But just place here as an example. Since the unbiased weight technique is used, the reviewer thinks **at least** TUVR(CVPR2023) is a newer baseline that should be included in both the quantitative and qualitative comparisons on TanksandTemples, Scannet++, Replica datasets. (But more comparison with other newer methods will enhance the paper's experiment part.)\n\n2. The qualitative comparison is also insufficient (only compared with older baselines). It's better to include the qualitative comparison with newer baselines in Figure 4, Figure 11, Figure 12, Figure 13.\n\n3. Some works have both pure MLP version and hash grid version (like MonoSDF). The performance and efficiency of different versions have some differences. Since ND-SDF includes instant-NGP in its implementation, it's better to clearly indicate the version to compare within the table. (For example, \"MonoSDF(mlp)\", \"MonoSDF(grid)\".)\n\nMinor weakness:\n\n4. For Section 3.4, similar ideas of better ray sampling strategy have been investigated in some previous work. Like edge-aware sampling in EaNeuS[1], GaussianRoom[2], and uncertainty-aware sampling in DebSDF. Please consider adding a brief discussion about these previous methods.\n\n5. In Figure 2, \"GT depth\" and \"GT Normal\" should be changed to \"depth prior\" and \"normal prior\".\n\n6. The number of selected scenes in each dataset for the experiment has not been reported.\n\n7. The dataset conducted in the ablation study has not been reported.\n\nReference:\n\n[1] Edge-aware Neural Implicit Surface Reconstruction. Li et al. (ICME 2023)\n\n[2] GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction. Xiang et al. (arxiv 2024)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)