NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction
Authors: Yifan Wang
Di Huang
Weicai Ye
Guofeng Zhang
Wanli Ouyang
Tong He
Venue: NeurIPS 2024 poster (2024)
Keywords: SDF representation; Volume Rendering; High-Fidelity Surface Reconstruction
Primary area: machine_vision
TL;DR: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction with Intricate Structures

Abstract: Signed Distance Function (SDF)-based volume rendering has demonstrated significant capabilities in surface reconstruction. Although promising, SDF-based methods often fail to capture detailed geometric structures, resulting in visible defects. By comparing SDF-based volume rendering to density-based volume rendering, we identify two main factors within the SDF-based approach that degrade surface quality: SDF-to-density representation and geometric regularization. These factors introduce challenges that hinder the optimization of the SDF field. To address these issues, we introduce NeuRodin, a novel two-stage neural surface reconstruction framework that not only achieves high-fidelity surface reconstruction but also retains the flexible optimization characteristics of density-based methods. 
  NeuRodin incorporates innovative strategies that facilitate transformation of arbitrary topologies and reduce artifacts associated with density bias.
  Extensive evaluations on the Tanks and Temples and ScanNet++ datasets demonstrate the superiority of NeuRodin, showing strong reconstruction capabilities for both indoor and outdoor environments using solely posed RGB captures. Project website:
https://open3dvlab.github.io/NeuRodin/

NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction
Yifan Wang 12 Di Huang
Weicai 2e2,3,5 Guofeng Zhang?
Wanli Ouyang²
Tong He
Shanghai Jiao Tong University Shanghai Artificial Intelligence Laboratory
State Key Lab CAD&CG, Zhejiang University
maikeyeveicai@gaail.c
tonghe90@gmail.cmm
Figure 1: We present NeuRodin, novel two-stage framework designed for high-fidelity neural surface reconstruction with intricate structures. Requiring only posed RGB captures as inputs, NeuRodin not only recovers large-scale areas but also accurately reconstructs fine-graing ine-grained details.
Abstract
Signed Distance Function (SDe)-based volume rendering has demonstrated sig- nificant capabilities in surface reconstruction. Although promising, SDF-based methods often fail to capture detailed geometric structures, resulting in visible defects. By comparing SDF-based volume rendering density-based volume rendering, we identify two main factors within the SDF-based approach that de- grade surface quality: SDF-to-densin representation and geometric regularization. These factors introduce challenges that hinder the optimization of the SDF field. To address these issues, we introduce NeuRodin, novel two-stage neural surface re- construction framework that not only achieves high-fidelity surface reconstruction but also retains the flexible optimization characteristics of density- -bsed meth- ods. NeuRodin incorporates innovative strategies that facilitate transformation of arbitrary topologies and reduce artifacts associated with density bias. Extensive evaluations the Tanks and Temples and ScanNet++ datasets demonstrate the superiority NeuRodin, showing strong reconstruction capabilities for both door and outdoor environments using solely posed RGB captures. Project website: https: http::///pp/ddldeeee github. Mo/NeuRodin//
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a) Neuralangelo
(b) Instant-NGP
(c) Ours
Mesh
Figure 2: Comparative analysis of SDF-based and density-based volume rendering methods. (a) Neuralangelo [13] experiences difficulties with topologiaal transformations, leading to incorrect surfaces. (b) Instant-NGP [18] approximates the correct surface positioning yet produces noisy
surface. (c) Our method achieves high-quality surfaces with fine details.
Introduction
3D surface reconstruction [34, 16, 35, 28, 8, 24, 26, 40, 39, 14, 12, 2, 38, 25] is long-standing research topic in the field of computer vision. This process involves using images to recover the underlying 3D geometry, typically represented as meshes. These reconstructed meshes find diverse applications in various domains, including video games and augmented/virtual reality systems, In this paper, we specifically address the challenge of reconstructing 3D surfaces from posed RGB images.
Inspired by the density- representation [15] for task of novel view synthesis, recent works for neural surface reconstruction commonly introduce signed distance functions, (SDF) [28, 35] to recover high-quality geometry.
However, incorporating SDF to the density function is nontrivial and often fails to intricate geometric details. We illustrate this by comparing two methods for reconstructing the same scene: Instant- NGP [18], which employs density- volume rendering, and Neuralangelo [13], which utilizes SDF-based volume rendering. Both methods use a similar multi-resolution hash table representa- tion. The mesh is produced by is by ISDF-fusion [4] for Instant-NGP. As illustrated in Figure Instant-NGP reconstructs surfaces with accurate localization albeit with a certain roughness, while Neuralangelo produces smoother surfaces yet encounters issues in correctly positioning portions of the surface. This disparity underscores the limitations and highlights the need for improved modeling capabilities in SDF-based surface reconstruction methods.
Why do SDF-based surface reconstruction methods face challenges in accurately capturing intricate geometric details, and how can these methods be improved? In this paper, thoroughly analyze the reconstruction pipeline and identify two primary factors in current SDF-based pipelines that contribute suboptimal surface reconstruction:
SDF-to-density conversion: SDF-based volume rendering requires a conversion function to relate the SDF field with the density field. Existing methods use a conversion function that assigns uniform density across the same level sets, which restricts the representation of arbitrary non- negative density values. Additionally, there is no assurance that the geometric representation within a volume rendering framework will align perfectly with the implicit surface. This misalignment often results in accurate visual renderings on incorrect surfaces, due to inherent biases.
Geometric regularization: Regularization constraints imposed on the implicit surface can limit topological changes during optimization. These constraints often introduce biases, complicating the convergence of the model and hindering its ability to accurately represent complex geometries.
To tackle these challenges, we introduce NeuRodin (Figure 1), a high-fidelity 3D surface recon- struction method that innovatively overcomes the limitations previously outlined. Firstly, we refine the SDF-to-density conversion by transitioning from global scale parameter parameter. Unlike previous methods that enforce the same densities for points with identical SDF values, our approach enhances the flexibility and effectiveness of the SDF function by allowing adaptive density values. Secondly, we implement a novel loss function designed to align the max- imum probability distance with the zero-level set in volume rendering, improving the alignment of geometric representations. Additionally, We incorporates above innovations within a two-stage optimization framework to tackle the over-regularization imposed by geometric constraints. Initially, employ coarse optimization stage in which the SDF field operates similarly to density field,
local adaptive

---

exhibiting minimal influence from topological transformations. Subsequently, refinement stage is conducted to achieve a surface with enhanced smoothness. We also introduce the stochastic-step numerical gradient estimation technique to mantain a natural zero level set for the coarse stage. With the design described, our method enables high-fidelity surface reconstruction suited to both large-scale and intricate geometries.
We conducted extensive experiments on the Tanks and Temples dataset [1]] and the ScanNet++ dataset [41], where our method demonstrated superior performance over the previous state-of-the-ar in both indoor and outdoor environments. Notably, recognizing the lack of an established benchmark for ScanNet++, we executed comparative analyses using six baseline methods and established new benchmark for ScanNet++ reconstruction, significantly enriching the community resources and setting foundation for future research. In comparative tests, our model outperformed Neuralangelo on the Tanks and Temples training set, delivering superior results with only 1/8 the parameters the comparative model. Our approach excels in optimizing complex topological structures and preserving intricate details, enabling high-fidelity, fine-grained surface reconstruction.
2 Related Work
Multi-view 3D reconstruction. In traditional 3D surface reconstruction, methods based on Multi- View Stereo (MVS) have long been prevalent, serving as foundational approach for mining sparse geometric data from multiple views and generate detailed 3D models by comparing and analyzing the disparities across multiple camera perspectives. The traditional MVS methods [24], while effective in texture-rich domains, often stumble upon the challenge of processing ambiguous observations. The point clouds produced by traditional MVS methods suffer from noise, undermining the reliability of the surface triangle meshes reconstructed from these point clouds. For learning- based MVS methods [34, 44, 33, 32], the generated point clouds are still plagued by noise, leading consistently incomplete reconstructions.
Neural surface reconstruction. NeRF [16, 37, 10, 17] pioneers the use of neural network to represent neural radiance fields for novel view synthesis and optimizes these scenes through differentiable volume rendering. Following NeRF, subsequent research has combined implicit surfaces with differentiable volume rendering [19, 28, 35]. These methods typically represent implicit surfaces as SDF and use the zero-level set of SDFs to describe geometry, achieving high-quality reconstruction on individual objects. Various improvements have been made based on this foundation, including the incorporation of different positional encoding enhance representational capabilities [22, 29, 47, 48] and the introduction of additional priors to deal with surfaces that exhibit specular highlights or have low textures [43, 27]. Several studies refine the modeling of SDF-to-density conversion [46, 31] to address bias issues in density. Meanwhile, other works employ patch-match techniques to improve multi-view consistency [5, 7]. Neuralangelo [13] enhances the network's representational capability by introducing hash encoding. Additionally, it proposes numerical gradients and coarse-to-fin to-fine optimization strategies to enhance the quality of surface reconstruction.
3 Study on SDF-based Volume Rendering
3.1 Preliminary
Density-based volume rendering Density-based volume rendering methods model a 3D scene as a volume density field. Given camera position o and view direction d, the ray emitted from o in direction d is denoted as (r(t) = o + tdlt > 0}. A set n points is sampled along this ray. The predicted density (r((t) and geometry features (r((t) of the point r(t) are obtained from a geometry network dgeo- the density is parameterized by an activation function, such as ReLU, softplus, or the exp function, prior being output by the network.
color network &eoter predicts the color c(r(t), d), taking as inputs the geometry feature z(r(t)). and the viewing direction d. The rendered color of this ray can be calculated as:
((r) = f (())()(()))(((()) T(t) = exp (- ((oot(w)ddu)
(I)
3

---

The networks are trained to minimize a color loss Ceolor that quantifies the difference between the rendered colors and the ground truth colors:
Loolor 1 Σ (((())
(2)
FER
with R representing the set of m rays in each training batch and (r)) being the ground-trun truth color of ray r.
SDF-based volume rendering SDF-based volume rendering methods, such as NeuS [28] and VoISDF [35], combine volume rendering with an SDF representation. Unlike density-based methods, the geometry is represented as the zero level set of the SDF The predicted SDF f (r((t) and geometry feature (()) at the point r(() are obtained from the geometry network &ges- Then, the SDF f (r(()) is transformed into density (r(()) by a predefined function V/ and a global scale For instance, VoISDF defines the density as the scaled cumulative distribution function of the negative SDF:
28 exp
f(r(t))
if f(r(t)) 2 0, if f(r(t) < O.
a(((1)) v.(((r()))
(1
((r(o))
exp
Moreover, to encourage the SDF to have a unit norm gradient, the Eikonal loss [9] is often employed:
1
Lakk
mn E -1))
Incorporating this loss not only helps avoid suboptimal solutions the zero level set but also promotes smoothness. The networks are trained under the supervision of both the color loss Loolor and the Eikonal loss Leik-
3.2 Challenges in Previous SDF-Based Volume Rendering
State-of-the-ard SDF-based volume rendering techniques frequently fail to reconstruct surfaces with accuracy in scenarios where density-basee methods manage to renders realistic novel views. This disparity highlights the inherent limitations of SDF-based volume rendering approaches. To further elucidate these issues, explore the fundamental distinctions between SDF-based and density-based volume rendering. Please refer to the appendix for a more in-depth analysis.
Unsuitable assumption for SDF-to-density conversion. SDF-based volume rendering methods typically employ a predefined function and global scale parameter to convert SDF values into density, as described by Equation (3). These methods often result in uniform density values for points sharing identical SDF values. Such global scaling mechanism restricts the representation capability of the density field derived from the SDF field. Intuitively, previous ap-performing methods for novel view synthesis can generate arbitrary non-negative density values within (0, +oo). In contrast, incorporating SDF representation with a global scaling factor for surface reconstruction can only result in density values within (0, 2].
Bias of the density. When applying an Eikonal constraint or any form of smooth regularization to an SDF, the geometric representation within the rendering framework must align with that of the SDE Unfortunately, current SDF-based methods often fail to ensure this alignment, particularly at larger-scale parameters s as explored mathematically [46] to analyze this issue. Recent studies [31, 46, 3] have attempted to tackle this issue, proposing designs for SDF to density conversion that aim to minimize bias. Despite these advancements, these solutions still exhibit inherent biases. Additionally, the introduction of geometric regularization often exacerbates this bias, complicating model convergence and resulting in the creation of inaccurate surfaces. A more detailed analysis of this issue is provided in Section 4.2 and Appendix B to Appendix D.
Over-regularizztion of Geometry. To maintain a high-quality surface, previous methods often introduce geometric constraints, such as Eikonal loss or smoothness constraints. However, these global constraints result in excessive smoothing across all regions, both flat and intricate, leading to a loss of fine details. Moreover, in the framework of SDF-based volume rendering, the prediction of color typically necessitates being conditioned on normals following IDR [36], a characteristic that distinctly sets it apart from the dansity-based volume rendering approach. When optimizing the

---

color conditioned on the normal and explicitly constraining the SDF by geometric regularization, the optimization process restricts the topologiaal structure. Please refer to the Appendix F for the impact on the normal condition.
4 Method
4.1
Uniform SDE, Diverse Densities
To deal with the representation limitations of SDF-transformed density, instead of using global scale for the transformation from SDF to density, we have employed strategy akin that [30], which utilizes a non-linear mapping to obtain the unique scale associated with a given point r(t). More precisely, it is defined as follows:
((((t)),s(r()) z(r(t))) ggoo(((t)) (r(t)), (r((t) Va((()) (f(r(t)).
(5)
With this particular design, the density is not identical within the same SDF level set and can achieve any non-negative value through the continuous representation that maps an input coordinate to its corresponding scale.
This approach ensures that densities within the same SDF level set are no longer uniformly identical. Instead, they vary, achieving any non-negative value through continuous representation that maps input coordinates to their corresponding scales. This design greatly enhances the flexibility and accuracy of our density modeling, enabling more realistic and detailed reconstructions. More detailed analysis regarding the local scale can be found in Appendix A.
4.2 Explicit Bias Correction
The issue of bias represents critical concern frequently addressed within SDF-based volume rendering. As demonstrated Figure 3, is necessary to align the geometric representation under the volume rendering framework with that of the implicit surface. For the volume rendering framework, the most intuitive way to represent geometry is through the rendered distance:
D Dremderea (r) = ((t)a(r(t))tdL
We can also consider the position where wt) is maximized that is. the probability that the light ray arrives and collides is the greatest, or in other words, the location that contributes the most to the color as the geometric representation within the volume rendering framework:
Dprah (r) arg max w(l = arg max (r()). (E(0,+00)
rE(0,+0o)
We shall refer to Dprot (r) as the maxinum prob- ability distance. For an implicit surface, the zero level set offers a direct geometric representation. In an ideal scenario, irrespective of whether con- vergence has been achieved, the geometric rep- resentations of volume rendering (i.e., rendered distance and maximum probability distance) and the geometric representation of the implicit sur- face (i.c., zero level set) should be aligned, as illustrated in Figure (a). However, in the practi- cal optimization process, conflicts such as those depicted in Figure (b) may arise, leading to misalignment between the two representations.
Weight
SDF
A 8 Ideal situation (a)
A * B c Biased situation (b)
Figure 3: Visualization of the bias of the den- sity. (a) An ideal scenario where the geometry of the volume rendering scheme (A: maximun prob- ability distance and B: rendered distance) aligns precisely with the geometry of the implicit surface (C: zero level ser). (b) A biased scenario showcas- ing misalignment.
Multiple past methods have broached this topic, offering various solutions. For example, in TUVR [46], an unbiased model is proposed and it is mathematically proven that the zero-crossing point of the SDF a local maximum of rendering weight. However, bias still exists. The experimental analysis is presented in Section 5.3. In the work
5

---

of [3], penalty is imposed on the rendered distance to ensure that its SDF value is equal to zero. However, is important to note that the rendered distance subject significant distortion due to existing biases, particularly in the early stages of convergence as shown in Figure (b). We offer further analysis from Appendix B and Appendix D.
We propose an explicit bias correction in which we opt to deliberately align the maximum probability distance with the zero level Specifically, we define
1 Σ max (f(r(1" (bias)), 1- = PER
Lbi
arg max (()))((()), (E(0,+00)
(8)
where Ebins is bias correction factor. The loss function is designed to penalize the positive portion of [Ir(t + )bins), which encourages the SDF to take on negative values after the maximum probability distance. We have shown Appendix C that manually scheduling the lower bound of the local scale, coupled with the penalty after the point of maximum probability distance, can effectively alleviate bias issues. During the experiment, we approximate t* by directly using the sampled point with the largest wt),, which, despite introducing a certain degree of deviation, does not affect the overall effectiveness. Please refer to Appendix for details on the design.
4.3 Two-Stage Optimization to Tackle Geometry Over- Sener-eguuariztioon
Previous methods often produce incorrect surfaces due to over-regularization of geometry as shown in Figure 2 (a). However, we have discovered that methods based on density are not constrained by changes in topology as shown in Figure 2 (b), prompting to question whether the SDF field can be first optimized as freely as density field, then refine to a smooth surface by geometry regularization. We now propose novel two-stage optimization approach. This approach allows the optimization process to initially mimic density-based behavior in the first stage and subsequently refines to a smooth surface in the second stage.
For the first stage, our objective to tackle the over-regularizatton issue. An intuitive solution might be to eliminate downweight any geometric constraints and avoid conditioning the color on the predicted normal, but this approach often results in an unnatural zero level set [9, 28, 35, 36]. We experimentally validated in the Appendix H.6.
We have identified a simple but effective method to preserve the natural level sets of large-scale scale structures while allowing the formation of complex structures to be unimpeded by geometric regu- larization. Instead of applying geometric regularization directly to the gradient (f(((t)), we elect to impose them upon an estimated gradient V (r))), to which we introduce uncertainty through a th z-component of the estimated gradient
specific design. Specifically, the
vas(r()) = f (r(1) + Ex) (r(()= En)
where e, (e, 0,00 and U(0, 6
2e
The gradient is calculated through finite dif- ferences similar to those described in [13, 21]. However, the step size for gradient estimation at each iteration is stochastically sampled from a uniform distribution in the range of (0, mmax).
Using this technique, we observed that esti- mated larger-scale normals have smaller vari- ance, while fine details exhibit larger variance, depicted in Figure 4. This introduces uncer- tainty in geometric regularization, ensuring sta- bility for large features and flexibility for com- plex details. We provide further explanation in Appendix E.
(a) Reference image
(b) Variance of normal
Figure 4: The heatmap for the variance of the nor- mal predicted using random step.
During the initial stage, our goal to reconstruct the approximate, coarse structure of the 3D content. This is primarily addressed by tackling the issues of over-regularization and the bias in density estimation that we previously mentioned. We employ the stochastic-step numerical gradient estimation along with the explicit bias correction to address the initial reconstruction of the coarse

---

Metric
F-Score NeuS-NGP
Scene
Neur/Waap
CALMAP
NeuS
Geo-NeuS
MonoSDF
Neuralangelo"
Neuralangelo
Ours
Barn
0.22
0.55
0.29
0.33
0.46
0.49
0.61
0.70
0.70
Caterpillar
0.18
0.01
0.29
0.26
0.32
0.31
0.34
0.36
0.36
Courthouse
0.08
0.11
0.17
0.12
0.08
0.12
0.13
0.28
0.21
Ignatius
0.22
0.83
0.72
0.81
0.78
0.82
0.89
0.87
Mectingroom
0.08
0.19
0.24
0.20
0.08
0.23
0.22
0.32
0.43
Truck
0.45
0.45
0.44
0.42
0.45
0.47
Meun
0.15
0.21
0.38
0.35
0.37
0.39
0.43
0.50
0.51
Table 1: Quantitative evaluation of method the Tanks and Temples training subset. The
best performance and the second-best outcomes are highlighted for easy reference. Note that the hash grid parameters used in method is the same as Neuralangelo" which possesses 2 hash entries per resolution.
NeuS
Neuralangelo
Ours
GT Point Cloud
Figure 5: Quantitative comparison on the training subset of Tanks and Temples dataset. structure of the 3D content. Additionally, we utilize VoISDF's SDF-to-density conversion, with the local scale modeling delineated in Equation 5, facilitate this primary formulation. Consequently, the training loss at this stage is formulated as:
Ceaare
Leolor + Aeik (Vf) + Abias Coias.
(10)
During the refinement stage, we discontinue the use of the estimated gradients given that the funda- mental 3D content has been initially restored and the issue of over-regulaiization no longer presents a concern. In similar vein, move past the explicit bias correction, as the significant surface errors induced by bias were initially addressed in the first stage. We incorporate a standard Eikonal loss alongside smoothness constraint from PermutoSDF [22] to enforce local smoothness:
1 Σ (r(()) - (r(() + e.m(r(())) -1)² r.t
Symooth
(11)
7187
where ()(r)t) n(r(t)) T and is a random unit vector. Furthermore, the model nears convergence at this stage, we adopt the SDF-to-density conversion method proposed by TUVR [46], which ensures minimal bias and preserves fine object details. The loss function employed in second phase is defined as follows:
Lime Leolor + Leik (V + Asmoouth Samooth-
(12)
Experiments
Experimental setup. We carry experimental evaluations on two benchmark datasets: Tanks and Temples [1 [11] and ScanNet++ [41]. We include several baselines for comparisons: VoISDF [35]. NeuralWarp [5]. COLMAP [23]. NeuS [28]. Geo-NeuS [7], Neuralangelo [13] and MonoSDF [43]. We extract mesh through marching cube algorithm with a resolution of 2048 applied across all scenes and report the F-score for suface evaluation. More details are provided in the supplementary materials, Please refer to Appendix H for additional experimental results.
5.1 Tanks and Temples
NeuRodin outperforms previous state-of-the-ard methods terms of the average F-score. Owing to our explicit bias correction technique, the barn's 's roof maintains its structural integrity without
7

---

Ours Figure 6: Quantitative evaluation of our method on the Tanks and Temples advance subset.
Reference Image
MonoSDF
Neuralangelo
collapsing shown at the top of Figure 5, marked improvement over other methods which of- ten fail to prevent such collapse. Thanks to our two-stage optimization approach, we effectively mitigate the issue of excessive geometric regularization shown at the bottom of Figure 5. Conse- quently, NeuRodin achieves more detailed surface representation than Neuralangelo with 1/8 fewer parameters.
evident that our method out- performs previous work on the Tanks and Temples advance sub- set as shown in Table 2. In the comparison in Figure 6, we demonstrate enhanced accuracy and completeness when recon- structing large-scale surfaces, alongside capturing more fine- ame-iraieed details compared to Neuralan- gelo. Benefiting from our explicit bias correction in the first stage and the TUVR modeling employed in the second stage, fine structures restored the zero level set in the initial phase and maintain sufficiently small bias in the subsequent phase, thus enabling the refinement of high-quality surfaces.
Method
VolSDF
MonoSDI
Neuralangelo'
Neuralangelo
Ours 28.84
Mean
6.72
20.89
27.14
26.28
Table 2: Quantitative evaluation of our method versus prior work on the Tanks and Temples advance subset. The best performance and the second-best outcomes are highlighted for easy reference.
5.2 ScanNet++ Benchmark
Since public results are available for the ScanNet++ dataset, randomly selected 8 scenes to construct a benchmark. For more details and results on our ScanNet++ benchmark, please refer to the supplementary materials. Quantitative results are shown in Table 3. We surpassed the methods we compared against in most scenes and achieved comparable results to those with prior knowledge in terms of F-score. We provide more visual result ScanNet++ dataset in the supplementary.
F-Score f
Without Prior Neuralangelo" 0.507
With Prior
method
NeuS
VoISDF
Neuralangelo
Ours
MonoSDF-MLP MonoSDF-GH
Mean
0.455
0.391
0.564
0.638
0.439
0.642
Table 3: Quantitative evaluation of our method versus prior work on the ScanNet++ dataset. The best performance and the second-best outcomes are highlighted for easy reference.
5.3 Analysis
Ablation Study. To validate the efficacy of the proposed techniques, we performed an ablation study on scene Meetingroom from Tanks and Temples dataset. As illustrated in Figure (a), applying global scale for SDF-to-density conversion results in inaccurate surfaces, primarily due to the

---

(8)
Tel
GT Point Cloud
Figure 7: Ablation results. (a) Without local scale for SDF-to-density conversion. (b) Without stochastic-step numerical gradient estimation. (c) Without explicit bias correction. (d) Without stage-two refinement. (e) Full model.
Depth mup produced at iteration 7500 during the first stuge.
Final mesh
Referaee Image
Model A
Model B
Model C
Instant-NGi
Model B
Model C:
GT Mesh
Figure 8: Analysis on stochastic-step numerical gradient estimation. We visualize the produced depth maps at iteration 7500 of the first stage. Model A: Change stochastic-step numerical gradient estimation to analytical gradient. Model B: Change stochastic-step numerical gradient estimation to progressive numerical gradient estimation from Neuralangelo. Model C: Ours.
assumption of uniform density across the same level sets. This assumption leads the convergence of surfaces with subtler textures to incorrect locations. Figure 7 (b) demonstrates that the absence of stochastic-step numerical gradient estimation hinders the model's ability to form arbitrary topologies. leading to incorrect surfaces. The incorrect ground collapsing depicted in Figure 7 (c) is due to the bias in density, creating inconsistencies as shown in Figure 3(c). Without the application of our proposed explicit bias correction, this bias issue causes visibly incorrect surfaces. Figure 7 (d) presents the outcomes of optimization conducted in single phase; under stochastic-step numerical gradient estimation, the Eikonal loss's preference for smoothness is somewhat compromised, resulting in rougher surface finish. Finally, Figure 7 (e) showcases our full model, which achieves high-fidelity smooth surfaces while preserving most details. We conducted additional ablation experiments in Appendix H.6.
Analysis on Stochastic-step Numerical Gradient Estimation. We further demonstrate experimen- tally the impact of our stochastic-step numerical gradient estimation on the optimization process, as illustrated in Figure 8. The depth maps produced indicate that Model A is severely constrained by geometric regularization, making it difficult to alter its topological structure during optimization. Model B employs a progressive numerical gradient estimation technique that, even after 7500 steps, does not yield an accurate depth map. However, utilizing our stochastic-ster numerical gradient estimation, we achieve an approximately accurate scene geometry in as few as 7500 steps.
At this stage, the depth map of Model C is similar that of Instant-NGP, yet ours displays more natural and smooth depth profile. This suggests that our approach, similar to instant-NGP, is capable of freely altering topological shapes for optimization, while also maintaining a natural zero-level set surface.
Furthermore, final mesh of Model B still manifests inaccurate floor collapses, whereas mesh of model C, with our stochastic-step numerical gradient estimation, maintains correct and smooth floors. This is attributed to the fact that the floor, being large scale, allows stochasticity to continuously apply the Eikonal constraint on large-scale areas. This results in a natural zero level set in these vast regions. However, for progressive steps, once the step size is reduced beyond a certain point, it no longer imposes the Eikonal constraint on large-scale surfaces, resulting in unnatural zero level sets.
Analysis on Explicit Bias Correction. bias correction approach, we further conducted experiments to verify its potential as a plug-and-pll correction method independent of our full model. We implemented this correction technique our coarse stage and tested it across various renderers, including NeuS, VoISD, and TUVR. These experiments were designed to evaluate the adaptability and efficacy of our bias correction in diverse SDF-to -DF-to-density modelings.
To substantiate the versatility and effectiveness of our explicit
As depicted in Figure 9, the ceiling of the room displays pronounced bias issue that leads to collapse. However, with the application of our proposed explicit bias correction approach, the issue of ceiling collapse is significantly ameliorated.

---

wio explicit bias correction
(b) wi explicit bias correction
Reference Image
VolSDF
NeuS
TUVE
VoISDP
NeuS
TUVR
Figure 9: Analysis on explicit bias correction. We visualize the produced depth maps with and without our explicit bias correction for different SDF-to-density modelings at the first stage.
Conclusion
This paper proposes NeuRodin, two-stage framework for high- figh-fidelity neural surface reconstruction with intricate details. It introduces key designs to tackle SDF-based rendering challenges, notably the local scale adjustment for SDF-to-density conversion, which enables any non-negative value to be achieved, facilitating accurate density derivation from SDF. Additionally, an explicit bias correction method is employed ensure the geometry of the volume rendering scheme coherently aligns with that of the implicit surface, thereby preventing the emergence of incorrect surfaces. Finally, two- stage optimization strategy effectively resolves the issue over-regularization imposed by geometric constraints. Comprehensive experiments demonstrate that NeuRodin simultaneously delivers superior quality.
Acknowledgments and Disclosure of Funding
This work funded in part by the National Key R&D Program China No.20222Z0110110,, and Shanghai Artificial Intelligence Laboratory.
References
[1] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip- nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 5470-5479, 2022.
[2] Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang, Weijian Xie, Shangjin Zhai Nan Wang, Haomin Liu, Hujun Bao, and Guofeng Zhang. Pgsr: Planar based gaussian splatting for efficient and high-fidelity surface reconstruction. arXiv preprint arXiv:2406.000 20620666.6,, 2024.
[3] Decai Chen, Peng Zhang, Ingo Feldmann, Oliver Schreer, and Peter Eisert. Recovering fine details for neural implicit surface reconstruction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4330-4339, 2023.
[4] Brian Curless and Marc Levoy. volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 303-312, 1996.
[5] François Darmon, Bénédicte Bascle, Jean-Clément Devaux, Pascal Monasse, and Mathieu Aubry. Improving neural implicit surfaces geometry with patch warping. In Proceedings the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 6260-6269, 2022.
[6] Ainaz Eftekhar, Alexander Sax, Jitendra Malik, and Amir Zamir. Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans. Proceedings of the IEEÉ/CVF International Conference Computer Vision pages 10786-10796, 2021.
[7] Qiancheng Fu, Qingshan Xu, Yew Soon Ong. and Wenbing Tao. Geo-neus: Geometry-consissent neural implicit surfaces learning for multi view reconstruction. Advances in Neural Information Processing Systems, 25:3403-3416, 2022.
[8] Silvano Galliani, Katrin Lasinger, and Konrad Schindler. Massively parallel multiview stereopsis by surface normal diffusion. In Proceedings of the IEEE international conference on computer vision, pages 873-881, 2015.
10

---

[9] Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. In Proceedings of the 37th International Conference on Machine Learning, pages 3789-3799, 2020.
[10] Chenxi Huang, Yuenan Hou, Weicai Ye, Di Huang, Xiaoshui Huang, Binbin Lin, Deng Cai, and Wanli Ouyang. Nerf-det++: Incorporating semantic cues and perspective-aware depth supervision for indoor multi-view 3d detection. arXiv preprint arXiv:2402.1 14464, 2024.
[11] Arno Knapitsch, Jaesik Park, Qian- Yi Zhou, and Vladlen Koltun. Tanks and temples: Bench- marking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1-13, 2017.
[12] Hai Li, Weicai Ye, Guofeng Zhang, Sanyuan Zhang, and Hujun Bao. Saliency guided subdivision for single-view mesh reconstruction. In 2020 International Conference on 3D Vision (3DV), pages 1098-1107. IEEE, 2020.
[13] Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H Taylor, Mathias Unberath, Ming- Yu Liu, and Chen-Hsuan Lin. Neuralangelo: High-fidelity neural surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages -456-8465, 2023.
[14] Xiangyu Liu, Weicai Ye, Chaoran Tian, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang. Coxgraph: multi-robot collaborative, globally consistent, online dense reconstruction system. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8722-8728. IEEE, 2021.
[15] Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovit- skiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7210-7219, 2021.
[16] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
[17] Yuhang Ming, Weicai Ye, and Andrew Calway. idf-slam: End-to-end rgb-d slam with neural implicit mapping and deep feature tracking. arXiv preprint arXiv:2209.0791 07999, 2022.
[18] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1- 15, 2022.
[19] Michael Oechsle, Songyou Peng, and Andreas Geiger. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5589-5599, 2021.
[20] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, performefree deep learning library. Advances in neural information processing systems, 32, 2019.
[21] Albert Pumarola, Artsiom Sanakoyeu, Lior Yariv, Ali Thabet, and Yaron Lipman. Visco grids: Surface reconstruction with viscosity and coarea grids. Advances in Neural Information Processing Systems, 15608000-1871, 2022.
[22] Radu Alexandru Rosu and Sven Behnke. Permutosdf: Fast multi-view reconstruction with implicit surfaces using permutohedral lattices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1466-8475, 2023.
[23] Johannes L. Schonberger and Jan-Michael Frahm. Structure-from-mete motion- revisited. In Proceed- ings of the IEFE conference on computer vision and pattern recognition, pages 4104 2016.
11

---

[24] Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Mare Pollefeys. Pixelwise view selection for unstructured multi-view stereo. In Computer Mision-ECCH 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 501-518. Springer, 2016.
[25] Ziyu Tang, Weicai Ye, Yifan Wang, Di Huang, Hujun Bao, Tong He, and Guofeng Zhang. Nd-sdf: Learning normal deflection fields for high-fidelity indoor reconstruction. arXiv preprint arXiv:2408. 12598, 2024.
[26] Hoang-Hiep Vu, Patrick Labatut, Jean-Philippe Pons, and Renaud Keriven. High accuracy and visibility-consistent dense multiview stereo. IEEE transactions on paltern analysis and machine intelligence, 34(5):889-4)) 2011.
[27] Jiepeng Wang. Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. Neuris: Neural reconstruction of indoor scenes using normal priors. In European Conference on Computer Vision, pages 139-155. Springer, 2022.
[28] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2 10689, 2021.
[29] Yiming Wang, Qin Han, Mare Habermann, Kostas Danilidis, Christian Theobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In Pro- ceedings of the IEEE/VV International Conference on Computer Vision, pages 3295-3306, 2023.
[30] Zian Wang. Tianchang Shen, Merlin Damier-David, Nicholas Sharp, Jun Gao, Alexander Keller, Sanja Fidler, Thomas Müller, and Zan Gojeic. Adaptive shells for efficient neural radiance field rendering. ACM Transactions on Graphics (TOG), 42(6):1-15, 2023.
[31] Yuting Xiao, Jingwei Xu, Zehao Yu, and Shenghua Gao. Debsdf: Delving into the details and bias of neural indoor scene reconstruction. arXiv preprini arXiv:2308. 15536, 2023.
[32] Qingshan Xu, Weihang Kong, Wenbing Tao, and Marc Pollefeys. Multi-scale geometric consistency guided and planar prior assisted multi-view stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4))4945-4963, 2022.
[33] Qingshan Xu and Wenbing Tao. Multi-scale geometric consistency guided multi-view stereo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5483-5492, 2019.
[34] Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: unstructured multi-view stereo. Proceedings of the European conference on computer vision (ECCV), pages 767-783, 2018.
Depth inference for
[35] Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805 -810005-8815000 2021.
[36] Lior Yariy, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lipman. Multiview neural surface reconstruction by disentangling geometry and appearance. Advances in Neural Information Processing Systems, 33:2492 25002, 2020.
[37] Weicai Ye, Shuo Chen, Chong Bao, Hujun Bao, Mare Pollefeys, Zhaopeng Cui, and Guofeng Zhang. IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
[38] Weicai Ye, Xinyu Chen, Ruohao Zhan, Di Huang, Xiaoshui Huang, Haoyi Zhu, Hujun Bao, Wanli Ouyang, Tong He, and Guofeng Zhang. Datap-sfm: Dynamic-aware tracking any point for robust dense structure from motion in the wild. arXiv preprint arXiv:2411..333 133291, 2024.
[39] Weicai Ye, Xinyue Lan, Shuo Chen, Yuhang Ming, Xingyuan Yu, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. Pvo: Panoptic visual odometry. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9579-9589, June 2023,
12

---

[40] Weicai Ye, Xingyuan Yu. Xinyue Lan, Yuhang Ming, Jinyu Li, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. Deflowslam: supe-sipe scene motion decomposition for dynamic dense slam. arXiv preprini arXiv:2207.08779, 2022.
[41] Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high- fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12-22, 2023.
[42] Zehao Yu, Anpei Chen, Bozidar Antic, Songyou Peng Peng, Apratim Bhattacharyya, Michael Niemeyer, Siyu Tang, Torsten Sattler, and Andreas Geiger. Sdfstudio: A unified framework for surface reconstruction, 2022.
[43] Zehao Yu, Songyou Peng, Michael Niemeyer, Torsten Sattlr,, and Andreas Geiger. Monosdf: Exploring monocular geometric cues for neural implicit surface reconstruction. Advances in neural information processing systems, 35:25018 250322, 2022.
[44] Jingyang Zhang, Yao Yao, Shiwei Li, Zixin Luo, and Tian Fang. Visibility-awan multi-view stereo network. arXiv preprint wrXiv:2008.07928 2020.
[45] Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXi::2010004992 2020.
[46] Yongqiang Zhang, Zhipeng Hu, Haoqian Wu, Minda Zhao, Lincheng Li, Zhengxia Zou, and Changjie Fan. Towards unbiased volume rendering of neural implicit surfaces with geometry pri- ors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 4359-4368. 2023.
[47] Fuqiang Zhao, Yuheng Jiang, Kaixin Yao, Jiakai Zhang, Liao Wang, Haizhao Dai, Yuhui Zhong, Yingliang Zhang, Minye Wu, Lan Xu, et al. Human performance modeling and rendering via neural animated mesh. ACM Transactions on Gruphics (TOG), 41(6):1-17, 2022.
[48] Yiyu Zhuang, Qi Zhang, Ying Feng, Hao Zhu, Yao Yao, Xiaoyu Li, Yan-Pei Cao, Ying Shan, and Xun Cao. Anti-aliased neural implicit surfaces with encoding level of detail. In SIGGRAPH Asia 2023 Conference Papers, pages 1-10, 2023.
13

---

A Analysis of Local Scale in SDF-to-Density Transformation
We further illustrate the importance of the local scale in Figure A, using simple single plane scenario for explanation. This plane has low-texture region on the left and a rich-texture region on the right.
Without special constraints, the rendering weight should converge surface in the richly textured region and form a scattered distribution in the weakly textured region.
Dirac delta function at the
However, under the assumption of global scale factor, all areas on the plane follow the same distribution which will derived in the following sections. This means that both richly textured and weakly textured regions share the same density bias, preventing the surface from converging correctly to the richly textured surface with higher certainty.
Implicit Surface True Surface
Without local scale factor:
1
Same distribution
F1 A
Same bias t
F
Z7
Implicit Surface
With local scale factor
14
1. Different distribution
A
2. Different bias of * t
r2
Low-texture Area
Rich-texture Area
Figure 10: Explanation of the motivation behind the use of the local scale factor.
By introducing the local scale factor, the most significant difference is that the distribution of rendering weights along the ray is no longer uniform. The network can adaptively converge the richly textured regions, and the density bias in these areas is no longer affected by the low-texture regions.
In Adaptive shells, their focus is primarily rendering quality and they did evaluate surface reconstruction metrics. A straightforward application to surface reconstruction can lead to issues such increased density bias (as w(t) also a function of the scale factor s())). We tackle this by implementing special designs to ensure effectiveness, such as gradually scheduling the lower bound of the scale factor correct relatively small biases and the explicit bias correction.
B Analysis of Density Bias
In this section, we analyze the density bias and discuss the main disadvantage of previous work (NeuS [28], VoISDF [35] and TUVR [46]). According to Equation (8), the rendering weight maximum
point t" should satisfy
arg max (rt))
(13)
+x0)
The derivation of rendering weight w(t) (r()) respected to 1" should equal to zero:
@w(t)
((T(t))(r(t))
Øt
Ot
1=t²
1=
aT(t)
(r((t)) (o(((t))
T(t))
01
1=:
1===
(14)
(r(((t)) (r(())))
- 1"' a())du
exp
1=
0.
Then, we have
8² o((((t)) (r(r(t))
(15)
Öt
1==-
NeuS [28] and TUVR [46] seek a modeling approach for a (r(()) such that also satisfies the above equation at the point where the SDF value is zero f(r(t")) = O. NeuS only satisfies this under the
14

---

first-order approximation of the distribution of the SDF along the ray, while TUVR extends this to arbitrary distributions.
Although TUVR's modeling ensures that the derivative of the rendering weight is zero at the point ( where the SDF value is zero, this does not mean that 15 is the location of the global maximum of the weight.
During the optimization process, the distribution of rendering weight along the ray is complex non-convex function. Therefore, TUVR only guarantees that 1" is at local maxima. Therefore, bias persists throughout the optimization process. We have shown more analysis on TUVR in Section D.
Here we take VoISDF's SDF-to-density modeling (3) an example. And considering the simplest case, where the ray intersects with single plane. In other words, the implicit surface at the current iteration is single plane. In this case, if the angle between the ray and the plane is 0, and the distance from the ray's origin to the plane is d, then the distribution of the SDF along the ray can be expressed as f (r(()) sin + d. In Equation (15), the expression on the right-hand side is
(r(((()) a
Of(((t))
[(((t))
exp
(16)
2x²
ØL
And the left-hand side is
18 exp
-2f(r(0) 8
if f(((()) 2 O, if f(r(t) < O.
(r((()
(17)
f(((())
1
2 exp
If f(t*) 2 O, then we have
2f(r((t))
Of(r(t))
[f(((tt))
4s exp
exp
2s² Øt
(((((())
- Of(x(())
exp
(18)
Ot
t=
t* sin 0 2sin 0.
exp
8
We directly obtain the closed-form solution for t" only if sin 0 s 0.5
s In sin 0) + d
t
(19)
sin 0
If ftt) < O, then we have
t sin + d
sin ( exp
(tt 0 + d +
2
exp
(20)
8
Let exp ((-Isin 0 + d)/8) denoted as m. It's easy to see that the above equation has a solution m" 2 + sin 0 / sin" 0 + 4sin @ which can be obtained using the quadratic formula only if sin > 0.5, therefore
1- s in(1/m²) +d
(21)
sin 0
After organizing the two situations, can obtain the results:
In(k) + d
[2sin 0 if sin 0 < 0.5,
I:
(22)
sin
[1/m* sin > 0.5.
In this case, the closed-form solution for 1 can be directly obtained as t = d//sin sin 0. Then the distance between and gd is
In k 0
(23)
It can be observed that A is linear function of the scale factor If the scale factor s is relatively large, the implicit surface far from the regions that contribute most to the color. At that time, the geometry regularization on the implicit surface acts on the incorrect surface, resulting in visible defects the final mesh. Our explicit bias correction aims to minimize A during the coarse stage, so that the geometric regularization can be applied to the correct surface.
15

---

(a) Penalty on both sides of
(b) Penalty on SDF after
Figure 11: Visual results on ScanNet++ with different designs of our explicit bias correction. Penalty on both sides of will lead to erroneous surface
C Design of Explicit Bias Correction
Although aligning 1" with the SDF zero-crossing is intuitive, our experiments showed that this approach requires special design considerations. We tried simply constraining s(L") to approach zero but found that visible surface defects persisted. We also attempted to constrain the SDF values both before and after t", which led to very strange optimized surfaces, likely due to overly strong constraints on the SDF. We have shown the visual result Figure 11.
The key to our method's design lies in correcting large relative bias with explicit loss functions, while smaller biases corrected by gradually scheduling the lower bound of the scale factor. We empirically found that visible surface errors are always caused by t" is being before SDF zero-crossing points ( Therefore,
penalize the SDF value just after 1" trend towards negative values, which prevents 1" from being before 1o_ For the case where L" is after 1o we can directly address it by gradually scheduling the lower bound of the scale factor, the bias in this situation is relatively small.
We provided mathematical explanation under the assumption of a sufficiently small local surface. According to Section the distance between t* and t as Equation (23).
When A 0, t" is before to, and when & > 0, t" is after tº. We visualized the values of t under different 0 in Figure 12. We found that whent" before to (sin @ less than 0.5), the relative bias is significantly greater than when L" is after 1 Therefore, we penalize the SDF value just after /* to prevent /* being before !
before 2
t after 0
0.00
-0
0.14
-0.0
-020
The angle between the ray and the plane
Figure 12: The distance between t* and 1 as the angle between the ray and the single plane varies. We find that when t" precedes 4 a significantly larger relative bias is observed.
16

---

26-
55
16
45
SBF
Wesgle, Sunt
distannc
(a) Case of ray intersection
(b) Weight distribution VoISDE TUVR
Figure 13: A particular scenario for density hias. In this situation TUVR exhibits more pro- nounced bias compared to VolSDF.
D Analysis of TUVR
Previously, we mentioned that TUVR only proves t as a local maximum for rendering weights, not global one. Here, present scenario where bias exists in the modeling of TUVR. Consider wall composed of three planes in space, with light passing through it, as shown in Figure 13 (a). In this situation, the rendering weights, illustrated in Figure 13 (b), exhibit bias in TUVR; although TUVR ensures a local peak at the SDF zero crossing point, the rendering weight is greater at previous location. In this scenario, the bias in TUVR may even be greater than that in VoISDE.
Next, we analyze the unbiasedness of TUVR under the assumption of local scale factor. To simplify the equations, we will henceforth abbreviate all r(t) and all 0t (t). When using local scale factor, the density represented under TUVR modeling is:
-f(1) = s(T) exp
if f()) 2 0, if f(t) < 0.
(t)
(24)
2
(1
2 exp (CIT)) ))
When f(t) 2 O, the left-hand side of Equation (15) is:
d'(t) (1) exp (- S(1) )+
s1(
f(e) ToT
s())
f(t) 1 s((
f(1)
(001(()))++ s(()
s2(1)
exp
s(1))((0)
(25)
When ftt O. the above expression can be simplified to:
(1)
f'())
(t)
(26)
2t (t)l((())
And we only consider the scenario where the light ray enters the plane, namely f'(t) So we have
d'(t) -1( +1
(27)
st
At this point, d²(1) 1/)²((), so Equation (15) is only satisfied when s(()) meaning the scale factor is constant. Therefore, under the assumption of a local scale factor, TUVR's local unbiasedness (ensuring the SDF crossing point is local maximum in rendering weights) cannot be achieved. When f(t) < O, we can also arrive at a similar conclusion.
E Explanation of Stochastic Gradients
Our stochastic gradient estimation introduces some uncertainty into the true normals. For large-scale features, the Eikonal loss with varying step sizes can still be successfully minimized (since SDF near
17

---

(b) Case D Figure 14: Visual results on Tanks and Temples from iteration 25,000 at the first stage.
(a) Case E
(c) Case E
(d) Case D
a large-scale plane should satisfy the Eikonal equation for different step sizes). In other words, the variance of stochastic gradients small for large-scale features, making it easier for the model to minimize the Eikonal loss. However, for fine details, the random step sizes lead to high variance in the estimated normals, which reduces the impact of the Eikonal loss and makes it more challenging for the model to minimize samples with high variance.
F Impact of Color Conditioning on Normal
In Figure 14, we experimentally observed that in indoor scenes, when color conditioning on normals upplied, the optimization process becomes very slow and adversely affects the optimization results (both surface and rendering quality). However, this issue is nearly absent in outdoor scenes. We believe this primarily due to the convergence behavior of the scale factor. Indoor scenes often have many weakly textured areas, leading to larger scale factors and a greater scatter distribution. Additionally, color conditioning on normals, which intended for geometry disentanglement as mentioned in IDR and resonable for surface points, results in many details being optimized to incorrect positions before convergence is achieved.
However, our use of stochastic normals helps mitigate these erroneous surfaces. For detailed regions, the estimated normals have greater vari- ance, which alleviates the impact of incorrect surfaces. This is also demonstrated in our abla- tion experiments, as shown in the Table Cases D and E are as follows: Case D: Excludes the stage 1 Eikonal loss but incorporates color conditioning based on the estimated normal. Case E: Excludes the stage 1 Eikonal loss but includes color conditioning based on the analytical normal.
F-score 1 Courthouse (outdoor)
D
E
Full Model
0.11
0.11
0.21
Meetingroom (indoor)
0.35
0.21
0.43
Table 4: Impact of color conditioning on normal in different scenes.
G Experimental Details
G.1 Datasets
We carry out experimental evaluations on two benchmark datasets: Tanks and Temples [11] and ScanNet++ [41]. The Tanks and Temples dataset is characterized by its large-scale, diverse real-world scenes, both indoors and outdoors. For our experiments, we utilize six scenes from the training subset, consistent with the scenes employed in Neuralangelo, maintain comparability. Additionally, extend our validation to four expansive indoor scenes from the advanced subset to further assess the robustness of our method. Turning to the ScanNet++ dataset, it is distinguished by its high-qualit indoor scenes, supplemented with DSUR-tyality images. From this dataset, we have selected eight scenes for our analysis.
G.2 Baselines
For the Tanks and Temples dataset, our methodology is compared against several prominent methods, including: Neural [5], COLMAP [23], NeuS [28], Geo-NeuS [7], and Neuralangelo [13]. In the context of the ScanNet++ dataset, our approach is contrasted with methods lacking prior knowledge, such as VolSDF [35], NeuS [28], and Neuralangelo [13]. Additionally, we evaluate our approach against methods incorporating pretrained prior information, notably MonoSDF [43]. It should be noted that our efforts to reproduce Neuralangelo for indoor scenes were unsuccessful. Instead, we employed the implementation of Bakedangelo from [42], which serves an enhanced version of Neuralangelo. Bakedangelo utilizes the same proposal network as our setup. We discovered that manually adjusting the global scale of SDF-to-density conversion in Bakedangelo significantly
18

---

Scene
Metric
NeuS
VoISDF
Neuralangleo
Neuralangelo
Ours
MonoSDF-MLL MLP
MonoSDF-
Acc
0.053
0.053
0.326
0.121
0.166
0.041
0.064
Comp
0.065
0.078
0.039
0.053
0.029
0.038
0.027
0e7513c4d9
Prec
0.433
0.395
0.381
0.584
0.611
0.574
Recal
0.437
0.363
0.734
0.719
0.790
0.665
0.705
F-score
0.435
0.378
0,502
0,607
0.671
0.637
0.633
Acc
0.050
0.053
0.184
0.168
0.028
0.062
0.037
Comp
0.066
0.098
0,028
0.029
0.097
0.037
036bce3393
Prec
0.507
0,475
0.509
0.520
0,688
0.383
0.606
Recal
0.492
0.372
0.754
0.780
0.733
0.333
0.656
R-acore
0.499
0,417
0.608
0.624
0,710
0.356
0.630
Acc
0.047
0.039
0.121
0.092
0.044
0,059
0.041
Comp
0.088
0.097
0.060
0.050
0.053
0.113
0.045
108cc0b806
Prec
0,519
0.475
0,389
0.498
0.597
0.395
0.575
Recal
0.432
0.372
0.542
0.624
0.587
0,303
0,576
F-score
0.472
0.417
0.453
0.554
0.592
0,343
0.576
Acc
0.046
0.070
0.247
0.261
0.075
0.054
0.042
Comp
0.050
0.060
0,071
0.049
0.031
0.062
0.031
21d970d8de
Prec
0.526
0,383
0.403
0.403
0.588
0,365
0.580
Recal
0.565
0.410
0.595
0.656
0.726
0.371
0.676
F-score
0.545
0.396
0,481
0.499
0,650
0.368
0.624
Acc
0.042
0.043
0.073
0.079
11.034
0.043
0.032
Comp
0.075
0.071
0.038
0.034
0.038
0.060
0.031
355e5e32dh
Pree
0.534
0.501
0.575
0.575
0.672
0.439
0.657
Recal
0.465
0.447
0.701
0.730
0.689
0.404
0.683
F-score
0.497
0.475
0.632
0.643
0.681
0,421
0.669
Acc
0.094
(1080
0.254
01222
0.103
0,063
0.1199
Comp
0.174
0,212
0.057
0.030
0.044
0.171
0.044
57851 1c8a9
Pree
0.373
0.354
0.378
0,520
0.322
0.608
Recal
0.328
0.271
0.599
0.737
0.623
0.264
0.657
F-score
0.349
0.307
0.463
0.570
0.567
0,290
0.631
Acc
0.080
0.097
0.414
0.420
0.155
0.094
0.1
Comp
0.055
0.075
0.028
0.030
0.022
764d173c9k
Pree
0.503
0.457
0.437
0.457
0.657
0.704
0.708
Recal
0.496
0.436
0.771
0.799
0.732
0.713
0.813
F-score
0,500
0.446
0.557
0,582
0.693
0.709
0.757
Acc
0.170
0.085
0.070
0.050
Comp
0.116
0.285
0.076
0,242
0.056
0.105
0.080
0911414fb
Prec
0,348
0.347
0.289
0).755
0.545
0.418
0.651
Recal
0.331
0,256
0,470
0.403
0.530
0.363
0.589
R-score
0.294
0.537
0.389
0.618
Acc
0.073
0.066
0.246
0.184
0.086
0.061
0.055
Comp
0.086
0.122
0,049
0,063
0.039
0,085
0.040
Mean
Pree
0.468
0..43
0.420
0.490
0.606
0.455
0.620
Recal
0.366
0.676
0.427
0.669
F-score
0.455
0.391
0.507
0.564
0.638
0.439
0.642
Table 5: ScanNet++ Benchmark We established benchmarks for eight different scenes within the
ScanNet++ dataset.
mitigates the optimization issues encountered with Neuralangelo in indoor scenes. Additionally, we eliminated the unnecessary 360 unbound setting and background modeling for indoor scenes. Therefore, in the context of the Tanks and Temples advanced subset and ScanNet++, we report the results obtained with Bakedangelo as a substitute for those of Neuralangelo.
G.3 Evaluation Metrics
Mesh is extracted through marching cube algorithm with a resolution of 2048 applied across all scenes. For the Tanks and Temples dataset, the evaluation metries the training subset are computed using the official Python script provided by the dataset's maintainers ! Meanwhile, for the four scenes from the advanced subset, reconstructed results submitted to the evaluation server which calculates the evaluation metrics. For the ScanNet++ dataset, we calculate the F-score with a threshold of 0.025 compare the resultant meshes with the ground truth mesh, which is derived from the point cloud captured by laser scanner.
http:///tttttiiioooo///////aaaaa htnps:/www..aansssddeeneeee...o
toolbox/evaluation
19

---

G.4 Implementation Details
NeuRodin utilizes a multi-resolltion hash grid for encoding, spanning from 2 to 2 across 16 levels. Each hash entry possesses a channel size of 2 for room-level scenes such as ScanNet++, which is adjusted to 8 for larger-scale scenes like Tanks and Temples. The maximum number of hash entries for each resolution is set 2 We incorporate per-image appearance encoding the style of NeRF-W [15] while employing proposal network [1] based on a compact hash grid. For the outdoor scene, we model the background using an additional network [45] with the hash grid. For the ScanNet++ datasets, we sample 512 pixels per iteration. In the case of the Tanks and Temples dataset, we sample 1024 pixels during the first stage and escalate to 8192 pixels for the second stage. We set the weights Asi and Asmoath to be 0.01 and 0.005, respectively. For outdoor scenes, we set Abias to 0.1. For indoor scenes, we progressively increase Anias from 0.001 to 0.05 over the first 10,000 iterations through an exponential adjustment. Bakedangelo samples 8192 pixels per iteration in large indoor scenes of Tanks and Temples, aligning with the settings employed by Neuralangelo. In the context of room-level scenarios within ScanNet++, the batch size is adjusted to 1024 pixels per iteration. Our implementation of the method employs PyTorch [20] and utilizes the Adam optimizer, with learning rate of 0.001 applied to both the hash grid and the network, alongside weight decay set at 0.01. For the background model, we set the learning rate to 0.01. The total training steps amount to 300k, with the learning rate for the foreground model being decayed by factor of 10 at 160k and 240k steps. For the background model, we employ an exponential schedule for the learning rate, reducing it to 0.0001. For the proposal network, the learning rate is decayed by a factor of 3 at steps 150k, 225k, and 270k. All our experiments were conducted on an A100 40G GPU. We roughly require 7 GPU hours complete the reconstruction of an indoor scene from the ScanNet++ dataset. For large-scale scenes, the reconstruction takes approximately 18 GPU hours.
G.5 Implementation of the Explicit Bias Correction
In the actual implementation of explicit bias correction, when inferring the SDF at Ebins after the point r(t") along the ray in Equation (8), we also infer the SDF Chis mask beyond r(t") along the ray as mask. If f(r(t* + (biss mask)) is less than zero, we do not apply the bias loss to this particular ray. We have found that this approach effectively prevents incorrect alignments that may be caused by our approximate estimation of t*. For outdoor scenes, we simply determine whether ray is cast towards the background by checking if there exists a negative value of SDF at any sampled point along the ray. If so, we similarly refrain from performing bias correction. We have configured ébins mask to 0.001 for large-scale scenes, such the Tanks and Temples dataset. For room-level scenes, such as the ScanNet++ dataset, we have set it to 0.01.
G.6 Implementation Details of the Two-Stage Optimization
The local scale modeling in Equa- tion (5) can impede the model's convergence to the surface to some extent. Therefore, we manu- ally adjust the lower bound Sccarse for the scale prevent the ambi- guity that arises from excessively small scales. Analogous the manual adjustments made in the first phase, our objective in this stage is to facilitate convergence from volume rendering to surface rendering, thereby aligning the implicit surface with volume rendering completely. To achieve this, we exponentially increase the lower bound of the scale 8ime to substantial value. In all experiments, we set the value of scarse to 100 and Stine to 3000.
Metrie
F-Score (%)
Scene
VoIDDF
MonoSDF
Neuralangelo
Neuralangelo
Ours
Auditorium
3.16
10.97
14.32
14.09
16.03
Ballroom
11.61
29.30
32.21
28.93
33.10
Courtroom
7.71
21.58
36,53
32.81
33.53
Museum
4.41
21.71
25.49
29.28
32.71
Mean
6.72
20.89
27.14
26.28
28.84
Table 6: Quantitative evaluation of our method versus prior work on the Tanks and Temples advance subset. The best performance and the second-best outcomes are highlighted for easy reference.
20

---

69
97
106
114
122
NeRF
1.90
1.60
1.85
0.58
2.28
1.27
1,47
1.67
2.05
1.07
0.88
2.53
1.06
1.15
0.96
1.49
VoISDF
1.14
1.26
0.81
0.49
1.25
0.70
0.72
1.29
1.18
0.70
0.66
1.08
0.42
0.61
0.55
0.86
NeuS
1.00
1.37
0.93
0.43
1.10
0.65
0.57
1.48
1,09
0.83
0.52
1.20
0.35
0.49
0.54
0.84
CD
Neuralangelo
0.37
0.72
0.35
0.35
0.87
0.54
0.53
1.29
0.97
0.73
0.47
0.74
0.32
0.41
0.43
0.61
Table 7: Quantitative results on DTU Benchmark.
MonoSDD
Neuralangelo
Ours
GT Mesh
Figure 15: Quantitative evaluation of method the ScanNet++ dataset.
H
More Experimental Results
H.1
Experimental Results the Scannet++ Benchmark
For every scene, we use high-quality DSLR camera images from all frames for our experiments. For methods that not use prior knowledge, we downsample the images from 1752 1168 to 876 x 584 for training. For methods that do use prior knowledge, we do something similar to MonoSDF. We first crop the images to 1152 x 1152, then downsample them to 384 x 384 before feeding them into the Omnidata model [6] to predict geometry
First, scale the pose to be centered within a bounding sphere of radius equal to 1. Subsequently, we scale the camera pose by 0.8 to ensure that all scene boundaries algorithm to a cube with dimensions -1 to at a resolution of 2048. Then, we evaluate the metrics: Accuracy (Acc), Completeness (Comp), Precision (Pred), Recall (Rec), and F-score. The final result shown in Table 5. Here, we also present some visual results in Figure 15.
contained within the bounding sphere. We apply the marching cubes
H.2 Experimental Results on the DTU Benckmark
Although design is not specifically tailored for single-object datasets, validated method on the DTU Benchmark. The results are shown in the Table 7. We found that, even without special parameter tuning, method achieves results comparable to Neuralangelo and surpasses other baseline methods.
H.3 Experimental Results on the Tanks and Temples Advance Subset
In this section, present the individual results for each scene within the advanced subset of the Tanks and Temples dataset, as shown Table 6. Except for the Courtroom scene, our method significantly outperforms comparative approaches in the other three large-scale indoor scenes, demonstrating the effectiveness our approach. We present a portion of the mesh for the museum data in Figure 16 and more results in Figure 17, highlighting our method's significant capability in capturing details.
H.4 Comparison with TUVR
21

---

Since TUVR is not spen-souree and only reports metrics on the DTU dataset, we reproduced its unbiased SDF-to-densit technique and combined it with a hash grid to create the TUVR- Grid method for comparison. Additionally, we compared our results on the DTU dataset with the reported results in the paper that did not use MVS priors (TUVR-MLP). The results are shown in the table below.
F-score f
TUVR-Grid
Ours
Barn
0.57
0.70
Caterpillar
0.25
0.36
Courthouse
0.11
0.21
Meetingroom
0.03
0.43
Truck
0.33
0.47
Ignatius
0.66
0.86
Mcan
0.40
0.51
The performance of TUVR is not ideal on all datasets because its unbiased nature is not fully guaranteed, and it is also somewhat affected by over-regularization.
Table 8: Comparison with TUVR on the Tanks and Tem- ples training set.
24
37
40
55
63
65
69
83
97
105
106
110
114
118
122
Mean
TUVR-MLP
0.77
0.67
0.37
0.93
0.61
1.15
0.56
0.34
0.47
TUVR Grid
0.84
0.81
1.4
0.37
1.23
0.69
0.78
1.16
1.23
0.65
0.54
1.32
0.34
0.43
0.54
0.82
Ours
0.37
0.31
0.93
0.63
1.07
0.52
0.61
0.32
0.37
Table 9: Comparison with TUVR on the DTU Benchmark.
H.S
Image Reconstruction Comparison
While our design is primarily intended for surface reconstruction tasks, we have also compared the results of NeuS and Neuralangelo using the image quality evaluation methods from Neuralangelo, across different parameter scales.
As shown Table 10, with fewer parame- ters, our method (Ours-19) achieves image reconstruction quality close to that of Neu- ralangelo. With larger number of param- eters, our method (Ours-22) surpasses Neu- ralangelo in terms of image reconstruction quality.
PSNR
NeuS
Neuralangelo-22 27.24
Ours-19
Ours-22
Mean
24.58
26.90
27.67
Image Reconstruction Results on the Tanks and Temples training set.
Table 10:
H.6 More Ablation Study
We conducted additional ablation experiments on the Tanks and Temples training set and we included two more cases to verify the role of Eikonal loss in maintaining the natural zero level set during the first stage, as well as the impact of color condition on normals.
The five cases A: Without the
F-score
A
C
E
Full Model
local scale factor. B: Changing the stage 1 estimated gradient to the analytical gradient. C: With- out explicit bias correction. D: Without stage 1 Eikonal loss, but with color conditioning on the es- timated normal. E: Without stage Eikonal loss, but with color con- ditioning on the analytical normal.
Barn
0.70
0.58
0.68
0.54
0.56
0.70
Caterpillar
0.33
0.33
0.36
0.32
0.33
0.36
Courthouse
0.24
0.12
0.19
0.11
0.11
0.21
Meetingroom
0.37
0.29
0.38
0.35
0.21
0.43
Truck
0.46
0.39
0.47
0.42
0.43
0.47
Ignatius
0.81
0.79
0.86
0.77
0.78
0.86
Mean
0.49
0.42
0.49
0.42
0.40
0.51
Table 11: More ablation study Tanks and Temples training
set.
The quantitive results are in Ta-
ble 11. The results from cases A, B, and C demonstrate the effectiveness of the techniques we
proposed. The results from cases D and E indicate the necessity of applying Eikonal loss during the first stage.
I Limitation
Though NeuRodin has capabilities in reconstruction, it falls short in certain areas. Specifically, struggles to faithfully reconstruct the correct surface in areas that are textureless and less observed. NeuRodin is also incapable of handling situations with strong ambiguity. Additionally, density not guaranteed to be unbiased, hence, bias will always exist though the SDF-to-density conversion.
22

---

BEMBAL
Figure 16: Recovered mesh of the Museum data from the Tanks and Temples dataset.
Figure 17: More results from the Tanks and Temples dataset.
23

---

Lastly, the time taken for the reconstruction of large scenes is considerable, often requiring hours to complete. This could prove to inefficient in scenarios where time is of the essence.
Societal Impact
Our model achieves high-fidelity 3D reconstruction. The societal impact of this development is multifaceted. On one hand, enables significant advancements fields such as architecture and augmented reality, improving professional practices and potentially benefiting the public by enhancing the precision and interactivity of digital models. On the other hand, the increase computational power demands may lead to greater energy consumption, which poses environmental considerations. Moreover, there could be privacy concerns such technology is applied to reconstruct environments from personal data without consent. Overall, while this technology presents opportunities for progress and innovation, it also requires careful consideration of ethical and environmental implications.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the main claims made in the abstract and introduction accurately reflect the paper's 's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
25

---

Justification:
Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes)
Justification:
We present all implementation details in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
We have released the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
mublic/guides Conidddesooeeeuiissiiiiii... cereiisssoooooliiio Policylyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: //niss.. wtt.cc/ppbb.. wwtppcccpcc//ccccccccccccc.cclct gww.//gg//e///eeeeeeeecc httgg//de/ee/duubnntnnnnnn
onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the details of training and testing have been outlined in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
Completing the experiments required substantial utilization of graphics card
resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
27

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided sufficient information on the computer resources in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: h
Answer: [Yes]
Justification:
The research conducted in the paper conforms fully with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss potential negative societal impacts in our supplementary material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited all the datasets we have used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "Emo3zmb0cd", "reviewer": "Reviewer_o1gj", "summary": "This article introduces NeuRodin, a Signed Distance Function (SDF)-to-density-based neural surface reconstruction method. The author summarizes the two main factors, SDF-to-density representation and geometric regularization, which cause low-quality performance in SDF-based methods and improve them in the pipeline. They argue that the widely used global scale parameter may cause the identical density in the same SDF level set. The paper attempts to solve this problem by using a non-linear mapping to obtain this scale parameter and adding a bias of density.  For the geometric regularization, they mentioned some reasons such as the Eikonal loss and smoothness constraints that caused oversmoothness. Stochastic-step numerical gradient estimation and two-step training strategy are employed for over-regularization issue. The results overperform some baselines and ablation studies show the effectiveness of the modules.", "questions": "Is it possible to compute the proposed method under gird resolution of 2^22 just as nerualangelo did? This could be a fairer comparison in table 1.\nIt will be good to discuss the runtime compared with other baselines.", "limitations": "The authors only provide F-score for the evaluation metrics except for some single scenes in appendix. It will be good to provide more such as PSNR, SSIM, etc.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The mentioned factors theoretically may cause the low-quality performance. The pipeline is clear, and theories are reasonable. \nThe strategies and improvements enhance the performance of the model.\nThe shown depth maps examples also improved with the proposed strategies. This could be useful for other depth estimation tasks.", "weaknesses": "The authors only provide F-score for the evaluation metrics except for some single scenes in appendix. It will be good to provide more such as PSNR, SSIM, etc.\nThere are some minor mistakes in Table 5. The underline format in caption is not consistent with the italics format in the table.\nIs it possible to compute the proposed method under gird resolution of 2^22 just as nerualangelo did? This could be a fairer comparison in table 1.\nIt will be good to discuss the runtime compared with other baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback. Below, we address the concerns raised in the review.\n\n\n### **Image Reconstruction Comparision**\n> The authors only provide F-score for the evaluation metrics except for some single scenes in appendix. It will be good to provide more such as PSNR, SSIM, etc.\n\nWhile our design is primarily intended for surface reconstruction tasks, we have also compared the results of NeuS and Neuralangelo using the image quality evaluation methods from Neuralangelo, across different parameter scales. \n\n| Methods | NeuS |  Neuralangelo-22 | Ours-19 | Ours-22 |\n|:-------:|:----:|:----:|:-------:|:-------:|\n| PSNR    |   24.58   |   27.24   | 26.90   | 27.67   |\n\nAs shown, with fewer parameters, our method (Ours-19) achieves image reconstruction quality close to that of Neuralangelo. With a larger number of parameters, our method (Ours-22) surpasses Neuralangelo in terms of image reconstruction quality.\n\n### **Hash Grid with a Dictionary size of $2^{22}$**\n>  Is it possible to compute the proposed method under gird resolution of 2^22 just as nerualangelo did?\n\nWe also tested our method on the Tanks and Temples dataset with a hash dictionary size of 2^22 as shown in the table below. \n\n| Methods | Neuralangelo-19 | Neuralangelo-22 | Ours-19 | Ours-22 |\n|:-------:|:---------------:|:---------------:|:-------:|:-------:|\n| F-score    |   0.43   |   0.50   |         0.51        |  0.50  | \n\nWe found that increasing the number of parameters did not significantly improve the results but instead dramatically increased the running time (from 18 hours to 48 hours). The redundant parameters (with a dictionary size of $2^22$ resulting in over a hundred million parameters) did not enhance surface accuracy but did help fit the training images better (PSNR from 26.90 to 27.67). Neuralangelo addresses some surface defects through over-parameterization. However, we achieve the same effect with significantly fewer parameters, thanks to our specialized design.\n\nFurthermore, we test with a hash dictionary size of $2^{22}$ in a large-scale scenario from BlendedMVS dataset. By significantly increasing the number of parameters, the model is able to capture fine-grain details on images, thereby preserving more details in the reconstructed surface. In our PDF file related to the global response, Figure 1 displays the quantitative comparison.\n\n### **Training time**\n> It will be good to discuss the runtime compared with other baselines.\n\nAs mentioned in lines 504-506 of the original text, on the Tanks and Temples dataset, Neuralangelo requires nearly 48 GPU hours for optimization, whereas our method only requires approximately 18 GPU hours. This is due to our specialized design for addressing surface defects, which allows us to maintain performance without requiring as many parameters as Neuralangelo.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "beZJsjwiAu", "reviewer": "Reviewer_9bjU", "summary": "This paper introduces an innovative two-stage framework, NeuRodin, for neural surface reconstruction that significantly improves upon previous SDF-based methods. Locally adaptive parameters for SDF-to-density conversion and a novel explicit bias correction are introduced to enhance the fine reconstruction of SDF surfaces.", "questions": "Refer to the weaknesses section.", "limitations": "The authors discuss the limitations of the proposed model.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper thoroughly elaborates on the shortcomings of current SDF-based approaches and provides insightful and impactful plug-and-play solutions, such as SDF-to-density conversion and explicit bias correction.\n2. The paper is clearly written with detailed explanations of the methodology, the challenges addressed, and the solutions proposed.\n3. Strong experimental results demonstrate that the proposed framework can generate high-quality surface reconstruction results.", "weaknesses": "1. The two-stage optimization process proposed in the paper seems somewhat cumbersome, and finding a balance between over-regularization and fine reconstruction is challenging. This detailed design somewhat weakens the depth of discussion on the essential issues of SDF reconstruction. Easily portable modification modules, such as explicit bias correction and SDF-to-density conversion, could have a more far-reaching impact.\n2. The difference between the ideal situation discussed by the authors in Explicit Bias Correction and the SDF distribution in actual training seems to have a similar mechanism to the convergence difficulty of shape adjustment discussed in [1]. Is it possible to avoid the biased situation shown in Figure 3 by introducing depth or other supervision information?\n3. In the experimental section, Table 2 does not explain the measurement indicator of the data. Combined with Appendix 5, it can be inferred that the indicator is the percentage F-Score.\n4. There is a lack of comparison of image reconstruction results. Since color is used in the training process and the baseline method in the paper includes image reconstruction comparisons, why does this paper not provide qualitative or quantitative results of image reconstruction?\n\n[1] Yang, Huizong, et al. \"Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation.\" *Advances in Neural Information Processing Systems* 37 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback. The issues we mentioned regarding SDF-based rendering, such as the bias problem, might indeed be addressed through improved SDF-to-density modeling. While it is possible to tackle these issues directly through mathematical modeling, we believe that our detailed design can be very easily implemented within the existing framework and effectively and significantly improve surface quality. This also provides considerable value.\n\n### **Impact of Additional Signal Supervision**\n> Is it possible to avoid the biased situation shown in Figure 3 by introducing depth or other supervision information?\n\nWe tested monocular depth and normal supervision on an indoor scene from ScanNet++ to examine the density bias. We found that the density bias no longer caused significant surface errors. This aligns with intuition, as the impact of density bias depends on the convergence of the surface. Any SDF-to-density design ensures that when the scale factor approaches zero, the weights along the ray can approximate a Dirac distribution at the SDF zero-crossing points.\n\nWe believe that the bias issue is caused by photometric ambiguity, meaning that the surface cannot be accurately determined based on color alone. This is consistent with the outlier distribution in MVS methods. From the MVS perspective, the scattered distribution in low-texture areas of MVS results is similar to the weight distribution along the ray and is considered outliers and removed. However, our goal is dense surface reconstruction, and we need to uniquely determine the surface within this scattered distribution. Therefore, we select the part with the highest weight as the most reliable surface.\n\nWhen we have more reliable supervision signals, such as monocular depth and normals, these outliers decrease because the model can use these additional signals to accurately determine the surface.\n\n### **Image Reconstruction Comparision**\n> There is a lack of comparison of image reconstruction results. Since color is used in the training process and the baseline method in the paper includes image reconstruction comparisons, why does this paper not provide qualitative or quantitative results of image reconstruction?\n\nWhile our design is primarily intended for surface reconstruction tasks, we have also compared the results of NeuS and Neuralangelo using the image quality evaluation methods from Neuralangelo, across different parameter scales. \n\n| Methods | NeuS |  Neuralangelo-22 | Ours-19 | Ours-22 |\n|:-------:|:----:|:----:|:-------:|:-------:|\n| PSNR    |   24.58   |   27.24   | 26.90   | 27.67   |\n\nAs shown, with fewer parameters, our method (Ours-19) achieves image reconstruction quality close to that of Neuralangelo. With a larger number of parameters, our method (Ours-22) surpasses Neuralangelo in terms of image reconstruction quality.\n\n### **Addressing Typographical Errors and Omissions**\n> In the experimental section, Table 2 does not explain the measurement indicator of the data. Combined with Appendix 5, it can be inferred that the indicator is the percentage F-Score.\n\nThank you for your valuable feedback and for pointing out the typographical errors and missing elements in our manuscript. We sincerely apologize for these mistakes. We will carefully revise the manuscript to correct these issues and ensure that it meets the highest standards of clarity and accuracy.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jDvn3pOjOz", "reviewer": "Reviewer_JYLp", "summary": "The paper improved NeRF based surface reconstruction from two aspects: adaptive scale $s$ and a bias correction loss to reduce the bias, which encourages the SDF becomes negative after the maximum at $t^*$.", "questions": "While training $\\mathcal{L}_{bias}$, how to evaluate $t^*$? Perhaps, it is an iterative procedure to obtain the maximum value, but can you embed the iterations into training?\n\nHow to avoid the maximum $t^*$ behind the zero iso-surface?\n\nWhy only use F-score for validation? What about other scores, such as Chamfer distance.\n\nNeRF based training is slow, but the training times are not reported in the paper. Does it become slower due to the bias correction?", "limitations": "In the limitation section, it needs to more details about reconstruction of thin structures, sharp features, etc.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The position $\\mathbf{r}(t)$ based scale $s$ increases the degree of freedom, which has the potential to improve the accuracy.\n\nThe bias correction loss encourages the SDF to be negative after the maximum weight point.", "weaknesses": "The bias correction looks like partially. It doesn't punish negative SDF before $t^*$, so it is not completed.\n\nThe experiments is not comprehensive. TUVR is not compared, which also aims to reduce the bias.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback. We hope the following response can address your concerns.\n\n### **Design of Explicit Bias Correction**\n> The bias correction looks like partially. It doesn't punish negative SDF before $t^*$, so it is not completed.\n\nAs you mentioned, although aligning $t^*$ with the SDF zero-crossing is intuitive, our experiments showed that this approach requires special design considerations. We tried simply constraining $f(t^\\*)$ to approach zero but found that visible surface defects persisted. We also attempted to constrain the SDF values both before and after $t^\\*$, which led to very strange optimized surfaces, likely due to overly strong constraints on the SDF. We have shown the visual result in Figure 3 from our PDF file related to the global response.\n\nThe key to our method's design lies in correcting large relative bias with explicit loss functions, while smaller biases are corrected by gradually scheduling the lower bound of the scale factor. We empirically found that visible surface errors are always caused by $t^\\*$ is being before SDF zero-crossing points $t^0$. Therefore, we penalize the SDF value just after $t^\\*$ to trend towards negative values, which prevents $t^*$ from being before $t^0$. For the case where $t^\\*$ is after $t^0$, we can directly address it by gradually scheduling the lower bound of the scale factor, as the bias in this situation is relatively small.\n\nWe provided a mathematical explanation under the assumption of a sufficiently small local surface. In the VolSDF framework we used, In this case, if the angle between the ray and the plane is θ, and the distance from the ray’s origin to the plane is $d$, then the distribution of the SDF along the ray can be expressed as $f(r(t)) = −t\\sin θ + d$. By setting the partial derivative of the rendering weight with respect to $t$ to zero, we can directly obtain a closed-form solution for $t^\\*$:\n\n$$\nt^*=\\frac{s\\ln (k) + d}{\\sin\\theta},\n$$\n\nwhere $k=2\\sin\\theta$ if $\\sin \\theta \\leq 0.5$, and $k=(2+\\sin\\theta-\\sqrt{\\sin^2\\theta+4\\sin\\theta})^{-1}$ if $\\sin \\theta > 0.5$. \n\nIn this case, the closed-form solution for $t^0$ can be directly obtained as $t^0 = d/\\sin\\theta$. Then the distance between $t^*$ and $t^0$ is\n\n$$\n t^{\\Delta} = t^* - t^0 = \\frac{\\ln k }{\\sin\\theta}s.\n$$\n\nWhen $t^\\Delta < 0$, $t^\\*$ is before $t^0$, and when $t^\\Delta > 0$, $t^\\*$ is after $t^0$. In our PDF file related to the global response, we visualized the values of $t^\\Delta$ under different $\\theta$ in Figure 2. We found that when $t^\\*$ is before $t^0$ ($\\sin\\theta$ less than 0.5), the relative bias is significantly greater than when $t^\\*$ is after $t^0$. This aligns with our experimental findings, where visible erroneous surfaces are primarily caused by $t^*$ preceding $t^0$. Therefore, we penalize the SDF value just after $t^\\*$ to prevent $t^\\*$ being before $t^0$.\n\nWe will include the complete derivation in the appendix.\n\n\n### **Comparison with TUVR**\n> TUVR is not compared, which also aims to reduce the bias.\n\nSince TUVR is not open-source and only reports metrics on the DTU dataset, we reproduced its unbiased SDF-to-density technique and combined it with a hash grid to create the TUVR-Grid method for comparison. Additionally, we compared our results on the DTU dataset with the reported results in the paper that did not use MVS priors (TUVR-MLP). The results are shown in the table below.\n\nResults on Tanks and Temples dataset:\n| Methods |  TUVR-Grid | Ours |\n|:---:|:---:|:--:|\n| F-score | 0.33 | 0.51 |\n\nResults on DTU dataset:\n| Methods |  TUVR-Grid | TUVR-MLP | Ours |\n|:---:|:---:|:--:|:--:|\n| CD↓ | 0.89 | 0.71 | 0.60 |\n\nThe performance of TUVR is not ideal on all datasets because its unbiased nature is not fully guaranteed, and it is also somewhat affected by over-regularization.\n\n### **Evaluation of $t^*$**\n> how to evaluate $t^*$?\n\nIn lines 197-198 of the paper, we mentioned that we directly use the sampled point with the highest weight as the point where $t^\\*$ is located. This is because we perform importance sampling along the ray before rendering, and the sampled points are already concentrated in areas with higher weights. Therefore, there is no need for additional iterations to find $t^*$. Experimentally, this approach does not significantly impact the results.\n\n### **Other Evaluation Metrics**\n> Why only use F-score for validation? What about other scores, such as Chamfer distance.\n\nOn the Tanks and Temples benchmark, previous work only reported the F-score as a surface quality metric. We compared our F-score results with the reported F-scores from other methods, although we also evaluated additional metrics using our method. However, these additional metrics cannot be directly compared with other methods.\n\nAdditionally, we compared the PSNR metric for image reconstruction on the Tanks and Temples dataset and the Chamfer distance metric on the DTU dataset. The results are shown in the table below:\n\n| Methods | NeuS |  Neuralangelo-22 | Ours-19 | Ours-22 |\n|:-------:|:----:|:----:|:-------:|:-------:|\n| PSNR    |   24.58   |   27.24   | 26.90   | 27.67   |\n\n| Methods | NeuS |  TUVR-Grid | TUVR-MLP | Neuralangelo | Ours |\n|:---:|:---:|:---:|:--:|:--:|:--:|\n| CD↓ | 0.84 | 0.82 | 0.71 | 0.61 | 0.60 |\n\n### **Impact of Explicit Bias Correction on Training Time**\n> Does it become slower due to the bias correction?\n\nAfter incorporating the explicit unbias correction design, the time per iteration increased from approximately 59ms to 63ms. \u001dAfter 300,000 iteration, this results in an additional 20 minutes of training time. This is primarily because we need additional network inference to compute the SDF slightly beyond $t^*$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i7mwBiwsB1", "reviewer": "Reviewer_EWQD", "summary": "This paper focuses on improving SDF-based volume rendering and proposes a new pipeline to address issues stemming from SDF-to-density conversion and geometric regularization. First, it changes a global scale parameter to local adaptive value, allowing more flexible density values to be converted from SDF. Second, the method proposes a novel loss function to align the maximum probability distance in volume rendering with the zero-level set of the SDF representation. Third, the paper claims that SDF regularization may be too strong to allow flexible topological changes and thus proposes a two-stage training process. The coarse stage operates similarly to a density field without strong constraints, while the refinement stage then encourages enhanced smoothness.", "questions": "1. In Equation (8), why does it only penalize a single point instead of a region or multiple points (both SDF > 0 and SDF < 0)?\n\n2. Line 273: What does \"prior knowledge in terms of SDF\" refer to?\n\n3. Why would optimizing the color conditioned on the normal restrict topological change?", "limitations": "Yes, it's mentioned in the supplementary material.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed improvements are well-motivated and technically sound in general.\n\n2. The paper is well-written and easy to follow overall. It's great to have Figure 3 for illustration.\n\n3. The paper appropriately mentions related work that shares a similar idea and the problems of prior solutions.\n\n4. The authors compare the proposed method on two datasets and demonstrate its superiority over several baseline methods. They also provide qualitative examples for the ablation study.", "weaknesses": "1. It would be more convincing to include quantitative results and more qualitative examples for verifying the effectiveness of the proposed components in the ablation study. Currently, the ablation study is only conducted on a single example scene qualitatively.\n\n2. The idea of using an adaptive scale is not quite new, as it is also seen in previous works [1]. It would be better to discuss the differences with existing works sharing a similar idea. Also, it would be better to provide more motivating examples or analysis on why an adaptive scale is important, such as when to use a large scale and when to use a small scale.\n\n3. For the two-stage training, the motivation is to make the coarse stage more like a density field without strong constraints. The paper claims that \"eliminating or downweighting any geometric constraints often results in an unnatural zero-level set.\" However, this point is not verified in experiments. It is also strongly recommended to add an ablation variant that does not have the eikonal loss (using estimated gradient) in the coarse stage.\n\n4. It is common to use numerical gradients in calculating the eikonal loss. The benefit of sampling a step size should be analyzed and compared more thoroughly in the ablation studies.\n\n5. While the variance of stochastic gradients is understandable, it is unclear why this ensures stability for large features and flexibility for complex details. More explanation and analysis would be helpful.\n\n6. The proposed pipeline is built upon many techniques from TUVR. It would be better to directly compare with TUVR in the experiments.\n\n7. It would be beneficial to show results on the commonly used DTU dataset as well.\n\n8. The paper needs more careful proofreading and polishing:\n\n (a) Line 25: \"fails to intricate geometric details\"\n\n (b) Line 28: \"is produced by is by\"\n\n (c) Equation (11), the symbol 'n' is not explained.\n\n (d) Equation (8), the symbol 'd' is not explained.\n\n[1] Wang, Zian, Tianchang Shen, Merlin Nimier-David, Nicholas Sharp, Jun Gao, Alexander Keller, Sanja Fidler, Thomas Müller, and Zan Gojcic. \"Adaptive shells for efficient neural radiance field rendering.\" arXiv preprint arXiv:2311.10091 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your insightful feedback. For more comprehensive ablation study, comparison with TUVR, results on DTU and more explanation of stochastic gradients, please refer to our global response. Apologies for the brevity due to word constraints.\n\n### **Motivation of Local Scale Factor**\n> Why an adaptive scale is important.\n\nWe further illustrate the importance of the local scale in Figure 3 from our PDF file related to the global response, using a simple single plane scenario for explanation. This plane has a low-texture region on the left and a rich-texture region on the right. \n\nWithout special constraints, the rendering weight should converge to a Dirac delta function at the surface in the richly textured region and form a scattered distribution in the weakly textured region.\n\nHowever, under the assumption of a global scale factor, all areas on the plane follow the same distribution which will be derived in the following sections. This means that both richly textured and weakly textured regions share the same density bias, preventing the surface from converging correctly to the richly textured surface with higher certainty.\n\nBy introducing the local scale factor, the most significant difference is that the distribution of rendering weights along the ray is no longer uniform. The network can adaptively converge in the richly textured regions, and the density bias in these areas is no longer affected by the low-texture regions.\n\nIn Adaptive shells, their focus is primarily on rendering quality and they did not evaluate surface reconstruction metrics. A straightforward application to surface reconstruction may lead to issues such as increased density bias. We tackle this by implementing special designs to ensure effectiveness, such as gradually scheduling the lower bound of the scale factor to correct relatively small biases and the explicit bias correction.\n\n### **Design of Explicit Bias Correction**\n> In Equation (8), why does it only penalize a single point instead of a region or multiple points (both SDF > 0 and SDF < 0)?\n\nAlthough aligning $t^*$ with the SDF zero-crossing is intuitive, our experiments showed that this approach requires special design considerations. We attempted to constrain the SDF values both before and after $t^\\*$, which led to very strange optimized surfaces, likely due to overly strong constraints on the SDF. We have shown the visual result in Figure 3 from our PDF file related to the global response.\n\nThe key to our method's design lies in correcting large relative bias with explicit loss functions, while smaller biases are corrected by gradually scheduling the lower bound of the scale factor. We empirically found that visible surface errors are always caused by $t^\\*$ is being before SDF zero-crossing points $t^0$. Therefore, we penalize the SDF value just after $t^\\*$ to trend towards negative values, which prevents $t^*$ from being before $t^0$. For the case where $t^\\*$ is after $t^0$, we can directly address it by gradually scheduling the lower bound of the scale factor, as the bias in this situation is relatively small.\n\nWe provided a mathematical explanation under the assumption of a sufficiently small local surface. In the VolSDF framework we used, In this case, if the angle between the ray and the plane is θ, and the distance from the ray’s origin to the plane is $d$, then the distribution of the SDF along the ray can be expressed as $f(r(t)) = −t\\sin θ + d$. By setting the partial derivative of the rendering weight with respect to $t$ to zero, we can directly obtain a closed-form solution for $t^\\*$:\n\n$$\nt^*=\\frac{s\\ln (k) + d}{\\sin\\theta},\n$$\n\nwhere $k=2\\sin\\theta$ if $\\sin \\theta \\leq 0.5$, and $k=(2+\\sin\\theta-\\sqrt{\\sin^2\\theta+4\\sin\\theta})^{-1}$ if $\\sin \\theta > 0.5$. \n\nIn this case, the closed-form solution for $t^0$ can be directly obtained as $t^0 = d/\\sin\\theta$. Then the distance between $t^*$ and $t^0$ is\n\n$$\n t^{\\Delta} = t^* - t^0 = \\frac{\\ln k }{\\sin\\theta}s.\n$$\n\nWhen $t^\\Delta < 0$, $t^\\*$ is before $t^0$, and when $t^\\Delta > 0$, $t^\\*$ is after $t^0$. In our PDF file related to the global response, we visualized the values of $t^\\Delta$ under different $\\theta$ in Figure 2. We found that when$t^\\*$ is before $t^0$ ($\\sin\\theta$ less than 0.5), the relative bias is significantly greater than when $t^\\*$ is after $t^0$. This aligns with our experimental findings, where visible erroneous surfaces are primarily caused by $t^*$ preceding $t^0$. Therefore, we penalize the SDF value just after $t^\\*$ to prevent $t^\\*$ being before $t^0$.\n\nWe will include the complete derivation in the appendix.\n\n### **Impact of Color Conditioning on Normal**\n> Why would optimizing the color conditioned on the normal restrict topological change?\n\nIn Figure 6 from our PDF file related to the global response, we experimentally observed that in indoor scenes, when color conditioning on normals is applied, the optimization process becomes very slow and affects the optimization results. However, this issue is nearly absent in outdoor scenes. We believe this is primarily due to the convergence behavior of the scale factor. Indoor scenes often have many low-texture areas, leading to larger scale factors and a greater scatter distribution. Additionally, color conditioned on normals, which is intended for geometry disentanglement as mentioned in IDR and only resonable for surface points, results in many details being optimized to incorrect positions before convergence is achieved. \n\nHowever, our use of stochastic normals helps mitigate these erroneous surfaces. For detailed regions, the estimated normals have greater variance, which alleviates the impact of incorrect surfaces. This is also demonstrated in our ablation experiments, as shown in the table below:\n\n| F-score | D | E  | Full |\n|:-:|:-:|:-:|:-:|\n| Courthouse (outdoor) | 0.11 | 0.11 | 0.21 |\n| Meetingroom (indoor) | 0.35 | 0.21 | 0.43 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We express our sincere gratitude to all the reviewers for their valuable insights and constructive feedback on our work. We truly appreciate your dedicated efforts and the time you have devoted to evaluating our work. 
Here we address some common concerns raised by reviewers. We also have provided a PDF for visualizing results or charts to assist in illustrating and explaining the issues. Please check.
### **Image Reconstruction Comparision**
While our design is primarily intended for surface reconstruction tasks, we have also compared the results of NeuS and Neuralangelo using the image quality evaluation methods from Neuralangelo, across different parameter scales. 

| Methods | NeuS |  Neuralangelo-22 | Ours-19 | Ours-22 |
|:-------:|:----:|:----:|:-------:|:-------:|
| PSNR    |   24.58   |   27.24   | 26.90   | 27.67   |

As shown, with fewer parameters, our method (Ours-19) achieves image reconstruction quality close to that of Neuralangelo. With a larger number of parameters, our method (Ours-22) surpasses Neuralangelo in terms of image reconstruction quality.


### **Comparison with TUVR**
Since TUVR is not open-source and only reports metrics on the DTU dataset, we reproduced its unbiased SDF-to-density technique and combined it with a hash grid to create the TUVR-Grid method for comparison. Additionally, we compared our results on the DTU dataset with the reported results in the paper that did not use MVS priors (TUVR-MLP). The results are shown in the table below.

Results on Tanks and Temples dataset:
| Methods |  TUVR-Grid | Ours |
|:-:|:-:|:-:|
| F-score | 0.33 | 0.51 |

Results on DTU dataset:
| Methods |  TUVR-Grid | TUVR-MLP | Ours |
|:-:|:-:|:-:|:-:|
| CD↓ | 0.89 | 0.71 | 0.60 |

The performance of TUVR is not ideal on all datasets because its unbiased nature is not fully guaranteed, and it is also somewhat affected by over-regularization.

### **Comparison on DTU Benchmark**
Although our design is not specifically tailored for single-object datasets, we validated our method on the DTU Benchmark. The results are shown in the table below. We found that, even without special parameter tuning, our method achieves results comparable to Neuralangelo and surpasses other baseline methods such as TUVR.

| Methods | NeuS |  TUVR-Grid | TUVR-MLP | Neuralangelo | Ours |
|:-:|:-:|:-:|:-:|:-:|:-:|
| CD↓ | 0.84 | 0.82 | 0.71 | 0.61 | 0.60 |

### **More Ablation Study**

We conducted additional ablation experiments on the Tanks and Temples training set and we included two more cases to verify the role of Eikonal loss in maintaining the natural zero level set during the first stage, as well as the impact of color condition on normals. 

The five cases are:

A: Without the local scale factor

B: Changing the stage 1 estimated gradient to the analytical gradient

C: Without explicit bias correction

D: Without stage 1 Eikonal loss, but with color conditioning on the estimated normal

E: Without stage 1 Eikonal loss, but with color conditioning on the analytical normal

The quantitive results are as follows:

| Case    | A    | B    | C |  D   | E    | Full Model |
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
| F-score | 0.49 | 0.42 | 0.49 | 0.42 | 0.40 | 0.51       |

The results from cases A, B, and C demonstrate the effectiveness of the techniques we proposed. The results from cases D and E indicate the necessity of applying Eikonal loss during the first stage.

Here, we present the results of removing the Eikonal loss during the first stage (cases D and E), which numerically demonstrate the necessity of this loss in the first stage. If removed, the following issues arise:

1. Noisy surfaces behind the correct surface.
2. Some large-scale areas display challenging-to-fill holes in the subsequent stage.
3. Under this poor SDF initialization in stage one, the optimization in stage 2 is not rubust.

### **Training time**
As mentioned in lines 504-506 of the original text, on the Tanks and Temples dataset, Neuralangelo requires nearly 48 GPU hours for optimization, whereas our method only requires approximately 18 GPU hours. This is due to our specialized design for addressing surface defects, which allows us to maintain performance without requiring as many parameters as Neuralangelo.

### **More Explanation of Stochastic Gradients**

As discussed in the main paper, our stochastic gradient estimation introduces some uncertainty into the true normals. For large-scale features, the Eikonal loss with varying step sizes can still be successfully minimized (since SDF near a large-scale plane should satisfy the Eikonal equation for different step sizes). In other words, the variance of stochastic gradients is small for large-scale features, making it easier for the model to minimize the Eikonal loss. However, for fine details, the random step sizes lead to high variance in the estimated normals, which reduces the impact of the Eikonal loss and makes it more challenging for the model to minimize samples with high variance.

Meta-review: This paper makes a clear incremental contribution to SDF volume rendering. The discussion with the authors and reviewers efficiently clarified the points raised in the reviews and an acceptance consensus was reached.
Decision: Accept (poster)