From Transparent to Opaque: Rethinking Neural Implicit Surfaces with $\alpha$-NeuS
Authors: Haoran Zhang
Junkai Deng
Xuhui Chen
Fei Hou
Wencheng Wang
Hong Qin
Chen Qian
Ying He
Venue: NeurIPS 2024 poster (2024)
Keywords: NeuS
Transparent Modeling
Primary area: machine_vision
TL;DR: We discover that NeuS is unbiased for not only opaque but also transparent models, and propose a method to reconstruct transparent and opaque models simultaneously from multi-view images.

Abstract: Traditional 3D shape reconstruction techniques from multi-view images, such as structure from motion and multi-view stereo, face challenges in reconstructing transparent objects. Recent advances in neural radiance fields and its variants primarily address opaque or transparent objects, encountering difficulties to reconstruct both transparent and opaque objects simultaneously. This paper introduces $\alpha$-NeuS$\textemdash$an extension of NeuS$\textemdash$that proves NeuS is unbiased for materials from fully transparent to fully opaque. We find that transparent and opaque surfaces align with the non-negative local minima and the zero iso-surface, respectively, in the learned distance field of NeuS. Traditional iso-surfacing extraction algorithms, such as marching cubes, which rely on fixed iso-values, are ill-suited for such data. We develop a method to extract the transparent and opaque surface simultaneously based on DCUDF. To validate our approach, we construct a benchmark that includes both real-world and synthetic scenes, demonstrating its practical utility and effectiveness. Our data and code are publicly available at https://github.com/728388808/alpha-NeuS.

From Transparent to Opaque: Rethinking Neural Implicit Surfaces with a-NeuS
Haoran Zhang! Junkai Deng"; Xuhui Chen Fei Hou Wencheng Wang
Hong Qin', Chen Qian Ying He
Key Laboratory of System Software (CAS) and State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences
University of Chinese Academy of Sciences
College of Computing and Data Science, Nanyang Technological University
* *Department of Computer Science, Stony Brook University *SenseTime Research
zhanghr&ios cn junkai006de.. ntw. edu. sg {chenxh, houfei, whn]@ios ac. qinâcs ceinccs.stonyble stonybrook..ee..o.o edu qianchen@senset com yheentu. edu. sg
Abstract
Traditional 3D shape reconstruction techniques from multi-view images, such as structure from motion and multi-view stereo, face challenges in reconstructing transparent objects. Recent advances in neural radiance fields and its variants primarily address opaque or transparent objects, encountering difficulties to recon- struct both transparent and opaque objects simultancously. This paper introduces o-NeuS-an extension of NeuS that proves NeuS unbiased materials from fully transparent to fully opaque. We find that transparent and opaque surfaces align with the non-negative local minima and the zero iso-surface, respectively, in the learned distance field of NeuS. Traditional iso-surfacing extraction algo- rithms, such as marching cubes, which rely on fixed iso-values, are ill-suited for such data. We develop method to extract the transparent and opaque sur- face simultancously based on DCUDF. To validate our approach, we construct benchmark that includes both worldw and synthetic scenes, demonstrating its practical utility and effcctiivneess. Our data and code are publicly available at httpp://gittie giggtbtt.bb..... ctt. ctt//////888/////laaaaaaaaaaa NeuS
Introduction
Surface reconstruction from multi-view images has been an important area of research for decades. Traditional methods such as structure from motion (SIM) [I] and multi-view stereo (MVS) 2 calibrate images and reconstruct 3D geometry based on color consistency. Recently, the emergence of Neural Radiance Fields (NeRF)
S has revolutionized the field, producing impressive results novel view synthesis results via volume rendering. Its implicit surface-basee variants, such as NeuS []. VoISDF (5), -e-NeuS ( and NeuS2 77]. further advance this field by reconstructing high-quality geometry and appearance through the learning of signed distance fields (SDFs). However, these NeRF family methods are limited reconstructing opaque surfaces. Reconstructing transparent surfaces presents greater challenge, with relatively few investigations to date.
Recently, some works dealt with the refraction and reflection effects in the transparent scenes. For example, ReNeuS [8] effectively reconstructs opaque objects within transparent materials, such as glass, by assuming known parameters for these materials. Similarly, NeuS-HSR [ separates
'Equal contributions. CCoresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

reflections from the glass to reconstruct objects within thin transparent objects. While these methods successfully reconstruct the opaque objects behind or within transparent materials, they do not extend to reconstructing the transparent objects themselves.
To address the challenges described above, we propose new method, called a-NeuS, for the simultaneous reconstruction of thin transparent objects and opaque objects, Given that transparent objects are thin, we can disregard refraction effects. A key observation our work is that transparent surfaces induce local extreme values in the learned distance field of NeuS 4 during neural volumetric rendering. NeuS is unbiased, i.e., the maximum volume rendering weight coincides with the object surface, for opaque surface [ We advance the theory of NeuS and prove that NeuS is unbiased for all transparent and opaque surfaces. Under various opacities, the unbiased surfaces are either the non-negative local minimum or the zero level set of the distance field learned by NeuS. Thus, we are able to extract the unbiased surface for transparent and opaque surface reconstruction simultaneously. However, precise values of these non-negative local minima are unknown beforehand and can vary spatially, they are unsuitable for extraction by conventional iso-surfacing algorithms, such as marching cubes [10], which require specified fixed vso-value. To effectively extract the target geometry for transparent objects, we take the absolute value of the distance fields, making the unbiased surfaces become the local minima of the absolute distance field. Based on DCUDF [Л we introduce an optimization method to simultaneously extract the unbiased surfaces of the transparent and opaque surfaces.
To validate our approach, we construct benchmark containing real-world scenes and 5 synthesized scenes. Experimental results show that ax-euS effectively reconstructs both transparent and opaque objects all tested scenarios. To summarize, our main contributions are as follows:
1. We prove that the density functions proposed in NeuS 4 unbiased across continuum of material opacities, from fully transparent to fully opaque, thereby completing the theoretical framework of NeuS.
2. We show that transparent and opaque surfaces correspond to the non-negative local minima and the zeros of the learned distance field of NeuS, respectively.
3. We present a method for simultaneously extracting the unbiased surfaces corresponding to the target geometry of transparent objects and opaque objects based on DCUDF (II) from mixed SDF and unsigned distance field (UDF).
4. We construct a benchmark comprised of 5 real-world scenes and 5 synthetic scenes for validating our method.
2 Related Works
2.1 3D reconstruction from multi-view images
Reconstructing 3D objects from multi-view 2D images has been a research interest for decades, with wide range of approaches having been proposed. Traditional model structure recovery methods try to understand the images and infer the structure of the model. Notable examples in this category are voxel based approaches [2-16] and point cloud based approaches including SIM [ and MVS [2].
Recently, with the advancement of machine learning, volume rendering based approaches have achieved high-fidelity reconstruction quality. Based on 3D Gaussian splatting [17], many model
reconstruction methods are proposed, e.g., SuGaR 18] and 2D Gaussian splatting 19 Another category of method for surface reconstruction is NeuS [4] and VoISDF (5), based on NeRF [3] In particular, NeuS has gathered special attention and has spawned multiple descendants like Geo- Neus (0 and HE-NeuS (6. There also several studies that focus on non-watertight model reconstruction also by extending NeuS, including NeUDF (21]. NeuralUDF [22], NeAT 23 and
2S-UDF (2). However, these works all assume that the object is opaque.
2.2 3D reconstruction of transparent objects
Reconstruction of transparent objects presents significant challenge due to the complex light paths 30 caused by refraction and reflection, which hinder multiview stereo from effectively solving this problem (3I]. Traditional methods (32)44] use additional devices assumptions to

---

Table 1: Comparison with related works for respective use cases.
Method
Opaque
Transparent
Refraction
Reflection
Note
NeuS
Yes
No
No
No
ReNeuS g
Yes
No
Yes
Yes
1,3
NeuS-HSS 9
Yes
No
No
Yes
1
TransPIR 25
N/A
Yes
Yes
Yes
2.5
Li. 2020
26
N/A
Yes
Yes
2.4.5
NeTO 27
N/A
Yes
Yes
No
2
RefNeuS 22
Yes
No
No
Yes
aSurf 29
Yes
Yes
No
No
o-NeuS (Ours)
Yes
Yes
No
No
Notes:
Opaque object inside transparent container.
Focuses pure glass objects.
Assumes known container geometry and homogeneous background lighting.
Assumes maximum two light bounces.
Not involving volume rendering.
reconstruct transparent objects. Li et al. [26] used deep learning to further improve the quality of reconstruction without additional inputs, but tend produce over-smoothing results. With the development of NeRF 3]. some works have explored how to use neural rendering to reconstruct transparent objects 25 27 35-371 for capturing more details. However, they all just focus on transparent objects neglecting opaque objects.
There also works attempting capture the correct geometry of opaque objects under the in- fluence reflection and refraction, which are primarily caused by transparent objects. NeRFReN (38) and Ref-NeuS 28 reconstruct models by considering reflections in NeRF pipeline. NeuS-HSR 9 uses similar idea to model opaque objects inside transparent objects by separating the reflection effect. ReNeuS g considers both reflection and refraction model the opaque object inside glass, but needs strong assumption. These methods focus on the reconstruction of opaque models while overcoming the interference of reflection and refraction.
But none of them can reconstruct both transparent and opaque objects. They all have their own assumptions conditions for reducing the effect of reflection refraction. It is non-trivia to combine the two tasks. Please refer to Table T for a comprehensive comparison on the use cases. A concurrent work [39 proposed NeRF-based efficient rendering method for non-opaque scenes with baked quadrature fields. Another concurrent work, aSurf (29), extends Plenoxels [40] for modeling both transparent objects and opaque objects, while ignoring the effect of reflection and refraction. Concurrently, the work NU-NeRF [41] tries to model both transparent and opaque objects while recovering refractions. In this paper, we propose new algorithm to reconstruct thin transparent objects and opaque objects uniformly based on NeuS 4
Method
3.1
Unbiased density mapping in NeuS across opacities
NeuS 4 utilizes signed distance fields for surface representation and introduces a density distribution induced by these SDFs, thereby enabling neural volume rendering coupled with SDF learning. NeuS 4 proved that for opaque objects, the mapping from SDF to density in NeuS is unbiased, ensuring that the reconstructed surface is first-order approximation of the learned SDF. In this section, we further establish that the density mapping proposed NeuS is indeed unbiased across a continuum of material opacities, from fully transparent to fully opaque. This verification completes the theoretical framework of NeuS.
A surface is considered unbiased if the rendering weights attain the local maxima on the surface. This is essential to minimize the discrepancy between the surface and the desired result. NeuS 4 assumes the surface is opaque and proved the zero iso-surface is unbiased. For transparent surface,

---

boier Vagu
100
a S 0.5
0.5 < a <
a a+1
Figure I: Conceptual illustration of the signed distances along ray (black horizontal line) through scene containing single object (represented by vertical line segment). (a) When the rendered opacity a is less than or equal to 0.5, both the front and back faces of the object coincide with each other, aligning the maximal weight with the local minimum of the distance field. (b) When exceeds 0.5 but less than I, the back face, which is not rendered, is separated from the front face. The further away the back face is, the more opaque the rendered front face is. The maximum weight in this case aligned with the position of zero distance values. (c) For fully opaque surface, the back face is infinitely away. The scene can therefore be considered the single ray- mn--llane intersection discussed by NeuS [4].
observed that NeuS can also produce local minimum distance on the transparent surface, which inspired us to explore the properties of these local minima. We have the following theorem.
Theorem 1. Assuming single ray-plane intersection f the rendered opacity S 0.5, the learned distance field reaches a local minimum which non-negative, and the corresponding color weight maximum aligns with the distance local minimum. Otherwise, the distance local minimum is smaller than zero, and the corresponding color weight maximum aligns with the zero iso-surface.
Please refer to Appendix for the details of the proof. We simply sketch the proof here. Assume a is the opacity and the ray starting from point in direction d is rf)) o + td with parameter t. The density function of NeuS and HF-NeuS
σ (r(1)) = max (-s (1 (s(f(r(t))) cos(
(I)
where s is a learnable parameter, (.) is the sigmoid function and 0 is the angle between the ray direction and the gradient of the distance field f. The max operation avoids negative after crossing the local minimum m of the distance field. Assume the distance between origin of the ray and the plane is dg. Then, the opacity
1- udg Tfee
m
a
Liee odg om
0
-- zo 0.5, since and do are relatively large. For the sake of brevity, If m = O, the opacity = 1- 2 we simply state 0.5 the watershed value in the theorem.
The derivative of the rendering weight w(t) is w((t) = T(t) [o'(r(t)) - or(((t)) If the local maximum of weight occurs at /= o''')((((())) = (r((())). Taking the density function of NeuS into the above equation, we have (r((I") 0. if if m 0. Thus, if m 0, the zero iso-surface attains the largest rendering weight. We can also deduce that w (1) > if m > O, and thus the rendering weight continuously increases until touching the minimum of the distance field. Therefore, the local minimum of the distance field attains the largest rendering weight. In case of m = O, the point where the distance reaches minimum also achieves zero distance value, so the rendering weight is maximum at the distance local minimum.
The theorem can be explained follows. As illustrated in Figure I if I 0.5, the distance field local minimum is non neg-negative and the unbiased surface coincides with the local minimum. 0.5 < < the local minimum is negative and the unbiased surface coincides with the front zero iso-surface. If a - 1, the local minimum approaching negative infinity and the back zero iso-surface approaching infinity. Along with the opacity increasing, the front and back faces separate gradually from overlap to infinity, so that the integral of densities increases to infinity gradually.
With this assumption, we focus on first-order unbiasedness.

---

(b)
(c)
Figure 2: Illustration of our mesh extraction procedure. (a) The orange line denotes the input model, where the dashed line is transparent and the solid line is opaque. The color map illustrates the distance field f. (b) The iso-curve (red) is extracted. (c) The iso-curve is mapped to the local minima of the absolute distance field fe.
0.04
0.00
-0.02
(a) 0.0
(b) 0.005
(c) f
(d) fa
Figure 3: Comparisons of projection on the mixed SDF and UDF f and the absolute field fa. The cutting plane draws the distance field. The white line indicates the iso-surface and the orange line indicates the 0.005 iso-surface. (a) The extracted () iso-surface which attains the opaque surface exactly, but the transparent surface disappears. (b) The extracted 0.005 iso-surfaces. (c) Direct mapping on the original f would result in the opaque surfaces shrinking. (d) In contrast, after taking the absolute, all unbiased surfaces are properly extracted.
Remark. In NeuS, is learnable parameter that gradually converges a large value over the course of training. A larger value sharpens the edges and faces in the reconstructed model, enhancing overall quality. However, practice, cannot increase indefinitely due to numerical computation constraints, such the number of sample points. This caps at relatively high but finite values, which allows non-zero distance values to influence color calculation along the ray. Colors are derived from the the weighted sum of sampled radiance at points along the rays, and different distance minimum values contributes achieving different levels of opacity.
We believe that the tendency of larger s, combined with the color loss and the actual situations with MLP and numerical caleulation will achieve balance that leads the best results.
3.2 Unbiased surface extraction
As mentioned Section 3.1 we aim to extract the unbiased surface from the learned distance field. The unbiased surface is either the local minimum or the zero iso-surface depending on whether the local minimum neg-negative. Thus, the distance field learned by NeuS is not UDF or an SDF. As illustrated in Figure when < 0.5, the distance field is similar to UDF whose values are positive on both side and the unbiased surface is the local minimum. Since the local minimum greater than equal to zero, the distance field is not strict UDF. However, for simplicity, we still call such
5

---

distance field a UDF. When a > 0.5, the distance field is an SDF whose values are positive in front of the surface and negative behind the surface. The unbiased surface is the zero iso-surface and the zero is not an extreme value. Hence, the unbiased surface is a mixed SDF and UDE, which cannot be extracted using the conventional iso-surface extraction methods, e.g., marching cubes [10] In Figure 3 a), the zero iso-surface cannot extract the transparent hemisphere.
To extract the unbiased surface from the mixed SDF and UDE, follow the idea of DCUDF
extract the unbiased surface, As illustrated in Figure 2 a), given the mixed distance field f learned by NeuS, we extract the mesh M of non zero level set with user-specific iso-value r (r m) by marching cubes [0]] (Figure b) and ()). .M encloses the intended unbiased surface as an envelop. As DCUDF [T]] we compute a covering map project M back the local minima. However, > 0.5, the unbiased surfaces are not the local minima, but the zero iso-surface. In Figure 2b), the values of f inside the opaque box are smaller than the values outside. If we project M to the local minima of f. the curve would shrink into the box. Figure ((c) shows an example that the opaque surface shrinks we project M to the local minima of f. Nevertheless, as shown in Figure and dd). if we convert into its absolute values denoted by f the non non-negative local minima and the zero suo-surface of f are both the local minima of fa. Thus, f a UDF whose local minima are the unbiased surfaces. We are able to map M to the local minima extract the unbiased surfaces. Following DCUDF [TI), which employs a two-stage optimization process, we first solve a mapping Ti to project M to the local minima of f [11:
min Σ (((((P)) + A PEEMU
Σ w(px) || AT (p:))l where P.EM
ATi(pi) x1(p)) (p(p))
Σ T1(p))
P.EN(())
the Laplacian of the projected point Fi (p:) and is the of triangle centroids of M. ((() drives the point projecting to the local minima of fa. (p:))) denotes the I-ring neighboring vertices of p; E .M and (p;)) is a weight adaptive to the area of the adjacent triangle faces of ps. The second term is a Laplacian constraint that prevents the mesh from folding and self self-intersecting during optimization.
DCUDF [IT] further calculates a mapping to refine тi (M) in the stage two, which further reduces the fitting error. 7/, denotes the normal of the i-th triangle face of (M), whose centroid is encouraged to move along the normal direction 7h by penalizing the tangential displacements so as to prevent mesh folding and self-intersecting. The loss function of the refinement stage is [T1]
min Σ ( +A2 PEEMUC
E (T2 (p:)) TI (p:)) Tt:
P:EC
where x is the vector cross product. After projection, the initial M shrinks to the unbiased surfaces as illustrated in Figure Since the surface may contain non-manifold structures, e.g., the intersection of the transparent and opaque surfaces in Figure 2 we do not apply the min-cut postprocessing as .. Hence, the unbiased surface two-layer mesh that coincides together in regions of m 2 (i.e., a < 0.5), and a single-layer mesh in regions of m < 0 (i.c., a >> 0.5).
4 Experiments
4.1 Experimental settings
Due to the absence of relevant datasets, we have prepared a dataset comprised of synthetic scenes and 5 real-world scenes. The synthetic data rendered using Blender. The real data are captured by ourselves and the camera are calibrated with the help ArUco calibration boards.
Datasets.
Baselines. We compare method with the original NeuS 4 and NeUDF 21 which learns UDF from multi-view images.
Implementation details. Our training structure is the same as NeuS. We also followed the recom- mended configuration for the synthetic datasct by the authors of NeuS, without changing the loss

---

Table 2: Quantitative evaluation (x10-3) on the synthetic dataset. "g2d" is the Chamfer distance from the ground truth mesh to the reconstructed mesh and "d2g" measures the reverse. "CD" denotes the average of "g2d" and "d2g". The best results are marked in bold.
NeuS 4
(iso
NeuS 4
(iso
0.005)
Ours
g2d
d2g
CD
g2d
d2g
CD
g2d
d2g
CD
Snowglobe
65.22
5,16
35.19
7.07
6,46
6.77
4.73
4.37
4.55
Case
39.60
8.18
23.89
6.23
8.80
7.51
5.52
7.66
6.59
Bottle
7.91
4.77
6.34
6.19
8.14
7.16
3.14
4.22
3.68
Jug
11.59
10.41
11.00
5.33
9.36
7.34
6.44
4.67
Jar 76.79
4.45
40.62
11.61
7.92
9.77
11.89
5.87
8.88
mean
40.22
6.60
23.41
7.29
8.14
7.71
5.63
5.71
5.67
functions or their respective weights. That is, we chose A1 1.0 for color loss and X2 0.1 for Eikonal loss. All our trainings are without mask.
To extract the unbiased surface through DCUDF [T]. we choose to use 0.005 as the threshold for synthetic scenes, and 0.002 or 0.005 for real-world scenes. We conducted our experiments using almost the same setting as DCUDF. DCUDF employs two-stage optimization process. We performed 300 epochs for step and 100 epochs for step 2 respectively, which is the default setting of DCUDF. We used the VectorAdam optimizer as suggested by DCUDF. We set the weights Àι 500 and A2 0.5, which are different from DCUDF default setting.
The training process of NeuS typically takes about 9.75 hours and DCUDF convergence only requires few minutes single NVIDIA A100 GPU.
4.2 Synthetic data
In this section we focus on the Synthetic Blender dataset, where each synthetic dataset comprises 100 training images from different viewpoints. We compared the reconstruction results with NeuS and compared the Chamfer Distance (CD) with NeuS on the threshold O and 0.005. We report the effectively reconstruct
Chamfer distance results in Table The results indicate that our approach unbiased surfaces, both transparent and opaque. Please refer to Figure 4 for a qualitative comparison.
In comparison to the vanilla NeuS 4 from which we extract zero iso-surface, large portions of transparent surfaces are absent from the reconstructed mesh due to the positive minimum distances. Consequently, the one-way Chamfer distance from the ground truth to the reconstructed mesh is considerably high. iillustaessssssraee the percentage of sample points on the ground truth models, whose distances are smaller than the threshold. The final rates reflect the completeness of the reconstructed models. is evident that our method all achieves 100% completeness, indicating the absence of unnecessary holes in models. In contrast, if extracting the iso-surface, there are approximately 20% to 40% holes remaining. In comparison to the vanilla NeuS from which extract 0.005 iso-surface, the extracted surface does not correspond the maximum color weight, leading to sub-optimal reconstructed-to- toitted-tt-ggruuee to-ground-truit one-way Chamfer distance. Our method preserves transparent surfaces while being unbiased, leading the best Chamfer distances across all data.
4.3 Real-world data
We also capture 5 real-world scenes for validation. Due the absence of ground-truth mesh data, qualitative comparisons are conducted with NeuS on real-world scenes. The results are presented in Figure 6 The visual results demonstrate that method exhibits good reconstruction quality for both transparent and opaque surfaces, even in complex lighting conditions in real-world scenes. In contrast, NeuS with zero iso-surfaces is unable to extract completed surface, resulting in artifacts.
4.4 Discussion
Choice NeuS. We use NeuS 4 as backbone for reconstruction, which learns mixed SDF and UDF. However, during the projection stage of surface extraction, use the absolute value
7

---

Reference
Normal Map
NeuS
4
(section view)
Ours (section view)
Snowglobe
Case
Bottle
Jug
Jar
Figure 4: Qualitative comparison synthetic data. Our method uses NeuS for distance field learning, and as shown in the normal maps, vanilla NeuS is in fact capable of reconstructing surfaces with transparency. The difference between method and NeuS is drastic because NeuS cannot extract transparent surfaces where the distance field local minima are larger than zero with marching cubes, but theory confirms and extends NeuS's learning ability, extracting both the non-negative local minima and the zero iso-surface.
100%
snowglobe
100%
caRC
1005
bottle
100%
jug
1005
75%
75%
759-
75%
755-
25%
25%
25%
25%
25%--
Figure 5: Percentage of sample points on the ground truth mesh that the distance the reconstructed mesh is lower than given values. Blue: Ours, Orange: Zero iso-surface.
of the distance field. The absolute distance field fa resembles UDF. While directly using UDF learning methods could avoid the distance field conversion process, we select NeuS rather than UDF learning methods because the the SDF learning method NeuS is simple, stable and robust, and also capable of reconstructing details. We further compare with a UDF learning method NeUDF [21) We notice that other UDF learning methods including NeuralUDF 22 and 2S- 24 both take advantage of the opaque surface assumption, introducing an indicator function ray truncation

---

Reference
Normal Map
NeuS 4
Ours
Toy box
Toy Toy-cylinder
Sunglasses
Magician-Pe plane
Magician-bol bbx
Figure 6: Qualitative comparisons on real-world data.
NeUDF 21
Ours
NeUDF 21
Ours
6.42
4.55
Figure 7: Comparisons with UDF-based reconstruction method NeUDF 21 world data. While NeUDF can successfully reconstruct transparent surfaces and interior structures, it fails to preserve details and has difficulties reconstructing intricate structure. The Chamfer distances (x 10-3) are shown below the synthetic Snowglobe. The Snowglobe is shown in section view.
on synthetic and real-
strategy respectively. This leaves NeUDF [1] the only method that is theoretically capable of rendering multiple layers of surfaces in single ray.
Figure 7 the comparisons with NeUDF (1]. Since the zero distance value of NeUDF would result in opaque surface, the minimum distances of transparent objects learned by NeUDF are also

---

Reference
Normal
Mesh
Figure 8: The reflections and refractions in the input multi-view images (left) can lead to ambiguities in distance field learning (middle), preventing our method from extracting the desired surface (right). This phenomenon is particularly pronounced in the reconstruction of cylinders spheres under complex lighting conditions.
positive. We use DCUDF to extract surface instead of the default MeshUDF 42 used by NeUDR because MeshUDF could only extract the zero iso-surface. The models reconstructed by NeUDF are over-smoothed and lack many intricate structures. Qualitative measurement of the synthetic model also shows that NeUDF results larger Chamfer distance. This is due to the volatile nature of UDF learning, requiring additional regularizers for successful convergence, often sacrificing the reconstruction fidelity.
Limitations. Although our method has been effectively validated on both synthetic and real- worldworld data, it cannot handle all use cases. Our method, together with oSurf [29]. allows for simultancous reconstruction opaque and transparent objects. Other works either focus on the reconstruction of opaque objects, or pure-glase objects with refraction and reflection under certain assumptions. However, our method not designed handle the cases with complex lighting conditions like heavy refraction reflection. As shown in Figure 8 when using NeuS 4 as the backbone, scenes with reflection and refraction may yield ambiguous distance fields, preventing the acquisition the ideal surface. For these situations, on the one hand, improving the lighting conditions to minimize the occurrence of refraction and reflection can be considered. our experiments, we used polarizer to reduce reflection and model only thin transparent objects that have as little refraction as possible. Meanwhile, is also possible to use existing reflection removal algorithms [43]. On the other hand, replacing the backbone with models like Ref-NeuS (28) ReNeuS [8] (which focus opaque object reconstruction but not the transparent object itself) could considered. This will be one of our future research directions.
Conclusion
Overall, a-NeuS presents a new perspective on NeuS. We proved the unbiasedness of NeuS for transparent objects and extended the capability of NeuS to transparent surface and opaque surface reconstruction by proposing a unified theoretical and practical framework. Based on DCUDE, we extract the unbiased transparent surface and opaque surface simultancously for model reconstruction. We established benchmark consisting of synthetic and 5 real world scenes for validation. Our experiments have demonstrated the effectiveness of our proposed method, and its practical potentials.
Acknowledgments and Disclosure of Funding
This work was supported in part by the National Key R&D Program China (2023YFB3002901), in part by the Basic Research Project of ISCAS Project of ISCAS (ISCAS-ZD-202401), in part by the Ministry of Education, Singapore, under its Academic Research Fund Grants MOF-TTEP20220-000 & RT19/22) and the RIE2020 Industry Alignment Iund-Inn Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s).
in part by the Major Research
10

---

References
[1] Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision Cambridge University Press, 2 edition, 2004, doi: 10.1017/CH097801111 11151111166.
[2] Yasutaka Furukawa and Carlos Hernández. Multi-View Stereo: A Tutorial. Foundations and Trends in Computer Graphics and Vision, 9(1-2)1-148, 2015. ISSN 1572-2740. doi: 10.1561/06000000552
[3] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan Jan-Michael Frahm, editors, Computer Vision ECCV 2020. pages 405-421, Cham, 2020. Springer International Publishing. ISBN 978-3-0030-58452-88 doi: 10.1007/ 978-3-030--88552-8222
[4] Peng Wang, Lingje Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-vies viet Reconstruction. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 27171 277183. Curran Associates, Inc., 2021.
[5] Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume Rendering of Neural Implicit Surfaces. In M. Ranzato, A. Beygelzimer. Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advunces in Neural Information Processing Systems, volume 34, pages 4805-4815. Curran Associates, Inc., 2021.
[6] Yiqun Wang, Ivan Skorokhodov, and Peter Wonka. HF-NeuS: Improved Surface Reconstruction Using High- Figh-Frre Details. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 1966-1978. Curran Associates, Inc., 2022.
[7] Yiming Wang, Qin Han, Marc Habermann, Kostas Danillidis, Christian Theobalt, and Lingjie Liu. NeuS2: Fast Learning Neural Implicit Surfaces for Multi- view Reconstruction. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). pages 3272-3283, 2023.
doi: 10.1109/CCCV51000.22033..
[8] Jinguang Tong, Sundaram Muthu, Fahira Afzal Maken, Chuong Nguyen, and Hongdong Li. Seeing Through the Glass: Neural 3D Reconstruction Object Inside a Transparent Container. In 2023 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12555-12564, 2023. doi: 10. N..109/CVVR52229..0 20.0320022.0
[9] Jiaxiong Qiu, Peng- Tao Jiang, Yifan Zhu, Ze- Xin Yin, Ming-Ming Cheng, and Bo Ren. Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflections. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10. (1010//CVVRS27992222.0...
2023. doi:
[10] William E. Lorensen and Harvey E. Cline. Marching cubes: A high resolution 3D surface construction algorithm. SIGGRAPH Computer Graphics, 1164))163-169, August 1987. ISSN 0097-8930. doi: 10.1145/37402.37422.
[11] Fei Hou, Xuhui Chen, Weneheng Wang, Hong Qin, and Ying He. Robust Zero Level-Set Extraction from Unsigned Distance Fields Based on Double Covering, ACM Transactions Graphics, 42(6), December 2023. ISSN 0730-0301. doi: 10. 44/36183144
[12] J. De Bonet and P. Viola. Roxels: responsibility weighted 3D volume reconstruction. In Proceedings of the Seventh IEEE International Conference on Computer Vision, volume 1, pages 418-425 vol.1, 1999, doi: 10. 1110//CCV.11999.91113
[13] A. Broadhurst, T.W. Drummond, and R. Cipolla. A probabilistic framework for space carving. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume 1, pages 388-393 vol.1, 2001. doi: 10. 109100/C
[14] Mengqi Ji, Jinzhi Zhang. Qionghai Dai, and Fang, SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi View Stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence. 43(1
2021. doi: 10. 109.T000.......20...
[15] Abhishek Kar, Christian Häne, and Jitendra Malik. Learning a Multi- View Stereo Machine. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates. Inc., 2017.
[16] Jiaming Sun, Yiming Xie, Linghao Chen, Xiaowei Zhou, and Hujun Bao. NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video. In 2021 IEEE/CVF Conference on Computer Vision doi: 10.11099CVPP44443..222....11
and Pattern Recognition (CVPR), pages 15593 15602, 2021.

---

[17] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trunsactions Graphicx, 42(4), July 2023. ISSN 0730-0301. doi: 10.1 143/3592433.
[18] Antoine Guédon and Vincent Lepetit. SuGaR: Surface- Alignedd Gaussian Splatting for Efficient Mesh Reconstruction and and Pattern Recognition (CVPR), pages 5354-5363, 2024.
Mesh Rendering. In 2024 IEEE/CVF Conference on Computer Vision doi: 10. 10.1099CCVPS5273..2200000
[19] Binbin Huang, Zehao Yu, Anpei Chen, Andrcas Geiger, and Shenghua Gao. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. In ACM SIGGRAPH 2024 Conference Papers, SIGGRAPH '24, New York, NY, USA, 2024. Association for Computing Machinery. doi: 10. 10.1 (e.11//4445199.366777
[20] Qiancheng Fu, Qingshan Xu, Yew Soon Ong, and Wenbing Tao. Geo-Neus: Geometry-Consistenn Neural Implicit Surfaces I Learning for Multi-view Reconstruction. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 3403-3416 Curran Associates, Inc., 2022.
[21] Yu-Tao Liu, Li Wang, Jie Yang. Weikai Chen, Xiaoxu Meng, Bo Yang, and Lin Gao. NeUDF: Leaning Neural Unsigned Distance Fields with Volume Rendering. In 2023 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 237-247, 2023.
doi: 10.1 109000VP52229.9002222 2001272229222000033
[22] Xiaoxiao Long, Cheng Lin, Lingjie Liu, Yuan Liu, Peng Wang. Christian Theobalt, Taku Komura, and Wenping Wang. ND:: Learning Unsigned Distance Fields for Multi View Reconstruction of Surfaces with Arbitrary Topologies. In 2023 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 20834-20843, 2023. doi: 10. 121109/CVPR222222223.000
[23] Xiaoxu Meng. Weikai Chen, and Bo Yang. NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi- View Images. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 248-258, 2023. doi: 10.1 RA111//CVR57729.22222.. 202.00000..
[24] Junkai Deng, Fci Hou, Xuhui Chen, Wencheng Wang, and Ying He. 2S-UDF: A Novel Two-stage UDF Learning Method for Robust Non- Non-watertight Model Reconstruction from Multi-view Images. In 2024 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 5084-5093, 2024. doi:
10.1 1009/CVPRS2733.222..0000
[25] Mingqi Shao, Chongkun Xia, Dongxu Duan, and Xueqian Wang. Polarimetric Inverse Rendering for Transparent Shapes Reconstruction. IEFE Transactions on Multimedia, pages 1-11, 2024. doi: 10./ TMMM.2024.3371799
[26] Zhenggin Li, Yu- Yung Yeh, and Manmohan Chandraker. Through the Looking Glass: Neural 3D Re- construction of Transparent Shapes. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1259 1259-1268, 2020. doi: 10. 100100//CVR4420002220000000
[27] Zongcheng Li. Xiaoxiao Long, Yusen Wang. Tuo Cao, Wenping Wang, Fei Luo, and Chunxia Xiao. NeTO:Neural Reconstruction Transparent Objects with Self-Occlusion Aware Refraction Traciin, In 2023 IEEF/CVF International Conference on Computer Vision (ICCV), pages 18501-18511, 2023. doi: 10. 001000/0C011110000222.0
[28] Wenhang Ge, Tao Hu, Haoyu Zhao, Shu Liu, and Ying-Cong Chen. Ref-NeuS: Saabiguity-Reduccee Neural Implicit Surface Learning for Multi- View Reconstruction with Reflection. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4228-4237, 2023. doi: 10. 109/ICCV5100..0023....
[29] Tianhao Wu, Hanxue Liang, Fangeheng Zhong, Gemot Riegler, Shimon Vainer, Jiankang Deng, and Cengiz Oztireli. aSurff: Implicit Surface Reconstruction for Semi- Semi--raasparent and Thin Objects with Decoupled Geometry and Opacity. In 2025 International Conference on 3D Vision (3DV), 2025.
[30] Amit Agrawal, Srikumar Ramalingam, Yuichi Taguchi, and Visesh Chari. A theory of multi-layer flat refractive geometry. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3346-3353, 2012.
doi: 10. 11010//V...0112266881007
[31] Xiao Hu, François Lauze, and Kim Steenstrup Pedersen. Refractive Pose Refinement: Generalising the Geometric Relation between Camera and Refractive Interface. International Journal of Computer Vision, (116))1448-1470 March 2023, ISSN 0920-5691. doi: 10.1007/s1 2212266--0366333
[32] K.N. Kutulakos and E. Steger. A theory of reffratiive and specular 3D shape by light giht-path triangulation. In Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, volume 2, pages 1448 1455 Vol. 2, 2005. doi: 10.1109/1CC/22005.226
12

---

[33] Bojian Wu, Yang Zhou, Yiming Qian. Minglun Cong, and Hui Huang. Full 3D reconstruction of transparent objects. ACM Transactions on Graphics, 37(4), July 2018. ISSN 0730-0301. doi: 10. 10.1145/3197517 3201286.
[34] Jiahui Lyu, Bojian Wu, Dani Lischinski, Daniel Cohen-On and Hui Huang. Differentiable refraction-tracce mesh reconstruction of transparent objects. ACM Transactions on Graphics, 39(6). November 2020.
ISSN 0730-0301. doi:
[35] Dongqing Wang. Tong Zhang, and Sabine Süsstrunk. NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 317-327, 2023, doi: 10.1 10.1109/CCCV1110022222....
[36] Weijian Deng, Dylan Campbell, Chunyi Sun, Shubham Kanitkar, Matthew Shaffer, and Stephen Gould. Ray Deformation Networks for Novel View Synthesis of Refractive Objects. In 2024 IEEE/CVF Winter Conference Applications of Computer Vision (WACV), pages 3106-3116, 2024. doi: 10.1 1 000009.
[37] Fangzhou Gao, Lianghao Zhang. Li Wang, Jiamin Cheng, and Jiawan Zhang. Transparent Object Recon- struction via Implicit Differentiable Refraction Rendering. In SIGGRAPH Asia 2023 Conference Papers, SA '23, New York, NY, USA, 2023, Association for Computing Machinery, ISBN 9798400703157, doi: 10. 10.1145/3610448.361 381236.
[38] Yuan-Chen Guo, Di Kang, Linchao Bao, He, and Song-Hai Zhang. NeRFReN: Neural Radiance Fields with Reflections. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18388-18397,, 2022. doi: 10.1 109/ 202...008
[39] Gopal Sharma, Daniel Rebain, Kwang Moo Yi, and Andrea Tagliasacchi. Volumetric Rendering with Baked Quadrature Fields. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol, editors, Computer Vision ECCV 2024, pages 275-292, Cham, 2025. Springer Nature Switzerland, ISBN 978-3-031--33036-8
doi: 00.10079978---031--306-----
[40] Sara Fridovich-Keil Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance Fields without Neural Networks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5491-5500, 2022. doi: 10.1 20WCCVVRR2288.2222000..5
[41] Jia-Mu Sun, Tong Wu, Ling-Qt Qi Yan, Lin Gao. NU-NeRF: Neural Reconstruction of Nested Transparent Objects with Uncontrolled Capture Environment. ACM Transactions on Graphics, 43(6), November 2024. ISSN 0730-0301. doi: 10. 10.1145/3687757
[42] Benoit Guillard, Federico Stella, and Pascal Fua. MeshUDF: Fast and Differentiable Meshing of Unsigned Distance Field Networks. In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision ECCV 2022, pages 576 576-592, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-20062-55 doi: 10. 1000/97 --07778---33--1 3-033-200022---
[43] Simon Niklaus, Xuaner Cecilia Zhang, Jonathan T. Barron, Neal Wadhwa, Rahul Garg, Feng Liu, and Tianfan Xue. Learned Dual- View Reflection Removal. 202/ JEEE Winter Conference Applications of Computer Vision (WACV), pages 1-221, 2021. doi: 10. 11100/CV48660..0000 202244083300.2000000 100076..
[44] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, Henrik Aanæs. Large Scale Multi-view Stercopsis Evaluation. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 406-413, 2014, doi: 10.1 109/CVPR.2014.99.
13

---

A Proof of Theorem 1
Theorem 1, Assuming single ray-plane intersection, if the rendered opacity S 0.5, the learned distance field reaches a local minimum which is non-negative, and the corresponding color weight maximum aligns with the distance local minimum. Otherwise, the distance local minimum is smaller than zero, and the corresponding color weight maximum aligns with the zero iso-surface.
Proof. Before start our proof, we first provide discussion of the rendered opacity a. We define the rendered opacity a as the integral of the color weights w(f) along the entire ray r(t) = o + td. This definition coincides with the code that calculates the opacity, found NeuS's source code. Technically speaking, a can be defined by Eqn. (3
w(1) di
A discovery made by HF-NeuS T(t), as shown in Eqn.
66 connects the color weight w(t) with the accumulated transmittance
d T(t) -(t)= - =-w(t)
dt
Hence, can simplify the integral of the color weights wt) to the integral of the derivative of the accumulated transmittance T(t), connecting the rendered opacity a with T(t) itself directly, shown in Eqn. (5)
+0o
d
w((t
T(t)
dt
J0
dt
= =T(0)
lim T(t)
(5)
1 lim
T(t)
++000
Our proof is divided into two parts. In the first part, we prove that the distance field we described in the theorem could lead different rendered opacity a. After that, in the second part, we prove the unbiasedness claimed in the theorem.
Part I We first provide recapitulate of the density function defined in NeuS 4 NeuS 4 and HF-NeuS [6) both derive the same density formula, shown in Eqn. (6)
d (f(((()))) ((((t) di (.((((())) = - (f(((tt))) cos(O)
(6)
where (.) is sigmoid function parametered by 8, and 0 is the angle between the ray direction and the gradient the distance field f, and cos(O) < 0. NeuS 4 and HF-NeuS [6] further clip against 0 wherever cos(@) becomes positive, ensure non-negative density values, shown Eqn. (7). This process is effectively equivalent to back-face culling technique used in mesh rasterization pipelines.
(r(()) = max(o
We consider the scenario where the ray intersects with only one front-facing plane at f. = to- Let the local minimum of the distance field be m, the signed distance function f(r(t)) can be explicitly written as Eqn. ( We provide readers with Figure 9 to help understand the different terms in
Eqn. (8)
(1 (1- - lo) |cos(0)] + m,
2
f(r(t))
(t (t-to [ene(0a| |cos(0)] +m, m <
(8)
14

---

dist
f(r(t))
44 cos( + m
0
to
cos (0)
dist
to
loos
Tr
#)
Ar to 0
do = At ccos (0)
0
to
ray
ray
to
plane
(a)
(b)
Figure 9: Illustration for Eqn. (8. (a) The two red curves represent the two cases of the distance functions when m 2 0 and m <0 respectiivel (b) is an illustration showing the relationship the distance do to and (O).
When m 2 0, direct calculation of a shown in Eqn. 9 by substituting the variable with (r((t))
a 1 - lim T(t) b
= 1 exp
(r(t))
+00
(000s))))))0(000)) 1+e cos(0),0) dt
-se (f((e)((()))
1 exp
=f((((t)
1 exp
1 -=f((((() ccos(0) dt
(9)
-e-e((r(t)
1
cxp
1 (+-=)(()(o) d(sf(r(t)))
=(fo|cos(0))+mm)
e sto cO0))
1
+ esm
sto (os)
+ e
and m leads the resulting range of a E (0, - Jolo000)))00 Note that (o)cos(O) is the distance
between the camera origin and the surface. If we denote that distance dg, we can rewrite Eqn. (9) as a 14eam
-adg
0.
1-e- -dd
It's worth noting that when O. the distance field is the corresponding UDF of the scene. 2S- UDF 24 discovers that naively applying NeuS's density function on UDFs will lead transparency in rendered surfaces. Our theoretical result explains this observation.
15

---

When m < 0, direct calculation of a is shown in Eqn. (T)). calculated in similar manner.
1 lim T(1) 1-+00
1 - exp
(r((t)
toc
T+e-s/(r()) cos(O); 0) di
-se-=f(r(1)
1 exp
max
rut (ot(-m)//com())
-se (f((t)))
(10)
1
exp
1+6-)/(1()) cos((f) dt
rmm
-e (((((((
1 exp
1 e+e d(sf(r(t)))
stg]cos(0]]
1+ sta (0)
1 +
and < 0 leads to the resulting range of a E
1-e stgkwu(0)1
1).
Using the same notation above,
1dd T+e-am
I-c ndg 1).
rewrite Eqn. (T))
In this case, each front-facing plane's associated back-facing plane, which not rendered due to the clipping of the density, begin separate from the front-facing plane. A larger means further-away back face. When the back face is infinitely away, m will approach -o0. The rendered opacity a will then approach 1, which means fully opaque. This scene can be considered a single ray- ray-plane intersection, and is the scene on which NeuS and HF-NeuS 6 base their density function derivation.
Combining the cases together, different choices of m will cover every rendered opacity a from to 1. The watershed a of the two cases is --e-2do Since after training, the parameter converges large number, and the origin of the ray usually away from the surface (meaning that de is relatively large), the watershed a is approximately 0.5.
.2
Part II Only the places where cos(0) contributes the rendered color, and the color weight function wt) is continuous and smooth in this region. Therefore, Eqn. (II) shows the derivative of w() with respect t.
w'(t) [T())(((()))
(11)
(d)((r()r() -r((r())]
Should the local maximum occur at f = t", we get w/ (t*) = 0. Since T(t) > 0 is always true, Eqn. 12 should hold true:
=(r(f)))
(12)
Since
-s (1 11- I +e-s(((((n)) cos(O)
d'(((t))
(13)
5² (0)e -0)e=tf(((t") *f(r((*))
(1 (1+e0)/e())))
16

---

and
o² (r((t))) x² cos² (0)e (0)=-ia(r((())
(14)
(1 + (1+==s((x(=1))
solving Eqn.
12
get
1= e 1=--=/((())) =
= [(r(())
(15)
When m O, the zero distance field value position exists at t" = to. and the maximum weight position aligns with the zero distance field value position.
When m O, the position of the distance field minimum value is also the position of the zero distance value. Therefore, the maximum weight occurs at the zero distance position, which is also the distance field minimum position.
When m > 0, the distance field is nowhere zero. In this scenario, we have Vt < to.
d()( -f(0) < ((00)))-====0000000
(16)
w (1)
This means that the color weight is continuously increasing until the distance field reaches its minimum, implying that the maximum position the color weight is aligned with the position the distance field minimum.
And that completes the proof.
B Additional Experimental Results
B.1 More comparisons with NeUDF (11
23.21
18.24
Figure 10: More results of NeUDF 21 The Chamfer distances (x 10-3) are shown below synthetic models. The Jar is shown in section view.
We show results NeUDF [21 whose surfaces extracted by DCUDF 11 in Figure TO Generally speaking, compared with the results of our o-NeuS in the paper, the NeuS backbone outperforms the NeUDF backbone in quantitative and qualitative measures.
B.2 Empty transparent object
The majority of our data focuses on the objects where transparent and opaque share roughly the proportion. We include synthetic case where the vast majority of the object is transparent and without refraction or reflection. The results are shown in Figure
B.3 Experimental results on DTU dataset
We also conduct experiments on DTU dataset 44 which is opaque object dataset. Theoretically. our method when applied to pure opaque dataset, essentially equivalent
original NeuS [4]
We present the results selected data in Figure [2 Although the SDF learned by NeuS is not completely clean and may oscillate inside the object, it does interfere with the mesh extraction process with DCUDF [[]] shown in the top of Figure 12 Moreover, DCUDF can resolve
17

---

Ref. image
NeuS 4
Ours
Empty Jar
Figure 11: An empty transparent jar (section view).
DTU 55
DTU 114
DTU 122
NeuS B
0.422
0.349
0.473
Ours
0.422
0.347
0.472
Figure 12: Quantitative comparison on selected DTU dataset The Chamfer distances (x10-) 10-3) shown below each model. The slices of the absolute values of the SDF are shown and truncated to 0.2 for better visualization. The orange line the r = 0.002 level set used DCUDE.
mesh vertices more accurately than marching cubes 10,, which relies on interpolation to approximate the value location between grid vertices, therefore there cases where the mesh extracted by DCUDF can achieve a lower Chamfer distance than those by marching cubes. Similar results are also reported by the authors of DCUDF [I] (Table 3).
C Acknowledgments of Dataset
We thank the vibrant Blender community for heterogeneous models that drive our experiments. Specifically, we thank Rina Bothma for creating the bottle model, Joachim Bornemann for creating
the case model, Rudolf Wohland for creating the jar model, Eleanie for creating the strawberry model which is put inside the jar. These models are released under royalty-free license described
18

---

at httpps://ww blenderkit. http://ccc//ccccccceseeeeeeeee Furthermore, we thank MrSorbias for beginner-friendly tutorial on creating snow globe in Blender on YouTube.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and caw.utttiionncaatttoo reflect the paper's contributions and
scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have a dedicated section for limitations, in Section 4.4
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
20

---

Justification:
Our main theoretical result (Theorem 1) is proved extensively in Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
All of our experimental results are reproducible.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
21

---

Answer: [Yes]
Justification: Our data and code are publicly available at https:// com/
aaaa8880//llpaa-Nuuus NeuS
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https wwt.sSuwwisssssppoo.oooooooooooe
/nipps.cc/pulli///uuu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Implementation details are introduced in Section 4.1 Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Like almost all methods, our neural networks are randomly initialized. Our method does not depend on any special random numbers. We believe the different test cases in our experiments can explain the stability of our method.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
22

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Experiments Compute Resources are introduced in Section 4.1
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://ss//aarrss.cpppppp//////a
Answer: [Yes]
Justification:
The research conducted in the paper fully conforms to the NeurlPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: To the best of our imagination we cannot think of potential misuse of our proposed approach that leaves negative impact to the society.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
23

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper does not pose such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We use Blender models created by other people, which are released under royalty-free license that only prohibits reselling the assets in their original form. In our case, we render them into images which are in a different form. Furthermore, we do not plan to sell them. The creators of these assets are credited in dataset acknowledgments Appendix C
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
24

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
All our data and codes are well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "xSGcZzsvS2", "reviewer": "Reviewer_ruwJ", "summary": "The paper presents a theoretically justified method for surface reconstruction of transparent and opaque objects. The authors prove the theorem that the NeuS density function is unbiased even for transparent surfaces. They propose a method based on DCUDF for extracting unbiased surfaces. The method is validated on synthetic Blender datasets and real datasets captured by the authors. The method is compared quantitatively with NeuS at different threshold levels and qualitatively with NeUDF.", "questions": "Questions and suggestions:\n1. Related work on the method of 3D reconstruction of transparent objects is limited and should be reviewed. I suggest that the authors review the following papers: \"NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects\", ICCV 2023; \"Ray Deformation Networks for Novel View Synthesis of Refractive Objects\", WACV-2024; \"Volumetric Rendering with Baked Quadrature Fields\", 2023. All of these papers have applications to transparent object reconstruction and I would like to see how they relate to considered setup. \n2. While some of the previously mentioned papers do not consider datasets with opaque objects, your method is expected to be applicable to datasets with only transparent objects, so I suggest the authors experiment with this setup on publicly available datasets from \"Through the Looking Glass\": Neural 3D Reconstruction of Transparent Shapes\" and NEMTO. It should also be noted that some works consider publicly available data with both transparent and opaque objects, see Fig. 7 and Fig. 9 in the paper \"Volumetric Rendering with Baked Quadrature Fields\" (https://arxiv.org/pdf/2312.02202v1) from the NeRF-Synthetic and Mip-NeRF-360 datasets respectively.\n3. It looks like the authors of \"$\\alpha$Surf\" have released their code (https://github.com/ChikaYan/alphasurf), so I suggest the authors to check it again. \n4. It would be better to have an illustration for Eq. 8.\n5. Can you clarify training hyperparameters of your method like lambda coefficients, etc and explain their choice?\n\nTypos:\n1. Line 44, .e.g -> e.g.", "limitations": "The limitations of the method are adequately addressed in the Limitations section.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Theorem 1 is an important fact for the 3D reconstruction community as NeuS is a fundamentally justified method for surface reconstruction. \n2. The authors presented additional datasets with transparent and opaque objects. The number of open datasets with such properties is limited.\n3. The proposed algorithm for NeuS modification is a solid technical contribution that combines NeuS and DCUDF methods.", "weaknesses": "1. There are no comparisons of the method on publicly available data with transparent and opaque objects, see Question 2.\n2. My concern is that it is not clear whether the method can reconstruct scenes with only transparent objects. Comparison with transparent object methods is limited, see Question 1. The method hasn't been tested on transparent object datasets that have been used previously in the literature, see Question 2.\n3. Another concern is the number of competitors. In fact, there is only one quantitative evaluation with NeuS. A quantitative comparison of the method with NeUDF is not presented.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments.\n\n**Q: Related work on transparent objects.**\n\nA: Thank you for highlighting several pertinent studies that we had initially overlooked. We will incorporate a discussion of these works in our revised manuscript. The works [1,2] address the reconstruction of fully transparent objects by separating rendering from geometry or considering the variations in rays due to refraction and reflection. Sharma et al. [3] introduces multiple intersections during the rendering process through an additional network. It primarily focuses on efficient rendering rather than geometric reconstruction.\n\nReferences:\n\n[1] Wang D, Zhang T, Süsstrunk S. Nemto: Neural environment matting for novel view and relighting synthesis of transparent objects. in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2023: 317-327.\n\n[2] Deng W, Campbell D, Sun C, et al. Ray Deformation Networks for Novel View Synthesis of Refractive Objects. in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2024: 3118-3128.\n\n[3] Sharma G, Rebain D, Yi K M, et al. Volumetric Rendering with Baked Quadrature Fields. ECCV, 2024.\n\n\n**Q: Testing datasets with only transparent objects.**\n\nA: As mentioned in the limitations section, we \"model only thin transparent objects that have as little refraction as possible\". As an extension of NeuS, we primarily discuss NeuS's capability in reconstructing transparent objects. However, since refraction is not considered in NeuS, it is infeasible to use NeuS directly to reconstruct thick transparent objects with strong refraction and reflection. The related works, e.g., NEMTO, reconstruct thick transparent objects with significant refraction and reflection, whose goals are different from ours. We are glad to explore using works like NEMTO as a new backbone to extend the versatility of our method in future work.\n\nMeanwhile, we attach a result of a synthetic empty transparent object without refraction and reflection in Figure 3 of the supplementary one-page PDF file.\n\n**Q: Checking the code of $\\alpha$Surf.**\n\nA: Thank you for the reminder. As of our submission date, $\\alpha$Surf had not yet released its source code. We are keen to perform experimental comparisons with $\\alpha$Surf and have already conducted some preliminary experiments. The results are detailed in Figure 1 of the supplementary PDF file provided. \n\nWe notice that the $\\alpha$Surf repository is currently marked as \"a preliminary repo for submission of supplementary material\", indicating that it may not contain the final release code. Consequently, the results from these experiments should be viewed as preliminary.\n\n\n**Q: Illustration for Equation 8.**\n\nA: We added the illustration for Equation (8) to enhance clarity and understanding. Please refer to Figure 5 in the supplementary PDF file. (a) The two red curves represent the two cases of the distance functions when $m\\geq 0$ and $m<0$ respectively. (b) is an illustration showing the relationship of the distance $d_0$, $t_0$ and $\\cos(\\theta)$.\n\n**Q: Clarification of training hyperparameters.**\n\nA: Please refer to the combined responses to all reviewers for detailed evaluations.\n\n**Q: The quantitative evaluation with NeUDF.**\n\nA: Please refer to the combined responses to all reviewers for detailed evaluations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qZyNkXaFr6", "reviewer": "Reviewer_6LF5", "summary": "The paper proposed a extended version of NeuS that can reconstruct both transparent and opaque surfaces at the same time with unbiasedness. The unbiasedness is guaranteed by theoretical proof and a companying surface extraction method is also proposed. The authors evaluated their method on a small benchmark and demonstrated the methods’ effectiveness and superiority over other methods.", "questions": "I don’t have any question for this submission.", "limitations": "1. Is it possible to include participating media in your pipeline? e.g. if there’s some smog inside a jar, will the unbiasedness still apply? Or can the pipeline be generalized into a stochastic geometry perspective?\n2. If possible, can GT of real-world test cases be constructed with some ‘destroy’ to the object, like what NeRO did?", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Solid theoretical foundation: The paper provides a rigorous theoretical analysis, extending the unbiasedness proof of NeuS to cover a range of material opacities from fully transparent to fully opaque. This theoretical work completes and expands the framework of NeuS.\n2. Effective surface extraction method: The proposed surface extraction technique, which combines aspects of DCUDF with a novel approach to handling mixed SDF and UDF fields, appears well-designed and effective. It successfully extracts both transparent and opaque surfaces from the learned distance field.\n3. Comprehensive evaluation: The paper presents thorough comparative and evaluation results, both qualitative and quantitative. The experiments cover both synthetic and real-world datasets, providing strong evidence for the effectiveness of the proposed method. The quantitative metrics, particularly the Chamfer Distance comparisons, offer clear and convincing support for the method's superiority over baseline approaches.\n4. Discussion of limitations: The authors include a discussion of failure cases. The identified limitations, particularly regarding complex lighting conditions with heavy refraction or reflection, are reasonable and understandable given the scope of the work. This discussion provides valuable insights for future research directions.\n\nIn summary, this submission presents a solid contribution to the field, combining theoretical advancements, practical algorithmic development, and thorough empirical validation.", "weaknesses": "1. The evaluation set is relatively small, adding more synthetic cases would be fine.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments.\n\n**Q: The evaluation set is relatively small, adding more synthetic cases would be fine.**\n\nA: Thank you for your suggestion. We added a synthetic example in Figure 3 of the supplementary one-page PDF file as a test of an empty transparent object without refraction and reflection.\n\n**Q: Is it possible to include participating media in your pipeline? e.g. if there’s some smog inside a jar, will the unbiasedness still apply? Or can the pipeline be generalized into a stochastic geometry perspective?**\n\nA: Including semi-transparent media such as smog within a jar could cause the learned distances to oscillate near zero, simulating the presence of smog. The oscillation might result in multiple local minima, complicating surface extraction. If these local minima could be consistently excluded from the DCUDF (e.g., if the minima are all greater than the user-specified iso-value $r$), the intended surface could potentially be extracted. Theoretically, it is feasible to reconstruct the jar by minimizing smog interference, though practically, this approach may not be robust.\n\n**Q: If possible, can GT of real-world test cases be constructed with some \"destroy\" to the object, like what NeRO did?**\n\nA: Thank you for this insightful suggestion. However, capturing  transparent objects with an RGBD camera poses challenges due to their optical properties. We will explore this possibility in future research endeavors.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KVQI0OXlLK", "reviewer": "Reviewer_vpPg", "summary": "This paper proposes an adaptation to NeuS to extract semi-transparent surfaces. The method relies on training the original NeuS but with a modification to the mesh extraction. Instead of using marching cube; this paper proposes to extract the 0 level of the SDF field and also its local extrema that represent semi-transparent surfaces.  \n\nThe contributions are:\n- A theoretical derivation to show that the proposed method is sound\n- A new method to extract mesh\n\nThe method is evaluated on a custom dataset with both synthetic and real scenes of semi transparent objects.", "questions": "See limitations: the paper could benefit from an improved presentations. \n\nI expect from the rebuttal:\n- Results and discussion on opaque scene (DTU)\n- More explanations about the optimization of section 3.2", "limitations": "The limiations are correctly stated", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper is interesting, it clearly improves over vanilla NeuS.\n- The paper introduces a new dataset with transparent objects, the author say this dataset will be released on github", "weaknesses": "- Experiments and data: \n  - It would be nice to have an additional experiment showing the geometric accuracy of the new mesh extraction procedure with usual data, like DTU. I expect this method to perform worse than NeuS on DTU since this method can extract many spurious local minima of the SDF that are ignored with Marching Cube. \n  - While the proposed dataset is interesting, it may be too easy compared to real world. The synthetic data looks like specularities are disabled. However specularities are most present on transparent objects (glass, water) and this may be a harder challenge to solve. As stated in the limitation section, the proposed method cannot deal with reflections. \n\n\n\n- Presentation\n  - There could be a discussion about the role of the temperature parameter $s$ in equation (1). This parameter is used to map a SDF value to an opacity. It is introduced in Neus as a method to have a smooth optimization, starting from fuzzy surfaces and finishing with sharp ones. However in this paper s is also used to make sure nonzero SDF still produce nonzero opacity. \n  - From my understanding, this method follows NeuS training but this is never clearly stated\n  - What are the normal maps in Fig 4, 6, it looks like normals from the proposed method but it is not written in the caption.\n  - Lack of explanations about the optimization (Section 3.2). The paper shows two objective functions (l 167 and 176) it is not clear which one is used. Also there are no explanation about the optimization itself, the solver, stopping criterion, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments.\n\n**Q: Results and discussion on opaque models (DTU).**\n\nA: We have added additional results on the DTU dataset in Figure 4 of the supplementary PDF. The results of our $\\alpha$-Neus and the original NeuS are similar.\n\nWe show the absolute values of the SDF learned by NeuS, in which DCUDF performs optimization. We observe some local minima/maxima within the model in the distance field. Most of these local minima/maxima appear to be above $r$ and far away from the surface. Local minima larger than $r$ don't affect DCUDF. Thus the influence of local minima is small. In addition, DCUDF can resolve mesh vertices more accurately than Marching Cubes at the same resolution. This has been demonstrated by DCUDF -- see Table 3 of DCUDF paper -- resulting in better Chamfer distance values in some cases.\n\n**Q: More explanations about the optimization in Section 3.2.**\n\nA: Please refer to the combined responses to all reviewers for detailed evaluations.\n\n**Q: The normal maps in Figures 4 and 6.**\n\nA: The normal maps displayed in Figures 4 and 6 were generated using the official NeuS codebase for calculating normal maps. We will specify this clearly in the revision.\n\n**Q: The setting of NeuS.**\n\nA: Please refer to the combined responses to all reviewers for detailed evaluations.\n\n**Q: The role of $s$ in Equation (1).**\n\nA: In NeuS, $s$ is a learnable parameter that gradually converges to a large value over the course of training. A larger $s$ value sharpens the edges and faces in the reconstructed model, enhancing overall quality. However, in practice, $s$ cannot increase indefinitely due to numerical computation constraints, such as the number of sample points. This caps $s$ at relatively high but finite values, which allows non-zero distance values to influence color calculation along the ray. Colors are derived from the the weighted sum of sampled radiance at points along the rays, and different distance minimum values contributes to achieving different levels of opacity. \n\nWe believe that the tendency of a larger $s$, combined with the color loss and the actual situations with MLP and numerical calculation will achieve a balance that leads to the best results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QthkQZHrDp", "reviewer": "Reviewer_GRsP", "summary": "The paper proposes an extension of NeuS to also support transparent surfaces. It states a proposition that claims that NeuS is also unbiased for transparent surfaces, proposes an approach for simultaneous extraction of opaque and transparent surfaces, and presents a benchmark dataset.", "questions": "(1) I would like to see a quantitative evaluation against NeUDF too. Even though NeUDF seems to smooth the surface, $\\alpha$-NeuS also seems to introduce noise. I am curious about which one is closer to the ground truth surface.", "limitations": "Yes, the paper has a paragraph about limitations.", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "(1) The results are good. The proposed approach successfully reconstructs the opaque and transparent surfaces simultaneously.\n\n(2) The paper presents a benchmark dataset, which may help the empirical standardization of the field and paper writing in consequence.", "weaknesses": "(1) The current paper organization and dependency severely hinders its appreciation. It should have at least a background / overview section with a minimum of presentation and intuition behind the core concepts. I understand that this paper is an extension of NeuS, but assuming that reading NeuS is an unavoidable prerequisite to conceive an overview of the method is not a gentle choice regarding readers or reviewers. For example, the Abstract, Introduction, and Method sections refer to the concept of unbiased surfaces, which is only introduced at the second paragraph of section 3. Moreover, that is a loose definition since it depends on the concept of rendering weights, which is not introduced at all. The expression rendering weights is ambiguous since it could means a plethora of concepts in the context of the paper. Presenting the equation beforehand and pointing to $\\omega(t)$ would remove the ambiguity.\n\n(2) Also in the ambiguity realm, Theorem 1 is not a theorem per se in its current presentation. To be a theorem, a proposition should be a clearly stated assertion that is precise and unambiguous. To be precise, all symbols must be precisely defined beforehand or in the theorem body. Theorem 1 is based on the weight function $\\omega(t)$ presented in NeuS, but its current assertion is plain English, without any precise definitions provided. The proposition is not currently verifiable in that form. Overall the paper has a significant problem stating its definitions and symbols.\n\n(1) and (2) are the core reasons behind my current rating and, unfortunately, I do not think they could be solved without substantial rewriting and in rebuttal time. I think the problem addressed is important, and also that the paper has good results, but it is based on a core theorem that is not correctly stated. I advise the authors to rewrite the paper, following the guidelines:\n\n* Please take a more didactic approach. In practice, at least a background / overview section should be present. The paper should be self-contained in some extent. That would broaden the public for the paper, since readers could understand the paper in different levels and in increasingly difficulty. It would also ease the review process, possibly earning more favor in future reviews.\n\n* Please precisely present the symbols and equations before the theorem statement to remove ambiguities. The usual form of a theorem is \"Let {definitions and symbols}. {conditions using the symbols} then {conclusions using the symbols}.\" In case the logic is bidirectional, the form is \"Let {definitions and symbols}. {conditions / conclusions using the symbols} if and only if {conditions / conclusions using the symbols}\". Following those standards helps to smooth the reading flow and helps the reader to verify whether the theorem holds. Since the theorem is a core component of the paper, including a mathematician author or making one reviewing the paper before the submission could also be a good idea.\n\n* Another approach would be to remove the theorem and make a discussion based on empirical evidence. The current results are good, arguably sufficient for acceptance if the paper is written in such standpoint.\n\n(3) The paper criticizes $\\alpha$Surf in a few locations, saying it introduces noise and artifacts. I can see such phenomena in that reference, however I do not think it is good practice to criticize another work without showing comparison experiments. Since the work is still unpublished and the code source is unavailable I would like to politely ask the authors to remove such critics. A more polite way to contextualize would be to say that $\\alpha$Surf is a concurrent work, without judging its results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments.\n\n**Q: Overview of NeuS**\n\nA: We appreciate your feedback and will add the overview of NeuS in our revision. We prepare an initial overview as follows:\n\nNeuS is a surface reconstruction method based on Neural Radiance Field (NeRF) that uses volume rendering technique, which is differentiable, to render the image and optimize the scene representation. In the realm of volume rendering, the color of individual pixels is determined by the color of individual rays that shoots from the camera origin $\\mathbf{o}$ and passes through the corresponding pixel on the imaginary image plane in the 3D space, parametered by $\\mathbf{r}(t)=\\mathbf{o} + t\\mathbf{d}$. The color of the ray, $C(\\mathbf{r})$, is determined by a weighted integral of colors $c(\\mathbf{r}(t), \\mathbf{d})$ along every point of the ray,\n$$C(\\mathbf{r}) = \\int_0^{+\\infty}w(t)c(\\mathbf{r}(t), \\mathbf{d}) \\mathrm{d}t.$$\nNote that in practice, the upper bound of integral is not infinity but rather a fixed big value. This makes sense because it is expected that there is nothing of interest far away from the camera.\n\nIn the volume rendering framework, the assigned color weights $w(t)$ of each point on the ray is determined by the volume density $\\sigma(t)$ of the point and the accumulated transmittance $T(t)$ at this point,\n$$w(t) = T(t)\\sigma(\\mathbf{r}(t)).$$\n\nThe accumulated transmittance $T(t)$ at point $\\mathbf{r}(t)$ can be construed as the possibility that a ray, starting from its origin (camera position $\\mathbf{o}$) and traversing along the ray path, could arrive at this point without being hindered somewhere before. $T(t)$ is formulated as\n$$T(t) = \\exp\\left\\lparen-\\int_0^t\\sigma(\\mathbf{r}(u))\\mathrm{d}u\\right\\rparen.$$\n\nThe color $c(\\mathbf{r}(t), \\mathbf{d})$ is predicted by a neural network. But in contrast to NeRF where the neural network directly predict the volume density $\\sigma(\\mathbf{r}(t))$, NeuS predicts the signed distance value at the point $f(\\mathbf{r}(t))$ instead. NeuS further leverages a mapping to map the signed distance $f(\\mathbf{r}(t))$ to the volume density $\\sigma(\\mathbf{r}(t))$ at this point. The tight mapping of the signed distance field and the volume density distribution enables high fidelity SDF learning through back propagation.\n\nWhen tying the object geometry with volume rendering, NeuS puts forward two major requirements for the color weight function $w(t)$: unbiased and occlusion-aware. For the rendering to be \"unbiased\", the color weight function $w(t)$ should attain locally maximal value at the surface intersection point $\\mathbf{r}(t_0)$, which, in the opaque scenario, corresponds to the zero level set of the SDF, i.e., $f(\\mathbf{r}(t_0)) = 0$. NeuS proposes a mapping from distance to density that is unbiased in the first order. For the rendering to be occlusion-aware, whenever $0 < t_1 < t_2$ and $f(\\mathbf{r}(t_1)) = f(\\mathbf{r}(t_2))$, $w(t_1)$ should be greater than $w(t_2)$. This aligns with the everyday experience that the object nearer to the camera ($t_1$) should occlude the one ($t_2$) behind it. An overview of the distance-to-density mapping function proposed by NeuS can be found in Appendix A (Eqn. (6) and Eqn. (7)).\n\nWe use $\\alpha$ to denote the rendered opacity. The $\\alpha$ of a pixel is defined as the integral of all color weights $w(t)$ along the corresponding ray,\n$$\\alpha = \\int_0^{+\\infty} w(t) \\mathrm{d}t.$$\n\n**Q: The presentation of Theorem 1.**\n\nA: We appreciate your feedback and revise Theorem 1 as follows:\n\nTheorem 1: Consider a ray $\\mathbf{r}(t)$ intersecting a plane at a point $\\mathbf{r}(t_0)$. Let $f(\\mathbf{r}(t))$ be the learned distance function and $w(t)$ the rendering color weight. If $f(\\mathbf{r}(t))$ attains its minimum at $t=t_m$ and $\\alpha$ is the plane's opacity:\n+ If $\\alpha<0.5$, then $t_0=t_m$, $f(\\mathbf{r}(t_m))>0$, and $w(t)$ peaks at $t_m$. \n+ If $\\alpha \\geq 0.5$, then $f(\\mathbf{r}(t_m)) \\leq 0$ and $f(\\mathbf{r}(t_0))=0$, positioning $w(t)$'s maximum at $t$ with $f(\\mathbf{r}(t))=0$.\n\n**Q: The quantitative evaluation with NeUDF.**\n\nA: Please refer to the combined responses to all reviewers for detailed evaluations.\n\n**Q: A more polite way to contextualize would be to say that  $\\alpha$-Surf is a concurrent work, without judging its results.**\n\nA: Thank you for the guidance. We will revise our discussion to reference $\\alpha$Surf as a concurrent work. As mentioned by reviewer ruwJ, $\\alpha$Surf has recently released a preliminary version of its code, which we have evaluated and discussed in Figure 1 of the supplementary PDF. We observed that $\\alpha$Surf outperforms NeuS and NeUDF+DCUDF on the synthetic dataset, judged by the average Chamfer distances. We hypothesize that the observed surface noise in $\\alpha$Surf's results arises from the use of alpha shapes [1] to generate meshes from point clouds. Given the extensive research on producing more reliable meshes from point clouds [2][3][4][5], we anticipate future improvements in $\\alpha$Surf results. Given that the currently released code of $\\alpha$Surf is preliminary, we regard our existing comparison with $\\alpha$Surf as initial findings. \n\n[1] Edelsbrunner et al. On the shape of a set of points in the plane, IEEE Transactions on Information Theory, 29 (4): 551–559, 1983\n\n[2] Zhou et al. Learning a more continuous zero level set in unsigned distance fields through level set projection. ICCV. 2023: 3181-3192.\n\n[3] Liu et al. Ghost on the Shell: An Expressive Representation of General 3D Shapes. ICLR, 2024.\n\n[4] Chen et al. Neural dual contouring. ACM Transactions on Graphics (TOG), 2022, 41(4): 1-13.\n\n[5] Wang et al. Hsdf: Hybrid sign and distance field for modeling surfaces with arbitrary topologies. NeurIPS, 2022, 35: 32172-32185.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for their insightful and constructive comments.

**Q: The quantitative evaluation with NeUDF.**

A: We have included further experimental results related to NeUDF in Figure 1 and 2 of the supplementary one-page PDF file. Generally, the NeuS backbone outperforms the NeUDF backbone. Moving forward, we plan to explore additional methods within the NeuS family as potential backbones for further testing.

**Q: Training hyperparameters and more explanations about the optimization in Section 3.2.**

A: We followed the recommended configuration for the synthetic dataset by the authors of NeuS, without changing the loss functions or their respective weights.

We conducted our experiments using almost the same setting as DCUDF. DCUDF employs a two-stage optimization process. Initially, the optimization follows the formula mentioned in line 167 for the first stage, and subsequently, the formula in line 176 for the second stage. We will clarify these configurations in the revision of the paper. We performed 300 epochs for step 1 and 100 epochs for step 2 respectively, which is the default setting of DCUDF. We used the VectorAdam [1] optimizer as suggested by DCUDF. We set the weight $\lambda_1=500$ which is different from DCUDF's default setting.  We set $\lambda_2=0.5$ as suggested by DCUDF. We do not use the mini-cut postprocessing in DCUDF.

[1] Ling S Z, Sharp N, Jacobson A. Vectoradam for rotation equivariant geometry optimization. Advances in Neural Information Processing Systems, 2022, 35: 4111-4122.

Meta-review: This paper was reviewed by four experts in the field.  The paper received the following recommendations: Strong Accept, Weak Accept, Weak Accept, and Strong Reject.  All the reviewers agreed that the results are promising and the new dataset will contribute to the community.  On the theoretical contribution, however, the reviewers could not reach a consensus unfortunately.  The three positive reviewers highlight the theoretical contribution in their reviews, while the other reviewer strongly disagrees. Having read the submission, the reviews, and the author feedback, the AC decided to accept the paper.  Although Theorem 1 might have room to improve its presentation, it is acceptable to present it in a plain English in the main text with providing a detailed proof in the appendix as the authors did in the original submission. In addition to this point, the reviewers did raise some valuable concerns that should be addressed in the final camera-ready version of the paper. The authors are encouraged to make the necessary changes to the best of their ability.
Decision: Accept (poster)