HSDF: Hybrid Sign and Distance Field for Modeling Surfaces with Arbitrary Topologies
Authors: Li Wang
Jie Yang
Weikai Chen
Xiaoxu Meng
Bo Yang
Jintao Li
Lin Gao
Venue: NeurIPS 2022 Accept (2022)
Keywords: implicit field
hybrid representation
arbitrary topology
mesh extraction

Abstract: Neural implicit function based on signed distance field (SDF) has achieved impressive progress in reconstructing 3D models with high fidelity. However, such approaches can only represent closed shapes. 
Recent works based on unsigned distance function (UDF) are proposed to handle both watertight and open surfaces. 
Nonetheless, as UDF is signless, its direct output is limited to point cloud, which imposes an additional challenge on extracting high-quality meshes from discrete points.
To address this issue, we present a new learnable implicit representation, coded HSDF, that connects the good ends of SDF and UDF. In particular, HSDF is able to represent arbitrary topologies containing both closed and open surfaces while being compatible with existing iso-surface extraction techniques for easy field-to-mesh conversion. In addition to predicting a UDF, we propose to learn an additional sign field via a simple classifier. Unlike traditional SDF, HSDF is able to locate the surface of interest before level surface extraction by generating surface points following NDF~\cite{chibane2020ndf}. We are then able to obtain open surfaces via an adaptive meshing approach that only instantiates regions containing surface into a polygon mesh. We also propose HSDF-Net, a dedicated learning framework that factorizes the learning of HSDF into two easier problems. 
Experiments on multiple datasets show that HSDF outperforms state-of-the-art techniques both qualitatively and quantitatively.

HSDF: Hybrid Sign and Distance Field for Modeling Surfaces with Arbitrary Topologies
Li Wang 12, Jie Yang! Weikai Chen', Xiaoxu Meng', Bo Yang', Jintao Li and Lin Gao (53) f1.2
Beijing Key Laboratory of Mobile Computing and Pervasive Device Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences
Tencent Games Digital Content Technology Center
(wang] i 20s, yangji e01 @icct ac. on ixiaoxumeng, brandonyang @tencent. com
chenwk891 110mmm1mmm1..mmmmmmmm comm
{jtli, (ww.innnicttc......
NDI
Cluss
NDF
Ous
NDF
Oues
Figure I: We reconstruct three groups of representative objects with both open and closed surfaces using NDF [15] (left) and our proposed HSDF (right). Compared with the SOTA NDF [15]. our method achieves higher surface reconstruction quality and more consistent surface normals. All the results are reconstructed with an equivalent resolution.
Abstract
Neural implicit function based on signed distance field (SDF) has achieved im- pressive progress in reconstructing 3D models with high fidelity. However, such approaches can only represent closed surfaces. Recent works based on unsigned distance function (UDF) are proposed handle both watertight and open surfaces. Nonetheless, as UDF is signless, its direct output is limited to the point cloud, which imposes an additional challenge on extracting high-quality meshes from discrete points. To address this challenge, we present a novel neural implicit representation coded HSDF, which is a hybrid of signed and unsigned distance fields. In particular, HSDF is able to represent arbitrary topologies containing both closed and open surfaces while being compatible with existing iso-surface extraction techniques for easy feld-to-mes's conversion. In addition to predicting a UDF, we propose learn an additional sign field. Unlike traditional SDF HSDF is able to locate the surface of interest before level surface extraction by generating surface points following NDF [15]. We are then able to obtain open surfaces via an adaptive meshing approach that only instantiates regions containing surfaces into polygon mesh. We also propose HSDF-Net, dedicated learning framework that factorizes the learning of HSDF into two easier sub-problems. Experiments and evaluations show that HSDF outperforms the state-of-the-ar techniques both qualitatively and quantitatively.
"Corresponding author is Lin Gao (gaolin (gaolin@iet.ac.cmm.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Introduction
Recent advances in neural implicit representation [12, 40, 43, 48, 58] have set new state of the art in 3D modeling and reconstruction by breaking the previous barrier in resolution and topology. However, such approaches rely the signed distance function (SDF) which divides the space into inside and outside of the object, they are limited to representing closed shapes. To lift the limitation, methods based unsigned distance function (UDF) [15, 52, 50] are proposed such that much broader class of shapes containing open surfaces can be effectively represented and learned via deep neural networks.
However, as UDF signless, directly applying the iso-surface extracting technique, e.g. the Marching Cubes algorithm, would convert all open surfaces into the closed mesh. To generate open structures, these approaches have to convert the resulting UDF field into discrete points and then apply the Ball-Pivoting algorithm [4] (BPA) to obtain the meshing result. Nonetheless, the BPA technique is prone to introduce self intersections and disconnected surface patches with inconsistent normals (see Figure 1). In addition, BPA is highly sensitive to the input parameters and often requires per- per-shape parameter tuning in order to generate a complete meshing result. This hinders UDF-based approaches from being practically used in real-world applications mesh remains the prominent standard for modeling and rendering in both industry and academia.
To address the above issue, we present novel learnalle implicit representation, named Hybrid Sign and Distance Function (HSDF), that can faithfully represent complex geometry containing both closed and open surfaces, while being compatible with off-the-shelf iso-surface extraction methods, e.g. the Marching Cubes algorithm, for easy and high-quality field-to-mesh conversion. The key idea of HSDF is to integrate the advantages of both SDF and UDF while avoiding their respective shortcomings. We empirically find that the learning of UDF is quite robust and can generalize well to novel data. Therefore, to inherit the benefit of UDF and overcome its limitation, we propose to learn an additional sign field in addition to UDF via sign predictor. Unlike traditional SDF, HSDF is able to locate the open surface before performing level surface extraction. We achieve this by generating surface points via the gradient field of the unsigned distance function following NDF [15]. Hence, able to create local SDFs by multiplying the UDF with the sign field and cast complex shapes containing both closed and open surfaces by incorporating an adaptive meshing algorithm that only instantiates the regions containing surface points into polygon mesh.
As HSDF may be discontinuous at regions far from the surface due to the sudden change of signs, propose to factorize the learning of HSDF into two casier sub-tasks, each of which learns continuous function. Specially, we propose HSDF-Net that learns the unsigned distance and sign field individually using the same input in separate manner. A fusion framework is also proposed to faithfully fuse the predicted distance and sign field at the inference time. Finally, we introduce an adaptive masked Marching Cubes algorithm for converting HSDF into the high-quality mesh at flexible resolution efficiently. Extensive experiments show that HSDF can outperform the previous state-of-the-art methods in both qualitative and quantitative measurements. We summarize the key contributions as follows:
We introduce HSDF, novel neural implicit field that can faithfully represent complex shapes with closed and open surfaces while being compatible with existing level surface extraction techniques.
We propose dedicated learning framework that separates the decoding of sign and distance in accordance with the definition of HSDE.
We propose an adaptive masked Marching Cubes algorithm to generate meshes from HSDF efficiently.
2 Related Works
3D neural shape representation is a very fundamental and popular research topic computer vision and graphics. In this section, we will review some recent advances in neural shape representations from two aspects: Explicit & Implicit Representation. Some survey papers [30, 6, 1, 60] have comprehensively summarized the development of 3D shape representation.

---

Learning on 3D Explicit Representations, For learning on 3D shapes, various 3D representations are upplied to the learning-based approaches, such as point clouds, voxels, and polygonal meshes. The voxel represents the concrete 3D shapes with regular domain which is directly an extension of 2D) pixel grids, and it widely used in 3D deep learning. Many previous works [38, 18, 59] investigated this representation by extending the 2D deep learning operators to the 3D domain. However, 3D voxel representation suffers from low-resolution, high-computation, and memory- remooyycoonsupttoon drawbacks. Further, Wang et al.tried their best to address the above drawbacks by introducing the octree-based deep learning methods [54, 56, 55, 36]. The point cloud is flexible and easier to be captured by some portable 3D scanners (e.g. Microsoft Kinect), but the main challenge is that the fidelity is too low to represent high-quality shape geometry and topology. Due to its irregular and disordered structure, the traditional deep learning methods cannot directly be applied, Qi et al al.sirstly introduced PointNet [45] and PointNet+++ [46] for 3D classification and segmentation by pooling operation which is order-independent. Wang et al. [57] constructed sub-graph by KNN and applied the graph neural network on the point cloud to learn the local geometric features for shape analysis [8]. There is also work [29] that leverages hierarchical structure for point cloud understanding. Some works [1 I, 34, 26, 47, 35, 28] exploited the topology of polygonal meshes for shape analysis. There is a recent trend [24, 53, 31, 37, 42, 22, 62] of studying the deformable meshes represent the detailed geometry. The predefined connections of the vertices set to limit the fiexibility and topology.
Learning on 3D Implicit Representations. Implicit function is powerful tool capable of modeling arbitrary geometric details and topologiaal structures. With the help of deep neural networks, a complex shape can be implicitly represented by classifying the query point in/outside shape (Binary Occupancy) [39, 44, 17, 23, 49, 20, 13] or predicting the continuous signed distance (SDF) to the shape surface [12, 40, 43, 32, 48), where the sign of SDF determines the query point in/out-side of shape. The binary occupancy and signed distance function can only represent the watertight closed surface, since it is essential that partition the set of query points inside or outside of the shape. For the open surfaces (e.g. cloth, thin-shell models), Atzmon et al. [2] developed a deep learning approach- - Sign Agnostic Learning (SAL) for raw data with any inside/outside labeling, but the prediction of SAL is still SDF for the close surface reconstruction. Furthermore, the concurrent works NDF [16] and DUDE [52] proposed neural-based methods that predict the unsigned distance field to represent arbitrary surfaces without any closed surface data. NDF adopts the multi-scale encoding techniques from IF-Net [14] enhance the surface details. DUDE introduces a disentangled shape representation that utilizes the unsigned distance field (UDF) and normal vector field (nVF), which learns high- fidelity representations. Due to the gradient vanishing of UDF close to the surface, UDF fails to predict high-quality estimation of high-ordered geometry attributes (e.g. normal, tangent plane), CSPNet [51] represents such surfaces by a class of implicit representation - closest-surface-poin (CSP), which achieves local geometric attributes accurately and efficiently.
Recent concurrent works such as GIFS [63], DeepCurrents [41], MeshUDF [25], 3PSDF [9] and NDC [10] also focus on modeling open surfaces. DeepCurrents [41] combines the explicit boundary curves and the implicit field. And the final open surfaces are obtained by solving a minimal surface problem, which is not as flexible as pure implicit field to model arbitrary shapes. Methods like GIFS [63] and NDC [10] model the relationships between every two points or edge intersections, which is not as efficient as modeling the relationships between points and surfaces. And the re- constructed surface normals will be different from the ground truth meshes because point-surface relationships are missing. MeshUDF [25] proposes differential meshing algorithm for UDF. Since its meshing process is still based on the Marching Cubes algorithm and the signs required are decided in heuristic way, the outcome meshes suffer from flipped faces and have different normal directions from ground truth meshes. 3PSDF [9] proposes an implicit representation that divides the space into three classes, namely positive, negative and null. Null class enables 3PSDF to mask out certain regions in the space and represent open shapes. 3PSDF formulates the reconstruction task as a classification problem. It eases the learning difficulty but also limits the applications downstream tasks that require SDF continuity, such as neural rendering.
In contrast, HSDF can efficiently model arbitrary shapes with both open and closed surfaces and achieves high-quality reconstruction with more consistent surface normals similar to the ground truth using the proposed adaptive masked Marching Cubes algorithm.
3

---

Sign, (p)
Variise Cozr
Trilinear Interpolation
Sign Predictor
Car
Trilinear Interpolation
F
Field Fusion
Masked
Marching Ciibes
30 Caze
Trilinear Interpolation
Distance L
F
Predictor
(ps,(p)
Figure 2: HSDF-Net architecture. The input sparse point cloud is voxelized and encoded multi-scale manner into shape code. Next, the distance predictor takes the shape code and query point p as input to predict an unsigned distance Dis(p). The sign predictor takes the same input and predicts a signed value Sign(p). The field fusion module (Sec. 3.3) is proposed to fuse Dis(p) and (ign(p). The rightmost lamp example is reconstructed from test data using the proposed adaptive masked Marching Cubes algorithm (Sec. 3.4).
Method
In this section, we introduce the proposed shape representation and our HSDF-Net with an adaptive masked Marching Cubes algorithm into high-fidelity single-layer meshes. diagram of our system architecture shown in Fig. 2. We first define the proposed implicit shape representation (Sec. 3.1). Then we introduce the model structure of HSDF-Net and explain learning strategy (Sec. 3.2). Finally, we illustrate the approach to fuse the hybrid distance and sign together to form a HSDF (Sec. 3.3) and the adaptive masked Marching Cubes algorithm (Sec. 3.4).
3.1 Shape Representation
We define the target surface as S. Our proposed formulation takes a vectorized shape code z 2 and point p E IR as input, and predicts hybrid sign and distance function HSDF(p.z) R x 2 + R. we define p' as the closest point of p on the surface S and the surface normal of p' v. Then unsigned distance can be computed as UDF(p. 2) = |lp p' and the sign is calculated as STGN(p, 2) sign(v (p- - p')), where sign(-) returns the sign of the input. Specifically, STGN z,) is positive if v aligns with p' and negative otherwise. The overall HSDF expression can be defined as:
HSDF 2) SIGN 2) UDF 2)
sign(v (p p')) ||p p'l12
s.t. p' argmin |pp P=ll2
P.ES
By this definition, we are able to locate the surface of interest by searching for the points on the surface following the paradigm of [15] leveraging the gradient of UDF(.). In addition, the introduced sign field SIGN can convert the UDF into signed one, enabling the use of the iso-surface extraction technique for easy field-to-mest conversion. By incorporating adaptive masked Marching Cubes algorithm (detailed in Section 3.4) that only instantiates the regions containing surface points into a mesh, able model complex topology with open surfaces.
3.2 HSDF-Net: Model Architecture
For regions that far from the surface, the HSDF representation may be discontinuous as the sign field may undergo a sudden change. To ease the learning of HSDF, we factorize the learning into two sub- sub-problems: learning of the unsigned distance and the sign field. As depicted in Fig. 2, we propose HSDF-Net that contains three components: a shared volume encoder, a distance predictor, and sign predictor. The shape encoder extracts features from input sparse point clouds multi-scale manner, which follows IF-Net [14]. Then we feed encoded multi-scale shape code and query point p into the sign and the distance predictors in Fig. 2 to get sign prediction Sign(p) and distance prediction Dis Dis(p). (p). Next, we will introduce network components' formulation in detail.

---

Volume Encoder: Our goal is to reconstruct an arbitrary surface S from a sparse point cloud X X. The input point cloud first voxelized and encoded by 3D CNNs into multi-scale grid features F F1,... Fn,, n, F& F FR where K is the grid size and F& R is feature with C channels. If we denote the result of trilinear interpolation of grid feature F; at position as T; then V (p(p)) .......: Tn) can represent the concatenated shape code for input X point p and (P) IR F x..... x Fn defines the encoder function, see IF-Net [14].
Distance Predictor: The distance predictor pre- dicts the UDF between points and the ground truth surface. We formulate this function & ((TI..... Tw)): Fi X x Fn + RT, and we use fully connected layers with ReLU acti- vation functions to guarantee & 2 0. In general, the distance predictor could be formulated as Dis (p) ( Vx (P) R + R
B
(6) Sign
(b) Distunce
(c) Original HSDF
Sign Predictor:
We extend sign concept to bitrary shapes. For example, a T-shirt model has consistent outward face normal directions, and
we can easily tell the sign of a query point p even though a T-shirt is obviously an open sur- face because we will intuitively choose the clos- est point p' on the surface and use the normal direction of p' as a clue of the sign. Detailed formulation is defined above in Sec. 3.1. For cars with inner structures, we also compute the sign of query point p according the normal direction of its closest surface point, no matter whether p is inside the car or close to the outside surface.
Sign Function
(e) Distance Function
Optimized HSDF
postiive regaon
negative region
Optimirzal A:
Optimioal Point
lavel-eer surfice
8
0
sign function gradient
distance function gradient
2D illustration of sign and distance fu- sion. Assume the green line the target surface. Row 1: By multiplying sign and distance point- wise, we can obtain an original HSDF, where some points (e.g. A and B) may be wrongly signed. Row 2: By using gradients of sign function (i.e. yellow vectors) and gradients of distance function (i.e. brown vectors) to optimize the original fused HSDF, HSDF of points rectified (i.e. green line).
Figure 3:
Therefore, we extend the concept of "signed dis- tance" to a broader range of shapes like open shapes by combining "distance" and "sign" com- putation and prediction processes. We define ((T Tw)) Fi x x Fn H Rg as sign predictor. By composition we obtain our sign predictor (ig = (0 F (p) R3 + Eo which we treat as a regressor to the signed distance. Hence, the predicted binary sign can be computed as sSgnn (p))) R + 1-1,1] where sign(-) returns the sign of the input. And the ground truth binary sign can also be represented as STGN (P) =Sig(SDFF (p, Sx)) R 1.
can be effectively
Learning: We define the volume encoder, distance predictor and sign predictor as we, wa and ws. respectively. During distance regression training, we use L1 mini-batch loss to jointly optimize W. Wa:
Lu :=:=1 I== min (Disx(p).) min (UDF (x, Sx)
XEB
where B and P stand for mini-batch and sub-sample of points. Clamping the maximal regressed distance to value > forces the model capacity to focus more on the vicinity of the surface.
During sign predictor training, we use T., mini-batch loss to jointly optimize we W.:
C. = Σ Σ max (min (Signx(p) 8), -ō) max (min (SDF (p, Sx), -5)
XEB PEP
Finally, train HSDF-Net in end-to-end manner by optimizing the total loss: Liotal
La + L..
3.3 Sign and Distance Field Fusion
The HSDF-Net trains shared volume encoder and two decoders respectively for the sign and distance prediction. With this learning manner, we can obtain predicted sign function Signx (p) and predicted distance function (ps of query points p after training, Although Signæ (p) and
5

---

Disx(p) are both learned continuous function, they can be fused into hybrid sign and distance function HSDF(p, with our fusing algorithm illustrated in Fig. In Fig. 3, we plot a local region in 2D of learned sign function Signx(p) (P) and distance function Disx (p) where X and is the input sparse point cloud and the query point, respectively. The first row Fig- describes straightforward fusion strategy that multiply distance with sign pointwisely: HSDF X) Signx(p) Disx(p),
In the second row of Fig. 3, we design simple but effective optimization using the gradients of the predicted sign function Grad, V Viignx(D (p) and the gradients of the predicted distance function Grads ( to optimize the predicted signs points like
As described in Fig. 3, Gradg Gradd is negative as point A, then the sign will be optimized to negative and vice versa for optimizing point B. Since Grad, and Grady can be inferred directly using the backward operation of deep neural networks, this optimization is not time-consuming
3.4 Mesh Extraction
Our fused HSDF enables us to use efficient meshing methods like Marching Cubes. Hence, we propose an adaptive masked Marching Cubes algorithm. A 2D T-shirt example of fused HSDF is depicted on the left side in Fig. 4. Assume the green contour stands for the open surface, which disconnects at the collar, sleeves, and waist. The red/ blue region indicates where the sign is pos- itive/ negative.
A
Next, we set up Marching Cubes grid to mesh the surface S in green. All the grid points can be pushed to the underlying surface by the gradients of the distance function. Among them, take points like A, B for example. We define PA and Ри as the coordinate and B. Then the coordinate of the closest sur- face point of A (i.e. A') can be computed as Diis where PA PA Disx(PA) (PDis
Marching Cubes grid
Figure 4: A 2D illustration of mesh extraction our fused HSDF, Green contour indicates an open shape in 2D. We push Marching Cubes grid points like A and B to their closest points on the surface, i.e., A' and B', respectively, using the gradients of distance function (i.e. black vectors). All the boxes in red enclosing surfaces form a mask for Marching Cubes to extract the complex shapes into meshes.
A
[[VDiss(PPill represent unit vector towards the underlying surface. And we compute Pœ' and all other points by the same process. After all the grid points are pushed to the surface, we can easily pick out those boxes (in red) which enclose surfaces. And these selected cubes form a mask for us to find out where the surfaces lie (works for arbitrary shapes). Finally, simple masked Marching Cubes can be applied to our HSDF extract any complex shapes.
(Diss(PA)
Experiments
In this section, we validate HSDF on 3D shape reconstruction from sparse point clouds and demon- strate that HSDF can reconstruct watertight shapes par with SOTAs. And then we show that our approach outperforms SOTA techniques on reconstructing complex shapes such as cars with inner structures in terms of quality and speed.
4.1 Data Preparation and Metrics
HSDF Computation. The HSDF computation follows a similar manner as mesh-to-sdf [33]. For every spatial point p. find the nearest point p' on the mesh surface and its corresponding surface normal v. If the dot product of v and p' is negative, we assign its sign to be positive with its distance be Jp' p| and vice versa. Please refer the supplementary for further experimental details and results.
Computation for Complex Shapes. We compute HSDFs repaired ShapeNet dataset [19] on four typical categories with complex shapes, namely cars (7,497 models) with inner structures, chairs (6,579 models), lamps (2,316 models), and ships (1,851 models) with thin open surfaces. We also

---

(a) Input
(b) IF-Net
(c) NDF
(d) Ours
(c) GT
Figure 5: Comparison between different methods on reconstructing closed shapes (without inner structure) from point clouds. The first row and the third row are independent experiments. The second row visualizes the inner reconstruction of the example the first row. Input point number is 3, 000 for all the results.
attain HSDFs on 307 garments from MGN [5] because garments are also a typical category comprised of open surfaces.
Computation for Closed Shapes. order to compare the performance on watertight shapes, compute HSDFs 3,094 ShapeNet [7] cars, which have been converted to closed shapes by DISN [61]. If the model only contains closed surfaces, HSDF reduced to vanilla SDF, which also conforms to intuition, because HSDFs are the natural extension of SDFs represent wider range of shapes.
Metrics. We follow the commonly used reconstruction metrics to evaluate the reconstructed meshes' quality: Chamfer distance (CD) [3], Normal consistency (NC) [39] and Oriented Normal Consistency which takes face directions into account. OccNet [39] defines normal consistency score as the mean absolute dot product of the normals in one mesh and the normals at the corresponding nearest neighbors in the other mesh. In contrast, we define Oriented Normal Consistency as the mean dot product of the normals in one mesh and the normals at the corresponding nearest neighbors in the other mesh make it sensitive to flipped mesh faces. More implementation details demonstrated in supplemental materials, and We will release the code and data for facilitating future research.
4.2 Shape Fitting of Single Model
To demonstrate HSDFs can represent complex shapes with both closed and open surfaces, and can be learned by HSDF-Net, fit models using NDF [15] and our HSDF-Net. For NDF, extract meshes following the same process as NDF [15], namely using the Ball-Pivoting algorithm to mesh dense point clouds generated by trained NDF.
shown in Fig. 1, NDF results in poor qual- ity. The holes and flipped faces cannot be easily optimized by post-processing techniques like closing holes or recomputing normals. Our ap- proach can robustly reconstruct complex shapes with both open and closed surfaces shown in Fig. such as the dresses, plant leaves, and hu- man characters. We also compare the timing and storage consumption between two single-layer mesh prediction approaches: HSDF-Net with MCubes (ours) and NDF. As shown Table our approach outperforms NDF in time and storage budget.
Resolution
Cost
Method
64
128
256
NDF
89s
58m
780m
Time
Ours
5s
17s
95s
NDF
17M
64M
1276M
Storage
Ours
1M
3M
10M
Table 1: Comparison on average meshing time
and storage budget per model. Our HSDF can reconstruct single-layer meshes much faster with much lower storage budget.
4.3 Shape Reconstruction of Closed Surfaces

---

Datasct
MGN
Car
Chair
Ship
Lamp
Mcan
#Points
10K
3K
10K
3K
10K
3K
10K
3K
10K
3K
10K
3K
Input
0.879
1.70
0.683
1.91
0.814
2.42
0.327
0.933
0,350
1.08
0.611
1.61
Chamfer
W.GT
3.46
3.46
4.87
4.87
2.93
2.93
3.79
3.79
5.32
5.32
4.07
4.07
Distance
NDF
0.102
0.158
0.137
0.325
0.122
0.291
0.104
0.207
0.142
0.231
0.121
0.242
Ours
0.114
0.151
0.124
0.289
0.101
0.209
0.133
0.194
0.166
0.274
0.128
0.223
Normal
NDF
0.954
0.938
0.879
0.817
0.903
0.865
0.884
0.850
0.909
0.869
0.906
0.868
Consisteney
Ours
0.962
0.957
0.898
0.842
0.920
0.896
0.891
0.863
0.923
0.904
0.919
0.892
Oriented
NDF
0,517
0.390
0,383
0,310
0.423
0.344
0.445
0,358
0.541
0.505
0.462
0.381
NC
Ours
0.929
0.900
0.711
0.646
0.835
0.749
0.826
0.732
0.878
0.760
0.836
0.757
Table 2: Comparison on Chamfer distance x 10 10-1)000000 & Normal Consistency & Oriented Normal
Consistency which is sensitive to face normal directions.
Most of the SOTA such as OceNet [39] and IF-Net [14] require watertight training data. In
# Points
Input
OccNet
IF-Net
NDF
Ours
order to compare with these SOTA methods, train these methods along with NDF [15] on 3094 ShapeNet [7] cars pre-processed by DISN [61]. The pre-processing transform open shapes into closed shapes and removes all the interior contents of cars like seats. In Table 3, we compare the Chamfer Distance [21] of our method against all the baselines quantitatively with 300 input points and 3,000 input points. Our method achieves the best performance. In Fig. 5, we show that our HSDF' can achieve comparable quality as UDF-based SOTA [15] and SDF-based SOTA [14]. All the results are reported using test data unseen during training time.
300
12.5
2.53
2.76
1.51
1.34
3000
1.54
1.17
0.615
0.191
0.155
Table 3: Comparison on Watertight Cars (DISN dataset) using Chamfer- Distance(x10 (10 4)).
4.4 Shape Reconstruction of Complex Shapes
In order to demonstrate that HSDFs can reconstruct meshes with better quality efficiently, we train our HSDF-Net on five representative categories containing complex shapes illustrated in Sec. 4.1. In Table 2, we compare our method quantitatively against NDF and watertight ground truth (W.GT) on the complex shapes with 3,000 and 10,000 points as input. The vi- sual comparisons are shown in Fig. 6 where we can see thin open structures (i.e. the clothes, the back of the chair, and the hat of the lamp) and complex inner structures (i.e. the seats and windows, etc.). All these results are reported on test data unseen during training time. As depicted in Fig. 6 and Fig. 1, our HSDF the first representation that can efficiently reconstruct high-fidelity single-layer open mesh with reasonable face normal directions. These advantages are only possible because 1) HSDFs can model open shapes by hybrid distance and sign; 2) HSDF-Net can learn the prior knowledge form of signs, which results in reasonable face di- rections and also enables using marching cubes to accelerate meshing; 3) Our two-head training man- ner enables using deep neural networks (known to represent continuous functions) learn HSDFs. Al- though we computed HSDFs using surface normal information in training data illustrated in Sec. 4.1, our HSDF-Net is still robust to noise like flipped faces in training data. We can even use the learned HSDF-Net to reconstruct meshes with more reason- able face directions than their ground truth with only point clouds as input, see Fig. 7.
face normal direction in the
(a) Ours
(b) GT
(c) Ours
(d) GT
Figure 7: Reconstructed meshes with face di- rections corrected. Above are two examples from test data. And they have more reasonable face directions even than their noisy ground truth meshes which have flipped faces. This shows that HSDF-Net learns pattern of face normal directions from the prior knowledge, which can be robust to small noises.
4.5 Further Discussions
8

---

(a) Input
(b) W.GT
(c) NDF
(d) Ours
(e) GT
Figure 6: Shape reconstruction comparison of complex shapes. Dark brown area represents the back-faces. (a) input point clouds with 3k points for clothes, chairs and lamps; and 10k points for cars to sample complex inner structures. (b) the watertight version of GT pre-processed by DISN [61], which erases the inner structures and closes the open surfaces. This is the upper bound of SDF based approaches. (c) reconstructed meshes using BPA [4] on NDFs [15]. The results of NDF suffers because BPA is very sensitive to ball-radius, which have to tuned per-shape [50]. (d) our meshes reconstructed with the proposed HSDF, which preserves the thin shapes in (e) GT.
Necessity of the two-head training strategy We validate the necessity of our two-predictor training strat- egy for sign and distance by jointly training a single decoder to predict signed distance instead of two de- coders. All the results are from test data and reconstructed in the same condition except for the number of de- coders. As can see in column (b) in Fig. 8, although the HSDFs for train- ing are sampled in the same way as two-head training, joint training can never learn HSDFs correctly because discontinuous function is beyond what single DNN can express. In contrast, our results in column (c) are a plausible open surface that is similar to the ground truth.
(a) Input
(b) Joint
(c) two-head
(d)
Figure 8: Ablation study two-head training. The recon- structed meshes with joint learning strategy are shown as column (b) where all the open structures like collars and sleeves are wrongly shaped with watertight surfaces.
(a) Direct fusing (b) Grad optimize
(c) GT
Figure 9; Ablation study on sign and distance fusing strategy illustrated in Fig. 3. The gradient optimized HSDF can reconstruct meshes with better surface quality shown in (b) compared to (a).
Necessity of HSDF fusion We val- idate the quality improvement by inte- grating HSDF fusion as described in

---

Sec. 3.3. As shown in Fig. 9, the gradient optimization consistently removes the Moire pattern and produces smoother results than direct fusing.
Limitations The first limitation of our approach is the reconstruction accuracy of mesh boundaries. The marching cube algorithm cuts through every cube to extract the surfaces. The extraction becomes ill- ill-posed when cubes enclose the boundary surface of an open shape, which results in non-mann non-manifold structures. We envision that increasing the resolution of boundary cubes will alleviate this problem by decreasing the boundary cubes' size. The second limitation is that we rely on consistent and clean normals of the training data to compute HSDFs. Datasets without clear normals like unprocessed ShapeNet [7] have to be repaired [19] before training.
5 Conclusions
We propose a novel hybrid signed and distance field (HSDF) and a dedicated learning model HSDF- Net an intuitive but useful extension of SDF in previous works. Through sufficient experiments, we have shown that this natural extension leads to a substantial increase in modeling capacities for a much wider range of complex shapes. And we further validate the proposed HSDF can reconstruct arbitrary shapes containing open surfaces efficiently with high fidelity and consistent surface normals. In future work, we expect to explore more applications, such reconstructions from 2D images to exploit our representational capacities. And we will implement our proposed approach in Jittor [27], which is fully just-in-time (JIT) compiled deep learning framework.
6 Broader Impact
The proposed HSDF representation and the dedicated learning strategy can serve as fundamental tool for modeling 3D shapes with arbitrary topologies including both open and closed surfaces. Hence, it can have a positive impact to research fields such as computer vision, computer graphics and human-computer interaction, etc. Specifically, due the enhanced representing and meshing ability of HSDF, our method can reconstruct a wider range of complex shapes with high fidelity and consistent surface normals from raw scanning. This could benefit various real-world applications. including 3D reconstruction tasks from point clouds, images or voxels with thin structures and open surfaces. However, during data collection for our model training, particular care must be taken to ensure that the privacy and security of the owners of private models are not violated.
Acknowledgments. This work was supported by CCF-Tencent Open Fund, the Beijing Municipal Natural Science Foundation for Distinguished Young Scholars (No. JQ21013), the National Natural Science Foundation of China (No. 62061 1 136007 and No. 61872440), the Royal Society Newton Advanced Fellowship (No. R2 192151) and the Youth Innovation Promotion Association CAS.
References
[1] Eman Ahmed, Alexandre Saint, Abd El Rahman Shabayek, Kseniya Cherenkova, Rig Das, Gleb Gusev, Djamila Aouada, and Bjöm Ottersten. Deep learning advances on different 3D data representations: A survey, arXiv preprint arXiv:1808 01462, 1, 2018. 2
[2] Matan Atzmon and Yaron Lipman. Sal: Sign agnostic learning of shapes from raw data. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 2565-2574, 2020. 3
[3] Harry G. Barrow, Jay M. Tenenbaum, Robert C. Bolles, and Helen C. Wolf. Parametric correspondence and chamfer matching: Two new techniques for image matching, In Raj Reddy, editor, Proceedings of the 5th International Joint Conference on Artificial Intelligence. Cambridge, MA, USA, August 22-25, 1977, pages 659 663. William Kaufmann, 1977. 7
[4] F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. The ball- pill-pivoting algorithm for surface reconstruction. IEFF Transactions on Visualization and Computer Graphics, 5(4):349-359, 1999. 2,
[5] Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll. Multi-garment net: Learning to dress 3d people from images. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 November 2, 2019, pages 5419-5429, IECE, 2019.
[6] Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond Euclidean data, IEEE Signal Processing Magazine, 34(4):18 42, 2017. 2
10

---

[7] Angel X. Chang. Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan. Qi-Xing Huang, Zimo Li. Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository. CoRR, abs/1512.0300 2015. 7, 8, 10
[8] Haiwei Chen, Shichen Liu, Weikai Chen, Hao Li, and Randall Hill. Equivariant point network for 3d point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14514 14523,, 2021. 3
[9] Weikai Chen, Cheng Lin, Weiyang Li, and Bo Yang. 3psdf: Three-pole signed distance function for learning surfaces with arbitrary topologies. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, June 2022. 3
[10] Zhiqin Chen, Andrea Tagliasacchi. Thomas Funkhouser, and Hao Zhang. Neural dual contouring. ACM Transactions on Graphics (Special Issue of SIGGRAPH), 41(4), 2022. 3
[11] Zhiqin Chen, Kangxue Yin, Matt Fisher, Siddhartha Chaudhuri, and Hao Zhang. BAE-NET: Branched autoencoder for shape segmenttttio.. In IEEE Int. Conf. on Computer Vision (ICCV), 2019. 3
[12] Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5939-5948, 2019. 2.3
[13] Julian Chibane, Thiemo Alldicck, and Gerard Pons-Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6970-6981, 2020.
[14] Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6970-6981, 2020. 3, 5,
[15] Julian Chibane, Aymen Mir, and Gerard Pons-Moll. Neural unsigned distance fields for implicit function lcarning. In Advances in Neural Information Processing Systems (NeurIPS), December 2020. 2, 4, 7, 8,
[16] Julian Chibane, Aymen Mir, and Gerard Pons-Moll. Neural unsigned distance fields for implicit function learning. arXiv preprint arXiv:2010.1 13938, 2020. 3
[17] Julian Chibane and Gerard Pons-Moll. Implicit feature networks for texture completion from partial 3d data. In European Conference on Computer Vision, pages 717-725. Springer, 2020. 3
[18] Christopher B Choy, Danfei Xu, Jun Young Gwak, Kevin Chen, and Silvio Savarese. 3D-R2N2: A unificd approach for single and multi-view 3D object reconstruction. In ECCV, pages 028-644. Springer, 2016.
[19] Lei Chu, Hao Pan, Yang Liu, and Wenping Wang. Repairing made meshes via visual driven global optimization with minimum intrusion. ACM Trans. Graph., 38(6):55::1 155:118, 2019. 6, 10
[20] Boyang Deng, John P Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, and Andrea Tagliasacchi. Nasa neural articulated shape approximation. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK. August 23 23-28, 2020, Proceedings. Part VII 16. pages 612-628. Springer, 2020. 3
[21] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3D object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and puttern recognition, pages 605-613, 2017. 8
[22] Lin Gao, Jie Yang. Tong Wu, Yu-Jie Yuan, Hongho Fu, Yu-Kun Lai, and Hao(Richard) Zhang. SDM-NET: Deep generative network for structured deformable mesh. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2019), (4::::23::1223:111 2019. 3
[23] Kyle Genova, Forrester Cole, Avncesh Sud, Aaron Sama, and Thomas A Funkhouser. Deep structured implicit functions. 2019.
[24] Thibault Groucix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. AtlasNet: A rapier-måchó approach learning 3D surface generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018. 3
[25] Benoit Guillard, Federico Stella, and Pascal Fua. Meshudf: Fast and differentiable meshing of unsigned distance field networks. arXiv preprint arXiv:2111.14499, 2021.
[26] Rana Hanocka, Amir Hertz, Noa Fish, Raja Giryes, Shachar Fleishman, and Daniel Cohen-Or. Meshenn: network with edge. ACM Transactions Graphics (TOG), 38(4):1-12, 2019.
(27] Shi-Min Hu, Dun Liang, Guo-Yo Ye Yang, Guo-Wei Yang, and Wen- Yang Zhou. Jittor: novel deep lcarning framework with meta opeaatope and unified graph exccution. Science China Information Sciences, (3(222103):1222 2020. 10
[28] Shi-Min Hu, Zheng- -ing Liu, Meng- Guo, Jun-Xiona Xiong Cai, Jiahui Huang. Tai-Jiang Mu, and Ralph R. Martin, Subdivision- based mesh convolution networks, 2021. 3
11

---

[29] Yunhong WANG Huiqun WANG, Di HUANG, Gridnet: efficiently learning deep hierarchical representa- tion for 3d point cloud understanding. Frontiers Computer Science, 16(1):16301,, 2022. 3
[30] Anastasia Ioannidou, Elisavet Chatzilari, Spiros Nikolopoulos, and Ioannis Kompatsiaris. Deep learning advances in computer vision with 3D data: A survey. ACM Computing Surveys 50 2017.
[31] Dominic Jack, Jhony K Pontes, Sridha Sridharan, Clinton Fookes, Sarch Shirazi, Frederic Maire, and Anders Eriksson. Learning free-form deformations for 3d object reconstruction, In Asian Conference on Computer Vision, pages 317-333. Springer, 2018. 3
[32] Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, Thomas Funkhouser, et al. Local implicit grid representations for 3d scenes. In Proceedings of the IEEEJCVE Conference on Computer Vision and Pattern Recognition, pages 6001-6010, 2020. 3
[33] Marian Kleineberg. Calculate signed distance fields for arbitrary meshes. https:/ pettps:/githhee /github..cccooo
mari
2021. 6
[34] Alon Lahav and Ayellet Tal. Mcshwalker: Deep mesh understanding by random walks. ACM Transactions Graphics (TOG), 39(6):113,, 2020. 3
[35] Hsueh-Ti Derek Liu, Vladimir G Kim, Siddhartha Chaudhuri, Noam Aigerman, and Alec Jacobson. Neural subdivision. arXiv preprint arXiv:2005.01819 2020. 3
[36] Qinqing Liu, Peng-Shuai Wang. Chunjiang Zhu, Blake Blumenfeld Gaines, Tan Zhu, Jinbo Bi, and Minghu Song. Octsurf: Efficient hierarchical voxel-based molecular surface representation for protein-ligan affinity prediction. Journal of Molecular Graphics and Modelling, 105:107865. 2021.
[37] Shichen I Liu, Tianye Li, Weikai Chen, and Hao Li. Soft rasterizer: A differentiable renderer for image-based reasoning. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 7708-7717, 2019.
[38] Daniel Maturana and Sebastian Scherer. VoxNet: A 3D convolutional neural network for real-time object recognition. In 2015 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 922-928. IEEE, 2015. 3
[39] Lars Meseheder, Michael Occhsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3D reconstruction in function space. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 4460-4470, 2019. 7,8
[40] Mateusz Michalkiewicz, Jhony K Pontes, Dominic Jack, Mahsa Baktashmotlagh, and Anders Eriksson. Deep level sets: Implicit surface representations for 3d shape inference. arXiv preprint arXiv: /rXiv:/901.06802 2019. 2, 3
[41] David R. Palmer, Dmitriy Smirnov, Stephanie Wang. Albert Chern and Justin M. Solomon. Deepcurrents: Learning implicit representations of shapes with boundaries. CoRR, abs/211 1,09383, 2021.
[42] Junyi Pan, Xiaoguang Han. Weikai Chen, Jiapeng Tang, and Kui Jia. Deep mesh reconstruction from single rgb images via topology modification networks. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages
2019. 3
[43] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. DeepSDF: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 165-174, 2019. 2,
[44] Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In European conference on computer vision (ECCV 2020), 2020. 3
[45] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages
2017.
[46] Charles Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnet++: Deep hicrarchical feature learning on point sets in a metric space. arXiv preprint (Tt:::7706024133 2017. 3
[47] Yi-Ling Qiao, Lin Gao, Paul Rosin, Yu-Kun Lai, Xilin Chen, et al. Learning on 3d meshes with laplacian encoding and pooling. IEEE Transactions on Visualization and Computer Graphics, 2020. 3
[48] Edoardo Remelli, Artem Lukoianov, Stephan Richter, Benoît Guillard, Timur Bagautdinov, Pierre Baque, and Pascal Fua. Meshsdf: Diffcrentiable iso-surface extraction. arXiv preprint arXiv:2006. 03997, 2020. 2,
3
[49] Shunsuke Saito, Zeng Huang. Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Fixel-alinned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 2304 2314, 2019. 3
12

---

[50] Rahul Venkatesh, Tejan Karmali, Sarthak Sharma, Aurobrata Ghosh, R. Venkatesh Babu, Laszlo A. Jeni. and Maneesh Singh. Deep implicit surface point prediction networks. In Proceedings the IEEE/CVF International Conference on Computer Vixion (ICCV), pages 12653-12662, October 2021. 2,
[51] Rahul Venkatesh, Tejan Karmali, Sarthak Sharma, Aurobrata Ghosh, R Venkatesh Babu, László A Jeni, and Maneesh Singh. Deep implicit surface point prediction networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2021.
[52] Rahul Venkatesh, Sarthak Sharma, Aurobrata Ghosh, Laszlo Jeni, and Maneesh Singh. Dude: Deep unsigned distance embeddings for hi-fidelity representation of complex 3d surfaces. arXiv preprint arXiv:2011.022 1.02770. 2020. 2,3
[53] Nanyang Wang. Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3D mesh models from single RGB images. In Proceedings of the European Conference on Computer Vision (ECCV), pages 52-67, 2018. 3
[54] Peng-Shuai Wang. Yang Liu, Yu-Xiao Guo, Chun- Yu Sun, and Xin Tong. O-enn: Octree-based convolu- tional neural networks for 3d shape analysis. ACM Transactions On Graphics (TOG), 36(4):1-11, 2017.
[55] Peng-Shual Wang, Yang Liu, and Xin Tong. Deep actree-based enns with output-guided skip connections for 3d shape and scene completion. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 266-267, 2020. 3
[56] Peng-Shuai Wang. Chun-Yu Sun, Yang Liu, and Xin Tong. Adaptive o-cnn: A patch-hased deep represen- tation of 3d shapes. ACM Transactions on Graphics (TOG), 37(6):1-11, 2018.
[57] Yue Wang. Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein. and Justin M Solomon. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog), -8- 2019.
[58] Yujie Wang, Yixin Zhuang, Yunzhe Liu, and Baoquan Chen. Mdisn: Learning multiscale deformed implicit fields from single images. Visual Informatics, 6(2):41-49 2022. 2
[59] Jiajun Wu, Chengkai Zhang. Tianfan Xue, William T Freeman, and Joshua B Tenenbaum. Learning a probabilistic latent space of object shapes via 3d anberative-advessatt modeling, In Advances in Neural Information Processing Systems, pages 82- 90, 2016. 3
[60] Yun-Peng Xiao, Yu-Kun Lai. Fang-Lue Zhang, Chunpeng Li, and Lin Gao. A survey on deep geometry learning: From a representation perspective. Computational Visual Media, 6(2): (2):113-133, 133, 2020. 2
[61] Qiangeng Xu, Weiyue Wang. Duygu Ceylan, Radomír Mech, and UIrich Neumann. DISN: deep implicit surface network for high-quality single-view 3d reconstruction. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alchh- Buc, Emily B. Fox, and Roman Gareet,, editors, Advances in Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NewrIPS 2019, December 14, 2019, Vancouver, BC, Canada, pages 490 500, 2019. 7, 8.
[62] Jie Yang, Kaichun Mo. Ku-Kun Lai, Leonidas J Guibas, and Lin Gao. DSG-Net: Learning disentangled structure and geometry for 3d shape generation. arXiv preprint arXiv:2008. 055400 2020.
[63] Jianglong Ye, Yuntao Chen, Naiyan Wang, and Xiaolong Wang. GIFS: neural implicit function for general shape representation. CoRR, 0bs/2204.07126. 2022. 3
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 4.5.
(c) Did you discuss any potential negative societal impacts of your work? [No] Our work focuses on 3D) shape reconstruction and modeling, which is not related to any potential negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
13

---

(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? Yess] We will attach our code in the supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please refer to supplementary material.
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [No] This not necessary in our experimental settings.
(d) Did you include the total amount compute and the type of resources used (e.g. type of GPUs, internal cluster, or cloud provider)? [Yes] Please refer to supplementary material.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]] See Section 4.1.
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "qOETV4zKql", "reviewer": "Reviewer_gpXi", "summary": "This paper proposes a hybrid distance field approach to represent 3D object surfaces, called HSDF. The main idea behind it is to utilize the advantage of the unsigned distance function and use another network head to predict the sign of the distance field for each point. The network structure is inherited from IF-net with additional heads. To extract the mesh from HSDF, the authors propose a masked Marching Cube algorithm to extract the iso-surface from a union of the local bounding boxes to handle the arbitrary topology of the 3D shape. The experiments conducted in ShapeNet demonstrate the HSDF achieves good performance compared with other methods.", "questions": "1. Several recent works with similar ideas exist, such as [A]. It would be better if the authors could discuss and compare the differences.\n\n2. Discussion about robustness. The input of all comparison methods is a point cloud. Suppose the point cloud contains some unexpected noise, what about the performance drop or influenced by the perturbation of inputs? \n\n3. The notation in this paper might need further improvement.\n\n    a. $Sign$ in line 192 is a signed distance function, I think it should be $\\text{HSDF}(\\mathbf{p}, \\mathbf{X}) = \\text{sign}(Sign_\\mathbf{x}(\\mathbf{p}))*Dis_\\mathbf{x}(\\mathbf{p})$ in line 213.\n\n    b. line 184 and line 186, the mapping should be a binary output {-1, 1} rather than an interval $[-1, 1]$?\n\n\n\n[A] 3PSDF: Three-Pole Signed Distance Function for Learning Surfaces with Arbitrary Topologies (CVPR'22)\n\n", "limitations": "The authors have discussed the limitation adequately. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. The design of HSDF is simple but effective. The experimental result and ablation study also demonstrate its effectiveness.\n2. The motivation of the proposed approach is reasonable and the idea is easy to follow.\n3. The gradient optimization idea in Sec 3.3 is thought-provoking.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "eaHAgmV46_-", "reviewer": "Reviewer_oe6G", "summary": "This paper proposes an implicit neural representation called Hybrid Sign and Distance Field (HSDF) for modeling surfaces with arbitrary topologies. It employs a neural network to regress unsigned and signed distance fields separately. It then fuses the two fields by taking the distance from the unsigned distance field and the sign of the signed distance field. In this way, the unsigned distance field allows modeling of open surfaces, and the signed distance provides the directions of the surfaces. They also simply modified the marching cube algorithm to a masked version to mesh their learned field.\n\nThe overall idea of this paper is simple and interesting. However, some motivations and arguments from the paper are problematic. The presentation quality of the paper could also be further improved.", "questions": "I am impressed by the numbers in Table 1. However, I have no idea what causes the difference? Could you please give more explanation about the time and memory? It would also be great to include IF-Net in this table as well.\n\n\n\n", "limitations": "The authors have mentioned some limitations in the text.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. I like the idea of modeling the surface by combing a signed distance field and an unsigned distance field, which enables arbitrary topology and surface direction modeling. \n2. The idea of using two network heads to regress two fields is reasonable, as the combined one is not continuous.\n3. According to Table1, the proposed method seems to be more time and memory efficient.\n\n\n\nWeaknesses:\n1. As stated in Line 30, the main motivation of the proposed method is that applying the marching cubes algorithm to UDF would convert all open surfaces into closed meshes. However, I don't think this point is valid, as we can post-process the generated meshes by Marching Cubes to represent open surfaces. Specifically, we can remove the faces in regions that do not contain the surface.\nAlternatively, we can also apply the masked marching cubes to UDF to only extract regions containing the surfaces.\nMoreover, NDF work has provided a method to output arbitrarily dense point clouds, which can then be used to estimate consistent normal direction accurately.\nAs a result, the point that \"marching cubes algorithm cannot be applied to UDF for open surfaces\" is very vague. I would highly suggest authors sell other benefits of learning a sign function and rewrite the paper in another way.\n\n2. In section 3.4, the role of the signed function is not described. How does HSDF enable marching cubes? Why do we need a mask version of marching cubes? These points are not stated in the text.\n\n3. Figure 5, for the NDF method, do you use the dense point cloud generation technique introduced in their paper? If points are dense enough, we will be able to estimate consistent normals, apply marching cubes, and flip the faces as a post processing for BPA. \n\n4. In table 3, what's the meaning of only sampling 300 points. Much more points could be sampled to get a more accurate estimate.\n\n5. In Table 2, it would also be great to include a direction-sensitive normal consistency (original version).\n\n6. It would be better to include more recent UDF approaches in comparison.\n\n7. The presentation could be improved a lot:\n\n(a) Line 178: \"closest surface normal direction\" is confusing.\n\n(b) Line 183: What does $\\mathbb{R}_{0}$ mean?\n\n(c) Line 184: $\\mathbb{R}^{3} \\mapsto[-1,1]$ should be {-1, 1} instead of an interval.\n\n(d) Line 186: Please explicitly explain $\\mathcal{S}_{\\mathrm{x}}$.\n\n(e) Line 192: For the signed distance, why is the clamping in one direction?\n\n(f) Line 238: watertight meshes -> watertight meshes and open surfaces?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "UiBkByyU-hq", "reviewer": "Reviewer_nGbx", "summary": "As opposed to SDFs, this submission proposes to regress sign and distance separately, and extend sign to oriented open (ie. non watertight) meshes. Authors demonstrate it can represent both open and close surfaces, and propose a way to mesh it and merge sign and distance fields that reduces inaccuracies of both components.\n", "questions": "- In 3.1, how is the surface normal v computed? does it assume the surface is meshed, and uses triangle winding numbers? This is a crucial aspect of the submission that should be made clearer.\n- Why is the propose \"sign and distance fusion\" procedure required? Which field is noisy? Both?\n- Is Tab. 1 measuring meshing time only? or both network forward pass and meshing time? Is NDF using ball pivoting here?\n- Loss term L_s : why clamp a binary sign value? I guess there is a typo here, what is the real loss term?\n- While the introduction focuses on the generative/representation aspect of 3D shapes, the method section starts with an input point cloud. Could the shape representation be demonstrated in other settings, like auto-decoder?\n- What are the \"adversarial impacts\" of SDF and UDF mentioned in the introduction?", "limitations": "Authors have identified mesh boundaries and consistent normal orientations as main limitations, but do not quantify it. For example:\n- would a metric computed only on mesh borders really show that boundaries are not well represented?\n- across the main 3D datasets, which ones have consistent normals? What are pathological cases (Moebius strip)?", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- In the experiment section, metrics include both a surface proximity (CD) and smoothness (NC) components, and are in favor of the proposed approach agains NDF.\n- Results look visually appealing.\n- Ablations demonstrate the effectiveness of the sign+distance fusion.\n\n\nWeaknesses:\n- The main limitation is that learning a prior on triangle facets orientation requires datasets to be properly oriented, which might not always be possible. This huge limitation is only slightly mentioned in Fig. 7 but should be discussed more thoroughly and sooner.\n- The current formulation reads as if factoring SDFs into distance times sign was making the field easier to be learned by MLPs because it is more continuous. But in fact the sign network is highly non continuous.\n- The intro states that \"...\" , but this is not true, see NDF (Chibane 2020) for example. The proposed \"Masked Marching Cubes\" could be applied to normal SDFs, and should also be demonstrated in this case.\n- Wording: I do not find the word \"hybrid\" well chosen to describe the proposed representation. \"Factorised\" or \"Disentangled\" would be more appropriate.\n- The first 3 lines of the introduction only mention SDFs and forget about Occupancy Fields.\n- The \"sign and distance fusion\" is not motivated by any intuition.\n- Tab.2 evaluates oriented normal consistency for methods that are not supervised for it. This is a biased assessment.\n- Some references about representing/meshing open surfaces with UDFs are missing, such as \"Neural Dual Contouring\" or \"MeshUDF\". \"DUDE\" is mentioned, but not used as a baseline in the experiment section.\n- English quality could be improved.\n\nMinor: figures always appear 1 page before their reference in the text.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept