Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes an implicit neural representation called Hybrid Sign and Distance Field (HSDF) for modeling surfaces with arbitrary topologies. It employs a neural network to regress unsigned and signed distance fields separately. It then fuses the two fields by taking the distance from the unsigned distance field and the sign of the signed distance field. In this way, the unsigned distance field allows modeling of open surfaces, and the signed distance provides the directions of the surfaces. They also simply modified the marching cube algorithm to a masked version to mesh their learned field.

The overall idea of this paper is simple and interesting. However, some motivations and arguments from the paper are problematic. The presentation quality of the paper could also be further improved.
strengths_and_weaknesses: Strengths:
1. I like the idea of modeling the surface by combing a signed distance field and an unsigned distance field, which enables arbitrary topology and surface direction modeling. 
2. The idea of using two network heads to regress two fields is reasonable, as the combined one is not continuous.
3. According to Table1, the proposed method seems to be more time and memory efficient.



Weaknesses:
1. As stated in Line 30, the main motivation of the proposed method is that applying the marching cubes algorithm to UDF would convert all open surfaces into closed meshes. However, I don't think this point is valid, as we can post-process the generated meshes by Marching Cubes to represent open surfaces. Specifically, we can remove the faces in regions that do not contain the surface.
Alternatively, we can also apply the masked marching cubes to UDF to only extract regions containing the surfaces.
Moreover, NDF work has provided a method to output arbitrarily dense point clouds, which can then be used to estimate consistent normal direction accurately.
As a result, the point that "marching cubes algorithm cannot be applied to UDF for open surfaces" is very vague. I would highly suggest authors sell other benefits of learning a sign function and rewrite the paper in another way.

2. In section 3.4, the role of the signed function is not described. How does HSDF enable marching cubes? Why do we need a mask version of marching cubes? These points are not stated in the text.

3. Figure 5, for the NDF method, do you use the dense point cloud generation technique introduced in their paper? If points are dense enough, we will be able to estimate consistent normals, apply marching cubes, and flip the faces as a post processing for BPA. 

4. In table 3, what's the meaning of only sampling 300 points. Much more points could be sampled to get a more accurate estimate.

5. In Table 2, it would also be great to include a direction-sensitive normal consistency (original version).

6. It would be better to include more recent UDF approaches in comparison.

7. The presentation could be improved a lot:

(a) Line 178: "closest surface normal direction" is confusing.

(b) Line 183: What does $\mathbb{R}_{0}$ mean?

(c) Line 184: $\mathbb{R}^{3} \mapsto[-1,1]$ should be {-1, 1} instead of an interval.

(d) Line 186: Please explicitly explain $\mathcal{S}_{\mathrm{x}}$.

(e) Line 192: For the signed distance, why is the clamping in one direction?

(f) Line 238: watertight meshes -> watertight meshes and open surfaces?
questions: I am impressed by the numbers in Table 1. However, I have no idea what causes the difference? Could you please give more explanation about the time and memory? It would also be great to include IF-Net in this table as well.




limitations: The authors have mentioned some limitations in the text.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: As opposed to SDFs, this submission proposes to regress sign and distance separately, and extend sign to oriented open (ie. non watertight) meshes. Authors demonstrate it can represent both open and close surfaces, and propose a way to mesh it and merge sign and distance fields that reduces inaccuracies of both components.

strengths_and_weaknesses: Strengths:
- In the experiment section, metrics include both a surface proximity (CD) and smoothness (NC) components, and are in favor of the proposed approach agains NDF.
- Results look visually appealing.
- Ablations demonstrate the effectiveness of the sign+distance fusion.


Weaknesses:
- The main limitation is that learning a prior on triangle facets orientation requires datasets to be properly oriented, which might not always be possible. This huge limitation is only slightly mentioned in Fig. 7 but should be discussed more thoroughly and sooner.
- The current formulation reads as if factoring SDFs into distance times sign was making the field easier to be learned by MLPs because it is more continuous. But in fact the sign network is highly non continuous.
- The intro states that "..." , but this is not true, see NDF (Chibane 2020) for example. The proposed "Masked Marching Cubes" could be applied to normal SDFs, and should also be demonstrated in this case.
- Wording: I do not find the word "hybrid" well chosen to describe the proposed representation. "Factorised" or "Disentangled" would be more appropriate.
- The first 3 lines of the introduction only mention SDFs and forget about Occupancy Fields.
- The "sign and distance fusion" is not motivated by any intuition.
- Tab.2 evaluates oriented normal consistency for methods that are not supervised for it. This is a biased assessment.
- Some references about representing/meshing open surfaces with UDFs are missing, such as "Neural Dual Contouring" or "MeshUDF". "DUDE" is mentioned, but not used as a baseline in the experiment section.
- English quality could be improved.

Minor: figures always appear 1 page before their reference in the text.
questions: - In 3.1, how is the surface normal v computed? does it assume the surface is meshed, and uses triangle winding numbers? This is a crucial aspect of the submission that should be made clearer.
- Why is the propose "sign and distance fusion" procedure required? Which field is noisy? Both?
- Is Tab. 1 measuring meshing time only? or both network forward pass and meshing time? Is NDF using ball pivoting here?
- Loss term L_s : why clamp a binary sign value? I guess there is a typo here, what is the real loss term?
- While the introduction focuses on the generative/representation aspect of 3D shapes, the method section starts with an input point cloud. Could the shape representation be demonstrated in other settings, like auto-decoder?
- What are the "adversarial impacts" of SDF and UDF mentioned in the introduction?
limitations: Authors have identified mesh boundaries and consistent normal orientations as main limitations, but do not quantify it. For example:
- would a metric computed only on mesh borders really show that boundaries are not well represented?
- across the main 3D datasets, which ones have consistent normals? What are pathological cases (Moebius strip)?
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a hybrid distance field approach to represent 3D object surfaces, called HSDF. The main idea behind it is to utilize the advantage of the unsigned distance function and use another network head to predict the sign of the distance field for each point. The network structure is inherited from IF-net with additional heads. To extract the mesh from HSDF, the authors propose a masked Marching Cube algorithm to extract the iso-surface from a union of the local bounding boxes to handle the arbitrary topology of the 3D shape. The experiments conducted in ShapeNet demonstrate the HSDF achieves good performance compared with other methods.
strengths_and_weaknesses: 1. The design of HSDF is simple but effective. The experimental result and ablation study also demonstrate its effectiveness.
2. The motivation of the proposed approach is reasonable and the idea is easy to follow.
3. The gradient optimization idea in Sec 3.3 is thought-provoking.
questions: 1. Several recent works with similar ideas exist, such as [A]. It would be better if the authors could discuss and compare the differences.

2. Discussion about robustness. The input of all comparison methods is a point cloud. Suppose the point cloud contains some unexpected noise, what about the performance drop or influenced by the perturbation of inputs? 

3. The notation in this paper might need further improvement.

    a. $Sign$ in line 192 is a signed distance function, I think it should be $\text{HSDF}(\mathbf{p}, \mathbf{X}) = \text{sign}(Sign_\mathbf{x}(\mathbf{p}))*Dis_\mathbf{x}(\mathbf{p})$ in line 213.

    b. line 184 and line 186, the mapping should be a binary output {-1, 1} rather than an interval $[-1, 1]$?



[A] 3PSDF: Three-Pole Signed Distance Function for Learning Surfaces with Arbitrary Topologies (CVPR'22)


limitations: The authors have discussed the limitation adequately.
review_title: null

Meta-review: metareview: The reviewers agree that the paper's idea to include both sign and distance fields is a valuable contribution to 3D computer vision research.

Reviewers ask sensible clarifying questions (e.g. orienting the training data, sign network continuity) and the rebuttal's answers are illuminating and to the point.

A short notice on terminoloy: I agree that "adversarial" should not be used here as it has a special meaning for the wider NeurIPS audience.  Regarding other wording suggestions, I add no extra vote for or against.

confidence: Less certain
Decision: Accept