Shape as Line Segments: Accurate and Flexible Implicit Surface Representation

Authors: Siyu Ren, Junhui Hou
Primary area: learning on graphs and other geometries & topologies
Keywords: Implicit representation, 3D geometric, surface reconstruction

Abstract:
Distance field-based implicit representations like signed/unsigned distance fields have recently gained prominence in geometry modeling and analysis. However, these distance fields are reliant on the closest distance of points to the surface, introducing inaccuracies when interpolating along cube edges during surface extraction. Additionally, their gradients are ill-defined at certain locations, causing distortions in the extracted surfaces. To address this limitation, we propose Shape as Line Segments (SALS), an accurate and efficient implicit geometry representation based on attributed line segments, which can handle arbitrary structures. Unlike previous approaches, SALS leverages a differentiable Line Segment Field to implicitly capture the spatial relationship between line segments and the surface. Each line segment is associated with two key attributes, intersection flag and ratio, from which we propose edge-based dual contouring to extract a surface. We further implement SALS with a neural network, producing a new neural implicit presentation. Additionally, based on SALS, we design a novel learning-based pipeline for reconstructing surfaces from 3D point clouds. We conduct extensive experiments, showcasing the significant advantages of our methods over state-of-the-art methods.
The source code is available at https://github.com/rsy6318/SALS.

Decision: accept (poster)
Reviewer scores: 6, 8, 6, 6

Reviews and discussion:
<thread>
creation date:
1730282323671

thread_type:
official_review

<thread_object>
creator: reviewer_ea2y

<content>
summary:
The paper presents a new approach for implicitly encoding 3D surfaces. The idea is to encode how an arbitrary line segment intersects a surface, represented by a binary variable showing intersection or not, and a scalar within [0,1] showing the ratio of the intersection point along the line segment. The motivation for using such a line segment representation is that to extract surfaces from implicit fields, one usually has to find such intersection points for regular grid edges, as done in marching cubes and dual contouring. The paper does analysis of this line segmentation representation, presents a dual contouring adaptation for this representation, develops a learning based network design for generating such a representation, and conducts experiments for shape encoding and shape reconstruction from point clouds, where the method shows improved accuracy than alternative representations like SDF and UDF. 

The idea makes sense intuitively by directly catering for the surface extraction step when learning the implicit rep, and is novel as far as I know. Given the novelty of the approach, some questions can be answered to better understand the approach. See the questions below.

strengths:
The idea of representing shapes as intersections with respect to arbitrary edges is pursued and turned into a concrete design, and shows improved accuracy than alternative distance field representations due to its direct compliance with surface extraction algorithms like dual contouring.

The paper is well written and easy to read overall.

The paper shows the advantage of the new representation in two different tasks, i.e. shape encoding and shape reconstruction from point clouds. The dataset used is also challenging and diverse, including nonmanifold CAD shapes, scenes and open boundary shapes.

weaknesses:
There remain some key technical questions that can be explored for this novel representation.

- How to handle cases where a line segment should intersect the surface multiple times? Such cases should be common when the line segment is larger than the local feature size.

- I am not sure the proof for Theorem 1 is correct. Why are p and n not functions of u/v? What assumptions have to be made about the surface, to ensure this invariance to u/v?

- Eq(2), should there be a square of the dot product? Otherwise the minimization is not bounded from below.

- Sec.3.3, it's said that surface normal is determined as the gradient of s. But the gradient is taken with respect to which variable? u or v? What if they give different gradients?

- Related to the above question, the network producing the line-segment field should have certain invariance and equi-variance. That is, with respect to the switch of u/v, its output o (binary intersection) should be invariant, and its output s (ratio) should be equivariant. But the network design in its current form seems not preserve such properties.

The results frequently show holes that are undesirable. Such holes would not appear for SDF based methods. What can be done to fix such issues?

The ablation about normal vectors (Sec.4.3) is quite obscure, with only one visual example and minor differences. Can quantitative differences be observed with or without normal vector for dual contouring?

questions:
See weakness section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730372386862

thread_type:
official_review

<thread_object>
creator: reviewer_ssdj

<content>
summary:
This paper introduces a novel method for sharper surface reconstruction. It proposes to model the implicit surfaces as a field of line segments, in comparison to the previous fields for points.

strengths:
- A novel implicit representation of surfaces based on line segments is proposed. 
- For the novel line segments field, surfaces are extracted with edge-based dual contouring.
- The LSF-method is extend to surface reconstruction from point clouds. 
- Experimental demonstrations are provided for manifold and non-manifold structures.

weaknesses:
- Quantitative comparison are only provided for a limited amount of shapes in ABC dataset. 
- The improvement in surface quality appears incremental. The line segment field uses query grids of $128^3$. What is the counterpart resolution of point queries for the compared methods? 
- Given the method’s focus on sharper surface reconstruction, it is recommended that the authors also report metrics evaluated specifically around surface edges, similar to those used in ``Neural Dual Contouring (Chen et al., SIGGRAPH 2022)''.

questions:
- For the applications to Waymo data, are the data also rescaled to a unit cube?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730793366014

thread_type:
official_review

<thread_object>
creator: reviewer_1tcg

<content>
summary:
The paper proposes Shape as Line Segments (SALS), an accurate and efficient implicit geometry representation based on attributed line segments, which can handle arbitrary structures. The authors then design a novel learning-based pipeline for reconstructing surfaces from 3D point clouds. The experiments show the method produces better results than the state-of-the-art methods.

strengths:
- The paper's contributions are clearly defined, with comprehensive experimental validation on several benchmark datasets.

- Paper writing is clear and it’s easy to read.

weaknesses:
- Some results exhibit only minor differences, as seen in Figure 8, where the outcomes of Ours and GeoUDF appear quite similar. Presenting additional results, such as error maps or normal maps, may provide clearer visual distinctions.

- Lack of analysis of limitations. It would be interesting to evaluate how the proposed method performs on thin structures, such as leaves, or particularly complex geometric structures, like a basket, to better understand its applicability to different types of shapes.

questions:
- I suggest add normal maps, which may highlight the differences more clearly in certain results, such as in Fig. 11.
- Some related works need to be added. [1-3].

- In the network design, I’m interested in exploring the results of replacing max pooling with mean pooling, as it may provide more information of neighbors in feature aggregation.

- Discuss the limitations.

[1] Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction
[2] NeUDF: Learning Neural Unsigned Distance Fields with Volume Rendering
[3] NeAT: Learning neural implicit surfaces with arbitrary topologies from multi-view images
</content>
</thread_object>

</thread>
<thread>
creation date:
1730804365918

thread_type:
official_review

<thread_object>
creator: reviewer_hfqz

<content>
summary:
The paper presents a learnable surface extraction method based on edge-based dual contouring. Similar to Neural Dual Contouring (NDC), it uses a network to predict an intersection or crossing flag per edge. However, unlike NDC, it optimizes vertex locations within each cell using a Quadratic Error Function (QEF) rather than predicting them directly through the network. This optimization requires both the position of the intersection along each edge and its normal direction. Consequently, in addition to predicting the intersection flag, the network is also trained to output a scalar value representing the intersection position on the edge, using a concatenation of the corresponding cell vertices as input. Instead of the normal direction the paper uses the gradient of this intersection scalar  which is shown to  align with the the normal direction.

Extensive experiments validate the method’s effectiveness, leveraging the ABC and non-manifold ABC datasets to evaluate implicit representation in addition to DeepFashion3D, Synthetic Rooms, and Waymo datasets to illustrate generalization of  models trained with a subset of Thingi10k.

strengths:
- The paper is well-written and easy to follow.
- The focus on edges (segments) rather than points for predicting intersection positions, combined with the novel approach of linking the gradient with the surface normal, effectively leverages QEF optimization for vertex positioning.
- **Evaluation and performance:** The evaluations are thorough, showing clear improvements over state-of-the-art methods.

weaknesses:
- **Motivation and comparison with Neural Dual Contouring (NDC):** The paper could better clarify its approach relative to NDC, particularly why optimizing vertex locations with QEF outperforms direct network predictions.
- **Training and inference times:** A table comparing performance, training time, and inference time would provide clarity. The method appears slower in inference than others; for instance, NDC achieves sub-second inference at a 128 resolution. Then, the question is  how good is  the trade-of between performance and inference time of the proposed method ?
- **Dependence on K-NN size for point cloud surface reconstruction:** The paper lacks details on how the K-NN value is chosen for each dataset and point cloud density, which would help to understand its impact on performance.

questions:
- How good is the trade-of between performance and inference time of the proposed method compared to other methods?
- How the K-NN value is chosen for each dataset and point cloud density, which would help to understand its impact on performance?
- To solve the QEF optimization, the gradient the network w.r.t e one of the edge vertices should be used. Which one do you use? Also since they have different orientations how does this choice impact the stability of the optimization and the overall performance?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732346419156

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We sincerely thank the reviewers for the valuable time and effort in evaluating our work and recognizing its merits. We have carefully addressed the raised concerns and updated the manuscript accordingly (as reflected in the revised PDF). The modifications are highlighted in **red**, and the key improvements include:

* Sec. 2: Added a discussion on related works for implicit fields in the context of novel view synthesis.
* Sec. 3.1: **Enhanced** the proof of **Theorem 1** and updated Fig. 3.
* Sec. 3.2: Added an illustration explaining how to use Theorem 1 to compute the **normal vector** at the intersection point on a given line segment and the difference from Dual Contouring-based methods, NDC and DualMesh.
* Sec. 3.3: Included a detailed explanation of how a line segment is **input to the MLP**.
* Sec. 4.2 and 4.3: Detailed the experimental settings, including the **resolutions** of Marching Cubes, Dual Contouring, and their variants used in baseline methods for surface extraction.
* Fig. 11: Added **normal maps** for the reconstructed results.
* Appendix Sec. C: Expanded with additional details, including **network architecture**, solutions for surface **holes**, **error maps** for reconstructed results in Fig. 6, experimental results **validating** Theorem 1, evaluation of **sharp edges**, and **efficiency analysis**.
 * Appendix Sec. D: Added **error maps** for results in Fig. 8, evaluation of **sharp edges**, analysis of reconstruction accuracy for **K-NN patch sizes** under varying point cloud densities, results on variant datasets, and **efficiency analysis**.
* Appendix Sec. E: Discussed the **limitations** of our method.

We appreciate the reviewers' insights, which significantly helped us refine and strengthen our work.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732524320534

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
The authors are looking forward to Reviewers' feedback. Thanks for your time.

comment:
Dear **Reviewers**,

Thanks for your time and effort in reviewing our manuscript. In our previous response, we have addressed your comments directly and comprehensively. We are nearing the deadline for the discussion phase between the reviewers and authors. The authors understand that you may have a very busy schedule and would appreciate any further feedback you could provide.

Best regards, The authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a novel approach to represent surfaces and reconstruct them from point clouds. Specifically, the paper proposes line-segment fields, which model surfaces through an implicit function that can be queried for intersections with the surface and the intersection ratio between end points given a line segment. The paper further proposes an approach to surface extraction from the line segment fields based on dual contouring. Comparisons against representative baselines demonstrate state-of-the-art performance. The reviewers unanimously recommended to accept the paper and the AC agrees.