PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds
Authors: Yujia Liu
Stefano D'Aronco
Konrad Schindler
Jan Dirk Wegner
Venue: ICLR 2021 Poster (2021)
Keywords: deep neural network
3d point cloud
wireframe model

Abstract: We introduce PC2WF, the first end-to-end trainable deep network architecture to convert a 3D point cloud into a wireframe model. The network takes as input an unordered set of 3D points sampled from the surface of some object, and outputs a wireframe of that object, i.e., a sparse set of corner points linked by line segments. Recovering the wireframe is a challenging task, where the numbers of both vertices and edges are different for every instance, and a-priori unknown. Our architecture gradually builds up the model: It starts by encoding the points into feature vectors. Based on those features, it identifies a pool of candidate vertices, then prunes those candidates to a final set of corner vertices and refines their locations. Next, the corners are linked with an exhaustive set of candidate edges, which is again pruned to obtain the final wireframe. All steps are trainable, and errors can be backpropagated through the entire sequence. We validate the proposed model on a publicly available synthetic dataset, for which the ground truth wireframes are accessible, as well as on a new real-world dataset. Our model produces wireframe abstractions of good quality and outperforms several baselines.

Published as a conference paper at ICLR 2021
PC2WF: 3D WIREFRAME RECONSTRUCTION FROM RAW POINT CLOUDS
Yujia Liu, Stefano D'Aronco, Konrad Schindler, Jan Dirk Wegner Eco Vision Lab, Photogrammetry and Remote Sensing, ETH Zürich (fi rsstname lastname) ) @geed. baug. et.hz.
ABSTRACT
We introduce PC2WF, the first end-to-ent trainable deep network architecture convert a 3D point cloud into a wireframe model. The network takes as input an unordered set of 3D points sampled from the surface of some object, and outputs a wireframe of that object, i.e., sparse set of corner points linked by line segments. Recovering the wireframe a challenging task, where the numbers of both vertices and edges different for every instance, and a-priori unknown. Our architecture gradually builds up the model: It starts by encoding the points into feature vectors. Based on those features, it identifies pool of candidate vertices, then prunes those candidates to final set of corner vertices and refines their locations. Next, the corners linked with exhaustive set of candidate edges, which is again pruned to obtain the final wireframe. All steps are trainable, and errors can be backpropagated through the entire sequence. We validate the proposed model on a publicly available synthetic dataset, for which the ground truth wireframes are accessible, as well as on new real-world dataset. Our model produces wireframe abstractions of good quality and outperforms several baselines.
INTRODUCTION
Many practical 3D sensing systems, like stereo cameras or laser scanners, produce unstructured 3D point clouds. That choice of output format really just "smallest common denominator", the least committal representation that can be reliably generated with low-level signal processing. Most users would prefer a more efficient and more intuitive representation that describes the scanned object's geometry as compact collection of geometric primitives, together with their topological relations. Specifically, many man-made objects are (approximately) polyhedral and can be described by corners, straight edges and/or planar surfaces. Roughly speaking there are two ways to abstract point cloud into a polyhedral model: either find the planar surfaces and intersect them to find the edges and corners, e.g.. [Schnabel et al. (2007); Fang al. (2018); Coudron al. (018); directly find the salient corner and/or edges, e.g., Jung et al. (2016) Hackel et al. (2016).
A wireframe is graph representation of object's is shape, where vertices correspond to corner points (with high Gaussian curvature) that are linked by edges (with high principal curvature). Wireframes are a good match for polyhedral structures like mechanical parts, furniture or building interiors. In particular, since wireframes focus on the edge structure, they are best suited for piece-wise smooth objects with few pronounced crease edges (whereas they are less suitable for smooth objects without defined edges or for very rough ones with edges everywhere). We emphasise that wireframes are not only "compression technique" to save storage. Their biggest advantage in many applications is that they are easy to manipulate and edit, automatically or interactively in CAD software, because they make the salient contours and their connectivity explicit. Reconstructed wireframes can drive and help create 3D CAD models for manufacturing parts, metrology, quality inspection, well as visualisation, animation, and rendering.
Inferring the wireframe from noisy point cloud is challenging task. We can think of the process as a sequence of steps: find the corners, localise them accurately (as they are not contained in the point cloud), and link them with the appropriate edges. However, these steps are intricately correlated. For example, corner detection should "know" about the subsequent edge detection: curvature is affected by noise (as any of interest point detector testify), qualify corner

---

Published as a conference paper at ICLR 2021
3D location should also be the plausible meeting point of multiple, non-collinear edges. Here we introduce PC2WF, an end-to-end trainable architecture for point cloud to wireframe conversion. PC2WF is composed of sequence of feed-forward blocks, see Fig. for the individual steps of the pipeline. First a "backhone" block extracts a latent feature encoding for each point. The next block identifies local neighbourhoods ("patches") that contain corners and regresses their 3D locations. The final block links the corners with an overcomplete set of plausible candidate edges, collects edge features by pooling point features along each edge, and uses those to prune the candidate set to final set of wireframe edges. All stages of the architecture that have trainable parameters support back- propaaaapagannn such that the model can be learned end-to-end. We validate PC2WF with models from the publicly available ABC dataset of CAD models, and with new furniture dataset collected from the web. The experiments highlight the benefit of explicit, integrated wireframe prediction: on the one hand, the detected corners are more complete and more accurate than with generic corner detector; on the other hand, the reconstructed edges are more complete and more accurate than those found with existing (point-wise) edge detection methods.
2 RELATED WORK
Relatively little work exists on the direct reconstruction of 3D wireframes. We review related work for the subtasks of corner and edge detection, as well as works that reconstruct wireframes in 2D or polyhedral surface models and geometric primitives in 3D.
Corner Detection. Cornrr points play a fundamental role in both 2D and 3D computer vision. A 3D corner is the intersection point of > (locally) non-collinear surfaces. Corners have well-defined location and a local neighbourhood with non-trivial shape (e.g., for matching). Collectively the set of corners often is sufficient to tightly constrain the scale and shape of an object. Typical local methods for corner detection are based on the 2nd moment matrix of the points (resp. normals) in a local neighhourhood (Glomb) 2009; Sipiran & Bustos 2010, (201T), a forward generalisation of the 2D Harris operator first proposed for 2D images (Harris et al. 1988). Also other 2D corner
detectors like Susan (Smith & Brady 1997) have been generalised to 3D (Walter al., 2009)
Point-level Edge Point Detection A large body of work is dedicated to the detection of 3D points that lie on or near edges. The main edge feature is high local curvature (Yang & Zang, 2014). Related features, such as symmetry (Ahmed et 2018). change of normals (Demarsin et al.] (007). and eigenvalues (Bazazian et al.] [2015] & Wang. 2017) can also be used for edge detection. Haakel et al. (0166 2017) use combination of several features as input for a binary edge-point classifier. After linking the points into an adjacency graph, graph filtering can be used to preserve only edge points (Chen et al., (017)). EC-Net (Yu et al. 2018) is the first method we know of that detects edge points in 3D point clouds with deep learning.
Learning Structured Models. Many methods have been proposed to abstract point clouds into more structured 3D models, e.g., triangle meshes (Lin et al. 2004 Remondino 2003 Lin et al. 2004).
polygonal surfaces (Fang et al. [0188] Coudron et al. 2018), 8), simple parametric surfaces (Schnabel ct al. 2007] 2008 Li et al. (201)). From the user perspective, the ultimate goal would be to reverse- engineer CAD model (Gonzalez-Aguilera
al. 2012 Li et al., 2017b; Durupt et al. 2008).
Several works have investigated wireframe parsing in 2D images (Huang et al. 2018] Huang & Gao 2 LCNN (Zhou al. 2019a) is an end-to-end trainable system that directly outputs a vectorised wireframe. The current state-of-the-art appears to be the improved LCNN of Xuc al. (2020), which uses holistic "attraction field map" characterise line segments. Zou al. (2018) estimate the 3D layout of an indoor scene from single perspective or panoramic image with an encoder-decoder architecture, while Zhou al. (2019b) obtain a compact 3D wireframe representation from a single image by exploiting global structural regularities. Jung et al. (2016) reconstruct 3D wireframes of building interiors under a Manhattan-world assumption. Edges are detected heuristically by projecting the points onto 2D occupancy grids, then refined via least-squares fitting under swralleiism/orhhooonaiit constraints. Wireframe extraction in 2D images is simplified by the regular pixel grid. For irregular 3D points the task is more challenging and there has been less work.
Another related direction of research is 3D shape decomposition into sets of simple, parameterised primitives (Tulsiani et al.) (017)) Li et al. 2017a] Zou et al. [017]] [Sharma et al., 2018 Du et al. 2018). Most approaches use coarse volumetric occupancy grid input for the 3D data, which
2

---

Published as a conference paper at ICLR 2021
Vertex Detector
Vertex Localiser
Edge detector
M x C
PatchNNet
VertexNet
vertex1
EdgeNet
wireframe
PatchNet
VertexNet
vertexz
EdgeNet
FOGF
edges
EdgeNet
shared
VertexNet
vertexx
backbone
shared
C
PatchNNet
features
patch samples
patches
predicted vartices
edge samples
edges predicted edges
Figure 1: Illustration of our wireframe modelling architecture.
works well for shape primitive fitting but only allows for rather inaccurate localisation of edges and vertices. Li et al.) (2019) propose method for fitting 3D primitives directly to raw point clouds. A limiting factor of their approach that only fixed maximum number of primitive instances can be detected. This upper bound is hard-coded in the architecture and prevents the network from fitting more complex structures. Nan & Wonka (2017) propose Polyfit extract polygonal surfaces from point cloud. Polyfit first selects an exhaustive set of candidate surfaces from the raw point cloud using RANSAC, and then refines the surfaces. Similarly to our PC2WF, Polyfit works best with objects made of straight edges and planar surfaces. The concurrent work Wang et al. (2020) proposed a deep neural network that trained to identify a collection of parametric edges.
In summary, most 3D methods are either hand-crafted for restricted, schematic settings, only learn label individual "edge points". Here we aim for an end-to-end learnable model that directly maps a 3D point cloud to vectorised wireframe.
METHOD
Our full architecture is depicted in Fig. I In the following describe each block and, where necessary. discuss training and inference procedures. We denote a point cloud of N 3D points with P = { where P: R33 represents the 3D coordinates of point i. The 3D wireframe model of the point cloud PP denoted by W = (V,&) and it is defined, analogously to an undirected graph, by of vertices V which are connected by of edges V x V. As the vertices represent points in the 3D space we can think of them as being described by 3D coordinates E V C R
Overall Pipeline. Our feed-forward pipeline consists of four blocks: (1) a convolutional backbone that extracts per-point features; (2) vertex detector that examines patches of the point cloud and decides which ones contain vertices; (3) vertex localiser that outputs the location of the vertex within a patch; (4) an edge detector that instantiates candidate edges and prunes them to wireframe.
Backbone. PC2WF first extracts feature vector per 3D point that encodes its geometric context. Our pipeline is compatible with any (hand-coue hhand-coded or learned) function that outputs a fixed-siee descriptor vector per point. We use Fully Convolutional Geometric Features (FCGF, Choy al. 2019b). which are compact and capture broad spatial context. The backbone operates on sparse tensors (Choy et al. 2019ah and efficiently computes 32-dimensional features single, 3D fully-convolutional pass.
Vertex Detector.
This block performs binary classification of local patches (neighbourhoods in the point cloud) into those (few) that contain a corner and those that do not. Only patches detected to contain corner are passed on to the subsequent block. While one could argue that detecting the presence of a vertex and inferring its 3D coordinates are two facets of the same problem, we found it advantageous to separate them. Detecting first whether patch is likely to contain corner, then constructing its 3D coordinates reduces the computational cost, as we discard patches without corners early. Empirically it also reduces false positives and increases localisation accuracy.
3

---

Published as a conference paper at ICLR 2021
Our patch generation is similar to EC-Net Yuu et al. 2018). A patch contains a fixed number M of points, chosen by picking a patch centroid and finding the closest (M - points in terms of geodesic distance on the underlying surface. The geodesic distance avoids "shorteuts" in regions with thin structures or high curvature. To approximate the geodesic distance, the entire point cloud is linked into k-nearest neighbour graph, with edge weights proportional to Fuclidean distance. Point-to-point distances are then found as shortest paths on that graph. During training we randomly draw positive samples that contain corner and negative ones that do not. To increase the diversity of positive samples, the same corner can be included in multiple patches with different centroids. For the negative samples, we ensure draw not only patches on flat surfaces, but also several patches near edges, by doing this negative samples with strong curvature are well represented. For inference, patch seeds are drawn using farthest point sampling. The vertex detector takes as input the (indexed) features of the M points in the patch, max-pooled per dimension to achieve permutation invariance, and outputs a scalar vertex/no_veree vertex probability. Its loss function Spat is binary cross-cntropp cnttrop..
Vertex Localiser. The vertex detector passes on a set of P patches predicted to contain corner. The M points p; of such a patch form the input for the localisation block. The backbone features Si E TRC and the 3D coordinates p: E R of all M points are concatenated and fed into a multi-layer perceptron with fully connected layers, ReL.U activations and batch normalisation. The MLP outputs vector of M scalar weights wg, where Es wi 1, such that the predicted vertex location is the weighted mean of the input points, V = > w.pt. The prediction vy is supervised by a regression loss feeet = 10P Σ.¹ ||e; 1 summed over all P corner patches. Predicting weights wo rather than directly the coordinates è safeguards against extrapolation errors on unusual patches. We note that the localiser not only improves the corner point accuracy but also improves efficiency, by making possible to use farthest point sampling rather than testing every input point.
Edge Detector. The edge detection block serves to determine which pairs of (predicted) vertices are connected by an edge. Candidate edges eij are found by linking two vertices eú; and ej. see below. To obtain descriptor for an edge, we sample the N₂ equally spaced query locations Q {q j== along the line segment between the two vertices, including the endpoints: 4u = Ug f--1-11--- = ü, For each dk we find the nearest 3D input point (R)) (by Euclidean distance) and retrieve its backbone encoding fNN(K)- This yields an (N, x C') edge descriptor, which is max-pood max-pooled along its first dimension, inspired by the ROI pooling in Faster-RCNY (Ren et al. 2015) and LOI pooling in LCNN (Zhou) ct al. 2019a). The resulting _ N₂ x C*) descriptor is flattened and fed through an MLP with two tully connected layers, which returns a scalar aiigeho__egee score.
During training we select a number of vertex pairs from both the ground truth vertex list {2t and the predicted vertex list { and draw positive and negative sets of edge samples, E+, We empirically found that evaluating all vertex pairs contained in e+ 3+ E during training leads to large imbalance between positive and negative samples. Similar to Zhou et al. (2019a), we thus draw input edges for the edge detector from the following sets:
(a) Positive edges between ground truth vertices: This set comprises all true edges in the ground truth wireframe Ex E
(b) Negative edges between ground truth vertices: Two situations are relevant: "spurious edges", i..., connections between two ground truth vertices that do not share an edge (but lie on the same surface, such that there are nearby points collect features). And "inaccurate edges" where of the endpoints is not a ground truth vertex, but not far from one (to cover difficult cases not far from a correct edge). Formally:
8 =(Cer,;; = (w;, U;) 0 U/ E V,yy & Vp on ev,,, 3p' E P, 11p 1ipp--lla p'1l2 < <e}
U (et.pp = (cr, (ve, m/) E E E P. a. S ||u ps S (2)
(c) Positive edges between predicted vertices: Connections between two "correct" predicted ver- tices that have both been verified coincide with ground truth wireframe vertex up small threshold c... Formally:
gpred+ {ca,,, = (0;, ú2)1 min ||ú; Vell2 < c4, min 10 - =2112 < C+, (Ux, 01) E gat+} (d) Negative edges between predicted vertices: These are (i) "wrong links" as in Es,, between "correctly" predicted vertices; and (ii) "hard negatives" where exactly one of the two vertices is close
4

---

Published as a conference paper at ICLR 2021
to ground truth wireframe vertex, cover "near misses".
Epeed = (0;, 0/)) min iíū; uxll2 < c-, min |lū vzl12 < (c vi) E Est-}
U (e = (ū; B2) S |0 ux 2 S e2, 1 ull2 S E2, UR, vs E V}
Fig. 2 visually depicts the different edge sets. To train the edge detector randomly sample positive examples from - = Ex Epred+ and negative examples from -- = 8 U Errel As loss function go through the fully connected graph of edge candidates connecting any two (predicted) vertices. Candidates with a too high average distance the input points (i.e., not lying any object surface) are discarded. All others are fed into the edge detector for verification.
Ledge the balanced binary cross entropy. During inference,
(9) Examples in gs++
(b) Examples in g
(e) Examples goeed
(d) Examples in gorod.
Figure Positive and negative samples for edge verification during training. (a) Positive edge samples (green lines) between ground truth vertices (black dots). (b) "spurious" (left) and "inaccurate" (right) connections berween ground truth vertices. (c) positive samples (blue lines) between predicted vertices (blue dots). (d) "wrong links" (left) and "near misses" (right) between predicted vertices.
End-to-end training. When training the complete network, we minimise the joint loss function with balancing weights a and f: C Lpat + aLvert + BLedge
Non-Maximum Suppression (NMS) is standard post-processing step for detection tasks, to sup- press redundant, overlapping predictions. We use it (i) to prevent duplicate predictions of vertices that are very close to each other, and (ii) to prune redundant edges.
EXPERIMENTS
We could not find any publicly available dataset of point clouds with polyhedral wireframe annotations. Thus, we collect our own synthetic datasets: a subset from ABC (Koch et al. 2019) and Furniture dataset. ABC is collection of one million high-quality CAD models of mechanical parts with explicitly parameterised curves, surfaces and various geometric features, from which ground truth wireframes can be generated. We select subset of models with straight edges. It consists of 3,000 samples, which we randomly split into 2,000 for training, 500 for validation, and 500 for testing. For the furniture dataset have collected 250 models from Google 3DWarehouse for the categories bed, table, chair/sofa, stairs, monitor and render ground truth wireframes. Examples are shown in the supplementary material. To generate point clouds, we use the virtual scanner of Wang et al.) (017). We place 14 virtual cameras on the face centres of the truncated bounding cubes of each object, uniformly shoot 16,000 parallel rays towards the object from each direction, and compute their intersections with the surfaces. Finally, we jitter all points by adding Gaussian noise A AO0, 0.01). truncated to [-0.02, 0.02]. The number
points varies between As 100k 200k for different virtually scanned point clouds. For details, see the supplementary material.
4.1 EVALUATION MEASURES
We first separately evaluate the vertex detection and edge detection performance of our architecture, then assess the reconstructed wireframes in terms of structural average precision and of a newly designed wireframe edit distance.
Mean Average precision mAP for vertices. To generate high quality wireframes, the predicted vertex positions must be accurate. This metric evaluates the geometric quality of vertices, regardless of edge links. A predicted vertex is considered a true positive the distance to the nearest ground truth vertex is below threshold 7/- For given T) generate precision-recall curves by varying
5

---

Published as a conference paper at ICLR 2021
(a) vertex matching
(b) vertex translation/isectionn
(c) edited vertices (d) edge deletion insertion (c) ground truth
Figure 3:
Illustration of WED. (a) First match vertices to (e) ground truth; (b) move matched vertices,
insert missing ones and compute the cost; (d) delete and insert edges and compute the cost.
the detection threshold. Average precision (AP") is defined as the area under this curve, mAP is averaged over different thresholds 7/p-
Point-wise precision and recall for edges. We do not have baseline that directly extracts 3D wireframe edges. Rather, existing methods label individual input points as "lying on an edge", and produce corresponding precision-recall curves (Hackel et al., 2016 Yu et al., 2018). To this metric, we label 3D point as "edge point" its distance to the nearest wireframe edge is below the average point-to-point distance. As for AP", can now check whether a predicted "edge point" is correct up to Te. and compute AP and mean average precision (mAP²) over different V/e-
Structural average precision for wireframes. The above two evaluation metries looks ver- tices and edges separately, in terms of geometry. In order to comprehensively evaluate the generated wireframe, we also need take into account the connectivity. The structural av- erage precision (sAP, Zhou et al. 2019a) metric, inspired by mean average precision, can be used to assess the overall prediction. This metric checks the two end points (ö;, 0;) of a de- tected edge both match the end points (Ue, v1) of a ground truth edge up to threshold The i.e., mi | vk lll + minney [īū 1ill2 < Ylu: (Vk, 01) E E. We report sAP for different The and mean sAP (msAP).
Wireframe edit distance (WED). In order to topologically compare wireframes, we have designed quality measure based on edit distance. We consider wireframes geometric graphs drawn in 3D space. Distance measures commonly used in computational geometry and graph theory, such as Hausdorff distance and Frechet distance, resistance distance, geodesic distance, do not to apply to our situation. Inspired by [Cheng et al. (2009), we propose a wireframe edit distance (WED). is variant of the graph edit distance (GED), which measures how many elementary graph edit operators (vertex insertion/deletion. edge insertion/deletion, etc.) are needed to transform one graph to another. Instead of allowing arbitrary sequences operations, which makes computing the GED NP-hard, we fix their order, see Fig. 3 (1) vertex matching; (2) vertex translation; (3) vertex insertion: (4) edge deletion; (5) edge insertion. Let the ground truth wireframe W (Vat Eat) and the predicted wireframe W (Vo, ")). First, we match each vertex vg in W to the nearest ground truth vertex ugt in Wos then translate them a cost of c, [|o?ugd]] New vertices are inserted where ground truth vertex has match (at cost, as cost will accrue during edge insertion). After all vertex operations, we perform edge deletion and insertion based on ground truth edges at a cost of ce ||e|], where ||e|| is the length of deleted/inserted edges. The final WED is the sum of all cost terms.
4.2 RESULTS AND COMPARISONS
Vertex detection. We validate the PC2WF vertex detection module and compare Harris3D baseline (with subsequent NMS to guarantee a fair comparison) on the ABC dataset, Tab. [ Exem- plary visual results and precision-recal curves are shown Fig. 4 and Fig Harris3D often fails to accurately localise a corner point, and on certain thin structures it returns many false positives. The patch size used in our vertex detector has an impact on performance. Patch sizes between 20 and 50 achieve the best performance, where the precision stays at high level over a large recall range. Patch size leads to low precision (although still better than Harris3)). confirming that per por-point features, even when computed from larger receptive field, affected by noise and insufficient to find corners with acceptable false positive ratio. We refer the interested reader to the supplementary material for more exhaustive ablation study.

---

Published as a conference paper at ICLR 2021
Table Vertex detection results.
Dataset
ABC
furniture
Method
AP
AP
AP
mAP
AP8.02
AP8.0s
AP5.os
mAP
Harris3D + NMS
0.262
0.469
0.733
0.488
0.268
0.460
0.712
0.480
our method
0.856
0.901
0.925
0.894
0.847
0.911
0.924
0.894
E3E3
Figure 4: Vertex detection results of Harris3D (left) and ours (right) on the ABC dataset.
Edge prediction. Due to the limited number of methods able to extract wireframes from 3D point clouds. we compare PC2WF to recent algorithms that detect edge points, such as FREE (Bazazian et al. 2015), ECD (Ahmed et al. 2018), FEEM (Xia & Wang, 2017). Contour Detector (Hakell al. 2016) and EC-Net (Yu et al. 2018). We also compare with Polyfit (Nan & Wonka 201 7), which is the only method able reconstruct vectorial representation similar to from point cloud. Since Polyfit extracts planar surfaces of the object, we must convert that representation into wireframe, which amounts identifying the plane intersections. Precision-recall curves for all methods are shown Fig. 6 Visual results are shown in Fig. 8 PC2WF outperforms all baselines based hand-crafted features like FREE, ECD, FEEM and Contour Detector, well the deep learning method EC-Net, across all quality measures (Tab. 2 A visual comparison between the ground truth, PC2WF, EC-Net Yu al. (2018), and Polyfit is shown in Fig. 9 One clearly the advantage of predicting vector edges rather than "edge points". as they lead to a much sharper result. The inevitable blur of edge points across the actual contour line, and their irregular density, make vectorisation a non-trivial problem. Polyfit and PC2WF show comparable performance for this specific point cloud. PC2WF failed to localise some corners and missed some small edges, whereas Polyfit failed to reconstruct the lower part of the armchair.
Wireframe Reconstruction. We visualise PC2WF's output wireframes in Fig. 10 Structural precision-recall curves with various patch sizes are shown in Fig. For additional examples of output predictions, including cases with curved edges, please refer to the supplementary material. The results aligned with the ones of the vertex module. Moderate patch sizes between 20 and 50 achieve the best performance. Tab. 3 shows the structural average precision with patch size 50, which exceeds the of Polyfit. The results in terms of the wireframe edit distance shown in Tab. 4 for PC2WF and Polyfit. The average number of predicted vertices (21.7) and ground truth vertices (20.1) 1) per object are close, such that only few vertex insertions Although all predicted vertices need some editing, the WED, caused by those edits is small (0.2427), meaning that the predictions are accurate. On the contrary, the 2.6 edges that need be edited on
needed in most cases.
29
38-
0.5
25
Precsion
3.6
Precsion
FREE
3.6
0.5
Patch size=]
6.S.
ECD
25
Patch size
FEEM
Patch size=1
Patch s5ze=50
Contour
Patch size=20
42
100
62
22
size-50
31
Harris3D
62
PC2WF
51
Patch size- 100
Recall
Figure Vertex prediction
Figure 6: Edge detection ac- curacy on ABC dataset.
Figure Wireframe accu- racy ABC dataset.
accuracy ABC dataset.
7

---

Published as a conference paper at ICLR 2021
(a) FREE
(b) ECD
(c) FEEM
(d) Contour
(c) EC-Net
(f) our method
Figure 8: Comparison with edge point detection algorithms.
ground truth
(b) EC-Net
(c) PolyFit
PC2WF
Figure 9: Visual comparison of graph structures. Vertices are orange and edges are blue.
Table 2: Edge prediction results on the ABC subset and furniture dataset.
Dataset
ABC
furniture
Method
AP
AP
AP AP6.03
mAp
APC
AP6
APC A.0.03
mAP
FREE
0.089
0.365
0.622
0.359
0.076
0.355
0.610
0.347
ECD
0.401
0.703
0.834
0.646
0.405
0.699
0.840
0.648
FEEM
0.412
0.754
0.870
0.679
0.419
0.760
0.865
0.681
Contour
0.422
0.812
0,913
0.715
0.420
0.809
0.900
0.710
PolyFit
0.771
0.798
0.882
0,817
0.584
0.667
0.728
0.660
EC-Net
0.503
0.839
0.912
0.751
0.499
0.838
0.924
0.750
our method
0.826
0.907
0.931
0.888
0.851
0.951
0.974
0.925
Table 3: sAP evaluation on two datasets.
Dataset
ABC
furniture
Method
sAp0.03
SAP0.05
sAp0.07
msAP
sAp0.03
sAp0.05
sAp0.07
msAP
Polyfit
0.753
0.772
0.781 0.907
0.769
0.552
0.590
0.612
0.585
ours
0.868
0.898
0.891
0.865
0.901
0.925
0.897
average increase WED, lot more (1.4142) and dominate the WED, i.e., the inserted/deleted edges have significant length.
5
CONCLUSION
We have proposed PC2WF, an end-to-end trainable deep architecture to extract vectorised wireframe from raw 3ID point cloud. The method achieves very promising results for man-made, polyhedral
8

---

Published as a conference paper at ICLR 2021
14
(b)
(c)
(d)
(c)
Figure 10: Wireframe reconstruction results on ABC (a)(b)(c) and furniture (d)(e)(f) datasets. Top: input raw point clouds. Middle: predicted wireframe, Bottom: ground truth.
Table 4: Wireframe edit distance (WED) on ABC dataset.
pred #v
gt #v
edited #v
WED,
pred #e
gt #c
edited #e
WED,
WED
Polyfit
14.6
20.1
14.6
0.1251
21.9
30.2
10.2
4.7202
4.8453
ours
21.7
20.1
21.7
0.2427
32.8
30.2
2.6
1.4142
1.6569
objects, going one step further than low-level corner or edge detectors, while at the same time outperforming them on the isolated vertex and edge detection tasks. We see our method as one further step from robust, but redundant low-level vision to compact, editable 3D geometry. As future work we aim to address the biggest limitation of our current method which the inability to handle wireframes with strongly curved edges.

---

Published as a conference paper at ICLR 2021
REFERENCES
Syeda Mariam Ahmed, Yan Zhi Tan, Chee Meng Chew, Abdullah Al Mamun, and Fook Seng Wong. Edge and comner detection for unorganized 3d point clouds with application robotic welding. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018.
Dena Bazazian, Josep R Casas, and Javier Ruiz-Hidalgoo. Fast and robust edge extraction in un- organized point clouds. In Digital Image Computing: Techniques and Applications (DICTA), 2015.
Siheng Chen, Dong Tian, Chen Feng, Anthony Vetro, and Jelena Kovačević. Fast resampling of three- dimensional point clouds via graphs. IEEE Transactions on Signal Processing, 66(3):666-6)) 2017.
Otfried Cheong. Joachim Gudmundsson, Hyo-Sil Kim, Daria Schymura, and Fabian Stehn. Measuring the similarity of geometric graphs. In International Symposium on Experimental Algorithums, 2009.
Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019a.
Christopher Choy, Jaesik Park, and Vladlen Koltun. Fully convolutional geometric features. In IEEE International Conference Computer Vision (ICCV), 2019h.
Inge Coudron, Steven Puttemans, and Toon Goedemé. Polygonal reconstruction of building interiors from cluttered pointclouds. In Europeun Conference on Computer Vision (ECCV), 2018.
Kris Demarsin, Denis Vanderstraeten, Tim Volodine, and Dirk Roose. Detection of closed sharp edges in point clouds using normal estimation and graph theory. Computer-Aided Design, 39(4): 276-283, 2007.
Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. InverseCSG: Automatic conversion of 3d models to CSG trees. ACM Transactions on Graphics (TOG), 1-16) 2018.
Alexandre Durupt, Sebastien Remy, Guillaume Ducellier, and Benoît Eynard. From 3d point cloud to an enginecring CAD model: Virtual and Physical Prototyping. 3(2):51-59, 2008.
www.wledgge-rouuut-bbeeee approach for reverse engineering.
Hao Fang, Florent Lafarge, and Mathieu Desbrun. Planar shape detection at structural scales. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2018.
Przemystaw Glomb. Detection of interest points 3d data: Extending the Harris operator. In Computer Recognition Systems 3, pp. 103-111. Springer, 2009.
Diego Gonzalez- Aonzzale-Aguilraa, Aguilera, Susana Del Pozo, Gemma Lopez, and Pablo Rodriguez-Gonzaleezz Gonzalvez. From point cloud to CAD models: Laser and optics geotechnology for the design of electrical substations. Optics & Laser Technology, 4384 2012.
Timo Hackel. Jan Dirk Wegner, and Konrad Schindler. Contour detection in unstructured 3d point clouds. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2016.
Timo Hackel, Jan Dirk Wegner, and Konrad Schindler. Joint classification and contour extraction of large 3d point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 130:231-241 -245, 2017.
Christopher G Harris, Mike Stephens, et al. A combined corner and edge detector. In Alvey Vision Conference, volume 15(50), 1988.
Kun Huang and Shenghua Gao. Wireframe parsing with guidance of distance map. IEEE Access, 141036-141044 2019.
Kun Huang, Yifan Wang. Zihan Zhou, Tianjiao Ding, Shenghua Gao, and Yi Ma. Learning parse wireframes in images of man-man man-made environments. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
10

---

Published as a conference paper at ICLR 2021
Jachoon Jung, Sungchul Hong, Sanghyun Yoon, Jeonghyun Kim, and Joon Heo. Automated 3d wireframe modeling of indoor structures from point clouds using constrained least-squares adjustment for built BIM. Journal of Computing in Civil Engineering. 30(4):04015074, 2016.
Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Mare Alexa, Denis Zorin, and Daniele Panozzo. ABC: A big CAD model dataset for geometric deep learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. GRASS: Generative recursive autoencoders for shape structures. ACM Transactions Graphics (TOG), 36 (4):1-14, 2017a.
Lingxiao Li, Minbyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas Guibas. Supervised fitting of geometric primitives to 3d point clouds. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2019.
Yan-qin Li, Gui-zhong Xie, Fan-nian Meng, and De-hai Zhang. Using the point cloud data to recon- structing CAD model by 3d geometric modeling method in reverse engineering. In International Conference on Manufacturing Engineering and Intelligent Materials (ICMEIM), 2017b.
Hong-We Wei Lin, Chiew-Lan Tai, and Guo-Jin Wang. mesh reconstruction algorithm driven by an intrinsic property of a point cloud. Computer-Aided Design, 36 36(1):1-9, 2004.
Liangliang Nan and Peter Wonka. Polyfit: Polygonal surface reconstruction from point clouds. In IEEE International Conference on Computer Vision (ICCV), 2017.
Fabio Remondino. From point cloud to surface: the modeling and visualization problem. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 34, 2003.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in Neural Information Processing Systems (NeurIPS), 2015.
Ruwen Schnabel, Roland Wahl, and Reinhard Klein. Efficient RANSAC for point-cloud shape detection. Computer Graphics Forum, 26(2):214 226. 2007.
Ruwen Schnabel, Raoul Wessel, Roland Wahl, and Reinhard Klein. Shape recognition in 3d point- clouds. In International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), 2008.
Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. CSGNet: Neural shape parser for constructive solid geometry. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
Ivan Sipiran and Benjamin Bustos. A robust 3d interest points detector based on harris operator. In Eurographics Conference on 3D Object Retrieval (3DOR), 2010.
Ivan Sipiran and Benjamin Bustos. Harris 3d: a robust extension of the Harris operator for interest point detection on 3d meshes. The Visual Computer, 27(1 1):963, 2011.
Stephen M Smith and J Michael Brady, SUSAN - new approach to low level image processing. International Journal of Computer Vision, 23(1) ):55-88,, 1997.
Shubham Tulsiani, Hao Su, Leonidas J Guibas, Alexei A Efros, and Jitendra Malik. Learning shape abstractions by assembling volumetric primitives. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Nicolas Walter, Olivier Aubreton, Yohan D Fougerolle, and Olivier Laligant. Susan 3d operator, principal saliency degrees and directions extraction and a brief study on the robustness to noise. In IEEE International Conference on Image Processing (ICIP), 2009.
Peng-Shue Song-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun- Yu Sun, and Xin Tong. O-CNN: Octree-based convolutional neural networks for 3d shape analysis. ACM Transactions on Graphics (TOG), 36 (4):1-11, 2017.
11

---

Published as a conference paper at ICLR 2021
Xiaogang Wang, Yuelang Xu, Kai Xu, Andrea Tagliasacchi, Bin Zhou, Ali Mahdavi-Amiri, and Hao Zhang. Pie-net: Parametric inference of point cloud edges. Conference on Neural Information Processing Systems (NeurIPS), 2020.
Shaobo Xia and Ruisheng Wang. A fast edge extraction method for mobile LiDAR point clouds.
IEEE Geoscience and Remote Sensing Letters, 14(8):1288-122,, 2017.
Nan Xue, Tianfu Wu, Song Bai, Fu-Dong Wang, Gui-Song Xia, Liangpei Zhang, and Philip H. S. wireframe parsing. arXiv preprint arXiv:2003.0 00660006000,,,, 2020.
Torr. Holistically
Bisheng Yang and Yufu Zang. Automated registration of dense terrestrial laser-scanning point clouds using curves. ISPRS Journal of Photogrammetry and Remote Sensing, 95:109-121, 2014.
Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. EC-Net: edge- aware point set consolidation network. In European Conference on Computer Vision (ECCV), 2018.
Yichao Zhou, Haozhi Qi, and Yi Ma. End-to-end wireframe parsing. In IEEE International Conference Computer Vision (ICCV), 2019a.
Yichao Zhou, Haozhi Qi, Yuexiang Zhai Qi Sun, Zhili Chen, Li-Yi Wei, and Yi Ma. Learning to reconstruct 3d Manhattan wireframes from a single image. In IEEE International Conference on Computer Vision (ICCV), 2019b.
Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 3D-PRNN: Generating shape primitives with recurrent neural networks. In IEEE International Conference on Computer Vision (ICCV), 2017.
Chuhang Zou, Alex Colburn, Qi Shan, and Derek Hoiem. LayoutNet: Reconstructing the 3d room layout from single RGB image. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2018
12


Reviews:
[{"review_id": "xaxem-fh-N2", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\nThis paper introduces PC2WF, a neural network that turns 3D point clouds into a wireframe model. PC2WF encodes each point into a feature vector and uses them to predict the candidate corners. After that, line proposals are generated by connecting pairs of corners, and the point features along each line are pooled into its confidence value. By pruning the proposed lines, PC2WF generates the final wireframe represesntation.\n\n### Comments\n\nExtracting lines and wireframes from point clouds is a relatively new idea. However, there are lines of works about detecting 3D wireframe from single images for both indoor and outdoor scenes, in which people first detect 2D wireframes and lift them into 3d with optimization. I suggest that the author should also cite the following papers:\n\nZou, C., Colburn, A., Shan, Q., & Hoiem, D. (2018). LayoutNet: Reconstructing the 3d room layout from a single rgb image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2051-2059).\n\nZhou, Y., Qi, H., Zhai, Y., Sun, Q., Chen, Z., Wei, L. Y., & Ma, Y. (2019). Learning to reconstruct 3D Manhattan wireframes from a single image. In Proceedings of the IEEE International Conference on Computer Vision (pp. 7698-7707).\n\n#### Pros\n\n1. The writing of the paper is clear and the method is easy to understand. The figures are nice.\n\n2. The end-to-end framework is intuitive and reasonable for the tasks of wireframe extraction from point clouds.\n\n#### Suggestings/Questions\n\n1.  This paper only tested on the synthetic ABC datasets, in which models have fairly dense point clouds and relatively low noise. It would more convencing if the authors can show whether the algorithm works on real-world 3D scanning, e.g., redwood 3d-scan.\n\n2. It seems that EC-NET is the only data-driven method among all the baselines. Could you clarity in the experiment setting section that whether you re-train their models on your dataset with reasonable efforts? If so, could you comment on why their performance is bad (AP^e_0.01) in the experiment section?\n\n3. Will the source code be released?", "extra_scores": {}, "extra_text": {}}, {"review_id": "gN9ufJO8duo", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduced an end-to-end trainable network to predict 3D wireframes given an object point cloud using deep networks. It firstly use  network to cluster points and predict the vertex, and then use an edgenet to predict if an edge exists between two vertices. \n\nPros:\n+ This paper presents an interesting idea on transforming an unordered pointcloud to a structured graph and use networks to learn topologies.\n+ This paper presents some tricks, for example, selecting limited numbers of examples to prevent redundant computation and potential imbalance to train edgenet, and use geodesic distance for grouping points during patch formulation.\n+ This paper raised a new problem and created their own synthesized dataset based on existing datasets for evaluation.\n\nCons:\nMy biggest concern is the limited application of this method. There are already many previous researches on generating graph or wireframes given point cloud without training data pairs. There are also many recent efforts on generating meshes given point cloud using deep networks. Deep networks requires tons of training data, and also seems most of the experimental results in this paper are almost perfect point cloud. Due to these limitations, I don't think this paper has enough contribution for applications.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "acbM8TJn8Yd", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper introduces a supervised neural network predicting a wireframe structure from a 3D point cloud. The network takes a raw unordered 3D point cloud as input, processes it using FCGF architecture, and predicts three types of information: vertex existence in each patch, vertex location, and edge existence for each pair of vertices. In the experiments, the network is evaluated with two datasets, a subset of the ABC dataset and a set of 3D models from Google 3D warehouse. Also, it is compared with the baseline methods using four evaluation metrics, which are created to assess the accuracy of the predicted vertices, edges, and the overall wireframe graph structure. The results demonstrate the outperformance of the proposed method quantitatively and qualitatively.\n\n*** Strengths ***\nTo my knowledge, this is the first work of leveraging neural networks to predict the wireframe structure from a 3D point cloud, except for the following concurrent work:\nWang et al., PIE-NET: Parametric Inference of Point Cloud Edges, arXiv:2007.04883.\nWhile the architecture is simple, the proposed network shows the outperformance compared with the other non-learning-based methods as well as a previous edge detection network called EC-Net (Yu et al., 2018). The experiments are also carefully designed with two datasets and four evaluation metrics; the metrics include the precision of vertex prediction, edge prediction, and the entire graph structure prediction. The supplementary material also shows the details of network training and the performance changes depending on the choice of positive/negative edge sampling methods in the training.\n\n*** Weaknesses ***\nAs mentioned in the conclusion of the paper, the biggest limitation is that the proposed method can only predict straight lines as edges. Also, the experiments are conducted only on the synthetic data but not on any real scan data. Some expositions are also not clear to me, and I describe the details of my concerns below in 'Questions and Suggestions'.\n\n*** Questions and Suggestions ***\n- In the patch generation and vertex location prediction, how is it guaranteed that each patch has only one vertex? The vertex localizer cannot predict multiple vertices since it predicts the location as a weighted sum.\n- In the vertex localizer, wouldn't it be possible to combine binary classification loss for each point?\n- In the end-to-end training, if the network makes false positives of vertex detection (especially at the beginning of the training) and thus the predicted patch does not have a GT vertex, how is the vertex localizer trained?\n- In Section A.3 in supplementary, how is the backbone FCGF pretrained?\n- While the authors described that the network is trained end-to-end, wouldn't it be better to pretrain each of the three components separately and finetune all of them in the end-to-end fashion? Since the later components take the outputs of the previous components in the end-to-end training, I guess the later components can be biased by some wrong predictions at the beginning of the training.\n- For the vertex prediction, is the consistency across the patches checked? If a vertex is predicted in a region where two or more patches are overlapped, one can check whether all the overlapped patches predict the same vertex.\n- For the edge prediction, wouldn't it be also possible to check whether the predicted edge lies on the input point cloud? One can sample points over the predicted edge and compute the distance from the sampled points to the closest points in the input point cloud.\n\n*** Justification ***\nI think this work is worth to be published in ICLR in that it first proposed the problem of learning for wireframe prediction in 3D point clouds. This is a fundamental problem in computer vision/graphics not only for understanding the structure of the raw geometry but also for practically converting raw 3D scanned data into an editable form. While the proposed network is simple, the supervised approach outperformed all the classic non-learning methods in the experiments. The benchmarks including datasets and evaluation metrics will also inspire future research in this direction.\n\n\nI hope the authors discuss the following concurrent work in the revision:\nWang et al., PIE-NET: Parametric Inference of Point Cloud Edges, arXiv:2007.04883.\n\n\nIn short, I'll vote for acceptance once the authors address all my concerns above.\n\n*** Minor Comments ***\n- I could not find which numbers are used for parameters in the experiments: e.g., the number of points N, the number of patches M, and the weights in the loss function alpha and beta.\n- I suggest changing the title of Section C in the supplementary to 'Choice of Positive and Negative Edge Sampling' or a similar one.\n- In all tables, it would be good if 1) the best results are highlighted in bold (as done in Table 2) and 2) the caption describes which evaluation metric results are shown in the table.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "aZCjgnP1jxE", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a deep architecture to extract a wireframe model from a 3D point cloud. This is a problem of high interest, and the author claim that the approach they present is the first one to address this task, which is true to the best of my knowledge. Since both the approach and the evaluation are sound, this alone seem to warrant publication. There are however several weaknesses in the paper, and my accept recommendation is conditional to clear answer on each of them:\n\n1. An important point is the dataset that is introduced: will it be made publicly available? If not, this is a real issue, since it would create a high-cost entrance barrier for any following paper working on the same problem + it would makes comparison between the work and any other method impossible. Similarly, making the code available to allow comparison would be good.\n\n2. The paper and supplementary material show little qualitative results. I think results on a large nimber of random shapes from the test sets should be provided, it is otherwise very hard to figure out how well the pipeline is actually working. \n\n3.  related work on learning 3D shape decomposition into primitives is a bit quick/lacking, I would like discussion and ideally comparison with: https://openaccess.thecvf.com/content_cvpr_2017/papers/Tulsiani_Learning_Shape_Abstractions_CVPR_2017_paper.pdf seem very relevant to me, also GRASS https://dl.acm.org/doi/pdf/10.1145/3072959.3073637 , https://openaccess.thecvf.com/content_ICCV_2017/papers/Zou_3D-PRNN_Generating_Shape_ICCV_2017_paper.pdf ,https://openaccess.thecvf.com/content_cvpr_2018/papers/Sharma_CSGNet_Neural_Shape_CVPR_2018_paper.pdf , https://dl.acm.org/doi/pdf/10.1145/3272127.3275006 . While I realize it is not direct, I find it hard to believe that comparison with none of these works was possible.\n\n4. All the examples shown are with very dense point clouds and very little noise, an analysis of the influence of these two parameters would be necessary. Also, it's unclear how robust the method is to variations in sampling (with a common M parameter)\n\nSome smaller concerns/comments:\n- p4, the paper refers to \"testing all vertex pairs\" when describing the training, which is confusing. Also, the argument that using all pairs during training (as done during inference) would create an imbalance is not so clear since balanced BCE is used in the loss.\n- the task seem very well adapted to unsupervised training, proposing a completely supervised approach and not testing any unsupervised one is a bit underwhelming\n- to me, Li et al. 2019 would be the natural baseline, but it's not discussed in details/not compared to. Would a comparison be possible?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers were all on the positive side for this paper. Multiple reviewers liked the new and interesting task that this paper presents, found that the proposed method works well, is sufficiently compared to alternative approaches, and could serve as a solid baseline for future work in this area. The main limitation that reviewers noted was that results were shown only on the synthetic ABC dataset using dense point clouds with very little noise. The authors wrote *very* thorough responses to all reviewer questions. One reviewer noted that these responses answered all of their questions.

It is worth noting that a paper that solves a similar problem was recently published at NeurIPS ("PIE-NET: Parametric Inference of Point Cloud Edges"). This paper was not published as of the ICLR submission deadline, so it was judged as 'contemporary work' which the authors have no obligation to compare against. Nevertheless, they did attempt to make a comparison in their responses. I would ask that the authors include some discussion of this comparison in their final version of the paper.
Decision: Accept (Poster)