Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper introduced an end-to-end trainable network to predict 3D wireframes given an object point cloud using deep networks. It firstly use  network to cluster points and predict the vertex, and then use an edgenet to predict if an edge exists between two vertices. 

Pros:
+ This paper presents an interesting idea on transforming an unordered pointcloud to a structured graph and use networks to learn topologies.
+ This paper presents some tricks, for example, selecting limited numbers of examples to prevent redundant computation and potential imbalance to train edgenet, and use geodesic distance for grouping points during patch formulation.
+ This paper raised a new problem and created their own synthesized dataset based on existing datasets for evaluation.

Cons:
My biggest concern is the limited application of this method. There are already many previous researches on generating graph or wireframes given point cloud without training data pairs. There are also many recent efforts on generating meshes given point cloud using deep networks. Deep networks requires tons of training data, and also seems most of the experimental results in this paper are almost perfect point cloud. Due to these limitations, I don't think this paper has enough contribution for applications.

review_title: This paper presents an interesting idea to form a wireframe graph for unordered pointcloud using an end-to-end network. But it doesn't have enough applications due to limitations like big training data requirement, and perfect point cloud input.

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary
This paper introduces PC2WF, a neural network that turns 3D point clouds into a wireframe model. PC2WF encodes each point into a feature vector and uses them to predict the candidate corners. After that, line proposals are generated by connecting pairs of corners, and the point features along each line are pooled into its confidence value. By pruning the proposed lines, PC2WF generates the final wireframe represesntation.

### Comments

Extracting lines and wireframes from point clouds is a relatively new idea. However, there are lines of works about detecting 3D wireframe from single images for both indoor and outdoor scenes, in which people first detect 2D wireframes and lift them into 3d with optimization. I suggest that the author should also cite the following papers:

Zou, C., Colburn, A., Shan, Q., & Hoiem, D. (2018). LayoutNet: Reconstructing the 3d room layout from a single rgb image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2051-2059).

Zhou, Y., Qi, H., Zhai, Y., Sun, Q., Chen, Z., Wei, L. Y., & Ma, Y. (2019). Learning to reconstruct 3D Manhattan wireframes from a single image. In Proceedings of the IEEE International Conference on Computer Vision (pp. 7698-7707).

#### Pros

1. The writing of the paper is clear and the method is easy to understand. The figures are nice.

2. The end-to-end framework is intuitive and reasonable for the tasks of wireframe extraction from point clouds.

#### Suggestings/Questions

1.  This paper only tested on the synthetic ABC datasets, in which models have fairly dense point clouds and relatively low noise. It would more convencing if the authors can show whether the algorithm works on real-world 3D scanning, e.g., redwood 3d-scan.

2. It seems that EC-NET is the only data-driven method among all the baselines. Could you clarity in the experiment setting section that whether you re-train their models on your dataset with reasonable efforts? If so, could you comment on why their performance is bad (AP^e_0.01) in the experiment section?

3. Will the source code be released?
review_title: Review of PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds 

final_score: rating: 7: Good paper, accept
review_content: This paper presents a deep architecture to extract a wireframe model from a 3D point cloud. This is a problem of high interest, and the author claim that the approach they present is the first one to address this task, which is true to the best of my knowledge. Since both the approach and the evaluation are sound, this alone seem to warrant publication. There are however several weaknesses in the paper, and my accept recommendation is conditional to clear answer on each of them:

1. An important point is the dataset that is introduced: will it be made publicly available? If not, this is a real issue, since it would create a high-cost entrance barrier for any following paper working on the same problem + it would makes comparison between the work and any other method impossible. Similarly, making the code available to allow comparison would be good.

2. The paper and supplementary material show little qualitative results. I think results on a large nimber of random shapes from the test sets should be provided, it is otherwise very hard to figure out how well the pipeline is actually working. 

3.  related work on learning 3D shape decomposition into primitives is a bit quick/lacking, I would like discussion and ideally comparison with: https://openaccess.thecvf.com/content_cvpr_2017/papers/Tulsiani_Learning_Shape_Abstractions_CVPR_2017_paper.pdf seem very relevant to me, also GRASS https://dl.acm.org/doi/pdf/10.1145/3072959.3073637 , https://openaccess.thecvf.com/content_ICCV_2017/papers/Zou_3D-PRNN_Generating_Shape_ICCV_2017_paper.pdf ,https://openaccess.thecvf.com/content_cvpr_2018/papers/Sharma_CSGNet_Neural_Shape_CVPR_2018_paper.pdf , https://dl.acm.org/doi/pdf/10.1145/3272127.3275006 . While I realize it is not direct, I find it hard to believe that comparison with none of these works was possible.

4. All the examples shown are with very dense point clouds and very little noise, an analysis of the influence of these two parameters would be necessary. Also, it's unclear how robust the method is to variations in sampling (with a common M parameter)

Some smaller concerns/comments:
- p4, the paper refers to "testing all vertex pairs" when describing the training, which is confusing. Also, the argument that using all pairs during training (as done during inference) would create an imbalance is not so clear since balanced BCE is used in the loss.
- the task seem very well adapted to unsupervised training, proposing a completely supervised approach and not testing any unsupervised one is a bit underwhelming
- to me, Li et al. 2019 would be the natural baseline, but it's not discussed in details/not compared to. Would a comparison be possible?
review_title: first sound deep wireframe extraction from 3D PC

final_score: rating: 7: Good paper, accept
review_content: This paper introduces a supervised neural network predicting a wireframe structure from a 3D point cloud. The network takes a raw unordered 3D point cloud as input, processes it using FCGF architecture, and predicts three types of information: vertex existence in each patch, vertex location, and edge existence for each pair of vertices. In the experiments, the network is evaluated with two datasets, a subset of the ABC dataset and a set of 3D models from Google 3D warehouse. Also, it is compared with the baseline methods using four evaluation metrics, which are created to assess the accuracy of the predicted vertices, edges, and the overall wireframe graph structure. The results demonstrate the outperformance of the proposed method quantitatively and qualitatively.

*** Strengths ***
To my knowledge, this is the first work of leveraging neural networks to predict the wireframe structure from a 3D point cloud, except for the following concurrent work:
Wang et al., PIE-NET: Parametric Inference of Point Cloud Edges, arXiv:2007.04883.
While the architecture is simple, the proposed network shows the outperformance compared with the other non-learning-based methods as well as a previous edge detection network called EC-Net (Yu et al., 2018). The experiments are also carefully designed with two datasets and four evaluation metrics; the metrics include the precision of vertex prediction, edge prediction, and the entire graph structure prediction. The supplementary material also shows the details of network training and the performance changes depending on the choice of positive/negative edge sampling methods in the training.

*** Weaknesses ***
As mentioned in the conclusion of the paper, the biggest limitation is that the proposed method can only predict straight lines as edges. Also, the experiments are conducted only on the synthetic data but not on any real scan data. Some expositions are also not clear to me, and I describe the details of my concerns below in 'Questions and Suggestions'.

*** Questions and Suggestions ***
- In the patch generation and vertex location prediction, how is it guaranteed that each patch has only one vertex? The vertex localizer cannot predict multiple vertices since it predicts the location as a weighted sum.
- In the vertex localizer, wouldn't it be possible to combine binary classification loss for each point?
- In the end-to-end training, if the network makes false positives of vertex detection (especially at the beginning of the training) and thus the predicted patch does not have a GT vertex, how is the vertex localizer trained?
- In Section A.3 in supplementary, how is the backbone FCGF pretrained?
- While the authors described that the network is trained end-to-end, wouldn't it be better to pretrain each of the three components separately and finetune all of them in the end-to-end fashion? Since the later components take the outputs of the previous components in the end-to-end training, I guess the later components can be biased by some wrong predictions at the beginning of the training.
- For the vertex prediction, is the consistency across the patches checked? If a vertex is predicted in a region where two or more patches are overlapped, one can check whether all the overlapped patches predict the same vertex.
- For the edge prediction, wouldn't it be also possible to check whether the predicted edge lies on the input point cloud? One can sample points over the predicted edge and compute the distance from the sampled points to the closest points in the input point cloud.

*** Justification ***
I think this work is worth to be published in ICLR in that it first proposed the problem of learning for wireframe prediction in 3D point clouds. This is a fundamental problem in computer vision/graphics not only for understanding the structure of the raw geometry but also for practically converting raw 3D scanned data into an editable form. While the proposed network is simple, the supervised approach outperformed all the classic non-learning methods in the experiments. The benchmarks including datasets and evaluation metrics will also inspire future research in this direction.


I hope the authors discuss the following concurrent work in the revision:
Wang et al., PIE-NET: Parametric Inference of Point Cloud Edges, arXiv:2007.04883.


In short, I'll vote for acceptance once the authors address all my concerns above.

*** Minor Comments ***
- I could not find which numbers are used for parameters in the experiments: e.g., the number of points N, the number of patches M, and the weights in the loss function alpha and beta.
- I suggest changing the title of Section C in the supplementary to 'Choice of Positive and Negative Edge Sampling' or a similar one.
- In all tables, it would be good if 1) the best results are highlighted in bold (as done in Table 2) and 2) the caption describes which evaluation metric results are shown in the table.

review_title: This paper solves a novel problem, provides a comprehensive benchmark, and shows good results. I recommend acceptance, providing that the authors address all my comments.

Meta-review: Reviewers were all on the positive side for this paper. Multiple reviewers liked the new and interesting task that this paper presents, found that the proposed method works well, is sufficiently compared to alternative approaches, and could serve as a solid baseline for future work in this area. The main limitation that reviewers noted was that results were shown only on the synthetic ABC dataset using dense point clouds with very little noise. The authors wrote *very* thorough responses to all reviewer questions. One reviewer noted that these responses answered all of their questions.

It is worth noting that a paper that solves a similar problem was recently published at NeurIPS ("PIE-NET: Parametric Inference of Point Cloud Edges"). This paper was not published as of the ICLR submission deadline, so it was judged as 'contemporary work' which the authors have no obligation to compare against. Nevertheless, they did attempt to make a comparison in their responses. I would ask that the authors include some discussion of this comparison in their final version of the paper.
Decision: Accept (Poster)