Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: A Transformer-based network for point cloud completion called Semantic-Prototype Variational Transformer (SPoVT) is proposed. Firstly, SPoVT learns point cloud distributions for each semantic part by taking partial point cloud and additional associated part labels as inputs, allowing resampling point features for decoding and generation point clouds with varying resolutions. Secondly, A ratio predictor is deployed in SPoVT for predicting point number distributions for each segment part, which serves and guidance for point cloud completion and alleviates potentially dense or sparse completion for particular object parts. Finally, by learning prototypes and feature distributions for object parts, SPoVT is able to perform point cloud completion and manipulate at instance or part levels.
strengths_and_weaknesses: Strengths:
1.	The authors found an efficient way to introduce semantic label information in point cloud completion. Unsurprisingly, the performance is better than methods that only use partial point clouds as input.
2.	I like the idea of Semantic VAE, which can manipulate parts or objects with different point count distributions, making point cloud generation more flexible.
3.	The experimental content is sufficient.

Weaknesses:
1.	In Table 2, pre-trained DGCNN is used for semantic segmentation for each completion method for comparison. Under this condition, the comparison of the segmentation metric is not too meaningful, it just means that the segmentation of those methods + DGCNN is worse than the proposed method.
2.	Some writing problems. Please see the suggestions below.

questions: Questions:
1.	Please address the above Weakness 1.
2.	Is the dataset the complete PCN dataset, or the intersection of PCN and ShapeNetPart? Does the intersection contain the annotations of all GT objects in the PCN? 
3.	I noticed that you reproduced other methods for fair comparisons. I wonder if these methods are trained under the default configuration of the official implementation? Besides, both PMP-Net and PoinTr provide pre-trained models on PCN, how do these compare to your trained models?
4.	Just to be sure, like other completion methods, SPoVT needs to train the model separately for each category of objects, and you can't mix parts of different categories of objects, right?

Suggestions:
1.	Some suggestions for figures.
a)	Figure 1 is too cluttered, lacks annotations, and has a different style for each block. This is confusing.
b)	In Figure 3, It might be better to mark the number of points in red.
2.	The writing of the paper needs to be improved a little bit. 
a)	Some paragraphs are a little cluttered, like the Semantic VAE part, ablation studies part, etc.
b)	Typos, like “Tranformer-based” in the Introduction, “SWiVT” on Table 2, “mIOU”s , etc.

limitations: No.
Suggestions: Please consider adding a discussion section to the paper to describe the limitations of the method as possible, such as the need for additional semantic annotations, inference speed, complex training process, etc.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes the Semantic-Prototype Variational Transformer model for the 3D point cloud semantic completion problem.  Experiments on benchmark datasets show that the proposed method outperforms existing methods.
strengths_and_weaknesses: Strengths:
The proposed approach applies transformer-like structures with a semantic VAE module for 3D point cloud semantic completion problem. The authors also apply a coarse-to-fine strategy for improving the semantic completion, with the learning prototypes and feature distributions for each object part. The ablation analysis validates the efficacy of the proposed network structure.

Weaknesses: 
The motivation of using Transformer-structure for the encoding part for point cloud semantic completion seems not sufficient. Although the authors mentioned in the Introduction section that the reason is to better refine local geometry, in fact, point-based convolution methods such as PointConv [1] provide such utility and graph-based methods such as DGCNN [2] are also suitable for such task. A detailed discussion and comparisons in Related Works section are desired.

[1] Wu, Wenxuan, Zhongang Qi, and Li Fuxin. "Pointconv: Deep convolutional networks on 3d point clouds." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[2] Wang, Yue, et al. "Dynamic graph cnn for learning on point clouds." Acm Transactions On Graphics (tog) 38.5 (2019): 1-12.

questions: From Line 259 on Page 8, the authors perform an evaluation on the setting of varying resolution, I’m curious about the memory usage and runtime comparisons.
limitations: It seems there’s a lack of evaluation on the model runtime in the paper and supplemental. It’s suggested to provide it for objective evaluation.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors proposed a Semantic-Prototype Variational Transformer approach for the dense point cloud semantic completion problem. In addition to the partial input, the paper also used the semantic labels as input for the completion task. An encoder-decoder Transformer architecture is used to leverage both the pixel level input and the semantic label input, so that the semantic label can guide the completion task. The Variational AutoEncoder (VAE) is used for model training and inference. Empirical results showed that the semantic label helped to improve the completion quality, outperforming other baselines.
strengths_and_weaknesses: Strengths:
1) The paper proposed to use semantic label from other sources as additional input to the point cloud completion problem. With this extra source of information, the model is able to infer the ground truth point cloud with better accuracy.
2) The paper leveraged an encoder-decoder Transformer architecture to consume both the pixel level input and the semantic label input, so that the semantic label information can effectively guide the point cloud completion process. 

Weaknesses:
1) The comparisons in Table 2 showed the advantages of the proposed method. However, it is not clear to me whether the comparison is apples-to-apples. For example, is the semantic label information used in other baselines (such as PCN)? It would be nice to explicitly mention which methods used such additional input, and which methods did not. That way, we would know where the improvement comes from (from the semantic label, or from the model architecture).
questions: It has been reported in the literature that the VAE inference is prone to posterior collapse, which greatly degrades the quality of model inference. Have the authors encountered such problem in the VAE inference in this paper? Any procedures taken to alleviate this problem (e.g., through weight annealing)?


limitations: The model uses semantic labels as additional input for point cloud completion. A natural question is, what if such additional input is not available in practice? Is it practical to run a separate semantic labeler on the point cloud?
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose Semantic-Prototype Variational Transformer for semantic point cloud completion, which combines the tasks of point cloud completion and part segmentation. The experimental results on the five categories of the PCN dataset demonstrate that the proposed method outperforms the state-of-the-arts methods.
strengths_and_weaknesses: **Strengths**

- The paper is clearly written and well organized.

**Weaknesses**

- The motivation for introducing semantic labels in the point cloud completion is not clear. In other words, the proposed method can be regarded as point cloud completion + part segmentation.
- The experiments are only conducted on the five categories of the PCN dataset, which is not enough. PCN is a very old dataset. I suggest conducting experiments on the ShapeNet34/55 and MVP datasets.
- All competitive methods use DGCNN for part segmentation for a fair comparison. However, it is still not fair because the proposed method does not use DGCNN. Moreover, DGCNN is a very old baseline.
- Why the mIoU of PointTr are much worse than the rest methods? All of these methods adopt DGCNN for part segmentation.
- The experimental results are only compared on the synthetic dataset, how about the results on real-world scenarios?
questions: Please refer to the weaknesses.
limitations: Compared to the STOA completion methods (e.g. VRC-Net), the improvement of completion is small but requires more labor for data labeling. Moreover, the proposed method requires a model for each category. Do we really need to introduce the semantic information in point cloud completion?
review_title: null

Meta-review: metareview: The paper received mixed reviews. After rebuttal, reviewers Bq58 and QsJ3 decided to raise the rating to weak accept. So all the reviewers give positive ratings and think the authors have addressed their concerns well. Taking the comments of the reviewers into account, the AC decided to accept this paper at NeurIPS.
confidence: Certain
Decision: Accept