Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed a novel way to utilize DDPM's for point cloud completion. They use a conditional DDPM to generate a plausible "complete" point cloud from pure Gaussian noise, conditional on a partial subset of the point cloud points. Their main contribution is is in the design of the conditional feature extraction and denoising subnet, which essentially form dual path connected U-Net type structures with internal modules based on an improved PointNet++ design. Additionally, the authors propose a refinement network to mitigate the computational burden of needing so many diffusion steps, which seems to offer an acceptable performance tradeoff for increases computational efficiency. The method convincingly demonstrates superior performance over its competitors, with crisp details preserved on the datasets tested.
main_review: I thoroughly enjoyed this paper. Though the paper casts itself as a "point cloud completion" paper, I find that the architecture presented is more general and more convincingly powerful than previous DDPM models proposed for general 3D point cloud generation (Luo et al, CVPR'21 and Zhou et al, ICCV'21). 

Luo's approach, which was a best paper finalist for CVPR, is comparatively much weaker in its conditionalization strategy, whereas this proposed work utilizes a multi-scale dual U-Net type approach which allows multi-scale conditioning information to propagate through the network, which seems to add to additional fidelity. Zhou's approach uses point-voxel CNN's, making the (mostly empirical) argument that some kind of point ordering (i.e. voxelization) is necessary for diffusion and so something like PointNet++ cannot be used. This argument never sat well with me, so I was happy to see the authors in this paper also rebut Zhou's reasoning, and additionally present an easy and elegant fix-- to add the absolute coordinates to the PointNet feature (similar in spirit to the CoordConv approach of Liu et al, NeurIPS'18).

The paper is well-presented and easy-to-read, with a robust supplementary. All design choices were reasonably discussed, and I was happy to see that the authors stressed how Chamfer distance losses could be avoided using the DDPM design, a point which isn't always well-explained in the point cloud diffusion literature (even though the authors end up using CD-based loss anyway in the refinement module). I like and agree with the proposed minor modifications to the PointNet++ modules, as well as the use of self-attention instead of pooling. I'm not 100% sold on the refinement network-- I think the training procedure of generating 10 coarse models seems a bit ad hoc-- but I do think it represents an interesting trade-off, and I'm glad the authors included it.

Here's my wishlist for this paper:
1. Direct comparisons against Point-Voxel diffusion completion results. This is the most apples-to-apples comparison you could make. 
2. Multimodal completions. My guess is that the refinementNet probably hurts your ability to get diverse completion results? Perhaps with some stochasticity in the feature transfer module could be added to produce multimodal completions? If your method is unable to provide multimodal completions, you should discuss this limitation.
3. Representation learning results. Recently it was shown that point cloud completion can be a powerful pretext task for representation learning on point clouds (see Wang et al, "Unsupervised Point Cloud Pre-training via Occlusion Completion" ICCV'21). I would be very curious what would happen if you attempted to extract a representation from your trained feature transfer modules and use it for a different downstream supervised task. My guess is that it could very well outperform previous art. 
4. "In the wild" completion results: for example, training on shapenet/modelnet cars and then conditioning from bounding box crops of partially occluded car point clouds in KITTI
5. Point cloud generation with depth map conditioning modality (like as done in Point-Voxel diffusion)

I caught a few typos. Please comb over the paper one more time. For example, from the first couple pages I see: "for many downstreaming applications" and "it can iteratively moves a set of Gaussian noise" and "we propose the feature transform (FT) module to directly transmits encoded point features"
summary_of_the_review: Overall, I really like this paper. I would like to see the authors address the items in my wishlist (specifically iterms #1-3), but the paper stands pretty well on its own.

I also think that diffusion models and 3D point clouds is a nice and timely area of research, and this paper will likely be a nice reference point for others trying to combine DDPM's with 3D point cloud applications.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper tackles the problem of shape-level point cloud completion in a fully supervised setting. The big picture of the proposed method is to use a conditional DDPM to generate a noisy but complete point cloud given incomplete input, and then use another refinement network, also conditional on the incomplete input, to further refine the noisy point cloud. Both networks adopt a novel, dual-path network architecture, based on PointNet++, to allow localized guidance from the incomplete point cloud. Thanks to the 2-staged approach, up to 50x acceleration of evaluation speed is achievable using step jumping, with moderate performance drop. The model is evaluated on MVP, MVP-40 and Completion3D datasets.
main_review: Pros: 
State-of-the-art performer for point cloud completion on various datasets.
I am impressed by the comprehensiveness of the paper. All the key details and experiments are included and well-presented.
Furthermore, since the specific problem of point cloud completion using a DDPM is a relatively under-studied problem, there are many design decisions that need to be made and justified. The paper is very good in this respect.
Various components in the original PointNet++, such as up- and down-sampling modules, are carefully analyzed of their inner workings, taylored to the proposed setting, and evaluated with ablation studies.

Questions:
1. In some sense, the refinement network can be considered as one extra step to the DDPM generator. I wonder if you have any insight on how does the proposed two-stage generation + refinement paradigm compare with using a single stage DDPM, but with a larger model?

2. in section B.1, the author mentiond that the reported results are evaluated on a subset due to the slow evaluation speed. Since the confidence intervals of the evaluation metrics w.r.t. subsampling are not reported, I wonder if subsampling the test set can potentially lead to unreliable and unfair comparison with previous methods?

3. Regarding evaluation speed, I wonder how long does it take to evaluate one point cloud?

4. Regarding Table 5, is the refinement network retrained for experiments with different DDPM steps?

**Update:**
Thank the author for providing additional insights on how the 2-staged model can improve the result -- indeed, minimizing ELBO does not imply minimizing Chamfer distance. I am also glad to learn that the comparisons with baselines are fair. I thus retain my rating of accept.

summary_of_the_review: Good performance. Novel approach. Comprehensive paper. Contributions are well justified. some concern on the reliability of the reported numbers due to subsampling the test set.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. The proposed method effectively and efficiently extracts multi-level features from partially observed point clouds to guide completion. Moreover, it accurately manipulates spatial locations of 3D points to obtain smooth surfaces and sharp details. Experimental results demonstrate its state-of-the-art performance on the mainstream datasets and benchmarks.
main_review: **Strengths**

1. The manuscript is overall clearly written.
2. The authors propose a novel pipeline for point cloud completion.

**Weaknesses**

1. The figures in the manuscript are not clear enough. For example, the relationship between CGNet and DDPM is not clearly represented in Figure 1. The relationship of CGNet, RFNet, CFENet, and Denoise Net are not well represented. This unclarity makes me wonder that both CGNet and RFNet have CFENet and Denoise Net, but why do they have different names?
2. The authors claim that the generated results are more uniform than other methods. Please provide evidence.
3. It seems that the results of the Completion3D benchmark in Table 1 are not consistent with the online results [here](https://completion3d.stanford.edu/results). Could you explain the reason?
4. There are lots of methods in Tables 1-3. However, there are only a few methods in Figure 4. Please provide more qualitative results in Figures 4.
5. Please provide the running time comparison with other methods because you incoporate lots of time-comsuming operations.
summary_of_the_review: I'm leaning to reject this paper (Borderline Reject) *at this time* due to the following reasons:

1. Some claims in the manuscript lacks references. 
2. The results of the Completion 3D benchmark are not consistent with the online results.

**Update (11/26/2021)**

The authors address all of my concerns in the rebuttal. Therefore, I agree to accept the paper.
review_title: null

Meta-review: This paper presents an approach based on conditional denoising diffusion models for point cloud completion. The reviewers have recognized the significance of contributions, the clarity of presentation, and the comprehensivity of experiments. I am happy to recommend this paper for presentation at ICLR.
Decision: Accept (Poster)