A CONDITIONAL POINT DIFFUSION-REFINEMENT PARADIGM FOR 3D POINT CLOUD COMPLETION
Year: 2021

Abstract:
3D point clouds are an important data format that captures 3D information for real world objects. Since 3D point clouds scanned in the real world are often incomplete, it is important to recover the complete point cloud for many downstream applications. Most existing point cloud completion methods use the Chamfer Distance (CD) loss for training. The CD loss estimates correspondences between two point clouds by searching nearest neighbors, which does not capture the overall point density distribution on the generated shape, and therefore likely leads to nonuniform point cloud generation. To tackle this problem, we propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. PDR consists of a Conditional Generation Network (CGNet) and a ReFinement Network (RFNet). The CGNet uses a conditional generative model called the denoising diffusion probabilistic model (DDPM) to generate a coarse completion conditioned on the partial observation. DDPM establishes a one-to-one pointwise mapping between the generated point cloud and the uniform ground truth, and then optimizes the mean squared error loss to realize uniform generation. The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud. In terms of the neural network architecture, we develop a novel dual-path architecture for both networks. The architecture can (1) effectively and efficiently extract multi-level features from partially observed point clouds to guide completion, and (2) accurately manipulate spatial locations of 3D points to obtain smooth surfaces and sharp details. Extensive experimental results on various benchmark datasets show that our PDR paradigm outperforms previous state-of-the-art methods for point cloud completion. In addition, with the help of the RFNet, we can accelerate the iterative generation process of the DDPM by up to 50 times without much performance drop.

heading: 1 INTRODUCTION
text: With the rapid developments of 3D sensors, 3D point clouds are an important data format that captures 3D information owing to their ease of acquisition and efficiency in storage. Unfortunately, point clouds scanned in the real world are often incomplete due to partial observation and self occlusion. It is important to recover the complete shape by inferring the missing parts for many downstream tasks such as 3D reconstruction, augmented reality and scene understanding. To tackle this problem, many learning-based methods (Yuan et al., 2018; Yang et al., 2018; Tchapmi et al., 2019; Xie et al., 2020; Liu et al., 2020; Pan et al., 2021) are proposed, which are supervised by using either the Chamfer Distance (CD) or Earth Mover Distance (EMD) to penalize the discrepancies between the generated complete point cloud and the ground truth. However, CD loss is not sensitive to overall density distribution, and thus networks trained by CD loss could generate non-uniform point cloud completion results (See Figure 10 and 11 in Appendix). EMD is more distinctive to measure density distributions, but it is too expensive to compute in training. The absence of an effective and efficient training loss highly limits the capabilities of many existing point cloud completion networks.
We find that denoising diffusion probabilistic models (DDPM) (Sohl-Dickstein et al., 2015; Ho et al., 2020) can potentially generate uniform and high quality point clouds with an effective and efficient loss function. It can iteratively move a set of Gaussian noise towards a complete and clean point cloud. DDPM defines a one-to-one pointwise mapping between two consecutive point clouds in the diffusion process, which enables it to use a simple mean squared error loss function for
training. This loss function is efficient to compute and explicitly requires the generated point cloud to be uniform, as a one-to-one point mapping is naturally established between the generated point cloud and the ground truth. Point cloud completion task can be treated as a conditional generation problem in the framework of DDPM (Zhou et al., 2021; Luo & Hu, 2021). Indeed, we find the complete point clouds generated by a conditional DDPM often have a good overall distribution that uniformly covers the shape of the object.
Nonetheless, due to the probabilistic nature of DDPM and the lack of a suitable network architecture to train the conditional DDPM for 3D point cloud completion in previous works, we find DDPM completed point clouds often lack smooth surfaces and sharp details (See Figure 1 and Appendix Figure 12), which is also reflected by their high CD loss compared with state-of-the-art point cloud completion methods in our experiments. Another problem with DDPM is its inefficiency in the inference phase. It usually takes several hundreds and even up to one thousand forward steps to generate a single point cloud. Several methods (Song et al., 2020; Nichol & Dhariwal, 2021; Kong & Ping, 2021) are proposed to accelerate DDPM using jumping steps without retraining networks, which however, leads to an obvious performance drop when using a small number of diffusion steps.
In this work, we propose the Conditional Point Diffusion-Refinement (PDR) paradigm to generate both uniform and high quality complete point clouds. As shown in Figure 1, our PDR paradigm performs point cloud completion in a coarse-to-fine fashion. Firstly, we use the Conditional Generation Network (CGNet) to generate a coarse complete point cloud by the DDPM conditioned on the partial point cloud. It iteratively moves a set of Gaussian noise towards a complete point cloud. Following, the ReFinement Network (RFNet) further refines the coarse complete point cloud generated from the Conditional Generation Network with the help of partial point clouds. In addition, RFNet can be used to refine the low quality point clouds generated by an accelerated DDPM, so that we could enjoy an acceleration up to 50 times, while minimizing the performance drop. In this way, the completion results generated by our PDR paradigm demonstrate both good overall density distribution (i.e. uniform) and sharp local details.
Both CGNet and RFNet have a novel dual-path network architecture shown in Figure 2, which is composed of two parallel sub-networks, a Denoise subnet and a Condition Feature Extraction subnet for noisy point clouds and partial point clouds, respectively. Specifically, we propose Point Adaptive Deconvolution (PA-Deconv) operation for upsampling, which can effectively manipulate spatial locations of 3D points. Furthermore, we propose the Feature Transfer (FT) module to directly transmit encoded point features at different scales from the Condition Feature Extraction subnet to the corresponding hierarchy in the Denoise subnet. Extensive experimental results show that our PDR paradigm can provide new state-of-the-art performance for point cloud completion.
Our Key contributions can be summarized as: 1) We identify conditional DDPM to be a good model with an effective and efficient loss function to generate uniform point clouds in point cloud completion task. 2) By using RFNet to refine the coarse point clouds, our PDR paradigm can generate complete point cloud with both good overall density distribution (i.e. uniform) and sharp local details. 3) We design novel point learning modules, including PA-Deconv and Feature Transfer modules, for constructing CGNet in DDPM and RFNet, which effectively and efficiently utilizes multi-level features extracted from incomplete point clouds for point cloud completion. 4) With the help of our proposed RFNet, we can accelerate the generation process of DDPM up to 50 times without a significant drop in point cloud quality.

heading: 2 PROBLEM STATEMENT
text: In this paper, we focus on the 3D point cloud completion task. A 3D point cloud is represented by N points in the 3D space: X = {xj |1 ≤ j ≤ N}, where each xj ∈ R3 is the 3D coordinates of the j-th point. We assume the dataset is composed of M data pairs {(Xi,Ci)|1 ≤ i ≤ M}, where Xi is the i-th ground-truth point cloud, and Ci is the incomplete point cloud from a partial observation ofXi. The goal is to develop a model that completes the partial observation Ci and outputs a point cloud as close to the ground truth Xi as possible. For algebraic convenience, we let x ∈ R3N be the vector form of a point cloudX , and similarly c be the vector form of C.

heading: 3 METHODOLOGY
text: We consider the point cloud completion task as a conditional generation problem, where the incomplete point cloud C serves as the conditioner. We use the powerful generative model called denoising diffusion probabilistic models (DDPM) (Sohl-Dickstein et al., 2015; Ho et al., 2020) to first generate a coarse completion of the partial observation. Then we use another network to refine the coarse point cloud to improve its visual quality. Our point cloud completion pipeline is shown in Figure 1. We first briefly introduce the theory of DDPM in Section 3.1, and then describe detailed architecture of the Conditional Generation Network (CGNet) and ReFinement Network (RFNet) in Section 3.2 and Section 3.3.

heading: 3.1 BACKGROUND ON CONDITIONAL DENOISING DIFFUSION PROBABILISTIC MODELS
text: We assume pdata to be the distribution of the complete point cloud xi in the dataset, and platent = N (03N , I3N×3N ) to be the latent distribution, whereN is the Gaussian distribution. Then, the conditional DDPM consists of two Markov chains called the diffusion process and the reverse process. Both processes have length equal to T . We set T = 1000 in this paper.
The Diffusion Process. The diffusion process is a Markov process that adds Gaussian noise into the clean data distribution pdata until the output distribution is close to platent. The diffusion process is irrelevant of the conditioner, the incomplete point cloud ci. Formally, let x0 ∼ pdata. We use the superscript to denote the diffusion step t. For conciseness, we omit the subscription i in the following discussion. The diffusion process from clean data x0 to xT is defined as
q(x1, · · · ,xT |x0) = T∏ t=1 q(xt|xt−1), where q(xt|xt−1) = N (xt; √ 1− βtxt−1, βtI). (1)
The hyperparameters βt are pre-defined, small positive constants (See details in Appendix Section A.1). According to Ho et al. (2020), there is a closed form expression for q(xt|x0). We first define constants αt = 1− βt, ᾱt = ∏t i=1 αi. Then, we have q(x t|x0) = N ( xt; √ ᾱtx 0, (1− ᾱt)I ) .
Therefore, when T is large enough, ᾱt goes to 0, and q(xT |x0) becomes close to the latent distribution platent(xT ). Note that xt can be directly sampled through the following equation:
xt = √ ᾱtx 0 + √ 1− ᾱt , where is a Gaussian noise. (2)
We emphasize that q(xt|xt−1) can be seen as a one-to-one pointwise mapping as xt can be sampled through the equation xt = √ 1− βtxt−1 + βt . Therefore, the order of points in x0 is preserved in the diffusion process. However, it does not matter what kind of order we input the points in x0. That is because when T is large enough, xT will become a Gaussian distribution. Every point in a Gaussian distribution is equivalent and there is no way to distinguish one point from another.
The Reverse Process. The reverse process is a Markov process that predicts and eliminates the noise added in the diffusion process. The reverse process is conditioned on the conditioner, the incomplete point cloud c. Let xT ∼ platent be a latent variable. The reverse process from latent xT to clean data x0 is defined as
pθ(x 0, · · · ,xT−1|xT , c) = T∏ t=1 pθ(x t−1|xt, c), where pθ(xt−1|xt, c) = N (xt−1;µθ(xt, c, t), σ2t I).
(3)
The mean µθ(xt, c, t) is a neural network parameterized by θ and the variance σ2t is a time-step dependent constant. To generate a sample conditioned on c, we first sample xT ∼ N (03N , I3N×3N ), then draw xt−1 ∼ pθ(xt−1|xt, c) for t = T, T − 1, · · · , 1, and finally outputs x0.
Training. DDPM is trained via variational inference. Ho et al. (2020) introduced a certain parameterization for µθ that can largely simplify the training objective. The parameterization is σ2t = 1−ᾱt−1 1−ᾱt βt, and µθ(x t, c, t) = 1√αt ( xt − βt√ 1−ᾱt θ(x t, c, t) ) , where θ is a neural network taking noisy point cloud xt ∼ q(xt|x0) in equation (2), diffusion step t, and conditioner c as inputs. Then, the simplified training objective becomes
L(θ) = Ei∼U([M ]),t∼U([T ]), ∼N (0,I) ‖ − θ( √ ᾱtx 0 i + √ 1− ᾱt , ci, t)‖2, (4)
where U([M ]) is the uniform distribution over {1, 2, · · · ,M}. The neural network θ learns to predict the noise added to the clean point cloud x0, which can be used to denoise the noisy point cloud xt = √ ᾱtx 0 + √
1− ᾱt . Note that traditional CD loss or EMD loss is NOT present in Equation 4. The reason that we are able to use the simple mean squared error is because DDPM naturally defines a one-to-one pointwise mapping between two consecutive point clouds in the diffusion process as shown in Equation 1. Note that at each training step, we not only need to sample a pair of point clouds xi, ci, but also a diffusion step t and a Gaussian noise .

heading: 3.2 CONDITIONAL GENERATION NETWORK
text: In this section, we introduce the architecture of Conditional Generation Network (CGNet) θ. The inputs of this network are the noisy point cloud xt, the incomplete point cloud c, and the diffusion step t. We can intuitively interpret the output of θ as per-point difference between xt and xt−1 (with some arithmetic ignored). In addition, θ should also effectively incorporate multi-level information from c. The goal is to infer not only the overall shape but also the fine-grained details based on c. We design a neural network that achieves these features. The overall architecture is shown in Figure 2. It is composed of two parallel sub-networks similar to PointNet++ (Qi et al., 2017b), and they have the same hierarchical structure.
The upper subnet, which we refer as the Condition Feature Extraction subnet, extracts multi-level features from the incomplete point cloud c. The lower subnet, which we refer as the Denoise subnet, takes the noisy point cloud xt as input. We also add the diffusion step t, the global feature extracted from c, and multi-level features extracted by the Condition Feature Extraction subnet to the Denoise subnet. The diffusion step t is first transformed into a 512-dimension step embedding vector through positional encoding and fully connected (FC) layers (See Appendix Section A.1 for details), and then inserted to every level of the Denoise subnet. Similarly, the conditioner c is first transformed into a 1024-length global feature through a two-stage PointNet, and then inserted to every level of the Denoise subnet. The multi-level features extracted by the Condition Feature Extraction subnet are
inserted to every level of the Denoise subnet through Feature Transfer modules. Finally, the Denoise subnet is connected to a shared MLP and outputs θ(xt, c, t).
Additionally, while Zhou et al. (2021) argues PointNet++ cannot be used in a DDPM that generates point clouds, we find attaching the absolute position of each point to its feature solves this problem. See Appendix Section A.3 for detailed analysis. We also improve the backbone PointNet++ so that it manipulates positions of points more accurately.
In the next paragraphs, we elaborate on the building blocks of the improved PointNet++: Set Abstraction modules in the encoder, and Feature Propagation modules in the decoder, and Feature Transfer modules between the Condition Feature Extraction subnet and the Denoise subnet.
Set Abstraction (SA) Module. Similar to PointNet++, this module subsamples the input point cloud and propagates the input features. Assume the input is {xj |1 ≤ j ≤ Nl}, where xj is the 3D coordinate of the j-th point and Nl is the number of input points to the Set Abstraction module of level l. Each point has a feature of dimension dl. We concatenate these features with their corresponding 3D coordinates and group them together to form a matrix Fl of shape Nl × (dl + 3). The SA module first uses iterative farthest point sampling (FPS) to subsample the input points to Nl+1 points: {yk|1 ≤ k ≤ Nl+1}. Then it finds K neighbors in the input {xj |1 ≤ j ≤ Nl} for each yk. We denote the K neighbors of yk as {xj |j ∈ Bx(yk)}, where Bx(yk) is the index set of the K neighbors. See definition of neighbors in Appendix A.2. These neighbors and their features are grouped together to form a matrix Gin of shape Nl+1 × K × (dl + 3). Then a shared multilayer perceptron (MLP) is applied to transform the grouped feature Gin to Gout, which is of shape Nl+1×K × dl+1 and dl+1 is the dimension of the output feature. Finally, a max-pooling is applied to aggregate features from the K neighbors {xj |j ∈ Bx(yk)} to yk. We obtain the output of the SA module, the matrix Fl+1, which is of shape Nl+1 × dl+1. Note that we need to incorporate information of the diffusion step embedding and global feature extracted from the incomplete point cloud c to every SA module in the Denoise subnet as shown in Figure 2. We insert these information to the shared MLP that transforms Gin to Gout mentioned in the above paragraph. Specifically, we add them to the channel dimension of the intermediate feature maps in the shared MLP. Figure 3(a) illustrates this process in details. Inspired by the works (Pan et al., 2021; Zhao et al., 2020), we also replace the max-pooling layer in the SA module with a self-attention layer. Feature at yk is obtained by a weighted sum of the features of its K neighbors {xj |j ∈ Bx(yk)} instead of max-pooling, and the weights are adaptively computed through the attention mechanism. See Appendix A.4 for details of this attention layer.
Feature Propagation (FP) Module. Similar to PointNet++, this module upsamples the input point cloud and propagates the input features. In PointNet++, the features are upsampled from {yk|1 ≤ k ≤ Nl+1} to {xj |1 ≤ j ≤ Nl} by three interpolation: Feature at xj is a weighted sum of the features of its three nearest neighbors in {yk|1 ≤ k ≤ Nl+1}. We think that the three interpolation operation is not suitable in our task, because the interpolation operation may lose some information about the accurate positions of the points. See a detailed analysis in Appendix Section A.5.
We propose to use a Point Adaptive Deconvolution (PA-Deconv) module to upsample the point features. In the SA module, the features are mapped from set {xj |1 ≤ j ≤ Nl} to {yk|1 ≤ k ≤ Nl+1}. The key step is to find the neighbors {xj |j ∈ Bx(yk)} ⊆ {xj |1 ≤ j ≤ Nl} for each yk. Features at {xj |j ∈ Bx(yk)} are transformed and then aggregated to the point yk through either max-pooling or attention mechanism. Now in the FP module, we need to map features the other way around: from {yk|1 ≤ k ≤ Nl+1} to {xj |1 ≤ j ≤ Nl}. We can achieve this goal through a similar method. We find the neighbors {yk|k ∈ By(xj)} ⊆ {yk|1 ≤ k ≤ Nl+1} for each xj . Features at {yk|k ∈ By(xj)} are transformed through a shared MLP, and then aggregated to the point xj through attention mechanism. Similar to SA modules, we insert information of the diffusion step embedding and the global feature extracted from the incomplete point cloud c to the shared MLP in every FP module in the Denoise subnet. Finally, same as the original FP module in PointNet++, the upsampled features are concatenated with skip linked point features from the corresponding SA module, and then passed through a unit PointNet. The Feature Propagation module are applied four times and features are eventually propagated to the original input point cloud.
Feature Transfer (FT) Module. The FT module transmits information from the Condition Feature Extraction subnet to the Denoise subnet. Assume the point cloud at level l in the Condition Feature Extraction subnet is {zl|1 ≤ l ≤ Sl}, where Sl is the number of points at level l in the Condition Feature Extraction subnet. The FT module maps the features at points {zr|1 ≤ r ≤ Sl} to points at the same level in the Denoise subnet, which are {xj |1 ≤ j ≤ Nl}. Then the mapped features are concatenated with the original features at {xj |1 ≤ j ≤ Nl}. Next, the concatenated features are fed to the next level of the Denoise subnet. In this way, the Denoise subnet can utilize local features at different levels of the incomplete point cloud to manipulate the noisy input point cloud to form a clean and complete point cloud. The key step in this process is to map features at {zr|1 ≤ r ≤ Sl} to {xj |1 ≤ j ≤ Nl}. We adopt a similar strategy in the SA module. We find the neighbors {zr|r ∈ Bz(xj)} ⊆ {zr|1 ≤ r ≤ Sl} for each xj . Features at {zr|r ∈ Bz(xj)} are transformed through a shared MLP, and then aggregated to the point xj through the attention mechanism, which is a weighted sum of the features at {zr|r ∈ Bz(xj)}. We set a small distance to define neighbors in low level FT modules, so that they only query the adjacent parts of the incomplete point cloud c to preserve local details in it. Large distances are set to define neighbors in high level FT modules. This makes high-level FT modules have large receptive fields, so that they can query a large part of the incomplete point cloud to infer high level 3D structural relations. See detailed neighbor definitions in Appendix Section A.2.

heading: 3.3 REFINEMENT NETWORK
text: We denote the coarse point cloud generated by the Conditional Generation Network as U . We use another network of the same architecture shown in Figure 2 to predict a per-point displacement forU to refine it. The differences are that the input to the Denoise subnet becomes U and we do not need to insert the diffusion step embedding to the Denoise subnet. The predicted displacement are added toU to obtain the refined point cloud V : v = u+γ f (u, c), where v,u, c are the concatenated 3D coordinates of the point clouds V ,U ,C, respectively. γ is a small constant and we set it to 0.001 in all our experiments. f is the ReFinement Network. We use the Chamfer Distance (CD) loss between the refined point cloud V and ground truth point cloudX to supervise the network f :
LCD(V ,X) = 1 |V | ∑ v∈V min x∈X ||v − x||2 + 1 |X| ∑ x∈X min v∈V ||v − x||2, (5)
where |V | means number of points in V . If we also want to upsample points in U by a factor of λ, we can simply increase the output dimension of the network f . In addition to predicting one 3D displacement of each point inU , we predict another λ displacements. We consider each point in the refined point cloud V as the center of a group of λ points in the dense point cloud that we want to generate. The additional λ displacements are added to every point in V to form a dense point cloud. Figure 3(c) illustrates how we upsample every point in V by a factor of λ = 8.
When training the ReFinement Network f , parameters in the Conditional Generation Network θ are fixed. It is not practical to generate coarse point clouds U on the fly in the training process of f , because the generation process of DDPM is slow. Instead, we generate and save the coarse point clouds in advance. Due to the probabilistic nature of DDPM, we generate 10 coarse point clouds for each incomplete point cloud in the dataset to increase diversity of training data.

heading: 4 RELATED WORKS
text: Point cloud completion. Inspired by the pioneering work, PointNet (Qi et al., 2017a), researchers focus on learning global feature embeddings from 3D point clouds for completion (Yuan et al., 2018; Tchapmi et al., 2019), which however cannot predict local and thin shape structures. To address these challenges, following research works (Pan, 2020; Xie et al., 2020; Zhang et al., 2020; Wen et al., 2021; Yu et al., 2021; Pan et al., 2021) exploit multi-scale local point features to reconstruct complete point clouds with fine-grained geometric details. Recently, PointTr (Yu et al., 2021) and VRCNet (Pan et al., 2021) provide impressive point cloud completion results with the help of attention-based operations. Nonetheless, as a challenging conditional generation problem, point cloud completion has not been fully resolved. DDPM for point cloud generation. Luo & Hu (2021) are the first to use DDPM for unconditional point cloud generation. They use a Pointwise-net to generate point clouds, which is similar to a 2-stage PointNet used for point cloud part segmentation. However, the Pointwise-net could only receive a global feature. It can not leverage fine-grained local structures in the incomplete point cloud. Zhou et al. (2021) further use conditional DDPM for point cloud completion by training a point-voxel CNN (Liu et al., 2019), but the way they use the incomplete point cloud c is different from ours. They directly concatenate c with the noisy input xt, and feed them to a single pointvoxel CNN. This may hurt performance of the network, because the concatenated point cloud is very likely to be non-uniform. In addition, xt is very different from c for large t’s due to the large noise magnitude in xt. Feeding two point clouds of very different properties to a single network at once could be quite confusing for the network. The other major difference is that they do not refine or upsample the coarse point cloud generated by DDPM like we do.

heading: 5 EXPERIMENTS

heading: 5.1 DATASETS
text: We conduct point cloud completion experiments on the following three datasets. MVP. The MVP dataset (Pan et al., 2021) has 62,400 training partial-complete point cloud pairs and 41,600 testing pairs sampled from ShapeNet (Chang et al., 2015). Every partial point cloud has 2048 points. In particular, MVP dataset provides ground truth point clouds with different resolutions, including 2,048, 4,096, 8,192, and 16,384 points. MVP-40. The MVP-40 dataset (Pan et al., 2021) consists of 41,600 training samples and 64,168 testing samples from 40 categories in ModelNet40 (Wu et al., 2015). Its partial point clouds are sampled from complete point clouds with a pre-defined missing ratio, i.e., 50%, 25% and 12.5% missing. Both the partial and complete point clouds have 2048 points. Completion3D. It (Tchapmi et al., 2019) consists of 28,974 point cloud pairs for training and 1,184 for testing from 8 object categories in ShapeNet. Both the partial and complete point clouds have 2048 points. We find some pairs of the incomplete point cloud and complete point cloud have inconsistent scales in the Completion3D dataset. We correct the scales and use the corrected dataset in our experiments. See details in Appendix Section B.4.

heading: 5.2 EVALUATION METRICS
text: We use the Chamfer Distance (CD), Earth Mover Distance (EMD), and F1 score to evaluate the quality of the generated point clouds. CD distance is defined in Equation 5.
Earth Mover Distance. Consider the predicted point cloud V and the ground truth point cloud X of equal size N = |V | = |X|, the EMD loss penalizes their shape discrepancy by optimizing a transportation problem. It estimates a bijection φ : V ←→X between V andX:
LEMD(V ,X) = min φ:V←→X ∑ v∈V ∥∥v − φ(v)∥∥ 2 . (6)
F1 score. To compensate the problem that CD loss can be sensitive to outliers, we follow previous methods (Pan et al., 2021; Tatarchenko et al., 2019) and use F1 score to explicitly evaluates the distance between object surfaces, which is defined as the harmonic mean between precision LP(ρ)
and recall LR(ρ): LF1 = 2LP(ρ)LR(ρ)LP(ρ)+LR(ρ) , where LP(ρ) = 1 |V | ∑ v∈V [ min x∈X ∥∥x − v∥∥2 < ρ], LR(ρ) = 1 |X| ∑ x∈X [ min v∈V
∥∥x − v∥∥2 < ρ], and ρ is a predefined distance threshold. We set ρ = 10−4 for the MVP and Completion3D datasets, and set ρ = 10−3 for the MVP-40 dataset.

heading: 5.3 POINT CLOUD COMPLETION
text: We compare our point cloud completion method with previous state-of-the-art point cloud completion methods. The comparison is performed on MVP, MVP-40, and Completion3D datasets. Results are shown in Table 1. We also conduct multi-resolution experiments on the MVP dataset, and results are shown in Table 2. Detailed experimental setups are provided in Appendix Section B.1. We can see that our Conditional Point Diffusion-Refinement (PDR) paradigm outperforms other methods by a large margin in terms of EMD loss. We also achieve the highest F1 score and very low CD loss. Although VRCNet sometimes has lower CD loss than ours, it tends to put more points in the parts that are known in the incomplete point clouds, while put less points in the missing part (See Figure 10 in Appendix). In this way, its CD loss could be very low, but this non-uniformness is undesired and leads to very high EMD loss. We compare our method with other baselines in terms of visual quality of completed point clouds in Figure 4. We can see that our method generally has better visual quality. More samples are provided in Figure 9 and Figure 11 in Appendix.
Ablation Study. We study the effect of attention mechanism, Point Adaptive Deconvolution (PADeconv) module, and Feature Transfer (FT) module in term of training the Conditional Generation Network and the Refinement Network. The experiments are conducted on MVP dataset (2048 points) and results are shown in Table 3. “PA-Deonv & Att.” is our proposed complete network shown in Figure 2. “PA-Deonv” removes attention mechanism. “PointNet++” further removes PA-Deconv module. “Concate xt & c” removes FT modules. It concatenates c with xt as Zhou et al. (2021) do, and feed them to a single PointNet++ with attention mechanism and PA-Deconv. “Pointwise-net” only utilizes a global feature extracted from the incomplete point cloud. We can see that these proposed modules indeed improve the networks’ performance. Note that the conditional generation networks in Table 3 are trained without data augmentation. Experiments with data augmentation are presented in Appendix Section B.6. Data augmentation is necessary to prevent the conditional generation networks from overfitting on the training set, so that they can generate coarse point clouds of consistent distributions on the training set and test set, which is important to train the refinement network. All the refinement networks are trained using data generated by our proposed complete network trained with data augmentation. If the other networks use training data generated by themselves, they would have worse refinement results.
DDPM acceleration. Kong & Ping (2021) propose to accelerate the generation process of DDPM by jumping steps in the reverse process. The method does not need retraining of the DDPM. We directly apply their method to our 3D point cloud generation network. However, we observe a considerable performance drop in the accelerated DDPM. On the MVP dataset (2048 points), the original 1000-step DDPM achieves 10.7× 10−4 CD loss. CD losses of the accelerated 50-step and 20-step DDPMs increase to 13.2×10−4 and 18.1×10−4, respectively. Fortunately, we can generate coarse point clouds using the accelerated DDPMs and use another Refinement Network to refine them. The refined point clouds of the 50-step and 20-step DDPMs bear CD losses of 5.68 × 10−4 and 5.78× 10−4, respectively. Compared with the original 1000-step DDPM, which has a CD loss of 5.66 × 10−4, it’s quite temping to accept a slight drop in performance for an acceleration up to 50 times. Complete results of the acceleration experiment are presented in Appendix Section B.7.

heading: 5.4 EXTENSION TO CONTROLLABLE GENERATION
text: Our conditional PDR paradigm can be readily extended to controllable point cloud generation conditioned on bounding boxes of every part of an object. We sample points on the surfaces of the bounding boxes and regard this point cloud as the conditioner, just likes the incomplete point cloud serves as the conditioner for point cloud completion. We conduct experiments on the chair category of PartNet (Mo et al., 2019) dataset. Two examples are shown in Figure 5. It’s interesting that our method can generate a shape different from the ground truth in some details, but be still plausible.

heading: 6 CONCLUSION
text: In this paper, we propose the Conditional Point Diffusion-Refinement (PDR) paradigm for point cloud completion. Our method effectively leverages the strong spatial correspondence between the adjacent parts of the incomplete point cloud and the complete point cloud through the proposed Feature Transfer module, which could also infer high-level 3D structural relations. We make improvements of the backbone PointNet++ to make it capable of accurately manipulating positions of input points. Our method demonstrate significant advantages over previous methods, especially in terms of the overall distribution of the generated point cloud. We also find that our method has great potential to be applied in other conditional point cloud generation tasks such as controllable point cloud generation.

heading: APPENDIX

heading: A METHODOLOGY DETAILS

heading: A.1 DETAILS OF THE DDPM
text: Hyperparameters βt. We define hyperparameters βt in the diffusion process according to a linear schedule. We let β1 = 1× 10−4 and βT = 2× 10−2. Then, we define βt = t−1T−1 · (βT − β1), t = 1, 2, · · · , T .
Diffusion step embedding. The network needs to output different θ(xt, c, t) for different diffusion steps t ∈ {1, · · · , T}. We first use positional encoding (Vaswani et al., 2017) to encode each t into a 2dt dimensional vector φemb(t) = [sin(ψ(t)), cos(ψ(t))], where
ψ(t) = [ 10 4×0 dt t, 10 4×1 dt t, · · · , 10 4×(dt−1) dt t ] .
We set dt = 64 in experiments. Then, we use two fully-connected (FC) layers to transform φemb(t) into a 512 dimensional embedding vector (Ho et al., 2020). The first FC layer has input dimension 2dt = 128 and output dimension 512. The second FC layer has input dimension 512 and output dimension 512. Both layers are followed by the Swish activation function (Ramachandran et al., 2017).

heading: A.2 NEIGHBOR DEFINITION
text: In the Set Abstraction module and the Feature Transfer module, the neighbors are defined as the points that are within a specified distance to the center point. If a center point has more than K neighbors, we randomly select K neighbors from its neighbors. If a center point has less than K neighbors, we pad it with dummy neighbors that has the same position as the center point, and has features of zeros. These dummy neighbors are excluded from the max-pooling operation. And in the attention mechanism, the weights of the dummy neighbors are manually set to 0. In this way, we can guarantee that a center point can always find K neighbors. We set K = 32 in the Set Abstraction module and the Feature Transfer module. In the 4 levels of the Set Abstraction modules, the neighboring distance are set to 0.1, 0.2, 0.4, 0.8, respectively. In the 9 Feature Transfer modules, the neighboring distance are set to 0.1, 0.2, 0.4, 0.8, 1, 6, 0.8, 0.4, 0.2, 0.1, respectively. Coordinates of samples in all datasets are normalized to the range [−1, 1]. In the Point Adaptive Deconvolution (PA-Deconv) modules of the Feature Propagation modules, the K neighbors are defined as the K nearest neighbors of the center point, and we set K = 8 for all Feature Propagation modules.

heading: A.3 PROBLEMS OF VANILLA POINTNET++
text: Zhou et al. (2021) argue that PointNet++ can not be used to train a DDPM. We observe the same phenomenon in our experiments. We find that this is because the density of the input noisy cloud xt is too low for large t’s. Recall that xt = √ ᾱtx 0 + √
1− ᾱt , where is a Gaussian noise. ᾱt goes to 0 for large t’s. This means xt is close to a Gaussian noise when t is large. The density of a Gaussian noise is much lower than the complete point cloud x0. This is because points in x0 concentrate on the surface of some object and x0 is normalized to the range [−1, 1], while points from a standard Gaussian distribution could fill the whole space in the range of [−3, 3]. PointNet++ is originally designed to process point clouds like x0. It’s selection of distances to define neighbors described in Appendix Section A.2 is suitable for point clouds that have the same level of density as x0, but can not handle point clouds close to a Gaussian noise. Indeed, we find the average number of neighbors for each point in the four levels of the Set Abstraction modules are 22.3984, 29.9133, 29.3266, 27.8375, respectively, for 10 random shapes sampled from the MVP dataset. In constrast, the average number of neighbors for each point in the four levels are 1.0864, 1, 1, 1 for 10 random point clouds sampled from the Gaussian distribution. Note that each point itself is considered to be a neighbor of itself. This means most points do not have any neighbors besides itself in a Gaussian noise.
PointNet++ only utilizes the relative positions of input points. The input feature of each point to the first Set Abstraction module is its relative position to the center point, which is subsampled from the original input points by farthest point sampling. No information can be extracted when points do not have neighbors. This is the reason why PointNet++ can not be directly used to train a DDPM.
Our solution is to attach the absolute position of each point to its feature. This guarantees that a point at least can utilize its own position to decide which direction to move when it does not have neighbors. Afterall, a point cloud with large magnitude noises does not have many meaningful structures. There is not much information in the relative positions of points. Another solution is to change the definition of neighbors: From points within a specified distance to K-nearest neighbors. This guarantees that a point always hasK neighbors. We conduct experiments to compare these two solutions, and we find that their performances are basically the same. Therefore, we just stick to the original neighbor definitions in PointNet++.

heading: A.4 ATTENTION MECHANISM
text: In Section 3.2 in the main text, we mentioned that we use the attention mechanism instead of maxpooling to aggregate features at the neighboring points to the center point. We take the Set Abstraction module as an example to elaborate on the attention mechanism. Attention mechanism in the Feature Propagation module and Feature Transfer module is similarly designed.
Assume we want to propagate features from {xj |1 ≤ j ≤ Nl} to {yk|1 ≤ k ≤ Nl+1}. Each point in {xj |1 ≤ j ≤ Nl} has a feature of dimension dl. We concatenate these features with their corresponding 3D coordinates and group them together to form a matrix Fl of shape Nl × (dl + 3). We finds K neighbors in the input set {xj |1 ≤ j ≤ Nl} for each yk. These neighbors together with their features are grouped together to form a matrixGin of shape Nl+1 ×K × (dl + 3). Then a shared multi-layer perceptron (MLP) is applied to transform the grouped feature Gin to Gout, which is a matrix of shape Nl+1 ×K × dl+1 and dl+1 is the dimension of the output feature. In our attention mechanism, Gin (shape Nl+1 × K × (dl + 3)) will act like keys, Gout (shape Nl+1 × K × dl+1) will act like values, while the original features at {yk|1 ≤ k ≤ Nl+1} will act like queries. Since {yk|1 ≤ k ≤ Nl+1} is a subset of {xj |1 ≤ j ≤ Nl}, we can group the original features at {yk|1 ≤ k ≤ Nl+1} to form a matrix Q, which is of shape Nl+1 × (dl + 3). Q is first repeated K times into a matrix of shape Nl+1 × K × (dl + 3). Then this matrix is passed through a shared MLP and transformed into a matrix Q′, which is of shape Nl+1 × K × dquery. Next, we pass Gin through a shared MLP to transform it into a new matrix G′in, which is of shape Nl+1 ×K × dkey . We concatenate the query matrix Q′ with the key matrix G′in along the feature dimension. We denote this matrix as [Q′,G′in], which is of shape Nl+1 × K × (dquery + dkey). [Q′,G′in] is passed through a shared MLP to obtain the scores of all the K neighbors. We denote the scores as matrix S of shape Nl+1×K×dl+1. Note that S has the same shape asGout. And the scores S of the K neighbors are adaptively computed according to the feature at the center point yk and features at its K neighbors {xj |j ∈ Bx(yk)}. We apply a softmax operation along the neighbor dimension (the second dimension) of S to obtain the weight matrix of all the K neighbors. We denote it as W , which is of shape Nl+1 × K × dl+1. Note that we manually set the weights of the padded dummy neighbors to 0 in W . Then the weight matrix W and the value matrix Gout are dot producted along the neighbor dimension (the second dimension) to form the output matrix F ′, which is of shape Nl+1 × dl+1. Finally, F ′ is concatenated with the 3D coordinates of the set {yk|1 ≤ k ≤ Nl+1} to form output of the Set Abstraction module, Fl+1, which is a matrix of shape Nl+1 × (dl+1 + 3).

heading: A.5 PROBLEMS WITH THREE INTERPOLATION
text: The original PointNet++ uses three interpolation to upsample features in the Feature Propagation module. We think that the three interpolation operation is suitable for tasks like point cloud part segmentation, but not suitable in our task. Interpolation means that points close to each other have similar features. Points close to each other tend to have similar semantic labels in a clean point cloud, therefore it is meaningful to use interpolation operation to upsample features in the part segmentation task. However, in our task, the network need to predict a per-point displacement for all points in a noisy point cloud and move it towards a clean point cloud. Points close to each other
do not need to move in a similar direction in general. In fact, they may just need to move towards the opposite direction to form a smooth surface.
We also find that the three interpolation operation lacks the ability to manipulate positions of points accurately in small scales. We first elaborate on the three interpolation operation used in the original PointNet++. Assume we want to upsample features at {yk|1 ≤ k ≤ Nl+1} to {xj |1 ≤ j ≤ Nl}, where {yk|1 ≤ k ≤ Nl+1} is a subset of {xj |1 ≤ j ≤ Nl}. For each xj , assume its three nearest neighbors in {yk|1 ≤ k ≤ Nl+1} are {yk|k ∈ By,3(xj)}. Then feature at xj is obtained through the following equation:
f(xj) =
∑ k∈By,3(xj) w(yk, xj)f(yk)∑
k∈By,3(xj) w(yk, xj) , where w(yk, xj) =
1
||yk − xj ||2 , (7)
f(yk) and f(xj) are features at yk and xj , respectively. We can see that the value of f(xj) is determined by the relative distances between itself and its three nearest neighbors. However, in 3D space, the point that has a specific relative distances to three fixed points is not unique. In fact, the point can move freely on a curve.
Let’s take a very simple example, assume the three nearest neighbors of xj forms a regular triangle. If we move xj along the straight line that passes the center of the triangle and is perpendicular to the plane determined by the triangle, then xj will always have the same relative distances from the three points, which means xj will always have the same interpolated feature, as long as its movement is small enough that its three nearest neighbors do not change.
This property makes the three interpolation operation not able to distinguish some specific points in a small scale, since these points could have the same interpolated value. Therefore, three interpolation operation is not suitable for our task, as we need to accurately manipulate positions of points to make them form a meaningful shape with smooth surfaces and sharp details.

heading: B EXPERIMENT

heading: B.1 DETAILED EXPERIMENTAL SETUP
text: In all experiments, we use the Adam optimizer with a learning rate of 2× 10−4. For experiments of our PDR paradigm in Table 1 and Table 2 in the main text, we use data augmentation described in Appendix Section B.3. We train our Conditional Generation Network for 340 epochs, 200 epochs, and 500 epochs on the MVP, MVP-40, and Completion3D datasets, respectively. We save a checkpoint and evaluate the network’s performance on both the training set and the test set every 20 epochs. Since the generation process of DDPM is very slow, we randomly select 1600 samples from the training set and test set respectively for evaluation. (Test set of the Completion3D dataset has less than 1600 samples. Therefore, we use all samples in the test set for evaluation.) The checkpoint with the lowest CD loss is chosen as the best network. It is used to generate training data for the Refinement Network. We train the Refinement Network for 100 epochs, 150 epochs and 200 epochs on the MVP, MVP-40, and Completion3D datasets, respectively.
Note that we subsampling the test set only when we try to choose a best checkpoint in the training process of the conditional generation network in DDPM. After choosing the best checkpoint, we use it to generate training data to train the refinement network. However, when we evaluate the whole PDR paradigm (composed of the conditional generation network and refinement network) and compare with previous methods, we evaluate them on the complete test set. Therefore, the comparison result in Table 1 and Table 2 in the main text is reliable and fair.
For ablation studies in Table 3, the Conditional Generation Networks are trained without data augmentation for 300 epochs. All the Refinement Networks are trained on the same data generated by our proposed Conditional Generation Network trained with data augmentation. The Refinement Networks are trained for 100 epochs.
All baseline methods are rerun under the data augmentation described in Appendix Section B.3 according to their open source codes. And CD loss is chosen to train all the baseline methods. We will release our codes if the paper is accepted.

heading: B.2 DETAILS OF THE NETWORK STRUCTURE.
text: Detailed network structure is shown in Figure 6. We present the number of points and feature dimension in each level of the Feature Extraction network and the Denoise network. The distances to define neighbors are provided in Appendix Section A.2.

heading: B.3 DATA AUGMENTATION
text: We use rotation, mirror, translation, and scaling as data augmentation methods during training. Rotation is performed along the upward direction of the shapes. The upward direction is the y-axis in MVP dataset and Completion3D dataset, while upward direction in the MVP-40 dataset is the zaxis. And the rotation angle is uniformly sampled from the interval [−a, a], where a is a predefined hyper-parameter that controls the magnitude of the rotation.
Mirror operation is performed with respect to the two planes that are parallel to the upward direction: x = 0 plane and z = 0 plane for MVP dataset and Completion3D dataset, x = 0 plane and y = 0 plane for MVP-40 dataset. The mirror operation is performed with a probability ofm/2 with respect to the two planes, respectively. m is a predefined hyper-parameter the controls the probability of the mirror operation.
Same as some previous works (Wang et al., 2020; Xia et al., 2021), we observe that most objects in the MVP dataset and Completion3D dataset have reflection symmetry with respect to the xy plane. Therefore, we mirror the partial input with respect to this plane and concatenate the mirrored points with the original partial input for these two datasets. We subsample this concatenated point cloud from 4096 points to 3072 points by farthest point sampling to obtain a uniform point cloud. We label the original points with 1 and the mirrored points with −1. This concatenated point cloud is feed to both the Conditional Generation Network and the Refinement Network, so that they could learn whether an object has reflection symmetry and determine whether to utilize the mirrored points according to their −1 label. Translation is achieved by adding a randomly sampled 3D vector to every point in the incomplete point cloud and the complete point cloud. Each component of the 3D translation vector is sampled from a Gaussian distribution with zero mean and standard deviation of σ, where σ is a predefined hyper-parameter the controls the magnitude of the translation operation.
We also randomly samples a scaling factor uniformly from the interval [δlow, δhigh] when loading a training pair. The scaling factor is multiplied to the coordinates of all points in the incomplete point cloud and the complete point cloud.
We observe that data augmentations can prevent the network from overfitting on the training set, but could also lead to performance drop on the test set. Therefore, we use different data augmentation schemes to train the Conditional Generation Network in the DDPM and the Refinement Network. When training the Conditional Generation Network, we hope the network does not overfit on the training set, because we need it to generate training samples to train the Refinement Network. Therefore, we use data augmentations of large magnitudes to train the Conditional Generation Network. However, when training the Refinement Network, high performance is the top priority. Therefore, we use data augmentations of small magnitudes to train the Refinement Network. We also use data augmentation to train baseline methods. The data augmentations are the same ones that we use to train the Refinement Network. The details of the data augmentation is shown in Table 4.

heading: B.4 SCALE-INCONSISTENCY ISSUE OF THE COMPLETION3D DATASET
text: We find that many pairs of incomplete-complete point clouds have inconsistent scales in the Completion3D dataset. A few inconsistent examples are shown in Figure 7. Ideally, the incomplete point should overlap with the complete point cloud in 3D space, but many incomplete-complete pairs in the Completion3D dataset cannot overlap with each other due to inconsistent scales, which misleads
the network training. Moreover, the scale-inconsistency issue also gives rise to unreliable evaluation results, as we expect the network to predict a complete point cloud of a consistent scale with the incomplete point cloud. Therefore, it is necessary to correct the scales of these pairs before using the dataset.
We leverage the one-side CD loss to identify and correct these pairs. For a consistent pair of the incomplete point cloud C and complete point cloudX , the one-side CD loss should be very low:
L1-Side CD(C,X) = 1 |C| ∑ c∈C min x∈X ||c− x||2. (8)
We find the correct scale of the incomplete point cloud by optimizing the following problem
min δ L1-Side CD(δC,X). (9)
This optimization problem is solved by using the python package scipy.optimize.fmin for every pair of point clouds in the Completion3D dataset. We consider the pairs with a scale factor δ greater than 1.05 or less than 0.95 as inconsistent pairs, and then correct its scale inconsistency by multiplying the scale factor δ to these incomplete point clouds. In the training set, we find 2.81% pairs are inconsistent. The inconsistent pairs are also discovered in the validation set. We can not verify the test set because the ground truth complete point cloud is not released.
We did not use the online Completion3D benchmark server to evaluate our method and previous methods for the following reasons: 1) the website server was out of service, as it gave no feedback for any submissions at the time we conduct this work; 2) the ground truth complete point clouds in test set of the Completion3D dataset are not released, and hence we can not verify whether this inconsistency problem is also present in the test set. Therefore, we use the test set provided in the work (Wang et al., 2020), which contains 1200 pairs of incomplete-complete point clouds for testing. It contains the same set of objects as the test set of the original Completion3D dataset. After correcting inconsistent pairs in this test set, we evaluate our method and previous methods on this revised test set to achieve fair and reliable comparisons.

heading: B.5 COMPLETE EXPERIMENT RESULTS FOR MVP-40 DATASET
text: In Table 1 in the main text, we only present the completion result at the missing ratio of 50% and 12.5% for the MVP-40 dataset. We present the complete experiment result on MVP-40 dataset including result at the 25% missing ratio in Table 5 below.

heading: B.6 COMPLETE EXPERIMENT RESULTS FOR NETWORK ABLATION STUDY
text: In Section 5.3 in the main text, we conduct ablation study of our proposed network architecture, and results are shown in Table 3. Note that the conditional generation networks in DDPM in Table 3 are trained without data augmentation. However, it is actually very important to train the conditional generation networks with data augmentation, because we need to prevent it from overfitting on the training set, so that they can generate coarse point clouds of consistent distribution on the training set and test set to train the refinement network. Therefore, we provide the training results with data augmentation in Table 6. The data augmentation is specified in Table 4 for MVP dataset.
Same as Table 3 in the main text, “PA-Deonv & Att.” is our proposed complete network shown in Figure 2. “PA-Deonv” is our network without attention mechanism. “PointNet++” further removes the PA-Deconv module. “Concate xt & c” removes FT modules. It concatenates c with xt as Zhou et al. (2021) do, and feed them to a single PointNet++ with attention mechanism and PA-Deconv. “Pointwise-net” only utilizes a global feature extracted from the incomplete point cloud. We can see that these proposed modules indeed improve the networks’ performance. Our proposed networks achieve superior results both with and without data augmentation.
We also observe that networks generally achieve better performance on both the training set and test set without data augmentation, but they tend to overfit on the training set. This is undesirable because we need these conditional generation networks to generate training data for the refinement networks. It is very important for them to generate coarse point clouds that have consistent distributions on the training set and the test set. Indeed, we can see that the overfitting problem is largely mitigated in the presence of data augmentation.

heading: B.7 COMPLETE EXPERIMENT RESULTS FOR DDPM ACCELERATION
text: The complete experiment results of the DDPM acceleration is shown in Table 7. We can see that the quality of coarse point clouds generated by the accelerated DDPMs has dropped considerably. However, with the help of the Refinement Network, the performance drop of the final refined point clouds is slight. This demonstrates the strong refinement capability of our proposed network architecture shown in Figure 2 in the main text.

heading: B.8 GENERATION DIVERSITY OF THE PDR PARADIGM
text: In this section, we discuss whether the PDR Paradigm can generate diverse completion results for the same incomplete point cloud. Although there is no stochasticity in the refinement network, we find our PDR paradigm still demonstrates some kind of diversity in the completion results.
We know that DDPM itself is a probabilistic model and can generate diverse completion results. The refinement network receives a coarse completion from the DDPM and then refine it according to the condition point cloud, i.e., the incomplete point cloud. The final refined result surely depends on the condition point cloud, but also depends on the coarse point cloud received from the DDPM. The refinement network can only refine the coarse point cloud in a small scale, because we multiply the output of the refinement network by a small constant γ = 0.001 as described in Section 3.3 in the main text. Therefore, the overall sketch of the coarse point cloud will be preserved after the refinement. This explains why the PDR paradigm still bears low EMD loss as the DDPM, even though we use CD loss to train the refinement network, because the refinement network does not change the overall distribution of the coarse shape generated by DDPM.
Back to the diversity issue, if the coarse completion results from DDPM demonstrate diversity for the same incomplete point cloud, the refined point clouds will also demonstrate some diversity because the inputs to the refinement network are different. Figure 8 shows some examples where the PDR paradigm demonstrate diversity in the completion results.

heading: B.9 MORE VISUALIZATION RESULTS

Review summaries:
The paper tackles the problem of shape-level point cloud completion in a fully supervised setting. The big picture of the proposed method is to use a conditional DDPM to generate a noisy but complete point cloud given incomplete input, and then use another refinement network, also conditional on the incomplete input, to further refine the noisy point cloud. Both networks adopt a novel, dual-path network architecture, based on PointNet++, to allow localized guidance from the incomplete point cloud. Thanks to the 2-staged approach, up to 50x acceleration of evaluation speed is achievable using step jumping, with moderate performance drop. The model is evaluated on MVP, MVP-40 and Completion3D datasets.

In this paper, the authors propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion. The proposed method effectively and efficiently extracts multi-level features from partially observed point clouds to guide completion. Moreover, it accurately manipulates spatial locations of 3D points to obtain smooth surfaces and sharp details. Experimental results demonstrate its state-of-the-art performance on the mainstream datasets and benchmarks.

This paper proposed a novel way to utilize DDPM's for point cloud completion. They use a conditional DDPM to generate a plausible "complete" point cloud from pure Gaussian noise, conditional on a partial subset of the point cloud points. Their main contribution is is in the design of the conditional feature extraction and denoising subnet, which essentially form dual path connected U-Net type structures with internal modules based on an improved PointNet++ design. Additionally, the authors propose a refinement network to mitigate the computational burden of needing so many diffusion steps, which seems to offer an acceptable performance tradeoff for increases computational efficiency. The method convincingly demonstrates superior performance over its competitors, with crisp details preserved on the datasets tested.