A C ONDITIONAL P OINT D IFFUSION -R EFINEMENT P ARADIGM FOR 3D P OINT C LOUD C OMPLETION
A BSTRACT
3D point cloud is an important 3D representation for capturing real world 3D objects.
However, real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications.
Most existing point cloud completion methods use Chamfer Distance (CD) loss for training.
The CD loss estimates correspondences between two point clouds by searching nearest neighbors, which does not capture the overall point density distribution on the generated shape, and therefore likely leads to non-uniform point cloud generation.
To tackle this problem, we propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion.
PDR consists of a Conditional Generation Network (CGNet) and a ReFinement Network (RFNet).
The CGNet uses a conditional generative model called the denoising diffusion probabilistic model (DDPM) to generate a coarse completion conditioned on the partial observation.
DDPM establishes a one-to-one pointwise mapping between the generated point cloud and the uniform ground truth, and then optimizes the mean squared error loss to realize uniform generation.
The RFNet refines the coarse output of the CGNet and further improves quality of the completed point cloud.
Furthermore, we develop a novel dual-path architecture for both networks.
The architecture can (1) effectively and efficiently extract multi-level features from partially observed point clouds to guide completion, and (2) accurately manipulate spatial locations of 3D points to obtain smooth surfaces
and sharp
details.
Extensive experimental results on various benchmark datasets show that our PDR paradigm outperforms previous state-of-the-art methods for point cloud completion.
Remarkably, with the help of the RFNet, we can accelerate the iterative generation process of the DDPM by up to 50 times without much performance drop.
I NTRODUCTION
With the rapid developments of 3D sensors, 3D point clouds are an important data format that captures 3D information owing to their ease of acquisition and efficiency in storage.
Unfortunately, point clouds scanned in the real world are often incomplete due to partial observation and self occlusion.
It is important to recover the complete shape by inferring the missing parts for many downstream tasks such as 3D reconstruction, augmented reality and scene understanding.
To tackle this problem, many learning-based methods (Yuan et al., 2018; Yang et al., 2018; Tchapmi et al., 2019;
Xie et al., 2020; Liu et al., 2020; Pan et al., 2021) are proposed, which are supervised by using either the Chamfer Distance (CD) or Earth Mover Distance (EMD) to penalize the discrepancies between the generated complete point cloud and the ground truth.
However, CD loss is not sensitive to overall density distribution, and thus networks trained by CD loss could generate non-uniform point cloud completion results (See Figure 10 and 11 in Appendix).
EMD is more distinctive to measure density distributions, but it is too expensive to compute in training.
The absence of an effective and efficient training loss highly limits the capabilities of many existing point cloud completion networks.
noise step by step towards a coarse completion of the partial observation through a diffusion model (DDPM).
Then it refines the coarse point cloud by one step to obtain a high quality point cloud.
We find that denoising diffusion probabilistic models (DDPM) (Sohl-Dickstein et al., 2015; Ho et al., 2020) can potentially generate uniform and high quality point clouds with an effective and efficient loss function.
It can iteratively move a set of Gaussian noise towards a complete and clean point cloud.
DDPM defines a one-to-one pointwise mapping between two consecutive point clouds in the diffusion process, which enables it to use a simple mean squared error loss function for training.
This loss function is efficient to compute and explicitly requires the generated point cloud to be uniform, as a one-to-one point mapping is naturally established between the generated point cloud and the ground truth.
Point cloud completion task can be treated as a conditional generation problem in the framework of DDPM (Zhou et al., 2021;
Luo & Hu, 2021).
Indeed, we find the complete point clouds generated by a conditional DDPM often have a good overall distribution that uniformly covers the shape of the object.
Nonetheless, due to the probabilistic nature of DDPM and the lack of a suitable network architecture to train the conditional DDPM for 3D point cloud completion in previous works, we find DDPM completed point clouds often lack smooth surfaces and sharp details (See Figure 1 and Appendix
Figure 12), which is also reflected by their high CD loss compared with state-of-the-art point cloud completion methods in our experiments.
Another problem with DDPM is its inefficiency in the inference phase.
It usually takes several hundreds and even up to one thousand forward steps to generate a single point cloud.
Several methods (Song et al., 2020; Nichol & Dhariwal, 2021; Kong & Ping, 2021) are proposed to accelerate DDPM using jumping steps without retraining networks, which however, leads to an obvious performance drop when using a small number of diffusion steps.
In this work, we propose the Conditional P oint D iffusion- R efinement ( PDR ) paradigm to generate both uniform and high quality complete point clouds.
As shown in Figure 1, our PDR paradigm performs point cloud completion in a coarse-to-fine fashion.
Firstly, we use the Conditional Generation Network (CGNet) to generate a coarse complete point cloud by the DDPM conditioned on the partial point cloud.
It iteratively moves a set of Gaussian noise towards a complete point cloud.
Following, the ReFinement Network (RFNet) further refines the coarse complete point cloud generated from the Conditional Generation Network with the help of partial point clouds.
In addition,
RFNet can be used to refine the low quality point clouds generated by an accelerated DDPM, so that we could enjoy an acceleration up to 50 times, while minimizing the performance drop.
In this way, the completion results generated by our PDR paradigm demonstrate both good overall density distribution ( i.e. uniform) and sharp local details.
Both CGNet and RFNet have a novel dual-path network architecture shown in Figure 2, which is composed of two parallel sub-networks, a Denoise subnet and a Condition Feature Extraction subnet for noisy point clouds and partial point clouds, respectively.
Specifically, we propose Point
Adaptive Deconvolution (PA-Deconv) operation for upsampling, which can effectively manipulate spatial locations of 3D points.
Furthermore, we propose the Feature Transfer (FT) module to directly transmit encoded point features at different scales from the Condition Feature Extraction subnet to the corresponding hierarchy in the Denoise subnet.
Extensive experimental results show that
our
PDR paradigm can provide new state-of-the-art performance for point cloud completion.
Our Key contributions can be summarized as: 1)
We identify conditional DDPM to be a good model with an effective and efficient loss function to generate uniform point clouds in point cloud completion task.
By using RFNet to refine the coarse point clouds, our PDR paradigm can generate complete point cloud with both good overall density distribution ( i.e. uniform) and sharp local details.
3) We design novel point learning modules, including PA-Deconv and Feature Transfer modules, for constructing CGNet in DDPM and RFNet, which effectively and efficiently utilizes multi-level features extracted from incomplete point clouds for point cloud completion. 4)
With the help of our proposed RFNet, we can accelerate the generation process of DDPM up to 50 times without a significant drop in point cloud quality.
P ROBLEM STATEMENT
In this paper, we focus on the 3D point cloud completion task.
A 3D point cloud is represented by
N points in the 3D space:
X = { x j | 1 ≤ j ≤ N } , where each x j ∈ R 3 is the 3D coordinates of the j -th point.
We assume the dataset is composed of M data pairs { ( X i , C i )
| 1 ≤ i ≤ M } , where X i is the i -th ground-truth point cloud, and C
i is the incomplete point cloud from a partial observation of X i .
The goal is to develop a model that completes the partial observation C i and outputs a point cloud as close to
the ground truth X i as possible.
For algebraic convenience, we let x ∈ R 3 N be the vector form of a point cloud X , and similarly c be the vector form of C .
M ETHODOLOGY
We consider the point cloud completion task as a conditional generation problem, where the incomplete point cloud C serves as the conditioner.
We use the powerful generative model called denoising diffusion probabilistic models (DDPM)
(Sohl-Dickstein et al., 2015; Ho et al., 2020; Kong et al., 2020) to first generate a coarse completion of the partial observation.
Then we use another network to refine the coarse point cloud to improve its visual quality.
Our point cloud completion pipeline is shown in Figure 1.
We first briefly introduce the theory of DDPM in Section 3.1, and then describe detailed architecture of the Conditional Generation Network (CGNet) and ReFinement
Network (RFNet) in Section 3.2 and Section 3.3.
3. B ACKGROUND ON C ONDITIONAL D ENOISING D IFFUSION P ROBABILISTIC M ODELS
We assume p data to be the distribution of the complete point cloud x i in the dataset, and p latent =
N ( 0 3 N , I 3 N × 3 N ) to be the latent distribution, where N is the Gaussian distribution.
Then, the conditional DDPM consists of two Markov chains called the diffusion process and the reverse process.
Both processes have length equal to T .
We set T = 1000 in this paper.
The Diffusion Process.
The diffusion process is a Markov process that adds Gaussian noise into the clean data distribution p data until the output distribution is close to p latent .
The diffusion process is irrelevant of the conditioner, the incomplete point cloud c i .
Formally, let x 0 ∼ p data .
We use the superscript to denote the diffusion step t .
For conciseness, we omit the subscription i in the following discussion.
The diffusion process from clean data x 0 to x T is defined as
[Equation] [Equation]
q ( x t | x t −
[Equation]
N
[Equation]
x t
[Equation]
The hyperparameters β t are pre-defined, small positive constants (See details in Appendix Section A.1).
According to Ho et al.
(2020), there is a closed form expression for q ( x t | x 0 ) .
We first define constants α t = 1 − β t , ¯ α t = (cid:81) ti =1 α i .
Then, we have q ( x t | x 0 ) =
N (cid:0) x t ; √ ¯ α t x 0 , (1 − ¯ α t ) I (cid:1) .
Therefore, when T is large enough, ¯ α t goes to 0 , and q ( x T | x 0 ) becomes close to the latent distribution p latent ( x T ) .
Note that x t can be directly sampled through the following equation:
[Equation]
ϵ is a standard Gaussian noise
[Equation]
We emphasize that q ( x t | x t − 1 ) can be seen as a one-to-one pointwise mapping as x t can be sampled through the equation x t = √ 1 − β t x t − 1 + β t ϵ .
Therefore, the order of points in x 0 is preserved in the diffusion process.
However, it does not matter what kind of order we input the points in x 0 .
That is because when T is large enough, x T will become a Gaussian distribution.
Every point in a
Gaussian distribution is equivalent and there is no way to distinguish one point from another.
The Reverse Process.
The reverse process is a Markov process that predicts and eliminates the noise added in the diffusion process.
The reverse process is conditioned on the conditioner, the incomplete point cloud c .
Let x T ∼ p latent be a latent variable.
The reverse process from latent x T to clean data x 0 is defined as
[Equation]
) , where p [Equation] x [Equation]
=
[Equation] µ [Equation] x
[Equation]
c
[Equation]
, σ [Equation] [Equation] The mean µ θ ( x t , c , t ) is a neural network parameterized by θ and the variance σ 2 t is a time-step dependent constant.
To generate a sample conditioned on c , we first sample x T ∼ N ( 0 3 N , I 3 N × 3 N ) , then draw x t − 1 ∼ p θ ( x t − 1 | x t , c ) for t =
T, T − 1 , · · · , 1 , and finally outputs x 0 .
Training.
DDPM is trained via variational inference.
Ho et al.
(2020) introduced a certain parameterization for µ θ that can largely simplify the training objective.
The parameterization is σ 2 t = 1 − ¯ α t − 1 1 − ¯ α t β t , and µ θ ( x t , c , t )
= 1 √ α t (cid:16) x t − β t √ 1 − ¯ α t ϵ θ ( x t , c , t ) (cid:17) , where ϵ θ is a neural network taking noisy point cloud x t ∼ q ( x t | x 0 ) in equation (2), diffusion step t , and conditioner c as inputs.
Then, the simplified training objective becomes
[Equation]
where U ([ M ]) is the uniform distribution over { 1 , 2 , · · · , M } .
The neural network ϵ θ learns to predict the noise ϵ added to the clean point cloud x 0 , which can be used to denoise the noisy point cloud x t =
√ ¯ α t x 0 + √ 1 − ¯ α t ϵ .
Note that traditional CD loss or EMD loss is NOT present in Equation 4.
The reason that we are able to use the simple mean squared error is because DDPM naturally defines a one-to-one pointwise mapping between two consecutive point clouds in the diffusion process as shown in Equation 1.
Note that at each training step, we not only need to sample a pair of point clouds x i , c i , but also a diffusion step t and a Gaussian noise ϵ .
3. C ONDITIONAL G ENERATION N ETWORK
In this section, we introduce the architecture of Conditional Generation Network (CGNet) ϵ θ .
The inputs of this network are the noisy point cloud x t , the incomplete point cloud c , and the diffusion step t .
We can intuitively interpret the output of ϵ θ as per-point difference between x t and x t −(with some arithmetic ignored).
In addition, ϵ θ should also effectively incorporate multi-level information from c .
The goal is to infer not only the overall shape but also the fine-grained details based on c .
We design a neural network that achieves these features.
The overall architecture is shown in
Figure 2.
It is composed of two parallel sub-networks similar to PointNet++ (Qi et al., 2017b), and they have the same hierarchical structure.
The upper subnet, which we refer as the Condition Feature Extraction subnet, extracts multi-level features from the incomplete point cloud c .
The lower subnet, which we refer as the Denoise subnet, takes the noisy point cloud x t as input.
We also add the diffusion step t , the global feature extracted from c , and multi-level features extracted by the Condition Feature Extraction subnet to the Denoise subnet.
The diffusion step t is first transformed into a 512 -dimension step embedding vector through positional encoding and fully connected (FC) layers (See Appendix Section A.1 for details), and then inserted to every level of the Denoise subnet.
Similarly, the conditioner c is first transformed into a 1024 -length global feature through a two-stage PointNet, and then inserted to every level of the
Denoise subnet.
The multi-level features extracted by the Condition Feature Extraction subnet are
[Figure]
inserted to every level of the Denoise subnet through Feature Transfer modules.
Finally, the Denoise subnet is connected to a shared MLP and outputs ϵ θ ( x t , c , t ) .
Additionally, while Zhou et al.
(2021) argues PointNet++ cannot be used in a DDPM that generates point clouds, we find attaching the absolute position of each point to its feature solves this problem.
See Appendix Section A.3 for detailed analysis.
We also improve the backbone PointNet++ so that it manipulates positions of points more accurately.
In the next paragraphs, we elaborate on the building blocks of the improved PointNet++: Set Abstraction modules in the encoder, and Feature Propagation modules in the decoder, and Feature
Transfer modules between the Condition Feature Extraction subnet and the Denoise subnet.
Set Abstraction (SA) Module.
Similar to PointNet++, this module subsamples the input point cloud and propagates the input features.
Assume the input is { x j | 1 ≤ j ≤ N l } , where x j is the 3D coordinate of the j -th point and N l is the number of input points to the Set Abstraction module of level l .
Each point has a feature of dimension d l .
We concatenate these features with their corresponding 3D coordinates and group them together to form a matrix F l of shape N l × ( d l + 3) .
The SA module first uses iterative farthest point sampling (FPS) to subsample the input points to
N l +1 points: { y k | 1 ≤ k ≤ N l +1 } .
Then it finds K neighbors in the input { x j | 1 ≤ j ≤ N l } for each y k .
We denote the K neighbors of y k as { x j | j ∈ B x ( y k ) } , where B x ( y k ) is the index set of the K neighbors.
See definition of neighbors in Appendix A.2.
These neighbors and their features are grouped together to form a matrix G in of shape N l +1 × K × ( d l + 3) .
Then a shared multilayer perceptron (MLP) is applied to transform the grouped feature G in to G out , which is of shape
N l +1 × K ×
d l +1 and d l +1 is the dimension of the output feature.
Finally, a max-pooling is applied to aggregate features from the K neighbors { x j | j ∈ B x
( y k ) } to y k .
We obtain the output of the SA module, the matrix F l +1 , which is of shape N l +1 ×
d l +1 .
Note that we need to incorporate information of the diffusion step embedding and global feature extracted from the incomplete point cloud c to every SA module in the Denoise subnet as shown in
Figure 2.
We insert these information to the shared MLP that transforms G in to G out mentioned in the above paragraph.
Specifically, we add them to the channel dimension of the intermediate feature maps in the shared MLP.
Figure 3(a) illustrates this process in details.
Inspired by the works (Pan et al., 2021; Zhao et al., 2020), we also replace the max-pooling layer in the SA module with a self-attention layer.
Feature at y k is obtained by a weighted sum of the features of its K neighbors { x j | j ∈ B x
( y k ) } instead of max-pooling, and the weights are adaptively computed through the attention mechanism.
See Appendix A.4 for details of this attention layer.
Feature Propagation (FP) Module.
Similar to PointNet++, this module upsamples the input point cloud and propagates the input features.
In PointNet++, the features are upsampled from { y k | 1 ≤ k ≤ N l +1 } to { x j | 1 ≤ j ≤ N l } by three interpolation: Feature at x j is a weighted sum of the features of its three nearest neighbors in { y k | 1 ≤ k ≤ N l +1 } .
We think that the three interpolation operation is not suitable in our task, because the interpolation operation may lose some information about the accurate positions of the points
We propose to use a Point Adaptive Deconvolution (PA-Deconv) module to upsample the point features.
In the SA module, the features are mapped from set { x j | 1 ≤ j ≤ N l } to { y k | 1 ≤ k ≤
N l +1 } .
The key step is to find the neighbors { x j | j ∈ B x ( y k ) } ⊆ { x j | 1 ≤ j ≤ N l } for each y k .
Features at { x j | j ∈ B x ( y k ) } are transformed and then aggregated to the point y k through either max-pooling or attention mechanism.
Now in the FP module, we need to map features the other way around: from { y k | 1 ≤ k ≤ N l +1 } to { x j | 1 ≤ j ≤ N l } .
We can achieve this goal through a similar method.
We find the neighbors { y k | k ∈ B y ( x j ) } ⊆ { y k | 1 ≤ k ≤ N l +1 } for each x j .
Features at { y k | k ∈ B y ( x j ) } are transformed through a shared MLP, and then aggregated to the point x j through attention mechanism.
Similar to SA modules, we insert information of the diffusion step embedding and the global feature extracted from the incomplete point cloud c to the shared MLP in every FP module in the Denoise subnet.
Finally, same as the original FP module in PointNet++, the upsampled features are concatenated with skip linked point features from the corresponding SA module, and then passed through a unit PointNet.
The Feature Propagation module are applied four times and features are eventually propagated to the original input point cloud.
Feature Transfer (FT) Module.
The FT module transmits information from the Condition Feature Extraction subnet to the Denoise subnet.
Assume the point cloud at level l in the Condition
Feature Extraction subnet is { z l | 1 ≤ l ≤ S l } , where S l is the number of points at level l in the
Condition Feature Extraction subnet.
The FT module maps the features at points { z r | 1 ≤ r ≤ S l } to points at the same level in the Denoise subnet, which are { x j | 1 ≤ j ≤ N l } .
Then the mapped features are concatenated with the original features at { x j | 1 ≤ j ≤ N l } .
Next, the concatenated features are fed to the next level of the Denoise subnet.
In this way, the Denoise subnet can utilize local features at different levels of the incomplete point cloud to manipulate the noisy input point cloud to form a clean and complete point cloud.
The key step in this process is to map features at { z r | 1 ≤ r ≤ S l } to { x j | 1 ≤ j ≤ N l } .
We adopt a similar strategy in the SA module.
We find the neighbors { z r | r ∈ B z ( x j ) } ⊆ { z r | 1 ≤ r ≤ S l } for each x j .
Features at { z r | r ∈ B z
( x j ) } are transformed through a shared MLP, and then aggregated to the point x j through the attention mechanism, which is a weighted sum of the features at { z r |
r ∈ B z ( x j ) } .
We set a small distance to define neighbors in low level FT modules, so that they only query the adjacent parts of the incomplete point cloud c to preserve local details in it.
Large distances are set to define neighbors in high level FT modules.
This makes high-level FT modules have large receptive fields, so that they can query a large part of the incomplete point cloud to infer high level 3D structural relations.
See detailed neighbor definitions in Appendix Section A.2.
3. R EFINEMENT N ETWORK
We denote the coarse point cloud generated by the Conditional Generation Network as U .
We use another network of the same architecture shown in Figure 2 to predict a per-point displacement for U to refine it.
The differences are that the input to the Denoise subnet becomes U and we do not need to insert the diffusion step embedding to the Denoise subnet.
The predicted displacement are added to U to obtain the refined point cloud V : v = u + γ ϵ f ( u , c ) , where v , u , c are the concatenated 3D coordinates of the point clouds V , U , C , respectively.
γ is a small constant and we set it to 0 .in
all our experiments.
ϵ f is the ReFinement Network.
We use the Chamfer Distance (CD) loss between the refined point cloud V and ground truth point cloud X to supervise the network ϵ f :
[Equation]
where | V | means number of points in V .
If we also want to upsample points in U by a factor of λ , we can simply increase the output dimension of the network ϵ f .
In addition to predicting one 3D displacement of each point in U , we predict another λ displacements.
We consider each point in the refined point cloud V as the center of a group of λ points in the dense point cloud that we want to generate.
The additional λ displacements are added to every point in V to form a dense point cloud.
Figure 3(c) illustrates how we upsample every point in V by a factor of λ = 8 .
When training the ReFinement Network ϵ f , parameters in the Conditional Generation Network ϵ θ are fixed.
It is not practical to generate coarse point clouds U on the fly in the training process of ϵ f , because the generation process of DDPM is slow.
Instead, we generate and save the coarse point clouds in advance.
Due to the probabilistic nature of DDPM, we generate 10 coarse point clouds for each incomplete point cloud in the dataset to increase diversity of training data.
R ELATED W ORKS
Point cloud completion.
Inspired by the pioneering work, PointNet (Qi et al., 2017a), researchers focus on learning global feature embeddings from 3D point clouds for completion (Yuan et al., 2018; Tchapmi et al., 2019), which however cannot predict local and thin shape structures.
To address these challenges, following research works (Pan, 2020; Xie et al., 2020; Zhang et al., 2020;
Wen et al., 2021; Yu et al., 2021; Pan et al., 2021) exploit multi-scale local point features to reconstruct complete point clouds with fine-grained geometric details.
Recently, PointTr (Yu et al., 2021) and VRCNet (Pan et al., 2021) provide impressive point cloud completion results with the help of attention-based operations.
Nonetheless, as a challenging conditional generation problem, point cloud completion has not been fully resolved.
DDPM for point cloud generation.
Luo & Hu (2021) are the first to use DDPM for unconditional point cloud generation.
They use a Pointwise-net to generate point clouds, which is similar to a 2-stage PointNet used for point cloud part segmentation.
However, the Pointwise-net could only receive a global feature.
It can not leverage fine-grained local structures in the incomplete point cloud.
Zhou et al.
(2021) further use conditional DDPM for point cloud completion by training a point-voxel CNN (Liu et al., 2019), but the way they use the incomplete point cloud c is different from ours.
They directly concatenate c with the noisy input x t , and feed them to a single pointvoxel CNN.
This may hurt performance of the network, because the concatenated point cloud is very likely to be non-uniform.
In addition, x t is very different from c for large t ’s due to the large noise magnitude in x t .
Feeding two point clouds of very different properties to a single network at once could be quite confusing for the network.
The other major difference is that they do not refine or upsample the coarse point cloud generated by DDPM like we do.
E XPERIMENTS 5. D ATASETS
We conduct point cloud completion experiments on the following three datasets.
MVP.
The MVP dataset (Pan et al., 2021) has 62400 training partial-complete point cloud pairs and 41600 testing pairs sampled from ShapeNet (Chang et al., 2015).
Every partial point cloud has 2048 points.
In particular, MVP dataset provides ground truth point clouds with different resolutions, including 2048, 4096, 8192, and 16384 points.
MVP-40.
The MVP-40 dataset (Pan et al., 2021) consists of 41600 training samples and 64168 testing samples from 40 categories in ModelNet40 (Wu et al., 2015).
Its partial point clouds are sampled from complete point clouds with a pre-defined missing ratio, i.e. , 50%, 25% and 12.5% missing.
Both the partial and complete point clouds havepoints.
Completion3D.
It (Tchapmi et al., 2019) consists of 28974 point cloud pairs for training and 1184 for testing from 8 object categories in ShapeNet.
Both the partial and complete point clouds have 2048 points.
We find some pairs of the incomplete point cloud and complete point cloud have inconsistent scales in the Completion3D dataset.
We correct the scales and use the corrected dataset in our experiments.
See details in Appendix Section B.4.
5. E VALUATION M ETRICS
We use the Chamfer Distance (CD), Earth Mover Distance (EMD), and F1 score to evaluate the quality of the generated point clouds.
CD distance is defined in Equation 5.
Earth Mover Distance.
Consider the predicted point cloud V and the ground truth point cloud X of equal size N = | V | = | X | , the EMD loss penalizes their shape discrepancy by optimizing a transportation problem.
It estimates a bijection ϕ : V ←→ X between V and X :
[Equation]
F1 score.
To compensate the problem that CD loss can be sensitive to outliers, we follow previous methods (Pan et al., 2021; Tatarchenko et al., 2019) and use F1 score to explicitly evaluates the distance between object surfaces, which is defined as the harmonic mean between precision L P ( ρ ) and recall L R ( ρ ) :
L F1 =
2 L P ( ρ ) L R ( ρ ) L P ( ρ )+ L R ( ρ ) , where L P ( ρ ) =
| V | (cid:80) v ∈ V
[Equation]
(cid:21) , and ρ is a predefined distance threshold.
We set ρ = 10 − 4 for the
MVP and Completion3D datasets, and set ρ = 10 − 3 for the MVP-40 dataset.
[Table] [Table]
[Table]
5. P OINT C LOUD C OMPLETION
We compare our point cloud completion method with previous state-of-the-art point cloud completion methods.
The comparison is performed on MVP, MVP-40, and Completion3D datasets.
Results are shown in Table 1.
We also conduct multi-resolution experiments on the MVP dataset, and results are shown in Table 2.
Detailed experimental setups are provided in Appendix Section B.1.
We can see that our Conditional P oint D iffusion- R efinement (PDR) paradigm outperforms other methods by a large margin in terms of EMD loss, which is highly indicative of uniformness (Zhang et al., 2021).
We also achieve the highest F1 score and very low CD loss.
Although VRCNet sometimes has lower CD loss than ours, it tends to put more points in the parts that are known in the incomplete point clouds, while put less points in the missing part (See Figure 10 in Appendix).
In this way, its CD loss could be very low, but this non-uniformness is undesired and leads to very high EMD loss.
We compare our method with other baselines in terms of visual quality of completed point clouds in Figure 4.
We can see that our method generally has better visual quality.
More samples are provided in Figure 9 and Figure 11 in Appendix.
We also find that our PDR paradigm demonstrate some diversity in completion results as discussed in Appendix B.8.
Ablation Study.
We study the effect of attention mechanism, Point Adaptive Deconvolution (PADeconv) module, and Feature Transfer (FT) module in term of training the Conditional Generation Network and the Refinement Network.
The experiments are conducted on MVP dataset at the resolution of 2048 points and results are shown in Table 3.
“PA-Deonv & Att.” is our proposed complete network shown in Figure 2.
“PA-Deonv” removes attention mechanism.
“PointNet++” further removes PA-Deconv module.
“Concate x t & c ” removes FT modules.
It concatenates c with x t as Zhou et al.
(2021) do, and feed them to a single PointNet++ with attention mechanism and PADeconv.
“Pointwise-net” only utilizes a global feature extracted from the incomplete point cloud.
We can see that these proposed modules indeed improve the networks’ performance.
Note that the conditional generation networks in Table 3 are trained without data augmentation.
Complete experimental results with data augmentation are presented in Appendix Section B.6.
All the refinement networks are trained using data generated by our proposed complete dual-path network trained with data augmentation.
If the other ablated networks use training data generated by themselves, they would have worse refinement results.
[Figure]
DDPM acceleration.
Kong & Ping (2021) propose to accelerate the generation process of DDPM by jumping steps in the reverse process.
The method does not need retraining of the DDPM.
We directly apply their method to our 3D point cloud generation network.
However, we observe a considerable performance drop in the accelerated DDPM.
On the MVP dataset (2048 points), the original 1000 -step DDPM achieves 10 .
7 × 10 − 4 CD loss.
CD losses of the accelerated 50 -step and 20 -step DDPMs increase to 13 .
2 × 10 − 4 and 18 .
1 × 10 − 4 , respectively.
Fortunately, we can generate coarse point clouds using the accelerated DDPMs and use another Refinement Network to refine them.
The refined point clouds of the 50 -step and 20 -step DDPMs bear CD losses of 5 .
68 × 10 −and 5 .
78 × 10 − 4 , respectively.
Compared with the original 1000 -step DDPM, which has a CD loss of 5 . 66 × 10 − 4 , it’s quite temping to accept a slight drop in performance for an acceleration up to 50 times.
Complete results of the acceleration experiment are presented in Appendix Section B.7.
5. E XTENSION TO C ONTROLLABLE G ENERATION
Our conditional PDR paradigm can be readily extended to controllable point cloud generation conditioned on bounding boxes of every part of an object.
We sample points on the surfaces of the bounding boxes and regard this point cloud as the conditioner, just like the incomplete point cloud serves as the conditioner for point cloud completion.
We conduct experiments on the chair category of PartNet (Mo et al., 2019) dataset.
Two examples are shown in Figure 5.
It’s interesting that our method can generate a shape different from the ground truth in some details, but be still plausible.
C ONCLUSION
In this paper, we propose the Conditional P oint D iffusion- R efinement (PDR) paradigm for point cloud completion.
Our method effectively leverages the strong spatial correspondence between the adjacent parts of the incomplete point cloud and the complete point cloud through
the proposed
Feature Transfer module, which could also infer high-level 3D structural relations.
We make improvements of the backbone PointNet++ to make it capable of accurately manipulating positions of input points.
Our method demonstrate significant advantages over previous methods, especially in terms of the overall distribution of the generated point cloud.
We also find that our method has great potential to be applied in other conditional point cloud generation tasks such as controllable point cloud generation.
A PPENDIX A M ETHODOLOGY D ETAILS A. D ETAILS OF THE DDPM
Hyperparameters β t .
We define hyperparameters β t in the diffusion process according to a linear schedule.
We let β 1 = 1 × 10 − 4 and β T = 2 × 10 − 2 .
Then, we define β t = t − 1 T − 1 · ( β T − β 1 ) , t = 1 , 2 , · · · , T .
Diffusion step embedding.
The network ϵ needs to output different ϵ θ ( x t , c , t ) for different diffusion steps t ∈ { 1 , · · · , T } .
We first use positional encoding (Vaswani et al., 2017) to encode each t into a 2 d t dimensional vector ϕ emb ( t ) =
[sin( ψ ( t )) , cos( ψ ( t ))] , where
[Equation]
We set d t = 64 in experiments.
Then, we use two fully-connected (FC) layers to transform ϕ emb ( t ) into a 512 dimensional embedding vector (Ho et al., 2020).
The first FC layer has input dimension 2 d t = 128 and output dimension 512.
The second FC layer has input dimension 512 and output dimension 512.
Both layers are followed by the Swish activation function (Ramachandran et al., 2017).
A. N EIGHBOR D EFINITION
In the Set Abstraction module and the Feature Transfer module, the neighbors are defined as the points that are within a specified distance to the center point.
If a center point has more than K neighbors, we randomly select K neighbors from its neighbors.
If a center point has less than K neighbors, we pad it with dummy neighbors that has the same position as the center point, and has features of zeros.
These dummy neighbors are excluded from the max-pooling operation.
And in the attention mechanism, the weights of the dummy neighbors are manually set to 0 .
In this way, we can guarantee that a center point can always find K neighbors.
We set K = 32 in the Set
Abstraction module and the Feature Transfer module.
In the 4 levels of the Set Abstraction modules, the neighboring distance are set to 0 .
1 , 0 . 2 , 0 . 4 , 0 . 8 , respectively.
In the 9 Feature Transfer modules, the neighboring distance are set to 0 .
1 , 0 . 2 , 0 . 4 , 0 . 8 , 1 , 6 , 0 . 8 , 0 . 4 , 0 . 2 , 0 . 1 , respectively.
Coordinates of samples in all datasets are normalized to the range [ − 1 , 1] .
In the Point Adaptive Deconvolution (PA-Deconv) modules of the Feature Propagation modules, the
K neighbors are defined as the K nearest neighbors of the center point, and we set K = 8 for all
Feature Propagation modules.
A. P ROBLEMS OF V ANILLA P OINT N ET ++
Zhou et al.
(2021) argue that PointNet++ can not be used to train a DDPM.
We observe the same phenomenon in our experiments.
We find that this is because the density of the input noisy cloud x t is too low for large t ’s. Recall that x t = √ ¯ α t x 0 + √ 1 − ¯ α t ϵ , where ϵ is a Gaussian noise.
¯ α t goes to 0 for large t ’s.
This means x t is close to a Gaussian noise when t is large.
The density of a Gaussian noise is much lower than the complete point cloud x 0 .
This is because points in xconcentrate on the surface of some object and x 0 is normalized to the range [ − 1 , 1] , while points from a standard Gaussian distribution could fill the whole space in the range of [ − 3 , 3] .
PointNet++ is originally designed to process point clouds like x 0 .
It’s selection of distances to define neighbors described in Appendix Section A.2 is suitable for point clouds that have the same level of density as x 0 , but can not handle point clouds close to a Gaussian noise.
Indeed, we find the average number of neighbors for each point in the four levels of the Set Abstraction modules are 22 . 3984 , 29 . 9133 , 29 . 3266 , 27 .
8375 , respectively, for 10 random shapes sampled from the
MVP dataset.
In constrast, the average number of neighbors for each point in the four levels are 1 . 0864 , 1 , 1 , 1 for 10 random point clouds sampled from the Gaussian distribution.
Note that each point itself is considered to be a neighbor of itself.
This means most points do not have any neighbors besides itself in a Gaussian noise.
PointNet++ only utilizes the relative positions of input points.
The input feature of each point to the first Set Abstraction module is its relative position to the center point, which is subsampled from the original input points by farthest point sampling.
No information can be extracted when points do not have neighbors.
This is the reason why PointNet++ can not be directly used to train a DDPM.
Our solution is to attach the absolute position of each point to its feature.
This guarantees that a point at least can utilize its own position to decide which direction to move when it does not have neighbors.
Afterall, a point cloud with large magnitude noises does not have many meaningful structures.
There is not much information in the relative positions of points.
Another solution is to change the definition of neighbors: From points within a specified distance to K-nearest neighbors.
This guarantees that a point always has K neighbors.
We conduct experiments to compare these two solutions, and we find that their performances are basically the same.
Therefore, we just stick to the original neighbor definitions in PointNet++.
A. A TTENTION M ECHANISM
In Section 3.2 in the main text, we mentioned that we use the attention mechanism instead of maxpooling to aggregate features at the neighboring points to the center point.
We take the Set Abstraction module as an example to elaborate on the attention mechanism.
Attention mechanism in the
Feature Propagation module and Feature Transfer module is similarly designed.
Assume we want to propagate features from { x j |
≤ j ≤ N l } to { y k | 1 ≤ k ≤ N l +1 } .
Each point in { x j | 1 ≤ j ≤ N l } has a feature of dimension d l .
We concatenate these features with their corresponding 3D coordinates and group them together to form a matrix F l of shape N l × ( d l + 3) .
We finds K neighbors in the input set { x j | 1 ≤ j ≤ N l } for each y k .
These neighbors together with their features are grouped together to form a matrix G in of shape N l +1 × K × ( d l + 3) .
Then a shared multi-layer perceptron (MLP) is applied to transform the grouped feature G in to G out , which is a matrix of shape N l
+1 × K ×
d l +1 and d l +1 is the dimension of the output feature.
In our attention mechanism, G in (shape N l +1 × K × ( d l + 3) ) will act like keys, G out (shape
N l +1 × K ×
d l +1 ) will act like values, while the original features at { y k | 1 ≤ k ≤ N l +1 } will act like queries.
Since { y k | 1 ≤ k ≤ N l +1 } is a subset of { x j | 1 ≤ j ≤ N l } , we can group the original features at { y k | 1 ≤ k ≤ N l +1 } to form a matrix Q , which is of shape N l +1 × ( d l + 3) .
Q is first repeated K times into a matrix of shape N l +1 × K × ( d l + 3) .
Then this matrix is passed through a shared MLP and transformed into a matrix Q ′ , which is of shape N l +1 × K × d query .
Next, we pass G in through a shared MLP to transform it into a new matrix G ′ in , which is of shape
N l +1 × K × d key .
We concatenate the query matrix Q ′ with the key matrix G ′ in along the feature dimension.
We denote this matrix as [ Q ′ , G ′ in ] , which is of shape N l +1 × K × ( d query + d key ) .
[ Q ′ , G ′ in ] is passed through a shared MLP to obtain the scores of all the K neighbors.
We denote the scores as matrix S of shape N l +1 × K ×
d
l
+1 .
Note that S has the same shape as G out .
And the scores S of the K neighbors are adaptively computed according to the feature at the center point y k and features at its K neighbors { x j | j ∈ B x
( y k ) } .
We apply a softmax operation along the neighbor dimension (the second dimension) of S to obtain the weight matrix of all the K neighbors.
We denote it as W , which is of shape N l +1 × K ×
d
l
+1 .
Note that we manually set the weights of the padded dummy neighbors to 0 in W .
Then the weight matrix W and the value matrix G out are dot producted along the neighbor dimension (the second dimension) to form the output matrix
F ′ , which is of shape N l +1 ×
d l +1 .
Finally, F ′ is concatenated with the 3D coordinates of the set { y k | 1 ≤ k ≤ N l +1 } to form output of the Set Abstraction module, F l +1 , which is a matrix of shape
N l +1 × ( d l +1 + 3) .
A. P ROBLEMS WITH T HREE I NTERPOLATION
The original PointNet++ uses three interpolation to upsample features in the Feature Propagation module.
We think that the three interpolation operation is suitable for tasks like point cloud part segmentation, but not suitable in our task.
Interpolation means that points close to each other have similar features.
Points close to each other tend to have similar semantic labels in a clean point cloud, therefore it is meaningful to use interpolation operation to upsample features in the part segmentation task.
However, in our task, the network need to predict a per-point displacement for all points in a noisy point cloud and move it towards a clean point cloud.
Points close to each other do not need to move in a similar direction in general.
In fact, they may just need to move towards the opposite direction to form a smooth surface.
We also find that the three interpolation operation lacks the ability to manipulate positions of points accurately in small scales.
We first elaborate on the three interpolation operation used in the original
PointNet++.
Assume we want to upsample features at { y k | 1 ≤ k ≤ N l +1 } to { x j | 1 ≤ j ≤ N l } , where { y k | 1 ≤ k ≤ N l +1 } is a subset of { x j | 1 ≤ j ≤ N l } .
For each x j , assume its three nearest neighbors in { y k | 1 ≤ k ≤ N l +1 } are { y k | k ∈ B y, 3 ( x j ) } .
Then feature at x j is obtained through the following equation:
[Equation]
, where w ( y k , x j )
=
||
y k − x j || 2 , (7) f ( y k ) and f ( x j ) are features at y k and x j , respectively.
We can see that the value of f ( x j ) is determined by the relative distances between itself and its three nearest neighbors.
However, in 3D space, the point that has a specific relative distances to three fixed points is not unique.
In fact, the point can move freely on a curve.
Let’s take a very simple example, assume the three nearest neighbors of x j forms a regular triangle.
If we move x j along the straight line that passes the center of the triangle and is perpendicular to the plane determined by the triangle, then x j will always have the same relative distances from the three points, which means x j will always have the same interpolated feature, as long as its movement is small enough that its three nearest neighbors do not change.
This property makes the three interpolation operation not able to distinguish some specific points in a small scale, since these points could have the same interpolated value.
Therefore, three interpolation operation is not suitable for our task, as we need to accurately manipulate positions of points to make them form a meaningful shape with smooth surfaces and sharp details.
B E XPERIMENT B. D ETAILED E XPERIMENTAL S ETUP
In all experiments, we use the Adam optimizer with a learning rate of 2 × 10 − 4 .
For experiments of our PDR paradigm in Table 1 and Table 2 in the main text, we use data augmentation described in
Appendix Section B.3.
We train our Conditional Generation Network for 340 epochs, 200 epochs, and 500 epochs on the MVP, MVP-40, and Completion3D datasets, respectively.
We save a checkpoint and evaluate the network’s performance on both the training set and the test set everyepochs.
Since the generation process of DDPM is very slow, we randomly select 1600 samples from the training set and test set respectively for evaluation.
(Test set of the Completion3D dataset has less than 1600 samples.
Therefore, we use all samples in the test set for evaluation.)
The checkpoint with the lowest CD loss is chosen as the best network.
It is used to generate training data for the
Refinement Network.
We train the Refinement Network for 100 epochs, 150 epochs and 200 epochs on the MVP, MVP-40, and Completion3D datasets, respectively.
Note that we subsampling the test set only when we try to choose a best checkpoint in the training process of the conditional generation network in DDPM.
After choosing the best checkpoint, we use it to generate training data to train the refinement network.
However, when we evaluate the whole PDR paradigm (composed of the conditional generation network and refinement network) and compare with previous methods, we evaluate them on the complete test set.
Therefore, the comparison result in Table 1 and Table 2 in the main text is reliable and fair.
For ablation studies in Table 3, the Conditional Generation Networks are trained without data augmentation for 300 epochs.
All the Refinement Networks are trained on the same data generated by our proposed Conditional Generation Network trained with data augmentation.
The Refinement
Networks are trained for 100 epochs.
All baseline methods are rerun under the data augmentation described in Appendix Section B. according to their open source codes.
And CD loss is chosen to train all the baseline methods.
B. D ETAILS OF THE N ETWORK S TRUCTURE .
Detailed network structure is shown in Figure 6.
We present the number of points and feature dimension in each level of the Feature Extraction network and the Denoise network.
The distances to define neighbors are provided in Appendix Section A.2.
B. D ATA A UGMENTATION
We use rotation, mirror, translation, and scaling as data augmentation methods during training.
Rotation is performed along the upward direction of the shapes.
The upward direction is the y -axis in
MVP dataset and Completion3D dataset, while upward direction in the MVP-40 dataset is the zaxis.
And the rotation angle is uniformly sampled from the interval [ − a, a ] , where a is a predefined hyper-parameter that controls the magnitude of the rotation.
Mirror operation is performed with respect to the two planes that are parallel to the upward direction: x = 0 plane and z = 0 plane for MVP dataset and Completion3D dataset, x = 0 plane and y =plane for MVP-40 dataset.
The mirror operation is performed with a probability of m/ 2 with respect to the two planes, respectively.
m is a predefined hyper-parameter the controls the probability of the mirror operation.
Same as some previous works (Wang et al., 2020; Xia et al., 2021), we observe that most objects in the MVP dataset and Completion3D dataset have reflection symmetry with respect to the xy plane.
Therefore, we mirror the partial input with respect to this plane and concatenate the mirrored points with the original partial input for these two datasets.
We subsample this concatenated point cloud from 4096 points to 3072 points by farthest point sampling to obtain a uniform point cloud.
We label the original points with 1 and the mirrored points with − 1 .
This concatenated point cloud is feed to both the Conditional Generation Network and the Refinement Network, so that they could learn whether an object has reflection symmetry and determine whether to utilize the mirrored points according to their − 1 label.
Translation is achieved by adding a randomly sampled 3D vector to every point in the incomplete point cloud and the complete point cloud.
Each component of the 3D translation vector is sampled from a Gaussian distribution with zero mean and standard deviation of σ , where σ is a predefined hyper-parameter the controls the magnitude of the translation operation.
We also randomly samples a scaling factor uniformly from the interval [ δ low , δ high ] when loading a training pair.
The scaling factor is multiplied to the coordinates of all points in the incomplete point cloud and the complete point cloud.
We observe that data augmentations can prevent the network from overfitting on the training set, but could also lead to performance drop on the test set.
Therefore, we use different data augmentation schemes to train the Conditional Generation Network in the DDPM and the Refinement Network.
When training the Conditional Generation Network, we hope the network does not overfit on the training set, because we need it to generate training samples to train the Refinement Network.
Therefore, we use data augmentations of large magnitudes to train the Conditional Generation Network.
However, when training the Refinement Network, high performance is the top priority.
Therefore, we use data augmentations of small magnitudes to train the Refinement Network.
We also use data augmentation to train baseline methods.
The data augmentations are the same ones that we use to train the Refinement Network.
The details of the data augmentation is shown in Table
[Table] [Table]
B. S CALE -I NCONSISTENCY I SSUE OF THE C OMPLETION 3D D ATASET
We find that many pairs of incomplete-complete point clouds have inconsistent scales in the Completion3D dataset.
A few inconsistent examples are shown in Figure 7.
Ideally, the incomplete point should overlap with the complete point cloud in 3D space, but many incomplete-complete pairs in the Completion3D dataset cannot overlap with each other due to inconsistent scales, which misleads the network training.
Moreover, the scale-inconsistency issue also gives rise to unreliable evaluation results, as we expect the network to predict a complete point cloud of a consistent scale with the incomplete point cloud.
Therefore, it is necessary to correct the scales of these pairs before using the dataset.
We leverage the one-side CD loss to identify and correct these pairs.
For a consistent pair of the incomplete point cloud C and complete point cloud X , the one-side CD loss should be very low:
[Equation]
We find the correct scale of the incomplete point cloud by optimizing the following problem
[Equation]
This optimization problem is solved by using the python package scipy.optimize.fmin for every pair of point clouds in the Completion3D dataset.
We consider the pairs with a scale factor δ greater than 1 . 05 or less than 0 .
95 as inconsistent pairs, and then correct its scale inconsistency by multiplying the scale factor δ to these incomplete point clouds.
In the training set, we find 2 .
81% pairs are inconsistent.
The inconsistent pairs are also discovered in the validation set.
We can not verify the test set because the ground truth complete point cloud is not released.
We did not use the online Completion3D benchmark server to evaluate our method and previous methods for the following reasons: 1) the website server was out of service, as it gave no feedback for any submissions at the time we conduct this work; 2) the ground truth complete point clouds in test set of the Completion3D dataset are not released, and hence we can not verify whether this inconsistency problem is also present in the test set.
Therefore, we use the test set provided in the work (Wang et al., 2020), which contains 1200 pairs of incomplete-complete point clouds for testing.
It contains the same set of objects as the test set of the original Completion3D dataset.
After correcting inconsistent pairs in this test set, we evaluate our method and previous methods on this revised test set to achieve fair and reliable comparisons.
B. C OMPLETE E XPERIMENT R ESULTS F OR MVP-40 D ATASET
In Table 1 in the main text, we only present the completion result at the missing ratio of 50% and 12.5% for the MVP-40 dataset.
We present the complete experiment result on MVP-40 dataset including result at the 25% missing ratio in Table 5 below.
[Table]
B. C OMPLETE E XPERIMENT R ESULTS F OR N ETWORK A BLATION S TUDY
[Table]
In Section 5.3 in the main text, we conduct ablation study of our proposed network architecture, and results are shown in Table 3.
Note that the conditional generation networks in DDPM in Table 3 are trained without data augmentation.
However, it is actually very important to train the conditional generation networks with data augmentation, because we need to prevent it from overfitting on the training set, so that they can generate coarse point clouds of consistent distribution on the training set and test set to train the refinement network.
Therefore, we provide the training results with data augmentation in Table 6.
The data augmentation is specified in Table 4 for MVP dataset.
Same as Table 3 in the main text, “PA-Deonv & Att.” is our proposed complete network shown in
Figure 2.
“PA-Deonv” is our network without attention mechanism.
“PointNet++” further removes the PA-Deconv module.
“Concate x t & c ” removes FT modules.
It concatenates c with x t as Zhou et al.
(2021) do, and feed them to a single PointNet++ with attention mechanism and PA-Deconv.
“Pointwise-net” only utilizes a global feature extracted from the incomplete point cloud.
We can see that these proposed modules indeed improve the networks’ performance.
Our proposed networks achieve superior results both with and without data augmentation.
We also observe that networks generally achieve better performance on both the training set and test set without data augmentation, but they tend to overfit on the training set.
This is undesirable because we need these conditional generation networks to generate training data for the refinement networks.
It is very important for them to generate coarse point clouds that have consistent distributions on the training set and the test set.
Indeed, we can see that the overfitting problem is largely mitigated in the presence of data augmentation.
B. C OMPLETE E XPERIMENT R ESULTS F OR DDPM A CCELERATION
The complete experiment results of the DDPM acceleration is shown in Table 7.
We can see that the quality of coarse point clouds generated by the accelerated DDPMs has dropped considerably.
However, with the help of the Refinement Network, the performance drop of the final refined point clouds is slight.
This demonstrates the strong refinement capability of our proposed network architecture shown in Figure 2 in the main text.
[Table]
B. G ENERATION D IVERSITY OF THE PDR P ARADIGM
In this section, we discuss whether the PDR Paradigm can generate diverse completion results for the same incomplete point cloud.
Although there is no stochasticity in the refinement network, we find our PDR paradigm still demonstrates some kind of diversity in the completion results.
We know that DDPM itself is a probabilistic model and can generate diverse completion results.
The refinement network receives a coarse completion from the DDPM and then refines it according to the condition point cloud, i.e. , the incomplete point cloud.
The final refined result surely depends on the condition point cloud, but also depends on the coarse point cloud received from the DDPM.
The refinement network can only refine the coarse point cloud in a small scale, because we multiply the output of the refinement network by a small constant γ = 0 .
001 as described in Section 3.3 in the main text.
Therefore, the overall sketch of the coarse point cloud will be preserved after the refinement.
This explains why the PDR paradigm still bears low EMD loss as the DDPM, even though we use CD loss to train the refinement network, because the refinement network does not change the overall distribution of the coarse shape generated by DDPM.
Back to the diversity issue, if the coarse completion results from DDPM demonstrate diversity for the same incomplete point cloud, the refined point clouds will also demonstrate some diversity because the inputs to the refinement network are different.
Figure 8 shows some examples where the PDR paradigm demonstrate diversity in the completion results.
[Figure]
[Figure]
[Figure] [Figure]