Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
Authors: Tong Wu
Liang Pan
Junzhe Zhang
Tai WANG
Ziwei Liu
Dahua Lin
Venue: NeurIPS 2021 Poster (2021)
Keywords: Point Cloud Completion
Similarity Metrics

Abstract: Chamfer Distance (CD) and Earth Mover’s Distance (EMD) are two broadly adopted metrics for measuring the similarity between two point sets. However, CD is usually insensitive to mismatched local density, and EMD is usually dominated by global distribution while overlooks the fidelity of detailed structures. Besides, their unbounded value range induces a heavy influence from the outliers. These defects prevent them from providing a consistent evaluation. To tackle these problems, we propose a new similarity measure named Density-aware Chamfer Distance (DCD). It is derived from CD and benefits from several desirable properties: 1) it can detect disparity of density distributions and is thus a more intensive measure of similarity compared to CD; 2) it is stricter with detailed structures and significantly more computationally efficient than EMD; 3) the bounded value range encourages a more stable and reasonable evaluation over the whole test set. We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other. We can also use DCD as the training loss, which outperforms the same model trained with CD loss on all three metrics. In addition, we propose a novel point discriminator module that estimates the priority for another guided down-sampling step, and it achieves noticeable improvements under DCD together with competitive results for both CD and EMD. We hope our work could pave the way for a more comprehensive and practical point cloud similarity evaluation. Our code will be available at https://github.com/wutong16/Density_aware_Chamfer_Distance.

Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
Tong Wu', Liang Pan", Junzhe Zhang? Tai Wang Ziwei Liu?, Dahua Lin Joint Lab, The Chinese University of Hong Kong, LS-Lab, Nanyang Technological University, Shanghai AI Laboratory. "SenseTime Research, *Centre of Perceptual and Interactive Intelligence
Lin 1,3,5
{ut020, wt019, dhlin] cuhk. edu. hk, junzhe001 &e. att..ed..ee {liang. pan, ziwei liuuentu centuu.dd..... edu.ag
Abstract
Chamfer Distance (CD) and Earth Mover's Distance (EMD) are two broadly adopted metrics for measuring the similarity between two point sets. However, CD is usually insensitive to mismatched local density, and EMD is usually dominated by global distribution while overlooks the fidelity of detailed structures. Besides, their unbounded value range induces a heavy influence from the outliers. These defects prevent them from providing consistent evaluation. To tackle these problems, we propose a new similarity measure named Density-aware Chamfer Distance (DCD). It is derived from CD and benefits from several desirable properties: 1) it can detect disparity of density distributions and is thus a more intensive measure of similarity compared CD; 2) it stricter with detailed structures and significantly more computationally efficient than EMD; 3) the bounded value range encourages more stable and reasonable evaluation over the whole test set. We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other. We can also use DCD as the training loss, which outperforms the same model trained with CD loss on all three metrics. In addition, we propose a novel point discriminator module that estimates the priority for another guided down- sampling step, and it achieves noticeable improvements under DCD together with competitive results for both CD and EMD. We hope our work could pave the way for more comprehensive and practical point cloud similarity evaluation. Our code will be available at https: / thub. com http://ttonno/////oo 6/ Chamfer er Distance.
1 Introduction
Point cloud as one of the fundamental 3D representations is attracting increasing attention due to its efficiency, flexibility, and direct connection real-worid objects through 3D scanning devices. has been employed in a wide range of application scenarios and studied for various tasks [1, 4, 7, 11, 25, 33, 35, 27, 21, 36, 38]. A proper similarity measure between two point clouds is always crucial aspect for both guiding the training process and providing fair and reasonable evaluation. However, this is challenging design considering the unordered and irregular data form and varying point numbers.
of the most universally acknowl- edged metrics in various point cloud tasks. CD is a nearest- neaaret-neighhbuu-bbee based method and benefits from its efficient computation and flexible applicability for point sets with different point numbers. EMD
Chamfer Distance (CD) and Earth Mover's Distance (EMD)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Higher imbalance (mismotched density with GT)
Chamfer Distonce
Housdorff Distance
0.83
642
0.01
om
Larger
mine
0085
Higher noise intensity
2035
Hipher Imbolonse
Earth Mover's Distance
Density-awore Chamfer Distance
0.09
0.09
0.27
0.55
0.0%
0.00
255
212
1114
1536
2346
256
512
3334 3530
2043
68 8008S
REGL
will
50 #0005 WEEL W8.2 WEB4
Examples with noise and mismatched density
Metries: stotistics of the datoset
To generate the examples the left, a complete yet noisy shape (blue) with points is combined with a partial yet clean shape (pink) with 2048 points and then down-sammll back to 2048 points via FPS. Four point set distances are calculated between the generated shape and the ground truth. The results are averaged over the whole dataset to get the 3D histograms on the right: CD and HD are not sensitive to the mismatched density while highly influenced by noise when the intensity achieves certain level; EMD and DCD share similar pattern as n changes, but DCD more sensitive to the noise which also represents detailed structures.
Figure 1:
relies on solving an optimization problem find the least expensive one-to-one transportation flow between two point sets. Although sometimes considered to be more faithful to visual quality than CD [1, 13], it is significantly more computationally expensive. The desirable properties of a good similarity metric can be different across application scenarios. 1) For perception and registration, should be robust to sampling strategies and noisy points on detecting the similarity between the continuous surfaces represented by the discrete points. 2) For modelling and generation, it should be stricter with the quality of local point distributions, which is also crucial for visual quality. We mainly focus on the second aspect in this paper.
Take closer look at the two metrics above. The formulation of CD sometimes suffers from its intrinsic deficiencies: being insensitive to different density distributions while significantly influenced by outliers [22]. In particular, we specialize one concept closely related visual quality, namely matched density distribution between two point sets. This can also be denoted as 'balance", assuming the ground truth is uniformly distributed. We visualize the issue in Fig. I through the task of point cloud completion: 1) under a low noise level, CD hardly changes with the imbalance ratio: 2) it increases dramatically once the noise intensity rises to some extent: 3) under a high noise level, higher imbalance even results in a lower CD, which leads to tricky operation to reduce CD for the task: increasing the point density in the seen area with assured accuracy while reducing in the unseen area to lower down the risk from abnormal points. But severe imbalance also significantly affects the global appearance. On the contrary, EMD can steadily detect the change of distribution in Fig. 1. However, the requirement for one-to-one mapping is usually over harsh. Consequently, the optima of the transportation problem is dominated by the global distribution while ignoring the fine-structured local details [5], to be described in Sec. 3. Therefore, both CD and EMD are not ideally suitable for evaluating the quality of the generated shapes.
In this work, we mainly focus on CD's injustice as an evaluation metric and propose a new similarity measure named Density-aware Chamfer Distance (DCD) to tackle the challenges above. Specifically, DCD is derived from the original CD, while it benefits from higher sensitivity to distribution quality through fraction term of query frequency and higher tolerance to outliers through an approximation of Taylor Expansion. It shares a similar trend with EMD under a varying point distribution while being more computationally efficient and better at capturing the details. Moreover, due to their different focus, CD and EMD often encounter divergence when evaluating different methods, which makes them less reliable as consistent metrics, as to be demonstrated in Sec. 5. We empirically observe that DCD usually provides a more consistent and reliable evaluation, especially when the

---

Table 1: Comparison of properties among different metrics.
Metrics
assignment
efficient
bounded
density-aware
detail-aware
CD
nearest neighbour
X
EMD
optimization
X
X
DCD
nearest neighbour
results of CD and EMD contradict each other. Note that the proposed metric is also beneficial at dealing with ground truth with a non-uniform distribution such as curvature-hasee sampling, as to be discussed in the supplementary material.
Furthermore, we analyze the property of DCD as a loss function and compare it with LI and L2 versions of CD. We introduce minor adjustments to it that are critical for better training process. We then propose to make better use of the information from the query frequency mentioned above and design
MLP-based point discriminator inspired by the recent success of implicit functions [18, 14, 19]. is further integrated to our balanced design for a two-stage completion framework, where the output of the module can be viewed as the "importance" of each point and serve as priority for a following step of down- sown-sampling Finally, the guided lown-sampling operation benefits from removing outliers and preserving the critical points.
Extensive investigations are provided for the comparison among different metrics and methods for the task of point cloud completion. Experimental results validate that the proposed metric. Density-aware Chamfer Distance, successfully overcomes the aforementioned issues of CD. DCD can provide a more reliable evaluation when CD and EMD contradict each other, and it is proved to be more faithful to visual quality in Sec, 5 and a user study in the supplementary material. We validate its capacity loss function PCN [35] and VRCNet [17], showing that it not only helps reduce DCD itself but also significantly lowers down the EMD metric and surprisingly reduces CD as well compared with the network trained with CD. Our proposed balanced design also gains noticeable improvement under the new metric, competitive results for both CD and EMD, and superior visual quality in the experiments.
2 Related Works
Point Cloud Completion. Point cloud completion aims to recover a complete shape based
partial observation. Earlier works represent shapes with voxels [3, 6, 20), while PCN [35] first proposes use raw point data and leverages encoder-decoder structure generate global- feature-based coarse shape followed by the folding-based up-sampling [34]. Following works enhance the feature representation by techniques like attention mechanisms [28, 17, 16], hierarchical aggregation [8, 37], and grid structure for cubic feature sampling [32], etc.; the decoding process can also be examined to leverage, for example, a tree-structure [23]. iterative refinement [26, 31], multiple patches generation [13], or separated prediction for the seen and unseen [37]. etc. These works use either CD or EMD for evaluation, yet the two may be consistently satisfied due their different concerns, as to be shown in Sec. 5, and thus a more comprehensive metric is essential for a fair and reliable comparison.
Point Cloud Distance. The term "distance" refers to a non-negative function that measures the dissimilarity between two point sets. Considering the unordered strueture of point clouds, the shape-level distance usually comes from statistics of pair-wise point-level distances based on certain assignment strategy. Chamfer Distance (CD) is one of the most widely used metrics based on the nearest neighbour (Eqn. 1). There are variants of it used for training [34, 4] and some other similarly formed distances like Hausdorff [9, 2, 29]. Another generally adopted metric is Earth Mover's Distance (EMD), which relies on solving an optimization problem to find the best mapping function from one
the other. much more computationally expensive. Recently, Urbach et al [24] propose DPDist, which compares point clouds by measuring the distance between the surfaces that they were sampled on. However, it is estimated by a network rather than mathematical formulation, making it inconvenient and potentially unstable to be adopted into various tasks. Another closely related work is by Nguyen et al [15], who propose the sliced Wasserstein distance that has equivalent properties as EMD and similar computational complexity to CD, with the Monte Carlo method involved for approximation. In comparison, start from the opposite point of view by deriving new formulation based CD
sometimes considered to be more rational than CD [1. 13], but is
3

---

and result in a clean and explicit expression. Similarly, our DCD also shares the properties of EMD many cases (Fig. 1). while detects the detail preserving issue better than EMD.
3
Density-aware Chamfer Distance for Point Sets
3.1
Preliminaries
Chamfer Distance between two point sets Si and S2 is defined as:
dcn(Si, |S
Σ RE
min 9E82
1
ull2
min #ES1
|yy 1/|2-
122
Fach point E S1 finds its nearest neighbour in S2 and vice versa; all the point-level pair-wise distances are averaged to produce the shape-level distance. The simple and flexible formulation generalizes well across many tasks. Earth Mover's Distance is defined as:
dEMD(SI, S2)
min
-1=-5(z)1
(6:S1 22 JESI
relies on solving an optimization problem that finds a one-to-one bijection mapping á S1 + S2, thus only applicable when [S|] = -S2]. The pair-wise distances are then calculated between x and Φ(z). As computing the optimal mapping is computationally expensive and even hardly affordable, several approximation schemes [13, 12] have been developed relieve the computation burden.
3.2 Density-aware Chamfer Distance
Formulation and Interpretation. As discussed in Sec. 1, CD is not a comprehensive metric for evaluating visual quality for the generation tasks, e.g.. point cloud completion. We explain this from its formulation: 1) the square operation makes it intensively influenced by outliers, and the evaluation results have huge varying range across the dataset; 2) the nearest point query operation makes less sensitive to the issue of mismatched density distribution, and hence resulting in a less discriminative evaluation of visual quality. Therefore, we aim to propose the original formulation that not only preserves the capacity of similarity measure but also highly alleviates the problems above, namely Density-aware Chamfer Distance (DCD).
metric based on
Firstly, CD grows quadratically with point pair distances, which can be dominated by the worst cases and hence overlooking the others. To address this problem, we introduce the first order approximation of Taylor Expansion ex = Σn=o 2" /n!, i.e. ea 1 + 2 where = -||r y/12. Thanks to the nearest neighbour assignment, the condition that 0 is usually satisfied and the approximation is reasonable. Thus we have:
dcD(S1, ~
min (1 e (l- 2 yE82
min (1 - -1s-==l1,) xES
SS
S2
(3)
FESt
yESa
Considering the property of the exponential function (= < 0), each point-level distance is mapped to value between [0, 1]. As result, the formulation also sets a natural boundary of [o, for the overall shape distance. The approximation would be less accurate as the point-level distance gets away from zero, yet exactly helps to mitigate the over-sensitivity of the outliers by suppressing the square growth. We add another scale factor as in Eqn. 4 to adjust the sensitivity. The absolute distance value depends on this factor, and we reveal relative consistency across different choices of a in the supplementary material, and we fix a 1000 for evaluation in this paper.
Secondly, the ambiguity of CD is partially due its "blindness" that each point only considers its nearest neighbour in the other set while ignoring the surroundings. We denote the nearest neighbour assignment process by "query" for simplicity and present a simple example here: assume S₁ to be a uniform point cloud and S, an in-homogeneous one; consider two points y1, E S2 with yı located at a sparse area and y2 in relatively dense area. The calculation of (SD, S2) would likely get y1 frequently queried by points in S₁ due to the local sparsity, and the case for can be exactly the opposite. Thus we denote that y1 and y/2 are not equally critical in representing the shape. Furthermore, assume a subset S C S1 so that each point in 8 queries y and that ny |S points in this set are unaware of each other under the formulation of CD, and the contribution of to each point in Sy is not affected as ny gets larger, which is unreasonable.

---

xESI/yE5:
Assignment
(a) CD
(b) DCD
(c) EMD
Comparison of assignment strategies and distance calculation. CD and DCD take the nearest neighbour locally, and DCD further considers the point-specific query frequency; EMD forces one-to-one mapping. and the assigned pair points may locate far from each other with weaker physical meaning.
Figure 2:
Alternatively, denote that the overall contribution of each point to the evaluation system shall be normalized, and we introduce 1/ny to deal with the case where is being shared by multiple æs:
(pc =
1
- f
- -
+
Σ yE
1
-e- - n1
|
S2
zES1
(4)
where ý minges, || 21 2 = minges, |lv 2112, and denotes temperature scalar. Finally, consider the first term without loss of generality, each y contributes ny Σ e-11x-alla E [o. 1]
the overall distance metric (before averaging). A variants for DCD to deal with point sets with different number of points will be discussed in the supplementary material.
Comparison among the Distance Metries. We conduct brief comparison of the properties among the three metrics Table I, sketching their assignment strategies and computation schemes in Fig. As discussed above, both CD and DCD assign point pairs by querying the nearest neighbours, while DCD further considers the point-specific query frequency Ny and incorporates the property density distribution into the measurement. In comparison, EMD naturally forces equal query frequency of for each point via the mapping function, and the metric is highly sensitive to the global point distribution. However, the harsh constraint not only imposes significant increase in computational cost but also tends to sacrifice its attention to visual quality for optimal mapping. As shown in Fig. 2, the assigned pair of points could be located far from each other, and the distance less physically meaningful. Experimental results in Sec. 5 would also show that an over compacted and smoothed shape can be favored by EMD despite its loss of detailed structure.
In brief, DCD takes a step from CD and attempts to provide a rationale bridge towards EMD for a better sense of point distribution rather than being blinded by its nearest neighbour. Compared with EMD, not only more efficient but also stricter with local structures. and good preservation of detailed structures both important factors for the visual quality of the completion result. More examples are to be shown in Sec. 5 for better illustration.
balanced distribution
3.3 Application as an Objective Function
Gradient Analysis and Comparisons. Besides its usage as an evaluation metric, DCD is also expected serve an objective function to guide the training process. Considering the same nearest neighbour assignment, our analysis below will mainly focus on comparing CD (denoted by CD-T) and its L1 version (denoted by CD-P), which also widely adopted for training. The exponential formulation in DCD takes in the L2 distance and modifies the gradient curve while bounding the loss value between [0,1] at the same time.
We visualize the loss value and gradient curves of the three in Fig. (a and b). The gradient by CD-T grows linearly with the distance 1 of single point pair: it becomes rather small when / is close to zero and performs a heavy punishment on points with a large l. The CD-P loss, on the contrary, produces constant gradient. For the DCD loss, it would first and then approach and its gradient can be calculated as: ddpep(1)/8l
increases, 2ale at -ale /n, where dpcn(!) denotes contribution of one point pair with a distance of 1, and ny. The gradient first rises and then falls to zero, indicating that the loss is only effective to point pairs whose distance lies within a certain range. Similar to CD-T, it becomes small when reduces zero, which is more reasonable than the
5

---

gradient
gradient
gradient
PD 10
abhw-30c
DCD
abhw-502
4ghh 2000
34
0.00
0.00
0.06
0.12
0.00
0.01
0.08
0.13
0.00
0.05
0.30
0.35
0.00
0.05
0.30
0.35
b.
d.
Figure 3: presents the loss curves of CD-T, CD-P, and DCD and b presents the gradient curves of them; e and d visualizes the gradient for DCD with different and a, respectively.
constant gradient by CD-P, while it is also bounded by maximum and will not be so harsh to the points with an extremely large and thus stabilizes the training process.
Adjustment on DCD for Training. There are two characteristic components that decide the property of the curve, namely the hyper-parameter a and the query frequency n. As shown Fig. 3(c), larger promotes higher peak and smaller range for non-zero gradient, and we need to set a proper a during training based on the practical distance distribution. We empirically find that setting E [40, 100] best promotes the training. A larger n indicates more serious mismatching of density in a local region and leads to a higher loss value. When viewing both n and as variables, they are encouraged to be smaller to lower down the loss, but is not differentiable and cannot be directly changed by backpropagation. Meanwhile, a higher n reduces the gradient to / linearly (Fig. 3()),, which prevents one point from moving towards over-dense region, while it may also block the training process when the gradient to gets too small. As a result, we introduce another hyper-parameter À [0, 1] and replace n, and ng in Eqn. 4 with na and n to get the practical loss function for training.
Incorporating Balanced Design in Point Cloud Completion
4.1 Point Cloud Completion Framework
We adopt typical two-stage coarse-to-fine completion pipeline [35, 13, 26, 17] (Fig. 4(d)). In the first stage, we extract global feature from the partial observation and generate a complete yet coarse point cloud; the second stage. we then introduce local features with abundant geometry information and obtain the final output with more details, higher visual quality, and high fidelity to the input. More details of the network architecture and training loss are included the supplementary material.
We would like to highlight two observations regarding the point distributions here. First, a mean shape usually exists for each category, and there is obvious imbalance of density for different regions according to how commonly they are shared across the dataset (Fig. 4(a)); second, the evaluation results by CD does not perfectly align with human assessment: the trick that place more points in the seen region with high confidence usually boosts the CD performance while it introduces imbalanced distribution and hurts the benign global distribution the same time.
To address the problems above, we propose a simple yet effective method based on a current SoTA approach [17] that significantly boosts both the qualitative and quantitative results. Specifically, we tried our new metric as the objective function to replace LCD: we then propose a novel point discriminator which is supervised by carefully designed density-aware signal and estimates the importance of each point; the output from the point discriminator used as a priority for a final guided down-sampling process that helps remove outliers and maintain balanced distribution.
4.2 Point Discriminator
The generated points in a shape are usually not equally important. We can roughly group them into three categories with the help of query frequency n introduced in Sec. 3: 1) when n >> 0, the points with a higher n often lie in a sparse area and play a critical role in representing the shape: 2) when = 0, the point can lie on the ground truth surface while in an over-populated region and become relatively unimportant; 3) another case with n underlying surface and can be regarded as outlier that hurts the overall appearance. There
that the point locates far from the

---

Meon shope illustration
b.
Target formulatio
Point discriminator
Mean
Edge- Convolution
MLP
high
Variance
low
Frequently
queried
High probobility
Low probobiltt Loss
Torget
Prediction
d. Network architecture
Powrze
P
Down- sampler
Output
Coarse
Coarse
Poonrad
Fine Decoder
Encoder
Decoder
Fine Encoder
Po
Upsampler
Prrc
Figure a. visualization mean shape over the datasel; b. visualization of g(x) in Eqn. at instance level; c. module structure of the point discriminator; d. two-stage framework with guided down-sampling.
no hard boundaries among these cases above, but we carefully design a formulation g(x) in Eqn. to map them into proper value range for learning. Specifically, suppose we train the module with the ground truth Pgt and the coarse output P Ponarse when n O, we adopt a logarithmic function for n and introduce the expected frequency ProareProarsell as the denominator; we add bias of 1 to ensure g()) = 0 at the boundary and finally invert the sign to distinguish from the following cases. When
O, denote that the distance from point to Pat together with scaling factor t can serve as a suitable indicator to distinguish between case 2) and 3). To this end, we formulate a conjoint objective g(x) below:
min ||x - 8112= - yEP
0,
g(x)
[Pcoorse]
(5)
log2
n2 + 1)
fh > 0.
[Pat
Encouraged by the recent success of implicit functions [18, 14, 19], we train an MLP module along with the main network that learns to predict the target function g(æ), namely point discriminator. Specifically, it takes the local features of a point and its neighbours for edge convolution; the results are concatenated with the local feature ft. global feature /9, and point coordinates cz. denoted by
2 = f, f3, fu Cz), where x' E NN and N₂ denotes the neighbours for point æ. It outputs single scalar which is supervised by the target function (xx) with regression loss:
Lp
Σ
- 1z)g(x)ll2.
(6)
Pooarse
ZE
The discriminator would be used for deciding point sampling privilege at inference time.
Guided Down-sampling. In view of the imbalanced population issue discussed above, an intuitive idea to alleviate it would be to allow larger number of points generated via up-sampling before reducing the desired number via Furthest Point Sampling (FPS), However, FPS tends to select more points from outer regions, which increases the risk of including more outliers, and thus this operation usually results in a noisier output. As a result, we aim to take advantage the point diseriminator above for the down-sampling stage at inference time together with FPS. Specifically, for each point Prnarse: define point-wise existing probability by:
p(z) = a0-3- - -)7)== = - e(
Let s denote the scale for up- sapsampling, then x: is corresponding to points the Pitarse (up-sampled from P Pcoarse) and they all share the same p(x) E 1). an independent probability of each point being sampled. p+ Piterse is first down-sampled in this manner and then combined with Prec (up- sampled from Po) before the final FPS, which ensures that we remain pre-defined number of points. This process leverages the prior learned during training to effectively reduce the unreasonably located points at inference time (Fig. 7), relieving the side effect brought by FPS operation.
7

---

Table 2: Point cloud completion results in terms of CD x10*, EMD x x10², and DCD, lower is better.
aipplane
Methods
Metrics
Avg...
4.50
8.33
13.11
12.86
20.00
14.63
1.73
6.17
5.84
PCN
7.99
5.75
6.90
1199
5.32
6.60
5400
9.84
4.85
7.87
10.56
4.03
4.86
6.80
DCD
0.178
0.519
0.450
0.517
0.710
0.552
0.559
0.580
0.662
0.562
0.608
0:129
0.416
0.548
0.491
0.415
0.553
CD)
4.06
9.08
6.64
15.01
19.25
9.78
14.36
9.66
22.33
9:73
15.51
5.13
1.86
6.25
5.81
4.99
10.29
PCN++
BMD
3,44
3.75
3.85
4.65
9.00
3.56
4.69
4,22
6,13
3.85
4.39
2.62
2.75
5.60
3.771
3.07
4.27
DCD
0.478
0.454
0,451
0.574
0.661
0,504
0.517
0,540
0.617
0.534
0.563
0.389
0.360
0.527
0.447
0,393
0.508
CD
4.12
9.54
7.44
15.26
18.64
10:77
12.95
19.99
9.21
16.06
5.47
2.36
7.06
7.04
4.68
10.30
TopNet
6,30
4,00
7.01
7.0
8,09
6.27
6,80
DCD
0.536
0.558
0.548
0.650
0.508
0.599
0.600
0.678
0.558
0.622
0.492
0.457
0.572
0.542
0.496
0.598
CD
2.73
3.92
6.50
10.75
13:37
9.26
10.17
7,70
17:27
6.54
12.10
5.21
1.37
4.59
4.62
3.38
7.99
MSN
EMD
2.75
4.02
3.47
1.44
6.28
3.74
4.16
3.82
5.27
3.34
4.25
2.92
2,07
3.30
3.62
2.31
3.94
DCD
0,404
0,509
0,516
0,537
0,539
0,532
0
0.515
0.514
0.471
0.541
0.458
0.358
0.491
0.465
0,422
0.499
CD
2.20
7,92
5.60
7,49
8.15
7,45
7.52
5.20
11.90
4,88
7,39
4.53
1,15
3.90
3.44
3.22
6.09
VRC
4.65
6.58
5.25
4.45
4.56
5:27
0,509
0.699
0.488
0.515
0:438
0.527
0.401
0.470
0.319
0.452
2.72
9,0
6.58
9.99
11.55
9.38
9.80
17:23
10:34
7.87
VRC-EMD
EMD
2.50
3,65
3:23
4.35
5.31
3,61
3.93
3.58
5.17
3.19
3.97
2.69
2.05
3.06
3.48
2.39
3.62
DCD
0.369
0.483
0,473
0,502
0,499
0,509
0.450
0.478
0.547
0,423
0:487
0:424
0.349
0.445
0,430
0,370
0.461
CD
2.22
8,00
5.41
7,58
8.28
7.94
8.39
5,46
14.76
5.78
9.57
4.44
1,30
3.59
3.43
2.39
6.51
Our
BMD
2.29
4.43
3,46
3.92
3.98
3,89
5.34
3.13
3.91
3.02
3.38
2.39
3.67
0.447
0.427
0.451
0,445
0,423
0.426
0.504
0.453
0.401
0.365
5 Experiments
Dataset. We use the recently proposed MVP Dataset [17] for our study and experiments. It is a multi-view partial point cloud dataset covering 16 categories with 62,400 and 41,600 pairs for training and testing, respectively. It renders the partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model selected from ShapeNet [30], and the ground truth point cloud is sampled via Poisson Disk Sampling (PDS).
Comparison Methods and Metries. We include the following methods for comparison in the main experiments: PCN [35], TopNet [23], MSN [13], and VRCNet [17] (with the PSK module discarded to improve efficiency while slightly scarifying performance). PCN++ is simple extension of PCN [35] that generates the double number of points for training and down-sampled to the required point number inference time. We report per-class results CD, EMD, and DCD for clear comparison across methods and a view of the different properties of these metrics.
Implementation Details. All the models are trained using the Adam optimizer [10] with the learning rate initialized at l and decayed by 0.7 every 40 epochs. We use batch size of 32 and a total epoch of 80. We set a 1000 for the evaluation of DCD, and a E [40, 100) for training. We set E [0, 0.5] and 3 = 9, 7 = 1 for our approach in the main experiments. Our work is implemented with PyTorch and is run Tesla V100 GPU.
5.1 Comparison of the Methods
The main experimental results for point cloud completion on MVP dataset are reported in Table 2. Note that most networks trained with the CD loss except when specified. Early works, PCN [35] and TopNet [23] have relatively high loss for all the evaluated metrics, CD, EMD, and DCD. PCN with an additional up-sampling step, namely PCN++, notably lowers down EMD and DCD, while marginally raising CD for sacrifice; MSN [13] benefits from the multi-surface design and obtains low EMD and DCD results, while its CD performance is not that satisfying. The previous SoTA method, VRCNet [17] outperforms the other methods under the metric of CD and DCD, but we observe that its EMD loss is surprisingly high. We further replace the CD loss with EMD when training VRCNet (denoted as VRC-EMD). and it achieves the lowest EMD yet obviously higher CD than the original version. In comparison, our method reports the lowest DCD, second-lowest CD, and comparable EMD with VRC-EMD. Qualitative results (Fig. 5) show that our results are apparently superior in both the global balanced point distributions and local structures. user study in the supplementary material will further validate that our method benefits from higher visual quality.
8

---

DCD
3175
Case
PONH
VRC-EMD
3125
05
Case EMD
CD DCD
0.07
Case
3060
Case
3,05
VRC
OUR
CD
Case EMD 1 CD / DCD 1
CD &
+
0.00
0.000
0.0m
0.007
0.201
0.004
5.005
0.006
Partial Input
PCN++
VRC-EMD
VRC
Our
Ground Truth
EMD | CD DCD
6.23 | 4.69 0.52
5.31 3.29 0.50
5.30 8.10 0.55 0.55
5.07 3.89 0.45
11.95 4.10 0.60
7,83 4.40
5.24 5.03 0.44
5:45 3.85 0.41
2.61 2.58 0.42
2.24 0. 0.42
1.89 3.36 0.43
1.47 2:12 0033
examples from different methods that represent three typical cases: 1) EMD is low while CD is high; 2) CD is low while EMD is high; 3) both CD and EMD are low. DCD only lowered down in the third case. Upper left: close shot at the cases above. Upper right: visualization of the three metrics correlation between CD and EMD is weak and the scatter points form fan-shaped area, and 2) DCD (denoted by the point color) becomes lower towards the original point where both CD and EMD are low.
Figure 5: Comparison of CD, EMD, and DCD through examples and statistics. Below:
test set, we can observe that 1) the positive
5.2
Comparison of the Metrics
Consistency and Reliability. When taking closer look at all the CD-EMD-DCD tuples (either for each category for the averaged results) in Table the results by different metries do not exhibit clear positive correlation. But we also observe that for those that have a similar value of CD, DCD is usually dominated by their EMD performance, and for those with similar EMD results, DCD is highly correlated with CD. Although the law is not strictly held or theoretically proved, the scatter plot by each instance from different methods in Fig. 5 (upper right) also supports empirical observation. reveals that since CD and EMD focus on different aspects the point cloud, inconsistency and confusion may exist for the similarity measurement, which prevents either of them from being comprehensive metric. In comparison, DCD reflects the behaviors of them both and could only be reduced when both CD and EMD are relatively low that it serves as a more consistent, stable, and comprehensive metric, especially when CD and EMD encounter disparity. Our user study in the supplementary material further indicates that DCD more faithful metric visual quality.
Bounded Distance. One advantage of DCD is its bounded value range, which promotes an equal consideration for all samples in the dataset at the evaluation stage. In contrast, CD and EMD dominated by the worst cases the dataset due to their unbounded nature, as shown in Fig. Specifically, for CD, 80% of the normalized loss accumulation is contributed by only top 50% of the samples ranked by the shape-level distance, and 50% contributed by the top 25%; a similar case is also observed for EMD. This property makes them fail to comprehensively evaluate the quality of all samples in the dataset. Our DCD overcomes this problem by assuring clear [0.1] boundary for the distance value, enabling more comprehensive and stable statistical results the whole test set.
5.3 Ablation Study
We study the effectiveness of each component our method separately, including DCD loss the additional point up-sampling (PU), and the guided point down-sampling (GPD). As shown in Table 3,

---

Chamfer Distance
Earth Mover's Distance
Density-aware aware Chamfer Distance
35.00
0.8
Disribution Rccumulation
Distribtion
Dutribution
Accomulztion
Accumalation
0.5
0.3
0.5
0.3
000
10400
Ramiked instance id
20:00
Ranked instance
Ranleed Instancee (d
Figure 6: Distribution and accumulation (normalized to 1) evaluation results per-shape in the test set.
Table 3: Ablation Study.
Table 4: Evaluation results on three metrics when trained with each of them. denotes applying LEMD on the final outputs while LCD on intermediate ones.
Metric
DCD
PU
GPD
CD
EMD
DCD
Model
PCN [35]
VRC [17]
6.09
5.27
0.462
5.85
5.14
0.457
Metric
LCD
LEAD
LOCD
LCD
LEMD
LDCD
6.91
3.78
0.425
CD
9.78
10.70
9.36
7.87
6.65
3.68
0.422
EMD
6.80
3.97
4.71
5.27
3.62
5.14
6.51
3.67
0.420
DCD
0,553
0.537
0.526
0.462
0.461
0.457
leveraging LDCD loss usually outperforms the same model trained with LeD in all the three metrics; the additional point up-sampling (PU) significantly reduces EMD while increases CD at the same time; after guided down-sampling (GPD) is applied to the model with both Lpon and PU, achieve the lowest EMD and DCD and relatively low CD.
Ground Truth
Ground Truth
7.36
Ground truth
Ground
Ground Truth
11.93
3.60
Ground Truth
Figure 7: Examples of guided down sampling (from left to right) and evaluation in terms of CD x1
5.4 Performance as the Loss Function
We evaluate the effectiveness of DCD loss function by training networks with each of the three metrics (denoted by LCD. LEMD. and LDCD) and evaluate on all of them (denoted by CD, EMD, and DCD). We conduct experiments the baseline model PCN [35] and the SoTA VRCNet [17], as shown in Table 4. Compared with -tantnee networks, training with LEAD D produces the lowest EMD and lower dpen. yet it suffers from non-negligible scarification of CD and much heavier computational burden, which is to be discussed the supplementary material; training with LDCD will produce the lowest DCD, significantly reduce D. and the Lep-trained models (especially on PCN). The time consumption is also comparable with LCD. which further validates DCD's ss convenience and superiority an objective function. We a 50 or 100 here with 3 = 0 and replace all the LOD with LpCD. Experiments on how and 3 affect the performance are included in the supplementary material.
slightly reduce CD than
Conclusion
In this work, we propose a new similarity measure point clouds named Density-aware Chamfer Distance (DCD). It is bounded in value, effective in computation, and faithful to visual quality by considering both the density distribution and detailed structures. Our method achieves noticeable improvements under DCD and superior visual quality compared with previous works.
10

---

Acknowledgements. This study is supported in part by the NTU NAP, and under the RIE2020 Industry Alignment Fund Industry Collaboration Projects (IAF-ICP) Funding Initiative. as well as cash and in-kind contribution from the industry partner(s). It is also supported in part by Centre for Perceptual and Interactive Intelligence Limited, in part by the GRF through the Research Grants Council of Hong Kong under Grants (Nos. 14208417, 14207319 and 142035 (1551) 88) and ITTS/43 in part by CUHK Strategic Fund and CUHK Agreement 12193, in part by the Shanghai Committee of Science and Technology, China (Grant No. 20DZI 100800).
References
[1] Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning repre- sentations and generative models for 3d point clouds. In Proceedings of the 35th International Conference on Machine Learning, volume 80, pages 40-49. PMLR, 10-15 Jul 2018.
[2] Xuelin Chen, Baoquan Chen, and Niloy J Mitra. Unpaired point cloud completion on real scans using adversarial training. The International Conference on Learning Representations, 2019. 3 [3] Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner. Shape completion using 3d-encoder- predictor cnns and shape synthesis. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 5868-5877, 2017. 3
[4] Haowen Deng, Tolga Birdal, and Slobodan llic. 3d local features for direct pairwise registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3244-3253, 2019. 1,3
[5] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEÉE conference on computer vision
and pattern recognition, pages 605-613, 2017. 2
[6] Xiaoguang Han, Zhen Li, Haibin Huang, Evangelos Kalogerakis, and Yizhou Yu. High- resolution shape completion using deep neural networks for global structure and local geometry inference. In Proceedings of the IEEE International Conference on Computer Vision, pages 85-93, 2017. 3
[7] Pedro Hermosilla, Tobias Ritschel, and Timo Ropinski. Total denoising: Unsupervised learning of 3d point cloud cleaning. In Proceedings of the IEEE/CVF International Conference on Compuler Vision. pages 52-60, 2019.
[8] Zitian Huang, Yikuan Yu, Jiawen Xu, Feng Ni, and Xinyi Le. Pf-net: Point fractal network for 3d point cloud completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7662-7670, 2020. 3
[9] Daniel P Huttenlocher, Gregory A. Klanderman, and William Rucklidge. Comparing images using the hausdorff distance. IEFE Transactions on pultern analysis and machine intelligence,
15(9):850- 16350-863,111 1993. 3
[10] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6988 2014. 8
[11] Ruihui Lii, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Pu-gan: a point cloud upsampling adversarial network. In Proceedings of the IEEE International Conference on Computer Vision, pages 7203-7212, 2019.
[12] Tianyi Lin, Nhat Ho, and Michael Jordan. On efficient optimal transport: An analysis of greedy and accelerated mirror descent algorithms. In International Conference on Machine Learning,
pages 3982-3991. PMLR, 2019. 4
[13] Minghua Liu, Lu Sheng, Sheng Yang, Jing Shao, and Shi-Min Hu. Morphing and sampling network for dense point cloud completion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11596-1 11596-11603, 2020. 2, 3, 4, 6, 8
[14] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4460-4470, 2019. 3.7
[15) Trung Nguyen, Quang-Hieu Pham, Tam Le, Tung Pham, Nhat Ho, and Binh-Son Hua. Point-set distances for learning representations of 3d point clouds. arXiv preprint urXiv:2102.04011
2021. 3
[16] Yinyu Nie, Yiqun Lin, Xiaoguang Han, Shihui Guo, Jian Chang, Shuguang Cui, and Jian Jun Zhang. Skeleton-bridged point completion: From global inference to local adjustment. Advances
in Neural Information Processing Systems, 2020.
[17] Liang Pan, Xinyi Chen, Zhongang Cai, Junzhe Zhang, Haiyu Zhao, Shuai Yi, and Ziwei Liu. Variational relational point completion network. Proceedings of the IEEE/CVF Conference

---

Computer Vision and Pattern Recognition, 2021. 3, 6. 8, 10, 13
[18] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 165-174,
2019. 3, 7
[19] Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Con- volutional occupancy networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 523-540, Springer, 2020. 3,
[20] Edward J Smith and David Meger. Improved adversarial systems for 3d object generation and reconstruction. In Conference Robot Learning, pages 87-96, 2017.
[21] Yongbin Sun, Yue Wang. Ziwei Liu, Joshua Siegel, and Sanjay Sarma. Pointgrow: Autore- gressively learned point cloud generation with self-attention. In Proceedings of the IEEE/CVF
Winter Conference on Applications of Computer Vision, pages 61-70, 2020.
[22] Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. What do single-view 3d reconstruction networks learn? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages
2019. 2
[23] Lyne P Tehhapmi Vineet Kosaraju, Hamid Rezatofighi, Ian Reid, and Silvio Savarese. Topnet: Structural point cloud decoder. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, pages 383-392, 2019. 3, 8
[24] Dahlia Urbach, Yizhak Ben-Shabat, and Michael Lindenbaum. Dpdist: Comparing point clouds using deep point cloud distance. The European Conference on Computer Vision (ECCV), 2020.
3
[25] Tai Wang, Xinge Zhu, and Dahua Lin. Reconfigurable voxels: A new representation for lidar-based point clouds. In Conference on Robot Learning, 2020.
[26] Xiaogang Wang, Marcelo H Ang Jr, and Gim Hee Lee. Cascaded refinement network for point cloud completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 790-799, 2020. 6.6
[27] Yue Wang. Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning on point clouds. Acm Transactions On Graphics 388)),3(55):1112 2019.
[28] Xin Wen, Tianyang Li, Zhizhong Han, and Yu-Shen Liu. Point cloud completion by skip- attention network with hierarchical folding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1939,1948, 2020. 3
[29] Rundi Wu, Xuelin Chen, Yixin Zhuang, and Baoquan Chen. Multimodal shape completion via conditional generative adversarial networks. The European Conference on Computer Vision
(ECCV), August 2020. 3
[30] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 1912- 1920, 2015. 8 [31] Chulin Xie, Chuxin Wang, Bo Zhang, Hao Yang. Dong Chen, and Fang Wen. Style- based point generator with adversarial rendering for point cloud completion. arXiv preprint arXiv:2103.0255 02000.0253..00 2021.
[32] Haozhe Xie, Hongxun Yao, Shangchen Zhou, Jiageng Mao, Shengping Zhang, and Wenxiu Sun. Grnet: Gridding residual network for dense point cloud completion. arXiv preprint
vrXiv:2006.0376 2020. 3
[33] Guandao Yang. Xun Huang, Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan.
Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the
IEEE/CVF International Conference on Computer Vision, pages 4541-4550, 2019. 1
[34] Yaoqing Yang, Chen Feng. Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 206-215, 2018.
[35] Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. Pen: Point completion network. In International Conference on 3D Vision (3DV), pages 728-737. IEEE,
2018. 1,3, 8, 10
[36] Junzhe Zhang. Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, and Chen Change Loy. Unsupervised 3d shape completion through gan inversion. In CVPR, 2021. 1
[37] Wenxiao Zhang, Qingan Yan, and Chunxia Xiao. Detail preserved point cloud completion via separated feature aggregation. arXiv preprint arXiv:2007.00 023744, 2020. 3
12

---

[38] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma. Wei Li, Hongsheng Li, and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. In Proceedings of the European Conference Computer Vision, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See supplementary material.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material as a URL)? [No] We will release the code upon the paper accepted.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Sec. 5.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Sec. 5.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] MVP dataset [17].
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're
using/curating? [No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "yBla4mptCBH", "reviewer": "Reviewer_gwGK", "summary": "This paper starts with a discussion on the currently used point cloud similarity metrics (Chamfer distance and earth mover’s distance) discussing their drawbacks and proposes a novel similarity measure. It is based on the commonly used Chamfer distance and proposes two principal modifications to it:\n1) the distance is scaled exponentially to be robust to outliers and is bound to be in [0, 1];\n2) it is weighted by the number of neighboring matched points to account for the local density of the ground truth.\nNew metric is used for evaluation in the point cloud completion task for which the authors also propose a model based on prior work, but additionally apply outlier filtering by a trainable component and furthest point sampling to additional refine the output.\n\nExperiments show that the proposed modifications achieve state-of-the-art performance with respect to the proposed metric. The qualitative evaluations argue, that the proposed metric bridges the gap between the ability of CD to capture finer details and the ability of EMD to better account for the density distribution of the output.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n* The paper is well written and properly justifies the need for improved point cloud similarity measures. The proposed measure is introduced gradually with clear argumentation why each change is required.\n* Evaluations on the point completion task bring insightful information about the proposed measure.\n* The proposed trainable probabilistic filtering technique produce visually improved results.\n\nCons:\n* The main drawback is discussed in the supplementary and is the inability to use the proposed measure as a loss function because it is not differentiable. In that sense, it is not clear how to optimize existing models for this measure.\n* There are some minor corrections and clarifications required:\n1) $g(x)$ in equation 5 is a very specific function, which only takes some discrete values depending on the number of neighbors. Could you elaborate why it is used exactly this form of this function and why do you need to regress its values, rather than just classify, how many neighbors the point have?\n2) How exactly the probability in equation 7 is used? Do you sample according to it or just take a threshold?\n3) It is not stated clearly in the text to what exactly the abbreviations Sample and Filter in Table 3 are referring to. Filter is probabilistic trainable filtering and Sample is FPS?\n4) L238 number of epochs\n5) L272 more efficient\n\nOverall, I tend to acceptance. However, I would like to see my questions addressed in the rebuttal.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors provided a discussion on the importance of reliable similarity measures.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "XCgC_8N8mn6", "reviewer": "Reviewer_p1L4", "summary": "Post rebuttal comment: in my original review I have copied an incorrect paper summary from my notes (from another paper from my review batch). All remaining sections of this review were copied correctly. My deepest apologies about this, I have only noticed this mistake after the rebuttal period. Below I am copying the correct summary: \n\n---\nThis paper argues both commonly used evaluation measures for point cloud completion, Chamfer Distance (CD) and Earth Movers Distance (EMD) both suffer from certain limitations. Firstly, both are unbounded (certainly not a desirable property of evaluation measure) and sensitive to outliers. Furthermore, CD is insensitive to point cloud density and thus it doesn’t align well with human perception, while EMD tends to be insensitive to details, requires inputs of equal size, and is computationally expensive. \nTo alleviate this, the paper argues for an extension of CD, that transforms the Euclidean distance to a similarity measure, which is bounded and robust to outliers. Second, by normalizing the contribution of each point, the proposed variant of CD becomes sensitive to the local point cloud density, which is argued to be a desired property for the evaluation point cloud completion quality. \nOverall, the paper presents good arguments for the proposed metric being a good substitute for standard CD. The second part of the paper presents an extension to a two-stage point cloud completion network (PCN) that tackles the point imbalance issue wrt. the point discriminator. The experiments show that the proposed method obtains betters results in terms of BCD compared to baselines. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n\n**Pros**\n\n* This is certainly an interesting paper, pointing out significant drawbacks of existing evaluation measures for assessing point cloud completion. \n* The theoretical discussion of the drawbacks of existing metrics and the proposed extension of CD make perfect sense to me and looks like a nice contribution! \n* Looking at the metric comparison discussion, the proposed metric is computationally effective (which presents an advantage over EMD) and bounded (in contrast to both). It is difficult to interpret results when evaluation measures are unbounded. \n\n\n**Cons**\n\n\n* Given the focus of this paper (on the novel distance metric) and the claims on the “metric being superior in terms of visual quality”, I would have expected s very thorough analysis on this part, discussing the difference between CD, EMD, the proposed method as well as the method by [Nguyen et al, Arxiv:2102.04014], that is (similarly to the proposed method) computationally efficient and has similar properties as EMD. This is only mentioned in the related work. Such as study should also contain a visual human assessment; without it, I don’t see how the claims on this measure being “superior in terms of visual quality” can be justified.\n* At some point, it becomes unclear what the focus of this paper is: on a new evaluation measure for point cloud completion, or addressing an inherent imbalance problem of existing point cloud completion methods (Sec. 4). Related to that, It is difficult to draw good conclusions from Sec. 5.1; the proposed method obtains the best results overall in terms of the proposed BCD metrics, while results in terms of CD and EMD are mixed. On one hand, this is indeed what the criticism of existing metrics is (neither of them is comprehensive and weakly correlated), but this experiment somewhat mixes-up evaluation of new metrics and method, that is based on the same observations. Again, I find it very difficult to draw conclusions when the focus is not clear (new metrics vs. methods). However I can observe this: when looking at the class average score, neither CD or EMD reaches a consensus with the proposed BCD. \n* Why is the metric proposed by [Nguyen et al, Arxiv:2102.04014] excluded from the metric comparison and discussion (5.2)?\n* This doesn't impact my final rating, but still want to point out: this paper would certainly benefit from investing more time in improving the grammar, sentence structure, and presentation flow. At the bare minimum, I would recommend using grammar recommendation tools. For example, please do not start sentences with “And it would be helpful …” (L196). I would also suggest minimizing the usage of passive voice. \n\n\n**Justification**\n\nI do not think this paper is ready for publication at the moment. In principle, the proposed metric seems sound and makes perfect sense. However, I would have expected that the paper will have a strong focus on justifying the proposed evaluation measure; instead, the evaluation mixes up a new variant of point cloud completion network and the metric, making it difficult to draw conclusions. \nIn terms of experimental justification of the metric, I would have expected a human quality assessment (to support the claims on being superior when it comes to visual quality evaluation), a discussion and comparison to [Nguyen et al, Arxiv:2102.04014], and analysis across different datasets for point cloud completion (e.g., SemanticKITTI [Behley et al., ICCV’19]) to demonstrate that the proposed metric is versatile (i.e., conclusions are consistent across different scenarios, objects vs. scenes, and different sensors). \n\nThis being said, I am not an expert in this field and I will certainly keep an open mind. \n\n**Post rebuttal** \nThe authors provided a very good rebuttal. Let me quickly summarize:\n\n*Human assessment study* Authors point out that this study was conducted and reported in the supplementary. \nIn the future, please make that clear in the paper. Supplementary material is not meant to be read \"standalone.\" I refer to it when I want to see additional results or studies that didn't make it to the main paper (as most reviewers do). Apologies in case this study was referenced, and I've missed it. \n\n*Confusion in sec. 5.1.* I will trust the authors to revise this section.\n\n*Comparison to Nguyen et al.* Authors point out that this is only a pre-print, therefore I cannot really ask for this comparison. It would still be potentially interesting for the camera-ready.\n\nOverall, my main doubts were addressed well in the rebuttal, and I am ready to raise my ratings. As reviewer gwGK points out, this paper tackles a relevant problem with existing point cloud similarity measures, and I appreciate the core contribution.\n\nI would still like to clarify that after reading Sec. 3, I was expecting a thorough evaluation of the merit of the proposed similarity measure across different datasets and tasks. I think this paper would be stronger if it had a laser focus on the metric.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors did not discuss the limitations. I don't see any potential negative social impact of this work. \n\n**Post rebuttal** The authors did discuss limitations in the supplementary, my bad. ", "review_text": "", "extra_scores": {"ethical_concerns": "I see no ethical concerns here. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "4EUzLrg9WHY", "reviewer": "Reviewer_9BXU", "summary": "This paper compares Chamfer Distance (CD) and Earth Mover’s Distance (EMD) and then points out their limitations separately. To address the limitations, the authors propose a Balanced Chamfer Distance (BCD) which considers disparity of density distributions, locally detailed structures and has a bounded value stabilizing the training process. These issues are not considered in either CD or EMD. Furthermore, the paper proposes a point discriminator and up/down sampling to enhance the performance of PCN. The experimental results on point cloud completion demonstrate that BCD can overcome shortcomings in CD and EMD.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is well-motivated as it pinpoints the existing issues with respect to CD and EMD. Figure 1 supports the authors' claim. The Taylor Expansion based modification and the density based “query” estimation (at least) is novel to me. And I believe BCD would have a broader application in 3D vision.  \n\nA key problem is how to decide n_y and n_x, ie, the nearest neighbor assignment. Does it depend on an R^3 sphere whose radii is a hyper-parameter? If so, an ablative study is needed to evaluate the effect of n_y and n_x. Another problem is whether the choice of alpha in Eq.(4) will affect the effectiveness of BCD? Ie, is it possible for some values of alpha that BCD performs better than CD and EMD, but for some other values that BCD performs worse? If the performance of BCD heavily depends on the choice of alpha, it will hurt the merits of BCD. From Eq.(3) and Eq. (4), it is not obvious that BCD considers the global density distribution of point clouds as EMD. It still emphasizes the local structure. Despite the above issues, BCD is still an interesting idea for measuring similarity between point clouds.\n\nThe main concern is the point discriminator and up/down sampling presented in Sec. 4.2 are heavily coupled with PCN. as PCN adopts a two-stage coarse-to-fine completion process, which limits its application to other methods. One problem w.r.t this part is whether using point sampling privilege will cause significant computational overhead during inference. \n\nThe experiments are mainly conducted on point cloud completion and use CD, EMD and BCD as evaluation metrics. Though BCD achieves lower values in most (not all) cases (Table 2), I would like to point out that such evaluation is not efficient. As the loss function for completion will always push each metric to a lower value. To strongly support that BCD is more powerful than CD and EMD in 3D vision tasks (in a general sense), experiments on other tasks (ie, rigid registration) are preferred. As CD is usually used as the loss function for rigid registration. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "No ethical issue.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers appreciated the contribution of the paper, and thought it is a timely introduction of a new measure of comparison of point clouds. There were some doubts whether the BCD measure should be evaluated on point cloud completion task in particular. The authors are encouraged to evaluate the measure on other tasks.
Decision: Accept (Poster)