Balanced Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
Authors: Tong Wu
Liang Pan
Junzhe Zhang
Tai WANG
Ziwei Liu
Dahua Lin
Venue: NeurIPS 2021 Poster (2021)
Keywords: Point Cloud Completion
Similarity Metrics

Abstract: Chamfer Distance (CD) and Earth Mover’s Distance (EMD) are two broadly adopted metrics for measuring the similarity between two point sets. However, CD is usually insensitive to mismatched local density, and EMD is usually dominated by global distribution while overlooks the fidelity of detailed structures. Besides, their unbounded value range induces a heavy influence from the outliers. These defects prevent them from providing a consistent evaluation. To tackle these problems, we propose a new similarity measure named Density-aware Chamfer Distance (DCD). It is derived from CD and benefits from several desirable properties: 1) it can detect disparity of density distributions and is thus a more intensive measure of similarity compared to CD; 2) it is stricter with detailed structures and significantly more computationally efficient than EMD; 3) the bounded value range encourages a more stable and reasonable evaluation over the whole test set. We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention to both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other. We can also use DCD as the training loss, which outperforms the same model trained with CD loss on all three metrics. In addition, we propose a novel point discriminator module that estimates the priority for another guided down-sampling step, and it achieves noticeable improvements under DCD together with competitive results for both CD and EMD. We hope our work could pave the way for a more comprehensive and practical point cloud similarity evaluation. Our code will be available at https://github.com/wutong16/Density_aware_Chamfer_Distance.

Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completion
Tong Wu', Liang Pan", Junzhe Zhang? Tai Wang Ziwei Liu?, Dahua Lin Joint Lab, The Chinese University of Hong Kong, LS-Lab, Nanyang Technological University, Shanghai AI Laboratory. "SenseTime Research, *Centre of Perceptual and Interactive Intelligence
Lin 1,3,5
{ut020, wt019, dhlin] cuhk. edu. hk, junzhe001 &e. att..ed..ee {liang. pan, ziwei liuuentu aentuu.ddu.... edu.ag
Abstract
Chamfer Distance (CD) and Earth Mover's Distance (EMD) are two broadly adopted metrics for measuring the similarity between two point sets. However, CD is usually insensitive to mismatched local density, and EMD is usually dominated by global distribution while overlooks the fidelity of detailed structures. Besides, their unbounded value range induces a heavy influence from the outliers. These defects prevent them from providing consistent evaluation. To tackle these problems, we propose a new similarity measure named Density-aware Chamfer Distance (DCD). It is derived from CD and benefits from several desirable properties: 1) it can detect disparity of density distributions and is thus a more intensive measure of similarity compared CD; 2) it stricter with detailed structures and significantly more computationally efficient than EMD; 3) the bounded value range encourages more stable and reasonable evaluation over the whole test set. We adopt DCD to evaluate the point cloud completion task, where experimental results show that DCD pays attention both the overall structure and local geometric details and provides a more reliable evaluation even when CD and EMD contradict each other. We can also use DCD as the training loss, which outperforms the same model trained with CD loss on all three metrics. In addition, we propose a novel point discriminator module that estimates the priority for another guided down- sampling step, and it achieves noticeable improvements under DCD together with competitive results for both CD and EMD. We hope our work could pave the way for more comprehensive and practical point cloud similarity evaluation. Our code will be available at https: / thub. com http://ttonno/////oo 6/ Chamfer er Distance.
1 Introduction
Point cloud as one of the fundamental 3D representations is attracting increasing attention due to its efficiency, flexibility, and direct connection real-worid objects through 3D scanning devices. has been employed in a wide range of application scenarios and studied for various tasks [1, 4, 7, 11, 25, 33, 35, 27, 21, 36, 38]. A proper similarity measure between two point clouds is always crucial aspect for both guiding the training process and providing fair and reasonable evaluation. However, this is challenging design considering the unordered and irregular data form and varying point numbers.
of the most universally acknowl- edged metrics in various point cloud tasks. CD is a nearest- neaares-neighhbuu-bbee based method and benefits from its efficient computation and flexible applicability for point sets with different point numbers. EMD
Chamfer Distance (CD) and Earth Mover's Distance (EMD)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Higher imbalance (mismotched density with GT)
Chamfer Distonce
Housdorff Distance
0.83
642
0.01
om
Larger
mine
0085
Higher noise intensity
2035
Hipher Imbolonce
Earth Mover's Distance
Density-awore Chamfer Distance
0.09
0.09
0.27
0.55
0.0%
0.00
255
212
1314
1536
2346
256
512
3334 3530
2043
68 8008S
REGL
will
50 #0005 WEEL W8.2 WEB4
Examples with noise and mismatched density
Metries: stotistics of the datoset
To generate the examples the left, a complete yet noisy shape (blue) with points is combined with a partial yet clean shape (pink) with 2048 points and then down-sammll back to 2048 points via FPS. Four point set distances are calculated between the generated shape and the ground truth. The results are averaged over the whole dataset to get the 3D histograms on the right: CD and HD are not sensitive to the mismatched density while highly influenced by noise when the intensity achieves certain level; EMD and DCD share similar pattern as n changes, but DCD more sensitive to the noise which also represents detailed structures.
Figure 1:
relies on solving an optimization problem find the least expensive one-to-one transportation flow between two point sets. Although sometimes considered to be more faithful to visual quality than CD [1, 13], it is significantly more computationally expensive. The desirable properties of a good similarity metric can be different across application scenarios. 1) For perception and registration, should be robust to sampling strategies and noisy points on detecting the similarity between the continuous surfaces represented by the discrete points. 2) For modelling and generation, it should be stricter with the quality of local point distributions, which is also crucial for visual quality. We mainly focus on the second aspect in this paper.
Take closer look at the two metrics above. The formulation of CD sometimes suffers from its intrinsic deficiencies: being insensitive to different density distributions while significantly influenced by outliers [22]. In particular, we specialize one concept closely related visual quality, namely matched density distribution between two point sets. This can also be denoted as 'balance", assuming the ground truth is uniformly distributed. We visualize the issue in Fig. I through the task of point cloud completion: 1) under a low noise level, CD hardly changes with the imbalance ratio: 2) it increases dramatically once the noise intensity rises to some extent: 3) under a high noise level, higher imbalance even results in a lower CD, which leads to tricky operation to reduce CD for the task: increasing the point density in the seen area with assured accuracy while reducing in the unseen area to lower down the risk from abnormal points. But severe imbalance also significantly affects the global appearance. On the contrary, EMD can steadily detect the change of distribution in Fig. 1. However, the requirement for one-to-one mapping is usually over harsh. Consequently, the optima of the transportation problem is dominated by the global distribution while ignoring the fine-structured local details [5], to be described in Sec. 3. Therefore, both CD and EMD are not ideally suitable for evaluating the quality of the generated shapes.
In this work, we mainly focus on CD's injustice as an evaluation metric and propose a new similarity measure named Density-aware Chamfer Distance (DCD) to tackle the challenges above. Specifically, DCD is derived from the original CD, while it benefits from higher sensitivity to distribution quality through fraction term of query frequency and higher tolerance to outliers through an approximation of Taylor Expansion. It shares a similar trend with EMD under a varying point distribution while being more computationally efficient and better at capturing the details. Moreover, due to their different focus, CD and EMD often encounter divergence when evaluating different methods, which makes them less reliable as consistent metrics, as to be demonstrated in Sec. 5. We empirically observe that DCD usually provides a more consistent and reliable evaluation, especially when the

---

Table 1: Comparison of properties among different metrics.
Metrics
assignment
efficient
bounded
density-aware
detail-aware
CD
nearest neighbour
X
EMD
optimization
X
X
DCD
nearest neighbour
results of CD and EMD contradict each other. Note that the proposed metric is also beneficial at dealing with ground truth with a non-uniform distribution such as curvature-hasee sampling, as to be discussed in the supplementary material.
Furthermore, we analyze the property of DCD as a loss function and compare it with LI and L2 versions of CD. We introduce minor adjustments to it that are critical for better training process. We then propose to make better use of the information from the query frequency mentioned above and design
MLP-based point discriminator inspired by the recent success of implicit functions [18, 14, 19]. is further integrated to our balanced design for a two-stage completion framework, where the output of the module can be viewed as the "importance" of each point and serve as priority for a following step of down- sown-sampling Finally, the guided lown-sampling operation benefits from removing outliers and preserving the critical points.
Extensive investigations are provided for the comparison among different metrics and methods for the task of point cloud completion. Experimental results validate that the proposed metric. Density-aware Chamfer Distance, successfully overcomes the aforementioned issues of CD. DCD can provide a more reliable evaluation when CD and EMD contradict each other, and it is proved to be more faithful to visual quality in Sec, 5 and a user study in the supplementary material. We validate its capacity loss function PCN [35] and VRCNet [17], showing that it not only helps reduce DCD itself but also significantly lowers down the EMD metric and surprisingly reduces CD as well compared with the network trained with CD. Our proposed balanced design also gains noticeable improvement under the new metric, competitive results for both CD and EMD, and superior visual quality in the experiments.
2 Related Works
Point Cloud Completion. Point cloud completion aims to recover a complete shape based
partial observation. Earlier works represent shapes with voxels [3, 6, 20), while PCN [35] first proposes use raw point data and leverages encoder-decoder structure generate global- feature-based coarse shape followed by the folding-based up-sampling [34]. Following works enhance the feature representation by techniques like attention mechanisms [28, 17, 16], hierarchical aggregation [8, 37], and grid structure for cubic feature sampling [32], etc.; the decoding process can also be examined to leverage, for example, a tree-structure [23]. iterative refinement [26, 31], multiple patches generation [13], or separated prediction for the seen and unseen [37]. etc. These works use either CD or EMD for evaluation, yet the two may be consistently satisfied due their different concerns, as to be shown in Sec. 5, and thus a more comprehensive metric is essential for a fair and reliable comparison.
Point Cloud Distance. The term "distance" refers to a non-negative function that measures the dissimilarity between two point sets. Considering the unordered strueture of point clouds, the shape-level distance usually comes from statistics of pair-wise point-level distances based on certain assignment strategy. Chamfer Distance (CD) is one of the most widely used metrics based on the nearest neighbour (Eqn. 1). There are variants of it used for training [34, 4] and some other similarly formed distances like Hausdorff [9, 2, 29]. Another generally adopted metric is Earth Mover's Distance (EMD), which relies on solving an optimization problem to find the best mapping function from one
the other. much more computationally expensive. Recently, Urbach et al [24] propose DPDist, which compares point clouds by measuring the distance between the surfaces that they were sampled on. However, it is estimated by a network rather than mathematical formulation, making it inconvenient and potentially unstable to be adopted into various tasks. Another closely related work is by Nguyen et al [15], who propose the sliced Wasserstein distance that has equivalent properties as EMD and similar computational complexity to CD, with the Monte Carlo method involved for approximation. In comparison, start from the opposite point of view by deriving new formulation based CD
sometimes considered to be more rational than CD [1. 13], but is
3

---

and result in a clean and explicit expression. Similarly, our DCD also shares the properties of EMD many cases (Fig. 1). while detects the detail preserving issue better than EMD.
3
Density-aware Chamfer Distance for Point Sets
3.1
Preliminaries
Chamfer Distance between two point sets Si and S2 is defined as:
dcn(Si, |S
Σ RE
min 9E82
1
ull2
min #ES1
|yy 1/|2-
122
Fach point E S1 finds its nearest neighbour in S2 and vice versa; all the point-level pair-wise distances are averaged to produce the shape-level distance. The simple and flexible formulation generalizes well across many tasks. Earth Mover's Distance is defined as:
dEMD(SI, S2)
min
E -1=10(<)11e
(6:S1 22
JESI
relies on solving an optimization problem that finds a one-to-one bijection mapping á S1 + S2, thus only applicable when [S|] = -S2]. The pair-wise distances are then calculated between x and Φ(z). As computing the optimal mapping is computationally expensive and even hardly affordable, several approximation schemes [13, 12] have been developed relieve the computation burden.
3.2 Density-aware Chamfer Distance
Formulation and Interpretation. As discussed in Sec. 1, CD is not a comprehensive metric for evaluating visual quality for the generation tasks, e.g.. point cloud completion. We explain this from its formulation: 1) the square operation makes it intensively influenced by outliers, and the evaluation results have huge varying range across the dataset; 2) the nearest point query operation makes less sensitive to the issue of mismatched density distribution, and hence resulting in a less discriminative evaluation of visual quality. Therefore, we aim to propose the original formulation that not only preserves the capacity of similarity measure but also highly alleviates the problems above, namely Density-aware Chamfer Distance (DCD).
metric based on
Firstly, CD grows quadratically with point pair distances, which can be dominated by the worst cases and hence overlooking the others. To address this problem, we introduce the first order approximation of Taylor Expansion ex = Σn=o 2" /n!, i.e. ea 1 + 2 where = -||r y/12. Thanks to the nearest neighbour assignment, the condition that 0 is usually satisfied and the approximation is reasonable. Thus we have:
dcD(S1, ~
min (1 e (l- 2 + yE82
min (1 - -1s-==l1,) xES
SS
S2
(3)
FESt
yESa
Considering the property of the exponential function (= < 0), each point-level distance is mapped to value between [0, 1]. As result, the formulation also sets a natural boundary of [o, for the overall shape distance. The approximation would be less accurate as the point-level distance gets away from zero, yet exactly helps to mitigate the over-sensitivity of the outliers by suppressing the square growth. We add another scale factor as in Eqn. 4 to adjust the sensitivity. The absolute distance value depends on this factor, and we reveal relative consistency across different choices of a in the supplementary material, and we fix a 1000 for evaluation in this paper.
Secondly, the ambiguity of CD is partially due its "blindness" that each point only considers its nearest neighbour in the other set while ignoring the surroundings. We denote the nearest neighbour assignment process by "query" for simplicity and present a simple example here: assume S₁ to be a uniform point cloud and S, an in-homogeneous one; consider two points y1, E S2 with yı located at a sparse area and y2 in a relatively dense area. The calculation of (SD, S2) would likely get y1 frequently queried by points in S₁ due to the local sparsity, and the case for can be exactly the opposite. Thus we denote that y1 and y/2 are not equally critical in representing the shape. Furthermore, assume a subset S C S1 so that each point in 8 queries y and that ny |S points in this set are unaware of each other under the formulation of CD, and the contribution of to each point in Sy is not affected as ny gets larger, which is unreasonable.

---

xESI/yE5:
Assignment
(a) CD
(b) DCD
(c) EMD
Comparison of assignment strategies and distance calculation. CD and DCD take the nearest neighbour locally, and DCD further considers the point-specific query frequency; EMD forces one-to-one mapping. and the assigned pair points may locate far from each other with weaker physical meaning.
Figure 2:
Alternatively, denote that the overall contribution of each point to the evaluation system shall be normalized, and we introduce 1/ny to deal with the case where is being shared by multiple æs:
(pc =
1
- f
- -
+
Σ yE
1
-e- - n1
|
S2
zES1
(4)
where ý minges, || 21 2 = minges, |lv 2112, and denotes temperature scalar. Finally, consider the first term without loss of generality, each y contributes ny Σ e-11=-alla E [0. 1]
the overall distance metric (before averaging). A variants for DCD to deal with point sets with different number of points will be discussed in the supplementary material.
Comparison among the Distance Metries. We conduct brief comparison of the properties among the three metrics Table I, sketching their assignment strategies and computation schemes in Fig. As discussed above, both CD and DCD assign point pairs by querying the nearest neighbours, while DCD further considers the point-specific query frequency Ny and incorporates the property density distribution into the measurement. In comparison, EMD naturally forces equal query frequency of for each point via the mapping function, and the metric is highly sensitive to the global point distribution. However, the harsh constraint not only imposes significant increase in computational cost but also tends to sacrifice its attention to visual quality for optimal mapping. As shown in Fig. 2, the assigned pair of points could be located far from each other, and the distance less physically meaningful. Experimental results in Sec. 5 would also show that an over compacted and smoothed shape can be favored by EMD despite its loss of detailed structure.
In brief, DCD takes a step from CD and attempts to provide a rationale bridge towards EMD for a better sense of point distribution rather than being blinded by its nearest neighbour. Compared with EMD, not only more efficient but also stricter with local structures. and good preservation of detailed structures both important factors for the visual quality of the completion result. More examples are to be shown in Sec. 5 for better illustration.
balanced distribution
3.3 Application as an Objective Function
Gradient Analysis and Comparisons. Besides its usage as an evaluation metric, DCD is also expected serve an objective function to guide the training process. Considering the same nearest neighbour assignment, our analysis below will mainly focus on comparing CD (denoted by CD-T) and its L1 version (denoted by CD-P), which also widely adopted for training. The exponential formulation in DCD takes in the L2 distance and modifies the gradient curve while bounding the loss value between [0,1] at the same time.
We visualize the loss value and gradient curves of the three in Fig. (a and b). The gradient by CD-T grows linearly with the distance 1 of single point pair: it becomes rather small when / is close to zero and performs a heavy punishment on points with a large l. The CD-P loss, on the contrary, produces constant gradient. For the DCD loss, it would first and then approach and its gradient can be calculated as: ddpep(1)/8l
increases, 2ale at -ale /n, where dpcn(!) denotes contribution of one point pair with a distance of 1, and ny. The gradient first rises and then falls to zero, indicating that the loss is only effective to point pairs whose distance lies within a certain range. Similar to CD-T, it becomes small when reduces zero, which is more reasonable than the
5

---

gradient
gradient
gradient
PD 10
abhw-30c
DCD
abFw-502
4ghh 2000
34
0.00
0.00
0.06
0.12
0.00
0.01
0.08
0.13
0.00
0.05
0.30
0.35
0.00
0.05
0.30
0.35
b.
d.
Figure 3: presents the loss curves of CD-T, CD-P, and DCD and b presents the gradient curves of them; e and d visualizes the gradient for DCD with different and a, respectively.
constant gradient by CD-P, while it is also bounded by maximum and will not be so harsh to the points with an extremely large and thus stabilizes the training process.
Adjustment on DCD for Training. There are two characteristic components that decide the property of the curve, namely the hyper-parameter a and the query frequency n. As shown Fig. 3(c), larger promotes higher peak and smaller range for non-zero gradient, and we need to set a proper a during training based on the practical distance distribution. We empirically find that setting E [40, 100] best promotes the training. A larger n indicates more serious mismatching of density in a local region and leads to a higher loss value. When viewing both n and as variables, they are encouraged to be smaller to lower down the loss, but is not differentiable and cannot be directly changed by backpropagation. Meanwhile, a higher n reduces the gradient to / linearly (Fig. 3()),, which prevents one point from moving towards over-dense region, while it may also block the training process when the gradient to gets too small. As a result, we introduce another hyper-parameter À [0, 1] and replace n, and ng in Eqn. 4 with na and n to get the practical loss function for training.
Incorporating Balanced Design in Point Cloud Completion
4.1 Point Cloud Completion Framework
We adopt typical two-stage coarse-to-fine completion pipeline [35, 13, 26, 17] (Fig. 4(d)). In the first stage, we extract global feature from the partial observation and generate a complete yet coarse point cloud; the second stage. we then introduce local features with abundant geometry information and obtain the final output with more details, higher visual quality, and high fidelity to the input. More details of the network architecture and training loss are included the supplementary material.
We would like to highlight two observations regarding the point distributions here. First, a mean shape usually exists for each category, and there is obvious imbalance of density for different regions according to how commonly they are shared across the dataset (Fig. 4(a)); second, the evaluation results by CD does not perfectly align with human assessment: the trick that place more points in the seen region with high confidence usually boosts the CD performance while it introduces imbalanced distribution and hurts the benign global distribution the same time.
To address the problems above, we propose a simple yet effective method based on a current SoTA approach [17] that significantly boosts both the qualitative and quantitative results. Specifically, we tried our new metric as the objective function to replace LCD: we then propose a novel point discriminator which is supervised by carefully designed density-aware signal and estimates the importance of each point; the output from the point discriminator used as a priority for a final guided down-sampling process that helps remove outliers and maintain balanced distribution.
4.2 Point Discriminator
The generated points in a shape are usually not equally important. We can roughly group them into three categories with the help of query frequency n introduced in Sec. 3: 1) when n >> 0, the points with a higher n often lie in a sparse area and play a critical role in representing the shape: 2) when = 0, the point can lie on the ground truth surface while in an over-populated region and become relatively unimportant; 3) another case with n underlying surface and can be regarded as outlier that hurts the overall appearance. There
that the point locates far from the

---

Meon shope illustration
b.
Target formulatio
Point discriminator
Mean
Edge- Convolution
MLP
high
Variance
low
Frequently
queried
High probobility
Low probobiltt Loss
Torget
Prediction
d. Network architecture
Powrze
P
Down- sampler
Output
Coarse
Coarse
Poonrad
Fine Decoder
Encoder
Decoder
Fine Encoder
Po
Upsampler
Prrc
Figure a. visualization mean shape over the datasel; b. visualization of g(x) in Eqn. at instance level; c. module structure of the point discriminator; d. two-stage framework with guided down-sampling.
no hard boundaries among these cases above, but we carefully design a formulation g(x) in Eqn. to map them into proper value range for learning. Specifically, suppose we train the module with the ground truth Pgt and the coarse output P Ponarse when n O, we adopt a logarithmic function for n and introduce the expected frequency ProareProarsell as the denominator; we add bias of 1 to ensure g()) = 0 at the boundary and finally invert the sign to distinguish from the following cases. When
O, denote that the distance from point to Pat together with scaling factor t can serve as a suitable indicator to distinguish between case 2) and 3). To this end, we formulate a conjoint objective g(x) below:
min ||x - 8112= - yEP
0,
g(x)
[Pcoorse]
(5)
log2
n2 + 1)
fh > 0.
[Pat
Encouraged by the recent success of implicit functions [18, 14, 19], we train an MLP module along with the main network that learns to predict the target function g(æ), namely point discriminator. Specifically, it takes the local features of a point and its neighbours for edge convolution; the results are concatenated with the local feature ft. global feature /9, and point coordinates cz. denoted by
2 = f, f3, fu Cz), where x' E NN and N₂ denotes the neighbours for point æ. It outputs single scalar which is supervised by the target function (xx) with regression loss:
Lp
Σ
- 1z)g(x)ll2.
(6)
Pooarse
ZE
The discriminator would be used for deciding point sampling privilege at inference time.
Guided Down-sampling. In view of the imbalanced population issue discussed above, an intuitive idea to alleviate it would be to allow larger number of points generated via up-sampling before reducing the desired number via Furthest Point Sampling (FPS), However, FPS tends to select more points from outer regions, which increases the risk of including more outliers, and thus this operation usually results in a noisier output. As a result, we aim to take advantage the point diseriminator above for the down-sampling stage at inference time together with FPS. Specifically, for each point Prnarse: define point-wise existing probability by:
p(z) = a0-3- - -)7)== = - e(
Let s denote the scale for up- samsampling, then x: is corresponding to points the Pitarse (up-sampled from P Pcoarse) and they all share the same p(x) E 1). an independent probability of each point being sampled. p+ Piterse is first down-sampled in this manner and then combined with Prec (up- sampled from Po) before the final FPS, which ensures that we remain pre-defined number of points. This process leverages the prior learned during training to effectively reduce the unreasonably located points at inference time (Fig. 7), relieving the side effect brought by FPS operation.
7

---

Table 2: Point cloud completion results in terms of CD x10*, EMD x x10², and DCD, lower is better.
aipplane
Methods
Metrics
Avg...
4.50
8.33
13.11
12.86
20.00
14.63
1.73
6.17
5.84
PCN
7.99
5.75
6.90
1199
5.32
6.60
5400
9.84
4.85
7.87
10.56
4.03
4.86
6.80
DCD
0.178
0.519
0.450
0.517
0.710
0.552
0.559
0.580
0.662
0.562
0.608
0:129
0.416
0.548
0.491
0.415
0.553
CD)
4.06
9.08
6.64
15.01
19.25
9.78
14.36
9.66
22.33
9:73
15.51
5.13
1.86
6.25
5.81
4.99
10.29
PCN++
BMD
3,44
3.75
3.85
4.65
9.00
3.56
4.69
4,22
6,13
3.85
4.39
2.62
2.75
5.60
3.771
3.07
4.27
DCD
0.478
0.454
0,451
0.574
0.661
0,504
0.517
0,540
0.617
0.534
0.563
0.
0.360
0.527
0.447
0,393
0.508
CD
4.12
9.54
7.44
15.26
18.64
10:77
12.95
19.99
9.21
16.06
2.36
7.06
7.04
4.68
10.30
TopNet
6,30
4,00
7.01
7.0
8,09
6.27
6,80
DCD
0.536
0.558
0.548
0.650
0.508
0.599
0.600
0.678
0.558
0.622
0.492
0.457
0.572
0.542
0.496
0.598
CD
2.73
3.92
6.50
10.75
13:37
9.26
10.17
7,70
17:27
6.54
12.10
5.21
1.37
4.59
4.62
3.38
7.99
MSN
EMD
2.75
4.02
3.47
1.44
6.28
3.74
4.16
3.82
5.27
3.34
4.25
2.92
2,07
3.30
3.62
2.31
3.94
DCD
0,404
0,509
0,516
0,537
0,539
0,532
0
0.515
0.514
0.471
0.541
0.458
0.358
0.491
0.465
0,422
0.499
CD
2.20
7,92
5.60
7.49
8.15
7,45
7.52
5.20
11.90
4,88
7,39
4.53
1,15
3.90
3.44
3.22
6.09
VRC
4.65
6.58
5.25
4.45
4.56
5:27
0,509
0.699
0.488
0.515
0:438
0.527
0.401
0.470
0.319
0.452
2.72
9,0
6.58
9.99
11.55
9.38
9.80
17:23
10:34
7.87
VRC-EMD
EMD
2.50
3,65
3:23
4.35
5.31
3,61
3.93
3.58
5.17
3.19
3.97
2.69
2.05
3.06
3.48
2.39
3.62
DCD
0.369
0.483
0,473
0,502
0,499
0,509
0.450
0.478
0.547
0,423
0:487
0:424
0.349
0.445
0,430
0,370
0.461
CD
2.22
8,00
5.41
7,58
8.28
7.94
8.39
5,46
14.76
5.78
9.57
4.44
1,30
3.59
3.43
2.39
6.51
Our
BMD
2.29
4.43
3,46
3.92
3.98
3,89
5.34
3.13
3.91
3.02
3.38
2.39
3.67
0.447
0.427
0.451
0,445
0,423
0.426
0.504
0.453
0.401
0.365
5 Experiments
Dataset. We use the recently proposed MVP Dataset [17] for our study and experiments. It is a multi-view partial point cloud dataset covering 16 categories with 62,400 and 41,600 pairs for training and testing, respectively. It renders the partial 3D shapes from 26 uniformly distributed camera poses for each 3D CAD model selected from ShapeNet [30], and the ground truth point cloud is sampled via Poisson Disk Sampling (PDS).
Comparison Methods and Metries. We include the following methods for comparison in the main experiments: PCN [35], TopNet [23], MSN [13], and VRCNet [17] (with the PSK module discarded to improve efficiency while slightly scarifying performance). PCN++ is simple extension of PCN [35] that generates the double number of points for training and down-sampled to the required point number inference time. We report per-class results CD, EMD, and DCD for clear comparison across methods and a view of the different properties of these metrics.
Implementation Details. All the models are trained using the Adam optimizer [10] with the learning rate initialized at l and decayed by 0.7 every 40 epochs. We use batch size of 32 and a total epoch of 80. We set a 1000 for the evaluation of DCD, and a E [40, 100) for training. We set E [0, 0.5] and 3 = 9, 7 = 1 for our approach in the main experiments. Our work is implemented with PyTorch and is run Tesla V100 GPU.
5.1 Comparison of the Methods
The main experimental results for point cloud completion on MVP dataset are reported in Table 2. Note that most networks trained with the CD loss except when specified. Early works, PCN [35] and TopNet [23] have relatively high loss for all the evaluated metrics, CD, EMD, and DCD. PCN with an additional up-sampling step, namely PCN++, notably lowers down EMD and DCD, while marginally raising CD for sacrifice; MSN [13] benefits from the multi-surface design and obtains low EMD and DCD results, while its CD performance is not that satisfying. The previous SoTA method, VRCNet [17] outperforms the other methods under the metric of CD and DCD, but we observe that its EMD loss is surprisingly high. We further replace the CD loss with EMD when training VRCNet (denoted as VRC-EMD). and it achieves the lowest EMD yet obviously higher CD than the original version. In comparison, our method reports the lowest DCD, second-lowest CD, and comparable EMD with VRC-EMD. Qualitative results (Fig. 5) show that our results are apparently superior in both the global balanced point distributions and local structures. user study in the supplementary material will further validate that our method benefits from higher visual quality.
8

---

DCD
3175
Case
PONH
VRC-EMD
3125
05
Case EMD
CD DCD
0.07
Case
3060
Case
3,05
VRC
OUR
CD
Case EMD 1 CD / DCD 1
CD &
+
0.00
0.000
0.0m
0.007
0.201
0.004
5.005
0.006
Partial Input
PCN++
VRC-EMD
VRC
Our
Ground Truth
EMD | CD DCD
6.23 | 4.69 0.52
5.31 3.29 0.50
5.30 8.10 0.55 0.55
5.07 3.89 0.45
11.95 4.10 0.60
7,83 4.40
5.24 5.03 0.44
5:45 3.85 0.41
2.61 2.58 0.42
2.24 0. 0.42
1.89 3.36 0.43
1.47 2:12 0033
examples from different methods that represent three typical cases: 1) EMD is low while CD is high; 2) CD is low while EMD is high; 3) both CD and EMD are low. DCD only lowered down in the third case. Upper left: close shot at the cases above. Upper right: visualization of the three metrics correlation between CD and EMD is weak and the scatter points form fan-shaped area, and 2) DCD (denoted by the point color) becomes lower towards the original point where both CD and EMD are low.
Figure 5: Comparison of CD, EMD, and DCD through examples and statistics. Below:
test set, we can observe that 1) the positive
5.2
Comparison of the Metrics
Consistency and Reliability. When taking closer look at all the CD-EMD-DCD tuples (either for each category for the averaged results) in Table the results by different metries do not exhibit clear positive correlation. But we also observe that for those that have a similar value of CD, DCD is usually dominated by their EMD performance, and for those with similar EMD results, DCD is highly correlated with CD. Although the law is not strictly held or theoretically proved, the scatter plot by each instance from different methods in Fig. 5 (upper right) also supports empirical observation. reveals that since CD and EMD focus on different aspects the point cloud, inconsistency and confusion may exist for the similarity measurement, which prevents either of them from being comprehensive metric. In comparison, DCD reflects the behaviors of them both and could only be reduced when both CD and EMD are relatively low that it serves as a more consistent, stable, and comprehensive metric, especially when CD and EMD encounter disparity. Our user study in the supplementary material further indicates that DCD more faithful metric visual quality.
Bounded Distance. One advantage of DCD is its bounded value range, which promotes an equal consideration for all samples in the dataset at the evaluation stage. In contrast, CD and EMD dominated by the worst cases the dataset due to their unbounded nature, as shown in Fig. Specifically, for CD, 80% of the normalized loss accumulation is contributed by only top 50% of the samples ranked by the shape-level distance, and 50% contributed by the top 25%; a similar case is also observed for EMD. This property makes them fail to comprehensively evaluate the quality of all samples in the dataset. Our DCD overcomes this problem by assuring clear [0.1] boundary for the distance value, enabling more comprehensive and stable statistical results the whole test set.
5.3 Ablation Study
We study the effectiveness of each component our method separately, including DCD loss the additional point up-sampling (PU), and the guided point down-sampling (GPD). As shown in Table 3,

---

Chamfer Distance
Earth Mover's Distance
Density-aware aware Chamfer Distance
35.00
0.8
Disribution Rccumulation
Distribtion
Dutribution
Accomulztion
Accumalation
0.5
0.3
0.5
0.3
000
10400
Ramiked instance id
20:00
Ranked instance
Ranleed Instancee (d
Figure 6: Distribution and accumulation (normalized to 1) evaluation results per-shape in the test set.
Table 3: Ablation Study.
Table 4: Evaluation results on three metrics when trained with each of them. denotes applying LEMD on the final outputs while LCD on intermediate ones.
Metric
DCD
PU
GPD
CD
EMD
DCD
Model
PCN [35]
VRC [17]
6.09
5.27
0.462
5.85
5.14
0.457
Metric
LCD
LEAD
LOCD
LCD
LEMD
LDCD
6.91
3.78
0.425
CD
9.78
10.70
9.36
7.87
6.65
3.68
0.422
EMD
6.80
3.97
4.71
5.27
3.62
5.14
6.51
3.67
0.420
DCD
0,553
0.537
0.526
0.462
0.461
0.457
leveraging LDCD loss usually outperforms the same model trained with LeD in all the three metrics; the additional point up-sampling (PU) significantly reduces EMD while increases CD at the same time; after guided down-sampling (GPD) is applied to the model with both Lpon and PU, achieve the lowest EMD and DCD and relatively low CD.
Ground Truth
Ground Truth
7.36
Ground truth
Ground
Ground Truth
11.93
3.60
Ground Truth
Figure 7: Examples of guided down sampling (from left to right) and evaluation in terms of CD x1
5.4 Performance as the Loss Function
We evaluate the effectiveness of DCD loss function by training networks with each of the three metrics (denoted by LCD. LEMD. and LDCD) and evaluate on all of them (denoted by CD, EMD, and DCD). We conduct experiments the baseline model PCN [35] and the SoTA VRCNet [17], as shown in Table 4. Compared with -tantnee networks, training with LEAD D produces the lowest EMD and lower dpen. yet it suffers from non-negligible scarification of CD and much heavier computational burden, which is to be discussed the supplementary material; training with LDCD will produce the lowest DCD, significantly reduce D. and the Lep-trained models (especially on PCN). The time consumption is also comparable with LCD. which further validates DCD's ss convenience and superiority an objective function. We a 50 or 100 here with 3 = 0 and replace all the LOD with LpCD. Experiments on how and 3 affect the performance are included in the supplementary material.
slightly reduce CD than
Conclusion
In this work, we propose a new similarity measure point clouds named Density-aware Chamfer Distance (DCD). It is bounded in value, effective in computation, and faithful to visual quality by considering both the density distribution and detailed structures. Our method achieves noticeable improvements under DCD and superior visual quality compared with previous works.
10

---

Acknowledgements. This study is supported in part by the NTU NAP, and under the RIE2020 Industry Alignment Fund Industry Collaboration Projects (IAF-ICP) Funding Initiative. as well as cash and in-kind contribution from the industry partner(s). It is also supported in part by Centre for Perceptual and Interactive Intelligence Limited, in part by the GRF through the Research Grants Council of Hong Kong under Grants (Nos. 14208417, 14207319 and 142035 (1551) 88) and ITTS/43 in part by CUHK Strategic Fund and CUHK Agreement 12193, in part by the Shanghai Committee of Science and Technology, China (Grant No. 20DZI 100800).
References
[1] Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning repre- sentations and generative models for 3d point clouds. In Proceedings of the 35th International Conference on Machine Learning, volume 80, pages 40-49. PMLR, 10-15 Jul 2018.
[2] Xuelin Chen, Baoquan Chen, and Niloy J Mitra. Unpaired point cloud completion on real scans using adversarial training. The International Conference on Learning Representations, 2019. 3 [3] Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner. Shape completion using 3d-encoder- predictor cnns and shape synthesis. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition, pages 5868-5877, 2017. 3
[4] Haowen Deng, Tolga Birdal, and Slobodan llic. 3d local features for direct pairwise registration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3244-3253, 2019. 1,3
[5] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEÉE conference on computer vision
and pattern recognition, pages 605-613, 2017. 2
[6] Xiaoguang Han, Zhen Li, Haibin Huang, Evangelos Kalogerakis, and Yizhou Yu. High- resolution shape completion using deep neural networks for global structure and local geometry inference. In Proceedings of the IEEE International Conference on Computer Vision, pages 85-93, 2017. 3
[7] Pedro Hermosilla, Tobias Ritschel, and Timo Ropinski. Total denoising: Unsupervised learning of 3d point cloud cleaning. In Proceedings of the IEEE/CVF International Conference on Compuler Vision. pages 52-60, 2019.
[8] Zitian Huang, Yikuan Yu, Jiawen Xu, Feng Ni, and Xinyi Le. Pf-net: Point fractal network for 3d point cloud completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7662-7670, 2020. 3
[9] Daniel P Huttenlocher, Gregory A. Klanderman, and William Rucklidge. Comparing images using the hausdorff distance. IEFE Transactions on pultern analysis and machine intelligence,
15(9):850- 16350--63,111 1993. 3
[10] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6988 2014. 8
[11] Ruihui Lii, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Pu-gan: a point cloud upsampling adversarial network. In Proceedings of the IEEE International Conference on Computer Vision, pages 7203-7212, 2019.
[12] Tianyi Lin, Nhat Ho, and Michael Jordan. On efficient optimal transport: An analysis of greedy and accelerated mirror descent algorithms. In International Conference on Machine Learning,
pages 3982-3991. PMLR, 2019. 4
[13] Minghua Liu, Lu Sheng, Sheng Yang, Jing Shao, and Shi-Min Hu. Morphing and sampling network for dense point cloud completion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11596-1 11596-11603, 2020. 2, 3, 4, 6, 8
[14] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4460-4470, 2019. 3.7
[15) Trung Nguyen, Quang-Hieu Pham, Tam Le, Tung Pham, Nhat Ho, and Binh-Son Hua. Point-set distances for learning representations of 3d point clouds. arXiv preprint urXiv:2102.04011
2021. 3
[16] Yinyu Nie, Yiqun Lin, Xiaoguang Han, Shihui Guo, Jian Chang, Shuguang Cui, and Jian Jun Zhang. Skeleton-bridged point completion: From global inference to local adjustment. Advances
in Neural Information Processing Systems, 2020.
[17] Liang Pan, Xinyi Chen, Zhongang Cai, Junzhe Zhang, Haiyu Zhao, Shuai Yi, and Ziwei Liu. Variational relational point completion network. Proceedings of the IEEE/CVF Conference

---

Computer Vision and Pattern Recognition, 2021. 3, 6. 8, 10, 13
[18] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 165-174,
2019. 3, 7
[19] Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Con- volutional occupancy networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 523-540, Springer, 2020. 3,
[20] Edward J Smith and David Meger. Improved adversarial systems for 3d object generation and reconstruction. In Conference Robot Learning, pages 87-96, 2017.
[21] Yongbin Sun, Yue Wang. Ziwei Liu, Joshua Siegel, and Sanjay Sarma. Pointgrow: Autore- gressively learned point cloud generation with self-attention. In Proceedings of the IEEE/CVF
Winter Conference on Applications of Computer Vision, pages 61-70, 2020.
[22] Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. What do single-view 3d reconstruction networks learn? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages
2019. 2
[23] Lyne P Tehhapmi Vineet Kosaraju, Hamid Rezatofighi, Ian Reid, and Silvio Savarese. Topnet: Structural point cloud decoder. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, pages 383-392, 2019. 3, 8
[24] Dahlia Urbach, Yizhak Ben-Shabat, and Michael Lindenbaum. Dpdist: Comparing point clouds using deep point cloud distance. The European Conference on Computer Vision (ECCV), 2020.
3
[25] Tai Wang, Xinge Zhu, and Dahua Lin. Reconfigurable voxels: A new representation for lidar-based point clouds. In Conference on Robot Learning, 2020.
[26] Xiaogang Wang, Marcelo H Ang Jr, and Gim Hee Lee. Cascaded refinement network for point cloud completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 790-799, 2020. 6.6
[27] Yue Wang. Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph enn for learning on point clouds. Acm Transactions On Graphics 388)),3(55):1112 2019.
[28] Xin Wen, Tianyang Li, Zhizhong Han, and Yu-Shen Liu. Point cloud completion by skip- attention network with hierarchical folding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1939,1948, 2020. 3
[29] Rundi Wu, Xuelin Chen, Yixin Zhuang, and Baoquan Chen. Multimodal shape completion via conditional generative adversarial networks. The European Conference on Computer Vision
(ECCV), August 2020. 3
[30] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 1912- 1920, 2015. 8 [31] Chulin Xie, Chuxin Wang, Bo Zhang, Hao Yang. Dong Chen, and Fang Wen. Style- based point generator with adversarial rendering for point cloud completion. arXiv preprint arXiv:2103.0255 02000.0253..00 2021.
[32] Haozhe Xie, Hongxun Yao, Shangchen Zhou, Jiageng Mao, Shengping Zhang, and Wenxiu Sun. Grnet: Gridding residual network for dense point cloud completion. arXiv preprint
vrXiv:2006.0376 2020. 3
[33] Guandao Yang. Xun Huang, Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan.
Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the
IEEE/CVF International Conference on Computer Vision, pages 4541-4550, 2019. 1
[34] Yaoqing Yang, Chen Feng. Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 206-215, 2018.
[35] Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. Pen: Point completion network. In International Conference on 3D Vision (3DV), pages 728-737. IEEE,
2018. 1,3, 8, 10
[36] Junzhe Zhang. Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, and Chen Change Loy. Unsupervised 3d shape completion through gan inversion. In CVPR, 2021. 1
[37] Wenxiao Zhang, Qingan Yan, and Chunxia Xiao. Detail preserved point cloud completion via separated feature aggregation. arXiv preprint arXiv:2007.00 023744, 2020. 3
12

---

[38] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma. Wei Li, Hongsheng Li, and Dahua Lin. Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. In Proceedings of the European Conference Computer Vision, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See supplementary material.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material as a URL)? [No] We will release the code upon the paper accepted.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Sec. 5.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Sec. 5.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] MVP dataset [17].
(b) Did you mention the license of the assets? [No]
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're
using/curating? [No]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "tMvn7LoHKiL", "reviewer": "Reviewer_ziDa", "summary": "The paper proposes a new distance metric for measuring the difference between two point clouds (say P1 and P2). In the past, either Chamfer or Earth-Mover's distance was used. The new distance builds on top of Chamfer, and essentially adds more weight to a point in P1 where multiple points from P2 are matched to that point. The authors claim that the new metric is:\n\n- More distribution sensitive compared to Chamfer.\n- More detail sensitive compared to Earth-Mover.\n- Efficient to compute compared to Earth-Mover.\n- More robust in general.\n\nI must admit that I am not an expert in point cloud matching. But I mostly agree with the authors' assessments.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I am leaning towards weak reject. The contribution, although very simple, seems to do well on the point completion task. However, I have concerns on the limited amount of experiments.\n\nInsufficient experiments: My main issue with the work is that the experimentation is only on point completion. Chamfer is also used for many other application, e.g. ICP. If the argument is that the new metric is superior than Chamfer, then it needs to be shown on a few applications where Chamfer is usually the distance measure of choice. If the argument only applies to points completion, the scope of the problem seems too small. Alternatively, one could measure downstream metrics as well that uses point cloud matching, e.g. structured depth. Given how simple the new metric is, I feel that the experimentation has to be extended by a substantial amount.\n\nSome other major feedbacks and questions:\n- ln 241: why not using BCD as the loss?\n- Instead of using the new metric, could we just use the product of CD and EMD (or the sqrt(CD) * sqrt(EMD))? Wouldn't we have the same behavior as the ones in figure 4?\n\nNit:\n- Was the claim that the new metric is more robust substantiated anywhere?\n- Figure 1: FPS -> farthest point sampling (its not a trivial abbreviation)\n- ln 48. violently? better word choice please\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "1.5 - 3 hrs, didn't count exactly"}, "extra_text": {}}, {"review_id": "nwdzXSk5tiy", "reviewer": "Reviewer_b3Fh", "summary": "This paper proposes a new evaluation metric for point cloud processing, namely BCD (balanced CD). The authors began by discussing the shortcomings of the commonly used CD and EMD losses, which may result in inconsistent evaluation, especially for imbalanced point clouds. By some simple modifications, the BCD can alleviate the issues of CD loss and report more reliable results which are consistent with the visualization. The authors then explored the architecture for point cloud completion, focusing on intermediate coarse prediction. Some modifications are specifically designed to solve the issues of the current completion framework. The proposed completion network achieves the best results under most assessment criteria (CD, EMD, BCD).", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The authors begin by doing a thorough study of CD and EMD losses, two widely used assessment criteria for point cloud processing. The authors emphasize two critical aspects, the data balance (point distribution) and quality (close to the underlying surface). By theoretical analysis and experimental demonstration, the authors show CD and EMD address just one component and neglect the other, resulting in the contradiction between these two measures. Moreover, these two metrics are unbounded.\n\n2. The suggested BCD is derived by simply modifying the CD loss. More precisely, the suggested BCD differs from the original CD in the following ways: BCD introduces Nx, Ny to resolve the density imbalance issue; BCD realizes a bounded metric by using exponential distances rather than L2 distances. These modifications are simple and reasonable to some extend.\n\n3. The authors also provide a detailed discussion for the point cloud completion task. Similar to the proposed BCD metric, some modifications of the standard completion network are proposed to address the imbalance prediction issue of the intermediate coarse prediction. The newly suggested network can attain SOTA performance across almost all assessment criteria.\n\nIn summary, the reviewer appreciates the paper’s in-depth analysis of both the evaluation metrics CD, EMD and the completion networks. In the following, I list some potential limitations of this work:\n\n1. The newly introduced Nx and Ny seem only valid to process uniformly distributed ground truth point sets, which are also referred to in L45 as \"balancing.\" However, if the ground truth is not uniformly distributed, I'm curious if BCD can still provide a consistent result with such a design. I think the authors should have a more detailed discussion of this situation.\n\n2. The proposed BCD is only treated as an evaluation metric. Why not use it as the training metric to replace CD or EMD?\n\n3. Although the issues of CD and EMD are examined in-depth, the proposed BCD is also based on CD with minor modifications. I think that such changes may not entirely address the mentioned issues.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "lkJaB9ROwp5", "reviewer": "Reviewer_1bxc", "summary": "This paper proposes an improved chamfer distance to evaluate the similarity of two point clouds. ", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: This work is an improved version of chamfer distance. Based on my knowledge, it is an original work and the paper has already explained the difference to other works clearly. However, I think the novelty is limited.\n\nQuality: This work is technically sound and the claims are well supported. The method explains their strengths very well but does not mention the weakness that much. I think this paper is a work in progress.\n \nClarity: This paper is well-written and well organized. \n\nSignificance: This work is also medium significant.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors address the potential limitations in the supplement material.", "review_text": "", "extra_scores": {"ethical_concerns": "n/a", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "br9mnAMaZb", "reviewer": "Reviewer_UpWV", "summary": "The paper presents a new similarity measure named Balanced Chamfer Distance (BCD), which is derived from the original Chamfer Distance (CD) through an approximation of Taylor Expansion. \n\nThe authors propose a better sampling based on point query frequency to improve the overall performance for point cloud completion.\n\nThe authors also empirically demonstrate that BCD provides a more robust evaluation.", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**: \nThe proposed BCD makes a Tayler expansion to approximate the $x^2$ with the first order Tayler expansion $e^{x^2} \\approx 1 + x^2$ ($x^2 \\approx 1 - e^{x^2}$). Therefore, it bounds each point-level distance to a value between 0 and 1. Also, due to the exponential function, it potentially mitigates the outlier prediction when $x^2$ becomes large. \n\nTo further handle the imbalance ambiguity of CD, the authors propose to add one extra normalization term for each point-level distance by considering points’ surroundings, point-specific query frequency. Based on the frequency, the authors also designed a discriminator to guide point sampling to sample important points.\n\nThe reviewer considers the overall pipeline to be novel.\n\n**Quality**: The paper is intuitive and straightforward to implement. \nHowever, the major limitation the reviewer captures from the paper is that the BCD is only used during the test stage. As both CD and EMD are widely used for self-supervised learning for point-related problems, it seems to be quite unfortunate that the authors didn’t try to use BCD as the objective function to guide the learning tasks and report the numbers. This could be making it more useful for point cloud literature.\n\nAs the authors only use BCD for evaluation, the improvement made from the paper seems to mainly come from an empirical approach to conduct better sampling compared to FPS based on point query frequency. It is somewhat an incremental contribution. \n\n**Clarity**: It is not clear on the details of the proposed model. For example, the authors should have mentioned the baseline architecture without the balanced design. It is not clear to the reviewer whether the authors built their method upon PCN, PCN++, TopNet, MSN or VRC. Therefore, it is difficult for the reviewer to evaluate whether the balanced design indeed improves the performance.\n\n**Significance**: The authors propose a bounded version of Chamfer Distance, the Balanced Chamfer Distance, for distance measurement on point clouds. It is tackling important problems that are interesting to the point cloud literature.  However, its limitation is obvious that the metric is only limited for evaluation and can not be used for training as CD and EMD. \n\nIn the end, the method of the paper tends to ‘degrade’ to designing a better sampling based on point query frequency, which is unfortunately considered as an incremental contribution.\n\nTherefore, the reviewer tends to give a borderline rejection given its current version as the work seems to be still an on-going work that remains to explore the training with BCD. The authors have attempted solving an important research problem, the reviewer encourages them to further move forward along the direction.\n\n[1] A Point Set Generation Network for 3D Object Reconstruction from a Single Image\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The major limitation of the paper is that the metric is limited for evaluation only.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers all agreed that this submission should be rejected. The reviewers find that the method lacks sufficient novelty and significance, and that the experimental results are not strong enough. In particular, the results show that the proposed method does not provide a significant improvement when used as a training objective, and the experiments are not comprehensive enough for the task of evaluation. 

As a (very) minor side note, I also noticed that the paper claims that the EMD requires the point sets to have the same number of points, which certainly isn't true. The EMD is just the Wasserstein distance, which can be evaluated exactly between any two discrete distributions regardless of the (finite) number of points in the support.
Decision: Accept (Poster)