Deep Point Cloud Reconstruction
Authors: Jaesung Choe
ByeongIn Joung
Francois Rameau
Jaesik Park
In So Kweon
Venue: ICLR 2022 Poster (2022)
Keywords: Computer Vision
3D Geometry
Deep Learning based Point Cloud Understanding
Point Cloud Denoising
Point Cloud Upsampling

Abstract: Point cloud obtained from 3D scanning is often sparse, noisy, and irregular. To cope with these issues, recent studies have been separately conducted to densify, denoise, and complete inaccurate point cloud. In this paper, we advocate that jointly solving these tasks leads to significant improvement for point cloud reconstruction. To this end, we propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse stacked-hourglass network as for the initial densification and denoising, 2) a refinement via transformers converting the discrete voxels into continuous 3D points. In particular, we further improve the performance of the transformers by a newly proposed module called amplified positional encoding. This module has been designed to differently amplify the magnitude of positional encoding vectors based on the points' distances for adaptive refinements. Extensive experiments demonstrate that our network achieves state-of-the-art performance among the recent studies in the ScanNet, ICL-NUIM, and ShapeNet datasets. Moreover, we underline the ability of our network to generalize toward real-world and unmet scenes.


Published as a conference paper at ICLR 2022
DEEP POINT CLOUD RECONSTRUCTION
Jaesung Choelt, Byeongin Joung"t Francois Rameau', Jaesikk Park?, In So Kweon
KAIST', ?OSTECH?
ABSTRACT
Point cloud obtained from 3D scanning is often sparse, noisy, and irregular. To cope with these issues, recent studies have been separately conducted densify, denoise, and complete inaccurate point cloud. In this paper, we advocate that jointly solving these tasks leads to significant improvement for point cloud re- construction. To this end, we propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse sacked-hourglans network as for the ini- tial densification and denoising, 2) refinement via transformers converting the discrete voxels into 3D points. In particular, we further improve the performance of transformer by a newly proposed module called amplified positional encoding. This module has been designed to differently amplify the magnitude of the po- sitional encoding vectors based the points' distances. Extensive experiments demonstrate that our network achieves state-of-the-ar performance among the re- cent studies in the ScanNet, ICL-NUIM, and ShapeNetPart datasets. Moreover, we underline the ability of our network to generalize toward real-world and unmet scenes]
INTRODUCTION
3D scanning devices, such as LiDAR and RGB-D sen- sors, allows to quickly and accurately reconstruct scene as a 3D point cloud. This compact 3D representation is commonly used achieve various tasks in autonomous driving (Yang et al., 2018 Qi et al.) 2018 Shi et al.)
2020 Lang et al., 2019 Choe et al. 2021c b (019b),
Noisy
Sparse
Incomplete I
robotics (e et al.) 2021: 2020 Wang et al. 2019), or
3D mapping (Choy al. 2020 Choe et al. 2021a d).
Deep point cloud reconstruction
However, processing raw point cloud happens to be par- ticularly challenging due to their sparsity, irregularity and
sensitivity to noise (Xiang et al. 2019
Guo et al. 2020).
To address these issues, recent deep learning-based stud- ies have proposed to improve the quality of point cloud. Specifically, prior works that are dedicated to point cloud refinement can be categorized into three distinct sub- fields: (1) point upsampling Yuu et al. 2018b), (2) point denoising (Rakotosaona al. 2020). and (3) point com- pletion (Yuan et al.) 2018). Each of these tasks has its benefits and inconveniences. For instance, the point upsampling task usually handles noisy input points but do not explicitly deal with strong outliers. On the other hand, point denoising techniques have been designed filter noisy data but cannot densify the point cloud. Finally, point completion methods mainly focus on object-level full 3D completion but are known for their weak generalization to unmet environments unknown category of objects. Though there exist differences among the aforementioned tasks, their common goal is to improve proximity-to-surfe
Point cloud reconstruction. We propose a novel neural architecture that jointly solves inherent shortcom- ings in raw point cloud, such as noise, sparsity, and incompleteness.
Figure 1:
in the 3D point representation.
Thus, we believe in the complementary of these three sub-tasks and hypothesize that joint frame- work is greatly beneficial to deal with deteriorated and incomplete 3D point cloud. Accordingly, this paper challenges to resolve all these issues as single task, called point cloud reconstruction as
Both authors have equal contributions.

---

Published as a conference paper at ICLR 2022
depicted in Fig. [ To the best of our knowledge, this paper is the first attempt to jointly resolve the inherent shortcomings of point cloud obtained from 3D scanning devices: sparsity, noise, irregular- ity, and outliers. To this end, propose deep point cloud reconstruction network that consists in two stages: a voxel generation network and a voxel re-localization network.
In the first stage, the voxel generation network (Sec. 3.1) aims to densify voxels and remove out- liers via sparse convolution layer (Choy et al., 2019). Rather than to use k-Nearest Neighbor that sensitive to points' density (Mao al. 2019), we utilize voxel hashing with sparse convolution layer to understand absolute-scale 3D structures for densification and denoising. Despite its suc- cess, voxelization inevitably leads information loss due the discretization process. To provide a fine-grained reconstruction, in the 2nd stage, we propose a voxel re-localization network that con- verts discrete voxels into 3D points using transformer. Additionally, we increase the performance of transformer by the amplified positional encoding. In our analysis, spatial frequency plays an important role in the context of voxel-to- co-point conversion. Our new positional encoding strategy useful to infer descriptive and detailed point cloud by simply changing the amplitude of the encoding vector. Our contributions can be summarized as follows:
New problem formulation: point cloud reconstruction.
Novel two-stage architecture for voxel-to-point refinement.
A large number of experiments illustrating the generalization ability of our point cloud reconstruction to real 3D scans.
RELATED WORKS
Point cloud obtained from 3D scanning devices are known to contain various artifacts (Berger et al, 2017). such noise, outliers, irregularity, and sparsity. Point cloud reconstruction aims resolving these aforemmnnioned issues in order to provide fine- nne-grained 3D reconstructions. In this section, we introduce different approaches related to point cloud refinement.
Point Cloud Denoising. Without extra information, such as RGB images, point cloud denoising purely based the input point distribution is challenging task (Leel 2000 Avron et al.) 2010).
Recent deep learning- baaning-based strategies (Rakotosaona et al., 2020 Roveri et al.j 2018]] Hermosilla et al. 2019 Pistilli et al. 2020, Luo & Hul (021) demonstrate promising results. However, despite the large improvement provided by these architectures, these methods do not have the ability to densify the reconstruction.
Point Cloud Completion. A family of solutions called point cloud completion have recently been proposed alleviate the problem of incomplete reconstruction. One of the pioneer papers (Yuan et al. 2018) proposes to overcome the point incompleteness problem by generating points in both non non-observed and visible areas. To improve the completion process, recent approaches (Huang et al. 2020 Zhang et al. [020]] Wen et al.] [021] Xiang et al.) (02I) propose to include prior structural information of the objects. While these object-specific point completion techniques can lead to accurate results, they suffer from a lack of versatility since the object class is assumed to be known beforehand. Without such semantic level priori, the entire completion in occluded surface (e.g., behind objects) is doubtful task. Thus, in this paper, focus on reliable and generic object-agnostic approach that is able to complete visible areas within an arbitrary point cloud and to cope with sparse, noisy, and unordered point cloud.
Point Cloud Upsampling. Given a sparse 3D reconstruction, the goal of point cloud upsampling
is to densify the point cloud distribution. Conventional methods (Alexa et al. 2003 Huang al.
2013) rely on point clustering techniques to re-sample points within the 3D space. Recent deep
learning-based studies (Li al., 2021 Yu et al., 2018a Yifan al.) 2019 Yu al. 2018b) and
concurrent study Qiu et al. (2021) propose point densification networks under the supervision of dense ground truth points. Though some papers internally conduct point denoising (Yu et al.) [2018a]
Yifan et al. 2019), most of these works mainly assume that there are no strong outliers among the input points, such as flying-point cloud (You al. 2019).
2

---

Published as a conference paper at ICLR 2022
Voxel Generation Network (Sparac slacked boungiasa network)
Vaxel Re- Re-localization Network (TransSomen and smpliiiedd PE)
Vin
Vout
EriIEr Self-attention
Vexelized
Voxel-to ppoii
(vosel hoching)
Input point cloud Pin
Oulput point cloud Pout
Amplified
layer
Quety vexel
Neighbor voxel
Figure 2: Two-stage architecture for point cloud reconstruction. this figure, point cloud has been colorized for visualization purpose and we set the identical size of visualized points in Pin and Pour for fair comparison.
Surface Reconstruction. Point cloud meshing is common solution for surface reconstruc- tion (Hoppe et al. 1992 Curless & Levoy 1996). For instance, this process can be achieved by
converting point cloud implicit surface representation via signed distance function (SDF) and to apply Marching Cubes (Lorensen & Cline 1987) or Dual Contouring (Ju et al. (002) for meshing
(e.g., Poisson surface reconstruction (Kazhdan et al. 2006)). This strategies is known be effec- tive under favorable conditions but can hardly handle sparse and non-uniform points' distribution. Moreover, such technique tends to lead over-smoothed and undetailed reconstruction (Berger al. 2017). Similar limitations can be observed with recent deep learning-based SDF represen- tation Park et al. 2019). Many other solution have been developed for the problem of surface re- construction, such as, Delaunay triangulation (Rakotosaona et al. 2021 Boissonnat 1984 Kolluri ict al. 2004), alpha shapes (Edelsbrunner & Mücke, 1994), or ball pivoting (Bernardini et al. 1999).
Despite their compelling performances, these approaches require uniform, dense, and accurate puts since they usually preserve the original distribution of a given point cloud. This assumption still holds for the recent studies (Sharp & Ovsjanikov 2020: Rakotosaona al. 2021).
Compared to surface reconstruction, point cloud reconstruction can be viewed as a prior prior-process overcome natural artifacts: sparsity, noise, irregularity. Furthermore, in contrast to previous studies, our method are object-agnostic and generally applicable toward the real-world 3D point cloud. To this end, introduce our deep point cloud reconstruction network that consists of the voxel generation network and the voxel re-localization network.
DEEP POINT CLOUD RECONSTRUCTION
Given noisy and sparse point cloud Pin. point cloud reconstruction aims to generate dense and accurate set points Pout- For this purpose, we propose a deep point cloud reconstruction net- work composed of two stages: voxel generation network (Sec. 3.1) and voxel re-localization net- work (.2). The first network has been designed to perform densification and outliers removal in the sparse voxel space. The second network complements and improves this first stage by de- noising the data further and by converting voxels back to 3D points. The overall architecture is illustrated Fig. 2
3.1 VOXEL GENERATION NETWORK
Most previous studies (Li et al. 2021 Yifan al. 2019 Yu al. 2018a| Luo & Hu 202 attempts to resolve these problems by directly processing raw point cloud. However, it is known to be a challenging problem due the unordered nature the 3D point cloud. To tackle this issue, we propose to convert the 3D point cloud Pn, into sparse voxel representation Vnn through our voxel generation network. From this data, our network generates a refined and densified volume V A noticeable advantage of sparse voxel representation is its ability to preserve the neighbor connectiv- ity of point cloud a uniform and regular 3D grids (Guo et al., 2020) via voxel hashing (Choy et al.) 2019 Graham & van der Maaten 2017). Voxel hashing keeps tracking of the geometric positions of sparse tensors so that we can find the voxels' neighbors to apply sparse convolution. Moreover, sparse convolution layer
cover the consistent and absolute-scale scope of the local region and hierarchically capture its large receptive fields. Such property is then particularly beneficial to deal with inaccurate and unordered input point cloud P..

---

Published as a conference paper at ICLR 2022
V/n
mid
out
Sparse corw block
Generative spanse transposed onne
Sparse bransposed comv
Prune
Sparse interpolation
Figure 3: Voxel generation network. We design sparse stacked hourglass network that hierarchi- cally and sequentially generates and prunes voxels a coarse-to- carse-to-finee manner.
In order to denoise and densify Vm. our voxel generation network consists of sparse stacked hour- glass network as shown in Fig. 3 Our network architecture is close to the previous studies (Newell et al. 2016 Chang & Chen 2018 Im al. 2019; Choe et al., 2019a) in that it is composed of several hourglass networks for cascaded refinements of images or dense volumes.
the context
of this paper, scanning devices only reconstruct the surface of the structure, thus, the resulting voxelized volume inherently sparse. Therefore, to process this 3D volume efficiently, our voxel generation network takes advantage of the sparse convolution network (Choy al. 2019).
should be noted that the specificity of sparse convolution operation is particularly desirable for the tasks hand. First, generative sparse transposed convolution layer (Gwak et al. 2020) has the ability to create new voxels given a pre-defined upsampling rate r, which can be seen as a point upsampling methodology (Yu al. 2018b). Secondly, voxel pruning widely used for fine-grained volume rendering (Liu et al. 2020) and efficient computation (Choy et al. (019). In our context, voxel pruning is closely related outlier removal. Last, the joint use of generation and pruning enables our network to create an arbitrary number of points.
As illustrated in Fig. 3 we first compute local voxel features using sparse convolutional blocks in an and pruning layers
encoder part. The aggregated voxel features undergo the voxel generation layers decoder part. We use two hourglass networks to densify and prune voxels. To train the voxel generation network, we adopt Binary Cross-Entropy loss (BCE) to classify the status of estimated voxels into an occupied/empty state as:
Σ Vv log(th)) + (1 -yy) log(1 fv), vEV.
where Vv an estimated class of a generated voxel v from the network, and lv the ground truth class at the generated voxel .N, is the total number of voxels that we used to compute the BCE loss. v is on the 3D surface, yv becomes true class (i.e., =v=1). Furthermore, following the original loss design (Newell et al. 2016), we compute the BCE Loss LECE for both the final voxel prediction Vau and the intermediate results Vmii- In total, we compute the voxel loss L as:
Lvox LBCE + 0.5
Σ CBCE(Vi),
AG
NEV
where NG is the total number of voxels in the intermediate inference volume Vmid that we used to calculate the BCE loss. Along with the predicted voxels Vouns extract voxel features Fv using sparse interpolation as illustrated in Fig.
3.2 VOXEL RE-LOCALIZATION NETWORK
Despite the denoising and densification offered by the 1st stage network, voxelization inevitably leads to information loss. Thus, in this section, propose voxel re-localization network that converts the discrete voxels V into 3D points Pour ER Nx3, This conversion requires an un- derstanding of local geometry to describe the 3D surface group of point sets. Inspired by (Zhao et al. 2021 Choe et al. 2021d), we utilize self-attention as a point set operator.
Let us describe the detailed process in the voxel re-loralizzaiion network, which is illustrated in Fig. Given output voxels Vemn we collect the K(=8) closest voxels to each voxel v xx using the hash table that used in the 1st stage network. Then, obtain voxel set (= that
4

---

Published as a conference paper at ICLR 2022
V out
P Pout
V
CERCEDE
0000foo o0
Tranddomet
Transfomme encoder
Clusbering (vuned lendingg
encoder (sdtFittaniion layor)
V Voadl-to-polur
embodding
reroe-ttanttion layer)
vs
closn
f
V
Query voxel
Neighbor voxel
A Appliiid PE
Concalenale
Figure 4: Voxel re-localization network. The 2nd stage network involves transformers for voxel- to-point refinement. In particular, transformers are applied into self/cross-atteniie layers with our amplified positional encoding to compute the relation between a query voxel and its neighbor voxels.
is referenced to the query voxel v;. Similarly, we cluster a set of voxel features Fi= } k=1 that correspond to P,. As in Fig, 4 the 2nd stage network regresses the location of a query point (red voxel) using the geometric relation between the query voxel V; and its neighbor V (white voxels) through self/cross attention layers.
While this formulation close PointTransformer (Zhao al.] 2021), our voxel re-localization network has three distinctive differences. First, PointTransformer utilizes k-Nearest Neighbors to group 3D points but we re-use the voxel hashing that covers absolute scale local geometry. Second, network consists of both self-attention and cross-attention. In analysis, it is better to use two different attention layer than to solely adopt self-attention. Third, we design our own positional encoding system that controls the necessity of high-frequency signal by changing the amplitude of the encoding vector, called amplified positional encoding.
Originally, positional encoding has been introduced help transformer to understand the orders of tokens /Vaswani et al., 2017). When applied to other domains of applications, positional encoding is often used to preserve high-frreuency signals which tend to be disregarded by neural networks (Ra- haman et al. 2019 Ramasinghe & Lucey 2021). For example, positional encoding is proven to be essential synthesize sharp images from an implicit 3D neural representation (NeRF (Mildenhall ctt al. 2.20)). For our purpose, we need to consider spatial frequency to coherently re-locate query voxel V, by using its neighbor effective for the high- fregufryyy embedding. As in Fig, 4 we increase the amplitude of positional encoding when we need to move query voxel by a small amount of distance (i.e., high-frequency in 3D spatial domain). If query voxel is far away from its neighbor, we decrease the amplitude of the positional encoding. Based on the relation between high frequency signals and our voxel re-location scheme, we define our amplified positional encoding "ampPE as:
V, Our intuition is simple. Since positional encodings are mainly
7amppe (d, pos) = e eld "PEE(po))
(3)
YPE (pos) [..., sin (pos/ ppps/10000/// cos (pos/ 10000
(4)
where "ampPE an amplified positional encoding vectors and || | is the L1-norm. Following the conventions (Vaswani et al. 2017), 7PR is ID positional encoding vectors, pos represents position, i means dimension, and C is channel dimensions of voxel embedding. After we encode TampP aggregate this embedding through self- and cross-attention layers to
into the voxel embedding. capture local structures around the query voxel. Finally, the regressed offset Av is added to the original location of query voxel v. to update its location. To train the second stage network, calculate chamfer distance loss Laiss as:
Laist
[ min || - (11) + Nr. pEP,
Σ
min ||p - p12) pEP ynd
(5)
Np pEP
pEP,
5

---

Published as a conference paper at ICLR 2022
where Pred is the predicted point cloud, Pat indicates ground truth point cloud, p represents pre- dicted point, and means a ground truth point. Np and Np, are the number of prediction and ground truth point cloud, respectively. Through our two-stage pipelines, forms point cloud reconstruction. By jointly generating, removing, and refining points, our network robustly recover accurate point cloud and increase the level of proximity-to-surti
whole network per-
4 EXPERIMENT
In this section, we describe the implementation details of our network and we present a large series of assessments underlying the effectiveness of approach. Our point reconstruction network has been solely trained on the ShapeNet-Part Yi et al., 2016) dataset but tested on other real and synthetic datasets such ScanNet (Dai al. 2017) and ICL-NUIM (Handa al. 2014). These experiments highlight the ability of our solution to generalize well to other scenarios involving various levels of noise and different densities.
4.1
IMPLEMENTATION
We first train voxel generation network for 10 epochs using the Adam optimizer (Kingma & Ba 2014) with initial learning of 1e-3 and a batch size of 4. We decrease the learning rate by half for every 2 epochs. After that, we freeze the voxel generation network and initiate the training of the voxel re-localization using the inferred voxels Vour from the voxel generation network. The point re- location network is trained using identical hyper-parametts used for training the first stage network. Further details regarding the training rules are provided in Sec. 4.3
4.2 BASELINE APPROACHES
We conduct a thorough comparison against previous state-of-the-art techniques: PU for point cloud upsampling (Li et al. 202T). PD for point cloud denoising (Luo & Hu 202T). and PC for point cloud completion (Xiang et al. 2021). These three papers are currently positioned at the highest rank for each task and we utilize their official implementation for training and evaluation. ods are trained with the same data augmentation as competing approaches and its performances have been measured on various datasets: ShapeNet-part (Yi et al. 2016), ScanNet (Dai et al. (017), and ICL-NUIM (Handa et al. 2014) datasets. In particular, to the combination of the existing studies. For instance in Table I P((r=))- resents consecutive operation of point completion, and point upsampling and point denoising under upsampling ratio r=4. Note that we also take into account the point completion task since this task also considers point densification on visible surface.
meth-
analyze the efficacy of ours compared rep-
4.3 DATASET
NhapeNet dataset. We utilize 13 different categories of objects categories from the ShapeNet- Part dataset (Chang et al., 2015) Yi et al. (016). We extract dense and uniform 10K ground truth point cloud per object by using the Poisson Disk Sampling algorithm (Yuksel 20155) provided by Open3D (Zhou et al. 2018). Among these ground truth 3D points, we randomly sample 2048 points input P for the training networks. For the sake of fairness, we adjust the unit voxel length as 1vox=0000 =0.000 for training the networks. This is to produce comparable number of points than previous studies. We follow data augmentation scheme proposed by the previous point upsam- pling study (Li et al., (02T), such as random noise addition and random re-scaling. Additionally, we include random outliers that constitute less that percent of input points. train and validate the networks, carefully follow the official train/val/test split provided by (Yi et al. 2016).
ScanNet ICL-NUIM dataset. These datasets (Dai et al. 2017: Handa et al. 2014) are indoor datasets that provide RGB-D sequences and corresponding camera poses. We utilize these datasets to evaluate the generalization capability of ours and previous methods. Similar to the image- based reconstruction study (Yao et al.) 2018), firstly re-use the network parameters pre pre-trained in the ShapeNet dataset, then evaluate all methods in unmet datasets without fine-tuning networks. Further details are included in the supplementary material.

---

Published as a conference paper at ICLR 2022
ShapeNet- Fart dalaset
Methods
Fi
Pou
K-Chaamf. (.)
K=2
k=4
Percentage (<)
Comp.
f-score
0.6)
Acc.
Percentage (<)
Comp.
/-scure
1.0)
PC (rw4)
2048
8192
1.14
1.29
1.51
72.28
42.71
51.51
92.41
88,48
96.16
PU ()=1)
2048
8192
1.56
1.69
1.90
60.71
26.91
36.04
86.67
68.19
75.60
PointRecon Cho, 0200)
2048
8732
1.19
1.33
1.52
81,02
40.41
53.48
96.95
81.23
88.08
PC (r=4) + PU (r=4) - PD
2048
32987
1.14
1.48
1.50
56.55
49.72
52.43
87.13
82.73
84.67
PU (rw4) PC (rw4) PD
59.90
57.63
89.47
85.97
PointRecon (0.00000100
2048
15863
0.90
0.94
1.12
85.77
69.01
78.14
97.11
94.98
96.16
Nea dataset
Methods
Pn
Pon
C-Camm. (.)
Percentage (<) 0.5)
Perceniage (*)
1.0)
k=1
km2
km4
Acc.
Comp.
J-score
Acc.
Comp.
f-score
PC (r=16)
4098
65536
3.79
4.08
4.48
16.91
11.96
13.12
40.45
53.70
45.45
PC (r=1) (r=1)
64427
3.69
4.23
23.05
12.59
15.31
42.24
44.84
PU (y=10)
4096
65536
3.68
3.94
4.32
16.81
13.48
13.90
37.89
44.60
40.04
PU ()=1) -0 PC -
64578
2.66
23.30
21.86
64.11
$5.40
PU (y=4) - PD -C (r=4)
4096
63118
3.75
3.99
4.33
15.99
15.43
14,50
36.60
$4,70
43.10
PeintRecon (l.05000075)
36.83
44.72
68.34
76.72
NCIM dataset
Methods
Pn
Pon
A-Chaml. (.)
Percentage (1)
(.5)
Percenlage (<. 1.0)
Aoc.
Comp.
f-score
Acc.
Comp.
/ /scure
PC (r=16)
4098
63536
3.41
3.63
3.95
18.69
17.53
16.65
38.26
63.02
47.20
PC (r=4) PU (r=4) PD
4096
64451
3.61
3.81
4.11
21.62
11.41
14.05
45.34
42.57
43,32
P17 (y=16)
4096
65536
3.83
4.03
4.33
17.78
18.19
16.83
35.45
49.93
41.03
PU (r=() - PC (r=4) PD
4096
64872
3.61
3.94
4.42
27.55
8.64
12.42
54.42
33.58
40.60
P17 (r=4)
PD PC (r=4)
4096
63521
4.00
4.21
4.53
10.09
11.64
9.51
22.70
48.52
30.25
PointRecon (0.=0.0077))
4096
41322
1.87
1.99
2.18
67.78
44.97
53,36
89.09
70.52
78.141
PointRecon 20.05))
56902
2.14
2.26
42.85
80.26
67.78
75.41
PointRecon (1o -0.0000)
4096
99321
2.78
2.92
3.10
54.11
38.09
43.55
81.76
61,88
69.45
Table I: Quantitative results of point cloud reconstruction. Note that PC, PU, PD represent
point completion (Xiang et al., (02T), point upsampling (Li et al. [202T), and point denoising (Luo & Hu 202I). respectively. Please refer to the appendix for more results.
4.4 COMPARISON
For metric computation, we adopt Chamfer distance and percentage metries (e.g.. Accuracy, Com- pleteness, and f-score) Yao et al., 2018: Jensen al., 2014), as shown in Table [ In particular, we extend to measure the N-Nearest Chamfer distance (iee., k-Chamf that efficiently computes point-wise distance than Earth Mover's distance. Based on these criteria, we calculate the metrics of the whole methods.
As shown in Table 1 network achieves state-of-the-art performance compared the previous studies (Li et al. 2021; Luo & Hu 2021 Xiang et al. 2021). Moreover, the performance gap increases when we conduct generalization test ScanNet dataset and ICL-NUIM dataset. There are two dominant reasons behind the better generalization and transferability. First, each sparse convolution layer has consistent and absolute
receptive fields in 3D space. This local voxel neighborhood appears to be more robust than raw point cloud processing via k- Neeersstt Neighbor as adopted in the three baselines (Xiang et al., [022 Luo & Hu 2021 Li et al. 202I). This is also one initial processing of point cloud reconstruction, which
of the strength of using voxelization is conducted by our 1st network. Second, our network does not have task tasc-spe header networks tasks. Instead, Our network purely focuses the
or losses for hammppletio/usssappinn/oooooiiii inherent problems resided in raw point cloud. Such joint approach trains our network to improve the overall -tossffe- not the task-specific quality.
Regardless of the combined use of the baselines, our method systematically outperforms them. When we sequentially apply point densification and point denoising, the performances are not con- stantly improved in [ This observation is supported by the qualitative results in Fig. 5 While point denoising can effectively remove the remaining outliers that are generated from (Xiang al.) 2021) or (Li et al. [222]), it has chance to remove the wrong points sets. For instance in Fig. 4 the point denoising method erases part of the airplane's wing. In contrast, our voxel generation network conducts both voxel pruning and generation in each hourglass network so that it can effectively circumvent such case.
Precise equations of the metric computation are descrihed in the appendix.
7

---

Published as a conference paper at ICLR 2022
Inpui cloud
Ground truth
PU PD
PD
(a) Point cloud reconstruction results ScanNet dataset.
Input point cloud
Ground
Ours
far
close
(b) Point cloud reconstruction results ICL-NUIM dataset.
Input point cloud
PC
Ours
Ground truth
PU PD
PC - PD
(c) Point cloud reconstruction results ShapeNetPart dataset.
Figure 5: Point cloud reconstruction results using two baselines. Note that point cloud has been colorized for visualization purpose.
PC- CC-PD-J PU
PU PD PC
PC- PD
Our
Groun truth
Figure 6: Point cloud reconstruction results using three baselines, Note that point cloud has been colorized for visualization purpose.
8

---

Published as a conference paper at ICLR 2022
Type positional encoding No PE
Performance
preserve(V)
Performance
PE*
Tampli
Chaml.
7-score
1 stage
Sellaiiin
Cross-aitn
Chaml.
f-score
1.231
52,011
1.42
49,56
1.224
52.434
1.209
53.374
1.226
1.223
$2.692
1.195
55,488
1.195
55,488
Ablation study for amplified positional encoding.
(b) Ablation study for voxel re-localization.
Ablation study of our point cloud reconstruction network. In (a), PE* indicates positional encoding from NeRF Mildenhall al. (2020) In (b), S-hourglass represents the voxel generation network. Self-attn and Cross-attn mean self-attention layers and cross-attention layers, respectively. The evaluation is conduced in ShapeNet-Part dataset under lyox =0002, Pin -248..
Table 2:
4.5 ABLATION STUDY
In this section, we propose to evaluate the contribution of each proposed component (amplified po- sitional encoding and voxel re-localization network), through an extensive ablation study in Table
In Table (a), we validate amplified positional encoding in comparison with different styles of positional encodings, "PE (Vaswani et al.) 2017) and PE* (Mildenhall et al. (020). Also, we include an additional experiment that does not use any positional encodings (i.e., No PE in Table (a)). We measure the accuracy by calculating the Chamfer distance and f-score. It demonstrates that our method outperforms the previous embedding schemes. We deduce that by using the geometric po- sition priors of point cloud, we can intentionally magnify or suppress the position encoding vectors that are usually beneficial for the control of high-frey embedding 3D spatial domain.
In Table (b), we conduct experiments different composition of voxel re-localization network. Our network involves sparse stacked hourglass network (i.e., S-Hourglass), self-attention layers (ie., Self-attn in Table(]) and cross-attention layers (i.e.. Cross-attn in Table[]. By intentionally omitting certain modules in the voxel re-localization network, we measure the quality of point reconstruction using the Chamfer distance and f f-scores. Despite reasonable results from the voxel generation network (i.e. 1s stage in Table J.. it turns out that self- and cross-attention layers further increase the accuracy of of the point cloud reconstruction. In particular, the combined usage of self- and cross- layers outperforms other strategies. To ensure a fair comparison, the self-attention only method and cross-attention solely network adopts more transformers to balance with our final network design.
So far, we demonstrate the effectiveness of our network design through a fair comparison with previous studies and an extensive ablation studies. Our network aims point cloud reconstruction that can be applied in general cases to achieve high-quality point refinement.
CONCLUSION
Contrary to previous approaches individually addressing the problems of densification and de- noising, we propose an elegant two-stage pipeline to jointly resolves these tasks. As a result of this unified framework, the quality of sparse, noisy and irregular point cloud can be drastically improved. This claim has been validated through a large series of experiments underlying the relevance and effi- ciency of this joint refinement strategy. Specifically, qualitative and quantitative results demonstrate significant improvements in terms of reconstruction's 's accuracy and generalization when compared with prior techniques. Our point cloud reconstruction strategy paves the way towards more flexible, multi-tasks and effective architectures to refine point cloud's quality. Despite this success, there exist remaining issues, such as noise modeling for better denoising and voxet-voxel fusion design.
ACKNOWLEDGGENTT
This work was supported by (1) NAVER LABS Corporation [SSIM: Semantic and scalable in- door mapping], (2) ITTP grant funded by the Korea government(MSIT) (No.2019-0-0199) Artifcial Intelligence Graduate School Program(POSTECHI) and (3) Institute of Information and communi- cations Technology Planning and Evaluation (IIPP) grant funded by the Korea MoVIM (N0.2021-0-02068, Artificial Intelligence Innovation Hub)

---

Published as a conference paper at ICLR 2022
REFERENCES
Henrik Aanæs, Rasmus Ramsbøl Jensen, George Vogiatzis, Engin Tola, and Anders Bjorholm Dahl. Large-scale data for multiple-view stereopsis. International Journal of Computer Vision (IJCV), 120(2):153-1688 2016.
Marc Alexa, Johannes Behr, Daniel Cohen-Or, Shachar Fleishman, David Levin, and Claudio T. Silva. Computing and rendering point set surfaces. IEEE Transactions on visualization and computer graphics, 91):3-15, 2003.
Haim Avron, Andrei Sharf, Chen Greif, and Daniel Cohen-Or. 11-sparse reconstruction of sharp point set surfaces. ACM Transactions of Graphics (ToG), 2010.
Matthew Berger, Andrea Tagliasacchi, Lee M Seversky, Pierre Alliez, Gael Guennebaud, Joshua A Levine, Andrei Sharf, and Claudio Silva. A survey of surface reconstruction from point clouds. In Computer Graphics Forum. Wiley Online Library, 2017.
Fausto Bernardini, Joshua Mittleman, Holly Rushmeier, Claudio Silva, and Gabriel Taubin. The ball-pivoting algorithm for surface reconstruction. IEEE transactions on visualization and com- puter graphics, 5(4):349-359, 1999.
Jean-Daniel Boissonnat. Geometric structures for three-dimensional shape representation. ACM Transactions of Graphics (ToG), 1984.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song. Hao Su, al. Shapenet: An information-rict 3d 2015.
model repository. arXiv preprint
Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo matching network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Jaesung Choe, Kyungdon Joo, Francois Rameau, Gyumin Shim, and In So Kweon Seg- ment2regress: Monocular 3d vehicle localization in two stages. In Robotics: Science and Systems, 2019a.
Jaesung Choe, Kyungdon Joo, Francois Rameau, Gyumin Shim, and In So Kweon. Seg- ment2regress: Monocular 3d vehicle localization in two stages. In Robotics: Science and Systems, 2019b.
Jaesung Choe, Sunghoon Im, Francois Rameau, Minjun Kang, and In So Kweon. Volumefusion: Deep depth fusion for 3d scene reconstruction. Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 16086-16095, October 2021a.
Jaesung Choe, Kyungdon Joo, Tooba Imtiaz, and In So Kweon. Volumetric propagation network: Stereo-lidar fusion for long-range depth estimation. IEEE Robotics and Automation Letters, 6(3); 4672 0672,4679, 2021b. doi: 10.1109/RA..20211.6667
Jaesung Choe, Kyungdon Joo, Francois Rameau, and In So Kweon. Stereo object matching network. In IEEE International Conference Robotics and Automation (ICRA), pp. 12918-12924, 2021c. doi: 10. 00.100//44448
Jaesung Choe, Chunghyun Park, Francois Rameau, Jaesik Park, and In So Kweon. Pointmixer: Mip-mixer for point cloud understanding. arXiv preprint arXiv:2111.11188 2021d.
Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Christopher Choy, Wei Dong, and Vladlen Koltun. Deep global registration. In Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition (C (CVPR), 2020.
Brian Curless and Mare Levoy. A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 303-312, 1996.
10

---

Published as a conference paper at ICLR 2022
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 5828-5839, 2017.
Herbert Edelsbrunner and Emst P Mücke. Three-dimensional alpha shapes. ACM Transactions of Graphics (ToG), 1994.
Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object recon- struction from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Benjamin Graham and Laurens van der Maaten. Submanifold sparse convolutional networks. arXiv preprint 1706v:77060// 07700.0037,, 2017.
Yulan Guo, Hanyun Wang. Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for 3d point clouds: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence (PAMI), 2020.
Jun Jonno Gwak, Christopher Choy, and Silvio Savarese. Generative sparse detection networks for 3d single-shot object detection. In European Conference on Computer Vision (ECCV), 2020.
Ankur Handa, Thomas Whelan, John McDonald, and Andrew Davison. benchmark for rgb-d visual odometry, 3d reconstruction and slam. In IEEE Int. Conf. on Robotics and Automation. (ICRA), 2014.
Yisheng He, Wei Sun, Haibin Huang, Jianran Liu, Haoqiang Fan, and Jian Sun. Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Yisheng He, Haibin Huang, Haoqiang Fan, Qifeng Chen, and Jian Sun. Ffb6d: A full flow bidi- rectional fusion network for 6d pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Pedro Hermosilla, Tobias Ritschel, and Timo Ropinski. Total denoising: Unsupervised learning of 3d point cloud cleaning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. Surface re- construction from unorganized points. In Proceedings of the 19th annual conference computer graphics and interactive techniques, pp. 71-78, 1992.
Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Zhang, Edge-aware point set resampling. ACM Transactions of Graphics (ToG), 2013.
Zitian Huang, Yikuan Yu, Jiawen Xu, Feng Ni, and Xinyi Le. Pf-net: Point fractal network for 3d point cloud completion. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Sunghoon Im, Hae-Gon Jeon, Stephen Lin, and In Kweon. Dpsnet: End-to-end deep plane sweep stereo. In International Conference Learning Representations (ICLR), 2019.
Rasmus Jensen, Anders Dahl, George Vogiatzis, Engin Tola, and Henrik Aanæs. Large scale multi- view stereopsis evaluation. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
Tao Ju, Frank Losasso, Scott Schaefer, and Joe Warren. Dual contouring of hermite data. In Pro- ceedings of the 29th annual conference on Computer graphics and interactive techniques, pp- 339-346. 346, 2002.
Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. Poisson surface reconstruction. In Pro- ceedings of the fourth Furographics symposium Geometry processing, volume 7, 2006.
11

---

Published as a conference paper at ICLR 2022
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6990, 2014.
Arno Knapitsch, Jaesik Park, Qian- Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 2017.
Ravikrishna Kolluri, Jonathan Richard Shewchuk, and James F O'Brien. Spectral surface recon- struction from noisy point clouds. In Proceedings of the 2004 Eurographics/ACM SIGGRAPII symposium on Geometry processing, pp. 11-21, 2004.
Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang. and Oscar Beijbom. Point- pillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12697-12705, 2019.
In-Kwon Lee. Curve reconstruction from unorganized points. Computer aided geometric design, 17(2):111-77,, 2000.
Ruihui Li. Xianzhi Li, Pheng-Ann Heng, and Chi-Wing Fu. Point cloud upsampling via disentan- gled refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. In Advances Neural Information Processing Systems (Neurips), 2020.
William Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. ACM siggraph computer graphics, 21(4):163-169 1987.
Shitong Luo and Wei Hu. Score-based point cloud denoising. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), 2021.
Jiageng Mao, Xiaogang Wang, and Hongsheng Li. Interpolated convolutional networks for 3d point cloud understanding. In Proceedings of the IEEE/CVE International Conference on Computer Vision (ICCV), 2019.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), 2020.
Alejandro Newell, Kaiyu Yang. and Jia Deng. Stacked hourglass networks for human pose estima- tion. In European Conference on Computer Vision (ECCV), 2016.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
Francesca Pistilli, Giulia Fracastoro, Diego Valsesia, and Enrico Magli. Learning graph- convolutional representations for point cloud denoising. In European Conference on Computer Vision (ECCV), 2020.
Charles Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb-d data. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2018.
Shi Qiu, Saeed Anwar, and Nick Barnes. Pu-transformer: Point cloud upsampling transformer. arXiv preprint 1rXiv:21 2021.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In Int. Conf. on Machine Learning. (ICLM), 2019.
Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy Mitra, and Maks Ovsjanikov. Pointcleannet: Learning to denoise and remove outliers from dense point clouds. In Computer Graphics Forum. Wiley Online Library, 2020.
12

---

Published as a conference paper at ICLR 2022
Marie-Julie Rakotosaona, Paul Guerrero, Noam Aigerman, Niloy J Mitra, and Maks Ovsjanikov. Learning delaunay surface elements for mesh reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2021.
Sameera Ramasinghe and Simon Lucey. Learning positional embeddings for coordinate-mlps. arXiv preprint arXiv:21 2021.
Riccardo Roveri, A Cengiz Öztireli, loana Pandele, and Markus Gross. Pointpronets: Consolidation of point clouds with convolutional neural networks. Computer Graphics Forum. Wiley Online Library, 2018.
Nicholas Sharp and Maks Ovsjanikov. Pointtrinet: Learned triangulation of 3d point sets. In Euro- pean Conference on Computer Vision (ECCV), pp. 762-778. Springer, 2020.
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng
Li. Pv-renn: Point-voxel feature set abstraction for 3d object detection. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 10529-10538, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems (Neurips), 2017.
Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, and Silvio Savarese. Densefusion: 6d object pose estimation by iterative dense fusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3343-3352, 2019.
Xin Wen, Peng Xiang, Zhizhong Han, Yan-Pei Cao. Pengfei Wan, Wen Zheng, and Yu-Shen Liu. Pmp-net: Point cloud completion by learning multi-step point moving paths. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Chong Xiang, Charles R Qi, and Bo Li. Generating 3d adversarial point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, and Zhizhong Han. Snowflakenet: Point cloud completion by snowflake point deconvolution with skip-transformer. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), 2021.
Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Real-time 3d object detection from point clouds. Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 7652-7660, 2018.
Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. Mvsnet: Depth inference for unstruc- tured nulti-view stereo. In European Conference Computer Vision (ECCV), 2018.
Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (ToG), 35(6):1-12, 2016.
Wang Yifan, Shihao WU, Hui Huang, Daniel Cohen-Or, and Olga Sorkine-Homung. Patch-based progressive 3d point set upsampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger. Pseudo-lidar++. Accurate depth for 3d object detection in autonomous driving. International Conference on Learning Representations (ICLR), 2019.
Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Ec-net: edge- aware point set consolidation network. In European Conference on Computer Vision (ECCV), 201 10.
13

---

Published as a conference paper at ICLR 2022
Lequan Yu, Xianzhi Li, Chi-Wing Wiin Fu, Daniel Cohen-Or, and Pheng-Amn Heng. Pu-net: Point cloud upsampling network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018b.
Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. Pen: Point completion network. In 2018 International Conference on 3D Vision (3DV), 2018.
Cem Yuksel. Sample elimination for generating poisson disk sample sets. In Computer Graphics Forum, 2015.
Wenxiao Zhang, Qingan Yan, and Chunxia Xiao. Detail preserved point cloud completion via separated feature aggregation. In European Conference Computer Vision (ECCV), 2020.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Tor, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
Qian- Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3d: A modern library for 3d data processing.
arXiv preprint arXiv: 1801 00947,,099477 2018.
14

---

Published as a conference paper at ICLR 2022
APPENDIX
In this appendix. we describe further details of the proposed methodology, deep point cloud re- construction. First, we provide the additional quantitative results (Sec. A). Second, describe
precise equations of the metrics that we used for evaluation (Sec. B). Third, we present qualitative comparisons of ours and baseline methods (Sec. C
A QUANTITATIVE RESULTS
Along with Table of the manuscript, we provide additional results that we did not include in the manuscript.
ShapeSet-Par Part dallset Chamf. (L)
Methods
Fb
Fax
A--1
X-22
kmm4
Aue.
Peroenaae ( 0.5)
Compp.
|-ecorc
Anc.
Percenrage ())( LO)
Comp.
J-Scree
(rl)
2015
8192
1.14
1.29
1.51
72.28
42.71
31.31
92.41
88,48
911.16
(red)
1.56
26.91
Ours (00200))
2018
8732
1.19
1.33
1.52
81.02
40.41
53.48
96.95
$1,23
88.08
PL (1-8)
2048
16384
1.01
1.13
1.31
70.89
00.06
64.17
92.16
93.56
92.76
(r=8) PD
2048
16384
1.25
1.34
1.49
61.11
42,98
49.74
91.73
81.01
85.83
(p-4)
PU (y=d)
1.48
49.72
84.67
PU (r-8)
2048
16384
1.02
1.14
1.32
80.84
50.36
60:74
95.69
$2.21
88.05
PC (1=8) PD
2048
16384
1.29
1.39
1.56
71.79
43.88
53.57
94.68
75.41
83..55
(r-4)
PC
2048
1.51
1.59
$7.63
83.13
89.47
85.97
Ours (.0150)
15863
0.90
69.01
78.14
97.11
94.98
96.16
SeaniNet dotasee
Methods
P.
[=]
Ncarestt Chant. (1)
X=2
[=]
Acc.
Perceniage (f) 0.50
Comp.
Percentage (T)
Comp.
/-scom:
(r-8) PC (r=8) - PD
41.19
4098
32768
3.61
3.84
4.19
24.58
10.33
13.82
55.21
38.18
44.47
PI (p-8)
4095
32768
3.20
3.49
3.92
21.52
9,323
11.94
48.
36,10
39.72
PD
32768
3.01
3.29
3.71
30.24
18.04
63.36
42.15
49.68
Ours #000000)
70.49
73.24
Ours 000055))
1095
11022
1.73
1.86
2.05
63.67
38.55
17.77
91.03
70.67
79.36
RC (m-16)
40%
66536
3.79
4,18
4.48
16.91
11.96
13.12
40.45
53.70
45.45
(r=16) PD
4098
65536
3.71
3.93
4,23
21.93
15.05
17.33
51.57
47.38
49.12
(+=
42.24
(r=16)
4096
65536
3,68
3.94
4.32
16.81
13,48
13.90
37.89
44.60
40.04
PU (r-16) PD
4098
65536
2,86
3.10
3.44
26.06
21.04
23.36
$5.73
$3.56
$4.04
PL (r-4) (C())) PD
4096
2,66
2,86
3.14
22.80
23.30
21.86
49.64
64,11
$5.40
(r=()
$4,70
Ours (1m.
4096
55188
57.88
68.34
76.72
Ours (1w--000003
4096
76182
2.17
2.32
2.32
31.55
34.78
41.16
84.77
65.28
73.37
Ours Man -0.0000
4095
116265
2,86
3.02
3.26
40.42
30.24
34.08
76.37
58.12
65.61
KCNNUM dasaset
Methods
F7
Plat
[=]
1=2
Chamt.
[=]]
Acc.
Percentage (C)
Comp.
/-scure
0.53
Acc.
Percentage (*)
Comp.
(-score
PC (1=8)
4088
32768
3.32
3.59
3.97
21.61
12.28
13.87
44.32
50.T8
45.9%
PC (m-88)
32766
28.84
48.97
(r-8)
4096
32768
3.86
14.16
PU (r=3)) - PD
4008
32768
2.85
3.09
3.45
32.34
20.03
22.75
61.38
51.12
54, 96
Oun -0.0100)
4096
21395
1.50
1.61
1.79
75.81
48.13
58.44
92.99
74.57
N2.40
C -0.0080)
4096
30986
1.81
2,00
71.65
55.74
72.36
(r=10)
4038
65536
3:41
3.63
3.95
18.69
17.53
16.55
38.26
63.02
47:200
PC (r=16) PD
1098
65536
3.18
3.34
3.59
21.16
20.58
21.53
50.98
$7.43
53.84
PC (r=() (U PD
4096
3.61
3.81
4.11
21.62
11.11
14.05
45.34
422.57
43.32
PU (r=16)
65536
4.33
(U
1058
65536
2.50
3.00
3.29
27.16
27.11
26.18
53.15
60.50
56.26
PC PD
4096
3.61
3.94
4.42
27.55
8.64
12.42
54.42
33.5%
40.60
PU (U PD -+ PC (r=4)
4096
4.00
4.21
4.53
10.00
11.64
9.51
22.70
48.52
30.25
(l...000007)) Ours (lu.)) (066))
67.78
.006
56902
2.14
2.26
2.45
63. 40
42.85
80.26
86.84
67.78
75.41
Ours =0,05000000)
4096
99321
2.78
2.92
3.10
$4.11
38.09
43.55
81.76
61.88
69.45
Table 3: Additional quantitative results. We measure the quality of point reconstruction from ours, point completion (PC) (Xiang et al. 2021). point denoising (PD) (Luo & Huj 2021) and point upsampling (PU) (Li et al.) (021) using typical criteria: chamfer distance (Fan al. 2017) and percentage metric that consists of accuracy (Acc.) and completeness (Comp.) and f f-sore (Jensen al. 2014). For fair comparison, we test various conditions by changing the number of input point cloud P... the upsample ratio r, and voxel size 1 Note that identical colors mean the results from the same pre-trained weights for each method.
15

---

Published as a conference paper at ICLR 2022
B METRICS
B.I PERCENTAGE METRICS
Let describe the precise equation of the metrics that used in Table T of the manuscript and 3f the appendix. Let Par be ground truth point cloud, Preed a reconstructed points. For each inferred point pred its distance ground truth defined as:
dp-Par- min |pp gl12
(6)
REPar
where g is single point included the ground truth point cloud Por These distances are gathered to calculate Accuracy (i.e., Acc.) as:
100
Accuracy(danesh) =
Σ
Idp-PPor Sdhresh)
(7)
No prod PEP
where NY is the number of reconstructed points Preed [.] is the Iverson bracket, and dirrss is threshold distance. In similar manner, calculate the distance from ground truth point g to the reconstructed points Ppred as:
dg Ag-PPps PEP
min |/g pll2
(8)
These distances gathered calculate Completeness (i.e., Comp.)
100
Completeness (dunest) = Nr
Σ [dg datresh]
(9)
REPor
where Nperr is the number of ground truth point clouds. To calculate fl-score, Accuracy and Completeness as below:
f1 (--scor(ddirres)) 2 Accuracy(ddneed Completeness( (ditees))))))
(10)
Accuracy (ditresh) + Completeness/ (dunes)))
Note that ShapeNet dataset, we ddtreshh0.00m)) For ScanNet and ICL-NUIM dataset, we set dihresh -0.000000 0.0050 for evaluation. We mostly follow these metrics from (Jensen et al. 2014
Aanæs
al.
2016 Yao al. 2018 Knapitsch al. 2017).
B.2
CHAMFER DISTANCE
Before we explain the k-Nearest Chamfer distance, let us revisit the original Chamfer distance. Based on the equations Eq. 8 and Eq. 6 we describe the Chamfer distance (i.e., Chamf.) as:
(dp+Pen) + 1
Chamf.
Σ (dg-Ppm):
(11)
N
Na
PEPpol pood
gEPar
Since the original Chamfer distance only considers the closest distances, has difficulty in describ- ing the optimal distance between two point sets having different number of points (Fan al. 2017). However, rather than to use Earth Mover distance that requires quadratic computational power and memory consumption, we formulate k-Nearest Chamfer distance by considering the closest points instead of the single closest point (i.e., k=1). Let us define the distance from inferred point p pred the closest ground truth points E&T as follow:
1
P Por
Σ
gx
(12)
where gi is the k-th closest point to the inferred point p. Similarly, we calculate the distance from a ground truth point g to the k-closest reconstructed points ps as follows:
lg 1
(13)
P pra
Based on these equations, compute k-Nearest Chamfer distance (i.e., k-Chamf.) below:
k--Chamf. NNpps pePpe
Σ (1p + No REPor
Σ
(at (
(14)
16

---

Published as a conference paper at ICLR 2022
C
QUALITATIVE RESULTS
Color image
Input point cloud"
Ours
Ground truth
Color image
Input point cloud
Ours
Ground truth
Color image
Input point cloud
Ours
Ground truth
Color image
Input point cloud
Ours
Ground truth
Color image
Input point cloud
Ours
Ground truth
Color image
Input point cloud
Ours
Ground truth
Figure 7: Point cloud reconstruction results. Note that point cloud has been colorized for visu- alization purpose.
17


Reviews:
[{"review_id": "gpXV3qOMID", "reviewer": "Reviewer_dx4y", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The problem of point cloud reconstruction combines several existing important challenges in point cloud processing. The paper is well written and easy to follow, however, some details remain unclear and the evaluation choices inconclusive. \n\nMajor issues: \n\n\n- Since many scans are combined into a single scan, there is another challenge that is entirely overlooked - misalignment. If the idea is to combine all challenges into one, this should be another challange to address.  \n- Missing related work - There is a very large body of recent works on neural shape implicit representation which focuses on the surface reconstruction task. While I understand how this is a bit different from the proposed point cloud reconstruction, one could simply sample points on the zero level set and achieve point cloud reconstruction. Among these works are DeepSDF, Occupancy networks, SAL, SALD, DPDist, IGR,  Convolutional occupancy networks, PHASE, SIREN, DiGS, etc. These are missing particularly since in section 3.2 it is claimed that the local 3D shape is estimated using neighbour voxels. \n- Approach - in the described approach it was not clear to me how the final number of points is specified. Also, not clear how color information is determined for the newly generated points\n- Evaluation baselines: I find the choice of the baseline questionable. First, there was no unified approach used (denoising + completion + sampling). Second, It would make more sense to me to compare the proposed method on the other's task since it is expected that a denoising method will not do well for upsampling. Third, in the combined baselines (Snowflake-PC->Score-PD and Dis-PU->scorePD) it is not clear the order of application (it makes sense to first denoise and then upsample/complete but the text suggests the opposite which should also be reported). Finally, as mentioned in the related work comment, the paper lacks comparison to implicit representations surface reconstruction. \n- Evaluation datasets - The chosen datasets are commonly used and an understandable choice for this task. However, they are strongly biased towards planar geometries. For example, most shapes in ShapeNet and most scenes in ScanNet have large planes. I expected to see results on the Surface Reconstruction Benchmark (Berger et. al.) which is small but has a mix of complex geometries. \n-Evaluation - time and memory requirements are missing. From the implementation details it seems to be resource-heavy, how does that compare to the other baselines? \n- Lack of insight - The paper is missing an important subsection that provides some insight into what the different modules have learned and how is that beneficial to the task. This raises some question about novelty - how is this approach better than the sum of its technical parts? \n- In its current form, the paper is not self-contained and the method is not reproducible due to a lack of technical details. \n\nMinor issues: \n - There is an issue with the term \"continuous 3D points\" which repeats itself in the paper (section 3.2 for example). Point clouds are not continuous, they are sampled on a continuous surface. \n- Evaluation: The evaluation metrics were not properly specified. When performing standardized experiments it may be sometimes acceptable to commit this, however, in these case, the paper proposes a new task, therefore, the evaluation measures should be specified. \n- $V_{mid}$ is missing from Fig 2 and Fig3. One can only assume where it is in the architecture. It should be better specified.\n- Section 2 -> Point cloud upsampling \"input points input points\" typo. \n\n", "summary_of_the_paper": "The paper proposes a new method to jointly address the tasks of point cloud denoising, completion and upsampling. They call this joint problem \"Point cloud reconstruction\". The method consists of two main modules - a voxel generation module (to increase voxel density and remove outliers) implemented as an hourglass sparse convolutional neural network and a point relocalization module (to convert the discrete voxels back to point clouds) implemented using a transformer network. The results show good performance in the chosen metrics and datasets. ", "summary_of_the_review": "In summary, I find the paper well written and the problem interesting. The approach is well presented and technically sound for the most part. However, due to the lack of a significant body of works in the related work section as well as the missing evaluations I can not recommend this paper for acceptance at this stage. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "WOIHHU8tvw", "reviewer": "Reviewer_G4Aq", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper mainly has two strengths:\nFirst, a two-stage pipeline is innovatively proposed, which can enable the network to solve multiple tasks such as point cloud denoising, completion and reconstruction at one time, and the reconstruction accuracy and visualization results of this method can surpass the latest methods in various sub fields.\nSecond, the innovative use of amplified positive encoding and transformers to compute the relationship between a center voxel and its neighbor voxels has certain enlightening significance for the follow-up work.\n\nBut at the same time, this paper also has some shortcomings.\nFrom the experimental results, ShapeNet-Part dataset is comparable to SOTA results, and the generalization ability of ScanNet and ICL-NUIM datasets is significantly better than other methods, but the paper only compares the experimental results. It would be better if the paper could explain why the proposed new method can achieve such strong generalization ability, and why the generalization ability of the existing methods is poor.\n", "summary_of_the_paper": "This paper proposes an elegant two-stage pipeline to jointly solve the three tasks of point cloud densification, denoising and reconstruction at one time. Voxel densification and denoising are performed using a 3D spark stacked hourglass network, and then voxels are reconstructed into dense point clouds using transformers. The experimental results of ShapeNet exceed most existing methods, and have strong generalization ability.", "summary_of_the_review": "This paper creatively integrates different tasks such as point cloud denoising, completion and reconstruction into a two-stage task, and the generalization ability of the experimental results on ScanNet and ICL-NUIM datasets is much better than the latest existing methods. This paper creatively uses amplified positive encoding to compute the relationship between a center voxel and its neighbor voxels, which has certain enlightening significance for the follow-up work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "21EtpYnR0s", "reviewer": "Reviewer_veKS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Note: I'll use the the word \"multi-task\" to refer to the problem setting in this work, in contrast to the single-task baselines. I understand that \"multi-task\" slightly abused here as one may argue that the the studied problem is a unified single task rather than three independent ones. \n\nStrengths:\n1. The high-level idea makes lots of sense to me. Using voxelization to denoise and complete irregular point cloud is reasonable and smart. Besides, we can sample as many points as we want from a voxel grid to get a dense surface. I'm not 100% sure whether this idea has been studied before as I'm not an expert for these two tasks. If possible, the authors could further elaborate on this contribution.\n2. The empirical results are strong. The proposed framework greatly out-performs single task baselines. \n3. Writing is mostly clear and proper ablation has been conducted.\n\nWeakness:\n1. The presented results are great, but the baselines are not very considerate. This paper is arguing that the three tasks should be unified and hence multi-task joint-learning baselines should be considered. The simplest setting is to ensemble all three baselines together in an optimized order. Alternatives contain training each baseline methods on all three tasks jointly. \n2. Following above point, the proposed method should also be evaluated for each single tasks to demonstrate advantages. For example,  I would be only curious about how good is the proposed framework for point completion if my point cloud isn't noisy. Does this method improved over specified point-completion network?  \n3. I'm not fully convinced by the story of positional encoding. In the original form, the distance information is already expressed as the frequency of the signal. The magnitude is just redundant information. Why is it important? And what kind of effect it will cause precisely?  \n4. Lots of details are unclear. I list several that confuses me most: \n    - How many voxels are processed in the transformer on average? Since the complexity of the attention function is $O(N^2)$, does the proposed framework suffer from big memory consumption?\n    - For the tested dataset, especially the synthetic ones, how does the input noise/sparsity/incompleteness are created? ScanNet has lots of incomplete objects (eg., chair with no legs) but I don't find any qualitative results showing that the network can complete them, which is unsatisfactory. Also, regular synthetic noise (eg.,  Gaussian noise) are much simpler than real-world noise. How does the proposed framework deal with real-world noise? For example, MVS reconstructed point-clouds. Similar thing also holds for sparsity. It's unclear how does this framework work for real-world sparse point clouds (eg., the remote areas in Lidar data). If the authors think these are out-of-the-scope, then please clearly introduce the problem setting and limitations.\n   - How is the dimension of $P_out$ decided? I thought arbitrary number of points can be generated. If this is true, how are the numbers in Tab.1 decided? I found these values of $P_out$ are confusing and they are not consistent with prior work.\n\nMisc\n1. The difference between sparse (point up-sampling) and incomplete (point-completion) is not very clear. Very specific definition should be made in the very early stage of the writing so that there is no ambiguity between the challenges that these two problems pose. Besides, the experimental results should clearly demonstrate how does the proposed framework address each sub-problem (above two+de-noising) individually. Unfortunately, these results are either missing or not very clear right now.\n2. If find it mis-leading to term this task \"point-cloud reconstruction\". A maybe improper analogy is that we don't call joint image de-noising and super-resolution and image generation. If the authors really want to use the term reconstruction, maybe call it \"conditioned point cloud reconstruction\"? Or something else that is more accurate like \"point cloud refinement\"? I'm just making examples and throwing out random names. \n3. It's unclear to me what's the motivation of clustering (voxel hashing), and how important is this step? Why do we need a clustering before transformer?  And where are the cluster information used in the later framework? If I understand correctly, this is used because the voxel-to-point step only operates on center voxels and hence we want a locally augmented feature representation?\n\nMinor issues:\n1. Fig.2 input figure is barely visible. A better color-map should be used. \n2. 3D sparse Hourglass network is not a super popular network. It's also not implemented in popular sparse conv net framework (eg., Minkowski engine). Why this framework is adopted? Maybe the authors can share some related work to justify the advantage. It's not a common choice.\n\nSuggestions:\n1. In Tab.1, maybe the light color can be used to high-light the 2nd best among the baseline method? High-lighting the 2nd best among all including \"ours\" cannot fully demonstrate the advantages.", "summary_of_the_paper": "This paper presents a unified framework for doing point cloud upsampling, denoising, and completion jointly. The proposed framework contains two modules: a point-to-voxel auto-encoder, and a Point Re-localization transformer. It outperforms single-task baselines on multiple benchmarks by a great margin. ", "summary_of_the_review": "Summary: My current rating is mainly based on two reasons: 1) The experimental results are strong and the idea of utilizing voxelization for de-noising/refining/densify-ing point clouds makes lots of sense to me. In addition, the proposed transformer works well for further improving the results and recovering points from voxels. 2) Even though the numbers are great, I do think the experiments can be improved. Multi-task baselines should be devised and compared. It's not enough to only demonstrate the proposed method out-performs single-task baselines. Moreover, fine-grained experiments should also be conducted to help us understand the performance on each single tasks. \n\nI'll also use this section to justify my ratings below. This paper is mainly proposing a new problem setting and develops a new framework correspondingly.  Technical wise, this new framework doesn't carry significant new components. It does a great job choosing the proper sub-modules,  combining and slightly improving them for the final tasks. However, most of these components existed in the literature. Empirical wise, the results in Tab.1 is exciting and the proposed framework greatly improves over singe-task baselines. More strictly speaking, single tasks and two-task (X--> score-PD) baselines. The qualitative results in Fig.5 are also encouraging and clearly show the advantages. \n\nI have worked in the field of point-cloud recognition and pretty familiar with the recognition frameworks.  However, I'm not an expert of point-cloud de-noising/completion/densify-ing, and not very familiar with the SOTA methods in these fields. \n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1eRDAMb-B0q", "reviewer": "Reviewer_1Mes", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n\nStrengths:-\n1. The paper brings a new perspective solving two old problems in computer vision, and this is useful. I like the idea it is clean and simple.\n2. The two-stage novel architecture is simple, but apart from the additional positional embedding I don’t there is any other addition here. But the stage1 and stage2 architectures have been used in the past to solve problems. My understanding is that the network majorly befits from stage1 as a lot of qualitative results concentrate on completion.\n3. The numbers are better and change with voxel resolution but significantly better than the baseline approaches.\n4. Qualitative examples are good, and show the performance improvement but are limited, and would like to see more of them in the supplemental (which is missing)\n\n\n\n\n\nWeakness:-\n1. This paper needs a quite few presentation changes to make the idea clear and easy flowing. The initial section of the method section of the paper is up until 3.1 is clear and well explained while 3.2 needs a better explanation. I still do not understand the exact intuition behind the amplitude-based positional embedding and logic is not very clear. \n2. The baselines are not explained well at all, I’m not sure if the comparison to the baselines is fair because the exact characteristics of the Snowflake-PC → Score-PD are not explained, and likewise for Dis-PU. These sections need rewriting.\n3. There is no evaluation independently for Stage-1 all evaluations in the paper use Stage-1 and Stage-2. I think to justify the need for Stage-2 and see its impact it is important to set up some evaluation criteria for Stage-1 only. I understand that maybe doing the full evaluation on Stage-1 is not possible. \n4. Reiterating a point I mentioned earlier → More qualitative examples are needed.\n5. The paper claims like past work when they upsample they don’t have this scaling-up factor of $r$ but I think in this case this $r$ just manifests as the voxel density in stage 1.\n6. I’m also not clear what happens when multiple voxel centers move to the same location in 3D points.\n\n\n\n\n", "summary_of_the_paper": "The paper proposes an end-to-end unified approach to solve the subtasks in point cloud completion. The main selling point of the paper is that combine two existing methods for point cloud densification  & denoising (Stage 1) and point cloud completion (Stage 2). The paper does extensive experiments on 3 datasets and compares to 3 different baselines. They achieve SOTA performance on these datasets. The use of sparse convolution to process point clouds is quite interesting as an autoencoder (termed as hour glass) aligns with the task at hand.", "summary_of_the_review": "\n\nJustification:-\nI think the paper does interesting contributions but lacks extensive qualitative results and the “Snowflake-PC → Score-PD” and similar things make me wonder if things are fair. The performance gap between the method and baselines is quite huge and I’m not able to completely understand the exact difference in impact between Stage 1 and Stage 2. Depending upon the rebuttal by the authors I will change my rating upwards if given reasonable explanations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a unified framework for point cloud upsampling, denoising, and completion through a two-stage approach. It receives three reviews with three leaning to accept and one leaning to reject. Most of the reviewers like the proposed two-stage approach for its simplicity and demonstrated strong performance. The reviewer recommending marginally below the acceptance threshold expresses concerns about missing comparison to neural shape implicit representation and a lack of insights on what is learned by individual layers in the network. While the meta-reviewer agrees that having both would make the paper stronger, the meta-reviewer feels the paper has enough merit and would like to recommend its acceptance.
Decision: Accept (Poster)