D EEP P OINT C LOUD R ECONSTRUCTION
A BSTRACT
Point cloud obtained from 3D scanning is often sparse, noisy, and irregular.
To cope with these issues, recent studies have been separately conducted to densify, denoise, and complete inaccurate point cloud.
In this paper, we advocate that jointly solving these tasks leads to signiﬁcant improvement for point cloud reconstruction .
To this end, we propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse stacked-hourglass network as for the initial densiﬁcation and denoising, 2) a reﬁnement via transformers converting the discrete voxels into 3D points.
In particular, we further improve the performance of transformer by a newly proposed module called ampliﬁed positional encoding .
This module has been designed to differently amplify the magnitude of positional encoding vectors based on the points’ distances for adaptive reﬁnements.
Extensive experiments demonstrate that our network achieves state-of-the-art performance among the recent studies in the ScanNet, ICL-NUIM, and ShapeNetPart datasets.
Moreover, we underline the ability of our network to generalize toward real-world and unmet scenes.
I NTRODUCTION
[Figure]
We propose a novel neural architecture that jointly solves inherent shortcomings in raw point cloud, such as noise, sparsity, and incompleteness.
3D scanning devices, such as LiDAR and RGB-D sensors, allows to quickly and accurately reconstruct a scene as a 3D point cloud.
This compact 3D representation is commonly used to achieve various tasks in autonomous driving (Yang et al., 2018;
Qi et al., 2018; Shi et al., 2020; Lang et al., 2019; Choe et al., 2021c;b; 2019b), robotics (He et al., 2021; 2020; Wang et al., 2019), or 3D mapping (Choy et al., 2020; Choe et al., 2021a;d).
However, processing raw point cloud happens to be particularly challenging due to their sparsity, irregularity and sensitivity to noise (Xiang et al., 2019; Guo et al., 2020).
To address these issues, recent deep learning-based studies have proposed to improve the quality of point cloud.
Speciﬁcally, prior works that are dedicated to point cloud reﬁnement can be categorized into three distinct subﬁelds: (1) point upsampling (Yu et al., 2018b), (2) point denoising (Rakotosaona et al., 2020), and (3) point completion
(Yuan et al., 2018).
Each of these tasks has its own beneﬁts and inconveniences.
For instance, the point upsampling task usually handles noisy input points but do not explicitly deal with strong outliers.
On the other hand, point denoising techniques have been designed to ﬁlter noisy data but cannot densify the point cloud.
Finally, point completion methods mainly focus on object-level full 3D completion but are known for their weak generalization to unmet environments or unknown category of objects.
Though there exist differences among the tasks, their common goal is to improve proximity-to-surface in the 3D point representation.
Thus, we believe in the complementary of these three sub-tasks and hypothesize that a joint framework is greatly beneﬁcial to deal with deteriorated and incomplete 3D point cloud.
Accordingly, this paper challenges to resolve all these issues as a single task, called point cloud reconstruction as depicted in Fig.
To the best of our knowledge, this paper is the ﬁrst attempt to jointly resolve the inherent shortcomings of point cloud obtained from 3D scanning devices: sparsity, noise, irregularity, and outliers.
To this end, we propose a deep point cloud reconstruction network that consists in two stages: a voxel generation network and a voxel re-localization network.
In the ﬁrst stage, the voxel generation network (Sec. 3.1) aims to densify voxels and remove outliers via sparse convolution layer (Choy et al., 2019).
Rather than to use k -Nearest Neighbor that is sensitive to points’ density (Mao et al., 2019), we utilize voxel hashing with sparse convolution layer to understand absolute-scale 3D structures for densiﬁcation and denoising.
Despite its success, voxelization inevitably leads to information loss due to the discretization process.
To provide a ﬁne-grained reconstruction, in the 2 nd stage, we propose a voxel re-localization network that converts discrete voxels into 3D points using transformer.
Additionally, we increase the performance of transformer by the ampliﬁed positional encoding .
In our analysis, spatial frequency plays an important role in the context of voxel-to-point conversion.
Our new positional encoding strategy is useful to infer descriptive and detailed point cloud by simply changing the amplitude of the encoding vector.
Our contributions can be summarized as follows:
• New problem formulation: point cloud reconstruction.
• Novel two-stage architecture for voxel-to-point reﬁnement.
Unique embedding scheme for point cloud, ampliﬁed positional encoding.
• A large number of experiments illustrating the generalization ability of our point cloud reconstruction to real 3D scans.
R ELATED W ORKS
Point cloud obtained from 3D scanning devices are known to contain various artifacts (Berger et al., 2017), such as noise, outliers, irregularity, and sparsity.
Point cloud reconstruction aims at resolving these aforementioned issues in order to provide ﬁne-grained 3D reconstruction.
In this section, we introduce different approaches related to point cloud reﬁnement.
Point Cloud Denoising.
Without extra information, such as RGB images, point cloud denoising purely based on the input point distribution is a challenging task (Lee, 2000; Avron et al., 2010).
Recent deep learning-based strategies (Rakotosaona et al., 2020; Roveri et al., 2018; Hermosilla et al., 2019; Pistilli et al., 2020;
Luo & Hu, 2021) demonstrate promising results.
However, despite the large improvement provided by these architectures, these methods do not have the ability to densify the reconstruction.
Point Cloud Completion.
A family of solutions called point cloud completion have recently been proposed to alleviate the problem of incomplete reconstruction.
One of the pioneer papers (Yuan et al., 2018) proposes to overcome the point incompleteness problem by generating points in both non-observed and visible areas.
To improve the completion process, recent approaches (Huang et al., 2020; Zhang et al., 2020; Wen et al., 2021; Xiang et al., 2021) propose to include prior structural information of the objects.
While these object-speciﬁc point completion techniques can lead to accurate results, they suffer from a lack of versatility since the object class is assumed to be known beforehand.
Without such semantic level a priori, the entire completion in occluded surface ( e.g ., behind objects) is a doubtful task.
Thus, in this paper, we focus on a reliable and generic object-agnostic approach that is able to complete visible areas within an arbitrary point cloud and to cope with sparse, noisy, and unordered point cloud.
Point Cloud Upsampling.
Given a sparse 3D reconstruction, the goal of point cloud upsampling is to densify the point cloud distribution.
Conventional methods (Alexa et al., 2003; Huang et al., 2013) rely on point clustering techniques to re-sample points within the 3D space.
Recent deep learning-based studies (Li et al., 2021; Yu et al., 2018a; Yifan et al., 2019; Yu et al., 2018b) and concurrent study Qiu et al. (2021) propose point densiﬁcation networks under the supervision of dense ground truth points.
Though some papers internally conduct point denoising (Yu et al., 2018a; Yifan et al. 2019), most of these works mainly assume that there are no strong outliers among the input points, such as ﬂying-point cloud (You et al., 2019) [Figure]
Surface Reconstruction.
Point cloud meshing is a common solution for surface reconstruction (Hoppe et al., 1992; Curless & Levoy, 1996).
For instance, this process can be achieved by converting point cloud to an implicit surface representation via signed distance function (SDF) and to apply Marching Cubes (Lorensen & Cline, 1987) or Dual Contouring (Ju et al., 2002) for meshing ( e.g ., Poisson surface reconstruction (Kazhdan et al., 2006)).
This strategies is known to be effective under favorable conditions but can hardly handle sparse and non-uniform points’ distribution.
Moreover, such technique tends to lead to over-smoothed and undetailed reconstruction (Berger et al., 2017).
Similar limitations can be observed with recent deep learning-based SDF representation (Park et al., 2019).
Many other solution have been developed for the problem of surface reconstruction, such as, Delaunay triangulation (Rakotosaona et al., 2021; Boissonnat, 1984; Kolluri et al., 2004), alpha shapes (Edelsbrunner & M ¨ ucke, 1994), or ball pivoting (Bernardini et al., 1999).
Despite their compelling performances, these approaches require uniform, dense, and accurate inputs since they usually preserve the original distribution of a given point cloud.
This assumption still holds for the recent studies (Sharp & Ovsjanikov, 2020; Rakotosaona et al., 2021).
Compared to surface reconstruction, point cloud reconstruction can be viewed as a prior-process to overcome natural artifacts: sparsity, noise, irregularity.
Furthermore, in contrast to previous studies about point cloud reconstruction, our method aims to perform object-agnostic and well generalized point cloud reconstruction that is robust to overall shortcomings in the real-world 3D point cloud.
To this end, we introduce our deep point cloud reconstruction network that consists of the voxel generation network and the voxel re-localization network.
D EEP P OINT C LOUD R ECONSTRUCTION
Given a noisy and sparse point cloud P in , point cloud reconstruction aims to generate a dense and accurate set of points P out .
For this purpose, we propose a deep point cloud reconstruction network composed of two stages: voxel generation network (Sec. 3.1) and voxel re-localization network (Sec. 3.2).
The ﬁrst network has been designed to perform a densiﬁcation and outliers removal in the sparse voxel space.
The second network complements and improves this ﬁrst stage by denoising the data further and by converting voxels back to 3D points.
The overall architecture is illustrated in Fig.
3. V OXEL G ENERATION N ETWORK
Most previous studies (Li et al., 2021; Yifan et al., 2019; Yu et al., 2018a;
Luo & Hu, 2021) attempts to resolve these problems by directly processing a raw point cloud.
However, it is known to be a challenging problem due to the unordered nature of the 3D point cloud.
To tackle this issue, we propose to convert the 3D point cloud P in into a sparse voxel representation V in through our voxel generation network.
From this data, our network generates a reﬁned and densiﬁed volume V out .
A noticeable advantage of sparse voxel representation is its ability to preserve the neighbor connectivity of point cloud in a uniform and regular 3D grids (Guo et al., 2020) via voxel hashing (Choy et al., 2019; Graham & van der Maaten, 2017).
Voxel hashing keeps tracking of the geometric positions of sparse tensors so that we can ﬁnd the voxels’ neighbors to apply sparse convolution.
Moreover, sparse convolution layer can cover the consistent and absolute-scale scope of the local region and hierarchically capture its large receptive ﬁelds.
Such property is then particularly beneﬁcial to deal with an inaccurate and unordered input point cloud P in .
[Figure] + + + + + + + + + + + + 𝑉 out 𝑉 in C 𝐹 v 𝑉 mid In order to denoise and densify V in , our voxel generation network consists of a sparse stacked hourglass network as shown in Fig.
Our network architecture is close to the previous studies (Newell et al., 2016; Chang & Chen, 2018; Im et al., 2019;
Choe et al., 2019a) in that it is composed of several hourglass networks for cascaded reﬁnements of images or dense volumes.
In the context of this paper, scanning devices can only reconstruct the surface of the structure, thus, the resulting voxelized volume is inherently sparse.
Therefore, to process this 3D volume efﬁciently, our voxel generation network takes advantage of the sparse convolution network (Choy et al., 2019).
It should be noted that the speciﬁcity of sparse convolution operation is particularly desirable for the tasks at hand.
First, generative sparse transposed convolution layer (Gwak et al., 2020) has the ability to create new voxels given a pre-deﬁned upsampling rate r , which can be seen as a point upsampling methodology (Yu et al., 2018b).
Secondly, voxel pruning is widely used for ﬁne-grained volume rendering (Liu et al., 2020) and efﬁcient computation (Choy et al., 2019).
In our context, voxel pruning is closely related to outlier removal.
Last, the joint use of generation and pruning enables our network to create an arbitrary number of points.
As illustrated in Fig.
3, we ﬁrst compute local voxel features using sparse convolutional blocks in an encoder part.
The aggregated voxel features undergo the voxel generation layers and pruning layers in a decoder part.
We use two hourglass networks to densify and prune voxels.
To train the voxel generation network, we adopt a Binary Cross-Entropy loss (BCE) to classify the status of estimated voxels into an occupied/empty state as:
[Equation]
where ˆ y v is an estimated class of a generated voxel v from the network, and y v is the ground truth class at the generated voxel v .
N v is the total number of voxels that we used to compute the BCE loss.
If v is on the 3D surface, y v becomes true class ( i.e ., y v =1 ).
Furthermore, following the original loss design (Newell et al., 2016), we compute the BCE Loss L BCE for both the ﬁnal voxel prediction V out and the intermediate results V mid .
In total, we compute the voxel loss L vox as:
[Equation]
where N v is the total number of voxels in the intermediate inference volume V mid that we used to calculate the BCE loss.
Along with the predicted voxels V out , we extract voxel features F v using sparse interpolation as illustrated in Fig.
3. V OXEL R E - LOCALIZATION N ETWORK
Despite the denoising and densiﬁcation offered by the 1 st stage network, voxelization inevitably leads to information loss.
Thus, in this section, we propose a voxel re-localization network that converts the discrete voxels V out ∈ R N × 3 into 3D points P out ∈ R N × 3 .
This conversion requires an understanding of local geometry to describe the 3D surface as a group of point sets.
Inspired by (Zhao et al., 2021; Choe et al., 2021d), we utilize self-attention as a point set operator.
Let us describe the detailed process in the voxel re-localization network, which is illustrated in Fig.
Given output voxels V out , we collect the K (=8) closest voxels to each voxel v i ∈ R 1 × 3 using the hash table that we used in the 1 st stage network.
Then, we obtain a voxel set V i
= { v k } K k =1 that [Figure] Query voxel : Neighbor voxel [Figure]
Amplified PE [Figure] is referenced to the query voxel v i . Similarly, we cluster a set of voxel features F i = { f k } Kk =1 that correspond to P i .
As in Fig. 4, the stage network regresses the location of a query point p
i (red voxel) using the geometric relation between the query voxel v i and its neighbor V i (white through self/cross attention layers.
While this formulation is close to PointTransformer (Zhao et al., 2021), our voxel re-localization network has three distinctive differences.
First, PointTransformer utilizes k -Nearest Neighbors to group 3D points but we re-use the voxel hashing that covers absolute-scale local geometry.
Second, our network consists of both self-attention and cross-attention.
In our analysis, it is better to use two different attention layer than to solely adopt self-attention.
Third, we design our own positional encoding system that controls the necessity of high-frequency signal by changing the amplitude of the encoding vector, called ampliﬁed positional encoding .
Originally, positional encoding has been introduced to help transformer to understand the orders of tokens (Vaswani et al., 2017).
When applied to other domains of applications, positional encoding is often used to preserve high-frequency signals which tend to be disregarded by neural networks (Rahaman et al., 2019; Ramasinghe & Lucey, 2021).
For example, positional encoding is proven to be essential to synthesize sharp images from an implicit 3D neural representation (NeRF (Mildenhall et al., 2020)).
In our purpose, we need to consider spatial frequency to coherently re-locate a query voxel v i by using its neighbor set V i .
Our intuition is simple.
Since positional encodings are mainly effective for the high-frequency embedding.
As in Fig.
4, we increase the amplitude of positional encoding when we need to move a query voxel by a small amount of distance ( i.e ., high-frequency in 3D spatial domain).
If a query voxel is far away from its neighbor, we decrease the amplitude of the positional encoding.
Based on the relation between high frequency signals and our voxel re-location scheme, we deﬁne our ampliﬁed positional encoding γ ampPE as:
[Equation]
where γ ampPE is an ampliﬁed positional encoding vectors and (cid:107) · (cid:107) 11 is the L1-norm.
Following the conventions (Vaswani et al., 2017), γ PE is 1D positional encoding vectors, pos represents position, i means dimension, and C is channel dimensions of voxel embedding.
After we encode γ ampPE into the voxel embedding, we aggregate this embedding through self- and cross-attention layers to capture local structures around the query voxel.
Finally, the regressed offset ∆ v is added to the original location of query voxel v c to update its location.
To train the second stage network, we calculate a chamfer distance loss L dist as:
[Equation]
where P pred is the predicted point cloud, P gt indicates ground truth point cloud, ˜ p represents a predicted point, and p means a ground truth point.
N P pred and N P gt are the number of prediction and ground truth point cloud, respectively.
Through our two-stage pipelines, our whole network performs point cloud reconstruction.
By jointly generating, removing, and reﬁning points, our network can robustly recover accurate point cloud and increase the level of proximity-to-surface.
E XPERIMENT
In this section, we describe the implementation details of our network and we present a large series of assessments underlying the effectiveness of our approach.
Our point reconstruction network has been solely trained on the ShapeNet-Part (Yi et al., 2016) dataset but tested on other real and synthetic datasets such as ScanNet (Dai et al., 2017) and ICL-NUIM (Handa et al., 2014).
These experiments highlight the ability of our solution to generalize well to other scenarios involving various levels of noise and different densities.
4. I MPLEMENTATION
We ﬁrst train our voxel generation network for 10 epochs using the Adam optimizer (Kingma & Ba, 2014) with initial learning of 1 e - 3 and a batch size of 4.
We decrease the learning rate by half for every 2 epochs.
After that, we freeze the voxel generation network and initiate the training of the voxel re-localization using the inferred voxels V out from the voxel generation network.
The point relocation network is trained using identical hyper-parameters used for training the ﬁrst stage network.
Further details regarding the training rules are provided in Sec. 4.3.
4. B ASELINE A PPROACHES
We conduct a thorough comparison against previous state-of-the-art techniques: PU for point cloud upsampling (Li et al., 2021), PD for point cloud denoising (Luo & Hu, 2021), and PC for point cloud completion (Xiang et al., 2021).
These three papers are currently positioned at the highest rank for each task and we utilize their ofﬁcial implementation for training and evaluation.
all methods are trained with the same data augmentation as competing approaches and its performances have been measured on various datasets: ShapeNet-part (Yi et al., 2016), ScanNet (Dai et al., 2017), and
ICL-NUIM (Handa et al., 2014) datasets.
In particular, we analyze the efﬁcacy of ours compared to the combination of the existing studies.
For instance in Table 1, PC( r =4 ) → PU( r =4 ) → PD represents consecutive operation of point completion, and point upsampling and point denoising under upsampling ratio r =4 .
Note that we also take into account the point completion task since this task also considers point densiﬁcation on visible surface.
4. D ATASET ShapeNet dataset.
We utilize 13 categories of different objects categories from the ShapeNetPart dataset (Chang et al., 2015; Yi et al., 2016).
We extract dense and uniform 10 K ground truth point cloud per object by using the Poisson Disk Sampling algorithm
(Yuksel, 2015) provided by
Open3D (Zhou et al., 2018).
Among these ground truth 3D points, we randomly sample 2048 points as an input P in for the training networks.
For the sake of fairness, we adjust length of unit voxel as l vox =0 . for training our network.
This is to produce a comparable number of points than previous studies.
We follow a data augmentation scheme proposed by the previous point upsampling study (Li et al., 2021), such as random noise addition and random re-scaling.
Additionally, we include random outliers that constitute less that 5 percent of input points.
To train and validate the networks, we carefully follow the ofﬁcial train/val/test split provided by (Yi et al., 2016).
ScanNet / ICL-NUIM dataset.
These datasets (Dai et al., 2017; Handa et al., 2014) are indoor datasets that provide RGB-D sequences and corresponding camera poses.
We utilize these datasets to evaluate the generalization capability of ours and previous methods.
Similar to the image-based reconstruction study (Yao et al., 2018), we ﬁrstly re-use the network parameters pre-trained in
ShapeNet dataset, then evaluate all methods in unmet datasets without ﬁne-tuning networks.
Further details are included in the supplementary material.
[Table]
4. C OMPARISON
For metric computation [Table] we adopt Chamfer distance and percentage metrics ( e
[Table]
g ., Accuracy, Completeness, and f -score)
(Yao et al., 2018; Jensen et al., 2014), as shown in Table 1.
In particular, we extend to measure the k -Nearest Chamfer distance ( i [Table] e ., k -Chamf. 1 ) that efﬁciently computes point-wise distance than Earth Mover’s distance.
Also, this formulation counts more number of the nearest points than the original Chamfer distance ( i.e ., k =1 ).
Based on these criteria, we calculate the metrics of the whole methods
[Table] As shown in Table [Table] our network achieves state-of-the-art performance compared to the previous studies (Li et al., 2021;
Luo & Hu, 2021; Xiang et al., 2021).
Moreover, the performance gap increases when we conduct generalization test in ScanNet dataset and ICL-NUIM dataset.
There are two dominant reasons behind the better generalization and transferability.
First, each sparse convolution layer has consistent and absolute-scale receptive ﬁelds in 3D space.
This local voxel neighborhood appears to be more robust than raw point cloud processing via k -Nearest Neighbor as adopted in the three baselines (Xiang et al., 2021;
Luo & Hu, 2021; Li et al., 2021).
This is also one of the strength of using voxelization as an initial processing of point cloud reconstruction, which is conducted by our 1 st network.
Second, our network does not have task-speciﬁc header networks or losses for completion/upsampling/denoising tasks.
Instead, Our network purely focuses on the inherent problems resided in raw point cloud.
Such joint approach trains our network to improve the overall proximity-to-surface, not the task-speciﬁc quality.
Regardless the order in which we apply the baselines, our method systematically outperforms them.
When we sequentially apply point densiﬁcation and point denoising, the performances are not constantly improved in Table 1.
This observation is supported by the qualitative results in Fig.
While point denoising can effectively remove the remaining outliers that are generated from (Xiang et al., 2021) or (Li et al., 2021), it has chance to remove the wrong points sets.
For instance in Fig.
4, the point denoising method erases part of the airplane’s wing.
In contrast, our voxel generation network conducts both voxel pruning and generation in each hourglass network so that it can effectively circumvent such case. [Figure] [Figure]
[Table]
A BLATION STUDY
In this section [Table] we propose to evaluate the contribution of each proposed component (ampliﬁed positional encoding and voxel re-localization network), through an extensive ablation study in Table [Table]
In Table 2-(a)
[Table]
we validate our ampliﬁed positional encoding in comparison with different styles of positional encodings, γ PE (Vaswani et al., 2017) and PE* (Mildenhall et al., 2020).
Also, we include an additional experiment that does not use any positional encodings ( i
[Table]
e ., No PE in Table 2-(a)).
We measure the accuracy by calculating the Chamfer distance and f -score.
It demonstrates that our method outperforms the previous embedding schemes.
We deduce that by using the geometric position priors of point cloud, we can intentionally magnify or suppress the position encoding vectors that are usually beneﬁcial for the control of high-frequency embedding in 3D spatial domain.
In Table 2-(b), we conduct experiments on different composition of voxel re-localization network.
Our network involves sparse stacked hourglass network ( i.e ., S-Hourglass), self-attention layers ( i.e .,
Self-attn in Table 1), and cross-attention layers ( i.e ., Cross-attn in Table 1).
By intentionally omitting certain modules in the voxel re-localization network, we measure the quality of point reconstruction using the Chamfer distance and f -scores.
Despite reasonable results from the voxel generation network ( i.e . 1 st stage in Table 1), it turns out that self- and cross-attention layers further increase the accuracy of of the point cloud reconstruction.
In particular, the combined usage of self- and crosslayers outperforms other strategies.
To ensure a fair comparison, the self-attention only method and cross-attention solely network adopts more transformers to balance with our ﬁnal network design.
So far, we demonstrate the effectiveness of our network design through a fair comparison with previous studies and an extensive ablation studies.
Our network aims at point cloud reconstruction that can be applied in general cases to achieve high-quality point reﬁnement.
C ONCLUSION
Contrary to previous approaches – individually addressing the problems of densiﬁcation and denoising, we propose an elegant two-stage pipeline to jointly resolves these tasks.
As a result of this uniﬁed framework, the quality of sparse, noisy and irregular point cloud can be drastically improved.
This claim has been validated through a large series of experiments underlying the relevance and efﬁciency of this joint reﬁnement strategy.
Speciﬁcally, qualitative and quantitative results demonstrate signiﬁcant improvements in terms of reconstruction’s accuracy and generalization when compared with prior techniques.
Our point cloud reconstruction strategy paves the way towards more ﬂexible, multi-tasks and effective architectures to reﬁne point cloud’s quality.
Despite our success, there exist remaining issues, such as noise modeling for better denoising and point-voxel fusion design.
A CKNOWLEDGMENTS
This work was supported by (1) NAVER LABS Corporation [SSIM: Semantic and scalable indoor mapping], (2) IITP grant funded by the Korea government(MSIT) (No.2019-0-01906, Artifcial Intelligence Graduate School Program(POSTECH)) and (3) Institute of Information and communications Technology Planning and Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2021-0-02068, Artiﬁcial Intelligence Innovation Hub)
A PPENDIX
In this appendix, we describe further details of the proposed methodology, deep point cloud reconstruction.
First, we provide the additional quantitative results (Sec. A).
Second, we describe precise equations of the metrics that we used for evaluation (Sec. B).
Third, we present qualitative comparisons of ours and baseline methods (Sec. C) [Table]
A Q UANTITATIVE RESULTS
Along with Table 1 of the manuscript
[Table]
we provide additional results that we did not include in the manuscript.
[Table]
B M ETRICS B. P ERCENTAGE METRICS
Let us describe the precise equation of the metrics that we used in Table 1 of the manuscript and
Table 3 of the appendix.
Let P GT be ground truth point cloud, P pred a reconstructed points.
For each inferred point p ∈P pred , its distance to ground truth is deﬁned as:
[Equation]
where g is a single point included in the ground truth point cloud P GT .
These distances are gathered to calculate Accuracy ( i.e ., Acc.) as:
[Equation]
where N P pred is the number of reconstructed points P pred , [ · ] is the Iverson bracket, and d thresh is a threshold distance.
In a similar manner, we calculate the distance from ground truth point g ∈P GT to the reconstructed points P pred as:
[Equation]
These distances are gathered to calculate Completeness ( i.e ., Comp.)
as:
[Equation]
where N P GT is the number of ground truth point clouds.
To calculate f1-score, we use Accuracy and
Completeness as below:
[Equation]
Note that in ShapeNet dataset, we set d thresh =0 . 0200 .
For ScanNet and ICL-NUIM dataset, we set d thresh =0 .
0050 for evaluation.
We mostly follow these metrics from (Jensen et al., 2014; Aanæs et al., 2016; Yao et al., 2018; Knapitsch et al., 2017).
B. C HAMFER DISTANCE
Before we explain the k -Nearest Chamfer distance , let us revisit the original Chamfer distance.
Based on the equations Eq. 8 and Eq. 6, we describe the Chamfer distance ( i.e ., Chamf.)
as:
[Equation]
Since the original Chamfer distance only considers the closest distances, it has difﬁculty in describing the optimal distance between two point sets having different number of points (Fan et al., 2017).
However, rather than to use Earth Mover distance that requires quadratic computational power and memory consumption, we formulate k -Nearest Chamfer distance by considering the k closest points instead of the single closest point ( i.e ., k =1 ).
Let us deﬁne the distance from a inferred point p ∈P pred to the k closest ground truth points g k ∈P GT as follow: [Equation]
[Equation]
where g k is the k -th closest point to the inferred point p .
Similarly, we calculate the distance from a ground truth point g to the k -closest reconstructed points p k as follows:
[Equation]
Based on these equations, we compute k -Nearest Chamfer distance ( i.e ., k -Chamf.)
as below:
[Equation]
pred ) .
C Q UALITATIVE RESULTS
[-First of all, we found that the total number of citations of stacked hourglass network is 3364 (Newell et al., 2016b) and 
-]
[Figure]