Sample-Based Point Cloud Decoder Networks
Authors: Erich Merrill
Alan Fern
Venue:  (2020)
Keywords: point cloud
autoencoder

Abstract: Point clouds are a flexible and ubiquitous way to represent 3D objects with arbitrary resolution and precision. Previous work has shown that adapting encoder networks to match the semantics of their input point clouds can significantly improve their effectiveness over naive feedforward alternatives. However, the vast majority of work on point-cloud decoders are still based on fully-connected networks that map shape representations to a fixed number of output points. In this work, we investigate decoder architectures that more closely match the semantics of variable sized point clouds. Specifically, we study sample-based point-cloud decoders that map a shape representation to a point feature distribution, allowing an arbitrary number of sampled features to be transformed into individual output points. We develop three sample-based decoder architectures and compare their performance to each other and show their improved effectiveness over feedforward architectures. In addition, we investigate the learned distributions to gain insight into the output transformation. Our work is available as an extensible software platform to reproduce these results and serve as a baseline for future work.

Under review as a conference paper at ICLR 2020
SAMPLE-BASED POINT CLOUD DECODER NETWORKS
Anonymous authors
Paper under double-blind review
ABSTRACT
Point clouds are flexible and ubiquitous way to represent 3D objects with ar- bitrary resolution and precision. Previous work has shown that adapting encoder networks to match the semantics of their input point clouds can significantly im- prove their effectiveness over naive feedforward alternatives. However, the vast majority of work point-cloud decoders still based on fully-connected net- works that map shape representations to a fixed number of output points. In this work, we investigate decoder architectures that more closely match the semantics of variable sized point clouds. Specifically, we study sample-base maple-bbsed point-cloud point feature distribution, allowing
decoders that map shape representation an arbitrary number of sampled features to be transformed into individual output points. We develop three sample-based decoder architectures and compare their performance to each other and show their improved effectiveness over feedfor- ward architectures. In addition, we investigate the learned distributions to gain insight into the output transformation. Our work is available as an extensible soft- ware platform to reproduce these results and serve as a baseline for future work.
INTRODUCTION
Point clouds are an important data type for deep learning algorithms support. They are commonly used to represent point samples of some underlying object. More generally, the points may be extended beyond 3D space capture additional information about multi-sets of individual objects from some class. The key distinction between point clouds and the more typical tensor data types that the information content invariant to the ordering of points. This implies that the spatial relationships among points is not explicitly captured via the indexing structure of inputs and outputs. Thus, standard convolutional architectures, which leverage such indexing structure to support spatial generalization, are not directly applicable.
common approach to processing point clouds with deep networks is voxelization, where point clouds are represented by one more occupancy grid tensors (Zhou & Tuzel 2018), Wu et al. (2018)). The grids encode the spatial dimensions of the points in the tensor indexing structure, which allows for the direct application of convolutional architectures. This voxelization approach, however, is not appropriate in many use cases. In particular, the size of the voxelized representation depends on the spatial extent of the point cloud relative the spatial resolution needed to make the necessary spatial distinctions (such as distinguishing between different objects in LIDAR data). In many cases, the required resolution will be unknown or result in enormous tensors, which can go beyond the practical space and time constraints of an application. This motivates the goal of developing architectures that support processing point cloud data directly, so that processing scales with the number of points rather than the required size of an occupancy grid.
One naive approach, which scales linearly in the size of the point cloud, 'flatten' the point cloud into an arbitrarily ordered list. The list can then be directly processed by standard convolutional or fully-connected (MLP) architectures directly. This approach, however, has at least two problems. First, the indexing order in the list carries no meaningful information, while the networks do not encode this as a prior. Thus, the networks must learn to generalize in a way that is invariant to ordering, which can be data inefficient. Second, in some applications, it is useful for point clouds to consist of varying numbers of points, while still representing the same underlying objects. However, the number of points that can be consumed by the naive feedforward architecture is fixed.

---

Under review as a conference paper at ICLR 2020
PointNet (Qi et al., 2017) and Deepsets Zaheer et al. (2017) exhibit better performance over the MLP baseline with a smaller network by independently transforming each point into a high- -ishnsinnnional representation with single shared MLP that is identically applied to each individual point. This set of derived point features is then mapped to a single, fixed-sized dense shape representation using symmetric reduction function. As such the architectures naturally scale to any number of input points and order invariance is built in as an architectural bias, As a result, these architectures have been shown to yield significant advantages in applications in which point clouds are used as input, such as shape classification.
The success of PointNet and DeepSet style architectures in this domain shows that designing a network architecture to match the semantics of a point cloud results in a more efficient, and better performing network. Since point clouds are such useful object representation, it's natural to ask how we should design networks to decode point clouds from some provided shape representation. This would allow for the construction of point cloud auto-encoders, which could serve number of applications, such as anomaly detection and noise smoothing. Surprisingly, the dominant approach to designing such a differentiable point cloud decoder is to feed the dense representation of the desired object through single feedforward MLP whose result is then reshaped into the appropriate size for the desired point cloud. This approach has similar issues as the flat MLP approach to encoding point clouds; the decoder can only produce fixed-sized point cloud while point clouds are capable of representing objects at low or high levels of detail; the decoder only learns a single deterministic mapping from a shape representation a point cloud while we know that point clouds are inherently random samples of the underlying object.
The primary goal and contribution of this paper is to study how to apply the same lessons learned from the PointNet encoder's 's semantic congruence with point clouds to a point cloud decoder design. As such, we build on PointNet's principles to present the 'NoiseLeam' algorithm- a novel, simple, and effective point cloud decoding approach. The simplicity of the decoding architectures and the increase in performance are strong indicators that sample-hased decoders should be considered as default in future studies and systems. In addition, we investigate the operation of the decoders to gain insight into how the output point clouds are generated from a latent shape representation.
2 RELATED WORK
Point cloud decoders are relatively unexplored area of research. Among the works which describe an algorithm that produces point cloud, the majority focus their efforts on learning a useful latent shape representation that is then passed to a MLP decoder.
PU-Net (Yu et al. 2018) is one such example, in which they design a novel point cloud upsampling network which uses a hierarchical approach to aggregating and expanding point features into meaningful latent shape representation. To decode the learned shape representation into a point cloud, the latent vector is then passed through feedforward MLP to produce fixed number of points. This implies that the network would need to be retrained to allow for a different upsampling rate, which unlikely to be desired property of an upsampling algorithm.
TopNet (Tehapmi et al. 2019) recognizes the data inefficiency of using a single MLP to decode a point cloud and instead reorganizes their MLP into hierarchical tree structure which MLPs the same level share the same parameters. Their results show that addressing this inefficiency allows for better performance with smaller parameter count. Similarly, in "Learning Localized Generative Models for 3D Point Clouds via Graph Convolution" Valsesia et al. (2019) augments their decoder by assuming a graph structure over the decoded point cloud and employing graph convolutions. However, despite improved performance neither approach addresses the other issues that come with using MI.Ps to decode entire point clouds, namely the fixed-size output.
"Point Cloud GAN" (Li et al., 2018) and PointFlow (Yang et al.) 2019) take a different approach to producing a point set in a generative setting. Instead of learning a single mapping from any latent vector directly to its decoded point cloud, they learn function parameterized by the latent vector which transforms cow-dimensional Gaussian noise to a 3D point on the surface of the object described by the latent shape representation. This sampling based approach is more in line with the semantics of point clouds. First, an arbitrary number of points can be drawn from the Gaussian noise produce a point cloud consisting of that number of points without requiring any changes
2

---

Under review as a conference paper at ICLR 2020
Surpling
SN
Faantion
Stared Paine
Prenstie
Shaned
MFP
lrpec Cleed
Iedautinn maxc
Fanntion
Suspial
Poiri Fertens
Droale
Park
Laree Stape
(a) Point cloud encoder subnetwork
(b) Point cloud subnetwork
decoder
Figure 1: Diagrams demonstrating the operation of the PointNet style encoder network shared by all architectures evaluate, and the parallel decoder network shared by all the sampling approaches.
to retraining of the algorithm. Second, every individual point is decoded independently and identically, which avoids the data inefficiency issues that come with using MLPs to process set data.
While this sampling approach has several desirable properties and appears promising, it's unclear whether the algorithm is applicable outside of the GAN settings these two papers inhabit, if they re- quire specific bespoke loss functions to be trained effectively, if they capable of outperforming the baseline MI.P approach according to other metrics.
3 POINT-CLOUT DECODING ARCHITECTURES
A point cloud is a set of 3D points C = (pi..... PN). where each E R In general, each p; may have additional auxiliary information associated with via non-spatial dimensions. While all of our architectures easily generalize to include such information, in this paper, we focus on point clouds that exclusively encode shapes without auxiliary information.
A point cloud auto-encoder takes a point cloud C with n points and outputs a point cloud Ć with m points that is intended to represent the shape described by C. While often m, are interested in the general case when n and m may be different, which corresponds to up-scaling or down- scaling C. Each auto-encoder will be comprised of encoder E(C), which takes input point cloud and outputs a latent shape representation h in Rd, and a decoder D(h) which maps latent representation to output point cloud of the appropriate size. Thus, given input point cloud C, the auto-encoder output is given by C (EE(C)).
In this paper, we focus on the Chamfer distance as the measure of auto-encoder quality. Intuitively this loss function measures how well è matches C in terms of the nearest neighbor in C to each point in C and vice versa. Specifically, if dist(p, C) gives the distance between point and the nearest neighbor in point cloud Ć, our loss function is defined by T(C, C) n Epec dist(p, C) + 1 Epec: dist(p. C).
Since the focus of this paper is on point-cloud decoders, all of our architectures use the same point- cloud encoder architecture, while varying the decoder architecture. Below, first overview the common PointNet-style encoder used followed by a description of the four decoders considered in experimental analysis, which include three sample sample-based decoders.
3.1 POINTNET ENCODER ARCHITECTURE
PointNet (Qi et al.) 2017) handles unordered input by recognizing that symmetric function (such as element-wiss wise max or sum) produces the same result regardless of the order of its in- puts. PointNet thus learn single transformation function f that maps individual points to I-dimensional representation and then combines those representations via 9. That is, the latent encoding produced by PointNet for point cloud C {Pı,..., .p. is the / dimensional vector
3

---

Under review as a conference paper at ICLR 2020
As desired, E(C) is invariant to the ordering of points in C and applies to any number of points.
E(C) 9
We learn MLP representation of f. with input space R3, encoding points, and output space R,, encoding the latent representation or feature. We use max as the reduction function g to map the arbitrary number of resulting point features to single fixed-size latent shape representation. The hidden layers and size of the latent shape representation for each instantiation of this encoder architecture can be found in Table
3.2 SAMPLE-BASED DECODERS
Most prior work has used MLP decoders, which we consider here as a baseline approach. An MLP decoder is fully connected network that takes the latent shape representation input and outputs an m x 3 output vector, which represents the m output points. Accordingly, MLP decoders are parameterized by the number and size of their fully connected layers. In our experiments. each fully connected layer consists of parameterized ReLU units with a batch normalization layer.
Our main focus is on sample-bass decoders, which allow for an arbitrary number of outputs points to be produced for latent shape representation. In particular, given a latent shape representation h, each of our decoders is defined in terms of point feature sampling distribution 5(h). where the decoder produces point-cloud output by sampling m point features from S(h).
Once we have a set of M independently sampled point features from our sampling distribution S(h) we need to transform each one into triple representing that point's location. Note that we are now in an identical but opposite situation as the point cloud encoder. Whereas the encoder had to trans- form independent point samples of some underlying object into corresponding high-dimensional representations, our decoder now has to transform independently sampled high-dimensional point representations into point in space on the surface of the target object. Therefore, we simply apply the same style of PointNet encoding mechanism with different input and output tensor sizes to implement effective point feature decoder. The sizes of the hidden layers in our decoder network can be seen in Table I By applying the shared MLP point decoder to each sampled point feature, can directly decode point clouds of arbitrary size.
Below we descrihe three architectures for S, which are compared to each other and the baseline MLP decoder in our experiments.
NoiseAppend Decoder. NoiseAppend is similar to the sampling approach described in "Point Cloud GAN" by Li et al. (2018). They sample point features by simply sampling from a multivariate Gaussian distribution with zero mean and unit variance before appending the sampled noise to the latent shape vector. That is. S(h) = concat (O,
However, this requires us to decide how many elements of noise should be appended to the latent shape representation. Li et al. (2018) state that the size of the appended noise vector should be 'much smaller than' the size of the latent shape representation, but it's not clear how much noise is necessary to allow the decoder to fully represent the shape. Ultimately this is an additional hyper- parameter that needs to be investigated and tuned.
No Decoder. NoiseAdd builds on the concept of adding unit Gaussian noise to the latent shape vector with the goal of avoiding the additional hyperparameter that Noise introduces. This can be casily accomplished by treating the entire latent vector as the mean of a Gaussian distribution with unit variance. That is, S(h) = N I).
However, this violates the claim by Li et al. (2018) that the amount of noise introduced to the re- sulting point feature samples should be much smaller than the size of the latent shape representation itself. Therefore, it may be the case that uniformly adding noise to every element of the latent vector obscures the crucial information it represents.
Noise Decoder. Noisel earn attempts to instead learn a small separate function V(h) which predicts the log-variance of the desired point feature distribution. Specifically, S(h) N (h, ( I). We define V ((h)
small MLP, the size of which be seen in Table |
By allowing the network to choose the amount and location of noise to be added to the latent shape vector, we hope that will leamn both to add an appropriate amount of noise for the target shape while
4

---

Under review as a conference paper at ICLR 2020
Sih)
Cancel
(a) NoiseAppend
(b) NoiseAdd
(c) NoiseLearn
Figure 2: Diagrams of the different approaches deriving distribution from the latent shape representation h.
Table 1: Detailed description of architectures evaluated. MLP is not present in the 10k family it cannot be reduced to that parameter count while still producing 1024 points.
Network Archifecture
Emender Hidden
Latent
Appended
V(h)
Deender Hidden
Parameter Count
Hidden
61, 256, 256, 312
1024
1024, 312. 312,
4R33600
AM
NosseAppend
NoiseAdd
64, 64, 256, 256, 512
64,66,255 256, 256, 512
1024
2048.
2048, 512, 512, 256, 256, 128
512, 512. 256.
4868291
4802756
128,33
4573059
ML.P
64. 64, 156 256, 512
256
256. 512, 512,
1739456
NoiseAppend
64, 64, 256,
256, 512
256
256, 512, 256
1675643
2M
NoiseAdd
$12.5112,
1643076
Noisel zam
64,
128,
256
MLP
123.
64
256, 256, 128
545052
NoseAppend
128.
507459
128,
Noiset.com
64, 128, 128
64
512, 512, 256, 12%
503555
MI.
64, 256
64
32.
138048
NoiseAgpend
256, 128, 64
97923
NoiseAdl
256
256, 128, 64
03828
Noisel zom
64. 256
64
256, 178, 64
98115
NossAppeen
32
12595
MoiscAdd
NoiseLeam
64,32
32
(nonc)
64, 64, 16
12639
conserving the information necessary to accurately reconstruct without introducing any additional hyperparameters.
4 EVALUATION
We evaluated each decoding architecture by training several instantiations of each architecture on point cloud auto-encoding problem derived from the ModelNet40 dataset, which consists of over 12,000 3D models of 40 different common object classes. The dataset has a prescribed train/test split, with approximately 9800 models in the training dataset and 2500 in the test dataset. We randomly select 10% of the training data to use for validation during training.
Before training, each object model the ModelNet40 dataset is used to generate uniformly- sampled point cloud with 4096 points which is then scaled to fit within the unit sphere. For all auto- encoder network models, each iteration training, the point clouds are randomly downsampled to 1024 points before being used to update the network parameters. The helps reduce the computational cost of training and also encouraging better generalization. During training, each decoded point cloud consists of 1024 points.
We use the Chamfer distance as the loss function due its relative speed and capability to directly compare point clouds of unequal sizes without modification. Each network is trained for 100 epochs using the ADAM optimizer with an initial learning rate 10-3 where each epoch performs parameter update on each training example. The learning rate is decreased by factor of 10 at epoch 50 and epoch 80. We trained five instantiations of each of the four network architectures with each instantiation varying the number of parameters shown in Table (note that scale down the MLP for the smallest parameter setting). For each instantiation we ran the entire training process 15 times and all results show average performance across the 15 runs.
not able to
5

---

Under review as a conference paper at ICLR 2020
20
MLP Nolselopend Natsends NotseLearn
18
NoloeAppend
NoloeAod NolocLcarn
16
Avcrogo TropgoTccuuuo Uos
14-
12-
10-
5-
20
40
60
80
300
Epoch
Perameter count
Figure 3: Average validation losses of each architecture the 2M family during training.
Figure 4: Test performance of each archi- tecture at different parameter counts. Error bars show the 95% confidence interval of the mean.
All code and infrastructure for "push-button" replication of our experiments open-source (Github/Gittlab location removed for anonymous review code will be privately made available to reviewers through a comment approximately week after submission).
4.1 PERFORMANCE COMPARISON
Quantitative Results. Figure shows the validation loss along the learning curves for the 2M parameter instantiation each architecture. The relative ordering of the architectures consistent after the initial phase of training, with all curves flattening out by 100 epochs.
First, the large jumps in the MLP training (due to unstable training runs) show that it was much less stable to training compared the sample-based architectures. While effort spent trying to specialize the training configuration for the MLP, stability remained an issue [ In contrast the runs for each sample based architecture similarly to worst performing sample-hased architectureby the end of the learning curve.
stable and similar. Ignoring the MLP stability, it performs
The three sample based architectures show rapid progress early in learning and then settle into sistent ordering with NoiseLearn performing best, followed by NoiseAppend, and then NoiseAdd. This suggests that NoiseAdd's approach of adding uniform noise the latent representation may be obscuring information needed for accurate reconstruction. compared to NoiseAppend, which sep- arates noise from the shape representation. On the other hand, we see that while NoiseLearn also adds noise to the latent representation, it is able to outperform Noise This indicates the importance of being able to intelligently select how much noise to add to different components of the representation. Apparently, this allows NoiselLearn to avoid obscuring critical information in the latent representation needed for accurate reconstruction.
Figure shows the average test set performance after 100 epochs of training of each size instantiation of the four architectures (note the log scale). The appendix also shows more detailed results broken down for each of 5 selected object classes. The three sample based architectures show relatively consistent improvement in performance as the sizes grow by orders magnitude. Rather, the MLP shows initial improvement, but then performance decreases significantly past 100K parameters. We looked further into the behavior the MLP architecture for the larger parameter sets. We observed that the larger MLPs showed a similar decrease in performance on even the training data, indicating that the problem is not necessarily overfitting but also difficulty of the optimization. It is possible that with substantially more epochs the MLP performance would improve, but at great cost.
This indicates that the MLP architecture is much less efficient at exploiting larger network sizes than the more structured sample-based architectures. is possible that the architecture and training hyperparameters could be tweaked to improve the large MLP networks' performance, such as by adding additional regularization via weight decay or other mechanisms. However, we consider this tweaking to be outside the scope of this work, and note that none of the sampling based architectures required any such tweaking to achieve competitive performance at all parameter counts.
We attempted to rectify this by adding batch normalization to layers in the MLP decoder to make it more similar to the architecture of the sampling approaches. While this stabilizied training. it also prevented the MI.P architecture from achieving competitive performance, loss than doubled the best architectures'.

---

Under review as a conference paper at ICLR 2020
Figure 5: Examples of networks' auto-encoding results on several previously unseen objects.
Overall the results give strong evidence that the sample-based architectures encode a bias that is much better matched to decoding the class of shapes in ModelNet40 compared MLPs. The struc- tured sample-based architectures, compared to the MLP, result in more stable learning and the ability to continually improve as the architectures grow in size. Further, tecture, which avoids the need to specify hyperparameters to control the amount of noise performs the best, or near best, across all network sizes and number of epochs.
that the NoiseLearn archi-
Jssows Illustrative Qualitative Results. Figure 5 shows each network's performance three test objects not in the training data. The point cloud decoded by the MLP network appears be more evenly distributed spatially, while the sampling- -bsee approaches are better able to capture finer detail in the target shape, such as the stool's thin legs and crossbars. Among the sample-based approaches, no single approach is clearly dominant in terms of visual quality across the objects. It is interesting that all of the sample-based architectures tend miss the same type of object details, e.g. the jets on the plane or the leg cross bars on the chair, which may be due to limitations of the PointNet encoders sized and/or architecture. Nevertheless, quite interesting that single relatively small latent vector representation is able to encode the level of detail exibited in these results.
4.2 NOISE EXAMINATION
Each sampling architecture defines a function from the latent shape representation to point feature distribution. The underlying latent representation inherently defines the manifold of the encoded shape. Rather, the injected noise (either via appending or addition) can be viewed as playing the role of indexing locations the manifold for the generated point. Effectively, the primary difference between the sample-based architectures is how they use the noise to index locations and traverse the manifolds. Below aim better understand this relationship between noise and spatial indexing and how the architectures differ in that respect.
In Figure we demonstrate how each architecture uses noise by controlling the variance introduced to a trained network in two different ways. To examine how the decoder's output is influenced by individual elements of noise we show the output of these networks when all but one of the noise elements is held at constant near-zero value. In the lower plots, we show the decoder's s behavior when only receives the union the network learns to exploit individual elements of noise and how the decoder combines those elements to produce a point cloud that spans the entire shape.
noise elements above. This demonstrates both how
just examine the first five elements of noise in its noise vector. NoiseLearn predicts individual variances for each element in the dense shape encoding, enabling therefore presumably the biggest contribution to the decoded point cloud. The appendix contains additional examples of noise manipulation.
For NoiscAppend all of the noise is of equal magnitude,
to select the five elements of noise with the highest variance, and
The plots shown in Figure 6 give us some insight into how the networks use noise to complete the decoded shape. Each individual element of noise appears correspond to learned path along the

---

Under review as a conference paper at ICLR 2020
4
(a) NoiseAppend
(b) NoiseLearn
Figure 6: Behavior of NoiseAppend and Noisel earn when the noise is manipulated. The ith figure in the top row shows the decoded point clouds when all but the ith variance is set zero. The same figure in the bottom row shows the decoded point clouds when all but the first variances are set to zero. The faint red points show the network's input.
(a) =
(b) N =
N
Figure 8: Contribution of individual channels of noise when the NoiscAppend architecture is modi- fied to only append N elements of noise the shape representation.
surface of the learned shape. The final point cloud then seems to be produced by 'extruding' along those paths.
NoiseLearn's 's use of only four significant elements of noise suggests that in this domain only three four elements of noise is sufficient to achieve good coverage of the target shape. Figure g shows how individual noise channels change when the NoiseAppend architec- ture is modified to only append one, two, and three noise elements.
Assrag Cranter Disunne Leaa
With only one element of noise, we can see that the network ef- fectively has to learn single path that spans much of the target shape as possible. With two elements of noise, the network instead seems to learn individual 'loops' around the object which trans- formed and rotated as necessary. Once the network has access to three elements of noise, networks of learning small paths on the object's 'ss surface.
30'
207
207
Elements nobe appenced
Average final validation loss achieved by NoiseAppend networks when varying the number of elements of noise appended.
Figure
the same behavior the functional
If too little noise seriously hurt NoiseLearn's performance, does adding too much noise do the same? Figure 7 shows the NoiseAppend architecture trained with different amounts of added noise It appears that even when the noise vector is much larger than the dense shape representation, the decoder's overall performance is not impacted. However, note that adding large amounts of noise does significantly increase the parameter count, so there is a nontrivial cost to doing this.
the same performance dropoff present both extremes.
CONCLUSION
In this work, we evaluated and compared several realizations of sample-based point cloud decoder architecture. We show that these sampling approaches are competitive with or outperform the MLP approach while using fewer parameters and providing better functionality. These advantages over the baseline suggest that sample based point cloud decoders should be the default approach when network needs to produce independent point samples of some underlying function object. To further this this area of research, we provide a complete open-source implementation our tools used train and evaluate these networks.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Haoqiang Fan, Hao Su, and Lconidas J Guibas. A point set generation network for 3d object recon- struction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 605-613, 2017.
Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov. Point cloud gan. arXiv preprint 11X::2181. 2018.
Charles Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652-660, 2017.
Lyne P. Tchapmi, Vineet Kosaraju, Hamid Rezatofighi, Ian Reid, and Silvio Savarese. Topnet: Struc- tural point cloud decoder. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Diego Valsesia, Giulia Fracastoro, and Enrico Magli. Learning localized generative models for 3d point clouds via graph convolution. International Conference on Learning Representations, 2019. URL URL httpptts///ppeeeeeeeeee hwtp://wwwwwweeteeeeeeeeeeeeeeee Fot.://ett
Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer. Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud. In 2018 IEEE International Conference Robotics and Automation (ICRA), pp. 1887-1893, IEEE, 2018.
Guandao Yang, Xun Huang, Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In ICCV, 2019.
Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. Pu-net: Point cloud upsampling network. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan R Salakhutdinov, and Alexander J Smola. Deep sets. In Advances in neural information processing systems, pp- 3391-3401, 2017.
Yin Zhou and Oncel Tuzel. Voxelnet: End-to-end learning for point cloud based 3d object detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
A APPENDIX
A. FULL PERFORMANCE TABLES
The tables below show each architecture's average loss each individual class in the ModelNet40 dataset. The best-performing network is bolded for each object class.

---

Under review as a conference paper at ICLR 2020
Table 2: Average loss of 2M family architectures (# stddev) on test dataset
airplane
hathnub
hol
bench
hookshelf
MLP-2m
3.39
5:67
6.42 :: 2.40
NoiseApgend-22
NoiseAdd 2m
4.88
1.10
6.23
1.50
6.05
1.42
5.36
1.72
6,90
1.78
NosseLetam-Lz Zan
297 0.73
5.31
1.30
5.15
1.28
3387
1.27
5,95 + 1.51
MLP-2m
2.69
1.49
6,95
= 3.36
5,96
7.04
2.34
7.02 49.08
NoiseAppend-2 2m
4.13
1.40
10.10
4,46
6.34
1.08
6.78
2.25
5.22 +: 2,62
NoiseAdd-2m
1.70
9.45
2.86
7.46
4.32
curain
desk
door
dresset
MLP-2m
2.00
4.44
5,05 4.77
NoiscAppend-2n
7:13
NoiseAAd-2m
9.08 2.83
1.53
1.24
8.14
2.20
1.13 1: 1.05
1.24 = 1.77
7.63 270
2,85
0.88
6,00
1.70
0.61 2.52
6.32 = 1,48
Nower_pol
guilar
lamp
MI.P-3m
9.15 4.87
5.81
1,70
2.65 0.89
2.18
0.49
13.1
31.22
NoiseAgrend-20
983 3.96
7.16
1.79
2.17
2775
0.68
8.87
11.19
10.0
lapeop
mintel
monator
nightsstund
pesson
Mr.P-2m
3.79
7.29
NoiseAppead-2n
NoiseAdd--mm
4.73
6.78
1.38
6.53
2.08
7.67
1.94
5.08 = 2.00
Noisel Leim-2m
3.25 + 0,50
5.59
1,18
5.33 + 1.75
6.35
1,80
3,87 = 1,38
piano
plant
radio
range-hood
sink
MI.P-3m
10.70 2.85
10.06
5.18
5.73
1.57
7.57
2.84
9.16 4.50
NoiseAprend-20
8.67 2:10
9.87
4.82
5:34 1.90
6:83 1.53
6.85 = 2.09
1.77
1.77
5002
stairs
stool
table
S2nt
ML.P-3m
4.56
NoizeAgpend-
6.60
1.62
8.16 3.29
6.08
2.35
1.32
7.17 1.74
NuiseAdd (odd-ddd--m 2m
6.79
1.68
8.31
3.52
6.78
2.85
A-48
1.59
1.16 + 1.84
Noisel zamm2mm 2m
534
1.39
6,18
2.46
1.71 4.75
3.15
1 1.08
6,15 # 1.72
Inilet
IV stsi
vise
MLP-2m
7.92
6.92
2.56
5.92 3.49
5.53 2.49
6.12 3/03
NoiseAgpend
6.79
7,60 3.75
1.24
6.59 :: 1.46
1,47
Noisel Letmm
6.01
6.25
1:13
5,75 2: 0.92
Table 3: Average loss of 100k family architectures (# stddev) on test dataset
nirplane
bathnub
bed
bench
bookshelf
MLP-JO0K
4,35
1.27
5.70
1.57
6.37
2,06
NotseAppend
1.78
NaiseAdd-110k
3.98 = 1,48
7.69
1.61
7.00
1.40
0.66
2.20
8,06
2.33
NouseLcam- 100k
4.16 = 1.06
5.77
1.49
5.75
+: 1.36
4.91 :: 1.76
6.62 +: 1.94
bootle
MLP-JO0K
2.79
1.45
8.17
3.40
5.61
1.02
0.97
2.12
4.74 2.92
NoiscAppend 100k
3.70
d: 1.57
8,95
4.15
6.39
4: 1.14
7.07 4: 2.22
5.40 4: 3.13
NoiseAdd-1100
4.63
2.49
7.11 4.11
oup
curtain
desk
door
diesser
MLP-100K
7.48
3.60
2.86
2.82
1.38
5.75
1.52
NoiseAppend
9.03
NoiseAds-1O0k
10.00
2.82
4.02
1.30
9.77
4 2.54
3.82
1.16
9.18 + 1.92
NouseLzam- 100k
8.46
=: 2,87
3.12
1.14
$.20
2.45
2.67
+ 0.71
6.71 + 1,02
Bower.pot
glass_box
guiler
keyboord
MLP-J00K
9.10 =: 4.44
6.30
1.53
2.05
0.77
2.55
0.55
15.12 27.38
NoiscAppend 100k
9.65
=: 3.99
7.43
1.87
1.35
4 0.20
2.57
0.62
10.69 4: 13.8%
NoiseAdd-
11.61
0.84
13.45
NoseLLam-
9.27
laptop
mantel
moniloe
night_stand
person
MLP-100K
5.10
0.83
2.13
4.88
NoiseAgpend- 100k
NoiseAdd-1O0k
0.67 3.74
5.02
8.54 1.54
7.61
2.49
9.37
2.37
6.14
2,40
NouseL-am-- 100k
0.51
1.40 6.45
6.09 + 1.98
7.20
1.92
4.79 + 1.95
pinno
plan
hood
ML.--10k
8.70 2 2.19
10.70 5366
5.14 1.50
6.99 =: 1.82
9.18 4.13
NoistAppend- 100k
9.47
2.36
10:25 4: 5.30
5.91 2.11
7.24
4: 1.52
8.06
2.58
NoiseAdd-110k
11.04
5.00
2.0-4:2..5
1.82
NosseL.am-
2.25
1.48
sola
stain
st0ol
table
S0nt
MLP--10k
1.34
122:4:4 4.300
6.25
NoistAppend- 100k
6.72
1.50
4,90 1.80
7.19
NoiveAdd--1nk
7,68 = 1.96
4.33 10:57
8.56
3.57
1010 1.96
8.93 = 2.61
Noiiet-cam-10k 100k
6.43 = 1.51
3,40 9A8
6.30 + 2.45
4,31
166
6.76 + 1.83
Mr.F-100k
8.50 = 1.59
6.84 2.02
5.99 3.33
5.43 :: 1.77
5.59 1.52
NoiseAppend 100k
8.89 1,73
7.14 1.96
7.39 3.55
6.54 4: 1.33
6.81 1.33
NoiseAdd-
340-1:4.00 4.3.6
NoiseT.cam
10

---

Under review as a conference paper at ICLR 2020
Table 4: Average loss of 4M family architectures (# stddev) on test dataset
airplane
baihmb
bed
bench
bookshelf
MLP-Im
5.43
14331.2.1 2.38
10.00
2.81
9.42 1 2.72
10311
1 4.79
9.19
3:94
NolseAppead-4m
NoiseAdd Am
4.00 1.11
1.53 5.30
6.27
6.11
1.43
5,38
1.79
6,89
1.79
NotseLeam 4m
2.98 1. 0.72
1.29
5.11 1: 1.26
3.81
1.26
5.82 1.54
MLP-4m
4.39
2.38
14.06 4.90
9:01
2.77
13:31
3.09
9:45 6.15
NokseAppend-4m
3.3 + 1.40
9.61
3.89
6.39 + 1.12
6.02
2.188
5.32 2.70
NoiseAdd
4.40
vpp
currain
desk
door
dresar
MLP-4m
12007 3:08
15,88
4.61
10.15 1: 3.54
Ao
7.44
NoiscAdd-4in
9.21
285
3.64
1,26
8.35
2:27
3.06
0.82
7.21
# 1.80
Necsel zom-4cn
7.50 + 2.68
2881
0.89
5,90 + 1.64
2.49
0.63
6.27 1.48
glass
MLP-4a
12.55 3.96
10.98
3.75
5:44 2.36
4.24 1.24
24,85
42.31
NokseAppend-im
9.38 + 3.53
7.07
1.79
2.19 = 0.69
2.74
0.62
9.36 = 12.32
NoiscAdd--uu
3.88
2.11
2om--4n
laptop
mantcl
moniur
nightstand
person
Mr.P-4m
4.27
Ar
1.40 6.83
6.29
NoiscAdd-4mt
5.20 1.26
1.44
6.64 + 2.12
7:63
2.02
5.37
1.96
Nocsel 4m
3.27 0.53
5.57
1,18
5,19 + 1.72
1.77 6.24
3,74
1,31
piano
plant
hood
MLP-4m
1419 4.32
[3.97 = 7.85
10.28 1 3.85
13.17 4.07
14.79 = 5.58
NriseAppeni-im
849 + 2.00
9.77
4.76
5.85 + 1.81
6.68 1.43
6.70
2.11
NoiscAdd-44m
5,06
sola
stains
Stool
tbble
12.3
NriseAppemdl-mm
3.08
6.00
NolscAdd-4m
6.81 + 1.72
8.60
3.54
6.34 + 3.35
4.71 1.64
7.24 = 1.99
NoiseLaam-fe
5.77 1.37
2,42 6.20
4,46 1.37
3.16
1.08
5.97 1.38
MLF-4m
16.88 3.08
0..9 4.61
10.03 1: 6.06
9.11 4.15
9:31 5.13
Ariso-ppemi-im
797 + 1.73
6.79
1.74
7.28 +: 3.46
6.19 1.21
6.35 1.34
NotseLeem--fm
6.68 1:
1.63
6.15 2.90
5.64
11

---

Under review as a conference paper at ICLR 2020
A.2 NOISE MANIPULATIONS
Below are more examples of noise manipulation in an autoencoder setting in the style of Figure 6
A
12


Reviews:
[{"review_id": "SkeaYr229H", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper deals with a task of learning latent representations for 3D point clouds by introducing a series of particular point cloud decoders. The advantage of the proposed architectures is their ability of producing variable-size output 3D point cloud. While the work on point-based architectures is relevant to the current agenda in the vision community, the novelty of this paper is somewhat limited, as the proposed architecture is constrained to one class (PointNets) only, while the decoding network could just as easily be used with other encoders (e.g., DGCNN [1]). \n\n\nMajor comments:\n\nOne major shortcoming of the paper is its very limited experimental evaluation, that does not really allow to judge whether the proposed decoder architecture brings the real utility for most problems. For instance, authors only provide results for two distinct experiments: building an auto-encoder and understanding how each architecture uses noise. Much more tasks, such as shape denoising, upsampling, completion, or retrieval could have been explored, too. Only one (simple) dataset is used, i.e. ModelNet, whereas more complex datasets (e.g., ShapeNet, ABC, or datasets related to human shapes) could have been considered.\n\nI wonder why authors didn't compare to (or even cite) [1], which seems to be one of the earliest papers in the area. \n\nOther possible research questions are as follows. What is the effect of the proposed decoder networks on the learned latent representations in the entire architecture? Will the interpolatory and semantic properties of the learned latent spaces be preserved when using the autoencoders? \n\nHaving said that, I believe the paper cannot be accepted in the present form, however, I would change my decision, should more experiments validating the architecture be provided. \n\n\nMinor comments:\n\n* in definition of a point cloud, both n and N are used to indicate its size\n\n\n[1] Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019). Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG), 38(5), 146.\n[2] Achlioptas, P., Diamanti, O., Mitliagkas, I., & Guibas, L. (2018, July). Learning Representations and Generative Models for 3D Point Clouds. In International Conference on Machine Learning (pp. 40-49).", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "Hkx9xqR35B", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary\n\nThis paper studies the problem of 3D point cloud autoencoding in a deep learning setup, and in particular, the choice of the architecture of a 3D point cloud decoder. The main claim of the paper is the adaptation of the PointNet architecture (a set of fully-connected layers applied per point) from encoding to decoding purposes. To achieve that, the authors propose three similar mechanisms to inject noise in the bottleneck shape representation, which are used to sample per point feature vectors suitable for a PointNet-like decoder. These include appending a low dimensional gaussian noise sample to the bottleneck shape representation, adding individual gaussian noise sample to each bottleneck shape representation dimension or using trainable variances per bottleneck shape representation dimension. Resulting networks are optimized by minimizing Chamfer distance between the ground truth and output point clouds. The main motivation of their choice of these architectures is their ability to output an arbitrary size point clouds, compared to naive multilayered fully-connected network applied to a bottleneck shape representation and outputting a \"flattened\" fixed-size point cloud.\n\nTo verify their claims, the authors considered autoencoding task for ModelNet40 dataset. They compared a naive multilayered fully-connected decoder to PointNet-like decoders with all three proposed noise-injection mechanisms. This comparison was also performed for 5 variations of the proposed models differing is network size. To compare quantitatively, the authors present all results graphically, showing the dependency of resulting average Chamfer distance from the network size. This graph shows that using trainable per dimension variance for noise injection works consistently better, than other options; also, it shows inconsistent results for the baseline multilayered fully-connected decoder. In qualitative comparisons, the authors compare output point clouds visually and study the influence of sampling from various noise dimensions on the reconstructed points.\n\n\nReview\n\nAlthough the authors of this paper touch a very interesting topic in generative modeling of point clouds, I do not believe this paper should be published in its current state.\n\nCons:\n1. The main issue in this paper is the absence of comparisons to any external approaches. Even though comparison to their variation of the baseline model is provided, it is impossible to verify where their results stand compared to the related work. There is a whole stream of works in the same task using ShapeNet dataset, which the authors completely omit. Even if the authors do not want to perform experiments on ShapeNet, they, at least, could have provided comparison to AAE and PC-GAN results from PC-GAN paper [1] for generative modeling metrics on ModelNet40. As the authors stated, there are at least two works, PC-GAN and PointFlow, which allow sampling of the arbitrary size point clouds and comparison to these approaches is crucial (especially since the proposed approach is technically significantly simpler compared to hierarchical GANs from PC-GAN and continuous normalizing flows combined with VAEs from PointFlow papers).\n\n2. The paper misses numerous citations to related work in generative 3D shape modeling:\n- voxel grid based, (since the authors mention them): [2, 3, 4, 5, 6, 7]\n- point cloud based: [8, 9, 10]\n- graph based: [11, 12]\nAll these approaches propose various types of decoders for 3D shapes represented accordingly and some of them perform evaluations for pure generative, autoencoding task, which, again, can be used for comparison.\n\n3. I believe, there is a factual mistake and arguable diminishing of the current state of the art in the related work section:\nThe authors state, that both PC-GAN and PointFlow models \"inhabit GAN settings\". However, PointFlow is, in fact, an approximate likelihood model and is considered as an instance of Variational Autoencoder with a shape prior and a point cloud decoder implemented as continuous normalizing flows (a special case of invertible neural networks). The authors also state, that it is \"unclear... if they are capable of outperforming the baseline MLP approach\" when they talk about these papers, while, in fact, both PC-GAN and PointFlow explicitly show that by comparing to [9], which use the multilayered fully-connected baseline decoder. It is also not unclear if these approaches need specific loss functions, contrary to what authors say. They were designed so they need their particular loss functions, which does not disqualify them from the comparison. These papers are the main competitors of this work and should be addressed properly instead of being diminished.\n\nPros:\n1. Besides the cons, the paper is written clearly and is easy to follow.\n\n2. Rigorousness with respect to comparison of the proposed options (Table 1 and the fact that every model was retrained 15 times to obtain average result).\n\n3. Qualitative experiments with control over noise dimensions are novel and interesting.\n\nOverall, this could have been a paper about a nice and simple alternative to PC-GAN and PointFlow for arbitrary size point cloud modeling, but, unfortunately, it missed the opportunity to properly address the current state-of-the-art in its current version.\n\n\nSmall remarks and questions for authors:\n\n1. Citation of Valsesia at al. in the related work should be reorganized to be consistent with other citations. Omitting full names there and in case of PC-GAN will also be more consistent.\n\n2. All figures in the paper should be updated for better visibility of both graphics and text. Figures 1-4, 7 - text is barely visible, the scheme blocks and lines are too thin and overall lack readability. Figures 5-6, 8 - point clouds are almost invisible even with zooming, remove coordinate grid in the background, zoom in on the objects, so they take as much space in the images as possible, use bigger point sizes for better visibility.\n\n3. Maybe I do not follow something, why have you ignored applications of your point cloud decoders, and particularly generative modeling task and its evaluation protocols, presented for example in [9]?\n\n\n[1] C.-L. Li, M. Zaheer, Y. Zhang, B. Póczos and R. Salakhutdinov. Point Cloud GAN. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2019\n[2] C. Choy, D. Xu, J.-Y. Gwak, K. Chen, and S. Savarese. 3D-R2N2: A unified approach for single and multi-view 3d object reconstruction. In ECCV, 2016.\n[3] R. Girdhar, D. Fouhey, M. Rodriguez, and A. Gupta. Learning a predictable and gen- erative vector representation for objects. In ECCV, 2016.\n[4] J. Wu, C. Zhang, T. Xue, W. Freeman, and J. Tenenbaum. Learning a probabilistic latent space of object shapes via 3D generative-adversarial modeling. In NeurIPS, 2016.\n[5] M. Tatarchenko, A. Dosovitskiy, and T. Brox. Octree generating networks: Efficient\nconvolutional architectures for high-resolution 3D outputs. In ICCV, 2017.\n[6] S. R. Richter and S. Roth. Matryoshka Networks: Predicting 3d geometry via nested shape layers. In CVPR, 2018.\n[7] R. Klokov, J. Verbeek, and E. Boyer. Probabilistic Reconstruction Networks for 3D Shape Inference from a Single Image. In BMVC, 2019\n[8] T. Groueix, M. Fisher, V. Kim, B. Russell, and M. Aubry. AtlasNet: A papier-mâché approach to learning 3D surface generation. In CVPR, 2018.\n[9] P. Achlioptas, O. Diamanti, I. Mitliagkas, and L. Guibas. Learning Representations and Generative Models for 3D Point Clouds. In ICML, 2018.\n[10] P. Mandikal, K. Navaneet, M. Agarwal, and R. Babu. 3D-LMNet: Latent embedding matching for accurate and diverse 3D point cloud reconstruction from a single image. In BMVC, 2018.\n[11] N. Wang, Y. Zhang, Z. Li, Y. Fu, W. Liu, and Y. Jiang. Pixel2Mesh: Generating 3D\nmesh models from single RGB images. In ECCV, 2018.\n[12] C. Wen, Y. Zhang, Z. Li, and Y. Fu. Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation. In ICCV, 2019.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HkgILpeiFS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is about decoder networks for 3D point cloud data, i.e., given a latent vector that encodes shape information, the decoder network outputs a 3D point cloud. Previous approaches like PU-Net, or TopNet have used an MLP with a fixed number of output points for this task. The approach in this paper takes inspiration from the work of Li et al. and extends a sample based method for the decoder network. Specifically, the core contribution is to sample from a Gaussian distribution with the latent vector as mean and a variance computed by a small network conditioned on the latent vector. This sampled vector is then feed to an MLP to generate a 3D point estimate. The sampling approach is compared to the method of Li et al. (concatenating noise values to the latent value), and to directly adding Gaussian noise to the latent vector on auto-encoding shapes from the ModelNet40 dataset. Further studies on the impact of the noise on the reconstructed shapes are presented.\n\nI argue for a weak reject of this paper (in its current form) because (1) the presented method seems to be too incremental, and (2) I am missing experiments that compare to other state-of-the-art methods (e.g. on the task of 3D reconstruction from single images).\n\nRegarding the first point, let's start with the related work that is missing some very relevant paper. I would like to see the relation and comparison to implicit surface models (e.g., Park et al. \"DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation\", or Mescheder et al. \"Occupancy Networks: Learning 3D Reconstruction in Function Space\") that sample not based on a random vector, but on the 3D position itself. Further, I see Fan et al. \"A point set generation network for 3d object reconstruction from a single image\" in the references, but not where it is discussed in the paper. Further, Yang et al. \"FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation\" and Groueix et al. \"AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation\" seem to be quite relevant.\nFurther, the core contribution is to add a noise vector to the latent vector, instead of appending it to the latent vector as in Li et al. The presented evaluations in Fig. 3 and Fig. 4 show that this might be slightly beneficial, but I am not convinced how this performance increase would translate to a other tasks.\n\nThis is also my second major point of criticism: The presented evaluations are nice, but also seem like a toy problem. What would have been interesting is to compare the presented decoder network to state-of-the-art methods on other tasks presented in the literature. One example might be 3D reconstruction from a single image (cf. Fan et al.), or shape completion (cf. Park et al.). Even for the auto-encoding setting it would be possible to compare to other methods (cf. Yang et al., PointFlow).\n\nOne part that might lower the performance on the above mentioned task is the inferior encoder network (PointNet) that takes no neighbourhood information into account. There are quite a few stronger methods that can be considered, e.g. PointNet++, or any of the recently proposed point convolution methods.\n\nIn Fig. 3 and 4 the loss seems to be quite large, given that a Chamfer distance is used and the shapes are scaled to fit into a unit sphere. What function is used for dist? And are there severe outliers in the generated point clouds?\n\nOn page 5 it is stated that the Chamfer distance is used due to its relative speed. Relative to what?\n\nOn page 7 it is stated that the MLP results appear to be more evenly distributed. The last row in Fig. 5 suggests otherwise on the back of the chair. Also, the NoiseAdd and NoiseAppend seem to miss quite some details in the second row (or are just more noisy in general), which one is the proposed NoiseLearn?\n\nThings to improve the paper that dis not impact the score:\n- On the second pager, first line, there is an inconsistency in the citing style (use of parentheses)\n- What is meant with a \"parameterized ReLU\" on page 4?\n- The point cloud visualizations can be greatly improved. The grid is not necessary and the images are very small, making it hard to see details.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers have raised several important concerns about the paper that the authors decided not to address.
Decision: Reject