A Statistical Manifold Framework for Point Cloud Data
Authors: Yonghyeon Lee
Seungyeon Kim
Jinwon Choi
Frank C. Park
Venue: ICLR 2022 Submitted (2022)
Keywords: Riemannian Geometry
Point Cloud
Autoencoders

Abstract: A large class of problems in machine learning involve data sets in which each data point is a point cloud in $\mathbb{R}^D$. The reason that most machine learning algorithms designed for point cloud data tend to be ad hoc, and difficult to measure their performance in a uniform and quantitative way, can be traced to the lack of a rigorous mathematical characterization of this space of point cloud data. The primary contribution of this paper is a Riemannian geometric structure for point cloud data. By interpreting the point cloud data as a set of samples from some underlying probability distribution, the set of point cloud data can be given the structure of a statistical manifold, with the Fisher information metric acting as a natural Riemannian metric; this structure then leads to, e.g., distance metrics, volume forms, and other coordinate-invariant, geometrically well-defined measures needed for applications. The only requirement on the part of the user is the choice of a meaningful underlying probability distribution, which is more intuitive and natural to make than what is required in existing ad hoc formulations.  Two autoencoder case studies involving point cloud data are presented to demonstrate the advantages of our statistical manifold framework: (i) interpolating between two 3D point cloud data sets to smoothly deform one object into another; (ii) transforming the latent coordinates into another with less distortion. Experiments with synthetic and large-scale standard benchmark point cloud data show more natural and intuitive shape evolutions, and improved classification accuracy for linear SVM vis-\`{a}-vis existing methods.

Under review as a conference paper at ICLR 2022
A STATISTICAL MANIFOLD FRAMEWORK FOR POINT CLOUD DATA
Anonymous authors Paper under double-blind review
ABSTRACT
A large class of problems in machine learning involve data sets in which each data point is a point cloud in R.. Many applications involving point cloud data sets require a means of measuring not only distances, but also angles, volumes, derivatives, and other more advanced geometric notions. Existing approaches for the most part tend to be ad hoc, and lack coordinate-invariance. invariance, In this paper we develop a Riemannian geometric structure for point cloud data. By interpreting a point cloud samples from some underlying probability distribution, the space of point cloud data can be given the structure of statistical manifold, with the Fisher information metric acting on the part of the user is the choice of meaningful underlying probability dis- tribution, which is more intuitive and natural make than what is required in existing ad hoc formulations. Two autoencoder case studies involving point cloud data are presented to demonstrate the advantages of our statistical manifold frame- work: (i) interpolating between two 3D point cloud data sets to smoothly deform one object into another; (ii) finding an optimal set of latent space coordinates that minimizes distortion.
natural Riemannian metric. The only requirement
INTRODUCTION
A growing number of machine learning problems involve data sets in which each data point is point cloud in R.,, e.g., 3D point cloud obtained by depth cameras. Typical applications include measuring the degree of similarity betweeen two point clouds the point clouds may he measure- ments obtained from depth camera, for example- - for which distance metric on the space of point clouds is needed. Some widely used distance metrics used in this context include the Hausdorff dis- tance (both the original and averaged versions), the chamfer distance (Hausdorff 1914), and the earth mover distance (Rubner al. 2000).
However, the distance metric measures just one aspect of point cloud data; other applications may require more advanced concepts and tools. For example, in the case of moving point cloud, one may wish to measure some of its more dynamic aspects like its velocity or other quantities that require notion of higher-order derivatives. Applications such Monte Carlo sampling may require the construction of an isotropic Gaussian distribution on the underlying space, in which case the notion of "aaglle" is needed in addition distance (more technically, an inner product on the tangent
space is required (Girolami & Calderhead 2011 Gemici et al. 2016 Mallasto & Feragen 2018).
Related to the above, the idea of inter- preting a point cloud as samples from some underlying probability distribution is well-known, and has been applied to
Point doud space
Statistical manifold
1-1 mapping
problems ranging from point set registra- tion (Jian & Vemuril 2005] Wang et al.) 2006 Myronenko & Song 2010 Hasan- belliu et al. 2014 Zhou et al. 2014 Min et al. 2018 Li et al. 2022) 1) to point cloud de-noising (Zaman al. 2017 Luo & Hu 2021]. In these approaches, divergence measures from probability and informa-
M))
plet X.)
RED
Figure I: Illustration of statistical manifold obtained from the 1-1 mapping between the set of point cloud data and the space of probability density functions.

---

Under review as a conference paper at ICLR 2022
tion theory have been utilized to compute the similarity between point clouds. While effective for certain applications, these divergence measures still only capture just one aspect of point cloud data, and cannot be used to measure other quantities of more geometric nature.
For more advanced applications, a rigorous mathematical characterization of the space of point cloud data is essential ingredient being coordinate-invanl invariant and geometrically well-defined) analysis of the types described above, particularly one based on Riemannian geometry. As such, the first contribution of this paper is Riemannian geometric structure for the space of point cloud data.
more comprehensive, robust, and correct (in the sense of
The key idea behind our approach draws upon the information geometry framework of (Amari & Na- gaoka 2000) Amari 2016). in which the space of probability density functions is given the structure of Riemannian manifold (the statistical manifold), with the Fisher information acting as a natural Riemannian metric (the Information Riemannian metric, or riffmRiimnnninn metric for short).
The connection between point cloud data and information geometry is established by constructing a 1-1 mapping from the space of point cloud data to the space of probability density functions, i.e., a point cloud X = {T11,... Inp Ig E [P] is mapped to density function p(æ; X) on RDD in 1-1 fashion as illustrated in Figure
Two case studies involving autoencoders are presented to demonstrate the benefits of our approach. In the first case study, a pre-trained autoencoder is used to encode two 3D point clouds one repre- senting a cylinder, one a cone and the minimal geodesic (or path of shortest length) with respect to the aifo-Riemannian metric is then constructed between these two objects. The shape evolution ob- tained for the info-Riemannian metric is seen to be far more natural and intuitive than that obtained for the straight line interpolant in latent space.
In the second case study, we use the info-Riemannian statistical manifold framework to find a set of distortion minimizing latent space coordinates, in the sense that (Fuclidean) straight lines in the latent space closely approximate minimal geodesics on the statistical manifold. Such a set of coordinates offers a more discriminative representation for the data manifold (Chen et al., 2020) that results in, e.g., higher linear SVM classification accuracy vis-é-vis existing state-ot-the art methods. Experiments are carried out with both synthetic and standard benchmark datasets (ShapeNet (Chang et al. 2015), ModelNer (Wu et al. 2015)).
2 STATISTICAL MANIFOLDS AND THE FISHER INFORMATION METRIC
We begin by extending the original definition of a statistical manifold as follows:
Definition 1. Given m n--dimensional topological manifold" O and 1-1 map from θ the space of probability density functions 0 + p(z;8), 8). the image of this mapping, denoted S == {p(x; 0) e ). is an m-dimensional statistical manifold.
In the original definition θ is taken to be an open subset of IRm (Amari & Nagaoka 2000 Amari 2016)).
By endowing S with a Riemannian metric, S can be given the structure of a Riemannian manifold, allowing for lengths, angles, and volumes to be defined on S in coordinate-invarian manner. The Fisher information metric serves as a natural Riemannian metric on S: the elements (giy) of the Fisher information metric G(O) E pmxx can be expressed
(0)(0) = p(ar; 0) a Olog p(o 0) aogg p(x; 0) dx, i.j = 1,..., ,,,
(1)
00,
06,
are local coordinates on S. Defining infinitesimal length on S by ds² d G(O)d0, the length of a curve 0(t) on S can then be computed the integral fo" ds. Further details on statistical manifolds and the Fisher information metric can be found in, e.g., (Amari &
where 0 =
Nagaoka 2000
Efron & Hinkley 1978. Rissanen 1996, Han & Park 2014).
'A topological manifold locally Fuclidean Hausdorff topological space.
2

---

Under review as a conference paper at ICLR 2022
3 STATISTICAL MANIFOLD FRAMEWORK FOR POINT CLOUD DATA
With the above statistical manifold preliminaries, ture for the space of point cloud data. Section 3.1 defines a statistical manifold from the point cloud data, while Section 3.2 uses the Fisher information metric to construct a Riemannian met- ric for point cloud data. To keep the definitions and results simple, we shall assume throughout all point cloud data consists of exactly n distinct points in RD, i.c., each point cloud is of the form X = {æ1, .... Fn 24 E R2,"0 .T; # Iÿ i Φ j}. The set of all point clouds is denoted X'. Later we dis- cuss methods for dealing with point clouds that violate our assumptions. The proofs of propositions in this section are in Appendix B.
construct Riemannian geometrie struc-
3.1 STATISTICAL MANIFOLD OF POINT CLOUD DATA
The core idea for constructing the statistical manifold is to interpret a point cloud X as a set of samples drawn from underlying probability density function. Using kernel density esti- mator (Parzen 1962 Davis et al.) 2011). a parametric probability density function p(æ; X) can be defined in which X itself the parameter:
Definition Given a positive kernel function K TR. D
such that SRo K(u) du 1 and D x D symmetric positive-definite matrix E (the bandwidth matrix), the kernel density estimate
p(r: X
TV Σ
K(E (x--)(x-zu)) zx))
said to a statistical representation of the point cloud X E X. The set of statistical representa- tions is denoted S = {p(x: X) X E X).
To ensure that S is statistical manifold, recall from Definition Ihhat the following conditions need to be satisfied: (i) X is topological manifold; (ii) A 1-1 mapping h X - X p(z; X) must be defined. The first condition can be satisfied with the "distinct points" assumption:
Proposition 1 (Corollary 2.2.11. 1. in (Knudsen) 2018)). The set of point clouds in which each point cloud is set of distinct points of dimension D, is -.D-dimensi topological manifold.
To satisfy the second condition, additional assumptions needed. The following proposition pro- vides a sufficient condition for to be 1-1:
Proposition 2. If the set of functions {K(E (x- E F} are linearly independen for any arbitrary finite subset C IR." with [F| S 2n, the mapping X is 1-1.
With the above proposition, any kernel function that satisfies the linear independence condition is sufficient to ensure the existence of a 1-1 mapping h. For our purposes the standard and widely used normal kernel function satisfies the linear independence condition.
Proposition 3. Under the distinct points assumption, the standard (multivariate) normal kernel function
1 exp(- K(u)
u" u
with the scaled identity bandwidth matrix, i.e. E = I²1, satisfies the linear independence condition in Proposition
From Propositions 2 and 3 we have established that, under the distinct points assumption and using the standard normal kernel function, the mapping h X - S is 1-1; S can therefore be given the structure of statistical manifold. While other choices of kernel function are possible, throughout the remainder of the paper we use the standard normal kernel function. Figure 2 illustrates statistical manifold representations some example point clouds.
The linear independence of set of functions implies that only trivial linear combination of the functions cquals the function.
3

---

Under review as a conference paper at ICLR 2022
Point cloud
B = 0.1
k= 0.2
& = 0.3
k 0.5
k = 0.8
==10
X = 12
ht == 15
h 20
k 3.0
Figure 2: Probability heat maps for various is (the greener, the higher) for some examples from the ShapeNet dataset (Chang et al., 2015), where we set = k x MED for & (0, oo). MED denotes the median of the distances between the points in the point cloud and their nearest points.
3.2 INFORMATION RIEMANNIAN METRIC FOR POINT CLOUD DATA SPACE
We now equip the point cloud statistical manifold S with the Fisher information metric, which refer to as the info-Riemannian metric and denote by H. The first task is to define local coordinate system on the space of point clouds X. Toward this end, we use the matrix representation unique; given an
X R of point cloud X. Observe that the matrix representation permutation matrix P E RRxx, then X and PX represent the same point cloud X. Fortunately, this does not cause problems since p(æ; X) is defined in permutation-invarias p(z; PX) for any n x n permutation matrix P. Throughout we use italics to denote local coordinate representations, e.g., has local coordinates coordinates V E R
way, i.e., p(æ; X)
g=xD, the tangent vector V Tx has local
The info- aieemaiennnnaan metric can be expressed in local coordinates coordinates X as follows:
(x) = / p(z; X)
i log p(æ; X) Ô log p(æ; X) dx,
axu
0Xki
for i, = 1,..., n. and j. 1 = 1,.... D. Given two tangent vectors V. W E Tx with respective matrix representations V, W E RR their inner product then computed as follows:
(V = Σ Σ
(5)
1.k: 1,1ml
We note that the coordinate expression of the info-Riemannian metric (X) results in permutation-invariint inner product, i.e., Σ Hx)x(X)VV') = (PX)((P))P) for any n X n permutation matrix P, showing that the metric is geometrically well-defined.
Using the standard normal kernel function, the coordinate expression of the aifemniimnnnian metric Hight has a simple analytic expression as follows:
Proposition 4. With the standard (multivariate) normal kernel function and the bandwidth param- eter a, the information Riemannian metric is given by
K 2m K(
K(
(x
x2)(x - xxx dr.
(x)x(X)) = p(x; X)
(6)
g
Figure 3 shows that, given two moving point cloud data whose velocity matrices have equal Euclidean norm (i.e.. Vi) the velocity norms under the Info-Riemannian metric are significantly different: the veloc- ity A has value of 0. 0.26226 while the velocity B has value of 2.2 x 10-8 In particular, observe that the tangential velocity in the case B, which does not change the overall distribution of the point cloud, has a very small velocity norm under the info-Riemannian metric should.
Σ -
Figure 3: Two moving point clouds with different velocity matrices.
4

---

Under review as a conference paper at ICLR 2022
Stop 5
Step Stop
Step
Step
Step 30 Step
Step Step
Stop
Step 300
Euclidean
Reference
Initial
Info- Rllemannian
Figure 4: Random walks for point cloud data on the statistical manifold equipped with Euclidean metric (upper) and riemamiiannnn metric (lower). An initial shape is defined by slightly deform- ing the reference sphere with the addition of small velocity matrix V. Then, sequences of point clouds are generated by recursively adding randomly sampled velocity matrices normalized to have the same norm with V under each metric. Details of implementations are Appendix C.
As another illustrative example that highlights the difference between the info-Riemannian metric and Euclidean metric (i.e., (V, W) == Σm= Σn= Vü Figure shows random walks for point cloud data under these metrics. Consider a randomly sampled and normalized velocity matrix under Euclidean metric. 3D velocity vectors in the sampled velocity matrix are equally likely to point any direction. On the other hand, 3D velocity vectors in the sampled velocity matrix under mifemRiimannian metric are more likely point tangential directions (e.g., more likely the than the case A in Figure 3) As a result, unlike the standard Euclidean metric (top), the info- Riemannian metric (bottom) produces random walks that stay close to the initial sphere without significant changes in its overall distribution pattern.
4 APPLICATIONS TO POINT CLOUD AUTOENCODERS
Riemannian geometric formulations of autoencoders for representation learning have recently been introduced and extensively studied in (Shao et al., 2018 Arvanitidis et al. 2018 Yang et al., 2018a Chen et al. 2018, Kalatzis et al., 2020, Arvanitidis al., 2020) Chen et al., 2020p. In these works, the image of the decoder function is viewed as low-dimensional manifold embedded in the high- dimensional data space - we refer to this manifold as the decoded manifold and a Riemannian metric for the decoded manifold is obtained by projecting the data space Riemannian metric to this manifold. In contrast, this perspective cannot be reasonably extended to existing point cloud au- toencoders (e.g.. FoldingNet (Yang al., (2018)), AtlasNet (Groueix al. 2018). AtlasNetV2 (De- prelle et al., 2019), and TearingNet (Pang et al. 2021), due to the absence of a geometrically well-formulated Riemannian manifold structure.
In this section, using the info-Riemannian metric, we extend this perspective by defining a Rieman- nian metric for the decoded manifold of the point cloud autoencoder. With this info-Riemannian metric, we examine two case studies: (i) interpolation between two points of latent space via the minimal geodesic; (ii) learning an optimal set of latent space coordinates that best preserves dis- tances and angles (or intuitively, minimizes distortion).
Consider a point cloud decoder function with the m-dimensional latent space f R + R where the output is expressed in terms of the matrix representation. The projection of the info- Riemannian metric on the point cloud statistical manifold to the decoded manifold then expressed in latent space coordinates E R as follows:
G=b(z) :0 [=)==>>> D M
2,K=1 j,1=1
where y denotes the Jacobian of f. and the indices a, both range from to m. The latent space is then assigned Riemannian metric Gas the following two applications rely on G(z) E mm
Geodesic interpolation: The latent space metric G(z) can be used to find the minimal geodesic curve connecting two point clouds (i.e., the shortest length curve in the decoded manifold). Let 21, 22 be the encoded values of the two point clouds in the latent space RT In terms of the metric G(z), the geodesic curve connecting these two points can be determined as a solution the following optimization problem (Do Carmo 2016):
min
f 2(1) (())))()) dt, =(t)
(8)
5

---

Under review as a conference paper at ICLR 2022
subject to =(0) 21 and 21) = 22. Parametrizing 2t)) by cubic spline with fixed boundary points 21, 22 then leads to an unconstrained optimization problem. To avoid excessive memory consumption when computing the objective function and its gradient, instead of the usual Riemann sum approximation of the integral, we interpret the integral as an expectation over the uniform distribution i ~ (O, 1) and accordingly use the mini-batch sampling technique.
The latent space metric G(2) can be used to formulate regularization term when training an autoencoder to learn an optimal set of latent space coordi- nates; by "optimal" we mean G(z) = cl for some positive scalar c, so that the decoder preserves distances and angles as much as possible. Recently, a regularization technique for this purpose has been introduced in (Chen et al. 2020). Specifically, the following regularization term is added to the reconstruction loss function:
Learning optimal latent space coordinates:
[g-r[]]((x) cf }
where ||- ||e is the Frobenius norm, = B 1 T=)G(=)) and P is defined via the modified mix-up augmentation, i.c., 2 ~ P - 2 021 + (1 a)22 where 21, 22 are sampled from the
set of encoded training data and U(-7,,1+ n) for 1 > O. For latent spaces whose dimension m is large, in order to avoid the expensive and memory-consuming computation of G(z) E mRm use the following regularization term in the subsequent experiments:
Ez-p (Bur (allo cu" =='"]]]]]
(10)
where we use mini-batch sampling to estimate the expectations. This can be done much more efficiently since we need only compute the Jacobian- veetoor product, i.c., (J))2
EXPERIMENTAL RESULTS
We now verify the effectiveness of the Info-Riemannian metric for the two point cloud autoencoder applications described above using both a synthetic 3D basic shape dataset and standard benchmark point cloud datasets.
The synthetic 3D basic shape dataset consists of cylinders, cubes, cones, and ellipsoids with various aspect ratios of the shape parameters (e.g., radius versus height for the cylinder). We sample 512 points from the surface mesh the shapes using a greedy sample elimination algorithm, so that each sampled point is approximately the same distance from its neighborhood points (Yuksel 2015). Each point cloud is then normalized so that distance between the two farthest points is one. Further details about the synthetic 3D basic shape dataset generation are provided in Appendix D.
The standard benchmark point cloud dataset consists of ModelNet (Wu et al, 2015) and ShapeNet (Chang et al., 2013), where ModelNet consists of ModelNet10 and ModelNet40, each of which consist of 10 and 40 shape classes, respectively. Each point cloud data has 2048 points; we normalize these into a unit sphere as done in Yang et 2018b).
5.1 SYNTHETIC 3D BASIC SHAPE DATASET
In Section 5.1.I we use a dataset consisting of cones, cylinders, and ellipsoids, which are split into saining/validationnesst sets of size 3196/800/804. We then confirm the validity of the proposed metric by comparing the results of several shape interpolation methods in the latent space.
In Section 5.1.2 to study the effects of the regularization term when learning the optimal latent coordinates (with respect to the Info-Riemanniat metric), we use a dataset consisting of boxes, cones, and ellipsoids divided into rraniig/validationn// sets of size 720/240/240.
We use DGCNN as the encoder (Wang et al. 2019) and connecco neural network as the decoder. The latent space is assumed to be two-dimensional. For the reconstruction loss term, we use the Chamfer distance. The regularization term in Equation (9) is multiplied by a coefficient > and added to the reconstruction loss term. Further details about the network architectures and training are provided in Appendix D.
5.1.1 EXAMPLE 1: CONE, CYLINDER, AND ELLIPSOID
Figure 5 shows the test data encoded in the latent space together with the interpolation trajectories and generated point clouds from those interpolants. In the case of intra-class interpolations (i.e.,

---

Under review as a conference paper at ICLR 2022
Latent space
Interpolants
0.4
-0.2
0.4
-1
-0.75
-0.5
-0.25 Cone
0.25 Filipsold
Cylinder
Linear
Geodesic w/
Geodesic w/ Riemmnnmnnnn metric
Figure 5: Left: Latent space with linear and geodesic interpolants. The orange interpolants connect wide cylinder to tall cylinder, while the magenta interpolants connect cylinder to cone. Linear interpolants and geodesic interpolants under the Euclidean and info-Riemannian metrics are drawn dotted, dashed, and solid lines, respectively. Right: Generated point clouds from those inter- polants. To visually indicate which class generated point cloud belong to, we color these according the ratio of the Chamfer distances to the nearest point cloud for each class (see Appendix D). For example, when it is uncertain which class a generated data belongs to (i.e., the nearest distances to each class are similar), is assigned some color other than blue, red, or green.
interpolants between cylinders). the linear interpolant clearly passes through the red ellipsoid region in the latent space, with some of the generated point clouds clearly ellipsoids. The geodesic interpo- lations under the Fuclidean and info-Riemannian metrics both avoid ellipsoid regions in the latent space. In particular, the geodesic interpolants between two cylinders are also cylinders; this is well- aligned with human intuition. However, if look the generated point clouds detail, while the Rirmmmiinnnnnnn metric produces clearly blue cylinders, some of the generated point clouds with the Euclidean metric are non-blue cylinders (i.e., relatively closer the ellipsoid region) with noisy side surfaces. For the inter-class interpolations (i.e., interpolants between a cylinder and cone), the linear interpolant also clearly passes through the red ellipsoid region. The geodesic interpo- lation under the Euclidean metric produces many non-blue and non-green color shapes during the transition from cylinders to cones, while geodesic interpolation under the info-Riemannian metric produces such cases far less. Overall, it can be observed that the geodesic interpolants under the mifo-Riemannian metric have minimal shape class changes.
Figure shows the latent spaces produced by the regularized autoencoders using the Euclidean and Info-Riemannia Compared to the latent space in Figure 5 (trained without regularization). the fol- lowing observations can be made: (i) the encoded latent spaces of cones and cylin- ders are flattened, and (ii) the ellipsoids and cones become more discríminative. Furthermore, we note that the encoded latent space curves of cones and cylin- ders in the right figure (where the info- Riemannian metric is used) are clearly flatter than those in the left figure (where the Fuclidean metric is used). In other words, if we linearly interpolate between two cylinders or two cones, the interpolants in the right case will most likely remain in the same class, unlike the left case (the generated point clouds from the linear interpolants are provided in Appendix E).
Euclidean metric
Info-Riemannian metric
metries.
100
-20
-20
-10
0
10
20 Ellipsoid
-5
-2.5
0
2.5
5
7.5
Cylinder Corre
Figure 6: Latent spaces produced by regularized au- toencoders, each of which is trained with the Euclidean (Left) and Riemaniannnnn metric (Right). Represen- tative intra-class linear interpolants between two cylin- ders and two cones are drawn as black solid lines.
5.1.2 EXAMPLE 2: BOX, CONE, AND ELLIPSOID
Figure 7 shows the test data encoded in the latent space together with the visualization of the Rie- mannian metric, the fitted Gaussian Mixture Model and its samples, and pairwise Fuclidean dis-
7

---

Under review as a conference paper at ICLR 2022
Latent space
GMM
Samples
Pairwise distancee
Cono Ellipsold
Normalized distence
Vanilla AE
0.4
0.4
0
-0.4
-0.4
-0.5
0.5
-1
-0.5
0
0.5
Box
Cone Ellipsoid
Bok Cone Elllppoiid
Normalized distennee
Regularized KE
0
Box
Cone
Ellisoid
Rox
Cone
Filipsoid
Figure 7: From left to right: latent spaces with equidistant ellipse (<((===>)"0(<>>((= for center 2²) centered on some selected points and sampled points from interspaces, Gaussian Mixture Model (GMM) fitting results, generated samples from the GMM, and the heat map of the pairwise Euclidean distances in the latent space of all test data. The upper figure is a vanilla autoencoder trained without regularization, while the lower figure is trained with regularization (using the info-Riemannian metric). For the samples in the third column, we assign colors using the same method of Section 5.1.1 to visually express which classes the samples are likely to belong to.
tances. From the first column of Figure 7 by comparing the results with and without regularization, the following two key observations can be made: (i) in the vanilla autoencoder case (upper case), the major axes of the gray ellipses aligned with the decision boundary (i.e., hypersurface that par- titions the different class regions). which implies that shapes of different classes are actually more distant in the learned manifold (under the riffoRiiemaaiann metric) than shown in the latent space, and (ii) in the regularized autoencoder case (lower case), by encouraging the metric to be isotropic (i.e.. turning ellipses into circles), the gaps between different class regions are widened. The second column confirms that the components of the GMM are better separated after regularization; each component of the GMM on the regularized autoencoder generates high-quality, even samples from the same class shape as shown in the third column. The heat maps of the pairwise distances in the last column also indicate that shapes in different classes are more distant, and therefore more easily separable in the latent space of the regularized autoencoder.
5.2 STANDARD BENCHMARK DATA
To show that the regularization technique with the info-Riemannian metric can benefit unsupervised representation learning from the perspective of discriminative representation learning, we compare the transfer classification accuracy of ShapeNetCore.w .2 to ModelNet following the same experimen- tal procedure outlined in (Yang et al.] (0188)). When training autoencoders with ShapeNet, random rotations about axis parallel to the direction of gravity are applied to each point cloud. We use four different point cloud autoencoders: FcNer and FoldingNer adopted from (Yang et al., 20188), Point- CapsNet adopted from (Zhao et 2019). and DGCNN-FcNC using DGCNN (Wang et al. 2019); the latent space is 512-dimensional for all. The four autoencoders are trained with and without reg- ularization. In the former case, the regularization terms of Equation (TO) under both the Euclidean and RiffoRiimanniann metrics are used while varying the regularization coefficient A. We distinguish between regularized autoencoders using the Euclidean and Riemaniannnn metrics by an "+E" or "+I" after the network name. After network training is finished, we train linear SVM classifiers with the encoded data for ModelNet10 and ModelNer40. These are split into training/test sets of sizes 399 1/999 and 9843/2468, respectively. Further experimental detail are provided in Appendix D.
Table I shows a comparison of transfer classification accuracy from ShapeNet to ModelNet10 (MN10) and ModelNet40 (MN40) for various recent state-of-the-art methods. In the left column (Adopted from References), the numbers are adopted from previous papers (the experimental proce- dures may differ slightly from ours). In the right column (Implemented by Authors), we report the best numbers obtained (adopted from Appendix E). As shown in the right column, regularization using the Info-Riemannias metric improves classification accuracy over vanilla autoencoders, with
8

---

Under review as a conference paper at ICLR 2022
Table 1: Classification aceuracy by transfer learning for ModelNet10 (MN10) and ModelNet40
(MN40) from ShapeNet.
METHOD
MN40
MN10
METHOD
MN40
MN10
Adopted from References
Implemented by Authors
SPH (Kazhdan al. 20003)
68.2%
79.8%
FeNet
88.3%
93.5%
LFD Chen al. 22003)
75.5%
79.9%
FeNet + (ours)
89.3%
93.7%
VConv- (Sharma et al.
2016
75.5%
80.5%
FeNet + (ours)
90.4%
94.3%
3D-GAN (Wu ct al. 2016)
83.3%
91.0%
FoldingNet
89.3%
93.7%
Latent-GAN Achlioplas ct al. 2018
84.5%
95,4% %
FoldingNet + (ours)
88.9%
94.4%
FoldingNet (Ynng et al.
2018h
88.4%
94.4%
FoldingNet + (ours)
90.1%
94.5%
PointFlow Yang el al.
2019
86.8%
93.7%
PoinCCapsNet
87.2%
93.6%
Multi-Task (Hsssm & Haley 2019)
89. 19.1%
PointCapsNett + (ours)
88.1%
93.7%
PointCapsNer (Zhao et al.
2019)
89.3%
PointCapsNet + (ours)
88.5%
93,9%
DGCNN-FcNet
90.3%
94.5%
DGCNN-FoNet (ours)
89.9%
94.4%
DGCNN-FaNet + (ours)
91.0%
95.2%
higher accuracy compared the Euclidean metric case. Although their performance not directly comparable due to differences in the experimental procedures, it can be seen that our regularized autoencoders are comparable to the state-of-the the-art methods and, at least for our implementation, shows higher classification accuracy than vanilla autoencoders.
We also conduct additional experiments to determine how much more robust the representation obtained with our regularization approach is for noisy point cloud data. We add noise with different levels of standard deviation (1 16, 5%, 10%, and 20% of the diagonal length of the point cloud bounding box) to point cloud data (see Appendix D for details). Then FcNer is trained with and without regularization in the same way as above.
Table 2:
Classification accuracy by transfer learning for ModelNet10 (MN10) and ModelNet40 (MN40) from ShapeNet under the noise levels of I 1%, 5%, 10%, and 20%.
MN40
MN10
METHOD
1%
5%
10%
20%
1%
5%
10%
20%
FcNet
87.8%
83.2%
75.6%
64.5%
92.4%
91.9%
88.4%
79.8%
FoNet E. (ours)
86.6%
85.1
79.1%
70.4%
92.2%
91.1%
88.2%
82.6%
FoNet +1 (ours)
89.0%
86.6 %
81.4%
72.4%
93.3%
92.6% %
91.6%
84.8%
Table 2 shows a comparison of transfer classification accuracy in the presence of noise. As the noise level iccreases, the classification accuracy obviously decreases in both cases, but the reduction is more dramatic for the vanilla autoencoder case. Additional experimental results including semi- supervised classification task included in Appendix E. Overall, it is indeed somewhat surprising that unsupervised classification accuracy can be improved with a simple regularization technique in lieu of complex neural network architecture or loss function.
DISCUSSION AND CONCLUSION
This paper has proposed a new Riemannian geometric structure for the space of point cloud data. We have defined a statistical representation of point cloud data and constructed a statistical manifold in mathematically rigorous way. Then natural Riemannian metric Fisher information metric is assigned to the point cloud statistical manifold, which provides geometrically well-defined measures needed for applications. We demonstrate its advantages through two applications involving point cloud autoencoders: (i) minimal geodesic interpolants under info-Riemannian metric have minimal shape changes compared to the standard linear interpolants, and (ii) the optimal latent coordinates learned using method produce more discriminative representation spaces than existing methods.
As a potential issue, the "fixed number of points" assumption used our construction the sta- tistical manifold may be violated in real world problems. In such cases, we can easily mitigate this issue by matching the number of points in each point cloud through a simple upsampling/down- sampling algorithm. Further, the kernel function used in current implementations, the standard normal kernel function, may not be an optimal choice. Other choices can be explored to enhance algorithms long the conditions of Proposition 2 are satisfied.

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We have provided clear explanations of the assumptions and statements for the theoretical results, and included complete proofs of the propositions in Appendix B. Also, we have included experiment settings detail as much as possible in Appendix C and D such as the number of training/valice tion/test splits of datasets, data pre preppeoessing methods, neural network architectures, and hyper- parameters used training (e.g., batch size, number of epochs, learning rate, etc). particular, we have included the details of the synthetic dataset generation in Appendix D.
REFERENCES
Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. Learning representa- tions and generative models for 3d point clouds. In International conference on machine learning, pp. 40-49. PMLR, 2018.
Shun-ichi Amari. Information geometry and its applications, volume 194. Springer, 2016.
Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2000.
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: on the curvature of deep generative models. Proceedings of the 6th International Conference on Learning Representations (ICLR), 2018.
Georgios Arvanitidis, Søren Hauberg, and Bernhard Schölkopf. Geometrically enriched latent spaces. arXiv preprint arXiv:2008. 00565, 2020.
Paul Besl and Neil D McKay. Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, volume 1611, pp. 586 606. International Society for Optics and Photonics, 1992.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rie 3d model repository. arXiv preprint arXiv: (12.00012,, 2015.
Ding- Yun Chen, Xiao-Pei Tian, Yu-Te Shen, and Ming Oubyoung. On visual similarity based 3d model retrieval. In Computer graphics forum, volume 22, pp. 223-232. Wiley Online Library, 2003.
Nutan Chen, Alexej Klushyn, Richard Kurle, Xueyan Jiang, Justin Bayer, and Patrick Smagt. Met- rics for deep generative models. In International Conference on Artificial Intelligence and Statis- tics, pp. 1550-1550. PMLR, 2018.
Nutan Chen, Alexej Klushyn, Francesco Ferroni, Justin Bayer, and Patrick Van Der Smagt. Learning flat latent manifolds with vaes. arXiv preprint arXiv:2002.04888 2020.
Luca Cosmo, Antonio Norelli, Oshri Halimi, Ron Kimmel, and Emanuele Rodolà. Limp: Learning latent shape representations with metric preservation priors In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pp- 19-35. Springer, 2020.
Richard A Davis, Keh-Shin Lii, and Dimitris N Politis. Remarks some nonparametric estimates of a density function. In Selected Works of Murray Rosenblatt, pp. 95-100. Springer, 2011.
Theo Deprelle, Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. Learning elementary structures for 3d shape generation and matching. arXiv preprint arXiv:/908.0472. 04725, 2019.
Manfredo P Do Carmo. Differential geometry of curves and surfaces: revised and updated second edition. Courier Dover Publications, 2016.
Bradley Efron and David V Hinkley. Assessing the accuracy of the maximum likelihood estimator: Observed versus expected fisher information. Biometrika, 65(3):457-433, 1978.
10

---

Under review as a conference paper at ICLR 2022
Meviana C Gemici, Danilo Rezende, and Shakir Mohamed. Normalizing flows riemannian man- ifolds. arXiv preprint arXiv:161.00230 2016.
Mark Girolami and Ben Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2):
123-214, 2011.
Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-måché approach to learning 3d surface generation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 216-224, 2018
Minyeon Han and Frank C Park. Dti segmentation and fiber tracking using metrics multivariate normal distributions. Journal of mathematical imaging and vision, 49(2)3317-334, 2014.
Erion Hasanbelliu, Luis Sanchez Giraldo, and José C Príncipe. Information theoretic shape match- ing. IEEE transactions on pattern analysis and machine intelligence, 36( (6611))2436222 2451, 2014.
Kaveh Hassani and Mike Haley. Unsupervised multi-task feature learning on point clouds. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 8160-8171, 2019.
Felix Hausdorff. Grundziige der mengenlehre. 1914.
Felix Hausdorff. Felix Hausdorff Gesaommelleeeemllleeeeee Werke Band III: Mengenlehre (1927, 1935) Deskripte Mengenlehre und Topologie, volume 3. Springer-Verlag, 2008
Bing Jian and Baba C Vemuri. A robust algorithm for point set registration using mixture of gaus- sians. In Tenth IEEE International Conference Computer Vision (ICCV'05) Volume 1, vol- ume 2, pp. 1246-1251 IEEE, 2005.
Dimitris Kalatzis, David Eklund, Georgios Arvanitidis, and Søren Hauberg. Variational autoen- coders with riemannian brownian motion priors. Proceedings of the 37th International Con- ference on Machine Learning (ICML), 2020.
Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz. Rotation invariant spherical harmonic representation ume 6, pp. 156-164, 2003.
shape descriptors. In Symposium on geometry processing, vol-
Ben Knudsen. Configuration spaces in algebraic topology. arXiv preprint arXiv: 165, 2018.
Feiran Li, Kent Fujiwara, and Yasuyuki Matsushita. Toward unified framework for point set registration. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp- 12981-12987. IEEE, 2021.
Shitong Luo and Wei Hu. Score-based point cloud denoising. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 4583 4592, 2021.
Anton Mallasto and Aasa Feragen. Wrapped gaussian process regression on riemannian manifolds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5580- 5588, 2018
Zhe Min, Jiaole Wang, and Max Q-H Meng. Robust generalized point cloud registration using hybrid mixture model. In 2018 IEEE International Conference Robotics and Automation (ICRA), pp. 2812-8816 IEEE, 2018.
Andriy Myronenko and Xubo Song. Point set registration: Coherent point drift. IEEE transactions on pattern analysis and machine intelligence, 32(12):2262-2775. 2010.
Trung Nguyen, Quang-Hien Pham, Tam Le, Tung Pham, Nhat Ho, and Binh-Son Hua. Point-set distances for learning representations of 3d point clouds. arXiv preprint arXiv:2 102.04014, 2021.
Jiahao Pang, Duanshun Li, and Dong Tian. Tearingnet: Point cloud autoencoder learn topology- friendly representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 7453-7462, 2021.

---

Under review as a conference paper at ICLR 2022
Emanuel Parzen. On estimation of probability density function and mode. The annals of mathe- matical statistics, 3363):1065-1076, 1962.
Jorma J Rissanen. Fisher information and stochastic complexity. IEEE transactions on information theory, 42(1)):0-47, 1996.
Yossi Rubner, Carlo Tomasi, and Leonidas Guibas. The earth mover's distance as metric for image retrieval. International journal of computer vision, 40(2):99-121 121, 2000.
Hang Shao, Abhishek Kumar, and P Thomas Fletcher. The riemannian geometry of deep generative models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 315-323, 2018.
Abhishek Sharma, Oliver Grau, and Mario Fritz. Vconv-dae: Deep volumetric shape learning with- out object labels. In European Conference on Computer Vision, pp. 236-250. Springer, 2016.
Fei Wang, Baba C Vemuri, and Anand Rangarajan, Groupwise point pattern registration using novel edf-based jensen-shannon divergence. In 2006 IEEE Computer Society Conference on Computer Vision and Pultern Recognition (CVPR'OR) 066), volume 1, pp. 1283-1288. IEFE, 2006.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5): 1-12, 2019.
Jiajun Wu, Chengkai Zhang, Tianfan Xue, William Freeman, and Joshua B Tenenbaum. Learning probabilistic latent space of object shapes via 3d generative-adversaria modeling. In Proceed- ings of the 30th International Conference on Neural Information Processing Systems, pp. 82-90, 2016.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Guandao Yang, Xun Huang. Zekun Hao, Ming- Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4541-4550 2019.
Tao Yang, Georgios Arvanitidis, Dongmei Fu, Xiaogang Li, and Søren Hauberg. Geodesic clustering in deep generative models. arXiv preprint arXiv: 0188809944473 2018a.
Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 206-215, 2018b.
Cem Yuksel. Sample elimination for generating poisson disk sample sets. In Computer Graphics Forum, volume 34, pp. 25-32. Wiley Online Library, 2015.
Faisal Zaman, Ya Ping Wong, and Boon Yian Ng. Density- based denoising of point cloud. In 9th International Conference on Robolic, Vision, Signal Processing and Power Applications, pp- 287-295. Springer, 2017.
Yongheng Zhao, Tolga Birdal, Haowen Deng, and Federico Tombari. 3d point capsule networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 1009-1018, 2019.
Zhiyong Zhou, Jian Zheng, Yakang Dai, Zhe Zhou, and Shi Chen. Robust non-rigid point set registration using student's s-t mixture model. PloS one, 9(3)ee9131, 2014.
12

---

Under review as a conference paper at ICLR 2022
APPENDIX
A EXISTING GEOMETRIC/STATIITICCE METHODS FOR POINT CLOUD DATA
A.1 GEOMETRIC METHODS
The Hausdorff distance measures the distance between two non-emply subsets of metric space (Hausdorfi 1914 2008 Given two point clouds X = f.1...., I; E R2} and R and metric ||- 1 yll in R., the Hausdorff distance can be computed as
Y (yi,..., ....... Yn
follows:
max(max(min (iin || yll?), max || - )) zEX yEY
yeY #eX
The Hausdorff distance is susceptible to outliers; hence, in practice, the average Hausdorff distance used more often:
Σ min |- vlI +
min || yll Σ
X
eex
NEY
where |X| denotes the number of elements in the set X. A slightly modified version of this is often referred to as the Chamfer distance (Yang et al. 2018b). The popular point cloud registration algorithm ICP relies on these classes of metrics (Besl & McKay 1992).
Another popular similarity measure between two point cloud data is the Earth Mover's Distance (EMD) (Rbbee et al.) 2000
Σ min || d(z)]1
6-X-Y ZEX
where & is bijective mapping. Although the EMD is computationally more expensive than the above Hausdorff distances, comparing point clouds with optimal matching in EMD provides more robust and well-behaved similarity measure.
The Chamfer distance and EMD are often used measure the distances between two point clouds, but they are typically computationally expensive. Recently, the sliced Wasserstein distance and its variants have been proposed to more efficiently measure distances (Nguyen et al. 2021). another study, each point cloud data is represented as matrix of pairwise Euclidean distances between all points, and the Frobenius norm of the difference between the two matrices is used the distance between two point clouds Cosmo et al. (2020).
However, all these distance metrics measure just one aspect of point cloud data; other applications may require more advanced concepts and tools. For example, in the case of moving point cloud, one may wish to measure some of its more dynamic aspects like its velocity or other quantities that require a notion of higher-order derivatives. Applications such as Monte Carlo sampling may require the construction of an isotropic Gaussian distribution on the underlying space, in which case the notion of "angle" is needed in addition to distance (more technically, an inner product the tangent space is required (Girolami & Calderhead 2011 Gemici et al. 2016 Mallasto & Feragen 2018
We believe a rigorous mathematical characterization of the space of point cloud data is an essen- tial ingredient more comprehensive, correct, and robust analysis of the types described above, particularly one based on Riemannian geometry. In this paper, we rigorously define a Riemannian geometric structure for the space of point cloud data.
A.2 STATISTICAL METHODS
Interpreting the point cloud data of samples from some underlying probability distribution very intuitive and natural, and has been adopted in many previous works (Jian & Vemuri 2005 Wang al. 22066] Myronenko Song 2010 Hasanbelliu et al.) 2014 Zhou 2014 Min al. 2018 Li et al. 2021). Commonly, a mixture model is used to describe the point cloud, written as
follows:
&
p(a w. 0) =
13

---

Under review as a conference paper at ICLR 2022
where the ii weights and ((x(0,) are primitive density functions with parameter 8 (e.g., 6(x) can be a standard Gaussian with mean 8,). Given a point cloud := {z1...... 2 I2 E R }. the parameters w, 0 are either fit with data specified by the then the mixture model is used as a statistical representation of the point cloud data. Besides the most popular choice for Φ. the
Gaussian (Jin & Vemuri 2005), other choices such the t-distribution (Zhou al. 2014) or hybrid model (Min et al. 2018) have been explored.
The main purpose behind using statistical representations in existing works are use the well- known information-theoretii divergence measures such as the KL-divergence to compute the simi- larity between point clouds. However, these divergence measures (which are approximate distance metrics) only measure just one aspect of point cloud just like the distance measures discussed in I. Effective mathematical concepts and tools for defining and measuring other important geomet- ric aspects of point cloud are still lacking.
B PROOF OF THE PROPOSITIONS
B.1 PROOF OF PROPOSITION
Proof. Let's consider two point clouds {=i and {2)) To show that the mapping h X' is 1-1 (especially injective), we have to prove the following statement:
S
p(a; ( p(z; (=)
fu {zi)
(11)
The conditional statement can be rewritten follows:
Σ :( yz)) >
K(E 2:))
(12)
Let denote B = {1 n f and [B] = m, and assume that m < Then the above equation
reduced
Σ
K( (x ---1) y))
K( (x - 2)) = O. sE(a.)
Σ
(13)
VE(u ( -
Since the sets ) B and {2 - B are disjoint and each set has n elements, the LHS has 2(n m) S 2n terms and the terms are different to each other. Then, by the assumption, the above 2(n m) terms are linearly independent, so the above equation cannot hold by the definition the other words, there is a contradiction and we can find that the
of the linear independence. assumption < n is wrong. Therefore,
so {y: 12)=.-
B.2 PROOF OF PROPOSITION 3
Proof. First, let's think about the I-dimensional case, i.e., x E R. Consider an arbitrary point set {xr).n and the corresponding set of functions {K xx;)]; = 1, 1,2,...,AN N}, where the kernel function K is
u 2
K(u)
cxp(
(14)
2'T
Suppose there exist non-zero constants a1, 02 ..., such that
N
(5 xz)) 0 Vo R.
Σ Kái (s- x.))
Σ
as exp(
(15)
V2Ta
The above expression can be reduced as follows:
exp( x2a
x2 0 Vz R.
a, exp((xz))xxp
(16)
..., Let A; = a exp(p for = 1,2, ....., N, then the above equation is simplified as:
N
A cxp(xx,)) = Yr e R.
(17)
14

---

Under review as a conference paper at ICLR 2022
In the other words, it is enough to show that the functions in the set {exp(2,x))e 1,2, N) linearly independent.
There is useful tool for proving the linear independence of the functions:
Theorem 1 (Wronskian's Theorem), Let fI, f2....,ff fn : -I + R be n 1 times differentiable
functions interval I and the Wronskian W (f1. .... fu) be function I defined by:
(x)
f2(x)
fa(x) (a
fi(z)
f2(x)
(2)
(2
(2
Va E IR
(18)
(z)
f (x)
where this is the determinant of the square matrix constructed by placing the functions and their derivatives in appropriate way. Then, if the Wronskian of this set of functions is not identically zero, then the set of functions is linearly independent.
Using the above theorem, (exp(r;œ)) = 1, 2.,.... N}. Substituting zero for x, then we can obtain:
construct the Wronskian W(x) with the set of functions
1
21
T2
N
2
.2
(
N
(19)
N
The above can be simplified by the mathematical induction:
1
0
T1
EN
2
12
22
(20)
NN
01
22
(21)
N-1
N-1
JN
(I2 - x1) ... (xx, - 21)
(22)
N-2
N-2
32
(23)
(xt x
(24)
Since all the points {x; J 1= are distinct to each other, the above determinant value must be non-zero. Then by Wronskian's Theorem, the functions {exp(x;ar) 1,2, ..., N) are linearly independent, so are the functions {K(o 1 {x - xz))]i = 1,2,...,N)
Next, let's think about the general D-dimensional case with D > 2; R. The Wronskian's 's Theorem introduced above cannot be used, instead, we adopt a generalized Wronskian's Theorem applicable to multivariable functions.
15

---

Under review as a conference paper at ICLR 2022
Theorem 2 (Generalized Wronskian's Theorem), Let fI, f:....... RD + R be multivariable functions and the generalized Wronskian gW (fi, f2, ... fn) as a function defined by:
f(x)
Dif(z)
D2f(2)
ff.......... =
Vx E
(25)
Dn-1f(a)
where (<))= - (Ji greater than j. and all Dj are distinct for j = 1, ...,, - 1. We note that if at least one of the generalized Wronskians of this set of functions is not identically zero, then the set of functions is linearly independent.
(z)(2) are row vectors, Dj is any partial derivative order not
Similar to the 1-dimensional case I₁ E IR, consider an arbitrary point set { and the corre- sponding set of functions (K(o - (xx x;)>li = 1, .... N}. where the kernel function K is now:
u² v 2
K(u) V(2T)D exp(
(26)
Suppose that there exist non-zero constants 61, b2, be,..., bn such that
1 by exp( (x x;) );) (xx-
Σ boK (x- z1))
Vo E R..
(27)
(2TO)
The above expression can be reduced follows:
TTo
2 f/1
exp(
by exp(xx exp(
expp(=
Vo e R
(28)
2
Let w; = b, exp(p= 2. xi for i = 1,2, ............. N. Then the final expression of the equation is as follows:
N
Σ
wi exp((x = Vz R.
(29)
In other words, it is enough to show that the functions fexp(2 (f)(=f)))l 1,, N) are linearly independent.
To apply the generalized Wronskian theorem, we need to find proper set of partial derivatives. To find such set, our strategy is to use a coordinate transformation with a rotation matrix R E SO(D), ie., rotating vectors. To be more specific, we will find R that makes the first components of y: Rx; for i = 1,2, ..., N all different and use the transformed coordinates in the following proof. For this purpose, first show that such rotation matrix R exists.
Intuitively, we can find a vector r1 so that ...,.., Tn are projected to rı as all distinct points. If R has this r1 vector as the first column vector, then R can make the first components of yı Rr; for i = 1, 2.... N all different. More precisely, the first components of the two points 2y are same if ri is in the (D - --ddmensionn hyperplane {ri E R P [= (x; - xzj) = 0}. Since there are only ..... N(N - 1)/2 combinations of the pairs of (xe, I;) for i, 1, 2, ..., nI, we can find vector ri such
that
T(a x,) # 0 デ and í, = 2,2..... n.
(30)
We pick coordinate transform matrix R E SO(D) with this v as the first column vector. Then this R can make the first components of y Rx, for = 2,...., all different.
Using the R, rewrite the set of functions (exp(>² z)}i = 1, 1, . N} as follows:
{exp(x² x))i = 1,2, N) = (exp(x²' R² (=))) = 1, ...,N N) (ex ( 2)}i 1, 1.2....,NT N7.
(31)
(32)
16

---

Under review as a conference paper at ICLR 2022
where Ræ and y: Ræ, for i = 1,2, ...., N. We again note that the first elements of y for
? l,... are all distinct to each other.
To use the Theorem 2 in the transformed coordinates 2, pick generalized Wronskian gW (2) with the corresponding partial derivatives:
8 D1 - 821
8
8n-1
D2
0z ... Dn-1
(33)
0=1 0zn-1:
where 2 (21, 2D). With this generalized Wronskian and the set of functions fexp(y! 2) 1, ..,y Nr, substituting zero vector for E RD,, we can obtain:
1
21
y1
g0( =
V12
VIN
(34)
Vii
N
ViN
where Y1s is the first element of y/ for i 1,2, .... N. Since yiu's are all distinct to each other, can obtain, just like the I-dimensional case:
gW(() (yn 911)111)) +
(35)
j==1 S>j
By generalized Wronskian's Theorem, the functions (exp(xf'x)) x))i 1,2,..,, N) are linearly in- dependent, so are the functions {K(o 2 {x - xz))( 1,2,.., N), also in the D-dimensional case.
B.3 PROOF OF PROPOSITION
Proof. Since
Blog p(z:X)
8p(x:X) 8XU
the Riemannian metric Higkt in equation
8x
P(x:)]
/ ((xx X)
Öp(a; X) Öp(a; X)
0X dr.
pxxx;)]
0x
By plugging p(z; X) equation (3
Op(a:X)) Ox
we get the following expression:
Op(x; X)
i
n
Σ
1(2-1/2(22 xa))
0X
0X
KIE K(E-1/2 1/2 -z:))
0
JK(E -1 112 x1))E~11 1/2 θ
0X (s x;)
= JK 1//2 1 2,))
0x (x x.)
= (E-122-1126 x:))
JK: : R + RR is the Jacobian of the kernel function K. This consequently leads to
Hiju(X): = / p(x; x))::)) (Jx Ixtm,,, Σー I) (n(o. Σ Σー )}
dr,
(2n (hix,, x))2
where M1x,21;) E- 3- x.).
17

---

Under review as a conference paper at ICLR 2022
If K is the standard normal kemnel function, then get JK [x) -K(x)x". By plugging this in the above equation with Σ I²I, we get the following:
(Jx (K 2' (J A(x.02)) -- ))
f p(a; x)
Hxjx(X)=
dx
(Em=1 (h(2, Fmm))
= ff pla;; X)
K [h((a, x4) (h(a,
(x - x2) - -xx)
de.
(5m=i ((m(),)mm))
0¹
C RANDOM WALKS OVER THE POINT CLOUD RIEMANNIAN MANIFOLD
This section describes details the random walk experiments over the point cloud statistical man- ifold equipped with Riemannian metric Hight i, k ..,.. n, 1, 2, 3. First, initial shape defined by slightly deforming the reference sphere with the addition of small velocity matrix V. Then, sequences of point clouds generated by repeating the following process: given point cloud matrix X, reorder as a 3n-dimensional vector and the Riemannian metric at as 3n x 3n matrix, ii) sample velocity vector from the Gaussian distribution whose covariance the inverse of the Riemannian metric, iii) reorder the sampled velocity vector to n x 3 matrix representation and divide it by its norm induced by the metric, and iv) add it to X.
D IMPLEMENTATION DETAILS FOR THE EXPERIMENTS
D.1 SYNTHETIC 3D BASIC SHAPE DATASET GENERATION
For the synthetic 3D basic shape dataset, we define 5 shape classes consist of cylinder, cone, elliptic cone, ellipsoid, and box. Figure shows the representative shape of the shape corresponding each class and the shape parameters required to define the shape class. We again note that we sample 512 points from the surface mesh of the shapes using a greedy sample elimination algorithm, and each point cloud is then normalized so that distance between the two farthest points is one. In the process of normalizing the point cloud, the dimension of the shape parameter is reduced by one dimension. As example, in the case of cylinder, the cylinders are contained in two-dimensional space defined by radius dimension r and height dimension but is reduced to one dimension through normalization. In summary, cylinders and cones with two shape parameters placed in one-dimensional space, and elliptic cones, ellipsoids, and boxes with three parameters are placed in two-dimensional space.
Cylinder
Elliptic
Ellipsoid
h
h
h
d
radius height
radius height
width
width depth height
width depth d height
depth height
Figure 8: Representative shape to each class and the shape parameters required to define it. There are 5 shape classes including cylinder, cone, elliptic cone, ellipsoid, and box.
dataset consisting of cones, cylinders. and ellipsoids, which are split into training/ver http//aliddaaiooooo vadiinttt/ttttttt sets size 3196/800/804, The detail ranges of the exact value of the shape parameters shown in Table 3
In Section S.1.I
18

---

Under review as a conference paper at ICLR 2022
Table 3: The ranges of the shape parameters of the dataset used in Section 5.1.1
SHAPE
param
min
max
param
min
max
param
min
max
Cylinder
r
0.01
0.12
0.05
0.45
Cone
0.02
0.15
h
0.02
0.45
Ellipsoid
W
0.03
0.12
0.03
0.12
0.03
0.12
In Section 5.1.2 we dataset consisting of boxes, cones, and ellipsoids divided into training/val- idation/test sets of size 720/240/240.. The detail ranges of the aspect ratios of the shape parameters shown in Table
Table 4: The ranges of the shape parameters of the dataset used in Section 5.1.2
SHAPE
param
min
max
param
min
max
Elliptic cone
d/w
0.33
3
h/w
0.33
3
Ellipsoid
d w
0.33
3
h/w 0.125
0.33
Box
d/w 0.33
3
h/w
0.33
3
D.2 DETAILS FOR EXPERIMENTS ON SYNTHETIC 3D BASIC SHAPE DATASET
We used an encoder with a structure similar to the classification network used in DGCNN (Wang et al. 2019). The input point cloud with dimension 3x512 passes through five EdgeConv layers with point-wise latent space dimensions (64, 64, 128, 256) and max pooling layer (we do not use batch normalization layer unlike the original DGCNN classification network). then can obtain a 1024-dimensional feature vector. Other settings are the same (e.g.. k = 20, leaky relu activation) Then this feature vector again passes through three fully-connected neural networks with dimension (512, 256, 2) with leaky relu activation function and linear output activation function: the latent space woo-dimensional. For the decoder model, we simply use fully-connecter neural network as the decoder. The two-dimensional vector on the latent space passes through three fully-com fully-connected neural networks with dimension (256, 512, 3x522) with relu activation function and linear output activation function; the output is 3D point cloud with the number of points 512.
Section 5.1.1: To train the networks, we use ADAM with learning rate of 0.001 and batch size of 16; the total number of the epochs is 500. The mean value of MEDs of the dataset is 0.0339, and we use the bandwidth value & to 0.5. We use Chamfer distance as the reconstruction loss; for regularization figures, the regularization term with the version of Equation ( is multiplied by coefficient À = 107 with the Riffmmiimnnnnann metric and by a coefficient metric and added to the reconstruction loss term for each metric case. The value of 17 is set to be 0.0. For geodesic computation, we parametrize the curve 2(t) ) by a cubic spline with fixed boundary points 21, 22 and 10 control points. The control points first initialized with equally spaced linear interpolants between ž and 22. Then, for each iteration of optimization, we randomly sample 40 points on t; ~ (0, 1), = 1, 1,.. 40 and calculate an expectation 40 the sampled points as the approximation of the objective function. We use ADAM with a learning rate of 0.001 and the total number of the iterations is 5000.
with the Euclidean
1 Σte a(t,) G(z(t))) over
Section 5.1.2: To train the networks, we use ADAM with learning rate of 0.001 and batch size of 16; the total number of the epochs is 3000. The mean value of MEDs of the dataset is 0.0341, and we use the bandwidth value & to 0.5. We use Chamfer distance as the reconstruction loss, and the regularization term with the version of Equation (9) is multiplied by coefficient À 102 and added to the reconstruction loss term. The value of is set to be 0.0.
Color assigning method: To visually indicate which class generated point cloud belong to, color these according to the ratio of the Chamfer distances to the nearest point cloud for each class. In detail, the smallest value (distance to nearest point cloud) is found by comparing the distance between the given point cloud and all point clouds of each class in the dataset. Since we are using 3 shape classes in both examples, call the nearest distance to each class d1, d2, and d3. After
19

---

Under review as a conference paper at ICLR 2022
that, the vector d = (d1, d2. d3) is normalized with 2-norm that the 2-norm of the vector to be 1. Finally, the value 0.2 x coftmax(1/d1) 1/d2, 1/dy) is regarded as the ratio of the distances and a color is assigned to given point cloud according to this ratio (i.e., linear weighted sum in the RGB coordinate).
D.3 DETAILS FOR EXPERIMENTS ON STANDARD BENCIMARK DATASET
We use four different point cloud autoencoders: FeNet, FoldingNet, PointCapsNee, and DGCNN- FcNer; the latent space is 512-dimensional. For FoNel and FoldingNet. we use the exactly same point cloud autoencoder structures both adopted from (Yang et al. 2018b). For PointCapsNet, we also use the exactly same point cloud autoencoder structure adopted from (Zhao et al. 2011);; we use 16 x 32 capsules to restrict the latent space to reasonable size of 512. For DGCNN-FoNet, we use DGCNN classification network as encoder (i.e., the same encoder architecture used experiments on synthetic 3D basic shape dataset, see Appendix C.2), and the same decoder structure from FcNet as decoder (i.e., three fully-connected neural networks with dimension (1024, 2048, 3 3x4048 with relu activation function and linear output activation function). To train the networks, we use ADAM with a learning rate of 0.0001, betas of [0.9, 0.999], and weight decay of 0.000001 and batch size of 16; the total number of the epochs is 500. The mean value of MEDs of the dataset is 0.0356, and we use the bandwidth value k to 0.8. We use Chamfer distance as the reconstruction loss and regularization term with the version of Equation (10) with the value of 1) be 0.2. The regularization term is multiplied by various coefficients, where the values of the regularization coefficients are summarized in Appendix D.
D.4 DETAILS FOR EXPERIMENTS ON STANDARD BENCHMARK DATASET WITH NOISE
We use the exactly same point cloud autoencoder structures adopted from (Yang et al. 2018b), FeNet, where the latent space is 12-dimensional. We add noise to each point x in point cloud of the dataset (ShapeNet, ModelNet10, and ModelNet40) according to X + x-+ mv, where v uniformly sampled on the unit sphere and m is sampled from the Gaussian distribution with zero mean and different levels of standard deviation (1%, 5%, 10%, and 20% of the diagonal length of the point cloud bounding box). The training configuration is the same with the case of Appendix D.3 except the followings. The regularization term is multiplied by X = 8000. The mean values of MEDs of the dataset are 0.0320, 0.0364, 0.0442, and 0.579 for the cases of the noise levels 1%, 5%, 10%, and 20%, respectively, and we use the bandwidth value ls to 0.8.
E ADDITIONAL. EXPERIMENTAL RESULTS
E. SYNTHETIC DATASET
E. QUANTITATIVE RESULTS
In this section, we quantitatively evaluate how much the regularization term improves class sepa- rability. In addition to the example shown in Figure 7, more diverse synthetic datasets are made and experiments are conducted. We use datasets consisting of boxes, elliptic cones, and ellipsoids as the case in Figure 7. But in these additional experiments, for each shape class, we use more di- verse parameter configurations. In details, we generate short, normal, and tall shapes for each shape class, and the aspect ratios of the shape parameters are shown Table We conduct a total of 27 experiments with 3 combinations.
Similar to the experiment of Section 5.1.2, each dataset is divided into training/validationneess sets of size 720/240/240. Training configurations are the same with the experiment of Section 5.2. except that the mean values of MEDs are different to each other (but we consistently use the bandwidth value i to 0.5) and the total number of epochs is 500.
In the obtained representation spaces, after fitting the Gaussian mixture model by using the training and validation data, the clustering scores measured with the test data. The Normalized Mutual Information (NMI), Adjusted Rand Index (ADI), and Silhouctte Coefficient (SC) are used as metrics. The results shown in Table 6 For all three clustering measures, regularized autoencoder shows
20

---

Under review as a conference paper at ICLR 2022
The ranges of the shape parameters of the dataset used in quantitative analysis on synthetic dataset
Table
SHAPE
param
min
max
param
min
max
Elliptic cone short
d/w
0.33
h W
0.125
0.33
Elliptic cone normal
d/w 0.33
h W 0.33
3
Elliptic cone tall
d/w 0.33
h W h/WW 0.125
8
Ellipsoid short
d/w 0.33
0.33
Ellipsoid normal
d/w 0.33
h/w 0.33
3
Ellipsoid tall
d/w 0.33
h/w
8
Box short
d/w 0.33
h/w 0.125
0.33
Box normal
d/w 0.33
h// 0.33
3
Box tall
d/w 0.33
h/w
8
Table 6: Normalized Mutual Information (NMI), Adjusted Rand Index (ADI), and Silhouette Coef- ficient (SC) of the vanilla autoencoder and regularized autoencoder
MODEL
NMI
ADI 0.7209 = 0.2598
SC
Vanilla AE
0.7624 + 0.2132
0,4207 + 0.0453 0.5279 0.0817
Regularized AE
0.9484 + 0.1391
0.9368 0.1737
higher values, indicating that our regularization technique makes the autoencoder learn more optimal latent spaces.
Some representative examples are shown in Figure 9 The trend of the experimental results is similar the experimental results in Section 5.1.2. For all five results, the gray ellipses are aligned well with the decision boundary in the vanilla autoencoder (we show the decision boundary in Figure g while is not included the main manuscript due to lack of space). In the regularized autoencoder. these gray ellipses (or Riemannian metrics) try to become isotropic, boundaries get widened. As a result, different class clusters become farther away from each other. The interpretations about results GMM and pairwise distances are same as the ones in Section 5.1.2.
the gaps on the decision
E.1.2 LINEAR INTERPOLATIONS USING REGULARIZED AUTOENCODERS
The generated point clouds from the representative intra-class linear interpolants between two cylinders and two cones with the regularized autoencoders with the Euclidean metric and info- Riemannian metric are drawn in Figure 10
E.2 STANDARD BENCHMARK DATASET
E.2.1 PERFORMANCE ANALYSIS WITH VARYING REGULARIZATION COEFFICIENTS
Figure II shows graphs of the classification accuracy versus reconstruction error for the trained AEs measured on ModelNet datasets, for range of regularization coefficients. The reconstruction er- is measured by the modified Chamfer distance as (Yang al., 2018b). Compared vanilla metric (blue) show overall
autoencoders (red), regularized autoencoders under the higher classification accuracy regardless of the regularization coefficients. At the same time, they do not significantly increase the reconstruction eтror. On the other hand, when comparing the per- formance of regularized autoencoders under the Euclidean metric (green), most of these are clearly inferior to the vanilla autoencoder; the others perform worse. Overall, regularization under the Soo-Riemannian metric much more robust to the choice of regularization coefficients compared to using the Euclidean metric.
The linear SVM classification accuracy and reconstruction enror (modified Chamfer distance) ac- cording regularization coefficient are shown in Table and Table The tables also arranged
21

---

Under review as a conference paper at ICLR 2022
Latent space
Decision boundary
Riemannian metric
Pairwise distances
GMM
E
Vanilla
Short
Normal
Tall
Regular
Vanilla
Short Normal Normal
Regular
Vanilla
Normal Tall
Normal
Regular
Vanilla
Normal Tall Short
Regular
Vanilla
Tall
Tall
Tall
Regular
Figure 9: The representative five examples of the regularization experiments the synthetic dataset. From left to right: latent spaces, decision boundary according to the color assigning method intro- duced in Appendix D.2, latent spaces with equidistant ellipse (f2](z 2 ) (z=")(=-2") 1} for center 2²) centered on some selected points and sampled points from interspaces, Gaussian Mixture Model (GMM) fitting results, and the heat map of the pairwise Fuclidean distances in the latent space of all test data. For each experiment, the upper figure is vanilla autoencoder trained without regularization, while the lower figure is trained with regularization.
according to autoencoder models (FcNet vs. Foldingnet vs. PointCapsNet vs. DGCNN-FcNet, and regularization types (Vanilla Fuclidean vs. info-Riemannian).
22

---

Under review as a conference paper at ICLR 2022
Figure 10: The generated point clouds from the linear interpolants the regularized autoencoders with the Euclidean metric (Upper) and info- Riemanniann metric (Lower).
Info- Info-Riemannian metric
Euclidean metric
23

---

Under review as a conference paper at ICLR 2022
Modelnet40
Modelnet10
Classification acc. (%)
90
Classification acc. (%)
94.0
93.5
FcNet
FoNet
87
93.0
86
0.030
0.032
0,034
0.030
0.036
Reconstruction error
Reconstruction error
Modelnet40
Modelnet10
Classification acc. (%)
90.0
Classification acc. (%)
94.5
89.5
94.0
FoldingNet +
89.0
FoldingNet +
93.5
FoldingNet
88.0
87.5
93.0
0.029
0.030
0.031
0.032 0.033
0.030
0.032
0.034
0.036
Reconstruction error
Reconstruction
Modelnet40
Modelnet10
Classification acc. (%)
Classification acc. (%)
94
85
92
PointCapsNet PointCapsNet
PointCapsNet
90
75
88
0.05
0,10
0.050
0.075
0.100
0.125
Reconstruction error
Reconstruction error
Modelnet40
Modelnet10
Classification acc. (%)
Classification acc. (%)
90
DGCNN--FcNet +
94
DGCNN-FcNett DGCNN-FcNet
88
93
0.028
0.030
0.032
0.034
0.0300
0.0325
0.0350
Reconstruction error
Reconstruction error
Figure 11: Graphs of classification accuracy versus reconstruction eтror measured ModelNet datasets. More transparent markers have larger coefficients X; detailed values are Table and Table 10
F.2.2 LEARNING CURVES FOR NOISY POINT CLOUD DATA
Figure 12 shows how the linear SVM classification accuracy and reconstruction error (modified Chamfer distance) evolve as the training proceeds, where datasets are ModelNer10 and ModelNet40 and noise levels are 1 1%, 5%, 10%, and 20% (details about noise are in Appendix D.4). Compared to vanilla autoencoders (light colored lines), regularized autoencoders under Info-Riemannian metre (dark colored lines) show overall higher classification accuracy, while they show similar levels of reconstruction errors. The increase in classification accuracy becomes more pronounced as the noise level increases. Especially, when the noise level is 10% and 20%, the classification accuracy of the vanilla autoencoder and regularized autocncoder under Euclidean metric gradually decreases as the learning progresses (as the epoch increases). However, such phenomenons do not appear in the
24

---

Under review as a conference paper at ICLR 2022
regularized autoencoders under the Rifonnmmnnnian metric. This result implies that method is very advantageous situations where there noise in the data.
FcNet
FcNet + E
FcNet +
ModelNet40
ModelNet10
90
0.07
Reconstruction error
Classification acc. (%)
95
90
0.07
error
Noise
Classification acc.
80
0.06
0.06
1%
80
0.05
75
0.04
70
0.04
50
65
0.03
0.03
100
200
300
400
500
100
200
300
400
500
Epoch
Epoch
Classification (%)
0.060
Reconstruction error
Classification acc. (%)
90
0.060
Reconstruction error
0.055
85
0.055
5%
0.050
80
0.050
0.045
75
0.045
70
0.040
0.040
100
200
300
400
500
0
100
200
300
400
500
Epoch
Epoch
0.062
Classification (%)
0.060
Coonstrucen error
Classification acc. (%)
0.062
Sonnnstuuct error
75
0.058
0.060
70
0.058
0.056
10%
0.056
0.054
60
0.054
0.052
70
0.052
0.050
65
0.050
100
200
300
400
500
100
200
300
400
500
Epoch
Epoch
Classification (%)
0.076
Ceecoonan erro
Classification acc. (%)
0.076
8 65
0.074
0.074
0.072
0.072
20%
0.070
Reconstructt
50
60
85
0.068
0.068
50
0.066
0.066
0.064
40
0.064
100
500
Epoch
Epoch
Figure 12: Learning curves of classification accuracy and reconstruction error measured on Model-
Net datasets (ModelNet40 and ModelNet10) according to the noise levels (1%, 5%, 10%, and 20%). In each plot, the light colored lines are the result of the non-regularized autoencoders (i.c., FcNet), and the dark colored lines are the result of the regularized autoencoders (i.e., FeNet E and FeNet I).
E.2.3 EXPERIMENTAL RESULTS ON SEMI-SUPERVISED CLASSIFICATION
We train FoNer whose the latent space is 512-dimensional with and without regularization. The training configuration the same with the case of Appendix D.3 except the following: the regular- ization term of the regularized autoencoder (i.e., FeNet + D is multiplied by À 8000. In this case, when we training linear SVM classifier, we only use the different numbers training data (1%, 5%, 10%, and 50% of the overall training data).
25

---

Under review as a conference paper at ICLR 2022
Table 7 shows comparison of transfer classification accuracy according the percentage of train- ing label used. The overall trend is similar to the results the experiment with noise in Section 5.2 and Appendix E.2.2. Regularized autoencoders (i.e., FcNet + I) show overall higher classification accuracy compared to vanilla autoencoders (i.e., FcNet), while their reconstruction errors are not significantly different to vanilla autoencoders'. Moreover, the increase in classification accuracy becomes more pronounced as the label rate decreases. In other words, our performance is more effective as the number of labels decreases. Figure 13 shows how the linear SVM classification accuracy and reconstruction eπrоr evolve as the training proceeds, where label rate levels are 1 1,, 5%, 10%, and 50%. Also, similarly, when the label vanilla autoencoder gradually decreases the learning progresses (as the epoch increases), but such phenomenons do not appear in the regularized autoencoders. This result implies that our method also very advantageous in semi-supervised settings.
is 1%, the classification accuracy of the
Table 7: Classification accuracy by transfer learning for ModelNet10 (MN10) and ModelNet40 (MN40) from ShapeNet under the the percentages of labeled training data linear SVM classifier (50%, 10%, 5%, and 1%).
MN40
MNI0
50%
10%
5%
1%
50%
10%
5%
1%
FeNet
85.7%
78.0%
70.6%
50.3%
91.7%
90.1%
87.2%
74.1%
FeNet + (ours)
87.9%
81.6%
76.8%
57.4%
93.2%
91.2%
88.3%
78.
26

---

Under review as a conference paper at ICLR 2022
FcNet
FcNet
ModelNet40
ModelNet10
0.08
Reconstruction error
Classification acc. (%)
95
90
0.08
dssonstruction error
Label
0.07
85
Classification
0.06
80
0.06
50 %
75
60
0.05
0.05
0.04
65
0.04
60
40
0.03
55
0.03
100
200
300
400
500
100
Epoch
Epoch
Classification acc. (%)
0.08
Reconstruction error
Classification acc. (%)
0.08
Reconstructinn erTor
70
0.07
80
0.07
0.06
0.06
10%
50
0.05
0.05
0.04
60
0.04
40
0.03
0.03
100
200
300
400
500
100
200
300
400
500
Epoch
Epoch
Classification acc. (%)
0.08
Reconstruction error
Classification acc. (%)
0.08
Reconstruction error
70
0.07
80
0.07
5%
60
0.06
70
0.06
50
0.05
0.05
60
0.04
0.04
50
0.03
100
200
300
400
500
100
200
Epoch
Epoch
80
Reconstruccio connttuuciion error
0.08
0.07
0.07
1%
Classification
50
0.06
Classification
65
0.06
Reconstruction
0.05
0.05
40
0.04
0.04
35
0.03
50
0.03
100
200
300
400
500
100
200
300
400
500
Epoch
Epoch
Figure 13: Learning curves of classification accuracy and reconstruction error measured on Model- Net datasets (ModelNet40 and ModelNet10) according to the label rates (50%. 10%, 5%, and 1 1%). each plot, the light colored lines are result of the non-regularized autoencoders (i.e., FcNet), and the dark colored lines the result of the regularized autoencoders (i.e., FeNet I).
F COMPUTATIONAL ASPECT
First of all, the proposed info-Riimannian metric
(x)((X) Σ
K( ZumZi Tamai
(r,
[x) - 24 TT 44
K
Z.c.
requires to compute all pairwise distances between points in X. This can be computed in parallel using GPU, and it requires a similar level of computation to that of calculating the widely-used average Hausdorff distance metric between point cloud data.
27

---

Under review as a conference paper at ICLR 2022
Table 8: Ellapsed time per one epoch when training FeNet, FcNet + E, and FeNet
MODEL TIME (s) 90.97
FcNet
FeNet + 146.27
FoNet + 287.51
On the other hand, since Hiskt a huge 4D tensor, there may be a memory issue, but fortunately, there is need to store this tensor for both autoencoder application tasks studied in this paper: (i) geodesic interpolation and (ii) learning optimal latent space coordinates. Because the calculations commonly required perform both tasks be done without storing the tensor Hijki entirely. Specifically, the commonly-requiee calculations are as follows: given a neural network decoder m RaxD and latent value R with tangent vector (v² s.rn,, wm IR" need
to compute
Σ (F) (3)0
ij.kd.a.bb
where Jy is the Jacobian of f. This can be memory-efficienty done described in the pytorch-style pseudocode:
def
htt._kiieemmniaaaaeeelattttooooo ==*
2, sigma)
input: decoder function 'f'
torch. nn Module, output size:
boatch_size, num num_pats,, x x_dim) (batch_size, latent_dim) (batch_size, latent_dim)
latent value '2'
tangent vector 'v' bandwith 'sigma'
scalar (batch_size)
output: velocity square 'v_sq'
10
Jacobian vector product
x, torch. autchaautograd. .untooraa..uucttinaaa.... functicnal, (f, 2,
13
14
calculate Info-Riemanian metric
15
batch_size, num_pts, x_dim x.size()
delta_X (x.unsqueeze
x.unsqueeze / sigma
17
K_eelta_X Kernel (dita x_dim)]
18
d_detta X K_delta X.deelaax.vieett...e.. viee (batchhhs_sssiee,, num num_pts, num pts_ __eett__X K_delta_X X_delta_X. X. sum (dim=2) unsqueeze (2)
19
summation
22
v_sq torch, cinsum (' nxi, nxij, nij nx', d_eltt_ delta JV)
23
v_sg = torch, cinsum (' nx, nx -> n', v_59, v_sg)
24
return v_sq / (sigma ** 2)
def Kernel (
27
=**
28
input: vector 'u'
(batch_ssk size, vec siatch_size)
output: kernel values u
31
vec_dim u.size (1)
32
k_ 12 (veo_dim/2) terch. (-torch. norm (u, dim=1) x+222
33
return ku
Listing 1: Pytorch-style pseudocode for the calculation of velocity square under info-Riemannian metric
For training the regularized autoencoders, it is not only required compute the above quantity but also back prok-proe through it. In the following, we report how costly adding the regularization term is, compared to the vanilla autoencoder case. We measure the ellapsed time per one training epoch under GeForce RTX 3090. For representative examples, we use FcNet, FeNet + E, and FeNet I, and the ellapsed tims for each method is shown in Table
28

---

Under review as a conference paper at ICLR 2022
G DIFFERENT KERNEL CHOICES FOR DENSITY ESTIMATIONS
In this section, we qualitatively analyze how the statistical representation can change when a kernel function other than the Gaussian kernel function is used. As representative ones, compare the Gaussian, Uniform, Triangular, Epanechnikov, and Sigmoid kernel functions. All but Gaussian and Sigmoid kernel functions have the finite support. For visualization, comparison is made using 2d point clouds, and data in the shape of circles, ellipses, squares, stars, and hearts are used.
Figure 14 visualizes the probability distributions. Basically, the estimated probability distributions are not that different. However, when using kernel functions with finite support, estimated distribu- tions are not very smooth compared to using Gaussian Sigmoid kernel function that have infinite support. Overall, it is difficult to say definitively which choice is better, but we choose the one that produces smooth estimations, the Gaussian kernel.
Point Cloud
Gaussian
Uniform
Triangular
Epanechnikov
Sigmoid
Figure 14: Kernel density functions of different point cloud data (circle, ellipse, square, heart, and star) using different kernels (Gaussian, uniform, triangular, Epanechnikov, and sigmoid). We set the common bandwidth of kernels a MED, where MED denotes the median of the distances between the points in the point cloud and their nearest points.
29

---

Under review as a conference paper at ICLR 2022
Table Classification accuracy and reconstruction enror according to regularization coefficient. The table is also arranged according to model (FeNet vs. FoldingNet) and regularization type (Vanilla vs. Euclidean vs, info- mioo-iimaanian). For Riemannian metric cases, the regularization coefficients used in the actual experiments are x² times À shown in the table.
MODEL
METRIC
MD40 acc
MD40 recon
MD10 acc
MD10 recon
FeNet
Euclidean
0.0001
89.343598
0.029069
92.951542
0.029527
FeNet
Euclidean
0.0010
88.330632
0.030464
93.612335
0.030684
FeNet
Euclidean
0.0100
88.249595
0,029877
93.722467
0,030054
FeNet
Euclidean
0.1000
86.993517
0,029878
92.841410
0.030900
FcNet
Euclidean
1.0000
87.115073
0.031966
92.951542
0.034272
FeNet
Fuclidean
10.0000
86.709887
0.031523
92.400881
0.032318 18
FeNet
Euclidean
100.0000
85.696921
0.031841
92.511013
0.035172
FoNet
Euclidean
1000,0000
85.899514
0.035089
92.400881
0.036145
FeNet
Euclidean
10000.0000
86.345219
0.034149
92.400881
0.034811
FeNet
Riemannian
100.0000
89. $9.586710 10
0.029032
94.052863
0.029435
FeNet
Riemannian
500.0000
89 $9.829822
0.028944
94,273128
0.029928
FeNet
Riemannian
1000.0000
89.951378
0.028864
93.722467
0.029430
FeNet
Riemannian 2000.0000
90.194489
0.029165
94.052863
0.029606
FeNet
Riemannian 8000.0000
90.397083
0.028869
93.722467
0.028944
FeNet
Riemannian 10000.0000
$9.991899 896
0.029603
94.162998
0.030278
FeNet
Riemannian 20000.0000
89.748784
0.028980
93.832599
0.029412
FoNet
Riemannian
10000.0000
89 89.667747
0.029161
93.392070
0.028894
FeNet
Riemannian
100000.0000
89. 89.748784
0.028961
93.942731
0.030085
FeNet
Vanilla
0.0000
88.330632
0,028938
53.502203
0.029895
FoldingNet
Euclidean
0.0001
88. 38.897893
0.030540
94.162996
0.029130
Not
Euclidean
0.0010
87 844408
0.031623
94.273128
0.031726
FoldingNet
Euclidean
0.0100
87 52029
0.029126
93.612335
0.032219
FoldingNet
Euclidean
0.1000
87 641811
0.029391
93.722467
0,030318
FoldingNet
Euclidean
1.0000
88. 12809
0.030564
93.171806
0.031435
FoldingNet
Euclidean
10.0000
88 88.2901 13
0.032129
93.171806
0.032975
NoldingNet
Euclidean
100.0000
88 $8.087520
0.032728
94.383260
0.032885
FoldingNet
Euclidean
1000.0000
87. 722852
0.033305
92.951542
0.036244
FoldingNet
Euclidean
10000.0000
87 844488448
0.033546
93. 502203
0.036820
Not
Riemannian
100.0000
89 69.667747
0.029467
94.052863
0.030789
NoldingNet
Riemannian
500.0000
90.113452
0,029916
94.052863
0.029346
NoldingNet
Riemannian
1000.0000
89, 89.870340
0.030090
94.493392
0.029996
FoldingNet
Riemannian
2000.0000
89.546191
0.030471
94.273128
0.031142
FoldingNet
Riemannian
8000.0000
89.627229
0.0289499
94. 162999
0.029745
FoldingNet
Riemannian
10000.0000
89.505673
0.029926
94.052863
0.033548
FoldingNet
Riemannian
20000.0000
89.384117
0,032798
94.162996
0.029463
FoldingNet
Riemannian
50000.0000
89.708266
0.029262
94,273128
0.030002
FoldingNet
Riemannian
100000.0000
89.262561
0.029511
94.162996
0.030355
FoldingNet
Vanilla
0.0000
89.343598
0.029599
93.722467
0.030528
30

---

Under review as a conference paper at ICLR 2022
Table 10: Classification accuracy and reconstruction according regularization coefficient.
The table is also arranged according to model (PointCapsNet vs. DGCNN-FcNet) and regularization type (Vanilla vs. Euclidean vs. anfo-Riemannian). For Riemannian metric cases, the regularization coefficients used in the actual experiments are x² times shown in the table.
MODEL
METRIC
MD40 acc
MD40 recon
MD10 acc
MD10 recon
PointCapsNet
Euclidean
0,0001
88,087520
0,039522
93.722467
0,043112
PointCapsNet
Fuclidean
0.0010
87.601297
0,046227
93.171806
0.047749
PointCapsNet
Fuclidean
0.0100
87.155592
0.058326
92.621145
0.060827
PoinCCasNNet
Fuelidean
0.1000
86.628849
0.070735
91.519824
0.072615
PointCapsNet
Euclidean
1.0000
85.858995
0.087722
91.299559
0.087930
PointCapsNer
Euclidean
10,0000
83.954619
0,110708
90.638767
0,107402
PointCapsNet
Euclidean
100.0000
79.659643
0.124266
88.215859
0.115549
PointCaps Net
Euclidcan
1000.0000
76. 823339
0.128694
88.546256
0.121284
PointCapsNet
Euclidean
10000.0000
75.567261
0.130566
88.325991
0.121398
Point PointCapsNet
Riemannian
100,0000
88 492707
0.035034
93 93,942731
0.042224
PointCapsNet
Riemannian
1000,0000
88. 168558
0.035816
93.942731
0.039131
PointCapsNer
Riemannian
2000.0000
87.884927
0.036824
93.392070
0,040169
PointCapsNet
Riemannian
5000.0000
87.884927
0.035990
93.832599
0.039729
PointCapsNet
Riemannian
8000.0000
87.641815
0.037986
93.392070
0.040947
PointCaps
Riemannian
10000,00K
87.763371
0.037507
93.722466
0,040428
PointCapsNet
Riemannian
20000.0000
87.763371
0.038454
93.281938
0,048909
Point PointCapsNer NepsNet
Riemannian
50000.0000
87.641815
0.042233
93.281938
0.044333
PoinCCapsNet
Riemannian
100000.0000
87.317666
0.046607
93.171806
0.051187
PointCapsNet
Vanilla
0.0000
87.155592
0.033936
93.612335
0.037172
DGCNN-FaR -eNeet
Euclidean
0,0001
89.910859
0.028852
94.052863
0,029049
DGCNN-FcNet
Euclidean
0.0010
89.546191
0.029489
94.383260
0,029480
DGCNN-FcNet
Euclidean
0.0100
88 88.492707
0.030448
93.061674
0.030531
DGCNN- FcNet
Euclidean
0.1000
87.884927
0.030046
93.392070
0.030966
DGCNN-FcNE
Euclidcan
1,0000
87. 87.520259
0.030605
93,502203
0.034671
AGCNN-FcNG
Euclidean
10,0000
87.196110 10
0.032174
92.841410
0.032151
DGCNN-FoNet
Euclidean
100.0000
87.277147
0.032385
93.171806
0,033750
DGCNN-FcNet
Euclidean
1000.0000
87 77.682334
0.033359
93.171806
0.033906
DGCNN-FcNet
Euclidean
10000.0000
86.385737
0.034471
92.621145
0.036056
DGCNN-FcNet
Riemannian
100.0000
90.397083
0.028761
94.493392
0.028591
DGCNN-FcNet
Riemannian 1000.0000
90.964344
0.028278
94.493392
0,028806
DGCNN-FcNet
Riemannian 2000,0000
90.680713
0.028299
94.493392
0,028832
DGCNN-FcNet
Riemannian
5000.0000
90,883306
0.028520
94.493392
0.029128
DGCNN-FcNet
Riemannian
8000.0000
90.802269
0.028727
95.154185
0.028971
DGCNN-FeNet
Riemannian 10000.0000
90.680713
0.028583
94.383260
0.028820
DGCNN-FeNet
Riemannian 20000.0000
90.559157
0.028677
94.713656
0.029067
DGCNN-FcNet
Riemannian 50000.0000
90.761750
0.029794
94.713656
0.029695
DGCNN-FcNet
Riemannian
100000.0000
90.235008
0.028934
94.052863
0.031275
DGCNN-FcNet
Vanilla
0.0000
90.275527
0.028867
94.493392
0.029454
31


Reviews:
[{"review_id": "vvED9ZlOCOD", "reviewer": "Reviewer_yY6L", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "===STRENGTHS===\n\nNOVELTY\n\nThe proposed approach seems novel, and I am not aware of other works that propose a similar approach to equip a point cloud of a Fisher Information Metric. \n\nCLARITY\n\nThe paper is clear, and I had no trouble understanding it. The only parts a bit obscure to me are the paragraphs of Section 4; I would suggest giving the applications a bit more space. For example: how is implemented and solved the optimization problem of \"Geodesic Interpolation\"?\n\nSOUNDNESS AND REPLICABILITY\n\nBy my understanding, the method seems sound; while I have not performed an in-depth check, the passages seem reasonable to me. The appendix provides information to help the replicability of the results, explaining the implementation details.\n\n\n===WEAKNESSES===\n\nEXPERIMENTS\n\nMy major concern about this paper is the experimental setting for the following reasons:\\\na) Quantitative results are provided only on one experiment of shape classification; in this experiment, the comparisons with other methods are \"adopted from references\", and hence they are not directly comparable. I am convinced that the proposed method work in a similar level of performance, but at the moment, some sentences seem over-claims not well-grounded (e.g., \"the classification accuracy of our method is higher in most cases\"). The improvement over the vanilla baselines seems tight; can you provide some elaboration on the \"best\" cases, i.e., in which cases using the regularization produce the greatest improvement?\\\nConcretely: Given the tight margin between the proposed method and the ones taken from other papers, I think that it would be important to reproduce at least two SOTA methods (e.g., PointCaps and Multi-Task) in the exact same setting.\\\nb) The toy dataset is really interesting from the point of analysis, but only qualitative examples are provided. I would also suggest providing some quantitative results in this case that are highly controllable and naturally in correspondence, providing visualization of the reconstruction error. This would give insight into the proposed metric's behavior on different geometrical features (e.g., different curvature and sharpness).\\\nc) Only rigid objects have been shown, all with the same distributions, while one of the paper's claims is that defining probability distribution is \"likely quite natural and intuitive to the user\". I would suggest proposing at least some qualitative examples on different domains and different probability distributions. Are there issues in considering non-rigid objects? Which are other reasonable choices for probability distributions that respect the given assumptions?\n\nCONTEXT\n\nThe paper does not offer a proper context around the work. I would suggest providing at least two sections for related works: one on point clouds as a probability distribution (as stated in the text, this is not a novel idea), and one for defining metrics for point clouds and their latent representations (examples are [1] and [2]). I think that the correct positioning of the paper in the literature would help to clarify the novelty of the work and what it is actually proposing.\n\n\n===MINOR ISSUES AND QUESTIONS===\n\n1) the \"info-Riemannian\" name is introduced twice (page 2 and page 4)\n2) to me it is not completely obvious the term \"velocity\" on page 4; could you give me a better intuition\\definition of it?\n3) I would suggest coloring the point clouds of Figure 4 with a non-constant color to better track the movement of the points; I see the overall structure is better \"preserved\" in IR metric, but I would better understand their movements.\n4) the computational aspect seems untouched; can you provide some timings of your computation, e.g., comparing it with the geodesic metric computation over a mesh? \n\n[1]: Limp: Learning latent shape representations with metric preservation priors, Cosmo et al., 2020\n\n[2]: Point-set Distances for Learning Representations of 3D Point Clouds, Nguyen et al., 2021", "summary_of_the_paper": "The paper proposes treating the point clouds as probability distributions and equipping them with a Fisher Information Metric, providing a Riemannian geometric structure that can be used as regularization in AutoEncoders for training and latent interpolation. The experiments are performed on ModelNet and a toy dataset of toy shapes (cones, cylinders, ellipsoids). ", "summary_of_the_review": "I like the overall idea, and some qualitative results seem promising. However, some aspects require better experimentation and remain a bit obscure, particularly the effectiveness of the method, the positioning among other SOTA approaches, and the behavior in different conditions (e.g., different domains, probability distribution). ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "cNGt_O55n68", "reviewer": "Reviewer_21tj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength: \nI personally find that incorporating the proposed constraint to achieve Fisher information metric in the underlying latent space is a very interesting idea.\n\nAs the authors proposed, the underlying latent space created by the autoencoder does seem to be endowed with the desired metric. The experiments regarding the decision boundary is also interesting, as the data seems to be cleanly separated. \n\nThe task of point cloud classification does not really involve usage of pointwise correspondence, therefore may have worked to the advantage of the authors' proposal.\n\nWeakness:\nThe contribution of the paper is very limited, as it lacks a thorough review of the geometric/statistical representation of point clouds in other fields.\nThe description of representing points as a distribution is accurate, yet has already been explored in registration. The proposal in section 3 itself lacks novelty in that regard.\n\nIn the field of image/point cloud registration, there has already been decades of discussion regarding the usage of geometric and statistical representation, and the advantages of using Euclidean metric and Fisher information metric to compare the sets of points.\nIn the case of point set registration, it is widely known that geometric methods excel at cases where outliers are included, while statistical methods are robust to noise.\n\nSuch discussion has been completely omitted in this proposal, which I find disappointing.\nHere are some of the prior works, but there are many more:\n\nH. Chui and A. Rangarajan. \"A new point matching algorithm for non-rigid registration,\" CVIU 2003\n\nA. Myronenko and X. Song, “Point set registration: Coherent point drift,” TPAMI 2010\n\nB. Jian and B. Vemuri, \"Robust point set registration using gaussian mixture models,\" TPAMI 2010\n\nZ. Zhou et al. “Robust non-rigid point set registration using student’s-t mixture model,” PloS one, 2014\n\nThere has also been discussion regarding merging the two approaches:\n\nF. Li et al, \"Toward a unified framework for point set registration,\" ICRA 2021\n\nThe novelty of the paper lies in section 4, which I did find interesting. However, as it lacks a thorough review, the contribution of this section is insufficient. \nFor example, as stated above, each method is known to have some advantages and disadvantages in registration. I wonder if, in this proposal, the same can be said of cases under a different setting, such as point clouds with added noise, occlusion, etc.\nThe overall novelty lies only in the proposal of embedding the metric in the latent space, therefore thorough analysis is definitely necessary for its contribution to be sufficient.", "summary_of_the_paper": "This paper proposes to represent point cloud data as samples drawn from statistical distribution, and project the data to an underlying statistical manifold on which various analyses can be conducted in a manner that is more favorable than doing so in a regular Euclidean space. \n\nThe authors thoroughly describe how points are converted into samples from a probability density function, and explain the metric that determines the coordinate system on the statistical manifold, which describe the changes to the distribution of point clouds. \nTo introduce this metric to represent point clouds, the authors propose to use this to find minimal geodesic curve between two data in a latent space. They also propose to add a novel constraint to the error metric of autoencoders, so that the encoded latent space possesses the Fisher information metric.\n\nThe effect of this proposal is firstly analyzed by two synthetic experiments. First, the authors show that using the proposal, they are able to find the geodesic curve along the manifold that consists of data with the same label. Then, they demonstrate that the latent space of the constrained autoencoder successfully captures the Riemann metric, as samples of many classes are placed along a single line, or are cleanly separated. \nFinally, the authors transfer the learnt representation between dataset to demonstrate that the proposed metric space is suitable for representation of various point cloud data.", "summary_of_the_review": "Overall, I find the paper to be lacking background analysis, and thus also in novelty. \n\nThe autoencoder with the Fisher information metric as the metric of the latent space is an intriguing proposal. However, as this is the only contribution of the paper, the authors are required to thoroughly compare the geometric and statistical approaches. There has been decades of research that analyzed the advantages of the two representations, especially in the field of image and point set registration. Therefore, such knowledge had to be used in the analysis. As the paper lacks such analysis, I have to lean towards rejection.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "LxR7Li0Igx9", "reviewer": "Reviewer_FLA8", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of representing point clouds as probability distributions and considering those as points in a statistical manifold equipped with the Firsher information metric seems good and well-founded. I am not aware of any previous work that has exactly this take on point cloud analysis.\n\nOverall, I find the idea novel, the presentation clear and interesting.\n\nSome points that the authors can consider:\n\n- a weakness of the method is the estimation of the density. The result is of course very dependent on the chosen kernel density estimator and parameters of the kernels. As far as I could read, the authors use fixed values for the bandwidth without justification for the chosen values. A more comprehensive discussion of the effect of the density estimator and e.g. the bandwidth parameter would strengthen the paper.\n\n- I was at the first read confused about the statement that the mapping h:X\\to S being 1-1, and subsequently that the representation is invariant to permutations. I believe the manifold \\mathcal X can equivalently be considered the set of equivalence classes of point sets under the permutation group so that permutation invariance comes by construction, but perhaps this could be described more explicitly around Proposition 1.\n\n- Proposition 3: Isn't the linear independence assumption satisfied on all reproducing kernel Hilbert spaces? Perhaps the result can be stated much more generally.", "summary_of_the_paper": "The paper concerns data sets of point clouds in Euclidean space and proposes to analyze those as samples from underlying probability distributions. Each point cloud being represented by a probability distribution can now be seen as a point in a statistical manifold on which the Fisher information metric provides a natural Riemannian structure. The authors presents two experiments in this setup where they train autoencoders to represent point clouds, interpolate between point clouds using geodesics, and where the latent representation is trained so that straight line are close to geodesics.", "summary_of_the_review": "I believe the paper presents a novel take on analysis of point clouds.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "A2MXj8PSPfv", "reviewer": "Reviewer_PyPd", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- the proposed framework has potential, it may indeed sometimes (although certainly not in general) be useful to think about point clouds in this way\n- the framework definitely has a certain mathematical elegance\n- experiments in toy data like parametric primitives, Shapenet and Modelnet indicate that for \"perfect\" point clouds the proposed kernel density model is a valid regulariser\n- actually the most promising bit of the proposed view for me: an interesting way to construct a regulariser for (global) point cloud representations, although the paper only scratches the surface and does not deeply explore that aspect\n\nWeaknesses:\n- the work is in my view interesting, but not as \"rigorous\" and momentous as claimed; actually, behind the information-theoretic smoke screen it largely boils down to the popular and obvious standard assumption that a point cloud is a set of samples from an underlying surface, perturbed by (with the proposed kernel, Gaussian) random noise\n- the view as a distribution may be useful in some situations - typically when requiring a \"rough, global\" descriptor of the point cloud, for instance to recognise well-discriminated shape categories; it is almost certainly not a suitable view for many other (arguably more realistic and relevant) tasks - for instance the kernel smoothing will harm any local feature extraction or description; and it will likely also be challenged by point clouds that cover the object of interest incompletely or with varying density - in practice any point cloud that has been sensed, rather than rendered synthetically from a surface model (i.e., from an \"explicit version of the distribution\")\n- nothing is said to substantiate the claim that choosing the distribution is easier than making other reasonable assumptions; I am not sure how to make a reasonable choice for realistic distributions (say, autonomous vehicle Lidar)\n- the kernel density flavour suggests that the method might not scale all that well to realistic point cloud sizes. Perhaps it does, but the experimental setting with <2100 points per object evades the topic\n\nComment (neutral, I would not want to call this a strength or a weakness):\n- the paper sits in a strange place between a theory paper and a practical, algorithmic machine learning paper: a theory paper can be very valuable, but for my taste the theoretical \"contribution\" of the paper is fairly basic, it would have been nice to push further and see what more advanced theoretical analyses and conclusions the point cloud manifold view leads to; on the other hand, the practical \"evidence\" is limited to last-decade toy point clouds - nicely discriminative objects without any background, sampled with perfectly regular, even coverage and low, isotropic noise. If one claims practical utility, then one should nowadays really show at least \"semi-real\" point clouds (e.g., Scannet, semantic3d.net, KITTI, ...)\n", "summary_of_the_paper": "The paper studies 3D point cloud data. The core message is that one can regard a point cloud as a collection  of samples from an underlying distribution and, hence, one can imagine a manifold of point clouds and construct an associated distance metric via the Fisher information. That distance can then be used for certain manipulation and analysis tasks, e.g., to interpolate (\"morph\") between point clouds along the manifold, or to regularise representation learning. A central assumption of the paper is that it is easier to choose a meaningful probability distribution than to make other a-priori assumptions \"ad hoc\" about the point cloud. However it is not discussed how to actually choose a suitable distribution, the paper limits itself to Gaussian kernel density (seemingly also \"ad hoc\" for mathematical convenience).\n", "summary_of_the_review": "A rather theoretical paper, the proposed view is in some sense elegant, but not as revolutionary as claimed. Interesting theoretical or practical developments could perhaps build on the proposed view, although the paper does not push far enough in any directions to get to a real impact. Still, I have no objections against putting the idea out to the community, if the authors tone down their claims and acknowledge that they are throwing out a cute, raw idea; while neither exploring the theoretical implications in much depth nor engineering it into a strong algorithmic tool.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to interpret point cloud data in Euclidean space as samples from some underlying probability distribution. Thus a set of point cloud data can be given the structure of a statistical manifold with a Riemannian metric structure, namely the Fisher Information Metric.
Applications to point cloud autoencoders are then studied.

Reviewers generally agree that the idea of equipping point cloud data with the Fisher information metric is interesting and has potential. However, there are concerns that the theoretical properties of the proposed framework have not been explored in depth. Furthermore, the experimental work should be enhanced to demonstrate the practical utility of the proposed formulation.
Decision: Reject