A Statistical Manifold Framework for Point Cloud Data

1 INTRODUCTION . A growing number of machine learning problems involve data sets in which each data point is a point cloud in RD , e.g. , 3D point cloud obtained by depth cameras . Typical applications include measuring the degree of similarity betweeen two point clouds – the point clouds may be measurements obtained from a depth camera , for example – for which a distance metric on the space of point clouds is needed . Some widely used distance metrics used in this context include the Hausdorff distance ( both the original and averaged versions ) , the chamfer distance ( Hausdorff , 1914 ) , and the earth mover distance ( Rubner et al. , 2000 ) . However , the distance metric measures just one aspect of point cloud data ; other applications may require more advanced concepts and tools . For example , in the case of a moving point cloud , one may wish to measure some of its more dynamic aspects like its velocity or other quantities that require a notion of higher-order derivatives . Applications such as Monte Carlo sampling may require the construction of an isotropic Gaussian distribution on the underlying space , in which case the notion of “ angle ” is needed in addition to distance ( more technically , an inner product on the tangent space is required ( Girolami & Calderhead , 2011 ; Gemici et al. , 2016 ; Mallasto & Feragen , 2018 ) . Related to the above , the idea of interpreting a point cloud as samples from some underlying probability distribution is well-known , and has been applied to problems ranging from point set registration ( Jian & Vemuri , 2005 ; Wang et al. , 2006 ; Myronenko & Song , 2010 ; Hasanbelliu et al. , 2014 ; Zhou et al. , 2014 ; Min et al. , 2018 ; Li et al. , 2021 ) to point cloud de-noising ( Zaman et al. , 2017 ; Luo & Hu , 2021 ) . In these approaches , divergence measures from probability and informa- tion theory have been utilized to compute the similarity between point clouds . While effective for certain applications , these divergence measures still only capture just one aspect of point cloud data , and can not be used to measure other quantities of a more geometric nature . For more advanced applications , a rigorous mathematical characterization of the space of point cloud data is an essential ingredient to a more comprehensive , robust , and correct ( in the sense of being coordinate-invariant and geometrically well-defined ) analysis of the types described above , particularly one based on Riemannian geometry . As such , the first contribution of this paper is a Riemannian geometric structure for the space of point cloud data . The key idea behind our approach draws upon the information geometry framework of ( Amari & Nagaoka , 2000 ; Amari , 2016 ) , in which the space of probability density functions is given the structure of a Riemannian manifold ( the statistical manifold ) , with the Fisher information acting as a natural Riemannian metric ( the Information Riemannian metric , or info-Riemannian metric for short ) . The connection between point cloud data and information geometry is established by constructing a 1-1 mapping from the space of point cloud data to the space of probability density functions , i.e. , a point cloud X = { x1 , ... , xn | xi ∈ RD } is mapped to a density function p ( x ; X ) on RD in a 1-1 fashion as illustrated in Figure 1 . Two case studies involving autoencoders are presented to demonstrate the benefits of our approach . In the first case study , a pre-trained autoencoder is used to encode two 3D point clouds – one representing a cylinder , one a cone – and the minimal geodesic ( or path of shortest length ) with respect to the info-Riemannian metric is then constructed between these two objects . The shape evolution obtained for the info-Riemannian metric is seen to be far more natural and intuitive than that obtained for the straight line interpolant in latent space . In the second case study , we use the info-Riemannian statistical manifold framework to find a set of distortion minimizing latent space coordinates , in the sense that ( Euclidean ) straight lines in the latent space closely approximate minimal geodesics on the statistical manifold . Such a set of coordinates offers a more discriminative representation for the data manifold ( Chen et al. , 2020 ) that results in , e.g. , higher linear SVM classification accuracy vis-á-vis existing state-of-the art methods . Experiments are carried out with both synthetic and standard benchmark datasets ( ShapeNet ( Chang et al. , 2015 ) , ModelNet ( Wu et al. , 2015 ) ) . 2 STATISTICAL MANIFOLDS AND THE FISHER INFORMATION METRIC . We begin by extending the original definition of a statistical manifold as follows : Definition 1 . Given an m-dimensional topological manifold1 Θ and a 1-1 map from Θ to the space of probability density functions θ 7→ p ( x ; θ ) , the image of this mapping , denoted S : = { p ( x ; θ ) |θ ∈ Θ } , is an m-dimensional statistical manifold . In the original definition Θ is taken to be an open subset of Rm ( Amari & Nagaoka , 2000 ; Amari , 2016 ) ) . By endowing S with a Riemannian metric , S can be given the structure of a Riemannian manifold , allowing for lengths , angles , and volumes to be defined on S in a coordinate-invariant manner . The Fisher information metric serves as a natural Riemannian metric on S : the elements ( gij ) of the Fisher information metric G ( θ ) ∈ Rm×m can be expressed as gij ( θ ) : = ∫ p ( x ; θ ) ∂ log p ( x ; θ ) ∂θi ∂ log p ( x ; θ ) ∂θj dx , i , j = 1 , . . . , m , ( 1 ) where θ = ( θ1 , ... , θm ) are local coordinates on S. Defining infinitesimal length on S by ds2 = dθTG ( θ ) dθ , the length of a curve θ ( t ) on S can then be computed as the integral ∫ T 0 ds . Further details on statistical manifolds and the Fisher information metric can be found in , e.g. , ( Amari & Nagaoka , 2000 ; Efron & Hinkley , 1978 ; Rissanen , 1996 ; Han & Park , 2014 ) . 1A topological manifold is a locally Euclidean Hausdorff topological space . 3 STATISTICAL MANIFOLD FRAMEWORK FOR POINT CLOUD DATA . With the above statistical manifold preliminaries , we now construct a Riemannian geometric structure for the space of point cloud data . Section 3.1 defines a statistical manifold from the point cloud data , while Section 3.2 uses the Fisher information metric to construct a Riemannian metric for point cloud data . To keep the definitions and results simple , we shall assume throughout all point cloud data consists of exactly n distinct points in RD , i.e. , each point cloud is of the form X = { x1 , ... , xn |xi ∈ RD , xi ̸= xj if i ̸= j } . The set of all point clouds is denoted X . Later we discuss methods for dealing with point clouds that violate our assumptions . The proofs of propositions in this section are in Appendix B . 3.1 STATISTICAL MANIFOLD OF POINT CLOUD DATA . The core idea for constructing the statistical manifold is to interpret a point cloud X as a set of n samples drawn from some underlying probability density function . Using a kernel density estimator ( Parzen , 1962 ; Davis et al. , 2011 ) , a parametric probability density function p ( x ; X ) can be defined in which X itself is the parameter : Definition 2 . Given a positive kernel function K : RD → R such that ∫ RD K ( u ) du = 1 and a D ×D symmetric positive-definite matrix Σ ( the bandwidth matrix ) , the kernel density estimate p ( x ; X ) : = 1 n √ |Σ| n∑ i=1 K ( Σ− 1 2 ( x− xi ) ) ( 2 ) is said to be a statistical representation of the point cloud X ∈ X . The set of statistical representations is denoted S : = { p ( x ; X ) |X ∈ X } . To ensure that S is a statistical manifold , recall from Definition 1 that the following two conditions need to be satisfied : ( i ) X is a topological manifold ; ( ii ) A 1-1 mapping h : X → S , X 7→ p ( x ; X ) must be defined . The first condition can be satisfied with the “ distinct points ” assumption : Proposition 1 ( Corollary 2.2.11. in ( Knudsen , 2018 ) ) . The set of point clouds in which each point cloud is a set of n distinct points of dimension D , is an nD-dimensional topological manifold . To satisfy the second condition , additional assumptions are needed . The following proposition provides a sufficient condition for h to be 1-1 : Proposition 2 . If the set of functions { K ( Σ− 12 ( x− xi ) ) |xi ∈ F } are linearly independent2 for any arbitrary finite subset F ⊂ RD with |F| ≤ 2n , the mapping h : X → S is 1-1 . With the above proposition , any kernel function that satisfies the linear independence condition is sufficient to ensure the existence of a 1-1 mapping h. For our purposes the standard and widely used normal kernel function satisfies the linear independence condition . Proposition 3 . Under the distinct points assumption , the standard ( multivariate ) normal kernel function K ( u ) = 1√ ( 2π ) D exp ( −u Tu 2 ) ( 3 ) with the scaled identity bandwidth matrix , i.e. , Σ = σ2I , satisfies the linear independence condition in Proposition 2 . From Propositions 2 and 3 we have established that , under the distinct points assumption and using the standard normal kernel function , the mapping h : X → S is 1-1 ; S can therefore be given the structure of statistical manifold . While other choices of kernel function are possible , throughout the remainder of the paper we use the standard normal kernel function . Figure 2 illustrates statistical manifold representations of some example point clouds . 2The linear independence of a set of functions implies that only a trivial linear combination of the functions equals the zero function . 3.2 INFORMATION RIEMANNIAN METRIC FOR POINT CLOUD DATA SPACE . We now equip the point cloud statistical manifold S with the Fisher information metric , which we refer to as the info-Riemannian metric and denote by H. The first task is to define a local coordinate system on the space of point clouds X . Toward this end , we use the matrix representation X ∈ Rn×D of a point cloud X . Observe that the matrix representation is not unique ; given an n×n permutation matrix P ∈ Rn×n , then X and PX represent the same point cloud X. Fortunately , this does not cause problems since p ( x ; X ) is defined in a permutation-invariant way , i.e. , p ( x ; X ) = p ( x ; PX ) for any n×n permutation matrix P . Throughout we use italics to denote local coordinate representations , e.g. , X has local coordinates X ∈ Rn×D , the tangent vector V ∈ TXX has local coordinates V ∈ Rn×D . The info-Riemannian metric H can be expressed in local coordinates coordinates X as follows : Hijkl ( X ) : = ∫ p ( x ; X ) ∂ log p ( x ; X ) ∂Xij ∂ log p ( x ; X ) ∂Xkl dx , ( 4 ) for i , k = 1 , ... , n and j , l = 1 , ... , D. Given two tangent vectors V , W ∈ TXX with respective matrix representations V , W ∈ Rn×D , their inner product is then computed as follows : ⟨V , W⟩X : = n∑ i , k=1 D∑ j , l=1 Hijkl ( X ) V ijW kl . ( 5 ) We note that the coordinate expression of the info-Riemannian metric Hijkl ( X ) results in a permutation-invariant inner product , i.e. , ∑ Hijkl ( X ) V ijW kl = ∑ Hijkl ( PX ) ( PV ) ij ( PW ) kl for any n× n permutation matrix P , showing that the metric is geometrically well-defined . Using the standard normal kernel function , the coordinate expression of the info-Riemannian metric Hijkl has a simple analytic expression as follows : Proposition 4 . With the standard ( multivariate ) normal kernel function and the bandwidth parameter σ , the information Riemannian metric is given by Hijkl ( X ) = ∫ p ( x ; X ) K ( x−xiσ ) K ( x−xk σ ) ( ∑n m=1 K ( x−xm σ ) ) 2 [ ( x− xi ) ( x− xk ) T σ4 ] jl dx , ( 6 ) Figure 3 shows that , given two moving point cloud data whose velocity matrices have equal Euclidean norm ( i.e. , ||V||2 = ∑n i=1 ∑D j=1 V ijV ij ) , the velocity norms under the info-Riemannian metric are significantly different : the velocity A has a value of 0.2626 , while the velocity B has a value of 2.2 × 10−8 . In particular , observe that the tangential velocity in the case B , which does not change the overall distribution of the point cloud , has a very small velocity norm under the info-Riemannian metric as it should . As another illustrative example that highlights the difference between the info-Riemannian metric and Euclidean metric ( i.e. , ⟨V , W⟩ : = ∑n i=1 ∑D j=1 V ijW ij ) , Figure 4 shows random walks for point cloud data under these metrics . Consider a randomly sampled and normalized velocity matrix under Euclidean metric . 3D velocity vectors in the sampled velocity matrix are equally likely to point any direction . On the other hand , 3D velocity vectors in the sampled velocity matrix under info-Riemannian metric are more likely to point tangential directions ( e.g. , more likely the case B than the case A in Figure 3 ) . As a result , unlike the standard Euclidean metric ( top ) , the infoRiemannian metric ( bottom ) produces random walks that stay close to the initial sphere without significant changes in its overall distribution pattern .

Summary: This paper proposes to represent point cloud data as samples drawn from statistical distribution, and project the data to an underlying statistical manifold on which various analyses can be conducted in a manner that is more favorable than doing so in a regular Euclidean space.   The authors thoroughly describe how points are converted into samples from a probability density function, and explain the metric that determines the coordinate system on the statistical manifold, which describe the changes to the distribution of point clouds.  To introduce this metric to represent point clouds, the authors propose to use this to find minimal geodesic curve between two data in a latent space. They also propose to add a novel constraint to the error metric of autoencoders, so that the encoded latent space possesses the Fisher information metric.  The effect of this proposal is firstly analyzed by two synthetic experiments. First, the authors show that using the proposal, they are able to find the geodesic curve along the manifold that consists of data with the same label. Then, they demonstrate that the latent space of the constrained autoencoder successfully captures the Riemann metric, as samples of many classes are placed along a single line, or are cleanly separated.  Finally, the authors transfer the learnt representation between dataset to demonstrate that the proposed metric space is suitable for representation of various point cloud data.