Title: Grassmann Manifold Flows for Stable Shape Generation
Venue: NeurIPS 2023
Accepted: yes
Keywords: Generative Models

Geometric Deep Learning

Normalizing Flows

Shape Analysis

Grassmann Manifold
Abstract: Recently, studies on machine learning have focused on methods that use symmetry implicit in a specific manifold as an inductive bias.
Grassmann manifolds provide the ability to handle fundamental shapes represented as shape spaces, enabling stable shape analysis. 
In this paper, we present a novel approach in which we establish the theoretical foundations for learning distributions on the Grassmann manifold via continuous normalization flows, with the explicit goal of generating stable shapes.
Our approach facilitates more robust generation by effectively eliminating the influence of extraneous transformations, such as rotations and inversions, through learning and generating within a Grassmann manifold designed to accommodate the essential shape information of the object.
The experimental results indicated that the proposed method could generate high-quality samples by capturing the data structure.
Furthermore, the proposed method significantly outperformed state-of-the-art methods in terms of the log-likelihood or evidence lower bound.
The results obtained are expected to stimulate further research in this field, leading to advances for stable shape generation and analysis.

Grassmann Manifold Flows for Stable Shape Generation

Abstract

Recently, studies on machine learning have focused on methods that use symmetry implicit in a specific manifold as an inductive bias. Grassmann manifolds provide the ability to handle fundamental shapes represented as shape spaces, enabling stable shape analysis. In this paper, we present a novel approach in which we establish the theoretical foundations for learning distributions on the Grassmann manifold via continuous normalization flows, with the explicit goal of generating stable shapes. Our approach facilitates more robust generation by effectively eliminating the influence of extraneous transformations, such as rotations and inversions, through learning and generating within a Grassmann manifold designed to accommodate the essential shape information of the object. The experimental results indicated that the proposed method could generate high-quality samples by capturing the data structure. Furthermore, the proposed method significantly outperformed state-of-the-art methods in terms of the log-likelihood or evidence lower bound. The results obtained are expected to stimulate further research in this field, leading to advances for stable shape generation and analysis.

1 Introduction

Many machine learning algorithms are designed to automatically learn and extract latent factors that explain a specific dataset. Symmetry is known to be an inductive bias (that is, prior knowledge other than training data that can contribute significantly to learning results) for learning latent factors (Cohen & Welling (2016, 2017); Weiler & Cesa (2019); Satorras et al. (2021); Bronstein et al. (2021); Puny et al. (2022)). Moreover, they exist in many phenomena in the natural sciences. If a target M is invariant 1 when the operation g ∈ S designated by S applied to M , M has symmetry S . For example, a sphere remains a sphere even if a rotation R is applied; a symmetric shape remains symmetric even if a left-right reversal is applied.

Recent studies have focused on methods incorporating symmetry into models based on equivariance and invariance, when the data space forms a non-Euclidean space (a sphere S n or a special unitary group SU ( n ) ) (Cohen et al. (2018, 2019b); Graham et al. (2020); Haan et al. (2021); Boyda et al. (2021a)). Among them, discriminative and generative models for subspace data (e.g., shape matrices of point cloud data such as three-dimensional molecules and general object shapes) have been proposed as viable approaches using Grassmann manifolds, which are quotient manifolds obtained by introducing a function that induces invariance with respect to orthogonal transformations into

a set of orthonormal basis matrices namely Stiefel Manifold. Numerous studies have confirmed their effectiveness (Liu et al. (2003); Hamm & Lee (2008); Turaga et al. (2008); Harandi et al. (2011); Fan et al. (2011); Lui (2012); Huang et al. (2015, 2018); Souza et al. (2020); Haitman et al. (2021); Doronina et al. (2022); Souza et al. (2022)). Shape theory studies the equivalent class of all configurations that can be obtained by a specific class of transformation (e.g. linear, affine, projective) on a single basis shape (Patrangenaru & Mardia (2003); Sepiashvili et al. (2003); Begelfor & Werman (2006)), and it can be shown that affine and linear shape spaces for specific configurations can be identified by points on the Grassmann manifold. By applying this model to inverse molecular design for stable shape generation to discover new molecular structures, they can contribute to the development of drug discovery, computational anatomy, and materials science (Sanchez-Lengeling & Aspuru-Guzik (2018); Bilodeau et al. (2022)).

Continuous normalizing flows (CNFs; Chen et al. (2018); Grathwohl et al. (2019); Lou et al. (2020); Kim et al. (2020); Mathieu & Nickel (2020)) are generative models that have attracted attention in recent years along with the variational auto-encoders (V AE; Kingma & Welling (2014)), generative adversarial networks (GAN; Goodfellow et al. (2014)), autoregressive models (Germain et al. (2015); Oord et al. (2016)), and diffusion models (Sohl-Dickstein et al. (2015); Ho et al. (2020); Song et al. (2021); Huang et al. (2022); De Bortoli et al. (2022)). The CNF is a method with theoretically superior properties that allow rigorous inference and evaluation of the log-likelihood. It can be trained by maximum likelihood using the change of variables formula, which allows to specify a complex normalized distribution implicitly by warping a normalized base distribution through an integral of continuous-time dynamics. In this paper, we present a novel approach with the explicit goal of generating stable shapes. We employ the CNF to achieve stable shape generation on a Grassmann manifold; however, previous studies have lacked the theoretical foundation to handle the CNF on the Grassmann manifold. To construct this, we focused on the quotient structure of a Grassmann manifold and translated the problem of flow learning on a Grassmann manifold into that of preserving equivariance for orthogonal groups on a Stiefel manifold, which is its total space. The contributions of this study are as follows.

A theory and general framework for learning flows on a Grassmann manifold are proposed. In our setting, we can train flows on a Grassmann manifold of arbitrary dimension. To the best of our knowledge, this is the first study in which a CNF was constructed on a Grassmann manifold via a unified approach, focusing on the quotient structure.

The validity of the proposed approach was demonstrated by learning densities on a Grassmann manifold using multiple artificial datasets with complex data distributions. In particular, orthogonally transformed data were proven to be correctly learned without data augmentations by showing that untrained transformed (i.e., rotated or mirrored) data can be generated from the trained model. The model was evaluated on multiple patterns of training data and performed well with a small amount of training data.

The effectiveness of our approach was confirmed by its state-of-the-art performance in a molecular positional generation task.

2 Related Works

Symmetry-based Learning The concept of equivariance has been studied in recent years to leverage the symmetries inherent in data (Cohen & Welling (2017); Kondor & Trivedi (2018); Cohen et al. (2018, 2019b,a); Finzi et al. (2020); Haan et al. (2021)). Early work by Cohen & Welling (2017) indicated that when data are equivariant, they can be processed with a lower computational cost and fewer parameters. In the context of CNFs (Rezende & Mohamed (2015); Chen et al. (2018); Grathwohl et al. (2019)), which are generative models, Rezende et al. (2019), Köhler et al. (2020) and Garcia Satorras et al. (2021) proposed equivariant normalizing flows to learn symmetric densities on Euclidean spaces. Furthermore, the symmetries that appear in learning densities on a manifold were introduced by Boyda et al. (2021b) and Katsman et al. (2021) as a conjugate equivariant flow on SU ( n ) , which is a quotient manifold, for use in lattice gauge theory. However, a normalizing flow on a Grassmann manifold capable of handling subspace data has not yet been established.

Shape Data Analysis and Other Applications with Subspace on the Grassmann Manifold k -dimensional shape data (Begelfor & Werman (2006); Yoshinuma et al. (2016); Haitman et al. (2021);

Doronina et al. (2022)) such as point clouds are essentially represented as subspace (shape space (Sepiashvili et al. (2003); Srivastava et al. (2005); Begelfor & Werman (2006); Yataka & Fukui (2017)) data on a Grassmann manifold. Furthermore, subspace data can be obtained on many types of data and can provide advantages such as practicability and noise robustness. For example, multiview image sets or video data (Fan et al. (2011); Turaga et al. (2011); Lui (2012); Alashkar et al. (2016)), signal data (Srivastava & Klassen (2004); Gatto et al. (2017); Souza et al. (2019); Yataka et al. (2019)), and text data (Shimomoto et al. (2021)) are often provided as a set of feature vectors. Such raw data in matrix form is not very useful owing to its considerable size and noise. The analysis of its eigenspace, or column subspace, is important because alternatively, the raw data matrix can be well approximated by a low-dimensional subspace with basis vectors corresponding to the maximum eigenvalues of the matrix. Therefore, they inevitably give rise to the necessity of analyzing them on a Grassmann manifold.

3 Mathematical Preliminaries

In this section, the basic mathematical concepts covered in this paper are described. Further details on the fundamentals of a Grassmann manifold are summarized in the Appendix D.

3.1 Grassmann Manifold Defined as Quotient Manifold

Definition of a Grassmann Manifold A Grassmann manifold Gr( k, D ) is a set of k -dimensional subspaces span( Y ) ( Y is a matrix of k basis vectors and the function span( · ) is onto Gr( k, D ) ) in the D -dimensional Euclidean space R D , and is defined as Gr( k, D ) = { span( Y ) ⊂ R D ∣ ∣ dim(span( Y )) = k } (Absil et al. (2008)). The span( Y ) is the same subspace regardless of a k -dimensional rotation or k -dimensional reflection applied to Y on which it is spanned. With respect to the compact Stiefel manifold St( k, D ) := { Y ∈ R D × k ∣ ∣ Y ⊤ Y = I k } defined as the set of D × k -orthonormal basis matrices Y , the equivalence class of Y ∈ St( k, D ) determined from the equivalence relation ∼ 2 is defined by [ Y ] := π ( Y ) = { Y Q ∈ St( k, D ) | Q ∈ O ( k ) } , where π ( Y ) is a continuous surjection referred to as a quotient map. The equivalence class corresponds one-to-one with the k -dimensional subspace:

where Y ∈ St( k, D ) is the representative of [ Y ] . span( Y ) is termed invariant under ∼ . The quotient set composed of such [ Y ] as a whole can introduce the structure of a manifold (Sato & Iwai (2014)).

Definition 1. A Grassmann manifold as a quotient manifold is defined as follows:

where St( k, D ) / O ( k ) is the quotient manifold by the k -dimensional orthogonal group O ( k ) with the total space St( k, D ) , and π ( Y ) is the quotient map π : St( k, D ) → St( k, D ) / O ( k ) .

Tangent Space and Vector Field Let T [ Y ] Gr( k, D ) be the tangent space of [ Y ] ∈ Gr( k, D ) . As the point [ Y ] is not a matrix, T [ Y ] Gr( k, D ) cannot be represented by a matrix.

Therefore, treating these directly in numerical calculations is challenging. To solve this problem, we can use the representative Y ∈ St( k, D ) for [ Y ] and the tangent vector ξ h Y ∈ T h Y St( k, D ) , which is referred to as the horizontal lift of ξ [ Y ] ∈ T [ Y ] Gr( k, D ) , for ξ [ Y ] . These facilitate computation with matrices (Absil et al. (2008)). T h Y St( k, D ) is a subspace of the tangent space T Y St( k, D ) at Y , which is referred to as a horizontal space, and ξ h Y is referred to as a horizontal vector. The tangent bundle T Gr( k, D ) = ⋃ [ Y ] ∈ Gr( k,D ) T [ Y ] Gr( k, D ) that sums up the tangent spaces T [ Y ] Gr( k, D ) form vector fields

X : Gr( k, D ) → T Gr( k, D ) . A conceptual diagram of the various spaces is shown in Figure 1. Further details on these concepts can be found in Appendix D.4.

3.2 Manifold Normalizing Flow

Let ( M , h ) be a Riemannian manifold. We consider the time evolution of a base point x = γ (0) , γ : [0 , ∞ ) → M , whose velocity is expressed by a vector field X ( t, γ ( t )) . Intuitively, X ( t, γ ( t )) represents the direction and speed at which x moves on the curve γ ( t ) . Let T x M be the tangent space at x and T M = ⋃ x ∈M T x M be the tangent bundle. The time evolution of a point according to a vector field X : M× R → T M is expressed by the differential equation dγ ( t ) dt = X ( t, γ ( t )) , γ (0) = x . Let F X ,T : M → M be defined as the map from ∀ x ∈ M to the evaluated value at time T on the curve γ ( t ) starting at x . This map F X ,T is known as the flow of X (Lee (2003)). Recently, Mathieu & Nickel (2020) introduced the Riemann CNF (RCNF), wherein the random variable z ( t ) ∈ M is assumed to be time-dependent and the change in its log-likelihood follows the instantaneous change formula for the variable. This is an extension of the CNF (Chen et al. (2018); Grathwohl et al. (2019)) to a Riemannian manifold. Specifically, when p θ is the density parameterized by θ , the derivative of the log-likelihood is expressed as d log p θ ( z ( t )) dt = -div ( X θ ( t, z ( t ))) , where X θ is the vector field parameterized by θ and div ( · ) is the divergence. By integrating this over time, the sum of the changes in the log-likelihood with flow F X ,t 1 θ can be computed:

4 Invariant Densities from Grassmann Manifold Flow

This section provides a tractable and efficient method for learning densities on a Grassmann manifold Gr( k, D ) . The method for preserving the flow on Gr( k, D ) is non-trivial. Therefore, we derive the following implications.

Vector field on Gr( k, D ) ⇔ Flow on Gr( k, D ) ( Proposition 1 ).

Flow on Gr( k, D ) ⇔ Probability density on Gr( k, D ) ( Proposition 2 ).

Construction of a prior probability density for an efficient sampling ( Proposition 3 ).

The essence of the proofs is to show that Corollary 2 (Homogeneity Property (4) with regard to ∀ Q ∈ O ( k ) , which is equivariance in the horizontal space) is satisfied.

where ξ h Y is a horizontal lift at representative Y relative to ξ [ Y ] ∈ T [ Y ] Gr( k, D ) . We defer the proofs of all propositions to Appendix B. These series of propositions show that by using a prior distribution on Gr( k, D ) that can be easily sampled, a flow that generates a complex probability density distribution on Gr( k, D ) can be obtained.

4.1 Construction of Flow from Vector Field

To construct flows on Gr( k, D ) , we use tools in the theory of manifold differential equations. In particular, there is a natural correspondence between the vector fields on Gr( k, D ) and the flows on Gr( k, D ) . This is formalized in the following proposition.

Proposition 1. Let Gr( k, D ) be a Grassmann manifold, X be any time-dependent vector field on Gr( k, D ) , and F X ,T be a flow on a X . Let X be any time-dependent horizontal lift and F X ,T be a flow of X . X is a vector field on St( k, D ) if and only if F X ,T is a flow on St( k, D ) and satisfies invariance condition X ∼ X ′ for all F X ,T ∼ F X ′ ,T . Therefore, X is a vector field on Gr( k, D ) if and only if F X ,T := [ F X ,T ] is a flow on Gr( k, D ) , and vice versa.

Algorithm 1 Random Sampling from p Gr( k,D ) ([ X ]; [ M ] , U , V )

Require: A mean matrix M ∈ St( k, D ) , a rows covariance matrix U ∈ R D × D and a columns covariance matrix V ∈ R k × k .

1: Sample a Vec ( Z ) ∈ R Dk from Gaussian Distribution N Dk ( 0 , V ⊗ U ) and reshape to Z ∈ R D × k .

2: Compute a projected horizontal vector ξ h M = Z -M ( M ⊤ Z ) ∈ T h M St( k, D ) .

3: Compute a representative Y = R M ( ξ h M ) = M + ξ h M -( 1 2 M + 1 4 ξ h M )( I k + 1 4 ξ h ⊤ M ξ h M ) -1 ξ h ⊤ M ξ h M of the equivalence class [ Y ] ∈ Gr( k, D ) .

4.2 Construction of Probability Densities with Flow

We show that the flow on Gr( k, D ) induces density on Gr( k, D ) .

Proposition 2. Let Gr( k, D ) be a Grassmann manifold. Let p be the probability density on Gr( k, D ) and F be the flow on Gr( k, D ) . Suppose p is a density on St( k, D ) and F is a flow on St( k, D ) . Then, the distribution p F after transformations by F is also a density on St( k, D ) . Further, the invariance condition p F ∼ p F ′ is satisfied for all F ∼ F ′ . Therefore, p F := [ p F ] is a distribution on Gr( k, D ) .

In the context of the RCNF, Proposition 2 implies that the application of a flow on Gr( k, D ) to a prior distribution on Gr( k, D ) results in a probability density on Gr( k, D ) . Thus, the problem of constructing a probability density on Gr( k, D ) is reduced to that of constructing a vector field.

4.3 Prior Probability Density Function

To construct a flow on Gr( k, D ) , a prior distribution that is easy to sample as a basis for the transformation is required, although the method for constructing such a distribution is non-trivial. In this study, a distribution based on the matrix-variate Gaussian distribution is introduced as a prior on Gr( k, D ) that is easy to sample, and a flow on Gr( k, D ) is constructed.

Proposition 3. The distribution p Gr( k,D ) on a Grassmann manifold Gr( k, D ) based on the matrixvariate Gaussian distribution MN can be expressed as follows.

where M is an orthonormal basis matrix denoting the mean of the distribution, U is a positive definite matrix denoting the row directional variance, V is a positive definite matrix denoting the column directional variance, and ξ h M is a random sample from MN in an ( D -k ) × k -dimensional horizontal space T h M St( k, D ) . V Gr( k,D ) denotes the total volume of Gr( k, D ) defined by (113), R M denotes the horizontal retraction at M , and ∣ ∣ ∣ det ( ∇ ξ h M R M )∣ ∣ ∣ denotes the Jacobian.

A retraction is a map for communicating data between a manifold and its tangent bundles and is a first-order approximation of an exponential map (Zhu & Sato (2021)). Various methods of retraction have been proposed at (Absil et al. (2008); Fiori et al. (2015); Zhu & Sato (2021)); in particular, the one based on the Cayley transform is differentiable and does not require matrix decomposition. We use (40) as a Cayley transform based horizontal retraction. Further details are provided in Appendix D.6.

5 Learning Probability Densities with Flow

5.1 Training Paradigms

Using the results of Section 4, a flow model on a Grassmann manifold Gr( k, D ) is constructed. Herein, the flow model on Gr( k, D ) is expressed as GrCNF . In the proposed GrCNF, the probability density function in Section 4.3 is first constructed as a prior distribution. Subsequently, the vector field X θ : Gr( k, D ) × R → T Gr( k, D ) that generates the flow model F X θ ,T is constructed using a neural network, wherein stepwise integration on the manifold occurs in accordance with (3). The

HorP

MILs

Grad

Horizontal Vector

∈

Vector Field

RCNF (Mathieu & Nickel (2020)) framework is used for loss function, integration and divergence calculations. In the integration steps, the ordinary differential equation (ODE) is solved using the ODE solver with orthogonal integration. The learnable parameters are updated using backpropagation to maximize the sum of the computed log-likelihood. In order to efficiently compute gradients, we introduce an adjoint method to calculate analytically the derivative of a manifold ODE, based on the results of Lou et al. (2020).

5.2 Sampling Algorithm from Prior

We propose a sampling algorithm on Gr( k, D ) derived from the Cayley transform, according to the results of Section 4.3. The algorithm of sampling from a distribution on p Gr( k,D ) is described in Algorithm 1. Vec denotes the map of vertically concatenating matrices and converting them into a vector and I D is a D × D identity matrix.

5.3 Construction of Vector Field

We describe the method for constructing the vector field X θ . The learnable parameters θ are feedforward neural networks, which accept the representative Y ∈ St( k, D ) of [ Y ] ∈ Gr( k, D ) inputs and output a horizontal lift ξ h Y ∈ T h Y St( k, D ) of ξ [ Y ] ∈ T [ Y ] Gr( k, D ) that satisfies (4). The structure of X θ is important because it directly determines the ability of the distribution to be represented. To address these geometric properties, the following specific input and intermediate layers are used to obtain a O ( k ) -invariant function value v out . Figure 2 shows a conceptual diagram.

Input Layer The input layer maps the input representative Y to a point ζ h Y ∈ T h Y St( k, D ) . A horizontal projection HorP : St( k, D ) → T h Y St( k, D ) ∼ = R ( D -k ) × k is constructed as an input layer:

where W is the output of NN obtained with Y Y ⊤ as input. HorP is well-defined because it is uniquely determined regardless of how the representative Y ∈ [ Y ] is chosen.

Intermediate Layer To model the dynamics over the horizontal space, intermediate layers based on neural networks are constructed. The layers are particularly important for the ability to represent the distribution, and various configurations are possible such as concatsquash ( CS ) layer (Grathwohl et al. (2019)). The layer can be stacked multiply, and the overall power of expression increases with the number of layers. The input x of the first intermediate layer is x = Vec ◦ HorP . The last intermediate layer (one layer before the output layer) must be constructed such that it exhibits a potential function value. This is to obtain the gradient for the potential function in the output layer.

Output Layer We construct a gradient Grad : ( R , St( k, D )) → T h Y St( k, D ) as an output layer. In Grad , ξ h Y ∈ T h Y St( k, D ) is obtained by taking the gradient of the potential function v out :

where autograd ( v out , Y ) denotes automatic differentiation (Paszke et al. (2017)) ∂v out ∂ Y , P t Y ( Z ) denotes the projection of Z onto T Y St( k, D ) and sym( B ) = ( B + B ⊤ ) / 2 is a symmetric part of B . Grad is well-defined on Gr( k, D ) by the horizontal lift (Absil et al. (2008)).

5.4 ODE Solver with Orthogonal Integration

We present an ordinary differential equation (ODE) solver on Gr( k, D ) . Celledoni & Owren (2002) proposed an intrinsic ODE solver that was applicable to St( k, D ) and did not assume an outer Euclidean space. This solver works on St( k, D ) , thus it must be reformulated into a solver suitable for Gr( k, D ) . We introduce a solver on Gr( k, D ) via ODE operating on T h Y St( k, D ) , based on results from Section 4. This is an intrinsic approach regardless of whether the manifold has been embedded in a bigger space with a corresponding extension of the vector field. The ODE defined on T h Y St( k, D ) is obtained as:

where ϵ : [0 , ∞ ) → T h Y St( k, D ) is the curve on T h Y St( k, D ) , R Y denotes the horizontal retraction, ∇ γ R -1 Y can be calculated using (55) and Vec denotes the map of vertically concatenating matrices and converting them into a single vector. Further details on our ODE can be found in Appendix C.1.

5.5 Loss Function

The total change in log-likelihood using GrCNF can be calculated using the following equation.

where F X ,t 1 θ is a flow. In this study, we defined the loss function Loss for maximizing the loglikelihood: Loss = NLL = -log p θ ( Y ( t 1 )) where NLL denotes negative log-likelihood.

6 Experimental Results

In this section, we discuss the results of several experiments for validating GrCNF. Further details regarding each experimental condition, such as the architectures and hyperparameters used in training, can be found in Appendix C.3.

6.1 Generation and Density Estimation on Artificial Textures

We first trained GrCNF on five different Gr(1 , 3) (1-dimensional subspace in R 3 , that is, a line through the origin) data to visualize the model and the trained dynamics. The five datasets 3 were 2 spirals, a swissroll, 2 circles, 2 sines, and Target. We represented Gr(1 , 3) with a sphere of radius 1 centered at the origin by mapping a 1-dimensional subspace to two points on the sphere (a point on the sphere and its antipodal point) for visualization.

Result The five types of probability densities transformed by GrCNF and the results of data generation are shown in Figure 3. The top, middle, and bottom rows in the figure present the correct data, the generated data with GrCNF when the left-most column is the prior distribution, and the probability density on a Gr(1 , 3) obtained by training when the leftmost column was the prior distribution, respectively. Regarding the probability density, a brighter region corresponds to a higher probability. As shown in the figure, GrCNF can generate high-quality samples that are sufficiently accurate to the distribution of the correct data. To investigate whether GrCNF learns the distribution on Gr(1 , 3) , we generated antipodal points in Figure 3 with the trained GrCNF used in Figure 3. As shown in Figure 4, GrCNF generated exactly the same high quality samples as the original for the untrained antipodal points. This experimental result implies that the proposed flow accurately learns the distribution on a Gr( k, D ) and that all the orthogonal transformed samples can be obtained with equal quality by only training with arbitrary representatives.

Prior

2 spirals

Swissroll

2 circles

2 sines

Target

Data

Generated

Density

G

O

O

6.2 Comparison with Conventional Methods for DW4 and LJ13

Comparative experiments were performed using DW4 and LJ13-the two systems presented by Köhler et al. (2020). These datasets were generated synthetically by sampling from their respective energy functions using Markov chain Monte Carlo (MCMC) methods. Both energy functions (DW4 and LJ13) are ideal for analyzing the advantages of various methods when they exist on data that are equivariant for rotation and reflection, respectively. We used the datasets generated by MCMC that were employed in Köhler et al. (2020). The orthonormalized data Y were used for this study by applying Gram-Schmidt's orthogonalization to each column of P such that the D -dimensional data was a matrix with k orthonormal basis vectors (DW4: D = 4 , k = 2 ; LJ13: D = 13 , k = 3 ).

To match the experiment conducted in Garcia Satorras et al. (2021), 1 , 000 validation and testing samples each were used for both datasets. For DW4 and LJ13, different numbers of training samples, i.e., { 10 2 , 10 3 , 10 4 , 10 5 } and { 10 , 10 2 , 10 3 , 10 4 } , respectively, were selected, and their performance for each amount of data was examined. The proposed approach was compared with the state-of-theart E( n ) equivariant flow (E-NF) presented by Garcia Satorras et al. (2021) and simple dynamics presented by Köhler et al. (2020). In addition, comparisons with graph normalizing flow (GNF), GNF with attention (GNF-att), and GNF with attention and data augmentation (GNF-att-aug) (data augmentation by rotation), which are non-equivariant variants of E-NF, were performed. All the reported values are averages of cross-validations (three runs). Otherwise, the network structures of all these conventional methods were the same as that used in Garcia Satorras et al. (2021).

Result Table 1 presents the results of the cross-validated experiments (negative log-likelihood; NLL) for the test samples. The proposed GrCNF outperformed both the conventional non-equivariant models (GNF, GNF-att, and GNF-att-aug) and the conventional equivariant models (Simple dynamics, E-NF) in all data domains.

6.3 Comparison with Conventional Methods for QM9 Positional

A comparative experiment was performed using the QM9 Positional-a subset of the QM9 molecular dataset that considers only positional information. The purpose of this experiment was to evaluate the feasibility of generating a practical point cloud. The QM9 Positional comprises only molecules with 19 atoms/nodes, and each node has a 3-dimensional position vector associated with it. However, the likelihood of the molecules must be invariant to translation and rotation in 3-dimensional space; thus, the proposed model is suitable for this type of data. The dataset consisted of 13,831, 2501, and 1813 training, validation, and testing samples, respectively. In this experiment, we used evaluation metrics based on the NLL and Jensen-Shannon divergence (JSD; Lin (1991)), in accordance with the experiments conducted in Garcia Satorras et al. (2021). The JSD calculates the distance between the normalized histograms that are generated from the model and obtained from the training set, by creating a histogram of the relative distances among all the node pairs in each molecule.

GrCNF handles subspace data. Therefore, orthonormalized data Y was used only in the proposed GrCNF, as in PP ⊤ ≃ Y Λ Y ⊤ s.t. Λ is diagonal (Huang et al. (2015)), such that the k -dimensional point cloud data P of N points N × k matrix is a matrix with k orthonormal basis vectors ( D = 19 , k = 3 ). However, a complete point cloud generating task, such as QM9 Positional, must also store a scale parameter √ Λ such that P = Y √ Λ . Therefore, GrCNF incorporates an additional architecture to estimate √ Λ . In addition, the proposed GrCNF encounters difficulties in computing the NLL in distribution p ψ ( P ) of P ; thus, a new loss function is required to address this. In this study, the evidence lower bound (ELBO), i.e., the lower bound of the log-likelihood log p ψ ( P ) , was maximized using a variational inference framework. This is equal to the minimization of -ELBO( P ) . Further details regarding ELBO can be found in Appendix C.2.

We compared our GrCNF with the GNF, GNF-att, GNF-att-aug, Simple dynamics, Kernel dynamics, and E-NF methods. The network structure and experimental conditions for all these conventional methods were identical to those used in Garcia Satorras et al. (2021). In all the experiments, the training was performed for 160 epochs. The JSD values were the averages of the last 10 epochs for all the models.

Result Table 2 presents the NLL and JSD cross-validated against the test data. Although the proposed GrCNF could not directly compute the NLL in this problem setting, it outperformed all

Nearest QM9

Generated QM9

GrCNF

E-NF

x-y

y-z

z-x

x-y

y-z

z-x

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

3.8

-3.8

3.8

the other algorithms because the relation NLL ≤ -ELBO is true in general. With regard to the JSD, GrCNF achieved the best performance. Figure 2 presents the normalized histograms of the QM9 Positional molecular data and the data generated by GrCNF, in relation to the JSD of GrCNF. As shown, the histograms of the molecules generated by GrCNF are close to the histograms of the data set; thus, GrCNF can generate realistic molecules stably.

7 Conclusion

We proposed the concept of CNF on a Grassmann manifold (GrCNF) for stable shape generation. The proposed model is a generative model capable of handling subspace data; i.e., it is a CNF that considers the equivariance on the Stiefel manifold. Through suitable experiments, the ability of the proposed GrCNF to generate qualitatively in 1-dimensional subspace datasets in R 3 was confirmed. Further, GrCNF significantly outperformed existing normalizing flows methods in terms of the log-likelihood or ELBO for DW4, LJ13, and QM9 Positional. It was shown that GrCNF can be used to generate realistic data more stably.

Societal Impact and Limitations Our work is concerned with accurately modeling the data topology; thus, we do not expect there to be any negative consequence in our application field. The proposed theory and implementation are valid in a remarkably general setting, although there are limitations that can be addressed in future works: 1. The proposed method requires calculation of the Kronecker product at each ODE step (Section 5.4), which is computationally expensive. 2. Our current paper does not include experiments on high-dimensional data. Therefore, future experiments should be conducted using higher dimensional data, such as point clouds that represent general object geometry. However, the results presented in this paper still provide numerous valuable insights in the field of stable shape generation.

References

P.-A. Absil, R. Mahony, and R. Sepulchre. Optimization Algorithms on Matrix Manifolds . Princeton University Press, Princeton, NJ, 2008. ISBN 978-0-691-13298-3. URL https: //press.princeton.edu/absil .

Taleb Alashkar, Boulbaba Ben Amor, Mohamed Daoudi, and Stefano Berretti. A grassmann framework for 4d facial shape analysis. Pattern Recognition , 57:21-30, 2016. ISSN 0031-3203. URL https://doi.org/10.1016/j.patcog.2016.03.013 .

Begelfor and M. Werman. Affine invariance revisited. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06) , volume 2, pp. 2087-2094. IEEE Computer Society, 2006. URL https://doi.org/10.1109/CVPR.2006.50 .

Camille Bilodeau, Wengong Jin, Tommi Jaakkola, Regina Barzilay, and Klavs F. Jensen. Generative models for molecular discovery: Recent advances and challenges. WIREs Computational Molecular Science , pp. e1608, 2022. URL https://doi.org/10.1002/wcms.1608 .

Denis Boyda, Gurtej Kanwar, Sébastien Racanière, Danilo Jimenez Rezende, Michael S. Albergo, Kyle Cranmer, Daniel C. Hackett, and Phiala E. Shanahan. Sampling using su ( n ) gauge equivariant flows. Phys. Rev. D , 103(7):074504, 2021a. URL https://doi.org/10.1103/PhysRevD.103. 074504 .

Denis Boyda, Gurtej Kanwar, Sébastien Racanière, Danilo Jimenez Rezende, Michael S. Albergo, Kyle Cranmer, Daniel C. Hackett, and Phiala E. Shanahan. Sampling using su(n) gauge equivariant flows. Physical Review D , 103(7), 2021b. URL https://link.aps.org/doi/10.1103/ PhysRevD.103.074504 .

Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Velickovic. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. CoRR , abs/2104.13478, 2021. URL https: //arxiv.org/abs/2104.13478 .

Elena Celledoni and Brynjulf Owren. A class of intrinsic schemes for orthogonal integration. SIAM Journal on Numerical Analysis , 40(6):2069-2084, 2002. URL http://www.jstor.org/ stable/4100984 .

Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems , volume 31, pp. 6571-6583. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper/ 2018/file/69386f6bb1dfed68692a24c8686939b9-Paper.pdf .

Yasuko Chikuse. Statistics on Special Manifolds. Lecture Notes in Statistics , volume 174. Springer, 01 2003. URL https://doi.org/10.1007/978-0-387-21540-2 .

Taco Cohen, Maurice Weiler, Berkay Kicanaoglu, and Max Welling. Gauge equivariant convolutional networks and the icosahedral CNN. In Proceedings of the 36th International Conference on Machine Learning , volume 97, pp. 1321-1330. PMLR, 2019a. URL https://proceedings. mlr.press/v97/cohen19d.html .

Taco S. Cohen and Max Welling. Group equivariant convolutional networks. In Proceedings of The 33rd International Conference on Machine Learning , volume 48, pp. 2990-2999. PMLR, 2016. URL https://proceedings.mlr.press/v48/cohenc16.html .

Taco S. Cohen and Max Welling. Steerable cnns. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum?id=rJQKYt5ll .

Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cnns. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum?id= Hkbd5xZRb .

Taco S. Cohen, Mario Geiger, and Maurice Weiler. A general theory of equivariant cnns on homogeneous spaces. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019b. URL https://proceedings.neurips.cc/paper/2019/file/ b9cfe8b6042cf759dc4c0cccb27a6737-Paper.pdf .

Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet. Riemannian score-based generative modelling. In Advances in Neural Information Processing Systems , volume 35, pp. 2406-2422. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ 105112d52254f86d5854f3da734a52b4-Paper-Conference.pdf .

J. Dormand and J. P. Prince. A family of embedded Runge-Kutta formulae. Journal of Computational and Applied Mathematics , pp. 19-26, 1980. URL https://doi.org/10.1016/ 0771-050X(80)90013-3 .

Olga Doronina, Zachary Grey, and Andrew Glaws. Grassmannian shape representations for aerodynamic applications. In AAAI 2022 Workshop on AI for Design and Manufacturing (ADAM) , 2022. URL https://openreview.net/forum?id=1RRU6ud9YC .

Alan Edelman, Tomás A. Arias, and Steven T. Smith. The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications , 20(2):303-353, 1998. URL https://doi.org/10.1137/S0895479895290954 .

Lawrence Craig Evans and Gariepy Ronald. Measure Theory and Fine Properties of Functions, Revised Edition (1st ed.) . CRC Press, Revised Edition, 2015. URL https://doi.org/10.1201/ b18333 .

Yong Fan, Yong Liu, Hong Wu, Yihui Hao, Haihong Liu, Zhening Liu, and Tianzi Jiang. Discriminant analysis of functional connectivity patterns on grassmann manifold. NeuroImage , 56(4):2058-2067, 2011. URL https://doi.org/10.1117/12.844495 .

Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson. Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data. In Proceedings of the 37th International Conference on Machine Learning , volume 119, pp. 3165-3176, 2020. URL https://dl.acm.org/doi/10.5555/3524938.3525235 .

Simone Fiori, Tetsuya Kaneko, and Toshihisa Tanaka. Tangent-bundle maps on the grassmann manifold: Application to empirical arithmetic averaging. IEEE Transactions on Signal Processing , 63(1):155-168, 2015. URL https://doi.org/10.1109/TSP.2014.2365764 .

Victor Garcia Satorras, Emiel Hoogeboom, Fabian Fuchs, Ingmar Posner, and Max Welling. E( n ) equivariant normalizing flows. In Advances in Neural Information Processing Systems , volume 34, pp. 4181-4192. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/ paper/2021/file/21b5680d80f75a616096f2e791affac6-Paper.pdf .

Bernardo B. Gatto, Juan G. Colonna, Eulanda M. dos Santos, and Eduardo F. Nakamura. Mutual singular spectrum analysis for bioacoustics classification. In 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) , pp. 1-6, 2017. URL https: //doi.org/10.1109/MLSP.2017.8168113 .

Evan S. Gawlik and M. Leok. High-order retractions on matrix manifolds using projected polynomials. SIAM J. Matrix Anal. Appl. , 39:801-828, 2018a. URL https://doi.org/10.1137/ 17M1130459 .

Evan S. Gawlik and Melvin Leok. High-order retractions on matrix manifolds using projected polynomials. SIAM Journal on Matrix Analysis and Applications , 39(2):801-828, 2018b. URL https://doi.org/10.1137/17M1130459 .

Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In Proceedings of the 32nd International Conference on Machine Learning , volume 37, pp. 881-889. PMLR, 2015. URL http://proceedings.mlr.press/ v37/germain15.html .

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems , volume 27. Curran Associates, Inc., 2014. URL https://proceedings. neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf .

Simon Graham, David B. A. Epstein, and Nasir M. Rajpoot. Dense steerable filter cnns for exploiting rotational symmetry in histology images. IEEE Trans. Medical Imaging , 39(12):4124-4136, 2020. URL https://doi.org/10.1109/TMI.2020.3013246 .

Will Grathwohl, Ricky T. Q. Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. FFJORD: free-form continuous dynamics for scalable reversible generative models. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview.net/forum?id=rJxgknCcK7 .

Pim De Haan, Maurice Weiler, Taco Cohen, and Max Welling. Gauge equivariant mesh cnns: Anisotropic convolutions on geometric graphs. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum?id=Jnspzp-oIZE .

Ernst Hairer. Geometric Numerical Integration : Structure-Preserving Algorithms for Ordinary Differential Equations . Springer, 2nd ed. edition, 2006. URL https://doi.org/10.1007/ 3-540-30666-8 .

Yuval Haitman, Joseph M. Francos, and Louis L. Scharf. Grassmannian dimensionality reduction for optimized universal manifold embedding representation of 3d point clouds. In 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) , pp. 4196-4204, 2021. URL https://doi.org/10.1109/ICCVW54120.2021.00468 .

Jihun Hamm and Daniel D. Lee. Grassmann discriminant analysis: A unifying view on subspacebased learning. In Proceedings of the 25th International Conference on Machine Learning , ICML '08, pp. 376-383. Association for Computing Machinery, 2008. URL https://doi.org/10. 1145/1390156.1390204 .

Mehrtash T. Harandi, Conrad Sanderson, Sareh Shirazi, and Brian C. Lovell. Graph embedding discriminant analysis on grassmannian manifolds for improved image set matching. In CVPR 2011 , 2011. URL https://doi.org/10.1109/CVPR.2011.5995564 .

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems , volume 33, pp. 6840-6851. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/ 4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf .

Chin-Wei Huang, Milad Aghajohari, Joey Bose, Prakash Panangaden, and Aaron C Courville. Riemannian diffusion models. In Advances in Neural Information Processing Systems , volume 35, pp. 2750-2761. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_ files/paper/2022/file/123d3e814e257e0781e5d328232ead9b-Paper-Conference. pdf .

Zhiwu Huang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Projection metric learning on grassmann manifold with application to video based face recognition. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 140-149, 2015. URL https://doi. org/10.1109/CVPR.2015.7298609 .

Zhiwu Huang, Jiqing Wu, and Luc Van Gool. Building deep networks on grassmann manifolds. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) , pp. 3279-3286, 2018. URL https://dl.acm.org/doi/abs/10.5555/3504035.3504436 .

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pp. 448-456. PMLR, 2015. URL https://proceedings.mlr.press/v37/ioffe15.html .

Arieh Iserles, Hans Z. Munthe-Kaas, Syvert P. Nørsett, and Antonella Zanna. Lie-group methods. Acta Numerica , 9:215-365, 2000. URL https://doi.org/10.1017/S0962492900002154 .

Isay Katsman, Aaron Lou, Derek Lim, Qingxuan Jiang, Ser Nam Lim, and Christopher M De Sa. Equivariant manifold flows. In Advances in Neural Information Processing Systems , volume 34, pp. 10600-10612. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/ paper/2021/file/581b41df0cd50ace849e061ef74827fc-Paper.pdf .

Hyeongju Kim, Hyeonseung Lee, Woo Hyun Kang, Joun Yeop Lee, and Nam Soo Kim. Softflow: Probabilistic framework for normalizing flow on manifolds. In Advances in Neural Information Processing Systems , volume 33, pp. 16388-16397. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/ bdbca288fee7f92f2bfa9f7012727740-Paper.pdf .

Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations, ICLR 2014 , 2014. URL http://arxiv.org/abs/ 1312.6114 .

Jonas Köhler, Leon Klein, and Frank Noé. Equivariant flows: Exact likelihood generative learning for symmetric densities. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proceedings of Machine Learning Research , pp. 5361-5370. PMLR, 2020. URL http://proceedings.mlr.press/ v119/kohler20a.html .

Risi Kondor and Shubhendu Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In Proceedings of the 35th International Conference on Machine Learning , volume 80, pp. 2747-2755. PMLR, 2018. URL https: //proceedings.mlr.press/v80/kondor18a.html .

J.M. Lee. Introduction to Smooth Manifolds . Graduate Texts in Mathematics. Springer, 2003. ISBN 9780387954486.

Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory , 37(1):145-151, 1991. URL https://doi.org/10.1109/18.61115 .

Xiuwen Liu, A. Srivastava, and K. Gallivan. Optimal linear representations of images for object recognition. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. , volume 1, pp. I-I, 2003. URL https://doi.org/10.1109/CVPR. 2003.1211358 .

Aaron Lou, Derek Lim, Isay Katsman, Leo Huang, Qingxuan Jiang, Ser Nam Lim, and Christopher M De Sa. Neural manifold ordinary differential equations. In Advances in Neural Information Processing Systems , volume 33, pp. 17548-17558. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/ cbf8710b43df3f2c1553e649403426df-Paper.pdf .

Yui Man Lui. Advances in matrix manifolds for computer vision. Image Vision Comput. , 30(6-7): 380-388, 2012. URL https://doi.org/10.1016/j.imavis.2011.08.002 .

J.R. Magnus and H. Neudecker. Matrix Differential Calculus with Applications in Statistics and Econometrics . Wiley Series in Probability and Statistics. Wiley, 2019. URL https://doi.org/ 10.1002/9781119541219 .

Arak M. Mathai, Serge B. Provost, and Hans J. Haubold. The matrix-variate gaussian distribution. In Multivariate Statistical Analysis in the Real and Complex Domains , pp. 217-288. Springer International Publishing, 2022. URL https://doi.org/10.1007/978-3-030-95864-0_4 .

Emile Mathieu and Maximilian Nickel. Riemannian continuous normalizing flows. In Proceedings of the 34th International Conference on Neural Information Processing Systems , volume 33. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/ 1aa3d9c6ce672447e1e5d0f1b5207e85-Paper.pdf .

Cleve Moler and Charles Van Loan. Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Review , 45(1):3-49, 2003. URL https://doi.org/10.1137/ S00361445024180 .

Hans Munthe-Kaas. High order runge-kutta methods on manifolds. Applied Numerical Mathematics , 29(1):115-127, 1999. URL https://doi.org/10.1016/S0168-9274(98)00030-0 .

Aaron Van Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In Proceedings of The 33rd International Conference on Machine Learning , volume 48, pp. 1747-1756. PMLR, 2016. URL http://proceedings.mlr.press/v48/oord16.html .

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS 2017 Workshop on Autodiff , 2017. URL https://openreview.net/forum? id=BJJsrmfCZ .

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper/2019/file/ bdbca288fee7f92f2bfa9f7012727740-Paper.pdf .

Vic Patrangenaru and Kanti V. Mardia. Affine shape analysis and image analysis. In 22nd Leeds Annual Statistics Research Workshop , 2003. URL http://www1.maths.leeds.ac.uk/ Statistics/workshop/lasr2003/proceedings/patrangenaru.pdf .

Omri Puny, Matan Atzmon, Edward J. Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu, and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id= ywxtmG1nU_6 .

Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for activation functions. CoRR , abs/1710.05941, 2017. URL http://arxiv.org/abs/1710.05941 .

Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pp. 1530-1538. PMLR, 2015. URL https://proceedings.mlr.press/ v37/rezende15.html .

Danilo Jimenez Rezende, Sébastien Racanière, Irina Higgins, and Peter Toth. Equivariant hamiltonian flows, 2019. URL https://doi.org/10.48550/arXiv.1909.13739 .

Benjamin Sanchez-Lengeling and Alán Aspuru-Guzik. Inverse molecular design using machine learning: Generative models for matter engineering. Science , 361(6400):360-365, 2018. URL https://www.science.org/doi/abs/10.1126/science.aat2663 .

Hiroyuki Sato and Toshihiro Iwai. Optimization algorithms on the grassmann manifold with application to matrix eigenvalue problems. Japan Journal of Industrial and Applied Mathematics , 31: 355-400, 2014. URL https://doi.org/10.1007/s13160-014-0141-9 .

Víctor Garcia Satorras, Emiel Hoogeboom, and Max Welling. e ( n ) equivariant graph neural networks. In Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pp. 9323-9332. PMLR, 2021. URL https://proceedings.mlr.press/v139/satorras21a.html .

Sepiashvili, J.M.F. Moura, and V.H.S. Ha. Affine-permutation symmetry: invariance and shape space. In IEEE Workshop on Statistical Signal Processing, 2003 , pp. 307-310, 2003. URL https://doi.org/10.1109/SSP.2003.1289406 .

Erica K. Shimomoto, François Portet, and Kazuhiro Fukui. Text classification based on the word subspace representation. Pattern Analysis and Applications , 24(3):1075-1093, 2021. URL https://doi.org/10.1007/s10044-021-00960-6 .

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pp. 2256-2265. PMLR, 2015. URL https://proceedings.mlr.press/v37/ sohl-dickstein15.html .

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations , 2021. URL https://openreview.net/ forum?id=St1giarCHLP .

Lincon S. Souza, Bernardo B. Gatto, and Kazuhiro Fukui. Classification of bioacoustic signals with tangent singular spectrum analysis. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pp. 351-355, 2019. URL https://doi.org/ 10.1109/ICASSP.2019.8682493 .

Lincon S. Souza, Bernardo B. Gatto, Jing-Hao Xue, and Kazuhiro Fukui. Enhanced grassmann discriminant analysis with randomized time warping for motion recognition. Pattern Recognition , 97:107028, 2020. URL https://doi.org/10.1016/j.patcog.2019.107028 .

Lincon S. Souza, Naoya Sogi, Bernardo B. Gatto, Takumi Kobayashi, and Kazuhiro Fukui. Grassmannian learning mutual subspace method for image set recognition. Neurocomputing , 2022. ISSN 0925-2312. URL https://doi.org/10.1016/j.neucom.2022.10.040 .

Srivastava, S.H. Joshi, W. Mio, and Xiuwen Liu. Statistical shape analysis: clustering, learning, and testing. IEEE Transactions on Pattern Analysis and Machine Intelligence , 27(4):590-602, 2005. URL https://doi.org/10.1109/TPAMI.2005.86 .

Anuj Srivastava and Eric Klassen. Bayesian and geometric subspace tracking. Advances in Applied Probability , 36(1):43-56, 2004. URL http://www.jstor.org/stable/1428352 .

Pavan Turaga, Ashok Veeraraghavan, and Rama Chellappa. Statistical analysis on stiefel and grassmann manifolds with applications in computer vision. In 2008 IEEE Conference on Computer Vision and Pattern Recognition , pp. 1-8, 2008. URL https://doi.org/10.1109/CVPR.2008. 4587733 .

Pavan Turaga, Ashok Veeraraghavan, Anuj Srivastava, and Rama Chellappa. Statistical computations on grassmann and stiefel manifolds for image and video-based recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 33(11):2273-2286, 2011. URL https://doi.org/ 10.1109/TPAMI.2011.52 .

Maurice Weiler and Gabriele Cesa. General e (2) -equivariant steerable cnns. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper/2019/file/ 45d6637b718d0f24a237069fe41b0db4-Paper.pdf .

Zaiwen Wen and Wotao Yin. A feasible method for optimization with orthogonality constraints. Mathematical Programming , 142:397-434, 2013. URL https://doi.org/10.1007/ s10107-012-0584-1 .

Zhu Xiaojing. A riemannian conjugate gradient method for optimization on the stiefel manifold. Computational Optimization and Applications , 67:73-110, 2017. URL https://doi.org/10. 1007/s10589-016-9883-4 .

Ryoma Yataka and Kazuhiro Fukui. Three-dimensional object recognition via subspace representation on a grassmann manifold. In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods , pp. 208-216. SciTePress, 2017. URL https://doi.org/10.5220/0006204702080216 .

Ryoma Yataka, Kazuki Hirashima, Takafumi Matsuda, Tai Tanaka, Masato Gocho, and Masashi Shiraishi. Projection metric learning of updated-subspaces for radar target classification. In 2019 16th European Radar Conference (EuRAD) , pp. 5-8, 2019. URL https://ieeexplore.ieee. org/abstract/document/8904617 .

Takao Yoshinuma, Hideitsu Hino, and Kazuhiro Fukui. Personal authentication based on 3d configuration of micro-feature points on facial surface. In Image and Video Technology , pp. 433-446. Springer International Publishing, 2016. URL https://doi.org/10.1007/ 978-3-319-29451-3_35 .

Xiaojing Zhu and Chun-Yan Duan. On matrix exponentials and their approximations related to optimization on the stiefel manifold. Optimization Letters , 13, 07 2019. URL https://doi.org/ 10.1007/s11590-018-1341-z .

Xiaojing Zhu and Hiroyuki Sato. Cayley-transform-based gradient and conjugate gradient algorithms on grassmann manifolds. Adv. Comput. Math. , 47(4), 2021. URL https://doi.org/10.1007/ s10444-021-09880-9 .

A Appendix Overview

First, we present the details of the various proofs of Section 4 in Appendix B. Next, in Appendix C, we describe the network layers for the building GrCNF, and the detailed model architectures, hyperparameters, and implementation on the experiments. Finally, in Appendix D, we provide a summary of the fundamentals of a Grassmann manifold, which is the core concept of this study.

B Proofs

B.1 Proposition 1

First, we invoked the following two corollaries.

Corollary 1 (Diffeomorphism Invariance of Flows) . Let F : M→N be a diffeomorphism. If X is a smooth vector field over M and θ is the flow of X , then the flow of F ∗ X 4 is η t = F ◦ θ t ◦ F -1 , with domain N t = F ( M t ) for each t ∈ R .

Corollary 2 (Homogeneity Property) . The horizontal lift ξ h Y at representative Y ∈ St( k, D ) relative to ξ [ Y ] ∈ T [ Y ] Gr( k, D ) satisfies the following homogeneity (equivariance) property (4) with regard to ∀ Q ∈ O ( k ) .

Proof. π ( Y ) = π ( Y Q ) is true for ∀ Y ∈ St( k, D ) , Q ∈ O ( k ) . Therefore, π ( Y ) = ( π ◦ q ) ( Y ) is true when defined as q ( Y ) = Y Q . When the derivative dπ ( · ) [ · ] of both sides is applied to the horizontal lift ξ h Y of ξ [ Y ] , the following is obtained:

Moreover, from (98) which is definition of horizontal lift, the following equation is true.

Subsequently, we obtain the following equation.

.

Finally, the uniqueness of the horizontal lift yields ξ h Y Q = ξ h Y Q

Proposition 1. Let Gr( k, D ) be a Grassmann manifold, X be any time-dependent vector field on Gr( k, D ) , and F X ,T be a flow on a X . Let X be any time-dependent horizontal lift and F X ,T be a flow of X . X is a vector field on St( k, D ) if and only if F X ,T is a flow on St( k, D ) and satisfies invariance condition X ∼ X ′ for all F X ,T ∼ F X ′ ,T . Therefore, X is a vector field on Gr( k, D ) if and only if F X ,T := [ F X ,T ] is a flow on Gr( k, D ) , and vice versa.

Proof. Flow F X ,T on Gr ( k, D ) ⇒ Vector Field X on Gr ( k, D ) . Let θ : St( k, D ) × O ( k ) → St( k, D ) , ( Y , Q ) ↦→ Y Q be a map representing the right action of the orthogonal group. In addition, let F X ,T be a flow on Gr( k, D ) and F X ,T be a flow on St( k, D ) . These satisfy F X Q ,T ∼

F X ,T , F X Q ,T ∈ F X ,T , F X ,T ∈ F X ,T .

Thus, X ∼ X Q is true. Therefore, X is the horizontal lift of the vector field X on Gr( k, D ) and is unique for X .

Flow F X ,T on Gr ( k, D ) ⇐ Vector Field X on Gr ( k, D ) . Let θ : St( k, D ) × O ( k ) → St( k, D ) , ( Y , Q ) ↦→ Y Q be a map representing the right action of the orthogonal group. In addition, let X be a vector field over a horizontal bundle T h St( k, D ) on St( k, D ) and F X ,T be its flow. From the Corollary 1,

Note that d ( θ ) Y X = X Q is derived from the Corollary 2 and (4) in Zhu & Sato (2021). This indicates that F X ,T ∼ F X ′ ,T is true for any X , X ′ ∈ T h St( k, D ) that satisfies X ∼ X ′ . Thus, a new flow can be defined as F X ,T := [ F X ,T ] . This is a flow on a Gr( k, D ) . Because X is a vector field in a horizontal bundle T h St( k, D ) on St( k, D ) , it is a horizontal lift of the vector field X on Gr( k, D ) and is therefore unique for X .

Thus, the proof is complete.

B.2 Proposition 2

Proposition 2. Let Gr( k, D ) be a Grassmann manifold. Let p be the probability density on Gr( k, D ) and F be the flow on Gr( k, D ) . Suppose p is a density on St( k, D ) and F is a flow on St( k, D ) . Then, the distribution p F after transformations by F is also a density on St( k, D ) . Further, the invariance condition p F ∼ p F ′ is satisfied for all F ∼ F ′ . Therefore, p F := [ p F ] is a distribution on Gr( k, D ) .

Proof. Let θ : St( k, D ) × O ( k ) → St( k, D ) : ( Y , Q ) ↦→ Y Q be a map representing the right action of the orthogonal group.

where | det { J θ ( X ) }| = 1 is true because θ is the action of the orthogonal group. Therefore, as p F ( θ ◦ Y ) = θ ◦ p F ( Y ) is true, p F ∼ p F Q ∈ p F is true. Based on this, it can be concluded that the subject is satisfied.

B.3 Proposition 3

Proposition 3. The distribution p Gr( k,D ) on a Grassmann manifold Gr( k, D ) based on the matrixvariate Gaussian distribution MN can be expressed as follows.

where M is an orthonormal basis matrix denoting the mean of the distribution, U is a positive definite matrix denoting the row directional variance, V is a positive definite matrix denoting the column directional variance, and ξ h M is a random sample from MN in an ( D -k ) × k -dimensional horizontal space T h M St( k, D ) . V Gr( k,D ) denotes the total volume of Gr( k, D ) defined by (113), R M denotes the horizontal retraction at M , and ∣ ∣ ∣ det ( ∇ ξ h M R M )∣ ∣ ∣ denotes the Jacobian.

Proof. Let p Gr ([ X ]) be a probability density function on a Gr( k, D ) . From (101), let ( d X ) be the invariant measure on Gr( k, D ) and d ξ h M be the Lebesgue measure on T h M St( k, D ) . Subsequently, a change of variables was performed according to the following:

Suppose p Gr ([ X ]) is integrable with the probability measure [ d X ] on Gr( k, D ) defined by (114). Then, we obtain the following relation.

In addition, we obtain the following equation based on [ d X ] .

where MN ( ξ h M ) denotes the matrix-variate Gaussian distribution (Mathai et al. (2022)). Thus, the probability density function on Gr( k, D ) can be expressed as follows:

The Jacobian can be represented as ∣ ∣ ∣ det ( ∇ ξ h M R M )∣ ∣ ∣ = ∣ ∣ ∣ det { ( ∂R M ) ⊤ ( ∂R M ) }∣ ∣ ∣ 1 2 from Evans &Ronald (2015), where ∂R M = ∂R M ∂ ξ h M . Further, ∂R M can be computed as follows. First, we define the horizontal retraction R Y : T h Y St( k, D ) → St( k, D ) based on the Cayley transform from Zhu & Sato (2021).

This is a fixed time ( t = 1 ) version of (122). Next, for improved visibility in subsequent calculations, let E = ξ h M , F = 1 2 M + 1 4 E , G = ( I k + 1 4 H ) -1 , H = E ⊤ E . In addition, let D be defined as the operator for the derivative of a matrix by a matrix. Then, the derivative ∇ ξ h M R M = D X by E is as follows:

Finally, each derivative can be calculated as follows:

where ⊗ denotes the Kronecker product. K D,k is a Dk × Dk matrix K D,k = ∑ m i =1 ∑ n j =1 ( L i,j ⊗ L ⊤ i,j ) referred to as the commutation matrix, which denotes the transposition operation of D × k . Further, L i,j is a D × k matrix whose ( i, j ) component is 1 whereas all other components are 0.

deg=0°

deg=30°

deg=60°

deg=120°

deg=150°

For details on the formulae for matrix derivatives used in this proof, please refer to Magnus & Neudecker (2019).

p Gr( k,D ) ([ X ]) = p Gr( k,D ) ([ X ]; [ M ] , U , V ) is a probability distribution following mean [ M ] and matrix variance U , V . Using Gr(1 , 3) as an example, we qualitatively confirmed through visualization that p Gr(1 , 3) ([ X ]) is a density on Gr(1 , 3) . Gr(1 , 3) is a 1-dimensional subspace in a 3-dimensional space; that is, a space whose elements are lines passing through the origin in 3-dimensional space. For the visualization, we expressed Gr(1 , 3) by mapping a 1-dimensional subspace to two points on a sphere (one point on the sphere and its antipodal point) of radius 1 centered at the origin.

Figure 6 shows the density of p Gr(1 , 3) ([ X ]) with M = (1 . 0 , 0 . 0 , 0 . 0) ⊤ , U = σ 2 I 3 , V = I 1 , σ 2 = 0 . 5 . Each sphere in the figure indicates Gr(1 , 3) , with brighter spheres representing higher densities. The leftmost figure shows M as viewed from the front diagonally above, and the other figures present the views when the viewpoint is rotated clockwise around the z -axis by 30 ◦ to 150 ◦ with movement to the right. In the leftmost figure, the density is highly spread around M . In the other figures (particularly the rightmost one), the antipodal point ( -M = ( -1 . 0 , 0 . 0 , 0 . 0) ⊤ ) is densely spread out. This implies that when only one M is specified as the representative of the equivalence class [ M ] , the density around the other elements in the equivalence class [ M ] is as high as that around the representative. Thus, we can confirm that p Gr( k,D ) ([ X ]) has a density of Gr(1 , 3) .

C Experimental Details for Learning GrCNF

C.1 Details on ODE Solver with Orthogonal Integration

We explain in detail an ordinary differential equation (ODE) solver on Gr( k, D ) . Several studies on ODE solvers employed on a manifold M have been reported (Munthe-Kaas (1999); Iserles et al. (2000); Hairer (2006)). Hairer (2006) proposed a simple projection method that projected onto the manifold at each step and the symmetric projection method suitable for long-time integration. However, this method requires M to be a submanifold in Euclidean space, and thus cannot be applied on Gr( k, D ) . In contrast, Celledoni & Owren (2002) proposed an intrinsic ODE solver that was applicable to a Stiefel manifold and did not assume an outer Euclidean space. This solver works on a Stiefel manifold, thus it must be reformulated into a solver suitable for Gr( k, D ) , which is our problem setting. We introduce below a solver on Gr( k, D ) via ODE operating on the horizontal space T h Y St( k, D ) , based on results from Celledoni & Owren (2002) and Section 4. This is an intrinsic approach regardless of whether M has been embedded in a larger space with a corresponding extension of the vector field.

First, consider an ODE expressed using a vector field X θ on a curve γ ( t ) : [0 , ∞ ) → St( k, D ) such that for each time t .

where X θ is constructed by a neural network with parameter θ , as described in Section 5.3. Horizontal retraction R Y which is defined as (122) and is described in Appendix D.6, defines local coordinates of Gr( k, D ) in a neighborhood of the point [ Y ] . Thus, the solution of ODE can be expressed as:

where ϵ : [0 , ∞ ) → T h Y St( k, D ) is the curve on T h Y St( k, D ) . By differentiating (53) with t , the following equation is obtained:

Therefore, the ODE defined on T h Y St( k, D ) is obtained as (8):

where Vec denotes the map of vertically concatenating matrices and converting them into a single vector. ∇ γ R -1 Y can be calculated using the derivative of (123):

where X = γ ( t ) , M = Y -XX ⊤ Y , N = I k + X ⊤ Y , ∇ M = I Dk -( I k ⊗ XX ⊤ ) and ∇ N -1 = -( N -⊤ ⊗ N -1 ) ( I k ⊗ X ⊤ ) . Because (8) is an ODE on T h Y St( k, D ) ∼ = R ( D -k ) × k from Absil et al. (2008), it can be solved using an ODE solver such as Runge-Kutta methods that operate on Euclidean space. This study used Algorithm 5.1 presented in Celledoni & Owren (2002). In each step, first, (8) was solved using the Runge-Kutta method of order 5 as in Dormand & Prince (1980). Subsequently, the solution of ODE (52) was obtained by applying the solution ϵ to (53).

C.2 Loss Function based on Variational Inference

In the setting in Section 6.3, we used orthonormalized data Y as in PP ⊤ ≃ Y Λ Y ⊤ , where Λ is diagonal (Huang et al. (2015)), such that the k -dimensional point cloud data P of N points N × k matrix is a matrix with k orthonormal basis vectors. Thus, generating a complete point cloud requires the estimation of the scale parameters √ Λ to be P = Y √ Λ and a loss function that incorporates this. In this study, we approximated by maximizing the evidence lower bound (ELBO), which is the lower bound of the overall log-likelihood log p ψ ( P ) of p ψ ( P ) , using a variational inference framework. The loss function is the variational energy -ELBO( P ) with negative ELBO.

(a)

(b)

ELBO( P ) can be decomposed as follows.

where q ϕ ( Y | P ) is the inference model with parameter ϕ , p ψ ( P | Y ) is the decoder model with parameter ψ , p ψ ( Y | P ) is the posterior distribution with parameter ψ , and p θ ( Y ) is the prior distribution with parameter θ . Further, D KL ( q ϕ ( Y | P ) || p θ ( Y )) can be formulated using differential entropy as follows.

Thus, the final loss function is as follows.

Each term of the loss function can be calculated as follows.

Expectation of log-likelihood E q ϕ ( Y | P ) [log p ψ ( P | Y )] is the reconstruction log-likelihood of P . The expectation is estimated by Monte Carlo sampling.

Differential entropy In the decomposition of a point cloud P , there exists arbitrariness in the choice of Y and Λ , as in PP ⊤ ≃ Y Λ Y ⊤ . In this study, we assumed that the diagonal components of Λ are in descending order, and we restricted the decomposition arbitrariness to be an action Q ∈ O ( k ) . If we suppose that the action Q follows a uniform distribution when Y = XQ holds, then Y also follows a uniform distribution in the k -dimensional subspace span( Y ) . Although this is a uniform distribution on St( k, k ) , we can consider a uniform distribution on O ( k ) because St( k, k ) = O ( k ) . The probability density function of Q is represented by U O ( k ) ( Q ) in (111).

Therefore, the differential entropy of the decoder model can be calculated as follows.

Expectation of prior distribution We used (9) for the prior distribution p θ ( Y ) . Further, reparameterization was used to enable differentiable Monte Carlo estimation of expectations.

where it was assumed that Q is sampled from a uniform distribution, which follows Haar measure on O ( k ) . L is set L = 1 .

C.3 Implementation Details and Experimental setting

The following sections present more details about the network architectures, training hyperparameters, and experimental conditions for each of the experiments in Section 6.

C.3.1 Artificial Textures

Network Architecture The vector field was constructed with the specific input, intermediate, and output layers described in Section 5.3. The GrCNF architecture is shown on top in Table 3. Layers are denoted as Layer in the table, and were processed from top to bottom. Norm. and Act. denote the normalization and activation functions to be applied immediately after the Layer, and the Norm. and Act. were applied in that order. Further, Out Size denotes the output size after Act. Vec denotes the map of vertically concatenating matrices and converting them into a single vector. Moreover, only row Input denotes the size of the input data, not the input layer ( HorP ). (9) was used for the loss function.

Hyper-parameters The mean M and covariances U and V in the prior distribution on the Grassmann manifold were set to M = (1 . 0 , 0 . 0 , 0 . 0) ⊤ , U = σ 2 I 3 , and V = I 1 , σ = 0 . 3 , respectively. Other hyperparameters used during the training of GrCNF are shown in Table 4.

Implementation We used PyTorch (Paszke et al. (2019)) to implement the model and run the experiments. The CNF is based on the implementation 5 in Chen et al. (2018) and the framework of the RCNF (Mathieu & Nickel (2020)). Thus, the ODE was solved using the explicit and adaptive Runge-Kutta method (Dormand & Prince (1980)) of order 5, and worked by projecting each step onto a manifold (Hairer (2006)). The autograd in (7) was calculated with torch . autograd . grad (Paszke et al. (2017)) in PyTorch. The experimental hardware was built with an Intel Core i7-9700 CPU and a single NVIDIA GTX 1060 GPU with 6 GB of RAM.

The code used in the experiment to generate the data distributions on Gr(1 , 3) is shown in Listing 1. This implementation of the data distributions is based on the codes in Kim et al. (2020) and Grathwohl et al. (2019) 6 .

C.3.2 DW4 and LJ13

Network Architecture As in Appendix C.3.1, the vector field was constructed with the specific input, intermediate, and output layers described in Section 5.3. The GrCNF architecture is shown on the bottom left and right in Table 3. The bottom left and right were used for experiments on the DW4 and LJ13 datasets, respectively. The views presented in the table is the same as in Appendix C.3.1. (9) was used for the loss function. In addition, for architectures in methods other than GrCNF, please refer to Garcia Satorras et al. (2021).

Hyperparameters The mean M and covariances U and V in the prior distribution on the Grassmann manifold were set to M = I 4 × 2 , U = σ 2 I 4 , and V = σ 2 I 2 , σ = 0 . 3 for DW4 and M = I 13 × 3 , U = σ 2 I 13 , and V = σ 2 I 3 , σ = 0 . 3 for LJ13, respectively. Other hyperparameters used during the training of GrCNF are shown in Table 4. In addition, for the hyperparameters in methods other than GrCNF, please refer to Garcia Satorras et al. (2021).

Implementation The experimental hardware was built using a single NVIDIA Quadro RTX 8000 GPU with 48 GB of GDDR6 RAM. The other environments were the same as in Appendix C.3.1.

Full results In Tables 5 and 6, the same DW4 and LJ13 averaged results from Section 6.2 were reported; however, they included the standard deviations over the three runs.

C.3.3 QM9 Positional

Network Architecture On the QM9 Positional, we addressed the task of generating the molecular P by estimating the scale parameter √ Λ = diag ( { √ λ i } 3 i =1 ) , in addition to the generation of the orthonormal basis matrix Y with GrCNF. Because the molecular generation task requires a specialized loss function based on variational inference, we used (56), as explained in Appendix C.2. We designed two networks to achieve this. The first is the same GrCNF architecture as in previous experiments, and the second is a scale estimator. Table 7 shows the architectures. The left side of the table shows the GrCNF architecture and the right side shows the scale estimator. The scale estimator estimated one scale parameter from each of the three orthonormal basis vectors Y = { y i ∈ R 19 } 3 i =1 , for a total of three parameters { √ λ i ∈ R } 3 i =1 . With the orthonormal orthogonal basis matrix Y and the estimated scale parameter √ Λ , we generated a point cloud P = Y √ Λ . In this study, the overall architecture that generates P is also named GrCNF.

Hyperparameters The mean M and covariances U and V in the prior distribution on the Grassmann manifold were set to M = I 19 × 3 , U = σ 2 I 19 , and V = σ 2 I 3 , σ = 0 . 3 , respectively. In addition, for the hyperparameters in methods other than GrCNF, please refer to Garcia Satorras et al. (2021).

Implementation The experimental hardware was built using a single NVIDIA A100 GPU with 80GB PCIe of GDDR6 RAM.

D Fundamentals of Concepts Associated with Grassmann Manifold

D.1 Definition of Stiefel Manifold

Definition 2. An (orthogonal or compact) Stiefel manifold St( k, D ) is defined as the set of orthonormal bases of k -dimensional subspaces in the Euclidean space R D as in (68).

For Y ∈ St( k, D ) , the space span( Y ) spanned by its column vectors is the element of Gr( k, D ) .

St( k, D ) is a Dk -k ( k +1) 2 -dimensional compact manifold (Absil et al. (2008)).

D.2 Equivalence Relation

To define the equivalence relation ∼ 7 on a Stiefel manifold St( k, D ) , we introduce the following two lemmas.

Lemma 1. The necessary and sufficient conditions for span( Y 1 ) = span( Y 2 ) to hold for Y 1 , Y 2 ∈ St( k, D ) are as follows.

Proof. span( Y 1 ) = span( Y 2 ) ⇐ Y 2 = Y 1 Q . From the definition, Y ⊤ 1 Y 1 = Y ⊤ 2 Y 2 = I k , the following is obtained:

Thus, there exists a k -dimensional orthogonal matrix Q ∈ O ( k ) . As the subspace span( Y ) is invariant to coordinate transformations by orthogonal matrices, span( Y 1 ) = span( Y 2 ) is true.

span( Y 1 ) = span( Y 2 ) ⇒ Y 2 = Y 1 Q . From the assumption, we immediately concluded that Y 2 = Y 1 Q for Q ∈ O ( k ) .

Lemma 2. We define the equivalence relation ∼ on St( k, D ) to be Y 1 ∼ Y 2 whenever (70) is satisfied with respect to Y 1 , Y 2 ∈ St( k, D ) . The fact that a binary relation is an equivalence relation ∼ implies that the following three statements hold for ∀ Y 1 , Y 2 , Y 3 ∈ St( k, D ) .

Reflexivity Y 1 ∼ Y 1 .

Symmetry Y 1 ∼ Y 2 ⇒ Y 2 ∼ Y 1 .

Transitivity Y 1 ∼ Y 2 ∧ Y 2 ∼ Y 3 ⇒ Y 1 ∼ Y 3 .

Proof. With Lemma 1, we can confirm that it is valid as follows:

Reflexivity From Y 1 = Y 1 I , I ∈ O , we obtain Y 1 ∼ Y 1 .

Symmetry As Y 2 = Y 1 Q is obtained from Y 1 ∼ Y 2 , and Y 2 Q ⊤ = Y 1 , Q ⊤ ∈ O is true, then Y 2 ∼ Y 1 is obtained.

Transitivity Y 3 = Y 1 Q 1 Q 2 with Y 2 = Y 1 Q 1 and Y 3 = Y 2 Q 2 . Q 1 Q 2 is ( Q 1 Q 2 ) ⊤ ( Q 1 Q 2 ) = Q ⊤ 2 Q ⊤ 1 Q 1 Q 2 = Q ⊤ 2 Q 2 = I . Moreover, as ( Q 1 Q 2 ) ( Q 1 Q 2 ) ⊤ = Q 1 Q 2 Q ⊤ 2 Q ⊤ 1 = I holds, Q 1 Q 2 ∈ O is true. Therefore, we concluded Y 1 ∼ Y 3 .

The equivalence class of Y ∈ St( k, D ) is denoted by [ Y ] . In other words, [ Y ] is the set of all elements of St( k, D ) that are equivalent to Y , and the equivalence relation on St( k, D ) divides St( k, D ) into equivalence classes with no intersection. Thus, Y is then referred to as the representative of the equivalence class [ Y ] . The set of equivalence classes is denoted St( k, D ) / ∼ and is referred to as the

quotient of St( k, D ) by the equivalence relation ∼ . In addition, π : → St( k, D ) → St( k, D ) / ∼ is the natural projection that maps Y ∈ St( k, D ) onto its equivalence class [ Y ] . This projection π is surjection.

D.3 Quotient Manifold

Definition 3. Let M be a manifold with equivalence relation ∼ . The quotient space M / ∼ with ∼ of M is the set of all equivalence classes. Thus, M / ∼ := { π ( x ) | x ∈ M} , where π : M→M / ∼ is the natural projection and π ( x ) := { y ∈ M | y ∼ x } . Then, M is referred to as the total space or the total manifold. Moreover, M / ∼ is referred to as a quotient manifold of M if M / ∼ admits a differentiable structure.

Let M = M / ∼ be a quotient manifold. Further, suppose that M is endowed with a Riemann metric g , and let x = π ( x ) . The horizontal space T h x M is the orthogonal complement of the vertical space T v x M := T x π -1 ( x ) in the tangent space T x M and is defined as the follows:

The horizontal lift ξ h x ∈ T h x M of the tangent vector ξ x ∈ T x M at point x ∈ π -1 ( x ) is a tangent vector that is uniquely determined as dπ x ( ξ h x ) = ξ x (Absil et al. (2008)).

D.4 Grassmann Manifold Exploiting the Quotient Structure

D.4.1 Tangent Space on a Stiefel Manifold

We describe the relationship between tangent space T [ Y ] Gr( k, D ) on Gr( k, D ) and tangent space T Y St( k, D ) on St( k, D ) to relate the tangent vectors of a Grassmann manifold Gr( k, D ) to the tangent vectors of a Stiefel manifold St( k, D ) in a matrix representation. We take the derivative on both sides of Y ( t ) ⊤ Y ( t ) = I p in (68) by t and solve for t = 0 .

where ξ Y = d dt Y (0) is the tangent vector at Y 8 .

Definition 4. Define the tangent space T Y St( k, D ) at Y on the Stiefel manifold as follows:

Let matrix Y ⊥ ∈ R D × ( D -k ) be a matrix satisfying the following:

As [ Y Y ⊥ ] is an orthogonal matrix 9 , the column vectors of Y and Y ⊥ form an orthonormal basis in R D . Thus, any D × k matrix can be written in terms of the C ∈ R k × k and B ∈ R ( D -k ) × k coefficient matrices as follows:

where ξ Y = Y C + Y ⊥ B is inserted. The following equation is obtained.

Thus, the following equation is derived.

Thus, C is a k × k skew-symmetric matrix Skew( k ) . Therefore, we obtain the following as another representation of the tangent space on St( k, D ) .

D.4.2 Riemannian Metric on a Stiefel Manifold

In the tangent space T x M defined at each point x ∈ M on the manifold M , the inner product h is endowed as a bilinear map. This h is referred to as a Riemannian metric on a manifold, and the manifold M on which the Riemannian metric h is endowed is referred to as a Riemannian manifold ( M , h ) . We define the Riemannian metric g on St( k, D ) as follows.

This is the standard inner product of R D × k induced by T Y St( k, D ) , with T Y St( k, D ) ⊂ R D × k 1011 .

D.4.3 Tangent Space on a Grassmann Manifold

We describe the relation between tangent spaces T [ Y ] Gr( k, D ) and T Y St( k, D ) to relate tangent vectors in tangent spaces T [ Y ] Gr( k, D ) on Gr( k, D ) to tangent vectors ξ Y ∈ T Y St( k, D ) .

First, we define the vertical space T v Y St( k, D ) as a subspace of T Y St( k, D ) as follows.

where π : St( k, D ) → Gr( k, D ) is the natural projection defined by π ( Y ) = [ Y ] 12 . Thus, π converges all Y ′ ∈ St( k, D ) such that Y ∼ Y ′ to a point [ Y ] on Gr( k, D ) . Therefore, using (1), (88) can be transformed as follows.

However, ξ v Y ∈ T v Y St( k, D ) can be written as ξ v Y = Y S with S ∈ T I k O ( k ) .

Thus, we obtain the following formula.

Next, we define the horizontal space T h Y St( k, D ) as the orthogonal complement of T v Y St( k, D ) in T Y St( k, D ) endowed with the inner product (87).

Based on the fact that T v Y St( k, D ) is a subspace of T Y St( k, D ) and T h Y St( k, D ) is defined as its orthogonal complement, the direct sum decomposition is as follows.

where ⊕ denotes direct sum. Moreover, the tangent space is a linear space (Absil et al. (2008)). From (86), element Y C of T v Y St( k, D ) corresponds to the first term of (93); thus, (96) is formulated as follows:

Note that the horizontal vector ξ h Y is not necessarily an orthogonal matrix.

Finally, define the element ξ h Y ∈ T h Y St( k, D ) of the horizontal space at Y ∈ St( k, D ) for the tangent vector ξ [ Y ] ∈ T [ Y ] Gr( k, D ) at [ Y ] ∈ Gr( k, D ) as satisfying the following formula.

where dπ ( Y ) : T Y St( k, D ) → T [ Y ] Gr( k, D ) is the derivative dπ ( Y ) d Y of π : St( k, D ) → Gr( k, D ) at Y ∈ St( k, D ) . The ξ h Y ∈ T h Y St( k, D ) is referred to as the horizontal lift at Y ∈ St( k, D ) of [ Y ] ∈ Gr( k, D ) .

We describe the tangent space of Gr( k, D ) with the concept of horizontal lift.

Definition 5. Let T h Y St( k, D ) be a horizontal space on St( k, D ) . Then, we define the tangent space T [ Y ] Gr( k, D ) of the Gr( k, D ) as follows.

From the above, ξ [ Y ] ∈ T [ Y ] Gr( k, D ) is obtained from the map dπ ( Y ) [ ξ h Y ] when ξ h Y is obtained. The ξ [ Y ] is defined by an equivalence class and cannot be treated numerically in matrix form; however, it is sufficient to obtain the ξ h Y for actual numerical calculations. For ξ [ Y ] ∈ T [ Y ] Gr( k, D ) , there exists a ξ h Y ∈ T h Y St( k, D ) that uniquely satisfies (98). In other words, we can handle it in matrix form by using elements of the horizontal space of Stiefel manifolds through the concept of horizontal lifting. Figure 1 is a conceptual diagram of the tangent space representation of a Grassmann manifold by horizontal lift.

D.4.4 Riemannian Metric on a Grassmann Manifold

We define the Riemannian metric g of Gr( k, D ) through the concept of horizontal lift.

Definition 6. Let ξ h Y and η h Y be the horizontal lifts that become dπ ( Y ) [ ξ h Y ] = ξ [ Y ] and dπ ( Y ) [ η h Y ] = η [ Y ] , respectively. Then, we define the Riemannian metric on Gr( k, D ) as follows:

where B and D are matrices that are ξ h Y = Y ⊥ B and η h Y = Y ⊥ D , respectively.

D.5 Invariant Measures

Let the column vectors of matrix Y = { y 1 , · · · , y k } ∈ R D × k be the orthonormal basis that span the subspace span( Y ) ∈ Gr( k, D ) in R D , and the column vectors of Y ⊥ = { y k +1 , · · · , y D } ∈ R D × D -k be the orthogonal complementary space span( Y ⊥ ) of span( Y ) , respectively. Then, the following differential form can be defined.

where ∧ is the wedge product and the relation satisfies ω i ∧ ω i = ω j ∧ ω j = 0 and ω i ∧ ω j = -ω j ∧ ω i . The above equation is in k ( D -k ) -order differential form, which is an invariant measure on Gr( k, D ) (Chikuse (2003)).

If we define the matrix X ⊥ to be [ X X ⊥ ] for any point X = { x 1 , · · · , x k } ∈ St( k, D ) , the differential form for an invariant measure on St( k, D ) is defined as follows.

where ( d Q ) is the invariant measure of O ( k ) . The integral of (103), that is, the volume of St( k, D ) , can be evaluated as follows:

(104) can be computed as follows. First, the surface S D of the D -dimensional unit sphere can be defined as follows:

where V D is the volume of a D -dimensional sphere π D 2 Γ ( D 2 +1 ) r D and Γ ( D 2 ) is the gamma function. Then, the following equation is obtained.

where ( d X 1 ) is the differential form of St( k -1 , D -1) . Thus, (104) can be transformed as follows.

where Γ k ( D 2 ) is the multidimensional gamma function. The invariant measure ( d X ) is an unnormalized measure. A measure normalized to be a probability measure can be formulated as follows:

This is a uniform distribution on St( k, D ) . As St( k, k ) = O ( k ) , the volume V O ( k ) of O ( k ) can be represented using ( d Q ) as follows.

Furthermore, a measure normalized to be a probability measure can be represented by the following:

From the above, the probability density function U O ( k ) ( Q ) of the uniform distribution on O ( k ) is as follows:

As Gr( k, D ) is defined as a quotient manifold St( k, D ) / O ( k ) as in (2), the volume V Gr( k,D ) of Gr( k, D ) can be defined as follows.

The measure normalized to be a probability measure is expressed as:

̸

D.6 Retraction

In general, the points except the origin ( p (0) = x ) of the tangent space T x M at x on the manifold M are not elements on M ( p ( t ) ∈ T x M , t = 0 ). Therefore, if the result of the operation on the tangent space is to be used at another point on the manifold M , it is necessary to map p ( t ) to the manifold M . The map from a tangent space to a manifold is referred to as an exponential map. However, because the exponential map is computationally expensive, retraction based on numerical linear algebra is often used as an alternative (Zhu & Sato (2021)). Retraction is a method for approximating an exponential map to first order while maintaining global convergence in optimization algorithms on Riemannian manifolds. The most commonly used retractions on Gr( k, D ) are methods based on QR decomposition or singular-value decomposition (SVD) (Absil et al. (2008); Zhu & Sato (2021)). In addition, a retraction based on the Cayley transform is introduced in Zhu & Sato (2021). This retraction is closely related to the Cayley transform on St( k, D ) (Wen & Yin (2013); Xiaojing (2017); Zhu & Duan (2019)) and the Projected polynomial retraction (Gawlik & Leok (2018a)).

D.6.1 Exponential Map and Retraction

Geodesics on Gr( k, D ) can be expressed as the equivalence class [ exp Gr Y ( t ξ Y )] , where

Here, exp on the right-hand side is the matrix exponential, and B = [ 0 k -B ⊤ B 0 D -k ] ∈ skew ( D ) , where B satisfies ξ Y = Y ⊥ B . We can use a following exponential map that is mathematically equivalent to (115) (Edelman et al. (1998)):

Further, we can use the Padé approximation to approximate geodesics on Grassmann manifolds as follows:

where r m ( X ) is the m th-order diagonal Padé approximation to the matrix exponential exp( X ) . See the expression of r m ( X ) in Moler & Loan (2003). The simplest member of this class is surely the first-order Padé approximation

which is also known as the Cayley transform. From the error expression exp( Y ) = r m ( Y ) + O ( ∥ Y ∥ 2 m +1 ) of the Padé approximation, we have

which is also given by Theorem 3 in Gawlik & Leok (2018b).

D.6.2 Horizontal Retraction

From Definition 3 in Zhu & Sato (2021), (119) is a horizontal retraction, and

is a retraction on Gr( k, D ) as a quotient manifold defined by (2). This is because R satisfies the invariance condition that R Y ( t ξ h Y ) ∼ R Y ′ ( t ξ h Y ′ ) for all Y ∈ St( k, D ) , Y ′ ∈ St( k, D ) , ξ Y ∈ T h Y St( k, D ) and ξ Y ′ ∈ T h Y ′ St( k, D ) such that Y ∼ Y ′ and ξ Y and ξ Y ′ are horizontal lifts of ξ [ Y ] ∈ T [ Y ] Gr( k, D ) at Y and Y ′ , respectively.

In low-rank cases, we can obtain an economical version of (119) as follows (Zhu & Sato (2021)).

The inverse retraction ( R -1 [ Y ] ) h Y : St( k, D ) → T h Y St( k, D ) of R Y ( ξ h Y ) is the following:

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a normalizing flow approach applicable to generate data on Grassmann manifold. The proposed method utilize the fact that Grassmann manifold is a quotient space of Stiefel manifold on which numerical calculation is feasible. Specifically, the method first constructs the flow from vector field and then transform a probabilistic density using the flow. The vector field is learned through a neural network. A prior probabilistic density is defined as a matrix-variate Gaussian distribution on Grassmann manifold. The experiments are conducted on three different types of datasets among which the first one is synthetic and the last one is molecular point clouds. The experimental results show that the proposed GrCNF can generate the distribution on Grassmann manifold.\n\nSTRENGTHS:\nThis paper aims to resolve the problem of shape generation when the high-dimensional shapes intrinsically embed in a low-dimensional manifold, e.g., Grassmann manifold, using normalizing flow (NF). Extending NF from common Euclidean space to non-linear manifold is novel and meaningful, as the manifold hypothesis assumes that most high-dimensional data usually embeds in a low-dimensional manifold. The theoretical part looks sound. Experiments under three different settings verify the effectiveness of the proposed GrCNF to learn distributions on Grassmann manifold.\n\nWEAKNESSES:\nThe main weakness (also my concern) is, as also mentioned by the authors in Line 325 - 328, whether the proposed GrCNF is applicable to  high-dimensional data? E.g., point clouds containing thousands of points? If yes, it will make the paper very strong in both theoretical and application aspects. (I also want to know what real-world high-dimensional data embeds in Grassmann manifold?) If not applicable to high-dimensional data, it will weaken the paper.\n\nQUESTIONS:\nIn general:\n\n1. What's the meaning of \"stable shape\" in the title? Why does the generation process is stable?\n\nMethod parts:\n\n2. In Figure 1, what's the relation between T^v_YSt(k, D) and T^h_YSt(k, D) and why do we use T^h_Y, not use T^v_Y?\n\n3. In Line 106, it mentions that \"we use the representative Y ...\", so how to choose the representative Y?\n\n4. In Proposition 1, what's the relation between a vector field and a flow? I.e., how to compute a flow F_{X, T} given a vector field X?\n\n5. Why do we need retraction in Eq.(4) in Proposition 3?\n\n6. In Section 5.3 and Figure 2. does the output (through a neural network) is ensured in the tangent space of T^h_YSt(k, D) theoretically?\n\n7. In Section 5.4, what's the goal of the ODE solver? Could you use one or two sentences to summarize it.\n\nExperiment parts:\n\n8. In the experiment in Section 6.1, I'm not sure why the spiral (and swissroll and circle ...) are 1-dimensional data in R^3, as they contain x and y two coordinates.\n\n9. In the experiment in Section 6.2 and 6.3, is it possible to visualize the generated data, similar to Figure 3.\n\nImplementation parts:\n\n10. As the method is complicated in theory, will the code be released?\n\nLIMITATIONS:\nThe authors mentions the two limitations of the proposed method in Line 324 - 328.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the application of continuous normalizing flow (CNF) to the Grassmann manifold for shape generation. The authors establish the theoretical foundations for learning distributions on the Grassmann manifold using CNF, enabling the generation of stable shapes. The proposed approach offers a promising generative model capable of generating subspace data.\n\nSTRENGTHS:\nThe study represents the first exploration of applying a continuous normalizing flow (CNF) to the Grassmann manifold for shape generation.  \n\nThe experiments conducted demonstrate the effectiveness of the proposed approach in generating promising shapes, particularly subspace data with low dimensions.\n\nWEAKNESSES:\nI have concerns regarding the significant contribution of the paper to the domain, given the existence of [51]. To address these concerns, it would be beneficial for the authors to include a comparative analysis with [51] in the experiments to demonstrate any potential superiority of the proposed method. \n  \nFurthermore, it is not clear how the proposed approach technically differs from the Stiefel-based ODE [7]. To provide a clearer understanding, it would be helpful if the authors could provide an intuitive illustration of the transfer from the Stiefel-based ODE to the Grassmann-based ODE. \n  \nAdditionally, the paper's validation is limited to low-dimensional data, and there is a lack of empirical study on real-world, high-dimensional data such as point clouds. As a result, the current validation may not be sufficiently convincing to establish the effectiveness of the proposed approach in practical scenarios. Conducting experiments on high-dimensional data, such as point clouds, would enhance the credibility and generalizability of the proposed method.\n\nQUESTIONS:\nThere should indeed be a comma rather than a period before finishing the equation, as seen in Eq. 8 on lines 232-233.\n\nRegarding the performances for different D and k values on DWR, LJ13, and QM9, it would be valuable for the paper to provide an analysis and evaluation of the proposed method's performance in these scenarios. This would enhance the understanding of the method's effectiveness across different datasets and settings.\n\nThe reason why the proposed GrCNF outperforms the counterpart methods in Table 1 should be clarified in the paper. It would be beneficial for the authors to provide an explanation or analysis that highlights the specific advantages or characteristics of the proposed method that contribute to its superior performance.\n\nRegarding the stability of shape generation in the proposed method compared to other methods, the paper should provide a clearer explanation to address this question. It would be helpful for the authors to discuss the specific mechanisms or techniques employed in the proposed method that lead to more stable shape generation results.\n\nThe feasibility and necessity of bringing invertibility to normalizing flows could be an interesting aspect to explore. It would be valuable for the authors to discuss the potential benefits and drawbacks of incorporating invertibility into the proposed method, considering the existing literature and the specific requirements and objectives of the task at hand.\n\nLIMITATIONS:\nThe paper does not provide a thorough comparison with existing methods, such as [51], to demonstrate the superiority of the proposed approach. A comparative analysis would have helped assess the advancements of the proposed method over previous work.\n\nThe paper lacks empirical studies on real-world, high-dimensional data, such as point clouds. This limitation restricts the generalizability and applicability of the proposed approach to practical scenarios, as the effectiveness of the method has only been demonstrated on low-dimensional data. \n\nIt is not adequately explained how the proposed approach technically differs from the Stiefel-based ODE [7]. A clearer illustration of the transfer from Stiefel-based ODE to Grassmann-based ODE would have provided a better understanding of the method's technical contributions. \n\nWhile the paper claims that the proposed GrCNF outperforms counterpart methods, it does not provide a comprehensive explanation for this superiority. It would be beneficial to discuss and analyze the specific factors or characteristics of the proposed method that contribute to its improved performance.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends the previous Riemannian CNF to subspaces, i.e., Grassmann Manifold Gr(k,D), and tackle stable shape generation with nice transformation-invariant property. They derive the first and full mathematical formulation for learning invariant densities and flow on a Grassmann manifold. They successfully bring the complete model to life by using Stiefel Manifold to help construct the vector field effectively, with adapted ODE solver on Gr(k, D). They have also shown the superior performance of the proposed method on both toy and well-adopted public benchmarks compared to several existing seminar works.\n\nSTRENGTHS:\n[**originality**] the proposed work is the first one to explore continuous normalizing flow on subspace on the Grassmann Manifold. They not only creatively combine the success of quotient manifold for shape analysis, and the recent Riemannian CNF, but also contributed an elegant way of constructing the vector field through the connection to Stiefel manifold with horizontal lift. \n\n[**quality & clarity**] The writing quality is over the bar, with clean explanation on the motivation, intuition, and design. As a person who is not that familiar with manifold theory, I found that the mathematical derivation is straight-forward to understand. They have presented complete and detailed propositions and proofs throughout the main paper and supp. \n\n[**significance**]  The proposed framework is a pioneer work on exploring continuous normalizing flow on subspaces with nice equivariance-preserving property. Compared to existing baselines, this method is achieving SOTA performance on simple shape generation. It has great potential to benefit 3D molecules design, other high-dimensional geometry data generation including common objects.\n\nWEAKNESSES:\n1. From Figure 2, all that has been showing is on the Stiefel Manifold, without showing how it is connected to the target Grassmann manifold, which is confusing and not informative. Without this connection, I am not clear how the nice O(k)-invariant output v_out is achieved. The author might want to update the figure, or provide a better clarification;\n\n2. As the author also mentions, it will make the proposed work more significant if showing experimental results on real-world object shapes.   More importantly, I am interested in knowing the major challenges to use the developed framework to handle such object shape data;\n\n3. On the public benchmarks, the quantitative numbers are better than the baselines, it will make the comparison more complete if we can further show the visualizations of the generated samples;\n\nQUESTIONS:\n1. I am also wondering about the computation overhead, besides the expensive ODE solver, how many steps does it take to generate the final sample from a Gaussian density?\n\n2. When comparing with the baselines, the proposed method will instead take the orthonormalized data, I am wondering whether this is a fair comparison setting.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel concept: Continuous Normalizing Flows on a Grassmann Manifold (GrCNF), a model that presents a more efficient and accurate way to capture and represent the geometric features of data. The unique proposition of GrCNF is its application of the Grassmann Manifold to grasp data equivariance, a key aspect that delineates the transformation response of data. GrCNF's efficacy is demonstrated via several experiments. The results affirm its capability to accurately generate data that depict complex geometric shapes, and outperform current Normalizing Flow models in metrics like log-likelihood and ELBO for diverse test datasets.\n\nSTRENGTHS:\n+ GrCNF is a novel model to bring continuous normalizing flows (CNF) into the field of Grassmann manifolds, expanding its applications to data manifolds with geometric structures.\n+ GrCNF showed superior performance when compared to existing Normalizing Flow models in experiments with different datasets. It not only generated high-quality samples but also effectively learned the distribution on Gr(k, D).\n+ GrCNF is able to handle subspace data, making it a versatile tool for handling various types of complex data.\n\nWEAKNESSES:\n1. The use of the Kronecker product at each ODE step is a demanding process, which suggests potential improvements in computational efficiency could be beneficial. This exploration could lead to exciting developments in scalability and applicability.\n2. The study primarily focused on lower-dimensional data, and an examination of the GrCNF model's performance with higher-dimensional data could provide additional valuable insights.\n\nQUESTIONS:\nIn the artificial texture experiments, the antipodal points were generated using the trained GrCNF. While this approach demonstrates the learning capabilities of the model, it does not guarantee the accurate representation of all antipodal points in different contexts or datasets.\n\nLIMITATIONS:\n1. The proposed GrCNF has been tested on a limited variety of data types. Further exploration is required to understand its potential application and effectiveness on different data types, such as image, audio, and textual data.\n2. In the experiments, GrCNF is mainly compared with existing normalizing flows methods. The inclusion of other machine learning models, such as generative adversarial networks (GANs) or autoencoders, could have provided a broader perspective on GrCNF's performance.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is on developing techniques for sampling from probability distributions on Grassmann manifolds and utilizing it to generating shapes. The shapes are associated with the probability distributions on Grassmann and sampling from the distribution generates shapes. The sampling itself it based on flows under vector fields associated with the gradient of potentials (negative logs of pdfs). A large amount of paper is dedicated towards defining vectors fields and flows on a Grassmann manifold assuming one knows how to do it on a Stiefel manifold. Some of these constructions are implemented using deep neural networks. The paper provides some examples of shape and texture generation and compares it to some similar methods.\n\nSTRENGTHS:\n-- The paper is trying to make a connection between a number of distinct and important mathematical quantities: shapes, distributions on Grassmann manifolds, random sampling, gradient flows, etc. \n\n-- The geometry part of the paper is solid. The use of horizontal lift to remove the nuisance group is a standard technique in analysis on quotient spaces.\n\nWEAKNESSES:\n-- I find this paper somewhat confusing to read. The presentation relies on CNF and Riemannian CNF etc, while these quantities are standard flows under given gradient fields (associated with given pdfs). \n\n\n-- Perhaps the authors can clarify what is different in CNFs from a integral curves or flows under vector fields such as gradient vector fields. See for example, the paper by Liu et al., Optimal Linear Representations of Images for Object Recognition, IEEE TPAMI, May 2004. These are very standard constructions for either Stiefel or Grassmann manifolds. \n\n-- The connection with shapes is also unclear. What do the authors mean by a shape? They should explain precisely (preferably in mathematical terms) on how shapes are represented and how sampling from a distribution on a Grassmann becomes generation of a shape.\n\nQUESTIONS:\nSee my question listed above. \n\nThe authors need to clarify in simple layman's terms the big picture of their approach. This can be done without using undefined terminology and stating ideas precisely.\n\nLIMITATIONS:\nYes, they have listed some limitations of the computational steps of their framework.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-08"}