SlowGuess/NeurIPS_2023_OCR huggingface dataset:
RCNF (Mathieu & Nickel (2020)) framework is used for loss function, integration and divergence calculations. In the integration steps, the ordinary differential equation (ODE) is solved using the ODE solver with orthogonal integration. The learnable parameters are updated using backpropagation to maximize the sum of the computed log-likelihood. In order to efficiently compute gradients, we introduce an adjoint method to calculate analytically the derivative of a manifold ODE, based on the results of Lou et al. (2020).
5.2 Sampling Algorithm from Prior
We propose a sampling algorithm on  $\mathrm{Gr}(k,D)$  derived from the Cayley transform, according to the results of Section 4.3. The algorithm of sampling from a distribution on  $p_{\mathrm{Gr}(k,D)}$  is described in Algorithm 1. Vec denotes the map of vertically concatenating matrices and converting them into a vector and  $\mathbf{I}_D$  is a  $D\times D$  identity matrix.
5.3 Construction of Vector Field
We describe the method for constructing the vector field  $\mathsf{X}_{\theta}$ . The learnable parameters  $\theta$  are feedforward neural networks, which accept the representative  $\mathbf{Y} \in \mathrm{St}(k,D)$  of  $[\mathbf{Y}] \in \mathrm{Gr}(k,D)$  inputs and output a horizontal lift  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}} \in T_{\mathbf{Y}}^{\mathrm{h}}\mathrm{St}(k,D)$  of  $\pmb {\xi}_{[\mathbf{Y}]}\in T_{[\mathbf{Y}]}\mathrm{Gr}(k,D)$  that satisfies (4). The structure of  $\mathsf{X}_{\theta}$  is important because it directly determines the ability of the distribution to be represented. To address these geometric properties, the following specific input and intermediate layers are used to obtain a  $\mathcal{O}(k)$ -invariant function value  $v_{\mathrm{out}}$ . Figure 2 shows a conceptual diagram.
Input Layer The input layer maps the input representative  $\mathbf{Y}$  to a point  $\overline{\zeta}_{\mathbf{Y}}^{\mathrm{h}} \in T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$ . A horizontal projection  $\operatorname{HorP} : \operatorname{St}(k, D) \to T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D) \cong \mathbb{R}^{(D - k) \times k}$  is constructed as an input layer:
$$
\operatorname {H o r P} (\boldsymbol {Y}) = \boldsymbol {W} - \boldsymbol {Y} \boldsymbol {Y} ^ {\top} \boldsymbol {W}, \tag {6}
$$
where  $\mathbf{W}$  is the output of NN obtained with  $\mathbf{Y}\mathbf{Y}^{\top}$  as input. HorP is well-defined because it is uniquely determined regardless of how the representative  $\bar{\mathbf{Y}}\in [\mathbf{Y}]$  is chosen.
Intermediate Layer To model the dynamics over the horizontal space, intermediate layers based on neural networks are constructed. The layers are particularly important for the ability to represent the distribution, and various configurations are possible such as concatsquash (CS) layer (Grathwohl et al. (2019)). The layer can be stacked multiply, and the overall power of expression increases with the number of layers. The input  $\pmb{x}$  of the first intermediate layer is  $\pmb{x} = \text{Vec} \circ \text{HorP}$ . The last intermediate layer (one layer before the output layer) must be constructed such that it exhibits a potential function value. This is to obtain the gradient for the potential function in the output layer.
Output Layer We construct a gradient  $\operatorname{Grad}:(\mathbb{R},\operatorname{St}(k,D))\to T_{\mathbf{Y}}^{\mathrm{h}}\operatorname{St}(k,D)$  as an output layer. In  $\operatorname{Grad},\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}}\in T_{\mathbf{Y}}^{\mathrm{h}}\operatorname{St}(k,D)$  is obtained by taking the gradient of the potential function  $v_{\mathrm{out}}$  ..
$$
\operatorname {G r a d} \left(v _ {\text {o u t}}, \boldsymbol {Y}\right) = P _ {\boldsymbol {Y}} ^ {\mathrm {t}} \left(\operatorname {a u t o g r a d} \left(v _ {\text {o u t}}, \boldsymbol {Y}\right)\right) \text {s . t .} P _ {\boldsymbol {Y}} ^ {\mathrm {t}} (\boldsymbol {Z}) = \boldsymbol {Z} - \boldsymbol {Y} \operatorname {s y m} \left(\boldsymbol {Y} ^ {\top} \boldsymbol {Z}\right), \tag {7}
$$
where autograd  $(v_{\mathrm{out}},\mathbf{Y})$  denotes automatic differentiation (Paszke et al. (2017))  $\frac{\partial v_{\mathrm{out}}}{\partial Y}$ ,  $P_{Y}^{\mathrm{t}}(\mathbf{Z})$  denotes the projection of  $\mathbf{Z}$  onto  $T_{\mathbf{Y}}\operatorname {St}(k,D)$  and  $\operatorname {sym}(\pmb {B}) = (\pmb {B} + \pmb{B}^{\top}) / 2$  is a symmetric part of  $\pmb{B}$ . Grad is well-defined on  $\operatorname {Gr}(k,D)$  by the horizontal lift (Absil et al. (2008)).
5.4 ODE Solver with Orthogonal Integration
We present an ordinary differential equation (ODE) solver on  $\mathrm{Gr}(k,D)$ . Celledoni & Owren (2002) proposed an intrinsic ODE solver that was applicable to  $\mathrm{St}(k,D)$  and did not assume an outer Euclidean space. This solver works on  $\mathrm{St}(k,D)$ , thus it must be reformulated into a solver suitable for  $\mathrm{Gr}(k,D)$ . We introduce a solver on  $\mathrm{Gr}(k,D)$  via ODE operating on  $T_{\mathbf{Y}}^{\mathrm{h}}\mathrm{St}(k,D)$ , based on results from Section 4. This is an intrinsic approach regardless of whether the manifold has been embedded in a bigger space with a corresponding extension of the vector field. The ODE defined on  $T_{\mathbf{Y}}^{\mathrm{h}}\mathrm{St}(k,D)$  is obtained as:
$$
\frac {d \operatorname {V e c} (\epsilon (t))}{d t} = \nabla_ {\gamma} \overline {{R}} _ {\boldsymbol {Y}} ^ {- 1} \operatorname {V e c} \left(\mathrm {X} _ {\theta} (t, \gamma (t))\right) \text {s . t .} \gamma (t) = \overline {{R}} _ {\boldsymbol {Y}} (\epsilon (t)), \tag {8}
$$
where  $\epsilon : [0, \infty) \to T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$  is the curve on  $T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$ ,  $\overline{R}_{\mathbf{Y}}$  denotes the horizontal retraction,  $\nabla_{\gamma} \overline{R}_{\mathbf{Y}}^{-1}$  can be calculated using (55) and Vec denotes the map of vertically concatenating matrices and converting them into a single vector. Further details on our ODE can be found in Appendix C.1.
5.5 Loss Function
The total change in log-likelihood using GrCNF can be calculated using the following equation.
$$
\log p _ {\theta} \left(\boldsymbol {Y} \left(t _ {1}\right)\right) = \log p _ {\operatorname {G r} (k, D)} \left(F _ {\mathsf {X}, t _ {1} \theta} ^ {- 1} \left(\boldsymbol {Y} \left(t _ {1}\right)\right)\right) - \int_ {t _ {0}} ^ {t _ {1}} \operatorname {d i v} \left(\mathsf {X} _ {\theta} \left(t, \boldsymbol {Y} \left(t\right)\right)\right) d t, \tag {9}
$$
where  $F_{\mathsf{X},t_1\theta}$  is a flow. In this study, we defined the loss function Loss for maximizing the log-likelihood:  $\mathrm{Loss} = \mathrm{NLL} = -\log p_{\theta}\left(\boldsymbol {Y}(t_1)\right)$  where NLL denotes negative log-likelihood.
In this section, we discuss the results of several experiments for validating GrCNF. Further details regarding each experimental condition, such as the architectures and hyperparameters used in training, can be found in Appendix C.3.
6.1 Generation and Density Estimation on Artificial Textures
We first trained GrCNF on five different  $\mathrm{Gr}(1,3)$  (1-dimensional subspace in  $\mathbb{R}^3$ , that is, a line through the origin) data to visualize the model and the trained dynamics. The five datasets were 2 spirals, a swissroll, 2 circles, 2 sines, and Target. We represented  $\mathrm{Gr}(1,3)$  with a sphere of radius 1 centered at the origin by mapping a 1-dimensional subspace to two points on the sphere (a point on the sphere and its antipodal point) for visualization.
Result The five types of probability densities transformed by GrCNF and the results of data generation are shown in Figure 3. The top, middle, and bottom rows in the figure present the correct data, the generated data with GrCNF when the left-most column is the prior distribution, and the probability density on a  $\mathrm{Gr}(1,3)$  obtained by training when the leftmost column was the prior distribution, respectively. Regarding the probability density, a brighter region corresponds to a higher probability. As shown in the figure, GrCNF can generate high-quality samples that are sufficiently accurate to the distribution of the correct data. To investigate whether GrCNF learns the distribution on  $\mathrm{Gr}(1,3)$ , we generated antipodal points in Figure 3 with the trained GrCNF used in Figure 3. As shown in Figure 4, GrCNF generated exactly the same high quality samples as the original for the untrained antipodal points. This experimental result implies that the proposed flow accurately learns the distribution on a  $\mathrm{Gr}(k,D)$  and that all the orthogonal transformed samples can be obtained with equal quality by only training with arbitrary representatives.
<sup>3</sup>The code for the data distributions is presented in Appendix C.3.1.
Figure 3: Generated samples and probability densities using the GrCNF trained on each of the five distributions. (top) Ground truth data, (middle) Generated data with the GrCNF when the leftmost column represents the prior distribution, and (bottom) densities obtained via training.
Figure 4: Results of the GrCNF transformation for the antipodal point of the prior in the middle panel of the Figure 3. The antipodal points are those that pass through the origin of the sphere and appear in the southern hemisphere (leftmost). It is evident that the antipodal points are generated with exactly the same quality as in the Figure 3 for the un-trained antipodal points.
6.2 Comparison with Conventional Methods for DW4 and LJ13
Comparative experiments were performed using DW4 and LJ13—the two systems presented by Kohler et al. (2020). These datasets were generated synthetically by sampling from their respective energy functions using Markov chain Monte Carlo (MCMC) methods. Both energy functions (DW4 and LJ13) are ideal for analyzing the advantages of various methods when they exist on data that are equivariant for rotation and reflection, respectively. We used the datasets generated by MCMC that were employed in Kohler et al. (2020). The orthonormalized data  $\mathbf{Y}$  were used for this study by applying Gram-Schmidt's orthogonalization to each column of  $\mathbf{P}$  such that the  $D$ -dimensional data was a matrix with  $k$  orthonormal basis vectors (DW4:  $D = 4$ ,  $k = 2$ ; LJ13:  $D = 13$ ,  $k = 3$ ).
To match the experiment conducted in Garcia Satorras et al. (2021), 1,000 validation and testing samples each were used for both datasets. For DW4 and LJ13, different numbers of training samples, i.e.,  $\{10^{2},10^{3},10^{4},10^{5}\}$  and  $\{10,10^{2},10^{3},10^{4}\}$ , respectively, were selected, and their performance for each amount of data was examined. The proposed approach was compared with the state-of-the-art  $\mathrm{E}(n)$  equivariant flow (E-NF) presented by Garcia Satorras et al. (2021) and simple dynamics presented by Kohler et al. (2020). In addition, comparisons with graph normalizing flow (GNF), GNF with attention (GNF-att), and GNF with attention and data augmentation (GNF-att-aug) (data augmentation by rotation), which are non-equivariant variants of E-NF, were performed. All the reported values are averages of cross-validations (three runs). Otherwise, the network structures of all these conventional methods were the same as that used in Garcia Satorras et al. (2021).
Result Table 1 presents the results of the cross-validated experiments (negative log-likelihood; NLL) for the test samples. The proposed GrCNF outperformed both the conventional non-equivariant models (GNF, GNF-att, and GNF-att-aug) and the conventional equivariant models (Simple dynamics, E-NF) in all data domains.
Table 1: Negative log-likelihood comparison on the test partition of different methods for DW4 and LJ13 datasets for different amount of training samples averaged over 3 runs.
# of Samples	DW4	LJ13
\( 10^2 \)	\( 10^3 \)	\( 10^4 \)	\( 10^5 \)	10	\( 10^2 \)	\( 10^3 \)	\( 10^4 \)
GNF	-2.30	-7.04	-7.19	-7.93	6.77	-0.76	-4.26	-12.43
GNF-att	-2.02	-4.13	-5.25	-6.74	6.91	1.40	-6.81	-12.05
GNF-att-aug	-3.11	-4.04	-6.51	-9.42	2.95	-6.11	-13.94	-15.74
Simple dynamics	-1.22	-1.28	-1.36	-1.39	-1.10	-3.87	-3.72	-3.59
E-NF	-0.54	-9.89	-12.15	-15.29	-12.86	-15.75	-31.51	-32.83
GrCNF	-12.53	-13.74	-14.09	-16.07	-23.64	-44.24	-58.02	-58.71
Table 2: Results for the QM9 Positional dataset. (Left) Negative log-likelihood, -ELBO and JSD for the QM9 Positional dataset on the test data, and (Right) normalized histogram of relative distances between atoms for QM9 Positional and generated samples with GrCNF.
NLL	-ELBO	JSD
Simple dynamics	73.0	-	0.086
Kernel dynamics	38.6	-	0.029
GNF	-00.9	-	0.011
GNF-att	-26.6	-	0.007
GNF-att-aug	-33.5	-	0.006
E-NF	-70.2	-	0.006
GrCNF	NLL ≤ -85.3	-85.3	0.005
6.3 Comparison with Conventional Methods for QM9 Positional
A comparative experiment was performed using the QM9 Positional—a subset of the QM9 molecular dataset that considers only positional information. The purpose of this experiment was to evaluate the feasibility of generating a practical point cloud. The QM9 Positional comprises only molecules with 19 atoms/nodes, and each node has a 3-dimensional position vector associated with it. However, the likelihood of the molecules must be invariant to translation and rotation in 3-dimensional space; thus, the proposed model is suitable for this type of data. The dataset consisted of 13,831, 2501, and 1813 training, validation, and testing samples, respectively. In this experiment, we used evaluation metrics based on the NLL and Jensen-Shannon divergence (JSD; Lin (1991)), in accordance with the experiments conducted in Garcia Satorras et al. (2021). The JSD calculates the distance between the normalized histograms that are generated from the model and obtained from the training set, by creating a histogram of the relative distances among all the node pairs in each molecule.
GrCNF handles subspace data. Therefore, orthonormalized data  $\mathbf{Y}$  was used only in the proposed GrCNF, as in  $PP^{\top} \simeq Y\Lambda Y^{\top}$  s.t.  $\Lambda$  is diagonal (Huang et al. (2015)), such that the  $k$ -dimensional point cloud data  $P$  of  $N$  points  $N \times k$  matrix is a matrix with  $k$  orthonormal basis vectors ( $D = 19$ ,  $k = 3$ ). However, a complete point cloud generating task, such as QM9 Positional, must also store a scale parameter  $\sqrt{\Lambda}$  such that  $P = Y\sqrt{\Lambda}$ . Therefore, GrCNF incorporates an additional architecture to estimate  $\sqrt{\Lambda}$ . In addition, the proposed GrCNF encounters difficulties in computing the NLL in distribution  $p_{\psi}(P)$  of  $P$ ; thus, a new loss function is required to address this. In this study, the evidence lower bound (ELBO), i.e., the lower bound of the log-likelihood  $\log p_{\psi}(P)$ , was maximized using a variational inference framework. This is equal to the minimization of  $-\mathrm{ELBO}(P)$ . Further details regarding ELBO can be found in Appendix C.2.
We compared our GrCNF with the GNF, GNF-att, GNF-att-aug, Simple dynamics, Kernel dynamics, and E-NF methods. The network structure and experimental conditions for all these conventional methods were identical to those used in Garcia Satorras et al. (2021). In all the experiments, the training was performed for 160 epochs. The JSD values were the averages of the last 10 epochs for all the models.
Result Table 2 presents the NLL and JSD cross-validated against the test data. Although the proposed GrCNF could not directly compute the NLL in this problem setting, it outperformed all
Figure 5: Visualization results of generated samples in Section 6.3. We visualize Generated QM9 by GrCNF and E-NF, as well as the aligned data points from the QM9 dataset that are closest to the Generated QM9 samples. These visualizations illustrate that GrCNF is capable of generating realistic data points that align well with the existing dataset.
the other algorithms because the relation  $\mathrm{NLL} \leq -\mathrm{ELBO}$  is true in general. With regard to the JSD, GrCNF achieved the best performance. Figure 2 presents the normalized histograms of the QM9 Positional molecular data and the data generated by GrCNF, in relation to the JSD of GrCNF. As shown, the histograms of the molecules generated by GrCNF are close to the histograms of the data set; thus, GrCNF can generate realistic molecules stably.
We proposed the concept of CNF on a Grassmann manifold (GrCNF) for stable shape generation. The proposed model is a generative model capable of handling subspace data; i.e., it is a CNF that considers the equivariance on the Stiefel manifold. Through suitable experiments, the ability of the proposed GrCNF to generate qualitatively in 1-dimensional subspace datasets in  $\mathbb{R}^3$  was confirmed. Further, GrCNF significantly outperformed existing normalizing flows methods in terms of the log-likelihood or ELBO for DW4, LJ13, and QM9 Positional. It was shown that GrCNF can be used to generate realistic data more stably.
Societal Impact and Limitations Our work is concerned with accurately modeling the data topology; thus, we do not expect there to be any negative consequence in our application field. The proposed theory and implementation are valid in a remarkably general setting, although there are limitations that can be addressed in future works: 1. The proposed method requires calculation of the Kronecker product at each ODE step (Section 5.4), which is computationally expensive. 2. Our current paper does not include experiments on high-dimensional data. Therefore, future experiments should be conducted using higher dimensional data, such as point clouds that represent general object geometry. However, the results presented in this paper still provide numerous valuable insights in the field of stable shape generation.
A Appendix Overview
First, we present the details of the various proofs of Section 4 in Appendix B. Next, in Appendix C, we describe the network layers for the building GrCNF, and the detailed model architectures, hyperparameters, and implementation on the experiments. Finally, in Appendix D, we provide a summary of the fundamentals of a Grassmann manifold, which is the core concept of this study.
B.1 Proposition 1
First, we invoked the following two corollaries.
Corollary 1 (Diffeomorphism Invariance of Flows). Let  $F: \mathcal{M} \to \mathcal{N}$  be a diffeomorphism. If  $X$  is a smooth vector field over  $\mathcal{M}$  and  $\theta$  is the flow of  $X$ , then the flow of  $F_{*}X^{4}$  is  $\eta_t = F \circ \theta_t \circ F^{-1}$ , with domain  $N_{t} = F(M_{t})$  for each  $t \in \mathbb{R}$ .
Proof. See Lee (2003, Corollary 9.14).
Corollary 2 (Homogeneity Property). The horizontal lift  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}}$  at representative  $\mathbf{Y} \in \operatorname{St}(k, D)$  relative to  $\pmb{\xi}_{[\mathbf{Y}]} \in T_{[\mathbf{Y}]}\operatorname{Gr}(k, D)$  satisfies the following homogeneity (equivariance) property (4) with regard to  $\forall \mathbf{Q} \in \mathcal{O}(k)$ .
$$
\overline {{\xi}} _ {Y Q} ^ {\mathrm {h}} = \overline {{\xi}} _ {Y} ^ {\mathrm {h}} Q.
$$
Proof.  $\pi(\mathbf{Y}) = \pi(\mathbf{YQ})$  is true for  $\forall \mathbf{Y} \in \mathrm{St}(k, D), \mathbf{Q} \in \mathcal{O}(k)$ . Therefore,  $\pi(\mathbf{Y}) = (\pi \circ q)(\mathbf{Y})$  is true when defined as  $q(\mathbf{Y}) = \mathbf{YQ}$ . When the derivative  $d\pi(\cdot)[\cdot]$  of both sides is applied to the horizontal lift  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}}$  of  $\pmb{\xi}_{[\mathbf{Y}]}$ , the following is obtained:
$$
d \pi (\boldsymbol {Y}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} \right] = d (\pi \circ q) (\boldsymbol {Y}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} \right] = d \pi (q (\boldsymbol {Y})) \left[ d q (\boldsymbol {Y}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} \right] \right] = d \pi (\boldsymbol {Y} \boldsymbol {Q}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} \boldsymbol {Q} \right]. \tag {10}
$$
Moreover, from (98) which is definition of horizontal lift, the following equation is true.
$$
\boldsymbol {\xi} _ {[ Y ]} = d \pi (\boldsymbol {Y}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} \right] = d \pi (\boldsymbol {Y} \boldsymbol {Q}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y} \boldsymbol {Q}} ^ {\mathrm {h}} \right]. \tag {11}
$$
Subsequently, we obtain the following equation.
$$
\boldsymbol {\xi} _ {[ \mathbf {Y} ]} = d \pi (\mathbf {Y} \mathbf {Q}) \left[ \overline {{\boldsymbol {\xi}}} _ {\mathbf {Y} \mathbf {Q}} ^ {\mathrm {h}} \right] = d \pi (\mathbf {Y} \mathbf {Q}) \left[ \overline {{\boldsymbol {\xi}}} _ {\mathbf {Y}} ^ {\mathrm {h}} \mathbf {Q} \right]. \tag {12}
$$
Finally, the uniqueness of the horizontal lift yields  $\overline{\pmb{\xi}}_{\pmb{Y}\pmb{Q}}^{\mathrm{h}} = \overline{\pmb{\xi}}_{\pmb{Y}}^{\mathrm{h}}\pmb{Q}$ .
Proposition 1. Let  $\operatorname{Gr}(k, D)$  be a Grassmann manifold,  $X$  be any time-dependent vector field on  $\operatorname{Gr}(k, D)$ , and  $F_{\mathsf{X},T}$  be a flow on a  $X$ . Let  $\overline{X}$  be any time-dependent horizontal lift and  $\overline{F}_{\overline{X},T}$  be a flow of  $\overline{X}$ .  $\overline{X}$  is a vector field on  $\operatorname{St}(k, D)$  if and only if  $\overline{F}_{\overline{X},T}$  is a flow on  $\operatorname{St}(k, D)$  and satisfies invariance condition  $\overline{X} \sim \overline{X}'$  for all  $\overline{F}_{\overline{X},T} \sim \overline{F}_{\overline{X}',T}$ . Therefore,  $X$  is a vector field on  $\operatorname{Gr}(k, D)$  if and only if  $F_{\mathsf{X},T} := \left[\overline{F}_{\overline{\mathsf{X}},T}\right]$  is a flow on  $\operatorname{Gr}(k, D)$ , and vice versa.
Proof. Flow  $F_{\mathsf{X},T}$  on  $\mathbf{Gr}(k,D) \Rightarrow \mathbf{Vector~Field} \times \mathbf{on~Gr}(k,D)$ . Let  $\theta : \mathrm{St}(k,D) \times \mathcal{O}(k) \to \mathrm{St}(k,D), (\mathbf{Y},\mathbf{Q}) \mapsto \mathbf{Y}\mathbf{Q}$  be a map representing the right action of the orthogonal group. In addition, let  $F_{\mathsf{X},T}$  be a flow on  $\operatorname{Gr}(k,D)$  and  $\overline{F}_{\overline{\mathbf{X}},T}$  be a flow on  $\mathrm{St}(k,D)$ . These satisfy  $\overline{F}_{\overline{\mathbf{X}}\mathbf{Q},T} \sim$
${}^{4}{F}_{ * }$  denotes the pushforward,that is,another notation for the differential of  $F$  .
$$
\overline {{F}} _ {\mathsf {X}, T}, \overline {{F}} _ {\mathsf {X} \mathbf {Q}, T} \in F _ {\mathsf {X}, T}, \overline {{F}} _ {\mathsf {X}, T} \in F _ {\mathsf {X}, T}.
$$
$$
\begin{array}{l} \bar {X} (t, \bar {F} _ {\bar {X} Q, t} (\mathbf {Y} \mathbf {Q})) = \bar {X} (t, \bar {F} _ {\bar {X}, t} (\mathbf {Y}) \mathbf {Q}) (13) \\ = \frac {d}{d t} \left\{\bar {F} _ {\bar {X}, t} (\boldsymbol {Y}) \boldsymbol {Q} \right\} (14) \\ = \frac {d}{d t} (\theta \circ \bar {F} _ {\bar {X}, t}) (\mathbf {Y}) (15) \\ = d (\theta) _ {\boldsymbol {Y}} \left\{\frac {d}{d t} \bar {F} _ {\bar {X}, t} (\boldsymbol {Y}) \right\} (16) \\ = d (\theta) _ {\mathbf {Y}} \left\{\bar {\mathrm {X}} \left(t, \bar {F} _ {\bar {\mathrm {X}}, t} (\mathbf {Y})\right) \right\} (17) \\ = \bar {X} (t, \bar {F} _ {\bar {X}, t} (\mathbf {Y})) \mathbf {Q}. (18) \\ \end{array}
$$
Thus,  $\overline{\mathsf{X}}\sim \overline{\mathsf{X}}\mathbf{Q}$  is true. Therefore,  $\overline{\mathsf{X}}$  is the horizontal lift of the vector field  $\mathsf{X}$  on  $\operatorname {Gr}(k,D)$  and is unique for  $\mathsf{X}$ .
$\mathbf{Flow} F_{\mathbf{X},T}$  on  $\mathbf{Gr}(k,D) \Leftarrow \mathbf{Vector~Field} \times \mathbf{Gr}(k,D)$ . Let  $\theta : \mathrm{St}(k,D) \times \mathcal{O}(k) \to \mathrm{St}(k,D), (\mathbf{Y}, \mathbf{Q}) \mapsto \mathbf{Y}\mathbf{Q}$  be a map representing the right action of the orthogonal group. In addition, let  $\overline{\mathbf{X}}$  be a vector field over a horizontal bundle  $T^{\mathrm{h}}\mathrm{St}(k,D)$  on  $\mathrm{St}(k,D)$  and  $\overline{F}_{\overline{\mathbf{X}},T}$  be its flow. From the Corollary 1,
$$
\bar {F} _ {\theta_ {*} \circ \bar {X}, T} = \theta \circ \bar {F} _ {\bar {X}, T} \circ \theta^ {- 1} \tag {19}
$$