SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\boldsymbol {C} ^ {\top} + \boldsymbol {C} = \mathbf {0} _ {k}. \tag {85}
$$
Thus,  $\mathbf{C}$  is a  $k \times k$  skew-symmetric matrix Skew  $(k)$ . Therefore, we obtain the following as another representation of the tangent space on  $\operatorname{St}(k, D)$ .
$$
T _ {\boldsymbol {Y}} \operatorname {S t} (k, D) = \left\{\boldsymbol {Y} \boldsymbol {C} + \boldsymbol {Y} _ {\perp} \boldsymbol {B} \mid \boldsymbol {C} \in \operatorname {S k e w} (k), \boldsymbol {B} \in \mathbb {R} ^ {(D - k) \times k} \right\}. \tag {86}
$$
D.4.2 Riemannian Metric on a Stiefel Manifold
In the tangent space  $T_{\pmb{x}}\mathcal{M}$  defined at each point  $\pmb{x} \in \mathcal{M}$  on the manifold  $\mathcal{M}$ , the inner product  $h$  is endowed as a bilinear map. This  $h$  is referred to as a Riemannian metric on a manifold, and the manifold  $\mathcal{M}$  on which the Riemannian metric  $h$  is endowed is referred to as a Riemannian manifold  $(\mathcal{M}, h)$ . We define the Riemannian metric  $\overline{g}$  on  $\mathrm{St}(k, D)$  as follows.
$$
\bar {g} _ {\boldsymbol {Y}} \left(\bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}, \bar {\boldsymbol {\eta}} _ {\boldsymbol {Y}}\right) := \operatorname {t r} \left(\bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\top} \bar {\boldsymbol {\eta}} _ {\boldsymbol {Y}}\right) \quad \text {s . t .} \quad \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}, \bar {\boldsymbol {\eta}} _ {\boldsymbol {Y}} \in T _ {\boldsymbol {Y}} \operatorname {S t} (k, D), \boldsymbol {Y} \in \operatorname {S t} (k, D). \tag {87}
$$
This is the standard inner product of  $\mathbb{R}^{D\times k}$  induced by  $T_{\pmb{Y}}\mathrm{St}(k,D)$ , with  $T_{\pmb{Y}}\mathrm{St}(k,D)\subset \mathbb{R}^{D\times k}$  1011.
D.4.3 Tangent Space on a Grassmann Manifold
We describe the relation between tangent spaces  $T_{[\mathbf{Y}]} \mathrm{Gr}(k, D)$  and  $T_{\mathbf{Y}} \mathrm{St}(k, D)$  to relate tangent vectors in tangent spaces  $T_{[\mathbf{Y}]} \mathrm{Gr}(k, D)$  on  $\mathrm{Gr}(k, D)$  to tangent vectors  $\overline{\xi}_{\mathbf{Y}} \in T_{\mathbf{Y}} \mathrm{St}(k, D)$ .
First, we define the vertical space  $T_{\mathbf{Y}}^{\mathrm{v}} \operatorname{St}(k, D)$  as a subspace of  $T_{\mathbf{Y}} \operatorname{St}(k, D)$  as follows.
$$
T _ {\mathbf {Y}} ^ {\mathrm {v}} \operatorname {S t} (k, D) := T _ {\mathbf {Y}} \pi^ {- 1} ([ \mathbf {Y} ]), \tag {88}
$$
where  $\pi : \mathrm{St}(k, D) \to \mathrm{Gr}(k, D)$  is the natural projection defined by  $\pi(\mathbf{Y}) = [\mathbf{Y}]^{12}$ . Thus,  $\pi$  converges all  $\mathbf{Y}' \in \mathrm{St}(k, D)$  such that  $\mathbf{Y} \sim \mathbf{Y}'$  to a point  $[\mathbf{Y}]$  on  $\mathrm{Gr}(k, D)$ . Therefore, using (1), (88) can be transformed as follows.
$$
T _ {\mathbf {Y}} ^ {\mathrm {v}} \operatorname {S t} (k, D) = T _ {\mathbf {Y}} \left\{\mathbf {Y} \mathbf {Q} \mid \mathbf {Q} \in \mathcal {O} (k) \right\}. \tag {89}
$$
However,  $\overline{\pmb{\xi}}_{\pmb{Y}}^{\mathrm{v}}\in T_{\pmb{Y}}^{\mathrm{v}}\operatorname {St}(k,D)$  can be written as  $\overline{\pmb{\xi}}_{\pmb{Y}}^{\mathrm{v}} = \pmb {Y}\pmb{S}$  with  $S\in T_{I_k}\mathcal{O}(k)$
$$
\begin{array}{l} T _ {I _ {k}} \mathcal {O} (k) = T _ {I _ {k}} \operatorname {S t} (k, k) (90) \\ = \left\{\boldsymbol {I} _ {k} \boldsymbol {C} + \left(\boldsymbol {I} _ {k \perp} \boldsymbol {B} = \mathbf {0} _ {k}\right) \mid \boldsymbol {C} \in \operatorname {S k e w} (k), \boldsymbol {B} \in \mathbb {R} ^ {(k - k = 0) \times k} \right\} (91) \\ = \operatorname {S k e w} (k). (92) \\ \end{array}
$$
$$
\bar {g} _ {\boldsymbol {x}} (\boldsymbol {\xi}, \boldsymbol {\eta}) := g _ {\boldsymbol {x}} (\boldsymbol {\xi}, \boldsymbol {\eta}), \quad \boldsymbol {x} \in \mathcal {N} \subset \mathcal {M}, \boldsymbol {\xi}, \boldsymbol {\eta} \in T _ {\boldsymbol {x}} \mathcal {N} \subset T _ {\boldsymbol {x}} \mathcal {M}.
$$
$\overline{g}$  is an induced metric and  $(\mathcal{N},\overline{g})$  is a Riemannian submanifold of  $(\mathcal{M},g)$ . As  $\mathrm{St}(k,D)$  is a submanifold of  $\mathbb{R}^{D\times k}$ , we can define the standard inner product  $\pmb {A}\cdot \pmb {C} = \operatorname {tr}\left(\pmb {A}^{\top}\pmb {C}\right)$  of  $\mathbb{R}^{D\times k}$  as the induced metric  $\overline{g}$ . Thus,  $\mathrm{St}(k,D)$  is a Riemannian submanifold of  $\mathbb{R}^{D\times k}$ .
Suppose a set is given a suitable equivalence relation. A natural projection is a map that sends each element of a set to the equivalence class to which it belongs.
<sup>10</sup>The inner product  $\mathbf{A} \cdot \mathbf{C} = \mathbf{A}^{\top} \mathbf{C}$  of a vector is typically referred to as the standard inner product. Further, matrices are similarly defined with a standard inner product, defined as  $\mathbf{A} \cdot \mathbf{C} = \operatorname{tr}\left(\mathbf{A}^{\top} \mathbf{C}\right)$ . A space  $\mathbb{R}^{D \times k}$  such that the  $D \times k$  matrix  $\mathbf{A}$  is an element is referred to as a matrix space. The standard basis of the matrix space can be constructed by a matrix wherein only one element in the matrix is 1 and the remaining are 0. The matrix space is a linear space because it satisfies the linearity that is similar to that in case of a linear vector space.
11 When  $\mathcal{N}$  is a submanifold of a Riemannian manifold  $(\mathcal{M},g)$ , we define the Riemannian metric  $\overline{g}$  of  $\mathcal{N}$  to be:
Thus, we obtain the following formula.
$$
T _ {\boldsymbol {Y}} ^ {\mathrm {v}} \operatorname {S t} (k, D) = \left\{\boldsymbol {Y} \boldsymbol {C} \mid \boldsymbol {C} \in \operatorname {S k e w} (k) \right\}. \tag {93}
$$
Next, we define the horizontal space  $T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$  as the orthogonal complement of  $T_{\mathbf{Y}}^{\mathrm{v}} \operatorname{St}(k, D)$  in  $T_{\mathbf{Y}} \operatorname{St}(k, D)$  endowed with the inner product (87).
$$
\begin{array}{l} T _ {\mathbf {Y}} ^ {\mathrm {h}} \operatorname {S t} (k, D) := \left(T _ {\mathbf {Y}} ^ {\mathrm {v}} \operatorname {S t} (k, D)\right) ^ {\perp} (94) \\ = \left\{\overline {{\boldsymbol {\xi}}} _ {Y} ^ {\mathrm {h}} \in T _ {Y} \operatorname {S t} (k, D) \mid \operatorname {t r} \left(\overline {{\boldsymbol {\xi}}} _ {Y} ^ {\mathrm {h} \top} \overline {{\boldsymbol {\eta}}} _ {Y} ^ {\mathrm {v}}\right) = 0, \overline {{\boldsymbol {\eta}}} _ {Y} ^ {\mathrm {v}} \in T _ {Y} ^ {\mathrm {v}} \operatorname {S t} (k, D) \right\}. (95) \\ \end{array}
$$
Based on the fact that  $T_{\mathbf{Y}}^{\mathrm{v}} \operatorname{St}(k, D)$  is a subspace of  $T_{\mathbf{Y}} \operatorname{St}(k, D)$  and  $T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$  is defined as its orthogonal complement, the direct sum decomposition is as follows.
$$
T _ {\mathbf {Y}} \operatorname {S t} (k, D) = T _ {\mathbf {Y}} ^ {\mathrm {v}} \operatorname {S t} (k, D) \oplus T _ {\mathbf {Y}} ^ {\mathrm {h}} \operatorname {S t} (k, D), \tag {96}
$$
where  $\oplus$  denotes direct sum. Moreover, the tangent space is a linear space (Absil et al. (2008)). From (86), element  $YC$  of  $T_{\mathbf{Y}}^{\mathrm{v}}\operatorname {St}(k,D)$  corresponds to the first term of (93); thus, (96) is formulated as follows:
$$
T _ {\boldsymbol {Y}} ^ {\mathrm {h}} \operatorname {S t} (k, D) = \left\{\boldsymbol {\xi} _ {\boldsymbol {Y}} ^ {\mathrm {h}} = \boldsymbol {Y} _ {\perp} \boldsymbol {B} \mid \boldsymbol {B} \in \mathbb {R} ^ {(D - k) \times k} \right\}. \tag {97}
$$
Note that the horizontal vector  $\overline{\pmb{\xi}}_Y^{\mathrm{h}}$  is not necessarily an orthogonal matrix.
Finally, define the element  $\overline{\pmb{\xi}}_{\pmb{Y}}^{\mathrm{h}}\in T_{\pmb{Y}}^{\mathrm{h}}\mathrm{St}(k,D)$  of the horizontal space at  $\pmb {Y}\in \mathrm{St}(k,D)$  for the tangent vector  $\pmb {\xi}_{[Y]}\in T_{[Y]}\operatorname {Gr}(k,D)$  at  $[\pmb {Y}]\in \operatorname {Gr}(k,D)$  as satisfying the following formula.
$$
d \pi (\boldsymbol {Y}) \left[ \overline {{\boldsymbol {\xi}}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} \right] = \boldsymbol {\xi} _ {[ \boldsymbol {Y} ]}, \tag {98}
$$
where  $d\pi(\mathbf{Y}): T_{\mathbf{Y}} \mathrm{St}(k, D) \to T_{[\mathbf{Y}]} \mathrm{Gr}(k, D)$  is the derivative  $\frac{d\pi(\mathbf{Y})}{d\mathbf{Y}}$  of  $\pi: \mathrm{St}(k, D) \to \mathrm{Gr}(k, D)$  at  $\mathbf{Y} \in \mathrm{St}(k, D)$ . The  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}} \in T_{\mathbf{Y}}^{\mathrm{h}} \mathrm{St}(k, D)$  is referred to as the horizontal lift at  $\mathbf{Y} \in \mathrm{St}(k, D)$  of  $[\mathbf{Y}] \in \mathrm{Gr}(k, D)$ .
We describe the tangent space of  $\mathrm{Gr}(k,D)$  with the concept of horizontal lift.
Definition 5. Let  $T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$  be a horizontal space on  $\operatorname{St}(k, D)$ . Then, we define the tangent space  $T_{[\mathbf{Y}]}\operatorname{Gr}(k, D)$  of the  $\operatorname{Gr}(k, D)$  as follows.
$$
T _ {[ \mathbf {Y} ]} \operatorname {G r} (k, D) = \left\{\boldsymbol {\xi} _ {[ \mathbf {Y} ]} \mid d \pi (\mathbf {Y}) \left[ \overline {{\boldsymbol {\xi}}} _ {\mathbf {Y}} ^ {\mathrm {h}} \right] = \boldsymbol {\xi} _ {[ \mathbf {Y} ]}, \overline {{\boldsymbol {\xi}}} _ {\mathbf {Y}} ^ {\mathrm {h}} \in T _ {\mathbf {Y}} ^ {\mathrm {h}} \operatorname {S t} (k, D) \right\}. \tag {99}
$$
From the above,  $\pmb{\xi}_{[\mathbf{Y}]} \in T_{[\mathbf{Y}]} \operatorname{Gr}(k, D)$  is obtained from the map  $d\pi(\mathbf{Y}) \left[ \overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}} \right]$  when  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}}$  is obtained. The  $\pmb{\xi}_{[\mathbf{Y}]}$  is defined by an equivalence class and cannot be treated numerically in matrix form; however, it is sufficient to obtain the  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}}$  for actual numerical calculations. For  $\pmb{\xi}_{[\mathbf{Y}]} \in T_{[\mathbf{Y}]} \operatorname{Gr}(k, D)$ , there exists a  $\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}} \in T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$  that uniquely satisfies (98). In other words, we can handle it in matrix form by using elements of the horizontal space of Stiefel manifolds through the concept of horizontal lifting. Figure 1 is a conceptual diagram of the tangent space representation of a Grassmann manifold by horizontal lift.
D.4.4 Riemannian Metric on a Grassmann Manifold
We define the Riemannian metric  $g$  of  $\operatorname{Gr}(k, D)$  through the concept of horizontal lift.
Definition 6. Let  $\overline{\pmb{\xi}}_Y^{\mathrm{h}}$  and  $\overline{\eta}_{\mathbf{Y}}^{\mathrm{h}}$  be the horizontal lifts that become  $d\pi (\mathbf{Y})\left[\overline{\pmb{\xi}}_{\mathbf{Y}}^{\mathrm{h}}\right] = \pmb{\xi}_{[\mathbf{Y}]}$  and  $d\pi (Y)\left[\overline{\eta}_Y^{\mathrm{h}}\right] = \eta_{[Y]}$ , respectively. Then, we define the Riemannian metric on  $\operatorname {Gr}(k,D)$  as follows:
$$
g _ {[ \mathbf {Y} ]} (\boldsymbol {\xi} _ {[ \mathbf {Y} ]}, \boldsymbol {\eta} _ {[ \mathbf {Y} ]}) := \bar {g} _ {\mathbf {Y}} \left(\overline {{\boldsymbol {\xi}}} _ {\mathbf {Y}} ^ {\mathrm {h}}, \bar {\boldsymbol {\eta}} _ {\mathbf {Y}} ^ {\mathrm {h}}\right) = \operatorname {t r} \left(\boldsymbol {B} ^ {\top} \boldsymbol {D}\right), \tag {100}
$$
where  $B$  and  $D$  are matrices that are  $\overline{\pmb{\xi}}_{\pmb{Y}}^{\mathrm{h}} = \pmb{Y}_{\perp}\pmb{B}$  and  $\overline{\eta}_{\pmb{Y}}^{\mathrm{h}} = \pmb{Y}_{\perp}\pmb{D}$ , respectively.
D.5 Invariant Measures
Let the column vectors of matrix  $\mathbf{Y} = \{\pmb{y}_1, \dots, \pmb{y}_k\} \in \mathbb{R}^{D \times k}$  be the orthonormal basis that span the subspace  $\operatorname{span}(\mathbf{Y}) \in \mathrm{Gr}(k, D)$  in  $\mathbb{R}^D$ , and the column vectors of  $\mathbf{Y}_{\perp} = \{\pmb{y}_{k+1}, \dots, \pmb{y}_D\} \in \mathbb{R}^{D \times D-k}$  be the orthogonal complementary space  $\operatorname{span}(\mathbf{Y}_{\perp})$  of  $\operatorname{span}(\mathbf{Y})$ , respectively. Then, the following differential form can be defined.
$$
\begin{array}{c c} D - k & k \end{array}
$$
$$
\begin{array}{l} \left(d \boldsymbol {Y}\right) = \bigwedge_ {j = 1} \bigwedge_ {i = 1} \boldsymbol {y} _ {k + j} ^ {\top} d \boldsymbol {y} _ {i} (101) \\ = \left(\boldsymbol {y} _ {k + 1} ^ {\top} d \boldsymbol {y} _ {1} \wedge \dots \wedge \boldsymbol {y} _ {k + 1} ^ {\top} d \boldsymbol {y} _ {k}\right) \wedge \dots \wedge \left(\boldsymbol {y} _ {D} ^ {\top} d \boldsymbol {y} _ {1} \wedge \dots \wedge \boldsymbol {y} _ {D} ^ {\top} d \boldsymbol {y} _ {k}\right), (102) \\ \end{array}
$$
where  $\wedge$  is the wedge product and the relation satisfies  $\omega_{i}\wedge \omega_{i} = \omega_{j}\wedge \omega_{j} = 0$  and  $\omega_{i}\wedge \omega_{j} = -\omega_{j}\wedge \omega_{i}$ . The above equation is in  $k(D - k)$ -order differential form, which is an invariant measure on  $\mathrm{Gr}(k,D)$  (Chikuse (2003)).
If we define the matrix  $\mathbf{X}_{\perp}$  to be  $\left[ \begin{array}{cc}\mathbf{X} & \mathbf{X}_{\perp} \end{array} \right]$  for any point  $\mathbf{X} = \{\pmb{x}_1,\dots ,\pmb{x}_k\} \in \mathrm{St}(k,D)$ , the differential form for an invariant measure on  $\mathrm{St}(k,D)$  is defined as follows.
$$
\left(d \boldsymbol {X}\right) = \bigwedge_ {j = 1} ^ {D - k} \bigwedge_ {i = 1} ^ {k} \boldsymbol {x} _ {k + j} ^ {\top} d \boldsymbol {x} _ {i} \bigwedge_ {i <   j} ^ {k} \boldsymbol {x} _ {j} ^ {\top} d \boldsymbol {x} _ {i} = \left(d \boldsymbol {Y}\right) (d \boldsymbol {Q}), \tag {103}
$$
where  $(dQ)$  is the invariant measure of  $\mathcal{O}(k)$ . The integral of (103), that is, the volume of  $\mathrm{St}(k,D)$ , can be evaluated as follows:
$$
V _ {\operatorname {S t} (k, D)} = \int_ {\operatorname {S t} (k, D)} (d \boldsymbol {X}). \tag {104}
$$
(104) can be computed as follows. First, the surface  $S_{D}$  of the  $D$ -dimensional unit sphere can be defined as follows:
$$
S _ {D} = \left. \frac {d}{d r} \right| _ {r = 1} V _ {D} = D V _ {D} = \frac {D \pi^ {\frac {D}{2}}}{\Gamma (\frac {D}{2} + 1)} = \frac {2 \pi^ {\frac {D}{2}}}{\Gamma (\frac {D}{2})}, \tag {105}
$$
where  $V_{D}$  is the volume of a  $D$ -dimensional sphere  $\frac{\pi^{\frac{D}{2}}}{\Gamma\left(\frac{D}{2} + 1\right)} r^{D}$  and  $\Gamma\left(\frac{D}{2}\right)$  is the gamma function. Then, the following equation is obtained.
$$
\int_ {\operatorname {S t} (k, D)} \left(d \boldsymbol {X}\right) = S _ {D} \int_ {\operatorname {S t} (k - 1, D - 1)} \left(d \boldsymbol {X} _ {1}\right), \tag {106}
$$
where  $(dX_{1})$  is the differential form of  $\mathrm{St}(k - 1,D - 1)$ . Thus, (104) can be transformed as follows.
$$
V _ {\mathrm {S t} (k, D)} = \int_ {\mathrm {S t} (k, D)} (d \boldsymbol {X}) = \prod_ {i = 1} ^ {k} S _ {D} = \frac {2 ^ {k} \pi^ {\frac {D k}{2}}}{\Gamma_ {k} \left(\frac {D}{2}\right)}, \tag {107}
$$
where  $\Gamma_k\left(\frac{D}{2}\right)$  is the multidimensional gamma function. The invariant measure  $(dX)$  is an unnormalized measure. A measure normalized to be a probability measure can be formulated as follows:
$$
\left[ d \boldsymbol {X} \right] = \frac {1}{V _ {\operatorname {S t} (k , D)}} \left(d \boldsymbol {X}\right). \tag {108}
$$
This is a uniform distribution on  $\mathrm{St}(k, D)$ . As  $\mathrm{St}(k, k) = \mathcal{O}(k)$ , the volume  $V_{\mathcal{O}(k)}$  of  $\mathcal{O}(k)$  can be represented using  $(dQ)$  as follows.
$$
V _ {\mathcal {O} (k)} = V _ {\mathrm {S t} (k, k)} = \frac {2 ^ {k} \pi^ {\frac {k ^ {2}}{2}}}{\Gamma_ {k} \left(\frac {k}{2}\right)}. \tag {109}
$$
Furthermore, a measure normalized to be a probability measure can be represented by the following:
$$
\left[ d \boldsymbol {Q} \right] = \frac {1}{V _ {\mathcal {O} (k)}} \left(d \boldsymbol {Q}\right). \tag {110}
$$
From the above, the probability density function  $U_{\mathcal{O}(k)}(\boldsymbol{Q})$  of the uniform distribution on  $\mathcal{O}(k)$  is as follows:
$$
U _ {\mathcal {O} (k)} (\boldsymbol {Q}) = \frac {1}{V _ {\mathcal {O} (k)}} \quad \text {s . t .} \quad \boldsymbol {Q} \in \mathcal {O} (k), \tag {111}
$$
$$
\int_ {\mathcal {O} (k)} U _ {\mathcal {O} (k)} (\boldsymbol {Q}) (d \boldsymbol {Q}) = \int_ {\mathcal {O} (k)} \frac {1}{V _ {\mathcal {O} (k)}} (d \boldsymbol {Q}) = \int_ {\mathcal {O} (k)} [ d \boldsymbol {Q} ] = 1. \tag {112}
$$
As  $\mathrm{Gr}(k, D)$  is defined as a quotient manifold  $\mathrm{St}(k, D) / \mathcal{O}(k)$  as in (2), the volume  $V_{\mathrm{Gr}(k, D)}$  of  $\mathrm{Gr}(k, D)$  can be defined as follows.
$$
V _ {\operatorname {G r} (k, D)} = \int_ {\operatorname {G r} (k, D)} (d \mathbf {Y}) = \frac {V _ {\operatorname {S t} (k , D)}}{V _ {\mathcal {O} (k)}} = \frac {V _ {\operatorname {S t} (k , D)}}{V _ {\operatorname {S t} (k , k)}} = \frac {\pi^ {\frac {k (D - k)}{2}} \Gamma_ {k} \left(\frac {k}{2}\right)}{\Gamma_ {k} \left(\frac {D}{2}\right)}. \tag {113}
$$
The measure normalized to be a probability measure is expressed as:
$$
\left[ d \boldsymbol {Y} \right] = \frac {1}{V _ {\operatorname {G r} (k , D)}} \left(d \boldsymbol {Y}\right). \tag {114}
$$
D.6 Retraction
In general, the points except the origin  $(\pmb{p}(0) = \pmb{x})$  of the tangent space  $T_{\pmb{x}}\mathcal{M}$  at  $\pmb{x}$  on the manifold  $\mathcal{M}$  are not elements on  $\mathcal{M}(\pmb{p}(t) \in T_{\pmb{x}}\mathcal{M}, t \neq 0)$ . Therefore, if the result of the operation on the tangent space is to be used at another point on the manifold  $\mathcal{M}$ , it is necessary to map  $\pmb{p}(t)$  to the manifold  $\mathcal{M}$ . The map from a tangent space to a manifold is referred to as an exponential map. However, because the exponential map is computationally expensive, retraction based on numerical linear algebra is often used as an alternative (Zhu & Sato (2021)). Retraction is a method for approximating an exponential map to first order while maintaining global convergence in optimization algorithms on Riemannian manifolds. The most commonly used retractions on  $\mathrm{Gr}(k, D)$  are methods based on QR decomposition or singular-value decomposition (SVD) (Absil et al. (2008); Zhu & Sato (2021)). In addition, a retraction based on the Cayley transform is introduced in Zhu & Sato (2021). This retraction is closely related to the Cayley transform on  $\mathrm{St}(k, D)$  (Wen & Yin (2013); Xiaojing (2017); Zhu & Duan (2019)) and the Projected polynomial retraction (Gawlik & Leok (2018a)).
D.6.1 Exponential Map and Retraction
Geodesics on  $\mathrm{Gr}(k,D)$  can be expressed as the equivalence class  $\left[\exp_{\pmb{Y}}^{\mathrm{Gr}}\left(t\overline{\pmb{\xi}}_{\pmb{Y}}\right)\right]$ , where
$$
\exp_ {\boldsymbol {Y}} ^ {\operatorname {G r}} \left(t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}\right) = \left[ \begin{array}{l l} \boldsymbol {Y} & \boldsymbol {Y} _ {\perp} \end{array} \right] \exp (t \mathfrak {B}) \boldsymbol {I} _ {D \times k}. \tag {115}
$$
Here,  $\exp$  on the right-hand side is the matrix exponential, and  $\mathfrak{B} = \left[ \begin{array}{cc}\mathbf{0}_k & -\pmb {B}^\top \\ \pmb {B} & \mathbf{0}_{D - k} \end{array} \right]\in \mathrm{skew}(D)$  where  $\pmb{B}$  satisfies  $\overline{\xi}_{\pmb{Y}} = \pmb{Y}_{\perp}\pmb{B}$ . We can use a following exponential map that is mathematically equivalent to (115) (Edelman et al. (1998)):
$$
\exp_ {\boldsymbol {Y}} ^ {\operatorname {G r}} \left(t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}\right) := \left\{\boldsymbol {Y} \boldsymbol {V} \cos (\boldsymbol {\Sigma} t) + \boldsymbol {U} \sin (\boldsymbol {\Sigma} t) \right\} \boldsymbol {V} ^ {\top}, \tag {116}
$$
where  $\pmb{U},\pmb{\Sigma},\pmb{V}^{\top} = \mathrm{SVD}\left(\overline{\pmb{\xi}}_{\pmb{Y}}\right)$
Further, we can use the Padé approximation to approximate geodesics on Grassmann manifolds as follows:
$$
\boldsymbol {Y} (t) = \left[ \begin{array}{l l} \boldsymbol {Y} & \boldsymbol {Y} _ {\perp} \end{array} \right] r _ {m} (t \mathfrak {B}) \boldsymbol {I} _ {D \times k} \approx \exp_ {\boldsymbol {Y}} ^ {\operatorname {G r}} \left(t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}\right), \tag {117}
$$
where  $r_m(\mathbf{X})$  is the  $m$ th-order diagonal Padé approximation to the matrix exponential  $\exp(\mathbf{X})$ . See the expression of  $r_m(\mathbf{X})$  in Moler & Loan (2003). The simplest member of this class is surely the first-order Padé approximation
$$
\begin{array}{l} \bar {R} _ {\boldsymbol {Y}} \left(t \bar {\xi} _ {\boldsymbol {Y}}\right) := \left[ \begin{array}{l l} \boldsymbol {Y} & \boldsymbol {Y} _ {\perp} \end{array} \right] r _ {1} (t \mathfrak {B}) \boldsymbol {I} _ {D \times k} (118) \\ = \left[ \begin{array}{l l} \boldsymbol {Y} & \boldsymbol {Y} _ {\perp} \end{array} \right] \left(I _ {n} - \frac {t}{2} \mathfrak {B}\right) ^ {- 1} \left(\boldsymbol {I} _ {D} + \frac {t}{2} \mathfrak {B}\right) \boldsymbol {I} _ {D \times k}, (119) \\ \end{array}
$$
which is also known as the Cayley transform. From the error expression  $\exp (\mathbf{Y}) = r_m(\mathbf{Y}) + O\left(\| \mathbf{Y}\|^{2m + 1}\right)$  of the Padé approximation, we have
$$
\bar {R} _ {\boldsymbol {Y}} \left(t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}\right) = \exp_ {\boldsymbol {Y}} ^ {\operatorname {G r}} \left(t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}}\right) + O \left(t ^ {2 m + 1} \| \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} \| ^ {2 m + 1}\right), \tag {120}
$$
which is also given by Theorem 3 in Gawlik & Leok (2018b).
D.6.2 Horizontal Retraction
From Definition 3 in Zhu & Sato (2021), (119) is a horizontal retraction, and
$$
R _ {[ Y ]} \left(t \boldsymbol {\xi} _ {[ Y ]}\right) := \left[ \bar {R} _ {Y} \left(t \bar {\boldsymbol {\xi}} _ {Y} ^ {\mathrm {h}}\right) \right] \tag {121}
$$
is a retraction on  $\mathrm{Gr}(k, D)$  as a quotient manifold defined by (2). This is because  $\overline{R}$  satisfies the invariance condition that  $\overline{R}_{\mathbf{Y}}\left(t\overline{\xi}_{\mathbf{Y}}^{\mathrm{h}}\right) \sim \overline{R}_{\mathbf{Y}'}\left(t\overline{\xi}_{\mathbf{Y}'}^{\mathrm{h}}\right)$  for all  $\mathbf{Y} \in \mathrm{St}(k, D), \mathbf{Y}' \in \mathrm{St}(k, D), \overline{\xi}_{\mathbf{Y}} \in T_{\mathbf{Y}}^{\mathrm{h}}\mathrm{St}(k, D)$  and  $\overline{\xi}_{\mathbf{Y}'} \in T_{\mathbf{Y}'}^{\mathrm{h}}\mathrm{St}(k, D)$  such that  $\mathbf{Y} \sim \mathbf{Y}'$  and  $\overline{\xi}_{\mathbf{Y}}$  and  $\overline{\xi}_{\mathbf{Y}'}$  are horizontal lifts of  $\xi_{[\mathbf{Y}]} \in T_{[\mathbf{Y}]}\mathrm{Gr}(k, D)$  at  $\mathbf{Y}$  and  $\mathbf{Y}'$ , respectively.
In low-rank cases, we can obtain an economical version of (119) as follows (Zhu & Sato (2021)).
$$
\bar {R} _ {\boldsymbol {Y}} \left(t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h}}\right) = \boldsymbol {Y} + t \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h}} - \left(\frac {t ^ {2}}{2} \boldsymbol {Y} + \frac {t ^ {3}}{4} \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h}}\right) \left(\boldsymbol {I} _ {k} + \frac {t ^ {2}}{4} \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h} \top} \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h}}\right) ^ {- 1} \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h} \top} \bar {\boldsymbol {\xi}} _ {\boldsymbol {Y}} ^ {\mathrm {h}}. \tag {122}
$$
The inverse retraction  $\left(\overline{R_{[Y]}^{-1}}\right)_{\mathbf{Y}}^{\mathrm{h}}: \operatorname{St}(k, D) \to T_{\mathbf{Y}}^{\mathrm{h}} \operatorname{St}(k, D)$  of  $\overline{R}_{\mathbf{Y}}\left(\overline{\boldsymbol{\xi}}_{\mathbf{Y}}^{\mathrm{h}}\right)$  is the following:
$$
\begin{array}{l} \left(\overline {{R _ {[ Y ]} ^ {- 1} ([ X ])}}\right) _ {Y} ^ {\mathrm {h}} = \overline {{R}} _ {Y} ^ {- 1} (X) (123) \\ = 2 \boldsymbol {Y} _ {\perp} \boldsymbol {Y} _ {\perp} ^ {\top} \boldsymbol {X} \left(\boldsymbol {I} _ {k} + \boldsymbol {Y} ^ {\top} \boldsymbol {X}\right) ^ {- 1} (124) \\ = 2 \left(\boldsymbol {X} - \boldsymbol {Y} \boldsymbol {Y} ^ {\top} \boldsymbol {X}\right) \left(\boldsymbol {I} _ {k} + \boldsymbol {Y} ^ {\top} \boldsymbol {X}\right) ^ {- 1}. (125) \\ \end{array}
$$