SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
D _ {\alpha} (\mu \| \mu^ {\prime}) = \sup _ {E} (\mu (E) - \alpha \mu^ {\prime} (E)) = \int_ {Z} \left[ \frac {d \mu}{d \mu^ {\prime}} (z) - \alpha \right] _ {+} d \mu^ {\prime} (z) = \sum_ {z \in Z} [ \mu (z) - \alpha \mu^ {\prime} (z) ] _ {+},
$$
where  $E$  ranges over all measurable subsets of  $Z$ ,  $[\cdot]_{+} = \max\{\cdot, 0\}$ . Informally speaking, the privacy profile represents the set of all of the privacy parameters under which a mechanism provides differential privacy. Furthermore, the privacy profile can be computed using Theorem 5 of Balle and Wang [2018]:
$$
\delta_ {\mathcal {M}} (\varepsilon) = \sup  _ {\mathcal {D} \simeq \mathcal {D} ^ {\prime}} \left(\Pr \left[ L _ {\mathcal {M}} ^ {\mathcal {D}, \mathcal {D} ^ {\prime}} > \varepsilon \right] - e ^ {\varepsilon} \Pr \left[ L _ {\mathcal {M}} ^ {\mathcal {D} ^ {\prime}, \mathcal {D}} <   - \varepsilon \right]\right)
$$
where  $L_{\mathcal{M}}^{\mathcal{D},\mathcal{D}'}$  is the privacy loss random variable of the mechanism  $\mathcal{M}$  on inputs  $\mathcal{D} \simeq \mathcal{D}'$  defined as  $L_{\mathcal{M}}^{\mathcal{D},\mathcal{D}'} = \log (d\mu / d\mu')(\mathbf{z})$ , where  $\mu = \mathcal{M}(\mathcal{D}), \mu' = \mathcal{M}(\mathcal{D}')$ , and  $\mathbf{z} \sim \mu$ . It follows that for our Riemannian Gaussian mechanism  $\mathcal{M}$ , the privacy profile  $\delta_{\mathcal{M}}$  can be rewritten as
$$
\delta_ {\mathcal {M}} = \sup _ {\mathcal {D} \simeq \mathcal {D} ^ {\prime}} \int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A} p _ {\eta_ {2}, \sigma} (y) d \nu (y)
$$
where  $A \coloneqq \{y \in \mathcal{M} : p_{\eta_1, \sigma}(y) / p_{\eta_2, \sigma}(y) \geq e^\varepsilon\}$  and  $\eta_1 \coloneqq f(\mathcal{D}), \eta_2 \coloneqq f(\mathcal{D}')$ . This is exactly the left-hand side of (2).
The following theorem establishes a connection between GDP and privacy profile:
Theorem A.1 (Theorem 3.3 of Liu et al. [2022]). Let  $\mu_0 \coloneqq \sqrt{\lim_{\varepsilon \to +\infty} \frac{\varepsilon^2}{-2\log\delta_A(\varepsilon)}}$ . A privacy mechanism  $\mathcal{A}$  with the privacy profile  $\delta_A(\varepsilon)$  is  $\mu$ -GDP if and only if  $\mu_0 < \infty$  and  $\mu$  is no smaller than  $\mu_0$ .
Theorem A.1 implies that a mechanism with finite  $\mu_0$  is  $\mu$ -GDP for some privacy budget  $\mu$ . Note that Theorem 3.1 only tells us that Riemannian Gaussian distribution can be used to achieve  $\mu$ -GDP for some  $\mu$ . Therefore, to prove it, we only need to show that  $\mu_0$  is finite. We will show  $\mu_0 < \infty$  by demonstrate that  $-\log \delta_A(\varepsilon) = O(\varepsilon^2)$ . To do so, we will need the Bishop-Gromov comparison theorem:
Theorem A.2 (Bishop-Gromov; Lemma 7.1.4 in Petersen [2006]). Let  $M$  be a complete  $n$ -dimensional Riemannian manifold whose Ricci curvature satisfies the lower bound
$$
\operatorname {R i c} \geq (n - 1) K
$$
for a constant  $K \in \mathbb{R}$ . Let  $M_K^n$  be the complete  $n$ -dimensional simply connected space of constant sectional curvature  $K$  (and hence of constant Ricci curvature  $(n - 1)K$ ). Denote by  $B(p,r)$  the ball of radius  $r$  around a point  $p$ , defined with respect to the Riemannian distance function. Then, for any  $p \in M$  and  $p_K \in M_K^n$ , the function
$$
\phi (r) = \frac {\operatorname {V o l} B (p , r)}{\operatorname {V o l} B \left(p _ {K} , r\right)}
$$
is non-increasing on  $(0,\infty)$ . As  $r$  goes to zero, the ratio approaches one, so together with the monotonicity this implies that
$$
\operatorname {V o l} B (p, r) \leq \operatorname {V o l} B \left(p _ {K}, r\right).
$$
Bishop-Gromov comparison theorem not only gives us the control of volume growth of certain manifolds but also gives a rough classification by sectional curvature. Besides, this is a global property in the sense that  $p$  and  $p_{K}$  can be arbitrary points on the manifolds.
2It is known that a mechanism  $\mathcal{M}$  is  $(\varepsilon, \delta)$ -DP if and only if  $D_{e^{\varepsilon}}(\mathcal{M}(\mathcal{D}) \| \mathcal{M}(\mathcal{D}')) \leq \delta$  for every  $\mathcal{D}$  and  $\mathcal{D}'$  such that  $\mathcal{D} \simeq \mathcal{D}'$ .
A.1.1 Proof of Theorem 3.1
Proof. By A.1, we only need to show that for any  $\eta \in \mathcal{M}$ , when  $\varepsilon \to \infty$ ,
$$
\int_ {A} p _ {\eta , \sigma} (y) d \nu (y) = e ^ {- O (\varepsilon^ {2})}
$$
where  $A$  is given by
$$
A = \{y \in \mathcal {M} | p _ {\eta , \sigma} (y) / p _ {\eta^ {\prime}, \sigma} (y) > e ^ {\varepsilon} \}
$$
Let's consider  $\mathcal{M} \backslash A = \{y \in \mathcal{M} : p_{\eta, \sigma}(y) / p_{\eta', \sigma}(y) \leq e^{\varepsilon}\}$ . We have
$$
\begin{array}{l} \log \left(\frac {p _ {\eta , \sigma} (y)}{p _ {\eta^ {\prime} , \sigma} (y)}\right) \\ = \frac {1}{2 \sigma^ {2}} \left(d (\eta , y) ^ {2} - d (\eta^ {\prime}, y) ^ {2}\right) + C \\ \leq \frac {\Delta}{2 \sigma^ {2}} (2 d (\eta , y) + \Delta) + C, \quad \text {b y} \\ \end{array}
$$
where  $C = \log (Z(\eta ,\sigma)) - \log (Z(\eta ',\sigma))$  . Thus we have,
$$
d (\eta , y) \leq \frac {2 \sigma^ {2} (\varepsilon - C) - \Delta^ {2}}{2 \Delta} \Rightarrow \frac {p _ {\eta , \sigma} (y)}{p _ {\eta^ {\prime} , \sigma} (y)} \leq e ^ {\varepsilon}
$$
Let  $r = \frac{2\sigma^2(\varepsilon - C) - \Delta^2}{2\Delta}$ , note that since  $B_{\eta}(r) \subseteq \mathcal{M} \backslash A$ , we have  $A \subseteq \mathcal{M} \backslash B_{\eta}(r)$ . Thus, we only need to prove the following:
$$
\int_ {\mathcal {M} \backslash B _ {\eta} (r)} p _ {\eta , \sigma^ {2}} (y) d \nu (y) = e ^ {- O (\varepsilon^ {2})}
$$
when  $\varepsilon \to \infty$ . One can easily show the following inequality,
$$
\int_ {B _ {\eta} (2 r) \backslash B _ {\eta} (r)} p _ {\eta , \sigma} (y) d \nu (y) \leq Z (\eta , \sigma) e ^ {- \frac {r ^ {2}}{\sigma^ {2}}} (\operatorname {V o l} B (\eta , 2 r) - \operatorname {V o l} B (\eta , r)) \tag {4}
$$
By Theorem A.2, we have the following three cases:
1.  $K > 0$ . Then the standard space  $M_K^n$  is the  $n$ -sphere of radius  $1 / \sqrt{K}$ . (Vol  $B(\eta, 2r) - \operatorname{Vol} B(\eta, r)$ ) is obviously less than the volume of the whole space  $M_K^n$ . Thus we have
$$
\int_ {B _ {\eta} (2 r) \backslash B _ {\eta} (r)} p _ {\eta , \sigma} (y) d \nu (y) \leq Z (\eta , \sigma) e ^ {- \frac {r ^ {2}}{\sigma^ {2}}} s (n) \sqrt {K} ^ {1 - n} \tag {5}
$$
where  $s(n) = \frac{2\pi^{\frac{n}{2}}}{\Gamma(\frac{n}{2})}$  is a constant relative to the dimension  $n$ . One can easily find that as  $\varepsilon \to \infty$ ,  $B_{\eta}(2r)$  will cover the  $M_K^n$ , and the right-hand side of inequality 5 will approach to 0 as  $e^{-O(\varepsilon^2)}$ .
2.  $K = 0$ . Then the standard space  $M_K^n$  is the  $n$  dimensional Euclidean space  $\mathbb{R}^n$ . We have
$$
\int_ {B _ {\eta} (2 r) \backslash B _ {\eta} (r)} p _ {\eta , \sigma} (y) d \nu (y) \leq Z (\eta , \sigma) e ^ {- \frac {r ^ {2}}{\sigma^ {2}}} \frac {\pi^ {n / 2} r ^ {n}}{\Gamma \left(\frac {n}{2} + 1\right)}. \tag {6}
$$
The same with above, when  $\varepsilon \to \infty$ ,  $B_{\eta}(2r)$  will cover the  $\mathbb{R}^n$  and the right-hand side of 6 will approach to 0 as  $e^{-O(\varepsilon^2)}$ .
3.  $K < 0$ . The standard space  $M_{n}(K)$  is the hyperbolic  $n$ -space  $\mathbb{H}^n$ . The hyperbolic volume  $\operatorname{Vol} B(\eta_K, r)$  with any  $\eta_k \in \mathbb{H}^n$  is given by
$$
\operatorname {V o l} B \left(\eta_ {K}, r\right) = s (n) \int_ {0} ^ {r} \left(\frac {\sinh (\sqrt {- K} t)}{\sqrt {- K}}\right) ^ {n - 1} d t \tag {7}
$$
where the hyperbolic function is given by  $\sinh (x) = (e^x - e^{-x}) / 2$ . It's not hard to see that
$$
\sinh^ {n} (t) \leq \frac {(n + 1) e ^ {n t}}{2 ^ {n}}. \tag {8}
$$
Plugging the 8 into 7, we have
$$
\operatorname {V o l} B \left(\eta_ {K}, r\right) \leq s _ {n} \frac {n}{(n - 1) 2 ^ {n - 1} \sqrt {- K} ^ {n}} e ^ {\sqrt {- K} (n - 1) r}. \tag {9}
$$
Combining 9 and 4, we have
$$
\int_ {B _ {\eta} (2 r) \backslash B _ {\eta} (r)} p _ {\eta , \sigma} (y) d \nu (y) \leq c (n) Z (\eta , \sigma) e ^ {- \frac {r ^ {2}}{\sigma^ {2}} + \sqrt {- K} (n - 1) 2 r}. \tag {10}
$$
The principle part of the exponent of the right-hand side of 10 is still  $-\varepsilon^2$ . Thus, when  $\varepsilon \to \infty$ , it approaches to 0 as  $e^{-O(\varepsilon^2)}$ .
A.2 Theorem 3.2
Proof. Follows directly from Definition 2.2, Theorem 3.1 and Theorem 5 in Balle and Wang [2018].  $\square$
A.3 Corollary 3.2.1
Proof. In this proof, we will parameterize points on  $S^1$  using their polar angles.
On  $S^1$ , the Riemannian Gaussian distribution with footprint  $\eta$  and rate  $\sigma$  has the following density,
$$
p _ {\eta , \sigma} (\theta) = \frac {1}{Z _ {\sigma}} e ^ {- \frac {1}{2 \sigma^ {2}} (\theta - \eta \mod \pi) ^ {2}}, Z _ {\sigma} = \sqrt {2 \pi} \sigma \left[ \Phi \left(\frac {\pi}{\sigma}\right) - \Phi \left(- \frac {\pi}{\sigma}\right) \right].
$$
Note that since  $S^1$  has constant curvature, we can use Theorem 4.1 instead of Theorem 3.2
WLOG we assume  $\eta_1 = 2\pi - \frac{\Delta}{2}$  and  $\eta_2 = \frac{\Delta}{2}$  and thus  $d(\eta_1, \eta_2) = \Delta$ . Given an arbitrary  $\varepsilon$ , the set  $A$  takes the following form,
$$
A = \left[ \pi + \frac {\sigma^ {2} \varepsilon}{\Delta}, 2 \pi - \frac {\sigma^ {2} \varepsilon}{\Delta} \right].
$$
and it follows that we must have
$$
\varepsilon \in [ 0, \pi \Delta / (2 \sigma^ {2}) ]. \tag {11}
$$
$$
\begin{array}{l} \int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A} p _ {\eta_ {2}, \sigma} (y) d \nu (y) \\ = \frac {1}{Z _ {\sigma}} \left[ \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {1} - \theta \mod \pi) ^ {2}} d \theta - e ^ {\varepsilon} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {2} - \theta \mod \pi) ^ {2}} d \theta \right] \\ = \frac {1}{Z _ {\sigma}} \left[ \int_ {\pi + \frac {\sigma^ {2} \varepsilon}{\Delta}} ^ {2 \pi - \frac {\sigma^ {2} \varepsilon}{\Delta}} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {1} - \theta \mod \pi) ^ {2}} d \theta - e ^ {\varepsilon} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {2} - \theta \mod \pi) ^ {2}} d \theta \right] \\ = \frac {1}{Z _ {\sigma}} \left[ \int_ {\pi + \frac {\sigma^ {2} \varepsilon}{\Delta}} ^ {2 \pi - \frac {\sigma^ {2} \varepsilon}{\Delta}} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {1} - \theta) ^ {2}} d \theta - e ^ {\varepsilon} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {2} - \theta \mod \pi) ^ {2}} d \theta \right] \\ = \frac {1}{Z _ {\sigma}} \left[ \Phi \left(\frac {2 \pi}{\sigma} - \frac {\sigma \varepsilon}{\Delta} - \frac {\eta_ {1}}{\sigma}\right) - \Phi \left(\frac {\pi}{\sigma} + \frac {\sigma \varepsilon}{\Delta} - \frac {\eta_ {1}}{\sigma}\right) - e ^ {\varepsilon} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {2} - \theta \mod \pi) ^ {2}} d \theta \right] \\ = \frac {1}{Z _ {\sigma}} \left[ \Phi \left(- \frac {\sigma \varepsilon}{\Delta} + \frac {\Delta}{2 \sigma}\right) - \Phi \left(\frac {\sigma \varepsilon}{\Delta} + \frac {\Delta}{2 \sigma} - \frac {\pi}{\sigma}\right) - e ^ {\varepsilon} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\eta_ {2} - \theta \mod \pi) ^ {2}} d \theta \right]. \\ \end{array}
$$
We have evaluated the first integral, and now let's consider the second integral. For  $\varepsilon \leq \frac{\Delta^2}{2\sigma^2}$ , we have
$$
\begin{array}{l} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\mu_ {2} - \theta \mod \pi) ^ {2}} d \theta \\ = \int_ {\pi + \frac {\sigma^ {2} \varepsilon}{\Delta}} ^ {\mu_ {2} + \pi} e ^ {- \frac {1}{2 \sigma^ {2}} (\theta - \mu_ {2}) ^ {2}} d \theta + \int_ {\mu_ {2} + \pi} ^ {2 \pi - \frac {\sigma^ {2} \varepsilon}{\Delta}} e ^ {- \frac {1}{2 \sigma^ {2}} (\theta - (2 \pi + \mu_ {2})) ^ {2}} d \theta \\ = \sqrt {2 \pi} \sigma \left[ \Phi \left(\frac {\pi}{\sigma}\right) - \Phi \left(\frac {\pi}{\sigma} + \frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma}\right) + \Phi \left(- \frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma}\right) - \Phi \left(- \frac {\pi}{\sigma}\right) \right]. \\ \end{array}
$$
Thus for  $\varepsilon \leq \frac{\Delta^2}{2\sigma^2}$  we have,
$$
\begin{array}{l} \int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A} p _ {\eta_ {2}, \sigma} (y) d \nu (y) \\ = \frac {\sqrt {2 \pi} \sigma}{Z _ {\sigma}} \left[ \Phi \left(- \frac {\sigma \varepsilon}{\Delta} + \frac {\Delta}{2 \sigma}\right) - e ^ {\varepsilon} \Phi \left(- \frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma}\right) \right] \tag {12} \\ - \frac {\sqrt {2 \pi} \sigma}{Z _ {\sigma}} \left[ \Phi \left(\frac {\sigma \varepsilon}{\Delta} + \frac {\Delta}{2 \sigma} - \frac {\pi}{\sigma}\right) - e ^ {\varepsilon} \Phi \left(\frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma} + \frac {\pi}{\sigma}\right) \right] \\ - e ^ {\varepsilon}. \\ \end{array}
$$
Similarly, for  $\varepsilon >\frac{\Delta^2}{2\sigma^2}$  , we have,
$$
\begin{array}{l} \int_ {A} e ^ {- \frac {1}{2 \sigma^ {2}} (\mu_ {2} - \theta \mod \pi) ^ {2}} d \theta \\ = \int_ {\pi + \frac {\sigma^ {2} \varepsilon}{\Delta}} ^ {2 \pi - \frac {\sigma^ {2} \varepsilon}{\Delta}} e ^ {- \frac {1}{2 \sigma^ {2}} (\theta - (2 \pi + \mu_ {2})) ^ {2}} d \theta \\ = \sqrt {2 \pi} \sigma \left[ \Phi \left(- \frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma}\right) - \Phi \left(- \frac {\pi}{\sigma} + \frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma}\right) \right]. \\ \end{array}
$$
Thus for  $\varepsilon >\frac{\Delta^2}{2\sigma^2}$  we have,
$$
\begin{array}{l} \int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A} p _ {\eta_ {2}, \sigma} (y) d \nu (y) \\ = \frac {\sqrt {2 \pi} \sigma}{Z _ {\sigma}} \left[ \Phi \left(- \frac {\sigma \varepsilon}{\Delta} + \frac {\Delta}{2 \sigma}\right) - e ^ {\varepsilon} \Phi \left(- \frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma}\right) \right] \tag {13} \\ - \frac {\sqrt {2 \pi} \sigma}{Z _ {\sigma}} \left[ \Phi \left(\frac {\sigma \varepsilon}{\Delta} + \frac {\Delta}{2 \sigma} - \frac {\pi}{\sigma}\right) - e ^ {\varepsilon} \Phi \left(\frac {\sigma \varepsilon}{\Delta} - \frac {\Delta}{2 \sigma} - \frac {\pi}{\sigma}\right) \right]. \\ \end{array}
$$
Put (11), (12) and (13) together with Theorem 4.1, we have proved Corollary 3.2.1.
A.4 Homogeneous Riemannian Manifolds
For more detailed treatment on homogenous Riemannian manifolds and related concepts, refers to Helgason [1962], Berestovskii and Nikonorov [2020], Lee [2006] for details and Chakraborty and Vemuri [2019] for a more concise summary.
A.4.1 Group actions on Manifolds
In this section, we will introduce some basic facts about group action which will be used to introduce homogeneous Riemannian manifolds in later section. The materials covered in this section can be found in any standard Abstract Algebra texts.
Definition A.1. A group  $(G,\cdot)$  is a non-empty set  $G$  together with a binary operation  $\cdot : G\times G\to G$ ,  $(a,b)\mapsto a\cdot b$  such that the following three axioms are satisfied:
- Associativity:  $\forall a, b, c \in G, (a \cdot b) \cdot c = a(b \cdot c)$
- Identity element:  $\exists e \in G, \forall a \in G, a \cdot e = e \cdot a = a$ .
- Inverse element:  $\forall a \in G, \exists a^{-1} \in G, a \cdot a^{-1} = a^{-1} \cdot a = e$ .
Definition A.2. Let  $G$  be a group and  $X$  be an arbitrary set. A left group action is a map  $\alpha: G \times X \to X$ , that satisfies the following axioms:
$\alpha (e,x) = x$
-  $\alpha(g, \alpha(h, x)) = \alpha(gh, x)$
Note here we use the juxtaposition  $gh$  to denote the binary operation in the group. If we shorten  $\alpha(g, x)$  by  $g \cdot x$ , it's equivalent to say that  $e \cdot x = x$ , and  $g \cdot (h \cdot x) = (gh) \cdot x$
Note each  $g\in G$  induces a map  $L_{g}:X\to X,x\mapsto g\cdot x$
A.4.2 Homogeneous Riemannian manifolds, symmetric spaces and spaces of constant curvature
Let  $\mathcal{M}$  be a Riemannian manifold and  $I(\mathcal{M})$  be the set of all isometries of  $\mathcal{M}$ , that is, given  $g\in I(\mathcal{M}),d(g\cdot x,g\cdot y) = d(x,y)$ , for all  $x,y\in \mathcal{M}$ . It is clear that  $I(\mathcal{M})$  forms a group, and thus, for a given  $g\in I(\mathcal{M})$  and  $x\in \mathcal{M},g\cdot x\mapsto y$ , for some  $y\in \mathcal{M}$  is a group action. We call  $I(\mathcal{M})$  the isometry group of  $\mathcal{M}$ .
Consider  $o \in \mathcal{M}$ , and let  $H = \operatorname{Stab}(o) = \{h \in G \mid h \cdot o = o\}$ , that is,  $H$  is the Stabilizer of  $o \in \mathcal{M}$ . Given  $g \in I(M)$ , its linear representation  $g \mapsto d_x g$  in the tangent space  $T_x\mathcal{M}$  is called the isotropy representation and the linear group  $d_x \operatorname{Stab}(x)$  is called the isotropy group at the point  $x$ .
We say that  $G$  acts transitively on  $\mathcal{M}$ , iff, given  $x, y \in \mathcal{M}$ , there exists a  $g \in \mathcal{M}$  such that  $y = g \cdot x$ .
Definition A.3 ([Helgason, 1962]). Let  $G = I(\mathcal{M})$  act transitively on  $\mathcal{M}$  and  $H = \mathrm{Stab}(o)$ ,  $o \in \mathcal{M}$  (called the "origin" of  $\mathcal{M}$ ) be a subgroup of  $G$ . Then  $\mathcal{M}$  is called a homogeneous Riemannian manifold and can be identified with the quotient space  $G / H$  under the diffeomorphic mapping  $gH \mapsto g \cdot o, g \in G$ .
By definition, we have  $d(x,y) = d(g\cdot x,g\cdot y)$  for any  $g\in G$  and any  $x,y\in \mathcal{M}$ . More importantly, any integrable function  $f:\mathcal{M}\to \mathbb{R}$ , we have [Helgason, 1962]
$$
\int_ {\mathcal {M}} f (x) d \nu (x) = \int_ {\mathcal {M}} f (g \cdot x) d \nu (x)
$$
This property leads to Proposition 4.1.
Definition A.4 ([Helgason, 1962]). A Riemannian symmetric space is a Riemannian manifold  $\mathcal{M}$  such that for any  $x\in \mathcal{M}$ , there exists  $s_x\in G = I(\mathcal{M})$  such that  $s_x\cdot x = x$  and  $ds_{x}|_{x} = -I$ .  $S_{x}$  is called symmetry at  $x$ .
That is, a Riemannian symmetric space is a Riemannian manifold  $\mathcal{M}$  with the property that the geodesic reflection at any point is an isometry of  $\mathcal{M}$ . Note that any Riemannian symmetric space is a homogeneous Riemannian manifold, but the converse is not true.
Definition A.5 ([Vinberg et al., 1993]). A simply-connected homogeneous Riemannian manifold is said to be a space of constant curvature if its isotropy group (at each point) is the group of all orthogonal transformations with respect to some Euclidean metric.
Once again, a space of constant curvature is a symmetric space but the converse is not true.
A.5 Theorem 4.1
Proof. Let  $G$  be the isometry group of  $\mathcal{M}$ . Let  $\eta_1, \eta_2 \in \mathcal{M}$  be arbitrary points such that  $d(\eta_1, \eta_2) = \Delta$ . By Corollary 4.1, the set  $A$  reduces to  $A = \{y \in \mathcal{M} : d(\eta_2, y)^2 - d(\eta_1, y)^2 \geq 2\sigma^2\varepsilon\}$ .
What we need to show is the following, for any points  $\eta_1^{\prime},\eta_2^{\prime}\in \mathcal{M}$  such that  $d(\eta_1^\prime ,\eta_2^\prime) = \Delta$
$$
\int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A} p _ {\eta_ {2}, \sigma} (y) d \nu (y) = \int_ {A ^ {\prime}} p _ {\eta_ {1} ^ {\prime}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A ^ {\prime}} p _ {\eta_ {2} ^ {\prime}, \sigma} (y) d \nu (y)
$$
where  $A' = \{y \in \mathcal{M} : d(\eta_2', y)^2 - d(\eta_1', y)^2 \geq 2\sigma^2\varepsilon\}$ . It's sufficient to show
$$
\int_ {A} p _ {\eta_ {1}, \sigma} (y) = \int_ {A ^ {\prime}} p _ {\eta_ {1} ^ {\prime}, \sigma} (y) d \nu (y), \quad \int_ {A} p _ {\eta_ {2}, \sigma} (y) = \int_ {A ^ {\prime}} p _ {\eta_ {2} ^ {\prime}, \sigma} (y) d \nu (y). \tag {14}
$$
We can separate the proof into three cases: (1)  $\eta_1' = \eta_1, \eta_2' \neq \eta_2$ ; (2)  $\eta_1' \neq \eta_1, \eta_2' = \eta_2$ ; (3)  $\eta_1' \neq \eta_1, \eta_2' \neq \eta_2$ .
Case (1):  $\eta_1' = \eta_1, \eta_2' \neq \eta_2$ :
It follows that  $\eta_{2}$  is in the sphere centered at  $\eta$  with radius  $\Delta$ . (14) then follows from the rotational symmetry of the constant curvature spaces.
Case (2):  $\eta_1' \neq \eta_1$ ,  $\eta_2' = \eta_2$ :
Same as case (1).
Case (3):  $\eta_1' \neq \eta_1$ ,  $\eta_2' \neq \eta_2$ :
For any  $\eta_1' \neq \eta_1$ , there exists  $g \in G$ , such that  $g \cdot \eta_1 = \eta_1'$ . Denote  $\eta_2' = g \cdot \eta_2$ , we have
$$
\begin{array}{l} g A := \left\{g \cdot y: d \left(\eta_ {2}, y\right) ^ {2} - d \left(\eta_ {1}, y\right) ^ {2} \geq 2 \sigma^ {2} \varepsilon \right\} \\ = \left\{g \cdot y: d \left(\eta_ {2} ^ {\prime}, g \cdot y\right) ^ {2} - d \left(\eta_ {1} ^ {\prime}, g \cdot y\right) ^ {2} \geq 2 \sigma^ {2} \varepsilon \right\} \\ = \left\{y: d \left(\eta_ {2} ^ {\prime}, y\right) ^ {2} - d \left(\eta_ {1} ^ {\prime}, y\right) ^ {2} \geq 2 \sigma^ {2} \varepsilon \right\} \\ = A ^ {\prime}. \\ \end{array}
$$
Let  $F(y)\coloneqq p_{\eta_1,\sigma}(y)\mathbf{1}_A(y)$  , we have
$$
\begin{array}{l} \int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) \\ = \int_ {\mathcal {M}} F \circ L _ {g} ^ {- 1} (y) d \nu (y) \\ = \int_ {\mathcal {M}} p _ {\eta_ {1}, \sigma} (g ^ {- 1} \cdot y) \mathbf {1} _ {g A} (y) d \left(L _ {g} ^ {- 1}\right) ^ {*} \nu (y); \quad \text {c h a n g e o f v a r i a b l e f o r m u l a r}, \\ = \int_ {\mathcal {M}} p _ {\eta_ {1}, \sigma} (g ^ {- 1} \cdot y) \mathbf {1} _ {g A} (y) d \nu (y); \quad \nu \text {i s a G - i n v a r i a n t m e a s u r e}, \\ = \frac {1}{Z _ {\sigma}} \int_ {g A} e ^ {- \frac {1}{2 \sigma^ {2}} d (g ^ {- 1} \cdot y, \eta_ {1}) ^ {2}} d \nu (y) \\ = \frac {1}{Z _ {\sigma}} \int_ {g A} e ^ {- \frac {1}{2 \sigma^ {2}} d ((g g ^ {- 1}) \cdot y, g \cdot \eta_ {1}) ^ {2}} d \nu (y) \\ = \frac {1}{Z _ {\sigma}} \int_ {g A} e ^ {- \frac {1}{2 \sigma^ {2}} d (y, \eta_ {1} ^ {\prime}) ^ {2}} d \nu (y) \\ = \int_ {g A} p _ {\eta_ {1} ^ {\prime}, \sigma (y)} d \nu (y) \\ = \int_ {A ^ {\prime}} p _ {\eta_ {1} ^ {\prime}, \sigma (y)} d \nu (y). \\ \end{array}
$$
For  $\int_{A}p_{\eta_2,\sigma}(y)d\nu (y)$ , the proof is the same. Combine with the result of case (1), we have finished the proof for case (3).
A.6 Simulation Details
For sampling from Riemannian Gaussian distribution  $N_{\mathcal{M}}(\theta, \sigma^2)$  on  $S^1$  (section 4.2), we first sample from truncated normal distribution with  $\mu = 0$  and  $\sigma^2$ , then embed the sample to  $\mathbb{R}$  and lastly counter-wise rotate the sample with degree  $\theta$ .
For simulation on  $S^2$  (section 5.2), we choose the pair of  $\eta$  and  $\eta'$  to be  $(1,0,0)$  and  $(\cos(\Delta), (1 - \cos(\Delta)^2)^{1/2}, 0)$ . Though any pair  $\eta, \eta' \in S^2$  with  $d(\eta, \eta') = \Delta$  works, we simply choose this specific pair for convenience. For Fréchet mean computation, we use a gradient descent algorithm described in Reimherr and Awan [2019].
A.6.1 R Codes
For simulations in section 4.2, refer to R files euclid-functions.R & euclid_simulation.R for Euclidean space and sphere-functions.R & s1_simulation.R for unit circle  $S^1$ . For simulations in section 5.2, refer to R files sphere-functions.R & sphere_simulation.R.