SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Definition 4.1 (Definition 4.6.1 in Berestovskii and Nikonorov [2020]). A Riemannian manifold  $(\mathcal{M},g)$  is called a homogeneous Riemannian manifold if a Lie group  $G$  acts transitively and isometrically on  $\mathcal{M}$ .
Homogeneous Riemannian manifolds encompass a broad class of manifolds that are commonly encountered in statistics such as the (hyper)sphere [Bhattacharya and Bhattacharya, 2012, Mardia et al., 2000], the space of SPD Matrices[Pennec et al., 2019, Said et al., 2017, Hajri et al., 2016], the Stiefel manifold [Chakraborty and Vemuri, 2019, Turaga et al., 2008] and the Grassmann manifold [Turaga et al., 2008]. It's a more general class than Riemannian symmetric space, which is a common setting used in geometric statistics [Cornea et al., 2017, Asta, 2014, Said et al., 2018, Said, 2021, Chevallier et al., 2022]. Please refer to Appendix A.4 for more detail.
Informally, a homogeneous Riemannian manifold looks geometrically the same at every point. The transitive property required in the definition implies that any homogeneous Riemannian manifold  $\mathcal{M}$  only has one orbit. Therefore, we have the following proposition.
Proposition 4.1. If  $\mathcal{M}$  is a homogeneous Riemannian manifolds, then  $Z(\eta_1,y) = Z(\eta_2,y)$  for any  $\eta_{1},\eta_{2}\in \mathcal{M}$ .
Therefore, on homogeneous Riemannian manifolds, we can simplify the set  $A$  in Theorem 3.2 to  $A = \{y\in \mathcal{M}:d(\eta_2,y)^2 -d(\eta_1,y)^2\geq 2\sigma^2\varepsilon \}$ . To further simplify (2), we will need a much stronger assumption than homogeneous Riemannian manifolds. In particular, we require the condition of constant curvature.
Theorem 4.1 (Riemannian Gaussian Mechanism on Manifolds with Constant Curvature). Let  $\mathcal{M}$  be a Riemannian manifold with constant curvature and  $f$  be a  $\mathcal{M}$ -valued summary with global sensitivity  $\Delta$ . The Riemannian Gaussian distribution with footpoint  $f(\mathcal{D})$  and rate  $\sigma > 0$  is  $\mu$ -GDP if and only if  $\mu$  satisfies the following condition:  $\exists \eta_1, \eta_2$  such that  $d(\eta_1, \eta_2) = \Delta$  and  $\forall \varepsilon \geq 0$
$$
\int_ {A} p _ {\eta_ {1}, \sigma} (y) d \nu (y) - e ^ {\varepsilon} \int_ {A} p _ {\eta_ {2}, \sigma} (y) d \nu (y) \leq \delta_ {\mu} (\varepsilon) \tag {3}
$$
where  $A = \{y\in \mathcal{M}:d(\eta_2,y)^2 -d(\eta_1,y)^2\geq 2\sigma^2\varepsilon \}$
Theorem 4.1 tells us that instead of evaluating the integrals of 2 on every neighboring pairs  $\eta_{1}$  and  $\eta_{2}$ , we only need to check one such pair. Despite the convenience it provides us, evaluating the integrals remains a challenge, even for an elementary space like  $S^d$  with  $d > 1$ . To circumvent this challenge, we employ the MCMC technique for the integral computations. The main idea is simple: Let  $Y_{1}, Y_{2}, \ldots, Y_{n}$  be independent and identically distributed random variables from  $N_{\mathcal{M}}(\eta, \sigma^{2})$ , and denotes  $Z_{i} = \mathbf{1}\left\{\frac{1}{2\sigma^{2}}\left(-d(\eta_{1}, Y_{i})^{2} + d(\eta_{2}, Y_{i})^{2}\right) \geq \varepsilon \right\}$ . By the strong law of large number [Dudley, 2002], we then have
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} Z _ {i} \rightarrow \int_ {A} p _ {\eta , \sigma} (y) d \nu (y) \quad \text {a . s .}.
$$
By using  $\frac{1}{n}\sum_{i = 1}^{n}Z_{i}$  as an approximation, we avoid the challenge of evaluating the integrals analytically. The detailed algorithm is documented in Algorithm 2. It's known that any space of constant curvature is isomorphic to one of the spaces: Euclidean space, sphere, and hyperbolic space [Vinberg et al., 1993, Woods, 1901]. Therefore, Algorithm 2 offers a straightforward and practical method for assessing the privacy budget  $\mu$  on spheres and hyperbolic spaces. Furthermore, Algorithm 2 can be extended to a more general class of Riemannian manifold by sampling more than one pair of  $\eta$ ,  $\eta^{\prime}$  in step 1. The determination of the number of pairs being sampled and the selection method to ensure sufficient dissimilarity among the sampled pairs is an aspect that requires further investigation.
Algorithm 2 Computing  $\mu$  on Manifolds with Constant Curvature
Input: Sensitivity  $\Delta$ , rate  $\sigma^2$ , Monte Carlo sample size  $n$ , number of  $\varepsilon$  used  $n_{\varepsilon}$ , maximum  $\varepsilon$  used  $\varepsilon_{\mathrm{max}}$ , number of MCMC samples  $m$   
Output: privacy budget  $\mu$ ;  
1: Sample a random point  $\eta$  on  $\mathcal{M}$  and another point  $\eta'$  on the sphere center at  $\eta$  with radius  $\Delta$ .  
2: for each  $j$  in  $1, 2, \ldots, m$  do  
3: Sample  $y_{1j}, \ldots, y_{nj} \sim N_{\mathcal{M}}(\eta, \sigma^2), y_{1j}', \ldots, y_{nj}' \sim N_{\mathcal{M}}(\eta', \sigma^2)$ .  
4: Compute  $d_{ij} = d(\eta', y_{ij})^2 - d(\eta, y_{ij})^2$  and  $d_{ij}' = d(\eta', y_{ij}')^2 - d(\eta, y_{ij}')^2$  for  $i$  in  $1, 2, \ldots, n$ .  
5: for each  $\varepsilon$  in  $\{k, 2k, \ldots, n_{\varepsilon}k\}$  where  $k = \varepsilon_{\mathrm{max}} / n_{\varepsilon}$  do  
6: Compute  $l_{\varepsilon}^{(j)} = \sum_{i=1}^{n} \mathbf{1}(d_{ij} \geq 2\sigma^2\varepsilon) / n - e^{\varepsilon} \sum_{i=1}^{n} \mathbf{1}(d_{ij}' \geq 2\sigma^2\varepsilon) / n$ .  
7: end for  
8: end for  
9: Compute  $l_{\varepsilon} = \sum_{j=1}^{m} l_{\varepsilon}^{(j)} / m$  and  $\mu_{\varepsilon}$  via  $l_{\varepsilon} = \delta_{\mu}(\varepsilon)$  for each  $\varepsilon$ . Compute  $\mu = \max_{\varepsilon} \mu_{\varepsilon}$ .  
10: Return:  $\mu$ .
4.1 Sampling from Riemannian Gaussian Distribution
The one crucial step in Algorithm 2 involves sampling from the two Riemannian Gaussian distributions  $N_{\mathcal{M}}(\eta, \sigma^2)$  and  $N_{\mathcal{M}}(\eta', \sigma^2)$ . Since their densities (1) are known up to a constant, a Metropolis-Hasting algorithm would be a natural choice. In this section, we describe a general Metropolis-Hasting algorithm for sampling from a Riemannian Gaussian distribution on an arbitrary homogeneous Riemannian manifold [Pennec et al., 2019]. However, there are more efficient sampling algorithms that are tailored to specific manifolds (e.g., [Said et al., 2017, Hauberg, 2018]).
The Metropolis-Hasting algorithm involves sampling a candidate  $y$  from a proposal distribution  $q(\cdot |x)$ . The acceptance probability of accepting  $y$  as the new state is  $\alpha (x,y) = \min \{1,q(y|x)p_{\eta ,\sigma}(y) / [q(x|y)p_{\eta ,\sigma}(x)]\}$ . A natural choice for the proposal distribution  $q(\cdot |x)$  could be an exponential-wrapped Gaussian distribution [Galaz-Garcia et al., 2022, Chevallier et al., 2022]. Informally, it's the distribution resulting from "wrapping" a Gaussian distribution on tangent space back to the manifold using the exponential map. Given the current state  $x\in \mathcal{M}$ , we sample a tangent vector  $v\sim N(\mathbf{0},\sigma^2\mathbf{I})$  on the tangent space  $T_{x}\mathcal{M}$ . If  $\| v\|$  is less than the injectivity radius, we then accept the newly proposed state  $y = \exp_x(v)$  with probability  $\alpha (x,y)$ . Please refer to Section 2.5 in Pennec et al. [2019] for the detailed algorithm.
4.2 GDP on  $\mathbb{R}$  and  $S^1$
To evaluate the performance of Algorithm 2, we will conduct simulations on Euclidean space  $\mathbb{R}$  and unit circle  $S^1$  as the relation between  $\mu$  and  $\sigma$  is established in Dong et al. [2022] and Corollary 3.2.1.
We fix sensitivity  $\Delta = 1$  and let  $\sigma = k / 4$  with  $1\leq k\leq 16$ . For each  $\sigma$ , we determine the privacy budget  $\mu$  using two approaches: (i) using Algorithm 1 with  $n_{\varepsilon}$  set as 1000 for  $S^1$  and using  $\mu = 1 / \sigma$  for  $\mathbb{R}$ ; (ii) using Algorithm 2 with  $n = 1000$ ,  $n_{\varepsilon} = 1000$ ,  $m = 100$ ,  $\varepsilon_{\mathrm{max}} = \pi /(2\sigma^2)$  for  $S^1$  and  $\varepsilon_{\mathrm{max}} = \max \{10,5 / \sigma +1 / (2\sigma^2)\}$  for  $\mathbb{R}$ . Since Algorithm 2 is a randomized algorithm, we generate 20 replicates for approach (ii) for each  $\sigma$ . In Figure 1, the first panel plots the sample means of the  $\mu$  generated by approach (ii) (in grey with rectangular symbols) with error bars indicating the minimum & the maximum of the  $\mu$ 's and the  $\mu$  computed by approach (i) (in red with circular symbols). Additionally, as a comparison, we also plot the  $\mu = 1 / \sigma$  for Euclidean space (in blue with triangular symbols) in the first plot. As we see from the first plot, the GDP results on  $S^1$  are almost exactly the same as on  $\mathbb{R}$  for smaller  $\sigma$ . This is expected as manifolds are locally Euclidean,
Figure 1: First and Second plots: Red lines with circular symbols represent the relation between privacy budget  $\mu$  and rate  $\sigma$  on the unit circle  $S^1$ . Blue lines with triangular symbols represent the relation in Euclidean space. Gray lines with rectangular symbols plot the sample mean of the  $\mu$ , across the 20 repeats, computed at a variety of  $\sigma$  using Algorithm 2. The error bar indicates the minimum and maximum of the  $\mu$ 's. Refer to Section 4.2 for details. Third plot: Blue line with triangular symbols indicates the sample mean, across 100 repeats, of the Riemannian distances  $d(\bar{x}, \bar{x}_{\mathrm{laplace}})$  while the red line with circular symbols indicates the sample mean of the Riemannian distances  $d(\bar{x}, \bar{x}_{\mathrm{gauss}})$ . The error bands indicate the sample mean  $\pm 4\mathrm{SE}$ . Refer to Section 5.2 for details.
and the smaller the  $\sigma$  the closer the results will be. As the  $\sigma$  gets larger, the privacy budget  $\mu$  gets smaller on  $S^1$  compared to on  $\mathbb{R}$ . This can be explained by the fact that  $S^1$  is a compact space while  $\mathbb{R}$  is not. For the second panel, we plot the exactly same things for Euclidean spaces. As we can observe from both panels, our Algorithm 2 gives a fairly accurate estimation for larger  $\sigma$ . However, as  $\sigma$  gets smaller, Algorithm 2 has a tendency to generate estimates that exhibit a higher degree of overestimation.
In this section, we evaluate the utility of our Riemannian Gaussian mechanism by focusing on the task of releasing differentially private Fréchet mean. Specifically, we conduct numerical examples on the unit sphere, which is commonly encountered in statistics [Bhattacharya and Bhattacharya, 2012, Mardia et al., 2000]. As a comparison to our Riemannian Gaussian mechanism, we use the Riemannian Laplace mechanism implemented in Reimherr et al. [2021] to achieve GDP. Although the Riemannian Laplace mechanism is developed originally to achieve  $\varepsilon$ -DP, it's shown in Liu et al. [2022] any mechanism that satisfies  $\varepsilon$ -DP can achieve  $\mu$ -GDP with  $\mu = -2\Phi^{-1}(1/(1 + e^{\varepsilon})) \leq \sqrt{\pi/2\varepsilon}$ . Our results show significant improvements in utility for the privatization of the Fréchet mean when using our proposed Gaussian mechanism, as opposed to the Laplace mechanism, in both examples. In Section 5.1, we cover some basics on differentially private Fréchet mean. In Section 5.2, we discuss the numerical results on the sphere. Simulations are done in R on a Mac Mini computer with an Apple M1 processor with 8 GB of RAM running MacOS 13. For more details on each simulation, please refer to Appendix A.6. The R code is available in the GitHub repository: https://github.com/Lei-Ding07/Gaussian-Differential-Privacy-on-Riemannian-Manifolds
5.1 Differentially Private Fréchet Mean
For more details on Fréchet mean under the DP setting, please refer to Reimherr et al. [2021]. Consider a set of data  $x_{1},\ldots ,x_{N}$  on  $\mathcal{M}$ . The Euclidean sample mean can be generalized to Riemannian manifolds as the sample Fréchet mean, which is the minimizer of the sum-of-squared distances to the data,  $\bar{x} = \arg \min_{x\in \mathcal{M}}\sum_{i = 1}^{N}d(x,x_i)^2$ . To ensure the existence & uniqueness of the Fréchet mean and to determine its sensitivity, we need the following assumption.
Assumption 1. The data  $\mathcal{D} \subseteq B_r(m_0)$  for some  $m_0 \in \mathcal{M}$ , where  $r < r^*$  with  $r^* = \min\{\mathrm{inj}\mathcal{M}, \pi/(2\sqrt{\kappa})\}/2$  for  $\kappa > 0$  and  $r^* = \mathrm{inj}\mathcal{M}/2$  for  $\kappa \leq 0$ . Note  $\kappa$  denotes an upper bound on the sectional curvatures of  $\mathcal{M}$ .
Under Assumption 1, we can then compute the sensitivity of the Fréchet mean [Reimherr et al., 2021]. consider two datasets  $\mathcal{D} \simeq \mathcal{D}'$ . If  $\bar{x}$  and  $\bar{x}'$  are the two sample Fréchet means of  $\mathcal{D}$  and  $\mathcal{D}'$
respectively, then
$$
d \left(\bar {x}, \bar {x} ^ {\prime}\right) \leq \frac {2 r (2 - h (r , \kappa))}{n h (r , \kappa)}, \quad h (r, \kappa) = \left\{ \begin{array}{l l} 2 r \sqrt {\kappa} \cot (\sqrt {\kappa} 2 r) & \kappa > 0 \\ 1 & \kappa \leq 0 \end{array} \right.
$$
5.2 Sphere
First, we revisit some background materials on spheres, refer to Bhattacharya and Bhattacharya [2012], Reimherr et al. [2021] for more details. We denote the  $d$ -dimensional unit sphere as  $\tilde{S}^d$  and identify it as a subspace of  $\mathbb{R}^{d+1}$  as  $S^d = \{p \in \mathbb{R}^{d+1} : \|p\|_2 = 1\}$ . Similarly, at each  $p \in S^d$ , we identify the tangent space  $T_p S^d$  as  $T_p S^d = \{v \in \mathbb{R}^{d+1} : v^\top p = 0\}$ . The geodesics are the great circles,  $\gamma_{p,v}(t) = \cos(t)p + \sin(t)v$  with  $-\pi < t \leq \pi$  where  $\gamma_{p,v}$  denotes the geodesic starts at  $p$  with unit direction vector  $v$ . The exponential map  $\exp_p: T_p S^d \to S^d$  is given by  $\exp_p(0) = p$  and  $\exp_p(v) := \cos(\|v\|)p + \sin(v)v/\|v\|$  for  $v \neq 0$ . The inverse of the exponential map  $\log_p: S^d \setminus \{-p\} \to T_p S^d$  has the expression  $\log_p(p) = 0$  and  $\log_p(q) = \arccos(p^\top q)[q - (p^\top q)p][1 - (p^\top q)^2]^{-1/2}$  for  $q \neq p, -q$ . It follows that the distance function is given by  $d(p,q) = \arccos(p^\top q) \in [0,\pi]$ . Therefore,  $S^d$  has an injectivity radius of  $\pi$ .
We initiate our analysis by generating sample data  $\mathcal{D} = \{x_1, \ldots, x_n\}$  from a ball of radius  $\pi / 8$  on  $S^2$  and subsequently computing the Fréchet mean  $\bar{x}$ . To disseminate the private Fréchet mean, we implement two methods: (i) We first generate the privatized mean  $\bar{x}_{\mathrm{gauss}}$  by drawing from  $N_{\mathcal{M}}(\bar{x}, \sigma^2)$  employing the sampling method proposed by Hauberg [2018]. The privacy budget  $\mu$  is then computed using Algorithm 2. (ii) Next, we convert  $\mu$ -GDP to the equivalent  $\varepsilon$ -DP using  $\varepsilon = \log[1 - \Phi(-u/2)] / \Phi(-u/2]$ , and generate the privatized mean  $\bar{x}_{\mathrm{laplace}}$  by sampling from the Riemannian Laplace distribution with footprint  $\bar{x}$  and rate  $\Delta/\varepsilon$  using the sampling method introduced by You and Shung [2022].
Throughout these simulations, we fix the sample size at  $n = 10$  to maintain a constant sensitivity  $\Delta$ . With  $\Delta$  held constant, we let the rate  $\sigma = k / 4$  with  $1 \leq k \leq 12$ . The objective here is to discern the difference between the two distances  $d(\bar{x},\bar{x}_{\mathrm{gauss}})$  and  $d(\bar{x},\bar{x}_{\mathrm{laplace}})$  across varying privacy budgets  $\mu$ . The third plot in Figure 1 displays the sample mean of the Riemannian distances  $d(\bar{x},\bar{x}_{\mathrm{gauss}})$  (in red with circular symbols) and  $d(\bar{x},\bar{x}_{\mathrm{laplace}})$  (in blue with triangular symbols) across 1000 iterations with the error band indicating the sample mean  $\pm 4\mathrm{SE}$ . From observing the third plot, we see that our Gaussian mechanism achieves better utility, especially with a smaller privacy budget  $\mu$ . With larger  $\mu$ , the gain in utility is less pronounced. One obvious reason is that there are much fewer perturbations with larger  $\mu$  for both approaches, so the difference is subtle. The other reason is that Algorithm 2 has a tendency to overestimate  $\mu$  with smaller  $\sigma$ . Effectively,  $\bar{x}_{\mathrm{gauss}}$  satisfies  $\mu$ -GDP with a smaller  $\mu$  compared to  $\bar{x}_{\mathrm{laplace}}$ .
In this paper, we extend the notion of GDP over general Riemannian manifolds. Then we showed that GDP can be achieved when using Riemannian Gaussian distribution as the additive noises. Furthermore, we propose a general MCMC-based algorithm to compute the privacy budget  $\mu$  on manifolds with constant curvature. Lastly, we show through simulations that our Gaussian mechanism outperforms the Laplace mechanism in achieving  $\mu$ -GDP on the unit sphere  $S^d$ .
There are many future research directions. First of all, the framework established in this paper can be used for extending  $(\varepsilon, \delta)$ -DP to general Riemannian manifolds. There are several points of improvement around Algorithm 2 as well. Although Algorithm 2 provides us a general method of computing the privacy budget  $\mu$ , it lacks an error bound on its estimation. Furthermore, due to the random nature of Algorithm 2, a variance estimator of the output is desirable and can better inform the end user. Though we demonstrate the utility of our mechanism through simulation in Section 5, it's difficult to obtain a theoretical utility guarantee due to the lack of simple analytical relation between  $\mu$  and rate  $\sigma$ . Furthermore, although Algorithm 1 requires the manifolds to have constant curvature, it's possible to extend Algorithm 1 and Theorem 4.1 to a slightly more general class of manifolds. Additionally, the Riemannian Gaussian distribution defined in this paper is not the only way of extending Gaussian distribution to Riemannian manifolds as mentioned in Section 3. Potentially, the two other approaches, the wrapped distribution approach, and the heat kernel approach can also be
used to achieve GDP on Riemannian manifolds as well. In particular, there are many rich results around heat kernel on Riemannian manifolds (see Grigoryan [2009] for example). Incorporating the heat kernel in a privacy mechanism presents ample potential for novel and noteworthy discoveries.
We would like to express our greatest appreciation to Professor Xiaowei Wang of Rutgers University and Professor Xi Chen from the University of Alberta for their invaluable guidance, support, discussion, and expertise throughout the course of this research. This work was supported by the Canada CIFAR AI Chairs program, the Alberta Machine Intelligence Institute, the Natural Sciences and Engineering Council of Canada (NSERC), the Canada Research Chair program from NSERC, and the Canadian Statistical Sciences Institute. We also thank Qirui Hu from Tsinghua University and all the constructive suggestions and comments from the reviewers.
V. Arsigny, P. Fillard, X. Pennec, and N. Ayache. Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM Journal on Matrix Analysis and Applications, 29 (1):328-347, 2007. doi: 10.1137/050637996. URL https://doi.org/10.1137/050637996.
D. M. Asta. Kernel density estimation on symmetric spaces. In International Conference on Geometric Science of Information, 2014.
B. Balle and Y.-X. Wang. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 394-403. PMLR, 10-15 Jul 2018. URL https://proceedings.mlr.press/v80/balle18a.html.
B. Balle, G. Barthe, and M. Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, page 6280-6290, Red Hook, NY, USA, 2018. Curran Associates Inc.
A. Barachant, S. Bonnet, M. Congedo, and C. Jutten. Riemannian geometry applied to bci classification. Latent Variable Analysis and Signal Separation, 09 2010. doi: 10.1007/978-3-642-15995-4_78.
B. Barak, K. Chaudhuri, C. Dwork, S. Kale, F. McSherry, and K. Talwar. Privacy, accuracy, and consistency too: A holistic solution to contingency table release. In Proceedings of the Twenty-Sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS '07, page 273-282, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 9781595936851. doi: 10.1145/1265530.1265569. URL https://doi.org/10.1145/1265530.1265569.
M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7(85): 2399-2434, 2006. URL http://jmlr.org/papers/v7/belkin06a.html.
V. Berestovskii and Y. Nikonorov. Riemannian manifolds and homogeneous geodesics. Springer, 2020.
A. Bhattacharya and R. Bhattacharya. Nonparametric Inference on Manifolds: With Applications to Shape Spaces. IMS monograph. Cambridge University Press, 2012. ISBN 9781107019584.
M. Bun and T. Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. 14th Theory of Cryptography Conference, 2016. URL https://arxiv.org/abs/1605.02065.
R. Chakraborty and B. C. Vemuri. Statistics on the Stiefel manifold: Theory and applications. The Annals of Statistics, 47(1):415 - 438, 2019. doi: 10.1214/18-AOS1692. URL https://doi.org/10.1214/18-AOS1692.
P. Turaga, A. Veeraraghavan, and R. Chellappa. Statistical analysis on stiefel and grassmann manifolds with applications in computer vision. In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8, 2008. doi: 10.1109/CVPR.2008.4587733.
P. K. Turaga and A. Srivastava. Riemannian Computing in Computer Vision. Springer Publishing Company, Incorporated, 1st edition, 2015. ISBN 3319229567.
S. Utpala, A. Han, P. Jawanpuria, and B. Mishra. Improved differentially private riemannian optimization: Fast sampling and variance reduction. Transactions on Machine Learning Research, 2023a. ISSN 2835-8856. URL https://openreview.net/forum?id=paguBNtqi0.
S. Utpala, P. Vepakomma, and N. Miolane. Differentially private fréchet mean on the manifold of symmetric positive definite (SPD) matrices with log-euclidean metric. Transactions on Machine Learning Research, 2023b. ISSN 2835-8856. URL https://openreview.net/forum?id= mAx8QqZ14f.
E. B. Vinberg, V. Minachin, D. Alekseevskij, O. Shvartsman, and A. Solodovnikov. Geometry II: spaces of constant curvature. Springer, 1993.
L. Wasserman and S. Zhou. A statistical framework for differential privacy. Journal of the American Statistical Association, 105(489):375-389, 2010. ISSN 01621459. URL http://www.jstor.org/stable/29747034.
F. S. Woods. Space of constant curvature. Annals of Mathematics, 3(1/4):71-112, 1901. ISSN 0003486X. URL http://www.jstor.org/stable/1967636.
K. You and D. Shung. On the spherical laplace distribution, 2022.
P. Zanini, M. Congedo, C. Jutten, S. Said, and Y. Berthoumieu. Transfer learning: A riemannian geometry framework with applications to brain-computer interfaces. IEEE Transactions on Biomedical Engineering, 65(5):1107-1116, 2018. doi: 10.1109/TBME.2017.2742541.
M. Zhang and T. Fletcher. Probabilistic principal geodesic analysis. In C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. URL https://proceedings.neurips.cc/paper_files/paper/2013/file/eb6fdc36b281b7d5eabf33396c2683a2-Paper.pdf.
A.1 Theorem 3.1
First, we need to introduce the notion of privacy profile [Balle et al., 2018]:
The privacy profile  $\delta_{\mathcal{M}}$  of a mechanism  $\mathcal{M}$  is a function associating to each privacy parameter  $\alpha = e^{\varepsilon}$  a bound on the  $\alpha$ -divergence between the results of running the mechanism on two adjacent datasets, i.e.  $\delta_{\mathcal{M}}(\varepsilon) = \sup_{x\simeq x'}D_{e^{\varepsilon}}(\mathcal{M}(x)\| \mathcal{M}(x')$  where the  $\alpha$ -divergence  $(\alpha \geq 1)$  between two probability measures  $\mu ,\mu '$  is defined as