Title: Gaussian Differential Privacy on Riemannian Manifolds
Venue: NeurIPS 2023
Accepted: yes
Keywords: Differential Privacy

Gaussian Differential Privacy

Differential Geometry

Riemannian Manifold

Homogeneous Riemannian Manifold

Frechet Mean
Abstract: We develop an advanced approach for extending Gaussian Differential Privacy (GDP) to general Riemannian manifolds. The concept of GDP stands out as a prominent privacy definition that strongly warrants extension to manifold settings, due to its central limit properties. By harnessing the power of the renowned Bishop-Gromov theorem in geometric analysis, we propose a Riemannian Gaussian distribution that integrates the Riemannian distance, allowing us to achieve GDP in Riemannian manifolds with bounded Ricci curvature. To the best of our knowledge, this work marks the first instance of extending the GDP framework to accommodate general Riemannian manifolds, encompassing curved spaces, and circumventing the reliance on tangent space summaries. We provide a simple algorithm to evaluate the privacy budget $\mu$ on any one-dimensional manifold and introduce a versatile Markov Chain Monte Carlo (MCMC)-based algorithm to calculate $\mu$ on any Riemannian manifold with constant curvature. Through simulations on one of the most prevalent manifolds in statistics, the unit sphere $S^d$, we demonstrate the superior utility of our Riemannian Gaussian mechanism in comparison to the previously proposed Riemannian Laplace mechanism for implementing GDP.

Gaussian Differential Privacy on Riemannian Manifolds

Abstract

We develop an advanced approach for extending Gaussian Differential Privacy (GDP) to general Riemannian manifolds. The concept of GDP stands out as a prominent privacy definition that strongly warrants extension to manifold settings, due to its central limit properties. By harnessing the power of the renowned Bishop-Gromov theorem in geometric analysis, we propose a Riemannian Gaussian distribution that integrates the Riemannian distance, allowing us to achieve GDP in Riemannian manifolds with bounded Ricci curvature. To the best of our knowledge, this work marks the first instance of extending the GDP framework to accommodate general Riemannian manifolds, encompassing curved spaces, and circumventing the reliance on tangent space summaries. We provide a simple algorithm to evaluate the privacy budget µ on any one-dimensional manifold and introduce a versatile Markov Chain Monte Carlo (MCMC)-based algorithm to calculate µ on any Riemannian manifold with constant curvature. Through simulations on one of the most prevalent manifolds in statistics, the unit sphere S d , we demonstrate the superior utility of our Riemannian Gaussian mechanism in comparison to the previously proposed Riemannian Laplace mechanism for implementing GDP.

1 Introduction

As technological advancements continue to accelerate, we are faced with the challenge of managing and understanding increasingly complex data. This data often resides in nonlinear manifolds, commonly found in various domains such as medical imaging [Pennec et al., 2019, Dryden, 2005, Dryden et al., 2009], signal processing [Barachant et al., 2010, Zanini et al., 2018], computer vision [Turaga and Srivastava, 2015, Turaga et al., 2008, Cheng and Vemuri, 2013], and geometric deep learning [Belkin et al., 2006, Niyogi, 2013]. These nonlinear manifolds are characterized by their distinct geometric properties, which can be utilized to extract valuable insights from the data.

As data complexity grows, so does the imperative to safeguard data privacy. Differential Privacy (DP) [Dwork et al., 2006b] has gained recognition as a prominent mathematical framework for quantifying privacy protection, and a number of privacy mechanisms [McSherry and Talwar, 2007, Barak et al., 2007, Wasserman and Zhou, 2010, Reimherr and Awan, 2019] have been devised with the aim of achieving DP. Conventional privacy mechanisms, while effective for dealing with linear data, encounter difficulties when handling complex non-linear data. In such cases, a common approach, referred to as the extrinsic approach, is to embed the non-linear data into the ambient Euclidean space, followed by the application of standard DP mechanisms. However, as exemplified in the work of Reimherr et al. [2021], the intrinsic properties of such non-linear data enable us to achieve better data utility while simultaneously preserving data privacy. Therefore, it is imperative that privacy mechanisms adapt to the complexity of non-linear data by employing tools from differential geometry to leverage the geometric structure within the data.

Related Work Reimherr et al. [2021] is the first to consider the general manifolds in the DP literature. It extends the Laplace mechanism for ε -DP from Euclidean spaces to Riemannian manifolds. Focusing on the task of privatizing Frechet mean, it demonstrates that better utility can be achieved when utilizing the underlying geometric structure within the data. Continuing its work, Soto et al. [2022] develops a K-norm gradient mechanism for ε -DP on Riemannian manifolds and shows that it outperforms the Laplace mechanism previously mentioned in the task of privatizing Frechet mean. Similarly, Utpala et al. [2023b] extends ( ε, δ ) -DP and its Gaussian mechanism but only to one specific manifold, the space of symmetric positive definite matrices (SPDM). Equipping the space of SPDM with the log Euclidean metric, it becomes a geometrically flat space [Arsigny et al., 2007]. This allows them to simplify their approach and work with fewer complications, although at the expense of generality. In contrast to the task of releasing manifold-valued private summary, Han et al. [2022], Utpala et al. [2023a] focus on solving empirical risk minimization problems in a ( ε, δ ) -DP compliant manner by privatizing the gradient which resides on the tangent bundle of Riemannian manifolds. Working on tangent spaces instead of the manifold itself, they could bypass many of the difficulties associated with working under Riemannian manifolds.

Motivations Although the ε -differential privacy (DP) and its Laplace mechanism have been extended to general Riemannian manifolds in Reimherr et al. [2021], there are other variants of DP [Dwork et al., 2006a, Mironov, 2017, Bun and Steinke, 2016, Dong et al., 2022]. Each of them possesses unique advantages over the pure DP definition, and therefore their extensions should be considered as well. As one such variant, GDP offers superior composition and subsampling properties to that of ε -DP. Additionally, it's shown that all hypothesis testing-based privacy definitions converge to the guarantees of GDP in the limit of composition [Dong et al., 2022]. Furthermore, when the dimension of the privatized data approaches infinity, a large class of noise addition private mechanisms is shown to be asymptotically GDP [Dong et al., 2021]. These traits establish GDP as the focal privacy definition among different variants of DP definitions and therefore make it the most suitable option for generalizing to Riemannian manifolds.

Main Contributions With the goal of releasing manifold-valued statistical summary in a GDPcompliant manner, we extend the GDP framework to general Riemannian manifolds, establishing the ability to use Riemannian Gaussian distribution for achieving GDP on Riemannian manifolds. We then develop an analytical form to achieve µ -GDP that covers all the one-dimensional cases. Furthermore, we propose a general MCMC-based algorithm to evaluate the privacy budget µ on Riemannian manifolds with constant curvature. Lastly, we conduct numerical experiments to evaluate the utility of our Riemannian Gaussian mechanism by comparing it to the Riemannian Laplace mechanism. Our results conclusively demonstrate that to achieve GDP, our Gaussian mechanism exhibits superior utility compared to the Laplace mechanism.

2 Notation and Background

In this section, we first cover some basic concepts from Riemannian geometry. The materials covered can be found in standard Riemannian geometry texts such as Lee [2006], Petersen [2006], Pennec et al. [2019], Said [2021]. Then we review some definitions and results on DP and GDP, please refer to Dwork and Roth [2014], Dong et al. [2022, 2021] for more detail.

2.1 Riemannian Geometry

Throughout this paper we let M denote a d -dimensional complete Riemannian manifold unless stated otherwise. A Riemannian metric g is a collection of scalar products ⟨· , ·⟩ x on each tangent space T x M at points x of the manifold that varies smoothly from point to point. For each x , each such scalar product is a positive definite bilinear map ⟨· , ·⟩ x : T x M× T x M→ R .

Equipped with a Riemannian metric g , it grants us the ability to define length and distance on M . Consider a curve γ ( t ) on M , the length of the curve is given by the integral

where the ˙ γ ( t ) is the velocity vector and norm ∥ ˙ γ ( t ) ∥ is uniquely determined by the Riemannian metric g . Note we use ∥ · ∥ to denote the l 2 norm throughout this paper.

It follows that the distance between two points x, y ∈ M is the infimum of the lengths of all piecewise smooth curves from x to y , d ( x, y ) = inf γ : γ (0)= x,γ (1)= y L ( γ ) . In a similar fashion, we can introduce the notion of measure on M . The Riemannian metric g induces a unique measure ν on the Borel σ -algebra of M such that in any chart U , dν = √ det gdλ where g = ( g ij ) is the matrix of the Riemannian metric g in U , and λ is the Lebesgue measure in U [Grigoryan, 2009].

Given a point p ∈ M and a tangent vector v ∈ T p M , there exists a unique geodesic γ ( p,v ) ( t ) starting from p = γ ( p,v ) (0) with tangent vector v = ˙ γ ( p,v ) (0) defined in a small neighborhood of zero. It can then be extended to R since we assume M is complete. This enables us to define the exponential map exp p : T p M→M as exp p ( v ) = γ ( p,v ) (1) . For any p ∈ M , there is a neighborhood V of the origin in T p M and a neighborhood U of p such that exp p | V : V → U is a diffeomorphism. Such U is called a normal neighborhood of p . Locally, the straight line crossing the origin in T p M transforms into a geodesic crossing through p on M via this map. On the normal neighborhood U , the inverse of the exponential map can be defined and is denoted by log p . The injectivity radius at a point p ∈ M is then defined as the maximal radius R such that B p ( R ) ⊂ M is a normal neighborhood of p , and the injectivity radius of M is given by inj M = inf { inj M ( p ) , p ∈ M} .

2.2 Differential Privacy

We start this section with the definition of ( ε, δ ) -DP.

Definition 2.1 ([Dwork et al., 2006a]) . A data-releasing mechanism M is said to be ( ε, δ ) -differentially private with ε ≥ 0 , 0 ≤ δ ≤ 1 , if for any adjacent datasets, denoted as D ≃ D ′ , differing in only one record, we have Pr( M ( D ) ∈ A ) ≤ e ε Pr( M ( D ′ ) ∈ A )+ δ for any measurable set A in the range of M .

Differential privacy can be interpreted from the lens of statistical hypothesis testing [Wasserman and Zhou, 2010, Kairouz et al., 2017]. Given the outcome of a ( ε, δ ) -DP mechanism and a pair of neighboring datasets D ≃ D ′ , consider the hypothesis testing problem with H 0 : The underlying dataset is D and H 1 : The underlying dataset is D ′ . The smaller the ε and δ are, the harder this hypothesis testing will be. That is, it will be harder to detect the presence of one individual based on the outcome of the mechanism. More specifically, ( ε, δ ) -DP tells us the power (that is, 1 - type II error) of any test at significance level α ∈ [0 , 1] is bounded above by e ε α + δ . Using this hypothesis testing interpretation, we can extend ( ε, δ ) -DP to the notion of Gaussian differential privacy.

Let M ( D ) , M ( D ′ ) denote the distributions of the outcome under H 0 , H 1 respectively. Let T ( M ( D ) , M ( D ′ )) : [0 , 1] → [0 , 1] , α ↦→ T ( M ( D ) , M ( D ′ )) ( α ) denote the optimal tradeoff between type I error and type II error. More specifically, T ( M ( D ) , M ( D ′ )) ( α ) is the smallest type II error when type I error equals α .

Definition 2.2 ([Dong et al., 2022]) . A mechanism M is said to satisfy µ -Gaussian Differential Privacy ( µ -GDP) if T ( M ( D ) , M ( D ′ )) ≥ G µ for all neighboring datasets D ≃ D ′ with G µ := T ( N (0 , 1) , N ( µ, 1)) .

Informally, µ -GDP states that it's harder to distinguish D from D ′ than to distinguish between N (0 , 1) and N ( µ, 1) . Similar to the case of ( ε, δ ) -differential privacy, a smaller value of µ provides stronger privacy guarantees. As a privacy definition, µ -GDP enjoys several unique advantages over the ( ε, δ ) -DP definition. Notably, it has a tight composition property that cannot be improved in general. More importantly, a crucial insight in Dong et al. [2022] is that the best way to evaluate the privacy of the composition of many "highly private" mechanisms is through µ -GDP. More specifically, it gives a central limit theorem that states all hypothesis testing-based privacy definitions converge to the guarantees of µ -GDP in the limit of composition. Furthermore, Dong et al. [2021] shows that a large class of noise addition mechanisms is asymptotic µ -GDP when the dimension of the privatized data approaches infinity. These distinct characteristics position µ -GDP as the focal privacy definition among different variants of DP definitions, and we will extend the µ -GDP framework to general Riemannian manifolds in Section 3.

3 Gaussian Differential Privacy on General Riemannian Manifolds

Our primary objective in this study is to disclose a M -valued statistical summary while preserving privacy in a GDP-compliant manner. To this end, we first extend the GDP definition to general

Riemannian manifolds. In Definition 2.2, µ -GDP is defined through the optimal trade-off function T ( M ( D ) , M ( D ′ )) , which is challenging to work with on Riemannian manifolds. We successfully resolve this difficulty by expressing µ -GDP as an infinite collection of ( ε, δ ) -DP (Corollary 1 in Dong et al. [2022]). Since ( ε, δ ) -DP is a well-defined notion on any measurable space [Wasserman and Zhou, 2010], it readily extends to any Riemannian manifold equipped with the Borel σ -algebra. Following this methodology, we define µ -GDP on general Riemannian manifolds as follows.

Definition 3.1. A M -valued data-releasing mechanism M is said to be µ -GDP if it's ( ε, δ µ ( ε )) -DP for all ε ≥ 0 , where

and Φ denotes the cumulative distribution function of the standard normal distribution.

Similarly, we extend the notion of sensitivity to Riemannian manifolds as well.

Definition 3.2 (Reimherr et al. [2021]) . A summary f is said to have a global sensitivity of ∆ < ∞ , with respect to d ( · , · ) , if we have d ( f ( D ) , f ( D ′ )) ≤ ∆ for any two dataset D ≃ D ′ .

Following the extension of µ -GDP to Riemannian manifolds, it is crucial to develop a private mechanism that is compliant with µ -GDP. Given that the Gaussian distribution satisfies µ -GDP on Euclidean space [Dong et al., 2022], we hypothesize that an analogous extension of the Gaussian distribution into Riemannian manifolds would yield similar adherence to µ -GDP. (e.g., [Reimherr et al., 2021] for extending Laplace distribution to satisfy ε -DP on Riemannian manifolds). We introduce the Riemannian Gaussian distribution in the following definition.

Definition 3.3 (Section 2.5 in Pennec et al. [2019]) . Let ( M , g ) be a Riemannian manifold such that

We define a probability density function w.r.t dν as

We call this distribution a Riemannian Gaussian distribution with footprint η and rate σ and denote it by Y ∼ N M ( η, σ 2 ) .

The necessity for Z ( η, σ ) to be finite is generally of little concern, as it has been shown to be so in any compact manifolds [Chakraborty and Vemuri, 2019] or any Hadamard manifolds 1 -with lower-bounded sectional curvature [Said, 2021]. The distribution we introduce has been established to maximize entropy given the first two moments [Pennec et al., 2019, Pennec, 2006], and has already found applications in a variety of scenarios [Zhang and Fletcher, 2013, Hauberg, 2018, Said et al., 2017, Cheng and Vemuri, 2013, Zanini et al., 2018, Chakraborty and Vemuri, 2019]. When M = R d , the Riemannian Gaussian distribution reduces to the multivariate Gaussian distribution with a mean of η and a variance of σ 2 I .

However, it is critical to highlight that this extension is not the only possible method for integrating the Gaussian distribution into Riemannian manifolds. Other approaches are available, such as the heat kernel diffusion process articulated by Grigoryan [2009], or the exponential-wrapped Gaussian introduced by Chevallier et al. [2022]. For an in-depth exploration of sampling from the Riemannian Gaussian distribution defined above, we refer readers to Section 4.1.

Furthermore, the subsequent theorem underscores the ability of the Riemannian Gaussian distribution, as defined herein, to meet the requirements of Gaussian Differential Privacy.

Theorem 3.1. Let M be a Riemannian manifold with lower bounded Ricci curvature and f be a M -valued summary with global sensitivity ∆ . The Riemannian Gaussian distribution with footprint f ( D ) and rate σ satisfies µ -GDP for some µ > 0 .

While Theorem 3.1 confirms the potential of the Riemannian Gaussian distribution to achieve GDP, it leaves the relationship between the privacy budget ( µ ) and the rate ( σ ) undefined. The subsequence theorem establishes such a connection:

Theorem 3.2 (Riemannian Gaussian Mechanism) . Let M be a Riemannian manifold with lower bounded Ricci curvature and f be a M -valued summary with global sensitivity ∆ . The Riemannian Gaussian distribution with footprint f ( D ) and rate σ > 0 is µ -GDP if and only if µ satisfies the following condition, ∀ ε ≥ 0 ,

where A := { y ∈ M : p η 1 ,σ ( y ) /p η 2 ,σ ( y ) ≥ e ε } and η 1 := f ( D ) , η 2 := f ( D ′ ) .

Given the rate σ , Theorem 3.2 provides us a way of computing the privacy budget µ through the inequality (2). When M = R d , the set A enjoys a tractable form, A = { y ∈ R d : ⟨ y -η 2 , η 1 -η 2 ⟩ ≥ 1 2 (2 σ 2 ε/ ∥ η 1 -η 2 ∥ + ∥ η 1 -η 2 ∥ ) } , and the inequality (2) then reduces to,

where the equality holds if and only if σ = ∆ /µ , which reduces to the Gaussian mechanism on Euclidean spaces in Dong et al. [2022]. It's worth pointing out that on R d , the Pythagorean theorem allows us to reduce the integrals to one-dimensional integrals resulting in a simple solution for any dimension d . Unfortunately, the lack of Pythagorean theorem on non-Euclidean spaces makes it difficult to evaluate the integrals on manifolds of dimensions greater than one. It's known that any smooth, connected one-dimensional manifold is diffeomorphic either to the unit circle S 1 or to some interval of R [Milnor and Weaver, 1997]. Therefore, we encompass all one-dimensional cases by presenting the following result on S 1 .

Corollary 3.2.1 (Riemannian Gaussian Mechanism on S 1 ) . Let f be a S 1 -valued summary with global sensitivity ∆ . The Riemannian Gaussian distribution N M ( f ( D ) , σ 2 ) is µ -GDP if and only if µ satisfies the following condition, ∀ ε ∈ [0 , π ∆ / (2 σ 2 )] , h ( σ, ε, ∆) ≤ δ µ ( ε ) where

with C ( σ ) = Φ( π/σ ) -Φ( -π/σ ) .

In summary, Corollary 3.2.1 provides us the analytical form for the integrals in (2). By specifying the rate σ and sensitivity ∆ , it becomes feasible to compute the privacy budget µ , which is the smallest µ such that h ( σ, ε, ∆) ≤ δ µ ( ε ) for all ε ∈ [0 , π ∆ / (2 σ 2 )] . The systematic procedure is summarized in Algorithm 1. It is important to emphasize that the result in Corollary 3.2.1 together with the existing Euclidean space result covers all one-dimensional cases, and for manifolds with dimension d > 1 , we will tackle it in Section 4.

4 Numerical Approach

In Section 3, we demonstrate that our proposed Riemannian Gaussian distribution can be used to achieve GDP in Theorem 3.1 and 3.2. Furthermore, we document our method in Algorithm 1 to compute the privacy budget µ in S 1 . This and already existing Euclidean results encompass all the one-dimensional Riemannian manifolds. However, for general Riemannian manifolds of dimension

d > 1 , the integrals in inequality (2) are difficult to compute (see Section 3 for more explanation). One of the central difficulties is the dependence of the normalizing constant Z ( η, σ ) on the footprint η makes the expression of A intractable. To avoid this dependence, we introduce the concept of homogeneous Riemannian manifolds.

Definition 4.1 (Definition 4.6.1 in Berestovskii and Nikonorov [2020]) . A Riemannian manifold ( M , g ) is called a homogeneous Riemannian manifold if a Lie group G acts transitively and isometrically on M .

Homogeneous Riemannian manifolds encompass a broad class of manifolds that are commonly encountered in statistics such as the (hyper)sphere [Bhattacharya and Bhattacharya, 2012, Mardia et al., 2000], the space of SPD Matrices[Pennec et al., 2019, Said et al., 2017, Hajri et al., 2016], the Stiefel manifold [Chakraborty and Vemuri, 2019, Turaga et al., 2008] and the Grassmann manifold [Turaga et al., 2008]. It's a more general class than Riemannian symmetric space, which is a common setting used in geometric statistics [Cornea et al., 2017, Asta, 2014, Said et al., 2018, Said, 2021, Chevallier et al., 2022]. Please refer to Appendix A.4 for more detail.

Informally, a homogeneous Riemannian manifold looks geometrically the same at every point. The transitive property required in the definition implies that any homogeneous Riemannian manifold M only has one orbit. Therefore, we have the following proposition.

Proposition 4.1. If M is a homogeneous Riemannian manifolds, then Z ( η 1 , y ) = Z ( η 2 , y ) for any η 1 , η 2 ∈ M .

Therefore, on homogeneous Riemannian manifolds, we can simplify the set A in Theorem 3.2 to A = { y ∈ M : d ( η 2 , y ) 2 -d ( η 1 , y ) 2 ≥ 2 σ 2 ε } . To further simplify (2), we will need a much stronger assumption than homogeneous Riemannian manifolds. In particular, we require the condition of constant curvature.

Theorem 4.1 (Riemannian Gaussian Mechanism on Manifolds with Constant Curvature) . Let M be a Riemannian manifold with constant curvature and f be a M -valued summary with global sensitivity ∆ . The Riemannian Gaussian distribution with footpoint f ( D ) and rate σ > 0 is µ -GDP if and only if µ satisfies the following condition: ∃ η 1 , η 2 such that d ( η 1 , η 2 ) = ∆ and ∀ ε ≥ 0

where A = { y ∈ M : d ( η 2 , y ) 2 -d ( η 1 , y ) 2 ≥ 2 σ 2 ε } .

Theorem 4.1 tells us that instead of evaluating the integrals of 2 on every neighboring pairs η 1 and η 2 , we only need to check one such pair. Despite the convenience it provides us, evaluating the integrals remains a challenge, even for an elementary space like S d with d > 1 . To circumvent this challenge, we employ the MCMC technique for the integral computations. The main idea is simple: Let Y 1 , Y 2 , . . . , Y n be independent and identically distributed random variables from N M ( η, σ 2 ) , and denotes Z i = 1 { 1 2 σ 2 ( -d ( η 1 , Y i ) 2 + d ( η 2 , Y i ) 2 ) ≥ ε } . By the strong law of large number [Dudley, 2002], we then have

By using 1 n ∑ n i =1 Z i as an approximation, we avoid the challenge of evaluating the integrals analytically. The detailed algorithm is documented in Algorithm 2. It's known that any space of constant curvature is isomorphic to one of the spaces: Euclidean space, sphere, and hyperbolic space [Vinberg et al., 1993, Woods, 1901]. Therefore, Algorithm 2 offers a straightforward and practical method for assessing the privacy budget µ on spheres and hyperbolic spaces. Furthermore, Algorithm 2 can be extended to a more general class of Riemannian manifold by sampling more than one pair of η, η ′ in step 1. The determination of the number of pairs being sampled and the selection method to ensure sufficient dissimilarity among the sampled pairs is an aspect that requires further investigation.

Algorithm 2 Computing µ on Manifolds with Constant Curvature

Input: Sensitivity ∆ , rate σ 2 , Monte Carlo sample size n , number of ε used n ε , maximum ε used ε max , number of MCMC samples m

Output: privacy budget µ ;

1: Sample a random point η on M and another point η ′ on the sphere center at η with radius ∆ .

2: for each j in 1 , 2 , . . . , m do

3: Sample y 1 j , . . . , y nj ∼ N M ( η, σ 2 ) , y ′ 1 j , . . . , y ′ nj ∼ N M ( η ′ , σ 2 ) .

4: Compute d ij = d ( η ′ , y ij ) 2 -d ( η, y ij ) 2 and d ′ ij = d ( η ′ , y ′ ij ) 2 -d ( η, y ′ ij ) 2 for i in 1 , 2 , . . . , n .

5: for each ε in { k, 2 k, . . . , n ε k } where k = ε max /n ε do

6: Compute l ( j ) ε = ∑ n i =1 1 ( d ij ≥ 2 σ 2 ε ) /n -e ε ∑ n i =1 1 ( d ′ ij ≥ 2 σ 2 ε ) /n .

7: end for

8: end for

9: Compute l ε = ∑ m j =1 l ( j ) ε /m and µ ε via l ε = δ µ ( ε ) for each ε . Compute µ = max ε µ ε .

10: Return : µ .

4.1 Sampling from Riemannian Gaussian Distribution

The one crucial step in Algorithm 2 involves sampling from the two Riemannian Gaussian distributions N M ( η, σ 2 ) and N M ( η ′ , σ 2 ) . Since their densities (1) are known up to a constant, a Metropolis-Hasting algorithm would be a natural choice. In this section, we describe a general Metropolis-Hasting algorithm for sampling from a Riemannian Gaussian distribution on an arbitrary homogeneous Riemannian manifold [Pennec et al., 2019]. However, there are more efficient sampling algorithms that are tailored to specific manifolds (e.g., [Said et al., 2017, Hauberg, 2018]).

The Metropolis-Hasting algorithm involves sampling a candidate y from a proposal distribution q ( ·| x ) . The acceptance probability of accepting y as the new state is α ( x, y ) = min { 1 , q ( y | x ) p η,σ ( y ) / [ q ( x | y ) p η,σ ( x )] } . A natural choice for the proposal distribution q ( ·| x ) could be an exponential-wrapped Gaussian distribution [Galaz-Garcia et al., 2022, Chevallier et al., 2022]. Informally, it's the distribution resulting from "wrapping" a Gaussian distribution on tangent space back to the manifold using the exponential map. Given the current state x ∈ M , we sample a tangent vector v ∼ N ( 0 , σ 2 I ) on the tangent space T x M . If ∥ v ∥ is less than the injectivity radius, we then accept the newly proposed state y = exp x ( v ) with probability α ( x, y ) . Please refer to Section 2.5 in Pennec et al. [2019] for the detailed algorithm.

4.2 GDP on R and S 1

To evaluate the performance of Algorithm 2, we will conduct simulations on Euclidean space R and unit circle S 1 as the relation between µ and σ is established in Dong et al. [2022] and Corollary 3.2.1.

We fix sensitivity ∆ = 1 and let σ = k/ 4 with 1 ≤ k ≤ 16 . For each σ , we determine the privacy budget µ using two approaches: (i) using Algorithm 1 with n ε set as 1000 for S 1 and using µ = 1 /σ for R ; (ii) using Algorithm 2 with n = 1000 , n ε = 1000 , m = 100 , ε max = π/ (2 σ 2 ) for S 1 and ε max = max { 10 , 5 /σ +1 / (2 σ 2 ) } for R . Since Algorithm 2 is a randomized algorithm, we generate 20 replicates for approach (ii) for each σ . In Figure 1, the first panel plots the sample means of the µ generated by approach (ii) (in grey with rectangular symbols) with error bars indicating the minimum & the maximum of the µ 's and the µ computed by approach (i) (in red with circular symbols). Additionally, as a comparison, we also plot the µ = 1 /σ for Euclidean space (in blue with triangular symbols) in the first plot. As we see from the first plot, the GDP results on S 1 are almost exactly the same as on R for smaller σ . This is expected as manifolds are locally Euclidean,

GDP on S1

4

True

Euclidean

3

MCMC

2

1

0

0

1

2

3

4

GDPonR

4

Euclidean

3

MCMC

三

2

1

2

3

4

GDPonSphere

Riemannian distance

0.6

Gauss

Laplace

0.4

0.2

1

2

3

4

μ

and the smaller the σ the closer the results will be. As the σ gets larger, the privacy budget µ gets smaller on S 1 compared to on R . This can be explained by the fact that S 1 is a compact space while R is not. For the second panel, we plot the exactly same things for Euclidean spaces. As we can observe from both panels, our Algorithm 2 gives a fairly accurate estimation for larger σ . However, as σ gets smaller, Algorithm 2 has a tendency to generate estimates that exhibit a higher degree of overestimation.

5 Simulations

In this section, we evaluate the utility of our Riemannian Gaussian mechanism by focusing on the task of releasing differentially private Fréchet mean. Specifically, we conduct numerical examples on the unit sphere, which is commonly encountered in statistics [Bhattacharya and Bhattacharya, 2012, Mardia et al., 2000]. As a comparison to our Riemannian Gaussian mechanism, we use the Riemannian Laplace mechanism implemented in Reimherr et al. [2021] to achieve GDP. Although the Riemannian Laplace mechanism is developed originally to achieve ε -DP, it's shown in Liu et al. [2022] any mechanism that satisfies ε -DP can achieve µ -GDP with µ = -2Φ -1 (1 / (1 + e ε )) ≤ √ π/ 2 ε . Our results show significant improvements in utility for the privatization of the Fréchet mean when using our proposed Gaussian mechanism, as opposed to the Laplace mechanism, in both examples. In Section 5.1, we cover some basics on differentially private Fréchet mean. In Section 5.2, we discuss the numerical results on the sphere. Simulations are done in R on a Mac Mini computer with an Apple M1 processor with 8 GB of RAM running MacOS 13. For more details on each simulation, please refer to Appendix A.6. The R code is available in the GitHub repository: https://github. com/Lei-Ding07/Gaussian-Differential-Privacy-on-Riemannian-Manifolds

5.1 Differentially Private Fréchet Mean

For more details on Fréchet mean under the DP setting, please refer to Reimherr et al. [2021]. Consider a set of data x 1 , . . . , x N on M . The Euclidean sample mean can be generalized to Riemannian manifolds as the sample Fréchet mean, which is the minimizer of the sum-of-squared distances to the data, ¯ x = arg min x ∈M ∑ N i =1 d ( x, x i ) 2 . To ensure the existence & uniqueness of the Fréchet mean and to determine its sensitivity, we need the following assumption.

Assumption 1. The data D ⊆ B r ( m 0 ) for some m 0 ∈ M , where r < r ∗ with r ∗ = min { inj M , π/ (2 √ κ ) } / 2 for κ > 0 and r ∗ = inj M / 2 for κ ≤ 0 . Note κ denotes an upper bound on the sectional curvatures of M .

Under Assumption 1, we can then compute the sensitivity of the Fréchet mean [Reimherr et al., 2021]. consider two datasets D ≃ D ′ . If ¯ x and ¯ x ′ are the two sample Fréchet means of D and D ′

respectively, then

5.2 Sphere

First, we revisit some background materials on spheres, refer to Bhattacharya and Bhattacharya [2012], Reimherr et al. [2021] for more details. We denote the d -dimensional unit sphere as S d and identify it as a subspace of R d +1 as S d = { p ∈ R d +1 : ∥ p ∥ 2 = 1 } . Similarly, at each p ∈ S d , we identify the tangent space T p S d as T p S d = { v ∈ R d +1 : v ⊤ p = 0 } . The geodesics are the great circles, γ p,v ( t ) = cos( t ) p + sin( t ) v with -π < t ≤ π where γ p,v denotes the geodesic starts at p with unit direction vector v . The exponential map exp p : T p S d → S d is given by exp p (0) = p and exp p ( v )) := cos( ∥ v ∥ ) p + sin( v ) v/ ∥ v ∥ for v = 0 . The inverse of the exponential map log p : S d \ {-p } → T p S d has the expression log p ( p ) = 0 and log p ( q ) = arccos( p ⊤ q )[ q -( p ⊤ q ) p ] [ 1 -( p ⊤ q ) 2 ] -1 / 2 for q = p, -q . It follows that the distance function is given by d ( p, q ) = arccos( p ⊤ q ) ∈ [0 , π ] . Therefore, S d has an injectivity radius of π .

̸

We initiate our analysis by generating sample data D = { x 1 , . . . , x n } from a ball of radius π/ 8 on S 2 and subsequently computing the Fréchet mean ¯ x . To disseminate the private Fréchet mean, we implement two methods: (i) We first generate the privatized mean ¯ x gauss by drawing from N M (¯ x, σ 2 ) employing the sampling method proposed by Hauberg [2018]. The privacy budget µ is then computed using Algorithm 2. (ii) Next, we convert µ -GDP to the equivalent ε -DP using ε = log[1 -Φ( -u/ 2)) / Φ( -u/ 2)] , and generate the privatized mean ¯ x laplace by sampling from the Riemannian Laplace distribution with footprint ¯ x and rate ∆ /ε using the sampling method introduced by You and Shung [2022].

Throughout these simulations, we fix the sample size at n = 10 to maintain a constant sensitivity ∆ . With ∆ held constant, we let the rate σ = k/ 4 with 1 ≤ k ≤ 12 . The objective here is to discern the difference between the two distances d (¯ x, ¯ x gauss ) and d (¯ x, ¯ x laplace ) across varying privacy budgets µ . The third plot in Figure 1 displays the sample mean of the Riemannian distances d (¯ x, ¯ x gauss ) (in red with circular symbols) and d (¯ x, ¯ x laplace ) (in blue with triangular symbols) across 1000 iterations with the error band indicating the sample mean ± 4 SE. From observing the third plot, we see that our Gaussian mechanism achieves better utility, especially with a smaller privacy budget µ . With larger µ , the gain in utility is less pronounced. One obvious reason is that there are much fewer perturbations with larger µ for both approaches, so the difference is subtle. The other reason is that Algorithm 2 has a tendency to overestimate µ with smaller σ . Effectively, ¯ x gauss satisfies µ -GDP with a smaller µ compared to ¯ x laplace .

6 Conclusions and Future Directions

In this paper, we extend the notion of GDP over general Riemannian manifolds. Then we showed that GDP can be achieved when using Riemannian Gaussian distribution as the additive noises. Furthermore, we propose a general MCMC-based algorithm to compute the privacy budget µ on manifolds with constant curvature. Lastly, we show through simulations that our Gaussian mechanism outperforms the Laplace mechanism in achieving µ -GDP on the unit sphere S d .

There are many future research directions. First of all, the framework established in this paper can be used for extending ( ε, δ ) -DP to general Riemannian manifolds. There are several points of improvement around Algorithm 2 as well. Although Algorithm 2 provides us a general method of computing the privacy budget µ , it lacks an error bound on its estimation. Furthermore, due to the random nature of Algorithm 2, a variance estimator of the output is desirable and can better inform the end user. Though we demonstrate the utility of our mechanism through simulation in Section 5, it's difficult to obtain a theoretical utility guarantee due to the lack of simple analytical relation between µ and rate σ . Furthermore, although Algorithm 1 requires the manifolds to have constant curvature, it's possible to extend Algorithm 1 and Theorem 4.1 to a slightly more general class of manifolds. Additionally, the Riemannian Gaussian distribution defined in this paper is not the only way of extending Gaussian distribution to Riemannian manifolds as mentioned in Section 3. Potentially, the two other approaches, the wrapped distribution approach, and the heat kernel approach can also be

̸

used to achieve GDP on Riemannian manifolds as well. In particular, there are many rich results around heat kernel on Riemannian manifolds (see Grigoryan [2009] for example). Incorporating the heat kernel in a privacy mechanism presents ample potential for novel and noteworthy discoveries.

References

Arsigny, P. Fillard, X. Pennec, and N. Ayache. Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM Journal on Matrix Analysis and Applications , 29 (1):328-347, 2007. doi: 10.1137/050637996. URL https://doi.org/10.1137/050637996 .

M. Asta. Kernel density estimation on symmetric spaces. In International Conference on Geometric Science of Information , 2014.

Balle and Y.-X. Wang. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 394-403. PMLR, 10-15 Jul 2018. URL https://proceedings.mlr.press/ v80/balle18a.html .

Balle, G. Barthe, and M. Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , NIPS'18, page 6280-6290, Red Hook, NY, USA, 2018. Curran Associates Inc.

Barachant, S. Bonnet, M. Congedo, and C. Jutten. Riemannian geometry applied to bci classification. Latent Variable Analysis and Signal Separation , 09 2010. doi: 10.1007/978-3-642-15995-4_ 78.

Barak, K. Chaudhuri, C. Dwork, S. Kale, F. McSherry, and K. Talwar. Privacy, accuracy, and consistency too: A holistic solution to contingency table release. In Proceedings of the TwentySixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems , PODS '07, page 273-282, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 9781595936851. doi: 10.1145/1265530.1265569. URL https://doi.org/10.1145/1265530. 1265569 .

Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research , 7(85): 2399-2434, 2006. URL http://jmlr.org/papers/v7/belkin06a.html .

Berestovskii and Y. Nikonorov. Riemannian manifolds and homogeneous geodesics . Springer, 2020.

Bhattacharya and R. Bhattacharya. Nonparametric Inference on Manifolds: With Applications to Shape Spaces . IMS monograph. Cambridge University Press, 2012. ISBN 9781107019584.

Bun and T. Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. 14th Theory of Cryptography Conference , 2016. URL https://arxiv.org/abs/1605. 02065 .

Chakraborty and B. C. Vemuri. Statistics on the Stiefel manifold: Theory and applications. The Annals of Statistics , 47(1):415 - 438, 2019. doi: 10.1214/18-AOS1692. URL https: //doi.org/10.1214/18-AOS1692 .

Cheng and B. C. Vemuri. A novel dynamic system in the space of spd matrices with applications to appearance tracking. SIAM J. Img. Sci. , 6(1):592-615, jan 2013. doi: 10.1137/110853376. URL https://doi.org/10.1137/110853376 .

Chevallier, D. Li, Y . Lu, and D. Dunson. Exponential-wrapped distributions on symmetric spaces. SIAM Journal on Mathematics of Data Science , 4(4):1347-1368, 2022.

Cornea, H. Zhu, P. Kim, and J. G. Ibrahim. Regression models on riemannian symmetric spaces. Journal of the Royal Statistical Society. Series B (Statistical Methodology) , 79(2):463-482, 2017. ISSN 13697412, 14679868. URL http://www.jstor.org/stable/44682521 .

Dong, W. Su, and L. Zhang. A central limit theorem for differentially private query answering. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 14759-14770. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ 7c2c48a32443ad8f805e48520f3b26a4-Paper.pdf .

Dong, A. Roth, and W. J. Su. Gaussian differential privacy. Journal of the Royal Statistical Society Series B: Statistical Methodology , 84(1):3-37, 2022.

L. Dryden. Statistical analysis on high-dimensional spheres and shape spaces. The Annals of Statistics , 33(4):1643-1665, 2005. ISSN 00905364. URL http://www.jstor.org/stable/ 3448620 .

L. Dryden, A. Koloydenko, and D. Zhou. Non-euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. The Annals of Applied Statistics , 3(3):1102-1123, 2009. ISSN 19326157, 19417330. URL http://www.jstor.org/stable/30242879 .

M. Dudley. Real Analysis and Probability . Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2nd edition, 2002. doi: 10.1017/CBO9780511755347.

Dwork and A. Roth. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. , 9:211-407, aug 2014. ISSN 1551-305X. doi: 10.1561/0400000042. URL https: //doi.org/10.1561/0400000042 .

Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In Proceedings of the 24th Annual International Conference on The Theory and Applications of Cryptographic Techniques , EUROCRYPT'06, page 486-503, Berlin, Heidelberg, 2006a. Springer-Verlag. ISBN 3540345469. doi: 10.1007/11761679_29. URL https://doi.org/10.1007/11761679_29 .

Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In S. Halevi and T. Rabin, editors, Theory of Cryptography , pages 265-284, Berlin, Heidelberg, 2006b. Springer Berlin Heidelberg.

Galaz-Garcia, M. Papamichalis, K. Turnbull, S. Lunagomez, and E. Airoldi. Wrapped distributions on homogeneous riemannian manifolds. arXiv preprint arXiv:2204.09790 , 2022.

Grigoryan. Heat kernel and analysis on manifolds . American Mathematical Soc., 2009.

Hajri, I. Ilea, S. Said, L. Bombrun, and Y. Berthoumieu. Riemannian laplace distribution on the space of symmetric positive definite matrices. Entropy , 18(3), 2016. ISSN 1099-4300. doi: 10.3390/e18030098. URL https://www.mdpi.com/1099-4300/18/3/98 .

Han, B. Mishra, P. Jawanpuria, and J. Gao. Differentially private riemannian optimization. arXiv preprint arXiv:2205.09494 , 2022.

Hauberg. Directional statistics with the spherical normal distribution. In 2018 21st International Conference on Information Fusion (FUSION) , pages 704-711, 2018. doi: 10.23919/ICIF.2018. 8455242.

Helgason. Differential geometry and symmetric spaces . New York and London: Academic Press, 1962.

Kairouz, S. Oh, and P. Viswanath. The composition theorem for differential privacy. IEEE Transactions on Information Theory , 63(6):4037-4049, 2017. doi: 10.1109/TIT.2017.2685505.

M. Lee. Riemannian manifolds: an introduction to curvature . Springer Science & Business Media, 2006.

Liu, K. Sun, B. Jiang, and L. Kong. Identification, amplification and measurement: A bridge to gaussian differential privacy. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems , 2022. URL https://openreview.net/forum?id= UpNCpGvD96A .

V. Mardia, P. E. Jupp, and K. Mardia. Directional statistics . Wiley, 2000.

McSherry and K. Talwar. Mechanism design via differential privacy. Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS , pages 94-103, 11 2007. doi: 10.1109/FOCS.2007.66.

Milnor and D. W. Weaver. Topology from the differentiable viewpoint . Princeton university press, 1997.

Mironov. Rényi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF) , pages 263-275. IEEE, 2017.

Niyogi. Manifold regularization and semi-supervised learning: Some theoretical analyses. Journal of Machine Learning Research , 14(37):1229-1250, 2013. URL http://jmlr.org/papers/ v14/niyogi13a.html .

Pennec. Intrinsic statistics on riemannian manifolds: Basic tools for geometric measurements. Journal of Mathematical Imaging and Vision , 25:127-154, 07 2006. doi: 10.1007/s10851-006-6228-4.

Pennec, S. Sommer, and T. Fletcher. Riemannian geometric statistics in medical image analysis . Academic Press, 2019.

Petersen. Riemannian geometry . Springer, 3rd edition, 2006.

Reimherr and J. Awan. Kng: The k-norm gradient mechanism. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/ faefec47428cf9a2f0875ba9c2042a81-Paper.pdf .

Reimherr, K. Bharath, and C. Soto. Differential privacy over riemannian manifolds. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 12292-12303. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ 6600e06fe9350b62c1e343504d4a7b86-Paper.pdf .

Said. Statistical models and probabilistic methods on riemannian manifolds, 2021.

Said, L. Bombrun, Y. Berthoumieu, and J. H. Manton. Riemannian gaussian distributions on the space of symmetric positive definite matrices. IEEE Transactions on Information Theory , 63(4): 2153-2170, 2017. doi: 10.1109/TIT.2017.2653803.

Said, H. Hajri, L. Bombrun, and B. C. Vemuri. Gaussian distributions on riemannian symmetric spaces: Statistical learning with structured covariance matrices. IEEE Transactions on Information Theory , 64(2):752-772, 2018. doi: 10.1109/TIT.2017.2713829.

Soto, K. Bharath, M. Reimherr, and A. Slavkovi´ c. Shape and structure preserving differential privacy. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 24693-24705. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/ file/9c84feb75eae1ef6389f31b3ef050b6a-Paper-Conference.pdf .

Turaga, A. Veeraraghavan, and R. Chellappa. Statistical analysis on stiefel and grassmann manifolds with applications in computer vision. In 2008 IEEE Conference on Computer Vision and Pattern Recognition , pages 1-8, 2008. doi: 10.1109/CVPR.2008.4587733.

K. Turaga and A. Srivastava. Riemannian Computing in Computer Vision . Springer Publishing Company, Incorporated, 1st edition, 2015. ISBN 3319229567.

Utpala, A. Han, P. Jawanpuria, and B. Mishra. Improved differentially private riemannian optimization: Fast sampling and variance reduction. Transactions on Machine Learning Research , 2023a. ISSN 2835-8856. URL https://openreview.net/forum?id=paguBNtqiO .

Utpala, P. Vepakomma, and N. Miolane. Differentially private fréchet mean on the manifold of symmetric positive definite (SPD) matrices with log-euclidean metric. Transactions on Machine Learning Research , 2023b. ISSN 2835-8856. URL https://openreview.net/forum?id= mAx8QqZ14f .

B. Vinberg, V. Minachin, D. Alekseevskij, O. Shvartsman, and A. Solodovnikov. Geometry II: spaces of constant curvature . Springer, 1993.

Wasserman and S. Zhou. A statistical framework for differential privacy. Journal of the American Statistical Association , 105(489):375-389, 2010. ISSN 01621459. URL http://www.jstor. org/stable/29747034 .

S. Woods. Space of constant curvature. Annals of Mathematics , 3(1/4):71-112, 1901. ISSN 0003486X. URL http://www.jstor.org/stable/1967636 .

You and D. Shung. On the spherical laplace distribution, 2022.

Zanini, M. Congedo, C. Jutten, S. Said, and Y. Berthoumieu. Transfer learning: A riemannian geometry framework with applications to brain-computer interfaces. IEEE Transactions on Biomedical Engineering , 65(5):1107-1116, 2018. doi: 10.1109/TBME.2017.2742541.

Zhang and T. Fletcher. Probabilistic principal geodesic analysis. In C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Weinberger, editors, Advances in Neural Information Processing Systems , volume 26. Curran Associates, Inc., 2013. URL https://proceedings.neurips. cc/paper_files/paper/2013/file/eb6fdc36b281b7d5eabf33396c2683a2-Paper.pdf .

A Appendix

A.1 Theorem 3.1

First, we need to introduce the notion of privacy profile [Balle et al., 2018]:

The privacy profile δ M of a mechanism M is a function associating to each privacy parameter α = e ε a bound on the α -divergence between the results of running the mechanism on two adjacent datasets, i.e. δ M ( ε ) = sup x ≃ x ′ D e ε ( M ( x ) ∥M ( x ′ ) where the α -divergence ( α ≥ 1) between two probability measures µ, µ ′ is defined as

where E ranges over all measurable subsets of Z, [ · ] + = max {· , 0 } . 2 Informally speaking, the privacy profile represents the set of all of the privacy parameters under which a mechanism provides differential privacy. Furthermore, the privacy profile can be computed using Theorem 5 of Balle and Wang [2018]:

where L D , D ′ M is the privacy loss random variable of the mechanism M on inputs D ≃ D ′ defined as L D , D ′ M = log ( dµ/dµ ′ ) ( z ) , where µ = M ( D ) , µ ′ = M ( D ′ ) , and z ∼ µ . It follows that for our Riemannian Gaussian mechanism M , the privacy profile δ M can be rewritten as

where A := { y ∈ M : p η 1 ,σ ( y ) /p η 2 ,σ ( y ) ≥ e ε } and η 1 := f ( D ) , η 2 := f ( D ′ ) . This is exactly the left-hand side of (2).

The following theorem establishes a connection between GDP and privacy profile:

Theorem A.1 (Theorem 3.3 of Liu et al. [2022]) . Let µ 0 := √ lim ε → + ∞ ε 2 -2 log δ A ( ε ) . A privacy mechanism A with the privacy profile δ A ( ε ) is µ -GDP if and only if µ 0 < ∞ and µ is no smaller than µ 0 .

Theorem A.1 implies that a mechanism with finite µ 0 is µ -GDP for some privacy budget µ . Note that Theorem 3.1 only tells us that Riemannian Gaussian distribution can be used to achieve µ -GDP for some µ . Therefore, to prove it, we only need to show that µ 0 is finite. We will show µ 0 < ∞ by demonstrate that -log δ A ( ε ) = O ( ε 2 ) . To do so, we will need the Bishop-Gromov comparison theorem:

Theorem A.2 (Bishop-Gromov; Lemma 7.1.4 in Petersen [2006]) . Let M be a complete n -dimensional Riemannian manifold whose Ricci curvature satisfies the lower bound

for a constant K ∈ R . Let M n K be the complete n -dimensional simply connected space of constant sectional curvature K (and hence of constant Ricci curvature ( n -1) K ). Denote by B ( p, r ) the ball of radius r around a point p , defined with respect to the Riemannian distance function. Then, for any p ∈ M and p K ∈ M n K , the function

is non-increasing on (0 , ∞ ) . As r goes to zero, the ratio approaches one, so together with the monotonicity this implies that

Bishop-Gromov comparison theorem not only gives us the control of volume growth of certain manifolds but also gives a rough classification by sectional curvature. Besides, this is a global property in the sense that p and p K can be arbitrary points on the manifolds.

A.1.1 Proof of Theorem 3.1

Proof. By A.1, we only need to show that for any η ∈ M , when ε →∞ ,

where A is given by

Let's consider M\ A = { y ∈ M : p η,σ ( y ) /p η ′ ,σ ( y ) ≤ e ε } . We have

where C = log( Z ( η, σ )) -log( Z ( η ′ , σ )) . Thus we have,

Let r = 2 σ 2 ( ε -C ) -∆ 2 2∆ , note that since B η ( r ) ⊆ M\ A , we have A ⊆ M\ B η ( r ) . Thus, we only need to prove the following:

when ε →∞ . One can easily show the following inequality,

By Theorem A.2, we have the following three cases:

K > 0 . Then the standard space M n K is the n -sphere of radius 1 / √ K . (Vol B ( η, 2 r ) -Vol B ( η, r )) is obviously less than the volume of the whole space M n K . Thus we have

where s ( n ) = 2 π n 2 Γ( n 2 ) is a constant relative to the dimension n . One can easily find that as ε →∞ , B η (2 r ) will cover the M n K , and the right-hand side of inequality 5 will approach to 0 as e -O ( ε 2 ) .

K = 0 . Then the standard space M n K is the n dimensional Euclidean space R n . We have

The same with above, when ε →∞ , B η (2 r ) will cover the R n and the right-hand side of 6 will approach to 0 as e -O ( ε 2 ) .

K < 0 . The standard space M n ( K ) is the hyperbolic n -space H n . The hyperbolic volume Vol B ( η K , r ) with any η k ∈ H n is given by

where the hyperbolic function is given by sinh( x ) = ( e x -e -x ) / 2 . It's not hard to see that

Plugging the 8 into 7, we have

Combining 9 and 4, we have

The principle part of the exponent of the right-hand side of 10 is still -ε 2 . Thus, when ε →∞ , it approaches to 0 as e -O ( ε 2 ) .

A.2 Theorem 3.2

Proof. Follows directly from Definition 2.2, Theorem 3.1 and Theorem 5 in Balle and Wang [2018].

A.3 Corollary 3.2.1

Proof. In this proof, we will parameterize points on S 1 using their polar angles. On S 1 , the Riemannian Gaussian distribution with footprint η and rate σ has the following density,

Note that since S 1 has constant curvature, we can use Theorem 4.1 instead of Theorem 3.2 WLOG we assume η 1 = 2 π -∆ 2 and η 2 = ∆ 2 and thus d ( η 1 , η 2 ) = ∆ . Given an arbitrary ε , the set A takes the following form,

and it follows that we must have

Thus we have

We have evaluated the first integral, and now let's consider the second integral. For ε ≤ ∆ 2 2 σ 2 , we have

Thus for ε ≤ ∆ 2 2 σ 2 we have,

Similarly, for ε > ∆ 2 2 σ 2 , we have,

Thus for ε > ∆ 2 2 σ 2 we have,

Put (11), (12) and (13) together with Theorem 4.1, we have proved Corollary 3.2.1.

A.4 Homogeneous Riemannian Manifolds

For more detailed treatment on homogenous Riemannian manifolds and related concepts, refers to Helgason [1962], Berestovskii and Nikonorov [2020], Lee [2006] for details and Chakraborty and Vemuri [2019] for a more concise summary.

A.4.1 Group actions on Manifolds

In this section, we will introduce some basic facts about group action which will be used to introduce homogeneous Riemannian manifolds in later section. The materials covered in this section can be found in any standard Abstract Algebra texts.

Definition A.1. A group ( G, · ) is a non-empty set G together with a binary operation · : G × G → G, ( a, b ) ↦→ a · b such that the following three axioms are satisfied:

Associativity : ∀ a, b, c ∈ G, ( a · b ) · c = a ( b · c )

Identity element : ∃ e ∈ G, ∀ a ∈ G,a · e = e · a = a .

Inverse element : ∀ a ∈ G, ∃ a -1 ∈ G,a · a -1 = a -1 · a = e .

Definition A.2. Let G be a group and X be an arbitrary set. A left group action is a map α : G × X → X , that satisfies the following axioms:

α ( e, x ) = x

α ( g, α ( h, x )) = α ( gh, x )

Note here we use the juxtaposition gh to denote the binary operation in the group. If we shorten α ( g, x ) by g · x , it's equivalent to say that e · x = x , and g · ( h · x ) = ( gh ) · x

Note each g ∈ G induces a map L g : X → X,x ↦→ g · x .

A.4.2 Homogeneous Riemannian manifolds, symmetric spaces and spaces of constant curvature

Let M be a Riemannian manifold and I ( M ) be the set of all isometries of M , that is, given g ∈ I ( M ) , d ( g · x, g · y ) = d ( x, y ) , for all x, y ∈ M . It is clear that I ( M ) forms a group, and thus, for a given g ∈ I ( M ) and x ∈ M , g · x ↦→ y , for some y ∈ M is a group action. We call I ( M ) the isometry group of M .

Consider o ∈ M , and let H = Stab( o ) = { h ∈ G | h · o = o } , that is, H is the Stabilizer of o ∈ M . Given g ∈ I ( M ) , its linear representation g ↦→ d x g in the tangent space T x M is called the isotropy representation and the linear group d x Stab( x ) is called the isotropy group at the point x .

We say that G acts transitively on M , iff, given x, y ∈ M , there exists a g ∈ M such that y = g · x .

Definition A.3 ([Helgason, 1962]) . Let G = I ( M ) act transitively on M and H = Stab( o ) , o ∈ M (called the "origin" of M ) be a subgroup of G . Then M is called a homogeneous Riemannian manifold and can be identified with the quotient space G/H under the diffeomorphic mapping gH ↦→ g · o, g ∈ G .

By definition, we have d ( x, y ) = d ( g · x, g · y ) for any g ∈ G and any x, y ∈ M . More importantly, any integrable function f : M→ R , we have [Helgason, 1962]

This property leads to Proposition 4.1.

Definition A.4 ([Helgason, 1962]) . A Riemannian symmetric space is a Riemannian manifold M such that for any x ∈ M , there exists s x ∈ G = I ( M ) such that s x · x = x and ds x | x = -I . S x is called symmetry at x .

That is, a Riemannian symmetric space is a Riemannian manifold M with the property that the geodesic reflection at any point is an isometry of M . Note that any Riemannian symmetric space is a homogeneous Riemannian manifold, but the converse is not true.

Definition A.5 ([Vinberg et al., 1993]) . A simply-connected homogeneous Riemannian manifold is said to be a space of constant curvature if its isotropy group (at each point) is the group of all orthogonal transformations with respect to some Euclidean metric.

Once again, a space of constant curvature is a symmetric space but the converse is not true.

A.5 Theorem 4.1

Proof. Let G be the isometry group of M . Let η 1 , η 2 ∈ M be arbitrary points such that d ( η 1 , η 2 ) = ∆ . By Corollary 4.1, the set A reduces to A = { y ∈ M : d ( η 2 , y ) 2 -d ( η 1 , y ) 2 ≥ 2 σ 2 ε } .

What we need to show is the following, for any points η ′ 1 , η ′ 2 ∈ M such that d ( η ′ 1 , η ′ 2 ) = ∆ ,

where A ′ = { y ∈ M : d ( η ′ 2 , y ) 2 -d ( η ′ 1 , y ) 2 ≥ 2 σ 2 ε } . It's sufficient to show

̸

̸

We can separate the proof into three cases: (1) η ′ 1 = η 1 , η ′ 2 = η 2 ; (2) η ′ 1 = η 1 , η ′ 2 = η 2 ; (3) η ′ 1 = η 1 , η ′ 2 = η 2 .

̸

̸

̸

It follows that η 2 is in the sphere centered at η with radius ∆ . (14) then follows from the rotational symmetry of the constant curvature spaces.

̸

Same as case (1).

̸

̸

̸

For any η ′ 1 = η 1 , there exists g ∈ G , such that g · η 1 = η ′ 1 . Denote η ′ 2 = g · η 2 , we have gA := { g · y : d ( η 2 , y ) 2 -d ( η 1 , y ) 2 ≥ 2 σ 2 ε } = { g · y : d ( η ′ 2 , g · y ) 2 -d ( η ′ 1 , g · y ) 2 ≥ 2 σ 2 ε } = { y : d ( η ′ 2 , y ) 2 -d ( η ′ 1 , y ) 2 ≥ 2 σ 2 ε } = A ′ . Let F ( y ) := p η 1 ,σ ( y ) 1 A ( y ) , we have ∫ A p η 1 ,σ ( y ) dν ( y ) = ∫ M F ◦ L -1 g ( y ) dν ( y ) = ∫ M p η 1 ,σ ( g -1 · y ) 1 gA ( y ) d ( L -1 g ) ∗ ν ( y ); change of variable formular, = ∫ M p η 1 ,σ ( g -1 · y ) 1 gA ( y ) dν ( y ); ν is a G -invariant measure, = 1 Z σ ∫ gA e -1 2 σ 2 d ( g -1 · y,η 1 ) 2 dν ( y ) = 1 Z σ ∫ gA e -1 2 σ 2 d (( gg -1 ) · y,g · η 1 ) 2 dν ( y ) = 1 Z σ ∫ gA e -1 2 σ 2 d ( y,η ′ 1 ) 2 dν ( y ) = ∫ gA p η ′ 1 ,σ ( y ) dν ( y ) = ∫ A ′ p η ′ 1 ,σ ( y ) dν ( y ) .

For ∫ A p η 2 ,σ ( y ) dν ( y ) , the proof is the same. Combine with the result of case (1), we have finished the proof for case (3).

A.6 Simulation Details

For sampling from Riemannian Gaussian distribution N M ( θ, σ 2 ) on S 1 (section 4.2), we first sample from truncated normal distribution with µ = 0 and σ 2 , then embed the sample to R and lastly counter-wise rotate the sample with degree θ .

For simulation on S 2 (section 5.2), we choose the pair of η and η ′ to be (1 , 0 , 0) and (cos(∆) , (1 -cos(∆) 2 ) 1 / 2 , 0) . Though any pair η, η ′ ∈ S 2 with d ( η, η ′ ) = ∆ works, we simply choose this specific pair for convenience. For Fréchet mean computation, we use a gradient descent algorithm described in Reimherr and Awan [2019].

A.6.1 R Codes

For simulations in section 4.2, refer to R files euclid_functions.R & euclid_simulation.R for Euclidean space and sphere_functions.R & s1_simulation.R for unit circle S 1 . For simulations in section 5.2, refer to R files sphere_functions.R & sphere_simulation.R.

Peer reviews:
comments: SUMMARY:
The paper extends definition of Gaussian DP to General Riemannian Manifolds and proposed Riemannian Gaussian mechanism that shows to achieve GaussianDP. Paper also develops MCMC procedure for Riemannian Manifolds with constant curvature.

STRENGTHS:
* Extending GaussianDP to Riemannian Manifolds
* MCMC procedure for Riemannian manifolds of constant curvature.

WEAKNESSES:
The experimental setup is extremely limited.

* **Baselines** The experiment compares the $\epsilon$-differential privacy mechanism (Riemannian Laplace mechanism) with the $(\epsilon, \delta)$-differential privacy mechanism (GDP mechanism), but it fails to include baselines that use $(\epsilon, \delta)$-mechanisms. It is necessary to include comparison with the following:

     *  DP-Riemannian Optimization ($\epsilon, \delta$)-differential privacy mechanism (source: https://arxiv.org/abs/2205.09494)
 
     Additionally, it is recommended to compare with 
    *  Riemannian K-Norm Gradient mechanism ($\epsilon$-DP) (source: https://arxiv.org/pdf/2209.12667.pdf).

*  **Experiments used only one manifold and of max dimension 3 for n=10 samples:** Experiments were solely carried out on 2-dimensional ($S^1$) and 3-dimensional ($S^2$) manifolds. It is essential to extend the experimentation to include at least one additional manifold, such as a hyperbolic manifold, which represents another constant curvature manifold. Furthermore, it is advisable to introduce variations in the parameters $n$ and $d$ during the experiments.

Related work : 
Also related work on DP-Optimization Improved Differentially Private Riemannian Optimization: Fast Sampling and Variance Reduction , TMLR 2023, (https://openreview.net/pdf?id=paguBNtqiO)

QUESTIONS:
NA

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the problem of achieving Gaussian Differential Privacy (GDP) on Riemannian manifolds.  After defining what this concept means, they provide a mechanism capable of achieving GDP via a generalization of the Gaussian to manifolds.  The major challenge in implementing the approach is in finding a suitable scaling parameter, $\\sigma$, for the noise.  The authors define several equations that must be solved to find $\\sigma$, each which become more tractable as additional structural assumptions on the manifold are made.\n\nSTRENGTHS:\nThis is a timely paper as there has been an increased interest in manifolds in statistics/machine learning.  There have also been several recent works that consider privacy within these contexts as well.  The Gaussian mechanism is one of the most popular tools for achieving privacy, so it is quite natural to consider it in the context of manifolds.  The extension of the Gaussian mechanism to manifolds is much more challenging than the Laplace.\n\nWEAKNESSES:\nImplementing the mechanism seems quite challenging.  Just to find the scaling parameter $\\sigma$ is difficult in general, plus one must do something like MCMC to actually sample from the mechanism.  This limits the practical impact of the paper.\n\nQUESTIONS:\nDef 3.1: I would emphasize that this is a known equivalence from the Dong et al paper.  I think the authors should also explain a bit more why this definition is easier to work with on manifolds than the original gaussian dp.  In general, I’m not convinced that it truly is.  The GDP definition is simply a statement about the tradeoff between type 1 and type 2 errors, it really has nothing to do with the underlying space.  So the central challenge isn’t defining GDP for manifolds, but defining a mechanism that achieves it. \n\n“In contrast to the task of releasing manifold-valued private summary, Han et al. [2022] considers a different scenario where the private summary resides on the tangent bundle of Riemannian manifolds and extends (ε, δ)-DP and its Gaussian mechanism to general manifolds.\"\nThis seems a bit inaccurate since their intention was not to privately release the gradient but rather use the private gradient for the private summary which resides on the manifold.  I think the wording here should be clarified.\n\nMinor— I don’t think the authors ever actually define $\\Phi$.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo obtain manifold-value statistic with the Gaussian differential privacy (GDP), the authors extend the GDP from Euclidean spaces to general Riemannian manifolds. Then they proved that their proposed Riemannian Gaussian distribution make GDP tractable. Furthermore, they proposed an efficient numerical algorithm to compute the privacy budget μ on the manifolds of one dimension, especially on the manifolds with constant curvature. At last, they provided some simulation experiments to illustrate performance of their algorithms.\n\nDue to the importance of GDP among the variants of DP, it is meaningful to generalize GDP to Riemannian manifolds. They solved this question partially. Considering the importance of this question, the paper is worth to be accepted. Also, this paper is organized clearly except some typos. I have checked carefully some important details of proofs, so the results should be reliable.\n\nSTRENGTHS:\n1.\tThey research the extension of an important variant of DP—GDP and prove that they can achieve GDP on Riemannian manifolds by using Riemannian Gaussian distribution.\n2.\tFollowing previous works, their results make sense.\n\nWEAKNESSES:\nSome typos influence the readability of this paper. For example, ‘compliant’, not ‘compliment’ in subsection Main Contributions; and $N_m(\\eta,\\sigma^2)$  is correct, not $N_m(\\mu,\\sigma^2)$.\n2. For the case of manifolds of dimension $d>1$, they make a too strong assumption that the manifold with constant curvature to design an efficient numerical algorithm. To the best of my knowledge, this assumption can’t encompass enough manifolds (I can only think of Euclidean space, the sphere and Lobachevskij space). The motivation of this assumption is not strong in practice.\n\nQUESTIONS:\n1.\tThis paper mentioned all hypothesis testing-based privacy definitions converge to the guarantees of GDP in the limit of composition and when the dimension of the privatized data approaches infinity, a large class of noise addition private mechanisms is shown to be asymptotically GDP. Are these results valid on Riemannian manifolds?\n2. \tIt is hard to solving the case of manifolds of dimension $d>1$, which make us use numerical methods to compute the integral. This work uses MCMC to approximate the integral, how about other numerical integration methods?\n  3.\tThey claimed that their work is the extension of an important variant of DP, so this work may be valuable in practice. Thus they should explain the point clearly in the introduction and provide some real world numerical examples.\n\nLIMITATIONS:\nSee \"weakness\".", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies an extension of a Gaussian privacy mechanism on Riemannian manifolds, so as to extend the notion of mu-GDP for estimators with given global sensitivity. The idea is to sample a random point on the manifold with a Gibbs density (wrt to the Riemannian volume) whose potential is proportional to the squared distance to the given estimator (that would simply be a Gaussian density on a Euclidean space). It is well known that in order for such a distribution to be well defined, it is enough to assume that the sectional (or, equivalenty, Ricci) curvature of the manifold is bounded from below. The main challenges, then, reside in : (1) tuning the temperature parameter of the Gibbs distribution so as to obtain mu-GDP, and (2) sample from such a distribution. \n\n\nWhile giving general formulae for the choice of the temperature parameter, the authors study the case of homogeneous manifolds, as well as model manifolds with constant curvature, also providing simulation studies. \n\nDifferential privacy on non-linear spaces, in particular Riemannian manifolds, has not been studied a lot yet, and this paper aims at providing new, simple insights.\n\nSTRENGTHS:\n•\tThe question tackled in this paper is important, given the increasing need of dealing with non-linearity in data. \n•\tThe paper is written clearly and the main text is easy to follow.\n\nWEAKNESSES:\n•\tWhile the paper addresses an important question, the execution remains shallow. Laplace and Gaussian mechanisms can be very easily extended to Riemannian manifolds, and the computations that yield to determine the calibration of their parameters are very similar to the Euclidean case. One challenge that is barely addressed here (see the very brief remark just before Definition 4.1) is the understanding of the connexion between these parameters and the geometry (in particular, curvature bounds) of the ambient manifold. Instead, the authors choose to focus on very simple (and perhaps not realistic in most applications) manifolds, where everything becomes much simpler. Hence, I think that a big, if not essential, part of the picture is missing in this work.\n•\tThe presence of Definition 3.1 (mu-GDP mechanisms on manifolds) is obscure to me: A definition is already given in Definition 2.2, and I do not see why it should be different on manifolds. Should Definition 3.1 actually be a theorem? I mean that one should check that M is mu-GDP in the sense of Definition 2.2 if and only if it is (eps,delta(eps))-DP for all eps>0, with delta(eps) given by the formula that appears in Definition 3.1 (something similar to Theorem A.1).\n•\tSome proofs seem either wrong or incomplete to me, here are some pointers:\n\t\t- One line after Theorem A.1, this theorem is interpreted using the O(eps^2) notation, which completely forgets the dependence in mu. Hence, the whole proof of Theorem 3.1 forgets mu and is, therefore, imprecise/incomplete. \n\t\t- I did not understand the first line of the proof of Theorem 3.1. Also, there, what is x (it does not appear in the formula)?\n\t\t- In Equation 4, I think that the normalizing constant Z_{eta,sigma} is missing in the upper bound. \n\n\nSome minor remarks: \n•\tLine 92, there is a typo.\n•\tLine 99: « the » should be replaced with « a ».\n•\tLine 237: Remove « satisfies »\n•\tEvery citation should come to a pointer to a specific definition or result, especially when the citation refers to a whole book (e.g., Dudley, 2002)! \n•\tIn Theorem A.1, what is the ‘t’ in the subscript of mu_t?\n•\tAlso in Theorem A.1, « privacy profile » has not be defined anywhere.\n\nQUESTIONS:\nSee paragraph on weaknesses.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-05"}