Differentially Private Fréchet Mean on the Manifold of Symmetric Positive Definite (SPD) Matrices with log-Euclidean Metric
Authors: Saiteja Utpala
Praneeth Vepakomma
Nina Miolane
Venue: Accepted by TMLR ()

Abstract: Differential privacy has become crucial in the real-world deployment of statistical and machine learning algorithms with rigorous privacy guarantees. The earliest statistical queries, for which differential privacy mechanisms have been developed, were for the release of the sample mean. In Geometric Statistics, the sample Fréchet mean represents one of the most fundamental statistical summaries, as it generalizes the sample mean for data belonging to nonlinear manifolds. In that spirit, the only geometric statistical query for which a differential privacy mechanism has been developed, so far, is for the release of the sample Fréchet mean: the \emph{Riemannian Laplace mechanism} was recently proposed to privatize the Fréchet mean on complete Riemannian manifolds. In many fields, the manifold of Symmetric Positive Definite (SPD) matrices is used to model data spaces, including in medical imaging where privacy requirements are key. We propose a novel, simple and fast mechanism - the \emph{tangent Gaussian mechanism} - to compute a differentially private Fréchet mean on the SPD manifold endowed with the log-Euclidean Riemannian metric.  We show that our new mechanism has significantly better utility and is computationally efficient --- as confirmed by extensive experiments.

Published in Transactions on Machine Learning Research (02/2023)
Differentially Private Fréchet Mean on the Manifold of Sym- metric Positive Definite (SPD) Matrices with log-Euclidean Metric
Saiteja Utpala
muitejauupall@gmmacccco com
UC Santa Barbara
Pranceth Vepakomma
vepakomn&@mii edu
MIT
Nina Miolane UC Santa Barbara
winamiolane@uuss.... edu
Reviewed on OpenReview: https: // openreview. net/ forum? id= mAz80q214f
Abstract
Differential privacy has become crucial in the real-world deployment of statistical and ma- chine learming algorithms with rigorous privacy guarantocs. The carlicst statistical querics, for which differential privacy mechanisms have been developed, were for the release of the sample mean. In Geometrie Statisties, the sample Fréchet mean represents one of the most fundamental statistical summaries, as it generalizes the sample mcan for data. belonging to nonlinear manifolds. In that spirit, the only geometric statistical query for which differen- tial privacy mechanism has been developed, so far, is for the release of the sample Fréchet mean: the Riemannian Laplace mechanism was recently proposed to privatize the Fréchet mcan on complete Ricmannian manifolds. In many fields, the manifold of Symmetric Pos- itive Definite (SPD) matrices used to model data spaces, including where privacy requirements are key. We propose novel, simple and fast mechanism the langent Gaussian mechanism to compute differentially private Fréchet mean on the SPD manifold endowed with the Bog-Euclidean Riemannian metric. We show that mechanism has significantly better utility and is computationally efficient as confirmed by extensive experiments.
medical imaging
1 Introduction
Privacy-preserving computing is an active area of research which is necessitated by ethics, regulations, re- quirements for protections of trade secrets, or possible lack of trust amongst distributed data siloes. Privacy preservation is desired across several topologies of data sharing, be it from client devices to powerful central- ized entities or a in peer-to-peer fashion. Mistrust. in data sharing carries over not only the sharing of raw data but also in the sharing of results obtained from intermediate or complete computations. The ncod for stringent privacy protections often fueled by many privacy leakages and atlacks that continue to happen under various settings operating without the right level of
mechanisms.
In this context, differential privacy (DP) Dwoor et al. 2006 Dwork 2008 Dwork et al. 2014 Dwork 2006) has emerged as one of the leading mathematical definitions to ensure the preservation of privacy up to a chosen level. Privacy-preservie subsequently developed to privatize a wide range statistical and machine learning computations. The earliest queries, for which mechanisms have been proposed, were for the privatization of sample means in statistics, computed for data lying linear spaces. When data belong to nonlinear manifolds, the
mechonisms that satisfy the definition of differential privacy were

---

Published in Transactions on Machine Learning Research (02/2023)
Fréchet mean query (Fréchet 1948) is the foundational building block of geometric statistics that needs to be privatized. Our work proposes a new, simpler and faster, mechanism for private Fréchet means on the manifold of symmetric positive definite (SPD) matrices endowed with log-Euclidean metric.
1.1 Motivation
Fréchet mean: a building block in geometric statistics While traditional statistics studies data that lies on linear spaces, geometric statistics studies data that lies nonlinear spaces such as Riemannian man- ifolds, affine connection spaces, or stratified spaces (Pennec et al. 2019) Miolane 2016). Such analysis is fruitful as data might have inherent constraints that are well captured by the goometry of a nonlinear space Miolane al.] p: Myers et al. (2022). ). For instance, symmetric matrices constrained to have strictly positive eigenvalues are conveniently modeled as elements of the manifold of symmetric positive definite (SPD) matrices. Several extensions of traditional statistical analysis tools have thus been developed for the
manifold setting: regression has been generalized to geodesic regression (Fletcher) 2011 Thomas Fletcher 2013). principal component analysis (PCA) to principal geodesic analysis or geodesic PCA (Fletcher et al.
2004 Sommer et al. 2010: Huckemann et al. 2010), and mean shift to Riemannian mean shift clustering
Subbarao & Meer 2009; Caseiro et al.) 2012). In each of these algorithms, the computation of the sample Frechet mean generalizes the computation of the sample mean, and thus represents the most fundamental building block. The privatization of the Fréchet mcan is therefore the key clement required to privatize geometric statistical queries. Privacy-preserving geometric statistics is also crucial, as one of its main appli-
cation areas is medical imaging and computational anatomy (Pennec al. 2019 Miolane 2016 for which privacy requirements are often desirable.
Importance of the SPD manifold with log-Euclidean metric Symmetric positive definite (SPD) matrices model wide range of data, from medical images with Diffusion Tensor Imaging (DTT) (Basser et al. Pennec et al. 2006), to physiological signals with electronncephaloggrapyy (EEG) signals from brain-computer
interfaces (Yger et al.] 2016 Zanini et al.) 2017 Chevallier et al.] 2021). to 3D shapes (Tabia et al., 2014) to name lew. Given their central roles for medical data where privacy is of the utmost. importance [Lotan et al. 2020 Li et al. 2005), private statistical computations on the SPD manifold are a worthy endeavour. The SPD manifold can be equipped with different Riemannian metrics that provide elementary operations such as distance computations. The log- Eug-ddeiee metric, originally proposed in (Arsigny et al. 2006), has numerous advantages over another popular Riemannian metric called the affine invariant metric (Pennec et al.) 2006): (a) it is computationally faster, (b) it gives similar or better performances on several processing and learning tasks, (c) and quite importantly, it provides closed form expression for the Fréchet mean which otherwise requires solving an optimization problem.
Despite its importance for the processing of number Need for better and faster privacy mechanisms
of (medical) data, geometric stat isties currently stands understudied from the lens of differential privacy. The very recent work by (Reimherr et al. 2021) provides the first differentially private mechanism for the Fréchet mean. However, its utility- a measure of the mechanism's 's deviation from non-privatized computations makes impracticable on the manifold of SPD matrices as soon as we consider matrices of moderate size, e.g. 20 20 matrices. Consequently, there is need for better and faster privacy mechanisms on manifolds, starting with the SPD manifold.
1.2 Related Work and Contributions
Reinherr et al. 22021) were first to consider differential privacy in manifold setting and developed Riemanmian
Laplace mechanism by extending the standard Laplace mechanism (Dwork et al.) 2014) for linear spaces complete Ricmannian manifolds. It is based on Laplace distribution that originally proposed for SPD
matrices Hajri et al. 2016) based on distance of the affine invariant metric (Pennec et al. 2006), which they generalize to any manifold M equipped with a distance
p(x) exp (- [pll,.) Vr E M
(1)
2

---

Published in Transactions on Machine Learning Research (02/2023)
where E M, a E Ryo(positive reals) are parameters of the probability density p. Reimherr et al. (2021) show that the mechanism obtained achieves pure differential privacy and provides an upper bound for the expectation of its utility (a measure of the deviation from non-privatized computations) for the Fréchet mean query. Their method is applicable to various Riemannian manifolds that satisfy some regularity conditions.
Approximate differential privacy relaxes pure differential privacy (see Section 2 but provides significantly better utility for higher dimensions and is heavily used in real world applications Abadi al. (2016). In the Euclidean case, the Gaussian mechanism, where noise is added from standard Gaussian, satisfies approximate differential privacy. To this end, we make use of log Gaussian distribution Schwartzman 2016), an intrinsic distribution on SPD matrices, for deriving approximate differentially private mechanism. This relaxation helps us obtain better utility compared to Riemannian Laplace mechanism in terms of dimension, similar to standard Euclidean case. We summarize our contributions are as follows.
Mechanism A
DP
(DD))( A(D))
Theoretical Results
Riemannian Laplace (Reimherr et al. 2021 tangent Gaussian (Ours)
Pure DP
O(k")
Expectation of (²([(D), A(D)
Approx. DP
((I((1/5)A2)
Exact Distribution of p²(J(D), A(D))
Differences between existing [Reimherr et al. 2021 | and proposed mechanisms for private Fréchet mean queries on the manifold of k x k SPD matrices endowed with the log- Euclidean metric. The notation P(((D)) A (D)) represents the utility with D the dataset, A the mechanism under consideration, the log- Euclidean distance, f the Fréchet mean and 8 quantifies approximate differential privacy.
Table 1:
1. We propose a new and simple mechanism called the langent Gaussion Mechanism that privatizes any statistical summary on the manifold of Symmetric Positive Definite (SPD) matrices endowed with the log-Euclidcan metric. We prove that it achieves approximate differential privacy (Th. 2)
2. When the statistical summary is the Fréchet mean, we show that our mechanism obtains significant improvement in terms of utility over recent works which we demonstrate theoretically, and practically for data in higher dimensions. Further, our mechanism is computationally efficient and easily implementable.
3. We present the effectiveness of our mechanism on synthetic and real-world (medical) imaging data, the latter being represented via their covariance descriptors. To this aim, we also prove a theoretical bound on the radius of log- Euclidienn geodesic ball with the covariance descriptor pipeline (Tuzel et al. 2006 required for the applicability of our mechanism (Th.
Table highlights the technical differences between Reimherr el al. 2021 and our work.
2 Preliminaries and Notations
Elements of Riemannian Geometry Let M be d-dimensional smooth connected manifold and T.M be its langent space at point p E M. A Riemammian metric g on M is a collection of inner products 9p T.MM x T.M + R that vary smoothly with p. A manifold M equipped with a Riemannian metric g is called a Riemannian manifold. Importantly, the metric 9 gives a distance p on M. Let y [0, 1] + M be a smooth parametrized curve on M with velocity vector at t denoted as t E Ty The length of y is defined as Ly fo V V9 dtt and the distance p between any two points p.g E M is: p(p.g) pp Lyp
If in addition M is complete for p. then any two points p.g E M can be joined by length curve.
called a geodesic. We refer the reader to (Do Carmo & Flaherty Francis 1992 for a detailed exposition.
Lee
2006:
Hel Helgason
1979
Elements of Differential Privacy (DP) Let X be an input data space and M the manifold under consideration. Let f X + M be a manifold-valued statistical summary that requires privatization with respect to some sensitive dataset D of size n, Lee. D E x Two datasets D, D' E n are said to be adjacent
3

---

Published in Transactions on Machine Learning Research (02/2023)
if they differ by at most one data point. We denote adjacency as D ~ D'. The sensitivity of the summary f with respect to the distance p M is defined as:
= sup P(f(D), D-D'
(2)
which is the maximum amount of deviation that can occur in the output of f for adjacent datasets.
A mechamism A X M is randomized algorithm that takes a dataset D as input, and outputs a privatized version of the summary on D. The mechanism A satisfies (e, differential privacy (also pure differential privacy) if, for all adjacent datasets D D' and for all measurable sets S of M the following holds:
(D( S] S (xp(A) E
The intuition is that the change of single element of the data space X' does not. significantly alter the output distribution of the mechanism. As relaxation, the mechanism A satisfies (e, d)-differential privacy (also approzimate differential privacy) if, for all adjacont datasets ~ D' and for all measurable sets 5 of M:
P[A(D) E S) S exp ((A(()) E S) 6.
Intuitively, 8 can be thought of as the probability of privacy failure, when Eq. equation 3 is not guaranteed.
Let PA(D) be the density of the random variable Y A(D). Given adjacent datasets D D', the privacy loss function of A is defined as
(A...D = In (V) Ay E E
(4)
('AD))()))
and the privacy loss random variable LA,D,D' (A. (Balle & Wang 2018). Importantly for our derivations, both sufficient and sufficient & necessary conditions for the mechanism A to be (c, differentially private (DP) can be formulated in terms of LA,,,D'. The sufficient condition writes VD ~
D' I.LLA,D,D' 2 c < + A 5D- The sufficient & necessary condition is: VD ~ D': P'L.,L 2 c) - exp (c) (L),(L,, v -c] < 5 A O)-DP.
Fréchet Mean When the data space X' is equal to the manifold M, we will be interested in mechanisms that can privatize a specific statistical summary f called the Fréchet mean. The sample Fréchot mean X Fréchet 1948) of the dataset D (XI,...X,) }n) the manifold M is defined as
& plp E arg min Σ (g.
9EM
dee. we have in this case X f(D) for D M Intuitively, the Fréchet mean uses a property of the mean on linear spaces namely the fact that mean minimizes the sum of squared distances to the data points definition of mean on manifolds. Crucially, the Fréchet mean depends on the distance p and therefore M. We also note that the Fréchet mean might not always exist, and if
the Riemannian metric defined it exists it might not be unique supplementary materials. In practice, computing X generally requires optimization algorithms such as gradient descent on manifolds (Boumal 2020
3 Geometry of the SPD Manifold with Log Euclidean Metric
Manifold and vector space structures We now restrict M to be the manifold of symmetric positive definite (SPD) matrices:
SPD(K) = (x E 1 x X and Vu E \ (0) u Xu 0}.
(5)
which has dimension M(k+1) 2
The tangent space of the manifold SPD(K) at any point X E SPD(k) the vector space of symmetric matrices SYM(K). The mathematical construct (SPD(k), +,.) is not vector space
4

---

Published in Transactions on Machine Learning Research (02/2023)
under element-wist addition and element wise scalar multiplication. This can be seen from the observation that E (Rgo, X E (PD( = aX & SPD(K). Instead, SPD(k) is an open cone of Tg and, as such, naturally possesses smooth manifold structure which can further be equipped with different Riemannian metrics (Thanwerdas Pennec 2021 However, Arsigny et al. (Arsigny et al., 2007) showed in a surprising result that SPD(k) can be given a vector space structure (SPD(k), @ () via the operations B, @ defined Table 2 where Expm, Logm denote the matrix exponential and matrix logarithm. This fact is central for the proofs provided in the present paper.
OPERATION
NOTATION
EXPRESSION
Addition
X1 + X2
Expm Logm X₁ Logm X₂]
Subtraction
X1 θ X2
Expm Logm X1 Logm X2)
Scalar Multiplication
a a X
Expm (a. Logm
Table 2: Operations turning the manifold SPD(k) into a vector space. Expm and Logm denote the matrix exponential and logarithms, respectively. X2 belong to SPD(k) while a is a scalar.
Riemannian structure Arsigny et al. further define Riemannian metric on SPD(k), called the log- Euclidean metric, which induces the following distance:
PLE(X1, X2) = |[Logın X1 - Logm Xalla X2 SPD()),,
(6)
where F denotes the Frobenius norm matrices. Importantly, the log- Englidenn metric (Ariigy et al. 2006) gives a unique and simple closed form expression for the Fréchet mean terms matrix logarithm and matrix exponential
XLE = Expm
Logm X
1=1
for the dalaset XI,... X. E SPD(k).
Maps between spaces Lastly, we present maps that will help us define the differential privacy mecha- nism proposed in the next. section. Consider the map veed SYM(K) R ki+e [diag(XX) (X)", uppprrdaag(X)) where diag SYM(K) + R* and upperdiag SYM(K) + R 2 build vectors from the diagonal, and from the strictly upper diagonal entries, of the matrix X. The map veed is invertible and we denote by invveed its inverse. Specifically, the spaces SPD(K), and R KEID related follows:
-RR- defined veod(X)
are
Logm
veed
SPD(k)
SYM(A)
Expm
invveed
4 Tangent Gaussian Mechanism on SPD manifolds
We can now introduce our differential privacy mechanism for statistical summaries on the SPD(k) manifold. Let ; x + SPD(k) be any SPD(k)-valued summary that needs to be privatized. The proposed mechanism is based on the log Gaussian distribution on the SPD manifold (Schwartzman) 20166) which defined as follows. Consider mean M SPD(k) and tangent covariance Σ E SPD
k(++1) We can (i) first map the mean M to the tangent space SYM(K) of SPD(K) at the identity using the matrix Logarithm Logm, then
(ii) to IR +(++)) using the map veed introduced in the previous section, and (iii) consider whether the result follows a traditional Gaussian distribution.
Definition 1 (Log Gaussian Distribution on SPD(k) (Schwartzman 2016 Given a mean M E SPD(k), and a langent covariance E E SPD A(k+1) we say that X ~ LN(M,E) follows log Gaussion distribution
5

---

Published in Transactions on Machine Learning Research (02/2023)
Algorithm 1: tangent Gaussian Mechanism for f x SPD(k)
Inputs Dataset D of k x k SPD matricos of size n, sigma-type E ('classical', 'analytic' }. ALE the log- Eeg-Euclidean sensitivity of f, > 0, ò (0, 1) and additionally < 1 if sigma-type is "classical', the noise calibration subroutines CLASSIC, ANALYTIC which take ALE, c, and provide a.
Output: Private f(D)
if sigma-lype is 'classical' then CLASSIC(ALR, 6,0;; else ANALYTIC(A1n;s,0) Compute non private output fup := f(D)
Compute mean of Gaussian distribution: M := vecd fupl. M R "(4+1)
Sample from the Gaussian distribution in R 2 N (M
M0++)
Map sample to the SPD manifold: fp == Expm(invveed N] Return private fp
SPD(k) if veed[Logm X] [ovecd[Logm MJ.E) E) follows (regular) Gaussian distribution with mean veed Logm M and covariance malris Σ on R k(++))
The density p(X|M,2) 2 is then given by
J(X)
exp (-2 veod(L.ggm X - Logm voe X
voed
Logm Logm
(2m) # (det Σ
A; > X3 where d k(k+1) J(X) det Iles h()s;)g); and h()i,))) = (log Ai - -lgg
with Xi
2
eigencalues of the matrix X.
The definition of log Gaussian distribution on the SPD(k) manifold allows us to define our proposed tangent
Gaussian mechanism.
Definition (tangent Gaussian Mechanism). Consider any statistical summary f X SPD(k) on the manifold SPD(K) equipped with og-Fuclidean metric. Given o² > O, we define the tangent Gaussian mechanism ArG: x + SPD(k), as
ArG(D)= = X, where X LN
We now state main theorem, which shows that the privacy loss of the tangent Gaussian mechanism is normally distributed with mean and variance parametrized by the log- Euglidenn distance. Proof is given in Appendix A.2
Theorem (Distribution of Privacy Loss for the tangent. Gaussian Mechanism). Let Ara be a langent Gaussian mechanism with voriance o². Its privacy loss is normally distributed as
LAT. DA (DD fi
(
202
d²
This distribution is analogous to the distribution of the privacy loss for the Euclidean Gaussian mechanism. but with the log-Euclidean sensitivity instead of the Euclidean sensilivity (Dwork el. al. 2014 Balle &
Wang 2018). Consequently, our theoretical analysis of the tangent Gaussian mechanism deriving privacy guarantees from the distribution of the privacy loss above closely follows the steps of the analysis for the
Euclidean Gaussian case. Specifically, we can proceed in two ways with either a (1) classical approach where
sufficient conditions are used to show the mechanism (c, 5)DD as Dwork et. al. (2014). or with an (2) analytic approach where the utility is better by using sufficient and necessary conditions (Balle Wang 2018).
Theorem 2 (Privacy Guarantee of tangent Gaussian Mechanism). Consider Xm - SPD(k) with log- Euclidean sensilivily ALE-

---

Published in Transactions on Machine Learning Research (02/2023)
1. (Classical) Given e, (0,1), choosing ALEV / 2lm(1.25/0)ee, makes the tangent Goussian mechanism (c, -)dffferrenll privale.
(Analytic) Given e 2 0,8 (0.1) and & the cumulative distribution of the standard Gaussian, choosing any that satisfies ( Aig ca - -ex((())) mechanism (c, diffferrrellll private.
Aur
20 Din makes the tangent. Gaussian
Proofs are given in Appendix
A.3 Algorithm 1 shows the implementation of the mechanism.
5 Privatizing the Fréchet mean
In the previous section, f is any function that outputs summarr statistics on SPD(k). In this section, we seek to privatize the Fréchet mean of the log- Euclidean metric. We first compute its sensitivity and then provide its utility. In what follows, (MM (M), r) denotes an open geodesic ball of radius O<r do centered at M E SPD(K).
Theorem (Sensitivity of Log- Eug-Euciieean Fréchet Mean). Given data in (MM for some 0 < 0o and M E SPD( the sensitivity of the log- ogg-Eucide Fréchet mean verifies: ALE S 2r
Note above theorem also obtained from Reimhher et al. 2021 Theorem 2) by setting = 0. The utility of the tangent Gaussian mechanism for a Frechet mean query is then given below.
Theorem (Utility). Let ATG be the (classical) langent Gaussian mechanism, (M) geodesic ball of radius 0 00 and center M M containing the dataset D and f the Fréchet mean. The utility of the mechanism ATG is given by:
(I)) ATG()) ~ 0²xa-
ElPLE( (TG( 4r 1n(I11111 In(1.25/0)d
with d dim((PD(k)) k(k +
n2e2
where Xa represents the chi squared distribution with d degree of freedoms.
Proofs of Th. 3 and Th. are given in Appendix A.4 We compare these results with those of the Riemannian Laplace mechanism (Reimherr et al. 2021), denoted ARL-
Utility: We compare the utility in terms of size k of spd matrices k l because dependancy other factors n, are same. Utility of the Riemannian Laplace mechanism has an expectation = O(k4). By contrast, our tangent Gaussian mechanism provides
given by (fPL) ATG (D)) = (ln(11/))33) Hence our mechanism has significantly better utility in terms of dimension.
Pure DP vs Approx DP: It should noted that our privacy guarantees are weaker than Riemannian Laplace. In practice, ö is chosen be oryplographically small and typically < //nomee (2021).
Thcoretical Results: The authors of Reimherr et al. (2021) characterize the utility in terms of its expecta- tion (f A ADD)). By contrast, our results yield more complete picture, we derive the probability distribution of (f (D))) given that we are tailoring mechanism for flat geometry of SPD matrices with log- Euclidean metric.
6 Experiments
We use the Riemannian Laplace mechanism as the baseline and recall that this mechanism uses the Rieman- nian Laplace distribution equation 25 Efficient sampling from the Riemannian Laplace distribution is only
discussed for (i) SPDManifold with affine-nvvarian metric and (ii) Hypersphere with Euclidean metric in Reimherr et al. (2021) and we didn't find any sampling procedure from this distribution on SPD manifold with log- Euclidean metric in Reimherr et al. (2021): Hajri et al. (2016) and hence we used MCMC sampling in our experiments.

---

Published in Transactions on Machine Learning Research (02/2023)
0.4
Rie-Laplace
Rie-Laplace (anG-ClassicalOOurss
17.5
Rie-Laplace
Rie-Laplace
50
TanG-Classical(Ourss
TanG-Classical(Uuns)
Tanc-Clasikal(Ouus)
TanG-Analytcc(Ours)
TanG-Analytic(Ours)
TanG-Analyti((Ours)
TanG-AnalyyicCOur))
40
12.5
10
30
10.0
Pue
7.5
20
10
5.0
10
5
2.5
0.0
10
20
30
0.1
0.2
0.3
0.4
4.0
2.00
fanG-Clate 1101) 10-1)
TanG-Ciela 10-1)
2.5
TanG-Clata - 10"))
Fand-Clats- 10-1)
T 10
2.5
TAnG-Anw08 3077)
Tanf-Anuld 16
1.75
TanGvinets 10
1o-
Tans Gatd 16-")
Tans 1b
TanG 100
Anats-nats-1
A 20
TanG-Anuld - 10-91
1.50
Tand-Anats 10-9)
TAnt-Clel8 10-1)
TanG-Chel--1 10-9
1.25
TanG-cle8--10
An
30-1)
AanG 11-1
Anais
1.5
0.75
1.0
0.50
0.5
0.5
0.25
0.00
30
10
ExtG-Analytic TanG Analytia
ExtG-Analyyic Analltn
ExtG-Analytic
ExtG-Analytic
Analytnalytic
Taal--nalytic
Pelftz),
10 20 30 10 20
30
10 20
30
10
20
30
Figure 1: Utilities synthetic data for Rie-Lap the Riemannian Laplace mechanism Reiiheerr et al. 2021b, and TanG Classical, TanG-Cla and TanG Analylic, TanG-Ana our proposed langent Gaussian mech- anisms (classical and analytic versions), and Entl-Analytic the Extrinsic analytic gaussian mechanism for different matrix sizes k and privacy parameter e. PLE and PE denotes log- EggEuclidean and Euclidean distance respectively. Note output of extrinsic mechanism is not a SPD matrix and hence deviation measured in standard Fuclidean distance.
6.1 Experiments on Synthetic Datasets
The utility depends on privacy parameters (c, the size of the matrices, the dataset size n and the radius of the geodesic ball containing the datasct. The utilitics of the tangent Gaussian and Ricmannian Laplace mechanisms have the same dependency on n, r, such that their differentiating parameters k. Consequently, our experiments synthetic data fix n, and vary k.
We also consider Extrinsic approach suggested in Reimherr et al. 2002h where Fréchet mean is seen to be belonging to Symmetric matrix and noise from Euclidean normal distribution is added, specifically AEx (D) X, ~ invvecd (N (veed XLE, o²1)) for appropriate a. If is radius of log- Euclidean geodesic ball of data, extrinsic sensitivity is given by Agx = 2(exp (r) 1)/n (Reimherr et al. 2021 Proposition 1). It should be emphasized that resultant privatized Fréchet mean is no longer a SPD matriz. Hence Reinherr al. (2021) compared deviation between private and non private Fréchet mean in the standard Euclidean norm.
8

---

Published in Transactions on Machine Learning Research (02/2023)
We generate random & x k SPD matrices follows: (i) generate k real values (A1...., A6) uniformly in (e)", (ii) build D the diagonal matrix with Dú: = Xs for i E (1, k)), (iii) generate a k x k random orthogonal matrix E' with the Haar distribution, and (iv) build the SPD matrix as: X = EDE" This process generates SPD matrices, that can be shown to belong to the geodesic ball B B (T) with I the identity matrix:
[Logmm X 1 = E (In (m & vkr² vlr. We use n = 500 and r = 1/4 in our experiments and hence ALE S VK/1000.
Fig. [ (first) compares utilities using a fixed = 10-6 for our mechanism, MCMC burn-in of 50, 000 for the Riemannian Laplace mechanism, and different values of k E {2, 5, 10, 15, 25, 30) and (0.1, 0.2,0..0003... 0.3, 0.4). Each experiment is repeated 10 times, the results are averaged and the band (u- 20, u 2x) is shown, where I and are the mcan and standard deviation, respectively, of the associated result. The is small for our mechanism, and does not appear on the plots. The langent Gaussian mechanisms (ours) yield almost x10 utility improvement for larger k, for each e. Fig. [ (middle) shows that, as expected, our utility is not significantly impacted by different values of 8 E {10- 10-8, 10- Fig. (bottom) compares utilities between Extrinsic Gaussian mechanism (analytic) and tangent Gaussian mechanism (analytic) in Euclidean distance and shows proposed mechanism is better.
6.2
Experiments on Real-World Datasets
We run experiments on covariance descriptors of real-world images. Covariance descriptors (Tuzel et al.
2006
have been widely used for face and person recognition (Tuzel et al.
2007
Zhang & Li 2011
Pang
ct al. 2008 Križaj et al. 2013 Ma et al.] 2014 Cai et al.] 2010 Zeng et al.
2015
Matsukawa et al. 2016).
action and gesture recognition Cirujeda & Binefa 2014 Hussein et al.] 2013 Sharal et al. 2015). 3D shape
analysis (Tabia et al. 2014 Ma et al. 20044) medical imaging (Khan et al.
2015 Cirujeda et al. 2016) ; and
even recently as layers in neural networks Yu & Salzmann 2017) which makes them interesting data to privatize.
Let I Rxxuxe be image of height h, width and with channels, where is for gray scale images and 3 for RGB images. Let & Rxxuxe > Rhwxk be feature extractor of dimension k, Lee. ((I)(x) is a k-dimensional vector at each spatial coordinate x in the image's domain S. Given a small > 0, the covoriance descriptor R,, Rhxwxx + (P)(k) associated with & is defined as
1 Σ (o(Z)(x) - ()((()( w)² S
(Z) =
+ +n.I,
XES
where μ |SI-1 Σ Exss (II (x), and 17.l ensures R (I) E SPD(k) with 17 usually set to 10-6, Our experiments
follow Tuzel et al. 2006
Jayasunana et al. 2015 and use the covariance descriptors associated with the feature vector given as (x)) = y, I, (Zzl Izl Iaal Ianl V
where x x= (x. v), intensitics derivatives are denoted by I., Iy. Izz, Ing and we addod the intensity values I for cach channel compared to (Tuzel et al., 2006 Jayasumana et al. 2015). For gray scale images, (xI) is a 9- dimensional vector that makes R. (Z) a 9 9 SPD matrix, while for RGB images (xI)(x) 11-dimensional vector that makes (I) 11 11 SPD matrix. We are within the assumptions of Th. since such covariance descriptors belong to geodesic balls centered at I, as shown by the following theorem.
arctan
[Z
74
Theorem 5. Let (I) be the covariance descriptor associated with the feature vector ((I) above.
1. I TI is gray scale image, then [|Logm R, (T)lkp S /9 max (Iln [In[14++)))
2. I is RGB image, then [[Logm (I) S VII max (Ilnn), [In(16 + +)
Proof is given in Appenidx A.5
6.2.1 Experiments on medical imaging data
We use images from classes of the medical imaging datasets PATHMNIST (gray scale) and OctoMNIST (RGB) from MedMNNSTV2 (Yang et al. 2021), compute the 4 class-wise Fréchet means of their covariance
9

---

Published in Transactions on Machine Learning Research (02/2023)
10704
10356
N 1669
N 11035
Laplace
Rie Laplace
14
Laplace
Laplace
12
TanG 10-3)
TanG 10-9) 9)
TanG(6 10-9)
TanG 10-3)
PuelMD) AID)
10
TanG 10-1)
12
T 10-1)
12
TanGis 10-²)
TanG(o 100))
TanG 10 9)
30
TanGio- +
10
TanGio 10 "
TanGt5 10 $)
4
N 33734
N = 10463
N = 80D4
N = 46276
3.5
Rie Laplace
Rie Laplace
Rie Laplace
Rie Laplace
3.0
TanG - 10-1)
TanG 10-2)
10
TanG(8 - 10-9)
TanG - 10-2)
2.5
TanG - 10-1)
TanG 10-1)
TanG(6 10-1)
TanG 10-1)
TanG - 10 ?)
TanG 10"*)
TanGle 10 ?
155
TanG(o 10 ?)
3.5
3.0
2
0.5
3.5
0.2
0.6
0.5
2.5
0.2 0.4 0.6
0.8
0.2 0.4
0.5
0.8
0.2
0.4
0.5
0.8
Figure 2: Utilities on the private Fréchet means for different privacy parameters 6, and real-world datasets of sizes N. Top: PathMNIST (RGB images yiclding 11 x 11 SPD descriptors). Bottom: OctoMNIST (gray scale images yielding 9 x SPD matrices). Rie- Luppace is the Riemannian Laplacian mechanism (Reimherr let al.) ) and TanG the tangent Gaussian mechanism for different values of (ours). We also show the (n- 20, 1 20) band.
descriptors (17 = 10-5), which we privatize using the Riemannian Laplace and tangent Gaussian (analytical) mechanisms. We avoid using extrinsic approach because extrinsic sensitivity is extremely high Fig. shows the utilities for different values of e E {0.1, 0.3, 0.7, 0.9} and 8 {10- 90100, 7, 7,10-9), The datasets sizes N range from 8000 to 46276 images. The sensitivity of the Fréchet mean, required for the mechanisms, is calculated using Th. 5 and Th. 3 Each experiment is repeated 10 times and averaged and the band (p- 2o, + 2 is shown, where μ and a are the mean and standard deviation, respectively, of the associated result. Our mechanism also outperforms the Riemannian Laplace on real-world datasets, and the utility gap is higher for smaller values of N and c.
6.2.2 Experiments on standard imaging data
In this section, we perform additional experiments on standard image datasets. We choose MNIST, KMNIST (Clanuwat et al.] 2018) (gray scale images) and CIFAR10, FashionMNIST (Xiao et al. 2017 (RGB images) 8s datasels. We extract images from 4 classes for each datasel and compute the corresponding class-wise Fréchet means of their covariance descriptors (1 = 10 10-)), which we privatize using the Riemannian Laplace Mechanism (Reimherr et al. 2021] and our proposed mechanism tangent Gaussian (Analytic). Fig. 3 shows the utilities for different values of e E {0.1, 0.3, 0.5, 0.7, 0.9} and 8 E {10-5, 5,10-7,10 10.10- Each experiment is repeated 10 times, the results are averaged and the band (μ- - -2a l + 2n) is shown, where fu and a are the mean and standard deviation, respectively, of the associated result. Fig. 3 illustrates the better utility of our mechanism compared to the Riemannian Laplace mechanism.
7 Conclusion and Future Work
Differential privacy for geometric statistics and learning is at a very early stage. We proposed a tangent Gaussian mechanism that specific to the SPD manifold equipped with the log-Euclidcan metric, and that outperforms the only existing bascline. One limitation of our work is that the proposed mechanism is restricted to one manifold with one specific metric. While the Log-Euclidean metric is one of the most
10

---

Published in Transactions on Machine Learning Research (02/2023)
N
6990
N- 7141
Rie Laplace
Rie Laplace
34
Ale Laplace
14
Rie - Laplace
Tan = 10
TanG = 10-9
TanG(6 = 10-5)
TanG(ó= = 10-1)
TanGio =
10
TanGló 10"1)
22
TanG(6 - 10"1)
12
TanGie 10
DustftD) AID))
TanG =10-)
TanG(a= 10-9)
10
TanG(8= 10-
TanG(o= 1
02
0.8
N - 7000
N- 7000
N - 7000
N 7000
Rie- Laplace
14
Laplace
Rie Laplace
Laplace
TanGi5- 10065--100000 'I
TanG 10 5
Tansi5- 10 5)
TanGl6- 10 "
Puttft A/D0)
TanGi5- 1
TanGl6 - 10-9)
TanG 10-2)
TanG 10-1)
Tans(6 10-1)
TanG(6 - 10-2)
10
TanGi6 10-))
TanG(6- 10-1)
22
0.1
5.6
0.8
02
5/1
0.6
0.8
02
3.1
0.5
0.5
0.
0.4
2.5
0.5
N 6000
= 6000
N = 6000
= 6000
Rie Laplace
Rie - Laplace
Rle Laplace
Ale - Lappaae
TanG 11
TanG 10
TanG(8 = 10-5)
20
TanG(d= 10
TanG =00 ?
TanG(a=
TanG(e= ?)
TanG(o= 10
TanGlo 100")
TanGia 10
15
Tansie 10-1)
15
TanG(6 10"))
15
30
20
5
02
0.5
7000
N- 7000
N - 7000
N 7000
16
Rie- Laplace
Laplace
Rie Laplace
Laplace
TanGlō-10 3)
TanG(5 10 b
TanG - 10 ")
14
TanG(6 10 ")
TanGi6- -0
10
TanGi5 10 [)
20
TanGi6 - 10-1)
12
TanG 10-2)
Puttito AUo1)
TanGi6 10-
TanGls 10-*)
TanG(6 - 10-1)
TanG 10-1)
2
0.4
6.6
0.2
0.0
3.1
0.2 0.4
Figure 3: Utilities on the private Fréchet means for different privacy parameters 6, and real-world datasets of sizes N. First and Second Row: Fréchet mean from MNIST, KMNIST (Gray scale images yielding 9 9 SPD descriptors). Third and Fourth Row: Fréchet mean from CIFAR10, FashionMNIST (RGB images yielding 11 x 11 SPD descriptor). Rie- Rie-Lae means the Riemannian Laplacian mechanism. TanG means the tangent Gaussian Mechanism for different values of 5 (ours). We also show the mean-2+std, mean+ bands
important metrics on the SPD manifold, future work should investigate how to build a Gaussian mechanism that works on any complete Riemannian manifold. We could define such as mochanism using a Ricmannian Gaussian distribution derived in Penned (2006). The main challenge would be show that the associated procedure is (c, 8) differentially private. Future work can also seek to privatize other geometric statistical algorithms like geodesic regression principal geodesic analysis.
11

---

Published in Transactions on Machine Learning Research (02/2023)
References
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, 1yy Mironov, Kunal Talwar, and Li Zhang, Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communicalions securily, pp. 308 338, 2016. 3
Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Log- Log-euclidean metrics for fast and sim- ple calculus diffusion tensors. Magnetic Resonance in Medicine: An Officiall Journal of the International Society for Magnetic Resonance in Medicine, 56(2):411 421, 2006. 2
Vincent Arsigny, Pierre Fillard, Xavier Pennec, and Nicholas Ayache. Geometric means a novel vector space structure on symmetric positive-defiite matrices. SIAM journal on matria analysis and applications, 29(1):328 347, 2007. 5
Borja Balle and Yu-Xiang Wang. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In International Conference on Machine Learning, pp. 394-403. PMLR, 2018.
P. J. Basser, J. Matticllo, and D. Le Bihan. MR diffusion tensor spectroscopy and imaging. Biophysical Journal, 66(1):259 67. URL http:://h....aaccieeeeeeervrreet
Nicolas Boumal. An introduction to optimization on smooth manifolds. Available online, May, 3, 2020.
Yinghao Cai, Valtteri Takala, and Matti Pictikainen. Matching groups of people by covariance descriptor. In 2010 20th International Conference on Pallera Recognilion, pp. DifferentialPPrivcy.org 2021.
IEEE, 2010. g
Clément Canonne. What delta, and what difference does it make?
Rui Caseiro, Joao F Henriques, Pedro Martins, and Jorge Batista. Semi-intrinsic mean shift on riemannian manifolds. In L'uropean conference on computer vision, pp. 342 355. Springer, 2012.
Sylvain Chevallier, Emmanuel K Kalunga, Quentin Barthélemy, and Eric Monacelli. Review of riemannian distances and divergences, applied to ssvep-bascd bei. Neuroinformatics, 19(1):93-106, 2021. 2
Pol Cirujeda and Xavier Binefa. 4dcov: A nested covariance descriptor of spatio-temporal features for gesture recognition depth sequences. In 2014 2nd International Conference on 3D Vision, volume 1, pp. 657-664. TEFF, 2014. g
Pol Cirujeda, Yashin Dicente Cid, Hemning Müller, Daniel Rubin, Todd A Aguilera, Billy W Loo, Maximilian Dichn, Xavier Binefa, and Adrien Depeursinge. A 3-d ricsz-covariance texture model for prediction of nodule recurrence in lung ct. IEEE Iransactions on medical imaging, 35(12):2620 2630, 2016. g
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. arXiv preprint arXiv:1812.01715 2018, I0
Manfredo Perdigao Do Carmo and Flaherty Francis. Riemannian geomelry, volume 6. Springer, 1992.
Cynthia Dwork. Differential privacy. In Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo We- gener (eds.), Automota, Languages and Programming, pp. 1-12, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-35908-1
Cynthia Dwork. Differential privacy: A survey of results. International conference on theory and appli- cations of models of computation, pp. 1-19. Springer, 2008.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to scnsitivity in private data analysis. In Theory of cryplography conference, pp. 265 284. Springer, 2006. [
Cynthia Dwork, Aaron Roth, al. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sei, 9(3-4):211 1--477 2014.
19
12

---

Published in Transactions on Machine Learning Research (02/2023)
P. Thomas Fletcher, Conglin Lu, Stephen M Pizer, and Sarang Joshi. Principal geodesic analysis for the study of nonlinear statistics of shape. TEEE transactions on medical imaging, 23(8):995- 1000, 2004. 2
Thomas Fletcher. Geodesic regression on riemannian manifolds. In Proceedings of the Third International Workshop on Mathematical Foundalions of Compulational Maalomy-Geometricn and Stalistical Methods for Modelling Biological Shape Variability, pp. 75-86, 2011. 2
Maurice Fréchet. Les éléments aléatoires de nature quelconque dans un espace distancié. In Annales de l'instütut Henri Poincaré, volume 10, pp. 215-310, 1948. 2 4
Hatem Hajri, Toana Tlea, Salem Said, Lionel Bombrun, and Yannick Berthoumieu. Riemannian laplace distribution on the space of symmetric positive definite matrices. Entropy, 18(3):98, 2016. 2
Sigurdur Helgason. Differential geometry, Lie groups, and symmetric spaces. Academic press, 1979.
Stephan Huckemann, Thomas Hotz, and Axel Munk. Intrinsic shape analysis: Geodesic pea for riemannian manifolds modulo isometric lie group actions. Statistica Sinica, pp. 1-58, 2010.
Mohamed E Hussein, Marwan Torki, Mohammad A Gowayyed, and Motaz El-Saban. Human action recog- nition using a temporal hicrarchy of covariance descriptors on 3d joint locations. In Twenty-third inter- national joint conference on artificial intelligence, 2013. g
Sadeep Jayasumana, Richard Hartley, Mathieu Salzmann, Hongdong Li, and Mehrtash Harandi. Kernel methods on riemannian manifolds with gaussian rbf kernels. IEEE transactions on pattern analysis and machine intelligence, 37(1)):2464 2477, 2015.
Adnan Mujahid Khan, Korsuk Sirimukunwattana, and Nasir Rajpoot. A global covariance descriptor for nuclear atypia scoring in breast histopathology images. IEEE journal of biomedical and health informatics,
1965):1637-1647, 2015. g
Janez Križaj, Vitomir Struc, and Simon Dobrišek. Combining 3d face representations using region covariance descriptors and statistical models. In 2013 10th IEFE international conference and workshops on automatic face and gesture recognition (FG), pp. 1-7. IEEE, 2013. 9
John M Lee. Riemannian manifolds: on introduction curvature, volume 176. Springer Science & Business
Media, 2006. 3
Mingyan Li, Radha Poovendran, and Sreeram Narayanan. Protecting patient privacy against unauthorized release of medical images in group communication environment. Computerized Medical Imaging and Graphics, 29(5):367-383) 2005. 2
Eyal Lotan, Charlotte Tschider, Daniel K Sodickson, Arthur Caplan, Mary Bruno, Ben Zhang, and Yvonne W Lui. Medical imaging and privacy in the era of artificial intelligence: myth, fallacy, and the future. Journal of the American College of Radiology, 17(9):1159 1162, 2020. 2
Bingpeng Ma, Yu Su, and Frederic Jurie. Covariance descriptor based on bio inspired features for person re-identification and face verification. Image and Vision Computing, 32(6-7):379 379-3990, 2014. g
Tetsu Matsukawa, Takahiro Okabe, Einoshin Suzuki, and Yoichi Sato. Hierarchical gaussian descriptor for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 1363-1372. 2016.
Nina Miolane. Geometric statistics for computational anatomy. PhD thesis, Université Côte d' A'Aur, 2016. 2
Nina Miolane, Nicolas Guigui, Alice Le Brigant, Johan Mathe, Benjamin Hou, Yann Thanwerdas, Stefan Heyder, Olivier Peltre, Niklas Koep, Hadi Zaatiti, et al. Gcomstats: python package for ricmannian geometry in machine learning. Journal of Machine Learning Research, 21(233) 9, 2020. 24
Nina Miolane, Matteo Caorsi, Umberto Lupo, Marius Guerard, Nicolas Guigui, Johan Mathe, Yann Cabanos, Wojeiech Reise, Thomas Davies, António Leitão, et al. Icl 2021 challenge for computational geometry & topology: Design and results. Xrxiv preprint ar Xrxix22008 09810, 2021.
13

---

Published in Transactions on Machine Learning Research (02/2023)
Adele Myers, Saiteja Utpala, Shubham Talbar, Sophia Sanborn, Christian Shewmake, Claire Donnat, Johan Mathe, Rishi Sonthalia, Xinyue Cui, Tom Szwagier, et al. Tclr 2022 challenge for computational geometry & topology: Design and results. In Topological, Algebraic and Geometric Learning Workshops 2022, pp.
269-276. PMLR, 2022. 2
Yanwei Pang, Yuan Yuan, and Xuclong Li. Gabor-based region covariance matrices for face recognition.
IEEE Transactions on circuits and systems for video technology. 18(7):989-993, 2008. 9
Xavier Pennec. Intrinsic statistics on riemannian manifolds: Basic tools for geometric measurements. Journal of Malhemalical Imaging and Vision, 25(1):127-11) 154, 2006.
Xavier Pennec, Pierre Fillard, and Nicholas Ayache. A riemannian framework for tensor computing. Inter- national Journal of computer vision, 66( (661):41-666 66, 2006. 2
Xavier Pennec, Stefan Sommer, and Tom Fletcher. Riemannian geometric statistics in medicall image anal-
ysis. Academic Press, 2019. 2
Matthew Reimherr, Karthik Bharath, and Carlos Soto. Differential privacy over riemannian manifolds. Advances in Neural Information Processing Systems, 34, 2021. 2
24
Christian P Robert, George Casella, and George Casella. Monte Carlo slatistical methods, volume 2. Springer, 1999. 24
Armin Schwartzman. Lognormal distributions and geometric averages of symmetrie positive definite matrices. International Statistical Review, 84(3):456-44 1464456666666666 2016. 3
Amr Sharaf, Marwan Torki, Mohamed E Hussein, and Motaz El-Saban. Real-time multi-scale action detec- tion from 3d skeleton data. In 2015 IEEE Winter Conference on Applications of Computer Vision, pp. 998-1005. IEEE, 2015. g
Oleg Smirnov. Tensorflow ricmopt: a library for optimization on riemannian manifolds. arXiv preprint arXiv:2105.1388 2021. 24
Stefan Sommer, François Lauze, Soren Hauberg, and Mads Nielsen. Manifold valued statistics, exact principal geodesic analysis and the effect of linear approximations. In European conference computer vision, pp.
43-56. Springer, 2010. 2
Raghav Subbarao and Peter Meer. Nonlinear mean shift over riemannian manifolds. International journal of computer vision, 84(1):1-20, 2009. 2
Hedi Tabia, Hamid Laga, David Picard, and Philippe-Henri Gosselin. Covariance descriptors for 3d shape matching and retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4185 4192, 2014. 9
Yann Thanwerdas and Xavier Pennec. O (n)-invariant riemannian metrics on spd matrices. arXiv preprint arXiv:2109. 100.20569....,. 2021.
5
P Thomas Fletcher. Geodesic regression and the theory of least squares on ricmannian manifolds. Interna- tional journal of computer vision, 105(2):171 185, 2013. 2
Oncel Tuzel, Fatih Porikli, and Peter Meer. Region covariance: A fast descriptor for detection and classifi-
cation. In Europeon conference on compuler vision, pp. 589 600. Springer, 2006. 3 g
Oncel Tuzel, Fatih Porikli, and Peter Meer. Human detection via classification on riemannian manifolds. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8. IEEE, 2007. g
Saiteja Utpala, Andi Han, Pratik Jawanpuria, and Bamdey Mishra. Ricoptax: Riemannian optimization in jax. In OPT 2022: Oplimization for Machine Learning (NewrIPS 2022 Workshop). 24
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist novel image dataset for benchmarking machine learning algorithms. Xiv preprint Xrxiv 1-x::x108.0000 707.07747 2017. 10
14

---

Published in Transactions on Machine Learning Research (02/2023)
Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2: A large-scale light benchmark for 2d and 3d biomedical image classification. arXiv preprint Ar1:::1110.1420 14..1100..199.,... 2021.
Florian Yger, Maxime Berar, and Fabien Lotte. Riemannian approaches in brain-computer interfaces: a review. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(10):1733-176, 2016.
2
Kaicheng Yu and Mathieu Salzmann. Second-order convolutional neural networks. ar preprinl
NMXiv:1703.066 06817, 2017. 9
Paolo Zanini, Marco Congedo, Christian Jutten, Salem Said, and Yannick Berthoumieu. Transfer learning: A riemannian geometry framework with applications to brain-come compncr interfaces. IEEE Transactions on Biomedical Engineering, 65(5):1107 (6(5):1107-1116, 2017.
Mingyong Zeng, Zemin Wu, Chang Tian, Lei Zhang, and Lei Hu. Efficient person re-identification by hybrid spatiogram and covariance descriptor. In Proceedings of the IEEE conference computer vision and pollern recognilion workshops, pp. 48 56, 2015.
Ying Zhang and Shutao Li. Gabor-lbp based region covariance descriptor for person re-identification. Tn 2011 Siath International Conference on Image and Graphics, pp. 368 371. IEEE, 2011. g
A Proofs
Consider E N In this supplementary material, I|-liz2 and (,)2 denote the standard Euclidean inner product and the Euclidean norm on vectors. i..., for all n, y e IR.
P
(x. 12.8)12 Σ with.
| v /x,,x)22
Then, r) Ir denotes Frobenius inner product and Frobenius norm respectively, i.e., given A, B E RR
(A, = =
|||| = V A)
Lastly, Il-llz denotes the spectral norm of matrices. i.c., for all A Rick
| 122
||Al|2 = sup
||
|=
A.1 Useful Lemmas
In this section, we derive the distribution of the privacy loss. Its proof requires us to first. introduce the following definitions.
Definition 3 (Diffeomorphism and Isometry). A diffeomorphism between two manifolds Mi and M2 is an inwertible smooth function whose inverse is also smooth. diffeomorphism & between two Riemannian manifolds (M1,g1), (M2, is colled an isometry if it preserves distances i.e., Pa. (p.4) = Paz (o)p)))) ω(q)) for all p,q E M1.
Note that. Logm is a diffeomorphism from SPD(k) to SYM(k) and veed is a diffeomorphism from SYM(k) R "< making veed Logm diffcomorphism from SPD(k) R _ Importantly for our derivations the proofs of this Subsection, the operation veed Logm preserves the distances making it isometry.
15

---

Published in Transactions on Machine Learning Research (02/2023)
Lemma A.1 (veed Login is isometry). Let Logm SPD(A) SYM(K) be the matrix logarithm and let veed (YM(K)
+ R 11) be defined veed(X) = [diag(X)) v2 uppprrdiag(x)" Then veed Logm
(K)(K) + R #(++1)
is an isometry from SPD(k) equipped with the log- Eo-Euclidean metric to standard Euclidean with standard 1.2 metric, i.C.,
space R KE+))
(x, X2) P1,2 Logm X1, veed Logm X2),
(7)
where X2, E SPD(k). Hence we have that
||Logm X|FF = ||veed Logm X 11222
(8)
Proof. Let X1, X2 be elements of SPD(K). We have:
(ie(X1,,0)
Loogm X1 Logm X211
Σ
(Logm X1 Logm X2)
k
"(Logg (Logm X1 Logm X2)
Σ
(Logm X1 Logm X2)
Loggm X1 Logm X2)
i<j
12j
2.
(Logm X1 Logm X2)
[Logm X1 -Logm X2)
2 (pperdiag(Logm X, Logm X2)
|
+ ||diag (Logm.X, X1 Logm X2)
L2
[[oged[Logm X, Logm X2)
l/ve Login X1 - veed Logmn X2lli.2 = pia (vecd Logm X1, veed Logm X2).
from which we have Eq, equation 7 Eq. equation follows as
[Togin X PLE(X, T) P Logm X, veed Logm T) ||veed Logm X Il1,2
Now, we prove some useful properties of the Log Gaussian distribution, denoted LN, that will later. Essentially, we show that the Log Gaussian distribution behaves "nicely" with vector space structure of SPD(K). We recall that the vector space operations on the SPD manifold are defined as follows,
Xi @ X2 = Expm [Logm X1 + Logm X2)-
(9)
X, e X2 = Expm [Logm X1 [og
(10)
Lemma A.2. Take & E N. Let I denote the ie identity matriz, and consider M, C SPD(k), Σ SPD M(k+1) and Xa the chi-square distribution with degrees of freedom. Then:
LN(TE)) = X@M M CN,, X ~ LN((T02T) = (Logm L,go X)r ~ ~N(0,0² L C
(11)
(12)
X LN((1021) ||oggm X| ~ x x
(13)
16

---

Published in Transactions on Machine Learning Research (02/2023)
Proof. We first recall standard properties of multivariate normal distribution. Let m, E R2 and Σ, I E Rexp then following properties hold true.
(m, Σ) = a a ~ N(a m. L) (mm, Σ) a'x ~ Nía m, a² Ea).
(14)
(15)
x ~ (N0,00 = || o²
(16)
where x² denotes chi-square distribution. We prove the properties (a)-(c) below.
(a) Distribution of X x M.
vocd[Logm[X [X @ M]] (+) vocd[Logm Expm Dog X + log M]]]
veod[Logm X Logm M]
veod|Logm X] vecd M]
(w) N(veed[Loggm MM) 2)
where in (+) we used Eq. g and in (**) Eq. equation 14
(b) Distribution of (Logın C, Login X)F.
(Logm C, Logm X)
(veed[Logm CJ, vecd[Logm X|)12
N (Lvecd[LLogm C)2 12, veed[T.ogm CJ² o Logd C1)
N(0, o² Logm C1Il22)
N(00 |[Logm CII?).
where we used Eq. 8 in (*) and Eq.
(**).
(c) Distribution of ||oogm |
Logmn X | (+) Logm X1I (1x) o² x2xa X
where we used Eq. in (*) and Eq. (**) with p A(R+1)
2
As corollary, we give equivalent reformulation tangent. Gaussian mechanism that will useful in the rest of the proofs.
Corollary A.3 (Equivalent Reformulation of tangent Gaussian). Let ATG be a tangent Gaussian mechanism defined as (T(() = X, X ~ (N( Then, it is equivalently defined as:
Arc(f(D)) = f(D) NN.N (N(11021). (
Proof. The proof comes from Eq. of Lemma A.2
Now, we are ready to prove the distribution of the privacy loss of the tangent Gaussian Mechanism which is given Th. 1
17

---

Published in Transactions on Machine Learning Research (02/2023)
A.2 Proof of Th. 1
Theorem A.4 (Distribution of the privacy loss of the tangent Gaussian). Let ATG be tangent Gaussian mechanism with variance o². Its privacy loss is normally distributed as
DA N (PL(((D),((D') PLE ((D()
20²
o²
Proof. Assume that D, D' are adjacent datasets. Let V= = I(D) e S(D)). Consider the privacy loss random variable D. Let Y = ATG(D).
In PATa(D)( (Y) (ATa(D))())
(1) In PATa(D) (F(D)
PAra(D") (JD) () N)
(2)
T veed (f @ N) - Login f(D))
veed [Logm(((D) @ N) Logm f f(D)
2
I
veed [oogm(]()) e N) Logm S(D')
veed [Logm(/ @ N) - Logm S(D')
d²
(3)
veed Logm N
2 +
veed Logm f(D) - Logm Logm +Lgm' 1>
20
L2
20
(4)
veed Logm N)
2
+
/veod Logmn( @ N)) |
2
1,2
20
1.
[[ogin( (V ||
||oogmm N12]]
20
|Logm V + (Loom V, Logm N)
20
(
[|ogmm V + 2/ (, o² |Logm VI )
2x
|Logm VI
Logm 1
(7)
N
2a²
a²
(L)))( (i)()(D).
(8)
202
where we used following properties in each of the steps labeled above.
1. Equivalent reformulation of tangent Gaussian, Corollary. A.3
2. Density of Log Gaussian Distribution.
3. S(D) Φ N = Expm[Logm J(D) + Logm N).
Logm(V V @ N) Logm f (D) - Logm f(D) + Logm N.
Isometry of the veed operation, Eq
Eq. 12 in Lemma.
A.2
standard Gaussian property, Eq.
14
|Logg VI [LLogm J(D) Login S(P')l (iE)
18

---

Published in Transactions on Machine Learning Research (02/2023)
A.3
Proof of Th.
2
In this section give proof of privacy guarantee of the tangent Gaussian Mechanism.
Proof. The proof proceeds similarly to the proofs referenced below, by only replacing the standard sensitivity AL2 with respect to the Euclidean L2 metric, by ALE:
1. (Classical). See Th. A.1 (Appendix A Page 261) in Dwork el al. (0014)
2. (Analytic). See Th. 5, Th. 8, Th. 9 (Section 3) in Balle & Wang 2018
The fact that mechanism is manifold valued comes into play while deriving privacy loss (Taken care by Theorem 1). Once privacy loss (which is real valued scalar random variable) is derived, going from privacy loss to actual privacy guarantee wouldn't be affected whether mechanism is manifold-valus or not because both of the above proofs entirely rely on properties of one-dimensional euclidean Gaussian random variables. Specifically,
(Classical). Directly employs tail bound of one-dimensional Gaussian variable that Pfr exp(- 20
(Analytic). The method employs both the sufficient and necessary conditions of the (e, d) guarantee. Additionally, the algorithm avoids using tail bounds, since they may be loose, instead uses properties of Gaussian CDFs and employs binary scarch to solve analytically for a, given (c, See (Balle & Wang 2018 Algorithm 1) and discussion therein for more details.
A.4 Proof of Th. 3 and Th.
In this section, we prove the sensitivity of the Fréchct Mcan Theorem. 3 and then the utility of the tangent Gaussian Mechanism in Theorem. First we give the proof of 3
Proof. Consider N, < xo and M E SPD(k) such that (MM is a geodesic ball of radius and center M. Let D ~ D' be adjacent datasets of size E N that lic in B, (MM Without loss of generality, assume that they differ only by their last data point X, and X D X2,, X, and D' = (X,, X2, Let X,, X denote the Fréchet means of D and D' for the Pog-Fuclidean metric, which can be expressed in closed forms as mentioned in the main text. The log- EugEEucli distance between the Fréchet mcans writes:
PLe X X
Logmn Xi
Logm X
Logm X
Logm
Expm
Logm
Expm
n-1
Logm X;
Logmn X; + n
Logmm Xn,
Loggm X
n
1
[Logm X,, Noomm
n 1
19

---

Published in Transactions on Machine Learning Research (02/2023)
(1) ALE sup (L = sup OLe(Xn. X,) S PLLE (), + PLE(M, (M, xx)] _ D-DD'
(3) 2r
D.D''
where the closed form for the log- log-Euclidean Fréchet means in (+), the triangle incquality in (f) and assumption that data lies in (M) in (3).
Proof of Th. is given as follows,
Proof. Consider deviation (D),
|Logm f(D) Logm ATG (D)] (1) [|Logm Logm(f(D) @ N)
(2) |Toogm N|
(3) Xd=
where we use the following properties at each step:
(1) Corollary.
A.3
(2) f(D) + N Expm Logmm f(D) + Logm N).
Eq. 13 of Lemma.
A.2
Now we derive expression for (lG)
Blpio(
ATa())) (1) odd
(2) 24 LE In(1 (1.25/8)d
2
(3) 8 ln1.25//8
n²e²
where we use following properties at each step:
1. xa degrees of freedom.
E[c] d i.e., expectation of chi squared distributed random variable is mumber of
2. 22 for (c,8)-ArG from Th.
3. ALE < from Th.
A.5 Proof of Theorem 5
In this section derive log-Euclidean geodesic radius of covariance descriptors. We first prove following lemma that relates Logm X||r in terms of lower bound least eigenvalue and upper bound largest eigenvalue of X.
Lemma A.5. If E SPD(k) and let Amin (X), Amax (X) be the minimum and marimum eigenvalues of X. If l < Amin (X) and (X) L. Then, [Togm XIlr r S Vk max {[ln E) [In T.]).
20

---

Published in Transactions on Machine Learning Research (02/2023)
Proof. Consider,
(t)
Logmn X I
V [ X
/k mäx In X i=1
v max
|min In Asln ımáx In Xز²
i=l
1 Vh max
|| min 1=1
z drl. In mهي d,
vk max {Inn Amin
|In Amax
(17)
where (t) uses the fact that A R | S Vbl|Alla and (#) uses the fact that In is monotonically increasing. Now, split the derivation into two cases.
CASE Amin(X) 2 1. For In (minn(X) $ [In Amaxl= In L
|laa is increasing function, which gives |ln El <
Vk max {|ln Amiul, [In Amax | S VkiInn L| = vk max {Iln E), [In L|}
(18)
2. CASE XX) < 1. For x < 1, [n is decreasing function: [In Aminl & [I e). We further split the derivation into two sub-cases here
(a) SUB-CASE Amax 2 In this sub-case In Amaxi [In L| and In Amin [In El from which have that
La max {[ln Aminls [In Amax S Vk max {In E) [In L.-
(19)
(b) SUB-CASE Amax
1. In this sub-case [In L| < [In Amaxl S In Amial [ln El.
V max {Inn Aminl [In Amaxl) } s vkllnn El = V max {Inn E) [In L.}.
(20)
Based on Eq.
18 Eq.
19
Eq.
20 and Eq 17
We can conclude the lemma.
Lemma A.6.
Let (I) denote the covariance descriptor for image I for given vj > O, which is defined as
follows,
[ - ()(((I(()) - w) + In.1
(T) =
ZES
with,
I [
(TI) 2. J. II Iyl. Iunl
[Zyll V Inl² 2 + [Tyl² arctan
where x, y are grid positions of Image I, I denote pizel intensity values, IZz [yyl denotes first order intensity derivatives and [Izzl Iyyyl denotes the sccond order intensity derivatives then following holds,
I is grayscale image, then (I), 12 n.
2. TJ I is RGB image then (T)l2) I 14 n
21

---

Published in Transactions on Machine Learning Research (02/2023)
Proof. We have:
(2)12
- (x(
-1)² + +
S
XES
[I)(I) ()((()(()) ~)7112 + |1
S
max
[[((() --)(0)( -10² +1
XES
max (()([)(x)) - 1I1L2 +
XES
S max (x) 1
(21)
xE8
where we used following properties in each of the steps:
Triangle Inequality.
2. For all e IR, the spectral norm of 1-rank matrix aa is ||all22-
3. Consider the descriptor (II) x; yı I, [Z2] [Zyl Ianli Zyyl v VZ, ,, arctan
Then,
[,,
(x)) 2 0 for each x E and i kl....,k). This yields: (H), = (15) Exes (II) 2 Hence it implies that (x) xlli.22 < (x)()
Then, the following calculations provide an upper bound for [lx(Z) Specifically, we bound each of the elements constituting the descriptor (II) = v. I. [Iz] [zyl [Zaal, [Tyy /Zzz + [Zyl2, arctan
Normalized grid positions: Vx E S, 0 S x, y S 1.
2. Pixel intensity values C, for E [c] Vx E S, 0 0 C 1
3. First intensity derivatives [Z) [Zyl: The first intensity derivatives can be obtained by the convolution operation (denoted as x):
+1
0
-1
+1
+2
+1
1
I = I
+2
0
-2
In = I
0
4
4
+1
0
-1
2
-1
Since 0 < I(x) S 1, using the definition of the convolution operation yiclds Vx S, [x)(x s 1, [x)) 1.
Second intensity derivatives [Zzz| [Zull The second intensity derivatives can be obtained by the convolution operation (denoted as *)
+1
0
2
0
1
+1
+4
+6
+4
+1
+4
0
-8
0
4
0
0
0
0
I,z = I 1
1
+6
0
12
0
6
Iny I
-2
8
-
12
8
-2
32
32
+4
0
-8
0
4
0
0
0
+1
0
-2
0
1
+1
+4
+6
+4
+1
Since I(x) < 1, using the definition of the convolution operation yields [Ix((x) S
[x))
1.
22

---

Published in Transactions on Machine Learning Research (02/2023)
5. Norm of first intensity derivatives since [Ix(x] S 1, [x)(x) I have that bo E
S, ( + [x)² & 2
6. Angle intensity derivatives Note that for 2 0, 0 I arctana S
Hence we have that
Vx E S, arctan
These provide the following upper bounds on L2 norm of (x))(x),
for a gray scale image, vx E S (x)] - 12,
(22)
for RGB image, Vx S (x) s 14.
(23)
The claim follows by using Eq. 22 Theorem A.7 (Geodesic Radius of Covariance Descriptors).
Eq. 23 in Eq. 21
1. I is gray scale image, then [[Logm (Z)l S max max (In,)), [In + m)]
I is a RGB image, then [[ogg (I)(] I VII max ([lnn), [ln(14 + n)].
Proof. We first note that
Amin (R) Aminn
Σ (x(I) - (x) - 11) + +.
S
XES
Σ (x(I)(x) - (x(((() 15)T + Amina [17]
Amin
S
XES
0 + 77.
(24)
where we used Weyl's inequality for symmetric matrices in (1) and Amin of positive semi definite matrix is 2 0 and Amin(y 7 in (2).
For gray scale images, (Z) produces 9 x 9 matrix:
(+) 9 max (Inn E| [In LI]
|Togin (Z)l
v max [In [m(12 + +)
where we use Lemma. A.5 in (+) and 24
24 L = 11) and Lemma, A.6 L = 12 + 17) in (**)
For RGB images, R, (I) produces a 11 x 11 matrix:
Logm R, (I)
/ max max {[n [n L
11 max [[nn)) +))1
where we use Lemma. A.5 in (f) and Eq 24 (l 11) and Lemma. A.6 L 14 17) in (t), a similar fashion.
Note that in all of our experiments, we choose y) = 10 and hence In 13.8 domninates over In [n(1++n) 2.5 and [n111 + 7))l x 2.6.
23

---

Published in Transactions on Machine Learning Research (02/2023)
Rie Laplacel1 -10")
50
Rie Laplace 100
Ve-LLaplacee3-100
Laplace(
1.7
60
Rie-Laplace|5 11
40
Liplaap
time in seconds
50
(anG-Analytic(urss)
Put(f( A(D))
TanG-Classcal((Ours)
1.6
lanG-Amllytic(Ours)
redius predicted
30
1.5
OccoMNisTigreyscale) PathMNIST(RGB)
30
20
redius
20
0
1.2
10
20 30
10
20
30
k
(a)
(b)
Figure 4: (a) Computational times for Rie-Loplace(x) (z) the Riemannian Laplace mechanism with MCMC burn-in of x (10, 000; 30, 000; 50, 0003 (Reimherr et. al.) 2021), and TanG-Analylie the proposed langent
Gaussian mechanism (analytic version). (b) Utility with varying burn-ins for Rie-Laploce Plots (a, b) use
different matrix sizes k. Plot (c) explores if the bound from Th. 5 is tight in practice.
B
Experiments
All experiments were run on DELL XPS 17 9710 LaPTOP which has 32GB OF RAM, 11TH GEN INTEL(R) CORE 19-11900H 2.50GHz Processor. No GPUs were used in the experiments.
B.1
Implementation Details
Let k E N, M E SPD( a > 0 and PLE denote log- Euclidcan distance. The Ricmannian Laplace distribution with log-Euclidean distance is given by
(- (I.((X.M)
p(X Cas,o
exp
(25)
Note that sampling from Eq. equation 25 requires Markov Chain Monte Carlo (MCMC) methods (Robert et al. 1999), for which one needs to choose a proposal distribution that generates candidates on the SPD Manifold. We choose the Log Gaussian distribution as the proposal in our experiments given its simplicity and the fact that is quick to sample from. In all experiments, we found that using the log Gaussian distribution as proposal yields a stable acceptance ratio of 50% 65%. To summarize,
1. Initialize Xour at a random point of the manifold SPD(k).
2. For 1 + iterations
(a) Draw candidate from X CN 021).
(b) With probability exp(-PLE ( Xmean X)/o)/ exp(- PLE( Sesp--Le(Xcures (X/o) accept the generated candi- date and set Xeurr
The final sample is chosen based on a burn-in period of 50,000 steps. Both Riemannian Laplace and tangent Gaussian mechanism can be easily implemented using existing libraries like geomstats Miolane et al. (2020).
censorflow-riemopp Miolane et al. (2020): Smirnov 2021 ricoptax Utpala et al. all our experiments we
used geomstats Miolane et. al. (2020
B.2 Additional Experiments
We compare the times required to privatize the Fréchet mean using both mechanisms and varying e {2, 5, 10, 15, 20, 25, 30} in Fig. aa). Note that we used MCMC for Riemannian Laplace and its time depends on the burn-in that we choose in {10000, 30000, 50000). For & 30, Fig. ((a shows that Riemannian Laplace mechanism takes 14 see (burn-in 10000), 36 (burn-in 30000) and 73 (burn-in 50000) whereas
24

---

Published in Transactions on Machine Learning Research (02/2023)
our tangent Gaussian (Analytic) mechanism takes 1.3 microsec. Fig. 4 (b) considers the effect of the burn-in
the Riemannian Laplace's utility and finds significant difference for burn-ins in (10000, 30000, 50000).
Fig. shows that the bound derived in Th. 5 is tight in practice as illustrated by the ratio of the bound
obtained in Th and the practical bound.
B.3 Code
Code is attached with Supplementary Material.
25


Reviews:
[{"review_id": "Ref79WsbmL", "reviewer": "Reviewer_Shyk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths \n\n- *Motivation.* The focus of the article is clear and well motivated. The problem of designing methods to provide accurate and privacy-preserving statistical summaries of sensitive data is a very relevant topic in computer science, especially when considering medical applications (as mentioned in the paper). Moreover, Section 1.1. makes a good case that the tools of Riemannian geometry can naturally be used in several highly sensitive medical applications. Hence the need to implement privacy preserving mechanisms for data from Riemannian geometry. \n\n- *Soundness.* The proposed solution is a simple adaptation of an existing method in the differential privacy literature, but the adaptation seems technically sound (at least to my level of reading of the proofs). In addition, the proposed method is empirically tested on many synthetic and real-world datasets, including some datasets related to medical applications (which I really appreciated). \n\n- *Readability.* Overall, the paper is quite well written, easy to follow and well organized. \n\n### Weaknesses\n\n- *Hard to identify technical contributions.* The way the results are presented in the article gave me the impression that each result is an independent technical contribution. However, as I understand it, some of them are more important than others. Specifically, it seems that Theorems 1 and 4 represent the main technical contribution of the article.  In fact, Theorem 3 seems to me a direct corollary of [1, Theorem 2] when $\\kappa \\leq 0$ and Theorem 2 follows the steps of existing proofs in the differential privacy literature (as explained in the appendix). I think that highlighting this fact would improve the clarity of the paper's contributions.\n\n- *`Unfair' comparison to [1].* The performance claims of the paper are mostly based on a comparison with the Riemannian Laplace mechanism introduced in [1]. First of all, I am not very comfortable with this comparison because, in essence, the two mechanisms do not serve the same purpose. Indeed, the Laplace mechanism aims at providing *pure* differential privacy while the Gaussian mechanism aims at providing *approximate* differential privacy. In fact, *pure* differential privacy is generally considered to be a much stronger privacy primitive than its approximate counterpart, which essentially leads to worse performance compared to approximate differential privacy. I am not saying that this comparison is a deal breaker, but I would advise relying on it much less when making performance claims. Below I give two suggestions that could mitigate this problem. \n  -  *From a theoretical point of view.* Below Theorem 4, I would extend the paragraph entitled 'Theoretical results' to compare more thoroughly the obtained results with those of [1]. In particular, I would insist on the fact that approximate and pure differential privacy do not have the same semantics and that the comparison in terms of performance should be taken with a grain of salt. \n  -  *From the emprical point of view.* I think that the comparison with the Laplace mechanism can remain in the experiments, but similar to what was done in [1] for the Laplace mechanism, I would complement them by adding a comparison with an extrinsic approach. \n\n- *Minor comments.* Here, I simply point out a few typos to be corrected and small ambiguities to be cleared up. \n  -  Second line below Equation (1). 'provide**s** an upper bound for the expectation'\n  -  Above Table 2: 'Arsigny et al. (Arsigny et al. 2007)'\n  -  Second line of Section 4: ' that needs to **be** privatized'\n  -  Algorithm 1: I think the first line can be safely removed, all the information is already in the algorithm name and in 'inputs'. Also in the same algorithm, the subroutines 'CLASSIC' and 'ANALYTIC' are used without having been introduced before.\n  -  Last sentence of the first paragraph of Section 6: The sentence looks incomplete.\n\n[1] Matthew Reimherr, Karthik Bharath, and Carlos Soto. Differential privacy over riemannian manifolds. Advances in Neural Information Processing Systems, 34, 2021", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the problem of releasing a differentially private statistical query for data points that live in a Riemannian manifold. Specifically, it presents an extension of the well-known Gaussian mechanism to privately compute the Fréchet mean of a set of points living in the manifold of symmetric positive definite matrices. After presenting this new scheme, the article provides several results demonstrating the privacy and utility guarantees of the method. Finally, it presents a comparison of the numerical performance of the newly proposed Gaussian mechanism with the state-of-the-art solution for differentially private estimation of the Fréchet mean (i.e. the Riemannian Laplace mechanism). Based on experimental results (on synthetic and benchmark datasets), the proposed technique is claimed to outperform the Laplace mechanism both in terms of utility and computational efficiency. ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "For securing my recommendation for acceptance, I would ask to respond to my concerns regarding the 'unfair' comparison with [1] and to clarify the technical contributions of the paper.  Also,I would suggest providing a concrete proof for Theorem 2. To be honest, I am quite convinced that following the exact same steps as previous proofs would give the expected result, but as the paper is about manifolds, there might be some technical difficulties I am not seeing. Anyway, having a concrete explanation of why the results readily apply would help.", "broader_impact_concerns": "No concern", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aGObPoz51J", "reviewer": "Reviewer_2LiX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n1. The idea of special casing to the SPD manifold endowed with the log-Euclidean metric has real utility, since the speed-up with respect to prior work is quite considerable and this is still a use case seen in practice.\n\n2. The writing in this paper was for the most part very good and the presentation of the background material and the idea was very clean.\n\n3. The experiments are reasonably solid, covering a range of synthetic and real world datasets.\n\n### Weaknesses\n1. It should be well noted that the SPD manifold endowed with log-Euclidean structure is flat (i.e. geometrically it is as trivial as Euclidean space). Hence, it is very unsurprising that for this specific case the introduced mechanism is far faster than the much more general mechanism given in [38] (which applies to all complete Riemannian manifolds). The novelty of the paper is thus limited, as it needs only to lift the corresponding differential privacy result in the Euclidean setting to the SPD case. It does so using a variety of existing SPD constructs (e.g. log-Gaussian distribution on SPD(k) from [39]). The privacy guarantee can be lifted out of past Euclidean literature.\n\n### Verdict\nAs it stands, the paper does have some real-world utility. It presents a fast differential privatization method for any statistical summary on the SPD manifold endowed with log-Euclidean structure. The experiments demonstrate the method's utility for a number of real and synthetic datasets. Although I have some issues with the degree of novelty (as mentioned in the weaknesses section above), I think the paper (1) makes claims supported by accurate evidence and (2) would be interesting to at least some individuals in TMLR's audience, thereby meeting both criteria for acceptance. Therefore I recommend acceptance. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Motivated by both the growing use of the SPD manifold to model data as well as the need for differentially private machine learning algorithms, the paper proposes a fast differentially private Fréchet mean computation over the manifold of SPD matrices endowed with the Log-Euclidean metric. More specifically the paper proposes the \"Tangent Gaussian Mechanism\", which privatizes any statistical summary on the manifold of SPD matrices endowed with the log-Euclidean metric. When compared with previous work focused on the Fréchet mean, the proposed approach achieves higher utility and is considerably faster. Experiments on both synthetic and real-world data demonstrate the efficacy of the method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The paper writing is quite good already and the presentation, including figures, is clean. A couple of minor corrections are given below:\n\nPage 7: \"$\\Delta \\leq (2 * \\sqrt{k})/(4 * 500)$\" -> \"$\\Delta \\leq (2 \\cdot \\sqrt{k})/(4 \\cdot 500)$\" (please use \"\\cdot\" instead of \"*\")\n\nPages 9 and 11: Although it is understandable, please use better formatting than \"(avg - 2 * std, avg + 2 * std)\" for the 4 standard deviation band. Something like \"the $(\\mu-2\\sigma, \\mu+2\\sigma)$ band is shown, where $\\mu$ and $\\sigma$ are the mean and standard deviation, respectively, of the associated result\" would suffice.", "broader_impact_concerns": "I have no concerns regarding the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3eobelQIGl", "reviewer": "Reviewer_NgEb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Developing a toolbox for DP on manifolds is an interesting problem. Symmetric positive definite (SPD) matrices model a wide range of data, such as medical data, thus private statistical computations on the SPD manifold are a worthy endeavour. Despite its importance for the processing of a number of medical data, geometric statistics currently stands understudied from the lens of differential privacy.\n\nThe strengths of the paper are as follows.\n1. This paper proposed a tangent Gaussian mechanism that is specific to the SPD manifold equipped with the log-Euclidean metric, and the authors prove that it achieves approximate differential privacy.\n\n2. When the statistical summary is the Fréchet mean, the mechanism outperforms the only existing baseline and is computationally simple, does not rely on additional parameters.\n\nThe weaknesses of the paper:\nThe proposed mechanism is restricted to one manifold with one specific metric, while the log-Euclidean metric is one of the most important metrics on the SPD manifold.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the problem of differentially privacy for geometric statistics and learning. This work proposed a tangent Gaussian mechanism for private Fréchet means on the manifold of symmetric positive definite (SPD) matrices endowed with log-Euclidean metric.\n\nThe main contributions of the paper:\n1. This paper proposes a tangent Gaussian Mechanism which privatizes any statistical summary on the manifold of Symmetric Positive Definite (SPD) matrices endowed with the log-Euclidean metric. The authors prove that it achieves approximate differential privacy.\n\n2. When the statistical summary is the Fréchet mean, the authors show that the presented mechanism obtains significant improvement in terms of utility over recent works. The mechanism is computationally simple, does not rely on additional parameters, and is multiple orders of magnitude faster than the state-of-the-art.\n\n3. The mechanism on synthetic and real-world (medical) imaging data shows effective and the authors prove a theoretical bound on the radius of log-Euclidean geodesic ball with the covariance descriptor pipeline.\n\n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The method in the paper seems not novel. I think it can be seen as only a natural extension of the method proposed in Reimherr et al. (2021). I think the authors should emphasize the novelty their work and difference between their work and Reimherr et al.\n\n", "broader_impact_concerns": "There are no any concerns on the ethical implications of the work that would require adding a Broader Impact Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The response and revision provided by the authors convinced the reviewers: they all recommended acceptance without any additional requested changes. Therefore, I congratulate the authors for their work and recommend this paper to be accepted as is.
Decision: Accept (poster)