Gaussian Differential Privacy Transformation: from identification to application

1 INTRODUCTION . Recent years have seen explosive growth in the research and application of data-driven machine learning . While data fuels advancement in this unprecedented age of “ big data ” , concern for individual privacy has deepened with the continued mining , transportation , and exchange of this new resource . While expressions of privacy concerns can be traced back as early as 1969 ( Miller , 1969 ) , the concept of privacy is often perceived as “ vague and difficult to get into a right perspective ” ( Shils , 1966 ) . Through its alluring convenience and promise of societal prosperity , the use of aggregated data has long outstripped the capabilities of privacy protection measures . Indeed , early privacy protection protocols relied on the ad hoc enforcement of anonymization and offered little to no protection against the exposure of individual data , as evidenced by the AOL search log and Netflix Challenge dataset controversies ( Narayanan & Shmatikov , 2006 ; 2008 ; Barbaro & T. Zeller , 2006 ) . Differential privacy ( DP ) first gained traction as it met the urgent need for rigour and quantifiability in privacy protection ( Dwork et al. , 2006 ) . In short , DP bounds the change in the distribution of outputs of a query made on a dataset under an alteration of one data point . The following definition formalizes this notion . Definition 1.1 ( Dwork et al. , 2006 ) A randomized algorithm A , taking a dataset consisting of individuals as its input , is ( , δ ) -differentially private if , for any pair of datasets S and S′ that differ in the record of a single individual and any event E , P [ A ( S ) ∈ E ] ≤ e P [ A ( S′ ) ∈ E ] + δ . When δ = 0 , A is called -differentially private ( -DP ) . While the notion of ( , δ ) -DP has wide applications ( Dankar & El Emam , 2012 ; Erlingsson et al. , 2014 ; Cormode et al. , 2018 ; Hassan et al. , 2019 ) , there are a few notable drawbacks to this framework . One is the poor interpretability of ( , δ ) -DP : unlike other concepts in machine learning , DP should not remain a black box . Privacy guarantees are intended for human interpretation and so must be understandable by the users it affects and by regulatory entities . A second drawback is ( , δ ) -DP ’ s inferior composition properties and lack of versatility . Here , “ composition ” refers to the ability for DP properties to be inherited when DP algorithms are combined and used as building blocks . As an example , the training of deep learning models involves gradient evaluations and weight updates : each of these steps can be treated as a building block . It is natural to expect that a DP learning algorithm can be built using differentially-private versions of these components . However , the DP composition properties can not generally be well characterized within the framework of ( , δ ) -DP , leading to very loose composition theorems . To overcome the drawbacks of ( , δ ) -DP , numerous variants have been developed , including the hypothesis-testing-based f -DP ( Wasserman & Zhou , 2010 ; Dong et al. , 2019 ) , the momentsaccountant-based Rényi DP ( Mironov , 2017 ) , as well as concentrated DP and its variants ( Bun & Steinke , 2016 ; Bun et al. , 2018 ) . Despite their very different perspectives , all of these DP variants can be fully characterized by an infinite union of ( , δ ) -DP guarantees . In particular , there is a two-way embedding between f -DP and the infinite union of ( , δ ) -DP guarantees : any guarantee provided by an infinite union of ( , δ ) -DP can be fully characterized by f -DP and vice visa ( Dong et al. , 2019 ) . Consequently , f -DP has the versatility to treat all of the above notions as special cases . In addition to its versatility , f -DP is more interpretable than other DP paradigms because it considers privacy protection from an attacker ’ s perspective . Under f -DP , an attacker is challenged with the hypothesis-testing problem H0 : the underlying dataset is S versus H1 : the underlying dataset is S′ and given an output of an algorithm A , where S and S′ are neighbouring datasets . The harder this testing problem is , the less privacy leakageA has . To see this , consider the dilemma that the attacker is facing . The attacker must reject either H0 or H1 based on the given output of A : this means the attacker must select a subset R0 of Range ( A ) and reject H0 if the sampled output is in R0 ( or must otherwise reject H1 ) . The attacker is more likely to incorrectly reject H0 ( in a type I error ) when R0 is large . Conversely , if R0 is small , the attacker is more likely to incorrectly reject H1 ( in a type II error ) . We say that an algorithm A is f -DP if , for any α ∈ [ 0 , 1 ] , no attacker can simultaneously bound the probability of type I error below α and bound the probability of type II error below f ( α ) . The function f is called a trade-off function and controls the strength of the privacy protection . The versatility afforded by f can be unwieldy in practice . Although f -DP is capable of handling composition and can embed other notions of differential privacy , it is not convenient for representing safety levels as a curve amenable to human interpretation . Gaussian differential privacy ( GDP ) , as a parametric family of f -DP guarantees , provides a balance between interpretability and versatility . GDP guarantees are parameterized by a single value µ and use the trade-off function f ( α ) = Φ ( Φ−1 ( 1− α ) − µ ) , where Φ is the cumulative distribution function of the standard normal distribution . With this choice of f , the hypothesis-testing problem faced by the attacker is as hard as distinguishing between N ( 0 , 1 ) and N ( µ , 1 ) on the basis of a single observation . Aside from its visual interpretation , GDP also has unique composition theorems : the composition of a µ1- and µ2-GDP algorithm is , as expected , √ µ21 + µ 2 2-GDP . This property can be easily generalized to n-fold composition . GDP also has has a special central limit theorem implying that all hypothesis-testing-based definitions of privacy converge to GDP in terms of a limit in the number of compositions . Readers are referred to Dong et al . ( 2019 ) for more information . The benefits of DP come with a price . As outlined in the definition of DP , any DP algorithm must be randomized . This randomization is usually achieved by perturbing the intermediate step or the final output via the injection of random noise . Because of the noise , a DP algorithm can not faithfully output the truth like its non-DP counterpart . To provide a higher level of privacy protection , a stronger utility compromise should be made . This leads to the paramount problem of the “ privacy– utility trade-off ” . Under the ( , δ ) -DP framework , this trade-off is often characterized in a form of σ = f ( , δ ) : to achieve ( , δ ) -DP , the utility parameter ( usually the scale of noise ) needs to be chosen as f ( , δ ) . We will further discuss this formulation with examples in the next section . 1.1 OUR CONTRIBUTION . The general goal of this paper is to provide both a deeper theoretical understanding of and powerful practical tools for the GDP framework . To this end , we start with the analysis of privacy profiles under the condition of traditional ( , δ ) -DP , where we point out an often-overlooked partial order on ( , δ ) -DP conditions via implication . We choose this as the starting point for our work because ignoring this partial order will lead to problematic asymptotic analysis and compromised utility . Next , we broken down the GDP into two parts : a head condition and a tail condition and define an efficient , tractable , and visualizable tool , called the Gaussian differential privacy transformation ( GDPT ) . We first used the GDPT on the identification of GDP algorithms . Through the intermediate property of GDPT that we develop , we find an easy-to-verify criterion that can distinguish GDP mechanisms from non-GDP mechanisms . For GDP algorithms , this criterion provides a lower bound for the privacy protection parameter µ . This criterion can help researchers widen the set of available GDP algorithms and also gives an interesting characterization of GDP without an explicit reference to the Gaussian distribution . Following the identification of a GDP algorithm , the logical next step is to measure the exact value of µ . By development of numerical properties of the GDPT , we give a method to narrow down possible values of an optimal µ with an arbitrarily small and quantifiable margin of error . Lastly , we give three more applications of our newly developed tools . In the first , we revisit some established ( , δ ) -DP algorithms and improve their utility by accounting for the overlooked partial order on ( , δ ) -DP conditions provided by logical implication . In the second application , we make a comparison between -DP and µ-GDP and find that any -DP algorithm must be also µ-GDP . In the last application , we discuss the effect of subsampling using the GDPT . 2 BACKGROUND . An algorithm can be ( , δ ) -DP for multiple pairs of and δ : the union of all such pairs provides a complete image of the algorithm under the ( , δ ) -DP framework . In particular , an ( , δ ) -DP mechanismA is also ( ′ , δ′ ) -DP for any ′ ≥ and any δ′ ≥ δ . The infinite union of ( , δ ) pairs can thus be represented as the smallest δ associated with each . This intuition is formulated as a privacy profile in Balle & Wang ( 2018 ) . The privacy profile corresponding to a collection of ( , δ ) -DP guarantees Ω is defined as the curve in [ 0 , ∞ ) × [ 0 , 1 ] separating the space of privacy parameters into two regions , one of which contains exactly the pairs in Ω . The privacy profile provides as much information as Ω itself . Many privacy guarantees and privacy notions , including ( , δ ) -DP , Rényi DP , f -DP , GDP , and concentrated DP , can be embedded into a family of privacy profile curves and fully characterized ( Balle et al. , 2020a ) . A privacy profile can be provided or derived by an algorithm ’ s designer or users . Before proceeding with detailed discussions , we first give three examples of DP algorithms that are used throughout the paper . The first example we consider is the Laplace mechanism , a classical DP mechanism whose prototype is discussed in the paper that originally defined the concept of differential privacy ( Dwork et al. , 2006 ) . The level of privacy that the Laplace mechanism can provide is determined by the scale b of the added Laplacian noise . Given a global sensitivity ∆ , the value of b needs to be chosen as f ( , 0 ) = ∆/ in order to provide an ( , 0 ) -DP guarantee . Despite its long history , the Laplace mechanism has remained in use and study in recent years ( Phan et al. , 2017 ; Hu et al. , 2019 ; Xu et al. , 2020 ; Li & Clifton , 2021 ) . Our second example is a family of algorithms in which a noise parameter has the form σ = A −1 √ log ( B/δ ) . Examples include • the goodness of fit algorithm ( Gaboardi et al. , 2016 ) , • noisy stochastic gradient descent and its variants ( Bassily et al. , 2014 ; Abadi et al. , 2016 ; Feldman et al. , 2018 ) , and • the one-shot spectral method and the one-shot Laplace algorithm ( Qiao et al. , 2021 ) . Our third example comes from the field of federated learning : given n users and the number of messages m , the invisibility cloak encoder algorithm ( ICEA ) from ( Ishai et al. , 2006 ) is ( , δ ) -DP if m > 10 log ( n/ ( δ ) ) ( Ghazi et al. , 2019 ) . See also ( Balle et al. , 2020b ; Ghazi et al. , 2020 ) for other analysis of ICEA . For figures and numerical demonstrations in this paper , we use b = 2/∆ for the Laplace mechanism ; A = 2 , B = 1 , and σ = 2 for the second example , which we refer to as SGD ; andm = 20 and n = 4 for the ICEA . We omit the internal details of these methods and focus on their privacy guarantees : other than for the classical Laplace mechanism , whose privacy profile is known ( Balle et al. , 2020a ) , privacy guarantees are given in the form of a privacy–utility trade-off equation σ = f ( , δ ) . Given σ , it is tempting to derive the privacy profile by inverting the trade-off equation ( i.e. , as δA ( ) = min { δ | σ = f ( , δ ) } ) . However , in most cases , a privacy profile naively derived in this way is not tight and will lead to a problematic asymptotic analysis , especially near the origin , because of a frequently overlooked partial order between ( , δ ) -DP conditions , which we discuss below . 3 PRIVACY PROFILES AND AN EXACT PARTIAL ORDER ON ( , δ ) -DP CONDITIONS An ( 0 , δ0 ) -DP algorithm is trivially ( , δ ) -DP for any ≥ 0 and δ ≥ δ0 . However , this statement does not give a full picture of the relationships between different ( , δ ) -DP conditions . Theorem 3.1 Assume that 0 ≥ 0 and 0 ≤ δ0 < 1 . The ( 0 , δ0 ) -DP condition implies ( , δ ) -DP if and only if δ ≥ δ0 + ( 1− δ0 ) ( e 0 − e ) +/ ( 1 + e 0 ) . Corollary 3.1 Assume that ≥ 0 ≥ 0 and δ ≤ [ ( 1 + e ) δ0 − e + e 0 ] / ( 1 + e 0 ) . An ( , δ ) -DP algorithm is ( 0 , δ0 ) -DP . Corollary 3.2 Let A be an ( 0 , δ0 ) -DP algorithm with the privacy profile δA . Then δA ( ) ≤ δ0 + ( 1− δ0 ) ( e 0 − e ) + 1 + e 0 . ( 1 ) We remark that the bound given in ( 1 ) is tight in the following sense : there is a specific ( 0 , δ0 ) -DP algorithm A such that δA ( ) is exactly δ0 + ( 1− δ0 ) ( e 0 − e ) +/ ( 1 + e 0 ) . Corollary 3.1 states the exact partial order of logical implication on ( , δ ) -DP conditions . Taking this partial order into account , the privacy profile derived from the naive inversion of the trade-off function can be refined into δA ( ) = min ( { δ | σ = f ( 0 , δ0 ) and δ ≥ δ0 + ( 1− δ0 ) ( e 0 − e ) + 1 + e 0 } ) . Intuitively , the refined privacy profile not only considers ( , δ ) -DP provided directly by the trade-off function but also takes all pairs ( , δ ) inferred by Corollary 3.1 . Theorem 3.2 can itself be used as a tool to improve some DP results : we will discuss this improvement in Section 5 . With this approach , we can derive the privacy profile of our second example to be δSGD ( ) = min ( δ2 + ( 1− δ2 ) ( e 2 − e ) +/ ( 1 + e 2 ) , exp { − 2 } ) , where 2 ≈ 1.187 and δ2 ≈ 0.244 . The privacy profile of our third example is δICEA ( ) = min ( δ3 + ( 1− δ3 ) ( e 3 − e ) +/ ( 1 + e 3 ) , K/ } ) , where K = 4/e2 , 3 ≈ 1.159 , and δ3 ≈ 0.468 . Notice that δSGD and δICEA share a similar form : both are the smaller of two terms , one derived from Corollary 3.1 from a particular ( , δ ) -DP pair and the other from inverting the trade-off function . This is no coincidence . We leave the derivation to Appendix B . We next show the connection between GDP and the privacy profile : briefly , Gaussian differential privacy can be characterized as an infinite union of ( , δ ) -DP conditions . Theorem 3.2 A mechanism is µ-GDP if and only if it is ( , δµ ( ) ) -DP for all ≥ 0 , where δµ ( ) = Φ ( − µ + µ 2 ) − e Φ ( − µ − µ 2 ) . ( 2 ) This result follows from properties of f -DP and is formulated as Corollary 2.13 in Dong et al . ( 2019 ) . Prior to this general form , a similar expression for a special case appeared in Balle & Wang ( 2018 ) . From the definition of the privacy profile , it follows immediately that an algorithm A with the privacy profile δA is µ-GDP if and only if δµ ( ) ≥ δA ( ) for all non-negative . However , this observation does not automatically lead to a meaningful way to identify µ-GDP algorithms . Before proceeding with an analysis of privacy profiles , we give a few visual examples in Figure 1 . The left pane illustrates the privacy profiles of our examples . That of the Laplace mechanism is derived in Balle et al . ( 2020a ) as Theorem 3 : given a noise parameter b and a global sensitivity ∆ , the privacy profile of the Laplace mechanism is δ ( ) = max ( 1 − exp { ε/2 −∆/ ( 2b ) } , 0 ) . For the second and the third examples , we compare the naive privacy profiles obtained by inverting the trade-off function and the refined privacy profiles . The refined and naive privacy profiles take on notably different values around = 0 . The inverted trade-off functions suggest that ( 0 , δ ) can not be achieved by any choice of parameter σ . However , this is clearly not true , considering Theorem 3.1 . As shown in the right pane of Figure 1 , the Laplace mechanism ’ s privacy profile is below the 2-GDP and 4-GDP curves but crosses the 1-GDP curve , indicating that the Laplace mechanism in this case is 2-GDP and 4-GDP but not 1-GDP . The ICEA curve intersects all of the displayed GDP curves , so the algorithm is not µ-GDP for µ ∈ { 1 , 2 , 4 } . It is hard to tell whether or not the SGD curve crosses the 1-GDP curve and we can not say if it will cross the 2-GDP or even the 4-GDP curve at a large value of . These examples illustrate that we can not draw conclusions simply by looking at a graph . A privacy profile is defined on [ 0 , ∞ ) , so it is hard to tell if an inequality is maintained as increases . Previous failures of ad hoc attempts at privacy have taught that privacy must be protected via tractable and objective means ( Narayanan & Shmatikov , 2006 ; 2008 ; Barbaro & T. Zeller , 2006 ) . Performing this check via numerical evaluation yields similar problems : we can not consider all values of on an infinite interval ( or even a finite one , for that matter ) . Turning to closed forms for privacy profiles and δµ is also difficult : even if a given privacy profile is easy to handle , δµ presents some technical hurdles . The profile δµ and Φ are transcendental with different asymptotic behaviors for different values of µ and . This is clear from the right pane of Figure 1 : near = 0 , δµ is concave for µ = 4 but convex for µ = 1 . As a further complication , both the first and second terms in the definition of δµ converge to 1 as → ∞ , but the difference between them vanishes . Subtracting good approximations of two nearby numbers may cause a phenomenon called catastrophic cancellation and lead to very bad approximations ( Malcolm , 1971 ; Cuyt et al. , 2001 ) . Due to the risk of catastrophic cancellation , a good approximation of Φ does not guarantee a good approximation of the GDP privacy profile . These problems make it difficult to tightly bound δµ by a function with a simple form . To address the problem of differing asymptotic behaviours , we define the following two notions . Definition 3.1 ( Head condition ) An algorithmA with the privacy profile δA is ( h , µ ) -head GDP if and only if δA ( ) ≤ δµ ( ) when ≤ h. Definition 3.2 ( Tail condition ) An algorithm A with the privacy profile δA is ( t , µ ) -tail GDP if and only if δA ( ) ≤ δµ ( ) when > t. In particular , we define A as ( +∞ , µ ) -tail GDP if A is ( t , µ ) -tail GDP for some t < +∞ and define A as ( t , +∞ ) -tail GDP if A is ( t , µ ) for some µ < +∞ . The head condition checks the µ-GDP condition for near zero and the tail condition checks the µGDP condition for far away from zero . As such , the combination of ( , µ ) -head GDP and ( , µ ) -tail GDP is equivalent to µ-GDP .

Summary: The paper develops tools for analyzing private algorithms in the framework of Gaussian Differential Privacy (GDP). GDP is a strengthening of approximate differential privacy, similar in spirit to concentrated differential privacy (CDP): both GDP and CDP capture the privacy guarantees of the Gaussian noise mechanism, but generalize them in different directions. GDP has a number of nice analytical properties but analyzing the GDP guarantees of an algorithm can be difficult, as it involves comparing two different functions on $[0,\infty)$. The paper proposes a method to reduce this problem to comparing a function against a fixed constant function, making numerical or analytical GDP analysis of a given algorithm easier.