Identification, Amplification and Measurement: A bridge to Gaussian Differential Privacy
Anonymous Author(s)
AffiliationAddress
Abstract
Gaussian differential privacy (GDP) is a single-parameter family of privacy notionsthat provides coherent guarantees to avoid the exposure of sensitive individualinformation.
Despite the extra interpretability and tighter bounds under composition GDP provides, many widely used mechanisms (e.g., the Laplace mechanism)inherently provide GDP guarantees but often fail to take advantage of this newframework because their privacy guarantees were derived under a different background.
In this paper, we study the asymptotic properties of privacy profiles anddevelop a simple criterion to identify algorithms with GDP properties.
We proposean efficient method for GDP algorithms to narrow down possible values of anoptimal privacy measurement, µ with an arbitrarily small and quantifiable marginof error.
For non GDP algorithms, we provide a post-processing procedure that canamplify existing privacy guarantees to meet the GDP condition.
As applications,we compare two single-parameter families of privacy notions, ϵ -DP, and µ -GDP,and show that all ϵ -DP algorithms are intrinsically also GDP.
Lastly, we showthat the combination of our measurement process and the composition theorem of
GDP is a powerful and convenient tool to handle compositions compared to thetraditional standard and advanced composition theorems.
Introduction
Recent years have seen explosive growth in the research and application of data-driven machinelearning.
While data fuels advancement in this unprecedented age of “big data”, concern for individualprivacy has deepened with the continued mining, transportation, and exchange of this new resource.
While expressions of privacy concerns can be traced back as early as 1969 [27] the concept of privacyis often perceived as “vague and difficult to get into a right perspective” [34].
Through its alluringconvenience and promise of societal prosperity, the use of aggregated data has long outstripped thecapabilities of privacy protection measures.
Indeed, early privacy protection protocols relied on thead hoc enforcement of anonymization and offered little to no protection against the exposure ofindividual data, as evidenced by the AOL search log and Netflix Challenge dataset controversies[30, 31, 6].
Differential privacy (DP) first gained traction as it met the urgent need for rigour and quantifiability inprivacy protection [14].
In short, DP bounds the change in the distribution of outputs of a query madeon a dataset under an alteration of one data point.
The following definition formalizes this notion.
Definition 1.1 [14] A randomized algorithm A , taking a dataset consisting of individuals as its input, is ( ϵ, δ ) -differentially private if, for any pair of datasets S and S ′
that differ in the record of a single individual and any event E ,
[Equation]
When δ = 0 , A is called ϵ -differentially private ( ϵ -DP).
While the notion of ( ϵ, δ ) -DP has wide applications [12, 16, 10, 21], there are a few notable drawbacksto this framework.
One is the poor interpretability of ( ϵ, δ ) -DP: unlike other concepts in machinelearning, DP should not remain a black box.
Privacy guarantees are intended for human interpretationand so must be understandable by the users it affects and by regulatory entities.
A second drawbackis ( ϵ, δ ) -DP’s inferior composition properties and lack of versatility.
Here, “composition” refers tothe ability for DP properties to be inherited when DP algorithms are combined and used as buildingblocks.
As an example, the training of deep learning models involves gradient evaluations and weightupdates: each of these steps can be treated as a building block.
It is natural to expect that a DP learningalgorithm can be built using differentially-private versions of these components.
However, the DPcomposition properties cannot generally be well characterized within the framework of ( ϵ, δ ) -DP,leading to very loose composition theorems.
To overcome the drawbacks of ( ϵ, δ ) -DP, numerous variants have been developed, including thehypothesis-testing-based f -DP [35, 13], the moments-accountant-based Rényi DP [28], as well asconcentrated DP and its variants [9, 8].
Despite their very different perspectives, all of these DPvariants can be fully characterized by an infinite union of ( ϵ, δ ) -DP guarantees.
In particular, there isa two-way embedding between f -DP and the infinite union of ( ϵ, δ ) -DP guarantees: any guaranteeprovided by an infinite union of ( ϵ, δ ) -DP can be fully characterized by f -DP and vice visa [13].
Consequently, f -DP has the versatility to treat all of the above notions as special cases.
In addition to its versatility, f -DP is more interpretable than other DP paradigms because it considers privacy protection from an attacker’s perspective.
Under f -DP, an attacker is challenged with the hypothesis-testing problem
H 0 : the underlying dataset is S versus
H 1 : the underlying dataset is S ′ and given output of an algorithm A , where S and S ′ are neighbouring datasets.
The harder thistesting problem is, the less privacy leakage
A has.
To see this, consider the dilemma that the attackeris facing.
The attacker must reject either H 0 or H 1 based on the given output of A : this means theattacker must select a subset R 0 of Range( A ) and reject H
if the sampled output is in R 0 (or mustotherwise reject H 1 ).
The attacker is more likely to incorrectly reject H
(in a type I error)
when
R 0 is large.
Conversely, if R 0 is small, the attacker is more likely to incorrectly reject H 1 (in a type
II error).
We say that an algorithm A is f -DP if, for any α ∈ [0 , 1] , no attacker can simultaneouslybound the probability of type I error below α and bound the probability of type II error below f ( α ) .
Such f is called a trade-off function and controls the strength of the privacy protection.
The versatility afforded by f can be unwieldy in practice.
Although f -DP is capable of handlingcomposition and can embed other notions of differential privacy, it is not convenient for representing safety levels as a curve amenable to human interpretation.
Gaussian differential privacy(GDP), as a parametric family of f -DP guarantees, provides a balance between interpretability andversatility.
GDP guarantees are parameterized by a single value µ and use the trade-off functionf ( α ) =
Φ (cid:0) Φ − 1 (1 − α ) − µ (cid:1) , where Φ is the cumulative distribution function of the standard normaldistribution.
With this choice of f , the hypothesis-testing problem faced by the attacker is as hardas distinguishing between N (0 , 1) and N ( µ, 1) on the basis of a single observation.
Aside fromits visual interpretation, GDP also has unique composition theorems: the composition of a µ 1 - andµ 2 -GDP algorithm is, as expected, (cid:112) µ 21 + µ 22 -GDP.
This property can be easily generalized to n -foldcomposition.
GDP also has a special central limit theorem implying that all hypothesis-testing-baseddefinitions of privacy converge to GDP in terms of a limit in the number of compositions.
Readersare referred to [13] for more information.
1.1 Outline
The goal of this paper is to provide a bridge between GDP and algorithms developed under other DPframeworks.
We start by presenting an often-overlooked partial order on ( ϵ, δ ) -DP conditions inducedby logical implication.
Ignoring this partial order will lead to problematic asymptotic analysis.
We then break down GDP into two parts: a head condition and a tail condition.
We show that thelatter, through a single limit of a mechanism’s privacy profile, is sufficient to distinguish between
GDP and non-GDP algorithms.
For GDP algorithms, this criterion also provides a lower bound for theprivacy protection parameter µ and can help researchers widen the set of available GDP algorithms.
This criterion furthermore gives an interesting characterization of GDP without an explicit referenceto the Gaussian distribution.
The next logical step is to measure the exact privacy performance.
Interestingly, while the binary“GDP or not” question can be answered solely by the tail, the actual performance of a DP algorithmis determined by the head.
We define and apply the Gaussian Differential Privacy Transformation(GDPT) to narrow the set of potential optimal values of µ with an arbitrarily small and quantifiablemargin of error.
We further provide procedure to adapt an algorithm to GDP or improve the privacyparameter when results from the GDP identification and measurement procedures are undesirable.
Lastly, we demonstrate additional applications of our newly developed tools.
We first make acomparison between DP and GDP and show that any ϵ -DP algorithm is automatically GDP.
We thenshow that the combination of our measurement process and the GDP composition theorem is a morepowerful and convenient tool for handling compositions relative to traditional composition theorems.
Privacy profiles and an exact partial order on ( ϵ, δ ) -DP conditions
The benefits of DP come with a price.
As outlined in the definition of DP, any DP algorithm must berandomized.
This randomization is usually achieved by perturbing the intermediate step or the finaloutput via the injection of random noise.
Because of the noise, a DP algorithm cannot faithfully outputthe truth like its non DP counterpart.
To provide a higher level of privacy protection, a stronger utilitycompromise should be made.
This leads to the paramount problem of the “privacy–utility trade-off”.
Under the ( ϵ, δ ) -DP framework, this trade-off is often characterized in a form of σ = f ( ϵ, δ ) : toachieve ( ϵ, δ ) -DP, the utility parameter (usually the scale of noise) needs to be chosen as f ( ϵ, δ ) .
Therefore, an algorithm can be ( ϵ, δ ) -DP for multiple pairs of ϵ and δ : the union of all such pairsprovides a complete image of the algorithm under the ( ϵ, δ ) -DP framework.
In particular, an ( ϵ, δ ) -DPmechanism
A is also ( ϵ ′ , δ ′ )
-DP for any ϵ ′ ≥ ϵ and any δ ′ ≥ δ .
The infinite union of ( ϵ, δ ) pairscan thus be represented as the smallest δ associated with each ϵ .
This intuition is formulated as aprivacy profile in [5].
The privacy profile corresponding to a collection of ( ϵ, δ ) -DP guarantees Ω isdefined as the curve in [0 , ∞ ) × [0 , 1] separating the space of privacy parameters into two regions,one of which contains exactly the pairs in Ω .
The privacy profile provides as much information as Ωitself.
Many privacy guarantees and privacy notions, including ( ϵ, δ ) -DP, Rényi DP, f -DP, GDP, andconcentrated DP, can be embedded into a family of privacy profile curves and fully characterized [3].
A privacy profile can be provided or derived by an algorithm’s designer or users.
Before proceeding with detailed discussions, we first give three examples of DP algorithms that areused throughout the paper.
The first example we consider is the Laplace mechanism, a classical DPmechanism whose prototype is discussed in the paper that originally defined the concept of differentialprivacy [14].
The level of privacy that the Laplace mechanism can provide is determined by thescale b of the added Laplacian noise.
Given a global sensitivity ∆ , the value of b needs to be chosenas f
( ϵ, 0) =
∆ /ϵ
in order to provide an ( ϵ, 0) -DP guarantee.
Despite its long history, the Laplacemechanism has remained in use and study in recent years [32, 22, 36, 25].
Our second example isa family of algorithms in which a noise parameter has the form σ =
Aϵ − 1 (cid:112) log( B/δ ) .
Examplesinclude: the goodness of fit algorithm [18], noisy stochastic gradient descent and its variants [7, 1, 17]and the one-shot spectral method and the one-shot Laplace algorithm [33].
Our third example comesfrom the field of federated learning: given n users and the number of messages m , the invisibilitycloak encoder algorithm (ICEA) from [23] is ( ϵ, δ ) -DP
if m > 10 log( n/ ( ϵδ )) [20].
See also [4, 19]for other analysis of ICEA.
For figures and numerical demonstrations in this paper, we use b = 2 / ∆ for the Laplace mechanism;
A = 2 , B = 1 , and σ = 2 for the second example, which we refer to as SGD; and m = 20and n = 4 for the ICEA.
We omit the internal details of these methods and focus on their privacyguarantees: other than for the classical Laplace mechanism, whose privacy profile is known [3],privacy guarantees are given in the form of a privacy–utility trade-off equation σ =
g ( ϵ, δ ) .
Givenσ , it is tempting to derive the privacy profile by inverting g (i.e., as δ A ( ϵ ) =
min { δ | σ = g ( ϵ, δ ) } )
because an ( ϵ 0 , δ 0 ) -DP algorithm is trivially ( ϵ, δ ) -DP for any ϵ ≥ ϵ 0 and δ ≥ δ 0 .
However, inmost cases, a privacy profile naively derived in this way is not tight and will lead to a problematicasymptotic analysis, especially near the origin, because of a frequently overlooked partial orderbetween ( ϵ, δ ) -DP conditions below.
Theorem 2.1 Assume that ϵ 0 ≥ 0 and 0 ≤ δ 0 < 1 .
The ( ϵ 0 , δ 0 ) -DP condition implies ( ϵ, δ ) -DP ifand only if δ ≥ δ 0 + (1 − δ 0 )
( e ϵ 0 − e ϵ )
+ / (1 + e ϵ 0 ) .
Theorem 2.1 states the exact partial order of logical implication on ( ϵ, δ ) -DP conditions.
Though not being explicitly discussed in this form in previous literature on DP, this partial order can be implicitly derived from other results (e.g. proposition 2.11 of [13]).
Taking this partial order into account, the privacy profile derived from the naive inversion of the trade-off function can be refined into
[Equation]
Intuitively, the refined privacy profile not only considers ( ϵ, δ ) -DP provided directly by the trade-offfunction but also takes all pairs ( ϵ, δ ) inferred by corollary 2.1.
See figure 1 for comparsion beforeand after this refinement.
The identification of GDP algorithms
We next show the connection between GDP and the privacy profile: briefly, Gaussian differentialprivacy can be characterized as an infinite union of ( ϵ, δ ) -DP conditions.
Theorem 3.1 ([Corollary 2.13 [13])
A mechanism is µ -GDP if and only if it is ( ϵ, δ µ ( ϵ )) -DP for allϵ ≥ 0 , where
[Equation]
This result follows from properties of f -DP.
Prior to this general form, a expression for a special caseappeared in [5].
From the definition of the privacy profile, it follows immediately that an algorithm Awith the privacy profile δ A is µ -GDP if and only if δ µ ( ϵ ) ≥ δ A ( ϵ ) for all non-negative ϵ .
However,this observation does not automatically lead to a meaningful way to identify GDP algorithms.
Before proceeding with an analysis of privacy profiles, we give a few visual examples in Figure 1.
The left side of1 illustrates the privacy profiles of our examples.
That of the Laplace mechanism isderived in [3] as Theorem 3: given a noise parameter b and a global sensitivity ∆ , the privacy profileof the Laplace mechanism is δ ( ϵ ) = max(1 − exp { ε/ 2 − ∆ / (2 b ) } , 0) .
For the second and the thirdexamples, we compare the naive privacy profiles obtained by inverting the trade-off function withthe refined privacy profiles.
The refined and naive privacy profiles take on notably different valuesaround ϵ = 0 .
The inverted trade-off functions suggest that (0 , δ ) cannot be achieved by any choiceof parameter σ .
However, this is clearly not true, considering Theorem 2.1.
As shown in right side of Figure 1, the Laplace mechanism’s privacy profile is below the 2-GDP and4-GDP curves but crosses the 1-GDP curve, indicating that the Laplace mechanism in this case is2-GDP and 4-GDP but not 1-GDP.
The ICEA curve intersects all of the displayed GDP curves, so thealgorithm is not µ -GDP for µ ∈ { 1 , 2 , 4 } .
It is hard to tell whether or not the SGD curve crosses the1-GDP curve and we cannot say if it will cross the 2-GDP or even the 4-GDP curve at a large value ofϵ .
These examples illustrate that we cannot draw conclusions simply by looking at a graph.
A privacyprofile is defined on [0 , ∞ ) , so it is hard to tell if an inequality is maintained as ϵ increases.
Previousfailures of ad hoc attempts at privacy have taught that privacy must be protected via tractable andobjective means [30, 31, 6].
Performing this check via numerical evaluation yields similar problems: we cannot consider all valuesof ϵ on an infinite interval (or even a finite one, for that matter).
Turning to closed forms for privacyprofiles and δ µ is also difficult: even if a given privacy profile is easy to handle, δ µ presents sometechnical hurdles.
The profile δ µ and Φ are transcendental with different asymptotic behaviors fordifferent values of µ and ϵ .
This is clear from the Figure 1: near ϵ = 0 , δ µ is concave for µ = 4 butconvex for µ = 1 .
As a further complication, both the first and second terms in the definition of δ µconverge to 1 as ϵ → ∞ , but the difference between them vanishes.
Subtracting good approximationsof two nearby numbers may cause a phenomenon called catastrophic cancellation and lead to verybad approximations [26, 11].
Due to the risk of catastrophic cancellation, a good approximation ofΦ does not guarantee a good approximation of the GDP privacy profile.
These problems make itdifficult to tightly bound δ µ by a function with a simple form.
To address the problem of differing asymptotic behaviours, we define the following two notions.
Definition 3.1 (Head condition)
An algorithm A with the privacy profile δ A is ( ϵ h , µ ) -head GDP ifand only if δ A ( ϵ ) ≤ δ µ ( ϵ ) when ϵ ≤ ϵ h .
Definition 3.2 (Tail condition)
An algorithm A with the privacy profile δ A is ( ϵ t , µ ) -tail GDP if andonly if δ A ( ϵ ) ≤ δ µ ( ϵ ) when ϵ > ϵ t .
The head condition checks the µ -GDP condition for ϵ near zero and the tail condition checks theµ -GDP condition for ϵ far away from zero.
As such, the combination of ( ϵ, µ ) -head GDP and( ϵ, µ ) -tail GDP is equivalent to µ -GDP.
For now, we put the exact value of µ aside and consider onlythe qualitative question of how to identify a GDP algorithm by its privacy profile.
The followingtheorem answers this question.
Theorem 3.2 An algorithm A is GDP if and only if A is ( ϵ, µ ) -tail GDP for any finite ϵ and µ .
Interestingly, only the tail condition figures into the identification problem.
The reason for thisstems from theorem 2.1.
Any nontrivial ( ϵ, δ ) -DP algorithm must be (0 , δ ) -DP for some δ < 1 andtherefore must satisfy a head condition for some sufficiently large µ .
The only problem left is thetail.
However, it is not possible to check whether δ ( ϵ ) < δ µ ( ϵ ) for all values of ϵ .
To circumvent thisissue, we present a key lemma that underlies much of the theoretical analysis in this section and maycontinue to be useful in future developments.
Lemma 3.1 Define ˜ δ µ ( ϵ ) =
µe − a 2 / 2 √ 2 πa 2 , where a = − ϵµ + µ 2 .
It follows that lim
ϵ
→ + ∞ δ µ ( ϵ )
˜
δ µ ( ϵ ) = 1 .
Using the key lemma, a condition for identifying GDP algorithms is simple to formulate:
Theorem 3.3 Let µ t = (cid:113) lim
ϵ
→
+ ∞ ϵ 2 − 2log δ A ( ϵ ) .
An algorithm A with the privacy profile δ A ( ϵ )
isµ -GDP
if and only if µ t < ∞ and µ is no smaller than µ t .
Theorems 3.2 and 3.3 give a useful criterion characterizing GDP and deepen our understanding of
GDP.
Putting the exact value of µ aside, a GDP algorithm must provide an infinite union of ( ϵ, δ ) -DPconditions, where δ must be O ( e − ϵ 2 ) as ϵ → ∞ .
Refer to Appendices B.3 for proofs of Theorems.
The Gaussian differential privacy transformation
While the binary “GDP or not” question can be answered solely by the tail condition, the actualperformance of a DP algorithm is determined by the value of its privacy profile for small values ofϵ : intuitively, all ( ϵ t , µ ) -tail conditions are weaker than the corresponding ϵ t -DP condition, and thelatter provides almost no privacy when ϵ t > 10 .
A more detailed discussion will be presented in 4.2.
To solve the measurement problem, we first propose a new tool—the Gaussian differential privacytransformation (GDPT).
Definition 4.1 (GDPT) Let f be a non-increasing, non-negative function defined on [0 , + ∞ ) satisfying f (0) ≤ 1 .
The Gaussian differential privacy transformation (GDPT) of f is the function G fmapping [0 , ∞ ) to [0 , ∞ ) such that G f ( ϵ ) =
µ GDP ( ϵ, f ( ϵ )) , where µ GDP ( x, y ) is the implicit functiondefined by the equation δ µ ( x ) = y .
We highlight two critical features of the GDPT.
• The GDPT is order preserving: if f ( ϵ ) ≥ g ( ϵ ) , then G f ( ϵ ) ≥ G g ( ϵ ) .
• The GDPT of δ µ is G δ µ ( ϵ ) =
µ , a constant function.
The first of these two features derive from the monotonicity of δ µ ( ϵ ) .
Given a fixed µ , δ µ ( ϵ ) is astrictly decreasing continuous function of ϵ .
Given a fixed ϵ , δ µ ( ϵ ) is a strictly increasing continuousfunction of µ .
Therefore, µ GDP ( x, y ) is an increasing function of y : this leads to the order-preservingproperty.
The second property follows immediately from the definition of µ GDP .
By taking advantage of the order-preserving property, direct comparisons between δ µ and δ A areno longer necessary: instead, it is sufficient to compare their corresponding GDPTs.
Furthermore,appealing to the second property above, we need only compare G A to the constant function µ .
Thefollowing theorems formalize this insight.
Corollary 4.1 An algorithm A with the privacy profile δ A is µ -GDP if and only if µ ≥ sup( { G A ( ϵ ) |ϵ ∈ [0 , ∞ ) } ) .
Theorem 4.1 An algorithm A with the privacy profile δ A is ( ϵ h , µ ) -head GDP or ( ϵ t , µ ) -tail
GDPif and only if µ ≥ sup( { G A ( ϵ ) | ϵ ∈ [0 , ϵ h ]) or µ ≥ sup( { G A ( ϵ ) | ϵ ∈ ( ϵ t , ∞ )) , respectively.
Without the above results, we would be forced to search through a large family of functions for a singleδ µ that never crosses δ A anywhere on [0 , ∞ ) and has µ as small as possible.
Now, with Theorem4.1, we need only consider one function: the GDPT of δ A .
The tightest value µ is sup ϵ { G A ( ϵ ) } .
Now we revisit our previous three examples for which the limit in Theorem 3.3 is 0 , (cid:112) 1 / 2 , and + ∞ ,respectively.
From these evaluations, we can conclude that the Laplace mechanism and SGD are
GDP and that the privacy profile of the ICEA algorithm crosses every µ -GDP curve regardless ofhow large µ is, indicating that the ICEA algorithm is not GDP.
Left side of figure 2 shows the GDPTs of the three examples considered in this paper.
All three
GDPTs converge to a finite value as ϵ → 0 + .
This can be attributed to the fact that any algorithmproviding some non-trivial ( ϵ, δ ) -DP guarantee is (0 , δ ) -DP for some δ ∈ [0 , 1) .
For larger values ofϵ , the GDPT of the Laplace mechanism takes on a constant value of 0 , the GDPT of SGD convergesto a value that is approximately 0 .
7 , and the GDPT of the ICEA seems to be diverging.
Theseobservations are consistent with the values of 0 , (cid:112) 1 / 2 , and ∞ obtained from Theorem 3.3.
Oncean algorithm is confirmed to be GDP via Theorems 3.2 and 3.3, it is natural to be interested in theexact level of privacy protection, quantified by µ .
Nonetheless, plots are only good for visualizationand are not sufficient proof when verifying GDP.
We still need objective and tractable methods forobtaining bounds on GDPTs.
4.1 Measuring the head
Following the intuition outlined by definition 3.1 and 3.2, we decompose the GDP condition into headand tail conditions and first focus on finding µ such that A is ( ϵ, µ ) -head GDP.
Without additionalknowledge, finding sup { G A ( ϵ ) | ϵ ∈ [0 , ϵ h ] } , even for a finite ϵ h , seems computationally infeasible.
To solve this problem, we take advantage of the fact that µ GDP has a uniformly bounded partialderivative.
Theorem
[Equation]
With this result we can now only conduct the search on a list of ϵ as follows.
Theorem 4.3 Given ϵ h ≥ 0 , let D =
ϵ h /n and x i = iD + ϵ h for i ∈ { 0 , . . . , n + 1 } .
The GDPT of A , denoted by G A ( ϵ ) , is bounded between the two staircase functions
[Equation]
G
[Equation]
A
[Equation]
[Equation]
Specifically,
[Equation]
Refer to Appendix B.5 and B.6 for proofs of Theorem 4.2 and 4.3, respectively.
For any ϵ h < + ∞ , we can now bound any GDPT G A to any precision on [0 , ϵ h ] without fullpointwise evaluation because G A is bounded between G + A and G −A and each staircase function takeson only finitely many values.
The inequalities in (2) provide a viable way to estimate max G A ( ϵ ) :max i ∈{ 0 ,...,n } G −A ( x i ) and max i ∈{ 0 ,...,n +1 } G + A
( x i ) can be computed by choosing the maximum 256 value among 2 n + 2 evaluations of µ
GDP .
For the completeness, we provide a detailed description ofan efficient algorithm with a time complexity of O ( ϵ h /D
+ log 2 ( D ) log( ϵ h )) in Appendix E.
4.2 Understanding the tail
With Theorem 4.3, one can verify ( ϵ h , µ ) -head GDP conditions for arbitrarily large ϵ h and anarbitrarily precise approximation of µ .
While the error in µ can be quantified by D , one gap remains:ϵ h can be arbitrarily large but can never truly be + ∞ .
In this subsection, we discuss the gap between( ϵ h , µ ) -head GDP and true GDP (which is equivalent to (+ ∞ , µ ) -head GDP).
Before giving asolution, we intuitively illustrate the gap between ( ϵ h , µ ) -head GDP and true GDP.
Consider thefollowing two cases:
• GDP with catastrophic failure, where with probability 1 − p , A 1 functions properly asµ -GDP, with probability p , A 1 malfunctions and discloses the entire dataset; and• head-GDP with ϵ -DP, where A 2 is both ( ϵ h , µ ) -head GDP and ( ϵ h , 0) -DP.
The head GDP privacy guarantee lies strictly between those of A 1 and A 2 : specifically, δ A 1 ( ϵ ) <
δ ( ϵ ) <
δ A 2 ( ϵ ) .
In practice, µ is rarely above six in GDP and ϵ is rarely above 10 in ϵ -DP becausemore-extreme values provide almost no privacy protection [13].
If we verify the head condition upto ϵ h = 100 (which is not difficult because the time required for verification grows linearly) andtake µ = 6 , then p = δ µ ( ϵ h ) will be on the order of 10 − 43 .
Also, DP guarantee for ϵ this large israrely considered to provide real protection.
Hence, we conclude that the gap won’t make any notabledifference in practice with a proper choice of µ and ϵ h .
4.3 Amplification
In some cases, one may still wish to theoretically mend the gap discussed in the last subsection.
Thiscan be achieved by adding extra steps to perturb the output of the algorithm (i.e., via post-processing).
We propose the following “clip and rectify” procedure that can turn any ( ϵ h , µ ) -head GDP algorithminto a µ -GDP algorithm at the cost of some utility.
Theorem 4.4 Let A be an ( ϵ h , µ ) -head GDP algorithm with a numeric output.
Assume that −∞ <y − < y
+ < + ∞ .
Define C ( y ) = max(min
( y, y + ) , y − ) and R ( z )
=
z + v , where v is sampled from
Laplace( b ) with b =
( y + − y − )
/ϵ h .
Then R ◦ C ◦
A is µ -GDP.
Refer to Appendix B.7 for a proof of Theorem 4.4.
We remark that, in order to minimize the utilityloss, the bounds y − and y + should be properly or dynamically chosen and the head condition shouldbe verified to a value of ϵ h that is as large as possible.
On the other hand, the performance ( µ ) of a GDP algorithm may be bottlenecked by the value ofits privacy profile near the origin.
This problem can be remedied by subsample pre-processing, theimpact of which on privacy profiles has been thoroughly examined in [3].
The resulting privacyprofile is explicitly given in Theorems 8–10 of [3].
With the help of the GDPT, we can select differentsubsample ratios and measure µ .
For instance, the Laplace example in this paper was originally1 .
80 -GDP.
If we introduce a 50% - or 10% - Poisson subsampling before the Laplace mechanism, µwill be reduced to 0 .
98 or 0 . 28 , respectively.
Refer to F.2 for a complete graph of the new GDPTs.
While one could turn to other algorithms or design a new GDP mechanism in unfavourable cases wherea candidate algorithm is incompatible with GDP from the start, rectifying these incompatibilitiesvia pre- and post-processing may be more effective and efficient.
This is especially true in caseswhere raw data is not easily accessible.
In other cases, the DP mechanism might be inaccessible.
This is particularly common for users of proprietary software: while they are not able to identify andchange the algorithm distributed in binary code, users can still control sensitive information by onlyapproving a subset for release.
Applications
5.1 The Gaussian nature of ϵ -DP
and the Laplace mechanism
By our previous analysis of the GDPT, we know that being GDP means that a privacy profile hasa quickly vanishing tail (i.e., δ ( ϵ ) must be O ( e − ϵ 2 ) ).
It is remarkable that another single parameterfamily of DP conditions, the ϵ -DP conditions, is also a property that pertains to the tail of privacyprofiles.
For any ϵ 0 -DP algorithm, the privacy profile must be exactly 0 after ϵ 0 .
This suggests thatϵ -DP is stronger than GDP.
Next, we will quantify this intuition using the tools we developed above.
By Theorem 2.1, we know if A is ϵ 0 -DP, then in the worst case, δ A ( ϵ ) =
( e ϵ 0 − e ϵ )
+ / (1 + e ϵ 0 ) .
We consider the GDPT of δ A , denoted by G A .
It is easy to see that, for ϵ ≥ ϵ 0 ,
G A ( ϵ ) =
: we need only consider ϵ ∈ [0 , ϵ 0 ) .
Let G δ A ( ϵ ) be denoted by µ ϵ .
Using using the partial derivative of G A derived in Appendix B.5, we know that ∂∂ϵ G δ A ( ϵ )
= 311 √ 2 π exp (cid:110)(cid:0) µ 2 ϵ + 2 ϵ (cid:1) 2 / (8 µ 2 ϵ ) (cid:111) (cid:104) Φ( − µ 2 ϵ +
ϵ 2 µ ϵ ) − Φ( − µ 0 2 ) (cid:105) .
Then sign( ∂∂ϵ G δ A ( ϵ ) )
=
sign( µ ϵ − µ 0 −2 ϵ/µ 0 ) .
We can conclude that µ ϵ ≤ µ 0 and, further, that G A ( ϵ ) is strictly decreasing on [0 , ϵ 0 ) .
By
Theorem 4.1, we know that A is µ 0 -GDP.
This finding can be more generally formulated as thefollowing theorem.
Theorem 5.1 Any ( ϵ, 0)
-DP
algorithm is also µ -GDP for µ = − 2Φ − 1 (1 / (1 + e ϵ )) ≤ (cid:112) π/ 2 ϵ .[13]
pointed out that the DP guarantees of the Laplace mechanism are stronger than those correspondingly provided by ϵ -DP.
We reaffirm this difference by showing that it still exists under the GDPframework.
The Laplace mechanism satisfies µ -GDP for µ smaller than the bound given in Theorem5.1.
The GDPTs presented in Appendix F.1 illustrate this difference.
5.2 Handling composition with GDP
In practice, it is rare for a dataset to go through DP algorithms only once.
Multiple statistics may be ofinterest or one statistic may require multiple inquiries to acquire.
DP algorithms applied to the samedataset multiple times are usually still DP but with worse privacy parameters.
Composition theoremsquantitatively trace privacy loss and provide a privacy parameter for the ensemble.
However, notonly is exact composition an intrinsically (#P-)hard problem [29], but the conclusions of compositiontheorems are also often problematic.
Take traditional ( ϵ, δ ) -DP
as an example.
[24] gives an optimalcomposition theorem, but the composition of two ( ϵ, δ ) -DP algorithms cannot be characterized Method δ
[Table]
under the ( ϵ, δ ) -DP framework.
This result damages interpretability because the representation ofa composition will no longer be in two parameters.
This type of flaw is the major motivation fora GDP characterization of algorithms derived under other DP frameworks.
The composition of
GDP algorithms is easy, exact, and closed: the composition of a µ 1 - and µ 2 -GDP algorithm issimply (cid:112) µ 21 + µ 22 -GDP.
GDP also has a special central limit theorem which implies that, for allprivacy definitions that retain hypothesis testing with proper scaling, the privacy guarantee of acomposition converges to GDP in the limit.
In this subsection, we demonstrate that GDP is a powerfultool for composition by unifying other notions under the GDP framework and then using the GDPcomposition theorem.
As baselines, we select basic composition [14], advanced composition [15]and Rényi-DP [28].
We consider the 50-fold composition of 0 . 2 -DP algorithms.
In this setting, basic composition ispessimistic and says that the composition will be 10 -DP, which means there is next to no privacyguarantee.
According to corollary 1 of [28], the bound given by RDP is even more loose.
Refer to
Figure 3 for the results of other theorems.
We next consider composition using the proposed measurement method.
According to Theorem 5.1,a 0 .
2 -DP algorithm is 0 .
2505 -GDP.
If the algorithm is the Laplace mechanism, then the algorithm in Appendix E can tighten µ to 0 .
To compute µ for a 50-fold composition, we simply multiplythe original µ by √ 50 .
The result is 1 .
771 -GDP (or 1 . 691 -GDP for the Laplace mechanism).
Inthis case, distinguishing two neighbouring datasets is as hard as distinguishing between N (0 , 1) and
N (1 . 771 , 1) on the basis of a single observation.
In this particular case, the ground truth can be derived from the optimal composition theorem [24].
We present the results from the optimal composition theorem in Table 1 and Figure 3 for comparison,but we do not consider the optimal composition theorem to be generally superior because the groundtruth is not easy to compute and because the former method is not as interpretable and only works foralgorithms whose DP guarantees are fixed at ( ϵ, δ ) .
Conclusion and Future Work
In this paper, we provided both an analytic perspective of and engineering tools for the GDPframework.
By using the new notions we proposed, we devised solutions to three aspects of GDP:identification, amplification, and measurement.
The developments in this paper suggest numerous interesting directions for future work.
First, themore refined methods can be derived to expand the toolbox of rectification for more versatility.
Second, the measurement procedure can be combined with the rectification procedure.
Incrementallyintroducing more pre- and post-processing steps and dynamically checking whether privacy guaranteesare already satisfactory can also be explored.
Lastly, the idea underlying the GDPT can be generalizedto other parameterized DP notions like CDP or RDP to enrich tractability and visualizability in the DP literature.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] The limitations are given asconditions of theorems and properties.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
This work
is a development of mathematical tools for the DP framework and does not have a
immediate real-world impact.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] The complete proofs
are provided in the appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] There are noexperimental results in this work.
Figures are only for demonstrative reasons as main
results are supported by proofs.
(b) Did you specify all the training details (e g., data splits, hyperparameters,
how they
were chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A]
There is no randomness involved.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
The amount of computing resource needed is negligible.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
No such data used.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
No such data used.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
No human subjects involved.
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]