Identification, Amplification and Measurement: A bridge to Gaussian Differential Privacy
Abstract
Gaussian differential privacy (GDP) is a single-parameter family of privacy notions that provides coherent guarantees to avoid the exposure of sensitive individual information.
Despite the extra interpretability and tighter bounds under composition GDP provides, many widely used mechanisms (e.g., the Laplace mechanism) inherently provide GDP guarantees but often fail to take advantage of this new framework because their privacy guarantees were derived under a different background.
In this paper, we study the asymptotic properties of privacy profiles and develop a simple criterion to identify algorithms with GDP properties.
We propose an efficient method for GDP algorithms to narrow down possible values of an optimal privacy measurement, µ with an arbitrarily small and quantifiable margin of error.
For non GDP algorithms, we provide a post-processing procedure that can amplify existing privacy guarantees to meet the GDP condition.
As applications, we compare two single-parameter families of privacy notions, ϵ -DP, and µ -GDP, and show that all ϵ -DP algorithms are intrinsically also GDP.
Lastly, we show that the combination of our measurement process and the composition theorem of
GDP is a powerful and convenient tool to handle compositions compared to the traditional standard and advanced composition theorems.
Introduction
Recent years have seen explosive growth in the research and application of data-driven machine learning.
While data fuels advancement in this unprecedented age of “big data”, concern for individual privacy has deepened with the continued mining, transportation, and exchange of this new resource.
While expressions of privacy concerns can be traced back as early as 1969 [27], the concept of privacy is often perceived as “vague and difficult to get into a right perspective” [34].
Through its alluring convenience and promise of societal prosperity the use of aggregated data has long outstripped the capabilities of privacy protection measures.
Indeed, early privacy protection protocols relied on the ad hoc enforcement of anonymization and offered little to no protection against the exposure of individual data, as evidenced by the AOL search log and Netflix Challenge dataset controversies [30, 31, 6].
Differential privacy (DP) first gained traction as it met the urgent need for rigour and quantifiability in privacy protection [14].
In short, DP bounds the change in the distribution of outputs of a query made on a dataset under an alteration of one data point.
The following definition formalizes this notion.
Definition 1.1 [14] A randomized algorithm A , taking a dataset consisting of individuals as its input, is ( ϵ, δ ) -differentially private if, for any pair of datasets S and S ′
that differ in the record of a single individual and any event E ,
[Equation]
When δ = 0 , A is called ϵ -differentially private ( ϵ -DP).
While the notion of ( ϵ, δ ) -DP has wide applications [12, 16, 10, 21], there are a few notable drawbacks to this framework.
One is the poor interpretability of ( ϵ, δ ) -DP: unlike other concepts in machine learning, DP should not remain a black box.
Privacy guarantees are intended for human interpretation and so must be understandable by the users it affects and by regulatory entities.
A second drawback is ( ϵ, δ ) -DP’s inferior composition properties and lack of versatility.
Here, “composition” refers to the ability for DP properties to be inherited when DP algorithms are combined and used as building blocks.
As an example, the training of deep learning models involves gradient evaluations and weight updates: each of these steps can be treated as a building block.
It is natural to expect that a DP learning algorithm can be built using differentially-private versions of these components.
However, the DP composition properties cannot generally be well characterized within the framework of ( ϵ, δ ) -DP, leading to very loose composition theorems.
To overcome the drawbacks of ( ϵ, δ ) -DP, numerous variants have been developed, including the hypothesis-testing-based f -DP [35, 13], the moments-accountant-based Rényi DP [28], as well as concentrated DP and its variants [9, 8].
Despite their very different perspectives, all of these DP variants can be fully characterized by an infinite union of ( ϵ, δ ) -DP guarantees.
In particular, there is a two-way embedding between f -DP and the infinite union of ( ϵ, δ ) -DP guarantees: any guarantee provided by an infinite union of ( ϵ, δ ) -DP can be fully characterized by f -DP and vice visa [13].
Consequently, f -DP has the versatility to treat all of the above notions as special cases.
In addition to its versatility, f -DP is more interpretable than other DP paradigms because it considers privacy protection from an attacker’s perspective.
Under f -DP, an attacker is challenged with the hypothesis-testing problem
H 0 : the underlying dataset is S versus
H 1 : the underlying dataset is S ′ and given output of an algorithm A , where S and S ′ are neighbouring datasets.
The harder this testing problem is, the less privacy leakage
A has.
To see this, consider the dilemma that the attacker is facing.
The attacker must reject either H 0 or H 1 based on the given output of A : this means the attacker must select a subset R 0 of Range( A ) and reject H
if the sampled output is in R 0 (or must otherwise reject H 1 ).
The attacker is more likely to incorrectly reject H
(in a type I error)
when
R 0 is large.
Conversely, if R 0 is small, the attacker is more likely to incorrectly reject H 1 (in a type
II error).
We say that an algorithm A is f -DP if, for any α ∈ [0 , 1] , no attacker can simultaneously bound the probability of type I error below α and bound the probability of type II error below f ( α ) .
Such f is called a trade-off function and controls the strength of the privacy protection.
The versatility afforded by f can be unwieldy in practice.
Although f -DP is capable of handling composition and can embed other notions of differential privacy, it is not convenient for representing safety levels as a curve amenable to human interpretation.
Gaussian differential privacy (GDP), as a parametric family of f -DP guarantees, provides a balance between interpretability and versatility.
GDP guarantees are parameterized by a single value µ and use the trade-off function f ( α ) =
Φ (cid:0) Φ − 1 (1 − α ) − µ (cid:1) , where Φ is the cumulative distribution function of the standard normal distribution.
With this choice of f , the hypothesis-testing problem faced by the attacker is as hard as distinguishing between N (0 , 1) and N ( µ, 1) on the basis of a single observation.
Aside from its visual interpretation, GDP also has unique composition theorems: the composition of a µ 1 - and µ 2 -GDP algorithm is, as expected, (cid:112) µ 21 + µ 22 -GDP.
This property can be easily generalized to n -fold composition.
GDP also has a special central limit theorem implying that all hypothesis-testing-based definitions of privacy converge to GDP in terms of a limit in the number of compositions.
Readers are referred to [13] for more information.
1.1 Outline
The goal of this paper is to provide a bridge between GDP and algorithms developed under other DP frameworks.
We start by presenting an often-overlooked partial order on ( ϵ, δ ) -DP conditions induced by logical implication.
Ignoring this partial order will lead to problematic asymptotic analysis.
We then break down GDP into two parts: a head condition and a tail condition.
We show that the latter, through a single limit of a mechanism’s privacy profile, is sufficient to distinguish between
GDP and non-GDP algorithms.
For GDP algorithms, this criterion also provides a lower bound for the privacy protection parameter µ and can help researchers widen the set of available GDP algorithms.
This criterion furthermore gives an interesting characterization of GDP without an explicit reference to the Gaussian distribution.
The next logical step is to measure the exact privacy performance.
Interestingly, while the binary “GDP or not” question can be answered solely by the tail, the actual performance of a DP algorithm is determined by the head.
We define and apply the Gaussian Differential Privacy Transformation (GDPT) to narrow the set of potential optimal values of µ with an arbitrarily small and quantifiable margin of error.
We further provide procedure to adapt an algorithm to GDP or improve the privacy parameter when results from the GDP identification and measurement procedures are undesirable.
Lastly, we demonstrate additional applications of our newly developed tools.
We first make a comparison between DP and GDP and show that any ϵ -DP algorithm is automatically GDP.
We then show that the combination of our measurement process and the GDP composition theorem is a more powerful and convenient tool for handling compositions relative to traditional composition theorems.
Privacy profiles and an exact partial order on ( ϵ, δ ) -DP conditions
The benefits of DP come with a price.
As outlined in the definition of DP, any DP algorithm must be randomized.
This randomization is usually achieved by perturbing the intermediate step or the final output via the injection of random noise.
Because of the noise, a DP algorithm cannot faithfully output the truth like its non DP counterpart.
To provide a higher level of privacy protection, a stronger utility compromise should be made.
This leads to the paramount problem of the “privacy–utility trade-off”.
Under the ( ϵ, δ ) -DP framework, this trade-off is often characterized in a form of σ = f ( ϵ, δ ) : to achieve ( ϵ, δ ) -DP, the utility parameter (usually the scale of noise) needs to be chosen as f ( ϵ, δ ) .
Therefore, an algorithm can be ( ϵ, δ ) -DP for multiple pairs of ϵ and δ : the union of all such pairs provides a complete image of the algorithm under the ( ϵ, δ ) -DP framework.
In particular, an ( ϵ, δ ) -DP mechanism A is also ( ϵ ′ , δ ′ )
-DP for any ϵ ′ ≥ ϵ and any δ ′ ≥ δ .
The infinite union of ( ϵ, δ ) pairs can thus be represented as the smallest δ associated with each ϵ .
This intuition is formulated as a privacy profile in [5].
The privacy profile corresponding to a collection of ( ϵ, δ ) -DP guarantees Ω is defined as the curve in [0 , ∞ ) × [0 , 1] separating the space of privacy parameters into two regions, one of which contains exactly the pairs in Ω .
The privacy profile provides as much information as Ω itself.
Many privacy guarantees and privacy notions, including ( ϵ, δ ) -DP, Rényi DP, f -DP, GDP, and concentrated DP, can be embedded into a family of privacy profile curves and fully characterized [3].
A privacy profile can be provided or derived by an algorithm’s designer or users.
Before proceeding with detailed discussions, we first give three examples of DP algorithms that are used throughout the paper.
The first example we consider is the Laplace mechanism, a classical DP mechanism whose prototype is discussed in the paper that originally defined the concept of differential privacy [14].
The level of privacy that the Laplace mechanism can provide is determined by the scale b of the added Laplacian noise.
Given a global sensitivity ∆ , the value of b needs to be chosen as f ( ϵ, 0) =
∆ /ϵ
in order to provide an ( ϵ, 0) -DP guarantee.
Despite its long history, the Laplace mechanism has remained in use and study in recent years [32, 22, 36, 25].
Our second example is a family of algorithms in which a noise parameter has the form σ =
Aϵ − 1 (cid:112) log( B/δ ) .
Examples include: the goodness of fit algorithm [18], noisy stochastic gradient descent and its variants [7, 1, 17] and the one-shot spectral method and the one-shot Laplace algorithm [33].
Our third example comes from the field of federated learning: given n users and the number of messages m , the invisibility cloak encoder algorithm (ICEA) from [23] is ( ϵ, δ ) -DP
if m > 10 log( n/ ( ϵδ )) [20].
See also [4, 19] for other analysis of ICEA.
For figures and numerical demonstrations in this paper, we use b = 2 / ∆ for the Laplace mechanism;
A = 2 , B = 1 , and σ = 2 for the second example, which we refer to as SGD; and m = and n = 4 for the ICEA.
We omit the internal details of these methods and focus on their privacy guarantees: other than for the classical Laplace mechanism, whose privacy profile is known [3], privacy guarantees are given in the form of a privacy–utility trade-off equation σ = g ( ϵ, δ ) .
Given σ , it is tempting to derive the privacy profile by inverting g (i.e., as δ A ( ϵ ) =
min { δ | σ = g ( ϵ, δ ) } )
because an ( ϵ 0 , δ 0 ) -DP algorithm is trivially ( ϵ, δ ) -DP for any ϵ ≥ ϵ 0 and δ ≥ δ 0 .
However, in most cases, a privacy profile naively derived in this way is not tight and will lead to a problematic asymptotic analysis, especially near the origin, because of a frequently overlooked partial order between ( ϵ, δ ) -DP conditions below.
Theorem 2.1 Assume that ϵ 0 ≥ 0 and 0 ≤ δ 0 < 1 .
The ( ϵ 0 , δ 0 ) -DP condition implies ( ϵ, δ ) -DP
if and only if δ ≥
[Equation]
+
[Equation]
+
[Equation] + [Equation] Theorem 2.1 states the exact partial order of logical implication on ( ϵ, δ ) -DP conditions.
Though not being explicitly discussed in this form in previous literature on DP, this partial order can be implicitly derived from other results (e.g. proposition 2.11 of [13]).
Taking this partial order into account, the privacy profile derived from the naive inversion of the trade-off function can be refined into
[Equation]
Intuitively, the refined privacy profile not only considers ( ϵ, δ ) -DP provided directly by the trade-off function but also takes all pairs ( ϵ, δ ) inferred by corollary 2.1.
See figure 1 for comparison before and after this refinement.
The identification of GDP algorithms
We next show the connection between GDP and the privacy profile: briefly, Gaussian differential privacy can be characterized as an infinite union of ( ϵ, δ ) -DP conditions.
Theorem 3.1 ([Corollary 2.13 [13])
A mechanism is µ -GDP if and only if it is ( ϵ, δ µ ( ϵ ))
-DP for all ϵ ≥ 0 , where
[Equation]
This result follows from properties of f -DP.
Prior to this general form, a expression for a special case appeared in [5].
From the definition of the privacy profile, it follows immediately that an algorithm A with the privacy profile δ A is µ -GDP if and only if δ µ ( ϵ ) ≥ δ A ( ϵ ) for all non-negative ϵ .
However, this observation does not automatically lead to a meaningful way to identify GDP algorithms.
Before proceeding with an analysis of privacy profiles, we give a few visual examples in Figure 1.
The left side of1 illustrates the privacy profiles of our examples.
That of the Laplace mechanism is derived in [3] as Theorem 3: given a noise parameter b and a global sensitivity ∆ , the privacy profile of the Laplace mechanism is δ ( ϵ ) =
max(1 − exp { ε/ 2 − ∆ / (2 b ) } , 0) .
For the second and the third examples, we compare the naive privacy profiles obtained by inverting the trade-off function with the refined privacy profiles.
The refined and naive privacy profiles take on notably different values around ϵ = 0 .
The inverted trade-off functions suggest that (0 , δ ) cannot be achieved by any choice of parameter σ .
However, this is clearly not true, considering Theorem 2.1.
As shown in right side of Figure 1, the Laplace mechanism’s privacy profile is below the 2-GDP and 4-GDP curves but crosses the 1-GDP curve, indicating that the Laplace mechanism in this case is 2-GDP and 4-GDP but not 1-GDP.
The ICEA curve intersects all of the displayed GDP curves, so the algorithm is not µ -GDP for µ ∈ { 1 , 2 , 4 } .
It is hard to tell whether or not the SGD curve crosses the 1-GDP curve and we cannot say if it will cross the 2-GDP or even the 4-GDP curve at a large value of ϵ .
These examples illustrate that we cannot draw conclusions simply by looking at a graph.
A privacy profile is defined on [0 , ∞ ) , so it is hard to tell if an inequality is maintained as ϵ increases.
Previous failures of ad hoc attempts at privacy have taught that privacy must be protected via tractable and objective means [30, 31, 6].
Performing this check via numerical evaluation yields similar problems: we cannot consider all values of ϵ on an infinite interval (or even a finite one, for that matter).
Turning to closed forms for privacy profiles and δ µ is also difficult: even if a given privacy profile is easy to handle, δ µ presents some technical hurdles.
The profile δ µ and Φ are transcendental with different asymptotic behaviors for different values of µ and ϵ .
This is clear from the Figure 1: near ϵ = 0 , δ µ is concave for µ = 4 but convex for µ = 1 .
As a further complication, both the first and second terms in the definition of δ µ converge to 1 as ϵ → ∞ , but the difference between them vanishes.
Subtracting good approximations of two nearby numbers may cause a phenomenon called catastrophic cancellation and lead to very bad approximations [26, 11].
Due to the risk of catastrophic cancellation, a good approximation of Φ does not guarantee a good approximation of the GDP privacy profile.
These problems make it difficult to tightly bound δ µ by a function with a simple form.
To address the problem of differing asymptotic behaviours, we define the following two notions.
Definition 3.1 (Head condition)
An algorithm A with the privacy profile δ A is ( ϵ h , µ ) -head GDP
if and only if δ A ( ϵ ) ≤ δ µ ( ϵ ) when ϵ ≤ ϵ h .
Definition 3.2 (Tail condition)
An algorithm A with the privacy profile δ A is ( ϵ t , µ ) -tail GDP if and only if δ A ( ϵ ) ≤ δ µ ( ϵ ) when ϵ > ϵ t .
The head condition checks the µ -GDP condition for ϵ near zero and the tail condition checks the µ -GDP condition for ϵ far away from zero.
As such, the combination of ( ϵ, µ ) -head GDP and ( ϵ, µ ) -tail GDP is equivalent to µ -GDP.
For now, we put the exact value of µ aside and consider only the qualitative question of how to identify a GDP algorithm by its privacy profile.
The following theorem answers this question.
Theorem 3.2 An algorithm A is GDP if and only if A is ( ϵ, µ ) -tail GDP for any finite ϵ and µ .
Interestingly, only the tail condition figures into the identification problem.
The reason for this stems from theorem 2.1.
Any nontrivial ( ϵ, δ ) -DP algorithm must be (0 , δ ) -DP for some δ < 1 and therefore must satisfy a head condition for some sufficiently large µ .
The only problem left is the tail.
However, it is not possible to check whether δ ( ϵ ) < δ µ ( ϵ ) for all values of ϵ .
To circumvent this issue, we present a key lemma that underlies much of the theoretical analysis in this section and may continue to be useful in future developments.
Lemma 3.1 Define ˜ δ µ ( ϵ ) =
µe − a 2 / 2 √ 2 πa 2 , where a = − ϵµ + µ 2 .
It follows that lim
ϵ
→ + ∞ δ µ ( ϵ )
˜
δ µ ( ϵ ) = 1 .
Using the key lemma above, a condition for identifying GDP algorithms is simple to formulate:
Theorem 3.3 Let µ t = (cid:113) lim
ϵ
→
+ ∞ ϵ 2 − 2log δ A ( ϵ ) .
An algorithm A with the privacy profile δ A ( ϵ ) is µ -GDP if and only if µ t < ∞ and µ is no smaller than µ t .
Theorems 3.2 and 3.3 give a useful criterion characterizing GDP and deepen our understanding of
GDP.
Putting the exact value of µ aside, a GDP algorithm must provide an infinite union of ( ϵ, δ ) -DP conditions, where δ must be O ( e − ϵ 2 ) as ϵ → ∞ .
Refer to Appendices B.3 for proofs of Theorems.
The Gaussian differential privacy transformation
While the binary “GDP or not” question can be answered solely by the tail condition, the actual performance of a DP algorithm is determined by the value of its privacy profile for small values of ϵ : intuitively, all ( ϵ t , µ ) -tail conditions are weaker than the corresponding ϵ t -DP condition, and the latter provides almost no privacy when ϵ t > 10 .
A more detailed discussion will be presented in 4.2.
To solve the measurement problem, we first propose a new tool—the Gaussian differential privacy transformation (GDPT).
Definition 4.1 (GDPT) Let f be a non-increasing, non-negative function defined on [0 , + ∞ ) satisfying f (0) ≤ 1 .
The Gaussian differential privacy transformation (GDPT) of f is the function G f mapping [0 , ∞ ) to [0 , ∞ ) such that G f ( ϵ ) =
µ GDP ( ϵ, f ( ϵ )) , where µ GDP ( x, y ) is the implicit function defined by the equation δ µ ( x ) = y .
We highlight two critical features of the GDPT.
• The GDPT is order preserving: if f ( ϵ ) ≥ g ( ϵ ) , then G f ( ϵ ) ≥ G g ( ϵ ) .
• The GDPT of δ µ is G δ µ ( ϵ )
= µ , a constant function.
The first of these two features derive from the monotonicity of δ µ ( ϵ ) .
Given a fixed µ , δ µ ( ϵ ) is a strictly decreasing continuous function of ϵ . Given a fixed ϵ , δ µ ( ϵ ) is a strictly increasing continuous function of µ .
Therefore, µ GDP ( x, y ) is an increasing function of y : this leads to the order-preserving property.
The second property follows immediately from the definition of µ GDP .
By taking advantage of the order-preserving property, direct comparisons between δ µ and δ A are no longer necessary: instead, it is sufficient to compare their corresponding GDPTs.
Furthermore, appealing to the second property above, we need only compare G A to the constant function µ .
The following theorems formalize this insight.
Corollary 4.1 An algorithm A with the privacy profile δ A is µ -GDP if and only if µ ≥ sup( { G A ( ϵ ) | ϵ ∈ [0 , ∞ ) } ) .
Theorem 4.1 An algorithm A with the privacy profile δ A is ( ϵ h , µ ) -head GDP or ( ϵ t , µ ) -tail GDP
if and only if µ ≥ sup( { G A ( ϵ ) | ϵ ∈ [0 , ϵ h ]) or µ ≥ sup( { G A ( ϵ ) | ϵ ∈ ( ϵ t , ∞ )) , respectively.
Without the above results, we would be forced to search through a large family of functions for a single δ µ that never crosses δ A anywhere on [0 , ∞ ) and has µ as small as possible.
Now, with Theorem 4.1, we need only consider one function: the GDPT of δ A .
The tightest value µ is sup ϵ { G A ( ϵ ) } .
Now we revisit our previous three examples for which the limit in Theorem 3.3 is 0 , (cid:112) 1 / 2 , and + ∞ , respectively.
From these evaluations, we can conclude that the Laplace mechanism and SGD are
GDP and that the privacy profile of the ICEA algorithm crosses every µ -GDP curve regardless of how large µ is, indicating that the ICEA algorithm is not GDP.
Left side of figure 2 shows the GDPTs of the three examples considered in this paper.
All three
GDPTs converge to a finite value as ϵ → 0 + .
This can be attributed to the fact that any algorithm providing some non-trivial ( ϵ, δ ) -DP guarantee is (0 , δ ) -DP for some δ ∈ [0 , 1) (by theorem 2.1).
For larger values of ϵ , the GDPT of the Laplace mechanism takes on a constant value of 0 , the GDPT of SGD converges to a value that is approximately 0 .
7 , and the GDPT of the ICEA seems to be diverging.
These observations are consistent with the values of 0 , (cid:112) 1 / 2 , and ∞ obtained from
Theorem 3.3.
Once an algorithm is confirmed to be GDP via Theorems 3.2 and 3.3, it is natural to be interested in the exact level of privacy protection, quantified by µ .
Nonetheless, plots are only good for visualization and are not sufficient proof when verifying GDP.
We still need objective and tractable methods for obtaining bounds on GDPTs.
4.1 Measuring the head
Following the intuition outlined by definition 3.1 and 3.2, we decompose the GDP condition into head and tail conditions and first focus on finding µ such that A is ( ϵ, µ ) -head GDP.
Without additional knowledge, finding sup { G A ( ϵ ) | ϵ ∈ [0 , ϵ h ] } , even for a finite ϵ h , seems computationally infeasible.
To solve this problem, we take advantage of the fact that µ GDP has a uniformly bounded partial derivative.
Theorem
[Equation]
The first half of the inequality above is no surprise to us: the GDP privacy measurement µ is expected to be larger when ϵ is larger.
However, the second half allow us to only conduct the search on a finite list of ϵ without the concern of spikes in between.
We formulate this insight as the following theorem:
Theorem 4.3 Given ϵ h ≥ 0 , let d
= ϵ h /n
and x i = id for i ∈ { 0 , . .
. , n + 1 } .
For ϵ ≤ ϵ h , the
GDPT of A , denoted by G A ( ϵ ) , is bounded between the two staircase functions
G −A ( ϵ ) =
n +1 (cid:88) i = µ GDP ( x i , δ A ( x i +1 ))
× 1 ϵ ∈
[x i , x i+1 ) and G + A ( ϵ ) = n +1 (cid:88) i = µ GDP ( x i +1 , δ A ( x i )) × 1 ϵ ∈
[x i , x i+1 ) .
Specifically,
[Equation]
Refer to Appendix B.5 and B.6 for proofs of Theorem 4.2 and 4.3, respectively.
For any ϵ h < + ∞ , we can now bound any GDPT G A to any precision on [0 , ϵ h ] without full pointwise evaluation because G A is bounded between G + A and G − A and each staircase function takes on only finitely many values.
For any c > 0 , the inequalities in (2) provide a viable way to bound max ϵ ∈ [0 ,ϵ h ] G A ( ϵ ) in an interval with a length no greater than 1 /c .
First, a binary search algorithm (algorithm 2 in Appendix D) can yield µ + and µ − such that µ − ≤ µ GDP ( ϵ, δ ) ≤ µ + and µ + − µ − < b .
For future references, we use µ + GDP ( ϵ, δ, b ) and µ − GDP ( ϵ, δ, b ) to represent such outputs of µ + and µ − , respectively.
Therefore, we can naively go thorough all
G −A ( x i ) and G + A ( x i ) .
By picking n = (cid:6) √ 8 cπϵ h (cid:7) + 1 and b = 12 c , the true gap between max G −A ( ϵ ) and max G + A ( ϵ ) is less than 12 c and the error margin of the binary search estimate the µ GDP is also 12 c .
Therefore, the overall gap is bounded by 1 c .
As for complexity, each binary search has a time complexity of O (log( c )) and the number of binary searches is 2 n +
=
O
( ϵ h c ) .
The overall time complexity of this naive approach is O
( ϵ h c log( c )) .
For a complete pseudocode of this naive approach, refer to algorithm 3 in Appendix D.
By leveraging some properties of µ GDP and shuffling, the expected number of binary searches needed can be reduced from linear ( 2 n + 2 ≈ cϵ h ) to logarithmic ( O (log( cϵ h ))) .
Such reduction will eliminate the logarithmic term in the time complexity from the naive algorithm.
The improved algorithm is given as Algorithm 1 below.
Input: δ A , ϵ h , µ t , c .
(Privacy profile, searching range ϵ h , reciprocal of error margin)
n ← (cid:6) √ 8 cπϵ h (cid:7) +d ← ϵ h n − 1 µ − ←µ + ←
S =
[0, 1, · · · , n + 1 ]
Shuffle S for i = 0 to n + 1 do x − ← S
[ i ] d x + ← ( S [ i ] + 1) d
if δ µ + ( x − ) <
δ A ( x + ) then µ + ← µ + GDP ( x − , δ A ( x + ) , 12 c )) end if if δ µ − ( x + ) < δ A ( x − ) then µ − ← µ − GDP ( x + , δ A ( x − ) , 12 c ))
end if end for
Output: µ − , µ + ( upper of µ ).
We remark that this algorithm also has better accuracy than the naive algorithm because the lower and upper bounds will be closer while maintaining coverage.
Refer to Appendix D for a detailed explanation of this algorithm.
4.2 Understanding the tail
With Theorem 4.3, one can verify ( ϵ h , µ ) -head GDP conditions for arbitrarily large ϵ h and an arbitrarily precise approximation of µ .
While the error in µ can be quantified by D , one gap remains: ϵ h can be arbitrarily large but can never truly be + ∞ .
In this subsection, we discuss the gap between ( ϵ h , µ ) -head GDP and actual GDP (which is equivalent to (+ ∞ , µ ) -head GDP).
Before giving a solution, we intuitively illustrate the gap between ( ϵ h , µ ) -head GDP and actual GDP.
Consider the following two cases:
• GDP with catastrophic failure, where with probability 1 − p , A 1 functions properly as µ -GDP, with probability p , A 1 malfunctions and discloses the entire dataset; and • head-GDP with ϵ -DP, where A 2 is both ( ϵ h , µ ) -head GDP and ( ϵ h , 0) -DP.
The head GDP privacy guarantee lies strictly between those of A 1 and A 2 : specifically, δ A 1 ( ϵ ) <
δ ( ϵ ) <
δ A 2 ( ϵ ) .
As an interpretation of this inequality, a head GDP privacy guarantee is safer than the original GDP guarantee but with a minuscule probability of failure, and when combined with a very weak ϵ -DP condition, the head GDP will be stronger than the actual GDP.
In practice, µ is rarely above six in GDP, and ϵ is rarely above 10 in ϵ -DP because more extreme values provide almost no privacy protection [13].
If we verify the head condition up to ϵ h = 100 (which is not difficult because the time required for verification grows linearly) and take µ = 6 , then p = δ µ ( ϵ h ) will be on the order of 10 − 43 .
Also, DP guarantee for ϵ this large is rarely considered to provide real protection.
Hence, we conclude that the gap will not make any notable difference in practice with a proper choice of µ and ϵ h .
4.3 Amplification
In some cases, one may wish to theoretically mend the gap discussed in the last subsection.
This can be achieved by adding extra steps to perturb the output of the algorithm (i.e., via post-processing).
We propose the following “clip and rectify” procedure that can turn any ( ϵ h , µ ) -head GDP algorithm into a µ -GDP algorithm at some utility cost.
Theorem 4.4 Let A be an ( ϵ h , µ ) -head GDP algorithm with a numeric output.
Assume that −∞ < y − < y
+ < + ∞ .
Define C ( y ) = max(min
( y, y + ) , y − ) and R ( z )
=
z + v , where v is sampled from
Laplace( b ) with b =
( y + − y − )
/ϵ h .
Then R ◦ C ◦
A is µ -GDP.
Refer to Appendix B.7 for a proof of Theorem 4.4.
We remark that, in order to minimize the utility loss, the bounds y − and y + should be properly or dynamically chosen and the head condition should be verified to a value of ϵ h that is as large as possible.
On the other hand, the performance ( µ ) of a GDP algorithm may be bottlenecked by the value of its privacy profile near the origin.
This problem can be remedied by subsample pre-processing, the impact of which on privacy profiles has been thoroughly examined in [3].
The resulting privacy profile is explicitly given in Theorems 8–10 of [3].
With the help of the GDPT, we can select different subsample ratios and measure µ .
For instance, the Laplace example in this paper was originally 1 . 80 -GDP.
If we introduce a 50% - or 10% - Poisson subsampling before the Laplace mechanism, µ will be reduced to 0 .
98 or 0 . 28 , respectively.
Refer to E.2 for a complete graph of the new GDPTs.
While one could turn to other algorithms or design a new GDP mechanism in unfavourable cases where a candidate algorithm is incompatible with GDP from the start, rectifying these incompatibilities via pre- and post-processing may be more effective and efficient.
This is especially true in cases where raw data is not easily accessible.
In other cases, the DP mechanism might be inaccessible.
This is particularly common for users of proprietary software.
While they cannot identify and change the algorithm distributed in binary code, users can still control sensitive information by only approving a subset for release.
Applications 5.1 The Gaussian nature of ϵ -DP and the Laplace mechanism
By our previous analysis of the GDPT, we know that being GDP means that a privacy profile has a quickly vanishing tail (i.e., δ ( ϵ ) must be O ( e − ϵ 2 ) ).
It is remarkable that another single parameter family of DP conditions, the ϵ -DP conditions, is also a property that pertains to the tail of privacy profiles.
For any ϵ 0 -DP algorithm, the privacy profile must be exactly 0 after ϵ 0 .
This suggests that ϵ -DP is stronger than GDP.
Next, we will quantify this intuition using the tools we developed above.
By Theorem 2.1, we know if A is ϵ 0 -DP, then in the worst case, δ A ( ϵ ) =
( e ϵ 0 − e ϵ )
+ / (1 + e ϵ 0 ) .
We consider the GDPT of δ A , denoted by G A .
It is easy to see that, for ϵ ≥ ϵ 0 , G A ( ϵ ) =
: we need only consider ϵ ∈ [0 , ϵ 0 ) .
Let G δ A ( ϵ ) be denoted by µ ϵ .
Using the partial derivative of G A derived in Appendix B.5, we know that ∂∂ϵ G δ A ( ϵ [-) = √ 2 π exp (cid:110)(cid:0) µ 2 ϵ + 2 ϵ (cid:1) 2 / (8 µ 2 ϵ ) (cid:111) (cid:104) Φ( − µ 2 ϵ +
-][Equation]
− Φ
[Equation] − µ [Equation] (cid:105) .
Then sign( ∂∂ϵ G δ A ( ϵ ) )
=
sign( µ ϵ − µ 0 − 2 ϵ/µ 0 ) .
We can conclude that µ ϵ ≤ µ 0 and, further, that G A ( ϵ ) is strictly decreasing on [0 , ϵ 0 ) .
By Theorem 4.1, we know that A is µ 0 -GDP.
This finding can be more generally formulated as the following theorem.
Theorem 5.1 Any ( ϵ, 0)
-DP
algorithm is also µ -GDP for µ = − 2Φ − 1 (1 / (1 + e ϵ )) ≤ (cid:112) π/ 2 ϵ .
[13] pointed out that the DP guarantees of the Laplace mechanism are stronger than those correspondingly provided by ϵ -DP.
We reaffirm this difference by showing that it still exists under the GDP framework.
The Laplace mechanism satisfies µ -GDP for µ smaller than the bound given in Theorem 5.1.
The GDPTs presented in Appendix E.1 illustrate this difference.
5.2 Handling composition with GDP
In practice, it is rare for a dataset to go through DP algorithms only once.
Multiple statistics may be of interest or one statistic may require multiple inquiries to acquire.
DP algorithms applied to the same dataset multiple times are usually still DP but with worse privacy parameters.
Composition theorems quantitatively trace privacy loss and provide a privacy parameter for the ensemble.
However, not only is exact composition an intrinsically (#P-)hard problem [29], but the conclusions of composition theorems are also often problematic.
Take traditional ( ϵ, δ ) -DP
as an example.
[24] gives an optimal composition theorem, but the composition of two ( ϵ, δ ) -DP algorithms cannot be characterized under the ( ϵ, δ ) -DP framework.
This result damages interpretability because the representation of
[Table]
a composition will no longer be in two parameters.
This type of flaw is the major motivation for a GDP characterization of algorithms derived under other DP frameworks.
The composition of
GDP algorithms is easy, exact, and closed: the composition of a µ 1 - and µ 2 -GDP algorithm is simply (cid:112) µ 21 + µ 22 -GDP.
GDP also has a special central limit theorem which implies that, for all privacy definitions that retain hypothesis testing with proper scaling, the privacy guarantee of a composition converges to GDP in the limit.
In this subsection, we demonstrate that GDP is a powerful tool for composition by unifying other notions under the GDP framework and then using the GDP composition theorem.
As baselines, we select basic composition [14], advanced composition [15] and Rényi-DP [28].
We consider the 50-fold composition of 0 . 2 -DP algorithms.
In this setting, the basic composition is pessimistic and says that the composition will be 10 -DP, which means there is next to no privacy guarantee.
According to corollary 1 of [28], the bound given by RDP is even looser.
Refer to Figure 3 for the results of other theorems.
We next consider composition using the proposed measurement method.
According to Theorem 5.1, a 0 .
2 -DP algorithm is 0 .
2505 -GDP.
If the algorithm is the Laplace mechanism, then the algorithm in Appendix D can tighten µ to 0 . 2391 .
To compute µ for a 50-fold composition, we simply multiply the original µ by √ 50 .
The result is 1 .
-GDP ( 1 . 691 for the Laplace mechanism).
In this case, distinguishing two neighbouring datasets is as hard as distinguishing between N (0 , 1) and
N (1 . 771 , 1) on the basis of a single observation.
In this particular case, the ground truth can be derived from the optimal composition theorem [24].
We present the results from the optimal composition theorem in Table 1 and Figure 3 for comparison, but we do not consider the optimal composition theorem to be generally superior because the ground truth is not easy to compute and because the former method is not as interpretable and only works for algorithms whose DP guarantees are fixed at ( ϵ, δ ) .
However, by applying the GDPT, the privacy guarantee of the optimal composition theorem can be summarised as 1 . 420 -GDP .
Compared to the central limit theorem in [13] which yields µ = √ 2 (with an unknown asymptotic approximation error) in the same setting, the tractable numerical procedure of GDPT provides a satisfying result.
Conclusion and Future Work
In this paper, we provided both an analytic perspective of and engineering tools for the GDP framework.
By using the new notions we proposed, we devised solutions to three aspects of GDP: identification, amplification, and measurement.
The developments in this paper suggest numerous interesting directions for future work.
First, the more refined methods can be derived to expand the toolbox of rectification for more versatility.
Second, the measurement procedure can be combined with the rectification procedure.
Incrementally introducing more pre- and post-processing steps and dynamically checking whether privacy guarantees are already satisfactory can also be explored.
Lastly, the idea underlying the GDPT can be generalized to other parameterized DP notions like CDP or RDP to enrich tractability and visualizability in the DP literature.