Title: Unified Enhancement of Privacy Bounds for Mixture Mechanisms via $f$-Differential Privacy
Venue: NeurIPS 2023
Accepted: yes
Keywords: Differential privacy

$f$-DP

mixture mechanisms

shuffling

differentially private gradient descent
Abstract: Differentially private (DP) machine learning algorithms incur many sources of randomness, such as random initialization, random batch subsampling, and shuffling. However, such randomness is difficult to take into account when proving differential privacy bounds because it induces mixture distributions for the algorithm's output that are difficult to analyze. 
This paper focuses on improving privacy bounds for shuffling models and one-iteration  differentially private gradient descent (DP-GD) with random initializations using $f$-DP. 
We derive a closed-form expression of the trade-off function for shuffling models that outperforms the most up-to-date results based on $(\epsilon,\delta)$-DP.
Moreover, we investigate the effects of random initialization on the privacy of one-iteration DP-GD. 
Our numerical computations of the trade-off function indicate that random initialization can enhance the privacy of DP-GD.
Our analysis of $f$-DP guarantees for these mixture mechanisms relies on an inequality for trade-off functions introduced in this paper. This inequality implies the joint convexity of $F$-divergences. 
Finally, we study an $f$-DP analog of the advanced joint convexity of the hockey-stick divergence related to $(\epsilon,\delta)$-DP  and apply it to analyze the privacy of mixture mechanisms.

Unified Enhancement of Privacy Bounds for Mixture Mechanisms via f -Differential Privacy

Abstract

Differentially private (DP) machine learning algorithms incur many sources of randomness, such as random initialization, random batch subsampling, and shuffling. However, such randomness is difficult to take into account when proving differential privacy bounds because it induces mixture distributions for the algorithm's output that are difficult to analyze. This paper focuses on improving privacy bounds for shuffling models and one-iteration differentially private gradient descent (DP-GD) with random initializations using f -DP. We derive a closed-form expression of the trade-off function for shuffling models that outperforms the most up-to-date results based on ( /epsilon1 , δ ) -DP. Moreover, we investigate the effects of random initialization on the privacy of one-iteration DP-GD. Our numerical computations of the trade-off function indicate that random initialization can enhance the privacy of DP-GD. Our analysis of f -DP guarantees for these mixture mechanisms relies on an inequality for trade-off functions introduced in this paper. This inequality implies the joint convexity of F -divergences. Finally, we study an f -DP analog of the advanced joint convexity of the hockey-stick divergence related to ( /epsilon1 , δ ) -DP and apply it to analyze the privacy of mixture mechanisms.

1 Introduction

Differential privacy (DP, [16, 17]) is a rigorous mathematical framework for ensuring data privacy and has become a cornerstone of privacy-preserving data analysis over the past two decades. DP has found widespread applications in various data science fields, such as machine learning [12, 6, 44],

query answering [18, 15], and synthetic data generation [37, 48, 29, 28]. A randomized mechanism is considered differentially private if the outputs of two neighboring datasets that differ in at most one element are indistinguishable from each other. The closeness of these outputs can be measured in various ways, resulting in the definition of ( /epsilon1 , δ ) -DP in [16] and its various relaxations.

The distinguishability between the outputs can be measured by statistical divergences. For example, ( /epsilon1 , δ ) -DP is associated with the so-called hockey-stick divergence [35]. Another divergence relevant to differential privacy is the Rényi divergence [20, 38] which leads to Rényi DP [30, 10] and concentrated DP [11]. In addition to divergence-based DP, a hypothesis testing perspective on differential privacy was proposed in [43]. More recently, [14] established f -DP for differential privacy where the privacy is measured by the trade-off function of type I and type II errors.

In real-world applications of differential privacy, including differentially private machine learning, it is common to analyze the privacy budget of mechanisms that involve mixture distributions, where the mixture is introduced by stochastic components in the algorithm. Examples of such mechanisms include sub-sampled mechanisms [4, 52, 40, 31], shuffled mechanisms [13, 22, 23], and variants of the differentially private stochastic gradient descent (DP-SGD) algorithm [1, 9, 26, 3, 45] that involves random initialization and multiple rounds of mini-batch sampling. Recently, privacy amplification by iteration [24] has drawn much attention as it can be used to analyze the privacy bounds for DP-SGD [45, 2]) which leads to tighter privacy bounds compared to classical analysis based on the composition theorem [33, 39, 50].

While mixture mechanisms are essential in differentially private machine learning, the absence of an f -DP guarantee for their analysis remains a significant challenge. Moreover, existing divergencebased DP bounds for most of these mechanisms are not tight. This is primarily because the complex distribution resulting from the mixture makes it challenging to accurately quantify privacy guarantees. In order to illustrate this perspective, we consider the examples of shuffling models and DP gradient descent (DP-GD) with random initialization, as follows.

In shuffling models, each user's data record is locally privatized using a local DP algorithm [21]. Subsequently, a curator shuffles the dataset containing all users' data. The shuffling procedure introduces additional mixtures of binomial noise [22], thereby potentially amplifying the privacy provided by the local randomizer. Shuffling is commonly employed in machine learning algorithms for batch generation [45, 47]. To deal with this mixture, Hoeffding's inequality was used in previous literature [22, 23] that leads to the loss of information. Using f -DP in this paper, we derive an exact analytical trade-off function for the mixture of binomial distributions which is sharp.

In deep learning, random initialization is usually adopted in the stochastic gradient descent to enhance the performance of deep neural networks [36]. Intuitively, the inherent randomness introduced by initialization should contribute to the privacy amplification of DP-GD. However, Rényi differential privacy (DP) falls short in quantitatively measuring this randomness, even when applied to the simplest linear model. In this paper, we demonstrate how f -DP can effectively evaluate and quantify this inherent randomness from initialization.

Our contributions. This paper makes a two-fold contribution. Firstly, we propose a unified theory to analyze the privacy of mixture mechanisms within the framework of f -DP. Precisely, we derive an f -DP inequality for mixture distributions which implies the joint convexity of F -divergences for any convex F . We name this result the "joint concavity of trade-off functions", as it is a lower bound for trade-off functions. The tightness of the joint concavity is also investigated. Moreover, we propose the "advanced joint concavity of trade-off functions" which is an f -DP analog of the advanced joint convexity of the hockey-stick divergence and results in sharper bounds in certain cases.

Building on our inequality, we have refined the privacy analysis of both shuffling models and DP-GD with random initialization using f -DP. Specifically, for shuffling models, we obtain trade-off functions in a closed-form representation, leading to tighter bounds compared to existing state-of-the-art results based on ( /epsilon1 , δ ) -DP. As for DP-GD, given the challenges in the trajectory analysis of multi-step iterations, we have chosen to explore a more straightforward one-iteration DP-GD. We demonstrate that using random initialization significantly enhances the privacy of the output from a single iteration.

2 Preliminaries on differential privacy

Let D = { z i } n i =1 ⊂ Z be a fixed dataset of size n . Consider a randomized algorithm A : Z n → S that maps a dataset D to A ( D ) in some probability space S . Differential privacy requires that the change of one element in a dataset has a restricted impact on the output of A . Mathematically, we say A satisfies ( /epsilon1 , δ ) -DP for some /epsilon1 ≥ 0 and 0 ≤ δ ≤ 1 if

for any event S ∈ S and any neighboring datasets D 0 and D 1 . When δ = 0 , we simply call ( /epsilon1 , 0) -DP as /epsilon1 -DP. Based on the definition, we see that for small values of /epsilon1 and δ , it is challenging to distinguish between D 0 and D 1 based on the outputs of A ( D 0 ) and A ( D 1 ) , as the distribution of A ( D 0 ) closely resembles that of A ( D 1 ) .

The definition of ( /epsilon1 , δ ) -DP corresponds to the hockey-stick divergence. Let P and Q be two distributions with probability density functions (pdfs) p and q , respectively. The hockey-stick divergence between P and Q is defined by H γ ( P ‖ Q ) = ∫ ( p ( x ) -γ q ( x )) + dx for γ ≥ 1 with ( · ) + = max { 0 , ·} . With a little bit abuse of notations, in this paper, we define the divergence (or the trade-off function) between two random variables as the divergence (or the trade-off function) between their distributions. Then, a mechanism A is ( /epsilon1 , δ ) -DP if and only if H e /epsilon1 ( A ( D 0 ) ‖ A ( D 1 )) ≤ δ for any neighboring datasets D 0 and D 1 , which also implies H e /epsilon1 ( A ( D 1 ) ‖ A ( D 0 )) ≤ δ .

The Rényi-DP (RDP) is defined based-on the Rényi divergence. The Rényi divergence of order ˜ α > 1 between P and Q is given by

˜ For ˜ α = 1 or + ∞ , R 1 or R ∞ is the limit of R ˜ α as ˜ α tends to 1 or + ∞ . A mechanism A is said to satisfy ( α , /epsilon1 ) -RDP if R α ( A ( D 0 ) ‖ A ( D 1 )) ≤ /epsilon1 for any neighboring D 0 and D 1 .

˜ ˜ The distinguishability between A ( D 0 ) and A ( D 1 ) can be quantified using hypothesis testing, which aligns with the concept of f -DP. Consider a hypothesis testing problem H 0 : P v.s. H 1 : Q and a rejection rule φ ∈ [0 , 1] . We define the type I error as α φ = E P [ φ ] , which is the probability that we reject the null hypothesis H 0 by mistake. The type II error β φ = 1 -E Q [ φ ] is the probability that we accept the alternative H 1 wrongly.

The trade-off function T ( P, Q ) is the minimal type II error at level α of the type I error, that is,

We say a mechanism A satisfies f -DP if T ( A ( D 0 ) , A ( D 1 )) ≥ f for any neighboring datasets D 0 and D 1 . In particular, A is said to satisfy µ -GDP if it is G µ -DP, where G µ ( x ) = Φ ( Φ -1 (1 -x ) -µ ) , for µ ≥ 0 , is the Gaussian trade-off function with Φ being the cumulative distribution function (cdf) of N (0 , 1) . A is considered to be more private if the corresponding trade-off function takes larger values. When A achieves perfect privacy and A ( D 0 ) and A ( D 1 ) become completely indistinguishable, the trade-off function is Id( x ) = 1 -x . Consequently, for any trade-off function f , we have f ≤ Id .

We say a trade-off function is symmetric if T ( P, Q ) = T ( Q,P ) . Note that a trade-off function f may not necessarily be symmetric. But one can symmetrize it as shown in [14]. The symmetrization of a trade-off function will be used when we analyze the shuffled mechanisms.

3 Joint concavity of trade-off functions

Let { P i } i m =1 and { Q i } i m =1 be two sequences of probability distributions. Denote the probability density functions (pdfs) of P i and Q i as p i and q i , respectively. Consider the mixture distributions P w and Q w with pdfs p w = ∑ m i =1 w i p i and q w = ∑ m i =1 w i q i , where the weight w = ( w 1 , · · · , w m ) is such that w i ≥ 0 and ∑ m i =1 w i = 1 . The following lemma is to bound the trade-off function T ( P w , Q w ) . Upon finalizing this paper, we noted that Lemma 3.1 and Proposition 3.2 appeared independently in another paper [42, Theorem 8], where they served different applications.

Lemma 3.1 (Joint concavity of trade-off functions) . For two mixture distributions P w and Q w , it holds

The main idea of the proof is to make the mixture distributions more distinguishable by releasing the indices. Precisely, for X ∼ P w and Y ∼ Q w , let X | I be a random variable such that X | I = i ∼ P i with I being the indices, i.e., P [ I = i ] = w i . Let ( X | I, I ) be a random variable where we observe both X | I and the indices I . Then, the right hand side of Lemma 3.1 is the trade-off function T (( X | I, I ) , ( Y | I, I )) between two joint distributions. This is a lower bound for the trade-off function between mixture distributions because ( X | I, I ) → X is a data-independent post-processing procedure that only removes the observation of indices I , and DP is immune to post-processing [14, 19].

where α i ( t, c ) = P X ∼ P i [ q i p i ( X ) > t ] + c P X ∼ P i [ q i p i ( X ) = t ] is the type I error for testing P i v.s. Q i using the likelihood ratio test and α ( t, c ) = ∑ m i =1 w i α i ( t, c ) .

Under the setting of f -DP, we usually require that the trade-off function is symmetric. The symmetry of the trade-off function in Lemma 3.1 is guaranteed by the following proposition.

Proposition 3.2. Suppose that for each i , T ( P i , Q i ) is a symmetric trade-off function. Then the trade-off function T (( X | I, I ) , ( Y | I, I )) is symmetric.

The joint convexity of F -divergences plays an important role in the analysis of divergence-based DP for mixture mechanisms [4, 22]. We now show that Lemma 3.1 is an extension of the joint convexity of F -divergences, including the scaled exponentiation of the Rényi divergence and the hockey-stick divergence, to trade-off functions. A trade-off function is always convex and is thus differentiable almost everywhere. Thus, without loss of generality, we consider f i that is differentiable, symmetric, with f i (0) = 1 .

Proposition 3.3 (An application of Lemma 3.1 to the F -divergences) . Let D F ( P ‖ Q ) = ∫ F ( p ( x ) /q ( x )) q ( x ) dx be an F -divergence between any two distributions P and Q with some convex F . Then, for f i = T ( P i , Q i ) , we have

Conversion from a trade-off function to F -divergences is straightforward using Section B in [14]. However, conversion from an F -divergence to a trade-off function is highly non-trivial. In fact, F -divergence is an integral of a functional of the trade-off function over the whole space while Lemma 3.1 holds pointwisely, which is a local property. This explains why the divergence-based DP is not as informative as f -DP since some information is lost due to the integration.

4 Privacy analysis of the shuffled mechanisms

In this section, we explore the f -DP analysis of shuffled mechanisms. Drawing upon [22, 23], the shuffling procedure incorporates a mixture of binomial noise. This noise can be tightly bounded using our f -DP inequality for mixture distributions.

4.1 Theoretical privacy guarantee

In shuffling models, the record of each user is privatized by some local randomizer (such as a randomized response mechanism [41]) and all records are then shuffled by a curator. Mathematically, consider a dataset D = { z i } n i =1 ⊆ Z of size n and each data point z i is privatized by an local randomizer A 0 : Z : → ˜ Z that satisfies /epsilon1 0 -DP. Then, the mechanism A : Z n → ˜ Z n that maps D to ˜ D = {A 0 ( z i ) } n i =1 is /epsilon1 0 -DP. A shuffler A Shu ffl e takes the privatized dataset ˜ D as input and applies a uniformly random permutation to ˜ D , which introduces the mixture of binomial noise to A and results in privacy amplification.

As noted in [23], the shuffling procedure introduces mixtures of binomial distributions. More specifically, the outputs generated by the shuffled mechanism for two neighboring datasets result from post-processing random variables X ∼ P and Y ∼ Q with P = (1 -w ) P 0 + wQ 0 and Q = (1 -w ) Q 0 + wP 0 , where the weight w = 1 e /epsilon1 0 +1 , and the distributions P 0 and Q 0 are defined as ( A + 1 , C -A ) ∼ P 0 , and ( A,C -A + 1) ∼ Q 0 with A ∼ Binom( C, 1 / 2) and C ∼ Binom( n -1 , 2 / ( e /epsilon1 0 +1)) . It is easy to see that P 0 is the mixture of { ( A i +1 , i -A i ) } n -1 i =0 with weights w 0 i := P [ C = i ] and Q 0 is the mixture of { ( A i , i -A i +1) } n -1 i =0 with the same weights. In this context, Binom( k, p ) is a binomial distribution with parameters k ∈ N and p ∈ [0 , 1] and each A i is distributed as Binom( i, 1 / 2) . Advancing our analysis, we adopt the joint concavity, as outlined in Lemma 3.1, to establish a lossless bound for the trade-off function T ( P 0 , Q 0 ) .

Proposition 4.1. Let F i be the distribution function of Binom( i, 1 / 2) and let w 0 i = P [ C = i ] for C ∼ Binom( n -1 , 2 / ( e /epsilon1 0 +1)) . Then, we have T ( P 0 , Q 0 ) is a piecewise linear function with for each knot α ( t ) = ∑ n -1 i =0 w 0 i α i ( t ) := ∑ n -1 i =0 w 0 i F i ( i -i +1 t +1 ) .

Remark. Proposition 4.1 holds with equality and the bound for T ( P 0 , Q 0 ) is sharp.

Before stating our results for T ( P, Q ) , we define some notations related to f -DP. For a function g : R → R , let g ∗ ( y ) := max x { xy -g ( x ) } be its convex conjugate. For a trade-off function f , let C ( f ) = min { f, f -1 } ∗∗ be its symmetrization, where f -1 is the left inverse function of f , i.e., f -1 ◦ f ( x ) = x .

Theorem 4.2. The shuffled mechanism A Shu ffl e ◦ A is C ( f Shu ffl e ) -DP. Here f Shu ffl e ( α ( t )) is a piecewise linear function where each knot α ( t ) has the form

with F i being the distribution function of Binom( i, 1 / 2) and w 0 i = P [ C = i ] for C ∼ Binom( n -1 , 2 / ( e /epsilon1 0 +1)) , and the value of f Shu ffl e at a knot α ( t ) is with w = 1 1+ e /epsilon1 0 and Id( x ) = 1 -x being the identity trade-off function.

Remark. The bound in Theorem 4.2 is near-optimal. In fact, the proof of Theorem 4.2 is based on a post-processing procedure in [23], joint concavity (Proposition 4.1), and advanced joint concavity (Proposition 6.4). The post-processing procedure is sharp for specific mechanisms, such as the randomized response mechanism, as shown by Theorem 5.2 and Theorem 5.3 in [23]. Proposition 4.1 holds with equality and is optimal. The advanced joint concavity, which is an f -DP analog of the advanced joint convexity in [4], is optimal for specific distributions. Compared to existing analysis of shuffled mechanisms (e.g., [23]), the main advantage of using f -DP is that we avoid the use of Hoeffding's inequality and the Chernoff bound to bound the distance between P 0 and Q 0 in Proposition 4.1, which is adopted in [22, 23] and leads to loose bounds, to bound the mixture of binomial distributions. Moreover, Theorem 3.2 in [23] holds with an assumption /epsilon1 0 ≤ log ( n 8 log(2 / δ ) -1 ) , which is removed by using f -DP in our paper.

To convert f -DP to ( /epsilon1 , δ ) -DP, we use the primal-dual perspective in [14] and obtain the following Corollary.

Corollary 4.3. Let l ( t ) := -∑ n -1 i =0 w 0 i p i ( /floorleft i +1 -i +1 t +1 /floorright ) ∑ n -1 i =0 w 0 i p i ( /floorleft i -i +1 t +1 /floorright ) with p i being the probability mass function of Binom( i, 1 / 2) . Then, we have is ( /epsilon1 , δ ( /epsilon1 )) -DP for any /epsilon1 > 0 with

A Shu ffl e ◦ A f -DP

(a)

/epsilon1

0

= 5

.

444

>

4

.

444

0.00

0.25

0.50

0.75

1.00

0.00

0.25

0.50

0.75

1.00

Type I error

(b)

/epsilon1

0

= 4

.

444

(c)

/epsilon1

0

= 3

.

444

1.00-

0.75-

tradeoff

(e,8)

0.50-

f-DP

Id

0.25-

0.00-

0.00

0.25

0.50

0.75

1.00

Typelerror

1.00

0.75

error

=

0.50

Type

0.25-

0.00-

0.00

0.25

0.50

0.75

1.00

Type|error

4.2 Numerical results and comparisons

To the best of our understanding, the leading privacy analysis for shuffled mechanisms is given in [23]. In this section, we compare the privacy bounds from our Theorem 4.2 and Corollary 4.3 with those found in Theorem 3.2 of [23]. Additionally, we assess the tightness of our bound against the empirical lower bounds obtained through binary search.

Specifically, Figure 1 presents a comparison of the trade-off function derived from our Theorem 4.2 to that of [23]. This comparison clearly illustrates that f -DP offers tighter privacy bounds, given that its trade-off function aligns closer to the identity trade-off function.

In our Table 1, we compare the values of δ f -DP ( /epsilon1 ) , as derived from Corollary 4.3 with δ ( /epsilon1 ) in [23]. The results indicate that δ f -DP ( /epsilon1 ) is significantly smaller than δ ( /epsilon1 ) .

In Table 2, we present /epsilon1 f -DP alongside the numerical upper bound of /epsilon1 from [23] and the numerical lower bound determined by binary search. Given its closeness to the lower bound, our Theorem 4.2 can be considered near-optimal.

We compare δ f -DP obtained in Corollary 4.3 with the corresponding δ derived from [23] using a fixed value of /epsilon1 0 = 4 . 444 and n = 10000 . Notably, δ f -DP is significantly smaller than δ .

We compare /epsilon1 f -DP obtained from Corollary 4.3 with the corresponding numerical upper bound /epsilon1 derived from [23] using a fixed value of /epsilon1 0 = 4 . 444 and n = 10000 . For δ < 10 -7 , the bound in [23] fails as the assumption /epsilon1 0 ≤ log ( n 8 log(2 / δ ) -1 ) is violated while our theory removes this assumption and holds for all /epsilon1 0 . Moreover, we compare our theoretical upper bound with the empirical lower bound obtained by binary search in [23] which shows that our bound is near-optimal.

In summary, our non-asymptotic privacy bound for shuffled mechanisms outperforms Theorem 3.2 in [23]. This improvement is a result of our Proposition 4.1, which optimally refines Lemma A.4 in [23]. Besides Proposition 4.1, the remainder of our proof of Theorem 4.2 closely adheres to the methodology presented in [23]. Our near-optimal result is complicated due to its tightness. Thus, it is difficult to compare our result with the asymptotic bound in [23] analytically.

5 Privacy analysis of one-iteration DP-GD with random initialization

Asignificant challenge in the privacy analysis of the last-iteration model of DP-SGD lies in accounting for multiple randomization techniques used during iterations. This includes aspects like initialization, iterative steps, and sub-sampling. Since these techniques incorporate a mixture of random noise, the joint convexity of F -divergence becomes crucial in the privacy analysis of DP-SGD [45, 2]. Our Lemma 3.1, which provides a unified perspective on these convexity notations, has driven us to include it in the privacy analysis of DP-GD. Nevertheless, analyzing the trajectories from multi-step iterations remains complex. Therefore, our initial exploration is to investigate the effects of random initialization on a one-step iterate. It's noteworthy that in machine learning, training a deep neural network using (stochastic) gradient descent combined with random initialization is widely adopted [36]. The significance of random initialization in noisy gradient descent is also emphasized by [46] within the framework of Kullback-Leibler privacy.

Consider a dataset D = { ( x i , y i ) } n i =1 with x i ∈ R being the features and y i ∈ R being the labels. Let /lscript ( θ , D ) be a loss function and let g ( θ , D ) be the gradient of /lscript with respect to θ . The output of one-step iteration of DP-GD initialized at θ 0 with step-size 1 is given by

θ ( D ) = θ 0 -( g ( θ 0 , D ) + N (0 , σ 2 ) ) . (1) In the setting of random initialization, θ 0 is chosen as a Gaussian random variable. Without loss of generality, we consider θ 0 = I ∼ N (0 , 1) and rewrite θ ( D ) = s I ( D ) + N (0 , σ 2 ) with s I ( D ) = I -g ( I, D ) . θ ( D ) is a Gaussian random variable when the initialization I is given, that is, θ ( D ) | I = i ∼ N ( s i ( D ) , σ 2 ) . Thus, we can regard θ ( D ) as an infinite mixture of Gaussian distributions with continuous Gaussian weights { ϕ ( i ) } i ∈ R , where ϕ is the pdf of I and the corresponding trade-off function T ( θ ( D 0 ) , θ ( D 1 ) can be bounded using the joint concavity.

For simplicity, we define θ ( D ) | I as a random variable with a given initialization I . For two neighboring datasets D 0 and D 1 , it holds with X | I ∼ N (0 , 1) and Y | I ∼ N ( µ I , 1) for I ∼ N (0 , 1) , where µ I = ( g ( I, D 1 ) -g ( I, D 0 )) / σ . Theorem 5.1. Let θ ( D 0 ) and θ ( D 1 ) be defined in (1) for neighboring datasets D 0 and D 1 . Then, we have

with t I = -t µ I + µ I 2 and α ( t ) = E I [ Φ ( t I ) · 1 [ µ I ≤ 0] + Φ ( -t I ) · 1 [ µ I > 0] ] . Here Φ is the cumulative distribution function of N (0 , 1) and the expectation is taken with respect to I .

Remark. Note that Theorem 5.1 is instance-based privacy guarantee as it relies on the datasets. To extend it to the worst case, we let µ max I = max D 0 , D 1 {| g ( I, D max 1 ) -g ( I, D max 0 ) | / σ } be the sensitivity of the gradient with a given initialization I . As a result, θ ( D ) output by one-step DPGD is f -DP with f ( α ( t )) = E I [ Φ ( t max I -µ max I )] , where t max I = -t µ max I + µ max I 2 and α ( t ) = E I [ Φ ( -t max I )] . The worst case trade-off function is bounded for strongly convex loss functions with a bounded data domain.

To numerically evaluate the trade-off function in Theorem 5.1, we consider an example D 0 = { ( x i , y i ) } n i =1 with y i = ax i and x 2 i = 1 for some constant a and we defined D 1 by removing an arbitrary element in D 0 . Moreover, we assume that σ = 1 . Note that for this example without gradient clipping, the gradient is linear in I and θ ( D 0 ) is the sum of two Gaussian random variables which is Gaussian. Thus, the trade-off function has a closed-form representation. In general, the output is non-Gaussian and we should adopt Theorem 5.1. For example, if we consider gradient clipping [1, 9] and replace g ( θ , D ) by the clipped gradient

where the gradient of each data point g ( i ) is cut off by some constant c > 0 , then µ max I is given by

which is not Gaussian. In this example g c ( θ , D ) + N (0 , 1) is considered as c -GDP if we disregard the effects of random initialization since the sensitivity of g c is c .

We illustrate the trade-off function of Theorem 5.1 computed numerically in Figure 2, where we also compare it with c -GDP for a = 1 and varying values of c . Overall, the figure suggests that random initialization can amplify the privacy of DP-GD, as our bounds outperform those of c -GDP, which does not take into account the randomness of initialization. Furthermore, we observe that as c increases, the amplification effect caused by random initialization becomes more significant, since the difference between T (( X | I, I ) , ( Y | I, I )) and c -GDP also increases. This is reasonable, since the randomness resulting from initialization comes from I such that | a -I | ≤ c , whereas for | a -I | > c , µ I remains constant and no randomness is introduced. Thus, the random initialization introduces greater levels of randomness as c increases.

It is worth noting that in this example, without gradient clipping, we have µ max I = a -I and the dominate pair are two Gaussian distributions N (0 , 1) and N (0 , 2) . The Rényi DP fails to measure the privacy of initialization. In fact, it holds R ˜ α ( N (0 , 1) ‖ N (0 , 2)) = ∞ for ˜ α large enough. 1.00 1.00

0.00

0.25

0.50

0.75

0.00

0.25

0.50

0.75

1.00

Type I error

Type II error

(a)

c

= 0

.

5

0.00

0.25

0.50

0.75

0.00

0.25

0.50

0.75

1.00

Type I error

(b)

c

= 2

(c)

c

= 3

6 Optimality of joint concavity and advanced joint concavity

In this section, we first explore the sufficient and necessary conditions under which Lemma 3.1 holds with equality. While Lemma 3.1 is generally not sharp, we introduce an f -DP analog of the advanced joint convexity of the hockey-stick divergence from [4], yielding tighter bounds in certain applications.

Recall the distributions P = (1 -w ) P 0 + wQ 0 and Q = (1 -w ) Q 0 + wP 0 that appear in the shuffled mechanisms. Bounding the trade-off function T ( P, Q ) directly using the joint concavity leads to a loose bound (cf., Figure 3b). For the scenarios where Lemma 3.1 is not tight, we introduce the f -DP analog of the advanced joint convexity of ( /epsilon1 , δ ) -DP [4] that may lead to tighter bounds and we term it the "advanced joint concavity of trade-off functions".

The following proposition presents a necessary and sufficient condition for Lemma 3.1 to hold with equality.

/negationslash

Proposition 6.1. For m = 2 , Lemma 3.1 holds with equality if and only if w 1 p 1 + w 2 p 2 w 1 q 1 + w 2 q 2 ( X ) P = w 1 p 1 q 1 ( X ) + w 2 p 2 q 2 ( X ) with X ∼ P w , where for p i ( X ) /q i ( X ) = 0 / 0 and p j ( X ) /q j ( X ) = 0 / 0 with i = j , we set p i ( X ) /q i ( X ) = p j ( X ) /q j ( X ) .

/negationslash

It is not difficult to see that P 0 and Q 0 in shuffling models satisfy this necessary and sufficient condition when n = 2 .

As we discussed, Lemma 3.1 may not be sharp in general. The following lemma is about the advanced joint convexity of the hockey-stick divergence, which is a slight generalization of Theorem 2 in [4].

0.80

0.85

0.90

0.95

1.00

0.00

0.05

0.10

0.15

0.20

Type I error

Type II error

1.00-

0.75-

errc

Advanced

=0.50

-- Id

Jointconcavity

0.00-

0.000.25 0.500.751.00

Type I error

Lemma 6.2. For any non-negative /epsilon1 ′ , /epsilon1 0 , /epsilon1 1 , γ , and η satisfying exp( /epsilon1 ′ ) = (1 -w ) exp( /epsilon1 0 ) + w exp( /epsilon1 1 ) and exp( /epsilon1 0 )(1 -w ) γ +exp( /epsilon1 1 ) w η = exp( /epsilon1 ′ ) w, we have

Lemma 6.2 is reduced to the advanced joint convexity of the hockey-stick divergence in [4] when P 1 = Q 1 , by minimizing the right-hand-side of (2) with respect to γ , η , /epsilon1 0 , and /epsilon1 1 .

Recall the convex conjugate g ∗ of a function g defined by g ∗ ( y ) = sup x { xy -g ( x ) } and C ( f ) = min { f, f -1 } ∗∗ which is the symmetrization of f . We have the following advanced joint concavity of trade-off functions.

Lemma 6.3 (Advanced joint concavity) . Suppose that T ( P i , Q i ) is symmetric for each i . Then, for 0 ≤ w ≤ 1 , we have

for arbitrary 0 ≤ γ < w < η ≤ 1 , where F i,j ( x ) is given by F 1 ,i ( x ) := f 1 ,i ( x (1 -w )( η -γ ) ( η -w ) ) , and F 2 ,i ( x ) := f 2 ,i ( xw ( η -γ ) ( w -γ ) ) , and the trade-off functions are defined as f i,j = T ( P i , Q j ) for 1 ≤ i, j ≤ 2 . Moreover, for γ = η = w , it holds

Determining the trade-off functions using advanced joint concavity can be challenging in many practical situations. In fact, to apply the advanced joint concavity, one need to specify the choice of γ , η by maximizing the right-hand-side of Lemma 6.3. Therefore, in real-world applications, we often rely on both joint concavity and advanced joint concavity.

For P = (1 -w ) P 0 + wQ 0 and Q = (1 -w ) Q 0 + wP 0 in shuffling models, we have the following bound derived from Lemma 6.3.

Proposition 6.4. For P = (1 -w ) P 0 + wQ 0 and Q = (1 -w ) Q 0 + wP 0 with some weight 0 ≤ w ≤ 1 / 2 , we have T ( P, Q ) ≥ C (2 w Id + (1 -2 w ) T ( P 0 , Q 0 )) .

The equality in Proposition 6.4 does not hold exactly. However, this lower bound is almost the tightest closed-form expression. One may refer to Section E.1.1 in the appendix for the proof details.

7 Discussion

This paper provides refined privacy bounds for mixture mechanisms, including shuffling models and DP-GD with random initialization. For shuffling models, we present a bound that is tighter than existing results based on ( /epsilon1 , δ ) -DP. In the study of DP-GD, we demonstrate how random initialization can amplify privacy concerns. These bounds are derived using a unified f -DP approach based on the joint concavity and advanced joint concavity of trade-off functions. We also investigate the sharpness and other properties of these concavity notions.

In our future work, we plan to extend our analysis from one-step DP-GD to multi-step DP-SGD. For DP-SGD with multiple iterations, it is crucial to consider subsampling and privacy amplification by iteration in the privacy accountant, in addition to the randomness introduced by shuffling and random initialization. While there is an f -DP bound for subsampling provided in an independent work [42], as far as we know, there is limited research on f -DP results regarding privacy amplification by iteration.

Beyond DP-SGD, we intend to extend our theory to the privacy analysis of other key applications that involves various randomization techniques. These include the shuffled Gaussian mechanism for federated learning, as discussed in [25], and the composition of mixture mechanisms. For extending our theory to federated learning, we might adopt the f -DP framework outlined in [49]. Addressing the composition of mixture mechanisms demands examination of the tensor product between trade-off functions. This is a complex task, even when dealing with the simplest mixture mechanisms like sub-sampling, as highlighted in [51].

Acknowledgments

Weijie J. Su was supported in part by a Meta Research Award and NSF through CCF1934876.

Reza Shokri was supported by a Google PDPO Faculty Research Award, Intel within the www.privateai.org center, a Meta Faculty Research Award, the NUS Early Career Research Award (NUS ECRA award number NUS ECRA FY19 P16), and the National Research Foundation, Singapore under its Strategic Capability Research Centres Funding Initiative.

References

Martín Abadi, Andy Chu, Ian J. Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Edgar R. Weippl, Stefan Katzenbeisser, Christopher Kruegel, Andrew C. Myers, and Shai Halevi, editors, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016 , pages 308-318. ACM, 2016.

Jason M. Altschuler and Kunal Talwar. Privacy of noisy stochastic gradient descent: More iterations without more privacy loss. In NeurIPS , 2022.

Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy. Differentially private learning with adaptive clipping. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 17455-17466, 2021.

Borja Balle, Gilles Barthe, and Marco Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Gilles Barthe and Federico Olmedo. Beyond differential privacy: Composition theorems and relational logic for f-divergences between probabilistic programs. In Automata, Languages, and Programming: 40th International Colloquium, ICALP 2013, Riga, Latvia, July 8-12, 2013, Proceedings, Part II 40 , pages 49-60. Springer, 2013.

Raef Bassily, Adam D. Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In 55th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2014, Philadelphia, PA, USA, October 18-21, 2014 , pages 464-473. IEEE Computer Society, 2014.

David Blackwell. Equivalent comparisons of experiments. The annals of mathematical statistics , pages 265-272, 1953.

Jonathan M. Borwein and Adrian S. Lewis. Convex analysis and nonlinear optimization , volume 3 of CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC . Springer, New York, second edition, 2006. Theory and examples.

Zhiqi Bu, Jinshuo Dong, Qi Long, and Weijie Su. Deep learning with Gaussian differential privacy. Harvard Data Science Review , 2(3), Sep 30 2020. https://hdsr.mitpress.mit.edu/pub/u24wj42y.

Mark Bun, Cynthia Dwork, Guy N. Rothblum, and Thomas Steinke. Composable and versatile privacy via truncated CDP. In Ilias Diakonikolas, David Kempe, and Monika Henzinger, editors, Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018, Los Angeles, CA, USA, June 25-29, 2018 , pages 74-86. ACM, 2018.

Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. IACR Cryptol. ePrint Arch. , page 816, 2016.

Kamalika Chaudhuri, Claire Monteleoni, and Anand D. Sarwate. Differentially private empirical risk minimization. J. Mach. Learn. Res. , 12:1069-1109, 2011.

Albert Cheu, Adam D. Smith, Jonathan R. Ullman, David Zeber, and Maxim Zhilyaev. Distributed differential privacy via shuffling. In Yuval Ishai and Vincent Rijmen, editors, Advances in Cryptology - EUROCRYPT 2019 - 38th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Darmstadt, Germany, May 19-23, 2019, Proceedings, Part I , volume 11476 of Lecture Notes in Computer Science , pages 375-403. Springer, 2019.

Jinshuo Dong, Aaron Roth, and Weijie J. Su. Gaussian differential privacy. J. R. Stat. Soc. Ser. B. Stat. Methodol. , 84(1):3-54, 2022. With discussions and a reply by the authors.

Jinshuo Dong, Weijie J. Su, and Linjun Zhang. A central limit theorem for differentially private query answering. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 1475914770, 2021.

Cynthia Dwork. Differential privacy. In Michele Bugliesi, Bart Preneel, Vladimiro Sassone, and Ingo Wegener, editors, Automata, Languages and Programming, 33rd International Colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II , volume 4052 of Lecture Notes in Computer Science , pages 1-12. Springer, 2006.

Cynthia Dwork. Differential privacy: A survey of results. In Manindra Agrawal, Ding-Zhu Du, Zhenhua Duan, and Angsheng Li, editors, Theory and Applications of Models of Computation, 5th International Conference, TAMC 2008, Xi'an, China, April 25-29, 2008. Proceedings , volume 4978 of Lecture Notes in Computer Science , pages 1-19. Springer, 2008.

Cynthia Dwork, Moni Naor, Omer Reingold, Guy N. Rothblum, and Salil P. Vadhan. On the complexity of differentially private data release: efficient algorithms and hardness results. In Michael Mitzenmacher, editor, Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31 - June 2, 2009 , pages 381-390. ACM, 2009.

Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. , 9(3-4):211-407, 2014.

Cynthia Dwork and Guy N. Rothblum. Concentrated differential privacy. CoRR , abs/1603.01887, 2016.

Alexandre V. Evfimievski, Johannes Gehrke, and Ramakrishnan Srikant. Limiting privacy breaches in privacy preserving data mining. In Frank Neven, Catriel Beeri, and Tova Milo, editors, Proceedings of the Twenty-Second ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, June 9-12, 2003, San Diego, CA, USA , pages 211-222. ACM, 2003.

Vitaly Feldman, Audra McMillan, and Kunal Talwar. Hiding among the clones: A simple and nearly optimal analysis of privacy amplification by shuffling. In 62nd IEEE Annual Symposium on Foundations of Computer Science, FOCS 2021, Denver, CO, USA, February 7-10, 2022 , pages 954-964. IEEE, 2021.

Vitaly Feldman, Audra McMillan, and Kunal Talwar. Stronger privacy amplification by shuffling for Rényi and approximate differential privacy. In Nikhil Bansal and Viswanath Nagarajan, editors, Proceedings of the 2023 ACM-SIAM Symposium on Discrete Algorithms, SODA 2023, Florence, Italy, January 22-25, 2023 , pages 4966-4981. SIAM, 2023.

Vitaly Feldman, Ilya Mironov, Kunal Talwar, and Abhradeep Thakurta. Privacy amplification by iteration. In Mikkel Thorup, editor, 59th IEEE Annual Symposium on Foundations of Computer Science, FOCS 2018, Paris, France, October 7-9, 2018 , pages 521-532. IEEE Computer Society, 2018.

Antonious M. Girgis, Deepesh Data, Suhas N. Diggavi, Peter Kairouz, and Ananda Theertha Suresh. Shuffled model of differential privacy in federated learning. In Arindam Banerjee and Kenji Fukumizu, editors, The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event , volume 130 of Proceedings of Machine Learning Research , pages 2521-2529. PMLR, 2021.

Peter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, and Zheng Xu. Practical and private (deep) learning without sampling or shuffling. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , volume 139 of Proceedings of Machine Learning Research , pages 5213-5225. PMLR, 2021.

E. L. Lehmann and Joseph P. Romano. Testing statistical hypotheses . Springer Texts in Statistics. Springer, New York, third edition, 2005.

Ximing Li, Chendi Wang, and Guang Cheng. Statistical theory of differentially private marginal-based data synthesis algorithms. In The Eleventh International Conference on Learning Representations , 2023.

Ryan McKenna, Gerome Miklau, and Daniel Sheldon. Winning the NIST contest: A scalable and general approach to differentially private synthetic data. J. Priv. Confidentiality , 11(3), 2021.

Ilya Mironov. Rényi differential privacy. In 30th IEEE Computer Security Foundations Symposium, CSF 2017, Santa Barbara, CA, USA, August 21-25, 2017 , pages 263-275. IEEE Computer Society, 2017.

Ilya Mironov, Kunal Talwar, and Li Zhang. Rényi differential privacy of the sampled gaussian mechanism. arXiv preprint arXiv:1908.10530 , 2019.

Xiaosheng Mu, Luciano Pomatto, Philipp Strack, and Omer Tamuz. From Blackwell dominance in large samples to Rényi divergences and back again. Econometrica , 89(1):475-506, 2021.

Sewoong Oh and Pramod Viswanath. The composition theorem for differential privacy. CoRR , abs/1311.0776, 2013.

Maxim Raginsky. Shannon meets Blackwell and Le Cam: Channels, codes, and statistical experiments. In 2011 IEEE International Symposium on Information Theory Proceedings , pages 1220-1224. IEEE, 2011.

Igal Sason and Sergio Verdú. f-divergence inequalities. IEEE Trans. Inf. Theory , 62(11):5973-6006, 2016.

Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton. On the importance of initialization and momentum in deep learning. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013 , volume 28 of JMLR Workshop and Conference Proceedings , pages 1139-1147. JMLR.org, 2013.

Reihaneh Torkzadehmahani, Peter Kairouz, and Benedict Paten. DP-CGAN: differentially private synthetic data and label generation. CoRR , abs/2001.09700, 2020.

Tim van Erven and Peter Harremoës. Rényi divergence and Kullback-Leibler divergence. CoRR , abs/1206.2459, 2012.

Hua Wang, Sheng Gao, Huanyu Zhang, Milan Shen, and Weijie J Su. Analytical composition of differential privacy via the Edgeworth accountant. arXiv preprint arXiv:2206.04236 , 2022.

Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan. Subsampled Rényi differential privacy and analytical moments accountant. In Kamalika Chaudhuri and Masashi Sugiyama, editors, The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan , volume 89 of Proceedings of Machine Learning Research , pages 1226-1235. PMLR, 2019.

Yue Wang, Xintao Wu, and Donghui Hu. Using randomized response for differential privacy preserving data collection. In Themis Palpanas and Kostas Stefanidis, editors, Proceedings of the Workshops of the EDBT/ICDT 2016 Joint Conference, EDBT/ICDT Workshops 2016, Bordeaux, France, March 15, 2016 , volume 1558 of CEUR Workshop Proceedings . CEUR-WS.org, 2016.

Zhanyu Wang, Guang Cheng, and Jordan Awan. Differentially private bootstrap: New privacy analysis and inference strategies. arXiv preprint arXiv:2210.06140 , 2023.

Larry Wasserman and Shuheng Zhou. A statistical framework for differential privacy. J. Amer. Statist. Assoc. , 105(489):375-389, 2010.

Shirong Xu, Chendi Wang, Will Wei Sun, and Guang Cheng. Binary classification under local label differential privacy using randomized response mechanisms. Transactions on Machine Learning Research , 2023.

Jiayuan Ye and Reza Shokri. Differentially private learning needs hidden state (or much faster convergence). In NeurIPS , 2022.

Jiayuan Ye, Zhenyu Zhu, Fanghui Liu, Reza Shokri, and Volkan Cevher. Initialization matters: Privacyutility analysis of overparameterized neural networks. In NeurIPS , 2023.

Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential privacy library in PyTorch. CoRR , abs/2109.12298, 2021.

Jun Zhang, Graham Cormode, Cecilia M. Procopiuc, Divesh Srivastava, and Xiaokui Xiao. PrivBayes: Private data release via Bayesian networks. ACM Trans. Database Syst. , 42(4):25:1-25:41, 2017.

Qinqing Zheng, Shuxiao Chen, Qi Long, and Weijie J. Su. Federated f-differential privacy. In Arindam Banerjee and Kenji Fukumizu, editors, The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event , volume 130 of Proceedings of Machine Learning Research , pages 2251-2259. PMLR, 2021.

Qinqing Zheng, Jinshuo Dong, Qi Long, and Weijie J. Su. Sharp composition bounds for Gaussian differential privacy via edgeworth expansion. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proceedings of Machine Learning Research , pages 11420-11435. PMLR, 2020.

Yuqing Zhu, Jinshuo Dong, and Yu-Xiang Wang. Optimal accounting of differential privacy via characteristic function. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, International Conference on Artificial Intelligence and Statistics, AISTATS 2022, 28-30 March 2022, Virtual Event , volume 151 of Proceedings of Machine Learning Research , pages 4782-4817. PMLR, 2022.

Yuqing Zhu and Yu-Xiang Wang. Poission subsampled Rényi differential privacy. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of Machine Learning Research , pages 7634-7642. PMLR, 2019.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a framework for analyzing mixture distributions via $f$-DP and its tradeoff function. Additionally, the paper leverages this framework for improving bounds for shuffling mechanisms in DP, and the same framework to prove a statement about privacy of a single-step of gradient descent from random initialization.\n\nSTRENGTHS:\nThis paper addresses a fundamental problem in DP: computing effective estimates on additive mixtures of random variables. Methods of analysis for these mixtures are, to date, reasonably ad-hoc and certainly not known to be tight. This paper presents a clean unified approach, making connections to more abstract and clean mathematically-posed problems (IMO, this is the manner in which DP should imagine evolving itself, away from its CS + statistics formulations, which are often focused on data structures and properties of particular random variables). Not only does this paper formulate a nice presentation, but  to my knowledge it improves existing statements through this framework. Clear accept.\n\nWEAKNESSES:\nWriting some nits no presentation that came up from a stream of consciousness read below. Not necessarily all weaknesses, just comments.\n\n* I would prefer a slight reordering of the content. AFAICT, the main logical flow is: Lemma 4.1 -> Prop 4.3 -> Thm 3.1, with the rest of the stuff more or less as add-ons (though FWIW i have not yet read the section on advanced concavity). I would really prefer this logical flow to be more clear. Ideally, for me, we would present the proofs straight through, beginning with the lemmas. However, I get that this is CS and we don't really do that here. I think a reasonable option is to sketch the proof in the main body, or discuss that the major results seem to follow primarily from the fairly simple Lemma 4.1, plus some calculations, given the reduction introduced by [20]. This logical flow is simple and quite nice, and IMO it should be highlighted on its own terms.\n\n* Editorially, I would suggest pushing the DP-GD stuff into an appendix. I think it confuses the message, and it's not clear either how extensible it is (presumably it breaks after one step due to the injected dependence? Though I could imagine conditioning on stuff and invoking nested mixtures, this may be prohibitively difficult since IIUC Gaussian-ness will go away). \n\n* Confusion on 'the outputs of two neighboring datasets are post-processing of r.v.s $X$ and $Y$' (beginning of section 4). Going back to the reference, it seems like the technical meat there is effectively a good reduction to scalar-valued random variables (if  I read correctly, the content of their Thm 3.1)--presumably that is what is invoked here? But it reads as if the development were restricted to scalar-valued mechanisms (which I assume it is not).\n\n* Setup to 4.1: the way that $X$, $Y$, and $I$ are coupled is unstated. Meaning: in the setup here, it is possible that $I$ is independent of $X$ and $Y$ (in which case $X|I$ and $Y|I$ would just be $X$ and $Y$). I guess the coupling is something like, e.g. $X$ is coupled with $I$ such that $X|\\{I=i\\} \\sim p_i$? But if so, I don't see this stated anywhere before the proof of Lemma 4.1.\n\n* I know this is not how CS papers are generally written, but (being an old mathematician myself) I would really personally prefer to read this paper straight through and without applications, building up to an improved analysis (which I assume is Thm 3.1), then closing with discussion of relation to prior results and tightness.\n\n* Speaking of relation to prior results, and putting my CS hat back on: this could really use some deeper dives / commentary. I would recommend cutting the DP-GD stuff (or pushing to an appendix) and extending the discussion of relationship to prior bounds. For example: there is a table showing $\\delta$s for varying $\\epsilon$s at a fixed $\\epsilon_0$. What happens when we vary this $\\epsilon_0$ over a wide range--are the results here _always_ better (and qualitatively similar)? Or are there some regimes in which previous analyses were tighter?\n\n* Following the citations of [20] leads me to https://arxiv.org/pdf/2304.05007.pdf. Glancing through this paper, it seems to me that one major difference with the present work is that this work claims improved amplification relative to [20] _independent_ of mechanism structure, whereas that work improves their bounds in the case of some particular mechanisms (and their worst-case bounds, for general mechanisms, are identical). Is this correct? Can you speak to this relationship?\n\n* One confusing thing on a read-through: it is difficult to find the proof of proposition 4.3. This proof can be found in the 'technical details for section 3.1', which is not where a reader like me would look right away (given, in particular, that there is an appendix section on proofs for section 4).\n\nQUESTIONS:\nSimply a summary of some of my stream-of-consciousness comments above:\n\n* Could you comment more extensively on relation to prior results? Is the analysis here _always_ stronger?\n* Similarly, could you comment on the relation to https://arxiv.org/pdf/2304.05007.pdf? I listed my understanding of the way these relate above, but looking for some clarity here.\n\nLIMITATIONS:\nTightness or lack thereof of particular results was discussed. Negative social impacts not immediately applicable.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nOne of the main challenges the differential privacy frame work is facing these days, is the gap between the variety of randomization techniques applied in the machine learning community for various reasons, and our limited capability to prove the privacy amplifications they entail. Among primary examples we can mention random initialization and shuffling techniques.\n\nThe authors of this paper leverage the known connection between DP and hypothesis testing, specifically relaying on the newly presented notion of f-DP, to transition the analysis of the effect of these mechanism from the domain of privacy loss distribution to the domain of hypothesis testing, where they prove several key results. Combining these results with the known implications between f-DP and DP, they provide tighter bounds for privacy amplification by shuffling and new results for privacy amplification by random initialization (for the limited setting of one gradient step).\n\nSTRENGTHS:\nThis paper partially fills a long time gap in our understanding of the way privacy is improved by introducing various random steps to the learning algorithm. Unlike most previous works, the authors use the newly presented f-DP definition and the known implications between it and the DP definition, to analyze the effect of these randomization techniques on the trade-off function.\n\nThe contribution of this paper extends beyond its results, as the proof techniques presented in it might be used for the analysis of other potential amplification techniques as well.\n\nWEAKNESSES:\nAt first glance, I got the impression that the results of this paper aim to provide guarantees using a separate notion of privacy, which cannot be compared to DP. This is indeed not the case, thanks to the known implications between DP and f-DP, but I found the current presentation somewhat confusing regarding this point, and I recommend the authors clarify it in the final version.\n\nI found some of the choices made in the numerical evaluations presented in Figure 1 to be not so clear. First of all, the sample size was chosen to 1,000 while the reference paper used 10,000, and $\\delta$ was chosen as $1/n$ while it is often expected to be of the order $o(1/n)$.\n\nA more substantial issue with the comparison presented in Figure 1 has to do with the presented baseline. If I understand correctly, the baseline results were chosen to be those of the closed-form presented in [20], but that work contains tighter results presented in Theorem III2, which can be numerically evaluated, and should be presented as the more relevant comparison.\n\nMinor comments:\n\n* In line 104, the inequality if a functional inequality, but this notion was not presented before, so the notation is somewhat confusing.\n\n=======\n\n**Edit after rebuttal discussion:**\n\nThe authors response satisfied my remaining concerns.\n\nQUESTIONS:\ni will appreciate some clarification regarding the choices made in the evaluation presented in figure 1, and the chosen baseline.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

comments: SUMMARY:
Randomization is an essential too in deriving differentially private algorithms. However, sophisticated randomization techniques such as shuffling induce complicated distributions on outputs, making analysis of privacy loss difficult. The paper uses the framework of f-differential privacy to provide tighter analysis on the privacy guarantees of shuffled DP-SGD than existing methods do. Additionally, the authors show that randomized initialization can beneficially improve privacy guarantees for one step problems.

STRENGTHS:
-	The closed-form bounds for shuffled DP-SGD seem particularly important, even if the bounds are complicated. Table 1 does a good job of convincing the reader that the obtained bounds are significantly better than existing ones.
-	Moreover, the authors present some useful results for studying tradeoff functions in the f-DP framework. These will likely be useful for future investigations.

WEAKNESSES:
-	Results for random initialization for one-step SGD do not seem particularly useful? In particular, this contribution seems like a bit of a toy problem.
-	Some demonstration of the improved bounds for Shuffle-SGD would be useful to the reader. In particular, it would be convincing if there was some simple learning task on which the accuracy of the model produced by shuffle-SGD (under the new analysis) significantly outperformed the model produced under the old analysis. This should happen for any experiment, as the privacy budget savings seems to be significant per table 1.

QUESTIONS:
NA

LIMITATIONS:
-	The authors fairly discuss the limitations of their work, in particular their intentions to study multi-step SGD with random initialization.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies an important problem in DP: the privacy quantification based on a mixture of randomness. Based on f-DP, the authors point out the joint concavity of the tradeoff function. Two potential examples are proposed, including the shuffling model and the privacy amplification from random initialization.\n\nSTRENGTHS:\n+ The problem studied is important to the privacy analyses of many applications. In particular, I really like the idea to consider the privacy amplification from the random initialization.\n\n+ The introduction and the potential limitation are nicely written. Motivation is clear. \n\n+ The improvement upon the privacy analysis on shuffling model seem interesting.\n\nWEAKNESSES:\n- Though I really like the idea to take the randomness of initialization into account and study its privacy implication, the results presented in Section 3.2 are not convincing. The claimed simulatable f-DP guarantees seem to be instance-based, or is with respect to a given adjacent datasets $D_0$ and $D_1$. Moreover, even for the particular instance-based guarantee, there is no analysis provided, such as high confidence bound, to allow us really apply this bound to produce rigorous privacy guarantee. Moreover, (please correct me if I get it wrong), based on my understanding, applying the joint concavity, Theorem 3.5 is essentially determined by the average case of the \"local\" sensitivity given the random initialization. So, I guess current studies cannot be generalized to the standard worst-case DP guarantee. \n\n- The notations are confusing. For example, I did not find the formal definition of $(\\theta(D)|I,I)$ in Section 3.2. My best guessing is because there are two random sources? But why we must assume the noise and random initialization are in the same distribution? \n\n-  Section 3.1 is lack of theoretical analysis on how tighter the f-DP bound given in Theorem 3.1 is, as compared to previous works. Though the empirical improvement on the $\\delta$ numbers seem significant, given that the dependence of $\\epsilon$ on $\\delta$ is in logarithm, it is not very clear whether such improvement is general. Another missing issue is the explanation on the computational accuracy $10^{-6}$ in Table 1. Where is this restriction coming from? Moreover, I think one major motivation to introduce f-DP in [14] is for tighter composition. The authors may also want to take it into consideration.\n\nQUESTIONS:\n1. How to really apply the privacy amplification results in 3.2 under regular input-independent f-DP scenario?\n\n2. What the fundamental challenge to generalize the results to the subsampling case? I think the subsampled f-DP is already solved in \"Deep Learning with Gaussian Differential Privacy\" (https://arxiv.org/abs/1911.11607). \n\n3. For the theoretical analysis, can you characterize the improvement compared to previous works?  \n\n4. It would be more helpful if the authors can elaborate the background of shuffling model, at least why the output distributions from two adjacent datasets are in the form described at the beginning of Section 4.\n\nLIMITATIONS:\n## Update after reading the authors responses: \nI summarize my concerns and suggestions for the authors to improve this paper:\n1. As I said, I always like the idea of privacy amplification from random initialization and this is an important open question widely-recognized in DP community. But the authors's results on this part, from my opinion, is trivial given that it can only study closed-form iterate distribution for the first round and thus negligible amplification in DP-SGD, and the authors agreed to put it as a minor contribution in the revision.\n\n2. I think some claims in the paper are overblown and the limitations of the proposed methods are not properly discussed. At least to me, the joint convexity thing in the tradeoff function is not that surprising given that Poisson subsampling has already been studied in Gaussian DP. The authors also agreed that their improvement over the subsampling case is limited either for i.i.d. sampling or with a fixed batch size. \n\n3. For the main contribution to the shuffling model, I never say that there is no novelty in this paper. I agree that the paper presents tighter bounds on some special cases, for example with additional assumptions on the ordering tradeoff function or special mixture weights. But for the shuffling model, the advantage is only measured empirically without a clear asymptotic analysis. As I mentioned, a small constant improvement on the $\\epsilon$ can lead to exponential improvement on $\\delta$. The authors' response on this part is somewhat double talk and constantly say they simulate it and it is better than prior works. But my question is always how better it is and the authors did not  directly give an answer to show analytical bound of improvement. Anyway, I suggest the authors clearly discussing about both the advantages and limitations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper studies improving the analysis of privacy bounds for two randomization processes (privacy amplifications): shuffling models (where each user record is privatized by some local randomizer like randomized response mechanism) and differentially-private gradient descent (DP-GD, where a Gaussian noise is added to the gradient update at each iteration). Previous bounds were given for the standard $(\epsilon, \delta)$-DP bound, while this paper analyzes through $f$-DP, a differential-privacy bound considering the trade-off between type I and type II errors (like the $f$-score), using some joint convexity arguments.

When translating the $f$-DP bounds back to bounds on $\epsilon$ and $\delta$, this paper gets much tighter bounds that works for more general ranges of $\epsilon$ and $\delta$.

STRENGTHS:
The improvement in DP bounds are non-negligible and more general, and the presentation is convincing.

WEAKNESSES:
Certain critical aspects of the proofs might be better explained: the proof depends on some close-form expressions of piecewise linear functions (Theorem 3.1, Corollary 3.2, Proposition 4.3), which might have meaning beyond just those arithmetic manipulations or expressions as is currently shown in the paper. A one- or two-sentence description, or some simple examples, might help here.

QUESTIONS:
After Proposition 4.8, the paper writes “One may refer to the supplementary materials for details”, but this reader is unsure where are the supplementary materials.

LIMITATIONS:
The theoretical results in this paper does not have broader societal impacts.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-09