Title: Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement
Venue: NeurIPS 2024
Accepted: yes
Keywords: privacy preserving machine learning

differential privacy

differentially private stochastic gradient descent

fixed-size subsampled mechanisms

privacy amplification lemma
Abstract: Differentially private stochastic gradient descent (DP-SGD) has been instrumental in privately training deep learning models by providing a framework to control and track the privacy loss incurred during training.  At the core of this computation lies a subsampling method that uses a privacy amplification lemma to enhance the privacy guarantees provided by the additive noise. Fixed size subsampling is appealing for its constant memory usage, unlike the variable sized minibatches in Poisson subsampling. It is also of interest in addressing class imbalance and federated learning. Current computable guarantees for fixed-size subsampling are not tight and do not consider both add/remove and replace-one adjacency relationships. We present a new and holistic Rényi differential privacy (RDP)  accountant for DP-SGD with fixed-size subsampling without replacement (FSwoR) and with replacement (FSwR). For FSwoR we consider both add/remove and replace-one adjacency, where we improve on the best current computable bound by a factor of $4$. We also show for the first time that the widely-used Poisson subsampling and FSwoR with replace-one adjacency have the same privacy to leading order in the sampling probability. Our work suggests that FSwoR is often preferable to Poisson subsampling due to constant memory usage. Our FSwR accountant includes explicit non-asymptotic upper and lower bounds and, to the  authors' knowledge, is the first such RDP analysis of fixed-size  subsampling with replacement  for DP-SGD. We analytically and empirically compare fixed size and Poisson subsampling, and show that DP-SGD gradients in a fixed-size subsampling regime exhibit lower variance in practice in addition to memory usage benefits.

Differentially Private Stochastic Gradient Descent with Fixed-Size Minibatches: Tighter RDP Guarantees with or without Replacement

Abstract

Differentially private stochastic gradient descent (DP-SGD) has been instrumental in privately training deep learning models by providing a framework to control and track the privacy loss incurred during training. At the core of this computation lies a subsampling method that uses a privacy amplification lemma to enhance the privacy guarantees provided by the additive noise. Fixed size subsampling is appealing for its constant memory usage, unlike the variable sized minibatches in Poisson subsampling. It is also of interest in addressing class imbalance and federated learning. Current computable guarantees for fixed-size subsampling are not tight and do not consider both add/remove and replace-one adjacency relationships. We present a new and holistic R´ enyi differential privacy (RDP) accountant for DP-SGD with fixed-size subsampling without replacement (FSwoR) and with replacement (FSwR). For FSwoR we consider both add/remove and replace-one adjacency, where we improve on the best current computable bound by a factor of 4 . We also show for the first time that the widely-used Poisson subsampling and FSwoR with replace-one adjacency have the same privacy to leading order in the sampling probability. Our work suggests that FSwoR is often preferable to Poisson subsampling due to constant memory usage. Our FSwR accountant includes explicit non-asymptotic upper and lower bounds and, to the authors' knowledge, is the first such RDP analysis of fixed-size subsampling with replacement for DP-SGD. We analytically and empirically compare fixed size and Poisson subsampling, and show that DP-SGD gradients in a fixed-size subsampling regime exhibit lower variance in practice in addition to memory usage benefits.

1 Introduction

Differentially private stochastic gradient descent (DP-SGD) (Abadi et al., 2016) (DP-SGD) has been one of the cornerstones of privacy preserving deep learning. DP-SGD allows a so-called moments accountant technique to sequentially track privacy leakage (Abadi et al., 2016). This technique is subsumed by R´ enyi differential privacy (RDP) (Mironov, 2017), a relaxation of standard differential privacy (DP) (Dwork and Rothblum, 2016) that is widely used in private deep learning and implemented in modern DP libraries such as Opacus (Meta Platforms, 2024) and autodp (Zhu and Wang, 2019; Zhu et al., 2022). RDP facilitates rigorous analysis when the dataset is accessed by a sequence of randomized mechanisms as in DP-SGD. While other privacy accountant frameworks such

Rouzbeh Behnia

as f -DP (Dong et al., 2022), privacy loss distributions (PLD) (Koskela et al., 2020), Privacy Random Variable (PRV) (Gopi et al., 2021), Analytical Fourier Accountant (AFA) (Zhu et al., 2022), and Saddle-Point Accountant (SPA) (Alghamdi et al., 2023) have been proposed to improve conversion to privacy profiles, RDP is still one of the main privacy accountants used in popular deep learning tools and applications. Thus, we expect providing tighter bounds specific to RDP will have important practical implications for the community. Each iteration of DP-SGD can be viewed as a private release of information about a stochastic gradient. DP accounting methods bound the total privacy loss incurred by applying a sequence of DP-SGD mechanisms in training. Privacy of the overall mechanism is ensured through the application of two random processes: 1) a randomized mechanism that subsamples minibatches of the training data, 2) noise applied to each gradient calculation (e.g., Gaussian). Informally, if a mechanism M is ( ϵ, δ ) -DP then a mechanism that subsamples with probability q ∈ (0 , 1) ensures privacy ( O ( qϵ ) , qδ ) -DP; a result known as the 'privacy amplification lemma' (Li et al., 2012; Zhu and Wang, 2019). Computing the privacy parameters of a mechanism requires an accounting procedure that can be nontrivial to design. Abadi et al. (2016) compute DP parameters for the special case of Gaussian noise, which was later extended to track RDP parameters by Mironov (2017). These analyses are limited to Poisson subsampling mechanisms which produce minibatches of variable size. This creates engineering challenges in modern machine learning pipelines and also does not allow for sampling with replacement. Fixed-size minibatches are preferable as they can be aligned with available GPU memory, leading to higher learning throughput. In practice, fixed size minibathces are of interest in practical applications such as private federated learning. To address the issue, Wang et al. (2019) and Zhu et al. (2022) propose the latest fixed-size accounting methods. Wang et al. (2019) provide computable RDP bounds for FSwoR that cover general mechanisms under replace-one adjacency, while Zhu et al. (2022) give computable bounds for FSwoR under add/remove adjacency. Specifically, the study in (Wang et al., 2019) provides a general formula to convert the RDP parameters of a mechanism M to RDP parameters of a subsampled (without replacement) mechanism, but the bounds they provide are only tight up to a constant factor at leading order (i.e., comparing dominant terms in the asymptotic expansion in q ). Moreover, while the FSwoR results under add/remove adjacency relationship in (Zhu et al., 2022) are in parallel to our results, their accountant is not readily applicable to FSwoR under replace-one adjacency relationship.

In our work, we (1) present a new and holistic RDP accountant for DP-SGD with fixed-size subsampling without replacement (FSwoR) and with replacement (FSwR) that considers both add/remove and replace-one adjacency. We note that FSwoR is equivalent to shuffling the dataset at the start of every iteration and then taking the first | B | elements, which is conveniently similar to the default behavior of common deep learning libraries (e.g. Pytorch's DataLoader iterator object) while FSwR latter is implemented, e.g., in Pytorch's RandomSampler , and is widely used to address class imbalance in classifiers (Mease et al., 2007) and biased client selection in federated learning applications (Cho et al., 2022). (2) Our FSwoR improves on the best current computable bound under replace-one adjacency (Wang et al., 2019) by a factor of 4 . (3) Our FSwR accountant includes non-asymptotic upper and lower bounds and, to the authors' knowledge, is the first such analysis of fixed-size RDP with replacement for DP-SGD. (4) For the first time, we show that FSwoR and the widely-used Poisson subsampling have the same privacy under replace-one adjacency to leading order in the sampling probability. This has important practical benefits given the memory management advantages of fixed-size subsampling. Thus our results suggest that FSwoR is often preferable to Poisson subsampling. The implementation of our accountant is included in Supplementary Materials and is also publicly available to the community at https://github.com/star-ailab/FSRDP .

2 Background and Related Work

Mironov (2017) proposed a differential privacy definition based on R´ enyi divergence, RDP, that subsumes the celebrated moments accountant (Abadi et al., 2016) used to track privacy loss in DP-SGD. RDP and many other flavors of DP rely on subsampling for privacy amplification to ensure that privacy guarantees hold when DP-SGD is applied on samples (i.e., minibatches) of training data (Bun et al., 2018; Balle et al., 2018). Later works such as (Mironov et al., 2019; Zhu and Wang, 2019) propose a generic refined bound for subsampled Gaussian Mechanism that improves the tightness of guarantees in RDP. However, these methods also produce variable-sized minibatches. Bun et al. (2018), Balle et al. (2018), and Wang et al. (2019) propose fixed-size subsampled mechanisms. Balle et al. (2018) provide a general fixed-size subsampling analysis with and without replacement but their bounds focus on tight ( ϵ, δ ) -DP bounds for a single step, and are not readily applicable to DP-SGD.

The AFA developed by Zhu et al. (2022) is an enhancement of the moments accountant offering a lossless conversion to ( ϵ, δ ) guarantees and is designed for general mechanisms. However, in the context of DP-SGD, there are still technical and practical difficulties. For instance, in DP-SGD there is generally a large number of training steps, in which case the numerical integrations required by Algorithm 1 in Zhu et al. (2022) involve the sum of a large number of oscillatory terms. This can lead to numerical difficulties, especially in light of the lack of rigorous error bounds for the double (Gaussian) quadrature numerical integration approach as discussed in their Appendix E.1. In addition, the bounds in Zhu et al. (2022) are not readily applicable under replace-one adjacency. Due to these considerations, and the fact that RDP is still widely used, in this work, we focus on improving the RDP bounds from Wang et al. (2019).Hayes et al. (2024) uses the results in (Zhu et al., 2022) to provide a bound specific to DP-SGD when one is only concerned with training data reconstruction attacks rather than the membership inference attacks, which leads to a relaxation of DP to provide a bound that is only applicable to defending against data reconstruction and not membership inference. Our work can be viewed as a complimentary work to (Hayes et al., 2024) that is specific to DP-SGD and provides conservative bounds protecting against membership inference attacks (and thus any other type of model inversion attacks including training data reconstruction) in the RDP context.

In summary, to our knowledge, Wang et al. (2019) provide the best computable bounds in the fixedsize regime for RDP that are practical for application to DP-SGD. While the computations in (Wang et al., 2019) have the attractive property of applying to a general mechanism, in this work, we show that there is room for obtaining tighter bounds specific to DP-SGD with Gaussian noise. Our FS-RDP bounds address this theoretical gap by employing a novel Taylor expansion expansion approach, which precisely captures the leading order behavior in the sampling probability, while employing a computable upper bounds on the integral remainder term to prevent privacy leakage. For convenience, we provide the definition of RDP below.

Definition 2.1 ( ( α, ϵ ) -RDP (Mironov et al., 2019)) . Let M be a randomized mechanism, i.e., M ( D ) is a probability distribution for any choice of allowed input (dataset) D . A randomized mechanism is said to have ( α, ϵ ) -RDP if for any two adjacent datasets D,D ′ it holds that D α ( M ( D ) ∥M ( D ′ )) ≤ ϵ , where the R´ enyi divergence of order α > 1 is defined by

The definition of dataset adjacency varies among applications. In this work we consider two such relations: 1) The add/remove adjacency relation, where datasets D and D ′ to be adjacent if one can be obtained from the other by adding or removing a single element; we denote this by D ≃ a/r D ′ . 2) The replace-one adjacency definition, denoted D ≃ r -o D ′ , wherein D is obtained from D ′ by replacing a single element. In the next section we derive new RDP bounds when M is DP-SGD using fixed-size minibatches, with or without replacement.

3 R´ enyi-DP Bounds for SGD with Fixed-size Subsampling

In this section we present our main theoretical results, leading up to R´ enyi-DP bounds for SGD for fixed-size subsampling done without replacement (Thm. 3.3 for add/remove adjacency and Thm. 3.4 for replace-one adjacency) and with replacement (Thm. 3.7). In Sec. 3.4, we also compare our results with the analysis in (Wang et al., 2019) and show that our analysis yields tighter R´ enyi bounds by a factor of approximately 4 .

3.1 FS-RDP: Definition and Initial Bounds

Given a loss function L , a training dataset D with | D | elements, and a fixed minibatch size, we consider the DP-SGD NN parameter updates with fixed-size minibatches,

where t is the iteration number, the initial condition Θ D 0 is independent of D , η t are the learning rates, the noises Z t are Gaussians with mean 0 and covariance C 2 σ 2 t I , and the B D t are random minibatches

with fixed size; we denote the fixed value of | B D t | by | B | for brevity. We will consider fixed-size subsampling both without and with replacement; in the former, B D t is a uniformly selected subset of { 0 , ..., | D | -1 } of fixed size | B | for every t and in the latter, B D t is a uniformly selected element of { 0 , ..., | D | -1 } | B | for every t . We assume the Gaussian noises and minibatches are all independent jointly for all steps. Here Clip denotes the clipping operation on vectors, with ℓ 2 -norm bound C > 0 , i.e., Clip ( x ) := x/ max { 1 , ∥ x ∥ 2 /C } . We derive a RDP bound (Mironov, 2017) for the mechanism consisting of T steps of DP-SGD (2), denoted:

Specifically, we consider the cases without replacement, M FS woR [0 ,T ] ( D ) , and with replacement, M FS wR [0 ,T ] ( D ) . To obtain these FS-RDP accountants we bound D α ( M FS [0 ,T ] ( D ) ∥M FS [0 ,T ] ( D ′ )) , where D ′ is any dataset that is adjacent to D ; as previously stated, we will consider both add/remove and replace-one adjacency. By taking worst-case bounds over the input state at each step, as done in Theorem 2.1 in (Abadi et al., 2016), we decompose the problem into a sum over steps

where the time-inhomogeneous transition probabilities are,

Here N µ, ˜ σ 2 ( θ ) is the Gaussian density with mean µ and covariance ˜ σ 2 I , ˜ σ 2 t := C 2 η 2 t σ 2 t / | B | 2 , and the summation is over the set of allowed index minibatches, b , with normalization constant Z | B | , | D | = ( | D | | B | ) for FSwoR-RDP and Z | B | , | D | = | D | | B | for FSwR-RDP. When additional clarity is needed we use the notation p FSwoR t and p FSwR t to distinguish the transition probabilities for these respective cases.

3.2 FSwoR-RDP Upper Bounds

The computations thus far mimic those in (Abadi et al., 2016; Mironov et al., 2019), with the choice of subsampling method and adjacency relation playing no essential role. That changes in this section, where we specialize to the case of subsampling without replacement. First, in Section 3.2.1 we consider the add/remove adjacency relation; the proof in this case contains many of the essential ideas of our method but is simpler from a computational perspective. The more difficult case of replace-one adjacency will then be studied in Section 3.2.2

3.2.1 Add/remove Adjacency

In Appendix A we derive the following R´ enyi divergence bound for FSwoR-subsampled DP-SGD under add/remove adjacency.

Theorem 3.1. Let D ≃ a/r D ′ be adjacent datasets. With transition probabilities defined as in (5) and letting q = | B | / | D | we have

The key step in the proof consists of the decomposition of the mechanism given in Lemma A.1. We note that the r.h.s. of (6) differs by a factor of 1 / 4 in the variances from the corresponding result for Poisson subsampling in Mironov et al. (2019). This is due to the inherent sensitivity difference between fixed-size and Poisson subsampling under add/remove adjacency; see (28) - (29). To bound the r.h.s. of (3.1) we will employ a Taylor expansion computation with explicit remainder bound. As the r.h.s. of (6) has the same mathematical form as the Poisson subsampling result of Mironov et al. (2019), and therefore can be bounded by the same methods employed there, the Taylor expansion

method is not strictly necessary in the add/remove case. However, we find it useful to illustrate the Taylor expansion method in this simpler case before proceeding to the significantly more complicated replace-one case in Section 3.2.2, where the method in Mironov et al. (2019) does not apply.

The R´ enyi divergences on the r.h.s. of (6) can be written

where for any α > 1 , σ > 0 we define

Eq. (8) does not generally have a closed form expression. We upper bound it via Taylor expansion with integral formula for the remainder at order m (c.f. Thm. 1.2, (Stewart, 2022)):

where the remainder term is given by

Note that we do not take m →∞ and so H α,σ is not required to be analytic in order to make use of (9). Also note that (9) is an equality, therefore if we can compute/upper-bound each of the terms then we will arrive at a computable non-asymptotic upper bound on H α,σ ( q ) , without needing to employ non-rigorous stopping criteria for an infinite series, thus avoiding privacy leakage. The order m is a parameter that can be freely chosen by the user. To implement (9) one must compute the derivatives d k dq k H α,σ (0) and bound the remainder (10). We show how to do both of those steps for general m in Appendix B; in the following theorem we summarize the results for m = 3 , which we find provides sufficient accuracy in our experiments.

Theorem 3.2 (Taylor Expansion Upper Bound) . For q < 1 we have

where the remainder has the bound

and the M and ˜ B parameters are given by (41) and (44) respectively.

The reason for the complexity of the formulas (11) - (12) is the need to obtain a rigorous upper bound, and not simply an asymptotic result. Results for other choices of m are given in Appendix B.

Combining Theorem 3.2 with equations (4) - (8) we now arrive at a computable T -step FSwoR-RDP guarantee.

Theorem 3.3 ( T -step FSwoR-RDP Upper Bound: Add/Remove Adjacency) . Assuming q < 1 , the mechanism M FSwoR [0 ,T ] ( D ) , defined in (3) , has ( α, ϵ FSwoR [0 ,T ] ( α )) -RDP under add/remove adjacency, where

where M and ˜ R are given by (41) and (12) respectively.

More generally, using the calculations in Appendix B one obtains RDP bounds for any choice of the number of terms, m . The result (13) corresponds to m = 3 , which we find to be sufficiently accurate in practice; see Figure 6.

3.2.2 Replace-one Adjacency

Theorem 3.1 applies to add/remove adjacency, but our method of proof can also be used in the replace-one adjacency case, where it yields the following FSwoR-RDP upper bounds.

Theorem 3.4 (FSwoR-RDP Upper Bounds for Replace-one Adjacency) . Let D ≃ r -o D ′ be adjacent datasets. Assuming q := | B | / | D | < 1 , for any integer m ≥ 3 we have

where ˜ F α,σ t ,k is given by (80) and ˜ E α,σ t ,m ( q ) by (87) .

The derivation, found in Appendix C, follows many of the same steps as the add/remove-adjacency case from Theorem 3.1, though deriving bounds on the terms in the Taylor expansion is significantly more involved and the resulting formulas are more complicated. The decomposition from Lemma A.1 again constitutes a key step in the proof. For comparison purposes, in Appendix C.1 we derive the analogous result for Poisson-subsampling under replace-one adjacency; see Theorem C.9.

A corresponding T -step RDP bound, analogous to Theorem 3.3, for replace-one adjacency can similarly be obtained by combining Theorem 3.4 with Eq. (4).

Theorem 3.5 ( T -step FSwoR-RDP Upper Bound: Replace-one Adjacency) . Assuming q < 1 and for any integer m ≥ 3 , the mechanism M FSwoR [0 ,T ] ( D ) , defined in (3) , has ( α, ϵ FSwoR [0 ,T ] ( α )) -RDP under replace-one adjacency, where

Here ˜ F α,σ t ,k is given by (80) and ˜ E α,σ t ,m ( q ) by (87) .

We emphasize that one strength of our approach is that it provides a unified method for deriving computable bounds in both the add/remove and replace-one adjacency cases. We also note that Theorem 11(b) of Zhu et al. (2022), Theorem 6 of Balle et al. (2018), and Theorem 5 in Mironov et al. (2019) combine to provide an alternative method for deriving the add/remove result of Theorem 3.1 but not the replace-one result of Theorem 3.4. Unlike in the corresponding add/remove theorems, using m > 3 in Theorems 3.4 and 3.5 is often required to obtain acceptable accuracy, though we generally find that m = 4 is sufficient; see Figure 2.

3.3 FSwR-RDP Upper and Lower Bounds

We similarly obtain RDP bounds for DP-SGD using fixed-size minibatches with replacement, i.e., (2) with minibatches B D t that are iid uniformly random samples from { 0 , ..., | D | -1 } | B | . First we present upper bounds on the RDP of M FSwR [0 ,T ] ( D ) . The derivation follows a similar pattern to that of our results for FSwoR-subsampling, first using a probabilistic decomposition of the mechanism (see Lemma D.1) and then expanding (see Appendix D.1).

Theorem 3.6 (Fixed-size RDP with Replacement Upper Bound) . Assuming | D | > | B | and under add/remove adjacency, the mechanism M FSwR [0 ,T ] ( D ) has ( α, ϵ FSwR [0 ,T ] ( α )) -RDP with ( H α,σ given by (8) )

The functions H α,σ were previously bounded above in Theorem 3.2 (for order m = 3 ), and more generally in Appendix B (for general m ). It is straightforward to show that ˜ q = O ( q ) and ˜ a n =

O ( q n -1 ) . Therefore, recalling the asymptotics (11) of H α,σ , the behavior of ϵ FSwR [0 ,T ] ( α ) and ϵ FSwoR [0 ,T ] ( α ) is the same at leading order in q . We also present the following novel lower bounds on the R´ enyi divergence for one step of FSwR DP-SGD; see Appendix D.2 for the derivation.

Theorem 3.7 (Fixed-size RDP with Replacement Lower Bound) . Let the mechanism M FSwR t ( D,θ t ) denote the t 'th step of FSwR DP-SGD (2) when using the training dataset D and starting from the NN parameters θ t , i.e., the transition probabilities (5) where the average is over minibatches b that are elements of { 0 , ..., | D | -1 } | B | . In this result we consider the choice of | B | to be fixed but will allow D and θ t to vary. Suppose there exists θ t , d , and d ′ such that

i.e., when using the NN parameters θ t , the clipped gradients at the samples d and d ′ are anti-parallel and saturate the clipping threshold. Then for any integer α ≥ 2 and any choice of dataset size N we have the following worst-case R´ enyi divergence lower bound

The lower bound (17) provides a complement to one step of the upper bound from Theorem 3.6. We demonstrate the behavior of the lower bound (17) in Figure 1, where we employed the computational method discussed in Appendix D.2.1. Specifically, we show the FSwR-RDP lower bound as a function of α for fixed q and several choices of | B | . For small α , the lower bounds for different | B | 's are approximately equal. This holds up until a critical | B | -dependent threshold where there is a 'phase transition' to a regime with non-trivial | B | dependence (even for fixed q ) wherein the RDP bound increases quickly. This is in contrast to FSwoR-RDP which depends on | B | only

2

3

4

5

6

7

8

9

10

10

6

10

4

10

2

10 0

10 2

One-step RDP Lower Bound

|

B

| = 20

|

B

| = 30

|

B

| = 40

|

B

| = 60

|

B

| = 80

through the ratio q . The critical α is smaller for larger | B | , which implies worse privacy guarantees for larger | B | , with all else being equal. Intuitively, a larger minibatch size provides a greater chance for the element that distinguished D from D ′ to be selected multiple times in a single minibatch. If it is selected too many times then it can overwhelm the noise and become 'noticeable', thus substantially degrading privacy; this is the key property that distinguishes FSwRfrom FSwoR. In practice, this behavior implies that FSwR-RDP requires much smaller minibatch sizes than FSwoR-RDP in order to avoid this critical threshold and maintain privacy.

3.4 Comparison with (Wang et al., 2019)

We compare our fixed-size RDP upper bounds with the upper and lower bounds obtained by applying the general-purpose method from Wang et al. (2019) to DP-SGD with Gaussian noise. First we compare the results asymptotically to leading order in q and 1 /σ 2 t , which is the domain relevant to applications, and then we will compare them numerically using the full non-asymptotic bounds. Unless stated otherwise, from here on the phrase leading order implicitly refers to the parameters q and 1 /σ 2 t .

The method from Wang et al. (2019) applied to DP-SGD (App. E.1) gives the one-step R´ enyi bounds

In contrast, to second order in q , one step of our FS-RDP results under add/remove adjacency from Theorem 3.3 or Theorem 3.6 gives

while our FSwoR-RDP bound under replace-one adjacency from Theorem 3.4 gives

Therefore, to leading order, our methods all have the same behavior and give tighter RDP bounds by a factor of 4 than that of Wang et al. (2019). We emphasize that the adjacency relation used by Wang et al. (2019) is replaceone and therefore, strictly speaking, should be compared only with our Theorem 3.4, which also uses replace-one adjacency. Figure 2 shows a non-asymptotic comparison

0

10

20

30

40

50

60

10

6

10

5

10

4

One-step FSwoR-RDP Bound (r-o)

Thm.3.4 upper bound (m = 3)

Thm.3.4 upper bound (m = 4)

Thm.3.4 upper bound (m = 5)

Wang et al. upper bound

Wang et al. lower bound

between these RDP bounds, with ours being tighter significantly tighter than that of Wang et al. (2019). In particular, our bound from Theorem 3.4 with m = 4 remains close to the theoretical lower bound from Wang et al. (2019) over the entire range of α 's pictured, which is a sufficiently large range for practical application to DP, e.g., larger than the default range used in Meta Platforms (2024).

Following the setting in (Abadi et al., 2016), we provide corresponding ( ϵ, δ ) -DP guarantees in Figure 3 shows corresponding ϵ for δ ∈ { 1 e -4 , 1 e -5 , 1 e -6 , 1 e -7 , 1 e -8 , 1 e -9 , 1 e -10 } after 250 training epochs in DP-SGD. To translate RDP bounds to ( ϵ, δ ) -DP, we used Theorem 21 in (Balle et al., 2020), which is also used in the Opacus library. As shown in Figure 3, our FSwoR bounds with m ∈ { 3 , 4 , 5 } (solid lines) are close to the lower-bound provided in (Wang et al., 2019) (circles) over a range of δ 's that depend on the value of m . These guarantees are significantly tighter than the upper bound from Wang et al. (2019) (dashed line). In our experiments, we observe that m = 4 and m = 5 yields essentially the same results and we find that m = 4 suffices to obtain good results in practice while outperforming (Wang et al., 2019).

4 Comparing Fixed-size and Poisson Subsampling

In Sec. 3.4 we demonstrated that our R´ enyi-DP-SGD bounds with fixed-size subsampling in Thm. 3.3 are tighter than the general-purpose fixed-size subsampling method from Wang et al. (2019). However, the most commonly used implementations of DP-SGD rely on Poisson subsampling, not fixed-size, and there the comparison is more nuanced. In this section we compare Poisson and fixed-size subsampling and point out advantages and disadvantages of each.

4.1 Privacy Comparison

In terms of privacy guarantees, all else being equal, when using replace-one adjacency we find that DP-SGD with fixed-size subsampling yields the same privacy guarantees as Poisson subsampling to leading order; specifically, when moving from add/remove to replace one-adjacency, our fixedsize subsampling DP bounds do not change at leading order but the Poisson subsampling DP bounds change by a factor of 2 to match the fixed-size results. In contrast, when using add/remove adjacency we find that Poisson subsampling has a natural advantage over fixed-size subsampling by approximately a factor of 2 . Here we outline the reason for this intuitively.

The derivation of R´ enyi-DP bounds when using Poisson subsampling, (see Abadi et al. (2016); Mironov et al. (2019)) shares many steps with our derivation for fixed-size subsampling without replacement in Appendix A. In both cases, to leading order the bound on the R´ enyi divergence is proportional to ∥ µ -µ ′ ∥ 2 , where µ and µ ′ are the sum of the gradients under the adjacent datasets D and D ′ . When using Poisson subsampling, under add/remove adjacency, µ ′ differs from µ by the addition or deletion of a single term in the sum with probability q , while under replace-one adjacency, µ ′ differs from µ by the replacement of one element with probability q . Therefore in the former case, ∥ µ -µ ′ ∥ is the norm of a single clipped vector, hence it scales with the clipping threshold, C , while

in the latter, ∥ µ -µ ′ ∥ is the norm of the difference between two clipped vector, hence it scales with 2 C , as in the worst case the two vectors are anti-parallel. In fixed-size subsampling and under both adjacency relations, µ ′ differs from µ by replacing one element with another (due to the fixed-size minibatch constraint) and therefore ∥ µ -µ ′ ∥ scales like 2 C . After squaring and at leading order, this leads to a factor of 4 difference in the R´ enyi divergences between Poisson and fixed-size cases under add/remove adjacency while under replace-one adjacency the Poisson and fixed-size results agree; see Appendix C.1 for a more precise analysis of the latter comparison.

Translating this to ( ϵ, δ ) -DP then leads to an approximate factor of 2 difference in ϵ for the same δ under add/remove adjacency, as shown in Appendix G, while under replace-one adjacency the different subsampling methods lead to the same DP guarantees at leading order. When the higher order terms are taken into account, Poisson subsampling regains a slight privacy advantage under replace-one adjacency; see Figures 4 and 7.

4.2 Comparison on CIFAR10

Our numerical comparisons in Section 3.4 showed that FSwoR-RDP yields significantly tighter guarantees than its counterpart from Wang et al. (2019). It is also useful to compare the empirical privacy guarantees of fixed-size versus the Poisson subsampling RDP commonly used in DP implementations such as Opacus. To this end, we use CIFAR10 and a simple convolutional neural network (CNN) to compare the canonical Poisson subsampling RDP with FS-RDP. Our setup closely follows the procedure in (Abadi et al., 2016). Note that the goal of this comparison is not to achieve the state-ofthe-art privacy guarantees on CIFAR10 with any specific neural network. Instead we aim to compare the performance and guarantees with the alternative fixed-size and Poisson subsampling methods. Following Abadi et al. (2016), we used a simple CNN network that is pre-trained on CIFAR100 dataset non-privately. Our CNN has six convolutional layers 32, 32, 64, 128, 128, and 256 square filters of size 3 × 3 , in each layer respectively. Since using Batch Normalization layers leads to privacy violation, we used group normalization instead as a common practice that is also used the Opacus library (Meta Platforms, 2024). Our network has 3 group normalization layers of size 64, 128, and 256 after the second, fourth, and sixth convolution layer. Similar to Abadi et al. (2016), this model reaches the accuracy of 81.33% after 250 epochs of training in a non-private setting. For the Poisson subsampling RDP, we simply used the one in Opacus.

testing accuracy

Fig. 4 compares the privacy guarantees ( ϵ, δ ) for FSwoR-RDP and the method proposed by Wang et al. (2019) on the above CNN model after 250 training epochs with σ t = 6 , | B | = 120 , clipping threshold C = 3 . 0 , and learning rate lr = 1 e -3 on CIFAR10 following the same setting in Abadi et al. (2016). Fig. 4 also compares the testing accuracy of Poisson subsampled RDP in Opacus with our FSwoR-RDP. We ran each method 5 times with different random seeds for sampling: 0 , 1 , 364 , 2 , 560 , 3 , 000 , and 4 , 111 . Shaded areas show 3-sigma standard deviations around the mean. Both logical and physical minibatch sizes were set to 120 in Opacus. Experiments were run on a single work station with an NVIDIA RTX 3090 with 24GB memory. The runtime for all experiments was under 12 hours. As shown in Figure 4, FSwoRRDP yields substantially tighter bounds than (Wang et al., 2019) (top panel) and reaches the average accuracy of 63.87% after 250 epochs (vs. 61.57% for Poisson subsampled RDP) (bottom panel). Figure 4 indicates that for a fixed σ , FSwoR-RDP surpasses the accuracy of Poisson subsampled RDP in Opacus, but with slightly higher epsilon guarantees, as shown in the top panel. This behavior matches the approximate calculation from Eq. (150) as well as the discussion in Section 4.1; specifically, Poisson and FSwoRRDP agree to leading order but the higher order contributions give Poisson subsampling a slight privacy advantage at the same level

0

50

100

150

200

250

45

50

55

60

65

FS

woR

-RDP (m=10)

Poisson Subsampled RDP (Opacus)

of added noise. We emphasize that these results all apply to the replace-one notion of adjacency, as used in (Wang et al., 2019) and also our Theorem 3.4; we did not find existing computable RDP bounds for Poisson-subsampling under replace-one adjacency and so we used the new result in Theorem C.9. We conjecture that the performance difference observed in the bottom panel of Figure 4

is due to the difference in variance between Poisson and fixed-size subsampling, which we derive in Appendix F. Specifically, there we show that fixed-size subsampling has reduced variance compared to Poisson when the parameters are away from a minimizer. Additional experiments, gauging the sensitivity of FSwoR-RDP to key parameters including σ and | B | , are given in Appendix H.

4.3 Memory Usage Comparison

The RDP accountant used in common DP libraries assumes a Poisson subsampling mechanism that leads to variable-sized minibatches during the training. That is, the size of each mini-batch cannot be determined in advance. Allowing the minibatches to have variable sizes creates an engineering challenge: an uncharacteristically large minibatch can cause an out-of-memory error, and depending on how fine tuned the minibatch size is to the available GPU memory this could be a frequent occurrence. To tackle this issue, the Opacus implementation requires the additional complication of wrapping the Pytorch DataLoader object into a BatchMemoryManager object, to alleviate the memory intensive nature of Poisson subsampling. BatchMemoryManager requires privacy practitioners to define two minibatch sizes: one is a logical minibatch size for Poisson sampling and the other is a physical minibatch size that determines the actual space allocated in memory and is determined by the practitioner via a variable named max physical batch size (Meta Platforms, 2024). This design follows the distinction between 'lots' and minibatches described in the moments accountant (Abadi et al., 2016).

Our proposed FS-RDP does not require this memory management and is much less memory intensive. Figure 5 depicts the GPU memory footprints during 100 epochs of privately training the above CNN on CIFAR10 with FSwoR-RDP and three of the most common DP accountants: RDP (Mironov et al., 2019), f -DP (Dong et al., 2022), and PRV (Gopi et al., 2021). As shown in Figure 5, the memory usage of FS-RDP remains constant at 14,826 MB. However, the memory usage varies from 7,984 MB to 23,280 MB (almost twice larger than FS-RDP) for Poisson subsampled RDP in Opacus. Similar memory usage is observed for other Opacus accountants that use Poisson subsampling mechanism (i.e., f -DP and PRV). This fluctuating memory usage could result in undesirable outcomes in settings where resource-constrained devices are considered. For instance, in federated learning environments,

FS-RDP(Ours)

22500

RDP(Opacus)

f-DP(Opacus)

E

20000

PRV(Opacus)

/usage

17500

memory

15000

12500

GPU

10000

7500

0

20

40

60

80

100

erratic memory usage could result in a higher dropout rate and delay the global model's convergence (Liu et al., 2021). It could also undermine model's accuracy and fairness, preventing lower-end devices from participating in the training process (Imteaj et al., 2021).

5 Conclusion

Differentially private stochastic gradient descent with fixed-size minibatches has attractive properties including reduced gradient estimator variance and simplified memory management. Our work presents a holistic RDP accountant for DP-SGD with fixed-size subsampling without replacement (FSwoR) and with replacement (FSwR) and, in the FSwoR case, consider both add/remove and replace-one adjacency. As we showed theoretically and empirically, since FSwoR under replace-one adjacency leads to the same leading-order privacy guarantees as the widely-used Poisson subsampling, we suggest using the former over the latter to benefit from the memory management and reduced variance properties. For subsampling without replacement under replace-one adjacency, we obtained significantly tighter RDP bounds (4 times improvement) over the most recent computable results (Wang et al., 2019). For subsampling with replacement we obtained the first non-asymptotic upper and lower RDP bounds for DP-SGD. We also provided the first comparison of gradient estimator's variance and privacy guarantees between FS-RDP and Poisson subsampled RDP commonly used in DP libraries. Our analysis revealed that FS-RDP reduces the variance of the gradient estimator. We highlighted the memory usage advantages of FS-RDP over Poisson subsampled RDP, which makes it a practical choice for privately training large deep learning models. We made FS-RDP's implementation and its evaluations available online such that it can be added to common DP libraries.

References

Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. (2016). Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security , pages 308-318.

Alghamdi, W., Gomez, J. F., Asoodeh, S., Calmon, F., Kosut, O., and Sankar, L. (2023). The saddle-point method in differential privacy. In International Conference on Machine Learning , pages 508-528. PMLR.

Balle, B., Barthe, G., and Gaboardi, M. (2018). Privacy amplification by subsampling: Tight analyses via couplings and divergences. Advances in Neural Information Processing Systems (NeurIPS) , 31.

Balle, B., Barthe, G., Gaboardi, M., Hsu, J., and Sato, T. (2020). Hypothesis testing interpretations and R´ enyi differential privacy. In Chiappa, S. and Calandra, R., editors, Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , volume 108 of Proceedings of Machine Learning Research , pages 2496-2506. PMLR.

Bun, M., Dwork, C., Rothblum, G. N., and Steinke, T. (2018). Composable and versatile privacy via truncated CDP. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing , pages 74-86.

Cho, Y. J., Wang, J., and Joshi, G. (2022). Towards understanding biased client selection in federated learning. In International Conference on Artificial Intelligence and Statistics , pages 10351-10375. PMLR.

Dong, J., Roth, A., and Su, W. J. (2022). Gaussian differential privacy. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 84(1):3-37.

Dwork, C. and Rothblum, G. N. (2016). Concentrated differential privacy. CoRR , abs/1603.01887.

Folland, G. B. (1999). Real analysis: modern techniques and their applications , volume 40. John Wiley & Sons.

Gopi, S., Lee, Y. T., and Wutschitz, L. (2021). Numerical composition of differential privacy. Advances in Neural Information Processing Systems , 34:11631-11642.

Hayes, J., Balle, B., and Mahloujifar, S. (2024). Bounding training data reconstruction in dp-sgd. Advances in Neural Information Processing Systems , 36.

Imteaj, A., Thakker, U., Wang, S., Li, J., and Amini, M. H. (2021). A survey on federated learning for resource-constrained iot devices. IEEE Internet of Things Journal , 9(1):1-24.

Koskela, A., J¨ alk¨ o, J., and Honkela, A. (2020). Computing tight differential privacy guarantees using FFT. In Chiappa, S. and Calandra, R., editors, The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy] , volume 108 of Proceedings of Machine Learning Research , pages 2560-2569. PMLR.

Lebeda, C. J., Regehr, M., Kamath, G., and Steinke, T. (2024). Avoiding pitfalls for privacy accounting of subsampled mechanisms under composition. arXiv preprint arXiv:2405.20769 .

Li, N., Qardaji, W., and Su, D. (2012). On sampling, anonymization, and differential privacy or, k-anonymization meets differential privacy. In Proceedings of the 7th ACM Symposium on Information, Computer and Communications Security , pages 32-33.

Liu, J., Xu, H., Wang, L., Xu, Y., Qian, C., Huang, J., and Huang, H. (2021). Adaptive asynchronous federated learning in resource-constrained edge computing. IEEE Transactions on Mobile Computing .

Mease, D., Wyner, A. J., and Buja, A. (2007). Boosted classification trees and class probability/quantile estimation. Journal of Machine Learning Research , 8(3).

Meta Platforms, I. (2024). Opacus. https://opacus.ai/docs/introduction . (Accessed on 01/21/2024).

Mironov, I. (2017). R´ enyi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF) , pages 263-275. IEEE.

Mironov, I., Talwar, K., and Zhang, L. (2019). R´ enyi differential privacy of the sampled Gaussian mechanism. arXiv preprint arXiv:1908.10530 .

Stewart, D. (2022). Numerical Analysis: A Graduate Course . CMS/CAIMS Books in Mathematics. Springer International Publishing.

Van Erven, T. and Harremos, P. (2014). R´ enyi divergence and Kullback-Leibler divergence. IEEE Transactions on Information Theory , 60(7):3797-3820.

Wang, Y.-X., Balle, B., and Kasiviswanathan, S. P. (2019). Subsampled R´ enyi differential privacy and analytical moments accountant. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 1226-1235. PMLR.

Zhu, Y., Dong, J., and Wang, Y.-X. (2022). Optimal accounting of differential privacy via characteristic function. In International Conference on Artificial Intelligence and Statistics , pages 4782-4817. PMLR.

Zhu, Y. and Wang, Y.-X. (2019). Poission subsampled R´ enyi differential privacy. In International Conference on Machine Learning , pages 7634-7642. PMLR.

A Proof of Theorem 3.1: FSwoR-RDP Bound under Add/Remove Adjacency

In this appendix we prove a one-step R´ enyi bound for DP-SGD using subsampling without replacement under the add/remove adjacency relation, as stated in Theorem 3.1. To do this, we will show that the sum over minibatches b in (5) can be rewritten as an average of Gaussian mixtures with parameter q = | B | / | D | . This will then allow us to mimic several steps from the derivation in Mironov et al. (2019), which applies to Poisson subsampling. In the Poisson subsampling case the Gaussian mixture structure is more or less apparent from the start, but here it requires some additional work to isolate. We first consider the case where D ′ is obtained from D by removing one element. Without loss of generality we can let D ′ = ( D 0 , ..., D | D |-2 ) . Later we will show that the bound we derive in this case also bounds the case where D ′ has one additional element.

Noting that 1 N D ∑ b is simply the expectation with respect to the distribution of B D t , we start by showing that the distribution of B D t can be obtained from the distribution of B D ′ t through the introduction of two auxiliary random variables J and ˜ B as follows: Let J be a Bernoulli ( q ) random variable where q = | B | / | D | and let ( B ′ , ˜ B ) be random variables independent of J such that B ′ is a uniformly selected subset of { 0 , ..., | D | -2 } of size | B | (so B ′ ∼ B D ′ t ) and P ( ˜ B ∈ ·| B ′ = b ′ ) is the uniform distribution on subsets of b ′ of size | B | -1 . Defining the random variable B by

we have the following:

Lemma A.1. The distribution of B , defined in (20) , equals that of B D t .

Proof. It is clear that B is valued in the subsets of { 0 , ..., | D | -1 } having size | B | . Given b ⊂ { 0 , ..., | D | -1 } of size | B | we have

If | D | -1 ̸∈ b then b ⊂ { 0 , ..., | D | -2 } and the second term in (21) is zero, hence

as claimed. Now suppose | D | -1 ∈ b . Then b \ {| D | -1 } is a subset of { 0 , ..., | D | -2 } of size | B | -1 , hence

This completes the proof.

Using Lemma A.1, we can express the expectation with respect to B D t in terms of J , B ′ , and ˜ B . This will provide us with the desired decomposition of the sum in (5). The definition (20) is reminiscent

of the constructions in Proposition 23 in Appendix C of Wang et al. (2019), however our analysis will also result in tighter bounds for DP-SGD than one gets from the general-purpose method in Wang et al. (2019).

Using Lemma A.1 in the manner discussed above, the transition probabilities (5) for one step of DP-SGD with FSwoR-subsampling can be rewritten as follows:

Recalling that B ′ ∼ B D ′ t and noting that µ t ( θ t , B ′ , D ′ ) = µ t ( θ t , B ′ , D ) (as D ′ agrees with D except for the last missing entry, which by definition cannot be contained in B ′ ), the transition probabilities p t ( θ t +1 | θ t , D ′ ) can similarly be written

The outer expectations in the expressions (24) and (25) are now both with respect to the same distribution, with the key difference being captured by the parameter q . The construction (20) was introduced precisely so that we might obtain such a decomposition. It allows the next step to proceed as in DP-SGD with Poisson subsampling Abadi et al. (2016); Mironov et al. (2019). Namely, we use quasiconvexity of the R´ enyi divergences (i.e., the data processing inequality) Van Erven and Harremos (2014) to obtain

where, as in Abadi et al. (2016); Mironov et al. (2019), we used a change of variables to reduce the computation to 1-dimension in the last line.

Remark A.2 . We note that the above analysis can be applied to any divergence, D , that is quasiconvex in its arguments, not just the R´ enyi divergences. In the general case one similarly finds

For instance, this holds for all f divergences, including the important class of hockey-stick divergences that are important for other DP frameworks such as in Zhu et al. (2022).

Returning to (26), recalling that C is the clipping bound and that ˜ b ⊂ b ′ and | b ′ \ ˜ b | = 1 , we can compute

We note that this result is 4 times as large as the corresponding calculation in the case of Poisson subsampling, i.e., the sensitivity in the case of fixed-sized subsampling is inherently twice that of Poisson subsampling when using add/remove adjacency. This is because in Poisson subsampling when the minibatch from D differs from that of D ′ it is due to the inclusion of a single additional element. However, in fixed-size subsampling, when the minibatches are not identical then they differ by a replacement; this contributes more to the difference in means by a factor of 2 .

Combining (29) with the fact that scaling the means in Gaussian mixtures by the same factor c ≥ 1 can only increase the R´ enyi divergence, see Section 2 in Mironov et al. (2019), we obtain the following uniform R´ enyi bound.

Now we show that the right-hand-side of (30) also bounds the case where D ′ has one additional element. Repeating the steps in the above derivation, in this case we obtain

where ˜ q = | B | / ( | D | +1) ; the appearance of | D ′ | = | D | +1 here is due to the interchange of the roles of D and D ′ in the decomposition (20) for this case. Next use Theorem 5 in Mironov et al. (2019) to obtain

Quasiconvexity of the R´ enyi divergences implies the right-hand-side of (32) is non-decreasing in ˜ q . Therefore, as ˜ q ≤ q , we obtain

The upper bound here is the same as the one we obtained in (30) when D ′ had one fewer element than D . Finally, by changing variables we can rescale the mean to 1 and therefore, noting that

we obtain

This completes the proof.

B Taylor Expansion of H α,σ ( q )

In this appendix we provide details regarding the Taylor expansion (9) of H α,σ ( q ) (Eq. (8)), which can be written

In Section 3.2.1 we obtained one-step RDP bounds in terms of the R´ enyi divergence

To obtain the computable RDP bounds in Theorem 3.3 and also in Theorem 3.6 we must bound H α,σ ( q ) . We proceed by computing the coefficients in its Taylor expansion, including an explicit upper bound on the remainder term (10). First note that using the dominated convergence theorem (see, e.g., Theorem 2.27 in Folland (1999)), it is straightforward to see that H α,σ ( q ) is smooth in q and can be differentiated under the integral any number of times:

This justifies the use of Taylor's theorem with integral remainder to H α,σ for any order m ∈ Z + to arrive at

where

(note that the M σ, 1 = 0 , which is why the summation in Eq. (38) starts at k = 2 ) and the remainder term is given by

For integer k ≥ 0 , the M σ,k 's can be computed using the binomial theorem together with the formula for the moment generating function (MGF) of a Gaussian:

In particular, M σ, 0 = 1 and M σ, 1 = 0 . If α is an integer then the expansion (38) truncates at a finite number of terms (terms with k > α are zero), but for non-integer α one must bound the remainder term. To enhance numerical stability of the computations, in practice we add terms starting from the highest order and proceeding to the lowest order.

Bounding the Taylor Expansion Remainder Term:

To bound the remainder term (40) we need to bound dq m H α,σ ( sq ) for t, q ∈ (0 , 1) . We will break the calculation into two cases, where different methods are appropriate. Here we assume q < 1 .

In these bounds, it will be useful to employ the following definition for integer j ≥ 0

d m :

For j even we have B σ,j = M σ,j and for j odd we can use the Cauchy-Schwarz inequality to bound

as j ± 1 are both even. Therefore an explicitly computable upper bound for all integer j ≥ 1 is given by

where M σ,k is given by (41).

Case 1: α -m> 0 Using the bound x α -m ≤ 1 + x ⌈ α ⌉-m for all x > 0 we can compute

Therefore

Case 2: α -m ≤ 0 In this case we have and therefore

This leads to the following bound on the remainder

where ˜ B σ,m is defined by (44).

Explicit bounds for m = 3 : We find that m = 3 generally yields sufficiently accurate bounds. Here we specialize the above results to this case. For q < 1 we have

with

This completes the proof of Theorem 3.2.

To illustrate the use of these bounds, and the fact that m = 3 is often sufficient in practice, in Figure 6 we plot the FSwoR-RDP bound from Theorem 3.3 (which uses the expansion and bound (52) - (53)) for one step (i.e, T = 1 ) and with the parameter values σ t = 6 , | B | = 120 , and | D | = 50 , 000 . Note that the difference between the bound for m = 3 and m = 4 is negligible over a wide range of α (sufficient to cover the default α range needed by Meta Platforms (2024)). The value of q in this example is far from being unrealistically small; smaller values of q will only make the effect of m even less important. Therefore m = 3 is generally sufficient.

0

10

20

30

40

50

60

10

6

10

5

One-step FSwoR-RDP Bound (a/r)

Thm.3.3 upper bound (m = 3)

Thm.3.3 upper bound (m = 4)

C Proof of Theorem 3.4: FSwoR-RDP Bound under Replace-One Adjacency

We now prove the FSwoR-RDP bound under replace-one adjacency as stated in Theorem 3.4 above. Without loss of generality, we can assume that D and D ′ share all elements except the last. The derivation shares several initial steps with the add/remove case from Appendix A. Specifically, we apply Lemma A.1 to the transition probabilities for one step of DP-SGD with FSwoR-subsampling, both for D and for D ′ (as they now have the same number of elements), to obtain

similarly to (24). Now we again use quasiconvexity, this time in both arguments simultaneously, to bound the effect of the mixture (expectation) over ( b ′ , ˜ b ) by the worst case, yielding the one-step R´ enyi bound

Using the fact D and D ′ agree on the index minibatch b ′ , and hence µ t ( θ t , b ′ , D ) = µ t ( θ t , b ′ , D ′ ) , we see that P b ′ = P ′ b ′ . We can therefore change variables to center both of their means at zero, giving

The means of the Gaussians satisfy the bounds

where r t was defined in (29); the first two inequalities follow directly from (28) while the proof of the third is almost identical due to D and D ′ differing only in the last element.

Remark C.1 . Note that, similarly to Remark A.2, Eq. (56) remains true when the R´ enyi divergences D α are replaced by any other divergence D that is quasiconvex in both of its arguments, e.g., the hockey-stick divergences. This fact is useful for other differential privacy paradigms, though using the quasiconvexity bound does not always lead the tightest possible bounding pair of distributions; see Lebeda et al. (2024).

The key difference between the bound (56) and the corresponding result (26) in the add/remove adajcency case is that both arguments of the R´ enyi divergence on the right-hand side of (56) are now Gaussian mixtures with mixing parameter q ; in the add/remove adjacency case only one argument is a Gaussian mixture. This difference makes the following computations more involved, though the same Taylor expansion technique can be applied.

Using the definition of R´ enyi divergences we can write

Next we Taylor expand the argument of the logarithm:

where here the remainder term is given by

Using the formula d k dx k ( fg ) = ∑ k j =0 ( k j ) d j dx j f d k -j dx k -j g along the dominated convergence theorem to justify differentiating under the integral, the derivatives can be computed as follows:

Taylor expansion coefficients for k = 0 , 1 , 2 : Evaluating (61) at q = 0 we have

Note that we have not yet been able to maximize over b ′ , ˜ b ; doing so while incorporating the constraints (57) is nontrivial and constitutes one of the the primary difficulties in obtaining sufficiently tight bounds. We will make particular attention to the first few terms and then use looser (but still sufficiently tight in practice) approximations for the higher order terms. First note that F α (0) = 1 and

For k = 2 we can expand (62) and then evaluate the integrals by using the formula for the MGF of a Gaussian to obtain

Using the third constraints in Eq. (57) we obtain the bound

Elementary calculus along with the bounds ∥ ∆ µ t ( b ′ , ˜ b ) ∥ ≤ r t , ∥ ∆ µ ′ t ( b ′ , ˜ b ) ∥ ≤ r t then imply

where we used (34). These terms are sufficient to precisely capture the leading-order behavior.

Bounding the coefficients for k ≥ 3 : For the higher-order terms we obtain an upper bound which relies on the following lemmas. It will be useful to phrase various bounds in terms of the χ β divergences.

Definition C.2. For β > 0 define the χ β divergences

It will also be convenient to adopt the notation D χ 0 := 1 , despite it not defining a divergence.

The following property is a imply consequence of H¨ older's inequality.

Lemma C.3. Let γ ≥ β > 0 . Then

The next lemma will be used to obtain a uniform bound, i.e., not dependent on b ′ , ˜ b , or θ t . Lemma C.4. For k ≥ 0 an integer we have

where ˜ B was defined in (44) . These bounds also hold with ∆ µ t and ∆ µ ′ t interchanged.

Proof. D χ k is the f divergence corresponding to f β ( y ) = | y -1 | k , therefore it is jointly convex in ( Q,P ) . Therefore the same argument as in Section 2 of Mironov et al. (2019) (which only relies on the convexity of the divergence) implies

where we used (57) and (44). The second result in (69) is proven in a similar manner, where we must also change variables to center the second Gaussian at zero and then employ the third bound in (57).

The second lemma we require is a slight improvement on Lemma 24 of (Wang et al., 2019).

Lemma C.5. Let p, q, r be strictly positive probability densities and k ≥ 1 be an integer. Then

Remark C.6 . Note that a key difference between our technique and that of Wang et al. (2019) is that we only apply Lemma C.5 to the higher order terms (i.e., when k ≥ 2 ) while we bounded the leading contribution (66) more precisely. In contrast, Wang et al. (2019) used Lemma C.5 (their Lemma 24) on all terms.

Proof. We proceed by breaking the domain of integration into four regions, depending on the sizes of p, q, r and then use the appropriate bounds to replace one of the densities with another:

This completes the proof.

Lemma C.7. Let β 1 , β 2 > 0 with β 1 + β 2 and integer. Then

where ˜ B σ,k was defined in (44) .

Proof. Let p, q > 1 be conjugate exponents, i.e., they satisfy 1 /p + 1 /q = 1 . Using H¨ older's inequality we can compute

We now use a simple heuristic that allow for p, q to be chosen so as to minimize the right-hand side of this upper bound when β 1 , β 2 are large. Lemma C.4 together with (44) and (41) suggest that the two factors on the right-hand side approximately behave like exp(2 pβ 2 1 /σ 2 t ) and exp(2 qβ 2 2 /σ 2 t ) respectively. Therefore, to make the upper bound as tight as possible, we want to to minimize pβ 2 1 + qβ 2 2 subject to p > 1 , q = p/ ( p -1) . Simple calculus shows that the minimum occurs at p := 1 + β 2 /β 1 , q = 1 + β 1 /β 2 . Making this choice, using Lemma C.3 to convert to integer orders, and then employing Lemma 69 we find

In the case where β 1 + β 2 is an integer the above can be simplified via

We use these lemmas to bound the higher order derivatives as follows. First rewrite (62) as

The first family of integrals can be bounded using Lemma C.5 and then Lemma C.4 to obtain

The second family of integrals can be bounded by using Lemma C.7 (when j, k -j > 0 ) and Lemma C.4 (when j = 0 or j = k ), which yields

Therefore we find

where M and ˜ B are given by (41) and (44) respectively.

Bounding the remainder term: We now proceed to bound the remainder term (60). Using (61), for q < 1 we can compute

to upper bound the reciprocal of the denominator.

To bound the integrals K α,m,j we consider the following two cases.

Case 1: α -j ≤ 0 : In this case we use

to bound the first factor in the integrand. Then, using Lemma C.7 and Lemma C.4, we can compute

Case 2: α -j > 0 :

In this case use the bound x α -j ≤ 1 + x ⌈ α ⌉-j for all x > 0 , followed by the binomial formula and Lemma C.7 to compute

Putting the above two cases together we arrive at the remainder bound

Combining the above results, we obtain the following FSwoR-RDP bounds under replace-one adjacency:

where m ≥ 3 is an integer, ˜ F α,σ t ,k is given by (80), and ˜ E α,σ t ,m ( q ) is given by (87); both of these quantities are expressed in terms of M and ˜ B , as given by (41) and (44) respectively. This completes the proof of Theorem 3.4.

Remark C.8 . The same M 's appear many times during in the formulas for ˜ F α,σ t ,k and ˜ E α,σ t ,m ( q ) . Therefore, in practice, we compute all required values once and recall them as needed.

C.1 Comparison with Poisson Subsampling under Replace-one Adjacency

In this appendix we provide a precise comparison between RDP for fixed-size and Poisson subsampling under replace-one adjacency, complimenting the intuitive discussion in Section 4.1.

Here we let p Poisson t ( θ t +1 | θ t , D ) denote the transition probabilities for one step of DP-SGD with Poisson subsampling. By Mirroring the analysis in Section 2 of Mironov et al. (2019), but now under the assumption that D and D ′ have the same size and differ in only their last elements, we arrive at the one-step R´ enyi divergence bound

where T denotes the collection of indexes from { 0 , ..., | D | -2 } that were randomly selected during Poisson sampling.

This is similar to the result (56) for FSwoR-subsampling, except here the means satisfy the constraints

and these bounds are achieved when the differing elements from D and D ′ are anti-parallel and saturate the clipping threshold C . As we will show, this difference only contributes at higher order.

We now mirror the analysis from earlier in this section, taking the new constraints (90) into account. We again have a Taylor expansion of the form

The coefficient at k = 2 can be computed as in (62), which gives

This bound is optimal in that equality is achieved when the distinct elements in D and D ′ are anti-parallel and saturate the clipping bound. To leading order in 1 /σ 2 t we therefore have

which agrees with the leading order behavior of the corresponding term in the FSwoR-subsampling case (66). As the second order coefficient in the Taylor expansion controls the dominant behavior of the RDP bound, we can therefore conclude that Poisson and FSwoRsubsampling give the same DP bounds under replace-one adjacency to leading order.

Our method for bounding the higher order terms, as in (80), and remainder, as in (86), only uses bounds on the norms of ∆ µ ′ and µ ′ ; it does not incorporate the bound that couples these two quantities. Therefore the corresponding bounds on the higher order terms for Poisson subsampling are obtained by simply replacing r t with r t / 2 in our prior calculations; this is equivalent to replacing σ t with 2 σ t in (80) and (86). Therefore we arrive at the following non-asymptotic bound for Poisson subsampling under replace-one adjacency.

Theorem C.9 (Poisson-RDP Upper Bounds for Replace-one Adjacency) . Let D ≃ r -o D ′ be adjacent datasets. Assuming q := | B | / | D | < 1 , for any integer m ≥ 3 we have

where ˜ F α, 2 σ t ,k and ˜ E α, 2 σ t ,m ( q ) are computed via (80) and (87) respectively (with σ t replaced by 2 σ t ).

While our FSwoRand Poisson-subsampled RDP bounds agree to leading order, the effect of the higher order terms leads to Poisson subsampling having a slight privacy advantage over FSwoR-subsampling in practice, especially when larger α 's are required. This proven in Figure 7, where we see that the upper bound on Poisson-subsampled RDP (dashed line) is slightly below the theoretical lower bound for FSwoR-subsampled RDP derived in Wang et al. (2019) (circles) while our FSwoR-RDP bound from Theorem 3.4 (solid line) is slightly above the theoretical lower bound. All three are significantly below the upper bound from Wang et al. (2019) (dot-dashed line). We emphasize that the Poisson result does not contradict the lower bound from Wang et al. (2019), which only applies to FSwoR-subsampled RDP.

50

52

54

56

58

60

0.2

0.4

0.6

0.8

1.0

One-step FSwoR-RDP Bound (r-o)

1e

4

FSwoR upper bound (m = 5)

Poisson upper bound (m = 5)

Wang et al. upper bound

Wang et al. lower bound

D R´ enyi DP-SGD Bounds for Fixed-size Subsampling with Replacement

In this Appendix we provide a detailed derivation of R´ enyi DP-SGD bounds when using fixed-size subsampling with replacement (FSwR). In addition to the lower bound stated in Theorem 3.7, we also derive an upper bound.

As in case without replacement in Appendix A, a key step is a decomposition of the sampling distribution into a baseline (where D and D ′ agree) and a perturbation where they disagree. The derivation for subsampling with replacement is a non-trivial generalization of sampling without replacement, with Lemma D.1 below requiring a several key new ingredients, as compared to Lemma A.1.

First let D ′ be obtained from D by removing one element. Without loss of generality, assume it is the last (i.e., at index | D | -1 ) and call that element d . Let B ′ = ( B ′ 1 , ..., B ′ | B | ) where the components are iid Uniform( { 0 , ..., | D | ′ -1 } ) random variables, so that B ′ uniformly samples from { 0 , ..., | D | ′ -1 } with replacement. Now let N ∼ Binomial( | B | , | D | -1 ) and Π be a uniformly random permutation of (1 , ..., | B | ) . Assume that B ′ , N , and Π are independent. In the following lemma we show how sampling from D with replacement can be obtained from B ′ , N , and Π . In short, N will determine the number of entries of B ′ are replaced by | D | -1 and Π will determine the indices of the entries that are to be replaced.

Lemma D.1. For i = 1 , ..., | B | define

Then the B i are iid Uniform( { 0 , ..., | D | -1 } ) random variables, i.e., B := ( B 1 , ..., B | B | ) uniformly samples from { 0 , ..., | D | -1 } with replacement.

Proof. Let b ∈ { 0 , ..., | D | -1 } | B | and define n = |{ i : b i = | D | -1 }| . Let i 1 , ..., i n be the unique indices with b i j = | D | -1 . If B = b then we must have N = n , therefore if we let S n denote the set of permutations on { 1 , ..., n } we have

Using independence and that | D ′ | = | D | -1 and | S n | = n ! we can then compute

This holds for all b , therefore we can conclude that the distribution of B is the uniform distribution on { 0 , ..., | D | -1 } | B | . All the remaining claims then immediately follow from this fact.

Using Lemma D.1 we can decompose the transition probabilities for one step of DP-SGD with FSwR-subsampling as follows

Noting that µ t ( θ, b ′ , D ′ ) = µ t ( θ, b ′ , D ) , as D ′ differs from D only by removal of the last element, we now use quasiconvexity to compute

Remark D.2 . As in Remarks A.2 and C.1, Eq. (101) remains true when the R´ enyi divergences D α are replaced by any other divergence D that is quasiconvex in both of its arguments, e.g., the hockey-stick divergences. This fact is useful for other differential privacy paradigms.

D.1 Upper Bound on FSwR-RDP

The next stages in the calculation are more complex than in the case of sampling without replacement, as the first argument of the above R´ enyi divergence is a mixture of | B | +1 Gaussians, as opposed to simply two as in the case without replacement. This is because the binomial random variable N in (D.1) is playing the role that the Bernoulli random variable J did in Lemma A.1.

Start by rewriting Eq. (101) as

Let 1 < K ≤ | B | , choose ˜ q satisfying

and decompose the mixture of distributions as follows

The assumption (103) implies ˜ a n ∈ [0 , 1] and ˜ a 0 + ∑ K n =1 ˜ a n + ∑ | B | n = K +1 a n = 1 and therefore we can use convexity to bound (102) as follows:

where H was defined in Eq. (8) and can be bounded using the techniques from Appendix B.

Now suppose D ′ is obtained from D by adding one new element. By reversing the role of D and D ′ in Lemma D.1 and then mirroring the above derivation while also employing Theorem 5 in Mironov

et al. (2019) we arrive at

whenever ˜ q satisfies (103). Note that (103) also implies

due to the fact that a 0 is increasing in | D | and a n is decreasing in | D | for n ≥ 1 . These properties also imply

and therefore we can conclude that

whenever D ′ ≃ a/r D and any choice of ˜ q that satisfies

Note that if one chooses ˜ q = ( 1 + a 0 ∑ K n =1 a n ) -1 then ˜ a 0 = 0 . If one also takes K = | B | then ˜ q = 1 -(1 -| D | -1 ) | B | . This is the version of the result we present above in Theorem 3.6.

D.2 Lower Bound on FSwR-RDP

̸

Here we derive a lower bound on the one-step RDP of SGD with fixed-size subampling with replacement. To do this, suppose that all entries of D ′ are the same, equal to d ′ , and that the extra entry d in D satisfies d = d ′ . Moreover, suppose there exists θ t such that the clipped gradients at d and d ′ are anti-parallel with both having norm equal to the clipping bound, C . In this case, µ t ( θ t , b ′ , D ) and µ t ( θ t , Φ( b ′ , n, π ) , D ) are independent of both π and b ′ and

This also implies that (101) is an equality (and the argument of the divergence is independent of b ′ and π ). By changing variables and using (111) we can therefore write

This gives the one-step RDP lower bound

The integral over θ can be computed as follows. First note that summation in (113) is the average of N n,σ 2 t / 4 ( θ ) /N 0 ,σ 2 t / 4 ( θ ) under the Binomial distribution Binomial( | B | , | D | -1 ) . As a convenient shorthand we write this average as ∫ ...dn . With this notation, for integer α we can rewrite the argument of the logarithm in the lower bound (113) an iterated integral, then exchange the order of the integrals and evaluate the integral over θ using the formula for the MGF of a Gaussian as follows

Therefore we arrive at the lower bound

Recalling the definition of the dn i 's, this completes the proof of Theorem 3.7.

The lower bound (115) behaves somewhat differently from the one derived for subsampling without replacement in Wang et al. (2019) and restated in Appendix E.2. In particular, note that the explicit | B | and | D | dependence in the lower bound (113) cannot be removed simply by reparameterizing the expression in terms of q = | B | / | D | . In contrast, the FSwoR-RDP lower bound in Appendix E.2 depends on | B | and | D | only through q . This fact can be further illustrated through a simple loosening of the bound (113), where we bound the sum below by the term at n = | B | to obtain

For fixed ratio q = | B | / | D | , the right-hand-side of (117) approaches ∞ as | B |→∞ . Therefore we can conclude that ϵ ′ t ( α ) has nontrivial | B | dependence, even for fixed q as claimed. This accounts for

0

25

50

75

100

125

150

175

200

|

B

|

10

6

10

4

10

2

10 0

10 2

One-step RDP Lower Bound

=2.0

=3.0

=4.0

=5.0

=6.0

some of the difference in behavior between fixed-size subsampling with and without replacement. This behavior is also illustrated numerically in Figure 8, where we show the exact value of the lower bound (113) as a function of | B | for α = 2 (this case is straightforward to compute via the binomial theorem and MGF of a Gaussian) and for fixed q = 0 . 001 ; we show results for several values of σ . Note that initially each curve starts out horizontal, mirroring the case of FSwoR-RDP, which has no | B | dependence outside of q . However, for each σ there is a threshold value of | B | as which there is a 'phase transition', i.e., after which the explicit | B | dependence becomes important and the RDP value increases rapidly.

D.2.1 Comptutable FSwR-RDP Lower Bounds

When α is an integer the lower bound (116) can, in principle, be computed exactly by recalling that the integrals with respect to dn i are shorthand for the expectation with respect to the distribution Binomial( B, | D | -1 ) . Therefore the lower bound (116) consists of α nested sums of | B | +1 terms each. Clearly, when α is larger than 2 and | B | is not small iterated summation this quickly becomes computationally impractical. The looser lower bound (117) is easy to compute, but is much too inaccurate for most purposes. Here, for integer α , we will show how obtain lower bounds that are both practical and significantly more accurate.

Recall that the integrals with respect to the dn 's are actually finite sums, each over | B | +1 elements, and so in principle the above expression can be computed exactly but it is still not practical. We next show how to obtain recursively defined lower bounds that are significantly less computationally expensive than the O ( | B | α ) computations that the more naive formula (116) requires. To begin, for any integer k ≥ 2 define

For k = 2 we can reduce the computation to a single summation by using the formula for the MGF of the Binomial distribution:

For k > 2 we explicitly write out the integral over n k as a summation and then simplify to obtain F ( c, d ) (120)

As stated, the recursive formula provided by (119) and (120) is still too computationally expensive due to each recursive call requiring the summations over | B | +1 elements. However, noting that terms in the summations are all positive, we can obtain a practical lower bound by retaining only a small number of terms. More specifically, given choices of T k ⊂ { 0 , ..., | B |} for each k , if we recursively define

with

Then, by induction, we have F k ( c, d ) ≥ F T,k ( c, d ) for all k, T, c, d and therefore

for all α and all choices of T k , k = 2 , ..., α . By appropriately choosing the number of terms T k for each k one trades between accuracy and computational cost. In particular, if each | T k | = 1 then the computation of (123) only requires O ( α ) computations. If | T k | = ℓ > 2 for all k then the computation of (123) required O ( ℓ α -1 ) computations. In contrast, using the exact recursive formula (120) requires O ( | B | α -1 ) computations, which is intractable unless α or | B | is sufficiently small. However, despite this improvement, we should note that computational difficulties persist for cases where the number of nontrivial terms is too large.

In our experiments in Figure 9 we find that that using T k = {| B |} for k > 2 and T 2 = { 0 , ..., | B |} gives an accurate and computationally tractable lower bound. In the example in Figure 1 more terms were required; we show results obtained by using T k = { 0 , 1 , 2 , | B |} for k > 2 and T 2 = { 0 , ..., | B |} .

D.3 Comparison of Upper and Lower FSwR-RDP Bounds

Now we will compare our lower bound for FSwR-RDP from Theorem 3.7 with the upper bound derived in Appendix D.1. In Figure 9 we show the FSwR-RDP upper bound (solid line) and lower bound (circles) and, for comparison, we also show our bound on FSwoR-RDP (dashed line). Note that all three are very similar when α is close to 1 ; this is to be expected, due to them having the same leading order behavior in q , given by (19). However, as α increases the non-leading order behavior becomes more important and we observe that, after some threshold α , both the FSwR-RDP upper and lower bounds increase rapidly. A similar 'phase transition' was observed earlier in Wang et al. (2019) for subsampling without replacement, though for these parameters it occurs at much higher values of α . A key difference between these two cases is that, for FSwR-RDP, the phase transition can be brought on by increasing | B | while leaving q = | B | / | D | fixed, as demonstrated in Appendix D.2; see (117) and Figure 8. In contrast, the lower bound for FSwoR-RDP from Wang et al. (2019) depends on | B | and | D | only through their ratio q . We anticipate that the gap between the upper and lower bounds on FSwR-RDP in Figure 9 could be reduced by using a higher order expansion in q , but we leave that to future work. We also emphasize that the dashed FSwoR-RDP upper bounds is only for comparison purposes as it does not apply to the FSwRcase considered in this section.

which simplifies to

2

4

6

8

10

10

5

10

3

10

1

10 1

10 3

One-step RDP Bound

-RDP upper bound

-RDP lower bound (

n

=|

B

| term)

FSwoR

-RDP upper bound

E Comparison with Wang et al. (2019)

In Wang et al. (2019), R´ enyi-DP enhancement of fixed-size subsampling was analyzed in a framework that applies to arbitrary mechanisms. In this appendix we specialize their results to fixed-size subsampling DP-SGD with Gaussian noise. This will facilitate comparison with our results. Specifically, we will show that our specialized method results in a factor tighter bounds by approximately a factor of 4 .

E.1 Upper Bound from Wang et al. (2019), Theorem 9

For integer α ≥ 2 , the following the general fixed-size subsampling R´ enyi-DP enhancement bound (for one step) was derived in Wang et al. (2019):

(if α = 2 then the empty summation is interpreted to be zero), where ϵ t ( j ) denotes a R´ enyi-DP bound for the mechanism without random subsampling. For DP-SGD we have

and therefore we can use

Note that (125) uses the same bound on the means that we use in our method, and therefore (126) will yield a fair comparison. Comparing the leading order behavior of (124) with the leading order behavior of a single step of our result in Theorem 3.3, we see that our bound is smaller by a factor of 4 . In practice we find similar behavior even when including the higher order terms; see Figure 2.

For non-integer α ≥ 2 they use the following bound, which follows from convexity of K t ( α ) in α :

where ⌊ x ⌋ denotes the floor of x . For α ∈ (1 , 2) the convexity bound is

E.2 Lower Bound from (Wang et al., 2019), Proposition 11

A general lower-bound on fixed-size subsampling enhancement was also derived in (Wang et al., 2019); we repeat it here for convenience. For integer α ≥ 2 they show

where ϵ t ( j ) is again given by (126). In Figure 2 we demonstrate that our result is very close to this theoretical RDP lower bound.

E.3 Comparison with Replace-One Adjacency Bounds in Theorem 3.4

The analysis in Wang et al. (2019) is done under the replace-one adjacency definition and is therefore directly compatible to our Theorem 3.4. In Section 3.4 we showed that our replace-one and our add/remove adjacency results, Theorem 3.4 and 3.1 respectively, lead to the same RDP bounds to leading order in q and 1 /σ 2 t , including the factor of 4 improvement over Wang et al. (2019); this confirms that the improvement made by our approach is not reliant on the adjacency definition.

In this appendix we provide a conjecture as to the origin of this difference. The leading order behavior comes from the calculations (64)-(66), where we used the constraints on the means from (57):

However, if one doesn't use the constraint on the difference and only uses the first two upper bounds on the means then the best bound on the dot product becomes

and so the bound (66) is weakened to

which at leading order is a factor of 4 larger than our result (66) which was derived using all three constraints in (57). The strategies we employ differ from Wang et al. (2019) in a way that makes it difficult to pinpoint exactly what the essential differences are, however the above calculation leads us to conjecture that the difference originates in the failure to account for the constraint that couples ∆ µ t and ∆ µ ′ t . See also Remark C.6.

F Variance of Fixed-size vs. Poisson Subsampling

In this appendix we provide detailed calculations of the variance comparison that was discussed in Section 4.2. Let a i ∈ R , i = 1 , ..., | D | (for application to DP-SGD, one can take a i = ∇ θ L ( d i ) · v , the gradient of the loss at the i 'th training sample d i in the direction v ) and let | B | ∈ { 1 , ..., | D |} . Let J i , i = 1 , ..., | D | be iid Bernoulli( | B | / | D | ) random variables, B be a random variable that uniformly selects a subset of size | B | from { 2 , ..., | D |} , and R j , j = 1 , ..., | B | be iid uniformly distributed over { 1 , ..., | D |} . We will compare the variance of the Poisson-subsampled average,

with that of fixed-size subsampling with replacement,

and with fixed-size subsampling without replacement,

They all have the same expected value, equaling the average of the samples:

̸

and

The variances are

̸

̸

̸

and

̸

̸

Therefore, defining a 2 = 1 | D | ∑ | D | i =1 a 2 i , when | B | < | D | and a 2 = 0 we obtain the following variance relations

In particular, we see that fixed-size subsampling with replacement always has larger variance than fixed-size subsampling without replacement by the constant multiple 1 -1 / | D | 1 -q > 1 ; however, this factor is close to 1 when q is small. Therefore, for practical purposes we can consider them equivalent from the perspective of the variance. When | B | < | D | the ratio of fixed-size and Poisson variances is

̸

The Cauchy-Schwarz inequality implies 0 ≤ a 2 /a 2 ≤ 1 and so we see that Var[ Z B ] < Var[ Z P ] when a = 0 and | D | is sufficiently large; in DP-SGD the condition a = 0 corresponds to being away from a (local) minimizer or critical point of the loss. This calculation suggests that fixed-sized minibatches are better to use when one is away from the optimizer, as they lead to a (potentially substantial) reduction in variance, but they can become (slightly) worse than Poisson-sampled minibatches when one is close to a local minimizer (i.e., a ≈ 0 ). However, for a typical training-set size | D |≫ 1 we have (1 -1 / | D | ) -1 ≈ 1 and so the variance advantage of Poisson subsampling when a ≈ 0 is very minor. In addition, due to the effect of noise, DP-SGD cannot reach a = 0 . Therefore, from the perspective of the variance, we conclude that fixed-size subsampling (with or without replacement) is generally preferable over Poisson subsampling.

̸

G Comparison of Poisson and Fixed-size DP Bounds under Add/Remove Adjacency

Using the conversion from Proposition 3 in Mironov (2017), if a mechanism satisfies ϵ ′ ( α ) -RDP then it satisfies ( ϵ, δ ) -DP with

and

̸

̸

Under add/remove adjacency, to leading order, both Poisson and fixed-size subsampling satisfy RDP bounds with ϵ ′ ( α ) = αR for some R > 0 (see Abadi et al. (2016); Mironov et al. (2019)) and (19). Assuming that the optimal α is not close to one, and hence we can approximate α with λ := α -1 , minimizing over α gives

For t steps of fixed-size subsampling (with or without replacement), and with constant variance σ , Eq. (19) implies

In contrast, for Poisson subsampling the results in Abadi et al. (2016); Mironov et al. (2019) imply

Therefore R FS ≈ 4 R P at leading order and so (147) implies

From a privacy perspective, this suggests that ϵ for Poisson subsampling has an intrinsic advantage over fixed-size subsampling by approximately a factor of 2 when using the add/remove adjacency relation. Noting the scaling of (150) with σ , we see that fixed-size subsampling requires noise 2 σ to obtain the same privacy guarantee (at leading order) as Poisson sampling with noise σ . Tighter translation between RDP and ( ϵ, δ ) -DP, such as Theorem 21 in Balle et al. (2020) which is used by Meta Platforms (2024), alters this story somewhat but the general picture we have outlined here is corroborated by the empirical comparison provided in Section 4.2. We emphasize that this factor of two gap disappears when using replace-one adjacency; in that case Poisson subsampling and FSwoR subsampling lead to the same DP guarantees, as discussed in Section 4.1.

H Additional Experiments

To assess the sensitivity of our proposed method to important parameters such as σ and batch size | B | , we conducted a series of additional experiments to compare our proposed fixed-size method with the non-fixed size setting while varying values of B ∈ { 50 , 100 , 150 , 200 } and σ ∈ { 3 . 0 , 4 . 5 , 6 . 0 , 12 . 0 } . Figure 10 depicts the results of these experiments.

As shown in Figure 10, the results for varying values of σ and | B | align with what we reported in Section 4.2, showing that the proposed method is not sensitive to parameter settings.

a Performance with different values of

σ

∈ {

3

.

0

,

4

.

5

,

6

.

0

,

12

.

0

}

0=3.0

0=4.5

=6.0

0=12.0

60

70

55

09

60

60

50

50

45

accura

2

50

40

40

40

40

35

OE

30

30

25

FSwoR-RDP(m=10)

FSwoR-RDP (m=10)

FSwoR-RDP (m=10)

FSWoR-RDP (m=10)

Poisson Subsampled RDP

Poisson Subsampled RDP

Poisson Subsampled RDp

Poisson Subsampled RDP

20

50

100

150

200

250

20

0

50

100

150

200

250

20

50

100

150

200

250

20+

0

50

100

150

200

250

IB|=50

|B|=100

|B|=150

IB|=200

70

70

60

09

60

Ct

testing accuracy

40

50

50

SE

40

40

40

30

30

30

25

FSwoR-RDP (m=10)

FSwoR-RDP (m=10)

FSwoR-RDP (m=10)

Poisson Subsampled RDP

Poisson Subsampled RDP

Poisson Subsampled RDP

Poisson Subsampled RDp

20

50

100

150

200

250

20

0

50

100

150

200

250

20

50

100

150

200

250

20

50

100

150

200

250

b Performance with different values of batch size | B | ∈ { 50 , 100 , 150 , 200 }

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The paper provides the first fixed-size subsampled analysis of DP-SGD, which practically contributes to constant memory usage in DP applications. The improvements on the state-of-the-art fixed size method proposed by Wang et al. is clearly shown by theory and in practice on CIFAR-10.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Section 4.3 and Figure 4 clearly state the benefits and limitations with respect to both state-of-the-art fixed size sub-sampling methods and traditional non-fixed-size sampling supported by empirical results on CIFAR-10.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper provides rigorous and complete proofs for the upper-bounds obtained in the fixed-size regime, both for with and without replacement cases. All proofs and derivations are carefully double-checked to be correct and complete.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: In addition to meticulously stating all key parameters in the experiments in Section 4 and all plots, the code to exactly replicate our results in Section 4, accompanied with all the numerical analysis and generated plots throughout the paper was made available during submission. We will also make the privacy accountant publicly available upon acceptance.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We submitted the code of our privacy accountant to allow exact replication of our results. The repository includes the code to replicate the results in Section 4, an example of using our accountant on CIFAR 10 as well as all the numerical analysis and generated plots throughout the paper. We will also make the repository for our privacy accountant and experiments publicly available upon acceptance.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The key (hyper)parameters are meticulously stated in the experiment section as well as all numerical analyses.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We noted in the main body that Figure 4 was plotted with 3-sigma standard deviation around mean based on five runs each with different random seeds for sampling: 0 , 1 , 364 , 2 , 560 , 3 , 000 , and 4 , 111 . Also, the code that generates the plots were provided in our provided source code.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper states that the experiments were run on a single work station with an NVIDIA RTX 3090 with 24GB internal memory. The runtime for all experiments was under 12 hours.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We envision no negative impacts of our work. Not human subjects was used. Datasets are publicly available. Also, the code associated with this work is not envisoned to be used for malicious purposes.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The introduction and conclusion both highlight the positive societal impact of our work on privacy preserving machine learning by providing a new privacy accountant that advances the status quo in enhancing users privacy as machine learning and deep learning becomes more ubiquitous in individuals lives. As noted above, we do not envision any negative societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The dataset is a publicly available common benchmark. Also the the proposed accountant helps privacy-preserving ML and is not envisioned to be used in a malicious manner.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The CIFAR-10 dataset and Opacus library are both widely-used open-source and were properly cited in the main body. No other external assets was used.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The code comes with a readme file that includes an example of using our privacy accountnt and describes how to replicate the results in the experiment section.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: No human subject or crowdsourcing was involved in this work.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work does not involve crowdsouring or human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the Rényi differential privacy (RDP) guarantees of the subsamplied Gaussian mechanism with fixed-sized random minibatches, when the so called add/remove neighborhood relation of datasets is considered. It uses similar techniqes as were used in the paper \n\nMironov, Ilya, Kunal Talwar, and Li Zhang. \"R\\'enyi differential privacy of the sampled gaussian mechanism.\" arXiv preprint arXiv:1908.10530 (2019),\n\nwhich augmented the seminar paper Deep Learning with DP (Abadi et al., 2016) and gave rigorous RDP guarantees for the Poisson subsampled DP-SGD.\n\nThe paper also studies both analytically and numerically the benefits of carrying out subsampling with fixed-sized minibatches instead of Poisson subsampling.\n\nSTRENGTHS:\n- The analysis looks solid, generally a well-written paper\n- The variance analysis and the numerical experiments showing the benefits of fixed-size subsampling without replacement seem novel and interesting\n- The analysis for the \"with replacement\" subsampling seems interesting though it is quite limited\n\nWEAKNESSES:\n- Clearly the biggest deficit of the paper is that it does not take into account some of the recent research in this area. Most importantly, it overlooks the work\n\nZhu, Yuqing, Jinshuo Dong, and Yu-Xiang Wang. \"Optimal accounting of differential privacy via characteristic function.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2022. https://proceedings.mlr.press/v151/zhu22c/zhu22c.pdf\n\nThe main result of this paper is a special case of the Thm. 11 by Zhu et al. (2022) : If $(P,Q)$ are a dominating pair of distributions for the base mechanism in substitute relation, then the fixed-size random subsampling without replacement give a dominating pair $(q P + (1-q) Q, Q)$ for removal neighbours and $(P, (1-q) P + q Q)$ gives a dominating pair for add neighbors. Then, we know the dominating pair $(P,Q)$ of the Gaussian mechanism in case of substiture relation of datasets (pair of one-dimensional Gaussians) and the result of this paper's mmain result (Thm. 3.1) follows since if a pair if dominating pair for the hockey-stick divergence, by the Blackwell theorem it dominates also for other $f$-divergences. \n\nSince this \"without replacement\" result consitutes such a big part of the paper, I think this is a major deficit and there should be a major revision before accepting this paper.\n\nThis with replacement upper bound seems interesting, however my impression is that it is quite conservative (see e.g. Fig. 7 of the appendix). Only the experimental results for the lower bound in case of \"with replacement\" are given in the main text.\n\nQUESTIONS:\nCan you comment on the accuracy of the upper bound in case of \"with replacement\"? Is it still open/unclear, how tight those bounds are?\n\nCould you provide \"with replacement\" upper bounds also for the hockey-stick divergence, i.e., do you obtain a dominating pair of distributions for the hockey-stick divergence?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper analyzes Differentially Private-SGD with fixed batch size (with and without replacement), through the lens of Rényi-DP. The bounds without replacement have a much better constant than previous ones; the ones with replacement are brand new.\n\nSTRENGTHS:\n- The results are important and likely to be impactful in the niche of DP deep learning. It’s cool to see progress on DP analysis that gets closer to what practitioners actually do. \n- The paper is really well written and thorough in its discussions. I really enjoyed reading it.\n\nWEAKNESSES:\nIt’s a bit of a nitpick, but the paper keeps claiming that this analysis is DP-SGD specific, and that’s what enables the tight bounds. It seems like it is specific to:\n- the Gaussian mechanism (with specific subsampling approaches)\n- and I guess additive contributions of datapoints to the function?\nAnyway, it might be good to give the specifics in a slightly more general/abstract fashion, and then discuss why it applies to well to SGD? Just so that the reader doesn’t expect tight couplings with the optimization or something, and also because it may be slightly more broadly useful.\n\nMinor:\n- several citations seem off: the “standard differential privacy (DP)” citation is CDP, poisson sampling from RDP cites the RDP paper instead of this one I assume https://arxiv.org/pdf/1908.10530.\n- \"even without using the convexity technique” p. 6 -> this has never been mentioned before. You need more details here for context.\n\nQUESTIONS:\nSince you reduce to the same intermediary quantity as https://arxiv.org/pdf/1908.10530 (in Eq. 6), does it mean that:\n- the bound of that paper applies as-is to fixed batches using your proof (i.e., if one were using the Opacus accountant with fixed batches as a heuristic, it’s actually not a heuristic)?\n- if not, why is that?\n- if so, how does your new bound compares? (other than enabling the analysis with replacement, which is cool!)\nI think that the paper would benefit from discussing those to give a bit more context. S4.2 is related, as it gives an interesting discussion of where the different factor comes from (so the quantity is not identical I guess), but not all of this, like whether you could use the previous approximation as is, and how the two numerical procedures compare.\n\nAfter reading S4.2, I am wondering if the difference is a bit artificial, or at least tied to the notion of sensitivity. For instance, Poisson sampling would also pay a factor two under the change one definition (I think?) whereas your approach would mostly not change I believe (I only skimmed the appendix on sensitivity to I’m not completely sure, but intuitively it’d make sense—the section doesn’t seem to discuss the implications on Poisson vs. fixed batch). Is it the case that your approach pays a 2x factor, but also gets the stronger change-one for free, whereas Poisson gets the better guarantee under add/remove, but its equivalent under change-one? It would be an interesting thing if that’s the case I think (change-one is pretty popular in other applications, I don’t think it’s really less conventional as the paper claims).\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper first proves new privacy bound for the subsampled Gaussian mechanism under fixed-size sampling with and without replacement, which improves over the tightest known prior results in (Wang et al. 2019) by a constant of four. The proofs rely on a careful coupling of the sampling processes on neighboring datasets (similar to the design in Wang et al. 2019) to reduce the problem of analyzing divergence between Gaussian mixtures, for which the analyses in (Mironov et al. 2019) are applicable. \n\n- For fixed-size sampling without replacement, the paper proves integral upper bounds for the remainder term of the Taylor expansion of the privacy bound, to obtain tighter constants compared to the privacy bound in Wang et al. 2019. \n- The method extends to fixed-size sampling with replacement. It allows an upper bound that is similar to the upper bound for sampling without replacement in the leading term to sampling probability q. The authors also prove a lower bound under such settings and numerically investigate its dependence on the batch size and the Renyi divergence order.\n- The authors analytically showed that the empirical gradient variance is larger under Poisson sampling compared to fixed-size sampling, but the privacy bound is smaller under Poisson sampling compared to fixed-size sampling, indicating an interesting privacy-utility trade-off depending on the choice of sampling schemes, when fixing the sampling probability the same.\n\nSTRENGTHS:\n- An interesting way of computing DP guarantee for subsampled Gaussian mechanism, via computing integral upper bounds for the remainder term of its Taylor expansion.\n- The proposed method yields a tighter RDP guarantee for subsampled Gaussian mechanism under fixed-size sampling without replacement, compared to prior results (Wang et al. 2019) by a constant factor of four.\n- The method extends to fixed-size sampling with replacement. It allows an upper bound that is similar to the upper bound for sampling without replacement in the leading term to sampling probability q. The authors also prove a lower bound under such settings and numerically investigate its dependence on the batch size and the Renyi divergence order.\n\nWEAKNESSES:\n- The reason for the improved constant factor compared to (Wang et al.) is not crystal clear. Is the integral upper bound for the remainder term of Taylor expansion of the DP bound contributing to the tighter constant? Unfortunately, the Taylor expansion is not explained in much detail in the main paper (e.g., in Theorem 3.2, the crucial terms related to $A$ and $M$ are not presented nor explained).\n- Although it is interesting that the paper shows tighter constant in leading order term of the DP bound (via analytical approach), the value of this contribution in practice needs more explanation. As the divergence between Gaussian mixtures could be tightly computed numerically following (Mironov 2019.), it is unclear why we need an tighter analytical bound given by integral upper bound for the remainder term of Taylor expansion of the DP bound.\n\nQUESTIONS:\n1. The reason for improved tightness of DP bound by a constant factor (see weakness for more details)\n2. It is interesting that the paper shows that Poisson sampling enables smaller RDP bound than fixed-size sampling when keeping the sampling ratio the same. However, I wonder if it is a phenomenon that is unique to the add-or-remove-one notion of differential privacy. Could the authors comment on whether there will be a similar gap between Poisson sampling and fixed-size sampling when considering the replace-one notion of differential privacy?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the Renyi DP guarantees of a with- or without replacement subsampled Gaussian mechanism. Authors present a privacy analysis tailored for the subsampled Gaussian mechanism, which improves the earlier general bound for $\\epsilon(\\alpha)$ by Wang et al. 2019 by a factor of four. Authors show analytically, that the subsampling induced variance (i.e. the variance of the noise arising from using minibatches instead of full data) is smaller for without replacement subsampling than for the more commonly used Poisson subsampling. Authors also give a theoretical analysis of the differences between WOR and Poisson subsampling, showing that the Poisson subsampling leads to approximately half the $\\epsilon$ of the WOR sampling. Authors demonstrate empirically that for fixed noise level, the WOR leads to better accuracy than Poisson subsampling in a CIFAR10 based deep learning task, suggesting that the difference is due to the lower subsampling noise. Finally, authors show that the WOR sampling leads to more stable memory usage.\n\nSTRENGTHS:\nThe DP-SGD algorithm is by far the most widely applied tool for DP machine learning. Since WOR sampling is more commonly used in non-DP ML than Poisson subsampling, improving the privacy bounds for WOR sampling is a interesting and important contribution. The theoretical analysis based on Taylor expansion of the Renyi divergence is a novel contribution and allows stricter bounds than the general result by Wang et al. 2019. Furthermore, the fact that WOR results into smaller subsampling noise is an interesting finding as well. \n\nThe numerical results for the accounting highlight the significant improvement over the current state-of-the-art WOR sampling privacy accounting, showing over a factor of two improvement in the $\\epsilon$ after conversion to approximate DP bounds. Also the empirical comparison on memory usage demonstrates the benefits of WOR sampling.\n\nWEAKNESSES:\nWhile the presented analysis provides important insights and improvements over the previous RDP analysis for WOR sampled Gaussian mechanism, I wonder if the problem is already solved with the modern privacy accounting tools. For example, Zhu et al. 2022 solve the WOR sampled Gaussian mechanism in their characteristic function formalism. Given that the conversion from RDP to approximate DP is lossy, I would imagine the bounds presented in this work are more loose than the bounds of Zhu et al when converted to approx DP domain. Since the approx DP bounds are more commonly used than RDP bounds, I'm not sure if the tight RDP analysis is really needed. Or is there some practical reason, e.g. an implementation difficulty, that would not allow using the characteristic function accounting for this problem?\n\nZhu, Yuqing, Jinshuo Dong, and Yu-Xiang Wang. \"Optimal accounting of differential privacy via characteristic function.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2022.\n\nQUESTIONS:\n- As you acknowledge towards the end of Section 3.5, Wang et al. 2019 use the substitute neighbourhood relation which differs from the add/remove used in this paper. Can you clarify, is this difference taken into account in Figure 3?\n- Fig. 2: I'm a bit confused on the upper bounds show in this Figure. It seems that after some $\\alpha$, your proposed upper bound exceeds the one from Wang et al. Does this suggest that the Wang et al. bound is tighter in some regime of $\\alpha$?\n-  In the Appendix D, you derive the RDP bounds for the substitute relationship using mixture of Gaussians with weight $1-q$ on Gaussian centered at $0$ and weight $q$ for the \"adjacent\" point. A recent work by Lebeda et al. 2024, suggest that this might not be the worst-case distribution for woR-sampling (see Section 7 in Lebeda et al.). While their analysis is tailored for approximate-DP and not RDP, I wonder if the same holds RDP.\n\n**Typos and other minor things**\n- Eq. 57, extra parenthesis in the expression for $M_{\\sigma, 4}$\n- I guess the NN abbreaviation is never explained. However, I don't think using it is necessary for the paper to begin with as you analysis applies to any learning task using DP-SGD.\n- Fig. 2: the caption is overlapping with the axis label. Also, I think this figure is never referred to in text.\n- \"... even without using the convexity technique; ...\": which convexity technique are you talking about?\n- \"$a_i = \\nabla_\\theta L(d_i) \\cdot v$\", what is the $v$ here? Also, since the gradient is multidimensional, are you talking about dimension-wise variance?\n- \"addtional\"\n\nLebeda, Christian Janos, et al. \"Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Composition.\" arXiv preprint arXiv:2405.20769 (2024).\n\nLIMITATIONS:\nI think authors should still address whether other accounting tools can solve this problem more efficiently that their method in the approximate DP domain. Other than that I believe the limitations are well addressed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}