TAN without a burn: Scaling laws of DP-SGD
Authors: Tom Sander
Pierre Stock
Alexandre Sablayrolles
Venue: Submitted to ICLR 2023 (2023)

Abstract: Differentially Private methods for training Deep Neural Networks (DNNs) have progressed recently, in particular with the use of massive batches and aggregated data augmentations for a large number of steps. These techniques require much more compute than their non-private counterparts, shifting the traditional privacy-accuracy trade-off to a privacy-accuracy-compute trade-off and making hyper-parameter search virtually impossible for realistic scenarios. In this work, we decouple privacy analysis and experimental behavior of noisy training to explore the trade-off with minimal computational requirements. We first use the tools of Rényi Differential Privacy (RDP) to show that the privacy budget, when not overcharged, only depends on the total amount of noise (TAN) injected throughout training. We then derive scaling laws for training models with DP-SGD to optimize hyper-parameters with more than a $100\times$ reduction in computational budget. We apply the proposed method on CIFAR-10 and ImageNet and, in particular, strongly improve the state-of-the-art on ImageNet with a $+9$ points gain in accuracy for a privacy budget $\varepsilon=8$.

Under review as a conference paper at ICLR 2023
TAN WITHOUT A BURN: SCALING LAWS OF DP-SGD
Anonymous authors Paper under double-blind review
ABSTRACT
Differentially Private methods for training Deep Neural Networks (DNNs) have progressed recently, in particular with the use of massive batches and aggregated data augmentations for large number of steps. These techniques require much more compute than their non-private counterparts, shifting the traditional privacy- accuracy trade-off' to hawaacy-accuracyccomeepp parameter search virtually impossible for realistic scenarios. In this work, we decouple privacy analysis and experimental behavior of noisy training to explore the trade-off with minimal computational requirements. We first use the tools of Rényi Differential Privacy (RDP) to show that the privacy budget, when not overcharged, only depends on the total amount of noise (TAN) injected through- out training. We then derive scaling laws for training models with DP-SGD optimize hyper-pane parameters with more than 100x reduction in computational budget. We apply the proposed method on CIFAR-10 and ImageNet and, par- ticular, strongly improve the state-of-the-arr on ImageNet with +9 points gain in accuracy for a privacy budget ε 8.
trade-off and making hyper-
INTRODUCTION
Deep neural networks (DNNs) have become fundamental tool of modern artificial intelligence, producing cutting-edge performance in many domains such as computer vision (He et al. 2016), natural language processing (Devlin et al.) 2018) or speech recognition (Amodei et al., 2016 The performance of these models generally increases with their training data size (Brown et al. 2020 Rae al. 2021 Ramesh et al. 2022), y which encourages the inclusion of more data in the model's training set. This phenomenon also introduces potential privacy risk for data that gets incorporated.
Indeed, AI models not only learn about general statistics or trends of their training data distribution (such as grammar for language models), but also remember verbatim information about individual points (e.g.. credit card numbers), which compromises their privacy (Carlini et al, 2019 2021 Access to trained model thus potentially leaks information about its training data.
The gold standard of disclosure control for individual information Differential Privacy (DP) (Dwork al. 2006). Informally, DP ensures that the training algorithm does not produce very different models if a sample is added or removed from the dataset. Motivated by applications in deep learning, DP-SGD (Abadi et al. 2016) is an adaptation of Stochastic Gradient Descent (SGD) that clips individual gradients and adds Gaussian noise to their sum. Its DP guarantees depend on the privacy parameters: the sampling rate q = B/N (where B is the batch size and N is the number of training examples), the number of gradient steps S, and the noise variance o?.
Training neural networks with DP-SGD has seen progress recently, due to several factors. The first is the use of pre-trained models, with DP finetuning on downstream tasks (Li et al. [021] De al. 2022). This circumvents the traditional problems DP, because the model learns meaningful features from public data and can adapt to downstream data with minimal information. In the remainder of this paper, we only consider models trained from scratch, as we focus on obtaining information through the DP channel. Another emerging trend among DP practitioners is to use massive batch sizes at a large number of steps to achieve a better tradeoff between privacy and utility: Anil et al. (202I) have successfully pre-trained BERT with DP-SGD using batch sizes of 2 million. This paradigm makes training models computationally intensive and paramete (HP) search effectively impractical for realistic datasets and architectures.

---

Under review as a conference paper at ICLR 2023
Batch Scaling Law on ImageNet for fixed TAN
55
Baseline Previous SOTA
Ours Simulation
*
Top 1 Test Accuracy
Low Compate suash
40-
25
X128 FLOPS
20
128
256
512
1024 2048 4096 8192 16384 32768
Batch Size
Figure 1: Training with DP-SGD on ImageNet for constant number of steps S 72k. All points are obtained at constant a/ B, with rref = 2.5 and Bret 16384. The dashed lines are computed using linear regression on the crosses, and the dots and stars illustrate the predictive power of TAN. We perform low compute hyper-parameter (HP) search at batch size 128 and extrapolate our best setup for a single run at large batch size: stars show our reproduction of the previous SOTA from De iet al. (2022) and improved performance obtained under the privacy budget = 8 with a +6 points gain in top-1 accuracy. The shaded blue areas denote 2 standard deviations over three runs.
In this context, look at DP-SGD through the lens of the Total Amount of Noise (TAN) injected during training, and use to decouple two aspects: privacy accounting and influence of noisy up- dates the training dynamies. We first show that within wide range of the privacy parameters, the privacy budget is function only of the total amount of noise. Using the tools of RDP account- ing, we approximate by closed-form expression. We then analyze the scaling laws of DNNs at constant TAN and show that performance at very large batch sizes (computationally intensive) is (linearly) predictable from performance small batch sizes illustrated in Figure
In summary, our contributions are as follows:
We define the notion of Total Amount of Noise (TAN) and show that when the budget is not overcharged, it only depends on TAN;
We derive scaling laws and showcase the predictive power of TAN to reduce the computa- tional cost of typer-parameter tuning with DP-SGD, saving factor of 128 in compute on ImageNet experiments (Figure [ We then use TAN to find optimal privacy parameters, leading gain of +9 points under ε 8 compared the previous SOTA;
We leverage TAN to quantify the impact of the dataset size on the privacy/utility trade-off and demonstrate that doubling dataset size halves E while providing better performance.
BACKGROUND AND RELATED WORK
In this section, we review traditional definitions of DP and RDP. We consider randomized mecha- nism M that takes as input a dataset D and outputs a machine learning model 0 ~ M(D). Definition (Differential Privacy). A randomized mechanism M satisfies (e, 8)-DP (Dwork al.
2006 if. for any pair of datasets D and D' that differ by one sample and all subset R C Im MM),
P(M(D) E R) S PPM(DD) E R) exp(s) +
(1)
DP-SGD (Abadi et al. 2016)> is the most popular DP algorithm to train DNNs. selects samples uniformly at random with probability q B/N (Poisson sampling), clips per-sample gradients to norm C (clipc), aggregates them and adds (Gaussian) noise. With the parameters of the DNN:
1 Σ
clipa (Vol(0, (I,, v())) +N (0, C²02
Jnwisy B
(2)
B2
iEB
2

---

Under review as a conference paper at ICLR 2023
The traditional privacy analysis of DP-SGD is obtained through Rényi DP. Definition 2 (Rényi Divergence). For two probability distributions P and Q defined over R, the Rényi divergence of order of given Q
P(x) Q(x)
Da Q)) = a a-1 log 1
Definition (Rényi DP), randomized mechanism M: D R. satisfies (a, dRe-Ré differential privacy (RDP) if. for any D, D E DD' ththat differ by one sample, we have
Da(M(D) MM da.
Rényi DP convenient notion track privacy because composition additive: a sequence of two algorithms satisfying (a, da) and (o, d',) RDP satisfies (a, da + d',) RDP. In particular, the succession of S steps of (a, da) RDP mechanism satisfies Sd,) RDP. Mironov al. (2019)
show that each step of DP-SGD satisfies (a, Ja (a, R)D- with
go (a, 9) (11 g), o²) (N((10²) (0)
Finally, a mechanism satisfying (a, da J-déni--DDPP also satisfies (E (Mironov 2017) for ē = da + log( 1/6) Training for S steps with DP-SGD thus satisfies (ERDP, DD with
RDD min (g, log(1/8)
RDP is the traditional tool used to analyse DP-SGD, but other accounting tools have been proposed obtain tighter bounds (Gopi et al., 202T). In this work, we use the accountant due to Balle et al. (2020), whose output is referred to as which is slightly smaller than ERDP.
Training from Scratch with DP-SGD. Training ML models with DP-SGD typically incurs loss of model utility, and recent work have shown that using very large batch sizes improves the privacy/utility trade-off (Anil al. 2021 Li et al. 202T). [e et al. (2022) recently introduced Aug- mentation Multiplicity (AugMult), which averages the gradients from different augmented versions of every sample before clipping and leads to improved performance on CIFAR- 10. Computing per- sample gradients with mega batch sizes for large number of steps and AugMult makes DP-SGD much more computationally intensive than non-private training, typically dozens of times. For in- stance, reproducing the previous SOTA ImageNet of De et al. (2022) under 8 necessitates 4-day run using 32 A100 GPUs, while the non-private SOTA can be reproduced in a few hours with low-rank reparametrizationn of the weight matrices and diminish the computational cost of accessing per-sample gradients.
the same hardware (Goyal et al. 2017). Yu et 202 lb) propose
Finetuning with DP-SGD. Tramer & Boneh (2020) shows that handcrafted features are very com- petitive when training from scratch, but fine-tuning deep models outperforms them. Li et al. (202)) Yu et al. (2021a) fine-tune language models to competitive accuracy on several NL.P tasks. (002)) consider models pre-trained on JFT-300M and transferred to downstream tasks.
3 THE TAN APPROACH
We introduce the notion of Total Amount of Noise (TAN) and discuss its connections to RDP ac- counting. We then demonstrate how training with reference privacy parameters (gref, Fref, S) can be simulated with much lower computational resources using the same TAN with smaller batch size.
Definition 4. Let the individual signal- seanal-to-noiee to-noise ratio (and its inverse Σ, the Total Amount of Noise or TAN) be as follows:
1 = 92s
172
Σ2 20
3.1 MOTIVATION
We begin with a simple case to motivate our definition TAN. We assume a one-dimensional model, where the gradients of all points are clipped to C. Looking at Equation 2 in batch, the
3

---

Under review as a conference paper at ICLR 2023
50
8
40
40
S-1k
30
Sw2.5k
30
5=10k
20
5=70k
20
S=100k
5-200k
S=500k
10
0
0.2
0.2
Figure 2: Privacy budget function of the noise level with 77 constant. On both figures, each curve corresponds to a different number of steps S, and each point on the curve is computed at a sampling rate y such that constant. On the left, we use 7/ 0.13 (resulting in êTAN 1 in Equation 4 On the right, we use y 0.95 (êTAN = 8). We observe a "privacy wall" imposing 0.5 for meaningful level of privacy budget E, and
for constant ETAN-
expected signal from each sample is C/ B with probabillty q - B/N and otherwise. Therefore, the expected individual signal of each sample after S steps is SC/N, and its squared norm is C²C2/N² The noise at each step being drawn independently, the variance across 5 steps adds to SC²0²/1²² The ratio between the signal and noise is thus equal to (up a factor 1/22
S²C²2
q²s
N
= 2
2SC B
20
Denoting Tatep := q/V2o, we have n)² = Sn The ratio 0/9 is noted by Li et al. (022) as the effective noise. The authors found that for fixed budget and fixed S, the effective noise decreases with B. Our analysis goes further by analyzing how RDP accounting explains this dependency.
3.2 CONNECTION WITH PRIVACY ACCOUNTING
Intuitively, we expect that the privacy budget E only depends on the signal-to-noise ratio 7)- In Figure 2 plot
function of and S, at a fixed 1), and observe that is indeed constant, but only when σ > 2. On the contrary, when a gets smaller, & increases exponentially, creating a "Privacy Wall". We shed light on this phenomenon by looking the underlying RDP values.
We observe (see Figure 3 that when a > 2, go (o .) is close to ax²2/ /(0²²) angtep. Replacing that in the definition of ERDP (Equation Bp. we get
ERDP ~ 17² + min (a- 1)n² + log (L/0) -1²² + P2 -
2222/oog log (1/5) =: ETAN(1).
(4)
10
13
sola, g) anjap
12
10a
10
1.-
10a
30
20
α
0
u-0 0- 3.00
o-169
3.66
o-
Figure 3: Approximation of 9a (a, q) All curves correspond to distinct couples (g. a)) such that 1step 3.9 10-3 (used for ImageNet). The right plot corresponds an enlargement of the left plot: the ratio is very close to 1 for o > 2. The phase transitions on the left plot were also observed by Wang et al. (2018) and Abadi et al. (2016).
4

---

Under review as a conference paper at ICLR 2023
TAN: CIFAR-10 for B, 4096 and 2500
TAN: ImageNet for 8: 16384, Oer - 2.5
90
5-
S0k
Top Test Accuracy
85
Top Test Accuracy
40
Presious SOTA decm cm
Sw30k
80-
25
75
70
65
Oer 0.5
60-
durm 3
128
2048
4096
8192
16384
128
256
512
1024
2048
4096 8192 16384 32768
Batch Size
Batch Size
Figure 4: Test accuracies constant 7stp Bres/ (oes:/(2NN,,re and are (log) linearly decreasing with B. Dashed lines are computed using linear regression on the crosses. Shaded areas corre- spond std confidence intervals. (Left) CIFAR-I0 with Ne- for Sref steps. Each curve corresponds to a different value of Mstep- (Right) ImageNet with NF-ResNee-50 with various numbers of steps. The scaling law holds for various training configurations.
We verify this relationship empirically, and in particular choose 1) get a desired STAN in Figure Conversely, we observe that when a < 2, 9a becomes much larger than omatp which explains the blow-up in e from Figure Having this simple approximation for useful in practice because it allows for simple mental gymnastics: for instance, doubling the sampling rate q while dividing the number of steps S by should leave the privacy budget constant, which observe empirically.
3.3 SCALING AT CONSTANT TAN
Our RDP analysis in Section 3.2 suggests simple scaling strategy. Starting from (g, o, S). while < 2, we can double both q and a. This drastically improves privacy accounting (Figure 2 and the per step signal-to-noise ratio Maup remains constant. However, given that the number of steps S is fixed, each doubling of doubles the computational cost.
Batch Scaling Laws. We now analyse how this strategy affects the performance of the network.
In Figure 4 we perform this analysis on CIFAR-10 and ImageNet. We find that for triplets (q, o, for which q/a qvet/Oret (keeping Tatep constant), the performance of the network (log) linearly decreases with the batch size. This consistent with the (non moo--rivvate) work of Smith et al. (2020). which shows that for fixed number of steps. small batch sizes perform better in generalisation (i.e., test accuracy) than large batch sizes. The difference with our work that we consider noisy updates and observe (log) linear relationship between batch size and performance.
Choice of a. a < 2, simultaneously doubling q and a has small negligible negative impact on accuracy (Figure but it can greatly reduce the privacy budget (Figure 2h. Reciprocally, halving and 9 is slightly beneficial privacy guarantees almost unchanged (Figure 2 It also divides the computational cost by 2. This explains why state-of-the thrtt--atttttttt approaches heuristically find that mega-batches work well: blind grid search on the batch size and the noise level at constant privacy budget is likely to discover batches 2. Our analysis gives principled explanation for the sweet spot of E [2, 4] used by most state-of-the-art approaches (De et al. 2022
neutral to the performance (Figure 4 and if > 4, it keeps the
large enough to have
Li ei al. 2021).
Efficient TAN training. We go one step further and study training in the small batch size setting. We choose the optimal hyper-parameters (including architecture, optimizer, type of data augmenta- tion) in this simulated setting, and finally launch one single run at the reference (large) batch size, with desired privacy guarantees. On ImageNet, we target Bret 16.384 with B 128. We thus reduce the computational requirements by factor of x28x. Each hyper-paramete search in the ImageNet setup takes days using 32 A- 1-100 GPUs; we reduce less than a day on single GPU.
5

---

Under review as a conference paper at ICLR 2023
Table 1: Top-1 test accuracy when training ImageNet from scratch using DP-SGD. We train NF-ResNet-50 with = 2.5, hyper-parameters of Table 2 and (B, S) = (32768, 18k) (Table
Method
(e, 6)
Accuracy
Kurakin et al. (2022)
(13.2, 10 10-6)
6.2%
De et al.
2022
(original paper)
(8,8.10
32.4%
De et al.
2027
(our reproduction)
(8, 88,8.1077)
30.2%
Ours
(8,8.107))
39.2%
4 EXPERIMENTS
We leverage our efficient TAN training strategy and obtain new state-of-the-art results on ImageNet for (Table I We then study the impact of the dataset size on the pricacy/utility trade-off. We also demonstrate how our low compute simulation framework can be used to detect performance bottlenecks when training with noisy updates: our case, the importance of the order between activation and normalization in a WideResNer on CIFAR-10.
4.1 EXPERIMENTAL SETUP
We use the CIFAR-10 dataset Krizhevsky et al. 2009) which contains 50K 32 x 32 images grouped in 10 classes. The ImageNet dataset (Deng et al. 22009; Russakovsky al. 2014) contains 1.2 million images partitioned into 1000 categories. When using data augmentation, we always use Augmentation Multiplicity as detailed in Appendix C For both datasets, we train models from random initialization. On CIFAR- 10, we train 16-4 WideResNets ZZagorukko & Komodakis 2016). On Imagenet, we compare Vision Transformers (ViTs) (Dosovitskiy et al.) (2020), Residual Neural Networks (ResNets) (He et al., (016) and Normalizer-Free ResNets (NF-ResNets) (Brock et al. (022Tb). We fix 8 = 1/N where is the number of samples for each experiment and report the corresponding value of E. We use == 1 for the clipping factor in Equation 2 as we did not see any improvement using other values. We use the Opacus (Yousefpour et al., 2021) library in Pytorch (Paszke et al. 2019). We open-source the training code.
4.2 IMAGENET
In Section 4.2.1 we use our simulated training with constant TAN to find optimal hyperparame- ters low compute and improve performance for reference privacy parameters (Bref, Fref 5) (16384, 2.5, 72k). In Section we find optimal privacy parameters at constant TAN by chang- ing the number of steps S, leading to new state of the art on ImageNet for ε 8 (Table T).
4.2.1 HYPER- PARAMEER TUNING AT CONSTANT TAN
Hyper-parameter search. We a large pypeer-parmmeter search and report the best hyper- parameters in Table as well as the corresponding improvement for various batch sizes (at constant
Keeping Nstep and S constant, compare various changes in the training pipeline. We compare with the baseline De et al. (2022) (blue line in Fig- [ NeResNet-50, learning rate at 4, EMA decay at 0.99999, 4 random augmentations averaged over 3 runs). Each gain is compared to the previous column.
Table 2: Comparing optimal
Imagenet: ref 2.5, Byet 16,384, S 72K
B
(Ir, p. d)
decay
AugMult
AugTest
Total
128
(8,0,0)
+1.0
0.999
+1.2
(Ours, 8)
+3.0
+0.4
+5.6%
256
(8,0,0)
+0.8
0.999
+1.2
(Ours, 8)
+3.0
+0.7
+5.7%
512
(8,0,0)
+1.2
0.999
+1.1
(Ours, 8)
+2.8
+1.1
+6,2%
1024
(8,0,0)
+1.6
0.999
+1.2
(Ours, 8)
+2.3
+0.8
+5.9 %
16384
+0.8
+6.7%

---

Under review as a conference paper at ICLR 2023
Table Low compute simulation of privacy parameter search. We start from B 256 16384/64 and 5 = 72K. We use 2.5/64 4 for all runs and data augmentation.
Beef 256, Seet
72K
S
B
lr
Gain
9K
756
64
-6.22%
18K
512
32
+1.32%
72K
256
8
/
288K
128
2
-1.88%
Mstep and S). Each gain is compared the optimal hyper-parameters find at the previous column.
We search over learning rates E [1, 2, 4, 8, 12, 16). momentum parameters E [0, 0.1, 0.5, 0.9, I] and dampening factors d E [0, 0.1, 0.5, 0.9, 1]. We use exponential moving average (EMA) on the weights Tan & Le 2019) with a decay parameter in [0.9, 0.99, 0.999, 0.9999, 0.99999).
We try different types of data augmentation, that we referred to "RRC", "Ours" and "SimCLR", and try for each various multiplicity of augmentations (1, 2, 4, 8, 16) (see Appendix C for details).
RRC: a standard random resized crop (crop chosen at random with an area between 8% and 100% of the original image and random aspect ratio in [3/4, 4/3)),
Ours: random crop around the center with 20 pixels padding with reflect, random horizontal flip and color jitter;
SimCLR: the augmentation from Chen et al. (2020). including color jitter, grayscale, gaus- sian blur and random resized crop, horizontal flip.
We find (Table 2) that optimal parameters the same in each scenario of simulation, predicted in Section 3.3 We perform one run with these optimal parameters at B = 16384 which satisfies a multiple batch sizes only support our hypothesis and
privacy budget of 8. Note that batch scaling law, but it is sufficient to simulate only at B = 128. Our experiments indicate that AugMult is the most beneficial when the corresponding image augmentations are rather mild.
Testing with augmentations. We also test the model using majority vote the augmentations of each test image (AugTest column in Table We use the same type and number of augmentations as in training. It improves the final top-1 test accuracy. This is in line with recent line of work aiming at reconciling train and test modalities (Touvron et al. 2019). To provide fair comparison with the state of the art, decide not to include this gain in the final report in Table D and Table
Choice of architecture and optimizer. We have experimented with different architectures (ViTs, ResNets, NFResnets) and optimizers (DP-Adam, DP-AdamW, DP-SGD) (see Appendix [ for de- tails). Our best results are obtained using NFResnet-500 and DP-SGD with constant learning rate, which differs from standard practice in non- non-private training.
4.2.2 PRIVACY PARAMETER SEARCH AT CONSTANT TAN
While we kept S constant in previous experiments, we now explore constant TAN triplets (g, a, S) by varying S. We keep fixed to 2.5 and vary (B, starting from the reference (16384, 72K) at
Table Privacy parameter search. We use the optimal parameters described in Section 4.2.1 with 2.5 for one expensive run and compare it with our optimal result
Brof = 16384, S = 72K
S
B
g
br
Test acc
18K
32,768
8.00
32
39.2%
72K
16.384
7.97
8
36.1% 1%
7

---

Under review as a conference paper at ICLR 2023
Pre Activation VS Post Activation Noralization
F
Final Top 1 Test Accuracy
0.75
x
0.65
0.55
ReLU-GN, (simulation)
ReLU-GN, B=4096
GN-ReLU, (simulation)
GN-ReLU, B=4096
0.50
2
8
Augmentation Multiplicity Order
Figure 5: Impact of data augmentation on the test accuracy using pre-activation normalization vs CIFAR-10. We compare simulation (B, (512, g)
post-activation in 16-4- RideReeet and reference (Bref, Teer) ) = (4096, 3), both trained for S = 2,500 steps. Confidence intervals are 5 runs. Augmentation Multiplicity Order corresponds to
plotted with standard deviations the number of augmentations per image, or K in Appendix
constant 7) (2 Given that >> 2, stay in the almost constant privacy regime (see Figure 2): we indeed observe E se ÉTAN in Table 4 We scale the learning rate inversely to S to compensate for the decrease of the noisy updates' magnitude (Equation 2 Since performing this privacy parameter search is computationally intensive, we first simulate training using our scaling law at B 256 (with the same Msttp) and display our results in Table Our best results are obtained for 18k steps. Finally, we perform one computationally expensive run at S = 18k and B = 32768, with other syper-paraamters from Section 4.2.I and show the results Table 4
We note an improvement over our previous best performance at (B, a, S) (16384, 2.5, 72K) referred in Table 2 Overall, we improved performance by 9% when training from scratch on Im- ageNet with DP-SGD under = 8. We compare to our reproduction of the previous SOTA of De ctt al. (0222 at 30.2% (compared to the results reported the original paper (32.4%), we still gain 7% of accuracy). Thus, have shown how
TAN perform optimal privacy parameter search while simulating each choice of optimal parameters at a much smaller cost.
4.3 ABLATION
We now illustrate the benefit of TAN for ablation analysis. We study the importance of the order between activation and the normalization layers when training with DP-SGD, We also discuss how gathering more training data helps improve performance while decreasing the privacy budget. both experiments, we train Wideeeeees CIFAR-10, constant learning rate and studying (Bref, Teef S) = (4096,3 3, 2.5k) (reference private training).
Pre-activation Post-activation Normalization Normalization techniques such as BatchNorm (loffe & Szegedy 2015). GroupNorm (GN) (Wu & He 2018) or LayerNorm (Ba et al. 2016) help training DNNs. Note that BatchNorm is not compatible with DP-SGD because it is not amenable to per-sample gradient computations, we thus resort to GroupNorm. These normalization layers are usually placed between convolutional layers and activations (e.g., CONV-GN-ReI.U BBoock et al. 202 suggest that signal propagation improves when the order is reversed (to CONV-ReLU-GN),
We experiment with DP-SGD training using both orders of layers, and display our results in Figure We make two observations. First, the reverse order leads to significantly greater performance, and is more robust. Second, the standard order does not benefit from data augmentation. We observe that the two simulated experiments with B = 512 represented by lighter colors in Figure [5 standard deviations around the means) have the same properties. However, each simulation is times less
8

---

Under review as a conference paper at ICLR 2023
Table 5:
Impact of the training set size N on the privacy/utility trade-off. We start training on 10% of the data (No 5K). We use B 4,096, 3 and S 2,500, with post-activation normalization, and no augmentation. Standard deviations are computed over runs.
CIFAR-10-
B 4,096, S 2,500
N
Test acc (%)
5K
150.3
59.9 (+1)
25K
13.7
71.1 (#0.4)
40K
7.3
(-.1)
50K
7.1
74.0 (+0.5)
computationally expensive. Therefore, using TAN through our scaling law can facilitate studying variants of the network architecture while reducing the computational costs.
Quantity of Data and Privacy Budget We now look how collecting more data affects the tradeoff between privacy and utility. We show that doubling the data (from the same distribution) allows bettrr performance with half the privacy budget. To this end, train portions of the CIFAR-10 10 training set (N = 50k) and always report accuracies on the same test set. If we multiply by 3 the quantity of data No and keep the same (B, a, S), q (and thus 17), is divided by well. We divide ó by 3 for the accounting. We show in Table 5 the effects on and model accuracy.
On the hand, when using éTAN. we can predict the impact on the privacy budget. On the other hand, since the global signal-to-noise ratio Nn is held constant in all experiments, we expect to extract the same amount of information in each setup; adding more data makes this information richer, which explains the gain in accuracy. We show similar results for ImageNet in Appendix
CONCLUSIONS
We argue that the total amount of noise (TAN) is a simple but useful guiding principle to experiment with private training. particular, demonstrate that the privacy budget either a direct function of TAN or can be reduced. We further show that scaling batch size with noise level using TAN allows for ultra-efficient hyper-parameds search and demonstrate the power of this paradigm by establishing new state of the art for DP training on Imagenet.
5.1 LIMITATIONS
Non-private hyper-parameter search. We follow the standard practice of counting hyper- parameter search towards the privacy budget (Li et al. 2021 Anil et al. 2021). Theoretically, each training run should be charged on the overall budget, but practice is commonly assumed that the "bandwidth" of hhper-ypane parameters is too small to incur any observable loss of privacy (see also Liu Talwar (209)) for a theoretically sound way of handling this problem). available, one can use similar public dataset (such as ImageNet) to choose syper-parameetes, and then perform only limited runs on the private dataset. Finally, we note that training non-pon non-private models might not possible on sensitive data. In this case, our hyper-parameter transfer process can not be used.
5.2 FURTHER WORK
Better accounting. We believe that the important increase in the privacy budget e as the noise level decreases is a real phenomenon and not an artifact of the analysis. Indeed, DP assumes that the adversary has access to all model updates, as the case for example in Federated Learning. In such cases, noise level that is too low is insufficient to hide the presence of individual data points and makes it impossible to obtain reasonable privacy guarantees. In the centralized case however, the adversary does not intermediate models but only the final result of training. Some works have successfully taken into account this "privacy amplification by iteration" idea (Feldman et al. 2018 Ye & Shokri 2022) but results far limited convex models.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. Deep speech 2: End-to- end speech recognition in english and mandarin. In International conference on machine learning.
pp. 173-182. PMLR, 2016.
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi. Large-scale differen- tially private bert, 2021. URL httl/ttts///axiv.ooto org/ abs/2108. 01624
Jimmy Lei Ba. Jamie Ryan Kiros, and Geoffiey E. Hinton. Layer normalization, 2016. URL attps://arxiis wttpa//.w.../////1/...0000
Borja Balle, Gilles Barthe, Marco Gaboardi, Justin Hsu, and Tetsuya Sato. Hypothesis testing in- terpretations and Rényi differential privacy. In International Conference on Artificial Intelligence and Statistics, pp. 2496 2506. PMLR, 2020.
Andrew Brock, Soham De, and Samuel L. Smith. Characterizing signal propagation close the performance gap in unnormalized resnets, 2021a.
08692
Andrew Brock, Soham De, Samuel L.. Smith, and Karen Simonyan. High-performance large---scaaaaeeraaaaaaaaee image recognition without normalization, 2021b. URL htttpe://arriiv.org//a/////
06171
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu. Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL nttts://aaxiis o abs 2005. 14165
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium, pp. 267-284, 2019.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp- 2633-2650, 2021.
Ting Chen, Simon Korblithh Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. CORR, abs/2002.05709, 2020. URL https:
/20022222222 05709
Soham De, Leonard Berrada, Jamie Hayes, Samuel L. Smith, and Borja Balle. Unlocking
high-accuracy differentially private image classification through scale, 2022. URL https:
//a www.ai..wg.aaa//a.aaaaaaaa.aaaaa
13650
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 245,2555,200 2009. doi: 10. 10111099R...209550000 5500844.
Jacob Devlin, Ming- Ming Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint vrXiv:1810.04800 2018.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2020. URL https: rxiv www.verrg/aaee abs /20 11929
10

---

Under review as a conference paper at ICLR 2023
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Proceedings of the Third Conference on Theory of Cryptography, pp- 265-284, 2006.
Vitaly Feldman, Ilya Mironov, Kunal Talwar, and Abhradeep Thakurta. Privacy amplification by iteration. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pp. 521-532. IEEE, 2018.
Sivakanth Gopi, Yin Tat Lee, and Lukas Wutschitz. Numerical composition of differential privacy. Advances in Neural Information Processing Systems, 34:1 1631-1164444 2021.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, An- drew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in hour. arXiv preprint arXiv:1706.02677 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRR, abs/1502.03167, 2015. httptttp/p/taal/ org/
abs/1502.03161
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, Shuang Song, Steve Chien, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward training at imagenet scale with differential privacy. 2022. URL https:
//arxiv.orq/aant
Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners, 2021.
Jingeheng Liu and Kunal Talwar. Private selection from private candidates. In Proceedings of the 51 Annual ACM SIGACT Symposium on Theory of Computing (STOC), pp. (98-309, 2019.
Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Sym- posium (CSF), pp. 263-275. IEEE, 2017.
Ilya Mironov, Kunal Talwar, and Li Zhang. Rényi differential privacy of the Sampled Gaussian Mechanism, arXiv preprint arXiv: /908. 10530, 2019.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, Amelia Glaese, Johannes Welbl, Sumanth Dathathri, Saffron Huang, Jonathan Uesato, John Mellor, Irina Higgins, Antonia Creswell, Nat McAleese, Amy Wu, Erich Elsen, Siddhant Jayakumar, Elena Buchatskaya, David Budden, Esme Sutherland, Karen Simonyan, Michela Paganini, Laurent Sifre, Lena Martens. Xiang Lorraine Li, Adhiguna Kun- coro, Aida Nematzadeh, Elena Gribovskaya, Domenic Donato, Angeliki Lazaridou, Arthur Men- sch, Baantttp Lespiau, Maria Tsimpoukelli, Nikolai Grigorev, Doug Fritz, Thibault Sottiaux, Mantas Pajarskas, Toby Pohlen, Zhitao Gong, Daniel Toyama, Cyprien de Masson d' Autume, Yu- jia Li, Tayfun Terzi, Vladimir Mikulik, Igor Babuschkin, Aidan Clark, Diego de Las Casas, Au- relia Guy, Chris Jones, James Bradburr, Matthew Johnson, Blake Hechtman, Laura Weidinger, lason Gabriel, William Isaac, Ed Lockhart, Simon Osindero, Laura Rimell, Chris Dyer, Oriol Vinyals, Kareem Ayoub, Jeff Stanway, Lorrayne Bennett, Demis Hassabis, Koray Kavukcuoglu, and Geoffrey Irving. Scaling language models: Methods, analysis & insights from training go- pher, 2021. URL.tttpe /ttp/aaavvv////////aa///aaeeaaaa wwwaaxiw..rrggbb///11111/ 11446

---

Under review as a conference paper at ICLR 2023
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022. URL attps://arxiis org/ abs/
06125
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge, 2014. URL https://arxiv.oK
abs / 1tt./1499.0 0575
Samuel L. Smith, Erich Elsen, and Soham De. On the generalization benefit of noise in stochastic gradient descent. CoRR, abs/2006.15081, 2020.
15081
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International conference on machine learning, pp. 6105-6114. PMLR, 2019.
Hugo Touvron, Andrea Vedaldi, Matthijs Douze, and Hervé Jégou. Fixing the train-test resolution discrepancy. volume 32, 2019.
Hugo Touvron, Marthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. CoRR, abs/2012.12877,, 2020. URL http://///aax.vvoorr/aaa//212212 201b/2211..1877
wwg. abaa
Florian Tramer and Dan Boneh. Differentially private learning needs better features (or much more data). 2020.
Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan. Subsampled rényi differen- tial privacy and analytical moments accountant. CoRR, 10s.00088,0877, 2018. URL http:
7/arxiv. wwtpxxiw.rrggas///ooe/ aa/////a.//./....// 00087
Yuxin Wu and Kaiming He. Group normalization, 2018. URL http::://rxiv.org///aa
18494
Jiayuan Ye and Reza Shokri. Differentially private learning needs hidden state (or much faster convergence). arXiv preprint arXiv:2203.05363. 2022.
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj. Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential privacy library in PyTorch, 2021.
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janard- han Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. Differentially private fine-tuning of language models, 2021a.
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie- Yan Liu. Large scale private learning via low-rank reparametriiztion, 2021b. URL http:///wwwwwwo..c.aa//////./../
Sergey Zagonuyko and Nikos Komodakis. Wide residual networks. CoRR, absp///0.00/144 2016. URL httt:://axxxii org/ hbch//bs//eee 07146
12

---

Under review as a conference paper at ICLR 2023
A MORE DATA: IMAGENET
We show in Table 6 that similarly to the experiments CIFAR-10, doubling the training data on ImageNet improves the accuracy while diving by 2. We also demonstrate that our scaling strategy can accurately detect the gain of accuracy. We compare training on half of the ImageNet training set (N 600k) and the entire training set (NV 1.2M).
B CHOICE OF ARCHITECTURE AND OPTIMIZER
In this section, we give more details about our choice of architecture and optimizer on ImageNet. In particular, we noticed that DP-SGD without momentum is always optimal, even with ViTs, and that NF-ResNets-50 performed the best.
Architecture. When training with DP-SGD, the goal is to find the best possible local minimum within constrained number of steps S, and with noisy gradients. However, architectures and op- timizers have been developed to ultimately achieve the best possible final accuracy with normal updates. To illustrate this extremely, we train a Vision Transformer (ViT) (Dosovitskiy et al.! 2020) from scratch on ImageNet using DP-SGD, Touvron et al. (2020) have succeeded in achieving SOTA performance in the non-private setting, but with a number of training steps higher than convolution- based architectures. A common explanation is that ViT' have less inductive bias than CNNs: they have to learn them first, and that can be even harder with noisy gradients. And if they are successful, they have lost the budget for gradient steps to learn general properties of images.
We used our scaling strategy (keeping T[step and S constant) to simulate the DP training with differ- ent architectures low compute, studying noisy training without the burden of DP accounting. The best simulated results were obtained with a NFResNet-SU (Brock et al., 2021b) designed to be fast learners in terms of number of FLOPS. The worst results were obtained with ViTs, and intermediate results with classical ResNets. In Figure 6 we compare different training trajectories a ViT and Ne-ResNet.
Optimizer Using our simulation scheme, we found that DP-SGD with no momentum and a con- stant learning rate is the best choice for all architectures. We also tried DP-Adam, DP-Adam W with wide range of parameters. It is surprising find that this is the case for ViTs, without noisy, the Adam type optimizers perform better (Touvron et al. 2020). This highlights the fact that training with DP-SGD is different paradigm that requires its tools.
Using TAN allowed us to explore and compare different architectures and optimizers, which would have been computationally impossible in the normal DP training setting at B 16384.
C AUGMENTATION MULTIPLICITY
Augmentation Multiplicity (AugMult) was introduced by De et al. (2022) in the context of DP- SGD. The authors average the gradients of different augmentations of the same image before clip- ping the per-sample gradients, using the following formula (where Ç is standard Gaussian variable):
Table 6: Impact adding more data on ImageNet. The "Simulated Gain" column corresponds to the accuracy gain observe when simulating lower compute using scaling strategy for B = 256. The "Gain" column corresponds to the real gain at B 16384.
Imagenet: Oref 2.5, Bref 16384, 5 72k
E
STAN
Gain
Simulated Gain
0.6M
16.10
17.98
18.06
1.2M
8.10-7
8.00
8.26
+1.3%
+1.5%
13

---

Under review as a conference paper at ICLR 2023
NF-ResNet VS ViT on Imagenet with DP-SGD
VIT, AugMult=0
VIT, AugMult=16
0.5
VIT, 4
NF-ResNet-50, AugMult=0
Top 1 Test Accuracy
0.4
0.3
0.2
0.1
0.0
20
40
60
80
100
Percentage of Training
Figure 6: Training ViT from scratch on ImageNet with DP-SGD. We simulate training with our scaling strategy and 256. We observe that the accuracies not as good for NF-ResNets, and that Augmentation Multiplicity plays more important role.
w1+1 2+
17.
clipe
1 K
Σ
V (slool))
+N 0,
o² B
(5)
iEB:
jEK.
Compute scales linearly with the AugMult order K. Our intuition the benefits of AugMult is that difficult examples (or examples that fall out of the distribution) become casier when using this learned to
augmentation technique. On the other hand, without AugMult, simple examples be classified early in training, resulting in a gradient close to when used without augmentation. Because we are training for a fixed number of steps, budget). With AugMult, the network may still be able to learn from these examples. Figure 7 shows the histograms of the norms of the average over augmentations for each image of the per-sample gradients, before clipping and adding noise in equation 5 at different times of training.
waste of gradient steps (i.e. privacy
14

---

Under review as a conference paper at ICLR 2023
Histogram of the morm of gredients during training Augmult=O
Histogram of the merm of gradients during treining Augmult-BSIMCLL
0.32
42%
0.22
42%
57%
47%
0.00
71%
0.0
71%
85%
0.8
0.00
3.06
0.32
8.0
0.00
sam pla
par wampllll gradients
Figure 7: Histograms of the norms of the average across all augmentations for each image of the per-sample gradients, before clipping and adding noise. On the left, we see that without augmenta- tion, inereasing number of examples have their gradients going to zero during training. On the right, we see that when using a strong augmentation technique (SimCLR, (Chen et al. 2020)), the gradients more concentrated during all the training.
15


Reviews:
[{"review_id": "znFh_byyNT6", "reviewer": "Reviewer_FyxV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed to use the total amount of noise (TAN) to determine the privacy budget in Renyi DP. They observe scaling laws with TAN for DP-SGD which can be used to reduce the computational cost for hyper-parameter tuning. By the hyper-parameter tuning, the authors greatly improve the state-of-the-art on neural network training under privacy guarantees.", "summary_of_the_review": "This paper demonstrates the scaling law between the batch size and the test accuracy in private deep learning. This law and the new concept, TAN, are very useful in improving private learning results by DP-SGD when the computation resources are limited. This law is found in the results of ImageNet and CIFAR-10 experiments, but the reason behind it is still unknown and it may be difficult to reproduce the results since the code is not provided. ", "strength_and_weaknesses": "Major strengths:\n1. The observation of the scaling law is new. \n2. The results of hyperparameter tuning improve the state-of-the-art greatly.\n\nMinor strengths:\n1. This paper is easy to follow.\n2. The visualization of the experimental results is good.\n3. Figure 2 seems to deliver the same message as the linear part of Figure 3 (left) in (Li et al, 2021a), but the authors provide more explanations in Section 3.2.\n\nMajor weaknesses:\n1. The privacy guarantee parameter $\\varepsilon_{TAN}$ is an approximation of $\\varepsilon_{RDP}$ and it is only a lower bound. For strict privacy guarantees, we need an upper bound.\n2. There is no explanation for the reason for the scaling law. \n\nMinor weaknesses:\nSome of the statements in this paper are inconsistent with each other. \n1. In the left figure of Figure 4, the legend shows that $\\sigma_{ref}$ is constant for each line, but the caption says the $\\eta_{step}$ is constant. I also wonder why the authors choose to fix different values in Figure 4 for CIFAR-10 and ImageNet. Is it because the results of CIFAR-10 do not look as linear as of ImageNet?\n2. In Page 5, Choice of $\\sigma$, the authors claimed that if $\\sigma>4$, dividing it by 2 with halving q is likely to improve performance, but I cannot find experimental results supporting this.\n3. In Page 7, the authors say that they decide not to use the gain from 'testing with augmentations' in Table 1 and Table 4. In Table 2, I guess the Total column is the sum of the other columns for B=128, ..., 1024. Therefore, the last row, B=16384, also has the AugTest (+0.8) in the Total result (6.7%). I guess in Table 4, the test ACC for B=16384, 36.9%, is derived by adding the total improvement 6.7% to the baseline 30.2%. Therefore, I am not sure if the authors forget to remove the improvement from AugTest.\n4. In Page 9, in the last paragraph, the authors believe that the exponential increase in the privacy budget $\\varepsilon$ as the noise level $\\sigma$ decreases. I guess this is from Figure 2, but the x-axis for $\\sigma$ is log-scale. Therefore, it is not easy to identify whether there is a linear or exponential relationship between $\\varepsilon$ and $\\sigma$.\n\nBy the way, there are duplicate references (Li et al, 2021a and 2021b).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "O6bNfUSGkMc", "reviewer": "Reviewer_MS1Z", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Recent work shows that large batch size can give better privacy-utility trade-off in differentia private ML training. However, training with large batch sizes is computationally expensive. The paper observes (empirically) that the privacy budget of differential private ML training depends mostly on the total amount of noise added during training when the noise multiplier \\sigma>=2. At the same time, the accuracy of the final results can be predicted by the results on a corresponding hyper-parameter setting in which the batch size is small. Following these observations, the paper proposes to do hyper-parameter tuning for small batch sizes (which is computationally efficient) and then transform the hyper-parameters to the target large batch size. Experiments demonstrate that this approach works well in practice.", "summary_of_the_review": "\nOverall, the paper is well-written and the observations and the proposed approach can potentially be very useful for the community. It would be even better if the authors can demonstrate/verify the observation on more network architectures, datasets, and models (e.g., generative models).", "strength_and_weaknesses": "\nStrength:\n\n* The observations (batch scaling law, privacy budget computation) in the paper are new and interesting.\n\n* The proposed hyper-parameter tuning process can be very useful in practice, as it saves the computation requirement and therefore speeds up the development process.\n\nWeaknesses:\n\n* Overall, the writing is clear, but there are still some issues that need to be addressed:\n    - Equation 2: many of the notations here are not defined, e.g., clip_C, l, theta, C.\n    - Figure 4: B_ref, S_ref, \\sigma_ref are not defined. I can guess their meaning from the context, but all notations should be defined explicitly in the text.\n    - Page 5: it says \"Simultaneously doubling the batch size and \\sigma has a negligible or small impact on accuracy (Figure 4)\". However, in Figure 4(b), the impact on the accuracy is not small. Indeed, this is claimed in the next sentence \"Reciprocally, if \\sigma > 4, dividing it by 2 simultaneously with halving q is likely to improve performance.\" These two sentences contradict each other.\n    - Table 1: I guess \"8,8.\" should be \"8,\"?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "DFJvK7iLLI", "reviewer": "Reviewer_AFbM", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\n\nThe paper studies how to transfer the hyperparameters of differentially private training based on low-cost search. It shows the privacy budget only depends on the total amount of noise (TAN) injected throughout training. They derives a scaling law for training models with DP-SGD with more than a 100 times reduction in computational budget.", "summary_of_the_review": "How to tune DP-SGD is both theoretically and practically important for private learning. The paper makes a clear contribution towards tuning DP-SGD in principle. The concluding solution is solid and well-supported. I recommend its acceptance.\n\n\\#### After rebuttal and discussion with other reviewers \\#####\nAs discussed with other reviewers, the relation of TAN with privacy accounting has been derived in literature e.g., Lemma 2 and Proposition 3 of Bun et al. 2020. The paper is not fully aware of the literature, which undermines the novelty of the contribution. I change the score to 6.", "strength_and_weaknesses": "Strength: It studies an important problem of how to scale DP-SGD with hyper-parameter search. The paper provides a neat and simple relationship between the privacy budget and the other parameters, i.e., the noise multiplier, steps and the sampling rate. This relationship motivates a way of scaling the batchsize and the noise multiplier without affecting the privacy accountant.\n\nWeakness: \n1. There is no application of using TAN relation in language task, either pretraining or fine-tuning.\n\n2. The reference needs to cover more. Especially, Yu et al. 2021 starts the fine-tuning large language models with differential privacy and low-rank reparametrization. \nYu et al. 2021 Large Scale Private Learning via Low-rank Reparametrization.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7NC5gPCt6b", "reviewer": "Reviewer_6jSv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors argue that our ability to optimize a model trained with DP-SGD is governed by a quantity they call TAN (the total amount of noise), specifically $\\Sigma^2 = 2 \\sigma^2 N^2/(B^2 S)$, where $\\Sigma$ is the TAN, $\\sigma$ is the scale of the added Gaussian noise, $N$ is the dataset size, $B$ is the batch size and $S$ is the number of updates.\n\nAdditionally, they show empirically that when $\\sigma \\gtrsim 2$, the privacy parameter $\\epsilon$ becomes a simple function of $\\Sigma$ and $\\delta$. Since it is standard to set $\\delta \\approx 1/N$, in practice this means that the privacy is determined solely by $\\Sigma$, the TAN. Meanwhile, the privacy loss $\\epsilon$ is very large when $\\sigma \\ll 2$.\n\nThe authors propose to exploit this observation by tuning hyper-parameters with small $\\sigma$, small batch size $B$ and large privacy loss $\\epsilon$, before extrapolating those hyper-parameters to $\\sigma \\gtrsim 2$ while increasing the batch size to ensure that the TAN $\\Sigma$ is constant. This significantly reduces the privacy loss $\\epsilon$ however it also significantly increases the compute cost of training. Extrapolating hyper-parameters from cheap non-private training runs to expensive runs with tight privacy guarantees reduces the total computation required to achieve strong performance with DP.", "summary_of_the_review": "On the positive side, the authors identify an important and difficult problem, and provide some useful insights. However on the downside TAN is not a reliable measure of performance in practice, and the authors only achieve strong results on one dataset so it is not clear how reliable the methodology is.\n\nI have scored the paper weak accept, however I would encourage the authors to improve the clarity of the writing and to empirically explore in more detail how test accuracy depends on (batch size, step budget, added noise) under the constraint of constant TAN.", "strength_and_weaknesses": "Strengths:\n1) Integrating the constraints of DP-SGD with optimization theory, in order to reliably predict good hyper-parameter choices, is one of the most important problems that needs to be solved in order to make differentially private deep learning practical. I'm pleased to see the authors tackle this difficult and important problem.\n2) The metric proposed by the authors is intuitive and it appears to provide some helpful intuition.\n3) The authors achieve strong performance when training from scratch on ImageNet with DP-SGD.\n\nWeaknesses:\n1) If TAN were an accurate measure of optimizability, then we should expect the train accuracy to be roughly constant at fixed TAN. However in practice, the authors find in figure 1 that the test accuracy decays log-linearly at fixed TAN as the batch size rises. I feel that the current paper is quite confusing because this discrepancy is not explicitly stated anywhere. I'd encourage the authors to make this explicit, and to explore in more detail why this discrepancy arises (eg comparing test vs train performance). \n\n2) As discussed above, Figure 1 shows that TAN over-estimates the performance of large batch sizes. It would be nice to also see a figure at fixed batch size, sweeping the number of steps S, to see whether TAN under/over-estimates the performance of small/large step budgets.\n\n3) As a minor point, note that on some truly sensitive data we cannot train non-private models (even internally). It would be good to acknowledge that the hyper-parameter transfer process described here cannot be used in these cases.\n\n4) For many of the experiments, key hyper-parameters are lifted from prior work (eg step budget, target batch size). Is it not possible to directly infer good settings for these hyper-parameters using the TAN framework?\n\n5) The paper essentially only considers ImageNet (with some very limited experiments on CIFAR-10). Since very few groups have run experiments at this scale it is not clear how challenging the baseline from De et al. is in practice.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject