Title: A New Linear Scaling Rule for Differentially Private Hyperparameter Optimization
Venue: NeurIPS 2023
Accepted: no
Keywords: Differential privacy

deep learning
Abstract: A major direction in differentially private (DP) machine learning is DP fine-tuning: pretraining a model on a source of public data and transferring the extracted features to downstream tasks.
This is an important setting because many industry deployments fine-tune publicly available feature extractors on proprietary data for downstream tasks.
In this paper we propose a new linear scaling rule, a hyperparameter optimization algorithm that privately selects hyperparameters to optimize the privacy-utility tradeoff.
A key insight into the design of our method is that our new linear scaling rule jointly increases the step size and number of steps as $\varepsilon$ increases.
Our work is the first to obtain state-of-the-art performance on a suite of 16 benchmark tasks across computer vision and natural language processing for a wide range of $\varepsilon \in [0.01,8.0]$ while accounting for the privacy cost of hyperparameter tuning.

Abstract

A major direction in differentially private (DP) machine learning is DP fine-tuning: pretraining a model on a source of public data and transferring the extracted features to downstream tasks. This is an important setting because many industry deployments fine-tune publicly available feature extractors on proprietary data for downstream tasks. In this paper we propose a new linear scaling rule, a hyperparameter optimization algorithm that privately selects hyperparameters to optimize the privacy-utility tradeoff. A key insight into the design of our method is that our new linear scaling rule jointly increases the step size and number of steps as ε increases. Our work is the first to obtain state-of-the-art performance on a suite of 20 benchmark tasks across computer vision and natural language processing for a wide range of ε ∈ [0 . 01 , 8 . 0] while accounting for the privacy cost of hyperparameter tuning.

1 Introduction

Industry deployments make use of pretrained models [79] by fine-tuning on task-specific datasets [35; 6; 69] and serving consumer applications that span the range of modalities from portraiture [65] to chatbots [44]. A crucial component of interfacing machine learning models closely with user data is ensuring that the process remains private [74], and Differential Privacy (DP) is the gold standard for quantifying privacy risks and providing provable guarantees against attacks [20]. DP implies that the output of an algorithm e.g., the final weights trained by stochastic gradient descent (SGD) do not change much if a single datapoint in the dataset changes.

Definition 1.1 (Differential Privacy) . Arandomized mechanism M with domain D and range R preserves ( ε, δ ) -differential privacy iff for any two neighboring datasets D,D ′ ∈ D and for any subset S ⊆ R we have Pr[ M ( D ) ∈ S ] ≤ e ε Pr[ M ( D ′ ) ∈ S ] + δ

0.0

0.2

0.4

0.6

0.8

1.0

(Privacy Budget)

0

20

40

60

80

100

Learning Rate x Iterations

We estimate the best hyper-

parameters for the final run

with low privacy cost trials.

small

trial

final large

run

linear scaling

where D and D ′ are neighboring datasets if they differ in a single entry, ε is the privacy budget and δ is the failure probability.

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

Differentially Private Stochastic Gradient Descent (DP-SGD) [72; 1] is the standard privacypreserving training algorithm for training neural networks on private data, with an update rule given by w ( t +1) = w ( t ) -η t | B t | (∑ i ∈ B t 1 C clip C ( ∇ /lscript ( x i , w ( t ) )) + σξ ) where the changes to SGD are the per-sample gradient clipping clip C ( ∇ /lscript ( x i , w ( t ) )) = C ×∇ /lscript ( x i ,w ( t ) ) max( C, ||∇ /lscript ( x i ,w ( t ) ) || 2 ) , and addition of noise sampled from a d -dimensional Gaussian distribution ξ ∼ N (0 , 1) with standard deviation σ . These steps alter the bias-variance tradeoff of SGD and degrade utility, creating a challenging privacy-utility tradeoff. Recent work has made significant progress in closing the gap in performance between private and non-private fine-tuning of transformer-scale models [46; 52; 7; 51], but a key problem presents a concrete obstacle to implementing DP algorithms to power real-world consumer-facing machine learning applications.

The privacy analysis of current approaches for private training does not account for the cost of hyperparameter tuning, and DP-SGD additionally increases the hyperparameter tuning burden compared to vanilla SGD. These hyperparameters include the learning rate schedule, the clipping bound, the batch size, and the amount of noise to add at each iteration. Because private training introduces additional hyperparameters, biases optimization by clipping the gradient, and imposes privacy-utility tradeoffs for existing hyperparameters, it is challenging to apply hyperparameter selection strategies from non-private training, even on the same dataset. Furthermore prior SOTA work in private training does not use similar hyperparameters as non-private training so hyperparameter search algorithms cannot be leveraged from the broader literature. More specifically, conventional non-private training uses SGD with momentum [61] or AdamW [36] to train for hundreds of epochs. However, training for additional iterations in DP-SGD requires adding additional noise [27], and taking large step sizes (such as with momentum) with low signalto-noise ratio (SNR) can destabilize training [3]. Prior

Figure 2: We compare the best private and best non-private test accuracy performances of our method to prior work using models pretrained on ImageNet21k and fine-tuned on CIFAR10 and CIFAR100 datasets. Our results at ε = 1 include the cost of hyperparameter tuning via applying the linear scaling rule at ε ∈ [0 . 01 , 0 . 1] .

work aims to minimize the amount of noise that is added during training by utilizing early stopping, training for as little as a single iteration [51]. Prior work has either fixed these hyperparameters without explanation [7] or performed an extensive search to find the best values [15], but the hundreds of trials of hyperparameter tuning [51] go unaccounted for in the privacy analysis.

We propose a new linear scaling rule (Alg. 1, Fig. 1) that automatically selects hyperparameters to optimize the privacy-utility tradeoff of private fine-tuning. In particular, as our privacy budget increases from ε = 0 →∞ , we increase the step size and number of steps. Our method accounts for the privacy cost of hyperparameter selection by allotting a small portion of the budget to find the best hyperparameters at ε /lessmuch 1 and scaling these up to ε = 1 . We summarize our contributions:

We demonstrate that our new linear scaling rule reduces the computation and privacy cost of hyperparameter optimization by an order of magnitude without sacrificing performance

Linear scaling can obtain new SOTAs for both full fine-tuning and linear probing of both convolutional and transformer architectures across 20 vision and language tasks

We compare four model architectures for a set of five vision benchmarks and find that the privatenon private utility gap decreases as models improve, with the best model across all five tasks obtaining lossless performance of 99% accuracy for ε = 1 on CIFAR10

We find that linear scaling is robust to domain shifts between private training and test data

We find that models trained with our method can provide good performance even when there is a large shift between public and private data

We validate that models trained with our method can perform well for zero-shot classification

We provide our code as a part of our empirical evaluation.

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

Algorithm 1 DP-SGD with Linear Scaling

Inputs: Private dataset D , open source feature extractor F, number of classes C , privacy budget ε , momentum ρ = 0 . 9 , first search privacy budget ε 0 , second search privacy budget ε 1

Perform second hyperparameter search initialized at r ∗ 1 = ε 1 ε 0 · r 0 to obtain the best possible value of r 1 within the second overall privacy budget ε 1

Perform first hyperparameter search to obtain the best possible value of r 0 within the first privacy budget ε 0

Perform linear interpolation to estimate the slope α and bias b of the line r = αε + b given ( r 0 , ε 0 ) , ( r 1 , ε 1 )

Extract features from D using F: X = F ( D )

Set r ∗ = αε f + b given the estimated linear interpolation

Zero-initialize classifier w ← 0 C × d

Use privacy loss variable accounting to calibrate noise parameter σ given ε

Decompose the total step size r given by linear scaling into r = η × T

end for

Compute full-batch gradient according to Eq. 1 ∇ ( i ) = 1 | D | (∑ i ∈ D clip 1 ( ∇ /lscript ( x i , w ( i ) )) + σξ ) Take a step with momentum: v ( i ) ← ρ · v ( i -1) + ∇ ( i ) , w ( i ) ← w ( i -1) -ηv ( i )

Output: ( ε f + ε 0 + ε 1 ) -Private linear model w

2 A New Linear Scaling Rule

In this section we detail how our method chooses each hyperparameter in DP-SGD, prove the privacy guarantee of the overall hyperparameter selection process, and provide a theoretical analysis.

A new linear scaling rule The well-known linear scaling rule [29] proposes increasing the learning rate with the batch size. We propose a new linear scaling rule that details how to select all hyperparameters in DP-SGD. Our method first fixes full-batch, unit clipping norm, zero initialization and use SGD with momentum, and then jointly scales the learning rate and number of steps with ε . We provide extensive ablations of each design choice in our hyperparameter optimization algorithm in Appendix A.2. Prior work has exclusively taken small step sizes [51; 52; 7; 15; 9] on the order of { 10 -5 , 10 -3 } and works that train transformers have also trained for a small number of epochs { 1 , 3 } [51; 7]. While this works well to recover the bulk of the non-private performance when ε is very small, it is natural to expect that as ε →∞ we should increase the parameters of training to more closely resemble that of non-private training. In line with this insight, we propose a linear scaling rule: jointly increase the step size and number of steps linearly with ε . We make use of this simple yet powerful heuristic in the hyperparameter selection strategy that we use in all our experiments, outlined in Algorithm 1. Given a total privacy budget ε , we use an initial portion of this budget to do binary search (random search and grid search are also valid) on the meta-hyperparameter r = η × T for a small value of ε , and use this to estimate the best value of r for the desired overall privacy budget. We provide a privacy guarantee in 2. We note that linear scaling does not hold up forever: we are primarily interested with analyzing ε ≤ 1 , and show that in this range it holds (Fig. 3).

Linear Scaling is intuitive. Applying the linear scaling rule improves the cosine similarity between noisy weight updates and the optimal solution without degrading accuracy. First note that the classification accuracy of a linear model is scale-invariant; the optimal solution of Gradient Descent with total step size r is w ′ = w ∗ / ‖ w ∗ ‖ × r : the projection of w ∗ onto B r , the ball of radius r , and for linear models, the performance (top-1 accuracy) of w ′ is the same as the performance of w ∗ : Pred ( w ′ ( x )) = Pred ( w ∗ ( x )) ∀ x ∈ D . An important factor in the success of optimization is the angle between the gradient update ∇ i and w ′ : if all our updates point in the same direction, we can expect fast convergence. Let similarity(i) = ∇ i · w ′ ‖∇ i ‖·‖ w ′ ‖ . Suppose that ‖ w i ‖ = ‖ w ′ ‖ /lessmuch 1 , then adding Gaussian noise σξ where ξ ∼ N (0 , 1) to the update will significantly decrease the cosine similarity of the updated model and w ′ . If we decrease σ , it is easy to see that this mitigates the impact on the trajectory. However, we can equivalently keep σ constant and increase the scale of the parameters, and also decrease the impact of noise on the trajectory: similarity ( w i + σξ, w ′ ) < similarity ( α · w i + σξ, α · w ′ ) , ∀ α > 1 . Note that by increasing r we scale the optimal solution

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

while keeping its performance identical, and thus optimize the cosine similarity of the noisy update. Increasing the number of iterations and the learning rate linearly increases r but does not linearly increase σ due to the composition of Gaussian differential privacy [27], therefore the impact on the optimization trajectory is minimized.

Theory We introduce two theoretical results. We first analyze the privacy cost including hyperparameter tuning of DP-RAFT under Gaussian DP (GDP). In Thm. 2.3 we analyze the performance gap between hyperparameters for noisy gradient descent in terms of an upper bound in expectation on the distance between private and non-private iterates, and find that applying the linear scaling rule improves the upper bound on this distance. Proofs of all results are in Appendix A.5.

Corollary 2.2. Algorithm 1 is ( /epsilon1, Φ( -/epsilon1 · σ/ √ T + √ T/ 2 σ )) -e /epsilon1 · Φ( -/epsilon1 · σ/ √ T -√ T/ 2 σ )) -DP. Also, for n -fold repetition, the algorithm is ( /epsilon1, Φ( -/epsilon1 · σ/ √ n · T + √ n · T/ 2 σ )) -e /epsilon1 · Φ( -/epsilon1 · σ/ √ n · T -√ n · T/ 2 σ )) -DP

Proposition 2.1. Algorithm 1 is ( √ T/σ ) -GDP. Moreover, repeating Algorithm 1 for n times for hyper parameter search would be ( √ T · n/σ )-GDP.

Theorem 2.3. Let f be gradient descent that minimizes a α -strongly convex and β -smooth function /lscript with constant learning rate η ∈ (0 , 2 β ) over T iterations. Then we can bound the 'noisy radius' distance between the noisy iterate w T and the benign iterate w T b at iteration T in expectation: E [ | w T -w T b | ] ≤ ρη × ( ∑ T -1 i max( | 1 -ηα | , | 1 -ηβ | ) i ) .

Thm. 2.3 indicates that the distance between the noisy and non-noisy weights grows in a very controlled manner; at each iteration the divergence from the previous iteration is decreased by a factor strictly less than 1, and then we add some noise. The main idea of the proof is similar to the main result in Fang et al. [23] but is simpler because we only prove the result for linear models.

We apply this theorem to logistic regression (fine-tuning a linear model on extracted features). In this setting our theorem provides an upper bound on the radius of the range of solutions that DP-SGD produces. For linear models, this radius converts directly into an upper bound on the generalization error. If we use the linear scaling rule to scale r = η × T with ε , we expect that η remains appropriately bounded and T does not grow so large that the resulting noise creates significant model drift. Therefore, we find that increasing the quantity r = η × T improves this upper bound.

While our theorem only holds for linear models, we will show that it holds empirically for the deep GPT2 and RoBERTa models, in line with Li et al. [47] who find that even the updates of a large model lie in a low-dimensional space during fine-tuning.

3 Evaluation

We provide results on a range of image classification, distribution shift, and natural language processing tasks. Full results for all datasets and models can be found in Appendix A, including ablations on all steps of our method( A.2) and key hyperparameters( A.4).

Datasets. We evaluate the performance of our method on 20 benchmark tasks spanning the data modalities of CV and NLP. Image classification: ImageNet [16], CIFAR10, CIFAR100 [40], FashionMNIST [80], STL10 [11], EMNIST [12]. Because these image classification datasets are generally considered in-distribution of the pretraining data, we also provide results on a number of distribution shift datasets from the WILDS suite [38] that have been used to evaluate various fine-tuning techniques. CIFAR10 → STL, CIFAR10p1, CIFAR10C, CIFAR100 → CIFAR100C [31], Waterbirds [67], FMoW [10], and Camelyon17 [8]. These datasets are considered benchmark tasks for distribution shifts [42; 43; 53] and include data that is not in-distribution of the training data, making for a more realistic evaluation of the capabilities of our method to solve challenging tasks. We are the first to show that DP-SGD is capable of learning to handle distribution shifts without using any techniques from the distributionally robust optimization (DRO) literature [64]. For NLP tasks we consider text classification tasks from the GLUE benchmark [76]: SST-2, QNLI, QQP, MNLI(m/mm) and for next word generation we use PersonaChat [84], WikiText-2 [54], and Enron Emails [37].

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

3.1 Linear Scaling finds near-optimal hyperparameters with low privacy cost

We first provide a concrete example of the hyperparameter search with ε 0 on CIFAR10. Note that regardless of what strategy we use for hyperparameter search here, our total privacy cost as given by Proposition 2 must be strictly less than ε 0 . Binary search, random search, Bayesian optimization and grid search are all methods that we can use for the initial hyperparameter search. For this example, for the sake of simplicity we will use random search with 3 trials, with ε 0 = 0 . 01 · √ 3 , ε 1 = 0 . 05 · √ 3 , ε f = 0 . 9 , ε 0 + ε 1 + ε f = 1 . 0 . For ε 0 = 0 . 01 , we randomly sample r uniformly in the range [1,100]=2,20,100 and then randomly decompose this into (approximate) ( η, T ) pairs of [0.2, 10], [0.5, 40], [1, 100]. These in turn evaluate to accuracies of [91.79, 73.68, 67.21], so the best value of r at ε 0 = 0 . 01 is 2. We do a similar process at ε 1 = 0 . 05 and get a best r-value of 5. We do linear interpolation and obtain the line of best fit as r = 75 · ε +1 . 25 . Approximating this to r = 75 , we apply the linear scaling rule r = η × T and randomly decomposing this value of r into an ( η, T ) pair of [0.75, 100], we produce a final accuracy of 99.00 at ε f = 0 . 9 .

0.0

0.2

0.4

0.6

0.8

1.0

0

20

40

60

80

Test Accuracy

Accuracy via Linear Scaling

(a)

0.0

0.2

0.4

0.6

0.8

1.0

10

20

30

40

Total Step Size (

× T)

Total step size by Linear Scaling

Total step size by Grid Search

(b)

Linear Scaling outperforms prior hyperparameter search techniques. We validate the effectiveness of linear scaling against the grid search baseline. In Fig. 3 (right) we compare Alg. 1 to grid search. To avoid scale mismatch on the x-axis we do not account for the privacy cost of grid search, that does n = 100 trials (on the same scale as prior work [51]). It is trivial that linear scaling outperforms a naive grid search, but we also compare the effectiveness of linear scaling against the hyperparameter selection strategies used in prior work [51]. We apply linear scaling to the ViT model used in [51] on CIFAR100. Although [51] do not directly state the hyperparameters for their best results, they specify that they use 200 hyperparameter trials with Bayesian optimization. While they obtain RDP guarantees, these guarantees do not include the privacy cost of non-privately tuning hyperparameters. We apply the linear scaling rule to extrapolate a value of r from ε = 0 . 1 to ε = 1 , obtaining r = 20 = η (0 . 2) × T (100) . We recover performance of 82 . 7% for ε = 1 , a 2% improvement over the best result for DP-Adam in [51] while accounting for the privacy cost of hyperparameter tuning. They obtain their best result for DP-Adam at T = 10 , but we cannot compute the corresponding value of r because they do not provide η . However, because they use a clipping norm of 0 . 005 we can reasonably infer that their value of r is ≈ 1000 × smaller than ours. This is farther from the optimal non-private training, as evidenced by the performance gap.

∞

Linear Scaling scales to ImageNet In Table 4 we do a granular comparison between our method and [52; 51]. We observe that our method is competitive with [51] even when accounting for the privacy cost of hyperparameter search, and that the linear scaling rule holds up at the scale of ImageNet for very large values of r = η × T . The non-private accuracy of their closed-source model is 3 . 2% higher than our open-source model, and so the private accuracy at ε = 2 is also 3 . 2% higher.

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

However, ultimately our method and the method of Mehta et al. [51] are complementary, because their method introduces new hyperparameters that we intuit our linear scaling rule can optimize. We attempted to validate this intuition empirically but were unable to reproduce the results of Mehta et al. [51] because they and Mehta et al. [52] pretrain on the closed-source JFT dataset with billions of images. We note that all numbers we report for models pretrained on ImageNet-21k using first-order methods surpass those in [51], but for sufficiently small values of ε on harder datasets the secondorder methods they propose provide better performance. We note that the method in Mehta et al. [51] only works for vision tasks, whereas our approach works for both vision and language tasks.

Linear Scaling produces robust results. In Fig. 3 we report that following Algorithm 1 produces new state-of-the-art results for all values of ε , shown in Table 5. In Appendix A.1 we provide detailed computations of the linear interpolation for multiple datasets and in Appendix A.4 we provide full results across the entire hyperparameter search space. Our results validate that this rule is robust: we can move from one set of hyperparameters to another similarly performing set of hyperparameters by increasing the number of iterations T by a constant factor and decreasing the learning rate η by the same factor (or vice versa). We find that any inaccuracy incurred by estimating the best value of r with the linear scaling rule will not reduce accuracy by much compared to doing grid search for the optimal value of r , but does reduce the privacy cost of hyperparameter tuning immensely.

3.2 Linear Scaling enables empirical analysis

Many interesting questions in DP fine-tuning remain unanswered because of the immense computational overhead of evaluating hundreds of hyperparameter trials for each privacy budget, model architecture and dataset [51]. We now employ the linear scaling rule to efficiently answer key questions in DP fine-tuning for vision tasks.

Impact of model architectures on differential privacy Many pretrained model architectures are available [79] but prior work has generally engaged with a single architecture, e.g. beit [7] or ViT [52]. We leverage our method to answer three questions:

What model architectures can provide good DP classifiers?

Is the best model task-specific, e.g., is an architecture search required?

Does the private-non private utility gap depend on the model architecture?

We report our findings in Tab. 5. We evaluate multiple transformer architectures in ViT [19], beitv1 [4] and beitv2 [58], as well as the purely convolutional architecture Convnext [48]. We find that all architectures can serve as good backbones for high-accuracy DP classification. This is somewhat surprising because the different inductive biases of transformers and purely convolutional architectures tend to produce differently

Figure 5: We compare the best private and best nonprivate performances of all models on all datasets. Weuse the linear scaling rule to scale hyperparameters from ε = 0 . 1 to ε = 1 , so our privacy analysis includes the cost of hyperparameter tuning.

structured features, but we reason that the noise added by DP will 'smooth out' these decision boundaries regardless of architecture. We note that one architecture, beitv2, performs the best on all benchmarks and also has the highest non-private ImageNet accuracy [78]. We therefore recommend that practitioners do not worry about architecture search when fine-tuning as this can incur further privacy costs, and instead pick the best model available. We are encouraged to report that the private-non private utility gap diminishes with model accuracy, enabling us to report for the first time lossless privacy of 99 . 0% on CIFAR10 at ε = 1 . We expect that as pretrained models become even better, future works may even be able to attain lossless privacy on CIFAR100, that we note remains somewhat challenging for private fine-tuning. We harness these insights for our next analyses.

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

Linear Scaling is robust to distribution shifts. Benchmarking performance on datasets with distribution shifts is increasingly important because real-world problems almost always contain distribution shift between model training and inference [64]. Prior work in distributionally robust optimization (DRO) has addressed this problem by using knowledge of the relative imbalances between groups, but recent work with vision transformers has shown that linear probing can perform well on datasets with distribution shifts [53; 41; 43]. However there is no work

that evaluates the robustness of private models to distribution shifts. We leverage our method to answer three questions:

Can DP help when there is a domain shift from private fine-tuning to test?

Can DP help when there is a domain shift from public data to private fine-tuning?

Can DP fine-tuned models perform well in the zero-shot setting?

In Table 6 we compare the performance of our method across 8 benchmarks and find that the answer to all three of these questions is yes .

The Waterbirds dataset is a well-known benchmark for evaluating the robustness of models to spurious correlations. There is a domain shift between the private training data and the private test data created by class imbalance. We are surprised to find that in the absence of any other regularization methods, DP fine-tuning actually improves performance on the OOD split. We hypothesize that the lackluster OOD non-private performance is caused by the model overfitting to the spurious correlation in the training data, and that the inherent regularization of DP prevents the model from memorizing this spurious correlation. By comparing our results to Mehta et al. [53] we determine that this robustness is unique to DP rather than an artifact of the pretrained model. Although DP does significantly degrade the ID performance, in situations where minimizing OOD error is more important, we believe that DP by itself can mitigate the domain shift from private fine-tuning to test.

Because our central assumption in DP fine-tuning is that there is no privacy leakage from the pretraining data to the private training data, it is important to understand how DP fine-tuning performs when there is a distribution shift between public data and private data. fMoW [10] and Camelyon17 [8] are two datasets that represent a signficant distribution from the pretraining data (ImageNet). We observe a similar relationship between ID and OOD degradation as above, where the OOD degradation is somewhat mitigated by DP. If we compare our results on Camelyon to the best results in Ghalebikesabi et al. [25] we find that we can improve their best performance from 91 . 1% at ε = 10 to 93 . 91% at ε = 1 . Although performance on fMoW remains quite poor, we note that it is not significantly worse than in the non-private setting. We believe that DP fine-tuning from pretrained models remains a viable strategy even when the publicly available pretraining data has a very large distribution shift from the private target data.

We finally consider the zero-shot setting, where we fine-tune a model on CIFAR and then transfer it without updating any parameters to private test datasets that once again represent a distribution shift from CIFAR. We report the performance in the OOD column. For the more minute distribution shifts of STL and CIFAR10p1 we find that the fine-tuned classifier can achieve remarkable performance without ever updating parameters on these datasets; that is, we just remap the labels as per [42]. CIFAR10C and CIFAR100C represent larger distribution shifts and are used to benchmark the robustness of models to commonly reported image corruptions [31]. Our OOD performance on these larger distribution shifts is much worse, particularly for CIFAR100 where there is a > 20% degradation. Although this is lower than the top result on the RobustBench leaderboard [13] obtains

85%

but managed to achieve reasonable performance to distribution shifts in zero-shot classification

.

accuracy, we note that once again we used no additional methods beyond DP to ensure robustness

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

3.3 Linear Scaling for language modeling

Prior work has generally focused on either CV or NLP because the methods used in DP fine-tuning differ greatly across data modalities [46; 51]; here we show that our method extends to NLP by validating on text classification and language modeling tasks. We also update all parameters when fine-tuning, displaying that our method works for both linear probing and full fine-tuning. We fine-tune GPT-2 [63] with our method for three language modeling tasks that have been benchmarked in prior works [46; 70; 30] on private fine-tuning: Persona-Chat [85], WikiText-2 [54] and Enron Emails [37]. We also fine-tune RoBERTa-base on four tasks in the GLUE benchmark: SST2, QNLI, QQP and MNLI(m/mm) in Table 7.

While prior works mainly focus on ε in { 3 , 8 } , in this work we are also interested in smaller ε s like 0 . 1 . Appendix B.1 includes the details for the experimental set-up.

Linear scaling holds for NLP tasks We analyze the performance gap between estimated total step size and optimal total step size by grid search to understand how well linear scaling performs on language modeling tasks. Fig. 8 plots the optimal perplexity and perplexity by estimated total step size at different values of ε on Enron emails. We can see that the linear scaling rule generalizes well for reported values of ε and the perplexity by the estimated total step size is close to the optimal perplexity. From Table 7 we can see that linear scaling also holds across a range of tasks in the GLUE benchmark. We also have the result for WikiText-2 in Appendix B.3.

0.0

0.5

1.0

1.5

2.0

2.5

3.0

10

11

12

13

14

Perplexity

Perplexity by linear rule

Perplexity by grid search

ε

vs Test Perplexity.

0.0

0.5

1.0

1.5

2.0

2.5

3.0

0

5

10

Total step size ( ×

T

)

Total step size by linear scaling

Total step size by grid search

ε

vs Total Step Size.

The linear scaling rule outperforms prior results on differentially private language modeling tasks. We first run a qualitative evaluation on the previous benchmark SOTA [46] on PersonaChat trained with DP-SGD by following the linear scaling rule to increase the number of epochs.

We quantitatively validate the linear scaling rule on WikiText-2 and Enron email dataset and report the result in Table 10 respectively. We select training parameters and the total step size with Alg. 1.

We can see in Table 9 that we can push the perplexity under 18 for ε = 3 and ε = 8 ; this performance is competitive with the non-private baseline. Furthermore, even when pushing for a stricter privacy guarantee ε = 0 . 5 , we can still get perplexity of 21 . 25 , that is better than the result of ε = 8 in [46]. We also report the results of ablating these hyper-parameters and varying the number of layers trained in Appendix B.2.

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

For WikiText-2, a key observation is that when we compare our results to the best prior reported results in [70], for the same number of passes over the training data (20), we obtain lower perplexity for ε = 0 . 2 than they report for ε = 3 . That is, by just increasing the effective step size from ∼ 8 × 10 -6 to ∼ 8 × 10 -3 we can strengthen the privacy guarantee without degrading performance.

4 Related Work and Discussion

De et al. [15] and Cattan et al. [9] propose the use of large batch sizes and initializing the weights to small values near-zero to standardize training. However, they use ResNet architectures rather than modern vision transformers, and in Appendix A.2 we find that other techniques that they use such as data augmentation, fine-tuning the embedding layer, and weight averaging do not always improve performance. [7] do end-to-end training of the same beit architecture we use, but we crucially observe that updating all parameters incurs the curse of dimensionality and therefore it is better to only update the last layer. Besides vision tasks, Li et al. [46] and Yu et al. [82] provide methods for fine-tuning large language models under DP-SGD by proposing new clipping methods to mitigate the memory burden of per-sample gradient clipping. However, they do not achieve performance comparable to non-private models when fine-tuning a pretrained model on the PersonaChat dataset. We adapt their techniques to the hyperparameter settings that we show are optimal for DP fine-tuning, and produce similar performance to non-private fine-tuning on the PersonaChat dataset. Yu et al. [83] report compelling results by only updating a sparse subset of the LLMs with LoRA [33]. We fine-tune GPT2 and RoBeRTA; Basu et al. [5] also fine-tune BERT models.

Papernot and Steinke [57] propose an RDP hyperparameter optimization algorithm that requires selecting the number of trials at random with a random variable, and exhibits the greatest savings when the number of hyperparameter trials is large. By contrast our linear scaling rule needs only a small fraction of the overall privacy budget for hyperparameter search. Their evaluation only tunes the learning rate of a 3-layer CNN on MNIST. Our rule accounts for multiple hyperparameters (batch size, clipping norm, momentum, learning rate, number of iterations) and produces SOTA results.

Golatkar et al. [26]; Nasr et al. [55]; Amid et al. [2] treat < 10% of the private training dataset and public and use it to improve DP-SGD. Although we do not use any private data during pretraining, future work can tackle applying linear scaling to this alternate threat model.

An open challenge in DP training is how to privately and efficiently do hyperparameter tuning. We complement the existing body of work by introducing a new linear scaling rule to privately optimize hyperparameters. Our key insight is that we can interpolate between the early-stopping regime that is best for small ε and the regime of many iterations that is best for ε → ∞ as ε increases. We provide find that our method attains new state-of-the-art accuracy across 20 tasks, on benchmark image classification tasks, distribution shift datasets, and natural language modeling tasks.

5 Limitations

Assumptions. The key assumption in DP fine-tuning is that there is no privacy leakage between public data and private data. We take steps towards qualifying this assumption by evaluating on datasets with distribution shifts between public and private data. Scope of Claims. We evaluate 20 datasets across multiple data modalities with multiple model architectures for two types of fine-tuning methods, linear probing and end-to-end training of deep ( > 100 M param) transformers. Key Factors that Influence the Performance of Our Approach. The key parameter in the linear scaling rule is how to allocate privacy budget to the initial hyperparameter search. We find that with privacy budgets as small as ε = 0 . 01 we can still effectively forecast the linear trend to determine the best hyperparameters for the main privacy budget we consider ε = 1 . However, if we need to consider even smaller privacy budgets, it may be challenging to accurately extrapolate hyperparameters.

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

References

M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . ACM, oct 2016. doi: 10.1145/2976749.2978318.

E. Amid, A. Ganesh, R. Mathews, S. Ramaswamy, S. Song, T. Steinke, T. Steinke, V. M. Suriyakumar, O. Thakkar, and A. Thakurta. Public data-assisted mirror descent for private model training. In Proceedings of the 39th International Conference on Machine Learning , pages 517-535. PMLR, 2022.

E. Bagdasaryan and V. Shmatikov. Differential privacy has disparate impact on model accuracy, 2019. URL https://arxiv.org/abs/1905.12101 .

H. Bao, L. Dong, S. Piao, and F. Wei. Beit: Bert pre-training of image transformers, 2021. URL https://arxiv.org/abs/2106.08254 .

P. Basu, T. S. Roy, R. Naidu, Z. Muftuoglu, S. Singh, and F. Mireshghallah. Benchmarking differential privacy and federated learning for bert models, 2022.

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language models are few-shot learners, 2020. URL https://arxiv.org/abs/2005.14165 .

Z. Bu, J. Mao, and S. Xu. Scalable and efficient training of large convolutional neural networks with differential privacy. arXiv preprint arXiv:2205.10683 , 2022.

P. B´ andi, O. Geessink, Q. Manson, M. Van Dijk, M. Balkenhol, M. Hermsen, B. Ehteshami Bejnordi, B. Lee, K. Paeng, A. Zhong, Q. Li, F. G. Zanjani, S. Zinger, K. Fukuta, D. Komura, V. Ovtcharov, S. Cheng, S. Zeng, J. Thagaard, A. B. Dahl, H. Lin, H. Chen, L. Jacobsson, M. Hedlund, M. C ¸ etin, E. Halıcı, H. Jackson, R. Chen, F. Both, J. Franke, H. K¨ ustersVandevelde, W. Vreuls, P. Bult, B. van Ginneken, J. van der Laak, and G. Litjens. From detection of individual metastases to classification of lymph node status at the patient level: The camelyon17 challenge. IEEE Transactions on Medical Imaging , 38(2):550-560, 2019. doi: 10.1109/TMI.2018.2867350.

Y. Cattan, C. A. Choquette-Choo, N. Papernot, and A. Thakurta. Fine-tuning with differential privacy necessitates an additional hyperparameter search, 2022. URL https://arxiv.org/ abs/2210.02156 .

G. Christie, N. Fendley, J. Wilson, and R. Mukherjee. Functional map of the world, 2017. URL https://arxiv.org/abs/1711.07846 .

A. Coates, A. Ng, and H. Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics , pages 215-223. JMLR Workshop and Conference Proceedings, 2011.

G. Cohen, S. Afshar, J. Tapson, and A. van Schaik. Emnist: an extension of mnist to handwritten letters, 2017. URL https://arxiv.org/abs/1702.05373 .

F. Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein. Robustbench: a standardized adversarial robustness benchmark, 2021.

A. Cutkosky and H. Mehta. Momentum improves normalized SGD. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 2260-2268. PMLR, 13-18 Jul 2020. URL https://proceedings.mlr.press/v119/cutkosky20b.html .

S. De, L. Berrada, J. Hayes, S. L. Smith, and B. Balle. Unlocking high-accuracy differentially private image classification through scale, 2022. URL https://arxiv.org/abs/2204.13650 .

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical 454 image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 455 248-255, 2009. doi: 10.1109/CVPR.2009.5206848. 456

457

458

459

J. Diffenderfer, B. R. Bartoldson, S. Chaganti, J. Zhang, and B. Kailkhura. A winning hand: Compressing deep networks can improve out-of-distribution robustness, 2021. URL https: //arxiv.org/abs/2106.09129 .

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

J. Dong, A. Roth, and W. J. Su. Gaussian differential privacy, 2019. URL https://arxiv. org/abs/1905.02383 .

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2020. URL https://arxiv.org/abs/ 2010.11929 .

C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In Proceedings of the Third Conference on Theory of Cryptography , TCC'06, page 265-284, Berlin, Heidelberg, 2006. Springer-Verlag. ISBN 3540327312.

C. Dwork, N. Kohli, and D. Mulligan. Differential privacy in practice: Expose your epsilons! Journal of Privacy and Confidentiality , 9, 10 2019. doi: 10.29012/jpc.689.

U. Erlingsson, V. Feldman, I. Mironov, A. Raghunathan, K. Talwar, and A. Thakurta. Amplification by shuffling: From local to central differential privacy via anonymity, 2018. URL https://arxiv.org/abs/1811.12469 .

H. Fang, X. Li, C. Fan, and P. Li. Improved convergence of differential private SGD with gradient clipping. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=FRLswckPXQ5 .

R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. Imagenettrained cnns are biased towards texture; increasing shape bias improves accuracy and robustness, 2018. URL https://arxiv.org/abs/1811.12231 .

S. Ghalebikesabi, L. Berrada, S. Gowal, I. Ktena, R. Stanforth, J. Hayes, S. De, S. L. Smith, O. Wiles, and B. Balle. Differentially private diffusion models generate useful synthetic images, 2023.

A. Golatkar, A. Achille, Y.-X. Wang, A. Roth, M. Kearns, and S. Soatto. Mixed differential privacy in computer vision, 2022.

S. Gopi, Y. T. Lee, and L. Wutschitz. Numerical composition of differential privacy, 2021. URL https://arxiv.org/abs/2106.02848 .

P. Goyal, P. Doll´ ar, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. Accurate, large minibatch sgd: Training imagenet in 1 hour, 2017. URL https: //arxiv.org/abs/1706.02677 .

P. Goyal, P. Doll´ ar, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He. Accurate, large minibatch sgd: Training imagenet in 1 hour, 2018.

S. Gupta, Y. Huang, Z. Zhong, T. Gao, K. Li, and D. Chen. Recovering private text in federated learning of language models. In Advances in Neural Information Processing Systems (NeurIPS) , 2022.

D. Hendrycks and T. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations, 2019. URL https://arxiv.org/abs/1903.12261 .

E. Hoffer, T. Ben-Nun, I. Hubara, N. Giladi, T. Hoefler, and D. Soudry. Augment your batch: 497 better training with larger batches, 2019. URL https://arxiv.org/abs/1901.09335 . 498

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. Lora: 499 Low-rank adaptation of large language models, 2021. 500

P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson. Averaging weights leads to 501 wider optima and better generalization, 2018. URL https://arxiv.org/abs/1803.05407 . 502

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, 503 J. Wu, and D. Amodei. Scaling laws for neural language models, 2020. URL https://arxiv. 504 org/abs/2001.08361 . 505

506

507

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization, 2014. URL https: //arxiv.org/abs/1412.6980 .

508

509

510

511

512

513

B. Klimt and Y. Yang. The enron corpus: A new dataset for email classification research. In European conference on machine learning , pages 217-226. Springer, 2004.

P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gao, T. Lee, E. David, I. Stavness, W. Guo, B. A. Earnshaw, I. S. Haque, S. Beery, J. Leskovec, A. Kundaje, E. Pierson, S. Levine, C. Finn, and P. Liang. Wilds: A benchmark of in-the-wild distribution shifts, 2020. URL https://arxiv.org/abs/2012.07421 .

S. Kornblith, J. Shlens, and Q. V. Le. Do better imagenet models transfer better?, 2018. URL 514 https://arxiv.org/abs/1805.08974 . 515

A. Krizhevsky et al. Learning multiple layers of features from tiny images. Technical report, 516 Citeseer, 2009. 517

A. Kumar, A. Raghunathan, R. Jones, T. Ma, and P. Liang. Fine-tuning can distort pretrained 518 features and underperform out-of-distribution, 2022. URL https://arxiv.org/abs/2202. 519 10054 . 520

A. Kumar, A. Raghunathan, R. Jones, T. Ma, and P. Liang. Fine-tuning can distort pretrained 521 features and underperform out-of-distribution, 2022. URL https://arxiv.org/abs/2202. 522 10054 . 523

A. Kumar, R. Shen, S. Bubeck, and S. Gunasekar. How to fine-tune vision models with sgd, 524 2022. URL https://arxiv.org/abs/2211.09359 . 525

N. Lambert, L. Castricato, L. von Werra, and A. Havrilla. Illustrating reinforcement learning 526 from human feedback (rlhf). Hugging Face Blog , 2022. https://huggingface.co/blog/rlhf. 527

528

529

H. Li, P. Chaudhari, H. Yang, M. Lam, A. Ravichandran, R. Bhotika, and S. Soatto. Rethinking the hyperparameters for fine-tuning, 2020. URL https://arxiv.org/abs/2002.11770 .

530

531

532

533

534

535

536

537

538

X. Li, F. Tram` er, P. Liang, and T. Hashimoto. Large language models can be strong differentially private learners, 2021. URL https://arxiv.org/abs/2110.05679 .

X. Li, D. Liu, T. Hashimoto, H. A. Inan, J. Kulkarni, Y. T. Lee, and A. G. Thakurta. When does differentially private learning not suffer in high dimensions?, 2022. URL https://arxiv. org/abs/2207.00160 .

Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie. A convnet for the 2020s, 2022. URL https://arxiv.org/abs/2201.03545 .

X. Mao, Y. Chen, X. Jia, R. Zhang, H. Xue, and Z. Li. Context-aware robust fine-tuning, 2022. URL https://arxiv.org/abs/2211.16175 .

H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang. Learning differentially private recurrent 539 language models, 2017. URL https://arxiv.org/abs/1710.06963 . 540

541

542

H. Mehta, W. Krichene, A. Thakurta, A. Kurakin, and A. Cutkosky. Differentially private image classification from features, 2022. URL https://arxiv.org/abs/2211.13403 .

H. Mehta, A. Thakurta, A. Kurakin, and A. Cutkosky. Large scale transfer learning for differen543 tially private image classification, 2022. URL https://arxiv.org/abs/2205.02973 . 544

R. Mehta, V. Albiero, L. Chen, I. Evtimov, T. Glaser, Z. Li, and T. Hassner. You only need 545 a good embeddings extractor to fix spurious correlations, 2022. URL https://arxiv.org/ 546 abs/2212.06254 . 547

S. Merity, C. Xiong, J. Bradbury, and R. Socher. Pointer sentinel mixture models. In Inter548 national Conference on Learning Representations , 2017. URL https://openreview.net/ 549 forum?id=Byj72udxe . 550

M. Nasr, S. Mahloujifar, X. Tang, P. Mittal, and A. Houmansadr. Effectively using public data 551 in privacy preserving machine learning, 2023. URL https://openreview.net/forum?id= 552 5R96mIU85IW . 553

A. Panda, S. Mahloujifar, A. N. Bhagoji, S. Chakraborty, and P. Mittal. Sparsefed: Mitigating 554 model poisoning attacks in federated learning with sparsification, 2021. URL https://arxiv. 555 org/abs/2112.06274 . 556

N. Papernot and T. Steinke. Hyperparameter tuning with renyi differential privacy, 2021. URL 557 https://arxiv.org/abs/2110.03620 . 558

Z. Peng, L. Dong, H. Bao, Q. Ye, and F. Wei. Beit v2: Masked image modeling with vector559 quantized visual tokenizers, 2022. URL https://arxiv.org/abs/2208.06366 . 560

B. Polyak and A. B. Juditsky. Acceleration of stochastic approximation by averaging. Siam 561 Journal on Control and Optimization , 30:838-855, 1992. 562

V. Prabhu, S. Khare, D. Kartik, and J. Hoffman. Sentry: Selective entropy optimization via 563 committee consistency for unsupervised domain adaptation, 2020. URL https://arxiv.org/ 564 abs/2012.11460 . 565

N. Qian. On the momentum term in gradient descent learning algorithms. Neural networks , 12 566 (1):145-151, 1999. 567

S. Qiao, H. Wang, C. Liu, W. Shen, and A. Yuille. Micro-batch training with batch-channel nor568 malization and weight standardization, 2019. URL https://arxiv.org/abs/1903.10520 . 569

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are 570 unsupervised multitask learners. 2019. 571

H. Rahimian and S. Mehrotra. Frameworks and results in distributionally robust optimization. 572 Open Journal of Mathematical Optimization , 3:1-85, jul 2022. doi: 10.5802/ojmo.15. URL 573 https://doi.org/10.5802%2Fojmo.15 . 574

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis 575 with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision 576 and Pattern Recognition (CVPR) , pages 10684-10695, June 2022. 577

E. K. Ryu and S. P. Boyd. A primer on monotone operator methods. 2015. 578

S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural networks 579 for group shifts: On the importance of regularization for worst-case generalization, 2019. URL 580 https://arxiv.org/abs/1911.08731 . 581

V. Shejwalkar, A. Ganesh, R. Mathews, O. Thakkar, and A. Thakurta. Recycling scraps: 582 Improving private learning by leveraging intermediate checkpoints, 2022. URL https:// 583 arxiv.org/abs/2210.01864 . 584

Y. Shen, Z. Wang, R. Sun, and X. Shen. Towards understanding the impact of model size on 585 differential private classification, 2021. URL https://arxiv.org/abs/2111.13895 . 586

W. Shi, S. Chen, C. Zhang, R. Jia, and Z. Yu. Just fine-tune twice: Selective differential privacy 587 for large language models. arXiv preprint arXiv:2204.07667 , 2022. 588

C. Shorten and T. M. Khoshgoftaar. A survey on image data augmentation for deep learning. 589 Journal of Big Data , 6(1):60, Jul 2019. ISSN 2196-1115. doi: 10.1186/s40537-019-0197-0. 590 URL https://doi.org/10.1186/s40537-019-0197-0 . 591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

S. Song, K. Chaudhuri, and A. D. Sarwate. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing , pages 245-248, 2013. doi: 10.1109/GlobalSIP.2013.6736861.

I. Sutskever, J. Martens, G. Dahl, and G. Hinton. On the importance of initialization and momentum in deep learning. In S. Dasgupta and D. McAllester, editors, Proceedings of the 30th International Conference on Machine Learning , volume 28 of Proceedings of Machine Learning Research , pages 1139-1147, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR. URL https://proceedings.mlr.press/v28/sutskever13.html .

A. Team. Learning with privacy at scale, 2017. URL https://docs-assets.developer. apple.com/ml-research/papers/learning-with-privacy-at-scale.pdf .

H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jegou. Training dataefficient image transformers; distillation through attention. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 10347-10357. PMLR, 18-24 Jul 2021. URL https://proceedings.mlr.press/v139/touvron21a.html .

A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum?id=rJ4km2R5t7 .

Y.-X. Wang, B. Balle, and S. P. Kasiviswanathan. Subsampled renyi differential privacy and analytical moments accountant. In K. Chaudhuri and M. Sugiyama, editors, Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics , volume 89 of Proceedings of Machine Learning Research , pages 1226-1235. PMLR, 16-18 Apr 2019. URL https://proceedings.mlr.press/v89/wang19b.html .

R. Wightman. Pytorch image models. https://github.com/rwightman/ pytorch-image-models , 2019.

T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush. Huggingface's transformers: State-of-the-art natural language processing, 2019. URL https://arxiv.org/abs/1910.03771 .

H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017. URL https://arxiv.org/abs/1708.07747 .

A. Yousefpour, I. Shilov, A. Sablayrolles, D. Testuggine, K. Prasad, M. Malek, J. Nguyen, S. Ghosh, A. Bharadwaj, J. Zhao, G. Cormode, and I. Mironov. Opacus: User-friendly differential privacy library in pytorch, 2021. URL https://arxiv.org/abs/2109.12298 .

D. Yu, S. Naik, A. Backurs, S. Gopi, H. A. Inan, G. Kamath, J. Kulkarni, Y. T. Lee, A. Manoel, L. Wutschitz, S. Yekhanin, and H. Zhang. Differentially private fine-tuning of language models, 2021. URL https://arxiv.org/abs/2110.06500 .

D. Yu, H. Zhang, W. Chen, J. Yin, and T.-Y. Liu. Large scale private learning via low-rank reparametrization, 2021.

S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, and J. Weston. Personalizing dialogue agents: I have a dog, do you have pets too?, 2018.

S. Zhang, E. Dinan, J. Urbanek, A. Szlam, D. Kiela, and J. Weston. Personalizing dialogue agents: I have a dog, do you have pets too?, 2018. URL https://arxiv.org/abs/1801. 07243 .

Y. Zhu and Y.-X. Wang. Poission subsampled r´ enyi differential privacy. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 7634-7642. PMLR, 09-15 Jun 2019. URL https://proceedings.mlr.press/v97/zhu19c.html .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel hyperparameter tuning method in the presence of a privacy budget: linearly extrapolating from observations with very low privacy loss.\n\nSTRENGTHS:\nThe core technique presented here is certainly interesting and deserving of future study. The paper tackles an issue which is often unaddressed in the literature on training DP models: that of choosing hyperparameters subject to a privacy budget. This problem itself is also deserving of further study.\n\nWEAKNESSES:\n* A primarily empirical paper will live and die with the strength of its baselines (as well as its upper bounds in a case like this one where upper bounds on the efficacy of the technique can be computed). The baselines here are insufficiently strong, and do not seem to reflect the statements in the cited papers. The core technique _could_ be a component of a strong paper, but this paper is not it.\n\n* Some baseline issues: the citation problems with [51], [52] (detailed below). Lack of comparison to the 'naive baseline' of directly applying gaussian mechanism to results of grid search, say given known training statistics / optimal hparam values for nonprivate datasets (to avoid infinite regress, and here not so much of a problem since the experiments are all focused on public feature extractor settings). Lack of clear comparison to the 'upper bound' of _forgetting_ about the privacy cost of hparam search, which _should_ be an upper bound in _all_ scenarios considered here (IE, performing a sufficiently large grid search directly targeted at the problem at hand).\n\n* On [51]/[52], I see the reporeted CIFAR10 numbers from [51] as 98.8\\% at $\\epsilon=1$ and 98.9 at $\\epsilon=\\infty$ (table 1 of [the arxiv version](https://arxiv.org/pdf/2211.13403.pdf)). Is there a typo in figure 2 of the paper under submission? Similarly, [51] seems to claim 88.1\\% and 90.6\\% at the $\\epsilon=1, \\infty$ for CIFAR-100. I uncovered these discrepancies since the paper under submission seemed to present implausibly strong results to me--e.g. it should be _impossible_ to achieve at epsilon=1 what none of the cited papers achieved at epsilon=\\infty just by tuning hyperparameters (see figure 2). \n\n* The statements of timing on Imagenet seem wrong? The cited paper [51] seems to be pointing to a version from Nov 2022, clicking through to [52] seems to show a version uploaded in May 2022--so where are Jan 2023 and 'within the last month' coming from?\n\n* Some more consideration required in decomposition of $r$--do we know that random decomposition 'is enough'? Presumably it's not, since we _can_ generate an $\\eta$ for which the problem will presumably diverge?\n\nQUESTIONS:\nThe major questions I have for the authors here are the sources for both the claims on timing in relation to ImageNet (see weaknesses above) and the issues in citations (particularly with [51] and [52]).\n\nLIMITATIONS:\nSocietal impact not immediately applicable.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a linear scaling rule for finding the optimal value of the learning rate and number of training steps for differentially private SGD (DP-SGD). The idea is simple, small amount of privacy budgets are allocated for two initial DP learning rate optimization procedures, and then the values are extrapolated to bigger epsilon-values using linear scaling (as a function of epsilon). The work is mostly experimental, and the experimental results e.g. with CIFAR-10 show that for epsilon between 0 and 1, the scaling rule seems to nicely fit the optimal values found by the grid search.\n\nSTRENGTHS:\n- The idea seems very interesting and novel.\n- The paper is mostly written well and is easily readable.\n\nWEAKNESSES:\n- The technique is restricted to optimising the learning rate and the length of the training. I wonder if similar extrapolation (perhaps more generally polynomial extrapolation) could be used to find optimal the optimal hyperparameter values for other hyperparameters.\n\n- The technical part could be written more carefully. It remains unclear whether you use RDP or GDP. The hyperparameter tuning cost of the method by Papernot and Steinke is in terms of RDP, but you list theoretical results in terms of GDP. In the end of Alg. 1 you write that the total cost is \"$\\varepsilon_f + \\varepsilon_0 + \\varepsilon_1$\". Is that approximate DP? In case you use the classical composition result where you just add up the privacy parameters, what happens to the $\\delta$-parameters?\n\n- Some conclusions are a vaguely formulated/confusing. On p. 7 you have the subtitle \"Linear Scaling is robust to distribution shifts\", but then you seem to show and also claim in the subsequent text that DP itself is robust to distribution shifts. Somehow the message is vague here.\n\n- The contribution remains too thin in my opinion. There is really no theoretical or even heuristic explanation for the proposed scaling rule. There two theoretical results given, a GDP composition result (which is well known and should be cited as such) and another result of which importance I find difficult to judge.\n\nQUESTIONS:\nWhat does the following line in Alg. 1 mean: \"Use privacy loss variable accounting to calibrate noise parameter $\\sigma$ given $\\varepsilon$\" ?\n\n- I somehow find it hard to believe that you would get 99 percent test accuracy for CIFAR-10 with $\\varepsilon=1.0$. The SOTA results by De at al. (\"Unlocking High-Accuracy Differentially Private Image Classification through Scale\") are somewhere at 95 percent for  $\\varepsilon=1.0$. What made you to get this good results?\n\n- You mention in the description of the method that you scale up to $\\varepsilon=1.0$. Why is that? You seem to use varying $\\varepsilon$-values in the experiments.\n\nThere seem to be typos here and there, please go through the writing carefully. Here few examples:\nline 387-388: \" treat < 10% of the private training dataset and public\" -> \"as public\"\nline 394: \"We provide find that our method attains...\"\n\nI don't quite understand the following sentence:\n\"The key assumption in DP fine-tuning is that there is no privacy leakage between public data and private data.\"\nYou mean that there are not too big distribution shifts?\n\nLIMITATIONS:\nSome of the limitations are discussed in Section 5 but it could be expanded I think.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis study proposes a new algorithm for privately selecting hyperparameters subject to maximizing the model utility. The new algorithm draws inspiration from the linear scaling rule that suggests increasing learning rate as batch size increases. Given the number of hyperparameters in DP-SGD the proposed algorithm simply scales learning rate and number of iterations as the privacy budget increases. This introduces a new hyperparameter that is selected privately with a portion of the privacy budget while the rest is used to perform the normal hyperparameter search. The study provides brief theoretical intuition for why we can expect this linear scaling rule to more efficiently determine optimal hyperparamters compared to previous methods and extensive empirical evidence on 20 different benchmark datasets.\n\nSTRENGTHS:\n- One of the first papers to demonstrate improved privacy-utility tradeoffs that takes hyperparameter tuning into account. This is substantial as the field has mainly focused on evaluating the privacy-uility tradeoff without considering the privacy cost of hyperparameter tuning. As we move towards more practical implementations, this will be necessary. \n- Clever use of the linear scaling rule to perform hyperparameter search and the resulting algorithm is simple to use. \n- Extensive empirical evaluation and insightful analysis. For example, very few analyses have been done on the intersection of DP and distriutional shift. Yet, this linear scaling rule that is proposed holds in the presence of distribution shift.\n\nWEAKNESSES:\n- “We are 165 the first to show that DP-SGD is capable of learning to handle distribution shifts without using any 166 techniques from the distributionally robust optimization (DRO) literature” -> There are a couple of other papers that draw this connection. [1,2]\n- Lack of comparison to other private hyperparameter selection algorithms or hyperparameter free private learning algorithms [3, 4]\n- Unclear why the initial hyperparameter search can be done with such a small privacy budget even though this is a key factor driving the performance of the algorithm.\n\n[1] Kulynych, Bogdan, et al. \"What you see is what you get: Distributional generalization for algorithm design in deep learning.\" arXiv preprint arXiv:2204.03230 (2022): 13.\n[2] Hulkund, Neha, et al. \"Limits of Algorithmic Stability for Distributional Generalization.\" (2022).\n[3] Mohapatra, Shubhankar, et al. \"The role of adaptive optimizers for honest private hyperparameter selection.\" Proceedings of the aaai conference on artificial intelligence. Vol. 36. No. 7. 2022\n[4] Koskela, Antti, and Tejas Kulkarni. \"Practical differentially private hyperparameter tuning with subsampling.\" arXiv preprint arXiv:2301.11989 (2023).\n\nQUESTIONS:\n1. What is the intuition for why the initial hyperparameter search can be done with such a small privacy budget? Is it possible to simply randomly initialize $r_0$ and achieve similar performance?\n2. How does this method compare with optimization algorithms that reduce the need for hyperparameter tuning?\n\nLIMITATIONS:\nThe paper does address the technical limitations of the paper (specifically the assumption of access to public and private data). The main improvement for the limitations is to address the comparison to other tuning algorithms or optimization algorithms that don’t require as much tuning.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new method to conduct hyper parameter tuning for DP stochastic gradient descent. The method is based on a linear scaling rule, with two pilot runs using small PLBs and a third run chosen based on a linear extrapolation from the first two. The pilot runs are used to establish an estimate of the interpret and the slope that the total step size r would have with the PLB. The author uses this linear scaling rules to demonstrate that it works as well as grid search in optimizing for the accuracy in a suite of benchmark tasks, and attempts to apply this rule to perform empirical analysis on the potential of making existing model architectures DP and the issue of robustness against domain shifts.\n\nMy assessment, consisting of strengths, weaknesses, and questions, can be found in the sections below.\n\nSTRENGTHS:\nThe best thing about this paper is that it develops a method based on an intuition that is potentially worthwhile. This intuition is captured in the small paragraph in Section 2, titled Linear Scaling is Intuitive. What the authors have proposed is essentially a dimensional reduction to the hyperparameter search, and the reason why that works, in the sense that what you end up finding may not be so far off from a greedier search, is due to the geometry where you force the updates to be more congruent with each other. The whole idea of a linear scaling would otherwise be rather unremarkable, but if the author can further develop this intuition, formalize it and expand on it, it would contribute some insight to the literature.\n\nWEAKNESSES:\nThe most damning weakness of this paper is that it is written without due care. As a consequence, the main results and the accompanying algorithm are not correct as stated. I don’t suggest that the author is not capable of presenting the correct science -- to that question I do not know the answer. However, as things stand, the paper is not ready to be published.\n\nThe presentation in the introductory and main result sections wanders seemingly fluidly between epsilon-DP, (epsilon, delta)-DP and Gaussian DP:\n1. Definition 1.1 is given in the language of (epsilon, delta)-DP;\n2. The DP-SGD Definition is given without a quantification of its DP guarantee at all;\n3.  Algorithm 1, which employs the DP-SGD given before, states that its output is epsilon-DP, where an alleged PLB accounting between epsilon and sigma is not supplied. (In reality, a delta would be needed, so the provided guarantee is incorrect to begin with.)\n4. Then Proposition 2.1, which concerns Algorithm 1, gives a GDP guarantee in relation to sigma only, where sigma is not constructed as a function of epsilon (or the missing delta) in Algorithm 1;\n5. Corollary 2.2 now qualifies Algorithm 1 as (epsilon, delta)-DP, with a one line proof given in the Appendix citing another work and has no substance on its own.\n\nAll of the above is confusing at best. For a standard reader, a student coming into the DP world for example, these are not pedagogically informative.\n\nBack to Algorithm 1:\n1. It contains four privacy loss budget expressions: epsilon, epsilon_0, epsilon_1, and epsilon_f. Based on the context, am I to infer that epsilon is the sum of the rest of the three? \n2. The quantity r on the 12th line (beginning with Decompose). Is this a generic r, as you use it on line 7, or is it in fact referring to r* on line 9?\n3. When you speak of the “decomposition” or r, what is to be found exactly -- eta given r and T (my guess), T given r and eta (please explain), or both eta and T given r (please explain as well)? If my guess is correct, then do we know that the eta found here will automatically satisfy the condition given in Theorem 2.3? \n\nLine 143 begins with “We apply this theorem to logistic regression.” Then Line 151 continues, “While our theorem only holds for linear models…”. Nothing said between Line 143 and Line 151 constitutes a proof that Theorem 2.3 applies to linear models. This point should either be rectified with a formal analysis or deleted, so as to not be an exaggeration of contribution.\n\nSection 3.1 is misleading and should be thoroughly rewritten to rid all expressions of “randomly”, “sample”, and “uniformly”. The author picked the experimental values. No sampling, particularly random sampling nor uniform random sampling of values took place. It is not clear to what is “r = 75” an approximation (Line 179).\n\nIn addition, based on my reading of Section 3.2 I believe it should not be presented as is.  My understanding of what Section 3.2 does is that it uses the linear scaling rule proposed in this work to construct \"accuracy hypotheticals” for the listed models and datasets as well as the domain shift situations, and compare those numbers with existing experimental results. If that is the case, this is a dangerous operation. The linear scaling rule, when used as a heuristic to make tuning faster, is fine as the worst that could happen is that one misses out on the most efficient model tuning. However, the way that the rule is employed in Section 3.2 it is taken as a scientific theory between epsilon and accuracy. The accuracy numbers you get from it is no different than a terribly extrapolated number based on a linear model fitted with two data points. If you really want to use the linear scaling rule to poke at the said questions, actual experiments should be conducted to confirm these extrapolations. Of course, I may have misunderstood what was actually done and in particular, whether actual experiments were performed — although if so, what would be the contribution from the linear scaling rule?\n\nQUESTIONS:\nBelow are a set of minor comments.\n\n1. Please define SOTA. \n2. Please also define r, eta, and T before they are used for the first time. After the first time, there is no need to always state that one is the product of the other two.\n3. Lines 83-85: I understand that these are conclusions based on your empirical analysis in Sections 3.2 and 3.3. As the statements read, they cannot possibly be correct without qualifications to the specific characteristics of the models and benchmarks that you examine.  \n4. The fourth to last line in Algorithm 4 does not read properly.\n5. Line 106: “We provide a privacy guarantee in 2.” What is 2?\n6. The x axes of Figure 3 are not labeled.\n7. Line 196: “…their value of r is ≈ 1000× smaller than ours.” What you mean is that “their value of r is about a thousand times smaller than ours.”\n8. Line 216: “In Fig. 3 we report that following Algorithm 1 produces new state-of-the-art results for all values of ε, shown in Table 5.” This sentence seems to imply that you have applied Algorithm 1 to all Models and datasets listed in Figure 5 (which should be Table 5). This is contrary to my understanding that Section 3.2 is a thought experiment.\n\nLIMITATIONS:\nAs stated before, I believe the paper is written hastily to the point that the central results presented are incorrect, significantly harming the quality of the contribution and its readability. I am also concerned with the scientific merit of Section 3.2. These points are elaborated in detail in my comment section on Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-27"}