Differentially Private Image Classification from Features
Authors: Harsh Mehta
Walid Krichene
Abhradeep Guha Thakurta
Alexey Kurakin
Ashok Cutkosky
Venue: Accepted by TMLR ()

Abstract: In deep learning, leveraging transfer learning has recently been shown to be an effective strategy for training large high performance models with Differential Privacy (DP). Moreover, somewhat surprisingly, recent works have found that privately training just the last layer of a pre-trained model provides the best utility with DP. While past studies largely rely on using first-order differentially private training algorithms like DP-SGD for training large models, in the specific case of privately learning from features, we observe that computational burden is often low enough to allow for more sophisticated optimization schemes, including second-order methods. To that end, we systematically explore the effect of design parameters such as loss function and optimization algorithm. We find that, while commonly used logistic regression performs better than linear regression in the non-private setting, the situation is reversed in the private setting. We find that least-squares linear regression is much more effective than logistic regression from both privacy and computational standpoint, especially at stricter epsilon values ($\epsilon < 1$). On the optimization side, we also explore using Newton's method, and find that second-order information is quite helpful even with privacy, although the benefit significantly diminishes with stricter privacy guarantees. While both methods use second-order information, least squares is more effective at lower epsilon values while Newton's method is more effective at larger epsilon values. To combine the benefits of both methods, we propose a novel optimization algorithm called DP-FC, which leverages feature covariance instead of the Hessian of the logistic regression loss and performs well across all $\epsilon$ values we tried. With this, we obtain new SOTA results on ImageNet-1k, CIFAR-100 and CIFAR-10 across all values of $\epsilon$ typically considered. Most remarkably, on ImageNet-1K, we obtain top-1 accuracy of 88\% under DP guarantee of (8, $8 * 10^{-7}$) and 84.3\% under (0.1, $8 * 10^{-7}$).

Published in Transactions on Machine Learning Research (04/2023)
Differentially Private Image Classification from Features
Harsh Mchta
hershm
Google Research
Walid Krichene
walsdk@googgle.c
Google Research
Abhradeep Thakurta Google Research
alhakurta@google..ooo
Alexey Kurakin
kurukin
Google Research
Ashok Cutkosky
ashok ashok@cutkosky.comm com
Boston University
Reviewed on OpenReview: https: // / openreview. net/ forum? Cy6pLclmu
Abstract
In deep learning, leveraging transfer learning has recently been shown to be an effective strategy for training large high performance models with Differential Privacy (DP). Moreover, somewhat surprisingly, recent works have found that privately training just the last layer of a pro-trained model provides the best utility with DP. While past studies largely rely on using first-order differentially private training algorithms like DP-SGD for training large models, in the specific case of privately learning from features, we observe that computational burden often low enough to allow for more sophisticated optimization schemes, including second-order methods. To that end, systematically explore the effect of design parameters such as loss function and optimization algorithm. We find that, while commonly used logistic regression performs better than linear regression in the non-private setting, the situation is reversed in the private setting. We find that least-squares linear regression is much more effective than logistic regression from both privacy and computational standpoint, especially at stricter epsilon values (e < 1). On the optimization side, we also explore using Newton's method, and find that second-order information quite helpful even with privacy, although the benefit significantly diminishes with stricter privacy guarantees. While both methods second-order information, least squares more effective at lower epsilon values while Newton's method is more effective at larger epsilon values. To combine the benefits of both methods, we propose novel optimization algorithm called DP-FC, which leverages feature covariance instead of the Hessian of the logistic regression loss and performs well across all values we tried. With this, we obtain new SOTA results on ImageNet- CIFAR-100 and CIFAR-10 across all values of e typically considered. Most remarkably, on ImageNNet-1K, we obtain top-1 accuracy of 88% under DP guarantee of (8, 8 10-7) and 84.3% under (0.1, 8 10-7).
1
Introduction
Despite impressive performance, large machine learning models are susceptible attacks. Previous work has demonstrated successful membership attacks where the goal is to extract exact instances of training
Code: http://www///.eoooooooooooo/ooee

---

Published in Transactions on Machine Learning Research (04/2023)
examples that the model was trained on Shokri et al. 2017 Carlini et al 2019 2021 Choquette-Choo et al,
2020; Liu et al. 2021b: Balle et al. 2022; and models of larger size are known to be more likely to memorize training data. These attacks can be quite egregious if the model was trained on sensitive data like personal photos or emails. One approach to mitigate this risk is to train such models with privacy guarantees. In particular, Differential Privacy (DP) has become the gold standard in quantifying the risk and providing formal guarantees against membership attacks (Nasr et al., 2021
Informally, Differentially Privacy implics that adversary learns almost the same thing about individual data point independent of their presence absence in the training data set. More formally, DP is defined as follows. Two datascts D, D' are called neighboring datasets if one is obtained from the other by deleting one data point.
Definition 1.1 (Differential Privacy Dwork et al. 2006b a ). A randomized algoritlem A is (e, -)ddiffrrenlialy private if, for any pair of neighboring datasets D and D', and for all events S in the output range of A, we have
Pr(A(D) E ES] S e Pr[A(D') ES]+
where the probability is over the randomness of A.
In the field of deep learning, Differentially Private Stochastic Gradient Descent (DP-SGD) (Song et al., 2013 Bassily et al.) 2014 Abadi et al. 2016 is the most commonly used method for training models with DP guarantocs. While DP-SGD a fairly general algorithm, a naive application can suffer from several computational challenges. To make matters worse, the gap between performance of a model with and without privacy typically widens as the model is made larger. This stands as significant. obstacle to wider adoption and deployment of large practical models with privacy guarantees.
For the problem of Image Classification, several works have shown that transfer learning can be a very effective strategy in order to improve privacy-utility trade-off when formal privacy guarantees are required (Kurakin et al. 2022; Mehta et al. 2022: De et al., 2022). In this setting, a model is first pre-trained with -enniitns data without privacy guarantees, then fine-tuned on "sensitive' dataset over which a formal privacy guarantee required. Similar to previous works, we simulate several publicly available image classification benchmarks (like ImageNet) as "sensitive" datasets.
Interestingly, Mehta et al. (2022 De et al. [2022) observe that privately fine-tuning just the last layer of a pre-trained model (using DP-SGD) leads to state of the art results in the ImageNet-lk dataset. This is quite fortuitous, since privately ine-tuning the full model typically introduces siguificant computational challenges. We build on this observation, and perform a comprehensive exploration of various design parameters, including the choice of loss function and optimization algorithm, beyond simple DP-SGD. In this restricted setting of learning a single layer privately using features extracted from a pre-trained model, more sophisticated methods, such second-order methods, are computationally viable. Our main contributions are as follows:
Somewhat surprisingly, we find that linear regression solved using DP Least Squares performs much better than logistic regression solved using DP-SGD, especially at lower epsilons.
Postulating that the benefits largely stem from the use of second-order information in the least squares solution, we further explore using Newton's method to solve logistic regression. While Newton's method outperforms lincar regression in the non-private setting, with privacy constraints, largely because sanitizing the Hessian with logistic regression requires adding far more noise than in linear regression, where part of the Hessian can be shared across all classes.
find that it still performs worse
To combine the benefits of both, we introduce method which we call Differentially Private SGD with Feature Covariance (abbreviated as DP-FC) where we simply replace the Hcssian in Newton's 's method with sanitized Feature Covariance. Using Feature Covariance instead of Hessian allows us lo make use of second-order information in the training procedure while sharing it across classes and itcrations, which greatly reduces the amount of noise that nceds be added to sanitize it. This allows us to continue using logistic regression, which performs better in non- non-private setting, while benefiting from improved privacy-utility trade-off seen with linear regression in the private setting.
2

---

Published in Transactions on Machine Learning Research (04/2023)
With DP-FC, surpass previous state of the art results considerably on 3 image classification benchmarks, namely ImageNet-1k, CIFAR-10 and CIFAR-100, just by performing DP fine-tuning on features extracted from pre-trained model, see Table ! for summary. Consistent with previous works, we also find that performance increases as the pro-training dataset and the model are made larger.
Dataset
Epsilon
Previous SOTA
Accuracy
Method
Epochs (= Steps)
Pretraining DS
0.01
95.55 (0.98)
DP-LS
JFT
0.05
97.81 (0.22)
DP-FC
10
JFT
0.1
98.23 (0.12)
DP-FC
10
JFT
0.5
98.73 (0.04)
DP-FC
10
JFT
C-FAR-10
1.0
96.7
98.80 (0.05)
DP-FC
10
JFT
2.0
97.1
98.80 (0.03)
DP-FC
10
JFT
4.0
97.2
98.83 (0.02)
DP-FC
10
JFT
8.0
97.4
98.84 (0.02)
DP-FC
10
JFT
00
98.90 (0.00)
LS
JFT
0.01
76.96 (0.08)
DP-LS
1
121K
0.05
78.31 (0.33)
DP-LS
JFT
0.1
80.57 (0.26)
DP-LS
JFT
0.5
84.82 (0.38)
DP-FC
10
JFT
CIFAR-100
1.0
83.0
87.85 (0.14)
DP-FC
10
JFT
2.0
86.2
88.77 (0.16)
DP-FC
10
JFT
4.0
87.7
89.51 (0.17)
DP-FC
10
JFT
8.0
88.4
89.78 (0.08)
DP-FC
10
JFT
90.60 (0.00)
LS
1
JFT
0.01
81.99 (0.08)
DP-LS
JFT
0.05
83.74 (0.10)
DP-LS
JFT
0.1
84.28 (0.06)
DP-FC
JFT
0.5
86.04 (0.06)
DP-FC
10
JFT
ImageNet-1K
1.0
84.4
86.78 (0.07)
DP-FC
10
JFT
2.0
85.6
87.34 (0.04)
DP-FC
10
JFT
4.0
86.0
87.70 (0.03)
DP-FC
10
JFT
8.0
86.7
88.02 (0.08)
DP-FC
10
JFT
88.90 (0.00)
Newton
10
JFT
Table 1: Compilation of our best private Top-1 test accuracies. We report median and standard deviation across training runs with different seeds. All number are SOTA across all epsilons to the best of our knowledge. Previous state of the art for CIFAR-10 and CIFAR-100 were reported form Bu et al. (2022a) and for ImageNet- from De al. p022). We denote all sanitization steps including clipping. We set to 8 10-7 for ImageNet-1k, and 1 10-5 for CIFAR-10 and CIFAR-100. Interestingly, in the non-private setting, most previous works (including Zhai et al. 2021 use Linear Regression when finetuning from features but we found that Logistic Regression (solved using Newton's 's method) performs much better and leads to an impressive 88.9% accuracy when finetuning just the last layer. To put this perspective, this is only 1.1% less than the current state of the art non-private accuracy of 91% on ImageNet- lk (Yu et al. 2022b). We report extensive hyperparameter details in the appendix for reproducibility of our results.
be oo for non-private setting where we turn off
2 Private Learning from Features
this section, we describe the details of optimization strategies we considered, and state the privacy guarantces for each of them.
Given data set. D {(11, (1), (Xn, }n)), we optimize the function L: RRxd IR defined as follows
3

---

Published in Transactions on Machine Learning Research (04/2023)
(()) _ {(10). vis) =1
(1)
where n the number of examples, m the number of classes, 0 Rxdd the weight matrix to be learned, 2t is the feature vector for example i, Mij is the label of example i and class j, and e is a convex loss function. We assume that Vij E [0, 1] for all i.j. Additionally, we also use the short hand E(0; (x), = Dj= e((8g, Vey) where helpful in order to simplify the notation.
In the case of learning from features extracted from a pre-trained model, the feature vectors æ; are last layer features. Further, unless otherwise specified, we will assume to be the logistic loss, i.e. (ingittic y) = ly (2) (1 - y) log(1 - (z)) where o(z) = Ite Finally, we will also assume that each step of optimization considers the whole batch, which greatly simplifies both the privacy analysis of algorithms and the experiments. Rest of this section includes descriptions of several iterative solvers of this minimization problem, both in non-private and private settings. Some of these methods rely the fact that we are only interested in fine-tuning just the last layer, while others are more general. In the privacy analysis, we use zCDP (zero- Concentrated Differential Privacy) (Bun & Steinke 2016), but we state our empirical results always with final privacy guarantee in (e, 6)DD terms, as done in previous works.
2.1 DP-SGD
Arguably the most popular approach to solving the above minimization problem in the non-private setting is Stochastic Gradient Descent (SGD). In the full-batch sotting, at every itcration, SGD performs the update:
1 gt
o++1 0₂ -1e9t
(2)
n
where denotes the learning rate used for for iteration t.
DP-SGD a private variant of this algorithm and the baseline in all our experiments. Computationally, in order bound the sensitivity of each training example, Abadi et al. (2016) suggest computing a gradiemt for each example separately and clipping each to maximum norm of (a user-specified hyper-parameter):
g. +- 9t + N (0,
9 -
clip (Ve(0s (z.,
(3)
where clip(v) = min {I,
C
[ella]
After summing the elipped example gradients, a noise vector sampled from a Gaussian distribution with standard deviation oC' is added, where a a parameter that determines the privacy guarantee via the Gaussian mechanism.
As shown in Algorithm 4 once the gradient has been sanitized, are free to use it to accumulate statistics (e.g. first or second moment estimates) which are typically useful with the optimization process. Algorithm 4 presents a generalized version of DP-SGD where the gradients get processed in the traditional DP-SGD fashion, and are then passed to first-order optimizer as an input. This lets us instantiate DP versions of well-known first-order optimizers like SGD, Momentum and Adam. We employ DP-Adam in all our experiments. Finally, we omit the privacy analysis for the DP-SGD baseline since standard, but we do include in the appendix the details of the implementation used for translation from privacy constraints noise scale.
2.2 De-Newton
Since the optimization problem under consideration is a relatively simple convex problem, second-order DP algorithms can be viable. We first consider privatized version of the popular Newton's method and denote
4

---

Published in Transactions on Machine Learning Research (04/2023)
it DP-Newton. To control the sensitivity of each training example, naive approach is to compute per-example Hessians and clip their norm, in a similar fashion to example gradient clipping in DP-SGD. But even in our last-layer fine-tuning setting, this can be prohibitively expensive. For instance, training on features extracted from ViT-G for ImageNet-1k 1k fine-tuning, Hcssian tensor (in a block diagonal form) is of size [1000, 1664, 1664] with approximately 2.8B entries. In order avoid instantiating the Hessian for every training example, we instead choose to clip the feature vectors 24, then translate the clipping threshold into bounds on the Hessian and gradient norms. This is summarized in Algorithm |
Algorithm Differentially Private Newton's Method
Require: Data set. D - ((I1,,1,, .M (In, with (ri, Wi) E D, loss function: e R x R + R, regulariza- tion coefficient X, learning rate 7, clipping norm: C, number of iterations: T, noise multiplier: a, bound
on the second derivative of the loss: Bi
Clip all features: 2, clip(z;) for all i E {I,..., n where clip(u) min {
c Tolaa }
2: Randomly initialize On-
3: for = I....,TT , T do for j -....,m do
9t.j + Ei=1 Vi S. Mig).
6:
Hrj t E V Ri, Biy) + AI
7: Dt,J + g + W (0a, (cC)m)2) where N (oa. ( (0 indicates a d-dimensional vector cach of
whose coordinates is an iidd Gaussian with standard deviation oCvm/n.
His - Hos + N Oaxd, (B8y vm)) 2) where N [baxd: (18y vm 2)
8:
indicates a d x matrix
each of whose coordinates is an i..dd Gaussian with standard deviation oBnC²² (3nC2vmm/n 0441 + Pos - 1H Hus
9:
10: end for
11: end for
12: return 0r.
Theorem 2.1 (Privacy guarantee for Algorithm Suppose e is twice differentiable, and that for all (z, v,,)) (2((z < 1 and (z, v)1 < 8H. Then, Algoritho satisfies FDCCDPP
Here ll and l" denote the first and second derivatives of with respect to its first argument. We choose the bound the first derivative to be 1 without loss of generality (via scaling of E). Note that the assumptions of the theorem are satisfied for the sigmoid cross-entropy loss (i.e. logistic regression) with 3H 1 Indeed, (loogiati (2, (2, ly - (2)), and 1%giitti y) af2) * (1 - a(2) where ((2) = 1 Ite-z- The first expression is bounded by (since y, (2) [0, 1J) and the second expression is bounded by 1.
For the squared loss f(z, y) [ y)?; notice that (non-private) Newton's method reaches the solution in single step, regardless of where it is initialized. Therefore it suffices to initialize Algorithm [ at 0g = 0, and run it for a single step (T'=1) = 1). When initialized at 0 = O, we have 2 = 0 and the assumptions of the theorem are satisfied (first and second derivatives are bounded, with 3 = 1).
Proof. The crux of the proof is to ensure that Lines 7 and individually satisfy 2moz z -CDPP The rest of the proof is just simple composition of zCDP Bun & Steinke (2016) across classes and T iterations. To see this, first let D and D' be neighboring datasets and let. (x,y) be the differing datapoint between D and D'.
Now, notice that in the computation of gr, we have the following: [19:)(D) St., (D')l = (10 a), ).l. Thus, since | S C, and (e' 2).8,) 1:
-D) - gt)( c
(4)
From equation 4 it immediately follows that the computation of gt, for each {O..... T 1) and
each j (1,...,m) m) satisfies CDPP Now, moving to the sensitivity of H. in Line 8 have
5

---

Published in Transactions on Machine Learning Research (04/2023)
HD Ht,r(D') (L,( Uy))) Then, since ||ā||| C and [e"( 8/y1 S 3R, we have:
-D, S BgC²,
(5)
and equation 5 immediately implies that the computation of Haj for each t E {O....,T- - 1} and j E
[l....m) m) satisfies 2mg?
2m ZCP. This completes the proof.
2.3 DP-LS
One drawback of DP-SGD and DP-Newton is that each training example i contributes to the gradients (resp. Hessians) of all classes j E {l...., m), so the sensitivity (and hence the scale of required noise) increases with the number of classes. This is visible in Algorithm | where the amount of noise added for privacy protection scales with v (Lines 7 and B]. When the number of classes is large, this reduces signal-to-
ratio and can hurt quality. In this section, we develop a method that aims to address this issue. We take inspiration from the matrix factorization literature, in which one can separate the loss function into the contribution of positive and negative classes. We assume that labels are binary (Vi {0, 1}), and we consider the following quadratic loss function:
((0)
Σ
n
m
a Vig (x - vis)² 2 (x, 0,)
+
X 2 10,111
1=1
in other words, we take {(2,y) (v((2 y) a22)). The first term fits the positive labels (notice that this term vanishes when = 0), while the second term fits negative labels, and a is hyper-parameter that trades-off the two terms. This formulation was studied by Hu et al. (2008) and enjoys remarkable empirical
success Koren & Bell (2015). It turns out that this mothod also well-suited for privacy, as we shall discuss below.
By expanding the quadratic terms, we can write the loss as
[(0) = 2
E j-1
(0] - 20Jbb by + +a0] GO; + 10] 0,)
where for all j E {1....,m) m}
n
A₂ =
Σ
bj =
Σ E.y
G
Σ
x
(6)
i:g1y m1
The exact solution is then given by
0 [A; + aG - ATJ by.
Notice that the solution 8, for class j depends on class-specific statistics (the matrix Aj and the vector by), as well as the global quantity G. Algorithm 2 computes a private estimate of the solution by adding Gaussian noise to each of G, Ay, by (Lines 2 4 and respectively). then solving the linear system using the noised statistics (Line 6). This is variant of the popular sufficient statistics perturbation algorithm for DP linear regression. One crucial observation is that the noisy version of G' is only computed once (Line 2), and reused for all classos. This allows to control the scnsitivity of the solution w.r.t. each example; indeed, suppose example i only has one positive class jo, then that example only contributes to G, Aju, and bjo- In particular, the sensitivity of the solution (and hence the amount of noise we need to add) does not scale with the total number of classes, only with the number of positive classes per example. This represented by the parameter k in Algorithm
for singlo-class classification tasks, and even in multi-class tasks, we typically have k m). We now give the formal privacy guarantee:
2 (k is oqual
Theorem 2.2 (Privacy guarantee for Algorithm 2 Algorithm satisfies 202
3_ .CDP.
Proof. First, let D, D' be neighboring data sets that differ in the data point (z, y), and let G(D) be the global statistic in equation b computed data set D. Then [G(D) G(D)) | [2, and

---

Published in Transactions on Machine Learning Research (04/2023)
since ||ãl|2 I C (due to clipping), [[G(D) - G(D') I C2, thus the computation of G (Line 2 is 2n xCDP. Now moving to the computation of Ay: let A(D) be the concatenation of all the class-specific statistics, i.e. A(D) [Ai |Am (D)]. Notice that [A, (D = || 122 if yj = and 0 otherwise. Since by assumption, the number of positive classes per example is bounded by k, we have that [[A(D) - = S V [[,, which bounded by vEC due to clipping. Therefore the computation of all A, combined (Line 4 is -CDPP Finally, by a similar argument, we have that [[b(D) - b(D)) S VkC, and the computation of all by combined (Line 5) is z--DCDP
3 By simple composition of zCDP Bun & Steinke 2016), the algorithm is
26 2-DP.CDP
Algorithm 2 Differentially Private Least Squares
Require:
Data set D {(x1, y1), (In, 8n)} with (2i, Mi) E D, weight coefficient a, regularization coefficient X, maximum number of positive classes per example: k, clipping norm: C, noise multiplier: ø. Clip all features: 2, clip(z;) for all i [I....,n..
2: G + Σi= शे +N (faxa: (oC²)²), where N (ōaxa: (o(²)² indicates a d x d-matrix, each of whose coordinates is an ii.d. Gaussian with standard deviation oC
3: for = I,..., m do
4: A₁ r E Emsr,= T. + N (baxel (ovkCz)²)
5: bj - Emmryi 3i N (ōa, (oVKC))))
6: 0 + [A₁ aā + 'b;
end for
return 0.
2.4 DP-FC
From our early experiments, we found that Newton's method with logistic regression performs better than least squares linear regression in non-private sctting. But in the private setting, DP-LS outperforms DP- Newton, especially for lower values of epsilons. Notice that both are second-order methods, and both rely on estimating and inverting the Hessian (Line in Algorithm | and Line t Algorithm 2 the main advantage of DP-LS is that the private hessian computation does not have to composed over classes or iterations. In this section, in order to mitigate the tradc-off between the two methods, we introduce method called Differentially Private SGD with Feature Covariance (DP-FC) which leverages covariance of features to make use of second-order information without paying the cost of composition over classes or iterations. Indeed, since feature covariance neither depends on the model parameters nor the prediction, it can be shared across both classes and iterations. This described in Algorithm 3 The method can be interpreted as DP-SGD with preconditioning (where the approximate feature covariance G is used preconditioner). We empirically observe that this leads to greatly reduced sensitivity compared to DP-Newton and significant improvements in overall metrics across all values of epsilons we tried.
Theorem 2.3 (Privacy guarantee for Algorithm 3 Algorithm satisfies
(T+1) 20a
-2DDP
Proof. The crux of the proof is to ensure that the computation of Ō (Line 3) and ūr (Line 7 individually satisfy -DDPP The rest of the proof is by simple composition of zCDP Bun & Steinke (2016) (once for the G computation, and T times for the ÿ computation).
Let D and D' be neighboring datasots and let (x, y) be the differing datapoint between and D'.
In the computation of gt, we have the following: |g: (D) - ge (D')
(Vlip (z, v))l Cy. It immediately follows that the computation of ÿ. for each E {0.... ........ T - 1} satisfies 255 -CDP. Now, moving to the sensitivity of G. We have [[G(D) - G(D')I = || || S C2, since ||اا|| s Cg. It immediately follows that & satisfies -DPP, which completes the proof.

---

Published in Transactions on Machine Learning Research (04/2023)
Algorithm Differentially Private SGD with Feature Covariance (DP-FC) Method
Require:
Data set D = ((11,81)) (En, Bn)} with (I,, 14) D, loss function: é: Rmxd x R - R, learning rate 1), clipping norms: CG and Cg. number of itcrations: T, noise multiplier:
1: Clip Teatures for covariance computation: Iz + clip(r;) for all E (I,..., where clip(a)
x min {1, Cc [r] [
Compute the feature covariance: G= G (t,t),
G + (Daxa, ( (2a)) + AI
Randomly initialize 0u-
for 6 = 1...., do
6: 9t + c clip( (x)) where clip(g) = min {I, l }
7: 0t + 9t + +N (Gs, ( (oCe) )2)
01-1 + 0. G
end for
10: return 0r.
3 Empirical Results
Epsilon
Pretraining
Method
Epochs
NP
0.01
0.05
0.1
0.5
1.0
2.0
4.0
8.0
DP-LS
75.5
71.2
71.5
71.5
73.0
73.6
74.1
74.6
75.0
72.7
70.2
71.3
71.3
71.4
71.6
71.7
72.0
72.2
DP-Newton
10
78.2
66.5
71.0
71.4
71.9
71.9
72.3
73.0
74.2
ImageNeet22IK
72.9
71.9
72.1
72.5
72.9
72.9
72.9
72.9
72.9
DP-FC
10
77.1
71.8
71.9
72.1
75.4
76.3
76.8
77.0
77.1
73.8
39.1
54.2
63.3
68.3
DP-Adam
10
76.1
22.1
52.9
62.3
66.6
69.5
71.0
100
78.3
47.2
59.1
65.8
68.8
70.4
DP-LS
87.5
82.4
83.8
84.1
85.8
86.2
86.4
86.6
86.7
85.9
77.8
80.3
81.0
82.9
83.6
84.0
84.5
84.9
DP-Newton
88.9
76.0
79.7
80.1
81.8
82.9
83.1
84.7
85.3
JFT
85.8
82.1
83.8
84.3
85.1
85.4
85.5
85.6
85.6
DP-FC
10
88.4
81.0
83.1
83.7
86.1
86.8
87.4
87.8
88.0
84.8
68.7
74.0
82.1
83.7
84.4
84.8
84.8
DP-Adam
10
87.3
70.5
75.6
83.4
84.9
85.6
86.3
86.7
100
88.6
49.7
64.4
78.3
81.5
83.9
85.4
86.3
Table Comparison of Top-1 test accuracies when privately fine-tuning on Imagenet-IK. We denote accuracy < 20% with the symbol '.2 When pre-trained with JFT, we observe that DP-FC performs best for epsilon values ranging from 10.1, 8.0] whereas DP-LS is best for even lower epsilons. In the case of pre-training with ImageNet-21k, we find that DP-FC (10 epochs) outperforms all other methods across the board. For reference, we note that best non-private accuracy on ImageNet- 1k is 91% (Yu et al. 2022b). We used ViT-B/ 166 model for pre-training with ImageNet-21k and ViT-G/14 14 for JFT.
Datascts. We use 3 datasets for private finetuning, namely 1) ImageNet-1k (Deng et al. 2009 with 1k classes and 1.3M images 2) CIFAR-10 and 3) CIFAR-100. We also refer these as the private dataset for which want privacy guarantee. For pre-training, rely JFT-3B, ImageNet-21k and ImageNet-1K

---

Published in Transactions on Machine Learning Research (04/2023)
Epsilon
Pretraining
Method
Epochs
Non-Private
0.01
0.05
0.1
0.5
1.0
2.0
4.0
8.0
DP-LS
91.3
81.1
83.7
84.3
86.3
87.7
88.7
89.6
90.3
91.0
77.4
79.6
80.5
84.2
85.7
87.1
88.4
89.2
DP-Newton
10
91.6
79.7
81.4
82.7
86.1
87.5
89.0
88.6
90.2
ImageNet-
91.0
79.0
81.5
83.1
86.6
88.0
88.9
89.7
90.3
DP-FC
10
91.1
81.2
84.7
86.5
89.5
90.4
91.0
91.1
91.1
52.6
74.3
76.7
78.2
77.6
77.8
DP-Adam
10
82.3
56.4
78.3
76.7
82.0
82.6
82.4
82.2
82.7
100
87.5
40.9
63.8
72.3
83.6
87.4
87.6
DP-LS
96.5
94.5
95.2
95.4
95.8
96.0
95.5
96.2
96.3
96.5
94.3
94.9
95.2
95.6
95.7
96.0
96.1
96.2
DP-Newton
10
96.5
94.8
95.2
95.4
95.6
95.9
96.0
96.2
96.2
ImageNet-21K
96.6
94.6
95.2
95.5
95.9
96.0
96.2
96.3
96.3
DP-FC
10
96.6
94.8
95.6
95.8
96.1
96.3
96.5
96.5
96.5
95.2
90.0
94.7
95.1
95.1
95.1
95.1
95.1
95.2
DP-Adam
10
96.1
83.8
94.9
95.5
95.8
95.8
95.8
95.9
96.0
100
96.5
64.0
90.0
93.3
95.5
95.7
95.9
96.1
96.2
DP-LS
98.9
97.4
98.2
98.4
98.4
98.6
98.8
98.8
98.9
98.9
94.1
96.5
97.2
98.1
98.3
98.5
98.7
98.8
De-Newton
10
98.9
95.9
97.5
97.9
98.5
98.6
98.4
98.8
JFT
98.9
95.2
97.6
97.9
98.5
98.6
98.8
98.8
98.9
DP-FC
10
98.9
97.3
98.2
98.4
98.8
98.8
98.9
98.9
98.9
97.5
93.5
97.0
97.5
97.5
97.6
97.6
97.6
97.6
DP-Adam
10
98.7
87.6
97.7
98.1
98.5
98.6
98.7
98.7
98.7
100
98.9
79.2
93.2
96.3
98.3
98.6
98.6
98.8
98.8
Table Comparison of Top-1 test accuracies when private finetuning on CTFAR-10. We denote accuracy S 20% with the symbol - Similar other datasets, DP-FC (10 epochs) outperform all other methods almost across the board with single exception of epsilon of 0.01 when pre-training with JFT were DP-LS performs slightly better. For reference, note that best non non-private accuracy CIFAR-10 10 is 99.5% (Dsoovitskiy ct al. 2021). We used ViT-B/16 model for pre-training with ImageNet-21k and Imagenet- IK, and ViT-G/14 for JFT.
(as done in Zhai et al. 2021 For JFT, we intentionally chose slightly smaller version of the dataset i.e. JFT-3B instead of JFT-4B, enabling us to exactly follow Zhai et al. (2021) and thus lowering the risk of the project. Also note that, as done in recent works, none of our finetuning datasets in reality are sensitive datasets: we are only simulating a public/private private dataset split only for demonstration purposes (Kurakin 2022 De al. 2022). The JFT datascts are not publicly available but have been used extensively as pre-training dataset in the non-private setting to obtain state-of-the-arl results Dosovviskiy et al. 2021 Brock et al. 2021 Tolstikhin et al. 2021 Zhai al.) 2021). Similar to Mehta (2022), ), to make sure that our simulated "public" and "private" datascts capture & practical scenario, we carefully de-duplicate our pre-training datasets w.r.t. all splits of our finetuning
ct al.] 2022; Mchta et al.
ct al. [002)] : De et al.
datasets [Kolesnikov et al. 2020 Dosovitskiy et al. 2021 More details about this process can be found in the appendix.
Model variants. We evaluate the transfer learning capabilities of the Vision Transformer (ViT) (Dsovviskii et al. 2021 model family in our study. We follow the standard notation indicate the model size and the
9

---

Published in Transactions on Machine Learning Research (04/2023)
Epsilon
Pretraining
Method
Epochs
Non-Private
0.01
0.05
0.1
0.5
1.0
2.0
4.0
8.0
DP-LS
71.9
49.2
51.8
53.9
57.6
60.0
62.5
65.4
67.5
68.8
44.6
48.6
49.3
50.2
51.2
54.4
57.2
59.5
DP-Newton
10
72.1
36.4
48.2
49.8
50.7
51.8
53.5
56.1
59.5
ImageNet-
68.7
49.2
50.2
52.1
58.3
60.8
62.8
64.6
66.1
DP-FC
10
71.4
48.9
49.7
53.7
61.4
64.9
68.2
69.8
70.4
52.1
26.0
51.2
51.7
52.2
DP-Adam
10
57.3
20.0
33.5
51.2
54.4
55.6
57.5
56.3
100
67.9
40.2
58.4
63.7
DP-LS
83.9
77.2
77.7
78.1
79.8
80.7
81.4
81.9
82.4
83.0
76.5
77.2
77.2
77.6
78.3
79.0
79.5
80.5
DP-Newton
10
83.0
73.6
77.4
77.8
78.3
78.9
79.6
80.4
81.4
ImageNet-21KK
83.0
77.1
77.5
78.2
80.0
80.9
81.6
81.9
82.4
DP-FC
10
84.3
77.1
77.1
78.5
81.6
82.7
83.3
83.8
83.9
79.9
27.6
67.0
72.7
78.3
79.5
79.7
79.7
79.7
DP-Adam
10
82.0
55.7
67.5
78.6
80.7
81.5
81.5
81.6
100
84.6
29.7
45.2
71.3
76.2
79.5
81.1
82.2
DP-LS
90.6
74.9
80.3
82.5
85.5
87.7
88.4
88.9
89.9
73.1
72.6
73.4
78.5
80.9
82.8
84.6
85.9
De-Newton
10
89.9
69.6
75.7
76.7
77.6
80.0
82.9
85.4
JFT
89.9
73.5
78.6
81.0
85.2
86.7
87.7
88.3
88.6
DP-FC
10
90.1
72.1
75.9
79.0
86.2
88.1
89.0
90.0
90.1
83.5
27.9
61.8
71.3
79.7
82.1
83.4
83.5
83.5
DP-Adam
10
88.2
21.9
60.2
69.7
81.9
83.9
86.2
86.8
87.8
100
90.0
29.4
50.5
73.7
78.7
83.1
86.1
88.0
Table 4: Comparison of Top-1 test accuracies when private finetuning on CTFAR-100. We denote accuracy < 20% with the symbol --; Similar to other datasets, DP-FC outperforms all other methods for moderate privacy budgets whereas DP-LS performs slightly better for very strict privacy guarantees depending the pre-training dataset. For reference, note that best non-private accuracy CIFAR-100 is 96.08% (Foret ct al. 2021). We used ViT-B/16 model for pre-training with ImageNet-21k and Imagenet- and ViT-G/14 for JFT.
input patch size, for example, ViT-B/32 means the "Base" variant with 32x32 input patch size. Note that for ViT, compute requirements scales up as we reduce the patch size. We obtained features from ViT-G/14 model pre-trained on JFT-3B (Zhai et al. 2021] and VIT-B/16 pre-trained on ImageNet-21k and ImageNet-Ik. [Steiner et al. 2021).
Next, we present our main of private finc-tuning results and core observations on all datasets, namely ImageNet-1k (Table 2), CIFAR-10 (Table 3) and CIFAR-100 (Table
3.1 Better pre-training continues to improve private fine-tuning performance
We evaluate private fine-tuning performance on features extracted from pre-trained models sizes i.e. VIT- G/14 and VrT-B/16, pro-trained on 3 different datasets, namely JFT-3B, ImageNet-21K and ImageNet-IK, We do this quantify the extent to which the representation quality is improved by increasing pre-training dataset in combination of model size. As shown in Figure the model size and pre-training dataset size is
10

---

Published in Transactions on Machine Learning Research (04/2023)
100
100
100
90
Top-1 Accuracy
96
Top-1 Accuracy
80
70
JFT
60
121K
11K
50
50
0.1 0.5 1
2
0.1 0.5
0.1 0.5
2
4
8
(a) CIFAR-10
(b) CIFAR-100
(c) ImageNet-1K
Figure 1: Comparison of top-1 accuracies with private fine-tuning using DP-FC method on all 3 datasets across a range of epsilons. We observe that better pre-training helps even more for lower values of epsilon (stricter privacy guarantec).
increased, we contimue to see improvement in downstream private fine-tuning performance for all 3 datasets we consider. In addition, we make following observations from our results.
First, better pre-training can help even more at stricter privacy budgets. As shown in Figure I for both CIFAR-10 and CIFAR-100, when comparing features extracted from ViT-B/16 pre-trained with ImageNet-1KK and features from ViT-G/14 pre-trained with JFT, the improvement in performance at is larger than = 8. We see similar trend at even lower epsilons in Table 3 and Table
Second, features extracted from off-the-shelf pre-trained models can suffice for DP. Except for the fact that we deduplicate all splits of our pre-training dataset with our fine-tuning dalasets, we use the exact same procedure used to pre-train large vision models. This suggests that, in practice, a recipe where features extracted from large off-the-shelf vision model used to privately fine-tune a classifier can be quite effective for DP performance. Since there is no need for a special pre-trained model for use in DP, this considerably reduces the cost of training private image classifiers.
Lastly, private fine-tuning of high quality features closes the gap between private and non- private performance considerably. In the non-private setting, Zhai et al. (2021) obtain an impressive 90.45% top-1 accuracy by fine-tuning the whole ViT-G/14 model on Tmage/et-1I dataset. We observe that even by fine-tuning just the last layer, we can obtain as much as 88.9% top-1 accuracy on ImageNct- from the same sized model. Thus the marginal benefit of fine-tuning the whole model is <2% even in the non-private case. In the private case, fine-tuning of pre-extracted features with DP-FC at E 8 leads to state of the art 88% top-1 accuracy. On ViT-G/14, this represents 2.5% difference between best non-private and private performance. This is also just 3% below the best non-private accuracy of 91% on ImageNet-IK
Yu
et al. 2022b
3.2 Better optimizers improve privacy-utility trade-off
We observe that the choice of optimizer can have a significant impact on the privacy-utility trade-off.
First, optimizers that work well in the large batch regime are better for private training. Even in the non-private setting, it known that batch size is intimately tied to the optimization procedure and can lead to suboptimal
of resources if increased beyond certain point while keeping number of epochs constant (Goyal et al. 2017 You et al.] 2017 2019). The maximum batch size which can be used without jeopardizing the compity-comppte trade-off is also heavily dependent the choice of optimizer (Zhang et al, 2019). Though in the private setting, several works have observed that incrcasing the batch size leads to improved privacy-utility trade-off (Dormann et al. 2021 Li et al., 2014 Hoory et. al. 2021). It. is also empirically observed that the utility in large batch regime can be further improved by leveraging optimizers which work well in large batch regime, such as DP-Adam or DP-LAMB (Anil al.) 2021 Mchta al., 2022 Bu et al. 2022a)). Second-order methods such as LS or Newton are also well-suited the large-batch regime (they were developed and are typically used in the full-batch setting).
11

---

Published in Transactions on Machine Learning Research (04/2023)
Second, optimizers with faster convergence rates are advantageous in the private setting, because they can reduce the number of epochs required for convergence. Indeed, in typical privacy analysis of iterative methods, the analysis works by composition over iterations, which means that the privacy penalty scales with the number of visitations of the data. Thus, in addition to the computational benefit, any improvement in convergence rate directly helps training with privacy because it requires less noise to be added under the same privacy constraints. We empirically observe that this reduction in noise the case of DP-FC, in combination with sharing of the privatized Feature Covariance across classes and iterates, allow us to obtain better results with 10 epochs compared to even when using 100 epochs with DP-SGD.
4 Related Work
Differential privacy (Doork et al.) 2006bb is a popular method to guarantee privacy in a quantifiable way in many data-driven applications. To achieve differential privacy in machine learning, tasks practitioners commonly train models with privatized variation of gradient descent, called DP-SGD (Song et. al., 2013 Bassily et al. 2014; Abadi et al., 2016).
Despite theoretical guarantees, differentially private training has two major drawbacks which limits its wide adoption. First of all, DP-SGD can be much slower compared to regular SGD, if implemented naively. To address this, several deep learning architectures offer to vectorize the computation (Subramani et al.) 2020) or it is sometimes possible to bound the sensitivity of each example without calculating the gradient for every example separately, leading to a dramatic cost-reduction both in terms of memory and compute (Goodfellow) 2015; Li et al., 2022b Bu et al., 2022a ap. Although, in our work this cost is minimal since we only train the last layer. We do note that computational burden can still be an issue for optimization schemes like DP-Newton where we had to resort to feature clipping in order to produce bounds on sensitivity of the
gradient and the hessian.
In addition to the computational cost, model trained with differentially privacy usually suffer from so-called "utility loss", which means that accuracy (or any other quality metric) is worse (and sometimes significantly worse) compared to accuracy of non-private model (Dormann et al. 2021 Klause et al. 2022). Over the years, several lines of improvements have been proposed including adaptive clipping (Pichapati et al. 2019
Thakkar et al.] 2019 Bu al. 2022b Golatkar et al. 2022), param-efficient finetuning (Yu et al.) 2022a
Mehta et al. 2022 Bu et al. 2022c Cattan et al. 2022; Li ct al. 2022a) and even leveraging intermediate
checkpoints De et al. 2022 Shejwalkar et al.] 2022). One of the recent trends to improve utility of private models significantly involves various ideas related to transfer learning where previous works demonstrate
improved performance in the setting where we have access to large public or non-sensitive dataset of the
same modality as the private data (Kurakin et al. 2022: De ct al. 2022 Mchta et al. 2022 Tramèr & Bonch, 2021: Yu el. al.] [2022a] Li et al.] 2022b Kurakin et al., 2022 Hoory el al. 2021). b. Our work also leverages large pre-trained models in order to obtain high-quality features for private finetuning. In addition, similar to the works that focus on studying and reducing dimensionality of the model in the context of DP (Li et al.)
2021 Zhou et al. 2020), we focus solely on learning just the last layer privately, which can be seen as an implicit way to reduce dimensionality.
2022a Golatkar et al. 2022 Yu et al.) 2021 Zhang al.
In the context of differential privacy, several recent papers also advocate the use of large batch sizes in order to improve the privacy-utility tradeoff (Mehta et al., 2022 McMahan et al. 2018 Anil et al. 2021 Dormann let al. 2021 Hoory et al. 2021: Liu et al., 2021b Kurakin et al.] (022). ). Even though this work explicitly does not explore the alfect changing the batch size, the fact that we are able to obtain state of the art results in
the full batch setting may point to the effectiveness of large batch sizes in the context of DP. Further, our work also zooms in on differentially private linear and logistic regression. Several existing works have studied
differential private convex optimization (Chaudhuri ct al. 2011 Kifer et al. 2012 Song et al. 2013 Bassily
el al. 2014 Wu et al.] 2016 McMahan el al.] 2017 [Bassly by el al. 2019 Iyengar el al. 2019
Feldman et al.,
2020
Bassily et al. 2020 Song et al.] 2020 Andrew et al.) 2021
There is also a growing interest in the
spocial case of linear regression (Smith et al.
2017 Shhfffet 2019 Liu et al.] 2021a Cai et al. 2021 Varshney
ct al.
2022 Wang 2018) and even second order methods in the context of differential privacy (AvellaMMedina
et al.
2021 Chien et al. 2021 In this work, we illustrate empirically the extent. to which second order
12

---

Published in Transactions on Machine Learning Research (04/2023)
methods can help in DP. It would be quite interesting to see how other popular second order methods like Shampoo (Gupta et al. 2018 Anil et al. 2020) would fare in the context of DP.
5 Conclusion
In this work, we focus on private finctuning of image classification datascts using features extracted from a pre-trained model. Given that. finetuming just the features is significantly cheaper than fineluning the full model, we systematically explore optimization schemes which are perceived to be expensive in high- dimensional scttings. As illustrated on 3 finctuning datascts i.c. ImageNet-1lk, CIFAR-10 and CIFAR-100, find that DP-LS (Least Squares) perporrss DP-SGD with logistic regression, especially for lower values of epsilons. Given the intuition that 2nd order information may be the reason for superior performance of DP-LS, we also explore Newton's 's method with Logistic Loss. Noticing that the amount of noise required by Newton's method scales with the number of classes and iterations, we introduce an optimization scheme called DP-FC which replaces the hessian by the feature covariance matrix, that can be shared across classes and iterations. Using this insight, we demonstrate that it is indeed possible to get state of the results by just finetuning the last layer of pre-trained model with privacy constraints. We hope that our work significantly reduces the barrier in training private models.
References
Martín Abadi, Andy Chu, lan J. Goodfellow, H. Brendan McMahan, 1lya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proc. of the 2016 ACM SIGSAC Conf. on Computer and Communications Security (CCS'16), pp. 308-318, 318, 2016.
Galen Andrew, Om Thakkar, Hugh Brendan McMahan, and Swaroop Ramaswamy. Differentially private learning with adaptive clipping. A. Beygelzimer, Y. Dauphin, P. Liang, and Wortman Vaughan (eds.), Advonces in Neural Information Processing Systems, 2021. URL http::///pparrrree.nnnofooooooo
RUQ12WZR8
Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, and Yoram Singer. Scalable second order optimization for deep learning, 2020.
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Mamurangsi. Large-scale differentially private BERT. CoRR, abs/2108.01624, 2021. URL atttps//aaxxi..org/ab//2110/ 01624 01624
Marco Avella-Medina, Casey Bradshaw, and Po-Ling Loh. Differentially private inference via noisy optimiza- tion, 2021.
Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adversaries, 2022.
Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In Proc. of the 2014 IEEE 55th Annual Symp. on Foundations of Computer Science (FOCS), pp. 464-473, 2014.
Raef Bassily, Vitaly Feldman, Kunal Talwar, and Abhradeep Guha Thakurta. Private stochastic convex optimization with optimal rates. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelximer, Florence d' d'Alché-Bel Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Newral Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NewrIPS 2019, 8-14 December 2019, Vancouver, BC, Canada, pp. 11279 11288, 2019. URL httt://tppppeeeee caappers.iiis..o cctpspcr//pper//// 9306
Racf Bassily, Vitaly Feldman, Cristóbal Guzmán, and Kunal Talwar. Stability of stochastic gradient descent on nonsmooth convex losses. In NewrIPS, 2020. URL hntps:://proceedinggs wwwpeeedddinnn...ni...ee...eeeee www.iee..rrrppaaaeeeee ceurips..c//aperr2200/
13

---

Published in Transactions on Machine Learning Research (04/2023)
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, and Skye Wanderman-Milne. JAX: composable transformations of Python+NummPy programs, 2018. URL
Andrew Brock, Soham De, and Samuel L Smith. Characterizing signal propagation to close the performance gap in unnormalized resnets. In International Conference on Learning Represenlations, 2021. URL
Zhiqi Bu, Jialin Mao, and Shiyun Xu. Scalable and efficient training of large convolutional neural networks with differential privacy, 2022a.
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. Automatic clipping: Differentially private deep learning made easier and stronger. ArXiv, (bs/2206.07133,, 2022b.
Zhiqi Bu, Yu-Xiang Wang. Sheng Zha, and George Karypis. Differentially private bias-term only fine-tuning of foundation models. ArXiv, abs/2210.000366, 2022c.
Zhiqi Bu, XiaXia Wang, Sheng Zha, and George Karypis. Differentially private optimization on large model at small cost. ArXiv, abs/2220.000338, 2022d.
Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Theory of Cryptography Conference, pp. 635 658. Springer, 2016.
T Tony Cai, Yichen Wang, and Linjun Zhang. The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. The Annals of Statistics, 49(5):2825 2850, 2021.
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In Proceedings of the 28th USENIX Conference Security Symposim, 2019.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Kerbert-Voss Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting training data from large language models. In USENIX Security, 2021.
Yannis Cattan, Christopher A. Choquette-Choo, Nicolas Papernot, and Abhradeep Thakurta. Fine-tuning with differential privacy necessitates an additional hyperparameter search. ArXiv, abs/ Abs/2210.02156, 2022.
Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 12(Ma)) 1109, 2011.
Steve Chien, Prateck Jain, Walid Krichene, Steffen Rendle, Shuang Song, Abhradecp Thakurta, and Li Zhang. Private alternating least squares: Practical private matrix completion with tighter rates, 2021.
Christopher A. Choqueete-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only
membership inference attacks, 2020.
Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. Randaugment: Practical automated data augmentation with a reduced search space. 2020 TEFE/CVT Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Jun 2020. doi: (0.1009//ppr550008.22 202.0000000000033.0 URL
http://dx. doi. www.oorg......... 1ww9////////0099933338 2020. 00359
Soham De, Leonard Berrada, Jamie Hayes, Samuel L. Smith, and Borja Balle. Unlocking high-accuracy differentially private image classification through scale, 2022. URL
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largo-scale hicrarchical image database. In 2009 IEFE conference on computer vision and pottern recognition, pp. 248-255. Icce, 2009.
14

---

Published in Transactions on Machine Learning Research (04/2023)
Friedrich Dormann, Osvald Frisk, Lars Norvang Andersen, and Christian Fischer Pedersen. Not all noise is accounted equally: How differentially private learning benefits from large sampling rates. 2021 IEFF 31st International Workshop on Machine Learning for Signal Processing (MLSP), Oct 2021. doi: 10.1109/ mlsp52302. 2w1p5523222222..920 202199963777
URL http://dx www.dooo.o..ooo.oooooooo.o...o.. org/ 10. /ttp:/..1110mm1m12222.22o. 2tt.::23........99999..
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL httpps://openreeiiee wttp:/eewwe...eeeeoooooooreeddad
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Advances in EyppooooyyEURORRYPPP pp. 486 186-503, 2006a.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Shai Halevi and Tal Rabin (ods.), Theory of Cryptography, 2006b.
Ülfar Erlingsson, Vitaly Feldman, llya Mironov, Ananth Raghunathan, Kunal Talwar, and Abhradeep Thakurta. Amplification by shuffling: From local to central differential privacy via anonymity. In Timothy M. Chan (ed.), Proceedings of the Thirtieth Anmual ACM- SIAM Symposium on Discrete Algorithms, SODA 2019, San Diego, California, USA, January 6-9, 2019, pp. 2468 2479. STAM, 2019. doi: 10.1137/1. 9781611975452.151. URL https://doi: httpa/://doooooo/////// wro/o//g///// 11w./1.111119111 /11.1///19981111117688.11111
Vitaly Feldman, Audra McMillan, and Kunal Talwar. Hiding among the clones: A simple and nearly optimal analysis of privacy amplification by shuffling, arXiv preprint arXiv:2012. 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021. URL
2018. URL http://pwworriiio.ccccrrrrrraaaa
Roy Frostig, Matthew Johnson, and Chris Leary. Compiling machine learning programs via high-level tracing. pdf
Aditya Golatkar, Alessandro Achille, Yu-Xiang Wang, Aaron Roth, Michael Kearns, and Stefano Soatto. Mixed differential privacy in computer vision. 2022 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2022. doi: 10. (x000//cxpr52888.0222222
URL http:///x.ddi http://xxxxdi...oo www..od..ooo...oooooooooooooooo. org/
10. 1199///PR52288.....
Daniel Golovin, Benjamin Solnik, Subhodeep Moitra, Greg Kochanski, John Elliot Karro, and D. Sculley (eds.). Google Vizier: A Service for Black-Boz Optimization, 2017. URL htttp://ww.ket org/kdd2017
Ian J. Goodfellow. Efficient per-example gradient computations. ArXiu, abs/1510.01199, 2015.
Priya Goyal, Piotr Dollár, Ross B. Girshick, Picter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch SGD: training imagenet in hour, 2017.
Vineet Gupta, Tomer Koren, and Yoram Singer. Shampoo: Preconditioned stochastic lensor oplimization, 2018.
Shlomo Hoory, Amir Feder, Avichai Tendler, Sofia Erell, Alon Cohen, Itay Laish, Hootan Nakhost, Uri Stemmer, Ayclet Benjamini, Avinatan Hassidim, and Yossi Matias. Learning and evaluating a differentially private pre-trained language model. In Findings of the Association for Compulational Lónguislics: EMNLP 2021, pp. 1189, Punta Cana, Dominican Republic, 2021. URL httpp:://ccaatthooo/t//// org//02/ Findings-emmlp.1000
Yifan Hu, Ychuda Koren, and Chris Volinsky. Collaborative filtering for implicit feodback datascts. In Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, ICDM '08, pp. 263-272, 2008.
15

---

Published in Transactions on Machine Learning Research (04/2023)
Roger Iyengar, Joseph P Near, Dawn Song, Om Thakkar, Abhradeep Thakurta, and Lun Wang. Towards practical differentially private convex optimization. In 2019 TEEE Symposium on Securily and Privucy (SP), 2019.
Shiva Prasad Kasiviswanathan, Homin K. Tee, Kobbi Nissim, Sofya Raskhodnikova, and Adam D. Smith. What can learn privately? In 49th Annual IEEE Symp. Foundations of Computer Science (FOCS), pp. 531-540, 2008.
Daniel Kifer, Adam Smith, and Abhradeep Thakurta. Private convex empirical risk minimization and high-dimensional regression. In Conference Leorning Theory, pp. 25-1, 2012.
Helena Klause, Alexander Ziller, Daniel Rueckert, Kerstin Hammernik, and Georgios Kaissis. Differentially private training of residual networks with scale normalisation, 2022.
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): General visual representation learning. Lecture Notes in Computer Science, pp. 491-507, 2020. ISSN 1611-3349. doi: 00.1007/978-3-030-00 29
29. URL htttp://dx.doo..rrg..
Ychuda Koren and Robert Bell. Advances in collaborative filtering. Recommender systems handbook, pp.
77-118, 2015.
Alexey Kurakin, Shuang Song, Steve Chien, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward training at imagenet scale with differential privacy, 2022.
Mu Li, Tong Zhang, Yuqiang Chen, and Alexander J. Smola. Efficient mini-batch training for stochastic optimization. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledgee Discovery and Data Mining, KDD '14, pp. 661 670, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 47044999.9 doi: 10.1145 2222263332622666
URL stttps///doi http://0b..rrrr11..1444/3/333332 1145/2623330.. 2623612
Xuechen Li, Daogao Liu, Tatsunori Hashimoto, Huseyin A. Inan, Janardhan Kulkarni, Yin Tat Lee, and Abhradeep Thakurla. When does dilferentially private learning not sulfer in high dimensions? ArXie, abs/2207.00160) 2022a.
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In International Conference on Learning Representations, 2022b. URL hwt.eeeeeettttoorrrddddeBBe...
Xiyang Liu, Weihao Kong, and Sewoong Oh. Differential privacy and robust statistics in high dimensions, 2021a.
Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, and Yang Zhang. MI-doctor: Holistic risk assessment of inference attacks against machine learning models, 2021b.
H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. arXiv preprint 1r1i0:1710. 11100666, 2017.
H. Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. In International Conference on Learning Representations, 2018. URL https: //openreview.neteee. www.:neefooummiiddl...... 120b
Harsh Mchta, Abhradoep Thakurta, Alexcy Kurakin, and Ashok Cutkosky. Large scale transfer learning for differentially private image classification. 2022.
Iya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263 275. IEEE, 2017.
Iya Mironov, Kunal Talwar, and Li Zhang. Rényi differential privacy of the sampled gaussian mechanism, 2019.
16

---

Published in Transactions on Machine Learning Research (04/2023)
Milad Nasr, Shuang Songi, Abhradeep Thakurta, Nicolas Papernot, and Nicholas Carlin. Adversary instanti-
ation: Lower bounds for differentially private machine learning. 2021 TEEE Symposium Securily and Privecy (SP), May 2021. doi: 10.1109//p440001.2222.. 202100001...2..0009. URL http::/dx/xxxxmmm wttp:/.t........////////////////
2001.
Vonkatadhecraj Pichapati, Ananda Theertha Suresh, Felix X Yu, Sashank J Reddi, and Sanjiv Kumar. Adaclip: Adaptive clipping for private sgd. arxiv preprint orXiv: 19x.:198..0743, 2019.
Or Sheffet. Old techniques in differentially private linear regression. In Aurélien Garivier and Satyen Kale (eds.), Proceedings of the 30th International Conference on Algorilloiic Learning Theory, volume 98 of Proceedings of Machine Learning Research, pp. 789-827 PMLR, 22-24 Mar 2019. URL https:// procesdings.mlr.prsssse
mlr. wttpppse////sseeeeeeeeee html
Virat Shejwalkar, Arun Ganesh, Rajiv Mathews, Om Thakkar, and Abhradeep Thakurta. Recycling scraps: Improving private learning by leveraging intermediate checkpoints. ArXiv, abs/ abs/2210.01864, 2022.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18, 2017.
Adam Smith, Abhradeep Thakurta, and Jalaj Upadhyay. Is interaction necessary for distributed private learning? In 2017 IEEE Symposium on Security and Privacy (SP), pp. 58 77. IEEE, 2017.
Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pp. 245 248. IEEE, 2013.
Shuang Song, Om Thakkar, and Abhradcep Thakurta. Characterizing private clipped gradient descent on convex generalized linear problems. Xiv preprint arXiv:2006.0676 2020.
Xingyou Song, Sagi Perel, Chansoo Lee, Greg Kochanski, and Daniel Golovin. Open source vizier: Distributed infrastructure and api for reliable and flexible blackbox optimization. In Automated Machine Learning Conference, Systems Track (AutoMI-Conf Systems), 2022.
Andreas Steiner, Alexander Kolesníkov, Xiaohua. Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How to train your vit? data, augmentation, and regularization in vision transformers, 2021.
Pranav Subramani, Nicholas Vadivelu, and Gautam Kamath. Enabling fast differentially private sgd via just-in-time compilation and vectorization. arXie preprint orXiv:2010.00 0.00633, 2020.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, D. Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pallern Recognition (CVPR), pp. 9, 2015.
Om Thakkar, Galen Andrew, and H Brendan McMahan. Differentially private learning with adaptive clipping. arXiv preprint arXiv:1905, 03871, 2019.
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Peter Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. MLP-mixer: An all-MLP architecture for vision. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL httpp://openrreviw het/forrm?id==IIIOXXX
Florian Tramer and Dan Bonch. Differentially private learning needs better features (or much more data). In International Conference on Learning Representations, 2021. URL https://openreviek: http:///enrrviiiieeee/e////rrrr/ net/forum?id
YTWGvPFOQD-
Prateek Varshney, Abhradeep Thakurta, and Prateek Jain. (nearly) optimal private linear regression for sub-gaussian data via adaptive clipping. In Po-Ling Loh and Maxim Raginsky (ods.), Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pp. PMIRR 02 02-05 Jul 2022. URL httpps:/pproceedingg.... mlr. ptess//118//arrshaatte html html
17

---

Published in Transactions on Machine Learning Research (04/2023)
Yu-Xiang Wang. Revisiting differentially private linear regression: optimal and adaptive prediction & estimation in unbounded domain. ArXin, abs albs11833002590 2018.
Yu-Xiang Wang, Borja Balle, and Shiva Kasiviswanathan. Subsampled rényi differential privacy and analytical moments accountant. Journal of Privacy and Confidentialityy 10(2), Jun 2020. ISSN 2575-8527. doi:
10.29012/jpc.723. URL http: /ttt:://xx.wr http::/xxxddo.oooooo ott....r/0....00//.p...... 723
Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeffrey F. Naughton. Bolt-on differential privacy for scalable stochastic gradient descent-basod analytics, 2016.
Yang You, Igor Gitman, and Boris Ginsburg. Scaling SGD batch size to 32k for imagenet training, 2017.
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsich. Large batch optimization for deep learning: Training bert in 76 minutes, 2019.
Da Yu, Huishuai Zhang, Wei Chen, and Tie-Yan Liu. Do not let privacy overbill utility: Gradient embedding perturbation for private learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 8-7, 2021. OpenReview.net, 2021. URL /ttLittss///ppenee..eee..t
forum/1d=7aog0]__WW VYDO
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. Differentially private fine-tuning of language models. ArXio, abs/2110.06500 2022a.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models, 2022b.
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. Scaling vision transformers, 2021.
Guodong Zhang, Tala Li, Zachary Nado, James Martens, Sushant Sachdeva, George E. Dahl, Christopher J. Shallue, and Roger Grosse. Which algorithmic choices matter at which batch sizes? insights from a noisy quadratic model, 2019.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization, 2017.
Huanyu Zhang, Ilya Mironov, and Meisam Hejazinia. Wide network learning with differential privacy, 2021.
Yingxue Zhou, Zhiwei Steven Wu, and Arindam Banerjee. Bypassing the ambient dimension: Private sgd with gradient subspace identification, 2020.
Yuqing Zhu and Yu-Xiang Wang. Poission subsampled rényi differential privacy. In International Conference on Machine Learning, pp. 7634-7642. PMLR, 2019.
A Limitations
This work leverages a large properietary dataset called JFT-3B to pre-train ViT-G/14 model in order illustrate the benefits of scale on differential privacy with transfer learning. In order make our work more generalizable and reproducible, we also include results with models pre-trained with ImageNet-21k and ImageNet-1k.
Another limitation of our work may be the fact that our pre-training datasct is largely in-distribution with the private fine-tuning datasets. We would like to argue that, in practice, this is still valuable since it illustrates the effectiveness of the approach, and helps estimate the utility of gathering a public dataset to pre-train on, given a sensitive dataset that one wants privacy guarantce over. Finally, out of distribution performance is an interesting research question even in the non-private setting and its exploration in the context of privacy can be a direction of very valuable future work.
18

---

Published in Transactions on Machine Learning Research (04/2023)
We additionally note that we do not include any cost related hyperparmeter tuning in our privacy accounting and budget. This is in line with the setup of the baselines consider and allows for fair comparison with reported numbers. However, we recognize this as limitation of our work and more research in this direction a considerable opportunity.
In terms of societal impact, the biggest cost of this work training the largest ViT-G model on a large dataset and its energy impact. However, we argue that our results ultimately point towards amortizing and increasingly leveraging already trained models for high-performance DP training, and thus potentially reducing the overall energy consumption.
B Algorithmic details
B.1 Privacy Analysis details for DP-SGD
Algorithm 4 Generalized First Order Differentially Private Algorithm
Require:
Data set D (111,11),.--- (In, Hn)} with (x4, v: E loss function: & Rmxd x R8 + R, first order optimizer Opt, clipping norm: C, number of iterations: T, noise multiplier:
1: Randomly initialize On-
2: for t= 1....,, T' do
9t + + clip (Ve(0s: (24, where clip(u) v min { }
4: - + (oC))
5: 01-1 f single step of first order optimization with gradient Opt(Ee)
end for
T 0g or 0r.
7: return
1 Σ T
The privacy parameters (s, are functions of C, σ, [Br] [D), and the total number of iterations T. DP-SGD algorithm involves setting the right clipping norm C and the noise multiplier a given a privacy budget, batch and dataset size. The (s, guarantee is computed by analysis of the Gaussian Mechanism with privacy amplification by subsampling and composition across across iterations (Ksivvisaantthan et al. 2008 Bassily
et al. 2014 Abadi et al. 2016 Mironov] 2017:
McMahan et al. 2017 Mironov et al.) 2019 Erlingsson et al,
2019
Zhu & Wang 2019 Feldman et al. 2020 Wang et al., 2020). Our implementation relics on Tensorflow
Privacy codebase for conversion of (s and clipping norm C to/from noise multiplier a. We rely on the
default Rényi accountant implementation already open-sourced as part of Tensorflow Privacy library.
To put the epsilon-delta values in context, privacy guarantce for let's say E R 4 on ImageNet- 1K satisfies a much stronger property of xCDP (0.154 for = 4) which is by now industry standard.
C Pre-training Details
We conduct all our experiments in Jax Bradbury et al. 2018 Frostig et al. 2018) is framework that leverages just-in-time compilation using XLAA and does autoovectorizall of the backward pass. We leverage this functionality throughout our experiments. Finally, we conduct our experiments on TPUv4 architecture.
C.1 Pre-training with JFT-3B
Dataset. Contrary Mehta et al. (2022) ; De al. (2022) use smaller version of JFT, namely JFTT-B (instead of JFT-4B) for pre-training. We do this to lower the risk of the project and follow Zhai et al. (201)) exactly for pre-training ViT-G/14 model. JFT-3B dataset consists of nearly 3 billion images, annotated with a class-hicrarchy of around 30k labels via a semiautomatic pipclinc. As done previously, we ignore the
http::///shttoocccc/c/ooooooo//s hhtpps:/ httpss//www..mmoorooooorrr/////
19

---

Published in Transactions on Machine Learning Research (04/2023)
hierarchical aspect of the labels and use only the assigned labels targets for multi-label classification via sigmoid cross-entropy loss.
Deduplication. In order to both not inflate our results and break privacy guarantee offered by fine-tuning privately on ImageNet, we extend the deduplication process proposed by Kolesnikov et al. (2020) and deduplicate both JFT-3B with respect to all splits of ImageNet. We use a model based deduplication system which removes both exact and near-duplicates across common image transformation like crop, shift, resize etc.
Hyperparameters. At the pre-training stage, follow Zhai al. (2021) exactly and stick with the common practice of employing Adafactor optimizer with B1 0.9 and 32 0.999, with batch size of 32768, dropout rate of 0.0, clip global norm of 1, and high weight decay of 3.0 for the "head" and 0.03 for the "body'. In addition, we remove the additional [class] loken to save memory. Finally, all the models are pre-trained at resolution [224, 224], with inception crop followed by random horizontal flip pre-process. We also use reciprocal square-root schedule with linear learning rate warmup of 10k steps. Finally, ViT-G/14 model was pre-trained using 2048 TPUv3 chips.
C.2 Pre-training with ImageNet21k and ImageNetlk
Datasets. ImageNet-21k is a superset of ImageNet-k with 21k classes and 14M images (Deng et al. 2009 Similar to before, in order to both not inflate our results and break privacy guarantee, we extend the deduplication process proposed by Kolesnikov al. (2020) and deduplicate ImageNet-22k with respect to all splits of ImageNet-1k, 1k, CIFAR-10 and CIFAR-100. Similarly, we deduplicate ImageNet-1k with respect to all splits of CIFAR-10 and CIFAR-100.
Hyperparameters. At the pre-training stage, we stick with the common practice of employing Adam optimizer (even for ResNet) with 31 = 0.9 and 32 = 0.999, with a batch size of 4096. Unlike pre-training with JFT dataset, we follow recommendations from Steiner et al. (2021) to use AugReg strategy where we lower the weight decay to 0.1 (which gets multiplied by the learning rate) and don't use dropout but instead use data augmentation strategy called mediuml which combines Mixup with a = 0.2 (Zhang el. al. 2017) and RandAugment with / 15 and m (Cubuk et al.] (2020). We also use linear learning rate warmup until 10k steps and lincarly docay it until the end. Our model is pro-trained with 224x224-sized images.
Model
Dalaset
Epochs
Base 1)
TPU v4 hours
VIT-B/16 ImageNet-21k
300
10-3
2.7k
VIT-B/16
Imagenct-1K
300
10-3
0.4k
Table 5: Pre-training hyperparams. We used batch size of 4096, learning rale warmup of 10k sleps and then linear decay. Additionally, we set dropout rate to 0.0, clip global norm to and weight decay to 0.0001. We use images of resolution 224x224. Note that we intentionally keep the model size the same to illustrate the effect of larger pre-training dataset and its effect on private fine-tuning.
D Finetuning Details
D.1 Datasets
ImageNet-lk We fine-tune on ImageNet train split and present the Top-1 accuracies we obtain from the official test split. Following Mehta et al. (2022), used images of input resolution 256x256 which is central cropped from resolution of 384x384. Note that this is slightly lower resolution and without Inception Crop (Szegeyy et al. 2015) which is typically done in non-private setting. Finally, for training with DP, we fixed 8
to be 8e-7.
CIFAR-10 and CIFAR-100 Similar above, we finc-tune on train split and present the Top-1 accuracios we obtain from the official test split. We also changed the input resolution 256x256 which is central cropped from image of resolution 384x384. Again, this may look little unusual at first. for CIFAR-10 and
20

---

Published in Transactions on Machine Learning Research (04/2023)
CIFAR-100 since the original resolution of the images is 32x32. But first upsample them to 384x384 and then central crop them. We found that using higher resolution images made big difference in performance (even in non-private setting), especially when using features from a pre-trained model. Finally, for training with DP, we fixed to be le-5.
D.2 Compute requirement
To illustrate that our proposed methods don't 't impose significant compute overhead, we also report the exact compute required for each of our methods in Table 0 with ImageNet-11 features (largest dataset consider) obtained from ViT-G/14 pretrained with JFT-3B dataset (highest dimensional model consider).
As shown in Table 0 not only the absolute compute requirement for private finetuning using our methods is quite minimal, our best performing method (DP-FC) requires only slightly higher compute overhead over our baseline (DP-Adam) while outperforming in terms of quality significantly.
Protraining Datasel
Method
Epochs
TPUv4 hours
DP-LS
1
0.9
DP-Newton
10
2.4
JFT
DP-FC
10
1.2
0.2
DP-Adam
10
0.9
100
Table 6: Comparison of TPU v4 hours required measure of exact compute required for private finetuning with ImageNet-1k features obtained from ViT-G/14 pretrained with JFT-3B dataset.
D.3 Hyperparameter Tuning
All of our results consider full-batch setting with constant learning rate (when applicable). Additionally, following Mehta et al. 22022,, we set initial weights to 0.0.
To gain confidence in our proposed methods, we wanted to properly tune the important hyperparameters. Fortuitously, since we are only interested in learning just the last. layer from features, each training run can be relatively inexpensive. To properly tune hyperparameters, we employed a Baycsian optimization package called Vizier (Golovin al. 2017; Song el. al. 2022). For tuning hyperparameters, of the training set our validation set and report top-1 1 accuracies hyperparameters.
heldout 5% the test set by using the tuned
Note that, for a fair comparison, similar to previous work and our baseline (DP-SGD), we do not account for the privacy cost of hyperparameter tuning in our results. For proper tuning and trustful results, we made sure run more trails than required for tuning each method, although for all of tuning runs, however we found that the optimizer converged quite early on, suggesting that the cost of hyperparamter tuning can be further reduced.
21

---

Published in Transactions on Machine Learning Research (04/2023)
D.4 DP-Adam hyperparameters
Model
Pre-training DS
Fine-tuning DS
X
DP Clipping Norm (C)
VIT-G/14
JFT-3B
ImageNet- lk
1108,100
[10-*110 10
1.0
ViT-G/14
JFT-3B
CIFAR-10
[10-8100]
[10-8,109
0.005
ViT-G/14
JFT-3B
CIFAR-100
[10-8,10*]
[10-8,109
0.005
ViT-B/16
ImageNett221
ImageNet-1k
[10-8.108]
[10-8,109
0.005
VIT-B/16
ImageNeet22kk
CIFAR-10
[10-8,
100$,1091
0.005
ViT-B/16
ImagcNet-21k
CIFAR-100
[10-8,
[10 [10 10
0.005
VIT-B/16
ImageNet-lk
C-
100$ 10
110 10
0.005
VIT-B/16
ImageNet-1k
CIFAR-100
[10 1.0
10 110 10
0.005
Table 7: Fine-tuning hyperparams for DP-Adam. All models are trained in full-batch setting with constant learning rate and no dropout. When training the models with DP, we replace the global clipping with per example clipping norm as specified in the table. Following Mehta et al. (2022), we set initial weights to 0.0, bias to -1000 and train with sigmoid cross-entropy loss. Note that we employed a Bayesian optimization package called Vizier (Golovin al. 2017 Song al.) (2022) and used a total of 200 trials for jointly tuning both the learning rate and weight decay as specified the table.
D.5 DP-LS hyperparameters
Model
Pre-training DS
Fine-tuning DS
X
DP Clipping Norm (C)
VIT-G/14
JFT-3B
Imt-eNet-1k
110 10a)
[10-8 10
1.0
VIT-G/14
JFT-3B
CIFAR-10
[10-8,108
[10-8,108
0.005
ViT-G/14
JFT-3B
CIFAR-100
[10-8,10*
[10-8,10s]
0.005
VIT-B/16
ImageNee-211k
ImageNet-1k
[10-8,108]
[10-8,109]
0.005
VIT-B/16
ImageNet-21k
CIFAR-10
[10-8,
[10-8, 1081
0.005
VIT-B/16
ImageNet-21k
CIFAR-100
[10-8,
110*,110$1
0.005
VIT-B/16
ImageNet-1k
CIFAR-10
10-5.
10 10 10
0.005
VIT-B/16
ImageNet-1k
CIFAR-100
11-
10-8 10
0.005
Table 8: Fine-tuning hyperparams for DP-LS. All models are trained in full-batch setting. When training the models with DP, we replace the global clipping with per example clipping norm as specified in the table.
Specific to DP-LS, we clip the RHS with C and the gramians with C Interestingly, least squares is invariant the starting weights which takes an important confounding factor away from the private training procedure. Similar to DP-Adam, we employed a Bayesian optimization package called Vizier (Golovin al.) 2017 Song
et al. 2022 and used total of 200 trials for jointly tuning both the and weight decay specified in the table.
22

---

Published in Transactions on Machine Learning Research (04/2023)
D.6
DP-Newton hyperparameters
Model
Pro-training DS
Fine-tuning DS
A
DP Clipping Norm (C)
ViT-G/14
JFT-3B
ImageNet- lk
[10 10
[10-* 10
1.0
ViT-G/14
JFT-3B
CTFA R-10
[10-8
[10-8,109
0.005
VIT-G/14
JFT-3B
CIFAR-100
[10-8,10*]
[10-
0,005
VIT-B/16
ImageNet-221k
ImageNet-1k
[10-8.109]
[10-$.10%
0.005
VIT-B/16
ImageNet-221k
CIFAR-10
[10-8.109]
[10-
0.005
VIT-B/16
ImagcNet-21k
CIFAR-100
[10-8
[10 10
0.005
VIT-B/16
ImageNet-Ik
CIFAR-10
10-$,109
110
0.005
VIT-B/16
ImageNet-1k
CIFAR-100
[10 [10-
8, 8,09
[10 10
0.005
Table 9: Fine-tuning hyperparams for DP-Newton. All models trained in full-batch setting with constant learning rate and no dropout. When training the models with DP, we replace the global clipping with per example clipping norm as specified in the table. Specific to DP-Newton, we clip the features instead of the gradient and the Hessian. We do this save on computational cost since now we don't need to explicitly compute per-example gradient and the Hessian. For the privacy analysis, we use the clipping norm sanitize the gradient and C for the Hessian. Following Mehta et al. (2022), we set initial weights to 0.0 and train with sigmoid cross-entropy loss, We employed a Bayesian optimization package called Vizier (Golovin lt al. 2017 Song et al. 2022) and used a total of 300 trials for jointly tuning both the learning rate and weight decay specified in the table.
4.0
D.7 DP-FC hyperparameters
Model
Pretraining DS
Finetuning DS
A
DP Clipping Norm (C)
VIT-G/14
JFT-3B
ImageNet-1k
[10 10a
[10-8 10
1.0
VIT-G/14
JFT-3B
CIFAR-10
11 10
[10-8 10
0.005
ViT-G/14
JFT-3B
CIFAR-100
1008,10
[10-8,10*
0.005
VIT-B/16
ImageNet-21k
ImageNet-1k
1008.109
[10-8,199
0.005
VIT-B/16
ImageNet-21k
CIFAR-10
10-8,100
[10-8
0.005
VIT-B/16
ImageNet-21k
CIFAR-100
10 1008,100
[10 [10-8,
0.005
VIT-B/16
ImageNet-Ik
CIFAR-10
10 10-8,109
[10 [10-8 10a
0.005
VIT-B/16
ImageNet-1k
CIFAR-100
10 10 8 10
10-8 10
0.005
Table 10: Fine-tuning hyperparams for DP-FC. All models are trained in full-batch setting with constant. lcarning rate and no dropout. When training the models with DP, we replace the global clipping with per example clipping norm as specified in the table. Specific to DP-FC, we clip the per-example gradients with clipping norm and C2 for the gramian which shared across classes and iterations. Following Mehta et al. (2022), we set initial weights to 0.0, set initial bias 100. and train with sigmoid cross-cntropy loss. We employed Bayesian optimization package called Vizier (Golovin et al. 2017 Song al. 2022) and used total of 200 trials for jointly tuning both the learning rate and weight decay specified in the table.
23


Reviews:
[{"review_id": "VRcbpeRnId", "reviewer": "Reviewer_db1x", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe main strength is the novelty in the methodology, especially the DP-FC, which is well-explained by firstly define and experiment on DP-LS and DP-Newton. The paper is easy to follow and solid in terms of DP guarantee. In Section 2.3, DP-LS is carefully described and distinguished from existing work that establish on the closed-form solution of least square problems (maybe a missing reference to https://arxiv.org/abs/1803.02596). The experiments are convincing and supporting the claims well.\n\nWeaknesses:\nWhile this paper is well-written and fulfills its scope, the contributions are mostly empirical and on the privacy theory. From an algorithmic viewpoint, an extended discussion on the efficiency, the convergence theory, and the applicability will enhance the paper significantly. \n\nI understand that the efficiency is briefly discussed in Section 4, but no time/space complexity or empirical evidence such as training time per epoch and RAM occupation is present. This could be very meaningful to have given that second-order information is used, whereas most existing DP works use first-order ones.\n\nBesides, although this paper makes it clear that the focus is on the image classification, it is desirable to at least discuss whether this method works on language models and tasks. On a side note, it is commonly observed that last-layer fine-tuning does not work well with language models. Therefore, if DP-FC generally works on language models, it would be a great advantage. \n\nAlso, I highly recommend not to use JFT. JFT is a proprietary dataset that is not public. Any experiment on it is not reproducible if not hard-to-trust, it is not a good practice for the DP community. See more comments in 'Requested Changes'.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses specifically on the image classification and studies the last-layer fine-tuning. The authors propose three new second-order methods, DP-Newton, DP-FC, DP-LS, to fine-tune Vision Transformers on 3 datasets. The main contributions are on the methodology, which come with formal privacy guarantee, and on the image experiments. Through their analysis, we may advance our understanding of pretraining datasets and optimizers from an empirical perspective.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The tables are hard to follow when I tried to understand which model gives which results. I searched for a while in the appendix to find this important information (because larger models generally give better results, which must not be confused with the benefits of new methods). The models should be added to Table 1-4 as an additional column.\n\n2. Unfair comparison is observed in the tables. An experiment is of the following elements: (pretraining dataset, optimizer, training time/computation resource, model, fine-tuning method). Given that the main contribution of this paper is on the `optimizer' part, other parts should be set similar to previous results.\n\nIn Table 1, previous SOTA on CIFAR10 and CIFAR100 are pretrained on ImageNet21k and trained for 3 epochs, while the authors achieve new SOTA on a LARGER pretraining dataset JFT, MORE epochs (10 epochs) and LARGER model (vit-G 2B v.s. BeiT-large 0.3B). This is for sure not an apple-to-apple comparison because the model is different, the training cost is different, and the pretraining is different. I agree that it is not necessary to use the same setting as previous SOTA. However, the more different your setting is, the less convincing that your new optimizer is better. At least one experiment on CIFAR10/100 with the same pretraining dataset ImageNet21k is needed.\n\n3. Why in Table 2/3/4, DP-LS is only trained for 1 epoch, unlike DP-FC and DP-Newton which also show the 10 epoch results? The 1 epoch result shows that DP-LS is often the best method, so it may be true that it is also the best if trained for 10 epochs. Also DP-LS is strong when epsilon is large (e.g. Table 2, epsilon=8, DP-LS 1 epoch=86.7, DP-Newton=84.9, DP-FC=85.6), this contradicts your claim in the abstract \"least squares is more effective at lower epsilon values while Newton’s method is more effective at larger epsilon values\".\n\n3. Discuss efficiency. I would at least need the training time of each method. In the paper, most discussion is about the number of epochs, which is unimportant if DP-FC takes much longer to train one epoch than DP-SGD/Adam. This renders the claim in Section 3.2 \"(DP-FC) allow us to obtain better results with 10 epochs compared to even when using 100 epochs with DP-SGD\" not well-supported.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ykUiulBolI", "reviewer": "Reviewer_Diqz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The aim of this paper is to improve the utility of differentially private learning. It finds that by changing the loss function (from logistic regression to least-square linear regression), the performance is improved, moreover, by using second-order optimization methods, the performance can also be improved. This paper combines such two techs. to design the DP-FC algorithm which shows better results compared with previous SOTAs.\nHowever, the advantage is not significant as can be seen from Table 1 if compared with previous SOTAs. The cost of the proposed algorithm which is the computational cost (line 2 of alg. 4) actually offsets the advantage. \nOverall, the pre-training paradigm is not new, DP-FC offers limited improvements while also incurring quadratic computation costs with respect to the number of samples.  The proposed method may not constitute a satisfying manuscript for potential publication. \n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tries to improve the utility of differentially private learning by reforming the loss function and optimization approach. Results show somewhat improvement compared with previous work.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address my previous concerns ", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "XnqcQdc647", "reviewer": "Reviewer_nzZQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\nThe proposed methods seem like intuitive extensions to the standard DP-SGD, and under the limited number of gradient updates (<=100, with full batch in the experiments) the 2nd order information might indeed provide significant improvement to the learning (at least it would in non-DP case). The empirical result seem to indeed support this, with the higher order methods constantly outperforming the DP-SGD.\n\n## Weaknesses\nHowever, I do have some concerns which I hope the authors can address.\n \n### The bound for the linear regression loss\nIn the paragraph after Thm 2, you state that $|\\ell'(z,y)| \\leq 1$ holds for the squared loss for _all_ $(z,y)$. How can this be? You have $\\ell'(z,y)=z-y$, and now if your $z>1$ the claim obviously does not hold for $y=0$. I don't think there is a bound for the $z$ even if your input $x$ would be bounded as the regression parameters will be in $\\mathbb{R}^d$. If this does not hold, then I'm not sure if the reported $\\epsilon$ values for the DP-LS are correct.\n\n### Hyperparameter tuning\nThe appendix seems to suggest that you have used Bayes optimization with 200 trials in choosing the hyperparameters for you fine-tuning. If I'm correct, the privacy cost of this is not accounted in the reported $\\epsilon$ values (by the way: please state this explicitly). Now, in general I don't find the non-DP hyperparameter tuning such a big issue, as that is what many works in the field seem to do. However, for the particular method proposed, the hyperparameter tuning seems like a hugely important thing. For example, if you consider the single epoch, full batch (i.e. single step) gradient optimization, the hyperparameter search starts to resemble line search, which I would image has a huge effect on the performance. Therefore, if the results are really strongly dependent on the chosen hyperparameters, authors should definitely be more up-front about it. \n\nAlso, I would like the authors to better explain the hyperparameter tuning setting. Is there a separate validation set on which you tune the hyperparameters? Or are you just using the train split for the hyperparameter tuning?\n\n### Robustness of the results\nAll the tables report single top-1 accuracy results. This leaves me to wonder how robust the results actually are. For example, does the DP-Newton give you constantly top-1 accuracy of ~70% for the epsilon=0.01 setting of Table 2? What would happen if you would repeat the inference for example 10 times? Please, do add some error bounds for the methods. Otherwise it is really impossible to estimate the consistency of the methods.\n\n## Other comments/suggestions/questions\n- I believe the preconditioner used in Alg. 4 is a scaled version of the Fisher information matrix of the logistic regression log-probability. The Hessian for this particular problem will be block diagonal, and the $\\nabla^2 log p(y \\mid x, \\theta)$ is given as $\\sigma(\\langle\\theta, x\\rangle)(1-\\sigma(\\langle\\theta, x\\rangle)) x x^T$. All the blocks of the Fisher information will therefore be scaled with a scalar. Therefore, I think the learning procedure of Alg. 4 with the empirical covariance matrix as preconditioner resembles the one of natural gradients, and could be maybe motivated through it. It would be really interesting to see if adding the correct scaling to the preconditioner would yield even better results for the non-private classifier (this is just a curiosity though). \n\n- I think it would be fair if the Tables would include the best non-DP accuracy as well. Now it remains unclear how much the epoch limitation for example affects the non-DP baseline. I know that this is not probably attainable baseline, but it would give the reader relevant information of how well we should expect these algorithms to perform in non-DP case.\n\n- I guess in the evaluation of the linear regression based solution you simply pick the category $j$ with highest $\\langle \\theta_j, x_i\\rangle$?\n\n- I must say, that I'm bit puzzled by some results. For example, in Table 2, the results of a single step DP-Adam with $\\epsilon=0.01$ vs the DP-Newton in the same setting are really surprising. The only difference between these methods, as far as I understand, is the Hessian, and the gradients in DP-Adam should only have half the noise due to budget splitting in DP-Newton. I would imagine that if we do only a single step of the learning, the noisy (really noisy in this case) Hessian would not help the learning that much, but it seems that the difference is really drastic with DP-Newton giving top-1 accuracy of ~70% while DP-Adam <20%! Just to be clear, I'm not claiming that the results are wrong, but I would like to understand better how does this happen. Also, *making your source code available* would make it easier to understand the algorithms and the parameter choises used.\n\n### Typos and other minor things:\n- \"... only with the number of positive classes per example\": To me this would make more sense if it would be \"positive examples per class\", but maybe I misunderstood something. Or can an example (which I think is the sample?) have multiple different classes (which I believe are the labels)?\n- In the Appendix B.1 you state \"satisfies a much stronger property of zCDP ≤ 1 (0.154 for ε = 4) which is by now an industry standard.\". Is this because the conversion from zCDP to approximate DP is lossy? I'm not too familiar with the industry standards in terms of zCDP, so any citation that zCDP $\\leq 1$ is used in practice would be helpful.\n- It seems that in Algs 1 and 4 you have used $\\ell: \\mathbb{R}^{m \\times d}$ and in Alg 2 just $\\ell: \\mathbb{R}^{m \\times d}$. I think the latter one must be a typo right?\n- Are you considering the add/remove neighborhood relation for the privacy analysis? I guess so, as otherwise the sensitivities should have a $\\sqrt{2}$ before them. Might be good to mention it, now you have just stated that $D$ and $D'$ differ in a single data point, which could also refer to the substitute neighborhood.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies transfer learning for image classification models under differential privacy. The underlying task is the typical task in DP transfer learning, where a complex classification model is pre-trained using a publicly available data set, and then the model is fine-tuned for a sensitive private data with DP optimization methods. The previous approaches in the topic have used DP-SGD for the fine-tuning part.\n\nIn this paper, authors suggest three new algorithms for the private fine-tuning part with focus on large scale image classification models. First of the methods, the DP-Newton (Alg. 2) is a simple extension of DP-SGD, which uses the perturbed Hessian as well as the perturbed gradients. The main benefit of the method is to use the second order information for the optimization. The second method proposed, the DP least squares (DP-LS, Alg. 3), uses a least squares loss instead of the logistic loss to optimize the weights. Essentially, the model is a linear regression instead of logistic regression. For this approach, authors can write the optimal parameters in terms of sufficient statistics and, perturbing those achieve DP. The 2nd order information, or the Hessian, arises naturally in the sufficient statistics for this model. The final method, the DP feature covariance (DP-FC, Alg. 4), reverts back to the logistic loss, but uses the perturbed empirical feature covariance as a preconditioned for the gradients. The empirical covariance matrix is treated as an approximation of the 2nd order information, and compared to the DP-Newton, it only needs to be computed once, thus saving in DP budget.\n\nThe empirical performance of the methods seems really impressive, with DP-FC providing significant improvements to the SOTA (see Table 1).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I want authors to clarify the following points I raised in the weaknesses:\n- Is there a bound for the linear regression loss function's derivative\n- Clarification for the hyperparameter tuning setup\n- Some error bounds for the methods (more repeats)\n\nAdditionally, making the code available would help a lot.", "broader_impact_concerns": "I don't think the work needs a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper makes contributions to transfer learning of image classification under differential privacy. There are threoretical results proved for the three new proposed methods for differentially privately optimizing the fine-tuning aspect of training (model is trained non-privately on public data, and then only fine-tuned on sensitive data for which privacy concerns are important). There is substantial empirical evaluation that shows the efficacy of the proposed methods.

The reviewers have made several useful comments which the authors should incorporate into the main article. In particular, the authors should incorporate all the changes that they have agreed to make in response to the reviews. 

They should also be explicity about their intentions to release code. One reviewer has rightly suggested that in order for this paper to make most impact, and for reproducibility reasons, code release is highly desirable.
Decision: Accept (poster)