Differentially Private Learning Needs Hidden State (Or Much Faster Convergence)
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Differential privacy analysis of randomized learning algorithms typically relies oncomposition theorems, where the implicit assumption is that the internal state ofthe iterative algorithm is revealed to the adversary.
However, by assuming that theinternal state of the algorithm is not revealed, recent works prove a smaller privacybound for noisy gradient descent (on strongly convex smooth loss functions),compared with composition bounds.
In this paper, we signiﬁcantly improve privacyanalysis under the hidden state assumption.
We enable taking advantage of privacyampliﬁcation by sub-sampling and randomized post-processing, and extend theanalysis to “shufﬂe and partition” and “sample without replacement” stochasticminibatch gradient descent schemes.
We prove that, in these settings, our privacybound is substantially smaller than the composition bounds, notably after manyepochs of training (required for learning from high-dimensional data).
So, unlessthe DP algorithm converges fast, our privacy analysis shows that differentiallyprivate learning beneﬁts from hidden state privacy analysis.
We also empiricallyshow that, given a ﬁxed privacy budget, our analysis enables DP training of imageclassiﬁcation models with a better prediction accuracy, compared with prior work.
Introduction
Machine learning models leak sensitive information about their training data.
To protect user privacy,the widely-used differentially private training algorithm, DP-SGD [1], adds carefully calibrated noisein each gradient update for the model parameters.
This added noise guarantees that the modelstrained on two neighboring datasets are indistinguishable in probability distributions.
To quantify theindistinguishability, differential privacy analysis bounds the (moment of) likelihood ratio between apair of models trained on any two neighboring datasets.
Improving this privacy analysis is crucial forobtaining a higher (train and test) accuracy of the output model, under a constrained privacy budget.
For a vast majority of learning tasks (such as private image classiﬁcation), composition-based privacyanalysis of DP-SGD [1] is the mainstream analysis method, mainly due to its simplicity and broadapplicability.
However, private learning via DP-SGD usually suffers from a slow convergencein empirical prediction accuracy [10], under which the total privacy loss grows very large due tocomposing it over a large number of iterations.
This, in turn, results in overestimating the magnitudeof required noise for differentially private training, which worsens the privacy-accuracy trade-off.
To alleviate this problem, one popular direction is to design new variants of the DP-SGD algorithm,that converge faster, i.e. require a smaller number of iterations to reach a stable training accuracy.
This includes works that derive privacy preserving variants of fast optimization algorithms, such asperforming DP-SGD with momentum [22], adaptive gradient clipping [2, 32], adaptive selection ofstep-size and noise scale [3], and using pre-trained or hand-crafted features [36].
All these approachesaim for a faster training convergence for the DP learning algorithm, such that the composition analysis is applied to a smaller number of iterations.
This is to reduce the high performance cost ofoverestimating the privacy loss (and the required noise scale) due to DP composition.
Another line of work focuses on directly improving the privacy analysis of the DP-SGD algorithm.
To this end, prior work [18, 5, 15] suggests that hiding the internal state of noisy (S)GD, thereforeanalyzing the privacy bound for releasing only the last iterate results in a more accurate estimation ofprivacy loss, which can substantially reduce the performance cost of differential privacy.
By applyingthe last-iterate privacy analysis [18], Feldman et al.
[19] design new differentially private algorithmsthat achieve theoretically optimal excess risk with linear runtime for convex optimization, which isbetter than the prior privacy utility trade-off for the noisy SGD algorithm [7].
However, the analysisapplies only to one single epoch of training, thus it is unclear how such training performs in practicallearning settings (which require multiple epochs of training).
Chourasia et al.
[15] derive a strongprivacy guarantee for multiple epoch training with hidden-state noisy GD, under strongly convex lossfunction.
However, the analysis is limited to computationally expensive GD [20], and even if weattempt to directly apply it to stochastic mini-batch training, the analysis fails to model the privacyampliﬁcation by post-processing [18, 5] and mini-batch sub-sampling [11, 23, 7, 1, 4].
Thus, thebound under SGD learning becomes very large (as we also show in Figure 1).
Thus, under the hidden-state assumption over many epochs of training, the challenge is to computea small privacy bound for differentially private stochastic gradient descent algorithms by takingadvantage of privacy ampliﬁcations due to sub-sampling and randomized updates over mini-batches.
Contributions .
In this paper, we tackle this challenge, and bridge the gaps in the prior work onhidden-state (last-iterate) privacy analysis.
We model the privacy dynamics of noisy SGD, and showthe ampliﬁcation for hidden-state privacy bounds due to stochastic mini-batches, and demonstratethe empirical utility gain of our bound.
(i) For multi-epoch noisy stochastic mini-batch gradientdescent under “shufﬂe and partition” and iterative “sampling without replacement” , we prove newconverging privacy bounds (for strongly convex smooth loss function) that signiﬁcantly improvesover prior bounds [4, 28, 15].
Our proof relies on our new bounds for the privacy ampliﬁcationby post-processing (Lemma 4.2) and sub-sampling (Theorems 5.2 and 5.3).
(ii)
We evaluate theprivacy accuracy trade-off of practical learning tasks under our privacy dynamics analysis.
Weempirically show that for learning tasks that require many epochs of training, where DP-SGD [1] hasa slow convergence (e.g., logistic regression models trained from scratch), our privacy bound enablesachieving a much higher prediction accuracy than what the composition-based analysis brings about.
For tasks that converge within a few epochs (e.g., private ﬁne-tuning of pre-trained models), ourprivacy bound can cause a slightly better privacy accuracy trade-off.
Our results show that, to obtain high accuracy, it is crucial to apply our last-iterate privacy dynamicsanalysis for learning tasks with slow to moderate convergence.
Alternatively, it is not empirically socostly to use composition-based privacy analysis when the training process converges very quickly.
Preliminaries
Deﬁnition 2.1 (Differential Privacy [16, 17]) .
A randomized algorithm A is ( ε, δ ) -differentiallyprivate
if for any neighboring datasets D, D (cid:48) , and for all possible event S in the output space of A ,
[Equation]
(1) where we say D, D (cid:48) are neighboring if they are of the same size and differ in at most one data record.
Deﬁnition 2.2 ( ( α, ε ) -Rényi
DP [27]) .
A randomized algorithm A is said to satisfy ( α, ε ) -Rényidifferential privacy (or ( α, ε ) -Rényi DP for short), if for any neighboring datasets D and D (cid:48) ,
[Equation]
A
[Equation]
≤ ε, where R
[Equation]
µ [Equation]
θ ) ν ( θ ) (cid:19)
[Equation]
(cid:21) (2) where A ( D ) ( A ( D (cid:48) ) ) denote the distribution of output given input dataset D
( D (cid:48) ), and R α ( µ (cid:107) ν ) isthe Rényi divergence [33] of order α > 1 for two distributions with density µ ( θ ) and ν ( θ ) on R d .
Deﬁnition 2.3 (log-Sobolev Inequality [37]) .
A distribution ν over R d satisﬁes the log-Sobolevinequality (LSI) with constant c if for all smooth function g : R d → R with E θ ∼ ν (cid:2) g ( θ ) 2 (cid:3) < ∞ ,
[Equation]
[Table]
An overview of the problem and our approach
We analyze differential privacy loss of the noisy (stochastic) minibatch gradient descent Algorithm 1under two minibatch generation variants: (1) “shufﬂe and partition” (widely implemented in privacylibraries [42, 21]); (2) “sample without replacement” (analyzed extensively in prior results [11, 23, 4,39] despite its computational cost).
Our ultimate goal is to prove a worst-case upper bound of the
Rényi divergence between distributions of last-iterate parameters θ 0 K and θ (cid:48)
K trained on any twoneighboring datasets, that is tighter than composition and enables a better privacy accuracy trade-off.
Our starting point is a tight converging hidden-state privacy dynamics analysis in Chourasia et al.[15] for noisy gradient descent.
To extend this GD analysis to the stochastic minibatch setting, anatural solution is to conceptually view the updates that involve the (sensitive) differing data record(between neighboring datasets) as gradient descent on a smaller , iteratively changing dataset of size b(i.e.
, the size of a minibatch).
This simple extension results in the following naive privacy baseline.
Theorem 3.1 (Naive Privacy Dynamics Baseline [15]) .
If the loss function (cid:96) ( θ ; x ) is λ -stronglyconvex and β -smooth, and its gradient has (cid:96) 2 -sensitivity S g , then Algorithm 1 under "shufﬂe andpartition" and step-size η < 2 λ + β satisﬁes ( α, ε ) -Rényi
DP with ε ≤ αS 2 g λσ 2 b 2 (1 − e − ληK/ 2 ) .
However, as also shown in Figure 1, this naive privacy dynamics baseline converges slowly to a hugeconstant, which is signiﬁcantly worse than the bounds derived by composition [28, 1] after a largenumber of epochs.
This is because this naive baseline fails to capture the privacy ampliﬁcation due tothe stochasticity of minibatch sub-sampling, and the iterative noisy updates which amplify the privacyof preceding minibatches (referred to as ampliﬁcation by post-processing).
In this paper, however,we show how to compute a much tighter differential privacy bound for SGD, under hidden-stateassumption, while taking advantage of these privacy ampliﬁcations.
Our approach is as follows.
We decompose the distribution of last iterate parameters in Algorithm 1as a mixture of conditional distributions p ( θ 0 K ) =
(cid:80) B p ( B ) · p ( θ 0 K | B ) , given any possible minibatch sequence B .
We ﬁrst analyze the Rényi privacy loss for a ﬁxed minibatch sequence whilemodeling privacy ampliﬁcation by randomized post-processing (i.e., how much the randomizedgradient update improve the privacy due to updates on preceding mini-batches).
We then quantify theprivacy ampliﬁcation by subsampling under stochastic minibatches.
Privacy ampliﬁcation by randomized post-processing .
When a (ﬁxed) minibatch only contains theindices of shared data records between two neighboring datasets, then the deterministic minibatchgradient descent mapping does not cause any privacy loss.
Addition of the Gaussian noise thatfollows this deterministic update, however, serves as randomized post-processing which decreasesthe Rényi divergence between the two processes.
To compute this privacy ampliﬁcation, we pre- cisely model the change of parameter distributions with the Fokker-Planck equation (for diffusionprocess with zero drift).
Hidden-state privacy ampliﬁcation by sub-sampling .
Although privacy ampliﬁcation by a singlesub-sampling operation is well-studied [11, 23, 7, 1, 4], to the best of our knowledge, no prior boundsare applicable to hidden-state iterative resampling.
This is because under the hidden-state assumption,the number of mixing components in the last-iterate parameter distribution grows exponentially withthe number of epochs, which makes the mixture distribution very difﬁcult to analyze.
In this paper,we ﬁrst carefully couple the mixture components between the processes associated with neighboringdatasets.
We use the joint convexity of exponentiated Rényi divergence to bound how much the Rényidivergence between mixture distributions for the pair of processes is smaller than its worst case.
Improved privacy dynamics for noisy (ﬁxed) minibatch gradient descent
In this section, we quantify the privacy ampliﬁcation by randomized post-processing in Algorithm 1,during iterations that do not access the sensitive differing data between neighboring datasets.
Wethen combine it with Rényi DP composition during the remaining iterations to prove a convergingprivacy bound.
4.1 Privacy ampliﬁcation by randomized post-processing (additive Gaussian noise)
We ﬁrst explain the key lemma that proves exponentially decaying Rényi privacy loss under additive
Gaussian noise post-processing, when the parameter distributions satisfy the log-Sobolev inequality.
This is a new bound for the well-known "privacy ampliﬁcation by iteration" phenomenon [18, 5].
Lemma 4.1.
Let µ, ν be two distributions on R d .
Let f : R d → R d be a measurable mapping on 135 R d .
We denote N (0 , 2 tσ 2 · I d ) to be the standard Gaussian distribution on R d with covariancematrix 2 tσ 2 · I d .
We denote p t ( θ ) and p (cid:48) t ( θ ) to be the probability density functions for the distributionsf # ( µ ) ∗ N (0 , 2 tσ 2 I d ) and
f # ( ν ) ∗ N (0 , 2 tσ 2 I d ) respectively, where f # ( µ ) , f # ( ν ) denote the pushforward distributions of µ, ν under mapping f .
Then if µ and ν satisfy log-Sobolev inequality withconstant c , and if the mapping f is L -Lipschitz, then for any order α > 1 ,
[Equation]
where c t = (cid:16) L 2 c + 2 tσ 2 (cid:17) − 1 is the log-Sobolev inequality constant for distributions p t ( θ ) and p (cid:48)
t ( θ ) .
Proof Sketch.
The proof starts by modelling Gaussian noise as a diffusion process with zero drift.
We then bound the rate of Rényi divergence with the LSI constant for the process (following priorworks [37, 15]).
However, instead of assuming a ﬁxed LSI constant c for all intermediate distributions,we prove a more precise LSI constant c t that changes with t .
Complete proof is in Appendix C.1.
The partial differential inequality in Lemma 4.1 quantiﬁes the ampliﬁcation of Rényi privacy lossunder additive Gaussian noise.
By solving Equation (4) on t ∈ [0 , η ] , we prove the following lemmathat models recursive privacy dynamics during one step of noisy minibatch gradient descent.
Lemma 4.2.
Let D, D (cid:48) be an arbitrary pair of neighboring datasets that differ in the i 0 -th data point(i.e.
x i 0 (cid:54) =
x (cid:48) i 0 ).
Let B jk be a ﬁxed minibatch used (in iteration j of epoch k ) in Algorithm 1, whichcontains b indices sampled from { 1 , · · · , n } .
We denote θ jk and θ (cid:48)
jk as the intermediate parameters in
Algorithm 1 on input datasets D and D (cid:48) , respectively.
If the distributions of θ jk and θ (cid:48)
jk satisfy logSobolev inequality with a constant c , and if the minibatch GD mapping f ( θ )
= θ − η · 1 b · (cid:80)
i ∈ B jk (cid:96) ( θ ; x i )is L -Lipschitz for parameters θ , then the following recursive bound for Rényi divergence holds.
[Equation]
Proof Sketch.
When i 0 / ∈ B jk , we apply Lemma 4.1 on the deterministic minibatch gradient descentupdate f , and solve Equation (4) to obtain recursive Rényi privacy bound.
This proof is similar to [Figure]
i [Figure] [Figure]
i
[Figure]
i ff e r e n ti a l P r i v ac
[Figure]
that of [15, Theorem 2], however, dealing with the new LSI constant c t that changes with t introducesan additional technical difﬁculty.
When i 0 ∈ B jk , we use composition theorem to prove the additiverecursion in the second row of Equation (5).
The complete proof is in Appendix C.2.
Comparison with prior ampliﬁcation by post-processing bound .
Prior to this work, for convexand smooth loss functions, Feldman et al.
[18] derive a tight bound for the ampliﬁcation by postprocessing in noisy stochastic gradient descent.
In Appendix C.4, we show that our new recursiveampliﬁcation bound could recover this known tight bound for convex smooth loss functions, whileusing a different proof.
For (a more restrictive setting of) one epoch of noisy SGD on stronglyconvex loss function, Balle et al.
[5] further improve the bound in Feldman et al.
[18] via a carefulcoupling-based approach.
However, we do not see an easy way to extend the bound in Balle et al.
[4]to multiple epochs except by using Rényi DP composition (which would give a linearly worsening
Rényi DP bound with the number of epochs).
By contrast, our recursive ampliﬁcation bound easilyapplies to multiple epochs (under hidden state assumption, and without requiring composition overthe epochs), and enables converging privacy dynamics for Algorithm 1 on strongly convex smoothloss functions (as discussed in Section 4.2).
4.2 Improved privacy dynamics for (ﬁxed) minibatch gradient descent
By using the recursive privacy bound Lemma 4.2 and the non-overlapping property of minibatchpartitions of the dataset, we prove position-dependent Rényi DP bounds for Algorithm 1 as follows.
Theorem 4.3 (Privacy dynamics under strongly convex smooth loss) .
Conditioned on a ﬁxed sequenceof partitioned minibatches B 0 , · · · , B n/b − 1 in Line 3, if the loss function is λ -strongly convex,β -smooth and its gradient has (cid:96) 2 -sensitivity S g , then running Algorithm 1 for K ≥ 1 epochs withstep-size η < 2 λ + β , satisﬁes (
α, ε ) -Rényi
DP for data points in the batch B j 0 , with
[Equation]
n 2 b
[Equation]
( α
[Equation]
· 1 − (1 − [Equation] [Equation] [Equation]
n
[Equation]
b
[Equation]
n 2 b
[Equation]
1 − [Equation] n
[Equation]
+
[Equation]
[Equation]
( α ) (6) where ε j 0 ( α ) =
{+αηS 2 g 4 σ 2 b 2 · (1 − ηλ ) 2 · ( j − 1) · 1 (cid:80) j − 1 s =
+}
(1 − ηλ ) 2 s for any j = 1 , · · · , n b (we assume n b ≥ 2 ).
Proof Sketch.
We ﬁrst prove that the distribution of parameters θ jk satisﬁes LSI with a constant c jk thatdepends on k, j .
Then we plug the LSI constants into Lemma 4.2 and prove a recursive privacy boundfor data points in each batch B j 0 .
Finally, by carefully choosing which recursion to use and solve,we obtain the privacy bound in the theorem statement.
The complete proof is in Appendix C.5.
The above new privacy bound Theorem 4.3 quantiﬁes the privacy ampliﬁcation during iterations thatonly access the shared data point between neighboring datasets, and it is signiﬁcantly smaller thanthe naive privacy dynamics baseline derived from Chourasia et al.
[15] (that does not capture thisadditional privacy ampliﬁcation).
In Figure 1, we observe that for data points in the ﬁrst batch B 0 , ournew privacy dynamics bound is smaller by a multiplicative factor of approximately n/b (where b/nis the batch sampling ratio), and the improvement for data points in the last-batch is also signiﬁcant.
Bound improvement compared with Balle et al.
In the ﬁrst epoch (when K = 1 ), ourprivacy dynamics bound (based on recursive scheme) in Theorem 4.3 is of same order as the privacyampliﬁcation bound Balle et al.
[5, Theorem 5], as shown in Figure 1.
However, for training withmultiple epochs, Balle et al.
[5] use a coupling-based approach, and we do not see an easy way toextend their analysis except by using Rényi DP composition [27] over the epochs, which gives alinearly accumulating Rényi DP bound for multiple epochs (as K increases).
On the contrary, asshown in Figure 1, our improved privacy dynamics bound converges to a constant, thus signiﬁcantlyimproves over the Rényi DP composition of Balle et al.
[5, Theorem 5].
Privacy dynamics for noisy (stochastic) minibatch gradient descent
In this section, we further improve over the position-dependent privacy dynamics analysis in Section 4,by incorporating the effect of ampliﬁcation of privacy loss due to the stochastic minibatch sampling.
We ﬁrst prove the following lemma by the convexity of f -divergence with f ( x ) =
x α .
Lemma 5.1 (Joint convexity of scaled exponentiation of Rényi divergence) .
Let µ 1 , · · · , µ m andν 1 , · · · , ν m be distributions over R d .
Then for any RDP order α ≥ 1 , and any coefﬁcientsp 1 , · · · , p m ≥ 0 that satisfy p 1 + · · · + p m = 1 , the following inequality holds.
[Equation]
We provide a detailed proof for Lemma 5.1 in Appendix D.
This Lemma is our main tool forquantifying the additional privacy ampliﬁcation under stochastic minibatches in the rest of the section.
5.1 Privacy dynamics under shufﬂe and partition
The “shufﬂe and partition” batch generation scheme is commonly used for practical DP-SGDimplementations in privacy libraries [21, 42].
By using Theorem 4.3 and new bounds for the privacyampliﬁcation by shufﬂing in Algorithm 1, we prove the following Rényi DP guarantee.
Theorem 5.2 (Privacy dynamics under “shufﬂe and partition”) .
If the loss function (cid:96) ( θ ; x ) isλ -strongly convex, β -smooth, and if its gradient has ﬁnite (cid:96) 2 -sensitivity S g , then for nb ≥ 2 , running Algorithm 1 for K ≥ 1 epochs with step-size η < 2 λ + β , under “shufﬂe and partition” batchgeneration scheme, satisﬁes ( α, ε ) -Rényi DP with
[Equation]
where ε j 0 ( α ) =
αηS 2 g 4 σ 2 b 2 · (1 − ηλ ) {+2 · ( j − 1) · 1 (cid:80) j − 1 s =
+}
(1 − ηλ ) 2 s for any j = 1 , · · · , nb (we assume nb ≥ 2 ).
The above Rényi DP bound (8) in Theorem 5.2 is always smaller than the bound in Theorem 4.3 giventhe worst-case ﬁxed minibatch sequence (i.e., when the differing data point is in the last minibatchof each pass).
Therefore, Theorem 5.2 quantiﬁes the privacy ampliﬁcation due to shufﬂing, whencompared to the worst-case Rényi DP bound among all possible minibatch sequences.
We illustratethis ampliﬁcation in Figure 2 (a).
Moreover, our Rényi DP bound in Theorem 5.2 is strictly smallerthan composition-based privacy bound for DP-SGD [1, 28] after 4 · 1 λη +4 · nb epochs.
This is becausethe ﬁrst term in Equation (178) is strictly smaller than the composition-based privacy bound after4 · 1 λη epochs, and the second term in Equation (178) is strictly smaller than the composition-basedprivacy bound after 4 · ( n/b ) epochs (as we explain with more details in Appendix D.1). [Equation]
[Figure]
ε i n ( α , ε )
[Figure]
R é ny i D i ff e r e n ti a l P r i v ac y
[Figure] [Figure]
[Equation]
5.2 Privacy dynamics under sampling without replacement
We now similarly analyze privacy dynamics for another variant of noisy minibatch gradient descent(Algorithm 1), where we iteratively (re)sample b indices without replacement from { 1 , · · · , n } toobtain each minibatch.
That is, the minibatches used in different iterations (of one epoch) may overlap.
Theorem 5.3 (Recursive ampliﬁcation by sampling without replacement) .
If the loss function (cid:96) ( θ ; x )is λ -strongly convex, β -smooth, and if its gradient has ﬁnite (cid:96) 2 -sensitivity S g , then Algorithm 1 undersampling without replacement and stepsize η < 2 λ + β satisﬁes ( α, ε ) -Rényi
DP guarantee with
[Equation]
where the terms S jk ( α ) for k = 0 , · · · , K − 1 and j = 0 , · · · , n/b − 1 are recursively computed by
S 0 0 ( α ) = 1 ; S
j +1 k
[Equation]
S
[Equation]
S
[Equation]
b k
[Equation]
α ) .
In Figure 2 (b), we compare the above Rényi DP bound Theorem 5.3 with the baseline compositionbased privacy bound for DP-SGD [1, 27].
We observe that for RDP order α = 10 , 15 , our privacydynamics bound signiﬁcantly improves over the baseline composition-based bound after 50 epochs.
However, this improvement is smaller for large RDP order α = 20 .
Moreover, our privacy boundunder “sample without replacement” in Figure 2 (b) is larger than our privacy bound under “shufﬂe andpartition” in Figure 2 (a), which suggests room for future improvement of the bound in Theorem 5.3.
Evaluating privacy accuracy trade-off under improved privacy dynamics
In this section, we evaluate regularized logistic regression in a multi-class setting (with per-classbias).
We compare the optimal accuracy of ( ε, δ ) -DP models under our privacy dynamics bound,against the best known prior results [36] under DP-SGD [1, 28], for the same privacy budget ε and δ .
Experiment setting .
We run experiments on image classiﬁcation tasks.
The training dataset that wetake as input is D
=
( z 1 , · · · , z n ) , where each data record z i =
( x i , y i ) consists of the d -dimentionaldata input feature vector x i ∈ R d , and the label vector y i ∈ { 0 , 1 } c (in one-hot encoding).
To train a
[Table] [Table] [Table]
Reference
[Table]
This Paper [36] differentially private classiﬁer, we use Algorithm 1 under “shufﬂe and partition”, which is equivalentto practical implementations of DP-SGD [1] (as explained in Appendix E.1).
To satisfy the necessaryconditions for our privacy dynamics bound, we use regularized logistic regression to ensure strongconvexity, use feature clipping to ensure smoothness, and use gradient clipping to enforce ﬁnitegradient sensitivity (described in Appendix E.2, E.3 and E.4).
The formulas for these privacy bounds,are in Appendix E.5.
Our code is provided in the supplemental material.
Hyperparameter tuning .
In hyperparameter tuning, multiple accesses to the same private trainingdataset are required for training models under different hyperparameters, thus contributing to possiblyhigher privacy cost than the privacy bound for a single run of training algorithm.
However, most priorworks [1, 36] do not account for this additional privacy cost due to hyperparameter tuning.
Therefore,for a fair comparison with Tramèr and Boneh [36], we also do not account the privacy cost due tohyperparameter tuning.
Moreover, recent works [24, 30] also have proposed algorithms that reducethe privacy cost for hyperparameter tuning to a constant times the privacy cost of a single run.
6.1 Experiments on ScatterNet features
ScatterNet features
We follow Tramèr and Boneh [36] and similarly train regularized logisticregression models from scratch on hand-crafted ScatterNet [9, 29] features (extracted via dataindependent scale invariant feature transformation [25]).
This is because, to the best of our knowledge,
Tramèr and Boneh [36] provides the best known accuracy for logistic regression models trained fromscratch.
More details could be found in Section 2.1 and Appendix C.1 in Tramèr and Boneh [36].
Slow convergence of DP-SGD for training from scratch .
Tramèr and Boneh [36] show thatdifferentially private linear models trained on hand-crafted ScatterNet features improve the currentstate-of-the art test accuracy for end-to-end differentially private CNN models (both under DP-SGDanalysis).
We reproduce their test accuracy results for MNIST, Fashion-MNIST, and CIFAR-10 in thefollowing Table 1.
We observe that due to the restriction from the composition-based privacy bound,their training algorithm could only run for a limited number of epochs, e.g. 60 epochs for trainingon CIFAR-10.
However, the accuracy of the trained model continues to grow even after 60 epochs(discussed in detail in Appendix E.6), indicating that insufﬁcient number of allowed running epochsof DP-SGD restricts its accuracy for training differentially private model from scratch.
Improved accuracy under privacy dynamics Analysis .
Our tighter privacy bound in Corollary E.3allows the noisy minibatch gradient to run for signiﬁcantly more epochs than DP-SGD, within thesame ﬁxed privacy budget.
In Table 1, we show that our privacy dynamics bound enables in a betterprivacy accuracy trade-off for (regularized) logistic regression on hand-crafted features, than thatin Tramèr and Boneh [36] (which, to the best of our knowledge, is also the best reported test accuracyfor differentially private image classiﬁcation models trained from scratch in prior works).
[Table]
6.2 Experiments on Transfer Learning
In this section
[Table]
we empirically analyze the complementing setting of transfer learning, which typicallyonly requires a few epochs to reach a stable accuracy.
We use CIFAR-10 as the target private dataset,and differentially privately ﬁne-tune the last layer of the source pre-trained model (trained on anotherdataset), which is equivalent to training a logistic regression model on top of frozen second-to-lastlayer features of the source pre-trained model.
Source pre-trained model .
The problem of privately ﬁne-tuning a source model (trained from adifferent/public dataset) to a private target training dataset is well-studied [1, 31, 36].
The high qualityfeatures learned from pre-trained model usually enable a better privacy accuracy trade-off of thetarget learning task, than training from scratch.
We follow the setups in Tramèr and Boneh [36](which to the best our knowledge, attain model trained
13] model
Fast convergence of DP-SGD for ﬁne-tuning .
In our experiments, we observe that DP-SGD reachesstable accuracy within 50 epochs, for ﬁne-tuning pre-trained models on CIFAR-10.
This is muchfaster than training from scratch that typically requires a few times more epochs to converge.
Similar accuracy under privacy dynamics analysis .
Transfer learning tasks need relatively smallnumber of iterations to reach convergence.
However, our privacy dynamics bound is only smallerthan composition-based bound after O ( 1 λη + nb ) epochs (shown after Theorem 5.2), which suggeststhat composition may be tighter if the training only requires a few epochs.
Therefore, in Table 2, weobserve that our privacy dynamics analysis only enables slightly better privacy accuracy trade-off.
Conclusion
We prove a novel converging last-iterate privacy bound for noisy stochastic mini-batch gradientdescent on strongly convex smooth loss functions.
Our bound substantially improves the prior privacydynamics bound for noisy GD [15], by proving novel bounds for the additional privacy
ampliﬁcation(by post-processing and subsampling) during training with stochastic mini-batches.
We evaluate theprivacy accuracy trade-off for various learning tasks under our improved privacy dynamics bound,and compare it with that under DP-SGD.
We show that when the training process has a moderate toslow convergence, our privacy dynamics analysis enables better accuracy than that under compositionprivacy bound for DP-SGD.
This shows that to achieve the optimal privacy accuracy trade-off,differentially private learning needs to be evaluated by a last-iterate privacy bound, unless it has avery fast convergence (under which, in practice, the cost of composition bound is not signiﬁcant).
Future Work .
There is room for improving our privacy ampliﬁcation bound Theorem 5.2 undersmaller batch size b , because under one sampling step, the best known ampliﬁcation rate is as small as
O ( b 2 n 2 ) [28].
This suggests possibility for a better hidden-state privacy ampliﬁcation bound (withoutusing composition) for multiple iterations of sampling.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit
In your paper please delete this instructionsblock and only keep the Checklist section heading above along
with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes] See Section 1.
(b) Did you describe the limitations of your work?
[Yes] See Section 7.(c)
Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 4and 5.
(b) Did you include complete proofs of all theoretical results?
[Yes] See supplementalmaterial.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[Yes] See supplemen-
tal material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See supplemental material.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[Yes] See Table 1 and 2.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See supplemental material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See supplemental material.
(b) Did you mention the license of the assets?
[Yes] See supplemental material.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
See supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] See supplemental material.
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent
[Table]
[N/A]