R ECYCLING S CRAPS : I MPROVING P RIVATE L EARNING BY L EVERAGING I NTERMEDIATE C HECKPOINTS
Anonymous authors
Paper under double-blind review
A BSTRACT
All state-of-the-art (SOTA) differentially private machine learning (DP ML) methods are iterative in nature, and their privacy analyses allow publicly releasing the intermediate training checkpoints.
However, DP ML benchmarks, and even practical deployments, typically use only the ﬁnal training checkpoint to make predictions.
In this work, for the ﬁrst time, we comprehensively explore various methods that aggregate intermediate checkpoints to improve the utility of DP training.
Empirically, we demonstrate that checkpoint aggregations provide signiﬁcant gains in the prediction accuracy over the existing SOTA for CIFAR and StackOverﬂow datasets, and that these gains get magniﬁed in settings with periodically varying training data distributions.
For instance, we improve SOTA
StackOverﬂow accuracies to 22.7% (+0.43% absolute) for ε = 8 . 2 , and 23.84% (+0.43%) for ε = 18 .
Theoretically, we show that uniform tail averaging of checkpoints improves the empirical risk minimization bound compared to the last checkpoint of DP-SGD.
Lastly, we initiate an exploration into estimating the uncertainty that DP noise adds in the predictions of DP ML models.
We prove that, under standard assumptions on the loss function, the sample variance from last few checkpoints provides a good approximation of the variance of the ﬁnal model of a DP run.
Empirically, we show that the last few checkpoints can provide a reasonable lower bound for the variance of a converged DP model.
I NTRODUCTION
Machine learning models can unintentionally memorize sensitive information about the data they were trained on, which has led to numerous attacks that extract private information about the training data (Ateniese et al., 2013;
Fredrikson et al., 2014; 2015; Carlini et al., 2019; Shejwalkar et al., 2021;
Carlini et al., 2021; 2022).
For instance, membership inference attacks (Shokri et al., 2017) can infer whether a target sample was used to train a given ML model, while property inference attacks (Melis et al., 2019; Mahloujifar et al., 2022) can infer certain sensitive properties of the training data.
To address such privacy risks, literature has introduced various approaches to privacy-preserving
ML (Nasr et al., 2018; Shejwalkar & Houmansadr, 2021; Tang et al., 2022).
In particular, iterative techniques like differentially private stochastic gradient decent (DP-SGD) (Song et al., 2013; Bassily et al., 2014; Abadi et al., 2016b; McMahan et al., 2017) and DP Follow The Regularized Leader (DP-FTRL)
(Kairouz et al., 2021) have become the state-of-the-art for training DP neural networks.
For establishing benchmarks, prior works in DP ML (Abadi et al., 2016b; McMahan et al., 2017; 2018; Thakkar et al., 2019; Erlingsson et al., 2019;
Wang et al., 2019b; Zhu & Wang, 2019; Balle et al., 2020; Erlingsson et al., 2020; Papernot et al., 2020; Tramer & Boneh, 2020; Andrew et al., 2021; Kairouz et al., 2021; Amid et al., 2022; De et al., 2022; Feldman et al., 2022) use only the ﬁnal model output by the DP algorithm.
This is also how DP models are deployed in practice (Ramaswamy et al., 2020; McMahan et al., 2022).
However, the privacy analyses for the techniques used allow releasing/using all of the intermediate training checkpoints.
In this work, we comprehensively study various methods that leverage intermediate checkpoints to 1) improve the utility of DP training, and 2) quantify the uncertainty in DP ML models that is due to the DP noise.
Accuracy improvement using checkpoints: We propose two classes of aggregation methods based on aggregating the parameters of checkpoints, or their outputs .
We provide both theoretical and em- pirical analyses for our aggregation methods.
Theoretically, we show that excess empirical risk of the ﬁnal checkpoint of DP-SGD is log ( n ) times more than that of the weighted average of the past k checkpoints.
Here, n is the size of dataset.
Empirically, we demonstrate signiﬁcant top-1 accuracy gains due to our aggregations for image classiﬁcation (CIFAR10) and a next word prediction (StackOverﬂow) tasks.
Speciﬁcally , we show that our checkpoints aggregations achieve absolute (relative) prediction accuracy improvements of 3.79% (7.2%) at ε = 1 for CIFAR10 (DP-SGD), and 0.43% (1.9%) at ε = 8 . 2 for the StackOverﬂow (DP-FTRLM) SOTA baselines, respectively.
We also show that our aggregations signiﬁcantly reduce the variance in the performance of DP models over training
Finally, we show that these beneﬁts further magnify in more practical settings with periodically varying training data distributions.
For instance, we note absolute (relative) accuracy gains of 17.4% (28.6%) at ε = 8 for CIFAR10 over DP-SGD baseline in such a setting.
Uncertainty quantiﬁcation using checkpoints: There are various sources of randomness in a ML training pipeline (Abdar et al., 2021), e.g., choice of initial parameters, dataset, batching, etc.
This randomness induces uncertainty in the predictions made using such ML models.
In critical domains, e.g., medical diagnosis, self-driving cars and ﬁnancial market analysis, failing to capture the uncertainty in such predictions can have undesirable repercussions.
DP learning adds an additional source of randomness by injecting noise at every training round.
Hence, it is paramount to quantify reliability of the DP models, e.g., by quantifying the uncertainty in their predictions.
To this end, we take the ﬁrst steps towards quantifying the uncertainty that DP noise adds to DP ML training.
As prior work, Karwa & Vadhan (2017) develop ﬁnite sample conﬁdence intervals but for the simpler Gaussian mean estimation problem.
Various methods exist for uncertainty quantiﬁcation in ML-based systems (Mitchell, 1980; Roy et al., 2018; Begoli et al., 2019; Hubschneider et al., 2019; McDermott & Wikle, 2019; Tagasovska & Lopez-Paz, 2019; Wang et al., 2019a; Nair et al., 2020; Ferrando et al., 2022).
However, these methods either use specialized (or simpler) model architectures to facilitate uncertainty quantiﬁcation, or are not directly applicable to quantify the uncertainty in DP ML due to DP noise.
For e.g., a common way of uncertainty quantiﬁcation (Barrientos et al., 2019; Nissim et al., 2007; Brawner & Honaker, 2018; Evans et al., 2020) that we call the independent runs method, needs k independent (bootstrap) runs of the ML algorithm.
However, repeating a DP ML algorithm multiple times can incur signiﬁcant privacy and computation costs.
To address the above issue, we propose to use the last k checkpoints of a single run of a DP ML algorithm as a proxy for the k ﬁnal checkpoints from independent runs.
This does not incur any additional privacy cost to the DP ML algorithm.
Furthermore, it is useful in practice as it does not incur additional training compute, and can work with any algorithm having intermediate checkpoints.
Theoretically, we consider using the sample variance of a statistic f ( θ ) at checkpoints θ t 1 , . .
, θ t k as an estimator of the variance of f ( θ t k ) , i.e., the statistic at the ﬁnal checkpoint, and give a bound on the bias of this estimator.
As expected, our bound on the bias decreases as the “ burn-in ” time t 1 as well as the time between checkpoints both increase.
Intuitively, our proof shows that (i) as the burnin time increases, the marginal distribution of each θ t i approaches the distribution of θ t k , and (ii) as the time between checkpoints increases, any pair θ t i , θ t j approaches pairwise independence.
Both (i) and (ii) are proven via a mixing time bound, which shows that starting from any point distribution θ 0 , the Markov chain given by DP-SGD approaches its stationary distribution at a certain rate.
Empirically, we show our method provides reasonable lower bounds on the uncertainty quantiﬁed using the more accurate (but privacy and computation intensive) method that uses independent runs.
Related work on Checkpoint aggregations: (Chen et al., 2017; Izmailov et al., 2018) explore checkpoint aggregation methods to improve performance in (non-DP) ML settings, but observe negligible performance gains.
To our knowledge, De et al.
(2022) is the only work in the DP ML literature that uses intermediate checkpoints post training.
They apply an exponential moving average (EMA) over the checkpoints of DP-SGD, and note non-trivial gains in performance.
However, we propose various aggregation methods that outperform EMA on standard benchmarks.
I MPROVING ACCURACY BY AGGREGATING DP TRAINING CHECKPOINTS
In this section, we describe our checkpoint aggregation methods, followed by the experimental setup we use for evaluation.
Next, we detail our experimental results that demonstrate the signiﬁcant gains in accuracy of DP ML models due to checkpoints aggregations.
DP Preliminaries: Differential Privacy (DP) (Dwork et al., 2006) is a notion to quantify the privacy leakage from the outputs of a data analysis procedure.
A randomized algorithm M : D ∗ → Y is ( ε, δ ) -DP
if, for all neighbouring datasets D, D (cid:48) ∈ D ∗ (i.e., datasets that differ in one data sample) and all measurable sets of outputs S ⊆
Y , we have P [ M ( D ) ∈ S ]
≤
e ε · P [ M ( D (cid:48) )
∈ S ]+ δ .
We add
Gaussian noise to functions of bounded sensitivity to ensure DP.
We also use DP’s post-processing property , i.e., any analysis on the outputs of a DP algorithm does not worsen its DP guarantees.
We consider two state-of-the-art DP ML algorithms: 1) DP-SGD for the central learning setting, i.e., when all data is pooled at a central location, e.g., a server, and 2) DP-FTRL for the federated learning (FL) setting, i.e., when all private data is distributed across remote collaborative devices.
The privacy analyses for both of these techniques involves composition across training rounds, allowing the release of all intermediate checkpoints computed during training.
2. C HECKPOINTS AGGREGATION METHODS
We propose two types of aggregation methods: parameter aggregation , and output aggregation .
Parameter aggregations compute a function of the parameters of intermediate checkpoints from a DP run, and then use the resulting aggregate parameters for inference.
On the other hand, output aggregations compute a function of the outputs of the intermediate checkpoints, and use it for making predictions.
These classes can encompass a vast majority of possible aggregation algorithms, but for brevity, we experiment with two algorithms in each of the classes: exponential moving average and uniform past k average that aggregate parameters, and predictions average and labels majority vote that aggregate outputs.
Note that all of our aggregation algorithms post-process the checkpoints, and hence, do not incur any additional privacy cost.
Additionally, our aggregation methods are general, i.e., they are applicable to any training algorithm that computes intermediate checkpoints.
2.1. P ARAMETER AGGREGATION METHODS
Exponential moving average (EMA): EMA has been previously used (Tan & Le, 2019; Brock et al., 2021) to improve the performance of ML models at inference time.
Starting from the last checkpoint of the run, EMA assigns exponentially decaying weights to each of previous checkpoints; the weights are a function of the EMA coefﬁcient β t at step t .
During training, at each step t , EMA maintains a moving average θ tema that is a weighted average of θ t − 1 ema and the t th checkpoint, θ t .
This is formalized as follows:
θ tema =
(1 − β t ) · θ t − 1 ema + β t · θ t .
Following (Tan & Le, 2019; De et al., 2022), we use a warm-up schedule for the EMA coefﬁcient as: β t = min (cid:16) β, 1+ t 10+ t (cid:17) .
Uniform past k average (UPA): For step t of training, UPA computes the mean of the past k checkpoints, i.e., checkpoints from steps [ t − ( k − 1) , t ] .
We formalize this as: θ t,kupa = 1 k (cid:80) ti = t − ( k − 1) θ i .
. O UTPUT AGGREGATION METHODS
Output predictions averaging (OPA): For a given test sample x , OPA ﬁrst computes prediction vectors f
θ i ( x ) of the past k checkpoints, i.e., checkpoints from steps ∈ [ t − ( k − 1) , t ] , averages the prediction vectors, and computes argmax of the average vector as the ﬁnal output label.
We formalize OPA as ˆ y opa ( x ) = argmax (cid:16) 1 k (cid:80) ti = t − ( k − 1)
f
θ i ( x ) (cid:17) .
Output labels majority vote (OMV):
For a given test sample x , OMV computes output prediction vectors for x and the corresponding labels, i.e., argmax f
θ i ( x ) .
Finally, it uses the majority label among the k labels (breaking ties arbitrarily) as its ﬁnal output label.
We formalize OPA as ˆ y omv ( x ) =
Majority (cid:0) argmax ( f θ i ( x )) ti =
t − ( k − 1) (cid:1) .
2. E XPERIMENTS 2.2. E XPERIMENTAL SETUP
Datasets: We evaluate our checkpoints aggregation algorithms on three benchmark datasets in two different settings: CIFAR10 (Krizhevsky et al., 2009) and CIFAR100 for image classiﬁcation, and simulated federated learning over StackOverﬂow (Kaggle, 2018) for next word prediction.
Privacy guarantees are sample-level for CIFAR tasks and user-level for StackOverﬂow (as it is keyed by users).
For experiments with DP, we ﬁx the privacy parameter δ to 10 − 5 on CIFAR-10/CIFAR100, and 10 − 6 on StackOverﬂow, ensuring that δ < n − 1 , where n is the number of examples in
CIFAR10/CIFAR-100 and the number of users in StackOverﬂow.
Model architectures, and training details:
Following the setup of the state-of-the-art (SOTA) in (De et al., 2022), we train a WideResNet-16-4 with depth 16 and width 4 using DP-SGD (Abadi et al., 2016b) in JAXline (Babuschkin et al., 2020) for ε ∈ { 1 , 8 } .
For CIFAR100, we follow (De et al., 2022) and ﬁne-tune the last, classiﬁer layer of a WideResNet-28-10 pre-trained on ImageNet data.
For StackOverﬂow, we follow SOTA in (Kairouz et al., 2021; Denisov et al., 2022) and train a one-layer LSTM using DP-FTRL in TFF (Abadi et al., 2016a) for ε ∈ { 8 . 2 , 18 . 9 } .
For UPA, OPA and OMV, we treat k , the number of checkpoints to aggregate, as a hyperparameter and tune it using validation data.
All our results are computed over 5 runs of each setting.
We provide additional details of our experimental setup in Section B.1.
2.2. E XPERIMENTAL RESULTS
First, we discuss the results for original datasets and then for more practical periodic distribution shifting datasets.
Below, the tables present results for the ﬁnal training round/step, i.e., for the aggregates computed until the last step/round, while plots show results over the last k rounds for some k much smaller than total number of training steps/rounds.
For StackOverﬂow, due to large size of its test data, we provide plots for accuracy on validation data and tables with test accuracy.
CIFAR10 results: Table 1 and the left-most two plots in Figure 1 present the accuracy gains in
CIFAR10 for ε ∈ { 1 , 8 } .
For ε = 1 , OPA provides the maximum accuracy gain of 3.79%, while for ε = 8 , EMA provides maximum gain of 2.23%.
1 We note from Figure 1 that all checkpoints aggregations improve accuracy for all the training steps of DP-SGD for both ε ’s. Next, note from
Figure 1 that the accuracy of baseline DP-SGD has a high variance across training steps, i.e., based on the hyperparameters, DP-SGD can produce bad/good models which can be undesirable in practice.
However, checkpoints aggregations signiﬁcantly reduce the variance in accuracy of models, and therefore, increase the conﬁdence in the ﬁnal DP-SGD model.
[Figure]
[Table] [Figure]
StackOverﬂow results: Table 1 and the rightmost two plots in Figure 1 present the accuracy gains in
StackOverﬂow for ε ∈ { 18 . 9 , 8 . 2 } ; Figure 5 present the results for ε = ∞ .
The maximum accuracy gains are 0.57%, 0.43%, and 0.42% for ε of ∞ , 18.9, and 8.2, respectively.
We observe that UPA aggregation always provides the best accuracy.
Note that these improvements are very signiﬁcant, because there are 10,004 classes in StackOverﬂow data.
Similarly for CIFAR10, Figures 1 andshow that checkpoints aggregations improve accuracy for all training rounds by large margins and also signiﬁcantly reduce the variance of model accuracy across training rounds.
Note that these improvements are on top of the momentum optimizer used during training.
CIFAR100 results: Due to space constraints, we defer the results to Table 7 in Appendix B, and discuss the major observations here.
First, we improve the SOTA baseline of De et al.
(2022) 2 from
[Table]
=
[Table]
= [Table] [Table] [Table]
[Table]
70.6% to 75.51% at ε =
[Table]
and from 77.6% to 80.8% at ε = 8 .
To achieve these gains [Table] we perform ﬁne-tuning over the EMA checkpoint of ImageNet-trained WRN-28-10 instead of the ﬁnal checkpoint as in (De et al., 2022).
Subsequently [Table] we observe that for ﬁne-tuning using CIFAR100, checkpoint aggregations provide small accuracy gains: for both ε of 1 and 8, we observe maximum gains are 0.11% due to UPA and OPA, respectively.
2.2. A CCURACY IMPROVEMENTS IN PERIODIC DISTRIBUTION SHIFTING SETTINGS
In many real-world settings, for instance, in federated learning (FL) settings [Table] training data distribution may vary over time.
Zhu et al.
(2021) demonstrate the adverse impacts of distribution shifts in training data on the performances of resulting FL models.
Considering the practical signiﬁcance of such distribution shifts, we consider settings where the training data distribution has diurnal variations, i.e., it is a function of two oscillating distributions, D 1 and D 2 ; Figure 2 shows these distributions.
Such a scenario commonly occurs in FL training, e.g., when a model is trained using
FL with client devices participating from two signiﬁcantly different time zones.
[Figure]
Experimental setup: Following Zhu et al.
(2021), we consider a setting where training data is a combination of clients/samples drawn from two disjoint data distributions, which oscillate over time as Figure 2 shows.
Here the probability of sampling from, e.g., D 1 , at time t is p ( D 1 , t ) =
(cid:12)(cid:12) 2 t mod T T − 1 (cid:12)(cid:12) , where T is the period of oscillation of D { 1 , 2 } .
To simulate such diurnal distribution, for CIFAR datasets, we design D { 1 , 2 } such that D 1 and D 2 respectively contain the data from even and odd classes of the original data.
For StackOverﬂow, we design D { 1 , 2 } such that D 1 only has the questions from each user, while D 2 only has answers from them.
Then, we draw clients from D { 1 , 2 } .
Apart from data distribution, the rest of experimental setup is the same as before.
We use test and validation data same as for the original StackOverﬂow setting
CIFAR10 results: Table 2 and Figure 3 (left two plots) show accuracy gains for diurnal CIFAR10.
We note very large accuracy gains for both ε ∈ { 1 , 8 } ; the absolute accuracy gains are 7.45% and 17.37%, respectively, and are due to OPA.
Observe that, such diurnal settings have large variances in accuracy across training steps, but our aggregation methods almost completely eliminate the variances.
We note that the improvements in diurnal settings are signiﬁcantly more than that in original CIFAR10.
This is because in diurnal settings, the variances in model accuracy over training steps is very large, and hence, the beneﬁts of checkpoints aggregations magnify in these settings.
StackOverﬂow results: Table 2 and Figure 3 (rightmost two plots) present accuracy gains for diurnal StackOverﬂow.
The maximum accuracy gains are 0.09%, 0.44%, and 0.51% for ε of ∞ , 18.9, and 8.2, respectively.
In contrast to the diurnal CIFAR10 case, improvements in diurnal StackOverﬂow and StackOverﬂow are similar.
This is because the two distributions in diurnal CIFAR (completely different images from even/odd classes) are signiﬁcantly farther apart compared to the two distributions in diurnal StackOverﬂow (text from questions/answers). [Table] [Figure] [Table]
CIFAR100 results: Due to space constraints, we defer full results in Table 8 in Appendix B.
For
CIFAR100 also, we observe signiﬁcant gains due to checkpoints aggregations in PDS setting: for ε of 1 and 8, we note maximum gains of 4.96% (due to UPA) and 3.37% (due to EMA), respectively.
Similar to CIFAR10, the gains due to checkpoints aggregation are signiﬁcantly higher for PDSCIFAR100 than for non-PDS CIFAR100. [Table]
2. D ATA - DEPENDENT CHECKPOINT SELECTION FOR AGGREGRATION
So far
[Table] our aggregated methods operated over a ﬁxed sequence of checkpoints, e [Table] g., past k .
However, when some public data similar to the private training data is available [Table] we propose a data-dependent checkpoint selection method for aggregation: aggregate the checkpoints that perform the best on the (held-out) public data.
We expect this to improve over data-independent aggregation schemes.
We validate this hypothesis for StackOverﬂow dataset.
We use 10,000 random samples (less than 0.01% of the training data) from the StackOverﬂow validation data as the held-out public data.
Note that this is disjoint from the validation data that we use for hyperparameter tuning.
We omit the CIFAR datasets from this evaluation due to the lack of availability of additional similar data.
We compute accuracy of all training checkpoints on the held-out data, ﬁnd the best k checkpoints, and aggregate them as detailed above.
We tune the hyperparameter k using disjoint validation data.
Table 3 presents the gains due to data-dependent aggregations over the baseline and dataindependent aggregations; Tables 5 and 6 in Appendix B present full results.
We see that data- dependent aggregations outperform data-independent aggregations for all ε ’s. For instance, at ε of 8.2, accuracies of baseline, data-independent and data-dependent aggregations are 22.28%, 22.7% and 22.8%, respectively.
For ε ’s ∞ and 18.9, aggregating best past k checkpoints outperform aggregating past k checkpoints (baseline) by 0.16% (0.73%) and 0.08% (0.51%), respectively.
We make the same observation for diurnal PDS StackOverﬂow dataset—the maximum accuracy gains due to data-dependent aggregations are 0.29%, 0.58%, and 0.66% for ε of ∞ , 18.9, and 8.2, respectively.
2. I MPROVED E XCESS R ISK VIA TAIL AVERAGING
In this section, we formally show that checkpoint aggregations like uniform tail averaging provably improves the privacy/utility trade-offs, compared to that of the last checkpoint of DP-SGD.
To formalize the problem, we deﬁne the following notation: Consider a data set D
= { d 1 , . . .
, d n } and a loss function L ( θ ; D ) = 1 n n (cid:80) i =1 (cid:96) ( θ ; d i ) , where each of the loss function (cid:96) is convex and L -Lipschitz in the ﬁrst parameter, and θ ∈ C with C ⊆ R p being a convex constraint set.
We analyze the following variant of DP-SGD, which is guaranteed to be ρ -zCDP 4 .
θ 0 ← 0 p .
For t ∈ [ T ] , θ t +1 ← Π C
( θ t − η t ( ∇L ( θ t ; D ) + b t )) , where b t ∼ N (cid:16) 0 , L 2 T 2 nρ I p × p (cid:17) and Π C ( · ) being the (cid:96) 2 -projection onto the set C .
We will provide the utility guarantee for this algorithm by directly appealing to the result of Shamir & Zhang (2013).
For a given α ∈ (0 , 1) , UPA (Section 2.1.1) corresponds to the average of the last αT models, i.e., θ privupa = 1 αT
T [Equation] θ t .
One can also consider polynomial-decay averaging (PDA) with parameter γ ≥ 0 , deﬁned as follows:
[Equation]
For γ = 0 , PDA matches UPA over all iterates.
As γ increases, PDA places more weight on later iterates; in particular, if γ = cT , the averaging scheme is very similar to EMA (Section 2.1.1), since as t → T the decay parameter γ +1 t + γ approaches a constant cc +1 .
In that sense, PDA can be viewed as a method interpolating between UPA and EMA.
Theorem 2.1 (Adapted from Theorems 2 and 4 of Shamir & Zhang (2013)) .
Consider the DP-SGD algorithm above, and the associated parameters.
Then there exists choice of learning rate η t and the number of time steps T s.t.
the following are true for α = Θ(1) :
[Equation]
D
[Equation]
(cid:17) . Furthermore, for γ = Θ(1) , we have, E (cid:104) L (cid:16) θ privpda [ T ]; D (cid:17)(cid:105) − min θ
[Equation]
L ( θ ; D ) = O (cid:16)
[Equation]
Proof.
If we choose T = (cid:100) nρ (cid:101) and set η t appropriately, the proof of Theorem 2 (Shamir & Zhang, 2013) implies the following for
[Equation]
Setting α = Θ(1) gives the theorem’s ﬁrst part, and αT = 1 , i.e., 1 /α =
T = (cid:100) nρ (cid:101) gives the second.
The third follows from modifying Theorem 4 of Shamir & Zhang (2013) for the convex case (see the end of Section 4 of Shamir & Zhang (2013) for details).
The excess empirical risk for θ T is higher by factor of log( n ) in comparison to θ privupa and θ privpda [ T ] .
For the step size selections typically used in practice (e.g., ﬁxed or inverse polynomial step sizes), the last iterate will suffer from the extra log( n ) factor, and we do not know how to avoid it.
Furthermore, Harvey et al.
(2019) showed that this is unavoidable in the non-private, high probability regime.
Q UANTIFYING UNCERTAINTY IN PREDICTIONS DUE TO DP NOISE
In this section, we discuss our proposal of how to quantify the uncertainty that the differential privacy (DP) noise adds to the outputs of ML algorithms, without additional privacy cost or computation.
First, we discuss the issues with the current established approach of uncertainty quantiﬁcation in ML when used in DP setting, and then discuss our proposal and theoretical results.
Finally, we provide experimental results that demonstrate the practical utility of our approach.
(Naive) uncertainty quantiﬁcation, and its hurdles: As discussed in Section 1, although uncertainty quantiﬁcation has a long history of research, prior methods, including the most common independent runs method, are not applicable in DP settings.
The two major issues with the independent runs method in DP settings are: First, the additional runs of a DP-ML algorithm can incur a signiﬁcant increase in the privacy cost.
Second, the additional computation required in independent runs method can be prohibitive, e.g., in production FL.
Hence, it is not feasible to use the naive method for uncertainty quantiﬁcation in DP settings.
3. T WO BIRDS , ONE STONE : O UR UNCERTAINTY QUANTIFICATION PROPOSAL
To address the two hurdles discussed above, we propose a simple yet efﬁcient method that leverages intermediate checkpoints computed during a DP run.
Speciﬁcally, we substitute the k output models from the independent runs method with k checkpoints, e.g., the last k checkpoints, from a single DP run.
The rest of the conﬁdence interval computation is the same for both the methods.
Speciﬁcally, we compute the most likely classiﬁcation labels of the k models/checkpoints on a test input.
Use the set of k output labels as a sample from the student’s t -distribution and compute a conﬁdence interval of the mean of the distribution.
Finally, we use the average of multiple such intervals over multiple test inputs as a proxy for uncertainty of outputs of the DP ML algorithm.
3.1. T HEORY
In this section, we give a theoretical motivation for using checkpoints to estimate the variance of a statistic.
We show that under certain assumptions, if we take the sample variance of a statistic f ( θ ) computed at the checkpoints θ t 1 , . . .
, θ t k , in expectation it is a good approximation of the sample variance of the limiting distribution of DP-SGD.
As expected, the quality of the approximation improves by increasing the length of the “burn-in” time as well as the time between checkpoints.
To simplify the proof, we actually prove the theorem for DP-LD, a continuous-time version of DPSGD.
DP-LD can be deﬁned as follows.
We ﬁrst reformulate (unconstrained) DP-SGD with step size η as: (cid:101) θ ( t
[Equation] 1) η ← (cid:101) [Equation]
Notice that we have reparameterized (cid:101) θ so that its subscript refers to the sum of all step-sizes so far, i.e. after t iterations we have (cid:101) θ tη and not (cid:101) θ t .
Also notice that the variance of the noise we added is proportional to the step size η .
In turn, for any η that divides t , after t/η iterations with step size η , the sum of variances of noises added is 2 tσ 2 .
Now, taking the limit as η goes to 0 of the sequence of random variables { (cid:101) θ tη } t ∈ Z ≥ 0 deﬁned by DP-SGD, we get a continuous sequence { θ t } t ∈ R ≥ 0 .
In particular, if we ﬁx some t , then θ t is the limit as η goes to 0 of (cid:101) θ t deﬁned by DP-SGD with step size η .
This sequence is exactly the sequence deﬁned by DP-LD, which is more formally given by the following stochastic differential equation:
[Equation]
Here, W t is a standard Brownian motion and σ 2 is analogous to the variance of noise added in
DP-SGD.
In Appendix A we show the following:
Theorem 3.1 (Simpliﬁed version of Theorem A.1) .
Suppose L is 1-strongly convex and M -smooth, and σ = 1 in (1) .
Let 0 < t 1 < t 2 < . . .
<
t k be such that t
i +1 ≥ t i + γ, ∀ i > 0 and some γ .
Let { θ t i : i ∈ [ k ] } be the checkpoints, and f : Θ → [ − 1 , 1] be a statistic whose variance we wish to estimate.
Let S be the sample variance of f over the checkpoints, and V be the variance of f ( θ t k ) , i.e. the statistic at the ﬁnal checkpoint.
Then, for some “burn-in” time B that is a function of θ 0 , M, p , we have: | E [ S ] − V | = exp( − Ω(min { t 1 , γ } − B )) .
Intuitively, Theorem A.1 and its proof say the following: (i) As we increase t 1 , the time before the ﬁrst checkpoint, each of the checkpoints’ marginal distributions approaches the distribution of θ t k , and (ii) As we increase γ , the time between checkpoints, the checkpoints’ distributions approach pairwise independence.
So increasing both t 1 and γ causes our checkpoints to approach k pairwise independent samples from the distribution of θ t k , i.e., our variance estimator approaches the true variance in expectation.
To show both (i) and (ii), we build upon past results from the sampling literature to show a mixing bound of the following form: running DP-LD from any point initializationθ 0 , the R´enyi divergence between θ t and the limit as t → ∞ of DP-LD, θ ∞ , decays exponentially in t .
This mixing bound shows (i) since if t 1 is sufﬁciently large, then the distributions of all of θ t 1 , θ t 2 , . . .
, θ t k are close to θ ∞ , and thus close to each other.
This also shows (ii) since DP-LD is a Markov chain, i.e. the distribution of θ t j conditioned on θ t i is equivalent to the distribution of θ t j − t i if we run DP-LD starting from θ t i instead of θ 0 .
So our mixing bound shows that even after conditioning on θ t i , θ t j has distribution close to θ ∞ .
Since θ t j is close to θ ∞ conditioned on any value of θ t i , then θ t j is almost independent of θ t i .
[Figure]
(independent) runs, and using the last N checkpoints of a single run.
Empirical analysis: We compare the uncertainty quantiﬁed using the independent runs method and using our method; experimental setup is the same as in Section 2.2.1.
First, for both StackOverﬂow and CIFAR10 datasets, we do 100 independent training runs.
Then to compute uncertainty using the independent runs method, we take the ﬁnal model from k of these runs (chosen randomly), compute prediction labels for a given input, compute conﬁdence interval widths for the input, and ﬁnally, use the average of conﬁdence interval widths of a large number of inputs as the ﬁnal uncertainty estimate.
To compute uncertainty using our checkpoints based method, we instead select our k models to be the last k checkpoints of a random training run, and obtain average conﬁdence intervals as before.
Figure 4 shows the results for StackOverﬂow and CIFAR10.
Plots show intervals averaged overruns, i.e., by sampling k ﬁnal models 5 times or by using the last k models of 5 random runs.
We observe that uncertainty computed using the intermediate checkpoints consistently give a reasonable lower bound on the uncertainty computed using the independent runs.
C ONCLUSIONS
In this work, we explore methods for aggregating intermediate checkpoints to improve the utility of
DP training.
Using checkpoints aggregation methods, we show signiﬁcant improvements in prediction accuracy over the SOTA for CIFAR and StackOverﬂow datasets.
We also show that uniform tail averaging of checkpoints improves the ERM bound compared to the last checkpoint of DP-SGD.
Lastly, we prove that for some standard loss functions, the sample variance from last few checkpoints provides a good approximation of the variance of the ﬁnal model of a DP run.
We also conduct experiments demonstrating that the last few checkpoints can provide a reasonable lower bound for the variance of a converged DP model.
For future work, using checkpoint aggregates during DP training could be an interesting direction to further improve its utility.
Leveraging intermediate checkpoints to provide variance estimates for checkpoint aggregates could also be a promising direction
A P ROOF OF T HEOREM A.
For convenience/formality, we review the setup for the theorem we wish to prove.
Recall that given a loss function L , by taking the limit as η goes to 0 in DP-SGD, we recover the stochastic differential equation (1), restated here for convenience:
[Equation]
Note that the solutions θ t to this equation are random variables.
A key property of (1) is that the stationary distribution (equivalently, the limiting distribution as t → ∞ ) has pdf proportional to exp( −L ( θ ; D ) /σ ) under mild assumptions on L (which are satisﬁed by strongly convex and smooth functions).
To simplify proofs and presentation in the section, we will assume that (a) θ 0 is a point distribution, (b) we are looking at unconstrained optimization over R p , i.e., there is no need for a projection operator in DP-SGD and (1), (c) the loss L is 1-strongly convex and M -smooth, and (d) σ = 1 .
We note that (a) can be replaced with θ 0 being sampled from a random initialization without too much work, and (c) can be enforced for Lipschitz, smooth functions by adding a quadratic regularizer.
We let θ ∗ refer to the (unique) minimizer of L throughout the section.
Now, we consider the following setup: We obtain a single sample of the trajectory { θ t : t ∈ [0 , T ] } .
We have some statistic f : Θ → [ − 1 , 1] , and we wish to estimate the variance of the statistic for the ﬁnal value θ T , i.e. the variance V :=
Var ( f ( θ T )) .
To do so, we use the sample variance of the checkpoints at times 0 < t 1 < t 2 < t 3 < . . .
<
t k = T .
That is, our estimator is deﬁned as
[Equation]
2 where (cid:98) µ = 1 k (cid:80) ki =1 f ( θ t i ) .
Theorem A.1.
Under the preceding assumptions/setup, for some sufﬁciently large constant c , let γ = 12 M + ln( cM ( p + ln(1 / ∆) + (cid:107) θ 0 − θ ∗ (cid:107) 22 ))
+
c ln(1 / ∆) (recall that p is the dimensionality of the space).
Then, if t 1 > γ and t i +1 >
t i + γ for all i > 0 , for S, V as deﬁned above:
[Equation]
Before proving this theorem, we need a few helper lemmas about R ´ enyi divergences:
Deﬁnition A.2.
The R´enyi divergence of order α > 1 between two distributions P and Q (with support R d ), D α ( P , Q ) , is deﬁned as follows:
[Equation]
We refer the reader to e.g. van Erven & Harremos (2014);
Mironov (2017) for properties of the
R ´ enyi divergence.
The following property shows that for any two random variables close in R ´ enyi divergence, functions of them are close in expectation:
Lemma A.3.
[Adapted from Lemma C.2 of Bun & Steinke (2016)] Let P and Q be two distributions on Ω and g : Ω → [ − 1 , 1] .
Then,
[Equation]
Here, D 2 ( P||Q ) corresponds to R´enyi divergence of order two between the distributions P and Q .
The next lemma shows that the solution to (1) approaches θ ∞ exponentially quickly in R´enyi divergence.
Lemma A.4.
Fix some point θ 0 .
Assume L is 1-strongly convex, and M -smooth.
Let P be the distribution of θ t according to (1) for σ = 1 and:
[Equation]
Where c is a sufﬁciently large constant.
Let Q be the stationary distribution of (1) .
Then:
[Equation]
The proof of this lemma builds upon techniques in Ganesh & Talwar (2020), and we defer it to the end of the section.
Our ﬁnal helper lemma shows that θ ∞ is close to θ ∗ with high probability:
Lemma A.5.
Let θ ∞ be the random variable given by the stationary distribution of (1) for σ = 1 .
If L is 1-strongly convex, then:
[Equation]
Proof.
We know the stationary distribution has pdf proportional to exp( −L ( θ t ; D )) .
In particular, since L is 1-strongly convex, this means θ ∞ is a sub-Gaussian random vector (i.e., its dot product with any unit vector is a sub-Gaussian random variable), and thus the above tail bound applies to it.
We now will show that under the assumptions in Theorem A.1, every checkpoint is close to the stationary distribution, and that every pair of checkpoints is nearly pairwise independent.
Lemma A.6.
Under the assumptions/setup of Theorem A.1, we have:
[Equation]
Proof.
We assume without loss of generality ∆ is at most a sufﬁciently small constant; otherwise, since f has range [ − 1 , 1] , all of the above quantities can easily be bounded by 2, so a bound of O (∆) holds for any distributions on { θ t i } .
For (E1), by triangle inequality, it sufﬁces to prove a bound of O (∆) on | E [ f ( θ t i )] − E [ f ( θ ∞ )] | .
We abuse notation by letting θ t denote both the random variable and its distribution.
Then:
[Equation]
In ( ∗ 1 ) we use the data-processing inequality (Theorem 9 of van Erven & Harremos (2014)), and in ( ∗ 2 ) we use the fact e x − 1 ≤ 2 x, x ∈ [0 , 1] and our assumption on ∆ .
(E2) follows from (E1) by just using f 2 (which is still bounded in [ − 1 , 1] ) instead of f .
For (E3), note that since (1) is a (continuous) Markov chain, the distribution of θ t j conditioned on θ t i is the same as the distribution of θ t j − t
i according to (1) if we start from θ t i instead of θ 0 .
Let P be the joint distribution of θ t i , θ t j .
Let Q be the joint distribution of θ t i , θ ∞ (since (1) has the same stationary distribution regardless of its initialization, this is a pair of independent variables).
Let
P (cid:48) , Q (cid:48) be deﬁned identically to P , Q , except when sampling θ t i ,
[Equation]
(cid:107)
[Equation]
∆) we instead set θ t i = θ ∗ (and in the case of P (cid:48) , we instead sample θ t j from θ t j | θ t i = θ ∗ when this happens).
Let R denote this distribution over θ t i .
Then similarly to the proof of (E1)
we have:
[Equation]
Here ( ∗ 3 ) follows from the convexity of R ´ enyi divergence, and in our application of A.4, we are using the fact that for all θ t i ∈ supp ( R ) , (cid:107) θ t i − θ ∗ (cid:107) 2 ≤ √ p + (cid:112) 2 ln(1 / ∆) .
Furthermore, by
Lemma A.5, we know P and P (cid:48)
(resp.
Q and Q (cid:48) ) differ by at most ∆ in total variation distance.
So, since f is bounded in [ − 1 , 1] , we have:
[Equation] [Equation]
Then by applying triangle inequality twice:
[Equation]
Now we can prove (E3) as follows:
[Equation]
Proof of Theorem A.1.
We again assume without loss of generality ∆ is at most a sufﬁciently small constant.
The proof strategy will be to express E [ S ] in terms of individual variances Var ( f ( θ t i )) , which can be bounded using Lemma A.
[Equation]
We have the following:
[Equation]
From (2), we have the following:
[Equation]
i
[Equation]
y i
[Equation]
In the following
[Equation]
we bound each of the terms A , B
[Equation]
and C individually.
First, let us consider the term B .
We have the following:
[Equation]
Cov
[Equation]
Plugging Lemma A.6, (E3) into (4) we bound the variance of y i as follows:
[Equation]
We now focus on bounding the term C in (3).
Lemma A.6, (E1) and (E3) implies the following:
[Equation]
Plugging (6),(7), and (8) into (3), we have
[Equation]
Now, Lemma A.6, (E1) and (E2) implies ∀ i : | Var ( f ( θ t i ))
− Var ( f ( θ t k ))
| =
O (∆) .
So from (9) we have the following:
[Equation]
Plugging this bound back in (2), we have the following:
[Equation]
Which completes the proof.
A. O PTIMIZING THE N UMBER OF C HECKPOINTS
In Theorem A.1, we ﬁxed the number of checkpoints and gave lower bounds on the burn-in time and separation between checkpoints needed for the sample variance bound to have bias at most ∆ . We could instead consider the problem where T , the time of the ﬁnal checkpoint, is ﬁxed, and we want to choose k which minimizes the (upper bound on) mean squared error of the sample variance of { f ( θ iT/k ) } i ∈ [ k ] .
Here, we sketch a solution to this problem using the bound from this section.
The mean squared error of the sample variance is the sum of the bias and variance of this estimator.
We will use the following reparameterization of Theorem A.1:
Theorem A.7 (Reparameterized version of Theorem A.1) .
Let c 1 := 12 M + ln( c 2 M ( p + (cid:107) θ 0 − θ ∗ (cid:107) 22 )) , where c 2 is a sufﬁciently large constant.
Then if S is the sample variance of { f ( X iT/k ) } i ∈ [ k ] , V is the true variance of X T , and T/k > c 1 :
[Equation]
One can also show the variance of S is close to the true sample variance:
Lemma A.8.
If ¯ S is the sample variance of k > 1 i.i.d.
samples of θ T , then if c 2 is a sufﬁciently large constant, for c 1 as deﬁned in theorem A.7:
[Equation]
Proof.
Let x 1 , . . .
, x k be k i.i.d.
samples of f ( θ T ) , then since each x i is in the interval [ − 1 , 1] :
[Equation]
Giving the ﬁrst part of the lemma.
For the second part, let x i be the sampled value of f ( θ iT/k ) .
Then:
[Equation]
For some coefﬁcients c i,j,(cid:96),m , this can be written as (cid:80) i ≤ j ≤ (cid:96) ≤ m c i,j,(cid:96),m E [ x i x j x (cid:96) x m ] where (cid:80) i ≤ j ≤ (cid:96) ≤ m | c i,j,(cid:96),m | ≤ 2 .
By a similar argument to Theorem A.1, the change in this expectation if we instead use x i that are i.i.d.
is then at most exp (cid:16) − T/k − c 1 c(cid:17) as long as c 2 is a sufﬁciently large constant.
In other words, | E [ S 2 ] − E [ ¯ S 2 ] | ≤ exp (cid:16) − T/k
[Equation]
(cid:17) .
A similar argument applies to
E [ S ] 2 , giving the second part of the lemma.
Putting it all together, we have an upper bound on the mean squared error of the sample variance of:
[Equation]
Assuming k > 1 , T/k > c 1 .
Minimizing this expression with respect to k gives
[Equation]
which we can then round to the nearest integer larger than 1 to determine the number of checkpoints to use that minimizes our upper bound on the mean squared error.
Of course, if T < 2 c 1 then
Theorem A.1 cannot be applied to give a meaningful bias bound for any number of checkpoints, so this choice of k is not meaningful in that case.
We will bound the divergences D α ( P 1 , P 2 ) , D α ( P 2 , P 3 ) , D α ( P 3 , P 4 ) where P 1 is the distribution θ η that is the solution to (1), P 2 is a Gaussian centered at the point θ 0 − η ∇L ( θ 0 ; D ) , P 3 is a Gaussian centered at θ ∗ , and P 4 is the stationary distribution of (1).
Then, we can use the approximate triangle inequality for R ´ enyi divergences to convert these pairwise bounds into the desired bound.
Lemma A.9.
Fix some θ 0 .
Let P 1 be the distribution of θ η that is the solution to (1) , and let P 2 be the distribution
N ( θ 0 − η ∇L ( θ 0 ; D ) , 2 η ) .
Then:
[Equation]
Proof.
Let θ t be the solution trajectory of (1) starting from θ 0 , and let θ (cid:48) t be the solution trajectory if we replace ∇L ( θ t ; D ) with ∇L ( θ 0 ; D ) .
Then θ η is distributed according to P 1 and θ (cid:48)
η is distributed according to P 2 .
By a tail bound on Brownian motion (see e.g. Fact 32 in Ganesh & Talwar (2020)), we have that
[Equation]
Then following the proof of Lemmain Ganesh & Talwar (2020), w.p. 1 − δ ,
[Equation]
for some sufﬁciently large constant c , and the same is true w.p. 1 − δ over θ (cid:48) t .
Now, following the proof of Theorem 15 in Ganesh & Talwar (2020), for some constant c (cid:48) , we have the divergence bound
D
α
( P 1 , P 2 ) ≤ ε as long as:
[Equation]
In other words, for any ﬁxed η , we get a divergence bound of:
[Equation]
as desired.
Lemma A.10.
Let P 2 be the distribution
N ( θ 0 − η ∇L ( θ 0 ; D ) , 2 η ) and P 3 be the distribution
N ( θ ∗ , 2 η ) .
Then for η ≤ 2 /M :
[Equation]
Proof.
By contractivity of gradient descent we have:
[Equation]
Now the lemma follows from R´enyi divergence bounds between Gaussians (see e.g., Example 3 of van Erven & Harremos (2014)).
Lemma A.11.
Let P 3 be the distribution
N ( θ ∗ , 2 η ) and let P 4 be the stationary distribution of (1) .
Then for η ≤ 1 / 2 M we have:
[Equation]
Proof.
We have P 3 ( θ ) =
P 3 ( θ ∗ ) exp( − 14 η (cid:107) θ − θ ∗ (cid:107) 22 ) where P 3 ( θ ∗ ) = (cid:16) 14 πη (cid:17) d .
By Msmoothness of the negative log density of P 4 , we also have P 4 ( θ ) ≥ P 4 ( θ ∗ ) exp( − M 2 (cid:107) θ − θ ∗ (cid:107) 22 ) .
In addition, since P 4 is 1 -strongly log concave, P 4 ( θ ∗ ) ≥ (cid:0) 12 π (cid:1)
p/ 2 (as the 1 -strongly log concave density with mode θ ∗ that minimizes P 4 ( θ ∗ ) is the multivariate normal with mean θ ∗ and identity covariance).
Finally, for α ≥ 1 and η ≤ 1 / 2 M , we have α/ 4 η > ( α − 1) M/ 2 .
Putting it all together:
[Equation]
In ( ∗ )
[Equation]
we use the fact that α/ 4 η > ( α − 1) M/ 2 to ensure the integral converges.
Lemma A.12.
Fix some point θ 0 .
Let P be the distribution θ η that is the solution to (1) from θ 0 for time η ≤ 1 / 2 M .
Let Q be the stationary distribution of (1) .
Then:
[Equation]
Proof.
By monotonicity of R´enyi divergences (see e.g., Proposition 9 of Mironov (2017)), we can assume α ≥ 2 .
Then by applying twice the approximate triangle inequality for R´enyi divergences (see e.g. Proposition 11 of Mironov (2017)), we get:
[Equation]
The lemma now follows by Lemmas A.9, A.10, A.11.
Lemma A.4 now follows by plugging α = 2 , η = 1 / 2 M into Lemma A.12 and then using Theorem 2 of Vempala & Wibisono (2019).
[Table]
B M ISSING DETAILS FROM S ECTION 2.
Below we provide the experimental setup and results that are missing from Section 2.2.
B. E XPERIMENTAL SETUP Training hyperparameters:
CIFAR10 training : We use Jaxline (Bradbury et al., 2018) to train on CIFAR10 using DPSGD (Berrada et al., 2022).
For CIFAR10, we use a WideResNet with depth 16 and width 4.
We ﬁx clip norm to 1, batch size to 4096 and augmentation multiplicity to 16 as in (De et al., 2022).
Then, we set learning rate and noise multiplier, respectively, to 2 and 10 for ε = 1 and to 4 and 3 for ε = 8 .
For periodic distribution shifting (PDS) CIFAR10, we set learning rate and noise multiplier, respectively, to 2 and 12 for ε = 1 and to 4 and 4 for ε = 8 .
We stop the training when the intended privacy budget exhausts.
CIFAR100 training : For CIFAR100 also, we use Jaxline (Bradbury et al., 2018) and use DP-SGD to ﬁne-tune the last, classiﬁer layer of a WideResNet with depth 28 and width 10 that is pre-trained on entire ImageNet data.
We ﬁx clip norm to 1, batch size to 16,384 and augmentation multiplicity to 16 as in (De et al., 2022).
Then, we set learning rate and noise multiplier, respectively, to 3.
and 21.1 for ε = 1 and to 4 and 9.4 for ε = 8 .
For periodic distribution shifting (PDS) CIFAR100, we set learning rate and noise multiplier, respectively, to 4 and 21.1 for ε = 1 and to 5 and 9.4 for ε = 8 .
We stop the training when privacy budget exhausts.
We would like to highlight that we obtain a signiﬁcant improvement over the SOTA baseline of De et al.
In particular, unlike in (De et al., 2022), we ﬁne-tune the ﬁnal EMA checkpoint , i.e., the one computed using EMA during pre-training over ImageNet.
This modiﬁcation (without any additional checkpoints aggregations) gives a major accuracy boost of 5% (70.3% → 75.51%) for ε = 1 and of 3.2% (77.6% → 80.81%) for ε = 1 for the normal CIFAR100 baseline .
We obtain similarly high improvements by ﬁne-tuning the EMA of pre-trained checkpoints (instead of just the ﬁnal checkpoint) for the PDS-CIFAR100 case.
We leave the further investigation of this phenomena to the future work.
StackOverﬂow training : We follow the SOTA in (Kairouz et al., 2021; Denisov et al., 2022) and use TFF to train a one-layer LSTM (detailed architecture in Table 4 (Reddi et al., 2020)) on StackOverﬂow using DP-FTRLM full version from (Denisov et al., 2022).
StackOverﬂow is naturally user-partitioned data, and we process 100 clients in each FL round.
We train for 2048 FL rounds and set clip norm, noise multiplier, server learning rate, client learning rate, and server momentum, respectively, to 1, 0.341, 0.5, 1.0, 0.95 for ε = 18 .
9 and to 1, 0.682, 0.25, 1.0, 0.95 for ε = 8 . 2 .
For
PDS-StackOverﬂow, the same set of hyperparameter performs the best based on our tuning of the aforementioned hyperparameters.
Hyperparameters tuning of checkpoints aggregations: Here we provide the methodology we follow to obtain the best hyperparameters for our checkpoints aggregation methods (Section 2.1).
For
EMA, De et al.
(2022) simply use the EMA coefﬁcient that works the best in non-private baseline.
However, for each of the settings we consider, we tune EMA coefﬁcient in { 0.85, 0.9, 0.95, 0.99, 0.999, 0.9999 } and observe that the best EMA coefﬁcients for private and non-private settings need not be the same (Table 9).
For instance, for CIFAR10, for ε of 1 and 8, EMA coefﬁcient of 0.
and 0.99 perform the best and outperform 0.9999 by 0.6% and 0.3%, respectively.
Hence, we advise future works to perform tuning of EMA coefﬁcient.
Full results are given in Table 9.
[Table]
[Table] [Figure] [Table] [Figure]
[Table] [Figure]
For the past k checkpoints aggregation based methods, our tuning methodology is general and as follows: we use some validation data (which is disjoint from training data) and evaluate the efﬁcacy of aggregating k checkpoints where we vary k over a large enough range, and select the k that performs the best on average over 5 runs.
Finally, we present results on test data using the best k value.
Our hyperparameters tuning method is easy to replicate hence for conciseness, here we only provide the ranges of k that we use for tuning.
For CIFAR10 and CIFAR100, we tune k ∈ { 3 , 5 , 10 , 20 , ..., 200 } for both parameters and outputs aggregation.
For StackOverﬂow, we tune k ∈ { 3 , 5 , 10 , 20 , ..., 200 } for parameters aggregations (i.e., UPA) and for outputs aggregation (i.e.,
OPA and OMV).
However, as for outputs aggregations, one should store k checkpoints on device in the FL setting of StackOverﬂow, we reduce tuning range and tune k ∈ { 3 , 5 , 10 , 20 , ..., 100 } .
Details of data dependent checkpoints selection for StackOverﬂow: Here we provide precise method for selecting the best k of all checkpoints and aggregate them.
We perform these experiments for StackOverﬂow due to availability of additional data, and omit CIFAR datasets.
Speciﬁcally, we use 10,000 samples from the validation partition of the original StackOverﬂow data and use it as held-out data.
Note that held-out data is disjoint from both training and validation data we use for other parts of experiments.
We compute performance of all checkpoints on held-out data and order them from highest to lowest performing checkpoints.
Then we tune k as detailed in the previous section.
The aggregation function remains the same for all, but EMA, our aggregation methods from Section 2.1.
Recall that traditional EMA is computed over all the checkpoints computed during training.
We compute EMA over best k checkpoints as θ iema =
(1 − β ) · θ i − 1 ema + β · θ i , where i ∈ { 1 , 2 , ..., k } .
We keep EMA coefﬁcient, β , constant through out.
For EMA, we further tune β using validation data.
[Table] [Table] [Table]