Effectively using  public data in privacy preserving Machine learning
Authors: Milad Nasr
Saeed Mahloujifar
Xinyu Tang
Prateek Mittal
Amir Houmansadr
Venue: Submitted to ICLR 2023 (2023)
Keywords: Privacy preserving machine learning
dp-sgd
public data in privacy

Abstract: A key challenge towards differentially private machine learning is balancing the trade-off between privacy and utility. 
A recent line of work has demonstrated that leveraging  \emph{public data samples} can enhance the utility of DP-trained models (for the same privacy guarantees). 
In this work, we show that public data can be used to improve utility in DP models significantly more than shown in recent works.  
Towards this end, we introduce a modified DP-SGD algorithm that leverages public data during its training process. 
Our technique uses public data in two complementary ways: (1) it uses generative models trained on public data to produce synthetic data that is effectively embedded in multiple steps of the training pipeline; (2) it uses a new gradient clipping mechanism  (required for achieving differential privacy) which changes the \emph{origin} of gradient vectors using information inferred from available public and synthesized data. 
Our experimental results demonstrate the effectiveness of our approach in improving the state-of-the-art in differentially private machine learning across multiple datasets, network architectures, and application domains. 
Notably, we achieve a $75\%$ accuracy on CIFAR10  when using only $2,000$ public images;  this is \emph{significantly higher} than the  state-of-the-art which is $68\%$  for DP-SGD with the privacy budget of $\varepsilon=2,\delta=10^{-5}$ (given the same number of public data points).

Under review as a conference paper at ICLR 2023
EFFECTIVELY USING PUBLIC DATA IN PRIVACY PRE- SERVING MACHINE LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
A key challenge towards differentially private machine learning is balancing the trade-off between privacy and utility. A recent line of work has demonstrated that leveraging public data samples can enhance the utility of DP-trained models (for the same privacy guarantees). In this work, we show that public data can be used to improve utility in DP models significantly more than shown in recent works. Towards this end, we introduce modified DP-SGD algorithm that leverages public data during its training process. Our technique uses public data two complementary ways: (1) it uses generative models trained public data produce synthetic data that is effectively embedded in multiple steps of the training pipeline; (2) it uses new gradient clipping mechanism (required for achieving differential privacy) which changes the origin of gradient vectors using information inferred from available public and generated data from generative models. Our experimental results demonstrate the effectiveness of our approach in improving the state-of-the-arr in differentially private machine learning across multiple datasets, network architectures, and application domains. Notably, we achieve a 75.1% accuracy on CIFARIO when using only 2, 000 public images; this is significantly higher than the state-of-the-ant which is 68. 1% for DP-SGD with the privacy budget of & 2, 8 10-3 (given the same number of public data points).
INTRODUCTION
Machine learning is becoming an essential part of many technological advancements in various fields.
One major concern with usage of machine learning privacy of individuals whose data is used to develop the machine learning models. To tackle this concern, recent works De et al. (2022): Kurakin
(ct al. (2022); ; Abadi et al. (2016); Yu et al. (202I); Amid et al. (202I): ): Li et al. (202aa) suggest to train ML models with differential privacy (Dwork et al. (2014) guarantees. However, existing differential privacy (DP) techniques for ML impose large degradation to the utility of the trained models in comparison to non-private models. Recent works have made substantial improvements to the utility-privacy trade-off of such private ML techniques, (e.g., by scaling the hyper-parameters. (De) et al. (2022)), ) however, there still exists huge gap between the accuracy of DP-guaranteeing MI. mechanisms and their non- non-private alternatives, e.g., De et al. (2022) achieves an accuracy of 65% on CIFARIO (for ş 2.0 and 10 5) compared the 90% accuracy of non-private models.
In this work, we explore an emerging approach to close the utility gap between private and non-private models. Specifically, recent works (De et al. (2022); Kurakin et al. (2022); Abadi al. (2016); Yu et al. (202T)) show that leveraging publicly available (therefore. non-private) ) data can enhance the utility of DP-trained models without impacting their privacy guarantees. In such works, the public data is used to pre-train the model, and then the pre-trained model is fine-tuned with the private data while applying DP protections.
In this work, we show that, while recent works use public data to pre-train private models, public data can be used much more effectively in enhancing the utility of private models. To this aim, we design generic method to utilize public data in differentially private machine learning, an approach we call Differentially Private Origin Estimation Stochastic Gradient Descent (DOPE-SGD). Our work uses two complementary techniques to enhance the utility of differentially private models. First, it improves the quality of the noisy gradients based on the available non- non-private data. This helps by reducing the variance of the noise added the gradients in the DP model, therefore better preserving

---

Under review as a conference paper at ICLR 2023
the information in the original gradient vector (Section 3). Second, DOPE-SGD uses advanced data augmentation techniques to enhance the quality of the data used for training, therefore reducing overfitting to the public data and improving generalization.
Through extensive experiments we show that DOPE-SGD's use of public data along with data augmentation improves the privacy-utility trade-off's of private models by large margins. For instance, we show improvements up to 12.3% over DP-SGD models on the CIFAR10 10 dataset, pre-trained with the same public data. We also show improvements on language model both training from scratch (from 221 to 198 on a small BERT model) or fine-tuning (from 21.23 to 19.09 on GPT-2) with g = 1.0 and 8 10-5
BACKGROUND
Differential privacy (Dwork (201T): Dwork et al. (2014) is the gold standard for data privacy. It is formally defined as below:
Definition 1 (Differential Privacy). A randomized mechanism M with domain D and range R preserves (e, diffeerrent privacy for any two neighboring datasets D' DD and for any subset S C R we have:
Pr(M(D) S] e Pr[M(D') E ES
where is the privacy budget and 8 is the failure probability.
Several works have used differential privacy in traditional machine learning algorithms to protect the privacy of the training data (Li et al. (014);; CChaudhuri et al. (2011) Feldman al. (018): /Zhang et al. (2016); Bassily et al. (2014)) Many of these works (Feldman et al. 2018): Bassily y et al. (2014); Chaudhuri et al. (2011) use properties such convexity smoothness for their privacy analysis, which is not necessarily true in deep learning, and therefore, one cannot use many of such methods in practice. Abadi al. (2016) designed a deep learning training algorithm, DP-SGD, where they used gradient clipping to limit the sensitivity of the learning algorithm, and then add noise to clipped model gradient proportional to its sensitivity. As we know, training a deep learning model an iterative process. The main approach to analyze the privacy cost of private deep learning is to compute the privacy cost of the single step of the learning algorithm and then use composition method to calculate the overall privacy cost which is commonly done in RDP (Mironov] (2017)) instead of (e. -DP. One of the important features of differential privacy is the post-processing (Mironov] (2017); Dwork al. (2014) which we will utilize in this work. DP-SGD is now commonly used to train differentially private deep learning models.
3 DOPE-SGD: OUR IMPROVED DPSGD ALTERNATIVE
We can improve the utility-privacy trade-off for differentially private machine learning algorithms in three phases: (1) pretraining improve the initial point of private training, (2) use better algorithms for differentially private training, (3) do a post processing on the private models. Previous works showed the effect of using pretraining (Abadi al. (2016); Kurakin et al. (2022); De al. (202))) ) so this work we mainly focus on two last phases of private training.
3.1 DP-SGD WITH ADAPTIVE ORIGIN
One of the main steps of DP-SGD is to clip the gradient of each instance and add noise the gradient of each batch based on clipping value. Clipping the gradient results in a bias in the optimization process which makes convergence slower. One way to prevent this to use larger clipping values, however, this requires larger noise to obtain the desired privacy. The main idea of this work is clip the gradients around an estimate of the gradient instead of clipping the gradient around the origin, as shown in Figure] [ As result, we can potentially clip more harshly (i.e., small clipping values) around the carefully chosen centers with less bias in the optimization compared to DP-SGD and obtain better accuracy while maintaining privacy.
We first introduce a general algorithm called DP-SGDA (Algorithm D that uses adaptive origin selection. We use function that takes the history of the protocol and also some auxiliary
2

---

Under review as a conference paper at ICLR 2023
Notey Gradient
Clipping threshold
Gradient
Gradient
Vector
Vector
Noisy Graddant
Field
Clipping
threshold
Gradient Vector
Cligped Gradient Vecoor
Estimated Gradient Vector
Clipping the gradients around an estimation of the origin (purple vector) can increase the ratio of the clipped gradient vector to the added noise. As result the final privatized gradient vector closer to original gradient vector compared to clipping around zero.
Figure 1:
Algorithm 1 DP-SGD with Adaptive Origin (DP-SGDA)
Require:
training dataset D, adaptive origin function G, batch size n, learning rate n), noise scale gradient norm clip C, loss function 1, auxiliary information aur Initiate randomly
2: for E {T'} do
3:
B, < sample instances from dataset D
4: Vit] +
5:
I + C(V [1]........ ..... Vit - 1], aux).
6:
for all (at, v) E B, do
7:
(z.9) gradient 1(x,y) y)
(().)) xixC
8:
(z.9)
max C, Via.o 2)ll2)
9:
(2.0) + (a..)
V(t] [ (a,v)
11: end for
V < v + (W o
13:
0 0 [t.
14: end for
15:
return output 0
information as input and outputs a point ğ that will be used as the origin for the clipping operation. The following proposition states that any instantiation of this algorithm with this algorithm will satisfy DP guarantees.
Proposition 2. For any function G. DP-SGDA (Algorithm Ty obtains the same DP and RDP guaran- tees as DGD (Abadi et al.) (21()), when the clipping threshold C, sub-sampling rate and noise parameter equal (See Appendix F for detailed proof).
Theoretical Justification To show the benefit of our Algorithm | consider setting of the lipschitz loss function with concentrated gradients.
Definition 3 (Lipschitz and cradient-concentrate loss function). A loss function defined on a model space O and imput space is C-lipschitz if for all X and 0 E O, 38 (0,z)]l2 S C. The loss function is r-concentrated if for all 0 E O there exists point Co in the gradient space such that we have
dL 20 (0,x) coll2 Kr.
Va X;
Note that any C-lipschitz loss function L-concentrated. We call co the concentration point for We call oracle function C(-) concentration point oracle given model 0, it returns C(0) Cg-
3

---

Under review as a conference paper at ICLR 2023
Algorithm 2 Differentially Private Origin Estimation-SGD (DOPE-SGD)
Require: private training dataset D, non-private data D., non- poiittivate batch size ns learning rate 7).
private batch size n, noise scale o, gradient norm clip C, loss function
Initiate randomly
2: for E {T) do
3: B, < sample instances from dataset D
4: B, < sample nt, instances from dataset D.
5:
V < (L(..)
6:
V - 0
7:
for all (z, y) B, do
(2.9) + gradient of ((, v)
8:
9:
2)
+ V +
Vi) x C/ max(C, (V -V(1))
(..))
10:
11: end for
0 N((00
VG
14:
end for
15:
return output 0
Now we state the following proposition about the privacy cost of our algorithm comparison with the privacy cost of DP-SGD.
Proposition 4. Let X and θ be example and model spaces and let be C-lipschitz and concentrated loss function for X and @. Optimizing this loss function with clipping threshold r in Algorithmn 7 with concentration point oracle used as the adaptive origin function, will induce the same output (model) distribution as training with DP-SGD with clipping threshold C, as long as all examples in the training set are in X. Algorithm I achieves (c"² (Tln(1/1))) ln DD-DP, where as DP-SGD achieves (c (cf//Tim(1/0 ( -DP,, for constant sampling rate q, and saamiiii
number of iterations and sufficiently large a (See Appendix For detailed proof).
Proposition shows that our algorithm can reduce the sensitivity of the gradient update at each iteration from C to Given that L. our algorithm would obtain better privacy guarantees when the gradients are concentrated. Although these bounds are stated based on advanced composition theorem for approximate differential privacy, the privacy benefit of our algorithm holds for alternative notions of privacy, such as RDP, because of the reduced sensitivity in our algorithm.
Instantiation with (augmented) public data. To obtain the estimate of gradient vectors, we use the idea of (augmented) public data. Public data has been used previously in many traditional differentially private tasks. In this work, we show using public data to estimate can improve deep learning with differential privacy significantly. Algorithm summarizes Jusoitiimssmmmaiizess the design of our approach. We also take advantage of advanced augmentation techniques (such as diffusion models) to utilize the public data further. The exact data augmentation mechanism heavily depends the specific task and type of the dataset. We can use several techniques for data augmentation, which can range from using a basic shifting of images to designing a synthesizer (Mood (1950)) using complex generative models (Nichol & Dhariwal (202T)
3.2 ENSEMBLE OF PRIVATE MODELS
One of the properties of the way we analyze DP-SGD at the current state is that we assume every step of the iterative training is public and an adversary can use all of the intermediate steps for an attack. This might not be true in usual training of deep learning model, however, this assumption is currently necessary for the analysis. Although this assumption might increase the privacy budget (Nast et al. (2021)), it will allow us to all of the models in all iterations of the training without additional privacy cost (due to post procepsng feature differential privacy). One example of taking advantage of this fact is using "Fxponential Momentum Averages (EMA)" approach in differential private
4

---

Under review as a conference paper at ICLR 2023
models (De et al. (2022) We also show that by ensembling the intermediate models can further improve the accuracy of the predictions. We use two approaches to ensemble the models. First, we use the idea of majority voting in which to give a label for each input query the last n models and we use the label with the highest number of the votes for classification tasks (in the case that we are interested in the value of logits, such language models, last n models). Second, we show the effectiveness of taking the average models of last nt models and evaluating the inputs on the average instead of the target model.
take the average of logits among the
While the main objective of our work is on the best ways to utilize the public data in the training private models, this approach can be also used in cases where we do not have any additional data. We explore this in Appendix [ In the main text we always use the maximum between the two ensemble methods, but as shown in Appendix [ we see that both methods have very similar performance and using majority voting achieves slightly higher performance.
Why do ensemble methods work? Stochastic gradient descent with averaging has been studied as a way to cope with the excessive variance coming from stochasticity of SGD Bach (2014). One would expect the reduction of variance by averaging the resulting parameters of the last few models, assuming that the variance is caused by independent noise. Bach (2014) shows faster convergence on convex models by doing the simple averaging operation. In DP- and its variants, large portion of variance in the model comes from the added noise, which is independently sampled and added to the model after each iteration. This makes DP- SPD suitable for averaging methods as the variance is mostly the result of independent noise. It is important to note that in privacy analysis of DP-SGD and its variants, we always "pay" the privacy cost of all the intermediate models and using them as an ensemble will not violate the privacy guarantees.
We also note that several recent studies have tried to understand the privacy benefits of having hidden states during the course of training and not leaking the intermediate models Feldman et al. (2018) 8); Ye & Shokri (202)). They conjecture that a portion privacy cost spent for publishing the intermediate models in DPDSGD might be wasted, leading to sub-optimal trade-off between privacy and accuracy. The success of our ensemble methods can be seen as a verification of this conjecture and shows the possibility of better ways to use the privacy budget.
EXPERIMENTS
To evaluate the suggested techniques we consider two main scenarios, where public data is coming from similar distribution the training dataset and also where the data is from slightly different distribution. In the main body of this work, we focus on the CIFAR and WikiText datasets as they represent a majority of the applications of the deep learning models. In Appendix B effectiveness of our approach for other architectures and tasks.
we show the
Additionally, show that we can further improve the utility of the private models by including the public data in the training dataset. This way, with a fixed batch size, we can use a smaller sub- sampling rate in privacy analysis. Please note that also do pretraining on the public dataset. However, we found that while many of the public instances have zero loss after the pretraining it is still useful to include the public dataset in the training dataset. We explore this in our experiments in detail.
CIFARIO dataset. De et al. (2022) showed that using a WideResNet architecture for the CIFARIO dataset can improve the Wfate-ofthh-an differentially private model significantly. Therefore, we also use similar architecture for the majority of our experiments. In Appendix B we also evaluate smaller convolution neural network. Due the computation limitation we use WideResNet 6-4 as explained in De et al. (2022) since there gain of using the larger model is not very significant We also use CIFAR 100 as -uitt--dis public data.
WikiText-2 dataset. Our dataset setup mainly follows the previous work (Amid et al. (2021))
The texts are tokenized by the top 8k most frequent words and special token for the remaining words. The dataset then constructed as length-35 sequences and we have 59,675 data points in the
Training WRN40-4 eight 1100 in our setting takes more than 96 hours.
5

---

Under review as a conference paper at ICLR 2023
Table 1: An ablation study the effect of different techniques using training data for models trained on CIFAR-10 10 under (2, 10 -DDP. We use WRNI6-4 with all of the augmentation and optimization techniques as detailed in De et al. (2022). We use 2,000 images of the CIFARIO training dataset (4% of the whole dataset) as the public data which is sampled uniformly from each class. The public dataset is augmented 40,000 instances.
Setting
Test Acc (%)
Baseline (WRNI6-4 De et al. (2022) (cold)
64.9
+ Pretraining on the public data (warm)
68.1
+ Pretraining on the generated data using public data (warm-aug)
72.0
Including the generated data in the training dataset (extended)
73.7
+ Using DOPE-SGD (Algorithm) 2
74.8
+ Using Ensemble models
75.1
training set. We use 4% of the training set as in-distribution public data and WikiText-3 as imperfect out-of-distribution public data (we removed the overlapping part of WikiText-2 from WikiText-3). We use a BERT- Devlin et al. (2099) model with two blocks.
Settings: We show the results in several settings. In our experiments we use the term "warm" to describe a setting where we first train model without any privacy on the non-private data (as opposed "cold" where we do not pre-train). We also use "extended" to note case where we augment the training dataset by including the non-private dataset (which we use the augmented data). Please note that we did hyper-parameter tuning for each setting (as detailed in Appendix AA, the main body of this paper we focus on in in-distribution results for CIFAR10 and WikiText-22 datasets. We evaluate the other datasets and out-of-distribution datasets in Appendices
B andd]
4.1 RESULTS
Using the in-distribution public dataset can improve the utility of differentially private machine learning significantly. However, one the downsides of requiring an in-distribution public dataset is the cost of acquiring such data. As a result, assuming that can have access to large in-distribution public data is very strong. Therefore, we only assume that we have access to very limited in- distribution public data in this section. However, order to address the issue of small public dataset and prevent overfitting models on the public dataset, we use various augmentation techniques on the public dataset. In this section, we use small portion of the target's training dataset as the public dataset.
Main in-distribution Results. In addition to traditional augmentation techniques we train a generative model on the public dataset which can be seen ideal augmentation technique. This way, at each iteration, we can generate fresh samples from the generative model and calculate the average gradient over those samples. In particular for image datasets used DDPM generative models Nichol & Dhariwal (2021). DDPM models are currently slow in generating examples, therefore, in our implementations we generate a large set of data point from the generative models and use a random subsample of that for training the target model.
To understand the effect of each method described in this work, first we do ablation study when using different methods and algorithmss in Table for CIFAR10 10 dataset. One of the main approaches of using public data is pretrain the target model first on the public data which does not have any privacy cost and then do the private training. Since the public data for the in in-distribution dataset is limited (while we are still using traditional augmentation methods) there is only 3.5% increase in the accuracy. However, we apply idea of using generative models to augment the public data see that we can further improve. One important step that to the best of our knowledge has not been used before is to combine the public dataset (or the generated dataset public data) with the private data that is used during the private training. As shown in Table I only by including the public data in the training dataset
increase the accuracy (1.7%).
When we combine all of the methods for CIFAR1O we increase the accuracy of classification by 7.6 points when using only 4% of the data compared the case when we pretrain the training dataset

---

Under review as a conference paper at ICLR 2023
Table 2: Test accuracy/perplexity for models trained with differential privacy with = 10-6 and
using 4% of training of ITTF/A////iiieett.... dataset as the public data. The accuracy of the model trained on only augmented CIFARIO 10 dataset is 69.4% and perplexity is 240 on Wikitext-2 dataset.
CIFARIO 10 (Test Acc)
WikiText-2 (Test Ppl)
E
DPSGD (cold)
DPSGD (war))
DPSGD (warm-aug)
DOPE-SGD
DPSGD (cold)
DPSGD (warm)
DOPE-SGD
1.0
56.8%
60.1%
70.00 0%
72.1%
240
221
2.0
61.9%
68. 18.1%%
T2 .0.0%
75.1%
220
206
184
4.0
71.9%
72 4%
76.0%
77.9%
210
183
177
6.0
77.0%
77.1%
78.7%
80.0%
190
167
Table 3: Test accuracy of models trained with differential privacy with 8 = 10 and using 4% of training of CIFARIO dataset as the public data for different training algorithms.
Original
Ours
E
Method
Setting
Augmentation
DOPE-SGD
Mirror Gradient Descent Amid et al. (2021)
68.7%
70.5%
75.1%
Gradient Scaling Li al. 2022a
68.7%
69.1%
Mirror Gradient Descent Amid al.
(2021)
73.1%
74.5%
4
77.9%
Gradient Scaling
Li et al.
2022a
73.5%
74.1%
Mirror Gradient Descent Amid al.
(2021)
77.2%
78.2%
6
80.0%
Gradient Scaling
Li et al. 2022a
77.9%
78.1%
which is the common practice. We also evaluated our results for different privacy budgets as shown in Table 2 Our results suggest that the common practice of just using the public dataset to pretrain the models ineffective way of using the public data and by properly using this additional information we can improve the state of the private training significantly and we can achieve practical accuracies even more private regions. When and DP-SGD reduces which is excepted. Because in the higher privacy regimes we are adding less noise, therefore, the model has an easier time learning from the private data and does not rely on the public data as much. As we also show in Appendix our methods are also effective in smaller models, different datasets and imperfect datasets. One of the major downsides of our method the increase in the computation cost which we will discuss in Appendix C In Appendix [ we also show that when the public data is not from the same distribution, we do not see the same improvement as in-distribution data and we introduce changes in our approaches to improve the results.
increase the privacy budget, the gap between approach
Comparison. Recent works also proposed better algorithms to take advantage of public dataset. Amid et al. (2021) uses the idea of mirror gradient descent in traditional machine learning and extends the idea for deep models. All of the recent work and our work use public data to estimate the geometry of the gradient field. All of the recent work used different approach approximate the geometry estimation using the public data, [Amid et al. (2021) used the gradient of public data for first order approximation of the gradient geometry. Li et (2022a) used the public data re-scale the gradients of private data. In contrast our approach uses the gradient of the public data to recenter the private gradients. We compare the accuracy of different approaches on the CIFARIO 10 dataset. Unfortunately, previous works all use different architecture to evaluate their method and none of them used the current saute-of-the-ae architecture, therefore, we can not directly compare the accuracy numbers. However, we did our best to train and do a hyper-parameter tuning of both of the previous works for a fair comparison. We used the same public dataset all of the training runs.
Table 3 summarizes the results of our comparison. Original Setting refers to a setting where we use the method from the previous works and replace the neural network with the current state-of-the-ar method and do hyper-parameter tuning, As shown in Table 3 since the previous works do not use our data augmentation techniques there is a huge gap between our approach and previous works. Moreover, showed that augmentation techniques approach and can improve the accuracy of the existing mechanism as well. Finally, we see that using all of the techniques our approach achieves higher accuracy compared to the existing methods. In
also be used in any public data training
7

---

Under review as a conference paper at ICLR 2023
Table 4: Test accuracy of models trained with differential privacy using different amounts of public data that augmented to 40,000 in- stances on CIFARIO.
Table S: Test accuracy of models trained with differ- ential privacy using different amounts augmented data from 2, (00(4%) public data on CIFARIO.
Augmented Data Size
E 6
Public Data Size
E 2
E = 4
& = 6
5K
69.5%
73.8%
78.4%
500 (1%)
68.9%
72.1%
77.1%
10K
70.1%
75.3%
79.3%
1,000 (2%) 2,000 (4%)
73.5%
74.9%
78.4%
20K
70.9%
77.4%
79.9%
75.1%
77.9%
80.0%
40K
75.1%
77.9%
80.0%
Table 6: Finetuning WikiText-2 on pre- trained large language models using 4% of the training dataset as public data.
Table 7: Test accuracy models trained with user-level differential privacy with 10-5 on EMNIST using 2%
WikiText-2
users as public users.
8
DPSGD (warm)
Ours
E
DP-SGD (all)
DP-SGD (only-privare
DOPE-SGD
0.5
82.2%
83.1%
84.7%
0.25
21.30
19.16
1.0
83.4%
84.6%
85.2%
0.5
21.29
19.14
2.0
85.0%
85.5%
86.0%
1.0
21.23
19.09
4.0
86.2%
86.4%
87.1%
00
15.40
comparison using mirror gradient descent approaches achieves very high accuracy as well when we add all of our techniques top. Surprisingly Gradient Scaling approach does not scale as well as Mirror gradient descent when we add our techniques. One issue might be that the Gradient Scaling approach is not optimized for the cases where the size of public data is large.
Effect of Size of the Public Dataset. The size of the public dataset can affect the utility of learning significantly. We studied the effect of different amounts of public data Table As expected by having larger public dataset, the model can achieve better utility.
Effect of Size of Generated Data. One of the interesting trade-off's is to see how much the utility of a private model changes by increasing the number of augmentation that we generate for given public dataset. Table 3Jsooss the results of this study and as we can see adding more augmentations of the public data the training dataset increases the utility of the training. However, one the main limiting factors of using more augmentations is the computation cost of the augmentation. Specially, the current DDPM models are costly in both training and generating different augmentations.
Pretrained Large Language Models. All of the state-of-the-art models in language tasks currently use pretrained large language model and fine-tune them on their specific tasks. While this means that they are using much larger training dataset, is still very important show we can see the same improvements as in previous experiments in this setting. To this aim, we used pretrained GPT-2|Radfort al. (019) model (which is not trained on WikiText) and fine-tuned it using both DP-SGD and DOPE-SGD using 4% public data and when also first train it the public data without any privacy cost and then use private training. Table summmarzess our results, as we can see DOPE-SGD also outperforms DP-SGD in this setting even at small epsilons.
Extending to Federated Learning with User-Level Privacy. Another example of settings where we can use non-private data is federated learning. In many federated applications, users can indicate that they do not require any privacy protections, while this is only small number of the users, the learning algorithms can benefit significantly from such users. Amid et al. (202I) suggested using such users first pretrain the model, however, this is not a practical approach since the data of these users will change during the training of a federated learning task. Pretraining the model only on the non-private Therefore, our evaluations federated learning settings focuses on the cases where we do not pretrain on the data (cold setting).
will slow down the federated learning process and it won't 't be effective.
8

---

Under review as a conference paper at ICLR 2023
202
200
300
400
360
Figure 2: Comparison of DOPE-SGD and non-private's dot product with DP-SGD and non-private's dot product. Using DOPE-SGD will reduce the effect of the noise added by the Gaussian mechanism which can lead better performance (all experiments are in warm-aug setting).
For this setting, we used the EMNIST dataset which has 70,000 users and we uniformly divide the instances between the users and we assume fixed 2% of the users are non non-private. We use WideResNet model for this task and Fed-SGD McMahan et al. (2017) to train this model. In each round of the training we select subset of users update the model. We assume the computations and communications are done in a trusted environment, therefore, we can use the user-level central differential privacy definition to analyze this setting.
Table summarizes the results and the comparison with the existing approaches that can be applied in the federated learning setting. We compare our approach to using DP-SGD for this setting and show that by leveraging the public data we can achieve better results. One alternative to DP-SGD is to not apply differential privacy on the public user and only apply it on the private users. We compared to this baseline and we showed better performance. This result also shows the benefit of the Algorithm even when we do not use the other techniques described in this work.
Why DOPE-SGD Works? To answer this question we compute the inner product of the gradients of each batch when using DOPE-SGD with non-private gradients of the same batch and compare it with inner product the DP-SGD and non-private gradients (average between independent runs and after pretraining public dataset). Figure 2 shows this comparison, as we the dot product of DOPE-SGD is larger compared to DP-SGD especially in earlier iterations. As a result, we can achieve better convergence rates and privacy-utility trade-off.
CONCLUSION
As many of previous works showed that deep learning models leak information about their training datasets. New regulations will require the models to protect the privacy of the users that is being used to train large deep learning models. As result researchers are using differential privacy train large models. However, the main limiting factor of using differentially private machine learning is the degradation of the utility in most cases. To improve the utility-privacy trade-off, recent works started to use non-private data. The common practice of using public data is currently to just use pre-train the model and then use private learning on private dataset. In this work, our main goal is to show this approach is not the optimal technique to take advantage of non-private data in many cases. We can summarize our work in three practical steps that can be applied in any setting that uses non-private data. First, we should use advanced augmentation techniques (which include data synthesis approaches such as GAN, DDPM) on the public data to synthesize more non-private data points. Second, we should consider including the non-private dataset in the private data during the private training and finally, by using more advance private learning algorithms (c.g. DOPE-SGD, Amid et al. (202I), Li al. (2022a) which are designed to leverage public data to achieve better trade-offs. Using these methods we showed that we can improve the state-of- the-art in private learning significantly.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308 -18..1 ACM, 2016.
Ehsan Amid, Arun Ganesh, Rajiv Mathews, Swaroop Ramaswamy, Shuang Song, Thomas Steinke, Vinith M Suriyakumar, Om Thakkar, and Abhradcep Thakurta. Public data-assisted mirror descent for private model training. arXiv preprint arXiv:2112.00199) 2021.
Francis Bach. Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression. The Journal of Machine Learning Research, 15(1) 15(1):995-627 2014.
Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. 2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pp. 464-473. IEEE, 2014.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milnn and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://github com/gooqle/jax
Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 12(3). 2011.
Soham De, Leonard Berrada, Jamie Hayes, Samuel Smith, and Borja Balle. Unlocking high- accuracy differentially private image classification through scale. arXiv preprint 2rXiv:2204.136500 2022.
Jacob Devlin, Ming-Wer Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers). Association for Computational Linguistics, 2019.
Cynthia Dwork. Differential privacy. Encyclopedia of Cryptography and Security, pp. 338-340, 2011.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9334)::211 1-707, 2014.
Vitaly Feldman, Ilya Mironov, Kunal Talwar, and Abhradeep Thakurta. Privacy amplification by iteration. In 2018 IEEE 59th Annual Symposium Foundations of Computer Science (FOCS), pp. 521-532. IREF, 2018.
Alexey Kurakin, Steve Chien, Shuang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta. Toward training at imagenet scale with differential privacy. arXiv preprint arXiv:2201.12322 2022.
Haoran Li, Li Xiong, Lucila Ohno-Machado, and Xiaoqian Jiang. Privacy preserving rbf kernel support vector machine. BioMed research international, 2014, 2014.
Tian Li, Manzil Zaheer, Sashank J Reddi, and Virginia Smith. Private adaptive optimization with side information. arXiv preprint 200.0522,,.,5663, 2022a.
Xuechen Li. Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In International Conference on Learning Representations, 2022b.
Brendan McMahan, Fider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-effccientt learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282, PMLR, 2017.
Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263-275. IEFF, 2017.
10

---

Under review as a conference paper at ICLR 2023
Alexander McFarlane Mood. Introduction to the theory of statistics. 1950.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC Conference Computer and Communications Security, pp. 634 646, 2018.
Milad Nasr, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, and Nicholas Carlin. Adversary instantiation: Lower bounds for differentially private machine learning. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 866- 866-882. IEEE, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pp. 3162-8171. PMLR, 2021.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEEE, 2017.
Jiayuan Ye and Reza Shokri. Differentially private learning needs hidden state (or much faster convergence). arXiv preprint wrXiv:2203.05363 2022.
Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj. Jessica Zhao, Graham Cormode, and Ilya Mironov. Opacus: User-friendly differential privacy library in PyTorch. arXiv preprint 2rXiv:2109.12298, 2021.
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500, 2021.
Zuhe Zhang, Benjamin IP Rubinstein, and Christos Dimitrakakis. On the differential privacy of bayesian inference. In Thirtieth AAAI Conference Artificial Intelligence, 2016.
A IMPLEMENTATION DETAILS
We implemented Algorithm 2 and the related works in JAX (Bradbury et al. (2018) and we imple- mented Algorithm 2 in Opacus (Yousefpour et al.] (2021) and toavate-transformers library Li et al. 2022b Experiments in this work are the average between 5 independent runs. For each setting did a grid hhppr-pprameteeee search as mentioned in Table and picked one with the highest average.
While doing Syper--arameter training will increase the privacy cost, we didn't 't consider this in
privacy calculation similar to the previous works (Abadi et al. (2016) Amid et al. 2002 De et al.
(202)) Kurakin et al. (2022)).
Table 8: Set hyper-parameterr used in the hyper-tuning phase.
Parameter
Values
Learning rate Noise multiplier
[1,2,3,4
[1,2,3 4,5,8]
Public data sample size Clipping norm Batch size
160,460 122,0022600
1..5,088.1.0,1...
[512,1024,2048, 4096]
We summarize different settings in the paper in the followings:
Cold:
This refers to a setting where we do not pre-train on the available non-private dataset.
"The code is available (will be added after publishing).

---

Under review as a conference paper at ICLR 2023
Table 9: Comparison of the results on CIFARIO dataset in different settings for both DP-SGD and DOPE-SGD using ConvNet architecture (* the setting that does not utilize the public data). Public data in these experiments are 2,000 images from the dataset which are augmented 40,000 images.
SGD (Abadi et al. (2016)
DOPE-SGD (Alg
cold"
warm
cold-ext.
warm-ext.
cold
warm
cold-ext.
warm-ext.
1.0
46.7%
61.4%
48.0%
66.6%
67.2%
62.7%
67.8%
70.3%
2.0
50.1%
64.2%
51.9%
68.0%
70.1%
70.4%
71.8%
73.0%
4.0
54.7%
66.4%
57.4%
68.1%
75.7%
74.6%
77.0%
73.2%
6.0
58.4%
68.4%
59.4%
68.7%
76.2%
75.1%
78.1%
76.2%
Table 10: Comparison of the results on Purchase 10 dataset different settings for both DP-SGD and DOPE-SGD using Fully-Connected architecture. Public data in these experiments are 10,000 records.
DP-SGD (Abadi et al. (0166)
DOPE-SGD (Alg
cold"
warm
cold-ext.
warm-ext.
cold
warm
cold-ext.
warm-ext.
1.0
30.2%
32.6%
34.3%
36.8%
33.2%
35.3%
40.7%
42.1%
2.0
34.1%
35.7%
37.9%
39.2%
35.1%
37.4%
44.6%
45.6%
4.0
44.4%
45.9%
47.7%
49.2%
45.2%
48.5%
50.2%
52.3%
Warm:
We call setting warm where we first the model on all available non-private data (both augmented and original public data) and then train only on the private dataset.
Warm-aug: Similar to the previous setting we first train the model on the non non-private dataset and then train it on both non-private and private dataset.
Extended: We refer to a setting where we include the non-private (which includes the augmented dataset) dataset in the private training dataset.
B EVALUATING DIFFERENT ARCHITECTURES AND DATASETS.
To show that our techniques also be applied simpler model, used ConvNet network with three layers on CIFAR 10 dataset with batch size of 256. Table presents the accuracy this model in different settings used in Table 2 Similar to the results Table 2 we impressive improvement over DP-SGD.
We showed the effectiveness of our approach to the popular vision and language modeling tasks. However, many practical applications use categorical data. To show the effectiveness of our approach in such tasks, we focused on the Purchase dataset which is the shopping records for several thousand individuals. Each data record corresponds to one customer and has 600 binary features, we use Purchase 10 which has 10 classes. Similar to previous works Shokri al. (2017) Nasr al. (0188). we use 4 layers fully connected network. We used 90,000 instances the training dataset, 3,000 instances as the public data and 100,000 instances as training dataset. Table [aalle[0ssmmmaizze the results of our experiments. As we can see, the overall accuracy of this task is much lower than the previous datasets. However, we still see significant gain when using public-data and using Algorithm
C COMPUTATION COMPLEXITY
The main downside of our approach is the additional computation cost compared to the DP-SGD. There are two main components to these additional costs. First, the costs of the augmentation. As we know, augmentation techniques are used widely in training deep learning models. In this work we try on using diffusion models as our augmentation technique which have high computation costs. This can be the main source of the additional computation cost for work. As reference, training a diffusion model Nichol & Dhariwal (202T) on 2,000 cifar10 on a single V100 takes about 48 hours. Also generating 40,000 images using this model takes about week of compute. The second
12

---

Under review as a conference paper at ICLR 2023
30
20
DP.500.E
Epsilon
Figure 3: The effect of out-of-distribution data on Algorithm without clipping the public gradient (Equation
component of the cost, the additional computation cost since we have a larger non-privee non-private dataset which can increase the computation cost linearly. Moreover, Algorithm Sueer uses an additional gradient computation call on the public data which will increase the computation cost. Training model using our approach is about 2 3 times slower than using the same model using DP-SGD without augmentated images (without considering the time to generate the augmentations.).
D IMPERFECT PUBLIC DATA
One of the main downsides of the in-distribution public dataset is the cost collecting such a dataset. As result many applications starting the same data distribution for differentially private deep neural networks. We call such data sets imperfect data distributions which be from slight different distribution have noise data noisy labels. While many public data algorithms require the data to be in-distribution, the existing works showed that it is still helpful to pretrain the -uitt--dittiibution public datasets Amid et al. (2021): Li et al. (2022a). In many cases naively using the imperfect public dataset can negatively affect the utility of the learning performance.
other public datasets which not exactly from
This is because the main assumption in Algorithm il that the gradient on the public dataset is very close estimation of the private data. However, this not true for the out-of-distre distribbbut dataset. In order to fix this assumption, we rewrite Algorithm by modifying line 9 to the following two lines which allow us to limit the effect of the public data:
V x
max X, (
C
V ((V (Z.))
V3y)
(2)
ma (
(2'll2)
To show the effect of imperfect data in the training, we evaluate the effect of other datasets from different but similar distributions as public data (different distribution) and noisy public dataset. We used the 10FAR10 dataset as the public data dataset for CIFAR10. 10, and WikiText-3 for WikiText-2. While many of the previous works showed that pretraining on the dataset from a slightly different distribution (such as CIFAR 100 for CIFAR 10) can help improve the accuracy of the original model, we also show our additional techniques can also boost the accuracy. In particular, we observe that include all of the training dataset in the private training it will reduce the overall test accuracy, however, by including a random subset of the public dataset the private training part can improve the privacy-utility trade-off. In experiments for CIFAR 10 using 10% of the CIFAR 100 gave us
13

---

Under review as a conference paper at ICLR 2023
Table 11: An ablation study the effect of different techniques using imperfect public data
CIFARIO models trained under (2, 10 -DDP. We used CIFARI00 as public data for the CIFARIO dataset.
Settings
Test Acc (%)
Baseline (WRN16-4 pretrained on CIFARI00)
75.9%
+ Including the 10% of public data in the training dataset (extended)
76.4%
+
Using DOPE-SGD (Algorithm) 2 (warm)
77.1%
+ Using Ensemble models
77.3%
Table 12: Test accuracy/perplexity models trained with differential privacy with = 10-6 and
imperfect data (CIFARION 100 for CIFARIO and WikiText-3 for WikiText-2).
CIFARIO 10 (Test Acc)
WikiText-2 (Test Ppl)
DPSGD (warm)
DOPE-SGD
DPSGD (warm)
DOPE-SGD
1.0
68.9%
76.3%
79
77
2.0
76.4%
77.3%
78
68
4.0
79.2%
81.5%
76
6.0
82.5%
84.9%
75
62
the highest boost accuracy. For WikiText-2 dataset since the public dataset (Wikitext-3) is much more similar to the private dataset we do not see any downside of including all of the public data in private training.
we study the effect of each technique when the public data does not come from the same distribution as the private dataset. we can see from the results, similar to the in-distribution dataset, using DOPE-SGD improves the utility further. Finally, in Table 12 we show the results for different dataset and privacy budgets. As we see, in all settings our method outperforms current approaches.
In Table 11
E ENSEMBLE OF PRIVATE MODELS WITHOUT ADDITIONAL DATA
As mentioned in Section 3.2, we can also apply the idea of ensemble of the private models to improve the utility of the final model without additional privacy cost. In the main body of this work, show the effectiveness of this approach when we have public data. However, this approach can also be used in cases when do not have public data. In Figure 4 compare the accuracy of different ensemble approaches. As we see using majority voting can achieve higher accuracy compared to the other approaches. We also evaluate the effect of the number of models we use the ensemble in Figure 5 In our experiments we observed that we need at least 50 models in an ensemble to have noticeable gap between the final model and the ensemble model.
F PROOFS
Proof of Proposition 2
We start by analyzing the sensitivity of the gradient update rule. The update before adding noise and public gradient is equal to
(((..))
[ BBll-8 + E (E.)EBB (en(C./ Note that is data independent and comes from public data, therefore only need to understand the sensitivity of the sum of clipped gradients.
(p(=.) -X)x Since each example (z, y) only affects one of the clipped gradients max(C, ( () and each of these vectors have bounded by C. therefore the sensitivity of the sum is C.
14

---

Under review as a conference paper at ICLR 2023
No ensemblee..9p 919
EMA
Majortt
60
Actuwes
Figure 4: Comparison of the ensembling the last 100 models, the exponential moving average (EMA with decay rate 0.999) and the model without using any ensembling techniques for DP-SGD and any additional data on test accuracy of CIFARIO.
Majofty
71.5
71.0
70.5
70.0
30 50
300
200
300
Number
Figure 5: Comparison of the number of models in the ensemble the accuracy for WRN-4-16 on CIFARIO dataset with E = 4, in our implementation of De et al. (2022) we observed slight difference the accuracy and the reported accuracy in their work. De (2022) reported 71.4% for CIFARI 10 dataset with & = 4 but we were only able to achieve 71.1%.
15

---

Under review as a conference paper at ICLR 2023
Now, since adding Gaussian noise, each iteration is instantiation of the sub-sampled [B.l and noise Therefore, all the existing Gaussian mechanism with sensitivity C, sampling rate [
analysis for general DP-SGD (without additional assumptions) would also apply to Algorithm I
Proof of Proposition
As stated in proof of Proposition 2 each iteration of Algorithm
[ii and noise multiplier a/r. Therefore, each iteration Gaussian mechanism with sampling rate [
will be 21m111122//)) 8') -DP. Assuming e' 2m1.222155//0 1, the sub-sampled mechanism will be (2q- ='.q0')--D Then, by using advanced composition theorem for DP, we have that the
composition of all T steps is (4T'² 5² + 2q E' / log((18)), Tqq'' + -)-DP. Assuming 24 the composition of T mechanisms is (4qe') V 2T In (1/0),, T8' 8D-DP. Now setting 8 5/2 and 8' 8/2n, the entire mechanism is (4 v 22T ln(2.5 T/6) ln(2/6), 6D which in In order for the assumptions to be correct, we 2 (2.5 T/ō)). The privacy analysis for DP-SGD follows similarly, the only thing that changes is that we should use the clipping threshold instead of r.
e' In(1/h)
turn implies (4v109 90 (n
need o > max (2I11/23-T// In(2/3)
We would now prove that the output of Algorithm Dis same as DP-SGD. Note that since the loss function C-lipschitz,
clipping operation for DP-SGD is a non-operational. Similarly, for DP-SGDA, since we have r-concentrated gradients, the clipping operation is non-operational. Given that the clipping operation is the only difference between DP-SGD and DP-SGDA, and they both are non-operational, the output distributions of the two algorithms exactly the same.
16


Reviews:
[{"review_id": "jpNR4r0Q3Q", "reviewer": "Reviewer_d916", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes new techniques for using public data in differentially private machine learning and makes a significant improvement over the state-of-the-art solution. More specifically, they propose to use synthesized data provided by a generative model trained by the given public data, and a new gradient clipping mechanism that achieves higher accuracy with the help of public and synthesized data. ", "summary_of_the_review": "Overall I found no flaw in this paper and would recommend it. ", "strength_and_weaknesses": "Strength:\n- The techniques are well-motivated and presented. The intuitions behind the proposed techniques are well explained. A solid theoretic backbone is also provided for their claims.\n- Those techniques are novel and not very complicated. They also provide good experiments demonstrating the impacts and improvements in different settings and parameters. \n\nWeakness:\n- It's better to have proofs of proposition 2 and 4 in the appendix.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "YRnbENwLLq", "reviewer": "Reviewer_Y2rR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies multiple ways to incorporate public data to help private training,  (1) use pre-training to improve the initial point of private training, (2) use a novel DP-SGD algorithm during training, (3) post process on the private models. The contribution is complete, tackling aspects before/during/after the training. Ablation studies are done on CIFAR10 and Wikitext to show the performance.", "summary_of_the_review": "I think the paper is well-written but the experiments seem thin and the results aren't very exciting.", "strength_and_weaknesses": "Strength: This paper is well-written and the methodology is comprehensive, including different aspects of using public data (or public gradient) in the DP training. The experiments are carefully conducted and I appreciate the ablation study in Table 1. I think the new algorithms are interesting with good motivation.\n\nWeaknesses: 1. I think Algorithm 1&2 fall in the category of weighted sum between public and private gradients. Let's denote $g_i$ as per-sample grad, $\\hat g$ as public gradient like in Line 7 of Algorithm 1. Then the clipped residual is $c_i*(g_i-\\hat g)$ and Line 9 becomes $(1-c_i)\\hat g+c_i g_i$. Then the summed clipped gradient is $\\sum_i (1-c_i)\\hat g+\\sum_i c_i g_i$ and the noise is added. Clearly, this is a weighted sum that has been also studied in https://arxiv.org/abs/2111.00115 and several other works. So I think they should be discussed. 2. I wonder do the authors experiment with the average angle between clipped per-sample gradients and the public gradient at each iteration? This could be very helpful to convince that your new clipping helps reduce the bias (also, only the angle bias not the magnitude matters, as the latter can be recovered with adjusting learning rate). 3. The experiments do not show significant improvement but the algorithms are seemingly over-complicated.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KTl8ftUh_u", "reviewer": "Reviewer_uKD4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper uses several ideas to improve on the privacy-utility trade-off of DP-SGD using public data, beyond the traditional approach of pre-training on public data:\n1) use data augmentation to pre-train on augmented data (setting called \"warm\" in paper)\n2) include public data in training data (setting \"warm-aug\")\n3) at each DP-SGD gradient calculation step, before clipping and noise addition, recenter the gradient vector using a gradient estimate learnt from public data (technique called DOPE-SGD in paper)\n4) ensembling (I seem to understand that means \"using majority voting over n checkpoints\" on CIFAR-10 experiments)\n\n\nExperiments mainly on CIFAR-10, but also on language modelling are shown. Ablation and analytical experiments are conducted. The combination of methods improves on the state of the art.", "summary_of_the_review": "The paper's central claim of beating SOTA seems supported by experimental evidence on both CIFAR-10 and language modelling, in different settings. The description of experimental settings and protocols is unclear and leaves much underspecified. I see no major flaw in the work and it has potential to be improved. ", "strength_and_weaknesses": "Strengths:\n- convincing demonstration of superiority\n- good set of experiments\n\nWeaknesses:\n- exposition unclear, verging on obscure\n- experimental settings insufficiently documented\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8a2C2YtqJ_", "reviewer": "Reviewer_UAzg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the DP model training with public data. They utilize the public data in three phases:  pretraining, better optimization process and a post-process ensemble of private models. The experimental results show the significant improvement of their approach compared with the SOTA.", "summary_of_the_review": "The results of proposed approach are very promising. The missing baseline (see weaknesses) leads to a some uncertainty that how much the DOPE-SGD helps, which will influence the justification of significance. Thus I give the score of 6.", "strength_and_weaknesses": "**Strength**\n1. This paper is written very clear and organized.\n2. It clearly explains the intuition why the new proposed clipped gradient would be potentially better than clipping to the origin. The approach is overall well-motivated.\n\n**Weaknesses**\n\nThe experiments are mostly well-designed. However, it might miss one fair baseline, which is training only with (augmented) public data, including the new generated synthetic data. This baseline can justify the improvement from the second phase (new clipping method) and third phase (model ensemble) in the approach.\n\nMinor issue:\n1. Notation of *L* is re-used as both loss function in Algorithm 1&2 and the Lipschitz factor.\n2. \"Out-of-distribution public data\" is introduced in the dataset descriptions but is not mentioned in the later results in the main text. (I checked that it appears in the experiments in Appendix).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject