G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators
Authors: Yunhui Long
Boxin Wang
Zhuolin Yang
Bhavya Kailkhura
Aston Zhang
Carl A. Gunter
Bo Li
Venue: NeurIPS 2021 Poster (2021)
Keywords: Differential Privacy
Generative Models

Abstract: Recent advances in machine learning have largely benefited from the massive accessible training data. However, large-scale data sharing has raised great privacy concerns. In this work, we propose a novel privacy-preserving data Generative model based on the PATE framework (G-PATE), aiming to train a scalable differentially private data generator that preserves high generated data utility. Our approach leverages generative adversarial nets to generate data, combined with private aggregation among different discriminators to ensure strong privacy guarantees. Compared to existing approaches, G-PATE significantly improves the use of privacy budgets. In particular, we train a student data generator with an ensemble of teacher discriminators and propose a novel private gradient aggregation mechanism to ensure differential privacy on all information that flows from teacher discriminators to the student generator. In addition, with random projection and gradient discretization, the proposed gradient aggregation mechanism is able to effectively deal with high-dimensional gradient vectors. Theoretically, we prove that G-PATE ensures differential privacy for the data generator.  Empirically, we demonstrate the superiority of G-PATE over prior work through extensive experiments. We show that G-PATE is the first work being able to generate high-dimensional image data with high data utility under limited privacy budgets ($\varepsilon \le 1$). Our code is available at https://github.com/AI-secure/G-PATE.

G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators
Yunhui Long! Boxin Wang
Zhuolin Yang'
Bhavya Kailkhura²
Aston Zhang'
Carl Gunter!
Bo Li'
University Illinois, Urbana Champaign
2 Lawrence Livermore National Laboratory [ylong4, boxinw2, zhuolin5, 1zhang74, cgunter, 1bo)@illinois hiw.lllmoisseeuu
Abstract
Recent advances in machine learning have largely benefited from the massive accessible training data. However, large-scale data sharing has raised great privacy concerns. In this work, we propose a novel srivacyprreeervngg data Generative model based on the PATE framework (G-PATE), aiming to train scalable differ- entially private data generator which preserves high generated data utility. Our approach leverages generative adversarial nets generate data, combined with private aggregation among different discriminators to ensure strong privacy guaran- tees. Compared to existing approaches, G-PATE significantly improves the use privacy budgets. In particular, we train student data generator with an ensemble of teacher discriminators and propose a novel private gradient aggregation mechanism ensure differential privacy on all information that flows from teacher discrimi- nators to the student generator. addition, with random projection and gradient discretization, the proposed gradient aggregation mechanism is able effectively deal with high-dimensional gradient vectors. Theoretically, we prove that G-PATE ensures differential privacy for the data generator. Empirically, we demonstrate the superiority of G-PATE over prior work through extensive experiments. We show that G-PATE is the first work being able generate high-dimensional image data with high data utility under limited privacy budgets (s S 1). Our code available at https: //github............ com/ AI AI-secure/G-PATE
Introduction
Machine learning has been applied to wide range of applications such face recognition 30 39
21
22), autonomous driving (26), and medical diagnoses [8] 20). However, most learning methods rely on the availability of large-scale training datasets containing sensitive information such as personal photos or medical records. Therefore. such sensitive datasets are often hard to be shared due to privacy concerns [40] To handle this challenge, data providers sometimes release synthetic datasets produced by generative models learned on the original data. Though recent studies show that generative models such as generative adversarial networks (GAN) [14] can generate synthetic records that are indistinguishable from the original data distribution, there is no theoretical guarantee on the privacy protection. While privacy definitions such as differential privacy
[9] and Rényi differential privacy provide rigorous privacy guarantee, applying them to synthetic data generation is nontrivial.
27
Recently, two approaches have been proposed combine differential privacy with synthetic data gen- eration: DP-GAN (35) and PATE-GAN [37]. DP-GAN modifies GAN by training the discriminator using differentially private stochastic gradient descent. Though achieves privacy guarantee due to
"qqual contribution.
35th Conference on Neural Information Processing Systems (NcurIPS 2021)

---

the post processing property [10] of differential privacy, DP-GAN incurs significant utility loss on the synthetic data, especially when the privacy budget is low. In contrast, PATE-GAM trains differentialyy private GAN using Private Aggregation of Teacher Ensembles (PATE) (28]. Specifically, it trains set of teacher discriminators and a student discriminator. To ensure differential privacy, the student discriminator is only trained on records that are produced by the generator and labeled by the teacher discriminators. The key limitation of this approach is that it relies on the assumption that the generator would be able to generate the entire real records space to bootstrap the training process. If most of the synthetic records are labeled as fake by the teacher discriminators, the student discriminator would be trained on a biased dataset and fail to learn the true data distribution. Consequently, this trained generator would not be able to produce high-quality synthetic data. This problem does not exist for traditional GAN, where the discriminator is always able provide useful information to the generator since they can access the real data records rather than the synthetic data only.
The main contribution of this paper is new approach named G-PATE for training a differentially private data generator by combining the generative model with PATE mechanism. Our approach is based on the key observation that: It is not necessary to ensure differential privacy for the discrimina- tor in order to train a differentially private generator. As long as we ensure differential privacy on the information flow from the discriminator to the generator, it is sufficient to guarantee the privacy property for the generator. To achieve this, we propose a private gradient aggregation mechanism to ensure differential privacy all the information that flows from the teacher discriminators to the student generator. The aggregation mechamism applies random projection and gradient discretization to reduce privacy budget consumed by each aggregation step and to increase model scalability. Com- pared to PATE-GAN, our approach has three advantages. First, it improves the use of privacy budget by only applying it to the part of the model that actually needs to be released for data generation. Second, our discriminator can be trained on original data records since it does not need to satisfy differential privacy. Finally, G-PATE preserves better utility on high-dimensional data given its more efficient gradient aggregation mechanism.
Theoretically, we show that our algorithm ensures differential privacy for the generator. Empirically, we conduct extensive experiments on the Kaggle credit dataset and image datasets. To the best of our knowledge, this is the first work that is able to scale to nigh-dimensional face image dataset such as CelebA while still preserving high data utility. The results show that our method significantly outperforms all baselines including DP-GAN and PATE-GAN.
2 Related Work
Differential privacy [9] is notion that ensures algorithm outputs general information about its input dataset without revealing individual information. So far, researchers have proposed different methods to design differentially private statistical functions and machine learning models [2] 5 25 12).
Private aggregation of teacher ensembles (PATE) is proposed to train a differentially private classifier using ensemble mechanisms. Scalable PATE (2 improves the utility of PATE with a Confident- GNMax aggregator that only returns result if it has high confidence in the consensus among teachers. However, PATE and Scalable PATE are only applicable to categorical data (i.c.. class labels) as shown in prior work. Differentially private data generative models have also been proposed. Priview (31] generates synthetic data based on marginal distributions of the original dataset, PrivBayes [38] trains a differentially private Bayesian network, and MWEM [o] uses the multiplicative weights framework to maintain and improve distribution approximating a given data set with respect to set of counting queries. However, these approaches are not suitable for image datasets since the statistics they use cannot well preserve the correlations between pixels in an image.
Some recent work applies differential privacy to the training of GAN. DP-GAN 35 and DP-
CGAN B33 add Gaussian noise to the gradients of the discriminators during the training process. GS- WGAN 6 reduces gradient sensitivity using the Wassertein distance and uses gradient sanitization to ensure differential privacy for the generator. PATE-GAN (37) trains student discriminator using an ensemble of teacher discriminators. (E-MERF(1S) and PEARL (23) use differentially private embedding to train generative models on the embedding space. Concretely, DP-MERF uses random feature representations of kernel mean embeddings, and PEARL improves upon MERR by incorporating characteristic function that improves the generator's learning capability. Both MP-MERF and PEARL focus on generating a private embedding space on which the distance metric between synthetic and real data is computed (though there are no results reported for high-dimensional images). On the contrary, G-PATE focuses generating DP nigh-dimensional data by improving
2

---

Syntheto Samples
Teacher Discrmmntoos
Difierentaally Privtte Gradient Aggrogator
Student Data
Generator
Teacher 3
Data
Adversary
Advensary
Data Flow
Training
Grodient
Figure I: Model Overview of G-PATE.
The model contains three parts: student data generator, differentially private gradient aggregator, and an ensemble of teacher discriminators.
the model structure and the private gradient aggregation step, which orthogonal the embedding space optimization approaches as DP-MERF and PEARL.
3 Preliminaries
Differential Privacy. Differential privacy bounds the shift in the output distribution of a randomized algorithm that could be caused by small input perturbation. The following definition formally describes this privacy guarantee.
Definition ((c, Diffferrret Privacy). A randomized algorithm M with domain NI is (c, 6)- differentially private for all S C Range(M)) and for any neighboring datasets D and D', we have Pr[M((D E S] < exp(E) Pr[M(D') E S] +
Rényi Differential Privacy. Rényi differential privacy is natural relaxation of differential privacy. Defined below, its privacy guarantee is expressed in terms of Rényi divergence.
Definition ((A, () RDD). A randomized mechanism M said to guarantee (A, RDD with À > 1 if for any neighboring datasets D and D',
Dx (M(D))IM(D')) =
1 log EZ-M(D)
Pr M(D 2]
2-1
SE.
Pr.M (D') = x)
(A,e)-RDP implies (ss, -ddfffrrrett privacy for any given probability 8> 0. Theorem (From RDP to DP). If a mechanism M guarantees (X,ε)-RDP, then M guarantees (E+ log 8)-differential privacy for any 8 1).
A-I
Compared to DP, RDP supports easier composition of multiple queries and clearer privacy guarantee under Gaussian noise. Specifically, RDP could be easily composed by adding the privacy budget:
Theorem 2 (Composition of RDP). If mechanism M consists of sequence of M Mr such that for any i [k], M, guarantees (X, RDD- then guarantees (A, E E/=1,,) RDP.
Suppose f is real-valued function, and the Gaussian mechanism is defined as G.f(D) f(D)= N (0, o²) where V (0, o²) is normally distributed random variable with standard deviation a and mean O. The Gaussian mechanism provides the following RDP guarantee:
Theorem (RDP Guarantee for Gaussian Mechanism). If f has sensitivity 1, then the Gaussian mechanism G satisfies (X, X/ (2o²)) RDDP.
4 G-PATE: A Scalable Data Generative Method
In this section, we present our method named G-PATE. overview of the method is shown in Figure I Unlike PATE-GAN and DP-GAN, G-PATE ensures differential privacy for the information

---

flow from the discriminator to the generator. This improvement incurs less utility loss on the synthetic samples, so it can generate synthetic samples for higher dimensional and more complex datasets.
G-PATE makes two major modifications on the training process of GAN. First, we replace the discriminator in GAN with ensemble of teacher discriminators trained disjoint subsets of the sensitive data. The teacher discriminators do not need to be published, thus can be trained using non-private algorithms. In addition, we design gradient aggregator to collect information from teacher discriminators and combine them in a differentially private fashion. The output of the aggregator is gradient vector that guides the student generator to improve its synthetic samples.
Unlike PATE-GAN, G-PATE does not require any student discriminator. The teacher discriminators directly connected to the student generator. The gradient aggregator adds noise in the information flow from the teacher discriminators to the student generator to ensure differential privacy. This way, G-PATE uses privacy budget more efficiently and better approximates the real data distribution to ensure high data utility.
4.1 Training the Student Generator
Algorithm 1 Training the Student Generator.
To achieve better privacy budget efficiency, G-PATE only ensures differential privacy for the generator and allows the discrimi- nators learn private information.
hatch size m, number of teacher models n, num- ber of training iterations N, gradient clipping constant c, number of bins B, projected dimension k. noise parame- ters 0i and 02, threshold T, disjoint subsets of sensitive data S1, S2,..
Input:
To case privacy analysis, we decompose G-PATE into three parts: the teacher dis- criminators, the student generator, and the gradient aggregator. To prevent the propa- gation of private information, the student generator does not have direct access any information in any of the teacher discrim-
2: for number of training itcrations do
3: //Phase I: Pre- Pro-rrccs
4: Sample noise samples Z1, Z2, Zm
5: Generate fake samples G(z1). G(zz)..... G(zm)
for each synthetic image G(zj) do
7: //Phase II: Private computation and aggregation
8:
for each teacher model i do
9:
Sample data samples from S. Update the teacher discriminator D,
inators. Consequently, we cannot train the student generator by ascending its gradi-
10:
11:
Calculate the gradients Ax
based on loss of the discriminators. To
12:
end for
solve this problem, we calculate the back-
13:
AX, (Ax Ax (2).
Ax("))
propagated gradients on the fake record x by ascending x's gradients on the loss of the discriminator. This gradient vector can be viewed as an adversarial perturbation 17:
14:
Axpri <- DPGradAgg (AX,, e, B, k. 0101,
15:
I/Phase III: Proc-Processing
16:
81 + -G(zz) + Axjriv
end for
on that would cause the discriminator's
18:
Update the student generator G by descending its stochastic gradient on LG on (R1, &2,..., &m)
loss on x to increase. Therefore, adding the gradients to the generated fake record would teach the student generator how to improve the fake record. In each training iteration, the student generator updated in three steps: (1) A teacher discriminator generates the backpropagated gradients for each record produced by the student generator. (2) The gradient aggregator takes the gradients from all teacher models and generates a differentially private aggregation of them. (3) The student generator updates its weights based on the privately aggregated gradients. The process is formally presented in Algorithm [
19: end for
Backpropagating Gradients in the Discriminator. Let D be a teacher discriminator. Given fake record x, we use (z(z) to represent D's loss on æ. In each training iteration, the weights of D are updated by descending their stochastic gradients on LD-
For each input fake record x, generate gradient vector Ar that guides the student generator improving its output. By applying the perturbation on its output, the student generator would get an improved fake record f Az on which D has higher loss. Therefore, Ax is calculated as æ's gradients on Lp:
(Ln da
Ar
(1)
With the gradient vector Ax, the student generator can be trained without direct access to the discriminator's 's loss.

---

Updating the Student Generator. A student generator G learns to map random input to fake record (z)) so that z indistinguishable from a real record by D. Given the gradient vector Ar. the teacher discriminators have higher loss on the perturbed fake record o = Ar compared to the original fake record x. Therefore, the student generator learns to improve its fake records by minimizing the mean squared error (MSE) between its output G(2) and the perturbed fake record J.
La(z, = Σ (G(z)) - ";)²,
where k is the number of synthetic records generated per training iteration. To ensure differential privacy, instead of receiving the gradient vector from single discriminator, we train the student generator using a differentially private gradient aggregator that combines gradient vectors from multiple teacher discriminators. Details are provided in Section
4.2 Differentially Private Gradient Aggregation for G-PATE
G-PATE consists of student generator and an ensemble of teacher discriminators trained on disjoint subsets of the sensitive data. In each training iteration, each teacher discriminator generates gradient vector Ar that guides the student generator on improving its output records. Different from traditional GAN, in G- PATE, the student generator does not have ac- cess to the loss of any teacher discriminators, and the gradient vector is the only information propagated from the teacher discriminators the student generator. Therefore, to achieve differential privacy, it suffices add noise during the aggregation of the gradient vectors.
Algorithm 2 Differentially Private Gradient Ag- gregator (DPGradAgg). This algorithm takes list of gradient vectors and returns a differentially private aggregation.
I: Input: Concatenated gradient vectors from cach teacher model AX = (Ax((2) ..... Ax(*)), gradi- ent clipping constant c, number of bins B, projected dimension k noise paramcters and Ø₂, threshold T
2: Ko <- the dimension Ax(1)
3: R < a ko x k random projection matrix with each component randomly drawn from NO. :))
4: AU + AXR
5:
for each column uj AU do
6:
Clip uy to (-c,c)
However, the aggregators used in PATE and
7:
h 4- the histogram of uj with B bins of width 2c B
PATE-GAN are not suitable for aggregating
8:
Aufriv + Conf Confident-GlMax/f 01, 02, T)
gradient vectors because they are only appli- cable to categorical data. Therefore, we pro- pose differentially private gradient aggrega- tor (DPGradAgg) based on PATE. With gradi-
9: end for
10: Pu + (Auprin; ;.... Augtiy
11: Axpriv <- Au
12: Output: Axpriv
discretization, convert gradient aggregation into voting problem and get the noisy aggregation of teachers' votes using PATE. Additionally, we use random projection to reduce the dimension of vectors which the aggregation is performed. The combination of these two approaches allows G-PATE to generate synthetic samples with higher data utility, even for large scale image datasets, which is hard to be achieved by PATE-GAN. The procedure is formally presented Algorithm
Gradient Discretization. Since PATE is originally designed for aggregating the teacher models' votes the correct class label of an example, the aggregation mechanism in PATE only applies to categorical data. Therefore, we design a three-step algorithm to apply PATE on continuous gradient vectors. First, we discretize the gradient vector by creating histogram and mapping each element to the midpoint of the bin it belongs to. Then, instead of voting for the class labels as in PATE, a teacher discriminator votes for k bins associated with A: elements in its gradient vector. Finally, for each dimension, we calculate the bin with most votes using the Conf ideett aggregator (2 (Appendix[D) The aggregated gradient vector consists of the midpoints of the selected bins.
With gradient discretization, the teacher discriminators can directly communicate with the student generator using the PATE mechanism. Since these teacher discriminators are trained real data, they can provide much better guidance to the generator compared to the student discriminator in PATE- GAN, which is only trained on synthetic samples. Moreover, the confident-oo -Naan aggregator ensures that the student generator would only improve its output in the direction agreed by most of the teacher discriminators.
Random Projection. Aggregation of high dimensional vectors is expensive in terms of privacy budget because private voting needs to be performed each dimension of the vectors. To save
5

---

privacy budget, we use random projection [3] to reduce the dimensionality of gradient vectors. Before the aggregation, we generate random projection matrix with each component randomly drawn from Gaussian distribution. We then project the gradient vector into lower dimensional space using the random projection matrix. After the aggregation, the aggregated gradient vector projected back to its original dimensions. Since the generation of random projection matrix is data-independent. It does not consume any privacy budgets.
Random projection is shown to be especially effective on image datasets. Since different pixels of an image are often highly correlated, the intrinsic dimension of an image is usually much lower than the number of pixels 13]. Therefore, random projection maximizes the amount of information a student generator can get from single query the Confident GNMax aggregator, and makes possible for G-PATE to retain reasonable utility even on high dimensional data. Moreover, random projection preserves similar squared Fuclidean distance between high-dimensional vectors, therefore is beneficial to privacy protection both theoretically and empirically [36]
Privacy Guarantees
In this section, we provide theoretical guarantees on the privacy properties for G-PATE. To start with, we propose the following definition for differentially private data generative model.
Definition 3 (Differentially Private Generative Model). Let G' be generative model that maps set of points Z in the noise space Z set of records X in the data space X. Let D be the training dataset of G and A: 'D + G' be the training algorithm. We say that G is a (c, 5)-differentiilly private data generative model if the training algorithm A is (e, difffferretttllll private.
Definition 3 relaxes the definition of DP-GAN by focusing the protection only the generative model in a GAN. This relaxation saves privacy budget during training and improves the utility of the model. Moreover, the relaxation does not compromise the privacy guarantee for the synthetic data.
Lemma 1. Let G be an (e, 8) -iffeerentially private data generative model trained a private dataset D. For any Z E Z, the synthetic dataset X = =G(Z) is (e, differrentilll private.
Proof Sketch. Lemma is a consequence of the post-processing property of differential privacy. First, the random points Z are independent of the private dataset D. Second, one does not need to query the discriminator during the data generation process. Therefore, the synthetic dataset is generated by post-processing G' and is guaranteed to be (c, d)-differrnntially private.
Lemma D shows that differentially private generative model is able to support infinite number of queries to the data generator and can be used to generate multiple synthetic datasets.
Next, we justify the privacy guarantee of the G-PATE method. The following lemma justifies RDP of the gradient aggregator (Algorithm 2)
Lemma 2 (Rényi Differential Privacy of DPGradAgg). The output of the gradient aggregator
(DPGradAgg) proposed in Algorithm satisfies (>. Ez RDD,, where X > 1 and E is the data-dependent RDP budget with order for the Confident CiddettGGae GNNaa aggregator on the j-th projected dimension.
Proof Sketch. Lemma can be proved by combining the RDP guarantee of the Conf Confident-GNMar. aggregator and the post- pest-prcesssing property of RDP. We first divide the input of DPGradAgg into two categories. The first category contains data independent parameters, including the gradient clipping constant c, the number of bins B. the projected dimension k. the noise parameters σj and 02, and the threshold T. These parameters do not contain private information. The second category contains the gradient vectors AX (1) ..... Ax(m)), ). which are data-dependent and sensitive. Our privacy analysis focuses on the computation on AX. With random projection and gradient discretization, we convert AX into & histograms and pass the histograms into the Conf Confident deiiiddnttoo GNMax aggregator. Since Conf MidentGGMaa satisfies data-dependent RDP 22 the privacy guarantee nicely propagates to the output of DPGradAgg.
We analyze the RDP guarantee DPGradAgg by composing the privacy budget consumed by the Conf ieeiid privacy budget of the training algorithm is composition of the data-dependent Rényi differential privacy budget of the Conf Confident-Cue GGMMx aggregator over & dimensions. The data datandepn privacy
aggregator on each projection dimension. Therefore, the Rényi differential

---

budget for each Somfident-NNMye aggregation is dependent on 01, 02. and threshold T (Appendix D). The remaining parameters (e.g. gradient clipping constant e, number of bins B) do not influence the privacy guarantee.
The next theorem justifies RDP of the G-PATE training process.
Lemma (Rényi Differential Privacy of G-PATE). Let A be the training algorithm for the student generator (Algorithm D with N training iterations and l projected dimensions. The data-dependent Rényi differential privacy for A with order À > is Lisien (Eisjsk Exs) where Et,j is the data-dependent Rényi differential privacy for the Confident -NNaax aggregator the i-th ieraaion on the j-th projected dimension.
Proof Sketch. For the convenience of privacy analysis, we divide the each iteration in Algorithm T into three phases: pre-processing, private computation and aggregation, and post-processing. In the pre-processing phase, the generator produces fake samples without accessing the private data. In the private computation and aggregation phase, the teacher discriminators are updated based on private data. Each teacher discriminator also generates a gradient vector. These vectors are aggregated using the DPGradAgg algorithm. Based on Lemma [ the data-dependent RDP for this phase is Eisist ELj. In the proscesoosssing phase, the student generator is updated using the privately- aggregated gradient vector Axypriv It satisfies RDP because of the post-processing property. Finally, the RDP of Algorithm i composed over N training iterations.
The next theorem provides theoretical guarantee on the differential privacy of G-PATE.
Theorem 4 (Differential Privacy of G-PATE). Given sensitive dataset 'D and parameters 0 < 1, let G be the student generator trained by Algorithm|] There exists & > 0 and X > 1 so that G is a (e
log 1/8 8)-differentially private data generative model.
Theorem is the consequence of converting the Rényi differential privacy guarantee in Lemma 3 differential privacy (Theorem
6 Experimental Evaluation
We evaluate G-PATE against three state-of-the-aae models: DP-GAN, PATE-GAN and GS- WSSNG We first perform comparative analysis with baselines on the tabular and image datasets used in the corresponding works, including the Kaggle credit tabular dataset and the grayscale image datasets (MNIST and Fashion-MNISTT). In addition, we evaluate G-PATE on the privacy-sensitive large-scale high-dimensional face dataset CelebA.
6.1 Experimental Setup
Tabular Dataset. we the same Kaggle credit card fraud detection dataset 7 (Kaggle Credit) as in [37]. The dataset contains 284,807 samples representing transactions made by European cardholders' credit cards in September 2013, and 492 (0.2%) of these samples are fraudulent transactions. Each sample consists of 29 continuous features from a PCA transformation on the original features.
Image Datasets. To demonstrate the superiority of G-PATE to PATE-GAN on high dimensional image datasets, we train G-PATE on MNIST, MushionnMNISS (34), and the celebrity face datasets CelebA [24]. MNIST and Fashion-MNIST consist of 60,000 training examples and 10,000 testing examples. Each example is a 28 x 28 grayscale image, associated with a label from classes. The CelebA dataset contains 202,599 images aligned and cropped based on the human face. We create three datasets: SclebA-GenderrS is a binary classification datasct that uses the gender attributes as the labels and resizes the images to 32 x 32 x 3; to demonstrate the scalability of G-PATE we also create TelebA-Genderlu with the same label while resizing the images to 64 x 64 x 3; CelebA-Hair contains images as 64x64x3 with three hair color attributes (black/blodde/brow))) We follow the
official training and testing partition as 24].
Implementation Details. For the Kaggle Credit dataset, both the generator and discriminator net- works of G-PATE are fully connected neural network with the same architecture as PATE-GAN (37 We use random projection with 5 projection dimensions during gradient aggregation. We use the DCGAN (32] architecture on the image datasets. We set the projection dimensions to 10 during gradient aggregation. More details are provided in Appendix E.
7

---

Table I: Performance Comparison on the Tabular Dataset and Image Datasets. We compare G-PATE with DP-GAN, PATE-GAN, GS-WGAN and vanilla DC-GAN. Vanilla DC-GAN has no privacy protection. The best results are highlighted in bold. Table (a) presents AUROC of the classifier trained on synthetic data and tested on real data. The performance satisfying (1, 10 10-10000------ d-diffeerential privacy is evaluated over 4 different classifiers: logistic regression (LR), AdaBoost, bagging, and multi-layer perceptron (MLP).
(a) Data Utility (AUROC) on Kaggle (c) Data Utility (Accuracy) on Image Datasets. The table presents the classification accuracy of CNN models trained on the generated data and tested on real data evaluated under two private settings: E = 10 and = 1 given 5 = 10
Credit Tabular Dataset.
DC-
PATE-
DP-
GAN
GAN
GAN
G-PATE
LR
0.9230
0.8228
0.8720
0.925I
AdaBoost
0,9416
0.8959
0.8809
0.981
Dataset
DC-GAN
|| 5
DP-GAN
PATE-GAN
GS-WGAN
G-PATR
Bagging
0.9379
0.8877
0.8657
0.8964
Average
MLP
0.9444
0.9917
0.8925
0.8872
0.8743
0.8787
0.9072
MNIST
(5 = oc)
0.9653
||
||
10
1
0.4036
0.8011
0.6667
0.4168
0.8066
0.1432
0.8092
0.5880
(h) Visual Quality Evaluation on Image
Fashion- MNIST
0.8052
||
1
0.1053
0.4222
0,1661
0.5812
(5 oc)
Datasets using Inception Score (IS).
|| 10
0.6098
0.6218
0.6579
0.6934
CelebA- Gendert))
0.8002
|| 1
0.5201
0.4448
0.6293
0.7016
Dataset
Real data
DP- GAN
PATE- GAN
GS- WGAN
PATE
(5 oc)
|| 10
0,5409
0.5870
0.6326
0.7072
MNIST
9.36
10
1,00
1,00
1.19
1.46
8.59
3.60
5.16
ColehA- GendertT.)
0.8149
||
1
0,5330
0.6068
0,5901
0.6702
(6 oc)
Fashion-
MNIST
9.01
1.05
1,03
2.35
1.69
1.00
3.41
CelehA- Hair
0.7678
|| 10
|| 1
0.3447
0,5211
0.3789
0.6535
0,6136
0,3375
0.6897
0.4985
CelebA
1.88
10
1,00
1.00
1.15
1.16
1.00
1,00
1.17
1.37
(6 oc)
10 0.3920
0.3900
0,3725
0.6217
Table 3: Visualization of Generated Instances by G-PATE. Row 1 (real image), row 2 (g = 10,8
Table 2: Data Utility (Aceuracy) on Image Datasets given Small Privacy Budgets. G-PATE and baselines are evaluated following the same way as Table Ic given 8 10 and low 5 S 1.0.
10 10 and row 3 (E = 1, 8 = cach
each presents one image from each class (the left 5 columns MNIST images, and the right 5 columns are Fashion-MNIST images).
MNIST
Fashion MNIST
DP GAN
PATE GAN
WGAN
GS
G-PATE
GAN
DP
PATE GAN
WGAN
G5
G-PATE
0
1
2
3
4
0.2 0.1104
0.2176
0.0972
0.22.1
0.3021
0.3605
0.1000
0.1874
0.6
0.4
0.1524
0.1022
0.2559
0.3484
0.1029
0.1044
0.2478
0.4184
8.0988
0.1302
0.2977
0.3698
0.1001
0.114
0.5020
0.4233
4
f
J
4
0.8
0.3732
0.3571
0.3170
0.5377
8.3210
0.3659
0.1242
0.5258
1.0
0.4066
0-1688
0.3432
0.5881
0.3053
0.4222
0.1661
0.5812
To compare the data utility of different data generators, we follow the standard and train a classifier on the synthetic data and test it on the real data to benchmark the usefulness of the synthetic data for downstream tasks. Specifically, we report the AUROC of the classifiers for the binary-class tabular dataset, and the classification accuracy trained on CNN for the multi-class image datasets to measure the data utility. In addition, for image datasets, we also evaluate the visual quality of the synthetic images using Inception Score
Evaluation Metrics.
protocol 37
6
(IS) [8 and Fréchet
inception distance (FID) 17 More details can be found in Appendix E and F
6.2 Evaluation Results
Kaggle Credit. The Kaggle Credit dataset is highly unbalanced, to generate the unbalanced synthetic data. In the first step, we calculate a differentially private estimation of the class distribution in the training dataset using the Laplacian mechanism [10] with
take two-step approach
& = 0.01. In the second step, we train a a (0.99, 10 10-5) --if-frrreeet private data generator and use it generate data that follow the estimated class distribution. By the composition theorem of differential privacy [T), the data generation mechanism is (1, 10 -0-f)-differentially private.
To compare with PATE-GAN, we select 4 commonly used classifiers evaluated in 377 and report the AUROC of the 4 classifiers trained on the corresponding synthetic data. We evaluate G-PATE under the same experimental setup PATE-GAN for 1. The results for baselines are following (3), and we obtain a higher baseline performance for DC-GAN compared to the results reported in 37
Table La presents the data utility analysis based on AUROC between G-PATE and PATE-GAN on Kaggle Credit dataset. G-PATE outperforms both PATE-GAN and DP-GAN and is close to the vanilla DC-GAN which has no privacy protection. The high performance of G-PATE is partly due to the relatively low dimensionality of the Kaggle Credit dataset and the abundance of training examples. More experimental results on Kaggle Credit dataset are presented in Appendix A.
8

---

Table Analysis on the Hyper-parameters. We performed comprehensive studies on the hyper- syper--aramelers of G-PATE (the number of teachers, the projection dimensions, gradient clipping constant c, and the number of bins B) for MNIST and Fashion-MNIST with E = 1 and d = 10 10- "N/A" means "no projections".
Projection Dimennioos
of Teachers n
Gradient Clipping Constant c
# bins
10
20
N/A
2000
3000
4000
5e-4
Io-4
5o:5
1o-5
10
20
MINIST
0,4638
0.5880
0.5604
0.1141
04420
0.5218
0,5880
0.4754
0.5880
0.5505
0.4668
0.5880
0.5810
0.7066
Fashion
0.5129
0.5812
0.5172
0.1268
0.3997
0.4874
0,5812
0,5140
0.5567
0,5812
0.5339
0.5575
0.5812
0,5400
Image Datsets. To understand G-PATE's performance on image datasets, we evaluate the data utility and visual quality of the generated images with G-PATE, PATE-GAN, DP-GAN, and GS- -GAN on the MNIST, Fashion-MNIST and CelebA datascts. The analysis is performed under two private settings: E = 1, = 10 5 and = 10, 6 = 10-5.
For the data utility. We report the classification accuracy of the synthetic data in Table 1c G-PATE outperforms baselines under both settings, and there is a more significant improvement for the setting with stronger privacy guarantee (i.e., = 1). Specifically, we observe that on the Fashion-MNIST dataset the synthetic records generated by DP-GAN under this setting are close to random noise, while the model trained on G-PATE generated data retains an accuracy of 58.12%.
To demonstrate the scalability of our algorithm, we also conduct experiments on the sigh-dimensional face dateset CelebA. The synthetic data generated by G-PATE is highly DP-GAN can barely converge given the high-dimensionality of the data. Particularly, even with the strict privacy budget e = 1, the accuracy of the generated data by G-PATE on only around 10% lower than the vanilla DC-GAN. Moreover, although the dimensionality of is 4x larger than (endbA-G, the accuracy of both datasets is very close, which again demonstrates the scalability of G-PATE.
while
To better understand different generative models, we evaluate the visual quality of the generated image data, though it is not the main focus of the differentially private data generator. In particular, we visualize the generated DP images in Table 3 We also provide an quantitative analysis based on Inception Score for G-PATE and baselines in Table [b] G-PATE can consistently generate better images than baselines when E = 1, for which GS- WGAN does not converge, which demonstrates the superiority of G-PATE. When = 10, G-PATE achieves the best performance on the high- dimensional face dataset CelebA. Although GS-W SG-N has better visual quality on MNIST and Fashion-MNIs MNIST when which suggests that the data utility and visual quality are two orthogonal metrics, and it would be an interesting future direction to improve the visual quality. More evaluation details and evaluation of Fréchet inception distance (FID) can be found in Appendix F
10, G-PATE has the best data utility across different settings and datasets,
Analysis on the Syper-parametersy We perform comprehensive ablation studies on the number of teachers and the the projection dimensions to gain better understanding about G-PATE. As shown in Table G-PATE benefits from having more teacher discriminators. Under the same privacy guarantee, the number of noisy votes (0, and σ2)) remains the same, so the output of the noisy voting algorithm is more likely to be correct, and the model would get better performance. However, this benefit diminishes as the training set for each teacher model gets smaller with the increasing number of teachers, and 4000 teachers have already achieved satisfiable results. Table 4 also demonstrates the effectiveness of the random projection method, which improves the classification accuracy by around 47% With larger projection dimensions, the privacy consumption increases rapidly as we are accessing more private data. But if the projection dimension is too small, useful information can be lost during projection. We find the best trade-off when projection dimension equals to 10. G-PATE achieves better performance given samller bins (S 10). With larger bins, the teachers attain lower agreement rate, leading to worse performance.
Analysis under Limited Privacy Budgets. We conduct another set of ablation studies given limited privacy budgets. From Table 2 under small E on both MNIST and Gashion-MNISS datasets. The utility stably increases when the privacy budgets increase. Among different small e, G-PATE achieves significantly higher accuracy than baselines. In particular, the accuracy of G-PATE under = 0.6 is four times higher than DP-GAN, which indicates that G-PATE able to generate differentially private data with high utility under low privacy budgets.
and Figure 2] we can observe that G-PATE starts to converge even

---

MNIST (under small E)
Fashion-MNIST (under small E)
DP-GAN
DP-GAN
50
PATE-GAN
50
PATE-GAN
accuracy (%)
GS-WGAN
GS-WGAN
40
G-PATE
40
G-PATE
accuracy
30
30
20
10
10
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
MNIST
(b) Fashion-MNIST
Figure 2: Data Utility (Accuracy) Image Datasets given Small Privacy Budgets. The accuracy of G- PATE and baseline models is plotted under tight privacy budget 10 1.0. G-PATE consistently outperforms the baseline models under limited privacy budgets.
Agreement of Teachers, In the gradient aggregation step, G-PATE relies on the agreement of teacher models to select the gradient direction. When there is a high agreement rate among teacher models, the gradient aggregator is more robust to noise and more likely to select gradient direction that preserves higher utility. Intuitively, because the training partitions of teacher models come from the same dataset, we expect the teacher models to learn similar real data distribution. As a consequence, the gradients generated by the teacher models are expected be similar. Empirically, we evaluate the agreement rate of teacher models on MNIST with 4000 teacher models and 1. On average, the gradient aggregator achieves 60.35% agreement rate for teacher votes on the most agreed direction, and 39.1 11% on the second agreed direction, which suggests that teacher models have high agreement rates on the top agreement directions.
Evaluation under a Data-Indepen Inta-Indeeen Privacy Budget. We evaluated the performance of G-PATE on MNIST with a iata-independent privacy analysis. We followed the same evaluation process in Table Tc and replaced the privacy analysis with the data-ing data-independent privacy bound (29]. When E the utility (i.e., classification accuracy) of G-PATE on MNIST is 0.5483, outperforming the existing baselines by large margin, which demonstrates that G-PATE still achieves the highest data utility with data-independent privacy cost.
Ablation Study on the Use of PATE. To understand how the PATE framework contributes to the advantage of G-PATE, we trained DP-GAN model with gradient discretization and random projection applied to DPSGD on the MNIST dataset under = 1 and = = 10. The model achieves the and 0.5602 (s 10) respectively. The results demonstrate
classification accuracy of 0.2026 that the PATE framework contributes significantly to G-PATE's utility advantage. First, with the PATE framework, G-PATE only needs add noise one layer of projected gradients between the teacher discriminators and the student generator, so the dimension of the noise equals the data dimension after projection. On the contrary, DP-GAN adds noise all the gradients of the model, so the dimension of the noise equals to the model dimension. Since the data dimension is usually significantly lower than the model dimension, the PATE framework helps G-PATE to reduce amount of noise needed to achieve the same privacy guarantee, and therefore preserves better utility. Second, in G-PATE, we have 4000 teachers to vote over the projected gradients and choose the most agreed gradient direction to update the model, which eliminates the noise from the random projection, saves privacy cost, and ensures high utility of the gradients due to the high consensus teacher discriminators. In comparison, DP-GAN does not have teacher models, and thus the quantized gradients with random projection can contain lot of noise during aggregation, yielding worse performance.
7 Conclusion
We propose G-PATE, a novel approach for training a differentially private data generator for high- dimensional data. G-PATE is enabled by novel differentially private gradient aggregation mechanism combined with random projection. significantly outperforms prior work on both image and non- image datasets in terms of preserving data utility for generated datasets. Beyond the high utility compared with the state-of-the-ar differentially private data generative models under similar setting, G-PATE is also able to preserve high data utility even given small privacy budgets.
10

---

Acknowledgement
This work was performed under the auspices of the U.S. Department of Energy by the Lawrence Livermore National Laboratory under Contract No. DE-AC52-00NA27344 and LLNL LDRD Program Project No. 20-ER-014 (LLNL-CONN--05509)) the NSF grant No. 191100, NSF CNS 20-46726 CAR, and the Amazon Research Award.
References
[1] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308-318. ACM, 2016.
[2] R. Bassily, A. Smith, and Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In Foundations of Computer Science (FOCS), 2014 IEEE 55th Annual Symposium on, pages 464-473. IEEE, 2014.
[3] E. Bingham and H. Mannila. Random projection in dimensionality reduction: applications to image and text data. Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 245-250. ACM, 2001.
[4] L. Breiman. Bagging predictors. Machine learning. 24(2):123 1996.
[5] K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk mini- mization. Journal of Machine Learning Research, 12MMa)): 2011.
[6] D. Chen, T. Orekondy, and M. Fritz, Gs-wgan: A gradient-sanitized approach for learning differentially private generators. arXiv preprint arXiv:2006.08265. 2020.
[7] A. Dal Pozzolo, Caelen, undersampling for unbalanced classification. In 2015 IEEE Symposiun Series Computational Intelligence, pages 159-166. IEEE, 2015.
Johnson, and G. Bontempi. Calibrating probability with
[8] M. de Bruijne. Machine learning approaches in medical image analysis: From detection to diagnosis, 2016.
[9] C. Dwork. Differential privacy: A survey of results. In International Conference on Theory and Applications of Models of Computation, pages 1-19. Springer, 2008
[10] C. Dwork, A. Roth, et al. The algorithmic foundations of diffcrential privacy. Foundations and Trends@ in Theoretical Computer Science, 9(3-4):211-44 2014.
[11] Y. Freund, E. Schapire, et al. Experiments with new boosting algorithm, In ieml, volume 96, pages 148-156. Citeseer, 1996.
[12] A. Friedman and A. Schuster. Data mining with differential privacy. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 493-502. ACM, 2010.
[13] S. Gong, V. N. Boddeti, and A. K. Jain. On the intrinsic dimensionality of face representation. CoRR, abs//801 801.8966332 2018.
[14] I. Goodfellow, J. Pouget-Abadic M. Mirza, B. Xu, D. Warde-Farley, S. Orair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances neural information processing systems, pages 2672-2680, 2014.
[15] F. Harder, K. Adamczewski, and M. Park. Dp-merf: Differentially private mean embeddings with randomfcatures for practical privacy presevvpg data generation. In International Confer- Artificial Intelligence and Statisties, pages 1819-1827 PMLR, 2021.
[16] M. Hardt, K. Ligett, and F. McSherry, A simple and practical algorithm for differentially private data release. arXiv preprint 1rXiv:1012.47660 2010.

---

[17] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler. and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. arXiv preprint arXiv: vrXiv:7006.08000 2017.
[18] A. Khetan and S. Oh. Achieving budget-optimality with adaptive schemes in crowdsourcing. In Lee, M. Sugiyama, U. Luxburg, Guyon, and R. Gamett, editors, Advances in Neural Information Processing Systems, volume 29, pages 4844-4852. Curran Associates, Inc., 2016.
[19] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
[20] K. Kourou, Exarchos, K. Exarchos, M. V. Karamouzis, and D. I. Fotiadis. Machine learn- ing applications in cancer prognosis and prediction. Computational and structural biotechnology journal, 13:8-17, 2015.
[21] H. H. Li, X. Xu, x. Zhang, Yang, and B. Li. Nonlinear projection based gradient estimation for query efficient blackbox attacks. In International Conference on Artificial Intelligence and Statistics, pages 3142-3150. PMLR, 2021.
[22] H. Li, X. Xu, X. Zhang, S. Yang, and B. Li. Qeba: Query-efficient boundary- blackbox at- tack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 1221-1230, 2020.
[23] S. P. Liew, T. Takahashi, and M. Ueno. Pearl: Data synthesis via private embeddings and adversarial reconstruction learning. arXiv preprint arXiv:2106.04455 2021.
[24] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
[25] McSherry and K. Talwar. Mechanism design via differential privacy. In Foundations of Computer Science, 2007. 07. 48th Annual IEEE Symposium on, pages 94-103. IEEE, 2007.
[26] M. Menze and A. Geiger. Object scene flow for autonomous vehicles. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 3061-3070, 2015.
[27] I. Mironov. Renyi differential privacy. In Computer Security Foundations Symposium (CSF), 2017 IEEE 30th, pages 263-275. IEEE, 2017.
[28] N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar. Semi-supervised knowl- edge transfer for deep learning from private training data. In International Conference on Learning Representations, 2017.
[29] N. Papernot, S. Song. I. Mironov, A. Raghunathan, K. Talwar, and U. Erlingsson. Scalable private learning with PATE. In International Conference Learning Representations, 2018.
[30] O. M. Parkhi, A. Vedaldi, A. Zisserman, et al. Deep face recognition. In hmve, volume 1, page 6, 2015.
[31] W. Qardaji, W. Yang, and Li. Priview: practical differentially private release of marginal contingency tables. In Proceedings of the 2014 ACM SIGMOD) international conference on Management of data, pages 1435-1446. ACM, 2014.
[32] A. Radford, L. Metz, and Chintala. Unsupervised representation learning with deep convolu- tional generative adversarial networks. arXiv preprint 15Xiv:5511.066 1.06434, 2015.
[33] R. Torkzadehmahani, P. Kairouz, and B. Paten. Dp-cgan: Differentially private synthetic data and label generation. In Pruceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition Workshops, pages 0-0, 2019.
[34] H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.077 077477777777 2017.
[35] I.. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou. Differentially private generative adversarial network. arXiv preprint arXiv: 1802::8802.06739 2018.
12

---

[36] C. Xu, J. Ren, Y. Zhang, Z. Qin, and K. Ren. Dppro: Differentially private high- high-dimensional data release via random projection. IEEE Transactions on Information Forensics and Security, 12(12):3081-3093, 2017.
[37] Yoon, J. Jordon, and M. van der Schaar. PATE-GAN:
Generating synthetic data with differential privacy guarantees. In International Conference on Learning Representations, 2019.
[38] J. Zhang, G. Cormode, C. M. Procopiue, D. Srivastava, and X. Xiao. Privbayes: Private data release via bayesian networks. ACM Transactions on Database Systems (TODS), 42(4):25, 2017.
[39] Zhang, L. Li, H. Li, Zhang. Yang, and Li. Progressive-scale boundary blackbox attack via projective gradient estimation. arXiv preprint arXiv: 100.20000000 16600605,, 2021.
[40] Y. Zhang, R. Jia, H. Pei, W. Wang, B. Li, and D. Song. The secret revealer: Generative model- inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 253-261, 2020.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6.2 where we
discuss improving the visual quality is an interesting future direction.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)? [Yes] See supplementary materials.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 6.1 about the experiment setup.
(c) Did you report error bars (e-g., with respect to the random seed after running experi-
ments multiple times)? [No] We report the best test accuracy.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix [E]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "zYWn0U0jTtV", "reviewer": "Reviewer_ouRc", "summary": "This paper presents G-PATE, a scalable version of PATE-GAN, that uses a new private gradient aggregation mechanism to\nensure differential privacy on information that flows from the teacher discriminators to the student generator. Experimental results show that G-PATE can work on both tabular and image data and it also show better performance on these data especially for smaller epsilon values compared to existing work such as DP-GAN, PATE-GAN, and few more models. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea is simple yet effective. The results looks promising with some limitations. Here are some questions/suggestions:\n\n- There are few recent works on synthetic generative models using private embeddings [1][2]. How G-Pate is compared with such models? The author can discuss this in related work (and/or in experiments if it is possible)\n\n- Is there any reference that shows PATE-GAN can only apply on categorical data? Does the authors mean the teacher discriminators output should be categorical or the dataset itself? If the latter, there are works showed PATE-GAN can be used on both categorical and continuous features [3]. Please clarify this in the paper. \n\n- It would be better if the authors showed the results on more tabular data containing both continues and categorical features. \n\n- For Celeb data, the authors resized the images to 32*32 and 64*64. It is interesting to also see how the model performs on high quality images. \n\n- How G-PATE's training time is compared with Pate-GAN and DP-GAN?\n\n- It seems the visualization of generated images are not very good especially for small epsilons. If the quality of generated samples is low what's the advantage of using DP synthesizer vs DP classifiers? Also, how is the quality of generated samples for CelebA data? \n\n[1] https://export.arxiv.org/pdf/2002.11603\n\n[2] https://arxiv.org/abs/2106.04590\n\n[3] https://arxiv.org/pdf/2011.05537.pdf\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be better if the authors mention the limitations of the proposed work and list few future directions for the improvement.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "rk0pLyjx5c3", "reviewer": "Reviewer_f9CE", "summary": "The paper proposes a new method called G-PATE for generating differentially private datasets. G-PATE is motivated from the PATE framework, by training teacher discriminators on real datasets, and then privately aggregating the gradients from those discriminators to train the generator. To aggregate the gradients, the paper proposes to project the gradients to a lower dimension, discretize the gradients, and then use the voting mechanism from PATE to aggregate the gradients. The paper demonstrates that G-PATE achieves better fidelity-privacy trade-offs than the two prior techniques: DP-GAN and PATE-GAN.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Privacy-preserving data generation is an important problem in both the research community and the industry. GANs can generate high-fidelity (non-private) data much better than many alternatives, and therefore, there are many interests in applying GANs for privacy-preserving data generation. However, the two main classes of approaches in this direction (i.e. DP-GAN and PATE-GAN) achieve bad fidelity-privacy trade-offs. \n\nThe approaches in this paper are very interesting. The idea of only making the generator differentially private while leaving the discriminator sensitive is a very good innovation, which makes it substantially different from PATE-GAN despite that both are motivated by the PATE framework. The idea of using the voting mechanism from PATE to aggregate the gradients is also very interesting, which utilizes the benefit of having multiple teachers, and makes it substantially different from DP-GAN.\n\nThe paper also conducts experiments on many different datasets and evaluation tasks, and demonstrates the superiority over DP-GAN, PATE-GAN, and GS-WGAN.\n\nTherefore, I would regard G-PATE as a completely new class of approaches for solving this problem. I can see that this paper will open up many follow-up works and create a new opportunity on solving this important problem. I think this is a very good paper with solid contributions.\n\nI only have one minor suggestion.\n* There is an inconsistency in how you explain your proposed gradient aggregation mechanism. In many parts of the paper (e.g. abstract, introduction), you say you are aggregating the gradient in a privacy-preserving way. However, in section 4.1, you say you are generating perturbations for the generated images, and then solving a mean squared error objective over the generator. From my understanding, both of the two explanations are correct and are indeed equivalent, because the gradient of Eq. 2  is exactly (a scaled version of) \\delta x, the backpropagated gradient in the discriminator till the input layer. However, mixing these two explanations up in the paper does create some confusion. I would recommend using a consistent explanation, and mention the other as an alternative understanding of the approach in Section 4.1. But the current presentation is also fine.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors discussed the limitations, but not the potential negative societal impact of the work.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}, {"review_id": "odcLq0geUj-", "reviewer": "Reviewer_Hstn", "summary": "The paper studies the differentially private data generator problem. The authors propose the G-PATE algorithm based on the PATE framework.  The G-PATE trains a student data generator with an ensemble of teacher discriminators.  In their approach, they combine the random projection and gradient discretization to release the aggregated gradient privately.  Empirically, they demonstrate that G-PATE improves the privacy-utility trade-offs over prior work. \n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Clarity: The paper is generally written well though the authors reiterate their contributions too much. \n\nQuestion:  In the standard PATE, the confidence parameter T is set to |#teachers*0.7|.  I wonder if the authors use the same confidence threshold T in all gradient positions or T needs to be tuned at different gradient locations?\n\nWeak points:\n1. My main concern is that the privacy guarantee seems a little bit artificial to me. The privacy loss of G-PATE scales to both the number of queries and the model dimension after projection. DP-GAN can benefit from privacy amplification by subsampling though its privacy loss is model-dependent. Moreover, PATE and PATE-GAN are model-agnostic, and the privacy cost is proportional to the number of queries while G-PATE is model-dependent. It makes me surprised that the proposed G-PATE take the disadvantages of both PATE and DP-GAN can beat both approaches. I suspect the advantage of the proposed G-PATE comes from the random projection and gradient discretization instead of the PATE framework. It will be interesting to investigate whether a DP-GAN (combined with gradient discretization and projection) is comparable with G-PATE.\n\n2. The privacy cost comparison in the experiments might be unfair. G-PATE reports the privacy cost under data-dependent RDP while other baselines (DP-GAN) report the standard DP. Note that the data-dependent RDP itself is private and is much smaller compared to its data-independent counterpart. I encourage the authors to either report the data-independent privacy cost or use the similar smooth sensitivity from the original PATE to sanitize the privacy cost. \n\nStrong points:\n1. The proposed gradient aggregation mechanism is novel and can be incorporated into other differentially private deep learning frameworks. \n2. The experimental results looks good to me despite the privacy comparison issues mentioned above. \n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors did not discuss the potential negative societal impact. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7 hours."}, "extra_text": {}}, {"review_id": "5SY_nlKOWJ", "reviewer": "Reviewer_YNC3", "summary": "This study proposes a new method for generating differentially private synthetic datasets called G-PATE. \n\nThis approach combines GANs (Generative Adversarial Neural Networks) with the PATE framework (Private Aggregation of Teacher Ensembles). The proposal utilizes known techniques from the differential privacy literature, such as gradient clipping and discretization.\n\nThe new algorithm diverges from previous methods such as DP-GAN, PATE-GAN, and GS-WGAN by creating a differentially private aggregation mechanism for directly using information from teacher discriminators to train the student generator. The method presented in the paper improves upon the previous proposals.\n\nDPGradAgg is designed which aggregates, in a differentially private manner, adversarial perturbations created by each teacher discriminator. The privacy loss of this new method is proven and empirical evaluations on a variety of datasets (Kaggle, MNIST, Fashion-MNIST, CelebA) are shown. The method demonstrates improvements in terms of downstream utility for classification with both smaller and larger epsilon guarantees.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong points:\n1. The student generator is trained not by ascending the gradient based on the loss from the discriminator but by using the aggregated adversarial perturbations from discriminators.\n2. An interesting combination of established techniques, such as gradient clipping, discretization, and random projection, to enable the private release of the gradient at a relatively low privacy cost.\n\nWeak points:\n1. The experiments can be improved. For example, authors use the CelebA dataset but not with 40 labels but only with, e.g., the binary labels to determine the gender. Could the authors provide results, for instance, for the CIFAR-10 dataset?\n2. The quality of the generated images is low.\n3. The proposed idea to incorporate differential privacy into GANs is not novel and improves on existing methods (e.g., PATE-GAN).\n4. The method is computationally costly: could authors compare the running times with other methods? What is the execution time of different parts of G-PATE?\n\nDetailed comments:\n\n- Originality: \n\nG-PATE differs from a typical GAN. Specifically, in G-PATE, the student generator is trained not by ascending the gradient based on the loss from the discriminator but by using the aggregated adversarial perturbations from discriminators. The direct comparison between GAN and non-private G-PATE in Appendix C indicates that G-PATE is comparable with GAN but this was done only for the simplest dataset.\n\nThis work builds off of PATE-GAN but the method introduces a new mechanism for releasing aggregated gradients (specifically in this case using the aforementioned adversarial perturbations). To my knowledge, this gradient aggregation mechanism is novel and plays a key role in the success of G-PATE over its predecessors such as PATE-GAN & DP-GAN. The study makes it clear how G-PATE differs from other algorithms. \n\n- Quality:\n\nThe submission provides both theoretical analyses and empirical evaluations of the proposed method. The theoretical analyses focus on understanding the privacy guarantees of G-PATE. Using the composition and post-processing properties of RDP the authors demonstrate that the privacy loss incurred by the method is simply the composition of the data-dependent RDP from the Confident GNMax aggregator over each component of the random projection dimension. This privacy loss analysis is sound to me. Additionally, the authors use composition over each training iteration to determine the privacy loss of the entire training procedure, which is correct. This analysis demonstrates that their proposed mechanism DPGradAgg is indeed differentially private.\n\nThe claims that are made about improved data utility are demonstrated but the results about improved image quality are less conclusive. G-PATE demonstrates improvements across all epsilons and datasets in terms of data utility, but these evaluations can also be improved. Given the high variance in the results of DP learning algorithms, it is unclear whether some of the increases seen would hold over multiple runs. An example of this is MNIST G-PATE vs GS-WGAN at epsilon 10 for measuring the accuracy of a classifier trained on the synthetic data. This is especially true in the data utility results on the Kaggle Credit Tabular dataset between PATE-GAN and G-PATE. When looking at the Inception score results there is no conclusive evidence that G-PATE provides better image quality. The authors address this as a future work consideration in the Inception Score section of the paper at Line 358. A suggestion for the authors to improve the clarity on G-PATE being better than PATE-GAN would be to have graphs that plot these metrics for epsilons between 0.5-10.\n\n- Clarity:\n\nLine 81 is incorrect: \"PATE-GAN [33] trains a student generator using an ensemble of teacher discriminators.\" PATE-GAN trains a student discriminator using an ensemble of teacher discriminators.\n\nShowing the results in Table 1 using graphs over a range of epsilons would help present the improvements over previous methods.\n\nThe authors distinguish between their idea and related papers in both the introduction and the related work section. The main contributions of the paper are repeated in a few sections. This should be reduced to single statements without repetitions.\n\nLines 285 to 290: Theorem 4 is simply a direct copy of Theorem 1. There is no need for proof G.4 in the appendix.\n\nRegarding Table 4, what is the performance of more teachers? Why are there so many more teachers than in the standard PATE?\n\nThe lines from 325 to 329 are not entirely clear. The Laplacian Mechanism used there incurs only 0.01 of the privacy budget. The explanation that $\\epsilon=0.99$ for a data generator and that data generation mechanism has $\\epsilon=1.0$ is relatively a very small difference.\n\nThe code is in the supplementary material but it is not prepared to be run. The authors do not specify which versions of the libraries are used (which are rather old versions). For example, which versions of Keras or TensorFlow are used?\n\nTypos:\n\nLine 75 - I believe “us” should be removed? This sentence is not entirely clear.\n\nLine 325: We calculate (not calculates)\n\nLine 374: the privacy consumption increase\"S\"\n\n\n- Significance:\n\nThis paper can be useful for the community. The empirical results demonstrate good improvements in data utility (which is often the motivation of DP synthetic data over actual image quality).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "It would be great to see if PATE-G provides improved data utility for subgroups in downstream classification since this is a large challenge currently. This has a societal impact in that there are contentious debates between the impact of DP on the accuracy of analyses for subgroups and any improvements to this would help the adoption of DP. The authors did a good job of listing the limitations of their work especially based on the empirical evaluation at the end of their experimental section.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviews for the paper were mixed. Unfortunately, the reviewers still did not change the score even after the rebuttal. I am still supporting acceptance because of the novel gradient aggregation scheme in the paper.

In the following I list some of the concerns that may not have been addressed adequately in the rebuttal, and the authors need to take care of them.

1. There was a concern that the advantage of the proposed G-PATE comes from the random projection and gradient discretization instead of the PATE framework. The authors did some initial experiments on DP-GAN to show that that the gradient discretization actually impacts the improvement in accuracy. However, these experiments are fairly preliminary, and needs a thorough discussion in the main text of the paper.

2. The quality of the images generated were not that great.  There were some concerns about the efficacy of the algorithm at higher epsilons >10, when compared to other approaches. Since single digit epsilons seem to be mostly the industry standard as of now, it is important to understand the efficacy of the algorithm in this privacy regime.
Decision: Accept (Poster)