Year: 2019

Abstract:
We present a novel approach G-PATE for training a scalable differentially private data generator, which can be used to produce synthetic datasets with strong privacy guarantee while preserving high data utility. Our approach leverages generative adversarial nets to generate data and exploits the PATE (Private Aggregation of Teacher Ensembles) framework to protect data privacy. Compared to existing methods, our approach significantly improves the use of privacy budget. This is possible since we only need to ensure differential privacy for the generator, which is the part of the model that actually needs to be published for private data generation. In particular, we connect a student generator with an ensemble of teacher discriminators and propose a private gradient aggregation mechanism to ensure differential privacy on all the information that flows from the teacher discriminators to the student generator. Theoretically, we prove that our algorithm ensures differential privacy for the generator. Empirically, we provide thorough experiments to demonstrate the superiority of our method over prior work on both image and non-image datasets.

heading: 1 INTRODUCTION
text: Machine learning has been applied to a wide range of applications such as face recognition (Parkhi et al., 2015), autonomous driving (Menze & Geiger, 2015), and medical diagnoses (de Bruijne, 2016; Kourou et al., 2015). However, most learning methods rely on the availability of large-scale training datasets containing sensitive information such as personal photos or medical records. Therefore, such sensitive datasets are often hard to be shared due to privacy concerns. To handle this challenge, data providers sometimes release synthetic datasets produced by generative models learned on the original data. Though recent studies show that generative models such as generative adversarial networks (GAN) (Goodfellow et al., 2014) can generate synthetic records that are indistinguishable from the original data distribution, there is no theoretical guarantee on the privacy protection. While privacy definitions such as differential privacy (Dwork & Feldman, 2018) and Rényi differential privacy (Mironov, 2017) provide rigorous privacy guarantee, applying them to synthetic data generation is nontrivial.
Recently, two approaches have been proposed to combine differential privacy with synthetic data generation: DP-GAN (Xie et al., 2018) and PATE-GAN (Yoon et al., 2019). DP-GAN modifies GAN by training the discriminator using differentially private stochastic gradient descent. Though it achieves privacy guarantee due to the post processing property (Dwork et al., 2014) of differential privacy, DP-GAN incurs significant utility loss on the synthetic data, especially when the privacy budget is low. In contrast, PATE-GAN trains differentially private GAN using the PATE mechanism (Papernot et al., 2017). Specifically, it first trains a set of teacher discriminators and then train a student discriminator based on the trained ensemble of teacher discriminators. To ensure differential privacy, the student discriminator is only trained on records that are produced by the generator and labeled by the teacher discriminators. The key limitation of this approach is that it relies on the assumption that the generator would be able to generate the entire “real” records space to bootstrap the training process. If most of the synthetic records are labeled as fake by the teacher discriminators, the student discriminator would be trained on a biased dataset and fail to learn the true data distribution. Consequently, this trained generator would not be able to produce high-quality synthetic data. This problem does not exist on a traditional GAN, where the discriminator is always
able to provide useful information to the generator since they can access the real data records rather than the synthetic data only. In addition, the two stage training process of PATE-GAN makes it less scalable or flexible in terms of varying the number of teacher discriminators.
The main contribution of this paper is a new approach named G-PATE for training a differentially private data generator by combining the generative model with PATE mechanism. Our approach is based on the key observation that: It is not necessary to ensure differential privacy for the discriminator in order to train a differentially private generator. As long as we ensure differential privacy on the information flow from the discriminator to the generator, it is sufficient to guarantee the privacy property for the generator. To achieve this, we propose a private gradient aggregation mechanism to ensure differential privacy on all the information that flows from the teacher discriminators to the student generator. Compared to PATE-GAN, our approach has three advantages. First, we improve the use of privacy budget by only applying it to the part of the model that actually needs to be released for data generation. Second, our discriminator can be trained on original data records since it does not need to satisfy differential privacy. Finally, G-PATE is much more scalable given its simple framework architecture.
Theoretically, we show that our algorithm ensures differential privacy for the generator. Empirically, we conduct extensive experiments on the standard Kaggle credit card fraud detection dataset, as well as two image datasets MNIST and Fashion-MNIST. The results show that our method significantly outperforms all baselines including DP-GAN and PATE-GAN in terms of data utility.

heading: 2 RELATED WORK
text: Differential privacy (Dwork, 2008) is a notion that ensures an algorithm outputs general information about its input dataset without revealing individual information. Rényi differential privacy (Mironov, 2017) is a relaxation of differential privacy that allows tighter analysis on the composition of privacy budgets. Following the work of differential privacy, researchers have proposed different methods to design differentially private statistical functions and machine learning models (Bassily et al., 2014; Chaudhuri et al., 2011; Abadi et al., 2016; McSherry & Talwar, 2007; Friedman & Schuster, 2010). Recently, various approaches have been proposed for differentially private data generation. Priview (Qardaji et al., 2014) generates synthetic data based on marginal distributions of the original dataset, and PrivBayes (Zhang et al., 2017) trains a differentially private Bayesian network. However, these approaches are not suitable for image datasets since the statistics they use cannot well preserve the correlations between pixels in an image.
Both DP-GAN (Xie et al., 2018) and PATE-GAN (Yoon et al., 2019) apply differential privacy to the training process of generative adversarial networks. They both ensure differential privacy while training the discriminator, and the privacy property of the generator is guaranteed by the post processing property of differential privacy. Unlike their approaches, our method improves the use of privacy budget by only ensuring differential privacy on the generator, which is the part that actually needs to be released for synthetic data generation. This improvement allows us to incur much lower utility loss on the synthetic data under the same privacy constraint.
Private aggregation of teacher ensembles (PATE) is a method to train a differentially private classifier using ensemble mechanisms. It first trains an ensemble of teacher models on disjoint subsets of the sensitive training data. Then, a differentially private student model is trained on public data labeled by the teacher models. The privacy guarantee of PATE is more intuitive to understand because no teacher model can dictate the training of the student model. Compared to most differentially private classification method, PATE benefits from a tighter data-dependent privacy bound especially when teacher models are likely to reach consensus. Scalable PATE (Papernot et al., 2018) improves the utility of PATE with a Confident-GNMax aggregator that only returns a result if it has high confidence in the consensus among teachers. However, both PATE and Scalable PATE relies on the availability of public unlabeled data, and their aggregators are only applicable to categorical data (i.e., class labels). In contrast, our method does not rely on any public dataset and can generate synthetic samples that are differentailly private with respect to the private training dataset. We also design a differentially private gradient aggregator that works for continuous gradient vectors.

heading: 3 BACKGROUND
text: We recall the definition of differential privacy (DP), Rényi differential privacy (RDP), and some of their properties.

heading: 3.1 DIFFERENTIAL PRIVACY
text: Differential privacy bounds the shift in the output distribution of a randomized algorithm that could be caused by a small input perturbation. The following definition formally describes this privacy guarantee.
Definition 1 ((ε, δ)-Differential Privacy). A randomized algorithmM with domain N|X | is (ε, δ)differentially private if for all S ⊆ Range(M) and for any neighboring datasets D and D′:
Pr[M(D) ∈ S] ≤ exp(ε) Pr[M(D′) ∈ S] + δ.

heading: 3.2 RÉNYI DIFFERENTIAL PRIVACY
text: Rényi differential privacy is a natural relaxation of differential privacy. Defined below, its privacy guarantee is expressed in terms of Rényi divergence.
Definition 2 ((λ, ε)-RDP). A randomized mechanismM is said to guarantee (λ, ε)-RDP with λ > 1 if for any neighboring datasets D and D′,
Dλ (M(D)‖M (D′)) = 1
λ− 1 logEx∼M(D) [( Pr[M(D) = x] Pr [M (D′) = x] )λ−1] ≤ ε.
(λ, ε)-RDP implies (εδ, δ)-differential privacy for any given probability δ > 0.
Theorem 1 (From RDP to DP). If a mechanismM guarantees (λ, ε)-RDP, thenM guarantees (ε+ log 1/δλ−1 , δ)-differential privacy for any δ ∈ (0, 1).
Compared to DP, RDP supports easier composition of multiple queries and clearer privacy guarantee under Gaussian noise. Specifically, RDP could be easily composed by adding the privacy budget:
Theorem 2 (Composition of RDP). If a mechanismM consists of a sequnce ofM1, . . . ,M)k such that for any i ∈ [k],Mi guarantees (λ, εi)-RDP, thenM guarantees (λ, ∑k i=1 εi)-RDP.
Suppose f is a real-valued function, and the Gaussian mechanism is defined as follows: Gσf(D) = f(D) +N ( 0, σ2 ) ,
where N ( 0, σ2 ) is normally distributed random variable with standard deviation σ and mean 0. The Gaussian mechanism provides the following RDP guarantee:
Theorem 3 (RDP Guarantee for Gaussian Mechanism). If f has sensitivity 1, then the Gaussian mechanism Gσf satisfies ( λ, λ/ ( 2σ2 )) -RDP.

heading: 4 THE G-PATE METHOD
text: In this section, we present our method named G-PATE. An overview of the method is shown in Figure 1. Unlike PATE-GAN and DP-GAN, G-PATE ensures differential privacy for the information flow from the discriminator to the generator. This improvement incurs less utility loss on the synthetic samples, so it can generate synthetic samples for higher dimensional and more complex datasets.
G-PATE makes two major modifications on the training process of GAN. First, we replace the discriminator in GAN with an ensemble of teacher discriminators trained on disjoint subsets of the sensitive data. The teacher discriminators do not need to be published, thus can be trained using non-private algorithms. In addition, we design a gradient aggregator to collect information from teacher discriminators and combine them in a differentially private fashion. The output of the aggregator is a gradient vector that guides the student generator to improve its synthetic samples.
Unlike PATE-GAN, G-PATE does not require any student discriminator. The teacher discriminators are directly connected to the student generator. The gradient aggregator sanitizes the information flow from the teacher discriminators to the student generator to ensure differential privacy. This way, G-PATE uses privacy budget more efficiently and better approximates the real data distribution to ensure high data utility.

heading: 4.1 TRAINING THE STUDENT GENERATOR
text: To achieve better privacy budget efficiency, G-PATE only ensures differential privacy for the generator and allows the discriminators to learn private information. The privacy property is achieved by sanitizing all information propagated from the discriminators to the generator. To ease privacy analysis, we decompose G-PATE into three parts: the teacher discriminators, the student generator, and the gradient aggregator. To prevent the propagation of private information, the student generator does not have direct access to any information in any of the teacher discriminators. Consequently, we cannot train the student generator by ascending its gradient based on loss of the discriminators. To solve this problem, we propose the use of adversarial perturbation, which is a small manipulation on the fake record x that causes the discriminator’s loss on x to increase. The adversarial perturbation is calculated by ascending x’s gradients on the loss of the discriminator. It teaches the student generator how to improve its fake records. In each training iteration, the student generator is updated in three steps: (1) A teacher discriminator generates adversarial perturbations for each record produced by the student generator. (2) The gradient aggregator takes the adversarial perturbations from all teacher models and generates a differentially private aggregation of them. (3) The student generator updates its weights based on the privately aggregated adversarial perturbation. The process is formally presented in Algorithm 1.
Generating Adversarial Perturbations. Let D be a teacher discriminator. Given a fake record x, we use LD(x) to represent D’s loss on x. In each training iteration, the weights of D are updated by descending their stochastic gradients on LD. For each input fake record x, we generate an adversarial perturbation ∆x that guides the student generator on improving its output. By applying the perturbation on its output, the student generator would get an improved fake record x̂ = x + ∆x on which D has a higher loss. Therefore, ∆x is calculated as x’s gradients on LD:
∆x = ∂LD(a) ∂a ∣∣∣∣ a=x . (1)
Algorithm 1 - Training the Student Generator. The student generator is jointly trained with an ensemble of teacher discriminators. In each iteration, the student generator updates its weights based on an aggregated adversarial perturbation generated by the teacher ensemble.
Require: batch size m, number of teacher models n, number of training iterations N , gradient clipping constant c, number of bins B, projected dimension k, noise parameters σ1 and σ2, threshold T , disjoint subsets of sensitive data d1, d2, . . . , dn
1: for number of training iterations do 2: Sample m noise samples {z1, z2, . . . , zm} 3: Generate fake samples {G(z1), G(z2), . . . , G(zm)} 4: for i ∈ {1, . . . , n} do 5: Sample m data samples {x1, x2, . . . , xm} from di 6: Update the teacher discriminator Di by descending its stochastic gradient on LDi on both fake samples and real samples 7: for j ∈ {1, . . . ,m} do 8: Calculate the adversarial perturbation ∆x(i)j as xj’s gradients on LDi(xj) 9: end for
10: end for 11: for j ∈ {1, . . . ,m} do 12: ∆xj ← DPGradAgg(∆x(1)j ,∆x (2) j , . . . ,∆x (n) j , c, B, k, σ1, σ2, T ) 13: x̂j ← G(zj) + ∆xj 14: end for 15: Update the student generator G by descending its stochastic gradient on LG on {x̂1, x̂2, . . . , x̂m} 16: end for 17: return G
With the adversarial perturbation ∆x, the student generator can be trained without direct access to the discriminator’s loss.
Updating the Student Generator. A student generator G learns to map a random input z to a fake record x = G(z) so that x is indistinguishable from a real record by D. Given an adversarial perturbation ∆x, the teacher discriminators have higher loss on the perturbed fake record x̂ = x+∆x compared to the original fake record x. Therefore, the student generator learns to improve its fake records by minimizing the mean squared error (MSE) between its output G(z) and the perturbed fake record x̂.
LG(z, x̂) = 1
k k∑ i=1 (G(z)i − x̂i)2, (2)
where k is the number of synthetic records generated per training iteration. To ensure differential privacy, instead of receiving the adversarial perturbation from a single discriminator, we train the student generator using a differentially private gradient aggregator that combines adversarial perturbations from multiple teacher discriminators. Details are provided in Section 4.2.

heading: 4.2 DIFFERENTIALLY PRIVATE GRADIENT AGGREGATION FOR G-PATE
text: G-PATE consists of a student generator and an ensemble of teacher discriminators trained on disjoint subsets of the sensitive data. In each training iteration, each teacher discriminator generates an adversarial perturbation ∆x that guides the student generator on improving its output records. Different from traditional GAN, in G-PATE, the student generator does not have access to the loss of any teacher discriminators, and the adversarial perturbation is the only information propagated from the teacher discriminators to the student generator. Therefore, to achieve differential privacy, it suffices to add noise during the aggregation of the adversarial perturbations.
However, the aggregators used in PATE and PATE-GAN are not suitable for aggregating gradient vectors because they are only applicable to categorical data. Therefore, we propose a differentially private gradient aggregator (DPGradAgg) based on PATE. With gradient discretization, we convert gradient aggregation into a voting problem and get the noisy aggregation of teachers’ votes using
PATE. Additionally, we use random projection to reduce the dimension of vectors on which the aggregation is performed. The combination of these two approaches allows G-PATE to generate synthetic samples with higher data utility, even for large scale image datasets, which is hard to be achieved by PATE-GAN. The procedure is formally presented in Algorithm 2.
Algorithm 2 - Differentially Private Gradient Aggregator (DPGradAgg). This algorithm takes a list of gradient vectors and returns a differentially private aggregation of them.
Require: gradient vectors {∆x(1),∆x(2), . . . ,∆x(n)}, gradient clipping constant c, number of bins B, projected dimension k, noise parameters σ1 and σ2, threshold T
1: k0 ← the dimension of ∆x(1) 2: R← a k0 by k random projection matrix with each component randomly drawn from N (0, 1k ) 3: {∆u(1),∆u(2), . . . ,∆u(n)} ← {∆x(1)R,∆x(2)R, . . . ,∆x(n)R} 4: ∆u← empty list 5: for j ∈ 1, 2, . . . , k do 6: v ← a vector containing the jth element of all gradients in {∆u(1),∆u(2), . . . ,∆u(n)} 7: Clip v to (−c, c) 8: h← the histogram of v with B bins of width 2cB 9: j ← Confident-GNMax(h, σ1, σ2, T )
10: Append the midpoint of the j-th bin to ∆u 11: end for 12: ∆x← ∆uRT 13: return ∆x
Gradient Discretization. Since PATE is originally designed for aggregating the teacher models’ votes on the correct class label of an example, the aggregation mechanism in PATE only applies to categorical data. Therefore, we design a three-step algorithm to apply PATE on continuous gradient vectors. First, we discretize the gradient vector by creating a histogram and mapping each element to the midpoint of the bin it belongs to. Then, instead of voting for the class labels as in PATE, a teacher discriminator votes for k bins associated with k elements in its gradient vector. Finally, for each dimension, we calculate the bin with most votes using the Confident-GNMax aggregator (Papernot et al., 2018) (Algorithm 3 in Appendix A.4). The aggregated gradient vector is consisted of the midpoints of the selected bins.
With gradient discretization, the teacher discriminators can directly communicate with the student generator using the PATE mechanism. Since these teacher discriminators are trained on real data, they can provide much better guidance to the generator compared to the student discriminator in PATEGAN, which is only trained on synthetic samples. Moreover, the Confident-GNMax aggregator ensures that the student generator would only improve its output in the direction agreed by most of the teacher discriminators.
Random Projection. Aggregation of high dimensional vectors is expensive in terms of privacy budget because private voting needs to be performed on each dimension of the vectors. To save privacy budget, we use random projection (Bingham & Mannila, 2001) to reduce the dimensionality of gradient vectors. Before the aggregation, we generate a random projection matrix with each component randomly drawn from a Gaussian distribution. We then project the gradient vector into a lower dimensional space using the random projection matrix. After the aggregation, the aggregated gradient vector is projected back to its original dimensions. Since the generation of random projection matrix is data-independent. It does not consume any privacy budgets.
Random projection is shown to be especially effective on image datasets. Since different pixels of an image are often highly correlated, the intrinsic dimension of an image is usually much lower than the number of pixels (Gong et al., 2018). Therefore, random projection maximizes the amount of information a student generator can get from a single query to the Confident-GNMax aggregator, and makes it possible for G-PATE to retain reasonable utility even on high dimensional data. Moreover, random projection preserves similar squared Euclidean distance between high-dimensional vectors, therefore is beneficial to privacy protection both theoretically and empirically (Xu et al., 2017).

heading: 5 THEORETICAL GUARANTEES
text: In this section, we provide theoretical guarantees on the privacy properties for G-PATE. To start with, we propose the following definition for a differentially private data generative model.
Definition 3 (Differentially Private Generative Model). Let G be a generative model that maps a set of points Z in the noise space Z to a set of records X in the data space X . Let D be the training dataset of G andA : D 7→ G be the training algorithm. We say that G is a (ε, δ)-differentially private data generative model if the training algorithm A is (ε, δ)-differentially private.
Definition 3 relaxes the definition of a DP-GAN by focusing the protection only on the generative model in a GAN. This relaxation saves privacy budget during training and improves the utility of the model. Moreover, the relaxation does not compromise the privacy guarantee for the synthetic data.
Theorem 4 shows that a differentially private generative model is able to support infinite number of queries to data generator and can be used to generate multiple synthetic datasets.
Theorem 4. Let G be an (ε, δ)-differentially private data generative model trained on a private dataset D. For any Z ∈ Z , the synthetic dataset X = G(Z) is (ε, δ)-differentially private.
Theorem 4 is a consequence of the post-processing property of differential privacy. First, the random points Z are independent of the private dataset D. Second, one does not need to query the discriminator during the data generation process. Therefore, the synthetic dataset is generated by post-processing G and is guaranteed to be (ε, δ)-differentially private.
The next theorem justifies the Rényi differential privacy of the G-PATE method.
Theorem 5 (Rényi Differential Privacy of G-PATE). Let A be the training algorithm for the student generator (Algorithm 1) with N training iterations and k projected dimensions. The data-dependent Rényi differential privacy for A with order λ > 1 is ε = ∑ 1≤i≤N (∑ 1≤j≤k εi,j ) , where εi,j is the data-dependent Rényi differential privacy for the Confident-GNMax aggregator in the i-th iteration on the j-th projected dimension.
Theorem 5 holds because of the composition theorem of Rényi differential privacy (Theorem 2 in Appendix A.4). During the training process, the student generator accesses information about the sensitive dataset through the Confident-GNMax aggregator. It submits k queries per training iteration. Therefore, the Rényi differential privacy budget of the training algorithm is a composition of the data-dependent Rényi differential privacy budget of the Confident-GNMax aggregator over k dimensions and N iterations. The data-dependent privacy budget for each Confident-GNMax aggregation is dependent on σ1, σ2, and threshold T . We include the analysis in Appendix A.4. The remaining parameters (e.g. gradient clipping constant c, number of bins B) do not influence the privacy guarantee.
The next theorem provides a theoretical guarantee on the differential privacy of the G-PATE method.
Theorem 6 (Differential Privacy of G-PATE). Given a sensitive datasetD and parameters 0 < δ < 1, let G be the student generator trained by Algorithm 1. There exists ε > 0 and λ > 1 so that G is a (ε+ log 1/δλ−1 , δ)-differentially private data generative model.
Theorem 6 is the consequence of converting the Rényi differential privacy guarantee in Theorem 5 to differential privacy (Theorem 1 in Appendix A.4).

heading: 6 EXPERIMENTAL EVALUATION
text: We evaluate G-PATE against two state-of-the-art benchmarks: DP-GAN and PATE-GAN. To compare the performance of different data generators, we train a classifier on the synthetic data and test it on the original data to benchmark the quality of the synthetic data.
We first perform comparative analysis with PATE-GAN and DP-GAN on the datasets used in the corresponding works (i.e., Kaggle credit dataset and MNIST dataset). In addition, we evaluate G-PATE on the Fashion-MNIST dataset consisting of real-world images of clothes.

heading: 6.1 EXPERIMENTAL SETUP
text: To compare with PATE-GAN, we use the same Kaggle credit card fraud detection dataset (Dal Pozzolo et al., 2015) (Kaggle Credit) as in Yoon et al. (2019) 1. The dataset contains 284,807 samples representing transactions made by European cardholders’ credit cards in September 2013, and 492 (0.2%) of these samples are fraudulent transactions. Each sample consists of 29 continuous features which are the results of a PCA transformation from the original features.
To demonstrate the superiority of G-PATE to PATE-GAN on high dimensional image datasets, we train G-PATE on the MNIST and Fashion-MNIST datasets (Xiao et al., 2017). Each of them consists of 60,000 training examples and 10,000 testing examples. Each example is a 28 × 28 grayscale image, associated with a label from 10 classes. The examples in the MNIST dataset are images of handwritten digits between 0 and 9, and the examples in the Fashion-MNIST dataset are real-world images of clothes taken from the Zalando articles. Fashion-MNIST is proposed as a replacement for the MNIST dataset because it better represents modern computer vision tasks.
For the Kaggle Credit dataset, both the generator and discriminator networks of G-PATE are fully connected neural network with the same architecture as PATE-GAN (Yoon et al., 2019). We use random projection with 5 projected dimensions during gradient aggregation. We use the DCGAN (Radford et al., 2015) structure on both MNIST and Fashion-MNIST. We use random projection with 10 projected dimensions during gradient aggregation. More details are provided in Appendix A.5.

heading: 6.2 COMPARISON WITH DP-GAN AND PATE-GAN
text: Kaggle Credit. The Kaggle Credit dataset is highly unbalanced. In PATE-GAN, the ratio between positive and negative classes in the sensitive training set is assumed to be public information. In contrast, the G-PATE method does not rely on any public information about the sensitive training dataset. It calculates the ratio between positive and negative classes using Laplacian mechanism (Dwork et al., 2014) with ε = 0.01. We then train a (0.99, 10−5)-differentially private data generator and sample the synthetic records according to the noisy class ratios. By the composition theorem of differential privacy (Dwork et al., 2014), the data generation mechanism is (1, 10−5)-differentially private.
To compare with PATE-GAN, we select 4 commonly used classifiers evaluated in Yoon et al. (2019). The performance of a generator is measured by the AUROC of the 4 classifiers trained on the corresponding synthetic data. We evaluate G-PATE under the same experimental setups as PATEGAN for ε = 1.2 The results for PATE-GAN and DP-GAN are from Yoon et al. (2019), and we get a higher baseline performance from GAN compared to the baseline performance reported in Yoon et al. (2019).
Table 6 presents the comparative analysis between G-PATE and PATE-GAN on Kaggle Credit dataset. G-PATE outperforms both PATE-GAN and DP-GAN and is close to the original GAN which has no privacy protection. The good performance of G-PATE is partly due to the relatively low dimensionality of the Kaggle Credit dataset and the abundance of training examples. More experimental results on Kaggle Credit dataset are presented in Appendix A.1.
1PATE-GAN does not open source, so we directly compare with the results they reported. 2We reproduce the experimental setups of PATE-GAN to the best of our knowledge according to the paper.
MNIST and Fashion-MNIST. To understand G-PATE’s performance on image datasets, we perform comparative analysis between G-PATE and DP-GAN on the MNIST and Fashion-MNIST datasets3. We evaluate the generator by the 10-class classification accuracy of models trained on synthetic data and tested on real data (Table 2). The analysis is performed under two performance settings: ε = 1, δ = 10−5 and ε = 10, δ = 10−5. G-PATE outperforms DP-GAN under both settings, and there is a more significant improvement for the setting with stronger privacy guarantee (i.e., ε = 1). Specifically, we observe that DP-GAN fails to converge on the Fashion-MNIST dataset with ε = 1. The synthetic records generated by DP-GAN under this setting are close to random noise while the model trained on G-PATE generated data retains an accuracy of 51.74%.
Analysis on the Number of Teachers and the Projection Dimensions. We perform comprehensive ablation studies on the number of teachers and the the projection dimensions to gain better understanding about G-PATE. As shown in Table 3, G-PATE benefits from having more teacher discriminators. Under the same privacy guarantee, the number of noisy votes (σ1 and σ2) remains the same, so the output of the noisy voting algorithm is more likely to be correct, and the model would get better performance. However, this benefit diminishes as the training set for each teacher model gets smaller with the increasing number of teachers, and 4000 teachers have already achieved satisfiable results. Table 3 also demonstrates the effectiveness of the random projection method, which improves the classification accuracy by around 0.45.
3We are unable perform comparative study with PATE-GAN on image datasets because PATE-GAN does not report any results on images.

heading: 7 CONCLUSION
text: This paper proposes G-PATE, a novel approach for training a differentially private data generator by ensuring privacy property on the information flow from the discriminator to generator in GAN. G-PATE is enabled by a differentially private gradient aggregation mechanism combined with random projection. It significantly outperforms prior work on both image and non-image datasets. Moreover, G-PATE retains reasonable utility on complex image dataset for which DP-GAN can hardly converge.

heading: A APPENDIX
text: A.1 ADDITIONAL EVALUATION RESULTS ON KAGGLE CREDIT DATASET
In addition to AUROC, we also evaluate the AUPRC of the classification models trained on the synthetic data produced by different generative models. Table 4 presents the results. G-PATE has the best performance among all the differentially private generative models.
To understand the upper-bound of the classification models’ performance. We train the same classification models on real data and test it on real data. The results are presented in Table 5.
A.2 SYNTHETIC IMAGES GENERATED BY G-PATE
Figure 3 presents the synthetic images generated by G-PATE on MNIST and Fashion-MNIST. Images in the same column share the same class label. Row 1 contain real images in the training dataset; row 2 contain images generated by G-PATE when = 10, δ = 10−5; and row 3 contain images generated by G-PATE when = 1, δ = 10−5.
A.3 PERFORMANCE ANALYSIS ON NONPRIVATE GPATE
To understand how the GPATE training framework influence the performance of a GAN, we train a nonprivate GPATE with 10 teacher models. As shown in Table 6, the GPATE structure has a comparable performance to the vanilla GAN.
A.4 PRIVACY BUDGET OF CONFIDENT-GNMAX
The Confident-GNMax aggregator was proposed by Papernot et al. (2018) to support differentially private aggregation of the votes from multiple teacher models. For the completeness of this paper, in this section, we include the algorithm for the Confident-GNMax aggregator and its data-dependent RDP guarantee.
A.4.1 THE CONFIDENT-GNMAX AGGREGATOR
Algorithm 3 Confident-GNMax Aggregator. The private aggregator used in the scalable PATE framework Papernot et al. (2018).
Require: input x, threshold T , noise parameters σ1 and σ2 1: if maxi{nj(x)}+N (0, σ21) ≥ T then 2: return arg max{nj(x) +N (0, σ22)} 3: else 4: return ⊥ 5: end if
Algorithm 3 presents the Confident-GNMax aggregator proposed by Papernot et al. (2018). The algorithm contains two steps. First, it computes the noisy maximum vote
M1 = max i {nj(x)}+N (0, σ21).
Then, if the noisy maximum vote is greater than a given threshold, it uses the GNMax mechanism to select the output with most votes:
M2 = arg max{nj(x) +N (0, σ22)}.
Since each teacher model may cause the maximum number of vote to change at most by 1, M1 is equivalent to a Gaussian mechanism with sensitivity 1. Therefore, following Theorem 3, M1 with Gaussian noise of variance σ21 guarantees (λ, λ/2σ 2 1)-RDP for all λ > 1.
M2 could be decomposed into post-processing a noisy histogram with Gaussian noise added to each dimension. Since each teacher model may increase the count in one bin and decrease the count in another, the mechanism has a sensitivity of 2. Therefore, M2 with Gaussian noise of variance σ22 guarantees (λ, λ/σ22)-RDP (Papernot et al., 2018).
The data-dependent privacy guarantee for the GNMax mechanism M2 has been analyzed by Papernot et al. (2018):
Theorem 7. Let M be a randomized algorithm with (µ1, ε1)−RDP and (µ2, ε2)−RDP guarantees and suppose that there exists a likely outcome i∗ given a dataset D and a bound q̃ ≤ 1 such that q̃ ≥ Pr [M(D) 6= i∗]. Additionally, suppose that λ ≤ µ1 and q̃ ≤ e(µ2−1)ε2/ ( µ1 µ1−1 · µ2 µ2−1 )µ2 . Then, for any neighboring dataset D′ of D, we have:
Dλ (M(D)‖M (D′)) ≤ 1 λ− 1 log ( (1− q̃) ·A (q̃, µ2, ε2)λ−1 + q̃ ·B (q̃, µ1, ε1)λ−1 ) ,
where A (q̃, µ2, ε2) , (1− q̃)/ ( 1− (q̃eε2) µ2−1 µ2 ) and B (q̃, µ1, ε1) , eε1/q̃ 1 µ1−1 .
The parameters µ1 and µ2 are optimized to get a data-dependent RDP guarantee for any order λ. By applying Theorem 7 on M2, we obtain the data-dependent RDP budget for M2.
For any λ > 1, suppose ε1 is the RDP budget for M1 and ε2 is the data-dependent RDP budget for M2. Then, the RDP budget for the Confident-GNMax algorithm could be calculated as follows:
ε = { ε1 if output is ⊥, ε1 + ε2 otherwise.
A.5 MODEL STRUCTURES AND HYPERPARMETERS
G-PATE. For MNIST and Fashion-MNIST, the student generator consists of a fully connected layer with 1024 units and a deconvolutional layer with 64 kernels of size 5× 5 (strides 2× 2). Each teacher discriminator has a convolutional layer with 32 kernels of size 5 × 5 (strides 2 × 2) and a fully connected layer with 256 units. All layers are concated with the one-hot encoded class label. We apply batch normalization and Leaky ReLU on all layers. When ε = 10, we train 2000 teacher discriminators with batch size of 30 and set σ1 = 600, σ2 = 100. When ε = 1, we train 4000 teacher discriminators with batch size of 15 and set σ1 = 3000, σ2 = 1000. For Kaggle Credit dataset, we train 2100 teacher discriminators with batch size of 32 and set σ1 = 1500, σ2 = 600. For all three datasets, we use Adam optimizer Kingma & Ba (2014) with learning rate of 10−3 to train the models and clip the adversarial perturbations between ±10−4. The consensus threshold T is set to 0.5.
GAN. The structure of GAN is the same as the structure of G-PATE with a single teacher discriminator. The hyper-parameters are also the same as G-PATE.
DP-GAN. We use DP-GAN method mentioned in Xie et al. (2018) on both MNIST and FashionMNIST tasks. For the generator, we use FC Net structure with [128, 256, 512, 784] neurons in each layer, and the discriminator contains [784, 64, 64, 1] neurons in each layer. In each training epoch, the discriminator trains 5 steps and the generator trains 1 step. For both networks, 0.5 × ReLU(·) activation layers are used. Our batch size is 64 for each sampling, and sampling rate q equals to 646×104 . We bound the discriminator’s parameter weights to [−0.1, 0.1] and kept feature’s value between [−0.5, 0.5] during the forward process. In order to generate specific digit data, we concat one-hot vector, which represents digits categories, into each layer in both the disciminator and the generator.
Classification Models for MNIST and Fashion-MNIST. For each synthetic dataset, we trian a CNN for the classification task. The model has two convolutional layers with 32 and 64 kernels respectively. We use ReLU as the activation function and applies dropout on all layers.
Classification Models for Kaggle Credit. We implement 4 predictive models in Yoon et al. (2019) using sklearn: Logistic Regression (LogisticRegression), Adaptive Boosting (AdaBoostClassifier), Bootstrap Aggregating (BaggingClassifier) and Multi-layer
Perceptron (MLPClassifier). We use L1 penalty, Liblinear solver and (350:1) class weight in Logistic Regression. We use logistic regression as classifier in Adaptive Boosting and Bootstrap Aggregating, setting L2 penalty, number of models as 200 and 100. For Multi-layer Perceptron, we use tanh as the activation of 3 layers with 18 nodes and Adam as the optimizer.

Review summaries:
This paper presents a framework for Differentially private data generation that enables more accurate training of downstream learners without compromising on the privacy guarantees. The key insight is to introduce an ensemble of teacher discriminators in the GAN formulation instead of a single discriminator. The teachers are trained on separate subsets of the private data set. A gradient aggregator is also introduced for transmitting loss signal to the student without losing privacy by using adversarial perturbations and random projections. The authors provide theoretic guarantees of Renyi differential privacy and experiments on Kaggle Credit default dataset and MNIST.