PATE-GAN: GENERATING SYNTHETIC DATA WITH DIFFERENTIAL PRIVACY GUARANTEES
Year: 2019

Abstract:
Machine learning has the potential to assist many communities in using the large datasets that are becoming more and more available. Unfortunately, much of that potential is not being realized because it would require sharing data in a way that compromises privacy. In this paper, we investigate a method for ensuring (differential) privacy of the generator of the Generative Adversarial Nets (GAN) framework. The resulting model can be used for generating synthetic data on which algorithms can be trained and validated, and on which competitions can be conducted, without compromising the privacy of the original dataset. Our method modifies the Private Aggregation of Teacher Ensembles (PATE) framework and applies it to GANs. Our modified framework (which we call PATE-GAN) allows us to tightly bound the influence of any individual sample on the model, resulting in tight differential privacy guarantees and thus an improved performance over models with the same guarantees. We also look at measuring the quality of synthetic data from a new angle; we assert that for the synthetic data to be useful for machine learning researchers, the relative performance of two algorithms (trained and tested) on the synthetic dataset should be the same as their relative performance (when trained and tested) on the original dataset. Our experiments, on various datasets, demonstrate that PATE-GAN consistently outperforms the stateof-the-art method with respect to this and other notions of synthetic data quality.

heading: 1 INTRODUCTION
text: More and more large datasets are becoming available in a wide variety of communities. In the U.S. medical community, for example, the fraction of providers using electronic health records (EHR) increased from 9.4% in 2008 to 83.8% in 2015 [20]. The availability of large datasets presents enormous opportunities for collaboration between the data-holders and the machine learning community. However, many of these large datasets, especially EHR, include sensitive information that prevents data-holders from sharing the data.
The most common way to mitigate the privacy risk of sharing sensitive records is to de-identify the records - but it is by now well-known that records that have been de-identified can be easily re-identified by linking them to other identifiable datasets [30; 13; 24; 22; 14]. (This is especially true for medical records of patients who have rare diseases.) However, if the purpose of sharing the data is to develop and validate machine learning methods for a particular task (e.g. prognostic risk scoring), real data is not necessary; it would suffice to have synthetic data that is sufficiently like the real data.
∗: Equal contribution
Precisely what this means depends on how the synthetic data will be used. For example, the synthetic data may be used to train models that will be deployed directly on real data. In this setting it is important that these methods (which we trained entirely on synthetic data) perform as well as if they had been trained on real data. Another setting to consider is one in which data-holders wish to use the synthetic data to identify the best method(s) to be used on the real data [11]. In this setting, it is not important that training on synthetic data leads to good performance on real data, but rather that comparing two methods on the synthetic data results in conclusions similar to those that would have been drawn from comparing the two methods on the real data. We evaluate our method in both settings.
Generative Adversarial Networks (GAN) [19] provide a powerful method for using real data to generate synthetic data but it does not provide any rigorous privacy guarantees. Our method modifies the GAN machinery in a way that does guarantee privacy; the synthetic data is (differentially) private [12] with respect to the original data. To do this we modify the training procedure of the discriminator to be differentially private by using a modified version of the Private Aggregation of Teacher Ensembles (PATE) [25; 26] framework. The Post-Processing Theorem [12] then guarantees that the GAN generator - which is trained only using the differentially private discriminator - will also be differentially private and thus so will the synthetic data it generates. We call our proposed framework PATE-GAN.
Using two Kaggle datasets, two different real-world medical datasets and two UCI datasets, we evaluate the utility of the samples generated by PATE-GAN in various settings with various levels of differential privacy. In line with the settings outlined above, we consider two methods for evaluating the similarity of synthetic datasets with a real dataset. The first method, first proposed in [15], compares the predictive performance of models trained on the synthetic datasets and tested on the real dataset. The second method, which we propose for the first time here, compares the performance rankings of predictive models on the synthetic datasets with their performance rankings on the real dataset. We demonstrate that, for both of these methods, PATE-GAN consistently produces synthetic datasets that are ”more like” the original real dataset than the synthetic datasets produced by the state-of-the-art benchmark (DPGAN [32]).
The contributions of this paper can be summarized as follows: (1) we modify the PATE framework and apply it to GANs to generate synthetic data, (2) we demonstrate in the experiments section that using PATE to enforce differential privacy results in higher quality synthetic data than DPGAN using various real-world datasets, (3) we propose a novel new metric for evaluating the generated synthetic data.

heading: 2 RELATED WORKS
text: The most related previous work to this paper is DPGAN [32]. Like us, DPGAN proposes a framework for modifying the GAN framework to be differentially private, also relying on the PostProcessing Theorem to change the problem of learning a differentially private generator to learning a differentially private discriminator. Their work uses a technique introduced by [1] that provides a differentially private mechanism for training deep networks. The key idea is that noise is added to the gradient of the discriminator during training to create differential privacy guarantees. These ideas are also used in [2]. Our method is similar in spirit; during training of the discriminator differentially private training data is used, which results in noisy gradients, however, we use the mechanism introduced in [25] which we believe gives tighter differential privacy guarantees (via tighter bounds on the effect of a single sample) than those provided in [1]. This means that for the same privacy guarantees, our method is capable of producing higher quality synthetic data. For a visual representation of both PATE-GAN and DPGAN, see the Appendix.
The proposed model modifies the PATE framework [25; 26] for use in a generative model setting (specifically for use with GANs). The key to the GAN framework is that the discriminator is a differentiable module trained to classify samples as either real or generated. The PATE framework provides a differentially private mechanism for classification by training multiple teacher models on disjoint partitions of the data. To classify a new sample each teacher’s output is evaluated on the sample and then all outputs are noisily aggregated. This noisy aggregation, though, results in a classifier that is not differentiable with respect to the parameters of the generator. In order to overcome this problem we follow the idea of the student model, also proposed in [25], that
involves taking some public unlabelled data, labelling it using the standard PATE mechanism and then training the student using the resulting labelled data. Because access to any public data is often an unreasonable assumption in synthetic data generation, we adapt this training paradigm in a way that does not require public data by training the student using only outputs from the (differentially private) generator.
Some previous works generate synthetic data using summary statistics of the original data [23] or based on specific domain-knowledge [6]; however, those methods are limited to low-dimensional feature spaces, specific fields and do not provide any differential privacy guarantees. [9] generates synthetic patient records using a GAN framework. However, [9] focuses only on generating discrete variables, whereas PATE-GAN is capable of generating mixed-type (continuous, discrete, and binary) variables. Furthermore, [9] also does not provide any differential privacy guarantees and instead uses ad-hoc notions of privacy which are only validated empirically.
Finally, it is worth remarking that it is known to be hard in the worst-case to generate private synthetic data [31] and so techniques such as GANs are necessary to address this challenge.

heading: 3 BACKGROUND
text: Let us denote the feature space by X , the label space by Y and write U = X ×Y . Let the dimension of U be d. Suppose that X and Y are random variables over X and Y . We write U = (X, Y ) and x, y,u to denote realizations of X, Y and U, respectively. The dataset D consists of N samples of u, assumed i.i.d. according to PU denoted as D = {ui}Ni=1 = {(xi, yi)}Ni=1.

heading: 3.1 DIFFERENTIAL PRIVACY
text: We first provide some preliminaries on differential privacy [12] before describing PATE-GAN; we refer interested readers to [12] for a thorough exposition of differential privacy. We will denote an algorithm byM, which takes as input a dataset D and outputs a value from some output space, O. Definition 1. (Neighboring Datasets) Two datasets D,D′ are said to be neighboring if
∃x ∈ D s.t. D \ {x} = D′. (1)
Definition 2. (Differential Privacy) A randomized algorithm, M, is ( , δ)-differentially private if for all S ⊂ O and for all neighboring datasets D,D′:
P(M(D) ∈ S) ≤ e P(M(D′) ∈ S) + δ (2)
where P is taken with respect to the randomness ofM. Differential privacy provides an intuitively understandable notion of privacy - a particular sample’s inclusion or exclusion in the dataset does not change the probability of a particular outcome very much: it does so by a multiplicative factor of e and an additive amount, δ.
The following theorem, a proof of which can be found in [12], allows us to move the burden of differential privacy to the discriminator; the differential privacy of the generator will follow by the theorem.
Theorem. (Post-processing) LetM be an ( , δ)-differentially private algorithm and let f : O → O′ where O′ is any arbitrary space. Then f ◦M is ( , δ)-differentially private.

heading: 3.2 PRIVATE AGGREGATION OF TEACHER ENSEMBLES (PATE)
text: In this section we describe the PATE mechanism first defined in [25] and later improved upon by [26]. The PATE mechanism provides a differentially private method for classification, a core component of the GAN framework; the discriminator is a classifier trained to identify whether samples are real/fake.
In order to build a differentially private classifier, the dataset is first divided into k disjoint subsets D1, ...,Dk. k classifiers, T1, ..., Tk (referred to as teachers) are then trained separately on the k sub-datasets (i.e. Ti is only trained on Di). Given a new input feature vector x to classify, the
differentially private output is given by passing x to each of the k teachers, and then performing a noisy aggregation of the resulting outputs.
Formally, given the k teachers, m possible classes and an input feature vector, x, set nj(x) = |{Ti : Ti(x) = j}| for j = 1, ...,m (3)
so that nj(x) is the number of teachers that output class j for x. The output of the PATEλ mechanism for input x is then defined as
PATEλ(x) = arg max j∈[m] (nj(x) + Yj) (4)
where Y1, ..., Ym are i.i.d. Lap(λ) random variables. The following result, found in [25], follows from [12]. Theorem. The output of a single query to the PATEλ mechanism is ( 1λ , 0)-differentially private.
In order to apply this framework in the GAN framework, however, we require that the discriminator be differentiable, which the output of this classification mechanism is not (note that accessing the internal parameters of the teachers would violate differential privacy, the only thing we have access to in this case is the output). Instead, we draw on the PATE extension (also introduced in [25]) in which a student model is trained. This student model (after being trained) is free to access, not only its outputs given inputs but also its internal parameters. The model itself is differentially private.
Formally, the student, S, is a classifier that is trained by taking some public, unlabelled data, P = {xi}Ki=1, passing each sample, xi, through the (standard) PATE mechanism, to receive a differentially private label, ŷi, and forming a new (noisy-)teacher-labelled dataset P̂ = {(xi, ŷi)}Ki=1 on which the student is then trained.
Importantly, we can make the student differentiable - it can be modelled using any classifier, such as a neural net. Moreover, querying the student is “free” - there is no privacy cost associated with passing an input to the student and receiving an output, the only privacy cost is in acquiring the data on which to train the student. We state the following result which follows from the analysis in [25].
Theorem. The student, S, trained on the dataset P̂ where labels were generated according to the PATEλ mechanism using λ = K2 , is ( , 0)-differentially private with respect to the original data D.

heading: 4 PROPOSED METHOD: PATE-GAN
text: The proposed method builds on GAN and PATE frameworks. We replace the GAN discriminator with a PATE mechanism so that our discriminator is differentially private, but require the (differentiable) student version to allow back-propagation to the generator. We modify the implementation of the student, noting that the training paradigm presented in [25] is not appropriate for this setting due to the lack of publicly available data. Before training, we partition the dataset into k subsets, D1, ...,Dk, with |Di| = |D|k for ∀i.

heading: 4.1 GENERATOR
text: The generator, G, is as in the standard GAN framework. Formally it is a function G(·; θG) : [0, 1]d → U , parametrized by θG that takes random noise, z ∼ Unif([0, 1]d), as input and outputs a vector in U = X × Y . The generator will be trained to minimize its loss with respect to the student-discriminator. Given n i.i.d. samples of Unif([0, 1]d), z1, ..., zn, the empirical loss of G at θ for fixed S is defined by
LG(θG;S) = n∑ j=1 log(1− S(G(zj ; θG))). (5)
We will denote by PG the distribution induced by G over U .

heading: 4.2 DISCRIMINATOR
text: In the standard GAN framework, there is a single discriminator, D, that is trained in a directly adversarial fashion with G, where at each iteration either G is trying to improve its loss with respect
to D or D is trying to improve its loss with respect to G. In our proposed model, however, we replaceD with the PATE mechanism. This means we introduce k teacher-discriminators, T 1, ..., T k, and a student discriminator, S. A noticeable difference is that the adversarial training is no longer symmetrical: the teachers are now being trained to improve their loss with respect to G but G is being trained to improve its loss with respect to the student S which in turn is being trained to improve its loss with respect to the teachers.

heading: 4.2.1 TEACHER-DISCRIMINATORS
text: Formally, the teacher-discriminators (which we will refer to simply as teachers) are functions T1(·; θ1T ), ..., Tk(·; θkT ) : U → [0, 1] each parametrized by θiT . The teachers are given either a real sample from their corresponding partition of the dataset (i.e. Ti may receive a sample from Di) as input or a sample from the generator. The teachers are then trained to classify them.
Given n i.i.d. samples of Unif([0, 1]d), z1, ..., zn, we define the empirical loss of teacher i with weights θiT for fixed G by
LiT (θiT ) = − [ ∑ u∈Di log Ti(u; θ i T ) + n∑ j=1 log(1− Ti(G(zj); θiT )) ] . (6)
Each teacher is trained in the same way the discriminator is trained in a standard GAN framework, except that here the teacher only ever sees its partition of the real data.

heading: 4.2.2 STUDENT-DISCRIMINATORS
text: The main innovation of our paper comes from our implementation of the student-discriminator (which we will refer to simply as the student) in this setting. The differential privacy guarantee provided by the standard student model is only with respect to the original data, D, and not the public data, P , used to train the student. In our setting, where the entire focus is on generating synthetic data because no data is publicly available, we must propose a novel methodology to train the student without public data.
We first note, that the student training paradigm described in [25] would involve training the student using data similar to that used to train the generator - i.e. by taking an equal number of samples from each and then labelling those using the standard PATEλ mechanism (where here “labelling” refers to assigning them a real/fake label - not the label y present in the data). We consider the implications of training the student on teacher-labelled generated samples only.
We first observe that during training of the generator, the discriminator is only evaluated on samples from the generator itself, and not the real data, so by training the student only on generated samples we are in fact training it on the distribution we need it to perform well on. However, we note that if the student only sees unrealistic samples from the generator (i.e. generated samples that most teachers label as fake), then the student will not contain any information that the generator can use to improve its generated samples. It is therefore important that some of the generated samples the student is trained on are realistic. We then note that if Supp(PU ) ⊂ Supp(PG) then some of the generated samples will be realistic.
In order to ensure Supp(PU ) ⊂ Supp(PG), we normalize the data into [0, 1]d and then initialize the parameters of the generator randomly using Xavier initialization. It follows that Supp(P) ⊂ [0, 1]d ⊂ G([0, 1]d) = G(Supp(Z)) = Supp(G(Z)) when Z ∼ Unif([0, 1]d). We create our training data for the student by taking n i.i.d. samples of Unif([0, 1]d), z1, ..., zn, generating n samples using the generator, û1, ..., ûn with ûj = G(zj), and using the teachers to label these using PATEλ, setting rj = PATEλ(ûj). We train the student, S(·; θS) : U → [0, 1], to maximize the standard cross-entropy loss on this teacher-labelled data, i.e.
LS(θS) = n∑ j=1 rj logS(ûj ; θS) + (1− rj) log(1− S(ûj ; θS)). (7)
Although a priori the above mechanism does not appear to depend on the number of teachers, it should be noted that for fixed λ, more teachers results in the teacher-labelled dataset being less
noisy - the noise being added is smaller relative to the counts nj . This introduces a trade-off - for a small number of teachers, the noise may be too large and thus render the output meaningless; with a larger number of teachers, less data can be used to train each teacher, which may also render the output meaningless, even though the noise has a smaller effect. Finding the right balance in this problem is key. In our experiments, we use d real and d generated samples to train each teacher where d is the dimension of the input space. Although the utility of a single teacher may be low, by aggregating (even noisily) the resulting classifier actually has high utility. Moreover, by using a minimal number of samples for each teacher, the effect of any individual sample on the output is small (because there are more teachers and each sample can effect at most 1 teacher) which means that our differential privacy guarantees are tighter - if we used fewer teachers, the mechanism still assumes that, in the worst case, the presence (or absence) of a single sample can completely flip a teacher’s vote and so we still need to add the same noise.
We train G, T 1, ..., T k and S iteratively1, with each iteration of G consisting of first performing nT updates on all teachers, then performing nS updates of the student. We perform generator iterations until our privacy constraint, , has been reached. A block diagram of PATE-GAN can be found in the Appendix.
To calculate the privacy of our algorithm we use the moments accountant method given in [25] to derive a data-dependent privacy guarantee at run-time. Details of its definition, and key results we use can be found in the Appendix. We denote the moments accountant of PATE-GAN by α(l). The moments accountant allows us to more tightly bound the total privacy cost of our mechanism than standard composition theorems would, and moreover attributes a lower privacy cost to accessing the noisy aggregation of the teachers when the teachers have a stronger consensus with the intuition being that when the teachers have a strong consensus, a single teacher (and therefore a single sample) has a much lower influence on the output than when the votes (n0 and n1) are close. Pseudo-code for PATE-GAN can be found in Algorithm 1.
We now state the main theorem of the paper, which follows from the theory in [25].
Theorem 1. Algorithm 1, which takes as input δ > 0, a dataset, D, and outputs G and is ( , δ)differentially private.
The proof relies on applying the post-processing theorem where the discriminator corresponds to the mechanism M which takes outputs in O (in our case this corresponds to the weights of the discriminator), and the generator corresponds to the function f which maps from O to O (which corresponds to the weights of the generator). For full details of the proof and further details of the theory required for it, see the Appendix.

heading: 5 EXPERIMENTS
text: In this section, we use a real-world Kaggle dataset (Credit card fraud detection dataset [11]) to evaluate PATE-GAN against the state-of-the-art benchmark (DPGAN [32]). In addition, we provide high-level (average) results for five additional datasets (with various characteristics): MAGGIC [27], UNOS-Heart wait-list [7], Kaggle cervical cancer dataset [16], UCI ISOLET dataset and UCI Epileptic Seizure Recognition dataset. A more detailed breakdown of the results for these datasets is given in the Appendix. Details of all six datasets can be also found in the Appendix.

heading: 5.1 EXPERIMENTAL SETTINGS
text: To empirically validate the quality of the generated dataset we introduce three different trainingtesting settings. Setting A: train the predictive models on the real training set, test the performance of the models on the real testing set. Setting B: train on the synthetic training set, test on the real testing set ([15]), Setting C: train on the synthetic training set, test on the synthetic testing set. Note that the training set and the testing set are disjoint in both the real and synthetic datasets.
We are interested in two comparisons. If we see a high predictive performance on the real data for models that were trained on synthetic data (Setting B), we can infer that the synthetic data has
1The teachers can be trained in parallel.
Algorithm 1 Pseudo-code of PATE-GAN 1: Input: δ, D, nT , nS , batch size n, number of teachers k, noise size λ 2: Initialize: θG, θ1T , ..., θkT , θS , α(l) = 0 for l = 1, ..., L 3: Partition dataset into k subsets D1, ...,Dk of size |D|k 4: while ̂ < do 5: for t2 = 1, ..., nT do 6: Sample z1, ..., zn
i.i.d.∼ PZ 7: for i = 1, ..., k do 8: Sample u1, ...,un
i.i.d.∼ Di 9: Update teacher, Ti, using SGD
10: ∇θiT − [∑d j=1 log(Ti(uj)) + log(1− Ti(G(zj))) ] 11: for t3 = 1, ..., nS do 12: Sample z1, ..., zn
i.i.d.∼ PZ 13: for j = 1, ..., n do 14: ûj ← G(zj) 15: rj ← PATEλ(ûi) for j = 1, ..., n 16: Update moments accountant 17: q ← 2+λ|n0−n1|4 exp(λ|n0−n1|) 18: for l = 1, ..., L do
19: α(l)← α(l) + min{2λ2l(l + 1), log((1− q) (
1−q 1−e2λq
)l + qe2λl)}
20: Update the student, S, using SGD 21: ∇θS − ∑n j=1 rj logS(ûj) + (1− rj) log(1− S(ûj)) 22: Sample z1, ..., zn i.i.d.∼ PZ 23: Update the generator, G, using SGD 24: ∇θG [∑n i=1 log(1− S(G(zi))
] 25: ̂← min
l
α(l)+log( 1δ )
l
26: Output: G
captured the relationship between features and labels well. Moreover, synthetic data that does well in this setting can be used to train models without ever seeing the real data.
On the other hand, when we consider synthetic data for use in competitions such as Kaggle, we need synthetic data that allows researchers to do meaningful comparisons on the synthetic data. In this setting, the researchers will only be able to use the synthetic data as both the training and testing set, and will need to develop their algorithms using results on the synthetic data. Now it becomes important that the relative performance of two algorithms when trained and tested on the synthetic data (Setting C), is similar to their relative performance when trained and tested on the real data (Setting A). A simple requirement would be that if model 1 is better than model 2 on the real data, then model 1 is better than model 2 on the synthetic data. This allows researchers to use the synthetic data to choose the best method(s) to try on the real data (or rather to give to the data-holder to try on the real data).
For both comparisons, we use 12 different predictive models, shown in Table 1. We use two performance metrics to measure the capability of each model in predicting the label: (1) area under the receiver operating characteristics curve (AUROC), (2) area under the precision recall curve (AUPRC). Throughout the experiments we fix δ = 10−5 for use as input to PATE-GAN and DPGAN. We also report the performance of the original GAN framework (”GAN”), which serves to indicate an upper bound on performance and allows us to see how much performance is lost due to the two differential privacy mechanisms (PATE-GAN and DPGAN). The details of hyper-parameter optimization and benchmark implementations can be found in the Appendix.

heading: 5.2 RESULTS WITH SETTING B
text: In this subsection, we evaluate PATE-GAN and DPGAN in Setting B (trained on synthetic, tested on real) to understand whether or not the models are capturing the feature-label relationships well. Intuitively, if a synthetic dataset is such that a model trained on it performs well when performance is measured on real data, then the relationship between feature and label in the synthetic data is similar to that in the real data. In Table 1 we give the results for the Kaggle Credit dataset for all 12 predictive models. In Table 2, we give the performance on each dataset averaged across the 12 methods for each of the 6 datasets. A breakdown of the performance of each predictive model for each dataset can be found in the Appendix. Across all datasets, we see that PATE-GAN is capable of generating synthetic samples that better preserve the feature-label relationship (according to AUROC and AUPRC) than DPGAN.
We note that the performance of all models, including the original GAN model (i.e. PATE-GAN - or equivalently DPGAN - with (∞,∞)-differential privacy) in the high dimensional UCI ISOLET and UCI Epileptic Seizure Recognition datasets is lower than in lower dimensional datasets (when
compared to the baseline AUROC and AUPRC found in the Appendix). We do, however, see that both PATE-GAN and DPGAN show more significant decreases in performance than the original GAN in these high-dimensional settings. In the case of PATE-GAN, we believe this may be due to the fact that the student discriminator is trained only using data from the generator, and therefore requires that some of the generated data look somewhat realistic from the start, which is a harder requirement to satisfy as the data has more dimensions. On the other hand, in DPGAN, noise must be added to each component of the gradient (of the discriminator) and so in higher dimensions the norm of the noise added is larger. Note that in PATE-GAN, noise is added only to the teacher outputs, whose dimension (typically 1) does not depend on the dimension of the input data, and so this phenomena does not present itself in PATE-GAN. The results on both the UCI datasets would suggest that the loss from increasing noise norm (for DPGAN) is greater than from difficulty in randomly generating realistic samples (for PATE-GAN).

heading: 5.3 VARYING THE PRIVACY CONSTRAINT ( )
text: In Fig. 1, we investigate the trade off between privacy constraint and utility. In the table we report the average performance of AUROC over the 12 different predictive models for PATE-GAN and the benchmark for various (with δ = 10−5). As can be seen in Fig. 1, PATE-GAN is consistently better than DPGAN over the entire range of tested . We believe this is because the PATE mechanism allows us to more tightly bound the influence of a single sample on the discriminator, and hence we can provide tighter differential privacy guarantees - when the differential privacy guarantee is fixed, this results in higher quality synthetic data. Of course, as we increase (i.e. decrease the required privacy) both methods converge to the performance of GAN and the increase in performance of PATE-GAN over DPGAN becomes smaller.

heading: 5.4 SETTING A VS SETTING C: PRESERVING THE RANKING OF PREDICTIVE MODELS
text: As discussed at the beginning of this section, it is important that a synthetic dataset respects the ranking of models (in terms of their prediction performances) [21]. To evaluate this, we now introduce a new metric, which we refer to as the Synthetic Ranking Agreement (SRA). Suppose that we have L predictive models, f1, f2, ..., fL2. Furthermore, suppose that the performance of model i when trained and tested on the real data (Setting A) is Ai ∈ R and that the performance of model i when trained and tested on the synthetic data (Setting C) is Ci ∈ R. Then we define the Synthetic Ranking Agreement by
SRA({Ai}Li=1, {Ci}Li=1) = 1
L(L− 1) L∑ j=1 ∑ k 6=j I ( (Aj −Ak)× (Cj − Ck) > 0 )
(8)
where I is an indicator function. Note that the summand is 1 when the ordering of algorithms j and k are the same in both settings, and is 0 when the ordering in one setting differs from the ordering in the other.
2For the results in Table 3, we use the same 12 predictive models as used in Table 1
We compare the SRA of PATE-GAN and the benchmark for various (with δ = 10−5)3. As can be seen in Table 3, PATE-GAN achieves the best SRA across all values of .
In the Appendix, we perform a similar experiment in which we compare the ranking of features by their importance (determined by their absolute Pearson correlation coefficient with the label) on the original dataset and on the synthetic dataset (generated by PATE-GAN and the benchmark) and report the results using a metric that is identical to SRA, with the model performances ({Ai}, {Ci}) substituted for feature importances.

heading: 5.5 QUANTITATIVE ANALYSIS ON THE NUMBER OF TEACHERS
text: The number of teachers is a hyper-parameter of PATE-GAN and we choose the number of teachers among {N/10, N/50, N/100, N/500, N/1000, N/5000, N/10000} where N is the total number of samples. As we described in the previous section, there is a trade-off between number of teachers and the corresponding quality of the synthetic data. Table 4 quantitatively shows the trade-off between the number of teachers and the performance (in terms of both AUROC and AUPRC).

heading: 6 DISCUSSION
text: In this paper we introduced a novel methodology for generating differentially private synthetic data. Through several experiments we demonstrated the ability of our method to produce high quality synthetic data while being able to give strict differential privacy guarantees.
In order to apply PATE to the GAN setting, we needed to use the original GAN framework. Extending PATE to the regression setting so that, for example, a Wasserstein GAN can be used instead, is an open and interesting question, and a potential direction for future research.

heading: ACKNOWLEDGEMENT
text: The authors would like to thank the reviewers for their helpful comments. The research presented in this paper was supported by the Office of Naval Research (ONR) and the NSF (Grant number: ECCS1462245, ECCS1533983, and ECCS1407712).
3The ordering of models according to Table 1 is in fact quite consistent - the average agreed ranking probability (now applied to different folds of the data, rather than real vs. synthetic data) is 0.9273 (for AUROC). The rankings used are therefore sufficiently stable for this to be a meaningful metric.

heading: APPENDIX

heading: THEORY REQUIRED FOR THEOREM 1
text: Theorem 2. Algorithm 1, which takes as input δ > 0, a dataset, D, and outputs G and is ( , δ)differentially private.
In order to prove our theorem, we define the moments accountant [1] and state the theorems that the data-dependent privacy guarantees of the PATE mechanism rely on. For proofs of the results below, see [25] and the references therein. Definition 3. (Privacy Loss) Let M be a randomized algorithm taking outputs in a space O and D,D′ be neighbouring datasets. Let aux denote an auxiliary input. For an outcome o ∈ O, the privacy loss at o is defined as:
c(o;M, aux,D,D′) = log P(M(aux,D) = o) P(M(aux,D′) = o) . (9)
The privacy loss random variable C(M, aux,D,D′) is defined as c(M(D);M, aux,D,D′), i.e. the random variable defined by evaluating the privacy loss at an outcome sampled fromM(D). Definition 4. (Moments accountant) LetM be a randomized algorithm. The moments accountant is defined as:
αM(l) = max aux,D,D′
αM(l; aux,D,D′) (10)
where αM(l; aux,D,D′) = logE(exp(lC(M, aux,D,D′))) is the moment generating function of the privacy loss random variable and the max is taken over neighbouring datasets D,D′. Theorem 3. (Composability) Suppose that an algorithm M consists of a sequence of adaptive algorithms M1, ...,Mk where Mi outputs in Oi and takes inputs from Πi−1j=1Oj as well as the dataset D. Then for any output sequence o1, ..., ok−1 and any l
αM(l;D,D′) = k∑ i=1 αMi(l; o1, ..., oi−1,D,D′) (11)
where αM is conditioned on eachMi’s output being oi. Theorem 4. (Tail bound) Let M be a randomized algorithm. For any > 0, M is ( , δ)differentially private for
δ = min l
exp(αM(l)− l ). (12)
The following theorem combines Theorems 2, 3 and Lemma 4 from [25]. Theorem 5. (Data-dependent privacy guarantee for PATE) LetM be the PATE mechanism defined in Section 3 of the paper. Let n0, n1 be as defined in Equation 3 of the paper. Let q =
2+λ|n0−n1| 4 exp(λ|n0−n1|) .

heading: Then
text: αM(l) ≤ min{2λ2l(l + 1), log((1− q) (
1− q 1− e2λq
)l + qe2λl)}. (13)
Proof of Theorem 2. We use Theorem 5 to bound the moments accountant for each query to the PATE mechanism during the training of our algorithm (i.e. each time a generated sample is labeled by the teachers). Theorem 3 then allows us to sum the individual bounds for each query to bound the moments accountant of the entire algorithm. Theorem 4 then allows us to derive a value for given δ.

heading: DATA DESCRIPTIONS

heading: KAGGLE CREDIT CARD FRAUD DETECTION DATA DESCRIPTION
text: The Kaggle credit card fraud detection dataset [11] contains transactions made by credit cards in September 2013 by European cardholders and the label is whether or not the transaction is fraudulent. The total number of features is 29 (binary: 0, continuous: 29) and the number of samples in this dataset is 284,807. Among the 284,807 samples, 492 (0.2%) samples are fraudulent transactions.

heading: MAGGIC DATA DESCRIPTION
text: The Meta-analysis Global Group in Chronic Heart Failure (MAGGIC) dataset [27] is a collection of 30 different datasets from 30 different medical studies containing patients who experienced heart failure. We set the label of each patient as 1-year all-cause mortality, excluding all patients who are censored before 1-year. The total number of features is 29 (binary: 20, continuous: 9) and the number of patients in this dataset is 30,389. Among the 30,389 patients, 5,723 (18.8%) patients died within 1 year.

heading: UNOS DATA DESCRIPTION
text: The United Network for Organ Transplantation (UNOS) dataset [7] provides information about all patients in the U.S. who have received a transplantation or were on the wait-list during the period 1985-2015. In this paper, we focus on the patients who were on the heart transplant wait-list. The objective is to predict 1-year all-cause mortality. The total number of features is 20 (binary: 18, continuous: 2) and the number of patients in this dataset is 23,706. Among the 23,706 patients, 12,606 (53.2%) patients died within 1 year.

heading: KAGGLE CERVICAL CANCER DATA DESCRIPTION
text: The Kaggle cervical cancer dataset [16] was collected at ’Hospital Universitario de Caracas’ in Caracas, Venezuela. It contains demographic information, habits, and historic medical records. The total number of features is 35 (binary: 24, continuous: 11) and the number of patients in this dataset is 858. Among the 858 patients, 55 (6.4%) patients have positive biopsy.
UCI ISOLET DATA DESCRIPTION
The UCI ISOLET dataset https://archive.ics.uci.edu/ml/datasets/isolet was generated by speaking the name of each letter of the alphabet. The task is to classify each spoken letter as either a vowel or a consonant (binary classification). The total number of features is 617 and the number of samples in this dataset is 7797. Among the 7797 samples, 1500 (19.2%) samples are vowels.

heading: UCI EPILEPTIC SEIZURE RECOGNITION DATA DESCRIPTION
text: The UCI Epileptic Seizure Recognition dataset https://archive.ics.uci.edu/ml/ datasets/Epileptic+Seizure+Recognition was generated by recording brain activity. The task is to classify activity as seizure activity (binary classification). The total number of features is 179 and the number of samples in this dataset is 11500. Among the 11500 samples, 2300 (20.0%) samples correspond to seizure activity.

heading: DATA SUMMARY AND SETTING A PERFORMANCE
text: Table 5 summarises the 6 datasets we use and provides a baseline performance for a predictive model on each dataset - recall that Setting A refers to training and testing on the real data. The AUROC and AUPRC in this setting are upper bounds on the AUROC and AUPRC we could hope to achieve in Setting B.

heading: HYPER-PARAMETER OPTIMIZATION
text: In all experiments, the depth of the generator and discriminator (student-discriminator in our case) in both PATE-GAN and the DPGAN benchmark [32] is set to 3. The depth of the teacher discriminators is set to 1. The number of hidden nodes in each layer is d, d/2 and d (where d is the feature dimension), respectively. We use relu as the activation functions of each layer except for the output layer where we use the sigmoid activation function and the batch size is 64 for both the generator and discriminator. We set nT = nS = 5. Using cross validation, we select the number of teachers, k, among N/10 N/50 N/100 N/500 N/1000 N/5000 N/10000. The learning rate is 10−4 and we use Adam Optimizer to minimize the loss function.
We use tensorflow to implement PATE-GAN and DPGAN. For DPGAN we use the code from the following link: https://github.com/illidanlab. We use the sklearn package in python to implement the 12 predictive models: Logistic Regression (LogisticRegression), Random Forests (RandomForestClassifier), Gaussian Naive Bayes (GaussianNB), Bernoulli Naive Bayes (BernoulliNB), Linear Support Vector Machine (svm), Decision Tree (DecisionTree), Linear Discriminant Analysis Classifier (LinearDiscriminantAnalysis), Adaptive Boosting (AdaBoost) (AdaBoostClassifier), Bootstrap Aggregating (Bagging) (BaggingClassifier), Gradient Boosting Machine (GBM) (GradientBoostingClassifier), Multi-layer Perceptron (MLPClassifier), and XgBoost (XGBoostRegressor).

heading: ADDITIONAL RESULTS

heading: VARYING THE PRIVACY CONSTRAINT ( ) IN TERMS OF AUPRC
text: Similar to Fig. 1 in the main manuscript, Fig. 2 shows the trade off between the privacy constraint and utility, where utility is now measured in terms of AUPRC (rather than AUROC). We report the average performance in terms of AUPRC over the 12 different predictive models for PATE-GAN and the benchmark for various (with δ = 10−5). As can be seen in Fig. 2, PATE-GAN consistently outperforms DPGAN over the entire range of tested in terms of AUPRC as well.

heading: PRESERVING THE RANKING OF VARIABLE IMPORTANCE IN KAGGLE CREDIT DATASET
text: We compare the ranking of variables by their importance (according to absolute Pearson correlation coefficient with the label) on the original dataset and on both synthetic datasets. We report the results using agreed ranking probability. As can be seen in Table 6, PATE-GAN achieves consistently better agreed ranking probability across all values of tested (with δ = 10−5).
HIGH-DIMENSIONAL RESULTS: UCI ISOLET AND UCI EPILEPTIC SEIZURE RECOGNITION

heading: MAGGIC DATASET RESULT

heading: UNOS HEART WAIT DATASET RESULT

heading: KAGGLE CERVICAL CANCER DATASET RESULT

heading: BLOCK DIAGRAMS

heading: PATE-GAN
text: The two figures below indicate the iterative training procedure carried out by PATE-GAN; the figures correspond to a single generator update.
DPGAN [32]

Review summaries:
This paper considers using a GAN to generate synthetic data in a differentially private manner [see also https://www.biorxiv.org/content/early/2018/06/05/159756 ]. The key novelty is the integration of the PATE differential privacy framework from recent work. Specifically, rather than a single distinguisher as is usual in a GAN, there is a "student distinguisher" and several "teacher distinguishers". The student distinguisher is used as usual except that it does not have access to the real data, only the teacher distinguishers have access to the real data (as well as the synthetic data). The data is partitioned amongst the teacher distinguishers and their output is aggregated in a differentially private manner (and gradients are not revealed). The role of the teacher distinguishers is solely to correct the student distinguisher when it errs.

The paper studies the problem of generating synthetic datasets (while ensuring differential privacy) via training a GAN. One natural approach is the teacher-student framework considered in the PATE framework.  In the original PATE framework, while the teachers are ensured to preserve differential privacy, the student model (typically a GAN) requires the presence of publicly data samples. The main contribution of this paper is to get around the requirement of public data via using uniformly random samples in [0,1]^d.