Title: PATE-GAN: GENERATING SYNTHETIC DATA WITH DIFFERENTIAL PRIVACY GUARANTEES

Abstract: Machine learning has the potential to assist many communities in using the large datasets that are becoming more and more available. Unfortunately, much of that potential is not being realized because it would require sharing data in a way that compromises privacy. In this paper, we investigate a method for ensuring (differential) privacy of the generator of the Generative Adversarial Nets (GAN) framework. The resulting model can be used for generating synthetic data on which algorithms can be trained and validated, and on which competitions can be conducted, without compromising the privacy of the original dataset. Our method modifies the Private Aggregation of Teacher Ensembles (PATE) framework and applies it to GANs. Our modified framework (which we call PATE-GAN) allows us to tightly bound the influence of any individual sample on the model, resulting in tight differential privacy guarantees and thus an improved performance over models with the same guarantees. We also look at measuring the quality of synthetic data from a new angle; we assert that for the synthetic data to be useful for machine learning researchers, the relative performance of two algorithms (trained and tested) on the synthetic dataset should be the same as their relative performance (when trained and tested) on the original dataset. Our experiments, on various datasets, demonstrate that PATE-GAN consistently outperforms the stateof-the-art method with respect to this and other notions of synthetic data quality.

1. INTRODUCTION:

More and more large datasets are becoming available in a wide variety of communities. In the U.S. medical community, for example, the fraction of providers using electronic health records (EHR) increased from 9.4% in 2008 to 83.8% in 2015 [20] . The availability of large datasets presents enormous opportunities for collaboration between the data-holders and the machine learning community. However, many of these large datasets, especially EHR, include sensitive information that prevents data-holders from sharing the data.

The most common way to mitigate the privacy risk of sharing sensitive records is to de-identify the records -but it is by now well-known that records that have been de-identified can be easily re-identified by linking them to other identifiable datasets [30; 13; 24; 22; 14] . (This is especially true for medical records of patients who have rare diseases.) However, if the purpose of sharing the data is to develop and validate machine learning methods for a particular task (e.g. prognostic risk scoring), real data is not necessary; it would suffice to have synthetic data that is sufficiently like the real data.

Precisely what this means depends on how the synthetic data will be used. For example, the synthetic data may be used to train models that will be deployed directly on real data. In this setting it is important that these methods (which we trained entirely on synthetic data) perform as well as if they had been trained on real data. Another setting to consider is one in which data-holders wish to use the synthetic data to identify the best method(s) to be used on the real data [11] . In this setting, it is not important that training on synthetic data leads to good performance on real data, but rather that comparing two methods on the synthetic data results in conclusions similar to those that would have been drawn from comparing the two methods on the real data. We evaluate our method in both settings.

Generative Adversarial Networks (GAN) [19] provide a powerful method for using real data to generate synthetic data but it does not provide any rigorous privacy guarantees. Our method modifies the GAN machinery in a way that does guarantee privacy; the synthetic data is (differentially) private [12] with respect to the original data. To do this we modify the training procedure of the discriminator to be differentially private by using a modified version of the Private Aggregation of Teacher Ensembles (PATE) [25; 26] framework. The Post-Processing Theorem [12] then guarantees that the GAN generator -which is trained only using the differentially private discriminator -will also be differentially private and thus so will the synthetic data it generates. We call our proposed framework PATE-GAN.

Using two Kaggle datasets, two different real-world medical datasets and two UCI datasets, we evaluate the utility of the samples generated by PATE-GAN in various settings with various levels of differential privacy. In line with the settings outlined above, we consider two methods for evaluating the similarity of synthetic datasets with a real dataset. The first method, first proposed in [15] , compares the predictive performance of models trained on the synthetic datasets and tested on the real dataset. The second method, which we propose for the first time here, compares the performance rankings of predictive models on the synthetic datasets with their performance rankings on the real dataset. We demonstrate that, for both of these methods, PATE-GAN consistently produces synthetic datasets that are "more like" the original real dataset than the synthetic datasets produced by the state-of-the-art benchmark (DPGAN [32] ).

The contributions of this paper can be summarized as follows: (1) we modify the PATE framework and apply it to GANs to generate synthetic data, (2) we demonstrate in the experiments section that using PATE to enforce differential privacy results in higher quality synthetic data than DPGAN using various real-world datasets, (3) we propose a novel new metric for evaluating the generated synthetic data.

2. RELATED WORKS:

The most related previous work to this paper is DPGAN [32] . Like us, DPGAN proposes a framework for modifying the GAN framework to be differentially private, also relying on the Post-Processing Theorem to change the problem of learning a differentially private generator to learning a differentially private discriminator. Their work uses a technique introduced by [1] that provides a differentially private mechanism for training deep networks. The key idea is that noise is added to the gradient of the discriminator during training to create differential privacy guarantees. These ideas are also used in [2] . Our method is similar in spirit; during training of the discriminator differentially private training data is used, which results in noisy gradients, however, we use the mechanism introduced in [25] which we believe gives tighter differential privacy guarantees (via tighter bounds on the effect of a single sample) than those provided in [1] . This means that for the same privacy guarantees, our method is capable of producing higher quality synthetic data. For a visual representation of both PATE-GAN and DPGAN, see the Appendix.

The proposed model modifies the PATE framework [25; 26] for use in a generative model setting (specifically for use with GANs). The key to the GAN framework is that the discriminator is a differentiable module trained to classify samples as either real or generated. The PATE framework provides a differentially private mechanism for classification by training multiple teacher models on disjoint partitions of the data. To classify a new sample each teacher's output is evaluated on the sample and then all outputs are noisily aggregated. This noisy aggregation, though, results in a classifier that is not differentiable with respect to the parameters of the generator. In order to overcome this problem we follow the idea of the student model, also proposed in [25] , that involves taking some public unlabelled data, labelling it using the standard PATE mechanism and then training the student using the resulting labelled data. Because access to any public data is often an unreasonable assumption in synthetic data generation, we adapt this training paradigm in a way that does not require public data by training the student using only outputs from the (differentially private) generator. Some previous works generate synthetic data using summary statistics of the original data [23] or based on specific domain-knowledge [6] ; however, those methods are limited to low-dimensional feature spaces, specific fields and do not provide any differential privacy guarantees. [9] generates synthetic patient records using a GAN framework. However, [9] focuses only on generating discrete variables, whereas PATE-GAN is capable of generating mixed-type (continuous, discrete, and binary) variables. Furthermore, [9] also does not provide any differential privacy guarantees and instead uses ad-hoc notions of privacy which are only validated empirically.

Finally, it is worth remarking that it is known to be hard in the worst-case to generate private synthetic data [31] and so techniques such as GANs are necessary to address this challenge.

3. BACKGROUND:

Let us denote the feature space by X , the label space by Y and write U = X × Y. Let the dimension of U be d. Suppose that X and Y are random variables over X and Y. We write U = (X, Y ) and x, y, u to denote realizations of X, Y and U, respectively. The dataset D consists of N samples of u, assumed i.i.d. according to P U denoted as

D = {u i } N i=1 = {(x i , y i )} N i=1 .

3.1. DIFFERENTIAL PRIVACY:

We first provide some preliminaries on differential privacy [12] before describing PATE-GAN; we refer interested readers to [12] for a thorough exposition of differential privacy. We will denote an algorithm by M, which takes as input a dataset D and outputs a value from some output space, O. 

where P is taken with respect to the randomness of M.

Differential privacy provides an intuitively understandable notion of privacy -a particular sample's inclusion or exclusion in the dataset does not change the probability of a particular outcome very much: it does so by a multiplicative factor of e and an additive amount, δ.

The following theorem, a proof of which can be found in [12] , allows us to move the burden of differential privacy to the discriminator; the differential privacy of the generator will follow by the theorem.

Theorem. (Post-processing) Let M be an ( , δ)-differentially private algorithm and let f : O → O where O is any arbitrary space. Then f • M is ( , δ)-differentially private.

3.2. PRIVATE AGGREGATION OF TEACHER ENSEMBLES (PATE):

In this section we describe the PATE mechanism first defined in [25] and later improved upon by [26] . The PATE mechanism provides a differentially private method for classification, a core component of the GAN framework; the discriminator is a classifier trained to identify whether samples are real/fake.

In order to build a differentially private classifier, the dataset is first divided into k disjoint subsets D 1 , ..., D k . k classifiers, T 1 , ..., T k (referred to as teachers) are then trained separately on the k sub-datasets (i.e. T i is only trained on D i ). Given a new input feature vector x to classify, the differentially private output is given by passing x to each of the k teachers, and then performing a noisy aggregation of the resulting outputs.

Formally, given the k teachers, m possible classes and an input feature vector, x, set

EQUATION

so that n j (x) is the number of teachers that output class j for x. The output of the PATE λ mechanism for input x is then defined as

EQUATION

where Y 1 , ..., Y m are i.i.d. Lap(λ) random variables. The following result, found in [25] , follows from [12] .

Theorem. The output of a single query to the PATE λ mechanism is ( 1 λ , 0)-differentially private. In order to apply this framework in the GAN framework, however, we require that the discriminator be differentiable, which the output of this classification mechanism is not (note that accessing the internal parameters of the teachers would violate differential privacy, the only thing we have access to in this case is the output). Instead, we draw on the PATE extension (also introduced in [25] ) in which a student model is trained. This student model (after being trained) is free to access, not only its outputs given inputs but also its internal parameters. The model itself is differentially private.

Formally, the student, S, is a classifier that is trained by taking some public, unlabelled data, P = {x i } K i=1 , passing each sample, x i , through the (standard) PATE mechanism, to receive a differentially private label, ŷi , and forming a new (noisy-)teacher-labelled dataset P = {(x i , ŷi )} K i=1 on which the student is then trained.

Importantly, we can make the student differentiable -it can be modelled using any classifier, such as a neural net. Moreover, querying the student is "free" -there is no privacy cost associated with passing an input to the student and receiving an output, the only privacy cost is in acquiring the data on which to train the student. We state the following result which follows from the analysis in [25] .

Theorem. The student, S, trained on the dataset P where labels were generated according to the PATE λ mechanism using λ = K 2 , is ( , 0)-differentially private with respect to the original data D.

4. PROPOSED METHOD: PATE-GAN:

The proposed method builds on GAN and PATE frameworks. We replace the GAN discriminator with a PATE mechanism so that our discriminator is differentially private, but require the (differentiable) student version to allow back-propagation to the generator. We modify the implementation of the student, noting that the training paradigm presented in [25] is not appropriate for this setting due to the lack of publicly available data. Before training, we partition the dataset into k subsets,

D 1 , ..., D k , with |D i | = |D| k for ∀i.

4.1. GENERATOR:

The generator, G, is as in the standard GAN framework. Formally it is a function

G(•; θ G ) : [0, 1] d → U, parametrized by θ G that takes random noise, z ∼ Unif([0, 1] d ),

as input and outputs a vector in U = X × Y. The generator will be trained to minimize its loss with respect to the student-discriminator. Given n i.i.d. samples of Unif([0, 1] d ), z 1 , ..., z n , the empirical loss of G at θ for fixed S is defined by

EQUATION

We will denote by P G the distribution induced by G over U.

4.2. DISCRIMINATOR:

In the standard GAN framework, there is a single discriminator, D, that is trained in a directly adversarial fashion with G, where at each iteration either G is trying to improve its loss with respect to D or D is trying to improve its loss with respect to G. In our proposed model, however, we replace D with the PATE mechanism. This means we introduce k teacher-discriminators, T 1 , ..., T k , and a student discriminator, S. A noticeable difference is that the adversarial training is no longer symmetrical: the teachers are now being trained to improve their loss with respect to G but G is being trained to improve its loss with respect to the student S which in turn is being trained to improve its loss with respect to the teachers.

4.2.1. TEACHER-DISCRIMINATORS:

Formally, the teacher-discriminators (which we will refer to simply as teachers) are functions

T 1 (•; θ 1 T ), ..., T k (•; θ k T ) : U → [0, 1] each parametrized by θ i T .

The teachers are given either a real sample from their corresponding partition of the dataset (i.e. T i may receive a sample from D i ) as input or a sample from the generator. The teachers are then trained to classify them.

Given n i.i.d. samples of Unif([0, 1] d ), z 1 , ..., z n , we define the empirical loss of teacher i with weights θ i T for fixed G by

EQUATION

)

Each teacher is trained in the same way the discriminator is trained in a standard GAN framework, except that here the teacher only ever sees its partition of the real data.

4.2.2. STUDENT-DISCRIMINATORS:

The main innovation of our paper comes from our implementation of the student-discriminator (which we will refer to simply as the student) in this setting. The differential privacy guarantee provided by the standard student model is only with respect to the original data, D, and not the public data, P, used to train the student. In our setting, where the entire focus is on generating synthetic data because no data is publicly available, we must propose a novel methodology to train the student without public data.

We first note, that the student training paradigm described in [25] would involve training the student using data similar to that used to train the generator -i.e. by taking an equal number of samples from each and then labelling those using the standard PATE λ mechanism (where here "labelling" refers to assigning them a real/fake label -not the label y present in the data). We consider the implications of training the student on teacher-labelled generated samples only.

We first observe that during training of the generator, the discriminator is only evaluated on samples from the generator itself, and not the real data, so by training the student only on generated samples we are in fact training it on the distribution we need it to perform well on. However, we note that if the student only sees unrealistic samples from the generator (i.e. generated samples that most teachers label as fake), then the student will not contain any information that the generator can use to improve its generated samples. It is therefore important that some of the generated samples the student is trained on are realistic. We then note that if Supp(P U ) ⊂ Supp(P G ) then some of the generated samples will be realistic.

In order to ensure Supp(P U ) ⊂ Supp(P G ), we normalize the data into [0, 1] d and then initialize the parameters of the generator randomly using Xavier initialization. It follows that Supp

(P) ⊂ [0, 1] d ⊂ G([0, 1] d ) = G(Supp(Z)) = Supp(G(Z)) when Z ∼ Unif([0, 1] d ).

We create our training data for the student by taking n i.i.d. samples of Unif([0, 1] d ), z 1 , ..., z n , generating n samples using the generator, û1 , ..., ûn with ûj = G(z j ), and using the teachers to label these using PATE λ , setting r j = PATE λ (û j ). We train the student, S(•; θ S ) : U → [0, 1], to maximize the standard cross-entropy loss on this teacher-labelled data, i.e.

EQUATION

Although a priori the above mechanism does not appear to depend on the number of teachers, it should be noted that for fixed λ, more teachers results in the teacher-labelled dataset being less noisy -the noise being added is smaller relative to the counts n j . This introduces a trade-off -for a small number of teachers, the noise may be too large and thus render the output meaningless; with a larger number of teachers, less data can be used to train each teacher, which may also render the output meaningless, even though the noise has a smaller effect. Finding the right balance in this problem is key. In our experiments, we use d real and d generated samples to train each teacher where d is the dimension of the input space. Although the utility of a single teacher may be low, by aggregating (even noisily) the resulting classifier actually has high utility. Moreover, by using a minimal number of samples for each teacher, the effect of any individual sample on the output is small (because there are more teachers and each sample can effect at most 1 teacher) which means that our differential privacy guarantees are tighter -if we used fewer teachers, the mechanism still assumes that, in the worst case, the presence (or absence) of a single sample can completely flip a teacher's vote and so we still need to add the same noise.

We train G, T1 , ..., T k and S iteratively 1 , with each iteration of G consisting of first performing n T updates on all teachers, then performing n S updates of the student. We perform generator iterations until our privacy constraint, , has been reached. A block diagram of PATE-GAN can be found in the Appendix.

To calculate the privacy of our algorithm we use the moments accountant method given in [25] to derive a data-dependent privacy guarantee at run-time. Details of its definition, and key results we use can be found in the Appendix. We denote the moments accountant of PATE-GAN by α(l). The moments accountant allows us to more tightly bound the total privacy cost of our mechanism than standard composition theorems would, and moreover attributes a lower privacy cost to accessing the noisy aggregation of the teachers when the teachers have a stronger consensus with the intuition being that when the teachers have a strong consensus, a single teacher (and therefore a single sample) has a much lower influence on the output than when the votes (n 0 and n 1 ) are close. Pseudo-code for PATE-GAN can be found in Algorithm 1.

We now state the main theorem of the paper, which follows from the theory in [25] .

Theorem 1. Algorithm 1, which takes as input δ > 0, a dataset, D, and outputs G and is ( , δ)differentially private.

The proof relies on applying the post-processing theorem where the discriminator corresponds to the mechanism M which takes outputs in O (in our case this corresponds to the weights of the discriminator), and the generator corresponds to the function f which maps from O to O (which corresponds to the weights of the generator). For full details of the proof and further details of the theory required for it, see the Appendix.

5. EXPERIMENTS:

In this section, we use a real-world Kaggle dataset (Credit card fraud detection dataset [11] ) to evaluate PATE-GAN against the state-of-the-art benchmark (DPGAN [32] ). In addition, we provide high-level (average) results for five additional datasets (with various characteristics): MAGGIC [27] , UNOS-Heart wait-list [7] , Kaggle cervical cancer dataset [16] , UCI ISOLET dataset and UCI Epileptic Seizure Recognition dataset. A more detailed breakdown of the results for these datasets is given in the Appendix. Details of all six datasets can be also found in the Appendix.

5.1. EXPERIMENTAL SETTINGS:

To empirically validate the quality of the generated dataset we introduce three different trainingtesting settings. Setting A: train the predictive models on the real training set, test the performance of the models on the real testing set. Setting B: train on the synthetic training set, test on the real testing set ( [15] ), Setting C: train on the synthetic training set, test on the synthetic testing set. Note that the training set and the testing set are disjoint in both the real and synthetic datasets.

We are interested in two comparisons. If we see a high predictive performance on the real data for models that were trained on synthetic data (Setting B), we can infer that the synthetic data has Update teacher, T i , using SGD 10:

∇ θ i T - d j=1 log(T i (u j )) + log(1 -T i (G(z j )))

11:

for t 3 = 1, ..., n S do q ← 2+λ|n0-n1| 4 exp(λ|n0-n1|)

18:

for l = 1, ..., L do 19:

α(l) ← α(l) + min{2λ 2 l(l + 1), log((1 -q) 1-q

1-e 2λ q l + qe 2λl )} 20:

Update the student, S, using SGD 21:

∇ θ S - n j=1 r j log S(û j ) + (1 -r j ) log(1 -S(û j )) 22:

Sample z 1 , ..., z n i.i.d.

∼ P Z:



23::

Update the generator, G, using SGD 24:

∇ θ G n i=1 log(1 -S(G(z i ))

25:

ˆ ← min l α(l)+log( 1 δ ) l 26: Output: G captured the relationship between features and labels well. Moreover, synthetic data that does well in this setting can be used to train models without ever seeing the real data.

On the other hand, when we consider synthetic data for use in competitions such as Kaggle, we need synthetic data that allows researchers to do meaningful comparisons on the synthetic data. In this setting, the researchers will only be able to use the synthetic data as both the training and testing set, and will need to develop their algorithms using results on the synthetic data. Now it becomes important that the relative performance of two algorithms when trained and tested on the synthetic data (Setting C), is similar to their relative performance when trained and tested on the real data (Setting A). A simple requirement would be that if model 1 is better than model 2 on the real data, then model 1 is better than model 2 on the synthetic data. This allows researchers to use the synthetic data to choose the best method(s) to try on the real data (or rather to give to the data-holder to try on the real data).

For both comparisons, we use 12 different predictive models, shown in Table 1 . We use two performance metrics to measure the capability of each model in predicting the label: (1) area under the receiver operating characteristics curve (AUROC), (2) area under the precision recall curve (AUPRC). Throughout the experiments we fix δ = 10 -5 for use as input to PATE-GAN and DPGAN. We also report the performance of the original GAN framework ("GAN"), which serves to indicate an upper bound on performance and allows us to see how much performance is lost due to the two differential privacy mechanisms (PATE-GAN and DPGAN). The details of hyper-parameter optimization and benchmark implementations can be found in the Appendix. 

AUROC AUPRC:



GAN:



5.2. RESULTS WITH SETTING B:

In this subsection, we evaluate PATE-GAN and DPGAN in Setting B (trained on synthetic, tested on real) to understand whether or not the models are capturing the feature-label relationships well. Intuitively, if a synthetic dataset is such that a model trained on it performs well when performance is measured on real data, then the relationship between feature and label in the synthetic data is similar to that in the real data. In Table 1 we give the results for the Kaggle Credit dataset for all 12 predictive models. In Table 2 , we give the performance on each dataset averaged across the 12 methods for each of the 6 datasets. A breakdown of the performance of each predictive model for each dataset can be found in the Appendix. Across all datasets, we see that PATE-GAN is capable of generating synthetic samples that better preserve the feature-label relationship (according to AUROC and AUPRC) than DPGAN. We note that the performance of all models, including the original GAN model (i.e. PATE-GANor equivalently DPGAN -with (∞, ∞)-differential privacy) in the high dimensional UCI ISOLET and UCI Epileptic Seizure Recognition datasets is lower than in lower dimensional datasets (when compared to the baseline AUROC and AUPRC found in the Appendix). We do, however, see that both PATE-GAN and DPGAN show more significant decreases in performance than the original GAN in these high-dimensional settings. In the case of PATE-GAN, we believe this may be due to the fact that the student discriminator is trained only using data from the generator, and therefore requires that some of the generated data look somewhat realistic from the start, which is a harder requirement to satisfy as the data has more dimensions. On the other hand, in DPGAN, noise must be added to each component of the gradient (of the discriminator) and so in higher dimensions the norm of the noise added is larger. Note that in PATE-GAN, noise is added only to the teacher outputs, whose dimension (typically 1) does not depend on the dimension of the input data, and so this phenomena does not present itself in PATE-GAN. The results on both the UCI datasets would suggest that the loss from increasing noise norm (for DPGAN) is greater than from difficulty in randomly generating realistic samples (for PATE-GAN). In Fig. 1 , we investigate the trade off between privacy constraint and utility. In the table we report the average performance of AUROC over the 12 different predictive models for PATE-GAN and the benchmark for various (with δ = 10 -5 ). As can be seen in Fig. 1 , PATE-GAN is consistently better than DPGAN over the entire range of tested . We believe this is because the PATE mechanism allows us to more tightly bound the influence of a single sample on the discriminator, and hence we can provide tighter differential privacy guarantees -when the differential privacy guarantee is fixed, this results in higher quality synthetic data. Of course, as we increase (i.e. decrease the required privacy) both methods converge to the performance of GAN and the increase in performance of PATE-GAN over DPGAN becomes smaller.

Datasets:



5.4. SETTING A VS SETTING C: PRESERVING THE RANKING OF PREDICTIVE MODELS:

As discussed at the beginning of this section, it is important that a synthetic dataset respects the ranking of models (in terms of their prediction performances) [21] . To evaluate this, we now introduce a new metric, which we refer to as the Synthetic Ranking Agreement (SRA). Suppose that we have L predictive models, f 1 , f 2 , ..., f L2 . Furthermore, suppose that the performance of model i when trained and tested on the real data (Setting A) is A i ∈ R and that the performance of model i when trained and tested on the synthetic data (Setting C) is C i ∈ R. Then we define the Synthetic Ranking Agreement by

EQUATION

)

where I is an indicator function. Note that the summand is 1 when the ordering of algorithms j and k are the same in both settings, and is 0 when the ordering in one setting differs from the ordering in the other. We compare the SRA of PATE-GAN and the benchmark for various (with δ = 10 -5 ) 3 . As can be seen in Table 3 , PATE-GAN achieves the best SRA across all values of .

In the Appendix, we perform a similar experiment in which we compare the ranking of features by their importance (determined by their absolute Pearson correlation coefficient with the label) on the original dataset and on the synthetic dataset (generated by PATE-GAN and the benchmark) and report the results using a metric that is identical to SRA, with the model performances ({A i }, {C i }) substituted for feature importances.

5.5. QUANTITATIVE ANALYSIS ON THE NUMBER OF TEACHERS:

The number of teachers is a hyper-parameter of PATE-GAN and we choose the number of teachers among {N/10, N/50, N/100, N/500, N/1000, N/5000, N/10000} where N is the total number of samples. As we described in the previous section, there is a trade-off between number of teachers and the corresponding quality of the synthetic data. 

6. DISCUSSION:

In this paper we introduced a novel methodology for generating differentially private synthetic data.

Through several experiments we demonstrated the ability of our method to produce high quality synthetic data while being able to give strict differential privacy guarantees.

In order to apply PATE to the GAN setting, we needed to use the original GAN framework. Extending PATE to the regression setting so that, for example, a Wasserstein GAN can be used instead, is an open and interesting question, and a potential direction for future research.

ADDITIONAL RESULTS:



VARYING THE PRIVACY CONSTRAINT ( ) IN TERMS OF AUPRC:

epsilon (with delta = 10 -5 ) Similar to Fig. 1 in the main manuscript, Fig. 2 shows the trade off between the privacy constraint and utility, where utility is now measured in terms of AUPRC (rather than AUROC). We report the average performance in terms of AUPRC over the 12 different predictive models for PATE-GAN and the benchmark for various (with δ = 10 -5 ). As can be seen in Fig. 2 , PATE-GAN consistently outperforms DPGAN over the entire range of tested in terms of AUPRC as well.

PRESERVING THE RANKING OF VARIABLE IMPORTANCE IN KAGGLE CREDIT DATASET:

We compare the ranking of variables by their importance (according to absolute Pearson correlation coefficient with the label) on the original dataset and on both synthetic datasets. We report the results using agreed ranking probability. As can be seen in Table 6 , PATE-GAN achieves consistently better agreed ranking probability across all values of tested (with δ = 10 -5 ). 

PATE-GAN:



:

The teachers can be trained in parallel.

For the results in Table

3, we use the same 12 predictive models as used in Table1

The ordering of models according to Table1is in fact quite consistent -the average agreed ranking probability (now applied to different folds of the data, rather than real vs. synthetic data) is 0.9273 (for AUROC). The rankings used are therefore sufficiently stable for this to be a meaningful metric.

Figures and Tables:

(Neighboring Datasets) Two datasets D, D are said to be neighboring if ∃x ∈ D s.t. D \ {x} = D . (Definition 2. (Differential Privacy) A randomized algorithm, M, is ( , δ)-differentially private if for all S ⊂ O and for all neighboring datasets D, D : P(M(D) ∈ S) ≤ e P(M(D ) ∈ S) + δ

Pseudo-code of PATE-GAN 1: Input: δ, D, n T , n S , batch size n, number of teachers k, noise size λ Initialize: θ G , θ 1 T , ..., θ k T , θ S , α(l) = 0 for l = 1, ..., L 3: Partition dataset into k subsets D 1 , ..., D k of size |D| k while ˆ < do 5: for t 2 = 1, ..., n T do Sample z 1 , ..., z n i.i.d.

Figure 1: Average AUROC performance across 12 different predictive models trained on the synthetic dataset generated by PATE-GAN and DPGAN with various (with δ = 10 -5 ) (Setting B).

Figure 2: Average AUPRC performance across 12 different predictive models trained on the synthetic dataset generated by PATE-GAN and DPGAN with various (with δ = 10 -5 ) (Setting B).

Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10 -5 )-differentially private).

Table content: <table><tr><td/><td/><td colspan="2">PATE-GAN DPGAN</td><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>Logistic Regression</td><td>0.8950</td><td>0.8728</td><td>0.8720</td><td>0.4069</td><td>0.3907</td><td>0.3923</td></tr><tr><td>Random Forests [5]</td><td>0.9075</td><td>0.8980</td><td>0.8730</td><td>0.3219</td><td>0.3157</td><td>0.2926</td></tr><tr><td>Gaussian Naive Bayes [29]</td><td>0.8861</td><td>0.8817</td><td>0.8522</td><td>0.1963</td><td>0.1858</td><td>0.1601</td></tr><tr><td colspan="2">Bernoulli Naive Bayes [29] 0.8997</td><td>0.8968</td><td>0.8891</td><td>0.2169</td><td>0.2099</td><td>0.2069</td></tr><tr><td>Linear SVM [10]</td><td>0.7611</td><td>0.7523</td><td>0.7502</td><td>0.4473</td><td>0.4466</td><td>0.4464</td></tr><tr><td>Decision Tree [28]</td><td>0.9102</td><td>0.9011</td><td>0.8647</td><td>0.4071</td><td>0.3978</td><td>0.3672</td></tr><tr><td>LDA [3]</td><td>0.8710</td><td>0.8510</td><td>0.8487</td><td>0.1956</td><td>0.1852</td><td>0.1788</td></tr><tr><td>AdaBoost [17]</td><td>0.9143</td><td>0.8952</td><td>0.8809</td><td>0.4530</td><td>0.4366</td><td>0.4234</td></tr><tr><td>Bagging [4]</td><td>0.8951</td><td>0.8877</td><td>0.8657</td><td>0.3303</td><td>0.3221</td><td>0.3073</td></tr><tr><td>GBM [18]</td><td>0.8848</td><td>0.8709</td><td>0.8499</td><td>0.3057</td><td>0.2974</td><td>0.2773</td></tr><tr><td>Multi-layer Perceptron</td><td>0.9086</td><td>0.8925</td><td>0.8787</td><td>0.4790</td><td>0.4693</td><td>0.4600</td></tr><tr><td>XgBoost [8]</td><td>0.9058</td><td>0.8904</td><td>0.8637</td><td>0.3837</td><td>0.3700</td><td>0.3440</td></tr><tr><td>Average</td><td>0.8866</td><td>0.8737</td><td>0.8578</td><td>0.3453</td><td>0.3351</td><td>0.3219</td></tr></table>

Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10 -5 )-differentially private) over 6 different datasets. GAN is (∞, ∞)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.

Table content: <table><tr><td/><td/><td>AUROC</td><td/><td/><td>AUPRC</td><td/></tr><tr><td/><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>Kaggle Credit</td><td>0.8866</td><td>0.8737</td><td>0.8578</td><td>0.3453</td><td>0.3351</td><td>0.3219</td></tr><tr><td>MAGGIC</td><td>0.6574</td><td>0.6446</td><td>0.6286</td><td>0.3054</td><td>0.2952</td><td>0.2820</td></tr><tr><td>UNOS</td><td>0.6277</td><td>0.5996</td><td>0.5552</td><td>0.6554</td><td>0.6282</td><td>0.5862</td></tr><tr><td colspan="2">Kaggle Cervical Cancer 0.9268</td><td>0.9108</td><td>0.8699</td><td>0.5994</td><td>0.5460</td><td>0.4851</td></tr><tr><td>UCI ISOLET</td><td>0.8171</td><td>0.6399</td><td>0.5577</td><td>0.5561</td><td>0.2953</td><td>0.2146</td></tr><tr><td>UCI Epileptic Seizure Recognition</td><td>0.9173</td><td>0.7681</td><td>0.6718</td><td>0.8133</td><td>0.6512</td><td>0.5369</td></tr></table>

Synthetic Ranking Probability of PATE-GAN and the benchmark when comparing Setting A and Setting C for various (with δ = 10 -5 ) in terms of AUROC. The Synthetic Ranking Agreement of Original GAN is 0.9091, which is also attained by both PATE-GAN and DPGAN for = 50.

Table content: <table><tr><td/><td colspan="2">PATE-GAN DPGAN</td><td/><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>= 0.01</td><td>0.6909</td><td>0.5273</td><td>= 1</td><td>0.8364</td><td>0.8000</td></tr><tr><td>= 0.05</td><td>0.7455</td><td>0.6909</td><td>= 5</td><td>0.8909</td><td>0.8364</td></tr><tr><td>= 0.1</td><td>0.7818</td><td>0.7455</td><td>= 10</td><td>0.9091</td><td>0.8909</td></tr><tr><td>= 0.5</td><td>0.8000</td><td>0.7818</td><td>= 50</td><td>0.9091</td><td>0.9091</td></tr></table>

Table 4  quantitatively shows the trade-off between the number of teachers and the performance (in terms of both AUROC and AUPRC). Trade-off between the number of teachers and the performances (AUROC, AUPRC)

Table content: <table><tr><td colspan="2"># of teachers N/10</td><td colspan="3">N/50 N/100 N/500 N/1000 N/5000 N/10000</td></tr><tr><td>AUROC</td><td colspan="2">0.5425 0.6398 0.7638 0.8343 0.8737</td><td>0.8655</td><td>0.8282</td></tr><tr><td>AUPRC</td><td colspan="2">0.1273 0.2484 0.2900 0.3184 0.3351</td><td>0.3278</td><td>0.3092</td></tr></table>

Agreed ranking probability of PATE-GAN and the benchmark to order the features by variable importance in terms of absolute Pearson correlation coefficient HIGH-DIMENSIONAL RESULTS: UCI ISOLET AND UCI EPILEPTIC SEIZURE RECOGNITION

Table content: <table><tr><td/><td/><td colspan="2">DPGAN</td><td/><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>= 0.01</td><td colspan="2">0.8810</td><td>0.7963</td><td>= 1</td><td/><td>0.9048</td><td>0.8783</td></tr><tr><td>= 0.05</td><td colspan="2">0.8968</td><td>0.8148</td><td>= 5</td><td/><td>0.9127</td><td>0.8915</td></tr><tr><td>= 0.1</td><td colspan="2">0.8968</td><td>0.8333</td><td>= 10</td><td/><td>0.9153</td><td>0.8942</td></tr><tr><td>= 0.5</td><td colspan="2">0.9021</td><td>0.8545</td><td>= 50</td><td/><td>0.9153</td><td>0.9021</td></tr><tr><td>( , δ)</td><td/><td colspan="2">AUROC</td><td/><td/><td>AUPRC</td></tr><tr><td/><td>GAN</td><td colspan="3">PATE-GAN DPGAN</td><td>GAN</td><td>PATE-GAN DPGAN</td></tr><tr><td colspan="2">(10, 10 -5 ) 0.8171 (1, 10 -5 )</td><td>0.7688 0.6399</td><td>0.7390 0.5577</td><td colspan="2">0.5561</td><td>0.4734 0.2953</td><td>0.3831 0.2146</td></tr></table>

Average AUROC and AUPRC performance of 12 different predictive models trained on the synthetic datasets for = 1, 10 with δ = 10 -5 -Setting B using UCI ISOLET dataset. GAN is (∞, ∞)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.

Table content: <table><tr><td>( , δ)</td><td>AUROC</td><td/><td/><td>AUPRC</td><td/></tr><tr><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>(10, 10 -5 ) 0.9173 (1, 10 -5 )</td><td>0.8718 0.7681</td><td>0.8189 0.6718</td><td>0.8133</td><td>0.7662 0.6512</td><td>0.7201 0.5369</td></tr></table>

Average AUROC and AUPRC performance across 12 different predictive models trained on the synthetic datasets with various = 1, 10 with δ = 10 -5 -Setting B using UCI Epileptic Seizure Recognition dataset. GAN is (∞, ∞)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.

Table content: <table><tr><td colspan="2">MAGGIC DATASET RESULT</td><td/><td/><td/><td/><td/></tr><tr><td/><td/><td>AUROC</td><td/><td/><td>AUPRC</td><td/></tr><tr><td/><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>Logistic Regression</td><td>0.6645</td><td>0.6413</td><td>0.6415</td><td>0.3113</td><td>0.2951</td><td>0.2967</td></tr><tr><td>Random Forests</td><td>0.6492</td><td>0.6397</td><td>0.6147</td><td>0.2953</td><td>0.2891</td><td>0.2660</td></tr><tr><td>Gaussian Naive Bayes</td><td>0.6770</td><td>0.6726</td><td>0.6431</td><td>0.3258</td><td>0.3153</td><td>0.2896</td></tr><tr><td>Bernoulli Naive Bayes</td><td>0.6647</td><td>0.6618</td><td>0.6541</td><td>0.3008</td><td>0.2938</td><td>0.2908</td></tr><tr><td>Linear SVM</td><td>0.6410</td><td>0.6301</td><td>0.6322</td><td>0.2911</td><td>0.2904</td><td>0.2902</td></tr><tr><td>Decision Tree</td><td>0.6689</td><td>0.6598</td><td>0.6234</td><td>0.3163</td><td>0.3070</td><td>0.2764</td></tr><tr><td>LDA</td><td>0.6656</td><td>0.6433</td><td>0.6456</td><td>0.3118</td><td>0.2950</td><td>0.3014</td></tr><tr><td>AdaBoost</td><td>0.6524</td><td>0.6333</td><td>0.6190</td><td>0.3054</td><td>0.2890</td><td>0.2758</td></tr><tr><td>Bagging</td><td>0.6454</td><td>0.6380</td><td>0.6160</td><td>0.2912</td><td>0.2830</td><td>0.2682</td></tr><tr><td>GBM</td><td>0.6609</td><td>0.6470</td><td>0.6260</td><td>0.3106</td><td>0.3023</td><td>0.2822</td></tr><tr><td colspan="2">Multi-layer Perceptron 0.6390</td><td>0.6229</td><td>0.6091</td><td>0.2921</td><td>0.2824</td><td>0.2731</td></tr><tr><td>XgBoost</td><td>0.6604</td><td>0.6450</td><td>0.6183</td><td>0.3133</td><td>0.2996</td><td>0.2736</td></tr><tr><td>Average</td><td>0.6574</td><td>0.6446</td><td>0.6286</td><td>0.3054</td><td>0.2952</td><td>0.2820</td></tr></table>

Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10 -5 )-differentially private). GAN is (∞, ∞)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.

Table content: <table><tr><td colspan="3">UNOS HEART WAIT DATASET RESULT</td><td/><td/><td/><td/></tr><tr><td/><td/><td>AUROC</td><td/><td/><td>AUPRC</td><td/></tr><tr><td/><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td><td>GAN</td><td colspan="2">PATE-GAN DPGAN</td></tr><tr><td>Logistic Regression</td><td>0.6407</td><td>0.6155</td><td>0.5548</td><td>0.6691</td><td>0.6450</td><td>0.5901</td></tr><tr><td>Random Forests</td><td>0.6159</td><td>0.5950</td><td>0.5574</td><td>0.6436</td><td>0.6129</td><td>0.5768</td></tr><tr><td>Gaussian Naive Bayes</td><td>0.6323</td><td>0.6015</td><td>0.5343</td><td>0.6648</td><td>0.6371</td><td>0.5824</td></tr><tr><td>Bernoulli Naive Bayes</td><td>0.6213</td><td>0.6045</td><td>0.5763</td><td>0.6501</td><td>0.6363</td><td>0.6077</td></tr><tr><td>Linear SVM</td><td>0.6244</td><td>0.5979</td><td>0.5581</td><td>0.6486</td><td>0.6254</td><td>0.5892</td></tr><tr><td>Decision Tree</td><td>0.6209</td><td>0.6019</td><td>0.5590</td><td>0.6496</td><td>0.6284</td><td>0.5819</td></tr><tr><td>LDA</td><td>0.6403</td><td>0.6077</td><td>0.5530</td><td>0.6682</td><td>0.6406</td><td>0.5882</td></tr><tr><td>AdaBoost</td><td>0.6222</td><td>0.5928</td><td>0.5527</td><td>0.6404</td><td>0.6194</td><td>0.5826</td></tr><tr><td>Bagging</td><td>0.6084</td><td>0.5858</td><td>0.5493</td><td>0.6325</td><td>0.6074</td><td>0.5693</td></tr><tr><td>GBM</td><td>0.6374</td><td>0.6040</td><td>0.5585</td><td>0.6679</td><td>0.6352</td><td>0.5920</td></tr><tr><td colspan="2">Multi-layer Perceptron 0.6328</td><td>0.5927</td><td>0.5562</td><td>0.6629</td><td>0.6240</td><td>0.5856</td></tr><tr><td>XgBoost</td><td>0.6362</td><td>0.5956</td><td>0.5533</td><td>0.6676</td><td>0.6267</td><td>0.5880</td></tr><tr><td>Average</td><td>0.6277</td><td>0.5996</td><td>0.5552</td><td>0.6554</td><td>0.6282</td><td>0.5862</td></tr></table>

Performance comparison of 12 different predictive models in Setting B (trained on synthetic, tested on real) in terms of AUROC and AUPRC (the generators of PATE-GAN and DPGAN are (1, 10 -5 )-differentially private). GAN is (∞, ∞)-differentially private and is given to indicate an upper bound of PATE-GAN and DPGAN.

Table content: <table/>

Peer review (iclr2019):
 This paper considers using a GAN to generate synthetic data in a differentially private manner [see also https://www.biorxiv.org/content/early/2018/06/05/159756 ]. The key novelty is the integration of the PATE differential privacy framework from recent work. Specifically, rather than a single distinguisher as is usual in a GAN, there is a "student distinguisher" and several "teacher distinguishers". The student distinguisher is used as usual except that it does not have access to the real data, only the teacher distinguishers have access to the real data (as well as the synthetic data). The data is partitioned amongst the teacher distinguishers and their output is aggregated in a differentially private manner (and gradients are not revealed). The role of the teacher distinguishers is solely to correct the student distinguisher when it errs. What is strange about this setup is that the generator*s only feedback is from the gradients of the student distinguisher, which is never exposed to the real data. The entire training process relies on the generator producing realistic data by chance at which point the teacher distinguishers can provide positive feedback. (The paper remarks about this in the middle of page 5.) It*s surprising that this works, but there are experimental results to back it up. I think it would be appropriate to remark that generating private synthetic data is known to be hard in the worst case [ https://eccc.weizmann.ac.il/report/2010/017/ ] and therefore it is necessary to use techniques like GANs. Overall, I think the paper is interesting, well written, novel, and therefore appropriate for ICLR.