The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging
Authors: Georgi Ganev
Meenatchi Sundaram Muthu Selva Annamalai
Emiliano De Cristofaro
Venue: Accepted by TMLR ()

Abstract: Synthetic data created by differentially private (DP) generative models is increasingly used in real-world settings.
In this context, PATE-GAN has emerged as one of the most popular algorithms, combining Generative Adversarial Networks (GANs) with the private training approach of PATE (Private Aggregation of Teacher Ensembles).

In this paper, we set out to reproduce the utility evaluation from the original PATE-GAN paper, compare available implementations, and conduct a privacy audit.
More precisely, we analyze and benchmark six open-source PATE-GAN implementations, including three by (a subset of) the original authors.
First, we shed light on architecture deviations and empirically demonstrate that none reproduce the utility performance reported in the original paper.
We then present an in-depth privacy evaluation, which includes DP auditing, and show that \textit{all implementations leak more privacy than intended}.
Furthermore, we uncover \textit{19 privacy violations} and 5 other bugs in these six open-source implementations.
Lastly, our codebase is available from: \url{https://github.com/spalabucr/pategan-audit}.

Published in Transactions on Machine Learning Research (02/2025)
The Elusive Pursuit of Reproducing PATE-GAN: Benchmarking, Auditing, Debugging
Georgi Ganev University College London and SAS
georgi. ganev. 16 16wcll. uk
Meenatchi Sundaram Muthu Selva Annamalai University College London
meenatchi, anw.amaliiaaaamaaaa.aaa htt.maaammaa....a...... 22@uucl.ac. uk
Emiliano De Cristofaro
emiliomodeBes..urr.tt
University of California, Riverside
Reviewed on OpenReview: http://www....t..ooommmmoooooooe
Abstract
Synthetic data created by differentially private (DP) generative models is increasingly used in real-world settings. In this context, PATE-GAN has emerged one of the most popular algorithms, combining Generative Adversarial Networks (GANs) with the private training approach of PATE (Private Aggregation of Teacher Ensembles).
In this paper, we set out to reproduce the utility evaluation from the original PATE-GAN paper, compare available implementations, and conduct a privacy audit. More precisely, analyze and benchmark six open-source PATE-GAN implementations, including three by (a subset of) the original authors. First, shed light architecture deviations and empirically demonstrate that none reproduce the utility performance reported in the original paper. We then present an in-depth privacy evaluation, which includes DP auditing, and show that all implementations leak more privacy than intended. Furthermore, we uncover 19 privacy violations and 5 other bugs in these six open-source implementations. Lastly, our codebase is available from:
1 Introduction
synthetic data has been increasingly adopted share data within and across organizations while reducing privacy risks. The intuition is to train generative model the real data, draw samples from the model, and create new (synthetic) data points. As the original data may contain sensitive and/or personal information, synthetic data can be vulnerable to membership/ pproperty inference, reconstruction attacks, etc. (Hayes et al., 2019; Hilprecht et al., 2019; Chen et al., 2020; Stadler et al., 2022; Annamalai et al., 2024a; Ganev & De Cristofaro, 2023). Thus, models should be trained to satisfy robust definitions like Differential Privacy (DP) (Dwork et al., 2006; Dwork & Roth, 2014), which bounds the privacy leakage from the synthetic data. Combining generative models with DP has been advocated for or deployed by government agencies (NIST, 2018; Abowd et al., 2022; ONS, 2023; Hod & Canetti, 2024), regulatory bodies (ICO, 2023; FCA, 2024), and non-pree non-profit organizations (UN, 2023; OECD, 2023).
Numcrous DP generative models have been proposed (Zhang et al., 2017; Xie et al., 2018; Zhang et al., 2018; Jordon et al., 2019; McKenna el al., 2021; Zhang et al., 2021; McKenna et al., 2022), etc. Alas, new DP models are often published without a public/theee tthorughly reviewed codebase. Correctly implementing DP mechanisms and effectively communicating their propertics are often very challenging tasks in the real world (Cummings al., 2021; Houssiau et al., 2022b). This prompts the need for rigorous reproducibility efforts and auditing studies to confirm the utility and privacy claims of state-of-the-art DP generative models.

---

Published in Transactions on Machine Learning Research (02/2025)
original updated eyntheity turing borealis smartnoise
AUROC -24.57% -50.80%
-33.76% -77.38% -27.50% -26.92%
Table 1: Average reduction in AUROC (the random score 0.5 used as baseline) from the original paper (Jor- don et al., 2019) across twelve classifiers and four datasets (f = 1) for the six implementations.
Privacy Violations
Other Bugs
Data
Moments
Laplace
8
Labels
Implement.
PATE Partition
Accountant
Metadata Noise Scale Distribution Teachers Processing
original
updated
synthcity
turing
borealis
smartnoise
Table 2: Overview of the privacy violations and bugs found in each implementation.
In this paper, focus PATE-GAN (Jordon et al., 2019), which combines Private Aggregation of Teacher Ensembles (PATE) (Papernot et al., 2017; 2018) and Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) to train a generator in a privacy-preserving way using k disbhers-dssrimmnatorr and student- discriminator. PATE-GAN, published at ICLR'19, 19, is among the two most popular DP generative models for tabular data and the most popular deep learning model (with over 800 citations as of February 2025), remaining widely used, extensively studied, and re-implemented by researchers and practitioners. Moreover, PATE-GAN is very competitive vs. state-of-the-art models in terms of utility, privacy, and fairness, partic- ularly for high dimensional datasets (Rosenblatt et al., 2020; Ganev et al., 2022; 2024; Du & Li, 2024). We examine six public PATE-GAN implementations, including three by the original authors:
1. original', the first public code release in 2019 alongside the paper (Jordon et al., 2019);
2. updated?, released in 2021 and linked to from the latest version of the paper (Jordon et al., 2019); synthcity", the most recent implementation, included in the popular synthetic data benchmarking library synthcity (Qian et al., 2023);
turing', developed by the Alan Turing Institute, part of the TAPAS library (Houssiau al., 2022a);
5. borealis", part of Borealis Al's toolbox for private generation of synthetic data;
6. smartnoise included in OpenDP's popular library for DP synthetic data (OpenDP, 2021).
Our reproducibility study has two main objectives. First, we set to reproduce the utility performance reported in the original paper (Jordon et al., 2019) by studying and benchmarking the six implereenations.. Second, we empirically estimate PATE-GAN's privacy guarantees using DP auditing tools. In our experiments, we use all four publicly available tabular datasets (two Kaggle and two UCI) used in the original evaluation, a common image dataset (MNIST), and create a worst-case dataset as part of the DP auditing tests.
Our experimental evaluation yields several main findings:
We fail to reproduce the utility performance reported in the original paper (Jordon et al., 2019) in any of the six implementations we benchmark, with an average utility drop ranging from 25% to 77% across the four datasets (see Table 1);
All six implementations leak more privacy than they should i..., the empirical privacy estimates we obtain using black-box membership inference attacks are worse than the theoretical differential privacy bounds;
httpp://bicbnckee.orrg /ttb//bbnceeetog//mveeeeeeeeeeee www.ch/arraaaaaaaeeeeeaaaaaaaaaa httprd/chhaaammllooreetttaaaaeee htt.://rheeetaaaaapbbbeeeeeeeeee aansstte//lg/pptggaa///// GAN.py gan.py
2htpps: wtw.://mwwaanndddaaaaaaaaaaaltte
hhtpps: htt.aab..oo/vuuddcccaaaaaa/////s htt///bbobmnm///////////// habb//man/ssee//e//// hrt/syuthcity//luu
hhttps: htw.:///wwuuuuuuuuuinnnnniiiie/e
amain/sre/reppli wwtpn////ttt//////oooo lgggggn//aaaee
hatp/paat__ga.pyyyy gan.py
hctppe /ggtthub.comm/orrelllll////////a
baterratioooooo////aee baab/m models/pan patee//ppeee_/aee gan.py
hetpps
/synth/snsynnttii /snsynth hpttorch/toooottoe /nytooch/mn///ateg.g... /ntpeagga//aptppy
2

---

Published in Transactions on Machine Learning Research (02/2025)
As summarized in Table 2, we identify 19 privacy violations and 5 other bugs, predominantly in how PATE is implemented, e-g., partitioning and feeding data to the teachers-diseriminatoo,, tracking the privacy budget during model training, etc.
To facilitate open research and robust privacy (re-)implemennatiion,, we release our codebase, including the utility benchmark and privacy auditing tools; see
2 Preliminaries
Notation. In the rest of the paper, we denote a dataset containing N d-dimensional samples as D ( w:)] where 22 E X (the feature space) and yo E y (the label space).
Differential Privacy (DP). DP is mathematical formalization of privacy that limits the influence of the input data points on the output of a function. In the context of machine learning, DP bounds the probability of distinguishing hhether any particular record was used to train a model. Formally, a randomized algorithm A satisfies (e, 6DDD if, for all S S Range(AA) and for all neighboring datasets, D and D' differing in a single data record, it holds that (Dwork et al., 2006; Dwork & Roth, 2014):
Pr(A(D) 8) S e Pr(A(D') E S) -
The privacy budget c is positive number quantifying the privacy leakage (the lower, the better), while á is a very small number representing the probability of failure. The post-processing property guarantees that a DP mechanism's 's output can be used arbitrarily without additional privacy leakage.
PATE. The Private Aggregation of Teacher Ensembles (PATE) framework (Papernot et al., 2017; 2018) trains a DP discriminative model using semi-supervised learning techniques. A model trainer is assumed to have access to private dataset and an unlabelled public dataset. First, the private dataset is separated into & disjoint partitions, and teacher-classifer is trained on each partition (without privacy), then, the teachers predict the labels of the public dataset. Next, the predictions are noisily aggregated through the Laplace mechanism (Dwork et. al., 2006) to get DP labels. Finally, a student--lassifee is trained on the pairs of public records and noisy (DP) labels. The student-classifier can be released as it satisfics DP through the post-processing property (since it was trained on public data and DP labels).
PATE-GAN, In the context of synthetic data, a generative model G is typically fitted on D to capture probability representation and is later sampled to generate new data (of size N'), S ~ G(N'). PATE- GAN (Jordon et al., 2019) incorporates PATE into the standard mini-max generator vs. discriminator train- ing in a Generative Adversarial Network (GAN) (Goodfellow et al., 2014). We report its pseudo-code in Algorithm 1 in Appendix A.1. As before, D is first separated into k disjoint partitions. In each iteration, k teachors-discrimninttos. TI...Ta Tk, are trained on their corresponding data partition. Instead of using a public dataset, in PATE-GAN the generator G generates samples that are labeled by the teachers as "real" or "fake" through the PATE mechanism. The student-discrimintto S is then trained on these generated samples and noisy (DP) labels. Finally, the generator is trained by minimizing the loss on the student. Since the generator is only exposed to the
which in turn sees only "fake" generated sam- ples and noisy (DP) labels, PATE-GAN satisfies DP through the post-processing property. Throughout the training, the privacy budget spent in each iteration is tracked by an adjusted moments accountant (Abadi et al., 2016). In the accountant's ' calculations (see lines 16-19 in Algorithm 1), À denotes the scale of the Laplace mechanism added by PATE when aggregating the teachers' "fake" and "real" votes (ng and ni: respectively), while L the number of moments. As explained in the original paper (Jordon et al., 2019), finding the right balance between the noise level controlled by À (where larger values result in less added noise) and the number of teachers & is key to achieving meaningful aggregation. Finally, in Appendix A.2, we discuss discrepancies between PATE-GAN and PATE (Papernot et al., 2017).
DP Auditing. This entails empirically estimating the privacy leakage from DP model, denoted as cemp and comparing it with the theoretícal privacy budget, e. One key goal is for the estimates to be tight (i.c., Cemp R c), so that the audit can be effectively used to validate the DP implementation or detect DP violations in case cemp > e (Bichsel al., 2018; 2021; Niu al., 2022). To estimate Cemp, we use membership inference attacks (MIAs) (Shokri et al., 2017; Hayes et al., 2019), whereby an adversary attempts to infer
3

---

Published in Transactions on Machine Learning Research (02/2025)
whether a target record was part of the training data (i.e., (rr, E D), which closely matches the DP definition. The MTA process is run repeatedly as a distinguishing game; we randomly select between D and D' = D (uT,yT), pass it to the adversary, and get their predictions Pred = p and Pred' = {predd, pred...... These yield a false positive rate a and a false negative rate 3. 95% confidence upper bounds for the a and 8 (αr and 3) are lypically calculated using Clopper-Pearson intervals (Clopper & Pearson, 1934), but recent work has shown that using Bayesian credible intervals instead can dramatically improve the Cemp estimates (Zanella-Béguell Bayesian credible intervals may not produce statistically valid lower bounds, and therefore we mainly use Clopper-Pearson bounds throughout the paper. Finally, the upper bounds are converted into the lower bound Cemps as done in (Nast et al., 2021):
et al., 2023). However, Nasr et al. (2023) note that using
Cemp max (log ((1 ((1-a - (1-a-88)
-8) log 5) (1-3-5)/a)...
3 Related Work
DP Generative Models Benchmarks. There are multiple DP generative models for synthetic tabular data, including copulas (Li al., 2014; Gambs al., 2021), graphical models (Zhang et al., 2017; McKenna et al., 2021; Cai et al., 2021; Mahiou et al., 2022), workload//qeer--bsee (Vietri et al., 2020; Aydore et al., 2021; Liu et al., 2021; Vietri et al., 2022; McKenna et al., 2022; Maddock et al., 2023a), and deep generative models like VAEs (Acs et al., 2018; Abay et al., 2018), GANs (Xic et al., 2018; Zhang et al., 2018; Jordon et al., 2019; Alzantol & Srivastava, 2019; Frigerio el. al., 2019; Long et al., 2021), and others (Zhang et al., 2021; Ge et al., 2021; Truda, 2023; Vero et al., 2024).
A few benchmarking studies (Rosenblatt et al., 2020; Tao al., 2022; Ganev et al., 2024; Du & Li, 2024) compare PATE-GAN to other DP generative models. Although PATE-GAN is not the best-performing model at fidelity and in Aiilarity--bsser evaluations, it is among the best at downstream classification (for original, synthcity, smartnoise) compared to PrivBayes (Zhang et al., 2017), PrivSyn (Zhang et al., 2021), MST (McKenna et al., 2021), DPGAN (Xie et al., 2018), DP-WGAN (Alzantot & Srivastava, 2019), and TableDiffusion (Truda, 2023), especially on datasets with large number of columns (Rosenblatt et al., 2020; Ganev et al., 2024; Du & Li, 2024). On the other hand, Tao et al. (2022) claim that PATE-GAN (smartnoise) cannot beat simple baselines, although they only use narrow datasets (fewer than 25 columns).
None of these benchmarks use the datasets from the original paper (Jordon et al., 2019). Moreover, when developing new PATE-GAN implementations, the authors do not compare them to previous ones. Therefore, this prompls the need to evaluate them against each other and check how closely they resemble the original model (Jordon et al., 2019) in terms of architecture and utility performance.
DP Generative Models Auditing. Recent rescarch on DP auditing has focused on discriminative models trained with DP-SGD (Abadi et al., 2016) in both central (Jayaraman & Evans, 2019; Jagielski et al., 2020; Tramer et al., 2022; Nasr et al., 2021; 2023; Zanella- Béguelln et al., 2023; Steinke et al., 2023; Kong et al. 2024; Annamalai & De Cristofaro, 2024; Cebere et al., 2024) and federated (Maddock et al., 2023b; Andrew et al., 2024) settings. For generative models, Houssiau et al. (2022a) loosely audit MST (McKenna et al. 2021) using Querybased, a black-box attack which runs a collection of random queries, while Annamalai et al. (2024b) are the first to tightly audit PrivBayes (Zhang et al., 2017), MST (McKenna et al., 2021), and DP-WGAN (Alzantot & Srivastava, 2019) (and detect several De-related bugs) by proposing implementa- tion specific white-box attacks and worst-case data records. Also, Lokna et al. (2023) find floating points vulnerabilitics in MST (McKenna at al., 2021) and AIM (McKenna et al., 2022).
PATE-GAN has been mostly overlooked by previous work on DP auditing. Using novel shadow model- based black-box attack and manually crafted worst-case target, GroundHog, Stadler et al. (2022) show that PATE-GAN (original) leaks more privacy than it should, while van Breugel et al. (2023) show that PATE- GAN (synthcity) is less privale than non-DP models like CTGAN (Xu et al., 2019) and ADS-GAN (Yoon et al., 2020) using a density-based MLA. However, neither work explores the underlying reasons. In this work, we examine six PATE-GAN implementations to determine whether the privacy leakage is unique to a single implementation and dig deeper into the underlying reasons behind DP bugs/violation This number is comparable to, and often exceeds, the number of implementations studied in previous privacy attacks vs.
4

---

Published in Transactions on Machine Learning Research (02/2025)
(Jordon et al., 2019)
original
updated synthcity
turing borealis
smartnoise
Using PATE?
/
Teachers
N/1,000
10
10
10
N/1,000
X
1e-3
le-1
1e-3
a size (L)
100
100
100
5
le-5
le-5
1e-5
1/(NvN)
1e-5
1e-5
1/(NvN)
Framework
Tensor Flow
Tensor RMSProp
PyTorch Tensor Flow
PyTorch
PyTorch
Optimizer
Adam
Adam
Adam
Adam
Adam
Adam
Learning Rate
Te-4
le-4
le-4
1e-4
le-4
1e-4
1e44
Batch Size
64
128
64
200
128
64
64
Max Iterations
_
10,000
_1
1,000 or
100
_
Teachers Iterations
5
1
5
Student Iterations
5
1
10
1
5
5
Generator leratiins
1
10
1
Teachers Layers
1 layer layers
_1
{1}
{1)
_1
{d/2, 1}
{2d/3, d/3,
1)
Student Layers
{d, d, 1} {d, 13
(100, 1}
{d, d, 1}
{d/2, 1}
{2d/3, d/3.
1}
Noise Dimension
d/4 {d, d, d}
d
d/4 (d, d, d}
d/4
64
Generator Layers
{d, d/2, d}
{4d, 4d, d}
{100, d}
{2d, d} {64, 64, d}
(oaly single discriminator). &Epochs. Table 3: Architectures and hyperparameters of the six PATE-GAN implementations (N and are the number of data records and columns; the values in { denote the width of the corresponding layer).
Until there available privacy budget.
"heee
DP generative models for tabular data six DP implementations (Annamalai et al., 2024b), three (Stadler et al., 2022), and two (Houssiau et al., 2022a; Breugel et al., 2023; Annamalai et al., 2024a).
Reproducibility Studies. Overall, papers aimed at reproducing the computational experiments of previ- ously published work not only verify their empirical results but also ensure the reliability and trustworthiness of their claims. This is particularly important in fields like machine learning and security, where individual privacy may be al risk. For instance, recent work in this space includes (Nokabadi el al., 2024; Garg Tiwari, 2024; Feng al., 2024; Chaudhuri et al., 2024).
4 PATE-GAN Implementations
This section reviews the six implementations under study, as also summarized in Table 3. Algorithm 1 in Appendix A.1 reports the PATE-GAN algorithm as presented in the paper (Jordon et al., 2019), while we highlight any deviations in the six implementations Appendix E.
Architectures. In the first column of Table we list the architecture and hyperparameters of the PATE- GAN algorithm explicitly stated in the original paper (Jordon et al., 2019). However, find that moments accountant; rather, there is single discriminator (not is teachers and a student; see lines 2, 9-10, and 17 in Algorithm 1) that observes all the data and is trained for a given number of iterations since the privacy budget spent is not tracked (lines 4, 18-21, and 29). This a serious deviation that can compromise privacy. Also, updated and syntheity, released by subset of the original authors, do not use neural networks as teachers, but use Logistic Regression classifiers that are (re-) frittd from scratch on every itcration (lines 8-10): while this does not violate privacy, it might negatively affect the model's utility. Moreover, updated uses the RMSProp optimizer instead of Adam. Finally, borealis and smartnoise resemble the original algorithm the closest, although they change the networks' depth, with smartnoise opting for teachers with three layers instead of the default one. Also, all implementations have different network depths and noise dimensions, mostly depending on the input data.
original, as well as turing, do not actually implement the PATE framework
Data Support and Proccssing. The implementations also differ in the kind of data types they support and how they process the input data. For instance, original only runs on numerical and binary features, iee, categorical columns have most two distinct categories and are treated numerical data. Only
5

---

Published in Transactions on Machine Learning Research (02/2025)
Dataset
Paper (Jordon et al., 2019)
original updated synthcity turing borealis smartnoise
Kaggle Credit
0.8737
0.5000
0.5000
0.5172
0,5000
0.6300
0.7358
Kaggle Cervical Cancer
0.9108
0.9265
0.8739
0.8584
0.5000
0.9431
0.8025
UCI ISOLET
0.6399
0.8342
0.5861
0.6292
0.5000
0.6219
0.6152
UCI Epileptic Seizure
0.7681
0.6867
0.6187
0.7165
0.7425
0.6730
0.6963
Table 4: Average AUROC scores of the different implementations over the classifiers (e = 1).
two implementations, turing and syntheity, preserve the original data types (such as integers) in the newly generated synthetic data.
In terms of data processing, original and turing scale the numerical data between 0 and while smartnoise between -1 and 1. Also, synthcity transforms the numerical data and centers it around 0 by fitting Bayesian Gaussian Mixture model on every column and using standard scaler on the clusters. On the other hand, updated and borealis expect data that has already been processed/scaae do not return synthetic data in the scale of the input data (we adjust these models and use min-max scaling, similarly to the others). Apart from smartnoise, none of the implementations extract the data bounds in a DP way, which has been proven to lcak privacy (Stadler et al., 2022; Annamalai al., 2024b). (Note, fact, that turing allows for the bounds to be passed input).
and, consequently,
Remarks. The main goals of this paper are to reproduce the empirical experiments in the original pa- per (Jordon et al., 2019) and to test the privacy propertics of the implementations as they were released, without modifying them. We do so as these implementations are in the public domain and have been adopted and used in practice (e.g., both synthcity and smartnoise libraries have average monthly downloads exceed- ing 4,500). Specifically, we assume that the architectures, hyperparameters, and data processing decisions the authors have chosen for their respective implementations are optimal and do not change them unless stated otherwise. While we leave fixing discrepancies and bugs to future work, we have contacted the authors regarding these issues (see Sections 6.3 and 7) and have offered to assist them with addressing the bugs.
5 Utility Benchmark
In this paper, we reproduce the utility experiments on all public datasets from the original paper (Jordon et al., 2019), ie., Kaggle Credit, Kaggle Cervical Cancer, UCI ISOLET, and UCI Epileptic Seizure (we also use MNIST; see Appendix B.1).
Setup. We the evaluation criteria from (Jordon et al., 2019): for every synthetic dataset, we fit 12 classifiers (sce Appendix B.2) and report the Area Under the Receiver Operating Characteristic curve (AUROC) and the Area Under the Precision-Recall Curve (AUPRC) scores. For the sake of comparison, as done by the original authors in updated, we consider the best of the scores from 25 synthetic datasets, training five models and generating five synthetic datasets per model. We use an 80/20 split, i.c., using 80% of the records in the datasets to train the predictive/generatiiv ggneerative models and 20% for testing. Finally, we use two training-testing settings:
Setting A: train on the real training dataset, test on the real testing dataset;
Setting B: train on the synthetic dataset, test the real testing dataset (as done in (Esteban et al., 2017));
Hyperparameters. We set 8 = 10-5 (as in (Jordon et al., 2019)) and use the implementations' default hyperparameters, with a couple of exceptions. First, we set the maximum number of training iterations to 10,000 to reduce computation. In our experiments, this limit only reached for borealis and smartnoise with e 2 10. Consequently, we train synthcity for a set of iterations rather than epochs. Second, for updated, use À 0.001 to prevent the model from spending its privacy budget in just a few iterations. Finally, for all models, we set the number of teachers to N/1,000 following (Jordon et al., 2019), with the only exception being Kaggle Credit, where we set it to N/5.000 due to computational constraints; regardless,
As per
hwtps//rrrrrrgggsssseeaee eeynthhct//yyttttttyyyy and https:/ https://yyyistaats.ort http:://wppsssttta.o//a/a//////s
6

---

Published in Transactions on Machine Learning Research (02/2025)
Classifier
(Jordon et al., 2019)
original
updated
synthcity
turing
borealis
smartnoise
Logistic Regression
0.5109
0.5198
0.5271
0.5816
0.5304
0,5414
Random Forest
0.9529
0.7362
0.8187
0.9574
0.8283
0.8832
Ganssian Naive Bayes
0.3864
0.5385
0.9126
0.5000
0.6515
0.6551
Bernoulli Naive Bayes
0,6524
0.5185
0.9283
0.9680
0.5141
0.6517
Linear SVM
0,5181
0.5214
0.5333
0.5220
0.5460
0,5384
Decision Tree
0.6359
0.6614
0.7470
0.7003
0.6818
0.6769
LDA
0.5086
0.5229
0.5341
0.6226
0.5420
0.5440
AdaBoost
0.9162
0.6752
0.7588
0.9151
0.8582
0.7572
Bagging
0.8339
0.7446
0.8285
0.8635
0.8187
0.8386
GRM
0.9376
0.6512
0.7097
0.9188
0.7387
0.8235
MLP
0.5202
0.6194
0.7204
0.5682
0.5439
0.6267
XGBoost
0.8678
0.7149
0.5797
0.7927
0.8227
0.8194
Average
0.7681
0.6867
0.6187
0.7165
0.7425
0.6730
0.6963
Table 5: Performance comparison of 12 classifiers in Setting B (train on synthetic, test on real) in terms of AUROC on UCI Epileptic Seizure (e = 1). Baseline performance: 0.5000. Setting A (train on real, test on real) performance: 0.8103.
0.8
0.7
AUROC
0.6
0.5
baseline
original
synthcity
borealis
0.4
Paper
updated
turing
smartnoise
10-2
10
10
10
10a
epsilon (with delta 10 9)
Figure 1: Performance comparison of 12 classifiers (averaged) in Setting B (train synthetic, test real)
in terms of AUROC with various (with 5 10 10-5) on UCI Epileptic Seizure.
note that the difference in performance in the original paper (Jordon et al., 2019) is negligible. As done in (Jordon et al., 2019), we also consider the data bounds to be public, i.e., we not extract the bounds of the data in a DP manner (see Section 4) for smartnoise, thus saving its budget for the model training.
Analysis. In Table 4, report the AUROC scores for all datasets and implementations for c= 1. The AUROC scores are averaged over the 12 classifiers (recall that, for each classifier, consider the best of the scores from the 25 synthetic datasets). For completeness, we also list the scores reported in the paper (Jordon et al., 2019). Due to space limitations, we defer additional/ more detailed results to Appendix C more precisely, in Table 8-9 and Figure 6-7 for Kaggle Credit, Table 10-11 and Figure 8-9 for UCI Epileptic Seizure, and Figure for MNIST. Except. for three cases (two of which with original), all experiments underperform the results from (Jordon et al., 2019) (by 40.15% on average).
Kaggle Credit. Looking closer Kaggle Credit, which is (Jordon et al., 2019)'s main focus, we note that, out of the implementations that either do not neural networks teachers do not implement PATE correctly (original, updated, synthcity, and turing), only synthcity performs slightly better than random, i.c., AUROC 0.5. This is not surprising due to the extreme imbalance the dataset (only 0.17% of the instances have a positive label) and the known disparate effect DP (Bagdasaryan et al., 2019; Farrand et al., 2020; Ganev al., 2022). In other words, even a moderate data. imbalance can cause a disproportionate utility drop on the underrepresented class, which well captured by the AUROC metric. Although borealis and smartnoise achieve better results (0.6300 and 0.7358, respectively). thus supporting previous claims
7

---

Published in Transactions on Machine Learning Research (02/2025)
original
Dataset
Paper (Jordon et al., 2019)
Non-Conditional
Conditional
Kaggle Credit
0.8737
0.5000
0.8760
Kaggle Cervical Cancer
0.9108
0.9265
0.9330
UCI ISOLET
0.6399
0.8342
0.6927
UCI Epileptic Seizure
0.7681
0.6867
0.7029
AUROC A
-24.57%
-3.34%
Table 6: Average AUROC scores of (non-Jconditional original over the 12 classifiers (c 1) and reduction in AUROC (0.5 used as baseline) from the original paper (Jordon et al., 2019).
that PATE offers a reduced disparate effect (Uniyal et al., 2021; Ganev, 2022), there is still a considerable gap to the results reported in (Jordon et al., 2019), i.e. 0.8737. After further examining original (and the OpenReview discussion (Jordon et al., 2019)), we find that the implementation expects the synthetic data label distribution to be provided, and this is done to exactly match the counts in the training data. We consider this unaccounted privacy Icakage, which violates the privacy of the training data, and in our experiments, we do not pass the label distribution.
UCI Epileptic Seizure. Next, we focus the UCI Epileptic Seizure experiments as this: i) has the second most records (11.5k), ii) is high dimensional (179 columns), and iii) has the smallest imbalance (20%). For == 1, we report the AUROC scores in Table 5, and the scores for varying between 0.01 and 50 in Figure 1. Again, none of the implementations come close to the results reported in (Jordon et al., 2019) - note that, for UCI Epileptic Seizure, the authors only report the average scores across the 12 classifiers. Apart from original, which does not implement PATE, the only implementation that consistently achieves better utility with increasing is synthcity. In fact,, borealis and smartnoise's AUROC peak at = 1 then slightly drop to around 0.6, while for >> 1, updated and turing's drop significantly approaching the random baseline. This is unexpected and could be due to various reasons, c.g., overfitting and mode collapse. Morcover, synthcity has an early stopping criterion, which might be another contributing factor. Also nole that, even with DP processing disabled, smartnoise cannot be trained for c < 0.5. Additionally, in Figure 8 in Appendix C, we plot the mean AUROG scores across the 12 classifiers (rather than the maximum) alongside their standard errors to further confirm that the none of the implementations can reach the results reported in (Jordon et al., 2019) even when accounting for randomness.
Effect of Number of Teachers. We also experiment with different numbers of teachers-discrininatoo {N/50, N/100, N/500, 1/1,, 000, N/55, 000}, similar to the original paper (Jordon et al., 2019), on UCI Epilep- tic Seizure and present the results in Table 11. While the best AUROC scores for all implementations occur at N01,000, as claimed by (Jordon et al., 2019), not all models behave consistently. Only synthcity and borealis show improved results as the number of teachers reduced from N/50 to N/1, 000, with perfor- mance decreasing thereafter. By contrast, updated and smartnoise behave more randomly.
Effect of Conditional Generation. As mentioned carlier, unlike (Jordon et al., 2019), do not use the training labels distribution when generating synthetic data, as this would result in unaccounted privacy leakage. Nevertheless, we compare the performance of non non-conditional and conditional generation in Table 6 in an attempt to reach the results in (Jordon et al., 2019). Across all datasets, we observe that conditional generation gets closer to them, most notably for Kaggle Credit, where the AUROC score improves beyond random performance. Overall, the utility drop decreases from -24.57% with non-conditional generation to -3.34% with conditional, effectively reproducing the performance reported in (Jordon et al., 2019).
Effect of Default Hyperparometers. For completeness, we also evaluate all implementations using the hyper- parameters (and networks depths) specified in the original paper (Jordon et al., 2019), which are listed in the first column of Table 3). The results for all datasets are reported in Table 12 in Appendix C. As expected, the utility of all implenentations drops by an average of 17.5% compared to when their respective default hyperparameters are used.
Comparison with DPGAN. Finally, we compare the AUROC results of the six implementations to DP- GAN (Xie et al., 2018) as reported in (Jordon et al., 2019) (Kaggle Credit: 0.8578, Kaggle Cervical Cancer:
8

---

Published in Transactions on Machine Learning Research (02/2025)
data records only teacher has seen
data records other teachers have seen
updated
synthcity
borealis
smartnoise
600
400
200
0
2
1
4
2
4
teacher
teacher
teacher
teacher
Figure 2: Data records seen by the five teachers-discrimnaatee (e 1) on Kaggle Cervical Cancer.
seacher-discrimantor loss on first data subset seen
other teachers-diic socchrr-discrrmantors loss on same data subset
updated
synthcity
bo realli
sma rtnoise
cross entropy
0.02
0.02
0.6
0.6
0.4
0.4
0.01
0.01
0.2
0.2
0.0
0.0
50
100
150
15
500
1000
100
200
iteration
iteration
iteration
iteration
Figure 3: Cross entropy of the five Leachers-discriiiinators on a fixed subset of data (e = 1).
0.8699, UCT ISOLET: 0.5577, UCI Epileptic Seizure: 0.6718). For e 1, the PATE-GAN implementa- tions perform better in 13 of the 24 experiments, which contradicts the claim in (Jordon et al., 2019) that PATE-GAN is uniformly better than DP-GAN.
6 Privacy Evaluation
In this section, perform in-depth privacy evaluation of the six PATE-GAN implementations.
6.1 PATE-GAN Training
We start by tracking different aspects of the model's training procedure namely, the records seen by the teachers, the teachers' losses, and the accountant's moments. We do so over single training run on average-case dataset, i.c., Kaggle Cervical Cancer. We fix except for the moments accountant evaluation, in which we train the models for fixed 1,000 iterations thus obtaining large values. We also set d = 10-5, k = 5, x = 0.001, and use all other default hyperparameters.
Analysis. First, in Figure 2, we report the mumber of distinct data points provided as input to the five teachers. We observe that only borealis and smartnoise correctly partition the data into disjoint subsets and feed them to the corresponding teachers. While updated initially scparates the data, an indexing bug in the implementation results in all teachers only seeing the last teacher's data (the rest. is not used at all). On the other hand, synthcity samples records at each iteration, and every teacher ends up seeing all the data. Unfortunately, for updated and synthcity, this breaks one of the main PATE assumptions (i.c., every teacher can only see a disjoint partition of the data).
Next, in Figure 3, we report the losses the teachers during training, ie., the cross entropy of one teacher on its first seen data subset vs. the average of the others on the same subset. Since updated and synthcity use Logistic Regression instead of neural networks, their initial losses are much lower com- pared to borealis/ /twpm/mmaatisseeeeeee (around 30 times lower) as they (re-) fiet)fitted until convergence at every
9

---

Published in Transactions on Machine Learning Research (02/2025)
moments accountant
25
50
minimum estimated epsilon
updated
synthcity
borealis
smartnoise
10a
ostimated epsilon
10a
10
10a
109
10
10a
10a
10a
10a
10a
10a
10a
10
10
10a
10a
10a
10a
10
10a
iteration
iteration
iteration
iteration
Figure 4: Moments accountant values for 1,000 training iterations.
iteration. Whereas, for updated, the two losses are exactly the same since the classifiers are fitted on the same data, the losses of synthcity are much "jumpier" because the data changes at each iteration. As for borealis and smartnoise, their performance is as expected, with the loss on seen data (the blue line in the plot) being initially lower than the one on unseen data (orange line), and both smoothly getting lower with more iterations before converging to approximately the same value. In Figure 11 in Appendix C, plot the losses over 10 training runs, and observe the same patterns.
Finally, in Figure 4, we plot the 100 moments of the moments accountant over 1,000 iterations. Their scale is very different, with only borealis and smartnoise's values being identical. The first moment of synthcity (corresponding to the estimated privacy budget) much lower compared to updated, i.e., at iteration 1,000, synthcity is 30k vs. 50k for updated. After manually inspecting the code, we find an indexing bug for synthcity, which, unfortunately, makes it severely underestimate c. By contrast, borealis and smartnoise massively overestimate c, to around 320k (a multiple of 6 compared to updated) after 1,000 iterations. This is due to another bug in the privacy accountant, term is not scaled by the log operator. We (re-)list all the violations in Section 6.3.
6.2 DP Auditing of PATE-GAN
Our DP auditing procedure uses/adapts two membership inference attacks namely, GroundHog (Stadler et al., 2022) and Querybased (Houssiau et al., 2022a) and derives temp via the distinguishing game (An- namalai et al., 2024b) discussed in Section 2.
Adversarial Model. DP auditing is typically performed in one of two models: black-box, where the MIA adversary only has access to the synthetic data, or white-box, where they can also observe the trained generative model and its internal parameters. For the sake of our experiments, we focus on the former using two black-box attacks (GroundHog and Querybasod). We do so as black-box models are considered more realistic to execute in the real world, even though they yield less tight audits (as shown in (Houssiau et al., 2022a; Annamalai et al., 2024b)). In other words, auditing PATE-GAN implementations in the black-box model provides with measuring stick for privacy leakage.
GroundHog and Querybased attacks rely on different approaches to featurize, or reduce the dimensionality of, the input synthetic datasets. For the former, we use Fnaine, which extracts every column's min, max, mean, median, and standard deviation. For the latter, we run every possible query, i.e., all values in the dataset's domain (although, in practice, we limit the domain), and count the mumber of occurrences in the synthetic data. Once the features are extracted, we fit a Random Forest classifier to get Pred and Pred'.
Setup& & Hyperparameters. We run GroundHog choosing a real larget from the datasel by running
average-case dataset, i.e, Kaggle Cervical Cancer, as done in previous work (Meeus et. al.,
B.gg, in white-bux attack vs. GANs like LOGAN (Hayes et al., 2019), the adversary directly leverages the discriminator trained using DP-SGD and therefore private- to observe its output on the target record (higher coufideuce usually indicates that the record was part the training data). However, this approach does not directly apply to PATE-GAN, as the adversary cannot query the individual teachers, which are non-private, but only the student, which already DP.
10

---

Published in Transactions on Machine Learning Research (02/2025)
oretial epsilon
Querybased
theoretical epsilon
GroundHog
original
original
updated
updated
synthcity
synthcity
turing
turing
borealis
borealis
smartnoise
smartnoise
4
1
Cemp
Eemp
(a) Querybased on worst-case dataset.
(b) GroundHog on Kaggle Cervical Cancer
Figure 5: DP auditing with different black-box MIAs (e = 1, as per the dashed red lines).
2023; Annamalai et al., 2024b). This entails rumming MIAs limited number of times (we select 100) on a collection of records furthest away from the rest (we choose 64) and selecting the record yielding the highest AUC the target. For Querybased, 0) records, and a worst-case target record (1, 1, 1). This limits the number of queries run on the dataset, i.e., all possible combinations, to 8. We run both atlacks for 1,000 iterations, using the first 400 outputs to the classifier, the next 200 for validation (adjusting the optimal decision boundary), and the final 400 for testing. Finally, note that even if the adversary is 100% correct, the maximum attainable Cemp is around 4.7, limitation coming from the statistical power of the Clopper-Pearsee method (Nasr al., 2021).
manually craft worst-case dataset, consisting of 4 repeated (0, 0,
We train all models with 1, 5 10-5, and 0.001. For Querybased, use two teachers since there are only four/five records, while for GroundHog use five teachers. To reduce computation, set the maximum number of training iterations per generative models to 1,000, as we train 2,000 shadow models per setting (24,000 in total).
A Analysis. In Figure 5, we plot the empirical privacy estimates (temp) obtained with the different MIAs. For the Querybased attack (see Figure 5a), we run the MIA on a worst-case dataset and manually crafted target residing outside the bounds of the data, previous work (Nasr et al., 2021; 2023; Annamalai et al., 2024b) has shown that these (strong assumptions) are needed to andit DP-SGD and other DP generative models tightly. For all PATE-GAN implementations, see that temp > e, which means that we detect privacy violations across all of them.
Next, we relax the adversarial model by running MIAs on an averago-casc dataset, Kaggle Corvical Cancer, and a real larget selected from the data. Running GroundHog results cemp >> for all implementations (see Figure 5b). This strongly suggests we are successfully detecting other privacy violations beyond worst-case datasots/targets. which is the main auditing setting used by (Stadler al., 2022; Annamalai et al., 2024b) against other DP generative models (as discussed in Section 3). Overall, the fact that we obtain Cemp >> (implying large privacy leakage) against PATE-GAN implementations even in black-box model, further points to the severity of the privacy violations. The very significant privacy leakage observed in borealis and smartnoise might seem surprising, but it might be due to algorithmic and privacy analysis differences between PATE-GAN and the original PATE (Papernot et al., 2017), as discussed Appendix A.2.
We repeat both experiments using the Bayesian credible intervals from Gaanela-Béguelln et al., 2023) (see Figure 12 in Appendix C), and, similarly to the Clapper-Pearson method, we always get Cemp > We even achieve slightly higher estimates on 9 out of 12 occasions.
Remarks. Overall, our DP auditing procedure follows that of (Annamalai et al., 2024b) and uses elements of (Stadler et al., 2022; Houssiau et al., 2022a; B et al., 2023; Meeus et al., 2023).
Nevertheless, Annamalai et al. (2024b) only andit PrivBayes (Zhang al., 2017), MST (McKenna et al., 2021), and OP-N (Alzantot & Srivastava, 2019), while we audit PATE-GAN, They show that tight auditing for these models necessary to effectively identify privacy violations is only possible with worst-
11

---

Published in Transactions on Machine Learning Research (02/2025)
case datasets/targets and white-box access to the model. Furthermore, while they detect handful of privacy violations, they do not thoroughly explore why//hent these violations happen.
In contrast, we discover numerous privacy violations across all six implementations using only average-case datasets and black-box access to the model (Figure 5b), which are much more conservative and realistic assumptions. Additionally, we provide an in-depth analysis of these privacy violations, including finding the specific lines of code where they occur (Section 6.3).
6.3 Summary of Privacy Violations
We now provide an overview of the bugs and privacy violations we discover, also reported in Table 2. When applicable, we highlight them in Algorithms 2, 3, 4, 5, 6, and 7 in Appendix E.
As discussed, original and turing do not actually implement PATE teachers/studde (only single discriminator) and no moments accountant. Both use the Gaussian mechanism (McSherry & Talwar, 2007) instead of Laplace and have errors in the 8 scale, resulting in lower sensitivity. More precisely, original uses the XOR operator (*) instead of power (**) in Python, turing uses multiplication instead of division. Also, updated ingests less noise than necessary as it uses Laplace with standard deviation of À instead of 1/X.
In terms of data partition, neither original, updated, synthcity, nor turing separate the data correctly into disjoint sets and/or feed them to the teachers accordingly. Also, updated and synthcity use Logistic Regression instead of neural networks for the teachers. Moreover, synthcity, borealis, and smartnoise have errors in the moments accountant the former has indexing bug while the latter two skip log operator. At generation, original feeds the unperturbed training labels distribution.
Finally, we find that the majority of implementations (original, updated, synthcity, borealis) directly extract the data bounds from the data in a non-DP way. Also, updated and borealis do not return synthetic data the original scale, while turing rounds down all integer values when processing the generated data back to the original bounds.
7 Conclusion
This paper presented a benchmark evaluation of six popular implementations of PATE-GAN (Jordon et al., 2019), aiming to 1) reproduce their analysis of the synthetic data's utility on downstream classifiers analysis and 2) perform deeper privacy analysis. Alas, none of the implementations reproduces the utility reported in (Jordon et al., 2019), achieving, on average, 40% lower AUROC scores. Moreover, our privacy evaluation (including DP auditing) specific to PATE-GAN exposes privacy violations and bugs in all six implementations (cf. Table 2).
Numerous privacy-preserving technologies, including DP synthetic data, are already deployed critical industries such as healthcare and finance (NHS, 2021; Giuffrè Shung, 2023; FCA, 2024). Ensuring robust privacy protection in these applications is essential, as failures could lead to catastrophic breaches and the exposure of individuals' sensitive data. However, implementing DP mechanisms correctly in practice is highly complex, as many privacy-related bugs can be easily overlooked, or worse, challenging or even impossible to detect through manual code review. Papers like ours help automate the testing and verification of DP implementations" theoretical protections, or uncover discrepancies. Our auditing scheme can serve as an inspiration for researchers and developers to validate their own DP implementations. As a result, we believe our work will encourage more reproducible research and greater trust in the field.
Acknowledgements. We are grateful the TMLR Action Editor and reviewers for their valuable feedback and suggestions, which helped to significantly improve our paper. We also thank Bogdan Kulynych for pointing us to discrepancies between PATE-GAN and PATE.
Responsible Disclosure. In the spirit of responsible disclosure, in June 2024, we contacted the authors of the six implementations, sharing the detected DP violations via emails and GitHub issues, along with the exact lines of code where they occur and potential fixes. More specifically, we sent emails to the authors of or original and turing and raised 11 GitHub issues (see Table 13 in Appendix D). As of February 2025, none of the bugs have been fixed.
12

---

Published in Transactions on Machine Learning Research (02/2025)
References
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Nya Mironov, Kunal Talwar, and Li Zhang. Deep Learning with Differential Privacy. In CCS, 2016.
Nazmiye Ceren Abay, Yan Zhou, Murat Kantarcioglu, Bhavani Thuraisingham, and Latanya Sweeney, Pri- vacy preserving synthetic data release using deep learning. In ECML PKDD, 2018.
John M Abowd, Robert Ashmead, Ryan Cumings-Menon, Simson Garfinkel, Micah Heinock, Christine Heiss, Robert Johns, Daniel Kifer, Philip Leclere, Ashwin Machanavaijhala, al. The 2020 census disclosure avoidance system topdown algorithm. Harvard Dala Science Review, 2022.
Gergely Acs, Luca Melis, Claude Castelluccia, and Emiliano De Cristofaro. Differentially private mixture of generative neural networks. IEEE TKDE, 2018.
Moustafa Alzantot and Mani Srivastava. Diffcrential Privacy Synthotic Data Generation using WGANs. differential privacy synthetic data challenge, 2019.
huups///gihub..omm//
Galen Andrew, Peter Kairouz, Sewoong Oh, Alina Oprea, H Brendan McMahan, and Vinith Suriyakumar. One-shot Empirical Privacy Estimation for Federated Learning. In ICLR, 2024.
Ralph G. Andrzejak, Klaus Lehnertz, Florian Mormann, Christoph Rieke, Peter David, and Christian E. Elger. Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Phys. Rev. E, 2001.
Meenatchi Sundaram Muthu Selva Annamalai and Emiliano De Cristofaro. Nearly Tight Black-Bos Auditing of Differentially Private Machine Learning. In NeurIPS, 2024.
Meenatchi Sundaram Muthu Selva Annamalai, Andrea Gadotti, and Lue Rocher. A linear reconstruction approach for attribute inference attacks against synthetic data. In USENIX Securily, 2024a.
Meenatchi Sundaram Muthu Selva Annamalai, Georgi Ganev, and Emiliano De Cristofaro. "What do you want from theory alone?" Experimenting with Tight Auditing of Differentially Private Synthetic Data Generation. In USENIX Securily, 2024b.
Sergul Aydore, William Brown, Michael Kearns, Krishnaram Kenthapadi, Luca Melis, Aaron Roth, and Ankit A Siva. Differentially private query release through adaptive projection. In ICML, 2021.
Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. Differential privacy has disparate impact on model accuracy. In NeurIPS, 2019.
Benjamin Bichsel, Timon Gehr, Dana Drachsler-Cohen, Petar Tsankov, and Martin T. Vechev. DP-Finder: Finding Differential Privacy Violations by Sampling and Optimization. CCS, 2018.
Benjamin Bichsel, Samuel Steffen, Ilija Bogunovic, and Martin Vechev. DP-Sniper: Black- Discovery of Differential Privacy Violations using Classifiers. In IEEE S&P, 2021.
Kuntai Cai, Xiaoyu Lei, Jianxin Wei, and Xiaokui Xiao. Data synthesis via differentially private markov random fields. PVLDB, 2021.
Tudor Cebere, Aurélien Bellet, and Nicolas Papernot. Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model. arv:2:2405.1 14457, 2024.
Kamalika Chaudhuri, Chuan Guo, Laurens van der Maaten, Saeed Mahloujifar, and Mark Tygert. Guaran- tees of confidentiality via hanmersiyy-chppma---bbbi bounds. TMLR, 2024.
Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. Gan-lcaks: a taxonomy of membership inference allacks against generative models. In ACM CCS, 2020.
Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In KDD, 2016.
Charles J Clopper and Egon S Pearson. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 1934.
Ron Cole and Mark Fanty. ISOLET. UCI Machine Learning Repository, 1994.
Rachel Cummings, Gabriel Kaptchuk, and Elissa M Redmiles. "I need a better description': an investigation into expectations for differential privacy. Tn ACM CCS, 2021.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Yuntao Du and Ninghui Li. Towards Principled Assessment of Tabular Data Synthesis Algorithms. wXXiv2442.06680 2024.
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 2014.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006.
Cristóbal Esteban, Stephanie I. Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans, arXiv: 02663, 2017.
Tom Farrand, Fatemehsadat Mireshghallah, Sahib Singh, and Andrew Trask. Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy. In PPMLP; 2020.
FCA. Using Synthetic Data in Financial Services. https://www.feeet www.w.ca.orgg.o www.:ea...r...uuuubbieeee http://uiaaaatooooooratttet pogoot-ssing-yytthtti--tttt
mamancia--sorvcces..dde 2024.
Yunzhen Peng, Tim GJ Rudner, Nikolaos Tsilivis, and Julia Kempe. Atlacking bayes: On the adversarial robustness of bayesian neural networks. TMLR, 2024.
Kelwin Fernandes, Jaime Cardoso, and Jessica Fernandes. Transfer Learning with Partial Observability Applied to Cervical Cancer Screening. In IbPRIA, 2017.
Lorenzo Frigerio, Anderson Santana de Oliveira, Laurent Gomez, and Patrick Duverger. Differentially private generative adversarial networks for time series, continuous, and discrete open data. In IFIP SFC,, 2019.
Sébastien Gambs, Frédéric Ladouceur, Antoine Laurent, and Alexandre Roy-Gaumond. Growing synthetic data through difforontially-privaee vinc copulas. PETS, 2021.
Georgi Ganev. DP-SGD vs PATE: Which Has Less Disparate Impact on GANs? PPAI, 2022.
Georgi Ganev and Emiliano De Cristofaro. On the Inadequacy of Similarity- Privacy Metries: Recon- struction Attacks agains "Truly Anonymous Synthetic Data". arXin:2312005114 14, 2023.
Georgi Ganev, Bristena Oprisanu, and Emiliano De Cristofaro. Robin Hood and Matthew Effects: Differ- ential privacy has disparate impact on synthetic data. In ICML, 2022.
Georgi Ganev, Kai Xu, and Emiliano De Cristofaro. Graphical vs. Deep Generative Models: Measuring the Impact of Differentially Private Mechanisms and Budgets on Utility. In ACM CCS, 2024.
Shivank Garg and Manyana Tiwari. Unmasking the Voil: An Investigation into Concept Ablation for Privacy and Copyright Protection in Images. TMLR, 2024.
Chang Ge, Shubhankar Mohapatra, Xi He, and Ihab F. Ilyas. Kamino: Constraint- Differentially
Private Data Synthesis. PVLDDB 2021.
Mauro Giuffrè and Dennis Shung. Harnessing the power of synthetic data in healthcare: innovation, application, and privacy. NPJ Digital Medicine, 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NewrIPS, 2014.
Jamie Hayes, Luca Melis, George Danczis, and Emiliano De Cristofaro. Logan: membership inference attacks against generative models. In PoPETs, 2019.
Benjamin Hilprecht, Martin Härterich, and Daniel Bernau. Monte carlo and reconstruction membership inference attacks against generative models. In PoPETs, 2019.
Shlomi Hod and Ran Canetti. Differentially Private Release of Israel's National Registry of Live Births. arXiv:2405.0080 00267, 2024.
Florimond Houssiau, James Jordon, Samuel N Cohen, Owen Daniel, Andrew Elliott, James Geddes, Callum Mole, Camila Rangel-Smith, and Lukasz Szpruch. Tapas: toolbox for adversarial privacy auditing of synthetic data. NewrIPS Workshop Synthetic Data for Empowering ML Research, 2022a.
Florimond Houssian, Lue Rocher, and Alexanle de Montjoye. On the difficulty of achieving differential privacy in practice: user- guarantees in aggregate location data. Nature Communications, 2022b.
14

---

Published in Transactions on Machine Learning Research (02/2025)
ICO.
Privacy-enhancing technologies (PETs).
/ttp://oooorggaaaiiins///// 2023.
Matthew Jagielski, Jonathan Ullman, and Alina Oprea. Auditing Differentially Private Machine Learning: How Private is Private SGD? NewrIPS, 2020.
Bargav Jayaraman and David Evans. Evaluating differentially private machine learning in practice. In USENIX Securily, 2019.
James Jordon, Jinsung Yoon, and Mihaela, Van Der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In ICLR, 2019.
William Kong, Andres Medina, Monica Ribero, and Umar Syed. DP-Auditorium: Large Scale Library for Auditing Differential Privacy. In TEEE SBP, 2024.
Yann LeCun, Corinna Cortes, and CJ Burges. MNIST handwritten digit database. ATT Labs, 2010.
Haoran Li, Li Xiong, and Xiaoqian Jiang. Differentially private synthesization of multi-dimensionall data using copula functions. In EDBT, 2014.
Terrance Liu, Giuseppe Victri, and Steven Z Wu. Iterative methods for private synthetic data: Unifying framework and new methods. NeurIPS, 2021.
Johan Lokna, Anouk Paradis, Dimitar Dimitrov, and Martin Vechev. Group and Attack: Auditing Differ- ential Privacy. In CCS, 2023.
Yunhui Long, Boxin Wang, Zhuolin Yang, Bhavya Kailkhura, Aston Zhang, Carl A. Gunter, and Bo Li. G-PATE: Scalable differentially private data generalor via private aggregation of teacher discriminators. In NewrlPS, 2021.
Samuel Maddock, Graham Cormode, and Carsten Maple. FLAIM: AIM-basod Synthotic Data Generation in the Federated Setting. arXiv:2310.03447, 2023a.
Samuel Maddock, Alexandre Sablayrolles, and Pierre Stock. CANIFE: Crafting Canaries for Empirical
Privacy Measurement in Federated Learning. In ICLR, 2023b.
Sofiane Mahion, Kai Xu, and Georgi Ganev. dpart: Differentially Private Autoregressive Tabular, a General Framework for Synthetic Data Generation. TPDP, 2022.
Ryan McKenna, Gerome Miklau, and Daniel Sheldon. Winning the NIST Contest: a scalable and general approach to differentially private synthetic data. JPC, 2021.
Ryan McKenna, Brett Mullins, Daniel Sheldon, and Gerome Miklau. Aim: An adaptive and iterative
mechanism for differentially private synthetic data. PVLDB, 2022.
Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. In FOCS, 2007.
Matthieu Meeus, Florent Guepin, Ana-Maria Cretu, and Yves-Alexandre de Montjoye. Achilles' Heels: vulnerable record identification in synthetic data publishing. arXin:2306. 10308, 2023.
Milad Nasr, Shuang Songi, Abhradeep Thakurta, Nicolas Papernot, and Nicholas Carlin, Adversary Instan- tiation: Lower Bounds for Differentially Private Machine Learning. In IEEE S&P, 2021.
Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramer, Matthew Jagielski, Nicholas Carlini, and Andreas Terzis. Tight Auditing of Differentially Private Machine Learning. In USENIX Securily, 2023.
NHS. A&E synthetic data.
uk/ wwwdddeeentaal-prvvvoossssseeeee
2021.
NIST. 2018 Differential privacy synthetic data challenge. https://www.nist.. gww.ii.ttttoooooooooooe gowggoooo/tooo /ctl///////t pscr/ 2018.
Ben Niu, Zejun Zhou, Yahong Chen, Jin Cao, and Fenghua Li. DP-Opt: Identify High Differential Privacy Violation by Optimization. In WASA, 2022.
Adversarial Attacks Against Robust Transformer Trackers. TMLR, 2024. Fatemeh Nourilenjan Nokabadi, Jean-Frangohi Lalonde, and Christian Gagné. Reprodueibility Study
OECD. Emerging privacy-enhancing technologies.
https:/xwww.oecddlbbee www.eeedddddiibbbbrr..oo.... www.aaar..orooooto//oeoeeeeaaa /pape//
bf121be4-en, 2023.
15

---

Published in Transactions on Machine Learning Research (02/2025)
ONS. Synthesising the linked 2011 preserving its confidentiality. synthesising- the- linked- 2011-cee1sssseeee wew-sseenn--d-eeeeeeeeeeeeee---e
Census
and deaths
dataset
while
httpes//hhtaascienccaappss wwwp://wwwsuuoo.uuuuuu.. /os.gov.uk/
2023.
OpenDP. SmartNoise SDK: Tools for Differential Privacy Tabular Data. https:/ httss//github.oom. /ttp://ttii..c///////////// /opendp/ smartnoise-sde sdk, 2021.
Nicolas Papernot, Martin Abadi, Ulfar Erlingsson, lan Goodfellow, and Kunal Talwar. Semi-supervised knowledge transfer for deep learning from private training data. In ICLR, 2017.
Nicolas Papernot, Shuang Song, Πya Mironov, Ananth Raghunathan, Kunal Talwar, and Ulfar Erlingsson. Scalable private learning with pate. In ICLR, 2018.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python. JMLR, 2011.
Andrea Dal Pozzolo, Olivier Caclen, Reid A. Johnson, and Gianluca Bontempi. Calibrating Probability with Undersampling for Unbalanced Classification. In IEEE SSCI, 2015.
Zhaozhi Qian, Rob Davis, and Mihaela van der Schaar. Synthcity: a benchmark framework for diverse use cases of tabular synthetic data. In NewrIPS Datascts and Benchmarks Track, 2023.
Lucas Rosenblatt, Xiaoyan Liu, Samira Pouyanfar, Eduardo de Leon, Anuj Desai, and Joshua Allen. Differ- entially private synthetic data: Applied evaluations and enhancements. arXic:2011.055 1.55537, 2020.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shnatikov. Membership Inference Attacks against Machine Learning Models. In IEEE S&P, 2017.
Theresa Stadler, Bristena Oprisamu, and Carmela Troncoso. Synthetic data 1al--ann anonymization groundhog day. In USENIX Security, 2022.
Thomas Steinke, Milad Nasr, and Matthew Jagielski. Privacy Auditing with One (1) Training Run. In NewrIPS, 2023.
Yuchao Tao, Ryan McKenna, Michael Hay, Ashwin Machanavajjhala, and Gerome Miklau. Benchmarking differentially private synthetic data generation algorithms. In PPAI, 2022.
Florian Tramer, Andreas Terzis, Thomas Steinke, Shuang Song, Matthew Jagielski, and Nicholas Carlini. Debugging Differential Privacy: A Case Study for Privacy Auditing. orXie:2202.12113 2022.
Gianluca Truda. Generating tabular datasets under differential privacy. arXiv:2308. 14 2023.
UN. The United Nations Guide on privacy-enhancing technologies for official statistics. https:/ happs://unste /unstats.um. wwwp:/sstttttatttaaaataeeeae
2023.
Archit Uniyal, Rakshit Naidu, Sasikanth Kotti, Sahib Singh, Patrik Joslin Kenfack, Fatemehsadat Mireshghallah, and Andrew Trask. DP-SGD vs PATE: which has less disparate impact on model ac- curacy? PPML, 2021.
Boris van Breugel, Hao Sun, Zhaozhi Qian, and Mihaela der Schaar. Membership Inference Attacks against Synthetic Data through Overfitting Detection. AISTATS, 2023.
Mark Vero, Mislav Balunovié, and Martin Vechev. CuTS: Customizable Tabular Synthetic Data Generation. TCML, 2024.
Giuseppe Vietri, Grace Tian, Mark Bun, Thomas Steinke, and Steven Wu. New oracle-efficient algorithms for private synthetic data release. In ICML, 2020.
Giuseppe Vietri, Cedric Archambeau, Sergul Aydore, William Brown, Michael Kearns, Aaron Roth, Ankit Siva, Shuai Tang, and Steven Z Wu. Private synthetie data for multitask learning and marginal queries. NewrlPS, 2022.
Liyang Xie, Kaixiang Lin, Shu Wang, Fei Wang, and Jiayu Zhou. Differentially private generative adversarial network. arX1v:1802.06739, 2018.
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infand and Kalyan Voeramachaneni. Modeling tabular data using conditional gan. NewrIPS, 2019.
16

---

Published in Transactions on Machine Learning Research (02/2025)
Jinsung Yoon, Lydia N Drumright, and Mihaela Van Der Schaar. Anonymization Through Data Synthesis Using Generative Adversarial Networks (ADS-GAN). JBHT, 2020.
Santiago Bégueli, Lukas Wutschitz, Shruti Tople, Ahmed Salem, Victor Rühle, Andrew Paverd, Mohammad Nascri, Boris Köpf, and Danicl Jones. Bayesian estimation of differential privacy. In ICML, 2023.
Jun Zhang, Graham Cormode, Cecilia M Procopiuc, Divesh Srivastava, and Xiaokui Xiao. Privbayes: private data release via bayesian networks. ACM TODS, 2017.
Xinyang Zhang, Shouling Ji, and Ting Wang. Differentially private releasing via deep generative model (technical report). wxXi::180.00599 2018.
Zhikun Zhang, Tianhao Wang, Jean Honorio, Ninghui Li, Michael Backes, Shibo He, Jiming Chen, and Yang Zhang. PrivSyn: Differentially Private Data Synthesis. In USENIX Security, 2021.
17

---

Published in Transactions on Machine Learning Research (02/2025)
A PATE-GAN Algorithm and Differences with PATE
A.1 PATE-GAN Algorithm
Algorithm reports the original PATE-GAN training procedure (Jordon et al., 2019) discussed in Section 2.
Algorithm Psee of PATE-GAN
Input: 6, nT, nig batch size n, number of teachers k, noise size Initialize: fe, 0t...., 07, fs, a(I) 0 for 1 = 1,.... L
3: Partition dataset into h subsets DI...., Dr size [2]
R
while c do
for l2 L,,... R& do
6:
Sample 2,, nn for i == 1..... h do
i..d.
Pz
7:
Sample uI,... Un Lid. D;
8:
Update teacher, T,, using SGD
VoL 2 j log(T:(u;)) + log(1 - T(((()))))
10:
11:
end for
12:
end for
13:
for 13 L...., ns do
14:
Sample 21, ..,..., 2n 1..d
Pz
15:
for j = L..... n do
16:
û + G(z,) Tj + PATEX( for j = 1,...,
17:
18:
Update moments accountant
- 2+2/ng-mi
19:
(ooppA(no-nn) for 1 - I...,,T.T. do
20:
alI) - a(I) + min (2 1), log((11 (1tt2))) ) + 9 end for
21:
22:
23:
Update the student, S, using SGD
24:
Vog -E rj log S(0,) + (1 r;) log(1 5(0,))
25:
end for
Sample 21,..., 2n lid. Pz
26:
27:
Update the generator, G, using SGD
28:
VoG (Ei= log(1 S(G(2)))
E min +lgg))
29:
30:
end for
31:
end while
32:
Output: G
A.2 Differences between PATE-GAN and PATE
We now highlight key differences between the privacy analyses of PATE-GAN (Jordon et al., 2019) and PATE (Papernot et al., 2017) (recall that the former is directly based on the latter).
First, the notation used by (Jordon et al., 2019) and (Papernot et al., 2017) are inconsistent with each other. Specifically, in Scction 3.2 (Jordon et al., 2019) define the PATEş mechanism to add noise Lap(A) and satisfy 2 DD. However, in Theorem 2, (Papernot al., 2017) originally define the PATE, mechanism to add noise Lap(2) and satisfy (2y, DD. This has two consequences: i) unclear why PATE-GAN's privacy analysis is different by a factor of 2, and ii) the noise parameter in PATE-GAN is inversely related
Notation holded here for emphasis but the original papers appear unbalded.
18

---

Published in Transactions on Machine Learning Research (02/2025)
to y in PATE. However, the privacy analysis of PATE-GAN - specifically, the moments accountant (lines 18-21 in Algorithm 1) and Theorem 5 in (Jordon et al., 2019). both based on Theorem 3 and Lemma 4 in (Papernot et al., 2017) directly substitutes À for (not 5) and includes the factor 2. Moreover, updated and synthcity, the two implementations from the original authors implementing PATE, are inconsistent with each other. When aggregating the teachers' voles, the former adds noise Lap(X), while the latter ). Then, both update their accountants according to Algorithm (i.e., using 2). Overall, while the notations in PATE-GAN (Jordon al., /update/ synthcity might be inconsistent, synthcity seems to follow PATE (Papernot et al., 2017) the closest (but still omits the factor 2).
Next, we examine the moments accountant updates (line 21 in Algorithm 1). Theorem 5 in (Jordon et al., 2019) overlooks condition for 4, namely, y - stated by Theorem in (Papernot et al., 2017). Additionally, PATE-GAN's accountant update takes the minimum over two factors, whereas PATE's imple- mentation includes third one, y,.10
Finally, PATE (Papernot et al., 2017) explains that since the privacy budget analysis is data-dependent, should itself be released in a DP way. Their implementation follows this principle," 10 whereas PATE-GAN overlooks it and outputs the unperturbed e.
Overall, during our utility and privacy experiments, we follow PATE-GAN's analysis from (Jordon et al., 2019) and leave the analysis and resolution of these discrepancies with PATE (Papernot et al., 2017) for future work.
B Additional Preliminaries
B.1 Datasets
We use four of the original six datasets from (Jordon et al., 2019) the other two are not publicly available specifically, Kaggle Credit (Pozzolo et al., 2015), Kaggle Cervical Cancer (Fernandes et al., 2017), UCI ISOLET (Cole & Fanty, 1994), UCT Epileptic Seizure (Andrzejak et al., 2001). We also use MNIST (LeCun et al., 2010), the popular digits dataset. The main characteristies of these datascts are sunmarized in Table 7. When needed, we follow (Jordon et al., 2019) to convert the downstream tasks associated with the datasets into binary classification.
Kaggle Credit is a dataset consisting of 284,807 credit card transactions labeled as fraudulent or not. Besides the labels, there 29 numerical features. The dataset is highly imbalanced, only 492 transactions (0.17%) are actually fraudulent.
Kaggle Cervical Cancer contains the demographic information and medical history of 858 patients. There are 35 features, 24 binary/11 numerical, and a biopsy status label. Only 55 patients (6.4%) have positive biopsies.
The UCI ISOLET dataset has 7,797 featurized promunciations (617 numerical features) of a letter of the alphabet. (the label). We transform the task to classifying vowels vs. consonants. Out of the 7,797 letters, there are 1,500 (19.2%) vowels.
UCI Epileptic Seizure includes the brain activities encoded into 178 numerical vectors of 11,500 patients. Originally, there were five distinct labels, which we transformed into binary one to indicate whether there was a scizure activity. This results in 2,300 (20%) records with a positive label.
MNIST a benchmark image dataset consisting of 60,000 grayscale handwritten digits, each of size 28x28 pixels. The associated task is digit classification. This is the most balanced dataset, with the ratio of the least to the most frequent digit being 80.4%.
10 As per: http://///aaaaaaeeee/////////h/r http://aeeeea///aasssssssp
19

---

Published in Transactions on Machine Learning Research (02/2025)
Dataset
N
d
Imbalance
AUROC
AUPRC
Kaggle Credit
284,807
30
0.0017
0,8176
0.5475
Kaggle Cervical Cancer
858
36
0.0641
0.9400
0.6192
UCI ISOLET
7,797
618
0.1921
0.9678
0.9002
UCI Epileptic Seizure
11,500
179
0.2000
0.8103
0.7403
MNIST
60,000
785
0.8042
0.9960
0.9792
Table 7: Summary of the datasets used in our evaluations.
B.2 Classifiers and Evaluation Metrics
Following the original PATE-GAN paper (Jordon et al., 2019), we use the same 12 predictive models to eval- uate PATE-GAN's utility performance. Eleven them are from the popular Python library scikit-learn (Pe- dregosa et al., 2011); we list them with the names of the algorithms as found in the library in brackets Logistic Regression (LogisticRegression)) Random Forest (Random.ooresttaasfffee Gaussian Naive Bayes (BaussiannBB), Bernoulli Naive Bayes (BernoulliNB), Linear Support Vector Machine (LineaSSVC), De- cision Tree (Decision Tree Clossifis), Linear Discriminant Analysis Classifier ( Adaptive Boosting -AdaBoostClassffeer Bootstrap Aggregating (BaggingClassiifen Gradient Boosting Ma- chine ( Bostting Classifier) Multi-layer Perceptron (MLPClassifer)) The twelfth model is XGBoost
Analysis))
(XGBRegressor) from the library xgboost (Chen & Guestrin, 2016).
As done in (Jordon al., 2019), we report the Area Under the Receiver Operating Characteristic curve (AUROC) and the Area Under the Procision-Recal Curve (AUPRC) scores to quantify their performance the classification task.
Finally, all experiments are run on an AWS instance (m4.4xlarge) with 2.4GHz Intel Xeon E5-2676 v3 (Haswell) processor, 16 vCPUs, and 64GB RAM.
C Additional Experimental Results
In this Appendix, we present the AUROC and AUPRC scores for Kaggle Credit in Table 8-9 and Figure 6-7, and for UCI Epileptic Seizure in Table 10 and Figure 8 9. We also show the effect of number of teachers- discriminators and default hyperparametors in Table and 12, respectively. We discuss them in Scction 5. Additionally, we present further privacy results Figure and 12, which are discussed in Section
Kaggle Credit. On Kaggle Credit, in Setting A (train on real, test on real), we get lower AUROC score (0.8176) compared to both the original paper's (Jordon et al., 2019) Setting A (0.9438) and Setting B (0.8737) scores. This could be due to two reasons: 1) we set aside 20% of the data for testing, while the updated's 's authors do it with around 50% and 2) we fit the data preprocessor (min-max scaling) on the training data only; while they do on the combined training and test.
MNIST. Finally, we evaluate utility on MNIST. Unlike the four datasets studied thus far, MNIST i) balanced, with all labels approximately equally represented, and ii) more complex, requiring high-level correlations typically present in image data to be captured by the generative model. In Figure 10, report the AUROC scores of the six implementations at varying levels.
In contrast to the performance observed on the other datasets, there is distinct difference between original, turing and the remaining inplementations. The first. two achieve significantly higher utility, which is expected, as they do not implement PATE correctly. Consistent with prior observations, synthcity, shows slight improvement with higher c values, while updated, borealis, and smartnoise experience performance drops between and e 10. Overall, none of the six implementations demonstrate promising results on MNIST
20

---

Published in Transactions on Machine Learning Research (02/2025)
(Jordon et al., 2019)
original
updated
synthcity
turing borealis smartnoise
Logistic Regression
0.8728
0.5000
0.5000
0.2997
0.5000
0.8720
0.8158
Random Forest
0.8980
0.5000
0.5000
0.7809
0.5000
0.5076
0.7980
Gaussian Naive Bayes
0.8817
0.5000
0.5000
0.5000
0.5000
0.5000
0.5000
Bernoulli Naive Bayes
0.8968
0.5000
0.5000
0.6392
0.5000
0.5480
0.6003
Linear SVM
0. 0.7523
0.5000
0,5000
0.2599
0.5000
0.8622
0.8168
Decision Tree
0.9011
0.5000
0.5000
0.6068
0.5000
0.5721
0.7013
LDA
0.8510
0.5000
0.5000
0.2596
0.5000
0.5000
0.8192
AdaBoost
0.8952
0.5000
0.5000
0.5588
0.5000
0.5975
0.7749
Bagging
0.8877
0.5000
0.5000
0.7006
0.5000
0.5051
0.7688
GBM
0.8709
0.5000
0.5000
0.5342
0.5000
0.5928
0.6987
MLP
0.8925
0.5000
0,5000
0.3242
0.5000
0.8822
0.8144
XGBoost
0.8904
0.5000
0.5000
0.7426
0.5000
0.6202
0.7213
Average
0.8737
0.5000
0.5000
0.5172
0.5000
0.6300
0.7358
Table 8: Performance comparison of 12 classifiers in Setting B (train on synthetic, test on real) in terms of
AUROC on Kaggle Credit (e = 1). Baseline performance: 0.5000. Setting A (train on real, test on real)
performance: 0.8176.
(Jordon al., 2019)
original updated synthcity turing borealis smartnoise
Logistic Regression
0.3907
0.0017
0.0017
0.0013
0.0017
0.4847
0.2401
Random Forest
0.3157
0.0017
0.0017
0.0493
0.0017
0.0701
0,0370
Gaussian Naive Bayes
0.1858
0.0017
0.0017
0.0017
0.0017
0.0219
0.0017
Bernoulli Naive Bayes
0.2099
0.0017
0.0017
0.0359
0.0017
0.4027
0.1506
Linear SVM
0.4466
0.0017
0.0017
0.0012
0.0017
0.4027
0.2506
Decision Tree
0.3978
0.0017
0.0017
0.0023
0.0017
0.0023
0.0035
LDA
0.1852
0.0017
0.0017
0.0012
0.0017
0.0017
0.2699
AdaBoost
0.4366
0.0017
0.0017
0.1678
0.0017
0.2073
0.0927
Bagging
0.3221
0.0017
0.0017
0.0051
0.0017
0.0119
0.0564
GBM
0.2974
0.0017
0.0017
0.0572
0.0017
0.1866
0.0222
MLP
0.4693
0.0017
0.0017
0.0020
0.0017
0.4453
0.1840
XGBoost
0.3700
0.0017
0.0017
0.0590
0.0017
0.1254
0.0986
Average
0.3351
0.0017
0.0017
0.0320
0.0017
0.1635
0.1174
Table 9: Performance comparison of 12 classifiers in Setting B (train on synthetic, test on real) in terms of
AUPRC on Kaggle Credit (r 1). Baseline performance: 0.0017. Sctting (train on real, test on real)
performance: 0.5475.
0.9
0.8
0.7
AUROC
0.6
0.5
0.4
baseline
original updated
synthcity
borealis
0.3
Paper
turing
smart mnstnoise
10-2
10
100
10
102
epsilon (with delta 10-5)
Figure 6: Performance comparison of 12 classifiers (averaged) in Sctting B (train on synthetic, test on rcal)
in terms of AUROC with various e (with 5 = 10-5) on Kaggle Credit.
21

---

synthcity Published in Transactions on Machine Learning Research (02/2025)
(Jordon et al., 2019) original updated
synthcity
turing borealis smartnoise
Logistic Regression
0.4329
0.4717
0.4800
0.4543
0.4765
0.5152
Random Forest
0.8902
0.5754
0.6157
0.8789
0.6936
0.7572
Gaussian Naive Bayes
0.2568
0.4938
0.8621
0.2000
0.4338
0.5881
Bernoulli Naive Bayes
0.4435
0.2409
0.7970
0.8982
0.2282
0.4415
Linear SVM
0.4872
0.4713
0.4843
0.4922
0.5060
0.5128
Decision Tree
0.4045
0.4067
0.5138
0.4371
0.4304
0.3759
LDA
0.4676
0.4725
0.4835
0.3695
0,4990
0.4861
AdaBoost
0.7766
0.5808
0.4836
0.8035
0.7116
0.6086
Bagging
0.7213
0.5172
0.5548
0.6946
0.6269
0.7092
GBM
0.8563
0.5224
0.4861
0.7030
0.5548
0.6892
MLP
0.4780
0,5739
0.6487
0.4375
0.4867
0.5601
XGBoost
0.8094
0.5442
0.3615
0.6587
0.7136
0.6846
Average
0.6512
0.5854
0.4892
0.5643
0.5856
0.5301
0.5774
Table 10: Performance comparison of 12 classifiers in Setting B (train synthetic, test real) in terms of
AUPRC on UCI Epileptic Seizure (e = 1). Baseline performance: 0.2000. Setting A (train on real, test on real) performance: 0.7403.
baseline
Paper
turing
original
borealis
updated
smartnoise
AUPRC
0.1
0.0
10
10
10
101
102
epsilon (with delta 10-5)
Figure 7: Performance comparison of 12 classifiers (averaged) in Setting B (train synthetic, test real) in terms of AUPRC with various (with 5 = 10-5) Kaggle Credit.
0.8
AUROC
0.6
0.4
baseline
original updated
synthcity
borealis smartnoise
Paper
turing
0.2
10-2
10-1
100
10a
10a
epsilon (with delta 10-5)
Figure 8: Performance comparison of 12 classifiess (averaged) in Setting B (train on synthetic, test real) in terms of mean (not maximum) and standard error AUROC with various (with = 10-5) on UCT Epileptic Seizure.
22

---

Published in Transactions on Machine Learning Research (02/2025)
0.6
AUPRC
0.4
0.2
baseline
original updated
synthcity turing
borealis smartnoise
Paper
10
10
10
10
10
epsilon (with delta 10-5)
Figure 9: Performance comparison of 12 classifiers (averaged) in Setting (train on synthetic, test on real) in terms of AUPRC with various (with 8 10-5) on UCT Epileptic Seizure.
0.7
AUROC
0.6
0.5
baseline
original
synthcity turing
borealis
updated
smartnoise
0.4
10-2
10
epsilon (with delta 10
Performance comparison of 2 classifiers (Logistic Regression and Random Forest, averaged) Setting B (train on synthetic, test real) in terms of AUROC with various (with ó = 10 on MNIST.
Figure 10:
Implementation
N/50
N/100
N/500
N/1,000
N05.000
updated
0.6170
0.5944
0.6077
0.6187
0,6180
synthcity
0.6230
0.6447
0.6643
0.7165
0.6428
borealis
0.6170
0.6230
0.6332
0.6730
0,6363
smartnoise
0.6625
0.6275
0.6653
0.6963
0.6270
Table 11: Tradeoff between number of teachers and performances of 12 classifiers (averaged) in Setting B
(train synthetic, test on real) in terms of AUROC (with < 1 and 10-5) on UCT Epileptic Seizure.
original
updated
synthcity
turing
borealis
smartnoise
Dataset
P
P
P
Kaggle Credit
0.5000
0.5000
0.5000
0.5000
0.5172
0.5437
0.5000
0.5000
0.6300
0.6616
0.7358
0.5867
Kaggle Cervical
0.9265
0.8531
0.8739
0.6581
0.8584
0.6867
0.5000
0.5000
0.9431
0.8140
0.8025
0.7699
UCI ISOLET
0.8342
0.8355
0.5861
0.6001
0.6292
0.6091
0.5000
0.5000
0.6219
0.5980
0.6152
0.6009
UC1 Epileptic
0.6867
0.6876
0.6187
0.6223
0.7165
0.6888
0.7425
0.5000
0.6730
0.6069
0.6963
0.6768
AUROC A
-4.23%
-3.69%%
-17.78%
-25.00%
-00.49%
-24.10%
Table 12: Average AUROC scores of using the default hyperparameters (denoted as D) per implementation
and the hyperparameters specified in the paper (Jordon et al., 2019) (P) over the 12 classifiers (e 1)
well as reduction in AUROC from D to P.
23

---

Published in Transactions on Machine Learning Research (02/2025)
teacher-discrimantor loss on first data subset seen
other teachers-discriantton loss on same data subset
updated
synthcity
borealis
sma smartnoise
0.8
0.8
cross entropy
0.03
0.03
0.6
0.6
0.02
0.02
0.4
0.4
0.01
0.01
0.2
0.2
0.00
0.00
0.0
0.0
100
150
500
1000
100
200
iteration
iteration
iteration
iteration
Figure 11: Cross entropy of the five teachers-discriminator on fixed subset of data (e = 1) over 10 runs.
theoretical epsilon
Querybased
theoretical epsilon
GroundHog
original
original
updated
updated
synthcity
synthcity
turing
turing
borealis
borealis
smartnoise
smartnoise
2
3
Eemp
Comp
(a) Querybased on worst-case datasel
(b) GroundHog on Kaggle Cervical Cancer
Figure 12: DP auditing using Bayesian credible intervals (Zanella- Béguulln et al., 2023) with different black- box MIAs (c 1, as per the dashed red lines).
Responsible Disclosure
As mentioned in Section 7, we sent emails to the authors of original and turing, detailing our privacy concerns, and raised 11 GitHub issues, as detailed Table 13.
Implementation Privacy Violation/Bug GitHub Issue Link
updated updated updated
Data Partition
Teachers
Processing & Metadata
updated
Laplace Noise
synthcity
Data Partition
synthcity
Moments Accountant
synthcity
Teachers
synthcity
Metadata
https: http:://chhbbcoo/////cccccaaa/lr
borealis
Moments Accountant
borealis
Processing & Metadata
/ httppp///sttttuccco////aaaaaaaaa
smartnoise
Moments Accountant
https:/ httpee///sttuubbooo////eo///////
Table 13: Summary of the GitHub issues raised and their corresponding privacy violation bug.
E PATE-GAN Implementations
In Algorithms 2, 3, 4, 5, 6, and we highlight the deviations between the six implementations and the original paper's Algorithm 1 (Jordon et al., 2019).
24

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 2 Pseudo-code of PATE-GAN; original
1: Input: 0, nT, nss batch size n, number of teachers k, noise size À
2: Initialize: 8a, ....... of, fs. (l) = 0 for I..... L
Partition dataset into l subsets D Dx of size [D]
while < do
5: for i2 = ==
RT do Sample 21, On for
i.d. Pz
do
Sample u,...., Up 1..d. DD
8:
Update teacher, Tú, using SGD V Σ log(T:(u,)) + l T((()))
10:
-
11:
end for
12:
end for
13:
for is
= 1,
ns do
Sample Zi,..., Zn i.d.
14:
Pz
15:
for
j l...., n do
16:
û G(z,)
17:
Tj PATEX for j 1,...,n
18:
Update moments accountant
24-X/mg -11
19:
<- eexp A no -m
20:
for
/ 1, 1,....4..
a(1) - all + min (2 1), (1-22))) ge2l))3
21:
22:
end for
23:
Update the student, S, using SGD
24:
Vog Σラ= rj log S(ū,) + (1 (1-17) r3) log(1 S(û,))
25:
end for
Sample 21,..., 2n i.dd Pz
26:
27:
Update the generator, G, using SGD
28:
Vea D> log(1 - SG((,))
8 min (o)+
29:
30:
end for
31: end while
32: Output: G
25

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 3 Pseudo-code of PATE-GAN; updated
1: Input: D, nT, nss batch size n, number of teachers k, noise size À
2: Initialize: 8a, fj..... of, fs. alI) = 0 for = 1....,L L
Partition datasct into is subsets DI.... Dx of size [D]
k
while < do
5: for t2 = =
RT do
Sample 21, On i.d. Pz
for ,....,k do
Sample u1,..., Up 1..d. D Update teacher, Tú, using SGD
8:
10:
V
% E log(T:(u,)) + (T
11:
end for
12:
end for
13:
for is
1, Rs do
Sample Bi,..., Zn i.d.
14:
Pz
15:
for j l...., n do
16:
û G(z,)
17:
Tj PATEX (Q.) for j ,.
18:
Update moments accountant
2+ 2+
19:
<- ex -
20:
for / 1,...,,.
al1) - al) + ( lo, - -9) ( (1-22,) ) + ge2
21:
22:
end for
23:
Update the student, S, using SGD
24:
Vo. Σラ= rj log S(ū,) + (1 r3) log(1 S(û,))
25:
end for
Sample 21,..., 2n i.d. Pz
26:
27:
Update the generator, G, using SGD
28:
Vea D> log(1 S(((z))
(o)))) 29: t min 1
30:
end for
31: end while
32: Output: G
26

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 4 Pseudo-code of PATE-GAN; synthcity
1: Input: 6, nT, nss batch size n, number of teachers k, noise size À 2: Initialize: 8a, ....... of, fs. al) = 0 for I = 1,...,L L
Partition dataset into k subsets Di....y Dx of size [D]
while < do
5: for to = == RT do
Sample 21, On for
i.d. Pz
do
Sample u,...., Up 1..d. DD Update teacher, Tú, using SGD Vog Σ log(T)(u,)) + -og(11 ((((()))
8:
10:
11:
end for
12:
end for
13:
for is
= 1, ns do
Sample Bi,..., Zn i.d.
14:
Pz
15:
for j l...., n do
16:
û G(z,)
17:
Tj PATEX (Q.) for j I.
18:
Update moments accountant
2+ 2+
19:
<- ex no -
20:
for / 1,...,,.
al1) - al) + ( lo, - 9) (1-22,) ) g
21:
22:
end for
23:
Update the student, S, using SGD
24:
Vo. Σラ= rj log S(ū,) + (1 r3) log(1 S(û,))
25:
end for
Sample 21,..., 2n i.dd Pz
26:
27:
Update the generator, G, using SGD
28:
Vea D> log(1 S(((z))
(o)))))( 29: t min 1
30:
end for
31: end while
32: Output: G
27

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 5 Pseudo-code of PATE-GAN; turing
1: Input: D, nT, nss batch size n, number of teachers k, noise size À
2: Initialize: 8a, ......, of, fs. al) = 0 for I....,I.
Partition dataset into l subsets D Dx of size [D]
while < é do
5: for t2 = = RT do
Sample 21, On for
i.d. Pz
do
Sample u,...., Up 1.1d. DD
8:
Update teacher, Tú, using SGD VØg Σ log(T:(u,)) + log(l-
10:
- (G((()))
11:
end for
12:
end for
13:
for is
= 1,
ns do
Sample Zi,..., Zn i.d.
14:
Pz
15:
for
j l...., n do
16:
û G(z,)
17:
rj PATE, for j ,,...,n
18:
Update moments accountant
24-X/mg -
19:
x- ex A no -m
20:
for
| 1, 1,..,,,...
1), (1-22)) ge201)3
21:
a(1) - + min (
22:
end for
23:
Update the student, S, using SGD
24:
Vog Σラ= rj log S(ū,) + (1 (1-17) r3) log(1 S(û,))
25:
end for
Sample 21,..., 2n i.d. Pz
26:
27:
Update the generator, G, using SGD
28:
Vea D> log(1 - SG((,))
8 min (o)
29:
30:
end for
31: end while
32: Output: G
28

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 6 Pseudo-code of PATE-GAN; borealis
1: Input: 6,D,n7 nss batch size n, number of teachers k, noise size À
2: Initialize: fG, op.... f7, fs, a(I) = for 1 1,.... L
[2] Partition dataset into & subsets D1, Dj of size k
while < do
5: for t2 = L,... RT do
Sample 21, On i.d. Pz
for
do
Sample u,,..., Lid. D:
8:
9:
Update teacher, Tú, using SGD
VoL E log((T(u,)) + log(1 - (((())))))
10:
11:
end for
12: end for
13: for is l...., ns do
Sample Zj,..., Zn i.d.
14:
Pz
15:
for j l...., n do
16:
Új G(z,)
17:
Tj + PATEx for j ,,...,
18:
Update moments accountant
2+ 2+A/na--d
19:
Texp(A Ro -11
20:
for 1 1,...,11. do
al1) - all + min 1), 1og((1 -g) (1722)) ) 20200
21:
22:
end for
23:
Update the student, S, using SGD
24:
Vog Σラ= ry log S(ū,) + - ry) log(1 S(ū)))
25:
end for
Sample 2,,..., 2n iid. Pz
26:
27:
Update the generator, G, using SGD
28:
Vog (Ei log(1 S(G(z)))
é min +)
29:
30:
end for
31: end while
32: Output: G
29

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 7 Pseudo-code of PATE-GAN; smartnoise
1: Input: ò, NT, nls, batch size n, number of teachers k, noise size À
2: Initialize: fG, ....., f7, fs, a(I) = for 1 1,.... L
[2] Partition dataset into & subsets DI, Dj of size k
while < do
5: for t2 = L,... RT do
Sample 21, On i.d. Pz
for
do
Sample u,,..., Lid. D:
8:
9:
Update teacher, Tú, using SGD
VoL E log((T(u,)) + log(1 - (((())))))
10:
11:
end for
12: end for
13: for is l...., ns do
Sample Zi...., Zn i.d.
14:
Pz
15:
for j l...., n do
16:
Új G(z,)
17:
Tj + PATEx for j ,,...,
18:
Update moments accountant
2+ 2+A/na--d
19:
Texp(A Ro -11
20:
for 1 1,...,11. do
al1) - all + min 1), 1og((1 -g) (1-22)) 2022
21:
22:
end for
23:
Update the student, S, using SGD
24:
Vog Σラ= ry log S(ū,) + - ry) log(1 S(ū)))
25:
end for
Sample 2,,..., 2n iid. Pz
26:
27:
Update the generator, G, using SGD
28:
Vog (Ei log(1 S(G(z)))
é min +)
29:
30:
end for
31: end while
32: Output: G
30


Reviews:
[{"review_id": "IZrQqXHyak", "reviewer": "Reviewer_ztgz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- This work discover numerous privacy violations across all six implementations of PATE-GAN\n- The authors did a good job in auditing the implementations using MI attacks\n- They showed that the empirical epsilon is greater than the theoretical epsilon\n\n\n***Weaknesses:***\nThere is one major weakness with this study.\n\nWhile I am not entirely convinced, (see requested changes section), the basic experiments of reproducing the results of the paper as is, is missing. Specifically, what would have been the results when the code of the six implementations are run according to the original paper, including the hyperparameters?\n\nTo buttress that, since several implementation use different hyperparameters as shown in Table 3, surely, the results will differ. Hence, the need for a uniform assessment across all implementations using the same hyperparameter.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work provides a benchmark evaluation of six popular implementations of PATE-GAN [1]. The authors showed that the all six implementation leaks more privacy than intended ($\\epsilon_{emp} >> \\epsilon $). The goal of teh study is to reproduce the utility performance reported in the original paper, and using membership inference as auditor to estimate PATE-GAN’s privacy guarantees. Among other findings, they identified 17 privacy violations and 5 other bugs based on how PATE is implemented. \n\n\n[1] James Jordon, Jinsung Yoon, and Mihaela Van Der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In ICLR, 2019.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "My main worry is that I see the author's making exactly the same mistakes that the previous implementations made. Specifically, I would love to see the following analysis to justify the claim of this paper.\n1. Can the authors use exactly the same datasets, framework version e.g Tensoflow / Pytorch and python version, and report the results?\n2. While I am in favor of this paper provided 1 above is verified, rather than releasing the code, I would like to see a docker file. Such that the experiments can be reproduced.\n3. The claims of the paper are really concerning and interesting. For instance, \"We find that original and turing do not actually implement the PATE framework or a moments accountant\".  As also pointed out by the authors, \"This is a serious deviation that can compromise privacy\". I am not surprised the bugs are not fixed after contacting the authors, but I am interested in the response of the authors after you pointed out the errors.\n4. I don't understand the rationale behind using different hyperparameters for the same work.  \"Also, all implementations have different network depths and noise dimensions, mostly depending on the input data.\" Why would different authors have different implementations of the same work, especially in terms of the hyperparameters? I find this strange.\n\n5. One fundamental problem between the original paper [1] and your reimplementations is that there is no standard deviation reported. Could it be that the identified discrepancies is within the standard deviation? For instance, I find the results of original and borealis not to be entirely far apart.\n\n6. Rather than the claim that there are discrepancies in the results. As pointed out by the authors in \"Data Support and Processing\", the data processing differs across different implementations. So shouldn't the data processing be one / most important reason for the discrepancies? I will assume that each of the other implementations adapts the original implementation for fair comparison. If this is the case, I will expect some discrepancies as adaptation of an original code could lead to different result.\n\n7. Can the authors provide more details on the MI attacks (Querybased and GroundHog). How was it used. How many shadow models were used? etc. This saves the time in reading each of the papers.\n\n\n[1] James Jordon, Jinsung Yoon, and Mihaela Van Der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In ICLR, 2019.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ffv6AsmRmO", "reviewer": "Reviewer_jP3A", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1) The paper presents a detailed study of PATE-GAN implementations with respect to utility and privacy.\n\n2) The use of privacy auditing techniques, including membership inference attacks (MIA), demonstrates substantial privacy risks in these implementations.\n\n3) The paper  identifies the privacy violations and provides insights into the bugs and implementation errors that result to them.\n\n4) The open soursing of the code facilitates future research.\n\nWeaknesses:\n\nSome parts of the paper are not explained properly and are not clear. Please see the Requested Changes section for more details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper evaluates the reproducibility, privacy, and utility claims of PATE-GAN, by benchmarking six available PATE-GAN implementations against the resutls reported at the original paper. The contributions are as follows:\n\n1) The paper benchmarks six PATE-GAN implementations to reproduce utility metrics such as AUROC scores, demonstrating that none reproduce the original model's performance and often underperform.\n\n2) The paper presents an in-depth privacy audit using Differential Privacy (DP) techniques and discover significant privacy leaks in all six implementations.\n\n3) The paper identifies a series of privacy and implementation bugs, including 17 privacy violations and various issues affecting the DP guarantees.\n\n4) The code is open sourced to the research community.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) Could you please report the experimental results for figures 1 and 3 over multiple runs? what is the mean and standard deviation?\n\n2) It might be beneficial to add a section that addresses the ethical implications of using the DP models with the privacy violation in real-world applications, especially in applications such as healthcare or finance.\n\n3) Table 1 reports the reduction in AUROC from the original paper (Jordon et al., 2019). The drop is significant (from 42% to 80%). Are these results by running their code (even with the privacy bugs)? How can you explain this drop?\n\n4) Please mention what is the orange dotted line in Figure 3. Is it the same as the orange solid line?\n\n5) How the number of MIA and the number of collection datapoints was selected (at page 10 of the paper) ? \n\n6) Would it be possible to perform the DP auditing with different black-box MIAs (Figure 5) on the rest of the datasets (apart from the Kaggle Cervical Cancer ) ? What happens in the case when the theoretical epsilon is greater than 1? Do we observe privacy violation? by what margin?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "1UZu6VKVUW", "reviewer": "Reviewer_Geuw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This is a very timely paper showing the problems with existing DP approaches to PATE-GAN. The scale of the issue becomes even more striking, given that these frameworks were proposed by many different author groups, all suffering from different issues, but ultimately passing the associated risks to the end users whose data can be misused at the end. Therefore I can safely say that this work is of importance and of interest to the community (and not just the privacy community).\n\nI found no fundamental issues in this work, but have made a few additional requests in the comments below in order to improve the privacy bound estimation and show how these models perform in more realistic deep learning scenarios. Given that MIA results are already bad enough, I find the choice of attacks to be decent too and do not have any additional requests here. In general I find the work to be sound in the choice of techniques and methods.\n\nOne potential weakness for me is that I am still uncertain as to why is the loss of performance so drastic compared to the ones reported in the original papers? Is this purely because of the bugs or am I missing something? Because I find the 50+% to be extremely worrying and would like to clarify if this performance difference is to do with software issues, rather than misreported results in the original papers which were evaluated.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work empirically evaluates a number of publicly available implementations of PATE-GAN implementations. The work evaluates both the performance claims made in the respective works as well as an empirical privacy audit of the frameworks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "While I understand that these were the datasets described in the works making for a fair comparison, I would like to see how well (or how badly) the privacy-utility and the empirical epsilon estimation behave in settings which are a) more balanced with respect to the labels and b) use more complex underlying architectures and c) an additional epsilon. \n\nOne of the recent auditing frameworks introduced by Zanella-Beguelin, 2023 (cited in this work) significantly improves on the reported privacy bounds compared to the Clopper-Pearson approach, so it would be advisable to use the method proposed in this work for epsilon bound estimation.", "broader_impact_concerns": "Already covered in the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Many of the concerns by the reviewers have been addressed during the review process such running the experiments of the original paper (comment by reviewer ztgz) and considering more balanced datasets for experiments (comment by reviewer Geuw). 

I think the paper does great job in revealing inconsistencies in the results of previous PATE-GAN papers and in finding bugs in the existing implementations. Although there were some concerns from the reviewer ztgz about how solid the experiments are, I think they are very valuable and as requested, the authors have run the experiments of the original paper as well to check the methods' utilities. Similarly to the reviewer Geuw, I am flagging the Reproducibility Certification. The code for the comparisons can be found in the supplements and I sincerely hope the authors will release a code repository as they mention in the abstract. I also hope you can address the following questions.

A question: What are exactly the privacy violations in 'smartnoise' and 'borealies' that you mention? If I understood correctly, there are those $q$-dependent terms missing in the moments accountant (Alg. 6 and 7 in your paper). But leaving those terms out would only give overestimates of the epsilons, and thus you'd get valid DP bounds if everything else is correct? I just don't understand where do the privacy violations mentioned in Table 2 come from.

Related questions and comments: Did you use versions of 'smartnoise' and 'borealies' where you modified the moments accountant? If not, then what do you think, how is it possible that the empirical epsilons are larger than the theoretical ones since there do not seem to be other obvious bugs? If you used the modified moments accountant, then did you check that all the conditions of the underlying privacy analysis are met? It seems to me that the original work by [Jordron et al.](https://openreview.net/pdf?id=S1zk9iRqF7) is overlooking, e.g., the condition on $q$ that is stated in the original PATE paper by [Papernot et al.](https://arxiv.org/pdf/1610.05755). Also, looking at the original PATE analysis, should the Laplace noise parameter $\lambda$ in the moments accountant of PATE-GAN be replaced by $\lambda^{-1}$? I am mentioning these since I think it would be nice to get understanding on why the empirical epsilons are so large, especially for 'smartnoise' and 'borealies'. In any case this is a nice illustration of the usefulness of privacy auditing.

I still would recommend the authors to go over the writing carefully which was also commented by the reviewer jP3A. In particular:

- I think it would be good to add details about the PATE method, like what is the meaning of the parameter $\lambda$, how is the Laplace noise scaled with it? (seems to be different scalings in PATE-GAN and PATE papers)

- I could not find the meaning of the variables $n_0$ and $n_1$ used in the pseudocodes, I had to look up from the original paper. Please add those and desribe all the other variables as well in the pseudocodes ($L$ etc.).
Decision: Accept (poster)