Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper proposes a method of improving the generated samples of differential-private synthetic dataset using GANs by boosting them post training. They support their proposed method using theory, and then empirically show that it works on 3 types of machine learning tasks.

This paper presents a novel way of utilizing the sequence of generators and discriminators during training as they are already part of the privacy budget. So it significantly improves the quality of GAN-generated samples for different experiments under the same privacy budget.

The experiments provide evidence of the utility of the proposed method in all three tasks. The community can definitely benefit from this paper.
review_title: Sound theory, good results

final_score: rating: 7: Good paper, accept
review_content: This paper proposes an algorithm to process the sequence of generators and discriminators produced by any differentially private GAN training, in order to produce synthetic data of better quality. The paper also presents some empirical evaluations. 
There have been a few works on differentially private GAN, yet the noised required by differential privacy usually causes significant degradation in utility. So an algorithm for improving the results can be quite valuable. The evaluation was mainly on simple datasets, which is understandable as differentially private GAN can be quite hard. However, I think it might be helpful to do more experiments on more difficult datasets (like those usually used to evaluate non-private GAN) with various epsilons (even with very large epsilon), so that readers and future researchers can understand the limitations of the current sota.

The presentation is clear. The privacy aspect of the algorithm, e.g. which part needs privacy protection and what is the sensitivity of the score function, can be elaborated more.

review_title: Good progress in differentially private GAN

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This paper studies the differential private synthetic dataset generation. Unlike previous DP based GAN models, this paper aims to boost the sample quality of after the training stage. In particular, the final synthetic dataset is sampled from the sequence of generators obtained during GAN training. The distribution is obtained by a private two-player game between the privately selected discriminator and a sampler from the mixture of generators. The results are demonstrated on gaussian data and tabular data.


Pros: 1. The sample quality of private generative models is known to be not as good as non-private models. This paper provides a practical private post gan boosting algorithm to improve the sample quality. 

Cons:1. My main concern is on experiments. It is known that private generative models have bad sample quality on image data. Prior works on private synthetic generation papers usually show results on MNIST. It would be better if the authors could compare private PGB on MNIST dataset. 

2. It would be better to have an ablation study of the proposed PGB and discriminator rejection sampling. For example, in Figure~1, the baselines for both non-private gan and private gan are too bad. I am wondering whether the gain is from rejection sampling or the proposed PGB algorithm.


Questions:

1. I am curious about how to split epsilon for gan training and the post gan boosting. Are there any principled reasons for the split?

2. How do you calculate the sensitivity for the exponential mechanism?
review_title: A qualitative comparison with private generative models on MNIST is needed

Meta-review: This paper provides a privacy-preserving method to boost the sample quality after training a GAN. The reviewers were unanimous that this paper should be presented at ICLR, with an important contribution to privacy-preserving GANs.
Decision: Accept (Poster)