Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a simple strategy to improve DP-GANs by using more update steps for discriminator, adopting a step scheduler, and taking a larger batch size. The authors provide empirical findings to justify such improvement by linking generation quality with the discriminator accuracy though the training process. Experiments on MNIST and FashionMNIST show very promising results in terms of generation quality.
strength_and_weaknesses: Strength:

1. The proposed techniques are very effective to achieve much better FID on MNIST and FashionMNIST.

2. The empirical explanation of balancing the discriminator and maximizing generator steps taken when discriminator accuracy is high, is interesting and inspirational.

Weaknesses:

1. Overall, I think the proposed method is currently limited to an engineering technique, which might not be mature enough. However, I do believe it is highly potential to develop a principled method with high impact along the current direction. For example, I would recommend further elaborate on the step scheduler to be more adaptive and include more ablation studies. The authors might be able to draw some inspirations from StyleGAN2-ADA for such paradigm.

2. The improvement of utility accuracy is not as significant as FID. It would also be better to include an experiment on some larger datasets such as CelebA.

3. The writing and the organization could be improved.
- Although fancy, the title is not very informative.
- Figure 1 and figure 2 share the same purpose.
- Section 3.3 is duplicate to the previous content.
- Section 5.1 and 5.2 are not well aligned with the theme of Section 5 and the results are not well linked to the discriminator accuracy.
- In Algorithm 1, some symbols are used without definition.
- In Table 1, "(This work)" is ambiguous.
clarity,_quality,_novelty_and_reproducibility: The clarity and quality are good in general with some issues to be improved. 
This paper is novel in terms of empirical findings, while the proposed method is currently limited to an engineering technique.
The reproducibility is good with sufficient implementation details provided.
summary_of_the_review: This paper has interesting empirical findings and promising results, while at current stage the proposed solution is not mature enough to meet the standard of ICLR. However, along its direction, this work is potential to have an impact in the future, probably with an elaborated method or some theoretical support.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper provides two empirical findings on how to train differential private gans: larger batch size, and more discriminator steps. Experimental results show superior performance against existing baselines.
strength_and_weaknesses: Strength

1. In the literature of differential privacy gans, it seems like the utilization of more recent architectures and training tricks (e.g. StyleGAN2,3, …) for higher quality samples are mostly unexplored. This paper provides a good point that these “tricks” could largely boost the performance of private gans.

Weakness

1. From my perspective, this work is with limited novelty. Since larger batch size and take more discriminator steps has been studied widely in normal gan papers. It seems like natural attempts to try these tricks on private gans.
2. Several missing details. For example, how does (10, 10^-5)-DP calculated from B=128,sigma=1,C=1,T=450000 (in section 3.1)? Please provide the formal equation on calculating this. In addition, noise scales, n_D, and batch size also affect privacy, in section 5.1, how can you targeting the same \epsilon with different values of these?
clarity,_quality,_novelty_and_reproducibility: Some details are missing, code is not provided.
summary_of_the_review: Overall, this paper is a practical guide for training private gans. However the findings are empirical and rather trivial. In addition, some details are missing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors revisit the DPGAN paper and state that the added noise to discriminator training disrupts the balance between the generator and discriminator training. They show that by tuning the number of update steps taking by the discriminator for every generator update step, specifically, taking more steps significantly improves the results. They show that for MNIST \at epsilon = 10 their private GAN FID goes from 48.4  to 13.0 and the downstream accuracy of the classifier goes up from 83.2% to 95.0%. 
strength_and_weaknesses: ### Strength
The paper provides a simple yet effective method to improve the quality of the generated data of DPGAN. The experiments show significant improvement in the tested benchmarks. The authors discuss in length about whey more steps are needed in the case of DPGAN. In addition, the authors discuss the role of the batch size in DPGAN settings and the additional benefits of scheduling the discriminator frequency update (start low then increase). 

### Weaknesses
The novelty of the work is marginal. The method work best at \epsilon = 10, but on high privacy/ low \epsilon regime sometime the results are even worse.
clarity,_quality,_novelty_and_reproducibility: The paper is written in a clear and easy to follow fashion, the work might be important to the community although the novelty of the work is not substantial. The work could be easily be reproduce.
summary_of_the_review: The paper present a very slight modification, a focus on a parameter change in the DPGAN method. It propose to increase the discriminator update frequency when noise is introduced to the discriminator (as in the DPGAN training case). The paper discuss the motivation to their method at length. Finally, the authors show inconclusive experiments, where the method works for some regime of the noise level.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies strategies to train differential private gans, i.e., larger batch size and more steps to update the discriminator. 

All reviewers recognize the strength of this paper is its simple yet effective way to improve the quality of the generated data of DPGAN. But they also share the same major concern on the limited novelty of this paper. 
justification_for_why_not_higher_score: All reviewers thought the novelty of this paper is limited, as both the larger batch size and steps of tuning the discriminator are natural strategies in the literature. The overall approach is quite heuristic and lacks in-depth theoretical discussions.
Decision: Reject