Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The submission tackles the problem of training GANs with limited data.
It proposes to do so by adaptively regularizing GAN discriminators by replacing some real data by generated samples when the discriminator overfits, accoding to an overfitting heuristic.

A study of the proposed objectives show that the method keep the classical GAN optima while alleviating overfitting of the discriminator.

The method is tested on various datasets with limited data, and compared, using a StyleGAN-2 backbone, against other state-of-the-art methods for training with limited data.
main_review: 
Originality:
It might be worth mentioning that feeding the discriminator with generated data with the wrong label (label flipping) was a fairly well-known trick in 2016, discussed for example at the NeurIPS 2016 GAN Workshop [1, 2]. Connexions [3] had been drawn with label smoothing [4] and instance noise [5]. Using adaptive heuristics is also discussed in the same workshop presentation, and had been found unpractical at the time.
Of course, the proposed work is no less original, as modern GAN training dynamics have little to do with early implementations, they did not consider training in the low-data regime, and in any case, a detailed, peer-reviewed, report on label-flipping was missing. Still, discussing those connexions might be interesting.

Quality: 
The submission is technically strong. The proposed heuristic seems very ad hoc, but the method is demonstrated to be effective in a variety of settings, including when combined with other methods. Experiments are thorough, with multiple datasets and numbers of samples.
It is however tested only with the StyleGAN-2. As it is perhaps the most popular GAN variant, the study is still very interesting and relevant but could be even stronger if tried with BigGAN as well.

Clarity:
The submission is clear and easy to understand.

Significance:
The tackled problem is important (training with limited data) and the proposed method is a simple and very effective method that is likely to be of use to the community.

----------
[1] How to train a GAN?
Soumith Chintala. NIPS 2016 Workshop on Adversarial Training
https://sites.google.com/site/nips2016adversarial/

[2] soumith. https://github.com/soumith/ganhacks#6-use-soft-and-noisy-labels

[3] Instance Noise: A trick for stabilising GAN training
Ferenc Huszár. inFERENCe.
https://www.inference.vc/instance-noise-a-trick-for-stabilising-gan-training/

[4] Improved Techniques for Training GANs.
Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen. NeurIPS 2016.

[5] Amortised MAP Inference for Image Super-resolution.
 Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi, Ferenc Huszár. ICLR 2017.
limitations_and_societal_impact: One limitation that is not discussed is about the possible inadequacy of a theoretical analysis at optimality to draw conclusions about the effectiveness of the regularisation during training. This approach, however, is fairly common in works about GANs.

Otherwise, limitations and societal impacts are adequately discussed.

--------------------------------------------------------
Post-rebuttal:
I'd like to thank the authors for providing the additional clarification and additional results in their rebuttal.
After reading the different reviews and responses, I keep my positive score. The method is simple and effective, backed with strong empirical evidence.
I also concur with reviewer Kz3j's suggestion to clearly state and discuss the relation of the proposed method with related techniques (label smoothing, instance noise...), as well as the proximity between the objectives of training in low-data regime and reducing GAN instability.
In that regard, the results provided in the response to reviewer Kz3j and the discussions with reviewer GodP are already very interesting additions.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper tries to solve discriminator overfitting problem.

The authors propose adaptive pseudo augmentation (APA).
* APA employs the generator itself to augment the real data distribution with fake images.
  * i.e., Fake images are presented as "real" instances to the discriminator.
* Adaptiveness comes from measuring overfittingness of the discriminator.
* APA has similar theoretical properties to the original GAN, but with $(1-\alpha)p_\text{data} + \alpha p_g = p_g$.

APA improves systnesis quality in the limited-data regime.
* on FFHQ, AFHQ-Cat, CUB, Danbooru
The authors provide a theoretical analysis of APA, similar to the original GAN.

main_review: Originality.
* (+) This paper proposes a novel way to augment the real distribution by adding fake samples.
  * Existing methods transforms images to augment the real and/or fake distributions.

Quality.
* (+) APA has sound theoretical groundings.
* (+) APA improves StyleGAN2 in all cases (number of real samples).
* (+) APA is roughly robust to choice of hyperparameters.
* (-) Analyses are provided only on FFHQ 7k. Showing the similar trend on 1k and 5k would be better.
* (-) Benchmark datasets are not as diverse as ADA.
* (-) ADA is compared only on AFHQ-Cat, FFHQ-5k on FFHQ-70k.

Clarity.
* (+) Everything is clear to understand.

Significance.
* (+) APA is generally effective on different settings, even with existing standard augmentation methods.
* (+) APA brings negligible computational cost.
* (+) APA is simple.
limitations_and_societal_impact: Yes to both.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a new method for GAN training with limited data, named adaptive pseudo augmentation (APA). The key idea originates from the empirical observation that, when trained with limited data, the GAN discriminator tends to overfit easily, leading to poor generation performance. Therefore, the authors propose to adaptively add generated samples to the training data, so as to counteract the overfitting during training. Accordingly, they present the APA that is developed based on the output of the discriminator. Empirical experiments demonstrate the effectiveness of the proposed APA.
main_review: Originality. The presented APA is believed new and sound. More importantly, APA is quite easy to implement in practice. Related work is adequately cited and discussed. However, in most of the experiments, APA is only compared with StyleGAN2 that is not tailored for a generation with limited data. The ADA results should be reported, for example in Table 2, to highlight the advantages of the presented APA.

Quality. The presented APA is sound and seems to be a simple but powerful technique. The claims are well supported empirically. However, theoretically, it would be better if the authors discuss why applying APA leads to better performance than the vanilla GAN? After all, the optimum is the same.

Clarity. The paper is clearly written and quite easy to follow.

Significance. The presented APA is important for GAN training with limited data; it's likely to be reused by other researchers.

Other comments:

In the experiments, why did you use Lanczos filter to do the preprocessing? Please elaborate on it.

*********************************
After rebuttal
*********************************
Thanks for the detailed responses. Please address the concerns in the comments and revise the paper accordingly. 

limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper introduces a data augmentation strategy for training GANs, which proves to be especially effective in the low data regime. The essence of the idea is to replace real images by images produced by the generator, using an adaptive replacement probability in the discriminator training step. By classifying some fake images as real, discriminator overfitting is prevented, which normally leads to very poor training performance and instability when very little data is available. The proposed method (named APA) improves over or can be combined with another recent data augmentation scheme for the low data regime (such as ADA).
main_review: Strengths:

-	The proposed method (APA) is simple and can be easily applied to other data modalities, besides images, in contrast to ADA which requires image-specific transformations. At the same time, it performs better most of the time and can be applied in combination with ADA. 
-	The proposed approach is quite effective and helps to stabilize the training under limited training data.
-	The paper is sound and easy to read.


Weaknesses:

-  The core idea of the proposed method is adding label noise to the discriminator (switching the label of fake images to real), which is a well-known trick used for improving training stability of GANs [https://github.com/soumith/ganhacks]. One way was to flip the labels of the real images, adding pseudo-labels to the fake images, or to flip the labels of both fake and real with the fixed ratio. In the proposed approach, the labels are flipped for the fake images only. It’d be beneficial to see if the comparison can be provided between one-sided flipping techniques (adding pseudo-labels to either real or fake images) and a two-sided label flipping (adding pseudo-labels to both real or fake images). Also the paper should state clear that adding label noise is a well known technique to improve the stability of the GAN training.

-	There are also other techniques that can “cripple” discriminator, such as one-sided label smoothing [37], instance noise [39] or progressive augmentation of the discriminator [PA, Zhang et al., NeurIPS’2019]. As the proposed APA approach is quite close to those techniques, I think the comparison with them is required. 

-	The clarity of  Section 3.1 (Method) can be improved: the description of the adjustment of t is a little vague. Take the sentence “If lambda signifies too much/little overfitting regarding t, the probability p will be increased/decreased by one fixed step.”. In particular, how much is too much overfitting regarding t? What exactly is the step size? It seems like the step size is chosen such that p increases to 1 after exactly 500K images or it is increased in every single step? 

-	It would be beneficial to see how the APA method performs under low amount of training data, when the number samples is less than 1k images. 

-	Table 3: What happens if you combine ADA and APA on the full dataset?

-	In Section 4.3, it would be helpful to see an ablation on the hyper-parameter t. How was threshold t=0.6 chosen?


limitations_and_societal_impact: yes, the limitations are addressed in the paper
review_title: null

Meta-review: This paper proposes a very simple method of adaptively feeding generated instances into real data to prevent the overfitting of GAN Discriminator in the situation where the training data is limited. It is somewhat incremental in terms of novelty, considering existing label-noise techniques, and the algorithm in deciding the feeding probability is also somewhat heuristic. The theory also shows a trivial extension from original GAN paper. Nevertheless, all reviewers, including this AC, unanimously supported the acceptance of this paper because of its simplicity and strong empirical success. In the final version, please be sure to add comparison/discussion with the related technique mentioned by reviewers.
Decision: Accept (Poster)