Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a novel approach to training GANs with limited data, one that is motivated by the gap in the norms of the gradients between the target and generated images. They demonstrate applications of the proposed regularizer on various BigGAN architectures with and without data augmentation. 
strengths_and_weaknesses: **Originality and Significance:** Limited data (LD) training with GANs has been gaining popularity in the recent years, and the proposed regularization approach to training shows potential. The fact that it can be used in parallel with other schemes for LD training such as DiffAug is an advantage. The gradient gap in the discriminator is generally well analyzed and adequate  experimental validations are provided. 

**Presentation and Clarity:** The paper is clear to read and the flow is consistent. 
 - The synthetic experiments with Vanilla GAN consider a simplistic scenario — a similar experiment in 2D might have been more insightful, as the effect of different norm on the gradient vector (L1, L2), is clearer at least in 2D. Nevertheless, the given experiments help to give a clear intuition on the proposed method. 
 - The notation is confusing at times. For example, The additional parentheses when time indexing time $t$ seems unnecessary, but this is just a personal opinion.
 - Some proof read would help, as there are a few typos here and there. L233 L146 Forth -> Fourth. L262 improvement to 32.56 -> improvement **by** 32.58, just to name a few. 

**Literature Survey:** Most of the relevant literature has been cited and the paper includes relevant discussions on related works. 
questions:  - **Loss Formulation:** I have a fundamental concern with the form of the loss $\left(  \left\| \frac{\partial D}{\partial x_R} \right\|  - \left\| \frac{\partial D}{\partial x_F} \right\|\right)$. It is unclear how the samples are drawn. If $x_R$ and $x_F$ are drawn randomly in pair, then, the particular paining of the real and fake images will affect how this loss performs. While this issue might not be visible in the synthetic 1D experiment present, on the image scale, this might have effect. For example, are we making sure the pairs are drawn from the same class. In class-conditional GANs, it might be possible, bit, in unconditional variants, this cannot be guaranteed, and there is a strong possibility that the manifold nature of each class might itself affect how these terms interact. 
 - To address the above, maybe the authors could consider some ablation study where, given the choice of the sample batch, how permutations of the real and fake samples affects the penalty?
 - Alternatively, have the authors considered batch-level sample averages of the gradient norms? This would essentially bring the terms closed to the R1 and R2 penalties. 
limitations: —
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a new method for improving gan training under limited data. The key idea is to regularize the gradient norm of discriminator between real and fake samples. Authors have done empirical studies showing that this regularization can help avoid bad attractors within the GAN loss landscape. Authors have applied the proposed regularization on BigGAN, leading to improvement across multiple datasets.
strengths_and_weaknesses: Strengths:
+ The manuscript is easy to follow
+ The empirical studies provided by the authors are important to understand the effectiveness of proposed regularization

Weaknesses:
- Missing related work, and incorrect interpretation of related work. Specifically, [b] ([17] in manuscript) is not doing data augmentation in my understanding. And [a] is also related but is missed in the manuscript. Since both [a] and [b] are not doing data augmentation, I believe they should be compared as well.

[a] Data-Efficient Instance Generation from Instance Discrimination
[b] Deceive d: Adaptive pseudo augmentation for gan training with limited data.

- while at L30 authors claim "data augmentation can enhance the results, but the benefit is limited with insufficient data (Tab. 3)." In table 3 authors didn't evaluate sufficient data augmentation methods to support this claim.

- in the experiments, authors only compared to DiffAug and R_LC, which is insufficient. Authors should also include ADA, etc.

- while previous works targeting gan training with limited data mainly evaluate their methods on StyleGAN and unconditional image synthesis, authors mainly evaluate the proposed method on BigGAN and conditional image synthesis. It's better to stay consistent with previous methods to ensure a fair comparison. Or at least expand table 4.

-  It's great that authors provide empirical studies in Sec.3.3. It's better if authors include a theoretical analysis. 





questions: Overall I think the idea proposed in this manuscript looks interesting and effective to some extent. But I think the experiments are insufficient to validate the effectiveness of the proposed method. Detailed questions are covered in paper weaknesses.
limitations: Authors didn't include the discussion of limitations in the main manuscript. I think authors can briefly discuss in what cases the proposed method may fail or lead to marginal improvement.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces a new data-efficient training technique for GAN. The technique named Discriminator gradIent Gap (DIG) regularization aims to equalize the changes of discriminator’s judgments w.r.t its inputs (real and fake data) so that the discriminator can effectively learn representations for adversarial training without an imbalance issue. Experimental results demonstrate that GAN with the proposed DIG regularization exhibits better image generation results than the previous techniques (LeCam regularization and DiffAug).
strengths_and_weaknesses: Strengths

[S1] I think the proposed method is new to the GAN community. 

[S2] The empirical finding that the gap between the norms of the gradients of discriminator’s predictions increases when the GAN is trained with fewer data, seems to be very useful for future development of data-efficient training methods. 

[S3] The experiments in the paper show that the proposed DIG regularization is helpful for GAN training in data-hungry situations. To demonstrate the effectiveness of DIG regularization, the authors utilize various datasets (CIFAR10, CIFAR100, Tiny ImageNet, and CUB200), which means the proposed method can be used for real-world applications. 

[S4] The paper is well written and easy to follow. 

Weaknesses

[W1] I think the theoretical explanation of the relation between DIG and the training failure is needed. Although experimental results show that applying DIG regularization can resolve the problem of getting stuck in local attractors, I am not sure why this is possible by imposing DIG regularization.

[W2] FID scores in this paper do not match the results of previous papers [1, 2]. The FIDs of BigGAN on CIFAR10 in the papers [1, 2] are 8.57 and 8.08, respectively, while the FID of BigGAN in this paper is 10.53 (Table 1). Performance discrepancies occur for experiments across the paper, which significantly lowers the credibility of the paper.

[W3] Experiments using high-resolution images are missing. Verifying the usefulness of DIG regularization using MetFaces, AFHQv2, or FFHQ will make the paper more convincing.

[W4] Comparision with Adaptive Discriminator Augmentation (ADA) is missing. Since ADA is a standard method to train GAN with limited data, I think comparing DIG regularization with ADA is necessary. 

[1] Chen, T., Cheng, Y., Gan, Z., Liu, J., & Wang, Z. (2021). Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly. ArXiv, abs/2103.00397.
https://openreview.net/forum?id=BBVcs78PEDA

[2] Kang, M., Shim, W., Cho, M., & Park, J. (2021). Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training. NeurIPS.
questions: [Q1] Which gradient (gradient of a discriminator's prediction w.r.t real or fake sample) makes the DIG value large? I guess the norm of the gradient w.r.t real sample might be very large, as the discriminator tends to memorize training samples and predicts samples other than the training samples to be fake regardless of the realism of samples.
limitations: I cannot find any section that explains the limitations of the paper. Also, the described negative societal impact is not sufficient. I hope the authors will add more portions to tell the pros and cons of the paper and its societal impact.
review_title: null

Meta-review: metareview: The paper proposes a regularizer for limited-data GAN training. All three reviewers thought the experiments were adequate to demonstrate the method's usefulness and the writing was clear. The paper's biggest weakness seems to be unconvincing conceptual intutition and lack of theoretical justification (pointed out by reviewers Mjjy and Z5nH). This is a borderline paper but I recommend acceptance.
confidence: Less certain
Decision: Accept