Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, authors proposed a new training strategy which transfer the StyleGAN2-ada to the conditional version gradually by injecting conditional information into the generator and the objective function during the training phase. The proposed method is capable of training on limited data and generating high-quality images. However, the strategy of this method is only applicable to StyleGAN-like architecture and experiments can be improved.
main_review: Strengths:
- The writing of this paper is easy to follow. 
- The idea is novel and intuitive. The authors propose a new training strategy to transfer stylegan2-ada from an unconditional to a conditional approach. And in the results, the best FID was obtained.

Weaknesses & Questions: 
- The reason why this strategy only requires limited data has not been analyzed. Is it the stylegan-ada that reduces the need for data? If so, the "ada" strategy is not a contribution of this paper. As the authors claim: " The proposed method for training cGANs with limited data results not only in stable training…" . More experiments should be conducted to verify that the proposed method can reduce the need for data. For example, applying the proposed method to stylegan or stylegan2 and training on the same data.
- Are other architectures supported? The results provided are only implemented on stylegan-ada. However, these statements in Sections 1 and 2 may lead the reader to believe that the proposed approach is a general approach that can be applied to different cGANs architectures. Therefore, in addition to the results of stylegan2-ada, implementations on different architectures should be provided. For example, BigGAN, ContraGAN, etc.
- More qualitative results are compared. Tables 1 to 4 and Figure 2 provide quantitative results at different settings, however, providing more qualitative results at these settings can help to more visually assess the enhancements from the method.

Minors: 
- Details of L^D_uc, etc. should be specified, and are there additional hyper-parameters for each term?
- For better illustration, Ts and Te can be added to Figure 1 (first row), although the ranges of Ts and Te can be easily guessed.
- In Figures 1 and 2, "our" outperforms the unconditional version in the FID. Why is the Fid further reduced compared to the unconditional version?


summary_of_the_review: This paper provides an intuitive solution to move stylegan2-ada from an unconditional to a conditional approach and achieve the best performance on the FID metric, however more experiments, as listed above, should be given to help a better evaluation of all aspects of this work.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This work studies the problem of training class-conditional GANs in limited data settings. The authors first empirically demonstrate that class-conditioning results in mode collapse in a limited data regime whereas unconditional learning provided satisfactory generative ability. They perform a comprehensive analysis by gradually reducing the size of the training set in two ways, a) reducing the number of classes while keeping a number of 100 training images per class and b) reducing the number of images per class while using 50 classes in all cases. In both cases, the conditional GAN achieves a better FID for larger datasets while performance deteriorates significantly (experiencing mode collapse) when the dataset size is reduced. On the other hand, the unconditional model achieves consistently better FID in limited data settings. Based on this observation, the authors propose a method of injecting the class conditioning by transitioning from unconditional to the conditional case, in an incremental manner. They delineate the proposed architectural changes and training objectives. Authors base their experiments on StyleGAN2 with adaptive data augmentation (ADA) and four datasets. The empirical results suggest significant improvements compared to state-of-the-art methods and established baselines. The authors also perform a comprehensive ablation analysis to understand the contribution of different components.
main_review: Strengths:

[S1] The authors provide interesting observations of cGANs vs unconditional GANs in limited data settings which have not been observed or reported in previous works.

[S2] Comprehensive experiments are conducted to demonstrate the claim that class-conditioning causes mode collapse in limited data settings, whereas unconditional learning leads to satisfactory generative ability.

[S3] The authors propose a simple yet effective method for training cGANs that effectively prevents the observed mode-collapse and results in significant empirical improvements compared to state-of-the-art methods and established baselines.

[S4] The authors also perform a comprehensive ablation analysis over different components of their method.


Weaknesses:

[W1] The authors incorporate their method only in StyleGAN2 with ADA. While I understand that it is a recent state-of-the-art method for unconditional and class conditional image generation under a limited-data setup, it would be great if authors can verify the efficacy of their approach when applied to different models like BigGAN, etc.

[W2] The authors do not compare with some of the datasets that a recent method uses - Regularizing Generative Adversarial Networks under Limited Data (Tseng et al. 2021). Tseng et al. also perform experiments with GANs where they whittle down training data and they primarily conduct experiments with CIFAR10 and CIFAR100. The authors do not show their observations and experiments on C10 and C100. It would be nice to have results on these two datasets.

[W3 minor] The technical novelty of the paper is not much. Similar strategies of transitioning from one objective to include another objective can be seen often. For example, training a model using a pre-text task (contrastive learning, solving jigsaw, etc.) and then finetuning for the target task. Authors follow a similar approach where they kind of pretrain using unconditional learning and later include conditional learning. Having said that, the observations made as well as the application of this simple trick to mitigate mode collapse in cGANs are quite novel (empirically).

summary_of_the_review: The authors discover an interesting behavior of cGANs in limited data settings and propose a simple yet very effective method to solve the problem. The authors perform a comprehensive analysis of the problem followed by a comprehensive set of experiments and ablation analysis showing the efficacy of their method. Overall, the paper seems to make important observations and empirical contributions.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work address the limited training data issue for class-conditional GANs. Inspired by the observation that unconditional GANs produce more diverse results under the same setting, the authors propose a learning algorithm that transits the training of unconditional toward class-conditional settings. The authors report FID and KID scores on four datasets to verify the proposed approach.
main_review: Strength:
1. The observation is interesting. Indeed, the mode collapse problem happens more frequently in conditional GANs (including other conditional GANs such as image-to-image, tex-to-image generation).
2. The paper is well-written and easy to follow. The high-level idea and the technical details behind are well-explained. The modification that changes the concatenation to addition operation in the generator makes sense and enables the transition idea.

Concerns:
1. The transition design for the loss is unclear. 1) The goal is to transit the training from unconditional to conditional settings, 2) the "no transition" results in Table 2 shows that the combined loss harms the performance. I am wondering if the authors have tried the transition that the loss function in Equation 3 is set to $L = (1 - \lambda)L_{uc} + \lambda L_{c}$. In this case, the weight for the unconditional loss is zero after the transition.
2. How do the authors determine the subset of classes and images to be used for the experiments?
3. How do the authors compute the FID and KID scores? Are these scores the average of the per-class ones? Per-class FID/KID scores can verify if the proposed method (unconditional training in the beginning) makes the generator to produce images that does not correspond to the input class label.
4. Since the authors claim to address the mode collapse issue under limited data setting for conditional GANs. The authors can consider to use the precision and recall scores to better measure the diversity of the generated images, especially the recall score.
Kynkäänniemi et al, "Improved Precision and Recall Metric for Assessing Generative Models," NeurIPS 2019.
5. What is the total training iteration used for all experiments? According to the paper, $T_s$ is set to 2K while $T_e$ is set to 4K. Is there any case that the best model used for computing the score is from the iteration before the transition ends?
6. Does the proposed harm the performance if $T_e$ is set to be close to the final training iteration? What are the results of setting $T_e$ to 6K or 7K in Table 4?

Suggestions (not related to rating):
1. The paper is easy to follow for people familiar with the conditional setting in StyleGAN2. However, the conditional setting is not well-introduced in the original paper which may confuses people when they read this paper. The authors can consider to briefly introduce how conditional generation is achieved in StyleGAN2 (both generator and loss sides) before describing the proposed method.
2. The authors can consider to use "clean FID" score due to the robustness.
Parmar et al., "On Buggy Resizing Libraries and Surprising Subtleties in FID Calculation."
summary_of_the_review: Overall, this is a good paper that addresses the limited data issue for conditional GANs. However, I have two major concerns: the transition design for the loss, and evaluation metrics. Please see Weakness section in the main review.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors work towards training conditional GANs with limited data. Based on the observation that conditional GAN training suffers worse mode collapse than unconditional training, the authors proposed a training strategy that gradually injects conditional information into unconditional training. In other words, a learning curriculum that gradually replaces unconditional GANs training (losses and structure) with conditional ones is proposed.

Overall, the proposed learning curriculum is reasonable and achieved promising performance on data-efficient cGANs training.
main_review: Strengths:
This paper is easy to follow and understand. In my opinion, the authors' observation and proposed training strategy are reasonable. Experiments on four datasets demonstrated the superiority of the proposed strategy. 

Weaknesses:
1) In the paper, the authors claim that "the class-conditioning training is more robust to data volume reduction" is an intuitive belief. Based on such intuition, one contribution of this paper is pointing that cGANs are easier to suffer from mode collapse. However, I think this discovery may be widely known by the GAN community. Usually, a stronger condition would lead to worse diversity. For example, conditioning on semantic segmentations would largely decrease the generative diversity.

2) Although the solution/proposed strategy is simple and useful, some principles are not discussed or explored very well. According to experiments, unconditional training seems to correspond to better diversity yet worse quality, conditional training is the opposite. The proposed training seems to find a balance between them. However, is there always the best balance point and how can we arrive there? I believe that function $\lambda_t$ will largely influence training. For example, if we train cGANs with a large ratio and a relatively long time (e.g., increase $T_e$), will the trained generator collapse again? 
summary_of_the_review: Overall, the authors work on an interesting topic and present promising results.  In my opinion, the solution seems reasonable, yet there are still many problems that should be further explored. I highly suggest the authors further improve this work. However, I may not champion the current version.
review_title: null

Meta-review: This paper examines conditional GANs, which are found to lead to model collapse in low data settings. The paper proposes what appears to be a simple but effective method that addresses the issue. Reviewers were generally happy with the experiments and the utility of the observations and analysis. Code for the method was provided during the author response period. Only one reviewer did not vote to accept this paper, but they did acknowledge that the authors had addressed their concerns during the discussions with the authors. All others rated the paper as an accept.
The AC recommends accepting this paper.
Decision: Accept (Poster)