Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a simple yet effective way to mitigate the mode collapse issue of GANs. To this end, the authors propose the utilization of multiple discriminators that specialize on different subsets of the distribution. This process is implemented using multiple choice learning.
strengths_and_weaknesses: Strengths:

 A. The main idea is simple and clearly presented.

 B. The experimental results highlight the efficacy of the method in the terms of a number of metrics.


Weaknesses: 

A. In order to understand the significance of the proposed method, I’m missing a comparison with other clustering-based approaches, e.g.,

[1]. ClusterGAN : Latent Space Clustering in Generative Adversarial Networks

[2]. Diverse Image Generation via Self-Conditioned GANs


B.  I would like to see some more metrics that are specific to the diversity of the generated samples. e.g., IvOM [3],  Pairwise Distance [3].

[3]. Unrolled generative adversarial networks 


C. I would like to see how the approach performs on some mode-coverage benchmarks, e.g.,

 - Fashion-MNIST and partial MNIST of [4]

 - Stacked MNIST of  [5]

[4]. Rethinking Generative Mode Coverage: A Pointwise Guaranteed Approach

[5]. VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning

questions: I suggest that the authors address the issues raised in "weaknesses" section.
limitations: The authors have included an ethical considerations part, however I would also like to see a discussion regarding the limitations of the proposed method.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a generative adversarial network with multiple discriminators for GAN training. The proposed method guides each discriminator to have expertise in the subset of the entire data. For each sample, only k experts out of M discriminators are used. To address the over-confident problem, this paper proposes another loss, named non-expert training loss. To balance the assignment of discriminators, this paper proposes another loss, named balance loss. The proposed method is evaluated on several tasks, including unconditional image generation, image-to-image translation, and text-to-image synthesis. The results show that the proposed method outperforms the baselines.
strengths_and_weaknesses: Strengths:
1. The idea of using multiple discriminators is sensible. 
2. The proposed method is evaluated on various tasks, including unconditional image generation, image-to-image translation, and text-to-image synthesis. 
3. The experiments show that the proposed method outperforms GMAN. 

Weaknesses:
1. The proposed method is an extension of GMAN. The difference is that the proposed method selects part of the multiple discriminators. In my opinion, this extension is incremental. 
2. The extension (i.e., using part of the multiple discriminators) also introduces two problems, i.e. over-confident problem and balance problem. Although the authors propose two specific losses to tackle the above problems, both the over-confident problem and balance problem seem difficult to tune in practice. 
3. The advantages of the proposed method over GMAN are not clearly explained. This paper stated that “While GMAN focuses on the loss to the generator with parallel learning of discriminators, our strategy specializes each discriminator for more informative feedbacks to the generator”. This is not convincing to me. 
4. The resolutions of images used in the experiments are 32x32, 64x64, or 128x128. I think the effectiveness of the proposed method should be verified on larger datasets, such as FFHQ with 1024x1024. 

questions: Please refer to Weaknesses.
limitations: In practice, it may be difficult to tune the non-expert training loss and balance loss.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a new GAN framework named MCL-GAN. MCL-GAN comprises a single generator and multiple discriminators and follows a multiple-choice learning (MCL) strategy to forward signals of samples to the generator. By conveying the plentiful signals to the generator, the authors insist the generator can generate diverse samples, which means MCL is effective in overcoming the chronic mode-collapse problem. By performing image generation experiments using standard benchmark datasets, the authors demonstrate that the proposed MCL-GAN helps in increasing the diversity of generated images.
strengths_and_weaknesses: Strengths

(+) The motivation of MCL-GAN is reasonable. Also, MCL-GAN is simple, so I think researchers can implement MCL-GAN easily. 

(+) MCL-GAN does not require too many parameters to be burdensome. Multiple discriminators share their parameters, and all we need is stacking multiple fully connected layers parallelly upon the shared network.

(+) Toy experiments are appropriately designed and successfully support the main pillar of the paper. 

(+) Experiments using benchmark datasets demonstrate that the proposed MCL training is effective, and MCL-GAN can generate more diverse images than a baseline. 

Weaknesses

(-) Limited novelty. Although experimental results show the efficacy of MCL-GAN, I think the technical novelty of MCL-GAN is limited. I think introducing MCL into some frameworks is not new to the community. 

(-) I am not sure whether the proposed MCL-GAN can be scaled up for high-resolution image generation. This paper does not contain any experiment using high-resolution image datasets, such as ImageNet, AFHQ, and FFHQ.

(-) Using the old version of precision and recall [1] seems improper for measuring the diversity of generated images. I recommend the authors use the improved version of precision and recall [2] to quantify the diversity. The paper [2] has shown that the old precision and recall sometimes cannot capture the diversity well. The authors can compare their model with previous methods using StudioGAN library [3].

[1] Sajjadi, M.S., Bachem, O., Lucic, M., Bousquet, O., & Gelly, S. (2018). Assessing Generative Models via Precision and Recall. NeurIPS.

[2] Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., & Aila, T. (2019). Improved Precision and Recall Metric for Assessing Generative Models. ArXiv, abs/1904.06991.

[3] https://github.com/POSTECH-CVLab/PyTorch-StudioGAN.
questions: No
limitations: I understand that the authors are cautious about discussing the shortcomings of their work. So, I think it does not matter whether the authors provide the limitations of MCL-GAN or not. Instead, I highly recommend the authors write more about the negative societal impact of their work in the supplementary material.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a multi-discriminator generative adversarial network to address the issue of mode collapse. The discriminators are tied together through parameter sharing which helps reduce the training time.

strengths_and_weaknesses: Strength:

1.	Aim to solve the problem of mode collapse which is of interest to the community.

2.	The number of discriminators is determined adaptively.

3.	The proposed model is generic and can be applied to various variants of GANS as there is no architecture or loss specific constraints.

Weakness:

1.	Since the proposed model leverages multiple specialized discriminators, I suspect that the generator might overfit the training data. I request the authors to provide visualization of nearest neighbours (from the training split) of some randomly generated samples. They should also compute and report (1) Pixel Memorization Score (PMS) and (2) Inception Memorization Score (IMS) as proposed in the following paper.
https://proceedings.mlr.press/v161/mondal21a.html

2.	Although the authors report various metrics, there has been several drawbacks identified with the precision/recall metric. I suggest the authors to compute and compare density/coverage score instead as proposed in:
http://proceedings.mlr.press/v119/naeem20a/naeem20a.pdf


questions: My major concern is whether the proposed method is overfitting on the training set due to several specialized discriminators. Please see and address point 1 in weakness above.
limitations: The authors have discussed the limitations of their work and potential negative societal impact. According to the authors:
"we believe that the proposed approach can alleviate the bias and fairness issues by identifying the minority groups in a dataset effectively."

How does the proposed method identify the minority groups in a dataset is not very clear to me.
review_title: null

Meta-review: metareview: The paper considers a GAN setting with multiple discriminators, a topic that has been studied quite a few times before. It proposes a novel approach based on multiple choice learning in order to specialize discriminators to certain modes of the data distribution, with the aim of mitigating collapse. They demonstrate this clustering effect qualitatively, and present quantitative results on a number of tasks. Parameter-sharing keeps the computational burden in check, while the number of discriminators is determined adaptively.

Reviewers generally found the problem of interest, and the method simple, reasonably motivated, not burdensome in the additional hyperparameters, and generally applicable. A common complaint was the lack of high resolution results. While it is important not to unduly limit acceptance of papers wherein the authors are operating under a constrained compute budget, I believe that at this stage the request for some megapixel was reasonable, and of scientific interest as techniques which show promise at smaller scales often show little at larger scales. 

The authors did provide these results in rebuttal, along with a great number of other requested comparisons and revisions that I feel improve the empirical rigour of the work. As of the end of the discussion period, two reviewers (oqkx, FAZc) lean reject; FAZc did not respond to, or acknowledge, the rebuttal, in which it appears to me their concerns were addressed, and therefore I am discounting their score in my evaluation. oqkx's concerns can be summarized as a) incrementality; b) speculation on tuning difficulty; c) unconvinced by the explanation of advantages over GMAN; d) scale of experiments. The rebuttal seems to have addressed all of oqkx's concerns except a).

I don't believe that concerns around incrementality of the approach justify rejection in this instance. The method is quite clearly differently motivated from GMAN, the appeal to multiple choice learning is to my knowledge novel and appears sensible, the empirical results (which the authors have gone the extra mile to improve during the review period) are extensive and convincing. Despite the borderline scores, I tentatively recommend acceptance.
confidence: Less certain
Decision: Accept