Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: In the paper the generator is trained against the best discriminator. Would it also make sense to train the generator against the worst discriminator early on in some circumstances? For example, consider we have two discriminators: a linear one and a powerful deep network. Early on, the generator produces very different samples from real data. The deep model will perfectly classify fake vs. real, and training signals/derivatives would be close to 0. The linear model on the other hand may not be able to exactly classify fake from real, but it will still do something meaningful. Moving along the derivative of the linear classifier will roughly try to align the means of real and fake data.

The point I'm trying to address here is that if you have an ensemble of differently complex discriminators, you may not always want to follow the best one... Could the authors comment on this observation?
review_title: Training against the best discriminator

final_score: rating: 6: Marginally above acceptance threshold
review_content: This work brings multiple discriminators into GAN. From the result, multiple discriminators is useful for stabilizing. 

The main problem of stabilizing seems is from gradient signal from discriminator, the authors motivation is using multiple discriminators to reduce this effect.

I think this work indicates the direction is promising, however I think the authors may consider to add more result vs approach which enforce discriminator gradient, such as GAN with DAE (Improving Generative Adversarial Networks with Denoising Feature Matching), to show advantages of multiple discriminators.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: In this interesting paper the authors explore the idea of using an ensemble of multiple discriminators in generative adversarial network training. This comes with a number of benefits, mainly being able to use less powerful discriminators which may provide better training signal to the generator early on in training when strong discriminators might overpower the generator.

My main comment is about the way the paper is presented. The caption of Figure 1. and Section 3.1 suggests using the best discriminator by taking the maximum over the performance of individual ensemble members. This does not appear to be the best thing to do because we are just bound to get a training signal that is stricter than any of the individual members of the ensemble. Then the rest of the paper explores relaxing the maximum and considers various averaging techniques to obtain a ’soft-discriminator’. To me, this idea is far more appealing, and the results seem to support this, too. Skimming the paper it seems as if the authors mainly advocated always using the strongest discriminator, evidenced by my premature pre-review question earlier.

Overall, I think this paper is a valuable contribution, and I think the idea of multiple discriminators is an interesting direction to pursue.
review_title: Review

final_score: rating: null
review_content: 1. How is the max value computed? Is it approximated with a finite number of samples?

2. In Section 3.1's second paragraph, the "prime" functions (D', V') don't appear to be mentioned anywhere else. What are they?
review_title: Section 3.1 questions

final_score: rating: 7: Good paper, accept
review_content: The paper extends the GAN framework to accommodate multiple discriminators. The authors motivate this from two points of view:

(1) Having multiple discriminators tackle the task is equivalent to optimizing the value function using random restarts, which can potentially help optimization given the nonconvexity of the value function.

(2) Having multiple discriminators can help overcome the optimization problems arising when a discriminator is too harsh a critic. A generator receiving signal from multiple discriminators is less likely to be receiving poor gradient signal from all discriminators.

The paper's main idea looks straightforward to implement in practice and makes for a good addition to the GAN training toolbelt.

I am not very convinced by the GAM (and by extension the GMAM) evaluation metric. Without evidence that the GAN game is converging (even approximately), it is hard to make the case that the discriminators tell something meaningful about the generators with respect to the data distribution. In particular, it does not inform on mode coverage or probability mass misallocation.

The learning curves (Figure 3) look more convincing to me: they provide good evidence that increasing the number of discriminators has a stabilizing effect on the learning dynamics. However, it seems like this figure along with Figure 4 also show that the unmodified generator objective is more stable even with only one discriminator. In that case, is it even necessary to have more than one discriminator to train the generator using an unmodified objective?

Overall, I think the ideas presented in this paper show good potential, but I would like to see an extended analysis in the line of Figures 3 and 4 for more datasets before I think it is ready for publication.

UPDATE: The rating has been revised to a 7 following discussion with the authors.
review_title: Interesting ideas, needs more empirical results.

final_score: rating: null
review_content: This is interesting work which train multiple discriminators simultaneously. It is known that if discriminator is too strong generator will be hard to converge. Why to choose to train against the best discriminator? 
review_title: Pre-review Question

Meta-review: Using an ensemble in the discriminator portion of a GAN is a sensible idea, and it is well explored and described in this paper. Further clarification and exploration of how the multiple discriminators are combined (max versus averaging versus weighted averaging) would be good. The results are fairly strong, across a variety of datasets.
Decision: Accept (Poster)