Title: Generative Multi-Adversarial Networks
Authors: Ishan Durugkar
Ian Gemp
Sridhar Mahadevan
Venue: ICLR 2017
Keywords: Deep learning
Unsupervised Learning
Games

Abstract: Generative adversarial networks (GANs) are a framework for producing a generative model by way of a two-player minimax game.  In this paper, we propose the \emph{Generative Multi-Adversarial Network} (GMAN), a framework that extends GANs to multiple discriminators. In previous work, the successful training of GANs requires modifying the minimax objective to accelerate training early on. In contrast, GMAN can be reliably trained with the original, untampered objective. We explore a number of design perspectives with the discriminator role ranging from formidable adversary to forgiving teacher.  Image generation tasks comparing the proposed framework to standard GANs demonstrate GMAN produces higher quality samples in a fraction of the iterations when measured by a pairwise GAM-type metric.

TL;DR: GANs with multiple discriminators accelerate training to more robust performance.

Review discussion:
1. How is the max value computed? Is it approximated with a finite number of samples?
2. In Section 3.1's second paragraph, the "prime" functions (D', V') don't appear to be mentioned anywhere else. What are they?
1. The max is over the N players and is taken at each time step.
For example, at time t, each discriminator receives a loss, V_i(t), after assigning probabilities to the samples generated by the generator and those from the real dataset.  Let's say N=3 and the losses are V_1(t),V_2(t),V_3(t) = -2,-1,-3.  We then compute the max: \max\{ V_1(t),V_2(t),V_3(t) \} = \max\{ -2,-1,-3 \} = -1.  This means we backpropagate the loss of discriminator_2 to update the generator weights.
You can think of the standard GAN as following this same procedure, but computing the max over a single discriminator, which trivially returns the single discriminator: \max{ V_1(t) \} = V_1(t).
2. The primes are meant to distinguish between two instantiations of the discriminator and it's associated loss.  We will make this more clear.
The point we were trying to make is imagine you use a standard single-discriminator GAN with the discriminator and generator initialized with weights w_d and w_g respectively.  Assume we are NOT using dropout, so the training is entirely deterministic.  Let's run training to convergence and record the loss of the discriminator at each time step t.  Call this loss V'_1(t) as in the paper.
Now, let's use GMAN with N discriminators.  Initialize the generator weights to w_g and discriminator_1's weights to w_d just as in the previous experiment.  The N-1 other discriminators can be initialized randomly.  We still are NOT using dropout.  Let's run training to convergence and record the loss of each discriminator at time step t.  Call these losses V_1(t), ..., V_N(t) as in the paper.
The only difference between these two training runs is that the second run includes N-1 other discriminators.  The point that we are trying to make is that the mere existence of the N-1 discriminators changes the dynamics of the game because of the complex interactions between all the players.  This means that V_1(t) does not equal V'_1(t).
In fact, it's hard to make any claims on how much V_1(t) and V'_1(t) differ, therefore, it's hard to make any claims on how much \max\{ V_1(t), ..., V_N(t) \} and \max\{ V'_1(t) \} = V'_1(t) differ *(the max is over the players, not over time).
This argument is simply to say that, unfortunately, we CANNOT claim that taking the max over N discriminators returns a strictly higher loss than using a single discriminator.  Maybe this is obvious and we've only confused readers more, but we thought it was an important point to make.
In the paper the generator is trained against the best discriminator. Would it also make sense to train the generator against the worst discriminator early on in some circumstances?
For example, consider we have two discriminators: a linear one and a powerful deep network. Early on, the generator produces very different samples from real data. The deep model will perfectly classify fake vs. real, and training signals/derivatives would be close to 0. The linear model on the other hand may not be able to exactly classify fake from real, but it will still do something meaningful. Moving along the derivative of the linear classifier will roughly try to align the means of real and fake data.
The point I'm trying to address here is that if you have an ensemble of differently complex discriminators, you may not always want to follow the best one... Could the authors comment on this observation?
Thank you for your comment.  To be clear, Sections 3.1 and 3.2 discuss training against the best discriminator, however, the rest of the paper largely focuses on relaxing this assumption and exploring the exact train of thought you outline above.
1) Section 3.3 asks the question, "Is max too harsh a critic?".  Section 3.3.1 discusses replacing the max with a softmax.  3.3.2 discusses the effect that the arithmetic mean has on early training.  As you state in your example above, the training signals received from the deep model would be nearly zero, however, those of the linear classifier, while maybe not as accurate, might still be helpful to training.  The arithmetic mean returns (0+linear_signal)/2, allowing training signals to flow to the generator.  3.3.3 discusses this intuition from a probability density perspective.
2) We also explore the harmonic (HM, eqn5) and geometric means (GM, eqn4) in addition to the arithmetic (AM, eqn3).  It is known that min <= HM <= GM <= AM <= max.  This means that HM encourages training against the weakest discriminator (relative to the arithmetic mean).
However, we found that HM and GM did not perform as well as AM in our experiments.
This is interesting work which train multiple discriminators simultaneously. It is known that if discriminator is too strong generator will be hard to converge. Why to choose to train against the best discriminator?
Thank you for your comment. We explore both extremes through the perspective of multiple adversaries: training against the best discriminator vs worst (or at least more lenient).  Each extreme focuses on a different issue.
Training against the best discriminator focuses on optimality of the GAN solution.
1) GAN Solution Optimality: The original GAN proof was formulated around training against the best discriminator.  This means that we don't have any optimality guarantees when training against a suboptimal discriminator.
In practice, we never have access to the optimal discriminator, so we must train without these guarantees, however training against a better discriminator should bring us closer to the case where we DO have optimality guarantees.
Training against a more lenient discriminator focuses on GAN convergence to equilibrium.
2) GAN Convergence to Equilibrium: As you say, it is difficult to train a generator against a strong discriminator.
For this reason and others, in Section 3.3, we explore training against a more lenient discriminator (or ensemble).
In this interesting paper the authors explore the idea of using an ensemble of multiple discriminators in generative adversarial network training. This comes with a number of benefits, mainly being able to use less powerful discriminators which may provide better training signal to the generator early on in training when strong discriminators might overpower the generator.
My main comment is about the way the paper is presented. The caption of Figure 1. and Section 3.1 suggests using the best discriminator by taking the maximum over the performance of individual ensemble members. This does not appear to be the best thing to do because we are just bound to get a training signal that is stricter than any of the individual members of the ensemble.
Then the rest of the paper explores relaxing the maximum and considers various averaging techniques to obtain a ’soft-discriminator’. To me, this idea is far more appealing, and the results seem to support this, too. Skimming the paper it seems as if the authors mainly advocated always using the strongest discriminator, evidenced by my premature pre-review question earlier.
Overall, I think this paper is a valuable contribution, and I think the idea of multiple discriminators is an interesting direction to pursue.
Thank you for the review and the feedback!
Figure 1 was meant to depict the idea of multiple Discriminators providing feedback to the Generator, with the caption giving an example by specifying F:= max in section 3.1.
We will modify the caption so that the complete idea is more evident and less misleading.
We have also modified the layout of the paper to avoid the misunderstanding that was pointed out. We will be uploading the same soon.
The paper extends the GAN framework to accommodate multiple discriminators. The authors motivate this from two points of view:
(1) Having multiple discriminators tackle the task is equivalent to optimizing the value function using random restarts, which can potentially help optimization given the nonconvexity of the value function.
(2) Having multiple discriminators can help overcome the optimization problems arising when a discriminator is too harsh a critic. A generator receiving signal from multiple discriminators is less likely to be receiving poor gradient signal from all discriminators.
The paper's main idea looks straightforward to implement in practice and makes for a good addition to the GAN training toolbelt.
I am not very convinced by the GAM (and by extension the GMAM) evaluation metric. Without evidence that the GAN game is converging (even approximately), it is hard to make the case that the discriminators tell something meaningful about the generators with respect to the data distribution.
In particular, it does not inform on mode coverage or probability mass misallocation.
The learning curves (Figure 3) look more convincing to me: they provide good evidence that increasing the number of discriminators has a stabilizing effect on the learning dynamics.
However, it seems like this figure along with Figure 4 also show that the unmodified generator objective is more stable even with only one discriminator. In that case, is it even necessary to have more than one discriminator to train the generator using an unmodified objective?
Overall, I think the ideas presented in this paper show good potential, but I would like to see an extended analysis in the line of Figures 3 and 4 for more datasets before I think it is ready for publication.
UPDATE: The rating has been revised to a 7 following discussion with the authors.
Thank you for the review and the insights!
We agree that the GMAM metric is not an absolute score for evaluating GANs.
However, we do feel that it is an appropriate method to compare GAN models against each other. It is also an unsupervised method to evaluate GANs that are not limited to the domain of images. More detailed results for CIFAR and MNIST using the GMAM metric are included in the Appendix to show this. We have also included Inception scores for the CIFAR dataset in the Appendix to show that the GMAM metric compares well with how the score evaluates Generated images.
Figure 3 and Figure 4 plot the dynamics of the different models while learning. With multiple discriminators it is evident that variance in the Generator objective is reduced and the convergence to equilibrium is achieved up to twice as fast.
Further, the using the original unmodified objective with a single discriminator leads to saturation early on in the training process with the generator not getting meaningful signals. Using multiple discriminators mitigates this issue as well.
Figures 10 and 11 in the Appendix plot the same dynamics as Figures 3 and 4 on the CelebA dataset. Here training with the original unmodified objective and a single discriminator was not possible due to the saturation problem, while with multiple discriminators we could use the original objective.
For more empirical results, we have shall include plots similar to figures 3 and 4 (and 10, 11) for the CIFAR dataset as well. We shall also include results on one more dataset soon.
Please check the Appendix (SPECIFICALLY A.1 AND A.2) for additional results!
Thank you for the clarifications.
I remain unconvinced by the authors' arguments regarding the GMAM evaluation metric.
Even when taken as a relative evaluation score rather than an absolute one, the issues I raised still stand: in absence of evidence that the GAN game is converging, it is hard to make the case that the discriminator provides a good measure (even relative) of anything, including mode coverage and probability mass misallocation.
I am a bit puzzled by the added training curves for CIFAR10 (Figures 12 and 13).  Figure 12's caption claims that increasing the number of discriminators reduces variance, but at the same time the curves in Figure 13 overlap significantly, indicating that the standard deviations integrated over a 500 time steps window are comparable; isn't this contradictory?
Overall, I don't feel like the issues I raised in my review have been sufficiently well addressed and I stand by my original rating.
Thank you for reviewing our new figures.
With regards to the GMAM metric, we are not sure what you mean by convergence of the GAN game.  If you are referring to convergence of the weights of the nets, then true, we have not shown convergence.
However, if you're referring to the convergence of the minimax objective, we've clearly shown that in Figures 3, 10, and 12.
With respect to the latter, the game has converged and learning has reached equilibrium at which point it seems reasonable to compare different models against each other.
That all being said, the GMAM metric is not the focus of the paper - it is only one of the ways we evaluated our models.  We also evaluate the quality of the learned models using Inception scores and illustrative generated images as well as supplementary evidence of the learned model's quality.
With regards to Figures 12 and 13, no, these plots are not contradictory.  Figure 13 is produced by the following algorithm:
For run=1:5:
Measure standard deviation of the minimax objective over a sliding window of width 500 iterations
Plot mean of these 5 measurements
Therefore, low values indicate low standard deviation over a window which implies steady-state.
In short, Figure 12 displays standard deviation over runs (with the fill plot), whereas Figure 13 displays standard deviation over time.
By "convergence of the GAN game", I mean that the discriminator converges to the optimal D*(x) = q(x) / (q(x) + p(x)) at which the value function becomes proportional to the JS-divergence between p(x) and q(x).
I do agree however that there is other supporting evidence in addition to the GMAM metric that points to the additional discriminators having a beneficial effect on the learning dynamics.
Thank you for clarifying the relationship between Figures 12 and 13. I think other readers may benefit from this as well; I am willing to recommend acceptance provided you clarify the relationship between the two plots in the paper.
Thank you for your feedback! We changed the captions to the relevant figures (3,4,10,11,12,13) to emphasize the difference between the two plots.  We also made the distinction clearer in Section 5.2.1.  Please let us know if more clarification is required.
And thank you for clarifying what you meant by "convergence of the GAN game".  We understand your point.
Thank you. I updated my rating.
This work brings multiple discriminators into GAN. From the result, multiple discriminators is useful for stabilizing.
The main problem of stabilizing seems is from gradient signal from discriminator, the authors motivation is using multiple discriminators to reduce this effect.
I think this work indicates the direction is promising, however I think the authors may consider to add more result vs approach which enforce discriminator gradient, such as GAN with DAE (Improving Generative Adversarial Networks with Denoising Feature Matching), to show advantages of multiple discriminators.
Thank you for your comments. We did not compare against GAN with DAE because it is a parallel submission in this conference.
However, that paper contains Inception scores for their model on CIFAR-10 which can be directly compared against our model (SEE TABLE 6 IN APPENDIX A.3).

Meta-review:
role: PC
data: comment: Using an ensemble in the discriminator portion of a GAN is a sensible idea, and it is well explored and described in this paper. Further clarification and exploration of how the multiple discriminators are combined (max versus averaging versus weighted averaging) would be good. The results are fairly strong, across a variety of datasets.

Decision: Accept (Poster)