Title: Generative Adversarial Parallelization
Authors: Daniel Jiwoong Im
He Ma
Chris Dongjoo Kim
Graham Taylor
Venue: ICLR 2017
Keywords: Unsupervised Learning

Abstract: Generative Adversarial Networks (GAN) have become one of the most studied frameworks for unsupervised learning due to their intuitive formulation. They have also been shown to be capable of generating convincing examples in limited domains, such as low-resolution images. However, they still prove difficult to train in practice and tend to ignore modes of the data generating distribution. Quantitatively capturing effects such as mode coverage and more generally the quality of the generative model still remain elusive. We propose Generative Adversarial Parallelization (GAP), a framework in which many GANs or their variants are trained simultaneously, exchanging their discriminators. This eliminates the tight coupling between a generator and discriminator, leading to improved convergence and improved coverage of modes. We also propose an improved variant of the recently proposed Generative Adversarial Metric and show how it can score individual GANs or their collections under the GAP model.

TL;DR: Creating Synergy with Multiple Generative Adversarial Networks

Review discussion:
Thank you for your submission.
One experiment to measure mode coverage would be to synthesize simple data, such as MNIST digits, and then use a ConvNet to count if I get ~10% samples from each digit.
Did you think about doing such an experiment?
One potential issue with the evaluation proposed in this paper is the nature of the "GAM" metric, which relies on evaluating discriminator classification accuracy. It has been shown that discriminators often learn pixel-level artifacts in their filters to distinguish between real and fake samples. If this is the case... Why should I rely on a metric based on artifacts?
Thank you for your suggestion.
We included MNIST experiments according to the reviewer’s suggestion. We performed logistic regression which achieved ~= 99% accuracy rate. All of the models tested produced a fairly uniform distribution, though GAP seems to achieve a slightly higher Inception score than the baselines (see Figure 14).
Regarding the second question (proposed evaluation): The problem of “pixel-level” artifacts, of course, a limitation of the GAN training schema rather than the metric. We designed the metric based on the assumption of a valid training schema, however, we acknowledge this limitation and thank the reviewer for pointing this out. One important point is that increasing the number of discriminators should decrease the probability that they all learn the same artifacts.
Therefore this suggests that the GAM-II metric we proposed is more robust to such artifacts than the originally proposed GAM-I.
1. In Figure 3c and 4c, are the samples from GAP_{GAN4} generated from a single GAN? If so, were the samples from the non-chosen GANs also improved relative to single GAN training or were they degenerate?
2. Have you tried any more quantitative methods for assessing coverage in high-dimensional space?
For example, you could compute the average L2 distance from each test example to the nearest generated sample.
3. Did you consider alternate ways of evaluating sample quality such as the Inception Score (Salimans et al. 2016)? If so, did they correlate with your GAM-II results?
Thank you for your questions.
Regarding 1) Figure 3c and 4c are the samples from GAP_{GAN4} generated from a single GAN.
Although Figure 3c and 4c were the best ones chosen, we found that the samples from the other GANs within GAP produced superior examples compared to the baselines. We included samples from the other models within GAP_{GAN4}  in the supplementary material.
Regarding 2 & 3) we tried what Reviewer 1 suggested, which is looking at the uniformity of predicted classes compared to the distribution of training examples. We included these results in the paper (see the below forum).  We also implemented and reported the Inception score, on which GAP performs favourably.
Additionally, we computed log-likelihood estimates using AIS based on a recently proposed evaluation scheme (Wu et al 2016). With the code provided by Wu et al, we were able to apply it to GAP DCGANs and GAP_{combination} DCGANs.
But we have not been able to apply it to GRANs. We also included these results in the most recent revision of the paper.
Yuhuai Wu et al 2016, On the quantitative analysis of decoder based generative models (under ICLR review)
This paper proposes Generative Adversarial Parallelization (GAP), one schedule to train N Generative Adversarial Networks (GANs) in parallel. GAP proceeds by shuffling the assignments between the N generators and the N discriminators at play every few epochs.
Therefore, GAP forces each generator to compete with multiple discriminators at random. The authors claim that such randomization reduces undesired "mode collapsing behaviour", typical of GANs.
I have three concerns with this submission.
1) After training the N GANs for a sufficient amount of time, the authors propose to choose the best generator using the GAM metric. I oppose to this because of two reasons.
First, a single GAN will most likely be unable to express the full richness of the true data begin modeled. Said differently, a single generator with limited power will either describe a mode well, or describe many modes poorly.
Second, GAM relies on the scores given by the discriminators, which can be ill-posed (focus on artifacts). Since there is There is nothing wrong with mode collapsing when this happens under control.
Thus, I believe that a better strategy would be to not choose and combine all generators into a mixture.
Of course, this would require a way to decide on mixture weights. This can be done, for instance, using rejection sampling based on discriminator scores.
2) The authors should provide a theoretical (or at least conceptual) comparison to dropout. In essence, this paper has a very similar flavour: every generator is competing against all N discriminators, but at each epoch we drop N-1 for every generator. Related to the previous point, after training dropout keeps all the neurons, effectively approximating a large ensemble of neural networks.
3) The qualitative results are not convincing. Most of the figures show only results about GAP. How do the baseline samples look like? The GAN and LAPGAN papers show very similar samples.
On the other hand, I do not find Figures 3 and 4 convincing: for instance, the generator in Figure 3 was most likely under-parametrized.
As a minor comment, I would remove Figure 2. This is because of three reasons: it may be protected by copyright, it occupies a lot of space, and it does not add much value to the explanation.
Also, the indices (i_t) are undefined in Algorithm 1.
Overall, this paper shows good ideas, but it needs further work in terms of conceptual development and experimental evaluation.
Thanks for your review and suggestions on how to improve our paper. Please see the summary of improvements with respect to results and clarity in our response to Reviewer 3. We now respond to your specific concerns.
Concern 1: Choosing the single best generator using the [GAM-II] metric. In this work we viewed GAP as a training strategy and attempted to compare, “apples-to-apples” synthesis among single generators (not ensembles).
However, we agree with the reviewer that ensembling all GANs (e.g. as a mixture) has a richer modeling capacity and will likely improve quality of generated samples.
While we have not explored such a strategy in this work, we are interested in this extension and intend to pursue it.
Moreover, we recently found a paper that has a similar flavour to what the author suggested: AdaGAN: Boosting Generative Models (arXiv:1701.02386). It may be interesting to explore the use of GAP for training AdaGAN models. Please also see our response to Reviewer 2 for a discussion on the weakness of the GAM-II metric.
Concern 2: We should provide a theoretical (or at least a conceptual) comparison to dropout.
Thank you for pointing out the connection between GAP and dropout. We have added a conceptual comparison to Dropout in Section 3.1.
Concern 3a: The qualitative results are not convincing. Most of the [figures of generated samples] show only GAP results.
We agree that other papers often use this scheme to show samples of one GAN variant vs.
another. In our experience, visually distinguishing between the samples is difficult since our improvement mainly lies on mode coverage and stability. We’re not arguing that the visual quality of individual samples are better than the baseline training schemes.
Concern 3b: Figures 3 and 4 are not convincing; the generator in Figure 3 was most likely under-parametrized.
The architecture of the generator and discriminator for both individually-trained and GAP-trained models are exactly the same. Not only that, the hyper-parameters were also exactly the same. The only additional parameter tuned for GAP was swapping rate. Figure 3 and 4 are merely visualizations, and we noted that one should not draw strong conclusions from them, but we do feel that it is a fair comparison. The GAP-trained models appear to utilize the fixed number of parameters more effectively.
(1) If, instead of swapping, you were to simply train K GANs on K splits of the data, or K GANs with differing initial conditions (but without swapping) do you see any improvement in results?
Similarly, how about if you train larger capacity models with dropout in G and D? Since dropout essentially averages many models it would be interesting to see if the effects are the same.
(2) Did you compare against other models using the Inception score of Salimans et al.? Seems that this is becoming a widely used evaluation metric so I wonder why you chose not to use it.
(3) In figure 6 it appears that the validation costs remain the same as parallelization increase, but the training cost goes up and that is why the gap is shrinking. Does this really imply better generalization?
Thank you for your question. We would like to kindly ask the reviewer read the latest version (revised version of the paper, because we are curious whether this review was done based on first version of the paper)
Regarding 1) In our experiments (revised version of the paper), we trained 4 individual GANs  and compared with 4 GANs trained with GAP. As the reviewer said, these 4 individually trained GANs were initialized differently and the ordering of batch were permuted differently during the training. The results in table 2, 3, and 4 are based on the comparisons of 4 individually trained GANs versus 4 GANs with GAP applied. We report the best and worst error rates among 4 GANs using GAM-II and report the average and standard deviation of negative log-likelihood measured using AIS. We haven’t investigated on training large capacity models with dropout versus GAP(GANs).
Regarding 2) We report the inception score results in Table 5. More recently, (Wu et al. 2016. On the quantitative analysis of decoder-base generative model) have criticized that inception score heavily rely on visual quantity, and they have proposed metric based on log-likelihood. We followed Wu et al. 2016. paper and measured negative log-likelihood. This result is reported in Table 4.
Regarding 3) For sure, if the gap of training and validation cost is getting larger, then the discriminator is overfitting. Not only that, as shown in Figure 6, when the training cost goes down fast (e.g. GAP(DCGANx1) dis_tr), then naturally generators cost goes up. This means that the discriminator is getting too strong compared to the generator. This phenomena happens because discriminator error signal is larger than generator. It is ideal to keep the generator and discriminator balanced during the training.
Lastly, the learning of generator and discriminator are intertwined as they depend on each other during the training.
Hence, the generalization of discriminator can influence the generator learning.
This paper proposes to address the mode collapsing problem of GANs by training a large set of generators and discriminators, pairing them each up with different ones at different times throughout training. The idea here is that no one generator-discriminator pair can be too locked together since they are all being swapped. This idea is nice and is addressing an important issue with GAN training.
However, I think the paper is lacking in experimental results.
In particular:
- The authors need to do more work to motivate the GAM metric. It is not intuitively obvious to me that the GAM metric is a good way of evaluating the generator networks since it relies on the prediction of the discriminator networks which can fixate on artifacts. Perhaps the authors could explore if the GAM metric correlates with inception scores or human evaluations. Currently the only quantitative evaluation uses this criterion and it really isn't clear it's a relevant quantity to be measuring.
- Related to the above comment, the authors need to compare more to other methods. Why not evaluate inception scores and compare with previous methods.
Similarly, generation quality is not compared with previous methods. It's not obvious that the sample quality is any better with this method.
And now just repeating questions from pre-review section:
- If, instead of swapping, you were to simply train K GANs on K splits of the data, or K GANs with differing initial conditions (but without swapping) do you see any improvement in results?
Similarly, how about if you train larger capacity models with dropout in G and D? Since dropout essentially averages many models it would be interesting to see if the effects are the same.
- In figure 6 it appears that the validation costs remain the same as parallelization increase, but the training cost goes up and that is why the gap is shrinking. Does this really imply better generalization?
In summary, interesting paper that addresses an important issue with GAN training, but compelling results are missing.
Thank you for your thoughtful review. We have summarized the key improvements with respect to results and clarity in our response to Reviewer 3. To avoid duplicating text, we kindly refer you to that summary.
Both Reviewer 2 and 3 expressed some concern with the GAM-II metric’s reliance on discriminators, which can fixate on artifacts. This is an excellent point and we agree that GAM-II is flawed in this respect.
But we believe that it is an improvement over GAM (Im et al. 2016) which also relies on discriminators.
Despite its flaws, GAM-II has some other benefits compared to evaluation schemes like AIS and Inception scores: it can be performed online, it does not need any extra trained models nor data, etc.
That being said, the focus of our paper wasn't on the GAM-II metric, nor do we think that there is any single best metric for quantitatively evaluating generative models. We applied GAM-II as one of the evaluations in order to understand the performance of GAP. In response to the reviewer’s suggestions, we also measured negative log-likelihood of DCGANs based on the technique proposed by (Wu et al. 2016) -- see Table 4. We also reported the Inception score as suggested by all of the reviewers -- see Table 5.
This paper proposes an extension of the GAN framework known as GAP whereby multiple generators and discriminators are trained in parallel. The generator/discriminator pairing is shuffled according to a periodic schedule.
Pros:
+ The proposed approach is simple and easy to replicate.
Cons:
- The paper is confusing to read.
- The results are suggestive but do not conclusively show a performance win for GAP.
The main argument of the paper is that GAP leads to improved convergence and improved coverage of modes. The coverage visualizations are suggestive but there still is not enough evidence to conclude that GAP is in fact improving coverage. And for convergence it is difficult to assess the effect of GAP on the basis of learning curves. The proposed GAM-II metric is circular in that model performance depends on the collection of baselines the model is being compared with. Estimating likelihood via AIS seems to be a promising way to evaluate, as does using the Inception score.
Perhaps a more systematic way to determine GAP's effect would be to set up a grid search of hyperparameters and train an equal number of GANs and GAP-GANs for each setting.
Then a histogram over final Inception scores or likelihood estimates of the trained models would help to show whether GAP tended to produce better models.
Overall the approach seems promising but there are too many open questions regarding the paper in its current form.
* Section 2: "Remark that when..." => seems like a to-do.
* Section A.1: The proposed metric is not described in adequate detail.
Thank you for your thoughtful review. We have updated the paper and the results have improved (in terms of performance and consistency) since the initial submission. The key improvements are the following:
- We have updated the GAM-II results in Table 2 and 3.
- We evaluated GAP (DCGAN) based on maximum likelihood using AIS (Wu et al 2016).
- We also have looked at the distribution over class prediction based on samples generated by GAP and we included the inception score.
- Lastly, we have improved the clarity of the paper.
Regarding to the coverage of visualization, we agree with your comment and had acknowledged that the results are only suggestive.
Grid search over the hyper-parameter is a good suggestion. We will definitely try it!

Meta-review:
role: PC
data: comment: This paper was reviewed by three experts. While they find interesting ideas in the manuscript, all three point to deficiencies (problems with the use of GAM metric, lack of convincing results) and unanimously recommend rejection. I do not see a reason to overturn their recommendation.

Decision: Reject