Paper:
Fisher GAN
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework for training GANs. Fisher GAN defines a critic with a data dependent constraint on its second order moments. We show in this paper that Fisher GAN allows for stable and time efficient training that does not compromise the capacity of the critic, and does not need data independent constraints such as weight clipping. We analyze our Fisher IPM theoretically and provide an algorithm based on Augmented Lagrangian for Fisher GAN. We validate our claims on both image sample generation and semi-supervised classification using Fisher GAN.

Review:
This paper proposes a new criterion for training a Generative Adversarial Network and shows that this new criterion yields stability benefits.
The criterion is related to Fisher discriminant analysis and is essentially a normalized IPM.
The authors show that this criterion is equivalent to the symmetric chi-squared divergence.
One reason for not being fully enthusiastic about this paper is that the fact that the proposed criterion is equivalent to chi-squared, which is
an f-divergence, reduces the novelty of the approach and raises several questions that are not addressed:
- is the proposed implementation significantly different than what one would obtain by applying f-GAN (with the appropriate f corresponding
to the chi-squared divergence)? if not, then the algorithm is really just f-GAN with chi-squared divergence and the novelty is drastically reduced. If yes,
then it would be great to pinpoint the differences and explain why they are important or would significantly address the issues that are inherent
to classical f-divergences based GANs in terms of training stability.
- how do the results compare if one uses other f-divergences (for example non-symmetric chi-squared but other f-divergences as well)?
- because the criterion is an f-divergence, it may suffer from the same issues that were pointed out in the WGAN paper: gradients would vanish
for distributions with disjoint support. Is there a reason why chi-squared would not have the same issues as Jensen-Shannon or KL ? Or is
it that the proposed implementation, not being exactly written as chi-squared and only equivalent at the optimum in lambda, doesn't suffer from
these issues?
Regarding the experimental results:
The DCGAN baseline for inceptions scores (Figure 4) seem lower than what was reported in earlier papers and the current state of the art is more around 8
(not sure what it was at the time of the writing of this paper though).
The SSL results (Table 2) are ok when compared to the weight clipping implementation of WGAN but this is not necessarily the right baseline to compare with,
and compared to other algorithms, the results are not competitive.
My overall feeling is that this paper presents an interesting connection between the chi-squared divergence and the normalized IPM coefficient which
deserves further investigation (especially a better understanding of how is this connected to f-GAN with chi-squared). But the comparison with the
known suboptimal implementation of WGAN (with weight clipping) is not so interesting and the results are not really convincing either.
So overall I think the paper is below the acceptance bar in its current form.