user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step

Paper content:
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion.
Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost.
GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players’ parameters.
One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium.
Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence.
We show that this view is overly restrictive.
During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful.
We provide empirical counterexamples to the view of GAN training as divergence minimization.
Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail.
We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful.
This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.
Generative adversarial networks (GANs) BID9 are generative models based on a competition between a generator network G and a discriminator network D. The generator network G represents a probability distribution p model (x) .
To obtain a sample from this distribution, we apply the generator network to a noise vector z sampled from p z , that is x = G(z).
Typically, z is drawn from a Gaussian or uniform distribution, but any distribution with sufficient diversity is possible.
The discriminator D(x) attempts to distinguish whether an input value x is real (came from the training data) or fake (came from the generator).The
goal of the training process is to recover the true distribution p data that generated the data. Several
variants of the GAN training process have been proposed. Different
variants of GANs have been interpreted as approximately minimizing different divergences or distances between p data and p model . However,
it has been difficult to understand whether the improvements are caused by a change in the underlying divergence or the learning dynamics.We conduct several experiments to assess whether the improvements associated with new GAN methods are due to the reasons cited in their design motivation. We perform
a comprehensive study of GANs on simplified, synthetic tasks for which the true p data is known and the relevant distances are straightforward to calculate, to assess the performance of proposed models against baseline methods. We also evaluate
GANs using several independent evaluation measures on real data to better understand new approaches. Our contributions
are:• We aim to clarify terminology used in recent papers, where the terms "standard GAN," "regular GAN," or "traditional GAN" are used without definition (e.g., BID4 BID23 BID5 ). The original GAN
paper described two different losses: the "minimax" loss and the "non-saturating" loss, equations FORMULA0 and FORMULA0 of BID8 , respectively. Recently, it has
become important to clarify this terminology, because many of the criticisms of "standard GANs", e.g. , are applicable only to the minimax GAN, while the non-saturating GAN is the standard for GAN implementations. The non-saturating
GAN was recommended for use in practice and implemented in the original paper of BID9 , and is the default in subsequent papers BID20 BID23 BID5 BID18 1 . To avoid confusion
we will always indicate whether we mean minimax GAN (M-GAN) or non-saturating GAN (NS-GAN).• We demonstrate that
gradient penalties designed in the divergence minimization frameworkto improve Wasserstein GANs BID10 or justified from a game theory perspective to improve minimax GANs BID13 -also improve the non-saturating GAN on both synthetic and real data. We observe improved sample
quality and diversity.• We find that non-saturating
GANs are able to fit problems that cannot be fit by JensenShannon divergence minimization. Specifically, FIG1 shows a GAN
using the loss from the original non-saturating GAN succeeding on a task where the Jensen-Shannon divergence provides no useful gradient. FIG0 shows that the non-saturating
GAN does not suffer from vanishing gradients when applied to two widely separated Gaussian distributions.
We have shown that viewing the training dynamics of GANs through the lens of the underlying divergence at optimality can be misleading.
On low-dimensional synthetic problems, we showed that non-saturating GANs are able to learn the true data distribution where Jensen-Shannon divergence minimization would fail.
We also showed that gradient penalty regularizers help improve the training dynamics and robustness of non-saturating GANs.
It is worth noting that one of the gradient penalty regularizers was originally proposed for Wasserstein GANs, motivated by properties of the Wasserstein distance; evaluating non-saturating GANs with similar gradient penalty regularizers helps disentangle the improvements arising from optimizing a different divergence (or distance) and the improvements from better training dynamics.Comparison between explored gradient penalties: As described in Section 2.3, we have evaluated two gradient penalties on non-saturating GANs.
We now turn our attention to the distinction between the two gradient penalties.
We have already noted that for a few hyperparameter settings, DRAGAN-NS produced samples with mode collapse, while the GAN-GP model did not.
By looking at the resulting metrics, we note that there is no clear winner between the two types of gradient penalties.
To assess whether the two penalties have a different regularization effect, we also tried applying both (with a gradient penalty coefficient of 10 for both, or of 5 for both), but that did not result in better models.
This could be because the two penalties have a very similar effect, or due to optimization considerations (they might conflict with each other).Other
gradient penalties: Besides the gradient penalties explored in this work, several other regularizers have been proposed for stabilizing GAN training. BID22
proposed a gradient penalty aiming to smooth the discriminator of f -GANs (including the minimax GAN), which we refer to as f -GAN-GP, inspired by Sønderby et al. FORMULA0 and . Their
gradient penalty is different from the ones explored here; specifically, their gradient penalty is weighted by the square of the discriminator's probability of real for each data instance and the penalty is applied to data and samples (no noise is added). In Fisher-GAN
BID17 , an equality constraint that is added on the magnitude of the output of the discriminator on data as well as samples is directly penalized, as opposed to the magnitude of the discriminator gradients, as in WGAN-GP. Similar to WGAN-GP
, the penalty was introduced in the framework of integral probability metrics, but it can be directly applied to other approaches to GAN training. Unlike WGAN-GP, Fisher
GAN uses augmented Lagrangians to impose the equality constraint, instead of a penalty method. To the best of our knowledge
, this has not been tried yet and we leave it for future work.The regularizers assessed in this work (the penalties proposed by DRAGAN and WGAN-GP), as well as others (such as f -GAN-GP and Fisher-GAN) are similar in spirit, but have been proposed from distinct theoretical considerations. Future study of GAN regularizers
will determine how these regularizers interact, and help us understand the mechanism by which they stabilize GAN training and motivate new approaches.

assistant: text: This paper studies non-saturating GANs and the effect of two penalized gradient approaches, considering several thought experiments to demonstrate observations and validate them on real data experiments.