C OULOMB GAN S : P ROVABLY O PTIMAL N ASH E QUILIBRIA VIA P OTENTIAL F IELDS
A BSTRACT
Generative adversarial networks (GANs) evolved into one of the most successful unsupervised techniques for generating realistic images.
Even though it has recently been shown that GAN training converges, GAN models often end up in local Nash equilibria that are associated with mode collapse or otherwise fail to model the target distribution.
We introduce Coulomb GANs, which pose the
GAN learning problem as a potential ﬁeld, where generated samples are attracted to training set samples but repel each other.
The discriminator learns a potential ﬁeld while the generator decreases the energy by moving its samples along the vector (force) ﬁeld determined by the gradient of the potential ﬁeld.
Through decreasing the energy, the GAN model learns to generate samples according to the whole target distribution and does not only cover some of its modes.
We prove that Coulomb GANs possess only one Nash equilibrium which is optimal in the sense that the model distribution equals the target distribution.
We show the efﬁcacy of Coulomb GANs on LSUN bedrooms, CelebA faces, CIFAR-10 and the
Google Billion Word text generation.
I NTRODUCTION
Generative adversarial networks (GANs) (Goodfellow et al., 2014) excel at constructing realistic images (Radford et al., 2016; Ledig et al., 2016; Isola et al., 2017; Arjovsky et al., 2017; Berthelot et al., 2017) and text (Gulrajani et al., 2017).
In GAN learning, a discriminator network guides the learning of another, generative network.
This procedure can be considered as a game between the generator which constructs synthetic data and the discriminator which separates synthetic data from training set data (Goodfellow, 2017).
The generator’s goal is to construct data which the discriminator cannot tell apart from training set data.
GAN convergence points are local Nash equilibria.
At these local Nash equilibria neither the discriminator nor the generator can locally improve its objective.
Despite their recent successes, GANs have several problems.
First (I), until recently it was not clear if in general gradient-based GAN learning could converge to one of the local Nash equilibria (Salimans et al., 2016; Goodfellow, 2014; Goodfellow et al., 2014).
It is even possible to construct counterexamples (Goodfellow, 2017).
Second (II), GANs suffer from “mode collapsing”, where the model generates samples only in certain regions which are called modes.
While these modes contain realistic samples, the variety is low and only a few prototypes are generated.
Mode collapsing is less likely if the generator is trained with batch normalization, since the network is bound to create a certain variance among its generated samples within one batch (Radford et al., 2016; Chintala et al., 2016).
However batch normalization introduces ﬂuctuations of normalizing constants which can be harmful (Klambauer et al., 2017; Goodfellow, 2017).
To avoid mode collapsing without batch normalization, several methods have been proposed (Che et al., 2017; Metz et al., 2016; Salimans et al.,
Third (III), GANs cannot assure that the density of training samples is correctly modeled by the generator.
The discriminator only tells the generator whether a region is more likely to contain samples from the training set or synthetic samples.
Therefore the discriminator can only distinguish the support of the model distribution from the support of the target distribution.
Beyond matching the support of distributions, GANs with proper objectives may learn to locally align model and target densities via averaging over many training examples.
On a global scale, however, GANs fail to equalize model and target densities.
The discriminator does not inform the generator globally where probability mass is missing.
Consequently, standard GANs are not assured to capture the global sample density and are prone to neglect large parts of the target distribution.
The next paragraph gives an example of this.
Fourth (IV), the discriminator of GANs may forget previous modeling errors of the generator which then may reappear, a property that leads to oscillatory behavior instead of convergence (Goodfellow, 2017).
Recently, problem (I) was solved by proving that GAN learning does indeed converge when discriminator and generator are learned using a two time-scale learning rule (Heusel et al., 2017).
Convergence means that the expected SGD-gradient of both the discriminator objective and the generator objective are zero.
Thus, neither the generator nor the discriminator can locally improve, i.e., learning has reached a local Nash equilibrium.
However, convergence alone does not guarantee good generative performance.
It is possible to converge to sub-optimal solutions which are local
Nash equilibria.
Mode collapse is a special case of a local Nash equilibrium associated with suboptimal generative performance.
For example, assume a two mode real world distribution where one mode contains too few and the other mode too many generator samples.
If no real world samples are between these two distinct modes, then the discriminator penalizes to move generated samples outside the modes.
Therefore the generated samples cannot be correctly distributed over the modes.
Thus, standard GANs cannot capture the global sample density such that the resulting generators are prone to neglect large parts of the real world distribution.
A more detailed example is listed in the Appendix in Section A.1.
In this paper, we introduce a novel GAN model, the Coulomb GAN, which has only one Nash equilibrium.
We are later going to show that this Nash equilibrium is optimal, i.e., the model distribution matches the target distribution.
We propose Coulomb GANs to avoid the GAN shortcoming (II) to (IV) by using a potential ﬁeld created by point charges analogously to the electric ﬁeld in physics.
The next section will introduce the idea of learning in a potential ﬁeld and prove that its only solution is optimal.
We will then show how learning the discriminator and generator works in a Coulomb
GAN and discuss the assumptions needed for our optimality proof.
In Section 3 we will then see that the Coulomb GAN does indeed work well in practice and that the samples it produces have very large variability and appear to capture the original distribution very well.
Related Work.
Several GAN approaches have been suggested for bringing the target and model distributions in alignment using not just local discriminator information: Geometric GANs combine samples via a linear support vector machine which uses the discriminator outputs as samples, therefore they are much more robust to mode collapsing (Lim & Ye, 2017).
Energy-Based GANs (Zhao et al., 2017) and their later improvement BEGANs (Berthelot et al., 2017) optimize an energy landscape based on auto-encoders.
McGANs match mean and covariance of synthetic and target data, therefore are more suited than standard GANs to approximate the target distribution (Mroueh et al., 2017).
In a similar fashion, Generative Moment Matching Networks (Li et al., 2015) and MMD nets (Dziugaite et al., 2015) directly optimize a generator network to match a training distribution by using a loss function based on the maximum mean discrepancy (MMD) criterion (Gretton et al., 2012).
These approaches were later expanded to include an MMD criterion with learnable kernels and discriminators (Li et al., 2017).
The MMD criterion that these later approaches optimize has a form similar to the energy function that Coulomb GANs optimize (cf.
Eq.
However, all MMD approaches end up using either Gaussian or Laplace kernels, which are not guaranteed to ﬁnd the optimal solution where the model distribution matches the target distribution.
In contrast, the Plummer kernel which is employed in this work has been shown to lead to the optimal solution (Hochreiter & Obermayer, 2005).
We show that even a simpliﬁed version of the Plummer kernel, the low-dimensional Plummer kernel, ensures that gradient descent convergences to the optimal solution as stated by Theorem 1.
Furthermore, most MMD GAN approaches use the MMD directly as loss function though the number of possible samples in a mini-batch is limited.
Therefore MMD approaches face a sampling problem in high-dimensional spaces.
The Coulomb GAN instead learns a discriminator network that gradually improves its approximation of the potential ﬁeld via learning on many mini-batches.
The discriminator network also tracks the slowly changing generator distribution during learning.
Most importantly however, our approach is, to the best of our knowledge, the ﬁrst one for which optimality, i.e., ability to perfectly learn a target distribution, can be proved.
The use of the Coulomb potential for learning is not new.
Coulomb Potential Learning was proposed to store arbitrary many patterns in a potential ﬁeld with perfect recall and without spurious patterns (Perrone & Cooper, 1995).
Another related work is the Potential Support Vector Machine (PSVM), which minimizes Coulomb potential differences (Hochreiter & Mozer, 2001; Hochreiter et al., 2003).
Hochreiter & Obermayer (2005) also used a potential function based on Plummer kernels for optimal unsupervised learning, on which we base our work on Coulomb GANs.
C OULOMB GAN S 2. G ENERAL C ONSIDERATIONS ON GAN S
We assume data samples a ∈ R m for a model density p x ( . ) and a target density p y ( . ) .
The goal of
GAN learning is to modify the model in a way to obtain p x ( . )
=
p y ( . ) .
We deﬁne the difference of densities ρ ( a ) =
p y ( a ) − p x ( a ) which should be pushed toward zero for all a ∈ R m during learning.
In the GAN setting, the discriminator D ( a ) is a function D : R m → R that learns to discriminate between generated and target samples and predicts how likely it is that a is sampled from the target distribution.
In conventional GANs, D ( a ) is usually optimized to approximate the probability of seeing a target sample, or ρ ( a ) or some similar function.
The generator G ( z ) is a continuous function G : R n → R m which maps some n -dimensional random variable z into the space of target samples.
z is typically sampled from a multivariate Gaussian or Uniform distribution.
In order to improve the generator, a GAN uses the gradient of the discriminator ∇ a D ( a ) with respect to the discriminator input a = G ( z ) for learning.
The objective of the generator is a scalar function D ( G ( z )) , therefore the gradient of the objective function is just a scaled version of the gradient ∇ a D ( a ) which would then propagate further to the parameters of G .
This gradient ∇ a D ( a ) tells the generator in which direction ρ ( a ) becomes larger, i.e., in which direction the ratio of target examples increases.
The generator changes slightly so that z is now mapped to a new a (cid:48)
=
G (cid:48) ( z ) , moving the sample generated by z a little bit towards the direction where ρ ( a ) was larger, i.e., where target examples were more likely.
However, ρ ( a ) and its derivative only take into account the local neighborhood of a , since regions of the sample space that are distant from a do not have much inﬂuence on ρ ( a ) .
Regions of data space that have strong support in p y but not in p x will not be noticed by the generator via discriminator gradients.
The restriction to local environments hampers GAN learning signiﬁcantly (Arjovsky & Bottou, 2017; Arjovsky et al., 2017).
The theoretical analysis of GAN learning can be done at three different levels: (1) in the space of distributions p x and p y regardless of the fact that p x is realized by G and p z , (2) in the space of functions G and D regardless of the fact that G and D are typically realized by a parametric form, i.e., as neural networks, or (3) in the space of the parameters of G and D .
Goodfellow et al.
(2014) use (1) to prove convergence of GAN learning in their Proposition 2 in a hypothetical scenario where the learning algorithm operates by making small, local moves in p x space.
In order to see that level (1) and (2) should both be understood as hypothetical scenarios, remember that in all practical implementations, p x can only be altered implicitly by making small changes to the generator function G, which in turn can only be changed implicitly by small steps in its parameters.
Even if we assume that the mapping from a distribution p x to the generator G that induced it exists and is unique, this mapping from p x to the space of G is not continuous. To see this, consider changing a distribution p 1 x to a new distribution p 2 x by moving a small amount (cid:15) of its density to an isolated region in space where p 1 x has no support.
Let’s further assume this region has distance d to any other regions of support of p 1 x .
By letting (cid:15) → 0 , the distance between p 1 x and p 2 x becomes smaller, yet the distance between the inducing generator functions G 1 and G 2 (e.g. using the supremum norm on bounded functions) will not tend to zero because for at least one function input z we have: | G 1 ( z ) − G 2 ( z ) | (cid:62) d .
Because of this, we need to go further than the distribution space when analyzing GAN learning.
In practice, when learning GANs, we are restricted to small steps in parameter space, which in turn lead to small steps in function space and ﬁnally to small steps in distribution space.
But not all small steps in distribution space can be realized this way as shown in the example above.
This causes local Nash equilibria in the function space, because even though in distribution space it would be easy to escape by making small steps, such a step would require very large changes in function space and is thus not realizable.
In this paper we show that Coulomb GANs do not exhibit any local Nash equilibria in the space of the functions G and D .
To the best of our knowledge, this is the ﬁrst formulation of GAN learning that can guarantee this property.
Of course, Coulomb GANs are learned as parametrized neural networks, and as we will discuss in Subsection 2.4.2, Coulomb GANs are not immune to the usual issues that arise from parameter learning, such as over- and underﬁtting, which can cause local Nash Equilibria due to a bad choice of parameters.
2. F ROM C ONVENTIONAL GAN S TO P OTENTIALS
If the density p x ( . ) or p y ( . ) approaches a Dirac delta-distribution, gradients vanish since the density approaches zero except for the exact location of data points.
Similarly, electric point charges are often represented by Dirac delta-distributions, however the electric potential created by a point charge has inﬂuence everywhere in the space, not just locally.
The electric potential (Coulomb potential) created by the point charge Q is Φ C =
[Equation]
[Equation]
Qr , where r is the distance to the location of Q and ε 0 is the dielectric constant.
Motivated by this electric potential, we introduce a similar concept for GAN learning: Instead of the difference of densities ρ ( a ) , we rather consider a potential function Φ( a ) deﬁned as
[Equation]
with some kernel k ( a , b ) which deﬁnes the inﬂuence of a point at b onto a point at a .
The crucial advantage of potentials Φ( a ) is that each point can inﬂuence each other point in space if k is chosen properly.
If we minimize this potential Φ( a ) we are at the same time minimizing the difference of densities ρ ( a ) :
For all kernels k it holds that if ρ ( b ) = 0 for all b then Φ( a ) = 0 for all a .
We must still show that (i) Φ( a ) =
for all a then ρ ( b ) =
for all b , and even more importantly, (ii) whether a gradient optimization of Φ( a ) leads to Φ( a )
=
for all a .
This is not the case for every kernel.
Indeed only for particular kernels k gradient optimization of Φ( a ) leads to ρ ( b ) = 0 for all b , that is, p x ( b ) =
p y ( b ) for all b (Hochreiter & Obermayer, 2005) (see also Theorem 1 below).
An example for such a kernel k is the one leading to the Coulomb potential Φ C from above, where k ( a , b ) = 1 (cid:107) a − b (cid:107) for m = 3 .
As we will see in the following, the ability to have samples that inﬂuence each other over long distances, like charges in a Coulomb potential, will lead to GANs with a single, optimal Nash equilibrium.
2. GAN S AS E LECTRICAL F IELDS
For Coulomb GANs, the generator objective is derived from electrical ﬁeld dynamics: real and generated samples generate a potential ﬁeld, where samples of the same class (real vs. generated) repel each other, but attract samples of the opposite class.
However, real data points are ﬁxed in space, so the only samples that can move are the generated ones.
In turn, the gradient of the potential with respect to the input samples creates a vector ﬁeld in the space of samples.
The generator can move its samples along the forces generated by this ﬁeld.
Such a ﬁeld is depicted in Figure 1.
The discriminator learns to predict the potential function, in order to approximate the current potential landscape of all samples, not just the ones in the current mini-batch.
Meanwhile, the generator learns to distribute its samples across the whole ﬁeld in such a way that the energy is minimized, thus naturally avoids mode collapse and covering the whole region of support of the data.
The energy is minimal and equal to zero only if all potential differences are zero and the model distribution is equal to the target distribution.
Within an electrostatic ﬁeld, the strength of the force on one particle depends on its distance to other particles and their charges.
If left to move freely, the particles will organize themselves into a constellation where all forces equal out and no potential differences are present.
For continuous charge distributions, the potential ﬁeld is constant without potential differences if charges no longer move since forces are equaled out.
If the potential ﬁeld is constant, then the difference of densities ρ is constant, too.
Otherwise the potential ﬁeld would have local bumps.
The same behavior is modeled within our Coulomb GAN, except that real and generated samples replace the positive and negative particles, respectively, and that the real data points remain ﬁxed.
Only the generated samples are allowed to move freely, in order to minimize ρ .
The generated samples are attracted by real samples, so they move towards them.
At the same time, generated samples should repel each other, so they do not clump together, which would lead to mode collapsing.
Analogously to electrostatics, the potential Φ( a ) from Eq.
(1) gives rise to a ﬁeld E ( a )
=
− ∇ a Φ( a ) .
and to an energy function F ( ρ ) = 12 (cid:82) ρ ( a )Φ( a )d a .
The ﬁeld E ( a ) applies a force on charges at a which pushes the charges toward lower energy constellations.
Ultimately, the Coulomb
GAN aims to make the potential Φ zero everywhere via the ﬁeld E ( a ) , which is the negative gradient of Φ .
For proper kernels k , it can be shown that (i) Φ can be pushed to zero via its negative gradient given by the ﬁeld and (ii) that Φ( a ) =
for all a implies ρ ( a ) =
for all a , therefore, p x ( a ) =
p y ( a ) for all a (Hochreiter & Obermayer, 2005) (see also Theorem 1 below).
2.3. L EARNING P ROCESS
During learning we do not change Φ or ρ directly.
Instead, the location a = G ( z ) to which the random variable z is mapped changes to a new location a (cid:48) =
G (cid:48) ( z ) .
For the GAN optimization dynamics, we assume that generator samples a = G ( z ) can move freely, which is ensured by a sufﬁciently complex generator.
Importantly, generator samples originating from random variables z do neither disappear nor are they newly created but are conserved.
This conservation is expressed by the continuity equation (Schwartz, 1972) that describes how the difference between distributions ρ ( a ) changes as the particles are moving along the ﬁeld, i.e., how moving samples during the learning process changes our densities:
[Equation]
for sample density difference ρ and unit charges that move with “velocity” v ( a ) = sign ( ρ ( a ))
E ( a ) .
The continuity equation is crucial as it establishes the connection between moving samples and changing the generator density and thereby ρ .
The sign function of the velocity indicates whether positive or negative charges are present at a .
The divergence operator “ ∇· ” determines whether samples move toward or outward of a for a given ﬁeld.
Basically, the continuity equation says that if the generator density increases, then generator samples must ﬂow into the region and if the generator density decreases, they ﬂow outwards.
We assume that differently charged particles cancel each other.
If generator samples are moved away from a location a then ρ ( a ) is increasing while ρ ( a ) is decreasing when generator samples are moved toward a .
The continuity equation is also obtained as a ﬁrst order ODE to move particles in a potential ﬁeld (Dembo & Zeitouni, 1988), therefore describes the dynamics how the densities are changing.
We obtain
[Equation]
The density difference ρ ( a ) indicates how many samples are locally available for being moved.
At each local minimum and local maximum a of ρ we obtain ∇ a ρ ( a ) = 0 .
Using the product rule for the divergence operator, at points a that are minima or maxima, Eq.
(3) reduces to
[Equation]
In order to ensure that ρ converges to zero, it is necessary and sufﬁcient that sign ( ∇ · E ( a ))
=
sign ( ρ ( a )) , where ÷ a ρ ( a ) = 0 , as this condition ensures the uniform decrease of the maximal absolute density differences | ρ ( a max ) | .
2.3. C HOICE OF K ERNEL
As discussed before, the choice of kernel is crucial for Coulomb GANs.
The m -dimensional
Coulomb kernel and the m -dimensional Plummer kernel lead to (i) Φ that is pushed to zero via the ﬁeld it creates and (ii) that Φ( a ) =
for all a implies ρ ( a ) =
for all a , therefore, p x ( a ) =
p y ( a ) for all a (Hochreiter & Obermayer, 2005).
Thus, gradient learning with these kernels has been proved to converge to an optimal solution.
However, both the m -dimensional Coulomb and the mdimensional Plummer kernel lead to numerical instabilities if m is large.
Therefore the Coulomb potential Φ( a ) for the Coulomb GAN was constructed by a low-dimensional Plummer kernel k with parameters d (cid:54) m − 2 and (cid:15) :
[Equation]
The original Plummer kernel is obtained with d = m − 2 .
The resulting ﬁeld and potential energy is
[Equation]
The next theorem states that for freely moving generated samples, ρ converges to zero, that is, p x ( . ) =
p y ( . ) , when using this potential function Φ( a ) .
Theorem 1 (Convergence with low-dimensional Plummer kernel) .
For a , b ∈ R m , d (cid:54) m − 2 , and (cid:15) > 0 the densities p x ( . ) and p y ( . ) equalize over time when minimizing energy F with the low-dimensional Plummer kernel by gradient descent.
The convergence is faster for larger d .
Proof.
See Section A.2.
2. D EFINITION OF THE C OULOMB GAN
The Coulomb GAN minimizes the electric potential energy from Eq.
(6) using a stochastic gradient descent based approach using mini-batches.
Appendix Section A.4 contains the equations for the
Coulomb potential, ﬁeld, and energy in this case.
Generator samples are obtained by drawing N x random numbers
z i and transforming them into outputs x i =
G
( z i ) .
Each mini-batch also includes
N y real world samples y i .
This gives rise to a mini-batch speciﬁc potential, where in Eq.
(5) we use ρ ( a ) =
p y ( a ) − p x ( a ) and replace the expectations by empirical means using the drawn samples:
[Equation]
It is tempting to have a generator network that directly minimizes this potential ˆΦ between generated and training set points.
In fact, we show that ˆΦ is an unbiased estimate for Φ in Appendix
Section A.4.
However, the estimate has very high variance: for example, if a mini-batch fails to sample training data from an existing mode, the ﬁeld would drive all generated samples that have been generated at this mode to move elsewhere.
The high variance has to be counteracted by extremely low learning rates, which makes learning infeasible in practice, as conﬁrmed by initial experiments.
Our solution to this problem is to have a network that generalizes over the mini-batch speciﬁc potentials: each mini-batch contains different generator samples X
= x i for i = 1 , . . .
, N x and real world samples Y =
y i for i = 1 , . . .
, N y , they create a batch-speciﬁc potential ˆ Φ .
The goal of the discriminator is to learn E X , Y (ˆΦ( a )) =
Φ( a ) , i.e., the potential averaged over many mini-batches.
Thus the discriminator function D fulﬁlls a similar role as other typical GAN discriminator functions, i.e., it discriminates between real and generated data such that for any point in space a , D ( a ) should be greater than zero if the p y ( a ) > p x ( a ) and smaller than zero otherwise.
In particular
D ( a ) also indicates, via its gradient and its potential properties, directions toward regions where training set samples are predominant and where generator samples are predominant.
The generator in turn tries to move all of its samples according to the vector ﬁeld into areas where generator samples are missing and training set samples are predominant.
The generator minimizes the approximated energy F as predicted by the discriminator.
The loss L D for the discriminator and
L G for the generator are given by:
[Equation]
[Equation]
Where p ( a ) = 1 / 2 (cid:82) N ( a ; G ( z ) , (cid:15) I ) p
z
( z )d z + 1 / 2 (cid:82) N ( a ; y , (cid:15) I )
p y
( y )d y , i.e., a distribution where each point of support both of the generator and the real world distribution is surrounded with a Gaussian ball of width (cid:15)
I similar to Bishop et al.
(1998), in order to overcome the problem that the generator distribution is only a sub-manifold of R m .
These loss functions cause the approximated potential values D ( a ) that are negative are pushed toward zero.
Finally, the Coulomb GAN, like all other GANs, consists of two parts: a generator to generate model samples, and a discriminator that provides its learning signal.
Without a discriminator, our would be very similar to GMMNs (Li et al., 2015), as can be seen in Eq.
(33), but with an optimal Kernel speciﬁcally tailored to the problem of estimating differences between probability distributions.
We use each mini-batch only for one update of the discriminator and the generator.
It is important to note that the discriminator uses each sample in the mini batch twice: once as a point to generate the mini-batch speciﬁc potential ˆΦ , and once as a point in space for the evaluation of the potential ˆΦ and its approximation D .
Using each sample twice is done for performance reasons, but not strictly necessary: the discriminator could learn the potential ﬁeld by sampling points that lie between generator and real samples as in Gulrajani et al.
but we are mainly interested in correct predictions in the vicinity of generator samples.
Pseudocode for the learning algorithm is detailed in Algorithm 1 in the appendix.
2.4. O PTIMALITY OF THE S OLUTION
Convergence of the GAN learning process was proved for a two time-scales update rule by Heusel et al.
A local Nash equilibrium is a pair of generator and discriminator ( D ∗ , G ∗ ) that fulﬁlls the two conditions
[Equation]
for some neighborhoods U ( D ∗ ) and U ( G ∗ ) .
We show in the following Theorem 2 that for Coulomb
GANs every local Nash equilibrium necessarily is identical to the unique global Nash equilibrium.
In other words, any equilibrium point of the Coulomb GAN that is found to be local optimal has to be the one global Nash equilibrium as the minimization of the energy F ( ρ ) in Eq.
(33) leads to a single, global optimum at p y = p x .
Theorem 2 (Optimal Solution) .
If the pair ( D ∗ , G ∗ ) is a local Nash equilibrium for the Coulomb
GAN objectives, then it is the global Nash equilibrium, and no other local Nash equilibria exist, and
G ∗ has output distribution p x = p y .
Proof.
See Appendix Section A.3.
2.4. C OULOMB GAN S IN P RACTICE
To implement GANs in practice, we need learnable models for G and D .
We assume that our models for G and D are continuously differentiable with respect to their parameters and inputs.
Toward this end, GANs are typically implemented as neural networks optimized by (some variant of) gradient descent.
Thus we may not ﬁnd the optimal G ∗ or D ∗ , since neural networks may suffer from capacity or optimization issues.
Recent research indicates that the effect of local minima in deep learning vanishes with increasing depth (Dauphin et al., 2014; Choromanska et al., 2015;
Kawaguchi, 2016), such that this limitation becomes less restrictive as our ability to train deep networks grows thanks to hardware and optimization improvements.
The main problem with learning Coulomb GANs is to approximate the potential function Φ , which is a complex function in a high-dimensional space, since the potential can be very non-linear and non-smooth.
When learning the discriminator, we must ensure that enough data is sampled and averaged over.
We already lessened the non-linear function problem by using a low-dimensional
Plummer kernel.
But still, this kernel can introduce large non-linearities if samples are close to each other.
It is crucial that the discriminator learns slow enough to accurately estimate the potential function which is induced by the current generator.
The generator, in turn, must be even slower since it must be tracked by the discriminator.
These approximation problems are supposed to be tackled by the research community in near future, which would enable optimal GAN learning.
The formulation of GAN learning as a potential ﬁeld naturally solves the mode collapsing issue: the example described in Section A.1, where a normal GAN cannot get out of a local Nash equilibria is not a converged solution for the Coulomb GAN: If all probability mass of the generator lies in one of the modes, then both attracting forces from real-world samples located at the other mode as well as repelling forces from the over-represented generator mode will act upon the generator until it generates samples at the other mode as well.
E XPERIMENTS
In all of our experiments, we used a low-dimensional Plummer Kernel of dimensionality d = 3 .
This kernel both gave best computational performance and has low risk of running into numerical issues.
We used a batch size of 128.
To evaluate the quality of a GAN, the FID metric as proposed by Heusel et al.
(2017) was calculated by using 50k samples drawn from the generator, while the training set statistics were calculated using the whole training set.
We compare to BEGAN (Berthelot et al., 2017), DCGAN (Radford et al., 2016) and WGAN-GP (Gulrajani et al., 2017) both in their original version as well as when using the two-timescale update-rule (TTUR), using the settings from Heusel et al.
We additionally compare to MMD-GAN (Li et al., 2017), which is conceptually very similar to the Coulomb GAN, but uses a Gaussian Kernel instead of the Plummer Kernel.
We use the dataset-speciﬁc settings recommended in (Li et al., 2017) and report the highest FID score over the course of training.
All images shown in this paper were produced with a random seed and not cherry picked.
The implementation used for these experiments is available online 1 .
The appendix
Section A.5 contains an additional toy example demonstrating that Coulomb GANs do not suffer from mode collapse when ﬁtting a simple Gaussian Mixture of 25 components.
3. I MAGE D ATASETS
To demonstrate the ability of the Coulomb GAN to learn distributions in high dimensional spaces, we trained a Coulomb GAN on several popular image data sets: The cropped and centered images of celebrities from the Large-scale CelebFaces Attributes ( “CelebA” ) data set (Liu et al., 2015), the
LSUN bedrooms data set consists of over 3 million 64x64 pixel images of the bedrooms category of the large scale image database LSUN (Yu et al., 2015) as well as the CIFAR-10 data set.
For these experiments, we used the DCGAN architecture (Radford et al., 2016) with a few modiﬁcations: our convolutional kernels all have a kernel size of 5x5, our random seed that serves as input to the generator has fewer dimensions: 32 for CelebA and LSUN bedrooms, and 16 for CIFAR-10.
Furthermore, the discriminator uses twice as many feature channels in each layer as in the DCGAN architecture.
For the Plummer kernel, (cid:15) was set to 1.
We used the Adam optimizer with a learning rate of 10 −for the generator and 5 · 10 − 5 for the discriminator.
To improve convergence performance, we used the tanh output activation function (LeCun et al., 1998).
For regularization we used an L2 weight decay term with a weighting factor of 10 − 7 .
Learning was stopped by monitoring the FID metric (Heusel et al., 2017).
Once learning plateaus, we scaled the learning rate down by a factor of 10 and let it continue once more until the FID plateaus.
The results are reported in Table 1b, and generated images can be seen in Figure 2 and in the Appendix in Section A.7.
Coulomb GANs tend to outperform standard GAN approaches like BEGAN and DCGAN, but are outperformed by the Improved
Wasserstein GAN.
However it is important to note that the Improved Wasserstein GAN used a more advanced network architecture based on ResNet blocks (Gulrajani et al., 2017), which we could not replicate due to runtime constraints.
Overall, the low FID of Coulomb GANs stem from the fact that the images show a wide variety of different samples.
E.g. on CelebA, Coulomb GAN exhibit a very wide variety of faces, backgrounds, eye colors and orientations.
To further investigate how much variation the samples generated by the Coulomb GAN contains, we followed the advice of
Arora and Zhang (Arora & Zhang, 2017) to estimate the support size of the generator’s distribution by checking how large a sample from the generator must be before we start generating duplicates.
We were able to generate duplicates with a probability of around 50 % when using samples of size 1024, which indicates that the support size learned by the Coulomb GAN would be around 1M.
This is a strong indication that the Coulomb GAN was able to spread out its samples over the whole target distribution.
A depiction is included in Figure 3, which also shows the nearest neighbor in the training set of the generated images, conﬁrming that the Coulomb GAN does not just memorize training images.
3. L ANGUAGE M ODELING
We repeated the experiments from Gulrajani et al.
(2017), where Improved Wasserstein GANs (WGAN-GP) were trained to produce text samples after being trained on the Google Billion Word data set (Chelba et al., 2013), using the same network architecture as in the original publication.
We use the Jensen-Shannon-divergence on 4-grams and 6-grams as an evaluation criterion.
The results are summarized in Table 1a.
[Table]
C ONCLUSION
The Coulomb GAN is a generative adversarial network with strong theoretical guarantees.
Our theoretical results show that the Coulomb GAN will be able to approximate the real distribution perfectly if the networks have sufﬁcient capacity and training does not get stuck in local minima.
Our results show that the potential ﬁeld used by the Coulomb GAN far outperforms MMD based approaches due to its low-dimensional Plummer kernel, which is better suited for modeling probability density functions, and is very effective at eliminating the mode collapse problem in GANs.
This is because our loss function forces the generated samples to occupy different regions of the learned distribution.
In practice, we have found that Coulomb GANs are able to produce a wide range of different samples.
However, in our experience, this sometimes leads to a small number of generated samples that are non-sensical interpolations of existing data modes.
While these are sometimes also present in other GAN models (Radford et al., 2016), we found that our model produces such images at a slightly higher rate.
This issue might be solved by ﬁnding better ways of learning the discriminator, as learning the correct potential ﬁeld is crucial for the Coulomb GAN’s performance.
We also observed that increasing the capacity of the discriminator seems to always increase the generative performance.
We thus hypothesize that the largest issue in learning Coulomb GANs is that the discriminator needs to approximate the potential ﬁeld
Φ very well in a high-dimensional space.
In summary, instead of directly optimizing a criterion based on local differences of densities which can exhibit many local minima, Coulomb GANs are based on a potential ﬁeld that has no local minima.
The potential ﬁeld is created by point charges in an analogy to electric ﬁeld in physics.
We have proved that if learning converges then it converges to the optimal solution if the samples can be moved freely.
We showed that Coulomb GANs avoid mode collapsing, model the target distribution more truthfully than standard GANs, and do not overlook high probability regions of the target distribution. A CKNOWLEDGMENTS
This work was supported by Zalando SE with Research Agreement 01/2016, Audi.
JKU Deep Learning Center, Audi Electronic Venture GmbH, IWT research grant IWT150865 (Exaptation), H project grant 671555 (ExCAPE) and NVIDIA Corporation.
The authors would like to thank Philipp
Renz for fruitful discussions.
A A PPENDIX A. E XAMPLE OF C ONVERGENCE TO M ODE C OLLAPSE IN C ONVENTIONAL GAN S
As an example of how a GAN can converge to a Nash Equilibrium that exhibits mode collapse, consider a target distribution that consists of two distinct/non-overlapping regions of support Cand C 2 that are distant from each other, i.e., the target probability is zero outside of C 1 and C 2 .
Further assume that 50 % of the probability mass is in C 1 and 50 % in C 2 .
Assume that the the generator has mode-collapsed onto C 1 , which contains 100 % of the generator’s probability mass.
In this situation, the optimal discriminator classiﬁes all points from C 2 as ”real“ (pertaining to the target distribution) by supplying an output of 1 for them ( 1 is the target for real samples and 0 the target for generated samples).
Within C 1 , the other region, the discriminator sees twice as many generated data points as real ones, as 100 % of the probability mass of the generator’s distribution is in C 1 , but only 50 % of the probability mass of the real data distribution.
So one third of the points seen by the discriminator in C 1 are real, the other 2 thirds are generated.
Thus, to minimize its prediction error for a proper objective (squared or cross entropy), the discriminator has to output 1 / 3 for every point from C 1 .
The optimal output is even independent of the exact form of the real distribution in C 1 .
The generator will match the shape of the target distribution locally.
If the shape is not matched, local gradients of the discriminator with respect to its input would be present and the generator would improve locally.
If local improvements of the generator are no longer possible, the shape of the target distribution is matched and the discriminator output is locally constant.
In this situation, the expected gradient of the discriminator is the zero vector, because it has reached an optimum.
Since the discriminator output is constant in C 1 (and C 2 ), the generator’s expected gradient is the zero vector, too.
The situation is also stable even though we still have random ﬂuctuations from the ongoing stochastic gradient (SGD) learning: whenever the generator produces data outside of (but close to) C 1 , the discriminator can easily detect this and push the generator’s samples back.
Inside C 1 , small deviations of the generator from the shape of the real distribution are detected by the discriminator as well, by deviating slightly from 1 / 3 .
Subsequently, the generator is pushed back to the original shape.
If the discriminator deviates from its optimum, it will also be forced back to its optimum.
So overall, the GAN learning reached a local Nash equilibrium and has converged in the sense that the parameters ﬂuctuate around the attractor point (ﬂuctuations depend on learning rate, sample size, etc.).
To achieve true mathematical convergence,
Heusel et al.
(2017) assume decaying learning rates to anneal the random ﬂuctuations, similar to
Robbins & Monro (1951) original convergence proof for SGD.
A. P ROOF OF T HEOREM
We ﬁrst recall Theorem 1:
Theorem (Convergence with low-dimensional Plummer kernel) .
For a , b ∈ R m , d (cid:54) m − 2 , and (cid:15) > 0 the densities p x ( . ) and p y ( . ) equalize over time when minimizing energy F with the lowdimensional Plummer kernel by gradient descent.
The convergence is faster for larger d .
In a ﬁrst step, we prove that for local maxima or local minima a of ρ , the expression sign ( ∇ ·
E ( a ))
=
sign ( ρ ( a )) holds for (cid:15) small enough.
For proving this equation, we apply the Laplace operator for spherical coordinates to the low-dimensional Plummer kernel.
Using the result, we see that the integral ∇ · E ( a ) = − (cid:82) ρ ( b ) ∇ 2 a k ( a , b ) d b is dominated by large negative values of ∇ 2 a k around a .
These negative values can even be decreased by decreasing (cid:15) .
Therefore we can ensure by a small enough (cid:15) that at each local minimum and local maximum a of ρ sign ( ˙ ρ ( a ))
= − sign ( ρ ( a )) .
Thus, the maximal and minimal points of ρ move toward zero.
In a second step, we show that new maxima or minima cannot appear and that the movement of Φ toward zero stops at zero and not earlier.
Since ρ is continuously differentiable, all points in environments of maxima and minima move toward zero.
Therefore the largest | ρ ( a ) | moves toward zero.
We have to ensure that moving toward zero does not converge to a point apart from zero.
We derive that the movement toward zero is lower bounded by ˙
ρ ( a ) = − sign ( ρ ( a )) λρ 2 ( a ) .
Thus, the movement slows down at ρ ( a ) = 0 .
Solving the differential equation and applying it to the maximum of the absolute value of ρ gives | ρ | max ( t )
= 1 / ( λt + ( | ρ | max (0)) − 1 ) .
Thus, ρ converges to zero over time.
Proof.
For d = m − 2 , we have ∇ 2 k ( a , b ) =
δ ( a − b ) , where the theorem has already been proved for (cid:15) small enough (Hochreiter & Obermayer, 2005).
At each local minimum and local maximum a of ρ we have ∇ a ρ ( a ) = 0 .
Using the product rule for the divergence operator, Eq.
(3) reduces to
[Equation]
The term ∇ · E ( a ) can be expressed as
[Equation]
We next consider ∇ 2 a k
( a , b ) for the low-dimensional Plummer kernel.
We deﬁne the spherical
Laplace operator in ( m − 1) dimensions as ∇ 2 S m − 1 , then the Laplace operator in spherical coordinates is (Proposition 2.5 in Frye & Efthimiou (Efthimiou & Frye, 2014)):
[Equation]
Note that ∇ 2 S m − 1 only has second order derivatives with respect to the angles of the spherical coordinates.
With r = (cid:107) a − b (cid:107) we obtain for the Laplace operator applied to the low-dimensional Plummer kernel:
[Equation]
and in particular
[Equation]
For d (cid:54) m − 2 we have (2 + d − m ) (cid:54) 0 , and obtain ∇
[Equation]
b
[Equation]
< 0 ,
[Equation]
16) and ∂ ∂r ∇ 2 k
[Equation]
b [Equation] [Equation] and ∂
[Equation]
Therefore, ∇ 2 k ( a , b ) is negative with minimum − md(cid:15) − ( d +2) at r = 0 and increasing with r and increasing with (cid:15) for d (cid:54) m − 4 .
For d = m − 3 we have to restrict in the following the sphere
S τ ( a ) to τ < √ m(cid:15) and ensure increase of ∇ 2 k ( a , b ) with (cid:15) .
If ρ ( b ) (cid:54) = 0 , then we deﬁne a sphere S τ ( a ) with radius τ around a for which holds sign ( ρ ( b ))
=
sign ( ρ ( a )) for each b ∈ S τ ( a ) .
Note that ∇ 2 k ( a , b ) is continuous differentiable.
We have
[Equation] [Equation]
We bound ∇ 2 k ( a , b ) by 0 > ∇ 2 k [Equation] a , b [Equation]
d
[Equation]
d [Equation] d [Equation]
d
[Equation]
Using τ , we now bound (cid:12)(cid:12)(cid:12)(cid:82) T \ S τ ( a ) [Equation] ( b ) ∇ 2 a k ( a , b ) d b (cid:12)(cid:12)(cid:12) independently from (cid:15) , since ρ is a difference of distributions.
For small enough (cid:15) we can ensure
[Equation] [Equation]
Therefore we have
[Equation]
Therefore we have at each local minimum and local maximum a of ρ
[Equation]
Therefore the maximal and minimal points of ρ move toward zero.
Since ρ is continuously differentiable as is the ﬁeld, also the points in an environment of the maximal and minimal points move toward zero.
Points that are not in an environment of the maximal or minimal points cannot become maximal points in an inﬁnitesimal time step.
Since the contribution of a environment S τ ( a ) dominates the integral Eq.
(19), for (cid:15) small enough there exists a positive 0 < λ globally for all minima and maxima as well as for all time steps for which holds:
[Equation]
The factor λ depends on k and on the initial ρ .
λ is proportional to d .
Larger d lead to larger | ∇· E ( a ) | since the maximum or minimum ρ ( a ) is upweighted.
There might exist initial conditions ρ for which λ → 0 , e.g. for inﬁnite many maxima and minima, but they are impossible in our applications.
Therefore maximal or minimal points approach zero faster or equal than given by
[Equation]
In particular this differential equation dominates the global maximum | ρ | max of | ρ ( . ) | .
Solving the differential equation gives that at least
[Equation]
[Equation]
Thus d inﬂuences the worst case rate of convergence, where larger d with d (cid:54) m − 2 leads to faster worst case convergence.
Consequently, ρ converges to the zero function over time, that is, p x ( . ) becomes equal to p y ( . ) .
A. P ROOF OF T HEOREM
We ﬁrst recall Theorem 2:
Theorem (Optimal Solution) .
If the pair ( D ∗ , G ∗ ) is a local Nash equilibrium for the Coulomb
GAN objectives, then it is the global Nash equilibrium, and no other local Nash equilibria exist, and
G ∗ has output distribution p x = p y .
Proof.
( D ∗ , G ∗ ) being in a local Nash equilibrium means that ( D ∗ , G ∗ ) fulﬁlls the two conditions
[Equation]
for some neighborhoods U ( D ∗ ) and U ( G ∗ ) .
For Coulomb GANs that means, D ∗ has learned the potential Φ induced by G ∗ perfectly, because L D is convex in D , thus if D ∗ is optimal within an neighborhood
U ( D ∗ ) , it must be the global optimum.
This means that G ∗ is directly minimizing
L G ( G ; D ) = − 12 E p z
(Φ( G ( z ))) .
The Coulomb potential energy is according to Eq.
[Equation]
Only the samples from p x stem from the generator, where p x ( a ) = (cid:82) δ ( a − G ( z ))
p z ( z )d z .
Here δ is the δ -distribution centered at zero.
The part of the energy which depends on the generator is
[Equation] [Equation]
Theorem 1 guarantees that there are no other local minima except the global one when minimizing
F . F has one minimum, F = 0 , which implies Φ( a ) =
and ρ ( a ) =
for all a , therefore also p y
=
p x according to Theorem 1.
Each Φ( a ) (cid:54) = 0 would mean there exist potential differences which in turn would cause forces on generator samples that allow to further minimize the energy.
Since we assumed that the generator can reach the minimum p y
= p x for any p y , it will be reached by local (stepwise) optimization of − 12 E p z (Φ( G ( z ))) with respect to G .
Since the pair ( D ∗ , G ∗ ) is optimal within their neighborhood, the generator has reached this minimum as there is not other local minimum than the global one.
Therefore G ∗ has model density p x with p y
= p x .
The convergence point is a global Nash equilibrium, because there is no approximation error and zero energy F =is a global minimum for discriminator and generator, respectively.
Theorem 1 ensures that other local Nash equilibria are not possible.
A. C OULOMB E QUATIONS IN THE CASE OF F INITE S AMPLES
GANs are sample-based, that is, samples are drawn from the model for learning (Hochreiter &
Obermayer, 2005; Gutmann & Hyvärinen, 2012).
Typically this is done in mini-batches, where each mini-batch consists of two sets of samples, the target samples Y =
{ y i | i = 1 . . .
N y } , and the model samples X
= { x i | i = 1 . . .
N x } .
For such ﬁnite samples, i.e. point charges, we have to use delta distributions to obtain unbiased estimates of the the model distribution p x ( . ) and the target distribution p y ( . ) :
ˆ p y ( a [Equation] Y ) =
[Equation]
N y
N y (cid:88) i = δ ( a [Equation] y i ) , [Equation]
x
N x (cid:88) i =
δ ( a − x i ) , ˆ ρ
[Equation]
where δ is the Dirac δ -distribution centered at zero.
These are unbiased estimates of the underlying distribution, as can be seen by:
[Equation]
In the rest of the paper, we will drop the explicit parameterization with X and Y for all estimates to unclutter notation, and instead just use the hat sign to denote estimates.
In the same fashion as for the distributions, when we use ﬁxed samples X and Y , we obtain the following unbiased estimates for the potential, energy and ﬁeld given by Eq.
(5), Eq.
(6), and Eq.
[Equation]
, ˆ E
[Equation]
y
[Equation]
34) ˆ E ( y
[Equation]
) = − N y ∇ y
i ˆ F ( ρ )
[Equation]
These are again unbiased, e.g.:
[Equation]
If we draw samples of inﬁnite size, all these expressions for a ﬁxed sample size lead to the equivalent statements for densities.
The sample-based formulation, that is, point charges in physical terms, can only have local energy minima or maxima at locations of samples (Dembo & Zeitouni, 1988).
Furthermore the ﬁeld lines originate and end at samples, therefore the ﬁeld guides model samples x toward real world samples y , as depicted in Figure 1.
The factors N y and N x in the last equations arise from the fact that − ∇ a F gives the force which is applied to a sample with charge.
A sample y i is positively charged with 1 /N y and follows − ∇ y i F while a sample x i is negatively charged with − 1 /N x
and therefore follows − ∇ x i F , too.
Thus, following the force induced on a sample by the ﬁeld is equivalent to gradient descent of the energy F with respect to samples y i and x i .
A. M IXTURE OF G AUSSIANS
We use the synthetic data set introduced by Lim & Ye (2017) to show that Coulomb GANs avoid mode collapse and that all modes of the target distribution are captured by the generative model.
This data set comprises 100K data points drawn from a Gaussian mixture model of 25 components which are spread out evenly in the range [ − 21 , 21] × [ − 21 , 21] , with each component having a variance of 1.
To make results comparable with Lim & Ye (2017), the Coulomb GAN used a discriminator network with 2 hidden layers of 128 units, however we avoided batch normalization by using the
ELU activation function (Clevert et al., 2016).
We used the Plummer kernel in 3 dimensions ( d = 3 ) with an epsilon of 3 ( (cid:15) = 3 ) and a learning rate of 0.01, both of which were exponentially decayed during the 1M update steps of the Adam optimizer.
As can be seen in Figure 4, samples from the learned Coulomb GAN very well approximate the target distribution.
All components of the original distribution are present at the model distribution at approximately the correct ratio, as shown in Figure 5.
Moreover, the generated samples are distributed approximately according to the same spread for each component of the real world distribution.
Coulomb GANs outperform other compared methods, which either fail to learn the distribution completely, ignore some of the modes, or do not capture the within-mode spread of a
Gaussian.
The Coulomb GAN is the only GAN approach that manages to avoid a within-cluster collapse leading to insufﬁcient variance within a cluster.
[Figure]
A. P SEUDOCODE FOR C OULOMB GAN S
The following gives the pseudo code for training GANs.
Note that when calculating the derivative of ˆΦ( a i ; X , Y ) , it is important to only derive with respect to a , and not wrt.
X , Y , even if it can happen that e.g. a ∈ X .
In frameworks that offer automatic differentiation such as Tensorﬂow or
Theano, this means stopping the possible gradient back-propagation through those parameters.
while Stopping criterion not met do
• Sample minibatch of N x training samples { x 1 , . . .
, x N x } from training set • Sample minibatch of N y generator samples { y 1 , . . .
, y N y } from the generator •
Calculate the gradient for the discriminator weights:
[Equation]
• Calculate the gradient for the generator weights:
[Equation]
• Update weights according to optimizer rule (e
[Equation]
g. Adam):
[Equation]
end while
AMPLES
Images from a Coulomb GAN after training on CelebA data set.
The low FID stems from the fact that the images show a wide variety of different faces, backgrounds, eye colors and orientations.
[Table]