Title: TRAINING GENERATIVE ADVERSARIAL NETWORKS VIA PRIMAL-DUAL SUBGRADIENT METHODS: A LAGRANGIAN PERSPECTIVE ON

Abstract: We relate the minimax game of generative adversarial networks (GANs) to finding the saddle points of the Lagrangian function for a convex optimization problem, where the discriminator outputs and the distribution of generator outputs play the roles of primal variables and dual variables, respectively. This formulation shows the connection between the standard GAN training process and the primal-dual subgradient methods for convex optimization. The inherent connection does not only provide a theoretical convergence proof for training GANs in the function space, but also inspires a novel objective function for training. The modified objective function forces the distribution of generator outputs to be updated along the direction according to the primal-dual subgradient methods. A toy example shows that the proposed method is able to resolve mode collapse, which in this case cannot be avoided by the standard GAN or Wasserstein GAN. Experiments on both Gaussian mixture synthetic data and real-world image datasets demonstrate the performance of the proposed method on generating diverse samples.

1. INTRODUCTION:

Generative adversarial networks (GANs) are a class of game theoretical methods for learning data distributions. It trains the generative model by maintaining two deep neural networks, namely the discriminator network D and the generator network G. The generator aims to produce samples resembling real data samples, while the discriminator aims to distinguish the generated samples and real data samples.

The standard GAN training procedure is formulated as the following minimax game:

EQUATION

where p d (x) is the data distribution and p z (z) is the noise distribution. The generated samples G(z) induces a generated distribution p g (x). Theoretically, the optimal solution to (1) is p * g = p d and D * (x) = 1/2 for all x in the support of data distribution.

In practice, the discriminator network and the generator network are parameterized by θ θ θ d and θ θ θ g , respectively. The neural network parameters are updated iteratively according to gradient descent. In particular, the discriminator is first updated either with multiple gradient descent steps until convergence or with a single gradient descent step, then the generator is updated with a single descent step. However, the analysis of the convergence properties on the training approaches is challenging, as noted by Ian Goodfellow in (Goodfellow, 2016) , "For GANs, there is no theoretical prediction as to whether simultaneous gradient descent should converge or not. Settling this theoretical question, and developing algorithms guaranteed to converge, remain important open research problems.". There have been some recent studies on the convergence behaviours of GAN training (Nowozin et al., 2016; Li et al., 2017b; Heusel et al., 2017; Nagarajan & Kolter, 2017; Mescheder et al., 2017) .

The simultaneous gradient descent method is proved to converge assuming the objective function is convex-concave in the network parameters (Nowozin et al., 2016) . The local stability property is established in (Heusel et al., 2017; Nagarajan & Kolter, 2017) .

One notable inconvergence issue with GAN training is referred to as mode collapse, where the generator characterizes only a few modes of the true data distribution (Goodfellow et al., 2014; Li et al., 2017b) . Various methods have been proposed to alleviate the mode collapse problem. Feature matching for intermediate layers of the discriminator has been proposed in (Salimans et al., 2016) . In (Metz et al., 2016) , the generator is updated based on a sequence of previous unrolled discriminators. A mixture of neural networks are used to generate diverse samples (Tolstikhin et al., 2017; Hoang et al., 2017; Arora et al., 2017) . In (Arjovsky & Bottou, 2017) , it was proposed that adding noise perturbation on the inputs to the discriminator can alleviate the mode collapse problem. It is shown that this training-with-noise technique is equivalent to adding a regularizer on the gradient norm of the discriminator (Roth et al., 2017) . The Wasserstein divergence is proposed to resolve the problem of incontinuous divergence when the generated distribution and the data distribution have disjoint supports (Arjovsky et al., 2017; Gulrajani et al., 2017) . Mode regularization is used in the loss function to penalize the missing modes (Che et al., 2016; Srivastava et al., 2017) . The regularization is usually based on heuristics, which tries to minimize the distance between the data samples and the generated samples, but lacks theoretical convergence guarantee.

In this paper, we formulate the minimax optimization for GAN training (1) as finding the saddle points of the Lagrangian function for a convex optimization problem. In the convex optimization problem, the discriminator function D(•) and the probabilities of generator outputs p g (•) play the roles of the primal variables and dual variables, respectively. This connection not only provides important insights in understanding the convergence of GAN training, but also enables us to leverage the primal-dual subgradient methods to design a novel objective function that helps to alleviate mode collapse. A toy example reveals that for some cases when standard GAN or WGAN inevitably leads to mode collapse, our proposed method can effectively avoid mode collapse and converge to the optimal point.

In this paper, we do not aim at achieving superior performance over other GANs, but rather provide a new perspective of understanding GANs, and propose an improved training technique that can be applied on top of existing GANs. The contributions of the paper are as follows:

• The standard training of GANs in the function space is formulated as primal-dual subgradient methods for solving convex optimizations.

• This formulation enables us to show that with a proper gradient descent step size, updating the discriminator and generator probabilities according to the primal-dual algorithms will provably converge to the optimal point.

• This formulation results in a novel training objective for the generator. With the proposed objective function, the generator is updated such that the probabilities of generator outputs are pushed to the optimal update direction derived by the primal-dual algorithms.

Experiments have shown that this simple objective function can effectively alleviate mode collapse in GAN training.

• The convex optimization framework incorporates different variants of GANs including the family of f -GAN (Nowozin et al., 2016) and an approximate variant of WGAN. For all these variants, the training objective can be improved by including the optimal update direction of the generated probabilities.

2. PRIMAL-DUAL SUBGRADIENT METHODS FOR CONVEX OPTIMIZATION:

In this section, we first describe the primal-dual subgradient methods for convex optimization. Later, we explicitly construct a convex optimization and relate the subgradient methods to standard GAN training. Consider the following convex optimization problem:

EQUATION

where x ∈ R k is a length-k vector, X is a convex set, and f i (x), i = 0 • • • , , are concave functions mapping from R k to R. The Lagrangian function is calculated as

EQUATION

In the optimization problem, the variables x ∈ R k and λ λ λ ∈ R + are referred to as primal variables and dual variables, respectively. The primal-dual pair (x * , λ λ λ * ) is a saddle-point of the Lagrangian fuction, if it satisfies:

EQUATION

Primal-dual subgradient methods have been widely used to solve the convex optimization problems, where the primal and dual variables are updated iteratively, and converge to a saddle point (Nedić & Ozdaglar, 2009; Komodakis & Pesquet, 2015) .

There are two forms of algorithms, namely dual-driven algorithm and primal-dual-driven algorithm.

For both approaches, the dual variables are updated according to the subgradient of L(x(t), λ λ λ(t)) with respect to λ λ λ(t) at each iteration t. For the dual-driven algorithm, the primal variables are updated to achieve maximum of L(x, λ λ λ(t)) over x. For the primal-dual-driven algorithm, the primal variables are updated according to the subgradient of L(x(t), λ λ λ(t)) with respect to x(t). The iterative update process is summarized as follows:

EQUATION

where P X (•) denotes the projection on set X and (x) + = max(x, 0).

The following theorem proves that the primal-dual subgradient methods will make the primal and dual variables converge to the optimal solution of the convex optimization problem.

Theorem 1 Consider the convex optimization (2). Assume the set of saddle points is compact. Suppose f 0 (x) is a strictly concave function over x ∈ X and the subgradient at each step is bounded.

There exists some step size α (t) such that both the dual-driven algorithm and the primal-dual-driven algorithm yield x (t) → x * and λ λ λ (t) → λ λ λ * , where x * is the solution to (2), and λ λ λ * satisfies

EQUATION

Proof: See Appendix 7.1.

3. TRAINING GAN VIA PRIMAL-DUAL SUBGRADIENT METHODS:



3.1. GAN AS A CONVEX OPTIMIZATION:

We explicitly construct a convex optimization problem and relate it to the minimax game of GANs.

We assume that the source data and generated samples belong to a finite set {x 1 , • • • , x n } of arbitrary size n. The extension to uncountable sets can be derived in a similar manner (Luenberger, 1997) .

The finite case is of particular interest, because any real-world data has a finite size, albeit the size could be arbitrarily large.

We construct the following convex optimization problem:

EQUATION

)

where D is some convex set. The primal variables are

D = (D 1 , • • • , D n ), where D i is defined as D i = D(x i ). Let p g = (p g (x 1 ), • • • , p g (x n ))

, where p g (x i ) is the Lagrangian dual associated with the i-th constraint. The Lagrangian function is thus

EQUATION

When D = {D : 0 ≤ D i ≤ 1, ∀i}, finding the saddle points for the Lagrangian function is exactly equivalent to solving the GAN minimax problem(1). This inherent connection enables us to utilize the primal-dual subgradient methods to design update rules for D(x) and p g (x) such that they converge to the saddle points. The following theorem provides a theoretical guideline for the training of GANs.

Theorem 2 Consider the Lagrangian function given by ( 9) with D = {D : ≤ D i ≤ 1 -, ∀i}, where 0 < < 1/2. If the discriminator and generator have enough capacity, and the discriminator output and the generated distribution are updated according to the primal-dual update rules (5) and ( 6) with (x, λ λ λ) = (D, p g ), then p g (•) converges to p d (•).

Proof: The optimization problem ( 8) is a particularized form of ( 2), where

f 0 (•) = n i=1 p d (x i ) log(D i ), f i (•) = log(1 -D i ) and X = [ , 1 -] n .

The objective function is strictly concave over D. Moreover, since D is projected onto the compact set [ , 1 -] at each iteration t, the subgradients ∂f i (D (t) ) are bounded. The assumptions of Theorem 1 are satisfied.

Since the constraint (8b) gives an upper bound of D i ≤ 1/2, the solution to the above convex optimization is obviously

D * i = 1/2, for all i = 1, • • • , n.

Since the problem is convex, the optimal primal solution is the primal saddle point of the Lagrangian function (Bertsekas, 1999, Chapter 5) . Moreover, any primal-dual saddle point

(D * , p * g ) satisfies L(D * , p * g ) = max D∈D L D, p * g . Since D * is strictly inside D, we have ∂ D L(D * , p * g ) = 0. Since ∂ Di L(D * , p * g ) = 2p d (x i ) -2p * g (x i ), we have p * g = p d

, and the saddle point is unique. By Theorem 1, the primal-dual update rules will guarantee convergence of D (t) , p It can be seen that the standard training of GAN corresponds to either dual-driven algorithm (Nowozin et al., 2016) or primal-dual-driven algorithm (Arjovsky et al., 2017; Goodfellow et al., 2014) . A natural question arises: Why does the standard training fail to converge and lead to mode collapse? As will be shown later, the underlying reason is that standard training of GANs in some cases do not update the generated distribution according to (6). Theorem 2 inspires us to propose a training algorithm to tackle this issue.

3.2. ALGORITHM DESCRIPTION:

First, we present our training algorithm. Later, we will use a toy example to give intuitions of why our algorithm is effective to avoid mode collapse.

The algorithm is described in Algorithm 1. The maximum step of discriminator update is k 0 . In the context of primal-dual-driven algorithms, k 0 = 1. In the context of dual-driven algorithms, k 0 is some large constant, such that the discriminator is updated till convergence at each training epoch. The update of the discriminator is the same as standard GAN training. The main difference is the modified loss function for the generator update (13). The intuition is that when the generated samples have disjoint support from the data, the generated distribution at the data support may not be updated using standard training. This is exactly one source of mode collapse. Ideally, the modified loss function will always update the generated probabilities at the data support along the optimal direction.

The generated probability mass at x is p g

(x) = 1 m m i=1 1{G(z i ) = x},

where 1{•} is the indicator function. The indicator function is not differentiable, so we use a continuous kernel to approximate it. Define

EQUATION

)

where σ is some positive constant. The constant σ is also called bandwidth for kernel density estimation. The empirical generated distribution is thus approximately calculated as (17). There while the stopping criterion is not met do Sample minibatch m 1 data samples

x 1 , • • • , x m1 . Sample minibatch m 2 noise samples z 1 , • • • , z m2 . for k = 1, • • • , k 0 do

Update the discriminator parameters with gradient ascent:

EQUATION

)

end for Update the target generated distribution as:

EQUATION

)

where α is some step size and

EQUATION

)

With pg (x i ) fixed, update the generator parameters with gradient descent:

EQUATION

)

end while are different bandwidth selection methods (Botev et al., 2010; Hall et al., 1991) . It can be seen that as σ → 0, k σ (x -y) tends to the indicator function, but it will not give large enough gradients to far areas that experience mode collapse. A larger σ implies a coarser quantization of the space in approximating the distribution. In practical training, the kernel bandwidth can be set larger at first and gradually decreases as the iteration continues.

By the dual update rule (6) , the generated probability of every x i should be updated as

EQUATION

)

This motivates us to add the second term of (13) in the loss function, such that the generated distribution is pushed towards the target distribution (15).

Although having good convergence guarantee in theory, the non-parametric kernel density estimation of the generated distribution may suffer from the curse of dimension. Previous works combining kernel learning and the GAN framework have proposed methods to scale the algorithms to deal with high-dimensional data, and the performances are promising (Li et al., 2015; 2017a; Sinn & Rawat, 2017) . One common method is to project the data onto a low dimensional space using an autoencoder or a bottleneck layer of a pretrained neurual network, and then apply the kernel-based estimates on the feature space. Using this approach, the estimated probability of x i becomes

EQUATION

where f φ (.) is the projection of the data to a low dimensional space. We will leave the work of generating high-resolution images using this approach as future work.

3.3. INTUITION OF AVOIDING MODE COLLAPSE:

Mode collapse occurs when the generated samples have a very small probability to overlap with some families of the data samples, and the discriminator D(•) is locally constant around the region of the generated samples. We use a toy example to show that the standard training of GAN and Wasserstein may fail to avoid mode collapse, while our proposed method can succeed.

Claim 1 Suppose the data distribution is p d (x) = 1{x = 1}, and the initial generated distribution is p g (x) = 1{x = 0}. The discriminator output D(x) is some function that is equal to zero for |x -0| ≤ δ and is equal to one for |x -1| ≤ δ, where 0 < δ < 1/2. Standard training of GAN and WGAN leads to mode collapse.

Proof: We first show that the discriminator is not updated, and then show that the generator is not updated during the standard training process.

In standard training of GAN and WGAN, the discriminator is updated according to the gradient of (10). For GAN, since 0 ≤ D(x) ≤ 1, the objective funtion for the discriminator is at most zero, i.e.,

EQUATION

which is achieved by the current D(x) by assumption.

For WGAN, the optimal discrminator output D(x) is some 1-Lipschitz function such that

EQUATION

where ( 19) is due to the Lipschitz condition |D(1) -D(0)| ≤ 1. The current D(x) is obviously optimal. Thus, for both GAN and WGAN, the gradient of the loss function with respect to θ θ θ d is zero and the discriminator parameters are not updated.

On the other hand, in standard training, the generator parameters θ θ θ g are updated with only the first term of (13). By the chain rule,

EQUATION

where ( 21) is due to the assumption that D(x) is locally constant for x = 0. Therefore, the generator and the discriminator reach a local optimum point. The generated samples are all zeros.

In our proposed training method, when x = 1, the optimal update direction is given by ( 11), where pg is a large value because D(1) = 1. Therefore, by (13), the second term in the loss function is very large, which forces the generator to generate samples at G(z) = 1. As the iteration continues, the generated distribution gradually converges to data distribution, and D(x) gradually converges to 1/2, which makes ∂ pg(x) L(D(x), p g (x)) = log(2(1 -D(x))) become zero. The experiment in Section 5 demonstrates this training dynamic.

In this paper, the standard training of GANs in function space has been formulated as primal-dual updates for convex optimization. However, the training is optimized over the network parameters in practice, which typically yields a non-convex non-concave problem. Theorem 2 tells us that as long as the discriminator output and the generated distribution are updated according to the primal-dual update rule, mode collapse should not occur. This insight leads to the addition of the second term in the modified loss function for the generator (13). In Section 5, experiments on the above-mentioned toy example and real-world datasets show that the proposed training technique can greatly improve the baseline performance.

4. VARIANTS OF GANS:

Consider the following optimization problem:

EQUATION

Table 1 : Variants of GANs under the convex optimization framework.

Divergence metric

f 0 (D i ) f 1 (D i ) D * i Kullback-Leibler log(D i ) 1 -D i p d (xi) pg(xi) Reverse KL -D i log D i pg(xi) p d (xi) Pearson χ 2 D i -1 4 D 2 i -D i 2(p d (xi)-pg(xi)) pg(xi) Squared Hellinger χ 2 1 -D i 1 -1/D i pg(xi) p d (xi) Jensen-Shannon log(D i ) log(1 -D i ) -log(1/2) p d (xi) p d (xi)+pg(xi) Approximate WGAN D i -D 2 i -D i p d (xi)-pg(xi) 2 p d (xi)

Other metric

-1 2 D 2 i + D i D i -2 p d (xi)+pg(xi) p d (xi)

where f 0 (•) and f 1 (•) are concave functions. Compared with the generic convex optimization problem (2), the number of constraint functions is set to be the variable alphabet size, and the constraint functions are

f i (D) = f 1 (D i ), i = 1, • • • , n.

The objective and constraint functions in ( 22) can be tailored to produce different GAN variants. For example, Table 1 shows the large family of f -GAN (Nowozin et al., 2016) . The last row of Table 1 gives a new realization of GAN with a unique saddle point of D * (x) = 2 and p g (x) = p d (x).

We also derive a GAN variant similar to WGAN, which is named "Approximate WGAN". As shown in Table 1 , the objective and constraint functions yield the following minimax problem:

EQUATION

where is an arbitrary positive constant. The augmented term D 2 (x) is to make the objective function strictly concave, without changing the original solution. It can be seen that this problem has a unique saddle point p * g (x) = p d (x). As tends to 0, the training objective function becomes identical to WGAN. The optimal D(x) for WGAN is some Lipschitz function that maximizes E x∼p d (x) {D(x)} -E x∼pg(x) {D(x)}, while for our problem is D * (x) = 0. Weight clipping can still be applied, but serves as a regularizer to make the training more robust (Merolla et al., 2016) .

The training algorithms for these variants of GANs follow by simply changing the objective function f 0 (•) and constraint function f 1 (•) accordingly in Algorithm 1.

5. EXPERIMENTS:

5.1 SYNTHETIC DATA Fig. 1 shows the training performance for a toy example. The data distribution is p g (x) = 1{x = 1}. The inital generated samples are concentrated around x = -3.0. The details of the neural network parameters can be seen in Appendix 7.3. Fig. 1a shows the generated samples in the 90 quantile as the training iterates. After 8000 iterations, the generated samples from standard training of GAN and WGAN are still concentrated around x = -3.0. As shown in Fig. 1c and 1d , the discrminators hardly have any updates throughout the training process. Using the proposed training approach, the generated samples gradually converge to the data distribution and the discriminator output converges to the optimal solution with D(1) = 1/2. Fig. 2 shows the performance of the proposed method for a mixture of 8 Gaussain data on a circle. While the original GANs experience mode collapse (Nguyen et al., 2017; Metz et al., 2016) , our proposed method is able to generate samples over all 8 modes. In the training process, the bandwidth of the Gaussian kernel ( 14) is inialized to be σ 2 = 0.1 and decreases at a rate of 0.8 t 2000 , where t is the iteration number. The generated samples are dispersed initially, and then gradually converge to the Gaussian data samples. Note that our proposed method involves a low complexity with a simple regularization term added in the loss function for the generator update. 

5.2. REAL-WORLD DATASETS:

We also evaluate the performance of the proposed method on two real-world datasets: MNIST and CIFAR-10. Please refer to the appendix for detailed architectures. Inception score (Salimans et al., 2016) is employed to evaluate the proposed method. It applies a pretrained inception model to every generated image to get the conditional label distribution p(y|x). The Inception score is calculated as exp (E x {KL(p(y|x) p(y)}). It measures the quality and diversity of the generated images.

5.2.1. MNIST:

The MNIST dataset contains 60000 labeled images of 28 × 28 grayscale digits. We train a simple LeNet-5 convolutional neural network classifier on MNIST dataset that achieves 98.9% test accuracy, and use it to compute the inception score. The proposed method achieves an inception score of 9.8, while the baseline method achieves an inception score of 8.8. The examples of generated images are shown in Fig. 3 . The generated images are almost indistinguishable from real images.

We further evaluated our algorithm on an augmented 1000-class MNIST dataset to further demonstrate the robustness of the proposed algorithm against mode collapse problem. More details of the experimental results can be found in the Appendix.

5.2.2. CIFAR-10:

CIFAR is a natural scene dataset of 32 × 32. We use this dataset to evaluate the visual quality of the generated samples. Method Score Real data 11.24 ± 0.16 WGAN (Arjovsky et al., 2017) 3.82 ± 0.06 MIX + WGAN (Arora et al., 2017) 4.04 ± 0.07 Improved- GAN (Salimans et al., 2016) 4.36 ± 0.04 ALI (Dumoulin et al., 2016) 5.34 ± 0.05 DCGAN (Radford et al., 2015) 6.40 ± 0.05

Proposed method 4.53 ± 0.04 

7. APPENDIX:

7.1 PROOF OF THEOREM 1

The proof of convergence for dual-driven algorithms can be found in (Bertsekas & Tsitsiklis, 1989 , Chapter 3).

The primal-dual-driven algorithm for continuous time update has been studied in (Feijer & Paganini, 2010) . Here, we show the convergence for the discrete-time case.

We choose a step size α(t) that satisfies

EQUATION

Let z(t) = [x(t), λ λ λ(t)] T be a vector consisting of the primal and dual variables at the t-th iteration.

The primal-dual-driven update can be expressed as:

EQUATION

where

EQUATION

and

EQUATION

Since the subgradient is bounded by assumption, there exists M > 0 such that ||T (•)|| 2 2 < M , where ||.|| 2 stands for the L 2 norm. function L(•, •), (40) implies that there exists some δ > 0 such that for every t ≥ N 0 , T T (t)(z(t)z * (t)) < -δ. In this case, by summing (36) over N 0 ≤ t ≤ n -1, we have

EQUATION

Note that ||z(N 0 ) -z * (N 0 )|| 2 2 is bounded. By the choice of the step size (24), we have ||z(n)z * (n)|| 2 2 tends to -∞, which is contradicted with the fact that ||z(n) -z * (n)|| 2 2 ≥ 0. Therefore, there are infinitely many z(t) in A .

Consequently, we can find a large enough N 1 such that 2M In other words, for any > 0, there exists N 1 > 0 such that for all t ≥ N 1 , ||z(t) -z * (t)|| 2 ≤ 2 . Since it holds for all , it implies that there are infinitely many z(t) that belong to the saddle points.

The set of saddle points is compact by assumption. By the Bolzano-Weierstrass theorem, there must exist a subsequence of {z(t n )} that converges to a saddle point z 0 . For such subsequence, there exists some large enough n 0 such that t n0 ≥ N 1 and ||z(t n ) -z 0 || 2 2 ≤ , for every n ≥ n 0 . Since (42) holds for any saddle point z * (t), we replace z * (t) by z 0 and sum (42) over t n0 ≤ t ≤ n -1 to obtain

EQUATION

≤ ||z(t n0 ) -z 0 || 2 2 + 2M

n-1 t=N1 α 2 (t) (49)

EQUATION

)

This means that for any , there exists N 2 = t n0 such that for every t ≥ N 2 , ||z(t) -z 0 || 2 2 ≤ . That concludes the proof that z(t) converges to a saddle point.

7.2. 1000 CLASS MNIST DATASET:

We use an augmented version of MNIST dataset similar to the experiment conducted in (Che et al., 2016; Metz et al., 2016) . Each image in this dataset is created by randomly choosing three letter images from MNIST dataset. The three images are stacked as the R,G, and B channels into a color image. This dataset has 1000 distinct modes, corresponding to each combination of the ten MNIST classes in each channel.

We train a GAN and a classifier on this dataset. For each generated image, we apply the classifier to determine its label. We compute two metrics on this dataset. the number of modes the GAN generates, and the inception score computed using the classifier. We use the same architecture as (Metz et al., 2016) in our experiment. The result is shown in Table 7 .2. We find that the proposed method achieves a much better performance than unrolled GAN with 5 steps and comparable performance to unrolled GAN with 10 steps in terms of the number of predicted modes. However, since our method does not involve unrolling step, it is much more computationally efficient. Notice that although (Che et al., 2016) generates much more modes, it uses a more complex architecture, and such architecture is known to contribute to mode collapse avoidance on the 1000 Class MNIST dataset (Metz et al., 2016) . Compared to the baseline that does not use the second regularization term in (13), the proposed method achieves better inception score, and it generates more modes.

:

CONCLUSIONIn this paper, we propose a primal-dual formulation for generative adversarial learning. This formulation interprets GANs from the perspective of convex optimization, and gives the optimal update of the discriminator and the generated distribution with convergence guarantee. By framing different variants of GANs under the convex optimization framework, the corresponding training algorithms can all be improved by pushing the generated distribution along the optimal direction. Experiments on two synthetic datasets demonstrate that the proposed formulation can effectively avoid mode collapse. It also achieves competitive quantitative evaluation scores on two benchmark real-world image datasets.

Figures and Tables:

the primal-dual saddle point (D * , p * g ).

Training GAN via Primal-Dual Subgradient Methods Initialization: Choose the objective function f 0 (•) and constraint function f 1 (•) according to the GAN realization. For the original GAN based on Jensen-Shannon divergence, f 0 (D) = log (D) and f 1 (D) = log(2(1 -D)).

Figure 1: Performance of a toy example. Figure (a) shows the generated samples for different GANs.Figure (b) shows the discriminator output D(x) for GANs trained using the proposed method.Figure (c) and (d) show the discriminator output D(x) for standard GANs and WGANs.

Figure 1: Performance of a toy example. Figure (a) shows the generated samples for different GANs.Figure (b) shows the discriminator output D(x) for GANs trained using the proposed method.Figure (c) and (d) show the discriminator output D(x) for standard GANs and WGANs.

Figure 1: Performance of a toy example. Figure (a) shows the generated samples for different GANs.Figure (b) shows the discriminator output D(x) for GANs trained using the proposed method.Figure (c) and (d) show the discriminator output D(x) for standard GANs and WGANs.

Figure 3: Examples of generated images using MNIST and CIFAR dataset.

t=N1 α 2 (t) ≤ and ||z(N 1 )z * (N 1 )|| ≤ . Summing (42) over N 1 ≤ t ≤ n -1 we have ||z(n) -z * (n)|| 2 2 ≤ ||z(N 1 ) -z * (N 1 )|| 2

Table 2 shows the inception scores of different GAN models on CIFAR-10 dataset. The inception score of the proposed model is much better than the baseline method WGAN

Table content: <table><tr><td>MNIST</td><td>CIFAR</td></tr></table>

Inception scores on CIFAR-10 dataset. that uses similar network architecture and training method. Note that although DCGGAN achieves a better score, it uses a more complex network architecture. Examples of the generated images are shown in Fig. 3. Angelia Nedić and Asuman Ozdaglar. Subgradient methods for saddle-point problems. Journal of optimization theory and applications, 142(1):205-228, 2009. Tu Dinh Nguyen, Trung Le, Hung Vu, and Dinh Phung. Dual discriminator generative adversarial nets. arXiv preprint arXiv:1709.03831, 2017. Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. Stabilizing training of generative adversarial networks through regularization. arXiv preprint arXiv:1705.09367, 2017. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In Advances in Neural Information Processing Systems, pp. 2234-2242, 2016. Mathieu Sinn and Ambrish Rawat. Towards consistency of adversarial training for generative models. arXiv preprint arXiv:1705.09199, 2017. Akash Srivastava, Lazar Valkov, Chris Russell, Michael Gutmann, and Charles Sutton. VEEGAN: Reducing mode collapse in GANs using implicit variational learning. arXiv preprint arXiv:1705.07761, 2017.

Table content: <table><tr><td>Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers</td></tr><tr><td>using variational divergence minimization. In Advances in Neural Information Processing Systems,</td></tr><tr><td>pp. 271-279, 2016.</td></tr><tr><td>Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep</td></tr><tr><td>convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.</td></tr><tr><td>Kevin Roth,</td></tr></table>

Peer review (iclr2018):
 This paper formulates GAN as a Lagrangian of a primal convex constrained optimization problem. They then suggest to modify the updates used in the standard GAN training to be similar to the primal-dual updates typically used by primal-dual subgradient methods. Technically, the paper is sound. It mostly leverages the existing literature on primal-dual subgradient methods to modify the GAN training procedure. I think this is a nice contribution that does yield to some interesting insights. However I do have some concerns about the way the paper is currently written and I find some claims misleading. Prior convergence proofs: I think the way the paper is currently written is misleading. The authors quote the paper from Ian Goodfellow: “For GANs, there is no theoretical prediction as to whether simultaneous gradient descent should converge or not.”. However, the f-GAN paper gave a proof of convergence, see Theorem 2 here: https://arxiv.org/pdf/1606.00709.pdf. A recent NIPS paper by (Nagarajan and Kolter, 2017) also study the convergence properties of simultaneous gradient descent. Another problem is of course the assumptions required for the proof that typically don’t hold in practice (see comment below). Convex-concave assumption: In practice the GAN objective is optimized over the parameters of the neural network rather than the generative distribution. This typically yields a non-convex non-concave optimization problem. This should be mentioned in the paper and I would like to see a discussion concerning the gap between the theory and the practical algorithm. Relation to existing regularization techniques: Combining Equations 11 and 13, the second terms acts as a regularizer that minimizes [.lapha f_1(D(x_i))]^2. This looks rather similar to some of the recent regularization techniques such as Improved Training of Wasserstein GANs, https://arxiv.org/pdf/1704.00028.pdf Stabilizing Training of Generative Adversarial Networks through Regularization, https://arxiv.org/pdf/1705.09367.pdf Can the authors comment on this? I think this would also shed some light as to why this approach alleviates the problem of mode collapse. Curse of dimensionality: Nonparametric density estimators such as the KDE technique used in this paper suffer from the well-known curse of dimensionality. For the synthetic data, the empirical evidence seem to indicate that the technique proposed by the authors does work but I’m not sure the empirical evidence provided for the MNIST and CIFAR-10 datasets is sufficient to judge whether or not the method does help with mode collapse. The inception score fails to capture this property. Could the authors explore other quantitative measure? Have you considered trying your approach on the augmented version of the MNIST dataset used in Metz et al. (2016) and Che et al. (2016)? Experiments Typo: Should say “The data distribution is p_d(x) = 1{x=1}”.