Venue: ICLR 2018 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper formulates GAN as a Lagrangian of a primal convex constrained optimization problem. They then suggest to modify the updates used in the standard GAN training to be similar to the primal-dual updates typically used by primal-dual subgradient methods.

Technically, the paper is sound. It mostly leverages the existing literature on primal-dual subgradient methods to modify the GAN training procedure. I think this is a nice contribution that does yield to some interesting insights. However I do have some concerns about the way the paper is currently written and I find some claims misleading.

Prior convergence proofs: I think the way the paper is currently written is misleading. The authors quote the paper from Ian Goodfellow: “For GANs, there is no theoretical prediction as to
whether simultaneous gradient descent should converge or not.”. However, the f-GAN paper gave a proof of convergence, see Theorem 2 here: https://arxiv.org/pdf/1606.00709.pdf. A recent NIPS paper by (Nagarajan and Kolter, 2017) also study the convergence properties of simultaneous gradient descent. Another problem is of course the assumptions required for the proof that typically don’t hold in practice (see comment below).

Convex-concave assumption: In practice the GAN objective is optimized over the parameters of the neural network rather than the generative distribution. This typically yields a non-convex non-concave optimization problem. This should be mentioned in the paper and I would like to see a discussion concerning the gap between the theory and the practical algorithm.

Relation to existing regularization techniques: Combining Equations 11 and 13, the second terms acts as a regularizer that minimizes [\lapha f_1(D(x_i))]^2. This looks rather similar to some of the recent regularization techniques such as
Improved Training of Wasserstein GANs, https://arxiv.org/pdf/1704.00028.pdf
Stabilizing Training of Generative Adversarial Networks through Regularization, https://arxiv.org/pdf/1705.09367.pdf
Can the authors comment on this? I think this would also shed some light as to why this approach alleviates the problem of mode collapse.

Curse of dimensionality: Nonparametric density estimators such as the KDE technique used in this paper suffer from the well-known curse of dimensionality. For the synthetic data, the empirical evidence seem to indicate that the technique proposed by the authors does work but I’m not sure the empirical evidence provided for the MNIST and CIFAR-10 datasets is sufficient to judge whether or not the method does help with mode collapse. The inception score fails to capture this property. Could the authors explore other quantitative measure? Have you considered trying your approach on the augmented version of the MNIST dataset used in Metz et al. (2016) and Che et al. (2016)?

Experiments
Typo: Should say “The data distribution is p_d(x) = 1{x=1}”.

review_title: Interesting insights although gap between theory and practice should be discussed

final_score: rating: 7: Good paper, accept
review_content: In this paper, the authors study the relationship between training GANs and primal-dual subgradient methods for convex optimization. Their technique can be applied on top of existing GANs and can address issues such as mode collapse. The authors also derive a GAN variant similar to WGAN which is called the Approximate WGAN. Experiments on synthetic datasets demonstrate that the proposed formulation can avoid mode collapse. This is a strong contribution

In Table 2 the difference between inception scores for DCGAN and this approach seems significant to ignore. The authors should explain more possibly.
There is a typo in Page 2 – For all these varaints, -variants.

review_title: Important problem with well evaluated solution. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposed a framework to connect the solving of GAN with finding the saddle point of a minimax problem.
As a result, the primal-dual subgradient methods can be directly introduced to calculate the saddle point.
Additionally, this idea not only fill the relatviely lacking of theoretical results for GAN or WGAN, but also provide a new perspective to modify the GAN-type models.
But this saddle point model reformulation  in section 2 is quite standard, with limited theoretical analysis in Theorem 1.
As follows, the resulting algorithm 1 is also standard primal-dual method for a saddle point problem.
Most important I think, the advantage of considering GAN-type model as a saddle point model is that first--order methods can be designed to solve it. But the numerical experiments part seems to be a bit weak, because the MINST or CIFAR-10 dataset is not large enough to test the extensibility for large-scale cases. 
review_title: Clarity analysis, very good motivation, but relatively limited novelty

Meta-review: The paper makes a good theoretical contribution by formulating the GAN training as primal-dual subgradient method for convex optimization and providing convergence proof. The authors then propose a modified objective to standard GAN training, based on this formulation, that helps address the mode collapse issue.
One weak point of the paper as pointed out by reviewers is that that the experimental results are underwhelming and the approach may not scale well to high dimensional datasets / high-resolution images. Interestingly, the proposed approach is general enough to be applied to other GAN variants that may address this issue in future. I recommend acceptance.
Decision: Accept (Poster)