Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: https://openreview.net/pdf?id=SkxaueHFPB

The paper has some interesting ideas but I don’t think any of them are fully fleshed out.

I find the reporting of Inception Score highly suspect. The authors choose WGAN-GP as a baseline and report scores of ~4.5 vs ~5.5 with their modification. However the WGAN-GP paper reports an IS of 7.86 on CIFAR. Furthermore, current GAN SOTA on CIFAR is approaching IS=9. I am not making the argument that the authors ought to demonstrate SOTA results, however they should at least present results which are consistent with the published results of their chosen baseline.

The authors then make this statement:
“Thus, its superior performance supports the claim that ICR is the appropriate form of regularization for GANs. We emphasize that in our experiments we did not perform any architecture or hyperparameter tuning, and instead use a model intended to be used with WGAN gradient penalty”
This does not hold, since the numbers reported are far below the actual baseline.

Besides this major point, I am unconvinced by some of the mathematical statements in the paper. Much of the mathematical details are deferred to the original CGD paper. It is not really particularly reader-friendly to defer that to the CGD paper since they are seemingly crucial to the discussion here. Relative to the CGD paper some signs have been flipped and some definitions appear to be used in subtly different ways which makes for a very difficult read. I feel that far too much has been left as an exercise to the reader.

Concretely my concerns refer to the main discussion of the effect of the CGD as a regularizer:

The authors state:
“If some of the singular values of Dxy are very large, this amounts to approximately restricting the update to the orthogonal complement of the corresponding singular vectors” 

I don’t see how this is the case. The terms Dxy/Dyx aren’t really introduced or defined anywhere in this work. Assuming is the transpose of the other (?) then the update direction is:
A + B where A=inv(S) grad_x and B = inv(S)Dxy grad_y (and S = I + Dxy Dyx). So we have a term which is being affected by the smallest singular values of S and a term which is the orthogonal projection of grad_y onto Dxy, alternatively the ridge-regression fit of grad_y on Dxy which would attenuate directions corresponding to the *small* singular values (as is well known from the theory of ridge regularizers). I feel like there is much more to say here than what is discussed in the paper in very vague terms. 

Of course the effective rank of S, or the rate of decay of its singular values is crucially important. In practise I would assume the smaller SVs of Dxy to be difficult to estimate or the matrix to be rank deficient in which case they would simply be unity in the inverse whereas the directions corresponding to large singular values would be attenuated. So in this case it is the regularized orthogonal complement but its not clear (if the matrix is not full rank) that it is a meaningful direction (and again this is all highly dependent on the effective rank, too).

Further on it is mentioned: “For smoothly varying singular vectors, this can be though of as approximately constraining the trajectories to a manifold of robust play”.

First it is not at all clear to me what “smoothly varying singular vectors” are. Varying with respect to what? Secondly, the “manifold of robust play” has not been defined anywhere. 

Finally, figure 3 is quite bizarre to me. None of the quantities have been rigorously defined and so it seems like the relative effect of each of the arrows and the manifold have been drawn arbitrarily in order to fit the story, rather than to actually illuminate the true behaviour in an intuitive manner. 
B (defined above) has a very clear interpretation as a least-squares fit so I figure that any geometric interpretation of the CGD update direction could start from there.
review_title: Official Blind Review #5

final_score: rating: 8: Accept
review_content: The paper analyzes instability in training GANs, relates it to Nash equilibria, and proposes a novel training set-up based on competitive nashless games. The solution is related to other recently proposed work, but the paper brings additional insights into understanding it.

The analysis on conditions that lead to divergence or convergence, and of the proposed solution, are interesting. I recommend accepting.

I have some basic knowledge of GANs but am not deeply familiar with the field. The paper was accessible to me on a high level. Especially compelling to me were sections 2 and 3. The empirical study also seemed to yield positive results.

Suggestions for improvement:
- Additional proofreading would be beneficial.
- The scale of the axes in figure 1 is not clear, making it a little less compelling.
- Inception score is used as the only evaluation metric for the generators. Perhaps this is standard in the field, although human ratings would seem more reliable to me.
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The paper presents a novel training methodology for GANs to improve stability. The resulting regularization, termed implicit competitive regularization, updates the parameters of both the generator and the discriminator to be robust to one another. A framework for practical application of this approach is described -- this is done by a local Taylor approximation of the loss and updating each model’s parameters to this approximate model’s nash equilibrium.  The method is shown to prevent overfitting and produce high-performing models with consistent training.

The approach and insights are reasonable and the problem is worthwhile to approach. The method is clear and the associated code is appreciated. The results are interesting in terms of describing the ICR property and demonstrating its performance. 

The paper gives background and intuition to solidifying the CGD update. Are there additional algorithmic approaches that are possible and potentially more efficient with this understanding in mind? This I believe would help solidify the paper and build beyond CGD.

Some additional results that could clarify the benefits:
- A primary contribution of the training approach is training consistency. The distribution over many training runs should be provided in figures.
- Clearly due to the additional gradient calls the approach is computationally slower, as shown in Figure 4. If each approach is trained for the same amount of time, how does the performance compare?
- One may expect that the update may result in more conservative updates and thus potentially lower-performing policies in the limit. If there iterations were instead log-scale to show performance in high training iterations, is there any loss of performance in top-performing runs? 
- Could a similar approach be used to allow safe gradient updates according to a risk over the opponent’s possible updates, e.g., via CVar? This may also be a stable training procedure as well with less conservatism.  

The paper should be proofread, there are several minor typos throughout, e.g.:
- “generators producing that produce good” -> generators that produce good
- “This game is very similar similar” -> repeated word
- “GAN trainin.”

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper presents a new way of regularization in Generative Adversarial Network (GAN). It is well known that a naive training of GAN can fail to converge. Although GAN is relatively a new concept, many papers tried to introduce a good way of stabilizing GAN training. I believe that this paper is addressing the stability issues in the most fundamental and effective ways. The paper utilizes Competitive Gradient Descent proposed by Schäfer and Anandkumar in 2019 in training GAN. The intuition is that both players should predict what their opponent is going to do. This results in a convergence point where each agent becomes robust against changes of the other agent. The performance of the new method was demonstrated on CIFAR10.

The paper is definitely interesting. If this method works as well as the authors claim, it can significantly improve the practicality of GAN. The paper is very readable and understandable but many small typos and grammar errors can be found in the text. This can be easily corrected by the authors.

However, the contribution of this paper is questionable. The original CGD paper already applies it to train a GAN.

I would also appreciate if the method is tested on multiple other data sets. 

Overall, the paper is well-written, technically correct and interesting enough for the venue. However, as I pointed out above, the contribution should be more clearly stated.


review_title: Official Blind Review #1

Meta-review: The paper proposes to study "implicit competitive regularization", a phenomenon borne of taking a more nuanced game theoretic perspective on GAN training, wherein the two competing networks are "model[ed] ... as agents acting with limited information and in awareness of their opponent". The meaning of this is developed through a series of examples using simpler games and didactic experiments on actual GANs. An adversary-aware variant employing a Taylor approximation to the loss. 

Reviewer assessment amounted to 3 relatively light reviews, two of which reported little background in the area, and one more in-depth review, which happened to also be the most critical. R1, R2, R3 all felt the contribution was interesting and valuable. R1 felt the contribution of the paper may be on the light side given the original competitive gradient descent paper, on which this manuscript leans heavily, included GAN training (the authors disagreed); they also felt the paper would be stronger with additional datasets in the empirical evaluation (this was not addressed). R2 felt the work suffered for lack of evidence of consistency via repeated experiments, which the authors explained was due to the resource-intensity of the experiments. 

R5 raised that Inception scores for both the method and being noticeably worse than those reported in the literature, a concern that was resolved in an update and seemed to center on the software implementation of the metric. R5 had several technical concerns, but was generally unhappy with the presentation and finishedness of the manuscript, in particular the degree to which details are deferred to the CGD paper. (The authors maintain that CGD is but one instantiation of a more general framework, but given that the empirical section of the paper relies on this instantiation I would concur that it is under-treated.)

Minor updates were made to the paper, but R5 remains unconvinced (other reviewers did not revisit their reviews at all). In particular: experiments seem promising but not final (repeatability is a concern), the single paragraph "intuitive explanation" and cartoon offered in Figure 3 were viewed as insufficiently rigorous. A great deal of the paper is spent on simple cases, but not much is said about ICR specifically in those cases. 

This appears to have the makings of an important contribution, but I concur with R5 that it is not quite ready for mass consumption. As is, the narrative is locally consistent but quite difficult to follow section after section. It should also be noted that ICLR as a venue has a community that is not as steeped in the game theory literature as the authors clearly are, and the assumed technical background is quite substantial here. For a game theory novice, it is difficult to tell which turns of phrase refer to concepts from game theory and which may be more informally introduced herein. I believe the paper requires redrafting for greater clarity with a more rigorous theoretical and/or empirical characterization of ICR, perhaps involving small scale experiments which clearly demonstrates the effect. I also believe the authors have done themselves a disservice by not availing themselves of 10 pages rather than 8.

I recommend rejection at this time, but hope that the authors view this feedback as valuable and continue to improve their manuscript, as I (and the reviewers) believe this line of work has the potential to be quite impactful.
Decision: Reject