Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The authors are focusing in this paper on the development of LookAhead mechanism. Their contributions are the following:

1. They provide an improvement upon an already existing convergence rate for the minimization case.
2. They propose the application of the LookAhead mechanism (under the necessary adjustments) for the more general min/max framework.
3. These results are extensively justified by various numerical experiments.

The paper is well-written and easy to follow. Moreover, the core idea of applying the LookAhead for mix-max problems, as far as my knowledge goes, is  novel. In addition, without being an expert, the experiments seem promising and extensive.
That said, my concern would be regarding the theoretical part of the paper. I fully understand that the paper is primarily experimentally oriented. However some more theoretical analysis would be useful. More precisely, the theoretical justification for the improved convergence part for the minimization part seems to be more like a sketch rather than a rigorous mathematical proof. Overall, I suggest that it would be very interesting to see some provable theoretical guarantees starting with the  paper's motivation example that of the bilinear game.
review_title: This paper treats with algorithmic schemes that improve the training of GANs. These iterative methods are experimentally evaluate in a extensive manner. 

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper adapts a recently introduced Lookhead method in optimization to improve the training of minmax problems such as GANs. The main challenge is to address the variance of stochastic gradients and the rotational component in the Jacobian of the gradients. The algorithm proposed in this paper shows improvements over existing methods, in terms of convergence rate. The stability of existing algorithms is a major issue when applying GANs on image dataset, the proposed Lookhead minmax method also improves the stability. I find that the results on the bilinear game are convincing, but the results on GANs are much less convincing.  

There seems to me some inconsistency in the numerical results in Tab. 1 and Fig. 6. Are the same hyper-parameters used for LA-Alt-GAN on ImageNet? The median diverges in (c) of Fig. 6 while in Table 1, the FID is around 14.37? This is hard for me to interpret as the total number of iterations are the same, which is 500k. 

In terms of writing, I would recommend to focus on the minmax problem. In this sense, the section 3 is a bit distracting as it is about the optimization problem. The Algorithm 5 (page 19) about LA-AltGAN is important to understand the results, so it would be better to explain it more in the main body of the paper. Also what is LA-ExtraGrad algorithm? How Adam is applied in these algorithms? These proposed variants are not very clear to the reader. Also is the number of passes the same as the number of iterations? 

To make the results in Fig 5. more conclusive, I would recommend to zoom into the eigenvalues of LA-AltGAN, to see how small the imagery parts of the eigenvalues are. This is an interesting observation that it seems not very conclusive to show that LA-GAN shows no rotations.

Overall, I think both the results and the writing need to be improved. 

review_title: I find that the results on the bilinear game are convincing, but the results on GANs are much less convincing.  

final_score: rating: 6: Marginally above acceptance threshold
review_content: ##### Summary:

This paper proposes a lookahead-minmax algorithm for optimizing minmax problems such as GANs, which updates the parameters (of both the generator and the discriminator) with the extrapolation. With a bilinear example, the authors show that the use of lookahead-minimax allows for convergence in cases where other methods does not, and yields good performance under high variance. Experiments of generative performance on several well-known public datasets demonstrates the effectiveness of the proposed method.

##### Reasons for score:

The idea is intuitive and easy to follow. My major concern is about the scale of the experiments.

---
##### Pros: 
 
1. The paper is well-motivated and the idea is easy to understand. Figure 1 clearly demonstrates how lookahead-minmax can address the rotational nature in GAN’s training, which is further confirmed in Figure 5.
 
2. The advantages of the proposed lookahead-minmax is demonstrated with a bilinear example. For me, this helps to understand the merits of the proposed method.

3. The paper provides comprehensive experiments and describes the experimental settings in details. 
 
##### Cons: 
 
1. The contributions are a little bit confusing. The first contribution listed in the introduction (and Section 3) is an improved convergence guarantee, which seems an independent contribution. What is the relation between this theoretical result and the lookahead-minmax algorithm for GANs? Does the following analysis and conclusions rely on this improvement? After reading the paper, I found the rest of the paper does not necessarily build on the new convergence guarantee. I think it might be clearer to mention the most significant contribution first.
  
2. Some related works are missing. Gidel et al. 2019b presents the negative momentum method, which leads to a similar “backtracking step” in the vector field the gradients. CLC-GAN [Xu et al. arXiv:1909.13188] proposes to stabilize GANs’ training dynamics by interpreting the gradient flow as a dynamic system and manage it with closed-loop control, which is also related to this paper.
 
3. The experiments are conducted on 4 representative datasets. However, the image sizes (32*32) do not match the state-of-the-art GANs, which typically can generate larger images (e.g. Mescheder et al. 2018). One concern is whether the proposed method works well on larger datasets such as LSUN and ImageNet with higher resolutions. It would be more convincing if the authors can provide results on larger datasets.
  
---
##### Some typos: 
(1) In Eqn. (JVF): the second entry in the JVF $\nabla_{\phi}\mathcal{L}^{\theta}$ -> $\nabla_{\phi}\mathcal{L}^{\phi}$

(2) In the line below Eqn. (SB-G): It seems that $b$ and $c$ are of dimension $d \times n$.

(3) In paragraph of “Benchmark on CIFAR-10 …” of Page 7, “3.5 times larger model then ours” -> “3.5 times larger model than ours”

(4) In the same paragraph mentioned in (3), the references to tables are inconsistent. Both “Table 2” and “Tab. 2” are used.

(5) In the last line of Page 7, “LA—GAN” em dash -> en dash

review_title: Official Blind Review #1

final_score: rating: 7: Good paper, accept
review_content: Summary: This work extends the recently proposed lookahead optimizer (which was designed for single-objective optimization) to minimax optimization, particularly GAN training. The authors claim that the backtracking step in lookahead optimizer alleviates the notorious rotational behavior in GAN dynamics. Moreover, the authors argue that the lookahead optimizer implicitly handles the high variance in the small-batch setting. Both arguments are backed up by toy experiments on stochastic bilinear games.  Finally, on standard image datasets, the lookahead minimax algorithm outperforms some popular algorithms and achieves state-of-the-art performance on CIFAR-10.

Review: This paper is well-written and explains the main idea in a clear and effective manner. The empirical results of the introduced lookahead minimax algorithm are quite impressive given its simplicity. However, the lookahead minimax dynamics is not well-understood and no theoretical justification is provided in the paper. The bilinear game might not reflect the real GAN training as standard GDA never diverges in real GAN training. I tend to give a score of 7 or even higher, but I'm not satisfied with the current explanation of the performance gain. I think a better understanding of the dynamics is needed and I will increase my rating if the authors could improve the paper accordingly.

Here are detailed comments:

1. In the third paragraph of the introduction, the authors mentioned that empirically GANs often converge to a locally stable stationary point that is not a differential Nash equilibrium. Actually, this phenomenon occurs even for the simplest quadratic minimax games, see for example [1, 2, 3]. Moreover, Nash equilibrium might not even exist in general, Stackelberg equilibrium [2, 4] is probably a better solution notion.

2. The authors provide some theoretical results of the lookahead optimizer, but for single-objective optimization. I suggest the authors moving this section to the Appendix (as this section seems irrelevant) and replace it with the analysis of lookahead for minimax games. In particular, the authors argue many times in the paper that the lookahead algorithm is able to handle the rotational dynamics well, I think the authors should be able to verify that theoretically on quadratic minimax games if the argument is right.

3. The bilinear game might not reflect the real training dynamics of GANs. In bilinear games, simultaneous GDA with finite step sizes simply diverges, but it is not the case in practice for GAN training. To this end, strongly-convex strongly-concave minimax games might be better to model the underlying GAN dynamics. Strongly-convex strongly-concave quadratic minimax games shall be as simple as the bilinear cases and some theoretical analyses and simulation should be easy to do. 

4. In the paper, the authors chose Extragradient as one of their baselines. However, after inspecting the code, I find much fewer iterations are used for Extragradient. I understand the authors would like to keep the total gradient queries the same over different algorithms, but there is a similar algorithm (Optimistic gradient descent ascent [5, 6]) with only one gradient query per step and perform similarly with Extragradient. In this sense, I think it is fair to only use fewer iterations for Extragradient. 

5. For stochastic games (especially for SB-G), there is a better version of Extragradient which handles the variance well, see [7] for details.

6. In the original lookahead paper, it was mentioned that the slow weights can be understood as the exponential moving average (EMA) of the fast weights. In particular, the authors of the original lookahead paper did some analysis on a noisy quadratic model and showed lookahead reduces variance. In that simple noisy quadratic model, exponential moving average can effectively reduce variance, see section 3.4 of [8] for details. I think it is probably worth testing EMA on the toy experiments and mentioning it in the paper. I notice that the authors have already included EMA in section 5 and EMA improves the performance a lot. To this end, I think the advantages of lookahead can be potentially decomposed into two parts (though more works need to be done): (1) it suppresses the rotational part in game dynamics (I encourage the authors to analyze the spectrum of the dynamics); (2) it implicitly does exponential moving average and hence reduces the variance.

Overall, I think this paper is very strong on the empirical results. However, analysis on the dynamics of the lookahead algorithm is lacking. I will be happy to increase my rating if the authors could resolve some of my comments.

--------------------
**In the rebuttal, the authors resolved most of my concerns, therefore I increased my score to 7 as promised.**


Reference:
[1] On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games, 2019.

[2] On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach, 2020.

[3] GANs May Have No Nash Equilibria, 2020.

[4] Convergence of learning dynamics in Stackelberg games, 2019.

[5] Training GANs with Optimism, 2018.

[6] A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach, 2019.

[7] Revisiting Stochastic Extragradient, 2019.

[8] Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model, 2019.
review_title: Strong empirical results on standard benchmarks, but no theoretical result provided. 

Meta-review: The reviews were a bit mixed, with a general consensus towards acceptance. The authors were one of the first to extend lookahead to minimax optimization, and demonstrated its potential through thorough experiments. The theoretical results were not as strong or at least not very well presented. Overall, the authors made interesting contributions and this work is of general interest to the ICLR audience. Please consider further polishing the draft according to the reviewers' comments. The AC would also like to draw the authors' attention to the following issues discovered in an independent assessment:

(a) As the reviewers mentioned, how lookahead-minimax addresses rotational dynamics is not clearly presented. The current justification is a bit handwaving and speculative. 

(b) Please consider rewriting Section 3. If there is some new results on the minimization problem, state the results in a theorem and include all assumptions clearly and precisely. This is also useful for other people to reference your result. As the authors themselves pointed out, this result falls quite short of explaining or motivation lookahead. 

(c) Theorem 1, add e.g. in the citation before (Bertsekas, 1999). Theorem 2, in its current form, is quite weak in two aspects: (a) without checking its proof one can already see how to derive it in 1 line or 2. (b) if the base optimizer already converges, what is the point of having lookahead to converge as well? The potentially different convergence rate should be one's target here. It is certainly fine for the authors to not fully justify their proposed algorithm, as long as the authors (hopefully) are at least aware of the issues.

(d) Section 4 is a bit disappointing as one would have expected the authors to derive some qualitative results here (also raised by some reviewers).
Decision: Accept (Poster)