Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper studies the learning to optimize (L2O) for minimax optimization. Since L2O has been studied in a few works, extending L2O from continuous minimization to minimax is a straightforward idea and not super-novel. But it also is a non-trivial effort, as minimax problems are much harder and unstable to solve. 

The authors proposed to use two LSTMs with one shared reward, for updating min and max variables respectively. They presented a careful ablation study of design options such as (semi-)weight sharing between the two and their reward function, which is valuable for helping us understand what matters for L2O to work in minimax L2O.

The authors then presented two extensions to improve the generalization of Twin-L2O. The first one is based on curriculum learning to focus the meta-training gradually from easy to hard instances.  The second one is a minimax safeguard mechanism under a special case of solving convex-concave problems; the theory part seems to be a direct extension of (Heaton et al., 2020). 

The following suggestions are for the authors:

-	It is impressive to see that on relatively challenging minimax problems such as Seesaw, Twin-L2O can achieve one-magnitude higher-precision solutions than carefully tuned analytical algorithms. The number of iterations and MAC numbers needed for convergence are also comparable. I wonder whether the authors could also make a fair comparison on their running clock time? 

-	One further suggestion is that, it would be natural (and to the authors’ good) to combine enhanced L2O and safeguarded L2O together for solving convex-concave problems, so that we can get an impression on how large benefits it can lead to if we combine the best of the two L2O improvement ideas.

-	I appreciate the authors clearly and openly discussed the current work’s limitations by end of the paper. Although the paper was positioned as “proof of concept”, it could also be strengthened if some real problem can be demonstrated, e.g., training of a very simple GAN or so.

review_title: Reviews for L2O

final_score: rating: 7: Good paper, accept
review_content: This paper’s main contribution is to extend the L2O framework to solving minimax problems for the first time. 

Minimax optimization is in general unstable and harder to solve, challenging whether an L2O model can indeed figure out effective learning rules from data. Further, in order to design L2O for minimax problems, one has to decide to what extent the learning models for min and max updates should be coupled, and what reward shall be used (minimizing the negative cumulative objective value is no longer viable)

By discussing and comparing a number of design options, the authors find that two decoupled LSTMs sharing one variation-based reward is the best empirical design. They show this minimax L2O can display favorable empirical convergence speed on several testbed problems, compared against a number of analytical solvers.

More importantly, most L2O methods have little or no convergence guarantees, which constitutes another roadblock for broadening their practical usage, such as people often questioning whether they will diverge on some even slightly different problem or data. The authors presented Safeguarded Twin L2O, a preliminary theory effort saying that under some strong assumptions, it is possible to theoretically establish the general worst-case convergence of Twin-L2O. 

The proof draws and integrate two sources of ideas: (1) the safeguarded L2O technique recently developed for convex minimization (Heaton et al., 2020); and (2) Halpern iteration. For this part, it is unclear to me why Halpern iteration was chosen as the fallback method in their safeguarded L2O, since it is not a current popular or fast minimax solver. Is the safeguarded L2O framework also compatible with other convergent minimax solvers?

In Section 4.4, the authors said on unseen data, their safe-Twin-L2O remains to converge successfully and even faster than Halpern iteration. Is this really correct? As far as I understand, on an unseen distribution the optimization should “fall back” to exactly the Halpern iteration; so shouldn’t safeguarded L2O behave identically with Halpern on unseen data?

review_title: comments on #2099

final_score: rating: 7: Good paper, accept
review_content: Classical iterative minimax optimization algorithms display the unstable dynamics. Their convergence is often sensitive to the parameters and needs to be re-tuned for different problems to ensure convergence. Therefore, there is a practical motivation to develop L2O for minimax problems, so that we could meta-learn and adapt optimization rules to a special class of functions. 

To extend L2O from minimization to minimax where two groups of variables need be updated, the authors designed and explored a variety of model options. They find that using two LSTMs, with only their reward function shared, to benefit meta-learning most, particularly when the min and max updates are highly non-symmetric. The decoupled design is aligned with the experience in classical optimizers, e.g., the max step often needs for solution tracking. The authors also described both a curriculum training strategy, and a preliminary theory called safeguarding, to make L2O models be able to solve a wider range of problems. 

This paper’s contribution mainly lies in the engineering side, i.e., demonstrating meta learning or L2O can handle more complicated tasks/objectives than conventionally solving minimization. It is an interesting empirical study and is also done solidly. I believe this paper could attract interest and generate follow-up ideas from the L2O community.

On the math side, even though the authors tried to motivate their work from the limitation of classical minimax algorithms, I feel its impact may be limited for the optimization field, as it does not reveal many insights on how to design new minimax algorithms or providing better theory guarantees. 

Regarding the experiments, the authors demonstrated three simple testbed functions. As an empirical paper, it would definitely become stronger if the authors can prove their concept on some real minimax problems such as GAN or robust/private training.

The paper is in general well-written. With a lot of contents packed, the authors managed to organize and lay out their logic flow smoothly and clearly. I found just some typos: meta-learing -> meta-learning, draws and integrate -> draws and integrate, recently just introduce -> recently just introduced.



review_title: #Official Review 3

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary

The paper introduces the _learning to optimize_ (L2O) framework into the solution of minimax problems. The base model is composed of two decoupled LSTMs with a shared objective, with the two LSTMs being respectively responsible for the update of the min and max variables. On top of this, the authors further investigate two possible improvements.  One consists in applying curriculum learning to improve the generalization capability of the solver while the other uses safeguarding to guarantee convergence in convex-concave problems.
Numerical experiments are presented to justify the design choices of the base model and demonstrate the potential of minimax L2O.

---
### Pros
The paper is well-organized, easy to follow and provides a clear context for the problem that is studied.
This problem is particularly challenging and the authors manage to obtain some preliminary results.

---
### Score justification

I do not think the paper meets the acceptance criteria mainly due to the following reasons (all together):
1. Lack of clear motivation.
2. Lack of groundbreaking idea.
3. The definition of the loss function is not convincing.
4. The experiments do not provide strong evidence of the utility of the method either.

Although I fully understand this paper is just intended to be a proof of concept study that demonstrates the usefulness of L2O in minimax problems, I believe the authors should justify more the framework and their algorithmic choices (as done for the decoupled design).

---
### In more detail

1. While finding efficient algorithms for solving minimax problems is without doubt of increasing importance in machine learning today, in the paper there seems to be a lack of justification for the use of L2O in minimax problems.
In the literature, the L2O methodology has been mostly applied to relatively well-studied problems, such as sparse coding and function minimization. On the contrary, minimax optimization is much less well understood and there is little consensus on which optimization algorithm should be used when solving a particular minimax problem. In a sense, while meta learning approaches search for a universal solution to different problems, in minimax optimization people have not even agreed on the solution of a single problem.
Therefore, it would be beneficial if the authors could provide concrete examples for which they believe minimax L2O can really help.
2. In terms of the originality of the paper, the proposed models are basically combinations of existing ideas. While minimax L2O poses unprecedented challenges as claimed by the authors, this work does not seem to propose any dedicated solutions to address these challenges.
3. More importantly, I do not find what the authors propose as the loss function of the solver convincing. The definition of this loss function is probably one of the most important things in the framework. Nonetheless, I fail to see why encouraging stepwise progress in the two variables will necessarily lead to a solution of the problem. In my opinion, the objective (4) may lead to an unstable behavior of the generated iterates.
4. To finish, the experiments do not provide a strong motivation for the use of minimax L2O either.
5. A minor point: the proximal point of the safeguarding mechanism is not always computable so even for convex-concave problems safeguard Twin-L2O does not really offer a practical algorithm.
review_title: Interesting but the framework and the models should be better justified

Meta-review: The paper proposed Twin L2O (learning to optimize) for extending L2O from minimization to minimax problems. The authors honestly discussed the limitation of Twin L2O and proposed two improvements upon it with better generalization/transferability. While some reviewer had some concerns on the motivation of applying L2O to solve minimax problems and the motivation of the loss-function design (why objective-based one is chosen but not gradient-based one), the authors have done a particularly good job in the rebuttal. Even though this is more a proof-of-concept paper, it indeed has novel and solid contributions, and should be accept for publication.
Decision: Accept (Poster)