M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation
Authors: Junjie Yang
Xuxi Chen
Tianlong Chen
Zhangyang Wang
Yingbin Liang
Venue: ICLR 2023 poster (2023)
Keywords: L2O
Meta Learning
Generalization

Abstract:  Learning to Optimize (L2O) has drawn increasing attention as it often remarkably accelerates the optimization procedure of complex tasks by "overfitting" specific task type, leading to enhanced performance compared to analytical optimizers. Generally, L2O develops a parameterized optimization method (i.e., "optimizer") by learning from solving sample problems. This data-driven procedure yields L2O that can efficiently solve problems similar to those seen in training, that is, drawn from the same "task distribution". However, such learned optimizers often struggle when new test problems come with a substantially deviation from the training task distribution. This paper investigates a potential solution to this open challenge, by meta-training an L2O optimizer that can perform fast test-time self-adaptation to a out-of-distribution task, in only a few steps. We theoretically characterize the generalization of L2O, and further show that our proposed framework (termed as M-L2O) provably facilitates rapid task adaptation by locating well-adapted initial points for the optimizer weight. Empirical observations on several classic tasks like LASSO and Quadratic, demonstrate that M-L2O converges significantly faster than vanilla L2O with only $5$ steps of adaptation, echoing our theoretical results. Codes are available in https://github.com/VITA-Group/M-L2O.

Published as a conference paper at ICLR 2023
M-L20: TOWARDS GENERALIZABLE LEARNING-TO- OPTIMIZE BY TEST-TIME FAST SELF-ADAPTATION
Junjie Yang', Xuxi Chen?, Tianlong Chen², Zhangyang Wang', Yingbin Liang' The Ohio State University, Uniieeriity of Texas at Austin (yang. 4972, liang. 889) (0su0
Ixxchen, t tianlong, chen, at atlaswang y @ut exta...s..
ABSTRACT
Learning to Optimize (L20) has drawn increasing attention as it often remark- ably accelerates the optimization procedure of complex tasks by "overfitting" specific task types, leading to enhanced performance compared to analytical op- timizers. Generally, L20 develops a parameterized optimization method (i.e., "optimizer") by learning from solving sample problems. This data-driven procedure yields L20 that can efficiently solve problems similar to those seen in training, that is, drawn from the same "task distribution". However, such learned opti- mizers often struggle when new test problems come with substantial deviation from the training task distribution. This paper investigates a potential solution this open challenge, by meta-training test-time self-adaptation to an out-of-distribution task, in only a few steps. We theoretically characterize the generalization of L2O, and further show that our proposed framework (termed as M-L20) provably facilitates rapid task adapta- tion by locating well-adapted initial points for the optimizer weight. Empirical observations on several classic tasks like LASSO. Quadratic and Rosenbrock demonstrate that M-I.20 converges significantly faster than vanilla 1.20 with only steps of adaptation, echoing our theoretical results. Codes are available in https: : / thub. com wwwmco///////// MOIITA-GroupM----
I.20 optimizer that perform fast
INTRODUCTION
Deep neural networks are showing overwhelming performance various tasks, and their tremen- dous success partly lies in the development of analytical gradient-basee optimizers. Such optimiz- achieve satisfactory convergence on general tasks, with manually-crafted rules. For example, SGD (Ruder, 2016) keeps updating towards the direction of gradients and Momentum (Qian, 1999) follows the smoothed gradient directions. However, the reliance on such fixed rules can limit the ability of analytical optimizers to leverage task-specific information and hinder their effectiveness.
Leariing to Optimize (I.20), alternative paradigm emerges recently, aims at learning optimization algorithms (usually parameterized by deep neural networks) in data-driven way, to achieve faster convergence on specific optimization task optimizee. Various fields have witnessed the superior performance of these learned optimizers over analytical optimizers (Cao et al., 2019; Lv et al., 2017; Wichrowska al., 2017; Chen al., 10; Zheng al., 2022). Classic L20s follow a two-stage pipeline: at the meta meta-training stage, an L20 optimizer is trained to predict updates for the parameters of optimizees, by learning from their performance on sample tasks; and at the meta-lesting stage, the L20 optimizer freezes its parameters and is used to solve new optimizees. In general, L20 optimizers can efficiently solve optimizees that are similar to those seen during the meta- veetattainnnn stage, or are drawn from the same "task distribution".
However, new unseen optimizees may substantially deviate from the training task distribution. As L20 optimizers predict updates variables based the dynamics of the optimization tasks, such as gradients, different task distributions can lead to significant dissimilarity in task dynamics. Therefore, L20 optimizers often incur inferior performance when faced with these distinct unseen optimizees.
Such challenges have been widely observed and studied in related fields. For example, in the domain of meta-learning (Finn et al., 2017; Nichol & Schulman, 2018), we aim enable neural networks to

---

Published as a conference paper at ICLR 2023
be fast adapted to new tasks with limited samples. Among these techniques, Model-Agnostic Meta Learning (MAML) (Finn et al., 2017) is one of the most widely- algorithms. Specifically, in the meta-training stage, MAML makes inner updates for individual tasks and subsequently conducts pro-ppropagttion to aggregate the gradients of individual task gradients, which are used to update the meta parameters. This design enables the learned initialization (meta parameters) to be sensitive to each task, and well-adapted after few fine-tuning steps.
Motivated by this, we propose a novel algorithm, named M-L20, that incorporates the meta-adaption design in the meta-training stage of L20. In detail, rather than updating the L20 optimizer directly based on optimizee gradients, M-L20 introduces a nested structure to calculate optimizer updates by aggregating the gradients of meta- ueeated optimizees. By adopting such an approach, M-L20 is able to identify a well-adapted region, where only a few adaptation steps are sufficient for the optimizer to generalize well on unseen tasks. In summary, the contributions of this paper are outlined below:
To address the unsatisfactory generalization of L20 on incorporate a meta adaptation design into L20 training. It enables the learned optimizer to locate in well-adapted initial points, which can be fast adapted in only few steps to new unseen optimizees.
tasks, we propose to
We thcoretically demonstrate that our meta adaption design grants M-L20 optimizer faster adaption ability in out-o--distiibution tasks, shown by better generalization errors. Our analysis further suggests that training-like like adaptation tasks can yield better generalization performance, in contrast the common practice of using testing-like tasks. Such theoretical findings are further substantiated by the experimental results.
Extensive experiments consistently demonstrate that the proposed M-L.00 outperforms various baselines, including vanilla L20 and transfer learning, in terms of the testing performance within small number of steps, showing the ability of M-L20 to promptly adapt in practical applications.
RELATED WORKS
2.1 LEARNING TO OPTIMIZE
Learning to Optimize (L20) captures optimization rules in data-driven way, and the learned optimizers have demonstrated success on various tasks, including but not limited to black-box (Chen et al., 2017), Bayesian (Cao et al., 2019), minimax optimization problems (Shen et al., 2021), domain adaptation (Chen et al., 2020b; Li et al., 2020), and adversarial training (Jiang et al., 2018; Xiong& & Hsieh, 2020). The success of L20 is based on the parameterized optimization rules, which are usually modeled through a long short-term memory network (Andrychowicz et al., 2016), and occasionally as multi-layer perceptrons (Vicol et al., 2021). Although the parameterization is practically successful, it comes with the "curse" of generalization issues. Researchers have established two major directions for improving L20 generalization ability: the first focuses on the generalization to similar optimization tasks but longer training iterations. For example, Chen et al. (2020a) customized training procedures with curriculum learning and imitation learning, and Lv et al. (2017); Li et al. (2020) designed rich input features for better generalization. Another direction focuses on the generalization to different optimization tasks: Chen ct al. (202 studied the generalization for LISTA network on unseen problems, and Chen et al. (2020c) provided theoretical understandings to hybrid deep networks with learned reasoning layers. In comparison, our work theoretically studies general L20 and our proposals generalization performance under task distribution shifts.
2.2 FAST ADAPTATION
Fast adaptation is one of the major goals in the meta-learning area Finn et al. (2017); Nichol & Schulman (2018); Lee & Choi (2018) which often focuses on generalizing to new tasks with limited samples. MAML Finn et al. (2017), famous and effective meta-learning algorithm, utilizes the nested loop for meta-adaption. Following this trend, numerous meta-lcarning algorithms chose to compute meta updates more efficiently. For example, FOMAML (Finn et al., 2017) only updated networks by first-order information; Reptile (Nichol & Schulman, 2018) introduced an extra intermediate variable to avoid Hessian computation; HF-MAML (Fallah et al., 2020) approximated the one-step meta update by Hessian-vector production; and Ji et al. (2022) adopted multi-step approximation in updates. Meanwhile, many researchers designed algorithms to compute meta updates more wisely. For example, ANIL (Raghu et al., 2020) only updated the head of networks in the inner loop; HSML (Yao et al., 2019) tailored the transferable knowledge to different tasks; and MT-net (Lee & Choi, 2018) enabled meta-learner to learn on each layer's activation space. In terms of theories,
2

---

Published as a conference paper at ICLR 2023
(Fallah al., 2021) measured the generalization error of MAML; Fallah et al. (2020) captured the single inner step MAML convergence rate by Hessian vector approximation. Furthermore, Ji et al. (2022) characterized the multiple-step MAML convergence rate. Recently, LFT Zhao et al. (2022) combines the meta-learning design in Learning to Optimize and demonstrates its better performance for adversarial attack applications.
PROBLEM FORMULATION AND ALGORITTM
In this section, we firstly introduce the formulation of L20, and subsequently propose M-L20 for generalizable self-adaptation.
3.1 L20 PROBLEM DEFINITION
Most machine learning algorithms adopt analyt- ical optimizer, e.g. SGD, to compute parameter updates for general loss functions (we call it op- timizee or task). Instead, L.20 aims estimate such updates by model (usually a neural net- work), which we call optimizer. Specifically, the L20 optimizer takes the optimizee informa- tion (such as loss values and gradients) as input and generates updates the optimizee. In this work, our objective is to learn the initialization of the L20 optimizer on training optimizees and subsequently finetune it on adaptation op- timizees. Finally, we apply such an adapted optimizer to optimize the testing optimizees and evaluate their performance.
4--1
Ps
02-1
Optimizer
Optimizer
mith wesfiut
0
0 1-1
0
2-
T Optimizees samples
T Optimizees samples
Inputs to Optimizer
Updates to Optimizees
Figure 1: The pipeline of L20 problems.
We define ((0:E) the loss function where 0 the optimizee's parameter, & 1, 2. .N) denotes the data sample, then the optimizee empirical and population risks are defined as below:
{j
i(0) = N Σラ (0:E5)) 1(0) (c)
In L20, the optimizee's ' parameter θ is updated by the optimizer, an update rule parameterized by and we formulate it as m(2 (0,; (e). Φ). Specifically, 2N (0r; ()) denotes the optimizer model
input. It captures the t-th iteration's 's optimizee information and parameterized by 0g with the data batch G. Then, the update rule of 9 in 1-th iteration is shown as below:
(0) = (,) + m m ()), o) = (o) + m(22. 0).
(1)
The above pipeline is also summarized Figure where ls denotes the update epoch of the optimizer. Note that S refers to the data batch of size N used at t-th iteration while E, refers to the j-th single sample data batch Ç. For theoretical analysis, we only consider taking the optimizee's gradients as input to the optimizer for update, i.e., ze (0; Ge) (o)().())). Therefore, the optimizee's gradient at T-th iteration over the optimizer VeOr(o) (() takes form of
V -
(T+ Vim V1
+)), (2)
where we assume that Ø is independent from the optimizee's 's initial parameter Og and all samples are independent. The detailed derivation is shown in Lemma in the Appendix.
Next, we consider common initial parameter Og for all optimizees and define the optimizer empirical and population risks w.r.t. é as below:
(o( = ((O,())), (e( = Ec((O,(0):)) = -())())))))(((((()))))))
(3)
where (d) updates in such a fashion: (0) (c(0) + m(z((0,; (e), Φ). Typically, the L20 optimizer is evaluated and updated after updating the optimizees for 7'thh iterations. Therefore, the optimal points of the optimizer risks in Equation (3) are defined below:
3, = arg ming (T()), d. = arg ming (T()).
(4)
3

---

Published as a conference paper at ICLR 2023
3.2 M-L2O ALGORITHM
To improve the generalization ability of L2O, we aim at learning well-adapted optimizer initialization by introducing nested meta-update architecture. Instead of directly updating é, adapt for one step (namely an inner update) and define a new empirical risk for the optimizer as follows:
Gr(Ø) Ôr( a (GT(
(5)
where a is the step size for inner updates. Consequently, the optimal point of the corresponding updated optimizer is:
M arg ming (T))
(6)
Based such optimizer loss, introduce M-I.20 in Algorithm 1. Such nested update design has been proved effective in the field of meta-learning, particularly MAML (Fian et al., 2017). Note that Algorithm only returns the well-adapted optimizer initial point, which would require further adaptation in practice. We first denote the optimizees for training, adaptation, and testing by g (Ø)) gØ)), and gØ)), respectively, distinguish tasks seen in different stages. Next, we obtain the results of meta training, denoted by MM via Algorithm 1, and we further adapt it based on (Ø). The testing loss of M-L20 can be expressed follows:
o M aV (
where (6) denotes the meta testing loss. Note that 9 refers to empirical risk and refers to population risk.
Algorithm Our Proposed M-L20.
1: Input: Inner step size o, Outer learning stepsize 8z. Total epochs K, Epoch number per task S,
Optimizer initial point øi Muu Training task أ., Adaptation task g², Testing task g²
2: for k ===00 0, 1,.,,... K
do
3: if' (k; = 0: 0o ( = 0, (random initial) else 0o (o( = Or Or(
4: for
0,
do
5: (141("M))) = 0c( ml (
Note: (((((())x)) = = (o
6: end for
7: Compute (3 =9 ( Mr
aV (V
8: update: 2 )M( = Then= -B V.
9: end for
Output: ØMAK Testing Loss: ( a (
4 GENERALIZATION THEOREM OF M-L20
In this section, we introduce several assumptions and characterize M-L.20's generalization ability.
4.1 TECHNICAL ASSUMPTIONS AND DEFINITIONS
To characterize the generalization of meta adaptation, it is necessary to make strong convexity assumptions which have been widely observed (Li & Yuan, 2017; Du et al., 2019) under over- parameterization condition and adopted in the geometry of functions (Fallah et al., 2021; Finn et al., 2019; Ji et al., 2021).
A Assumption 1. We assume that the function ( is ststoongly convex. This assumption also holds for stochastic (Ø).
To capture the relationship between the L20 function (Ø) and the M-L2O function Gr(ø), we make the following optimal point assumptions.
Assumption We assume there exists non-trivial optimizer optimal point The- which is defined in Equation (6). Non-trivial means that V Vest( (TM)) F O and Ø) + T.. Then, based on the definition of T Mr and the existence of trivial solutions, for any T M4 have the following equation:
Φ = 00 a Vo07 (D)a))
4

---

Published as a conference paper at ICLR 2023
where is the step size for inner update of the optimizer. Note that we have defined the strongly-convex of landscape in Assumption 1 which validates the uniqueness of Φ1.
The aforemenniooed assumption claims that there exist meta optimal points which are different from original task optimal points. Such an assumption is natural in experimental view. In MAML experiments where single training task is considered, it is reasonable expect that the solution would converge towards a well-adapted point instead of the task optimal point. Otherwise, MAML would be equivalent simple transfer learning, where the learned point may generalize well to new tasks.
Assumption 3. We assume that L((0) is M -ipschittz (L, is L-Liscchita and (2((0) is p-Lipschitz for each loss function 4(0)(i = 2,2,3). 3). This assumption also holds for stochastic (,(0), (i, and (i) = 1,), We further assume that m(z, Φ) is Mma--ipschiin w.r.t Mmz-Lipschitz w.E.t @ and (O)) is pa- Lipschit..
The above Lipschitz assumptions are widely adopted previous optimization works (Fallah et al., 2021; 2020; Ji et al., 2022; Zhao et al., 2022). To characterize the difference between tasks for meta training and meta adaptation, we define 112 and Å12 as follows:
Assumption We assume there exist union bounds A12 and Â12 to capture the gradient and Hessian differences respectively between mela training task and adaptation task:
1 max
V pl (0) (ol2(0))l.
Â max V (0) (212(0))l-
Such an assumption has been made similarly in MAML generalization works (Fallah et al., 2021).
4.2 MAIN THEOREM
In this section, we theoretically analyze the generalization error of M-L20 and compare with the vanilla L20 approach (Chen et al., 2017). Firstly, characterize the difference of optimizee gradients between any two tasks (V (d) and (0)) in the following form:
Proposition Based on Equation (2), Assumptions and 4, we obtain
| ( - V (+) <
+ Mm=QT=i-2 Ef Anj),
where Ac: ((Q'A12)) AD; O(Q' D12 + Å12) and Q' (1 + M). Furthermore, characterize the task difference of optimizer gradient (Ø) as follows:
IV Vost( (0) (o)I( - + Q2T
where Q 1+ 1 Mm1Le A12 and Âς2 are defined in Assumption
Proposition 1 shows that the difference in optimizer gradient landscape scales exponentially with the optimizee iteration number T. Specifically, it involves the Q -1 term with gradient difference A12 and the TQT term with Hessian difference 7 Clearly, Q A dominates when increases, which implies that the gradient gap between optimizees the key component of the difference in optimizer gradient.
Theorem
Suppose that Assumptions 2, and hold. Considering Algorithm and Equation (7), we define d13 = l -311. set a min z Spo }. BR min(3. (3, m(++I] for B < 8 Then, with probability at least 1 - 8, we obtain
p
Elgi -aV ( (
Mgr 42
Lop + o
MAr 3VK
2 222M PVON
5(Mgr (1+ Lign ((1)
+
+ +Mar
+ M13
3K
+ +Mg a0
(1T))
where 1 Mm L, Mgr O(Q Lgr O(TQU-2 Q2T Par Q3T-3), K is total epoch number for training, N is the batch size for optimizer training.
- Pgr O
5

---

Published as a conference paper at ICLR 2023
To provide further understanding of generalization errors, we first make the following remark:
Remark (The choice of o:). In Theorem I, we set S Sppr or = Ol Tape '4qur thus the error term Mg - + Q A12) vanishes with larger T. we fix the iteration number T, then such an error term is determined by the gradient bound A12 and the Hessian bound Å12-
The key components that lead to Q dependeney the Lipschitz properties characterize the L20 loss landscape, e.g. the Lipschitz term LGT - O(TQ' 221-2) defined for V Vorr()). The reason is due to our nested update procedure (6) = 0c(0) ml mv 6). If we take the gradient of the last update term (r (Ø) ((Or())) over then it requires compute gradients iteratively for all t = 0, 1,.., T 1, which leads to the exponential term.
Consequently, it can be observed that the generalization enror of M-L20 can be decomposed into three components: (i) The first term determined by V tPo agg. Mar is dominated by the ByK
training epoch K. Such an error term characterizes how meta training influences the generalization;
(ii) The second term 2v2Maa reflects the empirical error introduced by limited samples; hence it is aV6N
controlled by the sample size N. (iii) The last two error terms capture task differences. Specifically, 513 measures the gap between training and testing optimal points, while A12 and ÂΙ2, which dominate the last error term and represent the gradient and Hessian union bounds, respectively, reflect the geometry difference between training and adaptation tasks.
For better comparison with L20, we make the following remark about generalization of M-L20 and Transfer Learninig.
Remark 2 (Comparison with Transfer Learning). We can rewrite the generalization error of M-L20 in Theorem in the following form:
( - aV aV - (0)))
Il - TMall Mar 11 -b'1l Mor al V -v 013
For L20 Transfer Learning, the generalization error is shown as below:
(0 aV (b)) (Ø.)
SMgrll0t Theall + Mar 1l 6.1l + Mgr all (Ok) V Vegt
Mgr
where 613 || - 222 and Wy represents transfer learning 120 learned point after K epochs.
The generalization error gap between M-L20 and Trasnfer Learning can be categorized into two parts:
(i) Difference between | The and 1 Thrali Il: (ii) Difference between || Vo07 ( k ( and V O V Vool 2 If we assume (p) = (+) then both differences can be characterized by the gap between | - @hrlll and |10b Pir- Il. Since
[|-
Thrk is trained to converge to dh, as K increases, it is natural to see that M-1.20 (1 Il enjoys smaller generalization error compared to Transfer Learning [10] 1l..l).
We further distinguish our theory from previous theoretical works. In the L20 area, Heaton et al. (2020) analyzed the convergence of proposed safe-L20 but not generalization while Chen al. (2020c) analyzed the generalization of guadratic-based L20. Instead, we develop the generalization on a general class of L.20 problems. In the meta-learning area, the previous works have demonstrated the convergence and generalization of MAML (Ji et al., 2022; Fallah et al., 2021). Instead, we leverage the MAML results in L20 domain to measure the learned point and training optimal point distance. Then, our L20 theory further characterizes transferability of learned point meta testing tasks. Overall, our developed theorem is based on both L20 and meta learning results. In conclusion, theoretical novelty lies in three aspects: ① Rigorously characterizing a generic class of L20 generalization. @ Incorporating the MAML results in our meta-learning analysis. @ Theoretically proving that both training-like and testing-like adaptation contribute to better generalization in I.20.

---

Published as a conference paper at ICLR 2023
5 EXPERIMENTS
In this section, we provide a comprehensive description of the experimental settings and present the results we obtained. Our findings demonstrate a high degree of consistency between the empirical observations and the theoretical outcomes.
5.1 EXPERIMENTAL CONFIGURATIONS
Backbones and observations. For all our experiments, we use a single-layer LSTM network with 20 hidden units as the backbone. We adopt the methodology proposed by Lv et al. (2017) and Chen et al. (2020a) to utilize the parameters' gradients and their corresponding normalized momentum to construct the observation vectors.
Optimizees. We conduct experiments on three distinct optimizees, namely LASSO, Quadratic, and Rosenbrock (Rosenbrock, 1960). The formulation of the Quadratic problem is min 2 AAx blI2 and the formulation of the LASSO problem is ming 1 |llAa bll2 + A/| where E Rdd We set À 0.005. The precise formulation of the Rosenbrock problem is available in Section A.6. During the meta-training and testing stage, the optimizees Etrann and Erett are drawn from the pre-specified distributions Diaan and Dres. respectively. Similarly, the optimizees Eadupt used during adaptation are sampled from the distribution Dadapt-
Rxd
Baselines and Training Settings. We compare M-L20 against three baselines: (1) Vanilla L2O, where we train a randomly initialized L20 optimizer on Eadupe for only 5 steps; (2) Transfer Learning (TL), where first meta-train randomly initialized L20 optimizer Euain- and then fine-tune on Eadapt for 5 steps; (3) Direct Transfer (DT), where we meta-train a randomly initialized L20 optimizer on Strain only. M-L2O adopts fair experimental setting, whereby we meta-train on Euain and adapt on the same Eadapt- We evaluate these methods using the same set of optimizees for testing (i.e., Eies),, and report the minimum logarithmic value of the objective functions achieved for these optimizees.
For all experiments, we set the number of optimizee iterations, denoted by T. to 20 when meta-trait training the L20 optimizers and adapting to optimizees. Notably, in large scale experiments involving neural networks as tasks, the common choice for T is (Zheng et al., 2022; Shen et al., 2021). However, in our experiments, we set T to 20 to achieve better experimental performance. The value of the total epochs, denoted by K, is set to 5000, and we adopt the curriculum learning technique (Chen et al., 2020a) to dynamically adjust the number of epochs per task, denoted by To update the weights of the optimizers (Ø), we use Adam (Kingma & Ba, 2014) with fixed learning rate of 1 x 10-
5.2 FAST ADAPTATION RESULTS OF M-I.20
Experiments LASSO optimizees. We begin with experiments on LASSO optimizees. Specifically, for Strain. the coefficient matrix A is generated by sampling from a mixture of uniform distributions comprising (U, 0.1), U0, 0.5), U00 1)}. In contrast, for ftett and Eadage the coefficient matrices A are obtained by sampling from a normal distribution with a standard deviation of a. We conduct experiments with o 50 and demonstrate that:
100, and report the results in Figures 2a and 2b. Our findings
① The Vanilla L20 approach, which relies on only five steps adaptation from initialization on Salagt, exhibits the weakest performance, as evidenced by the largest values of the objective function.
@ Although Direct Transfer (DT) capable of learning optimization rules from the training opti- mizees, the larger variance in coefficients among the testing optimizees renders the learned rules inadequate for generalization.
& The superiority of Transfer Learning (TL.) over DT highlights the values of adaptation when the testing optimizees deviates significantly from those seen in training, as the optimizer presumably able to acquire new knowledge during the adaptation process.
4 Finally, M-L20 exhibits consistent and notably faster convergence speed compared to other baseline methods. Moreover, it demonstrates the best performance overall, reducing the logarithm of the objective values by approximately 0.2 and 1 when a = 50 and a = 100, respectively. M- L2O's superior performance be attributed to its ability to learn well-adapted initial weights for optimizers, which enables rapid self-adaptation, thus leading to better performance in comparison to the baseline methods.
7

---

Published as a conference paper at ICLR 2023
Table Minimum logarithm loss of different methods on LASSO at different levels o. We report the 95% confidence interval from 10 repeated runs.
Methods
Vanilla L20
M-L20
DT
TL
10
0.033
-3.712
-..233+0.016
4.007
25
1.559
-3.433+0.001
+0.025
4.011
50
0.550
-3.285 40.013
-3.098 #0.021
-3,181 40.011
100
2.500
-2.408
-1.775
-1.961 +0.050
200
4..000
-1.396.40.0s
-0.453+0.075
-0.98240.086
Experiments on Quadratic optimizees. We continue to assess the performance of our approach on a different optimizee, i.e., the Quadratic problem. The coefficient matrices A of the optimizees are also randomly sampled from normal distribution. We conduct two evaluations, with or values of 50 and 100, respectively, and present the outcomes in Figure 3. Notably, the results show a similar trend to those we obtained in the previous experiments.
Otjeetiei
Log(Objective
2.5
M-
M-
M M-L.00
DT
DT
-2-
TL
TL
Vanilla L20
Vanilla L20
-2.5
50
100
150
50
100
150
Steps
Steps
(a) Performance on (0, 50")
(b) Performance on (o 100
Figure 2: Comparison of convergence speeds on target distribution of LASSO optimizees. We repeat the experiments for 10 times and show the 95% confidence intervals in the figures.
More LASSO experiments. We proceed investigate the impact of varying the standard deviation a of the distributions we used to sample the coefficient matrices A for Endapt and fuea- The minimum logarithm of the objective value for each method is reported in Table 1. Our findings reveal that:
① At lower levels of o, it is not always necessary. and may even be unintentionally harmful, to adaptation for normally trained I.20. Although M-I.20 produces satisfactory results, it exhibits slightly lower performance than TL., which could be due to the high similarity between the training and testing tasks. Since M-L20's objective is to identify optimal general initial points, L2O optimizers trained directly on related and similar tasks may effectively generalize. However, after undergoing adaptation single task, L20 optimizers may discard certain knowledge acquired during meta- training that could be useful for novel but similar tasks.
Nevertheless, as the degree of similarity between training and testing tasks is declines, as character- ized by an increasing value of a, M-L20 begins to demonstrate considerable advantage. For values of a greater than 50, M-I.20 exhibits consistent performance advantages that exceed 0.1 in terms of logarithmic loss. This observation empirically supports that the learned initial weights facilitate rapid adaptation to new tasks that are "outtof-distribution" and that manifest large deviations.
5.3 ADAPTATION WITH SAMPLES FROM DIFFERENT TASK DISTRIBUTION
In Section 5.2, we impose a constraint on the standard deviation of the distribution used to sample A, ensuring that is identical for both the optimizees for adaptation and testing. However, it noteworthy that this constraint not mandatory, given that our theory can accommodate adaptation and testing optimizees with different distributions. Consequently, we conduct an experiment on LASSO optimizees with varying standard deviations of the distribution, from which the matrices A for optimizee Ealapi is drawn. Specifically, we sample a with smaller values that more resemble the training tasks, well larger values that more similar the testing task (a 100).
In Theorem 1, we have characterized the generalization of M-L20 with flexible distribution adaptation tasks. The theoretical analysis suggests that similar geometry landscape (smaller
8

---

Published as a conference paper at ICLR 2023
Logs (og(Cljjcctvve)
Log(Objecthve
2-
M-L20
M-L20
0-
DT
-2-
TL
Vanilla
Vanilla L20
50
100
150
50
100
150
Steps
Steps
(a) Performance on 50,5
(b) Performance (((, 100")
Figure 3: Comparison of convergence speeds target distribution of Quadratic optimizees. We repeat the experiments for 10 times and show 95% confidence intervals are shown the figures.
112, 112) between the training and adaptation tasks, can lead to a reduction in generalization loss as defined in Equation (7). This claim has been corroborated by the results of our experiments, as presented Figure 4. When the a is similar to the training tasks (e.g., 10), implying smaller A12 and Āι2, M-L20 demonstrates superior testing performance. In con- clusion, incorporating training-like adaptation tasks can lead to better generalization performance.
LogOOjeetive)
-2.2-
M-120
TL
Meanwhile, it is reasonable to suggest that the task differences between adaptation and testing, denoted by (Aza, A2s). may also have an impact on M-L20's gener- alization ability. Intuitively, if the optimizer is required to adapt to testing optimizees, the adapted optimize should demonstrate strong generalization ability mizees that are similar. In order to have a deeper under- standing of the relationship between the generalization abil- ity and the difference between adaptation and testing tasks, we rewrite M-L20 generalization error in Theorem the 11 .Rrall:
-2.6
25
100
Figure 4: Performance on LASSO opti- mizees. We vary the standard deviation of the distribution used for sampling the weight matrix A for adaptation optimizees. We visualize both the mean and the confi- dence interval the figure.
other opti-
following form with o² = 2.. - aV (o),)) and 8
0 - aV o (m)rk) - (Ø.))
SM (1WMk ا- + || 2 + a of (pM)) - Vosi + TM
In Equation (8), M-L20 generalization error is partly captured by V ( v Vi (o which is controlled by difference in optimizers (i.e., (
From Proposition I, we know that this term is determined by difference in optimizees, denoted by Aza and Â23. Similar the results established in Theorem 1, we can deduce that superior testing performance is connected with smaller difference between testing and adaptation optimizees. This result has been demonstrated in Figure 4 where TL generalizes well with larger (more testing-like), Moreover, M-L20 also benefits from larger values (e.g., 100) in certain scenarios.
To summarize, both training-like and testing-like adaptation task can lead to improved testing performance. As shown Figure 4, training- like adaptation results better generalization in L20. One possible explanation is that when the testing task significantly deviates from the training tasks, it becomes highly challenging for the optimizer to generalize well within limited adaptation steps. In such scenarios, the training-like like adaptation provides a more practical solution.
CONCLUSION AND DISCUSSION
In this paper, we propose self- self-adapted L.20 algorithm (M-L200), which incorporated with meta adaptation. Such design enables the optimizer to reach well-adapted initial point, facilitating its adaptation ability with only a few updates. Our superior generalization performances in out- of-distribution tasks have been theoretically characterized and empirically validated across various scenarios. Furthermore, the comprehensive empirical results demonstrate that training-like like adaptation tasks contribute to better testing generalization, which is consistent with theoretical analysis. One potential future direction is to develop a convergence analysis for L20. It will be more interesting consider meta adaptation in analyzing 1.20 convergence from theoretical view.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENTS
The work of Y. Liang was supported in part by the U.S. National Science Foundation under the grants ECCS-21 13660 and DMS-2134145. The work of Z. Wang was supported in part by the U.S. National Science Foundation under the grant ECCS-2145346 (CAREER Award).
REFERENCES
Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffmmnn, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In Advances in neural information processing systems (NeurIPS), 2016.
Yue Cao, Tianlong Chen, Zhangyang Wang, and Yang Shen. Learning optimize in swarms. In Advances in Neural Information Processing Systems (NeurIPS), pp. 15018-15028, 2019.
Tianlong Chen, Weiyi Zhang, Jingyang Zhou, Shiyu Chang, Sijia Liu, Lisa Amini, and Zhangyang Wang. Training stronger baselines for learning to optimize. arXiv preprint arXiv: 20..000000000 2020a.
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang. and Wotao Yin. Learning to optimize: primer and benchmark. arXiv preprint arXiv:2103.1282. 128288 2021a.
Wuyang Chen, Zhiding Yu, Zhangyang Wang, and Animashree Anandkumar. Automated synthetic- to-real generalization. In International Conference on Machine Learning (ICML), pp. 1746-1756, 2020b.
Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Hyperparameter tuning is all you need for lista. Advances in Neural Information Processing Systems, 34:1 (678-11689) 2021b.
Xinshi Chen, Yufei Zhang, Christoph Reisinger, and Le Song. Understanding deep architecture with reasoning layer. Advances Neural Information Processing Systems, 33:1240-122 -2522 2020c.
Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matt Botvinick, and Nando De Freitas. Learning to learn without gradient descent by gradient descent. International Conference on Machine Learning (ICML), pp. 748-756, 2017.
Simon S Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh. Gradient descent provably optimizes pwerpparameterized neural networks. In International Conference on Learning Representations (ICLR), 2019.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient- model-agnostic meta-learning algorithms. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Generalization of model-agnostic meta- learning algorithms: Recurring and unseen tasks. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-leariig for fast adaptation of deep networks. In International Conference on Machine Learning (ICML), 2017.
Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. In International Conference Machine Learning (ICML), pp. 1920-1930, 2019.
Howard Heaton, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Safeguarded learned convex optimization. arXiv preprint arXiv:2003.0 2020.
Kaiyi Ji. Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In International Conference on Machine Learning (ICML), pp. 4882-4892, 2021.
Kaiyi Ji, Junjie Yang, and Yingbin Liang. Theoretical convergence of multi-step model-agnostic meta-learning. Journal of Machine Learning Research (JMLR), 2022.
10

---

Published as a conference paper at ICLR 2023
Haoming Jiang, Zhehui Chen, Yuyang Shi, Bo Dai, and Tuo Zhao. Learning to defense by learning to attack. arXiv preprint arXiv: arXiv:1811.01213, 2018.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Yoonho Lee and Seungjin Choi. Gradient-based meta-learning with learned layerwise metric and
subspace. In International Conference on Machine Leurning (ICML), 2018.
Chaojian Li, Tianlong Chen, Haoran You, Zhangyang Wang, and Yingyan Lin. Halo: Hardware- aware learning to optimize. In European Conference Computer Vision (ECCV), pp. 500-518. Springer, 2020.
Yuanzhi Li and Yang Yuan. Convergence analysis of two-layer neural networks with relu activation. Advances in neural information processing systems (NeurIPS), 2017.
Kaifeng Lv, Shunhua Jiang, and Jian Li, Learning gradient descent: Better generalization and longer
horizons. In International Conference on Machine Learning (ICML), pp. 2247-2255, 2017.
Alex Nichol and John Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint arXivv: 1oXiv:/803.000 02999, 2(3):4, 2018.
Ning Qian. On the momentum term in gradient descent learning algorithms. Neural networks, 12(1):
145-151, 1999.
Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of maml. In International Conference on Learning Representations, 2020.
HoHo Rosenbrock. An automatic method for finding the greatest or least value of a function. The computer journal, 3(3):175-184, 1960.
Sebastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.044477 (4746, 2016.
Shai Shalev-Shwartz. Ohad Shamir, Nathan Srebro, and Karthik Sridharan. Learnability, stability and uniform convergence. The Journal of Machine Learning Research (JMLR), 2010.
Jiayi Shen, Xiaohan Chen, Howard Heaton, Tianlong Chen, Jialin Liu, Wotao Yin, and Zhangyang Wang. Learning a minimax optimizer: pilot study. In International Conference on Learning Representations (ICLR), 2021.
Laurits Tani, Diana Rand, Christian Veelken, and Mario Kadastik. Evolutionary algorithms for hyperparameter optimization in machine learning for application in high energy physics. The European Physical Journal C, 81(2):1-9, 2021.
Paul Vicol, Luke Metz, and Jascha Dohl-Dickstein. Unbiased gradient estimation in unrolled com- putation graphs with persistent evolution strategies. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Pro- ceedings of Machine Learning Research, pp. 10553-10563. PMLR, 18-24 Jul 2021. URL https:/ htap:://ppoceediii pttpppeiiceen...c//o//p
.ariinnsmmm.......mmm.m.mm mlr www...rees...............m.mee.e /v13111991 wwll///viieo 9/vvcol22a html.
Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando de Freitas, and Jascha Sohl-Dickstein. Learned optimizers that scale and generalize. In International Conference on Machine Learning (ICML), 2017.
Yuanhao Xiong and Cho-Jui Hsieh. Improved adversarial training via learned optimizer, 2020.
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically structured meta-learning. In International Conference on Machine Learning (ICML), 2019.
Pu Zhao, Parikshit Ram, Songtao Lu, Yuguang Yao, Djallel Bouneffouf, Xue Lin, and Sijia Liu. Learning to generate image source-agnostic universal adversarial perturbations. In International Joint Conference Artificial Intelligence (IJCAI), 2022.
Wenqing Zheng, Tianlong Chen, Ting-Kues Hu, and Zhangyang Wang. Symbolic learning to optimize: Towards interpretability and scalability. arXiv preprint 20Xvv:2033.0657. 2022.
11

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1 RESTATEMENT OF ASSUMPTION
Assumption 5 (Restatement of Assumption 3). We assume Lipschitz properties for all functions (() = 1,2,3) follows:
a) 4(6 is M-Lisschitz, i.e., for any 01 and 82 [14(01) - 4(62)) E M - B2l1(i = 1,2,3).
b) (0( is L-Lipschitz, i.e. for any 01 and 0z. [0i( - v4(02) _ L| 02] 1,2,3) 3). c) (0) p-Lipschitz ie., for any 01 and 0z [0,) (02) S p|10 -021(i 1,2,3, m(z,0) 6) -ip-Lipschizz w.r.t. and M - w.r.t. Φ. i.e.,
() m(22, 2) [ [m(2z, ( 02)1 - Mm @1 622
- Mm || -22 for any and 22.
for any Φ1 and 02.
e) V²) is po-Lipschitz, i.e., for any Φ1 and 62. 11 (T(() -2
S pallor 62l1-
The above Assumptions (a)(b(e) also hold for stochastic (0), vi, and (g)((0(i 1,2,3).
A.2 PROOF OF SUPPORTING LEMMAS (LEMMA 12 CORRESPONDS TO PROPOSITION 1)
Lemma 1, Based on update procedure of (z(O), we obtain
T-1
T-1
VoOT(0) = Σ
II
(I (et
Proof. The (c( update process shown below:
0c+((0) = (,(6) + m(2,,).
we only consider = V=((0((0)GG)) = Vol(O.). then we obtain
v0l++1(0) V
= T Vem(v)(()))
(I Vim.VVVacce).oVVVii)))).lCll Vom(ol))))))
If we iterate the above equation from f to T, then obtain
T-1
ve0r(0)
Σ
1+ (1 (.0
j=it1
T (1
Vg Va00
i=1
We assume On is randomly sampled and independent from Φ. then we obtain
V (oOT(0)= = T-1
T-1 II
Σ
(1+ (1
==0
j=++1
Lemma 2. If we assume that (o((1) = -lo(02)) based on Assumption 3, then we obtain
[[0r(01) - -(r()))))( (((Mm) 1) - 11--1-111
Mm2
|1 &2ll = MMrll - -0211- (9)
Mm L
A12

---

Published as a conference paper at ICLR 2023
Proof. Based on the iterate procedure of (r()), we obtain
- - 8r(02)l
m(Voi((((60)))002))
(m (m
m mileeeeealhhooooommM/eelo..ll.l-
M Mmall01 2221 -
Σ
Mm MoalVo((((((i))) vo(((((02))
S(T M)2m2ll 22 + Mm
(o))(((()1)) (o2()(
(1)) $ - M2mll - 22 M L
T-1
11(((1)) (d2))l.
where (i) follows from Equation (1), (ii) and (iii) from Assumption 3. we further iterate from t to T, obtain
[10r(á1) (rph))-r((())))) (f())) (1) Mm2
11 d22 = Mor Φ2ll- MmiL
Lemma 3. we define Acli) Va $1), bused on Assumption and Lemma we obtuin
- S MAilloh - 6211,
where MAi Lmm + Lin LMg.
Proof. Based the definition of A:(Ø), have
|| |A(())) - A,()))
= -IVVe((Voi(e((0))) Φ1) V2m(Voi(((()))))0)
V=m((v(((0((1)) 22) V=m((v/((0(022)))))))
(
< Lmmlllo 02l1 L, V -o
L 11 021 -m
(10
(Tmm + Ioma [Moo))loh Φ211 MAill - Φ211
where (1) follows from Assumption 3, (ii) follows from Lemma 2.
Lemma 4. We first define B,(,) = Vim((Vo(ee(c))))))((((.. Based on the Lemma and
Assumption 3. we obtain
BB((1) (((d2) S Mx Φ211.
where MR: M MaapMoo
L.maa T.2 Moi-
A13

---

Published as a conference paper at ICLR 2023
Proof. Based on the definition of (o))), we have
[b( (((e))l
+ V.mmYo((00100)))411111000011 otwp:///0000//00mmmmm//oooo/a/aa
Mn 244
MmipMeell0r + hhapapaeeeeoaoa--annmm/////0ooa0
+ (0
(88)
MmapMerllor 22 + L d2 mm (o((O.((1)) - vo((0((<2))
S(MmipMor IJem2 T.ma M6:) 22 MBillø d2l1;
where (1) and (iii) follows from Assumption 3, (ii) follows from Lemma
L
Lemma 5. Based on Assumption 3 and Lemmas 1, 3 and 4, then obtain
[V - V0or(bb))l S LoT - Φ221.
(10)
where LOt IT-1 (1)
E T-1 Mm (1+ (m L) 1)2-i-2>0 23 MB(T+i-M-
Proof. Based on the definition of V voDT(0) in Lemma we obtain
( (02r(12)1
T-1 [+-Tmmtyii00---aaaa))00/0io---ad (mm
j=i+1
T-1
II [4 (0
j=f+1
T-1
[+ (+Biiiis((i) A((di)
II j=it1
I Br+i Ac((z))
T-1
I+ [++Bra Al(di)
II j=++1
(I+ (++Bri=)(0o)) Ac((b))
j=++1
T-1 +
T-1
II 1i1t1
I+ (++Br+i)(01)) Al(02) - II
I+ BT+1-=(02)) A1(02) (1+
j=it1
(18)
(1 (1) A.(02)) + Mm2
II (I+ Br++-s(di)) jmit1
T-1 II jmit1
(I+ (I+Brii-((b2))
(1+ 1 - A,(02)l + Mmz(1 + Mm L)) -i-2
[BT+i_((01) (
(1
- Qell + Mm2(1 + Mm L
A14

---

Published as a conference paper at ICLR 2023
T-1
M MB(T+i-))lo - 2211
T-1
T-1
Mn Mm1L)7---1 MAi
M1 + Mmi L)
Σ
MB(T+i-)) |1 621
j=it1
= Lerrllo Φ211,
where (i) is based on Lemma (ii) is based on the fact that A,(01) V2m(Vo(((((h)), Φ1). Br-t-j (61) = 2im(Vol(Or4-22)
Φ1) ( (tii follows from Assumption and (iv) follows from Lemma 3 and 4.
Lemma 6. Based on Lemmas 2, 5 and Assumption 3, we obtain
IV or (Ø1) - Vorr( S Lir 1l 2211,
where Lgr = Mor + LM Lgr is defined in Lemma 5, Mør is defined in Lemma 2.
Proof. We assume all functions share the same starting point do. then we have
|I (V - verr(02))
-
Vol(er()1)) vol(0r(0))) (btp::////ooffallooaooolo/1ll1)- ((rr(())
+ -oool(00r())))))0(((((()))))))))
68)
x M - T21l + LM p1 d2ll = (M + LM - Φ2ll = Lor 1101 6211.
where (1) from Assumption 3, (ii) from Lemma 2 andd 5.
Lemma 7. Based on the Lemma and Assumption 3. obtain
(o) V2br((2)) S Per Iloi - Φ211.
(11)
where Per = LeT + Mp + MATP.
Proof. We first compute the Lipschitz condition of V (o((O((o) as follows
= ( TiTTefdfaaTTTTiiiiccccCC-CTTTTTr
+ (0))) (V
(
2 Mar - 221 LoT LoT
= -Marp L)T 2211,
where (i) follows from Lemma 2, 5 and Assumption 3. Then, based on the definition of V ( we have
( V
= -
+1T - [V99:1c6//V//Vd4(((((61]1]
A15

---

Published as a conference paper at ICLR 2023
- +1T566/f001111110330000000000000
-Mer -MNea|/5555iiiicci-------/-11110 22 Mor
- -
+ + -Momp + M Φ21
-33LMor + Mp + M1 - d2ll = Pgr 11 - 0211,
where (i) follows from Lemma and
Lemma 8. we assume (o))) = ()) based on Assumption 3 and 4, we obtain
110 (6) ( S GBT:
where T is the iteration number and ouT = + Mm L) 42
Proof. Based on the iterative process of (,))), we obtain
(0) (+)
(
$ <110 + m (Vol (0) 1) - ()-1(0) mVVob((2-)).
[10)-1(0) 6 (0)] M. Ml vob201
<1(0)-100) (0) ( Vo
Vol2 1)
+ Mmi (0l2(0)11) ve(2(0?-1)1
(122
I (1 Mm+ 10 07_ (b) + Mm A12
where (t) follows from Equation (1), (ii) follows from Assumption 3, (tit) follows from Assump-
tion 4. If we iterate above inequalities from t to T 1, then we obtain:
A12
A
(0) - (0) s (1 Mm L)
OBT.
Lemma 9. Based on Assumptions 3 and 4, Lemma 8, we have following inequality:
||| C? S Aco.
where = (1= e {1,2}) and Acs = Lm (1 + Mn L)¹ A
Proof. Based on the definition of C., we obtain
|||| - c?Il = Vm ( Φ) - (
< <m Vol Volz(0!) -
Im A12 I. L - 0?
(0)
< Lmi (1 + Mm L) A = Aco
where (1) follows from Assumption 3 and 4, (ii) follows from Lemma 8.
A16

---

Published as a conference paper at ICLR 2023
Lemma 10. Then based Assumptions 3 and 4, Lemma 8, have following inequality:
I D21l I ADi.
where DJ= (ym((Vo 6) ()))) (i 0 T.j E 1,2), Anc Mmo(poo, A12) +
Lma Smaa(11+ + Mm 112 and σe, is defined in Lemma 8.
Proof. Based the definition of D/. obtain
|D D =IV m VV (ttp::/)__.iiillllcemmmcccc.C..0 V=
= =VVim(Vvdd())))))vv.c)) - (
+ V (1) V
< ( (+
+ 11
< Mmm V
+ 1.miL/// [ll)) (o.) + (o.) - vob(
Mmmi (01l0) - 0/ + 412) - Lm Lop + A12)
Mm (pos. + A12) + Lm M L) A12 Api.
where (i) follow from Lemma 8.
Lemma 11. Based on Assumptions 3, 4 and Lemma 1, we obtain
[V V (0)(
T-1
(L) Dcs + Mm Mua T-
Σ ADj)
s=++1
where Acs and ADj have been defined in Lemmas 9 and 10.
Proof. Based on the Lemma I, we obtain
| (+) +b) V (0))
T-1 II 1mit1
II + D +
(I+ D 0++ II (I+ D++
jmit1
T-1
II jme+1
(I + D C
T-1 II (I+ D++
(I D D 10 -C1++
Cf1
1mi+1
T-1 II jmit1
(I+ D D2 [10.11
(1 + Mma T
of Mmz(1 -maL)T-i-2
|D - D
T-1 T-
(1 Mm 7
T-i- Acs - Mm + Mm L
E
Aos AoI).
-=++1
A17

---

Published as a conference paper at ICLR 2023
where (i) follows from the definitions that DJ-
V,m(Voi,(()) () ( C Vmm V2 (O.) Φ), (ti) follows from Assumption and (iéi) follows from Lemma 9 and 10.
Lemma 12. (Correspond to Proposition 1) Based on Assumptions 3 and 4, Lemmas & and 11, we obtain
(o) - = = Q A12),
where Q 1=1+ MmIL.
Proof. We first consider Acs and A Dis we obtain
Acs m1 + M L)2 A12) 01Q'A12).
(12)
ADi =M + 112) + Lm1 L(1 + Mm L A12)
_ =0 11(Q'A12200 À 12) 01Q' A 12 + Α12)
(13)
where (i) follows because døi = (1 + M L)) 4.2 4 = O(Q' A12).
Furthermore, we consider the uniform bound for | [V (dbf(d))1, then we obtain
IV + (6) V aa 00){())1
T-1 QT---2 Q
Σ )
j=it1
@ QT--22 @Q A 212))
s=i+1
(T 11-1A12+(T -T- (7 + (QuT-i-2 )²-1)A12))
((T i 1) -2 A12)
(ii)
(jQ' LIQ--AA " + QT+ (12)
1=0
TQQ Q2T
where (i) follows from Lemma 11, (ii) follows from Equation (12) and Equation (13), (iii) follows because j = T i 1. Based the formulation of V Vorr(o) in Lemma 6, we have
| (0) v ( <MI ())()) v (b) Mor A12
@ " Q A12);
where (i) follows because MøT defined Lemma 2 satisfies that MAT = -QQT-1).
Lemma 13. Based on the Assumption and Lemma 2, we obtain
(1gr(()) - (d2))) < Mgrll d2211
where Mgr MM and Mør is defined in Lemma 2.
Proof. Based on the definition of gT (0), we have
(19r(01) - ((2)) - =1(((e(((1)) - 1(Or(02))
(3)
< ( - 0r 0r22))
A18

---

Published as a conference paper at ICLR 2023
(60)
S MM 11 Φ21
-M || - 02ll,
where (i) is based on Assumption 3, (tt) is based Lemma 2.
Lemma 14. Based on the proposition in Fallah et al. (2021), Assumptions I and 3, if we set
S min 21.1 BPur Mor }. Bk min(3, m(k++1) I((+tt))for ) B I 8 in Algorithm 1, then we have
P
M (1+ 310) (L
+ Por Mor
Mar VK
BIJMK I
& 0(1)
+
K
where Mgr is defined Lemma 13, Lir is defined in Lemma Por defiend Lemma 7.
Proof. Based on the Proposition 1 in Fallah et al. (2021), we obtain
M2 gr (1
("er + Pgr a gr
M Dr
E ((G+( (+ s O(1)
11
K
where (T( is defined in Equation (5). Based on the Assumption and the fact that Th
arg min (Ø). have
21 - 2 (c)
M (1 Bu)
(4 + Per
Por
Mgr
<0(1)
1r
K
VK
Lemma 15. Based on Assumption and Lemma 13, we have
- 2/2M
PV8N
where N is the sample size.
Proof. Based on Assumption 1 and Lemma 13, from Theorem 2 in Shalev-Shwartz et al. (2010), with probability at least 8, we have
4M2
9T (t!: (p!)
8pNN
Furthermore, based on Assumption and the fact that Φ! = arg min ())( we obtain
| - o!'2 < 2 (b)(Tt) - (b!)
2 4M2
8M2 8p2N
/ JuN
We take the square root from both side and obtain:
2 2v
1 0.1l
WV8N
with probability at least 8.
A.3 PROOF OF THEOREM 1
Based on our definition of generalization error for the algorithm,
(0 a avoot(hrxx) - -7
( 207 The + Zt &o aN (f -t(
A19

---

Published as a conference paper at ICLR 2023
(68)
_ Mor 10 Pis- + 0 + - aV aV aV -V (14)
<M 10 @ 1l + M Mr 0o 0 + Mr 0 0.11
+ Mor all V (Phse) Vost( || + M 0 V 00 MK) (x Vori
((ar + Ly a) ΦN o.. + M MMr 11 0.1 Mgn 11 -6.1
+ a 2 (OMK) - VoDI
where (i) follows from Assumption 2, (ii) follows from Lemma 13.
Furthermore, considering Algorithm 1, if we set a S min 2L 8por Mor b Sr min(3,
u(k+1)) for
B Σ & based on Lemma 12, 14, 15, with probability at least 1 - 6, we obtain
(O a ( (0.))
Mgr 112
Lgr - Por or
M BVK
2 BT μBV ON
(Mar Mgr Lyg a)
(1)
MM
V
3K
+ M M " + A12))
where 813 = |l 2il1 Q = Mm1 K the step number for update, N the sample size for training.
Then for Lipschitz term Mgr defined Lemma 13,
Mgr M = -
where Mør defined in Lemma 2 satisfies Mar = O(QT-1).
For Lipschitz term Tor defined in Lemma 6, first compute the order for Lør which is defined in Lemma 5, then we obtain
Lør =0
Q² M
MA
+
Σ
QT-i-2
Σ
MB(T+i-))
j=+
T-1
T-1
=0
Q Q + Σ
QT-i-2
Q T+i-j-1
==
j=it1
0
Σ
Q
Σ
20
O(TQT-2 - Q2T-2),
where (i) follows from Lemmas and 4. Then, we obtain
Lgr LM + LM
=O(TQT-2 22 Q2T-2)= - + Q2T
For Lipschitz term Pyr defined in Lemma 7, we have
Per = 3TM Lagr + MM + MaTP= = 33.
Then, the proof is complete.
A.4 PROOF OF REMARK
In terms of M-I200 generalization error, based on the Equation (14) in Appendix A.3, we have
( a ( -
<Map 11 The + à. - 0º + 02 - + a a aV -V
< <M 1 Throll + M M7 - &!'l + M Mgr / ( Voot(Ø), Ill
Mor 513,
A20

---

Published as a conference paper at ICLR 2023
where 613 | -211-
In terms of Transfer Learning L20 generalization error with learned initial point Φk, we have 07 a a (ff - (o))
<M 11 aV V0 21.
Mar 11 a 2 - ( a V ( - Ø. -b'll
<Marll 110 M= + Mgr 10 211l + M 17 | (0f -Vosi (M )1l + Mgr
where (1) follows from Ø @hs. aV 813 10 02.1-
Then, the proof is complete.
A.5 PROOF OF EQ. 8 IN SUBSECTION 5.3
We assume that / 23 aV 29T M.), then have
G av V of 9
< Mor
Mo 10 a VoOi mK) 221
Morll 11 aV Vo0 kM
k)
M + oaVost -.'ll V00i(02 - Vosi
33
SMar |0 TMall + MØr 10 25 M a
Marll 11>r-- --
Then, the proof is complete.
A.6 ADDITIONAL EXPERIMENTS
New Optimizees: Rosenbrock We conduct additional experiments with substantially different optimizees, i.e. Rosenbrock (Rosenbrock, 1960). In this case, the optimizes are required to minimize a two-dimensional non-convex function taking the following formulation:
f(z, v) (z 1)2 100 =²²)²
(15)
which is challenging for algorithms to converge to the global minimum (Tani al., 2021).
We specify Datapt and Dest to be the family of Rosenbrock optimizees with randomly sampled initial points from standard normal distribution. In contrast, the training optimizees still LASSO with mixture of uniform distribution from which the coefficient matrices are sampled. The experiments repeated for 10 times, with all the algorithms receiving identical adaptation and testing samples in each run. Figure A5a shows the curves of the logarithm of the objective values generated by different methods, where our proposed M-L20 outperforms other baselines significantly. At 500-th step, the (mean, standard deviation) of the logarithmic objective values for /anillla L20, TL. DT, are are ((0.977, 0. 0.225). ( (-21700, 1.312), 4.8664 0.395), (-6.832, 0.445)) which provides numerical supports of the advantage of our methods.
M-L20)
New Evaluation: Interpolation
To obtain new optimize weights, we employ a linear interpolation strategy between two adapted optimizers. The first one optimized on the optimizees that are similar to those used in training. and the second is optimized on the optimizees that are similar to those used in testing. We introduce factor a to control the interpolation between the two weights, denoted by wi and w2, respectively, and caluclate the new weights as follows:
w = aw + (1 W2
In Figure A5b, we present the mean values of the logarithmic loss, as well as the 95% confidence interval. The results of TL and M-I.20 validate our claim that adapting training-like optimizees tend to yield better performance than adapting to optimizees that more resemble the testing optimizees.
A21

---

Published as a conference paper at ICLR 2023
M-L20
-1.8-
Log(Otjactive)
TL
-2.1
M-L20
-2.4
Vanilla L20
100
200
300
400
500
0.00
0.25
0.50
0.75
1.00
Steps
(a) Convergence speeds on Rosenbrock optimizees.
(b) Convergence speeds on LASSO optimizees, with repeat the experiments for times, and present the different interpolation weights a. Both the mcan and 95% confidence intervals are shown in the figure.
the 95% confidence intervals are shown in the figure.
Figure A5: Visualization of additional experiment results.
A22


Reviews:
[{"review_id": "inZ-UWNBhr", "reviewer": "Reviewer_5uH7", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes the M-L2O algorithm, as a substitute for the L2O algorithm, and then provides theoretical analysis and numerical experiments of the M-L2O algorithm.", "summary_of_the_review": "In summary, the reviewer cannot provide a clear evaluation of the quality of this paper due to the lack of introduction on the problem formulation, assumptions,  and clarity on the notations. The reviewer encourages the authors to largely rewrite the paper to improve the clarity on the aforementioned issue in the rebuttal.", "strength_and_weaknesses": "Strength:\n1. The theoretical analysis looks solid\n2. The numerical experiments corroborate the theoretical results.\n\nWeakness:\n1. The L2O part problem definition needs more descriptions – in section 3.1, it would be much better to introduce the different notations with better characterizations:  (a) It would be better to provide more description on the relationship between $\\phi$ and $\\theta_t$. \n(b) Is $\\xi_j$ a vector data sample or it is a parameter that indexed the task number? \n(c) What is $z_t$? \n(d) In the sentence above equation, $\\zeta_t$ is referred to as a data sample, while in the first sentence of section 3.1, $\\xi_t$ is referred to as a data sample, what is the relationship between $\\zeta_t$ and $\\xi_t$? \n(e) Where does equation 2 come from? Perhaps the author can provide more introduction/motivation on why $\\nabla_\\phi \\theta_T(\\phi)$ has the form in equation (2)?\n2. The intuition of M-L2O is not clearly stated – why is the empirical risk (equation 5) related to MAML?\n3. The notations in section 3.2 are confusing – where do $g_T^1,g_T^2,g_T^3$ appear in Algorithm 1?\n4. The assumptions need clarifications:\n(a) (Minor) Assumptions 1, 3, and 4 seem to be standard, but it would be better to address their appearance in some prior papers.\n(b) (Major) Why can we directly assume $\\widetilde{\\phi}^1_*$ and $\\widetilde{\\phi}^1_{M*}$ satisfy the conditions in assumption 2?\n5. The main result (theorem 1) is a bit confusing – the last error term in Theorem 1 has an exponential dependency on $Q$, and the author proposes to set $\\alpha\\leq \\mathcal{O}(1/TQ^{3T-4}+Q^{4T-4})$ to avoid such exponential dependency, which implies that $\\alpha$ should be exponentially small. And if the reviewer understands correctly, the new $\\alpha$ term seems to be the major novelty of this paper (as suggested in equation 5), so does this mean that the proposed M-L2O actually does not differ too much from the original L2O since we will be setting $\\alpha\\to0$ anyway?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "fOaM5Z9vTb", "reviewer": "Reviewer_GEXL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Learning to Optimize (L2O) has emerged as a data-driven way to derive “learned optimizers” for specific problem classes. However, for very different unseen problems those learned optimizers can fail badly. This paper is the first to discuss L2O’s test-time self-adaptation to out-of-distribution tasks, providing both theoretical and empirical results.", "summary_of_the_review": "In summary, I think this paper targets a very important problem and critical limitation of L2O, that will improve the applicability of L2O. The proposed solution seems reasonable with a well defined nested optimization problem, and the associated theoretical analysis well justify the motivation of the proposed solution.\n\nThere are some minor concerns in the \"Weakness\" part, if the authors can well address my concerns I would be able to change my score.", "strength_and_weaknesses": "Strength:\n-\tThis paper tackles a very important problem and inherent limitation of L2O: its performance is not ensured on tasks different from those seen in training. If resolved, that can remove a notable hurdle for L2O in practice. \n-\tThe authors formulated an MAML-like nested optimization, to locate in well-adapted region where a few adaption steps enable optimizer to generalize well on unseen tasks. The formulation is solved by meta-training with theory backups.\n-\tTheoretical analysis reveals that training-like adaptation can mean better generalization, which experiment results support. This is a meaningful new insight. \n\nWeakness:\n-\tIt is hard for me to catch whether/what the main innovations are for the theory part. The idea seems to be a direct combination of MAML and L2O, both belonging to the meta learning family. \nThe authors are invited to elaborate more on: is their theoretical result reused/re-instantiated from some known meta-learning result? Or by some direct combination of MAML result and L2O result each? Or they have actually made noteworthy theory contributions? \nI believe the clarity of Section4 will benefit a lot by adding such discussions. Currently, I find it very difficult to assess the authors’ theoretical contributions as those are poorly contextualized. The whole Section 4 surprisingly has not cited or discussed a single theory paper!\n\n-\tThere are existing L2O works studying generalization by theory, such as (Chen 2020d). However, the authors fail to concretely discuss how their generalization results differ from/compare with Chen et. al.\nAnother relevant paper the authors fail to cite and discuss is: “safeguarded learned convex optimization”, Heaton et. al.\n\n-\tThe experiments are in general consistent and informative, but unfortunately on very simple test problems only. I would expect at least some constrained optimization, nonconvex optimization or NNs. The authors are encouraged to add some such result. This could be considered as a minor point, IF the authors are able to clarify their theoretical contribution to be major (rather than combining existing off-the-shelf ingredients).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_C7xmA-KVC", "reviewer": "Reviewer_2sEx", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to add meta-adaptation into the learn-to-optimize pipeline. The authors present an algorithm for adapting L2O inspired by MAML in algorithm 1. Under standard assumptions in convex optimization (strong convexity and Lipschitz continuity) the authors then derive generalization bounds of the proposed method. The authors verify the efficacy of the proposed method on LASSO and quadratic objectives functions with small synthetic datasets.  \n", "summary_of_the_review": "The paper combines meta-learning and learning to optimize, proposing a natural algorithm and deriving some generalization bounds for it. Experimentally, the authors show some benefits on small synthetic tasks with the lasso or quadratic objective. While the paper is technically sound, the paper does little to demonstrate any practical utility or new theoretical insight. Thus, I recommend rejection.\n\n", "strength_and_weaknesses": "Strengths:\n\n* The paper is relatively well written\n\nWeaknesses:\n\n* The novelty is limited.\n* The experiments are very limited, using toy problems and small synthetic datasets.\n* The assumptions of convexity might not hold in practice. \n* The utility or broader interest of the method is unclear.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "SuV1VVlmo7_", "reviewer": "Reviewer_Sfir", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of OOD generalization in L2O, i.e. meta-learning an optimizer that is effective on objectives not seen during training. It proposes an algorithm based on MAML (Finn et al. 2017) and analyzes its generalization ability, measured by the loss at testing time. Unsurprisingly, generalization depends on the difference between the training and testing optimizees, data size, and training length. The authors show that their algorithm M-L2O generalizes better than no meta-learning at all. Experiments are done on quadratic and LASSO benchmarks and confirm the theoretical results. M-L2O also improves on directly applying MAML to L2O.", "summary_of_the_review": "I think that the proposed algorithm is an interesting approach to tackle OOD generalization for L2O; it is clearly presented and the authors provide both theoretical and empirical analysis. However, there are some points described above (strength of assumption 2, remark 2, standard errors) that raise some questions about the claims made in this paper. \n\n# Update\n\nThe majority of the weaknesses I raised were adequately addressed by the authors, so I raised my score from 3 to 5.", "strength_and_weaknesses": "# Strengths\n- The presentation of the mathematical results is generally pretty good. The authors clearly state the assumptions and explain the results intuitively.\n- This paper makes distinctions between the optimizees used for adaptation and testing, which is unusual in the literature as far as I know. However, I think this is a better approximation of practical situations, where during testing we may have access to small amounts of data that may or may not be drawn from the current distribution of interest.\n- The proposed algorithm is a straightforward simple modification of directly applying MAML to L2O, basically adding a gradient step to the objective. The authors provide both theoretical and empirical analysis, and it seems M-L2O empirically outperforms the baselines.\n\n# Weaknesses\n- The notation is messy. In section 3.1, it is not clear where the different tasks (line 3 of Alg. 1, $g^1, g^2, g^3$) come in. I think it would be more helpful to explain the general structure of L2O algorithms first. $Q^i$ is not defined in Proposition 1.\n- Assumption 2 seems fairly strong. Intuitively it makes sense but I think it would be good to have some evidence, theoretical or empirical.\n- As far as I understand it, vanilla L2O does not contain any meta-learning, the optimizer is random. In this case, vanilla L2O would be a misnomer, and the main baselines of interest would be Transfer Learning and Direct Transfer. The figures do suggest that M-L2O may be able to improve over those two algorithms. Therefore, should Remark 2 compare between M-L2O and Transfer Learning?\n- Standard errors are not provided in the figures or tables. To judge which algorithm is statistically significantly better, uncertainty estimates would be helpful.\n\n## Minor\n- In Theorem 1 and Remark 2, are norm symbols missing from the left hand side of the inequality?\n- What form does the optimizer $m$ take in the experiments?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "25cmhR_QWq", "reviewer": "Reviewer_egHk", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper investigates the generalization of L2O on unseen test cases that substantially differ from the training distribution. It proposes a self-adapted L2O algorithm (M-L2O) incorporated with meta-adaptation. The generalization advantages of M-L2O over out-of-distribution tasks have been theoretically and empirically validated.", "summary_of_the_review": "Please refer to the comments on strengths and weaknesses. ", "strength_and_weaknesses": "This paper does seem to have merits, but they are largely compromised by the unclarity in the current manuscript. See below:\n\nStrength.\n- The unseen generalization is an open L2O challenge that was unexplored before. The authors demonstrate a novel solution that outperformed naïve baselines, such as direct transfer learning. \n- The authors presented theoretical evidence that their meta-adaption design grants M-L2O optimizer faster adaption ability for out-of-distribution tasks and can have smaller generalization errors, compared to vanilla L2O.\n\nWeakness & Questions.\n- It is unclear to me whether the theoretical assumptions made in Section 4.1 make sense or not for L2O. As far as I know, L2O adopts an LSTM to predict the update. How can an LSTM g function be strongly convex? Also, the Lipschitz condition can easily get trivial under recurrence.\n- Experiments are short of MANY details. For example, nowhere in the main text or supplement, did the authors report what L2O algorithm they actually used for all experiments! That shows the paper was finished in a hectic rush, lacking serious proofreading and also making the comparison fairness questionable. \n- The authors conclude that a training-like optimizer adaptation task outperforms a test-like one. That is non-intuitive, and I’m not fully convinced: what if we smoothly interpolate the parameters from training to testing, in a curriculum-learning way? I also suspect Figure 3 is because training/testing cases are still quite similar. \n- The optimization problems tested are too simple: only Lasso and quadratic.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster