Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:
This paper studies meta-learning in the mixed linear regression setting, focusing on the effect of the within-task step-size on performance. For over-parameterized, under-parameterized, and NTK regimes they derive expressions for test-time loss that suggest that negative or close-to-zero learning rates are optimal, and provide experiments that closely match these results. However, some aspects of the mathematical approach are unclear, and the work's impact is limited without an investigation of the consequences of the analysis.

Strengths:
1. Understanding meta-learning in simple settings and focusing on the effect of learning rate are worthwhile goals.
2. The authors provide experimental evidence closely following the outlined theoretical results.

Weaknesses:
1. The mathematical results in the paper are difficult to follow carefully. It is still unclear to me what objective is being solved by the meta-training procedure, and various derivations in the appendix seem non-rigorous, such as replacing denominator terms by expectations. Formal statements and some proof sketches would be helpful. See also Questions 1-4.
2. No experimental investigation beyond the limited settings studied theoretically. The authors’ investigation leads to a clear prescriptive suggestion—use of negative within-task step-sizes during meta-training—that should be investigated on actual data to get a sense of whether the analysis extends to practical settings. This would be useful to do in both linear settings (see e.g. the experiments in Denevi et al. (2019)) and for standard meta-learning tasks. See also Questions 5-6.
3. The analysis is limited in not being able to handle representation learning or the case of label shuffling among tasks. Motivating empirical work such as the paper of Raghu et al. (2020) suggests that representation learning is a key component of being able to meta-learn without inner loop updates, while theoretical work (e.g. Saunshi et al. (2020)) suggests that the linear setting studied here cannot account for successful meta representation learning.

Questions:
1. The paper seems to suggest the objective ignores task structure completely, but in this case how does the within-task step-size affect meta-training at all?
2. How many steps of MAML are used in the analysis?
3. The line “All of the above distributions apply independently to each task and dataset (training, validation, target, test).” seems to suggest that data for the same task is drawn from different distributions - is this the case and if so how it this at all justifiable in the meta-learning setting?
4. In what references can the Gaussian moment results (36-46) be found?
5. How closely do experiments follow the analysis in non-asymptotic settings?
6. In the NTK results, why not investigate agreement between theory and experiments using kernel matrices obtained from actual networks rather than probability distributions on symmetric matrices?

Notes:
1. “their performance doesn’t seem to stop improving when adding more data and computing resources” - why would we expect it to?
2. “A meta-learning problem is solved by a bi-level optimization procedure: an outer loop optimizes meta-parameters across tasks, while an inner loop optimizes parameters within each task (Hospedales et al. (2020)).” - not all algorithms do this, c.f. Reptile (Nichol et al., 2018).
3. “recent papers argue that a simple alternative to meta-learning is just good enough, in which the inner loop is removed entirely (Chen et al. (2020a), Tian et al. (2020), Dhillon et al. (2020), Chen et al. (2020b), Raghu et al. (2020)). This is surprising, because this approach merges all source tasks into a single big training set, and it does not even distinguish between different source tasks during training.” - the paper of Raghu et al. (2020) does not remove the inner loop during training and still distinguishes between source tasks, since gradient updates still use data from the same task. Actually merging all data into a single big training set was shown to perform poorly by Finn et al. (2017).
4. “In the problem of mixed linear regression, we prove that the optimal learning rate is always negative in overparameterized models. The same result holds in underparameterized models for small values of the learning rate.” - the second sentence is unclear. The learning rate is negative when it is small?
5. “none of these studies look at the effect of the learning rate” - in the convex setting, Khodak et al. (2019) connect the within-task learning rate to the task similarity.
6. “Note that each task is characterized by a different distribution of the data, and we separate the training Dt and validation data Dv.”  - it is unclear whether these distributions are empirical distributions or population distributions. If they are empirical distributions then minimizing (1) is not the “goal of meta-learning” but an objective used to achieve the goal. If they are population distributions, are Dt and Dv different? Why and how?
7. What does the equality-like symbol in (5) mean? “Asymptotically equal to”?
8. Numerous equation reference numbers are missing parentheses.
9. “learning of the meta-parameter ω is performed by the outer loop” - still unclear which objective is being used for this. (1)?

References:
- Denevi, Stamos, Ciliberto, Pontil. “Online-within-online meta-learning.” NeurIPS 2019.
- Khodak, Balcan, Talwalkar. “Adaptive gradient-based meta-learning methods”. NeurIPS 2019.
- Nichol, Achiam, Schulman. “On first-order meta-learning algorithms.” arXiv 2018.  

# Update after rebuttal phase

Thank you to the authors for engaging with reviewer comments. I think the paper is much clearer now, and the additional results in Figures 5 and 6 indicate that the analysis may be relevant for practical meta-learning settings. I am not sure of the necessity of the new data-generating plot for mixed linear regression in Figure 1 (my uncertainty here was resolved with words); the authors might consider using the space for putting Figure 6 (currently in the appendix) in the main paper, or for additional experiments. Two more notes:  
1. It may make the paper easier to read if the appendix were part of the same PDF as the main paper and not in the supplementary material.
2. While the experiments are perhaps not difficult to reproduce, code would be helpful to the community.

I am increasing my rating to a 6, as I believe the paper presents an interesting result with sufficient evidence. I am not giving a higher rating as I think the paper's impact would increase substantially with experiments on actual data, either in the mixed linear or deep net setting. For future versions of the paper, I encourage the authors to consider adding such results.
review_title: Potentially interesting but still incomplete investigation of step-sizes in meta-learning.

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: 

This paper uses random matrix theory to study meta-learning in mixed linear regression and finds that the optimal learning rate for the inner loop/training is negative in order to minimize the test/adaption error. The results are interesting and novel. However, there are some concerns regarding the practical relevance and presentation of the results. 

Major comments:

1. Implementation of negative learning rates in practice: This paper provides an interesting perspective that a negative learning rate could reduce the test error. My concern is with a negative learning rate, the training loss $\mathcal{L}^{(i)}$ in Eq. (3)  increases and the algorithm may not converge (at least on the training sets). In practice, for example, on deep learning models motivated by this paper in the first paragraph, how do you decide when to stop training parameters $\theta^{(i)}$ and $\omega$? How could you use the results in this paper to provide some guidance? 
2. Presentation of the main results: I would suggest the authors formally state the results in theorems or propositions. Currently,  the main results are presented in Eq. (5), (9), (10), (15), and (16), that seem to be informal and need clarification. First, how is $\bar{\mathcal{L}}^{test}$ defined? Second, what does $\simeq$ mean in this context? If it means ````" approximately equal to," then what is the order of the estimation error? Third, the results are derived using mean-squared loss as shown in Eq. (25) and (28) in the appendix. It is helpful to be explicit about the loss function in the main text. Fourth, the loss function does not seem to have a regularization term. In the overparameterized regime, would the model suffer from overfitting? 
3. Experiments: Could you elaborate more on why the theory matches the experiment pretty well in Figures 1.a, 2.a, and 3.a, while not the case in Figures 1.b, 2.b, and 3.b (especially Figure 3.b)? If I understand correctly, the data generating process in the simulation follows the assumption in the main results. Is it because the estimation error in $\bar{\mathcal{L}}^{test}$ (the terms omitted in the RHS of $\simeq$) is not negligible in finite samples? Also, is the curve in Figures 1.b and 3.b robust to the choice of parameters? It may be helpful to include a few other simulation setups in the appendix. 
review_title: Interesting findings on negative learning rates in meta learning

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary
The authors of this paper prove that the optimal learning rate for MAML is negative under mixed linear regression and nonlinear regression with overparametrized models. They verify that theoretic bounds align with numerical experiments.

### Comments
* As the authors note, "theoretical work is still lagging behind," so theoretical research to explain advances in meta-learning is useful.
* I think the clarity could be improved, especially in section 3. It was a bit difficult to understand notation and the exact setting. For instance, I think it'd be preferable to use a different Greek letter for the learning rates, rather than $\alpha_t$ and $\alpha_r$. 
* I performed a surface level check of the proofs, and the results look correct to me.
* It would be nice to discuss more the practical implications of the results or whether there still exists a divide between theory and practice.

### Recommendation / Justification
I vote that the paper is marginally above the acceptance threshold. I think the results presented are interesting, but it is not clear to me what the implications are. It would be nice if the authors explored whether the setting they analyze aligns with more commonly used meta-learning tasks.

### Questions
* In the last paragraph of Section 3, are you referring to the learning rates at meta-training and meta-testing time?

### Minor feedback 
* not only the performance -> not only [does] the performance
* These results help clarifying -> These results help clarify
* In general, I think it is better to reference equations as Equation 13 rather than 13.

review_title: Official Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary:
This paper gives much needed attention to the theoretical underpinnings of modern meta-learning algorithms such as MAML; it introduces a novel formal argument, discovers surprising implications and follows through to show that such predictions hold experimentally, despite being counterintuitive.


Strong points: 
- Novel analysis of a practically important aspect of MAML-like meta-learning algorithms: setting the learning rates for training and adaptation; surprising theoretical result (of negative learning rates being optimal in some cases during meta-training) is well evaluated in controlled conditions.
- Honest discussion of limitations and good intuition is provided for applicability of the work. This is not too hard to do, but so many papers don’t provide it. Great job!
- Writing is clear enough, although the paper is dense.
- The authors don’t discuss previous empirical works (e.g. Meta-SGD/LEO) where meta-learning of the learning rate leads to negative inner-loop learning rates for some parameters, but such experiments actually provide further evidence to back up their claims, this time in SOTA deep models.


Weak points:
- Unfortunately, the paper includes only toy-task experiments, even by the standards of  meta-learning research.


Recommendation and Rationale:
I strongly support acceptance because this paper contains much needed fundamental work on theoretical underpinnings of modern meta-learning.

review_title: Important theoretical study of MAML with counterintuitive findings, sheds light on empirical observations

Meta-review: While this paper would be significantly improved with experiments on real data, the reviewers all agreed that there is value in the ideas and simple experiments in this paper and all voted for acceptance after the discussion period.

We encourage the authors to consider adding an experimental evaluation in more realistic settings (e.g. with real data) in the final version of the paper.
Decision: Accept (Poster)