Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper studies the two learning rates \alpha and \beta used in Model-Agnostic Meta-Learning (MAML) algorithm by [Finn et al., 2017]. MAML is known to be difficult to train, and part of the reason why is the need to tune the two learning rates. Under simplifications, the paper derives some necessary conditions on \alpha and \beta for the MAML iterates to converge to local minima, and then verifies the theory by experiments on synthetic and real-world data.

Overall, I think this paper should be rejected. In my opinion, this paper analyzes a simplified setting which is far from the original MAML setting and yet derives conditions that are not very meaningful or useful. Although it is interesting to see some match between theory and experiments (especially Fig 4), the theory part seems to have room for improvement, and I will detail the reasons in the following.

I believe that the analysis is done on an overly simplified setting and easily breaks without such simplifications.
- The paper assumes that training set is equal to test set and these sets do not change over iterations, which is essentially equivalent to assuming full gradient access; this is different from the stochastic setting of MAML. 
- The "necessary condition" is derived in the case where there is only one task, and there is no discussion on generalizing to some other tasks. This means that the setting is too simplified so that it is not even a meta-learning setup.
- In the extension to multiple tasks, the paper derives "sufficient conditions" to a set of "necessary conditions" for convergence of MAML to local minima. This means that, if we consider the sets of events A = {convergence of MAML to local min}, B = {necessary conditions for A}, and C = {sufficient conditions for B}, A is a subset of B and C is a subset of B and nothing can be specified between A and C. In the extreme case, A and C may be even disjoint subsets of B. Thus, at least in theory, the conditions for multiple tasks do not tell us anything about convergence.
- The analysis relies on a number of "A is approximately equal to B" arguments without careful handling of errors, e.g., (3) and (4), and Tg = 0.

I also have concerns about the correctness of the analysis in the single task case.
- In Section 3.1.2 and eq (12), the paper reparametrizes \theta to v and analyzes updates on v. How is (12) obtained from the original update rule of \theta? In fact, the paper analyzes steepest GD but doesn't provide the explicit update rule; for example, is it steepest GD with respect to which norm? It'd be helpful to have the details in the main text.
- Even if (12) is true, there are pathological cases lying in a set of measure zero, where (12) converges even when (13) is not satisfied. For simplicity, consider v(t+1) = G v(t), where G is a real symmetric matrix. Assume all but one eigenvalue \lambda_1 are greater than 1 and |\lambda_1| < 1. Let v_1 be the corresponding eigenvalue for \lambda_1. Then, if v(0) = v_1, the sequence v(t) converges to zero. This means that the "necessary condition" derived in (16) may not actually be a necessary condition for convergence.

There are also some points in the main text that doesn't describe the setting clearly; if the reader has no prior knowledge of MAML algorithm, they may get confused.
- In the update rule (1) and (2), it'd be better to mention that \nabla_\theta L_\tau (\theta) is an *estimate* of the true gradient using the training data and test data, respectively. In their current status, the gradients in (1) and (2) will read as the full gradients of the population where the training/test data points are sampled from.
- Section 2.1 only introduces the case where the update (1) is done only once in the inner loop. However, later Section 2.2 and Section 3, the paper says "only one step is taken for update...". Without the prior knowledge that there are versions in which multiple updates (1) are done, the readers can easily get confused.

Minor comments
- The word "minima" used throughout should better be corrected to "local minima," as mere minima may be understood as "global minima."
- In the abstract, there is a phrase "in contrast to the case of using the normal gradient descent method." Which setting do you mean by the "normal gradient descent"?
- After equations (3) and (4), the paper says "The above is known as the first-order approximation...", but as far as I'm concerned, the first-order approximation version of MAML is (3) without the hessian term, not (4). This statement can potentially be misleading.
- Column vectors / row vectors are mixed up. In (3), the partial derivatives are row vectors, but in (4) g_\tau(\theta)'s are column vectors. However, right above (3), g_\tau(\theta) is a row vector this time.
- Did (5) come from the approximation (4)? If so, as \tilde L(\theta) is defined to be L(\theta'), the two sides of (5) are "approximately equal" not "equal."
- In Eq (19): P_\tau (\theta - \theta^*) -> (\theta-\theta^*).
- At the end of page 6, what do you mean by "not a sum but a mean"? Isn't sum of n things equal to the mean, except for a factor of 1/n?
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The authors study a method to help tuning the two learning rates used in the MAML training algorithm. First, they derive a necessary condition for the convergence of the gradient descent in the single task setting. The condition relies on the eigenvalues of the Hessian of the task loss. This condition is reminiscent of convergence criteria for gradient descent on quadratic objectives, and the authors make the interesting observation that the criteria for the exterior learning rate beta depends on the interior learning rate alpha. 
However, in the setting of interest that is the multitask learning, the trick used to analyse the eigenvalues doesn’t work anymore. To circumvent this issue, the authors provides a sufficient condition for the multitask equivalent of the necessary condition to hold.

The derivations are correct (few typos, see nitpick below) and the paper is nicely presented. I’m not a MAML expert so I’ll let the other reviewers judge how this paper compares to the current literature. However, I think that the empirical work can be pushed further. The experiments on Omniglot and MiniImagenet are coherent with the theory, but I am not completely sure of their impact. Indeed, the learning rate domain on which MAML converge seems coherent with what the authors are predicting, but it doesn’t allow us to choose the learning rates before training the model. It doesn’t help that the criteria relies on the spectrum of the Hessian evaluated at a critical point, which of course is not known before convergence in non quadratic setting.
The paper would be way more convincing if the authors could use their findings to describe some more precise heuristics to tune the learning rates, and conduct a proper comparison of its performance with other methods.

As a result, I think the paper is exploring an interesting direction, but that the empirical work might be too preliminary for publication.

Nitpick:

- Notations in 2.2 are a bit confused.The gradient of L wrt theta is denoted first with nabla, then with partial derivatives. Nabla is then used to denote the Jacobian of theta’ wrt theta. There is a small mistake in the 4th line of 2.2. Chain rule for gradient of L wrt theta’ does not give (d L / d theta) times gradient but instead give gradient times (d L / d theta)^T. The error is fixed when passing from (3) to (4) so it doesn’t impact the paper.
- Also, a transpose is missing in the taylor expansion on page 3.
- "A multilayer perceptron with two hidden units of size 40" -> layers?

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: In this paper, the authors tackled one important research problem in MAML, i.e., the optimization instability, by investigating the two learning rates. Though I appreciate the theoretic contribution this work makes, I am not sure with the practical significance of it.  Below please find my detailed comments. 

Pros:
-	In this work, the authors focused on an important problem – training MAML is kind of unstable and tricky, so that developing guidelines that stabilize MAML or its first-order approximations is of significance. 
-	This work theoretically discusses the relationship between the inner loop learning rate and the outer one, under a set of assumptions. 
-	The paper is well written and easy to follow.

Cons:
-	Some of the simplifications for proving are empirical, so that the proof itself is not that rigorous. 
  o	For example, In Section 3.1.1, the authors ignored Tg based on an observation that Tg is small. However, even in the Appendix, the authors did not explicate the experimental setting where they reach such a conclusion. Will Tg be always small then, in any task and any dataset?
-	The take-away of this work is not clear, in other words, it makes small contribution to the practical training of MAML. 
  o	Practically, we often train MAML in 5 or 10 steps instead of only one step. Will the conclusion apply to such setups? Or will the model itself diverge during the inner loop as more steps are taken, provided with a larger value of \alpha?
  o	Practically, we merely use vanilla SGD. The figure in the Appendix shows that the main conclusion actually does not apply to Adam, which disempower the practicability of the theoretical guideline. 
  o	How can we search the “largest possible” inner loop learning rate? Should we still use grid-search or heuristic search? In that case, what is the implication/shortcut that this proposed guideline brings? We still reach a stable training process by tuning the hyperparameters in a brute-force fashion.  
-	The baselines should be compared, to support the effectiveness of this proposed algorithm. For example, Behl et al. (2019) automatically tuning the learning rates during training definitely needs to be compared, since both aim to stabilize the training of MAML.
review_title: Official Blind Review #1

Meta-review: This paper theoretically and empirically studies the inner and outer learning rate of the MAML algorithm and their role in convergence. While the paper presents some interesting ideas and add to our theoretical understanding of meta-learning algorithms, the reviewers raised concerns about the relevance of the theory. Further the empirical study is somewhat preliminary and doesn't compare to prior works that also try to stabilize the MAML algorithm, further bringing into question its usefulness. As such, the current form of the paper doesn't meet the bar for ICLR.
Decision: Reject