Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes using a continuous-time formulation along with forward-mode differentiation to perform meta-learning. The setup follows that of ANIL: the weights of the last layer is part of the inner loop optimization while the initialization of these last-layer weights and the rest of the weights are treated as meta-parameters. The proposed method relies on this setup in order to efficiently compute derivatives, which the authors note have a low-rank structure that can be exploited. Furthermore, the weights do not need to be solved over time, and instead a much smaller vector can be solved instead, which is pretty neat. 
main_review: Pros:
  - Interesting use of forward-mode differentiation and structure of the gradients to compute meta-gradients efficiently.
  - Learning T as an early stopping is pretty interesting, and T will be increased if the train and test gradients are aligned, and decreased otherwise. (IMO, the authors can expand on interpreting this alignment a bit more after Eq 13.)

Cons:
  - IMO, the claims/explanations for continuous-time and forward-mode differentiation should be separated. SGD simply corresponds to an Euler discretization, and the same forward-mode tricks can be applied just as readily in the discrete-time setting as the continuous-time setting. Perhaps the only strictly continuous-time-related contribution is being able to learn T.
  - I don't have any major worries, but see my comments below.

Comments:
  - "Gradient-based Hyperparameter Optimization Over Long Horizon" discusses efficient forward-mode differentiation for discrete-time meta-learning. which should be worth discussing.
  - Since forward-mode can also be applied in the discrete-time setting, the main motivation for continuous-time seems to be being able to learn T. However, I'm not entirely convinced about its usefulness: 

1. The gradient wrt T is very local.
(i) How much does T change over the course of training? Is the learning useful or do you have to initialize it very well?
(ii) How stable is the optimization? For instance, if you were to initialize with a very large or very small T value, is the gradient sufficient for learning a reasonable value for T?

2. The interpretation of T:
(i) Often meta-learning methods (for instance MAML) motivate the use of a fixed number of iterations to learn good meta-parameters that are useful even with a low compute budget (small number of gradient descent iterations). However, in the continuous-time version, changing T changes the amount of computation. If you have a fixed compute cost, would learning T still be useful? This might be a fairer comparison to discrete-time methods as well.

 - Can you explain why COMLN is more expensive with the Euler integration scheme compared to RK? Are you using different step sizes for these methods?
summary_of_the_review: The approach is interesting; the main novelty is being able to train T, kind of like a learning early stopping. However, T also increases compute cost, which may not be a completely fair comparison to fixed-cost discrete-time approaches. Another minor concern is the lack of control for compute cost due to the training of T.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper describes an algorithm called COMLN for few-shot meta-learning. 
In COMLN, the base level loop is modelled as a continuous-time autonomous ODE that is the gradient vector field of the inner loss. 
The authors restrict their discussion to a meta-learning setting where the only base-level parameters are the parameters of a linear classifier. The meta-parameters are the parameters of a neural network that acts as feature extractor, the initialization point for adaptation of the classifier and the adaptation time $T$. 
The authors point out that, crucially, it is possible to treat $T$ as a learnable meta-parameter thanks to the continuous reformulation of the base level.
Meta-training is carried out by gradient descent on the meta-parameter, where the gradients are computed with an efficient implementation (in few-shot classification, with a small number of classes) forward mode differentiation that exploits the loss structure.
The authors present experiments on standard benchmark datasets, showing that COMLN achieves better performances than other comparable gradient-based methods. 
main_review: Strengths:
- The paper is clearly written, well organized and easy to follow
- I believe the method proposed is an interesting addition to the large family of meta-learning algorithms that target the (by-now-classical) few-shot learning setting.
- The possibility of seamlessly treating also the adaptation horizon as a meta-parameter opens up interesting avenues (which however are left as future work) 

Weaknesses:
1. I believe that the main weakness of the proposed approach is that it is quite specific to the setting treated in the paper, mainly few-shot classification with a small number of classes, where the loss is (unregularized) cross-entropy. In my view, the main bottleneck is runtime rather than memory, since forward-mode differentiation, not required to store the intermediate states, can always be implemented in a way that is efficient in memory. In time, however, the computation generally grows with the number of meta-parameters. The authors devise a clever decomposition to avoid most of the burden (which, however, still scales with the number of examples and classes), but fail to discuss the limitations of this approach (regarding the efficiency of the gradient computation). E.g., what if $\mathcal{L}$ is a generic function of the parameters? what if at the base level one wants to adapt also the rest of the parameters of the feature extractor or the bias of a regularizer [1]? In my view, the method remains interesting notwithstanding these potential limitations, but limitations should be clearly mentioned and discussed. Furthermore, I would like to see included in the paper also a runtime column in Table 1, to give a more complete picture of the complexity of the algorithms.
2. In my view, one of the most interesting novelties of this work is the possibility of treating the adaptation time as a meta-parameter. I believe that the paper in its current stage misses on the opportunity of exploiting this. On the one side, this is because of the lack of novelty in the experimental validation, on the other side, this could be potentially linked to the limitations of the method: longer adaptation would be extremely beneficial in the presence of tasks with a different number of examples, however, since the memory (but also the runtime I believe) scales with the number of example, the complexity of  COMLN could be too high in these settings. Could the authors comment on this?
3. Related work could be better mentioned. For instance, the idea of only learning base classifiers is also present in [2,3,4] and not only in ANIL. Furthermore, I believe that the distinction between so-called gradient-based and non-gradient based methods is misleading since also MetaOptNet uses gradients (the difference being that there they have a closed-form expression for those, but this is not at all dissimilar to iMAML).

Minor comments:
4. Clarity: Missing formal definition of the sensitivity state $\mathcal{S}(t)$. The loss $\mathcal{L}$ is used with different inputs: cf (2) and (8).

[1] Denevi, Giulia, et al. "Learning-to-learn stochastic gradient descent with biased regularization." International Conference on Machine Learning. PMLR, 2019.
[2] Franceschi, Luca, et al. "Bilevel programming for hyperparameter optimization and meta-learning." International Conference on Machine Learning. PMLR, 2018.
[3]  Bertinetto, Luca, et al. "Meta-learning with differentiable closed-form solvers." ICLR, 2019.
[4]  Lee, Kwonjoon, et al. "Meta-learning with differentiable convex optimization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. 

--------------------------------------------------
**Post rebuttal:** I thank the authors for their reply. I appreciate the derivations for the MSE and the regularized CE and the clarifications regarding the extensibility to the general case. I hope that these comments will be included in the final version. On the other hand, I am still convinced that including runtimes in Table 1 is quite important to convey a clearer picture (you could add 2+ lines for COML depending on which ODE solver is used). Regarding standard forward mode, please note that since we are discussing asymptotic behaviour the asymptotic runtime complexity of the two implementations is the same, while clearly, one implementation is more memory efficient than the other (this does not mean that the actual GPU/CPU runtime will be the same, but again, Table 2 reports $O$'s..) 
Furthermore, I still believe this work misses to capitalize on its main novelty, i.e. meta-learning task-dependent $T$'s, leaving somewhat the doubt that this is either impractical, or does not bring noticeable advantages. For these reasons, while I am overall positive about this work, I will leave my score unvaried.
summary_of_the_review: The paper is well-crafted and the method presented is an interesting addition to the few-shot meta-learning literature, with possibly some limitations that are not well discussed. Furthermore, the paper misses exploiting what probably is the main advantage of the method (meta-learning task-specific, or conditional, $T$'s). These two issues currently limit my score to a weak accept.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose a continuous-time formulation for the inner-loop adaptation in a MAML-like setup. Moreover, the authors propose an efficient forward-mode algorithm to learn the parameters of this inner-loop adaptation that is independent of the number of adaptation steps used. The authors show that this works well for a variety of meta-learning problems.
main_review: The use of continuous time formulation for the inner-loop adaptation step, while also being able to learn the length of time this adaptation should be simulated is an approach that promises to be quite powerful. Combining this with an efficient forward-mode algorithm for learning the parameters of this inner-adaptation, and meta-learning representations is overall a very significant and novel combination.

The writing is clear and the overall algorithm is well explained.
One issue I spotted: 
- in eqn. 11, A is not defined

The empirical evaluation of the approach is also thorough, as is the evaluation of computation and memory usage (which the proposed forward-mode approach is targeted towards).

The discussion of the related work seems complete to my knowledge.

The one major weakness of this approach (specifically the efficient forward mode differentiation method) seems to be that it cannot be used for non-cross-entropy losses. Are there alternative ways to achieve the same thing there?
summary_of_the_review: Overall a very strong paper with novel and significant contributions without major flaws.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose COMLN -- a gradient-based meta-learning algorithm with continuous-time inner loop adaptation. The authors discusses possible options for computing the meta-gradient (backpropagation through time, solve ODE backwards in time, and forward-mode differentiation), and identifies the forward-mode differentiation as the only method that is both numerically stable and has a memory cost that is constant w.r.t. the inner-loop length. The authors then derive a way to compute the vector-Jacobian products without explicitly constructing the Jacobians in forward-mode differentiation (specifically for the few-shot learning setting), which further reduces the memory cost. Also, by using a continuous formulation for the inner-optimization, the authors are able to compute meta-gradient w.r.t. horizon T and adapt that as well. Finally, the authors conduct experiments on standard few-shot image classification benchmarks, and show that COMLN outperforms other gradient-based meta-optimization methods in accuracy with a lot less memory cost.
main_review: Strengths:
* The forward-mode differentiation is well-justified. Good discussion on the alternatives and why they are not feasible for the gradient-based meta-learning case.
* The derivation for the vector-Jacobian product in forward-mode differentiation without explicitly constructing the Jacobian is novel and can greatly reduce memory cost in the few-shot setting.
* The adaptation w.r.t. T is interesting, and only possible in the continuous-time formulation. However, I have a few questions about this (see “Questions” section below).

Weaknesses:
* The memory cost scales with the number of data points available. This is not desirable as we would usually expect the performance to improve with more data points, instead of having to limit the amount of data due to constraints on computation resources.


Questions:
* Adapting both T and W0 sounds like two conflicting things to do. When T is adapted to be very large, won’t it drive the gradient w.r.t. W0 to zero (i.e. inner loop is allowed to converge, making W_T independent of initialization)? Does that match your observations in the experiments, and what are the justifications for adapting both? I imagine this conflicting adaptation would complicate the learning dynamics, and may be affected by how you choose the meta optimizers (e.g. if you use a different meta learning rate for T and W0, it might lead to different behaviours).
* What makes COMLN achieve higher accuracy than other gradient-based meta-optimizers? Is it because of the adaptive T? I’d like to see some ablation experiments about this.
* In the related work section, you wrote “Zhang et al. (2021) also introduce a formulation where the adaptation of prototypes follows a gradient vector field, but they finally opt for modeling it as a Neural ODE, possibly due to the challenges of applying the adjoint method we identified in Section 3.2”. From my understanding, it’s precisely due to the numerical instability of the adjoint method that you **don’t use** Neural ODE, and opt for forward-mode differentiation instead. Could you clarify that in more detail?

Feedback for improvements:
* Should further discuss why you choose to focus on gradient-based meta optimizers specifically, what are the pros and cons compared to non-gradient-based ones (like MetaOptNet).

A couple places where \citet should be used instead of \citep:
 * Page 7, third line under “experiments” section: Lee et al., 2019.
 * Page 9, third line of the paragraph before Section 7: Chen et al., 2020a.

summary_of_the_review: I recommend weak accept. I think the novel derivation of memory-efficient forward-mode differentiation is a significant contribution in the few-shot learning context. I would consider raising the score if my questions about adapting T are addressed.

=== Post rebuttal ===

The authors have addressed all my concerns. I increased the score to 8 (accept).
review_title: null

Meta-review: This paper addresses a continuous-time formulation of gradient-based meta-learning (COMLN) where the adaptation is the solution of a differential equations. In general, outer loop optimization requires backpropagating over trajectories involving gradient updates in the inner loop optimization. It is claimed that one of main advantages of COMLN is able to compute the exact meta-gradients in a memory-efficient way, regardless of the length of adaptation trajectory. To this end, the forward-mode differentiation is used, with exploiting the Jacobian matrix decomposition. All the reviewers agree that the derivation of memory-efficient forward-mode differentiation is a significant contribution in the few-shot learning. The paper is well written and has interesting contributions. Authors did a good job in responding to reviewers’ comments during the discussion period. What is missing in this paper is the discussion of some limitations of the proposed method.  This can be improved in the final version. All reviewers agree to champion this paper. Congratulations on a nice work.
Decision: Accept (Spotlight)