Continuous-Time Meta-Learning with Forward Mode Differentiation
Authors: Tristan Deleu
David Kanaa
Leo Feng
Giancarlo Kerg
Yoshua Bengio
Guillaume Lajoie
Pierre-Luc Bacon
Venue: ICLR 2022 Spotlight (2022)
Keywords: meta-learning
few-shot learning
dynamical systems

Abstract: Drawing inspiration from gradient-based meta-learning methods with infinitely small gradient steps, we introduce Continuous-Time Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of a gradient vector field. Specifically, representations of the inputs are meta-learned such that a task-specific linear classifier is obtained as a solution of an ordinary differential equation (ODE). Treating the learning process as an ODE offers the notable advantage that the length of the trajectory is now continuous, as opposed to a fixed and discrete number of gradient steps. As a consequence, we can optimize the amount of adaptation necessary to solve a new task using stochastic gradient descent, in addition to learning the initial conditions as is standard practice in gradient-based meta-learning. Importantly, in order to compute the exact meta-gradients required for the outer-loop updates, we  devise an efficient algorithm based on forward mode differentiation, whose memory requirements do not scale with the length of the learning trajectory, thus allowing longer adaptation in constant memory. We provide analytical guarantees for the stability of COMLN, we show empirically its efficiency in terms of runtime and memory usage, and we illustrate its effectiveness on a range of few-shot image classification problems.

Published as a conference paper at ICLR 2022
CONTINUOUS-TIN META- WITH FORWARD MODE DIFFERENTIATION
Tristan Deleu* David Kanaa
Leo Feng Guillaume Lajoie
Giancarlo Kerg Pierre-Lue Bacon
Yoshua Bengio 1.2
Mila Université de Montréal
ABSTRACT
Drawing inspiration from gradient-based meta-learning methods with infinitely small gradient steps, we introduce Continuous-Tinn Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of gradient vector field. Specifically, representations of the inputs are meta- leannee such that task- specific linear classifier is obtained as solution of an ordinary differential equation (ODE). Treating the learning process as an ODE offers the notable advantage that continuous, opposed
the length of the trajectory number of gradient steps. As a consequence, we can optimize the amount of adaptation necessary to solve new task using stochastic gradient descent, in addition to learning the initial conditions as is standard practice in gradient- traseed meta- meta-learning. Importantly, in order to compute the exact meta- veta-gradients required for the outer-loop updates, we devise an efficient algorithm based on forward mode differentiation, whose memory requirements the learning trajectory, thus allowing longer adaptation in constant memory. We provide analytical guarantees for the stability of COMLN, we show empirically its efficiency in terms of runtime and memory usage, and illustrate its effectiveness on a range of few-shot image classification problems.
fixed and discrete
not scale with the length of
INTRODUCTION
Among the existing meta-learning algorithms, gradient-based methods popularized by Model- Agnostic Meta-Learning (MAMI, Finn et al., 2017) have received a lot of attention over the past few years. They formulate the problem of learning based on few steps of gradient descent. An outer meta-optimization problem is then responsible for updating the meta-parameters of this learning process, such as the initialization of the gradient descent procedure. However since the updates at the outer level typically require backpropagating through the learning process, this class methods has often been limited to only a few gradient steps of adaptation, due to memory constraints. Although solutions have been proposed to alleviate the memory requirements of these algorithms, including checkpointing (Baranchuk, 2019), using implicit differentiation (Rajeswaran et al., 2019), or reformulating the meta-learning objective (Flennerhag et al., 2018), they are generally either more computationally demanding, or only approximate the gradients of the meta-learning objective (Nichol et al., 2018: Flennerhag al., 2020).
task inner optimization problem, typically
In this work, we propose a continuous-time formulation of gradient-based meta-learning, called Continuous-Tine Meta-Learning (COMLN), where the adaptation is the solution of differential equation (see Figure 1). Moving to continuous time allows us to devise a novel algorithm, based on forward mode differentiation, efficiently compute the exact gradients for meta-optimization, no matter how long the adaptation to a new task might be. We show that using forward mode differentiation leads stable algorithm, unlike the counterpart of backpropagation in continuous time called the adjoint method (frequently used in the Neural ODE literature; Chen et al., 2018) which tends be unstable in conjunction with gradient vector fields. Moreover as the length of
Correspondence to: Tristan Deleu <deleutri @ mila quila. CIIARR Senior Fellow, *CIFAR AI Chair Code is available https:/ / whw.b coom/rrri stande Lenddeleu//en jaa/jaxxxomm

---

Published as a conference paper at ICLR 2022
(a) Gradient-basse Meta-Learning
(b) COMLN
(
WT
Wo
Wo
dW VC(W(()
W W, - aVLWW)
Figure 1: Illustration of the adaptation process in (a) a gradient-based meta-learning algorithm, such as ANIL (Raghu et al., 2019), where the adapted parameters Wr are given after ''' steps of gradient descent, and in (b) Continuous- Time Meta-Learning (COMLN), where the adapted parameters (TT) are the result of following the dynamics of gradient vector field up to time T.
the adaptation trajectory is continuous quantity, as opposed a discrete number of gradient steps fixed ahead of time, we can treat the amount of adaptation in COMLN as meta-parameter-oon par with the initialization
we can meta-optimize using stochastic gradient descent. We verify empirically that our method is both computationally and memory efficient, and we show that COMLN outperforms other standard meta-learning algorithms on few-shot image classification datasets.
BACKGROUND
In this work, consider the problem of few-shot classification, that is the problem of learning classification model with only small number of training examples. More precisely for classification task we assume that we have access to a (small) training dataset Diraiu {m ymm) m== fit model on task T, and distinct test dataset Diess to evaluate how well this adapted model generalizes on that task. In the few-shot learning literature, it is standard to consider the problem of i-sshot N-way classification, meaning that the model has to classify among N possible classes, and there are only h examples each class Dirain so that overall the number training examples is M N.N We use the convention that the target labels ym. E (0, J are one-hot vectors.
2.1 GRADIENT- BASED META-LEARNING
Gradient- based meta-learning methods aim to learn an initialization such that the model is able to adapt to a new task via gradient descent. Such methods are often cast as a bi-level optimization process: adapting the task-specific parameters 0 in the inner loop, and training the (lask-agnostic) meta-parameters & and initialization Oy the outer loop. The meta-learning objective is:
min E, E, [C(T, Φb Drest)
6,.4 s.t. 0-+1 07- -ooL &; Derain)
0 = Oo
Vr P(T),
(2)
where T is the number of inner loop updates. For example, in the case of MAMI. (Finn et al., 2017), there is additional meta-pant pert-parameter other than the initialization (4 0); in ANIL (Raghu et al., 2019), 0 are the parameters of the last layer, and d are the parameters of the shared embedding network; in CAVIA (Zintgraf et al., 2019), 0 are referred as context parameters.
During meta-training, the model is trained over many tasks 7. The task-specific parameters are learned via gradient descent Dirain The meta- peta-parrmeters are then updated by evaluating the error of the trained model on the test dataset Diest At meta- testing time, the meta eett--raiind model is adapted on pptrii__.e applying (2) with the learned meta-parameters Og and T.
2

---

Published as a conference paper at ICLR 2022
2.2 LOCAL SENSITIVITY ANALYSIS OF ORDINARY DIFFERENTIAL EQUATIONS
Consider the following (autonomous) Ordinary Differential Equation (ODE):
dz dt = g (t):
z() = 2 (0),
(3)
where the dynamics g and initial value 2o may depend on some external parameters 6, and integration carried out from 0 to some time T. Local sensitivity analysis is the study of how the solution of this dynamical system responds to local changes in 0; this effectively corresponds to calculating the derivative dz (f)/d0. We present here methods to compute this derivative, with a special focus on their memory efficiency.
Adjoint sensitivity method Based the adjoint state (Pontryagin, 2018), and taking its root in control theory (Lions & Magenes, 2012), the adjoint sensitivity method (Bryson & Ho, 1969; Chavent al., 1974) provides efficient approach for evaluating derivatives of (=)(T); 0), function of
=(T) the solution of the ODE in (3). This method, popularized lately by the literature on Neural ODEs (Chen et al., 2018), requires the integration of the adjoint equation
da di = (tt) ((();
dL (z(=
a(T)
(4)
3z(1)
dz(T)
backward in time. The adjoint sensitivity method can be viewed continuous time counterpart to backpropagation, where the forward pass would correspond to integrating (3) forward in time from to T, and the backward pass to integrating (4) backward in time from T to O.
One possible implementation, reminiscent backpropagation through time (BPTT), is to store the intermediate values of z z(4) during the forward pass, and reuse them to compute the adjoint state during the backward pass. While several sophisticated checkpointing schemes have been proposed (Serban & Hindmarsh, 2003; Gholami et al., 2019), with different compute/memory trade-offs, the memory requirements of this approach typically grow with T; this is similar to the memory limitations standard gradient-basee -ased meta-learning methods suffer from the number of gradient steps increases. An alternative is to augment the adjoint state a(t) with the original state z(t), and to solve this augmented dynamical system backward in time (Chen al., 2018). This has the notable advantage that the memory requirements are now independent of T, since (() are no longer stored during the forward pass. but they are recomputed on the fly during the backward pass.
Forward sensitivity method While the adjoint method related to reverse-mode automatic differ- entiation (backpropagation), the forward sensitivity method (Feehery et al., 1997; Leis & Kramer, 1988; Maly & Petzold, 1996; Caracotsios & Stewart, 1985), on the other hand, can be viewed as the continuous-timg counterpart to forward (tangent- linear) mode differentiation (Griewank & Walther, 2008). This method based on the fact that the derivative S(t) & dz(t)/ the solution of the so-called forward sensitivity equation
ds 0g(z(((: 0)
dt = dz(() S(t) + (g(=(((;0) 0)
azo 80
S(0)
(5)
00
This equation can be found throughout the literature in optimal control and system identification (Betts, 2010; Biegler, 2010). Unlike the adjoint method, which requires an explicit forward and backward pass, the forward sensitivity method only requires the integration forward in time of the original ODE in (3), augmented by the sensitivity state S(t) with the dynamics above. The memory requirements of the forward sensitivity method do not scale with T either, but it now requires storing S(t), which may be very large; we will come back to this problem in Section 3.2. We will simply note here that in discrete-time this is the same issue afflicting forward-mode training of RNNs with real-tima recurrent learning (RTRL; Williams Zipser, 1989), or other meta- meta-learning algorithms (Sutton, 1992; Franceschi al., 2017; Xu et al., 2018).
3 CONTINUOUS ADAPTATION
In the limit of infinitely small steps, some optimization algorithms can be viewed as the solution trajectory of differential equation. This point of view has often been taken analyze their behavior
3

---

Published as a conference paper at ICLR 2022
Figure 2: Numerical instability of the adjoint method applied to the gradient vector field a quadratic loss function. The trajectory in green starting W (0) corresponds to the inte- gration the dynamical system in (8) forward in time up to T, and the trajectory red starting at W (Z') corresponds to its integration backward in time. Note that ''' was chosen so that (T) does reach the equlibriium/minimum of the loss W
(0))
((T)
W
Forward
Backward
(Platt & Barr, 1988; Wilson et al., 2016; Su et al., 2014; Orvicto & Lucchi, 2019). fact, some optimization algorithms such gradient descent with momentum have even been introduced initially from the perspective of dynamical systems (Polyak, 1964). As the simplest example, gradient descent with constant step size + 0+ (i.e. a tends to 0 by positive values) corresponds to following the dynamics of an autonomous ODE called a gradient vector field
dz
2++1 21 az (z()
-
(6)
0
dt
where the iterate =() is now continuous function of time L. The solution of this dynamical system is uniquely defined by the choice of the initial condition =(0) = 20-
3.1 CONTINUOUS CEMTNU META-LEARE LEARNING
In gradient-based meta-learning, the task-specific adaptation with gradient descent may also be replaced by gradient vector field in the limit of infinitely small steps. Inspired by prior work in meta-learning (Raghu al., 2019; Javed & White, 2019), we assume that an embedding network fo with paraataarseeerss + is shared across tasks, and only the parameters W of task-specific linear classifier are adapted, starting some initialization Wo. Instead of being the result of few steps of gradient descent though, the final parameters (T) now correspond to integrating an ODE similar to (6) up to a certain horizon T, with the initial conditions W = Wo. We call this new meta-learning algorithm Continuous-Time Meta-Learning" (COMLN).
Treating the learning algorithm as a continuous-tine process has the notable advantage that the adapted parameter (I') is now differentiable wrt. the time horizon T (Wiggins, 2003, Chap. 7), in addition to being differentiable wrt. the initial conditions Wo-which plays a central role gradient- based meta-learning, par with & and Wo. and to effectively optimize the amount of adaptation using stochastic gradient descent (SGD). The meta-learning objective of COMLN can be written
described in Section 1. This allows to view T meta-parameter
min E, [=W, (D(tstt W.Wo
dW, = = -V) (D.rra))) s.t.
W, = Wo
VT ~ p(T),
(8)
dt
where fa fa (Dtrain) ((f(( Dm (mm: ym) Dirain is the embedded training dataset, and fo (Dtrst) is defined similarly for Dirstt In practice, adaptation is implemented using a numerical integration scheme based on iterative discretization of the problem, such as Runge-Kutta methods. Although a complete discussion of numerical solvers outside of the scope of this paper. we recommend (Butcher, 2008) for comprehensive overview of numerical methods for solving ODEs.
3.2 THE CHALLENGES OF OPTIMIZING THE META-LEARNING OBJECTIVE
In order to minimize the meta-learning objective of COMLN, it is common practice to use (stochastic) gradient methods; that requires computing its derivatives wrt. the meta-parameters, which we call mela-grudients Our primary goal is to devise algorithm whose memory requirements do not scale with the amount of adaptation T;; this would contrast with standard gradient- based meta-learning methods that backpropagate through a sequence of gradient steps (similar to BPTT), where the
COMLN is pronounced chameleon.
4

---

Published as a conference paper at ICLR 2022
intermediate parameters are stored during adaptation (i.e. O for all t in (2)). Since this objective involves the solution W(T) of an ODE, we can use either the adjoint method, or the forward sensitivity method, in order to compute the derivatives wrt. &b and Wi (see Section 2.2).
Although the adjoint method has proven to be an effective strategy for learning Neural ODEs, in practice computing the state (t)) backward in time is numerically unstable when applied gradient vector field like the one (8), even for convex loss functions. Figure 2 shows an example where the trajectory of (!) recomputed backward in time (in red) diverges significantly from the original trajectory (in green) on quadratic loss function, even though the two should match exactly in theory since they follow the same dynamics. Intuitively, recomputing (()) backward in time for a gradient vector field requires doing gradient ascent on the loss function, which is prone to compounding numerical errors; this is closely related to the loss of entropy observed by Maclaurin et al. (2015). This divergence makes the backward trajectory of (t) unreliable to find the adjoint state, ruling out the adjoint sensitivity method for computing the meta-gradients in COMLN.
The forward sensitivity method addresses this shortcoming by avoiding the backward pass altogether. However, it can also be particularly expensive here in terms of memory requirements, since the sensitivity state S(I) in Section 2.2 now corresponds to Jacobian matrices, such as dW (L)/dW As the size d of the feature vectors returned by fa may be very large, this Nd x Nd Jacobian matrix would be almost impossible to store in practice: for example in experiments, it can be large as = 16, 600 for a ResNet-12 12 backbone. In Section 4.1, we will show how to apply forward sensitivity in memory-efficient way, by leveraging the structure of the loss function. This is achieved by carefully decomposing the Jacobian matrices into smaller pieces that follow specific dynamics. We show in Appendix D that unlike the adjoint method, this process is stable.
3.3 CONNECTION WITH ALMOST No INNER-LOOP (ANIL)
Similarly to ANIL (Raghu et al., 2019), COMLN only adapts the parameters W of the last linear layer of the neural network. There a deeper connection between both algorithms though: while our description of the adaptation in COMLN (Eq. 8) was independent of the choice of the ODE solver used to find the solution W(T) in practice, if we choose an explicit Euler scheme (Euler, 1913; roughly speaking, discretizing (6) from right to left), then the adaptation of COMLN becomes functionally equivalent to ANIL. However, this equivalence can greatly benefit from the memory- efficient algorithm compute the meta-gradients described in Section 4, based on the forward sensitivity method. This means that using the methods devised here for COMLN, we can effectively compute the meta- greta-gg of ANIL with a constant memory cost wrt. the number of gradient steps of adaptation, instead of relying on backpropagation (see also Section 4.2).
4 MEMORY-EFFICIEN META-GRADIENTS
For some fixed task r and (Im, ym) E Dirran, let Φm = f+ E Rd be the embedding of input Fm through the feature extractor fa. Since we are confronted with a classification problem, the loss function of choice L(W) is typically the cross-entropy loss. Böhning (1992) showed that the gradient of the cross-entropy loss wrt. W can be written as
VL(W: (Dirain)
1 M
Σpm (pm ym)
where Pm softmax( W @m) is the vector of probabilities returned by the neural network. The key observation here is that the gradient can be decomposed as a sum of M rank-one matrices, where the feature vectors &m are independent of W. Therefore we can fully characterize the gradient of the cross-entropy loss with M vectors Pm ym E IR as opposed to the full N x d matrix. This is particularly useful in the context of few-shot classification, where the number of training examples M is small, and typically significantly smaller than the embedding size d.
4.1 DECOMPOSITION OF THE
We saw in Section 3.2 that the forward sensitivity method was the only stable option to compute the meta-gradients of COMLN. However, naively applying the forward sensitivity equation would involve
5

---

Published as a conference paper at ICLR 2022
quantities that typically scale with d?, which can be too expensive in practice. Using the structure of (9). the Jacobian matrices appearing in the computation of the meta veta-gradients for COMLN can be decomposed in such a way that only small quantities will depend time.
Meta-gradients wrt. Wo By the chain rule of derivatives, it is sufficient to compute the Jacobian matrix (W(T)/dW in order to obtain the meta- detta-gradient wrt. Wo. We show in Appendix B.2 that the sensitivity state (t)/ /dWo can be decomposed as:
(W = dW
M
M
Σ
(10)
where @ is the Kronecker product, and each Be is an N x N matrix, solution of the following system of ODES?
dB. 1(i () (. Σ ( Bi di
Boli.j] 0, (11)
and A,(t), defined in Appendix B.2, is also an N x N matrix that only depends (t) and Φ;. The main consequence of this decomposition is that we can simply integrate the augmented ODE in (t), [i up to T to obtain the desired Jacobian matrix, along with the adapted parameters WT)). Furthermore, in contrast to naively applying the forward sensitivity method (see Section 3.2), the 2 matrices B, [i are significantly smaller than the full Jacobian matrix. In fact, we show in Appendix C that we can compute vector-Jacobian requiree for the chain uul-ussing only these smaller matrices, and without ever having to explicitly construct the full Nd x Nd Jacobian matrix (W with (10).
Meta- genta-gradietts wrt. & To backpropagate the error through the embedding network f., we need to first compute the gradients of the outer-loss wrt. the feature vectors Φ.n. Again, by the chain rule, we can get these gradients with the Jacobian matrices dW (T)T)/ll (T)/dmm. Similar to (10), we can show that these Jacobian matrices be decomposed as:
dw [ dΦm
M
ml oJ & a
8m (1) 8 Bo +
2
li.
(12)
where 8m (1) and z 2i(i,,j, m] are vectors of size N, that follow some dynamies; the exact form of this system of ODEs, as well as the proof of this decomposition, are given in Appendix B.3. Crucially, the only quantities that depend on time are small objects independent of the embedding size d. Following the same strategy as above, we can incorporate these vectors in the augmented ODE, and integrate it to get the necessary Jacobians. Once all the (t) d are known, for all the training datapoints, we can apply standard backpropagation through fo to obtain the meta-gradient wrt. Tb
Meta-gradient wrt. T One of the major novelties of COMLN is the capacity to meta-learn the amount of adaptation using stochastic gradient descent. To compute the meta- geeti-gradeent wrt. the time horizon T, we can directly borrow the results derived by Chen al. (2018) in the context Neural ODEs, and apply it to our gradient vector field in (8) responsible for adaptation:
dL W (T); foo dT
Tac(W (T); (T)(W(T); fo fo (D(et)) (W(T)
T "ac(w (T); fo (D.rain) @W(T)
(13)
The proof is available in Appendix B.4. Interestingly, we find that this involves the alignment between the vectors of partial derivatives of the inner-loss and the outer-loss at W (T), which appeared in various contexts in the meta- and the multi-task learning literature (Li et al., 2018; Rothfuss et al., 2019; Yu et al., 2020; Von Oswald et al., 2021).
2 Here used the notation B. make the dependence t explicit, without overloading the notation. more precise notation would be B (t).

---

Published as a conference paper at ICLR 2022
Table I: Memory required to compute meta-gradients for different algorithms. Exact: the method returns the exact meta- geeta-gradeetss. Full net.: the whole network is adapted, with number of meta- parameters |@). The requirements for checkpointing are taken from (Shaban al., 2019). Note that
typically M < d in few-shot learning.
Model
Exact
Full net.
Memory
MAML (Finn et al., 2017)
0(10) - T) T)
ANIL (Raghu et al., 2019)
Checkpointing (every vT steps)
0[10) VT)
iMAMI (Rajeswaran al., 2019)
V
0(10))
Forward sensitivity (naive)
X
ON MNdP) O(MPN2+MM)
COMLN
X
4.2 MEMORY EFFICIENCY
Although naively applying the forward sensitivity method would be memory intensive, we have shown Section 4.1 that the Jacobians can be carefully decomposed into smaller pieces. turns out that even the parameters W (t) can be expressed using the vectors 8m (t) from the decomposition in (12); see Appendix B.I for details. As a consequence, to compute the adapted parameters W well as all the necessary meta-gradients, it is sufficient to integrate dynamical system in [i, j]. s..(.))) (t), ze zeli. j.]} (see Algorithms 1 & 2 in App. A.1), involving exclusively quantities that are independent of the embedding size d. Instead, the size of that system scales with M the total number of training examples, which is typically much smaller than d for few-shot classification.
Table shows a comparison of the memory cost for different algorithms. It important note that contrary to other standard gradient-based meta-learning methods, the memory requirements of COMLN do not scale with the amount of adaptation (i.e. the number gradient steps MAML iMAML (Rajeswaran et al., 2019), which only returns an approximation of the meta veea-ggaddiens. We verified empirically this efficiency, both in terms of memory and computation costs, in Section 5.2.
& ANIL), while still returning the exact meta gesta-gradiens--un
EXPERIMENTS
For our embedding network fo, we consider two commonly used architectures in meta-learning: Conv-4, a convolutional neural network with 4 convolutional blocks, and ResNet-12, 12, a 12-layer residual network (He et al., 2016). Note that following Lee et al. (2019), ResNet- 12 does not include a global pooling layer at the end of the network, leading to feature vectors with embedding dimension d = 16,000. Additional details about these architectures are given in Appendix E. To compute the adapted parameters and the meta- geadiggaaseett in COMLN, we integrate the dynamical system described in Section 4.2 with a 4th order Sounge-Kutte method with a Dormand Prince adaptive step size (Runge, 1895; Dormand & Prince, 1980); we will come back to the choice of this numerical solver in Section 5.2. Furthermore to ensure that T >> parametrized it with exponential activation.
5.1 FEW-SHOT IMAGE CLASSIFICATION
We evaluate COMLN on two standard few-shot image classification benchmarks: the minilmageNet (Vinyals et al., 2016) and the rieredlmageNet datasets (Ren et al., 2018), both datasets being derived from ILSVRR-2012 (Russakovsky et al., 2015). The process for creating tasks follows the standard procedure from the few-shot classification literature (Santoro al., 2016), with distinet classes between the different splits. minilmagenet consists of 100 classes, split into 64 training classes, 16 validation classes, and 20 test classes. tieredImageNet consists of 608 classes grouped into 34 high-level categories from ILSVRC-2012, split into 20 training. 6 validation, and 8 testing categories-corresponeee corresponding to 351/97/160 classes respectively; Ren et al. (2018) argue that separating data according high-level categories results in more difficult and more realistic regime.
Table 2 shows the average accuracies of COMLN compared to various meta-learing methods, be tradient-based not. For both backbones, COMLN decisively outperforms all other gradient-based
7

---

Published as a conference paper at ICLR 2022
Table Few-shot classification on minilmageNet & migred The average accuracy (%) on 1,000 held-out meta-test tasks is reported with 95% confidence interval. / denotes mudieent-based meta-learning algorithms. * denotes baseline results we executed using the official implementations.
minilmageNet 5-way
fieredlmageNeet way
Model
Backbone
1-shot
5-shot
1-shot
5-shot
MAML (Finn et al., 2017)
Conv-4
48.70 + 1.84
63.11 + 0.92
51.67 + 1.81
70.30 = 1.75
ANIL (Raghu et al., 2019)
Conv-4
46.30 1 0.40
61.00 1 0.60
49.35 1 0.26
65.82 = 0.12
Meta-SGD (Li et al., 2017)
V
Conv-4
50.47 + 1.87
64.03 0.94
52.80 1 0.44
62.35 0.26
CAVIA (Zintgraf et al., 2019)
Conv-4
51.82 + 0.65
65.85 0.55
52.41 1 2.64
67.55 2.05"
iMAML (Rajeswaran et al., 2019)
Conv-4
49.30 + 1.88
59.77 0.73
38.54 = 1.37*
60.24 = 0.76
MetaOptNet- RR (Lee et al., 2019)
Conv-4
53.23 = 0.59
69.51 + 0.48
54.63 = 0.67
72.11 + 0.59
MetaOptNet-SVM (Lee al., 2019)
Conv-4
52.87 = 0.57
68.76 + 0.48
54.71 + 0.67
71.79 0.59
COMLN (Ours)
Conv-4
53.01 + 0.62
70.54 = 0.54
54.30 = 0.69
71.35 0.57
MAML (Finn et al., 2017)
ResNet-12
49.92 + 0.65
63.93 + 0.59
55.37 + 0.74
72.93 0.60
ANII (Raghu et al., 2019)
ResNet-12
49.65 + 0.65
59.51 + 0.56
54.77 + 0.76
69.28 0.67
MetaOptNet-RRR (Lee al., 2019)
ResNet-12
61.41 + 0.61
77.88 + 0.46
65.36 0.71
81.34 0.52
MetaOptNet-SVA. (Lee al., 2019)
ResNet-12
62.64 + 0.61
78.63 0.46
65.99 1 0.72
81.56 0.53
COMLN (Ours)
ResNet-12
59.26 + 0.65
77.26 0.49
62.93
81.13 0.53
meta-learning methods. Compared to methods that explicitly backpropagate through the learning process, such as MAML or ANIL., the performance gain shown by COMLN could be credited the longer adaptation 7 it learns, as opposed to small number of gradient steps uspslusually about 10 steps; this was fully enabled by our scale with the length of adaptation anymore (see Section 4.2). We analyse the evolution of T during meta- teata-training for these different settings in Appendix E.3. In almost all settings, COMLN is even closing the gap with strong non-gradient-based method like MetaOptNet; the remainder may be explained in part by the training choices made by Lee et al. (2019) (see Appendix E for details).
method to compute meta-gradients, which does not
5.2 EMPIRICAL EFFICIENCY OF COMLN
In Section 4.2, we showed that our algorithm to compute the meta-gradients, based on forward differentiation, had a memory cost independent the length of adaptation T. We verify this empirically in Figure 3, where compare the memory required by COMLN and other methods to compute the meta- geetiggradeenss on a single task, with a Conv-4 backbone (Figure 4 in Appendix E.2 shows similar results for 122.. To ensure aligned comparison between discrete and continuous time, we use a conversion corresponding to a learning rate a = 0.01 in (2); see Appendix E.2 for a justification. As expected, the memory cost increases for both MAML and ANIL the number of gradient steps increases, while it remains constant for iMAML and COMLN. Interestingly, we observe that the cost of COMLN is equivalent the cost of running ANIL for small number of steps, showing that the additional cost of integrating the augmented ODE in Section 4.2 to compute the meta-gratee seta-graddeens is minimal.
Increasing the length of adaptation also has an impact on the time it takes to compute the adapted parameters, and the meta veta-graddents. Figure (right) shows how the runtime increases with the amount of adaptation for different algorithms. We see that the efficiency of COMLN depends on the numerical solver used. When we use a simple explicit-Euler scheme, the time taken to compute the meta-gradients matches the one of ANIL; this behavior empirically confirms our observation in Section 3.3. When we use an adaptive numerical solver, such as Runge- Kunge-Kutta Kutta (RK) with a Dormand Prince step size, this computation can be significantly accelerated, thanks to the smaller number of function evaluations. In practice, we show in Appendix E. that the choice of the ODE solver has a very minimal impact the accuracy.
8

---

Published as a conference paper at ICLR 2022
T
10 10-210 210-1 10P
10t
10a
10a
104
109 10e
T
10
10
10P
10A
10a
10a
104
10c
106
MAMI.
MAML
Memory usage (in Gb)
1MAML
10a
ANIL
ANIL COMEN
Runtime (in ms)
COMLN (Euler)
101
COMEN (RK)
102
10"
10
10"
10a
102
10a
104
10F
106
107
10
109
10a
102
102
10
10a
109
107
109
Number of gradient steps
Number gradient steps
Figure 3: Empirical efficiency of COMLN on single 5-shot 5-way task, with Conv-4 backbone. (Left) Memory usage for computing the meta-gradients as function of the number of inner-gradies steps. The extrapolated dashed lines correspond the method reaching the memory capacity of Tesla V100 GPU with 32Gb of memory. (Right) Average time taken (in ms) to compute the exact meta-gradients. The extrapolated dashed lines correspond to the method taking over 2 seconds.
6 RELATED WORK
We are interested in meta-learning (Bengio et al., 1991; Schmidhuber, 1987; Thrun & Pratt, 2012), and in particular we focus on gradient-basee meta-learning methods (Finn, 2018), where the learning rule is based gradient descent. While in MAML (Finn et al., 2017) the whole network was updated during this process, follow-up works have shown that generally sufficient to share most parts of the neural network, and only adapt few layers (Raghu et al., 2019; Chen et al., 2020b; Tian et al., 2020). Even though this hypothesis has been challenged recently (Arnold & Sha, 2021), COMLN continuous- time
also updates only the last layer of a neural network, and therefore can be viewed extension of ANIL (Raghu et al., 201 2); see also Section 3.3. With its shared embedding network across tasks, COMLN is also connected to metric- meta-learning methods (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018; Bertinetto et al., 2018; Lee et al., 2019).
Zhang et al. (2021) introduced formulation where the adaptation of prototypes follows gradient vector field, but finally opted for modeling it as a Neural ODE (Chen et al., 2018). Concurrent to our work. Li al. (2021) also propose formulation adaptation based on gradient vector field, and use the adjoint method to compute the meta- veeta-gradients despite the challenges we identified in Sec. 3.2; Li al. (2021) also acknowledge these challenges, and they limit their analysis to relatively small values of T (in comparison to the ones learned by COMLN), hence further from the equilibrium, to cireumvent this issue altogether. Zhou et (2021) also uses gradient vector field to motivate a novel method with a closed-form adaptation; COMLN still explicitly updates the parameters following the gradient vector field, since there no closed- form solution of (8). As mentioned Section 3, treating optimization continuous caatinuous-time process has been used to analyze the convergence of different optimization algorithms, including the meta-optimization of MAML (Xu et al., 2021), or to introduce new meta-optimizers based on different integration schemes (Im al., 2019). Guo et al. (2021) also uses meta-learning to learn new integration schemes for ODEs. Although this a growing literature the intersection of meta-learning and dynamical systems, our work is the first algorithm that uses gradient vector field for adaptation in meta-lcarning (see also Li et al. (2021)).
Beyond the memory efficiency of our method, one of the main benefits of the continuous-time perspective is that COMLN is capable of learning when to stop the adaptation, as opposed to taking a number of gradient steps fixed ahead of time. However unlike Chen al. (2020a). where the number of gradient steps are optimized (up to a maximal number) with variational methods, we incorporate the amount of adaptation as a (continuous) meta- parameer that can be learned using SGD. To compute the meta- tittt--rrdentss, which is known to be challenging for long sequences in gradient- based meta-learning, we use noward--moe differentiation as an alternative to backpropagation through the learning process, similar to prior work in meta-learning (Franceschi et al., 2017; Jiwoong Im et al., 2021) and hyperparameter optimization over long horizons (Micaelli & Storkey, 2021). This yields the exact meta- veeta-gradients in constant memory, without any assumption the optimality of the imner optimization problem, which necessary when using the normal equations (Bertinetto et al., 2018), or to apply implicit differentiation (Rajeswaran et al., 2019).

---

Published as a conference paper at ICLR 2022
7 CONCLUSION AND FUTURE WORK
In this paper, we have introduced Continuous- Meta-Learning (COMLN), a novel algorithm that treats the adaptation in meta-learning as continuous-time process, by following the dynamics of a gradient vector field up to a certain time horizon T. One of the major novelties of treating adaptation in continuous time is that the amount of adaptation T is now a continuous quantity, that can be viewed as a meta parameeer and can be learned using SGD, alongside the initial conditions and the parameters of the embedding network. In order to learn these meta-parameters,, we have also introduced a novel practical algorithm based on forward mode automatic differentiation, capable of efficiently computing the exact meta-gradiens using an augmented dynamical system. We have verified empirically that this algorithm was able to compute the mneta-gradients in constant memory, making it the first gradient-bad -baed meta-learning approach capable of computing the exact meta- gradients with long sequences of adaptation using gradient methods. In practice, have shown that COMLN significantly outperforms other standard gradient-basee meta-learning algorithms.
In addition to having single meta- pattaeerrteterr T that drives the adaptation of all possible tasks, the fact that the time horizon can be learned with SGD opens up new possibilities for gradient-hased methods. For example, we could imagine treating 7 not task-specific parameter. This would allow the learning process to be more adaptive, possibly with different behaviors depending on the difficulty of the task. This is left as future direction of research.
shared meta-parameters, but
ACKNOWLEDGEMENTS
The authors are grateful Samsung Electronics Co., Ldt., CIFAR, and IVADO for their funding and Calcul Québec and Compute Canada for providing us with the computing resources.
10

---

Published as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We provide in Appendix A.1 a full description in pseudo-code the meta-training procedure (Algorithm 1), along with the exact dynamics of the ODE (Algorithm 2) and the projection operations (Algorithms 3 & 4) to avoid explicitly building the Jacobian matrices to compute Jacobian-vector products (see Section 4.1).
We also provide in Appendix A.2 a snippet of code in JAX (Bradbury et al., 2018) to compute the
adapted parameters (T), as well as all the necessary objects { [2, j]. sm (t), 2 j.... j. ml] to com- pute all the meta- reta-ggrdieets (see Section 4.2). We also give in Code Snippet 2 the code to compute the meta-gradien meta-gradients wrt. the initialization Wa and the integration time T. Computing the meta-gradients wrt. & involves non-minimal dependencies on Haiku (Hennigan et al., 2020), and therefore is omitted here. The full code is available at https: gighhb...b...... woon/tristandeleee jax- moxx-c
Data generation & hyperparameters We used the minemmaeeeet and maared datasets provided by Lee et al. (2019) in order to create the 1-shot 5-way and 5-shot 5-way tasks for both datasets. During evaluation, fixed set of 1,000 tasks sampled for each setting; this means that both architectures for COMLN have been evaluated using exactly the same data, to ensure direct comparison across backbones. A full description of all the hyperparameters used in COMLN given in Appendix E.
Reproducibility baseline results To the best of our ability, we have tried to report baseline results from existing work, to limit much possible the bias induced by running our own baseline experiments. The references of those works are given in Table 3. We still had to run CAVIA and iMAML on the remaining settings, since these results have not been reported in the literature. For both methods, we used the data generation described above.
CAVIA: We used the official implementation' We used the hyperparameters reported in (Zintgraf et al., 2019) for minilmageNet, and an architecture with 64 filters.
iMAML: We used the official implementation'. We used the hyperparameters reported in (Rajeswaran et al., 2019) 9) for minilmaeeNet 1-shot 5-way.
Table 3: References for the results provided in Table 2: O (Liu et al., 2019), O (Oh et al., 2021), O (Aimen et al., 2021), O (Arnold et al., 2021), and O are reported in their respective references (under Model). Recall that * denotes baseline results we executed using the official implementations.
miniImageNet 5-way
tieredImageNet 5-way
Model
Backbone
1-shot
5-shot
1-shot
5-shot
MAML (Finn et al., 2017)
Conv-4
48.70 1.84
63.11 t 0.92
51.67 = 1.81
70.30 1.75
ANIL (Raghu et al., 2019)
Conv-4
46.30 = 0.40
61.00 0.60
49.35 = 0.26
65.82 0.12
Meta-SGD (Li et al., 2017)
Conv-4
50.47 + 1.87
64.03 0.94
52.80 + 0.41
62.35 0.26
CAVIA (Zintgraf et al., 2019)
Conv-4
51.82 + 0.65
65.85 + 0.55
52.41 + 2.64
67.55 = 2.05*
iMAMI (Rajeswaran et al., 2019)
Conv-4
49.30 + 1.88
59.77 + 0.73*
38.54 + 1.37*
60.24 0.76*
MetaOptNet-RR RR (Lee et al., 2019)
Conv-4
53.23 + 0.59
69.51 + 0.48
54.63 = 0.67
72.11 1 0.59
MetaOptNet-SVM (Lee al., 2019)
Conv-4
52.87 1 0.57
68.76 0.48
54.71 1 0.67
71.79 0.59
COMLN (Ours)
Conv-4
53.01 + 0.62
70.54 0.54
54.30 + 0.69
71.35 0.57
MAML (Finn al., 2017)
ResNet-12
49.92 + 0.65
63.93 + 0.59
55.37 = 0.74
72.93 0.60
ANIL (Raghu et al., 2019)
ResNet-12
49.65 + 0.65
59.51 0.56
54.77 = 0.76
69.28 0.67
MetaOptNet-RR (Lee et al., 2019)
ResNet-12
61.41 1 0.61
77.88 0.46
65.36 1 0.71
81.34 0.52
MetaOptNet-SVM (Lee et al., 2019)
ResNet-12
62.64 1 0.61
78.63 0.46
65.99 1 0.72
81.56 1 0.53
COMLN (Ours)
ResNet- 12
59.26 + 0.65
77.26 0.49
62.93 - 0.71
81.13 0.53

---

Published as a conference paper at ICLR 2022
REFERENCES
Aroof Aimen, Sahil Sidheekh, and Narayanan Krishnan. Task Attended Meta-Learriig for Few-Shot Learning. arXiv preprint, 2021.
Sébastien MR Arnold and Fei Sha. Embedding Adaptation is Still Needed for Few-Shot Learning. arXiv preprint, 2021.
Sébastien MR Amold, Guneet S Dhillon, Avinash Ravichandran, and Stefano Soatto. Uniform Sampling over Episode Difficulty. arXiv preprint, 2021.
Dmitry Baranchuk.
Memory Efficient MAML, 2019. URL https://github. com/
Yoshua Bengio, Samy Bengio, Jocelyn Cloutier, and Jan Gecsei. Learning Synaptic Learning Rule. International Joint Conference on Neural Networks, 1991.
Luca Bertinetto, Joao Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with Differen- tiable Closed-Form Solvers. arXiv preprint, 2018.
John T Betts. Practical Methods for Optimal Control and Estimation Using Nonlinear Programming. SIAM, 2010.
Lorenz T. Biegler. Nonlinear Programming. Society for Industrial and Applied Mathematics, January 2010.
Dankmar Böhning. Multinomial Logistic Regression Algorithm. Annals of the Institute of Statistical Mathematics, 1992.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, and Skye Wanderman-Milne. JAX: composable transformations of Python+NumPy programs, 2018. URL //ghtbb.////bbb/t/ com google e/jejjjaa jax.
A. E. Bryson and Y. C. Ho. Applied Optimal Control. Blaisdell, New York, 1969.
John Charles Butcher. Numerical Methods for Ordinary Differential Equations. Wiley, 2008.
Makis Caracotsios and Warren E Stewart. Sensitivity analysis of initial value problems with mixed odes and algebraic equations. Computers & Chemical Engineering, 94):335-336,, 1985.
Chavent, RE Goodson, and M Polis. Identification of parameter distributed systems. Identification of function parameters in partial differenniaal equations, pp. 31-48, 1974.
Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural Ordinary Differential Equations. Advances in Neural Information Processing Systems, 2018.
Xinshi Chen, Hanjun Dai, Yu Li, Xin Gao, and Le Song. Learning To Stop While Learning To Predict. In International Conference on Machine Learning, 2020a.
Yutian Chen, Abram L. Friesen, Feryal Behbahani, Arnaud Doucet, David Budden, Matthew W Hoffman, and Nando de Freitas. Modular Meta-Learning with Shrinkage. Neural Information Processing Systems, 2020b.
John R Dormand and Peter J Prince. A family of embedded Runge- Kutge formulae. Journal of computational and applied mathematics, 1980.
Leonhard Euler. De integratione aequationum differentialium per approximationem. Opera Omnia, 1913.
William F Feehery, John E Tolsma, and Paul Barton. Efficient sensitivity analysis of large-scale differential-algebrais systems. Applied Numerical Mathematics, 25(1):41-54, 1997.
Chelsea Finn. Learning to Learn with Gradients. PhD thesis, UC Berkeley, 2018.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- Model-Agnostic Meta-Leaming for Fast Adaptation of Deep Networks. International Conference on Machine Learning (ICMI), 2017.
12

---

Published as a conference paper at ICLR 2022
Sebastian Flennerhag, Pablo G Moreno, Neil D Lawrence, and Andreas Damianou. Transferring knowledge across learning processes. arXiv preprint, 2018.
Sebastian Flennerhag, Andrei A Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell. Meta-Learning with Warped Gradient Descent. International Conference on Learning Representations, 2020.
Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-basee based hyperparameter optimization. In International Conference Machine Learning, 2017.
Golnaz Ghiasi, Tsung-Y Yi Lin, and Quoc Le. Dropblock: A regularization method for convolutional networks. In Neural Information Processing Systems, 2018.
Amir Gholami, Kurt Keutzer, and George Biros. ANODE: unconditionally accurate memory-efficien gradients for neural odes. CoRR, Mtss/1902.1029,, 2019. URL http:///xx/// 1902. 10298.
Andreas Griewank and Andrea Walther. Evaluating Derivatives. Society for Industrial and Applied Mathematics, January 2008.
Yue Guo, Felix Dictrich, Tom Bertalan, Danimir T Doncevic, Manuel Dahmen, Ioannis Kevrekidis, and Qianxiao Li. Personalized Algorithm Generation: A Case Study in Meta-Learning ODE Integrators. arXiv preprint, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, 2016.
Tom Hennigan, Trevor Cai, Tamara Norman, and Igor Babuschkin. Haiku: Sonnet for JAX, 2020. URL http::ttp//. / /gi thub. - com/ dttpdeeepppnn////// ddmhaiikkk
Daniel Jiwoong Im, Yibo Jiang, and Nakul Verma. Model-Agnostic Meta-Learning using Runge- Kutta Methods. arXiv preprint, 2019.
Khurram Javed and Martha White. Meta-Learning Representations for Continual Learning. In Advances in Neural Information Processing Systems, 2019.
Daniel Jiwoong Im, Cristina Savin, and Kyunghyun Cho. Online hyperparameter optimization by Real-Tima Recurrent Learning. arXiv preprint, 2021.
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with Differentiable Convex Optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
Jorge Leis and Mark A Kramer. The simultancous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Transactions Mathematical Software (TOMS), 14(1):45-66 1988.
Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy Hospedales. Learning to Generalize: MetanLearning for Domain Generalization. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
Shibo Li, Zheng Wang, Akil Narayan, Robert Kirby, and Shandian Zhe. Meta-Learning with Adjoint Methods. arXiv preprint, 2021.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-SGD: Leamingg to learn quickly for few-shot learning. arXiv preprint, 2017.
Jacques Louis Lions and Enrico Magenes. Non-homogeneous boundary value problems and applica- fions: Vol. 1, volume 181. Springer Science & Business Media, 2012.
Yanbin Liu, Juho Lee, Minseop Park, Sachoon Kim, Eunho Yang, Sung Ju Hwang, and Yi Yang. Learning to Propagate Labels: Transductive Propagation Network for Few-Shot Learning. Interna- fional Conference on Learning Representations, 2019.
13

---

Published as a conference paper at ICLR 2022
Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-basee hyperparameter optimization through reversible learning. In International conference on machine learning, pp. 11222.2 PMLR. 2015.
Timothy Maly and Linda R Petzold. Numerical methods and software for sensitivity analysis of differential-aa allgebbrai systems. Applied Numerical Mathemattics, 20(1-2):57-79, 1996.
Paul Micaelli and Amos Storkey. Gradient-based Hyperparameter Optimization Over Long Horizons. In Neural Information Processing Systems, 2021.
Alex Nichol, Joshua Achiam, and John Schulman. On First-Order Meta-Learning Algorithms. arXiv preprint, 2018.
Jachoon Oh, Hyungjun Yoo, ChangHwan Kim, and Se- Young Yun. BOIL: Towards Representation Change for Few-Shot Learning, International Conference on Learning Representations, 2021.
Antonio Orvieto and Aurelien Lucchi. Shadowing Properties of Optimization Algorithms. In
Advances in Neural Information Processing Systems, 2019.
John Platt and Alan Barr. Constrained Differential Optimization. In Neural Information Processing Systems, 1988.
Boris T Polyak. Some methods of speeding up the convergence of iteration methods. computational mathematics and mathematical physics, 1964.
USSR
Lev Semenovich Pontryagin. Mathematical theory of optimal processes. Routledge, 2018.
Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse?
towards understanding the effectiveness of maml. arXiv preprint, 2019.
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-Learning with Implicit Gradients. In Advances in Neural Information Processing Systems, 2019.
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S 7emel. Meta-learning for semi-supervised few-shot classification. In International Conference on Learning Representations, 2018.
Jonas Rothfuss, Dennis Lee, Ignasi Clavera, Tamim Asfour, and Pieter Abbeel. ProMP: Proximal Meta-Policy Search. International Conference on Learning Representations, 2019.
Carl Runge. Über die numerische auflösung von diffcrentialgleichungnn. Mathematische Annalen, 1895.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211122 1-22, 2015.
Andrei A Rusu, Dushyant Rao, Jakub Sygnowski. Oriol Vinyals, Razvan Pascanu, Simon Osindero, and Raia Hadsell. Meta-learning with Latent Embedding Optimization. arXiv preprint, 2018.
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. One-shot learning with memory-augmentee neural networks. arXiv preprint. 2016.
Jürgen Schmidhuber. Evolutionary principles in self-referentaal learning, or on learning how to learn: the meta-mela-... hook. PhD thesis, Technische Universität München, 1987.
Radu Serban and Alan C Hindmarsh. Cvodes: An ode solver with sensitivity analysis capabilities. Technical report, Technical Report UCRL-JP-200039, Lawrence Livermore National Laboratory, 2003.
Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots. Truncated Back- soeck-rrppggtton for Bilevel Optimization. In International Conference on Artificial Intelligence and Statistics, 2019.
14

---

Published as a conference paper at ICLR 2022
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical Networks for Few-shot Learning. In Advances in Neural Information Processing Systems, 2017.
Weijie Su, Stephen Boyd, and Emmanuel Candes, A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights. Advances in Neural Information Processing Systems, 2014.
Flood Sung, Yongxin Yang. Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
Richard S. Sutton. Adapting Bias by Gradient Descent: An Incremental Version of Diiaa-Ba--Delle In Proceedings of the 10th National Conference on Artificial Intelligence, 1992.
Sebastian Thrun and Lorien Pratt. Learning to learn. Springer Science & Business Media, 2012.
Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola. Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need? 2020.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Neural Information Processing Systems, 29:3630-3638, 2016.
Johannes Von Oswald, Dominie Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, and João Sacramento. Learning where to learn: Gradient sparsity in meta and continual learning, Advances Neural Information Processing Systems, 2021.
Stephen Wiggins. Introduction Applied Nonlinear Dynamical Systems and Chaos, volume Springer, 2003.
Ronald J. Williams and David Zipser. A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. Neural Computation, 1989.
Ashia C Wilson, Benjamin Recht, and Michael Jordan. A Lyapunov Analysis of Momentum Methods in Optimization. arXiv preprini, 2016.
Ruitu Xu, Lin Chen, and Amin Karbasi. Meta Learinng in the Continuous Time Limit In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics. PMLR, 2021.
Zhongwen Xu, Hado van Hasselt, and David Silver. Meta-Gradient Reinforcement Learning. In Advances in Neural Information Processing Systems, 2018.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn.
Gradient Surgery for Multi-Task Learning. Neural Information Processing Systems, 2020.
Baoquan Zhang, Xutao Li, Yunming Ye, Shanshan Feng, and Rui Ye. MetaNODE: Prototype Optimization as Neural ODE for Few-Shot Learning. arXiv preprint, 2021.
Yufan Zhou, Zhenyi Wang, Jiayi Xian, Changyou Chen, and Jinhui Xu. Meta- Leta-Learning with Neural Tangent Kernels. International Conference Learning Representations, 2021.
Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning. In International Conference on Machine Learning, pp. 7693-7702.
PMLR, 2019.
15

---

Published as a conference paper at ICLR 2022
Appendix
The Appendix is organized as follows: in Appendix A we give the pseudo-code for meta-training and meta-testing COMLN, along with minimal code in JAX (Bradbury al., 2018). In Appendix B prove the decomposition of the Jacobians introduced in Section 4, and we give the exact dynamics of 8m (t) and =z(i, J, ml in the decomposition of dW (t)ddm. omitted from the main text for concision. In Appendix C we show how the total derivatives may be computed from the Jacobian matrices without ever having to form them explicitly, hence maintaining the memory-efficiency described in Section 4.2. In Appendix D, we show that unlike the adjoint method described in Section 2.2, the algorithm used in COMLN compute the meta-gradients based on the forward sensitivity method is stable and guaranteed not to diverge. Finally in Appendix E we give additional details regarding the experiments reported in Section 5, together with additional analyses and results on the dataset introduced in Rusu et al. (2018).
A ALGORITHMIC DETAILS
A.I META-TRAINING PSEUDO-CODE
We give in Algorithm the pseudo-code for meta-training COMLN, based on distribution of tasks p(T). with references to the relevant propositions developed in Appendices B and C. Note that to simplify the presentation in Algorithm 1, we introduce the notations V, and θC, to denote the total derivative of the outer-loss, and the partial derivative of the loss L respectively, both computed at the adapted parameters (T). For example:
V, Wo fo
OC BC(W,(T); (D (W(T)
(ry Ctrain
dWo
Algorithm 1 COMLN Meta-training
Require:
task distribution p(r)
Initialize randomly & and Wo. Initialize T to small value 0.
loop
Sample batch of tasks B ~ p(r)
for all E B do
Embed the training set: fo (Dtrain)= [pmm ym ) M
s s(T), Br, 2r + ODESolve (0, 0, DYNAMICS, 0, 7) W,(T) - Wo -m 8m (b
D Algorithm Proposition
Embed the test set: fo (D,trst)
Compute the partial derivatives: @wer) Ctrain & Ct
V, Wo +- PROJECTw, (Sw Crsst .T,Φ))
Proposition 5 Proposition
V +- PROJECT @... (dw C ,T()), BT, zT, b)
Ss.... Clest v.
V, $ Backpropagation through fa, starting with V, om for all m
V,T (te(t) (Ow(r)
Proposition
end for
Update the meta-parameters:
Wo + Wo
ΣTEB V.Wo
[
① - Σ V
T T [
E V.T
end loop
The dynamical system followed during adaptation is given in Propositions 1 to 3 and is summarized in Algorithm 2. The solution of this dynamical system is used not only to find the adapted parameters W, (T), but also get the quantities necessary compute the meta-gradients for the updates of
16

---

Published as a conference paper at ICLR 2022
the meta- parameters Wo & @. The integration of this dynamical system, named ODESOLVE in Algorithm 1, is done in practice using a numerical solver such as Runge- Kutta methods.
Algorithm 2 Dynamical system
function DYNAMICS(WO o. fo (Dtrain) s(t))
W, (t) < W Em 8m (
Proposition
Pn (t) < (or
(t) + (Pagg(m (t)) Pm (t)pmm (t)
dam 1 (pm(t)= ymm
Proposition
dt
M
(b. dt
M
+ 1 A) (:(t) Σ m=1
(o) Φm) B. [mmm]]]
b Propositions & 3 (t))
ji, dl
f A.(( 1(i = f)5mml) (1) + 1(i m)8;(1) +
Σ
(o) d5))=1 [k,
j.
k=1
return ds/ /d, dBB//dt, dzs/dt end function
In Algorithms 3 & 4, give the procedures responsible for the projection of the partial derivatives OweryChes onto the Jacobian matrices dW (T) /dW) and dW (T') (d')/dm respectively, based on Propositions & 6. Note that Algorithm 4 also depends on the initial conditions Wo. which is implicitly assumed here for clarity of presentation. It is interesting to see that both functions use the same matrix C, and therefore this can be computed only once and reused for both projections.
Algorithm 3 Projection onto dW
Algorithm 4 Projection onto dW (T)
function PROJECT w. (V Br,q0 Φ)
function PROJECT o... (V(T), sT)), BT, ZT, s)
M
M
C Φ. VTT) Br[i,j]
C Φ
return VTT) - CT end function
D Dmnjj
2r11,5 m) V
return
8m (T) V + C. + @
end function
Finally in Algorithms 5 & 6, we show how to use COMLN at meta-test time on novel task, based on the learned meta- parrameeessssss Wo. & and T. This simply corresponds system in W (L) (equivalently, in 8m (L), see Proposition 1). Note that for adaptation during meta- testing, there no need to compute either [i J] 2 2i, ml, since these are only necessary to compute the meta-gradients during meta-training.
integrating dynamical
Algorithm 5 COMLN Meta-test
Algorithm 6 Dynamical system for adaptation
Require: A task r with a dataset Dirrin
function DYNAMICS(W. WW fa (Diraiu) s(t))
Require: Meta-parameters 4, Wo T
W, (t) + Wo Σ. sm (t (
Embed the training dataset: fo (Dtrain)
Pm. (t) softmax W, (t))))
(T) f- ODESolve(O, DYNAMICS O,T)
dsm /dl <- (pm. ym /M
W, (T) + Wo Em sm (T)
return ds/dt end function
return W-(T) of
17

---

Published as a conference paper at ICLR 2022
A.2 SOURCE CODE
We provide snippet of code written in JAX (Bradbury et al., 2018) in order to compute the adapted parameters, based on Algorithms & 2. The dynamics as function not only computes the vectors 8m (t) necessary to compute W (see Appendix B.1), but also B Be j] and z, j. ml] jointly in order to compute the meta-gradee veta-gradients, This snippet shows that various necessary quantities can be precomputed ahead of adaptation, such as the Gram matrix gram.
1 import jax, numpy as jnp
2
3 from jax import vmap, grad, nn, eps, tree_util
4 from jax. experimental import ode
5 from collections import namedtuple
6
7 State namedtuple ( State', ['s', 'B', '21])
9
M, = train_innuten inputs shape [0], train labels. shape
gram
wwtmuatmulllll (train_inputs, train_inputs.11 logits __ - inppmaatmn (train_inputs, w_0.1)
11
diag jnp. app.diag_indices (M)
13
def dynamics (state, _
15
preds mn. softmax (logits_0 jnp matmul (gram, state.s), axis=1) (vmap (ipp.daag) (preds) vmap (inp.outee outer) (preds, preds)) M
18
Update of s
ds (preds train lapels) /
20
Update of cross_pred jinsui injnnn--ij A, gram, state.B)
22
einsum
dB cps index_add (-ccrsssddc____pppp________s crosss_prdd, diag, A)
24
Update of
26
cross prod inp. expsum ('iln, ik, fmn->ijml', A, gran, state.
A jap amp.eimme einsum j1-sijk', A, state.s)
dz ops. index_ado (cross_prod, diag, A_s)
29
dz ops. index_add (dz, (diag(0), None, diag(ll), A_s)
30
31
return State (s=ds, B=dB, 2=-dz)
s State( 33
s=jnp.zeros: (M,
35
B-jnp. zeros (M, M, N, N)), z-jnp. zeros (M, M, M, N))
37
solution ode odeint (dynamics, state_0, inp.array (10., II))
39
state_T tree_util. teeetreeee_meee (lambda x[-1], solution) T 0 jnp.nnnaiti matmul (state_T.e T.s.T, crain_inputs)
40
Snippet of code in JAX to compute the adapted parameter W_T for a given task specified by the embedded training set train train_inputs s & train labels (note that here train inputs are the embedding vectors returned by fa) and an initialization 0.
Code Snippet I:
We want emphasize that all the information required compute the meta-gradients are available in state_T T found after integration of the dynamics function forward in time, and does not require any additional backward pass (apart from backpropagation through the embedding network fo). In particular, the meta- getaiggaaseets wrt. the initial conditions W, and the wrt. the time horizon T can be computed using the snippet of code available Code Snippet 2.
18

---

Published as a conference paper at ICLR 2022
41
grads_test grad (cross_enttttt (cross__nttroy_llss))) (MLT, test_inputs, test_labels) grads_train
(W_T, train_inputs, train_labeis)
43
amppeinsum. in, kn, 1jk1->j1', train inputs, grads state T.B)
45
grads_ grads_test jnp matmul (C.T, train_inputs) grads_T = -jnp. vdot train, grads_test)
47
Code Snippet 2: Snippet of code in JAX to compute the meta-gradients wrt. the meta meta- parameters w_0 and т, based on state_T found above. See Appendix B.4 & Appendix C for details.
The code compute the meta-gradients wrt. the meta- parameters of the embedding network + is not included here for clarity, as involves non-minimal dependencies on Haiku (Hennigan et al., 2020) in order backpropagate the error through the backbone network. The full code is available https: / /ggthub.....gggg com ri stande eu/ jaxx jax-ccommn
B PROOFS OF MEMORY EFFICIENT META-GRADIENTS
In this section, we will prove the results presented in Section slightly more general case, where the loss function L is the cross-entropy loss regularized with a proximal term around the initialization (Rajeswaran et al., 2019):
1
L W: fo (D (Drraln)
Σ m=1
Um T
log Pm
W Woll
(14)
M
where Pm. soft Φm). || is the Frobenius norm, and À is regularization constant. Note that we can recover the setting presented in the main paper by setting À == 0. The core idea of computing the meta-gradien efficiently based on the decomposition of the gradient (as well as the Hessian matrix) of the cross-entropy loss (Böhning, 1992). We recall this decomposition as the following lemma, extended include the regularization term:
Lemma 1 (Böhning, 1992). Let fo ('Drain))
[(Øm: yn)) be the embedded training set through the embedding network fa. and the regularized cross-entrops loss defined in (14). The gradient & C and the Hessian matrix 4 C of the regularized cross-entropy loss can be written as
Fo (D)rain)
1
Σ
(pm
v +(W Wo)
M
M
V'C(W; (o (Dtrain)
Σ
Am x on TT AI,
m=1
where Am aan==diaggen (diag(pm.) P. /M are N x matrices, and the Kronecker product.
This lemma is particularly useful in the context few-shot learning since it reduces the characteriza- tion the gradient and the Hessian from quantities size N x d and Nd d Nd respectively (where d is the dimension of the embedding vectors Φ) to M objects whose size is independent of of d-the embedding vectors & being independent of W. Typically in few-shot learning, M << d.
In order to avoid higher-order tensors when defining the different Jacobians and Hessians, we always consider them as matrices, with possibly an implicit "flattening" operation. For example here even though W is a N x d matrix, we treat the Hessian L as a Nd x N d matrix, as opposed to a 4D tensor. When the context is required, we will make this transformation from higher-order tensor to matrix explicit with an encoding of indices. Moreover throughout this section, we will only consider the computation of the veta-gradients for a single task т, and therefore we will often drop the explicit dependence of the different objects on (e.g. we will write (t)) instead of W+(t)); the meta- gradiegndsents are eventually averaged over a batch tasks for the update of the outer-loop. Appendix A for details in the pseudo-code. Finally, since only consider adaptation of the last layer of the neural network, the presentation here always made in the context of an embedded training set fo (Dirain) ((pm, ym J. M that went through the backbone fo.
19

---

Published as a conference paper at ICLR 2022
B.1 DECOMPOSITION OF (t)) FOR PARAMETER ADAPTATION
As direct application of Lemma 1, we first decompose the parameters (() into smaller quantities 8m (I) that follow the dynamies defined in Prop. Although this decomposition equivalent to W (t), in practice solving smaller dynamical system in m. (t) improves the efficiency of our method.
Proposition 1. Let fo (DDrriin) {( ym ) m== be the embedded training set through the embed- ding network fa. and W (t) be the solution of the following dynamical system
dW
dt V (o
W Wo.
where the loss function is the regularized cross-entropy loss defined in (14). The solution (() of this dynamical system can written as
W(1) = Wo
Σ 8m (l
where for all m, sm (t) is the solution of the following dynamical system:
dam m
1
dt M (pm(l) -ym Amm (1)
8m (0) = 0,
and pm (t) softmaax W(t))))t))))))) (t)@m) is the vector of predictions returned by the network at time t.
Proof.
Using Lemma 1, the function ((t) is the solution of the following differential equation:
d.
dt VWWW(()) (t); (()
1 M Σ (1m(1) - ym DT. X(W(1) Wo)
W Wo.
The proof relies on the unicity of the solution of a given autonomous differential equation given particular choice of initial conditions (Wiggins, 2003, Prop. 7.4.2). In other words, we can find another function (t) that also satisfies the differential equation above with the initial conditions (0)) Wu. then it means that for all have ((t) Wtt). Suppose that this function W(t) can be written as
M
((t) = Wo
Σ
8m
(t)
m=1
where 8m satisfies the dynamical system defined in the statement Proposition Then we have
fo):
(pm(t)
( ((W(t) Wo)
M dsm
M sm (t) ((W(t) Wo)
Φ m
dt
m=1
m=1
M
d8m
dW
dt
m=1
We have shown that (t) follows the same dynamics as W (t). Moreover, using the initial conditions sm (0) O, is clear that (0)) Wo. which are the same initial conditions the equation satisfied by (t)). Therefore we have W(t) W(t) for all t, showing the expected decomposition of (()..
20

---

Published as a conference paper at ICLR 2022
B.2 DECOMPOSITION OF THE JACOBIAN MATRIX dw (t)/dW
The core objective of gradient- based meta-learning methods is the capacity to compute the meta- gradients the initial conditions Wo- In order to compute this meta-gradient using forward-mode automatic differentiation, we want to first compute the Jacobian matrix dW (L)/dWo We show that this Jacobian can be decomposed into smaller quantities that follow the dynamics in Proposition 2.
Proposition 2. Let fo )D [( ym } be the embedded training set through the embed- ding network f4- The Jacobian matrix dWW (W)/dWW can be written as
M
M
(t)
= - ΣΣ
B,ii,j] & 6.07,
dWo
==
where & is the Kronecker product, and for all i,j, Beli. is N N matrix which the solution of the following dynamical system
[t.. = 1(:= j) - ABI(i, A,(t) dt
M 2m) Bl
Boli. 0,
m=1
and A,(t) (p: - (t())p(((TT) /M defined using the vectors of predictions at time t (t) softmax W (t)
Proof. We will use the forward sensitivity equation from Section 2.2 in order to derive this new dynamical system over B,ji,j) Recall that to simplify the notations we can write the gradient vector field followed during adaptation
dW
= (t):: (Dimin)) VVC(W(()) (()
(Dtrain).
dt
Introducing the matrix valued function S(t) (t) sensitivity state, we can use the forward sensitivity equations and see that S(t) satisfies the following equation
dS 0g ((WW()) (0) di (W()) S(L) + OW,
(W(())
S(0) =
V (();; fo (D.rain)) )()) + XI.
The rest of the proof is based on the unicity of the solution of given autonomous differential equation given particular choice of initial conditions (Wiggins, 2003, Prop. 7.4.2). In other words, if we can find another function S(t) that also satisfies the above differential equation with the initial conditions S(0) I, then it means that for all / have S(I) S(I). Suppose that this function can be written as
5(1==1 -1 B & 6.
where Be(i,)] satisfies the dynamical system defined the statement of Proposition 2. Then we have, using Lemmal 1:
V"c(W(t)) (D) (rain )) + AX
M
Σ A.(1) @ Φ. +NI
+ 1- Σ B, @
10 Sm + +YI
m.
A,())
Σ
(Ø. Φ pn 2 (m,
pm + +
B...
k.j.m
ER
'While the dynamical system defined here for the sensitivity state S(t) alone not exactly autonomous due to the dependence of the Hessian matrix on W (t), we could augment S(t) with W (t) to obtain an autonomous system on the augmented state. We come back to this distinction in Appendix D. The unicity argument still holds here (the augmented solution would be unique).
21

---

Published as a conference paper at ICLR 2022
M
= Σ 1(i j) - AB.(i,]] - (,(1) Σ
(0) (m) B m, @ 0
m=1
(i.1 dt
dS
& $
=
dt
We have shown that S(t) follows the same dynamics as S(t). Moreover, using the initial conditions Boli.j] 5] O, it is clear that S(O) I. which are the same initial conditions the equation satisfied by S(t). Therefore, we have S(t) S(t) for all showing the expected decomposition of S() (t)///dW,
B.3 DECOMPOSITION OF THE JACOBIAN MATRIX dW (t)/dmmm
Similar to Proposition 2, there exists a decomposition of the Jacobian matrix dW /dp. Recall that this Jacobian matrix appears in the computation of the gradient of the outer-loss wrt. the embedding vectors Φm. which is necessary in order to compute the meta-gradiee getta-gradents wrt. the meta- parameters of the embedding network fe using backpropagation from the last layer of fe-
Proposition 3. Let fa (Diraiu) {(oms ym ) be the embedded training set through the embed- ding network f4. For all m, the Jacobian matrix dW (ll())/oooo (t)/ddm can be written as
M (I = [ 8m ( (t) & Ia 1 Σ dΦm
M
M
B
m Wo
@ Φ
2l.. J. mlØ] & &
A=
(m(() (t) and j] are the solutions of the ODEs defined in Propositions 1 & 2, and for all where 8m
j. m, ze J. ml] is vector of length N solution of the following dynamical system:
dzeli.j. dt
A,c0s((0 1 -
(t) 1 mts,(t) + Az, A,, ml
Φ Φ2) ze Ie
with the initial conditions zoli, m] O.
Proof. The outline this proof follows the proof of Proposition 2: we first use the forward sensitivity equations get the dynamical system followed by the Jacobian matrix dW(tt) ddp. and then use unicity argument given that the new decomposition satisfies the same ODE. Recall that we use the following notation write the gradient vector field followed during adaptation:
dW di = (t)) (D!rain) A VVVC(W(()) (D)rain)
For fixed m, we introduce the matrix valued function S(t) dw (t)/ddd as sensitivity state. We can use the forward sensitivity equations, and see that S(t) satisfies the following equation
dS @g( (t), Φm) S(I) +
Ög( ( Φm)
dW 0
S(0)
di (W())
dΦp
CCW (t); fo (D.rain)) S(t) Wt & Rm
M
+ A. (1) Σ (1)) s0 pm 1 M (pm - ym) 00 Is
where make the direct dependence of g o.. explicit, and we used the decomposition of W(t) from Proposition Suppose that we define the function S(t) as
M
š(t)
9m (t) la Σ
B,üi Wo & Φ + E 2 m|o @
22

---

Published as a conference paper at ICLR 2022
where j. m] satisfies the dynamical system defined in the statement of Proposition 3. Then have, using Lemma 1:
M Σ
- -V2C(W(1), (t); fo (rrii (D ()((t) Am
Wo 8:(1)
@ Φm
M (pm (t) ymm @ la
1=1
Σ (t(t) 2 0. + 9m 2 La +
B₁ ml Wo 2 Φ + Σ
zeli.j. ml &Φ.
2.
M
Am (t) @ pm + Am (t)
Σ
(t)o @ om
1
(pm(t) ym @ Ia
M
t=1
1
M (pm(t) -ym) Atm(t)
@ la
dam. fet
M
M AB,(i, m] (t,
1(i= (i=mm)A(t) - -
Σ
(o) Φx B (k ml
Wo @
(t.m//mi
M
+
Σ (,(t) 1(i 1010)(((00)00)) (1 + 1(i = m)s, + Azlli,j,
+ > (o, xz) 1k.
-dzz(t.f.mm/d
dS
dl
We have shown that S(t) follows the same dynamics as S(t). Moreover using the initial conditions
(0) 0, [i] O, and zo(i, j, ml = O, we have S(O) 0, which are the same initial conditions the equation satisfied by S(t). Therefore have S(t) S(t) for all showing the expected decomposition of the Jacobian matrix S(t) = dW(()) (t)/do...
B.4 PROOF OF THE META-GRADIER WRT. T
The novelty of COMLN over prior work on gradient- meta-learning is the ability to meta-learn the amount of adaptation T using SGD. To compute the meta-gradient wrt. the time horizon T, can apply the result from Chen et al. (2018). Proposition Let fo (Dirain) and fo (Dteat be the embedding through the network fo of the training and test set respectively. The gradient of the outer-loss wrt. the time horizon T is given by:
dC(W(I);; (a (D df
- TOC(W (T); fo (D(ss)) Oc(W (T); for (Drraan)) ))
@W(T)
OW
Proof. We can directly apply the result from (Chen et al., 2018, App. B.2). which we recall here for completeness. Given the ODE dz/dt g(z(t)), the gradient the loss L(z(T)) wrt. T' given by
dc
ac
di di a (()(T))
where a(T) (z)
is the initial adjoint state (see Section 2.2). In case we have
a(T) ac (OC(W(T): (D.est)
g(x(T)) = (T): fo
OW (T)
23

---

Published as a conference paper at ICLR 2022
C PROJECTION ONTO THE JACOBIAN MATRICES
Once we have computed the Jacobian dW (T) wrt. some meta-parameee (in our case, either the initial conditions Wo the embedding vectors Φm), we only have to project the vector of partial derivatives onto to obtain the meta-gradients (vector-Jacobian product), using the chain rule:
dC W ((. WC(W fo (T( d0
(T);
d0
(T(
00
While we have shown how to decompose the Jacobian matrices in such a way that it only involves quantities that are independent of the dimension of the embedding vectors Φ. this final projection priori requires us to form the Jacobian explicitly. Even though this operation is done only once at the end of the integration, this may be overly expensive since the Jacobian matrices scale quadratically with d, which can be as high as d = 16,000 in our experiments.
Fortunately, we can perform this projection as function of 8. (T), Br j]. and [r, m). without having to explicitly form the full Jacobian matrix, by exchanging the order of operations.
Proposition 5. Let fo (Dtrai)) = {(om. ) and f (Dirst) be the embedded training set and test set through the embedding network fo respectively. Let Brlū,j jl, be the solution at time I of the differential equation defined in Proposition 2, and (TT) E IR Nxd xå the partial derivative of the outer-loss wrt. the adapted parameters (T)):
V(T) (T); (D.es)) OW(T)
that (.): (o); Vec Vec(V(T))
(T) dWo
dWo
Let Φ = loi...., .Mll E BMxd be the design matrix. The gradient of the outer-loss wrt. the initial conditions Wo can computed as
(C'W; (Dikstt)
dWo
where the rows of C E M are defined by
C = Σ
V(T) Br(i,)]-
Proof. In this proof, we will make the encoding of the indices for the Jacobian dW (d)/ more explicit, as mentioned in Appendix B. Recall from Proposition 2 that this Jacobian matrix can be decomposed as
dW (T)
Σ i
Br 1 20
dWo
Introducing the following notations
F
Σ Brli. & $ E Ndx
G Vec(V(()) FRRN
i
for all l E (o...., N 1} and (o....,d-11)
d
N I d-1
Gidl + y) =
V1 k x F dh + x, dl + yl
k=D
Vr x [00
B k,
[T))
N-1
=
Σ
((T)
d
1o V(T) Brbsll
24

---

Published as a conference paper at ICLR 2022
M
M
V(T) T Brli, jl,
Ø, [[TT b
C2
This shows that G is equal to C & up to reshaping. Using the full form of the Jacobian, including I. concludes the proof.
An interesting observation is that if the term C' Φ is ignored in the computation of the meta-gradient, we recover an equivalent of the first-order approximation introduced in Finn et (2017). Similarly, we can show that we can perform the projection onto the Jacobian matrix dW (T) / without having to form the explicit Nd x d matrix.
Proposition Let f& (Dtrain) {(om. ym) M= and f (D be the embedded training set and test set through the embedding network fa respectively. Let s. (T') be the solution at time T of the differential equation defined Proposition Br j] the solution of the one defined in Proposition and zr J. m) the solution of the one defined in Proposition 3. Let V(T) E xx be the partial derivative of the outer-loss wrt. the adapted parameters (T)):
V(T) @C(W(TT fo (Distt)
(T
Let Φ - 1 MM) E Mxd be the design matrix. The projection of V (T) onto the Jacobian matrix (T) dmmm can be computed as
Vec (VT))) T (T
8m (V)) Cm Wo + Dm
ddm
where Cm E RNN and Dm E RM are defined by
C B Brlim
Dm. Σ
æTli, m] VTT
Note that the definition of C in Proposition 6 matches exactly the definition of C' in Proposition 5, and therefore we only need to compute this matrix once to perform the projections for both meta-gradients.
Proof. Recall from Proposition that the Jacobian dW (T))//dó, can be decomposed
(W _ 8m (T) 2 la
M
06. + Σ श 2T|...mmog @ Φ
ddm
We will consider each of these 3 terms separately.
For the first term, let
F 8m (7) 8 La E TR Naxd
G Vec(V(()) T F ER.
and for y E {0. .o...... d d1]:
N-1d-1
N-1d-1
Gilvl =
Vr Vr 1
ldk x.
Vrik, = sm (T))
R=0 ==0
k= 0 ==0
8m
Vr y]
[sm (T) (TT)
Therefore, G, the projection of VTT) onto the first term of the Jacobian matrix is equal to 8m (T) V(T).
25

---

Published as a conference paper at ICLR 2022
For the second term, let
F2
Brlim & e Ndxd
G2 a Vec F2 R.,
1=1
and for y (O,...,d 11:
N-1d-1 G2lyl
Vr x F2 + x,y1
k=0 ==
M N-1N-1
Vr (k. [x]
Brli, m, k, W. (1
=VV(T)
N-1
ΣΣ
(T)) m, k
Wolkel
1=0
=[0Tv(297 Br]im
Φ V(T) Br(i, ml]
Woll.y] [Cm Wol,
Cmn
G2 the projection of ((T) onto the second term of the Jacobian matrix is equal to C. Wo.
Finally for the third term, let
F3 A
Zrli, m. @ E Naxd
G3 - (VT)) T F3 E Rd
and for y {O....,.o 1}:
N-1d-1
Gzly] Σ
1k F + x,81
k=0 m=0
Σ
VT [k [1(00
2T pr(.j,m [
1 km
D=0
-IV(T)bb)
(TT))))azrlii..... kl 0by4ll
arlisim V(
M
M
24 [. m]
(
ø = =|mmø]
D
G3 the projection of VTT) onto the third term of the Jacobian is equal to D,.
Note that the projection of (T) as defined in Proposition is not equal to the meta- grttdegr itself, as it is missing the term coming from the partial derivative of the outer-loss wrt. the embedding vector dc(W (T); fo (Die*t) /00 which is non-zero unlike the counterpart for W,. However, this projection is the only expensive operation required to compute the total gradient.
26

---

Published as a conference paper at ICLR 2022
D PROOF OF STABILITY
In this section, we would like to show that unlike the adjoint method (see Sections 2.2 and 3.2), our method compute the meta-gradients of COMLN is guaranteed be stable. In other words, under some small perturbations due to the ODE solver, the solution found by numerical integration going to stay close the true solution of the dynamical system. To do so, we use the concept of Lyapunov stability of the solution of an ODE, which we recall here for completeness:
Definition (Lyapunov stability; Wiggins, 2003). The solution @(t) of a dynamical system is said to be Lyapunov stable if. given E > O, there exists such that for any other solution æ(t) such that (0) =0))| < 6, we have (t) xxt)]l < & for all >
We would like understand the conditions needed for function f R. R", such that a trajectory (()) satisfying the following autonomous differential equation is (Lyapunov) stable:
dx di
f(x(t))
If assume for moment that f(x) (Lxx) where L is convex function, then for any two trajectories æ (t) & (t)()) of the dynamical system above, we can see that the function F' defined by
F(t) 2
= 2 x2(())1
satisfies F(t) S O, where we use the notation F(t) - dF/dt. Indeed, since V is positive semi-definite and by defining htt) (t x (t), we get
F(()= dF
dF
dh
dl
dh
di
(1=2(1) 21 (t)) (1)2()) (t)
(x2(() =1(()) (f(zz(t)) f(x)
= -N(IT [. (() ds h(t))
(15)
- 1 h(t) v(())==((m(())-- + + sh(t)) h(t) sh(t))
ds
20
where (15) follows from the fundamental theorem of calculus. Now since F(t) I 0, the distance between two trajectories decreases time increases, hence have Lyapunov stability for all trajectories or solutions of the above autonomous differential equation.
Now note that didn't 't actually need f to be of the specific form f(x) V L(x),, but having the Jacobian matrix Df negative semi-definite everywhere would have been sufficient. Hence we get the following statement
Proposition 7. If continuously differentiable function f : R R is such that the Jacobian matrix D is negative semi-definite everywhere, then any solution (t) of the autonomous differential equation
dæ
((x(()
di
with initial condition x(0) = xo is (Lyapunov) stable.
We extend the above result to the non-autonomous case. For this purpose, let define
dx = [(,,x(()))
(16)
dt
which induces the parametrized function ft x + x)). With this notation we can state the following result.
Proposition 8. If the Jacobian matrix D is negative ceni-defiite everywhere for all 2 0, then any trajectory an (t), solution of (16) with initial condition a=()) Pp, is (Lyapunov) stable.
27

---

Published as a conference paper at ICLR 2022
Proof. Let start with two trajectories æ1 (t) and æz((), which solutions to the above (16). By defining h(t) = Ø2(() - (t) and by applying the fundamental theorem of calculus, we get
f(t (2()) (t) fx(()(())
Lo- (1)(() sh()) h(1))
Following the same idea as already previously outlined, let us define F(t) = ||| (t) =22(1) and show that F()) s 0:
F(t) (x2(()) x (t)) (f(t))))( f(t, x1)
-((t) [. (ff(z)(())))))))))))))(() (h(() h(t)
ds 0
Hence the distance between two trajectories decreases as time t increases, and we have Lyapunov stability for all solutions of the non-autonomous differential equation above.
D.1 STABILITY OF THE FORWARD SENSITIVITY EQUATIONS
In this subsection let us start by considering the general case of solving the initial value problem to the following autonomous system of differential equations:
dW dt = g W (t),
W(0) = W.
(17)
dS 0g( (t), 0)
S(I) Og (t).
aw
S(0)
dt OW
00
Proposition 9. There exists solution of the autonomous system of differential equations in (17), which is also unique.
Proof. By Theorem 7.1.1 in Wiggins (2003), we know that there exists solution of the system of differential equations Eq. 17. Then, by Theorem 7.4.1 Wiggins (2003), we can conclude that this solution is unique upon the choice of initial conditions ((0) = Wo and S(0) = OW (0)/00.
Alternatively, one could also apply Theorems 7.1.1 & 7.4.1 in Wiggins (2003) to conclude that the initial value problem dWW /dt = gt) 0) with initial condition ( = Wo has a unique solution. This existence and uniqueness then gives rise to existence and uniqueness of a solution of the entire system of differential equations via S(t) = dW /dt by applying Clairnaut's 's theorem.
In our case, we have g( (t)).,), 0) - (T)) 8), where L a convex function in W. Hence the autonomous system of differential equations can be rewritten
dW
VL(W
W = Wo
dt
(18)
dS
8g (g(W(()) V (()
aw (0)
S(0) 00
dt
00
Proposition 10. Let W (t) be the solution of
dW
dt - (TC(W(T),0)
W(0) Wo
Then the trajectory Wtt) is (Lyapunov) stable.
Proof. Note that Dg (t)) = here. Since is convex. V is positive semi-definite, and hence Dy( V (1)) is negative semi-definite. Hence the result directly follows from Proposition 7. I
28

---

Published as a conference paper at ICLR 2022
Table The effect of the numerical solver on the performance of COMLN. The average accuracy (%) on 1, 000 held-out meta-test tasks is reported with 95% confidence interval. Note that for a given setting, the same 1, 000 tasks are used for evaluation, making both methods directly comparable. RK: 4th-order Runge-Kutta with Dormand Prince adaptive step size. Euler: explicit Euler scheme.
minilmageNet 5-way
Method
1-shot
5-shot
COMLN (RK)
53.01 0.62
70.54 + 0.54
COMLN (Euler)
53.00 0.83
70.50 0.72
In addition we can now also conclude that the solution of (18) is also (Lyapunov) stable, Corollary 1. The solution (tt,,, of (18) is (Lyapunov) stable.
This result is general to the application of the forward sensitivity equations on gradient vector field derived from a convex loss function. We can also show that when the gradient dW' /d0 exists and finite (recall that W is the minimizer of the loss function, and the equilibrium of the gradient vector field), then the solution of the system is also bounded, which guarantees us that our solution will not diverge (unlike the adjoint method applied to gradient vector field).
Proposition 11. Assuming that || (dWW-/de is finite, the solution
[W (4), S()) of Eq. 18 is bounded.
Proof. Let us define
1 |1s(0)1
V(t) = [(0))
Since (t)) and S(t) are continuous functions in t, verifying
dW*
(tt)
W
and
S(t) 71000 d0
8-000
Hence
dW
||
V(I)
W
1|
+ 2
< 20,
2
d0
where the last point follows from our assumption that dW (t)/dfl < xc. We also have that
1 ( 2
dW())
|| <0.
3 2
d0
V(I) being continuous composition of continuous functions, we conclude that V is bounded, and hence (t) and S(t) are bounded as well.
E EXPERIMENTAL DETAILS
E. CHOICE OF THE NUMERICAL SOLVER
In Section 5.2, we showed that the numerical solver had a significant impact on the time to compute the meta-gradients (but not the memory requirements). In Table 4, show that using explicit Euler instead of an adaptive scheme like Runge-Ken Kutta leads to very similar results. In all our experiments we therefore chose Runge- Kutta for its faster execution, and by convenience it is the default numerical solver available in JAX (Bradbury et al., 2018).
E.2 DETAILS FOR MEASURING THE EFFICIENCY
In Figures 3
4, we show the efficiency. both in terms of memory and runtime, of COMLN compared to other meta-learning algorithms. The computational efficiency (runtime) was measured as the average time taken to compute the gradients on a single 5-shot 5-way task from the miniImageNet dataset over 100 runs.
29

---

Published as a conference paper at ICLR 2022
T
10 300
10P
10a
10a
10P
104
10" 10P
T
10
20
10P
10A
10a
10a
104
10a
106
MAML.
MAML
Memory usage (in Gb)
1MAML
ANIL
ANIL COMEN
Runtime (in ms)
COMLN (Euler)
10a
COMEN (RK)
10
10a
10"
10a
102
10a
104
105
106
107
108
109
10a
102
10a
10
10
106
107
109
Number gradient steps
Number gradient steps
Figure 4: Empirical efficiency of COMLN single 5-shot 5-way task, with ResNet- 12 backbone; this figure is similar to Figure 3. (Left) Memory usage for computing the meta-gradients as a function of the number of inner-gradient steps. The extrapolated dashed lines correspond to the method reaching the memory capacity of a Tesla V100 GPU with 32Gb of memory. (Right) Average time taken (in ms) compute the exact meta-gradients. The extrapolated dashed lines correspond the method taking over 3 seconds.
Evolution of
Effect of leamiig T
70
COMEN
10a
60
Accuracy
10a
50
10a
mintImageNet
Net
10-1
I-shot
w-way S-shot
0.2
0.4
0.6
0.8
1
10" 10
10
10
Meta-training
T
Figure 5: (Left) Evolution the mntta-parameter controlling the amount adaptation necessary for all tasks during meta-training. Here, the backbone is ResNet-12 12. We normalized the duration of meta-training in [0, 1] to account for early stopping; typically the model for minilmageNet requires an order of magnitude fewer iterations. (Right) Comparison of COMLN (where T is learned, in red) to meta-learning with a fixed length of adaptation T' (in blue), on a 5-shot 5-way classification problem on the minilmageNee dataset (with Conv-4 backbone).
In order to ensure fair comparison between methods that rely on discrete number of gradient steps (MAML, ANIL, and iMAML), and COMLN which relies on a continuous integration time T. we added conversion between the number of gradient steps and T. This corresponds to taking learning rate of 0.01 in (2) (which is standard practice for MAML and ANIL on minilmageNet). This means that the number of gradient steps is 100x larger than T. This can be formally justified by considering an explicit Euler scheme for COMLN (see Section 3.3), with constant step size 0.01. The memory requirements is independent of the choice of the numerical solver. For the computation time, this correspondence between T and the number of gradient steps is no longer exact for COMLN (RK)-the the comparison with COMLN (Euler) is still valid though.
E.3 ANALYSIS OF THE LEARNED HORIZON
Since one of the advantage of COMLN compared to other gradient-based methods its capacity to learn the amount of adaptation through the time horizon T, Figure 5 (left) shows the evolution of this meta parameer during meta-tres trainnr.. We can observe that for the more complex dataset longer sequence of adaptation. Similarly
tieredlmageNet, COMLN appropriately learns
30

---

Published as a conference paper at ICLR 2022
Table 5: minilmageNet results using LEO embeddings and single linear classifier layer. The average
accuracy (%) on 1,000 held-out meta-test tasks is reported with 95% confidence interval. Results reported in (Rusu et al., 2018). *e Note that LEO uses more than a single linear classifier layer, but we add the numbers for completeness.
minilmageNet 5-way
Model
1-shot
5-shot
MAML (Finn al., 2017)
50.35 + 0.63
65.28 + 0.54
Meta-SGD" (Li et al., 2017)
54.24 0.03
70.86 - 0.04
Meta-SGD (Li et al., 2017)
50.57 0.64
69.09 0.53
iMAML (Rajeswaran et al., 2019)
5.62
69.52 + 0.51
R2D2 (Bertinetto et al., 2018)
50.33
70.38 0.52
LRD2 (Bertinetto et al., 2018)
50.41
70.29 + 0.52
LEAP (Flennerhag et al., 2018)
50.95 0.62
66.72 0.55
MetaOptNet (Lee et al., 2019)
40.60 + 0.60
50.94 + 0.62
LEORR (Rusu al., 2018)
61.76 0.08
77.59 + 0.12
COMEN (Ours)
50.39 + 0.63
70.06 + 0.52
within each dataset, it also learns to use shorter sequences of adaptation of 1-shot problems, possibly to allow for better generalization and to reduce overfitting.
Besides adapting the amount of adaptation to the problem at hand, learning T also has the advantage of saving computation while reaching high levels of performance. If we were to fix T ahead of meta-training (as is typically the case in gradient-basee meta-learning, where the number of gradient steps for adaptation is a hyperparameter) to a large value in order to reach high accuracy, as is shown in Figure 5 (right), then would induce larger computational costs early during meta-training compared to COMLN, which achieves equal performance while tuning the value of T. In COMLN, the value of T is relatively small at the beginning of -eta-traininn.
E.4 ADDITIONAL EXPERIMENT: PREPROCESSED miniIMAGENET DATASET
In addition to our experiments on the minilmageNet and tieredImageNet datasets in Section 5, we want to evaluate the performance of continuous-time adaptation in isolation from learning the embedding network. We evaluate this using preprocessed version of minilmageNet introduced in Rusu et al. (2018), where the embeddings were trained using a Wide Residual Network (WRN) via supervised classification on the meta-trane train set. For our purposes, we consider these embeddings & E R2600 as fixed, and only meta-learn the initial conditions W, as well as T.
The classification accuracies for COMLN and other baselines using pretrained embeddings shown in Table 5. COMLN achieves comparable or better performance with single linear classifier layer as other meta-learning methods in both the 5-way 1-shot and 5-way 5-shot classification tasks. The single exception is the 5-way 1-shot result of Meta-SGD from Rusu et al. (2018), which exceeded the performance of COMLN, However, our implementation of Meta-SGD achieved comparable performance to COMLN. This gap in performance is likely due to the additional data used in Rusu al. (2018) (meta-trane train and meta-validation splits) during meta- meta-training. as opposed to using only the meta-training set for all other baselines. These results show that isolated from representation learning, all these meta-learning algorithms (either gradient-based or not) perform similarly, and COMLN is exception.
One notable exception though is MetaOptNet (Lee et al., 2019), where the performance is not as high as the other baselines when the backbone network is not learned mesporeddeppie often being the best performing model in Table 2. Our hypothesis is that this diserepancy is due to the accuracy of the QP solver used in MetaOptNet, since learning individual SVMs on 1,000 held-out meta-test tasks leads to performance matching all other methods (about 50% for 5-way 1-shot and about 70% for 5-way 5-shot).
31

---

Published as a conference paper at ICLR 2022
E.5 EXPERIMENTAL DETAILS
For all methods and all datasets, we used SGD with momentum 0.9 and Nesterov acceleration, with decreasing learning starting 0.1 and decreasing according the schedule provided by Lee et al. (2019). For meta-training, we followed the standard procedure in gradient- based meta-learning. and meta-trained with fixed number of shots: for example in minilmageNet 5-shot 5-way, we only used tasks with k = 5 training examples for each of the N = 5 classes. This contrasts with Lee al. (2019), which uses larger number of shots during meta-training than the one used for evaluation (e.g. meta-training with & = 15, and evaluating on k 1). This may explain the gap in performance between COMLN and MetaOptNet, especially on I-shot settings. We opted to not follow this decision made by Lee et al. (2019) ensure fair comparison with other gradient-based methods, which all used the process described above.
Conv-4 backbone We used standard convolutional neural network with 4 convolutional blocks (Finn et al., 2017). Each block consists of a convolutional layer with 3 x 3 kernel and 64 channels, followed by a batch normalization layer, and max-pooling layer with window size and stride 2 The activation function is ReLU.
ResNet-12 backbone We largely followed the architecture from Lee al. (2019), which consists of 12-layer residual network. The neural network is composed of 4 blocks with residual connections 3 convolutional layers with k 64, 160, 320 and 640 channels respectively. The non-linearity functions and a max- mnax-pooling layer with window size and stride 2 x is applied at the end of each block. No global pooling is performed the end of the embedding network, meaning that the embedding dimension d = 16,000. The only notable difference with the architecture used by Lee et al. (2019) the absence of DropBlock (Ghiasi et al., 2018) regularization.
kernel. The convolutional layers in the residual block have
LEAKYRELU(O.)
32


Reviews:
[{"review_id": "wX-4bmMVvLv", "reviewer": "Reviewer_nRbR", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The paper is clearly written, well organized and easy to follow\n- I believe the method proposed is an interesting addition to the large family of meta-learning algorithms that target the (by-now-classical) few-shot learning setting.\n- The possibility of seamlessly treating also the adaptation horizon as a meta-parameter opens up interesting avenues (which however are left as future work) \n\nWeaknesses:\n1. I believe that the main weakness of the proposed approach is that it is quite specific to the setting treated in the paper, mainly few-shot classification with a small number of classes, where the loss is (unregularized) cross-entropy. In my view, the main bottleneck is runtime rather than memory, since forward-mode differentiation, not required to store the intermediate states, can always be implemented in a way that is efficient in memory. In time, however, the computation generally grows with the number of meta-parameters. The authors devise a clever decomposition to avoid most of the burden (which, however, still scales with the number of examples and classes), but fail to discuss the limitations of this approach (regarding the efficiency of the gradient computation). E.g., what if $\\mathcal{L}$ is a generic function of the parameters? what if at the base level one wants to adapt also the rest of the parameters of the feature extractor or the bias of a regularizer [1]? In my view, the method remains interesting notwithstanding these potential limitations, but limitations should be clearly mentioned and discussed. Furthermore, I would like to see included in the paper also a runtime column in Table 1, to give a more complete picture of the complexity of the algorithms.\n2. In my view, one of the most interesting novelties of this work is the possibility of treating the adaptation time as a meta-parameter. I believe that the paper in its current stage misses on the opportunity of exploiting this. On the one side, this is because of the lack of novelty in the experimental validation, on the other side, this could be potentially linked to the limitations of the method: longer adaptation would be extremely beneficial in the presence of tasks with a different number of examples, however, since the memory (but also the runtime I believe) scales with the number of example, the complexity of  COMLN could be too high in these settings. Could the authors comment on this?\n3. Related work could be better mentioned. For instance, the idea of only learning base classifiers is also present in [2,3,4] and not only in ANIL. Furthermore, I believe that the distinction between so-called gradient-based and non-gradient based methods is misleading since also MetaOptNet uses gradients (the difference being that there they have a closed-form expression for those, but this is not at all dissimilar to iMAML).\n\nMinor comments:\n4. Clarity: Missing formal definition of the sensitivity state $\\mathcal{S}(t)$. The loss $\\mathcal{L}$ is used with different inputs: cf (2) and (8).\n\n[1] Denevi, Giulia, et al. \"Learning-to-learn stochastic gradient descent with biased regularization.\" International Conference on Machine Learning. PMLR, 2019.\n[2] Franceschi, Luca, et al. \"Bilevel programming for hyperparameter optimization and meta-learning.\" International Conference on Machine Learning. PMLR, 2018.\n[3]  Bertinetto, Luca, et al. \"Meta-learning with differentiable closed-form solvers.\" ICLR, 2019.\n[4]  Lee, Kwonjoon, et al. \"Meta-learning with differentiable convex optimization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. \n\n--------------------------------------------------\n**Post rebuttal:** I thank the authors for their reply. I appreciate the derivations for the MSE and the regularized CE and the clarifications regarding the extensibility to the general case. I hope that these comments will be included in the final version. On the other hand, I am still convinced that including runtimes in Table 1 is quite important to convey a clearer picture (you could add 2+ lines for COML depending on which ODE solver is used). Regarding standard forward mode, please note that since we are discussing asymptotic behaviour the asymptotic runtime complexity of the two implementations is the same, while clearly, one implementation is more memory efficient than the other (this does not mean that the actual GPU/CPU runtime will be the same, but again, Table 2 reports $O$'s..) \nFurthermore, I still believe this work misses to capitalize on its main novelty, i.e. meta-learning task-dependent $T$'s, leaving somewhat the doubt that this is either impractical, or does not bring noticeable advantages. For these reasons, while I am overall positive about this work, I will leave my score unvaried.", "summary_of_the_paper": "The paper describes an algorithm called COMLN for few-shot meta-learning. \nIn COMLN, the base level loop is modelled as a continuous-time autonomous ODE that is the gradient vector field of the inner loss. \nThe authors restrict their discussion to a meta-learning setting where the only base-level parameters are the parameters of a linear classifier. The meta-parameters are the parameters of a neural network that acts as feature extractor, the initialization point for adaptation of the classifier and the adaptation time $T$. \nThe authors point out that, crucially, it is possible to treat $T$ as a learnable meta-parameter thanks to the continuous reformulation of the base level.\nMeta-training is carried out by gradient descent on the meta-parameter, where the gradients are computed with an efficient implementation (in few-shot classification, with a small number of classes) forward mode differentiation that exploits the loss structure.\nThe authors present experiments on standard benchmark datasets, showing that COMLN achieves better performances than other comparable gradient-based methods. ", "summary_of_the_review": "The paper is well-crafted and the method presented is an interesting addition to the few-shot meta-learning literature, with possibly some limitations that are not well discussed. Furthermore, the paper misses exploiting what probably is the main advantage of the method (meta-learning task-specific, or conditional, $T$'s). These two issues currently limit my score to a weak accept.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "d_YisKjPOZH", "reviewer": "Reviewer_PyBo", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n  - Interesting use of forward-mode differentiation and structure of the gradients to compute meta-gradients efficiently.\n  - Learning T as an early stopping is pretty interesting, and T will be increased if the train and test gradients are aligned, and decreased otherwise. (IMO, the authors can expand on interpreting this alignment a bit more after Eq 13.)\n\nCons:\n  - IMO, the claims/explanations for continuous-time and forward-mode differentiation should be separated. SGD simply corresponds to an Euler discretization, and the same forward-mode tricks can be applied just as readily in the discrete-time setting as the continuous-time setting. Perhaps the only strictly continuous-time-related contribution is being able to learn T.\n  - I don't have any major worries, but see my comments below.\n\nComments:\n  - \"Gradient-based Hyperparameter Optimization Over Long Horizon\" discusses efficient forward-mode differentiation for discrete-time meta-learning. which should be worth discussing.\n  - Since forward-mode can also be applied in the discrete-time setting, the main motivation for continuous-time seems to be being able to learn T. However, I'm not entirely convinced about its usefulness: \n\n1. The gradient wrt T is very local.\n(i) How much does T change over the course of training? Is the learning useful or do you have to initialize it very well?\n(ii) How stable is the optimization? For instance, if you were to initialize with a very large or very small T value, is the gradient sufficient for learning a reasonable value for T?\n\n2. The interpretation of T:\n(i) Often meta-learning methods (for instance MAML) motivate the use of a fixed number of iterations to learn good meta-parameters that are useful even with a low compute budget (small number of gradient descent iterations). However, in the continuous-time version, changing T changes the amount of computation. If you have a fixed compute cost, would learning T still be useful? This might be a fairer comparison to discrete-time methods as well.\n\n - Can you explain why COMLN is more expensive with the Euler integration scheme compared to RK? Are you using different step sizes for these methods?", "summary_of_the_paper": "This paper proposes using a continuous-time formulation along with forward-mode differentiation to perform meta-learning. The setup follows that of ANIL: the weights of the last layer is part of the inner loop optimization while the initialization of these last-layer weights and the rest of the weights are treated as meta-parameters. The proposed method relies on this setup in order to efficiently compute derivatives, which the authors note have a low-rank structure that can be exploited. Furthermore, the weights do not need to be solved over time, and instead a much smaller vector can be solved instead, which is pretty neat. ", "summary_of_the_review": "The approach is interesting; the main novelty is being able to train T, kind of like a learning early stopping. However, T also increases compute cost, which may not be a completely fair comparison to fixed-cost discrete-time approaches. Another minor concern is the lack of control for compute cost due to the training of T.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Qsd_MopBrWP", "reviewer": "Reviewer_Bwjf", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n* The forward-mode differentiation is well-justified. Good discussion on the alternatives and why they are not feasible for the gradient-based meta-learning case.\n* The derivation for the vector-Jacobian product in forward-mode differentiation without explicitly constructing the Jacobian is novel and can greatly reduce memory cost in the few-shot setting.\n* The adaptation w.r.t. T is interesting, and only possible in the continuous-time formulation. However, I have a few questions about this (see “Questions” section below).\n\nWeaknesses:\n* The memory cost scales with the number of data points available. This is not desirable as we would usually expect the performance to improve with more data points, instead of having to limit the amount of data due to constraints on computation resources.\n\n\nQuestions:\n* Adapting both T and W0 sounds like two conflicting things to do. When T is adapted to be very large, won’t it drive the gradient w.r.t. W0 to zero (i.e. inner loop is allowed to converge, making W_T independent of initialization)? Does that match your observations in the experiments, and what are the justifications for adapting both? I imagine this conflicting adaptation would complicate the learning dynamics, and may be affected by how you choose the meta optimizers (e.g. if you use a different meta learning rate for T and W0, it might lead to different behaviours).\n* What makes COMLN achieve higher accuracy than other gradient-based meta-optimizers? Is it because of the adaptive T? I’d like to see some ablation experiments about this.\n* In the related work section, you wrote “Zhang et al. (2021) also introduce a formulation where the adaptation of prototypes follows a gradient vector field, but they finally opt for modeling it as a Neural ODE, possibly due to the challenges of applying the adjoint method we identified in Section 3.2”. From my understanding, it’s precisely due to the numerical instability of the adjoint method that you **don’t use** Neural ODE, and opt for forward-mode differentiation instead. Could you clarify that in more detail?\n\nFeedback for improvements:\n* Should further discuss why you choose to focus on gradient-based meta optimizers specifically, what are the pros and cons compared to non-gradient-based ones (like MetaOptNet).\n\nA couple places where \\citet should be used instead of \\citep:\n * Page 7, third line under “experiments” section: Lee et al., 2019.\n * Page 9, third line of the paragraph before Section 7: Chen et al., 2020a.\n", "summary_of_the_paper": "The authors propose COMLN -- a gradient-based meta-learning algorithm with continuous-time inner loop adaptation. The authors discusses possible options for computing the meta-gradient (backpropagation through time, solve ODE backwards in time, and forward-mode differentiation), and identifies the forward-mode differentiation as the only method that is both numerically stable and has a memory cost that is constant w.r.t. the inner-loop length. The authors then derive a way to compute the vector-Jacobian products without explicitly constructing the Jacobians in forward-mode differentiation (specifically for the few-shot learning setting), which further reduces the memory cost. Also, by using a continuous formulation for the inner-optimization, the authors are able to compute meta-gradient w.r.t. horizon T and adapt that as well. Finally, the authors conduct experiments on standard few-shot image classification benchmarks, and show that COMLN outperforms other gradient-based meta-optimization methods in accuracy with a lot less memory cost.", "summary_of_the_review": "I recommend weak accept. I think the novel derivation of memory-efficient forward-mode differentiation is a significant contribution in the few-shot learning context. I would consider raising the score if my questions about adapting T are addressed.\n\n=== Post rebuttal ===\n\nThe authors have addressed all my concerns. I increased the score to 8 (accept).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "57QK-YeHsQA", "reviewer": "Reviewer_Us8H", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The use of continuous time formulation for the inner-loop adaptation step, while also being able to learn the length of time this adaptation should be simulated is an approach that promises to be quite powerful. Combining this with an efficient forward-mode algorithm for learning the parameters of this inner-adaptation, and meta-learning representations is overall a very significant and novel combination.\n\nThe writing is clear and the overall algorithm is well explained.\nOne issue I spotted: \n- in eqn. 11, A is not defined\n\nThe empirical evaluation of the approach is also thorough, as is the evaluation of computation and memory usage (which the proposed forward-mode approach is targeted towards).\n\nThe discussion of the related work seems complete to my knowledge.\n\nThe one major weakness of this approach (specifically the efficient forward mode differentiation method) seems to be that it cannot be used for non-cross-entropy losses. Are there alternative ways to achieve the same thing there?", "summary_of_the_paper": "In this paper, the authors propose a continuous-time formulation for the inner-loop adaptation in a MAML-like setup. Moreover, the authors propose an efficient forward-mode algorithm to learn the parameters of this inner-loop adaptation that is independent of the number of adaptation steps used. The authors show that this works well for a variety of meta-learning problems.", "summary_of_the_review": "Overall a very strong paper with novel and significant contributions without major flaws.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses a continuous-time formulation of gradient-based meta-learning (COMLN) where the adaptation is the solution of a differential equations. In general, outer loop optimization requires backpropagating over trajectories involving gradient updates in the inner loop optimization. It is claimed that one of main advantages of COMLN is able to compute the exact meta-gradients in a memory-efficient way, regardless of the length of adaptation trajectory. To this end, the forward-mode differentiation is used, with exploiting the Jacobian matrix decomposition. All the reviewers agree that the derivation of memory-efficient forward-mode differentiation is a significant contribution in the few-shot learning. The paper is well written and has interesting contributions. Authors did a good job in responding to reviewers’ comments during the discussion period. What is missing in this paper is the discussion of some limitations of the proposed method.  This can be improved in the final version. All reviewers agree to champion this paper. Congratulations on a nice work.
Decision: Accept (Spotlight)