Year: 2021

Abstract:
Drawing inspiration from gradient-based meta-learning methods with infinitely small gradient steps, we introduce Continuous-Time Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of a gradient vector field. Specifically, representations of the inputs are meta-learned such that a taskspecific linear classifier is obtained as a solution of an ordinary differential equation (ODE). Treating the learning process as an ODE offers the notable advantage that the length of the trajectory is now continuous, as opposed to a fixed and discrete number of gradient steps. As a consequence, we can optimize the amount of adaptation necessary to solve a new task using stochastic gradient descent, in addition to learning the initial conditions as is standard practice in gradient-based meta-learning. Importantly, in order to compute the exact meta-gradients required for the outer-loop updates, we devise an efficient algorithm based on forward mode differentiation, whose memory requirements do not scale with the length of the learning trajectory, thus allowing longer adaptation in constant memory. We provide analytical guarantees for the stability of COMLN, we show empirically its efficiency in terms of runtime and memory usage, and we illustrate its effectiveness on a range of few-shot image classification problems.

heading: 1 INTRODUCTION
text: Among the existing meta-learning algorithms, gradient-based methods as popularized by ModelAgnostic Meta-Learning (MAML, Finn et al., 2017) have received a lot of attention over the past few years. They formulate the problem of learning a new task as an inner optimization problem, typically based on a few steps of gradient descent. An outer meta-optimization problem is then responsible for updating the meta-parameters of this learning process, such as the initialization of the gradient descent procedure. However since the updates at the outer level typically require backpropagating through the learning process, this class of methods has often been limited to only a few gradient steps of adaptation, due to memory constraints. Although solutions have been proposed to alleviate the memory requirements of these algorithms, including checkpointing (Baranchuk, 2019), using implicit differentiation (Rajeswaran et al., 2019), or reformulating the meta-learning objective (Flennerhag et al., 2018), they are generally either more computationally demanding, or only approximate the gradients of the meta-learning objective (Nichol et al., 2018; Flennerhag et al., 2020).
In this work, we propose a continuous-time formulation of gradient-based meta-learning, called Continuous-Time Meta-Learning (COMLN), where the adaptation is the solution of a differential equation (see Figure 1). Moving to continuous time allows us to devise a novel algorithm, based on forward mode differentiation, to efficiently compute the exact gradients for meta-optimization, no matter how long the adaptation to a new task might be. We show that using forward mode differentiation leads to a stable algorithm, unlike the counterpart of backpropagation in continuous time called the adjoint method (frequently used in the Neural ODE literature) which tends to be unstable with gradient vector fields. Moreover as the length of the adaptation trajectory is a continuous quantity, as opposed to a discrete number of gradient steps fixed ahead of time, we can treat the amount of adaptation in COMLN as a meta-parameter—on par with the initialization—which we can meta-optimize using stochastic gradient descent. We verify empirically that our method is both computationally and memory efficient, and we show that COMLN outperforms other standard meta-learning algorithms on few-shot image classification datasets.

heading: 2 BACKGROUND
text: In this work, we consider the problem of few-shot classification, that is the problem of learning a classification model with only a small number of training examples. More precisely for a classification task τ , we assume that we have access to a (small) training dataset Dtrainτ = {(xm,ym)}Mm=1 to fit a model on task τ , and a distinct test dataset Dtestτ to evaluate how well this adapted model generalizes on that task. In the few-shot learning literature, it is standard to consider the problem of k-shot N -way classification, meaning that the model has to classify among N possible classes, and there are only k examples of each class in Dtrainτ , so that overall the number of training examples is M = kN . We use the convention that the target labels ym ∈ {0, 1}N are one-hot vectors.

heading: 2.1 GRADIENT-BASED META-LEARNING
text: Gradient-based meta-learning methods aim to learn an initialization such that the model is able to adapt to a new task via gradient descent. Such methods are often cast as a bi-level optimization process: adapting the task-specific parameters θ in the inner loop, and training the (task-agnostic) meta-parameters Φ and initialization θ0 in the outer loop. The meta-learning objective is:
min θ0,Φ
Eτ [ L(θτT ,Φ;Dtestτ ) ] (1)
s.t. θτt+1 = θ τ t − α∇θL(θτt ,Φ;Dtrainτ ) θτ0 = θ0 ∀τ ∼ p(τ), (2)
where T is the number of inner loop updates. For example, in the case of MAML (Finn et al., 2017), there is no additional meta-parameter other than the initialization (Φ ≡ ∅); in ANIL (Raghu et al., 2019), θ are the parameters of the last layer, and Φ are the parameters of the shared embedding network; in CAVIA (Zintgraf et al., 2019), θ are referred to as context parameters.
During meta-training, the model is trained over many tasks τ . The task-specific parameters θ are learned via gradient descent on Dtrainτ . The meta-parameters are then updated by evaluating the error of the trained model on the test dataset Dtestτ . At meta-testing time, the meta-trained model is adapted on Dtrainτ —i.e. applying Equation 2 with the learned meta-parameters θ0 and Φ.

heading: 2.2 LOCAL SENSITIVITY ANALYSIS OF ORDINARY DIFFERENTIAL EQUATIONS
text: Consider the following (autonomous) Ordinary Differential Equation (ODE):
dz dt = g ( z(t);θ ) z(0) = z0(θ), (3)
where the dynamics g and initial value z0 may depend on some external parameters θ, and integration is carried out from 0 to some time T . Local sensitivity analysis is the study of how the solution of this dynamical system responds to local changes in θ; this effectively corresponds to calculating the derivative dz(t)/dθ. We present here two methods to compute this derivative, with a special focus on their memory efficiency.
Adjoint sensitivity method Based on the adjoint state (Pontryagin, 2018), and taking its root in control theory (Lions & Magenes, 2012), the adjoint sensitivity method (Bryson & Ho, 1969; Chavent et al., 1974) provides an efficient approach for evaluating derivatives of L ( z(T );θ ) , a function of
z(T ) the solution of the ODE in Equation 3. This method, popularized lately by the literature on Neural ODEs (Chen et al., 2018), requires the integration of the adjoint equation
da dt = −a(t)
∂g ( z(t);θ ) ∂z(t)
a(T ) = dL ( z(T );θ ) dz(T ) , (4)
backward in time. The adjoint sensitivity method can be viewed as a continuous-time counterpart to backpropagation, where the forward pass would correspond to integrating Equation 3 forward in time from 0 to T , and the backward pass to integrating Equation 4 backward in time from T to 0.
One possible implementation, reminiscent of backpropagation through time (BPTT), is to store the intermediate values of z(t) during the forward pass, and reuse them to compute the adjoint state during the backward pass. While several sophisticated checkpointing schemes have been proposed (Serban & Hindmarsh, 2003; Gholami et al., 2019), with different compute/memory trade-offs, the memory requirements of this approach typically grow with T ; this is similar to the memory limitations standard gradient-based meta-learning methods suffer from as the number of gradient steps increases. An alternative is to augment the adjoint state a(t) with the original state z(t), and to solve this augmented dynamical system backward in time (Chen et al., 2018). This has the notable advantage that the memory requirements are now independent of T , since z(t) are no longer stored during the forward pass, but they are recomputed on the fly during the backward pass.
Forward sensitivity method While the adjoint method is related to reverse-mode automatic differentiation (backpropagation), the forward sensitivity method (Feehery et al., 1997; Leis & Kramer, 1988; Maly & Petzold, 1996; Caracotsios & Stewart, 1985), on the other hand, can be viewed as the continuous-time counterpart to forward (tangent-linear) mode differentiation (Griewank & Walther, 2008). This method is based on the fact that the derivative S(t) , dz(t)/dθ is the solution of the so-called forward sensitivity equation
dS dt
= ∂g ( z(t);θ ) ∂z(t) S(t) + ∂g ( z(t);θ ) ∂θ
S(0) = ∂z0 ∂θ . (5)
This equation can be found throughout the literature in optimal control and system identification (Betts, 2010; Biegler, 2010). Unlike the adjoint method, which requires an explicit forward and backward pass, the forward sensitivity method only requires the integration forward in time of the original ODE in Equation 3, augmented by the sensitivity state S(t) with the dynamics above. The memory requirements of the forward sensitivity method do not scale with T either, but it now requires storing S(t), which may be very large; we will come back to this problem in Section 3.2. We will simply note here that in discrete-time, this is the same issue afflicting forward-mode training of RNNs with real-time recurrent learning (RTRL; Williams & Zipser, 1989), or other meta-learning algorithms (Sutton, 1992; Franceschi et al., 2017; Xu et al., 2018).

heading: 3 CONTINUOUS-TIME ADAPTATION
text: In the limit of infinitely small steps, some optimization algorithms can be viewed as the solution trajectory of a differential equation. This point of view has often been taken to analyze their behavior (Platt & Barr, 1988; Wilson et al., 2016; Su et al., 2014; Orvieto & Lucchi, 2019). In fact, some optimization algorithms such as gradient descent with momentum have even been introduced initially from the perspective of dynamical systems (Polyak, 1964). As the simplest example, gradient descent with a constant step size α→ 0+ (i.e. α tends to 0 by positive values) corresponds to following the dynamics of an autonomous ODE called a gradient vector field
zt+1 = zt − α∇f(zt) −→ α→0+
dz dt = −∇f
( z(t) ) , (6)
where the iterate z(t) is now a continuous function of time t. The solution of this dynamical system is uniquely defined by the choice of the initial condition z(0) = z0.
W(0)
W(T)
W*
Forward Backward

heading: 3.1 CONTINUOUS-TIME META-LEARNING
text: In gradient-based meta-learning, the task-specific adaptation with gradient descent may also be replaced by a gradient vector field in the limit of infinitely small steps. Inspired by prior work in meta-learning (Raghu et al., 2019; Javed & White, 2019), we assume that an embedding network fΦ with meta-parameters Φ is shared across tasks, and only the parametersW of a task-specific linear classifier are adapted, starting at some initializationW0. Instead of being the result of a few steps of gradient descent though, the final parametersW (T ) now correspond to integrating an ODE similar to Equation 6 up to a certain horizon T , with the initial conditionsW (0) = W0. We call this new meta-learning algorithm Continuous-Time Meta-Learning1 (COMLN).
Treating the learning algorithm as a continuous-time process has the notable advantage that the adapted parameterW (T ) is now differentiable wrt. the time horizon T (Wiggins, 2003, Chap. 7), in addition to being differentiable wrt. the initial conditionsW0—which plays a central role in gradientbased meta-learning, as described in Section 2.1. This allows us to view T as a meta-parameter on par with Φ andW0, and to effectively optimize the amount of adaptation using stochastic gradient descent (SGD). The meta-learning objective of COMLN can be written as
min Φ,W0,T
Eτ [ L ( Wτ (T ); fΦ(Dtestτ ) )] (7)
s.t. dWτ dt
= −∇L ( Wτ (t); fΦ(Dtrainτ ) ) Wτ (0) = W0 ∀τ ∼ p(τ), (8)
where fΦ(Dtrainτ ) = {(fΦ(xm),ym) | (xm,ym) ∈ Dtrainτ } is the embedded training dataset, and fΦ(Dtestτ ) is defined similarly for Dtestτ . In practice, adaptation is implemented using a numerical integration scheme based on an iterative discretization of the problem, such as Runge-Kutta methods. Although a complete discussion of numerical solvers is outside of the scope of this paper, we recommend (Butcher, 2008) for a comprehensive overview of numerical methods for solving ODEs.

heading: 3.2 THE CHALLENGES OF OPTIMIZING THE META-LEARNING OBJECTIVE
text: In order to minimize the meta-learning objective of COMLN, it is common practice to use (stochastic) gradient methods; that requires computing its derivatives wrt. the meta-parameters, which we call meta-gradients. Our primary goal is to devise an algorithm whose memory requirements do not scale with the amount of adaptation T ; this would contrast with standard gradient-based meta-learning methods that backpropagate through a sequence of gradient steps (similar to BPTT), where the intermediate parameters are stored during adaptation (i.e. θτt for all t in Equation 2). Since this objective involves the solution W (T ) of an ODE, we can use either the adjoint method, or the forward sensitivity method, in order to compute the derivatives wrt. Φ andW0 (see Section 2.2).
Although the adjoint method has proven to be an effective strategy for learning Neural ODEs, in practice computing the state W (t) backward in time is numerically unstable when applied to a gradient vector field like the one in Equation 8, even for convex loss functions. Figure 2 shows an example where the trajectory ofW (t) recomputed backward in time (in red) diverges significantly from the original trajectory (in green) on a quadratic loss function, even though the two should match exactly in theory since they follow the same dynamics. Intuitively, recomputingW (t) backward in time for a gradient vector field requires doing gradient ascent on the loss function, which is prone to compounding numerical errors; this is closely related to the loss of entropy observed by Maclaurin
1COMLN is pronounced chameleon
et al. (2015). This divergence makes the backward trajectory ofW (t) unreliable to find the adjoint state, ruling out the adjoint sensitivity method for computing the meta-gradients in COMLN.
The forward sensitivity method addresses this shortcoming by avoiding the backward pass altogether. However, it can also be particularly expensive here in terms of memory requirements, since the sensitivity state S(t) in Section 2.2 now corresponds to Jacobian matrices, such as dW (t)/dW0. As the size d of the feature vectors returned by fΦ may be very large, this Nd×Nd Jacobian matrix would be almost impossible to store in practice; for example in our experiments, it can be as large as d = 16,000 for a ResNet-12 backbone. In Section 4.1, we will show how to apply forward sensitivity in a memory-efficient way, by leveraging the structure of the loss function. This is achieved by carefully decomposing the Jacobian matrices into smaller pieces that follow specific dynamics. We show in Appendix D that unlike the adjoint method, this process is stable.

heading: 3.3 CONNECTION WITH ALMOST NO INNER-LOOP (ANIL)
text: Similarly to ANIL (Raghu et al., 2019), COMLN only adapts the parameters W of the last linear layer of the neural network. There is a deeper connection between both algorithms though: while our description of the adaptation in COMLN (Eq. 8) was independent of the choice of the ODE solver used to find the solution W (T ) in practice, if we choose an explicit Euler scheme (Euler, 1913, roughly speaking, discretizing Equation 6 from right to left), then the adaptation of COMLN becomes functionally equivalent to ANIL. However, this equivalence can greatly benefit from the memory-efficient algorithm to compute the meta-gradients described in Section 4, based on the forward sensitivity method. This means that using the methods devised here for COMLN, we can effectively compute the meta-gradients of ANIL with a constant memory cost wrt. the number of gradient steps of adaptation, instead of relying on backpropagation (see also Section 4.2).

heading: 4 MEMORY-EFFICIENT META-GRADIENTS
text: For some fixed task τ and (xm,ym) ∈ Dtrainτ , let φm = fΦ(xm) ∈ Rd be the embedding of input xm through the feature extractor fΦ. Since we are confronted with a classification problem, the loss function of choice L(W ) is typically the cross-entropy loss. Böhning (1992) showed that the gradient of the cross-entropy loss wrt. W can be written as
∇L ( W ; fΦ(Dtrainτ ) ) = 1
M M∑ m=1 (pm − ym)φ>m, (9)
where pm = softmax(Wφm) is the vector of probabilities returned by the neural network. The key observation here is that the gradient can be decomposed as a sum of M rank-one matrices, where the feature vectors φm are independent ofW . Therefore we can fully characterize the gradient of the cross-entropy loss with M vectors pm − ym ∈ RN , as opposed to the full N × d matrix. This is particularly useful in the context of few-shot classification, where the number of training examples M is small, and typically significantly smaller than the embedding size d.

heading: 4.1 DECOMPOSITION OF THE META-GRADIENTS
text: We saw in Section 3.2 that the forward sensitivity method was the only stable option to compute the meta-gradients of COMLN. However, naively applying the forward sensitivity equation would involve quantities that typically scale with d2, which can be too expensive in practice. Using the structure of Equation 9, the Jacobian matrices appearing in the computation of the meta-gradients for COMLN can be decomposed in such a way that only small quantities will depend on time.
Meta-gradients wrt. W0 By the chain rule of derivatives, it is sufficient to compute the Jacobian matrix dW (T )/dW0 in order to obtain the meta-gradient wrt. W0. We show in Appendix B.2 that the sensitivity state dW (t)/dW0 can be decomposed as:
dW (t)
dW0 = I − M∑ i=1 M∑ j=1 Bt[i, j]⊗ φiφ>j , (10)
where ⊗ is the Kronecker product, and eachBt[i, j] is an N ×N matrix, solution of the following system of ODEs2
dBt[i, j]
dt = 1(i = j)Ai(t)−Ai(t) M∑ m=1 (φ>i φm)Bt[m, j] B0[i, j] = 0, (11)
and Ai(t), defined in Appendix B.2, is also an N ×N matrix that only depends on W (t) and φi. The main consequence of this decomposition is that we can simply integrate the augmented ODE in{ W (t),Bt[i, j] } up to T to obtain the desired Jacobian matrix, along with the adapted parameters W (T ). Furthermore, in contrast to naively applying the forward sensitivity method (see Section 3.2), the M2 matricesBt[i, j] are significantly smaller than the full Jacobian matrix. In fact, we show in Appendix C that we can compute vector-Jacobian products—required for the chain rule—using only these smaller matrices, and without ever having to explicitly construct the full Nd×Nd Jacobian matrix dW (t)/dW0 with Equation 10.
Meta-gradients wrt. Φ To backpropagate the error through the embedding network fΦ, we need to first compute the gradients of the outer-loss wrt. the feature vectors φm. Again, by the chain rule, we can get these gradients with the Jacobian matrices dW (T )/dφm. Similar to Equation 10, we can show that these Jacobian matrices can be decomposed as:
dW (t)
dφm = −
[ sm(t)⊗ I + M∑ i=1 Bt[i,m]W0 ⊗ φi + M∑ i=1 M∑ j=1 zt[i, j,m]φ > j ⊗ φi ] , (12)
where sm(t) and zt[i, j,m] are vectors of size N , that follow some dynamics; the exact form of this system of ODEs, as well as the proof of this decomposition, are given in Appendix B.3. Crucially, the only quantities that depend on time are small objects independent of the embedding size d. Following the same strategy as above, we can incorporate these vectors in the augmented ODE, and integrate it to get the necessary Jacobians. Once all the dW (t)/dφm are known, for all the training datapoints, we can apply standard backpropagation through fΦ to obtain the meta-gradients wrt. Φ.
Meta-gradient wrt. T One of the major novelties of COMLN is the capacity to meta-learn the amount of adaptation using stochastic gradient descent. To compute the meta-gradient wrt. the time horizon T , we can directly borrow the results derived by Chen et al. (2018) in the context of Neural ODEs, and apply it to our gradient vector field in Equation 8 responsible for adaptation:
dL ( W (T ); fΦ(Dtestτ ) ) dT = − [ ∂L ( W (T ); fΦ(Dtestτ ) ) ∂W (T ) ]> ∂L(W (T ); fΦ(Dtrainτ )) ∂W (T ) . (13)
The proof is available in Appendix B.4. Interestingly, we find that this involves the alignment between the vectors of partial derivatives of the inner-loss and the outer-loss at W (T ), which appeared in different contexts in the meta-learning and the multi-task learning literature (Li et al., 2018; Rothfuss et al., 2019; Yu et al., 2020).
2Here we used the notation Bt[i, j] to make the dependence on t explicit, without overloading the notation. A more precise notation would be B[i, j](t).

heading: 4.2 MEMORY EFFICIENCY
text: Although naively applying the forward sensitivity method would be memory intensive, we have shown in Section 4.1 that the Jacobians can be carefully decomposed into smaller pieces. It turns out that even the parametersW (t) can be expressed using the vectors sm(t) from the decomposition in Equation 12; see Appendix B.1 for details. As a consequence, to compute the adapted parameters W (T ) as well as all the necessary meta-gradients, it is sufficient to integrate a dynamical system in{ Bt[i, j], sm(t), zt[i, j,m] } (see Algorithms 1 & 2 in App. A.1), involving exclusively quantities that are independent of the embedding size d. Instead, the size of that system scales with M the total number of training examples, which is typically much smaller than d for few-shot classification.
Table 1 shows a comparison of the memory cost for different algorithms. It is important to note that contrary to other standard gradient-based meta-learning methods, the memory requirements of COMLN do not scale with the amount of adaptation T (i.e. the number of gradient steps in MAML & ANIL), while still returning the exact meta-gradients—unlike iMAML (Rajeswaran et al., 2019), which only returns an approximation of the meta-gradients. We verified empirically this efficiency, both in terms of memory and computation costs, in Section 5.2.

heading: 5 EXPERIMENTS
text: For our embedding network fΦ, we consider two commonly used architectures in meta-learning: Conv-4, a convolutional neural network with 4 convolutional blocks, and ResNet-12, a 12-layer residual network (He et al., 2016). Note that following Lee et al. (2019), ResNet-12 does not include a global pooling layer at the end of the network, leading to feature vectors with embedding dimension d = 16, 000. Additional details about these architectures are given in Appendix E. To compute the adapted parameters and the meta-gradients in COMLN, we integrate the dynamical system described in Section 4.2 with a 4th order Runge-Kutta method with a Dormand Prince adaptive step size (Runge, 1895; Dormand & Prince, 1980); we will come back to the choice of this numerical solver in Section 5.2. Furthermore to ensure that T > 0, we parametrized it with an exponential activation.

heading: 5.1 FEW-SHOT IMAGE CLASSIFICATION
text: We evaluate COMLN on two standard few-shot image classification benchmarks: the miniImageNet (Vinyals et al., 2016) and the tieredImageNet datasets (Ren et al., 2018), both datasets being derived from ILSVRC-2012 (Russakovsky et al., 2015). The process for creating tasks follows the standard procedure from the few-shot classification literature (Santoro et al., 2016), with distinct classes
between the different splits. miniImagenet consists of 100 classes, split into 64 training classes, 16 validation classes, and 20 test classes. tieredImageNet consists of 608 classes grouped into 34 high-level categories from ILSVRC-2012, split into 20 training, 6 validation, and 8 testing categories—corresponding to 351/97/160 classes respectively; Ren et al. (2018) argue that separating data according to high-level categories results in a more difficult and more realistic regime.
Table 2 shows the average accuracies of COMLN compared to various meta-learning methods, be it gradient-based or not. For both backbones, COMLN decisively outperforms all other gradient-based meta-learning methods. Compared to methods that explicitly backpropagate through the learning process, such as MAML or ANIL, the performance gain shown by COMLN could be credited to the longer adaptation T it learns, as opposed to a small number of gradient steps—usually about 10 steps; this was fully enabled by our memory-efficient method to compute meta-gradients, which does not scale with the length of adaptation anymore (see Section 4.2). We analyse the evolution of T during meta-training for these different settings in Appendix E.3. In almost all settings, COMLN is even closing the gap with a strong non-gradient-based method like MetaOptNet; the remainder may be explained in part by the training choices made by Lee et al. (2019) (see Appendix E for details).

heading: 5.2 EMPIRICAL EFFICIENCY OF COMLN
text: In Section 4.2, we showed that our algorithm to compute the meta-gradients, based on forward differentiation, had a memory cost independent of the length of adaptation T . We verify this empirically in Figure 3, where we compare the memory required by COMLN and other methods to compute the meta-gradients on a single task, with a Conv-4 backbone (Figure 4 in Appendix E.2 shows similar results for ResNet-12). To ensure an aligned comparison between discrete and continuous time, we use a conversion corresponding to a learning rate α = 0.01 in Equation 2; see Appendix E.2 for a justification. As expected, the memory cost increases for both MAML and ANIL as the number of gradient steps increases, while it remains constant for iMAML and COMLN. Interestingly, we observe that the cost of COMLN is equivalent to the cost of running ANIL for a small number of steps, showing that the additional cost of integrating the augmented ODE in Section 4.2 to compute the meta-gradients is minimal.
Increasing the length of adaptation also has an impact on the time it takes to compute the adapted parameters, and the meta-gradients. Figure 3 (right) shows how the runtime increases with the amount of adaptation for different algorithms. We see that the efficiency of COMLN depends on the numerical solver used. When we use a simple explicit-Euler scheme, the time taken to compute the meta-gradients matches the one of ANIL; this behavior empirically confirms our observation in Section 3.3. When we use an adaptive numerical solver, such as Runge-Kutta (RK) with a Dormand Prince step size, this computation can be significantly accelerated, thanks to the smaller number of function evaluations. In practice, we show in Appendix E.1 that the choice of the ODE solver has a very minimal impact on the accuracy.

heading: 6 RELATED WORK
text: We are interested in meta-learning (Bengio et al., 1991; Schmidhuber, 1987; Thrun & Pratt, 2012), and in particular we focus on gradient-based meta-learning methods (Finn, 2018), where the learning rule is based on gradient descent. While in MAML (Finn et al., 2017) the whole network was updated during this process, follow-up works have shown that it is generally sufficient to share most parts of the neural network, and to only adapt a few layers (Raghu et al., 2019; Chen et al., 2020b; Tian et al., 2020). Even though this hypothesis has been challenged recently (Arnold & Sha, 2021), COMLN also updates only the last layer of a neural network, and therefore can be viewed as a continuous-time extension of ANIL (Raghu et al., 2019); see also Section 3.3. With its shared embedding network across tasks, COMLN is also connected to metric-based meta-learning methods (Vinyals et al., 2016; Snell et al., 2017; Sung et al., 2018; Bertinetto et al., 2018; Lee et al., 2019).
Closely related to our work, Zhang et al. (2021) also introduce a formulation where the adaptation of prototypes follows a gradient vector field, but they finally opt for modeling it as a Neural ODE (Chen et al., 2018), possibly due to the challenges of applying the adjoint method we identified in Section 3.2. Zhou et al. (2021) also uses a gradient vector field to motivate a novel method with a closed-form adaptation, based on the NTK theory; COMLN still explicitly updates the parameters following the gradient vector field, since there is no closed-form solution of Eq. 8. As mentioned in Section 3, treating optimization as a continuous-time process has been used to analyze the convergence of different optimization algorithms, including the meta-optimization of MAML (Xu et al., 2021), or to introduce new meta-optimizers based on different integration schemes (Im et al., 2019). Guo et al. (2021) also uses meta-learning to learn new integration schemes for ODEs. Although this is a growing literature at the intersection of meta-learning and dynamical systems, our work is the first algorithm that uses a gradient vector field for adaptation in meta-learning.
Beyond the memory efficiency of our method, one of the main benefits of the continuous-time perspective is that COMLN is capable of learning when to stop the adaptation, as opposed to taking a number of gradient steps fixed ahead of time. However unlike Chen et al. (2020a), where the number of gradient steps are optimized (up to a maximal number) with variational methods, we incorporate the amount of adaptation as a (continuous) meta-parameter that can be learned using SGD. To compute the meta-gradients, which is known to be challenging for long sequences in gradient-based meta-learning, we use forward-mode differentiation as an alternative to backpropagation through the learning process, similar to prior work in meta-learning (Franceschi et al., 2017; Jiwoong Im et al., 2021) and hyperparameter optimization over long horizons (Micaelli & Storkey, 2021). This yields the exact meta-gradients in constant memory, without any assumption on the optimality of the inner optimization problem, which is necessary when using the normal equations (Bertinetto et al., 2018), or to apply implicit differentiation (Rajeswaran et al., 2019).

heading: 7 CONCLUSION AND FUTURE WORK
text: In this paper, we have introduced Continuous-Time Meta-Learning (COMLN), a novel algorithm that treats the adaptation in meta-learning as a continuous-time process, by following the dynamics of a gradient vector field up to a certain time horizon T . One of the major novelties of treating adaptation in continuous time is that the amount of adaptation T is now a continuous quantity, that can be viewed as a meta-parameter and can be learned using SGD, alongside the initial conditions and the parameters of the embedding network. In order to learn these meta-parameters, we have also introduced a novel practical algorithm based on forward mode automatic differentiation, capable of efficiently computing the exact meta-gradients using an augmented dynamical system. We have verified empirically that this algorithm was able to compute the meta-gradients in constant memory, making it the first gradient-based meta-learning approach capable of computing the exact metagradients with long sequences of adaptation using gradient methods. In practice, we have shown that COMLN significantly outperforms other standard gradient-based meta-learning algorithms.
In addition to having a single meta-parameter T that drives the adaptation of all possible tasks, the fact that the time horizon can be learned with SGD opens up new possibilities for gradient-based methods. For example, we could imagine treating T not as a shared meta-parameters, but as a task-specific parameter. This would allow the learning process to be more adaptive, possibly with different behaviors depending on the difficulty of the task. This is left as a future direction of research.
REPRODUCIBILITY STATEMENT
We provide in Appendix A.1 a full description in pseudo-code of the meta-training procedure (Algorithm 1), along with the exact dynamics of the ODE (Algorithm 2) and the projection operations (Algorithms 3 & 4) to avoid explicitly building the Jacobian matrices to compute Jacobian-vector products (see Section 4.1).
We also provide in Appendix A.2 a snippet of code in JAX (Bradbury et al., 2018) to compute the adapted parameters W (T ), as well as all the necessary objects { Bt[i, j], sm(t), zt[i, j,m] } to compute all the meta-gradients (see Section 4.2). We also give in Code Snippet 2 the code to compute the meta-gradients wrt. the initialization W0 and the integration time T . Computing the meta-gradients wrt. Φ involves non-minimal dependencies on Haiku (Hennigan et al., 2020), and therefore is omitted here. The full code is available in the Supplementary Materials.
Data generation & hyperparameters We used the miniImageNet and tieredImageNet datasets provided by Lee et al. (2019) in order to create the 1-shot 5-way and 5-shot 5-way tasks for both datasets. During evaluation, for each setting, a fixed set of 1, 000 tasks were sampled; this means that both architectures for COMLN have been evaluated using exactly the same data, to ensure direct comparison across backbones. A full description of all the hyperparameters used in COMLN is given in Appendix E.
Reproducibility of baseline results To the best of our ability, we have tried to report baseline results from existing work, to limit as much as possible the bias induced by running our own baseline experiments. The references of those works are given in Table 3. We still had to run CAVIA and iMAML on the remaining settings, since these results have not been reported in the literature. For both methods, we used the data generation described above.
• CAVIA: We used the official implementation3. We used the hyperparameters reported in (Zintgraf et al., 2019) for miniImageNet, and an architecture with 64 filters.
• iMAML: We used the official implementation4. We used the hyperparameters reported in (Rajeswaran et al., 2019) for miniImageNet 1-shot 5-way.

Review summaries:
The paper describes an algorithm called COMLN for few-shot meta-learning.  In COMLN, the base level loop is modelled as a continuous-time autonomous ODE that is the gradient vector field of the inner loss.  The authors restrict their discussion to a meta-learning setting where the only base-level parameters are the parameters of a linear classifier. The meta-parameters are the parameters of a neural network that acts as feature extractor, the initialization point for adaptation of the classifier and the adaptation time $T$.  The authors point out that, crucially, it is possible to treat $T$ as a learnable meta-parameter thanks to the continuous reformulation of the base level. Meta-training is carried out by gradient descent on the meta-parameter, where the gradients are computed with an efficient implementation (in few-shot classification, with a small number of classes) forward mode differentiation that exploits the loss structure. The authors present experiments on standard benchmark datasets, showing that COMLN achieves better performances than other comparable gradient-based methods. 

The authors propose COMLN -- a gradient-based meta-learning algorithm with continuous-time inner loop adaptation. The authors discusses possible options for computing the meta-gradient (backpropagation through time, solve ODE backwards in time, and forward-mode differentiation), and identifies the forward-mode differentiation as the only method that is both numerically stable and has a memory cost that is constant w.r.t. the inner-loop length. The authors then derive a way to compute the vector-Jacobian products without explicitly constructing the Jacobians in forward-mode differentiation (specifically for the few-shot learning setting), which further reduces the memory cost. Also, by using a continuous formulation for the inner-optimization, the authors are able to compute meta-gradient w.r.t. horizon T and adapt that as well. Finally, the authors conduct experiments on standard few-shot image classification benchmarks, and show that COMLN outperforms other gradient-based meta-optimization methods in accuracy with a lot less memory cost.

This paper proposes using a continuous-time formulation along with forward-mode differentiation to perform meta-learning. The setup follows that of ANIL: the weights of the last layer is part of the inner loop optimization while the initialization of these last-layer weights and the rest of the weights are treated as meta-parameters. The proposed method relies on this setup in order to efficiently compute derivatives, which the authors note have a low-rank structure that can be exploited. Furthermore, the weights do not need to be solved over time, and instead a much smaller vector can be solved instead, which is pretty neat. 

In this paper, the authors propose a continuous-time formulation for the inner-loop adaptation in a MAML-like setup. Moreover, the authors propose an efficient forward-mode algorithm to learn the parameters of this inner-loop adaptation that is independent of the number of adaptation steps used. The authors show that this works well for a variety of meta-learning problems.