C ONTINUOUS -T IME M ETA -L EARNING WITH F ORWARD M ODE D IFFERENTIATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Drawing inspiration from gradient-based meta-learning methods with inﬁnitely small gradient steps, we introduce Continuous-Time Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of a gradient vector ﬁeld.
Speciﬁcally, representations of the inputs are meta-learned such that a taskspeciﬁc linear classiﬁer is obtained as a solution of an ordinary differential equation (ODE).
Treating the learning process as an ODE offers the notable advantage that the length of the trajectory is now continuous, as opposed to a ﬁxed and discrete number of gradient steps.
As a consequence, we can optimize the amount of adaptation necessary to solve a new task using stochastic gradient descent, in addition to learning the initial conditions as is standard practice in gradient-based meta-learning.
Importantly, in order to compute the exact meta-gradients required for the outer-loop updates, we devise an efﬁcient algorithm based on forward mode differentiation, whose memory requirements do not scale with the length of the learning trajectory, thus allowing longer adaptation in constant memory.
We provide analytical guarantees for the stability of COMLN, we show empirically its efﬁciency in terms of runtime and memory usage, and we illustrate its effectiveness on a range of few-shot image classiﬁcation problems.
I NTRODUCTION
Among the existing meta-learning algorithms, gradient-based methods as popularized by ModelAgnostic Meta-Learning (MAML, Finn et al., 2017) have received a lot of attention over the past few years.
They formulate the problem of learning a new task as an inner optimization problem, typically based on a few steps of gradient descent.
An outer meta -optimization problem is then responsible for updating the meta-parameters of this learning process, such as the initialization of the gradient descent procedure.
However since the updates at the outer level typically require backpropagating through the learning process, this class of methods has often been limited to only a few gradient steps of adaptation, due to memory constraints.
Although solutions have been proposed to alleviate the memory requirements of these algorithms, including checkpointing (Baranchuk, 2019), using implicit differentiation (Rajeswaran et al., 2019), or reformulating the meta-learning objective (Flennerhag et al., 2018), they are generally either more computationally demanding, or only approximate the gradients of the meta-learning objective (Nichol et al., 2018; Flennerhag et al., 2020).
In this work, we propose a continuous-time formulation of gradient-based meta-learning, called
Continuous-Time Meta-Learning (COMLN), where the adaptation is the solution of a differential equation (see Figure 1).
Moving to continuous time allows us to devise a novel algorithm, based on forward mode differentiation, to efﬁciently compute the exact gradients for meta-optimization, no matter how long the adaptation to a new task might be.
We show that using forward mode differentiation leads to a stable algorithm, unlike the counterpart of backpropagation in continuous time called the adjoint method (frequently used in the Neural ODE literature) which tends to be unstable with gradient vector ﬁelds.
Moreover as the length of the adaptation trajectory is a continuous quantity, as opposed to a discrete number of gradient steps ﬁxed ahead of time, we can treat the amount of adaptation in COMLN as a meta-parameter—on par with the initialization—which we can meta-optimize using stochastic gradient descent.
We verify empirically that our method is both computationally and memory efﬁcient, and we show that COMLN outperforms other standard meta-learning algorithms on few-shot image classiﬁcation datasets.
[Figure]
B ACKGROUND
In this work, we consider the problem of few-shot classiﬁcation, that is the problem of learning a classiﬁcation model with only a small number of training examples.
More precisely for a classiﬁcation task τ , we assume that we have access to a (small) training dataset D train τ = { ( x m , y m ) }
Mm =1 to ﬁt a model on task τ , and a distinct test dataset D test τ to evaluate how well this adapted model generalizes on that task.
In the few-shot learning literature, it is standard to consider the problem of k -shot
N -way classiﬁcation, meaning that the model has to classify among N possible classes, and there are only k examples of each class in D train τ , so that overall the number of training examples is M = kN .
We use the convention that the target labels y m ∈ { 0 , 1 } N are one-hot vectors.
2. G RADIENT - BASED META - LEARNING
Gradient-based meta-learning methods aim to learn an initialization such that the model is able to adapt to a new task via gradient descent.
Such methods are often cast as a bi-level optimization process: adapting the task-speciﬁc parameters θ in the inner loop, and training the (task-agnostic) meta-parameters Φ and initialization θ 0 in the outer loop.
The meta-learning objective is:
[Equation]
where T is the number of inner loop updates.
For example, in the case of MAML (Finn et al., 2017), there is no additional meta-parameter other than the initialization ( Φ ≡ ∅ ); in ANIL (Raghu et al., 2019), θ are the parameters of the last layer, and Φ are the parameters of the shared embedding network; in CAVIA (Zintgraf et al., 2019), θ are referred to as context parameters.
During meta-training, the model is trained over many tasks τ .
The task-speciﬁc parameters θ are learned via gradient descent on D train τ .
The meta-parameters are then updated by evaluating the error of the trained model on the test dataset D test τ .
At meta-testing time, the meta-trained model is adapted on D train τ —i.e. applying Equation 2 with the learned meta-parameters θ 0 and Φ .
2. L OCAL SENSITIVITY ANALYSIS OF O RDINARY D IFFERENTIAL E QUATIONS
Consider the following (autonomous) Ordinary Differential Equation (ODE):
[Equation]
where the dynamics g and initial value z 0 may depend on some external parameters θ , and integration is carried out from 0 to some time T .
Local sensitivity analysis is the study of how the solution of this dynamical system responds to local changes in θ ; this effectively corresponds to calculating the derivative d z ( t )
/d θ .
We present here two methods to compute this derivative, with a special focus on their memory efﬁciency.
Adjoint sensitivity method Based on the adjoint state (Pontryagin, 2018), and taking its root in control theory (Lions & Magenes, 2012), the adjoint sensitivity method (Bryson & Ho, 1969; Chavent et al., 1974) provides an efﬁcient approach for evaluating derivatives of L (cid:0)
z ( T ); θ (cid:1) , a function of z ( T ) the solution of the ODE in Equation 3.
This method, popularized lately by the literature on
Neural ODEs (Chen et al., 2018), requires the integration of the adjoint equation
[Equation]
backward in time.
The adjoint sensitivity method can be viewed as a continuous-time counterpart to backpropagation, where the forward pass would correspond to integrating Equation 3 forward in time from 0 to T , and the backward pass to integrating Equation 4 backward in time from T to 0 .
One possible implementation, reminiscent of backpropagation through time (BPTT), is to store the intermediate values of z ( t ) during the forward pass, and reuse them to compute the adjoint state during the backward pass.
While several sophisticated checkpointing schemes have been proposed (Serban & Hindmarsh, 2003; Gholami et al., 2019), with different compute/memory trade-offs, the memory requirements of this approach typically grow with T ; this is similar to the memory limitations standard gradient-based meta-learning methods suffer from as the number of gradient steps increases.
An alternative is to augment the adjoint state a ( t ) with the original state z ( t ) , and to solve this augmented dynamical system backward in time (Chen et al., 2018).
This has the notable advantage that the memory requirements are now independent of T , since z ( t ) are no longer stored during the forward pass, but they are recomputed on the ﬂy during the backward pass.
Forward sensitivity method While the adjoint method is related to reverse-mode automatic differentiation (backpropagation), the forward sensitivity method (Feehery et al., 1997; Leis & Kramer, 1988; Maly & Petzold, 1996; Caracotsios & Stewart, 1985), on the other hand, can be viewed as the continuous-time counterpart to forward (tangent-linear) mode differentiation (Griewank & Walther, 2008).
This method is based on the fact that the derivative S ( t ) (cid:44) d z ( t )
/d θ is the solution of the so-called forward sensitivity equation
[Equation]
This equation can be found throughout the literature in optimal control and system identiﬁcation (Betts, 2010; Biegler, 2010).
Unlike the adjoint method, which requires an explicit forward and backward pass, the forward sensitivity method only requires the integration forward in time of the original ODE in Equation 3, augmented by the sensitivity state S ( t ) with the dynamics above.
The memory requirements of the forward sensitivity method do not scale with T either, but it now requires storing S ( t ) , which may be very large; we will come back to this problem in Section 3.2.
We will simply note here that in discrete-time, this is the same issue afﬂicting forward-mode training of
RNNs with real-time recurrent learning (RTRL; Williams & Zipser, 1989), or other meta-learning algorithms (Sutton, 1992; Franceschi et al., 2017; Xu et al., 2018).
C ONTINUOUS - TIME ADAPTATION
In the limit of inﬁnitely small steps, some optimization algorithms can be viewed as the solution trajectory of a differential equation.
This point of view has often been taken to analyze their behavior (Platt & Barr, 1988; Wilson et al., 2016;
Su et al., 2014; Orvieto & Lucchi, 2019).
In fact, some optimization algorithms such as gradient descent with momentum have even been introduced initially from the perspective of dynamical systems (Polyak, 1964).
As the simplest example, gradient descent with a constant step size α → 0 + (i.e. α tends to 0 by positive values) corresponds to following the dynamics of an autonomous ODE called a gradient vector ﬁeld
[Equation]
where the iterate z ( t ) is now a continuous function of time t .
The solution of this dynamical system is uniquely deﬁned by the choice of the initial condition z (0) =
z 0 .
[Figure]
3. C ONTINUOUS - TIME META - LEARNING
In gradient-based meta-learning, the task-speciﬁc adaptation with gradient descent may also be replaced by a gradient vector ﬁeld in the limit of inﬁnitely small steps.
Inspired by prior work in meta-learning (Raghu et al., 2019; Javed & White, 2019), we assume that an embedding network f Φ with meta-parameters Φ is shared across tasks, and only the parameters W of a task-speciﬁc linear classiﬁer are adapted, starting at some initialization W 0 .
Instead of being the result of a few steps of gradient descent though, the ﬁnal parameters W ( T ) now correspond to integrating an ODE similar to Equation 6 up to a certain horizon T , with the initial conditions W (0) =
W 0 .
We call this new meta-learning algorithm Continuous-Time Meta-Learning 1 (COMLN).
Treating the learning algorithm as a continuous-time process has the notable advantage that the adapted parameter W ( T ) is now differentiable wrt.
the time horizon T (Wiggins, 2003, Chap. 7), in addition to being differentiable wrt.
the initial conditions W 0 —which plays a central role in gradientbased meta-learning, as described in Section 2.1.
This allows us to view T as a meta-parameter on par with Φ and W 0 , and to effectively optimize the amount of adaptation using stochastic gradient descent (SGD).
The meta-learning objective of COMLN can be written as
[Equation]
where f Φ ( D train τ ) = { ( f Φ ( x m ) , y m ) | ( x m , y m ) ∈ D train τ } is the embedded training dataset, and f Φ ( D test τ ) is deﬁned similarly for D test τ .
In practice, adaptation is implemented using a numerical integration scheme based on an iterative discretization of the problem, such as Runge-Kutta methods.
Although a complete discussion of numerical solvers is outside of the scope of this paper, we recommend (Butcher, 2008) for a comprehensive overview of numerical methods for solving ODEs.
3. T HE CHALLENGES OF OPTIMIZING THE META - LEARNING OBJECTIVE
In order to minimize the meta-learning objective of COMLN, it is common practice to use (stochastic) gradient methods; that requires computing its derivatives wrt.
the meta-parameters, which we call meta-gradients .
Our primary goal is to devise an algorithm whose memory requirements do not scale with the amount of adaptation T ; this would contrast with standard gradient-based meta-learning methods that backpropagate through a sequence of gradient steps (similar to BPTT), where the intermediate parameters are stored during adaptation (i.e. θ τt for all t in Equation 2).
Since this objective involves the solution W ( T ) of an ODE, we can use either the adjoint method, or the forward sensitivity method, in order to compute the derivatives wrt.
Φ and W 0 (see Section 2.2).
Although the adjoint method has proven to be an effective strategy for learning Neural ODEs, in practice computing the state W ( t ) backward in time is numerically unstable when applied to a gradient vector ﬁeld like the one in Equation 8, even for convex loss functions.
Figure 2 shows an example where the trajectory of W ( t ) recomputed backward in time (in red) diverges signiﬁcantly from the original trajectory (in green) on a quadratic loss function, even though the two should match exactly in theory since they follow the same dynamics.
Intuitively, recomputing W ( t ) backward in time for a gradient vector ﬁeld requires doing gradient ascent on the loss function, which is prone to compounding numerical errors; this is closely related to the loss of entropy observed by Maclaurin et al.
This divergence makes the backward trajectory of W ( t ) unreliable to ﬁnd the adjoint state, ruling out the adjoint sensitivity method for computing the meta-gradients in COMLN.
The forward sensitivity method addresses this shortcoming by avoiding the backward pass altogether.
However, it can also be particularly expensive here in terms of memory requirements, since the sensitivity state S ( t ) in Section 2.2 now corresponds to Jacobian matrices, such as d W ( t ) /d
W 0 .
As the size d of the feature vectors returned by f
Φ may be very large, this Nd ×
Nd Jacobian matrix would be almost impossible to store in practice; for example in our experiments, it can be as large as d = 16 , 000 for a ResNet-12 backbone.
In Section 4.1, we will show how to apply forward sensitivity in a memory-efﬁcient way, by leveraging the structure of the loss function.
This is achieved by carefully decomposing the Jacobian matrices into smaller pieces that follow speciﬁc dynamics.
We show in Appendix D that unlike the adjoint method, this process is stable.
3. C ONNECTION WITH A LMOST N O I NNER -L OOP (ANIL)
Similarly to ANIL (Raghu et al., 2019), COMLN only adapts the parameters W of the last linear layer of the neural network.
There is a deeper connection between both algorithms though: while our description of the adaptation in COMLN (Eq.
8) was independent of the choice of the ODE solver used to ﬁnd the solution W ( T ) in practice, if we choose an explicit Euler scheme (Euler, 1913, roughly speaking, discretizing Equation 6 from right to left), then the adaptation of COMLN becomes functionally equivalent to ANIL.
However, this equivalence can greatly beneﬁt from the memory-efﬁcient algorithm to compute the meta-gradients described in Section 4, based on the forward sensitivity method.
This means that using the methods devised here for COMLN, we can effectively compute the meta-gradients of ANIL with a constant memory cost wrt.
the number of gradient steps of adaptation, instead of relying on backpropagation (see also Section 4.2).
M EMORY - EFFICIENT META - GRADIENTS
For some ﬁxed task τ and ( x m , y m ) ∈ D train τ , let φ m =
f Φ ( x m ) ∈
R d be the embedding of input x m through the feature extractor f Φ .
Since we are confronted with a classiﬁcation problem, the loss function of choice L ( W ) is typically the cross-entropy loss.
Böhning (1992) showed that the gradient of the cross-entropy loss wrt.
W can be written as
[Equation]
where p m = softmax( W φ m ) is the vector of probabilities returned by the neural network.
The key observation here is that the gradient can be decomposed as a sum of M rank-one matrices, where the feature vectors φ m are independent of W .
Therefore we can fully characterize the gradient of the cross-entropy loss with M vectors p m − y m ∈ R N , as opposed to the full N × d matrix.
This is particularly useful in the context of few-shot classiﬁcation, where the number of training examples
M is small, and typically signiﬁcantly smaller than the embedding size d .
4. D ECOMPOSITION OF THE META - GRADIENTS
We saw in Section 3.2 that the forward sensitivity method was the only stable option to compute the meta-gradients of COMLN.
However, naively applying the forward sensitivity equation would involve quantities that typically scale with d 2 , which can be too expensive in practice.
Using the structure of Equation 9, the Jacobian matrices appearing in the computation of the meta-gradients for
COMLN can be decomposed in such a way that only small quantities will depend on time.
Meta-gradients wrt.
W
By the chain rule of derivatives, it is sufﬁcient to compute the Jacobian matrix d W ( T ) /d
W 0 in order to obtain the meta-gradient wrt.
W 0 .
We show in Appendix B.2 that the sensitivity state d W ( t ) /d
W 0 can be decomposed as:
[Equation]
[Table]
where ⊗ is the Kronecker product
[Table]
and each B t [ i
[Table]
j ] is an N × N matrix, solution of the following system of ODEs
[Equation]
and A i ( t ) , deﬁned in Appendix B.2, is also an N × N matrix that only depends on W ( t ) and φ i .
The main consequence of this decomposition is that we can simply integrate the augmented ODE in (cid:8)
W ( t ) , B t [ i, j ] (cid:9) up to T to obtain the desired Jacobian matrix, along with the adapted parameters W ( T ) .
Furthermore, in contrast to naively applying the forward sensitivity method (see Section 3.2), the M 2 matrices B t [ i, j ] are signiﬁcantly smaller than the full Jacobian matrix.
In fact, we show in Appendix C that we can compute vector-Jacobian products—required for the chain rule—using only these smaller matrices, and without ever having to explicitly construct the full Nd ×
Nd Jacobian matrix d W ( t ) /d
W 0 with Equation 10.
Meta-gradients wrt.
Φ To backpropagate the error through the embedding network f Φ , we need to ﬁrst compute the gradients of the outer-loss wrt.
the feature vectors φ m .
Again, by the chain rule, we can get these gradients with the Jacobian matrices d W ( T ) /d
φ m .
Similar to Equation 10, we can show that these Jacobian matrices can be decomposed as:
[Equation]
where s m ( t ) and z t [ i, j, m ] are vectors of size N , that follow some dynamics; the exact form of this system of ODEs, as well as the proof of this decomposition, are given in Appendix B.3.
Crucially, the only quantities that depend on time are small objects independent of the embedding size d .
Following the same strategy as above, we can incorporate these vectors in the augmented ODE, and integrate it to get the necessary Jacobians.
Once all the d W ( t ) /d
φ m are known, for all the training datapoints, we can apply standard backpropagation through f Φ to obtain the meta-gradients wrt.
Φ .
Meta-gradient wrt. T
One of the major novelties of COMLN is the capacity to meta-learn the amount of adaptation using stochastic gradient descent.
To compute the meta-gradient wrt.
the time horizon T , we can directly borrow the results derived by Chen et al.
(2018) in the context of Neural
ODEs, and apply it to our gradient vector ﬁeld in Equation 8 responsible for adaptation:
[Equation]
The proof is available in Appendix B.4.
Interestingly, we ﬁnd that this involves the alignment between the vectors of partial derivatives of the inner-loss and the outer-loss at W ( T ) , which appeared in different contexts in the meta-learning and the multi-task learning literature (Li et al., 2018; Rothfuss et al., 2019; Yu et al., 2020).
[Table]
4. M EMORY EFFICIENCY
Although naively applying the forward sensitivity method would be memory intensive, we have shown in Section 4.1 that the Jacobians can be carefully decomposed into smaller pieces.
It turns out that even the parameters W ( t ) can be expressed using the vectors s m ( t ) from the decomposition in
Equation 12; see Appendix B.1 for details.
As a consequence, to compute the adapted parameters
W ( T ) as well as all the necessary meta-gradients, it is sufﬁcient to integrate a dynamical system in (cid:8)
B t [ i, j ] , s m ( t ) , z t [ i, j, m ] (cid:9) (see Algorithms 1 & 2 in App. A.1), involving exclusively quantities that are independent of the embedding size d .
Instead, the size of that system scales with M the total number of training examples, which is typically much smaller than d for few-shot classiﬁcation.
Table 1 shows a comparison of the memory cost for different algorithms.
It is important to note that contrary to other standard gradient-based meta-learning methods, the memory requirements of
COMLN do not scale with the amount of adaptation T (i.e. the number of gradient steps in MAML & ANIL), while still returning the exact meta-gradients—unlike iMAML (Rajeswaran et al., 2019), which only returns an approximation of the meta-gradients.
We veriﬁed empirically this efﬁciency, both in terms of memory and computation costs, in Section 5.2.
E XPERIMENTS
For our embedding network f Φ , we consider two commonly used architectures in meta-learning:
Conv-4, a convolutional neural network with 4 convolutional blocks, and ResNet-12, a 12-layer residual network (He et al., 2016).
Note that following Lee et al. (2019), ResNet-12 does not include a global pooling layer at the end of the network, leading to feature vectors with embedding dimension d = 16 , 000 .
Additional details about these architectures are given in Appendix E.
To compute the adapted parameters and the meta-gradients in COMLN, we integrate the dynamical system described in Section 4.2 with a 4th order Runge-Kutta method with a Dormand Prince adaptive step size (Runge, 1895; Dormand & Prince, 1980); we will come back to the choice of this numerical solver in
Section 5.2.
Furthermore to ensure that T > 0 , we parametrized it with an exponential activation.
5. F EW - SHOT IMAGE CLASSIFICATION
We evaluate COMLN on two standard few-shot image classiﬁcation benchmarks: the mini ImageNet (Vinyals et al., 2016) and the tiered ImageNet datasets (Ren et al., 2018), both datasets being derived from ILSVRC-2012 (Russakovsky et al., 2015).
The process for creating tasks follows the standard procedure from the few-shot classiﬁcation literature (Santoro et al., 2016), with distinct classes
[Figure]
between the different splits.
mini Imagenet consists of 100 classes, split into 64 training classes, 16 validation classes, and 20 test classes.
tiered ImageNet consists of 608 classes grouped into 34 high-level categories from ILSVRC-2012, split into 20 training, 6 validation, and 8 testing categories—corresponding to 351/97/160 classes respectively;
Ren et al.
(2018) argue that separating data according to high-level categories results in a more difﬁcult and more realistic regime.
Table 2 shows the average accuracies of COMLN compared to various meta-learning methods, be it gradient-based or not.
For both backbones, COMLN decisively outperforms all other gradient-based meta-learning methods.
Compared to methods that explicitly backpropagate through the learning process, such as MAML or ANIL, the performance gain shown by COMLN could be credited to the longer adaptation T it learns, as opposed to a small number of gradient steps—usually about 10 steps; this was fully enabled by our memory-efﬁcient method to compute meta-gradients, which does not scale with the length of adaptation anymore (see Section 4.2).
We analyse the evolution of T during meta-training for these different settings in Appendix E.3.
In almost all settings, COMLN is even closing the gap with a strong non-gradient-based method like MetaOptNet; the remainder may be explained in part by the training choices made by Lee et al.
(see Appendix E for details).
5. E MPIRICAL EFFICIENCY OF COMLN
In Section 4.2, we showed that our algorithm to compute the meta-gradients, based on forward differentiation, had a memory cost independent of the length of adaptation T .
We verify this empirically in Figure 3, where we compare the memory required by COMLN and other methods to compute the meta-gradients on a single task, with a Conv-4 backbone (Figure 4 in Appendix E.2 shows similar results for ResNet-12).
To ensure an aligned comparison between discrete and continuous time, we use a conversion corresponding to a learning rate α = 0 .
01 in Equation 2; see Appendix E. for a justiﬁcation.
As expected, the memory cost increases for both MAML and ANIL as the number of gradient steps increases, while it remains constant for iMAML and COMLN.
Interestingly, we observe that the cost of COMLN is equivalent to the cost of running ANIL for a small number of steps, showing that the additional cost of integrating the augmented ODE in Section 4.2 to compute the meta-gradients is minimal.
Increasing the length of adaptation also has an impact on the time it takes to compute the adapted parameters, and the meta-gradients.
Figure 3 (right) shows how the runtime increases with the amount of adaptation for different algorithms.
We see that the efﬁciency of COMLN depends on the numerical solver used.
When we use a simple explicit-Euler scheme, the time taken to compute the meta-gradients matches the one of ANIL; this behavior empirically conﬁrms our observation in
Section 3.3.
When we use an adaptive numerical solver, such as Runge-Kutta (RK) with a Dormand
Prince step size, this computation can be signiﬁcantly accelerated, thanks to the smaller number of function evaluations.
In practice, we show in Appendix E.1 that the choice of the ODE solver has a very minimal impact on the accuracy.
R ELATED WORK
We are interested in meta-learning (Bengio et al., 1991; Schmidhuber, 1987; Thrun & Pratt, 2012), and in particular we focus on gradient-based meta-learning methods (Finn, 2018), where the learning rule is based on gradient descent.
While in MAML (Finn et al., 2017) the whole network was updated during this process, follow-up works have shown that it is generally sufﬁcient to share most parts of the neural network, and to only adapt a few layers (Raghu et al., 2019;
Chen et al., 2020b; Tian et al., 2020).
Even though this hypothesis has been challenged recently (Arnold & Sha, 2021), COMLN also updates only the last layer of a neural network, and therefore can be viewed as a continuous-time extension of ANIL (Raghu et al., 2019); see also Section 3.3.
With its shared embedding network across tasks, COMLN is also connected to metric-based meta-learning methods (Vinyals et al., 2016;
Snell et al., 2017; Sung et al., 2018;
Bertinetto et al., 2018; Lee et al., 2019).
Closely related to our work, Zhang et al.
(2021) also introduce a formulation where the adaptation of prototypes follows a gradient vector ﬁeld, but they ﬁnally opt for modeling it as a Neural ODE (Chen et al., 2018), possibly due to the challenges of applying the adjoint method we identiﬁed in Section 3.2.
Zhou et al.
(2021) also uses a gradient vector ﬁeld to motivate a novel method with a closed-form adaptation, based on the NTK theory; COMLN still explicitly updates the parameters following the gradient vector ﬁeld, since there is no closed-form solution of Eq. 8.
As mentioned in Section 3, treating optimization as a continuous-time process has been used to analyze the convergence of different optimization algorithms, including the meta-optimization of MAML (Xu et al., 2021), or to introduce new meta-optimizers based on different integration schemes (Im et al., 2019).
Guo et al.
(2021) also uses meta-learning to learn new integration schemes for ODEs.
Although this is a growing literature at the intersection of meta-learning and dynamical systems, our work is the ﬁrst algorithm that uses a gradient vector ﬁeld for adaptation in meta-learning.
Beyond the memory efﬁciency of our method, one of the main beneﬁts of the continuous-time perspective is that COMLN is capable of learning when to stop the adaptation, as opposed to taking a number of gradient steps ﬁxed ahead of time.
However unlike Chen et al. (2020a), where the number of gradient steps are optimized (up to a maximal number) with variational methods, we incorporate the amount of adaptation as a (continuous) meta-parameter that can be learned using SGD.
To compute the meta-gradients, which is known to be challenging for long sequences in gradient-based meta-learning, we use forward-mode differentiation as an alternative to backpropagation through the learning process, similar to prior work in meta-learning (Franceschi et al., 2017; Jiwoong Im et al., 2021) and hyperparameter optimization over long horizons (Micaelli & Storkey, 2021).
This yields the exact meta-gradients in constant memory, without any assumption on the optimality of the inner optimization problem, which is necessary when using the normal equations (Bertinetto et al., 2018), or to apply implicit differentiation (Rajeswaran et al., 2019).
C ONCLUSION AND F UTURE W ORK
In this paper, we have introduced Continuous-Time Meta-Learning (COMLN), a novel algorithm that treats the adaptation in meta-learning as a continuous-time process, by following the dynamics of a gradient vector ﬁeld up to a certain time horizon T .
One of the major novelties of treating adaptation in continuous time is that the amount of adaptation T is now a continuous quantity, that can be viewed as a meta-parameter and can be learned using SGD, alongside the initial conditions and the parameters of the embedding network.
In order to learn these meta-parameters, we have also introduced a novel practical algorithm based on forward mode automatic differentiation, capable of efﬁciently computing the exact meta-gradients using an augmented dynamical system.
We have veriﬁed empirically that this algorithm was able to compute the meta-gradients in constant memory, making it the ﬁrst gradient-based meta-learning approach capable of computing the exact metagradients with long sequences of adaptation using gradient methods.
In practice, we have shown that
COMLN signiﬁcantly outperforms other standard gradient-based meta-learning algorithms.
In addition to having a single meta-parameter T that drives the adaptation of all possible tasks, the fact that the time horizon can be learned with SGD opens up new possibilities for gradient-based methods.
For example, we could imagine treating T not as a shared meta-parameters, but as a task-speciﬁc parameter.
This would allow the learning process to be more adaptive , possibly with different behaviors depending on the difﬁculty of the task.
This is left as a future direction of research.
R EPRODUCIBILITY STATEMENT
We provide in Appendix A.1 a full description in pseudo-code of the meta-training procedure (Algorithm 1), along with the exact dynamics of the ODE (Algorithm 2) and the projection operations (Algorithms 3 & 4) to avoid explicitly building the Jacobian matrices to compute Jacobian-vector products (see Section 4.1).
We also provide in Appendix A.2 a snippet of code in JAX (Bradbury et al., 2018) to compute the adapted parameters W ( T ) , as well as all the necessary objects (cid:8) B t [ i, j ] , s m ( t ) , z t [ i, j, m ] (cid:9) to compute all the meta-gradients (see Section 4.2).
We also give in Code Snippet 2 the code to compute the meta-gradients wrt.
the initialization W 0 and the integration time T .
Computing the meta-gradients wrt.
Φ involves non-minimal dependencies on Haiku (Hennigan et al., 2020), and therefore is omitted here.
The full code is available in the Supplementary Materials .
Data generation hyperparameters We used the mini ImageNet and tiered ImageNet datasets provided by Lee et al. (2019) in order to create the 1-shot 5-way and 5-shot 5-way tasks for both datasets.
During evaluation, for each setting, a ﬁxed set of 1 , 000 tasks were sampled; this means that both architectures for COMLN have been evaluated using exactly the same data, to ensure direct comparison across backbones.
A full description of all the hyperparameters used in COMLN is given in Appendix E.
Reproducibility of baseline results To the best of our ability, we have tried to report baseline results from existing work, to limit as much as possible the bias induced by running our own baseline experiments.
The references of those works are given in Table 3.
We still had to run CAVIA and iMAML on the remaining settings, since these results have not been reported in the literature.
For both methods, we used the data generation described above.
• CAVIA :
We used the ofﬁcial implementation 3 .
We used the hyperparameters reported in (Zintgraf et al., 2019) for mini ImageNet, and an architecture with 64 ﬁlters.
• iMAML : We used the ofﬁcial implementation 4 .
We used the hyperparameters reported in (Rajeswaran et al., 2019) for mini ImageNet 1-shot 5-way.
[-Algorithm function P ROJECT W 
-]
[-s m ( T ) (cid:62) V ( T ) + C m W 
-]
[Table]
[-B 
-]
[-with the initial conditions z 
-]
[-Moreover using the initial conditions s m (0) = 0 , B 
-]
[-[ i, j ] = 0 , and z 
-]
https://github.com/lmzintgraf/cavia/https://github.com/aravindr93/imaml_dev