R APID L EARNING OR F EATURE R EUSE ? T OWARDS U NDERSTANDING THE E FFECTIVENESS OF MAML
DeepMind A BSTRACT
An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning.
An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop ﬁnding a meta-initialization , from which the inner loop can efﬁciently learn new tasks.
Despite MAML’s popularity, a fundamental open question remains – is the effectiveness of MAML due to the meta-initialization being primed for rapid learning (large, efﬁcient changes in the representations) or due to feature reuse , with the meta-initialization already containing high quality features?
We investigate this question, via ablation studies and analysis of the latent representations, ﬁnding that feature reuse is the dominant factor.
This leads to the ANIL (Almost No Inner Loop) algorithm, a simpliﬁcation of MAML where we remove the inner loop for all but the (task-speciﬁc) head of the underlying neural network.
ANIL matches MAML’s performance on benchmark few-shot image classiﬁcation and RL and offers computational improvements over
MAML.
We further study the precise contributions of the head and body of the network, showing that performance on the test tasks is entirely determined by the quality of the learned features, and we can remove even the head of the network (the NIL algorithm).
We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly.
I NTRODUCTION
A central problem in machine learning is few-shot learning , where new tasks must be learned with a very limited number of labelled datapoints.
A signiﬁcant body of work has looked at tackling this challenge using meta-learning approaches (Koch et al., 2015; Vinyals et al., 2016;
Snell et al., 2017; Finn et al., 2017; Santoro et al., 2016; Ravi and Larochelle, 2016; Nichol and Schulman, 2018).
Broadly speaking, these approaches deﬁne a family of tasks, some of which are used for training and others solely for evaluation.
A proposed meta-learning algorithm then looks at learning properties that generalize across the different training tasks, and result in fast and efﬁcient learning of the evaluation tasks.
One highly successful meta-learning algorithm has been Model Agnostic Meta-Learning (MAML)
(Finn et al., 2017).
At a high level, the MAML algorithm is comprised of two optimization loops.
The outer loop (in the spirit of meta-learning) aims to ﬁnd an effective meta-initialization , from which the inner loop can perform efﬁcient adaptation – optimize parameters to solve new tasks with very few labelled examples.
This algorithm, with deep neural networks as the underlying model, has been highly inﬂuential, with signiﬁcant follow on work, such as ﬁrst order variants (Nichol and Schulman, 2018), probabilistic extensions (Finn et al., 2018), augmentation with generative modelling (Rusu et al., 2018), and many others (Hsu et al., 2018; Finn and Levine, 2017; Grant et al., 2018; Triantaﬁllou et al., 2019).
Despite the popularity of MAML, and the numerous followups and extensions, there remains a fundamental open question on the basic algorithm.
Does the meta-initialization learned by the outer loop result in rapid learning on unseen test tasks (efﬁcient but signiﬁcant changes in the representations) or is the success primarily due to feature reuse (with the meta-initialization already providing high quality representations)?
In this paper, we explore this question and its many surprising consequences.
Our main contributions are:
• We perform layer freezing experiments and latent representational analysis of MAML, ﬁnding that feature reuse is the predominant reason for efﬁcient learning.
• Based on these results, we propose the ANIL (Almost No Inner Loop) algorithm, a signiﬁcant simpliﬁcation to MAML that removes the inner loop updates for all but the head (ﬁnal layer) of a neural network during training and inference.
ANIL performs identically to MAML on standard benchmark few-shot classiﬁcation and RL tasks and offers computational beneﬁts over MAML.
• We study the effect of the head of the network, ﬁnding that once training is complete, the head can be removed, and the representations can be used without adaptation to perform unseen tasks, which we call the No Inner Loop (NIL) algorithm.
• We study different training regimes, e.g. multiclass classiﬁcation, multitask learning, etc, and ﬁnd that the task speciﬁcity of MAML/ANIL at training facilitate the learning of better features.
We also ﬁnd that multitask training, a popular baseline with no task speciﬁcity, performs worse than random features.
• We discuss rapid learning and feature reuse in the context of other meta-learning approaches.
R ELATED W ORK
MAML (Finn et al., 2017) is a highly popular meta-learning algorithm for few-shot learning, achieving competitive performance on several benchmark few-shot learning problems (Koch et al., 2015; Vinyals et al., 2016;
Snell et al., 2017; Santoro et al., 2016; Ravi and Larochelle, 2016; Nichol and Schulman, 2018).
It is part of the family of optimization-based meta-learning algorithms, with other members of this family presenting variations around how to learn the weights of the task-speciﬁc classiﬁer.
For example Lee and Choi (2018); Gordon et al.
; Bertinetto et al.
; Lee et al. ; Zhou et al. ﬁrst learn functions to embed the support set and target examples of a few-shot learning task, before using the test support set to learn task speciﬁc weights to use on the embedded target examples. Harrison et al.
(2018) also proceeds similarly, using a Bayesian approach.
The method of Bao et al. (2019) explores a related approach, focusing on applications in text classiﬁcation.
Of these optimization-based meta-learning algorithms, MAML has been especially inﬂuential, inspiring numerous direct extensions in recent literature (Antoniou et al., 2018;
Finn et al., 2018; Grant et al., 2018; Rusu et al., 2018).
Most of these extensions critically rely on the core structure of the MAML algorithm, incorporating an outer loop (for meta-training), and an inner loop (for task-speciﬁc adaptation), and there is little prior work analyzing why this central part of the MAML algorithm is practically successful.
In this work, we focus on this foundational question, examining how and why MAML leads to effective few-shot learning.
To do this, we utilize analytical tools such as Canonical
Correlation Analysis (CCA) (Raghu et al., 2017; Morcos et al., 2018) and Centered Kernel Alignment (CKA) (Kornblith et al., 2019) to study the neural network representations learned with the MAML algorithm, which also demonstrates MAML’s ability to learn effective features for few-shot learning.
Insights from this analysis lead to a simpliﬁed algorithm, ANIL, which almost completely removes the inner optimization loop with no reduction in performance.
Prior works (Zintgraf et al., 2018; Javed and White, 2019) have proposed algorithms where some parameters are only updated in the outer loop and others only in the inner loop.
However, these works are motivated by different questions, such as improving MAML’s performance or learning better representations, rather than analysing rapid learning vs feature reuse in MAML.
MAML, R APID L EARNING , AND F EATURE R EUSE
Our goal is to understand whether the MAML algorithm efﬁciently solves new tasks due to rapid learning or feature reuse .
In rapid learning, large representational and parameter changes occur during adaptation to each new task as a result of favorable weight conditioning from the meta-initialization.
In feature reuse, the meta-initialization already contains highly useful features that can mostly be reused as is for new tasks, so little task-speciﬁc adaptation occurs.
Figure 1 shows a schematic of these two hypotheses.
We start off by overviewing the details of the MAML algorithm, and then we study the rapid learning vs feature reuse question via layer freezing experiments and analyzing latent representations of models trained with MAML.
The results strongly support feature reuse as the predominant factor behind MAML’s success.
In Section 4, we explore the consequences of this, providing a signiﬁcant simpliﬁcation of MAML, the ANIL algorithm, and in Section 6, we outline the connections to meta-learning more broadly.
3. O VERVIEW OF MAML
The MAML algorithm ﬁnds an initialization for a neural network so that new tasks can be learnt with very few examples ( k examples from each class for k -shot learning) via two optimization loops:
• Outer Loop: Updates the initialization of the neural network parameters (often called the meta-initialization ) to a setting that enables fast adaptation to new tasks.
• Inner Loop: Performs adaptation : takes the outer loop initialization, and, separately for each task , performs a few gradient updates over the k labelled examples (the support set ) provided for adaptation.
More formally, we ﬁrst deﬁne our base model to be a neural network with meta-initialization parameters θ ; let this be represented by f θ .
We have have a distribution D over tasks, and draw a batch { T 1 , ..., T B } of B tasks from D .
For each task T b , we have a support set of examples S T b , which are used for inner loop updates, and a target set of examples Z T b , which are used for outer loop updates.
Let θ ( b ) i signify θ after i gradient updates for task T b , and let θ ( b )
= θ .
In the inner loop, during each update, we compute
[Equation]
for m ﬁxed across all tasks, where L S Tb ( f θ ( b ) m − 1 ( θ ) ) is the loss on the support set of T b after m −inner loop updates.
We then deﬁne the meta-loss as
[Equation]
where L Z Tb ( f θ ( b ) m ( θ ) ) is the loss on the target set of T b after m inner loop updates, making clear the dependence of f θ ( b ) m on θ .
The outer optimization loop then updates θ as
[Equation]
[Table]
At test time
[Table] we draw unseen tasks { T ( test ), ..., T ( test ) n } from the task distribution [Table] and evaluate the loss and accuracy on Z T ( test ) i after inner loop adaptation
[Table]
[Table]
3. R APID L EARNING OR F EATURE R EUSE ?
We now turn our attention to the key question: Is MAML’s efﬁcacy predominantly due to rapid learning or feature reuse?
In investigating this question, there is an important distinction between the head (ﬁnal layer) of the network and the earlier layers (the body of the network).
In each few-shot learning task, there is a different alignment between the output neurons and classes.
For instance, in task T 1 , the (wlog) ﬁve output neurons might correspond, in order, to the classes (dog, cat, frog, cupcake, phone), while for a different task, T 2 , they might correspond, in order, to (airplane, frog, boat, car, pumpkin).
This means that the head must necessarily change for each task to learn the new alignment, and for the rapid learning vs feature reuse question, we are primarily interested in the behavior of the body of the network.
We return to this in more detail in Section 5, where we present an algorithm (NIL) that does not use a head at test time.
To study rapid learning vs feature reuse in the network body, we perform two sets of experiments: (1) We evaluate few-shot learning performance when freezing parameters after MAML training, without test time inner loop adaptation; (2) We use representational similarity tools to directly analyze how much the network features and representations change through the inner loop.
We use the
MiniImageNet dataset, a popular standard benchmark for few-shot learning, and with the standard convolutional architecture in Finn et al.
Results are averaged over three random seeds.
Full implementation details are in Appendix B.
3.2. F REEZING L AYER R EPRESENTATIONS
To study the impact of the inner loop adaptation, we freeze a contiguous subset of layers of the network, during the inner loop at test time (after using the standard MAML algorithm, incorporating both optimization loops, for training).
In particular, the frozen layers are not updated at all to the test time task, and must reuse the features learned by the meta-initialization that the outer loop converges to.
We compare the few-shot learning accuracy when freezing to the accuracy when allowing inner loop adaptation.
Results are shown in Table 1.
We observe that even when freezing all layers in the network body, performance hardly changes.
This suggests that the meta-initialization has already learned good enough features that can be reused as is, without needing to perform any rapid learning for each test time task.
3.2. R EPRESENTATIONAL S IMILARITY E XPERIMENTS
We next study how much the latent representations (the latent functions) learned by the neural network change during the inner loop adaptation phase.
Following several recent works (Raghu et al., 2017; Saphra and Lopez, 2018;
Morcos et al., 2018; Maheswaranathan et al et al
[Figure]
2019; Gotmare et al., 2018; Bau et al., 2018) we measure this by applying Canonical Correlation Analysis (CCA) to the latent representations of the network.
CCA provides a way to the compare representations of two (latent) layers L 1 , L 2 of a neural network, outputting a similarity score between 0 (not similar at all) and 1 (identical).
For full details, see Raghu et al.
; Morcos et al.
In our analysis, we take L 1 to be a layer before the inner loop adaptation steps, and L 2 after the inner loop adaptation steps.
We compute CCA similarity between L 1 , L 2 , averaging the similarity score across different random seeds of the model and different test time tasks.
Full details are in Appendix B.
The result is shown in Figure 2, left pane.
Representations in the body of the network (the convolutional layers) are highly similar, with CCA similarity scores of > 0 .
9 , indicating that the inner loop induces little to no functional change.
By contrast, the head of the network, which does change signiﬁcantly in the inner loop, has a CCA similarity of less than 0 .
To further validate this, we also compute CKA (Centered Kernel Alignment) (Kornblith et al., 2019) (Figure 2 right), another similarity metric for neural network representations, which illustrates the same pattern.
These representational analysis results strongly support the feature reuse hypothesis, with further results in the
Appendix, Sections B.3 and B.4 providing yet more evidence.
3.2. F EATURE R EUSE H APPENS E ARLY IN L EARNING
Having observed that the inner loop does not signiﬁcantly affect the learned representations with a fully trained model, we extend our analysis to see whether the inner loop affects representations and features earlier on in training.
We take MAML models at 10000, 20000, and 30000 iterations into training, perform freezing experiments (as in Section 3.2.1) and representational similarity experiments (as in Section 3.2.2).
Results in Figure 3 show the same patterns from early in training, with CCA similarity between activations pre and post inner loop update on MiniImageNet-5way-5shot being very high for the body (just like Figure 2), and similar to Table 1, test accuracy remaining approximately the same when freezing contiguous subsets of layers, even when freezing all layers of the network body.
This shows that even early on in training, signiﬁcant feature reuse is taking place, with the inner loop having minimal effect on learned representations and features.
Results for 1shot MiniImageNet are in
Appendix B.5, and show very similar trends.
T HE ANIL (A LMOST N O I NNER L OOP ) A LGORITHM
In the previous section we saw that for all layers except the head of the neural network, the metainitialization learned by the outer loop of MAML results in very good features that can be reused [Figure] as is on new tasks.
Inner loop adaptation does not signiﬁcantly change the representations of these layers, even from early on in training.
This suggests a natural simpliﬁcation of the MAML algorithm: the ANIL (Almost No Inner Loop) algorithm .
In ANIL, during training and testing, we remove the inner loop updates for the network body [Figure] and apply inner loop adaptation only to the head .
The head requires the inner loop to allow it to align to the different classes in each task.
In Section 5.1 we consider another variant, the NIL (No Inner Loop) algorithm , that removes the head entirely at test time, and uses learned features and cosine similarity to perform effective classiﬁcation, thus avoiding inner loop updates altogether.
For the ANIL algorithm, mathematically, let θ =
( θ 1 , ..., θ l ) be the (meta-initialization) parameters for the l layers of the network.
Following the notation of Section 3.1, let θ ( b ) m be the parameters after m inner gradient updates for task T b .
In ANIL, we have that:
[Equation]
i.e. only the ﬁnal layer gets the inner loop updates.
As before, we then deﬁne the meta-loss, and compute the outer loop gradient update.
The intuition for ANIL arises from Figure 3, where we observe that inner loop updates have little effect on the network body even early in training, suggesting the possibility of removing them entirely.
Note that this is distinct to the freezing experiments, where we only removed the inner loop at inference time.
Figure 4 presents the difference between MAML and ANIL, and Appendix C.1 considers a simple example of the gradient update in ANIL, showing how the ANIL update differs from MAML.
Computational beneﬁt of ANIL:
As ANIL almost has no inner loop, it signiﬁcantly speeds up both training and inference.
We found an average speedup of 1.7x per training iteration over MAML and an average speedup of 4.1x per inference iteration.
In Appendix C.5 we provide the full results.
[Table] [Figure] Results of ANIL on Standard Benchmarks: We evaluate ANIL on few-shot image classiﬁcation and RL benchmarks, using the same model architectures as the original MAML authors, for both supervised learning and RL.
Further implementation details are in Appendix C.4.
The results in Table 2 (mean and standard deviation of performance over three random initializations) show that ANIL matches the performance of MAML on both few-shot classiﬁcation (accuracy) and RL (average return, the higher the better), demonstrating that the inner loop adaptation of the body is unnecessary for learning good features.
MAML and ANIL Models Show Similar Behavior: MAML and ANIL perform equally well on few-shot learning benchmarks, illustrating that removing the inner loop during training does not hinder performance.
To study the behavior of MAML and ANIL models further, we plot learning curves for both algorithms on MiniImageNet-5way-5shot, Figure 5.
We see that loss and accuracy for both algorithms look very similar throughout training.
We also look at CCA and
CKA scores of the representations learned by both algorithms, Table 3.
We observe that MAMLANIL representations have the same average similarity scores as MAML-MAML and ANIL-ANIL representations, suggesting both algorithms learn comparable features (removing the inner loop doesn’t change the kinds of features learned.)
Further learning curves and representational similarity results are presented in Appendices C.2 and C.3.
C ONTRIBUTIONS OF THE N ETWORK H EAD AND B ODY
So far, we have seen that MAML predominantly relies on feature reuse, with the network body (all layers except the last layer) already containing good features at meta-initialization.
We also observe that such features can be learned even without inner loop adaptation during training (ANIL algorithm).
The head, however, requires inner loop adaptation to enable task speciﬁcity. [Table]
[Table] In this section [Table] we explore the contributions of the network head and body.
We ﬁrst ask: How important is the head at test time, when good features have already been learned?
Motivating this question is that the features in the body of the network needed no adaptation at inference time, so perhaps they are themselves sufﬁcient to perform classiﬁcation, with no head.
In Section 5.1, we ﬁnd that test time performance is entirely determined by the quality of these representations, and we can use similarity of the frozen meta-initialization representations to perform unseen tasks, removing the head entirely .
We call this the NIL (No Inner Loop) algorithm.
Given this result, we next study how useful the head is at training (in ensuring the network body learns good features).
We look at multiple different training regimes (some without the head) for the network body, and evaluate the quality of the representations.
We ﬁnd that MAML/ANIL result in the best representations, demonstrating the importance of the head during training for feature learning.
5. T HE H EAD AT T EST T IME AND THE NIL (N O I NNER L OOP ) A LGORITHM
We study how important the head and task speciﬁc alignment are when good features have already been learned (through training) by the meta-initialization.
At test time, we ﬁnd that the representations can be used directly, with no adaptation, which leads to the No Inner Loop (NIL)
algorithm :
Train a few-shot learning model with ANIL/MAML algorithm as standard.
We use ANIL training.
At test time, remove the head of the trained model.
For each task, ﬁrst pass the k labelled examples (support set) through the body of the network, to get their penultimate layer representations.
Then, for a test example, compute cosine similarities between its penultimate layer representation and those of the support set, using these similarities to weight the support set labels, as in Vinyals et al. (2016).
The results for the NIL algorithm, following ANIL training, on few-shot classiﬁcation benchmarks are given in Table 4.
Despite having no network head and no task speciﬁc adaptation, NIL performs comparably to MAML and ANIL.
This demonstrates that the features learned by the network body when training with MAML/ANIL (and reused at test time) are the critical component in tackling these benchmarks
. 5. T RAINING R EGIMES FOR THE N ETWORK B ODY
The NIL algorithm and results of Section 5.1 lead to the question of how important task alignment and the head are during training to ensure good features.
Here, we study this question by examining the quality of features arising from different training regimes for the body.
We look at (i) MAML
[Table]
and ANIL training; (ii) multiclass classiﬁcation , where all of the training data and classes (from which training tasks are drawn) are used to perform standard classiﬁcation; (iii) multitask training , a standard baseline, where no inner loop or task speciﬁc head is used, but the network is trained on all the tasks at the same time; (iv) random features , where the network is not trained at all, and features are frozen after random initialization; (v) NIL at training time, where there is no head and cosine distance on the representations is used to get the label.
After training, we apply the NIL algorithm to evaluate test performance, and quality of features learned at training.
The results are shown in Table 5.
MAML and ANIL training performs best.
Multitask training, which has no task speciﬁc head, performs the worst, even worse than random features (adding evidence for the need for task speciﬁcity at training to facilitate feature learning.)
Using NIL during training performs worse than MAML/ANIL.
These results suggest that the head is important at training to learn good features in the network body.
In Appendix D.1, we study test time performance variations from using a MAML/ANIL head instead of NIL, ﬁnding (as suggested by Section 5.1) very little performance difference.
Additional results on similarity between the representations of different training regimes is given in Appendix D.2.
F EATURE R EUSE IN O THER META - LEARNING A LGORITHMS
Up till now, we have closely examined the MAML algorithm, and have demonstrated empirically that the algorithm’s success is primarily due to feature reuse, rather than rapid learning.
We now discuss rapid learning vs feature reuse more broadly in meta-learning.
By combining our results with an analysis of evidence reported in prior work, we ﬁnd support for many meta-learning algorithms succeeding via feature reuse, identifying a common theme characterizing the operating regime of much of current meta-learning.
6. O PTIMIZATION AND M ODEL B ASED META - LEARNING
MAML falls within the broader class of optimization based meta-learning algorithms, which at inference time, directly optimize model parameters for a new task using the support set.
MAML has inspired many other optimization-based algorithms, which utilize the same two-loop structure (Lee and Choi, 2018; Rusu et al., 2018; Finn et al., 2018).
Our analysis so far has thus yielded insights into the feature reuse vs rapid learning question for this class of algorithms.
Another broad class of meta-learning consists of model based algorithms, which also have notions of rapid learning and feature reuse.
In the model-based setting, the meta-learning model’s parameters are not directly optimized for the speciﬁc task on the support set.
Instead, the model typically conditions its output on some representation of the task deﬁnition.
One way to achieve this conditioning is to jointly encode the entire support set in the model’s latent representation (Vinyals et al., 2016; Sung et al., 2018), enabling it to adapt to the characteristics of each task.
This constitutes rapid learning for model based meta-learning algorithms.
An alternative to joint encoding would be to encode each member of the support set independently, and apply a cosine similarity rule (as in Vinyals et al. to classify an unlabelled example.
This mode of operation is purely feature reuse – we do not use information deﬁning the task to directly inﬂuence the decision function.
If joint encoding gave signiﬁcant test-time improvement over non-joint encoding, then this would suggest that rapid learning of the test-time task is taking place, as task speciﬁc information is being utilized to inﬂuence the model’s decision function.
However, on analyzing results in prior literature, this improvement appears to be minimal.
Indeed, in e.g. Matching Networks (Vinyals et al., 2016), using joint encoding one reaches 44.2% accuracy on MiniImageNet-5way-1shot, whereas with independent encoding one obtains 41.2%: a small difference.
More reﬁned models suggest the gap is even smaller.
For instance, in Chen et al. (2019), many methods for one shot learning were re-implemented and studied, and baselines without joint encoding achieved 48.24% accuracy in MiniImageNet-5way-1shot, whilst other models using joint encoding such as Relation Net (Sung et al., 2018) achieve very similar accuracy of 49.31% (they also report MAML, at 46.47%).
As a result, we believe that the dominant mode of “feature reuse” rather than “rapid learning” is what has currently dominated both MAML-styled optimization based meta-learning and model based meta-learning.
C ONCLUSION
In this paper, we studied a fundamental question on whether the highly successful MAML algorithm relies on rapid learning or feature reuse.
Through a series of experiments, we found that feature reuse is the dominant component in MAML’s efﬁcacy on benchmark datasets.
This insight led to the ANIL (Almost No Inner Loop) algorithm, a simpliﬁcation of MAML that has identical performance on standard image classiﬁcation and reinforcement learning benchmarks, and provides computational beneﬁts.
We further study the importance of the head (ﬁnal layer) of a neural network trained with
MAML, discovering that the body (lower layers) of a network is sufﬁcient for few-shot classiﬁcation at test time, allowing us to remove the network head for testing (NIL algorithm) and still match performance.
We connected our results to the broader literature in meta-learning, identifying feature reuse to be a common mode of operation for other meta-learning algorithms also.
Based off of our conclusions, future work could look at developing and analyzing new meta-learning algorithms that perform more rapid learning, which may expand the datasets and problems amenable to these techniques.
We note that our study mainly considered benchmark datasets, such as Omniglot and
MiniImageNet.
It is an interesting future direction to consider rapid learning and feature reuse in
MAML on other few-shot learning datasets, such as those from Triantaﬁllou et al.
A CKNOWLEDGEMENTS
The authors thank Geoffrey Hinton, Chelsea Finn, Hugo Larochelle and Chiyuan Zhang for helpful feedback on the methods and results.
A F EW -S HOT I MAGE C LASSIFICATION D ATASETS AND E XPERIMENTAL S ETUPS
We consider the few-shot learning paradigm for image classiﬁcation to evaluate MAML and ANIL.
We evaluate using two datasets often used for few-shot multiclass classiﬁcation – the Omniglot dataset and the MiniImageNet dataset.
Omniglot: The Omniglot dataset consists of over 1600 different handwritten character classes from 23 alphabets.
The dataset is split on a character-level, so that certain characters are in the training set, and others in the validation set.
We consider the 20-way 1-shot and 20-way 5-shot tasks on this dataset, where at test time, we wish our classiﬁer to discriminate between 20 randomly chosen character classes from the held-out set, given only 1 or 5 labelled example(s) from each class from this set of 20 testing classes respectively.
The model architecture used is identical to that in the original MAML paper, namely: 4 modules with a 3 x 3 convolutions and 64 ﬁlters with a stride of 2, followed by batch normalization, and a ReLU nonlinearity.
The Omniglot images are downsampled to 28 x 28, so the dimensionality of the last hidden layer is 64.
The last layer is fed into a 20-way softmax.
Our models are trained using a batch size of 16, 5 inner loop updates, and an inner learning rate of 0.1.
MiniImageNet: The MiniImagenet dataset was proposed by Ravi and Larochelle (2016), and consists of 64 training classes, 12 validation classes, and 24 test classes.
We consider the 5-way 1-shot and 5-way 5-shot tasks on this dataset, where the test-time task is to classify among 5 different randomly chosen validation classes, given only 1 and 5 labelled examples respectively.
The model architecture is again identical to that in the original paper: 4 modules with a 3 x 3 convolutions and 32 ﬁlters, followed by batch normalization, ReLU nonlinearity, and 2 x 2 max pooling.
Our models are trained using a batch size of 4.
5 inner loop update steps, and an inner learning rate of 0.01 are used.
10 inner gradient steps are used for evaluation at test time.
B A DDITIONAL D ETAILS AND R ESULTS : F REEZING AND R EPRESENTATIONAL S IMILARITY
In this section, we provide further experimental details and results from freezing and representational similarity experiments.
B. E XPERIMENTAL D ETAILS
We concentrate on MiniImageNet for our experiments in Section 3.2, as it is more complex than
Omniglot.
The model architecture used for our experiments is identical to that in the original paper: 4 modules with a 3 × 3 convolutions and 32 ﬁlters, followed by batch normalization, ReLU nonlinearity, and 2 × 2 max pooling.
Our models are trained using a batch size of 4, 5 inner loop update steps, and an inner learning rate of 0.01.
10 inner gradient steps are used for evaluation at test time.
We train models 3 times with different random seeds.
Models were trained for 30000 iterations.
B. D ETAILS OF R EPRESENTATIONAL S IMILARITY
CCA takes in as inputs L 1 = { z (1)1 , z
(1)2 , ..., z (1) m } and L 2 = { z (2)1 , z (1)2 , ..., z (2) n } , where L 1 , L 2 are layers, and z ( j ) i is a neuron activation vector: the vector of outputs of neuron i (of layer L j ) over a set of inputs X .
It then ﬁnds linear combinations of the neurons in L 1 and neurons in L 2 so that the resulting activation vectors are maximally correlated, which is summarized in the canonical correlation coefﬁcient.
Iteratively repeating this process gives a similarity score (in [0 , 1] withidentical and 0 completely different) between the representations of L 1 and L 2 .
We apply this to compare corresponding layers of two networks, net1 and net2, where net1 and net might differ due to training step, training method (ANIL vs MAML) or the random seed.
When comparing convolutional layers, as described in Raghu, we perform the comparison over channels,
[Figure]
ﬂattening out over all of the spatial dimensions, and then taking the mean CCA coefﬁcient.
We average over three random repeats.
B. S IMILARITY B EFORE AND A FTER I NNER L OOP WITH E UCLIDEAN D ISTANCE
In addition to assessing representational similarity with CCA/CKA, we also consider the simpler measure of Euclidean distance, capturing how much weights of the network change during the inner loop update (task-speciﬁc ﬁnetuning).
We note that this experiment does not assess functional changes on inner loop updates as well as the CCA experiments do; however, they serve to provide useful intuition.
We plot the per-layer average Euclidean distance between the initialization θ and the ﬁnetuned weights θ ( b ) m across different tasks T b , i.e.
[Equation]
across different layers l , for MiniImageNet in Figure 6.
We observe that very quickly after the start of training, all layers except for the last layer have small Euclidean distance difference before and after ﬁnetuning, suggesting signiﬁcant feature reuse.
(Note that this is despite the fact that these layers have more parameters than the ﬁnal layer.)
B. CCA S IMILARITY A CROSS R ANDOM S EEDS
The experiment in Section 3.2.2 compared representational similarity of L 1 and L 2 at different points in training (before/after inner loop adaptation) but corresponding to the same random seed.
To complete the picture, it is useful to study whether representational similarity across different random seeds is also mostly unaffected by the inner loop adaptation.
This motivates four natural comparisons: assume layer L 1 is from the ﬁrst seed, and layer L 2 is from the second seed.
Then we can compute the representational similarity between ( L 1 pre, L 2 pre), ( L 1 pre, L 2 post), ( L 1 post, L 2 pre) and ( Lpost, L 2 post), where pre/post signify whether we take the representation before or after adaptation.
Prior work has shown that neural network representations may vary across different random seeds (Raghu et al., 2017; Morcos et al., 2018; Li et al., 2015; Wang et al., 2018), organically resulting in CCA similarity scores much less than 1 .
So to identify the effect of the inner loop on the representation, we plot the CCA similarities of (i) ( L 1 pre, L 2 pre) against ( L 1 pre, L 2 post) and (ii) ( L 1 pre, L 2 pre) against ( L 1 post, L 2 pre) and (iii) ( L 1 pre, L 2 pre) against ( L 1 post, L 2 post) separately across the different random seeds and different layers.
We then compute the line of best ﬁt for each plot.
If the line of best ﬁt ﬁts the data and is close to y = x , this suggests that the inner loop adaptation doesn’t affect the features much – the similarity before adaptation is very close to the similarity after adaptation.
[Figure]
i
[Figure]
r i t y
[Figure]
adaptation from inherent variation in the network representation across seeds, we plot CCA similarity of of the representations before adaptation against representations after adaptation in three different combinations: (i) ( L 1 pre, L 2 pre) against ( L 1 pre, L 1 post), (ii) ( L 1 pre, L 2 pre) against ( L 1 pre, L 1 post) (iii)
( L 1 pre, L 2 pre) against ( L 1 post, L 2 post).
We do this separately across different random seeds and different layers.
Then, we compute a line of best ﬁt, ﬁnding that in all three plots, it is almost identical to y = x , demonstrating that the representation does not change signiﬁcantly pre/post adaptation.
Furthermore a computation of the coefﬁcient of determination R 2 gives R 2 ≈ 1 , illustrating that the data is well explained by this relation.
In Figure 8, we perform this comparison with CKA, observing the same high level conclusions.
[Figure]
The results are shown in Figure 7. In all of the plots, we see that the line of best ﬁt is almost exactly y = x (even for the pre/pre vs post/post plot, which could conceivably be more different as both seeds change) and a computation of the coefﬁcient of determination R 2 gives R 2 ≈ 1 for all three plots.
Putting this together with Figure 2, we can conclude that the inner loop adaptation step doesn’t affect the representation learned by any layer except the head, and that the learned representations and features are mostly reused as is for the different tasks.
B. M INI I MAGE N ET -5 WAY -1 SHOT F REEZING AND CCA O VER T RAINING
Figure 9 shows that from early on in training, on MiniImageNet-5way-1shot, that the CCA similarity between activations pre and post inner loop update is very high for all layers but the head.
We further see that the validation set accuracy suffers almost no decrease if we remove the inner loop updates and freeze all layers but the head.
This shows that even early on in training, the inner loop appears to have minimal effect on learned representations and features.
This supplements the results seen in
Figure 3 on MiniImageNet-5way-5shot.
[Figure]
C ANIL A LGORITHM : M ORE D ETAILS
In this section we provide more details about the ANIL algorithm, including an example of the ANIL update, implementation details and further experimental results.
C. A N E XAMPLE OF THE ANIL U PDATE
Consider a simple, two layer linear network with a single hidden unit in each layer: ˆ y ( x ; θ )
=
θ 2 ( θ 1 x ) .
In this example, θ 2 is the head .
Consider the 1-shot regression problem, where we have access to examples (cid:110)
( x ( t ) 1 , y ( t ) 1 ) , ( x ( t ) 2 , y ( t ) 2 ) (cid:111) for tasks t = 1 , . . .
, T .
Note that ( x ( t ) 1 , y ( t ) 1 ) is the (example, label) pair in the meta-training set (used for inner loop adaptation – support set), and ( x ( t ) 2 , y ( t ) 2 ) is the pair in the meta-validation set (used for the outer loop update – target set).
In the few-shot learning setting, we ﬁrstly draw a set of N tasks and labelled examples from our meta-training set: (cid:110) ( x (1)1 , y (1)1 ) , . . .
, ( x ( N ) 1 , y ( N ))
(cid:111) .
Assume for simplicity that we only apply one gradient step in the inner loop.
The inner loop updates for each task are thus deﬁned as follows:
[Equation]
where L ( · , · ) is the loss function, (e.g. mean squared error) and θ ( t ) i refers to a parameter after inner loop update for task t .
The task-adapted parameters for MAML and ANIL are as follows.
Note how only the head parameters change per-task in ANIL:
[Equation]
In the outer loop update, we then perform the following operations using the data from the metavalidation set:
[Equation]
Considering the update for θ 1 in more detail for our simple, two layer, linear network (the case for θis analogous), we have the following update for MAML:
[Equation]
For ANIL, on the other hand, the update will be:
[Equation]
Note the lack of inner loop update for θ 1 , and how we do not remove second order terms in ANIL (unlike in ﬁrst-order MAML); second order terms still persist through the derivative of the inner loop update for the head parameters.
C. ANIL L EARNS A LMOST I DENTICALLY TO MAML
We implement ANIL on MiniImageNet and Omniglot, and generate learning curves for both algorithms in Figure 10.
We ﬁnd that learning proceeds almost identically for ANIL and MAML, showing that removing the inner loop has little effect on the learning dynamics.
C. ANIL AND MAML L EARN S IMILAR R EPRESENTATIONS
We compute CCA similarities across representations in a MAML seed and an ANIL seed, and then plot these against the same MAML seed representation compared to a different MAML seed (and similarly for ANIL).
We ﬁnd a strong correlation between these similarities (Figure 11), which suggests that
MAML and ANIL are learning similar representations, despite their algorithmic differences.
(ANIL and MAML are about as similar to each other as two ANILs are to each other, or two MAMLs are to each other.)
C. ANIL I MPLEMENTATION D ETAILS
Supervised Learning Implementation:
We used the TensorFlow MAML implementation opensourced by the original authors (Finn et al., 2017).
We used the same model architectures as in the original MAML paper for our experiments, and train models 3 times with different random seeds.
All models were trained for 30000 iterations, with a batch size of 4, 5 inner loop update steps, and an inner learning rate of 0.01.
10 inner gradient steps were used for evaluation at test time.
Reinforcement Learning Implementation:
We used the open source PyTorch implementation of MAML for RL 1 , due to challenges encountered when running the open-sourced TensorFlow [Figure] implementation from the original authors.
We note that the results for MAML in these RL domains do not exactly match those in the original paper; this may be due to large variance in results, depending on the random initialization.
We used the same model architecture as the original paper (two layer
MLP with 100 hidden units in each layer), a batch size of 40, 1 inner loop update step with an inner learning rate of 0.1 and 20 trajectories for inner loop adaptation.
We trained three MAML and ANIL models with different random initialization, and quote the mean and standard deviation of the results.
As in the original MAML paper, for RL experiments, we select the best performing model overiterations of training and evaluate this model at test time on a new set of tasks.
[Figure]
r i
[Figure]
y
[Figure]
I
[Figure]
[Figure]
Training: 5way-1shot
Training: 5way-5shot
Mean (s)
Median (s)
Speedup
Mean (s)
Median (s)
Speedup
MAML 0.
[Table]
C. ANIL IS C OMPUTATIONALLY S IMPLER T HAN MAML
Table 6 shows results from a comparison of the computation time for MAML, First Order MAML, and ANIL, during training and inference, with the TensorFlow implementation described previously, on both MiniImageNet domains.
These results are average time for executing forward and backward passes during training (above) and a forward pass during inference (bottom), for a task batch size of
[Table]
and a target set size of 1. Results are averaged over 2000 such batches. Speedup is calculated relative to MAML’s execution time. Each batches’ images were loaded into memory before running the TensorFlow computation graph, to ensure that data loading time was not captured in the timing.
Experiments were run on a single NVIDIA Titan-Xp GPU.
During training, we see that ANIL is as fast as First Order MAML (which does not compute second order terms during training), and about 1.7x as fast as MAML.
This leads to a signiﬁcant overall training speedup, especially when coupled with the fact that the rate of learning for these ANIL and
MAML is very similar; see learning curves in Appendix C.2.
Note that unlike First Order MAML,
ANIL also performs very comparably to MAML on benchmark tasks (on some tasks, First Order
MAML performs worse (Finn et al., 2017)).
During inference, ANIL achieves over a 4x speedup over MAML (and thus also 4x over First Order MAML, which is identical to MAML at inference
[Table]
D F URTHER R ESULTS ON THE N ETWORK H EAD AND B ODY D. T RAINING R EGIMES FOR THE N ETWORK B ODY
We add to the results of Section 5.2 in the main text by seeing if training a head and applying that to the representations at test time (instead of the NIL algorithm) gives in any change in the results.
As might be predicted by Section 5.1, we ﬁnd no change the results.
More speciﬁcally, we do the following:
• We train MAML/ANIL networks as standard, and do standard test time adaptation.
• For multiclass training, we ﬁrst (pre)train with multiclass classiﬁcation, then throw away the head and freeze the body.
We initialize a new e.g. 5-class head, and train that (on top of the frozen multiclass pretrained features) with MAML.
At test time we perform standard adaptation.
• The same process is applied to multitask training.
• A similar process is applied to random features, except the network is initialized and then frozen.
The results of this, along with the results from Table 5 in the main text is shown in Table 7.
We observe very little performance difference between using a MAML/ANIL head and a NIL head for each training regime.
Speciﬁcally, task performance is purely determined by the quality of the features and representations learned during training, with task-speciﬁc alignment at test time being (i) unnecessary (ii) unable to inﬂuence the ﬁnal performance of the model (e.g. multitask training performance is equally with a MAML head as it is with a NIL-head.)
D. R EPRESENTATIONAL A NALYSIS OF D IFFERENT T RAINING R EGIMES
In Table 8 we include results on using CCA and CKA on the representations learned by the different training methods.
Speciﬁcally, we studied how similar representations of different training methods were to MAML training, ﬁnding a direct correlation with performance – training schemes learning representations most similar to MAML also performed the best.
We computed similarity scores by averaging the scores over the ﬁrst three conv layers in the body of the network.
[Table]