Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
Authors: Aniruddh Raghu
Maithra Raghu
Samy Bengio
Oriol Vinyals
Venue:  (2020)
Keywords: deep learning analysis
representation learning
meta-learning
few-shot learning

Abstract: An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, from which the inner loop can efficiently learn new tasks. Despite MAML's popularity, a fundamental open question remains -- is the effectiveness of MAML due to the meta-initialization being primed for rapid learning (large, efficient changes in the representations) or due to feature reuse,  with the meta initialization already containing high quality features? We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor. This leads to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML where we remove the inner loop for all but the (task-specific) head of the underlying neural network. ANIL matches MAML's performance on benchmark few-shot image classification and RL and offers computational improvements over MAML. We further study the precise contributions of the head and body of the network, showing that performance on the test tasks is entirely determined by the quality of the learned features, and we can remove even the head of the network (the NIL algorithm). We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly.

Published as a conference paper at ICLR 2020
RAPID LEARNING OR FEATURE REUSE? TOWARDS UNDERSTANDING THE EFFECTIVENESS OF MAML
Aniruddh Raghu
Maithra Raghu
Samy Bengio Google Brain
MIT
Cornell University & Google Brain ma i thrarrggaa com
eta
Oriol Vinyals DeepMind
ABSTRACT
An important research direction in machine learning has centered around develop- ing meta-learning algorithms to tackle few-shot learning, An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that con- sists of two optimization loops, with the outer loop finding meta- meta-initidn initializzaiion from which the inner loop can efficiently learn new tasks. Despite MAML's popu- larity, a fundamental open question remains - is the effectiveness of MAML due to the mnta-initialization being primed for rapid learning (large, efficient changes in the representations) or due feature reuse, with the meta-initialization already containing high quality features? We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor. This leads to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML where we remove the inner loop for all but the (task-specific) head of the underlying neural network. ANIL matches MAML'S performance on benchmark few-shot image classification and RL and offers computational improvements over MAML. We further study the precise contributions of the head and body of the network, showing that performance on the test tasks is entirely determined by the quality of the learned features, and we can remove even the head of the network (the NII. algorithm). We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly.
INTRODUCTION
central problem in machine learning is few-shot learning, where new tasks must be learned with very limited number of labelled datapoints. A significant body of work has looked at tackling this challenge using meta-learning approaches (Koch et al., 2015) Vinyals et al.] [2066] Snell et al.) Nichol and Schulman 2018). Broadly speaking, these approaches define a family of tasks, some of which are used for training and others solely for evaluation. proposed meta leaa-ie algorithm then looks learning properties that generalize across the different training tasks, and result in fast and efficient learning of the evaluation tasks.
2017 Finn et al.) 2017 Santoro al., 2016,] Ravi and Larochelle| Nichol
One highly successful meta-learning algorithm has been Model Agnostic Meta-Learning (MAML) (Finn al. (017). At a high level, the MAML algorithm is comprised of two optimization loops. The outer loop (in the spirit of meta-learning) aims to find an effective meta-initialization. from which the inner loop can perform efficient adaptation - optimize parameters to solve new tasks with very few labelled examples. This algorithm, with deep neural networks as the underlying model, has been highly influential, with significant follow on work, such as first order variants (Nichol and Schulman 2018), probabilistic extensions (Finn et al. 2018), augmentation with generative modelling (Rusu et al., 2018), and many others (Hsu et al. 2018: Finn and Levine 2017 Grant et al. 2018 Triantafillou al. 2019
"Equal contribution

---

Published as a conference paper at ICLR 2020
Despite the popularity of MAML, and the numerous followups and extensions, there remains fundamental open question on the basic algorithm. Does the mettainiitaliizatonn learned by the outer loop result in rapid learning unseen test tasks (efficient but significant changes in the representations) or is the success primarily due to feature reuse (with the meta-initialization already providing high quality representations)? In this paper, we explore this question and its many surprising consequences. Our main contributions are:
We perform layer freezing experiments and latent representational analysis of MAMI,, finding that feature reuse is the predominant reason for efficient learning.
Based on these results, we propose the ANIL (Almost No Inner Loop) algorithm, significant simplification to MAML that removes the inner loop updates for all but the head (final layer) of a neural network during training and inference. ANIL performs identically to MAML on standard benchmark few-shot classification and RL tasks and offers computational benefits over MAMI..
We study the effect of the head of the network, finding that once training is complete, the head can be removed, and the representations can be used without adaptation to perform unseen tasks, which we call the No Inner Loop (NIL) algorithm.
We study different training regimes, e.g. multiclass classification, multitask learning, etc, and find that the task specificity of MAML/ANIL at training facilitate the learning of better features. We also find that multitask training, popular bascline with no task specificity, performs worse than random features.
We discuss rapid learning and feature reuse the context of other meta-learning approaches.
RELATED WORK
MAML Finn et al. 2017) is a highly popular meta- meta-learning algorithm for few-shot learning, achieving competitive performance on several benchmark few-shot learning problems (Koch et 20.55 Vinyals
et al. 2016, Snell et al. 2017 Santoro et al. 2016; Ravi and I Larochelle, 60 and Schulman 2018). is part of the family of optimization- based meta-learning algorithms, with other members of this family presenting variations around how to learn the weights of the task-specific classifier. For
example Lee and Choi ;); Gordon et al. (2018); Bertinetto et al. (2018): Lee et al. (2019); ZZhou 2018) first learn functions embed the support set and target examples of few-shot learning task, before using the test support set to learn task specific weights to use on the embedded target examples. Harrison al. (2018) also proceeds similarly, using a Bayesian approach. The method of Bao et al. 2019) explores a related approach, focusing on applications in text classification.
al.
Of these optimization- hased meta-learning algorithms, MAML has been especially influential, inspir- ing numerous direct extensions in recent literature (Antoniou et al. 2018 Finn et al., 2018 Grant ct al. 20188 Rusu al., 2018). Most of these extensions critically rely on the core structure of the MAML algorithm, incorporating an outer loop (for meta-training), and an inner loop (for task-specific adaptation), and there is little prior work analyzing why this central part of the MAML algorithm is practically successful. In this work, we focus on this foundational question, examining how and why MAML leads to effective few-shot learning. To do this, we utilize analytical tools such as Canonical Correlation Analysis (CCA) (Raghu et al. 2017: Morcos et al., 2018) and Centered Kernel Alignment (CKA) (Kornblith et al. 2019) to study the neural network representations learned with the MAMI. algorithm, which also demonstrates MAML's ability to learn effective features for few-shot learning.
Insights from this analysis lead to simplified algorithm, ANIL, which almost completely removes the inner optimization loop with no reduction in performance. Prior works (Zintgraf et al. 20188 Javed and White 2019) have proposed algorithms where some parameters are only updated in the outer loop and others only in the inner loop. However, these works are motivated by different questions, such as improving MAML's performance or learning better representations, rather than analysing rapid learning feature reuse in MAMI...
3 MAML, RAPID LEARNING, AND FEATURE REUSE
Our goal is to understand whether the MAML algorithm efficiently solves new tasks due to rapid learning or feature reuse. In rapid learning, large representational and parameter changes occur during
2

---

Published as a conference paper at ICLR 2020
Rapid Learning
Feature Reuse
Task
Task
0
Task
Task 2
05
Task
Task 3 gg
loop
loog
Figure 1: Rapid learning and feature reuse paradigms. In Rapid Learning, outer loop training leads to parameter setting that is well-conditioned for learning, and inner loop updates result significant task specialization. Feature Reuse, the outer loop leads to parameter values corresponding to reusable features, from which the parameters do not move significantly in the inner loop.
adaptation to each new task as a result favorable weight conditioning from the meta-initialization. In feature reuse, the petaiinitializztion already contains highly useful features that mostly be reused as is for new tasks, so little task-specific adaptation occurs. Figure shows a schematic of these two hypotheses.
We start off by overviewing the details of the MAML algorithm, and then we study the rapid learning vs feature reuse question via layer freezing experiments and analyzing latent representations of models trained with MAML. The results strongly support feature reuse as the predominant factor behind MAML's success. Section 4 simplification of MAML, the ANIL algorithm, and in Section 6 we outline the connections to meta-learning more broadly.
we explore the consequences of this, providing significant
3.1
OVERVIEW OF MAMI.
The MAML algorithm finds an initialization for neural network so that new tasks can be learnt with very few examples (k examples from each class for k-shot learning) via two optimization loops:
Outer Loop: Updates the initialization of the neural network parameters (often called the meta-initializztion) to a setting that enables fast adaptation to new tasks.
Inner Loop: Performs adaptation: takes the outer loop initialization, and, separately for each task, performs a few gradient updates over the k labelled examples (the support set) provided for adaptation.
More formally, we first define our base model to be a neural network with meta-initialization parameters 0: let this represented by fo- We have have a distribution D over tasks, and draw batch (?i....,TT Tp} of B tasks from D. For each task Ti, we have support set of examples Sr, which are used for inner loop updates, and a target set of examples ZT,, which are used for outer loop updates. Let 0!0)) signify 0 after gradient updates for task Ti. and let 0% 0. In the inner loop, during each update, we compute
g( ((6) - aV 6th) Lsn, (fom (())
(1)
for fixed across all tasks, where Lsn, (Jaw inner loop updates. We then define the meta-loss
(0)) (o)L is the loss the support set of Th after
Lmeta (0)
Σ L2n, (0)) (
where Cz., (fa (0)) is the loss on the target set of Ti after m inner loop updates, making clear the dependence of fae on 0. The outer optimization loop then updates 0
0 1) L (0)

---

Published as a conference paper at ICLR 2020
Freeze layers
MinilnageNerr-Say-llhhoo
MinimaageNet:5way-5hhoo
None 1
46.9 + 0.2
63.1 = 0.4
46.5 + 0.3
63.0 + 0.6
1,2
46.4 + 0.4
62,6 + 0.6
1,2,3 1,2,3,4
46.3 + 0.4
61.2 0.5
46.3 + 0.4
61.0 - 0.6
Table 1: Freezing successive layers (preventing inner loop adaptation) does not affeec accuracy, support- ing feature reuse. test the amount feature reuse happening accuracy of the model when we freeze (prevent inner loop adaptation) a contiguous block of layers at test time, We find that freezing even all four convolutional layers of the network (all layers except the network head) hardly affects accuracy. This strongly supports the feature reuse hypothesis; layers don't 't have to change rapidly adaptation time; they already contain good features from the meta-initialization.
inner loop adaptation. we test
At test time, we draw unseen tasks (IT(test) " T(testt) ate the loss and accuracy on Zysteest after inner loop adaptation using S Systees) (e.g. loss is
from the task distribution, and evalu-
L Eegee (Fo12
3.2 RAPID LEARNING OR FEATURE REUSE?
We turn attention to the key question: Iy MAML's efficacy predominantly due to rapid learning or feature reuse? In investigating this question, there an important distinction between the head (final layer) of the network and the earlier layers (the body of the network). In each few-shot learning task, there is a different alignment between the output neurons and classes. For instance, in task 7i, the (wlog) five output neurons might correspond, in order, to the classes (dog, cat, frog, cupcake, phone). while for a different task, T2, they might correspond, in order, to (airplane, frog, boat, car, pumpkin). This means that the head must necessarily change for each task to learn new alignment, and for the rapid learning vs feature reuse question, we are primarily interested in the behavior of the body of the network. We return to this in more detail in Section where we present an algorithm (NIL) that does not use a head test time.
To study rapid learning vs feature reuse in the network body, we perform two sets of experiments: (I) We evaluate few-shot learning performance when freezing parameters after MAML training, without test time inner loop adaptation; (2) We use representational similarity tools to directly analyze how much the network features and representations change through the inner loop. We use the MinilmageNet dataset, a popular standard benchmark for few-shot learning, and with the standard convolutional architecture in Finn et al. (2017). Results averaged over three random seeds. Full implementation details are in Appendix B
3.2.1 FREEZING LAYER REPRESENTATIONS
To study the impact of the inner loop adaptation, we freeze a contiguous subset layers of the network, during the inner loop at test time (after using the standard MAML algorithm, incorporating both optimization loops, for training). In particular, the frozen layers are not updated at all to the test time task, and must reuse the features learned by the meta- intttaiiiiiiizniinnn that the outer loop converges to. We compare the few-shot learning accuracy when freezing to the accuracy when allowing inner loop adaptation.
Results are shown in Table We observe that even when freezing all layers in the network body, performance hardly changes. This suggests that the meta-initialization has already learned good enough features that can be reused as is, without needing to perform any rapid learning for each test time task.
3.2.2 REPRESENTATIONAL SIMILARITY EXPERIMENTS
We next study how much the latent representations (the latent functions) learned by the neural network change during the inner loop adaptation phase. Following several recent works (Raghu et al. 2017 Saphra and Lopez! 2018 Morcos et al., 2018 Maheswaranathan al.! 2019 Raghu al.
4

---

Published as a conference paper at ICLR 2020
CCA Similarity Before and After Inner Loop Adaptation
CKA Similarity Before and After
Inner Loop Adaptation
1.0
1.0
CCA. Similariy
0.8
CK4 Simil antty
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
Conv1
Conv2
Conv3
Comvel
Head
Conv1
Conv2
Conv3
Convel
Head
Figure 2: High CCA/CKA similarity between representations before and after adaptation for all layers except the head. We compute CCA/CKA similarity between the representation of layer before the inner loop adaptation and after adaptation. We observe that for all layers except the head, the CCA/CKA similarity almost 1, indicating perfect similarity. This suggests that these layers do not change much during adaptation, but mostly perform feature reuse. Note that there is slight dip in similarity in the higher cony layers (e.g. conv3, conv4); this is likely because the slight representational differences in conv I, conv2 have a compounding effect the representations of conv3, conv4. The head of the network must change significantly during adaptation. and this is reflected in the much lower CCA/CKA similarity.
2019; Gotmare et al. 2018 Bau et al. 2018) we measure this by applying Canonical Correlation Analysis (CCA) to the latent representations of the network. CCA provides a way to the compare representations of two (latent) layers L1, L2 of a neural network, outputting similarity score between (not similar at all) and 1 (identical). For full details, see Raghu al. (017);; Morcos al. (2018). In our analysis, we take L1 to be layer before the inner loop adaptation steps, and L2 after the inner loop adaptation steps. We compute CCA similarity between L1. L2. averaging the similarity score across different random seeds of the model and different test time tasks. Full details are in Appendix
B.2
The result is shown in Figure left pane. Representations in the body of the network (the convo- lutional layers) are highly similar, with CCA similarity scores of > 0.9, indicating that the inner loop induces little no functional change. By contrast, the head of the network, which does change significantly in the inner loop, has a CCA similarity of less than 0.5. To further validate this, we also compute CKA (Centered Kernel Alignment) (Kornblith al. 2019) (Figure (rigurerrrigt)) right), another similarity metric for neural network representations, which illustrates the same pattern. These repre- sentational analysis results strongly support the feature reuse hypothesis, with further results in the Appendix, Sections B.3 and B.4 providing yet more evidence.
3.2.3 FEATURE REUSE HAPPENS EARLY IN LEARNING
Having observed that the inner loop does not significantly affect the learned representations with a fully trained model, we extend our analysis to see whether the inner loop affects representations and features carlier on in training. We take MAML models at 10000, 20000, and 30000 iterations into training, perform freezing experiments (as in Section 3.2.I and representational similarity experiments (as in Section (.2.2
Results in Figure 3 show the same patterns from early in training. with CCA similarity between activations pre and post inner loop update on MinilmageNett-Sayy-See
being very high for the body (just like Figure 2. and similar to Table [] test accuracy remaining approximately the same when freezing contiguous subsets of layers, even when freezing all layers of the network body. This shows that even early in training, significant feature reuse is taking place, with the inner loop having minimal effect on learned representations and features. Results for Ishot MinilmageNet are in Appendix B.5 and show very similar trends.
THE ANIL (ALMOST NO INNER LOOP) ALGORITHM
In the previous section we saw that for all layers except the head of the neural network, the meta- initialization learned by the outer loop of MAML results in very good features that can be reused
5

---

Published as a conference paper at ICLR 2020
Test Set Accuracy Freezing Consecutive Layers Inner Loop
CCA Similarity Before and After
Inner Loop Adaptation
Iteration 10000: 61.3%; (1.2.3,4) 58.7%
Iteration 20000; None: 63.0%, (1.2.3,4): 61.3%
0.9
Test Accuracy
65
Iteration 30000: 63.1%; (1.2.3.4);
CCA Simlar
0.6
80
0.5
Iteration 10000
Iteration 20000
Iteration 30000
55
0.3
None
1.2
1,2.3
1,2,3,4
Conv1
Conv2
Conv3
Conve
Head
Frozen
Figure 3: Inner loop updates have little effect on learned representations from early on in learning. Left pane:
we freeze contiguous blocks of layers (no adaptation at test time), on SelaiiaggeeeeS---SSaaeleeae and almost identical performance. Right pane: representations of all layers except the head highly similar pre/post adaptation - i.e. features are being reused. This is true from early (iteration 10000) training.
MAML
ANIL
TT Taskn,
OL., (0)
OT, Taskr,
I1 Og 8ln abrus
Dhy,!0) a
50g
Greed
101.02.
(9L.02. Onsea)
OT.
Or
Taskr
Taskr,
Taskra
UT
Taskr,
BTa
Figure 4: Schematic of MAML and ANIL algorithms. The difference between the MAML and ANIL algorithms: in MAML (left), the inner loop (task-speciffe gradient updates applied to all parameters 8, which are initialized with the meta-initializzttinn from the outer loop. ANIL (right), only the parameters corresponding to network head Ohead are updated by the inner loop, during training and testing.
as is on new tasks, Inner loop adaptation does not significantly change the representations of these layers, even from early on in training. This suggests natural simplification of the MAML algorithm: the ANIL (Almost No Inner Loop) algorithm.
In ANIL, during training and testing, we remove the inner loop updates for the network body, and apply inner loop adaptation only to the head. The head requires the inner loop to allow align to the different classes in each task. In Section 5.1 we consider another variant, the NIL (No Inner Loop) algorithm, that removes the head entirely at test time, and uses learned features and cosine similarity to perform effective classification, thus avoiding inner loop updates altogether.
For the ANIL algorithm, mathematically, let (01, ..., ft) be the (meta-initialization) parameters for the 1 layers of the network. Following the notation of Section 3.1 let o( be the parameters after
m inner gradient updates for task TG. In ANII., we have that:
(b)) (0)
(0))
a (00) Cs. (Socs
Le. only the final layer gets the inner loop updates. As before, we then define the meta-loss, and compute the outer loop gradient update. The intuition for ANIL arises from Figure 3 where observe that inner loop updates have little effect on the network body even early in training, suggesting the possibility of removing them entirely. Note that this distinct to the freezing experiments. where we only removed the inner loop at inference time. Figure Secureppresee presents the difference between MAML and ANIL, and Appendix C.T considers simple example of the gradient update in ANIL, showing how the ANIL update differs from MAML.
Computational benefit of ANIL: As ANIL almost has no inner loop, it significantly speeds up both training and inference. We found an average speedup of 1.7x per training iteration over MAML and average speedup of 4.1x per inference iteration. In Appendix C.5 we provide the full results.

---

Published as a conference paper at ICLR 2020
Method Onniglot-20way--ssho
Onmiglot-20way--5hot
MinilmageNett-5ayy-shho
MinilmageNet--way-55hho
MAML
93.7 0.7 96.2 + 0.5
96.4 + 0.1 98.0 + 0.3
46.9 - 0.2 46.7 + 0.4
63.1 0.4
ANIL
61.5 + 0.5
Method
HalfCheeeah-Directiin HalfCheetah- Velocity
2D-Navigation
MAML
170.4 + 21.0
-139.0 + 18.9
-20.3 + 3.2
ANIL
363.2 + 14.8
-120.9 + 6.3
-20.1 + 2.3
Table 2: ANIL matches the performance of MAML on few-shot image classification and RL. On bench- mark shot classification tasks MAML and ANIL have comparable accuracy, and also comparable average return (the higher the better) standard RI. tasks (Finn et al. 2017).
MinilmageNett-waay
Loss curves
Accuracy curves
MAML train less
0.75
val
ANIL train loss
0.70
6.0
ANIL val loss
Accuracy
0.65
LO5s
5.5
5.0
0.55
4.5
0.50
MAML train acc
MAML acc
4.0
ANIL train
3.5
ANIL val acc
0.40
0
5000
10000
15000
20000 25000
30000
0
5000
10000 15000 20000 25000 30000 Training iteration
Training iteration
Figure 5: MAML and ANIL learn very similarly. Loss and accuracy curves for MAML and ANIL on Selalmaageeee-SSa--Shhehhee illustrating how MAML and ANIL behave similarly through the training process.
Results of ANIL on Standard Benchmarks: We evaluate ANIL on few-shot image classification and RL benchmarks, using the same model architectures as the original MAML authors, for both supervised learning and RL. Further implementation details are in Appendix C.4 The results in Table (mean and standard deviation performance over three random initializations) show that ANIL matches the performance of MAML both few-shot classification (accuracy) and RL (average return, the higher the better), demonstrating that the inner loop adaptation of the body is unnecessary for learning good features.
MAML and ANIL Models Show Similar Behavior: MAML and ANIL perform equally well on few-shot learning benchmarks, illustrating that removing the inner loop during training does not hinder performance. To study the behavior of MAML and ANIL models further, we plot learning curves for both algorithms on SinilmageNNet5Say--S55 accuracy for both algorithms look very similar throughout training. We also look at CCA and CKA scores of the representations learned by both algorithms, Table 3 We observe that MAMI.- ANIL representations have the same average similarity scores MAML-MAML and ANIL-ANIL representations, suggesting both algorithms learn comparable features (removing the inner loop doesn't 't change the kinds of features learned)) Further learning curves and representational similarity
Figure We see that loss and
results are presented in Appendices C.2 and C33
5 CONTRIBUTIONS OF THE NETWORK HEAD AND BODY
So far, we have seen that MAML predominantly relies on feature reuse, with the network body (all layers except the last layer) already containing good features at meta- intttaiiiiiliiitinnn. We also observe that such features can be learned even without inner loop adaptation during training (ANIL algorithm). The head, however, requires inner loop adaptation enable task specificity.
7

---

Published as a conference paper at ICLR 2020
Model Pair
CCA Similarity
CKA Similarity
MAML-MAML 0.51
0.83
ANIL-ANIL
0.51
0.86
ANIL-MAML
0.50
0.83
Table 3: MAML and ANIL models learn comparable representations. Comparing CCA/CKA similarity scores the of MAML-ANIL representations (averaged over network body). and MAML-MAML and ANIL. ANIL similarity scores (across different random seeds) shows algorithmic differences between MAML/ANIL does not result in vastly different types of features learned.
Method
Ommiglot-20way-Isho
Ommiglot-20way-55hhg
MinilnageNee--way-lshoo
MinilmageNeet-Swayy5Shh
MAML
93.7 = 0.7 96.2 + 0.5
96.4 + 0.1 98.0 + 0.3
46.9 + 0.2
63.1 = 0.4 61.5 0.5 62.2 + 0.5
ANIL
46.7 + 0.4
NIL
96.7 + 0.3
98.0 + 0.04
48.0 + 0.7
Table 4: NIL algorithm performs as well as MAML and ANIL on few-shot image classification, Perfor- mance of MAML, ANIL, and NIL on few-shot image classification benchmarks. We see that with no test-time inner loop, and just learned features, NIL performs comparably to MAML and ANIL, indicating the strength of the learned features, and the relative lack of importance of the head at test time.
In this section, we explore the contributions of the network head and body. We first ask: How important is the head at test time, when good features have already been learned? Motivating this question that the features the body of the network needed no adaptation at inference time, so perhaps they are themselves sufficient to perform classification, with no head. In Section 5.1 we find that test time performance is entirely determined by the quality of these representations, and we can use similarity of the frozen veta-initiallzzatin representations to perform unseen tasks, removing the head entirely. We call this the NIL (No Inner Loop) algorithm.
Given this result, we next study how useful the head is at training (in ensuring the network body learns good features). We look at multiple different training regimes (some without the head) for the network body, and evaluate the quality of the representations. We find that MAML/ANIL result in the best representations, demonstrating the importance of the head during training for feature learning.
5.1 THE HEAD AT TEST TIME AND THE NIL (No INNER LOOP) ALGORITHM
We study how important the head and task specific alignment are when good features have already been learned (through training) by the meta inttaaiiiiializttinn. At test time, we find that the representations can be used directly, with no adaptation, which leads the No Inner Loop (NIL) algorithm:
1. Train shot learning model with ANIL/MAML algorithm standard. We use ANIL training.
2. At test time, remove the head of the trained model. For each task, first pass the labelled examples (support set) through the body of the network, to get their penultimate layer repre- sentations. Then, for a test example, compute cosine similarities between penultimate layer representation and those of the support set, using these similarities to weight the support set labels, as in Vinyals et al. (0016).
The results for the NIL algorithm, following ANIL training, on few-shot classification benchmarks are given in Table 4 Despite having no network head and no task specific adaptation, NIL performs comparably to MAML and ANIL. This demonstrates that the features learned by the network body when training with MAML/ANIL (and reused at test time) are the critical component in tackling these henchmarks.
5.2 TRAINING REGIMES FOR THE NETWORK BODY
The NIL algorithm and results of Section 5.1 lead to the question of how important task alignment and the head are during training to ensure good features. Here, we study this question by examining the quality of features arising from different training regimes for the body. We look (i) MAMI.
8

---

Published as a conference paper at ICLR 2020
Method
MinilmageNett-Swyy--hhhhe
MinilmageNet--wayy-Shooo
MAML training-Nin head
48.4 = 0.3
61.5 0.8
ANIL training-NII. head
48.0 + 0.7
62.2 + 0.5
Multiclass training-NIL head
39.7 0.3
54.4 + 0.5
Multitask training-NIL. head
26.5 + 1.1
34.2 + 3.5
Random features-NIL head
32.9 = 0.6
43.2 + 0.5
NIL training-No -IL head
38.3 + 0.6
43.0 + 0.2
Table 5: MAML/ANIL training leads superior features learned, supporting importance of head at training. Training with MAML/ANIL leads to superior performance over other methods which do not have task specific heads, supporting the importance of the head at training.
and ANIL training; (ii) multiclass classification, where all of the training data and classes (from which training tasks are drawn) are used to perform standard classification; (iii) multitask training, standard baseline, where no inner loop or task specific head is used, but the network is trained on all the tasks the same time; (iv) random random,fatured features, where the network not trained at all, and features are frozen after random initialization; (v) NIL at training time, where there is no head and cosine distance on the representations is used to get the label.
After training. we apply the NIL algorithm to evaluate test performance, and quality of features learned at training. The results are shown Table [ MAML and ANIL training performs best. Multitask training, which has no task specific head, performs the worst, even worse than random features (adding evidence for the need for task specificity at training to facilitate feature learning.) Using NIL during training performs worse than MAML/ANIL. These results suggest that the head is important at training to learn good features in the network body.
study test time performance variations from using MAML/ANIL head instead of NIL. finding (as suggested by Section [5.1] very little performance difference. Additional results on similarity between the representations of different training regimes is given in Appendix D.2
In Appendix D.I
FEATURE REUSE IN OTHER META-LEARNING ALGORITTMS
Up till now, we have closely examined the MAML algorithm, and have demonstrated empirically that the algorithm's success is primarily due to feature reuse, rather than rapid learning. We now discuss rapid learning vs feature reuse more broadly in meta-learning. By combining our results with an analysis of evidence reported in prior work, we find support for many meta-learning algorithms succeeding via feature reuse, identifying common theme characterizing the operating regime of much of current meta-learning.
6.1 OPTIMIZATION AND MODEL. BASED META-LFARNING
MAML falls within the broader class of optimization based meta-learning algorithms, which at inference time, directly optimize model parameters for new task using the support set. MAML has inspired many other optimization based algorithms. which utilize the same two-loop structure (Lee) and Choi 2018 Rusu et al., 2018 Finn et al. 2018). Our analysis into the feature reuse vs rapid learning question for this class of algorithms. Another broad class of meta-learning consists of model based algorithms, which also have notions of rapid learning and feature reuse.
far has thus yielded insights
In the model-based setting, the meta-learning model's parameters are not directly optimized for the specific task on the support set. Instead, the model typically conditions its output some representation of the task definition. One way to achieve this conditioning is to jointly encode the entire support set in the model's 's latent representation (Vinyals et al. (016 [ung al. 2018). enabling it to adapt to the characteristics each task. This constitutes rapid learning for model based meta-learning algorithms.

---

Published as a conference paper at ICLR 2020
An alternative to joint encoding would be encode each member of the support set independently. and apply a cosine similarity rule (as in Vinyals et al. (2016)) to classify an unlabelled example. This mode of operation is purely feature reuse do not information defining the task to directly influence the decision function.
If joint encoding gave significant test-time improvement over non-joint encoding, then this would suggest that rapid learning of the test-time task is taking place, as task specific information is being utilized to influence the model's decision function. However, on analyzing results in prior literature, this improvement appears to be minimal. Indeed, in e.g. Matching Networks (Vinyals et al. 2016). using joint encoding one reaches 44.2% accuracy on MinilmageNet-Sway-
whereas with independent encoding one obtains 41.2%: a small difference. More refined models suggest the gap is even smaller. For instance, in Chen al. (2019), many methods for one shot learning were re-implemented and studied, and baselines without joint encoding achieved 48.24% accuracy in MinilmageNet-Sway-- Ishot, whilst other models using joint encoding such as Relation Net (Sung) et al. 2018) achieve very similar accuracy of 49.31% % (they also report MAML, at 46.47%). As result, we believe that the dominant mode of "feature reuse" rather than "rapid learning" is what has currently dominated both MAML-stylee optimization based meta-learning and model based meta-learning.
7 CONCLUSION
In this paper, we studied a fundamental question on whether the highly successful MAML algorithm relies rapid learning feature reuse. Through series of experiments, found that feature reuse the dominant component in MAML's efficacy on benchmark datasets. This insight led to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML that has identical performance on standard image classification and reinforcement learning benchmarks, and provides computational benefits. We further study the importance of the head (final layer) of neural network trained with MAML, discovering that the body (lower layers) of network is sufficient for few-shot classification test time, allowing us remove the network head for testing (NIL algorithm) and still match performance. We connected our results to the broader literature in meta-learning, identifying feature reuse to a common mode of operation for other meta-learning algorithms also. Based off of conclusions, future work could look at developing and analyzing that perform more rapid learning, which may expand the datasets and problems amenable to these techniques. We note that our study mainly considered benchmark datasets, such as Omniglot and MinilmageNet. It is an interesting future direction to consider rapid learning and feature reuse in MAML on other few-shot learning datasets, such as those from Triantafillou et al. (2019).
meta-lcarning algorithms
10

---

Published as a conference paper at ICLR 2020
ACKNOWLEDGEMENTS
The authors thank Geoffry Hinton, Chelsea Finn, Hugo Larochelle and Chiyuan Zhang for helpful feedback the methods and results.
REFERENCES
Antreas Antoniou, Harrison Edwards, and Amos Storkey. How to train your MAML. arXiv preprint arXiv:1810.09500, 2018.
Yujia Bao, Menghua Wu, Shiyu Chang, and Regina Barzilay, Few-shot text classification with distributional signatures. arXiv preprint arXiv: arXivv/908.000 09039 2019.
Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass.
Identifying and controlling important neurons in neural machine translation. arXiv preprint arXiv:181..01152 2018.
Luca Bertinetto, Joao Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differen- tiable closed-form solvers. arXiv preprint arXiv: 080000600.00816, 2018.
Wei-Yu Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. arXiv preprint arXiv:/904.0000 04232, 2019.
Chelsea Finn and Sergey Levine. Meta-learning and universality: Deep representations and gradient descent can approximate any learning algorithm. arXiv preprint orXiv:/7100 1622, 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1126-1135. JMLR. org, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine Probabilistic model-agnostic meta-learning. In Advances in Neural Information Processing Systems, pages
2018.
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Turner. Meta- Learning probabilistic inference for prediction. arXiv preprint arXivv/805.09)) 2018.
Akhilesh Gotmare, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation. arXiv preprint arXiv:1810.13244 2018
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- 2018.
based meta-leariig as hierarchical bayes. arXiv preprint
James Harrison, Apoorva Sharma, and Marco Pavone. Meta-learning priors for efficient online bayesian regression. arXiv preprint arXh::1807.08112, 2018
Kyle Hsu, Sergey Levine, and Chelsea Finn. Unsupervised learning via meta-learning. arXiv preprint 1rXiv:/810.02333: 2018.
Khurram Javed and Martha White. Meta-learning representations for continual learning. arXiv preprint arXiv:/905. 12588, 2019.
Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2, 2015.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. arXiv preprint arXiv:1905.00414 2019.
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10657-10665,, 2019.
Yoonho Lee and Seungjin Choi. Gradient based meta-learning with learned layerwise metric and subspace. arXiv preprint arXiv: 1801. 055558 2018.

---

Published as a conference paper at ICLR 2020
Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John E Hoperoft. Convergent learning: Do different neural networks learn the same representations? In FE@ NIPS, pages 196-212, 2015.
Niru Maheswaranathan, Alex H. Willams, Matthew D. Golub, Surya Ganguli, and David Sussillo. Universality and individuality in neural dynamics across large populations of recurrent networks. arXiv preprini arXiv: (00i:/9900005499 2019.
Ari S Morcos, Maithra Raghu, and Samy Bengio. Insights on representational similarity in neural networks with canonical correlation. arXiv preprint (xXv::8806.05599 2018.
Alex Nichol and John Schulman. Reptile: A scalable metalcarning algorithm. arXiv preprint
arXiv:/803.0222 02999, 2, 2018.
Maithra Raghu, SVCCA Code and Tutorials, btst:s///gtthbb..ccm/oggg///////
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. In Advances in Neural Information Processing Systems, pages 6076-6085, 2017.
Maithra Raghu, Chiyuan Zhang, Jon Kleinberg, and Samy Bengio. Transfusion: Understanding 2019.
transfer learning with applications to medical imaging. arXiv preprint
Sachin Ravi and Hugo Larochelle. Optimization model for few-shot learning. 2016.
Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osin- dero, and Raia Hadsell. Meta-learning with latent embedding optimization. arXiv preprint 1007. 2018.
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta- learning with memory-a neural networks. In International Conference Machine Learning, pages 1842-1850, 2016.
Naomi Saphra and Adam Lopez. Understanding learning dynamics of language models with SVCCA. arXiv preprini arXiv: 181 1.00225, 2018
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4077-4087 2017.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 1199-1208 2018.
Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. Meta-Dataset: A dataset of datasets for learning learn from few examples. arXiv preprint arXiv: 03096, 2019.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et Matching Networks for one shot learning. In Advances in neural information processing systems, pages 3630-3638, 2016.
Liwei Wang, Lunjia Hu, Jiayuan Gu, Zhiqiang Hu, Yue Wu, Kun He. and John E. Hopcroft. To what extent do different neural networks learn the same representation: A Neuron Activation Subspace Match Approach. In NeurlPS 2018, 2018.
Fengwei Zhou, Bin Wu, and Zhenguo Li. Deep meta-learning learning learn in the concept space. 2018
arXiv preprint
Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning. arXiv preprint arXivv1810..08 2018.
12

---

Published as a conference paper at ICLR 2020
A FEW-SHOT IMAGE CLASSIFICATION DATASETS AND EXPERIMENTAL SETUPS
We consider the few-shot learning paradigm for image classification evaluate MAMI. and ANII.. We evaluate using two datasets often used for few-shot multiclass classification the Omniglot dataset and the MinilmageNett dataset.
Omniglot: The Omniglot dataset consists of over 1600 different handwritten character classes from 23 alphabets. The dataset is split on a character-level, so that certain characters are in the training set, and others in the validation set. We consider the 20-way 1-shot and 20-way 5-shot tasks on this dataset, where test time, wish our classifier discriminate between 20 randomly chosen character classes from the held-out set, given only or 5 labelled example(s) from each class from this set of 20 testing classes respectively. The model architecture used is identical that in the original MAML paper, namely: modules with a 3 x 3 convolutions and 64 filters with stride of 2, followed by batch normalization, and a ReLU nonlinearity. The Omniglot images are downsampled to 28 x 28, so the dimensionality of the last hidden layer is 64. The last layer is fed into a 20-way softmax. Our models are trained using a batch size of 16, 5 inner loop updates, and an inner learning rate of 0.1.
NinilmageNet: The Minilmagenet dataset was proposed by Ravi and Larochelle (2016), and consists of 64 training classes, 12 validation classes, and test classes. We consider the 5-way 1-shot and 5-way 5-shot tasks on this dataset, where the test-time task is to classify among 5 different randomly chosen validation classes, given only and labelled examples respectively. The model architecture is again identical that in the original paper: 4 modules with 3 3 convolutions and 32 filters, followed by batch normalization, ReLU nonlinearity, and 2 x max pooling. Our models trained using batch size of 4. 5 inner loop update steps, and inner learning rate of 0.01 used. 10 inner gradient steps are used for evaluation at test time.
B ADDITIONAL DETAILS AND RESULTS: FREEZING AND REPRESENTATIONAL SIMILARITY
In this section, we provide further experimental details and results from freezing and representational similarity experiments.
B. EXPERIMENTAL DETAILS
We concentrate on MinilmageNet for our experiments in Section 3.2 as it is more complex than Omniglot.
The model architecture used for our experiments is identical to that the original paper: 4 modules with 3 x 3 convolutions and 32 filters, followed by batch normalization, ReLU nonlinearity, and x 2 max pooling. Our models are trained using a batch size of 4, 5 inner loop update steps, and an inner learning rate of 0.01. 10 inner gradient steps used for evaluation test time. We train models 3 times with different random seeds. Models were trained for 30000 iterations.
B.2 DETAILS REPRESENTATIONAL SIMILARITY
22)) where L1, L2 are layers, and (() is neuron activation vector: the vector outputs of neuron (of layer I.y) over set of inputs X. It then finds linear combinations of the neurons in L1 and neurons in L2 so that the resulting activation vectors are maximally correlated, which is summarized correlation coefficient. Iteratively repeating this process gives similarity score (in [0, 1] with 1 identical and completely different) between the representations of L₁ and L2.
CCA takes in as inputs L, = (21") 2000 ..... "......))) 20) and L2 = (2 (2) (1)
the canonical
We apply this to compare corresponding layers of two networks, netl and net2, where netl and net2 might differ due to training step, training method (ANIL vs MAML) or the random seed. When comparing convolutional layers, described in Raghu perform the comparison over channels,
13

---

Published as a conference paper at ICLR 2020
MinilmageNNttSSayyy Sway-1shet Weight differences after finetune
MinilmagnnNetSSway--a
Weight differences after finetune
Cand
64-
04
62-
044
30300 1500e
20093
2500
30093
33500
20093
20000
30099
Tutting
Figure 6: Euclidean distance before and after finetuning MinilmageNet. We compute average (across tasks) Euclidean distance between the weights before and after inner loop adaptation, separately for different layers. We observe that all layers except for the final layer show very little difference before and after inner loop adaptation, suggesting significant feature reuse.
flattening out over all the spatial dimensions, and then taking the mean CCA coefficient. We average over three random repeats.
B.3 SIMILARITY BEFORE AND AFTER INNER LOOP WITH EUCLIDEAN DISTANCE
In addition to assessing representational similarity with CCA/CKA. we also consider the simpler measure of Euclidean distance, capturing how much weights of the network change during the inner loop update (task-speeific finetuning). We note that this experiment does not assess functional changes inner loop updates well as the CCA experiments do; however, they serve provide useful intuition.
We plot the per-layer average Euclidean distance between the initialization 0 and the finetuned weights ( across different tasks Tb, i.e.
N (1(0.) (0:) N
1
across different layers I, for MinilmageNet Figure We observe that very quickly after the start of training, all layers except for the last layer have small Euclidean distance difference before and after finetuning, suggesting significant feature reuse. (Note that this despite the fact that these layers have more parameters than the final layer.)
B.4 CCA SIMILARITY ACROSS RANDOM SEEDS
The experiment in Section 3.2.2 compared representational similarity of Ta and T.2 different points in training (before/after inner loop adaptation) but corresponding to the same random seed. To complete the picture, it useful study whether representational similarity across different random seeds is also mostly unaffected by the inner loop adaptation. This motivates four natural comparisons: assume layer L1 from the first seed, and layer L2 is from the second seed. Then we can compute the representational similarity between (L, pre, L2 pre), (L1 pre. L2 post), (L, post, L2 pre) and (L, post, L2 post), where pre/post signify whether we take the representation before after adaptation.
Prior work has shown that neural network representations may vary across different random seeds (Raghu et al. 2017 Morcos al.) 2018 Li et al. 2015 Wang et al. 2018), organically resulting in CCA similarity scores much less than 1. So to identify the effect of the inner loop the representation, we plot the CCA similarities of (i) (Li pre, L2 pre) against (L1 pre, L2 post) and (ii) (Li pre, La pre) against (L1 post, Le pre) and (iii) (Li pre, La pre) against (Lı post, L2 post) separately across the different random seeds and different layers. We then compute the line of best fit for each plot. If the line of best fit fits the data and is close to y this suggests that the inner loop adaptation doesn't affect the features much the similarity before adaptation is very close to the similarity after adaptation.
14

---

Published as a conference paper at ICLR 2020
(Pre, Pre) vs (Pre, Post) CCA Similarity Across Seeds
(Pre, Pre) vs (Post, Pre)
(Pre, Pre) vs (Post, Post)
CCA Similarity Across Seeds
CCA Similarity Across Seeds
0.70
LR 9-
CCA Similarity (Post 1. Pre 22
0.70
1Ry=1.014x
CCA Similarity (Post 1, Post 2]
0.70
y=0.988x
0.05
Conv2
Convi
0.05
Conv?
Canvi
0.65
Conv2
Conv1
0.60
Conva
0.60
Conv3
0.60
Conv3
CCA Similarity (Pre
0.53
Conv4
0.55
0.55
0.50
0.50
0.45
0.45
0.45
0.40
0.40
0.40
0.35
0.35
0.35
0.35
0.40 0.45 0.50
0.60 0.65
0.70
0.35
0.00
0.45
0.50
0.55 0.60
0.35
0.40
0.50 0.55
0.60 0.65 0.70
CCA Similarity (Pre 1,
CCA Similarity (Pre
2)
CCA Similarity (Pre 1, Pre 2)
Figure 7: Computing CCA similarity pre/post adaptation across different random seeds further demon- strates that the inner loop docsn't change representations significantly. We compute CCA similarity of L from seed 1 and L2 from seed 2, varying whether we take the representation pre (before) adaptation post (after) adaptation. To isolate the effect of adaptation from inherent variation the network representation across seeds, we plot CCA similarity of of the representations before adaptation against representations after adaptation in three different combinations: (i) (L1 pre, L2 pre) against (L1 pre, L1 post), (ii) (L1 pre. Le pre) against (L1 pre, L1 post) (iii) (L1 pre. L2 pre) against (L1 post, L2 post). We do this separately across different random seeds and different layers. Then, we compute line of best fit, finding that in all three plots, it is almost identical to y = v, demonstrating that the representation does not change significantly pre/post adaptation. Furthermore a computation of the coefficient of determination R² gives R² Rs 1, illustrating that the data is well explained by this relation. In Figure 8 we perform this comparison with CKA, observing the same high level conclusions.
(Pre, Pre) vs (Post, Post) CKA Similarity Across Seeds
OKA Similariyy (Pest Post
0.90
0.00
18 y-0 844x 0.14 Convi
0.75
Comv2
Comva
0.70
Convll
0.75
0.95
Similarity (Pre
Figure 8: We perform the same comparison as in Figure but with CKA instead. There is more variation in the similarity scores, but we still see strong correlation between (Pre, Pre) and (Post, Post) comparisons, showing that representations do not change significantly over the inner loop.
The results are shown in Figure 7 In all of the plots, we see that the line best fit is almost exactly y (even for the pre/pre vs post/post plot, which could conceivably be more different as both seeds change) and a computation of the coefficient of determination R² gives R for all three plots. Putting this together with Figure 2, can conclude that the inner loop adaptation step doesn't 't affect the representation learned by any layer except the head, and that the learned representations and features are mostly reused as is for the different tasks.
B.5 MINIIMAGENET-SWANN SHOT FREEZING AND CCA OVER TRAINING
Figure Dishoss that from early on in training, on Seaellaageeet-Sway-laeeeeee between activations pre and post inner loop update is very high for all layers but the head. We further see that the validation set accuracy suffers almost no decrease if we remove the inner loop updates and freeze all layers but the head. This shows that even early on in training, the inner loop appears to have minimal effect on learned representations and features. This supplements the results seen in Figure 3 on MinillmgeNNet-5way-55e
that the CCA similarity
15

---

Published as a conference paper at ICLR 2020
Test Set Accuracy Freezing Consecutive Layers in Inner Loop
CCA Similarity Before and After
Inner Loop Adaptation
50
0.9
Test Accuracy
CCA Similarity
45
0.7
Iteration 45.4%; (1,2,3,4):
0.5
Iteration 20000; 46.8%; (1,2,3,4): 46.9%
Iteration 20000
0.4
Iteration 30000;
None: 46.9%; (1,2,3,4): 46.3%
Iteration 30000
0.3
None
1.2
1.2.3
1.2.3.4
Conv1
Conv2
Conv3
Conv4
Head
Layers Frozen
Layer
Figure 9: Inner loop updates have little effect on learned representations from early on learning. We consider freezing and representational similarity experiments for MouilmmgeNet-Sway Ishot. We see that early on in training (from as few as 10k iterations in), the inner loop updates have little effect on the learned representations and features, and that removing the inner loop updates for all layers hut the head have little-to-no impact the validation set accuracy.
C ANIL ALGORITHM: MORE DETAILS
In this section, we provide more details about the ANIL algorithm, including an example of the ANIL update, implementation details, and further experimental results.
C.I AN EXAMPLE OF THE ANII UPDATE
Consider simple, two layer linear network with single hidden unit in each layer: j(x; 0) = 02 (0)
In this example, 0g is the head. Consider the 1-shot regression problem, where we have access to
examples (ar V1 (x)) 2/22 } for tasks t 1,.... T. Note that (x]²) (f(')) is the (example,
(t) (t)
(t)
label) pair in the meta- veta-training set (used for inner loop adaptation support set), and (xl)) V/2 (1) is pair in the meta-validation set (used for the outer loop update- target set).
In the few-shot learning setting, we firstly draw set of N tasks and labelled examples from our meta- training set: [z] v1 ( (x (M) .91 } }. Assume for simplicity that we only apply one
gradient step in the inner loop. The inner loop updates for each task thus defined as follows:
(4) O), vi
((0) + 0
(1)
00, (L(5(=1)) (0)
の 0g
002
where L(.,)) -) is the loss function, (e.g. mean squared error) and 0( refers to a parameter after inner loop update for task i.
The task-adapted parameters for MAML. and ANIL are as follows. Note how only the head parameters change per-task in ANIL:
(t) MAML
10/00 0501
(3)
(8)
Oz 05
ANIL
In the outer loop update, we then perform the following operations using the data from the meta- validation set:
16

---

Published as a conference paper at ICLR 2020
N
aL(5(12 (1)
0 20)
01 t 01
(5)
001
N aL(j(a O y2
82 t 82
(6)
002
Considering the update for 0, in more detail for our simple, two layer, linear network (the case for 0₂
analogous), we have the following update for MAML.:
aL(i(12 (2)
MAMI ),32 (()
01 01 Σ
001
([0,-
aL ( 0). v1
(9)
0L ( 0),
(1)
(x02 10, ( =
0
122
(8)
062
001
For ANIL., on the other hand, the update will be:
N
aL(i(=2
(t)
(tt)
ANNN))) ).22 (1)
01 - 01
Σ
(9)
00,
((0=
(L(((Z)
:).
0a22 19 (
01 -02
(10)
002
Note the lack of inner loop update for 0,, and how we do not remove second order terms in ANIL (unlike in first-order MAML); second order terms still persist through the derivative of the inner loop update for the head parameters.
C.2 ANIL LEARNS ALMOST IDENTICALLY TO MAML
We implement ANIL on MinilmageNet and Omniglot, and generate learning curves for both algo- rithms in Figure 10 We find that learning proceeds almost identically for ANIL and MAML, showing that removing the inner loop has little effect on the learning dynamics.
C.3 ANIL AND MAML LEARN SIMILAR REPRESENTATIONS
We compute CCA similarities across representations in MAML seed and an ANIL seed, and then plot these against the same MAML seed representation compared to a different MAML seed (and similarly for ANIL). We find a strong correlation between these similarities (Figure 11}, which suggests that MAML and ANIL are learning similar representations, despite their algorithmic differences. (ANIL and MAML are about similar each other two ANILS each other, two MAMLs are to each other.)
C.4 ANIL IMPLEMENTATION DETAILS
Supervised Learning Implementation: We used the TensorFlow MAML implementation open- sourced by the original authors (Finn et al. (017). We used the same model architectures as in the original MAML paper for our experiments, and train models 3 times with different random seeds. All models were trained for 30000 iterations, with a batch size of 4, inner loop update steps, and an inner learning rate of 0.01. inner gradient steps were used for evaluation test time.
Reinforcement Learning Implementation: We used the open source PyTorch implementation of MAML for RL due challenges encountered when running the open-sourced TensorFlow
17

---

Published as a conference paper at ICLR 2020
MinilmageNet-5way-BBe 1shot
Loss curves
Accuracy curves
8.0
train
MAMIL val lass
loss
0.55
ANIL val loss
Accuracy
0.50
0.40
MAML val acc train
0.35
5.5
val
5000
10000
15000 20000 25000 30000
5000
10000
15000 20000 25000 30000 Training iteration
Training iteration
MinilmageNeet-wayy-Sshhh
Loss curves
Accuracy curves
MAMIL train loss
7.0
val loss
6.5
ANIL train loss
0.70
6.0
ANIL
0.65
Accuracy
88
0.60
0.55
MAML train acc
MAML
0.45
ANIL train acc
3.5
acc
0.40
10000
15000 20000
10000
15000 20000 25000 30000
Training iteration
Training iteration
Omniglot-20way-1site
Loss curves
Accuracy curves
18
MAMIL train loss
0.95
16
ANIL train loss
ANIL val loss
0.90
Accuracy
LO55
0.85
8
0.80
MAML train acc
MAML val acc
6-
train
0.75
ANIL val acc
0 5000 10000 15000
20000 25000 20000
5000
10000 15000 20000 25000 30000 Training iteration
Training iteration
Figure 10: ANIL and MAML on MinilmageNeet and Omniglot. Loss and accuracy curves for ANIL and MAML on (i) MinilmageNet Swaygeyy---www------e Ishot (ii) MinilmageNet-5w:: 5way-5shot (iii) Onniglot-20way-Issh These illustrate how both algorithms learn very similarly over training.
implementation from the original authors. We note that the results for MAML in these RL domains do not exactly match those in the original paper; this may be due to large variance in results, depending on the random initialization.
used the same model architecture as the original paper (two layer MLP with 100 hidden units in each layer), batch size of 40, inner loop update step with inner learning rate of 0.1 and 20 trajectories for inner loop adaptation. We trained three MAML and ANIL models with different random initialization, and quote the mean and standard deviation of the results. As in the original MAML paper, for RL experiments, we select the best performing model over 500 iterations of training and evaluate this model at test time
set of tasks.
18

---

Published as a conference paper at ICLR 2020
(MAML, MAML) (ANIL, MAML)
(ANIL, MAML) (ANIL, ANIL)
CCA Similarity
CCA Similarity
CCA Simllarity (ANIL. MAML)
0.70
y=30
+0.013
CCA Similarity (ANIL. ANIL)
0.70
y=0,984x +0.012
0.65
Convi
Conv2
0.65
Convl
Conv2
0.60
Conv3
0.60
Conv3
Conv4
Conv4
0.55
0.55
0.50
0.45
0.45
0.40
0.35
0.35
0.35
0.40
0.45 0.50 0.55 0.60 0.65 0.70
0.35
0.40
0.45 0.50 0.55 0.60 0.65 0.70
CCA Similarity (MAML, MAML)
CCA Similarity (ANIL, MAML)
Computing CCA similarity across different seeds of MAML and ANIL networks suggests these representations are similar. We plot the CCA similarity between an ANIL seed and a MAML seed, plotted against (i) the MAMI. seed compared to different MAMI. seed (ii) the ANII. seed compared to a different ANIL seed. We observe strong correlation of similarity scores in both (i) and (ii). This tells us that (i) MAMI. representations vary about much as MAML. and ANII representations (ii) ANIL representations vary about as much as MAML and ANIL representations. In particular, this suggests that MAML and ANIL learn similar features, despite having significant algorithmic differences.
Figure 11:
Training: 5way-1shot
Training: 5way-5shor
Mean (s)
Median (s)
Speedup
Mean (s)
Median (s)
Speedup
MAML
0.15
0.13
0.68
0.67
1
First Order MAMI.
0.089
0.083
1.69
0.40
0.39
1.7
ANIL
0.084
0.072
1.79
0.37
0.36
1.84
Inference: 5way-Ishot
Inference: 5way-5shot
Mean (S)
Median (s) Speedup
Mean (s)
Median (S) Speedup
MAML
0.083
0.078
0.37
0.36
ANIL
0.020
0.017
4.15
0.076
0.071
4.87
Table 6: ANIL offers significant computational speedup over MAML, during both training and inference. Table comparing execution times and speedups of MAML, First Order MAML, and ANIL during training (above) and inference (helow) on MinilmageNet domains. Speedup is calculated relative MAML's exccution time. We see that ANIL offers noticeable speedup over MAML, as a result of removing the inner loop almost completely. This permits faster training and inference.
C.5 ANIL IS COMPUTATIONALLY SIMPLER THAN MAML
Table/6]shows results from a comparison of the computation time for MAML, First Order MAML., and ANIL, during training and inference, with the TensorFlow implementation described previously, on both MinilmageNet domains. These results are average time for executing forward and backward passes during training (above) and forward pass during inference (bottom), for task batch size of 1, and target set size of 1. Results are averaged over 2000 such batches. Speedup is calculated relative to MAML's execution time. Each batches' images were loaded into memory before running the TensorFlow computation graph, to ensure that data loading time was not captured in the timing. Experiments were run on a single NVIDIA Xp GPU.
During training, we see that ANIL is as fast as First Order MAML (which does not compute second order terms during training), and about 1.7x as fast as MAML. This leads to a significant overall training speedup, especially when coupled with the fact that the rate of learning for these ANIL and MAML is very similar, see learning curves in Appendix C.2 Note that unlike First Order MAML, ANIL also performs very comparably to MAML on benchmark tasks (on some tasks, First Order MAML performs worse (Finn et al., (017)). During inference, ANIL achieves over a 4x speedup over MAML. (and thus also 4x over First Order MAMI., which is identical to MAML at inference
19

---

Published as a conference paper at ICLR 2020
Method
MinilnageNer:-Say-llttoo
MinilmageNett-Sway-Shhm
MAML Wraining-MAM. head
46.9 0.2
63.1 0.4
MAMI. training-NII head
48.4 + 0.3
61.5 + 0.8
ANIL traaiiinnnnAAA -NIL head
46.7 + 0.4
61.5 0.5
ANIL training-NILL head
48.0 = 0.7
62.2 0.5
Multiclass pretrain- MAML head
38.4 + 0.8
54.6 0.4
Multiclass pretrain-NIL head
39.7 = 0.3
54,4 0.5
Multitask pretrain-Ma. MALL head
26.5 + 0.8
32.8 + 0.6
Multitask pretrain-NII. head
26.5 + 1.1
34.2 + 3.5
Random features-MAML. head
32.1 + 0.5
43.1 + 0.3
Random features-NII. head
32.9 + 0.6
43.2 + 0.5
Table 7: Test time performance is dominated by features learned, with difference between NIL/MAML heads. We see identical performances of MAML/NIL heads test time, indicating that MAMI/ANII. training leads better learned features.
time). Both training and inference speedups illustrate the significant computational benefit of ANIL over MAML.
D FURTHER RESULTS ON THE NETWORK HEAD AND BODY
D.I TRAINING REGIMES FOR NETWORK BODY
We add to the results of Section 5.2 in the main text by seeing if training head and applying that to representations test time (instead of the NIL algorithm) gives in any change the results. might be predicted by Section 5.1 we find no change the results.
More specifically, we the following:
We train MAMI/ANII. networks standard, and do standard test time adaptation.
For multiclass training. we first (pre)train with multiclass classification, then throw away the head and freeze the body. We initialize a new c.g. 5-class head, and train that (on top of the frozen multiclass pretrained features) with MAML. At test time perform standard adaptation.
The same process is applied to multitask training.
A similar process applied random features, except the network initialized and then frozen.
The results of this, along with the results from Table 5 in the main text is shown in Table 7 We observe very little performance difference between using MAML/ANIL head and NIL head for each training regime. Specifically, task performance is purely determined by the quality of the features and representations learned during training, with task-specific alignment (i) unnecessary (ii) unable to influence the final performance of the model (e.g. multitask training performance equally with a MAML head is with NIL-head.)
time being
D.2 REPRESENTATIONAL ANALYSIS DIFFERENT TRAINING REGIMES
In Table & we include results using CCA and CKA the representations learned by the different training methods. Specifically, we studied how similar representations of different training methods were to MAML training, finding direct correlation with performance training schemes learning representations most similar to MAML. also performed the best. We computed similarity scores by averaging the scores the first three conv layers in the body of the network.
20

---

Published as a conference paper at ICLR 2020
Feature pair
CCA Similarity
CKA Similarity
(MAMI,, MAMI))
0.51
0.83
(Multiclass pretrain, MAML)
0.48
0.79
(Random features, MAML)
0,40
0.72
(Multitask pretrain, MAML)
0.28
0.65
Table 8: MAML training most elosely resembles multiclass pretraining, illustrated by CCA and CKA similarities. On analyzing the CCA and CKA similarities between different baseline models and MAML (comparing across different tasks seeds), we see multiclass pretraining results features most similar to MAML training. Multitask pretraining differs quite significantly from MAML-learned features, potentially due to the alignment problem.
21


Reviews:
[{"review_id": "r1lsg0nptH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper is exploring the importance of the inner loop in MAML. It shows that using the inner loop only for the classifier head (ANIL) results are comparable to MAML. It also shows that using no inner loop at all (NIL) is okay for test time but not for training time.\n\nIt is indeed interesting to understand the effect of the inner loop. But, as the authors noted (“Our work is complementary to methods extending MAML, and our simplification and insights could be applied to such extensions also”), for it to be useful I’d like to see whether these insights can be extended to SOTA models. MAML is less than 50% accuracy on 1-shot mini-imagenet while current SOTS models achieve 60-65%.\n\nThe NIL experiment that shows low performance when no inner loop is used in training time doesn’t make sense. This is basically the same as the nearest-neighbour family of methods, e.g. ProtoNet (Snell et al., 2017), which have been shown to perform similarly to (or even better than) MAML.\n\n\nAfter rebuttal:\nI do think it's important to also have that kind of analysis works. My main concern is with how ANIL and NIL are introduced as new algorithms and not just an ablation of the MAML method. Presented as new algorithms I tend to compare them against the leaderboard where they are very far from the top. I am keeping my previous rating.  ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1lUaFTfFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper claims to examine the reasons for the success of MAML---an influential meta-learning algorithm to tackle few-shot learning. It thoroughly investigated the importance of the two optimization loops, and found that feature reuse is the dominant factor for MAML’s success. Moreover, the authors proposed new algorithms---ANIL and NIL---which spend much less computation on the inner loop of MAML. They also discussed their findings in a broader meta-learning context. \n\nI think the paper should be accepted for the following reasons: \n\n1. The experimental study is thorough. \n\nThe experiments follow a rigorous design of hypothesis-checking style and the conclusions are supported by extensive results under various evaluations. \n\nThe findings are potentially helpful for many future works in this field. \n\n2. The paper is clearly written. \n\nIt is generally enjoyable to read, except for some minor things to improve: (1) Evaluation metrics in table-2, table-4 and table-5 had better be explicitly clarified in the captions (2) No subsection seems needed in section-6. \n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "H1xctUU2oB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "After  rebuttal period: I recommend accepting  this  paper.\n======================================\nSummary:\n\nThis paper attempts to understand if the success of MAML is due to rapid learning or feature reuse. The analysis shows that MAML is performing better mainly due to feature reuse. Authors use this result to derive a simpler version of MAML called ANIL. ANIL does not update the non-final layers of the network during inner loop training and still has similar performance to MAML.\n\nMy comments:\n\nOverall I think this is an interesting analysis paper which sheds some light on how MAML works, However, I see these analysis not just as a criticism towards MAML. I also see these analysis as a criticism against the meta-learning datasets that we use. All these datasets are artifically created from the same dataset and hence it might be very easy to reuse features to get good performance. I am not sure if the same analysis will hold if we consider a dataset where tasks are not this similar (like Meta-dataset, Triantafillou et al 2019). I encourage the authors to have this disclaimer in the end of the paper so that the community does not falsely conclude that MAML cannot do rapid learning.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Paper received mixed reviews: WR (R1), A (R2 and R3). AC has read reviews/rebuttal and examined paper. AC agrees that R1's concerns are misplaced and feels the paper should be accepted. 

Decision: Accept (Poster)