Title: Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
Authors: Aniruddh Raghu
Maithra Raghu
Samy Bengio
Oriol Vinyals
Venue: ICLR 2020
Keywords: deep learning analysis
representation learning
meta-learning
few-shot learning

Abstract: An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, from which the inner loop can efficiently learn new tasks. Despite MAML's popularity, a fundamental open question remains -- is the effectiveness of MAML due to the meta-initialization being primed for rapid learning (large, efficient changes in the representations) or due to feature reuse,  with the meta initialization already containing high quality features? We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor. This leads to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML where we remove the inner loop for all but the (task-specific) head of the underlying neural network. ANIL matches MAML's performance on benchmark few-shot image classification and RL and offers computational improvements over MAML. We further study the precise contributions of the head and body of the network, showing that performance on the test tasks is entirely determined by the quality of the learned features, and we can remove even the head of the network (the NIL algorithm). We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly.

TL;DR: The success of MAML relies on feature reuse from the meta-initialization, which also yields a natural simplification of the algorithm, with the inner loop removed for the network body, as well as other insights on the head and body.

Review discussion:
The paper claims to examine the reasons for the success of MAML---an influential meta-learning algorithm to tackle few-shot learning. It thoroughly investigated the importance of the two optimization loops, and found that feature reuse is the dominant factor for MAML’s success.
Moreover, the authors proposed new algorithms---ANIL and NIL---which spend much less computation on the inner loop of MAML. They also discussed their findings in a broader meta-learning context.
I think the paper should be accepted for the following reasons:
1. The experimental study is thorough.
The experiments follow a rigorous design of hypothesis-checking style and the conclusions are supported by extensive results under various evaluations.
The findings are potentially helpful for many future works in this field.
2. The paper is clearly written.
It is generally enjoyable to read, except for some minor things to improve: (1) Evaluation metrics in table-2, table-4 and table-5 had better be explicitly clarified in the captions (2) No subsection seems needed in section-6.
Thank you for the thorough reading of our paper and the positive feedback. We will add these edits you suggested in the next revision!
Since submission we have also added additional interesting experiments to test how important the features that layers learn in the meta-initialization is to performance. We do this by resetting contiguous blocks of layers at meta-initialization to the very start of training, finding that resetting the lowest layer leads to the greatest performance drop. These experiments further support the feature reuse paradigm and also show that features in the earliest layers are most important for good performance:
Layers Reset           MiniImageNet-5way-1shot               MiniImageNet-5way-5shot
None                                  46.7                                                  61.5
This paper is exploring the importance of the inner loop in MAML. It shows that using the inner loop only for the classifier head (ANIL) results are comparable to MAML. It also shows that using no inner loop at all (NIL) is okay for test time but not for training time.
It is indeed interesting to understand the effect of the inner loop.
But, as the authors noted (“Our work is complementary to methods extending MAML, and our simplification and insights could be applied to such extensions also”), for it to be useful I’d like to see whether these insights can be extended to SOTA models. MAML is less than 50% accuracy on 1-shot mini-imagenet while current SOTS models achieve 60-65%.
The NIL experiment that shows low performance when no inner loop is used in training time doesn’t make sense. This is basically the same as the nearest-neighbour family of methods, e.g. ProtoNet (Snell et al., 2017), which have been shown to perform similarly to (or even better than) MAML.
After rebuttal:
I do think it's important to also have that kind of analysis works. My main concern is with how ANIL and NIL are introduced as new algorithms and not just an ablation of the MAML method. Presented as new algorithms I tend to compare them against the leaderboard where they are very far from the top. I am keeping my previous rating.
Thank you for your review. We respectfully disagree with your overall assessment of our paper, and offer here some justification.
Firstly, the focus of this paper is understanding why the highly popular and influential MAML algorithm works, and *not* just about removing the inner loop. The key message is that somewhat surprisingly, there is significant feature reuse, and not much large adaptation.  ANIL and NIL arise as a consequence of this, providing useful insights into further understanding the meta-learning field.
In line with the goal of gaining *fundamental insights* into these algorithms, the scope of this paper was not to look at SOTA methods, but perform a thorough and detailed study of the main MAML algorithm, and have **entirely reproducible** experiments. To do this, we stuck to using the following standard, easy to use and open source repositories:
https://github.com/cbfinn/maml for the few shot classification results (Code for the few-shot learning experiments from the original MAML paper, open-sourced by the original authors).
https://github.com/tristandeleu/pytorch-maml-rl for RL experiments (Code to reproduce the RL results from the original MAML paper).
We show how these insights on feature reuse connect to how other metalearning algorithms e.g. (Matching Networks, Vinyals et al) work, in Section 6. Note that feature reuse is a key factor behind the performance of other metalearning methods that achieve near SOTA , e.g. (Meta-learning with differentiable convex optimization, Kwonjoon et al), which does no ‘adaptation’ at test time, and still achieves excellent performance on the benchmarks. By demonstrating the importance of feature reuse in MAML (which is not at all immediately evident, due to the inner loop optimization), and identifying that other metalearning methods also employ feature reuse effectively, we link understanding about how MAML works to how other algorithms work. This contribution goes beyond just removing the inner loop in MAML.
Regarding NIL at training time and ProtoNets: firstly, we again emphasize that NIL at training time was **not at all** the main focus of the paper.
Additionally, there are **several very important differences** between NIL and Protonet:
— ProtoNets compute prototypes (averages over representations from the same class in the support set), which we do not do, taking the raw cosine similarity between test examples and all support set examples.
— ProtoNets use euclidean distance, instead of cosine distance, which is explicitly stated in the paper to improve performance.
— ProtoNets use a *learning rate scheduler* which also helps performance, while for using NIL during training we simply use the Adam optimizer with the default settings used in the original MAML paper, as we are seeking to compare NIL as closely as possible with the original MAML paper, not search over potential optimization curricula to improve its performance.
— ProtoNets have a different training process, with a *larger* number of classes during training compared to testing. E.g for testing 5-shot performance, they perform 20-way classification during the meta-training stage.
These key differences make it very hard to compare performance when training with NIL to  performance from ProtoNets.
Since submission we have also added additional interesting experiments to test how important the features learned in the meta-initialization are. We reset contiguous blocks of layers at meta-initialization to the very start of training. These experiments further support the feature reuse paradigm and also show that features in the earliest layers are most important for good performance:
Layers Reset           MiniImageNet-5way-1shot               MiniImageNet-5way-5shot
None                                  46.7                                                  61.5
The MAML algorithm has been extensively developed in recent literature, and has been applied to a wide range of problems. Our work offers fundamental insight into why this highly popular algorithm is effective, thereby answering a central open question. In investigating this question, we conducted detailed, reproducible experiments, that are of interest to the community, as seen by public comments. Our paper provides an excellent foundation for future work, which could leverage our insights to both understand few-shot learning algorithms better and develop improved few-shot learning algorithms.
After  rebuttal period: I recommend accepting  this  paper.
Summary:
This paper attempts to understand if the success of MAML is due to rapid learning or feature reuse. The analysis shows that MAML is performing better mainly due to feature reuse. Authors use this result to derive a simpler version of MAML called ANIL. ANIL does not update the non-final layers of the network during inner loop training and still has similar performance to MAML.
My comments:
Overall I think this is an interesting analysis paper which sheds some light on how MAML works, However, I see these analysis not just as a criticism towards MAML. I also see these analysis as a criticism against the meta-learning datasets that we use. All these datasets are artifically created from the same dataset and hence it might be very easy to reuse features to get good performance. I am not sure if the same analysis will hold if we consider a dataset where tasks are not this similar (like Meta-dataset, Triantafillou et al 2019). I encourage the authors to have this disclaimer in the end of the paper so that the community does not falsely conclude that MAML cannot do rapid learning.
Thank you very much for your review and comments on our paper. We will update the latest version of our paper with the disclaimer you have mentioned. We think that exploring how our analysis applies to other datasets from Meta-dataset (Triantafillou et al 2019), or across more diverse tasks, will be interesting future work.

Meta-review:
role: PC
data: comment: Paper received mixed reviews: WR (R1), A (R2 and R3). AC has read reviews/rebuttal and examined paper. AC agrees that R1's concerns are misplaced and feels the paper should be accepted. 


Decision: Accept (Poster)