Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning . An especially successful algorithm has been Model Agnostic Meta-Learning ( MAML ) , a method that consists of two optimization loops , with the outer loop finding a meta-initialization , from which the inner loop can efficiently learn new tasks . Despite MAML ’ s popularity , a fundamental open question remains – is the effectiveness of MAML due to the meta-initialization being primed for rapid learning ( large , efficient changes in the representations ) or due to feature reuse , with the meta-initialization already containing high quality features ? We investigate this question , via ablation studies and analysis of the latent representations , finding that feature reuse is the dominant factor . This leads to the ANIL ( Almost No Inner Loop ) algorithm , a simplification of MAML where we remove the inner loop for all but the ( task-specific ) head of the underlying neural network . ANIL matches MAML ’ s performance on benchmark few-shot image classification and RL and offers computational improvements over MAML . We further study the precise contributions of the head and body of the network , showing that performance on the test tasks is entirely determined by the quality of the learned features , and we can remove even the head of the network ( the NIL algorithm ) . We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly . 1 INTRODUCTION . A central problem in machine learning is few-shot learning , where new tasks must be learned with a very limited number of labelled datapoints . A significant body of work has looked at tackling this challenge using meta-learning approaches ( Koch et al. , 2015 ; Vinyals et al. , 2016 ; Snell et al. , 2017 ; Finn et al. , 2017 ; Santoro et al. , 2016 ; Ravi and Larochelle , 2016 ; Nichol and Schulman , 2018 ) . Broadly speaking , these approaches define a family of tasks , some of which are used for training and others solely for evaluation . A proposed meta-learning algorithm then looks at learning properties that generalize across the different training tasks , and result in fast and efficient learning of the evaluation tasks . One highly successful meta-learning algorithm has been Model Agnostic Meta-Learning ( MAML ) ( Finn et al. , 2017 ) . At a high level , the MAML algorithm is comprised of two optimization loops . The outer loop ( in the spirit of meta-learning ) aims to find an effective meta-initialization , from which the inner loop can perform efficient adaptation – optimize parameters to solve new tasks with very few labelled examples . This algorithm , with deep neural networks as the underlying model , has been highly influential , with significant follow on work , such as first order variants ( Nichol and Schulman , 2018 ) , probabilistic extensions ( Finn et al. , 2018 ) , augmentation with generative modelling ( Rusu et al. , 2018 ) , and many others ( Hsu et al. , 2018 ; Finn and Levine , 2017 ; Grant et al. , 2018 ; Triantafillou et al. , 2019 ) . ∗Equal contribution Despite the popularity of MAML , and the numerous followups and extensions , there remains a fundamental open question on the basic algorithm . Does the meta-initialization learned by the outer loop result in rapid learning on unseen test tasks ( efficient but significant changes in the representations ) or is the success primarily due to feature reuse ( with the meta-initialization already providing high quality representations ) ? In this paper , we explore this question and its many surprising consequences . Our main contributions are : • We perform layer freezing experiments and latent representational analysis of MAML , finding that feature reuse is the predominant reason for efficient learning . • Based on these results , we propose the ANIL ( Almost No Inner Loop ) algorithm , a significant simplification to MAML that removes the inner loop updates for all but the head ( final layer ) of a neural network during training and inference . ANIL performs identically to MAML on standard benchmark few-shot classification and RL tasks and offers computational benefits over MAML . • We study the effect of the head of the network , finding that once training is complete , the head can be removed , and the representations can be used without adaptation to perform unseen tasks , which we call the No Inner Loop ( NIL ) algorithm . • We study different training regimes , e.g . multiclass classification , multitask learning , etc , and find that the task specificity of MAML/ANIL at training facilitate the learning of better features . We also find that multitask training , a popular baseline with no task specificity , performs worse than random features . • We discuss rapid learning and feature reuse in the context of other meta-learning approaches . 2 RELATED WORK . MAML ( Finn et al. , 2017 ) is a highly popular meta-learning algorithm for few-shot learning , achieving competitive performance on several benchmark few-shot learning problems ( Koch et al. , 2015 ; Vinyals et al. , 2016 ; Snell et al. , 2017 ; Santoro et al. , 2016 ; Ravi and Larochelle , 2016 ; Nichol and Schulman , 2018 ) . It is part of the family of optimization-based meta-learning algorithms , with other members of this family presenting variations around how to learn the weights of the task-specific classifier . For example Lee and Choi ( 2018 ) ; Gordon et al . ( 2018 ) ; Bertinetto et al . ( 2018 ) ; Lee et al . ( 2019 ) ; Zhou et al . ( 2018 ) first learn functions to embed the support set and target examples of a few-shot learning task , before using the test support set to learn task specific weights to use on the embedded target examples . Harrison et al . ( 2018 ) also proceeds similarly , using a Bayesian approach . The method of Bao et al . ( 2019 ) explores a related approach , focusing on applications in text classification . Of these optimization-based meta-learning algorithms , MAML has been especially influential , inspiring numerous direct extensions in recent literature ( Antoniou et al. , 2018 ; Finn et al. , 2018 ; Grant et al. , 2018 ; Rusu et al. , 2018 ) . Most of these extensions critically rely on the core structure of the MAML algorithm , incorporating an outer loop ( for meta-training ) , and an inner loop ( for task-specific adaptation ) , and there is little prior work analyzing why this central part of the MAML algorithm is practically successful . In this work , we focus on this foundational question , examining how and why MAML leads to effective few-shot learning . To do this , we utilize analytical tools such as Canonical Correlation Analysis ( CCA ) ( Raghu et al. , 2017 ; Morcos et al. , 2018 ) and Centered Kernel Alignment ( CKA ) ( Kornblith et al. , 2019 ) to study the neural network representations learned with the MAML algorithm , which also demonstrates MAML ’ s ability to learn effective features for few-shot learning . Insights from this analysis lead to a simplified algorithm , ANIL , which almost completely removes the inner optimization loop with no reduction in performance . Prior works ( Zintgraf et al. , 2018 ; Javed and White , 2019 ) have proposed algorithms where some parameters are only updated in the outer loop and others only in the inner loop . However , these works are motivated by different questions , such as improving MAML ’ s performance or learning better representations , rather than analysing rapid learning vs feature reuse in MAML . 3 MAML , RAPID LEARNING , AND FEATURE REUSE . Our goal is to understand whether the MAML algorithm efficiently solves new tasks due to rapid learning or feature reuse . In rapid learning , large representational and parameter changes occur during adaptation to each new task as a result of favorable weight conditioning from the meta-initialization . In feature reuse , the meta-initialization already contains highly useful features that can mostly be reused as is for new tasks , so little task-specific adaptation occurs . Figure 1 shows a schematic of these two hypotheses . We start off by overviewing the details of the MAML algorithm , and then we study the rapid learning vs feature reuse question via layer freezing experiments and analyzing latent representations of models trained with MAML . The results strongly support feature reuse as the predominant factor behind MAML ’ s success . In Section 4 , we explore the consequences of this , providing a significant simplification of MAML , the ANIL algorithm , and in Section 6 , we outline the connections to meta-learning more broadly . 3.1 OVERVIEW OF MAML . The MAML algorithm finds an initialization for a neural network so that new tasks can be learnt with very few examples ( k examples from each class for k-shot learning ) via two optimization loops : • Outer Loop : Updates the initialization of the neural network parameters ( often called the meta-initialization ) to a setting that enables fast adaptation to new tasks . • Inner Loop : Performs adaptation : takes the outer loop initialization , and , separately for each task , performs a few gradient updates over the k labelled examples ( the support set ) provided for adaptation . More formally , we first define our base model to be a neural network with meta-initialization parameters θ ; let this be represented by fθ . We have have a distribution D over tasks , and draw a batch { T1 , ... , TB } of B tasks from D. For each task Tb , we have a support set of examples STb , which are used for inner loop updates , and a target set of examples ZTb , which are used for outer loop updates . Let θ ( b ) i signify θ after i gradient updates for task Tb , and let θ ( b ) 0 = θ . In the inner loop , during each update , we compute θ ( b ) m = θ ( b ) m−1 − α∇θ ( b ) m−1LSTb ( fθ ( b ) m−1 ( θ ) ) ( 1 ) for m fixed across all tasks , where LSTb ( fθ ( b ) m−1 ( θ ) ) is the loss on the support set of Tb after m− 1 inner loop updates . We then define the meta-loss as Lmeta ( θ ) = B∑ b=1 LZTb ( fθ ( b ) m ( θ ) ) where LZTb ( fθ ( b ) m ( θ ) ) is the loss on the target set of Tb after m inner loop updates , making clear the dependence of f θ ( b ) m on θ . The outer optimization loop then updates θ as θ = θ − η∇θLmeta ( θ ) At test time , we draw unseen tasks { T ( test ) 1 , ... , T ( test ) n } from the task distribution , and evaluate the loss and accuracy on Z T ( test ) i after inner loop adaptation using S T ( test ) i ( e.g . loss is LZ T ( test ) i ( f θ ( i ) m ( θ ) ) ) .

Summary: The paper claims to examine the reasons for the success of MAML---an influential meta-learning algorithm to tackle few-shot learning. It thoroughly investigated the importance of the two optimization loops, and found that feature reuse is the dominant factor for MAML’s success. Moreover, the authors proposed new algorithms---ANIL and NIL---which spend much less computation on the inner loop of MAML. They also discussed their findings in a broader meta-learning context.