Meta-Learning without Memorization

1 INTRODUCTION . The ability to learn new concepts and skills with small amounts of data is a critical aspect of intelligence that many machine learning systems lack . Meta-learning ( Schmidhuber , 1987 ) has emerged as a promising approach for enabling systems to quickly learn new tasks by building upon experience from previous related tasks ( Thrun & Pratt , 2012 ; Koch et al. , 2015 ; Santoro et al. , 2016 ; Ravi & Larochelle , 2016 ; Finn et al. , 2017 ) . Meta-learning accomplishes this by explicitly optimizing for few-shot generalization across a set of meta-training tasks . The meta-learner is trained such that , after being presented with a small task training set , it can accurately make predictions on test datapoints for that meta-training task . While these methods have shown promising results , current methods require careful design of the meta-training tasks to prevent a subtle form of task overfitting , distinct from standard overfitting in supervised learning . If the task can be accurately inferred from the test input alone , then the task training data can be ignored while still achieving low meta-training loss . In effect , the model will collapse to one that makes zero-shot decisions . This presents an opportunity for overfitting where the meta-learner generalizes on meta-training tasks , but fails to adapt when presented with training data from novel tasks . We call this form of overfitting the memorization problem in meta-learning because the meta-learner memorizes a function that solves all of the meta-training tasks , rather than learning to adapt . Existing meta-learning algorithms implicitly resolve this problem by carefully designing the metatraining tasks such that no single model can solve all tasks zero-shot ; we call tasks constructed in this Implementation and examples available here : https : //github.com/google-research/ google-research/tree/master/meta_learning_without_memorization . way mutually-exclusive . For example , forN -way classification , each task consists of examples from N randomly sampled classes . The N classes are labeled from 1 to N , and critically , for each task , we randomize the assignment of classes to labels { 1 , 2 , . . . , N } ( visualized in Appendix Figure 3 ) . This ensures that the task-specific class-to-label assignment can not be inferred from a test input alone . However , the mutually-exclusive tasks requirement places a substantial burden on the user to cleverly design the meta-training setup ( e.g. , by shuffling labels or omitting goal information ) . While shuffling labels provides a reasonable mechanism to force tasks to be mutually-exclusive with standard few-shot image classification datasets such as MiniImageNet ( Ravi & Larochelle , 2016 ) , this solution can not be applied to all domains where we would like to utilize meta-learning . For example , consider meta-learning a pose predictor that can adapt to different objects : even if N different objects are used for meta-training , a powerful model can simply learn to ignore the training set for each task , and directly learn to predict the pose of each of the N objects . However , such a model would not be able to adapt to new objects at meta-test time . The primary contributions of this work are : 1 ) to identify and formalize the memorization problem in meta-learning , and 2 ) to propose a meta-regularizer ( MR ) using information theory as a general approach for mitigating this problem without placing restrictions on the task distribution . We formally differentiate the meta-learning memorization problem from overfitting problem in conventional supervised learning , and empirically show that naı̈ve applications of standard regularization techniques do not solve the memorization problem in meta-learning . The key insight of our metaregularization approach is that the model acquired when memorizing tasks is more complex than the model that results from task-specific adaptation because the memorization model is a single model that simultaneously performs well on all tasks . It needs to contain all information in its weights needed to do well on test points without looking at training points . Therefore we would expect the information content of the weights of a memorization model to be larger , and hence the model should be more complex . As a result , we propose an objective that regularizes the information complexity of the meta-learned function class ( motivated by Alemi et al . ( 2016 ) ; Achille & Soatto ( 2018 ) ) . Furthermore , we show that meta-regularization in MAML can be rigorously motivated by a PAC-Bayes bound on generalization . In a series of experiments on non-mutually-exclusive task distributions entailing both few-shot regression and classification , we find that memorization poses a significant challenge for both gradient-based ( Finn et al. , 2017 ) and contextual ( Garnelo et al. , 2018a ) meta-learning methods , resulting in near random performance on test tasks in some cases . Our meta-regularization approach enables both of these methods to achieve efficient adaptation and generalization , leading to substantial performance gains across the board on non-mutually-exclusive tasks . 2 PRELIMINARIES . We focus on the standard supervised meta-learning problem ( see , e.g. , Finn et al . ( 2017 ) ) . Briefly , we assume tasks Ti are sampled from a task distribution p ( T ) . During meta-training , for each task , we observe a set of training data Di = ( xi , yi ) and a set of test data D∗i = ( x∗i , y∗i ) with xi = ( xi1 , . . . , xiK ) , yi = ( yi1 , . . . , yiK ) sampled from p ( x , y|Ti ) , and similarly forD∗i . We denote the entire meta-training set asM = { Di , D∗i } Ni=1 . The goal of meta-training is to learn a model for a new task T by leveraging what is learned during meta-training and a small amount of training data for the new task D. We use θ to denote the meta-parameters learned during meta-training and use φ to denote the task-specific parameters that are computed based on the task training data . Following Grant et al . ( 2018 ) ; Gordon et al . ( 2018 ) , given a meta-training set M , we consider meta-learning algorithms that maximize conditional likelihood q ( ŷ∗ = y∗|x∗ , θ , D ) , which is composed of three distributions : q ( θ|M ) that summarizes meta-training data into a distribution on metaparameters , q ( φ|D , θ ) that summarizes the per-task training set into a distribution on task-specific parameters , and q ( ŷ∗|x∗ , φ , θ ) that is the predictive distribution . These distributions are learned to minimize − 1N ∑ i Eq ( θ|M ) q ( φ|Di , θ ) [ 1 K ∑ ( x∗ , y∗ ) ∈D∗i log q ( ŷ∗ = y∗|x∗ , φ , θ ) ] . ( 1 ) For example , in MAML ( Finn et al. , 2017 ) , θ and φ are the weights of a predictor network , q ( θ|M ) is a delta function learned over the meta-training data , q ( φ|D , θ ) is a delta function centered at a point defined by gradient optimization , and φ parameterizes the predictor network q ( ŷ∗|x∗ , φ ) ( Grant et al. , 2018 ) . In particular , to determine the task-specific parameters φ , the task training data D and θ are used in the predictor model φ = θ + αK ∑ ( x , y ) ∈D∇θ log q ( y|x , φ = θ ) . Another family of meta-learning algorithms are contextual methods ( Santoro et al. , 2016 ) , such as conditional neural processes ( CNP ) ( Garnelo et al. , 2018b ; a ) . CNP instead defines q ( φ|D , θ ) as a mapping from D to a summary statistic φ ( parameterized by θ ) . In particular , φ = aθ ◦ hθ ( D ) is the output of an aggregator aθ ( · ) applied to features hθ ( D ) extracted from the task training data . Then θ parameterizes a predictor network that takes φ and x∗ as input and produces a predictive distribution q ( ŷ∗|x∗ , φ , θ ) . In the following sections , we describe a common pitfall for a variety of meta-learning algorithms , including MAML and CNP , and a general meta-regularization approach to prevent this pitfall . 3 THE MEMORIZATION PROBLEM IN META-LEARNING . The ideal meta-learning algorithm will learn in such a way that generalizes to novel tasks . However , we find that unless tasks are carefully designed , current meta-learning algorithms can overfit to the tasks and end up ignoring the task training data ( i.e. , either q ( φ|D , θ ) does not depend on D or q ( ŷ∗|x∗ , φ , θ ) does not depend on φ , as shown in Figure 1 ) , which can lead to poor generalization . This memorization phenomenon is best understood through examples . Consider a 3D object pose prediction problem ( illustrated in Figure 1 ) , where each object has a fixed canonical pose . The ( x , y ) pairs for the task are 2D grey-scale images of the rotated object ( x ) and the rotation angle relative to the fixed canonical pose for that object ( y ) . In the most extreme case , for an unseen object , the task is impossible without using D because the canonical pose for the unseen object is unknown . The number of objects in the meta-training dataset is small , so it is straightforward for a single network to memorize the canonical pose for each training object and to infer the object from the input image ( i.e. , task overfitting ) , thus achieving a low training error without using D. However , by construction , this solution will necessarily have poor generalization to test tasks with unseen objects . As another example , imagine an automated medical prescription system that suggests medication prescriptions to doctors based on patient symptoms and the patient ’ s previous record of prescription responses ( i.e. , medical history ) for adaptation . In the meta-learning framework , each patient represents a separate task . Here , the symptoms and prescriptions have a close relationship , so we can not assign random prescriptions to symptoms , in contrast to the classification tasks where we can randomly shuffle the labels to create mutually-exclusiveness . For this non-mutually-exclusive task distribution , a standard meta-learning system can memorize the patients ’ identity information in the training , leading it to ignore the medical history and only utilize the symptoms combined with the memorized information . As a result , it may issue highly accurate prescriptions on the meta-training set , but fail to adapt to new patients effectively . While such a system would achieve a baseline level of accuracy for new patients , it would be no better than a standard supervised learning method applied to the pooled data . We formally define ( complete ) memorization as : Definition 1 ( Complete Meta-Learning Memorization ) . Complete memorization in meta-learning is when the learned model ignores the task training data such that I ( ŷ∗ ; D|x∗ , θ ) = 0 ( i.e. , q ( ŷ∗|x∗ , θ , D ) = q ( ŷ∗|x∗ , θ ) = ED′|x∗ [ q ( ŷ∗|x∗ , θ , D′ ) ] ) . Memorization describes an issue with overfitting the meta-training tasks , but it does not preclude the network from generalizing to unseen ( x , y ) pairs on the tasks similar to the training tasks . Memorization becomes an undesired problem for generalization to new tasks when I ( y∗ ; D|x∗ ) I ( ŷ∗ ; D|x∗ , θ ) ( i.e. , the task training data is necessary to achieve good performance , even with exact inference under the data generating distribution , to make accurate predictions ) . A model with the memorization problem may generalize to new datapoints in training tasks but can not generalize to novel tasks , which distinguishes it from typical overfitting in supervised learning . In practice , we find that MAML and CNP frequently converge to this memorization solution ( Table 2 ) . For MAML , memorization can occur when a particular setting of θ that does not adapt to the task training data can achieve comparable meta-training error to a solution that adapts θ . For example , if a setting of θ can solve all of the meta-training tasks ( i.e. , for all ( x , y ) in D and D∗ the predictive error is close to zero ) , the optimization may converge to a stationary point of the MAML objective where minimal adaptation occurs based on the task training set ( i.e. , φ ≈ θ ) . For a novel task where it is necessary to use the task training data , MAML can in principle still leverage the task training data because the adaptation step is based on gradient descent . However , in practice , the poor initialization of θ can affect the model ’ s ability to generalize from a small mount of data . For CNP , memorization can occur when the predictive distribution network q ( ŷ∗|x∗ , φ , θ ) can achieve low training error without using the task training summary statistics φ . On a novel task , the network is not trained to use φ , so it is unable to use the information extracted from the task training set to effectively generalize . In some problem domains , the memorization problem can be avoided by carefully constructing the tasks . For example , for N -way classification , each task consists of examples from N randomly sampled classes . If the classes are assigned to a random permutation ofN for each task , this ensures that the task-specific class-to-label assignment can not be inferred from the test inputs alone . As a result , a model that ignores the task training data can not achieve low training error , preventing convergence to the memorization problem . We refer to tasks constructed in this way as mutuallyexclusive . However , the mutually-exclusive tasks requirement places a substantial burden on the user to cleverly design the meta-training setup ( e.g. , by shuffling labels or omitting goal information ) and can not be applied to all domains where we would like to utilize meta-learning .

Summary: This paper analyses a pitfall of current meta-learning algorithms, where the task can be inferred from the meta-training data alone, leaving the task-training data unused. Such a meta-learner would generalise well on the meta-training tasks, but will fail to generalise on new tasks at test time. This kind of overfitting is formalised as the memorization problem. This problem is implicitly resolved in current meta-learning algorithms by constructing mutually-exclusive meta-training tasks, which is not easy to construct in all scenarios. The paper introduces an information-theoretic meta-regularizer which forces information extraction from the task data (D) by restricting information flow from meta-parameters (\theta) and input (x^*). Experimental evaluation with one gradient based and one contextual meta-learning method, on non-mutually-exclusive tasks bring out the mettle of the proposed regulariser.