How to train your MAML

1 INTRODUCTION . The human capacity to learn new concepts using only a handful of samples is immense . In stark contrast , modern deep neural networks need , at a minimum , thousands of samples before they begin to learn representations that can generalize well to unseen data-points ( Krizhevsky et al. , 2012 ; Huang et al. , 2017 ) , and mostly fail when the data available is scarce . The fact that standard deep neural networks fail in the small data regime can provide hints about some of their potential shortcomings . Solving those shortcomings has the potential to open the door to understanding intelligence and advancing Artificial Intelligence . Few-shot learning encapsulates a family of methods that can learn new concepts with only a handful of data-points ( usually 1-5 samples per concept ) . This possibility is attractive for a number of reasons . First , few-shot learning would reduce the need for data collection and labelling , thus reducing the time and resources needed to build robust machine learning models . Second , it would potentially reduce training and fine-tuning times for adapting systems to newly acquired data . Third , in many real-world problems there are only a few samples available per class and the collection of additional data is either remarkably time-consuming and costly or altogether impossible , thus necessitating the need to learn from the available few samples . The nature of few-shot learning makes it a very hard problem if no prior knowledge exists about the task at hand . For a model to be able to learn a robust model from a few samples , knowledge transfer ( see e.g . Caruana , 1995 ) from other similar tasks is key . However , manual knowledge transfer from one task to another for the purpose of fine-tuning on a new task can be a time consuming and ultimately inefficient process . Meta-learning ( Schmidhuber , 1987 ; Vilalta & Drissi , 2002 ) , or learning to learn ( Thrun & Pratt , 1998 ) , can instead be used to automatically learn across-task knowledge usually referred to as across-task ( or sometimes slow ) knowledge such that our model can , at inference time , quickly acquire task-specific ( or fast ) knowledge from new tasks using only a few samples . Meta-learning can be broadly defined as a class of machine learning models that become more proficient at learning with more experience , thus learning how to learn . More specifically meta-learning involves learning at two levels . At the task-level , where the base-model is required to acquire task-specific ( fast ) knowledge rapidly , and at the meta-level , where the meta-model is required to slowly learn across-task ( slow ) knowledge . Recent work in meta-learning has produced state of the art results in a variety of settings ( Wang et al. , 2016 ; Ba et al. , 2016 ; Zoph & Le , 2016 ; Andrychowicz et al. , 2016 ; Vinyals et al. , 2016 ; Li & Malik , 2016 ; Li et al. , 2017 ; Antoniou et al. , 2017 ; Brock et al. , 2017 ; Munkhdalai & Yu , 2017 ) . The application of meta-learning in the fewshot learning setting has enabled the overwhelming majority of the current state of the art few-shot learning methods ( Vinyals et al. , 2016 ; Ravi & Larochelle , 2016 ; Edwards & Storkey , 2016 ; Finn et al. , 2017 ; Li et al. , 2017 ; Munkhdalai & Yu , 2017 ) . One such method , known for its simplicity and state of the art performance , is Model Agnostic Meta-Learning ( MAML ) ( Finn et al. , 2017 ) . In MAML , the authors propose learning an initialization for a base-model such that after applying a very small number of gradient steps with respect to a training set on the base-model , the adapted model can achieve strong generalization performance on a validation set ( the validation set consists of new samples from the same classes as the training set ) . Relating back to the definitions of metamodel and base-model , in MAML the meta-model is effectively the initialization parameters . These parameters are used to initialize the base-model , which is then used for task-specific learning on a support set , which is then evaluated on a target set . MAML is a simple yet elegant meta-learning framework that has achieved state of the art results in a number of settings . However , MAML suffers from a variety of problems which : 1 ) cause instability during training , 2 ) restrict the model ’ s generalization performance , 3 ) reduce the framework ’ s flexibility , 4 ) increase the system ’ s computational overhead and 5 ) require that the model goes through a costly ( in terms of time and computation needed ) hyperparameter tuning before it can work robustly on a new task . In this paper we propose MAML++ , an improved variant of the MAML framework that offers the flexibility of MAML along with many improvements , such as robust and stable training , automatic learning of the inner loop hyperparameters , greatly improved computational efficiency both during inference and training and significantly improved generalization performance . MAML++ is evaluated in the few-shot learning setting where the system is able to set a new state of the art across all established few-shot learning tasks on both Omniglot and Mini-Imagenet , performing as well as or better than all established meta learning methods on both tasks . 2 RELATED WORK . The set-to-set few-shot learning setting ( Vinyals et al. , 2016 ) , neatly casts few-shot learning as a meta-learning problem . In set-to-set few-shot learning we have a number of tasks , each task is composed by a support set which is used for task-level learning , and a target set which is used for evaluating the base-model on a certain task after it has acquired task-specific ( or fast ) knowledge . Furthermore , all available tasks are split into 3 sets , the meta-training set , the meta-validation set and the meta-test set , used for training , validating and testing our meta-learning model respectively . Once meta-learning was shown to be an effective framework for few-shot learning and the set to set approach was introduced , further developments in few-shot learning were made in quick succession . One contribution was Matching Networks ( Vinyals et al. , 2016 ) . Matching networks achieve few-shot learning by learning to match target set items to support set items . More specifically , a matching network learns to match the target set items to the support set items using cosine distance and a fully differentiable embedding function . First , the support set embedding function g , parameterized as a deep neural network , embeds the support set items into embedding vectors , then the target set embedding function f embeds the target set items . Once all data-item embeddings are available , cosine distance is computed for all target set embeddings when compared to all support set embeddings . As a result , for each target set item , a vector of cosine distances with respect to all support set items will be generated ( with each distance ’ s column tied to the respective support set class ) . Then , the softmax function is applied on the generated distance vectors , to convert them into probability distributions over the support set classes . Another notable advancement was the gradient-conditional meta-learner LSTM ( Ravi & Larochelle , 2016 ) that learns how to update a base-learner model . At inference time , the meta-learner model applies a single update on the base-learner given gradients with respect to the support set . The fully updated base-model then computes predictions on the target set . The target set predictions are then used to compute a task loss . Furthermore they jointly learn the meta-learner ’ s parameters as well as the base-learners initializations such that after a small number of steps it can do very well on a given task . The authors ran experiments on Mini-Imagenet where they exceed the performance of Matching Networks . In Model Agnostic meta-learning ( MAML ) ( Finn et al. , 2017 ) the authors proposed increasing the gradient update steps on the base-model and replacing the meta-learner LSTM with Batch Stochastic Gradient Descent ( Krizhevsky et al. , 2012 ) , which as a result speeds up the process of learning and interestingly improves generalization performance and achieves state of the art performance in both Omniglot and Mini-Imagenet . In Meta-SGD ( Li et al. , 2017 ) the authors proposed learning a static learning rate and an update direction for each parameter in the base-model , in addition to learning the initialization parameters of the base-model . Meta-SGD showcases significantly improved generalization performance ( when compared to MAML ) across all few-shot learning tasks , whilst only requiring a single inner loop update step . However this practice effectively doubles the model parameters and computational overheads of the system . 3 MODEL AGNOSTIC META LEARNING . Model Agnostic Meta-Learning ( MAML ) ( Finn et al. , 2017 ) is a meta-learning framework for fewshot learning . MAML is elegantly simple yet can produce state of the art results in few-shot regression/classification and reinforcement learning problems . In a sentence , MAML learns good initialization parameters for a network , such that after a few steps of standard training on a few-shot dataset , the network will perform well on that few shot task . More formally , we define the base model to be a neural network fθ with meta-parameters θ . We want to learn an initial θ = θ0 that , after a small number N of gradient update steps on data from a support set Sb to obtain θN , the network performs well on that task ’ s target set Tb . Here b is the index of a particular support set task in a batch of support set tasks . This set of N updates steps is called the inner-loop update process . The updated base-network parameters after i steps on data from the support task Sb can be expressed as : θbi = θ b i−1 − α∇θLSb ( fθbi−1 ) , ( 1 ) whereα is the learning rate , θbi are the base-network weights after i steps towards task b , LSb ( fθ ( i−1 ) ) is the loss on the support set of task b after ( i − 1 ) ( i.e . the previous step ) update steps . Assuming that our task batch size is B we can define a meta-objective , which can be expressed as : Lmeta ( θ0 ) = B∑ b=1 LTb ( fθbN ( θ0 ) ) ( 2 ) where we have explicitly denoted the dependence of θbN on θ0 , given by unrolling ( 1 ) . The objective ( 2 ) measures the quality of an initialization θ0 in terms of the total loss of using that initialization across all tasks . This meta objective is now minimized to optimize the initial parameter value θ0 . It is this initial θ0 that contains the across-task knowledge . The optimization of this meta-objective is called the outer-loop update process . The resulting update for the meta-parameters θ0 can be expressed as : θ0 = θ0 − β∇θ B∑ b=1 LTb ( fθbN ( θ0 ) ) ( 3 ) where β is a learning rate and LTb denotes the loss on the target set for task b . In this paper we use the cross-entropy ( De Boer et al. , 2005 ; Rubinstein , 1999 ) loss throughout .

Summary: This work presents several enhancements to the established Model-Agnostic Meta-Learning (MAML) framework. Specifically, the paper starts by analyzing the issues in the original implementations of MAML, including instability during training, costly second order derivatives evaluation, missing/shared batch normalization statistics accumulation/bias, and learning rate setting, which causes unstable or slow convergence, and weak generalization. The paper then proposes solutions corresponding to each of these issues, and reports improved performance on benchmark datasets.