How to train your MAML

1 INTRODUCTION . The human capacity to learn new concepts using only a handful of samples is immense . In stark contrast , modern deep neural networks need , at a minimum , thousands of samples before they begin to learn representations that can generalize well to unseen data-points ( Krizhevsky et al. , 2012 ; Huang et al. , 2017 ) , and mostly fail when the data available is scarce . The fact that standard deep neural networks fail in the small data regime can provide hints about some of their potential shortcomings . Solving those shortcomings has the potential to open the door to understanding intelligence and advancing Artificial Intelligence . Few-shot learning encapsulates a family of methods that can learn new concepts with only a handful of data-points ( usually 1-5 samples per concept ) . This possibility is attractive for a number of reasons . First , few-shot learning would reduce the need for data collection and labelling , thus reducing the time and resources needed to build robust machine learning models . Second , it would potentially reduce training and fine-tuning times for adapting systems to newly acquired data . Third , in many real-world problems there are only a few samples available per class and the collection of additional data is either remarkably time-consuming and costly or altogether impossible , thus necessitating the need to learn from the available few samples . The nature of few-shot learning makes it a very hard problem if no prior knowledge exists about the task at hand . For a model to be able to learn a robust model from a few samples , knowledge transfer ( see e.g . Caruana , 1995 ) from other similar tasks is key . However , manual knowledge transfer from one task to another for the purpose of fine-tuning on a new task can be a time consuming and ultimately inefficient process . Meta-learning ( Schmidhuber , 1987 ; Vilalta & Drissi , 2002 ) , or learning to learn ( Thrun & Pratt , 1998 ) , can instead be used to automatically learn across-task knowledge usually referred to as across-task ( or sometimes slow ) knowledge such that our model can , at inference time , quickly acquire task-specific ( or fast ) knowledge from new tasks using only a few samples . Meta-learning can be broadly defined as a class of machine learning models that become more proficient at learning with more experience , thus learning how to learn . More specifically meta-learning involves learning at two levels . At the task-level , where the base-model is required to acquire task-specific ( fast ) knowledge rapidly , and at the meta-level , where the meta-model is required to slowly learn across-task ( slow ) knowledge . Recent work in meta-learning has produced state of the art results in a variety of settings ( Wang et al. , 2016 ; Ba et al. , 2016 ; Zoph & Le , 2016 ; Andrychowicz et al. , 2016 ; Vinyals et al. , 2016 ; Li & Malik , 2016 ; Li et al. , 2017 ; Antoniou et al. , 2017 ; Brock et al. , 2017 ; Munkhdalai & Yu , 2017 ) . The application of meta-learning in the fewshot learning setting has enabled the overwhelming majority of the current state of the art few-shot learning methods ( Vinyals et al. , 2016 ; Ravi & Larochelle , 2016 ; Edwards & Storkey , 2016 ; Finn et al. , 2017 ; Li et al. , 2017 ; Munkhdalai & Yu , 2017 ) . One such method , known for its simplicity and state of the art performance , is Model Agnostic Meta-Learning ( MAML ) ( Finn et al. , 2017 ) . In MAML , the authors propose learning an initialization for a base-model such that after applying a very small number of gradient steps with respect to a training set on the base-model , the adapted model can achieve strong generalization performance on a validation set ( the validation set consists of new samples from the same classes as the training set ) . Relating back to the definitions of metamodel and base-model , in MAML the meta-model is effectively the initialization parameters . These parameters are used to initialize the base-model , which is then used for task-specific learning on a support set , which is then evaluated on a target set . MAML is a simple yet elegant meta-learning framework that has achieved state of the art results in a number of settings . However , MAML suffers from a variety of problems which : 1 ) cause instability during training , 2 ) restrict the model ’ s generalization performance , 3 ) reduce the framework ’ s flexibility , 4 ) increase the system ’ s computational overhead and 5 ) require that the model goes through a costly ( in terms of time and computation needed ) hyperparameter tuning before it can work robustly on a new task . In this paper we propose MAML++ , an improved variant of the MAML framework that offers the flexibility of MAML along with many improvements , such as robust and stable training , automatic learning of the inner loop hyperparameters , greatly improved computational efficiency both during inference and training and significantly improved generalization performance . MAML++ is evaluated in the few-shot learning setting where the system is able to set a new state of the art across all established few-shot learning tasks on both Omniglot and Mini-Imagenet , performing as well as or better than all established meta learning methods on both tasks . 2 RELATED WORK . The set-to-set few-shot learning setting ( Vinyals et al. , 2016 ) , neatly casts few-shot learning as a meta-learning problem . In set-to-set few-shot learning we have a number of tasks , each task is composed by a support set which is used for task-level learning , and a target set which is used for evaluating the base-model on a certain task after it has acquired task-specific ( or fast ) knowledge . Furthermore , all available tasks are split into 3 sets , the meta-training set , the meta-validation set and the meta-test set , used for training , validating and testing our meta-learning model respectively . Once meta-learning was shown to be an effective framework for few-shot learning and the set to set approach was introduced , further developments in few-shot learning were made in quick succession . One contribution was Matching Networks ( Vinyals et al. , 2016 ) . Matching networks achieve few-shot learning by learning to match target set items to support set items . More specifically , a matching network learns to match the target set items to the support set items using cosine distance and a fully differentiable embedding function . First , the support set embedding function g , parameterized as a deep neural network , embeds the support set items into embedding vectors , then the target set embedding function f embeds the target set items . Once all data-item embeddings are available , cosine distance is computed for all target set embeddings when compared to all support set embeddings . As a result , for each target set item , a vector of cosine distances with respect to all support set items will be generated ( with each distance ’ s column tied to the respective support set class ) . Then , the softmax function is applied on the generated distance vectors , to convert them into probability distributions over the support set classes . Another notable advancement was the gradient-conditional meta-learner LSTM ( Ravi & Larochelle , 2016 ) that learns how to update a base-learner model . At inference time , the meta-learner model applies a single update on the base-learner given gradients with respect to the support set . The fully updated base-model then computes predictions on the target set . The target set predictions are then used to compute a task loss . Furthermore they jointly learn the meta-learner ’ s parameters as well as the base-learners initializations such that after a small number of steps it can do very well on a given task . The authors ran experiments on Mini-Imagenet where they exceed the performance of Matching Networks . In Model Agnostic meta-learning ( MAML ) ( Finn et al. , 2017 ) the authors proposed increasing the gradient update steps on the base-model and replacing the meta-learner LSTM with Batch Stochastic Gradient Descent ( Krizhevsky et al. , 2012 ) , which as a result speeds up the process of learning and interestingly improves generalization performance and achieves state of the art performance in both Omniglot and Mini-Imagenet . In Meta-SGD ( Li et al. , 2017 ) the authors proposed learning a static learning rate and an update direction for each parameter in the base-model , in addition to learning the initialization parameters of the base-model . Meta-SGD showcases significantly improved generalization performance ( when compared to MAML ) across all few-shot learning tasks , whilst only requiring a single inner loop update step . However this practice effectively doubles the model parameters and computational overheads of the system . 3 MODEL AGNOSTIC META LEARNING . Model Agnostic Meta-Learning ( MAML ) ( Finn et al. , 2017 ) is a meta-learning framework for fewshot learning . MAML is elegantly simple yet can produce state of the art results in few-shot regression/classification and reinforcement learning problems . In a sentence , MAML learns good initialization parameters for a network , such that after a few steps of standard training on a few-shot dataset , the network will perform well on that few shot task . More formally , we define the base model to be a neural network fθ with meta-parameters θ . We want to learn an initial θ = θ0 that , after a small number N of gradient update steps on data from a support set Sb to obtain θN , the network performs well on that task ’ s target set Tb . Here b is the index of a particular support set task in a batch of support set tasks . This set of N updates steps is called the inner-loop update process . The updated base-network parameters after i steps on data from the support task Sb can be expressed as : θbi = θ b i−1 − α∇θLSb ( fθbi−1 ) , ( 1 ) whereα is the learning rate , θbi are the base-network weights after i steps towards task b , LSb ( fθ ( i−1 ) ) is the loss on the support set of task b after ( i − 1 ) ( i.e . the previous step ) update steps . Assuming that our task batch size is B we can define a meta-objective , which can be expressed as : Lmeta ( θ0 ) = B∑ b=1 LTb ( fθbN ( θ0 ) ) ( 2 ) where we have explicitly denoted the dependence of θbN on θ0 , given by unrolling ( 1 ) . The objective ( 2 ) measures the quality of an initialization θ0 in terms of the total loss of using that initialization across all tasks . This meta objective is now minimized to optimize the initial parameter value θ0 . It is this initial θ0 that contains the across-task knowledge . The optimization of this meta-objective is called the outer-loop update process . The resulting update for the meta-parameters θ0 can be expressed as : θ0 = θ0 − β∇θ B∑ b=1 LTb ( fθbN ( θ0 ) ) ( 3 ) where β is a learning rate and LTb denotes the loss on the target set for task b . In this paper we use the cross-entropy ( De Boer et al. , 2005 ; Rubinstein , 1999 ) loss throughout .

Summary: In the work, the authors improve a simple yet effective meta-learning algorithm called Model Agnostic meta-learning (MAML) from various aspects including training instability, batch normalization etc. The authors firstly point out the issues in MAML training and tackle each of the issue with a practical alternative approach respectfully. The few-shot classification results show convincing evidence.