Meta-Learning with Latent Embedding Optimization

1 INTRODUCTION . Humans have a remarkable ability to quickly grasp new concepts from a very small number of examples or a limited amount of experience , leveraging prior knowledge and context . In contrast , traditional deep learning approaches ( LeCun et al. , 2015 ; Schmidhuber , 2015 ) treat each task independently and hence are often data inefficient – despite providing significant performance improvements across the board , such as for image classification ( Simonyan & Zisserman , 2014 ; He et al. , 2016 ) , reinforcement learning ( Mnih et al. , 2015 ; Silver et al. , 2017 ) , and machine translation ( Cho et al. , 2014 ; Sutskever et al. , 2014 ) . Just as humans can efficiently learn new tasks , it is desirable for learning algorithms to quickly adapt to and incorporate new and unseen information . Few-shot learning tasks challenge models to learn a new concept or behaviour with very few examples or limited experience ( Fei-Fei et al. , 2006 ; Lake et al. , 2011 ) . One approach to address this class of problems is meta-learning , a broad family of techniques focused on learning how to learn or to quickly adapt to new information . More specifically , optimization-based meta-learning approaches ( Ravi & Larochelle , 2017 ; Finn et al. , 2017 ) aim to find a single set of model parameters that can be adapted with a few steps of gradient descent to individual tasks . However , using only a few samples ( typically 1 or 5 ) to compute gradients in a high-dimensional parameter space could make generalization difficult , especially under the constraint of a shared starting point for task-specific adaptation . In this work we propose a new approach , named Latent Embedding Optimization ( LEO ) , which learns a low-dimensional latent embedding of model parameters and performs optimization-based meta-learning in this space . Intuitively , the approach provides two advantages . First , the initial parameters for a new task are conditioned on the training data , which enables a task-specific starting point for adaptation . By incorporating a relation network into the encoder , this initialization can better consider the joint relationship between all of the input data . Second , by optimizing in the lower-dimensional latent space , the approach can adapt the behaviour of the model more effectively . Further , by allowing this process to be stochastic , the ambiguities present in the few-shot data regime can be expressed . We demonstrate that LEO achieves state-of-the-art results on both the miniImageNet and tieredImageNet datasets , and run an ablation study and further analysis to show that both conditional parameter generation and optimization in latent space are critical for the success of the method . Source code for our experiments is available at https : //github.com/deepmind/leo . 2 MODEL . 2.1 PROBLEM DEFINITION . We define the N -way K-shot problem using the episodic formulation of Vinyals et al . ( 2016 ) . Each task instance Ti is a classification problem sampled from a task distribution p ( T ) . The tasks are divided into a training meta-set Str , validation meta-set Sval , and test meta-set Stest , each with a disjoint set of target classes ( i.e. , a class seen during testing is not seen during training ) . The validation meta-set is used for model selection , and the testing meta-set is used only for final evaluation . Each task instance Ti ∼ p ( T ) is composed of a training set Dtr and validation set Dval , and only contains N classes randomly selected from the appropriate meta-set ( e.g . for a task instance in the training meta-set , the classes are a subset of those available in Str ) . In most setups , the training set Dtr = { ( xkn , y k n ) | k = 1 . . .K ; n = 1 . . . N } contains K samples for each class . The validation setDval can contain several other samples from the same classes , providing an estimate of generalization performance on the N classes for this problem instance . We note that the validation set of a problem instance Dval ( used to optimize a meta-learning objective ) should not be confused with the held-out validation meta-set Sval ( used for model selection ) . 2.2 MODEL-AGNOSTIC META-LEARNING . Model-agnostic meta-learning ( MAML ) ( Finn et al. , 2017 ) is an approach to optimization-based meta-learning that is related to our work . For some parametric model fθ , MAML aims to find a single set of parameters θ which , using a few optimization steps , can be successfully adapted to any novel task sampled from the same distribution . For a particular task instance Ti = ( Dtr , Dval ) , the parameters are adapted to task-specific model parameters θ′i by applying some differentiable function , typically an update rule of the form : θ′i = G ( θ , Dtr ) , ( 1 ) where G is typically implemented as a step of gradient descent on the few-shot training set Dtr , θ′i = θ−α∇θLtrTi ( fθ ) . Generally , multiple sequential adaptation steps can be applied . The learning rate α can also be meta-learned concurrently , in which case we refer to this algorithm as MetaSGD ( Li et al. , 2017 ) . During meta-training , the parameters θ are updated by back-propagating through the adaptation procedure , in order to reduce errors on the validation set Dval : θ ← θ − η∇θ ∑ Ti∼p ( T ) LvalTi ( fθ′i ) ( 2 ) The approach includes the main ingredients of optimization-based meta-learning with neural networks : initialization is done by maintaining an explicit set of model parameters θ ; the adaptation procedure , or “ inner loop ” , takes θ as input and returns θ′i adapted specifically for task instance Ti , by iteratively using gradient descent ( Eq . 1 ) ; and termination , which is handled simply by choosing a fixed number of optimization steps in the “ inner loop ” . MAML updates θ by differentiating through the “ inner loop ” in order to minimize errors of instance-specific adapted models fθ′i on the corresponding validation set ( Eq . 2 ) . We refer to this process as the “ outer loop ” of meta-learning . In the next section we use the same stages to describe Latent Embedding Optimization ( LEO ) . 2.3 LATENT EMBEDDING OPTIMIZATION FOR META-LEARNING . The primary contribution of this paper is to show that it is possible , and indeed beneficial , to decouple optimization-based meta-learning techniques from the high-dimensional space of model parameters . We achieve this by learning a stochastic latent space with an information bottleneck , conditioned on the input data , from which the high-dimensional parameters are generated . Algorithm 1 Latent Embedding Optimization Require : Training meta-set Str ∈ T Require : Learning rates α , η 1 : Randomly initialize φe , φr , φd 2 : Let φ = { φe , φr , φd , α } 3 : while not converged do 4 : for number of tasks in batch do 5 : Sample task instance Ti ∼ Str 6 : Let ( Dtr , Dval ) = Ti 7 : Encode Dtr to z using gφe and gφr 8 : Decode z to initial params θi using gφd 9 : Initialize z′ = z , θ′i = θi 10 : for number of adaptation steps do 11 : Compute training loss LtrTi ( fθ′i ) 12 : Perform gradient step w.r.t . z′ : z′ ← z′ − α∇z′LtrTi ( fθ′i ) 13 : Decode z′ to obtain θ′i using gφd 14 : end for 15 : Compute validation loss LvalTi ( fθ′i ) 16 : end for 17 : Perform gradient step w.r.t φ : φ← φ− η∇φ ∑ Ti L val Ti ( fθ′i ) 18 : end while Figure 1 : High-level intuition for LEO . While MAML operates directly in a high dimensional parameter space Θ , LEO performs meta-learning within a low-dimensional latent space Z , from which the parameters are generated . E ncoder R elation N et D ecoder optimized in inner loop optimized in outer loop Inner loop optimization Inference Figure 2 : Overview of the architecture of LEO . Instead of explicitly instantiating and maintaining a unique set of model parameters θ , as in MAML , we learn a generative distribution of model parameters which serves the same purpose . This is a natural extension : we relax the requirement of finding a single optimal θ∗ ∈ Θ to that of approximating a data-dependent conditional probability distribution over Θ , which can be more expressive . The choice of architecture , composed of an encoding process , and decoding ( or parameter generation ) process , enables us to perform the MAML gradient-based adaptation steps ( or “ inner loop ” ) in the learned , low-dimensional embedding space of the parameter generative model ( Figure 1 ) . 2.3.1 MODEL OVERVIEW . The high-level operation is then as follows ( Algorithm 1 ) . First , given a task instance Ti , the inputs { xkn } are passed through a stochastic encoder to produce a latent code z , which is then decoded to parameters θi using a parameter generator1 . Given these instantiated model parameters , one or more adaptation steps are applied in the latent space , by differentiating the loss with respect to z , taking a gradient step to get z′ , decoding new model parameters , and obtaining the new loss . Finally , optimized codes are decoded to produce the final adapted parameters θ′i , which can be used to perform the task , or compute the task-specific meta-loss . In this way , LEO incorporates aspects of model-based and optimization-based meta-learning , producing parameters that are first conditioned on the input data and then adapted by gradient descent . 1Note that we omit the task subscript i from latent code z and input data xkn for clarity . Figure 2 shows the architecture of the resulting network . Intuitively , the decoder is akin to a generative model , mapping from a low-dimensional latent code to a distribution over model parameters . The encoding process ensures that the initial latent code and parameters before gradient-based adaptation are already data-dependent . This encoding process also exploits a relation network that allows the latent code to be context-dependent , considering the pairwise relationship between all classes in the problem instance . In the following sections , we explain the LEO procedure more formally . 2.3.2 INITIALIZATION : GENERATING PARAMETERS CONDITIONED ON A FEW EXAMPLES . The first stage is to instantiate the model parameters that will be adapted to each task instance . Whereas MAML explicitly maintains a single set of model parameters , LEO utilises a datadependent latent encoding which is then decoded to generate the actual initial parameters . In what follows , we describe an encoding scheme which leverages a relation network to map the few-shot examples into a single latent vector . This design choice allows the approach to consider context when producing a parameter initialization . Intuitively , decision boundaries required for fine-grained distinctions between similar classes might need to be different from those for broader classification . Encoding The encoding process involves a simple feed-forward mapping of each data point , followed by a relation network that considers the pair-wise relationship between the data in the problem instance . The overall encoding process is defined in Eq . 3 , and proceeds as follows . First , each example from a problem instance Ti = ( Dtr , Dval ) ∼ p ( T ) is processed by an encoder network gφe : Rnx → Rnh , which maps from input space to a code in an intermediate hidden-layer code spaceH . Then , codes inH corresponding to different training examples are concatenated pair-wise ( resulting in ( NK ) 2 pairs in the case of K-shot classification ) and processed by a relation network gφr , in a similar fashion to Oreshkin et al . ( 2018 ) and Sung et al . ( 2017 ) . The ( NK ) 2 outputs are grouped by class and averaged within each group to obtain the ( 2×N ) parameters of a probability distribution in a low-dimensional space Z = Rnz , where nz dim ( θ ) , for each of the N classes . Thus , given the K-shot training samples corresponding to a class n : Dtrn = { ( xkn , y k n ) | k = 1 . . .K } the encoder gφe and relation network gφr together parameterize a class-conditional multivariate Gaussian distribution with a diagonal covariance , which we can sample from in order to output a class-dependent latent code zn ∈ Z as follows : µen , σ e n = 1 NK2 K∑ kn=1 N∑ m=1 K∑ km=1 gφr ( gφe ( xknn ) , gφe ( xkmm ) ) zn ∼ q ( zn|Dtrn ) = N ( µen , diag ( σ e n 2 ) ) ( 3 ) Intuitively , the encoder and relation network define a stochastic mapping from one or more class examples to a single code in the latent embedding space Z corresponding to that class . The final latent code can be obtained as the concatenation of class-dependent codes : z = [ z1 , z2 , . . . , zN ] . Decoding Without loss of generality , for few-shot classification , we can use the class-specific latent codes to instantiate just the top layer weights of the classifier . This allows the meta-learning in latent space to modulate the important high-level parameters of the classifier , without requiring the generator to produce very high-dimensional parameters . In this case , fθ′i is a N -way linear softmax classifier , with model parameters θ′i = { wn | n = 1 . . . N } , and each xkn can be either the raw input or some learned representation2 . Then , given the latent codes zn ∈ Z , n = 1 . . . N , the decoder function gφd : Z → Θ is used to parameterize a Gaussian distribution with diagonal covariance in model parameter space Θ , from which we can sample class-dependent parameters wn : µdn , σ d n = gφd ( zn ) wn ∼ p ( w|zn ) = N ( µdn , diag ( σ d n 2 ) ) ( 4 ) In other words , codes zn are mapped independently to the top-layer parameters θi of a softmax classifier using the decoder gφd , which is essentially a stochastic generator of model parameters . 2As before , we omit the task subscript i from wn for clarity .

Summary: This work presents an extension of the MAML framework for "learning to learn." This extension changes the space in which "inner-loop" gradient steps are taken to adapt the model to a new task, and also introduces stochasticity. The authors validate their proposed method with regression experiments in a toy setting and few-shot classification experiments on mini- and tiered-Imagenet. The latter are well known and competitive benchmarks in few-shot learning.