Meta-Learning with Implicit Processes

1 INTRODUCTION . Few-shot learning ( also known as meta-learning ) is a defining characteristic of human intelligence . Its goal is to leverage the experiences from previous tasks to form a model ( represented by metaparameters ) that can rapidly adapt to a new task using only a limited quantity of its training data . A number of meta-learning algorithms ( Finn et al. , 2018 ; Jerfel et al. , 2019 ; Ravi & Beatson , 2018 ; Rusu et al. , 2019 ; Yoon et al. , 2018 ) have recently adopted a probabilistic perspective to characterize the uncertainty in the predictions via a Bayesian treatment of the meta-parameters . Though they can consequently represent different tasks with different values of meta-parameters , it is not clear how or whether they are naturally amenable to ( a ) the characterization of a principled similarity/distance measure between tasks ( e.g. , for identifying outlier tasks that can potentially hurt training for the new task , procuring the most valuable/similar tasks/datasets to the new task , detecting task distribution shift , among others ) , ( b ) active task selection given a limited budget of expensive task queries ( see Appendix A.2.3 for an example of a real-world use case ) , and ( c ) synthetic task/dataset generation in privacy-aware applications without revealing the real data or for augmenting a limited number of previous tasks to improve generalization performance . To tackle the above challenge , this paper presents a novel implicit process-based meta-learning ( IPML ) algorithm ( Sec . 3 ) that , in contrast to existing works , explicitly represents each task as a continuous latent vector and models its probabilistic belief within the highly expressive IP1 framework ( Sec . 2 ) . Unfortunately , meta-training in IPML is computationally challenging due to its need to perform intractable exact IP inference in task adaptation.2 To resolve this , we propose a novel 1An IP ( Ma et al. , 2019 ) is a stochastic process such that every finite collection of random variables has an implicitly defined joint prior distribution . Some typical examples of IP include Gaussian processes , Bayesian neural networks , neural processes ( Garnelo et al. , 2018 ) , among others . An IP is formally defined in Def . 1 . 2The work of Ma et al . ( 2019 ) uses the well-studied Gaussian process as the variational family to perform variational inference in general applications of IP , which sacrifices the flexibility and expressivity of IP by constraining the distributions of the function outputs to be Gaussian . Such a straightforward application of IP to meta-learning has not yielded satisfactory results in our experiments ( see Appendix A.4 ) . expectation-maximization ( EM ) algorithm to perform meta-training ( Sec . 3.1 ) : In the E step , we perform task adaptation using the stochastic gradient Hamiltonian Monte Carlo sampling method ( Chen et al. , 2014 ) to draw samples from IP posterior beliefs for all meta-training tasks , which eliminates the need to learn a latent encoder ( Garnelo et al. , 2018 ) . In the M step , we optimize the meta-learning objective w.r.t . the meta-parameters using these samples . Our delicate design of the neural network architecture for meta-training in IPML allows competitive meta-learning performance to be achieved ( Sec . 3.2 ) . Our IPML algorithm offers the benefits of being amenable to ( a ) the characterization of a principled distance measure between tasks using maximum mean discrepancy ( Gretton et al. , 2012 ) , ( b ) active task selection without needing the assumption of known task contexts in ( Kaddour et al. , 2020 ) , and ( c ) synthetic task generation by modeling task-dependent input distributions ( Sec . 3.3 ) . 2 BACKGROUND AND NOTATIONS . For simplicity , the inputs ( outputs ) for all tasks are assumed to belong to the same input ( output ) space . Consider meta-learning on probabilistic regression tasks:3 Each task is generated from a task distribution and associated with a dataset ( X , yX ) where the set X and the vector yX , ( yx ) > x2X denote , respectively , the input vectors and the corresponding noisy outputs yx , f ( x ) + ✏ ( x ) ( 1 ) which are outputs of an unknown underlying function f corrupted by an i.i.d . Gaussian noise ✏ ( x ) ⇠ N ( 0 , 2 ) with variance 2 . Let f be distributed by an implicit process ( IP ) , as follows : Definition 1 ( Implicit process for meta-learning ) . Let the collection of random variables f ( · ) denote an IP parameterized by meta-parameters ✓ , that is , every finite collection { f ( x ) } x2X has a joint prior distribution p ( fX , ( f ( x ) ) > x2X ) implicitly defined by the following generative model : z ⇠ p ( z ) , f ( x ) , g✓ ( x , z ) ( 2 ) for all x 2 X where z is a latent task vector to be explained below and generator g✓ can be an arbitrary model ( e.g. , deep neural network ) parameterized by meta-parameters ✓ . Definition 1 defines valid stochastic processes if z is finite dimensional ( Ma et al. , 2019 ) . Though , in reality , a task may follow an unknown distribution , we assume the existence of an unknown function that maps each task to a latent task vector z satisfying the desired known distribution p ( z ) , like in ( Kaddour et al. , 2020 ) .4 Using p ( yX |fX ) = N ( fX , 2I ) ( 1 ) and the IP prior belief p ( fX ) from Def . 1 , we can derive the marginal likelihood p ( yX ) by marginalizing out fX . Remark 1 . Two sources of uncertainty exist in p ( yX ) : Aleatoric uncertainty in p ( yX |fX ) reflects the noise ( i.e. , modeled in ( 1 ) ) inherent in the dataset , while epistemic uncertainty in the IP prior belief p ( fX ) reflects the model uncertainty arising from the latent task prior belief p ( z ) in ( 2 ) .5 Let the sets T and T⇤ denote the meta-training and meta-testing tasks , respectively . Following the convention in ( Finn et al. , 2018 ; Gordon et al. , 2019 ; Ravi & Beatson , 2018 ; Yoon et al. , 2018 ) , for each meta-training task t 2 T , we consider a support-query ( or train-test ) split of its dataset ( Xt , yXt ) into the support set ( or training dataset ) ( X st , yX st ) and query set ( or test/evaluation dataset ) ( X qt , yX qt ) where Xt = X s t [ X q t and X st \ X q t = ; . Specifically , for a N -way K-shot classification problem , the support set has K examples per class and N classes in total . Meta-learning can be defined as an optimization problem ( Finn et al. , 2017 ; 2018 ) and its goal is to learn meta-parameters ✓ that maximize the following objective defined over all meta-training tasks : Jmeta , log Y t2T p yX qt |yX st = X t2T log Z p yX qt |fX q t p fX qt |yX st dfX qt . ( 3 ) Task adaptation p ( fX qt |yX st ) is performed via IP inference after observing the support set : p fX qt |yX st = Z z p fX qt |z p z|yX st dz . ( 4 ) 3We defer the discussion of meta-learning on probabilistic classification tasks using the robust-max likelihood ( Hernández-Lobato et al. , 2011 ) to Appendix A.1 . 4p ( z ) is often assumed to be a simple distribution like multivariate Gaussian N ( 0 , I ) ( Garnelo et al. , 2018 ) . 5Our work here considers a point estimate of meta-parameters ✓ instead of a Bayesian treatment of ✓ ( Finn et al. , 2018 ; Yoon et al. , 2018 ) . This allows us to interpret the epistemic uncertainty in p ( fX ) via p ( z ) directly . The objective Jmeta ( 3 ) is the “ test ” likelihood on the query set , which reflects the idea of “ learning to learn ” by assessing the effectiveness of “ learning on the support set ” through the query set . An alternative interpretation views p ( fX qt |yX st ) as an “ informative prior ” after observing the support set . The objective Jmeta ( 3 ) is also known as the Bayesian held-out likelihood ( Gordon et al. , 2019 ) . In a meta-testing task , adaptation is also performed via IP inference after observing its support set and evaluated on its query set . Similar to GP or any stochastic process , the input vectors of the dataset are assumed to be known/fixed beforehand . We will relax this assumption by allowing them to be unknown when our IPML algorithm is exploited for synthetic task generation ( Sec . 3.3 ) . 3 IMPLICIT PROCESS-BASED META-LEARNING ( IPML ) . 3.1 EXPECTATION MAXIMIZATION ( EM ) ALGORITHM FOR IPML . Recall that task adaptation requires evaluating p ( fX qt |yX st ) ( 4 ) . From Def . 1 , if generator g✓ ( 2 ) can be an arbitrary model ( e.g. , deep neural network ) , then p ( fX qt |yX st ) and p ( fX qt ) can not be evaluated in closed form and have to be approximated by samples . Inspired by the Monte Carlo EM algorithm ( Wei & Tanner , 1990 ) which utilizes posterior samples to obtain a maximum likelihood estimate of some hyperparameters , we propose an EM algorithm for IPML : The E step uses the stochastic gradient Hamiltonian Monte Carlo ( SGHMC ) sampling method to draw samples from p ( fX qt |yX st ) ( 4 ) , while the M step maximizes the meta-learning objective Jmeta ( 3 ) w.r.t . metaparameters ✓ : Expectation ( E ) step . Note that since fX qt = ( g✓ ( x , z ) ) > x2X qt ( 2 ) , no uncertainty exists in p ( fX qt |z ) in ( 4 ) . So , p ( fX qt |yX st ) can be evaluated using the same generator g✓ ( 2 ) and the latent task posterior belief p ( z|yX st ) , as follows : Remark 2 . Drawing samples from p ( fX qt |yX st ) is thus equivalent to first drawing samples of z from p ( z|yX st ) and then passing them as inputs to generator g✓ to obtain samples of fX qt . Hence , given a task t , adaptation p ( fX qt |yX st ) ( 4 ) essentially reduces to a task identification problem by performing IP inference to obtain the latent task posterior belief p ( z|yX st ) . This is a direct consequence of epistemic uncertainty arising from p ( z|yX st ) and p ( z ) ( Remark 1 ) . In general , p ( z|yX st ) also can not be evaluated in closed form . Instead of using variational inference ( VI ) and approximating p ( z|yX st ) with a potentially restrictive variational distribution ( Garnelo et al. , 2018 ; Kaddour et al. , 2020 ; Ma et al. , 2019 ) , we draw samples from p ( z|yX st ) using SGHMC ( Chen et al. , 2014 ) . SGHMC introduces an auxiliary random vector r and samples from a joint distribution p ( z , r|yX st ) following the Hamiltonian dynamics ( Brooks et al. , 2011 ; Neal , 1993 ) : p ( z , r|yX st ) / exp ( U ( z ) 0.5r > M 1r ) where the negative log-probability U ( z ) , log p ( z|yX st ) resembles the potential energy and r resembles the momentum . SGHMC updates z and r , as follows : z = ↵M 1r , r = ↵rzU ( z ) ↵CM 1r+N ( 0 , 2↵ ( C B ) ) where ↵ , C , M , and B are the step size , friction term , mass matrix , and Fisher information matrix , respectively.6 Note that rzU ( z ) = rz log p ( z|yX st ) = rz log p ( z , yX st ) = rz [ log p ( yX st |fX st = ( g✓ ( x , z ) ) > x2X st ) + log p ( z ) ] can be evaluated tractably . Maximization ( M ) step . We optimize Jmeta ( 3 ) w.r.t . ✓ using samples of z . The original objective Jmeta = P t2T log ( Ep ( z|yXst ) [ p ( yX qt |fX qt = ( g✓ ( x , z ) ) > x2X qt ) ] ) is not amenable to stochastic optimization with data minibatches , which is usually not an issue in a few-shot learning setting . When a huge number of data points and samples of z are considered , we can resort to optimizing the lower bound Js-meta of Jmeta by applying the Jensen ’ s inequality : Jmeta Js-meta , P t2T Ep ( fXqt |yXst ) ⇥ log p ( yX qt |fX q t ) ⇤ = P t2T Ep ( z|yXst ) ⇥ log p ( yX qt |fX q t ) ⇤ . 6The sampler hyperparameters ↵ , C , M , and B are set according to the auto-tuning method of Springenberg et al . ( 2016 ) which has been verified to work well in our experiments ; more details are given in Appendix A.2.1 .

Summary: The paper proposes a meta-learning method based on implicit process (IP) framework in which each task is represented by a latent vector. The IP setup for meta-learning seems identical to that of Neural processes [1]. In that, the key challenge for adaptation to a task based on a context/support set $(X_c, Y_c)$ is inferring the distribution $p(z|X_c, Y_c)$ over latent vectors. While [1] use amortized (variational) inference with a variational Gaussian distribution to approximate the true $p(z|X_c, Y_c)$, the paper proposes to use stochastic gradient Hamiltonian Monte Carlo (SG-HMC) for sampling latent state vectors from $p(z|X_c, Y_c)$.