Empirical Bayes Transductive Meta-Learning with Synthetic Gradients

1 INTRODUCTION . While supervised learning of deep neural networks can achieve or even surpass human-level performance ( He et al. , 2015 ; Devlin et al. , 2018 ) , they can hardly extrapolate the learned knowledge beyond the domain where the supervision is provided . The problem of solving rapidly a new task after learning several other similar tasks is called meta-learning ( Schmidhuber , 1987 ; Bengio et al. , 1991 ; Thrun & Pratt , 1998 ) ; typically , the data is presented in a two-level hierarchy such that each data point at the higher level is itself a dataset associated with a task , and the goal is to learn a meta-model that generalizes across tasks . In this paper , we mainly focus on few-shot learning ( Vinyals et al. , 2016 ) , an instance of meta-learning problems , where a task t consists of a query set dt : = { ( xt , i , yt , i ) } ni=1 serving as the test-set of the task and a support set dlt : = { ( xlt , i , ylt , i ) } n l i=1 serving as the train-set . In meta-testing1 , one is given the support set and the inputs of the query set xt : = { xt , i } ni=1 , and asked to predict the labels yt : = { yt , i } ni=1 . In meta-training , yt is provided as the ground truth . The setup of few-shot learning is summarized in Table 1 . A important distinction to make is whether a task is solved in a transductive or inductive manner , that is , whether xt is used . The inductive setting is what was originally proposed by Vinyals et al . ( 2016 ) , in which only dlt is used to generate a model . The transductive setting , as an alternative , has the advantage of being able to see partial or all points in xt before making predictions . In fact , 1To distinguish from testing and training within a task , meta-testing and meta-training are referred to as testing and training over tasks . Nichol et al . ( 2018 ) notice that most of the existing meta-learning methods involve transduction unintentionally since they use xt implicitly via the batch normalization ( Ioffe & Szegedy , 2015 ) . Explicit transduction is less explored in meta-learning , the exception is Liu et al . ( 2018 ) , who adapted the idea of label propagation ( Zhu et al. , 2003 ) from graph-based semi-supervised learning methods . We take a totally different path that meta-learn the “ gradient ” descent on xt without using yt . Due to the hierarchical structure of the data , it is natural to formulate meta-learning by a hierarchical Bayes ( HB ) model ( Good , 1980 ; Berger , 1985 ) , or alternatively , an empirical Bayes ( EB ) model ( Robbins , 1985 ; Kucukelbir & Blei , 2014 ) . The difference is that the latter restricts the learning of meta-parameters to point estimates . In this paper , we focus on the EB model , as it largely simplifies the training and testing without losing the strength of the HB formulation . The idea of using HB or EB for meta-learning is not new : Amit & Meir ( 2018 ) derive an objective similar to that of HB using PAC-Bayesian analysis ; Grant et al . ( 2018 ) show that MAML ( Finn et al. , 2017 ) can be understood as a EB method ; Ravi & Beatson ( 2018 ) consider a HB extension to MAML and compute posteriors via amortized variational inference . However , unlike our proposal , these methods do not make full use of the unlabeled data in query set . Roughly speaking , they construct the variational posterior as a function of the labeled set dlt without taking advantage of the unlabeled set xt . The situation is similar in gradient based meta-learning methods ( Finn et al. , 2017 ; Ravi & Larochelle , 2016 ; Li et al. , 2017b ; Nichol et al. , 2018 ; Flennerhag et al. , 2019 ) and many other meta-learning methods ( Vinyals et al. , 2016 ; Snell et al. , 2017 ; Gidaris & Komodakis , 2018 ) , where the mechanisms used to generate the task-specific parameters rely on groundtruth labels , thus , there is no place for the unlabeled set to contribute . We argue that this is a suboptimal choice , which may lead to overfitting when the labeled set is small and hinder the possibility of zero-shot learning ( when the labeled set is not provided ) . In this paper , we propose to use synthetic gradient ( Jaderberg et al. , 2017 ) to enable transduction , such that the variational posterior is implemented as a function of the labeled set dlt and the unlabeled set xt . The synthetic gradient is produced by chaining the output of a gradient network into autodifferentiation , which yields a surrogate of the inaccessible true gradient . The optimization process is similar to the inner gradient descent in MAML , but it iterates on the unlabeled xt rather than on the labeled dlt , since it does not rely on yt to compute the true gradient . The labeled set for generating the model for an unseen task is now optional , which is only used to compute the initialization of model weights in our case . In summary , our main contributions are the following : 1 . In section 2 and section 3 , we develop a novel empirical Bayes formulation with transduction for meta-learning . To perform amortized variational inference , we propose a parameterization for the variational posterior based on synthetic gradient descent , which incoporates the contextual information from all the inputs of the query set . 2 . In section 4 , we show in theory that a transductive variational posterior yields better generalization performance . The generalization analysis is done through the connection between empirical Bayes formulation and a multitask extension of the information bottleneck principle . In light of this , we name our method synthetic information bottleneck ( SIB ) . 3 . In section 5 , we verify our proposal empirically . Our experimental results demonstrate that our method significantly outperforms the state-of-the-art meta-learning methods on few-shot classification benchmarks under the one-shot setting . 2 META-LEARNING WITH TRANSDUCTIVE INFERENCE . The goal of meta-learning is to train a meta-model on a collection of tasks , such that it works well on another disjoint collection of tasks . Suppose that we are given a collection of N tasks for training . The associated data is denoted by D : = { dt : = ( xt , yt ) } Nt=1 . In the case of few-shot learning , we are given in addition a support set dlt in each task . In this section , we revisit the classical empirical Bayes model for meta-learning . Then , we propose to use a transductive scheme in the variational inference by implementing the variational posterior as a function of xt . 2.1 EMPIRICAL BAYES MODEL . Due to the hierarchical structure among data , it is natural to consider a hierarchical Bayes model with the marginal likelihood pf ( D ) = ∫ ψ pf ( D|ψ ) p ( ψ ) = ∫ ψ [ N∏ t=1 ∫ wt pf ( dt|wt ) p ( wt|ψ ) ] p ( ψ ) . ( 1 ) The generative process is illustrated in Figure 1 ( a , in red arrows ) : first , a meta-parameter ψ ( i.e. , hyper-parameter ) is sampled from the hyper-prior p ( ψ ) ; then , for each task , a task-specific parameter wt is sampled from the prior p ( wt|ψ ) ; finally , the dataset is drawn from the likelihood pf ( dt|wt ) . Without loss of generality , we assume the log-likelihood model factorizes as log pf ( dt|wt ) = n∑ i=1 log pf ( yt , i|xt , i , wt ) + log p ( xt , i|wt ) , = n∑ i=1 − 1 n ` t ( ŷt , i ( f ( xt , i ) , wt ) , yt , i ) + constant . ( 2 ) It is the setting advocated by Minka ( 2005 ) , in which the generative model p ( xt , i|wt ) can be safely ignored since it is irrelevant to the prediction of yt . To simplify the presentation , we still keep the notation pf ( dt|wt ) for the likelihood of the task t and use ` t to specify the discriminative model , which is also referred to as the task-specific loss , e.g. , the cross entropy loss . The first argument in ` t is the prediction , denoted by ŷt , i = ŷt , i ( f ( xt , i ) , wt ) , which depends on the feature representation f ( xt , i ) and the task-specific weight wt . Note that rather than following a fully Bayesian approach , we leave some random variables to be estimated in a frequentist way , e.g. , f is a meta-parameter of the likelihood model shared by all tasks , for which we use a point estimate . As such , the posterior inference about these variables will be largely simplified . For the same reason , we derive the empirical Bayes ( Robbins , 1985 ; Kucukelbir & Blei , 2014 ) by taking a point estimate on ψ . The marginal likelihood now reads as pψ , f ( D ) = N∏ t=1 ∫ wt pf ( dt|wt ) pψ ( wt ) . ( 3 ) We highlight the meta-parameters as subscripts of the corresponding distributions to distinguish from random variables . Indeed , we are not the first to formulate meta-learning as empirical Bayes . The overall model formulation is essentially the same as the ones considered by Amit & Meir ( 2018 ) ; Grant et al . ( 2018 ) ; Ravi & Beatson ( 2018 ) . Our contribution lies in the variational inference for empirical Bayes . 2.2 AMORTIZED INFERENCE WITH TRANSDUCTION . As in standard probabilistic modeling , we derive an evidence lower bound ( ELBO ) on the log version of ( 3 ) by introducing a variational distribution qθt ( wt ) for each task with parameter θt : log pψ , f ( D ) ≥ N∑ t=1 [ Ewt∼qθt [ log pf ( dt|wt ) ] −DKL ( qθt ( wt ) ‖pψ ( wt ) ) ] . ( 4 ) The variational inference amounts to maximizing the ELBO with respect to θ1 , . . . , θN , which together with the maximum likelihood estimation of the meta-parameters , we have the following optimization problem : min ψ , f min θ1 , ... , θN 1 N N∑ t=1 [ Ewt∼qθt [ − log pf ( dt|wt ) ] +DKL ( qθt ( wt ) ‖pψ ( wt ) ) ] . ( 5 ) However , the optimization in ( 5 ) , as N increases , becomes more and more expensive in terms of the memory footprint and the computational cost . We therefore wish to bypass this heavy optimization and to take advantage of the fact that individual KL terms indeed share the same structure . To this end , instead of introducing N different variational distributions , we consider a parameterized family of distributions in the form of qφ ( · ) , which is defined implicitly by a deep neural network φ taking as input either dlt or d l t plus xt , that is , qφ ( dlt ) or qφ ( dlt , xt ) . Note that we can not use entire dt , since we do not have access to yt during meta-testing . This amortization technique was first introduced in the case of variational autoencoders ( Kingma & Welling , 2013 ; Rezende et al. , 2014 ) , and has been extended to Bayesian inference in the case of neural processes ( Garnelo et al. , 2018 ) . Since dlt and xt are disjoint , the inference scheme is inductive for a variational posterior qφ ( dlt ) . As an example , MAML ( Finn et al. , 2017 ) takes qφ ( dlt ) as the Dirac delta distribution , where φ ( d l t ) = θ K t , is the K-th iterate of the stochastic gradient descent θk+1t = θ k t + η∇θEwt∼qθkt [ log p ( dlt|wt ) ] with θ0t = φ , a learnable initialization . ( 6 ) In this work , we consider a transductive inference scheme with variational posterior qφ ( dlt , xt ) . The inference process is shown in Figure 1 ( a , in green arrows ) . Replacing each qθt in ( 5 ) by qφ ( dlt , xt ) , the optimization problem becomes min ψ , f min φ 1 N N∑ t=1 [ Ewt∼qφ ( dlt , xt ) [ − log pf ( dt|wt ) ] +DKL ( qφ ( dlt , xt ) ( wt ) ‖pψ ( wt ) ) ] . ( 7 ) In a nutshell , the meta-model to be optimized includes the feature network f , the hyper-parameter ψ from the empirical Bayes formulation and the amortization network φ from the variational inference .

Summary: This paper addresses the issue of meta-learning in a transductive learning setting. That is, it aims to learn a model from multiple tasks and make it generalise to a new task in order to solve it efficiently. In the transductive setting, the query set (i.e., containing the unlabeled test data) of the new task is taken into account when learning the model.