Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learning

1 Introduction . Recent advances in machine learning and neural networks have resulted in effective but parameterbloated , data-hungry models . When the training data for a target task of interest is insufficient , such overparameterized models may easily overfit to the training data and exhibit poor generalization abilities . To address this problem , several research efforts have focused on designing a learning strategy that can leverage the training data of other tasks for the sake of improving the performance of some specific target task ( s ) . Specifically , in the meta-learning ( also called learning-to-learn or lifelong-learning ) setting [ 6 , 21 ] , a meta-learner first extracts knowledge from a set of observed ( metatraining ) tasks and subsequently , this knowledge enables a base-learner to better adapt to the new , possibly data-limited target ( meta-testing ) task . The meta-learning framework has been successfully applied and made significant practical impact on computer vision [ 23 ] , language understanding [ 7 ] , reinforcement learning [ 9 ] and many other research fields . In parallel to its impressive empirical success , a series of theoretical works [ 24 , 18 , 3 , 22 ] study how meta-learning utilizes the knowledge obtained from the observed task data and how it generalizes to the unseen target task . Among the generalization bounds , PAC-Bayes bounds [ 15 , 12 ] are considered especially tight and have already been proposed for meta-learning [ 18 , 3 , 22 ] . However , there still remains a gap between these existing PAC-Bayesian bounds and their practical application ( especially in the few-shot setting ) , which originates from the assumption that the observed task environment T̃ and the target task environment T are the same . In the PAC-Bayesian meta-learning setting , a task environment T is a distribution from which ( D , m ) is drawn from , where D is the data distribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and m is the number of training examples for the task . Although there is research work studying the case of general environment change ( e.g . [ 19 ] ) or data domain change ( e.g . [ 11 ] ) , to the best of our knowledge , there is little work focusing on the case where only the number of training examples m̃ in the observed tasks and m in the target task do not follow the same distribution . In practice , such mismatch commonly happens , because there is usually significantly more data in observed tasks than the target tasks , especially in the few-shot case . Without explicitly addressing this mismatch , the scope of the current theory is severely limited , and it prohibits a useful analysis on practical meta-learning algorithms such as MAML [ 9 ] . For example , when the number of training examples m in the target task is small , the existing bounds yield a large generalization gap which grows with O ( 1/m ) . In this paper , we bring the theory closer to practice by studying the setting where there are significantly more training examples in the observed task than in the target task ( i.e. , m̃ m ) . In Section 3.1 , we study two practical meta-training strategies and provide their PAC-Bayesian bounds in Theorem 3 and Theorem 4 . Both results are able to bring down the scaling coefficient of the bound from O ( 1/m ) to O ( 1/m̃ ) . However , Theorem 3 introduces a penalty term in the bound that captures the discrepancy between the observed and target task environment . Motivated by MAML [ 9 ] , we show with Theorem 4 that we can eliminate the penalty term by utilizing a subsampling strategy , yielding a much tighter bound . This theoretical work also bridges the gap from practice to theory , as we further show that the maximum-a-posteriori ( MAP ) estimates of our bounds ( in which the base-learner and the hyperposterior are both approximated by Dirac-measures ) yield various popular meta-learning algorithms , including multi-task pretraining [ 23 ] , Reptile [ 16 ] and MAML [ 9 ] . In that sense , our PAC-Bayesian theories provide a different perspective for understanding and justifying these commonly used algorithms ( Section 3.2 ) . Lastly , in Section 4 , we propose PACMAML , a novel PAC-Bayesian meta-learning algorithm based on Theorem 4 . As opposed to MAML , our algorithm does not have higher-order derivatives in the gradient , and therefore represents a significant improvement in computational efficiency . In Section 5 , we conduct numerical experiments that empirically support the correctness of our theorems , and report the effectiveness of the new PACMAML algorithm , which obtains superior results on several few-shot benchmark datasets . 2 Preliminaries . We begin by reviewing the background and settings of the existing PAC-Bayesian bounds for metalearning . Our notation mainly follows that of [ 22 ] , which is itself adapted from [ 18 , 3 , 6 ] . PAC-Bayesian for Supervised Learning In supervised learning , a learning task is characterized by a data distribution D over a data domain Z where every example z = ( x , y ) . A hypothesis h from the hypothesis space H allows us to make predictions based on inputs x . The quality of the predictions is measured by a loss function l ( h , z ) , where the goal is to minimize the expected loss L ( h , D ) = Ez∼D l ( h , z ) . Typically , D is unknown and instead we are given a set of m observations S ∼ Dm = { zi ∼ D } mi=1 , in which case the empirical error on S is simply L̂ ( h , S ) = 1 m ∑m i=1 l ( h , zi ) . In the PAC-Bayesian setting , we assume that the learner has prior knowledge of the hypothesis space H in the form of a prior distribution P ( h ) . When the learner observes a training dataset S , it updates the prior into a posterior distribution Q . We formalize such a base learner Q ( S , P ) that takes a dataset and a prior as input and outputs a posterior . The expected error of the posterior Q is called the Gibbs error L ( Q , D ) = Eh∼Q L ( h , D ) , and its empirical counterpart is L̂ ( Q , S ) = Eh∼Q L̂ ( h , S ) . The PAC-Bayesian framework provides the following bound over L ( Q , D ) based on its empirical estimate L̂ ( Q , S ) . Theorem 1 ( [ 2 , 12 ] ) Given a data distribution D , a hypothesis space H , a prior P , a confidence level δ ∈ ( 0 , 1 ] , and β > 0 , with probability at least 1− δ over samples S ∼ Dm , we have for all posterior Q , L ( Q , D ) ≤ L̂ ( Q , S ) + 1 β ( DKL ( Q‖P ) + log 1 δ ) + m β Ψ ( β m ) ( 1 ) where Ψ ( β ) = logEh∼P Ez∼D exp ( β ( l ( h , z ) − L ( h , D ) ) ) . PAC-Bayesian for Meta-Learning In the meta-learning setting , the meta-learner observes different tasks τi = ( Di , mi ) during the meta-training stage , where all tasks share the same data domain Z , hypothesis space H and loss function l ( h , z ) . For each observed task τi , the meta-learner observes a training set Si of size mi which is assumed to be sampled i.i.d . from its respective data distribution Di ( that is , Si ∈ Dmii ) . We further assume that each task τi = ( Di , mi ) is drawn i.i.d . from an environment T , which itself is a probability distribution over the data distributions and the sample sizes . The goal of meta-learning is to extract knowledge from the observed tasks τi , which can then be used as prior knowledge for learning on new ( yet unobserved ) target tasks τ = ( D , m ) ∼ T . This prior knowledge is represented as a prior distribution P ( h ) over learning hypotheses h , and it is subsequently used by the base learner Q ( S , P ) for inference over the target tasks . In the meta-learning PAC-Bayes framework , the meta-learner presumes a hyper-prior P ( P ) as a distribution over priors P . Upon observing datasets S1 , . . . , Sn from multiple tasks , the meta-learner updates the hyper-prior to a hyper-posterior Q ( P ) . The performance of this hyper-posterior , also called the transfer-error , is measured as the expected Gibbs error when sampling priors P from Q and applying the base learner : R ( Q , T ) : = EP∼Q E ( D , m ) ∼T ES∼Dm [ L ( Q ( S , P ) , D ) ] . ( 2 ) While R ( Q , T ) is unknown in practice , it can be estimated using the empirical error , R̂ ( Q , Sni=1 ) : = EP∼Q [ 1 n n∑ i=1 L̂ ( Q ( Si , P ) , Si ) ] . ( 3 ) In [ 18 , 22 ] , the following PAC-Bayesian meta-learning bound is provided : Theorem 2 ( [ 18 , 22 ] ) Given a task environment T and a set of n observed tasks ( Di , mi ) ∼ T , let P be a fixed hyper-prior and λ > 0 , β > 0 , with probability at least 1 − δ over samples S1 ∈ Dm11 , . . . , Sn ∈ Dmnn , we have , for all base learner Q and all hyper-posterior Q , R ( Q , T ) ≤R̂ ( Q , Sni=1 ) + ( 1 λ + 1 nβ ) DKL ( Q‖P ) + 1 nβ n∑ i=1 EP∼Q [ DKL ( Q ( Si , P ) ‖P ) ] + C ( δ , λ , β , n , mi ) . ( 4 ) Here C ( δ , λ , β , n , mi ) contains Ψ and 1δ terms as in Eq . ( 1 ) ( see Appendix A.1 ) , and can be bounded by a function that is independent ofQ for both bounded and unbounded loss functions under moment constraints ( see details in [ 22 ] ) . From a Bayesian perspective , meta-learning attempts to learn a good hyper-posterior Q such that for all tasks in the task environment T , the divergence terms DKL ( Q ( Si , P ) ‖P ) would be substantially smaller in expectation when P ∼ Q compared to when P ∼ P , such as in the ordinary supervised learning setting of Eq. ( 1 ) . The hyperparameters λ and β can be adjusted to balance between the first three terms of the bound and the C function . Defining the harmonic mean of mi as m̃ = ( ∑n i=1 1/nmi ) −1 , a common choice is λ ∝ n and β ∝ m̃ * . In this case , the generalization gap R ( Q , T ) − R̂ ( Q , Sni=1 ) becomes at least O ( 1m̃ ) ( from the 3rd-term on the RHS of Eq.4 ) . In the next section , we examine an assumption in this bound which makes it impractical for the few-shot setting .

Summary: Existing meta-learning techniques assume that the number of samples for teach task are drawn from the same distribution.  Theorem 3 shows that a naive decoupling of the number of samples introduces a term to the gap that penalizes additional samples.  The paper bypasses this issue by introducing a fix-sized subset that is used to train a base model.  The additional samples are then used to evaluate the empirical risk, thus removing the aforementioned unsatisfactory term.  The MAML and Reptile algorithms are recovered by considering the derived bound and normal priors along with both delta posteriors (ie MAP solutions) and normal posteriors.  Results are reported on synthetic regression, image classification, and product review classification.