Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learning

1 Introduction . Recent advances in machine learning and neural networks have resulted in effective but parameterbloated , data-hungry models . When the training data for a target task of interest is insufficient , such overparameterized models may easily overfit to the training data and exhibit poor generalization abilities . To address this problem , several research efforts have focused on designing a learning strategy that can leverage the training data of other tasks for the sake of improving the performance of some specific target task ( s ) . Specifically , in the meta-learning ( also called learning-to-learn or lifelong-learning ) setting [ 6 , 21 ] , a meta-learner first extracts knowledge from a set of observed ( metatraining ) tasks and subsequently , this knowledge enables a base-learner to better adapt to the new , possibly data-limited target ( meta-testing ) task . The meta-learning framework has been successfully applied and made significant practical impact on computer vision [ 23 ] , language understanding [ 7 ] , reinforcement learning [ 9 ] and many other research fields . In parallel to its impressive empirical success , a series of theoretical works [ 24 , 18 , 3 , 22 ] study how meta-learning utilizes the knowledge obtained from the observed task data and how it generalizes to the unseen target task . Among the generalization bounds , PAC-Bayes bounds [ 15 , 12 ] are considered especially tight and have already been proposed for meta-learning [ 18 , 3 , 22 ] . However , there still remains a gap between these existing PAC-Bayesian bounds and their practical application ( especially in the few-shot setting ) , which originates from the assumption that the observed task environment T̃ and the target task environment T are the same . In the PAC-Bayesian meta-learning setting , a task environment T is a distribution from which ( D , m ) is drawn from , where D is the data distribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . and m is the number of training examples for the task . Although there is research work studying the case of general environment change ( e.g . [ 19 ] ) or data domain change ( e.g . [ 11 ] ) , to the best of our knowledge , there is little work focusing on the case where only the number of training examples m̃ in the observed tasks and m in the target task do not follow the same distribution . In practice , such mismatch commonly happens , because there is usually significantly more data in observed tasks than the target tasks , especially in the few-shot case . Without explicitly addressing this mismatch , the scope of the current theory is severely limited , and it prohibits a useful analysis on practical meta-learning algorithms such as MAML [ 9 ] . For example , when the number of training examples m in the target task is small , the existing bounds yield a large generalization gap which grows with O ( 1/m ) . In this paper , we bring the theory closer to practice by studying the setting where there are significantly more training examples in the observed task than in the target task ( i.e. , m̃ m ) . In Section 3.1 , we study two practical meta-training strategies and provide their PAC-Bayesian bounds in Theorem 3 and Theorem 4 . Both results are able to bring down the scaling coefficient of the bound from O ( 1/m ) to O ( 1/m̃ ) . However , Theorem 3 introduces a penalty term in the bound that captures the discrepancy between the observed and target task environment . Motivated by MAML [ 9 ] , we show with Theorem 4 that we can eliminate the penalty term by utilizing a subsampling strategy , yielding a much tighter bound . This theoretical work also bridges the gap from practice to theory , as we further show that the maximum-a-posteriori ( MAP ) estimates of our bounds ( in which the base-learner and the hyperposterior are both approximated by Dirac-measures ) yield various popular meta-learning algorithms , including multi-task pretraining [ 23 ] , Reptile [ 16 ] and MAML [ 9 ] . In that sense , our PAC-Bayesian theories provide a different perspective for understanding and justifying these commonly used algorithms ( Section 3.2 ) . Lastly , in Section 4 , we propose PACMAML , a novel PAC-Bayesian meta-learning algorithm based on Theorem 4 . As opposed to MAML , our algorithm does not have higher-order derivatives in the gradient , and therefore represents a significant improvement in computational efficiency . In Section 5 , we conduct numerical experiments that empirically support the correctness of our theorems , and report the effectiveness of the new PACMAML algorithm , which obtains superior results on several few-shot benchmark datasets . 2 Preliminaries . We begin by reviewing the background and settings of the existing PAC-Bayesian bounds for metalearning . Our notation mainly follows that of [ 22 ] , which is itself adapted from [ 18 , 3 , 6 ] . PAC-Bayesian for Supervised Learning In supervised learning , a learning task is characterized by a data distribution D over a data domain Z where every example z = ( x , y ) . A hypothesis h from the hypothesis space H allows us to make predictions based on inputs x . The quality of the predictions is measured by a loss function l ( h , z ) , where the goal is to minimize the expected loss L ( h , D ) = Ez∼D l ( h , z ) . Typically , D is unknown and instead we are given a set of m observations S ∼ Dm = { zi ∼ D } mi=1 , in which case the empirical error on S is simply L̂ ( h , S ) = 1 m ∑m i=1 l ( h , zi ) . In the PAC-Bayesian setting , we assume that the learner has prior knowledge of the hypothesis space H in the form of a prior distribution P ( h ) . When the learner observes a training dataset S , it updates the prior into a posterior distribution Q . We formalize such a base learner Q ( S , P ) that takes a dataset and a prior as input and outputs a posterior . The expected error of the posterior Q is called the Gibbs error L ( Q , D ) = Eh∼Q L ( h , D ) , and its empirical counterpart is L̂ ( Q , S ) = Eh∼Q L̂ ( h , S ) . The PAC-Bayesian framework provides the following bound over L ( Q , D ) based on its empirical estimate L̂ ( Q , S ) . Theorem 1 ( [ 2 , 12 ] ) Given a data distribution D , a hypothesis space H , a prior P , a confidence level δ ∈ ( 0 , 1 ] , and β > 0 , with probability at least 1− δ over samples S ∼ Dm , we have for all posterior Q , L ( Q , D ) ≤ L̂ ( Q , S ) + 1 β ( DKL ( Q‖P ) + log 1 δ ) + m β Ψ ( β m ) ( 1 ) where Ψ ( β ) = logEh∼P Ez∼D exp ( β ( l ( h , z ) − L ( h , D ) ) ) . PAC-Bayesian for Meta-Learning In the meta-learning setting , the meta-learner observes different tasks τi = ( Di , mi ) during the meta-training stage , where all tasks share the same data domain Z , hypothesis space H and loss function l ( h , z ) . For each observed task τi , the meta-learner observes a training set Si of size mi which is assumed to be sampled i.i.d . from its respective data distribution Di ( that is , Si ∈ Dmii ) . We further assume that each task τi = ( Di , mi ) is drawn i.i.d . from an environment T , which itself is a probability distribution over the data distributions and the sample sizes . The goal of meta-learning is to extract knowledge from the observed tasks τi , which can then be used as prior knowledge for learning on new ( yet unobserved ) target tasks τ = ( D , m ) ∼ T . This prior knowledge is represented as a prior distribution P ( h ) over learning hypotheses h , and it is subsequently used by the base learner Q ( S , P ) for inference over the target tasks . In the meta-learning PAC-Bayes framework , the meta-learner presumes a hyper-prior P ( P ) as a distribution over priors P . Upon observing datasets S1 , . . . , Sn from multiple tasks , the meta-learner updates the hyper-prior to a hyper-posterior Q ( P ) . The performance of this hyper-posterior , also called the transfer-error , is measured as the expected Gibbs error when sampling priors P from Q and applying the base learner : R ( Q , T ) : = EP∼Q E ( D , m ) ∼T ES∼Dm [ L ( Q ( S , P ) , D ) ] . ( 2 ) While R ( Q , T ) is unknown in practice , it can be estimated using the empirical error , R̂ ( Q , Sni=1 ) : = EP∼Q [ 1 n n∑ i=1 L̂ ( Q ( Si , P ) , Si ) ] . ( 3 ) In [ 18 , 22 ] , the following PAC-Bayesian meta-learning bound is provided : Theorem 2 ( [ 18 , 22 ] ) Given a task environment T and a set of n observed tasks ( Di , mi ) ∼ T , let P be a fixed hyper-prior and λ > 0 , β > 0 , with probability at least 1 − δ over samples S1 ∈ Dm11 , . . . , Sn ∈ Dmnn , we have , for all base learner Q and all hyper-posterior Q , R ( Q , T ) ≤R̂ ( Q , Sni=1 ) + ( 1 λ + 1 nβ ) DKL ( Q‖P ) + 1 nβ n∑ i=1 EP∼Q [ DKL ( Q ( Si , P ) ‖P ) ] + C ( δ , λ , β , n , mi ) . ( 4 ) Here C ( δ , λ , β , n , mi ) contains Ψ and 1δ terms as in Eq . ( 1 ) ( see Appendix A.1 ) , and can be bounded by a function that is independent ofQ for both bounded and unbounded loss functions under moment constraints ( see details in [ 22 ] ) . From a Bayesian perspective , meta-learning attempts to learn a good hyper-posterior Q such that for all tasks in the task environment T , the divergence terms DKL ( Q ( Si , P ) ‖P ) would be substantially smaller in expectation when P ∼ Q compared to when P ∼ P , such as in the ordinary supervised learning setting of Eq. ( 1 ) . The hyperparameters λ and β can be adjusted to balance between the first three terms of the bound and the C function . Defining the harmonic mean of mi as m̃ = ( ∑n i=1 1/nmi ) −1 , a common choice is λ ∝ n and β ∝ m̃ * . In this case , the generalization gap R ( Q , T ) − R̂ ( Q , Sni=1 ) becomes at least O ( 1m̃ ) ( from the 3rd-term on the RHS of Eq.4 ) . In the next section , we examine an assumption in this bound which makes it impractical for the few-shot setting .

Summary: Previous PAC-Bayesian Meta-learning bounds assume the number of samples per task follows the same distribution in the meta-training and in meta-testing.  This paper relaxes this assumption, allowing for a shift in the number-of-samples distribution from meta-training to meta-testing (Thm 3). In Thm 4, the paper suggests a learning strategy that omits the extra distribution shift of Thm 3, by using a smaller data set.  The paper shows how the popular meta-learning algorithms, Reptile, and MAML can be justified using the PAC-Bayes framework, and also introduces a new algorithm called PACMAML.  Experimental results show the method outperforms other algorithms on several few-shot benchmarks.