Bayesian Online Meta-Learning

Neural networks are known to suffer from catastrophic forgetting when trained on sequential datasets . While there have been numerous attempts to solve this problem for large-scale supervised classification , little has been done to overcome catastrophic forgetting for few-shot classification problems . Few-shot metalearning algorithms often require all few-shot tasks to be readily available in a batch for training . The popular gradient-based model-agnostic meta-learning algorithm ( MAML ) is a typical algorithm that suffers from these limitations . This work introduces a Bayesian online meta-learning framework to tackle the catastrophic forgetting and the sequential few-shot tasks problems . Our framework incorporates MAML into a Bayesian online learning algorithm with Laplace approximation or variational inference . This framework enables few-shot classification on a range of sequentially arriving datasets with a single meta-learned model and training on sequentially arriving few-shot tasks . The experimental evaluations demonstrate that our framework can effectively prevent catastrophic forgetting and is capable of online meta-learning in various few-shot classification settings . 1 INTRODUCTION . Image classification models and algorithms often require an enormous amount of labelled examples for training to achieve state-of-the-art performance . Labelled examples can be expensive and time-consuming to acquire . Human visual systems , on the other hand , are able to recognise new classes after being shown a few labelled examples . Few-shot classification ( Miller et al. , 2000 ; Li et al. , 2004 ; 2006 ; Lake et al. , 2011 ) tackles this issue by learning to adapt to unseen classes ( known as novel classes ) with very few labelled examples from each class . Recent works show that metalearning provides promising approaches to few-shot classification problems ( Santoro et al. , 2016 ; Finn et al. , 2017 ; Li et al. , 2017 ; Ravi & Larochelle , 2017 ) . Meta-learning or learning-to-learn ( Schmidhuber , 1987 ; Thrun & Pratt , 1998 ) takes the learning process a level deeper – instead of learning from the labelled examples in the training classes ( known as base classes ) , meta-learning learns the example-learning process . The training process in meta-learning that utilises the base classes is called the meta-training stage , and the evaluation process that reports the few-shot performance on the novel classes is known as the meta-evaluation stage . Despite being a promising solution to few-shot classification problems , meta-learning methods suffer from several limitations : 1 . Unable to continually learn from sequential few-shot tasks : It is mandatory to have all base classes readily available for meta-training . Such meta-learning algorithms often require sampling a number of few-shot tasks in every iteration for optimisation . 2 . Unable to retain few-shot classification ability on sequential datasets that have evident distributional shift : A meta-learned model is restricted to perform few-shot classification on a specific dataset , in the sense that the base and novel classes have to originate from the same dataset distribution . A meta-learned model loses its few-shot classification ability on previous datasets as new ones arrive subsequently for meta-training . We emphasise that the task mentioned in this paper refers to the few-shot task for meta-learning . This paper considers meta-learning a single model for few-shot classification in the sequential datasets and sequential few-shot tasks settings respectively . We introduce a Bayesian online meta-learning framework that can train a few-shot learning model under the sequential few-shot tasks setting and train a model that is applicable to a broader scope of few-shot classification datasets by overcoming catastrophic forgetting . We extend the Bayesian online learning ( BOL ) framework ( Opper , 1998 ) to a Bayesian online meta-learning framework using the model-agnostic meta-learning ( MAML ) algorithm ( Finn et al. , 2017 ) . MAML finds a good model parameter initialisation ( called meta-parameters ) that can quickly adapt to novel classes using very few labelled examples , while BOL provides a principled framework for finding the posterior of the model parameters . Our framework aims to combine both BOL and MAML to find the posterior of the meta-parameters . Our work builds on Ritter et al . ( 2018a ) which combines the BOL framework and Laplace approximation with block-diagonal Kronecker-factored Fisher approximation , and Nguyen et al . ( 2018 ) which uses variational inference with BOL to overcome catastrophic forgetting in large-scale supervised classification . An important reason to implement Bayesian inference over non-Bayesian methods for an online @ @ @ Newsetting is that BOL provides a grounded framework that suggests using the previous posterior as the prior recursively . Bayesian inference inherits an advantage for robust meta-learning ( Yoon et al. , 2018 ) to overcome training instability problems addressed by Antoniou et al . ( 2019 ) . BOL implicitly keeps a memory on previous knowledge via the posterior , in contrast to recent online meta-learning methods that explicitly accumulate previous data in a task buffer ( Finn et al. , 2019 ; Zhuang et al. , 2019 ) . Explicitly keeping a memory on previous data often triggers an important question : how should the carried-forward data be processed in future task rounds , in order to accumulate knowledge ? Finn et al . ( 2019 ) update the meta-parameters at each iteration using previous few-shot tasks in the task buffer . This defeats the purpose of online learning , which by definition means to update the parameters each round using only the new data encountered . Having to re-train on previous data to avoid forgetting also increases the training time as the data accumulate ( Finn et al. , 2019 ; He et al. , 2019 ) . Certainly one can clamp the amount of data at some maximal limit and sample from the buffer , but the final performance of such an algorithm would be dependent on the samples being informative and of good quality which may vary across different seed runs . In contrast to memorising the datasets , having an implicit memory via the posterior automatically deals with the question on how to process carried-forward data and allows a better carry forward in previous experiences . Below are the contributions we make in this paper : • We develop the Bayesian online meta-learning ( BOML ) framework for sequential few-shot classification problems . Under this framework we introduce the algorithms Bayesian online meta-learning with Laplace approximation ( BOMLA ) and Bayesian online meta-learning with variational inference ( BOMVI ) . • We propose a simple approximation to the Fisher corresponding to the BOMLA algorithm that carries over the desirable block-diagonal Kronecker-factored structure from the Fisher approximation in the non-meta-learning setting . • We demonstrate that BOML can overcome catastrophic forgetting in the sequential few-shot datasets setting with apparent distributional shift in the datasets . • We demonstrate that BOML can continually learn to few-shot classify the novel classes in the sequential meta-training few-shot tasks setting . 2 META-LEARNING . Most meta-learning algorithms comprise an inner loop for example-learning and an outer loop that learns the example-learning process . Such algorithms often require sampling a meta-batch of tasks at each iteration , where a task is formed by sampling a subset of classes from the pool of base classes or novel classes during meta-training or meta-evaluation respectively . The N -way K-shot task , for instance , refers to sampling N classes and using K examples per class for few-shot quick adaptation . An offline meta-learning algorithm learns a few-shot classification model only for a specific dataset Dt+1 where all base classes of Dt+1 have to be readily available for meta-training . For notational convenience , we drop the t + 1 subscript in this section , as there is only one dataset involved in offline meta-learning . The datasetDt+1 is divided into the set of base classes D̃ and novel classes D̂ for meta-training and meta-evaluation respectively . Upon completing meta-training on the base class set D̃ , the goal of few-shot classification is to perform well on an unseen task D̂∗ sampled from the novel class set D̂ after a quick adaptation on a small subset D̂∗ , S ( known as the support set ) of D̂∗ . The performance of this unseen task is evaluated on the query set D̂∗ , Q , where D̂∗ , Q = D̂∗\D̂∗ , S . Since D̂ is not accessible during meta-training , this support-query split is mimicked on the base class set D̃ for meta-training . Model-agnostic meta-learning We are interested in the well-known meta-learning algorithm MAML ( Finn et al. , 2017 ) . Each updating step of MAML aims to improve the ability of the metaparameters to act as a good model initialisation for a quick adaptation on unseen tasks . Each iteration of the MAML algorithm samples M tasks from the base class set D̃ and runs a few steps of stochastic gradient descent ( SGD ) for an inner loop task-specific learning . The number of tasks sampled per iteration is known as the meta-batch size . For task m , the inner loop outputs the task-specific parameters θ̃m from a k-step SGD quick adaptation on the objective L ( θ , D̃m , S ) with the support set D̃m , S and initialised at θ : θ̃m = SGDk ( L ( θ , D̃m , S ) ) , ( 1 ) where m = 1 , . . . , M . The outer loop gathers all task-specific adaptations to update the metaparameters θ using the loss L ( θ̃m , D̃m , Q ) on the query set D̃m , Q . The overall MAML optimisation objective is arg min θ 1 M M∑ m=1 L ( SGDk ( L ( θ , D̃m , S ) ) , D̃m , Q ) . ( 2 ) Like most offline meta-learning algorithms , MAML requires all base classes to be readily available for tasks-sampling at each iteration . We aim to overcome this limitation by meta-learning a model that can few-shot classify unseen tasks from the novel classes , while the tasks from the base classes arrive sequentially for meta-training . MAML also assumes a stationary task distribution during meta-training and meta-evaluation . Under this assumption , a meta-learned model is only applicable to a specific dataset distribution . When the model encounters a sequence of datasets with apparent distributional shift , it loses the few-shot classification ability on previous datasets as new ones arrive for meta-training . Our work also aims to meta-learn a single model for few-shot classification on multiple datasets that arrive sequentially for meta-training . We achieve these two goals by incorporating MAML into the BOL framework to give the Bayesian online meta-learning ( BOML ) framework that finds the posterior of the meta-parameters . 3 OVERVIEW OF OUR BAYESIAN ONLINE META-LEARNING APPROACH . Our central contribution is to extend the benefits of meta-learning to the Bayesian online scenario , thereby training models that can generalise across tasks whilst dealing with parameter uncertainty in the setting of sequential tasks or sequential datasets . Sequential datasets setting In this setting , online meta-training occurs sequentially on the @ @ @ Newdatasets D1 , . . . , DT . Each dataset Di can be seen as a knowledge domain with an associated underlying task distribution p ( Ti ) . A newly-arrived Dt+1 is separated into the base class set D̃t+1 and novel class set D̂t+1 for meta-training and meta-evaluation respectively , where the tasks in these two stages are drawn from the task distribution p ( Tt+1 ) . Sequential tasks setting The sequential tasks setting only involves one dataset D with an associ- @ @ @ Newated underlying task distribution p ( T ) , where D is separated into the base and novel class sets . In this setting , D̃1 , . . . , D̃t+1 denote the non-overlapping tasks formed from the base class set and they arrive sequentially for meta-training . These tasks D̃1 , . . . , D̃t+1 and the meta-evaluation tasks are drawn from the task distribution p ( T ) . Notationally , for both sequential tasks and sequential datasets settings , let D̃St+1 and D̃ Q t+1 denote the collection of support sets and query sets respectively from D̃t+1 , so that D̃t+1 = D̃St+1 ∪ D̃ Q t+1 . We are interested in a MAP estimate θ∗ = arg maxθ p ( θ|D̃1 : t+1 ) . Using Bayes ’ rule on the posterior gives the recursive formula p ( θ|D̃1 : t+1 ) ∝ p ( D̃St+1 , D̃ Q t+1|θ ) p ( θ|D̃1 : t ) ( 3 ) = p ( D̃Qt+1|θ , D̃St+1 ) p ( D̃St+1|θ ) p ( θ|D̃1 : t ) ( 4 ) = { ∫ p ( D̃Qt+1|θ̃ ) p ( θ̃|θ , D̃St+1 ) dθ̃ } p ( D̃St+1|θ ) p ( θ|D̃1 : t ) ( 5 ) where Eq . ( 3 ) follows from the assumption that each dataset is independent given θ . From the meta-learning perspective , the parameters θ̃ introduced in Eq . ( 5 ) can be viewed as the task-specific parameters in MAML . There are various choices for the distribution p ( θ̃|θ , D̃St+1 ) in Eq . ( 5 ) . In particular if we choose to set it as the deterministic function of taking several steps of SGD on loss L with the support set collection D̃St+1 and initialised at θ , we have p ( θ̃|θ , D̃St+1 ) = 1 { θ̃ = SGDk ( L ( θ , D̃St+1 ) ) } . ( 6 ) and this recovers the MAML inner loop with SGD quick adaptation in Eq . ( 1 ) . The recursion given by Eq . ( 5 ) forms the basis of our approach and the remainder of this paper explains how we implement this . In order to do so we give a mini tutorial in Appendix A on Bayesian online learning , Laplace approximation and variational continual learning .

Summary: The paper proposes an Bayesian approach to online meta-learning. This is done by lifting (approximate) sequential Bayesian inference from the model parameters to the meta-parameters. Two approaches are proposed to do this: (i) Laplace Approximation (LA), thereby extending Ritter et al’s method from online learning to online meta-learning; and (ii) VI, thereby extending Nguyen et al’s Variational Continual Learning (VCL) method in a similar way. Experiments are performed by converting existing meta-learning benchmarks into online settings in two ways, which they call “sequential tasks” and “sequential datasets” respectively. The experimental results show resistance to catastrophic forgetting in both of these experimental settings.