Generative vs. Discriminative: Rethinking The Meta-Continual Learning

1 Introduction . Deep learning methods have gained great success in a wide variety of tasks in various fields [ 26 , 28 ] . However , in many real-world scenarios , where the data samples are neither i.i.d . nor available all at once , most of them lose performance on the older data points ; a phenomenon known as catastrophic forgetting [ 13 ] . Continual learning , also called lifelong learning , aims to tackle this problem [ 48 ] . Animals as intelligent learning agents can easily learn a sequence of different tasks with minimum interference and forgetting [ 55 , 12 ] . When facing new problems , the brain does not rewire all of the wirings [ 37 , 5 ] but utilizes lifelong accumulated inductive biases to efficiently learn new concepts [ 9 , 4 ] . In the field of cognitive and neuroscience , there exist two special cases of concept learning named prototypical and exemplar-based learning [ 44 , 2 , 59 , 7 ] . Bayesian inference is another aspect of learning in the brain which aims to consider uncertainty [ 24 , 31 , 16 , 4 , 23 ] . In this paper , we integrate the hypothesis of generative concept learning and the Bayesian approach by considering a distribution over each concept . Figure 1 compares different characteristics and expressive power of the mentioned concept learning approaches . Inspired by the biological mechanism of learning , we suggest that the generative approach in category learning is an important principle for continual learning problems . Although the Bayesian approach requires proper priors about each concept , ∗Equal contribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . meta-learning can be used to provide data-driven prior knowledge or statistics from the previously encountered concepts similar to the biological intelligent systems [ 4 ] . Meta-learning is an endeavor to gather experience from solving related problems in order to solve new unseen ones more efficiently ; which have been used to improve the learning process in different areas [ 19 ] . In this work , we employ meta-learning to derive proper inductive biases as well as a powerful feature extractor for a continual learning problem from a collection of similar continual learning problems . By taking advantage of these inductive biases , we formulate the category learning problem from a probabilistic generative approach . Our main contributions are as follows : • We take motivation from cognitive and neuroscience about meta-continual learning in intelligent animals and especially make use of bio-inspired concept learning to eradicate catastrophic forgetting . • We investigate the problem of discriminative approaches in continual learning and propose generative classifiers as a general way of mitigating catastrophic forgetting . • Encouraged by generative classifiers , we propose a new discriminative classifier called PGLR , to reduce the inherent catastrophic forgetting problem of discriminative approaches . • Existing prototypical learning is improved by adding extra statistics through the Bayesian approach . • Different components of the proposed method can be implemented in a fully incremental setting which is useful in the shortage of data or computational resources . • We achieve state-of-the-art accuracy with a notable gap compared to the existing methods . 2 Related Works . Continual Learning : Three main approaches have been proposed to address the forgetting issue [ 33 ] . ( i ) Replay-based methods rely on reusing samples from previous training tasks either by keeping them in an explicit episodic memory [ 30 , 36 , 6 , 1 ] or generating pseudo-samples via a deep generative model [ 43 , 53 , 50 ] . ( ii ) Methods based on parameter isolation dynamically expand the model as the data for new tasks arrive by introducing task-specific weights [ 40 , 57 , 42 ] or masks [ 52 , 58 ] . ( iii ) Regularization-based methods consider a probability distribution , typically the Gaussian family , over parameters and penalize the substantial deviations from the prior distribution learned through the earlier tasks . Taking a Bayesian approach will lead to an incremental posterior update which is naturally consistent with continual learning [ 32 ] , while considering a point estimation for the parameters will result in a regularization term in the objective function [ 45 , 60 , 22 ] . Besides theoretical foundations , simplifying assumptions and approximations often make the regularizationbased methods fail when the number of tasks is large and they are more prone to catastrophic forgetting compared to the first two approaches [ 49 ] . Compared to the first approach , merely storing low dimensional sufficient statistics of categories , makes our model extremely memory efficient and applicable even when there are thousands of tasks or categories . However , our method is more related to the second approach in the sense that we isolate class-specific parameters . Like the Bayesian methods [ 32 ] , we also consider a distribution over parameters . Nevertheless in our method , using generative classifiers and restricting the distribution to the class-specific parameters enables exact closed-form solutions with zero forgetting . Most of the existing continual learning methods use a discriminative approach but there exist some prototypical or exemplar-based models which are related to our method in the sense that they utilize a generative approach for final classification [ 36 ] . However , they are not completely immune to forgetting and are also less memory efficient since they store a considerable number of high-dimensional input examples . Meta-Learning : Existing methods in the literature of meta-learning can be categorized into optimization-based [ 10 , 35 ] , black-box [ 41 , 14 ] , and metric-based [ 46 , 47 , 56 ] approaches . In this paper we use metric-based methods that meta-learn an embedding space in which a metric-based classifier performs well . Some methods called Bayesian meta-learning , approach the problem from a probabilistic point of view , obtaining a distribution over the problem-specific parameters , therefore capable of providing information about the uncertainty of a learned model [ 15 , 11 , 54 ] . We also provide a probabilistic framework capable of using various statistics of data and learnable prior distributions . Meta-Continual Learning : Both continual and meta-learning have mutual impacts on each other , introducing two new research branches [ 8 ] : Continual meta-learning [ 18 , 21 ] and meta-Continual learning [ 17 , 38 , 20 , 3 ] . In the second branch , various techniques from meta-learning are used to tackle catastrophic forgetting . Here , we especially focus on the settings in which explicit episodic meta-training on continual problems is used [ 20 , 3 ] . OML [ 20 ] meta-learns a representation network to avoid interference between the continual tasks and reduce the catastrophic forgetting . ANML [ 3 ] also takes the same approach but instead of merely having a representation network , it generates a data-dependent mask that modulates the activity of neurons in the prediction network . Motivated by them , we utilize episodic meta-learning to tackle the continual learning problem , but instead of having a discriminative approach , we propose a probabilistic generative method that is completely immune to forgetting . Our approach falls into metric-based meta-learning , as opposed to the prior methods that are optimization-based [ 20 , 3 ] . 3 Preliminaries . Meta-Learning : In the meta-learning setting , we have access to a set of meta-training problems Dmeta-train which help to learn a prior knowledge and thus generalize to unseen problems with data fromDmeta-test . Each problem is a pair of support and query sets , i.e . ( S , Q ) , and labels fromDmeta-train do not intersect with the labels in Dmeta-test . The mentioned prior knowledge can be encoded in the meta-parameters θ which is utilized by a problem-specific model parameterized by φ to solve a new problem instance . The overall learning procedure of the meta-parameters can be formulated as [ 11 ] : θ ? = argmax θ logP ( θ | Dmeta-train ) . ( 1 ) And the distribution of the problem-specific parameters , given a support set S , can be formulated as : P ( φ | S , Dmeta-train ) = ∫ θ P ( φ | S , θ ) P ( θ | Dmeta-train ) dθ ≈ P ( φ | S , θ ? ) P ( θ ? | Dmeta-train ) ∝ P ( φ | S , θ ? ) . ( 2 ) While some works model this distribution , others use a point estimate as : φ ? = argmax φ logP ( φ | S , θ ? ) . ( 3 ) Meta-Continual Learning : A meta-learning scenario where each individual problem ( S , Q ) is continual , i.e . the support-set is observed continually , is called meta-continual learning . We consider support stream as S = { S1 , S2 , ... , St , ... } , in which St = { ( xti , yti ) } |St| i=1 and the x t i and y t i are the ith data point and the corresponding label in St respectively . Each St is accessible only at time-step t due to continual learning restriction . The query-set Q is a set of samples sharing the same set of labels as those in S. The objective here is to perform well on Q after sequentially observing all of the samples available in S. We use the notation SX and SY for the data points and their labels in S respectively , and the same holds for QX and QY . Furthermore ScX shows the data points with the label y = c available in S , i.e . ScX = { x| ( x , c ) ∈ S } . In this paper , we consider a model immune to catastrophic forgetting if the accuracy and decision boundaries of the model in continual learning and i.i.d . scenarios remain identical . In the lifelong-learning literature , each St is called a learning task which may contain the data points and corresponding labels for multiple classes . With this in mind , three standard benchmarks are introduced for evaluation : task-incremental , domain-incremental , and class-incremental . The latest benchmark is known as the most difficult one in which each class is only observed in a single task while at the test-time the model is expected to classify across all observed classes from all tasks [ 49 ] . However , our proposed method is completely agnostic to the order in which data samples arrive and produces the same model , even in an extreme setting that we refer to as fully incremental . In this setting , only one sample is available at each time step and each data point is visited exactly once . In fact , the only assumption in this work is that the samples with the same label in S are i.i.d. , namely P ( Sc ) = Πx∈ScP ( xi ) . 4 Method . Although recently meta-learning representations for continual learning have been introduced to alleviate the catastrophic forgetting [ 20 , 3 ] , these methods still greatly suffer from catastrophic forgetting . One probable reason is that they have constructed their methods based on discriminative approaches . Meanwhile , it is reported that the human brain benefits from a generative approach in category learning [ 44 , 2 , 59 , 7 ] and this raises the question of whether humans capabilities in continual learning stems from their generative approach to the problem . While further studies by the neuroscience community are needed to investigate the exact relationship between biological category learning and the forgetting concept , here we provide some theoretical insights which shed light on the topic . In the following subsections , we elaborate on this by first discussing the drawbacks of the existing meta-continual learning methods which originates from their discriminative nature . Then , we show how the generative approach can resolve the catastrophic forgetting problem of meta-continual learning and finally , the proposed generative model for continual learning is presented .

Summary: This paper proposes a probabilistic generative approach using meta-learning for continual learning. It is inspired by cognitive and neuroscience about meta-continual learning to address catastrophic forgetting. It presents the problems occurring in the discriminative approaches and suggests how to handle them by using generative classifiers. Using new update scheme for discriminative classifiers, it shows that the catastrophic forgetting problem is reduced. It also improves the existing prototypical learning by Bayesian approach.