Bypassing Logits Bias in Online Class-Incremental Learning with a Generative Framework

1 INTRODUCTION . Humans excel at continually learning new skills and accumulating knowledge throughout their lifespan . However , when learning a sequential of tasks emerging over time , neural networks notoriously suffer from catastrophic forgetting ( McCloskey & Cohen , 1989 ) on old knowledge . This problem results from non-i.i.d distribution of data streams in such a scenario . To this end , continual learning ( CL ) ( Parisi et al. , 2019 ; Lange et al. , 2019 ) has been proposed to bridge the above gap between intelligent agents and humans . In common CL settings , there are clear boundaries between distinct tasks which are known during training . Within each task , a batch of data are accumulated and the model can be trained offline with the i.i.d data . Recently , online CL ( Aljundi et al. , 2019c ; a ) setting has received growing attention in which the model needs to learn from a non-i.i.d data stream in online settings . At each iteration , new data are fed into the model only once and then discarded . In this manner , task boundary is not informed , and thus online CL is compatible with task-free ( Aljundi et al. , 2019b ; Lee et al. , 2020 ) scenario . In real-world scenarios , the distribution of data stream changes over time gradually instead of switching between tasks suddenly . Moreover , the model is expected to quickly adapt to large amount of new data , e.g . user-generated content . Online CL meets these requirements , so it is more meaningful for practical applications . Many existing CL works deal with task-incremental learning ( TIL ) setting ( Kirkpatrick et al. , 2017 ; Li & Hoiem , 2018 ) , in which task identity is informed during test and the model only needs to classify within a particular task . However , for online CL problem , TIL is not realistic because of the dependence on task boundary as discussed above and reduces the difficulty of online CL . In contrast , class-incremental learning ( CIL ) setting ( Rebuffi et al. , 2017 ) requires the model to learn new classes continually over time and classify samples over all seen classes during test . Thus , online CIL setting is more suitable for online data streams in real-world CL scenarios ( Mai et al. , 2021 ) . Most existing online CIL methods are based on experience replay ( ER ) ( Robins , 1995 ; Riemer et al. , 2019 ) strategy which stores a subset of learned data in a replay memory and uses the data in memory to retrain model thus alleviating forgetting . Recently , in CIL setting logits bias problem in the last fully connected ( FC ) layer , i.e . softmax classifier , is revealed ( Wu et al. , 2019 ) , which is a main cause of catastrophic forgetting . In Figure 1a , we show in online CIL , even if ER is used , logits bias towards newly learned classes in the softamx classifier is still serious and the forgetting on old tasks is dramatic ( See Figure 1b ) . Although some works ( Wu et al. , 2019 ; Belouadah & Popescu , 2019 ; Zhao et al. , 2020 ) propose different methods to reduce logits bias , they all depend on task boundaries and extra offline phases during training so that not applicable for online CIL setting . In this paper , we propose to tackle the online CIL problem without the softmax classifier to avoid logits bias problem . Instead , we propose a new framework where training and inference are both in a generative way . We are motivated by the insight that generative classifier is more effective in low data regime than discriminative classifier which is demonstrated by Ng & Jordan ( 2001 ) . Although the conclusion is drawn on simple linear models ( Ng & Jordan , 2001 ) , similar results are also observed on deep neural networks ( DNNs ) ( Yogatama et al. , 2017 ; Ding et al. , 2020 ) recently . It should be noticed that in online CIL setting the data is seen only once , not fully trained , so it is analogous to the low data regime in which the generative classifier is preferable . In contrast , the commonly used softmax classifier is a discriminative model . Concretely , we abandon the softmax FC layer and introduce nearest-class-mean ( NCM ) classifier ( Mensink et al. , 2013 ) for inference , which can be interpreted as classifying in a generative way . The NCM classifier is built on the feature space on the top of previous network layers . Thanks to ER strategy , NCM classifier can utilize the replay memory for inference . As for training , inspired by a recent work ( Boudiaf et al. , 2020 ) , which shows pair-based deep metric learning ( DML ) losses can be interpreted as optimizing the feature space from a generative perspective , we introduce MultiSimilarity ( MS ) loss ( Wang et al. , 2019 ) to obtain a good feature space for NCM classifier . Meanwhile , we prove theoretically that MS loss is an alternative to a training objective of the generative classifier . In this way , we can bypass logits bias . To strengthen the model ’ s capable of learning from new data in complex data streams , we further introduce an auxiliary proxy-based DML loss ( Movshovitz-Attias et al. , 2017 ) . Therefore , our whole training objective is a hybrid of generative and discriminative losses . During inference , we ignore the discriminative objective and classify with the generative NCM classifier . By tuning weight of the auxiliary loss , our method can work well in different data streams . In summary , our contributions are as follows : 1 . We make the first attempt to avoid logits bias problem in online CIL setting . In our generative framework , a generative classifier is introduced to replace softmax classifier for inference and for training , we introduce MS loss which is proven theoretically to optimize the model in a generative way . 2 . In order to improve the ability of MS loss to learn from new data , we further introduce an auxiliary loss to achieve a good balance between retaining old knowledge and learning new knowledge . 3 . We conduct extensive experiments on four benchmarks in multiple online CIL settings , including a new task-free setting we design for simulating more realistic scenarios . Empirical results demonstrate our method outperforms a variety of state-of-the-art replay-based methods substantially , especially alleviating catastrophic forgetting significantly . 2 RELATED WORK . Current CL methods can be roughly divided into three categories : which are regularization , parameter isolation and replay-based respectively ( Lange et al. , 2019 ) . Regularization methods retain the learned knowledge by imposing penalty constraints on model ’ s parameters ( Kirkpatrick et al. , 2017 ) or outputs ( Li & Hoiem , 2018 ) when learning new data . They work well in TIL setting but poor in CIL setting ( van de Ven & Tolias , 2018 ) . Parameter isolation methods assign a specific subset of model parameters , such as network weights ( Mallya & Lazebnik , 2018 ) and sub-networks ( Fernando et al. , 2017 ) to each task to avoid knowledge interference and thus the network may keep growing . This type of method is mainly designed for TIL as task identity is usually necessary during test . The mainstream of Replay-based methods is ER-like ( Rebuffi et al. , 2017 ) , which stores a subset of old data and retrains it when learning new data to prevent forgetting of old knowledge . In addition , generative replay method trains a generator to replay old data approximately ( Shin et al. , 2017 ) . In the field of online CL , most of methods are on the basis of ER . Chaudhry et al . ( 2019b ) first explored ER in online CL settings with different memory update strategies . Authors suggested ER method should be regarded as an important baseline as in this setting it is more effective than several existing CL methods , such as A-GEM ( Chaudhry et al. , 2019a ) . GSS ( Aljundi et al. , 2019c ) designs a new memory update strategy by encouraging the divergence of gradients of samples in memory . MIR ( Aljundi et al. , 2019a ) is proposed to select the maximally interfered samples from memory for replay . GMED ( Jin et al. , 2020 ) edits the replay samples with gradient information to obtain samples likely to be forgotten , which can benefit the replay in the future . Mai et al . ( 2021 ) focus on online CIL setting and adopt the notion of Shapley Value to improve the replay memory update and sampling . All of the above methods are replay-based with softmax classifier . A contemporary work ( Lange & Tuytelaars , 2020 ) proposes CoPE , which is somewhat similar to our method . CoPE replaces softmax classifier with a prototype-based classifier which is non-parametric and updated using features of data samples . However , the loss function of CoPE is still discriminative and the way to classify is analogous to softmax classifier . Apart from the above ER based methods , Zeno et al . ( 2018 ) propose an online regularization method , however it performs very badly in online CL settings ( Jin et al. , 2020 ) . Lee et al . ( 2020 ) propose a parameter isolation method in which the network is dynamically expanded and a memory for storing data is still required . Therefore , the memory usage is not fixed and potentially unbounded . A recent work ( Yu et al. , 2020 ) proposes SDC , a CIL method based on DML and NCM classifier . However , SDC requires an extra phase to correct semantic drift after training each task . This phase depends on task boundaries and the accumulated data of a task , which is not applicable for online CIL . In contrast , our method is based on ER and classifies with replay memory and thus need not correct the drift . 3 ONLINE CLASS-INCREMENTAL LEARNING WITH A GENERATIVE FRAMEWORK . 3.1 PRELIMINARIES AND MOTIVATIONS . 3.1.1 ONLINE CLASS-INCREMENTAL LEARNING . CIL setting has been widely used in online CL literature , e.g . ( Aljundi et al. , 2019a ; Mai et al. , 2021 ) , and a softmax classifier is commonly used . A neural network f ( · ; θ ) : X → Rd parameterized by θ encodes data samples x ∈ X into a d-dimension feature f ( x ) on which an FC layer g outputs logits for classification : o = Wf ( x ) + b . At each iteration , a minibatch of data Bn from a data stream S arrives and the whole model ( f , g ) is trained on Bn only once . The training objective is cross-entropy ( CE ) loss : LCE = − C̃∑ c=1 t : c log ŷ : c , ŷ : c = eo : c∑C̃ c=1 e o : c ( 1 ) where C̃ is the number of classes seen so far . t is one-hot label of x and the subscript : c denotes the c-th component . The new classes from S emerge over time . The output space of g is the number of seen classes and thus keeps growing . At test time , the model should classify over all C classes seen .

Summary: To address the problem of logits bias in class-incremental learning with deep neural networks, this paper proposes to replace the typically used softmax output layer by a nearest-mean classifier (on the feature space), which can be interpreted as a generative classifier. To train the feature attractor in an online setting (i.e., without reliance on task boundaries), the paper proposes a hybrid loss consisting of a multi-similarity loss (which can be interpreted as an approximate generative objective) and an auxiliary “proxy” loss. The paper reports substantial performance improvements on a number of computer vision benchmark (including ones with natural images) over a selection of existing methods.  Additionally, the paper proposes a new “smooth dataset” benchmark for online class-incremental learning, whereby the transitions between classes are smooth rather than based on abrupt task transitions.