Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference

1 INTRODUCTION . Continual learning ( CL ) is a learning scenario where a model learns from a continuous and online stream of data and is regarded as a more realistic and practical learning setup than offline learning on a fixed dataset ( He et al. , 2020 ) . However , many CL methods still focus on the offline setup ( Kirkpatrick et al. , 2017 ; Rebuffi et al. , 2017 ; Saha et al. , 2021 ) instead of the more realistic online setup . These methods assume access to a large storage , storing the entire data of the current task and iterating on it multiple times . On the other hand , we are interested extensively in the more realistic online setup where only a small memory is allowed as storage . Meanwhile , even for the online CL methods , we argue they have room for more practical and realistic improvements concerning multiple crucial aspects . The aspects include the class distributions such as the disjoint ( Rebuffi et al. , 2017 ) or the blurry ( Aljundi et al. , 2019c ) splits and the evaluation metric that focuses only on the task accuracy such as average task accuracy ( Aavg ) . The two main assumptions on the class distributions in existing CL setups , i.e. , the disjoint and blurry splits , are less realistic for the following reasons . The disjoint split assumes no classes overlap over different tasks ; already observed classes will never appear again . This assumption is not plausible because already observed classes can still appear later on in real-world scenarios ( see Fig . 2 of ( Bang et al. , 2021 ) ) . On the other hand , in the blurry split ( Aljundi et al. , 2019c ) no new classes appear after the first task even though the split assumes overlapping classes over tasks . This is also not plausible as observing new classes is common in real-world scenarios . The typical evaluation metric such as Aavg in which the accuracy is measured only at the task transition is also less realistic . It implicitly assumes that no inference queries occur in the middle of a task . However , in real-world scenarios , inference queries can occur at any-time . Moreover , there is no explicit task transition boundary in most real-world scenarios . Thus , it is desirable for CL models to provide good inference results at any time . To accurately evaluate whether a CL model is effective at such ‘ any-time ’ inference , we need a new metric for CL models . In order to address the issues of the current CL setups , we propose a new CL setup that is more realistic and practical by considering the following criteria : First , the class distribution is comprised of the advantages from both blurry and disjoint . That is , we assume that the model continuously encounters new classes as tasks continue , i.e. , class-incremental and that classes overlap across tasks , i.e. , blurry task boundaries , while not suffering from the restrictions of blurry and disjoint . Second , the model is evaluated throughout training and inference such that it can be evaluated for any-time inference . We call this new continual learning setup ‘ i-Blurry ’ . For the i-Blurry setup , we first propose a plausible baseline that employs experience replay ( ER ) with reservoir sampling and a learning rate scheduling tuned for the online and task-free CL setting . While existing online CL methods are applicable to the i-Blurry setup , they perform only marginally better than our baseline or often worse . To better handle the i-Blurry setup , we propose a novel continual learning method , which improves the baseline in three aspects . We design a new memory management scheme to discard samples using a per-sample importance score that reflects how useful a sample is for training . We then propose to draw training samples only from the memory instead of drawing them from both memory and the online stream as is done in ER . Finally , we propose a new learning rate scheduling to adaptively decide whether to increase or decrease the learning rate based on the loss trajectory , i.e . a data-driven manner . To evaluate the algorithms in the new setup , we evaluate methods by conventional metrics , and further define a new metric called ‘ area under the curve of accuracy ’ ( AAUC ) which measures the model ’ s accuracy throughout training . We summarize our contributions as follows : • Proposing a new CL setup called i-Blurry , which addresses a more realistic setting that is online , task-free , class-incremental , of blurry task boundaries , and subject to any-time inference . • Proposing a novel online and task-free CL method by a new memory management , memory usage , and learning rate scheduling strategy . • Outperforming existing CL models by large margins on multiple datasets and settings . • Proposing a new metric to better measure a CL model ’ s capability for the desirable any-time inference . 2 RELATED WORK . Continual learning setups . There are many CL setups that have been proposed to reflect the realworld scenario of training a learning model from a stream of data ( Prabhu et al. , 2020 ) . We categorize them in the following aspects for brevity . First , we categorize them into ( 1 ) task-incremental ( task-IL ) and ( 2 ) class-incremental learning ( class-IL ) , depending on whether the task-ID is given at test time . Task-IL , also called multi-head setup , assumes that task-ID is given at test time ( Lopez-Paz & Ranzato , 2017 ; Aljundi et al. , 2018 ; Chaudhry et al. , 2019 ) . In contrast , in class-IL , or single-head setup , task-ID is not given at test time and has to be inferred ( Rebuffi et al. , 2017 ; Wu et al. , 2019 ; Aljundi et al. , 2019a ) . Class-IL is more challenging than task-IL , but is also more realistic since task-ID will not likely be given in the real-world scenario ( Prabhu et al. , 2020 ) . Most CL works assume that task ID is provided at training time , allowing CL methods to utilize the task ID to save model parameters at task boundaries ( Kirkpatrick et al. , 2017 ; Chaudhry et al. , 2018b ) for later use . However , this assumption is impractical ( Lee et al. , 2019 ) since real-world data usually do not have clear task boundaries . To address this issue , a task-free setup ( Aljundi et al. , 2019b ) , where task-ID at training is not available , has been proposed . We focus extensively on the task-free setup as it is challenging and being actively investigated recently ( Kim et al. , 2020 ; Lee et al. , 2019 ; Aljundi et al. , 2019c ) . We now categorize CL setups into disjoint and blurry setup by how the data split is configured . In the disjoint task setup , each task consists of a set of classes disjoint from all other tasks . But the disjoint setup is less realistic as the classes in the real-world can appear at any time not only in a disjoint manner . Recently , to make the setup more realistic , a blurry task setup has been proposed and investigated ( Aljundi et al. , 2019c ; Prabhu et al. , 2020 ; Bang et al. , 2021 ) , where 100 −M % of the sampels are from the dominant class of the task and M % of the samples are from all classes , where M is the blurry level ( Aljundi et al. , 2019c ) . However , the blurry setup assumes no class is added in new tasks , i.e. , not class-incremental , which makes the setup still not quite realistic . Finally , depending on how many samples are streamed at a time , we categorize CL setups into online ( Rolnick et al. , 2018 ; Aljundi et al. , 2019a ; Chaudhry et al. , 2019 ) and offline ( Wu et al. , 2019 ; Rebuffi et al. , 2017 ; Chaudhry et al. , 2018b ; Castro et al. , 2018 ) . In the offline setup , all data from the current task can be used an unlimited number of times . This is impractical since it requires additional memory of size equal to the current task ’ s data . For the online setup , there are many notions of ‘ online ’ that differs in each literature . Prabhu et al . ( 2020 ) ; Bang et al . ( 2021 ) refer online to a setup using each streamed sample only once to train a model while Aljundi et al . ( 2019c ; a ) refer online to a setup where only one or a few samples are streamed at a time . We follow the latter as the former allows storing the whole task ’ s data , which is similar to offline and more unrealistic than using the sample more than a few times . Incorporating the pros and cons of existing setups , we propose a novel CL setup that is online , task-free , class-incremental , of blurry task boundaries , and subject to any-time inference as the most realistic setup for continual learning . Continual learning methods . Given neural networks would suffer from catastrophic forgetting ( McCloskey & Neal , 1989 ; Ratcliff , 1990 ) , the online nature of streaming data in continual learning generally aggravates the issue . To alleviate the forgetting , there are various proposals to store the previous task information ; ( 1 ) regularization , ( 2 ) replay , and ( 3 ) parameter isolation . ( 1 ) Regularization methods ( Kirkpatrick et al. , 2017 ; Zenke et al. , 2017 ; Lee et al. , 2017b ; Ebrahimi et al. , 2020 ) store previous task information in the form of model priors and use it for regularizing the neural network currently being trained . ( 2 ) Replay methods store a subset of the samples from the previous tasks in an episodic memory ( Rebuffi et al. , 2017 ; Castro et al. , 2018 ; Chaudhry et al. , 2019 ; Wu et al. , 2019 ) or keep a generative model that is trained to generate previous task samples ( Shin et al. , 2017 ; Wu et al. , 2018 ; Hu et al. , 2019 ; Cong et al. , 2020 ) . The sampled or generated examplars are replayed on future tasks and used for distillation , constrained training , or joint training . ( 3 ) Parameter isolation methods augment the networks ( Rusu et al. , 2016 ; Lee et al. , 2017a ; Aljundi et al. , 2017 ) or decompose the network into subnetworks for each task ( Mallya & Lazebnik , 2018 ; Cheung et al. , 2019 ; Yoon et al. , 2020 ) . Since ( 1 ) , ( 2 ) , and ( 3 ) all utilize different ways of storing information that incurs parameter storage costs , episodic memory requirement and increase in network size respectively , a fair comparison among the methods is not straighforward . We mostly compare our method with episodic memorybased methods ( Wu et al. , 2019 ; Aljundi et al. , 2019a ; Bang et al. , 2021 ) , as they perform the best in various CL setups but also with methods that use both regularization and episodic memory ( Chaudhry et al. , 2018b ; Wu et al. , 2019 ) . Online continual learning . Despite being more realistic ( Losing et al. , 2018 ; He et al. , 2020 ) , online CL setups have not been popular ( Prabhu et al. , 2020 ) due to the difficulty and subtle differences in the setups in the published literature . ER ( Rolnick et al. , 2018 ) is a simple yet strong episodic memory-based online CL method . It employs reservoir sampling for memory management and jointly trains a model with half of the batch sampled from memory . Many online CL methods are based on ER ( Aljundi et al. , 2019c ; a ) . GSS ( Aljundi et al. , 2019c ) selects samples using a score based on cosine similarity of gradients . MIR ( Aljundi et al. , 2019a ) retrieves maximally interfering samples from memory to use for training . Different from ER , A-GEM ( Chaudhry et al. , 2019 ) uses the memory to enforce constraints on the loss trajectory of the stored samples . GDumb ( Prabhu et al. , 2020 ) only updates the memory during training phase and trains from scratch at the test time only using the memory . Unlike these methods , recently proposed RM ( Bang et al. , 2021 ) uses an uncertainty-based memory sampling and two-stage training scheme where the model is trained for one epoch on the streamed samples and trains extensively only using the memory at the end of each task , effectively delaying most of the learning to the end of the tasks . Note that the uncertainty-based memory sampling can not be implemented in the online CL setup and the two-stage training performs particularly worse in our i-Blurry setup . Our method outperforms all other online CL method introduced in this section while strictly adhering to the online and task-free restrictions .

Summary: The paper proposes a new problem setup in continual learning. As the title suggests, the paper focuses on online, task-free, class incremental, task blurry learning with any-time inference. The authors also came up with new baselines and importance-based memory management. They empirically tested their methods in the proposed problem setup.