Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning
Authors: Tianlong Chen
Sijia Liu
Shiyu Chang
Lisa Amini
Zhangyang Wang
Venue: Accepted by TMLR ()

Abstract: Class-incremental learning (CIL) suffers from the notorious dilemma between learning newly added classes and preserving previously learned class knowledge. That catastrophic forgetting issue could be mitigated by storing historical data for replay, which yet would cause memory overheads as well as imbalanced prediction updates. To address this dilemma, we propose to leverage "free" external unlabeled data querying in continual learning. We first present a CIL with Queried Unlabeled Data (CIL-QUD) scheme, where we only store a handful of past training samples as anchors and use them to query relevant unlabeled examples each time. Along with new and past stored data, the queried unlabeled are effectively utilized, through learning-without-forgetting (LwF) regularizers and class-balance training. Besides preserving model generalization over past and current tasks, we next study the problem of adversarial robustness for CIL-QUD.
Inspired by the recent success of learning robust models with unlabeled data, we explore a new robustness-aware CIL setting, where the learned adversarial robustness has to resist forgetting and be transferred as new tasks come in continually. While existing options easily fail, we show queried unlabeled data can continue to benefit, and seamlessly extend CIL-QUD into its robustified versions, RCIL-QUD. Extensive experiments demonstrate that CIL-QUD achieves substantial accuracy gains on CIFAR-10 and CIFAR-100, compared to previous state-of-the-art CIL approaches. Moreover, RCIL-QUD establishes the first strong milestone for robustness-aware CIL. Codes are available in https://github.com/VITA-Group/CIL-QUD. 

Published in Transactions on Machine Learning Research (06/2022)
Queried Unlabeled Data Improves and Robustifies Class- Incremental Learning
Tianlong Chen
wianlong.chennuutere...
University of Teros at Austin
Sijia Liu
liusiji5 Chuiitt5@ssu.com Smsuu edu
Michigan State University
MIT-IBM Walson AI Lab, IBM Reseorch
Shiyu Chang
chang87@uucsb. edu
University of California, Santa Barbara
Lisa Amini
lisa. aooo.aiiiiicuss.com www.:usw....ibm.mmmmmmmmmm ibm..com
MIT-IBM Watson AI Lab, IBM Research
Zhangyang Wang
Stloswoong@uterrs.eel
University of Tezas at Austin
Reviewed on OpenReview: https: openreview. net/ formm? id= oLulPPheeD
Abstract
Class-incremental learning (CIL.) suffers from the notorious dilemma between learning newly added classes and preserving previously learned class knowledge. That catastrophie forgetting issue could be mitigated by storing historical data for replay, which yet would cause memory overheads as well as imbalanced prediction updates. To address this dilemma, we propose to leverage "free" external unlabeled data querying in continual learning. We first present a CIL with Queried Unlabeled Data (CIL-QUD) scheme, where we only store a handful of past training samples as anchors and use them query relevant unlabeled examples effectively wittingwwwthout--orttttii (LwF) regularizers and class balance training.
each time. Along with new and past stored data, the queried unlabeled utilized, through Besides preserving model generalization over past and current lasks, next study the problem of adversarial robustness for CIL-QUD. Inspired by the recent success of learning robust models with unlabeled data, we explore a new robustnes--wwar CIL sctting, where the learned adversarial robustness has to resist. forgetting and be transferred as new tasks come in continually. While existing options easily fail, we show queried unlabeled data can continue to benefit, and seamlessly extend CIL-QUD into its robustified versions, RCIL- QUD. Extensive experiments demonstrate that CIL-QUD achieves substantial accuracy gains on CIFAR-10 and C-00 compared to previous state-of-the-ard CIL approaches. Moreover, RCIL-QUD establishes the first strong milestone for robustness-awe aware CIL. Codes are available in https:/ / /tt::://ittiee httggggtt.c.o/...... Conn/VITAGGroup/CII
1 Introduction
Most deep neural networks (DNNs) are trained when the complete datasct and all class information are available al once and fixed. However, real-world applications, such as robolics and mobile health, often demand learning classifiers contimually (Parisi et al., 2019), when the data and classes are presented and fitted soquentially. Such continual learning pose severely challenge for standard DNNs, where previous experiences easily get overwritten as more data and new tasks arrive, i.c., the notorious catastrophic forgetting (Goodfellow al., 2013; McCloskey & Cohen, 1989).

---

Published in Transactions on Machine Learning Research (06/2022)
This paper is focused on realistic yet challeng- ing setting of continual learning, called class- incremental learning (CIL) (Rebuffi et al., 2017; Belouadah & Popescu, 2019; 2020; Zhang et al., 2020). In CTT, the classifier model will need to be incrementally re-trained from time to time, when new classes are added. Ideally, the re-training should provide competitive multi-class classifier for all classes observed so far at any time. Un- fortunately, naively augmenting and fine-tuning the model to learn new classes will only see an abrupt degradation of performance on the origi- nal set of classes, when the training objective is adapted to the newly added set of classes. Several attempts have been made by storing past training data (Castro et al., 2018; Javed & Shafait, 2018;
--urs Previoous SOTA
--Ous --Baelline
Tuski SA
Taski RA
40
Average SA
Task2 NA
Average RA
Task2 RA
Tasks SA
TA
Tasks
Task3
Taske
Tuski RA (b) Robust Accurary (RA%) Our RCII-UUB
(4) Standard Accuracy (8A%) of CUL-
Figure 1: Summary of our achieved performance on CIFAR- 10, where CIL conducted over incremental learning tasks (each 2-class). Figure (a) presents the standard accuracy (SA) achieved by CIL-QUD versus the previous SOTA (Belouadah & Popescu, 2019). Figure (b) shows the robust accuracy (RA) achieved by RCIL-QUD versus the baseline of directly applying adversarial training (Madry et al., 2018) to CTT.
Rebuffi et al., 2017; Belouadah & Popescu, 2019;
2020), by generative models (Kemker & Kanan, 2018; Shin et al., 2017; Li et al., 2020), or by regularized fine-tuning (Aljundi et al., 2017; Li & Hoiem, 2017). However, they suffer from either excessive memory overhead or the so-called asymmetric information problem (Zhang et al., 2020), learning this CIL problem far from being resolved. More detailed discussions will be presented in Section 2.
We instead embrace another potential blessing: using publicly available unlabeled auxiliary data in CIL, which can be obtained at negligible costs (e.g., crawled from the web). The power of unlabeled data has recently drawn explosive interests, in multiple contexts such as semi-supervised learning (Chen et al., 2020d), self-training (Xie et al., 2020), and self-supervised learning (Chen et al., 2020e;c). With massive "free" unlabeled data serving pre-training or regularization, those prior works significantly reduced their reliance on labeled data, imes performing on par with fully supervised classification while using one to two magnitudes less labeled samples.
Particularly for CIL, the training data and labels are not only expensive to collect (same as standard learning), but also expensive to store after its own training task finishes. The memory cost. can become predominant, and further in many cases the legacy data cannot remain accessible after training, due to legal or proprietary reasons. That naturally makes external unlabeled data even more promising for CIL: it is a cheap substitute for past training data and needs not to be always stored. Lately, Zhang et al. (2020) presented a new paradigm that proved the concept of improving CIL with external unlabeled data. Each time as new tasks arrive, the authors first trained a separate new model for the new classes using labeled data, and then distilled the two teacher models (the old and new models) into one student model. The second step leveraged unlabeled auxiliary data for distillation, which the authors claimed would help debias the knowledge transferred from both teacher models, compared to using either old or new training data. Their method delivered solid accuracies on CIL classification benchmarks. However, their proposed method was limited by the inefficiency of performing tedious two-step training each time (first the new teacher model, and then the distillation); and the distillation step uses all unlabcled data (~1 million images their default case), which further causes significant. training burdens.
1.1 Our Contributions
We seek to push forward the utility and potential of unlabeled data in CIL, by asking two further questions:
Q1:Provided with massive unlabeled data, can we sample and leverage them with higher efficiency? Q2: From unlabeled data, can we harvest more "borus" " in other CIL performance dimensions besides accuracy?
Our answer Q1 draws the best idcas from two worlds. On the one hand, existing methods that store and replay past training data (Castro et al., 2018; Javed & Shafait, 2018; Rebuffi al., 2017: Belouadah & Popeseu, 2019; 2020) are still the most effective in overcoming catastrophic forgetting, despite its storage headache and
2

---

Published in Transactions on Machine Learning Research (06/2022)
sometimes the data privacy/copyrighe concerns. On the other hand, Zhang et al. (2020) pioncered on CIL with unlabeled data, but sacrificed training/ /daaa efficiency as above explained. To mitigate their respective challenges, we propose to integrate the complementary strengths of storing past data and leveraging unlabeled data; specifically, we only store a handful of historical samples, which would be used as anchor points to query the most relevand unlabeled dala. Aided by the weariing-wiuouutfforgeeae Hoiem, 2017), those queried unlabeled samples join the labeled data from the newly added class to balance between preserving the historical and learning the new knowledge. Our ablation experiments endorse that (1) such queried unlabeled data leads to state-of-the-art (SOTA) CIL performance while sacrificing no efficiency; and (2) using anchor-based query stably outperforms random selection, and stays robust to the unlabeled data distribution shifts and volume variations.
(LwF) regularizer (Li &
Our answer to Q2 is strongly motivated by the prevailing success of utilizing unlabeled data besides standard classification, e.g., improving robustness (Alayrac et al., 2019). We hope to extend and validate those benefits to CIL as well. Spocifically, adversarial robustness (Chen et al., 2020a) arises as a key demand when deploying applications. While continual learning has been so far focused on maintaining
DNNs to beeely//ecuityycrilieee accuracy across the stream of tasks, we consider it the same necessary if not more to examine whether the model can maintain robustness across old and new tasks: a critical step towards enabling trustworthy lcarning in the open world. This problem has been unfortunately largely overlooked and under-explored in the CIL regime. In fact, even one-shot transferability (Hendrycks et al., 2019; Shafahi et al., 2020; Goldblum et al., 2020; Chan et al., 2019) of robustness (from one source domain pre-trained model, to one target domain new task) has not been studied until recently, and was shown to be challenging. We fill in this research gap, by (1) for the first time, studying the calastrophic forgetting of robustness! and showing that it cannot be trivially fixed; and (2) extending our new framework with unlabeled data query to sustaining strong robustness in the CIL schemie, with several robustified regularizations. That provides both extra benchmark dimension for CIL, and significant advance in the study of transferable robustness beyond one-shot.
Our specific contributions are summarized below:
A novel framework of Flass-nncremental Learning with Queried Unlabeled Data (CIL-QUD), that seamlessly integrates handful of stored historical samples and unlabeled data through anchor- based query. CIT-QUD also carefully leveraged LwF regularizers (Ti & Hoiem, 2017) and balanced training (Zhang et al., 2019b) as building blocks. It has light overheads in the model, data storage, and training.
A first-of-its-innd study of preserving adversarial robustness in CIL, and an extension of CIL-QUD to its robustified version called RCIL-QUD. Specifically, we propose and compare two robust versions of LwF regularizers together with an add-on robust regularizer, which are built on the queried unlabeled data.
Experiments demonstrating that the power of unlabeled data can effectively extend to CIL, contribut- ing substantially to superior accuracy as well adversarial robustness both preserved without catastrophic forgetting. In Figure 1, CIL-QUD outperforms previous SOTA by large margins of 10.28% accuracy on CIFAR-10; on C1FAR-100, it also outperforms (Zhang et al., 2020) by a 1.19% margin. RCIL-QUD further establishes the first strong milestone for robustness-aware CIL, that significantly surpasses existing baselines.
2 Related Work
Class-incremest Learning Among numerous methods developed, one category of approaches (Wang et al., 2017; Rosenfeld & Tsotsos, 2018; Rusu et al., 2016; Aljundi et al., 2017; Rebuffi et al., 2018; Mallya et al., 2018) incrementally grow the model capacity to accommodate new classes, yet suffering from the explosive model size as well as inflexibility (e.g., requiring task ID at inference). Another category of solutions is based on transfer learning (Kemker & Kanan, 2018; Belouadah & Popescu, 2018), whose effectiveness yet hinges on the pre-training quality.
It similar the catastrophic forgetting of standard generalization but with different evaluation metrics. other words, robustifying models on a new task leads to robustness degradations on previous tasks, as shown in Figure (right).
3

---

Published in Transactions on Machine Learning Research (06/2022)
A popular and successful family of CIL methods (Li & Hoiem, 2017; Castro et al., 2018; Javed & Shafait, 2018; Rebulfi et al., 2017; Belouadah & Popescu, 2019; 2020) (partially) memorized past training data to fight catastrophic forgetting, bypassing the need for dynamic model capacity. Many of those algorithms viewed CIL as an imbalanced learning problem, where previous and newly added classes made an class size extreme disparity (He & Garcia, 2009; Buda et al., 2018). Learning without Forgetting (IwP) (Li & Hoiem, 2017) made an attempt to fix that dilemma through an LwF regularization via knowledge distillation. Although the original Luf' did not deposit past data, later works (Rebufli et al., 2017) augmented it with a memory bank of previous tasks and previously stored data. More follow-ups (Castro et al., 2018; He et al., 2018; Javed & Shafait, 2018; Rebuffi et al., 2017; Belouadah & Popescu, 2019; 2020) advanced the LwF regularization further. For example, IL2M (Belouadah & Popescu, 2019) utilized a second memory bank to store past class statistics obtained al past training, and incorporated those stored statistics to compensate for the previous (minority) classes' predictions.
Since storing past data inevitably incurred memory overheads, He et al. (2018) used GANs to generate exemplars for previous lasks, for supplying a re-balanced training set at each incremental state. The most. relevant work to ours is Zhang et al. (2020), which pioneered the usage of free external unlabeled data to boost CIL, as we have discussed in Section 1. In comparison, our proposed CIL-QUD clearly distinguishes itself from Zhang et al. (2020), by (1) using adaptively queried unlabeled samples based on stored anchors, instead of full unlabeled data; (2) avoiding two-stage training and instead adopting LwF-reguuariized fine- tuning; (3) being the first in CIL to combine training with class- balanced and randomly sampled data (Zhang et al., 2019b). Besides, neither Zhang et al. (2020) nor any other mentioned work ever touched the robustness preservation in CIT. More details on those differences are expanded in Section 3.
On the other hand, recent investigations (Wang et al., 2021; He & Zhu, 2021) also explore other alternative possibilities of leveraging unlabelled data to boost contimual learning. For example, Wang et al. (2021) replays unlabeled data sampled from a conditional generator, and utilizes a consistency regularization to learn an improved continual classifier. He & Zhu (2021) studics continual lcarning in a fully unsupervised mode by assigning unlabeled data with clustered pseudo labels. Meanwhile, Bateni et al. (2022) and Chen et al. (2020b) enable few-shot and efficient continual learning respectively, with the assistance of unlabeled data. A recent survey paper (Qu et al., 2021) also provides a good summary of current achievements in this field.
Adversarial Robustness and Its Transferablity DNNs commonly suffer from adversarial vulnerabil- ity (Goodfellow et al., 2014), and numerous defense methods have been invented (Madry et al., 2018; Sinha et al., 2018; Rony et al., 2019; Zhang et al., 2019a; Ding et al., 2020). However, most of them focus on attacking/ /dffeinin/deedinggg DNNs trained on single fixed dataset and task. Recently, a couple of works emerge to cogitate the transferability of adversarial robustness (Hendrycks et al., 2019; Shafahi et al., 2020; Goldblum et al., 2020; Chan et al., 2019; Chen et al., 2020a), from robust model traitedined on a source domain to another target domain. They revealed that directly fine-tuning on the target domain data will quickly overwrite the pre-trained robustness (Chen et al., 2020a). One neods to refer to cither adversarial finc-tuning on the larget domain (Chen et al., 2020a), or specific regularizations such as knowledge distillation or gradient matching to preserve the source domain robustness knowledge (Shafahi al., 2020; Goldblum et al., 2020; Chan et al., 2019). In comparison, we target at inheriting robustness across many sets of new data and tasks added contimually, where the conventional transfer learning could be viewed as its oversimplified case. To our best. knowledge, our proposed RCIL-QUD marks the first-ever effort to explore this new daunting setting.
3 Class-Incremental Learning with Queried Unlabeled Data (CIL-QUD)
In this section, we begin by presenting a brief background on class-incremental learning (CIL), and show our CLL-QUD that overcomes catastrophic forgetting by adaptively querying and leveraging unlabeled data.
3.1 CIL Preliminaries and Setups
As in Figure 2, CIL models are continuously trained over a sequential data stream, where a new classification task (consisting of unseen classes) could be added every time. This makes CIL highly challenging in contrast.
4

---

Published in Transactions on Machine Learning Research (06/2022)
to static learning and conventional (one-shot) transfer learning. Following Castro et al. (2018); He et al. (2018); Rebuffi et al. (2017), we consider a practical and challenging CIL setting with only small memory bank& S to store data from previous classes. Neither task ID nor order is pre-assumed to be known.
Let Ti, T2. ... represent a sequence of CIL tasks, and the ith task Ti contains data that fall in (k; - J. with
Class
Task
Class k2 + ~ Kg
Task 3
1 ~ k1
k-1) classes C, = [ck_-++1,000 +
Class-incremental Learner
ko = 1 by convention. At the deh incremental learning session, only have access to the training data associated with Ti and a limited number of stored data in S from the previous learning tasks. Let f(0,0 o x) denotes the mapping from input samples x E - T to the corresponding classes C, acquired from T1. .Te Here (0, ) denotes the parameters of a CTT. model updated
Class
Task 2
K1 + 11+1~ k2
CIL learns continuously from a sequential data stream where new tasks are added. Each new task i may contain several classes from class - h. The CTT. learner aims for multi-class classification for both previous and newly added classes.
Figure 2:
till task Tj. To be more specific, the CIL model consists of feature extractor x(0,x)) Rd with parameters
0, which maps the input x E Uأ_1 Ta to d-dimensional feature space. It is then followed by a multi-class (i.e., Ui-1 Cj) classifier with parameters o which maps the d-dimensional feature to the prediction vector (0, o! x) at the current task T,.
Problem statement. We now formally define our target CIL problem: Given a previously trained model f(O, 0.--)),x) x) under tasks Ti, Ti- the objective is to obtain an updated model f(0, .x) to preserve the genoralization ability and robustness among all learned tasks, even if one can only have the access to data from the memory bank S as well as the current. task Tj. Here the feature extractor (0, x) is shared over all seen tasks, but the scale of the multi-class classifier linearly increases along with the incremental classes.
x)
3.2 Anchor-based Query of Unlabeled Data
Catastrophic forgetting poses the major challenge to CIL. A major reason of forgetting is the asymmetric information between previous classes and newly added classes at each incremental learning stage. Existing CIL approaches (He & Garcia, 2009; Buda et al., 2018; Rebuffi et al., 2017; Belouadah & Popescu, 2019; 2020; Chu et al., 2016; Kemker & Kanan, 2018; Parisi et al., 2019) undertake the forgetting dilemma by training on new task data plus stored previous data. However, the inevitable storage limitation could still cause severe prediction biases as more classes come in. Chu et al. (2016); Kemker & Kanan (2018); Parisi et al. (2019) introduced balanced fine-tuning, yet. incurring the risk of over-fitting new new/minoria /minority classes. Zhang el al. (2020) tried to fix the dilemmaa by referring external unlabeled data. They did not store any past training data; but as above discussed suffered from considcrable training overhead.
We present our novel remedy of utilizing anchor-based unlabeled data query, to balance bet ween preserving the historical and learning the new knowledge. First, we store a small number of i.id randomly picked samples as "anchors" for every past training class, c.g., as few ten samples per class CIFAR-100. Then, during the next incremental processes, we query more auxiliary unlabeled samples from public sources with stored anchors, using certain similarity matches (see Section 3.4 for details). The queried samples are expected to present "similar" " and more relevant information to past training data, compared to random samples. Practically, unlabeled samples can be queried from public sources containing diverse enough natural images, e-g-; Google Images, that are not necessarily tied with previous classes. Next, we inject the previous information into learning new classes, by tuning with the learning- without-forgetting (LwF) regularizers (Li & Hoiem, 2017) on queried unlabeled data.
We next detail on the concrete regularizers L used. Let U donates the queried unlabeled data, LL can be chosen from either knowledge distillation (ACD²) (Hinton et al., 2015; Li & Hoiem, 2017) or feature transferring (FT*) (Shafahi et al., 2020), i.e., LT.W E [TD.FFT.
2XCD here denotes the regularization function of knowledge distillation, modified cross-entropy as in Li Holem (2017); FT denotes the regularization function of feature transferring, an Ep distance metric as Shafahi al. (2020).
5

---

Published in Transactions on Machine Learning Research (06/2022)
ICD is one of the most classical regularizers in CIL (Li & Hoiem, 2017; Castro al., 2018; He et al., 2018;
Javed & Shafait, 2018; Rebuffi et al., 2017: Belouadah & Popescu, 2019; 2020), but was usually applied on fully-stored previous data and/or newly added data. In our case, we enforce the output probabilities p(0, 0,,x) of cach queried unlabeled image x E U to be close to the recorded p(0) 6 x). The ICD regularization is then given by:
(1, == Exeu [KD (p(0, 6., x), p(0 0., x))
(1)
where 0 and Og present the parameters of current feature extractor and classifiers respectively, and 0 and O. stand for the previous ones.
FT (Shafahi et al., 2020) inherits previous knowledge by maximizing the similarity between current feature representations c x) and previons features -(θ,x) x) on unlabeled data x E U:
(L) == Exeu [T(((( x), p(,,x))
(2)
where (p-) is the feature extractor, 0 is defined the same as above, and FT is distance metric, which we choose to be & norm here. An interesling finding from our later experiments (Table and 2) is that, KCD outperforms FT in preserving generalization ability under standard CIL, but FT becomes more useful in the later robustness-awaro CIL.
3.3 Balanced Training with Auxiliary Classifiers
Another well-known romedy for the imbalanced classes is to ro-sample mini-batches (Chawla et al., 2002; Haixiang et al., 2017; Tahir el al., 2009; Wang et al., 2019; Zhang et al., 2019b). It artificially re-balances between majority classes and minority classes to alleviate the negative effect of skewed training data distributions. In this paper, spurred by the recent progress in long-tail visual recognition (Zhang et al., 2019b), we for the first time combine both random and class-balanced sampling strategies into CIL. Our designed network architecture /(0,, oc, x) for CII-QUD contains a shared feature extractor (00,x), x), a primary classifier trained with balanced data, and an auziliary classifier trained with randomly sampled data (Zhang et al., 2019b); see Fig. 3 and Sec. 3.4 for detailed illustration. In this way, the feature extractor learns from both class-balanced and randomly sampled data, where the former prevents the feature extractor from prediction preference toward majority classes, and the latter improves generalization in minority classos.
We notice that both classifiers satisfying results, but there also exists performance trade-off (Zhang et al., 2019b) between them. The primary classifier, trained with dlassballanee data, remembers more previous knowledge and performs better on the previous/ne previous/minority minority classos; the auxiliary classifier, trained with randomly sampled data, performs better on the new new/majority classes, at the cost of forgetting previous knowledge to some extent. That trade-off motivates us to design a classifier ensemble mechanism (CEM) for extra performance boosts. More details are referred to the supplement Section S1.1 and Algorithm 1.
3.4 Overall Framework of CIL-QUD
Our proposed framework, CIL-QUU, effectively integrates the above two remedies, as presented in Figure 3. First, we create Class-Balanced Batch (BeB) and Random Sample Batch (BRs) through balaneel and random sampling from stored previous data and incoming new data. BeB and ERS are both used in feature extractor training, while they are separately employed to the primary and the auxiliary classifier. Second, we query the auxiliary unlabeled data by finding K-ncarest-neighbors of previously stored labeled data over their feature embeddings. Details on the use of unlabeled data and feature embeddings can be found in section S2. We then randomly sample from queried unlabeled data to form an Unlabeled Data Batch (BUD). In this way, we inject more "similar" knowledge of previous classes into CIL models via BUD- Third, feeding the BUD together with BсB and ERS, the primary classifier produces a cross-entropy loss LCB for classification and a regularization term Lt.r for preventing forgetting. Similarly, the auxiliary classifier yields Lns and LL Note that LL#F is calculated on the unlabeled data batch, i.e, BUD. In brief, CIL-QUD is cast as the following regularized optimization problem:
min ((xx)) [LcB(] 1, x),,, 0.0,1.0.,2
BB.x,))BBs [CRs(J xc.,,)
(3)
+X (L, 0c.1) + LI 0o.2))
6

---

Published in Transactions on Machine Learning Research (06/2022)
LCB
0 0c,1
New Task Data
Stored Data
Llass-Balanced
L
Batch
Primary
Random Sample
Classifier
Batch
LRS
K Nearest
Neighbors
i c,2
LLWF
Unlabeled Data
(0))
Feature Extractor
Auxiliary
Unlabel Data Batch
Classifier
Previous Tasks
New Task
Figure 3: Overall framework of CUI-QUD. From left to right, before each training iteration, first create three batches: GalancBl Batch BeB contains balanced data between stored previous classes and newly added classes; Random Sample Batch BRS includes data randomly sampled from all stored data and new task data; Unlabeled Data Batch BUD consists of auxiliary unlabeled data queried by stored previous data using the Senearest-neighhee algorithm over the feature embeddings. Then, three batches are fed into feature extractor (p()), ResNet-18 (He et al., 2016). Red arrows (+), Blue arrous (+) and Green arrous (+) represent the correspouding feed forward paths. The primary classifier takes features from Ben and Bun calculate the objective Len and Lt..r. The auxiliary classifier
produces the objective LRS and LLWF with features from BaS and BUD- Please zoom-in for details.
where 1,,1, 0,2 donate the parameters from the primary and auxiliary classifiers respectively, a hyperpa- rameter, controlling the contributions of LwF regularizers on queried unlabeled data. In our case, À 0.5. For tuning of hyper parameters, we perform a grid search. Lter selected from (KD, depicted Equation 1 and 2. The auxiliary classifier serves as an implicit regularizer for preventing networks from over-fitting minority classes of previous data.
4 Robustified Class-Incremental Learning with Queried Unlabeled Data (RCIL-QUD)
In this section, we motivate the new setup of robustness-aware CIL, where a model has to maintain adversarial robustness across old and new tasks. Assisted by our unlabeled data query scheme, propose two robustified LwF regularizers together with an add-on robust regularizer that can be plugged in CIT-QUD, leading to the RCIL-QUD.
The vulnerability of DNNs raises critical demands for improving their robustness (Goodfellow et al., 2014). However, no formal assessment of adversarial robustness has been performed the CIL setting. is natural to suspect that catastrophic forgetting would make learned robustness hard sustain over new tasks too. Indeed, recent studies have identified such challenges, even in the much simpler case of transferring robustness from source to another target domain (Hendrycks et al., 2019; Shafahi et al., 2020; Goldblum et al., 2020; Chan et al., 2019). Morcover, compared to standard generalization, DNNs need significantly more data to achieve adversarially robust generalization (Schmidt et al., 2018; Madry et al., 2018; Shafahi et al., 2020; Kurakin et al., 2016; Zhai et al., 2020), which also challenges CTL. where previous classes may only have a handful of stored samples that is precisely why our queried eaternal data can become the necessary oid and the blessing.
To handle this new daunting setting, we first investigate the catastrophic forgetting of robustness. As shown in Figure (b) and later in Section 5.3, neither the conventional LwF techniques nor the direct application of adversarial training (AT) (Madry et al., 2018) (the most successful defense method) can prevent. the model's learned robustness from decaying over time. Here the direct application of AT refers to train over the worst-case losses penalized by standard LwF regularizers.
In proposed RCIL-QUD, in addition to incorporating worst-case (min-max) training losses AT, we propose to robusify our LwF regularizations with queried unlabeled data, denoted as LWF: for sustaining adversarial robustness in the CIL scenario. RCII--QUD formulation is depicted as:
CR
7

---

Published in Transactions on Machine Learning Research (06/2022)
Current Feature Extractor
FT F x+ + (),,
Previous Feature Extractor
(0
(0
X Unlabeled Data Figure 4: Illustrations of RFT regularization. From left to right, we first feed the perturbed unlabeled data x 8 through current feature extractor -)) and get feature vector yp00, + 8). Then we pass the clean unlabeled data x to previous feature extractor .) and obtain another feature vector 400, x). RFF enforces the similarity between two features, where FT is E norm.
x+8 8
Perturbed Unlabeled Data
min Exx.))BBn 0 00.
max (0, +8).0) [1
+ +Eix,))BBs
max CRs(1 (,.2,, 16
(4)
+71 L (0, 0c) + 72 (0)),,,
where 7I, 72 balance the effect between AT and robust regularizers. In our case, 71 0.05, 72 0.2. 8 is the adversarial perturbation generated by Projective Gradient Descent (PGD) (Madry et al., 2018). is the upper bound of perturbations under foo norm. In practice, RCIL-UD can involve C implementations from (RKDD, RFFT logether w. or w.o. RTC, in Equations 5, 6 and 7 below, to pass on historic robustness.
Robustified knowledge distillation regularizers (RKD).. As standard LwF regularizers are found to be unsuitable for preserving robustncss, we thus proposc a robustified knowledge distillation term, RAD,, via introducing min-max optimization into KDD:
max KD (p(0, 0,, 6), 6(, o., xi) [16] Le
Excu
where e, 8, P: 0, Gc, θ and O. are defined similarly as equation and equation Our results demonstrate that RKD is more effective for preserving adversarial robustness previous classes.
Robustifted feature transferring regularizers (RFT). We then study the robust version of feature transferring regularizer (FT) (Shafahi et al., 2020). As mentioned recently in Shafahi et al. (2020) robust transfer learning, the feature activations make a main source of robustness. Yet in CIL, the effectiveness of FT decays considerably more tasks arrive. The limitation of FT motivates us to build its novel robustified version, R.FT. It enforces the similarity between the current feature representation p(f, x + 5) of perturbed unlabeled data, and the previous representation x) of the clean unlabeled data, as shown in Figure 4. RFT is mathematically described as:
Exeu
max (T( 18 St
(6)
where FT has been defined in equation 2.
Robustified TRADES regularizers on CIL (RTC). To further unleash the power of unlabeled data in promoting robustness, we incorporate a SOTA robust defense called TRADES (Zhang et al., 2019a), as an add-on regularizer over RKD and RFT. That is owing to the fact that TRADES does not rely on data labels, unlike the adversarial training loss (Madry et al., 2018). The formulation is:
Excu
max KC O., x 6), 0..)) [0
where p is defined as above, and KCC is Kullback Leibler divergence. RTC boosts robustness via minimizing the "difference" between the prodication probabilities p(0, 0., of the current model on clean samples and p(0, 0g, x 5) of the current. model on adversarial samples. It is worth mentioning that RTC equation 7 is different from RKD equation 5, the former enforces the stability of predictions of the same (current) model (0,0) 0.) before and after input perturbations, while the latter promotes the prediction stability of the current model at input perturbations with respect to reference model, namely, the previously learned (0. 0.)-
8

---

Published in Transactions on Machine Learning Research (06/2022)
5 Experiments and Analyses
5.1 Implementation Details
Datasct, Memory Bank, and External Source. We evaluate proposed method on CIFAR-10 and CIFAR-100 datasets (Krizhevsky et al., 2009). We randomly split the original training dataset into training and validation set with a ratio of 9 1. We use all default data augmentations provided by Li & Hoiem (2017), and image pixels are normalized to [o. On CIFAR-10, we divide the 10 classes into splits of 2 classes with a random order (10/2- = 5 lasks); On CIFAR- 100, we divide 100 classes into splits of 20 classes with a random order (100/20
tasks). Namely, at each incremental time, the classifier dimension will increase by 2 for CIFAR-10 and 20 for CIFAR-100.
We set the memory bank to store 100 images per class for CIFAR-10 and 10 images for CIFAR-100 by default. The default external source of queried unlabeled data is 80 Million Tiny Image dataset (Torralba et al., 2008), while another source of ImageNet 32 x 32 (Deng et al., 2009) is investigated later. During each incremental session, we query 5, 000 and 500 unlabeled images per class for CIFAR-10 and CIFAR-100, respectively by default. Increasing the amounts of queried unlabeled data may improve performance further but incur more training costs. We use a buffer of fixed capacity to store 128 queried unlabclod images at each training iteration. More training and model selection details are referred the supplement.
Evaluation Metrics. We evaluate on ResNet18 (He et al., 2016) in terms of: (1) Standard Accuracy (SA): classification testing accuracy on the clean test dataset; and (2) Robust Accuracy (RA): classification testing accuracy on adversarial samples perturbed from the original test dataset. Adversarial samples are crafted by n-step Projected Gradient Descent (PGD) with perturbation magnitude e = 8/255 and step size a- = 2/255 for both adversarial training and evaluation, with set n = 10 for training and n = 20 for evaluation, following Madry et al. (2018).
Existing Baselines for Comparison. We include several classical and competitive CIL approaches for comparison: LwF-MC is the multi-class implementation of Li & Hoiem (2017), and LWF-MCMB is its variation with a memory bank of previous data. For a fair comparison, it always keeps the same memory bank size with ours. For iCaRL (Rebuffi et al., 2017) and IL2M (Belouadah & Popescu, 2019), we use their publicly available implementations with the same memory bank sizes as ours. IL2M requires a second memory bank for storing previous class statistics. DMC (Zhang et al., 2020) is the existing CIL work that also utilizes external unlabeled dała. We ensure DMC and CIL-QUD to query from the same external source.
Variants for Our Proposed Methods. We first consider the vanilla version of CIL-QUD with neither queried unlabeled data nor the auxiliary classifier as Baseline among our proposals. It is still equipped with LwF regularizers yet only applied on stored labeled data² from previous learned tasks. We next check more self-ablations:
a. CIL-QUD Variants: We study four variants of CIL-QUD explained as follows: i) Baseline Auxiliary classifiers: it equals to CIL-QUD without involving the usage of unlabeled data; ii) CIL.-QUD w. FT; iif) CIL-QUD KD; iv) CIL-QUD w. ICD + CEM is the variant approach equipped with classifiers ensemble mechanism (CEM).
6. RCIL-QUD Variants: We investigate ten variants of RCIL-QUD with different robust LwF regularizations on queried unlabeled data, as listed in Table 2. Classifier ensemble mechanism (CEM) is further applied to the top-performing settings. For all variants of CII--UUD and RCIL-UUD without CEM, we evaluate the performance of the primary classifier by default, while the auxiliary classifier results are included in the supplement S3.2.
Privacy Issues of the Unlabeled Data Collection. We believe that potential privacy concerns can be easily circunvented like by only querying from authorized and public datasets. To further inject privacy protection into our method, possible solutions include querying images with filtering of sensitive and offensive samples, or using generative replay: those are certainly feasible and can be our future works.
Ethical Issues of 80 Million Tiny Image Dataset. We clarify that most of the experiments are completed in Spring 2020 before the withdrawal (June 2020) of 80 Million Tiny Image datasct.
For fair comparison, all settings trained on the same amount of data in each training iteration. means that the baseline approach can store extra historical data by taking over the "bddget" for queried unlabeled data.
9

---

Published in Transactions on Machine Learning Research (06/2022)
Moreover, as demonstrated in Table 4, our approach achieve similar performance when it queries unlabeled data from 80 Million Tiny Image or TmageNet datasets. This provides an alternative and alleviates the potential ethical issues of adopting our methods for future researchers.
Takeaways: Based on the results from Tables 1 and 2, we observe: (1) in terms of the standard accuracy, CIL-QUD w. ICD + CEM establishes the SOTA performance; (2) in terms of the robust accuracy, CIL-QUD RFT + RTC CEM reaches superior performance.
5.2 Improved Generalization with CIL-QUD
The standard accuracies (SAs) of all CIL models are collected in this section. As shown in Table 1, several observa- tions could be drawn: (1) By incrementally adding our proposed components to the vanilla baseline, we find all of them contribute to preserving SA. Among the auxiliary classi- fier architecture (+2.37% SA), unlabeled data regularization LLWF (+4.52% SA from FT; +5.98% SA from IDD), and CEM (+2.65%), LLWF is the dominant contributor, which demonstrates that queried ex- ternal unlabeled data gently balances the asymmetric in- formation between previous classes and newly added classes
Table 1: Final performance for each task T of CUL-UUD, compared with SOTAs. Note that MTTupper and MTolowr train the model with multi-Lask learning scheme using full data and few stored data, respectively. They act as the empirical performance upper bounds and lower bounds for CII-QQUD.
CIFAR-10 (SAA)
Methods
Ti (%)
Ta (%)
Ts (%)
Ta (%6
Ts (%)
Average (%)
LwF-MC (Li Hoiem, 2017)
28.30
58.10
50.20
46,00
60.25
48.57
LWF-MCMB (Li & Huiem, 2017)
76.45
81.45
69.55
32.30
44.30
60.81
iCaRL (Rebuffi et 2017)
76.45
79.00
75.70
50.85
35.55
63. 63.51
IL2M (Belouadah & Popescu, 2019)
78.20
61.05
60.40
38.95
92.10
66.77
DMC (Zhang et al., 2020)
73.66
MT' (empirical lower hound)
62.75
59.30
51.10
37.40
491.95
$2,10
MTypprr (empiricca uppper bound)
95.75
95.75
91.60
89.10
92.30
92.90
Vauilla Baseline
78.05
61.05
59.75
36,75
92. 10
66.20
Baseline + Auxiliary Classifier
75.05
71.50
54.25
52.05
89.10
68.39
CIL-QUD w. FT
$1.10
80.40
63.50
75.40
62.15
72.91
CIL-QUD ACD
$2.05
77.55
72.05
66.15
74.05
74.37
CUI-QUD A.D CEM
84.40
79.50
69.44
66.85
84.95
77.02
CIFAR- R-00R---0000 (SA.)
Methods
Ti (%)
15 (%)
T5 (%)
Ta (%6
75 (%6
Average (%)
IL2M (Belouadah & Popescu, 2019)
22.05
18.70
33.45
32.35
62.90
37,89
DMC (Zhang et al., 2020)
40.32
39.75
42.10
46.12
59.36
43.53
MT Mmnee (empirical lower bound)
14.40
10.50
12.30
11.15
16.75
13.02
MT (empirical upper bound)
70.65
67.55
75.05
68.80
74.55
71.32
in CIL. (2) Comparing CIL- QUD w. IDD with CIT-QUD w. FT, ICD seems to yield ob-
CIL-QUD CUL-QUD KD CEM
30.85
39.30
52.65 65
41.80
67.
16.36
51.95
46.15
52.80
38.60
44.10
46.72
tains a larger SA boost than FT. (3) All results in Table show CIL-QUD significantly oulperforms previous both CIFAR-10 (+10.28% SA) and CIFAR-100 (at least +1.19% SA). It demonstrates that, aided by query unlabcled data, CIL-QUD heavily reduces the prediction bias towards hit the previous or the new classes. Note that our proposal also surpasses DMC (Zhang et al., 2020) by 1.19% SA on CIFAR-100, with the same amount of unlabeled data: that supplies evidence that anchor- query is more effective than blindly sampling. In addition, we report the average performance variation when incrementally training CIL models in Figure We observe that CIL-QUD w. KD stably does better in preserving previous knowledge besides superior final performance.
SOTA methods by substantial margins
T1
T1-T2
T1-T3
T1-T4
T1-T5
40
50
60
70
80
90
CIL-QUD w. KD
IL2M
LWF.MCMB
LWF.MC
Figure 5: Average performance variation when training CTL models incrementally over tasks on the CTFAR-10 dataset with selected approaches. T,-T, means the model have been trained with tasks 1, continually.
i (t
(1,2,3, 4,5))
10

---

Published in Transactions on Machine Learning Research (06/2022)
5.3 Improved Robustness with RCIL-QUD
Under the new robustness- aware CIL scheme, the robust accuracies (RAs) of all mod- els are collected in this section. As shown in Table 2 and S8, number of observations can be drawn consistently on CIFAR- 10 and CIFAR-100 datasets.
Table 2: Robust accuracy for each task Te of RCTI-QUD and its variants. MTAT and MTAT adversarially train the model with multi-task learning scheme using full data and few stored data, respectively. They act the empir- ical performance upper bounds and lower bounds for RCIL-QUD. Here we focus on reporting RAs, while their corresponding SAs are Table S8.
CIFAR 10 (RA)
Methods
Ti (%)
T2 (%)
Ta (%)
Ta (%)
T5 (%6
Average (%)
MTAT hnwer MTAT upper
22.18
15.99
18:42
4.29
7.66
13.71
54.00
58.70
39.60
28.30
21.35
40.39
First, FT regularizer sur- passes K.D by 12.13% SA and 4.3% RA on CIFAR- 10, which supports Shafahi et al. (2020); Boopathy et al. (2020)'s claim that regularizing feature presentations are cru- cial for robustness. Compared our empirical performance lower bound by MTAT lower (explained in Table 2 caption), ID even hurts RA. A pos- sible explanation is that ill- conditioned probabilities may obstacle preserving robustness.
RCIL-QUD w. XD
4.10
14.30
6.35
5.10
24.20
10.81
RCII-QUD RAD
21.00
37.90
29.25
3.90
16.30
21.67
RCIL-QUD w. RAD CEM
34.70
45. 10
25.50
11 00
36.95
30.77
RCIL-QUD w. RAP + RTC
37.00
30.00
30.60
16. 60
5.10
23.86
RCIL-QUD w. RED + RTC + CEM
48.15
50.85
39.40
19.60
42.75
40.15
RCIL-QUD w. FT
21.70
26.65
11.90
8.85
15.11
RCIL-QUD w. R.FT
34.05
6,05
RCIL-QUI RFT CEM
26.75
35.
21.50
6.85
23. 80
22.80
RCIL-QUD w. RFT RTC
32 22.455
29. 55
28.10
23
10. 13
24.67
RCIL-QUE w. RFT RTC + CEM
41.45
37.20
31.95
16..35
24.30
30.25
CIFAR- 100 (RA)
Methods
T1 (%)
Ta (%5
(%)
T4 (%)
(96)
Average (%)
MTAT lower
3.65
2.25
1.85
1.75
5.30
2.96
MTTT opper
23.40
16.35
20.40
19.20
29.05
21.68
RCIL-QUD w. RCD
9.50
5.05
10.10
13.00
21.50
12.43
RCIL-QUD w. RKD +CEM
15.10
8.60
13:20
15.55
21.45
14.78
RCIL-QUI QUD w. RFT
3.45
26.90
RCIL QUD RJT CEM
12.05
7.05
10.95
10.15
19.70
11.98
Second, robustified regulariz- ers on unlabeled data, RKD and R.FT, significantly increase RA compare with their standard versions, especially for RKDD (+11.039) SA and +10.86% RAA on CIFAR-10). It suggests that injecting advessarial signals into regularizations effectively improves CIL models' resilience. Third, RTC further pushes RAA higher, with slight TA degradation. As the red RA nunber shows in Table 2, it only has negligible gap (0.24%) to the empirical RA upper bound from MTAT which we consider as an almost insurmountable milestone for achievable robustness in CIL. Lastly, the classifier ensemble mochanism (CEM) continucs to benefit RAs on both CIFAR-10 and CIFAR-100.
5.4 Ablation Study
Do we indeed need query, and would random unlabeled data also work? CIL-QUD has al- ready outperformed DMC (Zhang et al., 2020) in Section 4.2, which endorses the query way for better SAs. We further conduct experiments to justify our query also benefits RAs in RCIL-QUD w. RFT + RTC. We first observe that leveraging queried unlabeled data leads to better generalization and robustness than just storing a few historical data from past lasks (i.e., Baseline in Figure 6). Then, systematically compare the effectiveness of the following methods on utilizing unlabeled data: (i) Feature KNN: finding K nearest-neighhors for each methods. Both RAs and corresponding SAs reported. stored anchor over feature embeddings; which is our default way; (ii) Largest Logit: finding unlabeled data with top K largest soft logits for each class; (iii) Random Pick: randomly sampling unlabeled data.
Feature KNN
Metric
Logit
SA
80
Random Pick RA
Accuracy (%)
Baseline
80
40
20
10
The number of leamned classes
Figure 6: Average performance of RCIL-QUD RFT + RTC on CIFAR-10 with different unlabeled data query
As shown in Figure 6, all three options utilizing unlabeled data steadily outperform the vanilla baseline, especially in terms of RA. Our default query mechanism with feature KNN performs the best.
11

---

Published in Transactions on Machine Learning Research (06/2022)
Table 3: Investigation of diverse sizes of the memory
bank S.
Table 4: Query unlabeled data from different sources.
CIL-QUD (Stambard Accuracy] CIFAR-10
Size per class
(5)
(%)
T: (%6)
(56)
(%)
Average (%)
Sources
CIL-QUD (Standardd Accuracy) on CIPAR-10
10
65.15
63.20
57.55
56.70
67.70
62.06
Ti (%)
(%)
Ta (%)
T. (%6
75 (%)
Average (%)
20
74.90
72.05
61.30
71.20
01.40
68.25
80 Million
82.05
77.55
72.05
66.15
74.05
71.37
30
82.30
71.75
06.35
63.65
71.10
71.03
ImogoNot
78.25
72:45
68.20
73.80
70.15
T2.57
100
82.05
77.55
72.05
66.15
74.05
74.37
RCIL-QUD (Rolusst Accuneyy) CIFAR-10
RCIL QUD
(Roboust Acturary)) on CIFAR 10
Sourees
Sioe per calss
T: (9)
| 75 (%6
T5 (56
7h (%)
75 (%)
Average (%)
(96)
75 176)
(76)
(1)
Average (%)
80 Million
32.45
29.55
28.10
23.12
10.13
24.67
10
24.10
27.64
10.98
11.40
0.31
17.30
ImageNet
34.35
27.90
26.03
18.75
9.37
23.28
20
28,95
27.90
20.30
20.05
9.05
21.89
26.20
21.05
100
32.45
29.55
28.10
23.12
10:13
24.67
The influence of "anchors" for query. We adopt two representative activate learning methods to choose the "anchors": (1) maximum entropy sampling (Lewis & Gale, 1994; Settles, 2009) termed as "Entropy"; and (2) core-set selection via proxy (Coleman et al., 2020) named as "SVP". Experiments of CIL-QUD w. KDD on CIFAR-10 have been carried out, with the settings from Table 1. We obtain: Random v.s. Entropy v.s. SVP 74.37% v.s. 73.91% v.s. 74.76%. Results imply that different choices of "anchors" have limited effects the achievable performance of our methods.
Changing the size of memory bank. Indicated by Rebufi et al. (2017); Belouadah & Popescu (2019), the size of memory back S plays key role in the CIL performance. We conduct an ablation of CIL-QUD w. IDD and RCIL-QUD w. RFFT + RTC with 10, 20, 50, 100 samples stored per class, as shown in Table 3. Note that we keep the number of queried unlabeled data the same default, across all settings for a fair comparison. We observe that CIL-QUD/ROE BUUDD/RCL--QQ RRCIL-QUD equipped with a large memory banks consistency achieve the superior performance, in term of both gencralization and robustness. That is understandable that more anchors ensure more relevant queried data. Meanwhile, we are also encouraged see CIL-QUD and RCIL-QUD already have competitive performance when each class has small as 20-50 stored samples, demonstrating nice sample and memory efficiency of our methods.
Switching to different sources of unlabeled data Since the 80 Million Tiny Image set is often considered to be from the same distribution with CIFAR-10/-100, we switch the unlabeled data source to ImageNet (32 x 32 downsampled versions), whose images are semantically more complex and noisy. Results are presented in Table 4. We observe that the achieved performance of CIL-QUD/RCIL-QQ stay robust under a different unlabeled data distribution.
6 Conclusions
We introduce unlabeled data queries and show that it effectively overcome the catastrophic forgetting in the standard CIL scheme, with the help of LwF regularizations. Through anchor-based query, unlabeled data brings in more relevant information to previous classes while alleviating the information asymmetry. The power of queried unlabeled data can further extend to preserving robustness in CIT,, opening up a new dimension of robustified CIL and establishing strong milestone results. Our results exemplify a significant reduction of the performance gap between incremental and non-ineremental learning, in terms of both SA and RAA. Similarly to other deep learning frontiers, unlabeled data evidently reveals the new promise for CIL too, if leveraged appropriately. We hope that our findings can broadly inspire people to dig deeper into utilizing unlabeled data for lifelong learning, knowledge transfer, and sustained robustness.
Acknowledgement
W.W. in part supported by the U.S. Army Research Laboratory Cooperative Research Agreement W911NF17- 2-0196 (TOBT REIGN), and IBM faculty research award.
12

---

Published in Transactions on Machine Learning Research (06/2022)
References
Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, pp. 12214-12223, 2019.
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gale: Lifelong learning with a network of experts. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 3366-3375, 2017.
Peyman Bateni, Jarred Barber, Raghav Goyal, Vaden Masrani, Jan- Willem van de Meent, Leonid Sigal, and Frank Wood. Beyond simple meta-learning: Multi-purpose models for multi-domain, active and continual few-shot learning. arXi preprint or orXiv:2201.05154 2022.
Eden Belouadah and Adrian Popescu. Deesil: Deep-shallow incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 0-0, 2018.
Eden Belouadah and Adrian Popescu. T12m: Class incremental learning with dual memory. In The IEEE International Conference on Computer Vision (ICCV), October 2019.
Eden Belouadah and Adrian Popescu. Scail: Classifier weights scaling for class incremental learning, 2020.
Akhilan Boopathy, Sijia Liu, Gaoyuan Zhang, Cynthia Liu, Pin- Yu Chen, Shiyu Chang, and Luca Daniel. Proper network interpretability helps adversarial robustness in classification. In International Conference on Machine Learning, pp. 1014 1023. PMLR, 2020.
Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249-25 22999929,,999 2018.
Francisco M Castro, Manuel J Marin-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to- end incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV), pp- 233 248, 2018.
Alvin Chan, Yi Tay, and Yew-Soon Ong. What it thinks important is important: Robustness transfers through input gradients, 2019.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321 357, 2002.
Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From supe-sup pre-training to fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 699-708, 2020a.
Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Long live the lottery: The existence of winning tickets in lifelong learning. In International Conference on Learning Representations, 2020b.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learningg, pp. 1597- 1607. PMLR, 2020c.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self self-supervised modols are strong semi-supervised learners. Advances in Neural Information Processing Systems, 33, 2020d.
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved basclines with momentum contrastive learning. Xiv preprint arXiv:2003.04 04277, 2020e.
Brian Chu, Vashisht Madhavan, Oscar Beijbom, Judy Hoffman, and Trevor Darrell. Best practices for fine-tuning visual classifiers to new domains. In European conference on computer vision, pp. 435-442. Springer, 2016.
13

---

Published in Transactions on Machine Learning Research (06/2022)
Cody Coleman, Christopher Yeh, Stephen Mussmann, Baharan Mirzasoleiman, Peter Bailis, Percy Liang, Jure Leskovec, and Matei Zaharia. Selection via proxy: Elfficient data selection for deep learning. In International Conference on Learning Representations, 2020. URL https://openreviee fttp:///ppppeeeeeeeeeeeeooorrrra met//forumm/ddd-tlg220v00
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, Imagenet: A large-scale hierarchical image databasc. In 2009 IEEE conference computer vision and pottern recognition, pp. 248-255. Iece,
2009.
Gavin Weiguang Ding, Yash Sharma, Kry Yik Chau Lui, and Ruitong Huang. MMA training: Direct input space margin maximization through adversarial training. In International Conference on Learning Representations, 2020. URL https:/ http:a://pperreie..eetteee nett/foruuriiccHeeerxx.8
Micah Goldblum, Liam Fowl, Soheil Feizi, and Tom Goldstein. Adversarially robust distillation, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 3996-4003, 2020.
Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical invostigation of calastrophic forgelling in gradient-based neural networks. arXiv preprinl arXim:1312. 6211, 2013.
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. Xiv preprint or 14x.:1112.6511 2014.
Guo Haixiang, Li Yijing, Jennifer Shang, Gu Mingyun, Huang Yuanyue, and Gong Bing. Learning from class-imbalanced data: Review of methods and applications. Expert Systems with Applications, 73:220-2333 2017.
Chen He, Ruiping Wang, Shiguang Shan, and Xilin Chen. Exemplar-supported generative reproduction for class incremental learning. In Brilish Machine Vision Conference, 2018.
Haibo and Edwardo A Garcia. Learning from imbalanced data. IEEE Transactions on knowledge and dala engineering, (19:))23 1284, 2009.
Jiangpeng He and Fengqing Zhu. Unsupervised continual learning via pseudo labels. arXiv preprinl arXiv:210.00716, 2021.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Dan Hendrycks, Kimin Lee, and Mantas Mazcika. Using pre-training can improve model robustness and uncertainty. Proceedings of the 36th International Conference on Machine Learning, 09 15 Jun 2019.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXie preprint arXiv:1503.0253 2015.
Khurram Javed and Faisal Shafait. Revisiting distillation and incremental classifier learning. In Asion Conference on Computer Vision, pp. 3-17. Springer, 2018
Ronald Kemker and Christopher Kanan. Fearnet: Brain- irsin-inspired model for incremental learning. In International Conference on Learning Representations, 2018. URL https://openrevil wtwpeeerwweee..eee/orooeeee merfforum?id= SJ.Xmf-Rb.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial machine learning at scale, arXie preprint arXiu:1611.01236 2016.
David D Lewis and William A Gale. A sequential algorithm for training text classifiers. In SIGIR '94, pp. 3-12. Springer, 1994.
Huaiyu Li, Weiming Dong, and Bao-Gang Hu. Incremental concept learning via online generative memory recall. IEFE Transactions on Neural Networks and Learning Systems, 2020.
14

---

Published in Transactions on Machine Learning Research (06/2022)
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE tronsactions on pottern analysis and machine intelligence, 40(12):2935-2941) 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. URL
httpp/ppwrrrreeee.eeeeerrmrraaaa net/foorumiderrD2lffbbb
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting single network to multiple tasks by learning to mask weights. In Proceedings of the European Conference Computer Vision (ECCV), pp- 67 82, 2018.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and molivation, volume 24, pp. 109-165 165. Elsevier, 1989.
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: review. Neural Networks, 2019.
Haoxuan Qu, Hossein Rahmani, Li Xu, Bryan Williams, and Jun Liu. Recent advances of continual learning in computer vision: An overview. arxiv preprint arXiv:2 11399, 2021.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEFE conference on Computer Vision and Pottern Recognition, pp. 2001 2010, 2017.
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Efficient paramctrization of multi-domain deep neural networks. In Proceedings of the IEEE Conference on Compuler Vision and Pallern Recognition, pp- 8119 8127, 2018.
Jérôme Rony, Luiz G Hafemann, Luiz S Oliveira, Ismail Ben Ayed, Robert Sabourin, and Eric Granger. Decoupling direction and norm for efficient gradient-based 12 adversarial attacks and defenses. In IEEE Conference on Compuler Vision and Pollern Recognition, pp. 1322-4331) 2019.
Amir Rosenfeld and John K Tsotsos. Incremental learning through deep adaptation. IEEE trunsactions on pattern analysis and machine intelligence, 2018.
Andrei A Rusu, Neil C Rabinowitz, Guillaune Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: NNi::1606.0471,, 2016.
Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry. Adversarially robust gencralization requires more data. In Advances in Neural Information Processing Systems, 2018.
Burr Settles. Active learning literature survey. 2009.
Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Goldstein. Adversarially robust. transfer learning. In International Conference on Leurning Represenlations, 2020.
URL https://openrevvie: http://wwwwe.oooorirrrrereoooooo
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990 2999, 2017.
Aman Sinha, Hongscok Namkoong, and John Duchi. Certifiable distributional robustness with principled adversarial training. In International Conference on Learning Representations, 2018. URL https:// openreview.nee wttpw/wwwwwwwwwttnneeeeee wwtp:forumm.dd-HH.tr.......
Muhammad Atif Tahir, Josef Kittler, Krystian Mikolajezyk, and Fei Yan. multiple expert approach the class imbalance problem using inverse random under sampling. In International workshop on multiple classifier systems, pp. 82 91. Springer, 2009.
15

---

Published in Transactions on Machine Learning Research (06/2022)
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on paltern analysis and machine intelligence, 30(11):1958 1970, 2008.
Liyuan Wang, Kuo Yang, Chongxuan Li, Lanqing Hong, Zhenguo Li, and Jun Zhu. Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning. In Proceedings of the IBEE/CVF Conference on Computer Vision and Pallern Recogriiioo,, pp. 5383 5392, 2021.
Yiru Wang, Weihao Gan, Jie Yang, Wei Wu, and Junjie Yan. Dynamic curriculum learning for imbalancod data classification. In Proceedings of the IEEE Internalional Conference Compuler Vision, pp. 5017-5020 2019.
Yu-Xiong Wang, Deva Ramanan, and Martial Hebert. Growing a brain: Fine-tuning by increasing model capacity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 2471 2480, 2017.
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10687-10698, 2020.
Runtian Zhai, Tianle Cai, Di He, Chen Dan, Kun He, John E. Hopcroft, and Liwei Wang, Adversarially robust generalization just requires more unlabeled data, 2020. URL https:// httpp////peerrrieeeeeffforrrr net/forum? id=H1gdAC4KDB.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-74 7482. PMLR, 2019a.
Junjie Zhang, Lingqiao Liu, Peng Wang, and Chunhua Shen. To balance not to balance: An embarrassingly simple approach for learning with long-tailed distributions, 2019b.
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C-C Jay Kuo. Class-incremental learning via deep model consolidation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131-1140, 1140, 2020.
16


Reviews:
[{"review_id": "S9vHyLLcdB", "reviewer": "Reviewer_vEWF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- The proposed technique is well motivated and clearly distinguished from prior works.\n\n- The authors show the performance of various variants of CIL-QUD and RCIL-QUD, which shows the importance of each technical component.\n\n- This paper discovers the catastrophic forgetting of adversarial robustness in continual learning. Further, the authors modify CIL-QUD into a min-max optimization objective along with the TRADES loss (called RCIL-QUD) to mitigate the above issue.\n\n**Weaknesses**\n- The phenomenon called catastrophic forgetting of adversarial robustness seems to be unclear to me, although the authors mention it in the Introduction section.\n\n- All the experiments are based on only one model, i.e., ResNet18.\n\n- The 'anchors' for each training class are randomly picked. I am concerned whether the choice of 'anchors' will influence the performance.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a continual learning framework boosted by adaptively queried unlabeled training data (namely CIL-QUD) to mitigate catastrophic forgetting and thus enhance natural generalization. Further, the authors propose a robust version of CIL-QUD (namely RCIL-QUD) to prevent catastrophic forgetting of adversarial robustness and thus improve robust accuracy. The experiments validate the effectiveness of the proposed framework.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "- Please give some description and experimental evidence for the phenomenon called catastrophic forgetting of adversarial robustness,\n\n- It would be better to show the results based on different models to validate the generalization capability of the proposed method on various models.\n\n- It would be helpful to report the mean and standard variance of all the results, to remove the concern incurred by randomness.\n\n- As the authors conduct lots of variants of CIL-QUD, I recommend giving a summary of the technique that can obtain state-of-the-art performance.\n", "broader_impact_concerns": "This paper leverages unlabeled data to boost continual learning. As the authors say in the paper, the unlabeled training data could be crawled from the Internet. Therefore, this would cause an invasion of personal privacy due to the unauthorized collection of the data.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "409kSEafgy", "reviewer": "Reviewer_VL3x", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nBorrowing ideas from class-imbalanced learning to tackle class-incremental learning is interesting and may be helpful for future CIL work, especially when we are allowed to have a small set of previous data from old classes. The paper also utilize unlabeled data by querying them by taking the nearest neighbors of the training data of previous tasks (CIL-QUD). In the latter sections, the paper studies adversarial robustness in CIL and proposes a robustified version of their CIL-QUD.\n\nWeaknesses:\n\nMotivation of the paper -- Zhang et al. 2019c's motivation for utilizing unlabeled data came from how storing past training data is unrealistic in the CIL problem. In the paper under review, we use both past training data and a large set of unlabeled data. Since this makes the problem setting easier (we have access to both), the paper further needs to motivate if this is a realistic scenario.\n\nEthical issues -- The experiments rely on 80 million tiny image dataset, but in 2020, the authors have withdrawn the dataset due to derogatory terms as categories and offensive images. See https://groups.csail.mit.edu/vision/TinyImages/ for the details. Although I am not an ethics expert, it seems problematic to continue using a withdrawn dataset with ethical issues.\n\nExperiments -- The experiments are conducted with only 5 tasks, while most class-incremental learning papers perform experiments with much more.  For example, Zhang et al. 2019c performs incrementally learning 5, 10, 20, and 50 classes at a time, with iCIFAR-100. It would be more informative to see if the proposed method can maintain accuracy with respect to a higher number of tasks.\n\nStory of the paper -- Although the paper has several contributions for the class incremental learning problem, the contributions seem to be independent from each other and it was hard to grasp the main/core contribution when I first read the paper. For example, the discussions and experiments that aim to investigate robustness is independent from the former sections, and the contribution of balanced learning is independent from using unlabeled data. In my opinion, the class-balanced/random sample component in the proposed algorithm is the most interesting contribution of the paper, but the paper does not emphasize this at all (e.g., the paper does not mention it in the abstract and the conclusion).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an algorithm and studies some aspects of class-incremental learning. The paper has a few contributions. The first contribution is to utilize ideas from class-imbalanced learning, since if we allow to keep some training data from previous tasks (classes), this will lead to an imbalanced class learning setup with a lot of samples for the current task/classes and a few samples for the old task/classes. The next contribution is to use unlabeled data for class-incremental learning. Although Zhang et al. 2019c also used unlabeled data for class-incremental learning, this paper chooses similar ones to the stored training data (from old tasks) instead of using all of the unlabeled data, which seems to be effective in experiments and the burden of training will be lower. Finally, the paper studies the adversarial robustness of class-incremental learning and proposes an robustified version of the proposed method.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "I wrote my main comments/suggestions to improve the paper in the previous section.\n\nThe important ones in my opinion are \"ethical issues\", \"experiments\", and \"motivation of the paper\". The \"story of the paper\" is less important.\n\nSome other minor comments are:\n- Figure 1's caption: summery is summary?\n- Page 2, Section 1.1: reply is replay?", "broader_impact_concerns": "The paper does not have a broader impact statement. However, I do not see any immediate concerns for future negative impact to our society. I wrote my comments for potential ethical isuses in the prevous section (Strengths and Weaknesses).", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YMIYCDh5Hj", "reviewer": "Reviewer_3wop", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The proposed class-incremental learning method is clear and straightforward. The adversarial robustness of the proposed method is also explored in this paper.\n2. The experimental results of the proposed method are quite good on CIFAR-10/-100 when compared with previous methods. The authors also conducted interesting ablation studies. For example, different query mechanisms, size of the memory bank, and sources of unlabeled data are examined experimentally.\n3. The paper is well-written and structured. The reading experience of this paper is enjoyable. \n\nWeaknesses:\n1. Using the stored data to query similar unlabeled data may cause large costs in computational resources and time, which makes it hard to be applied in real-time settings in practice.\n2. The related work presented in the paper is quite old. All references are in/before the year 2020. \n\nSome additional comments: \n1. I'm wondering why the result of the DMC method (Zhang et al., 2019c) on CIFAR-10 is not presented in the paper, only presenting it on CIFAR-100? It is an important baseline to be compared with the proposed method.\n2. The paper mentioned that the primary classifier and auxiliary classifier remember knowledge in different aspects so a classifiers ensemble mechanism (CEM) is applied for extra performance boosts. But from the experimental results, it seems to add CEM performs well than not adding it on CIFAR-10, but not on CIFAR-100. Compared with not adding CEM, adding it even makes a large performance drop (67.2 to 44.1) on one setting of CIFAR-100. So does it mean the performance of adding CEM is quite random (not sure whether it would help)?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposed a novel framework in class-incremental learning where additional unlabeled data are used by querying a small set of pre-stored past training data without the need of storing a large amount of past data. Specifically, the small set of past data, new coming data, and unlabeled data are effectively used in the proposed framework based on learning-without-forgetting regularizers and class-balance training. The adversarial robustness of the proposed method is also considered and explored through experiments on CIFAR-10/-100.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "1. I suggest the authors check and update the references since many arxiv papers presented in the references are already published, such as Zhang et al. (2019c) and Chen et al. (2020b). \n2. It would be better if the paper could include some discussions of recent related work in Section 2, i.e., related work in the recent two years. For example, Wang et al. ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning. In CVPR 2021.\n3. Fix a typo in the caption of Figure 1: Summery should be Summary.", "broader_impact_concerns": "I don't have any concerns about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This is an interesting paper working on an important problem "class-incremental learning". It proposed several components to improve (wrt natural accuracy) and robustify (wrt adversarial accuracy) class-incremental learning methods. Although TMLR doesn't emphasize novelty and significance, the two factors are both fairly high to the reviewers and me. The authors did a good job in the rebuttal to address the concerns from the reviewers, and then all reviewers voted for acceptance. Furthermore, given that most of the promised changes (>95%) have already been included, I feel the paper could be accepted as is.
Decision: Accept (poster)