Self-supervised Continual Learning based on Batch-mode Novelty Detection
Authors: Jingbo Sun
Jiaxin Zhang
Frank Y Liu
Mahantesh M Halappanavar
Deliang Fan
Yu Cao
Venue: Submitted to ICLR 2023 (2023)
Keywords: Continual Learning
Gradients-based
Mahalanobis Distance
Novelty Detection
out-of-distribution
self-supervised

Abstract: Continual learning (CL) plays a key role in dynamic systems in order to adapt to new tasks, while preserving previous knowledge. Most existing CL approaches focus on learning new knowledge in a supervised manner, while leaving the data gathering phase to the novelty detection (ND) algorithm. Such presumption limits the practical usage where new data needs to be quickly learned without being labeled. In this paper, we propose a unified approach of CL and ND, in which each new class of the out-of-distribution (ODD) data is first detected and then added to previous knowledge. Our method has three unique features: (1) a unified framework seamlessly tackling both ND and CL problems; (2) a self-supervised method for model adaptation, without the requirement of new data annotation; (3) batch-mode data feeding that maximizes the separation of new knowledge vs. previous learning, which in turn enables high accuracy in continual learning. By learning one class at each step, the new method achieves robust continual learning and consistently outperforms state-of-the-art CL methods in the single-head evaluation on MNIST, CIFAR-10, CIFAR-100 and TinyImageNet datasets.

Under review as a conference paper at ICLR 2023
SELF-SUPERVISED CONTINUAL LEARNING BASED ON BATCH-MODE NOVELTY DETECTION
Anonymous authors Paper under double-blind review
ABSTRACT
Continual learning (CL) plays key role in dynamic systems in order to adapt new tasks, while preserving previous knowledge. Most existing CL approaches focus on learning new knowledge in a supervised manner, while leaving the data gathering phase to the novelty detection (ND) algorithm. Such presumption limits the practical usage where new data needs to be quickly learned without being labeled. In this paper, we propose unified approach of CL and ND, in which each new class of the out-of-distribution (ODD) data is first detected and then added to previous knowledge. Our method has three unique features: (1) unified framework seamlessly tackling both ND and CL problems; (2) a self-supervised method for model adaptation, without the requirement of new data annotation;
(3) batch-mode data feeding that maximizes the separation of new knowledge vs. previous learning, which in turn enables high accuracy in continual learning. By learning one class at each step, the new method achieves robust continual learning and consistently outperforms state-of-the-at CL methods in the single- head evaluation on MNIST, CIFAR-10, 10, CIFAR-100 and TinylmageNet datasets.
INTRODUCTION
Machine learning methods have been widely deployed in dynamic applications, such as drones, self-driving vehicles, surveillance, etc. Their success is built upon carefully handcrafted deep neural networks (DNNs), big data collection and expensive model training. However, due to the unforesee- able circumstances of the environment, these systems will inevitably encounter the input samples that fall out of the distribution (OOD) of their original training data, leading to their instability and performance degradation. Such scenario has inspired two research branches: (1) novelty detection (ND) or one-class classification and (2) continual learning (CI.) or life-long learning. The former one aims to make the system be able to detect the arrival of OOD data. The latter one studies how to continually learn the new data distribution while preventing catastrophic forgetting Goodfellow et al. 2013) from happening prior knowledge.
While there exists a strong connection between these two branches, current practices to solve them are heading toward quite different directions. ND methods usually output all detected OOD samples as a single class while CL methods package multiple classes into a single task for learning. Such a dramatic difference in problem setup prevents researchers to form unified algorithm from ND to CL, which is necessary for dynamic systems in reality.
One particular challenge of multi-class learning in CL the difficulty in data-driven (i.e., self- supervised or unsupervised) separation of new and old classes, due to their overlap in the feature space. Without labeled data, the model can be struggling to find a global optimum that can suc- cessfully separate the distribution of all classes. Consequently, the model either ends up with the notorious issue of catastrophic forgetting, or fails learn the new data. To overcome this chal- lenge, previous methods either introduce constraints in model adaptation in order to protect prior knowledge when learning a new task Aljuuni et al. (2018): : Li Hoiem| (2018); Kirkpatrick et al.) (017); Rebuffi et al. 2017). expand the network structure increase the model capacity for new knowledge Rusu et al. (2016); Yoon et al. (2017). However, the methods with constraints may not succeed when the knowled wedee distribution of new task is far from prior knowledge distribution. On the other hand, the methods with a dynamic network may introduce too much overhead when the amount of new knowledge keeps increasing.
In this context, our aims of this work are: (1) Connecting ND and CL into one method for dynamic applications, (2) Completing ND) and CL without the annotation of OOD data, and (3) Improving the

---

Under review as a conference paper at ICLR 2023
Input Stream (top to bottom)
Unsupervised Batch Preparation
Gradient-based OOD Detection
Continual L Learning
Single-head Classification
IDD,
IDD
OoDs
Batch
Initial Classifter
DOD,
Batch
Binary Classifter IDD, vs .00000
OOD,
COD;
Batch
Memory
Binary Classifier (IDD, DO vs. 00D;
Training
Classifier for IDD,
Continual learning for 00D, Continual learning O0D;
Figure 1: The framework of our unified method for both novelty detection and one-class continual learning. The entire process is seff-supervised, without the need of labels for new data.
robustness and accuracy of CL in the single-head evaluation. We propose a self-supervised approach for one-class novelty detection and continual learning, with the following contributions:
A unified framework that continually detects the arrival of the OODs, extracts and learns the OOD features, and merges the OOD into the knowledge hase of previous IDDs. More specially, we train a tiny binary classifier for each new OOD class as the feature extractor. The binary classifier and the pre-trained IDD model is sequentially connected to form "N 1 classifier, where "N" represents prior knowledge which contains N classes and "1" refers to the newly arrival OOD. This CL process continues as "N + +++++1. 1..." (i.e, one-class CL), as demonstrated in this work.
A batch-mode training and inference method that fully utilizes the context of the input and maximizes the feature separation between OOD and previous IDDs, without using data labels. This method helps achieve the high accuracy in OOD detection and prediction in scenario where IDDs and OODs are streaming into the system, such as videos and audios.
Comprehensive evaluation multiple benchmarks, such MNIST, CIFAR-10 10, CIFAR- 100, and TinyImageNet. Our proposed method consistently achieves robust and a high single-head accuracy after learning sequence of new OOD classes one by one.
BACKGROUND
Most continual learning methods belong to the supervised type, where the input tasks are well- labeled. To mitigate catastrophic forgetting, three directions have been studied in the community: (1) Regularization methods Zeng et al. (2019); Aljundi et al. (2018); Li Hoiem (2018); Kirkpatrick al. (2017); Rebulfi et al. (2017); Zenke al. (2017) Ahn et al. (2019), which aim penalize the weight shifting towards the new task. This is realized by introducing a new loss constraint to protect the most important weights for previous tasks. Many metries to measure the importance of weights are proposed, such as the Fisher information matrix, distillation loss and training trajectory. (2) Rehearsal-based methods Lopez-Paz Pazz & Ranzato (2017): Chaudhry al. (2188; Rolnick al. 2019); Aljundi et al. (2019); Cha et al. (2021). which maintain a small buffer to store the samples from previous tasks. To prevent the drift of prior knowledge, these samples are replayed during methods are combined with the regularization methods to improve their performance. (3) Expansion- methods Rusu et al. (2016); Yoon et al.) (2017), 7): Schwarz et al. 20118; Li et al. (201)); Hung et al. (2019). which aim to protect previous knowledge by progressively adding new network branches for the new task. These methods are especially useful when the knowlede powledge base of previous tasks and that of the new task are overlapping. Although these three supervised methods improve the performance of continual learning, argue that the capability to automatically detect the task shifting and learn the new knowledge (i.e., sulf-super or unsupervised) is the most preferred solution by a realistic system, since expensive annotation process of the task is impractical in the field.
the middle of the training routine on the new task. Some rehearsal-bacc
2

---

Under review as a conference paper at ICLR 2023
There also exist self-supervised CL methods Rao et al. (2019); : Smith et al. (2021), which focus on the generation of the diiussiiiiiiiiaaiiee representations from the unlabelled sequence of tasks. However, their demonstration far are still limited to relatively simple and low-resolution datasets, such as MNIST and CIFAR-10.
3
METHODOLOGY
3.1 TERMINOLOOGY
Algorithm 1 Unification of One-class Novelty Detection and Continual Learning
Previous CL methods use task- based setup, where each task consists of multiple classes of training sam- plcs. The model is trained to learn each task (i.e., multiple classes) quentially. Different from previous methods, our proposed solution is an unified system that leverages the out- put from the novelty detector for CL. Therefore, we embrace the one-class per task setup as follows:
Input: IDD memory budget
AIDD
Xmer Xmem=), ).
IDD sequence
XIDD
(DD
XIDD..... XN IDD) OOD sequenee XOOD
(XODD..... p.p......oonn (ooD) trained model Mo
1: Mourrent + Mo
2: for i = to len len(XooD) do
3:
XOOD XooD + KoODli]
4:
Kmizture + XIDD + XODD
Xeurrent
5:
DIDD &- -DD estimator &mme Meurrent)
As shown in Algorithm [ the sys- tem is continuously exposed to stream of mixed input that con- tains both in-distribution data (IDD) XInD
6:
X Bi t movellyDeleclorr (Meurrent
LDDD DIDD, Xmixture)
7: Amem append(Xoon) Xpred OOD)
8:
Mearrent t Mearrent + B, //Merge Model
{XIDD..... X and one of out-of-distribution (OOD) data Xeurrent XOOD {Xo OOD .x.o....,X%ooo
9:
end for
10:
return Mewrrent
X .ooo) where each X' corresponds to single class in the dataset. XIDD denotes the IDD set containing all the classes that the system already recognizes. ROOD contains the OOD classes that the system is currently fac- ing and will encounter in the future. The primary task is filter out Xourrent from XDDD through the novelty detection engine, while learning the features of X current for continual learning. Once Xeurrent from XOOD into XIDD. and ran- XooDe is successfully detected and learned, we move XOOD
domly draw a new OOD class from XOOD as the new Xoupe This process continues until ROOD empty and all classes have been learned by the model, i.c., becoming IDDs. For simplicity, we denote the initial IDD classes and the first OOD class as X and XooD. respectively. When the system has successfully learned XOOD. we denote the updated IDD set as XIDD and the next OOD as XXoD Then S, DD XoOD) represents the status when the system just finishes learning XOOD- From the perspective of Our proposed method will detect and learn only one class each time.
learning, each XOOD can be considered as a single task.
3.2 ONE-CLASS NOVELTY DETECTION AND CONTINUAL LEARNING
Our method is built upon the recently proposed work on "Self-supervised gradient-based novelty detector" et al. (022)). The contributions of this method are two-fold. First, given a pre-trained model Mo that is able to classify ATDDD- statistical analysis evaluating the Mahalanobis distance in the gradient space is developed to threshold the OOD. Second, to further boost the performance, they introduce a self-supervised binary classifier, denoted Bo. which guides the label selection process to generate the gradients so that the Mahalanobis distance between the IDD and OOD data is further maximized. The primary OOD detector, which is based the Mahalanobis distance, interleaves with the binary classifier. As more data stream into the system, the OOD detection accuracy gradually improves through this closed-loop interaction.
This method can be naturally converted into a one-class CL solution. Upon the successful detec- tion of XoOD. the binary classifier By is well-trained to recognize X from previous XDDD- Intuitively, we can deploy B, upstream from the pre-trained IDD model Mu to filter out the newly learned XooD and leave the predicted XTDD samples to the downstream Mo for further classi- fication. This merged model Mi {Mo + Bo} will become the new baseline to detect the next XooD- Continuously, we can keep adding new binary classifier B, the previously merged model M; (M;- B for every new XbOD upon its detection. Eventually, we obtain this
3

---

Under review as a conference paper at ICLR 2023
chain structure (Fig. where the latest binary classifier is the most knowledgeable to recognize all KIDD and ROOD. During the inference phase, its job is to filter out the XOOD from mix- ture input that contains (KfDD. X'oo)- The remaining inputs LDDD = { xooo will be sent downstream to the next binary classifier to sequentially recognize the X This sequence continues until only XIDD is left and arrives at the original model Mo for final classification.
Algorithm Gradient- Novelty Detection
There are two advantages of this one- class learning method. First, after we merge new binary classifier to the existing structure, all weights are frozen and isolated from future train- ing towards the new OODs. This prevents the knowledge shifting and thus, minimizes catastrophic forget- ting. Second, each binary classifier only induces small memory over- head since its task is simple enough to be accomplished with very few layers of neurons. Compared with previous dynamic methods which either add an indeterminate amount of neurons each layer or make new branches using sub-modules, our method re- quires much lower memory.
function novelty
DIDD: Xmix)
2:
Initialize Brew
3:
X x- batch PeriichPPrtyytttiiii (X miz:
ADDD M)
4:
repeat 10 iterations
Xpred + [[], Xrred - oppe for each E X do
5:
6:
7:
B Bnew not trained then
8:
CMo t MMg cMg CB, t C,
pred
9:
else
10:
CMo f- ceust
MMg 3 CB t-
pred
B;
11:
end if
V fm) + V Veng fMo (2))2()) ... 11 cas (2)
ces
12:
noveltyScore. 4 [[]
13:
for each (jie, E.) in DIDD do
score_c + (Vfm(z)-Ho)) noveltyScore Lappend(score scooe c) end for
(V (x)
On the other hand, such sequen- tial learning model leaves two ques- tions to be answered: (1) How to port the original Mahalanobis dis- tance method into this merged model so that can detect the next OOD? (2) How achieve a high infer- ence accuracy of the binary classi- fier, such that the testing samples can go through multiple steps in this se- quence and reach the final classifier without losing the accuracy. We will address each problem in the subsec- tion 3.3 and respectively.
14:
noveltyScore + min min(novelltycoree]
16:
if noveltyScore < threshold then Xipo xIDDooppedd(z)
17:
18:
else
19:
X.red
(ponappeed(r))
20:
end if end for
21:
Re-initial & Train Brew using {.x pred x pred
22:
23:
return Xxpred Baes
24:
end function
3.3 GRADIENT-BASED NOVELTY DETECTOR
The stepping stone of the previous novelty detector (Algorithm 2 Algorithm 3) is to charac- terize the IDD and OOD in the gradient space. If model M has been trained to classify N classes in Xtrain XIDD will form class-wise multi-variant distribution DIDD (DIDp..... D Each DIDD ~ N(;, Σn corresponds to Gaussian distribution for a particular class i, and future IDDs XTBD (xyel...... X } will be within the range of corresponding Dj DDD On the contrary, due the low confidence towards the OOD, the model M will adjust itself fit the OOD properly by back- bock-propagating the gradients with abnormal magnitude and direction. Any deviant from D will be considered abnormal and therefore, a distance metric can be utilized to measure the nov- elty confidence. They propose to use the Mahalanobis Distance as the novelty confidence score Sun al. 2022):
(Xiror .... , Xtroin_Ny NN] the gradients collected from each XiDD
M₂ = (Velm(x) E (Velm(x) (ec)
To collect the gradient castom predicted and e M
V VefM(z), they further utilize two types of labels for back-propagation:
predicted = cEXioo
CM
4

---

Under review as a conference paper at ICLR 2023
eustom
CM
cEXidD
Measured by the softmax output from the trained model M, predicted and costom refer to the CMM
most and least possible class the input belongs to, respectively, leading to mild and aggressive back- propagated gradients. Since the goal is to detect the abnormal gradient from the OOD, using coustom for OOD maximizes weight update to the model and thus, the gradients will be even more likely to predicted for IDDs helps their gradients stay within DIDD and thus, reduces the possibility of false alarm. To guide the label selection, they introduce binary classifier into the system to predict the samples. Based on the prediction, the primary ND engine M collects the gradients with either CM distance towards DIDD.
fall out of the DIDD. On the other hand, using CM
crustom to measure the Mahalanobis predioted or CM
To make the above algorithm compatible with the unified ND model, we set up the following sce- nario for explanation. Assume the system just finishes the detection and learning on XooD and is in the status of S (XiDD:Xoo
The latest binary classifier B,, which can distinguish X XOOD and Xi DD is placed upstream from the previously merged model Mi to form
model M (M, + B. At this point, Mi+1 becomes the new baseline detect xoon- It is a quential structure composed of i++ number of the binary classifier, i.e., one for each learned OOD, and one original neural network Ma that predicts N classes and is placed the end of the chain. With these notations mentioned above, we introduce our solution as follows:
Gradient Collection: To make this merged model M4+1 (M: + B₂} capable of detecting the upcoming XOOD- we need to first conduct the DibD estimation on X = {DiDDD X from both M₁ and B₁ Only then we can start evaluating how far the XoOD deviates from the distributions in IDD. Given an input we propose to collect the gradients V.f8s (z) and V.fM, (x) separately
from both B, and M, Then we concatenate them together V.JB, V.BB,+M, (2:) = V.JB, (+) (LL, The overall gradient dimension becomes V [fM++1 = VMMb (V f.,ll.... ... II. where V VJM, is
the gradient collected from the original neural network, and f.B..... are the gradients from each binary classifier in the chain.
Class Selection: Similar as CM predicted and cM coustom to VeJM(z). here we introduce CB. custom control the gradient Vels, (x). Since there are only two possible predictions: IDD (label B:
predxcted and
predicted and use the flipped 0) and OOD (label 1), we can simply make the original prediction eg, custom result B,
predicted B.
fIS (X:
custom CB
fi. (X
Ideally we prefer to use the label &B. gradient V Vefs, (z) will be minimized. For XODD using cB, even more and thus, easier to be detected.
predicted for all the samples E {XiDD: (OOD). so that the custom will make their gradients stand out
Memory for DIDD Evalu- ation: The extra dimension from Vfs, requires the pre- estimated distributions in DIDD to be re-evaluated. For each class in git1 memory a constant size to re-evaluate DIDD after the ex- pansion of the gradient dimen- sion. In addition, the newly learned XooD becomes a new IDD class from the perspective of M and thus, DIDD needs to include D Before M to detect the next OOD, DIDD should include the distribution estimation of N +i+ + classes, in the dimension of V /M
Algorithm 3 Gradient-based Evaluation of IDD Distribution
1: function IDD_eetimaateeee _estimator (XIDD .,))
2: DIDD <
3: for each Mem in Xmmm do
4:
VelM. (Tg) = V VeJMo (..x)ll../I///...... (T)
IDD we assign a
5: Po No Σ V VefM (Ik)
Wk
6:
E.
Σ (Vefm, - (V)(V
( where (Ix, yk) E Xmem
7: DIDD hrnn-append Ec))
8: end for
9: return DIDD
10:
end function
5

---

Under review as a conference paper at ICLR 2023
Furthermore, small memory for each class helps improve the ND accuracy. Previous methods only rely on the predicted IDD and OOD from the novelty detector engine to train the binary classifier, with limited training accuracy and convergence speed when the prediction is not accurate enough. Instead, we propose to directly use the pre-stored IDD samples as the IDD training dataset. For the OOD, we still use the predicted OOD from the ND routine. Such a solution makes the training of the binary classifier more stable, by reducing mislabeled OODs. After the system converges towards current OOD, those predicted OOD samples, selected by the novelty detector with the assistance from the binary classifier, will be stored in the memory as the representation of this newly learned class, which will be used for DIDD evaluation and future training on the new binary classifier. Except for the first N classes which have the pre-labeled dataset, all the samples from the new coming OODs are self-selected by the engine itself.
3.4 MATC TRAINING AND INFERENCE
The success of the sequential classification,, from a sequence of binary classifiers, relies the high inference accuracy of each binary classifier. Even with very minor accuracy degradation from each of them, the accumulated accuracy will drop exponentially. To achieve such a goal, we propose to not only consider the sample itself but the context of the sample as well. If a testing sample is within the cluster of other samples that all come from the same class, its prediction will be biased towards that class as well. This inspires the idea of the batch-mode training and inference: Based on the features, we will cluster the samples in one class into batch first, before feeding them the engine. During the training phase, we introduce the new loss function as follows:
1 N-1
((X) = N
Σ ((f(X)).)
(5)
where each X' refers to single batch with samples of class i only. Different from traditional train- ing method where the ferd-forward operation is conducted on a single batch that contains randomly selected samples from N classes, send N individual one-class batches to the classifier all at once and calculate their average loss. Due to the nature of the BatchNorm layer in the neural network, find that the batch average from each class can be better separated and thus, the boundary of the classes is easier to be learned.
To prove the effectiveness of the batch-mode training, we evaluate the feature distribution of IDD and OOD using both the traditional and batch-mode method with CIFAR-10 dataset on a binary classifier, where IDD contains five classes and OOD contains one class. The batch size at 32 samples. As shown in Fig. (b)),), the result from batch-mode training is less intertwined, promising a higher chance to detect OOD, We exploit this property to improve the accuracy of each binary classifier.
One challenges in this approach is that only XIDD classes are available for creating the pure batches, but the OOD samples are mixed with IDD in the incoming stream, not pre- pre-labeled. Therefore, a pre- filtering operation is necessary to detect and prepare the OOD from the data stream. We propose to first divide the input stream into small consecutive batches and use a purity metric to localize the batches with the highest OOD percentage. In reality, the assumption of IDD/OOD batches is quite feasible. For instance. in a video audio stream, once OOD data appear, there will be multiple, continuous samples of OOD, rather than one glitching sample only.
Regarding the purity metric, we propose compare the mean of the features from the testing batch with the pre-estimated features of each class in XIDD. More specifically, assuming the system is in status S. (xi DD. XbOD). there are totally N + i classes in KiDD- Therefore, we expect the next input stream will contain a mixture of XOOD batch and N + i kinds of batches from previous classes. We use Mo to filter out the batches that contain the first N IDD classes, by comparing the To2 distance between the batch features viso (Xtsst) and the X A/DD features viMa (X. both extracted from Mo. Since Mo is trained by x/ if Xtest batch is dominated by the samples from XDDDD then the La distance between them should not be large. On the other hand, if the batch is mixed with samples from another class, then the Le distance will increase which helps us set threshold to separate them. The selected batches will be sent to the upstream of My. which is Bu, for another round of filtering to remove the batches that are dominated by X This procedure continues along the sequential structure from bottom to top until all the batches containing the N +i classes are filtered out. The remaining batches then only contain the new OOD X XoOD-

---

Under review as a conference paper at ICLR 2023
0.05
Algorithm Unsupervised Estimation of Batch Purity
IDD
Class D-4
OOD
function
batchhPuribyEsttmaato
0.04
CIFAR-10
Distribution
(X IDD M)
0.03
2: Divide X min sequence into batches with size 32
[X X Bt.
0.02
3:
X +
0.01
4:
for each XB E [o XBI...., X BBd do
5:
for each m along the chain [Ma. B B B,]
0.00
80
do
L2 Distance
6:
m is Mo then
Frame-based feature distribution
Xref < select ADDD from LDD
7:
0.05
8:
else if is B& where k [0, then
9:
X select XoOD from LDD
0.04
CIFAR-10
OOD
Class 5
Distribution
10:
end if
1.2 +- d d(vm B), 4m (XTD))
0.03
12:
if L2 < thresholdydD then Xuue
0.02
Batch size 32
13:
X mii aappend(X.)) XB)
14:
else if L2 > thresholdooD then
0.01
15:
Continue
0.00
16:
else Break end if
0
10
20
30
40
50
60
70
80
L2 Distance
17:
18:
end for
(b) Batch-based feature distribution
19:
end for
return XPruee
Figure The feature distribution of IDD and OOD using (a) traditional training and (b) batch-mode training.
21:
end function
MD: Class 0-4 vs. Class 5.9
25
CIFAR-10
IDD
OOD
B0: Class vs. Cless 6-8
L2 Distance
20
Xoop pure batches
B2: Class vs. Class 8-8
15
detected by 80
B3: Class vs. Closs
10
Afno
Xooo Class
Xijop pure batches
Xbon Class Xiog Claes
5
Xdoo: Class Xdoot Class
detacted by MD
Estimated in batch size 32
Moo
Xbo Xeax Koop
Xoop
Input Stream
Figure 3: L2 distance-based batch purity estimation using Mu and B IDD: (& Ipp X bOOD...... X3 XoOD) and OOD: XoOD- All data are collected using CIFAR-10 dataset. Fig. illlstrattes the process separate the pure batches from the CIFAR-10 input stream that consists of five types of IDDs ({ XD. and Bo-3. The gray and white area corresponds to the batches with 100% purity and mixture data, respectively. Starting from Mo (the red curve), XDD pure batches are collected by comparing the La distance with low threshold. The batches above threshold are sent to Bo to find the X X pure batches. This process continues until the stream data reaches B3 and all IDD and OOD pure batches are successfully separated.
........... } and one OOD (Xoop), using Mo .X
4 EXPERIMENTS
To prove the efficacy of our proposed method, we conduct several one-class ND and CL experi- ments using MNIST Deng (2012), CIFAR-10 10, CIFAR- 100 Krizhevsky & Hinton (2009) and Tiny- ImageNet[Le Yang (2015). All experiments are implemented using Py Torch Paszke et al. (2019) on NVIDIA GeForce RTX 2080 platform.
4.1 EXPERIMENTAL SETUP
Input Sequence and Memory Budget: Different from the multiple-class based setup, our method evaluated after every exposure new OOD class. For every new XbOD we mix it with the
same amount of randomly selected samples from XiDD. This mixed input stream is then sent the system for ND and CL. The input stream consists of the batches from N i classes in 2 Dp and

---

Under review as a conference paper at ICLR 2023
from the Xoon current class. Each batch is at the size of 32 frames for all the experiments. We also create the transition phases to mimic the input change from one class to another. Each transition phase last three batches long. The ratios of the mixture between the previous class and the next class are 1/4, 1/2 and 3/4. This transition setup is used in batch purity evaluation. For each dataset, the size of X DoD and X DD samples is shown in the first row of Table XOOD samples are selected from the training dataset while the DD are selected from the testing dataset. The reason for using testing rather than training on XiDD is because previously trained binary classifiers have already seen the training data of X DDD and we need to avoid any unfair evaluation in the current iteration. The second row of Table presents the memory budget for XIDD-
Network Structure and Training: For fair comparisons with previous methods, we select the structure of Mo shown in Table [ Row 3. The binary classifier has three convolution layers, one BatchNorm layer, and a Sigmoid classifier. This structure is used all experiments. For Mo training. standard Stochastic Gradient Decent is used with momentum 0.9 and weight decay 0.0005. The number of epochs is listed in Table [ Row 4. The initial learning rate set to 0.1 and is divided by 10 after reaching the 50% and 75% milestones. For the binary classifier, we train it in 100 epochs with the Adam optimizer Kingma& & Bal (2014) where the initial learning rate is set to 0.0002 and the decay rate is controlled by 31 0.5 and B3 0.999. To estimate the novelty in the gradient space, we collect the gradients from the last convolution layer of Mo and the second from the last convolution layer of each binary classifier.
Finoturing MAS
EWC
LuF
ICaRL
Our
Ours 2+8
Ours 5+5
EWC
POMA
CWM
Single Head Accuraloy
8
dP
A-GEM
Single Head Acourde
90-
80
70
A-GEM
Co2L
G8S
GEM
GEM
IcRL
ER
10
20
30
40
50
70
80
90
100
2
3
5
7
9
10
Number Classes
Number of Classes
(a) CIFAR-100
(b) CIFAR-10
Single Head Accurscy,
100
Head Accuracy
110
100-
CCRL
00-
OWM
70:
LwF
60:
DooL
ER
30
A-GEM
CaRL
Ours
A
40
80
80
100
120
140
180
180
200
2
3
5
7
9
10
Number Classes
Number of Classes
(c) ImageNet
(d) MNIST
Figure 4: Single-head accuracy of one-class novelty detection and continual learning using (a) CIFAR-100, (b) Imagelm, (c) CIFAR-10, and (d) MNIST.
4.2 IN-DEPTH ANALYSIS
All experiments are conducted in an unsupervised manner, which means all new OOD classes are not manually labeled, but purely rely on the prediction from the novelty detector engine. Compared with traditional CL algorithms which learn multiple classes in one shot, our method learns one class each time and takes more steps to achieve the same learning goal. To test how many classes can be learned before the accuracy starts to drop, we test our algorithm using CIFAR-100 100 by training a baseline model using 10 classes and then continually feeding 20 new classes the system one class after another. The inset from the Figure aa illustrates the accuracy curve from the actual testing result (red points) plus the extrapolation (dashed line). This curve proves that our method is able to stably learn multiple steps; the accuracy eventually drops as the error in each binary classifier accumulates through the sequential process.
Therefore, we design our further experiment as follows: For CIFAR-100 and Ting-mma we divide them into 10 tasks where each task contains 10 and 20 classes, respectively. After the base- line training with the first task, we test the algorithm performance by feeding the next task as the incoming OOD. Once all the classes from the new task are learned one by one, we terminate the baseline model using all the previous tasks. This new baseline
current iteration and retrain
8

---

Under review as a conference paper at ICLR 2023
will then be used for learning the next available task. This process continues till all the tasks have been tested. For each experiment on CIFAR-10 10 and MNIST, we train two baseline models using the first two classes and the first five classes mimic the 5-tasks (5T) and 2-tasks (2T) setup that used by other methods. We then feed the remaining classes to the baseline model to test the performance.
As shown in Fig. the single-head accuracy of all the experiments stay high value after consec- utively detecting and learning new classes from each checkpoint, which proves that our batch- method successfully boosts the performance of the binary classifier. For CIFAR-10 and MNIST, our method significantly improves the state-of-the-art even though is unsupervised. For Tiny- ImageNet, the accuracy is less stable due the increased complexity of the dataset, but overall the performance is still robust after learning 20 new classes.
ABLATION STUDY
We conduct four ablation studies to test how the performance is influenced by various input stream patterns using CIFAR-10 and CIFAR- 100 dataset. First, we conduct two experiments by feeding the input with 72% and 90%/10% of IDD/OOD mixture. As shown in Figs. Figs.55a))0 ([a)(b), the single-head accuracy of both experiments are worse than previous experiments using 50%/50% of IDD/OOD mixture. This is because fewer OOD samples increase the difficulty in OOD separation, especially the unsupervised training of the binary classifier. Second, we test our model with two smaller batch size for batch purity estimation. Figs. Figs.33))( (d) present the performance after dividing the input stream into smaller batches at the size of 16 and 8. The smaller the batch size is, the worse performance our model achieves. With fewer samples in each batch, the average feature estimated from the trained model becomes more diverse, which exacerbates the error in the purity estimation engine.
50%,25% EOG200 509 2-8 DDO000 254-259
IDDIOOD
DD/0CO
IDDO0OD 50%
18/00/000
9256/15 EDC000 159.229 00/000 92%
Single Head Accuracy
Single Head Accuracy
100
00-
70
80
40:
40
30-
20-
20
20
30
40
50
70
90
100
2
5 Number of Classes
10
Number of Classes
(a) CIFAR- 100: IDD/OOD ratio
(b) CIFAR-10: IDD/OOD ratio
215 bebdiicee2 5/5 36ktrac-33 215 1esth
Single Head Accuracy
100
50
batchsmentl
botchreee-16 16
batchsmee-32
Single Head Accuracy
3:5 babffcc-11 215 bebthacedd 515 beshroodd
70-
70
60
50-
50
36-
10:
10
10
20
30
40
50
60
70
80
90
100
2
5
8
10
Number of Classes
Number of Classes
CIFAR-100: Batch size
CIFAR- 10: Batch size
Figure 5: Ablation studies CIFAR-10 and CIFAR-100 with (a)(b) multiple ratios of IDD/OOD
mixture, and (c)(d) various batch sizes.
CONCLUSION
In this paper, we propose an unified framework for one-class novelty detection and continual learn- ing, by using a sequence of binary classifiers with the batch-mode technique. We demonstrate that our method successfully detects and learns consecutive OOD classes in an unsupervised setup, achieving stable single-head accuracy without triggering catastrophic forgetting, For instance, our method reaches 97.08% on CIFAR-10 10 in continual learning, better than the state-of-the-an The performance all other datasets is also among the top list, without the need of manually labeled training data. The success of this approach promises high stabilization, high learning accuracy, and practical usage by various dynamic systems.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. Uncertainty-bant -based continual learn- ing with adaptive regularization. Advances in neural information processing systems, 32, 2019.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019.
Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9516-9525, 2021. 2021.
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprini 18/2vv///220042::: 2018.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-144 2012.
Ian Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empiri- cal investigation of catastrophic forgetting in gradient-basee neural networks. arXiv preprint arXiv:1312.621/ 2013.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Steven C. Y. Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu- Song Chen. Compacting, picking and growing for unforgetting continual learning. ArXiv, abs/ 1bs/09900, 2019.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6988 6980, 2014.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barde Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3222 2017.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Tech- nical report, Citeseer, 2009.
Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. 2015.
Xilai Li. Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International Conference on Machine Learning, pp. 3925-3934, PMLR, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis 2947, 2018. doi: 10.1109TPAM.2211.2773888
and Machine Intelligence,
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems 32, pp. 8024 8035. Curran Associates, Inc., 2019. URL http:::ppppt...eeeeeeee www.ppppww....is....co.p cwtpppppppcpppppp..pppppppp 2015-pytorss an -n--mmeaaaiie--tttteee otw.a//eeeeet.ee.ioooooeeeooeeee
www.aaninnnniibbar.....mm Librar.r......
pddf
10

---

Under review as a conference paper at ICLR 2023
Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Teh, and Raia Hadsell. Continual unsupervised representation learning. Advances Neural Information Processing Sys- tems, 32, 2019.
Sylvestre-Alviss Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning, Advances in Neural Information Processing Systems, 32, 2019.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.0400 2016.
Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka Grabska- Barkkiska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for contin- ual learning. In International Conference on Machine Learning, pp. 4537. PMLR, 2018.
James Smith, Cameron Taylor, Seth Baer, and Constantine Dovrolis. Unsupervised progressive learning and the stam architecture. In IJCAI, 2021.
Jingbo Sun, Li Yang, Jiaxin Zhang, Frank Liu, Mahantesh Halappanavar, Deliang Fan, and Yu Cao. Gradient-based novelty detection boosted by self-supervised binary classification. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 8370-8377, 2022.
Jachong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: 01547, 2017.
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent pro- cessing in neural networks. Nature Machine Intelligence, 1(8):344-72, 2019.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995, PMLR, 2017.
A APPENDIX
Parameters
MNIST
CIFAR-10
CIFAR-100
Tiny-ImgNet
Size of OOD
4000
2000
500
1000
Memory Budget
4000
2000
500
500
Mo Train Epochs Mo
MT.P
'ResNet-34
ResNet-34
ResNet-18
30
70
200
200
He al. 2016
Table 1: Hyper-parameter: for each experiment.
11


Reviews:
[{"review_id": "mjwd9i7q77", "reviewer": "Reviewer_e2pH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new learner which can address novelty detection and continual learning jointly. Specifically, a batch-mode training and inference method is used to maximize the feature separation between OOD and IDD data samples. Experimental results on several continual learning benchmarks show the effectiveness of the proposed method.", "summary_of_the_review": "It is necessary to see the replies from the authors for the questions above.", "strength_and_weaknesses": "The motivation behind this work is convincing and interesting. Novelty detection and continual learning are really two related tasks in an open-world scenario, because most continual learning methods are limited under a closed world setting where all the incremental classes are fully labelled in a supervised learning fashion. However, there are still several concerns on this work:\n\n-- Although novelty detection is a binary classification task in general, one-class continual learning is not so practical in the applications. In contrast, the continual learner should be able to learn more classes during each incremental session. One potential solution is to consider the task of novel class discovery which learns to cluster the unlabeled samples from a set of classes. \n\n-- In this work, the authors discussed that the proposed method helps achieve the high accuracy in OOD detection and prediction in a\nscenario where IDDs and OODs are streaming into the system, such as videos and audios. However, there has no experiments on video and audio datasets. It lacks experiments to support the conclusion above.\n\n-- The proposed method is built based on existing novelty detection and continual learning methods. It needs more insightful clarifications on the new technical contributions in this work. Otherwise, the novelty would be limited.\n\n-- The training procedure might be expensive when reading the Algorithms in the paper. Apart from the accuracy performance, it is encouraged to consider the computational efficiency, especially for a continual learning model.\n \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "i71idK94cq", "reviewer": "Reviewer_PhMt", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a novelty detection based continual learning method. The scenario is practical, but the proposed method seems to be largely heuristic and the tested scenario is somewhat contrived. ", "summary_of_the_review": "From above reasoning, I think the paper considers an important variation of ND + CL setting, but the overall scheme requires many heuristics and it is not clear how they will generalize to other data settings. ", "strength_and_weaknesses": "Strength\n- The motivation of the paper is rooted in the practical scenario in which OOD data is continuously arriving while learning IDD data. \n- The paper attempted to merge the two method from ND and CL. \n- Gradient based novelty detection is adopted for the continual learning scenario. \n\nWeakness\n- The proposed method applies several heuristics, which is not clear whether it can be really generalized to settings other than the ones considered in the paper. For example, the threshold for novelty detection (Alg. 2) and threshold_OOD (in Alg.4) should play important roles on overall performance, but it is not clear how to choose them. If the thresholds were chosen to maximize the overall test performance, it violates the continual learning assumption. \n- CIFAR-10/100, MNIST seem to be too small/standard benchmarks, and testing on more enlarged / diverse datasets should be necessary, provided that one of the main concern of the paper is OOD novelty detection. The new class in CIFAR could be thought of as a novel data, but they might have still similar distribution. What happens when completely differently distributed data point comes? From this reasoning, I think the current form of the paper is somewhat limited for a publication.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Bzc62CVzx0K", "reviewer": "Reviewer_wx2w", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on unifying novelty detection and continual learning methods by extracting and learning from the OOD data and merging it with the IDD distribution. It uses a sequence of binary classifiers with a batch-mode technique to achieve this goal. The proposed method improves performance compared to prior CL methods on MNIST, CIFAR-10/100, and Tiny-ImageNet datasets. ", "summary_of_the_review": "The paper proposes an interesting combination of novelty detection and continual learning techniques; however, due to the missing discussion and comparison with prior task-free CL and self-supervised CL methods, it is difficult to position the work. Furthermore, the experimental evaluation is weak and lacks comparison with recent CL methods. Therefore, for these reasons, the paper is not ready for acceptance in its current form.", "strength_and_weaknesses": "## Originality and Quality\n### Strengths\n* The paper attempts to unify novelty detection and continual learning techniques, which are novel and might interest the community.\n* The experiments are conducted in an unsupervised setting, a comparatively more realistic setting for practical applications.\n## Weaknesses\n* The proposed framework resembles the existing task-free continual learning setting [1, 2], where it uses the gradient-based novelty detector to detect the OOD/new task examples. Thus, the novelty of the framework is limited. Further, it does not include a discussion and comparison with this setting. \n* While the paper focuses on the self-supervised CL setting, it does not compare or mention the prior methods [3, 4] that have investigated self-supervised continual learning.\n* The paper also needs a comparison with recent supervised CL methods [5, 6], which makes it challenging to evaluate the utility of the proposed method.\n* Many design choices and hyper-parameters need to be better ablated and justified. For example, the selection of the linear classifier architecture, learning rate, epochs used for training it (ensuring fair comparison here in comparison to without it), the effect of batch size in Figure 5, and its effect on performance.\n\n---\n\n## Clarity\nOverall, the algorithm and method are well-written and easy to follow. These suggestions should improve the paper's readability and overall presentation.\n* The details about the baselines should have been provided. Further, the information on the dataset split should be included when the datasets are described in the experiments section.\n* The notations in the algorithm can be improved significantly. For instance, use \\text when writing text in the algorithms, and use $\\text{argmin}, \\text{argmax}$, and $\\text{softmax}$ for better presentation.\n* The zoom in Figure 4a) highlights the proposed method instead of the comparison. In Figure 4c), the arrow disrupts the interpretation of other methods.\n* Many references do not reflect the conference proceedings bibliography.\n\n---\n\n## Reproducibility\n The code is not provided with the submission. Since the paper is empirical, it is necessary to provide the code to aid the reproducibility of future works.\n\n---\n## References\n[1] Aljundi et al, Task-Free Continual Learning. CVPR 2019.  \n[2] Wang et al. Improving Task-free Continual Learning by Distributionally Robust Memory Evolution. ICML 2022.  \n[3] Madaan et al. Representational Continuity for Unsupervised Continual Learning. ICLR 2022.  \n[4] Fini et al. Self-Supervised Models are Continual Learners. CVPR 2022.  \n[5] Buzzega et al. Dark Experience for General Continual Learning: a Strong, Simple Baseline. NeurIPS 2020.  \n[6] Saha et al. Gradient Projection Memory for Continual Learning. ICLR 2021.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "7u3Y-g5Tt0k", "reviewer": "Reviewer_t1EJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new method for continual learning with convolutional neural networks for image classification which is based on a novelty detection mechanism using gradients (Sun et al. 2022) and a iterative training and adding of binary IDD/OOD classifiers as new classes arrive in the input stream.\nThe accuracy of OOD is further improved by a batch-mode estimation where a pre-filtering step determine the most pure OOD batches from the mixed IDD/OOD batches.\nThe experimental results show that this approach is able to achieve a high performance with a large number of classes.\n", "summary_of_the_review": "Overall the paper proposes a novel and interesting approach for continual learning. But the presentation is lacking in some aspects making the paper difficult to read and understand.\nAlso the fact that a large memory buffer and an initial classifier is used for the first N classes may make the approach less suitable for real \"continual learning\" and potentially not comparable to the state of the art in the domain.\n\n\n", "strength_and_weaknesses": "Strengths:\n- Original approach for continual learning\n\nWeaknesses:\n- Paper difficult to read\n- Overloaded presentation and results difficult to interpret\n- Scalability may be an issue", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject