Bypassing Logits Bias in Online Class-Incremental Learning with a Generative Framework
Authors: Gehui Shen
Shibo Jie
Ziheng Li
Zhi-Hong Deng
Venue: ICLR 2022 Submitted (2022)
Keywords: continual learning
online class-incremental learning
catastrophic forgetting
deep learning

Abstract: Continual learning requires the model to maintain the learned knowledge while learning from a non-i.i.d data stream continually. Due to the single-pass training setting, online continual learning is very challenging, but it is closer to the real-world scenarios where quick adaptation to new data is appealing. In this paper, we focus on online class-incremental learning setting in which new classes emerge over time. Almost all existing methods are replay-based with a softmax classifier. However, the inherent logits bias problem in the softmax classifier is a main cause of catastrophic forgetting while existing solutions are not applicable for online settings. To bypass this problem, we abandon the softmax classifier and propose a novel generative framework based on the feature space. In our framework, a generative classifier which utilizes replay memory is used for inference, and the training objective is a pair-based metric learning loss which is proven theoretically to optimize the feature space in a generative way. In order to improve the ability to learn new data, we further propose a hybrid of generative and discriminative loss to train the model. Extensive experiments on several benchmarks, including newly introduced task-free datasets, show that our method beats a series of state-of-the-art replay-based methods with discriminative classifiers, and reduces catastrophic forgetting consistently with a remarkable margin.

Under review as a conference paper at ICLR 2022
BYPASSING LOGITS BIAS IN ONLINE CLASS- INCREMENTAL LEARNING WITH A GENERATIVE FRAMEWORK
Anonymous authors Paper under double-blind review
ABSTRACT
Continual learning requires the model to maintain the learned knowledge while learning from non-i.id data stream continually. Due to the single-pass training setting, online continual learning is very challenging, but it is closer to the real- world scenarios where quick adaptation to new data is appealing. In this paper, we focus online mnssemnnceeettall learning setting in which new classes emerge over time. Almost all existing methods are replay-based with softmax classifier. However, the inherent logits bias problem in the softmax classifier is main cause of catastrophic forgetting while existing solutions are not applicable for online settings. To bypass this problem, we abandon the softmax classifier and propose a novel generative framework based on the feature space, In our framework, a generative classifier which utilizes replay memory is used for inference, and the training objective is pair-based metric learning loss which is proven theoretically to optimize the feature space a generative way. order to improve the ability learn new data, we further propose hybrid of generative and discriminative loss to train the model. Extensive experiments on several benchmarks, including newly introduced task-free datasets, show that method beats series of state-of-the-ee replay-bases methods with discriminative classifiers, and reduces catastrophic forgetting consistently with remarkable margin.
INTRODUCTION
Humans excel at continually learning new skills and accumulating knowledge throughout their lifes- pan. However, when learning a sequential of tasks emerging over time, neural networks notoriously suffer from catastrophic
fo forgetting (McCloskey & Cohen 1989) on old knowledge. This problem results from non-i.i.d distribution of data streams in such a scenario. To this end, continual learning (CL) Parisi et al. 2019| Lange et al. 2019) has been proposed bridge the above gap between intelligent agents and humans.
In common CL settings, there are clear boundarics between distinct tasks which are known during training. Within each task, batch of data accumulated and the model can be trained offline with the i.i.d data. Recently, online CL (Aljundi et al. 2019c a setting has received growing attention in which the model needs to learn from non-i.i.d data stream in online settings. At each iteration, new data are fed into the model only once and then discarded. In this manner, task boundary is not informed, and thus online CL is compatible with task-free (Aljundi al. 2019b Lee et al. 2020) scenario. In real-world scenarios, the distribution of data stream changes over time gradually instead of switching between tasks suddenly. Moreover, the model is expected to quickly adapt large amount of new data, e.g. user user-generated content. Online CL meets these requirements, so it is more meaningful for practical applications. Many existing CL. works deal with task- iesk-innremettll learning (TIL) setting (Kirkpatrick et al., 2017) Li & Hoiem, 2018), in which task identity is informed during test and the model only needs to classify within particular task. However, for online CL problem, TIL is not realistic because of the dependence on task boundary discussed above and reduces the difficulty of online CL. In contrast, innsssinceemetall learning (CIL) setting (Rebufhi et al.) 2017) requires the model learn new classes continually
time and classify samples all classes during test. Thus, online CIL setting is more suitable for online data streams in roal-world CL scenarios Mai al. 2021)

---

Under review as a conference paper at ICLR 2022
Feretive wen Gemertee Charrter
Eefree contfer
0.5
C4
62
TARO
humber of taals
Tuak2
Tcak2
Tea
Tral
(b)
Figure I: Logits bias phenomenon of softmax classifier (left) and accuracy & forgetting on different tasks using softmax vs. generative NCM classifier (right). The results obtained with ER and 1K replay memory 5-task Split CIFARIO.
Most existing online CIL methods are based on experience replay (ER) (Robins) 1995 Riemer et al., 2019) strategy which stores subset of learned data in replay memory and uses the data in memory to retrain model thus alleviating forgetting. Recently, in CIL setting logits bias problem in the last fully connected (FC) layer, i.e. softmax classifier, is revealed (Wu et al. 2019), which is main cause of catastrophic forgetting. In Figure [a] we show in online CIL, even if ER is used, logits bias towards newly learned classes in the softamx classifier is still serious and the forgetting is dramatic (See Figure Tb). Although some works (Wu et al.) 2019 Belouadah & Popescu (019)] Zhao et al. 2020)) propose different methods reduce logits bias, they all depend task boundaries and extra offline phases during training so that not applicable for online CIL setting.
old tasks
In this paper, we propose tackle the online CIL problem without the softmax classifier to avoid logits bias problem. Instead, we propose new framework where training and inference are both in generative way. We are motivated by the insight that generative classifier is more effective low data regime than discriminative classifier which is demonstrated by Ng & Jordan 2001). Although the conclusion drawn on simple linear models (Ng Jordan 2001), similar results are also observed on deep neural networks (DNNs) YYggtaaa et al.] 2011 7 Ding et al.) 2020) recently. It should be noticed that in online CIL setting the data is seen only once, not fully trained, so it is analogous to the low data regime in which the generative classifier is preferable. In contrast, the commonly used softmax classifier is discriminative model.
Concretely, we abandon the softmax FC layer and introduce nearest-class-meat (NCM) classi- fier (Mensink et al. 2013) for inference, which can be interpreted as classifying in generative way. The NCM classifier is built the feature space the top of previous network layers. Thanks to ER strategy, NCM classifier can utilize the replay memory for inference. As for training, inspired by recent work (Boudiaf et al.) 2020), which shows pair-based deep metric learning (DML) losses can be interpreted as optimizing the feature space from a generative perspective, we introduce Multi- Similarity (MS) Joss (Wang et al. 2019) obtain good feature space for NCM classifier. Meanwhile, we prove theoretically that MS loss is an alternative to training objective of the generative classifier. In this way, we can bypass logits bias.
To strengthen the model's 's capable of learning from new data in complex data streams, we further introduce an auxiliary proxy-based DML loss (Movshovitz-AAiee Attias et al. (017). Therefore, our whole training objective is a hybrid of generative and discriminative losses. During inference, we ignore the discriminative objective and classify with the generative NCM classifier. By tuning weight of the auxiliary loss, method can work well in different data streams.
In summary, our contributions are as follows:
We make the first attempt to avoid logits bias problem in online CIL setting. our generative framework, generative classifier is introduced to replace softmax classifier for inference and for training, we introduce MS loss which is proven theoretically to optimize the model in a generative way.
2

---

Under review as a conference paper at ICLR 2022
2. In order improve the ability of MS loss to learn from new data, we further introduce an auxiliary loss to achieve a good balance between retaining old knowledge and learning new knowledge.
3. We conduct extensive experiments on four benchmarks in multiple online CIL settings, including a new task-free setting we design for simulating more realistic scenarios. Empiri- cal results demonstrate our method outperforms variety of statc-of-the-arr replay-based methods substantially, especially alleviating catastrophic forgetting significantly.
RELATED WORK
Current CL. methods can be roughly divided into three categories: which are regularization, parameter isolation and replay-based respectively (Lange ct al., 2019). Regularization methods retain the learned knowledge by imposing penalty constraints model's parameters (Kirkpatrick al. 2017) or outputs (Li & Hoiem 2018) when learning new data. They work well in TIL. setting but poor in CIL setting (van de Ven & Tolas 2018). Parameter isolation methods assign specific subset of model parameters, such as network weights (Mallya & Lazebnik (2018) and sub-networks (Fernando et al. 2017) to each task avoid knowledge interference and thus the network may keep growing. This type of method is mainly designed for TIL as task identity is usually necessary during test. The mainstream of Replay-basee old data and retrains it when learning new data to prevent forgetting of old knowledge. In addition,
methods ER-like like (Rebuffi et al. 2017). which stores subset of
generative replay method trains generator to replay old data approximately (Shin et al. 2017).
In the field of online CL, most of methods are on the basis of ER. Chaudhry et al. (2019b) first explored ER in online CL settings with different memory update strategies. Authors suggested ER method should be regarded as an important baseline as in this setting it is more effective than several existing CL methods, such as A-GEM (Chaudhry et al. 2019a). GSS (Aljundi et al., 2019c) designs new memory update strategy by encouraging the divergence of gradients of samples in memory. MIR (Aljundi et al. 22019a) is proposed to select the maximally interfered samples from memory for replay. GMED (Jin et al. 2020) edits the replay samples with gradient information to obtain samples likely to be forgotten, which can benefit the replay in the future. Mai et al. (202)) focus on online CIL setting and adopt the notion of Shapley Value to improve the replay memory update and sampling. All of the above methods are replay-based with softmax classifier. A contemporary work (Lange & Tuytelaars 2020) proposes CoPE, which is somewhat similar our method. CoPE replaces softmax classifier with prototype- based classifier which is non- non-parametric and updated using features of data samples. However, the loss function of CoPE is still discriminative and the way to classify is analogous to softmax classifier.
Apart from the above ER based methods, Zeno et al. 2018) propose an online regularization method, however it performs very badly in online CL settings (Jin et al.) 2020). [ee et al. (2020) propose a parameter isolation method in which the network is dynamically expanded and a memory for storing data is still required. Therefore, the memory usage is not fixed and potentially unbounded.
recent work (Yu et al., 2020) proposes SDC, CIL method based on DML and NCM classifier. However, SDC requires extra phase to correct semantic drift after training each task. This phase depends on task boundaries and the accumulated data of a task, which is not applicable for online CIL. In contrast, our method is based on ER and classifies with replay memory and thus need not correct the drift.
ONLINE CLASS-INCREMENTAL LEARNING WITH A GENERATIVE FRAMEWORK
3.1 PRELIMINARIES AND MOTIVATIONS
3.1.1 ONLINE CLASS- INCREMENTAL LEARNING
CIL setting has been widely used in online CL literature, e.g. (Aljundi et al. 2019a Mai et al. 2021 J
and softmax classifier is commonly used. A neural network f(:: 0) ))= : X Rd parameterized by encodes data samples æ E X' into a d-dimension feature f(x) on which an FC layer outputs logits
for classification: W V]()) b. At each iteration, minibatch of data B, from data stream S
3

---

Under review as a conference paper at ICLR 2022
arrives and the whole model (f.g) is trained on B, only once. The training objective is cross-entropy
(CE) loss:
LCE
Σ be log D:cr
y.c
e00
(1)
Ee= co.e
where Ö' the number of classes seen so far. is one-hot label of æ and the subscript c denotes the e-th component. The new classes from S emerge over time. The output space of g is the number of seen classes and thus keeps growing. At test time, the model should classify over all C' classes seen.
3.1.2 EXPERIENCE REPLAY FOR ONLINE CONTINUAL LEARNING
ER makes two modifications during online training: (1) It maintains replay memory M with limited size which stores a subset of previously learned samples. (2) When a minibatch new data B, is coming. it samples minibatch B, from M and uses B,, U B, to optimize the model with one SGD-like step. Then it updates M with B,. Recent works, c.g. (Aljundi et al., 20192) Mai et al. strong baseline, which combines ER with reservoir sampling (Vitter 1985) for memory update and random sampling for B,.. See Chaudhry et al. (2019b) for more details
2021) regard ER-reservoir
about it.
3.1.3 LOGITS BIAS IN SOFTMAX CLASSIFIER
Some recent works (Wu et al. 2019 Belouadah & Popescu 2019 Zhao et al., 2020) show in CIL scenarios, even with replay-based mechanism the logits outpuited by model always have strong bias towards the newly learned classes, which leads to catastrophic forgetting actually. preliminary experiments, we also observe this phenomenon in online CIL setting. We run ER-reservoir baseline on 5-task Split CIFARIO (each task has two disjoint classes) online CIL benchmark. In Figure Ia display the average logits of each already learned tasks over samples in test data after learning each task. The model outputs much higher logits on the new classes (of the task just learned) than old classes.
Following (Ahn & Moon 2020). we examine the CE loss in Eq I the gradient of Cce w.r logit O.c of class is Y: l[t.c = I]. Thus, if e is the real label y, i.e. t.c 1, the gradient is non-posin non-positive and model trained to increase o.., otherwise the gradient is non-negative and model is trained decrease O.c. Therefore, logits bias problem caused by the imbalance between the number of samples of the new classes and that of the old classes with a limited size of M. As mentioned in Section [] existing solutions (Wu et al., 2019; Belouadah & Popescu 2019 Zhao et al.) (0220) designed for conventional CIL need task boundaries to conduct extra offline training phases and even depend on the accumulated data of one task. They are not applicable for online CII. setting where task boundaries are not informed or even do not exist in task-free scenario.
3.2 INFERENCE WITH A GENERATIVE CLASSIFIER
Proposed generative framework is based on ER strategy, and aims to avoid the intrinsic logits bias problem by removing the softmax FC layer g and build a generative classifier on the feature space Z f(x). If the feature is well discriminative,
can conduct inference with samples in M instead a parametric classifier which is prone to catastrophic forgetting (Rebuffi et al. (017). We use NCM classifier firstly suggested by Rebuffi et al. (2017) for CL and show it is generative model. We use M. to denote the subset of class e of M. The class mean Mh is computed by n.M = 1 ExEM. fxx) During inference, the prediction for made by:
M.
y arg min [I(())) - 14.2
(2)
In fact, the principle of prediction in Eq (2 is to find Gaussian distribution N(f((" with the maximal probability for x". Therefore, assuming the conditional distribution p(zly = =N(<<ppM I) and the prior distribution p(y) is uniform, NCM classifier virtually deals with p(y|z) by modeling p(zly) in a generative way. The inference way is according to Bayes rule:
arg max plyl p()) arg max ((((")) (v)r())) The assumption about p(y) simplifies the analysis
4

---

Under review as a conference paper at ICLR 2022
and works well in practice. In contrast, softmax classifier models p(y|=*) in typical discriminative way.
As discussed above, online CIL is in a low data setting where generative classifiers preferable compared to discriminative classifiers (Ng & Jordan 2001). Moreover, generative classifiers are more robust to continual learning (Yogatama et al., 2017) and imbalanced data settings (Ding et al., 2020). At each iteration, B, UB also highly imbalanced. Considering these results, we hypothesis generative classifiers are promising for online CIL problem. should be noted our method only models a simple generative classifier p( =|y) on the feature space, instead of modeling p(æ) yy) on the input space using DNNs Yogatama et 2017), which is time-consuming and thus not suitable for online training.
3.3 TRAINING WITH PAIR-BASED METRIC LEARNING LOSS FROM GENERATIVE PERPECTIVE
To train the feature extractor /()) we resort to DML. losses which aim to learn a feature space where the distances represent semantic dissimilarities between data samples. From the perspective of mutual information (MI), Boudial et al. (2020) theoretically show the equivalence between CE loss and several pair-based DML losses, such as contrast loss (Hadsell et al. 2006) and Multi-Similariis (MS) loss (Wang et al.) 2019). The DML losses maximize MI between feature and label y in a generative way while CE loss in a discriminative way, which motivates loss to obtain a good feature space Z for the generative classifier.
train f(0) with pair-based DML
Especially, we choose the MS loss methods in the field of DML. Wang et al. (2019) point out pair-based DML losses can be seen as weighting each feature pair in the general pair weighting framework. As MS loss requires the feature f()) to be fg-normalized first, from now on, we use z; to denote the fz-normalized feature of x; and a feature pair is represented the form of inner product S.j- == x zj. To weight feature pairs better, MS loss is proposed to consider multiple types of similarity. MS loss on dataset D is formulated as follows:
training objective. MS loss is of the state-of-the-ar
1 (D) = D Σ
log(1 + Σ -S( + log
Σ B(S 111 JEN;
(3)
JEP:
where a, 8 and À are hyperparameters and P. and M₁ represent the index set of positive and negative samples of x resppectively. MS loss also utilizes the hard mining strategy to filter out too uninformative feature pairs, i.c. too similar positive pairs and too dissimilar negative pairs:
P: = (ilS
max Sa + +e} N₁ LilS > min Sik -
Vs. 7y:
where is another hyperparameter in MS loss. At each iteration use MS loss the union of new samples and replay samples LMs (B,, U B,) to train the model. The sampling of B, and update of M are the same ER-reservoir.
To show the connection between LMs and the generative classifier in Eq we conduct some theoretical analyses.
Proposition Assume dataset which has samples. For a generative model p(z, y), assume p(y) actually obeys the uniform distribution and p(z c) N(=lue I) where μ ng - E; c). For MS loss assume
{ y:) is class-balanced and has C classes each of
hard mining in Eq (4) is not employed. Then we have:
LMS Z LCen-Bin Bin
(5)
where E stands for upper than, up an additive constant and LGen-Bin Bin defined in the following:
1 logp(z,ly = c)
LGen Bin =
log p(pl(2/y v.) + nC
f=1
'The positive samples have the same labels æ. while the negative samples have different labels from æ..
5

---

Under review as a conference paper at ICLR 2022
The proof of Proposition]] I in Appendix. Proposition shows LMS is upper bound of LGen- Bin and thus is an alternative to minimizing LGen-Bin. The first term of Leen-Bin aims to minimize the negative log- likkeiiioooo of the class-conditional generative classifier, while the second term maximizes the conditional entropy H (Z Y) of labels Y and features Z. It should be noticed LGen- Bia depends on modeling p(zly). With uniform p(y), classifying using p(zly) equals classifying using p(2, v), and (Z is equivalent to ( Y), which can be regarded as regularizer against features collapsing. Thus, LGen- Bin actually optimizes the model in generative way. The assumptions in Proposition[]] 1 are similar with those in Section B.2 about NCM classifier. The difference lies in that NCM classifier uses (ur I computed on replay memory M to approximate {u²). Therefore, Proposition I reveals that MS loss optimizes the feature space in a generative way and it models p(zly) for classification which is consistent with the NCM classifier.
The real class means {x2) depend on all training data and change with the update of f(0) so that are intractable in online settings. MS loss can be efficiently computed as it does not depend on (u2) thus the model can be trained efficiently. During inference, we use approximate class means ue to classify. In Figure lb. on 5-task Split CIFAR 10 benchmark, we empirically show compared to softmax classifier, on old tasks, our method achieves much higher accuracy and much lower forgetting, which implies MS loss is an effective objective to train the model f and class mean μaa" of replay memory is a good approximation of p.?-
With discriminative loss like CE loss, the classifier models a discriminative model p(y|z). Therefore, if training with discriminative loss and inference with NCM classifier based the generative model p(zly), we can not expect to obtain good results. In the next section, experiments will verify this conjecture. In contrast, the way to train and inference are coincided in proposed generative framework.
3.4 A HYBRID GENERATIVE/DDSCRRIIINNI LOss
However, when addressing classification tasks, generative classifier has natural weakness, since modeling joint distribution p(ær, y) is much tougher than modeling conditional distribution p(y p(y)=) for NNs. Moreover, in preliminary experiments, we found if only trained with MS loss, the NCM classifier's performance degenerates as the expected number of classes in B, at each iteration increases. This phenomenon is attributed to the inadequate ability to learn from new data, instead of catastrophic forgetting. We speculate because the size of B,, is always fixed to a small value (e.g. 10) in online CIL settings, the number of positive pairs in B, decreases the expected number of classes increases.
To remedy this problem, we take advantage of discriminative losses for fast adaptation in online setting. To this end, we introduce Proxy-NCA (PNCA) (Movshovitz-Attie Attias et al. 2017). a proxy- based DML loss, an auxiliary loss. For each class, PNCA loss maintains "proxies" as the real feature to utilize the limited data in a minibatch better, which leads to convergence speed-up compared to pair- based DML losses. Concretely, when new class emerges, assign trainable proxy P. E Rd to it. PNCA loss is computed as:
LPNCA(D)
1 [D]
log E PP (2
Σ
_C (P
Movshovitz- et al. (017)) suggest all proxies have the same norm Np and all features have the
norm Ny. The latter satisfies as in MS loss the feature is fg-normalized, i.e. Nr = 1. We also set Np by normalizing all proxies after each SGD-like update. In this way, CINCA is equivalent to
CE loss with ez-oormalized row vectors of W and without bias b. and thus we use PNCA instead of CE loss to keep utilizing the normalized features of MS loss. Our full training objective is a hybrid of generative and discriminative losses:
(8) where y is hyperparameter to control the weight of LPNCA. In general, generative classifiers have a smaller variance but higher bias than discriminative classifiers, and using such a hybrid loss can achieve a better Mariinaraanee tradeoff (Bouchard & Triggs 2004). Thus we think introducing the discriminative loss LPNCA can reduce the bias of model so that boost the ability to learn from new data.
LHybrid= = LMS + 7CPNCA
should be noticed we train the model with L Ciybria (B, U,), while we only use NCM classifier in Eq ? for inference. In all experiments, we set = 2, B 50, e = 0.1 following Wang al. (2019)

---

Under review as a conference paper at ICLR 2022
Selt: CIAAR CO
SOEECITARSS
Sersochh AILO
6M02-44
GSp-Grady
50
04
10
2003 2000 3333
3000
1030
2000
2000
4630
500
10
5 10
Figure 2: The probability distribution on class the sample at each time step on Split CIFARIO (left) and Smooth CIFARIO (right).
Figure 3: Average Accuracy on already learned task during training.
and À = 0.5 which always works well in online CIL settings. We only need to tune hyperparemeters y in Eq g for different experiment settings.
EXPERIMENTS
4.1 EXPERIMENT SETUP
Datasets First, we conduct experiments on Split datasets which are commonly used in CIL and online CIL literature. On Split MNIST and CIFAR1O, 10, the datasets are split into 5 tasks each of which comprises 2 classes. On CIFARI00 and minilmageNet with 100 classes, we split them into or 20 tasks. The number of classes in each task is 10 or 5 respectively. For MNIST training following Aljundi et al. (2019)) and we use full training data for other datasets. To simulate a task-free scenario, task boundaries are not informed during training (Jin et al. 2020).
select 5k samples for
To conduct a thorough evaluation in task-free scenarios, we design a new type of data streams. For a data stream with C' classes, we assume the length of stream is n and My = n//C. We denote (t) as the occurrence probability of class c at time step t and assume (t) ~ (f((((2c - 1) ng n2) At pc)()) and normalize p(t) the parameters of
each time step I, calculate p(() (pi (4),- -..... Categorical distribution from which a class index c₁ is sampled. Then we sample one data of class o without replacement, In this setting, data distribution changes smoothly and there is no notion of task. We call such data streams as Smooth datasets. To build Smooth datasets, we set n 5k on CIFARIO and 40A on CIFAR100 and minilmageNet, using all classes in each dataset. For all datasets, the size of minibatch B, is 10. In Figure 2 we plot the probability distribution on class at each time step in the data stream generation process for Split CIFAR and Smooth CIFAR1O. For better comparison, we set the length of data stream of two datasets both 5k. Split CIFARIO has clear task boundaries and within one task the distribution on class is unchanged and uniform. However, On Smooth CIFAR1O, 10, the distribution class keeps changing and there is notion of task.
Baselines We compare our method against series of ofate-of-the-art online CIL methods, including: ER-reservoir, A-GEM, GSS, MIR, GMED, CoPE and ASER. We have briefly introduced them in Section Specially, we use GSS-greed and ASER,, which are the best variants in the corresponding paper. For GMED, evaluate both GMED ER and GMED-MIR. We also evaluate fine-tune baseline without any CL strategy. For all baselines and our method, the model is trained with epoch, i.e. online CL setting. In addition, the performances of i.i.d online and i.i.d offline also provided, by training the model 1 and 5 epochs respectively on i.i.d data streams. We reimplement all baselines except ASER,, whose results are from the original paper. Model Following Aljundi et al. (2019a). the model f is 2-layer MLP with 400 hidden units for MNIST and a reduced ResNet18 88 for other datasets. For baselines with ER strategy, the size of replay minibatch B, is always 10. The budget of memory M is 500 on MNIST and 1000 on others. We use a relatively small budget [M] to mimic a practical setting. All models are optimized by SGD. The single-head evaluation is always used for CIL. More details about datasets, hyperparameter selection and evaluation metrics are in Appendix.
4.2
MAIN RESULTS ON Split DATASETS
On Split datasets, we Average Accuracy and Average Forgetting after training all tasks (Chaudhry et al., 2019b) for evaluation, which are reported in Table and Table rebbeeileepe For each metric, we report the mean of 15 runs and the 95% confidence interval.
7

---

Under review as a conference paper at ICLR 2022
MNIST
CIFARIO
CIFAR100
CIFARI00
minilmageNet (10-task)
minilnageNNt
Methods
(5-task)
(5-task)
(10-tnsk)
(20-makk
(20-mnkk)
fin
0.06
18.40:0..17
6.26
361 0.22
4.43:10.19
3.1240.15
ER-reservoir
82, 22.34:2.448
39.88
11.59:40.26
8.95++0.26
10.24:0.41
8,33:40.66
A-GEM
25.99=1.62
18.01+0.117
6.48=0.18
3.66=0009
4,68:10.11
1.37:0.13
USS-Greedy
83.88-00.72
39.07422.02
10.78:+00.25
7.94=0.47
9.20++-0.1
7.76:+0.35
MIR
86.81=0.95
42.10+11.27
11.52:40.37
8.61
9,99::0449
7.93:10770
GMED-ER
81.71=1877
42.65:11.27
11.86:1:.36
9.16=0..7
9,53::0.66
8,1420.58
GMED MIR
88.70.20.81
44.53:12.23
11.58.1:0.51
8.48:10.37
9.24110.53
7.75.10.80
CoPF
87.58-00665
47.36-+-0.96
10.79:40..3
9.11=0..4
11.03-0.688
9.9240.61
ASER,
43,50=+1.44 ,40
14.00::0:40
12,20=
Ours
88.79=0.26
51.84+0091
15,56=20,39
13.65+00.55
16.05=0038
15.15=+0.66
Lid. online
86.35:00.64
62.37
20.62-+-0.48
20.62-1-0.48
18.02:00.63
18.02-4-0.63
iid. offine
92.44-00.61
79.90+t0.51
45.59:40.29
45.59440.29 29
38.63-20.59
38.634+0.59
Table 1: Average Accuracy of 15 runs on Splir datasets. Higher is better. indicates the results are from the
original paper.
MNIST
CIFARIO
CIFAR100
CIFARI00
minilmageNNtt (10-task)
minilmageNNt (20-tnsk)
Methods
(5-task)
(5-task)
(10-task)
(20-task)
fine-tune
99.24:20.09
85.45:0.63
51.60:0.77
65.51:0.78
41.12:20.2
52.99::0.89
ER-reservoir
18.33-=1.77
52.72+1
45.94440.55
57.31+0.71
36.05--0.88
47.70:+0.90
A-GEM
89.90=2.02
82.80=0.73
54.15:0::22
67.61+0053
43.31=0.52
54,47=00.78
GSS- SSS-Crredy
15.13:20.99
49.96112.82
44.30:10.57
53.87100544
36.17:1..58
45.91
MIR 9,71.1.1.39
44.34112.65
46.52:10.52
56.58-10.2
3..98
1..84
GMED-ER
16.21
44.93-t
46.35-+0.50
57.76-+0.94
15.22-11116
45.08++1.25
GMED-MIR
12.52=1.05
39.88++2.23
46.56:20.65
58.1440055
34.79=1001
45.50=
CoPE 9.51
40.01:1.880
36.51:10086
43.82:10.62
29.43:10.98
10.9911.02
ASER,
47.90+ .60
45.00:10.70
28. 28.00=1.33
Ours
9.36110.37
35.37 1.35
21.79.10.69
27.10.11110
21,26 0.59
1.088
Table 2: Average Forgetting of 15 runs Split datasets. Lower is better. indicates the results are from the original paper.
In Table I. we can find our method outperforms all baselines on all 6 settings. The improvement of our method is significant except on MNIST, where GMED-MIR is competitive with our method. An interesting phenomenon all existing methods do not have a substantial improvement over ER-reservoir on CIFAR 100 and minilmageNNt, except ASER. We argue for online CIL. problem, we should pay more attention to complex settings. Nevertheless, our method is superior to ASER obviously, especially on CIFAR10 and minilmageNet. T shows the forgetting of our method is far lower than other methods based on the softmax classifier, except on MNIST. Figure 3 shows our method is almost consistently better than all baselines during the whole learning processes. More results with various memory sizes can be found in Appendix.
Ablation Study We also conduct ablation study about training objective and classifier in Table 4 The ER reseeeevvvir corresponds to the first row and our method corresponds to the last row. Firstly, we find for ER-reservoir, replacing softmax classifier with NCM classifier makes substantial improvement on CIFAR10. However, it has no effect on more complex CIFAR 100 (row 1&2). Secondly, only using MS loss works very well on CIFARIO 10 while on CIFAR 100 poor ability to learn from new data limits its performance (row 3&6). Lastly, when hybrid loss is used, the NCM classifier is much better than proxy-based classifier (row 5&6), and MS loss critical for NCM classifier (row 4&6). Note that hybrid loss does not outperform MS loss much on CIFARCI This is because in Split-CIFA CIFAR 10, a minibatch of new data contains maximum of two classes, and thus the positive pairs are enough fo MS loss to learn new knowledge well. These results verify our statement in Section 3.2.
Comparison with Logits Bias Solutions for Conventional CIL Setting Although existing CIL. methods to alleviate logits bias are not applicable for online CIL settings as task boundaries are necessary, after being modified some ways they can be adapted to online CIL. To better reflect the contribution of our method, we adapt iCaRL (Rebuffi et al. 2017) and BiC (Wu et al.) 2019) to online CIL and compare modified iCaRL and modified BiC with our method in Table 3 replaces softmax classifier with NCM classifier and BiC uses a linear bias correction layer to reduce logits bias. and iCaRL is modified in the following way: at each iteration, we minimize binary CE loss used by iCaRL which encourages the model to mimic the outputs for all learned classes of the
iCaRL
8

---

Under review as a conference paper at ICLR 2022
02 Gren)
MNIST
CIFARIO
CIFARI00
minilmageNet (20-assk)
Methods
(5-task)
(5-task)
(20-lask)
modified
34.58
29.77
5.38
7.16
1CaRL
11.18
modifted
83.33
43.65
8.72
791
BiC
11.35
88.79
51.84
13.65
15.15
Ours
+0.26
+0.91
+0.35
+.036
(aL
Table 3: The comparison between modified iCaRL, modified BiC and method (Ours) Split datasets.
Figure 4: Comparison of training time (a) and test time (b) Split CIFARIO0 and minilmageNet. The number of tasks 10.
Training
Inference
CIFARIO
CIFAR100
CE loss
Sottmax (Dis)
39.88:11522
8.9520.26
CE loss
NCM (Gen)
44.46.10.95
8,96::0.38
Method
CIFARIO
CIFARI00
minilmageNet
MS loss
NCM (Gen)
51.72++1.02
9.99-0032
fine lune
10,02.10.03
1.02:0.033
1.02.10.04
PNCA loss
NCM (Gen)
41.91:11.78
9.31
RR-ressrvviir
20.89+42.07
3.8440.42
6.854+0.00
Hyhrid
Proxy (Dis)
48.16-+1.21
7.02+40.75
MIR GMED-MIR
18.75+42.53
4.35++0.53
1.09:t1004
Hybrid loss
NCM (Gen)
51.8440.91
13.65=00.35
18.78:42.31
3.68:20.48
7.22:40.81
Ours
34.18:10.81
10.54:0.38
12.24:00.19
Table 4: Ablation study on Split CIFARIO and 20-
M.d online
31.234-2.11
18.08+00.62
17.23-40.42
task Split CIFARI00, We show the performances of different combinations of losses and inference ways.
iid offline
48.37::1.23
42.6840.37
39.82-20:46
Dis: Discriminative, Gen: Generative.
Table 5: Final accuracy of 15 rums on Smooth datasets.
old model after the last iteration. We use reservoir sampling for memory update. NCM classifier is used for inference. For modified BiC, we use the linear bias correction layer of BiC to correct the logits for all learned classes only before test task boundaries As shown in Table modified iCaRL. performs very badly and the performances of modified BIC much worse than method. These results imply adapting existing methods online setting can not alleviate logits bias effectively for online CIL. Therefore, our method makes a substantial contribution for online CIL.
unavailable in online CIL setting.
Time Comparison In Figure 4a of our method only bit higher than ER-reservoir.
we report the training time of different methods. The training time
Most baselines, such as GSS, MIR, and GMED, improve ER-reservoir by designing new memory update and sampling strategies which depend on extra gradient computations and thus are time- consuming. The inference costs of softmax classifier and our NCM classifier are displayed in Figure 4b] We can find the extra time of NCM to compute the class means is (about 3%) slight, as the size of memory is limited.
4.3 RESULTS ON TASK-FREE Smooth DATASETS
In newly designed task-free Smooth datasets, the new classes emerge irregularly and the distribution on class changes at each time step. In Table we compare our method with several baselines on three smooth datasets. The metric is final accuracy after learning the whole data stream. We can find these datasets are indeed more complex as fine-tune can only classify correctly on the last class, which is due to the higher imbalance of data streams. For this reason, basclines such as ER-reservoir and MIR degrade obviously compared with split datsets. However, our method performs best consistently.
CONCLUSION
In this work, we tackle with online CIL problem from a generative perspective to bypass logits bias problem in commonly used softmax classifier. We first propose to replace softmax classifier with generative classifier. Then we introduce MS loss for training and prove theoretically that optimizes the feature space in generative way. We further propose hybrid loss boost the model's 's ability to learn from new data. Experimental results show the significant and consistent superiority of our method compared to existing state-of-the-art methods.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Hongjoon Ahn and Taesup Moon. A simple class decision balancing for incremental learning. CoRR, 13942 2020. URL https//arrav.ag//////00.....1
Rahaf Aljundi, Bugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Cac- cia, Min Lin, and Lucas Page-Caccia. Online continual learning with maxi-
mal interfered retrieval. In NeurIPS, 2019a. URL http://papers.ipssccoppap
coneline-contiuuall--ee learmingg
maaximll-mnterffreee
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In CVPR,
2019b. doi: 10.1 2..1109/CVP..2099.011111 URL http:://ppnacceess.. www.::tteweewcoooooooooooo www//oonenttttcceceeoeommmmme CVPR 2019/ html/Aljuni Task- Free Continual Learning CVPR 2019 |
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample se-
lection for online continual learning. In NeurlPS, 2019c. URL http://papers.missccopppp 9354-gradient faased-sampl wewpple--seeetoon-foooeo for- coniinall continual-leariing
Eden Belouadah and Adrian Popescu. IL2M: class incremental learning with dual memory. In ICCV, 2019.
Guillaume Bouchard and Bill Triggs, The tradeoff between generative and discriminative classifiers. 2004.
Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed. A unifying mutual information view of metric learning: Cross-entropy vs. pairwise losses. In ECCV, 2020. doi: 10.
URL https:///ooi.rr//(0))
1007/978-3-030-555533
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In ICLR, 2019a. URL htt////s///pppeeeeeeee/////ooooo www.ne///ooommiidiii.....
sC5FX
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Mare' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint 190. 2019b.
Xiaoan Ding, Tianyu Liu, Baobao Chang, Zhifang Sui, and Kevin Gimpel. Discriminatively- tuned generative classifiers for robust natural language inference. In EMNLP, 2020. URL
http::/www.advebboggaabbb/////od
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. CoRR, waap///71100700000 2017.
Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, 2006.
Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In ICLR Workshop, 2015. URL http://arrxivorr//bb///aa 111..//////22.2222222
Xisen Jin, Junyi Du, and Xiang Ren. Gradient based memory editing for task-free continual learning. CoRR, abs/2006.11 15294, 2020. URL http:://arri..or/aab/200....... 15294
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, al. Overcoming catastrophic forgetting in neural networks. PNAS, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Matthias De Lange and Tinne Tuytelaars. Continual prototype evolution: Learning online from non-stationary data streams. CoRR, abs/2009.0099 2020. http:///ss//aaiv.oor/aaa/////// (0919
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh, and Tinne Tuytelaars. Continual learning: A comparative study on how to defy forgetting in classification tasks. CoRR, abs/1999.08888, 2019. URL http://aarki.oryyaa/////8855335
10

---

Under review as a conference paper at ICLR 2022
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, 86(11))2278-23 -22244, 1998.
Soochan Lee, Junsoo Ha, Dongsu Zhang, and Gunhee Kim. A neural dirichlet process mixture model for task-free continual learning. ICLR, 2020. URL http:://pppomeeeee...rroooo/osss
Zhizhong Li and Derek Hoiem. Learning without forgetting. TPAMI, 2018.
Zheda Mai, Dongsub Shim, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongseong Jang. Online class-incremental continual learning with adversarial shapley value. AAAI, 0bs//209.00093, 2021.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to single network by iterative pruning. In CVPR, 2018
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. Elsevier, 1989.
Thomas Mensink, Jakob J. Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-based image classification: Generalizing to new classes at near-zero cost. TPAMI, 35, 2013. doi: 10.1 10.11099/PAMM.2013..8. URL https::/doi.org/10.1111 I0. 109.1119//A
Yair Movshovitz-Attias, Alexander Toshev, Thomas K. Leung, Sergey Ioffe, and Saurabh Singh. No fuss distance metric learning using proxies. In ICCV, 2017.
Andrew Y. Ng and Michael I. Jordan. On discriminative generative classifiers: A compari- son of logistic regression and naive bayes. In NIPS, 2001. URL httt:://papers. wwwp:immmssssppppeeeee
naiieei naive- bayes
German Ignacio Parisi, Ronald Kemker, Jose L., Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In ICLR, 2019. URL U httpss//wooarrreeewww/////////// TShhAtt
Anthony V. Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connect. Sci., 7(2), 1995. doi: 10. 1011800055400995500333333
URI. http://///h..a////////////////10
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NIPS, 2017.
Gido M. van de Ven and Andreas S. Tolias. Three scenarios for continual learning. In NeurIPS Continual Learning workshop, 2018
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for one shot learning. In Advances in Neural Information Processing Sys- tems 29: Annual Conference on Neural Information Processing Systems 2016. Decem- ber 5-10, 2016, Barcelona, Spain, pp. 3630-3638, 2016. URL http:///apprr..ips.cc/appppp 63:85-machingg-newwaa netpainnnnnetoooo-f--oooooo wtwrss--oo-oo-oso---eeiiiiii shot-le
Jeffrey S. Vitter. Random sampling with reservoir. ACM Transactions on Mathematical Software (TOMS), 1985.
Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew R. Scott. Multi-similarity loss with general pair weighting for deep metric learning. In CVPR, 2019. doi: 10. 10.1 10.1109/CVPP.201 00516. URL http://ppaaaccess..eccccc.oooooo
CVPR 2019/html/ /ang Multi Similarity Loss_ With General Pair Weighting for Deep Metrie Learning CVPR 2019 1at__ppperhtml
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR, 2019.
11

---

Under review as a conference paper at ICLR 2022
Dani Yogatama, Chris Dyer, Wang Ling, and Phil Blunsom. Generative and discriminative text classification with recurrent neural networks. CoRR, atbs///03.00898, 2017. URL http::////aar
abs/ 10s//103301188
Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In CVPR, 2020.
Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task agnostic continual learning using online variational bayes. arXiv preprint arXiv: 1803. 10123, 2018.
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairness in class incremental learning. In CVPR, 2020.
A PROOFS
A.1 PROPOSITION
Proof. For simplicity, we ignore the coefficient both in Las and LGen-Bin Bin in the proof. We denote the two parts in the summation of Lass as Lis and LMs respectively, i.e.:
MMS
1
log(1 + Σ 0a(88 211
jEP;
(9)
LMS
log
B(1)))))]
JEN,
Without hard mining, for Lits we have:
CMs
log[1 Σ
ca(Sis A)1
j:
logl Σ
e a -X1]
i=
A1]
T1g
(10)
> Jensen's inequality
Σ
- (Si1 X)]
170
j:M5=
C
Σ
Sij
110
12

---

Under review as a conference paper at ICLR 2022
where e stands for equal to, up to an additive constant. For LMs we can write:
Ms
log|1
+
Σ
((S A1 Ay
logl
Σ
(S
(11)
log e [(8
no - 1) j:
Jensen's inequality
Σ
Sij
RC 11)
According to Eq 10 and Eq 11
we have:
LMS
Σ
Sij +
Σ
(12)
Y40
(C
1)mo ju
t=1
Now we consider Laen- Bin Firstly, it can be written:
LGen- Bin
logp(z;ly= = v.)
(13)
Σ
logplz;ly= =))
c1..cg:
For convenience, we denote the features of data samples whose labels are e as (2 For the first part in the right hand of Eq 13 we have:
logp(zz/y vi)
||z - 1221
> p(zly= = = (izlu=c)-NNE ~N(2) I)
2
i=1
no
Icc 11
1=1
no
[|| 112 -2 1 + 11 112)
(14)
2n 1r. 11 +
+00+noll4 2011142 112)
D |=1 =1
140 10 212
e=
no
8g
2ec Zc3
140
Σ
Σ
Sij
2no
13

---

Under review as a conference paper at ICLR 2022
Keep in mind that the mean of class uel no 2i=1 Ze. For the second part in the right hand of Eq 13
1.
we have:
logp(z:ly c)
1||
11:1-1..1
e=
12:22112 +22, ge
D
(15)
C
22, Me
TT
- 11)m0 Σ 1u
D
11 =1
2
1=1
1
2
C-1
Σ
C no 710
cl ze:
2
==1
1
C-1
Σ
S4
Σ
Syp
14g
2no
J:8: =
According to Eq
14 and Eq
115)
we have:
LG Bin
1
Σ
Sij + (C - 1) f:
Σ Sig)
(16)
1=1
According to Eq
12 and Eq
16). we can obtain:
LMS 2 LGen-Bin Bin
(17)
B DETAILS ABOUT EXPERIMENT SETUP
B. SPLIT DATASETS
In Table we show the detailed statistics about Split datasets. On MNIST (LeCun et al. 1998). we ran- domly select 500 samples of the original training data for each class as training data stream, following previous works (Aljundi et al. 2019a Jin et al. 2020p. On CIFAR10 10 and CIFAR 100 (Krizhevsky jtt al. 2009). We use the full training data from which 5% samples are regarded as validation set. The original minilmageNet dataset is used for meta learning (Vinyals et al. 2016) and 100 classes divided into 64 classes, 16 classes, 20 classes respectively for meta-training meta-validation and learning. There
meta-test respectively. We merge all 100 classes to conduct 600 samples per class in minilmageNNet. We divide 600 samples into 456 samples, 24 samples and 120 samples for training, validation and test respectively. We do not adopt any data augmentation strategy.
B.2 SMOOTH DATASETS
For data stream with C' classes, we assume the length of stream N and no N. We denote (t) as the occurrence probability of class at time step t and assume po(t) _N(t) (2c- (0tt:(221))) 19m6 each time step f, we calculate p(t) = (p1 (t),... (t))) and normalize p(t) as the parameters of a Categorical distribution from which class index ct is sampled. Then sample without replacement. In this setting, data distribution changes smoothly and there is no notion of task. We call such data streams Smooth datasets.
ne At
data of class
Here we show the characteristics of Smooth datasets in detail. In Table we list the detailed statistics of Smooth datasets. Due the randomness in the data stream generation process, the number of
14

---

Under review as a conference paper at ICLR 2022
Dataset
MNIST
CIFARIO
CIFAR100
minilmageNet
Image Size
(1,32,32)
(3,32,32)
(3,32,32)
(3,84,84)
#Classes
10
10
100
100
# TTain Samples
5000
47500
47500
45600
# #Valid Samples
10000
2500
2500
2400
#Test Samples
10000
10000
10000
12000
#Task
10 20
10 or 20
#Classes per Task
2
2
10 or
10 or
Table 6: Details about Split datasets.
samples of each class is slightly imbalanced. On CIFARIO0 and minilmageNet we set the mean 400 to avoid invalid sampling when exceeding the maximum
number of samples of each class number of sample of one class in the original training set. The last rows of Table] 7 show the range of number of samples of each classes in experiments. For example, in repeated runs, 15 different Smooth CIFAR 10 data streams are established. The number of samples each class is in the interval [464, 536]. It should be emphasized that in all experiments the identical 15 data streams for fair comparison.
the same random seed to obtain
Some existing works (Zeno et al. 2018 Lee al. 2020) also experiment task-free datasets where a data stream is still divided into different tasks but the switch of task is gradual instead of abrupt. In fact, in their settings, task switching only occurs in part of time, so that in the left time the distribution of data streams still i.i.d. In contrast, in our Smooth datasets the distribution changes at the class level which simulates task-free class-incremental setting. In addition, the distribution of data streams is never i..d, which can reflect real-world CL scenarios better.
plot the curves of training loss of ER-reservoir Split CIFARIO and Smooth CIFAR10. When new task emerges (each task consists of 100 iterations), the loss on Split CIFARIO increases dramatically. Thus some methods can detect the change of loss inference the task boundaries although they are not informed and conduct additional offline training phase (Aljundi) jtt al. 2019b). However, such a trick is not applicable on Smooth datasets therefore only the real online CL methods can work.
In Figure
Dataset
CIFARIO
CIFAR100
minilmageNet
#Classes (C)
10
100
100
#Train Samples (length of data stream) (n)
5000
40000
40000
# Valid Samples
2500
2500
2400
#Test Samples
10000
10000
12000
Mean number of samples of one class (no)
500
400
400
Minimum number of samples of one class
464
357
354
Maximum number of samples of one class
536
450
446
Table 7: Details about Smooth datasets.
B.3 HYPERPARAMETER SELECTION
In our method, although there are several hyperparameters LMS and CPNCA, we only need to tune which is the weight of LPNCA in L The value of other hyperparameters in L is fixed as stated in the main text. We select y from [o, 0.1, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 2.0). Especially,
7 makes ybrid become LMs.
We follow previous works to use SGD optimizer in all experiments. However, we find compared to CE loss, wider range of learning rate should be searched in for DML losses. Previous works are based
on CE loss and always set 77 = 0.05 or 17 = 0.1 (Aljundi et al. 2019c a Tim et al. 2020 Mai et al. 2021). We find the optimal rn for LHybrid is often larger than 0.1. However, when larger 7) (e.g. 0.2 on MNIST and CIFAR10) is used, baselines based on CE loss will degrade obviously because the unstable results multiple runs. Thus, for baselines select 7) from (0.05, 0.1, 0.15, 0.2]. For
15

---

Under review as a conference paper at ICLR 2022
Split CIFAR10
Smooth CIFAR10
20
16-
LOG0
lteration
Iteration
Figure 5: The curves of training loss on Split CIFARIO (left) and Smooth CIFARIO (right). Split CIFARIO comprises of 5 tasks, 100 iterations per task.
method, we select from 10.05, 0.1. 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5]. The hyperparameters of our method are displayed Table
Split Datascts
Smooth Datascts
MNIST
CIFARIO
CIFARI00
minilmageNet
CIFARIO
CIFAR100
minilmageNet
0.1
0.1
0.5
0.1
1.0
1.0
0.05
0.2
0.35
0.2
0.25
0.5
0.2
Table 8: Hyperparamters of PNCA loss weight and learning rate 17 for our method on different datasets.
B.4 EVALUATION METRICS
Following previous online CL. works (Aljundi et al. 2019a Mai et al., 2021 we use Average Accuracy and Average Forgetting (Chaudhry al. (2019b) on Split datasets after traiming all T tasks, Average Accuracy evaluates the overall performance and Average Forgetting measures how much learned knowledge has been forgetten. Let ag,j denote the accuracy on the held-out of the j-th task after training the model on the first & tasks. For dataset comprised of T tasks, the Average Accuracy defined as follows:
AT
AT
(18)
T
J=1
and Average Forgetting can be written as:
f7
max
ai - DT Vj <T
(19)
1e(1..,, --13
T-1
Fr T Σ
E
(20)
We report Average Accuracy and Average Forgetting in the form of percentage. It should be noted that higher Average Accuracy is better while lower Average Forgetting is better.
On Smooth datasets, as there is no notion of task, we evaluate the performance with the accuracy on the whole held-out after finishing training the whole training set.
B.5 CODE DEPENDENCIES AND HARDWARE
The Python version is 3.7.6. We use the PyTorch deep learning library to implement our method and baselines. The version of PyTorch is 1.7.0. Other dependent libraries include Numpy (1.17.3), torchvision (0.4.1), matplotib (3. and scikit-leam (0.22). The CUDA version is 10.2. We run all experiments NVIDIA RTX 2080ti GPU. We will publish our codes once the paper is accepted.
16

---

Under review as a conference paper at ICLR 2022
C MORE EXPERIMENTAL RESULTS
C.I EFFECT OF PROXY-NCA Loss WEIGHT
In Table 6 show the effect of Proxy-NCA loss weight 7 split datasets. We can find on MNIST and CIFAR1O where the number classes is relatively small, although the best results are obtained with y 0.1, we have competitive results with y 0. In other words, only using MS loss is effective enough. However, on CIFAR 100 and minilmageNet, as discussed in main text, the expected number of classes in each minibatch is larger which reduces the probability of occurrence of positive pair so that limits the learning ability of pair-based MS loss. At this time, it is necessary to introduce auxiliary PNCA loss. When y is larger than the performance begins degrade, which implies that excessively focusing on discriminative PNCA loss will affect the performance of the generative NCM classifier.
Split MNIST
Split CIFAR
Spilt CIFAR100 (10-task)
52.6
13.0
23.0
-0.0
40.0
11.0
220
874
Split (20-task)
Split magelet (10.task)
Split minilmageNeet (20-task)
16.0
110
125
14.0
22.5
32.0
10.F
181
8.5
Figure 6: Average Accuracy valid set of method with different Proxy-NCA loss weight yy the Split datascts as reported in Table main text. We report the mean 15 rums.
Split
Split CIFAR10
Split CIFAR100 (1 (10-task)
En-reservoir
26-3
43.0
38.0
35.0
38.0
102
250
1866
106
Split CIFARIOO task)s)
Split minilma geNt (10 task)
Split mageltt (20
26.0
Vald TOCY
12.0
Vate
Vale
80
18
360
3003
2000
5000
566
3268
2000
5000
360
1000
2000
5000
Hemory
Hemory
Memory
Figure 7: Average Accuracy on valid set of re-rescrvorr and our method with different size of M.
17

---

Under review as a conference paper at ICLR 2022
C.2 RESULTS WITH DIFFERENT MEMORY SIZES
In Table 9 and Table 10. we report the performance of our method and all baselines compared in main text on Split CIFAR 100 and minilmageNet with 2k memory size. Our method performs best in all settings and the improvements are obvious, which similar with the results in 1K memory size settings reported in Table 1-2 of main text. We also report Average Accuracy of ER-reservoir and our method on valid set with a various of memory sizes. Our method outperforms ER-rescrvoir consistently which shows broad applicability of our method. The improvements are relatively small when memory size is 500 on CIFAR 100 and minilmageNet, which is due to the fact that when size of replay memory M is too small (5 samples per class on average), class mean ue cannot approximate the real class mean e well and thus proposed NCM classifier degrades.
C.3 COMPARISON WITH TRIPLET LOSS
Yu et al. (2020) proposes SDC method for conventional CIL based on NCM classifier and another pair-based DML loss, triplet loss (Hoffer & Ailon 2015). As discussed in Related Work of main text, SDC is not applicable for online CIL. To further clarify our contribution given this work, we evaluate the performance NCM classifier after training the model with the hybrid objective Crriplet + CPNCCA where Crriplet represents triplet loss. Please see Hoffer & Ailon (0055) and Yuu et al. 2020) for details about LTriplet-
We report the results of the hybrid objective involving triplet loss and compare it with our hybrid loss (LMs + CPNCCA) in Table [1 We can find our proposed loss superior to Cr + CCNCA consistently. In addition to replace softmax classifier with NCM classifier for online CIL problem, our contributions mainly reflect in introducing a hybrid of MS loss and PNCA loss the view of training the generative NCM classifier. We believe the above results show MS loss is critical for online CIL, and introducing MS loss not a trivial contribution, even if given the SDC work YYu
et al. 2020
CIFAR100
CIFAR100
minilmageNet
minilmageNet
Methods
(10-task)
(20-task)
(10-task)
(20-task)
fine-tune
6.26+1030
3.61
0.4
3.12+00.15
ER-reservoir
15.15=0.37
12.76+00.99
13.88+10.68
11.76:10.88
A-GEM Chaudhry et al, 201 (0)
6.5010.16
3.61
4.41=0.14
3.23+0.13
ESS-Greedy (Aljundi al.
2019c)
12.66-10.67
11.62.10.11
13.95-10.40
11.70
MIR (Aljundi et al. 201 9a)
15.11
12.45
14.22-10.93
12.35
GMED-ER (Jin et al. 2020
14.93-0.39
11.97=0.60
12.10
9.90+10955
GMED-MIR Jin et al. 2020
15.11
12.01=0.81
13.90+0.55
12.25
ASER, Mai et al. 2021)*
17.20+0.50
14.80+11.10
Ours
18.96
16.6910.76
19.17
17.10
i.i.d. online
20.42
20.62:0.48
18.02-10.63
18.02-10.63
i.i.d. offline
45.59
45.5910.29
38.63-10.59
38.63
Table 9: Average Accuracy of 15 runs on Split datasets. Higher is better. original paper. The size of memory M is 2k.
indicates the results are from the
18

---

Under review as a conference paper at ICLR 2022
CIFARI00
CIFARI00
minilmageNNe
minilmageNet
Methods
(10-task)
(20-task)
(10-task)
(20-task)
fine-tune
0..60
65.51
41.12:10.82
52.99110889
ER-reservoir
40.19:1.71
10.85
32.91=10799
44.93-10.11
A-GEM (Chaudhry et al. 2019a)
51.45=0.68
67.13=0.55
40.49:10.3
51.45=0.68
GSS-Greedy (Aljundı et al. 2019c
40.02=0.81
45.90=2.09
30.50=10.83
39.09=2.34
MIR (Aljundi et al. 201 9a)
39.82-10.79
4..5440.60
29.27+1.32
39.74+1.50
GMED-ER (Jin et al
2020
02.5
55.45
31.81+1.73
43.68+1123
GMED-MIR Jun
2020
43.78=0093
55.15=0.71
29.67=0.58
41.16:00.5
ASER, (Mai et al. 2021)
38.60+0.60
22.20=11.60
Ours
26.5910.58
29.53
17.82.00.36
24.10
Table 10: Average Forgetting of 15 runs on Split datasets. Lower is better. original paper. The size of memory M is 2k.
indicates the results are from the
MNIST
CIFARIO
CIFAR100
CIFAR100
minilmugeNet (10-task)
minilmageNet (20-task)
Loss
(S-task)
(5-task)
(10-tusk)
(20-task)
Creiste CPNCA
87 0.552
43. 1..56.11.63
13.21 15.56.10.39
10.93.10.34 13.65:10.35
14.31.10333 16.05:10.38
13.55.10.22 15.15-10.36
Lis + CPNCA (Ours)
08.29
5184
Table 11: Comparison between MS loss and Triplet Loss. We report Average Accuracy of 15 runs on Split datasets. The size of memory M lk.
19


Reviews:
[{"review_id": "zvbvkHSw7x8", "reviewer": "Reviewer_hxsn", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is somewhat original and the text is relatively clear.\n\nThe authors describe the proposed approach in detail, and they present plenty of experimental results. The experimental section in general has convinced me that the proposed approach does indeed outperform the state of the art.\n\nThe authors argue that their method outperforms the state of the art because it optimizes the feature space in a generative way and thus enjoys the benefits of using a generative classifier. In combination with the NCM classifier, the authors propose a hybrid generative-discriminative loss. \n\nThe authors show that the NCM classifier can be interpreted as a very rudimentary generative model (i.e., an isometric Gaussian). This makes sense but it does not justify using a NCM classifier instead of a softmax. Moreover, the authors are never in this paper involved in actually generating latent-space data. Finally, the NCM classifier can also be interpreted as a discriminative model as it divides the latent space into Voronoi cells, hence modelling the decision boundaries between the different classes.\n\nRegarding the experimental results, the authors should probably compare their approach to the SCR method as well (see [1]).\n\nIn my view, the major downside of this paper is that it looks like an exercise in achieving better numbers than the baselines, instead of an attempt to document and transmit the knowledge gained from the underlying study. Having read the paper twice, I cannot say that I learned something new, apart from the fact that the proposed method outperforms the state of the art. In my opinion, the explanation provided as to why that is is only speculation. A paper like this needs to describe in detail assumptions about the specific reasons leading to superior performance and test whether these assumptions hold water. In short, I think this paper will benefit substantially by attempting to discover the underlying factors behind the improved performance.\n\n\\\nQuestions\n- If deep-metric-learning losses and the cross-entropy both maximize the mutual information between features and labels why would you expect different results?\n- Can you explain what do you mean by stating that the MS loss optimizes the feature space in a generative way?\n- What is the motivation behind selecting only 5k training instance for split MNIST? The paper you referenced cites two papers none of which uses the split MNIST benchmark.\n- By comparing rows 3 and 6 of Table 4 it seems to me that using the hybrid loss instead of only the MS loss is only beneficial in the case of CIFAR-100. Why is that?\n\n\\\n[1] Mai, Z., Li, R., Kim, H., & Sanner, S. (2021). Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning. In Workshop on Continual Learning in Computer Vision at Conference on Computer Vision and Pattern Recognition (CVPR), 2021", "summary_of_the_paper": "The authors attempt to improve online continual learning (CL) performance by eliminating the logit bias of the classifier used. They use a nearest-class-mean (NCM) classifier and a multi-similarity metric learning loss coupled with an auxiliary loss to achieve a good plasticity-stability balance.", "summary_of_the_review": "In my view, the paper fails to adequately explain why the proposed method achieves the results it does, and there is a relevant baseline missing. Hence, I recommend rejection. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "uuiBiazV0ZD", "reviewer": "Reviewer_TwHY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n-\tIdentifying generative classification as a promising alternative to softmax-based discriminative classification, and demonstrating that nearest-class mean classification (NCM) can be interpreted as generative classification.\n-\tProposing a task-free solution for updating a feature extractor for NCM classification.\n-\tProposing an interesting new benchmark for online continual learning that might be of interest to others working on this problem (i.e., the “smooth datasets”).\n-\tThe reported performance of the proposed method is substantially better than the existing methods that are compared against.\n\n\nWeaknesses:\n-\tMy main issue with this paper is that its two main contributions (i.e., [1] proposing generative classification for class-incremental learning and [2] proposing an online variant of NCM classification for class-incremental learning) have been recently proposed by two other papers, but this paper does not discuss, compare against or cite either of them. Specifically, Van De Ven et al. (2021, CVPR-W; https://arxiv.org/abs/2104.10093) argued for addressing class-incremental learning using generative classification, and De Lange & Tuytelaars (2021, ICCV; https://arxiv.org/abs/2009.00919) proposed an online version of an NCM-based class-incremental learning method.\n-\tAnother relevant paper that would have been good to discuss (and compare against) is Lomonaco et al. (2020, CVPR-W; https://arxiv.org/abs/1907.03799), which also proposes an online class-incremental learning method that addresses the logits bias problem.\n-\tIt is unclear to me exactly what the authors mean by “online continual learning”. Does it mean that there are no clear task boundaries, or does it mean that task boundary information is not provided to the network? At the top of section 4.1, it is stated that “task boundaries are not informed during training”. However, it seems to me that because class labels are provided, task boundaries can trivially be derived? It is also unclear to me why task-incremental learning cannot be performed in the online setting? (As claimed in the introduction.)\n-\tI’m not sure how relevant the motivation of generative classification as a more efficient method in the low data regime is for this paper. My feeling is that for all problems considered in this paper, if the data is provided to the algorithm in an i.i.d. stream, the discriminative classifier would perform better than the generative classifier.\n\n\nMinor comments:\n-\tAn accuracy of 15% on CIFAR-100 or miniImageNet, although perhaps better than the compared methods, is not of any practical value for real-world applications. Perhaps this performance improvement is of interest from an academic perspective, but it does not seem to be of value for any practical applications. I think it would be good to discuss this.\n-\tWhat is meant by “virtual catastrophic forgetting”? It would be good to expand a bit on that.\n", "summary_of_the_paper": "To address the problem of logits bias in class-incremental learning with deep neural networks, this paper proposes to replace the typically used softmax output layer by a nearest-mean classifier (on the feature space), which can be interpreted as a generative classifier. To train the feature attractor in an online setting (i.e., without reliance on task boundaries), the paper proposes a hybrid loss consisting of a multi-similarity loss (which can be interpreted as an approximate generative objective) and an auxiliary “proxy” loss. The paper reports substantial performance improvements on a number of computer vision benchmark (including ones with natural images) over a selection of existing methods.\n\nAdditionally, the paper proposes a new “smooth dataset” benchmark for online class-incremental learning, whereby the transitions between classes are smooth rather than based on abrupt task transitions.\n", "summary_of_the_review": "As explained above, my main issue with this paper is that its two main contributions are not as novel as suggested in the paper, as similar contributions were recently made by three other papers (https://arxiv.org/abs/2009.00919 and https://arxiv.org/abs/2104.10093 and https://arxiv.org/abs/1907.03799). There are some differences with these papers, but I think it is necessary that this paper clearly discusses what it contributes on top of these papers. When that is done, it should be better possible to judge the additional contribution of this paper.\n\nMy current scores are my “expectations” for this paper after a discussion of the above papers (and corresponding moderation of some novelty claims) has been incorporated. If the authors can make a more convincing case that their paper makes important contributions on top of those made by these previous papers, I’d be happy to increase my score. \n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "UF_FB2ByUl", "reviewer": "Reviewer_6SSs", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n- S1: Providing a theoretical argument that the minimizing MS loss is optimizing a generative model with the assumption of uniform distribution of $p(y)$.\n- S2: Computational overhead due to the proposed method is negligible thus practically useful.\n- S3: Pretty compelling accuracy gain in CIFAR10 (5-task), CIFAR-100 (20-task), miniImageNet (10/20-task).\n\n**Weaknesses**\n- W1: The proposed method is a combination of existing loss. The novelty of technical contribution is not very strong.\n- W2: The proposed hybrid loss is argued that it is beneficial as the Proxy-NCA loss will promotes learning new knowledge better (first paragraph in Sec. 3.4), rather than less catastrophic forgetting. But the empirical results show that the proposed method exhibits much less forgetting than the prior arts (Table. 2). The argument and the empirical results are not well aligned.\n\t- Also, as the proposed method seems promoting learning new knowledge, it is suggested to empirically validate the benefit of the proposed approach by a measure to evaluate the ability to learn new knowledge (e.g., intransigence (Chaudhry et al., 2018)).\n- W3: Missing important comparison to Ahn et al.'s method in Table 3 (and corresponding section, titled \"comparison with Logits Bias Solutions for Conventional CIL setting\").\n- W4: Missing analyses of ablated models (Table 4). The proposed hybrid loss exhibits meaningful empirical gains only in CIFAR100 (and marginal gain in CIFAR10), comparing \"MS loss with NCM (Gen)\" and \"Hybrid loss with NCM (Gen)\". But there is no descriptive analysis for it.\n- W5: Lack of details of *Smooth* datasets in Sec. 4.3.\n- W6: Missing some citation (or comparison) using logit bias correction in addition to Wu et al., 2019 and Anh et al., 2020\n\t- Kang et al., 2020: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9133417\n\t- Mittal et al., 2021: https://openaccess.thecvf.com/content/CVPR2021W/CLVision/papers/Mittal_Essentials_for_Class_Incremental_Learning_CVPRW_2021_paper.pdf\n- W7: Unclear arguments or arguments lack of supporting facts\n\t- 4th para in Sec.1\n\t\t- 'It should be noticed that in online CIL setting the data is seen only once, not fully trained, so it is analogous to the low data regime in which the generative classifier is preferable.'\n\t\t\t- Why?\n\t- 5th line of 2nd para in Sec. 3.1.3\n\t\t- 'This problem becomes more severe as the the number of classes increases.'\n\t\t\t- Lack of supporting facts\n- W8: Some mistakes in text (see details in notes below) and unclear presentations\n\n**Note**\n- Mistakes in text\n\t- End of 1st para in Sec.1: intelligence agents -> intelligent agents\n\t- 3th line of 1 para in Sec. 3.2: we can inference with -> we can conduct inference with\n\t- 1st line of 1st para in Sec. 3.3\n\t\t- we choose MS loss as training... -> we choose the MS loss as a training...\n\t\t- MS loss is a state-of... -> MS loss is the state-of...\n\t- 6th line of Proposition 1: 'up to an additive constant and L' -> 'up to an additive constant c and L'\n- Improvement ideas in presentations\n\t- Fig. 1: texts in legends and axis labels should be larger\n\t- At the beginning of page 6: Proposition (1) -> Proposition 1. --> (1) is confused with Equation 1.\n\t- Captions and legend's font should be larger (similar to text size) in Fig. 2 and 3.", "summary_of_the_paper": "The paper proposes a hybrid loss of generative classification by NCM as a form of Multi-Similarity (MS) loss (Wang et al., 2019) and discriminative classification by Proxy-NCA (Movshovits-Attias et al., 2017). The paper contains a theoretical argument that the MS loss upper-bounds the 'Gen-Bin loss', which is a generative classification model. The hybrid loss shows decent gains in 'average accuracy' (Table 1) and large gain in 'forgetting' (Table 2) in the experiments with multiple datasets including MNIST, CIFAR-10/100, miniImageNet with smaller computational complexity than the prior arts (Fig. 4).", "summary_of_the_review": "Despite the benefits of the theoretical arguments and somewhat convincing results, lack of novelty and missing details prevents this paper from being a quality to be accepted now.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MFa_jueGmd", "reviewer": "Reviewer_aZyF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n•\tThe paper addresses an important and challenging problem, i.e., online continual learning.\n•\tThe motivation is clear.\n•\tThey conducted extensive experiments to show the consistent superiority of proposed method compared to existing state-of-the-art methods.\nCons:\n•\tThe hypothesis of Proposition 1 is derived in the case of supervised learning and seems to have few connections with continual learning. Why the multi-similarity loss alleviates catastrophic forgetting in continual learning?\n•\tThe hypothesis of this paper is based on that the feature is well discriminative. However, the feature extractor is usually underfitting in online setting. It is difficult to disentangle the effects of catastrophic forgetting from underfitting.\n•\tIf the memory size is very small, the reserved samples of old classes are apt to be lost, how to obtain a generative classifier?\n•\tIt is not suitable to train the model 5 epochs on i.i.d offline setting for almost datasets.\n", "summary_of_the_paper": "This work develops a novel generative framework to bypass logits bias in online continual learning. They utilized a multi-similarity loss to weight feature pairs better during training time. At test time, they leveraged reserved samples to generate a generative classifier instead of a softmax classifier. Furthermore, they introduce an auxiliary loss to improve the ability of multi-similarity loss to learn from new data.", "summary_of_the_review": "The motivition is clear, and experiments are comprehensive. \nHowever, some unclear explanations and analyses are existed.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents an approach for online continual learning where only a single pass over each task's data is allowed. Instead of the oft-used softmax classification setting in continual learning, the paper proposes to use the generative setting based on the nearest class mean (NCM). The paper claims that it avoids the logits bias problem in the softmax classifier and helps combat catastrophic forgetting.

While the reviewers found the basic idea interesting, there were concerns about novelty and lack of clarity regarding the reasons for improved performance. In particular, there are several aspects from existing work that are leveraged in this paper (e.g, replay, metric learning loss, combination of generative and discriminative classification, etc) but the paper lacks in establishing which of these components affect the performance and in what ways.

The authors and reviewers engaged in detailed discussions; however, the reviewers were still unsatisfied and did not change their assessment. Based on my own reading of the paper as well as going through the reviews and discussions, I too concur with their assessment. It would be a stronger paper if the paper could shed more light on the above aspects as well as address the other concerns raised by the reviewers. However, in the current shape, it is not ready for publication.
Decision: Reject