Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning
Authors: Wang Xinrui
Chuanxing Geng
Wenhai Wan
Shao-Yuan Li
Songcan Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: online continual learning
data stream
model throughput
Primary area: machine_vision
TL;DR: Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning

Abstract: Online continual learning (OCL) requires the models to learn from constant, endless streams of data. While significant efforts have been made in this field, most were focused on mitigating the \textit{catastrophic forgetting} issue to achieve better classification ability, at the cost of a much heavier training workload. They overlooked that in real-world scenarios, e.g., in high-speed data stream environments, data do not pause to accommodate slow models. In this paper, we emphasize that \textit{model throughput}-- defined as the maximum number of training samples that a model can process within a unit of time --  is equally important. It directly limits how much data a model can utilize and presents a challenging dilemma for current methods. With this understanding, we revisit key challenges in OCL from both empirical and theoretical perspectives, highlighting two critical issues beyond the well-documented catastrophic forgetting: (\romannumeral1) Model's ignorance: the single-pass nature of OCL challenges models to learn effective features within constrained training time and storage capacity, leading to a trade-off between effective learning and model throughput; (\romannumeral2) Model's myopia: the local learning nature of OCL on the current task leads the model to adopt overly simplified, task-specific features and \textit{excessively sparse classifier}, resulting in the gap between the optimal solution for the current task and the global objective. To tackle these issues, we propose the Non-sparse Classifier Evolution framework (NsCE) to facilitate effective global discriminative feature learning with minimal time cost. NsCE integrates non-sparse maximum separation regularization and targeted experience replay techniques with the help of pre-trained models, enabling rapid acquisition of new globally discriminative features. Extensive experiments demonstrate the substantial improvements of our framework in performance, throughput and real-world practicality.

Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning
Xinrui Wang 1.2
Chuanxing Gengl.2 Wenhai Wan Shao-yuan Li
Songean Chen 1:2
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics
MIIT Key Laboratory Pattern Analysis and Machine Intelligence
3 School of Computer Science and Technology, Huazhong University of Science and Technology
Abstract
Online continual learning (OCL) requires the models to learn from constant, end- less streams of data. While significant efforts have been made in this field, most were focused on mitigating the forastruphicforgeeinng issue to achieve better clas- sification ability, at the cost of much heavier training workload. They overlooked that in real-world scenarios, e.g., in high-speed data stream environments, data do not pause accommodate slow models. In this paper, emphasize that model throughput- defined as the maximum number of training samples that a model process within unit of time is equally important. It directly limits how much data a model can utilize and presents a challenging dilemma for current methods. With this understanding, empirical and theoretical perspectives, highlighting two critical issues beyond the well-documented catastrophic forgetting: (i) Model's ignorance: the single-past nature of OCL challenges models to learn effective features within constrained training time and storage capacity, leading trade-off between effective learning and model throughput; (ii) Model's myopia: the local learning nature of OCL on the current task leads the model adopt overly simplified, task-specific features and excessively sparse classifier, resulting in the gap between the optimal solution for the current task and the global objective. To tackle these issues, we propose the Non-sparse Classifier Evolution framework (NsCE) to facilitate effective global discriminative feature learning with minimal time cost. NsCE integrates non-sparse maximum separation regularization and targeted experience replay techniques with the help of pre-trained models, enabling rapid acquisition of new globally discrimi- native features. Extensive experiments demonstrate the substantial improvements of our framework performance, throughput and real-world practicality.
revisit key challenges in OCL from both
1 Introduction
Online continual learning (OCL) is the learning paradigm that enables models to learn continuously from dynamic data stream D {Di, D2,..., ......,, where D, {ar; Mi J.N, is the dataset of task i sampled from distribution μ4,- Existing OCL methods are designed to promote effective learning by mitigating catastrophic forgetting and improving model plasticity through various techniques like gradient regularization(4] contrastive Dearning/31], experience replay[ [16, 10] and knowledge distillation J.. However, these methods often fail consider the assessment of model throughput, an essential metric especially crucial for managing data streams with varying arrival speeds.
According [l, 81], model's 's throughput is defined as the maximum number of training sample data that a model can process within a unit of time. When the training speed of the model is slower than the speed that data stream arrives, the model is forced to discard some training data which wastes data
Corresponding author. Code link: http::/www...oooooooeeggggggggge Cowpeessiinnnmiii---ooonaaaallaa -nnalllnges--nn-Oliinee----- Continual-Leamingg Leamiing
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

and harms model's performance. Furthermore, maintaining a -ime accessible memory buffer, as required by current OCL methods, also proves challenging in real-world applications(38, 42]. Under these practical concerns, a reexamination of the challenges in OCI from both empirical and theoretical perspectives is in urgent need. this paper, we reveal that model's s ignorance caused by single pass nature of data streams and model's myopia resulting from the continuous arrival of tasks may be more impactful than the well-documented catastrophic forgetting phenomenon.
Model's ignorance. Our first focus lies in whether models can acquire sufficient discriminative features within the limited time during the single-pass training. To independently study this challenge and avoid other issues like catastrophic forgetting and task interference or collision caused by the continuous arrival of tasks, we introduce a relaxed setting for streaming data called single task setting. The data stream is sampled from unified task, allowing data from any category to appear at any moment with equal probability. Under this controlled setting, we have observed that, 1) models trained from scratch under-perform significantly compared to expectations. The tingl-pass nature of OCL inhibits the model's ability to fully harness the semantic information from the data stream, a phenomenon we term as model's ignorance; 2) It is also noticed that existing strategies like contrastive learning and knowledge distillation to mitigate this issue significantly increase the training time of the model, consequently reducing its throughput.
Model's myopia. Even if models can quickly achieve decent performance on individual tasks, performance degradation remains a persistent issue for existing OCL models. Previous studies often attribute this to the model forgetting previously learned information. But in this paper, we propose different perspective. We observed that, in OCL training, the model initially acquires perfect classification accuracy for a specific class, e.g., "car". However, there arrives a critical moment when the model becomes completely confused, mistakenly identifying a "car" as newly introduced class (e.g.. "truck"). We believe this confusion cannot solely be attributed to the model forgetting previous learned knowledge, as the forgetting process should be gradual rather than abrupt. Besides, as training progresses, the parameters in the final layer of the model's is' classifier become increasingly sparse. The emergence of such an excessively sparse classifier causes the model to focus on few discriminative features specialized for the current task. When the model is exposed to only a limited range of categories, this narrow focus on the current task restricts its capability to acquire features with broader discriminative power. We term this limitation model's myopia.
In addition to empirical verification, we adopt the Pac-Bayes theorem to provide insights into the dilemma between effective learning and model throughput. The upper bound of expected risk summation can be segmented into three terms correlated to empirical risk, model throughput and task divergence respectively. Our theory places a particular emphasis on model throughput, which has been long overlooked in the context of OCL. To the best of our knowledge, this is the first attempt to provide theoretical insights into the relationship between model throughput and performance in this area. Given that model needs to adapt to varying data flow rates to ensure its performance, this factor also warrants recognition in theoretical discussions. Plus, interestingly. model's ss myopia and forgetting can be perceived as two complementary aspects of the proposed task divergence term.
Built on the above analysis, we propose new OCL framework called Non-sparse Classifier Evolution (NsCE), which capitalizes on the benefits of pre-trained initialization. This framework introduces a non-sparse regularization term and employs a maximum separation criterion between classes, aimed at mitigating the issue of parameter sparsity while maintaining the model's 's ability in differentiating classes. As a regularization applied uniformly across tasks, it helps to minimize the differences in the distributions of model parameters between tasks. Furthermore, to enhance the model's throughput and diminish the reliance on a real-time memory buffer, we propose an efficient selective replay mechanism. By selectively replaying past experiences, we specifically target data from classes that the model frequently misidentifies and implement the targeted binary classification tasks on them during experience replay. This approach not only enhances model's throughput but also boosts its performance in handling high-speed data streams. Extensive experiments demonstrate that these techniques are crucial for deploying OCL model where high performance, real-world practicality and computational efficiency are all paramount.
2 Ignorance: Trade-off between Effective Learning and Model Throughput
In OCL,, the most extensively studied challenge is the issue of catastrophic forgetting when learning new tasks. However, the poor performance of existing OCL methods on some large datasets [68]

---

Data from different clesset
Traditional OCL Settieg
Task
Tadk
Taudickk
bak'T
Single Setting
Uniffed
27 witE DOTETTTT Coth
aia Aarutiis
25.3
Real-time accuracy of OCL models trained under the standard cross entropy loss Lee both with and without pre-trained models (pre-trained on ImageNet) under our designed single task setting and the impact of some commomly used strategies[16, 49, 49]. Results on additional datasets, influence of different pre-trained models (pre-trained on different datasets, using different backbones and different pre-train tasks) and implementation details are provided in Appendix C.3.
Figure 1:
inevitably leads us to question whether, before considering the issue of forgetting, the model can truly acquire sufficient discriminative features and subsequent classification capabilities. To isolate the challenge brought by the continual arrival of tasks and concentrate on model's behavior on the single pass data stream, first construct a data stream setting called single task setting. As displayed in Figure I, we consolidate classes from multiple tasks into a single unified task, where samples from different classes are introduced at random timestamps with equal probability. It ensures a stable and balanced data stream, mitigating concerns about catastrophic forgetting or class imbalance.
Unsatisfactory model performance. Under this controlled setup, simple supervised model is optimized using cross-entropy loss Lee E' Ee= Vi.c log(o(fe(x;)) where <(-) represents the classifier, f(-) denotes the feature extractor. We implemented common OCL strategies such as experience replay[16], contrastive learning[49, 64] and knowledge distillation[ [] make a compari- son. As illustrated in Figure 1, models trained from scratch fail to reach satisfactory performance in single-pass pass training scenarios, unlike those benefiting from pre- ge-ttrinee initialization. On CIFAR 100, model's 's average accuracy remains below 10% even without any inter-task interference, Single-pass nature of OCL prevents the model from fully leveraging the semantic information from the data stream, which we refer model's ignorance. Meanwhile, as we integrate additional techniques like experience replay, contrastive learning, and distillation chains, the model's 's accuracy progressively improves, from 10% to 20%. Under such circumstances, leveraging additional prior knowledge seems to be the only viable solution. Empirical evidence also supports this viewpoint, as displayed by the performance when using pre-trained initialization in Figurel. While the selection of an appropriate pre-trained model is beyond the primary focus of this paper, we investigate the effects various pre-training methods on different OCL downstream tasks in Appendix C.3.
Vooila
120
70
Nodl "haroupppt (esugter
20-
3 50
60-
40-
50
Vaelll CD
12.
SCS
DR DC
35
50
monzing 1ina Ontss)
Figure 2: Left: Throughput of the model trained using vanilla cross-entropy, experience rcplay[16]] supervised contrastive replay[49] and distillation chain[1]. Right: Performance(AAucc: Area Under the Curve of Accuracy) and running time of the above strategies on 10. "CE++" denotes that we compute and perform extra gradient descent per time step to match the delay of the compared- against strategies. All experiments are conducted under single task setting.
3

---

Decreased model throughput. Despite the partial effectiveness of commonly used strategies in mitigating model's ignorance, they inevitably extend the training time for the same amount of data and increase the demands on the memory buffer's real-time accessibility. As shown in Figure 2(I.eft) the integration of additional techniques consistently increases the training time. But in the context of OCI, this extended training duration leads to processing fewer data units per unit of time, resulting in lower model throughput. Given that the volume of training data is widely recognized as a crucial factor in determining model performance, this highlights a significant flaw in the current evaluation of OCL models. When the data stream's 's flow rate exceeds the training speed, model throughput and effective learning emerge as two interrelated factors subject to trade-offs.
More surprisingly, our findings suggest that these strategies are actually no more effective than simply training the model multiple times (CE++) the same data offset the delays caused by these methods. As illustrated in Figure 2, when handling data at the same flow rate, CE++ can achieve comparable model performance through the application of experience replay, contrastive techniques and distillation chains. Furthermore, the challenges of maintaining a continuously accessible real-time memory buffer, caused by issues like network connectivity and privacy protection, are frequently overlooked, adding further obstacles to the effective implementation of these strategies.
3 Myopia: Key Factor for Performance Degradation
Figure 3: Normalized confusion matrix of NCM classifier (green) and softmax classifier (CIFAR1O) (blue) with ImageNet supervised pre-trained initialization. Due to space limitations, we present partial training process in the main text. Comprehensive training process is in AppendixE.
In addition to issues of model's ignorance and decreased model throughput, we recognize that while pre-trained initialization allows the model to quickly classify training data, relying solely on this initialization does not ensure satisfactory overall performance. Performance degradation continues to pose challenges in OCL [68]. Most previous studies attribute it to the phenomenon of catastrophic forgetting, which caused by interference between current task and previously learned knowledge [66]. Some studies also suggest it is due to the model learning some trivial features[31, 68). To clarify this issue, we conduct a comprehensive monitoring of the model's predictions by visualizing the predicting confusion matrix throughout the entire training process. Specifically, the model is trained by vanilla cross cross-entropy loss (Eq. 1), without employing any other techniques.
T N
=
-LD 1=1
y te log(d" (f(x))) (2% vi) Pc-
We separately assess the discrimination ability of the classifier and feature extractor by evaluating the model's performance using softmax classifier and an online updating NCM (Nearest Class Mean) classifier[58] which are both very common approaches for prediction in OCL. For the NCM classifier, we compute a dynamic class mean prototype for each class using Equation 2 and apply momentum update with the parameter X. This update calculates the new class mean prototype vo new based on the n. data points from class in the current data stream and the previous prototypes old Then, class label of new data can be assigned to the most similar prototype.
new (1 - -)) + +
--E(xx (ax) 1y = c) n² = -arg min [1[(x) -Mall-
He
(2)
The
c=I...C

---

0.57
twen of of pew-tra inttelipatinn
0.230
mwan of wf wa you tratted inttioleaaionn
0.00
dass coatusion occurs truined antiilleation
0.225
MonnohWights. Closs
contucion
insirrnneed initralization
0.08
0.00
0.175
spuesity ofw pro truited Intrallittion
0.130
cless confissus occurs wi pre ia siseed instralppation
0.53
0.115
0.02
0.01
9.075
0.00
9.000
250
500
750
1000
1250
1500
1750
2000
Took
Figure 4: Left: averaged weights of the final FC layer for class 0 in CIFARIO. Right: s(w) (lower s(w) stands for increasing sparsity) the final FC layer for w" corresponds to class 0 CIFARIO. During the training of task 5, the class confusion occurs as Figure 3 where model classify "car" as "truck".
Although pre- pre-trained initialization provides the model with a broader perspective and prior knowledge, performance degradation still occurs with the introduction of new tasks. As illustrated in Figure 3, the precision for the class "car' dramatically drops from 0.95 to nearly 0 during the training of the fifth task. When closer examining the classes that the model confuses during training, such as 'car' and "cat', suppose that distinguishing between them becomes challenging when highly discriminative features from past tasks reappear in new classes. For example, 'car' and 'truck' share similar shapes and backgrounds, while "cat" and "dog" share similar textures, making differentiation difficult. Different from the well-documented catastrophic forgetting, we propose a different perspective that the confusion arises since the discriminative features or knowledge acquired from previous tasks may not be helpful in distinguishing these classes from some new classes in future tasks from the outset. In other words, the previously learned representations may not capture the essential characteristics necessary for effectively differentiating these classes with new classes and this is something perfectly normal even for us as humans. During the training process, the model naturally focuses on features and discriminant criteria that are more important for the current task. The independent arrival of tasks results in such myopic model[70], which is the key factor in the decline of OCL performance.
Plus, we observed that softmax classifiers are more frequently susceptible to performance degradation compared to NCM classifiers. As depicted in Figure 3, during the fourth and fifth tasks, predictions made by the softmax classifier are more biased towards classes in the current task compared to those made by the NCM classifier. Meanwhile, although the features extracted by the model are initially separable, biased classifier soon leads to significant confusion between classes, causing the features to lose discriminative power over time. Such phenomenons better displayed by the complete visualization in Figure15 15 (AppednixE).
To better analyze the reasons behind model's performance degradation and verify our suppose on model's myopia, we evaluate and subsequently visualize the sparsity as 1/s(-) and mean (-) of the parameters in the final fully connected layer of the classifier for each task, as detailed in Eq.3. We represent this final fully connected layer by a matrix W E RdxC' where d represents the feature dimension and C denotes the number of classes. The variable w refers to the e-th column vector extracted from W, and w. E Rd.
(|wr) + |wz| + +... /wal)/d w s(w)
m(w) = =V wp + w
max(|w1), |122, . |wal)
As depicted in Figure 4, the mean and sparsity of parameters in the classifier continuously decrease when each new task is introduced. Prior studies in continual learning [9, 37, 69] have linked this pronounced prediction bias towards recent tasks to the decreasing mean weights for old classes. Interestingly, unlike scenarios involving training from scratch, using pre-trained initialization prevents the model from arbitrarily classifying class as belonging to the current task. Moreover, although the mean weights for the old classes consistently decline, class confusion only manifests with the arrival of task 4. These observations all suggest that the reduction weights is not solely responsible for the observed bias in the classifier. This leads us question whether, during training
5

---

process, the model's criteria for categorizing become simpler. In other words, the model increasingly focuses solely on a limited set of discriminative features that it deems beneficial for the current task, and this focus is precisely the cause of model's myopia. This trend towards simplification illustrated in Figure 4(Right), where there is a noticeable increase in the sparsity of parameters associated with older tasks as new ones are introduced. While relying on few discriminative features is beneficial in traditional supervised learning settings, in OCL, the emergence of such an excessively sparse classifier causes inevitable class confusions across tasks.
4 Theoretical Analysis
In addition to the empirical analysis, we try to provide theoretical insights into the OCL problem and illustrate the aforementioned trade-off between adequate feature learning and model throughput. Specifically, we approach the OCL, problem from a Pac-Bayes perspective, as outlined in Theorem 4.1. Following the common notations used in Pac-Bayes literature [8], we define a model space H and loss function f: H x 2 + RT, which is bounded by constant K > O. Here, 2 represents the whole training set and jr stands for data distribution of task t. In line with the approach described in [34], we denote sequence of distributions (Q:)i=o.rr on H. representing the evolution of the model's learning process. Here, Qi is the distribution of the model parameters after training task with Qu representing the initial parameter distribution and T indicates the total number of tasks. Meanwhile, we denote the flow rate of the data stream (#sampls coming from data stream in the unit time) Ug, the model throughput (#smppes model can train in the unit time) as Um and A as the duration time of task in the data stream. The sum of expected risks R across different tasks can be Eng [((he.=5)]
written as Et t== BrenQ. 1 =)]] and the empirical risk R is T 2j=1
1724
Theorem 4.1. For any distributions μn; -.-s PTT over 2, let Dr be iid set with my min(us, Um ) samples sampled from bt as the datasel of task I, for any X >> and any online predictive sequence (Qu, Q1,.... Qr), the following inequality holds with probability 1 8;
XK
KL(Q.l1Q=-)) X
Tlog(T/0)
R R
min Um D
(4)
constantt
D
It is clear that the upper bound of R can be segmented into three terms, along with constant related to the task number T. They are identified as empirical risk R. model throughput term M, and task divergence term D. Among them, the model throughput term M is determined by the amount of data accessible to the model and this is directly influenced by the model's throughput om when data stream's 's flow rate v, exceeds Um- However, achieving lower empirical risk R. by data augmentations, knowledge distillation or training the model for multiple times typically requires more training time and consequently reducing model throughput Em, which theoretically explains the trf mentioned in Section2 between R and M.
When examining the task divergence term D more closely, we see that nearly all OCL methods aiming at addressing forgetting attempt to minimize the deviation from the parameter distribution of previous tasks by adjusting the current model parameter distribution. Our proposed concept of model's myopia offers a fresh perspective. By aligning the distribution of current model with future ones, the divergence term 'D can be also reduced. It leads us to considering the use of structural constraints (e.g. non-sparse regularization) or pre-trained initialization as promising strategies to enhance model performance. Mcanwhile, it's important to acknowledge that our analysis has limitations; the bound discussed is intended to illustrate the sum of generalization risk for each individual task and can not represent global expectation. We provide detailed discussions and proof in AppendixB.
5 Method
After conducting a series of analysis on the key challenges in OCL, in this section, we provide the NsCE framework to tackle these issues based on the utilization of pre-trained initialization. Our goal is not only to reduce the risks of model ignorance and myopia but also to enhance the throughput of OCL models. We aim to accelerate training speeds to keep pace with data stream progress and reduce the dependence of existing methods on real-time accessible memory buffers.

---

Non-sparse regularization. Unlike some previous methods that aim to acquire task-specific features capable of generalization, in this study, we acknowledge the unrealistic expectation of obtaining a model with absolute discriminative ability within limited scope of classes, even with the rich prior knowledge provided by a pre-trained model. Instead, our focus lies in ensuring the diversity of discriminant features during training and enabling the model to swiftly develop the ability to differentiate between categories from different tasks. As posited in the previous section, in the context of OCL, contrary to traditional settings where sparse classifier is often considered desirable for achieving high classification performance(22. 46], the overly sparse parameters can cause the model to focus solely a limited set of highly discriminative features, increasing the risk of model's s myopia. To mitigate this issue, we propose a straightforward idea of constraining the sparsity of the final fully connected layer of (softmax classifier) of the model. Our goal is to ensure that the model maintains a diverse set of discriminative features during training, allowing it to effectively handle different tasks without being overly biased towards specific features. Meanwhile, it helps reduce the divergence of model distribution across tasks. In specific implementation, considering that the max(.) function is easily affected by small number of outliers in the parameters, we opt to replace it with the l2 norm as a more robust alternative in our sparsity regularization:
(leff + |wo/| + | 1/dd
C.
(5)
wy + 2 + .... + wp?
Maximum separation. While smooth classifier can help to mitigate the model's myopia, it hinders the model's ability to rapidly perform classification in the current task. Moreover, in OCL, is also hard to have simultaneous access to data from all categories, especially when there are restrictions on the use of memory buffers. This leads to severe class imbalance during the learning process, which a well-recognized challenge in the context of continual learning[72, 76, 39J. Thus, we draw inspiration from the famous Neural Collapse [55] and Maximum Class Separation criterion[39],
also serves as structure constraints on model parameters to minimize the model distribution divergence across the tasks. For learned representations from different categories {f(x)) (x2), ... f. f(rc))) their cosine similarity should satisfy a maximum separation criterion and converge to ideal simplex equiangular tight frame (ETF), Veg, (f (f;)), ((z)) -c---
1 C 4p -- Ct 1111
Σ (((E))))(E)))) po)) Po = C. 1
C2
Dij
1
(6)
C. C
where ōi,5 is Kronecker delta symbol that designates the number if = j and 0 i + j. To address categories not present in the current task, we use the class mean in Eq.2 replace the representation of the corresponding category. Thus, we can denote our total loss function as:
L - + T(p L.)
Targeted experience replay. To enable the model to learn globally discriminative features and correct existing class confusion, we prioritize the categories that the model has previously struggled to distinguish when accessing the memory buffer. During experience replay, we compute confusion matrix to identify frequently confused categories. To address each group of confused categories, we devise a separate binary classification loss specifically designed to expedite the acquisition of discriminative abilities between these confused classes, as shown in Eq.8.
B
L. = i=1 m1
Σ
I(Cr m > T] Log((""(((())) + yi log(ø" (f(x)))))) m n.
(8)
8h represents a normalized confusion matrix and Cn,n > T indicates that the proportion of data belonging to class m being classified as class n exceeds the threshold r. To further enhance model's throughput and diminish the reliance on a real-time memory buffer, we impose limitations on the number of requests allowed to retrieve data from the memory buffer. Compared experience replay, our way of replay achieves higher model throughput and is more specifically targeted at addressing existing class confusion. For a complete description of the algorithm process, please refer to Algorithml. Moreover, our findings indicate that when selecting an appropriate pre- trained model, halting gradient back-propagation the feature extractor often enhances throughput without compromising performance. More discussions are provided in AppendixC.2.2.
traditional
7

---

Table 1: Best AAUC is highlighted in bold, second best is shown underlined.
CIEAR-TO
Methed
23
1122
25 26
20 1122
Finn
1/100
Frreg 1,50
Fing 1,10
Ping 1/103
Frag 1,50
Frog = 1,10
Frog 1/100)
Frog 1/50
Free 1/10
EWC]-1]
61:72077
15.5::12
91,23077
60.70..5
62.923.2
67.2111
6102003
72.6005
83921.1
Di [10
k1 61.5-122
367103
06:0100
59,240.9
61:1403
66:2-1
4101600
152-224
81.93..1
PASS
$2.2208
8526335
90.2812
61.261.3
02.2:09
07.8115
50.12311
78.1=122
83.500.
MC-SCD SAM[S]]
$18-08
4599:1
905-04
602204
03.1408
118:08
61:1:009
789-05
54.810
Ma 15
312221
50.3507
3532211
014210X
S64230
077-00
313311
[R]
62.6:05
35.4:04
61310.33
646:004
7122:12
S84:08
70
$4.10.
02910.
ASER[S0]
60.2:112
$4.4306
57.2810
62.3:0.9
70.9214
020:0.
83.740.4
SER [0]
138-002
959:14
M0.3405
615100
62.74003
71:2:114
52.1-08
73 739-017
DVC wdo Aud[29]
85.060.
89.250.7
57.920.5
$1.2206
67.420.8
5202209
90 90.1
82772..1
DVC[S]
61-1-01.
85.6104
90 1-01--
61810.
02:91100
10.7-119
516-107
122-111
55. 1-10
OCM wio Augi31]
1.
83.3414
90 2011
60910.55
$3A4166
01.:00
46.72112
78 7866110
85.13044
DCM w/ Augf [1]
12:1:29
95.212.1
61311.5
00:1:1
0022:077
S10114
1-8-12
$4.01099
OnPro AigloS]
6377408
52.8663
834:077
563188.4
6 Experiments
Before delving into the specifics of experimental results, we highlight a significant distinction in the utilization of memory buffers between our study and other works. In this work, we impose limitations on the number of requests allowed to retrieve data from the memory buffer. We evaluate our models on six image datasets, incorporating realistic task overlaps to mimic practical scenarios[43]. Training is harmonized using ViT architectures and the Adam optimizer, with consistent training batch size and initialization (MAE pre-training for ImageNet, supervised pre-training for others). We compare our NsCE method 8 replay based methods and 5 regularization based ones. To more effectively assess model performance over time, we employ AAUC as the primary metrics. Specifically, we evaluate the model's is accuracy on all previously encountered tasks at intervals of every 100 training iterations and use these data points to calculate AAUC. Due to limited space, we leave detailed descriptions of the implementation, evaluation metrics, datasets and comparison methods in AppendixC.1.
Table 2: AAUC on on large-scale real-world online domain-incremenn intrremntreetal data stream. We discard OCM on ImageNet due to its significantly higher runtime and computational memory costs.
Method
CLEAR- 10
CLEAR-100
ImageNer
10k
Freg = 1/100
Freq 1/50
Frey 1/100
Frey = 1/30
Frey = 1/500
ER
87.3+11.0
87.9=0.5
80.12006
82.02:0.8
55.620.4
DER++
87.410..5
88.1=0.9
78.51.111
80.410.6
46.510.4
EWC
88.01044
89.0=0.2
81.110.3
81.21:0.5
$0.91110
iCarl
86.21:0.8
87.1=1.1
77.110.8
80.4::0.9
17.1
SCR
84.1+-0.7
85.9-0.6
67.2+0.7
79.7+0A
52.5424
OCM
88.11:0.5
89.2=0.6
80.010.9
82.5+:100
XXXXX
DVC
86.41.0.3
07.0=007
79.4:0.22
80.21:077
$3.110.6
OnPro
86.9+144
88.1-099
80.4+0.3
82.3+-02
52.2+04
NSCE
89,210.7
91.4 088
S431104
83.7
61.6:0.7
Main Results and Analysis. We conduct a comprehensive evaluation of our method by comparing its performance with several existing state-of-the-arl OCL methods as well as various continual learning variants. Table1 displays the AAUC (area under the accuracy curve) for three synthetic benchmark datasets, showcasing the impact of different memory buffer sizes and replay frequencies. This evaluation metric offers more comprehensive assessment compared to the commonly used average accuracy[43]. The results demonstrate that our proposed method, NsCE, consistently outperforms other approaches. Notably, the performance improvement achieved by NsCE is particularly significant when the memory buffer size relatively small and the number of memory buffer access times is very limited. This finding highlights that the proposed framework helps prevent the model from excessively focusing on the current task which crucial when memory capacity or access frequency are constrained. Plus, we also evaluate NsCE on real-world domain incremental datasets large- scale image classification dataset. It allows us to assess the performance and generalizability of approach in real-world scenarios, where the challenges and characteristics may differ. Table2 demonstrates that NsCE can also enhance the performance in real-world domain incremental settings and complex data streams. More experimental results including model's last time accuracy, sensitivity analysis and evaluation on model throughput are provided in C.2. Moreover, we visualize the predictions of our proposed NsCE in Figure5. From the visualization, it is evident that our model quickly learns the current task while avoiding confusion between past categories and categories in the current task much as possible, effectively alleviating model's ignorance and myopia.
Ablation Studies. To investigate the specific effects of different proposed components, we conduct a series of ablation studies. From Table3, we can draw several observations: (1) Each component
8

---

Table 3: Ablation study of the proposed NsCE framework.
CIFARTO
CIFART00
PaneSa
CIEARTON
Method
0.14
01.94
0.TA
1k
vanilla Loc w/ L.
$2.2105
61.31033
55.6.00
10.1
55.6100.4
vanilla for w/ ER C,
84.5+1.3
64.5+0.7
62.000
77.6440
56.2+0.7
vanilla L. w/ E. E, &r
86.2=0.8
66.1=0.4
669-077
81.3#11
1..4
vanilla Cvr targeted ER
87.2+009
71,9:099
724-00
83.7-4066
58.2+11. 1.3
Figure 5: The detailed normalized confusion matrix (CIFAR10) evolution of our proposed NsCE framework (memory buffer size is 100 and replay frequency is 1/100).
propose provides performance improvements, among which targeted ER has the most obvious effect. (2) Constraints on classifier sparsity, as defined by L. prove to be more effective class incremental scenarios where model's 's myopia tends to be pronounced. (3) The maximum separation term Cp achieves consistent performance improvements across datasets.
80
141
Aitti 20 So000 01 Claos
181
775
30
25
65
CIFARL0
15
CIBAR10
CLEAR105
38
ImageNNtt
InspeMMt
30
50
6.1
0.2
0.3
0'4
0.
5000
10600 15000 20100 25100
Forward steps
Figure 6: Left: Sensitivity analysis on 7 and y. Right: Sparsity (1/s(w)) of the classifier under different algorithms.
Sensitivity analysis. We analyze the impact of the threshold targeted experience replay and the coefficient on the non-sparse maximum separate regularization. As depicted in Figure 6, we observe that as the threshold 7 increases, our proposed NsCE has a relatively lower area under the accuracy higher values
curve (A (UC)). This trade-off between performance and efficiency is expected, o T lead to fewer samples being replayed, resulting in improved model throughput but potentially compromising performance. Furthermore, approach demonstrates robust outcomes when the coefficient y is not too small, and it basically achieves the best performance when 7 = 0.01.
Classifier sparsity. We are also very interested in how sparsity would be affected by the proposed NsCE and methods focusing on re-arranged last layer weight updates. After implementing ER-ACE and ER-AMI. 12], we found that the phenomenon of parameters rapidly becoming sparse is indeed somewhat mitigated, though not as significantly as with our proposed regularization term L., as illustrated in Figure 6. While incorporating ACE AML can also boost performance for baselines

---

like ER and SCR. We believe that when ACE and AML nudge the learned representations to be more robust new future classes, they indirectly decrease the sparsity of the model parameters. For GSA[32], the sparsity is not affected. But we are not entirely sure whether this part is perfectly embedded if further tuning would help, as the authors only provide hyperparameters for CIFAR-100. For SS-II,, we did not find its implementation, so may be left for future works.
7 Conclusion
In this study, conduct thorough reevaluation of the major challenges in current OCL methods. We delve into the underlying causes of these challenges and the limitation of existing methods. Our analysis highlights two critical limiting factors: model's ignorance and myopia, which can have more significant impact than the widely recognized issue of catastrophic forgetting. Furthermore, we introduce the NsCE framework, which incorporates non-sparse maximum separation regularization and targeted experience replay techniques with a focus on balancing performance, throughput and practicality. Our work aims provide fresh perspective and inspire the OCL field prioritize both model's 's performance and efficiency in more real-world scenarios.
8 mcknowledgments and Disclosure of Funding
This work was supported by the National Science and Technology Major Project 2022ZD01 14801, Natural Science Foundation of China (NSFC) (Grant No.62376126), the National Key R&D Program of China (2022ZD01 14801), National Natural Science Foundation of China (61 (61906089), Natural Science Foundation of China (NSFC) (Grant No.621061022 Natural Science Foundation of Jiangsu Province (0222),,, Graduate Research and Practical Innovation Program at Nanjing University of Aeronautics and Astronautics (xcx)h20221601).
10

---

References
[1] A. Agrawal, N. Kedia, A. Panwar, J. Mohan, N. Kwatra, B. S. Gulavani, A. Tumanov, and R. Ramjee. Taming throughput-latency tradeoff in Im inference with sarathi-serve. arXiv preprint arXiv:2403.023 10, 2024.
[2] H. Ahn, Kwak, S. Lim, H. Bang, H. Kim, and T. Moon. Ss-il: Separated softmax for incremental learning. Prrennggnrrrrrreeeeereeeeeee Proceedings of the IEEEJCVF International conference on computer vision, pages 844-854 004-853,22 2021.
[3] R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the Luropean conference on computer vision (ECCV), pages 139-154, 2018
[4] R. Aljundi, E. Belilovsky, Tuytelaars, Charlin, M. Caccia, M. Lin, and L. Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems, volume 32, 2019.
[5] R. Aljundi, K. Kelchtermans, and Tuytelaars. Task-free continual learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 2019.
[6] R. Aljundi, M. Lin, Goujaud, and Y. Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019.
[7] P. Alquier. User-friendly introduction to pac-bayes bounds. arXiv preprint arXiv:2110.111 2021.
[8] Alquier, Ridgway, and N. Chopin. On the properties variational approximations gibbs posteriors. The Journal of Machine Learning Research, 17(1):8374-414, 2016.
[9] E. Belouadah and A. Popescu. 112m: Class incremental learning with dual memory. In Proceedings of the IEEE/CVF international conference on computer vision, pages 583-592, 2019.
[10] P. Buzzega, M. Boschini. A. Porrello, D. Abati, and S. Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15520-155300 2020.
[11] L. Caccia, R. Aljundi, N. Asadi, T. Tuytelaars, J. Pineau, and E. Belilovsky. New insights reducing abrupt representation change in online continual learning. ICLR, 2022.
[12] L. Caccia, R. Aljundi, T. Tuytelaars, J. Pineau, and E. Belilovsky. Reducing representation drift in online continual learning. arXiv preprint arXiv:2 1(3), 2021.
[13] H. Cha, J. Lee, and J. Shin. Co2l: Contrastive continual learning. In Proceedings of the IEEE/CVF International conference on computer vision, pages 9516-9525, 2021.
[14] A. Chaudhry, N. Khan, P. Dokania, and P. Torr. Continual learning in low-rank orthogonal subspaces. Advances in Neural Information Processing Systems, 33:9900-9911 2020.
[15] A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny. Efficient lifelong learning with a a-gem. ICLR, 2019.
[16] A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. K. Dokania, P. H. Torr, and M. Ran- zato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902. 10486, 2019.
[17] H. Chen, Y. Wang, T. Guo, C. Xu, Y. Deng, 7. Liu, S. Ma, C. Xu, C. Xu, and W. Gao. Pre-trained image processing transformer. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 12299-12310, 2021.
[18] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597-16077 PMLR, 2020.

---

[19] A. Chrysakis and M.-F. Moens. Online continual learning from imbalanced data. In International Conference on Machine Learning, pages 1952-1961. PMLR, 2020.
[20] M. De Lange and T. Tuytelaars. Continual prototype evolution: Learning online from non- stationary data streams. In Proceedings of the IEEF/CVF international conference on computer vision, pages 8250-82555 2021.
[21] C. de Masson Autume, S. Ruder, L.. Kong, and D. Yogatama. Episodic memory in lifelong language learning. Advances in Neural Information Processing Systems, 32, 2019.
[22] A. Dedieu, H. Hazimeh, and R. Mazumder. Learning sparse classifiers: Continuous and mixed integer optimization perspectives. The Journal of Machine Learning Research, 22(1):600866055 2021.
[23] J. Deng, W. Dong, R. Socher, L.,J. K. Li, and Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255, Icee, 2009.
[24] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800 2018
[25] Dhar, R. V. Singh, K.-C. Peng, Z. Wu, and R. Chellappa. Learning without memorizing. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 5138-5146, 2019.
[26] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 1 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
[27] P. Garg, R. Saluja, V. N. Balasubramanian, C. Arora, A. Subramanian, and C. Jawahar. Multi- domain incremental learning for semantic segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 761-771, 2022.
(28) Y. Ghunaim, A. Bibi, K. Alhamoud, M. Alfarra, H. A. Al Kader Hammoud, A. Prabhu, P. H. Torr, and B. Ghanem. Real-time evaluation in online continual learning: A new hope. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 11888-11888 1897, 2023.
[29] Y. Gu, X. Yang, K. Wei, and C. Deng. Not just selection, but exploration: Online class- incremental continual learning via dual view consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7442-7451, 2022.
[30] Y. Guo, W. Hu, D. Zhao, and B. Liu. Adaptive orthogonal projection for batch and online continual learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6783-6791, 2022.
[31] Y. Guo, Liu, and Zhao. Online continual learning through mutual information maximiza- tion. In International Conference on Machine Learning, pages $109-8126, PMLR, 2022.
[32] Y. Guo, Liu, and D. Zhao. Dealing with taskk class discrimination in online continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11878-11887, 2023.
[33] Y. Guo, M. Liu, T. Yang, and T. Rosing. Improved schemes for episodic memory-based lifelong learning. Advances Neural Information Processing Systems, 33:1023-1035, 2020.
[34] M. Haddouche and B. Guedj. Online pac-bayes learning. Advances in Neural Information Processing Systems, 35:25725-25738. 2022.
[35] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pages 16000-16009, 2022.
12

---

[36] P. Helber, B. Bischke. A. Dengel, and D. Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):211-2226, 2019.
[37] S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 831 831-839, 2019.
[38] M. Jung, S. A. McKee, C. Sudarshan, C. Dropmann, C. Weis, and N. Wehn. Driving into the memory wall: The role of memory for advanced driver assistance systems and autonomous driving. In Proceedings of the International Symposium on Memory Systems, pages 377-386, 2018.
[39] T. Kasarla, G. Burghouts, M. van Spengler. van der Pol, Cucchiara, and P. Mettes. Maximum class separation as inductive bias in one matrix. Advances in Neural Information Processing Systems, 35:19553-19566, 2022.
[40] G. Kim, S. Esmaeilpour, C. Xiao, and B. Liu. Continual learning based on ood detection and task masking. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 3856-3866, 2022.
[41] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. RUsu, K. Milan, J. Quan, T. Ramalho, A. Crabska-Barwinnse et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 11((13):351-33552 2017.
[42] J. Ko, C. Lu, M. B. Srivastava, J. A. Stankovic, A. Terzis, and M. Welsh. Wireless sensor networks for healthcare. Proceedings of the IEEE, 98(1 (1 1): 2010.
[43] H. Koh, D. Kim, J.-W. Ha, and J. Choi. Online continual learning on class incremental blurry task configuration with anytime inference. ICLR, 2022.
[44] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images, 2009.
[45] K.-Y. Lee, Y. Zhong, and Y.-X. Wang. Do pre-trained models benefit equally in continual learning? In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages
2023.
[46] T. Levy and F. Abramovich. Generalization error bounds for multiclass sparse linear classifiers. Journal of Machine Learning Research, 24(151):1-35, 2023.
[47] Z. Lin, J. Shi, D. Pathak, and D. Ramanan. The clear benchmark: Continual learning on real- world imagery. In Thirty-fifth conference on neural information processing systems datasets and benchmarks track (round 2), 2021.
[48] Z. Mai, R. Li, J. Jeong, D. Quispe, H. Kim, and S. Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28-51, 2022.
[49] 7. Mai, R. Li, H. Kim, and S. Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 3589-3599, 2021.
[50] A. Mallya and S. Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765-7773, 2018. 2018.
[51] S. V. Mehta, D. Patil, S. Chandar, and E. Strubell. An empirical investigation of the role of pre-training in lifelong learning. Journal of Machine Learning Research, 24(214):1-50, 2023.
[52] M. Mirza, M. Masana, H. Possegger, and H. Bischof. An efficient domain-incrementt learning approach to drive in all weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3001-3011, 2022.
[53] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wa, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. Advances Neural Information Processing Systems Workshop, 20
13

---

[54] G. Oren and L. Wolf. In defense of the learning without forgetting for task incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2209-2216 2021.
[55] V. Papyan, X. Han, and D. L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40):24652-22 -46663, 2020.
[56] A. Prabhu, P. H. Torr, and P. K. Dokania. Gdumb: A simple approach that questions our progress in continual learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pages 524-540. Springer, 2020.
[57] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, Agarwal, Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748 8763. PMLR, 2021.
[58] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[59] O. Rivasplata, I. Kuzborskij. C. Szepesvári, and Shawe-Taylor. Pac-bayes analysis beyond the usual bounds. Advances in Neural Information Processing Systems, 33: 1683--16885, 2020.
[60] D. Shim, Z. Mai, J. Jeong. S. Sanner, H. Kim, and J. Jang. Online inaremnnrremettll continual learning with adversarial shapley value. In Proceedings the AAAI Conference on Artificial Intelligence, volume 35, pages 1630-9638, 2021.
[61] S. Sodhani, S. Chandar, and Y. Bengio. Toward training recurrent neural networks for lifelong learning. Neural computation, (2(1):1-33 2020.
[62] G. Sokar, D. C. Mocanu, and M. Pechenizkiy. Learning invariant representation for continual learning. arXiv preprint arXiv:2101.06162 2021.
[63] G. M. Van de Ven and A. S. Tolias. Three scenarios for continual learning. arXiv preprint 1904:1900 2019.
[64] W. Wan, X. Wang, M.-K. Xie, S.-Y. Li, S.-J. Huang, and S. Chen. Unlocking the power of open set: A new perspective for open-set noisy label learning. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 15438- 15446, 2024.
[65] A. Wang, A. Singh. J. Michael, F. Hill, O. Levy, and S. R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. ICLR, 2019.
[66] L. Wang, X. Zhang, H. Su, and J. Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302. 2302002300 2023.
[67] L. Wang, X. Zhang, K. Yang, L. Yu, C. Li, L. Hong, S. Zhang, Z. Li, Y. Zhong, and J. Zhu. Memory replay with data compression for continual learning. ICLR, 2022.
[68] Y. Wei, Ye, Z. Huang, J. Zhang, and H. Shan. Online prototype learning for online continual learning, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 18764-18774, 2023.
[69] Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2019.
[70] W. Xinrui, W. Wan, C. Geng, S.-Y. Li, and S. Chen. Beyond myopia: Learning from positive and unlabeled data through holistic predictive trends. Advances in Neural Information Processing Systems, 36, 2024.
[71] J. Yang, R. Shi, D. Wei, 7. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023.
14

---

[72] Y. Yang, H. Yuan, X. Li, J. Wu, L. Zhang, Z. Lin, P. Torr, D. Tao, and B. Ghanem. Neural collapse terminus: A unified solution for class incremental learning and its variants. arXiv preprini arXiv:2308.0/440, 2023.
[73] J. Yoon, E. Yang. J. Lee, and S. J. Hwang. Lifelong learning with dynamically expandable networks. ICLR, 2018.
[74] K. You, Y. Liu, Wang, and M. Long. Logme: for transfer learning. In International Conference on Machine Learning, pages 12133-12143. PMLR, 2021.
Practical assessment of pre-trained models
[75] F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987-3995, PMLR, 2017.
[76] Z. Zhong, Cui, 7. Li, Lo, J. Sun, and J. Jia. Rebalanced siamese contrastive mining for long-tailed recognition. arXiv preprint arXiv:2203. / 1506, 2022.
[77] D.-W. Zhou, Q.-W. Wang, H.-J. Ye, and D.-C. Zhan. A model 603 exemplars: Towards memory-efficient class incremental learning. ICLR, 2023.
[78] D.W.. Zhou, H.-J Ye, D.-C. Zhan, and Z. Liu. Revisiting with pre-trained models: Generalizability and adaptivity all you need. arXiv preprint arXiv:2303.0733 073388 2023.
learning
[79] Z. Zhou, L.ZZ. Guo, L.-H. Jia, D. Zhang, and Y.-F. Li. Ods: test-time adaptation in the presence of open- world data shift. International Conference on Machine Learning, pages PMLR, 2023.
[80] Z.-H. Zhou. Stream efficient learning, arXiv preprint arXiv:2305.02211. 2023.
[81] Z.-H. Zhou. A theoretical perspective of machine learning with computational resource concerns. arXiv preprint arXiv:2305.02217 v3, 2023.
[82] F. Zhu, X.-Y. Zhang, C. Wang, Yin, and C.-L. Liu. Prototype augmentation and self- supervision for incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5871-5880, 2021.
15

---

A Related Works
Continual Learning. Continual learning is a research field dedicated to learning continuously while mitigating the forgetting of previously acquired knowledge(54, 52, 27, 52, 27, 63]. Most continual learning approaches employ three types of techniques. Regularization- saaabr approaches introduce regularization terms or constraints to the learning process preserve previously learned knowledge[41, 75, 3, 25]. Memory-based approaches utilize external memory buffers or re- play mechanisms to store and replay past data, allowing the model to retain access to previous experiences [33, 82, 15, 16, 60, 65]. Architecture-basee approaches involve modifying the model ar- chitecture to facilitate continual learning(61, 73, 50, 40]. Additionally, reducing storage overhead and minimizing dependence on hardware devices are issues of concern in the research community667, 77].
Online Continual Learning. Online continual learning (OCL) serves as a more realistic extension of continual learning. Unlike traditional batch learning, where the entire dataset for each task is available upfront, OCL operates in scenarios where data distributions dynamically change over time. In OCL,, similar to memory-based approaches in CL. most methods leverage real-time accessible memory buffer and employ various experience replay methods to mitigate the issue of forgetting[ 15, 16, 4, 6, 60, 5, 19, 14, 58, 21, 62, 67, 11]. Besides, other OCL methods aim to improve the learning of better features and classifiers in a single-par pass training manner[[88] Techniques like contrastive learning[ (9, 13], mutual information maximization 31], and prototype learning[82, 68] have been employed to enhance the discriminative abilities of the model and improve its performance. Moreover, there are other works that focus a more proper evaluation of existing algorithms[43, 28). Compared with methods that aim for better performance, we focus on rethinking key challenges in OCL and then design framework under more realistic throughput and storage constraints. Despite that online learning has been extensively studied by theoretical community, research on generalization bounds tailored for online continual learning remains scarce and our bound serves a simple attempt to bridge the performance and model throughput.
Online Continual Learning with Pre-trained Models. The utilization of pre-trained models has become a common approach in various machine learning tasks, including transfer learning[74, 17], natural language processing[24 [24] and class-incremental learning[51, 78]. While the effectiveness of pre-trained models has been well-established for these applications, only a few works[45] have explored their impact on OCL. These studies reveal underperforming algorithms can become very competitive when considering when using pre-trained models [35, 18, 57,30]. 30].
B Detailed Proof and Limitations
We first reintroduce the classical PAC-Bayes adapted from [7, 8] as the Lemma.
Lemma B.1 (Adapted from [8], Thm 4.1), Let D = (21,.... 2m) be an iid set sampled from the law μ. For any data-free prior P, for any loss function E bounded by K, any X >> 0, [0, 1], one has with probability 5 for any posterior Q E (H)::
1 m
En- 2,1 + KL(Q[IP) + log(1/8)
AK²
En ((hh, 2)1
< m Σ
X
2m
where MH) denotes the set of all probability distributions on H.
Remark B.2. TheoremB.1 1 is a special case of the original theorem from [8] as we take the case of a
bounded loss which implies the subgaussianity of the random variables E(., 2;) and then allows us to recover the factor AK2
Following [59], we introduce the notion of stochastic kernel which formalise properly data-dependent measures within the PAC-Bayes framework. First, for a fixed predictor space H. we set Σn to be the considered o-algebra on H. We denote MMH as the set of all probability distributions on H.
Definition B.3 (Stochastic kernels[59[). A stochastic kernel from DD 2m H is defined mapping Q : 2 x Σn + [0; 1] where
For any B E En, the function D (Z1, 2m) + Q(D B) is measurable,
* For any D E th the function B + QDD, B) is probability measure over H.
16

---

We denote by Stoch(D, H) the set of all stochastic kernels from S to H. and for a fixed S, we set QD = QD...) the data-dependent prior associated to the sample through Q.
Following TheoremB.3, we provide a formal definition of the online predictive sequence [34]: Definition B.4. A sequence of stochastic kernels (Pi) is denoted as an online predictive sequence if (i) for all i 2 1,5 ε zmh (D( .) is F measurable and (ii) for all i 2 2, (D(D..) > PD-1(D,.).
For P.Q Mi (H), the notation Q indicates that Q absolutely continuous wrt P (i.e. Q(A) = 0f P(A) = 0 for measurable A C H). Before giving detailed proof, let us first reclaim our theorem.
iid set with min( Us. Ae samples sampled from μt as the dataset of task t and um is the model throughput, v, the flow rate of the stream and & the the time of the data stream for task t. for any X > 0 and any online predictive sequence (used as both priors and posteriors) (Qu, QI.... Qr), the following inequality holds with probability 1 5:
=m.)) Theorem B.S. For any distributions J1:..., PrT over Z, D; = (
T
T
E
ER
[e(bes 25)1 +
E Ene Qe =tll
mc
j=1
KL
(Q-1))1111)
T
AK
Tlog(T/8)
2m
Proof. For each task / in OCI,, Dr. By applying T.eoremB.1, 1, we can have that let DD = (21,.., Zm,) be an iid set sampled from the law pa. For data-free free prior Qt-1, for any loss function e bounded by K, any À > 0, 6 E [0, 1]. one has with probability 1 for a data-dependent posterior Q E &M1HH):
consider Qt-1 prior since it doesn't 't depend the dataset
BRUnQ, [E(he 25]]
KL(QallQ--)
AK
log(ō)
Ene-r [E [l(he, 2)]] S
m1
2m
We then make 8 = 8/T and take an union bound on all tasks to ensure with probability 8 for any e 1.2....T:
En Q. [e(he. [[)]
(I(QuIlQ:-1)
AK2 log(T/6)
BayaQ, [Bz,ng (((,e, x]]] <
+
m4
X
2m,
A
Then, by taking a sum on all tasks, we can have the following result with probability 55:
Ena.q [E(he:25)
KL (T
Qu [E (ece,, 2I]1
[e(he,
mp
X
1=1
XK²
log (Tg(T/))
+
2m
X
T mc Enenq [e(he:2))
T
XK2
me
min (Us, Um ).
R
M
T Tlog(T/8)
const
D
17

---

The flexibility of the classical PAC-Bayes bound allows the stochastic kernels Qr to be either data- dependent distributions or not, as stated in TheoremB.1. I. In the case of data-dependent distributions, the only available prior we can select in the predictive sequence Qi is the initial prior distribution Qo- Meanwhile, it is possibly the largest term in the divergence term D. This emphasizes the significance of a good initialization and pre-trained model in achieving favorable results. Furthermore, by examining LemmaB.5, we can observe that the derived bound deteriorates as the number of tasks T increases. This deterioration arises from the growing number of new tasks, which makes online continual learning more challenging. As the model needs to adapt and accommodate an expanding set of tasks, the learning process becomes increasingly complex and prone to performance degradation.
Furthermore, the third model throughput term in the bound emphasizes the significance of model throughput, as it directly impacts v. Many existing techniques in OCL, such as data augmentation, knowledge distillation and gradient constraints all increase the training time, consequently reducing the amount of data (m²) that the model can process when the data stream's 's flow rate surpasses the v. lower model throughput not only hampers the practicality of the OCL method but also restricts the model's generalization ability..
Compared traditional generalization bounds that only consider the final output of an algorithm, the left side of TheoremB.5 5 evaluates the performance of the model at each time step. This distinction crucial because in continual learning scenarios, the model's ' performance should be assessed and monitored throughout the learning process, rather than solely focusing on the final outcome. Indeed, considering the performance of the model at each time step can be seen as a compromise that aligns the generalization gap with a notion of regret. Compared with the regret bound provided in [34], the deteriorated convergence rate mainly caused by the fact that at each time step we don't have an access to all the past data to predict the future as the projected Online Gradient Descent (OGD) algorithm. In summary, this is just a simple and natural extension of [8] in the context of OCL. However, we believe that TheoremB.5 already provides some theoretical guidance for the issues that OCL needs to address. In future work, we hope to obtain a more in-depth theoretical analysis specifically for this problem.
Limitations. First, is important to clarify that the bound discussed is specifically used to illustrate the generalization risk associated with each individual task, rather than representing global risk. This limitation means that our current theoretical analysis can not extend data streams composed of disjoint tasks. Despite this constraint, we believe that it does not detract from the main findings presented in the core sections (model's ignorance and myopia) of our paper. Minimizing the distribution divergence across tasks remains one of the most fundamental concepts in OCL, despite our theoretical results can not fully reflect the benefits of mitigating model's forgetting or myopia. Part of the reason is that such a sum of risk is easier to account for the problem of the trade-off between effective learning and model throughput. On the other hand, it is actually extremely challenging to directly establish the relationship between the expected risk of the posterior distribution Qi and the empirical losses of the entire training process. Actually, research on generalization bounds tailored for online continual learning remains scarce and our bound serves simple attempt in this area and extension of LemmaB.1.
C Setups and Additional Experiments
C.1 Experiment Setups
Memory buffers. Before delving into the specifies experimental results. it is essential highlight a significant distinction in the utilization of memory buffers between this paper and other works. Traditional methods typically employ a real-time accessible memory buffer, where at each time step i, the model receives a mini-batch of data X U X², drawn i.i.d from D, and the memory buffer B, respectively. However, in this work, we impose limitations on the number of requests allowed to retrieve data from the memory buffer. Furthermore, we will assess the time and storage overhead incurred by any additional computations, including data augmentation, knowledge distillation, and gradient calculations. For our experiments, we employ three distinct memory sizes along with their corresponding experience replay frequencies, as presented in each respective table. In contrast to existing based methods that sample a small batch of data from the memory buffer at every training iteration, we evaluate the performance of OCL methods under the assumption that they only have access to the memory buffer every 10, 50, 100 or 500 training iterations. This approach allows us to evaluate the methods under various throughput requirements and is more aligned with the sif-site storage of data and models in real-world scenarios.
18

---

Datasets. We use 6 image classification datasets in the evaluation including CIFARIO, CIFAR 100, EuroSat, CLEARIO, CLEAR100 and ImageNet[44, 36, 47, 23]. CIFARIO has 10 classes with 40,000 for training and 10,000 for testing. It is split into 5 disjoint tasks with 2 classes per task. CIFARION 100 has 100 classes with 40,000 for training and 10,000 for testing. is split into 20 disjoint tasks with classes per task. EuroSat has 10 classes with 17,799 for training and 7,000 for testing. It is split into 5 disjoint tasks with 2 classes per task. CLEARIO, 10, CLEAR 100, two continual image classification benchmark datasets with a natural temporal evolution of visual concepts in the real world that spans decade (2004-2014), We adopt the "streaming" protocols for CL that always test on both seen data and data in the (near) future. ImageNet has 1000 classes with 1,281,167 for training and 50,000 for testing. It is split into 200 disjoint tasks with 5 classes per task. All the methods are trained in supervised manner and tested on seen classes at any given time. In our experiments, we employ a blurry task boundary as suggested by [43] instead of the conventional disjoint task boundary to better reflect realistic and practical scenarios. Specifically, in the process of data arrival, there is partial overlap (set at 10%) between the data at the boundaries of different tasks, rather than being completely disjoint.
Implementation details. For CIFARIO, CIFAR100, and EuroSat, we utilize the ViT-Tiny as a backbonn and ViT-Base [26] for CLEARIO, IO, CLEAR 100 and ImageNet. We train the model with AdamW optimizer for all the datasets and comparing methods. For all the methods compared, we set the same batch size (10) and replay batch size (10) for fair comparisons. We reproduce all baselines in the same environment with their source code and default settings. For the methods requiring a real-time memory buffer compute some exclusive variables, we ensure the correct calculation of these variables by increasing the frequency of access to the memory while ensuring a same replay frequency. For the pre-trained models used in Table 2, 3.4 and we use the MAE pre-trained initialization [35] for ImageNet and supervised pre-trained initialization for other datasets.
Compared baselines. We conducted a comparison of our NCE approach with 13 baselines, as shown in Table5, consisting of 8 replay-based OCL baselines and 5 offline CL baselines. To ensure a fair comparison, we implemented vanilla experience replay on the 3 offline CL baselines, running all approaches for one epoch.
The traditional metric, Average accuracy (Aavg), is commonly used in continual learning. However, Agva only provides information about the model's performance at specific moments of task transitions, which may occur only 5 to 10 times in most OCL setups. This temporal sparsity of measurement makes it insufficient to deduce conclusions about the model's any-time inference capability. In this paper, we use alternative evaluation metrics: Area Under the Curve of Accuracy (AAuc) and Last Accuracy. Inspired by the work of [43], we measure accuracy more frequently by evaluating after every An samples, instead of only at discrete task transitions. This new metric is equivalent to the area under the curve (AUC) of the accuracy-to-f of samples curve for continual learning methods, with An 1. We refer to it as Area under the curve of accuracy (A calculated as AAUC = Ein f(i An)- An. Additionally, we include Last Accuracy as another evaluation metric. Last Accuracy simply refers to the model's accuracy after it has processed all the data in the data streams.
Evaluation metrics.
Device. All the experiments are implemented on NVIDIA RTX2080ti and RTX4090ti. It is notable that all results on training efficiency, model throughput and inference time are done on RTX2080ti.
C.2 More Detailed Experimental Results
C.2.1 Last Accuracy
In addition to AAUC, we also evaluate the last accuracy of various OCL methods. We include these two evaluations because A Ave allows to assess the real-time performance of the model, while the last accuracy measurement reflects the model's performance after processing the entire data stream. As shown in Table4 and 5, Even without employing data augmentation and knowledge distillation, our NsCE framework still achieves comparable results. This is particularly evident when faced with more stringent constraints on memory buffer size and replay frequency.
C.2.2 NsCE Lite.
In addition to enhancing model throughput through constraining memory replay, we also consider the possibility of not fine-tuning the entire network since we have already utilized pre-trained model. However, when faced with large-scale data streams with changing data distributions, it becomes challenging for the model to adapt to new data without fine-tuning. In such cases, we evaluate the
19

---

Table 4: Last Accuracy synthetic online class-increment setting. Best is highlighted in bold, second best is shown underlined.
CIPAR
Fing 1/100
Fire 1,50
Frg
1,10
Frag 1/100
Pray = 1,50
Fing 1/300
Firg 1,200
Frog
1/50
Sieg 1,10
501 1::::12
90104-0-11
V1. 3234003
095104
70:24417
731--122
77 77M-10
NT. 27.540.
EWC
$5:5:0.4
02442015
942210
67.920.5
660021.1
70421.
75.781.1
$4.5009
30.2211
DAR -11
61811.
92.0609
88.7110
MC-SGD SAME51)
50.:06
70.00.4
73.1408
889:07
0733203
[R]
91.3:0.7
92.02:44
0,
71,510.5
73.3:0.5
736:04
76.3:03
10
93.40.9
MIR [
921=100
92 1500
44 1:11111
0K 0K..0
74.1:00
7472488
74,422
50 90L16O
6622:0.
90.22:12
03.021.1
67.910.5
7102:04
7233:08
74.3200
$59=0.5
92.910.5
[CR
02.2
69.9109
717440.
743-108
K7 8-8-017
64.2411
wio Aus[29]
714102
7102206
74.2342
OUM
14.::13
43.5614
92001.1
64.210.3
64.7004
718212
787=10
80.450.5
DCM wf A1g
656:11
$9.5::14
93 01610.5
73711.88
73.5210 8
739:00
74.0310
15
03..1.77
UnPro wi
92.2.199
92 92.1
94.100
69.420.5
7377200
741:016
708424
88 886
95.3&1.1
NSCE
90.
950:14
708613
739217
737214
849917
91.7088
944:446
Table 5: Last Accuracy on real-world online domain- incremental setting and large scale data stream. Best is highlighted in bold, second best is shown underlined.
Method
CLEAR-10
CLEAR-100
ImageNet M =
M 0.14
0.26
3 = II
M 2K
10E
Freg = 1/100
Freg = 1/50
Freg = 1/100
Freg = 1/50
Preg = 1/500
ER
23.4+022
93.5=0.6
88,5+0.9
140.2
17.0+
DER
92.7+004
93.420.8
87.920.4
88.9100.
43.82:077
EWC
93.0110.6
92.120.7
8..3
89.6110.9
46.01:0.4
iCarl
91.4+-0.9
92,8+1.2
8.
85.9+-0.8
47.9+110
SCR
894400.6
89.8=0.6
83.4+0.6
87 1.:01.1
46.3+11
OCM
92.11005
92.7=1.3
87.9+0.9
86.7100.6
DVC
91.3110.88
91.9=0.4
88.0:10.6
88.1100.5
45.910.5
92.24-1.2
9.65
88.0+1..
88.3
NsCE
93.910.7
94,2=111
90.11088
89.2116
49.812.4
features learned by our model (Eq.2) during training on the data stream. If the model has acquired sufficiently discriminative features for the current task, we believe that only updating the classifier layer would suffice to achieve optimal results. We refer to this lightweight framework as NsCE Lite, detailed Algorithml.
We conducted tests on our lightweight version of NsCE using the smallest memory buffer and lowest replay frequency on six datasets, as presented in Table6. In most cases, this lightweight framework also achieves comparable results with NsCE, particularly on relatively simple datasets like CIFARIO and EuroSat, where NsCE Lite even outperforms the original NsCE approach. The potential reason for this improvement may be that when NsCE Lite encounters simpler datasets, despite constraining the sparseness of the classifier parameters with the dataset, model's 's myopia caused by intensified parameter sparsity exists not only in the classifier but also in the feature extraction process. This becomes especially apparent when the model acquires highly separable features. Further training may cause the model excessively focus on discriminative features that lack generalization ability. Hence, introducing a detach operation to the feature extractor f(() can effectively mitigate the model's myopia, especially when the model has attained satisfactory performance on the current task. By detaching the feature extractor, the model can retain the learned features while allowing for independent updates and adjustments to the classification layer or other components.
C.3 Discussions on Utilization of Pre-trained Models
Model's ignorance and myopia: new perspectives. The current performance bottleneck of the OCL method serves as the initial motivation for our study on the application of pre-trained models in OCL, As shown by results in Table7 from [68], even the best methods can only reach about 30% on CIFARI 100 and 20% accuracy on TinylmageNet. Although the exploration of these methods may be meaningful to the community, such performance is completely unworthy of discussion for
Table 6: Comparison results between proposed NsCE and NsCE Lite (AAuc). The methods that exhibit the best performance with pre- trained models are highlighted in bold.
CHOMMI
CLIAIEOO
137/314
RERITLY
NA02 Lite
96
7239-00
761-99
929912
20

---

Algorithm NsCE (Lite)
Input: Data stream D, encoder f, classifier ø Initialization: Memory buffer M < fhacc, =
for to T do for each mini-batch X in D, do
M (U, X)
if acc, > Uhreshold then 0y.detach()
end if
p= (x)
Compute online class mean and y" by Eq.2 0, Lee + 7Cp + L.) by Eq.7
if Replay then
Compute Confusion Matrix on Memory buffer 03 t L. by Eq.8
end if
Caculate the accuracy ace, on current task by y
end for end for
practical problems. Furthermore, when review previous work from the perspective of model's ignorance and myopia, we observe that many techniques originally developed for continuous learning, such as knowledge distillation and dark experience replay, may not be fully applicable to OCL scenarios. In OCL, the model requires more than just relying on past cogmition. It needs the flexibility to dynamically adjust and update its features and classification criteria. The improvement in performance of OCL methods often stems from alleviating model ignorance through replaying past data and incorporating augmentation methods. These approaches help the model adapt and refine its representations, reducing the impact of myopia and enabling better performance in OCL settings.
Inspiration of using pre-trained models. Inspired by how humans learn quickly and effectively, we realize our ability to recognize new class is typically built upon fundamental cognitive abilities and prior knowledge. In fact, for most intelligent life forms, the process of learning begins with the acquisition of fundamental concepts and knowledge. For instance, humans possess innate abilities for perception and instinctual drive to seek advantages while avoiding disadvantages. These foundational aspects of learning form the basis upon which more complex cognitive abilities and knowledge are built. We anticipate that pre-trained models, which have demonstrated success across various domains, can play a similar role as fundamental knowledge that enables a high-throughhut, gehhperrforance OCL model supporting any-time inference.
Table 7: Average accuracy of state-of- the-art art methods without the pre-trained initialization. (The results are directly copied from [68])
CIFAR-TO
CIFAR-T00 Mf 1k
Tiny mage Net
Method
0.1k
M 0.2k
0.3k
M 0.5k
24
M
26
4k
iCaRL[S8]
31.0+1.2
33.9=099
42,0+0.9
128800.
1654004
17.6=0.5
5,0=0.3
6.6=04
7.4
DER++100
31.5429
39.7+27
509918
16.0:0.6
2141009
23.9110
3.7204
5.120.8
68100.6
PASS[82]
33.74222
33,7+22
33,7422
7.5+0.7
7.5+0.7
7.50077
0,5=0.1
0.5=0.1
0.5+0.1
AGEM[15]
17.740.3
17.5+0.3
17.540.2
58+00.1
5.940.1
58+0.11
0.8+0.1
0.8+0.1
0.+00.1
GSS[6]
18.4.0.2
194.1.77
25.2.009
8.110.2
9.4.0.5
10.1
1.110.1
1.510.1
2410.4
ER[
19444066
20.9+0.9
26.04122
8.740.3
9.940.5
10.740.8
1.240.1
1,5+02
2.04022
MIR[4]
20.7+007
23.540.8
29.9+112
9,7400.3
11.240.4
13.0=0.7
1.420.1
19=002
2.9+0033
GDamb[56]
23.34-1.3
27.1+0.7
34.0408
8.240.2
11.040.4
15.3+0.3
4.6-0.3
6.6+0.2
10.0+0.3
ASER[60]
20.0++1.0
22.8+00.
31.6+111
11.0+:0.3
13.5+0..3
17.620.4
2.2-0.1
4.2=0.6
8.400.7
SCR[49]
40.24:1.3
48.5+11.5
$9.12:1.
19.3:0.6
26.31005
32.720,3
8.220.3
14.720.3
19.510.3
CoPE
33.54:32
37.3+2.2
42,9435
11.6+0.7
14.6+11.
16.8=0.9
2.1
2.3204
2.5+0.3
DVC[29]
35.24:17
41.61277
53.842.
13410.7
20.34110
25.21166
4.920.6
7,5200
10.91111
0CM]31]
47.511.7
59.610.4
34.....5
20.910.77
OnPro[68]
57.81111
63.5+110
72.610.8
22,74007
30.011.4
35.920.6
11.920.3
16.9=04
22.110.4
Pre-trained model is not a one-size-fits-all solution. Although pre-trained models have demon- strated significant performance improvements across various datasets compared with the performance trained from scratch (as illustrated in Table7), crucial acknowledge that they are not one- size-fits-all solution. The limitations manifest in multiple aspects, depicted in Table8, 9 and
21

---

Euioon
hell di un serree Foe Pecio Male
Vell di wifi 1itt
Varia Peprrise Hait
Oerla CD witi
1000
2000
1000
2000
1400
1900
2000
fauds Doricsse Heals
MDP bncieee
Figure 7: We evaluate the real-time accuracy of models on currently seen classes (w/) and (w/o) pre-trained models under our designed single task setting, as well the impact of experience replay frequency on CIFAR, EuroSAT, CLEAR and ImageNet.
Table 8: Performance (AAuc) under our single task setting. We illustrate the impact brought by pre-trained models (models pre-trained on ImageNet by Masked Auto Encoder(35]) with different network architectures over various datasets. The networks that exhibit the best performance with pre-trained models are highlighted in bold, while the networks that achieve the best performance without pre-trained models are shown underlined.
Model
CIFARIÓ
CIFAR100
EuroSat
SVHN
TissueMNIST
VTT-T pretrain
31.31
9.77
55.71
54.16
43.68
VIT-T w/ pretrain
93.54
61.97
76.09
88.24
59.84
+62.23
+52.20
+20.38
+34.08
+16.16
VITSS w/o pretrain
37.64
6.95
51.81
36.86
42.78
VIT-S w/ pretrain
90.38
79.49
78.02
93.33
60.04
+52.74
+72.54
+26.21
+56.47
+17.26
10, the benefits (or drawbacks) of utilizing pre-trained model vary depending the dataset and network architecture employed. Pre-trained models do not consistently yield substantial gains across all datasets. The effectiveness of pre-training depends on several factors, such the degree of domain similarity between the pre-training and target tasks, the size and quality of the pre-training dataset, some specific characteristics of the target dataset and even the structure of backbone networks matters. For CIFAR, EuroSat, SVHN[53], and TissueMNIST[[11, the distributional discrepancy between the pre-training and downstream task data gradually increases. It evident that pre- trained models tend to provide more benefits when the pre-trained data share common semantics with the downstream tasks. Surprisingly, even for the same dataset and pre-training approach, the choice of model architecture can significantly impact the performance of the model, as observed in Table8,
Table 9: Performance (A under our single task setting. We illustrate the impact brought by pre- trained models (models pre-trained on CLIP[S7]) with different network architectures over various datasets. The networks that exhibit the best performance with pre-trained models are highlighted in bold, while the networks that achieve the best performance without pre-trained models are shown underlined.
Model
CIFARIO I0
CIFAR100
EuroSat
SVHN
TissueMNIST
Res50 w/o pretrain
38,58
14.04
37.64
88.04
43.72
Res50 w/ pretrain
86.44
79.27
65.31
92.10
60.31
+47.86
+65,23
+27.67
+4.06
+16.59
ViT-S w/o pretrain
37.64
6.95
51.81
36.86
42.78
ViT-S w/ pretrain
83.70
76.87
59.36
90.09
49.37
A
+55.24
+69.92
+7.55
+53.23
+6.59
22

---

Table 10: Performance (AAUC) under our single task setting. We illustrate the impact brought by pre-trained models (supervised models pre-trained on ImageNet) with different network architectures over various datasets. The networks that exhibit the best performance with pre-trained models are highlighted in bold, while the networks that achieve the best performance without pre- preitrained models are shown underlined.
Model
CIFARIO
CIFARIO0
EuroSat
SVHN
TissueMNIST
Res18 w/o pretrain
30.62
16.21
30.14
81.45
39.57
Rc w/ pretrain
43.60
53.41
35.68
85.60
39.40
A
+12.98
+37.20
45.54
+4.15
-0.17
Res50 w/o pretrain
38.58
14.04
37.64
88.04
43.72
Res50 w/ pretrain
49.21
56.73
46.20
88.91
48.60
+14.63
+42,69
+8.56
+0.87
+4.88
2 w/o pretrain
32.10
11.17
29.52
87.69
42.38
WRN28-2 w/ pretrain
46.83
37.57
36.67
87.78
39.41
+14.73
+26.40
+7.15
+0.09
-2.97
WRN28-8 w/o pretrain
26.66
13.22
21.24
90.17
44.35
WRN28-8 w/ pretrain
57.13
44.23
29.91
90.93
44.05
+30.47
+31.01
+8.67
+0.76
-0.30
VT-T w/o pretrain
31.31
9.77
55.71
54.16
43.68
ViT-T w/ pretrain
91.49
57.55
76.40
90.07
53.35
A
+60.18
+47.78
+10.70
+35.91
+9.67
VIT-S w/ pretrain
37.64
6.95
51.81
36.86
42.78
VIT-S w/ pretrain
92.88
76,87
79.40
95.11
57.64
A
+55.24
+69,92
+17.59
+58.25
+14,86
9 and 10 on SVHN and TissueMNIST. These findings highlight the nuanced nature of leveraging pre-trained models. When comparing networks based on convolutional neural networks (CNN), it is evident that transformer-based models tend to derive more benefits from pre-trained models. Furthermore, we also discover that different pre-training methods also exert significant influence on the model's performance. However, despite conducting these experiments, we have not yet been able to discern definitive rules for selecting pre-trained models. Careful consideration and experimentation necessary to identify the optimal combination of pre-training and downstream task settings for achieving the desired performance improvements. The overall efficacy of pre-training is influenced by various elements, including the domain alignment between the pre-training and target tasks, the volume and integrity of the pre-training dataset, specific pre-training scheme, particular attributes of the target dataset and the architecture of the backbone network. Generally, we posit that possessing insight into the expected data distribution of upcoming tasks enables the selection of pre-trained model trained on comparable distribution, which is prudent and dependable approach. We believe the selection of appropriate pre-trained models remains an important open question for many areas including OCL.
D Detailed Discussions on Efficiency and Feasibility of Current OCL Methods
In this section, we present empirical observations on the efficiency and feasibility of current OCL methods. We will discuss these observations from several key aspects: requirements on memory buffer, model throughput, and performance, By examining these aspects, we aim to provide insights into the practicality and effectiveness of existing OCL methods in real-world applications.
D.1 Requirements on Real-time Memory Buffers
As we stated before, OCL serves as more realistic extension of continual learning, unlike traditional batch learning, where the entire dataset for each task is available upfront, operates in scenarios where data distributions dynamically change over time. However, we find that, there is very limited research that specifically addresses the accessibility of memory buffers during training in the context of OCL. In this study, argue that such assumption is highly unrealistic in real-world environment. Most existing replay-based OCL methods exhibit some flaws when applied to real-world applications:
(1) As illustrated in Figure8, notable characteristic of replay- based methods is their tendency to sample a larger proportion of data from the memory buffer compared to the incoming data stream. However, such a continuous sampling process significantly restricts the throughput of the model for streaming data. has been observed that the time required to retrain memory data is typically 3-5 times longer than that for new data. Similarly, some data augmentations also significantly reduce
23

---

Skipped data
Sumpling prucess from memorr (may nut be accessible)
Online stream
Training delay
Streaming
Environment
mode
Traning
Sampling
Memory buffer
Text (any time inforence)
Possible sampling falture
Figure 8: Common framework of replay-based OCL methods. Possible sampling failure and training delay due to the mismatch between model training speed and the data stream flow rate are two primary concerns.
the model throughput. This prolonged training time not only results in increased training delays but also leads to more skipped data, reducing the amount of available data that can be effectively utilized within a given time frame, highlighted in[28, 80].
(2) Furthermore, there typical assumption that the memory buffer needs to store dozens of samples for each class help the model to enable efficient review of past classes. However, when dealing with large-scale datasets such as ImageNet [23], the storage overhead becomes impractical, particularly for data that needs to be real-time accessible and stored in local memory. Not to mention that in real-world scenarios, the number of categories in the data stream we encounter is constantly increasing. Such limitations become evident when the system lacks continuous access past data, severely restricting the model's learning capacity, as illustrated in Figure 7. Additionally, most existing methods actually require the memory buffer, model and data being processed be stored in GPU memory simultaneously to avoid latency during access, This discrepancy between the existing methods and the practical scenario further diminish the practicality of current OCL methods.
(3) In real-world upplications, such as autonomous vehicles [38] or sensor networks [42], ensuring the real-time accessibility of the memory buffer presents a sigmificant challenge, especially when the learning system is deployed in terminal equipment. The constraints of computing resources, privacy concerns and network connectivity all hinder the sampling process in the memory buffer, thereby diminishing the feasibility of existing OCL methods. For example, the latency caused by data transfer makes difficult for the model to synchronize and obtain data from both the memory buffer and the incoming data stream. Additionally, due to privacy and copyright concerns, in most cases, cannot store data from the data stream arbitrarily. For instance, real-world autonomous driving scenarios, data cannot be uploaded to data centers at any time. Meanwhile, the data stored in data centers usually undergoes strict scrutiny to avoid the risk of infringing on privacy or the commercial copyright of another party.
In our research, we simulate the situation where the memory buffer, model and data stream are stored separately by limiting the number of accesses to the memory, which means cannot replay the desired data at any given moment. Nevertheless, it is important to acknowledge that there may still be a gap between the scenarios we simulated and real-life applications. However, we firmly believe that taking this step is beneficial to the community.
D.2 Model Throughput and Performance
33-
Sarlo Callas
104
104
Galfer Jor 1aa
1041
Figure 9: Training time of iCarl[58] under different replay frequencies across datasets.
24

---

08.10 B1
Figure 10: Training time of Experience Replay (ER)[16] under different replay frequencies across datasets.
Figure 11: Averaged model throughput of OCL methods on datasets (Replay frequency as 1/100).
In recent years, various OCL methods have been proposed [48, 31, 68, 66, 79], among them, replay- based techniques that interleave past experiences with new data have emerged as a predominant component. It aims to consolidate feature learning from earlier tasks while mitigating catastrophic forgetting through the constant re-exposure of few old data. However, the evaluation of OCL methods often overlooks assessment of model throughput, critical metric especially for data streams with different coming speed. To assess the performance and efficiency of popular OCL methods more effectively, we first record the running time as shown in Figure9 and Figure 10. It evident that as the replay frequency increases, the training time of the model (every 200 training iterations) also significantly increases. Consequently, this leads substantial reduction in the model's throughput. In comparison, our experimental settings, including frequencies of 1 1/50 and 1/100, considerably shorten the training time and enhance the model's throughput compared to fetching data from the memory buffer for each training iteration. Moreover, we evaluate the averaged model throughput of 1l OCL methods on 6 datasets. As shown in Figure 1, our proposed NsCE and NsCE Lite improve model throughput while ensuring good performance.
EDC
ICB
SCA
Deto
Figure 12: AAUC (Area Under the Curve of Accuracy) and running time on CIFAR10 and CIFAR100. represents for different replay frequency of 1/100, 1/50. 1/10.
We also visualize the running time and A AUC of various OCI models with a pre-trained initialization. As shown in Figure 12, existing methods indeed achieved improvements in model performance, but
25

---

they often come at the cost of slower training speed. Interestingly. as shown in prior [2rk[28], if we utilize the available extra time to increase the frequency of replay and train the model multiple times on the memory, the performance gains are often greater compared to using state-of-the-an techniques such as various regularization or knowledge distillation techniques.
In addition to training time, we also compare the inference time between our NsCE method and existing approaches as it also serves as an important metric when doing the real-time inference. As indicated in Table II, our method achieves inference time comparable to ER. However, due to the requirement to compute feature similarity or the need for extra projector, methods like iCaRL and OnPro exhibit slower inference speeds.
Table 11: Comparison the inference time between NsCE and some popular methods.
Methods
CIFARIO
EuroSat
ImageNet
ER
22s
16s
6min43s
iCaRL
31s
26s
9min45s
OnPro
24s
20s
7min09s
NCE
22s
18s
6min50s
D.3 Conclusion
In addition to efficiency, achieving a balance between model performance, throughput, and practicality is of great concern. The aforementioned practical limitations highlight the necessity for innovative approaches that can adapt to resource-constraing environments and effectively address the challenges in OCL. Considering the difficulties encountered in real-world scenarios, simple alternative is to limit the frequency of memory access throughout the entire training process. This approach not only improves training throughput but also eliminates the need for real-time storage, thereby alleviating requirements related to hardware specifications, network connectivity, and privacy concerns.
However, this alternative approach introduces new challenges in avoiding model myopia and potential forgetting. While pre-training models can expedite the learning of valuable features, sampling less from memory makes the model to excessively concentrate on the current task, increasing the risk of model myopia and catastrophic forgetting.
E Forgetting Phenomenon
We meticulously record the changes in model classification results when using a linear classifier without pre-training initialization, using linear classifier with pre-training initialization, and using demonstrated by Figure14 and Figure 15,
a prototype classifier with pre-training initialization. pre-trained initialization allows the model to retain previously learned discriminant information without indiscriminately dividing the data into the current class. In our evaluation, we specifically assess the classifier results of CIFAR 10 using linear softmax and the NCM prototype classifier. To gain insights into the model's classification performance, we visualize the classification results on the test set at the beginning and end of each task, as shown in Figure 14. We calculate the model's classification confusion matrix to analyze the results. observations reveal the following:
Pre-trained initialization helps the model rapidly achieve performance the current task while providing a broader perspective to avoid mindlessly classifying past classes as part of the eurrent task (as the comparison in red and blue in Figure15).
The linear softmax classifier demonstrates a quicker acquisition of improved discriminative abilities for data within the current task compared the NCM classifier (as the comparison in green and blue in Figure 15). However, it is more prone to misclassifying categories from previous tasks as belonging to the current task. resulting in a decline overall performance.
When using pre-trained model and having continuous access to the memory buffer, as illustrated in Figure 14, we can quickly achieve good overall performance.
To further clarify the difference of our recognized model's 's myopia and the catastrophic forgetting, we implement existing anti-forgetting techniques against myopia and benchmark our NsCE framework against prevalent methods both with and without experience replay. Our results in Table 12 show that popular gradient-baat -aaed regularization methods such as EWC and AGEM do not effectively prevent performance degradation. Their performance are only on par with simple supervised
26

---

Table 12: Comparison of anti-forgetting techniques and our method.
Methods
CIFRRI w/ IR
CHARIO LR
CHBARJO0 LR
CIFARIO ER
LuroSa ER
EuroSar
Baseline
82.6
71.6
61.3
EWC
81.7
70.9
60.7
4044
61.0
31.4
AGIEM
78.6
72.3
50.2
37.9
56.4
39.7
OsPro
NoCE
86.2
79.8
66.1
46.8
72,4
45.6
Training phase:
Discriminative feature: color /
Task a:
cucumber: green
Data Datastres stream:
banana: yellow
Discriminative feature: color /
Task b:
brick: red
butters yellow
Testing phase:
red: brick
green: cucumber V
yellow; banana or butter *
Color can serve classification criteria
Color does not work
more
Color, which is the most discriminative feature for task (banana vs. cucumber) and task b (butter vs. brick), is precisely the reason why the model confuses butter and banana.
Figure 13:
learning baseline and it's the same case with or without pre-trained initialization. Additionally, techniques like SCR and OnPro, which use contrastive learning to improve feature discrimination, do not consistently enhance performance. Notably, the gains seen with SCR and OnPro largely stem from use of augmented samples, a tactic not used in earlier gradient-based methods or our NsCE approach. These findings underscore the idea that model myopia is more pressing concern than often-addressed catastrophic forgetting in the context of OCL.
In addition to the empirical evidence, we give a simplified example of the model's myopia. As shown in Figure 13, the discriminative features or attributes for the current task may exactly be the cause of confusion when dealing with future categories, which means the model's cognition for each class must dynamically evolve with the arrival of new data.
600 930 070000 EEGOOE
BJS DIOURA
4434m
Akpbie-
0060076 600
34000000
020022
08332
les K00 231393375 020.4
Gar
der
2038
20333
5038
D03
020038 600.3
000000
kene 2015200 misenne
34
-44
1ag
2011 0000 S100E00
5028640 338 330000 000033
110
Figure 14: Normalized confusion matrix after fine-tuning memory buffer with size of 500.
27

---

XX
501
Figure 15: The normalized confusion matrix (CIFAR10) evolution of linear softmax classifier without pre-trained initialization (red) and NCM classifier (green), linear softmax classifier (blue) with supervised pre-trained models ImageNet.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Section 2&3
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Appendix C
Guidelines:
The answer NA means that paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Section Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the code and details in Appendix B.1 and our supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
30

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the code and details in Appendix B. and the code link in the first page.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nip) for more details.
public/gute suuiides/ooeSuuuisseeeeeeeeeeeee
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /nips..c.. httpc//pppcpcguuueeeeeeeeeeeeeee guwgge/ddeseeee /Code/ooddesssseeeeeeee httpl////guiee//eeeuuiiiiiiiiss cesssioooolii onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
We provide the code and details in Appendix B.1 and our supplementary materials.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
31

---

The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: In Section5
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In Appendix B.1
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https htt///nuriip..o http:////c/.p..cetccccccccceeeea
32

---

Answer:
[Yes]
Justification: I have checked
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Not included.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not included.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring
33

---

that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have cited mentioned them in our paper and code.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have included them the supplementary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
34

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: No crowdsource.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: Not included.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
35


Reviews:
[{"review_id": "6b1oNSOyC6", "reviewer": "Reviewer_WX6u", "summary": "The paper identifies and formalizes main challenges specific to OCL. Notably the authors highlight the need for a stronger focus on ignorance (the inability of the online learner to fully converge) and throughput. Similarly, the authors identify Myopia, which corresponds to learning sparse feature, as a potential issue. Therefore they propose a simple yet effective non-sparse regularization strategy, combined with a maximum separation and targeted experience replay strategy to solve myopia with minimal computation overhead. A comparison with state-of-the-art approaches and pre-trained models shows that the proposed approach outperforms existing methods.", "questions": "See weaknesses.", "limitations": "The limitations have been correctly discussed in appendix.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The experiments seem to have been realized with care\n- The paper is well written\n- The experimental results are compelling\n- I appreciate the defined challenges as I would also agree that Ignorance is an important topic in OCL as the focus is shifting away from forgetting in recent studies\n- I appreciate the effort of providing a theoretical analysis\n- the code is available", "weaknesses": "**Major Weaknesses**\n\n1. What is the justification behind this limited request on the memory buffer? Why not include experiments in the traditional setup with $Freq=1$?\n2. Some references to related work on budget CL are missing. I would suggest the authors to clarify what is the difference between their analysis with regard to the throughput and the analysis of computationally budgeted continual learning [1]. It would be beneficial to include such methods in the comparison table.\n3. What is the justification behind the currently used metric $A_{AUC}$? I would like the authors to report the Average Accuracy, which should be the metric of interested in continual learning.\n4. A graph showing the weight sparsity with and without the sparsity-regularization term could also showcase better the impact of the loss. At the moment, there is no clear demonstration of how much myopia has been solved. Overall the presentation of how the proposed approach solves the introduced challenges could be improved.\n5. The authors should clarify the relation between the defined challenges and the usual stability-plasticity trade-off challenges of continual learning [6]. What is the relation between ignorance and plasticity? Is Myopia related to stability?\n6. Figure captions must be enlarges. Confusion matrices are also barely readable.\n\n**Minor Weaknesses**\n\n7. I believe the findings of figure 2 right hand side have been discussed in previous studies such as [5]. Such references could be included in the discussion.\n8. Table 1 readability could also be improved (number size specifically).\n9. figure 2, right hand side has incorrect markers.\n10. If I understand correctly, the \"single task setting\" is just training for one epoch. If so, I would advise to introduce it as such.\n11. equation 3 should be reference in figure4 caption.\n12. l184 : \"This trend towards simplification is illustrated in Figure 4(Right), where there is a noticeable increase in the sparsity of parameters associated with older tasks as new ones are introduced.\" You might want to define the sparsity as $1-s(w)$ as current definition can be misleading. A high sparsity is obtained with a low $s(w)$ value.\n13. I wonder how would the sparsity be affected by methods such as SS-IL [2], ER-ACE [3] or GSA [4] , which focus on re-arranged last layer weight updates.\n\n**Typos**\n\n- caption of Figure 4 : \"class 0 corresponding to class0\"\n- l167: Appednix\n\n**References**\n\n[1] Prabhu, Ameya, et al. \"Computationally budgeted continual learning: What does matter?.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[2] Ahn, Hongjoon, et al. \"Ss-il: Separated softmax for incremental learning.\" Proceedings of the IEEE/CVF International conference on computer vision. 2021.\n\n[3] Caccia, Lucas, et al. \"Reducing representation drift in online continual learning.\" arXiv preprint arXiv:2104.05025 1.3 (2021).\n\n[4] Guo, Yiduo, Bing Liu, and Dongyan Zhao. \"Dealing with cross-task class discrimination in online continual learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[5] Buzzega, Pietro, et al. \"Rethinking experience replay: a bag of tricks for continual learning.\" 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.\n\n[6] Wang, Maorong, et al. \"Improving Plasticity in Online Continual Learning via Collaborative Learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "_Respected Reviewer WX6u,_ We first thank you for your valuable and insightful feedback, and for recognizing our motivation and theoretical analysis. Below, we address your concerns in a point-by-point manner and welcome further discussion if anything remains unclear.\n\nQ: _Justification behind the limitation on the request frequency on the memory buffer_  \nA: We address this question from three aspects.\n1. Our initial intuition for limiting request frequency to the memory buffer is based on practical considerations. In real-world scenarios, such as autonomous vehicles and sensor network data classification where OCL could be applied, ensuring real-time accessibility of the memory buffer without incurring training delays is typically impractical. This is detailed in Appendix D.1.\n2. When take the traditional setting ($Freq=1$ and replay a small part of data in the memory each time step), the training time is typically much longer and significantly reduce model's throughput as illustrated in Figure 9 & 10, let alone replaying the whole memory buffer at $Freq=1$. \n3. Additionally, we find that using a proper pre-trained initialization and setting $Freq=1$ enables all baselines to achieve very high performance as shown in the following table. A high-frequency replay on a moderately sized memory buffer naturally resolves most issues like forgetting, myopia and even ignorance because it makes the setting more akin to an offline, non-continual one. This, to some extent, contradicts the intuition of the OCL setting in my opinion. Here, we take the performance $A_{AUC}$ on EuroSat as an example, for each time step we replay $10$ percent of the data in memory buffer:\n\n|     | M=0.1K | M=0.2K | M=0.5K | M=1K |\n|  ----  | ----  |  ----  | ----  | ----  |\n| ER        |86.4|89.5|89.8|91.3|\n| OnPro |84.6|87.6|90.0|90.9|\n| NsCE w/o target replay|87.8|89.4|90.4|91.0|\n\nQ: _Comparison with computationally budgeted continual learning_  \nA: We address this question from three aspects.\n1. _Some missing references:_ We first thank the reviewer for pointing out the missing related work on budgeted continual learning. We will cite those references in the revised version.\n\n2. _Difference between our analysis and theirs:_ The main difference is that in our setting, data is strictly treated as an online stream, meaning no revisit is allowed except for the data in the memory buffer. The starting points of the two papers are different: they aim to address the CL problem with limited training iterations, whereas we strive to train a model to match the original speed of the input data stream. Notably, in our experiments, the computational budget is much lower than the mentioned paper. In our setting ($Freq=1/50 or Freq=1/100$), the computational budget $\\mathcal{C}$ allows revisiting less than $0.1$ percent of all observed data at given step (in average), whereas in their paper, this proportion is $25-50$ percent.\n\n3. _Including computationally budgeted continual learning methods in the comparison table:_ We take the suggestion and add some baseline methods in computationally budgeted continual learning denoted as CBCL with ACE (We find the implementation of CBCL is very close to ER). For results for larger datasets like CLEAR and ImageNet, we will leave them in the future version. It appears that the method performs slightly better than ER in our original Table 1, but it is less competitive compared to our method.\n\n|     | CIFAR10 M=0.1K Freq=1/100 | CIFAR100 M=0.5K Freq=1/50 | EuroSat M=0.1K Freq=1/100 |\n|  ----  | ----  |  ----  | ----  |\n| CBCL w/ ACE       |84.0 $\\pm$ 1.0|62.4 $\\pm$ 0.8|60.9 $\\pm$ 0.6|\n|     | CIFAR10 M=0.5K Freq=1/10 | CIFAR100 M=2K Freq=1/10 | EuroSat M=0.5K Freq=1/10 |\n| CBCL w/ ACE       |89.2 $\\pm$ 0.4|72.7 $\\pm$ 1.2|85.1 $\\pm$ 0.8|\n\nQ: _The relation between the defined challenges and the usual stability-plasticity trade-off_  \nThe stability-plasticity trade-off is well-known in continual learning. Techniques to prevent forgetting often limit new knowledge acquisition. We thank the reviewer for giving us chances to clarify relationships between ignorance vs. plasticity and myopia vs. stability.\n\n1. We first acknowledge that both ignorance and plasticity emphasize the need to improve the model’s capability to acquire new knowledge. However, our concept of ignorance extends beyond plasticity by incorporating the model's throughput. As illustrated in Figure 2 and Figure 11, although the method in [6], referred to as CCL or Distillation Chain in our work, enhances plasticity for OCL, it significantly reduces the model's throughput. This reduction potentially harms its practicality and performance, as detailed in Section 2.\n2. Our examination of model myopia aims to refine the concept beyond its interpretation in the context of stability. Previous literature focusing on model stability attributes the model's decreasing performance to the interference or overlap in the feature representations or gradient contention of new and old classes, advocating for safeguarding existing knowledge to mitigate forgetting. Conversely, we contend that this performance degradation arises more from the model's limited exposure to subsets of classes during training, which causes it to favor features unique to these subsets. This leads to a myopic classification perspective with poor generalization. I think as we mentioned in our theoretical analysis (Section 4), our proposed concept of model's myopia actually offers a fresh perspective.\n   \nOverall, our defined new challenges serve as an extension or an independent aspect (the other side of the coin) compared to the traditional stability-plasticity trade-off.\n\n**Note: Due to strict space limits, our responses to the other review questions are consolidated in the Author Rebuttal above. We apologize for any confusion caused by not responding individually. Please refer to the Author Rebuttal or let us know if you have any other questions to discuss further.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7VUHn7bq03", "reviewer": "Reviewer_u1kw", "summary": "This paper identifies two previously overlooked challenges in online continual learning (CL): model ignorance and myopia. In response, it introduces a new framework called Non-sparse Classifier Evolution (NsCE). NsCE features non-sparse maximum separation regularization and targeted experience replay techniques designed to quickly learn new globally discriminative features. Experimental results show significant enhancements in both model performance and throughput.", "questions": "1. Is there any insight on how the size of the memory buffer relates to the number of learning tasks? The size of memory buffers seem quite random across the datasets.\n2. Does performance depend on the order of the tasks/domains? Sometimes it will influence the results as different data distributions come from different tasks.\n3. Is there a concern about over confidence or under confidence with this approach? Given that modern deep neural networks are particularly prone to issues with confidence levels, an analysis of this aspect would be valuable.\n4. See the weakness.", "limitations": "NA", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-motivated by the shortcomings of existing OCL methods. It tackles a significant learning problem, and the focus on model throughput from both empirical and theoretical perspectives is a pertinent challenge in many practical applications involving data streams.\n2. The strength of the paper lies in its thorough empirical evaluation, which convincingly illustrates the concepts of model ignorance and myopia. Additionally, the introduction of the term \"myopia\" is intriguing, and the accompanying theoretical analysis offers convincing insights into this issue.\n3. The analysis of model parameter sparsity is interesting, and the proposed method with pre-trained initialization aims to address the issues of model ignorance and myopia, ultimately achieving good empirical performance.\n4. It's the first time theoretical results have included discussions of model throughput, which represents a noteworthy contribution.", "weaknesses": "1. For the model's ignorance, following the general expectation of transfer learning that pre-trained models facilitate fast learning, it is not surprising that using pre-trained models improves the performance of OCL. So how the pre-trained model improve the learning speed is still not clear\n2. As highlighted in Section 4.1 of the article, many existing methods struggle with large-scale volatile datasets. While employing pre-trained models can partially mitigate the issue of model ignorance, it is still uncertain whether this approach offers a definitive solution. Although Appendix B.2 addresses this concern, a definitive and clear solution to fully resolve this problem has not yet been established.\n3. It's better to illustrate why to claim that using $max()$ function is easily affected by a small number of outliers in the parameters.\n4. The paper claims that a smooth classifier can help mitigate the model's myopia, but it hampers the model's ability to perform rapid classification on the current task. Are there any insights into why this occurs?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "_Respected Reviewer u1kw,_ We first thank you for your valuable and insightful feedback, and for recognizing our empirical evaluation and theoretical insights. Below, we address your concerns in a point-by-point manner and welcome further discussion if anything remains unclear.\n\nQ: _How the pre-trained model improve the learning speed_  \nA: We first appreciate your insightful question. As stated in Section 2, we highlighted that there is a significant trade-off between effective learning and model throughput. Meanwhile, Figure 1 clearly shows that a proper pre-trained initialization enables the model to rapidly achieve good performance on continual tasks. This, in turn, reduces the need for extensive training steps to mitigate ignorance, thereby increasing the learning speed. Certainly, fully exploiting pre-trained models to improve learning speed in OCL is a very important topic, and we hope to address this in future works.\n\nQ: _A clear solution on how to resolve the issue of model ignorance when using pre-trained models._  \nA: First, we must acknowledge that, up until now, using an appropriate pre-trained initialization is the only efficient way we have found to resolve ignorance. However, our research also indicates that the efficacy of pre-training is influenced by various factors, including the domain alignment between the pre-training and target tasks, the volume and quality of the pre-training dataset, specific attributes of the target dataset, and the architecture of the backbone network. Generally, we posit that having insight into the expected data distribution of upcoming tasks allows for the selection of a pre-trained model trained on a similar distribution, which is a prudent and dependable approach.\n\nQ: _Why to claim that using $max()$ function is easily affected by a small number of outliers in the parameters._  \nA: Thanks for your suggestion. By definition, the max() function selects the highest value in the dataset. It naturally does not consider the frequency or distribution of other values, which may cause some unexpected problems in optimization or evaluations.\n\nQ: _Why a smoother classifier sometimes hampers the model's ability to perform rapid classification on the current task._  \nA: We have observed this phenomenon in the sensitivity analysis and ablation study. The reasons behind it are straightforward: compared to a sparse classifier, a smoother one tends to give vague predictions, which often indicate degrading performance. Therefore, it is important to ensure that $\\mathcal{L}_{ce}$ remains the dominant term in optimization, while $\\mathcal{L}_s$ serves more as a regularization.\n\nQ: _How the size of the memory buffer relates to the number of learning tasks?_  \nA: Since each dataset comprises a varying number of classes, it is typically the case that datasets with more classes require the storage of more data, leading to a need for a larger memory buffer. Our experimental setup largely adheres to the parameters established by [1]. \n\n[1] Wei Y, Ye J, Huang Z, et al. Online prototype learning for online continual learning. ICCV, 2023. \n\nQ: _Is there a concern about over confidence or under confidence with this approach?_  \nA: We appreciate your insightful question and agree that the overconfidence or underconfidence of a model's predictions is a promising aspect for understanding the model's behavior during continual learning, especially in online scenarios. In fact, model myopia, recency bias in the classifier, and even ignorance can be considered from the perspective of the model's prediction confidence. For example, the bias towards the current task can be perceived from the perspective of overconfidence, and the $\\mathcal{L}_s$ we proposed can help alleviate this overconfidence. However, how to evaluate the confidence level and how existing techniques addressing issues like overconfidence will perform in OCL remains unclear and warrants further exploration. We hope to leave it our future works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MocYRBBfDP", "reviewer": "Reviewer_abk8", "summary": "This paper revisits the core challenges of Online Continual Learning (OCL) in high-speed data stream environments, identifying two significant obstacles: the model's ignorance and myopia. It then introduces a non-sparse classifier evolution framework (NsCE) designed to effectively address these issues. Additionally, the authors offer some theoretical guarantees from a PAC-Bayes perspective, providing insights into the robustness and generalization capabilities of the proposed method.", "questions": "1. See the weakness.\n2. There are some typos and grammatical errors, for example, \"Plus, we also evaluate NsCE on real-world domain incremental datasets and large-scale image classification datasets.\" An \"and\" is missing here.\n3. Some existing literatures like [A] have also talked about the training delay in OCL. It's better to cite it in the main text.\n4. The performance improvements appear relatively marginal in terms of the last accuracy reported in Tables 4 and 5, yet there is a significant boost in performance on the AUC metric. Could there be specific reasons behind this discrepancy?\n\n[A] Y. Ghunaim, A. Bibi, K. Alhamoud, M. Alfarra, H. A. Al Kader Hammoud, A. Prabhu, P. H.Torr, and B. Ghanem. Real-time evaluation in online continual learning: A new hope. ICCV2023", "limitations": "The authors have adequately addressed the limitations.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The analysis of factors beyond forgetting in OCL is enlightening, especially in highlighting the suboptimal performance of current approaches.\n2. The inclusion of model throughput as a factor in OCL is crucial, and the analysis from a Pac-Bayes perspective offers compelling insights.\n3. The examination of sparsity and the proposed Non-sparse Classifier Evolution (NsCE) in continual learning is noteworthy. The method is straightforward, easy to implement, and has proven effective in experiments.\n4. The paper is generally well-structured and easy to understand.", "weaknesses": "1. While the authors have imposed constraints on the number and frequency of memory buffer accesses, these conditions still mimic laboratory settings rather than real-world applications. It would be beneficial to provide examples of real-world scenarios where such restrictions are applicable and realistic.\n2. Despite that the analysis of model throughput as a factor in OCL is intriguing, it seems that the authors did not give a perfect strategy. More analysis on how to improve the model throughput and the relationship with pre-trained models is needed.\n3. Some experiments in the Appendix is better to be concluded in the main text as they also serve as some important validations on the proposed methods like results in Table6, Table11 and Table12.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "_Respected Reviewer abk8,_ We first thank you for your valuable and insightful feedback, and for recognizing our analysis from a Pac-Bayes perspective and proposed method. Below, we address your concerns in a point-by-point manner and welcome further discussion if anything remains unclear.\n\nQ: _Discussion on more real-world scenarios where such restrictions are applicable and realistic._  \nA: We agree with your perspective on the importance of realistic experimental settings and hope to address your concerns from two perspectives. \n\n1. Evaluations on real world settings: In fact, we have included two real-world continual image classification benchmark datasets with a natural temporal evolution of visual concepts in the real world that spans a decade (2004-2014) into the experiments to include both class incremental and domain incremental settings.  \n2. Real-world applications: Similar to answer to reviewer u1kw, our initial intuition for limiting request frequency to the memory buffer is based on practical considerations. In real-world scenarios, such as autonomous vehicles and sensor network data classification where OCL could be applied, ensuring real-time accessibility of the memory buffer without incurring training delays is typically impractical.\n\nQ: _More analysis on how to improve the model throughput and the relationship with pre-trained models._  \nA: To further improve model throughput, we provided a NsCE Lite version in Appendix C.2.2, which enhances throughput by not fine-tuning the entire network. We found that, in most cases, this lightweight framework achieves comparable results to NsCE, particularly on relatively simple datasets. However, we must acknowledge that such approaches heavily rely on the selection of pre-trained models. For detailed guidelines on selecting pre-trained models, please refer to Appendix C.3.\n\nQ: _Some typos, missing citations and experimental results in main text._  \nA: We feel sorry for any confusion or inconvenience for you. We will cite the mentioned paper in the main text and carefully proof-read our text to ensure that no grammar mistakes or typos still exist.\n\nQ: Reasons behind the discrepancy between $A_{AUC}$ and Last accuracy.  \nA: It should be noted that $A_{AUC}$ and last accuracy actually reflect different aspects of the model's performance. $A_{AUC}$ mainly assesses the real-time performance of the model, while the last accuracy measurement reflects the model's performance after processing the entire data stream. Our focus is on achieving a high-performance, anytime inference OCL model with minimal time and memory costs. Therefore, last accuracy is not our first priority. But even without employing data augmentation and knowledge distillation, our NsCE framework still achieves comparable results in last accuracy with SOTA OCL methods.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,  
We sincerely appreciate your time and effort in reviewing our manuscript and offering valuable suggestions. Based on some of the reviews, we provide a pdf including a figure showing the redefined weight sparsity value during the training with and without the sparsity-regularization term and displaying the impact of some methods (GSA,ACE,AML) which focus on re-arranged last layer weight updates as the comparison.

Due to the strict space limits, regarding how the proposed approach solves the introduced Myopia could be improved, whether the sparsity would be affected by methods that focus on re-arranged last layer weight updates and the evaluation metrics, we hope to provide a unified answer here.

Q: _How the proposed approach solves the introduced Myopia  could be improved_  
A: We first appreciate your insightful question and suggestion to create a graph showing the weight sparsity with and without the sparsity-regularization term. We have included this in Figure 1 of the attached PDF. It can be seen that the proposed sparse regularization term $\mathcal{L}_s$ prevents the classifier from becoming excessively sparse during training. Additionally, we want to clarify that myopia is addressed by the overall design of our proposed method including the sparsity-regularization term, NsCE, as illustrated in Figure 5 of our main text. From the visualization, it can be seen that our model quickly learns the current task while minimizing confusion between past categories and those in the current task.

Q: _Whether the sparsity would be affected by methods that focus on re-arranged last layer weight updates_  
A: After the reviewer's reminder, we are also very interested in how sparsity would be affected by methods focusing on re-arranged last layer weight updates. After implementing ER-ACE and ER-AML, we found that the phenomenon of parameters rapidly becoming sparse is indeed somewhat mitigated, though not as significantly as with our proposed regularization term $\mathcal{L}_s$, as illustrated in Figure 1 of the attached PDF. Additionally, incorporating ACE or AML can also boost performance for baselines like ER and SCR. We believe that when ACE and AML nudge the learned representations to be more robust to new future classes, they indirectly decrease the sparsity of the model parameters.

For GSA, the sparsity is not affected. Due to very limited time, we are not entirely sure whether this part is perfectly embedded or if further tuning would help, as the authors only provide hyperparameters for CIFAR-100. For SS-IL, we did not find its implementation, so it may be left for future works.

Overall, re-arranging the latter layer weight updates is an interesting and important problem in the area of OCL. We genuinely appreciate this insightful suggestion and plan to conduct detailed exploration of the dynamics of latter layer weight parameters during training in future works. This may lead to more intuitive designs.

Q: Justification behind $A_{AUC}$ (Area under the Accuracy Curve) but not Average Accuracy $A_{avg}$  
A: At this point, we respectfully disagree with the reviewer's opinion. Compared to $A_{avg}$, $A_{AUC}$ is typically perceived as a more suitable and modern evaluation metric for the OCL scenario, especially in boundary-free settings like ours. It can be seen as a refined version of $A_{avg}$ to prevent misinterpretation on model's real-time performance. $A_{AUC}$ was first proposed by [1] and has been widely adopted as a crucial metric in existing OCL literature to substitute the old commonly used $A_{avg}$, such as [2], [3], [4]. It helps address the limitation of $A_{avg}$, which only provides evaluation about model's performance at specific task transition points, usually occurring only 5-10 times in most OCL setups.

$A_{AUC}$ measures accuracy more frequently—at least 20 times—by evaluating it after every $\Delta n$ samples ($\Delta n=500$ for EuroSat). Instead of taking an average on these observed accuracy, $A_{AUC}$ computes the area under the accuracy-to-number-of-samples curve $A_{AUC}=\frac{1}{n}\sum_{i=1}^t acc(i \cdot \Delta n) \cdot \Delta n$ (actually similar to the average taken on more observations $A_{avg}=\frac{1}{t}\sum_{i=1}^tacc(i\cdot \Delta n)$), providing a more precise evaluation than simply taking average accuracy ($n$ denotes the total number of training data). As illustrated by the following table (we directly copy the performance of RM[5] in [1]), a higher $A_{AUC}$ typically induces higher $A_{avg}$ and good real-time inference performance, but a high $A_{avg}$ does not necessarily represent good real-time inference performance. We highly respect reviewer's suggestion and plan to include a more detailed discussion including the comparison between these two metrics (including both numerical results and illustrations) in our revised version.

|     | CIFAR10 | CIFAR100 | TinyImageNet | ImageNet |
|  ----  | ----  |  ----  | ----  |----  |
| $A_{AUC}$ |23.00 $\pm$ 1.43|8.63 $\pm$ 0.19|5.74 $\pm$ 0.30|6.22|
| $A_{avg}$ |61.52 $\pm$ 3.69 |33.27$\pm$ 1.59|17.04 $\pm$ 0.77|28.30|


[1] H. Koh, D. Kim, J.-W. Ha, and J. Choi. Online continual learning on class incremental blurry task configuration with anytime inference. ICLR, 2022.  
[2] Moon J Y, Park K H, Kim J U, et al. Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning ICCV 2023.  
[3] Koh H, et al. Online boundary-free continual learning by scheduled data prior. ICLR, 2023.  
[4] Seo M, Koh H, Jeung W, et al. Learning Equi-angular Representations for Online Continual Learning. CVPR, 2024.  
[5]Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. CVPR, 2021

Q: _Definition of sparsity $s(w)$._  
A: We redefine sparsity score as $1/s(w)$ so that parameters with higher sparsity will have a higher sparsity score as shown in the attached PDF.

Meta-review: This paper identifies some key issues in online continual learning, such as model ignorance and myopia, and offers ways to mitigate these using a non-sparse classifier evolution framework which allows efficient learning of globally discriminative features.

All the reviewers appreciate the paper for the careful study of the challenging in online continual learning and there was a consensus towards acceptance. The reviewers had some questions and the authors' rebuttal seems to have addressed those and the reviewers seem satisfied with the response. 

Given the unanimously positive reviews, I recommend the paper for acceptance.
Decision: Accept (poster)