Adaptive Retention & Correction: Test-Time Training for Continual Learning
Authors: Haoran Chen
Micah Goldblum
Zuxuan Wu
Yu-Gang Jiang
Venue: ICLR 2025 Poster (2025)
Keywords: Continual Learning; Computer Vision; Transfer Learning
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Continual learning, also known as lifelong learning or incremental learning, refers to the process by which a model learns from a stream of incoming data over time. A common problem in continual learning is the classification layer’s bias towards the most recent task. Traditionally, methods have relied on incorporating data from past tasks during training to mitigate this issue. However, the recent shift in continual learning to memory-free environments has rendered these approaches infeasible. In this study, we propose a solution focused on the testing phase. We first introduce a simple Out-of-Task Detection method, OTD, designed to accurately identify samples from past tasks during testing. Leveraging OTD, we then propose: (1) an Adaptive Retention mechanism for dynamically tuning the classifier layer on past task data; (2) an Adaptive Correction mechanism for revising predictions when the model classifies data from previous tasks into classes from the current task. We name our approach Adaptive Retention & Correction (ARC). While designed for memory-free environments, ARC also proves effective in memorybased settings. Extensive experiments show that our proposed method can be plugged in to virtually any existing continual learning approach without requiring any modifications to its training procedure. Specifically, when integrated with state-of-the-art approaches, ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R datasets, respectively

Published as a conference paper at ICLR 2025
ADAPTIVE RETENTION & CORRECTION: TEST-TIME TRAINING FOR CONTINUAL LEARNING
Haoran Chen 1.2 Micah Goldblum Zuxuan Wu
Yu-Gang Jiang'
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University 2 Shanghai Collaborative Innovation Center of Intelligent Visual Computing 3 New York University
ABSTRACT
Continual learning, also known as lifelong learning or incremental learning, refers to the process by which model learns from stream of incoming data over time. A common problem in continual learning is the classification layer's bias towards the most recent task. Traditionally, methods have relied on incorporating data from past tasks during training to mitigate this problem. However, the recent shift in continual learning to memory- environments has rendered these approaches infeasible. In this study, we propose a solution focused on the testing phase. We first introduce a simple Out-of-Tart Detection method, OTD, designed to accurately identify samples from past tasks during testing. Using OTD, we then propose: (1) an Adaptive Retention mechanism for dynamically tuning the classifier layer on past task data; (2) an Adaptive Correction mechanism for revising predictions when the model classifies the data from previous tasks into classes of the current task. We name approach Adaptive Retention & Correction (ARC). Although designed for memory-free environments, ARC also proves effective in memory- based settings. Extensive experiments show that our proposed method can be plugged in to virtually any existing continual learning approach without requiring any modifications to its training procedure. Specifically, when integrated with state-of-the-ae approaches, ARC achieves an average performance increase of 2.7% and 2.6% on the CIFAR-100 and Imagenet-R R datasets, respectively. Code is available at Github Link.
INTRODUCTION
Deep learning has seen remarkable advances over the past decade (Krizhevsky et al., 2012; Ren et al., 2015; He al., 2016; Ronneberger et al., 2015; Long al., 2015; Dosovitskiy et al., 2021), largely due to the availability of large static datasets. However, real-life applications often require continually updating models on new data. A major challenge in this dynamic updating process is the phenomenon of catastrophic forgetting (Kirkpatrick et al., 2016; French, 1999), where a model's 's performance on previously learned tasks drastically degrades. To address this issue, various continual learning (CL) methods (De Lange et al., 2021; Hadsell et al., 2020; Mai et al., 2022) have been developed to provide a more sustainable and efficient approach to sequential model adaptation.
The community has identified two primary sources of catastrophic forgetting: 1) overfitting of the feature extraction network on new tasks (Li and Hoiem, 2017: Kirkpatrick et al., 2016; Rebuffi et al., 2017); and 2) bias towards new classes introduced by the linear classifier (Wu et al., 2019; Chrysakis and Moens, 2023; Belouadah and Popescu, 2019). Throughout the years, most works have been focusing on addressing forgetting of the representation layer. More recently, however, Zhang et al. (2023) show that for pretrained models, using a small learning rate for the representation layer and a larger learning rate for the classifier can effectively mitigate the overfitting issue, thereby relieving catastrophic forgetting. In this work, we take step further and show that even for models overfitted on the latest task, the amount of information lost on the representation layer is minimal. As such, we primarily focus on tackling the latter issue, i.c., mitigating classifier bias.
Corresponding author.

---

Published as a conference paper at ICLR 2025
Latest
95
90
Accurccy
85
80
Shared Classifier
Independent Classifier
Start
A
Ao
Aa
Aa
A1
Ag A1o
40
120
140
Figure I: Toy example using pretrained VIT. Since the backbone network is fixed, the only cause of performance degradation is the bias introduced by the classifier.
Distribution of incorrect prediction for first task samples after training on the latest task. As is shown, the predictions are skewed the top, indicating the presence classification bias.
Figure 2:
First studied by Wu et al. (2019), classifier bias relates to the fact that the final classification layer tends misclassify samples from previous tasks into classes from the current task due imbalanced training data. To illustrate, we present a toy example in Figure I, where we perform linear probing on Split Imagenet-R using a pretrained ViT (Dosovitskiy et al., 2021). The dark blue line in the graph shows the performance when an independent classifier is trained for each specific task, and the model selects the appropriate one for testing. The light blue line shows the performance of testing with only single shared classifier that expands to accommodate new classes with each new task. As shown, the model experiences almost no performance drop when evaluated using independent classifiers, whereas there is a severe performance drop when using shared classifier. To further demonstrate the cause of this performance degradation, we show Figure the distribution of incorrect predictions for data from the first task. It is evident that the classifier tends to favor classes of the current task, highlighting the inherent bias in the predictions of the shared classifier.
To address performance degradation in the classification head, previous methods (Wu al., 2019; Belouadah and Popescu, 2019. 9; Chrysakis and Moens, 2023) often rely on including an additional memory dataset when training. However, concerns prompted recent CL approaches (Wang et al., 2022d;c; Smith el al., 2023) to develop methods that operate without memory, making these memory-dependent methods less appealing. Nonetheless, notice that an often-overlooked aspect of continual learning is the availability of data from previous tasks during inference. This availability offers unique opportunity to retrospect prior knowledge without the need for any additional memory storage.
computational costs and privacy have
In practice, we often do know what task samples come from when they arrive. Therefore, our first step involves developing effective method to correctly identify data from past tasks. Drawing inspiration from Dfist--Distriittion (OOD) detection (Liang et al., 2017: DeVries and Taylor, 2018), we propose simple Out-of-Task detection method, OTD, by leveraging the model's output confidence and predicted class. Specifically, OTD built on top of the following two empirical findings:
1. Predicting that a test sample comes from past task with high confidence generally indicates accurate classification of data from previous task.
2. Predicting that a test sample comes from the current task with low confidence suggests likely misclassification of data from previous task.
With these two categories predictions, we design two independent approaches tailored to each. Specifically, we propose Adaptive Retention & Correction (ARC) which consists of:
Adaptive Retention, where we utilize accurately classified samples from previous tasks and re-balance the classification layer in online manner with one gradient update.
2. Adaptive Correction, where we correct predictions made on potentially misclassified samples from previous tasks by adjusting the classifier's preference for the most recent task's classes.
2

---

Published as a conference paper at ICLR 2025
Given that ARC does not interfere with model training, evaluate its compatibility with existing methodologies. To this end, we conduct extensive experiments on popular continual learning bench- marks by integrating ARC into both classical and state-of-the-art methods in the class-incrent incremental setting, and show that ARC boosts their performance by a large margin. While designed for the memory-free scenario, our experiments show that ARC is equally effective using memory not. Specifically, on the Split CIFAR- dataset, incorporating ARC can increase the average accuracy by 2.7%, with DER (Yan al., 2021) showing the most significant improvement of 4.5%. On the Split Imagenet-R dataset, ARC can increase the average accuracy by 2.6%, with iCarl. (Rebuffi et al., 2017) exhibiting the largest increase of 6.6%. In summary, our contributions are three-fold:
We show that the strategic use of output confidence levels together with the output class can help distinguish between data from past tasks that is correctly classified and data from past tasks that misclassified into classes from the current task, and propose OTD, a simple Out-of- Task detection method.
We leverage the fact that models can interact with samples from past tasks during inference and propose Adaptive Retention & Correction (ARC). where the classifier weight is re- balanced and predictions on potentially misclassified images are rectified.
Extensive experiments show that integrating ARC into both traditional and state- offic-o--thh-artt methods produce significant improvements, with an average increase of 2.7% on Split CIFAR-100 and an average increase of 2.6% on Split Imagenet-R.
2
RELATED WORKS
2.1 CONTINUAL LEARNING
Continual Learning (CL) (Kirkpatrick et al.. 2016; Li and Hoiem, 2017; Dhar et al., 2019; Hung et al., 2019; Douillard al., 2022) has been an area of active research aiming enable models learn continuously from stream of data while retaining previously acquired knowledge. A significant challenge in CL. is catastrophic forgetting (French, 1999), where a model loses previously learned information upon learning new data. To address this notorious phenomenon, a plethora of approaches have focused specifically on tackling classifier bias. For example, BiC (Wu et al., 2019) introduces an additional bias correction layer that trains on validation dataset composed of both data from old tasks and data from the current task. IL2M (Belouadah and Popescu, 2019) directly adjusts the logit values for classes of the current task using learned statistics. OBC (Chrysakis and Moens, 2023) proposes re- reightig the classifier layer based on the memory dataset.
Recently, the field of CL has witnessed a paradigm shift influenced by the exponential growth in model scales, where researchers are increasingly leveraging large pretrained models without any memory data (Wang al., 2022e; Smith et al., 2023; Wang al., 2022d; Chen et al., 2023). Within this context, prompt-based methods have gained significant attention. For example, L2P (Wang al., 2022d) introduces prompt pool from which task-specific prompts are dynamically selected. Dualprompt (Wang et al., 2022c) extends the idea by leveraging both task-specific and task-agnostic prompts. However, since traditional methods on tackling classifier bias mostly rely on leveraging data from the past task, methods like BiC can't 't operate under the current memory-free paradigm. To address this, a most recent work SI.CA (Zhang et al., 2023) proposes to store the sample means and covariance matrix, and models the class feature as Gaussian distribution. After full training on each task, the classifier is further balanced on features generated from it.
2.2 OUT OF DISTRIBUTION DETECTION
Out-of-distribution (OOD) detection (Yang et al., 2021; Salehi et al., 2021) is a technique used in machine learning to identify data points that differ significantly from the training data distribution. These data points, known as out-of-distribution samples, do not belong to any of the classes seen during training. OOD detection methods typically involve using score function to measure the uncertainty or confidence of the model's predictions and flagging inputs with low confidence as auttof-diitribution. For example, maximum softmax probability (MSP) is used as the score function by Hendryeks and Gimpel (2016). Following this, the subsequently introduced ODIN (Liang et al., 2017) established solid baseline method for the OOD detection community. Recently, due their
3

---

Published as a conference paper at ICLR 2025
1.0
1.0
1.0
0.8
0.8
Accuracy
0.6
0.6
0.6
0.4
0.4
0.4
VIT-S
Flneure VIB86166
Plactane VITAL/16
0.2
S
0.2
BrG
0.2
Pubke
Joint Triring
Juin Traising
Joini Trising
0.0
0.0
0.0
2
9
10
2
10
2
10
1.0
1.0
1.0
0.8
0.8
0.8
Accuracy
0.6
0.6
0.6
0.4
0.4
0.4
Pizeline Remeels
Findlng Benatt0
Finzline Remett0)
0.2
0.2
0.2
Poree Recan
Joint Trining
Joun Trahingg
Joiei Triiing
0.0
0.0
0.0
2
8
9
10
2
2
# of Tasks Seen
Tasks Soen
Tasks Seen
Figure 3: Performance of linear probing with various model architecture and size.
close approximation to probability density, energy score based score functions (LeCun et al., 2006; Wang al., 2022a) have gained significant prominence. Relating to continual learning, when models train on data of the most current task, data from the previous tasks can be seen as out-of-distribution data, providing us the opportunity to identify these data using OOD techniques.
2.3 TEST TIME ADAPTATION
Test Time Adaptation (TTA) focuses adjusting the model deployment accommodate and possibly shifting data distributions using unlabeled test data Liang et al. (2023); Kundu et al. (2020); Schneider al. (2020); Sun al. (2020). In TTA, one of the representative methods is the usage of pseudo-labels (Arazo et al., 2020). For example, in SHOT (Liang et al., 2020), class centroids are obtained by calculating weighted sum of feature vectors similar k-means clustering. Then, pscudo-labels are assigned to each test sample according to nearest centroid classifier. Recent works also investigated more practical situation of continual test time adaptation (Song et al., 2023), where models are continuously adapted to different target environments. For example, COTTA (Wang et al., 2022b) introduces weight-averaged and augmentation-averrae pseudo-labels and leverages teacher model to produce more accurate predictions. However, few works have tried to directly apply TTA approaches to enhance CL methods, and we are among the first to analyze such possibility.
METHODS
3.1 PRELIMINARY NOTATION
In this work, we primarily consider the slass-incremental learning problem in the image classification setting, where image data arrives sequentially and models have no access to the task identity during consider training the model where each task has a corresponding dataset denoted by D(X' y*). In class-incremental learning. ) n 9 and P: # P, for i # j. where P; denotes the probability distribution over samples from task i. In other words, each task has distinct classes and a distinct data distribution. A common assumption in class-incremental learning is that the incremental size, i.e., number of classes in each task, remains constant. In this work, we denote this size by 8.
inference. Formally,
total of N tasks T = (Ti, ..,, T..,,,)
We define models by
Ma(-) hag (f (-))
(1)
with parameters 0 (0sps Sem, (+) represents the feature extractor parameterized by 0pps- and hag (.) denotes the classification layer parameterized by Oas-
4

---

Published as a conference paper at ICLR 2025
3.2 UNDERSTANDING CATASTROPHIC FORGETTING IN PRETRAINED MODELS
Recent developments in large pretrained models have prompted the continual learning community to also shift towards using these pretrained models, leading to a plethora of well-established works. However, whether the learning dynamics change in this context is not well understood. In this work, we find that the amount of information lost when adapting to a new task surprisingly minimal. This contrasts with the prevailing expectation in current literature that forgetting would be primarily attributed changes in the backbone of the model, especially since most current works on pretrained models focus heavily on representation layers. To justify this statement, we leverage models that are fully trained on the latest task and conduct linear probing on data from each preceding task. Here, different from Zhang et al. (2023), we do not pose any constrains on the learning rate, allowing the model to fully adapt to the incoming task.
Specifically, train models the Imagenet-R R dataset with an incremental step of 20, meaning that there is a total of 10 tasks. After finished training on each new task T,, we perform linear probing on the learned model using data from earlier tasks T; (i < n.). We experiment with different architectures and sizes, including ViT-S/16, ViT-B/16, ViT-L/16, and ResNet-18, 18, ResNet-50, ResNet- 101. No memory is used throughout the process. The results shown in Figure 3. We see here that regardless of the specific model architecture or size, the performance of linear probing consistently achieves high performance comparable joint training. Since linear probing keeps the representation layer fixed, we conclude that the representation layers of pretrained models are adept at retaining knowledge as new tasks are introduced, and they clearly are not the primary source of catastrophic forgetting. Thus, we argue that the key issue for the large performance gap between finetuning and linear probing lies the bias introduced by the classifier, leading us to the proposed approach ARC.
3.3 OUT-OF-TASK DETECTION
In the following, we will build up a method for de-biasing model's predictions towards the current task. As briefly discussed in the introduction, test samples arrive with no task identity, so the first step towards de- de-biasing the model's 's predictions will require accurately distinguishing between data from previous tasks and data from the current task. To do so, we will use the model's own predictions to distinguish cases where test sample comes from previous task or the current task.
Case 1. For data classified into the classes of past tasks, if the model, after training on the current task, still produces tigh-coofidence predictions, it is highly likely that this data indeed belongs to past tasks. Based on this, we propose the following assumption:
Assumption 1. At task To for test image æ with output logits E IR let 3 be pre-defined threshold value. If:
exp
arg mna > Ej=1 exp
S.. (t 1)
and
exp
max a- 15iSet
3,
EØ=1 exp
then we assume a sample from the past task that is correctly classified.
Case 2. For data classified into classes of the current task, we need to take more care when considering the model's confidence level. Due to the presence of classifier bias, low-confidence prediction likely indicates that the test sample comes from a previous task and is misclassified into the current task. However, there is also the likelihood of it being a particularly difficult sample from the current task. To differentiate between the two situations, we introduce a new metric w specifically designed for continual learning, and propose the following assumption:
Assumption 2. At task Tr, for a test image x with output logits 2 E R let y be a pre-defined threshold value. If:
exp 2i
arg max
> 8 (1 -1-1 1)
Σe exp Zj
and
exp Z
7, where è
max ISi
55 exp 2;
Dj=1
then assume is sample from previous tasks misclassified to the current task.
5

---

Published as a conference paper at ICLR 2025
Algorithm 1 Adaptive Retention & Correction (ARC)
Inputs: Current task identity Incremental step size s; Testing image æ: Initially predicted class ÿ: Adaptive Retention threshold 3; Adaptive Correction threshold y; Task-based Softmax Score temperature nettorrk Mo(-) he... (f0rpa (+)) with parameters 0 1(rpss Oets
1: if g < s (t 1) then
2: Obtain confidence c of Assumption 1. if ee>. / then
3:
4:
Update Gct with Equation (3). Recalculate y using the updated classifier.
5:
6:
end if
7:
else
8:
Obtain of Assumption
9: if w < y then
Calculate TSS (Si, S2,..., according to Definition Modify the output according to task T;, arg max; S.
11:
12: end
13:
end if
Here, è represents the highest predicted class probability when masking all the logits correspond- ing to classes of the current task. The intuition behind this design is that by isolating the logits specific to past tasks from those associated with the current task, the model is when predicting æ class from past tasks verses as a class from the current task. Consequently, if the ratio of and is low, this suggests the model's confidence that is from the current task is significantly lower than the model's confidence that a is from past task, and we can assume with high probability that is indeed misclassified sample from a past task.
get sense of how confident
3.4 ADAPTIVE RETENTION & CORRECTION
Based on OTD, we design two independent approaches, Adaptive Retention and Adaptive Correction, that leverage the distinct characteristics of the data following Assumption and Assumption 2 to mitigate classifier bias. Together, they form the proposed Adaptive Retention & Correction approach. overview of ARC shown in Algorithm
3.4.1 ADAPTIVE RETENTION
We first introduce Adaptive Retention using data following Assumption 1. Traditional methods for reducing classifier bias mainly rely on training with data from the past task and are not applicable in memory- free situations. However, by using OTD, we can still leverage data from past tasks, but this time adaptively during the testing phase. To do so, we utilize the predicted labels as supervisory signals and train the classifier with the backbone network frozen using cross-entropy loss. In this way, the tendency within the classifier towards classes of the current task can be effectively mitigated by re-balancing weights towards previous classes. Furthermore, since predicted labels inevitably introduce errors and uncertainty, we employ an additional entropy minimization loss. This approach enhances the model's confidence in its predictions, thereby offsetting the adverse effects of noise and ensuring more reliable outputs during testing. Finally, to align more closely with practical applications where test samples arrive in an online manner, we choose train the classifier with only one gradient update on each sample.
Formally, for test sample with predicted label and following Assumption obtain the cross-entropy and entropy minimization loss:
LCE = -jj log and LEM = -En log
where Pe the predicted probability for class Se. and train the classifier using the combined objective:
LCE LEM-

---

Published as a conference paper at ICLR 2025
3.4.2 ADAPTIVE CORRECTION
We now develop method correct the model's predictions for test samples following Assumption 2. A naive approach would be to mask out the probabilities corresponding to the current task classes and reassign the sample to the class with the highest probability among the remaining ones. However, such a method defaults to treating all past tasks equally, which we find suboptimal. In reality, in addition to the tendency towards the current task, there also classification bias among all past tasks, meaning that the classifier would again favor more recent past tasks than earlier ones. To illustrate this phenomenon, again, we refer to the example shown in Figure 2. As is clearly demonstrated, among all the past tasks, the model tends to misclassify images from task Ti into recent past tasks, ie., task Tg. T's and Ty.
Nonetheless, the aforementioned issue can be addressed through straightforward extension of the naive approach. We first present the following definition:
Definition 1. At task Tú for a test image with the model output logits IR and a scaling temperature T, we define Task-based Softmax Score (TSS) for each task T,, E {1..... t] as
exp exp2k/ 2k/ (
S
max
Ej=1 exp 23 T
with the purpose of measuring our confidence that the test sample a originates from task T,. The rationale behind this score similar Assumption 2 in that by isolating the logits specific each task i from those associated with subsequent future tasks, we can minimize classification bias both between past and current tasks and among past tasks themselves, ensuring that the scores obtained can more accurately reflect the true probability for the sample belonging to each task. We also scale the logits using a temperature T > 1 to accommodate the imbalanced denominator for different tasks. Finally, we adjust the prediction accordingly.
4
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We conduct our main experiments on two popular benchmark datasets for continual learning: Split CIFAR-I00 and Split Imagenet-R. CIFAR-I00 is a relatively simple dataset comprising 100 classes. Imagenet-R, on the other hand, includes 200 classes with data sourced from various styles such as cartoons, graffiti, and origami. The presence of both semantic and covariate shifts makes one of the most challenging datasets for continual learning. We also conduct experiments on another challenging benchmark, 5-dataset, where unlike the first two, it is a composite of five distinct image classification datasets: CIFAR-10, MNIST, FashionMNIST, SVHN, and notMNIST. The diversity in data distribution across these datasets offers us our method.
comprehensive evaluation of
Dataset Split. We use 'Inc-n' to denote the data split setting, where denotes the number of classes for each incremental stage. For example, Split CIFAR- 100 Inc5 means that there are 5 classes for each training step, resulting in total of 20 tasks. For fair comparison, split the classes following the order of Sun et al. (2023).
Evaluation metrics. In this work, we mainly use the conventional metrics Average Accuracy and Forgetting. Here, Average Accuracy reflects the overall performance of the model on all tasks after finished sequential training, and Forgetting measures the degree of performance decline on previously learned tasks. Formally, let RG, be the classification accuracy of task T; after training on task Tū, then Average Accuracy Ag for task Tr is defined as
T
AB =
Rrs
(4)
T
and Forgetting is defined as
T-1
F= T T-1
Σ
(Ri,5 - RI.).
(5)
1=1
7

---

Published as a conference paper at ICLR 2025
Method
Memory-Free
Split CIFAR-I00 Inc5
Split CIFAR-100 Inc10
Split Imagenet-R Incl0
Split Imagenet-R Inc20
Ag
J
As
Ag
F
As
Finetune
27.2
68.4
29.8
41.3
61.5
28.6
ARC
66.0 (+1.4)
24.7 (-2.5)
71.2 (+2.8)
10.8 (-19.0)
50.2 (+1.4)
35.3
(-6.0)
63.9 (+2.4)
15.2 (-13.4)
iCarl
74,9
24.6
78.7
21.0
56.5
34.3
61.0
29.0
ARC
81.7 (+6.8)
9.6 (-15.0)
81.8
(-15.4)
62.3 (+5.8)
(-1..1)
(+6.6)
14.4 (-14.6)
Der
X
70.7
27.7
79.6
18.9
64.3
29.6
76.7
14.7
ARC
(+3.3)
(:5.4)
84.1 (+4.5)
10.3 (-8.6)
(+1.7)
6.4
Memo
72.3
24.7
76.3
21.5
61.9
25.8
66.2
21.1
ARC
74.7 (+2.4)
16.5 (-8.2)
80.0 (+3.7)
10.3 (-11.2)
63.1 (+1.2)
17.7 (-8.1)
68.2 (+2.0)
11.9 (-9.2)
83.2
69.9
5.4
72.6
4.2
ARC
82.8 (+4.7)
8.0 (-2.0)
86,2 (+3.0)
5.7 (-3.1)
73.3 (43.4)
4.7 (-0.7)
75.1 (+2.5)
3.5 (-0.7)
DualPrompt
77.1
8.1
82.0
7.5
5.9
69.1
4.7
ARC
83,4 (+6.3)
4.4 (3366)
85.0 (+3.0)
3.8 (.3.7)
68.7 (+2.7)
4.2 (-1.7)
70.5 (+1.4)
3.5 (-1.2)
CodaPrompt
79.6
5.7
86,9
5.1
70,5
5.6
73.2
5,7
ARC
(+1.8)
5.8 (+0.1)
88.0
(-1.0)
(+2.1)
2.9
(+2.2)
2.0
SLCA
89.3
7.8
91.1
6.2
75.1
12.8
78.7
9.1
+AC
90.4 (+1.1)
5.9 (-1.9)
91.8 (+0.7)
(-1.9)
80.4 (+5.3)
5.3 (-7.5)
81.3 (+2.7)
3.5 (-5.6)
Table 1: Results of incorporating ARC into both classical and state-of-the methods on Split CIFAR-100 and Split Imagenet-R, We mainly use the implementation provided by Sun et al. (2023).
Implementation details. We use the repository provided by Sun et al. (2023) of our method. This repository is a pre-trained model- based continual learning toolbox consisting of both classical and most recent state-of-the-an methods. Specifically, we test ARC by incorporating it with Finetune, iCarL (Rebuffi et al., 2017), Memo (Zhou et al., 2022), Der (Yan et al., 2021), L2P (Wang et al., 2022d), DualPrompt (Wang et al., 2022c), CodaPrompt (Smith et al., 2023), and SLCA (Zhang et al., 2023). Finetune, iCarL, Memo, and Der are equipped with memory buffer of 2000 and 4000 for Split CIFAR-100 and Split Imagenet-R, respectively, whereas L2P, DualPrompt, CodaPrompt and SLCA are memory-frees For all tested methods, utilize ViT-b- 16 backbone pretrained on Imagenet-21K We set the training configuration to be the same as provided.
the effectiveness
4.2 COMBINING ARC WITH STATE-OF-THE-ARY CONTINUAL LEARNING METHODS
Results on benchmark with shared distribution. We report results on Split CIFAR- --0 and Split Imagenet-R in Table 1. These two datasets, in their entirety, are drawn from the same underlying distribution. We test our methods on both short and long sequence where there is total of 10 and 20 tasks, respectively. We see here that integrating ARC can significantly improve the Average Accuracy and reduce the Forgetting of state-of-th the-art methods. Specifically, for short sequence, all methods experience average Ag boost of 2.7% on both Split CIFAR- 100 and Split Imagenet-R. Similarly, the drop for F reaches an average of 8.0% and 7.0%. On the other hand, for long sequence, all methods experience an average Ag boost of 3.5% and 3.0%, and average FF drop of 4.8% and 5.5%, respectively. We note that the overall improvements is more significant on long sequence, indicating that ARC is more compatible with scenarios where the task number is large.
Method
5-dataset Inc10
Method
Split CIFAR-100 Incl0 As
Split Imagene-R Inc20 As
As
F
iCarl.
78.7
21.0
61.0
iCarL
89.8
5.0
TENT
80.1 (+14)
12.4 (-8.6)
61.6 (+0.6)
21.9 (-7.1)
+ +ACC
91.9 (+2.1)
2.9 (-2.1)
+ +ACC
81.8 (+3.1)
5.6 (-15.4)
67.6 (+6.6)
14.4 (-14.6)
Der
79.6
18.9
76.7
14.7
Der
95.9
1.1
TENT
81.8 (42.2)
16.0 (-2.9)
77.1 (40.4)
11.2
+ AACC
96.8 (+0.9)
(-1.0)
+ ARC
84.1 (+4.5)
10.3 (-8.6)
78.4 (+1.7)
6.4 (-8.3)
Memo
91.9
3.8
Mcmo
76.3
21.5
66.2
TENT
77.1 (40.8)
18.9 (-2.6)
65.8 (-0.4)
21.3 (40.2)
+ ARC
93.0 (+2.1)
0.6 (-3.2)
+ ARC
80.0 (+3.7)
10.3 (-11.2)
68.2 (+2.0)
11.9 (-9.2)
L2P
82,5
4.9
83.2
8.8
72.6
4.2
TENT
86.0 (42.8)
4,9 (-3.9)
73.2
2.7(-1.5)
+ ARC
87.8 (+5.3)
2.4 (-2.5)
+ ARC
86,2 (43.0)
5.7 (-3.1)
75.1 (+2.5)
3.5 (-0.7)
Table 2: ARC 5-dataset.
Table 3: Performance comparison with TENT.
Results on benchmark with more diverse distribution. We report results on 5-dataset Table 2. Unlike the other two benchmarks, the data in 5-dataset exhibit much more diverse distribution. Still,
8

---

Published as a conference paper at ICLR 2025
incorporating ARC into existing methods leads to notable improvements, where Ag increases by an average of 2.6%, and decreases by an average of 2.2%. This highlights the robustness and efficacy of ARC across different benchmark scenarios, making it valuable for practical usage.
Comparison with TTA method. Since Test Time Adaptation (TTA) is a special type of model correction, we also report performance comparison with representative TTA method, TENT (Wang et al., 2020) in Table 3. As demonstrated, while TENT exhibits some performance improvements, the magnitude is consistently lower compared ARC. Specifically, on Split CIFARIO0 Ine 10, TENT's average performance gains are 1.8% lower than those of ARC, and on Split ImageNet-R Inc20, the gap mcreasess 2.9%. We hypothesize that there are two primary reasons for this. First, TENT, and similar TTA methods, adapt to all incoming test data, whereas our approach selectively adapts only to past task data. Second, TENT's adaptation is comparable to Adaptive Retention but lacks the ability to identify and correct potentially misclassified data from previous tasks, i.e., Adaptive Correction, further limiting effectivvness.
4.3 IN-DEPTH ANALYSIS OF ARC AND ITS PROPERTIES
Detailed contribution. We further present the specific contributions of Adaptive Retention and Adaptive Correction individually in Table 4. The average increase using Adaptive Retention is 1.8% on Split CIFAR- 100 and Split Imagenet-R, R, and the average increase using Adaptive Correction is 1.1% on Split CIFAR-I00 and 1.2% on Split Imagenet-R. Notably, the performance boost when applying Adaptive Correction to prompt-based methods is low. We find that the small size of this boost is due to the small number of test samples for these prompt based methods that satisfy Assumption 2. For example, on Split Imagenet-R, only 1.9% of test samples satisfy Assumption 2 for L2P, while for iCarL, 7.4% test samples satisfy this assumption.
Split CIFAR-100
Split Imagenet-K Ada. Ret.
Method
Split Imagenet-R
Ada. Self-Ret.
Ada, Self-Cor.
Ada. Cor.
Assump- Accuracy
Assump. Aceuracy
Pinetune
-2.4
+0.4
40.2
+2.3
Finetune
83,2%
70.7%
iCart.
2.99
10.2
14.6
11.9
iCarl.
92.4%
86.5%
-0.6
43.9
40.2
+1.5
Der
Memo
+1.4
+2.3
+1.4
40.6
Mema
85.6%
70.6%
L2P
-2.7
+0.3
+2.0
+0.2
L2P
87.7%
60.0%
DualPrompe
DuslPrompt
$1.6%)
68.6%
CODA-Prompt
-1.0
+0.1
+1.9
+0.3
CODA Prompt
92.8%
62.4%
SLCA
-0.4
+0.2
+2.0
+0.7
SLCA
92.1%
73.3%
Table 4: Specific contribution of Adaptive Reten- Table 5: Empirical validation for Assump- tions 1 and 2 on Split Imagenet-R.
tion and Adaptive Correction.
Empirical validation of assumptions 1 and 2. To validate Assumption and Assumption 2, we present the empirical results on Split Imagenet-R R in Table 5, where the reported results indicate how well our assumptions hold. The proportion of data following Assumption I that are indeed samples from previous tasks correctly classified by the model is 88.4% averaged on all methods. Likewise, the proportion of data following Assumption that are indeed samples from previous tasks misclassified to the current task is 71.9% averaged on all methods. These results demonstrate the overall effectiveness of the proposed assumptions.
4.4 ABLATION STUDIES
Adaptive Retention. We present in Figure 4a the ablation studies on the objective of the training process of Adaptive Retention, where the effect of the cross-entropy loss Le and the entropy minimization loss LFM on the Split CIFAR- 100 dataset is tested. As is shown, while for I.2P, the use of LCE results in a slightly worse performance, general, both are important components of Adaptive Retention.
Adaptive Correction. We further present in Figure 4b the ablation studies on our design of Adaptive Correction. We first validate the necessity of the scaling temperature value for TSS. We see that when it is excluded, Finetune, iCarL, Der and Memo would experience an average performance drop of 0.6%. We also test our design of for Assumption 2. Specifically, we examine whether over the two possible outcomes of low-confidence prediction is necessary. To do so, we compared against letting c 7, i.e., making represent the raw confidence for the predicted class. As is
concern

---

Published as a conference paper at ICLR 2025
Method
Loe
LEM
Average Accuracy
Method
T
Average Accuracy
79.6
70.5
86.6
Finetune
x
70.5
86.2
/
/
71.2
V
x
83.4
x
81.3
DualPrompt
X
V
84.5
iCarL
x /
/
81.6
V
85.0
V
81.8
X
87.1
/
X
83.2
CodaPrompt
X
86,6
Der
x
83.5
88.0
/
5
84.1
91.4
SLCA
91.7
Memo
X
79.7
91.8
80.0
(a) Ablation for Adaptive Retention.
(b) Ablation for Adaptive Correction.
Figure 4: Ablation studies for specific components of Adaptive Retention and Adaptive Correction on Split CIFAR-100.
80
80
ICarL
DualPromat
ICarL
DualFromot
78
78
Mama
Average Accuracy
76
Average Accuracy
76
74
74
72
72
70
70
60
66
0.55
0.60
0.65
0.90
0.95
0.9
1.0
Y
Figure Ablation study on threshold and 7 of Assumption and Assumption
demonstrated, such change would result in an average performance degradation of 0.5%, indicating the success of our specific design of w.
Hyperparameter. Figure 5 depicts the ablation study for hyperparameter 33 and Assumption and Assumption 2, where we experiment on of the tested methods (iCarL, DualPrompt, Memo and L2P). We test / ranging from 0.6 0.9, and y from 0.6 to 1.0. As clearly shown, the performance shows relatively small fluctuation when using different threshold values, again indicating the robustness of ARC.
CONCLUSION
In this study, we addressed the critical issue of classifier bias continual learning, especially in the current paradigm of leveraging pretrained models under memory-free settings, during the testing phase. first built an Out-of-Task detection method, where we identified two special cases of testing data: 1) data from previous tasks that is correctly classified and 2) data from previous tasks that is misclassified to the current task. Consequently, we proposed two independent approaches, Adaptive Retention and Adaptive Correction, specifically tailored for each case. Together they form approach Adaptive Retention & Correction (ARC). Adaptive Retention focuses on dynamically tuning the classifier layer and Adaptive Correction implements training-fres bias correction mechanism. By leveraging the ability to access past task data during inference, ARC effectively mitigates classifier bias without requiring any modifications the training procedure. Extensive experiments show that integrating ARC to existing methods can significantly boost their overall performance, demonstrating the effectiveness and robustness of our method.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Eric Arazo, Diego Ortego, Paul Albert, Noel E O' O'Connor, and Kevin McGuinness, Pseudo-labeling and confirmation bias deep semi-supervised learning. In IJCN, 2020. 4
Eden Belouadah and Adrian Popescu. 112m: Class incremental learning with dual memory. In ICCV, 2019. 1, 2,3
Haoran Chen, Zuxuan Wu, Xintong Han, Menglin Jia, and Yu-Gang Jiang. Promptfusion: Decoupling stability and plasticity for continual learning. arXiv preprint arXiv:2303.07223, 2023. 3
Aristotelis Chrysakis and learning. ICLR, 2023. 2,
Moens. Online bias correction for task-free continual
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting classification tasks. TPAMI, 2021. 1
Terrance De Vries and Graham W Taylor. Learning confidence for out-of-distribution detection in neural networks. arXiv preprint 1802::8802..4880 2018. 2
Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuai Peng, Ziyan Wu, and Rama Chellappa. Learning without memorizing. In CVPR, 2019. 3
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2021. 1,2
Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In CVPR, 2022. 3
Robert French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 1999. 3,3
Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and outtof-distribution examples in neural networks. In ICLR, 2016.
Ching- Yi Hung, Cheng- Tu, Cheng-En Wu, Chien-Hung Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. NeurlPS, 2019.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neural networks. PNAS, 2016. 1,
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. NeurlPS, 2012.
Jogendra Nath Kundu, Naveen Venkat, R Venkatesh Babu, et al. Universal source-free domain adaptation. In CVPR, 2020.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujic Huang. tutorial on energy-based learning. Predicting structured data, 2006. 4
Zhizhong Li and Derek Hoiem. Learning without forgetting. TPAMI, 2017. 1,3
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In /CMI., 2020. 4
11

---

Published as a conference paper at ICLR 2025
Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey test-time adaptation under distribu- tion shifts. arXiv preprint arXiv:2303.1536. 2023. 4
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribttion
image detection in neural networks. arXiv preprint arXiv:1706.0000 026900 2017. 2, 3
Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015. 1
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online
continual learning in image classification: An empirical survey, Neurocomputing, 2022. 1
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl:
Incremental classifier and representation learning. In CVPR, 2017. 1, 8
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object
detection with region proposal networks. In NeurIPS, 2015. 1
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015. 1
Mohammadreza Salehi, Hossein Mirzaei, Dan Hendrycks, Yixuan Li, Mohammad Hossein Rohban, and Mohammad Sabokrou. A unified survey on anomaly, novelty, open-set, and ouitt--distribution detection: Solutions and future challenges. arXiv preprint arXiv:2110.14a. 14051, 2021. 3
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. NeurIPS, 2020.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante- Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In CVPR, 2023. 2, 3, 8
Junha Song, Jungsoo Lec, In So Kweon, and Sungha Choi. Ecotta: Memory-efficient continual test-time adaptation via self-distileed regularization. In CVPR, 2023.
Hai-Long Sun, Da- Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Pilot: A pre-trained model-based
continual learning toolbox. arXiv preprint arXiv:2309.00 00000907717,,. 2023. 7.8 8
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In ICMI,, 2020. 4
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. 2020. 9
Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: logit matching. CVPR, 2022a. 4
with virtual-
Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In CVPR, 2022b. 4, 14
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Ya Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. arXiv preprint arXiv:2204.04799 04799, 2022c. 2, 3,
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In CVPR, 2022d. 2, 3
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR, 2019. 2,
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In CVPR, 2021. 8
12

---

Published as a conference paper at ICLR 2025
Jingkang Yang. Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized uuttof-distribbution detection: A survey. arXiv preprint arXi:2210..1334, 2021.
Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. Slca: Slow learner with classifier alignment for continual learning on pre-trained model. In CVPR, 2023. 1, 5,
Da- Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficient incremenaa learning, arXiv preprint 132/8 2022. 8
Da-Wei
13


Reviews:
[{"review_id": "aCZi7LLORK", "reviewer": "Reviewer_1cF7", "summary": "The paper presents a novel approach to class-incremental memory-free continual learning by integrating out-of-distribution detection with model self-correction. Specifically, the authors observe that performance degradation in continual learning largely stems from the classifier head and suggest adapting it on test samples using an entropy minimization loss while refining its predictions through a modified softmax score. Experimental results on Split CIFAR and Split ImageNet-R demonstrate the method's robustness and effectiveness across both replay-based and memory-free continual learning techniques.", "questions": "Did you run the hyperparameter search for the adaptive self-retention procedure on the same dataset that was used for evaluation?\n\nHave you tried applying your method to other continual learning algorithms, like BEEF, EASE, MEMO, and FOSTER?\n\nIn line 88, you formulate the two observations as empirical findings. However, in Section 3.3, they are called assumptions. Do you have any empirical results that show to what extent these actually hold?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The submission offers a comprehensive analysis of the dynamics of catastrophic forgetting, with a focus on recency bias, and proposes a simple and effective method to alleviate it that can be combined with any continual learning method.\n\nThe experimental evaluation is rigorous, covering relevant baselines, standard datasets, and appropriate ablations. The new approach demonstrates robustness to method-specific hyperparameter choices and provides a performance boost across continual learning algorithms.\n\nThe paper is well-written and structured, with clearly presented arguments, consistent notation, and high-quality figures that effectively convey the main points.", "weaknesses": "The main weakness of the paper is that it makes assumptions about the continual learning setting that are not well justified. Computational constraints are often more critical than memory limitations (see Prabhu et al. 2023, *Computationally Budgeted Continual Learning: What Does Matter* and Roth et al. 2024, *A Practitioner's Guide to Continual Multimodal Pretraining*). A significant limitation of the proposed method is its requirement for additional gradient updates during inference, even if only a single update per sample is needed.\n\nAdditionally, performing a hyperparameter search for the adaptive self-retention procedure could lead to overfitting on the test samples.\n\nTo improve the evaluation, it would be best to: a) quantify the computational cost of the self-retention procedure and b) ensure hyperparameter tuning is conducted on a separate dataset from the evaluation dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FdMX6wfoTX", "reviewer": "Reviewer_ALn4", "summary": "The authors observe that the main issue of forgetting is not in model itself but the classifier layer. Base on such observation, they introduce a way to balance the calssifying layer between current task and previous task. Instead of naively mask the current task output, the authors argue that it is not suitable since the balance between previous class is also imbalance. They come up with a new algorithm to tackle the problem by two assumptions, retention and correction. With the new algorithm ARC, it gain the performance on various datasets and methods.", "questions": "1. What if the incoming sample from the past task is not allow to use during inference? For example, it is deployed to the embbedding device that need real-time inference, will additional training add much overhead? Or is it possible to not update at every step? It will be interesting to provide such results.\n2. Can this approach extend to other computer vision task such as detection and segmentation? It would be great if the methods can also fix the problem in such scenario by re-balancing the classifcation head in object detection.\n\nMinor Correction\n1. At Table 1, it seems strange that for CodaPrompt, the position of the difference for Split CIFAR-100 Inc5 is oppositve. The difference sign should be 5.8 $\\textcolor{red}{(+0.1)}$ instead of 5.7 $\\textcolor{green}{(-0.1)}$.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The observation of forgetting majorly happen in classifer sounds reasonable and inspring. The proposed method, ARC, also solve it well. \n2. ARC can be generally applied to different methods is a big advantage that it can plug and play easily with existing countinual learning techniques.", "weaknesses": "1. Need to report the training time during inference, since it might add overhead when deployed to the real-world.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "11NTgV5V2f", "reviewer": "Reviewer_EsC3", "summary": "This paper addresses the challenge of reducing classification bias in class incremental learning (CIL) by utilizing test samples. The authors introduce a novel method called Adaptive Retention & Correction (ARC), which dynamically adjusts classifier layers using test samples confidently identified as belonging to previous tasks. Additionally, ARC corrects test samples that are mistakenly classified as part of the current task.", "questions": "1. The hyperparameters used in the reported results are not fully detailed. Were the same hyperparameters applied across different methods and datasets? It seems that $\\beta$ will influence the number of test samples used for training, and there is a trade-off between the number of samples and the accuracy of the pseudo-labels. How were the hyperparameters selected?\n\n2. In the discussion following Assumption 2, the authors state that a low ratio of $c$ and $\\hat{c}$ indicates a sample is more likely misclassified as belonging to a past task. However, Figure 5 suggests that a larger corresponds to better performance, which seems contradictory. Could the authors clarify this point?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- **Originality:** The method of adaptively adjusting and correcting model predictions using test sample confidence is novel. \n- **Quality:**  Leveraging confidence scores to detect past task samples and using these pseudo-labeled samples to train the model represents a reasonable strategy for mitigating classification bias in CIL.", "weaknesses": "1. While the motivation to correct misclassified past task samples is sound, the proposed Task-based Softmax Score (TSS) may be problematic. For the earlier tasks, as the temperature $T^{(t−i)}$ increases, the probability distribution flattens, which could hinder the model’s ability to recognize samples from past tasks.\n\n2. Although the experiments demonstrate the effectiveness of ARC, the paper lacks comparisons with other test-time adaptation (TTA) benchmarks, such as CoTTA. Additionally, the introduction of extra data for classifier training (referred to as retention) raises concerns about increased training costs that should be addressed.\n\n3. The paper lacks clarity in several areas, as detailed below:\n- 3.1 The description of the experimental setup in Figure 1 is inadequate. For instance, is the class-incremental learning setup applied in this toy example? How are the two types of classifiers trained and tested? Which task’s accuracy is measured in the figure?\n- 3.2 Does the ARC algorithm run after training each task? If so, what is the additional time required for training and inference? Additionally, should line 9 in the pseudocode be corrected to $w \\leq \\gamma$?\n- 3.3 The scaling temperature used in $S_i$ and the total number of tasks are both denoted as $T$. Are these referring to the same setting?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uVLz5mTd9i", "reviewer": "Reviewer_hK7E", "summary": "This paper proposes to reduce the task-recency bias present in the last layer when training in a Continual Learning environment. This well-known problem remains under-explored in the case of memory-free methods and presents additional challenges. In particular, the authors propose to leverage test data in order to adapt the weights of the last layer by differencing old and newer data using prediction confidence. The authors also propose an alternative to the softmax to rebalance the model’s logits, which as often bias toward later tasks. Eventually, this paper shows significant improvement by combining their approach with current state-of-the-art methods.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The idea of focusing on the debiasing the last layer for later prompt-based memory-free methods is interesting and under-explored\n- The intuition behind focusing on test-time adaptation by taking into context the specifics behaviour of continually trained model is good and intuitive\n- The method performances are appealing and it seems quite robust to hyper-parameter, which is crucial in Continual Learning where hyper-parameter search is unrealistic\n- Most sections are well-written and easy to follow", "weaknesses": "### Major weaknesses\n- There is a lack of detail on the overall training procedure. How many epochs are used for training? Is this correction and retention procedure applied after each task? In that case, this would correspond to storing unlabeled memory data.\n- I don’t think the formulae in definition 1 is well defined. The inequality under the max does not make much sense as $s(i-1) \\leq i \\leq s.i$ is impossible in many cases. Also, taking $T>1$ implies that the output of the softmax will be lower (flattened) for earlier tasks (low $i$ values), but I believe the objective is the opposite; the authors want to obtain higher softmax values for earlier tasks. I also do not really understand the usage of a maximum operator here. Overall, this definition is very confusing.\n- There is a lack of detail on the overall training procedure. How many epochs are used for training? Is this correction and retention procedure applied after each task? In that case, this would correspond to storing unlabeled memory data.\n- This method is connected to Test-Time Adaptation and some discussion in the related work would be appreciated. While TTA methods are rightfully compared in section 4.2, such discussion could be included more thoroughly in the related work.\n- The impact of the temperature is not presented in the paper\n- The notation for the temperature and the number of tasks is the same, $T$, which is confusing\n- in Figure 1, is the independent classifier trained for doing classification of one task only? Then is the problem a TIL problem? If this is the case, the performances are comparing a TIL problem to a CIL problem, which is unfair since the CIL problem is much harder than the TIL problem.\n- l. 202 the distribution are defined as different between tasks but in 4.2, the incremental dataset are defined as IID. I believe this is a mistake, can you elaborate?\n### Minor weaknesses\n- The experiments of figure 3 is quite unclear. It depends on the optimization strategy and the number of epochs for instance. Also, do you use the same fixed learning rate for joint training?\n- I believe a confusion matrix would be more clearer than Figure 1 at least, maybe even Figure 1 and 2\n- How much data do you use for linear probe?\n- I believe the difference in performances in figure 3 between finetune and probe makes sense and justifies the claim that the representation knowledge is still transferable across task.  However, I do not see why the joint training performances are lower in most cases. Could you elaborate?\n- the code is not shared\n### typos\n- The title is spelled wrong.\nIf the authors can clarify my concerns, I would happily increase my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)