Hyperparameters in Continual Learning: A Reality Check
Authors: Sungmin Cha
Kyunghyun Cho
Venue: Accepted by TMLR ()

Abstract: Continual learning (CL) aims to train a model on a sequence of tasks (i.e., a CL scenario) while balancing the trade-off between plasticity (learning new tasks) and stability (retaining prior knowledge). The dominantly adopted conventional evaluation protocol for CL algorithms selects the best hyperparameters (e.g., learning rate, mini-batch size, regularization strengths, etc.) within a given scenario and then evaluates the algorithms using these hyperparameters in the same scenario. However, this protocol has significant shortcomings: it overestimates the CL capacity of algorithms and relies on unrealistic hyperparameter tuning, which is not feasible for real-world applications. From the fundamental principles of evaluation in machine learning, we argue that the evaluation of CL algorithms should focus on assessing the generalizability of their CL capacity to unseen scenarios.  Based on this, we propose the Generalizable Two-phase Evaluation Protocol (GTEP) consisting of hyperparameter tuning and evaluation phases. Both phases share the same scenario configuration (e.g., number of tasks) but are generated from different datasets. Hyperparameters of CL algorithms are tuned in the first phase and applied in the second phase to evaluate the algorithms. We apply this protocol to class-incremental learning, both with and without pretrained models. Across more than 8,000 experiments, our results show that most state-of-the-art algorithms fail to replicate their reported performance, highlighting that their CL capacity has been significantly overestimated in the conventional evaluation protocol.

Published in Transactions on Machine Learning Research (04/2025)
Hyperparameters in Continual Learning: A Reality Check
Sungmin Cha New York University
sungmin.cha
Kyunghyun Cho
kyunglyyun.cho Cryy.
New York University & Genentech
Reviewed on OpenReview:
Abstract
Continual learning (CL) aims to train model on sequence of tasks (í.e., CL scenario) while balancing the trade-off between plasticity (learning new tasks) and stability (retaining prior knowledge). The dominantly adopted conventional evaluation protocol for CL algorithms selects the best hyperparameterss (e.g., learning rale, mini-batch size, regularization strengths, etc.) within a given scenario and then evaluates the algorithmss using these hyperparameters in the same scenario. However, this protocol has significant shortcomings: overostimates the CL capacity of algorithms and relies on unrealistic hyperparameler tuning, which is not feasible for real-world applications. From the fundamental principles of evaluation in machine learning, we argue that the evaluation of CL algorithms should focus assessing the gencralizability of their CL capacity to unseen scenarios. Based on this, we propose the Generalizable Two-phase Evaluation Protocol (GTEP) consisting of hyperparameter tuning and evaluation phases. Both phases share the same scenario configuration (e.g.. number of tasks) but are generated from different datasets. Hyperparameters of CL algorithuss are tuned in the first phase and applied in the second phase to evaluate the algorithms, We apply this protocol to class-incremental learning, both with and without pretrained models. Across more than 8,000 experiments, our results show that most state-of-the-art algorithms fail to replicate their reported performance, highlighting that their CL capacily has been significantly overestimated in the conventional evaluation protocol.
1 Introduction
In recent, years, extensive research has been conducted on continual learning (CL) with the goal of effectively learning knowledge from a sequence of tasks (Wang et al. 2023). A neural network model in such CL scenarios faces crucial tradc-off between learning new knowledge from novel tasks (plasticity) and maintaining knowledge previous tasks (stability) (Mermillod et al. 2013). To address this inherent trade-off, numerous algorithms have been proposed for successful CT. in various domains (Wang et al. 2023). In these domains, learning (class-
many CL studies have focused on classification, primarily concentrating on IL) (Masana et al., 2020) without or with pretrained models (Zhou et al.) 2024a). However, deploying CL. algorithms requires careful hyperparameter tuning. Figure illlstrates the conventional evaluation protocol, including hyperparameter (e.g., learning rate, mini-batch size, etc.) tuning, dominantly employed in both offline and online class-incrementall learning (Zhou et al. 2022 Boschini et al., 2022, Zhou al.,
2024b Smith et al. 2023 Seo et al.) 2024). Moreover, similar evaluation protocols have been extensively adopted in other arcas of CL, including semantic segmentation (Cha et al.] 2021b Yuan & Zhao 2024
test-time adaptation Yoo et al.]
2024 Lee et al. 2024), federated learning (Piao et al. 2024), self-supervised
learning (Fini et al. 2022 Cha et. al. 2024), and large language models (Ke et al. 2023
Wu et al. 2024
Many algorithms have been considered stato-of-tho-art based on performance validated through the conven- tional evaluation protocol. However, this raises two issues: First, the hyperparameter tuning method used in this protocol is not applicable to real-world CL. scenarios. Second, it results in evaluation overfitting to

---

Published in Transactions on Machine Learning Research (04/2025)
Conventional Evaluation Protocol for CL
CL Scenario
Hyper Parameter Tuning
Evaluation using H
Task
Task
Task
(DF D D
Train 0 1T
(Dir D
Train 0:7
D
Dir
Dir
D
Poat
Post
CL (H)
Eval.
CL (3
Eval.
D
Dic
Select best H
Generate
Reinit
Sample H
Report Evaluation Result
Dataset
CL scenario
Figure This figure illustrates the conventional evaluation protocol. First, a CL scenario is constructed using a benchmark dataset, where each task has its own training, validation, and test sets. Second, to find the best hyperparameters H", model is sequentially trained up to the final task using sampled hyperparameters. After training for each task the model 0, is evaluated using the validation dataset. This process is repeated across various hyperparameter settings, and the best hyperparameters H. are selected based on a performance metric. Finally, a new model is trained using the CL algorithm with the best hyperparameters H"' in the same CL scenario, and report the evaluation result on the test dataset. Note that in most studies, results are reported using only Daa, without separate Lest set (i.e., Dro = Deni) (Zhou al.
2023a Sun el al. 2023
given scenario and dataset, which in turn leads to an overestimation of their CL capacity. In other words, this protocol only assesses performance in seen scenario but fails to evaluate generalizability to new, unseen ones an essential aspect for real-world applications. While several alternative evaluation protocols and hyperparameter tuning methods have been proposed, they also have limitations: 1) they require to tune additional hyperparameters for their methods (Delange et al. 2021 Liu et al. 2023). or 2) they are only applied to a few old algorithms, and have not gained widespread acceptance (Chaudhry et al., 2018b Chen et al. 2023 Bornschein et al. )023). As a result, the issues with the conventional evaluation protocol have been largely ignored, and remains the dominant evaluation protocol for evaluating CL algorithms until now.
In this paper, we aim to reveal the limitation of the conventional evaluation protocol by revisiting the fundamen- tal principles of evaluation in machine learning. From this perspective, argue that the evaluation of CL algo- rithms should prioritize assessing the generalizability of each algorithm's CL capacity across unseen scenarios.
To address this issue, we propose a revised evaluation pro- the Generalizable Two-phase Evaluation Protocol (GTEP) which consists of two distinct phases: hyperpa- rameter tuning phase and an evaluation phase. The core motivation behind GTEP is to assess the generalizability of CL algorithms by first identifying hyperparameters in seen scenario that closely resembles an unseen scenario,
HT Phase
90
poO0
Eval Phase
85
AvgAcc
80
and then evaluating the algorithm's performance in the actual unseon scenario. Both phases share the same CL
75
scenario configuration (e.g., number of tasks and classes per task), but use disjoint datasets to enable a more re- alistic assessment. During the hyperparameter tuning phase, models are trained under various hyperparameter configurations, and the best-performing set is selected. These best hyperparameters are then applied in the evaluation phase, where performance is measured on separate dataset, serving as a reliable indicator of the algorithm's capacily to generalize in unseen scenarios As an initial application of GTEP, we focus on the most actively studied setting in CL class- innss-inrremenall learning (class-IL) evaluating algorithms both with and without pretrained models (Wang al., 2023). Basod on approximately 8,000 experiments, we report. the following key findings:
70
188
WA
TOSTER NEMO 12P DanlPrompt CASE (2022) (2023) (2024)
(2020)
(2022) (2023)
Figure 2: Results both phases.
First, as illustrated in Figure 2 most Srate-of-tho-at class-IL algorithms achieve strong performance during the hyperparameter tuning phase, which mirrors the conventional evaluation
evaluuuaiiii protocol. However,
2

---

Published in Transactions on Machine Learning Research (04/2025)
some algorithms exhibit limited generalizability in their CL capacity when evaluated on unseen scenarios (f.e. the evaluation phase). This issue is particularly pronounced in recent algorit
Second, further analysis revcals that many of these algorithms suffer from long training times, substantial parameter requirements, high performance variance across different CL scenarios, indicating that they are less efficient than previously assumed.
Our extensive experimental results using the proposed evaluation protocol expose critical flaws in the conventional evaluation paradigm, which systematically inflates the perceived effectiveness of CL algorithms. These findings call for a fundamental shift in how CL methods are assessed. We urge the community to adopt more rigorous and generalizable evaluation standards across domains to enable truly robust and scalable CL. In this context, we propose our GTEP protocol as minimal yet essential step toward achieving this goal.
2
Related Work
Continual Icarning
Continual lcarning (CL) research has boen conducted in various domains (Wang et al.
2023
Parisi et al. 2019 Delange et al.
2021
Masana al.
2020).
In the beginning, the CL research focus on exploring diverse approaches (Li & Hoiem
task- incremental learning (Parisi et al.
2019
Delange et al.
2021
2017
Yoon et al. 2017). As the field progressed, attention shifted to the more challenging scenario, class-incremental learning (class-IL) (Masana et al. 2020
Aljundi et al. 2018 Chaudhry et al. 2018a Cha et al.
2021a
This shift. leads to the investigation of exemplar-based methods, involving the effective utilization of exemp plar
memory storing a subset of the dataset from previous tasks (Rebuffi et al., 2017 Zhao et al.) 2020 Cha
ct al. 2023a). Since then, using the excmplar memory has bocome standard, with several mothods building on Chis foundation. Reguularization-assed methods, which mitigate catastrophic forgetting by introducing a
novel regularization (Wu et al. 2019 Douillard et al. 2020), and model expansion-base
methods, which
dynamically expand model capacity to balance the trade-off between stability and plasticity, have become the
most powerful approach, achieving state-of-the-art performance (Wang et al. 2022b Yan et al. 2021 Zhou
et al. 2022
Wang et al. 2022a)
Class-IL using pretrained models has recently gained considerable attention for achieving strong performance without relying on the exemplar memory (Zhou et al., 2024a). Prompt- methods enable class-IL through prompt learning while keeping the pretrained model frozen. These approaches have evolved over time, incorporating techniques such as using prompt pool (Wang al. 2022d prompt combination (Wang et al., 2022cp, decomposed prompt. (Smith et al. 2023), and prompt generation (Jung et al., 2023). Additionally, representatiion-assed methods derive class prototypes from the pretrained model and use them for classifi- cation (Zhou et al.) 2023b). To enhance the scparability of these prototypes, several recent mothods have locused on reducing class-wise correlation (McDonnell et al. 2024 Zhou el al. 2024b).
Evaluation and hyperparameter tuning of CL Several papers have proposed new evaluation metrics and protocols for the proper assessment of CL algorithms in classification. Traditionally, accuracy-based metries (e.g., final and average accuracy) have been used as the primary metrics of evaluating performance of CL. algorithms (Parisi et al.) 2019 Masana et al. 2020 Chaudhry et al., 2018a).. However, recent studies have highlighted limitations of these metrics, particularly regarding computational costs (Prabhu et al.) 2023) and learned representations (Cha et al.] 2023b). Delange et al. (2021) introduced a hyperparameter tuning method for task-incremental learning, which involves first conducting a maximum plasticity search and then selecting the best hyperparameterss using stability decay. Similarly, Liu et al. (2023) proposed a hyperparameter selection method for class-IL based on a bandit algorithm. However, both approaches entail additional training costs and the need to tune extra hyperparameters.
Several prior studies have proposed evaluation protocols similar in spirit to ours (Chaudhry et al. 2018b;
Chen et al.) 2023 Bornschein el al. 2023 Michel el al. 2023: Wang el al. 2024), incorporating a separate hyperparameter tuning phase. However, while such approaches have advanced evaluation practices in specific contexts, these protocols have predominantly been appliod limited set of carlier algorithms within narrowly defined domains. In particular, they fail to comprchensively address the issues of the hyperparameter tuning in the face of dynamic, real-world conditions. Despite these efforts-and ongoing discussions about the need
3

---

Published in Transactions on Machine Learning Research (04/2025)
CL Scenario
Hyperparameter Tuning Phase
Result
Task
Task
Task
H
(D 1: D Dval
Train 1
Dg
Di
Dy
H[
64
Add (H,, P+)
H
78
pat
Poat
Dott
CL Alg.(H()
Eval.
Hg
84
Generate CL scenario
Dataset
Reinit 0
(D#T)
Sample H
Hf
54
CL Scenario
Evaluation Phase
Task
Task
Task T
(D!r" D
Train 11r
D Pat
Dir
D
Report Evaluation
Diat
D
CL Alg.(F())
Eval.
Result
Dataset FDF)
Generate CL scenario
H( H[3
Figure 3: Illustration of the proposed evaluation protocol. Both phases share the same CL scenario configuration (e.g., the number of tasks and number of classes in each task) but they are generated from distinct datasets (D#T and D Best hyperparameters Then, the evaluation phase access a CT. algorithm by training model using them. Note that evaluating an algorithm solely based on the results from the hyperparameter tuning phase is identical to the conventional evaluation protocol without using D
selected in the hyperparameter tuning phase.
for more rigorous CT, evaluation Mundt et al. 2022 the conventional evaluation protocol continues to dominate the assessment of state-of-the-art CL algorithms across wide range of domains.
We believe the contimued use of the conventional evaluation protocol stems from a lack awareness in the research community regarding inherent flaws. In this context, our paper makes two distinct contributions: 1) we introduce revised evaluation protocol specifically designed to assess the generalizability of each algorithm's is CL capacity more accurately, and 2) unlike previous studies, paper comprehensively exposes the shortcomings of the conventional evaluation protocol, supported by extensive experimental validation.
3 Protocol for Evaluating the Generalizability of Continual Learning Algorithms
3.1 Motivation: Limitations of hyperparameter tuning in conventional CL evaluation
As shown in Figure the primary flaw of the conventional evaluation protocol is that it optimizes an algorithm's is typerparameters in a given CL scenario and then evaluates the algorithm using those hyperparameters in the same scenario. Surprisingly, many studies have reported their results by directly tuning hyperparameters on validation data without considering a separate
set (i.e., set D.HT DHT)) ), as seen in studics such as Wu et al. (2019) Douillard et al. (2020); Zhao al. (2020); Yan et al. (2021): Wang al. (2022b): ; Zhou et al. (2022); Wang et al. (2022a Zhou al. (2023b 2024b), and others. Note that this approach is only feasible in experimental scenarios where all task data is always available. Consequently, this hyperparameter tuning method fails to capture the real challenges of CL and is not applicable to rcal-world situations. While many studies partially address this limitation by reporting robust. performance across various experiments with some fixed or minimally adjusted hyperparameters (Wang et al. 2022a Zhou et al. 2024b), these evaluations are still based on given scenarios (i.e,, seen scenarios), making challenging assess whether the algorithms would perform equally well in unseen scenarios. Nevertheless, this conventional protocol remains the predominant. evaluation protocol for assessing algorithms across most CL. domains.
4

---

Published in Transactions on Machine Learning Research (04/2025)
Algorithm 1: The Generalizable Two-phase Evaluation Protocol
Input: A CL algorithm A, a model 0, the dataset for the hyperparameter tuning phase D#T the dataset for the evaluation phase D the number of random samplings R, the number of trials S, and the number of hyperparameters K.
Output : Final evaluation result (PE) for a CL algorithm A in the evaluation phase
1. {(H,, P + #yperparaanteer/unnnggg A, D&T RR, K)
2. H" < SellerBestityprraranneteerr ))
3. PE
Evaluation (0, A, DE, ,,,,,, ,H,SS)
3.2 Generalizable Two-phase Evaluation Protocol (GTEP) for CL evaluation
Given the previously discussed issues with the conventional evaluation protocol, the key question becomes: What hyperparameter tuning and evaluation protocol should be used to properly assess CL algorithms? Note that effective evaluation in machine learning should prioritize realistic methods tailored to each learning scenario, rather than rigidly adhering to assumptions (e.g., i...d) for theoretical convenience. In this regard, argue that evaluating the generalizability of each algorithm's 's CL capacity is essential. For example, consider real-world CL scenario where an algorithm is applied to a CL scenario consisting of sequence of tasks. Since the entire task data would not be fully accessible at once, the conventional hyperparamcter luning method cannot be applied. In such cases, a reasonable approach is to construct a simulated CL scenario, reflecting the expected actual CL scenario, using a benchmark or available dataset. This involves identifying the best hyperparameters scenario. In other words, one of the basic evaluation protocols consistent with the fundamental principles of evaluation in machine learning is to tune hyperparameters in seen scenarios (e-g., simulated scenarios) and test them in unseen scenarios (e.g., actual scenarios).
the simulated scenario and then applying them to the actual CL
Building on the above concept, we propose a revised evaluation protocol consisting of two phases, the Gen- eralizable Two-phase Evaluation Protocol (GTEP); hyperparameter tuning and evaluation. Figure 3 and Algorithm ! outlines the overall process. The key idea is that CL scenarios for the hyperparameter tuning and evaluation phases are generated from different datasets (d.e., D # DF) but share the same scenario configu- ration (e.g., the number of tasks and classes per task), based on expectations on the actual scenario. In the hyperparameter luning phase, the goal is to identify the best hyperparameters for the CL algorithm. In the evaluation phase, these hyperparameters are ap- plied to assess the algorithm's CL capacity in unseen scenarios, providing a more realistic measure of its generalizability.
Algorithm Pscudo algorithm of the hyperpa- rameter huning phase
Input: : A CL algorithm A, a model 0, the dataset for the hyperparameter tuning phase D#T, the number of random samplings R, the number of trials S, the number of hyperparameters K, and the function that generates CL scenario F. Output {(H,, P 1. Result < 0
2. for + to R. do
3.
for k+ + to K do
4.
h4 + RRadomSSamplohhee)
5. Hr + (hi, (hk)
The pseudo algorithm of the hyperparameter tuning phase is outlined in Algorithm 2 First,, we randomly sample hyperparameters hg from a predefined set Set and build list of selected hyperparameters Hy. Next, generate predefined CL scenario using the func- tion F with shuffled class orderings. Afterward, the model 0 is trained using the selected hyperparameters Hr, the CL algorithm A, and the training dataset DHT Performance (PHT) then measured on the validation dataset DHT, This phase returns multi- ple sets of hyperparameters and their corresponding
6.
for s s 1 to s do
7.
Initialize 0
8.
DHT DHT +- F(Shuffle( (FT))
9.
p!l TrainCL(A, D Dua 8, H.))
10.
PHT + 1/4 EES1P! P!T
11. Add (Hr, PHHT)) to Result
5

---

Published in Transactions on Machine Learning Research (04/2025)
performance. Next, using the Solectiesdiyyepparaaccae function in Algorithm I select the best hyperparameterrs, denoted as H² Note that the hyperparameter tuning phase is identical to the conventional evaluation protocol. However, we only use the results from this phase to select the best hyperparameters.
In the evaluation phase (shown in Algorithm 3), we train a model 0 using the CL algorithm with the best hyperparameters H". The trained model is then tested on the validation dataset Deat- The final performance metric the averaged performance (PE) of the trained model across multiple class orderings, which serves the evaluation criterion for the CL algorithm.
Algorithm 3: Pseudo algorithm of the evaluation phase
Input A CL Algorithm A, model 8, the dataset for the Eval phase DE the best hyperparameter value H", the number of trials S, the number of hyperparameters K, and the function that generates CL scenario F.
Output : Final ovaluation result (PR) for A for 8 <-1 S do
2. Initialize 0
DE D val + F(Shuffec(D")) (FF)
4. P - TrainCL(A, D D vats 0, H*)
5. pE + 1 1322 1 PE
To find the best hyperparameters for each algorithm, optimize both algorithm-sppeiifc hyperparameters (e.g., regularization strength) and general hyperparamcters (e.g., learning rate and batch size). During the hyperparameter tuning phase, we train the model with R sets of randomly selected hyperparameters and account for S task orderings per set. In the evaluation phase, we assess the performance across S task orderings as well. In this paper, we set = 30 and S = for all experiments, We also take into account various similarity scenarios between the hyperparamctor tuning dataset (D#T) and the evaluation datasot somewhat. (DE). High similarity indicates that the characteristies of the dataset used in the actual scenario predictable, allowing
generate scenario for the hyperparameter tuning phase using a similar dataset. Conversely, low similarity suggests umpredictability,, indicating that the datasets used to generate scenarios in both phases differ significantly. Unlike previous studies (Standley et al. 2020 Fifty et al. 2021 Zamir et al. 2018 we consider similarity in the simplest form at the dataset. level. Specifically, high similarity refers to splitting the same dataset (c.g., CIFAR-100) into two disjoint subsets, which are then used in each phase. On the other hand, low similarity involves using completely different datasets in each phase. Evaluating each algorithm under both similarity cases offers a comprehensive understanding of the generalizability of its CL capacity. Furthermore, these efforts toward more rigorous evaluation underscore key methodological differences from previously proposed protocols (Chaudhry et al. 2018b Chen et al. 2023 Bornschein et al. 2023: Mundt et al.) 2022), further motivating the need for a revised evaluation framework. Note that the high-level concept underlying our proposed protocol is broadly applicable across diverse CL domains. By adapting the CL scenario generation process-denode
as F in Algorithms 2 and g to refloct domain-spocific characteristics (e.g., class imbalance within lasks, blurred task boundaries, or alternative CL domains), the protocol can be effectively applied wide range of settings that involve CL scenarios.
4 Experimental Results
In this section, we present extensive experimental results using proposed protocol within the most actively studied domain of continual lcarning (CL)-clas-increeennd lcarning (class-IL) (Wang et al. 2023) considering both settings without and with pretrained models (Masana el al. 2020 Zhou et al. 2024a 2023a). We begin by evaluating whether the best hyperparameters identified during the hyperparameter tuning phase outperform the original ones (i.e., the previously reported best hyperparameters for cach algorithm). We then assess the continual learning capacity of each algorithm across different levels of scenario similarity and compare their cost efficiency.
6

---

Published in Transactions on Machine Learning Research (04/2025)
4.1 glass-incremental learning without pretrained models
Experimental settings We conduct the hyperparameter tuning and evaluation phases using benchmark datasets, shown in Table From ImageNet-11 (Dnng et al., 2009), we derive two subsets, ImageNet- 100-1 and ImageNet-100-22 100-2, each containing 100 randomly selected non-overlapping classes. To account for varying dataset similarities, we further divide CIFAR-100 (Krizhevsky et al.) 2009) and PetageNet-100-3 into disjoint classes, generating CIFAR-50-1, CIFAR-50-2, ImageNet-50-1, and ImageNet-50-2. We focus on two primary class-incremental learning (class-IL.) scenarios (Masana et al., 2020): 10 Tasks, where the model learns an equal number of classes from each task, and 6 Tasks, where the model learns half of the total classes in the first task then evenly distributes the remaining classes evenly across subsoquent tasks. Note that evaluating using both scenarios has been widely considered the proper assessment of each algorithm (Masana et al. 2020 Zhou et al. 2023a) The table presents the configuration of the number classes (C) and tasks (T) for each scenario.
We conduct experiments using ResNet (He et al. 2016).
Table 1: Scenarios and datasets.
We employ two key performance metrics commonly used
Scenario
D
D
for evaluating class- IT. algorithms (Masana et al.] 2020):
10 Tasks
Acc is final classification accuracy for the entire valida- tion dataset after training the final lask, and AvgAcc
(C10xT10)
ImageNett 100-1
ImageNet. 100.2
Tasks
1 + Acce where Ace; denotes accuracy on the valida- tion data up to task t. The hyperparamcters that yield the highest harmonic mean of Acc and AvgAce are selected during the hyperparameter tuning phase, as most studies evaluate algorithms by simultaneously considering both metrics.
[C50xT1 C10xT5)
10 Tasks
ImageNet-50-1, CIFAR-50-1
ImageNet-50-2, CIFAR-50-2
(C5xT10)
Tasks
(C25xT1 + C5x
Baselines We evaluate nine major class- algorithms, including replay- methods (Replay, iCaRL Re buffi et al. 2017), and WA (Zhao al., 2020)) and regularization--aseed methods (BiC (Wu al. 2019 and PODNet (Douillard et al. (020j) and expansion-based methods (DER (Yan et al. 2021). FOSTER (Wang ct al. 2022b), and BEEF (Wang et al. 2022at). Note that we use the partially implemented DER, as neither PyCII nor the official DER code includes the implementation details for masking and pruning. Replay serves as a naive baseline, where a model is fine-tuned using both the exemplar memory and the current task's dataset. Note that these algorithms have demonstrated progressively improved performance in the order of their publication. Among them, FOSTER, BEEF, and MEMO are recognized as the current state-of -ate--o--hhh--r-e theh-att, reporting superior performance that surpasses DER by small margin. We conduct experiments using the implementation code proposed in PyCIL (Zhou et al. 2023a). The size of the exemplar memory is set to 2000 for Ne0 and 1000 for ImageNet-50 and CIFAR-50 variants. More details on setlings, predefined hyperparameter sets and selected hyperparameters are presented in Section A.1 of the Appendix.
Experiments using original and selected hyperparameters To demonstrate whether the hyperparam- eters identified during the hyperparameter luning phase achieve better performance than those previously reported, we conduct experiments with both sets of hyperparameters. Figure [[a] presents results on DHT ImageNet-100-- showing that using the bost hyperparameters (H²) generally outperforms the origi- nal ones across all algorithms except BEEF. Note that the performance differences among DER, FOSTER, and MEMO are within their respective standard deviations. Considering the hyperparameter tuning phase aligns with the conventional evaluation protocol, this graph indicates that each algorithm reflects the performance trends observed in their respective papers, gradually improving over time in accordance with the order of publication. On the other hand, we confirm that BEEF significantly sensitive to hyperparameters, as it occasionally results in NaN (Not a Number) in training loss for specific sceds, even when using the original hyperparameters.
In the evaluation phase, we apply the best hyperparameters to assess the CL capacity in unseen scenarios generated by DE. Note that, due to differences in the datasets between these phases, the final performance may vary across phases, even when using identical hyperparameters for each algorithm. Figure 4(b) presents experimental results. The graph shows that the CL capacity of the state-of the---rtttt-ttttee algorithms (i.e., FOSTER, BEEF, and MEMO) is significantly inferior to that of older algorithms, such as WA, BiC and PODNet. Additionally, BEEF again produces NaN values for certain seeds. In contrast, DER demonstrates superior generalizability of its CL capacity, consistently maintaining strong performance in both phases.
7

---

Published in Transactions on Machine Learning Research (04/2025)
Repley
75
Original
CeRL
(201
H
75
BIC
70
(2019)
WK
AvgAcc
(2020)
(2020)
DER
65
(2021)
BEEF
50
60
(2023)
MEMD
Replay
[CaRL (2017)
DIC
WA
FODNet DER TOSTER
BEEr MEMO (2020) (2021) (2022) (2023) (2023)
DHT (ImageNet-100-1)
DF (ImageNet-100-2)
(2019)
(2020)
(a) Results on D ImageNet-100-1 Figure 4: Experimental results (AvgAce) on the 10 Tasks scenario using ImageNeet100-11 for D and ImageNet- 100-2 for DE (high similarity). The term Original and H" refer to the use of reported hyperpa- rameters and hyperparameters selected from protocol, respectively, BEEF constantly returns NaN in training loss at specific seeds so could not report its performance.
(b) Comparing results on D and DE
Replay
BiC(2019)
PODNet(2020) DER(2021)
FOSTER(2022) BEEF(2023)
MEMO(2023)
(CaRL(2017)
WA(2020)
60
50
40
30
High similarity tasks)
High similarity (6 tasks) DM CIFAR50-1 D = CIFAR50-2
similarity tasks) DMT CIFAR50-1 D imageNet50-2
similarity (6 tasks)
D CIFAR50-1
D CIFAR50-1 D ImageNet50-2
CIFAR50-2
70
60
High similarity (10 tasks) D ImageNet50-1 D ImageNet50-2
High similarity (6 tasks) n ImageNet50-1
Low similarity (10 tasks) D = ImageNet50-1
Low similarity (6 tasks) Df! ImageNet50-1
D = ImageNet50-2
CIFAR50-2
Df CIFAR50-2
Figure 5: Bar graphs depict the experimental results from the evaluation phase. The Y-axis represents final classification accuracy (Acc). The parenthesos next to each algorithm indicate the publication year. The bar graphs in the first row show the experimental results using the best hyerparameters selected in the hyperparameter tuning phase with "T = CIFAR-50-11 while the graphs in the second row show the results using DT ImageNet-50-11. In cases of using ImageNet-50-11 or ImageNet-50-2. we encountered challenges in obtaining results for BEEF due NaN issues.
Experiments across diverse similarity cases To broadly assess the generalizability of each algorithm's CL capacity, conduct experiments across various similarity cases. The bar graphs in the first row of Figure display results for both high and low similarity cascs, using the best hyperparameters selected during the by peprrarraeeeer luning phase using #T CIFAR-50-1. In most cases, iCaRL performs worse than Replay, and BiC also struggles some cases (e.g., tasks in both high and low similarity settings). Additionally, both WA and PODNet outperform other regularization- basod methods, with PODNet particularly excelling in the 6 Tasks. Lastly, the current state-of-the-art Srrrhods-FOSTERR BEEF, and MEMO -ehMi-t lower performance compared to DER, with BEEF again showing significant sensitivity, especially ImageNet-50-2.
8

---

Published in Transactions on Machine Learning Research (04/2025)
The second row of Figure presents results using the best hyperparameters selected based on D ImageNet-50-1. The trends are consistent with previous experiments: DER maintains superior performance in most cases, although FOSTER surpasses DER in the low similarity case (6 tasks). Additionally, BEEF suffers from NaN issucs in training loss for certain seeds. Additionally, from the results of the high- and low-similarity cases, we observe that BiC and MEMO are particularly sensitive to hyperparaneters.
1e6
90
BEEF
DER
BIC
MEMO
Training time (hour)
BEEF
Replay
ACC+
70
KanLi2017)
FOSTER
BIC(2019)
60
PODNet2020)
WA(2020)
2
DER(20Z1)
FOSTER
POONet
50
(OSTERIZOZ2)
RFFF12023
Replay
PODNet
1
Replay ICaRL
WA
DER
40
MEMO120231
45
50
55
60
65
45
50
55
Acc
(a) Performance graph
(b) Number of parameters
(c) Total training time
Figure 6: Experimental analysis the evaluation phase. All experimental results are obtained by first
identifying the best hyperparameters using CIFAR-50-1 (10 Tasks) in the hyperparameter tuning phase, then evaluating each algorithm using CIFAR-50-2 (10 Tasks) in the evaluation phase. (b) and (c) show results after training up to the final task.
Additional analysis Figure 6(a) shows the evaluation results for each task t in the evaluation phase, with shaded areas representing the standard deviation across 5 trials. From these graphs, evident that DER consistently outperforms current state-of-the-art algorithms (i.e., FOSTER, BEEF and MEMO). Considering the standard deviation, the performances of FOSTER and MEMO nearly indistinguishable. Among the remaining algorithms, WA demonstrates relatively better performance while BEEF performs similarly to the order algorithms.
Recent studies have increasingly focused on evaluating CL algorithms based on their training costs, particularly in terms of GPU usage and energy consumption (Prabhu et al. 2023 Chavan et al. 2023). However, these evaluations were often conducted by either limiting the number of training itcrations or comparing costs under a fixed number of training epochs. Building on this, we extend the analysis by examining the final model size and total training time for each algorithm, using their best hyperparameters to ensure fair and comprehensive comparison of efficiency. Figures 6(b) and [(c) present scatter plots showing achieved accuracy, total parameter counts, and training times. DER. performs the best and requires relatively less training time. Nevertheless, it exhibits considerable inefficiency in the total number of parameters, which increases linearly with the number of tasks, raising concerns about its actual cost-efficiency as a CL. algorithm. On the other hand, BiC, BEEF, and MEMO fail to demonstrate superior performance while requiring similar or longer training times compared to DER, highlighting their serious inefficiency.
4.2 Class-incremental learning with pretrained models
Experimental details We conduct both the hyperparameter tuning and evaluation phases using widely used datasets in class-nnermmentl learning (class-
Table 2: Scenarios and datasets.
II.) with pretrained models, including CUB-200 (Wah et al.. 2011), ImageNet-R. (Hendrycks et al. 20211a,. and ImageNet- A (Hendryeks et al. 2021b), all of which contain 200 classes.
Scenario
D
DE
20 Tasks
CUB-200, ImageNet-R
ImageNet-R, ImtgeNet-A
(C10xT20)
To explore diverse similarity cases,
divide these datasets
10 Tasks
into two disjoint subsets, as outlined in Table 2 Following Sun et al. (202)). we consider Iwo major class-IL scenarios: 20 Tasks and 10 Tasks, where the model learns an equal number of classes each task. Note that the Tasks scenario has been commonly regarded as the standard for better evaluating algorithm performance due to the need to handle more tasks.
(C20xT110
20 Tasks
CUB-100-1,
CUB-100-2,
(C5xT20)
ImageNet-R-1
ImageNet-R-2.
10 Tasks
ImageNet-A-2
(C10xT10)
For all experiments, employ the VIT B16

---

Published in Transactions on Machine Learning Research (04/2025)
model, which is pretrained ImageNet (Dosovitskiy et al. 2020 The best hyperparameters selected based on the same metrics: the harmonic mean of Acc and AvgAcc.
Baselines We select six major algorithmms: prompt-based methods (L2P (Wang et al. 2022d Dual-
Prompt (Wang et al. 2022c) and CODA-Prompt (Smith et al. 2023 ) and representation based methods (Adam-Adapter (Zhou al. 2023b), Ranpac (McDonnell et al. 2024) and EASE (Zhou al., (024b)): )
Within each category, CODA-Prompt and EASE represent current state-of-the-ar algorithms. Although DAP (Jung et al. 2023) reports better performance within the prompt-based method category, exclude it due to fairness issues in comparison, mentioned in Zhou et al 2024a All experiments are conducted using code implemented in PILOT (Sun et al. 2023). Details on experimental settings, predefined hyperparameter A.2 of the Appendix.
sets and the best hyperparamelerss are proposed in Section
95
Original
1(
90
DualiPrompt
80
(2023)
AvgAcc
CODA Prompt (2023)
85
70
Adem
(2023)
Ranpac
80
(2023)
60
(2024)
75
L2P (2022)
DualPoopptODD--rropt
Adam (2023)
Ranpac (2023)
EASE (2024)
DE (CUB-200) (ImageNet-R) (ImageNet-A)
(2023) (2023)
(b) Comparing results on D and DE Figure 7: Experimental results (AvgAce) for 10 Tasks scenario using CUB-200 200 for D ImageNet-R, and ImageNet-A for D (low similarity). The term Original and H refer to the use of original hyperparameters and the hyperparameters selected from our protocol, respectively.
(a) Results on D CUB-200
Experiments using original and selected hyperparameters To verify best hyperparameters selected in the hyperparameter tuning phase, we conduct experiments on D CUB-200 using both the original and selected hyperparameters of each algorithm. Figure ((a) demonstrates that using the selected hyperparameters leads to better performance across all algorithms. Additionally, the performance of each algorithm gradually improves in accordance with their publication order, reported in the respective papers. However, Ranpac and EASE achieve similar performance, with differences falling within their standard devialions.
Following our evaluation protocol, we apply the best hyperparameters for each algorithm in the evaluation phase. We conduct experiments for two evaluation phases using ImageNet-R and ImageNet-A as D and Figure 7(b) shows experimental results. From these results, we confirm the following observations: First, among the prompt-based algorithms (solid lines), DualPrompt exhibits degraded performance compared to L2P in both evaluation phases. Additionally, CODA-Prompt demonstrates superior performance in all cases, although it shows nearly identical performance to L2P in the ImageNet-R. In the case of the representation-basee algorithms (dashed lines), Ranpac consistently demonstrates superior performance across all datasets; however, we observe some instability in specific scenarios, as will be shown in the following analysis. Furthermore, EASE, recognized as the current state-of-the-are shows significantly poorer performance both evaluation phases.
Experiments across diverse similarity cases Figure presents the experimental results evaluated in the evaluation phase. Similar to trends reported in Zhou et al. (2024a), representation- based methods generally outperform nompt-based methods. However, significant differences are observed under the proposed evaluation protocol: First, the prompt-based methods have reported substantial performance improvements over previous al algorithms (e.g., 7-10% increases on the CUB200 dataset. for each algorithm (Zhou el al., 2024a)). ). However, the proposed evaluation protocol reveals either no significant performance difference between them (e.g., low similarity (20 tasks) using ImageNet-R-2 where an older algorithm outperforms a newer one (e.g., high similarity (20 tasks) using CUB100-2 in the first of the graph). Second, the current state-of-the-art eepresentation-based method, EASE,, often
the first row of the graph) or cases
10

---

Published in Transactions on Machine Learning Research (04/2025)
L2P(2022) DualPrompt(2023)
CODA-Prompt(2023) Adam(2023)
Ranpac(2023) EASE(2024)
High similarit tasks) High similarity tasks) similanty (20 tosks)
similanty tasks)
Low aimiaarty tasks]
sim arity tasks
Dd CUB100-1
D CUB100-1
n CUB100-1
D CUB106-1
pef CUB100-1
pel CU8100-1
Of CUB100-2
Of CUB100-2
ImageNat-R-22
ImageNet-R-2
ImageNet-A-2
OF ImageNct-A-2
80
70
60
30
40
30
High similarity tasks) similarity tasks) similarity (20 tasks)
Low similanty tasks)
Low similarty tasks]
similarity tasks) D"r ImageMet- 8-1 of ImageNet
D imageNet-K- 8-1
OF ImageNet Ded ImageNiet--11 D CUB100-1
17 I-ageNet-R-1
t ImageNeet--11
of ImageNet D ImageNat-R.2
DF CUR100-2
ImageNet A2
Figure Bar graphs depict the experimental results from the evaluation phase. The Y-axis represents
final accuracy (Acc). In the legend, the parentheses next each algorithm indicate the publication year. The bar graphs in the first row show the experimental results using the best hyerparameters selected in the hyperparameter tuning phase with D CUB100-1, while the graphs in the second row display the results using #TT ImageNet-R-1.
le6
2.5
Prompt
QuaPrampo120231
Ranpaci2023) EASE12024]
Напрac
-
CODA Prom pt(223))
2.0
parameters
80
1.5
DealPrompt
EASE
15
L2P
DealPrompt
EASE
ACC
50
1.0
10
Prompt
0.5
L2P
Ranpac
Adam
Tha
5
Adam
0.0
0
25
30
35
40
45
25
30
40
45
ACc
Acc
(a) Performance graph
(b) # trainable params
(c) Training time
Figure 9: Experimental analyses in the evaluation phase. All experimental results obtained by first identifying the best hyperparameters using ImageNet-R--1 (20 Tasks) in the hyperparameter tuning phase, then evaluating each algorithm using ImageNet-A-2 (20 Tasks) in the evaluation phase. (b) and (c) show the results after training up to the final task.
underperforms compared to Ranpac, especially in low similarity cases (e.g., low similarity (10 tasks) using ImageNet-R-2 in the first row of the graph). Lastly, while Ranpac achieves the best performance in most cases, it exhibits significantly degraded performance in several low similarity cases (e.g., low similarity (20 tasks) using TmageNet-A-2 in the first row of the graph). This degradation attributed to considerable performance instability in certain tasks. Furthermore, by comparing the results of the high- and low-similarity cases, we observe that Adam and Ranpac highly sensitive hyperparameters. Specifically, while Ranpac generally achieves strong performance in the high-similarity case, its performance is sometimes reversed in the low-similarity case.
Additional analysis As we already confirmed in the previous experiments, Figure ((a) illustrates that Ranpac suffers from significant instability in certain tasks, resulting in a substantial increase in standard deviation (shaded arca). Furthermore, we observe that the stato-of-tho-art algorithms, EASE and CODA- Prompt in their respective categories, do not consistently outperform baseline algorithms like Adam and DualPrompt in many cases, highlighting lack of generalizability in their CL capacity.
11

---

Published in Transactions on Machine Learning Research (04/2025)
Figures 9(b] and [[c] display the number of trainable parameters and training times with the best hyperpa- rameters. For prompt-based algorithms, training times are comparable; however, CODA-Prompt requires more parameters while delivering lower performance compared to DualPrompt. Among representation-baese methods, the oldest algorithm (i.e., Adam) achieves the best performance with minimal costs in terms of trainable parameters and training time.
In Section B of the Appendix, we present additional experimental results, including training graphs and numerical data related to the results discussed in the manuscript.
5 Concluding Remarks
Limitations of the conventional evaluation protocol The conventional evaluation protocol, which remains the standard for assessing continual learning (CL) algorithms, exhibits critical shortcomings. Most notably, it assumes access to repeated training within the same scenario for hyperparameter tuning an unre- alistic assumption in real-world applications. This approach not only fails reflect practical CL scttings but also systematically overestimates the actual CL capacity of algorithms. According to fundamental principles of machine learning evaluation, the generalizability of a model particularly to unseen scenarios should be the primary metric of interest. In this context, we introduced the Generalizable Two-phase Evaluation Protocol (GTEP), which separates hyperparameter tuning and evaluation across different scenarios, thereby enabling a more faithful assessment of generalizability across varying degrees of similarity.
Summary of experimental findings Our experiments yield three key insights. First, many algorithums show strong performance in seen scenarios but struggle generalize unseen ones, indicating that their success under the conventional protocol often stems from overfitting. Second, several algorithms are highly certain task orders or unstable performance across
sensitive to hyperparameters, leading to failures lasks. Third, even algorithms that generalize well under our protocol (GTEP) often require substantial training time or parameter usage, undermining the cost-efficiency goal of CL. Although our experiments focus on classiincrementll learning, these limitations likely extend to other CL. domains that rely on the same conventional evaluation practices.
Toward more realistic evaluation: key takeaways We argue that the proposed GTEP offers a principled and realistic framework for evaluating CL algorithms. To advance CL rescarch meaningfully, recommend that future evaluations across all CL domains incorporate the following minimum criteria:
Generalization Check using GTEP: Does the algorithm outperform baselines in the evaluation phase when using hyperparameters selected from a separate hyperparameter tuning phaso?
Efficiency and Stability Check: Is the algorithm more cost-efficient than the baselines (e.g., in terms of training/inference /inference time, parameter count, GPU usage)? Does it consistently maintain stable performance across different tasks and scenarios (e.g., varying task orders)?
6 Limitations and Future Work
Our study is not without limitations. First, implementing GTEP requires repeated training trials, which can be computationally demanding. In our experiments, we conducted 30 random trials per algorithm with 5 seeds per trial. Encouragingly, we observed that most algorithms converged to their optimal hyperparameters within trials, with negligible gains beyond that point. Nevertheless, the development of more sample-efficient tuning strategies such as targeted search based on hyperparameter importance remains an important direction for future research. Second, experiments assume predictable CL scenarios, where the number of tasks and class distributions are known in advance, reflecting structured and practical deployments. However, evaluating the robustness of algorithms in unpredictable or adaptive scenarios (c.g., unknown task boundaries or shifting data distributions) is a critical next step, particularly for adaptive methods that dynanicllly adjust hyperparameters. Lastly, our experiments focus on offline class-incremie incremental learning. Extending GTEP to online CL scttings and broader CL domains-such as class incremental semantic segmentation, continual self-supervised learning continual reinforcement learning, and continual learning for LLMs will be essential for validating the scalability and practical utility of CL algorithms in each domain.
12

---

Published in Transactions on Machine Learning Research (04/2025)
7 Acknowledgement
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (ITTP) with grant funded by the Ministry of Science and ICT (MSIT) of the Republic of Korea in connection with the Global AT Frontier Lab International Collaborative Research. This work was also supported by the Samsung Advanced Institute of Technology (under the project Next Generation Deep Learning: From Pattern Recognition to AI) and the National Science Foundation (under NSF Award 1922658) This work was supported in part through the NYU IT High Performance Computing resources, services, and staff expertise. We would especially like to thank Shenglong Wang for his generous support in sctting up the experimental environment, which was instrumental in enabling our extensive experiments.
References
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Timne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 134, 2018.
Jorg Bornschein, Alexandre Galashov, Ross Hemsley, Amal Rannen-Triki, Yutian Chen, Arslan Chaudhry, Xu Owen Hc, Arthur Douillard, Massimo Caccia, Qixuan Feng, et al. Nevis' 22: A stream of 100 tasks sampled from 30 years of computer vision research. Journal of Machine Learning Research, 24(308):1-77, 2023.
Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Class-incremental continual learning into the extended der-verse. IEEE transactions on pattern analysis and machine intelligence, 45(4):5497-5512, 2022.
Sungmin Cha, Hsiang Hsu, Taebaek Hwang, Flavio Calmon, and Taesup Moon. {CPR): Classiferr-rooeecionn regularization for continual learning. In International Conference on Learning Representations, 2021a.
URL https://opearevview http://.......oooo/od//vavaaaa..
Sungmin Cha, YoungJoon Yoo, Taesup Moon, et al. Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems, 34: 10919 10930, 2021b.
Sungmin Cha, Sungjun Cho, Dasol Hwang, Sunwon Hong, Moontac Lec, and Tacsup Moon. Rebalancing batch normalization for exemplar-based class-incremental learning. In Proceedings of the IEEE/CV /CF Conference on Computer Vision and Pottern Recognition, pp. 20127-201 201366, 2023a.
Sungmin Cha, Jihwan Kwak, Dongsub Shim, Hyunwoo Kim, Moontae Lee, Honglak Lee, and Taesup Moon. Towards more objective evaluation of class incremental learning: Representation learning perspective, 2023b.
Sungmin Cha, Kyunghyun Cho, and Taesup Moon. Regularizing with pseudo-negatives for continual self-supervised learning. In Forty-first International Conference Machine Learning, 2024.
Arslan Chaudhry, Puncet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the Europeon Conference on Computer Vision (ECCV), pp. 532 547, 2018a.
Arslan Chaudhry, Marc'Aureli Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong lcarning with a-gem. arXiv preprint 1x1::1812.000.22 2018b.
Vivek Chavan, Paul Koch, Marian Schlüter, and Clemens Briese. Towards realistic evaluation of industrial continual learning scenarios with an emphasis on energy consumption and computational footprint. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11506-1155 2023.
Jiefeng Chen, Timothy Nguyen, Dilan Gorur, and Arslan Chaudhry, Is forgetting less a good inductive bias for forward transfer? Xix preprint or 2rXi:22303.08007, 2023.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Matthias Delange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classificat ion tasks. JEBE Transactions on Pattern Analysis and Machine Intelligence, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pollern recognition, pp. 248 255. leee, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. aixi preprint arXiv:2000. 11999, 2020.
Arthur Douillard, Matthicu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Computer Vision-F.CCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Part XX 16, pp. 86 102. Springer, 2020.
Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Processing Systems, 34:27503 27516, 2021.
Enrico Fini, Victor G Turrisi Da Costa, Xavier Alameda-Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal. Self-supervised models are continual learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9621-9630, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-77777 770-778, 2016.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVP international conference on compuler vision, pp. 8340 1340-8349, 2021a.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 15262 15271, 2021b.
Dahuin Jung, Dongyoon Han, Jihwan Bang, and Hwanjun Song. Generating instance-level prompts for rehearsal-free continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11847-11857, 2023.
Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. Continual pre-training of language models. In The Eleventh International Conference Leorning Representations, 2023. URL https:://openreview net/forunii ftt///fmummmmdmmmmmm P GDIItaI3o
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Daeun Lee, Jaehong Yoon, and Sung Ju Hwang. Becotta: Input-dependent online blending of experts for continual test-time adaptation. Forty-first International Conference on Machine Learning, 2024.
Zhizhong Li and Derck Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2955 2947, 2017.
Yaoyao Liu, Yingying Li, Bernt Schiele, and Qianru Sun. Online hyperparameter optimization for class- incremental learning. arXiv preprint arXiv:2301. 000000020000 2023.
Mare Masana, Xialei Liu, Bartlomicj Twardowski, Mikel Menta, Andrew Bagdanov, and Joost van de Weijer. Class-incremental learning: survey and performance evaluation on image classification. arXiv preprint ar Xiv:20010. 15277, 2020.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Mark D McDonnell, Dong Gong, Amin Parvanch, Ehsan Abbasnejad, and Anton van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. Aduances in Neural Information Processing Systems, 36, 2024.
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The continuum from catastrophic lorgetting to age-limited learning effects. Fronliers in psychology, 4:504, 2013.
dilemma: Investigating the
Nicolas Michel, Maorong Wang, Ling Xiao, and Toshihiko Yamasaki. Rethinking momentum knowledge distillation in online continual learning. arXiv preprint WXXi:2309.08870 2023.
Martin Mundt, Steven Lang, Quentin Delfosse, and Kristian Kersting. CLEVA-Compass: A continual Icarning evaluation assessment compass to promote research transparency and comparability. In International
Conference on Learning Representations, 2022. URL. htttps://openreviet/ http:///www......e..trtr//aaaaaa http::/.....popoorrrraa////////a
German I Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Newral Networks, 113:54 71, 2019.
Hongming Piao, Yichen Wu, Dapeng Wu, and Ying Wei. Federated continual learning via prompt-basee baset dual knowledge transfer. In Forty-first International Conference Machine Learning, 2024.
Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K Dokania, Philip HS Torr, Ser-Nam Lim, Bernard Ghanem, and Adel Bibi. Computationally budgeted continual learning: What does matter? In Proceedings of the IERE/CVF Conference on Computer Vision and Paltern Recognition, pp.
2023.
Sylvestre Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pollern Recognition, pp. 2001-2010, 2017.
Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, and Jonghyun Choi. Learning equi-angular representations for online continual learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 23933 23942, 2024.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascanto-Bonilla Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free contimual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11909 11919, 2023.
Trevor Standley, Amir Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Which tasks should be learned together in multi-task learning? In International conference on machine learning, pp. 11100-1332 PMLR, 2020.
Hai-Long Sun, Da-We Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Pilot: A pre-trained model-based continual learning toolbox. arXiv preprint -Xxx:2309007111 2023.
Catherine Wal, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200- 200-2011 dataset. 2011.
Fu-Yun Wang, Da-Wei Zhou, Lin Liu, Han-Jia. Ye, Yatao Bian, Do-Chuan Zhan, and Poilin Zhao. Beef: Bi- compatible class-incremental learning via energy-based expansion and fusion. The Eleventh International Conference on Learning Representations, 2022a.
Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for class-incremental learning. In European conference on computer vision, pp. 398 414. Springer, 2022b.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint ar 20X.:22302.00487, 2023.
15

---

Published in Transactions on Machine Learning Research (04/2025)
Maorong Wang, Nicolas Michel, Ling Xiao, and Toshihiko Yamasaki. Improving plasticity in online continual learning via collaborative learning. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pottern Recognition, pp. 23460 23469, 2024.
Zifeng Wang, Zizhao Zhang, Sayna Ebraahii,, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference Computer Vision, pp. 631 648. Springer, 2022c.
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognátion, pp. 139-149, 2022d.
Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trane Vu, and Gholamreza Haffari. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.0136, 2024.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceediiggs of the IEEE/CVF Conference on Compuler Vision and Paltern Recognition, pp. 374-382, 2019.
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 3014 3023, 2021.
Jayeon Yoo, Dongkwan Lee, Inseop Chung, Donghyun Kim, and Nojum Kwak. What how and when should object detectors update in continually changing test domains? In Proceediiggs of the IEEE/CVE Conference on Computer Vision and Pottern Recognition, pp. 13354-233633 2024.
Jachong Yoon, Eunho Yang, Jcongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: 708.0100550 01547, 2017.
Bo Yuan and Danpei Zhao. survey on contimual semantic segmentation: Theory, challenge, method and application. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese. Taskonomy: Disentangling task transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3712-3722, 2018.
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairnoss in class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pollern recognition, pp. 13208-12 -33177, 2020.
Da-Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficient class-incremental learning. arXiv preprint 2r 13218, 2022.
Da-Wei Zhou, Fu- Yun Wang, Han-Jia Ye, and De-Chuan Zhan. Pycil: a python toolbox for class-incremental learning. SCIENCE CHINA Information Sciences, 66(9):197101-, 2023a. doi: https:/ http:://doi.or//0.11000 s11432-022-3600-y
Da-Wei Zhou, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need. arXiv preprint W=Xiw:2303.07333 2023b.
Da- Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. Continual learning with pre-trained models: A survey. Xrx preprint orXix:2401.16335 163.66 2024a.
Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based
learning, ar preprint arXiv:2403. 12030, 2024b.
16

---

Published in Transactions on Machine Learning Research (04/2025)
A Additional Details on Experimental Settings
A.1 Class-incremental learning without pretrained model
Experimental details We conduct all experiments using PyCIL (Zhou et al. enviromment: Python 3.8, PyTorch 1.13.1, and CUDA 11.7. We use ResNet-18 and ResNet-32 architectures for our experiments. For class-ncrementall learning without a pretrained model, we employ the SGD optimizer with a momentum of 0.9 across all methods, consistent with their respective implementations. Other hyperparameters, however, are sampled during the hyperparameter luning phase.
2023a in the following
Table 3: Hyperparameters for training the first task.
Hyperparameters
Values
Init epochs
200
Init learning rate
0.1
[60, 120, 170]
Init milestonees
(Only applied when 'StepLR' is selected)
Init learning rate decay
0.1
Init weight decay
0.0005
Predefiend hyperparameters Recent studies have demonstrated that newer algorithms perform better when trained for more epochs the first task and fewer epochs subsequent tasks (Masana et al. 2020 Additionally, it is known that performance on the first task significantly impacts overall performance (Cha et al. 2023b To apply this approach consistently across all algorit we train a model on the first task using the hyperparameters listed in Table 3 Subsequently, we train that model with randomly sampled hyperparameters starting from the second task.
12
8
4-
2
Replay
ICARL
DIC
VA
PODNet
DER
FOSTER DEEF (2022)
MEMO (2023)
(2017)
(2019)
(2020)
(2020)
(2021)
(2023)
Figure 10: # of hyperparameters.
Figure 10 shows the number of hyperparameters for each algorithm. We consider both algorithm-spedde and general hyperparameters in the hyperparameter tuning phase. Table presents the sets of predefined hyperparameters considered for each algorithm. Note that 'Epoch', 'Num milestones', 'LR decay", 'Batch size', 'Weight decay', and 'LR scheduler' are commonly considered hyperparameters for all algorithms. Additionally, both 'Num milestones' and 'Lr decay' are applicable only when 'StepLR' is selected as a scheduler. The others are specific hyperparameters of each algorithm. We consider all the hyperparameters necessary for implementing each algorithm. For instance, even specific algorithm uses the same value for particular hyperparameter across all experiments (e.g., fixing the strength of an additional regularization aimed to find the best hyperparameteer for (e.g., setting the strength as and finding the best value of it in the hyperparameter tuning phase). We determine the range of values for the predefined hyperparameters based on the following criteria. First, for general hyperparameters, we establish the range to include all optimal values reported by each algorithm. For specific hyperparameters related to each algorithm, we not only include the optimal values report in the papers but also considered the full range of values that were explored during their hyperparameter searches.
1), we
When the LR scheduler is set StepLR, the milestones must be determined. To achieve this, we generalize the process of random sampling based the milestones used in existing algorithms. First, randomly
17

---

Published in Transactions on Machine Learning Research (04/2025)
sample num milestones. Based this sampling, the milestones for the StepLR are set according to the following rule: For example, if N milestones is set to 2, the milestones are defined as (2/5c (4/5]. If set to 3, the milestones become (2/7ch epoch (60ch* Similarly, for 4 milestoncs, the values are (epoch"(2/9) spoch*(4/9) epoch (6/9), cpoch"(8/9)]- However, note that the nun milestones is ignored when another LR schduler is selected.
Table 4: The predefined set of hyperparametes for class-IT without a pretrained model.
Algorithm
Hyperparameter Name
Epoch
(30, 70, 120, 160, 200 0.05, 0.1, 0.15, 0.2, 0.3)
LR
Num
[2, 3, 4]
All algorithms
milestones
LR
[0.1, 0.3, 0.5]
decay
Batch
[32, 64, 128, 256, 512]
size
Weigh
[0.0001, 0.0005, 0.001, 0.005]
decay
LR
[SStepL'', 'Cosine']
Scheduler
iCaRL, BIC, WA and FOSTER
[0.5, 1, 1.5, 2, 2.5] [0.5, 1, 1.5, 2, [0.05, 0.1, 0.15, 0.2, 0.3)
(KD)
A
BIC, WA and FOSTER
(KD)
Split
BiC
ratio
iCaRL, PODNet, DER. and MEMO
(0.5, 1, 1.5, 2,
(Aux)
FOSTER
[0.5, 1, 1.5, 2,
(FF)
FOSTER
81
0.93, 0.95, 0.97, 0.99
FOSTER
0.93, 0.95, 0.97, 0.99
Num
PODNet
[10, 20, 30, 50, 100]
proxy
Post FT epochs
[5. 10, 20, 30, 50)
PODNet, FOSTER and BEEF
[30, 70, 120, 160, 200] (FOSTER BEEF)
Post FT
PODNet
[0.001, 0.003, 0.005, 0.007, 0.01]
LR
PODNet
Adaptive factor
[True, False]
Energy
BEEF
[0.001, 0.005, 0.01, 0.02, 0.05)
weight
Logit
BEEF
[1.1, 1.4, 1.7, 2.0, 2.3] [16, 32, 64, 128, 256]
alignment
Exemplar
MEMO
batch size
18

---

Published in Transactions on Machine Learning Research (04/2025)
Original hyperparameters The following shows the original hyperparameters of each algorithm reported in PyCII.
Replay: ep_70 Ir 0.1 Ir decay 0.1 batch 128 w srraa__0..002sschdduee _eteepeeeersssppprrrrreee
BiC: ep 170 Ir 0.1 Ir decay 0.1 batch 128 w decay 0.0002 scheduler steplr
2_2 lambda, kd spslit ratio_0i _01
PODNet: ep ep_ 160 milestone 2 Ir 0.1 Ir decay 0.1 batch 128 "__decay_00 0.0005 scheduler _cosine
lambda_e c 5 lambda f 1.0 nb proxy 10 ft epochs 20 ft Irate 0.005 1.005_adaptive lib___dattve_facttot factor__chh
FOSTER: ep 170 0.1 decay 0.1 batch 128 w decay 0.0005 scweduler__eooeeee cosine T lambda, kd fe beta 0.96 0.97 0.97_comp______
comp_ op 130
MEMO: ep_ 170 milestone_ 3 Ir 0.1 Ir decay 0.1 batch 128 w __ ddccy docaa_oo00000 0..0002sschdulorr lambda aux examplar bs 64
iCaRL: ep 170 101 Ir decay 0.1 batch 128 0.0_00000022_cccuueeee 2_2_lambda aux
WA: ep 170 milestone 3 Ir 0.1 Ir decay 0.1 batch 128 decay 0.0002 scheduler steplr
2.0 lambda kd 0
DER: ep 170 Ir 0.1 decay 0.1 batch 128 w decay 0.0002 scheduler steplr
lambda aux
BEEF: ep_170 milestone_ 4 Ir 0.1 Ir decay 0.1 batch. 128 w Secay_000003 110000__cheuuler_e__ cosine fusion_ ep 60 _nerrgy--__ w 0.01 logits__alll 1.7
Note that setting 'lambda kd 0' for both BIC and WA indicates the use of their adaptive rule.
19

---

Published in Transactions on Machine Learning Research (04/2025)
of each algorithm selected in the hyperparameter tuning phase using ImageNet-100 (10 Tasks). Best hyperparameters (ImageNNet-100, 10 Tasks) The following represents the best hyperparametens
Replay: ep 70 milestone 3 Ir 0.2 Ir decay 0.1 batch 64 w decay 0.0001 scheduler steplr
BiC: ep 120 milestone 3 0.1 Ir decay 0.1 batch 32 w decay 0.0001 scheduler steplr
T 1 lambda, kd 3.0_ ratio 0.1
PODNet: ep_30 3p_30__mllsstt milestone 4 4_ 0.05 Ir 1_055_005__111111111111111111111 le________e docay 0.1 0.1 batch 64 64_w @4_w_docay dccaa_00.001111_______1 0..0001sschddui scshcheeer__sspprrrr steplr lambda c_ lambda f 1.5 nb proxy_oooooo_ 20 ft epochs 5 Irate _a0ee__0000... 1010005___ppiveeeee adaptive_factoor False
FOSTER: ep_30 milestone 4 1000_005__00 Ir delaa_0.111 1_1__011__dlee 64 64_ w T lambda kd 1.5
beta 0.93 0.97 comp ep 160
MEMO: ep 120 milestone 3 Ir 0.15 Ir decay 0.1 batch 512 w decay 0.001 scheduler steplr
ambda aux 0.5 0.5__.eee0000 ____xamplar bs_ 32
iCaRL: ep_eep_222000 200 mii_meeee 3 Ir 0.15 Ir decay 0.1 batch 64 ww decay_ec__pp 1ec_0001_shedulerccoooe 2.5 lambda aux
WA: ep 120 milestone 3 Ir 0.1 Ir decay 0.1 batch 32 decay 0.0001 scheduler steplr
T lambda, kd 3.0 split ratio 0.1
DER: ep_ 200 milestone 3 Ir 0.15 Ir decay 0.1 batch 64 w decay_0.00011 Laa___sshhcue__sssee lambda aux
BEEF: ep 120 milestone Ir 0.2 decay 0.3 batch 128 w decay_0.0000 0..0000000eeeltte scheduler steplr 0.2 fusion op 30 energy 0.02 logits align 2.3
Best hyperparameters (ImageNet-100, 6 Tasks) The following represents the best hyperparametens of each algorithm selected in the hyperparameter tuning phase using ImageNet-100 (6 Tasks).
Replay: ep_70. 70 milestone 3 Ir 0.2 Ir decay 0.1 batch 64 decay _0.00000001111 cecoo_sshhuule_sstieee steplr
BiC: ep 30 milestone Ir 0.05 Ir decay 0.1 batch 64 w decay 0.0001 scheduler scheduler_steph 1 lambda. kd 1.5 split ratio 0.1
PODNet: ep_30 30 milestone 2 Ir 0.15 Ir decay_0.. batch ______ddee__ v__eecy_____ de0ca__00011,. 1:0001_scheuulrr scheduler_sseplr lambda c lambda f 0.5 nb proxy_ 100 ft. epochs 10 ft. Irate 0.000
1rate_0.007.______ _d000_adapptvv__Bae__ factor False
FOSTER: ep_ 70
_p_70 milestone 3 Ir 0.05 Ir decay__a 0.1 batch 2.5 lambda kd 0.5 fe beta 0.95 0.93 comp ep
ba2 512 w_ dee_deacooy00000 000000000011111000 1..0001_scheduler_coom cosine
MEMO: ep_ 120 milestone 3 Ir 0.15 Ir decay 0.1 batch 512 w decay 0.001 1eoodulhheuulerre_ lambda aux 0.5 examplar bs 32
iCaRL: ep 200 milestone 3 Ir 0.15 Ir decay 0.1 batch 64 w decay 0.0001 scheduler cosine T_5 La__llmbdaaaaaa aux__
WA: cp_ 170 170 Ir 0.1 Ir decay 0.1 batch 128 decay d.oeaa_00000____sa________ ochedu2sshhcuulerr steplr
T 2 lamnda.. kd
DER: ep 200 milestone 3 0.15 Ir decay 0.1 batch 64 decay 0.0001 scheduler cosine lambda lambda_aux aux
BEEF: cp_ 30 milestone 4 Ir 0.05 decay 0.1 batch. 128 w 0.000l_schedulerr _stppl 0.01 logits align 1.4
fusion ep 70 energy
Best hyperparameters (CIFAR-50, 10 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using CIFAR-50 (10 Tasks).
20

---

Published in Transactions on Machine Learning Research (04/2025)
Replay: ep_1600 1160_milestone 3 Ir 0.15 Ir decay__oo 0.3 batch 32 (lecay_0.000) n..0001sscheuule__cem cosine
BiC: ep 200 milestone 2 0.1 decay 0.1 batch 32 w decay 0.0001 scheduler cosine
T 10.5__lmbba kd 0.5 split Lttps__rprtt_________________ 0.2
PODNet: ep_70 m 2 0.1 Ir decay 0.1 0.t__atchh_____. batch 32 w_ w__eeaa__ Daacay_0.0000 lambda c lambda
nb proxy 10 ft. epochs 30 ft. Irate 0.007 adaptive_ factor
_actttreeeeaarrrrrooo False
FOSTER: ep 120 milestone 3 Ir 0.1 Ir decay 0.5__attchh_________ batch 32
T lambda. kd 1.5 fe 0.5 beta 0.97 0.93 comp ep 160
MEMO: ep_ 120 milestone Ir 0.05 Ir decay 0.3 batch 32 w decay 0.0005 scheduuerr scheduler_steepp
lambda aux 0.5 examplar bs 16
iCaRT: ep 70 milestone 3 Ir 0.05 Ir decay 0.3 batch 32 w decay 0.001 scheduler_ccostee
T 225___ammbba _ux__
WA: ep 160 milestone 4 Ir 0.05 Ir decay 0.1 batch 64 decay 0.001 scheduler cosine
T T_2 lambda_kkd 3
DER: ep_200 milestone Ir 0.2 Ir decay 0.1 batch 256 w docaa__0001__ss_ee_e____e 10.001_schcdulee cosine lambda
BEEF: ep_ ep_200 200_milestone. 3r 1015 1.15_Ir_decay decay_ 10_00.11 batch 128 l_ ddeedyyyy decaa__00001. Srrry__00000_ssccccleee__ooom
fusion__pp 2atianeep_200000 200_000_______________211001 _000200eeeeee enerey_er______ w 0_02 logits align__2_ 2.3 2.3
Best. hyperparameters (CIFAR-50, 6 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using CIFAR-50 (6 Tasks).
Replay: 70 10_70_mileen milestone 2 Ir 0.05 Ir decay 0.1 batch 32 decay 0.0000000111011 1rooo_schduule_oooo cosine
BiC: ep_ 120 120 milestone 2 Ir 0.05 Ir decay 0.3 batch 32 w decay decca__0.0001 schesnuler_cosinee 2.5 lambda. kd 1.5 split ratio 0.3
PODNet: ep_ ep_30 30__mlestooee___ 3 Ir 0.05 Ir "d__deaay___ 0.5 batch 64_ l___w_dee__ _dccyy secay_0.0003 0..0055sschduulee scheduler cicoduulr_ccosinee
lambda e lambda 3 nb proxy 30 ft. epochs 50 ft. Irate 0.003 adaptive factor False
FOSTER: ep 70 milestone 2 Ir 0.05 Ir decay 0.1 batch 64 1.5 1_____ammdda
lambda kd 1 fe beta 0.97 0.93 comp ep_ 200
MEMO: ep 160 milestone 4 0.05 Ir decay 0.1 batch 32 w decay 0.001 scheduler cosine
lambda aux 0.5 _=3mmaaaa bs 256
iCaRL: ep__ep__10000 120 milestone 2 Ir 0.05 Ir decay 0.1 batch 32 w decay 10000055schhdlllee
T lambda. _ambda__ax__ aux_
WA: ep_ 160 milestone 3 Ir 0.05 Ir decay 0.1 batch 64 w decay 100ay__0.0000 1ec00001_sheeulee__ooooo 2 lambda. kd 1.5
DER: ep_ 120 milestone Ir 0.05 docay 0.5. batch w decay 0.001 schedulee cosine lambda aux 1.5
BEEF: ep 30 milestone 4 Ir 0.05 Ir decay 0.1 batch 128 decay 0.0001 scheduler scheduler_stepp. steplr
fusion_ _arinn_ep____ _0_70 energy w 0.01 logits align 1.4
Best hyperparameters (ImageNet-50) 10 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using ImageNet-50 (10 Tasks).
Replay: ep 70 milestone 3 Ir 0.2 Ir decay 0.1 batch 64 decay 0.0001 scheduler steplr
21

---

Published in Transactions on Machine Learning Research (04/2025)
BiC: ep ep_ 120 ep_120 milestone 3 Ir 0.1 Ir decay 0.1 batch 32 w scco___0..00_sscccullrrr steplr T lambda. kd 3 split, ratio 0.1
PODNet: ep_30 1p_30__mlesssss milestone 4 Ir 0.2 Ir decay_ 0.3 batch 64 w w_w_ewa________e decca__0.0005______1111 oww..000sscccuueeerrr...... cosine lambda lambda.
nb proxy 50 ft epochs 20 ft Irate 0.007 adaptíve factor__Thae True
FOSTER: ep 120 120 milestone 3 Ir 0.1 Ir decay 0.1 batch
b.t_hach 32 w decay 0.0001 Beaol__shedeee_eeee steplr
T 11laamm lambda. kd fe beta 0.99 0.93 comp_ ep 160
MEMO: ep 120 milestone 3 Ir 0.1 Ir decay 0.1 batch 32 w decay 0.0001 Baap____hedee___e___ _tepprrstprrrrr
lambda aux examplar bs 256
iCaRL: ep 200 milestone 3 Ir 0.15 decay 0.1 batch 64 w decay 0.0001 scheduler_cosiiieee T 2.5 _a__aambbaa____ _ux__
WA: ep 200 milestone 3 Ir 0.15 Ir decay 0.1 batch 64 w decay 0.0001 scheduler cosine
2.5 lambda kd 2
DER: ep_e___2220 200 _0002000mmiae 1000_mmlstooe___1111111111111 3 Ir 0.15 Ir decay 0.1 batch 64 w decay_0.0001, 0.00011scheculer_comooo cosine lambda aux
BEEF NaN
Best hyperparameters (ImageNet-500, Tasks) The following represents the best. hyperparameters of each algorithm selected in the hyperparameter tuning phase using ImageNet-50 (6 Tasks).
Replay: ep_ 2p_200 milessooe 2 Ir 0.2 Ir decay 0.3 batch 32 w decay Bea_l__shedlle_setee
BiC: ep 120 milestone 3 Ir 0.1 Ir decay 0.1 batch 32 decay 0.0001 scheduler steplr
T 1 lambda kd 3 split ratio 0.1
PODNet: ep milestone Ir 0.2 Ir decay 0.3 batch 64 w decay__..000 scheduler cosine lambda c lambda 1 nb proxy 50 ft epochs 20 ft Irate 0.007 0.007 _at_atvve_aaattorrr__o__
factor True
FOSTER: 30 milestone 4 Ir 0.2 Ir decay 0.3 batch 64 w decay 0.0005 see___schh__eeeee__ee cosine lambda
fe beta 0.97_0.99 _t9___000________1 comp_ _pmpp_pp__0 _p_ 120
MEMO: ep_ee____000 120 milestone 3 Ir 0.15 Ir decay 0.1 batch 512 decay__o.00111 sww011schhhhhorrr_........ _sepl lambda aux 0.5 examplar bs 32
iCaRL: ep_ 200 milestone 3 Ir 0.15 decay 0.1 batch 64_ w_ dee_daayy00011111__ 1..000000000sdd0 Setp__0000_schhhlee__oooo T 2.5 lambda aux
ep_200_______00
WA: ep 120 milestone 4 Ir 0.1 Ir decay 0.5 batch 64 decay 0.0005 scheduler steplr
T lambda, kd_
DER: ep 120 milestone 4 Ir 0.3 Ir decay 0.5 batch 128 decay 0.0005 scheduler cosine lambda aux 0.5
BEEF: NaN
22

---

Published in Transactions on Machine Learning Research (04/2025)
A.2 Experimental settings for class-incrementar learning with pretrained model
Experimental details
For experiments using the proposed evaluation protocol class-incremmnnal learning algorithus with a pretrained model, we employ the PILOT (Sun et al, 2023] code for each algorithm.
The experimental sctup closely followed PILOT's environment, using Python 3.8, PyTorch 2.0.1, and CUDA 11.7.
14
10
6
4
2-
L2P
DuslPrompt CODA Prompt (2023)
Adam (2023)
FASF (2024)
(2023)
(2023)
Figure 11: # hyperparameters.
Pretrained hyperparameters The process of selecting hyperparameters for algorithms using a pretrained model is similar to the previous experiments. We comprchensively consider both general hyperparameters and algorithm-specific ones, finding the best hyperparameters during the tuning phase. Figure [] shows the number of hyperparameters for each algorithm. The predefined hyperparameters used for this process are listed in Table 5 Using the selected hyperparameters, we train each algorithm across the entire CL scenario. The range of cach hyperparameter of algorithm. Unlike the algorithms without pretrained models, which use the same optimizer (i.e., SGD), different optimizers have been used across algorithms in this case, optimizer. For hyperparameters of the optimizer that were not sampled, use the default values provided in PyTorch.
basod on values reported in previous work for cach type
also perform sampling for the
23

---

Published in Transactions on Machine Learning Research (04/2025)
Table 5: The predefined set of hyperparametes for class-IL IL with a pretrained model.
A Algorithm
Hyperparameter Name
3. 5, 10, 15, 20, 25 (for L2P, DualPrompt. CODA-Pormpt) / [5, 10, 15, 20, 25, 30
Epoch
(for Adau--Adapter, Ranpac, EASE)
[0.000875, 0.001375, 0.001875, 0.002375, 0.0025 (for L2P, DualPrompt. CODA-Pormpt) / [0.01, 0.02, 0.03, 0.04, 0.05]
All algorithms
LR
(for Adame Ranpac, EASE)
Num
[2, 3, 4]
milestones
LR
[0.1, 0.3, 0.5]
decay
Batch
18, 16, 24, 48, 64, 128)
size
(for L2P, DualPrompt, CODA-Prompt, Adam-Adapter
[0, 0.0001, 0.0005)
Weigh decay
(for 122, DualPrompt, CODA-Prompt)
/ [0.0001, 0.0005, 0.001, 0.005
(for Adam-Adapter, Ranpac, EASE)
LR
['steplr', 'cosine', 'constant) ]
Scheduler
Optimizer
'sg''' 'adam', 'adamw'
L2P, DualPrompt
M Size
10, 15, 20, 25, 30
L2P
Length (Ipp)
12, 4, 6, 8, 10)
L2P
Top k
2, 4, 6, 8, 10
L2P, DualPrompt
[0.1, 0.3, 0.5]
DualPrompt
Prompt length of (Lg)
5, 10, 15, 20, 30
Drappt
Length (Le)
5, 10, 15, 20, 30
CODA-Promph
Pool size
30, 50, 100, 200, 300
CODA-Prompt
Prompt length
(4, 8, 16, 24, 32)
CODA-Prompt
Orthogonality Mu
0..2, 0.1, 0.01, 0.001,
Adam-A Addmte, Ranpac, EASE
FFN num M
[4,8,16,32,64
Ranpac
[5000, 10000, 15000, 20000]
Ranpac
Prompt token num
[3, 5, 10, 20, 30, 50)
EASE
0.001, 0.05, 0.1, 0.15, 0.2)
Original hyperparameters in PILOT.
The following shows the original hyperparameters of each algorithm reported
L2P ep ep_10 10 milestone Ir 0.001875 Ir decay batch 32 w decay size__i_e_______________i__ 10 length 5 top k 5 lamb_ _0.1
scheduler scheduler_constannt constanttooptimzzer adam size
DualPrompt_ ep_10_
10 milestone 4 Ir 00.001 Ir decay 0.0 batch 24 w [_l___eeee_ decay _0.0 wwnstanttttnt...ccocc _otsssttttttttooooiiietttt aapimizee__ddmm size 10 T. 5 L. g top k lamb 0.1
CODA-Prompt ep 50 milestone 2 Ir 0.001 Ir decay 0.0 batch 128 decay 0.0
scheduler_ schecller__cosine_pppieeeee
_ddm __ol size__iiieeee 100 p length 8 __ootho___m__ _m 0.0
I.I_000 Adam 10 11__0___mmettte_______ 3r 0.05 Ir de_addcc___ 0.0 batch 16 w w_deccay_.omm
scheduler constant Botordull_ccosttnt___. -ptiimizerr sgd ffn num 100
Ranpac_ ep_1ep______________111._ 10 milestooe 2 0.05 Ir Ir_ decay _00 batch 16 w decay _0005 Baaeduler_constaale _costanttooptiiiizzr sgd ffn num 64 M 10000 pt num
EASE_ep__2 _________m___mmmeee__e_________ 4__r______________ 4__r__005__11111111111______111_ 1o5_Ir_dccay 0.0 batch 16 w "d_deca__00 0.005
scheduler cosine optimizer sgd [fn num 64 alpha 0.1
24

---

Published in Transactions on Machine Learning Research (04/2025)
Best hyperparameters (CUB-200, 20 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using CUB-200 (20 Tasks).
L2P: ep_2p__2___2..e__ 20 mi0_mletetoon__ 2 Ir 0.002375 Ir decay 0.5 batch 64 decay_000000 0.0001 scheduler_cconsannt constant _nnnmint__otimizer adamw_ size 15 length 6 top k 4 lamb 0.1
DualPrompt: ep 25 milestone 3 Ir 0.000875 Ir decay 0.1 batch 48 decay 0.0005
acheduler_connteenee optimizer adamw size 20 L 5 L g 30 top
lamb 0.3
CODA-Prompt: ep_ 25 milestone 2 Ir 0.000875 Ir decay_ 0.3 _0.3___33___00 _..00.3________ batch 24_w w deeaa__0.000s 0.0005
scheduler steplr optimizer sgd pool size 30 e p length ortho mu 0.01
Adam: ep _p__5__mlesstoee_____.. cchoduler_cosinnee pessie__ptimizer sgd ffn num
milestone 4 Ir 0.05 _r decay_ 0.5 batch 48 w8w
w decay 0.0005 0.0005
Ranpac: ep_30 30 _im_eeeee 4 1_.00.00001 Ir decay 0.1 batch 8 w decay 0.0005
schoduler_ cosiibllrrccoice____ cossnnc_optiiiizerrr.r..oee _gdd_ ffn num 32 _2_M___ 20000_pt__0001 1i__nm__
EASE: 1p_15 op_15__ilestonee_________ milemmioo_e..4 4 Ir 0.02 Ir decay_eyyooo 0.5 batch 128 w_ decay__..000 0.001
scheduler cosine
_osiine optimizerr sgd ffn num 8 alpha 0.01
Best hyperparameters (CUB-200, 10 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter luning phase using CUB-200 (10 Tasks).
L2P: 25 milmmitestonee 2 02200001185__111 decay_eyyyoo batch 64 ww l__ de__dd_ay000033 0.0005
scheduler cosine optimizer adamw size 10 length top k 6 lamb 0.5
DualPrompt: ep 25 milestone 3 Ir 0.0025 Ir decay 0.5 batch 128 decay 0.0005
scheduler_stepe optimizer _gdd___d_____ size 20 L e 10 L g 10 top k 1 lamb 0.5
CODA-Prompt: ep 25 milestone 3 Ir 0.0025 Ir decay 0.3 batch 64 decay 0
cestneeoptimizer adamw pool size 100 P length 8 _orthh mu
Adam: ep__0 20 _0_milestone____ 3 Ir 0.04 Ir decay_0 0.3 batch 8 w 0.0005
scheduler Saheuller_steplrr cttssst_ppppiiizeeee optimizer sgd ffn num 32
Ranpac: e0_30_________ _0_30_mii mi0_milee 4 0.02 Ir decay_ 0.3 batch _6_ w_decay__ 1ecca__000001 schoduler _otiiiiorptiierr__oo sgd ffn num 64 M 10000 pt num 3
0.0001
EASE: ep_ 15 milestone 4 Ir 0.01 Ir decay 0.3 batch 64 w decay 0.0005
scheduler steplr optimizer sgd ifn num alpha 0.05
Best. hyperparameters (ImageNet-R, 20 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using ImageNet-R. (20 Tasks).
L2P_ep cp__p_____ 25 milestone 3 Ir 0.000875 Ir docay 0.5 batch _4_ ______oooe "_dcca 0 scheduler steplr optimizer adamn size 10 length 10 top
lamb 0.5
DualPrompt: ep 15 milestone 4 Ir 0.001875 Ir decay 0.5 batch 128 decay
scheduler steplr optimizer adam size 20 L e 30 L g 5 top k lamb 0.5
CODA-Prompt: ep 15 milestone 2 Ir 0.002375 Ir decay 0.1 batch 48 decay 0.0001 scheduler schecller__cosie__opeeeeee cosinne_ottimizer_eeee adamw __poool size 300 length__h___ 32 ortho mu 0.001
Adam: ep_ 20__2_mmmstoonecmcmm 3 Ir 0.05 Ir decay 0.5 batch 64 w decay_eeeoooe 0.001
scheduler constant optimizer sgd [Tn num 64
Ranpac: 20__20mmmletonee 2 2r_rr 0.005 decay 0.3 batch 24 w_ ___deca___0000 0.0005
scheduler constant optimizer sgd (fn num 16 M 15000. pt. 20
25

---

Published in Transactions on Machine Learning Research (04/2025)
EASE: ep ___15_____ milestone_ _4 0.04 Ir ir_deca__ 0.1 batch 24 w 0.0001
scheduler _oosssant optimizer sgd ffn num 16 alpha 0.15
Best. hyperparameters (ImageNet-R, 10 Tasks) The following represents the best hyperparameterss of each algorithm selected in the hyperparameter tuning phase using ImageNet-R (10 Tasks).
L2P: ep_ 25 milestone_ Ir 0..00030 decay_yyooo 0.5 batch 128 decay 0 scheduler constant optimizer adamw size 20 length 6 top k 10 lamb 0.3
DualPrompt: ep_25 25 33_milestoon___ 2 Ir 0.001375 Ir decay 0.3 batch 128 w __decay__ dec_a_eeca00000 0.0005 caneduler_constane optimizer adamw size 30 I. 20 I. g 20 top k lamb 0.3
CODA-Prompt: ep 20 milestone 2 Ir 0.001375 Ir decay 0.1 batch 48 w decay_ scheduler steplr optimizer adam pool size 300 length 8 ortho mu 0
Adam: 30_30_mmm 200_milestonn_2221111111_1__00__ 2r 0.05 Ir delcddeyyyoo 0.1 batch 64__ d__deea__ou 1000a 0.001
scheduler cosine optimizer sgd (fn num 32
Ranpac: ep milestone 3 Ir 0.03 Ir decay batch 24 w decay 0.0001
scheduler feeplr__pttmiieee sgd ffn num 64 M. 20000 pt num 20
30 EASE: ep_30_______0__00000_000022e_e milesmtone _4 Ir 0.05 Ir "eccy 0.3 batch 128 Mecaa__0.001 0.001
scheduler cosine optimizer adam IIn num 16
Best hyperparameters (CUB-100-1, 20 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using CUB-100-1 (20 Tasks).
L2P: ep 20 10_mmlestone__ 3 Ir 0..00.00235511 Ir decay__yoo 0.3 batch 128 decay 0.0005 scheduler constant _on optimizer adamn size 20 length 8 top k lamb 0.5
DualPrompt: ep_2_ 25 25_milestone__ scheduler constant optimizer adam size 15 T. e 15 T. g 20 top k lamb 0.5
_011 0.001375 _eeecy______ 0.1 batch 128 w _______ee_e w_decay__
CODA-Prompt: ep 10 milestone 4 Ir 0.0025 Ir decay 0.3 batch 64 decay 0 constant
_ optimizer adam pool size _00 p length 4 ortho mu 0.001
Adam: cp_ 5 __milestooeee____ 1..0 Ir decay 0.5 batch 16 w_ decay d___eca___o00 0.0005
scheduler constant optimizer sgd ifn num 4
Ranpac: 15_15 10_15_mmleston 4 0.04 Ir decay 0.1 batch _28_________ "__deeay____ 0.0001 scheduler _osiiee optimizer sgd (fn num M 15000 pt num
EASE: ep 10 10_10 milestone 4 Ir 0.01 Ir decay 0.1 batch 16 w decay 0.001
scheduler constantt optimizer sgd ffn num 32 alpha 0.05
Best hyperparamcters (CUB-100-1, 10 Tasks) The following represents the best hyperparametors of each algorithm selected in the hyperparameter luning phase using CUB-100-1 (10 Tasks).
L2P: ep_ 25 milestone 2 Ir 0.0025. Ir decay 0.3 batch 128 w decay 0 scheduler cosine Lesineeoptimizerr adam size___ 20 length 4 top k 6 lamb 0.5
DualPrompt: _p_25 25_milcstone 4 Ir 0.002375 decay 0.3 batch 128 ____w__eeeee _ddcc decay_000005 0.0005 e g 15 top
scheduler cosine optimizer adamw size
lamb 0.5
CODA-Prompt: ep__5_ _p_25_milestone milestone 3 lr 0.001375 Ir decay 0.1 batch w_ ___deeaa__o00 scheduler cosine optimizer adamw pool size 50 p. length 4 ortho
26

---

Published in Transactions on Machine Learning Research (04/2025)
Adam: 25 milestone 2 Ir 0.05 Ir decay__yoo.. 0.3 batch 24 w 0.0001
scheduler steplr optimizer sgd ffn num 32
Ranpac: ep_ ep_25 milestone 3 0.02 Ir decay 0.3 batch 16 w "___deaa__0000
scheduler constant constantt_optmiizer sgd ffn num 16 M 10000 pt 20
EASE: ep_15 15 milestone 3 Ir 0.03 Ir decay 0.5 batch 128 w decay 0.0001
scheduler constant _otiimieer sgd fin num 64 alpha 0.05
Best hyperparameters (ImageNet-R-1, 20 Tasks) The following represents the best hyperparameters of each algorithm selected in the hyperparameter tuning phase using ImageNet-R--1 (20 Tasks).
L2P: ep_15__ milestone 2 Ir 0.002375 Ir decay 0.5 batch 48 w decay_ 15 scheduler tensine cosine optimizer adamw size 25 length 6 top k 10 lamb 0.3
DualPrompt: ep milestone 3 Ir 0.001875 decay 0.1 batch 128 w decay 0
scheduler_steppi optimizer adam size 10 L e 30 L g 30 top k lamb 0.1
CODA-Prompt: ep 15 milestone 2 Ir 0.002375 decay 0.5 batch 64 w decay 0.0001
scheduler cosine optimmiee adammw pool size 100
length 4 ortho mu 0.01
Adam: ep_ 25 milestone 3 Ir 0.04 Ir decay 0.3 batch 24 decay__e_lll
scheduler ccheduler_constannt _ottraantppttmizer____ ssgddiiirr_sdd_____ ffn num 16
Ranpac: ep milestone 2 0.02 Ir decay 0.3 batch w decay 0.0001
scheduler_ cosine_optimizer sgd ffn
8 M 20000 pt num 10
EASE: ____1___ilsttte_____ 4_ 0.003 _r "r__deay___a 0.5 batch 16 wwww decay__oooll 0.0005 11
scheduler constant optimizer sgd Iīn num 64 alpha 0.05
Best hyperparameters (ImageNet-R-1, 10 Tasks) The following represents the best hyperparametens of each algorithm selected in the hyperparameter tuning phase using ImageNet-R-1 (10 Tasks).
L2P: ep 20 milestone 3 Ir 0.000875 Ir decay 0.1 batch 24 decay 0.0001 scheduler_coste cosine _octinn_ttimizerrrrooooee adamw size 20 _enghhhgh___ 8 top k lam_a_b___ 0.5
DualPrompt: ep ep___ milestone 4 Ir 0.000875 decay__ya 0.3 batch 64_ lamb 0.1
_www__ _____ddoo decay
scheduler cosine optimizer adam size 15 30 g 20 top
15 15_milestone___ 3 Ir lr__r_____ 0.001375 __cccy___a 0.5 batch 64_w w_deeay 0.00500.0005 0.0005 CODA-Prompt: ep_15
scheduler constant optimizer adamn e pool size 300 e p length ortho mu 0.01
Adam: ep 25 milestone 3 Ir 0.04 Ir decay 0.3 batch 24 w decay 0.001
scheduler constant_optimiann _sgd ffn mum 16
Ranpac: ep 25 milestone 4 Ir 0.05 Ir decay 0.1 batch 24 decay 0.0001
scheduler scheduler_conntanne
constant _ptimmizerree sgd ffn num 64 M 20000 pt. 10
0.04 scheduler constant, _ptimmizrr sgd ffn num 64 alpha 0.2
EASE: [r__10_milsstoee 0.04 Ir decay_ 0.1 _.100.1aaaaaa batch 24 w decay 0.001
24
27

---

Published in Transactions on Machine Learning Research (04/2025)
B Additional Experimental Results on the Evaluation Phase
B.1 Discussion on the number of hyperparameters
We believe that the mumber of hyperparameters can influence the evaluation protocol proposed in this paper. This is because a larger number of hyperparameters allows an algorithm to fit. more aggressively during the hyperparameter tuning phase, making it easier to achieve higher performance. However, such overfitting can lead to poor generalization in the evaluation phase. This trend can be observed Figure 4(b) Among the evaluated algorithms, Foster has the largest number of hyperparameters (see Figure 10). The experimental results show that while Foster reports outstanding (overfitted) performance during the hyperparameter tuning phase, its performance significantly deteriorates in the actual evaluation phase, We believe this finding further
underscores the necessity of our proposed protocol for properly assessing each CL algorithm.
B.2 Instability issues with BEEF baseline
During experiments with the BEEF baseline, encountered persistent NaN issues when training ImageNet-scalle datasets using ResNet-18. First, verified the hyperparameters used in our experiments to ensure they matched those specified in the original BEEF paper (Section C.2). Specifically, we confirmed that the learning rate of 0.1 (with a StepLR scheduler) and a mini-batch size of 256 were consistent with the settings in the original experiments. After confirming that there were no discrepancies in the hyperparameters, we investigated community reports related the issuc. In a relevant thread on the PyCIL repository, another user had reported similar NaN problems and suggested lowering the learning rate as potential solution. Following this advice, we conducted additional experiments using lower learning rates, testing values of [0.001, 0.005, 0.01, 0.015, 0.02]. Despite these adjustments, none of the configurations avoided NaN results across all seeds. Representative results for two configurations that showed relatively better performance are presented in Table [ These findings suggest that the NaN issue is not due to hyperparameter misconfiguration but rather an intrinsic instability in the BEEF algorithm.
Hyperparameters
Acc / AvgAcc
Seed 0
Seed 1
Seed 2
Seed 3
Seed 4
BEEF (HP1)
49.52 / 65.57
49.24
59.22
NaN
NaN
NaN
NaN
BEEF (HP2)
48.62
65.14
46.40
58.31
NaN
NaN
NaN
NaN
Table 6: Performance of BEEF with different hyperparameters on ImageNet-100 100 dataset.
Further investigation into the BEEF code within the PyCIL implementation revealed that the instability originates from the adversarial learning process used by BEEF. The process generates adversarial examples to compute the energy loss, which leads to extreme value growth in the feature maps. Specifically, the feature maps from the copied model (_ [_ntwork__oo copy) undergo unbounded amplification due to adversarial samples, ultimately resulting in NaN values during training. This instability appears to be an algorithmic flaw rather than a minor numerical issue. To mitigate this, we suggest applying regularization techniques, such as L2 or L1 regularization, during the adversarial example generation process. However, we acknowledge that such regularization may affect the performance of BEEF, as its reported success on CIFAR and ImageNet datasets could depend the current unregulated adversarial process. Given that BEEF exhibits instability across multiple seeds and hyperparameter configurations, we conclude that the NaN issues are an inherent limitation of the algorithm, particularly when applied to large-scale datasets like ImageNet. As a result, we report these findings to ensure transparency and will continue to investigate potential solutions future work.
B.3 Result tables
Class-IL without a pretrained model (D#T = IImageNet-100-1)
Performance comparison between original and our found Hyperparameters Based on the above experimental results, we can make the following observations. First, some algorithms achieve better performance with the original hyperparameters (e.g., FOSTER and MEMO). However, in contrast, there are cases where the best hyperparameters we identified lead to better performance. These results demonstrate that.
28

---

Published in Transactions on Machine Learning Research (04/2025)
Table 7: The experimental results of class-IL without a pretrained model (using original hyperparameters). Each result represents the Final Accuracy and Average Accuracy (Ace AvgAAc). The values parentheses represent the standard deviation.
10 Tasks
D ImageNNet100-11
DE = ImageNeet100-2
(Acc / AvgAcc)
Replay
41.21(1.06)
59.82(1.48)
41.00(1.46)
61.73
iCaRL
40.50(1.19)
60.12(1.41)
41.24(1.31)
(2.87
BiC
39.61(2.39)
(1.27(1.59)
37.62(3.73)
(6.25(1.17)
WA
53.34
(8.92(1.54)
57.81
(3.62
PODNet
46.66(1.11)
(4.13
48.86(1.15)
67.38(1.92)
DER
61.96(1.04)
(2.10
15.73(1.06)
76.12(1.10)
FOSTER
60.68(0.71) / 69.97(1.70)
(3.93(1.06)
(1.50)
BEEF
NaN
NaN
MEMO
(1.59) / 70.04(1.62)
63.42(0..88 / 75.25(1.21)
Table 8: The experimental results of class-IL without a pretrained model (using D ImageNNe-100-1) in the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
10 Tasks
DHT ImageNet-100-1
D ImageNet-100-2
(Ace / AvgAcc)
Replay
44.78(1.19) 59.85(0.95)
44.27(1.05) / 61.49(0.87)
iCaRL
42.58(1.06) (1.27((..26
42.44(1.50)
(3.39
BiC
54.22(1.27)
(0.74
58.77(0.96)
(1.81
WA
54.66(00.00)
(9.54)1141
59.89(1.18)
(1.93
PODNet
55.33(0.933)
(8.77(1.522)
57.48(0.94)
71.76(1.62)
DER
(0.42
(2.97
70.23(0.46)
77.12(1.20)
FOSTER
58.36(0.85)) (1.99
61.46(0.98)
68.41(1.23)
BEEF
NaN
NaN
MEMO
57.91(0.54)) 71.25(1.41)
61.94(0.78)
71.35
using the original hyperparameters does not always guarantee optimal performance across all CL scenarios. Furthermore, they show that performing hyperparameter tuning using the evaluation protocol we propose is more appropriate and practical approach for such CL scenarios.
29

---

Published in Transactions on Machine Learning Research (04/2025)
Table 9: The experimental results of class-IL without a pretrained model (using D ImageNett100011 the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
6 Tasks
DBT ImmaeeNet-100
DB = ImageNet-2000
(Acc / AvgAcc)
Replay
(2.41)
53.81(1.72)
43.26(1.38)
49.28(0.53)
iCaRL
46.62(1.54)
57.27(0.73)
45.64(1.49)
59.18(0.54)
BiC
37.14
36.42(1.89)
38.43(2.53)
(3.89(3.07)
WA
58.72(1.02)
(5.58(1.55)
60.58(1.35)
69.77(1.71)
PODNet
67.22(0.67)
75.05(1.16)
65.51(1.83)
75.82(1.03)
DER
72.20(0.51)
(7.68
(5.63
81.19
FOSTER
69.48(0.500 (1.18
71.62(1.08)
78.29(1.14)
BEEF
74.67(0.14)
(8.92(0.54))
75.09
81.31(0.50)
MEMO
59.91(0.877 67. 67.22(1.63)
62.80(3.16)
68.77(6.26)
Class-IL without a pretrained model (DIT CIFAR-50-1)
Table 10: The experimental results of class-IL without a pretrained model (using D CIFAR-50-1) the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc) AvgAcc). The values in parenthescs represent the standard deviation.
10 Tasks
DE CIFAR-50-2
DE ImageNet-55022
(Acc / AvgAcc)
Replay
45.42(2.19)
65.88(1.97)
(2.57
60. 60.72(1.58)
iCaRL
47.12(2.80)
(6.71
42.44(1.00) / (1.65)
BiC
52.83(2.83)
69.16(2.30)
49.52(1.16)
(7.0991.74)
WA
52.83(2.13)
(9.85(2.32)
53.64(1.47)
67.75(1.90)
PODNet
51.20(1.76)
69.47(0.13)
51.70(1.19)
67.86(1.67)
DER
63.51(1.98)
(1.04(1.24)
63.40(1.02)
72.67(1.62)
FOSTER
60.00(2.72)
72.29 (2.09)
62.09(1.83)
70.24(1.50)
BEEF
57.24(1.48)
72.26(2.05)
NaN
MEMO
60.72(2.41)
73.78
51.91(1.59) / 68.0062.10)
30

---

Published in Transactions on Machine Learning Research (04/2025)
Table 11: The experimental results class-IL IL without a pretrained model (using D CIFAR-50-1) the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
6 Tasks
DE CIFAR-50-2
DC ImageNet-50-22
(Acc / AvgAcc)
Replay
48.00(1.98)
59.86(1.03)
46.30(1.31)
55.67(0.64)
iCaRL
46.09)1.51)
59.14(1.39)
46.21(1.72)
57.79(1.06)
BiC
58.22(1.20)
68.16(1.96)
46.26(3.26)
59.07(3.87)
WA
(1.07
70.56(0.51)
61.47(0.72)
69.67(0.63)
PODNet
(2.62
72.62(0.75)
64.30(0.78)
73.56(1.01)
DER
67.98
75.88(0.78)
70.68(0.75)
76.56(0.95)
FOSTER
(0.55 (3.77
69.86(0.45)
75.27(0.83)
BEEF
65.51(1.29)
72.98(0.50)
NaN
MEMO
64.64(1.54) 73.50(0.83)
51.40(3.39)
62.
Class-IL without a pretrained model (D#T ImaaeNNet500-1
Table 12: The experimental results of class-IL without pretrained model (using DHT ImageNet-50-1)
the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc) AvgAcc). The values in parenthescs represent the standard deviation.
10 Tasks
DE ImageNet-550-2
DE = CIFAR-50-2
(Acc / AvgAcc)
Replay
43.71(0.81)
58.75(1.60)
44.19(2.17)
63.57(1.50)
iCaRL
39.41(1.46) 59.51(1.70)
41.59(3.10)
62.42(2.85)
BiC
51.26(1.39)
(5.433
51.22(3.67)
(6.41)2.92)
WA
50.85(0.79)
(1.79)
57.72(1.92)
71.39
PODNet
51.31(1.24)
(7.23
18.19(1.17)
(1.77
DER
(4.89(1.16)
74.15(1.56)
63.64(1.32
75.22(1.21)
FOSTER
61.57(0.77) / 72.38 (2.38(1.20)
58.64(2.15)
72.89(1.81)
BEEF
NaN
NaN
MEMO
57.56(1.24) (2.266(2.27
58.99(1.01)
72.43(1.81)
31

---

Published in Transactions on Machine Learning Research (04/2025)
Table 13: The experimental results of class-IL without pretrained model (using D ImageNet-500-1 the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
6 Tasks
DE --mageNet-50-2
DE = CIFAR-50-2
(Acc / AvgAcc)
Replay
42.82 (1.43)
(3.50
42.28(0.71)
52.18(1.31)
iCaRL
42.47(1.73)
(4.65(1.85)
40.24(2.64)
52.89
BiC
44.68(2.81)
(4.19(2.93)
39.65(1.32)
(9.49(1.46)
WA
55.68(0.07)
(4.69(0.72)
56.14(1.99)
(4.08(1.60)
PODNet
64.10(0.80)
72.50(0.81)
(1.33
71.27(1.07)
DER
70.28(0.98)
(6.14(1.00)
64.76(1.06)
72.89
FOSTER
68.40(1.08) 75.02(0.94)
65.31(0.26)
73.80(0.68)
BEEF
NaN
NaN
MEMO
50.92(1.25) (0.93(1.67)
50.58(2.62)
60.66(2.65)
Class-IL with pretrained model (D"T CUB-200
Table 14: The experimental results of class-TT. with a pretrained model (using original hyperparameters)
Each result represents the Final Accuracy and Average Accuracy (Acc AvgAAc). The values parentheses represent the standard deviation.
10 Tasks
D = CUB-200
(Acc AvgAcc)
L2P
72.32(0.62)
76.82(0..0))
DualPrompt
68.74(0.54))
74.39
CODA-Prompt
75.19(0.33)
80.27(0.9))
Adam
71.21(1.06)
77.52(1.24)
Ranpac
78.27(0.57)
83.24(0044)
EASE
77.07(0.19) / 82.65
Table 15: The experimental results of Blass-IL with a pretrained model (using D = CUB-200) in the hyperparameter tuming phase. Each result. represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
20 Tasks
DE ImageNet-R
DE ImageNet-A
(Acc / AvgAcc)
L2P
(9.93
75.90(0.23)
10.92(1.53)
51.24(1.39
DualPrompt
67. 67.20(0.78)
(3.79(0.64)
11.00(1.07)
54.12(0..6)
CODA-Prompt
68.63(0.64) 74.66(0.84)
48.20(1.05)
57.94(0.88)
Adam
67.70(1.38) 74.44(1.35)
49.61(0.29)
59.67(0.80)
Ranpac
78.72(0.40) 83.71(0.56)
62.95(1.41)
68.64(2.58)
EASE
61.94(0.06)
68.36(0.63)
(9.37(0.12)
59.48(0.75)
32

---

Published in Transactions on Machine Learning Research (04/2025)
Table 16: The experimental results of class-IL with a pretrained model (using D CUB-200) in the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
10 Tasks
DE ImageNet-R
DE ImageNet-A
(Acc / AvgAcc)
L2P
71.86(0.66)
(7.42
45.13(1.25)
53.57(0.92)
DoapPrompt
66.33(0.42)
73.03(0.60)
39.97(2.32)
52.58(0.70)
CODA-Prompt
72.86(0.44)
(8.49(0.99)
51.63(0.50)
61.00(0.47)
Adam
72.68(0.77) (9.09(0.89)
57.03(0.47)
66.50(1.22)
Ranpac
79.59(0.29)
84.46(0.41)
66.14(0.40)
73.63(1.05)
EASE
61.96(0.06)
67.74(0.67)
49.32(0.48)
58.30(0.86)
Class-IL with a pretrained model (D#T_ = ImageNet-R)
Table 17: The experimental results of class-IL with pretrained model (using D ImageNet-R) in the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
20 Tasks
D CUB-200
DE ImageNet-A A
(Ace / AvgAcc)
L2P
63.76(1.81)
76.39(1.18)
36.97(1.31) / 16.78(0.71)
DualPrompt
68.78(0.78) 79.67(1.04)
17.54(0.799) / 55.91(0.84)
CODA-Prompt
67.92(2.11) 79.65(1.93)
50.07(0.29)
59.76(0.58))
Adam
85.38(0.19) 90.87(0.90)
53.86(1.11)
63.99(2.61)
Ranpac
89.86(0.22)
93.44(0.788)
38.53(31.11)
/ 67 67.65(3.37)
EASE
79.29(1.22)
87.58(1.19)
53.99(1.05) / 64.11
Table 18: The experimental results of class-IL with pretrained model (using DHT ImageNet-R) in the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Ace AvgAcc). The values in parentheses represent the standard deviation.
10 Tasks
D F = CUB-200
D = ImageNet-A
(Acc / AvgAcc)
L2P
69.75(1.79) 79.92(1.24)
43.50(0.99)
50.06(1.18)
DualPrompt
71.74(1.01) 82.22(1.10)
39.47(0.79)
50.63(0.94)
CODA-Prompt
72.30(1.11) (3.30
52.39(0.38)
61.87(1.01)
Adam
85.90(0.17) (0.9930.88)
56.63(0.78)
65.94(1.45)
Ranpac
89.99(0.29) 93.36(0.833
63.78(1.52)
71.70(1.88)
EASE
74.00(0.78) (3.69().74)
(4.76(1.36)
66.11411665
33

---

Published in Transactions on Machine Learning Research (04/2025)
Class-IL with pretrained model (D"T CUB-100-1)
Table 19: The experimental results of class-TT. with a pretrained model (using DAT CUB-100-1) in the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
20 Tasks
D CUB-100-2
D InageNet-RR22
Image.Net-Al
(Acc AvgAcc)
L2P
54.12(3.59)
68..3(3.73)
66.01
72.17(1.04)
28.08(2.38)
39.18(2.75)
Prapprompt
59.83(1.63)
(3.54(2.68)
65.51(0.32)
71.58(0.68)
33.90(2.26)
(4.84(2.25)
CODA-Prompt
58.16 71.05
66.73(0.61)
73.06(0.46)
30.62(0.82)
41.70(1.70)
Adam
85.95(0.08)
(0..6))
67.77(0.84)
74.53(1.74)
43.93(0.09)
55. 55.63(2.69)
Rampac
89.52(0.335
90.52(2.96)
74.53(0.28)
79.80(0.81)
30.30(22.41)
45.87(4.57)
EASE
85.19(0.49)
(0.94)
67.17(0.29)
73.61(0.75)
44.11(0.29)
55.42(2.83)
Table 20: The experimental results of class-IT. with pretrained model (using DBT CUB-100-1) in the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
10 Tasks
D = CUB-100-2
DE ImageNet-R-22
ImageNel-A-2
(Ace / AvgAoc)
L2P
66.15(1.41) / 76.68(1.49)
70.11
(0.67
34.96(0.92) / 44.98(2.26)
DualPrompt
67.20(2.59)
78.28(1.68)
(8.29(0.49)
74.32(0.89)
38.43(1.52)
/ 49.15(2.43)
CODA-Prompt
68.37(2.71)
78.93(2.57)
70.35(0.81)
(5.59(0.90)
37.23(1.87)
47.48(1.85)
Adam
86.76(0.21)
90.75(0.46)
72.73(0.27)
79.42(0..5)
44.81(0.85)
55.08(2.22)
Ranpac
90.60(0.36)
93.08(0.65)
80.40(0.3)
85.00(0.47)
49.56(2.52)
57.60(1.96)
EASE
85.86(0.10) 90.11(0.26)
63.36(0.03)
69.36(0.95)
43.88(0.15)
54.49(2.64)
34

---

Published in Transactions on Machine Learning Research (04/2025)
Class-IL with pretrained model (D"T ImageNectR-1)
Table 21: The experimental results of class-IT, with a pretrained model (using DHT ImageNet-R-1) the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc AvgAcc). The values in parentheses represent the standard deviation.
20 Tasks
D ImageNet-R-22
D CUB-100-2
ImageNet-AA22
(Acc AvgAce)
L2P
66.15(0.85)
71.93(1.13)
51.04(1.45)
66.04(1.71)
25.13(2.27)
34.21(2.51)
Domppt
65.77(0.78)
(1.87
57.13(3.40
71.15(2.25)
31.96(2.49)
41.71(1.76)
CODA-Prompt
66.44(0.66)
72.62(0.36)
57.24(1.90)
71.27(11.95)
30.48(1.62)
41.30(2.56)
Adam
70.69(0.73)
77.86(0.51)
86.35(0.14)
90.83(0.56)
44.25(0.86)
55.84(2.75)
Ranpac
76.15(0.93)
81.68(0.94)
73.73(31.52)
89.58(2.03)
35.06(15.86)
47.04(6.07)
EASE
75.16(0.68)
81.68(0.71)
76.36(2.61) 84.35(2.55)
42.49(1.76)
54.40(3.21)
Table 22: The experimental results of class-IL with a pretrained model (using D = ImageNet-Ra. the hyperparameter tuning phase. Each result represents the Final Accuracy and Average Accuracy (Acc
AvgAcc). The values in parenthescs represent the standard deviation.
10 Tasks
D ImageNoctR-2
D CUB-100-2
Image.Net-A-2
(Ace AvgAce)
L2P
70.35(0.64)
75.66(0.30)
63.71(2.33)
74.62(1.61)
29.10(1.24)
38.80(1.44)
DualPrompt
69.97(0.25)
(0.93
66.66(1.12)
78.11(1.43)
32.42(0.68)
42.31(2.02)
CODA-Prompt
72.17(0.46)
(7.50
66.98(1.3)
/ 78.70(0.988
37.04(1.49)
(6.47(2.45)
Adam
72.84(0.67) 79.69
85.26(0.41)
89.77(0.45)
37.36(2.72)
48 18.622(4.) (4.07)
Raupac
80.70(0.50)
(85.28(0.4))
91.09(0.51)
91.63(3.51)
41.98(19.61)
58. 79..77))
EASE
(8.33(0.41)
(0.82(0.71)
79.70(1.47)
(6.23(1.59)
42.49(0.69)
53 (2.69(2.61)
35

---

Published in Transactions on Machine Learning Research (04/2025)
B.4 Training graphs
Class-IL without a pretrained model (DHT CIFAR50-1, D CUB50-2)
90
90
80
80
AcC:
70
(CaRL(2017)
Replay
AcC₁
70
(CaRL(2017)
4eplay
BiC/201.) 9)
3IC/2019)
60
WA(2020)
POIDNe((222)
60
VAA2002D
2DDNct(2020)
DERI2021)
DER(2021)
50
FOSTER(2022)
50
(0STER(2022)
BEEF(2023)
M#M0(7023]
MEMO(2023)
40
40
2
(a) 10 tasks
(b) tasks
Figure 12: Experimental results on the evaluation phase.
Class-IL without pretrained model (D CIFAR50-1, D ImageNet50022
90
90
80
70
Rep ay
60
Jeplay
ACC.
70
BIC:2019)
50
3IC(2019)
Ww/2020)
WA[2020)
60
P2DN))(2200)
10
PODNet(2020)
DERIZ021)
DER(2021)
50
TOSTER|2022)
30
(OSTER(2022)
BEEF(2023)
SEEF(2023)
20
MEMO(2023)
40
2
5
8
9 10
2
4
6
(task number)
t (task number)
(a) 10 tasks
(b) tasks
Figure 13: Experimental results on the evaluation phase.
Class-IL without a pretrained model (D IImageNNe50-1,, D ImageNNe50-22)
90
80
80
70
70
Rep ay
60
Seplay
ACC:
CaRL/2017)
(cALl20177)
(IC1)019)
50
SiC(2019)
WA/2020)
WAI2020:
50
PODNe5(2020)
40
2ODNet(2020)
DER(2021)
FOSTER/2022)
30
=OSTER(2022)
40
RFFF(2022)
3FFF(2023)
MEMO12023)
20
MEMO(2023)
2
5
8
9
10
2
4
6
(a) 10 tasks
(b) tasks
Figure 14: Experimental results on the evaluation phase.
36

---

Published in Transactions on Machine Learning Research (04/2025)
Class-IL without pretrained model (D#I
ImageNee50-11, D CIFAR50-2)
90
80
70
70
Reş
60
Replay
ACC
CaRL(2017)
AcC+
(CARL(2017)
60
BiC;2019) 9)
50
3iC(2019)
WATZUZU)
PODNet(2020)
40
DERI2021)
DER(202
FOSTER/2022)
30
FOSTER(2022)
40
BEEF(2023)
3EEF(2023)
MEAAD)2023)
20
MEMO(2023)
30
5
9
2
(a) 10 tasks
(b) tasks
Figure 15: Experimental results the evaluation phase.
Class-IL with pretrained model (D"T CUB100-1, CUB100-2)
110
DualPrompti2023) CODA- CODA-Prompt Prompp)(223)
Ranpac(2023)
100
EASE(2024)
AcCt
AcC:
85
80
80
12P(2022)
70
75
DualPrompt(2023)
CODA Prompt:2023)
70
ADAM[2073)
Rangac(2023)
65
EASEI2024)
50
1617
18120
2
3
5
8
9
10
(a) 20 tasks
(b)
10 tasks
Figure 16: Experimental results on the evaluation phase.
Class-IL with a pretrained model (DHT CUB100-1, D
ImageNet-RR,
110
95
.2P(2022)
ADAM112023
DualPromptizoza)
Ranpa:(2023)
100
COD4-Fremgt/2023
EASE(2024)
90
90
85
AcCt
ACC+
80
_2P(2022)
70
Due (toPPommp/22233
75
CODA-Promet12031
ADAM(2023)
60
70
3anpac12023)
EASEI2024)
9101122124411077188889
3
7
10
(a) 20 Lasks
(b) 10 tasles
Figure 17: Experimental results on the evaluation phase.
37

---

Published in Transactions on Machine Learning Research (04/2025)
Class-IL with pretrained model (D"T
ImageNet-R-1,.D ImageNettR-2)
100
100
L2P12022]
95
Pramptt22222
95
C0DM-Prompt22023
ADAMI2023)
90
Ranpac(2023)
AcC
85
FASE(2024)
ACCr
85
80
80
L2P120221
75
DualPromptt2021
CODA-Prumpt:025)
70
AnAM(2023)
Ran2ac(20223
EASE(2024)
65
1011 1617
19120
(a) 20 lasks
(b) 10 tasks
Figure 18: Experimental results on the evaluation phase,
Class-IL with pretrained model (DHT ImageNet-R-1,, D CUB100-2)
100
100
90
AcC
AcC+
80
L2P(2022)
DualFrompa12023)
DualPtomptt2023
COCA PODA-Promattien Cro at(2023)
60
CODA-Prompt2003)
ADAM(2023)
ADAM)2003)
Ranpe:12023)
50
Ranpar
EAS5(2024)
EASEI2024)
40
10 1011111811111188888
2
3
9
10
(a) 20 tasks
(b) tasks
Figure 19: Experimental results on the evaluation phase.
Class-IL with a pretrained model (DHT --m D ImageNeet-A22)
90
90
L2P(2022)
ADAM[2023)
2203
Rarps:(2023)
80
80
Prompt|22223
FASF(2074)
70
70
ACC:
AcC,
60
400
.2P(2022)
40
Sua/Prompt/2023)
40
30
ADAM(2023)
30
Ranpac(2023)
_ASL(2924)
20
021111231115111111110
3
10
(a) 20 tasks
(b) tasks
Figure 20: Experimental results the evaluation phase.
38


Reviews:
[{"review_id": "CGvMaODyLg", "reviewer": "Reviewer_2Qws", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. The paper tackles an important issue permeating the field of Continual Learning. As far as I know, this is the first effort to decouple the hyperparameter tuning and evaluation phases in the way that the paper proposes.  \n2. The paper includes experiments with and without pre-trained models, accommodating the current practices in machine learning where starting from pre-trained weights is becoming increasingly the norm.  \n3. The visualizations are clear and the algorithms are helpful in understanding the methodology\n\n## Weaknesses\n\nThe paper has flaws regarding lack of baselines and especially the writing.\n\n### Writing\n\n1. As far as I understand, the entire paper boils down to decoupling the hyperparameter and evaluation phase. This is done by selecting disjoint data subsets for each phase and running them multiple times to produce a robust estimate of both the optimal hyperparameters and the evaluation performance. Even though this is simple,easy to understand and makes sense, the reader needs to go through the algorithms 1-3 to understand it. Specifically, the introduction obfuscates the actual contribution (only briefly mentioned next to Figure 2).  \n2. The intro, related work (last paragraph), 3.1 and 3.2 seem to be repeating the same arguments. It would be a better use of real estate to clarify the proposed methodology, rather than repeating the shortcomings of previous studies on a high-level. For instance, the shuffled class orderings are first mentioned in page 5 (left of line 6 in Algorithm 2); this is an important aspect that can be mentioned in the intro to clarify why the proposed protocol produces more realistic results.  \n3. Lack of discussion on data availability. Performance is obtained on 100-class subsets of ImageNet1k; this might not be possible for larger benchmarks (more classes). This is a minor issue but should be mentioned.  \n4. Properly introducing terms or methodology:   \n   1. “Original hyperparameters”: first seen in first paragraph of Section 4\\. Does this refer to the hyperparameters of the original paper on the same benchmark and the same model? It is not defined  \n   2. Harmonic mean to select hyperparameters: the paper does not mention why the harmonic mean is used. The paper also departs from previous works that use forgetting as a metric.  \n   3. Similarity cases: how is similarity calculated? Task similarity is an active area of research, e.g., in multi-task learning [1,2] of transfer learning [3]. Therefore, the methodology to assess similarity is not trivial and should be discussed.  \n5. **Computational cost**: An issue for previous approaches/surveys is the additional computational cost for the HT tuning, e.g. the first sentence of the second page mentions the additional hyperparameters for Delange et al. 2021 and Liu et al. 2023\\. However, the methodology here requires $R\\cdot S=30\\cdot 5=150$ times to retrain. This is mentioned as a limitation but there should be a study/comment on how these numbers affect results. If new papers choose to employ the proposed protocol, they should be able to opt for lower computational cost.\n6. Clarifying the contribution (connected with point 2): The paper cites several other survey works, mentioning their shortcomings on a high-level. It would be beneficial to be more specific on what are the issues of previous works that this paper addresses. For instance, pg 3 “Other studies have proposed evaluation protocols similar to ours (Chaudhry et al., 2018b; Chen et al., 2023; Bornschein et al., 2023). However, these protocols have only been applied to a limited number of older algorithms in specific domains” \\-\\> it is not clear if the paper only modernizes these efforts to include newer algorithms.\n\n### Experiments\n\n1. BEEF baseline: the authors mention that “BEEF constantly returns NaN in training loss at specific seeds so we could not report its performance.” This is either an issue with the BEEF algorithm or an implementation mistake. In either case, the baseline should not be included in this state.  \n2. No 2024 baseline for non-pre-trained models: The authors say “this graph indicates that each algorithm reflects the performance trends observed in their respective papers, gradually improving over time in accordance with the order of publication.” but to not include more recent baselines.  \n3. Limited baselines: the baseline selection seems to favor expansion-based methods (in terms of recency \\- see point 2). However, more replay-based methods can be included, e.g., [4] has many citations. In general, the work from De Lange [5] also proposes an evaluation protocol but includes many more baselines.\n\n[1] Standley, Trevor, et al. \"Which tasks should be learned together in multi-task learning?.\" International conference on machine learning. PMLR, 2020\\.\n\n[2] Fifty, Chris, et al. \"Efficiently identifying task groupings for multi-task learning.\" Advances in Neural Information Processing Systems 34 (2021): 27503-27516.\n\n[3] Zamir, Amir R., et al. \"Taskonomy: Disentangling task transfer learning.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018\\.\n\n[4] Buzzega, Pietro, et al. \"Dark experience for general continual learning: a strong, simple baseline.\" Advances in neural information processing systems 33 (2020): 15920-15930.\n\n[5] De Lange, Matthias, et al. \"A continual learning survey: Defying forgetting in classification tasks.\" IEEE transactions on pattern analysis and machine intelligence 44.7 (2021): 3366-3385.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a novel hyperparameter-tuning and evaluation protocol for Continual Learning, and specifically for class-incremental (C-IL) settings. The authors argue that previous works and surveys employ an unrealistic evaluation protocol where hyperparameters are selected in a given scenario and evaluation occurs in the same scenario. This methodology breaks the standard ML principles since it does not evaluate performance on unseen scenarios and actually leads to overestimating the capability of algorithms.\n\nTherefore, the paper splits the hyperparameter tuning (HPT) and evaluation phase explicitly and uses different tasks (classes for C-IL) for each of these phases. The evaluation of the methods becomes more robust since they consider many different instances of a setting for each of these two phases (e.g., the order of tasks changes between each experiment of the HPT phase). The paper includes experiments with and without pre-trained networks, with appropriate baselines in each case.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The requested changes are:\n\n1. Fixing the writing issues outlined above. These would help clarify the contribution and better place the paper in terms of the related work  \n2. Extend the baselines and remove BEEF. Clarify the experimental settings  \n3. Figure 3: the notation for both CL scenarios (HT and E phase) are the same, confusing the reader. It would also be nice (if it does not clutter the figure) to include the number of task orderings $S$ because this is an important part of the protocol  \n4. “Additionally, note that the high-level concept of the proposed…for both phases”: The generality of the protocol is a strength of the paper and imo should be mentioned in the introduction  \n5. Related work: mention in the first paragraph what are class and task incremental settings to help the reader.  \n6. Typos/Phrasing:  \n   1. Overcome catastrophic forgetting \\-\\> mitigate catastrophic forgetting  \n   2. In this regards \\-\\> in this regard  \n   3. Table 2: missing ImageNet-A-1?", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QUD1UWKNXh", "reviewer": "Reviewer_9NTA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- The idea of GTEP is significant and interesting, especially for the purpose of tuning hyper-parameters in CL. For instance, one can use available datasets to construct CL benchmarks for the hyper-parameter tuning phase and then use the best hyper-parameters to actually run a CL algorithm.\n- The contributions of this paper are useful for the CL community.\n- The paper is clear and well-written.\n- The experiments are extensive, with all the details given in the appendix.\n\n**Weaknesses:**\n- Although GTEP is indeed beneficial for tuning hyper-parameters of CL algorithms, the conventional pipeline for evaluating CL algorithms can still be defended if hyper-parameter tuning is not required (e.g., when the default hyper-parameter setting can already achieve a good performance). Maybe this should be noted somewhere in the paper.\n- For better comparisons, the paper should also report the performance of the default hyper-parameter setting on the evaluation datasets. This could be added as an additional column in the tables in appendix B.1 or shown as a figure in the main text. Currently, the presented results can help compare the CL algorithms across the two phases, but it may also be useful to show that hyper-parameter tuning benefits CL performance compared to the usual default hyper-parameters, even when tuning on a different benchmark.\n- There are some minor typos and issues (see below).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new protocol for evaluating continual learning (CL) algorithms, called the Generalizable Two-phase Evaluation Protocol (GTEP), which specifically focuses on CL performance using the hyper-parameters tuned on another dataset. Through the new protocol, the paper empirically shows that the performance of several SoTA CL methods has been overestimated, i.e., they \"overfit\" to the scenarios seen during the hyper-parameter tuning phase but suffer from decreased performance on an unseen dataset during the evaluation phase. The paper also shows that some CL algorithms are highly sensitive to hyper-parameters, resulting in unstable or even undefined accuracy in some cases.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Critical:**\n- Please add the performance of the default hyper-parameter settings on the evaluation datasets and some discussions for these additional results.\n\n**Minor:**\n- In Appendix B.1, please explain what the two numbers in each cell mean. I suppose they are Acc and AvgAcc, but please add an explanation for clarity.\n- In the introduction, when \"hyper-parameter\" is first mentioned, maybe the authors could give some examples (e.g., learning rate, regularization strengths, etc.) for better clarity.\n- In the title, the colon (\":\") should be on the first line.\n- On page 2, second paragraph from the bottom: it should be \"... which is almost identical **to** the conventional evaluation protocol.\"\n- In the caption of Figure 3: the last symbol should be $D^E$.\n- On page 5, the 5th line of the first paragraph, there should be no white space at the beginning of the line.\n- On line 5 of Algorithm 2, I believe $h_0$ should not be there?\n- On page 9, last line, it should be \"... sets **and** the best hyperparameters ...\"\n- On page 11, the \"Experiments across diverse similarity cases\" paragraph, \"order\" should be \"older\".", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mJwANjTIrG", "reviewer": "Reviewer_PZXg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strenghts\n- This paper addresses a problem that I similarly consider critical in the community. Indeed a new and fairer evaluation protocol such as the one proposed is needed. This is for me the main strength of this paper.\n- The paper is very well written\n- The experiments are thorough and highlight the problem at hand\n- The perspectives of the work are reasonable\n\n# Weaknesses\n- Some additional discussion on the impact of the dissimilarity of $D^{HT}$ and $D^E$ could be included\n- Since this protocol uses a fixed number $R=30$ of sets of HPs, the more the number of HPs, the less likely it is that the method will reach optimal performances. I think it makes sense because a larger number of HP should be a problem in Continual Learning, however, it would be interesting to discuss how the number of HP impacts the robustness of the method to this evaluation protocol. Similarly, we could imagine ranking the HPs by importance to optimize better methods with a large number of HPs.\n\nOverall I think it is an excellent paper and the weaknesses I have pointed out remain minor.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the widely understudied problem of hyper-parameter (HP) estimation in the context of Continual Learning. Its contributions are:\n- Formulating the problem of incorrect HP estimation in Continual Learning\n- Proposing a novel evaluation protocol adapted to the described problem\n- Re-evaluating various state-of-the-art approaches using this newly introduced evaluation protocol\n- highlighting current limitations in evaluation proposed methods in the Continual Learning community", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Despite the thorough related work section, I believe recent studies have leveraged partially similar protocols in Online Continual Learning, on algorithms that are for the most part quite recent, contrary to the claim made in the Related Work section of the paper. Namely, [1, 2] describe such strategies in their paper.  Of course, as this is not the main focus, such work does not undermine the contribution of the current paper, but I believe at least one of these references should be included in the discussion.\n- This point is not a request but a suggestion. An interesting additional information would be the performance obtained on $D^E$ when applying a hyper-parameter search on it. This would act as an upper-bound performance that can be expected, and give some insight on why the performances are increasing overall going from ImageNet-100-1 to ImageNet-100-2 in Figure 4. Maybe ImageNet-100-2 is a relatively easier dataset?\n - typo: p 11: \"where an ~order~ *older* algorithm outperforms a newer one\"\n- In my opinion, Figure 5 would be clearer if the row showed the experimental results for the same $D^E$ but different $D^{HT}$. I am more interested in comparing the performances on the same dataset $D^E$ but with differently found HP.  I understand that such information is present in each \"column\" but I believe having this information in each line would make it more easily readable.\n\n[1] Michel, Nicolas, et al. \"Rethinking Momentum Knowledge Distillation in Online Continual Learning.\" Forty-first International Conference on Machine Learning. 2024.\n\n[2] Wang, Maorong, et al. \"Improving Plasticity in Online Continual Learning via Collaborative Learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "broader_impact_concerns": "I did not identify any broader impact concern related to this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes the *Generalizable Two-phase Evaluation Protocol (GTEP)* for continual learning, which decouples hyperparameter tuning and evaluation phases to better reflect real-world constraints. The authors demonstrate, via extensive experiments, that conventional protocols overestimate performance and fail to generalize. The paper was reviewed by three reviewers. All reviewers appreciated the motivation and empirical rigor of the study. While some concerns were raised about baseline coverage and writing, these were satisfactorily addressed in the rebuttal. Given that the claims are well-supported and the work is relevant to the TMLR audience, I recommend acceptance with minor revisions. Congratulations!

Please ensure the camera-ready version:
- Refines Figure 3 and Figure 5 for clarity (notation and layout).
- Slightly streamlines the introduction to highlight the main contribution earlier.
- Includes the additional references suggested for related work.
- Addresses remaining typos and phrasing inconsistencies.
Decision: Accept (poster)