CLDyB: Towards Dynamic Benchmarking for Continual Learning with Pre-trained Models
Authors: Shengzhuang Chen
Yikai Liao
Xiaoxiao Sun
Kede Ma
Ying Wei
Venue: ICLR 2025 Poster (2025)
Keywords: continual learning
dynamic benchmarking
Primary area: datasets and benchmarks

Abstract: The emergence of the foundation model era has sparked immense research interest in utilizing pre-trained representations for continual learning~(CL), yielding a series of strong CL methods with outstanding performance on standard evaluation benchmarks. Nonetheless, there are growing concerns regarding potential data contamination within the massive pre-training datasets. Furthermore, the static nature of standard evaluation benchmarks tends to oversimplify the complexities encountered in real-world CL scenarios, putting CL methods at risk of overfitting to these benchmarks while still lacking robustness needed for more demanding real-world applications. To solve these problems, this paper proposes a general framework to evaluate methods for Continual Learning on Dynamic Benchmarks (CLDyB). CLDyB continuously identifies inherently challenging tasks for the specified CL methods and evolving backbones, and dynamically determines the sequential order of tasks at each time step in CL using a tree-search algorithm, guided by an overarching goal to generate highly challenging task sequences for evaluation. To highlight the significance of dynamic evaluation on the CLDyB, we first simultaneously evaluate multiple state-of-the-art CL methods under CLDyB, resulting in a set of commonly challenging task sequences where existing CL methods tend to underperform. We intend to publicly release these task sequences for the CL community to facilitate the training and evaluation of more robust CL algorithms. Additionally, we perform individual evaluations of the CL methods under CLDyB, yielding informative evaluation results that reveal the specific strengths and weaknesses of each method.

Published as a conference paper at ICLR 2025
CLDYB: TOWARDS DYNAMIC BENCHMARKING FOR CONTINUAL LEARNING WITH PRE-TRAINED MODELS
Shengzhuang Chen', Yikai Liao², Xiaoxiao Sun Kede Ma Ying Wei
CCity University of Hong Kong
Nanyyang Technological University *Zhejiang University
3 "Australian National University
shengzhuang. chenêmy. cityu.edu.hk. edu.hk kede malcityu. edu. hk ying. wwwe2 euu...uuuuuuuo.o. edu.cn cn
ABSTRACT
The advent of the foundation model era has sparked significant research interest leveraging pre-trained representations for continual learning (CL), yielding se- ries of top-performing CL methods on standard evaluation benchmarks. Nonethe- less, there growing concerns regarding potential data contamination during the pre-training stage. Furthermore, standard evaluation benchmarks, which are typically static, fail to capture the complexities of real-world CL scenarios, result- ing in saturated performance. To address these issues, we describe CL. on dynamic benchmarks (CLDyB), general computational framework based Markov deci- sion processes for evaluating CL methods reliably. CLDyB dynamically identifies inherently difficult and algorithm-dependenn tasks for the given CL methods, and determines challenging task orders using Monte Carlo tree search. Leveraging CLDyB, we first conduct joint evaluation of multiple ods, leading to a set of commonly challenging and generalizable task sequences where existing CL. methods tend to perform poorly. We then conduct separate evaluations of individual CL methods using CLDyB, discovering their respective strengths and weaknesses. The source code and generated task sequences are pub- licly accessible at https: //ththub.. thub. com/ a CCDDB.
of-the---r--rtt CL meth-
INTRODUCTION
The field of machine learning is undergoing a paradigm shift driven by the emergence of foundation moadels--argg-scaall neural networks pre-trained massive datasets-the demonstrate remarkable adaptability across a wide range of downstream tasks. Within this transformative landscape, contin- ual learning (CL), a computational methodology for incrementally updating models while retaining prior knowledge (Ring, 1997). faces both unprecedented opportunities and critical challenges. Op- portunities arise from the synergistic relationship between CL. and foundation models. On one hand, foundation models stand to gain from CL techniques by continually adapting to evolving data dis- tributions without catastrophic forgetting. On the other hand, CL methods can leverage the general- purpose and robust representations embedded within foundation models powerful initializations, potentially accelerating convergence and improving performance on sequential tasks. Challenges, however, primarily stem from the inadequacy of conventional CL benchmarks (Rebuffi et al., 2016; Hendrycks et al., 2020) in providing reliable evaluations of CL methods with foundation models due to two critical limitations.
The first is data contamination. The exponential growth in the volume of pre-training data for foundational models elevates the risk of prior exposure downstream CL tasks. This has pre- cipitated performance saturation in current benchmarks, diminishing their utility of comparing CL methods through marginal performance gains. More fundamentally, this overlap in data distributions calls into question whether recent progress in CL. stems from genuine algorithmic developments or merely reflects the exploitation more advanced foundation models (Janson al., 2022: Galashov et al., 2023). The second is performance saturation. Current benchmarks are inherently static,
"Corresponding authors.

---

Published as a conference paper at ICLR 2025
which assume that tasks are randomly sampled from fixed datasets and presented in unstructured, sequential order. This is a clear oversimplification of real-world CL scenarios, which involve dy- namic and evolving task sequences that are significantly more varied and challenging. Consequently, CL methods may overfit these simplified benchmarks while failing to generalize in more complex and practically demanding environments.
In light of the pressing demand for reliable evaluation protocols, we introduce CL on dynamic benchmarks (CLDyB), a computational benchmarking framework designed to accelerate algorith- mic progress in CL with foundation models. Specifically, we formulate dynamic benchmarking for CL as an infinite-state Markov decision process (MDP, Puterman, 1994), where the sequential task selection and model adaptation are optimized over potentially unbounded time horizon. The state at time step encapsulates the consolidated model parameters after incremental training on all observed tasks up to 1. This parameter vector serves as a compact representation of the model's current knowledge retention (i.e,, stability) and its adaptability to new tasks (i.e., plasticity). The action corresponds selecting task from combinatorially large pool of candidate tasks, followed by stochastic optimization of model parameters on the selection task. The reward defined as the immediate, quantitative measure of how challenging the selected task is given the current state.
Due to the infinite state space, the vast action space, and the high computational cost of reward evaluation, finding an exact solution to this MDP (for instance, via dynamic programming) is com- putationally intractable. To address this, the proposed CLDyB first reduces the action space using greedy task sampling and clustering. This yields much smaller feasible set of diverse and difficult tasks that are more likely to challenge the original foundation model (as a way of mitigating data contamination) and the continually updated models at each time step. Second, it employs Monte Carlo tree search (MCTS, Coulom, 2006) with truncated lookahead to identify the (sub)optimal task each time step.
We instantiate CLDyB for caassincrremenal learning in visual recognition (Zhou et al., 2024b) one of the most prominent challenges in CL research. We first apply CLDyB jointly to a group of representative CL. methods, generating a standardized set of challenging and generalizable task sequences that yield reliable and indicative evaluation results. We then perform multi multi-diimnnionnl assessment of individual CL methods using CLDyB, assessing critical dimensions such as classifi- cation accuracy, robustness to task sequence variations, and memory efficiency. This allows us to inspect distinct performance characteristics and vulnerabilities of different CL methods.
In summary, our principal contributions include
CLDyB: A general computational benchmarking framework designed to inform reliable progress in CL, especially with pre-trained models;
Standardized CL benchmark: CLDyB establishes a set of challenging and generalizable task sequences for both current and future CL methods;
multi-dimensionl assessment: CLDyB enables comprehensive assessment of individual CL methods across multiple dimensions.
2 PRELIMINARIES
Class-incremental learning aims to build a generic classifier capable of recognizing an expanding set of classes by incrementally integrating new knowledge while maintaining performance on pre- viously learned tasks (Zhou et al., 2024b). More formally, a CL algorithm CL(-) incrementally trains parameterized classifier f on sequence of N classification tasks, collectively denoted as ()) [Til TeN+1 == (Ti, T2...., ]. Each task Tr contains [Ta] (image, label) pairs {a 1// 1j=1- di- vided into training. validation, and test subsets. Note that all tasks have disjoint sets of class labels. At time step f, the classifier is updated according to ft CL(ft-1, Te), where fo denotes the pre- trained foundation model. A fundamental constraint in CL is that data from past future tasks remain inaccessible while training on the current task. This presents central challenge: Training to incrementally recognize
classes (i.e., high plasticíty), while mitigating the catastrophic forgetting of previously learned classes (i.e., high stability). To quantify the plasticity and forgetting of ft, employ two standard metrics: Average learning accuracy (ALA, Riemer et al. 2018) and
2

---

Published as a conference paper at ICLR 2025
the average forgetting measure (AFM, Chaudhry et al., 2018):
ALA Tect1)
Ace (fx, Tk),
(1)
k=
AFM (ft, T<t+1)
Acc (sk. Tka) Acc (fe, Ts)
k=1
Tc)(J:T) represents the empirical classification accuracy of classifier f the test split of task T. Assuming there are CL methods, each associated with its respective parameterized classifier at time step t, denoted [ and Fi := 11(% Ji we compute the ALA and AFM scores by J
further averaging over these M models, i.e., ALA(F T<(+1) = ΣM ALA(S() (") .T<+1) and AFM Tcc+11) M ΣM AFM(S T<(41).
PROPOSED FRAMEWORK: CLDYB
In this section, we first give the general problem formulation of dynamic benchmarking for CL, and then introduce the proposed CLDyB as specific instance.
3.1 PROBLEM FORMULATION
Our core idea sequentially construct algorithn-dependent task sequences that consistently chal- lenge current CL methods. Mathematically, this sequential task construction can be formulated as an infinite-star MDP (S,A, A, R), with a potentially unbounded time horizon (i.e., the task length is unknown or N + oc). At time step t, the observable state comprises the set of M CL mod- els Ft-1 E S, each having been sequentially trained on a common sequence of selected tasks up to time step L. We assume policy μr S + A that maps the current state to an action, which involves selecting specific task T, from feasible set of tasks (i.e., the action space Ad). After that, probabilistic state transition is induced from F to Fe through the stochastic optimiza- Tr), for tion of F,-1 using their respective CL methods on the selected task: f() = CL (1(1. i {1, (1,2..... .,, M). This transition incurs an immediate reward:
R (Fr Te(+1)= = AFM (Fi, T<(41) ALA (F, (F.TT))))
(3)
which encourages forgetting while inhibiting plasticity, thereby facilitating the identification of chal- lenging tasks to mitigate data contamination (Yeom et al., 2017). Consequently, our objective trans- lates finding the optimal policies x² ={p] (F that maximize the cumulative rewards:
arg max lim
Σ a 1-1 ((
J (F))
where a E (0, is discount factor, Fo (5 MM and (Fu) the initial state value function.
Solving Problem (4) to obtain an exact solution presents two primary computational challenges. First, the solver shall efficiently navigate the vast and discrete set of feasible tasks (i.e., Ad) at each time step, which grows combinatorially with problem complexity (e.g., the total number of classes). Second, the solver shall accurately and efficiently evaluate the value function (i.e., Jr (Fo)) to facilitate challenging task identification. Nevertheless, the complexity of this evaluation can vary significantly based on the chosen continual learning (CL) method and task properties, such as the number of classes per task. Moreover, the computation may become intractable due to the infinite state space and potentially unbounded time horizon. the subsequent subsections, we describe our proposed solution, CLDyB, which effectively addresses these challenges through two key strategies: 1) Greedy task sampling and clustering to construct much smaller candidate task set, and 2) MCTS to identify the (sub)optimal task. Fig. illustrates the system diagram of CLDyB.
3

---

Published as a conference paper at ICLR 2025
Candidate task set. construction
Greedy task sampling
Functional task clustering
Sample the
Candidate bask 2.7
k- t-manns
Compute pairwise class potentials Y
Repeat to sample multiple tasks
clustering
Sample sutsoquant
clanses groadilyy w Egp. 15)
Pairwise class potential Cp,
Reduced action space Ae
Clustered action space A.
ptimal task identification
0.4
Backpropagation: Sum
Rollout simulation:
Expansion: Perform
Selection: Select (1() the task J. at time step
over immediate and L-
Train Fj iteratively on actions from At to obtain+
step lookahead rewards
this rollout task sequence
potential next state
Potential states and actions
Selected states actions
Figure 1: System diagram of the proposed CLDyB for dynamically constructing task sequences that challenge current CL methods. At time step i, CLD first performs candidate task set construc- tion by greedy task sampling (see Eqn. (5)) and functional task clustering. This results in reduced and clustered action space Ar. facilitating task evaluation. It then performs optimal task identifi- cation by maximizing the estimated current state value function (see Eqn. (7)) using MCTS, which consists of four steps: 1) Expansion, 2) rollout simulation, 3) backpropagation, and 4) selection. The pseudocode for CLDyB can be found Algorithm of the Appendix.
3.2 CANDIDATE TASK SET CONSTRUCTION
Given a data pool D, of C2 classes for caasseincremettll learning at time step t, we define a task Te subset of images containing K classes, where K < C.. Consequently, the number of possible tasks in A, is given by the binomial coefficient (x:) which grows rapidly as C₁ increases. This makes an exhaustive search over A, infeasible, as each task evaluation often necessitates a full cycle of continual model training, significantly increasing the overall computational cost.
Greedy task sampling. To construct more compact candidate task set Å,, propose greedy task sampling strategy that leverages the intuition that task difficulty in multi-class classification is inversely correlated to the separability of class pairs in feature space (He al., 2020). Specifically, define the sampling probability over task T, to be proportional to the product of its pairwise class potential functions:
M (Cu, Co), where (CC,, Co) = M Σ
P(T) x
II [I..
(CosSim uu "e,"
(5)
Here, Cn and C, represent the subsets of images belonging to classes and v. respectively. The prototypes un and of are computed by the mean feature representations of these subsets using ((4)) CosSim(:,.) computes the cosine similarity between two feature representations, and s(.) is a linear rescaling function that maps the similarity scores to the range [0, 1]. where the maxi- mum value is set to 1 and the minimum to 0. This additional rescaling ensures that the computed pairwise potentials are well-suited for probability computation (i.e., preventing negative values). By assigning higher sampling probabilities tasks with larger products of pairwise potentials, prior- itize challenging tasks, for which the classifiers F, struggle to discriminate between the included classes in their respective feature spaces. This will force F.-1 focus refining their feature
4

---

Published as a conference paper at ICLR 2025
representations for more difficult cases, thereby improving their capacity to handle more complex class relationships.
Drawing inspiration from (Kohli et al., 2013), we introduce greedy sampler that sequentially lects K classes to form task Tr. Initially, to ensure task diversity, the first class is uniformly sampled from all feasible classes in the data pool D.. Subsequently, during the k-th iteration, the class n is chosen to maximize the product of pairwise potentials with respect to the previously selected k - 1 classes in V
u = arg max
II VEVA
(CC,,Co) for ( K}.
(6)
and we incorporate ui into Vē_1 1 to form V. This greedy sampling iterates until K distinct classes are identified. The entire procedure is further repeated multiple times to generate the candidate task set As. detailed description of our sampler is provided Algorithm 1.
Functional task clustering. The sampled tasks in A are likely exhibit skewed distribution with respect to their enabled functional skills of classifiers. This is because the M classifiers naturally form distinct clusters based on their functional skills. Consequently, uniform task sampling from A disproportionately encourages selecting tasks from the cluster characterized by the dominant functional skills, thereby limiting exposure to tasks that would challenge classifiers in diverse ways. To enhance task diversity functional space and promote more efficient task evaluation, we propose additionally clustering tasks in Ar based on their induced functional skills, followed by ancestral sampling to populate a further condensed candidate task set A
Specifically, for each task, we compute an M- -ddimensional feature vector by evaluating the aver- age negative log-likelihood of predictions made by M k-nearest neighbors (k-NN) classifiers with feature representations from Ft-1- We partition tasks into functionally coherent clusters using the k-means algorithm, operating on the derived task representations. The final condensed candidate task set A, is populated through ancestral sampling: First uniformly selecting a cluster and then ran- domly sampling task from the chosen cluster. A detailed description is provided in Algorithm 2.
3.3 OPTIMAL TASK IDENTIFICATION
Despite substantial action space reduction via greedy task sampling and clustering, optimizing the initial state value function defined Eqn. (4) remains computationally challenging. This difficulty arises from the combinatorial complexity of task ordering and the long-term interdependencies be- tween task selection and model adaptation. To address these, we employ MCTS (Coulom, 2006), a widely adopted approach for online sequential decision-making (Puterman, 1994), particularly well- suited to our CL setup. Specifically, MCTS approximates the current state value (F,) within a search tree, comprising four steps. 1) Rollout simulation: Recursive tree growing via MC simula- tions until a predefined computational budget is exhausted; 2) Backpropagation: Update values and visit counts for all nodes in the visited path; 3) Selection: Choose the task Te that leads the state with the maximum estimated immediate value; 4) Expansion: Train all classifiers continually on the selected task T; and transit from the current state F to the next state F,
Unfortunately, in open-ended CL environments, MC simulations are resource-intensive due to 1) the necessity of (N ()-p rollouts to evaluate the value function J+ (J.), involving continual model training across tasks at all these steps, and 2) the indefinite task length N. We propose approximating the value function using a truncated horizon approach, leading to
(CL(((Fr
Ti)) so max R(F,-1, T.EAA
T.) + Σ ((CL(FF-148: TT4k) Te+x+1); k=
where Pt Te π represents the optimized policy time step t. The terms Te- for k E (I,...,,) L} are sampled uniformly from their respective feasible action spaces, and are ex- cluded from the set of optimization variables (i.e., they are not included the arg max operation). The hyperparameter L controls the trade-off between computational tractability and approximation fidelity. For simplicity, we set L 1 and defer the adoption of advanced value function approxima- tion methods, such as value network learning (Silver et al., 2016), to future research.
5

---

Published as a conference paper at ICLR 2025
4 RELATED WORK
Class-incremental learning. Recent advances in CL with foundation models reveal two key ad- vantages: inherent resistance to catastrophic forgetting and strong generalization to downstream tasks (Ostapenko et al., 2022; Zhang al., 2023), rendering it rapidly evolving research fron- tier. A prominent technical direction is to integrate CL methods with parameter- efficientt fine-tuning strategies, yielding representative methods like orthogonal projection (Liang & Li, 2024; Qiao al., 2024), model expansion (Wang et al., 2022; Smith et al., 2022; Wang et al., 2023a), and model en- sembling (Gao et al., 2023; Zhou et al., 2024a). Alternatively, ppresennatiion-based methods focus on maintaining stable feature representations within foundation models. This is often achieved by freezing the backbone after the initial task learning (Zhou al., 2023a) or employing low learning rates (Zhang et al., 2023). non-parametric classifiers are then progressively constructed, which can be enhanced by random projections (McDonnell al., 2023) or the inclusion of intermediate repre- sentations (Ahrens et al., 2023). A more detailed discussion of conventional CL methods is deferred to Appendix B.I.
Dynamic benchmarking. The rapid advancement of foundation models, particularly large language models (LLMs), has necessitated a re-evaluation of traditional static benchmarks, often inadequate in providing reliable assessment due to issues like data contamination (Shi et al., 2023; Zhou et al., 2023b) and oversimplification of real-world testing scenarios (Kiela et al., 2021; McIntosh et al., 2024). Dynabench (Kiela et al., 2021) and DynaBoard (Ma et al., 2021) propose continuously evolve test sets with crowd- human labels to enable more accurate LLM evaluations. Recog- nizing the substantial costs involved in human labeling, recent efforts have been made automate the process of dynamic test set generation, leveraging directed acyclic graphs (Zhu et al., 2023) and multiple agents (Wang et al., 2024). Additionally. dynamic programming has been used for test set selection, enhancing the scalability of evaluating a large number of models at ten thousand scales (Prabhu et al., 2024). The proposed CLDyB represents one of the initial endeavors to establish dynamic benchmarking protocols for CL methods, particularly those with foundation models.
EXPERIMENTS
In this section, we first present the experimental setups, and then evaluate nine state-of-the-ard CL. methods both jointly and separately using CLDYB, followed by insightful analysis.
5.1 EXPERIMENTAL SETUPS
Data pool. To build the playground for CLDyB, we assemble a total of 26 datasets, consisting of 2, 505, 185 images across 2, 403 categories. These datasets are categorized into two main groups: photographic image datasets for the main experiments and Al-generated image datasets for addi- tional analysis. The included classes cover broad spectrum of domains, levels of granularity, cultural contexts, and time periods. Details are provided in Appendix A.
CL methods. We select total of nine CL methods for evaluation based Competitive performance that prioritizes recently published methods, achieving state-of-the the-art re- sults on static CL benchmarks, and 2) sufficient representativeness that ensures the selected methods encompass a wide range of design philosophies.
main criteria: 1)
Implementation details. All experiments adhere to the standard blass-incremente learning pro- tocols, with results evaluated using three widely adopted metrics: 1) Average accuracy (Acc), 2) average retention' (AR, Chaudhry al. 2018), and 3) ALA (Riemer et al., 2018). Higher values in- dicate better CL performance but also suggest that the benchmark presents a lower level of difficulty for CL. methods being evaluated. During training, hyper-parameters for each method are selected using the validation sets of the first three tasks, following Chaudhry et al. (2018). Each task in the CLDyB sequences is a 20-category classification problem (i.e., K = 20) with a sequence length HeT-Base-Sup2111 SSp221K (Dosovitskiy al., 2021) as the foundation model. All main experiments are repeated across three random runs, and mean results are reported. To ensure fair comparison of task selection strategies, the first task is randomly chosen
of ten (i.e., N 10). All CL methods employ ViT-Base
Equiialent to the negation of AFM in Eqn. (2).

---

Published as a conference paper at ICLR 2025
Table Final average accuracy (%) of CL methods on various benchmarks, "f" indicates results di- rectly copied from the respective papers, while "x" highlights statistically significant results. AFEC and CLSER are excluded from comparison here due differences in backbone architectures.
Method
CIFAR- ImageNet-R
CLDyB (Ours)
Held-out
ER (Rolnick et al., 2018)
67.9
55.1
54.8
79.9
DualPrompt (Wang ct al., 2022)
86.5
68.1
41.9
70.8
LAE (Gao al.. 2023)
85.6
72.7
48.1
71.1
HiDe-Prompt (Wang al., 2023a)
92.6
75.1
62.5
84.9
SLCA (Zhang et al., 2023)
91.5
77.0
S6.2
80.3
RanPAC (McDonnell et al., 2023)
92.2
78.1
56.9
81.0
PGP (Qiao et al., 2024)
86.9
69.3
44.0
68.7
SRCC (per column & held-out)
0.643
0.643
0.964
N/A
KRCC (per column & held-out)
0.429
0.429
0.905
N/A
and fixed, maintaining consistent initial conditions. Additional details on the selected CL methods can be found in Appendix B.2.
5.2 MAIN RESULTS
CLDyB identifies commonly challenging task sequences. We first conduct a joint evaluation of five CL methods, i.e., AFEC (Wang al., 2021), CLSER (Arani al., 2022), HiDe-Prompt (Wang et al., 2023a), RanPAC (McDonnell et al., 2023), and PGP (Qiao et al., 2024) using CLDyB, while reserving the remaining four CL methods, i.e., ER (Rolnick al., 2018), DualPrompt (Wang et al., 2022), LAE (Gao et al., 2023), and S1.CA (Zhang et al., 2023), for independent testing. The results, presented in Figs. 2 and 8, demonstrate that the CLDyB sequences pose significant challenges not only for the five CL methods subjected to CLDyB but also for the four CL methods that were not exposed to it. Notably, compared to random task sequences drawn from the data pool, performance on the CLDyB sequences results in 26% reduction in Final Acc and a 9% reduction in Final AR across the evaluated methods, as shown in Fig 6. These cross-validation results underscore the generalizability of the CLDyB sequences, manifesting themselves strong CL benchmark.
AFBC
CLSER
DuslPrompi
LAC
IlDee-rrmpl
SLCA
RamPAC
PGP
0.00
0.90
Age
0.80
0.65
2
Time
Figure 2: Joint evaluation of the five CL. methods (represented by solid lines) using CLDyB with generalization to the other four CL methods (represented by dashed lines).
We present some qualitative visualizations of the CLDyB sequences in Figs. 14 and 18, illustrating that the chosen tasks, originated from various datasets, form sequences in which similar tasks are interspersed with dissimilar ones. We believe this pattern closely resembles real-world CL scenarios with inherent task diversity.
CLDyB sequences produce reliable evaluation results. We compare the performance of CL methods across two static CL benchmarks (Krizhevsky, 2009; Hendrycks et al., 2020), CLDyB sequences, and held-out set-the latter comprising randomly ordered tasks derived from datasets that are excluded from prior benchmarks. Spearman's rank correlation coefficient (SRCC) and Kendall's rank correlation coefficient (KRCC) used assess aligmment between each bench- mark and the held-out set. From the results in Table we find that the CLDyB sequences achieve the highest correlations, offering compared to static benchmarks. Such strong alignments also highlight that CLDyB holds great promise to bridge algorithmic advancements in CL research real-world applications.
accurate performance reflection in unseen CL scenarios
CLDyB enables multi-dimensional assessment of individual CL methods. By applying CLDyB separately to each CL method, seek to measure its worst-case performance, formalized

---

Published as a conference paper at ICLR 2025
in Eqn. (4). Accordingly, we define the Robustness of CL method the average Final Ace across multiple CLDyB sequences paired with that method. Additionally, we record the Memory Efficiency? in MB, alongside the three standard metries introduced in Sec. 5.1.
From the results in Fig. 3, we have several interest- ing observations. First, methods incorporating replay mechanisms, such as CLSER (Arani et al., 2022), HiDe- Prompt (Wang al., 2023a), SLCA (Zhang et al., 2023), and RanPAC (McDonnell et al., 2023), gener- ally exhibit greater resistance to forgetting, resulting in higher Final Acc. On the downside. these methods tend to be less memory efficient. Second, exemplar- replay methods, specifically ER (Rolnick et al., 2018) and CLSER, demonstrate significantly better model plas- ticity, whereas parameter-efficient fine-tuning methods like PiDee-Proptt and LAE (Gao et al., 2023) show limited Final ALA, indicating weaker forward transfer capabilities an aspect often neglected in current CL re- search. Third, more advanced methods typically outper- form their predecessors in terms of performance, but they PGP versus DualPrompt and CLSER versus ER). Last, we highlight that while the simple ER does not achieve satisfactory Final Ace and AR, it remains competitive in terms of Final ALA, Robustness, and Memory Efficiency, making it an overall top-ranked CL method.
Final Acc
ItDe-thampi SLCA
Memory Efficienccy
Rank
Dailhhrrge
AFEC
ALA
Robustness
Figure 3: Multi-dimensional assess- ment of CL methods using CLDyB, with higher values on the axes rep- resenting better performance. The comparison highlights their respective strengths and weaknesses, especially how they handle the trade-offs across different dimensions.
are not always more robust (e.g., PGP
5.3 FURTHER ANALYSIS
CLDyB identifies individually challenging task sequences. In Fig. 4, we visualize the CLDyB sequences for each individual CL method by examining: (a) The Acc trajectories on the commonly challenging CLDyB sequences, and (b) the feature similarity between the selected two tasks, ie., the average cosine similarity between all possible pairs of image features. This similarity is computed for every task pair within each CLDyB sequence, giving rise to 2D task-to-task similarity matrix. Each matrix is then flattened into a vector, forming rows of the dendrogram in Fig. 4(b). The cluster- ing patterns in the dendrograms indicate common characteristics shared by certain CL methods. For example, both ER and CLSER rely on exemplar replay, while SLCA and Prommp use classifier calibration.
A closer inspection of Fig. 4(b) reveals that the individually challenging CLDyB sequences offer crucial insights into the limitations of different CL methods. Specifically, LAE and AFEC show increased sensitivity to task sequences with significant distribution shifts, as CLDyB tends to select dissimilar tasks for both methods. We hypothesize that this vulnerability may stem from the use of moving average weight ensembles in LAE and parameter regularization in AFEC, both of which are less effective in handling large model updates required for certain CL settings. In contrast, CLSER and ER are more susceptible to sequences of similar tasks, which is consistent with prior theoretical findings, indicating a correlation between task similarity and forgetting (Lee et al., 2021).
Data pool of CLDyB is expandable over time with Al-generated image data. The data pool of CLDyB is not intended to remain static. We illustrate this concept by showing how CLDDy can offer valuable feedback for selecting new datasets, enabling the data pool to expand dynamically over time. This continuous evolution helps mitigate the performance saturation issue commonly seen in static benchmarks.
We initially employ CLDyB to evaluate CL methods over three tasks. Based on the analysis of the tree-search history, we observe that tasks related to animals consistently yield higher rewards com- pared to other tasks with different object categories. This suggests that these animal-related tasks may present particular challenges for the selected CL methods. In response, we employ genera- tive diffusion models, in particular SDXL (Podell et al., 2024), to efficiently expand the data pool,
Equivalent to the negative memory storage (detailed in Appendix B.3).
8

---

Published as a conference paper at ICLR 2025
AFBC
LAB
0.9
1AB
-08
AFBC
PGP
PGP
0.7
SLCA
Talk socummy
SLCA
DualPromps
IGDe-Prompl
CLSSER
-0.5
CLSER
0.5
HiDe-Prompt
RanPAC
RanPAC
0.3
DualPrrmpl
(a)
(b)
Figure 4: Dendrograms of CL. methods. (a) Ace trajectories on the commonly challenging CLDyB sequences. (b) Stack of flattened versions of the 2D task-to-task similarity matrices obtained on individually challenging CLDyB sequences. The task similarity values are normalized by s(.) in Eqn. (5) to the range [0, 1] for improved visualization. Both dendrograms exhibit noticeable consis- tency their hierarchical structures, reflecting commonality in CL methods.
adding generated images of novel animal categories. In Fig. 5, we compare the performance of the CL methods on upcoming tasks selected by CLDyB using both the original and the augmented data pools.
We find that the Al-generated classes are indeed frequently selected, indicating their prominence in the CLDyB sequences (see Fig. 21). This leads to noticeable drop in Final AA and AR, which verifies the added difficulty of the augmented data pool relative the original, and confirms that our data pool is readily expandable to resolve performance saturation. Additionally, the tasks de- picted Fig. 21 demonstrate variations image styles, such as shifts from photographic images to sketches and paintings, pointing out potential difficulty faced by current CL methods in adapting to stylistic changes over time. In particular, rehearsal-fie -fee methods like AFEC (Wang et al., 2021) and PGP (Qiao al., 2024) exhibit more substantial performance degradations when encountering task sequences with style transitions.
Oxiginal APEC Augmented
Osiginal CLSER Augmented
HiDe-Promst
RasPAC Ohiginal
Onggial
HDDr Procupt Augmented
-4-
RanPAC Augmented
PGP Augmented
0.01
0,10
Acc
0.50
0.90
-0.50
0.70
stop
Figure 5: Performance comparison of CL methods on upcoming tasks selected by CLDyB from the original and the augmented data pools. Additional diffusion-generatee images are added to the data pool at time step 2 (denoted by the dashed line).
5.4 ABLATION STUDIES
To isolate the contributions of individual components in CLDyB, we systematically remove each of them, resulting in four degenerates: 1) Sampling tasks randomly (as a performance lower bound), 2) replacing greedy task sampling with uniform sampling from the classes within each dataset in the data pool for candidate task set construction, 3) disabling functional task clustering, and 4) substituting MCTS for selecting the next task with method that consistently chooses tasks most similar in representation to previous tasks, as suggested in (Bell Lawrence, 2022).
In Fig. 6, is evident that the full version of CLDyB outperforms all degenerates in identifying more challenging task sequences, which leads to lower ACC and AR, thus validating the necessity each component. Furthermore, assess the effectiveness of CLDYB in handling long task sequences,

---

Published as a conference paper at ICLR 2025
(3) Random
(d) CLD finctional lask clustering
CLDyD
(b) CLDy we greedy task sampling
(d) CLDy B weo MCTS
0.00
0.90
-0.33
0.90
0.83
4005
0.70
-0.08
0.70
-0.33
0.50
stop
Figure 6: Average performance comparison of the nine CL methods on the CLDyB sequences and those generated by four CLDyB degenerates,
conduct experiments on an extended sequence consisting of 40 tasks, with results shown in Fig. 9. It is clear that, on average, CL methods consistently underperform on CLDyB sequences compared to random sequences. Finally, Fig. 10 illustrates that CLDyB remains effective in dynamically iden- tifying commonly challenging sequences to combat data contamination and performance saturation using alternative foundation models (i.e., CLIP-ViT-Base (Radford et al., 2021)).
6 CONCLUSION AND DISCUSSION
We have introduced CLDyB, a dynamic computational benchmarking framework that systematically identifies challenging and algoritmn-dependent task sequences based on MDPs. To make sequen- tial task generation computationally feasible, CLDyB makes two reasonable approximations: ac- tion space reduction via greedy task sampling and clustering, and value function approximation via MCTS. By evaluating nine state-of-the-art CL methods, we demonstrated that CLDyB sequences expose critical weaknesses in existing methods while providing reliable evaluations aligned with real-world generalization. CLDyB represents a paradigm shift in evaluating CL methods. By transi- tioning from static benchmarks to dynamic and Ilgorithm-ddeenden evaluation protocols, we pave the way for CL methods that genuinely adapt to open-world complexity. We hope that CLDyB and its generated task sequences provide foundation for developing next-generation CL methods that strike a better balance between stability and plasticity.
While CLDyB represents a significant step towards reliable and realistic CL benchmarking, several promising directions remain for future research. First, one of our primary motivations for CLDyB is the issue of data contamination. However, quantifying data contamination precisely is challenging due to inaccessible pre-training data and the need to assess its overlap with downstream tasks. While heuristic approaches (e.g., task difficulty metries) offer approximations, they may conflate contami- nation with other factors influencing difficulty, limiting their reliability. Although empirical studies show that CL performance becomes inflated when pre-training and CL tasks overlap significantly, defining threshold of contamination beyond which CL evaluations lose validity remains difficult. Addressing this could refine benchmark design and strengthen evaluation protocols for CL methods. Second, the development of self-evolving data pools through human-Al collaboration and gener- ative models presents an important avenue. Although we demonstrated preliminary success using diffusion-generaate image data by further exploiting known failure scenarios, a systematic approach is needed to automatically and dynamically expand tasks in the data pool in terms of quality, diffi- culty, and diversity. Third, the current MCTS- approximation of the value function and incurs high computational costs. Hybrid methods that in- tegrate learned value sstimators-withone the need for computationally expensive continual model dddatatioo-aalonssidee tree search could improve scalability. Fourth, although our experiments have thus far focused on image recognition, CLDyB is readily extendable to other domains, including video, text, and embodied AI. Last, establishing a community evolution of CLDyB-enaale enableed benchmarks is highly desirable. Inspired by Dynabench (Kiela et al., 2021), web-based interface could facilitate the submission of new datasets to expand the data pool and accommodate new CL models while ensuring backward compatibility for fair and consistent progress tracking.
method, while effective, offers only coarse
platform for the collaborative
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
We would like to thank Xuelin Liu for assistance with the plots and diagrams. This work was supported in part by the Hong Kong RGC General Research Fund (11220224), the CityU Applied Research Grant (9667264), and an Industry Gift Fund (9229111).
REFERENCES
Hongjoon Ahn, Donggyu Lee, Sungmin Cha, and Taesup Moon. Uncertainty -bsedd continual learn- ing with adaptive regularization. ArXiv, abs/ 1999,11611..,...1 2019.
Kyra Ahrens, Hans H. Lehmann, Jae Hee Lee, and Stefan Wermter. Read between the layers: Leveraging intra-layer representations for rehearsal-free continual learning with pre-trained mod- els. ArXiv, abs/2312.08888 1.088888. 2023.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Conference on Neural Information Processing Systems, 2019.
Zach Aluza. CNFood-241: Chinese Food Dataset, 2024. URL https: // /ttp://wwwwww///// wwwastsszachaluaeee cnwood-ooooooooooooooooooooooooo 241. Accessed: 2024-09-23.
http:://wwwwwwgwl/.e/e/e kaggle. com/
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general contin- ual learning method based on complementary learning system. In International Conference on Learning Representations, 2022.
Sourav Banerjee. Indian Food Images Dataset, 2023. URL https://www hwggs/////wwwkklll.ee.. kaggle. com/ images dataset. Accessed:
datasets /tr////mmuuvvaarrrreeeee 2024-09-23.
Sourav Banerjee.
Animal Image
Dataset:
90
Different Animals,
2024.
URL
https://www kaggle com wateeets/aamscuaabaanae jee/
animal- - animal-image-dene www.aatttttt-d-feeeeeeeee
Accessed: 2024-09-23.
Samuel J. Bell and Neil D. Lawrence. The effect of task ordering in continual learning. ArXiv, abs/2205.13322. 2022.
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101-Mining discriminative com- ponents with random forests. In European Conference on Computer Vision, pp. 146-461, 2014.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. ArXiv, abs/2004.07211, 2020.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co²L: Contrastive continual learning. In IEEE Interna- tional Conference on Computer Vision, pp. 9496-9505, 2021.
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. ArXiv, abs/1812.00420,, 2018.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Bench- mark and state of the art. Proceedings of the IEEE, 105(10):1865-1883 2017.
Rémi Coulom. Efficient selectivity and backup operators in Monte Carlo tree search. In Interna- lional Conference on Computers and Games, pp. 72-83, 2006.
Li Deng. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 2966):111-142, 2012.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Representations, 2021.

---

Published as a conference paper at ICLR 2025
Mehrdad Farajtabar, Navid Azizan, Alexander Mott, and Ang Li. Orthogonal gradient descent for continual learning. ArXiv, abs/1910.07104% 2019.
Alexandre Galashov, Jovana Mitrovic, Dhruva Tirumala, Yee Whye Teh, Timothy Nguyen, Arslan Chaudhry, and Razvan Pascanu. Continually learning representations at scale. In Conference on Lifelong Learning Agents, pp. 534-547, 2023.
Jhair Gallardo, Tyler L.. Hayes, and Christopher Kanan. Self-supervised training enhances online continual learning. ArXiv, 14010, 2021.
Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, and Jing Zhang. A unified continual learning framework with general parameter-efficien tuning. In IEEE Interna- tional Conference on Computer Vision, pp. 11449-11459 11459, 2023.
Andrei Grigorev. Clothing Dataset Full, 2024. Sataaetss/agriigoerr/ooooo
URL https://wwe kaggle com/ dataset full. Accessed: 2024-09-23.
Gunshi Gupta, Karmesh Yadav, and Liam Paull. La-MAML: Look-ahead meta learning for continual learning. Conference on Neural Information Processing Systems, pp. 11588- 11598, 2020.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9726-9735 2020.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Introducing EuroSAT: A novel dataset and deep learning benchmark for land use and land cover classification. JEEE International Geoscience and Remote Sensing Symposium, pp. 204-202 2018
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Lixuan Zhu, Samyak Parajuli, Mike Guo, Dawn Xiaodong Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of aut-of-distributin gener- alization. In IEEE International Conference on Computer Vision, pp. 1320-83329 2020.
Muhammad T. Islam. Jute Pest Dataset, 2024. URL https: www kaggle.ggg............kkk..kk.kk com/ cow/dsv/ 8332009. Accessed: 2024-09-23.
Paul Janson, Wenxuan Zhang, Rahaf Aljundi, and Mohamed Elhoseiny. A simple baseline that questions the use of pretrained-modde in continual learning. In Conference on Neural Informa- fion Processing Systems Workshop on Distribution Shifts: Connecting Methods and Applications, 2022.
Khurram Javed and Martha White. Meta-learning representations for continual learning. In Confer- ence on Neural Information Processing Systems, 2019.
Kaiska. Apparel Dataset, 2024. URL https:/ www. htt.g//....o/ttttttttteaaaaaaaes esparel-datase Accessed: 2024-09-23.
Parneet Kaur, Karan Sikka, Weijun Wang, Serge Belongie, and Ajay Divakaran. FoodX-251: A dataset for fine-grained food classification. arXiv preprint arXiv:1907.06167, 2019.
Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Talat, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. Dynabench: Rethinking benchmarking in NLP. In Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp- 4110-4124, 2021.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan. Tiago Ramalho, Agnieszka Grabska-Bann Barwinska, Demis Hass- abis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 11<(13)3521-3556 2017.
12

---

Published as a conference paper at ICLR 2025
Pushmeet Kohli, Anton Osokin, and Stefanie Jegelka. A principled deep random field model for image segmentation. In IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp- 1971-1978, 2013.
Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. Master's Thesis, Uni- versity of Toronio, 2009.
Vencer Lanz. Sea Animals Image Dataset, 2024. URL https: / www wwg.aagglllee.e e com/ httpt/////veneeaae/////eeee
image- Accessed: 2024-09-23.
Ya Le and Xuan Yang. Tiny ImageNet Dataset, 2015. URL htttttts: Q.t.s....tt..//pppprsttcccc.. com/ dataset/tine ttt.attttiitnnmmteeeeeeee -innnnee..nnnt.... Accessed: 2024-09-23.
Sebastian Lee, Sebastian Goldt, and Andrew M. Saxe. Continual learning in the teacher-student setup: Impact of task similarity. In International Conference on Machine Learning, pp. 6109- 1119,2021,
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40 (11))22935-29445)
Yan-Shuo Liang and Wu-Jun Li. InfLoRA: Interference-free low-rank adaptation for continual learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23638- 23647, 2024.
Weiduo Liao, Ying Wei, Mingchen Jiang, Qingfu Zhang, and Hisao Ishibuchi. Does continual learning meet compositionality? New benchmarks and an evaluation framework. In Conference on Neural Information Processing Systems, pp. 13519,,3513, 2023.
Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The CLEAR benchmark: Continual learn- ing on real-world imagery. In Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
Vincenzo Lomonaco and Davide Maltoni. CORe50: A new dataset and benchmark for continuous object recognition. In Conference on Robot Learning, pp. 17-26, 2017.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. In Conference on Neural Information Processing Systems, pp. 6470-6479, 2017.
Zhiyi Ma, Kawin Ethayarajh, Tristan Thrush, Somya Jain, Ledell Wu, Robin Jia, Christopher Potts, Adina Williams, and Douwe Kiela. Dynaboard: An evaluation-as-a-sevve platform for holistic next-generation benchmarking. In Conference Neural Information Processing Systems, pp. 10351-10367, 2021.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv: 2013.
Mark D. McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abhasnejad, and Anton van den Hengel. RanPAC: Random projections and pre-trained models for continual learning. In Conference on Neural Information Processing Systems, pp. 12022-12053, 2023.
Timothy R. McIntosh, Teo Susnjak, Tong Liu, Paul Watters, and Malka N. Halgamuge. Inadequa- cies of large language model benchmarks in the era of generative artificial intelligence. ArXiv, abs/2402.0088D/ 2024.
Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investiga- tion of the role of pre-training in lifelong learning. ArXiv, 2112.09153, 2023.
Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner. Variational continual learn- ing. In International Conference on Learning Representattonns 2018.
Oleksiy Ostapenko. Timothée Lesort, Pau Rodr'iguez. Md Rifat Arefin, Arthur Douillard, Irina Rish, and Laurent Charlin. Continual learning with foundation models: An empirical study of latent replay. In Conference on Lifelong Learning Agents, pp. 60-91, 2022.
13

---

Published as a conference paper at ICLR 2025
James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman. Object retrieval with large vocabularies and fast spatial matching. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.
Greg Piosenka. Balls Image Classification, 2024a.
URL https://www www.:w.w..lll....m..o com/
datasets hwtpeees//ppsseeaaaaeee -
Accessed: 2024-09-23.
Greg Piosenka. Sports Classification, 2024b. URL https://www Accessed: 2024-09-23.
gpiosenka/ htt/spportsss caaessscllssiii wwwassliaaattaca
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. In International Conference on Learning Representations, 2024.
Ameya Prabhu, Vishaal Udandarao, Philip H. S. Torr, Matthias Bethge, Adel Bibi, and Samuel Albanie. Lifelong benchmarks: Efficient model evaluation in an era of rapid progress. ArXiv, abs/2402.19472, 2024.
Martin L.. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., 1994.
Xiaoman Qi, Panpan Zhu, Yuebin Wang, Liqiang Zhang, Junhuan Peng, Mengfan Wu, Jialong Chen, Xudong Zhao, Ning Zang, and P. Takis Mathiopoulos. MLRSNet: A multi-label high spatial resolution remote sensing dataset for semantic scene understanding. ISPRS Journal of Photogrammetry and Remote Sensing, 169:337-350, 2020.
Jingyang Qiao, Zhizhong Zhang, Xin Tan, Chengwei Chen, Yanyun Qu. Yong Peng, and Yuan Xie. Prompt gradient projection for continual learning, In International Conference on Learning Representations, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. Interna- tional Conference on Machine Learning, pp. 8748-8763, 2021.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. iCaRL: Incremental classifier and representation learning. In IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 5533-5542, 2016.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning learn without forgetting by maximizing transfer and minimizing interfer- ence. ArXiv, abs/1 110. 19900, 2018.
Mark B. Ring. CHILD: A first step towards continual learning. Machine Learning, 28:77-104, 1997.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P. Lillicrap, and Greg Wayne. Experience replay for continual learning. In Conference on Neural Information Processing Systems, 2018.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In International Conference Learning Representations, 2021.
Jonathan Schwarz, Wojeiech M. Czarnecki, Jelena Luketina. Agnieszka Crabska-Barwinsee Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable frame- work for continual learning. ArXiv, abs/1805 00000000677....0 2018.
Kritik Seth. Fruit and Vegetable Image Recognition, 2024. URL com/ com/datade dat.setstsssss...o.....oo..s..st wwssets/kritiee wtw.:iiieehh/uutttt-deeeeeeeeeee wtw.aable--mmmae-rrooottttiioiio Accessed: 2024-09-23.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. ArXiv, Abs/2310.167888 2023.
14

---

Published as a conference paper at ICLR 2025
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Conference on Neural Information Processing Systems, 2017.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, L. Sifre, George van den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Mare Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 429(758))44444488 2016.
Adilya Sisodiya. One Piece Anime, 2024. URL https: www. www.kk.agggg.e. com/ datasets/ adityy2803/ adityy22003/me--peccee-e wwwp:////eeeeeeeeeeeeeeeeeeeeeee
amee. Accessed: 2024-09-23.
James Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante- Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogério Schmidt Feris, and Zsolt Kira. CODA-Prompt: Continual decomposed autention-basse prompting for rehearsal-free continual learning. In IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 11909-11919, 2022.
Stealth Technologies. Rock Classification, 2024. URL https: / /www. kwgglaggggl.lll. com/ classification Accessed: 2024-09-23.
Phuc Thai. Butterfly Image Classification, 2024. URL https://www wwwgle.agele...... com/ 09-23.
Accessed: 2024-
Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. YFCC100M: The data in multimedia research. In Communi- cations of the ACM, volume 59, pp. 64-73, 2016.
Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews, Razvan Pascanu, and Yee Whye Teh. Functional regularisation for continual learning with Gaussian processes. In International Conference on Learning Representations, 2020.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, (88)) 2605, 2008.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech- UCSD Birds-200-2011 Dataset. 2011.
Liyua Wang, Mingtian Zhang, Zhongfan Jia, Qian Li, Chenglong Bao, Kaisheng Ma, Jun Zhu, and Yi Zhong. AFEC: Active forgetting of negative transfer in continual learning. In Conference on Neural Information Processing Systems, pp. 22379-22399 2021.
Liyua Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, and Jun Zhu. Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality. In Con- ference Neural Information Processing Systems, pp. 69054-69076, 2023a.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5362 5383, 2023b.
Siyuan Wang, Zhuohan Long, Zhihao Fan, Zhongyu Wei, and Xuanjing Huang. Benchmark self- evolving: A multi-agent framework for dynamic LLM evaluation. ArXiv, abs/2402.1 1143,, 2024.
Zifeng Wang. Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer G. Dy, and Tomas Pfister. DualPrompt: Complementary prompting for rehearsal-free continual learning, ArXiv, abs/2204.04799, 2022.
Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. SUN database: Lcrge-scale scene recognition from abbey to zoo. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3485-3492, 2010.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine leamn ing: Analyzing the connection to overfitting. IEEE Computer Security Foundations Symposium, pp. 268-282 2017.
15

---

Published as a conference paper at ICLR 2025
Jachong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal-basee continual learning. ArXiv, abs/2106.01085, 2021.
Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. SLCA: Slow learner with classifier alignment for continual learning on pre-trained model. In IEEE International Conference on Computer Vision, pp. 19091-19101, 2023.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 mil- lion image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine 2017.
Intelligence, 40(6):1452-1
Wei Zhou, Han-Jia Ye, De chuan Zhan, and Ziwei Liu. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need. ArXiv, abs/2303.07338, 2023a.
Da-Wet Zhou, Hai-Long Sun, Han-Jia Ye, and De chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23554-23564, 2024a.
Da- Ra-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Class- incremental learning: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 46(12)99851-9873 2024b.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision- language models. International Journal of Computer Vision, pp. 2337 2348, 2021.
Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. Don't make your LLM an evaluation benchmark cheater. ArXiv, Abs/2311.01164 2023b.
Kaijie Zhu, Jiaao Chen, Jindong Wang, Neil Zhenqiang Gong, Diyi Yang, and Xing Xie. Dal::: Dynamic evaluation of large language models for reasoning tasks. In International Conference on Learning Representations, 2023.
16

---

Published as a conference paper at ICLR 2025
APPENDIX
A CLDYB DATA POOL
A.1 CLDYB DATA POOL CONSTRUCTION
Photographic image data. This portion of the CLDyB data pool includes 2, 043 classes from 22 publicly available image recognition datasets, and can be categorized into two major domains:
Natural and biological scenes. This category includes images of fauna, flora, and various natural elements and ecosystems, spanning multiple levels of granularity, from coarse-grand -eass--rained categories (e.g., different animal species in Animal-90 (Banerjee, 2024)) to fine-grained distinc- tions (e.g., specific butterfly species in Butterfly-70 (Thai, 2024)).
Man-made scenes. This category contains images depicting clothing, food, building, and en- tertainment scenes, capturing rich diversity of cultural and historical contexts from multiple geographic regions and time periods. For example, it includes Food-101 (Bossard et al., 2014), CNFOOD-221(Aluz) 2024) with Chinese cuisine, and Indian Food Images (Banerjee, 2023), which together represent variety of regional culinary traditions.
Diffusion-gemerated image data. In addition to photographic images, data pool incorporates four Al-generated image datasets by SDXL (Podell et al., 2024) to expand its data distribution and introduce new challenges for CL. These datasets contain 360 classes across three animal categories and one product category, created using the following prompts:
'A high-quality image of a kind of animal: {animal name)'
"A high-quality sketch image of a kind of animal: {animal name)'
'A high-qualiny {image style) image of kind of animal: {animal name)"
'A high-quality image of a kind of product: {product name)'
Table 2 gives the details of these datasets, which retain the licensing terms of their original sources whenever explicitly stated. In cases where no specific license is provided, the dataset is distributed under the CC BY-NC 4.0 license", permitting use only for non-commercial purposes.
A.2 CURRENT CL BENCHMARKS
The standard practice for evaluating class-incremental learning methods, first established in (Re- buffi et al., 2016), involves partitioning the classes of a labeled dataset into sequentially ordered tasks with non-overlapping class labels for training and testing. Commonly used datasets include MNIST (Deng, 2012), CIFAR-100 (Krizhevsky, 2009), CUB-200 (Wah et al., 2011), and various ImageNet derivatives (Le & Yang, 2015; Hendrycks al., 2020). Additionally, benchmarks like OmniBenchmark and VTAB (Zhou et al,, 2023a) aggregate multiple standard classification datasets, each regarded an individual task. Beyond remixing available datasets, several new datasets have been specifically curated, including Core50 (Lomonaco & Maltoni, 2017), which features diverse objects in varying contexts, and CLEAR (Lin et al., 2021), which captures natural temporal pro- gressions of visual concepts. Furthermore, Liao et al. (2023) introduced CGQA and COBJ bench- marks to assess the compositional generalization abilities of CL methods. However, existing CL benchmarks remain static, and do not explicitly account for assessment of CL methods with strong foundation models.
17

---

Published as a conference paper at ICLR 2025
Table 2: Statistics of the CLDyB data pool. For certain datasets, the number of classes (i.e., # of classes) is smaller than the original, we have selected only those with fewer than 45 images.
Type
Dataset
# classes
# images
Fruits- Vegetable (Seth, 2024)
35
3,047
Jute-Pest (Islam, 2024)
17
7.151
Animal-90 (Banerjee, 2024)
90
5,400
Sca-Animal (Lanz, 2024)
13,711
Rock (Technologies, 2024)
3,687
Butterfly-75 (Thai, 2024)
75
6,499
Clothing 20 (Kaiska, 2024)
17
5,325
Apparel (Grigorev, 2024)
16,170
FoodTOT (Hossard et 2014)
101
101,000
FondX251 (Kaur et al., 2019)
118,441
Indian-Food (Banerjee. 2023)
80
4,000
Photographic image data
CNFOOD241 (Aluza, 2024)
240
170,835
Oxford5k (Philbin et al., 2007)
17
5,063
SUN397 (Xiao al., 2010)
339
17,355
Places365 (Zhou al., 2017)
1,803,400
RESISC45 (Cheng et al., 2017)
45
5,100
AT-RGB (Helber et al., 2018)
10
27,000
MIRSNet (Qi al., 2020)
46
109, 161
FGVC-Aircrafi (Maji et al., 2013)
100
10,000
Balls30 (Piosenka, 2024a)
3,708
Sports (Phosenka, 2024b)
13,492
One-Piece-Anime (Sisodiya, 2024)
18
11,737
Product-71
71
7,095
Animal-Multi-Style
133
19,962
Diffusion-generatee image data
Animal-Sketch
8,494
Animal
8,292
CLDyB data pool
Combined
2,403
2,505,185
B
CL METHODS
B. TRADITIONAL CL METHODS
Traditional class-incremental learning methods can be broadly divided into five categories: I) Replay-based, regularizaatio--aased, 3) model-based, 4) optimization-based, and 5) representation-bee based approaches. Specifically,
Replay-based methods use memory buffer ing new tasks, with recent emphasis on exemplar selection (Aljundi et al., 2019; Yoon et al., 2021), constraint optimization (Lopez-Paz & Ranzato, 2017: Chaudhry et al., 2018), and gener- ative replay (Shin et al., 2017).
and replay previous samples when learn-
Regularization-basee methods (Kirkpatrick al., 2017) penalize significant changes in CL methods to preserve acquired knowledge. These include regularization of important parameters, where the importance measured by the Fisher information matrix (Kirkpatrick et al., 2017), weight uncertainty (Ahn et al., 2019), and variational posterior (Nguyen et al., 2018), well as functional regularization that leverages Gaussian processes (Titsias et al., 2020), knowledge distillation (Li & Hoiem, 2016; Buzzega et al., 2020), and contrastive learning (Cha et al., 2021).
Model-based methods often involve model expansion (Schwarz al.. 2018; Wang et al., 2023a) and parameter isolation (Wang et al., 2022) to manage task-specific knowledge.
Optimization-based methods employ techniques like orthogonal gradient projections (Fara- jtabar et al., 2019; Saha et al., 2021) and meta-learning (Riemer et al., 2018; Gupta et al., 2020) to mitigate negative interference between tasks.
Representation-basee methods exploit the advantages of meta-learning (Javed & White, 2019), self-supervised learning (Gallardo al., 2021), and large-scale pre-training (Mehta al., 2023) to improve feature representations in model initialization and adaptation.
We refer interested readers to (Wang al., 2023b) for comprehensive survey.
18

---

Published as a conference paper at ICLR 2025
B.2 SELECTED CL METHODS FOR CLDYB
We provide a brief description of the selected CL methods for CLDyB. All CL methods work seam- lessly with foundation models of varying backbones. Table summarizes the design principles of different CL. methods for better comparison.
ER (Rolnick al., 20 is the simplest replay-based method that stores previous exemplars in a fixed memory buffer, and replays them during training to mitigate catastrophic forgetting.
AFEC (Wang et al., 2021) enhances elastic weight consolidation (Kirkpatrick et al., 2017) by introducing second Fisher information matrix derived from the current and task-specific fine- tuned models as regularization. Inspired by biological neural networks, this approach enables selective forgetting of old knowledge conflicting with new experiences. moving beyond rigid retention strategies.
CLSER (Arani et al., 2022) employs two models: stable one (for long-term memory) and plastic one (for short-term memory), updated via distinct exponential moving average (EMA) strategies. During training. it incorporates the mean squared error between the working model and the most confident EMA model as a form of distillation loss.
(Wang et al., 2022), prompt-based CL method, improves upon L2P (Zhou et al., 2021) by attaching prompts to multiple attention layers. It also distinguishes task-agnostic and task-specific prompts. enhancing the model's ability to learn both global (cross-task) task) and local (within-task) knowledge.
DualPrompt
LAE (Gao et al., 2023) maintains an online parameter-efficient module through regular training and updates an offline sarameter-efficient module using EMA. During inference, it employs routing strategy by selecting the more confident prediction from the two modules.
HiDe-Prompt (Wang et al., 2023a) enhances prior prompt-based methods with several im- provements, including prompt-ensemble strategy, a contrastive feature constraint, and a joint optimization of three tasks: within-task prediction, task-identity prediction, and task-adaptive prediction.
SLCA (Zhang et al., 2023) employs dual learning rate strategy: the feature encoder (back- bone) is fine-tuned with smaller learning rate to maintain stable feature extraction, while the classifier is updated with a larger learning rate. Additionally, SLCAA mitigates classifier forget- ting by modeling and replaying class-wise feature distributions, ensuring robust calibration of the classifier over time.
PGP (Qiao et al., 2024) demonstrates that orthogonalizing prompt gradients via singular value decomposition prevents forgetting.
RanPAC (McDonnell et al., 2023) freezes the backbone after initial task training, and applies non-learnable random projections to enhance feature separability. It further decorrelates class prototypes using the inverse of the Gram matrix of the projected features, achieving strong performance without the of any memory buffer.
Table 3: Summary of design principles of the nine selected CL methods for CLDyB.
Replay
Training
Regularization
Model
Method
Exemplar Statistic
Full
Parameter- efficient
Functional -based
Parameter Orthogonality -based
Isolation or expunsion
Ensemble
based
AFEC
CLSSRR
DualPrompt
LAE
HiDe-Prompt
SI.CA
RanPAC
PGP
v
B.3
MEMORY FOOTPRINT CALCULATION
In the calculation of memory footprint, we consider three main sources beyond the ViT backbone: 1) learner parameters including model replicas (e.g. old and EMA-updated models) and parameter-
19

---

Published as a conference paper at ICLR 2025
efficient modules, 2) dataset metadata containing per-task statisties like mean and variance, and 3) image storage retaining raw 8-bit 224 x 224 RGB exemplars from previous tasks. All data are stored in 32-bit floats.
20

---

Published as a conference paper at ICLR 2025
C
PSEUDOCODE FOR CLDYB
We present the pseudocode for CLDyB in Algorithms 1, 2, and
Algorithm A. Cr B, K)
Require: Data pool D. continually learned classifiers Fi-1 { -11111 - # candidate tasks to
M
sample B, and # of classes per task K
Return: Ar
Reduced action space
1: (CCC,, Co) = M
1
E (CosSim (u
(u 2 o. Vu, U E Dr
> Compute pairwise potentials
2: A +- 0
3: for j {1..... B} do
Sample each task
4:
VØ
< 1
5:
for E {1,
K}
do
> Sample each class in the task Sample the first class uniformly
6:
if k
then
7:
" [Di]]
8:
else
D Sample subsequent classes according to Eqn. (6)
9:
u = arg max, nevr (CC,, C.)
end
11:
V/ < V U
end for
13: A f- AUU
Add the task to the candidate set
14: end for
Algorithm A. Funct:oonaTTaslluusserriidd,, Ft-1, B,C)
(1) Require: Candidate task set A,, continually learned classifiers F.-1 = {1-1s i=m
# of candidate
tasks sample B, and of task clusters C
Return: Ar
D Condensed action space
1: G + 0lÅ.X
Initialize the task feature matrix
2: for { 1
A do [14...,]]
3: for j E {1..... M do
Compute the average negative log- log-likelihood of sample using the j-th k--N classifier (2)
predictions for task (.())
(7 log logr!") (i) P:-1 built from ,(,)
4: G[i,)] =
logr:-11yl (y)=)
5: end for
6: end for
7:
T
..... [()) }
KMeans G, C)
> Cluster A into C' task clusters
8:
A +
9: for E (1,2,.... B} do
Sample each task without replacement Sample one task cluster uniformly Sample one task from the cluster uniformly
10:
j~ [C]
T, T,(1)
12:
7(0)+
- 7(11 T,
> Remove the task from the cluster
13:
Ae A.
> Add the task to the candidate set
14: end for
21

---

Published as a conference paper at ICLR 2025
Algorithm 3 CLDyB
Require: Data pools (Dt) foundation models Fo 10
Ji= CL methods (CL
jii of
candidate tasks B and B, # of task clusters C, and # classes each task K b CLDyB task sequence
Return: TeN+1
1: Tei f 0
2: for fl..... N do
3:
A, Greedd/|sskSampllnn((0.P....,,
> Algorithm
4:
A Fuuctisonaltssschhsserrmm__,.F B.C)
D Algorithm
5: T MCTS(current state - Ft-1, action space Ac, value function = (F.-i)
Identify the optimal task according to Eqn. (7)
6:
F. (J+ (J1
1==1 where f(" = CL (1 (1-1: & Train each classifier on the selected task
7: Tet+1 + Tr
Add the task to the CLDyB sequence
8: end for
D ADDITIONAL RESULTS
D.1 CLDYB SEQUENCES WITH VARYING DIFFICULTY LEVELS
While the main text describes CLDyB as selecting the hardest task at each time step (via greedy maximization of Eqn. (7)), we extend it to generate sequences with varying difficult levels. This can be done by replacing greedy task sampling with task reward-base based sampling:
exp (J7, (Fr-1)/T) p Ti ΣTiEa, exp (J.((T
where we slightly abuse the math notation by putting the action (i.e,, Te or Ti') in the subscript of the value function J instead of the policy l4 to be optimized. The temperature hyperparameter controls the difficulty bias. Lower values (r + 0) prioritize high-reward (challenging) tasks, mimicking greedy sampling, while higher t values promote diversity by flattening the probability distribution. All other components in CLDyB remain unchanged.
Using reward-based sampling within CLDyB, we generate two additional sets of sequences at the medium and easy difficulty levels, retaining the original as hard sequences. Table 4 presents the results, where we observe two key trends. First, performance improves predictably as difficulty decreases, with the hard sequences providing performance lower bounds. Second, while minor ranking fluctuations occur, results across all CLDyB sequences remain strongly correlated with those held-out sequences.
Table 4: Final average accuracy (%) of CL methods on CLDyB sequences at the easy, medium, and hard levels.
Method
Easy
Medium
Hard
Held-out
ER (Rolnick al., 2018)
76.3
60.5
79.9
AFEC (Wang et al., 2021)
72.5
51.9
49.5
76.3
CLSER (Arani al., 2022)
75.4
60.7
57.0
81.1
DualPrompt (Wang al., 2022)
69.0
53.7
41.9
70.8
LAE (Gao et al., 2023)
70.1
50.9
48.1
71.1
HiDe-Prompt (Wang et al., 2023a)
80.1
67.6
62.5
84.9
SI.CA (Zhang al., 2023)
77.2
59.7
56.2
80.3
RanPAC (McDonnell et 2023)
77.0
62.2
56.9
81.0
PGP (Qiao et al., 2024)
69.2
52.0
44.0
68.7
SRCC (per column & held-out)
0.867*
0.833*
0,983*
N/A
KRCC (per column & held-out)
0.722*
0.667*
0.944"
N/A
D.2 ADDITIONAL FIGURES
Figs 7 12 provide additional results under various conditions, Specifically,
22

---

Published as a conference paper at ICLR 2025
Fig. 7 evaluates CL methods on random sequences.
Fig. compares the commonly challenging CLDyB sequences with random sequences.
Fig. tests the robustness of CLDyB long-sequence, commonly challenging scenarios.
Fig. 10 validates CLDyB with CLIP- ViT-Base (Radford et al., 2021) commonly challenging scenario, stronger foundation model pre-trained much larger web-scale dataset, FYCC- 100M (Thomee et al., 2016), in comparison to VTT-Base-Sup21 K (Dosovitskiy et al., 2021).
Fig. 11 assesses CL methods on the commonly challenging CLDyB sequences, when combining Viī-Base-Supi (Dosovitskiy et al., 2021) and CL.I-ViT-Base (Radford et al., 2021).
Fig. 12 benchmarks CL methods separately using the hybrid configuration from Fig. 11.
Fig. 13 compares CL methods on the CLDyB sequences, discovered from the original data pool and its augmented version with AI Al-generated image data, respectively.
BRR
APEC
CLSER
DuilPrompt
LAE
HiDe-Prompe
SLCA
REaPAC
PGP
0.90
4:18
0.80
Time stop
Figure 7: Performance comparison of CL. methods random sequences drawn from the CLDyB data pool.
D.3 VISUALIZATION OF CLDYB SEQUENCES
Figs to 17 show I-SNE visualizations (van der Maaten Hinton, 2008) tasks in the commonly and individually challenging CLDyB sequences. Figs. 18 to 20 provide sample images from the commonly challenging CLDyB sequences. Additionally, Fig 21 demonstrates the effect of adding Al Al-generated image data to the CLDyB data pool over time.
23

---

Published as a conference paper at ICLR 2025
0.90
0.00
0.90
CLDvB
10.05
-0.10
00.15
0.50
0.82
step
(a) ER
0.50
CLDsB
0.85
Asc
0.70
-0,00
0.80
-0.15
0.70
Time step
(b) AFEC
0.00
0.86
-0.05
0.85
Aoc
0.34
0.82
Randem
(c) CLSSR
Rasbam
CLDsB
0.35
0,10
0.80
0.50
.0.20
0.75
050
0.40
0.70
Time step
(d) DualPrompt
0.50
CLD,B
Randon
0.00
0505
0.80
0.35
.0.15
0.70
Time
(e) LAE
Figure 8: Joint evaluation of the five CL methods using CLDyB relative to random sequences. All methods use the ViT-Base backbone (Dosovitskiy et al., 2021), pre-trained on ImageNet-21K.
24

---

Published as a conference paper at ICLR 2025
0.55
-0.01
0.35
Randar CLDyB
4005
0.75
0.70
Time
HiDe- Prompt
0.90
0.90
CLDyb
0.85
0.80
0.70
.0.10
step
(g) SLCA
RanCom
-0.02
Acc
00.04
0.80
0.70
0.75
0.70
-0.08
0.65
6
Time step
(h) RanPAC
0.00
CLDvB
0.70
-0.10
0.60
.0.20
step
(i) PGP
Figure 8: Continued. Independent testing of the four CL methods using the same CLDyB sequences as in Figs. 8(a)-(e), relative to random sequences.
Random
0.00
0190
0.90
4.05
0,30
0.70
4.10
0.60
0.50
10
25
30
35
40
10
15
20
25
50
35
43
10
15
20
40
Timte slop
Figure 9: Average performance comparison of CL methods on random and long commonly chal- lenging CLDyB sequences (when N 40).
Random
CLDyB
0.93
4.05
0.95
0.80
4.10
0.90
-015
0.85
0.6
4.20
0.50
Time step
Figure 10: Average performance comparison of CL methods on random and CLDyB sequences, with ELIP-ViT-BEL (Radford et al., 2021) as the foundation model.
25

---

Published as a conference paper at ICLR 2025
Ron.dom
CLyya
0.00
0.90
3 0.80
0.85
0.70
000
Time stop
Figure 11: Average performance comparison of CL methods random and CLDyB sequences,
using a mixture of ViT-Base-Supee SupplK (Dosovitskiy et al., 2021) and CLIP-ViT-Base (Radford et al., 2021).
0.00
CLDvB
0.91
00.0
0.90
0.85
0.85
step
(a) ER
Rasdan
0.50
CLDsB
0.90
Acc
.0.0
0.50
0,30
0.80
Time step
(b) AFEC
0.5
CLD,B
00.05
0.90
0.85
0.85
(c) CLSSRR
Rasbata
0.95
CLDsB
00.05
Apc
0.50
-0.10
0700
0.85
Time step
(d) DualPrompt
0.50
Bandom
0.00
0.90
0.50
CLD,B
0.05
0.85
0.70
0.80
0.0
.0.2
2
Time
(e) LAE
Figure 12: Performance comparison of individual CL methods random and commonly challeng- ing CLDyB sequences.
26

---

Published as a conference paper at ICLR 2025
0.0
0.01
Randem CLDyB
-0.00
-0.1
Time
(f) HiDe-Prompt
0.00
0.90
0.95
0.50
-0.10
0.92
0.90
Randlm
650
CLDy9
40400
step
(g) SLCA
Randon
0.95
-0.01
0.90
Ace
-0.02
0.85
0.80
0.04
0.75
Time step
(h) RanPAC
0
0.95
0.90
CLDsB
Ace
0.0
0.80
step
(i) PGP
Figure 12: Continued. Performance comparison of individual CL methods random and com- monly challenging CLDyB sequences.
APEC Original
CLSER Original
HiDe Prooe-Prrgtttt Original
RanPAC Original
PGP Original Augmettdd
APEC Augmented
Augmented
HiDe-Prompt Augmented
RanPAC Augmented
Ane
0.0
0.10
0.90
0.80
Time
Figure 13: Performance comparison of CL methods on upcoming tasks selected by CL.DY from the original and the augmented data pools. Additional geneuiiinnnggertteddddeeeeeeeedd images are added the data pool at time step = 2 (denoted by the dashed line).
27

---

Published as a conference paper at ICLR 2025
Task
Task
Task
Task
Task
Task 10
Task
Task 8
Task 7
Task
Task 11
Task
Task 13
Task
Task
Task 20
Tusk
Task
Task 17
Task
Figure 14: t-SNE visualizations of tasks in the commonly challenging CLDyB sequences.
28

---

Published as a conference paper at ICLR 2025
Task
Task
Task
Task 4
Task
Task 10
Task
Task
Task 7
Task
Task 11
Task 12
Task 13
Task 14
Task 15
Task 20
Task 19
Task 18
Task 17
Task
Figure 15; t-SNE visualizations of tasks in the individually challenging CLDyB sequences for ER.
29

---

Published as a conference paper at ICLR 2025
Task
Task
Task
Task
Task
Task 10
Task
Task
Tack
Task
Task 11
Task 12
Task 13
Task 14
Task 15
Task 20
Task 19
Task 18
Task 17
Task 16
Figure 16: t-SNE visualizations of tasks in the individually challenging CLDyB sequences for LAE.
30

---

Published as a conference paper at ICLR 2025
Task
Task
Task
Task 4
Task
Task 10
Task
Task 8
Task 7
Task
Task 11
Task 12
Task 13
Task 14
Task
Task 20
Task
Task 18
Task
Task
Figure 17: t-SNE visualizations of tasks in the individually challenging CLDyB sequences for Ran- PAC.
31

---

Published as a conference paper at ICLR 2025
Class
12
13
16
17
18
Figure 18: Sample images in a commonly challenging CLDyB sequence (seed-O).
Class
11
12
13
14
15
16
17
18
Figure 19: Sample images in a commonly challenging CLDyB sequence (seed-1).
32

---

Published as a conference paper at ICLR 2025
Class
Task
Figure 20: Sample images in commonly challenging CLDyB sequence (sced-2).
Class
9
10
11
12
13
14
15
16
17
18
19
A
Task
9
Figure 21: Sample images in a commonly challenging CLDyB sequence from the augmented data pool by Al-generated images after time step t = 2. Tasks consisting of Al-generated images are selected time steps {3, 5, 7.9).
33


Reviews:
[{"review_id": "eJYdUJn6IO", "reviewer": "Reviewer_kAFn", "summary": "This paper evaluates the existing CL approaches based on pre-trained foundation models over dynamic benchmarks, raising an important question of whether the existing methods benefit from data contamination, thereby resulting in higher overall continual learning performances, thus inducing less forgetting in the network over the sequential learning tasks.", "questions": "Please refer to weaknesses section.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper is well written and the problem setup is mostly clear.\n2. This paper raises an interesting question involving the evaluation strategy of the recently proposed continual learning approaches exploiting pre-trained foundation models to mitigate catastrophic forgetting in the network while learning from sequentially arriving data.", "weaknesses": "While I find the paper interesting, questioning the existing evaluation strategy and raising a possibility that the recently proposed continual learning approaches involving pre-trained foundation models may benefit from data contamination, thereby suffering less from catastrophic forgetting over sequential learning, I have a few concerns about this paper.\n\n1. This paper considers that the foundation models are pre-trained on large-scale datasets, like ImageNet-1k or ImageNet-21k, thereby, considers creating a dynamic benchmark with various datasets ranging across various types. However, the authors failed to consider that there are other large-scale datasets, such as JFT-300M, which might be the superset of the datasets considered by the authors in this paper, thereby raising a crucial question of what would happen in such a case. Furthermore, it is very much possible to train a foundation model with no supervision by scraping all images from the internet, thereby pre-training with a superset of these considered datasets, which raises the same question.\n\n2. This paper does not consider online continual learning setup [1], which I believe is a crucial variant of continual learning research. However, I believe this paper cannot handle such a scenario.\n\n[1] Prabhu, Ameya, Philip HS Torr, and Puneet K. Dokania. \"Gdumb: A simple approach that questions our progress in continual learning.\" In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pp. 524-540. Springer International Publishing, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rIJBpOpezy", "reviewer": "Reviewer_RbVT", "summary": "This paper proposes a “dynamic” framework for evaluating continual learning algorithms in a way that addresses two issues: i) the potential data contamination inherent in the methodology of using pretrained models as a starting point for continual learning, and ii) lack of realism in prior benchmarks due to their static nature. \n\nSpecifically, the proposed benchmark is dynamic in that it forms sequences of tasks using M CL algorithms. These algorithms are used to determine 1) the most challenging tasks, and 2) the most challenging ordering of them. For the former, the authors use a Markov Random Field construction with pairwise potentials encoding the similarity between pairs of classes (using their prototypes in embedding space). They employ a greedy algorithm for picking K classes that are most similar (higher pairwise potentials), starting with a randomly-chosen class, to ensure enough diversity. They then perform additional clustering and sample uniformly from clusters, to ensure enough diversity in the set of K-way classification problems that are chosen. Then, for picking the hardest sequence, they formulate an online sequential decision-making problem where the reward is defined based on a notion of difficulty (increased forgetting and decreased task accuracy). They solve the problem using Monte-Carlo Tree Search (MCTS). To mitigate the expensive nature of MCTS (due to the rollouts, which in this case require training models on different sequences of tasks), they use a look-ahead variation that is more compute-efficient at the expense of an approximation bias.\n\nThey propose two modes of using their dynamic benchmark: the first finds task sequences that are “commonly challenging” for a pool of considered CL algorithms / backbones, while the second finds task sequences that are individually challenging for a given CL algorithm / backbone. They perform several analyses with different CL algorithms on different task sequences.", "questions": "- The authors argue (at the start of Section 3.1) that when there is data contamination in the context of a pretrained model, the resulting high performance on CL tasks does not provide a meaningful comparison between CL algorithms. Why is this the case? I agree of course that if there is data contamination, this may lead to overall inflated performance (compared to had there not been data contamination), but why are the relative differences between the performance of different CL algorithms not meaningful?\n\n- There seems to be a hypothesis that “hardest” task sequences are more realistic. But this isn’t always in the case: many practical sequences of interest may not be adversarially difficult. And it might not be the case that the relative ranking between CL methods at the “hardest” sequences is the same as that for average-case / realistically-occurring sequences. Is there some empirical evidence that hardest tasks align best with realistic scenarios (in terms of relative ranking of CL methods)? It may make sense to offer different levels of difficulty (e.g. easy, medium and hard), and observe how the relative rankings of CL methods vary across these?\n\n- How compute intensive is it, overall, to find the hardest tasks and hardest sequence? Is the computational cost required a function of M? What are the relevant trade-offs to consider?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper is for the most part well written and easy to follow (see some exceptions in Weaknesses and Questions below)\n\n- The proposed benchmark is well motivated and well-thought out (though see some questions below regarding some exceptions) and seems like a useful contribution to the CL community.\n\n- Empirically, the authors show that the proposed approach for finding “generally challenging” sequences leads to task sequences that are challenging for “held out” CL methods too (that weren’t used during the sequence selection), making it a meaningful benchmark for measuring progress of new algorithms. \n\n- On the other hand, the sequences generated by the benchmark individually for different CL algorithms can lead to an understanding of that algorithm’s limitations, and capture similarities between different CL algorithms in terms of their failure modes, which is very interesting.\n\n- Several interesting and thorough analyses are considered, including the ability to expand the benchmark using generated data, and ablation studies of various components.", "weaknesses": "- Some clarity issues. For example, under Equation 2, certain symbols (e.g. the set of algorithms and models A, and F) are defined that aren’t used in Equations 1 and 2, but become relevant later and it’s not clear how they are integrated into the quantities AFM and ALA – is there an average missing?\n\n- Related to the above, in Section 4.1, the authors discuss the metrics used for evaluation and mention AA, AR and ALA, but not AFM. Why are there different metrics considered here now? Why is AFM excluded from evaluation? \n\n- Another important clarity issue regarding the experimental setup: the authors mention CL algorithms and backbones, and for the former, discuss that they were selected to ensure high performance and sufficient diversity / coverage. But how were the backbone selected? Are all CL algorithms applied on the same pretrained backbone, or are there different pretrained models and architectures considered? This feels like a very important aspect that is not discussed.\n\n- When defining intra-class difficulty, it’s not clear that the proposed operationalization of “difficulty” corresponds with the issue of data contamination, which the authors use as motivation for choosing difficult tasks. Instead, the chosen approach seems to correspond to capturing difficulty as the difficulty to discriminate between pairs of classes based on their embedding-space prototypes, which isn’t *directly* related to having seen those classes at pretraining time (it is to some extent of course since what is seen at pretraining time influences the embedding function). It would be great to add some discussion about this. An alternative approach would be to independently choose the K “hardest” classes, rather than picking based on pairwise relationships (this might be seen as more directly capturing potential data contamination). This could be done e.g. by picking the classes whose examples are predicted with the smallest possible confidence in a classification task containing all classes. Have the authors considered such alternative approaches?\n\n- Regarding the results and experimental setup in Table 1, can’t one claim that “Heldout” isn’t truly held-out due to potential data contamination issues that the authors have pointed out elsewhere in the paper? Has care been taken to alleviate this concern here? If not, can these results regarding the correlation of performance between Heldout and other benchmarks be confounded by potential data contamination?\n\n- In the caption of Figure 1, the authors claim that the figure shows the insufficient sensitivity to performance differences, but I don’t see how this argument can be made without knowing whether the evaluated methods do in fact perform differently. If we knew that, we could say it’s a limitation of the benchmark for not capturing this. But if we don’t know that, we can’t rule out that these methods are actually indistinguishable from one another in terms of their performance. Could you comment on this?\n\n- Insufficient discussion / description of the chosen CL algorithms (and families of CL algorithms in general). It is hard to follow the discussion of e.g. the trade-offs of different algorithms (based on Fig 4) without any description of those algorithms. Consider swapping the order of section to have Related Work appear before the experiments to address this.\n\n- Insufficient discussion of other benchmarks. While the authors describe some in the related work, they don’t discuss what blind spots of those benchmarks (that are also dynamic), the proposed benchmark addresses. Further, in analyses like the one in Table 1, it would have been great to compare against such dynamic benchmarks rather than static ones like CIFAR-100 and ImageNet-R.\n\n- Related work: it would be great to also discuss Nevis’22 (see References below) which also is motivated by building a more realistic benchmark for continual learning.\n\n- Relationship to related work: are the trade-offs observed in Figure 4 between different families of methods novel? Have prior work pointed out similar trade-offs? What are the new findings here?\n\n- In Equation 4, why are these objectives chosen in particular, and why with that particular weighting? \n\n\nMinor:\n\n“While continue learning approaches” → “While continual learning approaches” (line 062)\n\n\nReferences\n\nNevis’22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research. Bornschein et al. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fk4yD5gtg0", "reviewer": "Reviewer_BrAw", "summary": "The paper benchmarks different approaches in CL. The main contribution of this work is the use of optimization and tree search to come up with more challenging tasks.", "questions": "How does the approach ensure that the CL methods is not biased by the choice of the training sample. If uniform random is being used, I would understand it, however, in this case, there seems to be some form of search and it is easy to derail this search.\n\nThe metric is confusing, a smaller average accuracy seems to be the best choice, why? should it not be the other way round.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Dynamic benchmarking seems like a novel idea. \nThe evaluation methodology is scalable\nThe metrics make sense\nA well written paper\nThe graphics are very nice.", "weaknesses": "Why do we talk about benchmarking pre-trained models on CiFAR100, not even a CL dataset, nor an LLM dataset.\n\nThe training can only work with Pre-trained models, no starting from scratch.\n\nThe difficulty of choosing tasks is based on  some clustering approach, I would have thought that using some directional gradient to pick a task that pushes the cost in a particular direction to be much more reasonable for selecting the new tasks. .", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)