Class Incremental Learning from First Principles: A Review
Authors: Neil Ashtekar
Jingxi Zhu
Vasant G Honavar
Venue: Accepted by TMLR ()

Abstract: Continual learning systems attempt to efficiently learn over time without forgetting previously acquired knowledge. In recent years, there has been an explosion of work on continual learning, mainly focused on the class-incremental learning (CIL) setting. In this review, we take a step back and reconsider the CIL problem. We reexamine the problem definition and describe its unique challenges, contextualize existing solutions by analyzing non-continual approaches, and investigate the implications of various problem configurations. Our goal is to provide an alternative perspective to existing work on CIL and direct attention toward unexplored aspects of the problem.

Published in Transactions on Machine Learning Research (anuary/202))
Class Incremental Learning from First Principles: A Review
Neil Ashtckar
ncc.5096 edu
Artificial Intelligence Research Laboratory Pennsylvania State University
Jingxi Zhu
jq25678@psut edu
Artificial Intelligence Research Laboratory
Pennsylvania State University
Vasant G Honavar
ohonavorGist.per edu
Artificial Intellignnce Research Laboratory Pennsylvania State University
Reviewed OpenReview: https: // openreview. net/ forun? id= sZdtTJIIng
Abstract
Contimual learning systems attempt to efficiently learn over time without forgetting previ- ously acquired knowledge. recent years, there has been an explosion of work on continual learning, mainly focused on the class-incremental learning (CTL.) setting. In this review, we take a step back and reconsider the CIL problem. We reexamine the problem definition and describe its unique challenges, contextualize existing solutions by analyzing non-contimual approaches, and investigate the implications of various problem configurations. Our goal is to provide an alternative perspective to existing work on CIL and direct attention toward unexplored aspects of the problem.
1 Introduction
In the past. decade, machine learning has made tremendous progress on wide range of applications including computer vision, natural language processing, recommender systems, robotics, and more. The dominant approach deep lcarning - generally performs well given large datasets in the i.i.d. setting, in which many passes may be made over the same data during training. However, existing methods fail when new data arrives, as users must either (i) store the old data and retrain jointly on both the old and new data, thus incurring significant memory and computation costs, or (ii) choose between learning from the new data and retaining previously learned knowledge. The trade-off described in (ii) is commonly known the stability- plasticily dilemma (Carpenter & Grossberg, 1988). Recent work on continual learning has focused ways to balance the stability-plastiiitty dilemma, with many solutions proposed to alleviate catastrophic forgetting, referring to the loss of old knowledge when learning anew (McCloskey & Cohen, 1989).
In this review, we consider supervised classification problems in the continual setting. Specifically, we focus on the class-incremental learning (CIL) problem sctting, in which new classes of data are introduced over time (Van de Ven & Tolias, 2018). We seek to thoroughly understand both the problem statement and existing solutions, with focus unexplored directions.
Our work differs from recent continual learning review articles in several key ways, as summarized in Table 1. First, we specifically address the CIL setting, unlike De Lange et al. (2021); Mundt et al. (2023); Wang et al. (2024), and Verwimp et al. (2024). Second, we provide a novel categorization of existing approaches based on the shortcomings of non-continual methods, thus motivating and contextualizing recent work. Third, and perhaps most importantly, we focus on understanding and refining the CIL problem statement. This differs from Belouadah et al. (2021); Mai et al. (2022); Masana et. al. (2022) and Zhou et al. (2024), which

---

Published in Transactions on Machine Learning Research (annary//022)
Table 1: Brief summary of review articles on continual learning (CL), class- inassminceeeetall learning (CIL), and mask-inremenntl learning (TIL).
Review
Submission Year
Content
Reviews and evaluates TIL. approaches, focused on relatively early work
De Lange et al. (2021)
2019
Summarizes work on CIL with empirical evaluations on image classification benchmarks
Belouadah al. (2021)
2020
Focuses on empirical evaluations in online CIL setting over various performance metrics
Mai et al. (2022)
2020
Reviews work CIL in the context of image classification with
Masana et al. (2022)
2020
evaluations across various lask-splits and replay strategies
Summarizes work on CL and establishes connections to open-set recognition and active learning
Mundt et al. (2023)
2020
Reviews deep learning approaches to CIT. with memory-aligned evaluations image classification benchmarks
Zhou et (2024)
2023
Broad overview of work CL covering computer vision, natural language processing, reinforcement learning, etc.
Wang et al. (2021)
2023
Summarizes and evaluates CIL methods specifically designed for the few-shot setting (FSCIL)
Tian et al. (2024)
2023
Outlines real-world CL applications in which "continual learning is nol choice" and describes general directions for future work
Verwimp et al. (2024)
2023
Defines the CIL problem statement under resource constraints, describes properties of successful solutions
Ours
2024
summarize and evaluate existing solutions. Instead, we investigate the challenging aspects of the problem and analyze the properties of successful CIL system. Our main contributions are outlined below:
In Section 2, we argue that the CIL problem should be redefined using constraints on memory and compute. We show that a commonly used problem statement is imprecise and admits extremely inefficient solutions.
In Section 3, we describe key challenges introduced by CIL. We show that addressing these challenges both necessary and sufficient for solving the broader problem. In particular, we highlight the importance of across-task discrimination and provide corresponding performance metric.
In Sections 4 and 5, consider "naive" approaches which were not designed for continual learn- ing. We analyze their strengths, weaknesses, and required modifications in the CIL setting, thus motivating existing work. Within this framing, we provide overview of existing work.
In Section 6, we investigate the dimensions of the CIL problem. We characterize when the problem is easy versus hard, and describe how existing work addresses various problem configurations. Notably, we investigate class-to-task assignment and provide a surprising example illustrating its importance.
In Section 7, we conclude with suggestions for future work. Specifically, we argue that future solutions can and should make assumptions regarding the data generating process, as certain aspects of the world will remain fixed as new tasks are introduced.
2

---

Published in Transactions on Machine Learning Research (January /2025)
2 Problem Statement
In this section, we formally define our problem statement. We focus our discussion on the class- inssemeactetttll learning (CIL) setting as proposed in Van de Ven & Tolias (2018). We borrow notation and definitions from Zhou et al. (2024) and Kumar et al. (2023). First, we state the objective of CTL, then we provide two forms of restrictions imposed on potential solutions. These restrictions are crucial for ensuring that the problem is well defined and that solutions are non-trivial (i.e., CIL could be trivially solved by storing and repeatedly retraining on all of the data, though this highly inefficient). We argue that restrictions in the form of resource constraints are more appropriate than restrictions the availability of previously learned data.
2.1 Class-Incremental Learning
In CIL, we wish to soquentially learn a set of tasks 1...., T. For each task & E {1...., T), we are given a corresponding dataset D; = {aa uf )==] sampled from an unknown, underlying distribut ion Da ~ The Each task is a supervised classification problem with feature vectors af and categorical labels of Ma. One or more classes are included in each task, and classes are disjoint across tasks (Y& n Y = 0, wk # k and _f_ Xx= Y).
The overall goal is to learn a classifier x + Y with strong generalization performance across all classes. iast-identiiiiation information is not available to the model at inference. The classificr should perform woll on all previously observed classes after learning each task: after learning task k, performance should be strong on all classes Wi:k- At task k this goal can be expressed as learning ft from the model's hypothesis space H such that:
fī argmin JEE (T.g)~~ia
E
[Tfa(()) + v)l
(1)
2.2
Restrictions while Learning
2.2.1 Old Restriction:
Unavailability of Previous Data
Much of the existing work on CIL assumess that the data from previous tasks is not available when learning the current task. Formally, when learning task k, the model can only access Dz, while D1.k-1 are inaccessible. Many approaches to CIL (Kim et al., 2022; Van De Ven et al., 2021; Zhu et al., 2021a; Zhang et al., 2020a; Tao et al., 2020) as well as a recent survey on deep CIL (Zhou et al., 2024) explicitly include this restriction in the problem statement.
Completely restricting access to previous task data may appear reasonable. This restriction disallows trivial approaches which retrain on all of the data every task. Also, security and/or privacy concerns are often cited as reason to restrict access to old data. However, we argue that this restriction is both unnecessary and imadeguate for building useful CIL systems. To understand why, consider the following critiques.
Critique #1: Inefficient solutions are allowed. First, if the unavailability of previous data is the only restriction (i.e., there are no other restrictions on memory and/or compute), then extremely inefficient solutions are allowed. a recent analysis on the ImageNet-1K dataset, Harun et al. (2023a) finds that several highly-cited CIL algorithms actually use more compute" than trivially retraining on all of the data at every task! In one respect, such algorithms defeat the purpose of using continual learning.
Critique #2: Security and privacy violations may persist. Second, note that deep neural networks have the capacity to essentially "memorize" training data (Zhang et al., 2021) and that decoding schemes can be used to reconstruct data from a trained network (Haim et al., 2022). As discussed in Verwimp et al. (2024), these observations lead us to question whether restricting access to old data truly addresses security or privacy concerns, as sensitive data could simply be recovered from trained models. This concern is not specific to deep neural networks even simple generative or prototype- basod classifiers may memorize data or be susceptible to data leakage.
Measured in total number of backprooagation updates
3

---

Published in Transactions on Machine Learning Research (anuary/2025)
Critique #3: Imprecise problem definition. Third, if previous task data cannot be stored but previ- ously learned model(s) can be stored, then it becomes necessary to formally answer the question "What is a model?" or at least answer "What is the difference between a model and a dataset?" The existence of models such as K-nearest neighbors (which store the entire dataset) and prototype- based classifiers (which store representatives which may be very similar to the data) make these questions difficult to answer. Note that some carly work has defined learning algorithms (models) with respect to data compression in the PAC learning framework (Blumer et al., 1987; Takimoto & Maruoka, 1993), though these definitions have not been referenced in the modern continual learning literature. For this reason, we argue that framing the CIL problem based on the unavailability of previous data results in an imprecise problem definition.
A note regarding relacations of this restriction. We acknowledge that many approaches to CIL consider a relaxation of this restriction in which a relatively small subset of previous task data is available when learning the current task (see the discussion on replay-based approaches solutions are typically allowed to store either a fixed number of samples per-class (Rolnick et al., 2019) or fixed number of total samples (Wang el. al., 2023). While we feel that this framing is a step in the right direction, we argue that it is still lacking. This is because (i) it does not include restrictions on compute, (ii) the number of stored samples is constrained but the size of the predictive model is not, and (iii) unnccessary assumptions are made regarding data storage. Drawback (i) allows computationaallyineefi inffficcet solutions, which may be undesirable for CIT applications. Drawback (ii) implies that the total memory cost stored samples and model parameters may not be appropriately measured or constrained. Finally, drawback (ii) may introduce artificial constraints limiting solution performance. For example, it may be suboptimal to store the same number of samples for each class. Alternatively, storing task-specific model components can sometimes be better use of given memory budget as compared to storing data samples, discussed in Zhou et al. (2023).
Section 5). In this framing,
2.2.2 New Restriction: Resource Constraints
An alternative framing of the CIL problem restricts solutions using resource constraints instoad of restricting access to previous task data. Recent work frames the continual learning problem as computationally con- strained reinforcement learning (Kumar et al., 2023). In this framing, continual learning agent attempts to maximize its average reward? under restrictions on memory and compute. For example, memory constraints may limit the total size of model, while computational constraints may limit the number of floating point operations performed when learning each task.
Resource constraints prevent trivial retraining using all of the data at every task, either because storing the entire dataset would require too much memory and/or because full retraining would be too computationally expensive. Because trivial retraining infeasible, continual learning methods are necessary. We argue that framing the CIL problem in terms of resource constraints rather than data availability results in a more appropriate problem statement and will lead to the development of truly useful, real-world CIL systems.
Resource constraints can take many forms. For example, edge devices performing CTT may have strict latency or power constraints, permitting only a small number of lcarning updates (Yoshikiyo et al., 2022; Wang et al., 2022b). On the other hand, large GPU servers performing CIL may be relatively unconstrained with respect to memory, and are instead constrained by the monetary cost of compute (e.g., renting large instances on cloud computing platforms may be prohibitively expensive, see Prabhu ct al. (2023b) for details). In this case, it may be desirable to develop CIL algorithms which jointly optimize for classification performance well as computational efficiency. Here, resource constraints are not fixed, but rather become part of the optimization objective.
In summary, we emphasize the importance of including application-specific resource constraints as part of the CIT problem definition. Doing so avoids the issues caused by restricting the availability of previons task data: resource constraints formally define the space of potential solutions and disallow loopholes, there is no need to formally define "model" versus "data", and the problem not made artificially difficult by misconstrued privacy or security concerns. In the following sections, we will consider the resource-onsstrained version of the CIL problem statement.
2In CIT, average reward equivalent average negative log-loss (I.e., cross-entropy loss) over all classes.
4

---

Published in Transactions on Machine Learning Research (January /2025)
2.3 Related Problem Statements
We briefty describe other continual classification problem statements below, based on the categorization given in Van de Ven & Tolias (2018). While the focus of this work is CIT, note that the arguments for the resouree-conttranned setting in Section 2.2 are applicable to CIL well as TIL, DII, and OCL, and extend to continual learning problem settings even beyond classification.
Few-Shot Inasemnncremttal Learning (FSCIL). FSCIL involves contimually learning new classes of data given only a small number of samples per class. This sctting is typically addressed using some form of pretraining: learning on a large, general-purpose dataset with many samples prior to continual learning. FSCIL inherits the challenges of CIL described in Section 3.1, with the added difficulty of unreliable empirical risk minimization, i.e., avoiding overfitting when learning from small sample size (Wang et al., 2020).
Incremnnremnttll Learning (TIL). The TIL problem setting is very similar to that of CIL: new classes are introduced incrementally, and the classifier must efficiently adapt while maintaining previously learned knowledge. However, in TIL, task-identifiers are provided at inference. This removes the need for learning across-task discrimination, discussed in Section 3.1. TIL can be viewed as an "casier" version of CIL.
Domain-Incremennal Learning (DIL). In DII. the set of classes is fixed, but the distribution of features for each class changes over time. Similarly to CIL and TIL, the classifier must continually learn while miti- gating forgetting. Unlike CIL and TIL, this challenge must be addressed at the level of feature distributions rather than at the class-level. Task-identifiers are unavailable at inference. Therefore, the key challenges described in Section 3.1 for CIL each have counterparts for DIL.
Online Continual Learning (OCL). OCL involves learning from small batches of data introduced incre- mentally (Mai et al., 2022). This differs from (non-online) CTL, in which all of the data from a given class is available when lcarning its corresponding task. In other words, samples from a single class may be split across multiple timesteps. In the extreme case, samples may be introduced one at time. This introduces the additional challenge of aggregating knowledge across timesteps for samples within given class. Note that OCL is a broad categorization which is not specific to classification. In addition, these problem sottings can be combined. For example, CIL + DIL + OCL is sometimes referred to as Task-Fre Continual Learning (TFCL. (Aljundi et al., 2019a).
2.4 Real-world CIL Example: Social Media Post Classification
To motivate our problem statement, we outline a potential real-world system which fits the resource- constrained CIL setting. We discuss the problem of real-time, social media post classification, with new classes introduced incrementally. Classes could correspond to post content (lext, images, or videos) with applications for fraud detection, content moderation, targeted advertising, and more. Note that this prob- lem may not be trivially solvable using hashtags or interest tagging, as posts may be implicitly referring to given topic and fraudulent content may be intentionally obfuscated. The problem may be framed "traditional" CIT, or online CIT, depending on how the data is collected and training is scheduled.
Popular social media platforms receive an extremely high volume of posts according to Twitter's official blog, an average of 500 million tweets were shared every day in 2014 - and new classes may be introduced in a short time period corresponding to current events or newly developed types of fraud. Further, posts may nced to be classified in near real-time in order to avoid the spread of misinformation or limit online scams. These concerns motivate the resouree-constrainedd setting the extremely high volume of posts necessitates computationally efficient learning, and the real-time nature of the application requires a low inference cost.
In such sctting, the entire sequence of training data is stored by default (i.c., posts from several years ago are available on a user's profile and therefore must be stored on the platform's server). This removes the "unavailability of previous data" restriction discussed in Section 2.2 and places on the focus instead on computational efficiency. Note that this shift in focus significantly changes the solution space. Solutions leveraging K-nearest neighbors or other forms of weighte learning may be appropriate, and such techniques could be learned atop pretrained representations see Prabhu et al. (2023b) for one such approach.
5

---

old tasks as new tasks Published in Transactions on Machine Learning Research (anuary/2025)
Such approach may serve starting point, with modifications necessary to meet performance and efficiency goals. For example, the pretrained representation may need to be continually fine-tuned in order to improve classification accuracy, requiring some form of replay and/or knowledge distillation (see Section for details). Further, even if all of the training data is stored, computational constraints may limit the amount of data used when learning continually, necessitating an intelligent sampling stralegy. Finally, efficient inference techniques (e.g., forms of locality-sensitive hashing) may be required to meet near real-time classification constraints. Note that the example of social media post classification is not unique: applications ranging from cybersecurity to epidemiology involve similarly high volumes of data and may require fast inference. For further examples of real-world continual learning applications, see Verwimp et al. (2024).
3 Key Challenges
It is worth understanding what makes the CIL problem challenging as compared to learning in the non- continual setting. We outline three main challenges: balancing stability and plasticity, learning across-task discrimination, and transferring knowledge across tasks. None of these challenges are present in the non- continual setting, which can be thought of as a single task. We first describe these challenges then discuss why they are significant.
3.1 Challenges
Balancing stability and plasticity. The stability-plasticity dilemma refers to the ability to learn new tasks while maintaining previously learned knowledge (Carpenter & Grossberg. 1988). Loss of stability is typically referred to as forgelling or calastrophic forgelling (McCloskey & Cohen, 1989), due to its empirically observed severity and can be measured as the degradation of model performance
are learned. Lack of plasticity can be defined using intransigence (Chaudhry et al., 2018a). Intransigence reflects a model's inability to acquire new knowledge, measured as the difference in performance across the continual and non-continual settings.
To provide formal definitions, we introduce notation originally proposed in Lopez-Paz & Ranzato (2017). Let R T be the train-test performance matrix, with elements indicating some performance measure* for given classifier. Element R., denotes the performance on the test set of task j immediately after learning the training set of task i. We assumne that tasks are learned in the order in which they are presented: tasks 1....i-1 are learned in order before task i is learned. For all of the following metrics with subscripts i and j. we assume that task is learned before task j. Formally, i S in Eas. 2, 4, 5, and 6.
The amount of forgetting on task after learning tasks i i l...., . can be quantified as:
F.,
max kE(f....j-1)
R3, R
(2)
The max operation included in order to account for backward knowledge transfer see discussion following Eq. 6 for details. The intransigence on task i after learning tasks 1.....i i can be quantified as:
R; R
Here, R denotes the performance of a reference model on task This reference model is learned in the non-continual setting, i.e, jointly trained on all data Df_ Performance in the non-continual setting is often used an upper bound for performance in the continual setting, and the corresponding performance gap indicates an inability to continually learn new tasks. For fair evaluations, the reference model and continual learning model should have similar architectures.
We provide simplified notation for pains of tasks detailed notation for overall forgetting and kuowlodge transfer given in Lopez-Paz & Ranzato (2017); Chaudhry al. (2018b); Didz-Rodrig al. (2018) "Classification accuracy commonly used performance measure. general, measures should be chosen such that higher values correspond to better performance.

---

Published in Transactions on Machine Learning Research (anuary/2025)
Learning across-task discrimination." At inference, CIL methods must perform classification across all learned classes without the aid of task-identifiers. Namely, after learning task k, the model should be able to perform classification over all classes IIA- This requires the ability to discriminate between classes within each task and across all tasks. Note that the definitions of stability and plasticity defined above are insulficient for ensuring that across-task discrimination is learned. This is because these definitions only consider within-task performance. CIL solutions must include some mechanism to learn and maintain across-task discrimination.
To formally define across-task discrimination, consider the normalized confusion matrix M populated after learning tasks 1, .,. Matrix M has dimensionality 1Y1, x [1::],, with element Mm indicating the proportion of instances of class m which are predicted as class n after evaluation on a test set. In other words, the rows of M correspond to the actual classes, while the columns M correspond to the predicted classes. To simplify notation, let C, be the set of indices corresponding to the classes task For example, C1 = (l..... [1] C2 [[ 1....., [1 + |2ll, and so on. Within a task, the assignment of indices classes is arbitrary. Consider a model which has learned tasks ,,...,.: -,, and consider task i such that j- The model's across-task task discrimination performance with respect to task i can be defined as:
ATD..
Σ mEC;
Σ M nEC:
[C
m,
This quantity simply measures how often the model predicts class from the correct task al inference. Visually, it corresponds to the values within a square subset of the confusion matrix along the diagonal.
Transferring knowledge across tasks. It is reasonable to assume that there will be similarity across tasks in a given CIL application. Ideally, knowledge accumulated from old tasks should improve performance when learning new tasks, and learning new tasks should also improve performance on old tasks. In other words, both forward transfer and backward transfer are desirable. Consider two tasks, and j, with < j. The forward transfer from earlier tasks I,...,i to later task j can be measured as:
FWT,, = R
(5)
This definition of forward transfer is equivalent to the model's zero-shot performance task j. The backward transfer from later tasks i + 1, ......j to earlier task i can be measured as:
BWT,, = max kE(
R,,- RE,s
(6)
This definition of backward transfer is similar to the negation of forgetting in Eq. 2. Forgetting measures how much performance on an earlier task decreases when learning later tasks, while backward transfer measures how much performance on an earlier lask increnses when learning later tasks. These metrics are typically reported only when their values are nonnegative: when backward transfer is positive, it is reported rather than "negative forgetting", and vice versa. Note that Eqs. and both include the max operation this is because both definitions are cumulative, For example, knowledge may be transferred backward and later forgotten, or knowledge may forgotten and later re-lcarned through backward transfer. Eqs. 2 and 6 account for such situations. Forgetting and backward transfer could alternatively be defined without the max operation (replacing the RE,; term with R.,)) resulting in a non-cumulative definitions.
3.2 Why these challenges?
Successful within-task discrimination is a consequence of stability and plasticity. While within- task discrimination is a necessary component of successful CIL systems, we do not include it as a key
5 AAross-task discrimination is sometimes called cross-task discrimination, inter-task separability, inter-task confusion (Masana et al., 2022; Huang al., 2023; Nori & Kim, 2024) We vertical bars indicate the numher of elements within a set (cardinality). Here, [Y1gl is the total number of classes learned tasks ......j

---

Published in Transactions on Machine Learning Research (anuary/2025)
challenge. Instead, we argue that achieving and maintaining strong within-task performance is consequence of sufficient stability and plasticity. Recall that R., denotes the performance on task j after learning task i. Strong within-task performance can be formalized as follows: immediately after task is learned, R should be sufficiently high for all previously learned tasks j I i. This should hold for all tasks E {I.....T) T).. Note that the terms "strong performance" and "sufficiently high" are subjective, and can be defined based on application-specfic and/or task-specific criteria.
This formalization simply requires that performance be strong for the most recently learned task as well as for all of the other previously learned tasks. Strong performance on the most recently learned task is equivalent to low intransigence in Eq. 3. Maintaining performance on prior tasks is equivalent to low forgetting in Eq. 2. Therefore, sufficient stability and plasticity results in successful within-task discrimination.
Learning across-task discrimination is necessary for CIL. As proved in Kim et al. (2022), strong within-task discrimination and strong tarossttattliisciiinaaooon are together necessary and sufficient con- ditions for strong CIL performance. In other words, a model will have strong overall performance if and only if it is can accurately distinguish between classes from the same task and across classes from different tasks. Learning and maintaining across-task discrimination is therefore key challenge in CIL.
Strong across-task performance can be formalized as follows. Immediately after learning task j. ATD,,, (Eq. 4) should be sufficiently high for all previously learned tasks i S j. This condition should hold for each task E (I.....T}. T}. This formalization is very similar to the formalization of strong within-task performance, with ATD used instead of R.,j- However, there is an important difference between the within-task and across-task discrimination problems: the across-losk problem grows as new lasks are introduced, while each within-task problem has a fixed number of classes. It may be necessary recalibrate performance expectations as the number of tasks grows and across-task discrimination increases in difficulty."
In addition, we argue that across-task discrimination is a larger part of the overall CIL problem as compared to within-task discrimination. To provide evidence for this claim, we first discuss a commonly used CIL benchnark the split CIFAR-100 dataset- then discuss the general case. The CIFAR-100 dalaset contains 100 classes, and is often divided into 10 tasks, each containing 10 classes, to form the "split" version for CIL evaluations. Given sample at inference after all tasks have been learned, the model should predict its corresponding class out of the 100 total classes. This requires distinguishing the correct class (i) versus 9 incorrect classes from the same task and (ii) versus 90 incorrect classes from ht tasks. In this example, across-task discrimination dominates the CIL problem, as the model must predict across many more classes out-of-task versus within-task.
This observation extends beyond any specific dataset. Unless one task contains more than half of all of the learned classes, it will always be true that across-task discrimination is a larger part of the CIL problem compared to within-task discrimination. While couple of recent studics SSoutif-Corment Guo et al., 2023) focus specifically on the across-task discrimination problem in CIL, are not aware of any prior work which explicitly mentions this observation. This is noteworthy considering that many CIL approaches heavily emphasize the avoidance of within-task forgetting while placing relatively little emphasis on across-task discrimination - see Section 5 for details.
et al, 2021;
Knowledge transfer can improve both performance and resource efficiency. Exploiting knowledge transfer between similar tasks could improve model's predictive performance and decrease both memory and computational costs. For example, sharing parameters across tasks could decrease overall model size compared to learning isolated submodels for each task. A model could exploit forward transfer (Eq. 5) by initializing new, task-specific parameters to the values learned on a prior, similar task, thus reducing the number of iterations required for convergence. Such approaches may be necessary to achieve a desired level of predictive performance under resource constraints.
7 "Arrssstask discrimination is generally harder given more tasks. For example, would expect stronger performance when discriminating between tasks compared tasks. See Michel et al. (2023) for difficuly-adjusted metrics for CIL. 8 Class to-tsk assignment typically performed randomly or based on the sequential class labels, e.g., classes 0 are assigned to the first task, 19 assigned to the second class, etc.
8

---

Published in Transactions on Machine Learning Research (January /202))
4 Naive Approaches
In this section, we discuss the application of traditional (i.e., non-continual) machine learning classifiers in the CIL setting. This is "naive" in the sense that these models would be expected to fail in the CIL setting, they are not designed for continual learning. We discuss three types of classifiers: (1) single discriminative model for all tasks, (2) task-specific discriminative models, and (3) class-specific generative models. These approaches are illustrated Figure 1.
Some of the observations in this section may seem obvious or redundant in light of recent work on continual learning. However, we feel that this section is important it provides principled way of developing solutions to the CIL problem. This section also serves to contextualize existing work and provide insights for potential future work. Table 2 and Table 3 summarize the main takeaways. Table 2 outlines how each naive approach addresses the "Key Challenges" described in Section 3, while Table 3 outlines the memory and compute requirements for each approach, relevant to the resource-constrained problem statement defined in Section 2.
4.1 Background: Generative and Discriminative Classifiers
Discriminative Classifiers. Probabilistic discriminative classifiers directly model P(y Discriminatively trained models such as logistic regression and deep learning classifiers typically accomplish this by optimizing for cross-entropy loss:
N
C
CCE = - Σ
yico log Di.
where Lee is the cross-entropy loss on a training set with N samples and C classes. The true labels are represented as y and the predicted labels are represented as ÿ. For sample i, y, and 9. are C-dimensional vectors indexed by y: is a one-hot vector and D; is a vector of predicted probabilities. A regularization term penalizing large weights is often included in the overall loss, though this term is not relevant to discussion in this section.
Generative Classifiers. Probabilistic generative classifiers take an indirect approach to modelling P(y[z). Namely, generative classifiers model the joint distribution P(z, y), factorized as P(x))) then use Bayes' rule for classification:
P(y P(x P(x))))(p)
P(x)
When classifying a single sample, the denominator P(x) is a constant and can be ignored. In the numerator, P(y) be modelled by simply counting the number of instances in given class. Therefore, the key challenge is modelling P(æly). This can be accomplished by creating a separate generative model for each class. These models can be very simple, such Gaussian Naive Bayes, which represents each feature's distribution for given class as a univariate Gaussian. Alternatively, more complex models such as variational autoencoders could be used to learn each class conditional distribution.
4.2 Naive Approaches
Approach #1: Single Discriminative Model for All Tasks. We start by discussing the use of a single, discriminativell-triiin model in the CIL, setting. For example, consider a neural network classifier trained with rops loss. Before learning the first. lask, the nelwork is instantiated with |Y neurons in its output layer, corresponding to the number of classes in the first task. The network is trained until convergence on the first task's data D,. Next, |22 additional neurons with randomly-inttializee weights are added the network's output layer prior to learning the second task. The network trained to convergence the second task's data D2, and the process repeats for the remaining tasks.
9

---

Published in Transactions on Machine Learning Research (January 22025)
Task
Task 2
Task I
Task
Task 1
Task 2
(a) Single Discriminative
(b) Task-Specific Discriminative
(c) Class-Specific Generative
Figure I: Illustration of naive approaches to CIL. Two classification tasks are shown: dog versus cat and apple versus orange, shown light red and light blue boxes respectively. Stylized neural network implementations for each approach are illustrated, with output node color corresponding task. Discriminative approaches include one output node for each class, used to prodict P(ylæ)). The generative approach includes three output nodes for each class, illustraling variational auloencoder which allempls to reconstruct the inpul. Importance sampling can then be used to predict P(æ|y) see the implementation described in Van De Ven et al. (2021) for details. Task sequence illustration inspired by Zhou et al. (2024). Best viewed in color.
This approach will almost certainly result in severe forgetting, as it does not include any mechanism to maintain knowledge learned on previous tasks. In other words, this approuch highly favors plasticily over stability. In addition, it is unlikely that across-task discrinénation will be learned, given that the model only performs discriminative learning within each task. Finally, portial knowledge transfer is realized. When learning task k + 1, the model is initialized with knowledge from task k, thereby allowing forward transfer between consecutive tasks.
The memory cost and inference compute required for a single discriminative model are both approximately constant. Here, we assume that the number of output layer weights is small relative to the model's total number of weights. The required training compute scales linearly with the number of tasks.9 9
Approach #2: Task-Specific Discriminative Models. Instead of using single discriminative model for all tasks, separate discriminative models could be used for each specific task. For example, a neural network with |Y1] output layer neurons could be trained on D1, then a separate, randomly-initialized network with Y2/ output layer neurons could be trained on D2, and so on.
When using this approach, both stability and plasticity are achieved an entirely new model is learned for each task (plasticity), and models do not interfere with another (stability). However, it is still unlikely that across-task discrimination is learned lypically inadequate for performing across-task discrimination. Lastly, knowledge transfer will not be realized, since specifpe models are completely separate and mandomly-initializeed.
previously discussed, learning within-task discrimination is
The memory, training compute, and inference compute all scale linearly with the number of tasks in this context, as new models must be allocated and trained for each task. As task identification is not provided at inference, all models must be run when classifying test data.
Generative models serve as alternative naive solution to CIT. For example, consider training separate variational autoencoder (VAF) to model the conditional probability distribution P(*]øk) for each class k. When learning the first task, [Y1] VAËs are each trained in parallel on class-specific subsets T,. When learning the second task, |Y₂] additional VAEs are each trained in parallel on class-specffic subsets of T2. This process repeats for all tasks.
Approach #3: Class-Specific Generative Models.
In this approach, stability and plasticity are both achieved as additional model(s) are allocated for each task. In addition, lasoss-lask discréménation is learned. This is because generative models are learned at the class- level rather than at the task-level, therefore class-to-task assignment is irrelevant. Still, knowledge transfer remains unrealized since class-specific models are learned independently.
Given that class-specific models are trained, the required memory, training compute, and inference compute all scale linearly with the number of classes. At inference, all models are run and Bayes' rule (more precisely, the numerator of Eq. 8) is used to make predictions.
In practice training compute may scale sublinearly with the number of tasks due forward knowledge transfer, c.g., learning tasks I,...,& may improve the convergence rate when learning task + 1. In general, without any assumptions on the task structure, training compute O(T).
10

---

Published in Transactions on Machine Learning Research (annary//022)
Interestingly, note that learning generative classifier in the CIL setting is essentially the same learning generative classifier in the non-continual setting, as discussed in Van De Ven et al. (2021). In other words, training a generative classifier on a set of classes in the CIL setting will result in the exact same model as training the classifier in the non-contimual setting. This because generative classifiers are trained at the class-level, and do not require simultaneous access to data from multiple classes order learn (unlike discriminative classifiers). As a result, the predictive performance of generative classifiers is unaffected by both class-to-task assignment and task order.
Table 2:
Key challenges addressed by naive (i.e., non-continual) approaches to CTL. A red x-mark indicates that a given challenge is unaddressed, while a green check mark indicates that a given challenge addressed. An asterisk indicates that a challenge is partially addressed.
Across-Taas Discrimination
Knowledge Transfer
Stability
Plasticity
Single
x
x
Discriminative Model
Task-Specith
x
Discriminative Models
Class-Specific
x
Generative Models
Table 3:
Memory and compute requirements for naive (i.e., non-continual) approaches to CIL. The total number of tasks to be learned is T, while the total number of classes to be learned is C. Note that C = 1.)
Training Compute
Inference
Memory
Compute
Single
0(1)
O(T)
0(1)
Discriminative Model
Task-Specific
((T)
O(T)
O(T)
Discriminative Models
Class-Specific
O(C)
O(C)
O(C)
Generative Models
5
Existing Work
The observations in Section suggest natural solutions to the CIL problem. Each of the three naive ap- proaches serve as a starting point when developing CIL solutions, with modifications necessary to address the key challenges described in Section 3 under the resource constraints described in Section 2. In this section, we provide a high-level, non-comprehensiee overview of existing work on CIL within this framing, covering the main idcas proposed in work published prior to September 2024. We describe the major categorics of approaches and outline the key design choices for each category. An illustration these categories shown in Figure? 2.
11

---

Published in Transactions on Machine Learning Research (annary//025))
5.1 Modifying Single Discriminative Model: Replay and Regularization
To succeed in the CIT setting, single discriminative model must. be supplemented with mechanisms to avoid forgetting (i.e., maintain stability) and learn across-task discrimination. The majority of work on the CIL sctting focuses on this direction (Zhou et al., 2024). Here, we discuss two categorics of approaches - replay and regularization - used lo supplement discriminative models in the CIL setting.
Replay. This set of approaches allows models to revisit previous task data when learning new tasks. A relatively small subset of previous task data is stored in a replay buffer. When new tasks arrive, the model is trained jointly on the new task data as well as the old replay data. Because the loss (Eq. 7) is optimized over samples from both the current task and previous tasks, forgetting is mitigated and across-task discrimination is learned. The data in the replay buffer is often referred to as class exemplars, and replay is sometimes called rehearsal." 10
In general, replay-based methods are designed with the goal of limiting the size of the replay buffer while maintaining strong performance on previous tasks. The design of such methods requires answers to three key questions: (1) Which samples should be stored? (2) How should these samples be stored? (3) How should these samples be replayed during training?
To address question (1), previous work has proposed the use of herding algorithms (Welling, 2009) to select "prototypical" samples (Rebuffi et al., 2017), maximizing sample diversity with respect to model gradi- ent updates (Aljundi et al., 2019b), and selecting difficul-to-classifyy samples based on prediction entropy (Chaudhry et al., 2018b).
Solutions proposed to address question (2) include storing learned representations (Iscen et al., 2020) or compressed features (Zhao et al., 2021) rather than raw features, or learning a generative model in order to create synthetic data for replay (Shin et al., 2017; Hu et al., 2018; Kemker & Kanan, 2018) (sometimes called generative replay or rseudo-rehearsal). While generative replay approaches require learning generative model, this model is used to generate data for a downstream discriminative model, hence their grouping with other discriminative models.
To answer question (3), most approaches (Rebuffi et al., 2017; Chaudhry et al., 2018b; Aljundi et al., 2019b; Zhao et al., 2021; Iscon et al., 2020; Shin et al., 2017: Hu et al., 2018; Kemker & Kanan, 2018) use a straightforward application of replay- for each lask, the model minimizes the loss over both the current task data and the previous task data from the replay buffer. Alternatively, some approaches (Lopez-Paz & Ranzato, 2017; Wang et al., 2021; Zeng et al., 2019; Tang et al., 2021) frame learning the current task as constrained optimization problem in which model parameters may only be updated such that the loss on the replay data does not increase (or increases only by a small amount).
Replay-based approaches have been observed to be biased towards the recently learned classes. This bias is due data imbalance, as there is typically much more data available in the current task as compared to the previous tasks. Several methods have been proposed to mitigate this bias. These methods include the addition of a simple rectification layer used to adjust predictions (Wu et al., 2019), making use of statistical information from previous classes for de- (Belouadah & Popescu, 2019), and compensating for bias at the gradient level during optimization (Guo et al., 2023).
Rogularization. This second set of tochniques attempts to retain previously Iearned knowledge using a regularization term in the loss function. This regularization term penalizes changes in model behavior with respect to old tasks. Note that this penalty is used to mitigate forgetting, though it is typically insufficient for learning across-task discrimination (Lesort et al., 2019). Regularization approaches fall into two subcategories: parameter-based and data-based regularization (De Lange et al., 2021).
Parameter-based regularization approaches attempt to retain parameter values which are important for strong performance on previously-learned tasks. These parameters should not change when learning new tasks, otherwise forgetting may occur. Intuitively, only the unimportant parameters for tasks 1,..., k may be modified when learning task + Many approaches in this subcategory use Fisher Information as an
More precisely, rehearsal refers methods which use replay data for retraining (Bagus Gepperth, 2021). Some methods instead use replay data to generate constraints while learning new tasks.
12

---

Published in Transactions on Machine Learning Research (annurry/022)
importance measure (Kirkpatrick et al., 2017; Lee et al., 2017; Chaudhry et al., 2018a; Yang al., 2019; 2023b). Some approaches compute parameter importance after tasks have been learned (Kirkpatrick et al., 2017) while others compute importance during learning (Zenke et al., 2017; Chaudhry et al., 2018a).
Data- based regularization methods typically leverage knowledge distillation to mitigate forgetting. Knowl- edge distillation traditionally used transfer knowledge from larger teacher model a smaller student model in an attempt replicate the teacher's performance using the student (Hinton al., 2015). In con- timual learning, the model(s) trained previous tasks serve the teacher, and the model trained the current task serves as the student. The student model attempts to learn the current task while replicating the performance of the teacher on previous tasks, accomplished via an additional distillation term in the loss function. Distillation may be performed at the level of the logits the final layer of the neural network (Li & Hoiem, 2017: Rebuff et al., 2017: Zhang et al., 2020b; Hou et al., 2018; Smith et al., 2021; Lee et al., 2019; Zhou al., 2021) or the level of the learned representation an intermediate layer of the network (Simon et al., 2021; Lu et al., 2022; Kang al., 2022; Hu et al., 2021b). Some approaches (Rebuffi et al., 2017; Zhao et al., 2020; Lee al., 2019) combine data-based regularization with replay to improve distillation performance using set of exemplars from previous tasks.
Similar to replay, regularization-bast approaches tend to be biased toward the current task data. In addi- tion, regularization-based approaches attempt to maintain separability across classes within each previously learned task, but are typically unable to learn separability across tasks. number of methods have been proposed in an effort to address these issues. (Zhao et al., 2020) and (Hou et al., 2019) ensure that weights have comparable magnitudes across both old and new classes achieve fair classification, (Ahn al., 2021) separates softmax activations in the output layer to avoid task-recency bias, and (Castro et al., 2018) uses balanced fine-tuning to stabilize predictions.
Teacker Netwark
Replay old task data
H
Behavior
Learn new task data
Task weights Task weights plastic
Student Network
(a) Data Replay
(b) Parameter Regularization
(c) Data Regularization
X
Task Tesk-specific subnetiour
Match in Intout space
Shared subnetwork
Templates
Task-specific aubactvork
(d) Parameter Isolation
(c) Template-Basod
Figure 2: Illustration of existing work on CIL. Each subfigure depicts a stylized neural network, representing the high-level idea behind a given category of approaches. "Template-based" approaches include generative classifiers and hybrid models. These approaches match input samples to learned templates class prototypes or class-conditional distributions in order to perform classification. Best viewed in color.
5.2 Modifying Task-Specific Discriminative Models: Parameter Isolation
Both stability and plasticity can be achieved by allocating separate discriminative models for each task. However, this approach lacks mechanisms to learn across-task discrimination and exploit knowledge transfer. In addition, learning completely separate models for each task may be inefficient with respect to memory and compute. Parameter isolation methods address these challenges.
13

---

Published in Transactions on Machine Learning Research (anuary/2025)
Parameter Isolation. This family of methods allocates sets of discriminatively-traaaeee parameters (i.e. submodels) specific to each task that may not be modified when learning other tasks. Parameter isolation is a simple way to avoid forgetting - if previously learned knowledge is never modified, then it will never be overwritten when learning continually. However, parameter isolation methods introduce a new problem: at inference, they require an additional lask-selector model used to predict the task label order to choose the relevant task-specific submodel. In other words, most parameter isolation approaches decompose the CIL problem into task prediction (TP) and within-task prediction (WP) (Kim et al., 2022). Approaches to TP include using model prediction entropy to infer task label (Wortsman et al., 2020), creating separate network as dedicated task predictor (Abati et al., 2020), and utilizing out-of-distribution detection models (Kim et al., 2022).
Parameter isolation methods can be divided into two subcategories fixed architecture and dynamic archi- tecture based on their approach to WP. Fixed architecture methods start with a fixod-size network and allocate task-spocific subnetworks during training. Approaches to subnetwork allocation include the use of weight masks (Serra et. al., 2018; Wortsman et al., 2020), genetic algorithms (Fernando el al., 2017), and pruning (Mallya & Lazebnik, 2018). Alternatively, dynamic architecture methods do not have fixed size, and expand as necessary in order to learn new tasks. These approaches are concerned with efficiently expand- ing model size as new tasks arrive in order to minimize memory cost while maintaining strong performance. Varions techniques have been proposed to accomplish this goal, including dynamically creating and deleting neurons (Yoon et al., 2018), framing the problem as reinforcement learning (Xu Zhu, 2018) or neural architecture search (Li al., 2019), using compression (Schwarz et al., 2018) or distillation (Wang et al., 2022a) to lower the model's memory footprint, and focusing expansion specific components to better transfer knowledge across tasks (Zhou et al., 2023). Interestingly, Kim & Han (2023) finds that many CIL methods heavily favor stability over plasticity, and proposes holding earlier layers stable while expanding later layers in order to strike a fairer balance between old and new tasks.
5.3 Modifying Class-Specific Generative Models: Generative Classifiers, Hybrids, and Pretraining
Recall that generalive classifiers are a natural fit. for CTL - they achieve both stability and plasticity, and learn across-task discrimination without requiring a task-prediction model. However, traditional generative classifiers suffer from poor memory and compute scaling and lack mechanisms to leverage knowledge transfer.
In addition, the predictive performance of generative classifiers is typically inferior to that of discriminative classifiers in the non-continual setting. This statement is supported by wealth of thcoretical and empirical evidence, see Vapnik (1998) and Ng & Jordan (2001). At high level, this is because generative classifiers solve a more difficult problem - intermediatell modelling P(xly) - while discriminative classifiers solve an easier problem directly modelling P(y x). Generative classifiers and hybrid generativediisrimnnatre models designed for CIL attempt to remedy these issucs through modifications which improve resource efficiency, leverage knowledge transfer, and boost predictive performance.
Generative Classifiers. In addition to the large body of work on discriminative classifiess for CIL, there exists smaller body of work focused generative classifiers. Van De Ven et al. (2021) proposes generative classifiers for the CIL setting and provides results with a class-specific variational autoencoders (VAFs) as a proof-of-concept. Other approaches also utilize VAËs (Ye & Bors, 2021a; 2023) as well as generative adver- sarial networks (GANs) (Ye & Bors, 2020; 2021b) but focus on ways to efficiently expand the architecture and exploit knowledge transfer between submodels.
Hybrid Models. Some approaches cannot be clcanly categorized as gencrative or discriminative. These approaches include protolype- based classifiers, one-class classifiers, and hybrid generative-dieriinnaaiee clas- sifiers. Many of these approaches retain the desirable properties of generative classifiers while avoiding the drawbacks of strictly discriminative classifiers.
Prolotype-based classifiers (Zhu el al., 2021b; De Lange & Tuylelaars, 2021) learn protolypical representa- tions for each class and use nearest neighbor matching for classification. Similar to Naive Bayes, prototype- basod classifiers are natural continual learners, as new prototypes can be learned as new classes are introduced (Grossberg, 2020; Ashtckar & Honavar, 2023). One-class classifiers (Hu et al., 2021a; Sun et al., 2023) learn independent models for each class. These models are optimized to predict high values for in-class data
14

---

Published in Transactions on Machine Learning Research (anuary/2025)
similar to conditional generative models though one-class classifiers use different objective functions (Hu et al., 2020).
Hybrid iineraaivedissrinnnatie classifiers (Kirichenko et al., 2021), combine the lack-of-forgetting of gen- erative models with the strong classification ability of discriminative models. Generative techniques can be used to model features "holistically" (Hu et al., 2021a), while discriminative techniques can be used to enhance separation between classes. More generally, prospective modeting (Tian al., 2024) may be neces- sary to address the across-task discrimination challenge: when learning class it is unclear which features will be discriminative against future class j. Generative of eseudo-generatee techniques model both the features which are discriminative across the previously learned classes and the features which may be discriminative with respect to future classes. Other approaches in this direction include allocating "virtual" or "reserved" space in the learned representation as a praxy for future classes (Zhou et al., 2022a; Song et al., 2023; Zhou et al., 2022b).
be used to
Pretraining. Another calegory of approaches avoids continual representation learning through the use of pretrained models. Wang et al. (2022d) and Wang et al. (2022c) introduce the idea of learning to prompt for continual learning. These approaches learn to instruct pretrained transformers to perform tasks in the continual setting. Other approaches learn simple generative classifiers on features produced by fixed pretrained networks (Yang et al., 2023a; McDonnell et al., 2023). For example, Yang et al. (2023a) learns a Naive Bayes classifier atop features generated by a network trained on ImageNet in order to perform downstream image classification tasks.
Pretraining is particularly relevant in the few-shot setting (FSCIT.), in which only small amount of data is available when learning cach class. Many approaches to FSCIL focus learning in the later network layers with a fixed, pretrained represenlation. Shi et al. (2021) observes that FSCIL performance improves when starting from a pretrained network which has converged to a flat local minima in the loss landscape. In a flat local minima, small parameter changes will leave performance relatively unaffected, thus mitigating forgetting while learning new classes. Other approaches to FSCIL utilize meta-learning, or "learning how to learn", typically accomplished through bi-level optimization. Chi al. (2022) extends the classic model- agnostic meta-learning (MAML) algorithm first proposed in Finn al. (2017) for the FSCIL setting.
6 Dimensions of the Problem
Given the abundance of work on CIL, it is natural to ask when each approach should be used. We argue that the appropriate choice of approach depends on the specific dimensions of given CIL problem. In this section, we outline five such dimensions: resource construints, task size, task similarity, task order, and class-to-task assigament. We explain each dimension, describe how it influences the difficulty of the CIL problem, and discuss how it addressed by existing approaches.
Resource Constraints. Resource constraints are arguably the most important dimension of the CIL problem: constraints on available memory and compute define the set of potential solutions. Generally speaking, stricter resource constraints make the problem harder, while looser constraints make the problem easier. For example, given no resource constraints, CIL can be trivially solved by retraining on all learned data D1:k at every task k. This approach matches the predictive performance in the non-continual setting and renders continual learning methods unnecessary. On the other hand, if an application requires tight memory /nd/or compute budgets, then more care required design bespoke continual lcarning solutions. These solutions should be designed with the goal of approaching and ideally, matching the predictive performance of model learned in the non-continual setting in which data from all the tasks is available.
Recent work indicates that replay may be the most promising deep learning solution to CIL under strict resource constraints. Specifically, replay-based approaches have been shown to outperform regularization- based and parameter isolation- based approaches when given comparable budgets for memory (Zhou et al., 2024) and compute (Prabhu et al., 2023a). Notably, Harun et al. (2023b) proposes SIESTA a wake-sloop algorithm utilizing replay which matches the performance of non-contimual learner on the ImageNet- 1K dataset under tight. compute budget. Other promising approaches for resource-efficient CIT. with neural
15

---

Published in Transactions on Machine Learning Research (anuary/2025)
networks include pretraining (McDonnell et al., 2023), layer-specific plasticity (Zhou et al., 2023), dynamic miggh//gradient-nnsiing (Wang et al., 2022b), and sharpness-aware minimization (Ren & Honavar, 2024).
Task Size. Performing CIL with larger tasks (tasks containing more classes) is typically easier than per- forming CIL with smaller tasks. More precisely: for a chosen dataset, CIL methods tend to perform better given a few large tasks as compared to many small tasks see Masana et al. (2020) and Zhou et al. (2024) for empirical evidence on the split CIFAR-100 and ImageNet-IK datasets. Consider the extreme cases: (i) there is only one task containing all classes and (ii) each task contains single class. Case (i) is equivalent to the non-continual setting, so continual learning is unnecessary. Case (ii) is a version of CIL which does not include within-task discrimination, as there is only one class-per-task. In other words, the CIL problem in case (ii) is entirely across-task discrimination. For existing methods, learning across- Lask discrimination tends to be more challenging than learning within- task discrimination (Soutif-Cormeray Cormerais et al., 2021; Guo et al., 2023), therefore case (ii) can be categorized as the "most difficult" version of CTL.
To understand why, consider the following. Recall that discriminative classifiers require simultaneous access to data from all classes in order to effectively learn, and that discriminative classifiers typically outperform generative classifiers. As described in Section 5, across-task discrimination can be addressed in one of two ways: with a discriminative classifier supplemented by replay, or with generative seaattive/ssuuooggee soruuddoggnetttvee clas- sifier. Resource constraints restrict the amount of old data stored and/or the number of learning updates completed during replay, thus limiting the performance of discriminative approaches. On the other hand, generative/pseudl peee:/sseooooooatiiiee techniques offer lower performance than their discriminative counterparts. Ei- ther way, we would expect the performance on across-tast -asoss-taskaaa discrimination to be lower than the performance within-task discrimination.
Task Similarity. There are various ways to define similarity between classification lasks. These definitions can be categorized as either model-dependent - based on a particularly model architecture or training process - or model-agnostic- - independent of the type of model or training procedure used. Model-dependent similarity can be defined using knowledge transfer (see Eq. and Eq. 6), learned parameter values (Lee et al., 2021), or the distance between pretrained task embeddings (Achille et al., 2019). Model-agnostic similarity can be defined using optimal transport (Alvarez- Melis & Fusi, 2020; Liu et al., 2025) and/or conditional entropy (Tran et al., 2019; Tan et al., 2021).
CIT, approaches such as replay exploit task similarity through parameter initialization: when learning task k the model is initialized with the knowledge from tasks 1,..., 1. This can help reduce the number of training iterations when learning task k thus improving computational efficiency. Approaches such as replay, regularization, partial parameter isolation, and generative modelling exploit task similarity through parameter sharing. Sharing a subset of parameters across tasks can reduce computational and memory costs. In addition, both parameter initialization and parameter sharing have the potential to improve predictive performance, evidenced by the large body of work on transfer learning (Zhuang et al., 2020).
While task similarity can be exploited to improve learning, attempting to learn similar tasks presents ad- ditional challenges. Ramasesh et al. (2021) and Nguyen et al. (2019) empirically show that increasing task similarity often increases the severity of catastrophic forgetting. Through theoretical analyses of continual linear regression, Evron et al. (2022) and Lin et al. (2023) find that forgetting is greatest when learning tasks with intermediate similarity. Namely, regression tasks with common features but misaligned weights" are the most prone to forgelting. Designing CIL. solutions which leverage task similarity without int introducing across-task interference remains an interesting problem.
Task Order. The performance of most CIL methods depends on the order in which tasks are presented. For example, Masana et al. (2020) evaluate several popular CIT, methods across various class (and therefore task) orderings of the CIFAR-100 dataset and report wide variations in performance. Further, the method which performs best also depends on the ordering! This indicates the importance of evaluating CIL methods over multiple task orderings and class-to-tasl assignments.
11 Formally, let the optimal linear regression weights for task and task be unj and mz.. The "misaligned weights" condition corresponds to (mi. un)) < 0, as described in Lin et al. (2023)
16

---

Published in Transactions on Machine Learning Research (January (lamury/20255)
(a) Task 1: red/blue
(b) 'Task 2: green/yellow
(c) Task red/yellow
(d) Task 2: green /blue
Figure 3: Example illustrating the importance of class-to-task assignment. For given set of classes, (a) and (b) illustrate one class-to-task assignment, while (c) and (d) illustrate another. Data samples are represented shaded circles, with color indicating class label. Learned decision boundaries are represented with black dotted lines. Figure inspired by Lesort et al. (2019). Best viewed in color.
Curriculum learning is a subfield of machine learning which attempts to build training schedules in order to improve learning efficiency and performance (Bengio et al., 2009). While continual learning is concerned with efficiently learning
of sequential tasks, curriculum learning instead concerned with designing task sequence which can be efficiently learned. For given dataset, curriculum is ordering of the dataset's samples to be learned during training. Curricula include learning "casy" samples before "hard" samples, learning "hard" samples before "easy" samples (anti-curriculum), and emphasizing sample diversily early in training (Xin Wang, 2022). Note that curricula are typically designed at the sample-level, though Pentina et al. (2015) extends these idcas to the task-level.
Most work on CTT assumes that task order fixed (i.e., not chosen by the model) and that models have no knowledge of future tasks prior to learning. Several papers (Ruvolo & Eaton, 2013; Yang & Li, 2021; Bell & Lawrence, 2022; Mantione-Holmes al., 2023) relax this assumption and attempt to find the task/class order which maximizes overall classification performance. Bell & Lawrence (2022) propose ordering tasks such that the distance between optimal task-specific parameters is small. However, their results are counterintuitive performance on computer vision benchmarks is strongest when learning tasks in an order which results in large parameter changes. Mantione- Mlantione-Holmes et al. (2023) report similar results. Inspired by psychology, they find that an interleaved task order (learning dissimilar tasks adjacently) improves performance on NLP classification problems. Lin al. (2023) provide evidence supporting this observation through theoretical analysis of linear models.
When tasks are interleaved, the model is exposed to diverse set of classes all throughout training. This may better approximate the non-continual setting in which data from all classes is available, thus explaining the relatively strong performance on interleaved orderings. Interestingly, Yang & Li (2021) come to a different conclusion: learning similar tasks adjacently results in the strongest overall performance on image classification tasks. These findings may be application-sppeciie, or the differences may be reconciled by considering the various experimental setups and definitions of task similarity used across studies.
Class-to-Task Assignment. The way in which classes are assigned to tasks affects the performance of most CIL methods. Klasson et al. (2023) provides empirical evidence of this phenomenon across various replay- based methods on the split MNIST series and CIFAR-10 10 datasets with randomized class-to-tast to-k assignments. As with task order, this dimension of the problem has been observed to affect CIL performance, though this effect is not well understood.
Figure illustrates the importance of tlass-t--tssk assignment on a simple CIL problem with linear classifiers learned on each task. Here, we given set of four classes, and we consider two possible class-to-task assignments. The first assignment shown in subfigures 3a and 3b results in roughly the same within- task decision boundaries for both lasks, though across-task discrimination must be learned separately. The second assignment shown in subfigures 3c and 3d results in nearly orthogonal within-task decision boundarics, and across-task discrimination is implicitly learned. In other words, the CIL problem in the second assignment can solved by task-specific discriminative models alone. 13
12Curiicula can also be defined using instance selection sample reweighting Xin Wang (2022) for details For example, two separate logistic regression models for Task and for Task 2 could used, with predictions made based on the maximum logit activation across both models
17

---

Published in Transactions on Machine Learning Research (anuurry2025)
We hypothesize that the performance of CIL models will be stronger when classes are assigned to tasks such that the similarity of classes within-task is greater than the similarity of classes across-lask. Here, class similarity can be quantified using the performance of a given classifier for a set of classes, if classifier performance is low, then the classes can be said to be similar, and if classifier performance is high, then the classes can be said to be dissimilar. In other words, we hypothesize that CIL performance will be higher if the across-task discrimination subproblem is casier relative to the within-task discrimination subproblem. We expect this hypothesis to be true given that across-task discrimination is a larger part of the overall CIL problem compared to within-task discrimination, as described in Section 3. It may be reasonable to expect this condition to hold in real-world applications. This is because new tasks are likely to result from focused data collection and/or from temporally correlated changes in the world.
7 Conclusion and Future Work
At this point, we take a step back and reflect on our progress. We began by reexamining the CIL problem definition, arguing that common framings neglect important considerations such as computational cost. Next, we explored the factors which make CIL unique and decomposed the problem into three key challenges. With these challenges in mind, we analyzed non-continual approaches to the problem. The shortcomings of these approaches served to motivate and contextualize existing work on CTL. After summarizing existing work, we investigated various dimensions of the problem in order to better understand specific problem configurations and their corresponding solutions. Now, we look ahead and suggest polential directions for future work.
7.1 Future Work
Developing computationally efficient solutions. In Section 2, we discuss why the CTL problem should be defined based on resource constraints rather than data availability. Note that the arguments supporting the resource-constrained problem statement extend beyond the CIL setting. We argue that other continual learning settings such as task incremental learning (TIL), domain incremental learning (DIL), continual reinforcement learning, etc. should also be defined with respect to resource constraints.
As analyzed in Verwimp el al. (2024), the majority of continual learning papers at recent * 14 highly constrain memory, but do not constrain computational cost. These constraints may be counterproductive, as "memory is cheap, but compute expensive" nowadays given contemporary hardware and cloud computing platforms (Lomonaco & Carta, 2023). Instead, it may be more appropriate to constrain or optimize for low computational costs under large (or even unlimited) memory budgets.
The discussion in Section 2.4 on a potential real-world CIL system provides preliminary ideas in this di- rection: matching-basee or other locally-weighted classifiers could be learned atop a continually evolving representation, updated via replay and/or knowledge distillation. While continual representation learning has been well-studied under the "unavailability of previous data" problem statement (Section 2.2.1), rel- alively little work has addressed this problem in the context of computational efficiency beyond Li et al. (2022); Harun et al. (2023b), and Prabhu et al. (2023a).
Making assumptions regarding the data-genorating process. Most work on CIL assumes that models have no knowledge of future lasks prior to learning, as ment ioned in Section 6. We argue that this assumption is unnecessarily restrictive and unlikely to hold in practice. To understand why, consider the manifold hypothesis, which states that roal-world, high-dimensional datascts tend to be concentrated along low- dimensional manifolds (Narayanan & Mitter, 2010). For example, real-world images tend to be highly structured, and are unlikely to resemble random collection of pixels. In application of CIT for image classification, all classes are likely to share some common structure. If this structure is known apriori, learning may be made casier." 15
14BCCV '22 CVPR '23 NeurIPS '22, and ICML '23
18 Iutcreestngg many CIL approaches for image classification do make use of this common structure for example, by sharing convolutional layers across tasks. However, the assumption that classes or tasks have a common structure is often not explicitly stated or formalized.
18

---

Published in Transactions on Machine Learning Research (anuary/2025)
As another example, consider continual learning application leveraging physics-informed neural networks (Howard et al., 2024). The laws of physics remain constant as new tasks are presented (this fact is known before any tasks are learned). In this example, one aspect of the data-generating process is fixed, while other aspects may vary. For a given application, it may be possible to explicitly spocify which aspects of the dala-gengg process are changing. Such specifications could be used to build better inductive biases, thus improving learning efficiency and performance. Doing so may also circumvent some of the negative results regarding the difficulty of continual learning (Knoblauch et al., 2020).
Science versus engineering in continual learning. Looking back at the past eight years of work continual learning, we observe the following high-level trend. Early work (2016 2019) introduced new categories of approaches to the problem: Elastic Weight Consolidation (Kirkpatrick et al., 2017) introduced parameter regularization, Learning without Forgetting (Li & Hoiem, 2017) introducedd knowledge distillation, Progressive Neural Networks (Rusu et al., 2016) introduced parameter isolation, and so on. More recent work has focusod on refining the best-performing solutions and understanding why they work. This trend is analogous to the exploration-eexplotaaiioo trade-off in reinforcement learning early work explored the solution space, while recent work exploits the most successful solutions.
Work on continual learning can be very loosely divided into two categories: science and engineering. Scien- tifie work includes proposing novel algorithms (often inspired by neuroscience), understanding why various techniques succeed or fail, and proving theoretical results regarding complexity, performance, convergence, etc. Engineering work is concerned with developing efficient and effective solutions for specific applications. While there is a great need for further engincering advances in continual learning, there is an equal, if not greater, need for elucidating the theoretical underpinnings of continual learning. Future engineering work will likely focus specific, real-world applications, moving beyond the current general-purpose, somewhat contrived benchmarks such as the split versions of the MNIST, CIFAR-100, and ImageNet-11K 1K datasets. We expect that improvements in practical continual learning applications will lead to an increased adoption of continual learning in industry, further driving progress.
7.2 Actionable Insights
We conclude with several high-level actionable insights for future CIL research.
1. Address the resource-constrainet setting. As discussed in Section 2, framing CIL (and con- timual learning more generally) based on the unavallability of previous data may lead to undesirable outcomes. This framing is imprecise, may not adequately address security or privacy concerns, and permits inefficient solutions. We argue that the resource-constrained setting does not suffer from these drawbacks and is more appropriate for real-world applications, particularly with its inclusion of compute constraints.
2. Ensure that plasticity is available for future tasks. Much of the work on continual learning focuses on avoiding catastrophic forgetting (i.e,, maintaining stability). However, the focus on avoiding forgetting often neglects plasticity, limiting performance on future tasks.
3. Across-task discrimination dominates the CIL problem - build solutions accordingly. At inference, the notion of separate tasks in CIL is somewhat artificial: the one "true" task is to ac- curately make predictions across all previously learned classes. Therefore, across-task discrimination is almost always a larger part of the problem as compared to within-task discrimination.
4. Dimensions of the CIL problem are poorly understood and ripe for future work. Open questions remain concerning the impact of task similarity, class/lask order, and class-to-Lask assign- ment on CIL performance. Answering these questions could lead to better CIL algorithms as well as a deeper understanding of the broader continual learning problem.
Use (rcasonable) assumptions about your data to make learning casier. It may be possible to make assumptions regarding the data generating process in specific continual learning applications (i.e., physics-informed informed learning). Doing so could improve learning efficiency and performance.
19

---

Published in Transactions on Machine Learning Research (January
Acknowledgments
This work was supported in part by grants from the National Science Foundation (2226025) and the Na- tional Center for Advancing Translational Sciences, and the National Institutes of Health (UL1 TR002014) and by the Center for Artificial Intelligence Foundations and Scientific Applications and the Institute for Computational and Data Sciences at Pennsylvania State University.
References
Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehte- shami Bejuordi. Conditional channel gated networks for lask-aware continual learning. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp.
2020.
A Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C Fowlkes, Stefano Soatto, and Pietro Perona. Task2vec: Task embedding for meta-learning, In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6430-6439, 2019.
Hongjoon Ahn, Jihwan Kwak, Subin Lim, Hyeonsu Bang, Hyojun Kim, and Taesup Moon. Ss-il: Separated softmax for incremental learning. In Proceedings of the IEEE/CVF International conference on computer vision, pp. 844 853, 2021.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Proceedings of the IEEE/CVE Conference on Compuler Vision and Pallern Recognilion, pp. 1224-11126,, 2019a.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample lection for online continual learning. In Advances in Neural Information Processing Sys- tems, volume 32, 2019b. URL pff.pe.pff....
neurip.. wttpppppp.rppppppppppppppppe..ep wwtpppper.illle/ppeeeeeeeeeeeeee filees/papee//211//iee
4B22c3o07aa44466cfccafffftb Papeee
David Alvarez-Melis and Nicolo Fusi. Geometric dataset distances via optimal transport. Advances in Newral Information Processing Systems, 18:21428-21433 2020.
Neil Ashtekar and Vasant G Honavar. A simple, fast algorithm for continual learning from high-dimensional data. In Eleventh International Conference Learning Representations. Openreview. net, 2023. URL
https:// / net net//forum?id-TPPDiieeEE
Benedikt Bagus and Alexander Gepperth. An investigation of replay-based approaches for continual learning. In 2021 International Joint Conference on Neural Networks (IJCNN), pp. 9, 2021. doi: 10.1109/
Samuel J Bell and Neil D Lawrence. The effect of task ordering in continual lcarning. Xi preprint arXiv:2205. 13323, 2022. URL https://doi. org/10.485500 00.4448550//00 arxiv 2205. 13323.
Eden Belouadah and Adrian Popescu. I12m: Class incremental lcarning with dual memory. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 583-592, 2019.
Eden Belouadah, Adrian Popescu, and Ioannis Kanellos. A comprehensive study of class incremental learn-
ing algorithms for visual tasks. Neural Networks, 135:38 54, 2021. ISSN 0893-6080. doi: https://
www.w/edeeeccerre.......... hctp://///cccccccarrrrcee///p/p URL http:://ww..cceeenedrrrttt.tto.e
$08936008020304202.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41 48, 2009.
Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, and Manfred K Warmuth. Occam's razor. Informa- lion processing lellers, 24(6):377-388 1987.
Gail A. Carpenter and Stephen Grossberg. The art of adaptive pattern recognition by self-organizing neural network. Computer, 21 (3):777 88,, 1988.
20

---

Published in Transactions on Machine Learning Research (anuary//023)
Francisco M Castro, Manuel J Marin-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteck Alahari. End- to-end incremental learning. In Proceedings of the European conference computer vision (ECCV), pp. 233 248, 2018.
Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philiip H. S. Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Computer Vision ECCV 2018, pp. 556 572. Springer International Publishing, 2018a. doi: 10. 1111077/778--000--1222 https://doi.co org/ 10.
33. URL
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference computer vision (ECCV), pp. 532 547, 2018b.
Zhixiang Chi, Li Gu, Huan Liu, Yang Wang. Yuanhao Yu, and Jin Tang. Metafscil: A meta-learning approach for few-shot class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 14175,, 2022.
Matthias De Lange and Tinee Tuytelaars. Continual prototype evolution: Learning online from non- stationary data streams. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8250-8259, 2021.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on poltern analysis and machine intelligence, 44(7):3366 3385, 2021.
Natalia Dodr-godr,,,, Vincenzo Lomonaco, David Filliat, and Davide Maltoni. Don't forget, there is more than forgetting: new metrics for continual learning. In Workshop on Continual Learning, NewrIPS 2018
(Neural Information Processing Systems, 2018.
Itay Evron, Edward Moroshko, Rachel Ward, Nathan Srebro, and Daniel Soudry. How catastrophic catastrophic forgetting be in linear regression? In Po-Ling Loh and Maxim Raginsky (eds.), Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pp. 1928-4074 PMLR, 02-05 Jul 2022. URL
mlr. ppess/vv72 v///v///v//e/222ee evron222. html.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks, 2017.
Chelsea Finn, Picter Abbeel, and Sergey Levine. Model-agnostic meta-lcarning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135. PMLR, 2017.
Stephen Grossberg. A path toward explainable ai and autonomous adaptive intelligence: Deep learning, adaptive resonance, and models of perception, emotion, and action. Frontiers in Neurorobotics, 14, 2020. TSSN 1662-5218. doi: 1.3389/fnbot.2000 2020.00033 URT. http:://www.ffooo/ www.ttiessi..or//rrttccc... 3389/fnbot. 2tw.:/b.t..2...0000000
Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task class discrimination in online continual learning. In Proceedings of the IEEE/CC /CVF Conference on Computer Vision and Pattern Recognition, pp.
11878 11778-1187, 2023.
Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. Reconstructing training data from trained neural networks. Advances in Neural Information Processing Systems, 35:22911-22922 2022.
Md Yousuf Harun, Jhair Gallardo, Tyler Hayes, and Christopher Kanan. How efficient are today's continual learning algorithms? Tn Proceedings of the TEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2430 2435, 2023a.
Md Yousuf Harun, Jhair Gallardo, Tyler L. Hayes, Ronald Kemker, and Christopher Kanan. SIESTA: Efficient online continual learning with sleep. Thansactions on Machine Learning Research, 2023b. ISSN 2835-8856. URL https: https:///ppnreeiee: http:///pmeeeeeeeeeeeeee/aaa/aaa http://e/eeeo/mmm////////eee
21

---

Published in Transactions on Machine Learning Research (anuary/2025)
Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. ArXiv, abs/1503.02531 2015. URL https://api. www.nti sennattteccooar....ororooooo
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Lifelong learning via progressive distillation and retrospection. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 437-452, 2018.
Saihui Hou, Xinyu Pan; Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incre- mentally via rebalancing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 831-839, 2019.
Amanda Howard, Yucheng Fu, and Panos Stinis. A multifidelity approach continual learning for physical systems. Machine Learning: Science and Technology, 5(2):02504,, may 2024. doi: 10.1088/2632-2 ad45b2. URL https::/ http::://xx..o.oorgg..... 1001...0/001.1100/2222-2-1-35350
Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting for continual learning via model adaptation. In International conference on learning representations, 2018.
Wempeng Hu, Mengyu Wang, Qi Qin, Jinwen Ma, and Bing Liu. Hrn: A holistic approach to one class learning. Advances in newral information processing systems, 33:19111 191244, 2020.
Wenpeng Hu, Qi Qin, Mengyu Wang, Jinwen Ma, and Bing Liu. Continual learning by using information each class holistically, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7797-7805, 2021a.
Xinting Hu, Kaihua Tang, Chunyan Miao, ShangS Hua, and Hanwang Zhang, Distilling causal effect of data in idaremnncrellttal learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 3957-3966, 2021b.
Bingchen Huang, Zhineng Chen, Peng Zhou, Jiayin Chen, and Zuxuan Wu. Resolving task confusion in dynamic expansion architectures for class incremental learning. Proceedings of the AAAI Conference Artificial Intelligence, 37(1):908 916, Jun. 2023. doi: org/index. index.dexxx. Bandex.ppp/AAiee http://pp//aaaarrrrrre/e//e////e
URL https://ojs.ant
Ahmet Iscen, Jeffrey Zhang, Svetlana Lazebnik, and Cordelia Schmid. Memory-efficient incremental learning through feature adaptation. In Computer Vision -icoon-CCVV-C 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVT 16, pp. 699-715. Springer, 2020.
Minsoo Kang, Jacyoo Park, and Bohyung Han. Class-incremental learning by knowledge distillation with adaptive feature consolidation. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 16071-16088 2022.
Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. In Inter- national Conference on Learning Representations, 2018. URL https://arxxv.orr/abb////t
Dongwan Kim and Bohyung Han. On the stability-plasticity dilemmma of class-incremnntaal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20196 20204, June 2023.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. A theoretical study on solving continual learning. Advances in Newral Information Processing Systems, 35:5065-50777 2022.
Polina Kirichenko, Mehrdad Farajtabar, Dushyant Rao, Balaji Lakshminarayanan, Nir Levine, Ang Li, Huiyi Hu, Andrew Gordon Wilson, and Razvan Pascanu. Task-agnostic continual lcarning with hybrid probabilistic models. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021. URL http:////ppppeeeeeeeeee////rraaa
22

---

Published in Transactions on Machine Learning Research (anuary//022)
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Demis Hassabis, Claudia Kieran Milan, John Quan, Tiago Ramalho, Agnieszka catastrophic forgetting in neural networks. Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming
Proceedings of the National Academy of Sciences, (14(13):3521-3222 2017. doi: 10.1073 10..07///a..111111 1pa...611835114
URL https: http://ww.ppaat. ppaas.or//aaie..o...
pna.s 1armaas.6111311111 1611835114.
Marcus Klasson, Hedvig Kjellstrom, and Cheng Zhang. Learn the time to learn: Replay scheduling in continual learning. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https:
//openreview.oo (.lpparrriee..nt//fouunitiiiennn net//foruumiddd/aaaATTggg
Jeremias Knoblauch, Hisham Husain, and Tom Diethe. Optimal continual learning has perfect memory and is np-hard. In International Conference on Machine Learning, pp. 5327-5337. PMLR, 2020.
Saurabh Kumar, Henrik Marklund, Ashish Rao, Yifan Zhu, Hong Jun Jeon, Yueyang Liu, and Benjamin Van Roy. Continual Learning as Computationally Constrained Reinforcement Learning. arXiv e-prints, art 2i3: .04345, July 2023. doi: 10.48550/az2 /tt//arXivvv//// 2307.04345.
Kibok Lec, Kimin Lee, Jinwoo Shin, and Honglak Lec. Overcoming catastrophic forgetting with unlabeled data in the wild. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 312-321, 2019.
Sang-Woo Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung-Tak Zhang. Overcoming calastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
Sebastian Lee, Sebastian Goldt, and Andrew Saxe. Continual learning in the teacher-student setup: Impact of task similarity. In International Conference on Machine Learning, pp. 6109-6119. PMLR, 2021.
Timothée Lesort, Andrei Stoian, and David Filliat. Regularization shortoomings for continual learning. ArXie, abs/1912.03049. 2019. URL http:://api..eentiicccoooo.... httpaaaiiiicitoooooooooorrraaaaa orgg/CorpusID::2888733.
Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International Conference Machine Learning, pp. 3925-300 3934. PMLR, 2019.
Yingcong Li, Mingchen Li, M. Salman Asif, and Samet Oymak. Provable and efficient continual repre- sentation learning. ArXiv, abs/2203.02026, 2022. URL https://api. semanticscholar htt.:/acccolaaaoorrorrrorppsppee 247244826.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935- 2944299777777 2017.
Sen Lin, Peizhong Ju, Yingbin Liang, and Ness ShrofT. Theory on forgetting and generalization ion of continual learning. In International Conference on Machine Learning, pp. 21078 21100. PMLR, 2023.
Xinran Liu, Yikun Bai, Yuzhe Lu, Andrea Soltoggio, and Soheil Kolouri. Wasserstein task embedding for measuring task similarities. Neural Networks, 181:106796, 2025. ISSN 0893-6080. doi: https:// 10t./00....//...onnc.... 202.119..02..166999 URL $0893608024007202.
doi.org 10. doi..org/1..1011
Vincenzo Lomonaco and Antonio Carta. Continual learning beyond catastrophic forgetting in class- incremental learning scenarios. Conference on Lifelong Learning Agents (CoLLAs), 2023. URL https:
//www. www..slideshare
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017. URL uttpttttss//pprreettiii wewriiiinnnnnneuis...... http//pprffflleee
Yichen Lu, Mei Wang, and Weihong Deng. Augmented geometric distillation for data-free incremental person reid. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7329 0339-733, 2022.
23

---

Published in Transactions on Machine Learning Research (anuary/2025)
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Samner. Online continual learning in image classification: An empirical survey. Neurocompulingg 469:28-5) 51, 2022. ISSN 0925-2312. doi:
nww.mm. aTticCee/pa//009253311111999
newcom.0021.100002.. 20.1.10.021. URL https: http:://ww..cieenedirrec.. com/science/
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks single network by iterative pruming, In Proceedings of the IEEE conference on Computer Vision and Pallern Recognation, pp- 7765 (775--773, 2018.
Gabriel Mantione- Mantione-Holmes, Justin Leo, and Jugal Kalita. Utilizing Priming to Identify Optimal Class Or- dering to Alleviate Catastrophic Forgetting. In 2023 TEEE 17th International Conference on Semantic Computing (ICSC), pp. 37 64, Los Alamitos, CA, USA, February 2023. IEEE Computer Society, doi: URL https://doi. wew.conpttreosocccoc....... www.wo..go......o 11w.1//10//II11......
1011099ICSC51113.20000 2023.00015
Marc. Masana, Bartlomiej Twardowski, and Joost van de Weijer. On class orderings for incremental learning. ArXio, abs/2007.02144 2020. URL https://api. www.stiimmcoolleeee........ http::::app..emmatticccooooo
org/ heg/CorpusIDD... : 220363521.
Marc Masana, Xialei Liu, Bartlomicj Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Wei- jer. Class-incremental learning: survey and performance evaluation on image classification. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 45(5):5513-5555 2022.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Mark D. McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. Ranpac: Random projections and pre-trained models for continual lcarning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 12022-12053 Curran Associates, Inc., 2023. URL
mewrreedddnnnnnnnnn..ereo.e...se cc cappaper__pppp files/ pdf.
Nicolas Michel, Giovanni Chicrchia, Romain Negrel, Jean-François Bercher, and Toshihiko Yamasaki. New metrics for analyzing continual learners. MIRU2023 Symposium on Image Recognition und Undersland- ing, 2023. doi: 10.48550/ 1088555//AXIV 220.0.006....
Martin Mundt, Yongwon Hong, Tuliia Pliushch, and Visvanathan Ramesh. A wholistic view of continual lcarning with deop neural networks: Forgotten Icssons and the bridge to active and open world learning, March 2023. ISSN 0893-6080. URL http:://dooi..ggg. 1016/j neunet. 2023.01.2.... 01.
Hariharan Narayanan and Sanjoy Mitter. Sample complexity of testing the manifold hypothesis. In Proceed- ings of the 23rd International Conference on Neural Information Processing Systems Volume 2, NIPS'10, pp. 1786 1794, Red Hook, NY, USA, 2010. Curran Associates Inc.
Andrew Ng and Michael Jordan. On discriminative vs. generative classifiers: A comparison logistic regression and naive bayes. In T. Dictterich, S. Becker, and Z. Ghahramani (eds.), Advances in Neural Informalion Processing Systems, volume 14. MIT Press, 2001. URL htttps:/ nww.ips..erripp.cc
paper http:///i/ggooooooo////////////a
pdf.
Cuong V. Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, and Stefano Soatto. Toward understanding catastrophic forgetting in continual lcarning, 2019. URL https://doi.
48550/arXiv. 1908.01091.
Milad Khademi Nori and Il Min Kim. Task confusion and catastrophic forgetting in learning: A mathematical framework for discriminative and generative modelings. Tn The Thirly-eighth Annual Conference on Neural Information Processing Systems, 2024. URL forum?id=Tj6wwslljjn
Anastasia Pentina, Viktoriia Sharmanska, and Christoph Lampert. Curriculum learning of multiple tasks. In Proceedings of the IEFE conference on computer vision and poltern recognition, pp. 2015.
24

---

Published in Transactions on Machine Learning Research (anuary/2025)
Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puncet K Dokania, Philip HS Torr, Ser-Nam Lim, Bernard Ghanem, and Adel Bibi. Computationally budgeted continual learning: What does matter? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3698 3707, 2023a.
Ameya Prabhu, Zhipeng Cai, Puneet Dokania, Philip Torr, Vladlen Koltun, and Ozan Sener. Online contin- ual learning without the storage constraint, 2023b. URL https:://oi..org/////// 48550//rXiv.. /2855/0000000.....///////111 2305. 09253.
Vinay Venkatesh Ramasesh, Ethan Dyer, and Maithra Raghu. Anatomy of catastrophic forgetting: Hidden representations ions and task semantics. In International Conference on Learning Represenlations, 2021. URL http:://ppppereeeeeeeeeeeeororre wtt.://ww.....oorro/oooooaooooaa id=LhYequUSSu..
https://openreviier net/forum?
Sylvestre Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pallern Recognilion, pp. 2001- 1001-2010, 2010, 2017.
Weijieying Ren and Vasant G Honavar. Esacl: An efficient continual learning algorithm. In Proceedings of the 2024 SIAM International Conference on Data Mining (SDM), pp. 163 171, 2024. doi: 10.1137/1. 9781611978032.19, URL https://eppie http::////pbbbaa........ siam. wwt../o//////b////a/eee/ 11133//1..9711119999222.281
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experi- ence replay for continual learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché
E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL https:
cepppppccppp....pppprrrr htt..c/ppeee..lleeppppeeeeeeeeee fitls//papeer//11//eeee fa7cdfad1 a www.aaaaaaaaeeoeeaaaaaaaa.a..... Pappiffff.c--pe.re.o.am pdf.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaune Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. ArXiu, abs/160 00ss/666..06611, 2016. URL https://apl. semanticscholar. wrwpaaaoooo//oooriitteeo 15350923.
Paul Ruvolo and Eric Eaton. Active task selection for lifelong machine learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 27, pp. 862 868, 2013.
Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska~ Barwinskk,, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for continual learning. In International conference on machine learning, pp. 4528-4523 PMLR, 2018.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard allention to the task. In International conference on machine learning, pp. 4548 4557. PMLR, 2018.
Guangyuan Shi, Jiaxin Chen, Wenlong Zhang, Li-Ming Zhan, and Xiao-Ming Wu. Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima. Advances in neural information pro- cessing systems, 34:6747 777177711111 2021.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
Christian Simon, Piotr Koniusz, and Mehrtash Harandi. On learning the geodesic path for incremental learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognátion, pp. 1591 1600, 2021.
James Smith, Yen-Chang Hsu, Jonathan Balloch, Yilin Shen, Hongxia Jin, and Zsolt Kira. Always be dreaming: A new approach for data-free dass-incremennal learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9374 9384, 2021.
Zeyin Song, Yifan Zhao, Yujum Shi, Peixi Peng, Li Yuan, and Yonghong Tian. Learning with fantasy: Semantio-aware virtual contrastive constraint for few-shot class-nneremental learning. In Proceedings of the TEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 24183 24192, 2023.
25

---

Published in Transactions on Machine Learning Research (anuary//022)
A. Soutif-Cone Contff-Cormerais, Mare Masana, J. van de Weijer, and B. Twardowski. On the importance of cross-task features for class-incremental learning, June 2021.
Wenju Sun, Qingyong Li, Jing Zhang, Danyu Wang, Wen Wang, and YangLi-ao Geng. Exemplar-free free class incremental learning via discriminative and comparable parallel one-class classifiers. Pattern Recognition, 140:109561, 2023.
Fiji Takimoto and Akira Maruoka. Relationships between learning and information compression based pac learning model. Systems and computers in Japan, 24(8):47 58, 1993.
Yang Tan, Yang Li, and Shao-Lun Huang. Otce: A transferability metric for cross-domain cross-task representations. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognation (CVPR), pp. 15779-15788 June 2021.
Shixiang Tang, Dapeng Chen, Jinguo Zhu, Shijic Yu, and Wanli Ouyang. Layerwise optimization by gradient decomposition for continual learning. In Proceedings of the IEEE/CVF conference on Compuler Vision and Pattern Recognition, pp. 9634 9643, 2021.
Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, and Yihong Gong. Dopology-preservine class- incremental learning. In Computer Vision-FCCO FCCC 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Port XIX 16, pp. 254 270. Springer, 2020.
Songsong Tian, Lusi Li, Weijun Li, Hang Ran, Xin Ning, and Prayag Tiwari. A survey on few-shot class-incremental learning. Newral Nelworks, 169:307 324, 2024. ISSN 0893-6080. doi: https:// 1ttpg///...01//..eeeee www.eeeww..t.....ooom 2023.00.39. URL S08S3608023006019.
Anh T Tran, Cuong V Nguyen, and Tal Hassner. Transferability and hardness of supervised elassification tasks. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 1395 1395-1405, 2019.
Twitter's official blog. The 2014 #ycarontwitter, 2014. URL https://blog. http:///o.....ooooofooccccccaaaa au/a/// 2014/the- -14------t--errrtutttteee html. Accessed: 2024-11-14.
Gido M Van de Ven and Andreas S Tolias. Three continual learning scenarios. In NewrIPS Continual Learning Workshop, volume 2018. URL https://doi. ort.//.......... 48550/arXiis 1904.9907777
Gido M Van De Ven, Zhe Li, and Andreas S Tolias. Class-incremennall learning with generative classifiers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 3611-3620, 2021.
Vladimir N. Vapnik, Statistical Learning Theory. Iitey--nterscieene, 1998.
Eli Verwimp, Rahaf Aljundi, Shai Ben-David, Matthias Bethge, Andrea Cossu, Alexander Gepperth, Tyler L. Hayes, Eyke Hüllermeier, Christopher Kanan, Dhireesha Kudithipudi, Christoph H. Lampert, Martin Mundt, Razvan Pascanu, Adrian Popescu, Andreas S. Tolias, Joost van de Weijer, Bing Liu, Vincenzo Lomonaco, Tinne Tuytelaars, and Gido M van de Ven. Continual learning: Applications and the road forward. Trunsactions Machine Learning Research, 2024. ISSN 2835-8856. URL https:://pperev/////
Fu- Yun Wang, Da-Wet Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for class-incremental learning. In European conference computer vision, pp. 398-414 Springer, 2022a.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46 (8):5362 5383, 2024. doi: 10. TTPAM...02433333399
Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 184-193, 2021.
26

---

Published in Transactions on Machine Learning Research (anuury//022)
Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni. Generalizing from few examples: A survey on few-shot learning. ACM Compul. Surv., 53(3), June 2020. ISSN 0360-0300. doi: 10. 10.1145/3366252 URL https://doi.org org/10..//1. 10. 1145/3386252.
Zifeng Wang, Zheng Zhan, Yifan Gong, Geng Yuan, Wei Niu, Tong Jian, Bin Ren, Stratis loannidis, Yanzhi Wang, and Jennifer Dy. Sparcl: Sparse continual learning on the edge. Advances in Neural Information Processing Systems, 35:20366 20:00,,0-00300, 2022b.
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jemnifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. European Conference Computer Vision, 2022c.
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Paltern Recognition, pp. 139 149, 2022d.
Zifeng Wang, Zheng Zhan, Yifan Gong, Yucai Shao, Stratis loannidis, Yanzhi Wang, and Jennifer Dy. Dualhsic: ssic-bottleneck and alignment for continual learning. In Proceedings of the 40th International Conference on Machine Learning, ICML''2 JMLR.org, 2023.
Max Welling. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Leurning, ICML '09, pp. 1121 -128, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605585161.
doi: 10.1145/16:33441555555
URL https://dol http://ddo..org///// oooo:.org//1... 1145/
1553374.1553517.
Mitchell Wortsman, Vivek Ramamuja, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. Advances in Neural Information Processing Systems, 1411573--11844, 2020.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CU /CF conference on computer vision and pallern recognition, pp. 374 382, 2019.
Wenwu Zhu Xin Wang, Yudong Chen. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(09):4555-4576, sep 2022. ISSN 1939-3539. doi: 10.1109/TPAMI. 002193069908.
Ju Xu and Zhanxing Zhu. Reinforced continual learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 907-916, 2018.
Yang Yang, Da- Wei Zhou, De-Chuan Zhan, Hui Xiong, and Yuan Jiang. Adaptive deep models for incremen- tal learning: Considering capacity scalability and sustainability. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19 pp. 74-82, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450362016. doi: 00.1145/39920003333666 URL https: org. 10. 1145/3292500 3330865.
Yang Yang, Zhiying Cui, Junjie Xu, Changhong Zhong, Wei-Shi Zheng, and Ruixuan Wang. Continual learning with bayesian model based on fixed pre-trained feature extractor. Visual Intelligence, 1(1) (1):5, 2023a.
Yang Yang, Da-Wei Zhou, De-Chuan Zhan, Hui Xiong, Yuan Jiang, and Jian Yang, Cost-effective incremental deep model: Matching model capacity with the least sampling. IEFE Transactions on Knowledge and Data Engineering, 35(4):3575 3588, 2023b. doi: SLLU///TKKDE2222333222222
Zhaonan Yang and Huiyun Li. Task ordering matters for incremental learning. In 2021 International Sympo- stum on Networks, Computers and Communications (ISNCC), pp. 1-6, 2021. doi: 10.1109/ISNCC550 2021.9115907
27

---

Published in Transactions on Machine Learning Research (anuary/2025)
Fei Ye and Adrian G Bors. Learning latent representations across multiple data domains using lifelong vaegan. In Compuler Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23 3828, 2020, Springer, 2020.
Proceedings, Part XX 16, pp.
Fei Ye and Adrian G Bors. Lifelong mixture of variational autoencoders. IEEE Transactions on Neural
Networks and Learning Systems, 34(1):461 474, 2021a.
Fei Ye and Adrian G Bors. Lifelong twin generative adversarial networks. In 2021 IEEE International Conference Image Processing (ICIP), pp. 1289-1293. IEEE, 2021b.
Fei Ye and Adrian G Bors. Wasserstein expansible variational autoencoder for discriminative and generative continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18665-18675, 2023.
Jachong Yoon, Eunho Yang. Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically ex- pandable networks. In 6th International Conference on Learning Representations, ICLR 2018, 2018. URL httts://openrrvieev http:/ppaawvvvvvveeeeer/orrrrras mettfforum/ddds27sftt00
Shinsci Yoshikiyo, Naoko Misawa, Chihiro Matsui, and Ken Takeuchi. Edge computation-in-monon for in-situ class-incremental learning with knowledge distillation. In 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2953-2957, 2022. doi: 10. 20000//SCCA4477550222..828
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent processing neural networks. Nature Machine Intelligence, 1(8):364 372, 2019.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Contimual learning through synaptic intelligence. In International conference on machine learning, pp. 3995. PMLR, 2017.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Commun. ACM, 64(3):107-11) February 2021. ISSN 0001-0782. doi: 10.1145/3446776. URL https://dol http://do..oggg/1///
org/10..///.. 1tt///..1115//9 1145/3446776
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C- C Jay Kuo. Class- class-incremental learning via deep model consolidation. In Proceedings of the IEEE/CVF winter conference applications of computer vision, pp. 1131-1140, 2020a.
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C- C Jay Kuo. Class-incremental learning via deep model consolidation. In Proceedings of the IEEE/CVP Winter Conference on Applications of Computer Vision, pp. 1131-11480 1140, 2020b.
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairness in class incremental learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 13208 13217, 2020.
Hanbin Zhao, Hui Wang, Yongjian Fu, Fei Wu, and Xi Li. Memory-efficient class-incremental learning for image classification. IEEE Transactions on Neural Networks and Learning Systems, 33(10):5966-599) 5977, 2021.
Da- Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Co-transport for class-incremental learning. In Proceedings of the 29th ACM International Conference on Multimedio, pp. 1645-1654, 2021.
Da-Wer Wei Zhou, Yun Wang, Han-Jia Ye, Liang Ma, Shiliang Pu, and De-Chuan Zhan. Forward compatible few-shot dlass-incremente. learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9046-9056, 2022a.
Da-Wei Zhou, Han-Jia Ye, Liang Ma, Di Xic, Shiliang Pu, and De-Chuan Zhan. Few-shot inassincrementll learning by sampling multi-phase tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, (2211))1281611221 2022b.
28

---

Published in Transactions on Machine Learning Research (anuaryy/022)
Da- Wei Zhou, Qi- Wang, Han-Jia Ye, and De- Chuan Zhan. A model or 603 exemplars: Towards memory- efficient class-incremental learning. In The Eleventh International Conference Learning Represenlations. 2023. URL https://doi. org/ 10. 40.4850/arxiiv 2205. 13218.
Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Class-Incremental Learning: A Survey IEEE Transactions Pattern Analysis & Machine Intelligence, 46(12):9851- 9873, December 2024. ISSN 1939-3539. doi: 10.1109 2000/T/ URL https://doi. 3429388.
ieeeconputersociety. wttpt/rrrciiet..og11....1111
Fei Zhu, Zhen Cheng, Xu-yao Zhang, and Cheng-lin Liu. Class-incremental learning via dual aug- mentation. In M. Ranzato, A. Beygekimer, Y. Dauphin, P.S. Liang, and J. Wortman Vanghan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 14306 14318. Curran Associates, Inc., 2021a. URL https://proon attps://proceedings neurips cttpipp..cppppprrrpp f 7T58bb88c660066228862222111-9999 Paper. Paper......a...............
Fei Zhu, Xu- Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Lin Liu. Prototype augmentation and self- supervision for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5871-5880, 2021b.
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yougchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43-76, 2020.
29


Reviews:
[{"review_id": "oU4kRH61e7", "reviewer": "Reviewer_dnW6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper offers a novel practical perspective on CIL by focusing on resource constraints and highlighting the often-overlooked challenge of across-task discrimination. This reframing is valuable and can lead to more practical research directions.\n- The paper is well-organized and clearly explains complex concept. The literature review seems complete and described thoroughly. \n- The paper further defines challenges and metrics on how these challenges can be addressed and measured.\n\nWeaknesses:\n- There is a disconnect between between the proposed framework (resource constraints and novel metrics) and the subsequent discussion of existing CIL methods. Sections 4, 5, and 6 do not analyze existing methods within the context of the proposed memory/compute framework or utilize the novel metrics introduced in Section 3. \n- The paper is positioned as a review and has no experiments, but it also proposes a novel way to look at a new problem and defines novel metrics without actually evaluating existing methods using them. I would suggest the author to answer the following questions in the introduction to make it really clear what is going on. What are the actionable lessons of the paper? Why do authors think that the audience would find this paper valuable? What are the claims of this paper?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This review paper re-examines Class Incremental Learning (CIL), arguing for a problem definition centered around resource constraints (memory and compute) rather than more traditional focus on data availability. The setting with restricted access to past data seems unrealistic and can potentially lead to inefficient solutions. \n\nFrom this perspective, they highlight several key challenges in CIL and defines a metric for each of them: Balancing stability and plasticity, Learning across-task discrimination and Transferring knowledge across tasks.\n\nThe paper proceeds to explore several naive approaches to tackle CIL and finally provides a literature review of papers while framed around naive approaches and challenges they above.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- The disconnect between the resource-constrained framework (Section 3) and the review of existing methods (Sections 4-6) is a major weakness that needs to be addressed better in the paper.\n- I would at least mention briefly other types of continual learning, such as task-incremental and domain-incremental learning. Also, in the era of LLM it is worth mentioning that there are papers on few-shot incremental learning where we need to learn a new class on top of large pre-trained based clasifier.\n- I would suggest to the authors to provide more detailed examples of real world systems that fit the settings outlined in the paper and which of the reviewed method would be best to suit this problem.", "broader_impact_concerns": "No Broader Impact Statement is required as this is a review work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "38mjNgz54t", "reviewer": "Reviewer_Ep3w", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Weaknesses:\n- This paper introduces/argues with terms and concepts that are not conventional and lack justifications:\n    - For instance, \"We argue that restrictions in the form of resource constraints are more appropriate than restrictions on the availability of previously learned data.\" - this is not quite valid, in my opinion. In the modern era of machine learning, where we train foundation models with ever-increasing sizes, the resources required for continual learning are not our primary concern, especially in class incremental learning settings. So, the continual learning literature motivates the problem setup as the unavailability of previously learned data for various reasons, including privacy concerns. \n- The argument becomes more problematic in section 2.1. \n    - If we remove the old restriction and we have all the previous data available, we do not need a specialized continual learning solution. We can simply rely on a supervised solution. The entire point of the literature and the existing methods is to make sure to learn from new data while not forgetting the old ones. Even rehearsal-based solutions that store a few samples per old class perform considerably higher than solutions that do not store previous samples.\n    - Again, Critique #1 is not a significant problem. Also, the training compute time is less of a concern compared to inference compute time.\n    - Critique #2 is also not a strong argument. There is ongoing research to solve such privacy concerns of memorizing training data. Also, decoding usually requires access to model weight, which is not often available for deployed ML systems as a server. Even keeping that argument aside,  by no means such an excuse allow for doing more privacy concerning developments. \n    - I find Critique #3 even less intuitive. The example regarding K-nearest neighbours is also not completely valid, and it can't always memorize all data. On top of that, it's a classic algorithm. No recent methods on CIL use such methods. In modern machine learning, there is a **clear** distinction between a **model** and a **dataset**. \n    - Under 2.2.2. Yes, it's a valid point that we want to reduce the compute cost as much as possible, but not at the expense of performance. After all, training happens once; deployment cost is what matters, and the argument has no indication of a solution to reducing that. \n        - Edge devices usually delete the model, not train it - unless it's a federated learning system.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The machine learning community has made significant strides in various applications, including computer vision and natural language processing, through the dominant approach of deep learning. However, this success is hindered by the stability-plasticity dilemma, where existing methods struggle to accommodate new data without forgetting previously learned knowledge. Continual learning has emerged as a promising solution, with a focus on alleviating catastrophic forgetting. Specifically, Class-Incremental Learning (CIL) addresses the supervised classification problem in continual learning, where new classes of data are introduced over time. This review delves into the CIL setting, examining the problem statement, existing solutions, and unexplored directions, with the ultimate goal of understanding and refining the CIL problem statement to drive future advancements.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "- To begin with, the reformulation of CIL is full of arguments that are not backed by evidence, theory, or any strong logic. The whole section needs to go. The whole paper requires a full rewrite with the existing definition of CIL established by prior works in this literature.\n- For the remaining paper, there are no experimental results to compare different methods. The paper must include such results to understand the effectiveness of different methods and do the comparison.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YGwDXyi2yf", "reviewer": "Reviewer_EWou", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n1)\tThe paper is well-organized and easy to follow.\n\n2)\tThe perspective is novel, and directing attention to unexplored aspects of the CIL problem.\n\n3)\tThe article redefines the problem of Class Incremental Learning (CIL), explicitly pointing out the shortcomings in the existing definition, especially in the discussion of resource constraints. This re-examination of the problem provides a clearer direction and framework for subsequent research, and helps to advance the field.\n\n4)\tThe article delves into three core challenges involved in class incremental learning: stability-plasticity balance, cross-task discriminative learning, and knowledge migration. These analyses not only explain the complexity of CIL compared to traditional machine learning, but also suggest directions worth exploring for future research.\n\n5)\tThe article not only summarises existing methods, but also points out their shortcomings in terms of computational efficiency, cross-task differentiation, and so on. In particular, the discussion of ‘naive’ approaches provides valuable background information for understanding the difficulties and directions of CIL development.\n\n\n**Weaknesses:**\n\n1)\tAlthough the paper differentiates itself from other reviews of continuous learning, it covers fewer of some of the most recent advances in the field of class-based incremental learning (CIL) beyond 2023, particularly emerging hybrid methods and new benchmarking. This may limit its reference value to researchers seeking to understand the latest advances in the field.\n\n2)\tThe challenges of class-incremental learning are discussed primarily at a high level. A more nuanced breakdown with a focus on specific subfields (e.g., task-agnostic versus task-aware incremental learning) could provide a more granular understanding and be beneficial to different categories of readers.\n\n3)\tThe section dedicated to discussing \"naive\" approaches such as single discriminative models, task-specific models, and class-specific generative models, while providing historical context, takes up significant space and repeats insights that are well-known. This may not add much value to the paper’s intended audience, who may already be familiar with these basic approaches.\n\n4)\tThe argument for framing CIL in terms of resource constraints instead of unavailability of past data is interesting. However, the discussion could be expanded to address specific practical implementations and comparisons, especially around the computational cost of different methods. This would make the argument more compelling and actionable for real-world applications.\n\n5)\tWhile the conclusion does suggest some future directions, the suggestions are somewhat broad and lacking in concrete specifics on how certain challenges (e.g., learning across-task discrimination or computational efficiency) might be addressed. This limits the practical impact and future utility of the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides an alternative perspective to existing work on CIL. Considering resource constraints revisiting the problem definition and describing its unique challenges. A principled way of developing solutions to the CIL problem is provided by analysing discontinuous methods and providing an overview of existing work. Five specific dimensions of the CIL problem are then analysed and how existing work deals with various problem configurations is described. Recommendations for future work are summarised at the end of the article, arguing that future solutions can and should make assumptions regarding the data generation process.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1)\tIt is recommended that the authors add a discussion of recent research results in the field of Class Incremental Learning (CIL) beyond 2023, especially hybrid models and new benchmarking methods. You can discuss recent review papers such as;  Class-incremental learning: A survey; A survey on few-shot class-incremental learning; and others.\n\n2)\tThe paper provides a broader discussion of the challenges in class-based incremental learning, and it is recommended that this section be augmented with some more detailed analyses of sub-domains, such as the differences between task-independent incremental learning and task-related incremental learning, as well as the distinctive challenges they each face. This would allow for a more nuanced discussion and meet the needs of a diverse group of readers.\n\n3)\tThe paper discusses too much about the basic ‘naive’ methods (e.g., single discriminant model, task-specific model, category-generating model), and it is suggested to simplify this part of the paper and devote more space to the in-depth analyses and comparisons of the existing state-of-the-art methods, so as to emphasise the innovativeness and unique insights of the paper.\n\n4)\tWhen discussing the impact of resource constraints on the CIL problem, it is recommended that this section be further refined and expanded, especially the specific implementation, comparison and analysis of experimental results for different resource constraints (e.g. computation and memory), to provide more empirical support and make this part of the discussion more robust and comprehensive.\n\n5)\tIn the future work section, it is suggested that more specific research directions are proposed, especially providing actionable suggestions on how to deal with cross-task discrimination learning, knowledge transfer, and how to optimise in terms of computational efficiency. This will make the article more instructive for future research.\n\n6)\tNeed to summarise previous incremental learning reviews and plot them in a table showing the differences between this review compared to them, etc.\n\n7)\tClearly state the timeframe covered by this review, i.e. all classes of incremental learning methods discussed are research results published before a specific point in time. For example, ‘This review covers all relevant work published up to October 2024’.", "broader_impact_concerns": "1)\tThe CIL framework proposed in the article may rely on storing some of the old data or generating data for playback by generating models. However, this may lead to privacy and data security issues, especially when models are trained on tasks containing sensitive data, with an increased risk of data leakage. Models may ‘remember’ old data and inadvertently disclose sensitive information, especially when applied in domains such as healthcare or finance. It is recommended that the authors explore how to enhance privacy protection mechanisms in CIL.\n\n2)\tClass incremental learning (CIL) methods may inherit biases in existing data when applied to real-world scenarios. For example, when applied to sensitive domains such as autonomous driving or facial recognition, models may amplify racial, gender, or other social biases in the data as they incrementally learn new tasks. Discussion of these social and ethical issues is lacking in the paper, so it is recommended that content be added on how to address bias and fairness issues in CIL methods.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This survey reframes class-incremental learning (CIL) as learning to classify under resource constraints. Two reviewers found this perspective valuable, and the survey thorough, especially in its treatment of prior related work, including prior surveys. The remaining reviewer challenged the reformulation of CIL and viewed the submission as insufficient without any novel results. While indeed the survey doesn't present new results of its own, the majority of reviewers assessed that there is enough of a fresh perspective and targeted discussion of areas for future research for this survey to be a valuable contribution to TMLR, especially after the authors' revision to better connect the reformulation with existing work in CIL. I agree with this assessment and recommend acceptance.
Decision: Accept (poster)