Class Incremental Learning via Likelihood Ratio Based Task Prediction
Authors: Haowei Lin
Yijia Shao
Weinan Qian
Ningxin Pan
Yiduo Guo
Bing Liu
Venue: ICLR 2024 poster (2024)
Keywords: Class Incremental Learning
Continual Learning
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Class incremental learning (CIL) is a challenging setting of continual learning, which learns a series of tasks sequentially. Each task consists of a set of unique classes. The key feature of CIL is that no task identifier (or task-id) is provided at test time. Predicting the task-id for each test sample is a challenging problem. An emerging theory-guided approach (called TIL+OOD) is to train a task-specific model for each task in a shared network for all tasks based on a task-incremental learning (TIL) method to deal with catastrophic forgetting. The model for each task is an out-of-distribution (OOD) detector rather than a conventional classifier. The OOD detector can perform both within-task (in-distribution (IND)) class prediction and OOD detection. The OOD detection capability is the key to task-id prediction during inference. However, this paper argues that using a traditional OOD detector for task-id prediction is sub-optimal because additional information (e.g., the replay data and the learned tasks) available in CIL can be exploited to design a better and principled method for task-id prediction. We call the new method TPL (Task-id Prediction based on Likelihood Ratio). TPL markedly outperforms strong CIL baselines and has negligible catastrophic forgetting. The code of TPL is publicly available at https://github.com/linhaowei1/TPL.

Published as a conference paper at ICLR 2024
CLASS INCREMENTAL LEARNING VIA LIKELIHOOD RATIO BASED TASK PREDICTION
Haowei Lin', Yijia Shao?, Weinan Qian?, Ningxin Pan", Yiduo Guo and Bing Liu Institute for Artificial Intelligence, Peking University "Stanford University
3 Wangxuan Institute of Computer Technology, Peking University
*Department of Computer Science, University of Illinois Chicago
1 "linhaowei i@kku. edu, on 3 (ypqwn, 2100017816, yiduo ) wwt.tskkuuuu.....p.... edu. #liubBuic.edu
ABSTRACT
Class incremental learning (CIL) is a challenging setting of continual learning, which learns series of tasks sequentially. Each task consists of a set of unique classes. The key feature of CIL is that no task identifier (or task-id) is provided at test time. Predicting the task-id for each test sample a challenging problem. An emerging theory-guides approach (called TIL+OOD) is to train task-specific model for each task in a shared network for all tasks based on a mask-incremental The model for each task
learning (TIL) method to deal with catastrophic is an out-of- -iit--i-iittiiii (OOD) detector rather than a conventional classifier. The OOD detector can perform both within-task (in-distribution (IND) class prediction and OOD detection. The OOD detection capability is the key to task-id prediction during inference. However, this paper argues that using traditional OOD detector for task-id prediction is sub-optimal because additional information (e.g.. the replay data and the learned tasks) available in CIL can be exploited to design better and principled method for task-id prediction. We call the new method TPL (Task-id id Prediction based on Likelihood Ratio). TPL. markedly outperforms strong CIL. baselines and has negligible catastrophic forgetting.
INTRODUCTION
Continual learning learns a sequence of tasks. 1,2, T. incrementally (Ke & Liu, 2022: De Lange et al., 2021). Each task consists of set of classes to be learned. This paper focuses the challenging CL. setting of flass-incremental learning (CIL.) (Rebuffi et al., 2017). The key challenge of CIL lies in the absence of task-identifier (task-id) in testing. There is another CL setting termed kask-incremental learning (TIL), which learns a separate model or classifier for each task. In testing, the task-id is provided for each test sample so that it is classified by the task specific model.
main assumption of continual learning is that once a task is learned, its training data is no longer accessible. This causes catastrophic forgetting (CF), which refers to performance degradation of previous tasks due to parameter updates in learning each new task (McCloskey & Cohen, 1989). An additional challenge specifically for CIL is infer-task class separation (ICS) (Kim al., 2022b). That is, when learning new task, it hard to establish decision boundaries between the classes of the new task and the classes of the previous tasks without the training data of the previous tasks. Although in replay-based methods (Rebuffi et al., 2017; Kemker & Kanan, 2017; Lopez-Paz & Ranzato, 2017 a small number of training samples can be saved from each task (called the replay data) help deal with CF and ICS to some extent by jointly training the new task data and the replay data from previous tasks, the effect on CF and ICS is limited as the number of replay samples very small.
An emerging theoretically justified approach to solving CIL is to combine a TIL technique with an muttof-distribution (OOD) detection method, called the TIL+OOD approach (Kim al., 2022b). The TIL method learns model for each task in shared network. The model for each task is not
"Correspondance author. Bing Liu <1 iubbuic.c.. edu>> The code of TPI. is publicly available at https:t///// /ttp:////ggttttttteeee com/1 inhoowe T/IPI.

---

Published as a conference paper at ICLR 2024
traditional classifier but an OOD detector. Note that almost all OOD detection methods can perform two tasks (1) in-distribution (IND) classification and (2) nut-of-distribution (OOD) detection (Vaze et al., 2022). At test time, for each test sample, the system first computes a task-id prediction (TP) probability and a within-task prediction (WP) prohability (Kim et al., 2022b) (same as IND) classification) for each task. The two probabilities are then combined to make the final classification decision, which produces state-of-the-al results (Kim et al., 2022b; 2023). In this approach, WP is usually very accurate because uses the task-specific model. TP is the key challenge.
There is related existing approach that first predicts task-id and then predicts the class of the test sample using the task-specific model (Rajasegaran et al., 2020; Abati et al., 2020; Von Oswald et al., 2019). However, what is new is that Kim et al. (2022b) theoretically proved that TP is correlated with OOD detection of each task. Thus, the OOD detection capability of each task model can be used for task-id prediction of each test sample. The previous methods did not realize this and thus performed poorly (Kim et al., 2022b). In Kim et al. (2022b), the authors used the THI. method HAT (Serra et al., 2018) and OOD detection method CSI (Tack et al., 2020). HAT is parameter isolation method for TIL, which learns a model for each task in a shared network and each task model is protected with learned masks to overcome CF. Each task model is an OOD detector based on CSI.²
Our paper argues that using traditional OOD detectors is not optimal for task-id prediction as they not designed for CIL and thus do not exploit the information available in CIL for better task-id prediction. By leveraging the information in CIL,, we can do much better. A new method for task-id prediction is proposed, which we call TPL (Task-id Prediction based Likelihood Ratio). It consists of two parts: (1) new method to train each task model and (2) a novel and principled method for task-id prediction, i.e., to estimate the probability of a test sample æ belonging ((([=)).. We formulate the estimation of (() as a binary selection problem between two events "æ belongs to t" and "æ belongs to t ta is t's complement with regard to the universal set Ucn. which consists of all tasks that have been learned, i.e., Ucu. = 21,2,-. .T} and t" = Ucn [t}-
task t, i.e.,
The idea of TPL is analogous to using OOD detection for task-id prediction in the previous work. However, there is a crucial difference. In traditional OOD detection, given a set UND of in- distribution classes, we want to estimate the probability that a test sample does not belong to any classes in UND. This means the universal set UOOD for OOD detection includes all possible classes in the world (except those in UNND), which is at least very large not infinite in size and we have no data from UOOD. Then, there is way we can estimate the distribution of UooD. However, can estimate the distribution of Ucn based on the saved replay data² from each task in CIL. This allows us to use the likelihood ratio of Pr: and Peo to provide a principled solution towards the binary selection problem and consequently to produce the task-id prediction probability P(t) [=) as analyzed in Sec. 4.1, where Pe is the distribution of the data in task / and Pe- is the distribution of the data in tf (all other tasks than f), i.e., t's complement (t = Ucn - {t)).
The proposed system (also called TPL) uses the learned masks in the TIL method HAT for overcoming CF but the model for each task within HAT is not a traditional classifier but a model that facilitates task-id prediction (Sec. 3). At test time, given a test sample, the proposed likelihood ratio method is integrated with a logit-based score using an energy function to compute the task-id prediction probability and within-task prediction probability for the test sample to finally predict its class. Our experiments with and without using pre-trained model show that TPL. markedly outperforms strong basclines. With a pre-trained model, TPL has almost no forgetting or performance deterioration. We also found that the current formula for computing the forgetting rate is not appropriate for CIL.
2 RELATED WORK
OOD Detection. OOD detection has been studied extensively. Hendrycks & Gimpel (2016) use the maximum softmax probability (MSP) as the OOD score. Some researchers also exploit the logit space (Liang et al., 2017; Liu et al., 2020a; Sun et al., 2021), and the feature space to compute the distance from the test sample to the training data/IND distribution, e.g., Mahalanobis distance (Lee et al., 201 8b) and KNN (Sun et al., 2022). Some use real/generated OOD data (Wang et al., 2022d; Liu et al., 2020a; Lee et al., 2018a). Our task-id prediction does not use any existing OOD method.
2 (Kim et al., 2023), was also shown that based on this approach, CII. is leamable.. 3 In our case, the saved replay data are used to estimate the distrihution of Ucn rather than to replay them in training a new task like replay-hased methods. Also, our work is not about online continual learning.
2

---

Published as a conference paper at ICLR 2024
Continual Learning (CL). Existing CL methods are of four main types. (1) iigularrzzaton--aasee methods address forgetting (CF) by using regularizers in the loss function (Kirkpatrick et al., 2017; Zhu et al., 2)21) or orthogonal projection (Zeng et al., 2019) to preserve previous important pa- rameters. The regularizers in DER (Yan et al., 2021) and BEEF (Wang et al., 2022a) are similar to OOD detection but they expand the network for each task and perform markedly poorer than our method. (2) Replay-based methods save few samples from each task and replay them in training new tasks (Kemker & Kanan, 2017: Lopez-Paz & Ranzato, 2017; Li et al., 2022). However, replaying causes data imbalance (Guo et al., 2023; Xiang & Shlizerman, 2023; Ahn et al., 2022). (3) Parameter isolation methods train a sub-network for each task. HAT (Serra al., 2018) and SupSup (Wortsman et al., 2020) are two representative methods. This approach is mainly used in task-incremental learning (TIL.) and can eliminate CF. (4) TIL+OOD based methods have been discussed Sec. 1.
Recently, using pre-trained models has become standard practice for CL. in both NLP (Ke et al., 2021a;b; 2023; Shao et al., 2023). and computer vision (CV) (Kim et al., 2022a; Wang et al., 2022e). See the surveys (Ke & Liu, 2022; Wang et al., 2023; De Lange et al., 2021; Hadsell et al., 2020).
Our work is closely related to CIL methods that employ a TIL technique and a task-id predictor. iTAML (Rajasegaran et al., 2020) assumes that each test batch is from single task and uses the whole batch to detect the task-id. This assumption is unrealistic. CCG (Abati et al., 2020) uses separate network to predict the task-id. Expert Gat e (Aljundi et al., 2017) builds a distinct auto-encoder for each task. HyperNet. (Von Oswald et al., 2019) and PR-Ent (Henning et al., 2021) use entropy predict the task-id. However, these systems perform poorly as they did not realize that OOD detection is the key to task-id prediction (Kim et al., 2022b), which proposed the TIL+OOD approach. Kim et al. (2022b) gave two methods HAT+CSI and SupSup+ (Kim et al., 2022b). These two methods do not use a pre-trained model or replay data. The same approach was also taken in MORE (Kim et al., 2022a) and ROW (Kim al., 2023) but they employ a pre-trained model and replay data in CIL. These methods have established a state-of offte-of-the-ar performance. We have discussed how our proposed method TPI. different from them in the introduction section.
OVERVIEW OF THE PROPOSED METHOD
Preliminary. Class incremental learning (CIL) learns sequence of tasks 1,..,, T. Each task t has an input space X(0), a label space y)(), and a training set DDt) {(xr)) (6) (0) ) drawn i.i.d. from yj
P Prttyoo. The class labels of the tasks are disjoint, i.e., yo6) n ()(()) = 0, vi + k. The goal of CIL is to learn function f x )0() to predict the class label of each test sample
Kim et al. (2022b) proposed a theory for solving CIL. It decomposes the CIL probability of a test sample x of the j-th class v/ (t) in task / into two probabilities (as the classes in all tasks are disjoint),
Pwr = P(w)") ax, ())(())).
(1)
The two probabilities on the right-hand-side (R.H.S) define the CIL probability on the sefd-hand-side (L.H.S). The first probability on the R.H.S. the within-task prediction (WP) probability and the second probability the R.H.S. is the task-id prediction (TP) probability. Existing TIL+OOD methods basically use a traditional OOD detection method to build each task model. The OOD detection model for each task is exploited for estimating both TP and WP probabilities (see Sec. 1).
Overview of the Proposed TPL. This paper focuses on proposing a novel method for estimating task-id prediction probability, i.e., the probability of test sample æ belonging to (or drawing from (t) the distribution of) a task I, i.e., P(I æ)) in Sec. 1. The WP probability P(y) [x, i) can be obtained directly from the model of each task.
The mask-based method in HAT is used by our method to prevent CF. Briefly, in learning each task, learns model for the task and also set of masks for those important neurons to be used later to prevent the model from being updated by future tasks. In learning a new task, the masks of previous models stop the gradient flow to those masked neurons in back propprropagttion, which eliminates CF. In the forward pass, all the neurons can be used, so the network is shared by all tasks. We note that our method can also leverage some other TIL methods other than HAT to prevent CF (see Appendix G).
The proposed method TPL is illustrated in Figure 1. It has two techniques for accurate estimation of P(I[a)), one in training and one in testing (inference).
3

---

Published as a conference paper at ICLR 2024
Network Architecture
Inference Pipeline
①
Inactivated
@Activated Task- teak-specific feature Mx:; pf")
Logits for task f(h(x; s/"); 0*)
Within-task logits
appaa
Task
XX0XXXXX0 e
Adapter (HAT)
XXXX
Task-id
MLP
4
Norm
1 ... 20
Other-task class
Softmax
Classifier for task
Task
XXXXXXXX
Adapter (HAT)
CIL probability:
P P(y xc)
P 1) x P(12x)
aooxa
Task-id
Softmax
Multi-Headd Attention
Desimbation of other todes
Thambation of
Ejx0) ESxO
Norm
PAX B
Task
000000
Embedded
X0000
Task-id
Patches
logit score Figure 1: Illustration of the proposed TPL. We use pre-trained transformer network (in the grey box) (see Sec. 5.1 for the case without using pre-trained network). The pre-trained network is fixed and only the adapters (Houlsby al., 2019) inserted into the transformer important note that the adapter (in yellow) used by НAТ learns all tasks within the same adapter. The yellow boxes on the left show the progressive changes to the adapter as more tasks are learned.
Trainable modules
Feature-based likelihood ratio score
trainable adapt specific tasks. It is
(1) Training: In the original HAT, each model is a traditional supervised classifier trained with cross-entropy. However, for our purpose predicting task-id, this is insufficient because it has no consideration of the other classes learned from other tasks. In TPI, each model for task is trained using the classes y of task and an extra class (called O, for others) representing the replay buffer data Buf <t of all the previous tasks. This enables each model to consider not only the new task data but also previous tasks' data, which facilitates more accurate computation (t(æ)).
For each task f, its model consists of a feature extractor I(x;; ((()) (partially shared with other tasks based on HAT), and a task-specific classifier f(z; ((t)). When learning task t, the model receives the training data D(f) and the replay data Buf (stored in memory buffer). Then we minimize the loss:
((((1), (()) ((x.))>DDeeuuu [c (h((;; )) B())) 8)] + CHAY
(2)
where LeE is the cross- cross-entropy loss, Lurr is the regularization loss used in HAT (see Appendix G). (2) Testing (or inference): We follow eq. (1) compute the CIL probability. The WP probability (P xx, t)) for each test sample is computed through softmax on only the original classes y(t) of task E, the first on the right of eq. (3) (also the top right part in Figure 1). The class not used in inference. Note that the probabilities for different tasks can be computed in parallel.
(8) x)) P (
softmax (f(b(z;;) (1));; )(())) P(((æ)
(3)
The class v/" with the highest probability will be predicted as the class for test sample x. We discuss the proposed method for computing task-id prediction probability P(t]æ) (see the bottom right part in Figure 1) in the next section. Training will not discussed any further.
ESTIMATING TASK-ID PREDICTION PROBABILITY
4.1 THEORETICAL ANALYSIS
As noted in Sec. 1, we estimate the TP probability P(t[æ) by predicting whether a sample æ is drawn from the distribution P, of task t or drawn from the distribution of t's complement to.
We also calibrate the prohabilities from different task models, but it has little effect (see Appendix B).
4

---

Published as a conference paper at ICLR 2024
Le., Pte. We denote the universal set Ucn of all tasks (or task-ids) that have been learned, i..., Ucn = (1,2,-. ......... } and t" Ucu - {t). From a frequentist perspective, our objective can be formulated as binary hypothesis test:
Ho x Pe U.S. H1 : x Per
Using the Neyman- Panaasn lemma (Neyman & Pearson, 1933), we can derive theorem that demonstrates the principled role of likelihood ratio in this task (the proofs are given in Appendix E):
Theorem 4.1 A test with rejection region R defined as follows a unique uniformly most powerful (UMP) test for the hypothesis test problem defined in eq. (4):
R : {x p
do)-
where An is threshold that be chosen obtain specified significance level.
Theorem 4.2 The UMP test for hypothesis test defined in eq. (4) maximizes the Area Under the Curve (AUC) of binary classification between P. and Ptc.
Theorems 4.1 and 4.2 highlight the importance of detecting samples that do not belong to task t based on low density (x( and high t density pr (z)
Note that in traditional OOD detection, the system has no access to the true OOD distribution P:e but only P. (IND distribution). Some existing methods resort to a proxy distribution Por succh as uniform distribution (Nalisnick et al., 2018) or an auxiliary data distribution (Lin & Gu, 2023) as the universal set U is the set of all classes in the world and the universal set of all OOD classes for task t denoted by UOOD is very large if not infinite. This approach can lead to potential risks. For instance, consider scenario where Pt- (O, 0.01) and P. (0, 1). It is apparent that pc( > (c(1), but O is more likely to belongs to P.= than I as 0.1 = Pr (0) (0) < (c(1))p=(()) = 0.1 49.5, We further show the failure cases in real CIL scenarios in Appendix H.
Good News for CIL. In CIL, the IND distribution Pe for task l can be interpreted as the marginal distribution P Pxit, while Pte corresponds mixture distribution P Proce) comprising the individual marginal distributions (Pxte)) fe (which can be estimated based on the saved replay data), each of which is assigned the equal mixture weight, Consequently, we have the knowledge of Pa in CIL. thereby offering an opportunity to estimate Pe to be used to compute task-id prediction Pt more accurately. This leads our design of TPL in the following subsections.
4.2 COMPUTING TASK-ID PREDICTION PROBABILITY
We now present the proposed method for computing the task-id prediction probability P(((æ). which has three parts: (1) estimating both P, and P:, (as analyzed Sec, 4.1) and computing the likelihood ratio, (2) integrating the likelihood ratio based score with a logit-based score for further improvement, and (3) upplying a softmax function on the scores for all tasks to obtain the task-id prediction probability for each task. The three parts correspond to the bottom right part of Figure 1.
4.2.1 ESTIMATING P, AND P.e AND COMPUTING LIKELIHOOD RATIO
Guided by Theorem 4.1, we design task-id prediction score based on the likelihood ratio (x)))/p-((x). However, due to the challenges in directly estimating the data distribution within the high-dimensional raw image space, we instead consider estimation in the low-dimensional feature spuce. Interestingly. many distance-based OOD detection scores can function as density estimators that estimate the IND density p(x)) the feature space (see Appendix E.4 for justifications). For instance, MD (Mahalanobis Distance) (Lee et al., 2018b) estimates distributions using Gaussian mixture models, while KNN (Sun et al., 2022) uses non- con-paramttric estimation. Our method TPL also uses the two scores to estimate distributions (i.e., P. and P. in our case).
To connect the normalized probability density with unnormalized task-id prediction scores, we leverage energy-based models (EBMs) to parameterize P, and Pt.. Given test sample æ, it has density Pt (x) = exp{E, (x) in P., and density Pro (x: = exp{ (x=())) (x)) in Pie, where 71, Za are normalization constants that ensure the integral of densities (x(x) and pis (ær) equal 1,
5

---

Published as a conference paper at ICLR 2024
and E.),), E-) are called energy functions." Consequently, we can design feature- ceature-based task-id prediction score using the Likelihood Ratio (LR), which is also shown at the bottom right of Figure 1:
L (z) (atllp((/p/eeeee (a)) (x) - Er (x) + log log /2/1).
(5)
Since log(Zz/Zz) a constant, it can omitted in the task-id prediction score definition:
LR (x) = E, (x)) (x))
(6)
Since the energy functions (-) and (.) need not to be normalized, we estimate them with the above scores. We next discuss how to choose specific (-) and E--(-) for eq. (6).
For in-task energy (x(x)) of a task, we simply adopt an OOD detection score (z), which the OOD score for MD and is defined as the inverse of the minimum Mahalanobis distance of feature bax;; ((()) to all class centroids. The details of how S (x) is computed are given in Appendix F.I.
For out-of-task energy (x:) of task, we use replay data from other tasks for estimation. Let Bufs- be the set of buffer/replay data excluding the data of classes in task t. We set Ete (x) = dxxww( Bufr.). where dxww(æ, Buf{e) is the k-nearest distances of the feature h ((;; ()(() to the set of features of the replay Buf,- data. If dgNw(æ Buffe) ) is small, it means the distance between and replay Buft- data is small the feature space. The vanilla KNN score is ((t) (xn)(x)) = -xxww( D(!)), which was originally designed to estimate (x(x)) using the training set D(t), Here we adopt it to estimate pe (ar) using the replay data (Bufte). Finally, we obtain,
s (ar) = a: -)DD (x))
dRNw(ar, Buf(=)
E+(w)
(x)
where a is a syper-parameter to make the two scores comparable. This is a principled task-id prediction score as justified in Sec. 4.1.
Remarks. We can also use some other feature-based estimation methods instead of MD and KNN in (ræ( The reason why we choose MD to estimate P, is that it does not require the task data at test time (but KNN does), and we choose KNN to estimate P:- because the non con-paamettiic estimator KNN is high performing (Yang et al., 2022) and we only the saved replay data for this. We will conduct an ablation study using different estimation methods for both P, and Pee in Sec. 5.3.
4.2.2 COMBINING WITH LOGIT- BASED SCORE
To further improve the task-id prediction score, we combine the feature-based SLE score with a logit-based score, which has been shown quite effective in OOD detection (Wang et al., 2022c).
We again develop an energy-based model (EBM) framework for the combination that offers a principled approach to composing different task-id prediction scores. Specifically, to combine the proposed based sif (-) score with a logit-based score (an energy function) Sugi (-), we can make the composition
Ecomposition (x:) = log(exp(a1 s (x) - expla2 (x))),
(8)
where 0j and a2 are scaling terms to make different scores comparable. As noted in (Du et al., 2020), the composition emulates OR gate for energy functions.
To choose logit-based method for sh(((( in eq. (8), we opt for the simple yet effective method MLS
score s(t) (x:), which defined as the maximum logit of (also shown on the right of Figure 1).
Our final score s (æ), which integrates feature-based s(0 (-) and the logit-based s(U)s scores, uses the composition Eq. 8:
5 (x) = log (x ((i) exp( S (x) + dRNw(x, Buft D1f
5 In EBMs, the density p(x) is typically defined as - (oo(-E(())// Since our task-id prediction score is defined to measure the likelihood that the test sample helongs a task, the energy function here is defined as positively related to the probability density.

---

Published as a conference paper at ICLR 2024
where 31 and B2 are scaling terms, which are given by merging in eq. (7) and dr, 02 in eq. (8). Since the scale of dxvw(-) is near to 1, we simply choose 31 and B2 to be the inverse of empirical means of MD (x) estimated by the training data D(t) make different scores comparable:
s(t)) (as(x) and s
Σ DED
(L)( (
1
Σ DED
S (x:)
(10)
B1
D
B2
D
Remarks. We exploit EBMs, which are known for their flexibility but suffering from intractability. However, we exploit EBMs' ftexibility to derive principled task-id prediction score following Theo- rem 4.1 and eq. (8), while keeping the tractability via approximation using OOD scores (MD, KNN, MLS) in practice. This makes our proposed TPL maintain both theoretical and empirical soundness.
4.3 CONVERTING TASK-ID PREDICTION SCORES TO PROBABILITIES
Although theoretically principled shown in Sec. 4.1, our final task-id prediction score is still un- normalized energy function. We convert the task-id prediction scores for all tasks (i.e., { (5 (x))?=; to normalized probabilities via softmax:
P(t|z)) = softmax ([st) ((2) (1P))) (x), ... 1T 1T),
(11)
where is a temperature parameter. To encourage confident task-id prediction, we set a low temperature 7 0.05 to produce a low entropy task-id preidction distribution for all our experiments.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
CIL Baselines. We use 17 baselines, including 11 replay methods: iCCRLL (Rebuffi et al., 2017),
A-GEM (Chaudhry al., 2018), EEIL (Castro et al., 2018), GD (Lee et al., 2019), DER++ (Buzzega et al., 2020), HAL (Chaudhry et al., 2021), DER (Yan et al., 2021), FOSTER (Wang et al., 2022b), AFC (Kang et al., 2022), BEEF (Wang et al., 2022a), MORE (Kim et al., 2022a), ROW (Kim et al., 2023), and 6 non-replay methods: HAT (Serra et al., 2018), ADAM (Zhou et al., 2023), OWM (Zeng et al., 2019), PASS (Zhu et al., 2021), SLDA (Hayes & Kanan, 2020), and L2P (Wang et al., 2022c). We follow (Kim et al., 2022b) to adapt HAT (which is a TIL method) for CIL and call it HATCIL- Implementation details, network size and running time are given in Appendix 1.1.
Datasets. To form sequence of tasks in CIL experiments, follow the common CIL setting. We split CIFAR-10 10 into 5 tasks (2 classes per task) (C10-5T). For CIFAR-100, we conduct two experiments: 10 tasks (10 classes per task) (C100-10T) and 20 tasks (5 classes per task) (C100-20T). For TinyImageNet, we split 200 classes into 5 tasks (40 classes per task) (T-ST) and 10 tasks (20 classes per task) (T-10T). We set the replay buffer size for CIFAR-10 as 200 samples, and CIFAR-100 and TinyImageNet as 2000 samples following Kim et al. (2023). Following the random class order protocol in Rebuffi et al. (2017), we randomly generate five different class orders for each experiment and report the averaged metrics over the 5 random orders. For fair comparison, the class orderings are kept the same for all systems. Results on a larger dataset are given in Appendix D.
Backbone Architectures. We conducted two sets of experiments, one using pre-trained model and one without using a pre-trained model. Here we focus on using a pre-trained model as that is getting more popular. Following the TIL+OOD works (Kim et al., 2022a; 2023), TPL uses the same DeiT-S/16 model (Touvron et al., 2021) pre- trained using 611 classes of ImageNet after removing 389 classes that similar or identical to the classes of the experiment data CIFAR and TinylmageNet to prevent information leak (Kim et al., 2022a; 2023). To leverage the pre-trained model while adapting to new knowledge, insert an adapter module (Houlsby al., 2019) each transformer layer except SLDA and L2P.? The adapter modules, classifiers, and layer norms are trained using HAT while the transformer parameters are fixed to prevent CF. The hidden dimension of adapters is 64
The systems HAT+CSI and Sup+CSI I in (Kim al., 2022b) (which are based on the TIL+OOD paradigm but do not use a pre- pre-trained model) are not included as they are much weaker because their contrastive learning and data augmentations do not work well with pre-trained model.
SIDA fine-tunes only the classifier with fixed feature extractor and 1.2P trains learnable prompts.
7

---

Published as a conference paper at ICLR 2024
Table I: CIL ACC (%) "-XT": X number of tasks. The best result in each column is highlighted in bold. The baselines are divided into two groups via the dashed line. The first group contains non-replay methods, and the second group contains replay-based methods. Non-CL (non-continull learning) denotes pooling all tasks together to learn all classes as one task, which gives the performance upper bound for CIL. ATA is the average incremental ACC (%). Last is the ACC after lcarning the final task. See forgetting rate results in Appendix C.2. The pink rows also show the results of Non-Cl Con-CLpeu and TPL which use DeiT Pre-trained with Full ImageNet.
C10-ST
C100-10T
C100-20T
T-ST
T-1OT
Average
Last
AIA
Last
AIA
Last
AIA
Last
AIA
Last
AIA
Last
AIA
Non-CL
97.01
2014
82
87.20
82.76-112
87
87.53
72.52
77.05
72.52
77,03
81.27
85.16
OWM
41.69 663
56.00 1.6
21.39
40.10 86 16.98 41 32.58
24.55
th
45.18
17.52
35.75
24.43
41,92
ADAM
83.92*051
90.33 90.33
61,21
72.55-94
58.99-ns
20.89-05:
50.11
61,853851
49.68
61. 61,44494
60.78
71.41
PASS
86.2)5110
89, 05,055713
68.90
77.01
66,775111
7644=122
61.03=
67.12
58.34-344
67,33-163
68.
75.38
82.40"Mu2
91 1666
62,91
73.99-936
59.54-94:
09.12-1
59.22/mm
69.38-114
54,034921
65.63-1
63.62
73.84
SLDA
88. 64 100t
93.54400
77.72 67.82
67
78.51
57.95-0ms
66.03
57.93 1i
67.39
68.02
76.64
L2P
73.59*415 B4.60
61,72*981
72.88-1 It
53,81-1.99
66.52-1A:
59.12
67.81
51,09=114
61.59=1.59
60.47
71.28
ICRII
89. 12
68.90
76..00
69.15 15
77.06
53.13
61.36
51,88
63.56-514
66.12
73.64
A-GEM
56.33
68. 19
25.21
43.83
21.99
35.97"
30.53
49.26 196
21,90
39.58 11.0
31.19
47.37
EEIL
82.34*M15
90.50
68.08*951
$1.10-931
63.79-905
79.5
53.34
66.63-944
50.38-197
66.54=9M
63.59
76.86
GD
89. 16.465
94, 94,2246:7
64.36*931
80,51
60.10-944
78.43-947
53.01
67,5]
42.48=155
63.91=940
61.82
76.92
DER
84.63429
89.0 16:20
69.73-8M
80.64-274
70.03-14:
81.72-174
58.84 1231
66,55
54.20)
67.14-140
66.89
77.01
HAL
84.38+2.00
87.00*127
67,174130
77 77,42=213
67,37
77.85
$2.80
65,31
55,25=300
64,482148
65,39
74,41
DER
86.79-1a.
92.83+1.10
73.30
$2.89
72.00
82.79 1177
59.57
70.32
57,18
70.21
69.77
79.81
FOSTER
86.09
91.54 Hoer
71.69 1874
$1,16 99
72:91 -919
83.02
54.44 103
69.95
55,70 19/0
70.00
68.17
79.13
BEEF
87. 87.10
93.10
72.09*948
71888 $1.911955
81,45-034
61.419038
71.21
58.16 16
71.16-952
70.13
79.77
MORE
89. 89.164006
94,234052
70.23#221
81,24
70.53=1.8
81_59=m0
64,972128
74.03+141
63.06-126
72.74=104
71.59
80.77
ROW
90.974mm9
94.454021
74.72404
82.872041
74.60-212
83. 83,12
65.11
74.16
63.21
43.2
72,91-212
73.72
81.50
TPL. (ours)
92.33
95,11
76.53
84.10-031
76.34-113
84.46
68.64
76.77
67,20
75.72
76.21
83.23
Non-C3l es
96.9 1000
97.96
83.61-935
89.72-0010
83.61-933
88, 89 89-419
85.55
88 201
85,71-814
88, -336
87.08
90.70
TPLm
94.80
96.89
82.43-011
82.43
88.28
80.86
87.32
84.06
89.19
83.87 1907
27/40"
85, 85.22
89.42
for CIFAR-10, and 128 for CIFAR-100 and Net For completeness, we also report the results of TPL using DeiT-S/16 Pre-trained with the Full ImageNet (called TPLPH1) in the pink rows of Table 1. The results without using pre-traine pre-trained model are given in Appendix D.2.
Evaluation Metries. We use threepopular metries: (1) accuracy after learning the final task (Last in Table 1), (2) average incremental accuracy (ATA in Table 1), and (3) forgetting rate (see Table 6 in Appendix C.2, where we also discuss why the current forgetting rate formula is not appropriate for CIL, but only for TIL. The definitions of all these metrics are given in Appendix C.
5.2 RESULTS AND COMPARISONS
Table I shows the CIL accuracy (ACC) results. The last two columns give the row averages. Our TPL performs the best in both average incremental ACC (AIA) and ACC after the last task (Last). Based on AIA, TPL's forgetting (CF) is almost negligible. When the full ImageNet data is used in pre-training (pink rows), TPLpF1 has almost no forgetting in both AIA and Last ACC.
Comparison with CIL baselines with pre-training. The best-performing replay-based baseline is ROW, which also follows the TIL+OOD paradigm (Kim et al., 2022b). Since its OOD score is inferior to our principled S1æ(2). ROW is greatly outperformed by TPL. L.. The ACC gap between our TPL and the best exemplar-free method PASS is even greater, 68. 25% (PASS) vs. 76.21 % (TPL) in Last ACC. TPL also markedly outperforms the strong network expansion methods DER, FOSTER ER, and BEEF'
Without pre-training. The accuracy re- sults after learning the final task without pre- pre-training are given in Table 8 of Ap- pendix D.2. We provide a summary in Ta- ble 2 here. As L,P, SLDA, and ADAM are designed specifically for pre-trained back- bones, they cannot be adapted to the non- pre- pre-training setting and thus are excluded here. Similar to the observation in Table 1, our TPL
Table 2: CIL ACC (%) after learning the final task without pre-training (average over the five datasets used in Table I). The detailed results are shown in Table 8 of Appendix D.2.
OWN
PASS
EEIL
HAL
A.GEM
HAT
iCaRL
24.7
30.6
46.3
45.8
39.5
DER++
DER
FOSTER
BEEF
MORE
ROW
(Purs)
46.5
54.
52.2
534
51.2
53.1
$7.5
achieves the overall best results (with ACC of 57.5%), while DER ranks the second (54.2%).
5.3 ABLATION STUDY
Performance gain. Figure 2(a) shows the performance gain achieved by adding each proposed technique. Starting from vanilla HATCIL with an average Last ACC of 63.41% % over all datasets, the proposed likelihood ratio LR score (HAT+LR) boosts the average Last ACC to 71.25%. Utilizing the OOD detection method MLS (HAT+MLS) only improves the ACC to 68. 68.69% The final composition of L.R and MI. boosted the performance 76.21% %.
8

---

Published as a conference paper at ICLR 2024
100
MD KNN Constant
1.00
69.22
70.79
71.93
TPL (MSP)
890
HAT+MLSS
ALLACC (Loe))
0.80
(PL(E
HAT+LR
Er
70.73
72.22
74.01
TPL (MLS)
71.70
73.98
76.21
Constant MD Exz
KNN
C10-ST C100-10T C108-33
T-10T
C10-T
C100-10T
C100-20T
1-47
7-107
Dalend
Donaset
(a) Performance Gain
(b) Different E, v.s. Ege
(c) Different Bingir
Figure 2: Ablation Studies. Fig (a) illustrates the achieved ACC gain for each of the designed techniques the five datasets; Fig (b) displays the average ACC results obtained from different choices E, and E for eq. (7); Fig (c) showcases the results for various selections of Eiegar for TPL in eq. (9).
Different Er v.S. Ege. Recall that the key insight behind the LR score lies in the estimation of likelihood ratio. Figure 2(b) presents the average Last ACC results across 5 datasets, employing various approaches estimate P, and Ptr. In this context, the term Constant refers the use of a uniform distribution as the distribution of Par, where the energy function is a constant mapping. Our TPL approach is equivalent to employing (E, MD, E,e = KNN). The results reveal the following: (1) The incorporation of the Pt. distribution estimation is beneficial compared to assuming a uniform distribution. (2) As Pa can only be estimated using the replay data, the biigh-peforming KNN method outperforms MD. However, since MD can estimate P. without task l's training data during the test phase, it proves to be more effective than KNN when serving as Eg.
Different logit-based scores. Although (m)s()) is used as the logit-based score in Section 4.2.2, alternative logit-based scores can also be considered. In this study, we conduct experiments using popular logit-based scores MSP (Hendrycks & Gimpel, 2016), EBO (Liu et al., 2020a), and MLS (their definitions are given in Appendix F.2). The results presented in Figure 2(c) indicate that EBO and MLS yield comparable results, with average Last ACC of 75.76%, and 76.21 % respectively, while MSP has inferior performance with average Last ACC of 71.32%.
Smaller replay buffer sizes. The accuracy after learning the final task with smaller replay buffer sizes are given in Table 9 of Appendix D.3. We provide summary as Table 3, which shows that when using a smaller replay buffer, the perfor- mance drop of TPL is small. The goal of using the replay data in TPL to compute the likeli- hood ratio (LR) score, while traditional replay methods focus on preventing forgetting (CF).
Table 3: ACC (%) after learning the final task (Last) with smaller replay buffer sizes (average over the five datasets in Table 1). The detailed results shown in Table of Appendix D.3. The replay huffer size is set as 100 for CIFAR-10. 10, and 1000 for CIFAR- 100 and TinyImageNe
iCaRL
A-GEM
BEIL
GD
DER+
HAL
63.60
31.15
58.24
$4.39
62.16
60.21
DER
FOSTER
BEEF
MORE
ROW
TPL
Note that CF is already addressed by the TIL method HAT in our case. Thus our method TPL is robust with fewer replay samples.
68.32
66.86
68.94
71.44
72.70
75.56
More OOD methods. To understand the effect of OOD detection CIL, applied 20 OOD detection methods to CIL and drew some interesting conclusions (see Appendix A). (1) There exists linear relationship between OOD detection AUC and CIL ACC performances. (2) Different OOD detection methods result in similar TIL (task-incremental learning) ACC when applying HAT.
More pre-trained models (visual encoders). We also study TPL with different pre-trained models in Appendix D.5 (MAE, Dino, ViT and Dei'' of different sizes). We found the pre-trained models based on supervised learning outperform self-supervised models both CIL and TIL.
6 CONCLUSION
In this paper, we developed novel approach for class incremental learning (CIL) via task-id prediction based on likelihood ratio. Recent studies (Kim et al., 2022a;b; 2023) suggested that OOD detection methods can be applied to perform task-id prediction in CIL and thus achieve the state-of-the performance. However, we argue that traditional OOD detection is not optimal for CIL as additional information in CIL can be leveraged to design a better and principled method for task-id prediction. Our experimental results show that our TPL outperforms strong baselines and has almost negligible catastrophic forgetting. Limitations of our approach are discussed in Appendix J.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
We sincerely thank Baizhou Huang of Peking University, Shanda of Carnegie Mellon University, and the anonymous reviewers of ICLR 2024 for providing valuable suggestions this work.
ETHICS STATEMENT
Since this research involves only classification learning using existing datasets downloaded from the public domain and our algorithms are not for any specific application but for solving the general problem of continual learning, we do not feel there are any possible ethical issues in this research.
REPRODUCIBILITY STATEMENT
The source code of TPL has been public at https:://// /ttts:///gtthbb com/ foom/linhaowe www.nnanaaniiii.... 1/TPL. The proofs of Theorems 4.1 and 4.2 are provided in Appendix E. The training details and dataset details are given in Sec. 5.1 and Appendix I.
REFERENCES
Davide Abati, Jakub Tomezak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. Conditional channel gated networks for task-aware continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
3931-3940, 2020. 3.
Hongjoon Ahn, Jihwan Kwak, Subin Lim, Hyeonsu Bang, Hyojun Kim, and Taesup Moon. Ss- il: Separated softmax for incremental learning. In Proceedings of the IEEE/CVF International conference on computer vision, pp. 844-853, 2021. 3
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with network of experts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3366-3375, 2017.
Jihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung- Woo Ha, and Jonghyun Choi. Online continual learning on contaminated data stream with blurry task boundaries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9275-9284, 2022. 40
Abhijit Bendale and Terrance E Boult. Towards open set deep networks. In Proceedings of the IEEE
conference on computer vision and pattern recognition, pp. 1563-1572, 2016. 17
Prashant Bhat, Bahram Zonooz, and E. Arani. Consistency is the key to further mitigating catastrophic forgetting in continual learning. In CoLLAs, 2022. URL https:// semant icscholar.
org/Corppnn ID: 250425811111 4244866.... 40
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33: 10115522299900 2020. 7
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. Proceedings of the International Conference Computer Vision (ICCV), 2021. 27
Francisco M Castro, Manuel Marín-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In Proceedings of the European conference on computer vision
(ECCV), pp. 233-248, 2018. 7
Arslan Chaudhry, Mare' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.0044 2018. 7
Arslan Chaudhry, Albert Gordo, Puneet Dokania. Philip Tomr and David Lopez-Paz. Using hindsight to anchor past knowledge in continual learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 6993-7001, 2021. 7
10

---

Published as a conference paper at ICLR 2024
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions pattern analysis and machine intelligence, 14(7):3366-3385, 2021. 1,
Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li. Vos: Learning what you don't 't know by virtual outlier synthesis. In International Conference on Learning Representations, 2021.
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models.
Advances in Neural Information Processing Systems, 33:6637-6647, 2020. 6
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050-1059. PMLR, 2016. 17
Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In International Conference on Machine Learning. pp. 8109-8126 PMLR, 2022.
40
Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task class discrimination in online continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11878-11888 11887, 2023. 3
Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 24(12):1028-1044 2020. 3
Tyler Hayes and Christopher Kanan. Lifelong machine learning with deep streaming linear discriminant analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. pp. 220 220-221,222 2020.
Kaiming Hc. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016. 18, 24
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders are scalable vision learners. CoRR, abs/2111,06377,, 2021. URL https::/tt//att
arg//bs/21111 06377. 27
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016. 2, 9, 17
Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. arXiv preprint 1812.081000400 2018. 17
Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi. Jacob Steinhardt, and Dawn Song. Scaling ouist--diitribution detection for real-world settings. arXiv preprint /91.v 2019. 17
Christian Henning, Maria Cervera, Francesco D' Angelo, Johannes Von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F Grewe, and João Sacramento. Posterior meta-replay for continual learning. Advances in Neural Information Processing Systems, 144449, 2021. 3
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Pfrfmeter-cfficient transfer learning for nlp. In International Conference on Machine Learning, pp. 2790-2799. PMLR, 2019. 4, 7, 18
Rui Huang, Andrew Geng, and Yixuan Li. On the importance gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems, 34:677-689, 2021. 17
Kishaan Jeeveswaran, Prashant Bhat, Bahram Zonooz, and E. Arani. Birt: Bio-inspired replay 2023. URL https:
in vision transformers for continual learning. ArXiv, //api... semantiiiii commool htr.aaaiicccor.rroororrrrrrrtrto c 255855775 40
11

---

Published as a conference paper at ICLR 2024
Minsoo Kang, Jaeyoo Park, and Bohyung Han. Class inassinncremntll learning by knowledge distillation with adaptive feature consolidation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16071-16088 2022.
Zixuan Ke and Bing Liu. Continual learning of natural language processing tasks: A survey, arXiv preprint arXhi:2211. 2022. 1,3
Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. Achieving forgetting prevention and knowledge transfer in continual learning. Advances in Neural Information Processing Systems, 34: 22443-22456, 2021a.
Zixuan Ke, Hu Xu, and Bing Liu. Adapting bert for continual learning of sequence of aspect sentiment classification tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 4746-4755, 2021b.
Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. Continual pre-training of language models. In The Eleventh International Conference on Learning Represen- tations (ICLR-2023), 2023. 3
Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. arXiv preprint arXiv:1711.10560 2017. 1,3
Gyuhak Kim, Bing Liu, and Zixuan Ke, A multi-head model for continual learning via out-of- distribution replay. In Conference on Lifelong Learning Agents, pp. 548-563. PMLR, 2022a. 9, 20, 21,38 38
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. theoretical study on solving continual learning. In Advances in Neural Information Processing Systems, 2022b. 1, 2, 3, 7, 8, 9, 17, 20, 24, 35
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, and Bing Liu. Learnability and algorithm for continual learning. JCML-2003, 2023. 2, 3, 7, 9, 17,
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Bariskkaaaaii et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526,, 2017. 3
Alex Krizhevsky and Geoff Hinton. Convolutional deep belief networks on cifar- 10. Unpublished manuscript, 40(7) 2010. 38
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009. 38
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7:7, 2015. 38
Kibok Lee, Kimin Lee, Jinwoo Shin, and Honglak Lee. Overcoming catastrophic forgetting with unlabeled data in the wild. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 312-321, 2019. 7
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations, 2018a. 2
Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of--istribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018b. 2, 5, 17,31
Guodun Li, Yuchen Zhai, Qianglong Chen, Xing Gao, Ji Zhang, and Yin Zhang Continual few-shot intent detection. In Proceedings of the 29th International Conference on Computational Linguistics, pp. 333-343, 2022.
12

---

Published as a conference paper at ICLR 2024
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of ofitt--diitriiuttio image detection in neural networks. arXiv preprint arXiv:1706.02690, 2017. 17
Haowei Lin and Yuntian Gu. Flats: Principled out-of-distribution detection with feature-based like-
lihood ratio score. ArXiv, abs/2310.05083, 2023. URL httpt::ttpsppppt semant icscholar.
org/Corpus ID: 2668311773.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li, Energy- -uittr-distributtin detection. Advances in Neural Information Processing Systems, 33:2 43:21464- 2020a. 2, 17
Y. Liu, A. A. Liu, Y. Su, B. Schiele, and Q. Sun. Mnemonics training: Multi-class incremental learning without forgetting. In 2020 IEEE/CVF Conference Computer Vision and Putlern Recognition (CVPR), 2020b. 21
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017. 1,3,7 21
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks single network by iterative pruning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765-7773, 2018. 35
Mare Masana, Tinne Tuytelaars, and Joost Van de Weijer. Ternary feature masks: zero-forgetting for task-ineremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3570-3579, 2021. 35
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan, Do deep generative models know what they don't know? arXiv preprint arXiv: 2018. 5
Ibrahima Ndiour, Nilesh Abuja, and Omesh Tickoo. techniques and probabilistic modeling of features. arXiv preprint arXiv:20/2.042250 2020. 17
detection with subspace
Jerzy Neyman and Egon Sharpe Pearson. Ix. on the problem of the most efficient tests of statistical
hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of Mathematical or Physical Character, 231(694-066:28993
1933. 5, 28
Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. itaml: An incremental task-agnostic meta-learning approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588-13597, 2020. 2, 3
-ylvesstre-Alviee Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017. 1,7
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 (3):211-22) 1-222, 2015. doi: 10.1007/s1 12630001-111----11111 23
Andrei Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1XXi:16666006771 2016. 35
Joan Serra, Didac Suris. Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pp. 4548 4557. PMLR, 2018. 2, 3, 33, 38
Yijia Shao, Yiduo Guo, Dongyan Zhao, and Bing Liu. Class-incremental learning based on label generation, arXiv preprint arXiv:2306. 12619, 2023.
13

---

Published as a conference paper at ICLR 2024
Yiyou Sun, Chuan Guo, and Yixuan Li. React: Out-of- detection with rectified activations. Advances in Neural Information Processing Systems, 14:144-157, 2021. 2, 17
Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. arXiv preprint arXiv:220.00550, 2022. 2, 5, 17
Jihoon Tack, Sangwoo Mo. Jongheon Jeong, and Jinwoo Shin. Csi: Novelty detection via contrastive
learning on distributionally shifted instances. Advances in neural information processing systems, 33:11833 1852, 2020. 2
George R Terrell and David W Scott. Variable kemell density estimation. The Annals of Statistics, pp. 1265, 1992. 36
Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. Advances in Neural Information Processing Systems, 32, 2019. 17
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé
Jégou. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning, pp. 10347-10357. PMLR, 2021. 7, 17
S Vaze, K Han, A Vedaldi, and A Zisserman. Open-set recognition: A good closed-set classifier is all you need? In International Conference Learning Representations (ICLR), 2022. 2
Johannes Von Oswald, Christian Henning, Benjamin F Grewe, and João Sacramento. Continual learning with hypernetworks. arXiv preprint arXiv:1906.0065 2019. 2, 3
Fu-Yun Wang, Da-Wei Zhou, Liu Liu, Han-Jia Ye, Yatao Bian, De-Chuan Zhan, and Peilin Zhao Beef: Bi-compatible inaseinncremetal learning via ameryy-bases expansion and fusion. In The Eleventh International Conference on Learning Representations, 2022a. 3.7
Yu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for dass-incrrmental learning. In European conference computer vision, pp. 414. Springer, 2022b. 7
Haoqi Wang. Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of cutoof-distribution with virtual- logit matching. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 1921-4930, 2022c. 6, 17
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application, 2023. 3
Mengyu Wang, Yijia Shao, Haowei Lin, Wenpeng Hu, and Bing Liu. Cmg: A class-mixed generation approach to diit----ittiitttion detection. Proceedings of ECML/KKDD-2022 2022d. 2, 17
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 139-149, 2022e. 3,
Hongxin Wei, Renchunzi Xie, Hao Cheng, Lei Feng. Bo An, and Yixuan Li. Mitigating neural network overconfidence with logit normalization. 2022. 17
Ross Wightman. Pytorch image models. attps://githubs wtw.:ttwww...o....ittt..tttttt 2019. 27
Svante Wold, Kim Esbensen, and Paul Geladi. Principal component analysis. Chemometrics and intelligent laboratory systems, 2(1-3):37-52, 1987. 36
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. Advances in Neural Information Processing Systems, 33:15173-15184 2020. 3, 35
Jinlin Xiang and Eli Shlizerman. Tkil: Tangent kernel optimization for class balanced incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3539-3539, 2023.
14

---

Published as a conference paper at ICLR 2024
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 3014-3023, 2021. 3,7
Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. Openood: Benchmarking generalized outttf-distribution detection. 2022. 6, 18
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. Proceedings of the IEEE/CVF international conference computer vision, pp.
2019. 17
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependeet processing in neural networks. Nature Machine Intelligence, 1(8):364-372, 2019. 3,7
Jingyang Zhang, Nathan Inkawhich, Randolph Linderman, Yiran Chen, and Hai Li. Mixture outlier exposure: Towards out-of-distribution detection fine-grained environments. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5531-5540, 2023. 17
Da- Rei Zhou, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Revisiting class-increne iass-incremental learn- ing with pre-trained models: Generalizability and adaptivity are all you need. arXiv preprint 23030:3003.07388, 2023. 7
Fei Zhu, Xu- Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu. Prototype augmentation and self-supervision for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5871-5880, 2021. 3,7
15

---

Published as a conference paper at ICLR 2024
Appendix of TPL
Table of Contents
A Comprehensive Study TIL++ODD based methods
17
A.1 A unified CIL method based TIL+0OD
17
A.2 Experimental Sctup
17
A.3 Results and Analysis
18
B Output Calibration
20
Evaluation Metrics
21
C.I Definitions of AIA. and Last accuracy
21
C.2 Rectified Forgetting Rate for CIL
21
D Additional Experimental Results
23
D.1 CIT. Experiments on a Larger Dataset
23
D.2 CIL Experiments without Pre-training
................................
24
Experiments on Smaller Replay Buffer Size
Siee............................
25
D.4 Ablation on 31 Experiments hased More Pre-trained Models
................................
26
D.5
................................
27
E Theoretical Justifications
28
E.1
Preliminary
28
E.2 Proof of Theorem 4.1
28
E.3 Proof of Theorem 4.2
28
Distance-Based OOD Detectors are IND Density Estimators
F
Additional Details about TPL
31
E1
Computation of the MD Score
31
F2
Computation of based Scores
31
F33
Pscudo-code
31
Details of HAT
33
Training
................................
33
G.2 Inference
................................
34
G.3 Remarks
34
H Visualization of Task Distribution
36
Implementation Details, Network Size and Running Time
38
L.1 Implementation Details Baselines
38
1.2 Hardware and Software
38
I.3 Computational Budget Analysis
38
J Limitations
40
16

---

Published as a conference paper at ICLR 2024
A A COMPREHENSIVE STUDY ON TIL+OOD BASED METHODS
Based on the previous research on CIL that combines TIL method and OOD detection (Kim et al., 2022b;a; 2023), we make a thorough study on this paradigm by benchmarking 20 popular OOD detection methods. This study provides comprehensive understanding of these methods and draws some interesting conclusions, which we believe are beneficial to the future study of CIL.
A.1 A UNIFIED CIL METHOD BASED ON TIL+OOD
We first describe how to design a CIL method based on TIL+OOD paradigm. As HAT already a near-optimal solution for TIL (Kim al., 2023) which has almost no CF, choose HAT as the TIL method and only introduce variants in the OOD detection parts.
Categorization of OOD detection methods. Usually, OOD detection technique consists of two parts: (1) Training a classifier that can better distinguish IND data and OOD data in the fea- ture / logits space; (2) Applying an inference-time OOD score compute the INDness of the test case æ. For training techniques, a group of methods such as LogitNorm Wei et al. (2022), MCDropout Gal & Ghahramani (2016), Mixup Thulasidasan al. (2019), CutMix Yun al. (2019) apply data augmentation or confidence regularization to improve OOD detection; another series of methods such as OE Hendrycks et al. (2018), Mix0E Zhang al. (2023), CMG Wang et al. (2022d), vos Du et al. (2021) use auxiliary OOD data in training. For inference-time techniques, the OOD score can be computed based on feature (the hidden representations) or logits (the unnor- malized softmax score). GradNorm Huang et al. (2021), KL ML-MMachiing Hendrycks al. (2019), OpenMax Bendale & Boult (2016), ODIN Liang et al. (2017), MSP Hendrycks & Gimpel (2016), MLS Hendrycks et al. (2019), EBO Liu et al. (2020a), ReAct Sun et al. (2021) methods, and Res dual Ndiour et al. (2020), KNN Sun et al. (2022), MDS Lee et al. (2018b) are feature-based methods. Recently, VIM (Wang al., 2022c) is based on both feature and logits. We study the aforementioned 20 OOD detection methods in this section. For simplicity, we use MSP (maximum softmax probability) the OOD score for the training-time techniques, and adopt standard supervised learning with cross-entropy loss for the inference-time OOD scores.
logit-based
unified method for CIL based on TIL+OOD. We introduce unified CIL method that is based on TIL+OOD paradigm, which can be compatible with any OOD detection techniques. The details are follows.
We first recall that HAT learns each task by performing two functions jointly in training: (1) learning model for the task and (2) identifying the neurons that are important for the task and setting masks on them. When a new task is learned, the gradient flow through those masked neurons is blocked in the backward pass, which protects the models of the previous tasks to ensure no forgetting (CF). In the forward pass, no blocking is applied so that the tasks can share a lot of parameters or knowledge. Since (I) can be any supervised learning method, in our case, we replace it with an OOD detection method, which performs both in-distribution (IND) classification and out-of disttibbittiitti (OOD) detection.
In testing, first predict the task-id to which the test instance belongs, and then perform the IND classification (within task prediction) in the task to obtain the predicted class, Let S(x; t) be the OOD score æ in task based on task i's model, the task-id can be predicted by identifying the task with the highest OOD score:
== arg max S(x;
We use argmax here as OOD score is defined to measure the IND-ness the literature. Note that OOD score is produced by any OOD detection method.
A.2 EXPERIMENTAL SETUP
Backbone Architecture. Following the main experiment in our paper, DeiT-S/16 (Touvron et al., 2021) that is pre-trained using 61 I classes of ImageNet after removing 389 classes that are similar or identical the classes of the experiment data CIFAR and TinylmageNet prevent
17

---

Published as a conference paper at ICLR 2024
Table 4: Average AUC and ACC results based on pre-trained DeiT five datasets with five random seeds. Bold and underlined numbers indicate the best and second-best results, respectively. ?: logit-based OOD scores, : leature-based OOD scores, Q: training time techniques. The systems are divided into three categories by dashed lines. The first category includes post-hoe OOD detectors, the second category includes methods that exploit surrogate OOD data, and the last category includes methods that employ special training strategies.
CH0-T
C100-10T
C100-20T
T-ST
T-I0T
Average CIL
Medood
0OD
CIL
DOD
CIL
DOD
CIL
O0D
CIL
OOD
CIL
OOD
GraNNorm ?
73,85237
43.5 43.5
84.0
48.62
80,8
22.9
80.8 80.8
57. 57.04646
83.1
47.4
80.5
43.9
KI-Manching
75.2
48.4
48.751044
626
77.8
47 47.6
73.6-12
31 11.5tmmm
80.7
40.5
OpenMax V
94,1
84, 34.
84,7222
50ge002
90.6
57.9
74.1 1919
48.1
71,7
32.0
83.0
54.6
ODIN
92,3
75.6-402
89,8
61,7
92.5+
53.2-1541
85
64,5
88,5
59,5
86.4
62.9
MSP 12
93,5
82,43013
89.023
62
91,726
59
59.5t-34
81,5 10
59 59.24-921
$4.0
88.0
63.6
MLS 17
94,0-311
84ggint9
91.0-31
92,74833
86.1 86,1-991
65.4-
88,5
61.4 61.4
90.4
69.0
EBO 1
94,059
$1.gim
90.8
69. ja0
92.5
64.1
65 65.1
88,4-
61.4
90.1
69.0
RoAct
94,0a31
84.ginm
69.1 1022
92,54534
64,1
86,24999
65 63.4
88,4
61.4408
90,4
69.0
92.8
76,71025
85,98234
61.5
90,2853:
54.8
74,70400
49.9
79,2
44.4
84.6
57.5
Residual
92.4
83.0
85.0
64.72105
89,8
61,3
78,0
53. 53,741024
$1.0
85,4
62.8
MDS A
92.6-3-2
86.9
09.0
91,28531
65,4
82,0
60
84,4
56.9
87.4
VIM 4
95,4
59,0
91.0
72.8
93,5
69.8
86,359005
65.9
88,7
63.1
91.0
72.1
CMG o
42,3
50,7
85.2
56.2 $6.2
90,14254
$3.1 $3.1
80,55M12
56 9
83,1 53,1
50,5
86.2
54.5
VOS 6
93.1 23.1
82. 14031
86.7 2122
59.4 in
90.9 90.9
56.
82,4
19.71
81.8
87.8
61.8
OE Q
94.5
84.3
90,9#213
66,72131
94,0
66.24440
82,4
60,5
85,7
56.4
89.5
66.8
MixOF Q
93.6-2-2
91,7
67
62.
85
87,4
56.0
90.4
66.1
LogitNorm
93.1
14.3
91.9
81,7
84.6
53.4665
88.1
63.7
MCDropour 6
92,4
80,71077
87,8
61,7
91,38511
57,54044
80g000s
57.54000
84.0aas
$2.24002
87,3
68.5
Mixup
89.8
73
64.0
91,3
55,4
83,2
61.5
85.6
56.041044
87.9
66.5
CutMix 6
90.4
00.02001
$9,5
62.4
90.9
$0.6
82.6
61.34
85,1
55.2
87.7
59.7
information leak. We insert an adapter module (Houlsby et al., 2019) at each transformer layer. The adapter modules, classifiers and the layer norms are trained using HAT while the transformer parameters are fixed to prevent forgetting in the pre-trained network.
Evaluation Protocol. We compute AUC for OOD detection on each task model. The classes of the task are the IND classes while the classes of all other tasks of the dataset are the OOD classes. The evaluation metric for CIL is accuracy (ACC), which is measured after all tasks are learned. We report the average AUC value over all the tasks in each dataset, and the ACC of each dataset. Note that as KNN needs the training data at test time, in the CIL setting, we can only use the saved replay data of each task for its OOD score computation as the full data of previous tasks are not accessible in CIL. We also compute TIL accuracy in using difference OOD training- -ime techniques.
A.3 RESULTS AND ANALYSIS
The experiment results are given in Table 4, which allow to make some important observations.
(1) OOD detection and CIL performances. The OOD detection results in AUC here have similar trends those in Yang al. (2022) except KNN, which was considered as one of the best methods. But it is weak here because, as indicated above, in CIL, KNN can only use the replay data (which is very small) for each task compute the OOD score. The CIL performances of the top OOD methods are competitive compared to CIL basclines in Table 1.
(2) Similar TIL Performance. We present the detailed results of TIL accuracy, for the OOD detection baselines as table 5. It is evident from the results that there is minimal variation in the performance of OOD detection methods across different datasets, with average values of 9..22 95.7 12, 97.6+t0.10 1.0, 84.3+0.86, 88. 1 10.86 for C10-5T, C100-10T, C100-20T, T-5T, and T-10T, respectively. The observation that the majority of OOD methods exhibit negligible impact on the IND classification performance aligns with a previous benchmark study on OOD detection conducted by Yang et al. (2022). Notably, the aforementioned study investigated the finding on ResNet architecture (He et al., 2016) without pre-training, whereas experiments involved a pre-trained DeiT model. This suggests that the finding extends to various model backbones.
(3) Linear relationship between OOD AUC and CIL ACC. We plot the relationship between OOD AUC and CIL performances in fig. 3. Interestingly, we see a linear relationship with Pearson correlation coefficients of 0.976, 0,811, 0.941, 0.963, and 0.980 for the 5 datasets, respectively. This finding suggests that improving OOD AUC can bring about a linear improvement of x1.5 ~ 3.4 (which is the slopes of the fitted linear function) on CIL. ACC. Note that we are not conditioning
18

---

Published as a conference paper at ICLR 2024
Table 5: Average IND classification ACC (TIL accuracy) results of pre-trained DeiT five datasets with five random seeds. The baselines are divided into three categories Table the main text. The first category includes post-hoc detectors, the second category includes methods that exploit surrogate OOD data, and the last category includes methods that employ special training strategics. V: logit-hased 0OD scores, A feature- based OOD scores, Ø: training-time techniques. Note that the post-hoc methods only differ in the OOD score computation, which means they share the same trained model and thus have the same TIL ACC.
C10-5T C100-10T
C100-20T
T-ST T-10T
Average
GradNorm KL-Matching v
OpenMax V
ODIN <
MSP
MLS
99.20
95.71
97.50
84,40
88.10
92.98
EBO
ReAct V
KNN
Residual
VIM
CMG <
99. 20
95.70
97.61
84.52
88,30
93.07
VOS
99.20
95.66
97.59
84,51
88.30
93.05
99.17
95.73
97.72
84.46
88.64
93.14
MixOE
99.21
95.82
82.06
86.53
92.28
LogitNorm
99.20
95.71
97.56
84.55
87.87
92.98
MCDropout O
99.20
95.60
97.62
84,51
88.23
93.03
Mixup O
99.24
95.57
97.71
85.04
88.47
93.21
CutMix O
99.20
84.54
88.40
90
C19-ST
C100-10T
C100-207
T-ST
70
CIL Rosults (ACC)
50
1s
85
90
86
Figure 3: The correlation between OOD (AUC) and CIL (ACC) results. Each point denotes the AUC and ACC one method in table on the same dataset.
on TIL accuracy (2) above showed the TIL results are similar for different training time OOD techniques.
19

---

Published as a conference paper at ICLR 2024
B OUTPUT CALIBRATION
We used the output calibration technique to balance the scales of task-id prediction scores for different tasks, which is motivated by Kim al. (2022a;b), Even if the task-id prediction of each task-model is perfect, the system can make an incorrect task-id prediction the magnitudes of the outputs across different tasks are different. As the task task-specific modules are trained separately HAT, it useful to calibrate the outputs of different task modules.
To ensure that the output values comparable, calibrate the outputs by scaling of and shifting (t) 02 parameters for each task. The optimal parameters {(o1") o ) E Rº (T the number of tasks) can be found by solving optimization problem using samples the replay buffer Buf.
Specifically, we minimize the cross-entropy loss using SGD optimizer with batch size 64 for 100
epochs find optimal calibration parameters {(o) 02 )
(1) (f)
Leatibration (m)) log p(y[z),
where ply a)) is computed using eq, (3) and calibration parameters:
pl, 1x) (x) of |sottmax (f(h(z; ) S(x;;) t
Given the optimal parameters { (4) ( ) we make the final prediction as:
P(Uj (0) [x)
i = arg max
20

---

Published as a conference paper at ICLR 2024
C EVALUATION METRICS
C.I DEFINITIONS OF AIA. AND LAST ACCURACY
Fllowing (Kim al., 2022a), we give the formal definitions of average incremental accuracy (AIA in Table in the main text, denoted as AaAA ) and accuracy after learning the final task (Last in Table 1
in the main text, denoted Atas)) Let the accuracy after learning the task be:
A #correctly classified samples in U D(A))
#samples in U2=11 Diest
Let T be the last task. Then, Atn A(ST) and Aua
A(SA)
Here ((k) denotes the test-set for task k, and # denotes "the number of". To put it simply, A(St) means the accuracy of all the test data from task I to task t.
C.2 RECTIFIED FORGETTING RATE FOR CIL
Apart from the classification accuracy (ACC), report another popular CIL evaluation metric average forgetting rate. The popular definition of average forgetting rate is the following
F(0
Σ
(A ((").
where A!") is the accuracy of task i's test set on the CL model after task is learned (Liu et al., 2020b), which is also referred to as buckward transfer in other literature (Lopez-Paz & Ranzato, 2017).
However, this formula is only suitable for TIL but not appropriate for CII. As the task-id for each test sample is given in testing for TIL, all the test samples from task i will be classified into one the classes of task i. If there is no forgetting for TIL model, then A will be equal to A (2 t) in such a within-task classification, where the number of classes fixed. But in CII,, the task-id not provided in testing and we are not doing within-task classification. Even if in the Non-CI setting as more tasks or classes are learned, the classification accuracy will usually decrease for the same test set due to the nature of multi-class classification. That is. A < ( (i < f) is usually true as task there are fewer learned classes than at task t. Their difference not due to forgetting.
Furthermore, as we discussed in Section 1, forgetting (CF) is not the only issue of CIL. Inter-class separation (ICS) another important one. When considering the performance degradation of each task during continual learning, it is hard to disentangle the effects of these two factors. Our rectified average forgetting rate metric for CIL considers both forgetting and ICS. Two new average forgetting rates for CIL are defined, one for the case where we use the Last accuracy as the evaluation metric and one for the case where we use AIA as the evaluation metric:
(1)
1
(A (1,NC)) 11
CIL, Last
2
F(0) AIA
1
Σ
(()
t
CIL
Lest"
where NCL means Non-CL, (1 the accuracy of task i's test set on the CL model after task is learned, and A (L.NCL) is the accuracy of task i's test set on the Non-CL model that learns all tasks from to t. If the test dataset sizes are the same across different tasks, then F Lay is equal to Atan, NCL Ataut where Ajaz is defined in appendix C. and Agast NCL is the Atas of Non-CL. CL. The intuition of introducing NCL performance is to address the loss of accuracy for each task when more tasks are learned.
21

---

Published as a conference paper at ICLR 2024
Table 6: Forgetting rate (%) for CIL the five datasets of the baselines in Table 1. The lower the the better the method
C10-ST
C100-10T
C100-20T
T-ST
T-10T
Average
Method
(
CIL Los
CRL Lar
CIL
CIL Lis
AMA
F:T
Lort
CIL
CIL Lest
J
CIL.M.
CIL Last
CIL Al
OWN
$4.10
41.01
61.37
47.10
65.78
54.95
47.97
31.85
55.00
41.28
56.84
43.24
ADAM
11.87
6.68
21.55
14.65
23.77
16.64
22.41
15.18
22.84
15.59
20.49
13.75
PASS
9.58
7.98
13.86
10.19
15.99
11.11
11.49
9.91
14.18
9.70
13.02
9.78
HAT
iCaRL
39,46
28.82
57.55
43.37
60.77
51.56
41.99
27.77
50.62
37.45
50.08
37.79
A GEM
8.24
7.27
13.86
10.70
13.61
10.47
19.39
15.67
20.64
13.47
15.15
11.52
FEIL
13.45
6.51
14.68
6.10
18.97
7.99
19.18
10.40
22.14
10.49
17.68
8.30
GD
6.63
2.79
18.40
6.69
22.66
9.10
19.51
9.52
30.04
13.12
19.45
8.24
DER+
6.71
15.34
HAL
11.41
10.01
15.59
9.78
15,39
9.68
19.72
11.72
17.27
12:55
15.88
10.75
DER+
11.16
8.00
13.03
12.73
5.81
16.68
10.48
18.32
9.89
14.38
8.15
FOSTER
9.70
5.47
11.07
6.04
9.85
4.51
18.08
7.08
16.82
7,03
13.10
6.03
BEEF
8.69
3.91
10.67
5.29
10.88
6,08
11.11
5.82
14,36
5.87
11.14
5.39
MORE
4.39
ROW
4.82
2,56
8.04
4,33
8,16
4.41
7.41
2.87
9.31
4.12
7.55
3.66
TPL
1.90
6.23
6.42
3.07
0.26
5.32
1.31
5.06
TFLM
2.04
1.07
1.18
1.44
2.25
1.57
1.49
1.07
1.84
1.26
1.86
1.28
Table 6 shows the average forgetting rates of each system based on the new definitions. We clearly observe that TPL. and TPL.F have the lowest average forgetting rates on the five datasets among all systems. SLDA and L.2P not included they use different architectures and cannot take the Non-CL results in Table in the main text as the upper bounds or NCL results needed in the proposed formulas above.
is important to note that for both TPL and TPL the forgetting rate mainly reflects the performance loss due the ICS problem rather than the traditional catastrophic forgetting (CF) caused by network parameter interference in the incremental learning of different tasks because the TIL method HAT has effectively eliminated CF in TPL and TPLpF.
22

---

Published as a conference paper at ICLR 2024
D ADDITIONAL EXPERIMENTAL RESULTS
D.1 CIL EXPERIMENTS ON A LARGER DATASET
Table 7: The CIL ACC after the final task on ImageNet380-10T. We highlight the best results in bold.
HATeI 71,20
ADAM
SLDA
PASS
L2P
iCaRL
A-GEM
EEIL
65.78
65,27
47,89
62,23
30.36
63.375004
DER++
HAL
DER
FORSTER
BEEF
MORE
TPL
66.5342M
64.83
69.19
68.07-18
70.07
72.10
74.52
78.49
To the performance of our proposed TPL on large-scale datasets, we ImageNet-Ik (Rus- sakovsky et al., 2015), widely recognized benchmark dataset frequently examined in the CIL literature. However, due to the nature of our experiments, which involve DeiT backbone pretrained on 611 ImageNet classes after excluding 389 classes similar to those in CIFAR and TinyImageNet, we cannot directly evaluate our model on the original ImageNet dataset to avoid potential information leak.
To overcome this limitation, we created new benchmark dataset called ImageNet-380. We ran- domly selected 380 classes from the remaining 389 classes, excluding those similar to CIFAR and Tine from original set classes in the full ImageNet dataset. This new dataset consists of approximately 1,300 color images per class. For ImageNet-380, into 10 tasks, with each task comprising 38 classes. We set the replay buffer size to 7600, with 20 samples per class, which is commonly used number in replay-based methods. We refer these experiments as ImageNet380-1011. For other training configurations, we kept them consistent with the experiments conducted on T-10T.
divided the classes
The CIL Last ACC achieved after the final task on ImageNer380-10TT can be found in Table 7. Notably, the results obtained by TPL still exhibit significant improvement over the baselines, with 3.97% higher ACC compared to the best baseline method ROW. The results further provide strong evidence supporting the effectiveness of our proposed TPL system.
23

---

Published as a conference paper at ICLR 2024
D.2 CIL EXPERIMENTS WITHOUT PRE-TRAINING
Table 8: CIL accuracy (%) after the final task (last) based on ResNet-18 without pre-training over 5 runs with random seeds. "-XT": X number of tasks. The hest result in each column is highlighted in hold.
C10-ST
C100-10T
C100-20T
T-ST
T-10T
Average
OWM
51,8
28.9
20.26
10.0
8.6
24.7
PASS
47.3
33.0
25,0#t099
28.4+051
19.1
30.6
BEIL
64.5 $4.5
52.3 t:0.3
48 48.0%10.4
38.2#0054
28.7#088
46.3
GD
65.5
51.4
50,3
38.9
29.5
47.1
HAL
63.7
51.3 1:22
48 4
38.1 40.97
30.3 10.301100 L.5
46.4
A-GEM
64.7
50,5 23.73
47.3
37.3
29.4 29.4
45.8
HAT
62.7
41.1 #0.93
25.6 25.6##051
38.5
29.8
39.5
iCaRI.
63.4
51.4
47.8 47.8
37.0
28.3=t0.18
45.6
DER+
66.0
53,7
46.6*1.4
35.8
30,5
46.5
DER
62. 1 #097
64.5 -0.85
62.5
43,6 43.6+t0.77
38.3 38.3#082
54.2
FOSTER
65.4 1.00
62.5
62.54:0.84 1:0.84
56,3#0071
40.5 40.92
36.4 36.440885
52.2
BEEF
67.3
60.9
56,72072
44. 20.85
37.9
53.4
MORE
70,6 20.74
57.5 $7.5 t0.68
51.3#0099
41.2
35.4
51.2
ROW
74.6 +0.8
58.2
52.1 52.1 #091
42,3
38.2
53.1
TPL
78,4
62.2 62.2 +0.52
55,8
48.2
42.9
57.5
The experimental setup and results for CIL baselines without pre-training presented in this section.
Training details. We follow Kim et al. (2022b) to use ResNet-18 (He et al., 2016) for all the datasets (CIFAR-10, CIFAR-100, TinylmageNet) and all the baselines excluding OWM. MORE, ROW, and our TPL are designed for pre-trained models, and we adapt them by applying HAT on ResNet-18 18. All other baselines adopted ResNet-18 in their original paper. OWM adopts AlexNet as it is hard apply the method to the ResNet. For the replay-based methods, we also use the same buffer size as specified in Section 5.1. We use the hyper-parameter suggested by their original papers. For MORE, ROW, and our proposed TPL, we follow the typer-parameten used in HAT for training.
Experimental Results. The results for CIL Last accuracy (ACC) after the final task are shown in Table 8. We can observe that the detwork-expannsee and approaches that predict task-id based on TIL+OOD (MORE, ROW) are two competitive groups of CIL baselines. Our proposed TPL achieves the best performance DER achieves the best on C100-10T and C100-20T. Overall, our proposed TPL achieves the best average accuracy over the 5 datasets (57.5%) while DER ranks the second (with an average ACC of 54.2%).
based approaches (DER, FOSTER, BEEF)
C10-ST, T-5T, and T-10T, while
24

---

Published as a conference paper at ICLR 2024
D.3 CIL EXPERIMENTS ON SMALLER REPLAY BUFFER SIZE
Table 9: CIL. accuracy (%) after the final task (Last) with smaller replay buffer size over 5 runs with random seeds. "-XT": X numher of tasks. The best result in each column is highlighted in bold. The replay buffer size is set 100 for CIFAR-10, and 1000 for CIFAR- 100 and TinylmageNet. The pre-trained model is used.
C10-ST
C100-10T
C100-20T
T-ST
T-10T
Average
iCaRL
86.08
66.964208
68, 16.177771
47.27
49,51
63.60
A-GEM
56.64
23.18
20.76
31.44
23.73
31.15
EEIL
77.44 73.44
62.95
57.86
48.36
44.59=1.72
58.24
GD
85.96-1164
57.1741106
50,30
46.09
32.41
54.39
DER+
80.09
64.89-12.18
65.84-1144
50.7442244
49.24
62.16
HAL
79.16
62.65
63.96*1149
48. 48.17
47.11
60.21
DER
1:144
72.31 20078
70.25 20.98
58.07
55,85=1.23
68.32
FOSTER
84.99
70.25
71.14
53.35
54.58
66.86
BEEF
86.20
70.87+2.77
70,44 1..24
65.15
57.02=
68.94
MORE
88.13
71.69
71.29
64.1744077
61.90
71.44
ROW
89.70
73.63
71,86 71.007
65.42
62.87=0053
72.70
TPL
91.76 91.76.10.44 10.44
75.83
4.0228
75.65
75.65
68.08 10.61
66.48 10.47
75.56
25

---

Published as a conference paper at ICLR 2024
D.4 ABLATION ON $1 AND 32
Table The CIL accuracy (%) after leraning the final task (Last) of TPL C10-5T for different S1 and 82.
f2
15.0
17.5
20.0
22.5
25.0
31
91.9
91.9
91.8
92.5
92.2
0.6
92.1
91.0
92.2
91.6
92.0
0.7
91.8
92.1
92.3
92.4
91.9
0.8
92.0
92.4
91.7
91.9
91.7
0.9
92.3
92.5
92.0
91.9
91.5
31 and B2 are scaling hyper-parameters used in the definition of task-id prediction score
(1 (ar). Table 10 shows an ablation for different 31 and 32 on C10-5T, which indicates that 81 and B2 do not affect results much. For other hyper-parameters used in TPL, see Appendix I.1 for more details.
26

---

Published as a conference paper at ICLR 2024
D.5 EXPERIMENTS BASED ON MORE PRE-TRAINED MODELS
Table 11: TIL and CIL accuracy results (after last task learned) for TPL based on different pre pre-trained visual encoders or models.
Visual Encoder
C10-ST
C100-10T
C100-20T
T:ST
T-JOT
Averuge
Pre-training
TIL
CIL
TIL
CIL
TIL
CIL
TIL
CIL
TIL
CIL
TIL
CIL
DeiT-smull-- IN661 (TPL.
supervised
99.20
92.33
95.71
76.53
97:50
76.34
84.40
68.64
88.10
67.20
92.98
76.21
ViF-tiny
96.80
91.18
95.36
76.79
97.52
75.83
82.37
71.85
85.10
70.18
91.83
77.21
Dei DaiE-riny
ViT-small
supervised
98.85
99.43
90.79
95.57
94.80
97.51
84.52
14.01
97.67
98.76
73.21
83.94
83.58
89.49
81.80
72.46
86.54
91,30
71.71
80.94
95.30
92.29
76.44
DeiT-small (TPL
99.24
94,86
96.79
82.43
97.78
50.86
89.78
84.06
92.51
83.87
95.22
85.22
SiT-small-Dino
self-sueerviied
98.75
87.82
94.51
73.83
96.95
72.62
80.77
68.78
82.49
65.97
90.69
73.80
VIT-bsseMMB
sell-supervised
99.09
88.82
93.42
67
96.77
69.52
79 58
65 65.94
81,76
90.12
70.97
In this section, we conduct ablation study on the visual encoder (pre-trained model/network) To prevent data contamination or information leak, TPL uses DeiT-S/16 pre-trained with 61 classes of ImageNet (DeiT-small-IN661) after removing 389 classes that overlap with classes the continual learning datasets. Here we dismiss this limitation and experiment more open-sourced tre-trained visual encoders trained using the full ImageNet. The results are reported in Table 11, which also includes the TIL accuracy results if the task-id is provided for each test instance during testing. We show DeiT-small-INN611 which is our TPL, the first row of the table. It is the backbone used in our main experiments (Table 1) (IN: ImageNet). The other visual encoders are open-sourced in the timm (Wightman, 2019) library. We note that vanilla VIT and DeiT are pre-trained on ImageNet using supervised training, thereby there exists an information leak for CIFAR and IingeeeteeNet datasets used in continual learning. The details of the models are as follows:
ViT-tiny: The full model name timm is -iii_ttny__iieeeei fallyppacch.. 12__16____2 ___2224". It trained on ImageNet
(with additional augmentation and regularization) using supervised learning.
DeiT-tiny: The full model name in timm is "deit_ tat___iiyy__hhhh66_1 ___2""__ It is trained on ImageNet (with additional augmentation and regularization) using supervised learning and distillation.
ViT-small: The small version of ViT
(_22""). ViT-small- The small version of ViT trained with self-supervised DINO method (Caron et al., 2021).
DeiT-small: The small version of DeiT
ViT-base-MAE: The base version of ViT trained with self-supervised MAE method (He et al., 2021).
Analysis. We found that different pre-trained visual encoders have varied CIL results. Compared to DeiT-small-IN661, the pre-trained small VIT and DeiT that the full ImageNet conduct supervised learning have overall better performance, which is not surprising due to the class overlap we discussed above. For the sule-supervised visual encoders (Dino, MAE), the performances worse than the supervised pre-trained visual encoders. We hypothesize that the variance between different visual encoders is mainly rooted in the learned feature representations during pre-training. It is an interesting future work to design optimal pre- pre-training strategy for continual learning that does not need supervised data.
27

---

Published as a conference paper at ICLR 2024
E THEORETICAL JUSTIFCAAIOS
E.1 PRELIMINARY
In this section, we first give some primary definitions and notations of statistical hypothesis testing. rejection region and uniformly most powerful (UMP) test.
Definition (statistical hypothesis testing and rejection region) Consider testing a null hypothe- sis Ho : 0 E Oo against an alternative hypothesis H1 O₁ where Og and O are subsets the parameter space θ and On n 1 0. A test consists of a test statistic T(X), which is a function of the data æ, and rejection region R, which a subset of the range of If the observed value of falls in R, we reject Ho-
Type I error occurs when we reject true null hypothesis Hg. The probability of making Type I error is usually denoted by a. Type error occurs when we fail to reject a false null hypothesis willing to risk rejecting Ho when it is true.
Ho. The level of significance is the probability
Typically a = 0.1,0.05, 0.05, 0.01 are used.
Definition 2 (UMP test) Denote the power function BR( (x(T(x) E R), where Pg denotes the probability measure when 0 the true parameter. A test with a test statistic and rejection region R is called a uniformly most powerful (UMP) test at significance level o if it satisfies two conditions:
1. supage, (x( a.
2. v0 11,3R(0) 2 BR (0) (0) for every other test L with rejection region R' satisfying the first condition.
From the definition we the UMP test ensures that the probability of Type I error is less than (with the first condition), while achieves the lowest Type II error (with the second condition). Therefore, UMP is considered optimal solution in statistical hypothesis testing.
E.2 PROOF OF THEOREM 4.1
Lemma E.1 (Neyman & Pearson, 1933) Let {X1, X2, ..., Xn) be random sample with likelihood
function I(f). The UMP test of the simple hypothesis Ho = Og against the simple hypothetis
Ha 8 = 02 level has rejection region of the form:
L(0o)
L(O.)
where k is chosen so that the probability of type I error is a.
Now the proof of Theorem 4.1 is straightforward. From Lemma E.1, the UMP test for Equation (4) the main text has a rejection region of the form:
(x (
Ao
Pre (x)
where Å is chosen that the probability of a type error is
E.3 PROOF OF THEOREM 4.2
Note that the AUC is computed as the area under the ROC curve. A ROC curve shows the trade-off between true positive rate (TPR) and false positive rate (FPR) across different decision thresholds. Therefore,
28

---

Published as a conference paper at ICLR 2024
AUC
(TPR) d(FPR)
(12)
(1 -PRR d(TPR)
(13)
Bie d(1 - (re)
(14)
J/ (8() (3)(()))
(15)
where FPR and TPR are false positive rate and true positive rate. Therefore, an optimal AUC requires UMP lest of any given level a =3x(0.)) except on null set.
E.4 DISTANCE-BASED OOD DETECTORS ARE IND DENSITY ESTIMATORS
In this section, show that (a)( (MD: Mahalanobis distance) and Sxwv(x) (KNN: k-nearest neighbor) defined in Equation (16) and Equation (17) are IND (n-distribution) density estimators under different assumptions (We omit the superscript (1) for simplicity):
(MD(2) m/min( = mo) " 1 pe),
(16)
cE>
SKNN(a:; (a;; D) = --|12
;
(17)
In Equation (16), Me the class centroid for class e and E is the global covariance matrix, which
are estimated on IND training corpus D. In Equation (17), || -112 is Euclidean norm, z" = x //||2||
denotes the normalized feature z, and D denotes the set of normalized features from training set D. kNN(="; D*) denotes the k-nearest neighbor of 2t in set D
Assume we have feature encoder & X -> Th"", and training time we empirically observe n IND
samples 1 ((x2)...6(xx,.)
Analysis of the MD score. Denote E to be the covariance matrix of 6(x). The final feature we extract from data is:
=xx) = A (())
where AAT Σ. Note that the covariance of = I.
Given class label we assume the distribution =(x]ec) follows Gaussian ((A Me; Immedi- ately we have μe to be the class centroid for class under the maximum likelihood estimation. We can now clearly address the relation between MD score and IND density (p(æ));
(x)) = 1/(-2 max (In p(zlc)) m In 2x)
cEy
Analysis of KNN score. The normalized feature xxx) d(xx) (=))]] used for OOD detection. The probability density of z can be attained by:
p(z' B(z.r) r
p(z) lim
[B(2r)]
where B(z,r) = (2': |||² - 2112 S r ||'1|| 13
Assuming each sample 2(x;)) is i.i.d with probability mass 1/n, the density can be estimated by KNN distance. Specifically,, r = || kNN(=)]l2. p(z' E B(z,r) = k/n and [B(2r)] = n (m 1)/2
r( 2 +1) m- + o(1m where r Fuler's gamma function. When n large and k/n is
small, we have the following equations:
m + 1) kI m-1
p(x) ~
n (m (m-1))/ 2
29

---

Published as a conference paper at ICLR 2024
kI m=1 I (m 1
1): m (p(x)))
Skuv( R
2
Recall that the CIL methods based on the TIL+OOD paradigm (i.e., MORE and ROW) use Sud (a) to compute the task-prediction probability. As analyzed above, the MD score is in fact IND density estimator, which means SaD (a) measures the likelihood of the task distribution P,. Therefore, the TIL+OOD methods ignores the likelihood of the distribution of other tasks (Pı=), which may fail to
make the accurate task prediction. We put the detailed analysis in Appendix H.
30

---

Published as a conference paper at ICLR 2024
F ADDITIONAL DETAILS ABOUT TPL
F.1 COMPUTATION OF THE MD SCORE
Mahalanobis distance score (MD) an OOD score function initially proposed by Lee et al. (2018b), which is defined as:
(a = 1/ min ((h(x) -me -m))))
ccy
where (()) is the feature extractor of tested OOD detection model M. M4 is the centroid for class c and is the covariance matrix. The estimations of Mr and E are defined by
Σ h(x),
Me
No DF DE
Σ cE|D" WED
Σ
- (c) - po)
where iinin {x: (x, v) E Diminn = c). Donin is the training set, N is the total number of training samples, and Ne is the number of training samples belonging to class c.
In the CIL setting, have to compute Me and Σ for each task (assuming all classes in the task have the same covariance matrix) with trained task-specific model M Specifically, after training on the 1-th task dataset D(O), we compute:
k = No
Σ
(z)
Vc 22(4)
(18)
(m)))
( [
Σ
Σ
(h(x;; u (M(x; -
(19)
cE|])O]
(x,o)ED())
We put the memory budget analysis of the saved class centroids He and co-variance matrices (() in Appendix I.3.
F.2
COMPUTATION OF LOGIT- BASED SCORES
In Section 5.3, compare the performance of TPL with different logit-based scores MSP, EBO, and MLS. They are defined as follows:
s(()
(M) = max softmax (f(((z;)))) ((0)); 801))
D4
(20)
j=1
Pil
s(t) 5 (x) log
exp (1((200000))) )1))))))
(21)
==1
DDa max j=1
((0));
(22)
In traditional OOD detection works, they are tested effective in estimating the probability of " belongs to IND classes". Thus we can adopt them to design our TPL, method to estimate the probability of "æ belongs to task L".
F.3 PSEUDO-CODE
To improve reproducibility, provide the detailed pseudo-code for computing Syrz(x) (using Equa- tion (9) in the main text) as Algorithm 1. Then we give the pseudo-code for the CIL training Algorithm 2 and testing for TPI. as Algorithm 3.
31

---

Published as a conference paper at ICLR 2024
Algorithm 1 Compute TPL Score with the (-th Task-specific Model M
Input: Buf": replay buffer data without classes of task a: sample; task-id; M the trained t-th task model (M( with feature extractor lt(;; xf*) and classifier f(æ; 010); ); {ul Jeepros pre-computed class centroids for task I; E(; pre-computed covariance matrix for task I; k: KNN nyper-parameter; 1/ pre-computed empirical mean of SMis());; 1/8 pre-computed empirical mean Smp (x).
Return: TPL Score (T))
1: SMLs(w) + maxe f(h(2;; o(0); )²
2: (m +- 1/ (mincexco ((((a))((()) - =- (01) (n)))
3: SML((#) + SmLs(a)) 1
4: SmD(a) (x) (x) * (,35)
5: x h(z; (())))))
6: for á; in Buf do
7:
2; ==; ob // 1
8:
d, 112 2
9: end for
{d ),ar"' orteed(1g Bud")
10:
11: STPL(x) +- loglexp( -Sap((-sss(()))) + exp( (Mm(()) dr
Algorithm 2 CIL Training with TPL
Initialize an empty replay buffer Buf
2: for training data D(t) of each task do
3: for each batch (az,, My) C DD() U Buf. until converge do
4: Minimize Equation (2) (in the main text) and update the parameters with HAT
5: end for
6: Compute {nL (1) Feeya using Equation (18)
7: Compute (() using Equation (19)
8: Compute 1/ 1/8(t) using Equation (10)
9: Update Buf with D(()
10: end for
Train the calibration parameters (( ) following Appendix B
Algorithm 3 CIL Testing with TPL
Input:
test sample æ Return: predicted class à
1: Compute STPI [x; t) with each task model ( using Algorithm 1.
2: Preicition with Equation (3): arg max, af [sofmaxx (f(((;; ((1)); ((0))) -(xx) S(x;t)+0f
32

---

Published as a conference paper at ICLR 2024
G DETAILS OF HAT
G.1 TRAINING
For completeness, we briefly describe the hard attention mechanism of HAT (Serra al., 2018) used in TPL. In learning the task-specific model M(t for each task, TPL at the same time trains a mask for each adapter layer. To protect the shared feature extractor from previous tasks, their masks are used to block those important neurons so that the new task learning will not interfere with the parameters learned for previous tasks. The main idea is to use sigmoid to approximate 0-1 gate function hard attention to mask or unmask the information flow to protect parameters learned for each previous task.
The hard attention at layer and task é defined as:
(t) ai
sigmoid(s (
where is temperature scaling term, sigmoid(-) denotes the sigmoid function, and e a learnable embedding for task t. The attention is element-wise multiplied to the ouptut by of layer I as
h/ = x h1
The sigmoid function converges to a 0-1 binary gate as goes to infinity. Since the binary gate not differentiable, fairly large s is chosen to achieve differential pseudo gate function. The pseudo binary value of the attention determines how much information can flow forward and backward between adjacent layers. Denote h4 ReLU
bi), where ReLU(-) is the rectifier function.
For neurons of attention aj with zero values, we can freely change the corresponding parameters in Wi and b, without interfering the output h/. The neurons with non-zero mask values are necessary to perform the task, and thus need a protection for catastrophic forgetling (CF).
Specifically, during learning task modify the gradients of parameters that are important in performing the previous tasks 1, 1,2,... - so they are not interfered. Denote the accumulated mask by
max (a (<t-1) aaaa-1)) ((-1)).
where max(...) is an element-wise maximum and the initial mask a, (0) is defined as a zero vector. (<t) a collection of mask values at layer where a neuron has value if has ever been activated previously. The gradient of parameter Wij. is modified as
aj
(c0) (ct) Vulga = min(a (1.-1))
(<() is the i-th unit of aj (<)) The gradient flow is blocked if both neurons in the current where a layer and j in the previous layer have been activated. We apply the mask for all layers of adapters except the last layer. The parameters in last layer do not require protection as they are task-specific parameters.
A regularization is introduced to encourage sparsity in aj and parameter sharing with a (<t) The capacity network depletes when a (<t) becomes an all-one vector all layers. Despite a set of new neurons can be added in network any point in training for more capacity, we utilize resources more efficiently by minimizing the loss:
Lieg ΣE 2 a (50)
1
E1,,11- 0, (<))
The intuition of this term is regularize the number of masked neurons. Then the loss of HAT defined as the second term from the R.H.S. of Eq (7) in the main text is:
CHAT Lreg:
where μ is a hyper parammtr to balance the optimization of classification objective and HAT regular- ization.
33

---

Published as a conference paper at ICLR 2024
G.2 INFERENCE
In CIL without task identifier during inference, we are required to forward input data across each task to derive task-specific features (² for each task = 1, 2, ... .T This approach can result in computation overhead, especially with extended task sequences. This section proposes to use parallel computing (PC) to mitigate this by achieving comparable time efficiency to one-pass CL methods, albeit with trade-off of increased memory usage by factor of T compared to the one-pass CL methods for latent feature storage, a vector of 384 floating point numbers.
Consider the model M comprising two components: feature extractor h(-) and classifier f()). Un- like the standard model, where both extractor and classifier are universal across tasks, our model uses shared feature extractor with task-specific classifiers for each task. We analyze the computational costs for each component separately.
For the feature extractor, break it down into layers. Each layer involves an affine transformation (with weight W, and bias b,) followed by an activation function, specifically ReLU(-)). In the standard model, computation at each layer follows:
h = ReLU
(23)
For our model, the computation is extended to:
(c) htg ai (t) & ReLu - bi), t=
(24)
Here, dj denotes the stored hard attention at layer / for task t. Comparing the equations, the additional operation in our method is the element-wise product, which is efficiently parallelizable
by vectorizing the sets {h1 (t)) )T_ (h(1)T_ > and (a, TT_ into matrices H, H and A, This parallelization achieves near-equivalent time consumption the standard model, with the trade-off of f'ld increase in memory usage as our feature Hi and Hi-1 are T times larger compared to h and h
For the classifier, each task employs simple affine transformation (weight (t) and bias b())). The standard model uses W E Hxx and E IRC where H and C' represent the hidden size and class count, respectively. The standard model's computation is:
logits Whe+ +
(25)
In contrast, our method involves T' task-specific classifiers, each with weight ( E #x and bias (()) E f. The task-specific logits are calculated as:
logits (t) w h
(1) h (t)
(26)
By vectorizing task-specific weights, features, and biases, time consumption is maintained at levels comparable the standard model, and memory consumption remains unaffected.
In conclusion, our approach achieves similar time efficiency as the standard model in CL inference, the expense of increased runtime memory for storing sse-iipe features. This trade-off also allows flexibility between memory usage and time consumption by adjusting the level of parallelism, offering a balance between runtime duration and memory requirements. For example, one can reduce the running time memory by forwarding the input more times to the model with lower parallelism level each time.
G.3 REMARKS
is important note that our method can also leverage some other TIL methods to prevent CF other than HAT. The reason we chose HAT is to make it easy to compare with previous techniques (e.g..
This aligns with adapter implementation approach.
34

---

Published as a conference paper at ICLR 2024
MORE and ROW) based on the same setting. For example, et al., 2020), which has already been applied in Kim et al. (2022b) to construct an effective TIL+OOD similar to that using HAT. Furthermore,
also exploit SupSup (Wortsman
method and the empirical performance also compatible with other architecture- TIL methods such as PNN (Rusu et al., 2016), PackNet (Mallya & Lazebnik, 2018) 8) or Ternary Masks (Masana al., 2021).
method
35

---

Published as a conference paper at ICLR 2024
H VISUALIZATION OF TASK DISTRIBUTION
The key theoretical analysis behind TPL, theorems 4.1 and 4.2, suggest that an accurate estimate of Pt= (the distribution of the other tasks, i.e., the complement) is important. If we only estimate P. (the feature distribution of task t) when doing task prediction and assume Pt to be uniform distribution (which was done by existing methods analyzed in Appendix E.4), there will be potential risks as shown by the toy example on ID Gaussian Section 4.1.
Intuitively, the failure happens when P. and Pt. have some overlap. A test case 21 may have higher likelihood in Pe than another test case 22, but it gets even higher likelihood in Ptr, then æ will be less likely to be drawn from Pt. In this section, plot P. and Peo to demonstrate this phenomenon.
Recall that in our CIL scenario, we compute the likelihood ratio for each task to estimate the task prediction probability. For each task I, we compute the likelihood of input data under the task distribution Pt. and use the saved replay data from the other tasks to estimate P.c. Notice that as we do conduct task prediction training, all the discussion here is about inference or test time and we have the small among of saved data from all the learned tasks in the memory buffer. Our goal is to analyze whether P, and P:e have some overlap.
Specifically, we draw the feature distribution of the five tasks on C1O-ST after all tasks are learned under the pre-training setting. To facilitate an accurate estimation of Pe and Pte. we use the whole training set of CIFAR-10 to prepare the extracted feature it contains more data. The vanilla features are high-dimensional vectors in R334, and we use Principled Component Analysis (PCA) (Wold et al., 1987) to project the vectors into IR2 We then use Kernel Density Estimation (KDE) (Terrell & Scott, 1992) to visualize the distribution (density) of the data. The figures are shown as follows:
Distribution
Distribution Task Distribution other tasks
Distribution other tasks
Distribution Distribution of Task 3 Distribution on of other tasks tthertasks
Distribution Distribution of Task Distribution of other tasks
Distribution
Distribution of Tosk 4 Biution of other tasks
Figure 4: Visualization of feature distribution of Task (1=1,2,3,4.55 data and other 4 tasks. We use the trained task- feature extractor h(æ; (()) to extract features from the the training data that belongs to task t (which represent P.) and the training data that belongs the other 4 tasks (which represent Ptc).
36

---

Published as a conference paper at ICLR 2024
Notice that the task distributions (Pt) are bimodal there two classes in each task in CIO-ST. The most interesting observation that the distribution of the other tasks (Pre} } have overlap with the task distribution {P:). This indicates that the failure may happen
analyzed above. For example, we draw a demonstrative failure case the existing TIL+OOD methods (i.e., MORE and ROW) in Task 1 prediction in Figure 5. In this Figure, TIL+ methods will compute the task prediction probability solely based on the high likelihood of Pt (t = 1), while our TPL will consider both high likelihood of P. and low likelihood of Pær. In this case, the red has higher likelihood in P, than the green star (0.9 v.s. 0.4). However, the likelihood ratio between P; and Pte of the red star is lower than the green star (3 v.s. 20). Therefore, using likelihood ratio between Pa and Pa is crucial in estimating the task prediction probability.
Distribution Distribution of Task 1 Distribution of other tasks
Likelibood of
Likelibood
Likelilood
Task
0.02
0.9
0,3
Figure 5: failure of TIL+OOD methods that predict the task based the likelihood of Pa (e.g., MORK and ROW). the figure, the star has higher likelihood Pa (f = 1) than green star. However, the likelihood ratio between P, and Pe of the red star is lower than the green star. The correct choice is to accept the green star to be from Task instead of the red star.
37

---

Published as a conference paper at ICLR 2024
1 IMPLEMENTATION DETAILS, NETWORK SIZE AND RUNNING TIME
I.1 IMPLEMENTATION DETAILS OF BASELINES
Datasets. We use three popular image datasets. (1) CIFAR-10 (Krizhevsky & Hinton, 2010) consists of images of 10 classes with / 10,000 training / testing samples. (2) CIFAR-100 (Krizhevsky ct al., 2009) consists of images with 50,000/ 10,000 training testing samples. (3) & Yang, 2015) has 120,000 images of 200 classes with 500 50 images per class for training testing.
(Le
Implementation of CIL baselines (pre-trained setting). For CIL baselines, we follow the experi- ment setups as reported in their official papers unless additionally explained in Section 5.1. For the regularization hyper-parameter and temperature annealing term 8 (see Appendix G) used in HAT, follow the baseline MORE and use μ4 0.75 and 400 for all experiments recommended in (Serra et al., 2018). For the approaches based on network expansion (DER, BEEF, FOSTER), we expand the network of adapters when using pre-trained backbone. For ADAM, we choose the ADAM (adapter) version in their original paper, which is the best variant of ADAM. As some of the baselines are proposed to continually learn from scratch, we carefully tune their byper-parameters to improve the performance to ensure fair comparison. The implementation details of baselines under non-pre- pre-tanngni setting are shown in Appendix D.2.
Hyper-parameter tuning. Apart from 31 and B2 discussed in Section 5.3, the only hyper-parameters used in our method TPI. are 7, which is the temperature parameter for task-id prediction, and which is the hyper-parameee parameter of dgwv( (z, Buf") in Equation (7). The value of y and & are searched from (0.01, 0.05, 0.10, 0,50, 1.0, 2.0, 5.0, 10.0} and {1, 2, 5, 10, 50, 100), respectively. We choose y 0.05, and le 5 for all the experiments as they achieve the overall best results.
Training Details. To compare with the strongest baseline MORE and ROW, we follow their setup (Kim et al., 2022a; 2023) to set the training epochs 20, 40, 15, 10 for CIFAR-10 10, CIFAR-100, T-5T, T-1OT respectively. And we follow them to use SGD optimizer, the momentum of 0.9, the batch size of 64, the learning rate of 0.005 for C10-5T, T-5T, T-10T, C100-20T, and 0.001 for C100-10T.
I.2 HARDWARE AND SOFTWARE
We run all the experiments on NVIDIA GeForce RTX-2080Ti GPU. Our implementations are based on Ubuntu Linux 16.04 with Python 3.6.
1.3
COMPUTATIONAL BUDGET ANALYSIS
I.3.1 MEMORY CONSUMPTION
We present the network sizes of the CIL systems (with pre-trained network) after learning the final task in Table 12.
With the exception of SLDA and 1.2P, all the CIL methods studied utilize trainable adapter modules. The transformer backbone consumes 21.6 million parameters, while the adapters require 1.2M for CIFAR- 10 and 2.4M for other datasets. In the case of SLDA, only the classifier on top of the fixed pre-trained feature extractor is fine-tuned as it requires a fixed feature extractor for all tasks, while L2P maintains a prompt pool with 32k parameters. Additionally, each method requires some specific elements (e.g., task embedding for HAT), resulting in varying parameter requirements for each method.
As mentioned in Appendix F.1, our method also necessitates the storage of class centroids and covariance matrices. For each class, we save a centroid of dimension 384, resulting in a total of 3.84k, 38.4k, and 76.8k parameters for CIFAR-10, 10, CIFAR- and TinylmageNet, respectively. The covariance matrix is saved per task, with a size of 384 x 384. Then the parameter count of covariance matrix can be computed by T x 384 x 384 for dataset with T tasks. Consequently, the total parameter count is 737.3k, 1.5M, 2.9M, 737.3k, and 1.5M for C10-5T, C100-10T, C100-20T, T-5T, and TOT, respectively. It is worth noting that this consumption is relatively small when compared to certain replay-based methods like iCaRL and HAL, which require a teacher model of the same size the training model for knowledge distillation.
38

---

Published as a conference paper at ICLR 2024
Table 12: Network size measured in the number of parameters (# parameters) for each method without the memory buffer.
C10-ST
C100-10T
C100-20T
T-ST
T-10T
OWM
24.1M
24.4M
24.7M
24.3M
24.4M
ADAM
22.9M
24.1M
24.1M
24.1M
24.1M
PASS
22.9M
24.2M
24.2M
24.3M
24.4M
CTT
24.1M
24.4M
24.7M
24.3M
24.4M
SLDA
2166M
21.6M
2166M
21.7M
21.7M
L2P
217M
21.7M
217M
21.8
218M
iCaRL
22.9M
24.1M
24.1M
24.1M
24.1M
A-GEM
26.5M
31.4M
31.4M
31.5M
31.5M
EEIL
22,9M
24.IM
24.1M
24.1M
24.1M
GD
22,9M
24.1M
24.1M
24.1M
24.1M
DER+
22,9M
24.1M
24.1M
24.1M
24.1M
HAL
22.9M
24.1M
24.1M
24.1M
24.IM
DER
27.7M
45.4M
69.1M
33.6M
45,5M
FOSTER
28.9M
46,7M
74.2M
35.8M
48. IM
BEEF
30.4M
48.4M
82.3M
37.7M
50.6M
MORE
23.7M
25.9M
27.7M
25.1M
25.9M
ROW
23.7M
26.0M
27.8M
25.2M
26.0M
TPL
23.7M
25.9M
27.7M
25.1M
25.9M
I.3.2 RUNNING TIME
The computation for method TPL is very efficient. It involves standard classifier training and likelihood raio score computation, which employed some OOD detection methods. The OOD score computation only involves mean, covariance computation, and KNN search, which are all very efficient with the Python packages scikit-leate
and fa ss 10 We give the comparison in running time in Table 13. We use HAT as the base as MORE, ROW and TPL all make use of HAT and MORE and ROW are the strongest basclines.
Table 13: Average running time measured in minutes per task (min/T) for three systems.
C10-5T
C100-10T
C100-20T
T-ST
T-10T
HAT MORE
17.8 min/T
17.6 min/T
9.4 min/T
28.0 min/T
9.48 min/T
20.6 min/
23.3 min/T
11.7 min/T
32.8 min/T
11.2 min/T
ROW
21.8 min/T
25.2 min/T
12.6 min/T
34.1 min/T
min/T
TPL
20.7 min/
23.3 min/T
11.7 min/T
32.5 min/T
11.2 min/T
http://scikii-laamm.rg/stt
39

---

Published as a conference paper at ICLR 2024
J LIMITATIONS
Here we discuss the limitations of our proposed method TPL.
First, our TPL method relies on the saved data in the memory buffer like traditional replay-based methods, which may have privacy concerns in some situations and also need extra storage. We will explore how improve task-id prediction in the CIL setting without saving any previous data in our future work. Second, TPL. uses a naive saving strategy that samples task data randomly to put in the memory buffer for simplicity. In our future work, we would also like to consider better buffer saving strategies (Jeeveswaran et al., 2023) and learning algorithm for buffer data (Bhat et al., 2022), which may enable more accurate likelihood ratio computation. Third, this paper focuses on the traditional offline CIL problem set-up. In this mode, all the training data for each task is available upfront when the task arrives and the training can take any number of epochs. Also, the label space ) of different tasks are disjoint. In online CIL (Guo et al., 2022), the data comes stream and there may be clear boundary between tasks (called Blurry Task Setting (Bang et al., 2022)), where the incoming data labels may overlap across tasks. is an interesting direction for our future work to explore how to adapt method to exploit the specific information in this setting.
40


Reviews:
[{"review_id": "uvrZ4G7nHt", "reviewer": "Reviewer_fLUE", "summary": "The authors propose to use out-of-distribution ideas to solve the gap between task incremental and class incremental learning, indirectly predicting the task label and using it to refine the class prediction. They pose that using a low forgetting method such as HAT and pairing it with a good task-prediction from the ood-inspired setting, allows better estimates of both the intra- and inter-task probabilities, which leads to better performance in CIL scenarios.", "questions": "It is mentioned that HAT prevents CF, but it actually only mitigates it. It is discussed later in the appendix that a very large sigmoid is used in order to force an almost binary mask to promote more of that CF mitigation. However, how relevant is that the masks are binary and that the sigmoid is close to a step function? Would then a method that guarantees no forgetting such as PNN [Rusu et al. 2016], PackNet [Mallya et al. 2018] or Ternary Masks [Masana et al. 2020] be more suitable for the proposed strategy? How do you deal with HAT running out of capacity when the sequence gets longer?\n\nIn Table 1, which of these results are using the task label at inference time? For example, HAT needs the task label. So are the results of HAT comparable here with the other methods? Or is HAT having a forward pass with each task label and then using some heuristic to pick the class?\n\nFor the experiments on running time, in Table 9 of the appendix it is only shown the running times for the 4 methods that have the same base strategy. How do those compare with all the other methods, because I would assume that for large sequences of tasks, it might become quite a limiting factor to have to forward each sample/batch T times. I would argue that is a relevant discussion to have in the main manuscript.\n\nIn the introduction it is mentioned \"This means the universal set [...] includes all possible classes in the world [...], which is at least very large if not infinite in size...\". Is there some paper or relevant source to back this? One of the papers that comes to mind is [Biederman, 1987], which states that there are between 10k to 30k visual object categories that we can recognize in images. And that would hint towards learning an estimate of the distribution for objects in images would not be such unfeasible (specially now with foundational models).\n\nIn conclusion, I find the idea interesting and relevant. However, the small extension from existing related work, and the lack of a better discussion of the limitations and motivation/relevance for the community could be improved.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The proposed method is simple and well explained, backed up with justification of why they choose the likelihood ratio strategy. The idea of using ood ideas to overcome the task-ID limitation of TIL is interesting and aligns with the continual learning community directions. The experimental results are compared with a large array of existing methods and state-of-the-art approaches.", "weaknesses": "The proposed method is for the most part an extension of existing previous work, which requires a replay buffer, pretrained models and the need for a forward pass for each task learned. Therefore, the advantage of not needing the task label at inference is not well contrasted with the limitations (mostly mentioned at the end of the appendix only). I would expect further discussion and justification about how these benefits and limitations balance in the main part of the manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pgwYli5f30", "reviewer": "Reviewer_SU84", "summary": "This paper proposes a novel method for class incremental learning (CIL) by directly predicting the task identifiers to perform the task-wise prediction. Using the energy based models, the given model computes the scores for each task based on the Mahalanobis distance and KNN distance, and estimate the task label. Furthermore, the proposed model actively utilizes the pre-trained model by just training the adapter module to efficiently train the parameters. In the experiment, the given algorithm outperforms the baselines in both CIFAR and Tiny-ImageNet dataset. In addition, the authors show the effectiveness of using each component in the ablation study.", "questions": "Already mentioned in the Weakness section", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. By directly estimating the task identifier, the proposed algorithm outperforms other baselines in the benchmark dataset.\n\n2. Since the proposed model utilize the task-wise classifier, it can be robust to the class imbalance problem which can occur when the difference between the size  of replay buffer and training data are large.", "weaknesses": "1. I wonder the proposed methods can achieve high task-prediction accuracy. Different from the ideal situation, the accuracy may be lower than we expected. if the semantics across different classes are similar, the task-prediction accuracy can be low, and the overall performance also can decrease. \n\n2. Can this method outperform other baselines when it does not use the pre-trained model in ImageNet-1K? Furthermore, if the dataset used for pre-training are randomly selected (i.e. Randomly extract 500 classes from ImageNet-1K), can this method outperform other baselines? Since ImageNet-1K or other large datasets contain similar classes, the task-prediction is much harder than CIFAR or Tiny-ImageNet", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gRmat3j09v", "reviewer": "Reviewer_h9ds", "summary": "In this paper, the authors address the challenge of task identification (task-id prediction) in Class Incremental Learning (CIL). They propose a novel method named TPLR (Task-id Prediction based on Likelihood Ratio), which enhances task-id prediction by utilizing replay data to estimate the distribution of non-target tasks. This approach allows for a more principled solution compared to traditional Out-of-Distribution (OOD) detection methods that cannot estimate the vast universe of non-target classes due to lack of data.\n\nTPLR calculates the likelihood ratio between the data distribution of the current task and that of its complement, providing a robust mechanism for task-id prediction. The method is integrated into the Hard Attention to the Task (HAT) structure, which employs learned masks to prevent catastrophic forgetting, adapting the architecture to facilitate both task-id prediction and within-task classification.\n\nThe authors demonstrate through extensive experimentation that TPLR substantially outperforms existing baselines in CIL settings. This performance is consistent across different configurations, including scenarios with and without pre-trained feature extractors. The paper's contributions offer significant advancements for task-id prediction in CIL, proposing a method that leverages available data more effectively than prior approaches.", "questions": "- While the writing quality is satisfactory, I would argue for a friendlier approach to outlining the proposed method. First, outline the key ingredients. Then explain how they interact. Finally cross-reference these with the existing figure. \n- The existing figure is a bit too 'noisy' in terms of the information it is showing and the order it is showing it in. Consider reorganizing it so it can be read from left to right, top to bottom and with more emphasis on the key ideas and less detail that can distract from that.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Originality:\n\n- TPLR's innovation lies in its unique application of likelihood ratios for task-id prediction, an approach that distinctively diverges from traditional OOD detection methods.\n- The paper creatively leverages replay data to estimate the data distribution for non-target tasks, which is a novel use of available information in the CIL framework.\n- Integration of TPLR with the HAT method showcases an inventive combination of techniques to overcome catastrophic forgetting while facilitating task-id prediction.\n\nQuality:\n\n-The methodological execution of TPLR is of high quality. It is underpinned by a strong theoretical framework that is well-articulated and logically sound.\n- Extensive experiments validate the robustness and reliability of TPLR, demonstrating its superiority over state-of-the-art baselines.\n\nClarity:\n\nThe paper writing quality is satisfactory.\n\nSignificance:\n\nTPLR's ability to outperform existing baselines marks a significant advancement in the domain of CIL, potentially influencing future research directions and applications.\nThe paper's approach to using replay data for improving task-id prediction could have broader implications for continual learning paradigms beyond CIL.", "weaknesses": "The key weakness of this work I would argue is its overly complex presentation. I find that the organization of the paper can easily distract and confuse the reader, often finding myself fishing for key details of the main method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VDBNe9Hjt1", "reviewer": "Reviewer_aMSi", "summary": "In this paper, the author proposes the use of a Likelihood Ratio to identify the Task-id for Class Incremental Learning (CIL). Traditionally, out-of-distribution (OOD) detectors were used for task identification, but this paper introduces a new method called TPLR (Task-id Prediction based on Likelihood Ratio) that leverages additional information like replay data and learned tasks for more effective and principled task-id prediction. TPLR outperforms traditional CIL approaches.", "questions": "Please refer the weakness;", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "The motivation and method sound solid to me. I agree with the author that task-id prediction is pivotal for CIL under specific circumstances.\n\n- Motivation is clear and straightforward:  The author argues that using a traditional OOD detector is not optimal for task predictions and here they leverage the information in the CIL process for task-id prediction.\n\n- The proof and methods in section 3 and 4 look good to me.", "weaknesses": "1. The writing requires improvement. The author frequently used abbreviations and jargon, especially in the introduction, which occasionally left me puzzled. It would be beneficial if these terms were interpreted more straightforwardly. \n\n2. The related works are also unclear: \n- Although the author clarifies their focus on Class incremental learning, which doesn't provide the task-id during inference, it remains ambiguous whether they are using a memory buffer (rehearsal-based) or are memory-free (online CIL). I suggest the author address this in the introduction and related works.\n- Some recent benchmarks are missing: The author left memory-free (non-replay-based approaches) CIL in related works. The author also left balanced CIL works, e.g., SS-IL, TKIL.\n\n3. Experimental settings:\n- Table 1 is impressive, but the comparisons seem biased. The author claims they compared with 17 baselines, including 11 replay-based and 6 non-replay-based. From my understanding, the author requires a memory buffer, as indicated in the \"Overview of the Proposed TPLR\", equation 2.\n-  It would be more equitable if the author juxtaposed their method with replay-based CIL. Specifically, the author should draw a clear comparison with methods using task-id prediction, highlighting the advantages of their technique. \n- One import baseline is missing: AFC[3]\n  \n\n4. The inference setting remains unclear. Does the author predict both the task-id and class-id simultaneously? Is there any fine-tuning step involved? Typically, some fine-tuning follows the task-id prediction. e.g., iTAML. If the author's method circumvents this, it could be seen as a distinct advantage. Therefore, I recommend the author incorporate a discussion about the computational load when integrating likelihood ratio predictions, elucidating the benefits and drawbacks of this model.\n\n5. Lacks Visualizations: Could the author add a real visualization of data distribution, like the \"Feature-based likelihood ratio score\" in Figure 1. It will be strong evidence the TPLR works well.\n\n\n[1] Ss-il: Separated softmax for incremental learning. In Proceedings of the IEEE/CVF International conference on computer vision \n\n[2] TKIL: Tangent Kernel Optimization for Class Balanced Incremental Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision \n\n[3] Class-incremental learning by knowledge distillation with adaptive feature consolidation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)