On Solving Class Incremental Learning in Continual Learning
Anonymous Author(s)
AffiliationAddress email
Abstract
Continual learning (CL) is concerned with learning a sequence of tasks incrementally.
There are two popular CL settings, class incremental learning (CIL) and taskincremental learning (TIL).
A major challenge of CL is catastrophic forgetting(CF).
While a number of techniques are already available to effectively overcome
CF for TIL, CIL remains to be highly challenging.
So far, little study has beendone to provide a theoretical guidance on how to solve the CIL problem.
Thispaper performs such a study.
It first shows that probabilistically, the CIL problemcan be decomposed into two sub-problems: within-task prediction and task-id prediction .
It further proves that task-id prediction is correlated to out-of-distribution(OOD) detection, which connects CIL and OOD detection and at the same time,offers a principled approach to solving CIL.
Experiments have been conducted toempirically verify the theoretical result.
Based on the result, new CIL methods arealso designed, which outperform strong baselines by a large margin.
Introduction
Continual learning aims to incrementally learn a sequence of tasks [1].
Each task consists of a set ofclasses to be learned together.
A major challenge of CL is catastrophic forgetting (CF).
Two main
CL settings have been extensively studied: class incremental learning (CIL) and task incrementallearning (TIL) [2].
In CIL, the learning process builds a single classifier for all tasks/classes learnedso far.
In testing, a test instance from any class may be presented for the model to classify.
No priortask information (e.g., task-id) of the test instance is provided.
Formally, CIL is defined as follows.
Class incremental learning (CIL).
CIL learns a sequence of tasks, 1 , 2 , ..., T .
Each task k has atraining dataset D k = { ( x ik , y ik ) n k i =1 } , where n k is the number of data samples in task k , and x ik ∈ Xis an input sample and y ik ∈ Y k is its class label.
All Y k ’s are disjoint and (cid:83)
Tk =1 Y k
=
Y .
The goalof CIL is to construct a single predictive function or classifier f : X → Y that can identify the classlabel y of each given test instance x .
In the TIL setup, each task is a separate classification problem (e.g., one task could be to classifydifferent breeds of dogs and another task could be to classify different types of birds).
Here, onemodel is built for each task in a shared network.
In testing, the task-id of each test instance is providedand the system uses only the specific model for the task (dog or bird classification) to classify the testinstance.
Formally, TIL is defined as follows.
Task incremental learning (TIL).
TIL learns a sequence of tasks, 1 , 2 , ..., T .
Each task k has atraining dataset D k = { (( x ik , k ) , y ik ) n k i =1 } , where n k is the number of data samples in task k ∈ T ={ 1 , 2 , ..., T } , and x ik ∈ X is an input sample and y ik ∈ Y k ⊂ Y is its class label.
The goal of TIL is to construct a predictor f : X × T → Y to identify the class label y ∈ Y k for ( x, k ) (the given testinstance x from task k ).
Several techniques are already available to effectively overcome CF for TIL (with almost no CF)
However, CIL remains to be highly challenging.
This work focuses on CIL.
Before discussing theproposed work, let us recall the traditional machine learning (ML) paradigm, which builds a classifierbased on the training data of a set of classes.
The resulting classifier is then used to classify testinstances of the same set of classes.
This ML paradigm is said to make the closed-world assumption ,meaning that the classes seen in testing must have been seen in training.
However, in many real-lifeapplications, there are unknowns in the testing or application environment, which is called the openworld .
In open world learning, the training (or known) classes are called in-distribution (IND) classes.
A classifier built for the open world can (1) classify test instances of training/IND classes to theirrespective classes, which is called IND prediction , and also (2) detect test instances that do not belongto any of the IND/known classes but some unknown classes, called out-of-distribution (OOD) classes,which is called OOD detection .
In fact, many OOD detection algorithms can perform both INDprediction and OOD detection [5].
This paper conducts a theoretical study of CIL.
Instead of focusing on the traditional PAC generalization bound [6, 7], we focus on how to solve the CIL problem.
We first decompose the CIL probleminto two sub-problems in a probabilistic framework: within-task IND prediction (WIP) and task-idprediction (TP).
WIP means that the prediction for a test instance is only done within the classes ofthe task to which the test instance belongs.
TP simply predicts the task-id.
TP is needed because in
CIL, task-id is not provided in testing.
This paper then proves based on the popular cross-entropy lossthat (1) the CIL performance is bounded by WIP and TP performances, and (2) TP and task OODdetection performance bound each other (which connects CL and OOD detection).
These theoreticalresults provide a principled approach to solving the CIL problem, i.e., designing algorithms that canachieve good WIP performance and TP performances.
2 Since WIP is basically IND prediction foreach task and many OOD techniques perform both IND prediction and OOD detection, to achievea good CIL performance, a strong OOD detection algorithm is needed.
Any improvement in OODdetection performance means an improvement in CIL performance.
Based on the theoretical guidance, several new CIL methods are designed, including techniques basedon the integration of a TIL method and an OOD detection method for CIL, which outperforms strongbaselines by a large margin.
This combination is particularly attractive because TIL has achievedno forgetting, and we only need a strong OOD technique that can perform both IND prediction and
OOD detection to learn each task to achieve strong CIL performances.
Related Work
Despite the fact that numerous CL approaches have been proposed, little study has been done toprovide a theoretical guidance on how to solve the problem.
Most existing approaches belong toseveral categories.
We briefly review a list of representative methods, but the list of citations isby no means exhaustive.
Using regularization [8] and knowledge distillation [9] to minimize thechange in previous models are two popular approaches [10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21].
Memorizing some old examples and using them to adjust the old models in learning a new task isanother popular approach (called replay ) [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32].
Several systemslearn to generate pseudo training data of old tasks and use them to jointly train the new task, calledpseudo-replay [33, 34, 35, 36, 37, 38, 39, 40, 41].
Orthogonal projection learns each task in anorthogonal space to other tasks and thus have minimum interference [42, 43, 44].
Our theoreticalanalysis is applicable to any continually trained classification model.
We try some representativemethods from the categories to demonstrate our idea in experiment.
Parameter isolation is yet another popular approach, which makes different subsets (which mayoverlap) of the model parameters dedicated to different tasks using masks [3, 45] or finding asub-network for each task by pruning [46, 4, 47].
This approach is particularly suited for the TILsetting.
Several methods have almost completely overcome forgetting.
HAT [3] and CAT [45] protectprevious tasks by masking the important parameters.
PackNet [46], CPG [47] and SupSup [4] findan isolated sub-network for each task and use it in inference.
HyperNet [48] initializes task-specificparameters conditioned on task-id.
ADP [49] decomposes parameters into shared and adaptive parts2 By no mean do we claim this is the only approach. to construct an order robust TIL system.
CCLL [50] uses task-adaptive calibration on convolutionlayers.
A network of experts is proposed in [51].
We include several approaches in this category inour experiment to show the efficacy of our analysis.
Our methods designed based on the proposed theoretical results make use of two parameter isolationbased TIL methods and two OOD detection methods.
The latest strong OOD detection method CSIin [5] helps produce very strong CIL methods.
CSI is based on data augmentation [52] and contrastivelearning [53].
Excellent surveys of OOD detection include [54, 55].
Some methods have been proposed to use a TIL method for CIL with an additional task-id predictiontechnique.
iTAML [56] requires the test samples to come in batches and each batch must be froma single task.
This is not practical as test samples usually come one by one.
CCG [57] builds aseparate network to predict the task-id.
Expert Gate [51] constructs a separate autoencoder for eachtask.
HyperNet [48] and PR-Ent [58] use entropy to predict the task id.
Since none of these papersproposed a theoretical study, they did not know that a strong OOD detection algorithm is the key.
Ourmethods based on OOD detection perform dramatically better.
Several theoretical studies have been made on lifelong/continual learning.
However, they focus ontraditional generalization bound.
[6] proposes a PAC-Bayesian framework to provide a learningbound on expected error in future tasks by the average loss on the observed tasks.
The work in [59]studies the generalization error by task similarity and [7] studies the dependence of generalizationerror on sample size or number of tasks including forward and backward transfer.
[60] shows thatorthogonal gradient descent gives a tighter generalization bound than SGD.
Our work is very differentas we focus on how to solve the CIL problem.
Our work is thus orthogonal to the existing theoreticalanalysis.
Our theoretical finding is applicable to tasks without any underlying relation in similarity.
CIL by Within-Task IND Prediction and Task Prediction
This section presents our theoretical study.
It first shows that the CIL performance improves ifthe within-task IND prediction (WIP) performance and/or the task-id prediction (TP) performanceimprove, and then shows that TP and OOD detection bound each other, which indicates that CILperformance is controlled by WIP and OOD detection, which connects CL and OOD detectionresearch.
Finally, we study the necessary conditions for a good CIL model, which includes a good
WIP, and a good TP (or a good OOD detection).
3.1 CIL Problem Decomposition
This sub-section first presents the assumptions made by CIL based on its definition and then proposesa decomposition of the CIL problem into two sub-problems.
A CL system learns a sequence { T i } i =1of tasks, where T i is the domain of task i and each task consists of domains of classes as T i = ∪ j S
i,j ,where j indicates the j th class in task i .
Based on the definition of class incremental learning (CIL)(see Section 1), the following assumptions are implied,
Assumption 1.
The domains of all classes of the same task are disjoint, i.e., S i,j ∩ S i,j ′ = ∅ , ∀ j ̸ = j ′ .
Assumption 2.
The domains of tasks are disjoint, i.e., T i ∩ T i [-
-]′
= ∅ , ∀ i ̸ = i ′ .
For any ground event D , the goal of a CIL problem is to learn P ( x ∈ S i,j | D ) .
This can be decomposed into two probabilities, within-task IND prediction (WIP) probability and task-id prediction(TP) probability.
WIP probability is P ( x ∈ S i,j | x ∈ T i , D ) and TP probability is P ( x ∈ T i | D ) .
Wecan rewrite the CIL problem using WIP and TP based on the two assumptions,
[Equation]
where i 0 means a particular task and j 0 a particular class in the task.
Remark 1 .
Eq.
2 shows that if we can improve either the WIP or TP performance, or both, we canimprove the CIL performance.
In the following sub-sections, we develop this further concretely to derive the sufficient and necessaryconditions for solving the CIL problem in the context of cross-entropy loss as it is used in almost allsupervised CL systems.
3.2 CIL Improves as WIP and/or TP Improve
The study here is based on a trained CIL model and not concerned with the algorithm used in trainingthe model.
We use cross-entropy as the performance measure of a trained model as it is the mostpopular loss function used in supervised CL.
For experimental evaluation, we use accuracy followingclassification papers.
Denote the cross-entropy of two probability distributions p and q as
[Equation]
For any x ∈ T , let y to be the CIL ground truth label of x , where y i 0 ,j
= [-
-]1 if x ∈ S i 0 ,j [-
-]0 otherwise 139 y i,j = 0 .
Let ˜ y be the WIP ground truth label of x , where ˜
y
i 0 ,j 0 = 1 if x ∈ S i 0 ,j 0 otherwise ˜ y i 0 ,j = 0 .
Let ¯ y be the TP ground truth label of x , where ¯ y i
= 1 is x ∈ T i 0 otherwise ¯
y i =
Denote
[Equation]
where H WIP , H CIL and H TP are the cross-entropy values of WIP, CIL and TP, respectively.
Wenow present our first theorem.
The theorem connects CIL to WIP and TP, and suggests that by havinga good WIP or TP, the CIL performance improves as the upper bound for the CIL loss decreases.
Theorem 1.
If H TP ( x ) ≤ δ and H WIP ( x ) ≤ ϵ , we have H CIL ( x ) ≤ ϵ + δ.
The detailed proof is given in Appendix A.1.
This theorem holds regardless of whether WIP and
TP are trained together or separately.
When they are trained separately, if WIP is fixed and we letϵ =
H WIP ( x ) , H CIL ( x ) ≤ H WIP ( x ) + δ , which means if TP is better, CIL is better.
Similarly, if
TP is fixed, we have H CIL ( x ) ≤ ϵ + H TP ( x ) .
When they are trained concurrently, there exists afunctional relationship between ϵ and δ depending on implementation.
But no matter what it is, whenϵ + δ decreases, CIL gets better.
Theorem 1 holds for any x ∈ T that satisfies H TP ( x ) ≤ δ or H WIP ( x ) ≤ ϵ .
To measure the overallperformance under expectation, we present the following corollary.
Corollary 1.
Let U ( T ) represents the uniform distribution on T .
i)
If E x ∼ U ( T ) [ H TP ( x )] ≤ δ ,then E x ∼ U ( T ) [ H CIL ( x )] ≤ E x ∼ U ( T )
[ H WIP ( x )]
+ δ .
Similarly, ii) E x ∼ U ( T ) [ H WIP ( x )] ≤ ϵ , then 155 E x ∼ U
( T ) [ H CIL ( x )] ≤ ϵ + E x ∼ U ( T ) [ H TP ( x )] .
The detailed proof is given in Appendix A.2.
The corollary is a direct extension of Theorem 1 inexpectation.
The implication is that given TP performance, CIL is positively related to WIP.
Thebetter the WIP is, the better the CIL is as the upper bound of the CIL loss decreases.
Similarly,given WIP performance, the better TP performance results in the better CIL performance.
Due to thepositive relation, we can improve CIL by improving either WIP or TP using their respective methodsdeveloped in each area.
3.3 Task Prediction (TP) to OOD Detection
Building on Eq. 2, we have studied the relationship of CIL, WIP and TP in Theorem 1.
We nowconnect TP and OOD detection.
They are shown to be dominated by each other to a constant factor.
We again use cross-entropy H to measure the performance of TP and OOD detection of a trainednetwork as in Sec. 3.2.
To build the connection between H TP ( x ) and OOD detection of each task,we first define the notations of OOD detection.
We use P[-
-]
′ i ( x ∈ T i | D ) to represent the probabilitydistribution predicted by the i th task’s OOD detector.
Notice that the task prediction (TP) probabilitydistribution P ( x ∈ T i | D ) is a categorical distribution over n tasks, while the OOD detectionprobability distribution P [-
-]′
i ( x ∈ T i | D ) is a Bernoulli distribution.
For any x ∈ T , define
[Equation]
In CIL, the OOD detection probability for a task can be defined using the output values correspondingto the classes of the task.
Some examples of the function is a sigmoid of maximum logit value or amaximum softmax probability after re-scaling to 0 to 1.
The following theorem shows that TP and
OOD detection bound each other.
Theorem 2.
i)
If H TP ( x ) ≤ δ , let P ′
i ( x ∈ T i | D ) =
P ( x ∈ T i | D ) , then H OOD,i ( x ) ≤ δ, ∀ i =1 , . . . , n . ii)
If H OOD,i ( x ) ≤ δ i , i = 1 , . . . , n , let P ( x ∈
[-P 
-]T
i | D ) =
P ′ i ( x ∈ T i | D ) (cid:80) j P ′ j ( x ∈ T j | D ) , then H TP ( x ) ≤( (cid:80) i 1 x ∈ T
i e δ i )( (cid:80)
i 1 − e − δ i ) , where 1 x ∈ T i is an indicator function.
See Appendix A.3 for the proof.
As we use cross-entropy, the lower the bound, the better theperformance is.
The first statement (i) says that the OOD detection performance improves if the TPperformance gets better (i.e., lower δ ).
Similarly, the second statement (ii) says that the TP performance improves if the OOD detection performance on each task improves (i.e., lower δ i ).
Besides,since ( (cid:80) i 1 x ∈ T i e δ i )( (cid:80)
i 1 − e − δ i ) converges to 0 as δ i ’s converge to 0 in order of O ( | (cid:80) i
δ i | ) , wefurther know that H TP and (cid:80) i H OOD,i are equivalent in quantity up to a constant factor.
In Theorem 1, we studied how CIL is related to WIP and TP.
In Theorem 2, we showed that TP and
OOD bound each other.
Now we explicitly give the upper bound of CIL in relation to WIP and OODdetection of each task.
The detailed proof can be found in Appendix A.4.
Theorem 3.
If H OOD,i ( x ) ≤ δ i , i = 1 , . . . , n and H WIP ( x ) ≤ ϵ , we have
H CIL ( x ) ≤ ϵ + ( (cid:88) i 1 x ∈ T i e δ i )( (cid:88)
i 1 − e − δ i ) , where 1 x ∈ T i is an indicator function.
3.4 Necessary Conditions for Improving CIL
In Theorem 1, we showed that the good performance of WIP and TP is sufficient to guarantee agood performance of CIL.
In Theorem 3, we showed that the good performance of WIP and OODis sufficient to guarantee a good performance of CIL.
For completeness, we study the necessaryconditions of a well-performed CIL in this sub-section.
Theorem 4.
If H CIL ( x ) ≤ η , then there exist i) a WIP, s.t.
H WIP ( x ) ≤ η , ii) a TP, s.t.
H TP ( x ) ≤ η ,and iii) an OOD detector for each task, s.t.
H OOD,i ≤ η, i = 1 , . . . , n .
The detailed proof is given in Appendix A.5.
This theorem tells that if a good CIL model is trained,then a good WIP, a good TP and a good OOD detector for each task are always implied.
Moreimportantly, by transforming Theorem 4 into its contraposition, we have the following statements: Iffor any WIP, H WIP ( x ) > η , then H CIL ( x ) >
η .
If for any TP, H TP ( x ) > η , then H CIL ( x ) > η .
If for any OOD detector, H OOD,i ( x ) > η, i = 1 , . . . , n , then H CIL ( x ) > η .
Regardless of whether
WIP, TP and OOD detection are defined explicitly or implicitly by a CIL algorithm, good WIP, good
TP and good OOD detection are necessary for a good CIL performance.
Remark 2 .
It is important to note that our study in this section is based on a CIL model that has alreadybeen built.
In other words, it tells the CIL designers what should be achieved in the final model.
Clearly, one would also like to know how to design a strong CIL model based on the theoreticalresults, which also considers catastrophic forgetting (CF).
One effective method is to make use of astrong existing TIL algorithm, which can already achieve no or little forgetting (CF), and combine itwith a strong OOD detection
algorithm (as mentioned earlier, most OOD detection methods can alsoperform WIP).
Thus, any improved method from the OOD detection community can be applied to
CIL to produce improved CIL systems (see Sections 4.3 and 4.4).
Recall in Section 2, we reviewed prior works that have tried to use a TIL method for CIL with atask-id prediction method [48, 51, 56, 57, 58].
However, since they did not know that the key to thesuccess of this approach is a strong OOD detection algorithm, they are weak (see Section 4).
New CIL Techniques and Experiments
Based on Theorem 3, we have designed several new CIL methods, each of which integrates anexisting CL algorithm and an OOD detection algorithm.
The OOD detection algorithms that we usecan perform both within-task IND prediction (WIP) and OOD detection.
Our experiments have two goals: (1) to show that a good OOD detection method can help improve the accuracy of an existing
CIL algorithm, and (2) to fully compare two of these methods with strong baselines to show that theyoutperform the existing strong baselines considerably.
4.1 Datasets, CL Baselines and OOD Detection Methods
Datasets and CIL Tasks.
Four popular benchmark image classification datasets are used, fromwhich six CIL problems are created following the recent methods [20, 29, 21].
(1) MNIST consistsof handwritten images of 10 digits with 60,000 examples for training and 10,000 examples for testing.
We create a CIL problem ( M-5T ) by splitting it into 5 tasks where each task consists of 2 consecutiveclasses.
(2) CIFAR-10 consists of 32x32 color images of 10 classes in 60,000 samples with 50,000training samples and 10,000 test samples.
We create a CIL problem ( C10-5T ) by splitting it into 5tasks with 2 consecutive classes per task.
(3) CIFAR-100 consists of 60,000 32x32 color images ofwhich 50,000 are training and 10,000 are testing samples.
We create two CIL problems by splitting100 classes into 10 tasks ( C100-10T ) and 20 tasks ( C100-20T ), where each task has 10 and 5 classes,respectively.
(4) Tiny-ImageNet has 120,000 64x64 color images of 200 classes with 500 images perclass for training and 50 images per class for testing.
We create two CIL problems by splitting 200classes into 5 tasks ( T-5T ) and 10 tasks ( T-10T ), where each task has 40 and 20 classes, respectively.
Baseline CL Methods.
We include different families of CL methods to verify our theoreticalresult: regularization , orthogonal projection , replay , and parameter isolation .
EWC [8], MUC [20],and PASS [21] are regularization-based methods.
OWM [42] is an orthogonal projection method.
For replay methods, we use LwF [9], iCaRL [24], Mnemonics [61], BiC [27], DER++ [29], and
Co 2 L [32].
Finally, for parameter isolation, we use CCG [57], HyperNet [48], HAT [3], SupSup [4](or Sup), and PR [58].
We use the official codes for the baselines except for Co 2 L, CCG, and PR.
For these three systems, we copy the results from their original papers as CCG does not released itscode and we are unable to run Co 2 L and PR on our machines.
OOD Detection Methods .
Two OOD detection methods are used to verify the theoretical resultsby combining them with the above existing CL algorithms.
Both these methods can also performwithin-task IND prediction (WIP).(1).
ODIN :
Researchers have proposed several methods to improve the OOD detection performanceof a trained network by post-processing [62, 63, 64].
ODIN [64] is a representative method.
It addsperturbation to input and applies a temperature scaling to the softmax output of a trained network.(2).
CSI :
It is a recently proposed OOD detection technique [5] that is highly effective.
It is basedon data and class label augmentation and supervised contrastive learning [65].
Its rotation dataaugmentations create distributional shifted samples to act as negative data for the original samples
forcontrastive learning.
We will discuss the details about CSI in Appendix D.
4.2 Training Details and Evaluation Metrics
Training Details.
For backbone structure, we follow [21, 29, 4].
AlexNet-like architecture [66] isused for MNIST and ResNet-18 [67] is used for CIFAR-10.
For CIFAR-100 and Tiny-ImageNet,
ResNet-18 is also used as CIFAR-10, but the number of channels are doubled to fit more classes.
Allthe methods use the same backbone architecture except for OWM and HyperNet, for which we usetheir original architectures.
OWM uses AlexNet.
It is not obvious how to apply the technique to the
ResNet structure.
HyperNet uses a fully-connected network and ResNet-32 for MNIST and otherdatasets, respectively.
We are unable to change the structure due to model initialization argumentsunexplained in the original paper.
For the replay methods, we use memory buffer 200 for MNIST and
CIFAR-10 and 2000 for CIFAR-100 and Tiny-ImageNet as in [24, 29].
We use the hyper-parameterssuggested by the authors.
If we could not reproduce the result, we use 10% of the training data as avalidation set to grid-search for good hyper-parameters.
For our proposed methods, we report thehyper-parameters in Appendix F.
All the results are averages over 5 runs with random seeds.
Evaluation
Average classification accuracy over all classes after learning the last task.
The final classprediction depends on implementation (see Prediction Methods below).(2).
Average AUC (Area Under the ROC Curve) over all task models for the evaluation of OODdetection.
AUC is the main measure used in OOD detection papers.
Using this measure, we showthat a better OOD detection method will result in a better CIL performance.
Let AUC k be the AUCscore of task k .
It is computed by using only the model (or classes) of task k to score the test data oftask k as the in-distribution (IND) data and the test data from other tasks as the out-of-distribution(OOD) data.
The average AUC score is: AUC = (cid:80)
k AUC k /n , where n is the number of tasks.
Note that we don’t study forgetting rate because to show that better WIP and TP (or OOD detection)result in better CIL performance (Sec.4.3), we use existing systems and perform only post-processingusing ODIN, which does not affect the original training and thus does not change their forgetting(CF) profile.
It is not straightforward to change their existing training algorithms to include a new
OOD detection method that needs training, e.g., CSI, except for the TIL (task incremental learning)methods, e.g., HAT and Sup.
For these two TIL methods, we can simply switch their methods forlearning each task with CSI (see Sec.4.4).
HAT and Sup have little or no forgetting.
Prediction Methods: The theoretical results in Sec.
3 states that we use Eq.
2 to perform the finalprediction.
The first probability (for WIP) in Eq. 2 is easy to get as we can simply use the softmaxvalues of the classes in each task.
However, the second probability in Eq.
2 (for TP) is trickier aseach task is learned without the data of other tasks.
There can be many options.
We take the following approaches for prediction (which are a special case of Eq.
2, see below):(1).
For those approaches that use a single classification head to include all classes learned so far, wepredict as follows (which is also the approach taken by the existing papers.)
[Equation]
where f ( x ) is the logit output of the network.(2).
For multi-head methods (e.g., HAT, HyperNet, and Sup), which use one head for each task, weuse the concatenated output as
[Equation]
where (cid:76) indicate concatenation and f ( x ) i is the output of task i .
These methods (in fact, they are the same method used in two different settings) is a special case of
Eq.
2 if we define OOD i as σ (max f ( x ) i ) , where σ is the sigmoid.
Hence, the theoretical results in
Sec. 3 are still applicable.
We present a detailed explanation about this prediction method and someother options in Appendix C.
These two approaches work quite well.
4.3 Better OOD Detection Produces Better CIL Performance
The key theoretical result in Sec.
3 is that better OOD detection will produce better CIL performance.
Recall our considered methods ODIN and CSI can perform both WIP and OOD detection.
Applying ODIN :
We first train the baseline models using their original algorithms, and then applytemperature scaling and input noise of ODIN at testing for each task (no training data needed).
Moreprecisely, the output of class j in task i changes by temperature scaling factor τ i of task i as
[Equation]
10) and the input changes by the noise factor ϵ i as
[Equation]
where ˆ y is the class with maximum output value in task i .
This is a positive adversarial exampleinspired by [68].
The values τ i and ϵ i are hyper-parameters
and we use the same values for all tasksexcept for PASS, for which we had to use a validation set to tune τ i (see Appendix B).
[Table]
[Table]
[Table]
Tab. 1 shows the results on C100-10T. The CIL resultsclearly show that the CIL performance increases if the
AUC increases with ODIN.
For instance, the CIL of
DER++ and Sup improves from 53.71 to 55.29 and 44.58to 46.74, respectively, as the AUC increases from 85.99to 88.21 and 79.16 to 80.58.
It shows that when thismethod is incorporated into each task model in existing trained CIL network, the CIL performance of theoriginal method improves.
We note that ODIN does notalways improve the average AUC.
For those experienceda decrease in AUC, the CIL performance also decreasesexcept LwF.
The inconsistency of LwF is due to its severe classification bias towards later tasks as discussedin BiC [27].
The temperature scaling in ODIN has asimilar effect as the bias correction in BiC, and the CILof LwF becomes close to that of BiC after the correction.
Regardless of whether ODIN improves AUC or not, thepositive correlation between AUC and CIL (except LwF)verifies the efficacy of Theorem 3, indicating better OODdetection results in better CIL performances.
Applying CSI : We now apply the OOD detectionmethod CSI.
Due to its sophisticated data augmentation,supervised constrative learning and results ensemble, itis hard to apply CSI to other baselines without fundamentally change them except HAT and Sup (SupSup) asthese methods are parameter isolation-based TIL methods.
We can simply replace their model for trainingeach task with CSI wholesale (the full detail is given Appendix D).
As mentioned earlier, both HAT and SupSupas TIL methods have almost no forgetting.
Table 2 reports the results of using CSI and ODIN, where ODIN is a weaker OOD detection methodthan CSI.
Both HAT and Sup improve greatly as the systems are equipped with a better OOD detectionmethod CSI.
In summary, these experiment results empirically demonstrate the efficacy of Theorem 3, i.e., the CILperformance can be improved if a better OOD detection method is used.
[Table]
[Equation]
[Table]
[Table]
Full Comparison of HAT+CSI and Sup+CSI with Baselines
We now make a full comparison of the two strong systems (HAT+CSI and Sup+CSI) designed basedon the theoretical results.
These combinations are particularly attractive because both HAT and Supare TIL systems and have little or no CF.
Then a strong OOD method (that can also perform WIP
[Table]
(within-task IND prediction)) will result in a strong CIL method.
Since HAT and Sup are memory free CL methods, HAT+CSI and Sup+CSI also do not need to save any previous task data.
Tab. 3shows that HAT and Sup equipped with CSI outperform the baselines by large margins.
DER++, thebest replay method, achieves 66.0 and 53.7 on C10-5T and C100-10T, respectively, while HAT+CSIachieves 87.8 and 63.3 and Sup+CSI achieves 86.0 and 65.1.
The large performance gap remainsconsistent in more challenging problems, T-5T and T-10T. We note that Sup works very poorly on
M-5T, but Sup+CSI improved it drastically, although still very weak compared to HAT+CSI.
Due to the definition of OOD in the prediction method and the fact that each task is trained separatelyin HAT and Sup, the outputs f ( x ) i from different tasks can be in different scales, which will resultin incorrect predictions.
To deal with the problem, we can calibrate the output as α i f ( x ) i + β
i anduse OOD i =
σ ( α i f ( x ) i + β i ) .
The optimal α ∗ i and β ∗ i for each task i can be found by optimizationwith a memory buffer to save a very small number of training examples from previous tasks likethat in the replay-based methods.
We refer the calibrated methods as HAT+CSI+c and Sup+CSI+c.
They are trained by using the memory buffer of the same size as the replay methods (see Section 4.2).
Tab. 3 shows that the calibration improves from their memory free versions, i.e., without calibration.
We provide the details about how to train the calibration parameters α i and β i in Appendix E.
Conclusion
This paper proposed a principled guidance on solving the highly challenging continual learningproblem, class incremental learning (CIL).
It decomposed the CIL prediction into within-task indistribution prediction (WIP) and task-id prediction (TP).
It further theoretically demonstrated that
TP is correlated to out-of-distribution (OOD) detection, and showed that CIL problem can be solvedby WIP, TP and/or OOD detection.
It also proved that a good performance in the three is necessary fora good CIL model.
Experimental results verified the efficacy of the theoretical results.
Furthermore,based on the theoretical guidance, several new CIL methods have been designed.
They outperformthe strong baselines by large margins.
Limitations: There are many options to define WIP and TP (or OOD).
This paper took a simple way.
Although it performed well, in our future work, we will try to study how to optimize them.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ??.
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Conclusion(c)
Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sec. 3.1.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper-
imental results (either in the supplemental material or as a URL)?
[Yes] Included in
Supplementary Materials.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Sec.
4.2 and Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] Cited the relevant
repositories the attached codes.
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]