Exploring Example Inﬂuence in Continual Learning
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Continual Learning (CL) sequentially learns new tasks like human beings, withthe goal to achieve better Stability (S, remembering past tasks) and Plasticity (P,adapting to new tasks).
Due to the fact that past training data is not available, it isvaluable to explore the inﬂuence difference on S and P among training examples,which may improve the learning pattern towards better SP.
Inspired by Inﬂuence
Function (IF), we ﬁrst study example inﬂuence via adding perturbation to exampleweight and computing the inﬂuence derivation.
To avoid the storage and calculationburden of Hessian inverse in neural networks, we propose a simple yet effective
MetaSP algorithm to simulate the two key steps in the computation of IF and obtainthe S- and P-aware example inﬂuence.
Moreover, we propose to fuse two kindsof example inﬂuence by solving a Dual-Objective Optimization (DOO) problem,and obtain a fused inﬂuence towards SP Pareto optimality.
The fused inﬂuencecan be used to control the update of model and optimize the storage of rehearsal.
Empirical results show that our algorithm signiﬁcantly outperforms state-of-the-artmethods on both task- and class-incremental benchmark CL datasets.
We willmake our code open-source.
Introduction
By mimicking human-like learning, Continual Learning (CL) aims to enable a model to continuouslylearn from novel knowledge (new task, new class, etc.)
in sequential order.
The major challenge in CLis to harness catastrophic forgetting and knowledge transition, namely the Stability-Plasticity dilemma,with Stability(S) showing the ability to prevent performance drops for old tasks and Plasticity(P)referring if the new task can be learned rapidly and unimpededly.
Intuitively speaking, a robust CLsystem should achieve outstanding S and P through sequential learning.
The sequential paradigm means CL does not access past training data.
Comparing to traditionalmachine learning, the training data in CL is thus more precious.
It is valuable to explore the inﬂuencedifference on S and P among training examples.
Following the accredited inﬂuence chain “ DataModel - Performance ”, exploring this difference is equivalent to tracing from performance back toexample difference.
With appropriate control, this may improve the learning pattern towards better
SP.
On top of this, the goal of this paper is to explore the reasonable inﬂuence from each trainingexample to SP, and apply the example inﬂuence to CL training .
To understand example inﬂuence, one classic successful technique is the Inﬂuence Function (IF)
[18],which leverages the derivation chain rule from a test objective to training examples.
However, directlyapplying the chain rule leads to computing the inverse of Hessian with the complexity of O ( nq 2 + q 3 )( n denotes number of examples and q is parameter size), which is computationally intensive and may
Out-Of-Memory in neural network.
In this paper, we propose a novel meta-learning algorithm called
MetaSP to compute example inﬂuence via simulating IF.
We design on the basis of the rehearsal-based
CL framework, which avoids forgetting via retraining a part of old data.
First, a pseudo update isheld with example-level perturbations.
Then, two validation sets sampled from seen data are used to
[Figure]
compute the gradients on example perturbations.
The gradients are regarded as the example inﬂuenceon S and P. As shown in Fig.
1(a), examples can be distinguished by the value of inﬂuence on S and P.
To leverage the two independent kinds of inﬂuence in CL, we need to take full account of the inﬂuenceon both S and
P.
However, the inﬂuence on S and P may interfere with each other, which leads usto make a trade-off.
This can be seen as a Dual-Objective Optimization (DOO) problem, whichaims to ﬁnd solutions not dominated (no other better solution) by any other one, i.e. Pareto optimalsolutions [8].
We say the solutions as the example inﬂuence on SP.
Following the gradient-based
MGDA algorithm [10], we obtain the fused example inﬂuence on SP by meeting the KKT condition,as illustrated in Figure 1(b).
Finally, we show the fused inﬂuence can be used to control the update of model and optimize thestorage of rehearsal in Figure 1(c).
On one hand, the fused inﬂuence can be directly used to controlthe magnitude of training loss for each example.
On the other hand, under a ﬁxed memory budget,the fused inﬂuence can be used to select appropriate examples storing and dropping, which keeps therehearsal memory always have larger positive inﬂuence on SP.
In summary, our contributions are four-fold: 1) Inspired by the inﬂuence function, we study CL fromthe perspective of example difference and propose MetaSP to compute the example inﬂuence on
S and P. 2)
We propose to trade off S and P inﬂuence via solving a DOO problem and fuse themtowards SP Pareto optimal.
We leverage the fused inﬂuence to control model update and optimizethe storage of rehearsal.
The veriﬁcation contribution: by considering the example inﬂuence, inour experiments on both task- and class-incremental CL, better S and more stable P can be observed.
Related work
Continual Learning.
Due to many researchers’ efforts, lots of methods for CL have been proposed,which can be classiﬁed into three categories.
The regularization-based methods [17] are based onregularizing the parameters corresponding to the old tasks and penalizing the feature drift.
Theparameter isolation based methods [12, 20] generate task-speciﬁc parameter expansion or sub-branch.
Rehearsal-based methods [23, 7, 19, 6, 1, 2, 22, 3] tackle the challenge of SP dilemma by retaininga subset of old tasks in a stored memory buffer with bounded resources.
Although the existingmethods apply themselves to achieve better SP, they fail to explore what contributes to the Stabilityand Plasticity inside the training data.
In this work, we explore the problem in the perspective ofexample difference, where we argue that each example contributes differently to the SP.
We focus ourwork on the Rehearsal-based CL framework in order to omit the divergence between models, whileevaluating the old data’s inﬂuence simultaneously.
Example Inﬂuence.
In recent years, as the impressive Interpretable Machine Learning (IML) [21]develops, people realize the importance of exploring the nature of data-driven machine learning.
Examples are different, even they belong to the same distribution.
Because of such difference, theexample contributes differently to the learning pattern.
In other words, the inﬂuence acquired inadvance from different training examples can signiﬁcantly improve the CL training.
Some studiespropose a similar idea and use the inﬂuences to reweight or dropout the training data [25, 11, 30].
Incontrast to complicated model design, a model-agnostic algorithm estimates the training example inﬂuence via computing the derivation from a test loss to a training data weight.
One typical examplemethod is the Inﬂuence Function [18], which leverages a pure second-order derivation (Hessian) withthe chain rule.
In this paper, to avoid the expensive computation of Hessian inverse, we design a Meta
Learning [16] based method, which can be used to control the training.
Demystifying example inﬂuence on SP3.1 Preliminary: rehearsal-based CL
Given T different tasks w.r.t.
datasets {D 1 , · · · , D T } , Continual learning (CL) seeks to learn them insequence.
For the t -th dataset (task), D t = { ( x ( n ) t , y ( n ) t ) } N t n =1 is split into a training set D trn t and a testset D tst t , where N t is the number of examples.
At any time, CL aims at learning a multi-task/multi-classpredictor to predict tasks/classes that have been learned (say task-incremental and class-incremental
CL).
To suppress the catastrophic forgetting, the Rehearsal-based CL [24, 19, 26, 6, 14] builds asmall size memory buffer M t sampled from D trn t for each task ( i.e. , |M t | (cid:28) |D trn t | ).
At trainingphase, the data in the whole memory M = ∪ k<t M k will be retrained together with the current tasks.
Accordingly, a mini-batch training step of task t in rehearsal-based CL is denoted as
[Equation]
and
[Equation]
(1) where (cid:96) is the empirical loss.
θ t is the trainable parameters at task t and is updated from scratch.
3.2 Example inﬂuence on Stability and Plasticity
Deﬁnition 1 (Stability and Plasticity) Suppose the parameter of a model is initialized to θ 0 .
At thetraining on the t -th task, given test sets of an old task D tst k ( k < t ) and the current task D tst t , the
Stability S kt and Plasticity P t can be evaluated by:
[Equation]
where p ( D 1 | θ , D 2 ) represents the performance (accuracy in classiﬁcation) of D 1 conditioned to themodel θ training on D 2 .
p ( D| θ ) denotes the performance of D tested on the model θ .
The S of a task is evaluated by the performance difference on the test set after training on any latertasks, which is also known as Forgetting [6].
The P of a task is deﬁned as the ability to integrate newknowledge, which is regarded as the test performance of this task.
As many existing CL methodsdemonstrate, the SP inevitably interferes mutually.
Deﬁnition 2 (Example Inﬂuence on SP)
At the training on the t -th task, with a sampled examplex trn ∈ D trn t , the example inﬂuence from x trn to Stability S kt and Plasticity P t for k < t can beevaluated by the gap from deleting it then retraining the model:
[Equation]
where D trn t /x trn denotes the dataset D trn t without the training example x trn .
However, deleting every example to compute full inﬂuences is impractical due to the highly computational cost.
Instead, the performance change can be indicated by the loss change, which leads to aderivable way to approximate the inﬂuence:
[Equation]
where (cid:15) is the weight perturbation to the training example and def = means deﬁne.
This inﬂuence can becomputed by the Inﬂuence Function [18] that will be introduced in the next section.
Meta learning on Stability and Plasticity
4.1 Inﬂuence function for SP
A mini-batch, B , from the training data is sampled, and the normal model update is
[Equation]
a b a b a b … c a b a b a b … a b … c
Task t
Task tTask t +
[Figure]
update
[Figure]
IF
[Figure]
IF
Input
[Figure]
𝜽
[Figure]
𝜽
[Figure]
𝑬
[Figure]
[Figure]
𝜽 ෡𝜽 𝑬, ℬ
[Figure]
[Figure]
ℬ 𝐈 ∗
[Figure]
ℬ
[Figure]
ℬ
[Figure]
Continual Learning
In Inﬂuence Function (IF) [18], a small weight perturbation (cid:15) is added to the training example x trn ∈ B
[Equation] [Equation]
We can easily promote this to the mini-batch
[Equation]
(5) where L denotes the loss vector for a mini-batch and E ∈ R |B|× 1 denotes the perturbation on eachexample in it.
It is easy to know that the example inﬂuence I
( D tst , B ) is reﬂected in the derivative∇ E (cid:96) ( D tst , ˆ θ E ,x )
[Equation]
E = 0 .
By the chain rule, the example inﬂuence in IF can be computed by
[Equation]
where H = ∇ 2 θ (cid:96) ( B , ˆ θ ) is a Hessian.
Unfortunately, the inverse of Hessian requires the complexity
O ( |B| q 2 + q 3 ) and huge storage for neural networks (maybe Out-Of-Memory), which is challengingfor efﬁcient training.
In Eq.
(6), we have I ( D tst , B ) =
[ I ( D tst , x trn ) | x trn ∈ B ] and ﬁnd the loss will get larger if
I ( D tst , x trn ) > 0 , which means the negative inﬂuence on the test set D tst .
Similarly, I ( D tst , x trn ) <
0means the positive inﬂuence on the test set D tst .
Fortunately, the second-order derivation in IFis not necessary under the popular Meta Learning paradigm such as [16], instead we can easilyget the derivation like IF through a one-step pseudo update.
In the following, we will introduce asimple yet effective meta-based method, named MetaSP, to simulate IF at each step with a two-leveloptimization to avoid computing Hessian inverse.
4.2 Simulating IF for SP
Based on the Meta Learning paradigm, we transform the example inﬂuence computation into solvinga meta gradient descent problem named MetaSP .
For each training step in a rehearsal-based CL, wehave two mini-batches data B old and B new in respect to old and new tasks.
Our goal is to obtain theinﬂuence on Stability and Plasticity from every example in B old ∪ B new .
Note that both S-aware and Paware inﬂuence are applied to every example regardless of old or new tasks.
That is, the contributionof an example is not deterministic.
Data of old tasks may also affect the new task in positive,and vice-versa.
In rehearsal-based CL, we turn to computing the derivations ∇ E (cid:96) ( V old , ˆ θ ) | E =0 forexample inﬂuence.
[Equation]
Training batches
[Equation]
Validation batches
[Equation]
Pareto
[Equation]
influence on SP
[Equation]
// Pseudo update
[Equation]
Gradient from old val loss
[Equation]
Gradient from new val loss
[Equation]
// Optimal fusion hyper-parameter
[Equation]
// Influence fusion
To compute the derivation, as shown in Fig. 2(a), our MetaSP has two key steps:(1)
Pseudo update .
This step is to simulate Eq.
(5) in IF via a pseudo update
[Equation] [Equation]
where L denotes the loss vector for a mini-batch combining both old and new tasks.(2) Compute example inﬂuence .
This step computes example inﬂuence on S and P for all trainingexamples as simulating Eq.
Based on the the pseudo updated model in Eq.
(7), we compute Sand P-aware example inﬂuence via two validation sets V old and V new .
Noteworthily, because the testset D tst is unavailable at training phase, we use two dynamic validation sets V old and V new to act asthe alternative in the CL training process.
One is sampled from the memory buffer ( V old ) representingthe old tasks, and the other is from the seen training data representing the new task ( V new ).
With Einitialized to 0 , the two kinds of example inﬂuence are computed as
[Equation]
Generally, each elements in two inﬂuence vectors I ( V old , B ) and I ( V new , B ) represents the exampleinﬂuence on S and P. Similar to IF, elements with positive value mean negative inﬂuence whileelements with negative value mean positive inﬂuence.
Using Inﬂuence for Continual Learning
5.1 Before using: inﬂuence for SP Pareto Optimal
As shown in Eq.
(8), the example inﬂuence is equal to the derivation from validation loss of old andnew tasks to the perturbations E .
However, the two kinds of inﬂuence are independent and interferewith each other.
That is, using only one of them may fail the other performance.
We prefer to ﬁnd asolution that makes a trade-off between the inﬂuence on both S and P. Thus, we integrate the twoinﬂuence
I ( V old , B ) and I ( V new , B ) into a DOO problem with two gradients from different objectives.
[Equation]
The goal of Problem (9) is to obtain a fused way that satisﬁes the SP Pareto Optimality.
Deﬁnition 3 (SP Pareto Optimality)1.
( Pareto Dominate ) Let E a , E b be two solutions for Problem (9) , E a is said to dominate E b ( E a ≺
E b ) if and only if (cid:96) ( V , ˆ θ E a , B ) ≤ (cid:96) ( V , ˆ θ E b , B ) , ∀V ∈ {V old , V new } , and (cid:96) ( V , ˆ θ E a , B ) <
(cid:96) ( V , ˆ θ E b , B ) ,∃V ∈ {V old , V new } .2.
( SP Pareto Optimal ) E is called SP Pareto optimal if no other solution can have better values in(cid:96) ( V old , ˆ θ E , B ) and (cid:96) ( V new , ˆ θ E , B ) .
Inspired by the Multiple-Gradient Descent Algorithm (MGDA) [10], we transform Problem (9) to amin-norm problem.
Speciﬁcally, according to the Karush-Kuhn-Tucker conditions [13], we have
[Equation]
θ
Referring to the study from Sener et al.
[28], the optimal γ ∗ is easily computed as
[Equation]
Thus, the SP Pareto inﬂuence of the training batch can be computed by
[Equation]
This process can be seen in Fig. 2(a).
Different from the S-aware and P-aware inﬂuence, the integratedinﬂuence consider the Pareto Optimal to both S and P, i.e. , reducing the negative inﬂuence on S or Pand keeping the positive inﬂuence on both S and P.
Then we will introduce how to leverage exampleinﬂuence in CL training, our algorithm can be seen in Alg. 1.
5.2 Model update using example inﬂuence
Algorithm
Input: Initialized θ t , Training set D trn t ,
Memory M , Learning rate α
Output: Trained θ t 1 for i = 1 : ITER_NUM do
B new ∼ D trn t ; if t = 1 then θ t
= θ t − α · ∇ θ (cid:96) ( B new , θ t ) ; else
B old ∼ M , V old ∼ M , V new ∼ D trn t ;
I ∗ ← M ETA SP( B old , B new , V old , V new );
θ t = θ t − α · ∇ θ ( (cid:96) ( B old ∪ B new , θ t ) +( − I ∗ ) (cid:62) L ( B old ∪ B new , θ t )) ; end 11 end
With the computed example inﬂuence in eachmini-batch, we can easily control the modelupdate of this mini-batch to adjust the trainingtowards an ensemble positive direction.
Givenparameter θ from the previous iteration the stepsize α , the model can be updated in traditional
SGD as θ ∗ =
θ − α · ∇ θ ( (cid:96) ( B , θ )) , where
B = B old ∪ B new .
By regularizing the updatewith the example inﬂuence I ∗ , we haveθ ∗ =
θ − α ·∇ θ (cid:0) (cid:96) ( B , θ ) + ( − I ∗ ) (cid:62) L ( B , θ ) (cid:1) .
MetaSP offers regularized updates at every stepfor rehearsal-based CL, which leads the CLtraining to better SP but with only the complexity of O ( |B| q + vq ) ( v denotes the validationsize) compared with that of IF, O ( |B| q 2 + q 3 ) .
We show this application in Fig. 2(b).
By updating like the above equation, we can make use of theinﬂuence of each example to a large extent.
In this way, some useless examples are restrained andsome positive examples are emphasized, which may improve the acquisition of new knowledge andthe keeping of old knowledge simultaneously.
5.3 Rehearsal selection using example inﬂuence
Rehearsal in ﬁxed budget needs to consider storing and dropping to keep the memory M havingthe core set of all old tasks.
In tradition, storing and dropping are both based on randomly exampleselection, which ignores the inﬂuence difference on SP from each example.
Given inﬂuence I ∗ ( x )representing contributions from example x to SP, we further design to use it to improve the rehearsalstrategy under ﬁxed memory budget.
The above example inﬂuence on S and P is computed inmini-batch level, we can promote it to the whole dataset according to the Law of Large Numbers, andthe inﬂuence value for the example x is the value of expectation over batches, i.e. , E ( I ∗ ( x )) .
[Table]
The ﬁxed-size memory is divided averagely by the seen task number.
After task t ﬁnishes its training,we conduct our inﬂuence-aware rehearsal selection strategy as shown in Fig. 2(c).
For storing, weﬁrst cluster all training data into |M| t groups using K-means to diversify the store data.
Each groupis ranked by its SP inﬂuence value, and the most positive inﬂuence on both SP will be selected tostore.
For dropping, we rank again on the memory buffer via their inﬂuence value, and drop the mostnegative |M| t example.
In this way, M always stores diverse examples with positive SP inﬂuence.
Experiments
6.1 Datasets and implementation details
We use three commonly used benchmarks for evaluation: 1) Split CIFAR-10 [31] consists of 5 tasks,with 2 distinct classes each and 5000 exemplars per class, deriving from the CIFAR-10 dataset; 2)
Split CIFAR-100 [31] splits the original CIFAR-100 dataset into 10 disjoint subsets, each of which isconsidered as a separate task with 10 classes; 3) Split Mini-Imagenet [29] is a subset of 100 classesfrom ImageNet [9], rescaled to 32 × 32.
Each class has 600 samples, randomly subdivided intotraining (80%) and test sets (20%) .
Mini-Imagenet dataset is equally divided into 5 disjoint tasks.
We employ ResNet-18 [15] as the backbone which is trained from scratch .
We use Stochastic Gradient
Descent (SGD) optimizer and set the batch size 32 unchanged in order to guarantee an equal numberof updates.
Also, the rehearsal batch sampled from memory buffer is set to 32.
We construct the SPvalidation sets in MetaSP by randomly sampling 10% of the seen data and 10% of the memory bufferat each training step.
We set other hyper-settings following ER tricks [4], including 50 total epochsand hyper-parameters.
All results are averaged over 5 ﬁxed seeds for fairness.
To better evaluate the CL process, we suggest evaluating SP with three metrics as follows.
We use thesign function 1 ( · ) to represent if the prediction of model is equal to the ground truth.
First Accuracy ( A 1 = 1 T (cid:80) t (cid:80) x i ∈D tst t 1 ( y i , θ t ( x i )) ):
For each task, when it is ﬁrst trained done, we evaluateits testing performance immediately, which indicates the Plasticity, i.e. , the capability of learningnew knowledge.
2) Final Accuracy ( A ∞ = 1 T (cid:80) t (cid:80) x i ∈D tst t 1 ( y i , θ T ( x i )) ):
This metric is the ﬁnal
Positive IF on S
[Figure]
IF
[Figure]
IF
[Figure]
P
[Figure] IF [Figure] P
[Figure]
I n f l u e n ce s t a ti s ti c s
I n f l u e n ce p r opo r ti on
[Figure]
performance for each task, which indicates Stability, i.e. , the capability of suppressing catastrophic forgetting.
Mean Average Accuracy ( A m = 1 T (cid:80) t (cid:16) 1 t (cid:80) k ≤ t (cid:80) x i ∈D tst k 1 ( y i , θ t ( x i ))
(cid:17) ):
This metriccomputes along CL process, indicating the SP performance after each task trained done.
4) Backward
Transfer ( BWT = 1 T − 1 (cid:80) T − 1 t =1 (cid:80) ( x,y ) ∈D tst t ( 1 ( y, θ T ( x )) − 1 ( y, θ t ( x )))
=
TT − 1 ( A ∞ − A 1 ) ): Thismetric is the performance drop from ﬁrst to ﬁnal accuracy of each task.
6.2 Main comparison results
We compare our method against 8 rehearsal-based methods (GDUMB [22], GEM [19], AGEM [6],
HAL [5], GSS [2], MIR [1], GMED [27] and ER [7]).
What’s more, we also provide a lower boundthat train new data directly without any forgetting avoidance strategy (Fine-tune) and an upper boundthat is given by all task data through joint training (Joint).
In Table 1, we show the quantitative results of all compared methods and the proposed MetaSP inclass-incremental and task-incremental settings.
First of all, by controlling the training accordingto the inﬂuence on SP, the proposed MetaSP outperforms other methods on all metrics.
With thememory buffer size growth, all the rehearsal-based CL get better performance, while the advantagesof MetaSP are more obvious.
In terms of the First Accuracy A 1 , indicating the ability to learn newtasks, our method outperforms most of the other methods with a little numerical advantage.
In termsof the Final Accuracy A ∞ , which is used to measure the forgetting, we have an obvious improvementof an average of 3.17 for class-incremental setting and averagely 1.77 for task-incremental settingw.r.t.
the second best result.
This shows MetaSP can signiﬁcantly keep stable learning of the newtask while suppressing the catastrophic forgetting.
It is because although the new tasks may havelarger gradient to dominant the update for all rehearsal-based CL, our method improves the examplewith positive effective and restrain the negative-impact example.
In terms of the Mean Average
Accuracy A m , which evaluates the SP throughout the whole CL process, our method shows itssigniﬁcant superiority with an average improvement of over 4.44 and 1.24 w.r.t the second best resultsin class-incremental and task-incremental setting.
The complete results with std. can be viewed inthe Appendix.
Moreover, with the proposed rehearsal selection strategy (Ours+RehSel), we haveour A ∞ improved, which means the selected example according to their inﬂuence has a clear abilityfor reducing catastrophic forgetting.
With our Rehearsal Selection (RehSel) strategy, we have animprovement of 0.77 on A ∞ , but A 1 and A m have uncertain performance.
This means better memorymay bring in worse task conﬂict.
6.3 Analysis of dataset inﬂuence on SP
As shown in Fig.
3, we count the example with positive and negative inﬂuences on old task (S), newtask (P), and total SP in Split-CIFAR-10.
At each task from task 2, we have 500 ﬁxed-size memory and 10,000 new task data.
We ﬁrst ﬁnd
that most data of old tasks has a positive inﬂuence on S and anegative inﬂuence on P, while most data of new tasks has a positive inﬂuence on P and a negativeinﬂuence on S.
Even so, some data in both new and old tasks has the opposite inﬂuence.
Then, forthe total SP inﬂuence, most of memory data has positive inﬂuence.
In contrast, examples of newtasks have near equal number of with positive and negative SP inﬂuence.
Thus, by clustering andstoring examples via higher inﬂuence to rehearsal memory buffer, the old knowledge can be kept.
Bydividing all example inﬂuences equally into 5 groups from the minimum to the maximum, we ﬁndmost examples have mid-level inﬂuence, and they are the main part of the dataset.
Also, the numbersof examples with large positive and negative inﬂuence are small, which means unique examplesare few in the dataset.
The two observations suggest we should consider the example difference toimprove model training.
6.4 Analysis on SP Pareto optimum
[Figure]
In this paper, we propose to convert the S-aware and
P-aware inﬂuence fusion into a DOO problem and usethe MGDA to guarantee the fused solution is an SP
Pareto optimum.
As shown in Fig.
4, we show thecomparison of the First Accuracy and Final Accuracycoordinate visualization for all compared methods.
Wealso evaluate with only stability-aware (Ours only S)and with only Plasticity-aware (Ours only P) inﬂuence.
Obviously, with only one kind of inﬂuence, ourmethod can already get better SP than other methods.
The integration of two kinds of inﬂuence yield an evenmore balanced SP.
On the other hand, the existingmethods cannot approach the SP Pareto front well.
6.5 Training time
[Table]
We list the training time of onestep update and total update overheadfor all compared methods for Split
CIFAR-10 dataset.
In one-step update,we evaluate all methods with a batch on one update.
Our method takes more time than other methodsexcept for GEM, because of the pseudo update, backward on perturbation and inﬂuence fusion.
Toguarantee the efﬁciency, we utilize our proposed method only in the last 5 epochs among the total,and the rest are naive ﬁne-tuning (See details in the Appendix).
The results show the strategy is asfast as other light-weight methods but achieve huge improvement on SP.
We also use this setting forall the comparison in Table 1.
Conclusion
In this paper, we proposed to explore the example inﬂuence on Stability-Plasticity (SP) dilemma inrehearsal-based continual learning.
That is, different examples contribute to stability and plasticitydifferently.
To achieve that, we evaluated the example inﬂuence via small perturbation instead ofthe computationally expensive Hessian-like inﬂuence function and proposed a simple yet effective
MetaSP algorithm.
At each iteration in CL training, MetaSP builds a pseudo update and obtains the
S- and P-aware example inﬂuence in batch level.
Then, the two kinds of inﬂuence are combinedvia an SP Pareto optimal factor and can support the regular model update.
Moreover, the exampleinﬂuence can be used to optimize rehearsal selection.
The experimental results on three popular CLdatasets veriﬁed the effectiveness of the proposed method.
We list the limitation of the proposedmethod.
The proposed method relies on rehearsal selection, which may affect privacy and extrastorage is needed.
The proposed method is not fast enough for online continual learning.
In mostsituations, however, we can leverage our training tricks to reduce the time.
(3) Our method is limitedin the extremely small memory size.
Large memory size means better remembering and an accuratevalidation set.
The proposed method does not perform well when the memory size is extremely small.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A] We use public dataset and opensourced code.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]