Active Learning for Continual Learning: Keeping the Past Alive in the Present
Authors: Jaehyun Park
Dongmin Park
Jae-Gil Lee
Venue: ICLR 2025 Poster (2025)
Keywords: active learning
continual learning
Fisher information
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: Propose a novel active continual learning method based on the accumulative informativeness to avoid catastrophic forgetting

Abstract: *Continual learning (CL)* enables deep neural networks to adapt to ever-changing data distributions. In practice, there may be scenarios where annotation is costly, leading to *active continual learning (ACL)*, which performs *active learning (AL)* for the CL scenarios when reducing the labeling cost by selecting the most informative subset is preferable. However, conventional AL strategies are not suitable for ACL, as they focus solely on learning the new knowledge, leading to *catastrophic forgetting* of previously learned tasks. Therefore, ACL requires a new AL strategy that can balance the prevention of catastrophic forgetting and the ability to quickly learn new tasks. In this paper, we propose **AccuACL**, **Accu**mulated informativeness-based **A**ctive **C**ontinual **L**earning, by the novel use of the Fisher information matrix as a criterion for sample selection, derived from a theoretical analysis of the Fisher-optimality preservation properties within the framework of ACL, while also addressing the scalability issue of Fisher information-based AL. Extensive experiments demonstrate that AccuACL significantly outperforms AL baselines across various CL algorithms, increasing the average accuracy and forgetting by 23.8% and 17.0%, respectively, on average.

Published as a conference paper at ICLR 2025
ACTIVE LEARNING FOR CONTINUAL LEARNING: KEEPING THE PAST ALIVE IN THE PRESENT
Jaehyun Park', Dongmin Park², Jae-Gil Leel*
1 KAIST, 2 KRAFTON
(apark813, jaegill @kaist....... ae.kr,
com
ABSTRACT
Continual learning (CL) enables deep neural networks to adapt to ever-changing data distributions. In practice, there may be scenarios where annotation is costly, leading to active continual learning (ACI.), which performs active learning (AI.) for the CL scenarios when reducing the labeling cost by selecting the most infor- mative subset is preferable. However, conventional AL strategies are not suitable for ACL, they focus solely learning the knowledge, leading to catas- forgetting of previously learned tasks. Therefore, ACL requires a new AL strategy that can balance the prevention of catastrophic forgetting and the ability to quickly learn new tasks. In this paper, we propose AccuACL, Accumulated informativeness-basel Active Continual Learning, by the novel of the Fisher information matrix as a criterion for sample selection, derived from theoretical analysis of the Fisher-optimality preservation properties within the framework of ACI., while also addressing the scalability issue of Fisher information- AL. Extensive experiments demonstrate that AccuACL significantly outperforms AL baselines across various CL algorithms, increasing the average accuracy and forgetting by 23.8% and 17.0%, respectively, on average.
INTRODUCTION
Continual learning (CL), learning scenario to adapt models continuously evolving data distribu- tions, is essential in our dynamic world (Thrum 1995). Numerous CL methods have been advanced with the common goal of preserving past know ledge while acquiring new knowledge across the CL. tasks (Abraham and Robins] 2005 Kim et al., 2023b) Mermillod et al. 2013). While most studies in CL assume that the evolving data distributions are fully labeled, this is rarely the case in practice. For example, fraud detection systems in financial services must continuously learn to recognize new fraud patterns. However, the process of annotating these unique patterns expensive, since it requires professional analysis in the field (Lebichot et al. (2024). As result, active continual learning (ACL) becoming a key challenge for effectively mitigating the limited labeling budget, by querying the most important examples at each CL task that maximize the model's performance over all observed tasks (Cai et al., 2022: Perkonigg et al. 2021 Vu et al. 2023).
However, conventional active learning (AL) strategies are not suitable for ACL scenarios, because they are mainly designed to query the examples relevant to the knowledge about a new task. That is, in both uncertainty-basee and diversity- based AL strategies, the examples that the model has not encountered are highly prioritized (Ash et al. 2019 Sener and Savarese 2017; Settles 1995). Figure a first illustrates the inappropriateness of the conventional AL strategies for ACL scenarios. The unlabeled data with diverse feature importance is continuously received for each task. At task I, these AL strategies typically pay more attention to new features (i.e., Features and 4), and accordingly, the examples mainly involved with the new features are selected by the active learner. That is, the AL strategies focus only on quickly learning new tasks and neglect preventing catastrophic forgetting. Thus, the past knowledge involved with Features and 2 can be forgotten after learning task t. Failing to capture the crucial features of the past knowledge causes catastrophic forgetting (French) 1999), which results in significant performance degradation even compared to random querying, as shown in Figure (b).
"Coreepooningg author.

---

Published as a conference paper at ICLR 2025
Task1=1
New
Uolancing
AccuACL
Acca, Infa.
Ampesiinate
Acce Infs.
capabiliiis of
Sellett
Accuney
Fenuurs
2
Forgeting
8
28
(N) Stndba
Canadrrphhi Fergenting
Conventional
AL
Notseen
Select
Catsstrophic Porgetting
Taak Number
5
(a) Accumulated informativeness in ACL.
(b) AL performance CL benchmarks.
Figure 1: Overview of AccuACL; (a) Unlike conventional AL strategies that only focus on the new task and cause catastrophic forgetting, AccuACL balances the prevention of catastrophic forgetting and the ability to learn new tasks quickly, by defining the accumulated informativeness; (b) shows the catastrophic forgetting of the conventional AL strategies on SplitCIFAR100 100.
This paper offers a novel viewpoint on the query strategies ACL. Combined with the evolving data distributions of CL, it is very important not to forget the knowledge learned from past tasks. As example with Figure D a), all of the four features are important in ACI. because they have appeared one of the past or new tasks. However, the data for past tasks is inherently not accessible in CL., which makes challenging to identify the important features from past tasks at the new task. In summary, the key technical challenge in the paper lies in answering the question "what examples in the new task contribute to the preservation of past knowledge?"
To answer the aforementioned question, formulate the accumulated informativeness as novel standard for informativeness in ACL, which balances the prevention of catastrophic forgetting and the ability quickly learn new tasks. It enables assessing an example's informativeness with respect to both past and new tasks. Then, we introduce AccuACI (Accumulated informativeness- based Active Continual Learning). an algorithm based on the theoretical analysis of the Fisher information-basee AL (Sourati et al.) [2017 Zhang and Oles 22000,, that maximizes the accumulated informativeness. We model the accumulated informativeness via the Fisher information matrix, through the approximation with small memory buffer commonly adopted by many echearsal-based CL methods, the model parameters, and the unlabeled data pool for the new task, as illustrated in D a). As result, AccuACL becomes prefer the examples that comprehensively contain the four important features (i.e., Features I~44 in Figure aa). Furthermore, because Fisher information- based AI is combinatorial optimization problem, we develop query algorithm based on two theoretical properties that an optimal labeled subset should possess. To the best of our knowledge, this is the first ACL study that offers the ability to avoid catastrophic forgetting.
Figure
Extensive experiments with four different CL methods on three CL benchmarks, SplicCIFAR 10, SplicCIFAR100, and AplitTinyImageNe show that AccuACL significantly boosts CL approaches, outperforming conventional AL baselines by 23.8% and 17.0%, in terms of average accuracy and forgetting, respectively, on average. As shown Figure lb), AccuACI. always achieves dominance in both metrics throughout the entire sequence of tasks. The source code is available at https www.e/oom/aaiitttmaeae
2 RELATED WORK
Active Learning (AL). AL is research field that focuses on the selection of unlabeled data points for labeling by an oracle, which provides supervision, especially in domains where labeling requires significant costs (Ren et al.) 2021] Tharwat and Schenck [2023). This process allows us to optimize model performance under labeling budget constraints. Many strategies exist for selecting informative approaches measure model prediction uncertainty and select the most uncertain examples (Roth and Small 2006) [Settles] 1995] Wang and Shang 2014). Diversity-based methods prioritize diverse examples to reduce redundancy between selected examples and capture broader range of patterns and complexities in the data pool (Sener and Savarese 2017)
examples from unlabeled data. Uncertainty-bast
Hybrid approaches use gradient space embedding to select diverse and uncertain examples (Ash
al. 2019).. Fisher information-bran based methods measure the asymptotic value of unlabeled data with theoretical guarantee (Sourati et al., 2017] Zhang and Oles) 2000). Notably, Kirsch and Gal (022)) demonstrate that modern AI. algorithms optimize the same Fisher-based objective.
2

---

Published as a conference paper at ICLR 2025
Continual Learning (CL). CL has gained interest way adapt models to new tasks over time.
Numerous research has investigated different methods to address the stability- plasticity dilemma (Mer
millod et al. 2013). Rehearsal bebsed methods store (Aljundi et al. 2019 Buzzega al.) 2020 Caccia
et al.
2022 Rahaf and Lucas] 2019 Rolnick et al. | Liang and Li, 2024] Kim et al. [023a Lin
et al.
2024) or generate (Shin al. 2017) subset of examples from past tasks at constant cost.
These examples are then replayed for the new task to retain past knowledge. Regularization-bee baaedd
methods discourage significant changes to model parameters that are essential for past tasks (Aljundi
et al. 2018: Kirkpatrick et al. 2017). Dynamic-structte saaniii-struttur-baaaae
methods generate distinct modules
to augment the ability to learn new tasks (Yan et al., 2021 Zhou et al.)
2023] Wang al. (023).
Rehearsal-free or prompt-based methods are gaining popularity in the CL field, using pre-trained
models and prompt tuning to adjust to data distribution shifts (Wang et al. 2022a (b.
Active Continual Learning (ACL). There has been lack of research effort on ACI., Vu et al. (0023)
support our goal of tackling the impracticality of fully-supervised datasets in real-world settings; however, instead of developing their own AL approach for CL, they recommend a combination of AL and CL strategies for various CL scenarios. Ayub and Fendley (2022) discuss ACL in a few-shot situation; however, they choose examples based on their proposed uncertainty measure, which focuses on learning new tasks. Perronigg et al. (202T) address the need for ACL in a medical domain; however, because they assume that the distribution is gradually changing, their focus is on detecting the change in the distribution to determine when to label incoming images.
PRELIMINARY
3.1 PROBLEM SETUP: ACTIVE CONTINUAL LEARNING
Consider a sequence of tasks T = (1..... TT. where the input space X₂ and label space V/ shift as
the tasks progress. This study focuses on class-incremental learning. where V, n.v = 0 (t' t). In
the active continual learning (AC1.) framework, at task I, an unlabeled dataset U, = (x)
Dx,
provided. Within labeling budget constraint bt, we query the label of the selected subset S. C U, to the oracle labeler A(-) to obtain L, {xx] vt) ! where yi ~ Dy, jx (x)). The objective of ACL to identify the most informative subset S such that, when the parameter 0 is trained by S it minimizes expected arbitrary error across all encountered data distribution D = DxixXhin Formally, at each task L,
S arg min ((x,)) [<(x, y: 0.)] s.l. Ô = arg min L (0, S, 1,,
(1)
where é can be the cross-entropy error for classification tasks. Note that AL for each CL. task E T is performed through multiple rounds, as in conventional AL..
3.2 FISHER INFORMATION- BASED ACTIVE LEARNING
The Fisher information matrix is often used to quantify the information conveyed to the model parameters, indicating their significance in modeling data distribution (Lehmann and Casella 2006). A model parameter 0 E 0 with a high Fisher information is essential for modeling the distribution, and altering its value hinders the model performance. Fisher information-basee AL assumes the true model parameter 0", in which the underlying conditional distribution (y = p(- -|x 0*). Under this premise, Fisher information-ben based AL seeks to select an optimal subset S C U to label, which minimizes the discrepancy between the trained model parameter Ô and the true model parameter 0- In particular, when the discrepancy is formulated as the log-likelihood ratio c(x, y, 0, 0*) = log p(y|x; 0) log p(y/x; 0²), it results in simpler objective function that leverages the Fisher information matrices (Zhang and Oles 2000). Formally,
S* arg min Ex [((x. y: 0)] s.t. Ô -= arg min L(O:A(S))
(2)
SCU,s
arg min Ex-r [B (x) (x y, O ] s.l. 1 arg min A(S)
(3)
Sc0,
arg min tr () Sc0,181sb
1
(4)
candidate
largel
3

---

Published as a conference paper at ICLR 2025
Algorithm I AccuACL
INPUT:
initial model parameter Ou. CL tasks {1..... T). unlabeled task data (Ui,.... Ur}. oracle labeler A. per-round budget b, active learning round R, memory buffer M + 0.
for
T
CL Tasks
2: Querying set S < Random(U1,
Start with initial random querying
3: Unlabeled data pool U <- U -
4: 0, 4- arg ming CCL 00- 1,-1,
5:
for
to R do
6:
Far - f(0;; M)
{(0:
Fisher information embedding (
7:
Fi - FM (1 A) FU
Get target Fisher information (
4.2
8:
+ arg max (2)) exp (-Dis(o
Over-sample by distribution score ($ 4.5.1
i8 -1)
9:
+ arg max (b)
Fu,alla
Sample by magnitude score ($ 4.5.1
10:
- U U x + SUU
11:
0, - arg min, La (0,,
12:
M <- MemoryUpdatee@Ce A(S))
D. Update the memory buffer with new labeled samples
OUTPUT: Or (final CL model)
where b is the labeling budget, A(.) is the oracle labeler, and (0; is the Fisher information matrix over an arbitrary subset S, which is equivalent to covariance matrix of the score function Vo logply ply) x 0) E Tel0), formally expressed as
5)
ΣΣ XES DEC
ply x: V logplulx; 0) logp(ylx; 0
(5)
1S|
The target Fisher information matrix ;()" quantifies the importance of parameters in modeling the distribution D (or the unlabeled data pool U), while the candidate Fisher information matrix S) quantifies the importance of parameters in modeling the candidate subset S. Intuitively, training a model using S" puts emphasis on properly estimating the essential parameter specified by the target Fisher information matrix. For the AL scenario, 0* is approximated by the estimated parameter from the initially labeled data.
4 ACCUACL: PROPOSED ACL METHOD
In this section, we develop a novel ACL strategy that queries data, which prevents catastrophic forgetting and, at the same time, learns new tasks quickly. First, establish the accumulated informativeness to formulate an optimal informativeness measure that accounts for both preventing forgetting and facilitating rapid learning (in & 4.1.. Second, show that the Fisher-based ACL promising approach for integrating accumulated informativeness into the query strategy (in (..2p. Third, provide efficient approach for approximating the Fisher information matrix to improve scalability (in $4..3. Finally, we introduce AccuACL, Accumulated informativeness-bant Active Continual Learning, which is derived from two unique properties that the approximated Fisher- based ACL should satisfy (in 8 4.4) Furthermore, we provide a complexity analysis of AccuACL, demonstrating its practical usability. The overall methodology provided in Algorithm T
4.1 ACCUMULATED INFORMATIVENESS
We commence by introducing the accumulated informativeness for ACL, which quantifies the amount to which a subset (or sample) S, C. U, at task is beneficial for enhancing the performance within the framework of ACL, Confined to task t, the informativeness of S, about unlabeled dataset U, is
INFo U+) ((x,))~A(())) [p 0.)] s.t. Ô, arg min Lai (0; (0.0
(6)
0
which is the expected likelihood produced by 0, Here, θ, is initialized using the parameter 0,-1 at AL algorithms, without considering the requirements of CL, work by selecting a subset that maximizes INF U,). the context ACL, however, it is crucial to consider the influence of the candidate subset S, previous tasks, represented by the previous unlabeled data pool
task 1- Conventional AL
4

---

Published as a conference paper at ICLR 2025
U1--1- Thus, to achieve balance between preventing forgetting of past knowledge and facilitating efficient learning of new information, the accumulated informativeness ACCUINFO(S, U1a) is defined as composite of [S; U2), representing the capacity for rapid acquisition of new tasks, and (NF U1.-1), denoting the capacity to prevent catastrophic forgetting,
ACCUINFO(S,, U1+)) = f (INFO((S; Ut) INFo U1a-1)),
where f()) can be any function to combine the two informativeness measures.
4.2 ACCUMULATED INFORMATIVENESS AND RUSHER-BASER ACL
From Eq. (I). we can extend the objective in Eq. to the ACL problem. In contrast to conventional AL, the objective of CL is to maximize the generalization performance across all encountered tasks. The Fisher-based AL objective for each task in ACL formulated as
S
arg min S.CUv,SS.Isb, tr| (,) ( U1.)]
(8)
-ACCUINFO(S,.U1..
where ULZt is the whole collection of unlabeled data from all encountered tasks {1.....4) .). by is the la- beling budget for task L, and 07 is the true model parameter such that 0; = arg maxe Ep,, [p(y] x; 0)]. The targer Fisher information matrix [0,; U1:1) represents the importance of parameters for all observed tasks. That is, the trace function in Eq. (8) represents the accumulated informativeness in Eq. (7) As a result, the optimal subset S guarantees accurate estimate of essential parameters for all observed tasks.
In Theorem[4.] 4.1 we show that the target Fisher information matrix in Eq. (8 can be decoupled into two matrices representing past and new information, respectively, with À balancing the informativeness indicates AL that prioritizes preventing catastrophic forgetting, whereas a value close to O indicates an AL that prioritizes quick learning of new tasks. That Eq. (9) offers practical form of the function f(..-) that combines the two informativeness measures for past and new tasks in Eq. 7
between the two. number close
Theorem 4.1. Let Una be the unlabeled data pool for all seen tasks until task t. Then, the target Fisher information matrix can be divided into past and new information matrices such that
I(0,; Uv.) = I(0,; U1 A11
I(Oc;U.)
past information
new information
with the optimal value of the balancing parameter X
Uni-1l
10
Proof. The complete proof is available in Appendix
However, it is infeasible directly apply existing optimization technique to Fisher-based AL, as developed for conventional AI., to select examples based on Eq. (8) (Ash et al.) 2021). The reason is that the unlabeled data pool U1:-1 of the past tasks is not available in CL scenarios. Consequently, we leverage a small-sizze
memory buffer M, C -1s-11 generally maintained in rehearsal-
CL (Aljundi et al.) 2019 Buzzega al. 2020 Rolnick al. 2019). The estimated target Fisher information matrix for task t is defined as
[(0,; U1a) [0;; Mc, U.) A I(0;; M.) X) IOO,; A 101--1l
(10)
U1a
Please refer Appendix B for further analysis of the estimation.
4.3 FISHER INFORMATION EMBEDDING
For deep neural networks with numerous parameters, directly solving Eq. ( infeasible owing to the
computationally expensive and cine-consuming calculation of the inverse of the Fisher information matrix, which is cubic in complexity. Recent research, such BAIT (Ash et al., 2021), reduces the computational cost by using the last linear classification layer to obtain the Fisher information matrix and online approach update the inverse matrix via Woodbury matrix identity. However,
5

---

Published as a conference paper at ICLR 2025
since matrix inversion is still needed, it is computationally demanding, confining its application to scall-scale datasets such as MNIST (LeCun et al.) 1998) and CIFARIO (Alex) 2009).
Accordingly, propose the Fisher information embedding, which is the diagonal component of the Fisher information matrix, to reduce spatial and temporal complexity from square to linear. Formally, the Fisher information embedding f(0,; x) of an example x is expressed
f(0,;x)
Σ p(y|x; 0t) (V.. logply|x; 0.)] R2 VEC
(11)
which is equivalent to the diagonal component of Eq. The use of the diagonal elements of the Fisher information matrix is known to effectively prevent catastrophic forgetting when model is adapted to shifting data distributions (Kirkpatrick et al. 2017] Pennington and Worah] 2018). While
previous studies have used the diagonal Fisher information as a regularization method gradient descent, we use it as representation for each example.
Moreover, we consider solely the last linear classification layer to compute the Fisher information matrix, based the premise that the penultimate layer representation approximates convex model (Ash et al. 2021). I. The computation the Fisher information embedding necessitates [C] backward propagation since the gradient must be derived throughout the whole class space, which may be impractical for large-scale datasets. In Theorem [42 we show that its calculation can be reduced into a single forward operation, thereby avoiding the need for heavy computations proportional to ICI. This embedding is used to diagonally approximate the Fisher information matrix of a dataset D F(0,; D) [D] EXED x). Accordingly, we obtain the embedding version of our target Fisher information F,, U,) = X F(O,; M,) + (1 (1- F(O,
Theorem 4.2. Let K be the number of classes, d be the number of embedding dimensions, 0!' (wi1...., aKa) E RR be the parameters of the last linear classification layer, and h00;;)) E Rd be the embedding of an example x. Then, the (k, i)-th component of the Fisher information embedding f00;; x) can be formally expressed as
K
f(0x:)kA = Σ 1=1
p(y|x; 0.) Vww.. logp(ulx;00) P1 Pk Px)
(12)
where &o [1, E [1, Pk
= Exneep"* exp*x. the softmax probability of an example x for the class
k. and 2x.k the logit for the class of the example x.
Proof. The complete proof is available in Appendix
4.4 SETRER--PTIMALITY.OO O PROPERTIES
Since the Fisher-based AL objective in Eq. (8) is not submodular (Ash al.) 202I), greedy op- timization does not guarantee bounded approximation. Furthermore, understanding Eq. (4) for multivariate models is challenging since it involves the inverse of the Fisher information matrix. Hence, it is difficult to get clear intuition of what examples most beneficial for optimizing the objective function. However, we show that by simplifying the objective function from a Fisher information matrix to a Fisher information embedding,
get intuitions about what properties x E S should possess. The objective function of Eq. (4) can be rewritten as
10
S arg min
Σ
(13)
S. S.CU
where the i-th components of the targel Fisher information embedding F(0;; Mr, U,) and the candidate Fisher information embedding F(0;; St) are t, and s,, respectively. As the value of t, does not change, we find two properties that the s; should have under two different conditions.
Property 1. Position-Wise Optimality: If we fix all Fisher embeddings s; : Vi & k, larger 8y will be closer to optimizing Eq. (13). That is, having large information generally for all s; Vi E [0, e. beneficial for optimization.

---

Published as a conference paper at ICLR 2025
Property 2. Distribution- Wise Optimality: If we assume (F(00s)))):: = k where & is arbitrary constant, we demonstrate through Theorem 4.3) that subset S, whose candidate Fisher information embedding F(O,; St) has similar distribution with that of the target Fisher information embedding F(O,; M,, U+) will be beneficial for Eq. (3)).
Theorem 4.3. Let 84 be the i-th component of the Fisher information embedding of an arbitrary subset S.. Under (t)
/ 1o, 1s:12 k, the optimal 8; that minimizes Eq. 13) is
8. ct//s where 2
10 2/3
Proof. The complete proof is available in Appendix D
4.5 PUTTING THEM ALL TOGETHER: ACCUACL
4.5.1
GREEDY QUERY STRATEGY
We propose a novel greedy query strategy based on the two properties discussed in Section 4.4 AccuACL. successfully constructs batch of examples with the properties at task t.
Magnitude Score. Based on Property 1, in order to reward the examples with higher Fisher infor- mation, we propose the magnitude score M(0,, x), which is the fo-n of the Fisher information embedding M(0.. x)
Distribution Score. Based on Property 2, order reward the examples with similar information distribution to F(0;; Mr, U2), we propose the distribution score D(0,, x, M,, U.), which is the Jensen-Shannon divergence (Lin 1991 between the distributions of f(,; x) and F(O,; Mr, Uz),
D(O,, x, Mr, U.) = (xp(- ()); ())lo M,, Ut))))
(14)
where o (z) E is the softmax function, and (-s(- is the Jensen-Shannon divergence.
Merger of the Two Scores. In order to select the examples that satisfy both properties, we overly- sample the subset that ranks highest according to D-)), and then further narrow down by selecting its subset that ranks highest according to (.). Intuitively, after identifying important parameters for the past and the new through the target Fisher information F(O;; M,, U.), AccuACL prioritizes sample selection to preserve the stability of past parameters while effectively optimizing those important for the new task.
4.5.2 COMPLEXITY ANALYSIS
Space Complexity. Owing to the Fisher information embedding, for each AL round, AccuACL has space complexity of O((m + n))KK). where m is the memory buffer size, is the data pool size, d is the embedding dimensionality, and K is the total number of classes, which significantly less than the space complexity of O(mm - nd K2) required for the query strategy of BAIT (Ash et al. 2021).
Time Complexity. For each AL round, AccuACL has runtime complexity of O(n n d. log d.K., ). where n log n is induced by selecting the examples with the highest score, and dK log d.K is induced by selecting the dimensions with the highest Fisher information for measuring D(-). On the other hand. BAIT (Ash et al.] (2021) has time complexity of O(bndK + n(dK)?). where is the labeling budget, which is very expensive compared to AccuACL.
5 EXPERIMENT
5.1 EXPERIMENTAL SETUP
Algorithms. We compare AccuACL with six AL algorithms in combination with four rehearsal- -base CL methods. The implementation details can be found in Appendix
Active learning: (1) Uniform randomly selects examples from the unlabeled data at each task, (2) Entropy (Settles 1995) selects the examples for which the model is the least certain by select- ing those with the highest entropy of predicted probability distribution, (3) LeastConfidence (Wang
and Shang 2014) selects the examples that have the smallest confidence the highest probability
7

---

Published as a conference paper at ICLR 2025
Table Performance comparison of AL baselines and AccuACI combined with rehearsal-bee -based CL. methods on SpliCCIFAR 10. Spli and AplitTinyImageNN. The best and the second-best results are in bold and underline, respectively.
SpleCIFARRO
SpliCTFARR00
SplicTinllnngeee
Continual
Learning
Learning
Active
M=100
M=200
M=500
M=1000
M=2100
M=50000
Abott
Fall)
Asc)
Foll)
Alo(7)
Flot.)
Azolt:
Fioll
AloCC)
Flo(
Aboil)
Flo(D)
Full
20.1
93.3e12
26.3=as
85.9
12.6ea.
75.0sa7
17.9
68.5
7.7
60.4sb
11,5a02
54.1a05
Uniform
20.4.50
27.1
26.7-13
67.2
10.9:44
63.7-86
16.7:04
56.7.m3
6.8-43
45.3-03
8.9
42.5
Entropy
19.7a1a
81.7
23.6.14
76.5.23
8.5
64.7.a6
11.0am
4.7.a1
44.1
5.510a
42.8
LeastConf
19.9
81.0ans
22.5
76.4
8.8
66.3
11.3aa3
63.5
4,8
44.4
5.7
42.7
ER
&Center
76.2e15
23/4-2
71.8sts
9.7
66.1
14.7s07
60.0sn2
6.0=03
7.4ss
BAIGE
19.4
81.0
25.1-15
73.5
9.0
66.4-85
12,5
62.2
5.8-03
45.5:03
7:2
43 43.0
BAIT
18.4
82.3
23.3
76.5
AccuACL
20.7a19
77.9
26.9.n
20.3
14.1
55.8:39
22.0a111
44.5
7,3
41.9:03
10.5a10
37,5a19
Full
22.9.s
88.9.66
27.8
82.0
10.1436
67.9kax
10.8
67.3
7,2.03
54.5.0
8.0i
53.0.1
Uniform
19.3am0
76.722
23.6.14
21.2-a1
2.90s4
57.6.36
7.9
57.4
533011
42.0ua2
5.3
42.1
Entropy
18.0
75.4451
17.1
76.4=s
7.0
57.2-26
7.3
56.1
4.0=a1
39.1-0
1.3352
10.0eaa
LeastConf
1844
77.8
20.6.11
72.0:55
7.1581
58.2=b2
7.2502
57.1
3.9.92
40.0
4,3663
39
39.8
G88
kConter
19.1
7788:17
19.6.nx
75.1-93
7.1485
59.3.13
7.5106
56.2
5.1.43
41.2-11
5.1403
BADGE
18.6119
78.6119
20.6
74.
7.7a0s
57.7
7.4
57.5.20
4,5
40.4.a7
4,5a82
41.3100
BAIT
17.5
76.8
AccuACL
26,5
68.2
30,0:0s
61.3p1a
8:4
53.7.17
8.4504
54,3c10
5.7004
39.8
5.8502
38.4:17
Full
40.0a11
68.6115
48.7
57,5
30.6a12
$1.1
40.1214
38.0=1a
10.3,03
55,3at1
19.6
32.2.0a
Uniform
39.2
49.6
31.9-21
27.6+30
38.9-14
35.9ea0
21.5
11.3e01
29.0=044
15.2
10.7
Entropy
32.3:06
62.9e02
47.5=43
38.6
21.3400
48.6:10
31.7
27 3:0
81:-11
79.7:1
13.1403
9.7404
LeasiConf
338812
62.1156
45
43.
48.0
33.1
27.0..
8:5
28.9.07
13:3
9,5105
DER
kCenter
37.0a11
$5.1
47.0
39.6.33
25.9
43.4.a5
35.0za7
24.9
10.7
27.9-ax
14.4104
11.2
BADGE
36.4223
57.9266
51,0=21
35.8:3k
21.8a04
45,6210
34.1a10
27.7
9.7g01
28.5
14.7262
10.8
BATT
36.7
56.5
49.7
36.4
AccuACI
44.2
40.4
50.1
18
30.5
27.0
36.3
15.0
12.5
24.0
15.7
11.400
Full
57.6212
27.9
63.7
22/4p1a
34,9
34.6
40.1
30,5:n3
16.8
36.5=0
20.2
30.9:02
Uniform
41.3:13
25.9-18
49.6.15
20.0-34
28.4
30.0-a6
34,2:0s
25.5-11
12:3-10
22.6-09
14.610
23.2
Entropy
42.7.mn
30.3:16
47.0
28.8.24
24.9
37.
31.5
31.4
9,5
27.8
11.9un2
23:4
LeastConf
41S814
32.5
47.4215
27.8:24
25.7a34
35.7a33
30.9a1a
31.7
9.9u03
28.5
11.8
23.9ata
ACE
&Center
36.8:06
33.0a22
43.2=3a
29.6
27.4306
34.0=a7
35.1
28 8:00
11,4202
29.4=05
13.7
25 3e01
BAIGE
3133
32.3:19
47.8-0
26.7-07
35.0-04
11.1
28.5.00
13.5403
BAIT
41.2
33.3
48.0
28.3
AccuACL
43.7a18
20.8:15
48.1
17.7
28.4
26.1
33.9
20.9
13,4001
24.9.03
16.1a04
memory dusing executoo.
class, (4) kCenterCrreddy (Sener and Savarese 2017) tries to select the most diverse examples that are farthest from each other in feature space, (5) BADGE (Ash et al. (201)) is a hybrid approach that selects both diverse and uncertain examples, (6) BAIT (Ash et al. 2021) reduces the complexity of Fisher-based AL, making it suitable for use in deep learning environments, and Full is the
fully supeyssue setting.
Continual learning: (1) ER (Rolnick et al. 2019) stores random examples from previous tasks, (2)
GSS (Aljundi et al. 2019) stores the examples that can diversify the gradients, (3) DER++ (Buzzega
et 2020) further uses knowledge distillation to enhance stability, and (4) ER-ACE (Caccia et al.
2022
reduces abrupt changes on representations to discourage disruptive parameter updates.
Datasets. We use three popular CL benchmark datasets in class-incremental scenarios. SplitCI- FARIO, SplitCIFAR 100, and IplicTinyImageNe which are all derived from the original CIFAR 10, CIFARIOO (Alex 2009), and TinyImageNet (Le and Yang 2015), respectfully. SpliCCIFRR 10 splits 50K training images in CIFARIO into five tasks. where each task includes 10K images for 2-way classification. SplitCIFAR splits 50K training images in CIFAR 100 into ten tasks, where each task includes 5K images of 10-way classification. SplitTinylmageNN
divides 100K training images in TinylmageNet into ten tasks, where each task involves 10K images of 20-way classification.
Metrics. We use two commonly-used metrics for CL. average accuracy (Ar) and forgetting (Fr) De Lange al. 2021 The average accuracy Ar / ΣT= @r,j averages all the test accuracy dT.z. which is the accuracy of the j-th task measured after learning all T tasks. On the other hand, the forgetting Fr- = - -1 S3T averages all forgetting SJ.,, where f3.T measures the difference between the accuracy of task j measured after learning task j and task T.
5.2 MAIN RESULTS
Overall Performance. Table I shows the overall performance of AccuACL and six AL baselines, combined with four rehearsal- CL. methods. Overall, AccuACI achieves the best performance
8

---

Published as a conference paper at ICLR 2025
15
Memory Usage (MB3)
80
Elapsed Time (min)
Entropy
6
LeastConf
100
60
eastConn
kenter
akCeneer
4
BADGE
40
OBADGE
BAIT
AecuACL
5
BADGE
AccuACE
LeuntCood
Estrups
Diitern #Cooter
AccuACL
2
500 1000 1500 2000
2500
3000
Queried Examples
(a) Avg. acc. for different labeling budgets.
(b) Memory usage.
(c) Querying runtime.
Figure 2: Comparison of AL. strategies: (a) average accuracy of SplicCIFAR on ER for different labeling budget per task; (b) memory consumed for selecting 100 examples for single task in SplicCIFARIO; (c) elapsed querying time for selecting 1000 examples for every task SplitCIFAR
64
30
Performance of baseline AL methods with memory buffer examples as an initially selected subset.
Table 2:
SpliCIIARI00
SplirTaylmageNet
63
AL Method
A1o(t)
Pio(J)
A1o(f)
Pio(4)
kCenter
27.4
34.0a07
11.4a02
29.4.us
kCenter Mem
26.5s0s
35.3
11.1303
29.7=u9
62
0.50
0.67 0.75 0.80
1
22
BADGE BADGE Mem
26.5
35.9.04
11.1004
28.5a05
26.2+a7
36.0a03
11.0,03
29.0.u
Figure 3: Effect of the parameter for SplicCIFAR10 on task 4.
AccuACI.
28.4:06
26.1
13.4
24.9.5
in most cases, outperforming other AL baselines by 23.8% and 17.0% in average accuracy and forgetting, respectively in average. This superior performance indicates that considering the ability to prevent forgetting is as significant as the ability to acquire new knowledge quickly in query strategy boost the performance in ACL. On the other hand, most AL baselines that focus only learning new tasks actually under-perform even Uniform because they are biased too much towards new tasks, confirming our motivation. Also, BAIT not scalable large datasets with many classes, including SplicCIFAR 100 and SpliTinylmageNet, because of its expensive query strategy. In contrast, owing our efficient query algorithm, AccuACL succeeds in scaling to these datasets. For further analysis on the impact of task ordering, please refer to Appendix F1
Moreover, Figure aa) illustrates the performance trend of average accuracy as the per-task labeling budget varies. The results consistently show that AccuACL outperforms the AL baselines across different labeling budget settings, highlighting its robustness and adaptability. Interestingly, AccuACL reaches the optimal point even without using the entire dataset for training, which is also shown in several settings in Table D This phenomenon aligns with findings in core-set selection for CL (Yoon) et al., 2022), where the goal is to select high-quality subsets from larger pool of data enhance performance. In the context of ACL, AccuACL. can be viewed as selecting high-quality examples in unsupervised manner. The experimental settings in which ACL surpasses CL mostly attributed to AccuACL, indicating AccuACL's effectiveness in identifying example- wise quality. For further analysis on the labeling budget in a single task, please refer to Appendix F22
Efficiency. Figures (b)) and Z depict the memory usage and time spent for querying, respectively, for different AL algorithms, tested SplitCIFAR 10(M=100). Due to our effective approximation in Section 4.3 and querying algorithm in Section 4.5) AccuACL can perform querying with reasonable time and space consumption. Conversely, BAIT requires 4 times more time and 85.5 times more space resources than AccuACL.
5.3 MORE RESULTS AND ABLATION STUDIES
Effect of X. Figure illustrates the influence of À on the fourth task of SplitCIFAR10, 10, trained with ER, which is the parameter that dictates the quality of data to prioritize during query selection. The different values of À selected by our theoretical value in Theorem 4.1 for different tasks in SplitCIFARIO. For a fair comparison, we choose identical initial points for the first round of training, select subsets for various À values, and then re-train it examine the impact of change in X. It is

---

Published as a conference paper at ICLR 2025
Table 3: Learning accuracy(LA1o). forgetting(F1o), and average accuracy(A1o) comparison between AL baselines and AccuACL on SpliCIFAR 110 trained with DER++
AL Method
Uniform
Entropy
LeastConf
kCenter
BADGE
AccuACL
LAio(1)
62.2#06
64,9
65.8
65.0=a3
65.9
56,640.3
Fiv(4)
38.7213
46.9206
48.9
43,4:00
45.6+10
29.3
A1o(1)
27.4:06
22.6:04
21.7:003
25.9=00
24.8:04
30.0
observed that the forgetting rises when À is small because AccuACL focuses the new task, and vice versa. Furthermore, the accuracy reaches the optimal at the theoretical value of X.
AL Baselines with Memory. To show how effectively AccuACL uses the information of past data via a memory buffer, we run experiments on the greedy algorithms, kCenter and BADGE, to use the examples in the memory buffer into the algorithms. Surprisingly, as in Table 2 using the memory buffer as a starting point even degrades the performance of the original AL algorithms. We conjecture that additionally using the memory buffer may force AL algorithms to concentrate on learning new information even more as they attempt to choose the examples that are the most dissimilar from past data. This finding indicates that simply providing AL with past data is not sufficient, whereas AccuACL properly balances past and new knowledge through data-driven information. For further analysis on the impact of using a memory buffer in AccuACL, please refer to Appendix F.3
initially selected subset to incorporate the past knowledge
Learning Accuracy in ACI. Learning accuracy (Mizzadeh et al. 2022) is an essential evaluation for assessing the plasticity of CL algorithms, which is the accuracy of the new task immediately after lcarning. Within the context of ACL, it indicates the efficacy of AL algorithms in quickly adapting to new tasks. Accrodingly, we perform additional experiment evaluating the learning accuracy of AL strategies on the SplitCIFAR 100 dataset(M==000) optimized by DER++. As can be seen in Table 3, traditional AL baselines achieve superior learning accuracy, consistent with our motivation that traditional AL approaches emphasize fast adaptation to new tasks. On the other hand, AccuACL surpasses the AL baselines in forgetting, as AccuACL specifically tackles another essential perspective of ACL learning properties: prevention of catastrophic forgetring. By successfully balancing these two learning properties, AccuACL exhibits state-of-the-art performance in average accuracy, highlighting its value in overall ACL performance. Moreover, as AccuACI. selects examples that mitigate recency bias in linear classifiers (Mai et al. 2021), it allows reliable Fisher information matrix calculation for the upcoming AL rounds.
Scoring Methods. Table shows the effect of different scor-
Table 4: Effect of scoring measures.
Querying based solely on M(-) focuses exclusively on learning new information. ne- glecting the target Fisher information defined in Section 4.2 This approach leads to high forgetting and low accuracy. In contrast, querying based solely D(-) emphasizes the distri- bution of information across parameters. While this approach selects examples that balance information between past and new tasks, it neglects the amount of each example's informativeness. As a result, although it may reduce forgetting, it does not greatly enhance average accuracy. AccuACL employs D(-) as the primary selection criterion for querying, aligning with the objective of our study to sustain a balance between the two learning properties of ACI. ACI-preeenting catastrophic forgetting and quickly learning new tasks. Ensuring the magnitude of informativeness after establishing the balance has shown effectiveness through empirical evaluation.
ing methods defined in Section 4.5
SpliCCIFAR100
Score
Aio(t)
Fio(J)
M(-)
33.0a14
28.4109
D(-)
33.2+as
20.2+0.3
AccuACL
33.9
20.9
CONCLUSION
In this paper, we introduce a new perspective for the query strategy in ACL, by effectively balancing the prevention of forgetting and quick learning of new tasks, even with limited access to the data for past tasks. Consequently, we propose AccuACL, leveraging the Fisher information matrix to efficiently convey the learned knowledge across tasks and precisely measure the new knowledge without labels. Extensive experiments confirm that AccuACL substantially improves many CL methods in ACL scenarios.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2020-1120886.. DB4DL: High-Usability and Performance In-Memory Distributed DBMS for Deep Learning, 50% and No. 2S-2022-11220157 Robust, Fair, Extensible Data-Centric Continual Learning, 50%). Jae-Gil Lee was partly supported by Samsung Electronics Co., Ltd. (10201211-08051-00))
REFERENCES
Wickliffe C Abraham and Anthony Robins. Memory retention-th the synaptic stability versus plasticity dilemma. Trends in Neurosciences, 28(2):73-78, 2005.
Krizhevsky Alex. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. https//wwww.csttromm
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of European Conference on Computer Vision, pages 139-154, 2018.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in Neural Information Processing Systems, 32, 2019.
Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, and Sham Kakade. Gone fishing: Neural active learning with fisher embeddings. Advances in Neural Information Processing Systems, 34:8927- 8939, 2021.
Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv: (906::9996771 2019.
Ali Ayub and Carter Fendley. Few-shot continual active learning by robot. Advances in Neural Information Processing Systems, 35:30612-30624 2022.
Jihwan Bang. Heesu Kim, YoungJoon Yoo, Jung- Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory diverse samples. Proceedings of IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2218-8227, 2021.
Jihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung- Woo Ha, and Jonghyun Choi. Online continual learning contaminated data stream with blurry task boundaries. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9275-9284, 2022.
Jihwan Bang. Sumyeong Ahn, and Jae-Gil Lee. Active prompt learning in vision language models. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27004-27014, 2024.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark expe- rience for general continual learning: strong, simple baseline. Advances in Neural Information Processing Systems, $3:15920-1592 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change online continual learning. In Proceedings of International Conference on Learning Representations, 2022.
Lile Cai, Ramanpreet Singh Pahwa, Xun Xu, Jie Wang, Richard Chang, Lining Zhang, and Chuan- Sheng Foo. Exploring active learning for semiconductor defect segmentation. In Proceedings of IEEE International Conference on Image Processing, pages 1796-1800, 2022.
Antonio Carta, Lorenzo Pellegrini, Andrea Cossu, Hamed Hemati, and Vincenzo Lomonaco. Avalanche: A pytorch library for deep continual learning. Journal of Machine Learning Re- search, 24(363): 2023.
11

---

Published as a conference paper at ICLR 2025
Aristotelis Chrysakis and Marie-Francine Moens. Online continual learning from imbalanced data. In Proceedings of International Conference on Machine Learning, pages 1952-1961. PMLR, 2020.
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7)3336-3385, 2021.
Robert M French. Catastrophic forgetting connectionist networks. Trends Cognitive Sciences, (4):128-135, 1999.
Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, and Bing Liu. Learnability and algorithm for continual learning. In Proceedings of International Conference on Machine Learning, pages 16877-16896. PMLR, 2023a.
Sanghwan Kim, Lorenzo Noci, Antonio Orvieto, and Thomas Hofmann. Achieving a better stability- plasticity trade-off via auxiliary networks in continual learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 11930-11939, 2023b.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A et al. Overcoming
Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114 (13):3521-3526, 2017.
Andreas Kirsch and Yarin Gal. Unifying approaches in active learning and active sampling via fisher information and information-theorrtic quantities. Transactions on Machine Learning Research, 2022.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. Technical report, Stanford University, 2015.
Bertrand Lebichot, Wissam Siblini, Gian Marco Paldino, Yann Aël Le Borgne, Frédéric Oble, and G. Bontempi. Assessment of catastrophic forgetting in continual credit card fraud detection. Expert Systems with Applications, 249(123445); 2024.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of IEEE, 86(11):222 2324,, 1998.
Erich L Lehmann and George Casella. Theory of point estimation. Springer Science & Business Media, 2006.
Yan-Shuo Liang and Wu-Jun Li. Loss decoupling for task-agnostic continual learning. Advances in Neural Information Processing Systems, 36, 2024.
Haowei Lin, Yijia Shao, Weinan Qian, Ningxin Pan, Yiduo Guo, and Bing Liu. Class incremental learning via likelihood ratio based task prediction. In Proceedings of International Conference on Learning Representationss 2024.
Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory, 37(1):145-151) 1991.
Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3589-3599, 2021.
Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The stability-plassicity dilemma: Inves- tigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in Psychology, 4:504, 2013.
Seyed Iman Mirzadeh, Arslan Chaudhry, Dong Yin, Huiyi Hu, Razvan Pascanu, Dilan Gorur, and Mehrdad Farajtabar. Wide neural networks forget less catastrophically. In Proceedings of International Conference Machine Learning, pages 15699-15717. PMLR, 2022.
Dongmin Park, Yooju Shin, Jihwan Bang, Youngjun Lee, Hwanjun Song, and Jac-Gil Lee. Meta- netry-net: Resolving initty-nformmaiveness. dilemma in open-set active learning. Advances in Neural Information Processing Systems, 35:31416-31429, 2022.
12

---

Published as a conference paper at ICLR 2025
Jeffrey Pennington and Pratik Worah. The spectrum of the fisher information matrix of single- hidden-layer neural network. Advances in Neural Information Processing Systems, 31, 2018.
Matthias Perkonigg, Johannes Hofmanninger, and Georg Langs. Continual active learning for efficient adaptation of machine learning models changing image acquisition. In Proceedings of International Conference on Information Processing in Medical Imaging, pages 649-660. Springer, 2021.
Aljundi Rahaf and Caccia Lucas. Online continual learning with maximally interfered retrieval. Advances in Neural Information Processing Systems, 32, 2019.
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Yao Huang, Zhihui Li, Brij Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning, ACM Computing Surveys, 54(9):1-40, 2021.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances Neural Information Processing Systems, 32, 2019.
Dan Roth and Kevin Small. Margin- based active learning for structured output spaces. In Proceedings of European Conference Machine Learning, pages 413-424. Springer, 2006.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprini 1xXiv:7708.0080 2017.
Burr Settles. Active learning literature survey. Science, 10(3):237-304, 1995.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in Neural Information Processing Systems, 30, 2017.
Jamshid Sourati, Murat Akcakaya, Todd Leen, Deniz Erdogmus, and Jennifer G Dy. Asymptotic analysis of objectives based on fisher information in active learning. The Journal of Machine Learning Research, (((I):1123-110 2017.
Alaa Tharwat and Wolfram Schenck. A survey active learning: State-of-the-art, practical chal- lenges and research directions. Mathematics, (4):8820, 2023.
Sebastian Thrun. A lifelong learning perspective for mobile robot control. Intelligent Robots and Systems, pages 201-214, 1995.
Thuy-Trang Vu, Shahram Khadivi, Dinh Phung, and Gholamreza Haffari. Active continual learning: Labelling queries in a sequence of tasks. arXiv preprint arXiv:2305.039233 2023.
Dan Wang and Yi Shang. A new active labeling method for deep learning. In Proceedings of International Joint Conference on Neural Networks, pages 112-119. IEEE, 2014.
Pu-Yun Wang, Da-Wei Zhou, Liu Liu, Han-Jia Ye, Yatao Bian, De-Chuan Zhan, and Peilin Zhao BEEF: Bi-compatible class-ineremental learning via energy-based expansion and fusion. In Proceedings of International Conference on Learning Representations, 2023.
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In Proceedings of European Conference Computer Vision, pages 63 1-1664 Springer, 2022a.
Zifeng Wang, Zizhao Zhang, Chen-Ve Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 139-149, 2022b.
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of IEEE/VV Conference on Computer Vision and Pattern Recognition, pages 3014-3023 2021.
Jaehong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal- based continual learning. In Proceedings of International Conference on Learning Representations, 2022.
13

---

Published as a conference paper at ICLR 2025
Tong Zhang and Frank J. Oles. A problems. In Proceeding of probanation analysis the value of unlabeled data for classification PMLR. 2000.
Conference on Machine Learming pages 1191-1198.
Da-Wei Zhou, Qi-Wei Wang. Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficien -ane-iccreee. learning. In Proceedings of International Conference on Learning Representations, 2023.
14

---

Published as a conference paper at ICLR 2025
Active Learning for Continual Learning: Keeping the Past Alive in the Present
(Appendix)
A COMPLETE PROOF OF THEOREM 4.
Theorem 4.1. Restated. Let ULa be the unlabeled data pool for all seen tasks until task t. Then, the larget Fisher information matrix can be divided into past and new information matrices as
I [(0 Un.)) = X I(0;; U1--1) +(1- [()(UU)
(15)
past information
new information
(16)
with optimal value of the balancing parameter
101:--1 10
Proof.
K
(1a)==))=
Σ
Σ
plyl:; 0.) V.
(17)
|U1:c|
sEULa y=1
|11-11
Σ
Σ
p(yl=; 0,) V.
(18)
U1a
U1--1
=EU1 y=1
lUel
K
ply|:; x; 0,) V 8. O.)
(19)
[U&]
ZEU, y=1
U1a-1
|Uel
x= U12-1
[(0;; U1=-1) +
((O;UU,),
(20)
U1a
Uvtl
B ANALYSIS OF THE APPROXIMATION IN SECTION 4.2
In Theorem B.1 assess the effectiveness of our approximation I(0;; Me, U.) in comparison to the true I (v... This theorem indicates that the difference between their variances decreases as the size of memory buffer increases.
Theorem B.1. Let U, be the unlabeled data pool a task E (1.....
IT and M, be the memory buffer M. C U of a certain sitearsallbaseee CL algorithm. Then, the difference of variance
between two empirical Fisher information matrices I I(;;; U1a) and I is 10m--1
[ha-al-IMp
of
where of is the inherent variance of the Fisher information of past tasks tl,...,t- 1}.
Proof. Based on the optimal value of X derived from Theorem
we have
I(0;; U1a) = [1L-11| |U,1
I(0;: ((0-1)))11++ + 1 (1- |1+-11
I(0;U
(21)
[Uel
T(0,; M., U.) = |1==-1
I(0;; M,) + (1 |1e--1)) I(0,; U.)
(22)
[U,]
[U2]
As relying the Monte Carlo assumption of the Fisher information matrices (Sourati et al. 2017), we can employ the central limit theorem to assess the reliability of its estimates in relation to the ground truth. Given that two datasets from past and new tasks are independent, we have
U1--1 [
UUa-ill + (1
of
111-11
2
Var U..))
(23)
[U
U
|U1--11
of
(1-
|U11--1
Var [(O;; (,,0,M,UU0 M,,UD)
(24)
[U,]
[M
[U,]'
15

---

Published as a conference paper at ICLR 2025
where of and d. are its inherent variance of Fisher information of the past tasks and the new task, respectively. Then, we can show that the discrepancy two estimations in the perspective of its variance can be formulated
1010-1 1U1:e1?
(inrill-MMr o7.
Var [I (0,; U1:)] Var [I (0,; Mr, U.)
(25)
MMll
which converges to 0 when the cardinality of Mt reaches that of U1:-11
C COMPLETE PROOF OF THEOREM 4.2
Lemma C.1. For an arbitrary example let K be the number of classes, 0.01 = (wi1,... wxa) E T be the parameters of the last linear classification layer, and h(0,; E Rd be the embedding of an example x. The logit TR of is class is le log p(c], 0,) log Pe- where p. = [(2) ΣT exp"; is the probability of belonging to the class e [1, K]. Then, for the example x, the (i. component of the gradient with respect to 0 is
LT h((0;x) and the log-likelihood of x belonging exp"*
occ owij
(1[i
-p) h(0:)))
(26)
Proof. A similar derivation can be found in Ash et al. (2019)
dlc
alog
Pe
az:
(27)
Owsj 0z,
Owij
alog
0pc
0z:
(28)
Ope.
0z:
Owij
1
(o = cl - (p)) h(0;)),
(29)
Pr.
=(1 p.)) h(0:x),
(30)
Theorem 4.2. Restated. Let K be the number of classes, d be the number of embedding dimen- sions, 0.00 (Wii,..., WKd) E RR be the parameters of the last linear classification layer, and h(0,; x) E Rd be the embedding of an example x. Then, the (k, i)-th component of the Fisher information embedding f(0; x) can be formally expressed as
((0G:))R+ = Σ y=1
p(y|x: (.) wes log (gpp((xx00)) pa pp)))(0;;;
(31)
where ls E [1, [1, 11
Pa
ΣS exp'xx x, the softmax probability of an example x for the class
exp=x. *
k. and zx. the logit for the class of the example x.
Proof. The Fisher information matrix is defined the covariance of score function. When consider only diagonal components, it can be formally expressed as Eq. (32), since the expectation derive the
of the score function is 0 (Sourati et al. 2017). Using the results in Lemma C.I
16

---

Published as a conference paper at ICLR 2025
Fisher embedding
K
f(0,; x)k
Σ
plyl1, 0.) IV wns log x, 0.)]
(32)
y=1
@bLy
Σ
Py
(33)
Öurki
K
Σ
Py (1 = =y] - -pk)² h(0:x);
(34)
Pk ph) + pk
Σpu Py
h((0x)?
(35)
[Pe(1 pk:) + pē(11 pk)] h(0:x),
(36)
= P4 - px) x)?.
(37)
D COMPLETE PROOF OF THEOREM 4.3
Theorem 4.3. Restated. Let be the i-th component of the Fisher information embedding of an
arbitrary subset Sy. Under [0,.. S,)ll2 VIn0, |0. | = k, the optimal 8, that minimizes Eq. 13)
8; = =d///3 where e= = 22 Σ 0. 2/3
Proof. We aim find subset S, that satisfies arg ming, E.6.1 s;" when F(;,, S,)ll R+ = |0.1 l1/8; with a constraint [F, (:)ll= = k. Lagrange multipliers can be employed. To solve the Lagrange multiplier method, we define the constraint, by
As we aim to find the minimum of (s,,,..,,
[81 s),,)), the function that we wish to minimize, and
10
(($
$10.1)
k22 = 0,
(38)
1=1
10,1
t
[(1.....,
(39)
i=1
Then, we are able to construct the Lagrangian function [(81, 8101); A)
as
L(s .je......,9,,..11 (s1.....,.j)). 1g(s1....... s001).
(40)
Then, we find the extrema for every element 8,, Vi E (1,...,,(0A) 001) by taking the partial derivative of
10,|
10.
L(s [|0):...,$$0),:1)=
=
t
X
Σ
3 k2
(41)
1=1
1=1
0f(s1.... [[.s......8 810.1)
2s: = 0, Vi E (1....,0.1)
(42)
0s
t
ti 1/3
81
(43)
2X
17

---

over-sample two Published as a conference paper at ICLR 2025
Finally, each value of s, to find the value of the Lagrange multiplier À to substitute and finalize the proof.
10+1
4 )2/3 =
k
(44)
2X
1=1
10c1
(22)2/3
(45)
k
(2/01] 1i=1
0..
2//33/22
(: 8 2 0)
(46)
2h3/2
V 1/3
8
2/3
(47)
i=
8.8
Ct 1/3
(48)
E IMPLEMENTATION DETAILS
Table 5: Detailed experiment settings.
Hyperparamters
SpliCIIFAR10
SpliCCIFAR100
ImageNeny
architecture training epoch
ResNet18
ResNet18
ResNet18
30
16
32
Training
optimizer
SGD
SGD
Configuration
momentum
0.9
weight decay
10-1
10-6
learning rate (Ir)
0.01
0.01
0.01
scheduler
Cosine Anncaling
Cosine Anncaling
Cosine Anncaling
For each CL task, we perform multi-round ACL following the conventional AL setup (Ash et al. 2019). For each experiment, we use two different sizes for a memory buffer, (100, 200} for SplitCT- FARIO, (500, 1000} for SplitCIFARI00 100, and (2000, 5000} for SplitTinylmagehi 2000, and 3000 examples during ten rounds of AL for SplitCIFAR10, SplitCIFAR 100, and SplitTiny- ImageNet for each task, respectively. In terms of over-sampling in Section 4.5
We query 1000,
times the original query size.
The overall optimization and training setups shown in Table Among the four CL algorithms, GSS (Aljundi et al. 2019) and DER (Buzzega et al. 2020) require hyperparameters to fix. For GSS, the number of random samples for determining the maximal cosine similarity is set 5. For DER, the values of a and | are given weights of 0.1 and 0.5, respectively. represents the weight allocated the mean-son equare error, while 3 represents the weight assigned the cross-entropy error. All AL baselines, except for BAIT (Ash et al.) 2021), need no additional hyperparameters to be specified. The oversampling rate for forward greedy optimization configured as 2. For AccuACL, we choose the top-10 embeddings per class for SplitCIFAR10 and the top-5 embeddings for SplitCIFAR 100 and SplicTiyylmageNee in order to compute the distribution score. We conduct our experiment based on the Avalanche codebase (Carta et al. 2023 All experiments are implemented with PyTorch 1.12. and performed with NVIDIA GeForee RTX 4090 24GB on CUDA version 12.0. All experiments except for BAIT repeated three times, and the average and standard deviation are reported.
F EXTENDED EXPERIMENTS
F.1 ROBUSTNESS OF ACCUACL WITH VARYING TASK ORDERS
To further verify the robustness of our technique, we run additional experiments on the 100CCIFAR100M/=50000 dataset, where the original data is split into ten tasks of ten classes each
18

---

Published as a conference paper at ICLR 2025
Table 6: Performance comparison of AccuACL and Uniform across multiple task order permutations on ApliCIIFARI0((M--533) trained with ER.
Sequential
Perm,
Perm.
Perm. #3
Perm. #4
AL Method
(1o(t)
Fia(4)
Abo(t)
Fio(4)
Au((t)
Fio(4)
Aio(t)
Fio(L)
A1o(t)
Fin(4)
Uniform
10.9:04
63.7=0s
11.7404
62.0e03
11.+02
61,0+12
11.9-04
62,5s05
11.904
61,4e05
AccuACL
14.1-07
25.8
14.7.11
52.8.11
14.6205
52.4108
15.4208
53.9
15.8104
51.2.0.4
in sequential order. To investigate the impact of task order, we generate four random permutations of the tasks. Table Table demmonsrates that AccuACL consistently outperforms the second-best method, Uniform, in all permutations. Our results indicate that our method is highly robust to changes in task order, further establishing its efficacy.
F.2 PERFORMANCE ANALYSIS OF AL STRATEGIES THROUGHOUT AL ROUNDS
Entropy BADGE
AccuACL LeastConf
Uniform kCenter
30
50
25
t 45
20
40
15
10 15
20
25
0
10
15
20
25
AL Round
AL Round
(a) Avg. accuracy over AL rounds.
(b) Forgetting over AL rounds.
Figure 4: Comparison AL strategies: (a) average accuracy of SplitCIFAR on ER throughout
AL rounds; (b) forgetting of SplicCIFAR100 on ER throughout AL rounds.
Figures 4 a) and (b) show the performance of ER over the AL rounds at the fifth task on SplitCI- FAR 100. Here, AccuACL performs the best, and the performance gap to the AL baselines gets larger as the AL rounds progress. Notably, AccuACI. shows very low forgetting, indicating its superiority in preventing catastrophic forgetting. Interestingly, the performances peak around the tenth round of AL. Since the AI. was conducted up 10 rounds for previous tasks, extending beyond 10 rounds for the new task results in imbalanced data distribution, likely causing the observed performance drop.
F.3 EFFICACY OF FISHER INFORMATION MATRIX ESTIMATION VIA MEMORY BUFFER
Table 7: Comparison of forgetting and average accuracy between AccuACL and AccuACLFull on DpliCCIFAR10((/-111 using DER++
SplitCIFAR100
AL Method
Aio(t)
F1o(1)
Uniform
35.9+07
21.5=005
Entropy
31.7402
27.5=08
LeastConf
33.1+10
27.0=06
kCenter
35.0407
24.9=005
BADGE
34.1420
27.7=07
AccuACI
36,3404
15.0=as
AccuACLFull
33.9109
14.2:008
To investigate the influence of using memory buffer for estimating the target Fisher information matrix in rehearsal- CI., we design modified version of AccuACI, AccuACI.uul. In contrast
19

---

Published as a conference paper at ICLR 2025
to the original AccuACL, which employs memory buffer, AccuACLFull leverages all data from previously seen tasks to approximate the target Fisher information matrix. As shown in Table 7 AccuACL achieves competitive results compared to AccuACLFull, indicating that minimal memory buffer (1000 examples) suffices for adequate estimation. While AccuACI reduces forgetting due more comprehensive parameter estimation, results in lower average accuracy. We hypothesize that AccuACLFull's extensive data use of the past improves its capacity to retain important parameters from previous tasks that cannot be effectively captured by replaying memory buffer. However, in practical scenario of rehearsal-bars -ased CL, estimating the Fisher information matrix using only the memory buffer strikes a better balance between retaining past knowledge and integrating new tasks by efficiently selecting new task examples without conflicts with the memory data, instead of attempting to preserve every knowledge of the past.
G LIMITATIONS & FUTURE WORKS
While AccuACI. has repeatedly shown efficacy in ACI, scenarios, we have yet to validate the suitability of our approach in realistic CL situations, such noisy labels (Bang et al.) 2022 Park ct al. 2022). blurry tasks (Bang et al.) 2021). and imbalanced data (Chrysakis and Moens (020). Moreover, AccuACL necessitates the of memory buffer from rehearsal-basee CL, rehearsal- free approach AccuACL will be a promising approach, possibly with the use of regularization- basdd CL methods further accurately maintain the target Fisher information matrix without a memory buffer. Furthermore, as AL for vision-language models (VLMs) (Bang et al.) (022) has shown possibilities, devising ACL strategy for VLMs to be promising research field.
20


Reviews:
[{"review_id": "tiHphRX72n", "reviewer": "Reviewer_k3jB", "summary": "This paper addresses Active Continual Learning (ACL) and particularly the problem of catastrophic forgetting when active learning and continual learning meet. The authors propose propose AccuACL, (Accumulated informativeness-based Active Continual Learning,) which attempts to achieve an optimal balance between learning new knowledge while preserving past knowledge in ACL. They model the accumulated informativeness via the Fisher information matrix,  through the approximation with a small memory buffer, the model parameters, and the unlabeled data pool for the new task. The are theoretical guarantees and extensive experimentation.", "questions": "What are the characteristics of problems where the method works well? Additionally, what are the characteristics of problems where the method does not perform well?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The novelty of the method is the application of active learning in continual learning. A technique that is applied in active learning is modified to balance learning and prevention of forgetting in the continual setting domain.\n\nSince the proposed solution has a theoretical basis, this basis can be extended to continual learning. The paper includes a theoretical analysis of space and time complexity and also the Fisher Optimality preserving properties.\n\nThe proposed method and the intuition behind it are clearly explained in the paper.\n\nThe experimentation evaluates the proposed methods from a number of perspectives.", "weaknesses": "What are the open problems or future directions related to your work? Adding this to the paper would improve the paper's discussion of broader impact and potential future work. This should be additional to the limitations section in the Appendix.\n\nTypos: Please do a grammar check on your paper. There is a typo in the Abstract “informativeness”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XJJyiihdzc", "reviewer": "Reviewer_XnjR", "summary": "This paper introduces a novel active learning (AL) approach for active continual learning (ACL) that effectively balances the prevention of catastrophic forgetting with the capacity to quickly learn new tasks. The authors present an accumulated informativeness-based method, termed AccuACL, which extends Fisher information-based AL to a class-incremental setting. By leveraging Fisher information embeddings, AccuACL estimates informativeness through a combination of magnitude and distribution scores, offering a more efficient way to rank data for selection. Experiments across four continual learning (CL) methods on three CL benchmarks show that AccuACL significantly improves CL performance, outperforming traditional AL baselines in terms of both accuracy and forgetting metrics.", "questions": "1. In the main experiments, it’s intriguing to see ACL methods, especially AccuCL and random, outperform training on the full dataset. What annotation budget is used, and how many runs are conducted in this experiment? Also, why is there no standard deviation for the BAIT rows?\n2. Figure 2 shows an interesting trend: accuracy and forgetting metrics peak around AL round 10 and then decline for all AL methods. Could you provide an explanation for this phenomenon?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper addresses active continual learning (ACL), an underexplored area in the intersections of AL and CL.\n- In addition to promising experimental results, the paper provides a solid theoretical foundation for the proposed method, as well as an efficient approximation strategy for computing accumulated informativeness scores.\n- The authors include analyses of both time and space complexity, demonstrating the feasibility of implementing AccuACL.", "weaknesses": "-  Some experiment details like annotation budgets and number of runs are not provided, which are crucial for reproducing the results.\n-  Further analysis of the specific data points chosen by each AL method across different CL settings could offer more insight into optimal query selection. Additionally, an examination of annotation budget sensitivity (as many methods peak around round 10 in Figure 2, suggesting diminishing returns) and the effect of task order (since CL can be highly sensitive to task sequencing) would add depth to the findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FWIJVdfD2E", "reviewer": "Reviewer_Fzgq", "summary": "The paper proposes an active learning method for continual learning that balances the prevention of catastrophic forgetting and the ability to quickly learn new tasks.", "questions": "See the previous sections", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Studying active learning in the continual learning setting is interesting.", "weaknesses": "1.\tThe novelty of the paper is somewhat limited, as the use of the Fisher information matrix and replay mechanisms are already well-known methods in continual learning.\n\n2.\tThe citations in the related work section are outdated, giving the impression that the authors may not be fully aware of the latest advancements in continual learning, particularly beyond replay and regularization-based methods. Please see the survey [1] and the related work section of [2] and [3]. \n\n3.\tIt would be beneficial to examine how the system and baseline models perform as the number of queries increases.\n\n4.\tThe baseline methods used in this paper are also outdated. See those in [2] and [3]. \n\n5.\tThe current accuracy results are quite low and significantly fall short of existing state-of-the-art methods.\n\n6.\tGiven that your method selects only a small number of samples, it would be essential to compare it with recent few-shot continual learning approaches [4], [5], [6], and [7].\n\n7.\tCatastrophic forgetting does not seem to be the main problem anymore. Inter-task class discrimination may pose a greater challenge. Please refer to [2] and [3]. \n\n8.\tThe results on the full dataset are notably low too, and it is unclear what method was used in the “full” setting. Please see the papers mentioned above for comparison.\n\n9.     The paper is overly formal and not easy to follow. Some more descriptions of intuitions will be helpful. \n\n10.   The space completely is very high for continual learning.\n\n[1] Wang et al. A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024.\n\n[2] Lin et al., \"Class Incremental Learning via Likelihood Ratio-Based Task Prediction,\" ICLR 2024.\n\n[3] Wang et al. BEEF: Bi-compatible class-incremental learning via energy-based expansion and fusion. ICLR 2023. \n\n[4] Mazumder et al. Few-Shot Lifelong Learning. AAAI-2021 \n\n[5] Zhou et al. Forward compatible few-shot class-incremental learning. CVPR- 2022.\n\n[6] Song et al. Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning. CVPR-2023 \n\n[7] Tian et al. A survey on few-shot class-incremental learning. Neural Networks. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6Jz6jpBM8f", "reviewer": "Reviewer_fGgu", "summary": "The paper focuses on active continual learning (ACL) and proposes AccuACL (Accumulated Informativeness-based Active Continual Learning) to effectively balance the prevention of catastrophic forgetting and the rapid learning of new tasks. This method proposes the accumulated informativeness modeled by the Fisher information matrix, which is further approximated by the diagonal Fisher information embedding to reduce the calculation cost. Experimental results show that AccuACL outperforms existing active learning baselines across various continual learning algorithms.", "questions": "1. In active learning, why is unlabeled data not utilized in an unsupervised manner to further enhance the training process?\n\n2. In Figure 3, the choice of lambda depends on the amount of unlabeled data. Does this conclusion also hold in imbalanced datasets?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation is clear and well-justified, aiming to enhance previous methods by addressing both forgetting and rapid learning in subset selection for training.\n\n2. The proposed method is based on the two theorems, making the algorithm be more confident and efficient.\n\n3. Experimental results demonstrate the strength of the proposed method, significantly improving accuracy while reducing forgetting.\n\n4. The paper is well-organized and most components are clear and easy to understand.", "weaknesses": "**Major**\n\n1. The method employs buffer data to evaluate the current parameter importance with respect to the historical task. However, a small buffer can not adequately represent the complete data distribution of historical tasks, which is easy to overfit. Then the network may memorize these buffer samples and exhibit low parameter sensitivity, leading to an incorrect Fisher information matrix.\n\n2. The paper focuses solely on the Fisher information matrix of linear classifiers. However, linear classifiers in CL often exhibit task-recency bias, making them prone to favoring new classes [1,2,3]. This raises concerns about the accuracy of the Fisher information matrix calculation. How can an unbiased Fisher information matrix be derived from a biased linear classifier?\n\n3. The description of the greedy algorithm for subset selection is unclear (line 6 in Algorithm 1 requires further explanation). My understanding is as follows:\n\n   a) Randomly sample a subset and calculate the Fisher information embedding according to Theorem 4.2.\n\n   b) Calculate the magnitude score and distribution score of the current subset.\n\n   c) Randomly sample a lot of subsets and repeat steps a) and b). After that, comparing scores across these subsets, and then, selecting the subset with the highest score as the final selection subset $X$.\n\n4. If my understanding is correct, how to guarantee the convergence of the greedy algorithm and ensure that it can choose the optimal subset? Additionally, detailed settings should be provided, such as the subset sampling method, whether subsets intersect, and the size of the subsets.\n\n5. Regarding the budget experiment. There is no clear definition of budget and no experiments pointing out the budget limitation.\n\n\n**Minor**\n\n1. In Table 3 (lines 472-474), how to incorporate historical knowledge into the greedy algorithm? Why use the memory buffer as the initial subset?\n\n[1] Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. CVPR, 2021.\n\n[2] Cba: Improving online continual learning via continual bias adaptor. ICCV, 2023.\n\n[3] Overcoming recency bias of normalization statistics in continual learning: Balance and adaptation. NIPS, 2024.\n\nIf all my concerns are addressed, I'll gladly increase the score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dxK0Fju6qn", "reviewer": "Reviewer_aS7J", "summary": "This paper focuses on the scenario that performs Active Learning (AL) in the Continual Learning (CL) process. The paper states that previous AL methods only focus on learning new tasks, which leads to catastrophic forgetting. The authors introduce a novel method called **AccuACL** that improves fisher information-based AL to make a balance between learning new tasks while preserving old knowledge. Empirical results show that **AccuACL** outperforms existing AL methods in standard CL metrics. Overall, this paper opens an important problem and propose a reasonable method to mitigate it.", "questions": "### Method\n1. In Section 4.5, why uses distribution score first and then magnitude score later. Does the order of these two score matters? This is not discussed in Section 5.3 either. \n### Future work\n1. Does using regularization-based methods can further improve ACL performance? Maybe the authors can consider this idea to further improve ACL.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The problem is important, especially for online deployed models. \n* The extensive experiments partly support paper's claims and the effective of the method. Meanwhile, ablation studies help to understand the proposed method. \n* The paper provides sufficient details and code in Appendix for readers to reproduce the experiments.", "weaknesses": "### Method and theory \n1. The proposed method only works for memory-based continual learning methods. It can not be applied on naive continual learning or regularization-based continual learning methods. I understand that memory buffer is necessary for leveraging existing AL method, but I encourage authors to add this as limitations or propose future directions to handle this issue. \n2. In Figure 1, it seems like **AccuACL** is good for selecting new samples that related to the learned *features*. However, it is not clear what are those features, and Section 4 does not explain this part clearly either. \n3. More background for Active Learning is needed for CL community readers.\n\n### Experiment\n1. While theoretical space complexity shows **AccuACL** is more efficient than BAIT, no experiment results are support this claim. An additional experiment for memory usage is needed, like Figure 2(c) for time complexity.\n2. In Introduction and Section 5.2, authors claims that:\n> \"these AL strategies typically pay more attention to new features...\", \"most AL baselines that most AL baselines that focus only on learning new tasks often perform worse the Uniform.\"\n\nHowever, these claims are not confirmed by the experiments. Compared with the Uniform strategy, some existing AL baselines have better forgetting score, but worse average accuracy. This phenomenon might indicates that existing AL baselines are not learning new tasks well either. Learning accuracy metric [1] may be able to support paper's claim. If the claim is true, I expect to see existing AL methods might have higher Learning accuracy. But I am not sure what will the final results be. I encourage authors to do quick additional experiments on CIFAR-10 to see their claims is true or not. \n\n# Reference\n[1] Wide Neural Networks Forget Less Catastrophically, ICML 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)