Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
Authors: Run He
Di Fang
Yicheng Xu
Yawen Cui
Ming Li
Cen Chen
Ziqian Zeng
Huiping Zhuang
Venue: ICML 2025 poster (2025)
Keywords: Continual learning
Class-incremental learning
Exemplar-free
Semantic shift estimation
Primary area: deep_learning
TL;DR: This paper addresses semantic shift and decision bias in exemplar-free class-incremental learning via a dual-projection shift estimation and a classifier reconstruction.

Abstract: Exemplar-Free Class-Incremental Learning (EFCIL) aims to sequentially learn from distinct categories without retaining exemplars but easily suffers from catastrophic forgetting of learned knowledge. While existing EFCIL methods leverage knowledge distillation to alleviate forgetting, they still face two critical challenges: semantic shift and decision bias. Specifically, the embeddings of old tasks shift in the embedding space after learning new tasks, and the classifier becomes biased towards new tasks due to training solely with new data, hindering the balance between old and new knowledge. To address these issues, we propose the Dual-Projection Shift Estimation and Classifier Reconstruction (DPCR) approach for EFCIL. DPCR effectively estimates semantic shift through a dual-projection, which combines a learnable transformation with a row-space projection to capture both task-wise and category-wise shifts. Furthermore, to mitigate decision bias, DPCR employs ridge regression to reformulate a classifier reconstruction process. This reconstruction exploits previous in covariance and prototype of each class after calibration with estimated shift, thereby reducing decision bias. Extensive experiments demonstrate that, on various datasets, DPCR effectively balances old and new tasks, outperforming state-of-the-art EFCIL methods. Our codes are available at https://github.com/RHe502/ICML25-DPCR.

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
Run He Di Fang' Abstract
Yicheng Xu² Yawen Cui
Ming Li
Cen Chen
Ziqian Zeng'
Huiping Zhuang'
Semantic Shift Embodding Space
(b) Decision Blas towards New Tasks
Exemplar-Free Class-Incremental Learning (EF- CIL) aims to sequentially learn from distinct cat- egories without retaining exemplars but easily suffers from catastrophic forgetting of learned knowledge. While existing EFCIL methods lever- age knowledge distillation to alleviate forgetting. they still face two critical challenges: semantic shift and decision bias. Specifically, the embed- dings of old tasks shift the embedding space after learning new tasks, and the classifier be- comes biased towards new tasks due training solely with new data, hindering the balance be- tween old and new knowledge. To address these issues, we propose the Dual-Projection Shift Es- timation and Classifier Reconstruction (DPCR) approach for EFCIL. DPCR effectively estimates semantic shift through a dual dual- projecctoo,,,,, which combines a learnable transformation with row- space projection to capture both task-wise and category-wise shifts. Furthermore, mitigate decision bias, DPCR employs ridge regression to reformulate a classifier reconstruction process. This reconstruction exploits previous in covari- ance and prototype each class after calibration with estimated shift, thereby reducing decision bias. Extensive experiments demonstrate that, on various datasets, DPCR effectively balances old and new tasks, outperforming state-of-tt of-the-art EF- CIL methods. Our codes are available at htt ttts: / ///itt//l com RHe /I//5/22///22
O
Emebddings extracted by old backhone
mebddings new lask extracted by new backbone Guxtracted by new backhone Decision foundaries
Emebddings old task
... Semantic shift
Figure I. After learning new tasks, (a) the embeddings old tasks undergo semantic shift in embedding space, (b) and the embed- dings are more likely to be classified as new classes.
ner. This paradigm addresses that data is only accessible at specific times or locations, facilitating the accumulation of machine intelligence in dynamic real-world scenarios. How- ever, CIL encounters the challenge of catastrophic forgetting (CF) (Belouadah et al., 2021), where the model rapidly loses previous knowledge when acquiring new information. To address forgetting, exemplar-bass CIL (EBCIL) (Hou et al., 2019; Zheng et al., 2024; Bian et al., 2024) stores a portion of previous data as exemplars for replaying. However, this method may be limited by privacy concerns storage ca- pacity constraints. By contrast, Exemplar-free CIL (EFCIL), which does not store previous samples (Rypešć et al., 2024; Masana et al., 2023), is more applicable in practical situa- tions.
Recent EFCIL approaches mainly leverage knowledge distil- lation to mitigate CF (Li & Hoiem, 2017: Feng et al., 2022: Gao al., 2024a). However, their performance can be con- strained by two key challenges: semantic shift in learned representations caused by incremental updates to the back- bone, and decision bias in classifier training due to the absence of historical data (see Figure 1). These challenges lead to stability-plasttiii dilemma (Mermillod et al., 2013), where the model struggles to balance preserving old knowl- edge (stability) with acquiring new knowledge (plasticity). Specifically, when adapting the backbone new tasks, the embeddings of old classes inevitably shift in the embedding space (Yu et al., 2020), reducing the compatibility of up- dated representations with previously learned classes and thus compromising stability (Zhu et al., 2021a). Meanwhile, updating the classifier via iterative back poop-apopagttion (BP) without access to previous data often results in prefer-
ICML25 -DPCR.
1. Introduction
Class- Hass-incremental learning (CIL) (Zhou al., 2024; Rebuffi et al., 2017) enables models to acquire knowledge from distinct categories arriving sequentially in task-wise man-
'South China University of Technology "Institute of Science Tokyo 'The Hong Kong Polytechnic University "Guangdong Lab- oratory of Artificial Intelligence Digital Economy (SZ). Corre- spondence to: Huiping Zhuang <hpzhuang @ ceriiianngcuuuuuuuuc.. scut.edu.en>.
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Frec Class-Incremental Learning
ence for the newly learned data, introducing decision bias (Zhuang al., 2024c; Hou al., 2019). This bias skews the classification towards new tasks', imposing preference on plasticity at the expense of stability.
shifts across tasks and categories, DP enables DPCR to effectively address semantic shift.
To address the decision bias, we formulate the RRCR. RRCR establishes BP-ffree classifier training framework based on ridge regression, mitigating biases arising from new tasks. When combined with the estimated shift, RRCR yields a less biased classifier.
Existing solutions struggle to simultaneously address both semantic shift and decision bias, resulting in imbalance be- tween stability and plasticity. For instance, several works resort freezing backbone to prevent alterations of the old representation thus avoid semantic shift (Zhuang al., 2022b; Goswami et al., 2023; He et al., 2024b). However, this strategy restricts the backbone's ' ability to adapt rep- resentations for new tasks, thereby highly limiting plas- ticity. To tackle decision bias, several methods adopt a Nearest-Class Mean (NCM) classifier based on prototypes to avoid direct classifier updates (Hou et al., 2019; Goswami et al., 2024; Gomez-Vill et al., 2024). However, as non- parametric classifiers, NCM classifiers heavily depend on the quality of the learned representations, which are sus- ceptible to degradation caused by semantic shift. Also, the absence of trainable parameters could limit their adaptabil- ity across task. To address plaltiiiy dilemma, it is imperative to propose a novel EFCIL technique capable of continuously adapting the model to new tasks, effectively addressing semantic shift, and maintaining well-balanced parametrized classifier.
Extensive experiments on various benchmark datasets demonstrate that, our DPCR effectively handles semantic shift and decision bias, outperforming the state-of-the-art (SOTA) EFCIL methods with a good balance between stability and plasticity.
2. Related Works
EFCIL can be categorized into regularization-bart based CIL (Li & Zeng, 2023; Zhao et al., 2024; Gao et al., 2024b), prototype- based CIL (Zhu et al., 2021a; Toldo & Ozay, 2022; Wang et al., 2023), and analytic continual learning (Zhuang et al., 2022b; He et al., 2024a; Zeng et al., 2025).
Regularization-based CIL imposes additional constraints activation key parameters to mitigate forgetting. Knowledge Distillation (KD) is often employed to formu- late constraints to mitigate forgetting. For instance, KD can be utilized to restrict activation changes (Li & Hoiem, 2017), establish attention distillation loss (Dhar et al., 2019). Other methods impose constraints on crucial weights from previous tasks (Kirkpatrick et al., 2017). To quantify the importance of parameters, EWC (Kirkpatrick et al., 2017) utilizes a diagonally approximated Fisher information ma- trix. Various techniques employ different modeling strate- gies, such as the path integral of loss (Zenke et al., 2017) and changes in gradients (Aljundi et al., 2018). However, the constraints imposed on model updates can hinder the learning of new tasks (Zhuang et al., 2024b).
In this paper, we propose a Dual-Projection Shift Estimation and Classifier Reconstruction (DPCR) approach to handle the semantic shift and decision bias simultaneously. DPCR incorporates a dual projecctonn (DP) to address the seman- tic shift, and employs ridge perresiio-bbsee classifier reconstruction approach (RRCR) to mitigate the decision bias while benefiting from a parametrized classifier. DP formulates a learnable Task-wise Semantic Shift Projection (TSSP) together with a Category Information Projection (CIP), enabling DPCR to accurately estimate shift and im- proves stability. RRCR reformulates the classifier train- ing as BP-free classifier reconstruction process using ridge regression. RRCR can leverage previous information en- coded in covariance and prototype of each class to obtain a well-balanced classifier with calibration with shift. Our key contributions are summarized as follows:
Prototype-based CIL employs prototypes to preserve deci- sion boundaries across old and new tasks. The prototypes are typically the feature means of learned classes and mul- tiple studies have investigated how to reform past feature distributions from prototypes (Petit et al., 2023; Malepathi- rana et al., 2023; He et al., 2024b). These techniques include prototype augmentations (Zhu et al., 2021b), prototype se- lection (Zhu et al., 2022) or generating pseudo-features (Rypešé al., 2024). The reformed distribution is used to guide classifier training to maintain old boundaries. How- ever, due to semantic shift, previous prototypes may become inaccurate. Some techniques address this issue by estimat- ing prototype shift (Shi & Ye, 2023; Malepathirana et al., 2023). For examples, SDC (Yu et al., 2020) utilizes Gaus- sian kernels to capture the shift by estimating the translation of feature means, ADC (Goswami et al., 2024) uses ad- versarial samples to improve the accuracy of estimation, and LDC (Gomez- Villa et al., 2024) introduces a learnable
We present DPCR, an EFCII technique that addresses both semantic shift and decision bias, ultimately balanc- ing stability and plasticity in EFCIL.
To address the semantic shift, we propose the DP to cap- ture the shift. DP comprises TSSP for learning task- specific shifts and CIP for offering category-specific information. By comprehensively capturing semantic
'This phenomenon of model's tendency to favor new task is also termed as task-ree mesk-recency bias (Rebufli et al., 2017). Several works attribute it to the biased classifier (Zhuang et al., 2024c). We use decision bias to clarify that the bias originates from classifier.
2

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
projection to estimate the shift. However, they can obtain less effective estimations since they either only consider the embedding translation (Yu et al., 2020; Goswami et al., 2024) or focus solely on task-wise shift (Gomez- Villa et al., 2024). Moreover, LDC needs iterative BP-training to get the projector, inducing more computation cost. Prototype-based CIL achieves SOTA performance but faces the dilemma of stability and plasticity. That is, the representation is plagued by semantic shift and the classifier may still be biased by replaying previous distribution via BP.
model have met. The goal of EFCIL is train the model with Diaia at task & without access of data from previous tasks and validate the model on Di.d after training in task t.
In task t, to adapt the model to new tasks and reduce for- getting, we follow LwF (Li & Hoiem, 2017) to use KD on logits with cross-entropy loss to train the model. In task t, the loss function is
Lep = (h Th (1o, (2i), + (LLd
(I)
where
Analytic continual learning (ACL) is a recently devel- oped branch of EFCIL that uses closed-form solutions to train the classifier. AL (Guo & Lyu, 2001; Zhuang et al., 2022a; 2021) a technique that utilizes least squares (I.S) to yield closed-form solutions to neural networks training, and ACIL (Zhuang et al., 2022b) first introduces AL to the CIL realm. With frozen backbone, ACII is developed by reformulating the CIL procedure into a recursive LS form obtained by solving a ridge regression problem. The ACL family has demonstrated significant performance and has been widely adapted to various scenarios (Zhuang et al., 2023; Fang al., 2024; Yue et al., 2024; Xu et al., 2024). However, a common issue among existing ACL methods is the limited plasticity due to the frozen backbone. Our DPCR is inspired by ACL and uses the ridge regression to construct a less biased classifier. However, our DPCR does not need to freeze backbone and strikes a good balance between stability and plasticity.
Ced = Lee (fa (2))), hT (fo, (2))).
(2)
a is the weighting parameter of KD, and 1g hr" are auxiliary classifiers in task - 1 and respectively.
Although with KD, training on new task could still leads to semantic shift (Li al., 2024) and decision bias(Gao et al., 2025). DPCR formulates DP and RRCR to address these issues respectively.
3.2. Shift Estimation via Duol-Projection
In this section, we propose the detailed formulation of DP to capture the semantic shift. DP is consisted of a task-wise semantic shift projection (TSSP) to capture the semantic shift between two tasks and a category- information projection (CIP) to provide category-related information.
Task-wise Semantic Shift Projection. Since we can access the backbone 0:-1 at task L, the semantic shift between the backbone 0, and 9.-1 can be estimated by a learnable model that captures the difference between embeddings extracted by 6:-1 and 6, Inspired by LDC (Gomez-Vill et al., 2024), we introduce a linear projection parameterized by pt-1 E Rddd to transform the embeddings obtained by 8 to those obtained by 0..
3. Proposed Method
In this section, we provide a detailed description of the proposed method. The paradigm of DPCR has three parts: incremental representation learning, shift estimation via dual-projection, and ridge regression-based classifier recon- struction, An overview of DPCR is shown in Figure 2.
In task suppose embedddings of the i-th input extracted by the backbone 0, and 0,- are denoted as x = fo. (Xe,) and و 0.-1 fer- (X,,,) respectively, and the one-hol hot label is onchot(yr.)). then the embeddings and one-hot hot label of all the input data in task-t can be stacked as x" x. and Y, respectively:
3.1. Incremental Representation Learning
Prior to further description, we provide definitions of EFCIL here. In EFCIL, the model learns from training data a task- wise manner. For an EFCIL problem consisting T-tasks, in task f, the dataset is denoted as D,- = {x,,, ve. H. where Xe,i E [xe is the i-th input image in task l, i denotes the label of the i-th input data, and N₂ is the total number of samples in task t. The label set of task is denoted as C2 and C, nC, = 0i * f) in EFCIL. In this paper, we focus on the cold-start setting (Magistri et al., 2024), where the model is initialized randomly and the all tasks contains the same number of categories (i.e., |Co = C,₁l = C.j E 1 T). In task t, the backbone is denoted as fo. c + Rd (parameterized by 0.) and the embeddings extracted by 0, is classified via a mapping gd + 18L, where d is the feature dimension and 4 = tC is the number of classes that the
onchot(gg.)) onchot(gn.)
x
x
(3)
0c-1 3r,
onehot (g...)
Then the objective is to obtain a projection that satisfies X" X pt We encode the information of se- mantic shift in pt-1-st by minimizing the difference be- tween these two terms. In task t, the optimization problem is
argmin Euse = | - x 1-1111 p 2-14
p
(4)
3

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Frec Class-Incremental Learning
(a) Incremental Representation Learning
(b)
via Dual-projection
Eq. (12)
1900
P
f
TSSP
D
Lx
D
Lyse
CIP
DOod
fe.
for
p p4-1-t
labels
Lrce
TSSP CIP AA
Ta
Task
(c) Ridge Regression-based Classifier Reconstruction
New Info
D
for
4 Mie
yoeC
W
W
Dual Projection
Integrated Info for CR
{ 1.s Picti-1a
p peed
RRCR
CN
TSSP
1
Store for next task
Old Info
CIP
Calibrated
Old Info
Figure An overview our proposed DPCR. (a) At task I, the backbone is first trained with new data to learn new representation. (b) After the representation learning. shift estimation is conducted with dual projeerojtttion (DP), which is consisted of TSSP and CIP. (c) With the DP. the RRCR reconstructs the classifier based on calibrated covariance and prototypes.
where || is Frobenius-norm. The optimal solution to Eq. (4) is pr- pr-1=t x x" where T denotes transpose operation and II-11* is the Moore-Penrose (MP) inverse (also referred as generalized inverse pseudo-inverse) (Guo & Lyu, 2001). If x is of full-column rank, x.:-1f
ple but effective training-free row space projection to pro- vide category-related information. Given the class E C--1, rows in - t-iic (stacked matrix of class c's embeddings) span the space that contains class c-related informa- tion. To coordinate the TSSP with category information, we further project p onto each class's row space. How- ever, it is memory iemtory-nttesive to store X X due to the large quantity of samples. Here we use the uncentered covariance P
(x," - x Here we approximate the MP- inverse by adding a negligible term e = 10-9 to avoid ill-matrix condition, then the projector can obtained as
X t- x t-1,c construct row space projector
pt 1-+
(x - x. + el) x T x (5)
since it shares the same row space X X x-I.e By applying
where I is the identity matrix.
singular vector decomposition (SVD) to $ 1--1ee we have
By learning the projection, we can model the shift between two tasks and calibrate the previous embeddings. That is, given _ we can estimate X = pt However, TSSP does not consider specificity of each cat- egory, and it can be sub-optimal since all the categories within task are different but share same estimation of shift. Here we provide toy case in Figure 2 (b) to illustrate this issue. For task-t 1 that contains "dog" and "plane", TSSP can estimate the shift to task t and the embeddings of "dog" and "plane" can be calibrated. The calibrated em- beddings can move to the central position in space spanned by the embeddings of task-t due to the objective of task- wise error in Eq. (4). However, this process is sub-optimal since the "dog" is more similar to "cat" ideally. To fur- ther separate the calibrated embeddings, we should provide category-related information. We present the CIP to address this issue.
UL11,05 Er-1,o1
= -.I =VV(( 81-11
(6)
where U2-1.0 = -U 12 and Σ. E Ee Ei-L, diag(o), (1 o2, ..., σr) contains the non-zero singular values a; (i 0 r, is the rank of & ( (111))) Σi- contains 0 singular values, and Ui-1, [ui, U2 ..., u. contains the first singular vec- tors that span the row space of 1
Then we can constructed the projector of CIP as UTL. and the task-wise projector with category information can be constructed as
p!-1+t UULL
With p!-1-t the category- shift can also be captured.
The estimation of class within task t - can be extended to previous tasks. Without loss of generality, for the class
Category Information Projection. CIP formulates sim-

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Frec Class-Incremental Learning
c E C, E 1: : I 1, the embeddings extracted by Og can be estimated task by task. That is, estimating xf++ = x%, Pi in task i then propagate this process in the subsequent tasks. This process can be formulated as follow,
Here, Th is the covariance, pi, is the prototype of class c and N₂ is the number of samples in class
As shown in Eq. (13), the classifier can be constructed via covariance and correlation of each task without directly accessing previous data or embeddings. That is, the pre- vious information are encoded in 4% and H information T and H can be calculated directly and the classifier can be reconstructed. The formulation of re- construction leverages the encoded information of previous (9)
X9. 0L 3
X!:
0 i.
P
++
i.c' The new
where
pijjt1 = -p- Ü 1,0 1,0
tasks, then the classifier is not obtained by BP training with only new data, thereby reducing the bias issue.
ÜY.0, Σr 1,00 1,07 UIT =)) & =
x² i,
(10)
However, due to the constrain of EFCIL,, the embeddings of previous tasks cannot be extracted by 0, then the covariance and correlation can not be constructed directly. What we can directly obtained are the information extracted by 0,, i.e., 0: Th i.e> Ml,e, and H. and semantic shift affect the encoded information of previous tasks. Here, we incorporate the shift information estimated by dual projectooo in Section 3.2 to calibrate the semantic shift.
When implementing DP, the covariance can be calibrated in each task and such accumulative operation in Eq. 8 is not necessarily needed. In our implementation, the estima- tion is conducted across two adjacent tasks (please refer to Algorithm 1).
3.3. Ridge Regression-based Classifier Reconstruction
To address the decision bias, we formulate the RRCR to reconstruct the classifier based on encoded information of previous tasks.
Without loss of generality, here we consider the calibration in task t and assume this process has also been applied in previous task. Suppose the old information based on 0z-1 (i.e., { ( " - [ - are reserved, we can calibrate them with the dual--rojectton. The information after cali- bration in task & can be estimated via
Given the joint learning problems from task 1 to with dataset Disi = Dyrai U DDiuinn U... U Dirain, the embeddings after backbone 0g and the labels can be stacked as X and Y1.. The training of the classifier can be formulated as a ridge regression learning problem as follows
oe =
x.: 1,0
X i,c
=
-Ptt 1
p
0: 1.c
ppt pt-1-st 1-s
1.c
(16)
argmin W.
|| - X Will + 7 |/el.
(II)
1
Fi,c Mic
0 0: 1--p--
(17)
Ne
j=1
where y is the regularization factor of ridge regression and W. is the optimal weight for all the tasks that have seen. The optimal solution to Eq. (11)is is
C Σ OEC.
H i.c
T. Notlie
Vicc-
(18)
With the calibrated information of old classes and co- variance and prototypes newly computed in task 4, the integrated information set for RRCR can be formed by 0h
We-
(12)
The Eq. (12) can be further decomposed in wiie form, that is
t Then the classifier is reconstructed by
Let n p = {&%
9 " f 1i--1 U &% de JeEE. te pecc
W.-
X T
X be
+7I)
TY..
W. =
0.
H H
(19)
El oEC.
eEC,
i=1
1=1
9, t.
+ -11 -1
H
(13)
where
1,00
E= cEC.
1=1 oEC,
Σ
H Σ
H
(20)
1,c:
1.c
where
cEC,
OEC.
H
Σ oEC,
X T
Y
Σ
No yh,er
B.
(14)
Φ 0:
c 0, t,o
H = Σ
H
(21)
1.c
oEC.
oEC,
CEC,
N₂
With the Eq. (19), we reconstruct the classifier with both (15) rectified old information and knowledge from new cate- gories. The old information after calibration is encoded in
0
0,
0
0r
1
1,0
1,0
Hi.-
No
I
j=1
5

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Frec Class-Incremental Learning
@ and Hr, This help adjust the old decision boundary to adapt classifier to new tasks. Also, the new information is included in the form of &6 and H Thus the classifier can both ensure plasticity by including new information and maintain the stability by rectifying old boundary.
and we run three times for each methods with three different class orders and report the mean results. We set the regular- ization factor for ridge regression Y 200, 2000 and, 2000 and ImageNet- 100 respec-
for CIFAR- 100, tively. The implementation details of hyperparameters in compared methods can be found in the Appendix B.
For the subsequent learning task, we can store the inte- grated information set of calibrated covariance and pro- IcEC. totypes T,e Mi,eJ i=1: to construct & and ir, The memory consumption is d² + for each class.
Evaluation Metries. We adopt two metrics for evaluation, including average incremental accuracy and final accuracy. The overall performance is evaluated by the average incre- mental accuracy Auvg +- where A, indicates the average test accuracy after learning task t obtained by testing the model on Di.t- The other metric final accuracy A measures the model's final-task performance after all training agenda. Ar is an important metric as it reveals the gap between CIL and joint training, a gap that CIL strides to close.
During the training of DPCR, only the covariance and proto- type ( p need to be stored. Semantic shift estimated by the DP is used to calibrate the old information set (i.e., (#2 (00-1)
te .Pi Jimlit-1 for task é 1) then discarded.
Category-wise Normalization. In RRCR, calibrating with DP can introduce numerical imbalance to the classifier since P!= is not unitary. To address this problem, we propose a simple yet effective category-wise normalization (CN). Given the weight of the classifier W, = [[11, W2. ..., Wicl. where wj is the weight vector for j-th class, the CN is implemented
4.2. Comparative Study with State-of-the-art Methods
We evaluate our DPCR and various EFCIL methods us- ing the metries Auvy and Ams in the cold-start setting for T = 10 and T = 20. The comparison includes LwF (Li & Hoiem, 2017), SDC (Yu et al., 2020). PASS (Zhu et al., 2021b), ACIL (Zhuang et al.. 2022b). FeCAM (Goswami et al., 2023), DS-AL (Zhuang et al., 2024b), ADC (Goswami et al., 2024), and LDC (Gomez- Villa al., 2024). Among these methods, ACIL and DS-AL are ACL tech- niques that freeze the backbone after the first task, LwF is regularization based approach that utilizes KD as con- straint, while PASS, SDC, FeCAM, ADC, and LDC are prototype-based methods. PASS replays augmented proto- types on the classifier, FeCAM improves upon NCM classi- fier with a Mahalanobis distance-based metric while keeping the backbone frozen, and the other prototype- based methods estimate the semantic shift of prototypes and employ the NCM classifier. The experimental results are tabulated in the Table. 1. We also provide the results of EBCIL methods in Appendix C.
W =
W
2
WIC
(22)
[22 1
|wcel22
After the learning of Eq. (19), the CN is applied on the clas- sifier. The pseudo-code of learning agenda of our proposed DPCR is summarized as Algorithm in Appendix A.
Complexity Analysis. Here, we provide the complexity analysis of DP and RRCR. Suppose F is the FLOPs of the model, the time complexity of DP O(FN, NNd +dC + (dd)) is the sum of those of the feature extraction O (FNN,, calculation of the task-wise projection matrix O(N,d +d²), and the rectification of each class O(d + Cdd). Similarly, the time complexity of RRCR is O N. +dCC++++++ +dP++
+
4. Experiments
4.1. Experiment Setting
Main Results. As shown in Table I, for the final accuracy As, our DPCR demonstrates competitive performance on various settings of T on all the benchmark datasets. For example, the performance of DPCR on CIFAR-100 out- performs the second best method LDC with the gaps of 3.64% and 2.28% under the setting of T = 10 and 20 re- spectively. Similar leading patterns are also observed on Tiny-ImageNer and ImageNet- 100. It is worth mentioned that methods that freeze the backbone (e.g., ACIL, FeCAM, and DS-AL) are ineffective and demonstrate poor perfor- mance, whereas simple baselines like LwF and SDC can perform better. This observation highlights the limitations of freezing the backbone. Although freezing backbone can climinate the semantic shift, it sacrifice the adaptation in new tasks then obtain low performance. In contrast to ACIL
Dataset and CIL Protocol. We validate DPCR and com- pared methods on three popular benchmark datasets in EF- CIL: 100 (Krizhevsky et al., 2009), Tiny-ImageNet (Russakovsky et al.. 2015), and ImageNet-100 (Douillard et al., 2020). For the CII. evaluation, we follow the cold- start setting adopted in various EFCIL works (Magistri et al., 2024; Goswami et al., 2024). The model initialized ran- domly and all the categories are partitioned into T Tttasks evenly. We report the results of T 10 and 20. To validate the performance on fine-grained and large-scale datasets, we further provide the results on CUB200 (Wah et al., 2011) and ImageNet-Ik Ik (Russakovsky et al., 2015).
Implementation Details. All the experiments are con- ducted with the backbone of R8 (He et al., 2016)
6

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-nnerementa Learning
Table The last-task accuracy and average incremental accuracy compared methods and our proposed DPCR on three benchmark datasets. The data is reported as average after 3 runs with different class orders. Results in bold are the best within the compared methods the same setting.
CIFAR- 100
TinyImageNer
ImageNet 100
Methods
T=10
T=20
T=10
T=20
T=10
T=20
A
Awe
A
Asv
A
Awe
A
Awy
.Ar
Awv
A
Aavz
LwF (2017)
42.60
58.51
36,34
51.52
26.99
42.92
18.80
33.05
42.25
61.23
30.11
50.40
SDC (2020)
42.25
58.43
33.10
48.68
23.86
40.66
13.45
29.70
37.68
60.33
23.64
45.52
PASS (2021b)
44.47
55.88
28.48
42.65
23.89
36.82
12.50
25.38
36.52
52.02
19.59
31.55
ACIL (2022h)
35.53
50.53
27.22
39.58
26.10
41.86
21.40
33.60
44.61
59.77
33.05
48.58
FeCAM (2023)
34,82
49.14
25.77
41.21
29.83
42.19
22.69
34.48
41.92
58.21
28.64
43.04
DS-AL (2024b)
36.83
51.47
28.90
40.37
27.01
40.10
21.86
33.55
45,55
60.56
34.10
49.38
ADC (2024)
46.80
62.05
34.69
52.16
32,90
46.93
20.69
36. 36.14
46.69
65.60
32,21
52.36
LDC (2024)
46.60
61.67
36.76
53.06
33.74
47.37
24.49
38.04
49.98
67.47
34.87
54.84
DPCR (Ours)
50.24
3.64
63.21
11.54
38.98
2.22
54.42
1.30
35,20
1.46
47.55
10.18
26.54
2.06
38,09
10.05
52.16
12.18
67.51
10.04
38.35
13.48
57.22
12.36
and DS-AL, DPCR can take advantage of incremental repre- sentation learning with semantic shift estimation, enabling it to achieve strong performance. Compared to methods that estimate prototype shift (e.g., SDC, ADC, and LDC), our DPCR exhibits superior performance, showcasing the effective shift estimation of DP. We further analyze DP and RRCR in the following sections. Regarding Af, DPCR consistently delivers excellent performance, surpassing the compared methods.
methods ImageNet- 1k (Russakovsky al., 2015) with T=10 and the results are tabulated in Table 3. We use the same backbone of ResNet-18 18 and same under the same seed of 1993 to conduct the experiments. The hypeparameters are adopted the same as those in ImageNet- 100 experiments. As depicted in Table our DPCR can still outperform the compared methods.
Table 3. Comparative results on ImageNet- with T = 10.
ImageNet-Ik (T=10)
Ar (%)
Awy (%)
Evolution Curves of Task-wise Accuracy. To provide a comprehensive view the results, we display the evolution curves task-wise accuracy (i.c., Ar,t t E : 7) on all datasets. As depicted in Figure 3, starting with similar accuracy in task DPCR attains the highest performance after learning all tasks.
LwF (Li & Hoiem, 2017)
22.01
42.40
ACIL (Zhuang et al., 2022b)
32.28
46.61
DS-AL (Zhuang et al., 2024b)
33.67
48.84
ADC (Goswami al., 2024)
31.34
50.95
LDC (Gomez-Villa et al., 2024)
35.15
53.88
DPCR (Ours)
35.49
54.22
Validation on Fine-grained Dataset. To validate our DPCR on fine-grained dataset, we further include the re- sults on CUB200 (Wah et al., 2011) with T=5 in starts setting with the same seed of 1993. As shown in Table 2, our DPCR achieves superior results than other methods. On fine- -ne-ggained dataset like CUB200, DPCR can take ad- vantage of class-specific information via CIP, thus performs better in the fine-grained scenario.
4.3. Ablation Study
Ablation on Components in DPCR. We perform an abla- tion study of DP and CN on CIFAR-100 with T = 10. The results are presented in Table 4. Here, the baseline is to use the ridge regression to reconstruct the classifier (i.e., RRCR only) after the incremental representation learning. Task- wise semantic shift projection (TSSP), category information projection (CIP), and category-wise normalization (CN) incorporated into the RRCR sequentially.
Table 2. Comparative results CUB200 with T 5.
CUB200 (T=5)
A (%)
Auyg (%)
LwF (Li & Hoiem, 2017)
25,40
36.38
ACIL (Zhuang et al., 2022b)
21.14
33.14
Table 4. Ablation study of DPCR on CIFAR- 100 with
10.
DS-AL (Zhuang et al., 2024b)
21.28
32.36
Components
Ar (%)
Any (%)
SDC (Yu et al., 2020)
24.24
36.00
RRCR
32.17
44.89
ADC (Goswami et al., 2024)
28.84
39.44
RRCR+TSSP
40.86
55.76
LDC (Gomez-Villa et al., 2024)
28.70
39.09
TRCP+IS
45.56
62.15
DPCR (Ours)
29,51
39.44
RRCR-TSSP+CIP-CN
51.04
64.44
Validation on ImageNet-lk. To validate the performance on large-scale dataset, we compare DPCR and compared
As indicated in Table 4, using RRCR only exhibits low
7

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class- Class-nncremental Learning
(a) CIF
(b) Iiny-ImageNet
(c) ImageNet-
Accuracy (%)
60
60
13:41
1.44
60
1:246
40
100
75-
60
75
Accuracy
50
3:62
40
11.1
50
5::70
20
25
25
5
10
15
20
10
15
20
10
15
20
1.WF
SDC
PASS
ADC
LDC
ACH.
DS-AL.
FeCAM
DPCR (Ours)
Figure 3. Evolution curves of task-wise accuracy.
performance. While RRCR can produce a well-balanced classifier, it could struggle with semantic shift across learn- ing tasks. The inclusion of TSSP leads to a substantial im- provement in performance. This highlights that TSSP can mitigate the effects semantic shift. Furthermore, the ad- dition of CIP results further performance enhancements, underscoring the efficacy of incorporating projections with category-specine information. By introducing CN, the nu- merical challenges associated with dual- -rojjejjetttinn can be mitigated, leading to even better performance for our DPCR.
estimated shift. That is, both task-wise shift and category- specific details are encoded by linear transformation, while ADC only accounts for embedding translations and LDC solely focuses on spec-ipe shifts. This observation un- derscores the effectiveness of qual-projection in semantic shift estimation.
Old Classes
(b) New Classes
90
win C1P
wio CIP
CIFAR- 100
Accuracy (%6)
80
70
w/
70
60
60
Table Comparison of DP-NCM with other NCM-based methods.
50-
40
CIFAR- 100
Tiny- ImageNet
10
Methods
T=10
T=20
T=10
T=20
70
w/o CIP
W/lo CIP
AAr Ang
Ar
Awg
Ar
Awg
A
Aun
Tiny-ImagoNer
Accuracy (%)
60-
CIP
40
CIP
ADC
47.65
62.63
35.17 52.16
30.71
41.81
18.63 31.55
50
LDC
47.40 62.39
37.10
53.28
32.90
43.67
23.57
34.08
30
DP-NCM
49.19 63.47
37.64 53.86
33.47
43.86
24.90 35.22
20
10
DP Effectively Estimates Semantic Shift. The NCM clas-
Figure 4. Stability
Stability-plasticeee plasticity analysis with and without CIP.
sifier heavily relies on the backbone's 's representation and
CIP Enhances Both the Stability and Plasticity. To il- lustrate the impact of CIP, present a visualization of task-wise classification accuracy CIFAR-100 with 7=10 for both old (stability) and new classes (plasticity) in Figure 4. Without CIP, the performance on old tasks deteriorates consistently across all tasks and similar pattern of degra- dation can also observed on the performance on new classes. This observation highlights the significant influence of CIP enhancing both stability and plasticity and support the benefit of incorporating category-related information. Furthermore, the results on Tiny-ImageNed align with those on CIFAR-100, demonstrating the robustness of CIP across different datasets.
it can be good performance indicator of semantic shift estimation. As SOTA methods in EFCIL, ADC and LDC es- timate the semantic shift of prototypes and utilize an NCM classifier based on calibrated prototypes. To showcase DP's performance in semantic shift estimation, we follow ADC and LDC to leverage an NCM classifier (denoted as DP- NCM) and compare the results. Since the representation learning process remains consistent among the compared methods, we use exactly the same backbone after learning each task and compare the estimation methods fairly. As demonstrated in Table 5, the dual-projection is effective with the NCM classifier, surpassing state-of tf-th-ar- methods across various validation scenarios. This superiority can be attributed to the richer information encapsulated in the
Reducing Decision Bias via RRCR. In this section, we
8

---

Semantic Shift Estimation via Dual- Dual-Projection and Classifier Reconstruction for Exemplar-Fre Class-Incremental Learning
(a) CIFAR-100
(b) Tiny-ImageNet
with and without DP after the first two tasks on CIFAR-100 using UMAP. As shown in Figure 6, after learning task 2, the decision boundaries are biased to the new classes and become misaligned with the embeddings of class O. With DP, these decision boundaries can be corrected to accom- modate the shifted embeddings. This observation further support DP's effectiveness of semantic shift estimation with RRCR.
60
Accuracy (%)
50
30
40
20
PASS LDC DP-NCM Ours
PASS
LDC DP-NCM Ours
Old Class Ave.
New Class Ave.
Final Ace.
Figure 5. Balance Effect of Classification Reconstruction on Sta- bility and Plasticity.
5. Conclusion
In this paper, we propose the DPCR for EFCIL. To address the semantics shift issue, we proposed the dual projectoon (DP) to estimate the semantic shift with task-wise se- mantic shift projection (TSSP) and a category information projection (CIP). TSSP estimates the shift across tasks and CIP injects sategory-relatee information via row-space pro- jection. To address decision bias, we propose the ridge regression-baset based classifier reconstruction. The classifier is reconstructed in the form of uncentered covariance and pro- totypes of each class and the estimated semantic shift can be compensated into the new classifier without accessing pre- vious data. Extensive experiments have been conducted validate our DPCR and the results demonstrate that, ious datasets, DPCR outperforms offe--of-thearr methods with good balancing of stability and plasticity.
conduct comparison between different classifier training techniques and showease the balancing effect of RRCR. We select PASS, LDC, and implement the DP-NCM proposed in section 4.3 as the compared methods. Among these meth- ods, PASS replays augmented prototypes on the classifier, while the other two methods employ the NCM classifier. In Figure 5, we present the accuracy of old classes, new classes, and the final accuracy after 10-task training on CIFAR-100 and Tmay--m These results shows that the accuracy of new classes in the compared methods signif- icantly surpasses that of old classes, whereas our approach demonstrates a less biased pattern of accuracy between old and new classes. Moreover, our performance outperforms DP-NCM, which only differs in classifier training compared with DPCR. These findings indicate that classifier recon- struction can have superior balancing effect compared to other classifier training techniques, reducing the decision bias. Reconstructing the classifier through ridge regres- sion utilizes accurate information of previous tasks and eliminates the dependency on BP to update the classifier, thereby preventing the direct overwriting of previous de- cision boundaries with only current data. Additionally, in comparison to NCM, our classifier does not solely rely on the representation thus will not be directly affected by the se- mantic shift. With better balance of stability and plasticity, our methods can achieve superior final accuracy.
A Acknowledgment
This research was supported by the National Natural Science Foundation of China (62306117, 624061 14, 62472181), the Guangzhou Basic and Applied Basic Research Foun- dation (2024A04)3611), GJYC program of Guangzhou (2024D03J0005), National Key R & D Project from Minister of Science and Technology (2024YFA1211500, the Fundamental Research Funds for the Central Uni- versities (2024ZYGX7R074 Guangdong Basic and Ap- plied Basic Research Foundation (2024A1515010220, 2025A1515011413 and South China University of Technology-TCI. Technology Innovation Fund.
(a) Boundary without DP
(b) Boundary with DP
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
References
Figure 6. DP's impact decision boundaries viewed after UMAP.
Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., and Tuytelaars, T. Memory aware synapses: Learn- ing what (not) to forget. In Computer Vision ECCV 2018, pp. 144-161, Cham, 2018. Springer International
Visualization of Calibrating Decision Boundaries. To showcase the impact of the projectio in reducing de- cision bias, we visualize the changes in decision boundaries

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Clas-Inneremenaal Learning
Publishing. ISBN 978-3-030-01219-9, doi: 10.1007/ 978-3-030-00229--9 9.
Gomez-Villa, A., Goswami, D., Wang, K., Bagdanov, A. D., Twardowski, B., and van de Weijer, J. Exemplar-free con- tinual representation learning via learnable drift compen- sation. In Computer Vision - ECCV 2024, pp. 473-490 Cham, 2024. Springer Nature Switzerland. ISBN 978-3- 031-72667-5. doi: 10. N0.1007/978-3-000 1111007/778-3-001-77666-----
Belouadah, E.,, Popescu, A., and Kanellos, I. A comprehen- sive study of class incremental learning algorithms for visual tasks. Neural Networks, 135:38-52 15:33-54,225 2021.
Bian, A., Li, W., Yuan, H., Wang, M., Zhao, Z., Lu, A., Ji, P., Feng, T., et al. Make continual learning stronger via c-flat. Advances in Neural Information Processing Systems, 2024.
Goswami, D., Liu, Y., Twardowski, B. and van de Weijer, J. FeCAM: Exploiting the heterogeneity of class distribu- tions in exemplar-free continual learning, In Advances in Neural Information Processing Systems, volume 36, pp. Curran Associates, Inc., 2023.
Dhar, P., Singh, R. V., Peng, K.-C., Wu, Z., and Chellappa, R. Learning without memorizing. In Proceedings of the IEEF/CVF Conference Computer Vision and Pallern Recognition (CVPR), pp. 5133-5141 June 2019. doi: 10.1109/CVPR22201.00022
Goswami, D., Soutif- Commif-C A., Liu, Y., Kamath, S., Twardowski, B., and Van De Weijer, J. Resurrecting old classes with new data for exemplar-free continual learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 08525-285344 June 2024. doi: 10. N0.1109/CVPR52733. 2024.02695.
Douillard, A., Cord, M., Ollion, C., Robert, T., and Valle, E. PODNet: Pooled outputs distillation for small-tasks incremental learning. In Computer Vision ECCV 2020, pp. 86-102, Cham, 2020. Springer International Publishing. ISBN 778-3-030-58565-3 doi: 10.1007/ 978-3-033--88565-566
Guo, and Lyu, R. Pseudoinverse learning algorithm for feedforward neural networks. Advances in Neural Networks and Applications, 1:321-326, January 2001.
Fang, D., Zhu, Y., Lin, Z., Chen, C., Zeng. Z., and Zhuang, H. AIR: Analytic imbalance rectifier for continual learn- ing, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Deep resid- ual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, June 2016. doi: 10.1 10.100/CVP.22006.990
Feng, T., Wang, M., and Yuan, H. Overcoming catastrophic forgetting in incremental object detection via elastic re- sponse distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
He, R., Zhuang, H., Fang, D., Chen, Y., Tong, K., and Chen, C. REAL: Representation enhanced analytic learning for exemplar-fres class-incremental learning, 2024a.
Gao, Z., Xu, K., Zhuang, H., Liu, L., Mao, X., Ding, B., Feng, D., and Wang. Less confidence, less forgetting: Learning with humbler teacher in exemplar-free class- incremental learning. Neural Networks, 179:106513, 2024a. ISSN 0893-6080. doi: 10.10166j.neunet22222 106513.
He, Y., Chen, Y., Jin, Y., Dong, S., Wei, X., and Gong, Y. DYSON: Dynamic feature space self self-organization for online task-free class incremental learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattrrn Recognition (CVPR), pp. 23741-23751, June 2024b. doi: 10.1 D0.1009/CCPPP332333302222222222
Gao, Z., Zhang, X., Xu, K., Mao, X., and Wang, H. Stabi- lizing zero-shot prediction: A novel antidote to forgetting in continual vision-language tasks. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomezak, J.. and Zhang, C. (eds.), Advances in Neural Informa- fion Pro Processing Systems, volume 37, pp. 128462-128488. Curran Associates, Inc., 2024b.
Hou, S., Pan, X., Loy, C. C., Wang. Z., and Lin, Learn- ing unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 831-839, June 2019, doi: 10.
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Des- jardins, G., Rusu, A. A., Milan. K., Quan, J., Ramalho, T.,
Gao, Z., Han, S., Zhang, X., Xu, K., Zhou, D., Mao, X., Dou, Y., and Wang, H. Maintaining faimesss in logit-based knowledge distillation for class-incremental learning. Proceedings of the AAA/ Conference on Artifi- cial Intelligence, 39(16);16733-1677) Apr. 2025. doi: 10. v. 33916.33382..
A., Hassabis, D., Clopath, C., Kumaran, D., and Hadsell, R. Overcoming catastrophic forgetting in neural networks. Proceedings of the Na- tional Academy of Sciences, 114(13):3521-3522 2017. doi: 10. 10.3073/ 1ataaas...11111 1611835114.
10

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-nneremental Learning
Krizhevsky, A., Nair, V., and Hinton, G. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009, URL https:: /wwww cs. toronto. edu/~ / kriz/ far. far.mm.l.
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H.
iCaRL: Incremental classifier and representation learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 5533-5542, July 2017. doi: 10. (0.1109/CVPR.20177
Li, D. and Zeng, Z. CRNe:: fast continual learning framework with random theory. IEEE Transactions Puttern Analysis and Machine Intelligence, 45(9):10731- 10744, September 2023, ISSN 1939-3539. TPAMI.20233222885.
Russakovsky, O., Deng, J., Su, H.,, Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. ImageNet large scale visual recognition chal- lenge. International Journal of Computer Vision, 115(3):
doi: 10.1109/
211-252, 2015.
Li, Q., Peng, Y., and Zhou, J. FCS: Feature calibration and separation for non-exemplar class incremental learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pullern Recognition (CVPR), pp- 28495-285044 June 2024. 2024.02692.
Rypešć, G., Cygert, S., Trzcinski, T., and Twardowski, B. Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning. In The Thirty- eighth Annual Conference on Neural Information Pro- cessing Systems, 2024.
doi: 10.1109/CVVR52733...
Li, Z. and Hoiem, D. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 40(12):2933 2947,, December 2017. ISSN 1939- 3539. doi: 10.1109/PAM..221..77730
Shi, W. and Ye, M. Prototype reminiscence and aug- mented asymmetric knowledge aggregation for non- exemplar
learning. In Proceedings of the IEEE/CVF International Conference on Com- puter Vision (ICCV), pp. 1772-1781, October 2023. doi: 10.1 10.10099C5511000222...00000
Magistri, S., Trinci, T., Soutif, A., van de Weijer, J., and Bagdanov, A. D. Elastic feature consolidation for cold start exemplar-fin free incremental learning. In The Twelfth International Conference on Learning Representations, 2024.
Toldo, M. and Ozay, M. Bring evanescent representations life in lifelong class incremental learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16732-16741, June 2022. doi: 10.1 D0.100//CVPS222682222.00001
Malepathirana, T., Senanayake, D., and Halgamuge, S. NAPA-VQ: Neighborhood aware prototype augmenta- tion with vector quantization for continual learning. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 11674-11684 October 2023. doi: 10. 10.11091CCCV1110022223..00
Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The Caltech-UCSD birds-200-2011 dataset. (CNS- TR-201 1-001), 2011.
Wang. F.Y.,, Zhou, D.-W., Ye, H.J.,, and Zhan, D.-C. FOSTER: Feature boosting and compression for class- incremental learning. In Computer Vision ECCV 2022, pp. 398-414, Cham, 2022. Springer Nature Switzerland. ISBN 978-3-031-19806-9 doi: 10.1007/ 978-3-031-19906-99
Masana, M., Liu, X., Twardowski, B., Menta, M., Bag- danov, A. D., and van de Weijer, Class-incremental learning: Survey and performance evaluation image classification. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(5):5513-5533 May 2023. doi: 10.1099/PAAM..2222.211111
Wang. S., Shi, W., He, Y., Yu, Y., and Gong, Y. Non- exemplar class-incremental learning via adaptive old class reconstruction. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, pp. 4524 4534, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701085. doi: 10. 14833581783.361111
Mermillod, M., Bugaiska, A., and BONIN, P. The stability- plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects. Frontiers in Psychology, 4:504, August 2013. ISSN 1664- 1078. doi: 23388//fssyg.2113..00
Petit, G., Popeseu, A., Schindler, H., Picard, D., and Dele- zoide, B. FeTrIL:: Feature translation for exemplar- free class-incremental learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision (WACV), pp. 3911-3920, January 2023. doi: 10.1 N..1109/WCVVV 56688.202222 200.8.0000.000900
Xu, Y., Chen, Y., Nie, J., Wang, Y., Zhuang, H., and Oku- mura, M. Advancing cross-domain discriminability in continual learning of lasiunnl models. In The eighth- Annual Conference on Neural Information Processing Systems, 2024.

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning
Yu, L.,, Twardowski, B., Liu, X., Herranz, L.,, Wang, K., Cheng, Y., Jui, S., and Weijer, J. d. Semantic drift compensation for class-incremental learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition ( (CVPR), June 2020. doi: 10.1109/CVP2422000200000
on Computer Vision and Pattern Recognition (CVPR), pp. 5871-5880, June 2021b. doi: 10.11090CVPR444773.011 00581.
Zhu, K., Zhai, W., Cao, Y., Luo, J., and Zha, Z.-J. Self- sustaining representation expansion for non-exemplar class-incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 9296-9305, June 2022. doi: 10. 111119CCVPR2268222222222.
Yue, X., Zhang, X., Chen, Y., Zhang, C.,, Lao, M., Zhuang. H., Qian, X., and Li, H. MMAL: Multi-modal ana- lytic learning for exemplar-free audio-visual class in- cremental tasks. In Proceedings of the 32nd ACM In- teraaiional Conference on Multimedia, MM '24, pp- 2428-2433 New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400706868. doi: 10.1 N0.1145/36644473666 1607.
Zhuang, H., Lin, Z., and Toh, K.-A Correlation projection for analytic learning of classification network. Neural Processing Letters, 53(6):3893-3311 Dec 2021. ISSN 1573-773X. doi: 10./0/ 1063-021-10570-2.
Zhuang, H., Lin, Z., and Toh, K.-A. Blockwise recursive moore peeeesperreeeeee inverse for network learning. IEEE Trans- actions on Systems, Man, and Cybernetics: Systems, 52 (5):3237-3250, May 2022a. doi: 10.1109/TSMC:202.. 3064241.
Zeng. Z., Wang. J., Wu, L., Lu, W., and Zhuang, H. 3D- AOCL: Analytic online continual learning for imbalanced 3d point cloud classification. Alexandria Engineering Journal,
0000005390000 2025. ISSN 1110-0168. doi: 20.1016/j.aej.2000 w0...ae..2222100 10.037.
Zhuang. H., Weng, Z., Wei, H., Xie, R., Toh, K.-A,, and Lin, Z. ACII:: Analytic class-incremental learning with abso- lute memorization and privacy protection. In Advances in Neural Information Processing Systems, volume 35, pp. 11602-11614. Curran Associates, Inc., 2022b.
Zenke, F., Poole, B., and Ganguli, S. Continual learn- ing through synaptic intelligence. In Proceedings of the 34th International Conference on Machine Learning, vol-
ume 70 of Proceedings of Machine Learning Research, pp. 3987-3995. PMLR, August 2017.
Zhuang, H., Weng, Z. He, R., Lin, Z., and Zeng, Z. GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7746-7755, June 2023. doi:
Zhao. X., Wang, H., Huang, W., and Lin, W. A statisti- cal theory of regularization-bast based continual learning. In Forty-first International Conference on Machine Learn- ing, 2024.
Zheng, B., Zhou, D.-W., Ye, H.-J., and Zhan, D.-C. Multi-layer rehearsal feature augmentation for class- incremental learning. In Forty-first International Confer- ence on Machine Learning, 2024.
10.1 20.1009/CVPR5522222222232
Zhuang, H., Chen, Y., Fang, D.,, He, R., Tong, K., Wei, H., Zeng, Z., and Chen, C. GACL: Exemplar free generalized analytic continual learning. The Thirty-eighuh Annual Conference on Neural Information Processing Systems, 2024a.
Zhou, D.-W., Wang, F-Y., Ye, H.-J,, and Zhan, D.-C. PyCIL: python toolbox for dlas--ncremmnnal learning. SCI- ENCE CHINA Information Sciences, 66(9):197101, 2023.
Zhuang, H., He, R., Tong, K., Zeng, Z., Chen, C., and Lin, Z. DS-AL: A dual-stream analytic learning for exemplar- free class-incremental learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15): 17244, March 2024b. doi: 10.1669/aaiii...... w38.:
doi: 10. 10111000/ 14221112-22-222000
Zhou, D.-W., Wang, Q.- Qi, Z.H., Ye, H.JJ., Zhan, D.-C., and Liu, Z. class-incremental learning: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 46(12):955199873 2024. doi: 10.1 1L1109/TPAMM.22244422222.
Zhuang, H., Liu, Y., He, R., Tong, K., Zeng, Z., Chen, C., Wang. Y., and Chau, L.-P. F-OAL: Forward-onle online analytic learning with fast training and low memory foot- print in class incremental learning, In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024c.
Zhu, F. Cheng, Z. Zhang, X.-y., and Liu, C.-l. Class- incremental learning via dual augmentation. In Advances in Neural Information Processing Systems, volume 34, Curran Associates, Inc., 2021a.
pp. 14306-14311
Zhu, F., Zhang, X.-Y,, Wang, C., Yin, F., and Liu, C.-L. Pro- totype augmentation and self-supervision for incremental learning. In Proceedings of the JEEE/CVF Conference
12

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Claseenncremental Learning
A. Pseudo-Code of Proposed DPCR
The pseudo-code of the learning agenda of DPCR task is proposed Algorithm 1.
Algorithm Learning Agenda of DPCR in Task
Input: D., X, O, integrated information set of task t - 1 { Mice -
h--1 0--1 1
Outputtintegrated information set of task { &
1i:1 new backbone Oz, and classifier Wi-
Incremental Representation Learning for number of epoch do
Update O, by minimizing Crep in Eq. (1).
end for
// Shift Estimation via Dual Projection:
Calculate x-- and X with Dr, 0--1 and Or. Obtain task-wise projection pi- by (5). for i in (i = 1, t 1)do
for in C; do
Obtain Uh, by Eq. (6) with $ Pe
Obtain P/- by Eq. (7) with U
// Rectification:
Obtain Sh and fhe by Eq. (16) and Eq. (17)
1.c
Obtain Tr. via Eq. (18).
be
end for
Obtain Th E.G
|C,1 &% Ve H H.
end for // Ridge Regression-based Classifier Reconstruction for e in C, do
Obtain Th: + x x uie + 1 0.T
No -===1 x t,ed'
Obtain Hº:-- Notlle yt.c.
end for
Calculate D. Deed, t, H. + Eisd, H
te
Update { 0 8: ) +- ($.. d ICEC. f,c Pi U ($0.
t. l
cEC.
1.c-
Obtain the classifier weight W, by Eq. (19).
Obtain the classifier weight W/ with CN by Eq. (22).
B. Implementation Detail
We mainly follow the implementation details of hyperparameters in ADC (Goswami et al., 2024) for the backbone training. For all the compared methods, we the experiments three times with non-shuffled class order and two shuffled class orders generated by random seeds of 1992 and 1993. The detailed experimental settings are listed below.
Data Augmentation. As implemented in PyCIL (Zhou et al., 2023), for CIFAR-100, we use the augmentation policy which consists of random transformations including contrast or brightness changes. For the other datasets, we use the default setting of augmentations which include random crop and random horizontal flip. We use the same set augmentations for all the methods.
LwF (Li & Hoiem, 2017). For LwF, in the first task, we use starting learning rate of 0.1, momentum of 0.9, batch size of 128, weight decay of 5 and trained for 200 epochs, with the learning rate reduced by factor of 10 after 60, and 160 epochs, on all benchmark datasets. For subsequent tasks, we use an initial learning rate of 0.05 for CIFAR- 100 and ImageNet-100 and 0.001 for Imageee.e The learning rate is reduced by a factor of 10 after 45 and 90 epochs and the model is trained for a total of 100 epochs. We set the the temperature to 2 and the regularization strength to 10 for CIFAR- 100 and Tiny- ImageNet and 5 for ImageNet-100 100. For the backbone training in SDC, ADC, LDC, ACIL, DS-AL, and our DPCR, we use the same setting LwF.
13

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Frec Class-nneremental Learning
SDC (Yu al., 2020). For SDC, the 0.3, 0.3, 1.0 for the Gaussian kernel CIFAR-I00, Tiny-ImageNer, and ImageNet-100 respectively.
PASS (Zhu et al., 2021b). For PASS, we tune Aproto from {0.1, 0.5, 1.0, 10) use Aprit 0.1, 10 for CIFAR-100, Tiny-ImageNet, and ImageNet-100 respectively. We follow the implementation in PyCIL (Zhou et al., 2023) to use
Afkd = 10.
ADC (Goswami al., 2024). For ADC, we follow the implementation ADC paper to use a = 25, i = 3, and m 100 on all the benchmark datasets.
LDC (Gomez- Voolle-Villaaaao et al., 2024). For LDC, we follow the official implementation to Adam optimizer learn linear layer with a learning rate of 0.001 and epoch of 20.
ACIL (Zhuang et al., 2022b). For ACIL, we use the buffer size of 10000 for all the benchmark datasets.
FeCAM (Goswami et al., 2023). For backbone training of FeCAM, we use the official implementation of 200 training epochs with initial learning rate of 0.1 and batch size of 128 in the first task. Then the backbone frozen. For the hyperparameters in FeCAM, we set 8 = 0.1, 0 1, 022
DS-AL (Zhuang et al., 2024b). For DS-AL, compensation ratio C' from (0.5, 1.0, 1.5, 2.0). We use C = 1.0, 1.5, 1.0 for CIFAR-100, Imny-lmageNet, and ImageNet- 100 respectively.
the same setting of buffer size of 10000 ACIL and tune the
DPCR (ours). For DPCR, we use the = 200, 2000 and 2000 for CIFAR-100 100, Iiny-lmageNet, and ImageNet-100 respectively and the is set be 1e-9 for all experiments.
C. Comparison with EBCIL Methods
In this section, we provide the results for comparing with EBCIL methods, including iCaRL (Rebuffi et al., 2017), PODNet (Douillard et al., 2020) and FOSTER (Wang et al., 2022). The experiments are conducted on CIFAR- 100 and ImageNet- 100 with the memory size M = 500, 1000 for storing exemplars. Since these methods do not offer official implementation on Tmne-Ima we skip this dataset for correctness. The experiments are conducted with PyCIL repository (Zhou et al., 2023) and we follow the official implementation of hyperparameters with the same seed of 1993.
As shown in Table. 6, even comparing with the EBCIL methods, DPCR can have competitive performance. When the memory size is 500, all the EBCIL methods perform poorly and our DPCR outperforms them with considerable gap. With the increased memory size of 1000, the advantage of replaying exemplars begins to emerge. However, our DPCR can still achieve the second best or even best results with T=10. Since EFCIL could experience more intensive forgetting without the aid of exemplars (He et al., 2024a), these results showcases our DPCR's capability of resisting forgetting.
Table Comparison with EBCIL methods. Results in bold are the best within the compared methods and data underlined are the second best under the same setting of T.
CIFAR-100
ImageNet- 100
Memory Size
Methods
T=10
T=20
T=10
T=20
A
Aayg
A
Aavg
A
Aavg
A
Aavg
iCaRL-CNN (Rebuffi al., 2017)
500
22.28
45.91
20.12
41.59
22.08
44.86
14.64
35.40
iCaRL-NCM (Rebuffi et al., 2017)
500
39.56
58.16
33.67
52.39
36.14
57.47
28.44
48.71
PODNet (Douillard al., 2020)
500
31.90
51.17
23.44
42.42
38.10
57.10
25.70
43.56
FOSTER (Wang et al., 2022)
500
37.15
49.18
30.85
40.06
41.52
60.35
36.40
38.86
iCaRL-CNN (Rebuffi al., 2017)
1000
32.33
53.15
28.45
49.92
31.78
52.35
22.56
43.47
iCaRL-NCM (Rebuffi et al., 2017)
1000
46.78
62.62
40.71
57.47
44.46
62.60
35.68
54.24
PODNet (Douillard al., 2020)
1000
35.57
55,00
28.71
49.09
42.26
60.72
31.20
49.31
FOSTER (Wang et al., 2022)
1000
40.42
53.15
41.79
55,41
54.52
66.51
47.86
61.93
DPCR (ours)
49.59
62.13
40.72
54.91
53.24
68.18
40.82
57.94
14

---

Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Frec Class-Incremental Learning
D. Extended Experiments of CN
CN Is Beyond the Regularization. In this section, we provide the study on CN and regularization factor 7, The regularization of ridge regression important and affects the classification performance (Zhuang et al., 2024a). Also, since the normalization can be used restraint the value of weights then regarded as regularization, here we conduct joint study of y by and the CN module. As shown in Figure 7, without CN, the performance is benefited by adjusting the y, The performance first increase then decrease with increasing 7. The results are consistent with those in (Zhuang et al., 2024a), where appropriate regularization addresses over-fitting then enhances performance while large y could lead to over regularized classifier. However, we can find that, the effect of introducing CN is beyond the regularization. The results with CN are overall good regardless of the degree of regularization (i.e., value of 7). This demonstrates that the CN not just the regularization, brings numerical stability for DPCR and could enhance the performance. With CN, from the Figure ? 200, 2000 and 2000 can achieve overall good results.
(a) CIFAR-100
(b)Tiny-mmageNet
(e)
Anng (%)
20
20M
20
2002
28
208
40
50
(%)))
40
30
30
10
100
200
500
18
22
56
20A
10
100
280
500
24
24
100
10 100 200
500
FA
24
52
708
without CN
with CN
Figure 7. The effect of CN with different regularization factor 7 three datasets.
15


Reviews:
[{"review_id": "ydw3eJvkbw", "reviewer": "Reviewer_spyP", "summary": "The paper studies exemplar-free class-incremental learning and focus on addressing two major problems of semantic shift and decision bias. The authors propose to use dual-projection to estimate semantic shift which includes both class-wise and task-wise shifts, and a ridge regression based classifier instead of a learnable classifier to address the problem of decision bias. The proposed method achieves better stability-plasticity trade-off among existing methods in EFCIL benchmarks.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer spyP\nThank you for your constructive feedback. We provide detailed responses to your concerns below. The discussion and experiments will be properly included. Also, we will check and revise all the writing mistakes, and include references of ridge regression.\n### Q1: Comparison between RRCR and NCM/Mahalanobis-distance classifier in FeCAM.\nResponse:  RRCR is a parametrized classifier where the decision boundaies can be directly modified by training the parameters. NCM and Mahalanobis-distance classifiers are non-parametric classifiers that utilizes feature distribution and absence of trainable parameters in limits their adaptability across task. Although they can avoid decision bias and achieve better performance over existing parametrized classifiers, RRCR mitigates this via reconstruction then keep the benefit as a parametrized approach. To validate this, we compare RRCR with NCM and Mahalanobis on CIFAR-100 (T=10) using the same frozen backbone after task 1 (following FeCAM). RRCR surpasses both. \n\n|CIFAR-100(T=10)|A_last(%)|A_avg(%)|\n|-|-|-|\n|FeCAM|30.41|45.59|\n|NCM|18.77|36.16|\n|**RRCR**|**33.28**|**49.16**|\n\nRRCR also better adapts to semantic shift during CIL via reconstruction. FeCAM freezes the backbone to maintain consistent covariances, but this impedes plasticity. Once the backbone evolves, prior class covariances become outdated and hard to update without storing previous data due to centralization to obtain covariance. RRCR, by contrast, recalibrates using ${\\hat{{\\Phi}}{c}^{\\theta{t}}, \\hat{{\\mu}}{c}^{\\theta{t}}}$ via matrix operations, avoiding the need to retain prior data or embeddings.\n### Q2: Motivation of RRCR.\nResponse: In this paper, we propose RRCR to address decision bias in parametrized classifier during CIL. Non-parametric classifiers like NCM lack lack adaptability due to their static structure and inability to directly modify decision boudaries. While techniques like ADC/LDC try to offset semantic shift, they remain limited by non-learnable classifiers. Existing parametrized classifiers can directly learn from labels but tend to become biased without old data. RRCR fills this gap, offering the benefits of a learnable model while correcting bias via reconstruction. We will clarify this motivation in lines 066–069.\n### Q3: Evaluation on fine-grained datasets.\nResponse: As suggested, we include the results on fine-grained dataset CUB-200 with T=5 in cold-start setting with the same seed. Results shows that DPCR also leads the compared methods.\n|CUB200(T=5)|A_last(%)|A_avg(%)|\n|-|-|-|\n|LwF|25.40|36.38|\n|ACIL|21.14|33.14|\n|DSAL|21.28|32.36|\n|SDC|24.21|36.00|\n|LDC|28.70|39.09|\n|ADC|28.84|39.44|\n|**DPCR**|**29.51**|**40.62**|\n### Q4: Covariance vs. Gram Matrix.\nResponse: We will replace \"uncentered covariance\" to \"gram matrix\".\n### Q5: Compare with FeCAM. \nResponse: We compare DPCR with FeCAM and obtain the results of FeCAM via the official implementation under the same seed of 1993. Our DPCR outperforms FeCAM and this can be attributed to that FeCAM needs to freeze backbone thus limits the plasticity. Our DPCR can take advantage of incremental representation learning with semantic shift estimation then a good stability-plasticity balance.\n|CIFAR-100|T=10||T=20||\n|-|-|-|-|-|\n|(%)|A_last|A_avg|A_last|A_avg|\n|FeCAM|34.82|49.14|25.77|41.21|\n|**DPCR**|**50.08**|**63.43**|**41.62**|**56.01**|\n|Tiny-ImageNet|T=10||T=20||\n|(%)|A_last|A_avg|A_last|A_avg|\n|FeCAM|29.83|42.19|22.69|34.48|\n|**DPCR**|**35.50**|**47.73**|**26.67**|**38.80**|\n|ImageNet-100|T=10||T=20||\n|(%)|A_last|A_avg|A_last|A_avg|\n|FeCAM|41.92|58.21|28.64|43.04|\n|**DPCR**|**53.46**|**68.20**|**40.76**|**57.81**|\n### Q6: Acknowledge LDC and add discussion of LDC, ADC, and FeTrIL.\nResponse: We will include the discussion as follow. To estimate the semantic shift, ADC uses adversarial samples to estimate the translation of prototypes, and LDC introduces a learnale projection. However, ADC only considers the shift of translation, neglecting other transformation including scaling, and LDC only estimates the shift across tasks without class-specific information. Moreover, LDC needs iterative BP-training to get the projector, inducing more computation cost. Inspired by the learnable projector in LDC, DPCR achieves shift estimation via DP with low-cost closed-form solution and class-specific information. FeTrIL uses prototype-based pseudo-features to rebalance classifier training and employs a LinearSVC. However, it freezes the backbone, reducing plasticity. Also, LinearSVC's effect in CIL is not thoroughly studied. \n### Q7: Decision bias vs. Task-recency bias?\nResponse: Task-recency bias refers to the model's tendency to favor new tasks. Several works attribute it to the bias in classifier. We use “decision bias” to clarify that the bias originates in the classifier. While similar in outcome, we prefer the more specific term **decision bias** for better understanding and will clarify this in the text.", "claims_and_evidence": "Most claims are validated.\nSome claims need better explanation and validation:\n1. Why using Ridge Regression-based Classifier Reconstruction is better than other methods like NCM or FeCAM [X] which also employs covariance based classification? What is the motivation for introducing this RR based classifier? Does RRCR improve over Mahalanobis-distance based classification as done in [X] for EFCIL?\n2. Line 066-069: “However, its effectiveness heavily depends on the quality of the learned representations, which are susceptible to degradation caused by semantic shift.” This motivation is neither clear nor justifies introducing a new type of classifier. Existing methods like LDC, ADC exactly solve the semantic shift problem. I would ask the authors to justify why it is required to introduce a Ridge Regression-based Classifier in the context of EFCIL and what are the benefits of using this.\n\n[X] FeCAM: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. In Advances in Neural Information Processing Systems, 2023.", "methods_and_evaluation_criteria": "The evaluation criteria is appropriate for the EFCIL setting. The compared methods are very relevant and recent.\nAdditionally, I would propose to include evaluation on some fine-grained classification datasets similar to what is done in ADC [CVPR 24] for a more rigorous experimentation.", "theoretical_claims": "I do not find any errors in the theoretical formulation.\nMinor suggestion: It is important to mention that the covariance matrix referred to in equation 15 is not centered on the mean and also not normalized. This can be referred to as gram matrix as done in [Y].\n\n[Y] Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems. 2023", "experimental_designs_or_analyses": "The experimental setup is sound and valid, and following the standard practice in EFCIL. The analysis and ablation studies are extensive.", "relation_to_broader_scientific_literature": "The contributions of this paper aims to address problems of semantic shift and task-recency bias discussed in several existing CL works.", "essential_references_not_discussed": "1. Most essential references are discussed. \n2. Some discussion on FeTrIL [WACV 23] would be useful in the context of the paper since they also propose an alternative SVM-based classifier instead of training linear classifier. Comparison with a recent work FeCAM [NeurIPS 23] could be added which also uses covariance of features for classification.\n3. It is important to acknowledge that learning a projector for estimating semantic shift in prototype means was also proposed by LDC (ECCV 24). While the proposed method builds on top of this concept, the authors do not acknowledge this in the paper.\n4. While the paper focus largely on semantic shift, it does not have much discussion on existing drift compensation or resistant methods like LDC, ADC. Adding some discussion on this to better understand existing methods solving the same problem would add more clarity. Why these methods are not good enough or how is the proposed method addressing their limitations? This would improve the motivation of the proposed method.\n5. References to ridge regression theory is missing.", "other_strengths_and_weaknesses": "Strengths\nThe paper is well written and organized, with extensive experiments and ablations, good illustrations.\n\nWeaknesses:\nDiscussed above. My main concern is that the motivation for using Ridge Regression-based Classifier Reconstruction is not convincing. More discussions on \"why use RRCR\" is important.\nSome more comparisons, discussion and validation as mentioned above will improve the paper.\n\nI am willing to increase my rating if the authors address my major concerns.", "other_comments_or_suggestions": "Writing mistakes:\n1. Line 189 (right side) - “Then we can constructed the projector of CIP as”.", "questions_for_authors": "1. How is decision bias different from task-recency bias which is discussed in many CL works? It is better to stick to existing terms instead of defining the same thing with a different term.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "szFPKUfs2C", "reviewer": "Reviewer_YwAB", "summary": "This paper proposes a method called Dual-Projection Shift Estimation and Classifier Reconstruction (DPCR) to solve two key challenges in Exemplar-Free Class-Incremental Learning (EFCIL): semantic shift and decision bias. It reconstructs the classifier through the dual projection mechanism and ridge regression, effectively balancing the learning of new and old knowledge. Experimental results show that DPCR outperforms the existing EFCIL method on multiple benchmark datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer YwAB\n\nThank you for your constructive and detailed feedback. We provide detailed responses to your concerns below. The discussion and experiments below will be properly included in the manuscript.\n\n\n### Q1: Lack of evaluation on lager datasets like ImgaNet1000.\nResponse: As suggested, we validate DPCR and compared methods on ImageNet1000 with T=10. We use the same backbone of ResNet-18 and same under the same seed of 1993 to conduct the experiments. The hypeparameters are adopted the same as those in ImageNet-100 experiments. The results are reported below and our DPCR is still competitive on ImageNet1000. These results will be included in the manuscript. \n\n|ImageNet-1k(T=10)|A_last|A_avg|\n|-|-|-|\n|LwF|22.01|42.40|\n|ACIL|32.28|46.61|\n|DSAL|33.67|48.84|\n|LDC|35.15|53.88|\n|ADC|31.34|50.95|\n|**DPCR**|**35.49**|**54.22**|\n\n### Q2: Lack of complexity analysis\nResponse: As suggested, we will include the complexity analysis. Since the incremental representation learning is common in the CIL realm, here we analyze the complexity of DP and RRCR. The complexity analysis will be included as follow. \n\nSuppose $F$ is the FLOPs of the model ($F \\approx 1.8\\times10^{9}$ for ResNet-18),  the time complexity of DP $\\mathcal{O}(FN_t + N_td^2 + tC(d^3 + C^2d))$ is the sum of those of the feature extraction $\\mathcal{O}(FN_t)$, calculation of the task-wise projection matrix $\\mathcal{O}(N_td^2+d^3)$, and the rectification of each class $\\mathcal{O}(d^3+C^2d)$. Similarly, the time complexity of RRCR is $\\mathcal{O}(d^2N_c + dtC + td^2+d^3)$.\n\n### Q3: Some figures lack of horizontal labels.\nResponse: Thank you for pointing out this. We will check and include the horizontal labels for all the figures.", "claims_and_evidence": "This paper claims to address semantic shift via proposed DP, and decision bias via RRCR. The claims are supported by the experiments.", "methods_and_evaluation_criteria": "The methods are illustrated clearly. The evaluation criteria is common. A slight drawback is the lack of the evaluation on large datasets like ImageNet1000. Besides, this paper lacks evaluation on the complexity of the method.", "theoretical_claims": "There are no new theoretical results.", "experimental_designs_or_analyses": "The experiment designs and analyses are adequate. But there are figures lacking of horizontal labels (Figure 4).", "relation_to_broader_scientific_literature": "No", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "No", "other_comments_or_suggestions": "Overall, this paper proposes a novel method in NECIL, and seems achieving promising results with reasonable approaches. Although with some drawbacks, the paper seems good.", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jNngNTifjk", "reviewer": "Reviewer_V6Ai", "summary": "This paper focuses on Exemplar-Free Class-Incremental Learning (EFCIL), which aims to solve the problem of catastrophic forgetting without retaining any exemplars. Existing methods alleviate forgetting by storing distributional information about past tasks. However, as models are updated, past distribution information becomes outdated due to semantic shifts, and thus cannot properly represent past classes. This paper proposes a Dual-Projection Shift Estimation method to estimate and calibrate this shift. In addition, this paper uses ridge regression for offset estimation to reformulate classifier training as a reconstruction process, thereby alleviating the classifier's bias towards new categories.The proposed method achieved outstanding performance on multiple benchmark datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Response to Reviewer V6Ai\n\nThank you for your constructive and detailed feedback. We provide detailed responses to your concerns below. The discussion and experiments below will be properly included in the manuscript.\n\n### Q1: Elaborate the advantages of proposed methods over existing works [R1,R2,R3].\n\nResponse: The limitations of existing works [R1,R2,R3] can be summarized as follow respectively. 1. Toldo et al [R1] propose to train multi-layer perceptrons or variational autoencoders to estimate the semantic shift, intensifying the training cost.  EFC [R2] utilizes the Gaussian kernels to capture the shift by estimating the translation of feature means, potentially neglecting other types of shift transformations including scaling. PRL [R3] proposes a feature projection to align the current embedding space to a pre-defined latent space, rather than estimating the semantic shift. Also, PRL does not consider different classes within a training task. Our DPCR provides a DP to estimate the semantic shift in both efficient and effective way via a linear projection can be calculated directly without iterative BP-training. Also, the CIP in DP includes class-specific knowledge. To demonstrate the effectiveness of DPCR, we further include the experiments compare with EFC and PRL (we did not find the released code of [R1]). The results show the high performance of our DPCR. \n \n|CIFAR-100|T=10|| T=20||\n|-|-|-|-|-|\n|(%)|A_last|A_avg|A_last|A_avg|\n|EFC|45.01|59.44|34.12|48.63|\n|PRL|45.32|57.08|28.43|41.98|\n|**DPCR**|**49.59**|**62.13**|**37.79**|**54.48**|\n\n### Q2: Does CIP need to learn and store a linear projector for each class at each subsequent incremental phase?\nResponse: CIP is a training-free projection to include the class-specific knowledge and it does not need to be stored. Actually, during the training of DPCR, CIP is calculated from the uncentered covariance via SVD (see Eq. (6) and (7)) and it will be used to calibrate the old information set with TSSP and then discarded. Only the covariance and prototype  ${\\hat{{\\Phi}}{c}^{\\theta{t}}, \\hat{{\\mu}}{c}^{\\theta{t}}}$ need to be stored in DPCR. For the operations in Eq. (8) and Eq (9), they are used to demonstrate how the old features of far previous task are calibrated of without loss of generality and this process does not be necessarily performed in the implementation of DPCR. In our implementation, the calibration is conducted across two adjacent tasks (see Algorithm 1). We will further clarify this to avoid potential misunderstanding.\n\n### Q3: Using SVD seems to result in information loss.\n\nResponse: CIP is used to inject class-specific information via projecting the projector in TSSP to significant directions. The construction of projector for CIP maintains all the singular vectors corresponding to non-zero singular values and only discards the information of null space, which we believe not necessarily needed. To validate the effect of information of null space, we perform an experiment on CIFAR-100 with all the singular vectors to construct the projector of CIP (denoted as DPCR-wn). The results are shown below and we find that including null space information does not affect the performance and thus the information loss may not have large impact on the performance. \n \n|CIFAR-100|T=10|| T=20||\n|-|-|-|-|-|\n|(%)|A_last|A_avg|A_last|A_avg|\n|DPCR-wn|49.59|62.14|38.04|54.25|\n|DPCR|49.59|62.14|38.04|54.25|\n\n### Q4: Lack of details of training.\n\nResponse: As indicated in line 310 (left), the implementation details are provided in Appendix B. We will move them to the manuscript to avoid possible omission. \n\n### Q5: Experiments on larger datasets are beneficial.\n\nResponse: As suggested, we conduct the validation on ImageNet-1000 with T=10 using ResNet-18 under the same seed of 1993. The hypeparameters are the same as those in ImageNet-100 experiments. The results shows our DPCR is still competitive on ImageNet1000. \n\n|ImageNet-1k(T=10)|A_last|A_avg|\n|-|-|-|\n|LwF|22.01|42.40|\n|ACIL|32.28|46.61|\n|DSAL|33.67|48.84|\n|LDC|35.15|53.88|\n|ADC|31.34|50.95|\n|**DPCR**|**35.49**|**54.22**|\n\n[R1] Bring evanescent representations to life in lifelong class incremental learning. CVPR 2022. \n\n[R2] Elastic feature consolidation for cold start exemplar-free incremental learning. ICLR 2024 \n\n[R3] Prospective Representation Learning for Non-Exemplar Class-Incremental Learning. NeurIPS 2024.", "claims_and_evidence": "The paper illustrates the claims made with figures and supports them by citing previous work. The paper provides experiments that demonstrate that the proposed classifier reconstruction can strike a better balance between plasticity and stability.", "methods_and_evaluation_criteria": "The proposed method is meaningful for the domain of Exemplar-Free Class-Incremental Learning it focuses on. Semantic shift estimation has been extensively studied in EFCIL. The method proposed in this paper further considers the semantic shift differences between different classes compared to previous work, although this entails a more complex computation.", "theoretical_claims": "I have specifically checked the correctness of the proofs for the theoretical claims in this paper.", "experimental_designs_or_analyses": "Yes. This paper follows the previous experimental design on three benchmark datasets. Experiments on larger datasets are beneficial to further demonstrate the effectiveness and generalizability of the proposed method.", "relation_to_broader_scientific_literature": "1. Compared to previous work that only focused on task-level shifts, this paper further considers the differences in semantic shifts between different categories.\n2. This paper uses ridge regression to reconstruct the classifier instead of retraining the general classifier to solve the decision bias of the classifier.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths:\n\n1. This paper considers both semantic shifts and decision biases.\n2. The proposed method achieves state-of-the-art performance on multiple datasets.\n\nWeaknesses:\n\n1. There have been some previous works that estimate semantic drift and feature projection [1,2,3]. The advantages of the proposed method over previous works and how it solves the limitations of previous works should be elaborated to help readers understand the novelty of the method.\n2. Referring to Eq.8 and Eq.9, does the CIP need to learn and store a linear projector for each class at each subsequent incremental phases? If so, then the additional computational and storage costs incurred by these projectors should be discussed, compared to the case where they are not used. Is this going to be a significant burden on large-scale datasets?\n3. The steps of using uncentered covariance and singular vector decomposition seem to result in information loss. The error caused by this information loss gradually accumulates as the number of stages experienced by the class learned earlier increases.\n4. Lack of details on training, such as learning rate and weight of distillation loss, etc.\n\nReference:\n[1] Bring evanescent representations to life in lifelong class incremental learning. CVPR 2022.\n[2] Elastic feature consolidation for cold start exemplar-free incremental learning. ICLR 2024\n[3] Prospective Representation Learning for Non-Exemplar Class-Incremental Learning. NeurIPS 2024.", "other_comments_or_suggestions": "None.", "questions_for_authors": "Please see weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper presents an approach for exemplar-free class-incremental learning that simultaneously addresses the challenges of semantic shift and decision bias. The paper received mixed reviews, with primary concerns focusing on missing results for fine-grained datasets, lack of comparison with existing methods, and an unclear motivation for the Ridge Regression-based Classifier Reconstruction (RRCR) component. Several of these issues were clarified during the rebuttal phase; however, one reviewer further requested other clarification regarding RRCR, which was subsequently provided by the authors. The AC has reviewed both the paper and the rebuttal and agrees that all major concerns have been satisfactorily addressed. As such, the paper can be accepted.
Decision: Accept (poster)