Rethinking Continual Learning with Progressive Neural Collapse
Authors: Zheng Wang
Wanhao Yu
Li Yang
Sen Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Continual Learning
Neural Collapse
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Continual Learning (CL) seeks to build an agent that can continuously learn a sequence of tasks, where a key challenge, namely Catastrophic Forgetting, persists due to the potential knowledge interference among different tasks. On the other hand, deep neural networks (DNNs) are shown to converge to a terminal state termed Neural Collapse during training, where all class prototypes geometrically form a static simplex equiangular tight frame (ETF). These maximally and equally separated class prototypes make the ETF an ideal target for model learning in CL to mitigate knowledge interference. Thus inspired, several studies have emerged very recently to leverage a fixed global ETF in CL, which however suffers from key drawbacks, such as *impracticability* and *limited performance*. To address these challenges and fully unlock the potential of ETF in CL, we propose **Progressive Neural Collapse (ProNC)**, a novel framework that completely removes the need of a fixed global ETF in CL. Specifically, ProNC progressively expands the ETF target in a principled way by adding new class prototypes as vertices for new tasks, ensuring maximal separability across all encountered classes with minimal shifts from the previous ETF. We next develop a new CL framework by plugging ProNC into commonly used CL algorithm designs, where distillation is further leveraged to balance between target shifting for old classes and target aligning for new classes. Extensive experiments show that our approach significantly outperforms related baselines while maintaining superior flexibility, simplicity, and efficiency. Our code is available at https://github.com/yourname/ProNC.

Published as a conference paper ICLR 2026
RETHINKING CONTINUAL LEARNING WITH PROGRESSIVE NEURAL COLLAPSE
Zheng Wang' Wanhao Yu² Li Yang"
Sen Lin
*Deaarmeet of Computer Science, University of Houston, USA 2 DDpaatment of Computer Science, University of North Carolina at Charlotte, USA
ABSTRACT
Continual Learning (CL) seeks build an agent that can continuously learn sequence of tasks, where a key challenge, namely Catastrophic Forgetting, persists due to the potential knowledge interference among different tasks. On the other hand, deep neural networks (DNNs) are shown to converge to a terminal state termed Neural Collapse during training, where all class prototypes geometrically form a static simplex equiangular tight frame (ETF). These maximally and equally separated class prototypes make the ETF an ideal target for model learning in CL to mitigate knowledge interference. Thus inspired, several studies have emerged very recently to leverage a fixed global ETF in CL, which how- ever suffers from key drawbacks, such as impracticability and limited performance. To address these challenges and fully unlock the potential of ETF in CL, we propose Pro- gressive Neural Collapse (ProNC), a novel framework that completely removes the need of fixed global ETF in CL., Specifically, ProNC progressively expands the ETF target in principled way by adding new class prototypes as vertices for new tasks, ensuring maximal separability across all encountered classes with minimal shifts from the previous ETF, We next develop new CL framework by plugging ProNC into commonly used CL algorithm designs, where distillation is leveraged balance between target shifting for old classes and target aligning for new classes. Experiments show that our approach significantly out- performs baselines while maintaining superior flexibility, simplicity, and efficiency. Our code is available at /estps:/gqithn
Edge AI Lab/Paab... /ProC
1 INTRODUCTION
Continual Learning (CL) has gained much attention in recent years, aiming to mimic the extraordinary human abilities to lear different tasks in a lifelong manner. A key challenge here is Catastrophic Forgetting, iee. deep neural networks (DNNs) exhibit a pronounced tendency to lose previously acquired knowledge when trained new tasks (McCloskey & Cohen 1989). Within the spectrum of CL scenarios, class- incremental learning (CIL) presents the most formidable setting (Masana et al. 2022). where the model must not only address the current task by differentiating intra-task classes but also retain the knowledge of prior tasks by distinguishing historical classes from newly introduced ones. Yet, achieving this dual objective remains particularly challenging, as evidenced by the suboptimal performance of existing CL methods.
Recent studies (e.g., et al. (2000) have identified a compelling empirical phenomenon in DNN train- ing termed Neural Collapse (NC). During the terminal phase of training-wiin the training error asymp- totically approaches zero zero-the last-layer features of samples within the same class converge to their class- specific mean, while the means of all classes align with their corresponding classifier prototypes. These pro- totypes further collapse geometrically to form the vertices of Simplex Equiangular Tight Frame (ETF). This phenomenon results in four critical properties: (1) Feature Collapse: Features from samples within
Corresponding author: centrl@centrall.. uuueuu

---

Published as a conference paper ICLR 2026
the same class converge their class-specific mean, effectively eliminating within-class variability. (2)
ETF Geometric Alignment: The class-specift means for all classes align with the vertices of an ETF. (3) Ctassifee-Prottotyy Equivalence: These class means further align with the weights of the linear classifier. (4) Decision Simplification: Predictions reduce to a nearest-class. mean rule, where test samples are assigned the class whose feature mean is the closest.
A key idea emerging here is that the elegant NC prop- erties of DNN training naturally characterize an ideal model for CL: All classes seen far will have nearly- zero within-class variability, and their corresponding class-means equally and perfectly separated. Several studies (Yang et al. 2023b][Dang et al.] (025) Yang et al.) 2023a) have recently emerged to leverage NC and pre-
60.9
60
Value
52.4 51.7
K=10
Accuracy (ACC)
K-100
K=200
K=500
30.1
K- K=1000
30
define a fixed global ETF as the model training target in
20
CL, which however suffer from significant limitations: 1)
Setting the number of vertices in the predefined ETF re- quires the knowledge of total class number encountered during CL before learning the first task, which is clearly not practical. While Yang et al.] (2023b) posits that in- creasing the total class number k can overcome these lim- itations, fig. D shows that the consequently diminished angle between vertices actually degrades performance. 2) When the total class number is very large, the distance between any two vertices in the predefined ETF will be very small. Pushing class means towards these closely located vertices will unnecessarily hinder class discrimination, especially in early stages of CL when the number of seen classes that the model has to discriminate among is much smaller and the distance between these class means is larger. 3) NC posits that ETFs emerge naturally from feature convergence during training. Predefining the ETF contradicts this emergent behavior, as random initialization of the ETF risks geometric misalignment between learned features and the imposed topology.
Cifar10 cifar100
Tiny-Ima ge Net
Figure 1: Accuracy under different sizes of pre- defined ETF in NCT
To handle these limitations, key insight is that the number of vertices in the target ETF for model training should match the total number of classes seen so far to achieve maximal across-class separation anytime in CL. Thus inspired, instead of relying on a predefined ETF with critical design flaws, we seek to develop a novel approach that can appropriately adapt the ETF target for CL in order to fully unleash the potential of NC. To this end, our contributions can be summarized as follows:
I) principled approach for ETF expansion. By rethinking the objective of CL classification as facilitat- ing progressive NC with growing ETF after learning each new task, we propose novel approach, namely ProNC, to dynamically adjust the target ETF during CL. Specifically, ProNC first extracts the initial ETF target that emerges from first task training, and then expands the ETF target by adding types as vertices prior to new task learning, to ensure maximal separability across all encountered classes without causing dramatic shifts from the previous ETF. In principle, ProNC can be broadly applied in CL frameworks as new type of feature regularization.
class proto-
A simple and flexible framework for CL based on ProNC. We next develop new CL framework by plugging ProNC into commonly used CL algorithm designs. In particular, building upon the standard cross- entropy loss for new task learning, we introduce two additional losses, i.e., the aligmment loss and the distillation loss. The former seeks to push the learned class features towards the corresponding target ETF provided by ProNC, whereas the later follows standard idea of knowledge distillation to mitigate feature shifts for old classes. A nearest-ETF classifier will be used to replace the standard linear classifier.
3) Comprehensive experiments for performance evaluation. We perform comprehensive experiments on multiple standard benchmarks for both CIL and TIL, evaluate the effectiveness of our CL approach com- pared with related baseline approaches. It can be shown that our approach significantly outperforms the
2

---

Published as a conference paper ICLR 2026
baselines especially on larger datasets and also enjoys much less forgetting, without introducing more com- putation costs. In particular, extensive ablation studies are conducted to justify the benefits of ProNC in terms of maximizing feature separation among different classes and minimizing feature shifts across CL.
2 PRELIMINARIES
Problem Setup. We consider general CL setup where a sequence of tasks T (1) arrives sequentially. Each task t is associated with a dataset I3; = {(xx,in }/ containing N, input-label pairs. A fixed capacity model parameterized by will be trained to learn one task time. This work focuses two widely studied settings: caaseinncremetal learning (Class-IL) and eakkincceeental learning (Task-IL). In both settings, there is no overlap in class labels across tasks, ensuring ID, nrr 0 for any two distinct tasks 1 * L'. In Class-IL, the model does not have task specific information, requiring all data to be classified through a unified global classifier. In Task-IL, task-specific identifier is provided, enabling classification via dedicated task-level classifiers.
Neural Collapse. To formally characterize the NC phenomenon (Papyan al. 2020) emerged during terminal training phases of DNNs, it is necessary to first define the simplex ETF geometry.
Definition 1 (Simplex Equiangular Tight Frame). A simplex equiangular tight frame (ETF) is a set of vectors JK E _dd (d > K - 1) with the following properties. 1) Equal Norm: All vectors have identical t-- norm, i.e., ||+||22 = 1, vk E f1...., ......KK. 2) Equiangularity: The inner product between any two distinct vectors is minimal and constant, i.e., O22 Ck2 KK1 Vky # k2. Then, simplex ETF can be constructed
1
from an orthogonal basis U E IRdxK (where U Ix) via:
E E=
-=VK=-
K K-1 (IK - k)
where E =(ee,,--. ek]] is the ETF matrix, Ig the identity matrix, and 1 the all-ones vector.
The NC phenomenon can then be characterized by the following four properties (Papyan et al. 2020 (NC1): The last-layer features of samples within the same class collapse their within- class mean, resulting in vanishing intra-class variability: the covariance Σ(k)
W + 0, where Σ(k) Avg, {(UR, - (pu.) - pa) fuas is the feature of sample i in class k, and sas. is the within-class feature mean of class k; (NC2): The centered class means {āz} align with a simplex ETF, where fug (pL haee-M))/lll pG)//lmx
E Ma; (NC3): The centered class means align with their corresponding classifier prototypes, i.e., i4 = Wk [ Il S < K, where Wi. the class prototype of class k; (NC4): Under NC1-NC3, predictions reduce to a nearest-class-center rule, i.e., arg max; (js, Wx) arg min; || - p ll. where f is the last-layer feature a sample for prediction.
uell and the global mean Nc = K
3 CONTINUAL LEARNING WITH PROGRESSIVE NEURAL COLLAPSE
3.1 PROGRESSIVE NEURAL COLLAPSE
To completely remove the need of predefining global fixed ETF as the feature learning target for CL, we next seek to answer the following two important questions: 1) How should the base ETF target be initialized? How should the ETF target be adapted during CL.?
1) ETF initialization after first task. Previous studies YYang et al. 2023b) randomly initialize the ETF target, which could lead potential misalignment between the predefined ETF and learned features during task learning. Note that after the training of Task 1, last-layer class feature means {Ne) converge an ETF EdxKi, where Me E Rd and K₁ is the number of classes in Task 1. Thus motivated, the initial ETF should be extracted from the first task training to address the misalignment, which leads to an ETF target that matches the number of classes in Task 1.
3

---

Published as a conference paper at ICLR 2026
However, in practice it is difficult to fully reach the asymptotic convergence regime of model training with zero training loss, such that the learned class feature means MR. { for Task 1 will not strietly satisfy the ETF properties, as also corroborated in our empirical observations. Here fa.. is the empirical feature mean over samples within the class e. To handle this, key step is to find the right ETF target that is closest to Mx, after the training for Task converges, i.e., E = argming g ||M,, Ell To this end, based on Definition]] [ we have the following theorem that characterizes the nearest ETF after Task 1:
Theorem 1. Let Let /
(IK. X.. 1k, 1k.
and the SVD of U' is WEV Then the ETF
matrix E can be obtained as follows:
WL (Ix, A 1
E E-=
(2)
Given the learned class feature means MX,, Theorem immediately indicates three-step procedure to construct the initial ETF target that aligns well with the task learning: 1) Construct U' from Mx,, 2)
Conduct SVD on U', 3) Construct E* based on Equation 2
2) ETF expansion prior to new task learning. Given the initial ETF matrix E E where the number of vertices matches the number of classes in Task 1, the next step is to progressively expand the ETF target as new classes come in with new tasks, in order to achieve two objectives: 1) the number of newly expanded vertices in the new ETF target will match the number of new classes in the new task; 2) the vertices in the new ETF target that match old classes will not significantly shift from their original positions in the old ETF target, so as to reduce catastrophic forgetting.
A key insight here is that the ETF matrix E is indeed determined by the corresponding orthogonal basis U as shown in Equation (T). and keeping the orthogonal basis unchanged when expanding the ETF will in principle reduce the shift from the old ETF, This motivates novel ETF expansion strategy by expanding the constructing orthogonal basis, which includes two steps:
(Step a) Let K₁ be the total number of classes until any task t. When a new task t 2 arrives with K,-K, classes, the ETF target expansion will be triggered before learning task 6, which seeks to obtain a new target E, with K2 vertices from the previous ETF target E,-1 with K vertices. In particular, the original orthogonal basis U,-1 E Rd K, of E,-1 will be expanded to U, E Rd by appending K2 K new orthogonal vectors. These new vectors are generated via Gram-Schmidt orthogonalization against the existing K,- basis vectors in U.- ensuring that U, retains orthonormality across all K2 vectors.
(Step b) Substituting U, and K; into Equation T will lead to an expanded ETF target with K2 vertices. This extended ETF serves as the predefined geometric configuration for feature learning in task t, maintaining uniform angular separation and maximal equiangularity among all seen classes
3.2 A CONTINUAL LEARNING FRAMEWORK BASED ON PROGRESSIVE NEURAL COLLAPSE
In what follows, we seek to incorporate the idea of progressive neural collapse (ProNC) into commonly used CL algorithm designs, where the model will be trained to push the learned class features towards the progressively expanded ETF for each task during CL. More specifically, we focus on the model training for tasks Z 2. whereas the first task learning follows a standard supervised learning procedure with widely used loss functions, e.g., cross-entropy loss Loe. For tasks t > 2, we first apply ProNC to generate a newly expanded ETF target E, before learning task t. The loss function design for model training will include three different loss terms, i.e., supervised term, an alignment term, and distillation term. The first supervised term follows the standard cross-entropy loss to facilitate intra-task classification, while we introduce the other two loss terms below in detail.
1) Alignment with the ETF target. The alignment loss pushes the learned class features towards the ETF target E, je t. which characterizes the cosine similarity between the feature Mkli for sample i
4

---

Published as conference paper ICLR 2026
ft
extend
ETF,
3TF
Dr
M 1
f--1
Distillation
cur
Figure 2: Overflow of our proposed CL framework for new task learning based on the mixture of current task data and replay data. The new model fe is trained towards the expanded ETF target, with forgetting further reduced based on feature distillation.
in class and the corresponding vertex ex, in the ETF E, for class (Yang et al. 2022 2023b
Lalign (1 (x.,e) 2 ( 1)2.
(3)
Here p corresponds to the normalized feature extracted from the last layer of the current model when learning task t. small Lalign implies that the learned normalized feature for each sample aligns well the corresponding ETF vertex for the class that the sample belongs to, minimizing the intra-class variability while forcing different class feature means equally separated. This indeed provides a new type of feature regularization based
progressively expanded ETF, which can be widely adopted and is also very powerful as shown later in our experimental results.
Distillation further reduce forgetting. While ProNC expands the ETF target without dramatically shifting from the old ETF, the vertices that map to old classes will inevitably change after ETF expansion due to geometric properties of ETF. To handle this and further reduce catastrophic forgetting, we next borrow the idea of knowledge distillation, which is widely used technique in CL to reduce catastrophic forgetting (Rebufti et al. 2017 Hou al., 2019; Buzzega et al. 2020: Yang al. 2023b). In particular, we consider a typical lation loss which characterizes the cosine similarity between the normalized features, for the same data sample, from the current model and that from the model obtained after learning the previous task, to maintain the simplicity and flexibility of our framework:
(1) Laisau ( K,i (e.3) ((P ((( Mk.s T 1
(t-1) (t)
(t-1) is the normalized last layer feature for the sample in class after task 1. Here Pii
To best unlock the potential of ProNC, we also leverage the standard data replay in the CL framework: the replay data from previous tasks will be mixed together with the current data before the current task learning, such that each minibatch of data during model training will include both replay data and current data. Combining all three loss terms will lead to the final instance-loss function:
C = Lee Å1 Lalign + >2 Laissill,
(5)
which is averaged over all samples in a minibatch for model update. Here ê₁ and A2 are coefficients to balance between pushing the learned features towards the target ETF and reducing the vertex shift from the previous ETF for minimizing forgetting. The overall workflow is shown in Figure 2
Inference. During inference for model performance evaluation, instead of using the linear classifier for classification, leverage the widely used cosine similarity (Peng et al. 2022) Gidaris & Komodakis 2018 Wang et al.) 20188] Hou et al. 2019) between the extracted sample feature from the learned model and the vertices in the target ETF the classification criteria. Specifically, for any testing data sample let μ denote the normalized last layer feature extracted from the tested model. Then the class prediction result will be arg max; 14, Ck. where eg is the k-th column vector in the target ETF.
5

---

Published as a conference paper ICLR 2026
4 EXPERIMENTAL RESULTS
4.1 EXPERIMENTAL SETUPS
Datasets and baseline approaches. We evaluate the performance on three standard CL benchmarks, i.e., Seq-CIFAR-10 (Krizhevsky et al.) 2009) that partitions 10 classes into 5 sequential tasks, Seq-CIFAR 100 Krizhevsky et al. 2009) that comprises 10 tasks with 10 classes per task, and Seq-Tiny Imaggmeeeee (Le & Yang 2013] divides 200 classes into 10 sequential tasks, and consider both Class-IL and Task-IL scenarios. We evaluate CL approach against state-of-the-art (SOTA) approaches, including various replay-based (011)), iCaRL. (Rebuffi et al. (017). GEM (Lopez-Paz & Ranzato 2017). (Buzzega al. 2020), DER++ (Buzzega et al. (2020), STAR (Eskandar 2025), and contrastive learning based approaches, i.c., Co²L (Cha et al. MNCLL (Dang et al., 2025). We also compare our approach with NCT
approaches, i.e., ER (Riemer et al.
GSS (Aljundi et al. 2019). DER
ct al.
2025), CSReL Tong et al.
2021
CILA (Wen al., 2024).
YYng et al. (023b) that predefines a fixed global ETF target.
Implementation details and evaluation metrics. To ensure a fair comparison, we train a ResNet-18 18 back- bone (He et al.) 2016) using the same number of epochs and batch size for all approaches. For contrastive learning approaches Co CILA, and MNC* we follow their original implementations by removing the final classification layer of ResNet-18 18, and append two-layer projection head with ReLU activation to map backbone features into a d-dimensional embedding space (d = 128 for Seq-CIFAR- Seq-TinylmageNerr The training epochs are 50 for CIF-CI0 and Seq-CIFAR-100, and 100 for Seq- instead of 500 epochs for the initial tasks, 100 -Seq-CIFAR-IA 10 and Seq- CIFAR- -000 and 50 Soq4TinnImmagehhe for incremental tasks. A separate linear classifier is subsequently trained on the frozen embeddings for these contrastive learning methods. For STAR ( (Eskandar et al.) (025 ), we employ the performance of ER+SARR, since method can be seen adding a regularization term ER. Hyperpa- rameter details are in Appendix BB.2 We consider two standard evaluation metrics in CL. i.e., final average
d = 256 for
accuracy (FAA) and average forgetting (FF). Let T denote the total number of tasks and of be the model accuracy on the i-th task after learning the task & E [1,7"] The FAA and FF are defined as:
FAA = + E. af T-1
FF
T-2
maxte (D.., of - 0g Tr-
i=
4.2
MAIN RESULTS
Table T shows the performance comparison for both Class-IL and Task-IL under 200 and 500 memory budgets. It is clear that our approach significantly and consistently outperforms all the baseline approaches across all considered CL settings, datasets, and buffer sizes. In particular, the performance improvement in our approach becomes more substantial larger datasets and under a smaller buffer size. For example, consider a buffer size of 200. On Seq- CIFAR-IOO our approach outperforms the best baseline approaches, i..., DER for Class-IL and NCT for Task-IL, by 37.65% and 13.0 13.04%. respectively. On Seg-TinylmageNet, our approach outperforms the best baseline approaches, i.c.. CSREL for Class-IL and NCT for Task-IL, by 59.32% and 31.08%, respectively. buffer size is 200, especially for Task-IL when we focus on differentiating classes within a specific task. This implies that our approach is more robust to different buffer sizes and the principle of ProNC can even work well with small amount of replay data. Moreover, by leveraging the NCT in more principled manner, our approach dominates the previous approach NCT where a predefined global ETF target hinders class discrimination by unnecessarily forcing class means towards closely located vertices.
particular, the performance of our approach is outstanding when the
Besides, it can be seen from Table [ that among most scenarios, our approach and NCT achieve much less forgetting compared to other baseline approaches, while our approach shows even better forgetting than NCT in 8 out of 12 settings. The reason is that, instead of simply constraining the shifts from old features important weights in previous studies, the ETF target offers additional fixed goal from which the
6

---

Published as a conference paper at ICLR 2026
Table 1:
Performance comparison under various setups. All results are averaged over multiple runs. The final version with error bars is in the appendix.
Soq-CIFAR-ID
Soq-CIFAR-100
Seq- Ninylmgeee
Buffer
Method
Class-IL
Task-IL
Class-IL
Task-IL
Class-IL
Task-IL
FAA (FF)
FAA (FF)
FAA (FF)
FAA (FP)
FAA (PF)
FAA (FF)
ER
Riemer et al_
2019
44.79 (39.30)
91.19 (6.07)
21.78 (75.66)
60.19 60.19 (27.38)
8.49 (76.53)
38.17 (40.47)
&CaLL
Reeeam et p01r
49.02 (23.52)
88.99 (25 34)
28 (47.20)
51.43 (36.20)
7.53 (31.06)
28.19 (42.47)
GHM
aape7-Pa7
2017
25.54 (80.36)
90.44
20:75 (77 40)
58.84 (29.5))
GSS
Aujandi et al 2019
39.07 (72.48)
88.8 (8.49)
19.42 (77.62)
55.38 (32.81)
DER
Fizz
61.93 (35.79)
91 (6.08)
31.23 (62.72) 72)
63.09 (25,98)
11.87 (64 83)
40,22 (40.43)
200
DER+
Buzzoga
(32.59)
(5.16
99)
(23,91)
11.34 (73.47)
(39.02)
LODE
Liung & 1a 2126
68.01 (24.63)
93.11 (4.75)
26.65(44.29)
71.23 (18.75)
15.13 (64)
51.42 (29.66)
Co
Cha BF. 2014
51.27 (30.17)
(2.91)
18.09 (61.14)
19.19 (27.83)
12.95 (62.04)
38.40 (40.75)
CILA
Wen
2024
(37.52)
(64.01)
(33.07)
12.98 (63.11)
37.32 (41,40)
MNC
(ung ct al 2025
$1.09 (33.74)
85.07 (4.90)
15.81 (62.51)
43.91 (39.79)
10.57 (59.68)
32.78 (45.10)
STAR
65.94 (15.99)
95. (2.06)
38.15 (42.17)
79.53 (17.32)
13.64 (68.51)
43.01 (39,16)
CSRc
Toony of
7176
37.46 (26, 34)
69.22 (17.16)
29.06 (58 23)
66,99 (23,20)
18.14 (7)
45.04 (34.12)
NCT
Yang ct
20236
51.59 (22.48)
80.63 (1.41)
26.38 (27
75.75 (4.79)
10.95 (49. 33)
52.71 (15.88)
Ours
65.58 (32.75)
96.86 (0.65)
42,99 (36.07)
85.63 (4,40)
27.44 (42.81)
69.09 (9.42)
ER
Risneer et al 2019
57.74 (43.22)
93 61.6116 (3, 50)
22.35 (73.08)
73.98 (16.23)
9.99 (75.21)
48 64 (30.73)
iCaRL
Reovani et tin p01
47.55 20)
88.22 (22 61)
33.25 (10.99)
58.16 (27 90)
9.38 (37.30)
31.55 (39.44)
GEM
Lapee-rrr
2017
92.16 (78.9))
25.54 (71.34)
66.31 (20
GSS
(oljondi of al 2019
49.73 (89.18)
(6. 37)
21.92 (74. 12)
60.28 (26,57)
DER
bu2zgg et di 20e0
70.51 (24.02)
(3.70)
41.36 (19.07)
71.73 98)
17.75 95)
51.78 (28.21)
500
DER+
72.70 (22.38)
93.88 (4.66)
38.20 (49.
74.77 (15.75)
19.38 (58.75)
51,91 (25,47)
LODE
Liung & La 2026
75.91 (18.18)
94.19 (3.94)
40.01 (32.58)
80.06 (8.96)
20.5 (56.51)
61.49 (18.61)
Co'l.
Cha ér Br MAA1
61.78 (17.79)
89.51 (2.65)
26.64 (48.60)
62.12 (23.47)
18.71 (49.64)
50.74 (13.80)
CILA
Wen
2034
67.82 (18.22)
(45.67)
(16.98)
(64.14)
(27.83)
MNC
Daitz 2 mil 1416
52.20 (27.88)
85.94
22.29 (46.09)
56.43 (24.29)
11.52 (44.96)
36.12 (39.00)
STAR
(Eacandir et p02a
69.19 (12.21)
95. (2.38)
47. 56 (28 68)
78.28 (15.43)
21.31 (57.2)
59.32 (19.49)
CSRe
Trop 21163
(29.
73.30 (16.89)
22 (47 03)
75.09 (15.55)
22.13 (50.01)
$1,94 (28.20)
NCT Yang ol am. 212510
60,93 (13.82)
81 (3.84)
33.77 (27 85)
75.87 (5 (5.06)
18.24 (50.90)
62,30 (6.84)
Ours
73.95 (26.75)
96.95 (0.24)
48.94 (24.32)
86.38 (4.35)
29,06 (38,55)
69.77 (9,52)
model should not shift the new features too far away. These results indicate the huge potential of leveraging NC and ETF in guiding the feature learning for CL.
More interestingly, in contrast to previous replay-based approaches (Riemer et al. 2019. Rebufñ et al. 2017| Lopez-Paz Ranzato 2017: Aljundi et al. 2019: Buzzega et al. 2020) which require a replay buffer in principle, our approach can even work without replay. In table 2 buffer. Compared with contrasiie-leaminn--aass (Dang et al., 2025), our method still achieves superior performance. Specifically, with an empty replay buffer, our approach can achieve a FAA of 32.28%. 28% for Class-II. and 84.62% for Task-IL Seq-CIFAR- 100, and 24.43% for CIass-IL and 68.08% for Task-IL on Seq- TinyImageeet.. Remarkably, these results almost surpass all baselines in table even when those baselines use a buffer size of 200 on Seq-CIFAR- 100, and for Seq-Tiny TinyImageeee.. without a memory buffer, ProNC outperforms all baselines using buffer sizes of both 200 and 500. This phenomenal performance implies that our approach indeed offers new and powerful feature regularization based on ProNC, which can be widely applied in various CL scenarios.
Table 2: Performance comparison with buffer size zero
Buffer
Method
See- CIFARRIA00000
Seq Tiny ImageNet
Class-IL
Task-IL
Clas--L
Task-IL
Co'L
Cha
202
26.06
(40.02)
(65.
38.98 (40.77)
MNC
Ding o 2022
30.48 (53 03)
56.69137.002
14.04 (54.25)
42.59 (37 89)
Ours
32.28 (45.92)
84.62 (4.39)
24.43 (46.14)
68.08 (9,81)
we report the results for both Class-IL and Task-IL under an empty memory methods such as Co2L (Cha et al., 2021) and MNC3L
4.3 ABLATION STUDY
In what follows, we will conduct various ablation studies to build a comprehensive understanding of our approach, where most studies are for Class-Il. on Seq-CIFAR-100 with buffer size 500.
Feature learning behaviors. To understand the superior performance of our approach in contrast to important baseline approaches, we first delve into the feature learning behaviors during CL by characterizing different
7

---

Published as conference paper ICLR 2026
3.0
1--
1e-1
3.6
2.0
1.5
8.0
3.0
7.5
2,8
Dur
0.5
2.6
7.0
(a) Cosine similarity of feature means between any two differ- ent classes, averaged over all seen classes.
(b) Std of cosine similarity of feature means between any different classes same task, averaged over tasks.
(c) Cosine similarity between best previous features and final features for same class, av- eraged over all seen classes.
16.150
-3-
-4-
18.170
Ou/
OU/
NCT
-5
NET DER+
19,180
DER+
20.190
Thikk
60 70
(d) Cosine similarity between one class feature mean and the classitier prototype of any seen different class.
(e) Cosine similarity between one class feature mean and the classifier prototype of another class in the same task.
(f) Performance robustness our approach under different values of the hyperparameters A2).
Figure: 3: In (a)-(e), X-axis is the task ID during CL and Y-axis is the (average or std) value of the corre- sponding cosine similarity. In (f), X-axis the accuracy FAA and Y-axis is the value of (A1, A2).
types of feature correlations. In particular, based on our analysis of the ETF target and Table I compared NCT Yang et al.) 2023b) with a predefined global ETF, our approach should enjoy the following benefits:
A lower cosine similarity between different class feature means, which is also closer the theoretically maximum separation K. for K, seen classes until task according to Defini- tion This can be confirmed in Figure where we calculate Avgaye (( (1Tm2_pe11
Table 3: ProNC as feature regular-
ization
Method
FAA
Pg-PG [pa]- + k of features after learn- ing every task t, averaged for any two seen classes k and k'. Thanks ProNC, our approach achieves smaller value, which is also closer 0, than NCT. In NCT, with a global ETF of Kglabal classes, the pairwise inner product between any two normalized ETF vertices is 11 - 1). which is very close 0 when Kglobal is large (e.g., - 1/9999 for Kglobal = 1000), so carly-task classes are packed with relatively small angular separation. 2) All class feature means should be almost equally separated within the same task. To show this, we evaluate the standard deviation (std) of across-class cosine similarity for the same task, averaged over all seen tasks in CL, i.e., Avg(std( and k are in the same task. Figure 3b shows that our approach achieves a smaller average std (also closer to 0) than NCT, implying the different class means closer to equal separation. Besides, also follow the
ICaRL.
Rebull et
2017
33.25
ICaRL
PraNC
38.87
L.CCIR
Hou et
37.69
LAICIR With PToN
40.50
ER Riemer ala
2019
22.35
b025
with STAR
ER with ProN(
48.94
DER [Ruzzega al 2020
38.20
DER WITTODE
2023
DER+ with STAR (Fokmmhrr cl
39.77
DRR with ProNt.
47.34
XDER (Buzzega et 2020
49.93
XOKR
( Tpe-Pol) PPy
Py
))), where classes k
8

---

Published as a conference paper at ICLR 2026
metrics in NCT (Yang et al. 2023b) evaluate 1) the cosine similarity between the feature mean of class k and the classifier prototype wy of a different class k', averaged over all seen classes until task t. 2) the same cosine similarity but for classes within the same task, which averaged over all seen tasks during CL. As shown in Figure 3d and [3] our approach achieves lower cosine similarity compared to NCT and DER++ which implies easier class discrimination.
Moreover, Table | demonstrates the exceptional performance of our approach in addressing forgetting. To understand this, we characterize the cosine similarity between 1) the learned features from the best perform- ing previous model and 2) that from the final model. for the same class, averaged over all seen classes so far. This is consistent to the definition of forgetting. As shown in Figure 3c our approach achieves a higher similarity than both NCT and DER++ indicating its superior capability in handling feature shifts during CL and then minimizing forgetting based on ProNC.
Generality of our approach. As discussed in Section our ap- proach introduces a novel and effective feature regularization method based on ProNC, which can be generally incorporated into different CL frameworks. To further verify this, we conduct more experiments by plugging ProNC into established replay-based methods (iCaRL (Rebulli et al. 2017). LUCIR (Hou et al. 2019)) ER (Riemer et al. DER++ (Buzzega et al. 2020), and XDER (Buzzega et al.,
Table 4: Impact of different loss de- sign
Loss Combination
FAA
Cings + Lawill
48,94
2019.
Callann + I2-Norm loss
49,42
12-Norm loss Eaisin
48.61
2020 For studies with feature- wise distillation (iCaRL and LU- CIR), we keep all the components the same as the original designs in these studies (including the classifier design) and only incorporate
2u-Norm loss + No-r loss
48.66
ProNC with the alignment loss. For studies without feature-wist distillation (ER and DER++), in addition to the alignment loss, we also add feature- wise distillation and keep all the original components. Table shows the performance comparison among iCaRL, LUCIR, ER, DER++, XDER, and the versions enhanced with the ProNC regularization. For the SOTA regularization STAR (Eskandar et al. (025) and LODE Liang & Li (202)), the training of current and buffer data needs to be separated, making it unsuitable for iCaRL and LUCIR. Clearly, incorporating the ProNC regularization yields substantial performance improvements over both the original design, LODE, and STAR, which proves the potential and generality of ProNC as a feature regularization for CL.
Flexibility of our designs. Our approach is very flexible in the sense that the loss terms in the final objective function can be replaced by other designs. To show this, we replace the cosine-similarity in the loss functions, i.e., falggn and Luisin, by using a standard 12-norm, and con- Seq-CIFAR- CIF with
Table 5: Impact of different com- ponents.
Variant
Performance
duct experiments under the Class-IL setting a memory buffer size of 500. As shown in Table 4 the performance of our approach is stable under different design combinations when we
Ours
48.94
(a) w/o Eoe
44.97
(b) wlo Langa
23,22
(e) w/o Caum
19.96
(d) w/ predefined base ETF
44.99
replace the cosine similarity in any of the two loss terms, which further
(e) predefined global ETF
corroborate the flexibility of our approach. In principle, our approach can be generally applied with a loss function that seeks to minimize the
(f) w/ linear classifier
44.49
distance between the learned features and the ETF target/old features.
Impacts of different components. To understand the impact of different design components on our approach, we investigate six different variants, as shown in Table Here in (a)-(c) we remove one of the three loss terms in Equation (5), respectively. Clearly, removing either Lalign or Ldistill will significantly degrade the performance, highlighting the importance of the designed ETF target and the right balance between learning stability and plasticity. On the other hand, while removing Zee will slightly decrease the performance, this variant still outperforms the baseline approaches as shown in Table [ indicating the benefit of our approach mainly from Lalign and Lasail- In (d), we predefine base ETF target and expand it for new tasks based on ProNC, and the degraded performance shows the benefit of naturally aligning the base ETF with feature
9

---

Published as a conference paper ICLR 2026
learning in the first task. In (e), we replace the entire ProNC with a predefined fixed global ETF as in (Yang) ee al. 2023b) and the performance drops dramatically, corroborating the importance of ProNC for setting an appropriate ETF target in new task learning. In (f), we replace our cosine similarity based classifier by using the standard linear classifier, and the performance drop further highlights the usefulness of ETF in classification by providing equally separated feature representation targets for different classes.
More comparison with contrastive learning based approaches. Con- trastive learning based approaches usually suffer from high computa- tion costs due to the nature of con- trastive learning with data augmenta- tion. In the original implementation of these approaches, e.g., Co (Cha et al. 2021) and MNC (Dang al. 2025). 500 training epochs are used for the initial task, and followed by different epochs for each subsequent task, i.c., 100 epochs for CIF-CIIA-- and 50 epochs for Seq-TinylmageNet. To further demonstrate the superior performance of our approach, we evaluate our approach under the standard setup, against these approaches under their original implementation in Table Clearly, while the performance of the contrastive learning based approaches improves with more training epochs, our approach still achieves significantly better performance with an even shorter training time.
Table 6: FAA, FF and total training time comparison with con- trastive learning based approaches under their training setups.
Buffee
Mathod
See CIFAR 100
See Tmpgengee
Clase-ll II
Tock-Il
Time(S)
Claseil.
Thak-Il
Timer
Cu'l MNC
Clustal al
20217
27.35 (67. 52)
42.37 (36.22)
4362
13.85 (73.25)
42.37 (47.11)
12494
200
TAKiy if 101
54.4 157 4m
50 at 1% 66)
11434
15:53:50 (8.52-17.07)
64,59 (51. 76)
19923
42.99 (36.07)
$5.63 (4.48)
1482
27.44
(42.91)
69099 (9.42)
12157
CLLL LNC
Cla al
2021
57.02 (51.23)
62.44 (26.31)
4380
20.12 (65.15)
53.04 (39.22)
12100
500
Tiai at 2015
40.25 (45-09)
63.35 (34.29)
3979
20:31 (46-08)
53.46 (25.45)
12669
48.91124.11
$6.38 (4.35)
1588
29,06 (38.58)
69.77 (9.53)
12140
Hyperpurameter robustness. We conduct experiments to demonstrate the impact of A and X2 in Equa- tion (5). As shown in Figure 31 the performance of our approach is very stable under a wide range of X values, indicating the robustness of our approach.
5 CONCLUSION
Neural collapse in DNN training characterizes an ideal feature learning target for CL with maximally and equally separated class prototypes through simplex ETF, whereas recent studies leverage this by predefin- ing a fixed global ETF target, suffering from impracticability and limited performance. To address this and unlock the potential of NC in CL, we propose a novel approach namely progressive neural collapse (ProNC), by initializing the ETF align with first task leaning and progressively expanding the ETF for each new task without significantly shifting from the previous ETF. This will ensure maximal and equal separability across all encountered classes anytime during CL, without the global knowledge of total class numbers in CL. Building upon ProNC, we introduce a simple and flexible CL framework with minimal changes on standard CL frameworks, where the model is trained to push the learned sample features towards the cor- responding ETF target and distillation with data replay is leveraged to further reduce forgetting. Extensive experiments have demonstrated the dominating performance of our approach against ofte--of-the-ar base- line approaches, while maintaining superior efficiency and flexibility. One limitation here is that assume clear task boundaries and it is interesting to see how our approach can be extended to more general setups. We hope this work will serve initial steps for showcasing the great potentials of NC in facilitating better CL algorithm design and inspire further research in CL community along this interesting direction.
10

---

Published as conference paper ICLR 2026
REPRODUCIBILITY STATEMENT
explanations of implementation details for ProNC presented. The details of hyperparameters, equipment, and code platform are presented in B.2 Furthermore, the code of our implementation is submit- ted together with the paper as supplement materials.
ETHICS STATEMENT
We have read and followed the ICLR Code of Ethics.
ACKNOWLEDGEMENTS
We want thank all reviewers and area-chairs for their efforts toward improving this work.
REFERENCES
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pp. 139-154, 2018.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019.
Zalán Borsos, Mojmir Mutny, and Andreas Krause. Coresets via bilevel optimization for continual learning and streaming. Advances in neural information processing systems, 33:14879-14890 2020.
Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, and Simone Calderara. Class-
incremental continual learning into the extended der-verse. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15920-15930, 2020.
Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co2l: Contrastive continual learning. In Proceedings of the IEEE/CVF International conference on computer vision, pp. 9516-9525, 2021.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision (ECCV), pp. 532-547, 2018a.
Arslan Chaudhry, Mare'Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv: 2018b.
Aristotelis Chrysakis and Marie-Francine Moens. Online continual learning from imbalanced data. In International Conference on Machine Learning, pp. 1952-1961. PMLR, 2020.
Trung-Anh Dang, Vincent Nguyen, Ngoc-Son Vu, and Christel Vrain. Memory-effiiient continual learning with neural collapse contrastive. In Winter Conference Applications of Computer Vision (WACV), 2025.

---

Published as a conference paper ICLR 2026
Masih Eskandar, Tooba Imtiaz, Davin Hill, Zifeng Wang, and Jennifer Dy, Star: Stability-iduucing weight perturbation for continual learning. In The Thirteenth International Conference on Learning Representa- tions, 2025.
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International conference on artificial intelligence and statistics, pp. 3762-3773. PMLR, 2020.
Qiankun Gao, Chen Zhao, Bernard Ghanem, and Jian Zhang. R-dfcil: Relation-guided representation learn- ing for data-free class incremental learning. In European Conference on Computer Vision, pp. 423 423-439. Springer, 2022.
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4367-4375, 2018.
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified elassifier incre- mentally via rebalancing. In 2019 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 831-839. IEEE, 2019.
Ching-Y Yi Hung. Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. Advances in neural information processing systems, 32, 2019.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barrinskww et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 115(13)3521-3356, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Sang-Woo Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung-Take -aan Zhang. Overcoming catas- trophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12))2935-299) 2017.
Yan-Shus Liang and Wu-Jun Li. Loss decoupling for agnk-agn continual learning, Advances in Neural Information Processing Systems, 36: 1151-11167. 2023.
Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Beyond not-forgetting: Continual learning with back- ward knowledge transfer. Advances in Neural Information Processing Systems, 16165 2022a.
Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Trgp: Trust region gradient projection for continual
learning. arXiv preprint arXiv:2202.0294 2022b.
Hao Liu and Huaping Liu. Continual learning with recursive gradient optimization. arXiv preprint 125. 2022.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
12

---

Published as a conference paper ICLR 2026
Mare Masana, Xialei Liu, Bartlomijj Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Weijer. class-incremente learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513 5533,, 2022.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard Turner. Variational continual learning. arXiv preprint arXiv: 10628, 2017.
Vardan Papyan, XY Han, and David I. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, (irrees,,11(400)) 117(40):2452222466) 2020.
Can Peng, Kun Zhao, Tianren Wang, Meng Li, and Brian Lovell. Few-shot insremenntl learning
from an open-set perspective. In European Conference on Computer Vision, pp. 382-397. Springer, 2022.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv: 1910, 2018.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations. International Conference on Learning Representations, ICLR, 2019.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances neural information processing systems, 32, 2019.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint a6Xiv: 04671, 2016.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.09762 2021.
Junao Shen, Qiyun Hu, Tian Feng, Xinyu Wang, Hui Cui, Sensen Wu, and Wei Zhang. In2nect: Inter-class and intra-class neural collapse tuning for semantic segmentation of imbalanced remote sensing images. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 68 6814-682, 2025.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. Ger: Gradient coreset based replay buffer selection for continual learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 99-108, 2022.
Ruilin Tong, Yuhang Liu, Javen Qinfeng Shi, and Dong Gong. Coreset selection via reducible loss in continual learning. In The Thirteenth International Conference on Learning Representations, 2025.
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2018
13

---

Published as a conference paper ICLR 2026
Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing Hong, Shifeng Zhang, Zhenguo Li, Yi Zhong, and Jun Zhu, Memory replay with data compression for continual learning. arXiv preprint arXiv:2202. 06592, 2022.
Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covari- ance for continual learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 184-19199 193, 2021.
Yichen Wen, Zhiquan Tan, Kaipeng Zheng, Chuanlong Xie, and Weiran Huang. Provable contrastive con- tinual learning. In International Conference on Machine Learning, pp. 52819-52838. PMLR, 2024.
Robert Wu and Vardan Papyan. Linguistic collapse: Neural collapse in (large) language models. Advances in Neural Information Processing Systems, 37:137432-137473. 2024.
Liang Xie, Yibo Yang, Deng Cai, and Xiaofei He. Neural collapse inspired attreciion-replliio--blaaad loss for imbalanced learning. Neurocomputing, 227:60-70, 2023.
Lu Xie, Weigang Li, and Yuntao Zhao. Hard example learning based on neural collapse for class-imbalanced semantic segmentation. Applied Soft Computing, pp. 112755, 2025.
Li Yang. Sen Lin, Junshan Zhang. and Deliang Fan. Grown: Grow only when necessary for continual learning. arXiv preprint arXiv:21 10.00908, 2021.
Yibo Yang, Shixiang Chen, Xiangtai Li, Liang Xie, Zhouchen Lin, and Dacheng Tao. Inducing neural col- lapse in imbalanced learning: Do we really need learnable classifier at the end of deep neural network? Advances in neural information processing systems, 35:37991-38002, 2022.
Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr and Dacheng Tao. Neural collapse inspired feature-llassifier alignment for few-shot class incremental learning. arXiv preprint arXiv:2302.03004 2023a.
Yibo Yang. Haobo Yuan, Xiangtai Li, Jianlong Wu, Lefei Zhang, Zhouchen Lin, Philip Torr, Dacheng Tao, and Bernard Ghanem. Neural collapse terminus: A unified solution for class incremental learning and its variants. arXiv preprint arXiv:2308.01 01746 2023b.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically ex- pandable networks. arXiv preprint 1rXiv: 01547, 2017.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning, pp. 3987-3995, PMLR, 2017.
Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task-agnostic continual learning using online variational bayes with fixed-point updates. Neural Computation, 33(1 11:31113139111101 3177, 2021.
Didi Zhu, Zexi Li, Min Zhang, Junkun Yuan, Jiashuo Liu, Kun Kuang, and Chao Wu. Neural collapse anchored prompt tuning for generalizable vision-language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 4631 1640, 2024.
Hui Zou, Trevor Hastie, and Robert Tibshirani. Sparse principal component analysis. Journal of computa- tional and graphical statistics, 15(2):265-22 2006.
14

---

Published as a conference paper ICLR 2026
APPENDIX
A RELATED WORK
Continual Learning. In general, existing CL approaches on standard neural networks can be divided into several categories.
1) Regularization-ben based approaches seek to regularize the change on model parameters that are important previous tasks (Znke et al.] |207]] |Chaudhry et al., 2018a). For instance, EWC (Kirkpatrick et al. (2077) penalized updating important weights characterized based on Fisher Information matrix. MAS (Aljundi jet al. 2018) characterized the weight importance based on the sensitivity of model updates if this weight is changed. Liu & Liu (2022) proposed an approach that recursively modified the gradient update to mini-
mize forgetting. The Bayesian framework has also been substantially investigated to implicitly penalize the
parameter changes (Lee et al. 2017 Nguyen et al. 2017 Zeno al. 2021
2) Memory-based approaches store information for previous tasks, which have shown very strong perfor- mance and can be further divided into two categories, i.e., rehearsal-based and orthogonal-projecte approaches (Riemer et al.] [0188] [Chaudhry et al., [018b] [Rolick et al.) 2019)
based
approaches. Rehearsal-basee store subset of data for previous tasks and replay them together with current data for new task learning- Some studies focused on how to select and manage the replay data to achieve better performance and effi- ciency, such as the use of reservoir sampling (Chrysakis & Moens 2020), coreset-based memory selection (Borsos et al., 22000; Tiwari et al. 2022; Tong ct al. (025), and data compression (Wang et al. 2022). Some other studies investigated how to utilize the replay data, such as imposing constraints on gradient up- date Chaudhry et al., [018bb Eskandar et al. 2025). combining with knowledge distillation (Rebuffi et al.,
2017
Buzzega et al., 20200 Hou et al. 2019; Gao et al., 2022), contrastive learning based approaches (Cha 2024). The use of generative data has also been explored in CL for rehearsal-based
et al. 2021 Wen et al.
approaches (Shin et al.
based approaches Wang et al.] 2021 Saha et al., 2021 Lin et al., 2202bb a) store gradient or basis information to reconstruct the input subspaces of old tasks, so as to modify the model parameters only along
2017). Instead of storing data samples,
(Farajtabar al., 2020
the direction orthogonal to these subspaces.
3) Architecture- based approaches freeze the important parameters for old tasks, train the remaining param- eters learn new tasks and expand the model if needed. Notably, PNN (Rusu et al. 2016) preserved the weights for previous tasks and progressively expanded the network architecture to learn new tasks. LwF (Li & Hoiem 2017) split the model parameters into two parts, where task-shared parameters are used to extract common knowledge and task-specific parameters are expanded for new tasks. Some studies (Yoon et al. 2017 Hung al. 2019 Yang et al. 2021) combined the strategies of weight selection, model pruning and expansion.
Neural Collapse. The NC phenomenon during the terminal state of DNN training was first discovered in (Papyan et al. 2020), which has further motivated a lot of studies on understanding NC. For example, NC
has been investigated under different settings, e.g., imbalanced learning (Yang al. 2022 Xie al. 2023)
and also been applied in different domains, e.g.. semantic segmentation (Shen et al. 2025 Xie et al. 2025
and language models (Wu & Papyan, 2024 Zhu et al.) 2024). Very recently, several studies have emerged to leverage NC to facilitate better CL algorithm designs. (Yang et al., 2023a) first proposed to
fixed
global ETF target for feature-classifier aligmment in few-shot CL with an ETF classifier, whereas (Yang et al. 2023b) applied the same idea to more general CL setups. (Dang et al. 2025) further integrated this Idea with contrastive learning based CL. However, as mentioned earlier, the reliance on a fixed global ETF suffers from critical drawbacks, which we aim to address in this work.
15

---

Published as conference paper ICLR 2026
B EXPERIMENT DETAILS
B.I DATASETS
Based CIFAR-10 dataset (Krizhessky al. 2009), this benchmark partitions 10 classes into 5 sequential tasks (2 classes per task), and each class has 5000 and 1000 32 x 32 images each for training and testing, respectively;
Seq CIFAR-10:
Seq CIEAR-100:
Constructed from CIFAR- 100 (Krizhevsky et al. 2009). comprises tasks with 10 classes per task, and each class has 500 and 100 32 x 32 images each for training and testing, respectively;
Seq Nett Adapted from the TinyImageNet dataset (Le Yang 2015), this benchmark divides 200 classes into 10 sequential tasks (20 classes per task), and each class has 500 and 50 64 x 64 images each for training and testing, respectively.
B.2 IMPLEMENTATION DETAILS
All experiments conducted single RTX 4090 GPU. For all datasets, employ modified ResNet18 network architecture (He et al., 2016), where the kernel size of the first convolutional layer is modified from 7x7 to 3x3, and the stride is changed from 2 to The batch size number of training epochs is set to 50 for Sequential CIFAR-10 and Sequential CIFAR-100 and 100 for Sequential TinylmageNer. For the buffer size, we use 200 and 500 in the main comparison table.
32 across all experiments. The
For ProNC, we consider the following hyperparameters: learning rate (7)), weight of the alignment loss (A1). weight of the distillation loss (A2), momentum (mom), and weight decay (wd). The hyperparameters are selected through grid search. The chosen hyperparameters are presented in Table 7 and their corresponding search spaces are provided in Table 8
Table 7: Hyperparameters of ProNC
Method
Buffer size
Dataset
Hyperparameter
Seq-CIFAR-10
1): 0.01, A 13, X2: 90, mom: 0, wd: 0
200
Seq Beg-CIFAR-100
1): 0.03, A1 18, X2: 170, mom: 0, wdl 0
r/: 0.03, A1:20, ;2:00;; 165, mom: 0, wd: 0
Ours
Seq- Beq-CIFAR-10
17: 0.01, A1:12, >2: 80 mom: 0 wd:
500
Seq-CIFAR- 100
17: 0.03, A1:18 A2: 170, mom: 0, wd:
Tin-Tiny-mageeNet
T/: 0.03, A1:20, A2: 200,
mom:: 0, wdl: 0
Table 8: Search Spaces for Hyperparameters
Hyperparameter
Values
"
(0.01, 0.03, 0.05.0.1, 0.3
A1
{5, 10, 12, 13, 18, 20}
{50, 75, 80, 90, 165, 170, 200}
mom wd
(0.0.9)
(0, 10 1,5 10
Our code is implemented based on the Continual Learning platform Mammoth (Boschini et al. 2022), which is also provided in the supplementary materials.
16

---

Published as conference paper ICLR 2026
C MORE RESULTS
C.I FINAL AVERAGE ACCURACY WITH ERROR BARS
Table 9: Final average accuracies comparison under various setups. All results are averaged over multiple runs.
Buffer
Method
Seq
Seg 100
Soy-Tiay ImageNet
Class-Il.
Task-II
Cluss-ll.
Tink-1l.
Class-ll.
Task-II.
ER
[Riemer et al 2019
44.70
1.86
91.19
0.94
21.78 + 0-48
60:19
1.01
8-49 0.16
3s.17 + 2,00
iCaRL
(Kebuuh
2019
19.02
3.20
88.89
2.13
28
0.91
51.43
1.47
7.53
7.53:
26.19 1.47
GBM
Kanvaao
25
0.66
68S
A4junar or
39.07
5.50
+8.8
18.42
05.38
1.34
200
DER++
DER
Buzzeg o1
7020
61.93 1.79
64.88 1.17
0.60 91.92
9.91
25.13
31.23
0.51
1.38
66.80
63.09
1.09
11.87 1:377:000 0.78
11.34 +: 1.17
43.06 1.16
10.22 1: 0.67
Co
0.00
39.08
0.66
91.36
0.08
19.19
0.53
1.02 53.93
12.98
0.10
37.32
1.87
MNC
(Duns
2025
52.20 1.56
85.94
0:22
15.81
0.48
43.91 0.76
10.57 + 1.66
32. 78
3.54
STAR (Escanarr
62.10
18.29
13.06
3.23
CSR
A1/6
37
1.57
69.22
3.03
20.06
66 66.99 0.35
18.14 3.10
45.04
NCT
Yiing 4l
202.6
51.50 0.41
0.446 96.86
80.63
26.398
0.57
0.17 76.75
10:95-
1.46
52.71
4.12
Ours
0.15 65.58
0.
42.99
0.85
87. 0.73
27.
1.00
69.09
0.65
FR
Riemer
2019
2.48
1.52
48.64
iCaR1
Kchullne st
707
47.55 3.95
2.62 92.16
88.22
33.25
58. 08.16:21
9:38 1.58
31.55 3.27
GBM
(Lape- P9r X Kinzalis
2017
26.2
1.26
0.64
20.54
0.45
66:31
0.86
GSSS
Aurmni er mi 2019
40.73 4.788
02 1.57
21.92
01.34
60.28 1.18
DER+
Mazzess orate RUZO
72.70 1.36
0.30
35.20
74.77 0.31
19.38
1.41
51.00
Co
Chi e Hill 20MI
61.78 4.22
89.51
2:45
20.64
1.42
0.19 02.32
18. 0.54
30.74 1: 1.24
CILA
67 02 0.33
95.29
0.24
31.27
0.17
68
18.09
0.19
49.19 1: 0.91
1.56
22.20
0.29
0.01
36.32
STAR
(Eskandar er 1012
00.15 3.53
96.36
0.37
24.45
1.70
74.06 1.63
15.19 2.61
55.06
2.07
CSRc
Troy 20 ai 2025
40.82 4.09
6.92 73.30
39.22
1.70
75.09
0.78
22
0.35
51.94 2: 0..2
NCT
Vinng
0.94
Ours
73.95 0.68
0.44
0.43
20.06-1
C.2 FINAL FORGETTING WITH ERROR BARS
Table 10: Final forgetting comparison under various setups, All results are averaged over multiple runs.
Buffer
Method
Seq-Cifar-10
Seq-Cifar-101
Seg--Tnny-Imagehe
Class-Il
Task-11.
Class-1l.
Task-11.
Class-Il.
Task-11.
et
00131111
0.51
10.47
(C
2017
23
25.34
1.64
47,20
1.23
36,20
1.85
31,06 1.91
42.47
2:47
GEM
Lapez
Kenzaid
2017
50.36 6.22
9.57 2: 2.06
77.40
1.09
29.59
1.66
2019
72.48
2.05
77.62
0.76
32.81
1.75
200
DER
20p0
32
2.32
5.16-000 0.21
1.52 01.90
23.91
0.55
73.47 1.23
39,02
0.43
Co-L
30.17 8.57
2,91 5.25
61.11
0.69
36.81
0.63
62.04 0.28
10.75 3.56
CILA
MNC
Dang et al
33.
1.65
4.90 0.15
62.51
0.40
39.79
0.98
50.68
2.02
45.10
1.57
STAR
21.7 3.16
6.23 2.06
68:40
8.02
27.84
8.01
67 43
1.48
34.78
2.08
[oog M0.
NCT
Yang 21
20230
22.48
19.50 12.75
1.41 1.09
27.40
L5
0.07 4.40 0.82
4.79
4.47 42.81
19.83
15.86
0.85
Odrs
4.71
0.65 0.08
36.07
0.51
0.56
9.42
0.58
ER
[Riemer
2019
43.22
3.50 0.53
73,06
0.78
16.23
1.06
0.54 37.30
75.21
30.73 0.82
2010
3.97
40.99
27.90
GEM [opgz For & Kunzaii 201
78.
6.58
5.00 0.96
0.78 74.12
71.34
20.14
1.13
G5S
2011
59.
4.00
1.56 6.37
0.42
57
1.34
21.02
DER
Mazzeea et 2010
22.38
4.41
4.66 1.15
2.19 48.60
49.18
15.75
0.48
58.75
1.93
25.47 1.00
Co
17.79 3.36
1.00 2.65
1.34
23.47
0.27
40.64
1.14
13.80
2.10
M
Dane el al 2013
27 88
2.75
0.42 3.15
46.00
0.58
24.29
0.50
44.96
0.09
39.00
0.17
STAR
(P
18.50
2.51
2.38
0.34
53.11
1.57
15.22
0.96
63.42 3.19
15.19
2.61
CSRe
29.18 6.91
16.99 3.19
47.03
2.57
15.55
0.76
1.87 50.01
28.20
0.36
NCT
Ming 01
13.82
3.64
3.84
24.37
0.75
30.90
0.87
Ours
26.75 1:
0.14
24.32 11 0.24
4.35
0.17
38.58 0.3
9.52 0.96
17

---

Published as conference paper ICLR 2026
C.3 TIME COST
To evaluate the computation efficiency of our approach, we summa- rize in Table the average training time per epoch and per task for all approaches. It can be seen that our approach is more efficient than all considered replay-based approaches except iCaRI. iCaRL. achieves higher efficiency by employing binary cross-entropy loss, which enjoy a constant time complexity per class (O(1), compared the linear class-dependent complexity (O(C), C' being the total number classes) of conventional cross-entropy loss. Meanwhile, the per-epoch training time of approach is comparable with the recent contrastive learning based approaches. In summary, by lever- aging NC in principled way, our approach not only significantly outperforms baseline approaches by setting new SOTA performance, but also maintains high efficiency due the simplicity of our framework and robustness buffer sizes. This highlights the great potential of our approach in practical resource-limite CL scenarios.
Table 11: Average training time per
epoch and per task (seconds).
Method
Epoch
Task
ER Riemer et 2019
3.03
151,48
iCaRt. (iceum m0 2017
1.600
84.77
GEM
Lowe
2017
3.26
162.64
DER
[R0ZZe orale 20:0
3.07
153.21
DER
B02205 et mu20
5.03
253.34
Co'L
CIA
2.77
138.59
MNC (U 1025
2.82
141,01
CSRal
Toin Mi di pi2a
8-42
433.35
NCT
(Ying et 204430
3.41
172.02
Ours
2,91
146.71
D PROOF OF THEOREM
Based the definition of ETF, we know that E can be expressed as
K IK
1
1 1 k.
(6)
K1
K1
where U RdxK denotes the corresponding orthogonal matrix.
K1-1 Mr, (IK, Ki 1K, 1k.) have
Because U
K2
M K1 U
[K,
1
1x, K,
K1-1
K1
This is true since Ix: its pseudoinverse. It is clear that
1 ki 1,, is an orthogonal projection matrix with rank K 1 and also equal to
Ki
MR, E"
K1 U
IK
1
1x, 1k:)
1 K1 U IK
1k, kk
K1
K1
V Ki
Kι
K1 - (U' -U") Ik.
Ki
1 Ki
1
K1-1 a scalar and Ix, Ki 1x, ; 1, is fixed projection matrix, finding E* minimize |[K,, | is equivalent to finding an orthogonal matrix U* that minimizes ||U' U
Since
K1
To this end, the following lemma (Zou al. 2006) characterizes the nearest orthogonal matrix to any real matrix.
Lemma (Nearest Orthogonal Matrix via SVD). Let A E mm be a real matrix with Singular Value Decomposition (SVD) A WEVT, where W m and V pax are orthogonal matrices, and R is a diagonal matrix of singular values with s = min(m,n). The nearest orthogonal matrix Q E Rmxn A under Frobenius is:
WV
18

---

Published as conference paper ICLR 2026
Therefore, given the SVD of U' as WEVT, the orthogonal matrix closest to U can be represented as U-= = WV which will lead to an ETF matrix
E
K1
WVT
[K,
1k, 1k,)
V K
Ki
E LLM USAGE
During this project, we did not use LLM.
19


Reviews:
[{"review_id": "c8bW3Az4tz", "reviewer": "Reviewer_cHhJ", "summary": "This paper extends the Neural Collapse Terminus (NCT; Yang et al., 2023b, arXiv) and the ICLR 2023 work by Yang et al. (2023a), aiming to address their limitations when applying the neural collapse (NC) phenomenon to continual learning (CL). In NC-based CL, the target ETF (simplex equiangular tight frame) is typically predefined, which requires knowledge of the total number of classes and may degrade discriminability when the class number becomes large. To overcome this, the paper proposes Progressive Neural Collapse (ProNC), a method that dynamically adjusts and expands the target ETF throughout the CL process, building on the NCT framework. Experimental results show that ProNC achieves consistent and significant improvements over existing CL baselines, particularly compared with NCT. Moreover, the proposed regularization approach proves beneficial even when combined with other CL frameworks, as confirmed through ablation studies.", "questions": "1. Could the authors elaborate more explicitly on how the proposed ProNC framework differs technically from NCT (Yang et al., 2023b)?\nBeyond the dynamic ETF expansion, are there any additional algorithmic or theoretical components that are genuinely new rather than adapted from NCT?\n2. Some reproduced results (e.g., ER on Seq-CIFAR-100 Task-IL) are considerably lower than in prior works such as GPM (ICLR 2021).\nCould the authors detail the reproduction settings (e.g., data augmentation, optimizer, training epochs) and justify whether these differences could account for the gap?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a clear exposition of the background NC theory and explains its own contributions in a well-structured manner.\n2. The proposed method is well-motivated by the identified limitations of prior NC-based CL works, and the use of Theorem 1 introduces a moderately novel and theoretically grounded component.\n3. Comprehensive experiments demonstrate consistent and noticeable gains over a range of baselines, supporting the empirical validity of the approach.", "weaknesses": "1. The technical novelty remains limited compared with the preliminary works (Yang et al., 2023a,b). The paper reads largely as a continuation of this prior line of research, where NC-based CL formulations have already been thoroughly explored.\n2. The second main contribution—the ProNC-based CL framework—largely mirrors the loss formulation of NCT (Yang et al., 2023b). While Section 3.1 introduces a genuinely new idea, Section 3.2 appears nearly identical to the corresponding part in NCT.\n3. Some reproduced baselines yield noticeably lower accuracies than those reported in their original papers. For example, ER on Task-IL with Seq-CIFAR-100 using ResNet-18 has been reported above 70% (e.g., GPM, ICLR 2021), yet only 60.19% here, raising concerns about the faithfulness of baseline reproduction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jhwcyoYvz4", "reviewer": "Reviewer_hVBn", "summary": "This paper proposes Progressive Neural Collapse (ProNC), a continual learning framework inspired by the Neural Collapse phenomenon.\nProNC progressively constructs and expands an Equiangular Tight Frame (ETF) to align class features across tasks.\nAfter each task, it estimates the “closest ETF” from class means and expands the basis for new classes using Gram–Schmidt orthogonalization, keeping all class representations approximately equiangular.\nThe model jointly optimizes cross-entropy, feature-alignment, and distillation losses.\nExperiments on Seq-CIFAR-10/100 and Seq-TinyImageNet under both Class-IL and Task-IL show consistent gains over strong baselines (DER++, NCT, Co2L, STAR) with good efficiency and generalization.", "questions": "When the number of tasks grows large, does ETF orthogonality noticeably degrade? Would periodic re-fitting help?\nCan ProNC remain stable with higher-dimensional embeddings (e.g., ViT features) or larger datasets such as ImageNet-100?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Grounded in Neural Collapse geometry, offering an interpretable view of feature alignment in continual learning. Achieves strong results without complex contrastive or generative modules.\n2. Works as a plug-in regularizer across different CL frameworks (e.g., ER, iCaRL, DER++).", "weaknesses": "1.\tThe method assumes clear task segmentation (task-aware setting); its applicability to task-free or online CL remains untested.\n2.\tAs the ETF expands over many tasks, orthogonality may gradually degrade; this possible effect is not analyzed experimentally.\n3.\tGram–Schmidt expansion could become unstable when the number of classes approaches the embedding dimension; only small-scale datasets and ResNet-18 (d ≤ 512) were tested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wB1NuCxuIx", "reviewer": "Reviewer_fz6X", "summary": "The paper introduces Progressive Neural Collapse (ProNC), a continual learning framework inspired by *Neural Collapse*. This phenomenon describes how deep networks, at convergence, produce class features that collapse to a single point for each class creating simplex equiangular tight frames (ETFs), creating orthogonal class prototypes and maximizing separation between classes. ProNC proposes to progressively expand the ETF as new tasks arrive, maintaining geometric consistency and feature separability without prior knowledge of all class counts. The framework integrates three loss components: cross-entropy for supervision, an alignment loss enforcing ETF-based feature geometry, and a distillation loss preserving past representations (to mitigate forgetting). The method is evaluated on standard benchmarks (Seq-CIFAR10/100 and Seq-TinyImageNet) under Class-IL and Task-IL setups, reporting substantial performance gains across datasets and memory budgets.", "questions": "While initializing the ETF at the end of the first task close to the optimal solution of the optimization problem is an interesting idea, for subsequent tasks the ETFs are initialized orthogonally to the previous ones but otherwise randomly, without any guiding heuristic. Do you have any thoughts on how this initialization process could be improved for later tasks?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The idea of progressively adapting the ETF target during continual learning without knowing the number of total classes in advance is novel and addresses the shortcomings of fixed ETF methods for CL.\n\nThe paper is built on a convincing motivation. The reasoning is coherent and carefully developed, making the overall argument both logical and easy to follow.", "weaknesses": "### **Major Weaknesses**\n\n1. **Questionable baseline performance values**: in Table 1, several baseline results (Co$^2$L, CILA, MNC$^3$L, STAR) are notably lower than those reported in their original papers (where they surpass the results from the proposed ProNC). This discrepancy indicates possible reproduction or configuration issues, undermining the fairness and credibility of the comparison and invalidating the paper’s main “state-of-the-art” claim.\n\n2. **Missing baselines**: some important methodologies are missing in the experimental evaluation:\n   - XDER (Boschini et al., TPAMI 2022, also cited by the authors);\n   - GCR (Tiwari et al., CVPR 2022, also cited by the authors);\n   - LODE (Liang and Yi, NeurIPS 2023).\n\n3. **Limited novelty relative to NCT**: NCT (Yang et al., 2023, also cited by the authors and reported in the main table) already introduced ETFs in class-incremental learning. As far as I can tell, the main innovation of the proposed method compared to NCT is the removal of the \"known number of classes in advance\" assumption. While this is an interesting modification, it alone does not seem sufficient to constitute a truly novel methodology.\n\n4. **Limited significance of ablation studies**: most ablation experiments in Figure 2 examine only a very narrow range of values (from about 2e-1 at best to 1e-2 in some cases) and consider just one dataset. Such a limited scope restricts the interpretability of the results, as the observed variations may not represent a consistent behavior across the full cosine similarity range (–1 to 1).\n\n### **Minor Weaknesses**\n5. Clarity issue in Section 3.1 (point 2): the explanation of the proposed methodology in this part is somewhat convoluted and would benefit from clearer exposition to improve readability and understanding.\n\n6. Missing no-buffer baseline for ProNC: Table 1 does not report ProNC’s performance without a replay buffer (which is reported in the text only). Also inserting it in the main table would increase clarity and allow a fair comparison w.r.t. replay-free methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)