YoooP: You Only Optimize One Prototype per Class for Non-Exemplar Incremental Learning
Authors: Jiangtao Kong
Zhenyu Zong
Tianyi Zhou
Huajie Shao
Venue: Accepted by TMLR ()

Abstract: Incremental learning (IL) usually addresses catastrophic forgetting of old tasks when learning new tasks by replaying old tasks' raw data stored in memory, which can be limited by its size and the risk of privacy leakage. Recent non-exemplar IL methods store class centroids as prototypes and perturb them with high-dimensional Gaussian noise to generate synthetic data for replaying. Unfortunately, this approach has two major limitations. First, the boundary between embedding clusters around prototypes of different classes might be unclear, leading to serious catastrophic forgetting. Second, directly applying high-dimensional Gaussian noise produces nearly identical synthetic samples that fail to preserve the true data distribution, ultimately degrading performance. In this paper, we propose YoooP, a novel exemplar-free IL approach that can greatly outperform previous methods by only storing and replaying one prototype per class even without synthetic data replay. Instead of merely storing class centroids, YoooP optimizes each prototype by (1) shifting it to high-density regions within each class using an attentional mean-shift algorithm, and (2) optimizing its cosine similarity with class-specific embeddings to form compact, well-separated clusters. As a result, replaying only the optimized prototypes effectively reduces inter-class interference and maintains clear decision boundaries. Furthermore, we extend YoooP to YoooP+ by synthesizing replay data preserving the angular distribution between each class prototype and the class's real data in history, which cannot be obtained by high-dimensional Gaussian perturbation. YoooP+ effectively stabilizes and further improves YoooP without storing real data. Extensive experiments demonstrate the superiority of YoooP/YoooP+ over non-exemplar baselines in terms of different metrics. The code is released at https://github.com/Snowball0823/YoooP.git.

Published in Transactions on Machine Learning Research (07 (07/2025)
YoooP: You Only Optimize One Prototype per Class for Non-Exemplar Incremental Learning
Jiantto Kong
jkong01 @wwm. edu
Deportment of Computer Science William & Mary
Zhenyu Zong
zzong edu
Department of Computer Science William & Mory
Tianyi Zhou
Liaryi'@und edu
Department of Computer Science University of Maryland
Huajie Shao
hshao @wmn. edu
Deportment of Computer Science William & Mary
Reviewed on OpenReview: https: // openrevie.. net/ forum? Woeee 447eee6l..hhhhhhhtte24444
Abstract
Incremental learning (IL) usually addresses catastrophic forgetting of old tasks when learning new tasks by replaying old tasks' raw data stored memory, which can be limited by its size and the risk of privacy leakage. Recent non-exemplar IL methods store class centroids prototypes and perturb them with high-dimensional Gaussian noise to generate synthetic data for replaying. Unfortunately, this approach has two major limitations. First, the boundary between embedding clusters around prototypes of different classes might be unclear, leading to serious catastrophic forgetting. Second, directly applying high- dimensional Gaussian noise produces nearly identical synthetic samples that fail to preserve the true data distribution, ultimately degrading performance. In this paper, we propose YoooP, novel exemplar-free IL approach that can greatly outperform previous methods by only storing and replaying one prototype per class Instead of merely storing class centroids, YoooP optimizes each prototype by (1) shifting it to high-density regions within cach class using an attentional mean-shift algorithm, and (2) optimizing its cosine similarity with class-specific embeddings to form compact, well- separated clusters. As result, replaying only the optimized prototypes effectively reduces inter-class interference and maintains clear decision boundaries. Furthermore, we extend YoooP to YoooP+ by synthesizing replay data preserving the angular distribution between each class prototype and the class's real data in history, which cannot be obtained by high-dimensional Gaussian perturbation. YoooP+ effectively stabilizes and further improves YoooP without storing real data. Extensive experiments demonstrate the superiority of YoooP YoooP/Yoooll YYooo++ over non- oon-exemplar baselines in terms of different metrics. The code is released at httts://githu/: wtt./S/hoooaaa11022//oooooo
without synthetic data replay.
1 Introduction
Catastrophic forgetting McCloskey & Cohen 1989) refers deep neural networks forgetting the acquired knowledge from the previous tasks disastrously while learning the current task. This stands in stark contrast.

---

Published in Transactions on Machine Learning Research (07/2025)
to human learning, where new knowledge is integrated without erasing prior understanding. To bridge this
gap, incremental learning (TL.) Gepperth & Hammer (2016); Wu et al. (2019); Douillard et al. (2022); Wang
lt al. (2022a); Goswami et al. (2024) has emerged as a paradigm that enables Al systems to continuously
learn from evolving data.
In the past few years, a variety of methods Roady et al. (2020); Cong et al. (2200 Wang et al. 20021 Xue
let al.) (2022) have been proposed to mitigate catastrophic forgetting in IL. In this work, we are interested in a very challenging scenario, called class-incremental learning (CIL). CIL is particularly challenging because it requires the model to recognize all learned classes without any task identifier during inference. CIL is especially susceptible to catastrophic forgetting duc to overlapping feature representations between old and new lasks Zhu et al. (2021b). To address this issue, many prior studies have adopted exemplar-based
: Wu et al. (2019); Zhao et al. (0000;; Wang et al. (202bb) that store a subset of old class samples a memory buffer for replay. However, these methods face inherent limitations
approaches Rebufhi et al.] (2017::
related to memory capacity and privacy. Thus, non-eremplar-be based methods Li Hoiem (2017): Lopez-Paz &
Ranzato (2017); Mallya & Lazebnik (2018); Cong et. al. (2020); Xue et al. (2022) have been proposed, which avoid storing raw data by relying on regularization, parameter isolation, or generative models to mitigate
catastrophic forgetting. Unfortunately, solely applying regularization is often insufficient, parameter isolation
increases network size, and generative models can be unstable.
Recently, prototype-basee based methods Zhhu et al.] (2021b :): Petit et al.] (2023); Rypešć et al. (2024); Magistri al. (2024) have attracted attention in non-exemplar CIT. These approaches store sin- gle class-mean prototype as the class centroid for each old class and replay synthetic data augmented from these prototypes in future tasks. Notably, PASS Zhu ct al. (2021b) augments stored proto- types via high-dimensional Gaussian noise. Several follow-up works, such as EFC Magistri et al.] (2024) and AdaGauss Rypešé ct al. (2024), also rely Gaussian-based prototype augmentation by storing class means and covariances to sample synthetic fea- tures. However, our observations reveal that such an augmentation strategy may actually degrade pre- diction accuracy (Sec. 4.1). The underlying issue is twofold. First, simply generating class-mean pro- totypes without further optimization leads to dif- fuse clusters of embeddings with unclear decision boundaries between classes (Sec. 4.3 Consequently, the clusters around prototypes are less representa- live, resulting in serious catastrophic forgetting while training future tasks. Second, as shown in Fig. [a)- Prototype Augmentation, dircctly assuming the class feature distribution can be approxinated by Gaus- sians, and adding high-dimensional Gaussian noise prototypes yields nearly identical synthetic sam- ples that fail to capture the true distribution of old tasks' class embeddings, and decrease per performance.
Saant Pail
Mon
(II) Prototype Optimization
Original Distribution
skared Prenne
Riffert Dker burkky
Sharel Dinsteriis
Shewll musen
(II) Protstype pe Augmentation
(IV)
Prototype Augmentation
(a) Previous Methods
(b) Our Methods
Comparison of previous prototype-based (Ieft) methods with our YoooP/Yoooky (right). Pre- vious methods (c.g., PASS) typically store and average all class embeddings to form a class-mean prototype, which is then augmented inappropriately (e.g., by di- rectly adding high-dimensionall Gaussian noise). In contrast, YoooP adopts mini-batel attentional mean- shift method, which constructs represcntative proto- types from a small batch of embeddings. Prototype optimization further refines these prototypes to form a more compact feature space. YoooP+ further enhances performance by synthesizing data with stored original angular distribution of old tasks. Comparing (I) and (II) demonstrates the benefit of prototype optimization in forming compact feature spaces, while comparing (III and (IV) shows the advantage of our augmentation strategy in preserving realistic sample distributions.
Figure 1:
Motivated by these limitations, we propose to opti- mize prototype learning and develop a novel proto-
type augmentation strategy for CIT. Tn this work, we introduce YoooP, new non-exemplar CIT, method that stores and replays only one representative prototype per class without relying on synthetic data. The main challenge is obtaining well-separated protolypes that capture the essence of their respective classes. To achieve this, we introduce prototype optimization (Fig. I (II). First, we employ a hani-batch attentional
2

---

Published in Transactions on Machine Learning Research (07/2025)
mean shift-based method to shift each class prototype toward high-density regions of its corresponding
embeddings (the moving path in Fig. | (TT). Next, we optimize the angular distance between the protolype and the class-specific embeddings to form tight, compact clusters (comparing the (I) and (II) in Fig. [ As a result, each prototype becomes highly representative of its class. Consequently, only replaying the prototype can effectively reduce inter-class interference and maintain clear decision boundaries. Building on YoooP, we further develop YoooP+, which extends our approach with a novel prototype augmentation technique. Instead of directly adding high-dimensional Gaussian noise, which implicitly assumes that class feature distributions follow Gaussian patterns, YoooP+ synthesizes data by combining rotation matrix in high-dimensional space with the stored angular (cosine similarity) distribution between each class prototype (IV), this strategy produces synthetic data that more
and its corresponding real features. As shown in Fig. [ laithfully reflects the true embedding distribution of old classes and yields higher-quality samples.
Our contributions are four-fold: 1) We propose YoooP; a novel non-exemplar CTL algorithm that stores and replays a single prototype per class without synthetic data. 2) To the best of our knowlodge, we are the first to explore protolype optimization in CIL, which effectively reduces inter-class interference and maintains clear decision boundaries. 3) We extend YoooP to YoooP+, which develops new prototype augmentation technique that synthesizes high-quality data reflective of the original distribution. 4) Extensive evaluations on multiple benchmarks demonstrate that both YoooP and YoooP+ significantly outperform non-exemplar baselines in terms of average incremental accuracy, average accuracy, and average forgetting.
2 Related Work
Regulariization-based method. This method aims to alleviate catastrophic forgetting by introducing additional regularization terms correct the gradicnts and protoct the old knowlodge learned by the model Li & Hoiem (2017; Rannen et al. (2017;; ; Kirkpatrick et al. [2017); | Lee et al.. (2017) : Liu al. (2018); Masana et al. (0022). Existing works mainly adopt weight regularization to reduce the impact of learning new knowledge on the weights that are important for old tasks. However, it is very hard to design reasonable and reliable metrics to measure the importance of model parameters. Thus, solely using regularization-bant
methods is always insufficient.
Parameters isolation-based method. This line of work can be divided into dynamic network expansion and static network expansion. Dynamic network expansion methods adopt individual parameters for each task, they need a large memory to store the extended network for each previous task during training Yoon et al, (2017): : Ostapenko et al.) (2019); Yan et al. (2021); Xue et al. (2022). Conversely, static network expansion approaches [Serra et al. (2018); Mallya & Lazebuik (2018); Mallya el al. (2188) ;); Zhuu et al.] (2022) dynamically
expand the network if its capacity is not large enough for new tasks, and then adapt the expanded parameters
into the original network. Those methods can achiove remarkable performance, but they are not applicable
to large nunber of lasks.
Data replay-based method. This solution Wu et al. (2018); Rostami et al. (2019); Cong et al. 2020))
mainly employs deep generative models to generate synthetic samples of old classes in order to mitigate
privacy leakage. Most existing works Shin et al.] (2017): Rios & Itti (018);; Ostapenko et al. 2019 Lesort
et al. (2019) focus on Variational Autoencoder (VAF) and Generative Adversarial Network (GAN). However,
these methods suffer from the instability of generative models and inefficient training for complex datasets.
Prototype-based method. Recent works Zhu et al. 2021bba Petit et al. (2023): Goswami et al. 2023
Rypość et al. (2024): Magistri et al. (2024) avoid generating pscudo samples by storing class-representative prototypes and adapting prototype- based strategies to mitigate catastrophic forgetting. A key line of work, including PASS Zhu et al. 22021b). IL2A Zhu et al. (021a), FeTrIL Petit et al. (2023). AdaGauss [Rypešć et al. (2024), and EFC Magistri et al. (024)) uses these prototypes to generate synthetic samples for replay-based training. PASS directly adds high-dimensions incorporate adaplive covariance modeling or elastic attraction to improve sample realism. However, all these methods rely on unoptimized class-mean mean centroids and Gaussian assumptions, which may lead to unrealistic or overlapping synthetic samples that poorly capture class semantics, espocially in high-dimensional feature spaces. In contrast, FeCAM Goswami et al. (2023) does not generate synthetic data, but instead uses a Mahhalanohis-disttane---sss classifier built. upon saved means and covariances. Yet, this method freezes the
Gaussian noise to stored prototypes, while AdaGauss and EFC
3

---

Published in Transactions on Machine Learning Research (07 (225))
Memory of class prototypes
(b)
Prototype Augmentation
Prototype
Augmestation
prototypes: (Pa. & Cra)
(b)
Task 1:
G
(G,
EP.. X
Ps 4
Task
C2
YoooP+
Task t- 1:
G--1
C2
No
CLe-2
lec (0.w)
(P
Creise statlaity Teaagrame
(PL)
Store Protocypes
x = R(P.. x
Go
Lr((,w)
Prototype Optimization
(ep
Classifier Gl::w)
Protetype Optimization
F(:0)
(7 k)
YoanP: min (0. w) (0) + (0..
GL
Sk.1
4x,2
Current Data & F(x,
Features
YounP:: min L, w) = hep (0) + 1ec. (9.
(D))
a
Figure 2: Framework the proposed YoooP and YoooP+. YoooP only needs to replay one stored protolype for each class while YoooP+ trained on synthetic data generated from stored prototypes. (a) Prototype optimization aims to learn compact feature space and obtain representative prototype for each class. (b) Protolype augmentation aims to generate synthetic data of old classes from stored protolypes and angular distribution using a m-dimensional space rotation matrix.
feature extractor after the first. task and thus struggles under large task shifts or limited prior knowledge, due to its inability to adapt learned features. Another recent study Goswami et al. (0224) has leveraged class-mean prototypes as classifiers, using a Nearest Class Mcan (NCM) approach to correct prototype drift during sequential training. Nevertheless, compared to standard gradient-based classifiers, NCM is limited in its ability to learn complex decision boundaries when task distributions shift significantly. In this work, we propose prototype optimization during training to obtain compact and discriminative clusters. On top of that, we introduce an ingular-based prototype augmentation strategy that avoids unrealistic sampling in Euclidean space, thereby generating high-quality synthetic data and effectively mitigating catastrophic forgetting.
3 Proposed Method
In this section, we first describe YoooP, which optimizes the prototype for each class using an attentional mean-shift method. To further improve prediction accuracy, we extend YoooP to YoooP+ by generating synthotic data from the stored prototypes.
Problem Description. Given a sequence of lasks, each associated with a set of classes C, and a training set Dt e (I,, vi)i Tht with y Ct, caassinnremmenal learning (CIL) aims train a model f(x; [0, w]) G(F(x;)); that predicts probabilities for all classes CI. & U Cs without catastrophic forgetting. The model consists of a feature extractor F(: 0), which produces compatt representations, and classifier G(; w), with predictions given by (aftmax(((F(x;))))) w)). Since the parameters f and w are updated solely using the current task's data, the model typically suffers from catastrophic forgetting. To address this challenge, we propose YoooP and its extension Y to preserve clear class boundaries as tasks are learned sequentially.
3.1 YoooP
YoooP comprises two main components: (i) prototype optimization learn a compact feature space and obtain representative prototype pe for each class, and (ii) new task learning with prototype replay.
4

---

Published in Transactions on Machine Learning Research (07 (07/2025)
For (i), we propose mini-batch attentional mean-shift- method to shift each class prototype toward high-density regions of its embeddings. Then, we optimize the angular distance between the prototype and the class-specific embeddings to form a tight, compact cluster. As a result, each prototype becomes highly representative. Consequently, replaying only the optimized prototype effectively reduces inter-class interference and maintains clear decision boundaries.
For (ii), when learning new task, we augment its training set with the stored prototypes from previous tasks. In the YoooP, replaying only the memorized prototypes of old classes can efficiently retain clear boundaries between classes and mitigate catastrophic forgetting.
3.1.1 Prototype Optimization
To obtain a representative prototype without storing all sample embeddings (unlike the traditional class-mean prototype), we propose mini-batch attentional mean-shift- based method. Specifically, for task-t with classes Cr, and for each class & E Cr, we construct graph of sample representations z; F(2;; 0) connected to the prototype Pk. We then shift pk toward a high-density region by moving it toward a weighted average of the normalized representations of all samples in class-k and subsequently normalizing phs i.c.,
Pk (1-x)pk +A
Σ ORS ||:|||2
Pk +-
Pk
(1)
1pxl22"
#Eine]:gumk
where À controls the step size of the mean-shift and n, is the size of the training set for task-t. Unlike the original mean-shift algorithm, the weights agss are determined by learnable dot-product attention between each sample 2 and the prototype Ph. i.e.,
(2 Pk)
- (āx)) āg
[ā.,1 ag.m. āR,i = c(2s.
(2)
| Ilpall2"
In practice, when ny is large, we apply a mini-batch version of Eq.. over multiple steps, replacing i E [m+]] with E B, where B is a mini-batch.
After obtaining a representative prototype for each class, we optimize the similarity between class-specific embeddings and their prototype, form compact cluster of each class in the feature space. To achieve that, We train the representation model F(;;0) to produce 2; = F( 0) such that each sample is close to its class prototype and distant from other classes' prototypes. This is achieved by minimizing the loss as follows,
1 111 Σ
(0)
Σ REC, (Ein.]:a.=
e ((c(2.,)))
(3)
C& n4
where ((:))) :) is a cross-entropy loss, in which the first argument is the logits defined by a similarity measure c).,-), and the second argument k is the ground truth label. Optimizing this loss ensures that embeddings concentrate around their respective prototype, forming compact clusters. Thus, replaying only one prototype per class effectively reduces harmful inter-class interference while training future tasks.
3.1.2 New Task Learning with Prototype Replay
To mitigate catastrophic forgetting of previous tasks' classes C1:-1, YoooP replays stored class prototypes during new task training. Specifically, we augment the training set for task t with prototypes from previous classes C1+-1. The classification objective for all classes Ci. is,
[0,
Σ Σ
A(c(E, ([c(2,,0;)) (y)terr CCa--ll REC
1
Σ
e((o(pe (i wy k), (4)
C.. T4 KEC. iE(ne):us=
Furthermore, to preserve the performance of learned prototypes while training future tasks, it is necessary to retain the performance of extractor F(:;0) on previous tasks. Following previous work Hou et al. (001 Zhu
et al. 2021b), we employ knowledge distillation (KD) Hou et al. (019)) as follows,
(0)
A 1
Σ ne sE
[F(;;; 0) F(.;0,1))
(5)
5

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Thus, the overall training objective at task t is,
YoooP: : min (0, w) = (0) + L,,c(0, +y (.))
(6)
0,a0
where y is the weight of KD loss.
In summary, (0) shifts the current task's s embeddings toward their class protolype, forming compact clusters. (0, w) trains the model using both current task data and the replayed prototypes, while K (0) preserves the extractor's performance on previous tasks. Together, these objectives enable the model to learn new lasks without forgetting previous ones.
3.2 YoooP+
Although protolype-only replay in YoooP is highly effective in mitigaling calastrophic forgetting, il is still insufficient to reflect the true embedding distribution of old classes without replaying raw instances, Hence, we propose an extension, YoooP+, which replays synthetic data augmented from the stored prototypes.
3.2.1 Prototype Augmentation.
To generate high-quality synthetic data that matches the real embedding distribution of old classes, we propose novel prototype augmentation strategy- Our method draws synthetic data for each class from the real angular distribution between the class prototype and its specific embeddings. To simplify the augmentation, we first normalize each prototype to unit vector, generate syn- thetic data in the normalized space, and then rotate the synthetic data back. As shown in Fig.
we sample cosine similarity values from the stored real angular distribution, P(ãg.i), which is repre- sented by a histogram with N, bins. These sampled cosine similaritics are used to generate synthetic data for each class. Consequently, the angular distribution between each class prototype and its synthetic data faithfully preserves P(ãz,s). In contrast, approaches like PASS add high-dimensional noise to saved proto- types, causing significant divergence from the actual angular distribution.
YoooP+
YOOOP+ Synthetic data
PASS
Real data
Prototype of class-k
PASS Synthetic data
Pv
Estanate from
Sample from histogram
Gaussian noise to Pr.
Estimate
able to synthesize data point z that has a similar angu- lar distance to the prototype Pk as for replaying. This leads to YoooP+ whose replay of each previous class is conducted on multiple synthetic data points instead of single prototype.
Specifically, by using the stored P(ãx,,);
Freq
Fruq of
cos 0 histugram
histugram
In particular, we firstly derive rotation ma- trix R(ps, u) that can recover Pk from a unit vec- tor u [1, ... 0]
Figure 3: Synthetic data distributions of YoooP+ and PASS with prototype augmentation: YoooP+ pre- serves the original angular distribution by a histogram with N₂ bins.
an unit m-sphere, i.e., Ps= = R(p,. u) x u. To synthesize a sample 2/ of class-k as a proxy to z; (a previously learned sample of class-k), we then randomly draw v; in the vicinity of u, i.e.
Us = [ā c., ,em]: āsss ~ P(aks)
(7)
To ensure |ell|22 1, draw e, (O, for E (2..... .m) at first and then rescale them by e, +- V ( tei) "rE Er - 6G. Thereby, we have uf v; مg.s whose distribution approximates the distribution of cosine similarity Cz.i between real sample z; and its associated class prototype Ps-
Next, we create 2f from vú As Ps= = R(ps, u) x u, we apply the same rotation matrix R(pk, u) in to obtain z/, i.e.,
2 = R(pk, u2 x vz-
(8)
6

---

Published in Transactions on Machine Learning Research (07 (07/2025)
This operation preserves the similarity between and 8s in the transformed space between pa and 2.. By sampling a synthetic data point z/ for each removed real sample zi, we construct replay dataset for all seen classes C1a-11
3.2.2 New Task Learning with Prototype Augmentation
After generating the synthetic data, Y learns new task task-t by replaying the syuthetic dataset Dj as follows,
D. k() lo 24 = R(pa, u) vi = [ā,, 62, ,, ,mll
(9)
The training objective for task-t with replayed synthetic data
(1,c+(0, - [r
Σ Th kEC, dE(me]/u
1
Σ
l(c(z, w).
(10)
D
(a.N)ED;
Overall, the training objective L00,w) w) of YoooP+ at task-t combines the prototype-earning loss, the synthetic-data replay angmented loss, and the KD loss as follows,
YoooP+: min (0, w) = (e, + (0, + (0).
(11)
8.w
3.3 Practical Improvement to YoooP +oooP+
While the KD loss constrains the feature extractor to maintain representational consistency with the previous model, it only applies to current-task data. As a result, cannot fully prevent parameter drift the absence of old-task examples. To complement this, we adopt two lightweight yet effective techniques at the parameter level to further mitigate forgetting.
Model Interpolation. We apply model interpolation to retain the knowledge of the previous model 0:-1 and avoid overfitting to the current task. Specifically, after learning task-t, we update the current 0, by the following interpolation between 8,-1 and or, i.c.,
O₂ 5 (1-1133)--- + 30.,
(12)
where 3 E [0, and we set 8 = 0.6 in experiments. Since 0: is mainly trained on task-t, such simple interpolation between f--1 and 0, leads to a more balanced performance on all tasks.
"Partial Freezing" of Classifier. Following Li Hoiem (2177), instead of completely freezing the classifier parameters for previously learned classes, Wk, E C1:--1, we scale down the gradients of these parameters by small factor Specifically, the gradient update is modified as follows,
Vm, w) a aV (0,,w); Vk C1.t-1
(13)
This strategy helps prevent significant drift of the classifier parameters for previously learned classes.
We provide the complete procedure of YoooP and YoooP+ in Algorithm I in Appendix A
4 Experiment
In this section, we first compare the proposed YoooP and YoooP+ with non-exemplar-basee baselines in three datasets. Then we assess the quality of synthetic data augmented from memorized prototypes. Lastly, we do ablation studies to explore the impact of different key components. We also explore the sensitivity of hyper-parameters in Appendix
Datasets. To better evaluate the performance of our proposed methods, we perform on three different scale datascts: CIFAR-100 Krizhevsky et al. (2009), TinyImageNet Yao & Miller (2015), and a subset of ImageNet- Russakovsky al. (2015) (Sub- Ssub-ImgeNNet). CIFAR- 100 contains 100 classes of images, which
include 50,000 training images and 10,000 test images, and the image size is 32 32. TinyImageNet contains
7

---

Published in Transactions on Machine Learning Research (07/2025)
200 classes of images, which include 100,000 training images and 10,000 test images, and the image size is 64 x 64. For the Sub-ImageNet, the detailed description and the results are shown in Appendix C
Experimental settings. We implement all experiments using PyTorrc Paszke 2019), and compare our methods with baselines provided by PyCIL Zhou et al. (2021), a popular toolbox for continual incremental learning (CTT.). Following prior works Zhu et al. (2021b a): we adopt ResNet-18 He et al. (2016) as the backbone network for all methods. For YoooP and YoooP+, we train with the SGD optimizer with an initial learning rate of 0.01, which is decayed by a factor of 0.1 every 20 epochs. Models are trained for 60 epochs per task using batch size of B 256. For knowledge distillation loss (Eq. 6 for YoooP and Eq. [1 for YoooP+), we set a large weight 7 30. Additionally, we save the cosine similarity distribution into a histogram consisting of N₂ = 100 bins within the interval [o, 1J for prototype analysis (Fig. 3 For evaluation, we consider two standard CIL settings: "zero-base," where the total classes are evenly split into multiple incremental phases (i.e. 5 or 10 phases) trained sequentially, and "half-base," " where half of the classes are learned initially and remaining classes are evenly incremented. Following previous practice Zhu et al.) (2021b), all classes in each dataset are arranged in a fixed random order, and reported results are averaged across three runs with fixed random seed for reproducibility. Additionally, we also perform 20 phases under different settings on TinylmageNet in Appendix D
Baselines. We compare our proposed YoooP and YoooP+ methods with representative non-exemplaa--bsse
methods, including LwF Li & Hoiem 2017 PASS Zhu et al. (2021b). SSRE Zhu et al. (2022), 1L2A Zhu
et al. (2021a), FeTrIL Petit et al. (2023), FeCAM Goswami et al. (2023), EFC Magistri et al. (0024), AdaGauss Rypešć et al. (2024), DS-AL Zhuang et al. (2024) and ADC Goswami et al. (2024). Notably, ADC and FeCAM employ an NCM classifier Rebuffi et al. (2017) while +rooP/YYooP++ and other baseline methods perform with a normal classifier with softmax. To further show the impressive performance of our proposed methods, we also compare the performance with some exemplar- based methods on TinylmageNet in Appendix
Protocol. We evaluate all methods using three commonly adopted metrics in IL: average incremental accuracy Rebuffi et al. (2017) (ATA), the average accuracy after training the last task Rebulfi et al. (2017) (AA), and average forgetting Chaudhry et al. (2018) (AF).
4.1 Main Results
In this section, we compare the performance of our proposed methods YoooP and YoooP+ across various datasots under different settings with various basclincs and discuss the results.
Evaluation on zero-base setting. Next, we evaluate the proposed methods against baseline approaches on CIFAR-100 and TinylmageNet datasets under zero-base settings with 5 and 10 phases. As shown in Tab. D our proposed YoooP consistently outperforms non-NCM basclines that rely on the standard classifier layer. Specifically, compared to PASS, which achieves the best results among non-NCM baselines, YoooP achieves accuracy improvements of 4.91% and 7.05% (ALA) and 8.47% and 8.97% (AA) under 5 and 10 phasos, respectively, on CIFAR-100. On TinyImageNet, the advantages of YoooP are even more pronouneed, exceeding PASS by 13.15% and 13.58% (AIA), and by 17.99% and 13.95% (AA) under the respective phases. This impressive performance is because YoooP leverages prototype optimization, enabling the model to learn more discriminative and compact representations. To further mitigate catastrophic forgetting, YoooP+ augments the training with synthetic data derived from these optimized prototypes. Consequently, YoooP+ surpasses PASS by 9.28%, 11.36% (ATA), and by 13.95%, 19.79% (AA) on CIFAR-100. On TinyTmageNet, the performance gap is even more notable: 14.81%, 16.32% for AIA, and 20.27%, 17.94% for AA under the respective phases. Additionally, YoooP surpasses ADC and FeCAM, which employ the Nearest Class Mean (NCM) classifier, on TinylmageNet under zero-base settings, and YoooP+ significantly surpasses ADC on both datasets under base settings. We also show the performance while training the model incrementally on each task in the upper row in Fig.
Evaluation on half-base setting. We first evaluate the proposed methods under half-base settings with 5 and 10 phases. Results shown in Tab. p indicate that YoooP/YoooPP consistently performs well across CIFAR-100 and TinylmageNet datasets. On CIFAR-100, YoooP+ achieves performance compa- rable to FeTrIIL DS-AL, and FeCAM, which attain strong results by training the model only on the
8

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Table 1: Average incremental accuracy (ATA) and average accuracy after training the last task (AA) of the proposed YoooP + and basclines on CIFAR-100 and TinylmageNet under zero-base setting with different phases."b0-10" means zero-base with 10 phases, "b0-5" means zero-base with 5 phases. (*) denotes methods using " "Nearest Class Mean" (NCM) classifier. Bold: the best among non-exemplar methods. Underline: the second best among non-exemplar methods.
Datasets
CIFAR-100
Tiny FinymageNeet
AIA
[%]+
AA
[6]t
AIA
[7]
AA
[%]t
Method
bo-5
b0-10
b0-5
b0-10
b0-5
b0-10
b0-5
b0-10
LwF
Li & Hoiem
2017
58.95
47.73
39.87
25.88
46.44
35.00
29.50
17.80
SSRE
Zhu et al.
(2022
58.05
46.58
40.99
29.75
47.13
38.54
29.90
22.78
IL2A
Zhu et al.
2002aa
59.91
42.92
43.69
27.21
42.16
33.72
25.67
22.46
FeTrIL
Petit et al.
[2023
61.41
48.61
44.54
31.07
44.55
36.51
28.23
20.59
DS-AL
Zhuang et al. (2024))
66.79
56.74
50.30
42.76
49.62
47.24
36.89
33.71
EFC
Magistri et al. (2024)
67.52
59.96
50.47
45.24
54.12
48.03
36.92
34.12
AdaGauss Rypešč et al.
(2024
68.42
59.13
56.02
46.32
55.58
50.51
41.33
36.52
FeCAM
Goswami et
2023
61.27
48.93
45.78
32.11
54.37
46.03
39.96
32.02
ADC Goswami al. 2024
68.60
61.93
57.49
47.00
60.09
53.51
47.46
37.32
PASS
Zhu et al. (001bb)
60.33
51.94
43.61
35.81
47.11
40.15
31.05
26.31
PASS
w/o Aug
59.02
55.47
45.19
42.41
50.25
42.71
36.27
30.17
YoooP (Ours)
65.24
58.99
52.08
44.78
60.26
53.73
49.04
40.26
YoooP+ (Ours)
69.61
63.30
57.56
49.60
61.92
56.47
51.32
44.25
Table 2: Average incremental accuracy (AIA) and average accuracy after training the last task (AA) of the proposed YoooP/ YYooP++ and baselines on CIFAR-100 and TinyImageNe under half-base setting with different phasos."half-10 mcans half-base with 10 phasos, "half-5" means half-base with 5 phases. (*) denotes methods using "Nearest Class Mean" (NCM) classifier. Bold: the best among non-exemplar methods. Underline: the second best among non non-exemplar methods.
Datasets
CIFAR-100
TinylmageNet
AIA
%lf
AA [%]1
AIA
[
AA
[7]1
Method
half-5
half-10
half-5
half-10
half-5
half-10
half-5
half-10
LwF Li Hoiem
2017
49.00
37.46
27.93
17.09
34.46
23.34
21.34
13.56
SSRE
Zhu el al. 2022
63.38
61.29
52.35
49.27
50.45
47.88
40.48
39.47
IL2A
Zhu et al.
2021a
62.97
52.44
48.89
33.08
45.27
43.34
34.54
33.73
FeTriL
Petit et al. (2023
67.57
67.43
58.38
57.45
51.79
50.10
42.51
41.76
DS-AL Zhuang a..
(2024
68.42
68.37
61.35
61.23
58.41
58.08
50.92
48.86
FFC Magistri et al. 2024
67.93
64.88
56.55
54.83
56.64
53.18
44.67
40.35
AdaGauss Rypesč et al. 2024
65.07
62.35
58.14
54.82
58.33
56.00
50.09
47.90
*FeAM Goswami et al. 2023
67.17
67.01
60.21
59.93
60.25
60.02
50.64
50.64
*DC Goswarni
2024
65.62
61.71
54.31
49.12
52.36
47.12
43.00
37.18
PASS Zhu 2021b
64.10
57.41
54.80
45.71
48.61
39.99
38.69
30.23
PASS w/o Aug
59.34
55.41
49.64
43.02
46.06
42.84
37.12
33.56
YoooP (Ours)
66.19
58.99
56.61
47.09
62.30
58.66
53.94
49.40
YoooP+ (Ours)
67.40
61.83
58.54
50.81
65.57
61.72
58.56
52.40
initial task while keeping it fixed for future tasks. Although FeTriL, DS-AL, and FeCAM effectively preserve initial task knowledge, those stratogy severely restricts the model's capacity to adapt and im- prove representations for subsequent lasks, particularly when these difler significantly from the initial task.

---

Published in Transactions on Machine Learning Research (07 (07/202) 2205)
100
90
90
80
H55RE
85
+1.2A
80
70
4FeTnL
65
Average Accuracy [%)
70
Averaga Accuracy(s)
Accuracy!
60
55
AAdaGauss
55
50
Avergg
HPxCAM
40
30
BPASS
30
20
BPA55
35
20
25
10
w/o su
40
120
200
20
60
140 number of classes
100
eToooP
number classes
number of classes
number classes
(a) CIFAR-2ero-5
(b) CIFAR-Zero-10
(c)
Tiny-zero-5
(d) T'iny-zero-l1 10
85
80
80
4LWF
ASSRE
75
70
70
M12A
Average Accurac/S8
65
Accurec
Average Accuracy(5)
Accuracy
50
EDS-AL
55
50
OEFC
45
40
AAdaGauss
Average
45
40
35
30
AADC
35
25
30
20
DPASS
25
15
20
10
50
60
70
80
90
100
50
60
70
80
90
100
100
120 140 160 180 200
100
120 140 160 180 200 DYccoo++
number of classes
number of classes
number of classes
number of classes
(c)
CIBAR-half-5
(f)
CIFAR-balf-10
(g) Tiny-half-5
(h) Tiny-half-10
Figure 4: Performance comparison of each task across different methods on CIFAR-100 and TinylmageNet under different settings while training model incrementally. "zero-5,10"; "zero-base" with 5 and 10 phases settings. 1ha'f-5,10": "half-base" with 5 and 10 phases settings.
Thus, the FeTrIL, DS-AL, and FeCAM fail to achieve great performance if the initial task contains little knowledge and the future tasks have a large gap from the first task. Therefore, the proposed YoooP and YoooP+ beat the FeTrIL, DS-AL, and FeCAM on both datasels under zero-base seitings, and YoooP+ surpasses those three methods on Iiagemt under half-base settings. Specifically, on Tinylm- ageNet, YoooP outperforms DS-AL by 3.89% and 0.58% (ATA), and 3.02% and 0.54% (AA) under the respective phases. Furthermore, YoooP+ exhibits even greater improvements over DS-AL, achieving 7.16% and 3.64% (ATA), and 7.64% and 3.54% (AA). YoooP and YoooP+ also clearly surpass ADC by large margin, reinforeing our methods' capability to dynamically capture richer class representations, particularly on more challenging datasets like Tiny- ImtgeNet. The incremental accuracy progression under the half-base setting is depicted in the bottom row of Fig.
Table 3: Average forgetting (AF) of the proposed YoooP ToooP//YooPP YYooo++ and baselines on CIFAR-100 and Tiny- ImageNet under zero-base setting with different pba 10" means zero- base with 10 phases, "b0-5" means zero- base with 5 phases. (*) denotes methods using "Nearest. Class Mean" (NCM) classifier. Bold: the best among non-exemplar methods. Underline: the second best among non-exemplar methods.
AF [%]]
CIFAR-100 Tiny ImageNe
Methods
b0-5
b0-10
b0-5
b0-10
LwF Li & Hoiem
2017
43.80
51.80
45.79
54.40
SSRE Zhu al. 2022
15.44
12.13
16.31
19.94
IL2A Zhu et al. 2021a
26.94
25.07
20.89
26.10
TTTTIL Petit al. 2023
18.88
16.14
15.13
15.32
*ADC Goswami et al. (2024
19.15
21.21
15.35
23.92
PASS Zhu et al. PASS w/o Aug
2021b
23.66
18.78
22.00
20.69
28.11
29.55
24.01
26.00
YoooP (Ours)
21.24
22.69
16.49
25.02
YoooP+ (Ours)
18.75
15.41
14.28
19.24
Evaluation on Average Forgetting. We also evaluate the average forgetting (AF) under zero-bass settings CIFAR-100 and TinyImageNet. As shown in Tab. proposed methods, YoooP and YoooP+, achieve
10

---

Published in Transactions on Machine Learning Research (07/2025)
relatively low average forgetting compared to most baseline methods. Although some baselines exhibit lower AF, this is primarily due to their sharp accuracy decline in initial incremental phases followed by persistently low performance, as shown in Tab. [ and clearly illustrated by the incremental accuracy curves in the bottom row of Fig. 4 In contrast, YoooP and YoooP+ effectively maintain higher incremental accuracy (AIA and AA) throughout all phases (Tab. D. genuinely mitigating capastrophic lorgetting while maintaining superior overall performance.
Discussion. Our experimental results yield three main findings: (1). Comparing PASS w/o Aug and PASS in Tab. and Tab. 2 directly adding high-dimensional Gaussian noise for prototype augmentation can hurt performance, especially under zero-base scttings and the half-base setting with 10 phases on TinyImageNet. This is because high-dinensional Gaussian noise produces sparse, nearly equidistant data (the curse of dimensionality). In contrast, YoooP+ generates syuthetic data in angular space using a rotation matrix while preserving the real angular distribution, which consistently improves prototype performance. (2). The ADC with NCM shows an advantage only in the CIFAR-100 base setting. As shown in Tab. D and Tab. 2 ADC outperforms YoooP on CIFAR-100 under zero-base conditions but does not surpass YoooP+ and fails to outperform YoooP in other settings. Furthermore, in the half- base configuration, several non-NCM methods outperform ADC. This occurs because NCM relies solely on class-mean prototypes and avoids gradient updates to mitigate catastrophic forgetting, limiting its ability to learn complex decision boundaries when task distributions shift significantly. In contrast, a normal classifier with softmax benefits from gradient updates that enable the learning of complex boundaries and better generalization. In the half-base setting, where the initial task has abundant data and later tasks have fewer samples, the pronounced distribution shift leads to performance drop (comparing the performance of ADC between the two tables) and the loss of ADC's initial advantage. (3). The proposed YoooP and YoooP+ consistently outperform other non-cxemplar-based methods across different datasets under various settings. They achieve higher AIA and AA while maintaining lower AF, demonstrating robust performance.
4.2 Comparison of Synthetic Data for YoooP+ and PASS
In this experiment, we randomly selected five classes from the CIFAR-100 dataset (task t) under a zero-base setting with 10 phases. We then compared the angular distributions of synthetic data generated from stored prototypes by YoooP+ and PASS, as illustrated in Fig. 5 The upper row of Fig. 5 pertains to the original distribution (also the angular distribution stored in YoooP+), representing the cosine similaritics between the representations 0); and the stored protolypes for each class. In contrast, the bottom row of Fig. 5 shows the distributions of cosine similarities between the same prototypes and the synthetic data generated by PASS. We observe that PASS produces synthetic samples whose cosine similarities are heavily concentrated near 1.0, indicating that these augmented data are nearly identical to the stored prototypes. This narrow distribution leads to reduced diversity and, consequently, diminished performance, as also evidenced in Fig. 4 Tab. [ and Tab. 2 In comparison, YoooP+ draws its synthetic data based on the original cosine-similarity distribution (lop row). This stralegy more effectively restores the representations of the original data and helps YoooP+ generate higher-quality synthetic samples than PASS.
4.3 Comparison of the representations for YoooP+ and PASS
In addition, we compare the learned representations produced by YoooP+ and PASS. Fig. 0 shows the distribution of representations
CIFAR- 100 under zero-base setting with 10 phases for the first three tasks. Because YoooP+ employs prototype optimization in the first task, it forms more compact class clusters than PASS. In Tasks 2 and 3, YoooP+ continues to encode input data within well-defined boundarics, whereas PASS does not. In Fig. G (b), (c), (e), and (I), the light gray points represent data from previous tasks. We observe in (b) and (c) that YoooP+ keeps old and new tasks well separated, while PASS struggles to distinguish between the distributions of old and current tasks. This is because YoooP+ not only creates compact clusters via prototype optimization but also synthesizes high-quality data from the original cosine-similarity distribution, effectively preserving boundaries for old tasks.
11

---

Published in Transactions on Machine Learning Research (07/ (07))
102
60
Original
Original
Original
Ofiginll
250
60
50
20
10
5.0
0.2
04
02
1/2
60
0.2
03
to
02
03
10
0.4
3.
022
03
(a)
(b)
(d)
PASS-Class-]
PASS-Clas-
33
E00
12
12
30
022
5.6
63
82
0.4
5.6
6-3 02
0.4
225
8.0
62
64
2:6
03
80
02
6.1
27
13
(f)
(g)
(h)
Figure 5: Top row: histograms of the original cosine similarity ãg.i (Eq. 2 between each class's prototype and the real samples (top). The augmented samples of YoooP+ are drawn from the original histograms.
Bottom row: histograms the cosine similarity between each class's prototype and the augmented samples
for PASS. PASS fails to preserve the distribution of the original data.
(a) YoooP+ for Task 1
(b) YoooP- for Task 2
(c) YoooP+ for Task 3
6-0
D-19
10
20
21
22
23
24
(d) PASS for Task
(e) PASS for Task 2
(n PASS for Task 3
Figure 6: Visualization of the distribution of representations encoded by YoooP+ and PASS CIFAR-100 base-O phase 10 setting. The lighter gray points in "Task 2' and "Task 3' represent the distribution of the previous tasks' data.
4.4 Ablation Studies
We conduct ablation studies to assess the influence of three key components on model perfor- mance: prototype optimization (P), synthetic data replay, and model interpolation (MI). Fig. 7 presents the results on CIFAR-100 under the zero-base setting with 10 phases. In particular, YoooP (Class Mean) uses the classmmaan for prototype generation (similar PASS) while still employing
12

---

Published in Transactions on Machine Learning Research (07/202) (025)
the prototype optimization loss in Eq. 3 YoooP
95
(-P) omits protolype optimization, YoooP (-MT) ex-
cludes model interpolation, and YoooP (-P-MI) re-
85
YoooP
moves both. From Fig. 7 prototype optimization proves crucial, as YoooP (-P) sulfers substantial drop accuracy compared to the full YoooP. When comparing YoooP and YoooP (Class Mean), YoooP achieves slightly higher accuracy while utilizing mini-batch attentional mean-shift-base, method that requires fewer stored sample embeddings than the conventional class-mean approach (see Appendix F Moreover, YoooP+ (which incorporates prototype augnentation) further improves prediction accuracy relative to YoooP. Although model interpolation (MI) contributes modest performance boost by retaining prior knowledge and ensuring current-task perfor- mance, the difference between YoooP (-P) and the
75
Aaverage Accuracy
65
YoooP (Class Mean)
55
(-MI)
45
YoooP (-P)
35
25
YoooP (-MI-P)
10
20
30
40
50
60
70
80
90
100
number of classes
Figure 7: Ablation study of different components in YoooP+. "-M" means without model interpolation, "_P" mcans without prototype optimization.
other variants indicates that prototype optimization ion remains the most critical factor.
5 Conclusion
In this work, we developed two aon-exemplar-bass based methods, YoooP and YoooP+, for insseinncrmeetall lcarning. Specifically, YoooP only needs to store and replay one optimized prototype for each class without generating synthetic data from stored prototypes. As an exlension of YoooP, YoooP+ proposed to create synthetic data from the stored prototypes and the stored distribution of cosine similarity with the help of high-dimensional rotation matrix. The evaluation results multiple benchmarks demonstrated that both YoooP and YoooP+ can significantly outperform the baselines in terms of accuracy and average forgetting. Importantly, this work offered new perspective on optimizing class prototypes for exemplar-free CIT. We also show more experimental results in Appendix
Acknowledgments
Rescarch reported in this paper was sponsored in part by NSF CPS 2311086, NSF CIRC 716152, and Faculty Research Grant at William & Mary 141446.
13

---

Published in Transactions on Machine Learning Research (07 (025)
References
Arsian Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the Europeon Conference on Compuler Vision (ECCV), pp. 532 547, 2018.
Yizong Cheng. Mean shift, mode seeking, and clustering. IEEE transaclions on pallern onalysis and machine intelligence, 17(8):790-799 1995.
Yulai Cong, Miaoyun Zhao, Jianqiao Li, Sijia Wang, and Lawrence Carin. Gan memory with no forgetting. Advances Newral Information Processing Systems, 33:16481-16494, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255. leee, 2009.
Arthur Douillard, Alexandre Ramé, Guillamme Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVV Conference on Compuler Vision and Pattern Recognition, pp. 9285 9295, 2022.
Alexander Gepperth and Barbara Hammer, Incremental learning algorithms and applications. In European symposium artificial newral networks (ESANN), 2016.
Dipam Goswami, Yuyang Liu, Bartlomicj Twardowski, and Joost Van De Weijer. Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. Advances in Newral Information Processing Systems, 36:6582-6500 6595, 2023.
Dipam Goswami, Albin Soutif- Coremr-,,,meais, Yuyang Liu, Sandesh Kamath, Bart Twardowski, Joost Van
De Weijer, et al. Resurrecting old classes with new data for exemplar-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 28525 28534, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilci Wang, and Dahua Lin. Learning unified classifier incrementally via rebalaneing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 831-839, 2019.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barrinska,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521 3526, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Wor Ha, and Byoung-Take Tak Zhang, Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
Timothóe Lesort, Hugo Caselles-Dupré, Michael Garcia-Ortiz, Andrei Stoian, and David Filliat. Generative models from the perspective of conllnual learning. In 2019 International Joinl Conference on Neurul Networks (IJCNN), pp. 1 8. IEEE, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2600 2017.
Xialei Liu, Mare Masana, Luis Herranz, Joost Van de Weijer, Antonio M Lopez, and Andrew D Bagdanov. Ro- tate your networks: Better weight consolidation and less catastrophic forgetting. In 2018 24th International Conference on Pattern Recognition (ICPR), pp. 2262-2263 IEEE, 2018.
David Lopez-Paz and Marc' Aurclio Ranzato. Gradient episodic memory for continual learning. Advances in neuraa information processing systems, 30, 2017.
14

---

Published in Transactions on Machine Learning Research (07/)025)
Simone Magistri, Tomaso Trinci, Albin Soutif, Joost van de Weijer, and Andrew D. Bagdanov. Elastic feature consolidation for cold start exemplar-free incremental learning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openrevier net//ooruPiee719X22FFtt
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp. 7765 7773, 2018.
Arun Mallya, Dillon Davis, and Svetlana Lazebník. Piggyback: Adapting single network to multiple tasks by learning to mask weights. In Proceedings of the European conference on computer vision (ECCV), pp.
67-82, 2018.
Mare Masana, Xialei Liu, Barttomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Weijer. Class-incrementee learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513 3-3333553333- 2022.
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEEE/CVE conference on compuler vision and poltern recognition, pp. 11321 11329, 2019.
A Paszke. Pytorch: An imperative style, high-performance deep learning library. orXiv preprint 1912.0912.011 2019.
Grégoire Petit, Adrian Popescu, Hugo Schindler, David Picard, and Bertrand Delezoide. Fetril: Feature translation for exemplar-free doss-incremental learning. In Proceedlings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pp. 3911 3920, 2023.
Amal Rannen, Rahaf Aljundi, Matthew B Blaschko, and Tinne Tuytelaars. Encoder based lifelong learning. In Proceedings of the IEEE International Conference Computer Vision, pp. 1320 1328, 2017.
SylvestreAAlvise Rebuffi, Alexander Kolesníkov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pottern Recognition, pp. 2001-2010, 2017.
Amanda. Rios and Laurent Iuti. Closed-loop memory gan for continual learning. arXie preprinl arXiv:1811.01146, 2018
Ryne Roady, Tyler L Hayes, Hitesh Vaidya, and Christopher Kanan. Stream-51: Streaming classification and novelty detection from videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition Workshops, pp. 228 229, 2020.
Mohammad Rostami, Soheil Kolouri, and Praveen K Pilly. Complementary learning for overcoming catas- trophic forgetting using experience replay. arXiv preprint arXiv:1903.0 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-250 252, 2015.
Grzegorz Rypešé, Sebastian Cygert, Tomasz Trzciński,, and Bartlomiej Twardowski. Task-recency bias strikes
back: Adapting covariances in exemplar-fre free class incremental learning. Advances in Neural Information Processing Systems, 37:63268-63288 2024.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International Conference on Machine Learning, pp. 2018.
PMLR,
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. A Advances in neural information processing systems, 30, 2017.
15

---

Published in Transactions on Machine Learning Research (07 (07/2025)
Fu-Yum Wang, Da-Wei Zhou, Liu Liu, Han-Jia Ye, Yatao Bian, De-Chuan Zhan, and Peilin Zhao. Beef: Bi- compatible class-incremental learning via energy-based expansion and fusion. In The eleventh international conference on learning representations, 2022a.
Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. Foster: Feature boosting and compression for class-incremental learning. arXiv preprint 2Xi::2204.04666 2022b.
Liyuan Wang, Kuo Yang, Chongxuan Li, Lanqing Hong, Zhenguo Li, and Jun Zhu. Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning. In Proceedings of the IBEE/CVF Conference on Computer Vision and Pallern Recogriiiio,, pp. 5383 5392, 2021.
Chenshen Wu, Luis Herranz, Xialei Liu, Joost Van De Weijer, Bogdan Raducanu, et al. Memory replay gans: Learning to generate new categories without forgetting. Advances in Neural Information Processing Systems, 31, 2018.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedinggs of the IEEE/CVI Conference on Compuler Vision and Paltern Recognition, pp. 374-382, 2019.
Menggi Xue, Haofei Zhang, Jie Song, and Mingli Song. Meta-atlention for vit-backed continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 150-159, 2022.
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp- 3014 3023, 2021.
Leon Yao and John Miller. Tiny imagenet classification with convolutional neural networks. CS 231N, 2(5):8, 2015.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang, Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: 778.700.1100 01547, 2017.
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairness in class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 13208 13217, 2020.
Da-Wei Zhou, Fu-Yun Wang, Han-Jia Ye, and De-Chuan Zhan. Pycil: A python toolbox for class-incremental learning. Xixi preprint arXiv:2112. 12539, 2021.
Fei Zhu, Zhen Cheng, Xu- Xu-Yao Zhang, and Cheng-lin Liu. Class-incremental learning via dual augmentation. Advances in Neural Information Processing Systems, 34:14306-14318, 2021a.
Fci Zhu, Xu-Yao Zhang, Chuang Wang, Fci Yin, and Cheng-Lin Liu. Prototype augmentation and self- supervision for incremental learning. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pattern Recognition, pp.
2021b.
Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, and Zue Zha. Self-sustaining representation expansion for non-exemplar inaseinncementll learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9296 9305, 2022.
Huiping Zhuang, Run Hc, Kai Tong, Ziqian Zeng, Cen Chen, and Zhiping Lin. Ds-al: A dual-stream analytic learning for exemplar-free class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 17237- 17244, 2024.
16

---

Published in Transactions on Machine Learning Research (07 (07/2025)
A YoooP and YoooP+ Algorithms
We summarize the proposed YoooP and YoooP+ algorithms as follows. Regarding parameters S steps and iterations in Algorithm [ they are determined by the ratio of Dataset Size to Mini-batch size. More specifically, is the number of total steps per epoch for each task. For the number of steps R used update the prototypes in each task, we simply set R S in our experiments. For computing prototype Pk, randomly choose one sample from cach class k and use its embedding the initial prototype. Our protolype updale method is an extension of the classical mean-shift algorithm Cheng (1995) that replaces the pre-defined kernel similarity with a learnable attention score. As stated in Cheng (1955), random initialization of Pı can guarantee the independence of the solution to the initialization problem.
Algorithm 1: YoooP and YoooP+
input :Training data Di.r with classes Ci.,, epochs E, steps S, ilerates R, number of bius No learning rale
17, a, 3
initialize Memory M < 0, Distribution P 0, 0,
for t== 1-=1-1 do
for epoch=1 do
3
Compute features 2 = F( for (It, y:) E Dr;
Compute prototype ps for every class E C, by iterating Eq. for R iterations;
6
Save prototypes: . 4 M {(pt, k E C.):
Save distribution of cosine similarity using histogram: P P (P(an) & C.
for step=1 + S do Draw mini-batch data (e., ~ D::
if YoooP+ then
10
Data Synthesis: create a mini-batch of data (z/, k) previous classes from prototypes (pe k C1a-1) and the distribution of cosine similarity (P(āx.) & E C.) in Eqs. Compute loss (6, w) in Eq. on the two mini-bate bani-batches;
11
12
else
Draw mini-batch of prototypes (px, k) from k E Cra-1; Compute loss [., w)) in Eq. 6 6n the two mini-batches;
14
Update feature extractor: 0 +- 0 vVo.. w);
16
Update classifier for & E C₁: wk + wa V Le w)); Update classifier for C1:-11: wk + wa 1) (a V w2 (0, w));
17
18
Model interpolation: f (1 8)0' 38; Save current lask model 0' t- 0;
19
output
Feature extractor F(:;0) and classifier G(:; w)
B Effect of Hyper-parameters
We also explore some important hyper-parameters the estimation accuracy of YoooP+ as follows.
B.1 Effect of À in mean-shift method.
Tab. ssoows the impact of prpee-parameter À from Eq, [ in mean-shift- method on CIFAR100. We can obscrve that as X increases from 0.3 to 0.9, the prediction accuracy of YoooP+ almost remains the same. Thus, we can conclude that the proposed method is not sensitive to À that controls the steps in the attentional mean-shift-basse algorithm. We choose = 0.6 in our experiments.
B.2 Effect of parameter 3 in model interpolation.
We also investigate the influence of hyperparameter 3 from Eq. 12 in model interpolation on the prediction accuracy, as shown in Fig. 8 (a). It can be seen that when 0.6, the proposed method has the best performance. We change the / from 0.3 0.8, the performance just slightly drops when the 3 0.8. Thus conclude that the proposed method is not sensitive to B.
17

---

Published in Transactions on Machine Learning Research (07/2025)
Table 4: Effect of À on the accuracy of different tasks on CIFAR-100 with base-O 10 phases setting, We can
that method not sensitive to X.
Base-0-10 10 phases Average Accuracy on CIFAR-100 with different X.
[%]
A
Task 1
Taks 2
Task
Task 4
Task 5
Task
Task 7
Task 8
Task
Task 10
0.3
92.99
76.67
70.79
63.91
60.36
57.57
55.69
53.80
51.26
49.39
0.4
91.85
76.45
70.65
64.57
61.34
57.84
55.26
53.25
51.15
49.59
0.5
91.09
76.12
70.02
63.97
61.85
58.08
55.79
53.83
51.53
49.15
0.6
91.83
76.42
70.62
64.78
61.51
57.99
55.49
53.44
51.31
49.60
0.7
91.77
76.67
71.03
64.83
61.45
57.95
55.54
53.48
51.16
49.54
0.8
91.89
76.71
70.77
64.86
61.55
58.07
55.60
53.45
51.31
49.63
0.9
91.84
76.74
70.74
64.86
61.59
58.14
55.51
53.46
51.13
49.56
0.0.3
95
90
Average Accuracy [%)
0.4
85
+011
80
Average Accuracy [%6]
+.0.5
0.001
70
++0.6
65
+0.005
60
55
* 0.001
000.7
50
+0.0001
++0.8
40
35
20
40
60
80
90
100
number of classes
number of classes
Ablation-3
(b) Ablation-a
Figure 8: The ablation study of a and B on the prediction results.
B.3 Effect of small factor a.
Table 5: Average incremental accuracy (AIA) and average accuracy after training the last task (AA) of the proposed YoooP YoooP/YoooPP: and baselines Sub-ImageNet under base setting and half-base setting with 10 phases. "b0-10" means zero-base with 10 phases, "half-10" means half- base with 10 phases. (*) denotes methods using "Near- Class Mean" "aan"((N (NCM) classifier. Bold: the best among non-exemplar methods. Underline: the second best among non-exemplar methods.
Lastly, we study the effect of small factor a from Eq. 13 the "Partial Freezing" of the classifier on the model performance. As illustrated in Fig. (b). it be observed that when is O larger than 0.005, the prediction accuracy will drop slightly. As is a very small value, such as 0.001, the proposed method has the best performance. Hence, we choose a = 0.001 in our experiment.
Sub-Imagehe
AIA 1%1
AA 1%]
Methods
b0-10
half-10
b0-10
half-10
C Evaluation Sub-ImageNet Dataset
LwF Li & Hoiem
2017
30.64
27.46
15.62
16.62
SSRF Zhu al. 2022
41.12
60.41
24.43
51.20
In this section, we further evaluate our methods on larger Sub-ImageNet datasct created by randomly selecting 100 classes Deng al. (2009). This exper- iment demonstrates how YoooP and YoooP+ com- pare against various non-exemplar-basee based approaches under different settings. We exclude IL2A from the comparison due to its prohibitively high GPU memory requirements on Sub-ImageNet.
FeTrIL Petit
12023
46.73
72.55
28.84
64.06
*ADC Goswami et al.
2024
65.90
53.18
50.66
40.26
PASS
Zhu al.
2021bb
44.10
64.57
27.70
53.54
YoooP (Ours)
68.28
75. .6
52.86
68.96
YoooP (Ours)
70.35
78. 24
56.20
70.54
As shown Tab. 5 and Fig. g YoooP and YoooP+ consistently outperform the non-cxemplar basclines both zero-base and half- base configurations. line with our earlier findings (Sec. 4.1). FeTrIL continues perform well in the half-base setting but fails in the zero-base scenario, likely because it cannot. learn sufficient.
18

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
90
LWF
80
70
Average Accuracy|96
Average Accuracy|
60
FoT//l
50
50
+DA
40
P
30
YoooP
10
20
40
50
60
70
80
100
70
100
number of classes
number of classes
(a) tunnggSub--rrrr--asepppuaas
(b) SegeeSSuubball-bbsw-aasssse
Figure 9: Accuracy comparison different non-exemplar-basel and exemplar-based methods on ImageNet- Subset under different settings using 3 random seeds.
initial knowledge when starting with no base classes. Meanwhile, ADC with NCM classifier excels under zero-base conditions but suffers large performance drop in the half-base setting. This is primarily due to the substantial distribution shift between tasks in the half-base scenario, which is especially pronounced large-scale dataset like Sub-InageNet.
D Evaluation on 20 phases setting.
Table 6: Average incremental accuracy (ALA) and average accuracy after training the last task (AA) of the proposed YoooP YYooP + and baselines on TinyImageNet under zero- base setting and half-base setting with 20 phases "b0-2s."b0-20" means zero-base pero-base with 20 phases, "half-20" means half- base with 20 phases. (*) denotes methods using "Near- est Class Mean" (Naa"((CM)) classifier. Bold: the best among non-exemplar methods. Underline: second best among non-exemplar methods.
We further evaluate our proposed methods Tiny- ImageNet under more extended incremental settings, specifically 20-phase scenarios in both vero-base and half-base configurations. As shown in Tab. 6 FeTrIL, FeCAM, and DS-AL, contimue to perform well the half-base selting (earning second place) but expe- rience severe performance drop in the zero-base setting. Meanwhile, ADC with an NOM classifier achieves the second-best AIA under the pero-base setting but is consistently out performed by YoooP and YoooP+, particularly in the half-base configu- ration. Comparing to the PASS, the performance of our proposed methods surpass it in a clear gap across various settings. These findings reinforce our carlier observations that relying on the performance of the initial tasks (as in FeTrIL, FeCAM, EFC, and DS-AL) NCM classifier (as in ADC) limits adaptabil- ity when the task distribution shifts substantially. By contrast, YoooP and YoooP+ maintain strong performance across both zero- base and half- base 20- phase scttings, highlighting their robustness in more demanding incremental learning scenarios.
MhooP/YoooP11
TinyImageNet
AIA [%]t
AA
1%
Methods
b0-20
half-20
b0-20
half-20
LwF
Li & Hoiem
2017
22.93
13.39
9.57
5.62
SSRE Zhu et
2022
25.78
46.20
15.25
38.84
IL2A Zhu et al. 2021a
25.39
39.90
17.73
29.43
FeTHiL Petit al. 2023
31.84
48.96
18.38
41.44
DS-AL Zhuang el a..
2024
33.74
58.17
24.58
50.79
EFC Magistri et al.
2024
42.31
56.01
28.27
47.21
AdaGauss Rypese et al. 2024
44.21
50.31
30.26
41.95
"FeCAM Guswami el al. 2023
38.61
59.32
25.30
54.61
*ADC Goswami et PASS
2024
46.75
34.64
29.24
25.70
Zhu et al.
2021b
33.76
33.24
19.36
25.21
YoooP (Ours)
46.00
48.19
30.97
39.31
YoooP+ (Ours)
48.36
51.27
31.26
42.28
E Compare with Exemplar-based methods
To showcase the strong performance of our proposed methods, we also compare them with several exemplar- basod approaches, including iCaRL, Rebuffi et al. (2017). BiC Wu al. (2019), and WA Zhao al. (2020), under zero-base and half-base settings with and 10 phases on TinylmageNe Following prior works Hou
et al. (2099) Rebuffi et al. (017), these exemplar-based methods store 20 raw samples per old class, selected
19

---

Published in Transactions on Machine Learning Research (07 (07/202) 2025)
90
W/A
Average Accuracy| [%6]
70
70
*: BIC
Average Accuracy[%6
*--CaRL-
50
50
CNN
*-CaRL NME
30
30
YoooP
10
10
YoooP
40
80
120
160
200
40
60
80
100
120
140
160
180
200
number of classes
number of classes
(a) Tiny-zero- baro--baee-phaesh
(b) Tiny-zero- base-phases10
80
80
KWA
60
60
BIC
Average Accuracy
Average Accuracy($6
C-CRRL CNN
40
40
* XiCaRL-
NME
20
YoooP+
100
120
160
200
110
130
150
170
200
number of classes
number of classes
(c) Tiny-half-basee paaseess-
(d) Sin-haal-hase-phasee]
Accuracy comparison of different exemplar-based methods TinyImageNel under different. settings using 3 random seeds.
Figure 10:
via herd election Rebulfi et al. (2017) It is important to note that exemplar-based methods typically achieve higher accuracy than non-exemplar based methods, they can store data and thus preserve richer information Rebuffi et al. 2017 Hou et al. (2019). This makes fair comparison particularly challenging.
Fig. 10 shows the performance of different approaches on TinyImageNet. Notably, although our YoooP and YoooP+ do not store raw data, they still achieve comparable performance zero- base settings and even surpass some exemplar-bar based methods in half base-base settings. This result underscores the effectiveness of our prototype optimization and augmentation strategies, demonstrating that even without raw data storage, our approach can overcome the inherent advantages of exemplar-based methods under certain conditions.
F
Evaluation of Additional Storage Requirements
We further evaluate the additional storage requirements of our proposed mini-batch attentional mean-shift- based method presented in Eq. [ Specifically, this metric quantifies the number of sample embeddings that must be stored for generating prototypes of classes. In our approach, only the embeddings from a mini-batch are relained, whereas the conventional class-mean mot requires storing the embeddings of all samples within each class. Tab. presents a comparison these storage requirements for individual tasks under different settings on TinyImageNet datasct. As shown in Tab. 7 our methods require significantly fewer stored embeddings than class-mest approaches, demonstrating clear advantage in terms of memory efficiency.
20

---

Published in Transactions on Machine Learning Research (07 (025))25)
Table Number of stored embeddings utilized for generating prototype in each task of our proposed method and class mean method in the baselines under different settings on TinylmageNet.
Additional Storage Requirements under different settings TinylmageNet: #
Our method
Settings
Class-mean
batch size: 128
batch size: 256
base0-phases5
128
256
20000
base0-phases10
128
256
10000
halff-base-phases5
128
256
10000
half-base-phasessll
128
256
5000
G Evaluation on the confusion matrix
To effectively demonstrate the results of Class-Tncrementar Learning (CIT.), conduct comparison of the classification confusion matrices for PASS, YoooP, and YoooP+ on CIFAR-100 under the zero-base 10-phase setting. The confusion matrix, shown in Fig. 11 illustrates correct predictions on the diagonal and misclassifications in off aff-diagonal entries.
Comparing Fig. 11 a) and Fig. 11 b), we observe that the diagonal entrics' temperatures in YoooP are higher than those PASS, which means the accuracy higher, particularly for classes 20 to 100. This indicates that YoooP excels in learning the current task while minimizing forgetting of the old tasks. Consequently, YoooP cxhibits enhanced plasticity and stability compared to PASS.
Further comparing Fig. 11 (b) and Fig. 11 11,,, notice that the diagonal entries' temperatures for classes 0 to 40 in YoooP+ are higher than those in YoooP. Notably, YoooP tends to classify more into recent tasks. + effectively mitigates this tendency and reduces bias through the proposed prototype augmentation, resulting in superior performance.
true classes
true classes
60
60
90
300
300
100
0.0
predic predicted classes
predicted classes
predicted classes
(a) PASS
(b) YoooP
(c) YoooP+
Figure 11: Confusion matrix comparison of different methods under CIFAR-100 base0-10 setting.
21

---

Published in Transactions on Machine Learning Research (07 (07/2025)
H Evaluation on the efficiency
Table 8: Efficiency Tiny TinylmageNel under zero-base 10 phases setting, One GPU NVIDIA RTX A6000,
BS=256, M-10 G=10", T=10
Methods
MACst
Parameters
GPU-Memo/
Time/Epoch/
+oooP+
571.26 G
11.17 M
9.86 GB
22.07 s
YoooP
571.26 G
11.17 M
9.53 GB
17.10 s
ADC
571.26 G
11.17M
9.53 GB
17.00 s
PASS
2.28 T
11.18 M
>50 GB
40.64
FeTril
571.26 G
11.17 M
10.72 GB
349.15
SSRE
633.31 G
12.40 M
13.23 GB
50.17 8
LwF
571.26 G
11.17 M
9.53 GB
16.90 8
IL2A
2.74 T
11.17 M
>50 GB
97.4
To show that both YoooP and YoooP+ designed to be lightweight, we provide detailed analysis of the computational cost and implementation details for prototype optimization and synthetic data replay in Tab. E As shown in the table, our YoooP' method achieves competitive even lower MACs, GPU memory, and training time per epoch compared to other basclines (e.g., ADC, PASS, FeTrIL). Specifically, YoooP+ requires 571.26 .66 G MACs, 9.86 GB GPU memory, and 22.07 seconds per epoch, demonstrating that protolype optimization and synthetic data replay introduce minimal overhead.
22


Reviews:
[{"review_id": "WNSN2c3JDu", "reviewer": "Reviewer_8KDr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- YoooP and YoooP+ outperform strong baselines across multiple benchmarks and learning settings, showing consistent gains in AIA and AA. \n- The methods perform competitively despite not storing any real data, which is important for privacy-aware or resource-constrained applications.\n- Unlike previous works that use Gaussian noise, YoooP+ generates data in angular space, leading to more realistic and diverse synthetic samples.\n\nWeakness: \n- Evaluation is limited to CIFAR-100 and TinyImageNet. Testing on larger datasets (if possible) would strengthen generalizability claims.\n- While the paper introduces several accuracy-related metrics to evaluate performance, it does not provide comparisons in terms of memory usage or computational cost.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents YoooP and its enhanced variant YoooP+, two exemplar-free methods for class-incremental learning that address the challenges of catastrophic forgetting without relying on stored data from previous tasks. YoooP introduces a prototype-based framework that optimizes class representations through a mini-batch attentional mean-shift mechanism, producing compact and discriminative prototypes. YoooP+ further improves performance by generating synthetic data in angular space, preserving the original distribution of class features using a rotation-based augmentation strategy. The authors demonstrate that this approach results in higher-quality replay samples compared to prior methods that rely on Gaussian noise. Through comprehensive experiments on CIFAR-100 and TinyImageNet under both zero-base and half-base settings, the paper shows that YoooP and YoooP+ consistently outperform existing non-exemplar methods in terms of accuracy and forgetting. The work is further supported by ablation studies and analysis of representation quality, validating the effectiveness of the proposed components.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Include runtime/memory comparisons with key baselines  to quantify the computational trade-offs introduced by prototype optimization and model interpolation.\n- Clarify implementation details for prototype optimization and synthetic data replay, particularly the cost and frequency of these operations during training.", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i4wmyT4IGf", "reviewer": "Reviewer_7TYA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths - \n1. The paper proposes an interesting approach for optimizing prototypes and generating synthetic samples around prototypes to obtain distributions closer to original distributions.\n\nWeaknesses-\n1. The motivation is heavily based on a single paper - PASS [CVPR 2021] since FeTrIL does not augment prototypes and instead samples from a gaussian distribution using class distributions with the assumption that similar classes have similar feature distributions. The paper is not discussed or based on more relevant and recent works on non-exemplar IL [a,b,c].\n\n2. FeCAM [a] proposed to use class distributions in the form of class covariances to better represent classes and then used a Mahalanobis-distance based classifier for non-exemplar IL. The authors ignore any discussion with covariance based methods [a,b] and does not include them in the experiements. [a, b, c] performs better than FeTrIL in most settings and are relevant baselines for non-exemplar IL. For instance, it is not clear why one needs to optimize the prototypes or generate synthetic samples when they can simply use the class covariances and adapt them continually?\n\n3. While prototype optimization is interesting, it is not clear why the authors do not consider using class covariances in any form to improve class representations. The motivation for the proposed class centroid optimization is not so coherent in the paper. Why is the average of the class features not a good prototype? Can the authors show some analysis on how distant is the optimized prototype from the class feature centroid? \n\n4. How good is the proposed generation of synthetic samples compared to sampling features from a gaussian using the class mean and covariances?\n\n5. The method has several components and the implementation in Algorithm 1 says that it involves more complexity. The training needs several steps in each epoch which is computation intensive. Can the authors discuss more on this? The improvements does not look very impressive given the complexity and combination of several components in the method.\n\n6. How is the inference done? While it is a prototype-based method and the authors stress on using one optimized prototype per class, it is not using the prototypes for inference. How good is the performance if you use the optimized prototypes for NCM classification? Is this better than standard NCM with class feature centroids?\n\n[a] \"Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning.\" Advances in Neural Information Processing Systems 36 (2023).\n\n[b] \"Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning.\" Advances in Neural Information Processing Systems 37 (2024).\n\n[c] \"Elastic Feature Consolidation For Cold Start Exemplar-Free Incremental Learning.\" The Twelfth International Conference on Learning Representations, 2024.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper focus on non-exemplar continual learning using class-wise prototypes and discuss the limitations of methods like PASS which uses a single class centroid as the prototype and replays synthetic features. The authors propose a mini-batch attentional mean shift-based method to construct more representative prototypes. The authors also propose to generate synthetic features by exploiting the angular distribution between each class’s prototypes and its features. The proposed method outperforms competitive non-exemplar IL methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would request the authors to address the points in weaknesses mentioned above.", "broader_impact_concerns": "The paper does not have ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yjzz3Wbaht", "reviewer": "Reviewer_obox", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1. This paper tackles the non-exemplar class-incremental learning problem, an important problem in the machine learning field.\n2. The proposed method is rigorously evaluated on standard benchmark datasets and demonstrates better performance compared to existing approaches.\n3. The paper proposed a novel prototype augmentation technique, which is effective.\n\nCons:\n\n1.\tComparisons with other SOTA methods are missing, such as [1] and [2]. For example, on Cifar100 b50in10 setting, the AIA of DS-AL[1] is 68.4%, but the proposed method is 67.4%.\n2.\tMultiple things are combined in the Yooop+ method - three loss components and two tricks, where feature distillation is quite a strong regularization, preventing forgetting. However, mixing feature distillation with model interpolation is confusing.\n3.\tThe ablation is not clear about every single participant. I want to know the contribution of the feature distillation loss item and \"Partial Freezing\" of the Classifier.\n4.\tThe method's reliance on synthetic data increases computation time, yet the paper omits runtime comparisons with existing methods.\n5.\tThe method proposes a plug-in module so it can be combined with other methods. I hope to see the results.\n6.\tWhile the approach leverages prototypes for loss construction, both the backbone and the classifier remain trainable throughout the learning process. The current title may not fully capture the scope of the proposed method.\n\n[1] DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning. AAAI2024\n\n[2] Task-recency bias strikes back: Adapting covariances in Exemplar-Free Class Incremental Learning. NeurIPS2024", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper tackles the non-exemplar class-incremental learning problem, an important problem in the machine learning field. The authors present an enhanced prototypical network framework with novel loss functions specifically designed for this problem. The proposed approach demonstrates competitive performance through evaluations on benchmark datasets, outperforming existing methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See Weaknesses.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)