Two-Way Is Better Than One: Bidirectional Alignment with Cycle Consistency for Exemplar-Free Class-Incremental Learning
Authors: Hongye Xu
Bartosz Krawczyk
Venue: ICLR 2026 Poster (2026)
Keywords: continual learning
exemplar free
exemplar free class incremental learning
class incremental learning
exemplar-free
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Continual learning (CL) seeks models that acquire new skills without erasing prior knowledge. In exemplar-free class-incremental learning (EFCIL), this challenge is amplified because past data cannot be stored, making representation drift for old classes particularly harmful. Prototype-based EFCIL is attractive for its efficiency, yet prototypes drift as the embedding space evolves; thus, projection-based drift compensation has become a popular remedy. We show, however, that existing one-directional projections introduce systematic bias: they either retroactively distort the current feature geometry or align past classes only locally, leaving cycle inconsistencies that accumulate across tasks. We introduce bidirectional projector alignment during training: two maps, old$\to$new and new$\to$old, are trained during each new task with stop-gradient gating and a cycle-consistency objective so that transport and representation co-evolve. Analytically, we prove that the cycle loss contracts the singular spectrum toward unity in whitened space and that improved transport of class means/covariances yields smaller perturbations of classification log-odds, preserving old-class decisions and directly mitigating catastrophic forgetting. Empirically, across standard EFCIL benchmarks, our method achieves unprecedented reductions in forgetting while maintaining very high accuracy on new tasks, consistently outperforming state-of-the-art approaches. The code is available
at https://github.com/HXuSz11/BiCyc_ICLR2026.

Published as a conference paper at ICLR 2026
TWo-WAY Is BETTER THAN ONE: BIDIRECTIONAL ALIGNMENT WITH CYCLE CONSISTENCY FOR EX- EMPLAR-FREE CLASS-INCREMENTAL LEARNING
Hongye Xu
Bartosz Krawezyk
Chester F. Carlson Center for Imaging Science Rochester Institute of Technology hx523 90 ritt.edu
Chester F. Carlson Center for Imaging Science Rochester Institute of Technology ba rtosz. krawczykkrit.edd
ABSTRACT
Continual learning (CL) seeks models that acquire new skills without erasing prior knowledge. In exemplar-free class-incremental learning (EFCIL), this challenge is amplified because past data cannot be stored, making representation drift for old classes particularly harmful. Prototype-based EFCIL is attractive for its ef- ficiency. yet prototypes drift as the embedding space evolves; thus, projection- based drift compensation has become popular remedy. We show, however, that existing one-directional projections introduce systematic bias: they either retroac- tively distort the current feature geometry or align past classes only locally, leav- ing cycle inconsistencies that accumulate across tasks. We introduce BiCye, a bidirectional projector alignment approach with a cycle-consisteccy objective: two maps, old->new and new->old, are optimized with stop- gradient gating so that transport and representation co-evolve. Analytically, loss contracts the singular spectrum toward unity in whitened space and that im- proved transport of class means/covariance. yields smaller perturbations of clas- sification log-odds, preserving old-class decisions and directly mitigating catas- trophic forgetting. Empirically, across standard EFCIL benchmarks, our method achieves unprecedented reductions in forgetting while maintaining very high ac- curacy on new tasks, consistently outperforming of approaches. The code is available at httpt:ts/////o...... pettps://qithur /qihbbb....tttggg/ggg com/ com/HHXuuzl/BBCCC ICLR2026
prove that the cycle
INTRODUCTION
Continual learning (CL) studies models that learn from a stream of tasks without retraining from scratch or erasing prior knowledge (Parisi et al.) 2019 Lange et al.) 2022] Zenke et al.) 2017). A widely used protocol is inass-inccementll learning (CIL), where tasks introduce disjoint labels and
the learner must recognize all seen classes at test time without task identifiers. While rehearsal with
stored exemplars often curbs forgetting (Lopez-Paz & Ranzato 2017 Riemer al. 2018 Pham et al. 2021 Caccia et al. [222]] Wang et al. [022b Yang et 20.3), privacy or memory constraints motivate the exemplar-fres variant (EFCIL), which prohibits retaining raw inputs. Among the many
directions to mitigate forgetting (Zenke et al. 2017: Lopez-Paz Paz & Ranzato 2017 Schwarz et al.
Lope--
2018
Aljundi et al.)
2018
Riemer et al. 2018 [Srraa et al. [2088] [aaha et al. [020 | Pham et al.
2021
Caccia et al.
2021
Deng et al. 2021 1] Cha et al., 2021] Wang et al. 2022a b. Slim et al.,
2022
Wang et al.
2023
Yang et al. 2023 Shi & Wang 2023 Wang et al. 2024), prototype-
based EFCIL has emerged as a compelling compromise: the model caches compact class statistics
(means/covariancee)) and inference proceeds via nearest-prototype or Bayes scores achieving strict no-memory operation with modest compute.
The core difficulty in prototype-based EFCIL representation drift: as the backbone adapts to new tasks, the embedding geometry shifts and previously cached statistics become stale, biasing predictions toward recent classes. Existing EFCIL solutions to drift largely follow two routes that differ in how they balance stability and plasticity.
Covariance and geometry modeling. This route improves robustness by shaping the feature geometry the decision metric, commonly keeping the backbone partially/fixed to limit drift.

---

Published as a conference paper at ICLR 2026
FeTrIL (Petit et al. 2023) freezes the backbone and translates features to synthesize pseudo featuress
for past classes, trading some plasticity for stability. FeCAM (Goswami et al. 2023) argues that Eu- clidean metrics suboptimal under non-stationarity and adopts anisotropic (Mahalanobis) scoring with classwwise covariances, typically with a frozen extractor. PASS (Zhu et al. 2021 strengthens old-class representations via prototype augmentation and self-supervision without exemplars. These methods effectively mitigate forgetting by stabilizing or re- re-weighting the geometry, but they largely avoid cross-space transport; the price of stability potentially limited adaptation new tasks.
Prototype drift compensation. A second-and increas- ingly dominant- retains backbone plasticity and explicitly transports outdated prototypes into the current space. SDC (Yu et al., 2020) projects new features to- ward the old space and updates old prototypes accord- ingly. ADC (Goswami al.) 2024) estimates drift adver- sarially by pushing new samples toward old prototypes, then "resurrects" past classes. LDC (Gomez- Villa et al. 2024) replaces hand-crafted updates with learnable drift module that scales across regimes. EFC (Magistri et al. 2024) performs affinity-weighted, class-wise shifts that update prototypes in tandem with classifier training. Ada- Gauss (Rypešć al., (024) follows the learned-projector path but transports full Gaussian class statistics (means and covariances) into the new space for Bayesian infer- ence rather than only moving class means. DPCR (He) al., 2025) mitigates representation drift by estimating task-to-task semantic shift from stored class statistics and then analytically re-calibrating the classi- fier via ridge regression using only these cached moments, avoiding any exemplar replay. Despite strong performance. the prevailing paradigm here is two-stage: first train the new task (often with eggularrzation/diiiilaatooo then learn a post-hoc adapter neww. This paradigm leaves residual inconsistencies between spaces: transport is optimized only after the fact, and cycle errors accumu- late over tasks.
10-pp Lan.paa Aug. ALC 17) (130722001
UDC PECOYAAL
Aducctio tlousts
ICAPRON
Datter
25 up Lest-Takk Aug. Acc. 1N) mennnaddi
TinyImageNet (T=10): Our training algorithm yields solid performance gains over statc-of-the-ar EFCIL methods.
Figure 1.
Our idea: from two-stage to near single-stage transport. Motivated by the limitations of two- stage drift compensation, we propose bidirectional cycle consistency that evolves adapter training into the main task optimization that transport and representation co-evolve. Concretely, during each new task we jointly learn two maps-A: Zold - Znew and D Znew - Zold wath-with stop-grudient targets to prevent retrograde updates on the evolving representation and cycle-consistency objective that regularizes the pair toward a near-bijection on the data support. Analytically, we show that the cycle loss in whitened space equals |!ÃD /II and contracts the singular spectrum of AD toward one; and that smaller alignment/cycle errors yield tighter bounds on the perturbation of classification log-odds, preserving old-class decisions. After the main stage, a brief consolidation fine-tune is applied: inference uses Gaussian Bayes classifier built from transported old-class statistics and freshly estimated current-task statistics.
Contributions.
Bidirectional cycle consistency(BiCye) within training. We formulate paired projections A (old-neww and D (new oldd learned during the task, with stop-gradient gating and cycle loss that enforces near-inverse behavior addsupp the asymmetry and post-hoc mismatch of prior two-stage, one-way pipelines.
preserving transport for drift mitigation. Our transport keeps old-class ge- ometry stable as the backbone changes, yielding reduced recency bias and higher knowl- edge retention.
Theory-grounde alignment. We prove that minimizing the cycle loss contracts the sin- gular spectrum toward unity in whitened space and derive bounds linking mean/covariance transport errors to classification log-odds stability, explaining the observed reduction in forgetting.
Near single-stage pipeline with strong results. By collapsing adapter learning into the main stage (with short consolidation fine-tune), our method strikes excellent balance
2

---

Published as a conference paper at ICLR 2026
fx
Old
Old Backbone
7old
Zom
Bidirectional
Current Dataset
D.
Cycle
Bieyc 2ond
Conristency
Lu
New Backbone
New
Classification
LeE
Zati
Train the backbone continually
Bidirectional alignment
Cycle consistency
Figure Overview. (1) Train: the current backbone ft leams on Dr (producing know, while frozen ft-1 provides zata) with task loss LCE. (2) Bidirectional alignment: jointly learn distiller D Knew + Zold and adapter A
using Cui- (3) Cycle consistency: enforee Ao D == I and Do ze I with Leye, yielding near-bijective, geoseeny--reeervvnng transport. Old Gaussian prototypes mapped forward A, and all classes are evaluated in new space.
between preserving stability (i.e., preventing drift) and maintaining plasticity, substantially reducing forgetting and maintaining or improving new-task accuracy across CIFAR- -100, TinyImageNet, ImageNet-100,, and CUB-200 under multiple splits. We discuss limitations in the experiments section.
PRELIMINARIES
2.1
PROBLEM DEFINITION
Continual learning (CL) aims to train model on stream of tasks while preserving previously acquired knowledge. In the class-incremental scenario considered here, each task / E {1....,77 T) introduces disjoint label set C, with C, ne, = for i + j. After learning task model must recognize any class in Cia == U C; without a task identifier at test time.
Let ft X' + Rd denote the feature extractor after completing the first / tasks. During training on task the learner has access only to D, {(r;; y/s) yi E C.J. The absence of any prior-task data defines the exemplar-free class-incremental setting.
2.2 PROTOTYPE-BASEET EXEMPLAR-FREE CIL
exemplar-free class-incremental learning (EFCIL), the learner is prohibited from retaining raw samples from prior tasks. In the absence of replayed data, common strategy is to summarize past knowledge with prototypes one representative feature mean per seen class. Focusing on a single transition t-1 + after completing task f-1 the learner stores for each class C1a-1 the prototype.
+-1 n
1
Σ f-1(2)) IED:
D.
where D. collects all examples of class encountered up step f- This summary is compact its memory scales as O( d) for feature dimension anand can be used inference time either directly with nearest-class-mean rule to regularize subsequent training.
Prototype drift. When adapting the backbone from ft-1 to ft on D., the representation changes to fit the new classes and, side effect, the geometry of the feature space shifts. Hence, prototypes computed under f--1 become stale once ft deployed. Denote the updated class mean, its vector displacement, and its norm by
f De
Σ f((x), ZED:
A = p 1-1
8 =
Larger 5 indicates stronger prototype drift, which biases decisions toward recently learned classes.
3

---

Published as a conference paper at ICLR 2026
Because earlier samples retained, g cannot be recomputed exactly; mitigating compen- sating for this drift under the exemplar-fi free constraint motivates the two-stage strategy below.
2.3 PRIOR DRIFT COMPENSATION PARADIGM
widely adopted recipe to handle prototype drift in EFCIL proceeds in two stages.
Stage I: in-task regularization (backward alignment via D). During task t, the old backbone fi-1 is frozen and used as a teacher, while the current backbone ft is trained on the new data Dr the student. Let 9 denote the classifier head (shared task-specific). For each x Dj we define ft=1 Znew = (z) and the corresponding logits lold = g(Zoua, fnow ( ). The student optimized with the usual cross-entropy on new labels and distillation/regularzzttoo term that constrains either features or logits relative to the teacher:
Ls1 = D( CE((new, y) + À ((oew(()) Oota (z))
(3)
where Φ is either z (feature) or & (logit), and D stands for distillation/regulaiizzee operator with >> balancing the terms. This stage constrains the update of f, using only Dr, thereby limiting the growth of o for past classes.
Stage I:: post-hoc prototype transport (adapter learning). After training ft. both ft-1 and fo are frozen and an adapter A is learned on De to map old features into the new space. Concretely, A is fitted paired features (fr-1(x), fe f()) by minimizing
min EZED A| (fi-1<z) fx(=)
with A instantiated as a global translation operator, a class-conditioned translation, or a learnable MLP/linear projector (details vary across works; see appendix[A...I] Once trained, A is applied to the cached prototypes from prior steps to relocate them into the current feature space:
i A (m(-1)
CE C1
These transported prototypes { } are then used by the classifier at inference under f., effectively compensating for the shift induced by the update from ft-1 to Jr.
Our Research Objective. In the two-stage paradigm, the regularization term in Stage (often a distillation loss) pulls the new encoder ft toward the frozen teacher ft-1, whereas the Stage II adapter transports old prototypes forward from the space of ft-1 to that of ft. Functionally, these two modules act in opposite directions; structurally, prior work (Rypešć al. (024)) instantiates the distiller with the same architecture as the adapter but applies it in the reverse direction (Enew
Zold vs. Zold > 2new). Our objective is make this duality explicit already in Stage I: we co-learn forward adapter A and a backward distiller D, during Stage I, enforcing bidirectional alignment and cycle consistency both function (mutual inverses on features) and structure (mirrored/tied parameters), so that prototype transport becomes more accurate by design.
METHODOLOGG
3.1
SETUP
Let fi-1 be the frozen old backbone from task f-1 and fi the backbone being trained at task t. For an input x,
Zold = fx) R² Tnew = fe(x) E RRd.
Unless otherwise noted, evaluation is performed in the new feature space of ft using a Bayes classi- fier (see Appendix A.2), with class statistics estimated from D, (new classes) or transported into the new space (old classes). We instantiate two lightweight maps: distiller D Rd + Rd (new-xold) and an adapter A Rdd -> Rdd (old->new), implemented as linear layers or shallow MLPs. We use the stop-gradient operator stopgrad(-) throughout.
3.2 JOINT TRAINING WITH BIDIRECTIONAL CYCLE CONSISTENCY
We train fc, A, and D jointly on D,, combining standard classification with teacher-student regular- ization and our bidirectional/cycle consistency. Let g be the task-specific classifier head with logits fnew = (. ). For brevity, we denote the bidirectional alignment + consistency cycle module as Bicyc( Zold, Znew) (see Fig. 2)
4

---

Published as a conference paper at ICLR 2026
Bidirectional alignment. We seek (i) backward compatibility by making Snew projectable to the old space via D, and (ii) a forward map A that transports old prototypes into the current space used for ivaluation-withow dragging ft backward. Concretely,
(6) The first term updates fe and D (feature-level distillation, new->old). The second term up- dates A only (detached target), ing the plasticity of ft. In a Gausssian view, minimizing equation 6 reduces transport er- rors Gold->nww A Zold) Znew 2 and Enew-rold Ei] (neww) Zold /| which bound prototype mean/covariance mismatch after transport and help control margin drift.
L = || (Znew) - Zota 2 - [ - stopgrad(Znew))
A chases the evolving new space (old-neww without reduc-
Cycle consistency. While Lai aligns both directions, does not by itself prevent degeneracies (e.g., rank loss in weakly correlated directions). We therefore add a cycle loss that nudges the compositions toward identity on the data support:
Leye = (Zneww))))) stopgrad( (new) )12 + + ([X((())))))) stopgrad(zau))lz
Targets are detached, so Loye stabilizes (A, D) without pulling ft. Spectrally, enforcing =oD / and DeA# zz I contracts the singular values of the composed transports toward 1, curbing rank/energy loss and promoting near-isee wear-isometric geometry preservation. Thus Lsi lowers transport error (alignment) while Laye regularizes the transport operators mnjr-bijection); together they yield faithful prototype transport and empirically reduce forgetting without sacrificing plasticity. We denote the weighted sum of the bidirectional alignment and cycle-consistency losses by:
Bicyc (Zald, Znew) == Am Las + Aeyc Laye
We analyze the cycle objective under centered features and full-rank covariances on the data support, passing to whitened variables žold = End
--1/2 2old and &new Wnew Znew- this space E| [een & = I, and the expected cycle error equals the squared Frobenius distance of Åπ to T. We now state the resulting contraction property.
Theorem (Cycle contraction). Let Σou = E[zoldZola) T and Enew = E[ [neew be full-rank on the data support and define whitened variables žold = 2old Zold. Znew = Mnoh Znew with induced maps
Ā /- A/1/22 and Î =// 1//// Let M = Añ T. If the features are centered, then:
E Mnnw new 112 = ||
By Mirsky/Hoffman-Wiilee
Horn & Johnson 2013) Σd= (ox (ĀD) - 1)2 s I and < JM J.. In particular, if || then 1 - | S of <
hence max; Iox (AD) 1 1|
1 ||M and x(AD) <
M+|
T- M [a Consequently, minimizing Loye drives the singular values
of AD toward 1 on the data support, preventing rank loss and preserving geometry. Proof in Ap- pendix A.2.I
Corollary (Decision stability for classification). Let old-class statistics be transported as A Apg-1 [1 and (for linear A) 52. AA Assume evaluation uses the Bayes rule with Gaussian class-conditionals (nC, Σt) and priors Te, with log-scores f. (x:) as in Appendix A.1.2 Define mean
transport errors 8, == | - v ((zy) 1. If the alignment error 22 Sold-neww = Alll
know 11 and the
cycle error &cyc,oew = E ADD Anew Znew Znew are small, then:
de S / Fold-noow
ES - -E'-E'133 S CIV= + C2 Ecyc.new.
(10)
For any class pair (i,j) and any x. let mzy (z) = 14((x) (x)( be the Bayes margin. Then the induced change in log-odds satisfics (4, (2) (4, (6)1 x Cp(8, + 8,) + CzI - >¹
Consequently, if C₁u (8, + 6,) + CzJlš' E'1l2< (z), the Bayes decision between i and j at remains unchanged after transport. Proof in Appendix A.2.2
Pitfall of anti-collapse loss. For features z E I
BxS let 3L (2- (2-2). The AdaGauss
anti anti-collapse loss (Rypešé et al. 2024) is
8
Lac =
min min(chol(E))) B))
(11)
S
5

---

Published as a conference paper at ICLR 2026
Table 1: Average incremental (Ama.) and last-task average (Ana) accuracy (%. mean + std. five runs) on CIFAR-100 and TinylmageNet when training the feature extractor from scratch. Best results are bold.
CIFAR-I00
TinyImageNet
Method
T=10 10
T=20
T= 10
T-20
Aust
Amc
Alss
Aic
Auast
Ainc
A
Ainc
EWC
1.99=1.9
50.411.7
17.011.6
34.241..
18.5+1.8
34.31:..
11.3119
26.8:2..5
LwFrocvie
31.91111
51.8.1..5
17.611.2
39.211.77
27.1
39.61.2.0
15.2.1..6
31.5.12.1
SDCcvPR20
40.6=0.9
56.2+13
32.3+100
46.6+14
29.5=1.1
43.8+1.5
26.3+1.2
40.6+1.7
PASScvPR21
30.81:.2
48.311.1
17.610.8
31.11113
24.520.6
39.5+1.0
18.5+14
30.4119
FeTrILwAcv22
34.9=0.5
51.2+1.1
23.3114
37.9+122
31.0=0.9
45.3+1.8
25.9
39,9#1.2
FeCAMNennPS22
32.410.5
48.71009
21.1111.0
34.511.3
30.910.9
44.9114
24.910.8
37.9114
EFCICLR21
43.5+00.8
58.1+1.2
32.4+0.9
47.0+t1.3
34.5+1.1
47.9+1.5
28.4+1.2
42.1+1.6
ADCcvPR24
46.5=1.2
61.41:.6
35.1#1.4
51.7#1.8
32.3=1.5
43.0#1.9
18.1
36.0+2.1
LDCEcoV24
45.4=1.6
59.5#1.9
35.5+1.9
51.9+2.3
34.2=1.1
46.8+1.6
24.9+2.2
38.2+2.7
AdaGaussNeur1522
46.8-1.2
60.91:00
37.9110
54.410.8
32.9=0.9
45.811.
27.51:12
39.51111
DPCRICML2025
50.2=00.7
62.8+:111
39.8+122
54.8+0.9
33.9+188
46.9+0099
25.6+:0.7 39.340.6
BiCyc (Ours)
50.6=0.9
63.2+:1.3
41.5+1.11 56.5+1.3 35.4=0.8 49.1+14
30.2+1.11 44.2+11.
In practice, mini-batch E can be non-SPD or rank-deficient, causing Cholesky failures and poten- tially inflating scale near ill-conditioning. We enforce SPD via symmetrization and shrinkage, with jittered Cholesky and eigenvalue flooring as fallback:
S = L (-+E"), Đ= tr S
1l,
(12)
and, for very small batches, we optionally use diagonal approximation Eddngg = (alg(dag(2)) (D)). The robust objective is
Crob
min(chol(E)a. 3).
(13)
Total Stage-I loss and gradient routing. terms yields:
Combining the classification, cycle, and anti-collapse
Liotal LCr (Cnew, v) + +Bicy knew) + a Crob
(14)
learn new classes
Here, Lce and the first term of equation 6 update fo (and D); the second term of equation uupdates stabilizes (A, D) without reducing the plasticity of fi- Importantly, if gradients from the adapter are allowed to flow into ft, A and D become adversarial, severely weakening D's regularization and causing sharp performance drops. After Stage I, we freeze ft-1, fe, and D, and perform a low-learning-ral fine-tuning of A on D, to sharpen transport without re-optimizing from scratch.
only (detached target); and equation 7
4 EXPERIMENTS
Baselines. We benchmark our approach against broad set of exemplar-fred inass-incremmntall learn- ing (EFCIL) methods. Classic regularization baselines EWC (Kirkpatrick et al., (001)) and LwF (Li & Hoiem 2016) are executed using the reference OCL implementation (Mai et al. 2022). Con-
approaches- SDC YYu et al.. 2020), PASS (7hu al. 2021 FeTril. (Pe- tit ct al. 2023), FeCAM (Goswami al.) (023), EFC (Magistri et al. 2024), ADC (Goswami al.) 2024). LDC (Gomez- Villa et al. 2024), and AdaGauss (Rypesé et al. 2024 are run with the au- thors' public codebases as distributed via FACIL (Masana al. 2023 PyCIL (Zhou et al. 2023) or the official repositories. Unless otherwise noted, we preserve the original data augmentations and default syper-parameter reported by each paper.
temporary state-of-the-ee
Implementation details and reproducibility. We build on the public AdaGauss codebase and add the components introduced in this work. Unless stated otherwise, all experiments use ResNet-

---

Published as a conference paper at ICLR 2026
Table 2: Average incremental (Amm)) and last-task average (Ansa accuracy (%. mean - std. over five runs) on ImageNet- 100 and CUB-200. Best results are bold. ! results excerpted from (Gomez Villa al. 2024 : results excerpted from (He et al. 2025).
ImageNet-100
CUB-200
Method
T=10
T-20
T=10
T=20
Aust
Amc
Alsa
Aiec
Aust
Ainc
Aust
Anc:
EWC
25.1
40.6:13.33
13.7:2.1
29.2:2255 15.8:0.7 32.6:0.55 12.3:0..8 27.2:10.6
LwFecovie
33.4
51.54:1.6
18.6.1.66
41.3:119 30.411.11 46.1 19.41..6 34.7.11.8
SDCcvPR20
1..4
50.1+1.6
19,4#1.0
36.5+1.4
50.3+1.3
60.5+1.2
27.9+1.4 40.1
PASScvPR21
1..4
45.71:0.2
14.4::.2
31.710.4 27.0110.9
42.310.9
18.1=1. 36.911.1
FeTrlL
36.2+:1.2
52.64-0.6
26.6+:1.5
42.44:211 36.940077
48.2:10.6
34.6+1.0
45.3+009
FeCAMNeu
38.711.0
54.810.55
29.0.1..3
44.612.0 40.2.0088
54.911.0
36.2.1..1 48.911.3
EFCICLR21
50.9+1.1
6,,3+112
38.6+1.2
50.5+1.55 51.040.6
63.3+0.7
46.1+1100
59.3+1.3
ADCCVPR24
3..
55.5
25.1
43.4 49.5+0.9
58811.1
35.4114
48.3+14
LDCEcoV24
11.2
+9.4'
28.5+1.7
46.5+2.7 47.5+0.7
55.7+1.3
27.2+1.1 39.8+2.1
AdaGausSNaurrPS24
51.1=112
65.0111.4
42.61:.6
57.4119 52.911088
63.4113
45.0
57.01:10
DPCRICML2025
49.9110.8
64.811.11
37.311.6
54.710.77
BiCye (Ours)
52.7.10.9
66.81114
43,8.11.4 58.2.1.88 53.7.0..7 64.0. 43.7 55.911.2
Table 3: Last-task average forgetting (Fim)) (%, mean + std. five runs) of drift compensation methods when training the feature extractor from scratch, Best results are bold.
CIFAR-100
Tiny ImylImaeNNet
ImageNet-100
CUB-200
Method
T=10
T=20
T=10
T=20
T=10
T=20
T=10
T=20
Finss
Fin
Fus
Fist
Fiss
Fiast
Fus
Fust
LWFECCVIG
23.
31.211.8
21.9+1.9
33.5=2.4
42.1
48.1
16.5#1.1
21.7+1.4
SDCevpR20
34.8+1.7
35.9+1.9
25.1 29.4-2.1
44.6+2.00
54.44:2.3
10.9+1.3
17.3+1.1
EFC1C1824
23.111.1
24.71:1.
23.512.4
30.1=3.0
21.511.9
23.8:2..5
10.710.7
14.811.7
ADCevrR24
21.91:1.1
31.0+:..6
30.24..0
36.8
32.41166
33.41:.8
12.8
21.3
LDC2oev24
21.7+:1.9
25.6+2.3
24.74:25
30.7+2.1
25.7=1.7
32.9+2.3
13.6+1.2
23.9
AdaGausSMmur1S24
16.7:114
21.0::..5
18.711.2
23.1=1.
20.610.9
22.911111
11.610.77
16.9113
BiCye (Ours)
13.5+:1.3
16.6::0.9
12.0+0.9
18.9=1.1
18.211.
20.81:14
11.3+0,9
17.5+1.
backbone trained from scratch (He et 2016) with a batch size of 256 images per iteration, fol- lowing AdaGauss. For CIFAR-100 (Krizhevsky) 2009). TinylmageNet (Le & Yang, 2015). and ImageNet- 100 (Deng et al. 2009). we train for 200 epochs using SGD (fixed learning rate 10-1 weight decay 5 x 10 ") For CUB-200 (Wah et al. 2011]. we adopt a split learning rate: 1 x 10- for the backbone and 1 x 1 for the heads. The distileer and adapter are trained with learning rate 5 x 10-2 and weight decay 1 x 10-4 In the from-scratch regime we set 5=== and > the learning rate is decayed by a factor of 10 at epochs {60, 120, 180). After Stage I, we fine-tune the adapter for 30 epochs using SGD (initial learning rate 1 x 10-2, weight decay 5 x 10 10-4).
All other hyperparameters follow AdaGauss verbatim. In particular, we adopt its default settings for prototype storage and sampling, and therefore do not discuss additional computational overhead. For completeness, we note that the public AdaGauss code reports TinyImageNe results averaged over splits formed from the first 100 classes, which is slightly misaligned with common balanced partitions. To enable an apples-to-apples comparison, our tables present the corrected numbers under the standard balanced partitioning.
Evaluation metrics. We report three standard measures: the last-task average accurucy Apst its rumning mean. the average incremental accuracy Ane and the last-task average forgetting Fast- Dataset specifics, hyper-parameter schedules, and metric definitions are provided in the Ap- pendix A.3.2

---

Published as a conference paper at ICLR 2026
Accuracy
+0.0
Abe 951
UBA
62
VIL
1
+373
12
3K3
44
+254
14
44
+LR
+23
-117
+370
424
Figure 3. CIFAR-I00 (T=10): Per-step. per-task accuracy gains (A, percentage points) of Ours over Ada- Gauss, EFC, and LDC. Improvements concentrate on earlier tasks, indicating stronger retention and reduced forgetting.
4.1 MAIN RESULTS
Tables and 2 report fhiiing-from-scratee results balanced CIFAR-100, TinylmageNet,
ImageNet- 100, and CUB-200 (te over five runs). CIFAR-100: compared to AdaGauss,
gain +3.8/+3.3 pp T=10 and +3.6/+2.1 pp at T=20. DPCR# is competitive, but method
still slightly leads on all CIFAR- 100 settings (e.g., +0.44+1.44144 pp at T- and +1.7/41.7 pp at 20))..
TinyImageNet: improvements over AdaGauss are +2.5/43.3 pp 7= and +2.7/+4.7 pp T=20; the margins over the second-best (EFC) are +0.9/+1.2 pp 17=)1) and +1 +1.8/+2, pp (7=20). DPCR again trails our method, with gaps of about +1.1
7'=10 and +4.6/+4.9 pp at 7'=20. ImageNet-100: vs. AdaGauss we obtain +1.6/+1.8 pp T'0 and +1.2/+0.8 pp at T at 7'=10 our Ams is best (runner-up LDC +1.3 pP). while Anc is 2.6 pp below the best (LDC'). Under our protocol, DPCR* is clearly weaker than AdaGauss and ours: at T=10 it trails our method by about 2.8/2.0 pp A (and is already slightly below AdaGauss by 1.2/0.2 pp), while 7-20 the gap ours further widens to 6.5/3.5 pp (with AdaGauss still ahead of DPCR by 5.3/2.7 pp). For T-20 we achieve the best Alast and Ainc (runner-up AdaGauss: +1 +1.2/40.8 pp). Under our protocol, rerumning public LDC code at T=10 yields 41.7 + 1.5% and =m. 1.77% CUB-200 (ImageNet pre-trained): our performance is close to AdaGauss (vs. AdaGauss: +0.8/+0.6 pp at T=10,- 1 pp at T=20), while the 20-split setting trail EFC by 2.4/3.4 pp). DPCR does not report its CUB-200 hyperparameter configuration under our training protocol, so the cor- responding entries marked "_" in Table 2 With pretrained backbone, practitioners typically adopt a very low backbone learning rate, which keeps cross-task feature drift small and thus limits the incremental gains of our method.
Per-step advantage on CIFAR-100 (7=)0). As shown in Figure 3 across three baselines, our method shows consistently positive accuracy gain throughout training, with the largest gains on older tasks (lower-right region in each heatmap). Against EFC, margins often exceed +15-20 pp at mid/late steps; versus LDC, we sustain +6-11 pp on most old tasks; and relative to AdaGauss we obtain +5-10 pp improvements that persist to the final step. The concentration of positive A on early tasks indicates significantly smaller forgetting:
Table 4: CIFAR-I00, Contributions of Cui and Leye.
Components
T=10
T=20
Lm
Lae
Ana(%)
Amc(%)
Ama(S)
Anc(%)
x
46.8+12
60.91LL0
37.911.0
54.4=0.8
49,411..0
62.111111
40.2
1..8=1.0
47.8+1.1
61.8+1.0
39.0+111
54.9-0.9
50.61009
63.2-11.3
41.5.1111
56.5
accuracy on initial tasks decays far less under ours while recent tasks remain competitive, yielding superior plasticity-stabiiee trade-off.
4.2 ADVANCE IN FORGETTING
As shown in Table across the three balanced, achieves the lowest forgetting. On CUB-200, however, most methods fine-tune from a pretrained backbone, so the gaps forgetting are much smaller than in the cromtscratch regime.
datasets, our method
8

---

Published as a conference paper at ICLR 2026
Table 5: Adapter Strategy vs. Architecture: Ablation on CIFAR-100
(a) Direct prototype projection vs. projection with post-training adapter fine-tuning. Arrows indicate the preferred direction.
(b) Adapter/Distiller Architectures: MLP shows ab- solute scores, others report A vs. MLP
Ablation
T=10
T= 20
Ablation
T=10
T=
Aus f
Fint 4
Auat" f
Fiut 4
Aua
1
4
Aua
Flat
MLP
50.6
13.5
41.5
16.6
Direct projection
49.9
17.6
Linear
-3.5
+1.2
-42
+1.3
fine tuning (vs. Direct)
+0.7
-1.7
+2.6
-1.0
CrossAtteation
-2.7
-1.0
MoF.
-3.7
-3.6
-2.8
-4.9
4.3 EFFECT OF Lai AND Leve
Notably, our approach delivers especially strong preservation of prior knowledge when training from scratch.
As summarized Table 4 on CIFAR-I00 enabling either loss improves both Alsst and Aic over the AdaGauss baseline, and enabling both yields the best results across the 10- and 20-task splits. This pattern matches the roles established in Sec. 3 Lui (Eq. 6 reduces the new told caature-transpor errors that bound prototype mean/covariance mismatch, while Leye (Eq. 7 contracts the spectrum of AD toward 1, mitigating rank loss and promoting near-isometric transport. Used together, they simultaneously lower transport error and preserve geometry, explaining consistent gains in Austt and Ame. Empirical diagnostics corroborate this: Figs. 4 and (CIFAR- T=10) show lower symmetric KL between transported and ground-truth class Gaussians and singular-value spectra of AD that are tighter and more concentrated at than AdaGauss, indicating better distributional transport and more stable decision boundaries.
4.4 ABLATION: DIRECT PROJECTION VS. POST-TRAINING FINE-TUNING.
The adapter learned via bidirectional cycle consistency can be used as is to map old-class prototypes into the new space. We compare this "Direct projection" with additional post- training fine-tuning of the adapter. On CIFAR-100, direct projection achieves Ams==99.. and 11.2 split, and =3.9 9 and = at 20 -task split. Fine- tuning yields consistent gains: +0.7 points in Atsst and -177 in Fins at 10 -task split, and larger -2.6 1.0 at 20-task split. These results indicate that while the yyllecconsistent adapter already provides strong zero-shot projection, a adjustment further aligns prototypes to the new feature geometry- an effect that becomes more pro- nounced as the task sequence lengthens.
AdaGauss
Ours
10 -task
Figure 4. Task-O stability via SymKL (J). On the fixed task-0 data, we compare Gaussian fits from models f=1...=1... the task-0 reference using symmet- KL (Eqs. BT): mean over classes. Our method maintains smaller divergence i.e., a closer match to the original distrihution- than AdaGauss.
brief post-trainioe
4.5 ABLATION: ADAPTER/DISTILLES ARCHITECTURE
Because method learns bidirectional maps between old and new feature spaces, the adapter/distiller architecture di- rectly affects performance. Beyond the linear shallow MLP adapters common in prior work, we test lightweight but richer alternatives-crosss-eee -ttttatttnton and sparse MoE-tt to probe whether conditional//oolinear mappings better track representation drift. reepootss CIFAR- 100 results for the 10- and 20-task splits. Across both splits, multilayer adapters consistently outperform a single linear map: rela- tive to an MLP baseline, the linear variant lowers Aum by 3.5-4.2 points and increases Fist by 1.2-1.3 points. Within the multilayer family, cross-attention favors stability, reduc- ing forgetting (AFisst -0.7 to -1.0) at the expense accuracy (AAttt to -49) with only moderate accuracy drops (-2.8 to -377). If new and old features differed by a single global affine transform, linear adapter would suffice; the observed trade-offss instead point
Adass
OURS
Figure 5. Near-isometry on task-O under continual updates. AD-% in [0.9. 1.1] for models after t=1.. 9; our method consistently preserves geometry better than AdaGauss.
2.7 to -6.0), whereas sparse MoE delivers the largest forgetting gains (-3.6

---

Published as a conference paper at ICLR 2026
content-dependent, anisotropic drift, which conditional/nonlineaa adapters model more faith- fully. All variants share identical training schedules; a parameter- matched linear control a natural follow-up to isolate capacity from architecture.
4.6 PROTOTYPE DRIFT FROM ORACLE MEANS CIFAR-100
To assess how well each method preserves old-class geometry, Fig. reports proto- type drift on CIFAR-100 100 with the 10-task split. After training Task 9, we freeze the backbone and, for every old class c, com-
dURS
OURS
A6362055 LOC
pute the maintained prototype ee and an oracle prototype de given by the empirical feature mean all samples of class c un- der the final backbone. The drift for class c is defined as || 12
Source look
Fan 67
(a)
(b)
Panel 6a averages this drift the ten classes of each source task, while Fig- ure 6bb plots the full per-class distribution over all 90 old classes. Our method yields both lower average drift and a tighter dis- tribution at small values than AdaGauss and LDC, indicating less accumulated dis- tortion of old-class prototypes.
Figure 6. CIFAR-100 (T=10). Drift between maintained
prototypes and oracle protolypes (empirical class means) after completing Task 9. For each of the 90 old classes (Tasks 0-8 8). we measure & distance feature space tween the maintained prototype and its oracle prototype. (a) Per-source-tase average drift for the three methods. (b) His- togram of per-class drift over all old classes.
4.7 PARAMETER OVERHEAD IN THE 64-DIMENSIONAL SETTING
Setup. Following AdaGauss (Rypešć et al. 2024). all experiments use a ResNet-18 backbone fol- lowed by 512-66 linear reduction and two-layer MLP projector D (new~old) in the S=64 space. Our bidirectional variant simply adds a second MLP A (old->new) with the same archi- tecture. Both A and D are MLPs F$ g mS=2048).
R with width multiplier m=32 (hidden size
Parameter count. two-layer MLP with biases in this setting has
# #paramSMLP 2m5² (m+1)S
#paramsMLP 264,256
for S=64, m2 Thus AdaGauss already uses one such projector D (x 0.26M parameters), and our bidirectional version adds one more (A), for extra
A#params 264,256
on top of the published AdaGauss model. Since standard 18sNet-18 backbone has about 11M parameters, the additional adapter increases the total parameter count by only (We use this shared 14-dimensional configuration in all experiments.)
CONCLUSIONS, LIMITATIONS, AND FUTURE WORKS
Conclusions. We presented bidirectional crift-compensation framework for exemplar-fre free class- incremental learning that jointly learns old to new and new to old projectors with -ragdrnd gating and cycle consistency. Our analysis links least-squares projectors to CCA and shows how re- ducing alignment and cycle error stabilizes prototype margins. Experiments across standard EFCIL benchmarks demonstrate the new ofate-of-the-ar forgetting reduction while maintaining excellent new-task accuracy.
Limitations.
The current formulation assumes centered features, and second-order (Gaussian) pro- the data support. The method may
totype statisties; its theory is local to small alignment be sensitive to covariance estimation and hyperparameters in low-data regimes.
Future works. We plan to develop uncertainty- aware and class iimallnnc--robenn prototype trans- port, and derive non-asyoe asymptotic ceneralizzaiionfoogettin bounds beyond Gaussian assumptions. We also plan integrate test-time adaptation and multi-modal backbones under strict memory budgets.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Rahaf Aljundi, Francesca Babiloni. Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights reducing abrupt representation change in online continual learning. In ICIR, 2021.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. ICCV, 2021.
Hao Chen and Yin Xia. A normality test for high-dimensional data based on the nearest neighbor approach. Journal of the American Statistical Association, 118(541):719-335 2023. doi: 10.
1080/01621459 20221. 19535077.
Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, and Pheng-Ann Heng. Flattening sharp- ness for dynamic gradient projection memory benefits continual learning. NeurIPS, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In CVPR, pp. 245, 2009. doi: 10. 10.1100//CVPP.220955206666
Bruno Ebner and Norbert Henze. Tests for multivariate normality critical review with emphasis on weighted (2 /-sttaistics. TEST, 29()) 892, 2020. doi: 10.1007/s 1771117009000000--11
Alex Gomez-Viile Villa, Dipam Goswami, Kai Wang, Bagdanov Andrew, Bartlomiej Twardowski, and Joost de Weijer. Exemplar-fie continual representation learning via learnable drift compen- sation. In ECCV, 2024.
Dipam Goswami, Yuyang Liu, Bartlomiej Twardowski, and Joost van de Weijer. Fecam: Feature covariance and mahalanobis metric for incremental learning. In Neurips, 2023.
Dipam Goswami, Albin Soutif-Cormerais, Yuyang Liu, Sandesh Kamath, Bartlomiej Twardowski, and Joost van de Weijer. Resurrecting old classes with new data for exemplarfree continual learn- ing. In CVPR, 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.
Run He, Di Fang, Yicheng Xu, Yawen Cui, Ming Li, Cen Chen, Ziqian Zeng, and Huiping Zhuang. Semantic shift estimation via dual-projection and classifier reconstruction for exemplar- class- incremental learning. In ICMI. 2025.
Roger Horn and Charles R. Johnson. Matrix Analysis. Cambridge University Press, 2nd edition, 2013.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, and et al. Overcoming catastrophic forgetting in neural networks. In Proceedings of the National Academy of Sciences, 2017.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory G. Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Trans. Pattern Anal. Mach. Intell., 44(7):336--385, 2022.
Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS231N, 2015.
Zhizhong Li and Derek Hoiem. Learning without forgetting. In ECCV, 2016.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning- NeurlPS, 2017.
Simone Magistri, Tomaso Trinci, Albin Bagdanov. Elastic feature consolidation for cold start exemplarfree incremental learning. In ICI.R, 2024.
Joost de Weijer, and Andrew D.
11

---

Published as a conference paper at ICLR 2026
Simone Magistri, Tomaso Trinci, Albin Soutif- Cormerais, Joost van de Weijer, and Andrew D. Bag- danov. Efc++: Elastic feature consolidation with prototype re-baalnn for cold start exemplar- free incremental learning, 2025. URL htRLltts///xxii..rrr.. http://wwww...rraaa///aas///////
abs wttp:///////b//3...11333.
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online
continual learning in image classification: An empirical survey. Neurocomputiing 469:28-51, 2022. ISSN 0925-2312. doi: http:///bbboog////...oooumoomm.d
URL https:// www. www.scienccc. rect. cww.coooo/oo.oo.. http:////..aaaaaoccc../////////2 www.:/acccc.cttt.taaaaaraoaaaaar
Mare Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, and Joost van de Weijer. Class-incremental learning: Survey and performance evaluation image clas- sification. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(5):5513-5533,
2023. doi: 10. 10.11000 1000TPAA1120222.2134777
German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Con- tinual lifelong learning with neural networks: A review. Neural Networks, 1 113:54-71, 2019.
Grégoire Petit, Adrian Popescu, Hugo Schindler, David Picard, and Bertrand Delezoide. Fetril: Feature translation for exemplar-free class-incremental learning. In WACV, January 2023.
Quang Pham, Chenghao Liu, and Steven Hoi. Dualnet: Continual learning, fast and slow. NeurIPS, 2021.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning learn without forgetting by maximizing transfer and minimizing interfer- ence. In ICLR, 2018.
Grzegorz Rypeść, Sebastian Cygert, Tomasz Trzciński, and Bartlomiej Twardowski. Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning. NeurIPS, 2024.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. In ICLR, 2020.
Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for contin- ual learning. In ICML, 2018.
Yee Whye
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML, 2018.
Haizhou Shi and Hao Wang. A unified approach to domain incremental learning with memory: Theory and algorithm. In NeurIPS, 2023.
Habib Slim, Eden Belouadah, Adrian Popescu, and Darian Onchis. Dataset knowledge transfer for class-incremental learning without memory. In WACV, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention all you need. In NeurlPS, 2017.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. 2011.
Liyuan Wang, Xingxing Zhang, Qian Li, Jun Zhu, and Yi Zhong. Coscl: Cooperation of small continual learners is stronger than big one. In ECCV, 2022a.
Zhenyi Wang, Li Shen, Le Fang, Qiuling Suo, Donglin Zhan, Tiehang Duan, and Mingchen Gao. Meta-leamiig with less forgetting on large-scale non-stationary task distributions. In ECCV, 2022b.
Zhenyi Wang, Yan Li, Li Shen, and Heng Huang. unified and general framework for continual learning. In ICLR, 2024.
Zifeng Wang, Zheng Zhan, Yifan Gong, Yucai Shao, Stratis loannidis, Yanzhi Wang, and Jennifer Dy. Dualhsic: Hsic-hottleneck and alignment for continual learning. In ICMI., 2023.
12

---

Published as a conference paper at ICLR 2026
Enneng Yang, Li Shen, Zhenyi Wang, Tongliang Liu, and Guibing Guo. An efficient dataset densation plugin and its application to continual learning. NeurIPS, 2023.
Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang. Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In CVPR, 2020.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICMI,, 2017.
Da-We Zhou, Fu-Yun Wang, Han-Jia Ye, and De-Chuan Zhan. Pycil: a python toolbox for class- incremental learning. SCIENCE CHINA Information Sciences, 66(9):197101, 2023. doi: https:
Fei Zhu, Xu- Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu. Prototype augmentation and self-supervision for incremental learning. In CVPR, 2021.
A APPENDIX
This appendix provides additional details, proofs, metries, and supplementary results. It organized as follows:
Method details.
A.1.1
(transport view of drift compensation), AA.1.2 (Bayes classifier),
A.3133 (pseudo-coe)
Theory. $A.2.1 (Proof of Theorem 1), A.222
(Proof of Corollary 2).
(accuracy metrics), similarity metrics), $4..44 (near-isometry metric).
Experimental protocol. A.3.1 (sctup), $A.3.2
A.3.3
(distribution-
Additional results and ablations. tiller/adapter architecture ablations), of classifier).
$A.4.
(additional visualizations).
AA.4.
(dis-
A.4.3 by hyperparameter sensitivity), A.4.4 (choice
Additional discussion and diagnostics. A.5.I sian modeling in prior work), $A.3.2 (Gaussianity diagnostics), A.5.3 (t-SNE snap-
(prototype- based EFCIL and Gaus-
shots). AA..4 (intuitive view of bidirectional cycle consistency), $A..5 AdaGauss, full- covariance prototypes, and robustness).
Limitations and disclosure. $A.6. (ImageNet-Is K limitations), closure).
A.6.2
(LLM usage dis-
A.1 METHOD DETAILS
A.1.1 PROTOTYPE-DRIFF COMPENSATION: A TRANSPORT PERSPECTIVE
In the main paper, we adopt vectorial notion of prototype drift. For each previously-seen class E C1.-1, the backbone update from f to fi induces the feature-mean displacement
! 4 1 !-1
8
|
(15)
where - P.. - (xr is the (unknown) class mean under the updated encoder ft. Because EFCIL forbids storing past raw samples, 5.o cannot be recomputed exactly, and cached prototypes ! become stale once Sa is deployed.
Most drift-compensation pipelines in prior work follow a two-stage pattern: Stage constrains the backbone update using only D.,
Lsi E Bi.yiED CE(g(f(z)) y) D (2), Pold (z))
(16)
with & E (S(),, fo)( and D generic distillation/regularieer Stage II then learns forward adapter A, (with frozen f:) by aligning paired features ft D., and transports old prototypes:
A, E arg min ((f-,)A((f--())) f((x)) A
p
Adp
(17)
13

---

Published as a conference paper at ICLR 2026
This transport view unifies existing drift-compensation recipes each be seen as instantiating either global/class-wiss translation A²(z) A or learned projector A, applied cached prototypes.
TTansporr- summary of prior methods. Below cast representative EFCIL approaches as special cases of Eq. equation 17 For consistency, we denote the encoders by ft-1 and ft (some works write Pt) and use D, for the current-task data.
Semantic Drift Compensation (SDC) Yu et al. (020). SDC estimates global shift from new-task samples and uses as a translation adapter:
[(((() S--((a)) A, = 1ig = Me + A D.
RED;
Adversarial Drift Compensation (ADC) (Goswami et al. 2024). For each old class c, ADC selects a current- sarrent-samplee & that adversarially driven towards the vicinity of u - (in the old space), and takes the resulting pairwise feature gap as class-wise translation:
A f(()e) fe-1(
(2)
M Å
Learnable Drift Compensation (LDC) (Gomez- Villa et al. 2024 LDC directly learns projector the adapter:
Ga E arg min EEED, [e) fc Ac(z) Go(>), at Gdue ())
This captures non-linear, potentially class-dependent deformations.
EFC (EFM-weighted transport). (Magistri et al. 2024 EFC computes a weighted aver- age of per-sample shifts using pseudo-metric induced by the Empirical Feature Matrix E (estimated after task t t- Let || v E Each E D, casts vote for class c with weight
[1fr-1(z;) - 11
Wee = exp
20
yielding the class class-wise transport
A EnED, Wa,s (fe(z,) fx;)
H t-1
+ A
En,DD We,s
AdaGauss. Like LDC, AdaGauss first learns forward projector Go by aligning paired features on D,:
Go E arg min EzED, (esp
A((2) Go(z).
Unlike LDC-whiel directly transports old means via μ = Gd --daaas els each old class Gaussian and transports the distribution by Monte Carlo push- forward (see Alg. []] Stage II):
AdaGauss mod-
Um Ma Σ)-1), Un Go(um) Ac(um):
..... M,
followed by re-estimation in the new space:
M
M (pm - T...
Um:
S
Σ m=1
(um
M
M
When Go (equivalently A,) is affine, this reduces in closed form pushing moments (f 50 (Ajt!-1 A
14

---

Published as a conference paper at ICLR 2026
A.1.2 BAYES CLASSIFIER IN THE NEW FEATURE SPACE
Let = fx(x) E F be the feature of input a at task and let each seen class Ci:e be represented in the new space by a Gaussian prototype (ue, Σe) (means and covariances trans- ported/estimated as in Sec. A.I.Ip. The Bayes score is the class conditional quadratic form
8c(x)) = (=- (=-Ma) De (= (z-Me),
(18)
and the lask agnostic prediction (TAg) is
(atg(a)) = arg min (x(x)). cEC1a
(19)
When a task aware (TAw) report is required, we restrict the argmin to the current task's label set C₁:
grAw (2) =
arg min se(x).
(20)
nECs
A.1.3 PSEUDO-CODE FOR OUR ALGORITHM
Algorithm I specifies the end-to-end procedure for each task t: learns the current backbone fo under classification with bidirectional aligmment and cycle consistency (via A and D), and updates the class prototypes by transporting stored Gaussians into the current feature space for inference.
A.2 THEORY
A.2.1 PROOF OF THEOREM 1
Proof of Theorem I (Cycle contraction). Let := AD / and note that by definition of whitening, El nnew ] = I (features are taken to be centered; otherwise replace = by its centered version). Then
M Enew 112 E2 T M M Tr(M" M E((nne
Tr(M M) [M
(21)
which yields the stated identity.
For the consequence, write the singular values of AD {ax) Since M = AD - I, Weyl's inequality gives max I - 1|s ||M]] ||M][e- Thus minimizing Laye = M Enew 113= [ forces || = 0, hence 0% - for all k. In particular, when the loss is small, all singular values of AD lie in tight neighborhood of preventing rank/energy loss and preserving local geometry
on the data support.
A.2.2 PROOF OF COROLLARY 2
Proof of Corollary 2 (Decision stability for classification). Fix a class e and abbreviate μ = pe: = Σ 7 = 1/g Σ 5. Au = j 14, AΣ = E - Σ. The Bayes log-score is
(e(x) log ne log det Σ
(x n1)
(z= pd) A first-order expansion in (Aμ, AΣ)
2
gives the perturbation
1 5 -
Ex(x) (x()) - Th +
(x- p)) + Rx(x), (a- T
(22)
where R_(x)) = 0 || by the identities log det Σ- + AΣ) llog + Tr(E AE) - 0(IAE)) and (Σ -1 - A'' 0( ()
Taking absolute values and applying Cauchy-Schwarz and spectral norm bounds,
[(((x)) - tc(x)
< ce | + c || ||= - 4 + | 1 1 + O11 + |
(23)
for constants (( cf2) depending only on || || (and dimension via standard inequalities). For a pair (i,j), the log-odds perturbation satisfies by triangle inequality
(e. - Ł2, - ((4-19)) 22) s +| |++ (2)) (2)) (24)
15

---

Published as a conference paper at ICLR 2026
Algorithm Bidirectional Cycle Consistency (EFCIL)
Inputs:
Task stream (Pa P old backbone ft-1 (frozen); current backbone ft (leamable); classifier head gi adapter A _edh - R. (old (olw: distiller D _ +R (new->old);
hyperparameterss Ani Aeye, learning
v, IJa, TJD; batch size B:
anti-collapse hyperparameters K, 9, per-class sample count M for distribution transport.
Outputs: Trained ft. A, for each transported means covariances for inference.
Initialization: Copy fa < fi-1; randomly initialize A, D; freeze fi-1.
for L...., ' do
# Stage Joint training on current task Dr
while not converged do
Sample minibatch {(x, v) D.-
Fold f- (e-1(z)
no gradient
Fnew <- [T(
Eaom <- ((
Bidirectional alignment:
Csi [I Zota 50 A(et||A(aaa) A(200)
Ascm
(dettcc))
Cycle consistency:
Leye + || (new )} 2now
(thatarh)) (
Sold
((latach)
Classification: LCE CC((Ceenn (new: V)
Robust anti-collapse on features:
(Znew 2) (Znew 2); 5+ 1 (x++))):: E+ tr(2.) B
Σ
1 sT
2
d Σe min(chol(S))x,,))
Lac
Total: C Lee Auhri + Acyeheye Crob
end while
# Stage II: Distribution transport via sampling + adapter fine-tuning (optional) D. with small LR by minimizing [|A(zau) 2now
Freeze ft-1, fr, D; fine-tune for each old class E C1:--1 do
(detach)
Load stored stats (Je [-1 D.-1).
Sample old features: draw U {um ) JM NN (((--1,2?-1)
Push-forward to space: V = (Mm J.M with Um +- A A(um).
Re-estimate in space:
_e M7 > mm1 Um.
M-T Em (vm (2m - R..TT
end for
Estimate new-class stats under fa from Dz: (mé, 2x) for all C.. Build new prototype collection using ((et heche and ((Me, Ei) Jesd. Store (nE, E for the next task.
end for
where Cu, Ox: absorb bounded factors of || (oll on the evaluation support. Now set d, = - - and invoke the cansport-fidellin bounds used in the corollary,
8 S / V
1 E912 S Cive S -new + O2 Ecyc,new:
(25)
to obtain
( -G) S C5 55) + CEIlS' E'112.
(26)
the right-hand side is strictly smaller than the Bayes margin miy(x) = (e((x) - (z(x)) then the sign of the log-odds is unchanged and the Bayes decision between and j at preserved. as claimed.
A.3 EXPERIMENTAL PROTOCOL
A.3.1 EXPERIMENTAL SETUP
We utilize a workstation equipped with an NVIDIA RTX 6000 Ada GPU and a Xeon Gold 6448Y CPU to run all the experiments.
Datasets. We evaluate our method on four canonical continual-leaming benchmarks CIFAR-100 100, TinylmageNet, ImageNet- 100 and CUB-200. Each benchmark is instantiated with multiple class-
16

---

Published as a conference paper at ICLR 2026
incremental task splits that every training image is seen exactly once; only the granularity of the partition changes. We use the official train/val (or test) partitions supplied with each dataset.
CIFAR-100 consists of 50,000 training and 10,000 test images of size 32 x 32 drawn from 100 classes.
Tiny-ImageNet contains 100,000 training and 10,000 validation images at 64 x 64 resolu- tion spanning 200 classes.
ImageNet-100 (also referred ImageNet-Subbett) includes 130,000 training and 5,000 validation images at the original ImageNet resolution of 224 x 224 for 100 classes.
CUB-200 comprises 11,788 bird photographs 5,994 for training and 5,794 for test- ing covering 200 fine-grained species. All images are center-cropped and resized to 224 x 224 to match ImageNet preprocessing.
Testing, All results are reported with test batch size of 512 and no test-time augmentations. The code will be made publicly available the time of publication.
A.3.2 ACCURACY METRICS
We evaluate continual learning along three complementary axes: (i) aggregate predictive perfor- mance on seen tasks, (ii) distributional alignment between stored prototypes and the current test distribution, and (iii) near-isometry of the learned transport between old new representations. This subsection formalizes the first axis.
We report the last-task average accuracy A its running mean average incremental accuracy Ame, and the last-task average forgetting Fins. Let a (K) denote accuracy on task after training up task K, and let [.|] be the number of classes introduced step Then
last Σ Di=1
K
C:la
a
(K)
K K
1
ΣR C.l (K) EK Ee Cil
Amc
Fist
(27)
ΣK
EEIIC Ic11
K
last
j=1
where f(
Imaxigisk af of 14 and A is Ana evaluated at step j.
A.3.3 DISTRIBUTTIONSSIILAAN EVALUATION METRICS
To study prototype drift, we compare stored Gaussian prototypes test-time class statistics under the current backbone. Let fo(-) S denote the feature map, and for each class let (Tes E.e) be the stored prototype. Given a held-out set Dies, we compute
Σ fo(#),
m Cov((fo()) Ditl)) E RSSS
Diest
rED
For numerical stability, all expressions involving covariances use Tikhonov regularization Σ Σ el with small 0.
We report three per-class discrepancies that emphasize complmmettry aspects of drift; lower values better.
(1) Prototype Mean Drift (μ-L2).
L-L2, 11 Hill2-
(28)
(2) Covariance Drift (Frobenius).
TT >-F. = 11.5 Eill vu((E -5: 2T}
(29)
17

---

Published as a conference paper at ICLR 2026
(3) Symmetric KL Between Gaussians.
1 DKL(W(P1 E1) [1 Ez))
[u(22 Σn) + (#2 E2 (w2 pa) S In det Στ
2
det 2,
(30)
with S the feature dimension and inverses/determinants taken on regularized covariances. We report the bi-directional form:
SymKL DEL (N(Per Ed) (1 E2) DKL (N(H Ed) [ Ed)-
(31)
Aggregation over Class Set. For any per-class statistic ma E {u-12es E-F SymKT,)) and class set C (e.g., task slice), we report its mean and dispersion:
m
C
Σ
me:
std(m)
V
Viel
Σ REC.
(me -m)²
(32)
AEC
A.3.4 AD-% IN (0.9, 1.1]
Finally, to probe geometry preservation of the old+>new mapping, we measure the fraction of singu- lar values of the composed map that lie tight unit band. Consistent with See. Sec.[A.1.1] let ft-1 and ft be the frozen previous and current encoders at task i, and let A, (old-new and D, (new-xold) be the learned maps. On held-out split V/ restricted to the newly introduced classes C. extract paired features
Zold = f(-((()) E R
Znew = f((x) E R
x E EEV, v(z) E Cc.
stack them Zold, Znow E RSXN and form least squarss surrogates:
D. (Zola Z (Zuew Znow)²
A.
(Zuow Zola) (Zold Zola)*
Let {ar):s = (A, Dr) be the singular values. We report
AD-% in [0.9, 1.1 100 Eifassa(ADi) s11).
(33)
(If Ag or Dr is a single linear layer, its weight can replace the corresponding surrogate.)
Interpretation. Higher AD-% in [0.9, 1.1] indicates that A,D; is closer to isometry with less spectral shrinkage/expansion. This complements Sec. A.3.3 improved near-isometry typically co- incides with lower symmetric KL, indicating better preservation of the class-conditional geometry across tasks.
A.4 ADDITIONAL RESULTS AND ABLATIONS
A.4.1 ADDITIONAL VISUALIZATIONS
Figure tracks prototype drift on the fixed task-O validation split over steps t=1.. (CIFAR- 100, T=10), reporting the mean L2 shift of class centers (μ-L2; Eq. 28 and the Frobenius change of covariances (E-Fro: Eq. (99 with std across classes; smaller is better. Our method exhibits consistently lower center and covariance drift than AdaGauss, indicating closer alignment to the original task-O distribution, i.e., reduced degradation of old-class statistics as ft evolves.
Relation to the main findings. These curves complement the diagnostics in Sec. A.3.3 observe both lower symmetric KL between transported and ground-truth Gaussians and a higher fraction of singular values for A, within [0.9, 1.1] (near-isometry), each pointing better distributional transport and geometry preservation under our bidirectional + cycle training.
A.4.2 ADDITIONAL DETAILS ON DISTILLER/ADAPTHER ARCHITECTURE ABLATIONS
Setup and parity. All adapter/distiller variants Table 5b are trained under an identical data pipeline, optimization schedule, and loss configuration; only the architectural family of the adapter/distiller changes. Each map takes an S-dimensional feature and returns an S-dimensional output. Unless noted, dropout is disabled and LayerNorms default e.
18

---

Published as a conference paper at ICLR 2026
AceGouu
AlcOxo>
0.5
OLRS
20.0
15.0
125
75
Figure 7. CIFAR-100 (T==0): Prototype drift task-0 under continual updates (L). Using the fixed task- validation split, for each step 1=1... ...9 we evaluate model trained up to step Left: prototype mean drift j--L2 (Eq. (2); Right: covariance drift E-Frobenius (Eq. 29) Curves show mean=std over classes (Eq. 32)
smaller is better. OURS exhibits consistently lower center and covariance drift than AdaGauss.
Linear. single affine projection W E RS8 without bias (i.e., 2 + W2). This variant is parameter- and compute-light, and serves to illustrate the contribution of our objective under mini- mal capacity.
MLP (default). Unless stated otherwise, we instantiate the adapter/distiller two-layer MLP I +R'S m -RS with GELU nonlinearity, no residual connection, and no dropout. We set the width multiplier to m= (hidden size 32S),, which matches the capacity used in main experiments.
Cross-Attentie (XAttn). We use single cross-attention block with pre-LayerNorm, 8 heads, and FFN with SwiGLU and expansion 4x (hidden size 4S),, followed by linear projection back to S; dropout is disabled. Queries are produced from current (student) features and keys/values from frozen previous- task (teacher) features, following the standard encoder-decoder attention pattern Vaswani et al. 2017)
Mixture-of-Experts (MoE). We optionally replace the projection MLP with a sparse MoE (Switch-style) comprising 4 experts. A lightweight router (LayerNorm + linear) performs top-1 routing per sample; the selected expert a SwiGLU FFN with expansion 4x (hidden size 4S) and linear projection back to S; dropout is disabled.
Interpretation and scope. Table 5b compares representative lightweight instantiations of Lin- car/ML.P/AAttt/Mol under common training protocol. Since parameter counts and FLOPs nat- urally co-vary across families, the absolute margins are best read as robustness across common, footl-footprint configurations rather than as strict capacity-matehie ranking.
Symbolic capacity accounting (per map). Let S denote the feature dimension and mS the MLP hidden size. Ignoring biases, LayerNorm, and constants:
Linear:
(
2-layer MLP:
(m2m 5o)
(S m - S: default m=32)
1-block XAttn:
4
8S²
~ 12s²
QK/V10
FEN
(4x)
Sparse MoE (4 experts, top-1):
(SS
852
router
expert per sample
A.4.3 PARAMETER SENSITIVITY AND CHOICE OF DEFAULT HYPERPARAMETERS
In the main experiments did not perform extensive grid search. Instead, chose the scales of the bidirectional and cycle- coaee--oosstencyy losses based on their rough magnitude: =m= and A were selected so that the additional terms had a similar order of contribution the task loss and KD
19

---

Published as a conference paper at ICLR 2026
Table CIFAR-I00: Sensitivity of Ani Aeye and
Settings
T=10
T=20
Abi
Aeyo
Alast (%)
A inc (%)
AJast( %
Aine (%)
5
50.6
64.2
41.5
56.5
0
47.8
61.8
39.0
54.9
5
0
49.4
63.1
40.2
55.8
0
0
46.8
60.9
37.9
54.4
5
0
49.7
63.3
39.2
55.2
5
0.5
51.0
64.4
42.4
56.1
5
1
2
48.7
62.9
42.6
56.5
0.5
47.4
62.3
39.8
55.3
51.3
64.7
40.0
56.4
10
47.2
60.9
38.8
53.8
0.5
50.4
64.1
40.8
55.7
5
51.9
64.5
42.9
57.0
loss. For the anti-collapse loss we inherit the default scaling factor a=1 from AdaGauss; our robust formulation preserves the role and typical scale of this regularizer.
The parameter-sensstivity study in Table 6 varies Asi E {0, 0.5, 1, 10}, Azye E {0, 0.5, 1, 2, and (0, 0.5, 1. shows that alternative configurations can slightly outperform our default choice on CIFAR-106 Nevertheless, for all main results we retain the original defaults to avoid the impres- sion that gains are driven by aggressive hyperparameter tuning: the proposed bidirectional module consistently better than the baselines across a reasonably wide region of the hyperparameter space.
A.4.4 CHOICE OF CLASSIFIER
Table 7: Linear classifier vs. Bayesian classifier on CIFAR-100.
Classifier
T=10
T=20 20
Alss f
Aic
Ajs f
Ame f
Bayesian
50.6
64.2
41.5
56.5
Linear (sampling)
51.1
64.7
40.8
55.7
Our main experiments use the Bayesian classifier described in Sec. A.1.2 which predicts by Maha- lanobis distance stored Gaussian prototypes. We additionally evaluate linear classifier trained from these Gaussians (Table (7). Following the public AdaGauss implementation, we construct synthetic training set by sampling features from each class-wise Gaussian ([pes Ee)) and optimize single linear head over all seen classes with standard cross-entropy (denoted Linear (sampling)). the EFC literature (Magistri al. 2024 2025), this procedure is often referred to as Gaussian rebalancing.
As shown in Table 7 the sampling ences in Agst and Ane within fraction of percentage point for both T=10 and T=20. This indicates that our conclusions are not sensitive to the choice between a Bayesian classifier and a Caussian-samplinn linear head.
linear head closely matches the Bayesian classifier: differ-
A.5 ADDITIONAL DISCUSSION AND DIAGNOSTICS
A.5.1 PROTOTYPE-BASED EFCIL AND GAUSSIAN MODELING IN PRIOR WORK
Prototype-based strategies are a well-established line of work exemplar- free class-incremental learning (EFCIL). Broadly, existing methods differ in how they represent class prototypes (means vs. Gaussians) and how they use them (direct classification vs. pseudo-feature feature rehearsal vs. drift compensation).
20

---

Published as a conference paper at ICLR 2026
Mean prototypes with synthetic feature rehearsal. PASS stores one feature mean per class and performs prototype rehearsal by injecting Gaussian noise around these means to synthesize pseudo- features, which are mixed with current-task data to train the classifier; self-supervised rotation head is added to further stabilize the backbone. FeTTrIl. also stores class means, but produces old-class pseudo-features by geometric translation of real features from the current task, f = fnew + Hold Mnew, and uses these translated features together with new-class features to train linear classifier.
Explicit Gaussian prototypes and covariance-aware classification. FeCAM estimates per-class means and covariances and performs Bayes/Mahalanobis classification this Gaussian space. EFC represents each class as a Gaussian prototype (Me, Ee) and samples from these Gaussians to perform asymmetrie prototype rehearsal (PR-ACE), while explicitly compensating prototype drift across tasks. AdaGauss likewise models each class as N (Nte: Σ,) and introduces an anti-collapse regu- larizer based on the Cholesky factor of E. to prevent rank deficiency and feature collapse, together with covariance-adaplatio mechanisms that update (1, Σ) across tasks.
Mean-only drift compensation. LDC stores one mean prototype per class and learns forward projector that maps old-space means into the new feature space after each task. ADC uses adversari- ally perturbed current-tase task inputs whose embeddings lie near old-class means and uses the resulting feature displacements to estimate how old means should move in the space. Neither LDC nor ADC explicitly models full Gaussian structure.
Our position. Our work does not introduce Gaussian prototypes a concept; instead, build on Gaussian-based EFCIL (FeCAM, EFC, AdaGauss) and mean-based drift compensation (LDC, ADC). Our novelty lies in how prototypes are transported across tasks: we introduce bidirectional projector with cycle consistency that is trained online during backbone optimization, with theoretical guarantees linking the cycle loss to spectral contraction and classification stability.
A.5.2 MEASURING THE ADHERENCE OF FEATURES TO GAUSSIAN ASSUMPTIONS
On the suitability of multivariate normality tests. Mardia's 's multivariate normality test is not well aligned with the geometry and scale of continual-learing vision features. It relies on third- and fourth-order moments (multivariate skewness and kurtosis), and its asymptotic calibration assumes moderate dimension and i.i.d. samples. In high-dimensional settings with complex dependence structures and large sample sizes-as in deep feature spaces of EFCIL benchmarks-this test can become overly restrictive and reject even when deviations are mild and do not affect downstream methods Ebner Henze (2020); Chen & Xial (2023). Prior analyses also highlight sensitivity to sample size and dimensionality, motivating the use of graphical or geometric diagnostics.
Instead of relying on global hypothesis test, we assess Gaussianity through geometric, class- wise visualization in a low-dimensional embedding space. For fixed subset of classes, we periodically extract their validation features across the training sequence, embed them with t-SNE, and over- lay the corresponding fitted class-conditional Gaussians by plotting their one- and two-standard- deviation ellipses. This procedure directly reveals whether the learned representations form com- pact, approximately elliptical clusters that are stable over time.
A.5.3 T-SNE SNAPSHOTS OF TASK- -O CLASSES ON CIFAR- 100 (10 TASKS)
To better understand how feature distributions evolve over time, we conduct t-SNE study on the balanced CIFAR-100 benchmark with T=10 equally sized tasks. We fix the ten classes introduced at task 0 and, after finishing tasks 0, 3, 6, and 9, extract their validation features and project them with t-SNE, For each snapshot in Figure g we fit a Gaussian to the features of each class and visualize its one- and wwo-standard-deviatt
regions with solid and dashed ellipses, respectively.
Across all stages, the per-class clusters remain roughly unimodal and are well covered by a single Gaussian, rather than fragmenting into multiple disjoint modes. This suggests that the dominant challenges in EFCIL are not due to a gross mismatch of the Gaussian prototype assumption, but rather due drift of class statistics and reduced separability as the representation evolves.
21

---

Published as a conference paper at ICLR 2026
6.
30
0802
C4033
10
-3
-22
-40
-30
-30
-00
20
2N0 dm
After task
(b) After task 3.
001
30
15
30
CH12
10-
5085 dm
-005 dm
(c) After task 6.
(d) After task 9.
Figure 8. t-SNE of task-O classes CIFAR-100 with T==10. We project validation features of the same ten classes after training tasks 0, 3, 6, and 9. Solid and dashed ellipses mark the and two-standard-deviattion regions of fitted Gaussian for each class.
A.5.4 INTUITIVE VIEW BIDIRECTIONAL CYCLE CONSISTENCY AND LOW-DRIFT REGIMES
From post-hoc adapters to in-task bidirectional alignment. Most drift-compensation pipelines follow two-stage pattern: during Stage I the new encoder ft is regularized toward f:-1 (often via distillation), and only in Stage II is adapter A trained post hoc to map old features into the new space. Our goal
make this duality explicit and move it inside Stage I: we jointly learn a distiller D: Fnew + Sold and an adapter A: Zold Anew while the backbone is still being optimized. Intuitively, D regularizes Je toward the frozen teacher, while A learns the forward transport used for prototype relocation under the evolving representation.
Why this is adversarial training. Cycle consistency in Eq. rather than adversarial game: applying D then A (or A then D) should approximately return the original feature on the data support. A key design choice is that both Loi and Leyo are implemented with stop-gradient targets: (=(aaa) - stopgrad Znew I| updates only, so A follows the evolving St rather than dragging it; the cycle terms update (A, D) but not St. stabilizing the maps without reducing backbone plasticity.
self-consistency constraint
An intuitive reading of Theorem and Corollary 2. Theorem analyzes the cycle loss in whitened feature space where each side has identity covariance; the expected cycle error is the squared Frobenius distance between the composed map AD and the identity. Minimizing Leye therefore encourages near-isometry
the data support (singular values close to 1), preventing rank/energy loss. Corollary 2 then links transport fidelity of means/covariances to bounded pertur- bations of Bayes scores; if the perturbation smaller than the margin, old decisions preserved.
22

---

Published as a conference paper at ICLR 2026
Why CUB-200 shows smaller gains. On CUB-200 fine-tune from ResNet- 18 with a very low backbone learning rate, so representation drift modest. In this low-drift regime, the incremental benefit of aggressively regularizing and transporting features is naturally smaller; empirically the method behaves closer to AdaGauss.
A.5.5 ON ADAGAUSS, FULL-COVARIANC PROTOTYPES, AND THE NEED FOR ROBUSTNESS
AdaGauss and full-covariance prototypes. AdaGauss represents each class c with a Gaussian prototype Ni,,, Me) and uses these Gaussians both for classification (via Bayes classifier) and for Gaussian sampling to train the prototype adapter. To make this feasible and numerically stable, AdaGauss introduces an anti-collapse loss that regularizes class-wise covariance matrices through Cholesky factorization.
Why dimensionality reduction is necessary. The Cholesky-based anti-collapse term and Gaussian sampling require each Σ, to be symmetric positive-definite In exemplar-fie incremental settings, keeping the feature dimension at 512 (ResNet-18 penultimate layer) often makes reliable full-rank covariance estimation difficult. AdaGauss therefore applies a learned linear reduction layer after the ResNet-18 backbone, mapping 512 64, improving the aample-t-dimeenionn ratio and stabilizing covariance estimation.
Why dimensionality reduction alone is still not sufficient. Even after projecting to S 64, the mini-batch covariance can be non-SPD or ill-conditioned in realistic EFCIL regimes (c.g., small B, correlated features, or imbalance), leading to Cholesky failures and unstable scales. We there- fore adopt robust anti-collapse variant: symmetrization, shrinkage, and jitter (and fallbacks when needed), aimed at guaranteeing numerical stability without changing the role the regularizer.
Implications for our method. Following the public AdaGauss codebase, we retain the 512-64 projection. All Gaussian prototypes, anti-collapse losses, and transport maps are defined in the same S 64 feature space, enabling direct comparisons and controlled ablations.
A.6 LIMITATIONS AND DISCLOSURE
A.6.1 LIMITATIONS OF IMAGENET-LE KK EXPERIMENTS
Under our current setup, scaling this protocol to the full 1K-class ImageNet dataset would require several weeks of continuous GPU time, making such an experiment unrealistic for the present study. Consequently, we restrict large-scale evaluation to ImageNet- 100, whose class count still exposes the challenges of our setup while remaining computationally feasible.
A.6.2 LLM USAGE DISCLOSURE
We used ChatGPT (OpenAI) as a writing copilot to critique and polish the prose (clarity, tone, and grammar). The model was not used to generate technical content, figures, or results, nor to design experiments or draw conclusions. The authors take full responsibility for claims and the accuracy of the paper.
23

---

Published as a conference paper at ICLR 2026
We sincerely thank the readers for their time and careful reading of this paper.
24


Reviews:
[{"review_id": "lBQTQobrBf", "reviewer": "Reviewer_zrCc", "summary": "This paper addresses exemplar-free class-incremental learning (EFCIL), where models must learn new classes sequentially without storing past data, making them vulnerable to catastrophic forgetting due to representation drift. The authors critique existing drift compensation methods that use one-directional projections (either post-hoc adapters mapping old-new features or distillers pulling new-old), showing these introduce systematic bias and cycle inconsistencies that accumulate across tasks. They propose a bidirectional alignment framework that jointly learns two projectors—an adapter A (old-new) and distiller D (new-old)—during each task's training, coupled with a cycle-consistency loss that enforces near-inverse behavior using stop-gradient gating to prevent retrograde interference with the evolving backbone. The authors provide theoretical analysis proving that minimizing cycle loss contracts the singular spectrum of the composed maps toward unity in whitened space, and that improved transport of class means and covariances yields smaller perturbations of classification margins, directly mitigating forgetting. Extensive experiments across CIFAR-100, TinyImageNet, ImageNet-100, and CUB-200 demonstrate state-of-the-art performance.", "questions": "On Gaussian Assumption and Distribution Modeling.  \nQ1: Validation of Gaussian Approximation.   \n\nCould you provide empirical evidence that the Gaussian approximation is reasonable for the datasets studied?  \n For example:  \n- Visualizations of class distributions in feature space (e.g., t-SNE/UMAP with confidence ellipses)\n- Quantitative goodness-of-fit tests (e.g., Mardia's multivariate normality test, or KL divergence between empirical class distributions and fitted Gaussians)\n- Analysis showing whether classes that deviate more from Gaussianity exhibit larger forgetting or worse transport\n\nThis would help assess whether the theoretical foundations align with the empirical reality of your experiments.\n\nQ2: Robustness to Non-Gaussian Distributions. \nHave you tested the method on datasets with known multi-modal or long-tailed class distributions?  \n\nFor instance:\n\n- What happens on fine-grained datasets where within-class variance is high (cars, aircraft)?\n- Can you provide ablations using mixture-of-Gaussians or non-parametric kernel density models instead of single Gaussians?\n- How does performance degrade when classes are artificially made more multi-modal (e.g., by merging semantically different subsets)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper effectively motivates the work by clearly articulating limitations of prior one-directional approaches (systematic bias, local alignment, cycle inconsistencies), making the contribution well-grounded.\n- The paper provides theoretical analysis that formally connects cycle consistency to spectral contraction and classification stability. The proof that minimizing cycle loss contracts singular values toward unity and bounds perturbations of classification log-odds provides principled justification for the approach.\n- The bidirectional alignment with cycle consistency integrated during training (rather than post-hoc) is genuinely innovative. This addresses a clear gap in existing two-stage methods that optimize transport only after representation learning is complete, leading to accumulated cycle inconsistencies.\n- The evaluation is thorough, spanning four datasets (CIFAR-100, TinyImageNet, ImageNet-100, CUB-200) with multiple task splits (T=10, T=20), and comparing against 10 competitive baselines. The consistency of improvements across settings strengthens the claims.", "weaknesses": "- The theoretical analysis and distribution transport (Algorithm 1, Stage II) heavily rely on modeling classes as Gaussians with full covariance matrices. This assumption may not hold for complex, multi-modal, or long-tailed distributions, and the paper doesn't validate whether this approximation is reasonable or explore robustness when it's violated.\n- On CUB-200 (where methods fine-tune from ImageNet pretrained weights), the improvements are modest or negative (Table 2). The authors acknowledge that low learning rates for pretrained backbones reduce drift, but this limits the method's applicability to an important practical scenario where pretrained models are standard.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XA4zNA59RG", "reviewer": "Reviewer_J5qi", "summary": "This paper addresses the problem of exemplar-free class-incremental learning, where a model must incrementally learn new classes without storing previous examples. The core challenge is \"prototype drift\" - as the feature extractor adapts to new classes, previously stored class prototypes (means/covariances) become outdated in the evolving feature space. \nThe authors identify a fundamental limitation in existing two-stage approaches: (1) training the new model with distillation from the old model, followed by (2) learning a post-hoc adapter to map old prototypes to the new feature space. They demonstrate that this creates systematic bias due to one-directional projections that either retroactively distort current feature geometry or leave cycle inconsistencies that accumulate across tasks.\nTheir solution introduces bidirectional alignment with cycle consistency during the main training stage. Specifically, they jointly learn two mappings: a distiller D (z_new → z_old) and an adapter A (z_old → z_new), with stop-gradient operations and a cycle-consistency objective (A∘D ≈ I and D∘A ≈ I). Theoretically, they prove this contracts the singular spectrum toward unity in whitened space and reduces perturbations in classification log-odds. Empirically, their method achieves state-of-the-art results across multiple benchmarks (CIFAR-100, TinyImageNet, ImageNet-100, and CUB-200), consistently reducing forgetting while maintaining high accuracy on new tasks.", "questions": "- The paper doesn't sufficiently explain why the specific cycle loss formulation in Equation 7 (with stopgrad on targets) prevents degeneracies better than alternative cycle consistency implementations. A brief comparison to other possible formulations would strengthen the technical justification.\n\n- The motivation of using such a cycle consistency scheme is not very clear. Why can it bring performance gain?\n\n- Some theoretical sections (particularly Section 3.2) could benefit from more intuitive explanations alongside the formalism to improve accessibility.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper makes a compelling case that existing one-directional projection approaches create systematic bias due to post-hoc mismatch. The insight that bidirectional alignmeœqqnt with cycle consistency should be integrated into the main training stage (rather than treated as a separate post-hoc step) is conceptually sound and well-motivated.\n\n2. The theoretical analysis in Sections 3.2 and Appendix A provides rigorous justification for the approach. Theorem 1 showing how cycle loss contracts the singular spectrum toward unity, and Corollary 2 linking transport errors to classification log-odds stability, provide solid mathematical grounding for the empirical observations.\n\n3. The evaluation across four standard benchmarks with multiple task splits (T=10, T=20) demonstrates consistent improvements over state-of-the-art methods. The ablation studies (Tables 4 and 5) effectively isolate the contributions of bidirectional alignment (L_bi) and cycle consistency (L_cyc).", "weaknesses": "1. While the method shows strong performance overall, there's minimal discussion of scenarios where gains are marginal or negative. For example, on CUB-200 with T=20, the method trails EFC by 2.4/3.4 pp in accuracy metrics, but this isn't thoroughly analyzed.\n\n2. The paper mentions the adapter/distiller is \"lightweight\" but doesn't quantify the additional computational overhead compared to baseline methods. Given that efficiency is critical in continual learning, concrete metrics on training time, inference time, and memory requirements would be valuable.\n\n3. The method introduces new hyperparameters (λ_bi, λ_cyc, α) that are set to fixed values across experiments. A sensitivity analysis showing how results vary with these parameters would strengthen the claims of robustness.\n\n4. The theoretical analysis assumes centered features and Gaussian class distributions. The paper doesn't sufficiently address how violations of these assumptions in practice might affect performance, nor does it quantify how well these assumptions hold in the experimental settings.\n\n5. While the paper compares against many recent methods, some recent approaches like those using test-time adaptation or more sophisticated covariance modeling aren't included in the comparisons.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XKcnTKfOhz", "reviewer": "Reviewer_tKrY", "summary": "This paper proposes the a new method for predicting the prototype drift in the exemplar-free class-incremental learning (EFCIL), especially for the setting of training from scratch. This paper specifies the problem of existing drift compensation methods, which only consider transforming old prototypes to the new distribution while the representation learning is conducted in a reverse direction, i.e., pulling new backbone to the old one. To address this problem, this paper utilizes a distiller to transform the new feature to the old and a adapter to adapt old features to new distribution. A cycle consistency loss is proposed to avoid rank loss and theoretical analysis is provided to validate the necessarity of it.  Comparative experiments with other methods and ablation studies are conducted to verify the effectiveness of proposed methods.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well written and easy to follow. \n2. The perspetive of proposing bi-directional loss for the prototype drift compensation is novel.\n3. Theorectial analysis of the cycle loss is decent and sound.", "weaknesses": "Overall, I think this paper is good, but I have several concerns concentrated on the experiments and implementation.\n1. The selected methods in the comparative studies are a bit old. The most up-to-date methods in the comparison are published in 2024. Since this paper is submitted to ICLR 2026, more recent methods should be included, for example, the DPCR [1].\n2.  The extra amount of parameters of the adapter and distiller should be elaborated. This version does not indicate what architecture of the adapter and distiller is used to provide the performance of the comparative studies. Also, from Table 5, it can be witnessed that a performance drop occurs when using achitecture other than MLP. However, from the Appendix A.10, for a ResNet-18 whose output dimension is 512, the parameters of the MLP itself is $512\\times 512\\times 32 \\times 2 \\approx 16.78M$, which is much more larger than the  linear $512\\times 512 \\times 2 \\approx 0.52M$ and even the backbone ResNet-18 (11.69M). As such, does the performance of this method only benefit from more parameters? Also, what will happen if a smaller MLP is used? It is not reasonable that a adapter can be larger than the backbone. \n3. The effect of different $\\lambda_{\\text{bi}}$ and $\\lambda_{\\text{cyc}}$ is not studied.\n4. Lack of results under the setting of training from half. Since another pipeline of EFCIL is training from half, it is interesting to validate the proposed method under that setting.\n5. Compatibility with other types of classifiers should be studied. For continual learning, there are other types of classifier other than the Bayesian classifier, for example, the most common fully connected network. It is interesting to discuss and validate the compatibility of proposed method with them.\n\n[1] Run He, et al. Semantic Shift Estimation via Dual-Projection and Classifier Reconstruction for Exemplar-Free Class-Incremental Learning. In ICML 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kv17pPO7Nn", "reviewer": "Reviewer_A1WY", "summary": "The paper studies exemplar-free class-incremental learning (EFCIL) and claims that one-directional projection of class prototypes from old to new embedding space introduces systematic bias that accumulates over tasks. The authors propose to use a bidirectional projector which aligns prototypes during training using a cycle-consistency objective function which ensures a near-bijection between the pairs. The authors further analyze that reducing the cycle loss better preserves old-class boundaries. The method finally uses a Gaussian Bayes classifier for inferene which uses current-task statistics for new classes and projected statistics for old classes. The proposed method outperforms several recent EFCIL baselines across multiple benchmarks achieving a better stability-plasticity trade-off.", "questions": "1. What is the impact of transporting covariances and estimating from a normal distribution instead of directly projecting the class means as done is LDC [ECCV 24]? Is it necessary to also transport covariances since storing class-wise covariances adds storage requirements? How much does this improve the estimation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper is very well-written with a good introduction, motivation and theoretical formulations.\n2. The authors discuss and formulate the concept of prototype drift compensation very well including recent works.\n3. The extensive experiments with thorough ablations are appreciated. The evaluation with distribution similarity metrics is a valuable addition.", "weaknesses": "1. It would be nice if the authors could discuss and analyze more on the claim that one-directional projection of class prototypes accumulates bias over tasks to better motivate the method.\n2. Some analysis with respect to the oracle prototype drift could be added. For instance, comparing the projected prototypes using different methods with the oracle prototypes (computed using all old class data) could be used to compare prototype estimations from different methods (see Fig. 6 in ADC [CVPR 24]). This would also highlight how much scope is there to further improve the prototype estimation.\n3. A more detailed discussion of the Bayes classifier for the inference stage (not mentioned in the pseudo-code) would add some clarity.\n4. Figure 2 visibility could be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)