Learn and Ensemble Bridge Adapters for Multi-domain Task Incremental Learning
Authors: Ziqi Gu
Chunyan Xu
Wenxuan Fang
Xin Liu
Yide Qiu
Zhen Cui
Venue: NeurIPS 2025 poster ()
Keywords: Multi-domain task incremental learning; Schrödinger Bridge
Primary area: applications
TL;DR: We propose LEBA, a framework that mitigates forgetting and enhances generalization in multi-domain continual learning via Schrödinger bridge-based optimization and progressive adapter ensembling.

Abstract: Multi-domain task incremental learning (MTIL) demands models to master domain-specific expertise while preserving generalization capabilities. 
Inspired by human lifelong learning, which relies on revisiting, aligning, and integrating past experiences, we propose a Learning and Ensembling Bridge Adapters (LEBA) framework. 
To facilitate cohesive knowledge transfer across domains, specifically, we propose a continuous-domain bridge adaptation module, leveraging the distribution transfer capabilities of Schrödinger bridge for stable progressive learning.
To strengthen memory consolidation, we further propose a progressive knowledge ensemble strategy that revisits past task representations via a diffusion model and dynamically integrates historical adapters.
For efficiency, LEBA maintains a compact adapter pool through similarity-based selection and employs learnable weights to align replayed samples with current task semantics. 
Together, these components effectively mitigate catastrophic forgetting and enhance generalization across tasks.
Extensive experiments across multiple benchmarks validate the effectiveness and superiority of LEBA over state-of-the-art methods.

Learn and Ensemble Bridge Adapters for Multi-domain Task Incremental Learning
Ziqi Gu', Chunyan Xu Wenxuan Fang', Xin Liu", Yide Qiu', Zhen Cui
School of Computer Science and Engineering, Nanjing University of Science and Technology Nanjing Seetacloud Technology School of Artificial Intelligence, Beijing Normal University
Abstract
Multi-domain task incremental learning (MTIL) demands models to master domain- specific expertise while preserving generalization capabilities. Inspired by human lifelong learning 2Σ. which relies on revisiting, aligning, and integrating past experiences, we propose Learning and Ensembling Bridge Adapters (LEBA) framework. To facilitate cohesive knowledge transfer across domains. specifically, we propose a codiinuousdoomalee bridge adaptation module, leveraging the distri- bution transfer capabilities of Schrödinger bridge for stable progressive learning. To strengthen memory consolidation, we further propose a progressive knowledge ensemble strategy that revisits past task representations via diffusion model and dynamically integrates historical adapters. For efficiency, LEBA maintains a com- pact adapter pool through similarity-based selection and employs learnable weights to align replayed samples with current task semantics. Together, these components effectively mitigate catastrophic forgetting and enhance generalization across tasks. Extensive experiments across multiple benchmarks validate the effectiveness and methods.
superiority of LEBA over
Introduction
Deep learning has made strides 3 5), particularly in the realm of large-scale foundation models (6] 7]. with recent research further validating these advancements. However, traditional fully-supervised training methods struggle to address this challenge due to the high computational cost involved in integrating new-coming data with historical datasets. Incremental learning [8] 9. also known as
continual learning, provides an effective method by incrementally learning classes, with each training task focusing solely on new-coming samples. Many methods [10] II [2] have actively addressed
the challenges of continual learning, such as knowledge graph preservation [B], self-supervised learning [14,, and replay data [5]. While these methods demonstrate potential in memorization and scalability, they mainly focus incremental learning from batched data of homogeneous domain.
In contrast, nulti-domain task incremental learning (MTIL) the focus of this work mork-iims to learn from sequence of heterogeneous domains. The paradigm requires an effective transfer and adaptation across diverse domains while incrementally learning new ones, where catastrophic forgetting may be even more severe. Specifically, the model should not only maintain stability in retaining knowledge from previously learned domains, but also develop generalization capabilities for unseen domains, referring to the problem of zero-shot. Recently, liniun-language models as well knowledge distillation have been used for zero-shot MTIL. For instance, incorporating zero-shot generalization into CLIP has proven effective in mitigating knowledge degradation [16]. Further, MoE- [17] designs task-specific and task-independent components and leverages Mixture-of-F -x-perrtsss [18] for adaptive task learning. These methods offer promising advancements for zero-shot MTIL.
CCoresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Fundamentally, MTIL requires not only domain-speciin knowledge but also the ability to capture the cross-domain transfer. A well-constructed cross-domain transfer mode ensures both the mitigation of catastrophic forgetting and better generalization in an unseen domain. In this process-akin to human learning 1 2] rreviiiinn, aligning and integrating past experiences become crucial for memory consolidation. To this end, two key challenges need to be addressed: i) How to establish incremental transfer from previously learned adapters to current adapter? ii) How to replay knowledge beyond the constraints of task order and
features?
To address the above issues, in this work, we propose Learning and Ensembling Bridge Adapters (LEBA), a novel framework for multi- -omminn task incremental learning. LEBA introduces an incre- mental bridge- bridge-transfer mechanism to align the latent distributions of current and previous adapters, facilitating effective cross- cross-domain knowledge transfer. Specifically, we design continuous domaan bridge adapters that act as incremental knowledge bridges across sequential tasks. These adapters ensure knowledge cohesion and inheritance between existing and new task domains, thereby stabi- lizing the incremental learning. The integrated transfer mechanism not only mitigates catastrophic forgetting effectively but also promotes progressive model optimization throughout the task sequence.
During sequential domain learning, LEBA also enhances memory consolidation by actively revisiting past experiences. Unlike traditional methods [8] [9] that rely on storing subsets of prior samples for replay- constrained by task order and data characteristics, we propose a progressive knowledge ensemble method, which could flexibly revisit prior knowledge without these constraints. By lever- aging a pretrained diffusion model [20], our LEBA could reconstruct samples from any previously tasks. To optimize memory efficiency. we maintain a compact adapter pool by selectively preserving representative adapters through similarity-based matching. Furthermore, since different adapters may interpret replayed samples in distinct ways, we introduce a learnable weighting way to tailor the replay process to individual sample attributes. By adaptively integrating historical knowledge with new task adaptation, LEBA can achieve superior performance and generalization capabilities.
In summary, our primary contributions are four-fold: i) propose learning and ensembling bridge adapters framework for MTIL, facilitating knowledge transfer and mitigating catastrophic forgetting; ii) design continuous-domait bridge adaptation align and transfer domain knowledge across sequential tasks; iii) introduce progressive knowledge ensemble regardless of task-learning sequence, enabling flexible integration of prior knowledge; iv) report tate-of-the-art results on two task settings.
2 Related work
Multi-domain task incremental learning: Although the above method exhibits promising per- formance in incremental learning, it struggles to address a critical capability of vision-language incremental models: zero-shot transfer to unseen knowledge. In contrast to incremental learning, which centers on knowledge from a single domain, multi-domain incremental learning requires the sequential acquisition of knowledge from multiple domains. This mode necessitates that the incremental model not only incrementally learn new tasks and mitigate catastrophic forgetting but also effectively transfer knowledge across range of diverse domains. Notably relevant is ZSCL (16), which employs parameter regularization in the incremental learning of large-scale models. Addition- ally, MoE-Ade MoE-Adapters [17] enhance learning by integrating task-specific components into the CLIP model, thereby boosting its adaptability.
Incremental learning: Previous works in incremental learning have focused on developing variety of architectures (21), including memory-based regularization-basee and dynamic-based models. Memory- methods preserve historical knowledge by storing it within a memory bank, which is
periodically accessed and updated during incremental learning 19 10 22 [5]. Regularization- based methods integrate explicit regularization terms into the weights to mediate between previous and new-coming tasks [23 24 25 or data [26] 9. Dynamic methods tackle incremental learning by progressively augmenting the baseline with new parameters, such neurons, branches, prediction
heads (2 28
30
31.
Schrödinger Bridge: Schrödinger Bridge (SB) B2 33 is a conditional diffusion model that solves an entropy-regulariee optimal transport problem aimed at identifying the diffusion process between two distributions. Recently, Liu et al. 34 have introduced a tractable special case of dynamic stochastic bridges, which has demonstrated notable efficiency in image manipulation tasks such as image restoration and super-resolution real-world datasets. Moreover, Schrödinger bridges belong

---

to class of neural stochastic differential equations that, in contrast to diffusion models, facilitate the translation of samples across arbitrary domains with minimal transport costs. The learning of
these SBs typically involves two main algorithmic approaches: flow matching [5 which distills SBs between mini-batches using optimal transport; iterative proportional fitting [6. which focus
on iteratively minimizing transport costs by training models on input-output pairs generated by the
models themselves. Together, these methods have enhanced the flexibility and efficiency of learning in the context of both Schrüdinger bridge models (37 and broader stochastic dynamic frameworks.
3 The Proposed Method
3.1 Problem Formulation
Multi-domain task incremental learning (MTIL) involves sequentially learning from stream of labeled task domains, where historical data becomes unavailable in subsequent stages. The goal is to evaluate not only the model's adaptability to incremental learning but also its resistance to catastrophic forgetting. Formally, given a sequence of T task domains, denoted as {S 1/= we want to learn an incremental model (or adapter) O' based on the current task state S as well as previous available models. Each task domain St consists of a dataset and a semantic set C, defined as S == (D',C*) for the t-th domain. The dataset D' usually consists of input-label pairs, denoted as D = (1) v/ Mr
where N₂ is the total number of samples in task St. The semantic set C ) describes certain semantic information (c.g., class information gf), with M, denoting the number of distinct class names. In this incremental paradigm, task domains are typically non-overlapping in their class labels, ie., for any two task domains S S ,S', C n = Ø if + j. A conventional solution of MTIL is to
finetune the previous model via: O' +- 0 X (C(5)) where is a supervised loss function (e.g.. cross entropy over class labels) and is the learning rate. However, balancing new-domain adaptation with catastrophic forgetting remains a challenging problem, despite some existing efforts [16 17 to mitigate this problem.
In contrast, we propose to learn a cross-domain adapter θ by revisiting and aligning past knowledge. Concretely, we formulate multi-domain task incremental learning as:
t arg min w
ta
Σ 2--
Sala(at 0' g Pi; W)),):
(I)
supervised info.
knowlodge alignment
where the replayed sample a is sampled from a generator conditioned on historical semantic concepts [[]) i.e, o7 (((C)) d); a dynamic adapter pool Pk of size K is introduced to
store useful historical adapters, i.e., Px ye,,,, with ja E {1..... ,t 1); the weights quantify the relevance of replayed sample o to the adapters in the pool Pk. while g()) denotes feature extractor; the alignment loss ÇA over an operator A. parameterized by r. measures distribution
similarity between responses of current adapter and historical adapters in Px- By integrating supervised learning with historical knowledge alignment, our LEBA could mitigate catastrophic forgetting and enhance generalization to unseen comains-mirronti human learning processes where memory consolidation relies on revisiting past experiences.
3.2 Overview
Building on the formulation in Eqn. I our framework focuses on two key components: i) designing A as a cross-domain adapter and ii) dynamically integrating knowledge in Px- To this end, we propose Continuous-domain Bridge Adaptation (CBA) in Section 3.3 and Progressive Knowledge Ensemble (PKE) in Section 3.4 In CBA, rather than optimizing O' solely via the supervised loss (s. we design bridge-matching adapter O' aligned with historical adapters through distribution transfer. A diffusion- based generator G is used to synthesize samples from the observed concept set to facilitate knowledge alignment. In PKE, we construct dynamic buffer pool Pи (size K) to store historically significant adapters, balancing computational efficiency with knowledge retention. To address discrininability variations among adapters, we design an adaptive ensemble way with learnable weight w, ensuring both alignment and discriminative inference. Together, these components enable incremental learning to refine adapters and dynamically enhance knowledge transfer. Alongside these components, our framework incorporates vision-languag backbone with dedicated encoders for
3

---

Algorithm LEBA Training Procedure
Input: Task sequence S {(D', It it==,..... I,..., TT. diffusion based generator 9i continuous-domale bridge adapter I"; adaptive weight w; incremental model @
Output: Incremental model 6², adaptive weights and continuous-done -domain bridge adapter I I: Initialize model ( geneerator G, weight w. and adapter r
2: for 1 to T do -Spppervieed update with current task data 4: Train @ on St=1 = (D'=)," (C"=)
5:
i 1 then
6:
Progressive Knowledge Ensemble
7:
Construct adapter pool Px from previous adapters {O') Jt=n with ja E (1,.... , 11
8:
Generate replay samples Σ from generator & conditioned on semantic concepts { 16-1 13=1
9:
Compute adaptive weights w for each replay sample /
10:
Evaluate similarity of current adapter @ and update adapter pool Px
11:
# Continuous- Domain Bridge Adaptation
12:
Construct continuous-domait bridge adapter with replay data o and adaptive weight w Joint Optimization
13:
14:
Update θ', w, and I' by minimizing the total loss Goaa (Eqn. 12
15:
end if
16:
Update the incremental model for the next domain
17:
@ O*
18:
end for
processing image and semantic information. Specifically, we extract both image and text features for label prediction by reformulating the feature extractor g decomposing form:
g( c, 0²) = gimg (x1, mmgg)) @ ghzt Sixt),
where Jimg and ftx denote the image and text encoders, respectively. @ represents the element-wise product. Following common practice, we initialize these encoders using pre-trained models (e.g.. CLIP 38) backbones, with additional adaptation layers learned for task speciffc fine-tuning. Please note the adapter parameters that are structured as O = g img: Ofar). The subsequent subsections elaborate on the details of CBA and PKE, followed by the L.BA training optimization. The LEBA training process is shown in Algorithm T
3.3
Continuous-Domall Bridge Adaptation
The adapter requires not only Nomain-specifff adaptation capabilities but also the ability to facilitate cross-domain knowledge transfer. Effective knowledge transfer should simultaneously mitigate catastrophic forgetting while improving generalization performance on unseen domains. To achieve this, we leverage the Schrödinger Bridge (SB) mechanism to facilitate inter-task knowledge transfer by aligning probability distributions between current and previous adapters. Specifically, we design a continuous-domas bridge adapter for cross-domain distribution transfer.
Given a sample 24 the feature Z1 = = ci, O²) P1 encoded by the current adapter Of follows the probability distribution Pt. Similarly, for a historical adapter in the buffer pool Px. O E Px (where ja < { f 1), the feature Zo = g c; O²) Po be obtained. The continuous-domain bridge adapter can be formalized as:
dZn. [fnm 3m V log V Zm., m)]dm + V dW ms
Zu Po:
(3)
dZm = [f 3. V log (Zm. m)]dm + / WmdW m.
Z1 ~ Pı,
where Po and Pi denotes the source and target distributions, denotes the time-step, { W refer to the standard Wiener process and its time reversal and (.fm,mm) are the drift and diffusion
coefficients. The pair of functions {w, $ is said to solve the following coupled PDEs. The Eqn. and its time-reversal are directly derived from the Bokker-Planch equation 39 corresponding the
SDE in Eqn.44) as follows:
dZm fm.mm + / dW.
Zo ((,00),
m:
dbm dm + / B/
Z ~ V(, 1).

---

Since v and & contain complex drift terms that are difficult to compute, we simplify a certain form for the boundary distributions Po and Pt. We define the energy potential functions (+) Po(.) := 6.).) and ((,,1) P (-, 1), where da (-) is the Dirac delta distribution centered on a E R. This choice ensures that the diffusion process becomes computationally manageable.
Consequently, we approximate both the forward and backward with the following Gaussian posterior as:
Zm (m Zu. 21)
(Zmi Hm: Emn),
0²
m
s.t. Pm
Zo +
Z1 Σm
o + +
m
2
+ o
Om
+ o
where of - S dm' and Thh = J dm represent the cumulative noise variances in the forward and backward directions. We take P(Z,, 7,)= = (n) P (n) and f= = 0, and construct tractable SB between individual knowledge distribution from Zo and Pi (1 (n)
Based on the adapter formulation, we derive an approximate reverse SDE from Eqn.(3 to simulate the transfer from Z to Zo by estimating the score function log V (Zms m mLTT,G)) = e( Ct, Zm, m; I)/ Sm. formally:
dmm = (Bm/ h ( Ci Zm, m.; D)dm + v BB mm.
where P1 denotes the distribution of Z1, i.e., Z ~ (z [. Gi), and is a continuous-domail bridge realized through a neural network parameterized by I. The adapter network is optimized to approximate the score function Vz log (2 (2 / [2,, Ci) by minimizing the following objective function:
| Cr Bm, m;
Zm
Zo
2
SCBA = Egg...
Om
2
where m E u( 1J) and Zm ~ a( 1m Lo, Z1) are defined in Eqn.
For the above formula, propose to construct continuous-domain bridge adapter to connect the previous distribution (from prior adapters) to the current distribution (from the new adapter). This enables the incremental model to mitigate catastrophic forgetting while optimizing performance effectively. By facilitating smooth transitions between past and present tasks, our approach offers a novel framework for understanding and implementing incremental learning.
3.4 Progressive Knowledge Ensemble
The absence of historical samples presents fundamental challenge to revisiting prior knowledge in MTIL. Existing methods [99 g typically preserve features from selected past samples for replay, but remain constrained by task order and data-specific dependencies, thereby limiting their robustness against catastrophic forgetting. To overcome this limitation, we draw inspiration from human learning 2 and propose progressive knowledge ensemble that enables flexible knowledge reuse.
To enable flexible knowledge reuse, dynamically maintain adapter pool Px of size K to store useful historical adapters. For each new adapter O', we measure its similarity to those in the pool and update the pool by replacing the least representative one when necessary. Formally, the similarity between the current adapter and the j-th adapter in the pool is computed as:
= (o(a, cf, (lg c, O²)), s.t. j 1,2,...,
8
where Ty denotes the similarity between the current adapter and the j-th adapter in the dynamic adapter pool, computed over the current domain samples using KL divergence [40]. A threshold-baig based strategy determines whether the current adapter should replace an existing one.
How can replay samples be obtained without relying on traditional methods such as storing sample features or sesk-eeeuential replay? Inspired by human learning-wee -where individuals can recognize whether a concept has been previously encountered we propose an alternative strategy. Specifically, each replayed sample ž is drawn from diffusion- generator & conditioned on previously encountered semantic concepts { i.e, Z. ~ G((C)) ((;)))=1: :)), where denotes the generator
parameters. Let the replay set be denoted X consisting of [X samples, i.c., X (
5

---

Meanwhile, an learnable weight is computed for each replay data T/ which can be formulated as:
SPkE = COS Wj Dimg (T. S ing Ment (ct, O)mz)) Wj w1,42, Wk:
where wi represents the similarity between the replay sample 7 Through this process, we obtain an adaptive weight to evaluate the similarity between the replay sample I; and the previous adapters, thereby providing a more accurate previous distribution to construct the Schrödinger bridge, further enhancing knowledge transfer between current and previous adapters.
3.5 Optimizing the LEBA
To define the loss function Sootal in the above process, we can utilize replay data T. (see Section 3.4 and the constructed continuous-domann bridge adapter (see Section 3.3 to facilitate knowledge transfer and integration between adapters. Formally, considering that each adapter has different understanding of the replay data T; in the k-th(i.e., k > 1) incremental task, its probability distribution can be rewritten as:
k
20 =
Σ Wj (gimg (T;, mmmgg @ Itan (.at)) 1=0
(10)
where Zo denotes the integrated predictive distribution over all adapters for the replay sample T. Consequently, the final objective loss for the continuous-domain bridge adapter, based Eqn. can be reformulated as follows:
| E(ē,, cs. Zm, m, 1
Zm
Zu
SROBA = Ez
(11)
m
All components are ultimately integrated into the unified LEBA framework, thereby preserving the learned knowledge distribution across tasks. The final optimization objective can be formally defined as:
Stotal (CE(O") + (RCBAA(T + $(rkε(w),
(12)
where Y and 3 are balance factors. We construct a continuous-doman bridge adapter between different adapters by replaying samples (instead of stage-wise replay). Our LEBA effectively mitigates catastrophic forgetting by maintaining the probability distribution of the learned knowledge. Furthermore, our LEBA not only facilitates knowledge transfer across domains but also enables the incremental model to retain knowledge of previous tasks while learning new ones.
Experiment
4.1 Experimental Setting
We evaluate our LEBA in the multi-domain task incremental learning(MTII)) 16. In this configuration, tasks are sourced from multiple domains, each necessitating unique domain knowledge to achieve high accuracy. The MTIL benchmark comprises 11 tasks and contains a total of 1, 1,201 classes. We evaluate the method using different task orders: the first follows alphabetical order
Datasets:
(Order-I): Aircraft [41), Caltech101 [42] CIFARI00 (43) DTD [44, EuroS [45), Flowers [46),
Food 47 MNIST [88). OxfordPet 49 StanfordCars (50), and SUN397 (51) The second uses
a random order (Order-1I): StanfordCars. Food, MNIST, OxfordPer, Flowers, SUN397, Aircraft, Caltech101, DTD, EuroSAT, and CIFAR100. By default, experiments are conducted using Order-I.
To evaluate LEBA in the multi-task incremental learning (MTIL) setting, we follow the protocol introduced in 7SCL [16), which includes three metrics: "Transfer". "Last", and "Average". "Transfer" measures the model's zero-shot generalization to unseen tasks, while "Last" evaluates its ability retain knowledge from previous tasks. "Average" captures overall performance by averaging the results of "Transfer" and "Last". However, these metrics do not explicitly quantify the extent of forgetting across tasks. To address this, introduce new "Preserve" metric, which captures forgetting dynamics by analyzing the lower triangular portion of the accuracy matrix. Formally, "Preserve" is defined as Preserve TCT-172 Σ Σ acc,.j, where acc;,j denotes
Evaluation Metrics:

---

Table I: Comparison with state-of-the-art methods on the multi-domain task incremental learning benchmark (Order-I) in terms of "Transfer", "Average", "Last" and "Preserve" scores (%).
Airerait
Caltech101
CIFART00
DTD
EuoSAY
Flowers
MNNSS
OxfordPe
SENDS
Method
Average
Zero-shot
24.3
88.4
68.2
44.6
54,9
71.0
88.5
59.4
89.0
64.7
65.2
65.3
Full Fine-tume
62.0
95.1
89.6
79.5
98.9
97.5
92.7
99.6
94.7
89.6
81.8
89.2
ContinualFFT
67.1
46.0
32.1
35.6
35.0
57.7
44.1
60.8
20.5
46.6
44.5
LwF [9
74.5
56,9
39.1
51.1
52.6
72.8
60.6
75.1
30.3
55.9
58.9
Transfer
56.6
39.3
46.0
77.4
60.5
WISE-FT 58
73.5
55.6
35.6
41.55
47.0
68.3
51.9
69.3
26.8
51.9
52.3
ZSCL [6
86.0
67.4
45.4
50.4
69.1
$7.6
61.8
$6.8
60.1
66.8
68.1
MaB-Adaaters
87.9
68.2
44,4
.
70.7
$8.7
59.7
89.1
64.5
65.5
68.9
ContinualFFT
25.5
81.5
59.
53.2
64.7
$1.8
63.2
64.3
69.7
31.8
49:7
55.9
La/P (
36.3
86.9
72.0
99.0
73.7
60.0
73.6
74.8
8000
37.3
58.1
64.7
Average
WISE-FT
86.5
64.3
68.7
5RL7
71.1
70.5
75.8
54.6
60.7
ZSCL 16
45.1
92.0
80.1
61.3
79..
81.6
89.6
75.2
$5,9
6417
68.0
75.4
MoB-Adapiers
50.2
91.9
83.1
69.4
78.9
84.0
899
73.7
89.5
67.7
66.9
76.7
79.8
Continual-F7
31.0
89.3
65.8
67.3
BR.9
71.1
85.6
99.6
92.9
77.3
B1.1
77.3
LaxP ()
26.3
87.5
71.9
66.6
79.9
66.9
83.8
99.6
92.1
66.1
80.4
74.6
Last
iCaRL
35.8
93.0
77.0
70.2
83.3
88.5
90.4
86.7
93.2
81.2
81.9
80.1
WiSE-FT
86.9
99.6
81.3
77.7
2SCL 16
40.6
92.2
81.3
70.5
94.8
90.5
91.9
98.
93.9
85.3
80.2
83.6
MoE-Adagsts ir
49.8
92.2
86,1
78.1
95.7
94L.3
89.5
98.1
89.9
$1.6
80.0
85.0
95.2
97.2
97.3
N2.4
87.3
Contimul-FF
N7.2
61.5
84.4
68.5
96.2
71.3
LaFF [
25.4
84.4
69.3
62.4
75.2
63.8
79.5
97.4
89.5
63.2
69.3
Preserve
KaaRL [57
30.5
91.1
74.6
66.4
79.2
83.1
86.5
82.1
89.4
76.2
74.8
WISE.FT 5X
85.6
82.3
75.1
97.5
74.9
ZSCL 16
44.1
92.5
82.5
70.7
95.9
91.2
91.9
98.8
94.2
85.3
80.0
MoEl-Adipters
17
50.0
92.3
86.3
78.8
95.4
25.0
89.5
98.2
89.65
81.6
82.4
Ours
53.7
95.4
87.0
80.4
96,9
97.4
89.5
99.1
89.6
88,8
84,5(+2.1)
the accuracy on j-th domain after training on i-th task. This metric provides a more comprehensive assessment of the model's ability to preserve learned knowledge in the MTII.
Implementation Details: Following previous work [66,, we adopt CLIP with ViT-B/16 [2 as the backbone for all experiments. Each task's adapter is composed using LoRA (53. For generative replay, we employ the Stable Diffusion-V - 1.4 model [54] capable of generating samples that closely approximate the original data in both fidelity and discriminative quality. The continuous-domain
bridge adapter implemented as a four-layer MLP. We set the balancing factors ? = 0.1 and 0.4, and use step size of = 20 and an adapter selection threshold of 7 0.3, and an adapter pool containing K adapters. Optimization is performed using the Adam W optimizer (55), with label smoothing (56) applied to improve baseline performance. For the MTIL benchmark, batch size of 64 and search the learning rate a within {1 x 10-3....,, x 10-5). All experiments conducted using PyTorch on NVIDIA GeForce RTX 4090 GPUs.
4.2 Comparison with State-of-the-art Methods
Table T presents the detailed results of the "Transfer", "Avg", "Last", and "Preserve" metrics the MTIL benchmark across all evaluated methods and datasets. Zero-shot refers to the predic- tion performance of the initial CLIP model without any task-specific adaptation, while Fine-tune represents the accuracy achieved by fully fine-tuning on each dataset, serving as an upper bound in the absence of forgetting. The results reveal that both zero-shot prediction and newly learned knowledge suffer from performance degradation under incremental learning. While existing methods partially mitigate this issue, they generally fail preserve strong zero-shot capabilities. Our proposed method, LEBA (denoted as Ours). consistently outperforms the strongest MoE-Adapter across most tasks, demonstrating superior overall performance and more favorable slability-plastiityy trade-off. Further validation shows that the CBA module, by constructing a continuous-domait bridge adapter, effectively integrates and revisits previously learned knowledge while adapting new domains; by coupling supervised learning with historical knowledge alignment (e.g., feature/adapter consistency), the model enables smooth knowledge transition and durable retention. In parallel, the PKE mod- ule addresses task-order limitations, enabling flexible reuse of prior knowledge regardless of the incremental sequence via progressive knowledge ensemble and selective routing. These components mitigate catastrophic forgetting and enhance zero-shot generalization to unseen categories.
7

---

Table Comparison with state-of-the-art methods on the multi-domain task incremental learning benchmark (Order-II) in terms of "Transfer", "Average", "Last" and "Preserve" scores (%).
Food
MNNSTT
OxfordPor
Flowers
SUND9
Aicrafi
Caltech101
DTD
EueSAT
CIFARTOR
Method
Average
Zero-shot
64.7
88.5
99.4
89.0
71.D
65.2
24.3
88.4
44.6
54.9
6M.2
65.3
Full Fine-tune
89.6
92.7
99.6
94.7
97.5
81.8
62.0
95.1
79.5
98.9
89.6
89.2
Continua-FFT
85.9
59.6
57.9
40.0
46.7
11.1
70.0
30.5
26.6
37.7
46.6
LwF [9
87.8
58.5
71.9
46.6
57.3
12.8
B1.4
34.5
34..5
46.8
53.2
Transfer
72.3
WISE-FT 58
87.2
57.6
67.0
45.0
54.0
12.9
78.6
35.5
28.4
44.3
31.1
ZSCL [6
88.3
57.5
$4.7
68.1
64..
21.1
B8.2
45.3
55.2
68.2
61.1
MaB-Adagters
88.8
39.5
89.1
64.4
18.1
86.99
43.7
54.6
68.2
64.3
Continual-FT
42.1
70.5
92.2
N0.1
54.5
99.
19.8
7H.1
41.01
38.1
42.3
56.2
LaaP y
49.0
77.0
92.1
85.9
66.3
67.2
20.9
84.7
44.6
45.5
10.5
62.2
Average
WISE-FT
79.3
91.9
63.4
65.2
83.7
45.4
48.2
ZSCL 16
81.7
91.3
91.1
91.0
82.9
72.5
33.6
89.7
$3.5
62.8
69.9
74.5
MoB-Adapiers
84.9
89.9
89.3
91
86.
72.2
33,4
89.4
53.3
61.4
69.9
74.7
W7.2
Continual-F7
24.0
67.3
99.1
44.3
67.0
29.5
92.3
61.3
81.0
88.1
67.4
LaxP [
34.6
69.6
99.3
$8.7
61.1
72.5
32.5
88.1
65.6
90.9
87.9
71.9
Last
iCaRL
46.0
$1.5
91.3
82.8
66.55
72.2
16.3
91.6
68.1
83.2
87.8
71.6
WiSE-FTI
87.6
72.2
2SCL 16
78.2
91.1
97.6
92.5
87.4
78.2
45.0
92.3
72.7
96.2
86.3
83.4
MoE-Adagtts ir
84.1
88.5
94.0
91.8
94.1
77.8
50.4
93.3
77.1
$1.7
86.6
84.1
K8.9
99.2
95.2
86.4
Contimul-FF
89.1
76.2
LaFF [
47.4
76.1
90.1
83.6
63.8
64.5
17.2
80.7
41.8
43.4
60.7
Preserve
K.aRL [57
49.2
74.3
75.6
71.2
55.7
57.6
10.3
76.8
39.5
40.8
55.1
WISE
60.3
ZSCL 16
77.1
89.2
95.1
90.2
85.6
77.5
42.9
90.6
72.1
94.2
MoE--Adipters
17
81.2
87.6
97.5
85.
90.9
74.1
48.2
91.4
74.1
97.1
82.0
Ours
85.4
88.2
98.8
92.3
96.2
78.6
49.6
94.2
78,7
85.8/+2.9)
Table 3: Performance comparison of CBA module and PKE module of LEBA
Airerft
CalechhO1
CIFARIO
DTD
EuroAAT
Flowers
Food
LNNW
OxfordPal
Melhod
Cau
Arerage
Transter
Baseline
87.2
67.1
43.3
48.4
08.5
86.9
57.1
87.7
63.2
63.2
67.3
+CBA
88.1
67.6
44.1
49.1
69.4
87.5
58.2
64.3
68.1
-CRA.PKE
Average
+CBA
53.5
94.3
83.2
69.6
78.0
77.9
54.5
88.
74.1
88.6
68.7
61.4
77.2
-CBA+PKE
53.9
94.9
83.8
70.8
79.8
85.1
89.1
74.8
89.3
69.2
65.8
77.9
Lust
+CBA
Baseline
52.1
$4.2
94.4
93.8
$4.3
96.8
78.3
76.6
96.3
98.3
95.2
96.4
86.7
87.5
98.3
97.1
89.1
89.2
84.2
79.6
78.1
86.2
84.7
-CRA.PKE
87.3
Proserve
+CBA
53.4
94.2
86.7
78.2
95.8
95.6
88.
988
98.6
88.7
88.
86.1
83.5
83.6
-CBA+PKE
53.7
95.4
87.0
80.4
96.9
97.4
89.5
99.1
89.4
88.7
84.5
4.3 Ablation Study
This section focuses on analyzing the effectiveness of the proposed LEBA method. All experiments conducted in multi-domain task incremental learning setting, with additional analysis available in the supplementary material.
Effectiveness of different modules: We conduct experiments to assess the effectiveness of the proposed CBA and PKE, with detailed results shown in Table The results clearly demonstrate that incorporating the CBA module consistently improves performance over the baseline, highlighting its effectiveness in enhancing cross-task knowledge adaptation and generalization. Furthermore, the integration of the PKE module leads additional gains across all evaluation metrics, particularly in preserving prior knowledge and maintaining strong performance on the most recent tasks. This indicates that the PKE module plays crucial role in mitigating catastrophic forgetting while enabling forward transfer. It should be noted that the PKE module cannot be used alone. Overall, the synergistic effect of CBA and PKE contributes stable and consistent improvements, validating the robustness of the proposed LEBA architecture MTIL.
T--NE visualization analysis:
We present t-SNE visualizations of ZSCL,, MoE-Adapter, and LEBA (Ours) on the Flowers and Aircraft tasks, using the final models obtained after completing all incremental sessions, shown in Fig. From visual perspective, LEBA exhibits more distinct

---

Aircrait
Floveer
ZSCL
MoE-Adapters
Owr
ZSCL
MoE-Adaplers
Ouus
Figure I: The t-SNE visualizations illustrate the representation evolution across multi-domain task-incremental learning sessions for various methods on two benchmark datasets.
The ablation experi- ments on the m-step.
Figure 2:
The ablation experi- -igerre 4: The ablation experi- ments on the adapter pool K. ments of computational cost.
Figure 3:
class-wise separability compared other baselines, suggesting improved representation stability and task-specific disentanglement. Notably, LEBA demonstrates stronger robustness in multi-domain continual learning, effectively consolidating previously acquired knowledge while flexibly adapting to novel tasks. These results highlight LEBA's advantage in achieving both knowledge retention and forward transfer in complex task-incremental scenarios.
The quantity of m-step: To assess the impact the iterative step size m on adapter integration in the LEBA framework, we plot the accuracy trends of "Transfer", "Average", "Last", and "Preserve" metrics as a function m-step as shown in Fig. 2 Across all metrics, the performance remains remarkably stable with increasing m, exhibiting minimal fluctuation. This consistency suggests that the model effectively preserves previously acquired knowledge while maintaining robust performance throughout the incremental learning process. The observed stability underseores the adaptability and robustness of our LEBA framework in mitigating catastrophic forgetting and sustaining high learning capacity, particularly in retaining and transferring knowledge across long sequences of tasks in MTIL.
Effectiveness of adapters pool size K: To investigate how the size of the adapter pool influences performance and resource efficiency in LEBA, evaluate the model under varying values of K. As shown in Fig. increasing the adapter pool size K yields only marginal changes across all four evaluation metries, with the model maintaining consistently high performance under different settings. This phenomenon consistent with observations in Mixture-of--xxperr models, where an excessive number of experts may lead to performance degradation [59] indicating that LEBA robust to the choice of pool size and does not depend on retaining large number of adapters to sustain its effectiveness. Given the negligible performance improvement beyond K 2, and considering the trade-off between memory overhead and model complexity, we select K 2 as the default configuration ensure efficient memory usage.
Computational cost: To evaluate the computational efficiency of LEBA, we compare the "Average" and memory consumption across different methods, shown in Fig. 4 Compared to existing approaches, LEBA achieves higher accuracy with significantly lower memory usage. This indicates that our method not only improves performance but also offers superior efficiency in resource utilization. These results further highlight the effectiveness of LEBA in multi-task incremental learning, demonstrating its advantage in balancing accuracy and computational cost.
Conclusion
In this paper, we propose a novel LEBA framework designed mitigate catastrophic forgetting in multi-domain task- inkkinccreeetal learning. A core component of LEBA is the continuous domain bridge adaptation establish stable transfer pathway between adapters, effectively aligning the distributions of previous and current tasks. Furthermore, our progressive knowledge ensemble departs from traditional replar paradigms by removing the dependency on task-learning order, allowing the model to revisit and integrate prior knowledge flexibly. Extensive experiments validate the effectiveness of LEBA enhancing both knowledge retention and transfer. future work, we plan to extend LEBA to broader AI domains beyond vision- languaee tasks.

---

6 Acknowledgement
This work was supported by the National Natural Science Foundation of China (Grant Nos. 62372238 and 62476133) and the National Science and Technology Major Project of China (Grant No. 2024ZDD5524600) and the Fundamental Research Funds for the Central Universities (Grant No.
11300-312200502507).
References
[1] Gido M Van de Ven, Hava T Siegelmann, and Andreas S Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature Communications, 11(1))4069, 2020.
[2] Liyuan Wang, Xingxing Zhang, Qian Li, Mingtian Zhang, Hang Su, Jun Zhu, and Yi Zhong. Incorporating neuro-inspired adaptability for continual learning in artificial intelligence. Nature Machine Intelligence, ((12):3366111) 1368, 2023.
[3] Yide Qiu, Shaoxiang I Ling, Tong Zhang, Bo Huang, and Zhen Cui. Unikg: benchmark and universal embedding for large-scale knowledge graphs. arXiv preprint arXiv:2309.05269, 2023.
[4] Xiantao Hu, Ying Tai, Xu Zhao, Chen Zhao, Zhenyu Zhang, Jun Li. Bineng Zhong, and Jian Yang. Exploiting multimodal spatial-temporal patterns for video object tracking. In Proceedings of the Conference Artificial Intelligence, volume 39, pages 3581-3589, 2025.
[5] Yide Qiu, Tong Zhang, Bo Huang, and Zhen Cui. Global variational convolution network for semi-supervised node classification on large-scale graphs. In Proceedings of the Chinese Conference on Putlern Recognition and Computer Vision, pages 192-204. Springer, 2023.
[6] Yuanzhi Wang, Ziqi Gu, Yide Qiu, Shuaizhen Yao, Fuyun Wang, Chunyan Xu, Wenhua Zhang, Dan Wang, Zhen Cui, et al. Mmm-rs: A multi-modal, multi- gsd, multi-scene remote sensing dataset and benchmark for text-to-image generation. Advances Neural Information Processing 2024.
Systems, 37:12151-1216
[7] Shuaizhen Yao, Xiaoya Zhang, Xin Liu, Mengyi Liu, and Zhen Cui. Stdd: Spatio-temporal dual diffusion for video generation. Proceedings of the Computer Vision and Pattern Recognition Conference, pages
2025.
[8] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in Neural Information Processing Systems, 30, 2017.
[9] Zhizhong li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-2947, 2017.
[10] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in Neural Information Processing Systems, 30, 2017.
[11] Ziqi Gu, Chunyan Xu, Jian Yang, and Zhen Cui. Few-shot continual infomax learning. In Proceedings of the International Conference on Computer Vision, pages 19224-19233, 2023.
[12] Ziqi Gu, Chunyan Xu, and Zhen Cui. Grassmann graph embedding for few-shot class incremen- tal learning. In Proceedings of the Chinese Conference Pattern Recognition and Computer Vision, pages 179-191. Springer, 2023.
[13] Li Sun, Junda Ye, Hao Peng, Feiyang Wang, and S Yu Philip. Self-supervised continual graph learning in adaptive riemannian spaces. In Proceedings of the Conference Artificial Intelligence, volume 37, pages 4633-4642, 2023.
[14] Enrico Fini, Victor G Turrisi Da Costa, Xavier Alameda-Pinedas Elisa Ricci, Karteek Alahari, and Julien Mairal. Self-supervised models are continual Iearners. In Proceedings of the Conference Computer Vision and Pattern Recognition, pages 9621-9630 2022.
[15] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experi- ence replay for continual learning. Advances Neural Information Processing Systems, 32, 2019.
10

---

[16] Zangwei Zheng, Mingyuan Ma, Kai Wang. Ziheng Qin, Xiangyu Yue, and Yang You. Preventing zero-shot transfer degradation continual learning of dision-language models. In Proceedings of the International Conference on Computer Vision, pages 19125-19136, 2023.
[17] Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang. Huchuan Lu, and You He. Boosting continual learning of vision-language models via nixture-of-experts adapters. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 23219-23230, 2024.
[18] Saced Masoudnia and Reza Ebrahimpour. Mixture of experts: a literature survey. Artificial Intelligence Review, 42:275-295, 2014.
[19] Michael Hersche, Geethan Karunaratne, Giovanni Cherubini, Luca Benini, Abu Sebastian, and Abbas Rahimi. Constrained few-shot glass-incremental learning. Proceedings of the Conference Computer Vision and Pattern Recognition, pages 9057-9067, 2022.
[20] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 6851, 2020.
[21] Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Alevs Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(7):3366-338 3385, 2021.
[22] Frantzeska Lavda, Jason Ramapuram, Magda Gregorova, and Alexandros Kalousis. Continual classification learning using generative models. arXiv preprint arXiv: 10612, 2018.
[23] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision, pages 139-154, 154, 2018.
[24] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwing et al. Overcoming catastrophic forgetting in neural networks. National Academy of Sciences, 1 114(13):3521-3222 2017.
[25] Sang-Wop Woo Lee, Jin- Kim, Jaehyun Jun, Jung-Wee Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances Neural Information Processing Systems, 30, 2017.
[26] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In Proceedings of the Conference Computer Vision and Puttern Recognition, pages 831-839, 2019.
[27] Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord. Dytox: Trans- formers for continual learning with dynamic token expansion. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 9285-9295, 2022.
[28] Zhiyuan Hu, Yunsheng Li, Jiancheng Lyu, Dashan Gao, and Nuno Vasconcelos. Dense network expansion for class incremental learning. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 11858-11861 1167, 2023.
[29] Fei Ye and Adrian G Bors. Self-evolved dynamic expansion model for task-free continual learning. Proceedings of the International Conference on Computer Vision, pages 22102- 22112, 2023.
[30] Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the Conference Computer Vision and Patlern Recognition, pages 3014-3023, 2021.
[31] Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with a network of experts. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 3366-3375, 2017.
2017.
11

---

[32] Valentin De Bortoli, James Thorton,, Jeremy Heng, and Arnaud Doucet. Diffusion schrödinger bridge with applications score-based generative modeling, Advances in Neural Information Processing Systems, 2021.
[33] Tianrong Chen, Quan-HHomg Liu, and Evangelos A. Theodorou. Likelihood training of schrödinger bridge using forward-backwe sdes theory. In Proceedings of the International Conference on Machine Learning, 2022.
[34] Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A. Theodorou, Weili Nie, and Anima Anandkumar. sb:: Image-to-image schrödinger bridge. In Proceedings of the International Conference on Machine Learning, 2023.
[35] Yaron Lipman, Ricky TQ Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
[36] Guan-Homg Liu, Yaron Lipman, Maximilian Nickel, Brian Karrer, Evangelos A Theodorou, and Ricky TQ Chen. Generalized schrödinger bridge matching. arXiv preprint arXiv:2310.02233, 2023.
[37] Beomsu Kim, Gihyun Kwon, Kwanyoung Kim, and Jong Chul Ye. Unpaired image-to-imagee translation via neural schrödinger bridge. arXiv preprint arXiv:2305. 5586,, 2023.
[38] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, pages 8748-8763. 2021.
[39] Hannes Risken. Fokker- planer-ppanck equation. The Fokker-Planee Equation: Methods of Solution and Applications, pages 63 95, 1989.
[40] Monroe D Donsker and SR Srinivasa Varadhan. On a variational formula for the principal eigenvalue for operators with maximum principle. National Academy of Sciences, 72(3):780- 783. 1975.
[41] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.5154 2013.
[42] Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, pages 178-178, 2004.
[43] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. Technical Report, University of Toronto, Toronto, Ontario, 2009.
[44] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the Conference Computer Vision and Pattern Recognition, pages 6606-3613, 2014.
[45] Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2226, 2019.
[46] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Proceedings of the Conference on Computer Vision. Graphics & Image, pages 722-729, 2008
[47] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food- frodd1001-minng discriminative components with random forests. In Proceedings of the European conference on computer vision, pages 446-461, 2014.
[48] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. Signal Processing Magazine, 1416)11 2012.
12

---

[49] Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 3498-3505, 2012.
[50] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the International Conference on Computer Vision Workshops, pages 554-561, 2013.
[51] Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, pages 3485-3492, 2010.
[52] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprins arXiv: 19929, 2020.
[53] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li. Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Proceedings of the International Conference Machine Learning, 1(2):3 2022.
[54] Robin Rombach, Andreas Blatmaann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[55] lya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 1rXivv1711.0500,, 2017.
[56) Rafael Müller, Simon Korbllith, and Geoffrey E Hinton. When does label smoothing help? Advances Neural Information Processing Systems, 32, 2019.
[57] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the Conference Com- puter Vision and Pattern Recognition, pages 2001-2010, 2017.
[58] Yuxuan Ding, Lingqiao Liu, Chunna Tian, Jingyuan Yang, and Haoxuan Ding. Don't stop learning: Towards continual learning for the clip model. arXiv preprint 277.222207.09248, 2022.
[59] Chufan Shi, Cheng Yang, Xinyu Zhu, Jiahao Wang, Taiqiang Wu, Siheng Li, Deng Cai, Yujiu Yang, and Yu Meng. Unchosen experts can contribute too: Unleashing moe models' power by self-contrast. Advances in Neural Information Processing Systems, 37:136897-13692,, 2024.
[60] Edward Nelson. Dynamical theories of Brownian motion, volume 101. Princeton university press, 2020.
13

---

A Proof of Continuous-Domain Bridge Adaptation
To support our continuous-domain bridge adaptation, draw on the theory of Schrödinger bridges and their connection to stochastic differential equations (SDEs). Specifically, we model knowledge transition a bidirectional diffusion process governed by forward and reverse SDEs, whose densities evolve according to the Bokker-Plancee equation. By conditioning on the samples' representations, we obtain deeeddepndeett posterior that aligns with the solution of a Schrödinger bridge.
We begin by recalling that the density evolution of an Itô process is governed by the following stochastic differential equations (SDEs):
d2m [5 + 8m V log (Zmm;m)]dn + Bn m, Zo po:
(13)
d2m = [ 3m V log V [[)mm,))ee + v m: 21 ~ PI,
where these SDEs correspond to forward and reverse Schrödinger bridge process and described by the Fokker-Planck equation 39
(2, Öp(z, m) = V (fm p) + 3m Ap. =p-)))..
(14)
dm
dm can be interpreted as the Fokker-Planck equation for the SDE. The equivalence V p holds up additive constant, which vanishes when applying the
We suggest that the PDE ((zm)
"Vigg" operator or in the context of the Fokker-Planck equation (since all operators are linear). A
similar interpretation applies to the PDE (V(a, can be equivalently viewed from the reversed time 8m
coordinate as:
(8(5.))
(v 33.
(15)
(ū(2)))
V f. 13.
8s
where : 1- m. This implies that () 8) can be interpreted as the density (up to constant factor) of the SDE as:
dZm fmdm VBmdWm. Zo ((.0)) dzm fmdm + dW m,
(16)
Z ~ (V 1),
Eqn. 5) naturally follows by conditioning Nelson's duality 60 i.e., q(= (),mm)(( boundary pair (Zo, Z1),
4( <Zm 2i) = V (2m, m Zo) V Zm, ml Z1).
Because (2 ml Zo) and (Z ml [ are solutions to the Fokker-Planck equations, we can express the posterior as the product of two Gaussian distributions:
( m m Zo) S (m, m Z7)
7mm
Zo
7m
Z1 1 ))
= =exp
02
62
(17)
02 m
0 g2 m
-M
m
Zo
+
Z1, 82
I),
02
+
02
σ2
+ 02
+ +o2
where of o 8m dm and σ2 1 3m dm represent the analytical marginal variances of the
SDEs Eqn. 16 when = O. We demonstrate that z( Zo. 7.) is the marginal density of the
DDPM posterior p( Zn, 17, 7 First, observe that when f == O. pl Zm Zo, Zn+1) takes the form of analytic Gaussian:
p(Z,, Zo. Zn+1)
a²
o I),
(18)
Ninn: 03 + 02
Zo
2
2
0
+ 0
0 + o
where we define a = pm++ Bm dm as the accumulated variance between two consecutive time steps (m,, 11) ). It is evident that at the boundary m, = mN-1, we obtain:
q( |N 70. 7N) p - Zo. 7x)
(19)
14

---

because ON-1 Smw-1 F. dm = NN Assuming the relation holds at mn+1- it is sufficient to demonstrate shown in:
q Zo. ZN) - p(Zn 1o, Zn+1) dNNn+++
(20)
Since both and are Gaussians, the Gaussian with the mean as:
on
0 n+1
n
Zo
+
Zo
+
ZN
3 o
a
+ o
o
oat
(21)
o
n
Zo
+
ZN.
0
+
on + o
where we the fact that o + of constant for all and that o o o design. Similarly, the right-hand side of Eq
0
- -+11 by
2q contains the covariance as:
a
th 22 m
oe + t
a o
+
on
2 1 + t
+ + +
(22)
o ( + o + ( - J
0 0 o
2 o n+1
We demonstrate the consistency of the continuous-doman bridge adaptation posterior with the DDPM backward posterior. This validates the continuous-doman bridge adaptation knowledge transitions in our framework.
B Other Result
Experimental results of order-I: We provide detailed experimental results under order-I in Ta- This setup reflects standard incremental learning protocol, allowing for a fair comparison across methods. The results demonstrate the effectiveness of our approach under this specific task progression.
Table 4: The accuracy (%) of our method (Ours) on the MTII benchmark with order-I. Each row shows the performance on each dataset for the model trained after the corresponding task. The metrics for Transfer
Average
Last
and
Preserve are highlighted color.
Caltech101
CIFARIC0
OxfordPar
Atronn
DamSAT
Flowers
MNIST
SUN39P
DTD
Food
Curs
Transfer
88.5
68.3
44.8
49.4
70.2
88.6
60.9
89.1
64.8
64.2
69.2
Aircratt
$6.1
88.5
44.8
55.3
71.1
89.1
59.5
89.
64.8
Caltech10l
53.1
97.1
68.3
44.8
55.3
70.9
$8.5
59.5
89.1
64.8
65.6
CIFAR
$3.3
95.4
89.4
44.8
70.8
59.5
89.
64.8
65.6
52,5
95.0
86.2
81.3
42.7
68.9
88.5
62.7
89.
64.8
63.4
EuroSAT
52.9
95.0
86.2
81.2
98.3
68.9
S8.5
62.8
89.
64.8
63.6
Flowers
52.4
95.6
86.3
81.2
97.8
$8.5
62.7
89.
64.8
63.5
95.6
80.4
97.6
59.5
89.
64.8
MNIST
$1.9
95.6
87.4
80.4
96.9
97.5
89.5
99.1
89 ]
64.8
63.3
Oxfordeet
53.5
95.6
87.2
80.5
97.3
89.5
99.1
89.6
64.8
63.7
99.1
SUN397
55.1
95.2
87.4
78.8
97.3
99.1
89.6
88.8
82.4
87.3
Preserve
53.7
95.4
87.0
80.4
96.9
97.4
89.5
99.1
89.6
88.8
84.5
Average
53.9
91,9
83.8
70.8
79.8
85.1
89.1
74.8
89.3
69.2
65.8
77.9
Mixup training: Our LEBA adopts a phased training strategy: first trains new domain adapter independently, then fine-tunes it through cross-domain bridging, rather than mixing new and replayed data for joint optimization. This design choice motivated by the significant distributional shift across domains ciioiinsddiretlyyy mixing replayed samples with current task data would require the model to
15

---

align with multiple domains simultaneously, which can hinder adaptation to the new task. As shown in the [5 our experimental results validate this observation. Moreover, since the amount of data for new tasks is typically much smaller than the replayed historical samples, joint training may lead to overfitting on past tasks and suppress the domain- -specifc adaptation required for the new task.
Table 5: Performance comparison between LEBA and mixup training strategy
Transfer
Average
Last
Preserve
Mixup Training 68.3
76.2
85.4
82.6
Ours (LEBA)
69.2
77.9
87.3
84.5
We conduct ablation studies on the balance parameters 7 and B, as shown in the Tables and The best performance is achieved when y 0.1 and 3 0.4. Setting too high (e.g., 7 0.1) causes the CBA loss to dominate training, hindering the acquisition of new knowledge. Likewise, large 3 (e.g., B > 0.4) leads to overly smoothed integration weights, which diminish task- specific distinctions and negatively impact performance.
Balance factor analysis:
Table Sensitivity to balance factor Y
Table 7: Sensitivity to balance factor 3
Y
Avg
Last
Transfer
Preserve
Avg
Last
Transfer
Preserve
0.05
76.1
85.2
68.4
84.1
0.2
76.1
85.2
68.4
83.6
0.1
77.9
87.3
69.2
84.5
0.4
77.9
87.3
69.2
84.5
0.2
77.3
86.1
68.8
83.4
0.6
77.3
86.1
68.8
83.1
We evaluated the training time and memory usage of our method compared to other methods. As shown in Table 8 the proposed LEBA framework achieves better computational efficiency compared to existing baselines. It requires less GPU memory and converges faster during training. This demonstrates that LEBA not only improves performance but also reduces resource overhead.
Memory usage and training time:
Table 8: Comparison of memory usage and training time
Method
GPU (MiB)
Training Time (Min)
ZSCL 16
28,293
823.1
MoB-Adapter
26,294
803.4
Ours (LEBA)
24,698
786.6
Threshold 1) analysis: We investigate the effect of varying the threshold parameter which controls adapter selection in our method. As shown Table performance remains relatively stable across a range of 1) values, indicating the robustness of our approach. Notably, the best overall performance achieved when 7 = 0.3, suggesting an optimal balance between selective adapter reuse and new knowledge integration.
Table 9: Performance sensitivity to threshold y
1
Avg
Last
Transfer
Preserve
0.1
76.6
85,9
68.3
84.6
0.3
77.9
87.3
69.2
84.5
0.5
77.5
86.8
68.8
83.7
0.7
77.4
86.6
68.5
83.6
Other task order: we randomized the task order and conducted two independent experiments based on the resulting orders. The corresponding results are presented Table and Table with the task orders specified as: (a) [CIFAR 100, DTD, Aircraft, Flowers, Food, StanfordCars, MNIST, EUrOSAT, SUN397, OxfordPet, Caltech 101] and (b) [TuroSAT, OxfordPet, SUN397, DTD, CIFAR 100, Food, StanfordCars, MNIST, 101, Flowers, Aircraft]. Experimental results demonstrate that the
lnd
16

---

proposed LEBA consistently achieves superior performance compared to state-of ofeieeooftheeeee-e aef-th--art art methods across different randomized task orders, indicating its robustness to task order variations.
Table 10: Comparison of methods across four evaluation metrics for task order (a)
Method
Transfer
Average
Last
Preserve
ZSCL MoE-Adapters
67.23
75.89
84.21
81.35
68.67
76.21
85.36
82.65
Ours(LEBA)
69.42
77.66
87.13
84.01
Table Comparison of methods across four evaluation metrics for task order (b)
Method
Transfer
Average
Last
Preserve
ZSCL
60.95
75.12
83.54
85.32
MoB-Adapters
61.57
75.26
84.92
86.16
Ours(LEBA)
62.46
76.69
86.62
88.93
17

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The key contributions and scope claimed in abstract and Introduction T We provide experimental results in Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
The following the limitations of our method. Considering that the Schrödinger bridge method employed our paper utilizes relatively fundamental and classical techniques.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
18

---

Answer: [Yes]
Justification:
The relevant proof process has been shown in the proof of appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In the Sec. detailed algorithm, network framework and experimental setting are given.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
19

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: No codes are included in this submission, but the codes will be provide when the paper is accected.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
pub public//e /odee//uuud CooeeSuoeeeeeeeeeeeo wwwssssoosiiiooooooooooossissiis for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips cc/ wwtppcplwbicccooop
CodeSuumee Suwmissiomns.n...........s...m.u onP ciooooiopiiyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The content involved the question has been experimentally analyzed in the Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due to the increased training time, error bars were omitted from the presen- tation. It's worth noting that the absence of error bars in the previous MTIL method is consistent with our approach.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The computer resource requirements are given in the Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http::/://..aaaae.../c/////lle/a
Answer:
[Yes]
Justification:
We have reviewed the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
21

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The pre-model and datasets involved in the experiment have been mentioned in Sec. 4
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paaperswi haperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
23

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The LLM used only for writing, editing, formatting purposes and does not impact the core methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
hestte///eerriss.cc//ooeeeeeeeee LLM) for what should or should not be described.
Please refer to our LLM
24


Reviews:
[{"review_id": "1RCSGgV8Ee", "reviewer": "Reviewer_zShr", "summary": "The paper presents a framework for multi-domain task incremental learning that works by learning and using a cross-domain adapter to preserve previous knowledge as well as to transfer knowledge to new tasks. The contribution is two-fold: 1) a cross-domain adapter that aligns historical adapters through distribution transfer, and 2) a pool of adapters to store historically significant adapters. For 1), authors propose to use the Schrödinger Bridge (SB) mechanism to facilitate inter-task knowledge transfer by aligning probability distributions between current and previous adapters. For 2) the framework dynamically maintains an adapter pool of size K to store useful historical adapters. For each new adapter, its similarity to those in the pool is measured and the pool is updated accordingly. Experiments are run over a comprehensive set of datasets corresponding to 11 tasks, evaluating aspects like Transfer, Last task performance and Average performance. Additionally, forgetting is also measured. Ablation studies consider aspects such as removal of components and evaluation of the effect of hyperparameters.", "questions": "- What do you actually mean when you use the word \"ensemble\" in the context of the pool of adapters Pk?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper addresses an important problem of constructing cross-domain adapters in the multi-domain task incremental learning setting. \n- The paper’s contributions seem novel in the sense that two different methods are proposed: one for performing cross-domain adaptation while learning tasks sequentially, and the other for selectively maintaining a pool of adapters that might me more useful for preserving knowledge as well as for future tasks.\n- The paper is very well-written and presentation is excellent. \n\nWeaknesses:\n- The main weakness that I observe is in the experiments, where two possible task orders are used: alphabetically and random. Although I appreciate the consideration of multiple orders, I think these are insufficient to mitigate the effect that a particular task order may have on the results. I would expect to see at least a few (possibly random) orders, and average results of the metrics across these. \n- Another weakness that I observe is in the description of the pool of adapters, where the word “ensemble” is repeatedly used. Based on the design proposed for this method as described in section 3.4, it does not entail an “ensemble” as known, but rather it’s merely a pool that gets updated depending on the similarity of new adapters with previous ones.", "was_revised": true, "final_justification": "Based on the discussion with authors, I will maintain my score.", "rebuttal": "Thank you for your positive feedback on the novelty and writing quality of LEBA. For your concerns, we make the response below.\n\n**Q1: The main weakness that I observe is in the experiments, where two possible task orders are used: alphabetically and random. Although I appreciate the consideration of multiple orders, I think these are insufficient to mitigate the effect that a particular task order may have on the results. I would expect to see at least a few (possibly random) orders, and average results of the metrics across these.**\n\n**A1:** In accordance with your suggestion, we randomized the task order and conducted two independent experiments based on the resulting orders. \nThe corresponding results are presented in blow table, with the task orders specified as: (a) [CIFAR100, DTD, Aircraft, Flowers, Food, StanfordCars, MNIST, EuroSAT, SUN397, OxfordPet, Caltech101] and (b) [EuroSAT, OxfordPet, SUN397, DTD, CIFAR100, Food, StanfordCars, MNIST, Caltech101, Flowers, Aircraft].\nExperimental results demonstrate that the proposed LEBA consistently achieves superior performance compared to state-of-the-art methods across different randomized task orders, indicating its robustness to task sequence variations. \n\n\nComparison of methods across four evaluation metrics for task order (a)\n|Method|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|ZSCL [1]|67.23|75.89|84.21|81.35|\n|MoE-Adapters [2]|68.67|76.21|85.36|82.65|\n|Ours (LEBA)|69.42|77.66|87.13|84.01|\n\nComparison of methods across four evaluation metrics for task order (b)\n|Method|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|ZSCL[1]|60.95|75.12|83.54|85.32|\n|MoE-Adapters[2]|61.57|75.26|84.92|86.16|\n|Ours(LEBA)|62.46|76.69|86.62|88.93|\n\nFurthermore, by combining these results with those obtained under the randomized task order (Order II) presented in our manuscript, we compute the average performance across all four experimental settings, as shown below table.\nExperimental results demonstrate that our proposed LEBA consistently outperforms current state-of-the-art methods under randomized task orders.\n\nAverage performance across three randomized task orders\n|Method|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|ZSCL[1]|64.09|75.17|83.71|82.72|\n|MoE-Adapters[2]|64.84|75.39|84.79|83.90|\n|Ours|65.66|75.65|86.71|86.24|\n\n[1] Preventing zero-shot transfer degradation in continual learning of vision-language model ICCV 2023.\n\n[2] Boosting continual learning of vision-language models via mixture-of-experts adapters CVPR 2024.\n\n**Q2: Another weakness that I observe is in the description of the pool of adapters, where the word “ensemble” is repeatedly used. Based on the design proposed for this method as described in section 3.4, it does not entail an “ensemble” as known, but rather it’s merely a pool that gets updated depending on the similarity of new adapters with previous ones.\nWhat do you actually mean when you use the word \"ensemble\" in the context of the pool of adapters Pk?**\n\n**A2:** We appreciate your comment and would like to provide additional clarification regarding the ensemble mechanism.\nEnsemble learning typically refers to the process of combining multiple models to improve robustness and generalization. In our LEBA, the ensembling process doesn't happen in the construction of the adapter pool itself, but through the adaptive integration of prior knowledge as defined in Eqn.(9) and Eqn.(10) of the manuscript.\nSpecifically, we introduce a set of learnable parameters that assign sample-specific weights to each adapter in the pool. These weights are used to dynamically combine the outputs of historical adapters based on their relevance to the current task. This mechanism allows the model to selectively integrate previously learned knowledge in a principled and flexible manner, which aligns with the essence of ensemble learning.\nWe will also make the distinction clearer between the adapter pool construction and the ensemble operation performed over it."}, {"review_id": "ecRMG71vVa", "reviewer": "Reviewer_wSj1", "summary": "The paper addresses the multi-domain, task-incremental learning paradigm. The proposed method manages a pool of adapters from previous tasks. When training on a new task, the method aligns the feature distribution of the current adapter with those of previous ones. Instead of storing old data, the authors propose using a diffusion model to generate samples based on previously seen semantic concepts. The method is evaluated on two different orderings of 11 tasks.", "questions": "- What are the underlying reasons that a larger pool size (K) does not improve performance?\n- How many replay samples do you generate, and how does this number affect performance?\n- How is the \"semantic set\" actually computed? Is it based solely on class labels?\n- In line 193, what metric is used to identify the \"least representative\" sample?\n- Under what conditions do you replace an existing adapter, ignore the current one, or add a new one to the pool?\n- How were the hyperparameters selected? Do you have recommendations for setting them on other benchmarks?\n- Do you use the task ID at test time? Is the method applicable to multi-domain, class-incremental learning?\n- How many random seeds were used for the experiments, and what is the standard deviation of the results?\n- I expected that PKE combined with replaying generated samples would lead to higher gains than those reported in Table 3. Is there an intuition for why this module's contribution appears to be marginal?\n- What are the exact benefits that PKE offers over standard replay for addressing task-order sensitivity?\n- For heterogeneous learning, which approach is better: learning distinct knowledge specific to each domain, or forcing alignment between domains, which could be suboptimal if the domains are very different?", "limitations": "Not thoroughly addressed.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n- The paper addresses an interesting and important research problem that has received little attention: the presence of multiple, heterogeneous domains in a continual learning setting.\n- It proposes a new way of replaying experiences using diffusion models.\n- The proposed method achieves better performance than recent work.\n\nWeakness: \n- Some details are missing (see the questions section below).\n- Some claims could be more strongly supported. For instance, the claim of robustness to task order could be better supported by studying more than two different orders.\n- The computational cost of the approach and the usage of the diffusion model are not studied. (In line 309, the paragraph title is \"Computational Cost,\" but it discusses memory costs instead.)\n- It is unclear why an ensemble with more than two members does not help improve performance.", "was_revised": true, "final_justification": "The authors addressed one of my main concerns regarding the task order and provided extra experiments for some unclear points.\nThere is still some limitations and points of improvements for future work. The authors provided a discussion for them in the rebuttal and would be valuable to be included in the revision.", "rebuttal": "Thanks for your positive comments (an interesting and important research,  a new way of replaying experiences). All your questions have been answered below. \n\n**Q1: Why does increasing the adapter pool size beyond two not yield further performance gains?**\n\n**A1:** To evaluate the effect of the adapter pool size, we conduct ablation studies varying $K$. As illustrated in Figure 3 of the manuscript, the model performance tends to saturate when the adapter pool size reaches $K = 2$. Consequently, adding more adapters introduces fluctuations.Our LEBA uses learnable weights to assign importance to each adapter based on its relevance to the current replayed sample. As a result, a larger $K$ provides diminishing returns in terms of both representational diversity and performance gains. Given the negligible performance improvement beyond $K=2$, and considering the trade-off between memory overhead and model complexity, we select $K=2$ as the default configuration to ensure efficient memory usage.\n\n**Q2: How many replay samples do you generate?**\n\n**A2:** We generate 32 replay samples per iteration over 500 iterations for each task. To further investigate the impact of replay quantity, we conduct additional experiments by generating training samples over 500 iterations with either 16 or 64 samples per iteration. The results are shown in the below table.To balance performance and computational cost, our LEBA adopts the 500$\\times$32 replay setting, which achieves competitive results while maintaining efficiency.\n\nPerformance on varying numbers of replayed samples\n|Replay number|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|500$\\times$16|68.78|77.46|86.27|83.16|\n|500$\\times$32 (Ours)|69.20|77.90|87.30|84.50|\n|500$\\times$64|69.43|78.23|87.51|84.67|\n\n**Q3: How is the \"semantic set\" actually computed? Is it based solely on class labels?**\n\n**A3:** The semantic set refers to the collection of all classes encountered throughout the incremental learning process, and is defined solely based on the class labels.\nIn the manuscript, we elaborate on the definition of the semantic set as \" *The semantic set $\\mathcal{C}^t \\coloneqq \\{c_j^t\\}_{j=1}^{M_t}$ describes certain semantic information (e.g., class information $y_i^t$), with $M_t$ denoting the number of distinct class names.*\"\n\n**Q4: In line 193, what metric is used to identify the \"least representative\" sample? Under what conditions do you replace an existing adapter?**\n\n**A4:** Please allow us to offer a further clarification. In line 193, the entity being evaluated as the \"least representative\" is not a training sample, but rather a task-specific adapter from the dynamic adapter pool. To determine whether a newly learned adapter should replace an existing one, we compute the KL-divergence between the current adapter and each adapter in the pool, based on their responses to the current task samples. A fixed threshold is employed to guide the replacement decision: if the divergence is below the threshold, the adapter pool remains unchanged; otherwise, the newly learned adapter replaces the most divergent one, thereby updating the pool accordingly.\n\n**Q5: How were the hyperparameters selected?**\n\n**A5:** We present the hyperparameter studies and corresponding rationale in the supplementary material (Lines \\#519–\\#523). The same hyperparameter settings are used for the other benchmarks. \n\nSensitivity to balance factor $\\gamma$\n|$\\gamma$|Avg|Last|Transfer|Preserve|\n|-|-|-|-|-|\n|0.05|76.1|85.2|68.4|84.1|\n|0.1|77.9|87.3|69.2|84.5|\n|0.2|77.3|86.1|68.8|83.4|\n\nSensitivity to balance factor $\\beta$\n|$\\beta$|Avg|Last|Transfer|Preserve|\n|-|-|-|-|-|\n|0.2|76.1|85.2|68.4|83.6|\n|0.4|77.9|87.3|69.2|84.5|\n|0.6|77.3|86.1|68.8|83.1|\n\nThe best performance is achieved when $\\gamma = 0.1$ and $\\beta = 0.4$. Setting $\\gamma$ too high (e.g., $\\gamma > 0.1$) causes the CBA loss to dominate training, hindering the acquisition of new knowledge. Likewise, a large $\\beta$ (e.g., $\\beta > 0.4$) leads to overly smoothed integration weights, which diminish task-specific distinctions and negatively impact performance.\n\n**Q6: Do you use the task ID at test time? Is the method applicable to multi-domain class-incremental learning?**\n\n**A6:** Unlike class-incremental settings, task-incremental learning in MTIL typically assumes that the task ID is known at test time that follows the previous work [1,2]. Thus we also strictly follow the previous protocol in task-incremental learning [3]. Currently, our LEBA is not directly applicable to multi-domain class-incremental learning, as it is tailored for multi-domain task-incremental settings where task identities are known during training and inference, with each adapter aligned to a task-level distribution. In contrast, class-incremental learning requires unified classification without task labels, fundamentally altering the problem. Adapting our LEBA would require merging or aligning adapters across domains and classes, posing challenges in representation overlap and compatibility. We will explore such adaptations in future work.\n\n**Q7: How many random seeds were used for the experiments, and what is the standard deviation of the results?**\n\n**A7:** Following your suggestion, we have conducted experiments with five different random seeds and computed the standard deviations. The results are shown in the below table. The low standard deviations across all metrics demonstrate the robustness and stability of LEBA under randomized conditions.\n\nPerformance and standard deviation of LEBA across four metrics\n|Metric|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|LEBA (Ours)|69.20|77.90|87.30|84.50|\n|Std. Dev.|±0.09|±0.23|±0.13|±0.28|\n\n**Q8: Why does PKE with generative replay yield only modest gains in Table 3?**\n\n**A8:** We would like to clarify that the PKE module actually includes the replay of generated data (as described in Line \\#201). While the numerical improvements in Table3 may appear modest, the contribution of PKE lies not in delivering large standalone gains, but in enhancing robustness and consistency. Specifically, PKE adaptively integrates previously learned adapters during replay, facilitating stable knowledge reuse and sample-level alignment. These benefits are reflected in the consistent improvements observed across all four evaluation metrics.\n\n**Q9: What are the exact benefits that PKE offers over standard replay for addressing task-order sensitivity?**\n\n**A9:** Compared to standard data replay, the PKE module alleviates task-order sensitivity through two strategies.  \nFirst, it introduces task-order independence by leveraging a dynamic adapter pool with learnable weights, enabling the adaptive integration of prior knowledge regardless of task sequence. Second, it supports semantic-level replay, where generated samples are conditioned on accumulated semantic concepts rather than stored per-task instances, resulting in more balanced and representative replay across tasks.\n\n**Q10: For heterogeneous learning, is it better to learn domain-specific knowledge or enforce cross-domain alignment?**\n\n**A10:** The issue you mentioned remains an open research challenge. Different methodological paradigms offer distinct advantages: learning domain-specific knowledge emphasizes specialization, while enforcing cross-domain alignment promotes generalization.\nOur work falls into the latter category—focusing on task alignment across domains via a shared adapter integration framework. Although this may introduce some suboptimality when domain gaps are large, we mitigate this risk through adaptive adapter selection and semantic-aware replay, which promote stable transfer while preserving task-relevant knowledge.\n\n**Q11: The robustness claim would be stronger with results on more than two task orders.** \n\n**A11:** According to your suggestion, we randomized the task order and conducted two independent experiments based on the resulting orders. \nThe corresponding results are presented in blow table, with the task orders specified as: (a) [CIFAR100, DTD, Aircraft, Flowers, Food, StanfordCars, MNIST, EuroSAT, SUN397, OxfordPet, Caltech101] and (b) [EuroSAT, OxfordPet, SUN397, DTD, CIFAR100, Food, StanfordCars, MNIST, Caltech101, Flowers, Aircraft].\n\nComparison of methods across four evaluation metrics for task order (a)\n|Method|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|ZSCL [1]|67.23|75.89|84.21|81.35|\n|MoE-Adapters [2]|68.67|76.21|85.36|82.65|\n|Ours (LEBA)|69.42|77.66|87.13|84.01|\n\nComparison of methods across four evaluation metrics for task order (b)\n|Method|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|ZSCL[1]|60.95|75.12|83.54|85.32|\n|MoE-Adapters[2]|61.57|75.26|84.92|86.16|\n|Ours(LEBA)|62.46|76.69|86.62|88.93|\n\nExperimental results demonstrate that the proposed LEBA consistently achieves superior performance compared to state-of-the-art methods across different randomized task orders, indicating its robustness to task sequence variations. \n\n**Q12: In line 309, the paragraph title is \"Computational Cost,\" but it discusses memory costs instead.**\n\n**A12:** Sorry for the typo Figure 4 of our manuscript. It should be corrected to \"GPU Memory (MiB)\". Figure 4 on the manuscript reports GPU memory consumption, and we will revise this error accordingly in the next version. Furthermore, Table 8 of the supplementary material presents a comparison of training time, as shown in the table below, where our LEBA demonstrates slightly better efficiency compared to current state-of-the-art methods.\n\nComparison of memory usage and training time\n|Method|GPU (MiB)|Training Time (Min)|\n|-|-|-|\n|ZSCL|28293|823.1|\n|MoE-Adapter|26294|803.4|\n|Ours (LEBA)|24698|786.6|\n\n[1] Preventing zero-shot transfer degradation in continual learning of vision-language model ICCV 2023.\n\n[2] Boosting continual learning of vision-language models via mixture-of-experts adapters CVPR 2024.\n\n[3] Enhancing knowledge transfer for task incremental learning with data-free subnetwork NIPS 2023."}, {"review_id": "jImLc8Kvjm", "reviewer": "Reviewer_VGXi", "summary": "This paper presents a new framework for multi-domain task incremental learning (MTIL), called Learning and Ensembling Bridge Adapters (LEBA). LEBA contains two main technical components, continuous-domain bridge adaptation (CBA) and progressive knowledge ensemble (PKE). The CBA moduel aligns the latent distributions of current and previous adapters via Schrodinger Bridge, and the PKE module generates synthetic samples conditioned on previously seen semantic concepts (e.g., classes) using a pretrained diffusion model. Experiments consider two task orders and multiple benchmarks, showing that LEBA outperforms baselines such as ZSCL and MoE-Adapters.", "questions": "Please refer to the weaknesses above.", "limitations": "Not clearly presented.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "(Strengths)\n1. The paper is well-organized and generally well-written, despite verbose descriptions in some theoretical parts.\n2. Schrodinger bridge is firstly employed in the context of continual learning, which appeals some extent of novelty.\n3. The proposed method achieves strong empirical results across multiple metrics, Transfer, Average, Last, and newly proposed Preserve.\n4. The method is resource-efficient in memory and training time, making it suitable for practical MTIL applications.\n\n\n(Weaknesses)\n1. The claim that the method is “independent of task-learning sequence” is overstated. Replay samples are generated based on an accumulated set of prior class labels {$C_j$}$_{j=1}^{t-1}$, which are inherently tied to task order. Therefore, the replay process remains temporally dependent.\n2. The use of semantic prompts alone (i.e., class labels) to condition the diffusion model assumes that these labels are sufficient to recover representative samples of previous domains. However, no evidence is provided to validate the quality or fidelity of these generated samples relative to the original data distributions.\n3. The adapter pool size is fixed to K=2, and increasing it shows negligible gain, suggesting that either the replayed information is not diverse, or the adapter selection mechanism is too coarse.\n4. The CBA module, while well-formulated, does not introduce novel theoretical contributions beyond adapting existing SB-based score matching to the adapter alignment setting.\n5. The writing quality in the Related Work section could be improved for clarity and logical flow. For instance, the opening sentence refers vaguely to \"the above method\" without specifying which work is meant, creating confusion in a standalone section.\n6. The presentation of Section 3.3 is overly technical and difficult to follow. The notation is heavy and inconsistent at times, and the core intuition (i.e., aligning latent distributions of adapters using Schrödinger Bridge dynamics( is obscured by excessive mathematical detail.", "was_revised": true, "final_justification": "The rebuttal addresses several concerns and provides clarifications on task-order independence and sample fidelity, which are helpful. While some claims could still benefit from stronger empirical support, the authors’ willingness to temper statements and incorporate clarifications is encouraging. I will maintain a borderline rating, leaning toward acceptance.", "rebuttal": "Thanks for your positive comments on the well-organized structure, the extent of novelty, and the comprehensive experiments. The point-to-point answers are provided below.\n\n**Q1: The claim of being “independent of task order” is overstated, as replay relies on accumulated class labels, which are inherently tied to task sequence, making the process still temporally dependent.**\n\n**A1:** Thank you for raising this point. We would like to clarify our intended meaning behind the claim that our method is \"independent of task-learning sequence\". This statement refers specifically to the replay process in LEBA, which differs from standard replay-based methods that typically require explicit task identifiers or must associate each replayed sample with its original task or dataset. In contrast, LEBA generates replay samples based on an accumulated semantic set—i.e., the union of previously encountered class labels—without requiring knowledge of which specific task or dataset the sample originally belonged to.\nThis task-agnostic design is the motivation for constructing a semantic concept set, and it enables flexible and unified replay across all previously learned knowledge, thus reducing the system’s reliance on task sequence during knowledge consolidation.\nWe will revise this in the final version to better reflect this distinction and avoid overstating the claim.\n\n**Q2: Relying solely on class labels to condition the diffusion model assumes they suffice to recover representative samples, yet no evidence is provided to validate their fidelity to original distributions.**\n\n**A2:** Thank you for raising this point.\nWe would like to clarify that the generated samples used in the PKE module are produced by a diffusion-based model (DDPM) [1], which has demonstrated strong generative quality—achieving FID 3.17 and IS 9.46 on CIFAR dataset, indicating sufficient fidelity and discriminability relative to the original data distribution.\nIn the Table 3 of the manuscript, incorporating the PKE module which includes the use of these generated replay samples—results in a good performance gain across all evaluation metrics, confirming that the generated samples are informative and beneficial to the continual learning process.\nAs part of future work, we plan to explore post-processing strategies, or employ more advanced generative models to further enhance the fidelity and diversity of replay data.\n\n[1] Denoising Diffusion Probabilistic Models NIPS 2020.\n\n**Q3: The adapter pool size is fixed and increasing it shows negligible gain, suggesting that either the replayed information is not diverse, or the adapter selection mechanism is too coarse.**\n\n**A3:** We would like to clarify that the negligible gain from increasing the adapter pool size $K$ is not attributed to a lack of diversity in the replayed information or a coarse adapter selection mechanism. First, our replay samples are generated via a diffusion-based generator conditioned on the accumulated semantic set, ensuring high semantic diversity across tasks without relying on raw data storage. Second, adapter selection is guided by KL divergence, which replaces only semantically redundant adapters and preserves a compact yet diverse pool.\nAs illustrated in Figure 3 of the manuscript, the model performance tends to saturate when the adapter pool size reaches $K = 2$. \nAs a result, expanding the adapter pool offers negligible performance gains. \nSpecifically, LEBA employs learnable weights to assign importance scores to adapters based on their relevance to current replay samples. Consequently, larger values of $K$ lead to diminishing returns in both representational diversity and performance gains. Considering the minimal improvement beyond $K=2$ and the trade-off between memory overhead and model complexity, we adopt $K=2$ as the default setting to ensure efficient memory utilization.\n\n**Q4: The CBA module, while well-formulated, does not introduce novel theoretical contributions beyond adapting existing SB-based score matching to the adapter alignment setting.**\n\n**A4:** While the CBA module builds on established principles of Schrödinger Bridge (SB)-based score matching, its novelty lies in adapting and extending this framework to the unique setting of continual adapter alignment in multi-domain task incremental learning (MTIL).\nUnlike prior work that applies SB to generative modeling, we recontextualize it for cross-domain feature alignment across sequential tasks, using replayed samples and adapter responses as the distributions to be bridged. \nTo our knowledge, this is the first work to apply SB-based methods for aligning modular representations in continual learning, offering a new perspective on stable knowledge transfer under task shifts.\n\n**Q5: The writing quality in the Related Work section could be improved for clarity and logical flow.**\n\n**A5:** Thank you for the comment. We will refine the Related Work in the next version to improve its writing quality."}, {"review_id": "WmQrdXl5an", "reviewer": "Reviewer_SAmD", "summary": "This paper introduces LEBA (Learn and Ensembling Bridge Adapters), a novel framework designed to tackle the challenging problem of Multi-domain Task Incremental Learning (MTIL). The core idea is to mitigate catastrophic forgetting and enhance knowledge transfer across a sequence of diverse domains. The framework is built upon two key contributions: (1) Continuous-domain Bridge Adaptation (CBA), which, for the first time, applies Schrödinger Bridge theory to model the knowledge transition between adapters as an optimal transport problem, aligning the feature distributions of new and past tasks; and (2) Progressive Knowledge Ensemble (PKE), a strategy that circumvents the need to store past data by using a diffusion-based generator for replay. PKE further enhances efficiency by maintaining a small, dynamic pool of useful historical adapters and using adaptive weights to create a more accurate representation of past knowledge. Through extensive experiments, the authors demonstrate that LEBA achieves state-of-the-art performance on the MTIL benchmark, outperforming existing methods.", "questions": "1. Could the authors elaborate on the trade-off between the flexibility of generative replay and the potential performance impact from distributional bias in the generated samples? How sensitive is the framework to the quality of the generator G?\n\n2. The framework uses historical adapters' outputs as the target for alignment. How robust is this process if these historical adapters are themselves suboptimal? Do the PKE mechanisms (the dynamic pool and adaptive weights) sufficiently mitigate the risk of aligning to a flawed representation of the past?\n\n3. While KL-divergence looks a reasonable choice, could the authors justify that this single metric is sufficient to capture the multi-faceted concept of an adapter's \"usefulness\"? Have alternative, more sophisticated selection mechanisms (e.g., based on diversity) been considered?\n\n4. The fixed-size adapter pool ($P_K$) effectively ensures stability as the number of total tasks ($T$) grows. However, how does this stability mechanism interact with the replay batch size? Have the authors tested the framework's robustness with larger replay batches, and does a larger number of simultaneous alignment targets introduce new optimization challenges?\n\n5. The authors state that the code will be released upon acceptance.  Could they confirm if the release will include the necessary assets and scripts to faithfully reproduce the main results presented in Tables 1 and 2?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n1. The primary strength of this work lies in its novel formulation of the knowledge transfer problem in continual learning. Framing the alignment between past and present adapters as an optimal transport problem solvable by a Schrödinger Bridge is an interesting and original conceptual leap.  This approach moves beyond simple regularization or replay and introduces a more principled way to model the \"pathway\" of knowledge.\n\n2. The LEBA framework is well-designed, addressing multiple facets of the MTIL problem. The synergy between the CBA and PKE modules is a key strength. While CBA provides the theoretical foundation for knowledge alignment, PKE offers a practical and flexible solution for revisiting past knowledge without the constraints of traditional replay methods (i.e., being independent of task order). \n\n3. The paper provides compelling empirical evidence to support its claims. It compares LEBA against a strong and comprehensive set of baselines, including classic continual learning methods and recent state-of-the-art models, on a standard MTIL benchmark.  The use of multiple evaluation metrics and two different task orders (including a random one) demonstrates the robustness of the method.  Furthermore, the detailed ablation studies (e.g., on the adapter pool size and internal modules) effectively validate the contribution of each component of the proposed framework.\n\n**Weaknesses:**\n\n1. The framework relies on strong assumptions to make the elegant but complex theory computationally tractable. The Gaussian Bridge Path Assumption (Equation 5) is a significant simplification.  Real-world feature distributions from deep networks are often complex and multi-modal. Approximating the diffusion path as a simple Gaussian introduces a potential modeling error that could limit the optimality of the learned knowledge bridge. \n\n2. The use of KL-divergence as a sole metric for adapter usefulness (Equation 8) is a heuristic simplification of a complex concept.  Usefulness could also encompass factors like knowledge diversity or complementarity, which may not be fully captured by a single similarity score.\n\n3. The framework relies on samples from a pre-trained diffusion model.  These models are known to have their own biases and do not perfectly replicate the true data distribution, which could potentially mislead the alignment process.\n\n4. The framework is reactive, evaluating the importance of a sample after it has been generated.  There is no mechanism to guide the generator G to proactively create samples that are predicted to be most beneficial for mitigating forgetting at a given training stage.\n\n5. A key hyperparameter, the number of replayed samples  $|\\hat{X}^t|$ per training step, is not explicitly mentioned, which hinders exact reproducibility.", "was_revised": true, "final_justification": "Thanks a lot for the authors for their efforts to address the comments, which have been largely addressed. While I am not still fully convinced that the assumption (approximation) of the paper has been very well justified, I believe this revised version deserves a raised score.", "rebuttal": "Thank you for your positive comments (novel formulation of the knowledge transfer problem,  well-designed). All your questions and weeknesses have been answered below. \n\n**Q1: Could the authors elaborate on the trade-off between the flexibility of generative replay and the potential performance impact from distributional bias in the generated samples? How sensitive is the framework to the quality of the generator?**\n\n**A1:** For the question you asked, it should refer to some noises when using generated samples, which might result into potential performance impact although generative replay is flexible. For this, we mitigate this risk from two below aspects: 1) the use of a high-quality pretrained diffusion model (DDPM) [1], which achieves strong fidelity and diversity (e.g., FID 3.17, IS 9.46 on CIFAR), and 2) our LEBA focuses on semantic-level alignment using adapter responses to high-level features, rather than relying on low-level pixel fidelity. This design choice enhances robustness and ensures that minor generation artifacts do not adversely impact the alignment or learning. To evaluate the sensitivity of our LEBA to the quality of the generator, we conducted experiments using different versions of the diffusion model, as shown in below table. This indicates that our LEBA exhibits low sensitivity to the choice of generator.\n\nPerformance with different diffusion generator\n| Generator | Transfer | Average | Last  | Preserve |\n|-|-|-|-|-|\n| SD1.4| 69.2     | 77.9    | 87.3  | 84.5     |\n| SD1.5     | 69.03    | 77.73   | 87.48 | 84.31    |\n| SD2.1     | 68.94    | 78.12   | 87.51 | 84.74    |\n\n[1] Denoising Diffusion Probabilistic Models NIPS 2020.\n\n**Q2: If historical adapters are suboptimal, the PKE module mitigates this risk by adaptively weighting and integrating multiple prior adapters, reducing reliance on any single flawed representation and enhancing overall robustness.**\n\n**A2:** It is indeed common in incremental learning that historical adapters may be suboptimal, as no approach can fully preserve all prior knowledge perfectly over time. To address this, our framework incorporates the PKE module, which adaptively integrates $K$ historical adapters for each replay sample using learnable weights. This mechanism enables the model to prioritize more robust and relevant representations, rather than relying on any single adapter.\nIn Table 3 of the manuscript, incorporating the PKE module which includes the use of these generated replay samples—results in a good performance gain across all evaluation metrics, demonstrating its effectiveness in mitigating the impact of imperfect historical adapters through selective and weighted knowledge integration.\n\n**Q3: While KL-divergence looks a reasonable choice, could the authors justify that this single metric is sufficient to capture the multi-faceted concept of an adapter's \"usefulness\"? Have alternative, more sophisticated selection mechanisms (e.g., based on diversity) been considered?**\n\n**A3:** We chose KL-divergence as our primary metric because it effectively captures the task-specific similarity between adapter outputs, aligning well with our goal of preserving semantically relevant knowledge. \nTo further validate this choice, we also experimented with alternative metrics such as cosine distance and mutual information for adapter selection. As shown in the below table, these methods yield comparable performance across all four evaluation metrics, suggesting that KL-divergence is sufficiently effective in capturing the relevant aspects of an adapter’s “usefulness” in our setting. \n\nComparison of adapter selection strategies on four evaluation metrics\n|Method|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|Cosine Similarity|68.94|77.61|86.84|84.86|\n|Mutual Information|69.37|77.73|87.17|84.37|\n|LEBA (Ours)|69.20|77.90|87.30|84.50|\n\n**Q4: The fixed-size adapter pool  effectively ensures stability as the number of total tasks grows. However, how does this stability mechanism interact with the replay batch size? Have the authors tested the framework's robustness with larger replay batches, and does a larger number of simultaneous alignment targets introduce new optimization challenges? A key hyperparameter, the number of replayed samples per-training step, is not explicitly mentioned, which hinders exact reproducibility.**\n\n**A4:** \nFirstly, we confirm that the fixed-size adapter pool provides stability as tasks accumulate, and we have evaluated the framework under varying replay batch sizes.\nTo evaluate the interaction between this mechanism and replay batch size, we conducted experiments under different replay settings: 500$\\times$16, 500$\\times$32, and 500$\\times$64 (i.e., generating 16, 32, or 64 replay samples per iteration over 500 iterations for each task). The results are shown in below table. As shown, performance gains saturate beyond the 500$\\times$32 setting, indicating that LEBA is robust to variations in replay size. To balance performance and computational efficiency, we adopt the 500$\\times$32 configuration as the default throughout our experiments.\n\nSecondly, in our LEBA, increasing the number of simultaneous alignment targets does not introduce notable optimization challenges, as the alignment operates at the semantic feature level with lightweight adapter outputs rather than raw high-dimensional data. As shown in below table, performance remains stable even when the number of replay samples increases, suggesting that the optimization process is not adversely affected.\n\nPerformance on varying numbers of replayed samples\n|Replay number|Transfer|Average|Last|Preserve|\n|-|-|-|-|-|\n|500$\\times$16|68.78|77.46|86.27|83.16|\n|500$\\times$32 (Ours)|69.20|77.90|87.30|84.50|\n|500$\\times$64|69.43|78.23|87.51|84.67|\n\n**Q5: The authors state that the code will be released upon acceptance. Could they confirm if the release will include the necessary  scripts to faithfully reproduce the main results?**\n\n**A5:** We commit to releasing the corresponding code and script resources that can reproduce the main results if the manuscript is accepted.\n\n**Q6: The framework relies on samples from a pre-trained diffusion model. These models are known to have their own biases and do not perfectly replicate the true data distribution, which could potentially mislead the alignment process.**\n\n**A6:** We would like to clarify that the replay samples used in the PKE module are generated by a pretrained diffusion model (DDPM) [1], which has demonstrated strong generative quality. For example, on the CIFAR dataset, DDPM achieves FID 3.17 and IS 9.46, indicating high fidelity and discriminability relative to the original data distribution.\nWhile generative models may introduce certain biases, our LEBA relies more on the semantic alignment between adapter representations than on pixel-level realism. The consistent performance gains observed in Table 3 of the manuscript suggest that the generated samples are sufficiently informative for stable knowledge alignment and replay. \nAs part of future work, we plan to explore post-processing strategies, or employ more advanced generative models to further enhance the fidelity and diversity of replay data.\n\n[1] Denoising Diffusion Probabilistic Models NIPS 2020.\n\n**Q7: The framework lacks proactive guidance to generate samples optimal for mitigating forgetting.**\n\n**A7:** We have carefully reviewed the LEBA framework and concur that, as you noted, this aspect presents an opportunity for further enhancement. We appreciate your insightful suggestion and regard it as a direction for future improvements."}]

Author rebuttal:

Meta-review: This paper works on the multi-domain task incremental learning problem. In the paper, the authors propose a continuous-domain bridge adaptation module, leveraging the distribution transfer capabilities of the Schrödinger bridge for stable progressive learning. It was reviewed by four expert reviewers, and all of them recommended accepting this paper after the rebuttal period. According to the reviewers, the paper is very well-written and the presentation is excellent. It also proposes a new way of replaying experiences using diffusion models. Therefore, it is a clear acceptance. The authors are encouraged to include the additional results and discussions during the rebuttal period in the final version.
Decision: Accept (poster)