Null-Space Filtering for Data-Free Continual Model Merging: Preserving Stability, Promoting Plasticity
Authors: Zihuan Qiu
Lei Wang
Yang Cao
Runtong ZHANG
Bing Su
Yi Xu
Fanman Meng
Linfeng Xu
Qingbo Wu
Hongliang Li
Venue: ICLR 2026 Poster (2026)
Keywords: Continual Model Merging
Model Merging
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: NUFILT is a data-free continual model merging framework using null-space filtering and projection-aware adaptation, achieving transparency, fidelity, and 4–7% accuracy gains over baselines with minimal forgetting and lower overhead.

Abstract: Data-free continual model merging (DFCMM) aims to fuse independently fine-tuned models into a single backbone that evolves with incoming tasks without accessing task data.
This paper revisits two fundamental desiderata for DFCMM: stability, avoiding interference with earlier tasks, and plasticity, adapting faithfully to each new task. This poses a challenge that existing approaches fail to address: how to bridge data-level desiderata with parameter-space optimization to ensure stability and plasticity in the absence of task data.
To this end, we propose NUFILT(NUll-space FILTering) a data-free framework that directly links these desiderata into parameter-space optimization. Our key observation is that task vectors approximately align with representation subspaces, providing structural surrogates for enforcing stability and plasticity. 
Accordingly, we design a null-space projector that preserves prior responses by filtering overlapping components of new task vectors, ensuring stability.
We further introduce a lightweight LoRA adapter that injects complementary task-specific signals to enable plasticity.
The adapter is trained with a projection-based surrogate loss that preserves consistency with prior knowledge while introducing novel directions. 
This joint filtering–adaptation process enables the backbone to absorb new knowledge while retaining existing behaviors, with updates fused back in a layer-wise linear fashion without extra parameters or inference cost.
Theoretically, we establish approximate subspace alignment guarantees that justify null-space filtering. Empirically, NUFILT achieves state-of-the-art performance with minimal forgetting on both vision and NLP benchmarks, improving average accuracy by 4–7% over OPCM and WUDI-Merging, while narrowing the gap to fine-tuning and reducing computation overhead. The code is available at: https://github.com/zihuanqiu/NUFILT

Published as a conference paper at ICLR 2026
NULL-SPACE FILTERING FOR DATA-FREE CONTINUAL MODEL MERGING: PRESERVING STABILITY, PROMOT- ING PLASTICITY
Zihuan Qiul.,, Lei Wang', Yang Cao², Runtong Zhang', Bing Su', Yi Xu',
Fanman Meng' Linfeng Xu'. Qingbo Wu', Hongliang Li' 'University of Electronic Science and Technology of China, Chengdu, China DDlian University of Technology, Dalian, China 'Jiggan Technology
ABSTRACT
Data-free continual model merging (DFCMM) aims to fuse independently fine- tuned models into a single backbone that evolves with incoming tasks without accessing task data. This paper revisits two fundamental desiderata for DFCMM: stability, avoiding interference with earlier tasks, and plasticity, adapting faithfully to each new task. This poses a challenge that existing approaches fail to address: how to bridge deta-level desiderata with parameter-space optimization to ensure stability and plasticity in the absence of task data. To this end, we propose NUFILT (NUlI-space FILTering), a data-free framework that directly links these desiderata into warameter-spasee optimization. Our key observation is that task vectors approx- imately align with representation subspaces, providing structural surrogates for enforcing stability and plasticity. Accordingly, we design a null-space projector that preserves prior responses by filtering overlapping components of new task vectors, ensuring stability. We further introduce lightweight LoRA adapter that injects complementary task-specific signals to enable plasticity. The adapter is trained with a projection-based surrogate loss that preserves consistency with prior knowl- edge while introducing novel directions. This joint filtering filterin--ddaptatio adaptation process enables the backbone to absorb new knowledge while retaining existing behaviors, with updates fused back in a layer-wise linear fashion without extra parameters or inference cost. Theoretically, we establish approximate subspace alignment guarantees that justifyy null-space filtering. Empirically, NUFILT achieves oft the-art performance with minimal forgetting on both vision and NLP benchmarks, improving average accuracy by 4-7% over OPCM and WUDI-Merging, while narrowing the gap to fine-tuning and reducing computation overhead. The code is available at: https: /ht..b...uugggggggg wtt.:/cw.....aaeeae http://uuunnnuuUUUII0
INTRODUCTION
Modern machine learning systems are often deployed in dynamic environments where tasks arrive one after another. Training a new model from scratch for each task is computationally expensive and requires retaining all historical data, which is often impractical. Maintaining a separate checkpoint for every task also becomes infeasible due to memory and deployment constraints (McMahan et al. 2017 Fang et al. 2024 Qiu et al., 2024). A more appealing alternative is to reuse existing models either drawn from a model library or trained for new tasks and consolidate their knowledge into single
backbone that evolves over time (Zhou et al., 2024b Yu et al.) 2024b) Huang et al.) 2024a] Tang al.) 2025). However, this consolidation is expected to be performed while retaining only the merged model for storage efficiency, and without accessing any data in order to safeguard privacy, which makes the problem particularly challenging. These constraints highlight the urgent need for approaches that integrate knowledge across sequential tasks directly in parameter space without extra storage or data costs.
"Corresponding author

---

Published as a conference paper at ICLR 2026
Recent studies investigate data-free continual model merging (DFCMM) (Liu & Soatto 2023 Porrello ett al. 2025 Tang et al.] 2025), where tasks arrive sequentially and, at each step, only the newly fine-tuned task model and the previously merged backbone are available (see Fig. D. We revisit two fundamental desiderata for DFCMM: stability, avoiding interference with earlier tasks, and plasticity, adapting faithfully to each task. While these desiderata are inherently defined the data level, existing approaches lack principled translation into parameter-space objectives, As result, simple averaging or arithmetic updates often cause interference that undermines stability ([zmailov et al. 20188 Ilarrco et al. 2023); projection methods fail to preserve plasticity when task vectors are correlated (Tang et al. 2025] Yadav et al.) 2023); and adaptive strategies typically rely on auxiliary data, which violates the data-free constraint (ang et al.) Tang et al. 2024b Qiu et al.)
2025). Achieving both stability and plasticity under the strict requirements of DFCMM remains an open challenge.
Individual Fine- Tine-Tuning
Continual Model Merging
Desiderata
In this paper, we introduce NUFILT (NUII- space FILTering). novel framework for DFCMM that directly bridges data-level desiderata with parameter-sye space optimization. The key observation is that task vectors exhibit approximate alignment with representation sub- spaces, enabling structural surrogates for en- forcing stability and plasticity without data. To achieve stability, a null-space projector filters out components of the new task vector over- lapping with earlier subspaces, suppressing in- terfering activations. To achieve plasticity, a lightweight LoRA adapter performs projection- aware adaptation, optimized with data-free surrogate loss: updates are constrained main consistent with previous tasks while intro- ducing complementary signals along directions unique to the new task. Finally, the projector, task vector, and adapter are fused back into the backbone in a layer-wise linear fashion, so up- dates are absorbed without extra parameters or inference cost. Theoretically, we provide guarantees of approximate subspace alignment, establishing a rigorous foundation for null-space filtering. Em- pirically, we demonstrate that NUFILT achieves state-of-the-art art accuracy with minimal forgetting across both vision and NLP tasks. Compared to recent methods such as OPCM (Tang et al. 2025) and WUDI-Merging (Cheng et al.] 2025), NUFILT improves average accuracy by 4-7%, while substantially narrowing the gap to individual fine-tuning and reducing computation overhead.
STABILITY:
Eurnbeer
avoiding interforence
with carlier tasks:
On
O merged
PLASTICITY:
adapting faititully each bark.
%
6
ierggged
How bridge date-level with porameter- space optinization?
Orly weight access
Cars
6 merreed
Data-free
Figure Illustration of data-free continual model merging (DFCMM). At each step, only the current task model and the previously merged model are accessible, and the merging process is performed without access to any data. The merged model is ex- pected preserve prior knowledge (stability) while adapting efficiently to new tasks (plasticity).
To summarize, our main contributions are as follows:
We establish that task vectors exhibit approximate alignment with data representation subspaces, revealing geometric property that explains how task vectors interact with representations.
We propose NUFILT, data-free continual model merging framework that combines null-space filtering with projection-aware adaptation to enforce both stability and plasticity.
Extensive experiments on vision and NLP benchmarks demonstrate that NUFILT achieves state- of-the-art performance, surpassing prior methods in accuracy while better resisting forgetting.
RELATED WORK
Model Merging. Model merging provides an efficient alternative to multi-task or continual training by combining multiple fine-tuned models directly in parameter space, without revisiting their training data. Early efforts adopted simple weight averaging (Utans) 1996 Shoemake 1985). which can connect models through linear mode connectivity (Entezari et al. 2021 Ainsworth et al. 2022 but often suffers from severe performance degradation due to parameter interference. Task Arithmetic (TA) (Iharco et al. 2023) formalizes merging through task vectors, making merging operations more explicit. However, since standard fine-tunin tne-ttning rarely guarantees disentangled updates (Ortiz-Jimenez
et al. 2023). TA can amplify interference, prompting structured finetuning strategies (Ortiz-limenez
2

---

Published as a conference paper at ICLR 2026
et al. 2023 Liu & Soatto 2023 Porrello et al. 2025 To mitigate conflicts, several methods introduce additional structure. Sparsity-based approaches such as TiES-Merging (Yadav et al. 2023) prune redundant conflicting parameters, while reweighting schemes (Yu et al. 2024b) balance task contributions. Projection- baseed methods enforce orthogonality between task vectors to separate task directions (Tang al.] 2025 Cheng al. 2025), but struggled when tasks inherently correlated. Adaptive strategies further adjust merging process with auxiliary calibration data (Yang et al.) [Tan et al., 2025; Qiu al., 2025). but this departs from the strictly data-free setting. Overall, these methods reduce interference to varying degrees but either rely on data or assume strong independence between tasks, leaving the core challenge of simultaneously achieving stability and plasticity in the data-free continual setting largely unresolved.
Continual Learning. Continual learning tackles catastrophic forgetting (McCloskey & Cohen 1989), where models lose performance on earlier tasks when adapting to new ones. Solutions include
constraining parameter updates via importance weights (Kirkpatrick et al.] 2016, et al. 201
Aljundi al. 2018), preserving knowledge through distillation (Hou et al. 2019:; Douillard et al.
2020b, or replaying exemplars and surrogates (Rebuffi et al. 2016 Liu et al. 2021]. Other strategies
expand capacity with dynamic architectures (Lee et al., 2017 Qiu et al. al.]| |023] Zhou et al. 2024a) or adopt pwwameter-effiieeeeeee eeffcittteeeffiinttt modules such as adapters and prompts (Yu et al. [2024a] Huang et al.
2024b). Beyond traditional data-driven approaches, complementary direction explores continual learning through model merging. Early efforts merged fine-tuned checkpoints to alleviate forgetting
(Mirzadeh al. 2021] Wen al. 2023 Marczak al. (022)), but these typically relied on training
data or sequential fine-tuning, More recently, continual model merging (Jin et al. [0033] Liu &
Soatto 2023 Porrello al., 2025 Tang et al., 2025 Qiu al., 2025) has emerged, aiming to
fuse independently fine-tuned models directly in parameter space, avoiding both data access and checkpoint storage while improving scalability and privacy.
Orthogonality in Task Fusion. Orthogonality-biie -based methods aim to reduce interference between tasks by projecting updates into orthogonal spaces. These can be categorized into two types: orthogo-
nality between data and parameters (Farajtabar et al., 2220; Wang et al. [222] Liang & Li] 2024), and
orthogonality directly between parameter updates Wei et al.) Chenn et al. [025 Tang et al. (025).
Fang et al. 2025) project perturbations onto the null space of preserved knowledge, while Xiong et al. (2024) enforce orthogonality between task vectors, and Wang et al. (2023) learn new parameters within orthogonal low-rank subspaces to reduce interference. In contrast, our method projects each new task vector into the null space of previously merged task representations, but without requiring any task data. This differs from Fang al. (2025); ): Wang al. a (2023), which rely external data to estimate orthogonality constraints. Moreover, unlike Xiong et a (2024), which requires simultaneous access all task vectors, our approach operates sequentially using only the previously merged task vector and the current one, enabling efficient continual model merging.
BACKGROUND AND MOTIVATION
We formalize data- free continual model merging in Sec. 3.1
review representative approaches in Sec. 3.2 outline key desiderata for data-free merging Sec. 3.3 and analyze the geometric relation
between task vectors and representations in Sec. 3.4
3.1 PROBLEM SETTING
We study data-free continual model merging, where a sequence of task-specific models (0, are independently fine-tuned from a shared pre-trained model Og on labeled datasets D, with disjoint label sets C.. The goal is to obtain single merged model gmerrged that generalizes the union label space C1.1 = - 1=1 C.. Unlike conventional continual learning, we assume no access to raw training data. All knowledge integration must therefore occur directly in parameter space, via recursive merging of task-adapted checkpoints:
gmerged Merge (pmerrgeed 01)
(@tnerged 0
(1)
To expose the underlying structure, let T, = 6, On denote the task vector (lharco et al.) (002)). capturing the update induced by task f. Rather than merging checkpoints directly, reinterpret
3

---

Published as a conference paper at ICLR 2026
continual merging as learning transformed update:
O 0, + +h, (f= - (T)).
Here, F transforms task vectors to stay compatible with previously merged parameters, ensuring seamless integration of new updates.
3.2 REPRESENTATIVE CONTINUAL MODEL MERGING SOLUTIONS
Several continual model merging approaches can be viewed as special cases of task vector transfor- mation, each with different assumptions F(T)). Below we outline representative solutions:
Weight Averaging (WA) (Izmailov et al. 2018): Updates parameters by simple averaging, @merged (t- 1)0 + 0. This stabilizes optimization but assumes task compatibility and equal importance of checkpoints, making sensitive to semantic conflicts. @ Task Arithmetic (TA) (Ilharco et al., 2023p: Adds scaled task vectors, @ 0 Tt. where À tunable coefficrent. It can outperform naive averaging but lacks structural constraints, leading to scale sensi- tivity and task interference. @ Orthogonal Projection- based Continual Merging (OPCM) (Tang ct al. 2025): Projects each task vector onto the orthogonal complement of previous directions, @.merged
[ 17 - + (T.]] where p((=1)(-) removes overlapping subspaces. This enforces geometrie separation but struggles when task vectors are inherently entangled or non-orthogonal. AdaMerging (Yang et al.): Adapts coefficients using a small unlabeled test set, morrgod Omerrged + X (Di Tę- By exploiting test-time signals, it improves task alignment and mitigates scaling issues, but requires auxiliary data, violating the data-free assumption.
8o [Te-
3.3 DESIDERATA FOR DATA-FREE CONTINUAL MERGING
In continual model merging, updates for a new task should be incorporated without disrupting knowledge accumulated from earlier ones. This reflects the classical stability-plasticii balance in continual learning. To describe how transformed update - = F(T,") influences the merged model, we use two layer-wise quantities as surrogates for these effects. Let 88² &dd. be the (8) pretrained weights at layer 1, T the task vector for task i, and (1) E Rd: the corresponding layer activations.
@ Stability surrogate. This term measures how much the new update alters the model's responses on carlier tasks. For activations ((()) ~ Dist-1, we define:
By [e((0 + 7!") )o(0) gmerged. ,.)))
where l is a distance metric (e.g., squared E2).
@ Plasticity surrogate. This term evaluates how well the merged parameters follow the behavior of the task's individual model. For activations (l()) Dr, we define:
2 plas (F(()) EEwAD [ + =(())= (z()) (0 + )a
These two terms anchor the cubilitt-plasttonn trade-off within the parameter space used for merging. However, optimizing c(4) (7(()) and c(0) (=(()) not possible in the data-free setting, as neither Dist-1 nor D, accessible. This leads the central challenge:
How can continual merging achieve stability and plasticity without data?
3.4 APPROXIMATE ALIGNMENT BETWEEN REPRESENTATION AND TASK VECTOR SUBSPACES
We examine the geometric relation between task vectors and data representations. We hypothesize that task vectors tend to align with principal directions in the representation space that capture task-relevant information. To validate this, conduct subspace alignment analysis and provide a theoretical guarantee that task vector subspaces are approximately aligned with data subspaces, which in turn motivates null-space filtering approach.
4

---

Published as a conference paper at ICLR 2026
Mean acosa lagers
52th parcartle scrces liyess
E0OP 30r0n Lryers
Tex
Bacapaoo Altiidy
(a) Mean across layers
(b) 90th percentile across layers
(c) ECDF across layers
Figure 2: Subspace affinity between data and task vectors in ViT-B/16 across eight datasets. Heatmaps show diagonal dominance, and layer-wise empirical cumulative distribution functions (ECDF's confirm higher affinities for matched pairs.
For each task and layer I, let Vi E IRdixTa be the singular vectors of the representation covariance (xx)) J.. For the task vector 7(1), we compute its truncated SVD, 7(1) x (0)0), where
10(0 E Rdxxr is the top ro right singular vectors. We define the subspace affinity:
A(V vo)
IV Vall
d,)
(5)
rd
which lies in [0, 1] and measures the degree of alignment between the data and vector subspace.
Empirically, across ViT-B/16 models fine-tuned eight datasets, affinity heatmaps (Fig. 2 show strong diagonal dominance: matched data-vector pairs exhibit much higher affinities than mismatched ones. Layer-wise ECDFs (Fig. 2cb further reveal spectrum of overlaps, from high MNNIST-MNIST moderate (EuroSAT AT-RSSSS-SSSS to low (SVHN-DTD) These results indicate that task vectors are consistently aligned with tele-relevant representation directions, a trend observed in both vision and NLP tasks (additional results for more models provided in the Appendix C.2)
Theorem 1 (Approximate Subspace Alignment). Let 7()) Rdd, be the task vector at layer I, and H E R representation matrix of rank va with right singular vectors Va E Rdixra, Suppose T!)) admits the decomposition -!1) = To + E with To = BH, rank(To) = Td, where each row Ex of E satisfies ||E; l2 S k. Let or (To) denote the ra-th largest singular value of the matrix To.
ora (Tu) > Vd max; ke then for any Tu 2 Td. the span of the top vv right singular vectors V of 7!0) is approximately aligned with span(Va) in the sense that
A(V(V)) ((0) S 2
(6)
where A(V f0)= 1 ra
DIVTVVall and < dr. (To)- dd. max; W
Voh Vs
This theorem shows that (() consists of low-rank component aligned with the representation space plus a bounded perturbation, so its principal subspace is approximately aligned with the data subspace. Together with the empirical evidence, this provides both geometric intuition and theoretical justification for treating task vectors as carriers of task-relevant representation directions.
4 METHOD: NUFILT
Motivated by the above, we propose NUFILT (NUlI-space FILTering), a fate-free method that enforces structural constraints on The key idea is to leverage the geometry of prior tasks to suppress activations tied to earlier ones, enabling continual merging while mitigating forgetting.
4.1 STEPS IN NUFILT: FILTERING, ADAPTING & FUSING
The procedure of NUFILT unfolds in three steps, as illustrated in Fig.
Step O: Filtering. To prevent interference with prior tasks, we introduce null-space filter before applying each new task vector. For layer 1, we compute the cumulative update
(1) St-1 0,-1 (0)
(erged,
(7)
5

---

Published as a conference paper at ICLR 2026
Stepl: Null-space filtering
Step2: Data-free Adapting
Step3: Layer-wise Fusing
bato-free objective
L(A, - 1|5- -MM (,.,
opergn grorge tyPP,
Marged thet Victer
Null-pase
Task Mattor
Pagector
goornn dy
Pi + B,A,
Projector
A
Task Vester T,
Fusing
murred
Trainubble
Phoza
Figure 3: Overview of the NUFILT procedure. Filtering: the new task vector is processed through a sull-space projector that suppresses activations from previous tasks, ensuring stability to past knowledge. @ Adapting: within the filter, a lightweight LoRA adapter refines the update for the current task using data-free objective. O Fusing: the filter, task vector, and LoRA module merged back into the backbone, keeping the parameter count and inference cost unchanged.
1- and obtain its top-ry right singular vectors
S1-1- The null-space filter is defined as p())
v-(0)
Sr-1 S 1t-11
(T)
(8)
which removes components aligned with earlier task directions. Thus, for old task features an(1) span( (V-() 5(-1), we have p())(() = 0, ensuring stability to prior knowledge.
Step @: Adapting. Within the filtered subspace, we insert a low-rank LoRA adapter
p(O)
P B A!)
where 100h E Rrxxd, and B(") E Rd..x. This lightweight term restores task-specife flexibility and is trained with a data-free objective (Sec. [.2). allowing the merged model to recover the performance of the fine-tuned model on task t without requiring raw data.
Step O: Fusing. Finally, the filter, task vector, and LoRA module are merged into the backbone:
0 merged. (1) (1)) + T(") (P!") + A()
(10)
Since all operations linear, the composite update can be absorbed into the weight matrix, leaving the parameter count and inference cost identical to an individual model.
4.2 DATA-FREE OBJECTIVE OF NUFILT
From Theorem we obtain a data-free upper bound on parameter-representatinn interactions.
Corollary (Data-free upper bound). Let X E IR Nx,, with largest singular value a (X), rank ra. and right singular vectors Va. Let T E Rdoxd, with top-r,, right singular vectors V (r,, 2 ra). If Theorem
ensures 1 1 Vall?- I <², then for any p RRd
||(e-T)x < (x)( (11(-->VV12= + tra 11011--0 1112)
(11)
Subspace surrogate for losses. Under squared lz loss, the data losses (Fiq. are
Lstab = -BxxD,ccillfst xx Spmms = Ex Tt) X 11:
(12)
where tzc = -i= 7. Let X, and X,, denote feature matrices from old and new tasks. Applying Corollary with (p. r) (fsm i and (Fge, Tt). and denoting the corresponding misalignments by So and (n. we obtain the following data-free upper bounds:
[[(Fse - Fsc 1X < 2x1 [I(Fse Fz- S. 182 + +ro.3ll(fss - 111)
(13)
[[(Fg - xf (Xn) (2x))))(((( (I((fs Vo) + ralafllfs -noll3).
(14)

---

Published as a conference paper at ICLR 2026
Algorithm 1 NUFILT Procedure
Input: pre-trained model 0o: fine-tuned models (0:) rank parameters (ry, mi, rv): learning rate vy and maximum number iterations MaxIter.
Initialize: 0f 0,
for each task E .2....,22 } do for selected linear layer index E (1,......) L} do
p1d = 1 (.(0 V T A (0, mit)) B +
& null-space via (0 TO St-1r 5t-1
St-1
initialize LoRA components with low rank r
for iter to MaxIter
compute M = Sc-1
=())
+ T 11) p(") T. and via Eq. 17
(( B T
M + x()) 1 A! V
data-free objective (Eq. 16
A A - V
A11 L,
_ith B - -1 V B(t
update LoRA components
1==1 (0
+ + ( (p!0) + B A }
layer-wise fusion
Output: 0T geprrgodd
Thus when the misalignment Ç is small, both losses are governed by the terms ((fge-fst-a)) V and [|(5q: - Vll offering a projection-awee aware objective for effective data-free surrogates.
Projection-aware objective. At layer 1, the merged task vector (subtracting 80 from Eq. 10 is
( - + 7 (P!) + B(D A
(15)
Substituting Eq.
15 into the projection terms yields the compact objective:
[(A B(()
T (M rt B A V11
(16)
with
+ ( 2- T
(--()) Pill
V
M M
(1) 7t-1 + +Te
(1) p
(17)
Here the residual Tt
(1) B A bridges MV and T, ensuring stability to past tasks and plasticity the new one. Since Ti is generally non-square and low-rank, closed-form for (A B,!)) ) are unstable;
we thus optimize Eq. 16 via gradient descent. The full procedure is summarized in Algo. [
EXPERIMENTS
5.1 EXPERIMENTAL. SETUPS
Benchmarks and Protocols. We evaluate our approach on both vision, NLP tasks, and multimodal
tasks to assess scalability and generality. For vision, following (Ilharco al., [023] Wang al.). we adopt CL.IP-VI backbones (Radford et al. (021) and construct three groups of 8, 14, and 20 image classification tasks. We use publicly available ViT-B/32, ViT-B/16, and ViT-L/14 checkpoints, each fine-tuned up 20 datasets (Tang et al. 2024a). To ensure robustness to task order, all experiments are repeated over 10 random permutations (seeds 42-51). For NLP, we evaluate on eight GLUE tasks (Wang al. 2019) using Flan-T5-bare (Chung et al. 2024). For multimodal tasks, evaluate on 1.3 B (Liu et al. 2023). a widely used vision language model for instruction following. We adopt four multimodal generative benchmarks: IconQA (Lu et al. (02I). ImageNet captioning (Deng et al. 2009), ScienceQA (Lu et al. 2022), and VizWiz-Caption (Gurari et al. 2018). covering diagram reasoning, natural images, scientific QA, and low-vision real-world scenarios.
Implementation Details We insert the null-space filter in a cascaded fashion before selected linear layers of the backbone. All experiments share a single set of global hyper-parameters--aee both CLIP and Flan models, as well as all task orders and scales-withour task-specific tuning: null-space rank rp 128, LoRA rank ri = 64, and task projection rank r. Each task is adapted for 50 iterations with Adam at learning rate of 10-3
across

---

Published as a conference paper at ICLR 2026
Metrics and Baselines. We evaluate performance with two standard metrics: average accuracy (ACC) and backward transfer (BWT) (Lin et al. 2022). ACC is the mean accuracy of the final merged
model across all tasks, ACC = T Ei=1 a,(07 where ax(-) denotes accuracy on task { BWT
quantifies the effect of merging on past tasks by comparing their performance before and after the final merge: BWT 7 T-1 a. (O
(0, In addition to the methods in Sec. 3.2
we compare against TIFS- MAISSMMERNNGG (Yadav et al. 2023), MAGMAX-IND (Marczak et al. 2024p.
WEMOF (Tang et al. 2024b). and WUDI-MERGING (Cheng al. 2025). Detailed descriptions
provided in the Appendix B.3
5.2 MAIN RESULTS
Table Comparative results of continual merging methods, reporting average accuracy and backward transfer over ten task orders tstd. EP and DA denote method assumptions: the need for extra parameters or data access. Best results are in bold, and the second best are underlined.
Method
Requirement
VIT-B/32
VIF-B/16
VIT-L/14
EP/DA
tasks
tasks
20 tasks
tasks
14 tasks
20 tasks
8 tasks
14 tasks
20 takks
PRE-TRAINED
48.1
56.9
55.6
55.4
62.0
59.8
64.9
69.1
65.6
INDIVIDUAL
90.4
89.3
89.8
92.4
91.3
93.4
93.5
C. FINE-TUNED
/
79.8
67.4
62.6
82,9
72.2
68.2
90.0
70,9
77.7
WEIGHT AVERAGING
XI X
66.3 saa
65.4 so0
61.1 a30
72.3 sea
69.7 cn0
64.8 cm
80.0 cn0
77.5 :m0
71.1 100
TASK ARITHMETIC
X1X
66.5 sm
60.0 230
77.1 =3a
70,9 cn
64.2 :m0
82.1 =00
77.9 g/o
70.3 gm
TIES MERGING
X1 X
49.0 4102
66.2 sos
59.9 #27
66.8 #17
70.5 aas
63.0 :/4
64.3 670
78.0 cou
68.3 100
MAGMAX-IND
X1X
70.7 241
67.0 gm
76.7 2
67.0
62.5 :m0
81.4
71.2 g/o
71.2 gm
LW ADAMURGING
X1 /
53.4
59.8
59.7
59.9 223
64.3
61.52
68
73.1
66.9
MORA-WEMOR
/1 V
68.8
43
49.6 iss
47
67.9 3:
75
=4005
56.9
WUDI-MERGNG
X1
74.7
63.7 67.0
81
75.0
69.6
87.5
84.2
1
Is0-C
X1X
21.7
23.2 =1
67.6 23
2
29.7
23.0
86.9 =a5
KNOTS-TIES
X/
24
67.8 211
60.5
$7.9
71.6
62.7
68.3
78.8
09.7
TSV-M
X
68.2
58.8 231
69.2
63.1 2/
82.2
78.1 s/
70.5 g/
OPCM
X/X
71.9 201
65.7 232
81.8
77.1
70.3
87.0 204
83,5 ant
76.0 202
NUFILT (Ours)
X
83.6 282
78.0 sx2
71.0 #29
87.3 :21
83.1 203
78.1 :xp
91.6 :XI
89,2 cm
84.7 :03
WEIGHT AVERAGING
X/
-1..5
-8.0
a21
-9.7
-7.1 214
-7.3
-7.3 434
-5.8 :M
-6.4 =15
TASK ARITHMETIC
XIX
-133 4:s
+3.4
.4.2 110
-1.3 e84
-3.6 s84
-7.1
-1.8 ca
-3.3 va3
TIES MERGING
XI
-15.3
1,9 2G:
-15
55.5
1.4
.1.5
-13.0
11=a
MAGMAX-IND
XIX
.3
.3.4
.7.2
-6.1 813
-7.
-8.
a5.
-6.0
-6.5 m2:
LW ADAMERGING
X1
.3255
214.1
-22.7
-27.8 ga1
-22.1
-214
-24.3
1196
21.7
W MERIING
VIV
-0.44 180
-20.2 43x
-24.5 4:00
-18.0 162
-18.8 434
-25.8 am
-17.8 128
-16.8 153
22.9 mm2
XLX
117.0
-22.8
-26.0.4
-12.699
-169
-18.5
.7.3
.9.-m
-15.8
Iso-C
X/X
-1022
-104
-10.3
.6.7
-7.1
-9.
-3.
-3.7
-5.520
KNOTS-TIES TIES
XIX
-12,6499
1,9
+L.30
-13.5
1.00a1
.233
-11.6
0.3
-2.3
TSV-M
X/X
224.0
24
-31.7
-18.5
-22.7
-28.2
-13.0
-15622
-23332
OPCM
XIX
-6.3
.60
-78
448
2.1
-63322
26
43
-65.a
NUFILT (Qurs)
X1X
-2.7 2m
-5.7
-8.2
-1.6
-3.
-7.1
-1.10 201
-00
4,2
Results on Vision Tasks. The comparative results on vision tasks are summarized in Tab. 1 highlighting that our method, NUFILT, consistently outperforms prior merging techniques across various model architectures and task sequences, all without needing extra parameters or data access. Relative baselines like OPCM and WUDI-Merging, NUFILT achieves marked improvements in average accuracy while reducing backward transfer. For example, on ViT-B/32 with tasks, reaches 83.6% accuracy, outperforming OPCM by 8.1% and WUDI-Merging by 8.9%; for 20 tasks, the gains are 5.3% over OPCM. On ViT-L/144 it surpasses OPCM by 4.6% on 8 tasks and 8.7% on 20 tasks, with backward transfer limited to -1.1% versus -2.6% for OPCM on 8 tasks. Overall, NUFILT closes the gap to the individual fine-tuning benchmark, lagging by only 2.7% on ViT-L/14 for 8 tasks, underscoring its robustness in continual merging settings.
Results on NLP Tasks. The results for NLP tasks using the Flan-T5-base model on 8 tasks are detailed in Tab. 2 showing that NUFILT excels all merging strategies. Against comparable baselines such as OPCM and WUDi-Merging, NUFILT delivers notable gains in overall performance. It achieves an average accuracy of 83.7%, exceeding -eri-Me by 1.5% and OPCM by 3.1 1%, while limiting backward transfer to -1 -1.5% compared to -3.9% for WUDI-Merging and -2.5% for OPCM. NUFILT narrows the divide with individual fine-tuning, trailing by merely 2.7%, affirming its efficacy in continual merging for language models.
For STSB (NIP)), report Spearman's p. denoted by a.(-) for notational consistency.
8

---

Published as a conference paper at ICLR 2026
Results Multimodal tasks. The results for multimodal tasks using LLaVA-1.5-7B on four
benchmarks are summarized in Tab. Compared with strong baselines, NUFILT attains clear
improvements in average performance. It reaches overall score of 70.5%, surpassing OPCM by 2.9%, underscoring its effectiveness in continually merging multimodal models.
Table 2: Results of continual merging Flan-T5 T5-base models on 8 tasks, ordered alphabetically.
Method
EP/DA
CoL.A
MNLI
MRPC
QNLI
QQP
RTE
SST2
STSB
ACC
BWT1
PRE-TRAINED
69.1
56.5
76.2
88.4
82.1
80.1
91.2
62.2
75.7
INDIVIDUAL
83.4
87.5
91.5
85.4
93.6
88.7
86.4
TASK ARITHMETIC
X1
69.1
58.1
77.9
88.9
83.1
79.1
90.7
74.0
77.6
-4.6
WIES-MERGING
88.8
81.8
89.7
76.8
75.6
LW ADAMERGING
X1 /
69.1
58.1
77.9
88.9
83.1
79.1
90.7
74.2
77.6
-4.7
80.6
90.3
82.7
91.3
DIDI-MERGING
X1
71.2
73.4
79.2
89.7
82.9
79.1
93.1
88.2
W2.2
-3.9
72.9
78.7
90.3
83.8
73.7
NUFILT (Ours)
X1X
72.5
83.3
78.7
91.1
84.2
79.4
93.4
87.1
83.7
-1.5
Table 3: Results of continual merging LLaVA models on 4 tasks, ordered alphabetically.
TconQA
Image
SciQA
Average
INDIVIDUAL
75.5
96.0
83.7
64.8
80.0
PRE- TRAINED
14.1
40.9
61.7
41.0
39.4
47.6
TIES MERGING
69.6
74.4
75.5
44,6
66.0
NUFILT (Ours)
59.6
93.6
80.6
48.3
70.5
5.3 ABLATION AND ANALYSIS RESULTS
Ablation Each Component. We conduct an ablation study disentangle the contributions of the null-space filter and the data-free objective. Results are summarized in Tab, 4 (1) Simply accumulating task vectors without any constraint leads severe performance degradation. Both accuracy and backward transfer drop sharply the number of tasks increases, confirming that direct parameter addition suffers from catastrophic interference. (2) Projecting task vectors onto the null-space of previous tasks significantly stabilizes merging. Compared to naive merging, this strategy improves accuracy by more than 20 points across all task counts and reduces forgetting to near zero. (3) Using the data-free objective alone (without null-space filtering) recovers part of the performance but still suffers from negative backward transfer, especially when scaling to more tasks. Combining both components yields the best balance between stability and plasticity.
Table 4: Ablation study of NUFILT with CLIP ViT-B/32 over 8, 14, and 20 tasks.
Component
ACC(S)
BWT(%)
Method
Null Null-sgace/ LoRA
20
tasks
tasks
20 tasks
Naive Merging ginerges
(rrerend 7%)
X1
62.1
46.5asa
34.3
-18,5
-25.8ta2
-24.7
(2) Only Null-space (Bltauel guiceel r.P))
/1 x
80.0su
76.7a1e
67.0am
-1.7ass
-42a09
-6.2a1s
Data-free Objective
mull-space filter (dptersed gparred BBAD
75.8
63.7at
51.7am0
-10.2ass
-17,1277
-20 6axe
full method (8 (gueass guarnsd
(PP BeAc))
VTV
83.6am
78.0ata
71.0aw
-2.7
-5.7
-8.9
Hyper-paramed Sensitivity. We analyze the impact rank hyper-parameters in Fig. (a) Higher null-space rank rp initially reduces forgetting and boosts performance, but excessively large ranks reduce projection selectivity, impairing new task adaptation and lowering accuracy. (b) Low-rank adaptation enhances adaptability while controlling forgetting. Increasing rank rı improves accuracy steadily, with backward transfer remaining stable. (c) Projecting updates onto 12 improves new task adaptability. Moderate ranks yield consistent gains, while overly large T. reduces projection specificity, approaching full-space projection and diminishing benefits.
Evaluation of stability and plasticity. Following Eqs.
3 and compute c()) and c(l) layer-wise
and report their averages all layers. g(() measures activation changes earlier tasks, and c(()

---

Published as a conference paper at ICLR 2026
86-
A00 (N)
73
BWT (%0
ACC 1967
15
14 (96)
76-
72
20-
-15
0
B 18 Rank
32
64
100 200
Pank
Banke
(a) Vary Pp. To
(b) Vary Fi, Pp 128, 8
(c) Vary Fe. 128, 16
Figure 4: Hyper-pane parameter sensitivity analysis on the 8-task continual merging protocol. Setting r 0 corresponds to removing the associated component.
measures deviation from the individual model of the current task t. Since both are losses, lower values indicate better stability and plasticity. As shown Fig. 5 NUFILT achieves much lower stability loss, indicating minimal disturbance to earlier tasks, and consistently lower plasticity loss, showing closer alignment with the individual solution. The harmonic mean further confirms that NUFILT achieves the best balance between the two criteria.
200
500
200
Stabiity loss
150
Ploticty Loss
100
100
300
Tast Number
Tank Number
Task Number
Figure 5: Evaluation of stability and plasticity surrogates, and their harmonic mean on the CLIP ViT-B/32 8-task continual merging protocol. Lower values in all metrics indicate better performance.
Computation Overhead. Tab. compares
Table 5: Overhead of VIT-B/32 under 8-task continual merging: per-task iterations, solving time, total time, peak GPU memory, and final parameter count.
the computation overhead of three categories of methods. Training-free free approaches (TA,
OPCM) are the most efficient since they re- quire no optimization or data loading, but their accuracy limited. Test-time methods (LW AdaMerging, LORA-WEMOE) exhibit the largest overall runtime because both data loading and multi-step training add substan- tial latency. Compared with WUDI-Merging, NUFILT achieves lower solving time be- cause its LoRA-based updates operate on fewer parameters, though its total time is slightly higher owing to the additional SVD computation. Overall, NUFILT provides the best accuracy while maintaining competitive efficiency.
Model
Iters. Solv.
Tot.
GPU Mem.
Param.
ACC(%)
TA
87.5M
67.5
OPCM
76.6s
1.3GB
87,5 67.5M
75.5 am
LW ADAMURGING
47.1s
L66BB
a3.
LORA- WEMOR
50
72.9c
754.66
L88GB
103.7M
68.8
MERGING
50
28.9s
37.7s
2.0GB
87.5M
74.7
25
9.3s
129.8s
L.SGB
87.5M
83.3
NUFILT
100
35.9:
156.4s
L.SGB
87.5M
83.6
50
18.4s
138.9
L.8BB
87.5M
83.6
6 CONCLUSION
In this work, addressed the problem of data-free continual model merging, where independently fine-tuned models must be consolidated sequentially into a single backbone without access to task data or earlier checkpoints. The central challenge lies in enforcing stability and plasticity. To this end, we established that task vectors approximately align with representation subspaces, providing geometric foundation for continual merging. Building on this insight, we proposed NUFILT, novel framework that integrates null-space filtering to suppress interference with prior knowledge and projection-awe adaptation to recover task specifie plasticity via data-free surrogate objectives. Experiments across vision and NLP benchmarks demonstrated that NUFILT achieves oftte--f-the--tt results, improving over recent methods by 4-7% on average while substantially reducing forgetting and narrowing the gap to individual fine-tuning. We believe this work provides principled and practical step toward scalable, theoretically grounded solutions for continual model merging.
10

---

Published as a conference paper at ICLR 2026
Acknowledgments This work was supported in part by National Science and Technology Major Project (2021ZD01 12001). National Natural Science Foundation of China (No. NN0.6221119, 08 120002. U23A20286, 6207 (207,086), the Key Research and Development Project of Hainan Province (Grant No. ADYF2024(LALH)003 Liaoning Province Grant XLYC2403174, the Natural Science Foundation of Sichuan Province under Grant 20252NSFSC0475.
REFERENCES
Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models modulo permutation symmetries. arXiv preprint arXiv:2209.048: 04486,,,, 2022.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Mareus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pp. 139-154, 154, 2018.
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-10l-mining discriminative compo- nents with random forests. In European Conference on Computer Vision, pp. 446-461. Springer,
2014.
Gong Cheng, Junwei Han, and Xianqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865-138)) 2017.
Runxi Cheng, Feng Xiong. Yongxian Wei, Wanyun Zhu, and Chun Yuan. Whoever started the interference should end it: Guiding data-free model merging via task vectors. In Forty-second International Conference on Machine Learning, 2025.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetunee language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
Mircea Cimpoi, Subhransu Maji, lasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3606-3613, 2014.
Tarin Clanuwat, Mare Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature. In NeurIPS Workshop on Machine Learning for Creativity and Design, 2018
Adam Coates, Andrew Y Ng, and Honglak Lee. An analysis of single-layer networks in unsuper- vised feature learning. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 215-223. JMLR Workshop and Conference Proceedings, 2011.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN), pp.
2921-2926. IEEE, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 2448255, Icee, 2009.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. European Conference on Computer Vision, 2020.
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. The role of permutation invariance in linear mode connectivity of neural networks. arXiv preprint arXiv:2110.06290 2021.
Congyu Fang, Adam Dziedzic, Lin Zhang, Laura Oliva, Amol Verma, Fahad Razak, Nicolas Papernot, and Bo Wang. Decentralised, collaborative, and privacy-preserving machine learning for multi- hospital data. EBioMedicine, 101, 2024.
Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat- Seng Chua. Alphaedit: Null-space constrained model editing for language models. In The Thirteenth International Conference Learning Representations, 2025.

---

Published as a conference paper at ICLR 2026
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International conference on artificial intelligence and statistics, pp. 3762-3773.
PMLR. 2020.
lan J Goodfellow, Dumitru Erhan, Pierre-Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, William Cukierski, Yichuan Tang, David Thaler, Dong- Hyyn Lee, et al. Challenges in representa- tion learning: A report on three machine learning contests. In International Conference on Neural Information Processing, pp. 117-124. Springer, 2013.
Danna Gurari, Qing Li, Abigale Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3608-3617, 2018.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2226, 2019.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 831-839, 2019.
Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang. Chao Du, and Min Lin. Lorahub: Effi- cient cross-task generalization via dynamic loRA composition. In First Conference on Language Modeling, 2024a.
Linlan Huang. Xusheng Cao, Haori Lu, and Xialei Liu. Class-incremental learning with clip: Adaptive representation adjustment and parameter fusion. European Conference on Computer Vision, pp. 214-231. Springer, Spriner,,, 2024b.
Gabriel Ilhareo, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, 2023.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. In 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018, pp. 876 885. Association For Uncertainty in Artificial Intelligence (AUAI), 2018
Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietre and Pengxiang Cheng. Dataless knowledge fusion by merging weights of language models. In The Eleventh International Conference on Learning Representations, 2023.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, An- Demis
drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Hassabis, Claudia Clopath, Dharshan Kumaran. and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114:3521 - 3526, 2016.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- inn-ngrinedd categorization. In 2013 IEEE International Conference on Computer Vision Workshops, pp. 554-561. IEEF, 2013.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, 86(1) 86(11):2278-2322 1998.
Sang-Wor Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang, Overcom- ing catastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
12

---

Published as a conference paper at ICLR 2026
Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference- low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 33638-23644 2024.
Sen Lin. Li Yang, Deliang Fan, and Junshan Zhang. Beyond not-forgetting Continual learning with backward knowledge transfer. Advances Neural Information Processing Systems, 35: 16165-16177, 2022.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lec. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
Tian Yu Liu and Stefano Soatto. Tangent model composition for ensembling and continual fine-tuning, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18676-18686, 2023.
Yaoyao Liu, Bernt Schiele, and Qianru Sun. Rmm: Reinforced memory management for class- incremental learning, Advances in Neural Information Processing Systems, 34:3478-3490, 2021.
Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: new benchmark for abstract diagram understanding and visual language reasoning. arXiv preprint arXiv:21 10.2210..12244 13214, 2021.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521, 2022.
Daniel Marczak, Bartlomiej Twardowski, Tomasz Trzcinski, and Sehastian Cygert. Magmax: Lever- aging model merging for seamless continual learning. In European Conference on Computer Vision (ECCV), 2024.
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficienn learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. PMLR, 2017.
Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. Linear mode connectivity multitask and continual learning. In 9th International Conference on Learning Representations, 2021.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning, 2011.
Maria-Flena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp. 722-729. IEEE, 2008
Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Systems, 36:66727-66754, 2023.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C V Jawahar. Cats and dogs. 2012 IEEE Conference on Computer Vision and Patlern Recognition, pp. 3498-3505, 2012.
Angelo Porrello, Lorenzo Bonicelli, Pietro Buzzega, Monica Millunzi, Simone Calderara, and Rita Cucchiara. A second-ords order perspective on model compositionality and incremental learning. In The Thirteenth International Conference Learning Representations, 2025.
13

---

Published as a conference paper at ICLR 2026
Zihuan Qiu, Linfeng Xu, Zhichuan Wang, Qingbo Wu, Fanman Meng, and Hongliang Li. Ism-net: Mining incremental semantics for class incremental learning. Neurocomputingg 523:130-143, 2023.
Zihuan Qiu, Yi Xu, Fanman Meng, Hongliang Li, Linfeng Xu, and Qingbo Wu. Dual-consistency model inversion for non-exemplar class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24025-24035, 2024.
Zihuan Qiu, Yi Xu, Chiyuan He, Fanman Meng, Linfeng Xu, Qingbo Wu, and Hongliang Li. MINGLE: Mixture of null-space gated low-rank experts for test-time continual model merging. In The Thirty-ninth Annual Conference Neural Information Processing Systems, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PmLR, 2021.
Sylvestre-Alvise
Rebuffi, Alexander Kolesnikov, G. Sperl, and Christoph H. Lampert. icarl: Incre- mental classifier and representation learning. 2017 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 5533-5542, 2016.
Ken Shoemake. Animating rotation with quaternion eurves. In Proceedings of the 12th annual conference on Computer graphics and interactive techniques, pp. 245-254, 1985.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-1642, 2013.
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. Man computer: Bench- marking machine learning algorithms for traffic sign recognition. Neural Networks, 32:323-332, 2012.
Anke Tang, Li Shen, Yong Luo, Han Hu, Bo Du, and Dacheng Tao. FusionBench: A Comprehensive Benchmark of Deep Model Fusion, June 2024a.
Anke Tang, Li Shen, Yong Luo, Nan Yin, Lefei Zhang, and Dacheng Tao. Merging multi-task models via weight-ensembling mixture of experts. In International Conference Machine Learning, pp. 47778-47799. PMLR, 2024b.
Anke Tang, Enneng Yang, Li Shen, Yong Luo, Han Hu, Lefei Zhang, Bo Du, and Dacheng Tao. Merging on the fly without retraining: A sequential approach to scalable continual model merging. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Joachim Utans. Weight averaging for neural networks and local resampling schemes. Proc. AAAI-96 Workshop on Integrating Multiple Learned Models. AAAI Press, pp. 133-138. Citeseer, 1996.
Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco S Cohen, and Max Welling. Rotation equivariant cnns for digital pathology. arXiv preprint 1806v:1806.000 03962, 2018.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.0777 1180..0046.. 2018
Alex Wang. Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding, In 7th International Conference on Learning Representations, ICLR 2019, 2019.
Ke Wang, Nikolaos Dimitriadis, Guillermo Ortiz-Jimenez François Fleuret, and Pascal Frossard. Lo- calizing task information for improved model merging and compression. Forty-first International Conference Machine Learning.
14

---

Published as a conference paper at ICLR 2026
Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 184-193, June 2021.
Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan-Jing Huang. Orthogonal subspace learning for language model continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10658-10671, 2023.
Per-Ake Wedin. Perturbation bounds connection with singular value decomposition. B/T Numerical Mathematics, 12(1):99-111, 1972.
Yongxian Wei, Anke Tang, Li Shen, Zixuan Hu, Chun Yuan, and Xiaochun Cao. Modeling multi-task model merging as adaptive projective gradient descent. In Forty-second International Conference Machine Learning.
Haitao Wen, Haoyang Cheng, Heqian Qiu, Lanxiao Wang, Lili Pan, and Hongliang Li. Optimizing mode connectivity for class incremental learning. In International Conference on Machine Learning, pp. 36940-36957. PMLR, 2023.
Hermann Weyl. Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgle- ichungen (mit einer anwendung auf die theorie der hohlraumstrahlunng) Mathematische Annalen, 71(4):441-479 1912.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint
2017.
Jianxiong Xiao, James Hays, Krista Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE Conference on Computer Vision and Puttern Recognition, pp. 3485-3492. IEFF, 2010.
Feng Xiong, Runxi Cheng, Wang Chen, Zhanqiu Zhang, Yiwen Guo, Chun Yuan, and Ruifeng Xu. Multi-task model merging via adaptive weight disentanglement. arXiv preprint arXiv:2411.18779, 2024.
Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36:7093-7115, 2023.
Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. Adamerging: Adaptive model merging for multi-task learning. In The Twelfth International Conference on Learning Representations.
Jiazuo Yu, Yunzhi Zhuge, Lu Zhang. Ping Hu, Dong Wang. Huchuan Lu, and You He. Boosting continual learning of vision-language models via mixture-of- adapters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23219-23230, 2024a.
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning, 2024b.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. International conference on machine learning, pp. 3987-3995, 2017.
Da- Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23554-23564, 2024a.
Yuyan Zhou, Liang Song. Bingning Wang, and Weipeng Chen. Metagpt: Merging large language models using model exclusive task arithmetic. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 1711-1724, 2024b.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
The appendix is organized into sections, each providing supplementary explanations and supporting details.
Proof of Theoretical Results
I6
A.1
Proof of Theorem
A.2
Proof of Corollary
18
Additional Descriptions
B.1
Details of Dataset and Task Settings
19
B.2
Details Downstream Models
B.3
Details of Baselines
B.4
Hyper-pane paperr-prammttr Adjustment Guidelines
C
Additional Results
C.I
Detailed Overall Performance Results Extended Visualizations on Subspace Alignment
C.2
D
Discussions
23
D.I
Limitations
D.2
Broader Impacts
23
D.3
LLM Usage
23
A PROOF OF THEORETICAL RESULTS
A.1 PROOF OF THROREM
Theorem extends Proposition A.I Cheng et al. 2025) from individual task vectors to subspaces. While Proposition Proposition/AAJJdmmmooo demoonsstsss that each task vector can be approximated as linear combination of representation features, Theorem further establishes approximate alignment between the singular subspaces of task vectors and task representations.
Theorem A.1 (Weyl inequality for singular values (Weyl 1912)). For matrices A E Rmxx and F. E mmxn with singular values a, (A) and o [+
(A+ F ordered decreasingly (01 2 02 2
... 2 0), and Ell2 the operator norm of E, the following holds for j = 1, ..... ..mmiinmmm.... (, n):
loy(A+ o,(Aal ||
(18)
Theorem A.2 (Wedin's Sin-Thete Theorem (Wedin) 1972 ). Let M E Ruxx with SVD M
UEV U (To U.J.V VVo V1]. where Uo maxr and Vo gaax correspond
the top-r singular values. Let M M + H with SVD M ŪΣVT 1 = [0o Ú1),
(V V.l- Suppose there exist IR and A such that o, (M) a, o (M) A
Then the canonical angles between the singular subspaces satisfy
max dist Un), dist (Vo. vo)} <
max{]] max{]]HHolla |I Toll2) <
|
(19)
A
A
where dist (Ùo, Uo) = sin OŪū Uo)ll2 denotes the largest principal angle between sub- spaces.
16

---

Published as a conference paper at ICLR 2026
Proposition A.1 (Approximate Linear Combination CChng et al.
2025 ). Let 7()) denote the (1)
task vector of neuron k in linear layer I. Consider N input samples (In J and let nn,t denote the corresponding input to layer after t iteration for sample 2n during fine-tuning. Assume that the model before layer I is Ct- Ci-iipschttz continuous, and the gradient of loss is bounded in nn-norm by Gr, and the gradient of the loss with respect to the product 000 1 Innt
(1)
bounded by 1 k.- Then, the following inequality holds:
N
Tk
(1)
R.TT 1T
S t*
ΣΣ
Mo
(20)
==1
i=!
where 1 = N. Ct 9
r and B Et=
TI a(o!!) 6,1-1 19
(L,(0+)
Theorem A.3 (Approximate Subspace Alignment). Let r() E RRd,xdd be the task vector at layer I, and H E R xd. representation matrix of rank a with right singular vectors Va E IRdixTa, Suppose 7!0) admits the decomposition 7(1) To + E with To- = BH, rank(To) = ra where each row E& of E satisfies || 2 I k. Let (To))) denote the ra-th largest singular value of the matrix To- If or (To) > vdo max; V,, then for any vv 2 Tá. the span of the top To right singular vectors V of (1) is approximately ali aligned with span(Va) in the sense that
1- A-V(V)) V(1) S ?
(21)
where A(V A
((0)= ra
Vall? VVll? and (
Vd, max V2 or (26) max; Vt
Proof. The proof of Theorem I requires three ingredients. First, Proposition A. shows that task vectors can be approximated by linear combinations representation features, yielding decomposi- tion into a structured term plus bounded error. Second, Weyl's inequality (Theorem A.1 provides a perturbation bound for singular values, which ensures a nontrivial spectral gap under bounded error. Finally, Wedin's ' sin -in-Theta theorem (Theorem A..2) translates this spectral gap into guarantee of approximate alignment between the true and perturbed subspaces.
(!) of the task vector matrix From Proposition A.1 Apprrximmte Linear Combination), each row Tk satisfies
N Σ ==1
k.n (
n.T T12e S Lk
(22)
where Uu b (
E 7877 is per-neuron error bound, and 4; depends on the number of samples, Lipschitz constant, gradient bounds, and learning rates. This yields the decomposition (()) To + E, with To BH, B Pk and |E&112 < V for each row Eg. so || <
Vd, max; Mg.
(1) Exact case (E= = 0). In this case, (0) = To BH. Since H has rank the right singular space of is exactly span Va). Under the rank assumption rank(To) = Ta (which holds when d, 2 ra and B has sufficient rank, motivated by the NTK regime), To and H share the same dimminnnson right singular subspace. Hence span((...) svan(Va), so for any r. rd. A(Va. V) = 1.
(2) Perturbed case (E * 0). Apply Wedin's 's sin θ theorem A.2 Let Vo be the top ra right singular
vectors of To. and set a or, or,(To), (To). Define A
or, (To)
||]| The stability assumption
ora (Ta) Vdl max; VA 2 || ensures A 0. With rank(To) ra and (() To + F,
Weyl's inequality A.l gives |o,,+ (T()) o (To) I || Since orz+ (Tp) 0. this yields
Trati (7) < E 2
17

---

Published as a conference paper at ICLR 2026
By Wedin's 's theorem, the canonical angles & between (pan(V6) and span(Vr.)) satisfy
||
si (V6. 12 -
(23)
A
We have (V)) = VVan(Va) and |F S vdo max v. hence
Vdl, max; V
sin (V2,, Vra)ll2 S
(24)
Ora (To) - do max; 1
The subspace affinity then obeys
A(Va, Vr.)= = rd ==1
cos 2 fj 2 - -sin² 0 maxx.
(25)
Thus, we obtain
do max: Up
) 2 = )==²²
1 - A(Va, Vra) S sin? Omax
(Tr) vdo max; V
(26)
Finally, since enlarging the subspace only increases the overlap,
A(Va, V) 2 A(Va, Vra).
(27)
Hence, for any Tu 2 rd.
1 1- A(V,, V) $ 1 - A(Va, (ra) S 5
(28)
A.2 PROOF OF COROLLARY
Corollary A.1 (Data-free upper bound). Let X E RR Nxd: with largest singular value 0n 01 (X), rank rd and right singular vectors V4. Let Rd with top-r, right singular vectors V (re 2 ra). If Theorem 1 ensures 1 1 Vall? s 5², then for any p Rd.xxd
1|(e - 7) s 2x)(x) (I((- + ra 11p 11-1-10 -11
(29)
Proof. Let P P= VV be the projection onto span Decompose
T PX + (1 PIx
(30)
By |la + b11 s 221|a| + 2/161/2
- < - T)p 2 -
(31)
(1) Projection term
xP ( 13: = ( x).
trace (Ppcc(P(p-7)T66 x)PXT
Let B P(p-7)T(pp -)P 2 0 and C = X 2 0. Since C N ||C|| I, follows that
trace (BC
||||| tracc BB. Since ||C||2 || X1|2 xx( and
P = 11(p VI trace(B) B) = trace (p(p T) P) = -1(p -)
(32)
we obtain
((0-7)PX' 1 x of(x) lI 1)
(33)
18

---

Published as a conference paper at ICLR 2026
(2) Residual term. By ||AB|] | - ||
11(p (1 P)x 113 I 11l 11p-- (1x - P\1
(34)
Using the SVD X = V
[ - P)[ -race (1- -1))).
(35)
Since m3 (
|X - -) S
(36)
Noting that
traco (I- = rd 1 VI s ra5?.
(37)
we conclude
1X(1- - P))1 S r
(38)
Hence
(1- 1b of(()) 11
(39)
(3) Combine. Summing the two parts yields
1(0-+)xT11 | s 207 -1p--)-~VV0000 V1 + -20 ras? 1
(40)
as claimed.
B ADDITIONAL DESCRIPTIONS
B.1 DETAILS OF DATASET AND TASK SETTINGS
Overview of Vision Tasks To comprehensively examine continual model merging, we curated a
collection of 20 diverse image classification benchmarks, spanning natural objects, remote sensing. medical imagery, and text-rendered datasets. This selection largely follows prior practice in multi- domain evaluation (Tang et al., 2025). while ensuring balanced inclusion of datasets with varying granularity (from binary recognition to hundreds of categories). Specifically, the benchmarks include:
SUN397 (Xiao et al., 2010), Stanford Cars (Krause et al., 2213)), RESISC45 (Cheng et al.
2017).
EuroSAT (Helber al. ) SVHN (Netzer al., 2011). GTSRB (Stalkkamp et al. 2012).
MNIST (LeCun et al. 1998). DTD (Cimpoi et al. 2014)). Flowers 102 (Nilsback & Zisserman
2008 J.
PCAM (Veeling et al. 2018), FER2013 (Goodfellow al. 2013), Oxford-IIT Pet (Parkhi et al.
2012). STL-110 (Coates et al. 2011). CIFAR- and CIFAR-10 (Krizhevsky & Hinton] 2009).
Food-101 (Bossard et al. 2014), MaSHIOn-MNIST (iaoo et al. (2017). EMNIST (Cohen et al.
2017
KMNIST (Clanuwat et al. 22188) and Rendered SST-2 (Socher et al. 2013).
Overview of NLP Tasks In addition vision benchmarks, we also consider widely-used natural language understanding datasets. Specifically, we evaluate on the GLUE benchmark (Wang et al. 2018), which covers tasks ranging from single-sentence acceptability judgments to pairwise entail- ment and semantic similarity. For classification-orinneed datasets, report exact match accuracy, including CoLA (linguistic acceptability), MNLI (natural language inference), MRPC (paraphrase detection), QNLI (question gnusee entailment), QQP (duplicate question detection), RTE (recogniz- ing textual entailment), and SST-2 (sentiment classification). For STS-B, which measures semantic textual similarity, performance is reported using Spearman's p correlation coefficient.
Task Grouping We evaluate continual merging under three progressively larger task sets. For each setting, models are assessed using average accuracy (ACC) and backward transfer (BWT). To ensure robustness, we generate 10 random task orders for every group (Tab. 6). reporting the mean and standard deviation across runs.
8-Task Group: (1) SUN397, (2) Stanford Cars, (3) RESISC45, (4) EuroSAT, (5) SVHN, (6) GTSRB, (7) MNIST, (8) DTD.
14-Task Group: Extends the 8-task set with (9) Flowers102, (10) PCAM, (11) FER2013, (12)
Oxford-IIIT Pet, (13) STL.-10, (14) 100.
19

---

Published as a conference paper at ICLR 2026
20-Task Group: Builds upon the 14-task set by adding (15) CIFAR-10, (16) Food-101, (17) Fashion-MNIST, (18) EMNIST, (19) KMNIST, (20) Rendered SST-2.
For the NLP benchmarks, tasks are organized in alphabetical order: CoLA, MNLI, MRPC, QNLI, QQP, RTE, SST-2, and STS-B.
Table 6: Dataset orderings used for vision experiments in each task group.
Order Dataset Order (by ID)
(04 05 -0-000007777
03 106 +00
(07 07-0005 007-00--05--04-11 000000000-00----0000000000000000 0144--2206666----0000 -000--3-- -10))
(03 00==000-:04+000
-5
4 006-08--00-001 -00-000-0---000 -4 05)
00
02)
007--02--0--0----53
06)
00T=00---4------0--------- 06)
008-05--000007------------1--
001-044--5--00--00------------
10
(08 000-001-011-02-------------11000
(05
10
B.2 DETAILS DOWNSTREAM MODELS
In this section, we describe the evaluation protocol for both pre-trained and fine-tuned models across vision and natural language processing (NLP) downstream tasks.
report the zero-shot performance of pre-trained CLIP-VIT models as well as the test accuracy of task specifc fine-tuned models. Fine-tuned checkpoints are obtained from Hugging Face (https: httph/ugggnnggfagg respective dataset using a standard recipe. During fine-tuning, the visual encoder is updated while the text encoder remains fixed. Training follows consistent setup: cross-entropy loss, Adam optimizer, cosine annealing schedule, learning rate of 1 x 10-5, batch size of 128, and 4000 training steps.
For vision tasks (Tab. 7)
co/ ctoooaaaaoo tanganke), where each model is trained its
For NLP tasks, adopt the 8-task GLUE benchmark using Plan-T5-ore base. As summarized in Tab. 8 we compare pre-trained and fine-tuned Flan-T5- b5-b models across all GLUE tasks. Fine-tuning is conducted with learning rate 4 10-5 batch size 16, and 2000 training steps for each task.
20

---

Published as a conference paper at ICLR 2026
Table Performance of the CLIP pre-trained model and individually fine-tuned models on different vision downstream tasks.
SUN397
Cars
RESISC45
EaroSAT
SVHN
GTSRB
MNIST
DTD
Flowers
PCAM
Model
Fine-tumed Pre-trained
CLIP-VTT-B32
63.2
59.6
60.3
45.0
31.6
32.5
48.3
44.2
66.4
60.6
CLIP-VIT-B/6
65.5
64.7
66.4
54.1
52.0
43.5
51.7
45.0
71.3
54.0
CLIP-VIT-L/4
77.9
58.4
50.5
55.5
$1.2
CIIP-VIT-BA2
74.9
78.5
95.1
99.1
97.3
98.9
79.7
88.6
88.0
-LP--VTT-/16
78.9
85.9
96.6
99.0
97.6
99.0
99.7
82.3
94.9
90.6
CLIP-VIT-L/4 82.8
92.8
97.4
99.1
97.9
99.2
99.8
85.5
97.7
91.1
Oxfordii7PPa
FishioMNISS
RendoredSST2
FER2013
STL10
CIFARIO0
CIPARIO
Food101
EMNIST
KMNIST
Model
Pre-trained
CLIP-VYT-B/32
41.3
83.3
97.1
63.7
89.8
82.4
63.0
12.0
10.0
58.6
CLIP-VTT-B/16
46.4
88.4
98.3
66.3
90.8
87.0
67.3
12.4
11.2
60.6
CI-P-VIT-L/1
50.0
93.2
99.4
75.1
95.6
91.2
67.0
12.3
9.7
68.9
Fine-tanee Lan-ttneed
CLIP-VIT-B/22
71.6
92.5
97.5
88.4
97.6
88.4
94.7
95.6
98.2
71.3
CLIB-VYT-B/16
72.8
94.5
98.2
88.8
98.3
91.9
94.5
95.3
98.1
75.7
CLIP-VYT-L/14
75.9
95.7
99.2
93.0
99.1
94.8
95.3
95.4
98.3
80.5
Table 8: Performance of pre-trained and fine-tuned Flan-T5-bass models on the 8-task NLP GLUE benchmark.
CoLA
MNLI
MRFC
QNLI
QQP
RTE
SST2
STSB
Model
Flan-T5 T5-base (re-trained)
69.1
56.5
76.2
88.4
82.1
80.1
91.2
62.2
Flan-T5 Tann-T5-bace (Fine-tuned)
75.0
83.4
87.5
91.5
85,4
85.9
93.6
88.7
B.3 DETAILS OF BASELINES
In addition the methods presented in Sec. 3.2
introduce the following additional baselines.
Ties-Merging An extension of Task Arithmetic that alleviates parameter redundancy and sign conflicts during model merging (Yaday et al. 2023). For task 4, we first compute the task-specific difference vector 7, O2 0g. which is then trimmed and sign-normalized. The update is defined as T. Tie Tics -- Te), and the merged model is updated by gimergod @.eergdd + Ar, Ties
Maximum Magnitude Selection (MAGMAX). An extension of Task Arithmetic that selects, for each parameter dimension, the update with the larger absolute value (Marezak et al. 2024). Formally, 7 MMaMMa MagMax(7; MagMMa Tx), and the merged model is updated as omrerged
MagMax
gmerged + Xr: MagMax our experiments, we apply MAGMAX to merge individually fine-tuned models, denoted as MAGMAX-IND.
Weight-Ensembliag MoE (WEMOE). A mixture-of-experrs based merging strategy (Tang et al. 2024b). where task-specific MIP layers serve as experts and are aggregated via gating function. In the continual merging setting, however, WEMOE does not converge: the expert MLPs introduce excessive parameters, which makes learning the gating function unreliable with only a few unlabeled test samples. To mitigate this issue, we compress the MLP experts using LoRA, yielding the variant LoR LORA-WEMOR WEMOE. For test-time adaptation, we fine-tune the gating module using 5 randomly sampled instances per class from the test set each new task.
WUDI-Merging. A data-free merging method that reduces task interference by enforcing orthogonality between task vectors and their residual components (Cheng al. 2025 In the
21

---

Published as a conference paper at ICLR 2026
continual setting, each linear layer maintains two task vectors: the cumulative merged vector merged..() (leered,(() 0 and the current task vector 77 8 0
from previous tasks 7 0
(1) The merged vector Tm obtained by optimizing C E. m (Tm.. (T,,) I1 The
update follows gradient descent, and the final merged parameters are @.merged = 0g + Tm-
Details of Baseline Hlyper-parameters As summarized in Tab. we report the key hyperparameter settings for all baseline methods and task configurations. Notably, our approach employs a single fixed configuration applied uniformly across models (CLIP, Flan-T5), task scales (8, 14, 20), and all 10 task orders. This design highlights the robustness and generality of our method, ensuring that performance improvements do not stem from speciffe hyperparameter tuning.
Table 9: Hyperparameter settings for all baselines across different task configurations.
Method
Tasks
Scale Factor (A)
Top-k (%)
LR
Steps
Tp
8
0.3
TASK ARITHMETIC
14/20
0.1
8
0.3
20
TTES-MERGING
14/20
0.1
20
MAGMAX-IND
8/14/20
0.5
L.W. ADAMERGING
8/14/20
0.3
le-4
50
LORA- -EEMOE
8/14/20
0.3
le.4
50
64
WUDI-MERGING
8/14/20
le-5
50
OPCM
8/14/20
NUFILT (Ours)
8/14/20
le.3 50
128
64
8
B.4
HYPER-PARAMETER ADJUSTMENT GUIDELINES
This section provides brief guide for selecting the hyper-paramenn used in NUFILT.
Rank syper-parameters (Tp, To) Our sensitivity analysis shows that all three rank hyper- parameters exhibit unimodal behavior and can be tuned independently with a small validation set:
rp: Controls how much of the previous-task subspace is preserved. Larger values reduce forgetting; moderate values work consistently across backbones.
Low-rank correction term. Set smaller than Tp to avoid disrupting the preserved subspace; small ranks are stable.
re: Dimensionality of the task vector subspace. small value ensures stable alignment; increasing it introduces weaker singular directions and typically does not improve performance.
Number of Iterations. Performance improves initially but saturates quickly. A single tuned value (e.g.,50-100 iterations) generalizes across different ViT backbones.
Scaling Across Architectures. Sensitivity patterns are consistent for ViT-B/32, ViT-B/16, and ViT-L/14. The same hyperparameter settings work well across model sizes, with only minimal fine-tuning needed when desired.
C ADDITIONAL RESULTS
In this section, provide additional experimental results support the findings reported in the main paper. Specifically, we include: (1) detailed overall performance results CC.I: (2) extended
visualizations subspace alignment C22
22

---

Published as a conference paper at ICLR 2026
C.1 DETAILED OVERALL PERFORMANCE RESULTS
Tab. T0 expands on the average results in Tab. by reporting per-task average accuracy after continually merging 20 tasks. We compare six methods, WA, Task Arithmetic, Ties-Merging, MagMax-IND, OPCM, and our proposed NUFILT across three CLIP-ViT backbones (B/32, B/16, L/14). NUFILT achieves the highest accuracy on most tasks. These fine-grained results reinforce the main paper's 's findings, highlighting NUFILT's ability to improve performance on continual model merging.
C.2 EXTENDED VISUALIZZATIONS ON SUBSPACE ALIGNMENT
This section provides extended subspace aligmment visualizations for three additional models (ViT- B/32, ViT-L/14, Flan-T5-base; see Fig.
and to further validate the universal alignment of task vectors with task-relevant representations across architectures and layers. All visualizations follow unified 3x3 grid structure where columns represent model components (left: full model; middle: attention layers; right: MLP layers) and rows show visualization types (top: layer-wise mean affinity heatmaps; middle: layer-wise 90th percentile affinity heatmaps; bottom: layer-wise ECDF curves). The layer-wise ECDFs reveal consistent spectral overlaps: for VYT models (Fig. 0 and high (MNIST Data MNIIST Vector), moderate (EuroSAT Data - RESSSSC4 Vector), and low (SVHN Data DDTD Vector); for Flan-T5- (Fig. 8), high (RTE Data RTT Vector), moderate (RTE Data -QNLI Vector), and low (RTE Data SST-2 Vector). These results across both vision and NLP domains confirm the consistent alignment phenomenon originally observed Fig. 2 with all affimity heatmaps showing strong diagonal dominance where matched pairs exhibit significantly higher affinity than mismatched ones.
D DISCUSSIONS
D.1 LIMITATIONS
Similar prior approaches model merging, our NUFILT framework is built the assumption that all task-specific fine-tuned models [0.] originate from common pre-trained initialization 0o. The implications of this assumption-suuch subspace remaain insufficiently explored and merit further investigation. Our current experiments are restricted models sharing the same backbone (e.g., CLIP ViT variants, "lan-T5-base extending the framework to heterogeneous initializations or architectures represents an interesting avenue for future work.
its effect on the alignment of task vectors in the underlying
D.2 BROADER IMPACTS
This paper aims advance the Machine Learning field. Our work has potential societal impacts, but none require specific highlighting here.
D.3
LLM USAGE
In preparing this submission, we used large language models (LL.Ms) solely as an assistive tool for sentence-level editing, including grammar correction, spelling adjustments, and minor word-choice refinements. The LLM was not involved in research ideation, methodological design, experimental analysis, content generation beyond language editing. All substantive scientific contributions are solely those of the authors.
23

---

Published as conference paper at ICLR 2026
Table 10: Test set accuracy comparisons on different downstream tasks.
SUN397
RESISC45
EuroSAT
SVHN
GTSRB
MNIST
DTD
Flowers102
PCAM
Model
FINE-TUNED
45.4
WA
64.2
59.6
64.8
60.9
47.3
71.8
46.4
66.5
VIT-B332
C.TA
62.0
53.7
60.9
58.1
48.5
48.9
79.4
46.1
61.1
73.4
C.TIES
62.5
49.1
55.8
50.9
54.6
49.3
82.0
46.7
58.5
69.9
MAGMAX-IND
63.6
53.1
59.7
49.1
53.8
53.1
79.8
43.2
56.9
75.1
LW ADAMERGING
63.1
60.0
63.5
60.1
35.6
32,1
51.8
45.4
66.6
60.2
PORA-WEMON
51.4
45.8
63.3
43.5
42.9
34.6
46.5
47.5
WUDI-MERGING
OPCM
64.4
51.1
66.0
71.7
66.1
56.0
90.2
40.4
64.9
80.2
NUFILT (Ours)
60.3
53.6
67.0
76.6
87.4
85.6
988.1
51.8
55.5
82.4
C. PINE-TUNED
62.7
58.0
67.6
99.1
46.0
29.2
93.9
61.9
64.1
75,2
WA
67.1
64.6
69.3
63.4
62.4
52.7
80.7
46.6
71.8
63.1
VIT-1/16
TA
65.8
57.5
63.8
59.5
64.7
54.0
88.0
45.3
67.5
67.1
TIES
60.9
53.0
62.8
45.0
LW ADAMERGING
65.5
65.7
69.8
59.4
50.1
44.2
47.1
71.8
57.9
LORA-WEMOC
62.7
60.2
69.4
37.7
52.1
39.9
63.1
45.3
64.3
51.7
MUDI-MERGIT
64.6
55.1
70.1
65.3
69.1
60.4
90.0
48.4
68.8
79.3
OPCM
67.9
55.9
73.7
77.5
74.4
63.2
94.1
49.2
72.3
79.6
NUFILT (Ours)
64.9
54.5
77.8
83.7
88.9
83.7
98.1
51.8
70.0
86.1
FINE-TUNED
59.3
98.6
69.7
WA
69.5
VIT-1/14
TA
70.4
74.1
73.9
66.3
69.9
65.6
56.6
78.6
70.4
TIES
69.7
70.3
65.3
47.9
76.1
63.6
94.7
$4.4
77.9
72.3
MAGMAX-INI
73.1
73.7
75.6
64.6
73.7
68.8
94.6
56.1
78.0
71.7
LW ADAMERGNG
68.8
78.6
75.9
65.7
58.3
51.6
79.9
57.4
80.6
52.4
PERA-WEMOE
68.1
68.7
53.2
47.5
49.4
49.1
WUDI-MERGING
77.0
78.2
63.7
OPCM
73.1
78.3
82.4
80.2
80.8
80.4
61.6
84.8
76.3
NUFILT (Ours)
74.8
82.3
87.6
90.1
93.6
94.4
98.6
66.2
94.5
86.4
OxfordlUTPPot
FashionMNIST
RenderedSST2
FER2013
STL.0
CIFARIO0
CIFARIO
Food101
EMNIST
KMNIST
Model
C. FINE-TUNED
58.3
68.5
86.7
40.2
70.5
50.0
90.7
72.4
54.5
54.5
WA
50.2
84.1
97.0
69.8
92.7
80.4
71.3
15.0
11.5
61.8
VIT-B32
TA
51.4
82.3
94.9
64.6
91.4
71.9
73.9
17.8
12.2
59.9
TIES
49.5
81.3
95.2
63.7
91.2
70.2
73.7
17.8
16.9
59.8
MAGMAX-IND
79.9
94.6
68.7
91.9
73.8
74.3
18.3
ADAMERGING
89,9
LORA-WEMOE
44.6
72.5
86.1
40.1
63.8
63.8
10.3
12.8
55.7
NUDI-MERGING
60.7
80.3
93.1
60.9
85.4
62.8
76.1
38.2
21.9
70.2
OPCM
55.8
82.9
95.9
67.6
92.8
74.0
76.3
22,4
18.3
64.6
NUFILT (Ours)
62.5
81.6
95.1
63.1
91.1
66.0
85.4
42.6
43.5
71.6
C. FINE-TUNED
60.5
84,5
90.5
38.8
73.6
61.9
89.7
83.3
51.5
72.8
50.9
98.0
72.9
94.2
15.6
V7T-8/16
TA
97.0
93.1
TIES
50.4
87.9
96.3
63.1
78.0
75.0
23.4
24.9
61.5
MAGMAX-IND
57.7
88.8
97.5
71.5
94.4
81.3
77.2
24.5
25.0
59.4
LW ADAMERGING
46.8
88.9
98.1
69.2
91.4
86.6
67.2
17.2
11.0
59.2
CORA-WEMOE
45.6
91.2
92.3
41.3
64.3
78.1
48.0
23.5
16.6
52.7
WUDI- MERRING
64.7
91.5
95.9
67.5
90.2
78.3
81.4
50.8
30.5
OPCM
97.7
73.2
94.7
NUFILT (Ours)
92.3
97.0
70.5
71.0
72.1
C. FINE-TUNED
68.0
92.1
94.5
60.5
85.7
74.8
89.0
59.2
78.8
WA
$2.7
94.2
99.2
81.7
97.0
90.7
77.4
16.1
10.4
66.1
VILLIR 114
T'A
55.7
94.2
98.6
79.1
91.6
87.6
80.8
17.6
10.6
63.6
TIES
$7.6
93.5
97.8
74.0
95.6
84.7
79.7
20.2
12.6
58.4
MAGMAX IND
52.9
93.9
98.7
97.3
89.5
81.6
19.2
11.1
ADAMERGING
49.2
93.5
99.3
77.2
95.8
18.6
LORA-WE --RA-EEMOE
46.3
84.5
87.6
70.5
18.7
56.5
WUDI-MERGING
66.2
95.6
98.6
79.5
95.5
84.0
46.1
23.9
78.3
OPCM
61.8
95.4
99.2
83.0
97.8
90.9
86.0
26.4
14.7
71.0
NUFILT (Ours)
67.0
95.8
98.8
79.6
96.5
90.4
91.9
51.0
74.3
77.2
24

---

Published as a conference paper at ICLR 2026
Mean spnse lryers
Maan ECTD Ryars
Maan acoces bowm
Vedee
Tex: Vedeer
Tex
(a) Mean heatmaps: full model (left), attention layers (middle), and MLP layers (right).
Mode: NTT-822
Model VIT-612 (win)
Model WTT-GG (yio)
Tex:
(b) 90th percentile heatmaps: full model (left), attention layers (middle), and MLP layers (right).
Morae VIT-633
Modee VIT-003 (etr)
Medel VTF0002 (mip)
ECGF Some byvers
ECGF
50
1810
Busspaao
Buispado
(c) ECDF distributions: full model (left), attention layers (middle), and MLP layers (right).
Figure 6: Subspace affintyy between data and task vectors ViT-B/32 across eight datasets.
25

---

Published as a conference paper at ICLR 2026
Meen soose lraen
Maan SOOU boar
Maan acoees ligers
Vedke
Tex:
Tex
(a) Mean heatmaps: full model (left), attention layers (middle), and MLP layers (right).
Model UITLLIS
Mode: VTTU (M00))
Mode: VITULIE gripo
Tex:
Tex Wedden
(b) 90th percentile heatmaps: full model (left), attention layers (middle), and MLP layers (right).
Moddel VITU11
Modil VIT4 714 (ait))
Model VITUU1 (map))
ECGF SomeS bivers
ECGF
1000 1800
Sutspacoo
Sucspaso
Bucsppoo
(c) ECDF distributions: full model (left), attention layers (middle), and MLP layers (right).
Figure 7: Subspace affinity between data and task vectors ViT-L/14 across eight datasets.
26

---

Published as a conference paper at ICLR 2026
Wean ocnee Lryeng
Mean acvas lopara
Mean acros lapara
8 Tel Wet
Mean heatmaps: full model (left), attention layers (middle), and MLP layers (right).
8ith permentle scrost ingnee klddel Flov
perzentlle in sconos byywes Model
perzention beyees Model Top--T-baen
Chil
(b) 90th pereentile heatmaps: full model (left), attention layers (middle), and MLP layers (right).
ECOF 2000 Syyee
ECOP
197 420
277 hat 520
Bubssicce Alliity
Butspaco Alficly
Bucsppaoo Aftraty
(c) ECDF distributions: full model (left), attention layers (middle), and MLP layers (right).
Figure 8: Subspace affinity between data and task vectors in Flan-T5-base across eight datasets.
27


Reviews:
[{"review_id": "4F5rQPy3qv", "reviewer": "Reviewer_FVQh", "summary": "This paper presents NUFILT, a data-free continual model merging approaches with good empirical performance and theoretical grounding.", "questions": "1. Using LoRA. One key benefit of LoRA is that it updates only a lightweight set of parameters. In this case, the LoRA parameters are going to be incorporated to the base parameters, thus this advantage is lost. Is it necessary to use LoRA then, or maybe it still brings some training regularization?\n2. The empirical results demonstrate good accuracy but relatively poorer backward transfer (Table 1). Have you tried using a weighted sum for the two objectives in Equation 16 to enforce less forgetting?\n3. From the description of OPCM [1], the correct citation of the vision benchmarks should be [2] and [3].\n\n[1] Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging\n\n[2] Editing Models with Task Arithmetic\n\n[3] Localizing Task Information for Improved Model Merging and Compression", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The writing and formatting are good and clear.\n2. The authors provide detailed and intuitional theoretical derivations to arrive at the final solution.\n3. NUFILT achieves good empirical performance on both vision and NLP tasks, outperforming previous train-free and train-required merging methods on this task.", "weaknesses": "1. Potential unfair baseline comparison: The baseline train-free merging methods (e.g. TA, TIES) are implemented with theta_merged_t = theta_merged_t-1 + lambda tau_t. This implementation biases the model towards the early task vectors. For example, the optimal lambda for the first task is clearly 1.0, while later tasks have gradually reducing optimal lambda values. This probably explains the result of task arithmetic (Table 1) having lower performance than even weight averaging. I think the more fair comparison to these approaches should include a downscaling of previously merged task vectors (similar to the implementation of weight averaging), that is: theta_merged_t = (t-1)/t * theta_merged_t-1 + (1/t) * theta_0 + (lambda / t) * tau_t.\n\n2. Comparison to OPCM. The first step of NUFILT resembles OPCM, i.e., they both try to project the current task vector to the orthogonal space of previous merged vector. Given the similarity on the intuition, a more detailed comparison should be added to explicitly demonstrate the differences and novelties, especially on how it addresses the criticism to OPCM that it struggles when task vectors are inherently entangled or non-orthogonal (section 3.2).\n\n3. Introduced new terminologies: I do not agree it is necessary or worth to introduce the new terms transparency and fidelity, when they are just new names for the classical Stability-Plasticity trade-off in continual learning. They both measure the adaptiveness to new task and forgetting on previous tasks. Furthermore, the first contribution in the introduction that “formulating these two desiderata frames a new open challenge absent from prior work” seems an over-claim.\n\n4. Computational cost to train-free methods: Table 4 only compares the computational cost to the training-required baselines. It is necessary to compare to the training-free baselines (which have way lower computational cost) as well to demonstrate the performance-computation trade-off.\n\n5. Scalability to larger models: The paper demonstrates that it can scale to ViT and T5 models, but did not provide results with larger models (e.g., Llama), is it possible to scale to even larger language models given the training-required procedure?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yb7dYTGjlv", "reviewer": "Reviewer_SpsW", "summary": "This paper proposes NUFILT (NUllspace FILTering), a lightweight, data-free adaptation framework for Data-Free Continual Model Merging (DFCMM) aimed at improving transparency and fidelity. The authors argue that task vectors align with data representation subspaces and provide theoretical guarantees for approximate subspace alignment. Empirically, NUFILT reports 4–7% gains in average accuracy over prior work, with minimal backward transfer (i.e., limited forgetting of previously learned tasks), and requires only 50 optimisation steps.", "questions": "(1) Hyperparameter sensitivity: Fig. 4 suggests sensitivity to $r_p, r_l, r_v$. In a data-free setting, how were these tuned? Is there a data-free or proxy procedure (e.g., curvature/NTK proxies, spectral heuristics, or stability criteria) to select them?\n\n(2) Rank assumption: L296 assumes $r_v \\geq r_d$, yet the implementation fixes it to a very low value $r_v = 8$ which may not guarantee the above assumption is met. Does this have any implications?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "(1) Leveraging null-space filtering for data-free continual model merging is an interesting novel idea.\n\n(2) The adaptation strategy is lightweight and inexpensive, making it broadly applicable in data-restricted settings.\n\n(3) Empirical results validate that NUFILT improves average accuracy while maintaining low forgetting.", "weaknesses": "(1) The paper presents transparency and fidelity as distinct desiderata, but they are closely related and longstanding goals in continual learning. As such, their framing does not feel like a novel contribution.\n\n(2) Despite emphasizing transparency and fidelity, the paper does not quantitatively track or compare these properties for NUFILT vs. baselines.\n\n(3) Evaluation is limited to relatively older architectures (CLIP and T5). Results on modern LLMs would materially strengthen the paper’s relevance.\n\n(4) The work also lacks comparison against more recent merging techniques, such as ISO-Merging[1], KnOTS[2], TSV-M[3] which also use the SVD decomposition to align task vectors.\n\n(5) Notation and clarity issues:\n\n(a) Terms are introduced without definition (e.g., “representation covariance” at L214; use of $\\sigma$ at L249).\n\n(b) Overloaded symbols (e.g., T in Algorithm 1).\n\n(c) Non-standard or unclear formulations (e.g., Eq. (3–4): the product of X with task vectors is not well specified).\n\n(d) Undefined acronyms (e.g., ECDF).\n\nCollectively, these hinder readability and reproducibility.\n\nReferences:\n\n[1] Marczak, D., Magistri, S., Cygert, S., Twardowski, B., Bagdanov, A. D., & van de Weijer, J. (2025). No task left behind: Isotropic model merging with common and task-specific subspaces. arXiv preprint arXiv:2502.04959.\n\n[2] Stoica, G., Ramesh, P., Ecsedi, B., Choshen, L., & Hoffman, J. (2024). Model merging with svd to tie the knots. arXiv preprint arXiv:2410.19735.\n\n[3] Gargiulo, A. A., Crisostomi, D., Bucarelli, M. S., Scardapane, S., Silvestri, F., & Rodola, E. (2025). Task singular vectors: Reducing task interference in model merging. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 18695-18705).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LmZ2ucdMjU", "reviewer": "Reviewer_GLxk", "summary": "This paper proposes NUFILT for data-free continual model merging (DFCMM). Each new task’s update (task vector) is first projected onto the null space of a low-rank subspace spanned by the previously merged updates. Then a lightweight LoRA term is trained (using a projection-based surrogate objective) to recover task-specific signal. Finally, the projection and LoRA are fused back into the backbone so no extra parameters remain at inference. The paper motivates the projection with an “approximate alignment” claim between task-vector subspaces and data-representation subspaces, and reports gains over WA/TA, OPCM, WUDI-Merging and others on CLIP vision suites and GLUE with Flan-T5.", "questions": "see weekness", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper present a simple, practical recipe. The three-step pipeline is easy to follow. \n- The proposed method elegantly combines null-space projection with parameter-efficient LoRA adaptation, supported by solid mathematical reasoning and empirical evidence of subspace alignment. The approach is conceptually simple yet technically sound.\n- Experiments are thorough and well-controlled, covering vision and language tasks, multiple architectures, and extensive baselines. The ablations and sensitivity analyses are carefully executed, providing convincing evidence of robustness and practical usability.", "weaknesses": "I overall like this paper, which is clearly written and technically rigorous. I have only few questions.\n\n1. In Fig. 4(a), the accuracy drops when introducing a small projection rank \n(r_p = 1-4), even lower than using no projection (r_p = 0). Could the authors clarify why a small null-space rank initially hurts performance? Additionally, backward transfer first decreases and then improves as r_p increases, what causes this non-monotonic behavior?\n\n2. The subspace affinity is defined as $A(V_d,\\hat V)=\\frac1{r_d}\\|\\hat V^\\top V_d\\|_F^2\\in[0,1]$ , but Theorem 1 defines $A(V_d,\\hat V)=1-\\frac1{r_d}\\|\\hat V^\\top V_d\\|_F^2$ and claims $A\\le\\zeta^2$ (Eq. 6). I might have misunderstood. Please check this.\n\n3. The method repeatedly requires SVDs per layer on $\\tilde\\tau_{\\le t-1}$ and on $\\tau_t$  but runtime in Table 4 only reports the adaptation loop, not the SVD cost. \n\n4. Although the robustness analysis is informative, the chosen global ranks might still require adjustment for different model scales. A short discussion on adaptive or automatic rank selection would strengthen the paper’s practical perspective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ihBb6QKrpS", "reviewer": "Reviewer_m3Kf", "summary": "This paper proposes Null-space Filtering to address the data-free continual model merging problem: sequentially merging multiple fine-tuned models into a single backbone to cover all task label spaces, without the original training data and only with access to the current task model and the merged backbone network. The authors formalize the problem as two data-level desiderata: transparency and fidelity, and propose a strategy to map these data-level objectives into parameter space: based on the observation that the main subspace of the task vector is approximately aligned with the representation subspace. The paper provides detailed vision and NLP experiments showing that NUFILT outperforms multiple recent baseline methods in ACC and BWT metrics.", "questions": "1. What are the results of NUFILT when apply to larger scale models like qwen and llama ?\n2. Wiil the initialization of LoRA affect the merging result。\n\nIf the author can solve the question and the weakness well, i will raise my score.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing is quite easy to read and it was well-written\n2. The idea of ​​constructing a null-space projection based on empirical findings is both intuitive and mathematically connected, providing a reasonable path for \"replacing data supervision with parametric geometric structures in the absence of data.\"\n3. The paper studies multiple base models\n4. The gains from their proposed method of sampling are convincing and comprehensive", "weaknesses": "1. The use of Null Space has been applied in many reccent works, such as [1] [2] [3], so the contribution of the work is incremental. The author should further discuss the relation between this works.\n2. The paper uses quantities such as singular vectors representing the data, the largest singular value of a matrix, and σ1(X) to derive bounds (e.g., Corollary 1). However, in the strict case of data obscurity, how these quantities are estimated/upper bounded in practice needs to be more clearly explained. The current bridge between theoretical assumptions and practical implementations is not transparent enough.\n3. The paper claims to use a set of global hyperparameters, but does not provide guidance on hyperparameter adjustment for different backbones or longer sequences; in addition, it is unclear how the number of training iterations, rank selection, etc. scale to different model sizes.\n\n\n\n\n[1] Alphaedit: Null-space constrained knowledge editing for language models\n\n[2] Multi-task model merging via adaptive weight disentanglement\n\n[3] Orthogonal Subspace Learning for Language Model Continual Learning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)