Memory-Free Continual Learning with Null Space Adaptation for Zero-Shot Vision-Language Models
Authors: Yujin Jo
Taesup Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Continual Learning
Vision-Language Models
Parameter-Efficient Fine-Tuning
Null-Space Methods
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated remarkable zero-shot generalization, enabling deployment in a wide range of real-world tasks without additional task-specific training.
However, in real deployment scenarios with evolving environments or emerging classes, these models inevitably face distributional shifts and novel tasks.
In such contexts, static zero-shot capabilities are insufficient, and there is a growing need for continual learning methods that allow models to adapt over time while avoiding catastrophic forgetting.
We introduce NuSA-CL (Null Space Adaptation for Continual Learning), a lightweight memory-free continual learning framework designed to address this challenge.
NuSA-CL employs low-rank adaptation and constrains task-specific weight updates to lie within an approximate null space of the model's current parameters.
This strategy minimizes interference with previously acquired knowledge, effectively preserving the zero-shot capabilities of the original model.
Unlike methods relying on replay buffers or costly distillation, NuSA-CL imposes minimal computational and memory overhead, making it practical for deployment in resource-constrained, real-world continual learning environments.
Experiments show that our framework not only effectively preserves zero-shot transfer capabilities but also achieves highly competitive performance on continual learning benchmarks. 
These results position NuSA-CL as a practical and scalable solution for continually evolving zero-shot VLMs in real-world applications.

Published as a conference paper at ICLR 2026
MEMORY-FREE CONTINUAL LEARNING WITH NULL SPACE ADAPTATION FOR ZERO-SHOT VISION-LANGUAGE MODELS
Yujin Jo and Taesup Kim
Graduate School of Data Science Scoul National University
ABSTRACT
Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated remarkable zero-shot generalization, enabling deployment in wide range of real- world tasks without additional task-specific training. However, in real deployment scenarios with evolving environments or emerging classes, these models inevitably face distributional shifts and novel tasks. In such contexts, static zero-shot capabil- ities are insufficient, and there is growing need for continual learning methods that allow models to adapt over time while avoiding catastrophic forgetting. We introduce NuSA-CL (Null Space Adaptation for Continual Learning). a lightweight memory-free continual learning framework designed to address this challenge. NuSA-CL employs low-rank adaptation and constrains task-specific weight updates to lie within an approximate null space of the model's current parameters. This strategy minimizes interference with previously acquired knowledge, effectively preserving the zero-shot capabilities of the original model. Unlike methods relying on replay buffers or costly distillation, NuSA-CL imposes minimal computational and memory overhead, making it practical for deployment in resource-conse coource-constrained real-world continual learning environments. Experiments show that our frame- work not only effectively preserves zero-shot transfer capabilities but also achieves highly competitive performance on continual learning benchmarks. These results position NuSA-CL as practical and scalable solution for continually evolving zero-shot VLMs in real-world applications.
INTRODUCTION
Vision-language foundation models such as CLIP (Radford et al. 202T) have brought about major shift in artificial intelligence by enabling zero-shot generalization. Their powerful text-image aligned representations now serve as the perceptual core for a new generation of systems, including
Multimodal large language models (MLLMs) like LLaVA (Liu et al., 2023 2024) and Vision- language-action (VLA) models for robotics (Kim et al. 2024 Shukor et al. 2025). However, these advanced systems inherit critical limitation from their static backbones: In settings where data distributions and user requirements are constantly evolving, their knowledge is frozen. To bridge the gap between static foundation models and the demands of real-workk deployment without resorting to massive retraining, Continual Learning (CL. has emerged as a promising solution. CL allows models to incrementally acquire new knowledge while preventing catastrophic forgetting of both pre-trained and previously learned tasks.
Existing CL paradigms, however, face fundamental scalability wall. Storage-based methods, which rely on experience replay or reference data (Saha et al.] 202T: Wang et al., (202T), are inherently constrained by storage costs that grow linearly with the number of tasks. On the other hand, expansion- based methods introduce new modules for each task, such as adapters or prompts (Yu et al. 2024] Tang et al. 2024), forcing model's parameters and architectural complexity to grow unbounded over time. While effective on short-term benchmarks, these dominant approaches are ill-suited for
CCoresponding author.

---

Published as a conference paper at ICLR 2026
(1) Identifving Null Space
U
V
SVD
VPT
W
Up
Un
Σ
V
(3) Merge-and-Updane
(2) Null Space Constrained Adaptation
Un
W
AW
AW
M
Figure 1: The NuSA-CL framework. Starting with the weights from the previous task Wr-1, I. we first perform SVD to identify the intrinsic null space. new low-rank update AW, then learned under persistent constraint that confines it to this space. Finally, the update is merged to produce the new weights W, W + AW, completing the cycle.
true lifelong learning. Even many crrameter-efficien fine- tin-tuning (PEFT) techniques still depend on explicitly storing prior task information to mitigate interference (Liang & Li 2024 Lu et al. 2024)
We argue that overcoming this scalability wall requires paradigm shift from relying on external resources to enabling a model to adapt using only its intrinsic structure. We propose NuSA-CI. (Null Space Adaptation for Continual Learning), a continual learning framework that enables model with fixed capacity to efficiently reorganize its own knowledge to accommodate new information. NuSA-CL dynamically identifies an underutilized null space in the model's current weights via SVD before each new task and strictly confines all weight updates within these interference- dimensions throughout training. This data-agnostic process concludes by merging the update into the backbone, maintaining fixed parameter budget. By preserving the model's core knowledge, NuSA-CL enables stable continual adaptation, offering the ultimate form of scalability with zero storage overhead, zero auxiliary model load, and zero parameter growth which is crucial set of properties for resource-constraing environments such as autonomous agents or on-device AI.
Our contributions are summarized follows:
We propose NuSA-CL, novel memory-free and resource-efficient continual learning method for vision-language foundation models, designed to operate effectively in resource-constrainee environments without relying on memory buffers or knowledge distillation.
NuSA-CL. introduces a null space-constrained low-rank update strategy that integrates new knowledge into approximate null space of the pre-trained parameters, thereby preserving zero-shot generalization while enabling stable and incremental learning.
Our method demonstrates significant computational and memory efficiency, making it well- suited for real-world applications that demand lifelong adaptability under limited resources.
2 RELATED WORK
2.1 CONTINUAL LEARNING WITH PARAMETER -FFFICEENT
Continual learning (CL) aims to adapt models to a sequence of tasks without the catastrophic forgetting of previously acquired knowledge (Li & Hoiem 2017 Rebuffi et al. 2017 Chaudhry
2

---

Published as a conference paper at ICLR 2026
et al. 2019 Ding al. 2022). ) A central challenge is the stability-plastiitty trade-off. which is amplified in foundation models like CLIP, where forgetting undermines not only past tasks but general-purpose purpose zero-shot capabilities acquired during pre- pre-training (Wortsman al. 2022 Zhang et al. 2023 Tan et al., 2024). While full fine-tuning methods like ZSCL (Zheng et al. 2023) can be effective, they often require essource-intensiv techniques, hindering their scalability. Parameter-Efficient Fine-Tuning (PEFT) offers a lightweight alternative by restricting updates to a small subset of parameters. This family includes prompt-based methods that isolate task-specific knowledge (Wang et al. 2022b a and adapter- based methods that insert small, trainable modules for each task YYu et al., 2024]| Tang et al., 2024, Wu al. 2025 Wei et al.) 2024). However, many of these approaches still externalize new knowledge into task aas--specffic modules, which contributes to parameter growth over long task sequences. In contrast, our work focuses on adapting the model's
also the general
core weights within fixed parameter budget.
Our method targets the underlying zero-shot vision language model itself. Such encoders are increasingly used multimodal foundation components that extract visual features for larger systems, including MLLMs and VLA agents, so enabling them to continually absorb new visual knowledge under fixed parameter budget and without external storage is crucial. While higher-level components (e.g., language heads, instruction-tune instruction tuning task-specific adaptation, our method provides complementary mechanism the feature-encoding level rather than a direct competitor to full multimodal generation stacks.
decoders, or policy layers) can be continually trained via
2.2 ORTHIOGONAL PROJECTION AND NULL SPACE APPROACHES
To explicitly mitigate interference, orthogonal projection techniques constrain parameter updates to subspaces that are orthogonal to those encoding prior knowledge. Prior works typically project new updates away from subspaces identified using stored information, such as past data, features, or gradients (Wang et al. [021]] Saha et al., 2021 Zhao et al. 2023). For instance, InfLoRA (Liang Li 2024) adapts this concept to LoRA but still necessitates a memory bank of past gradients to enforce orthogonality. This reliance on external memory contrasts with our strictly memory-free approach. Our method distinct in that it derives the approximate null space intrinsically from the model's ss current weight structure via SVD, requiring access to storage of past data, features, or gradients.
2.3 SVD-GUIDED ADAPTATION IN FOUNDATION MODELS
Several recent methods explore using the spectral properties of a model's weights to guide adaptation, primarily for single-task fine-tuning (Lingam et al. 2024 Yang et al. 2025 Tang et al., 2025). These approaches involve adapting either principal components (Meng et al. 2024) conversely, minor low-energy components to minimize interference with pre-trained knowledge (Wang et al., (025). While insightful, these methods differ from our work in two fundamental ways. First, they designed for single-task adaptation, not the long-term, sequential learning required in CL. Second, and most critically, they use the low-energy subspace only for initialization, allowing the weight updates to deviate from this subspace during training. Our method, in contrast, enforces persistent constraint, ensuring updates are strictly confined to the dynamically identified null space. This enables stable, lifelong learning within a capacity model.
METHOD: NULL SPACE ADAPTATION FOR CONTINUAL LEARNING
The core of NuSA-CL is cyclical, data-agnostic adaptation process that enables a model to learn from new task while preserving previously acquired knowledge. For each task sequence, the process unfolds in three stages as illustrated in Figure [ (1) Null Space Identification via SVD: We begin with the model's current weights, Wt-1. and perform Singular Value Decomposition (SVD) to identify a low-energy subspace, that is, the intrinsic null space where prior knowledge is minimally encoded. (2) Constrained Adaptation: We then train a tpsk-spe low-rank update, AW, for the current task. Crucially, this update is persistently constrained to lie strictly within the identified null space throughout training. (3) Weight Merging: After training, the learned update is merged directly into the backbone weights, producing the updated model W, W 1 + AW, This evolved model,
3

---

Published as a conference paper at ICLR 2026
with its fixed parameter budget, then serves the starting point for the next task, where the eycle repeats.
3.1 IDENTIFYING THE INTRINSIC NULL SPACE
Let W E R be weight matrix from the model. We compute its SVD, = UEVT, analyze its spectral properties. We posit that the principal components, associated with high-energy singular values, encode the core knowledge of the model. To provide a principled basis for our approach, we first verify the existence of sufficiently large low-energy subspace. We identify the dimension of the principal space by finding the smallest integer that captures at least a fraction the total spectral energy:
Σ
o p W
The remaining dimensions constitute the intrinsic null space, spanned by the basis vectors (Un, Vx). For practical stability and to maintain consistent number of trainable parameters across all layers and tasks, we cap the dimension of our update by hyperparameter Tmax. The effective rank of our update is thus defined as r min d k, fmax).
We can now express the decomposition as:
Σ 0
0
U U= Un), V V Vn),
Σ
En
where (Up, 1/)) represent the top-k components of the principal subspace, and (Un, Vx.) span the approximate null space.
3.2 CONSTRAINED ADAPTATION WITHIN THE NULL SPACE
To prevent interference with existing knowledge during continual learning, impose persistent constraint that strictly confines all new parameter updates to the identified null space. We formulate the task-specific adaptation as LoRA-like low-rank update AW m=x= but with critical modification. Instead of learning two projection matrices, we define the update as:
AW = UV.
(3)
Here, the basis matrices Un and Vn are derived from the SVD of the frozen weight W and are themselves kept frozen during training for the current task. The intermediate matrix M E grxr is the only trainable component and initialized zero matrix for each new task. This formulation ensures that the update AW is mathematically guaranteed to be orthogonal to the principal subspace of W. thereby minimizing interference. This persistent constraint key distinction from prior work (Wang et al. 2025) that uses such subspaces only for initialization, after which updates are free to deviate.
3.3 CONTINUAL ADAPTATION VIA UPDATE MERGING
core component of NuSA-CL's scalability is its ability to operate within a fixed parameter budget. This is achieved by merging the learned low-rank update AW directly into the base weights after training on each task is complete. For given task t, the new weight matrix W, is computed as:
W, Wi + AW,
This apdaae-and-mergg cycle allows the model to sequentially accumulate knowledge from new tasks without adding any new parameters modules. The resulting model, with its updated weights W., then becomes the starting point for the next task, t + At the beginning the new task, the entire process repeats: the now-updtted weights W, decomposed via SVD to identify a new intrinsic null space, ensuring that the model always adapting in directions that are least disruptive to its full, accumulated knowledge.
4

---

Published as a conference paper at ICLR 2026
4 THEORETICAL MOTIVATION
In this section, we provide a theoretical motivation for our approach. We analyze the degree of interference in parameter space to demonstrate how persistent constraint provides principled mechanism for mitigating catastrophic forgetting. Our analysis shows that by freezing the null space basis vectors (Un, Vn,) and only learning the small intermediate matrix M defined in Eq. 3 the update direction is guaranteed to be nearly orthogonal to the dominant components of the existing model weights.
4.1 INTERFERENCE BOUND FOR SINGLE UPDATE
We first present a lemma that characterizes the interaction between the existing weights and a single task-specific update.
Lemma 1 (Bounded Interference via Null Space Constraint). W UEVT be the SVD of weight matrix, and let AW UnMV, be update restricted to its intrinsic null space. The interference in parameter space, measured by the Frobenius inner product, is bounded by:
(W, (W)))) ofill ||M
(5)
where o null == 0k+1 the largest singular value within the null space.
Proof. Expanding the inner product: (W, AW) = Tr(WT AW) = TRVVU UN,VEUU,,MV UV,,V TV, Since U. Un [0; I,] and VTV, [0; Ir]. the trace simplifies to: Tr(E, S ||2,1l2 || max
|
4.2 FORGETTING CONTROL IN CONTINUAL LEARNING
The above lemma naturally generalizes to the continual learning setting, where multiple tasks learned sequentially.
Theorem 2 (Cumulative Interference Bound), Let W. Wi- AW, where AW; U. U, V is the update for task t. The cumulative interference across T tasks is bounded by:
T
T null t.max
Σ (W....W)rr
Σ
[|M]]].
(6)
This result demonstrates that by constraining updates to low-energy subspaces, NuSA-CL. bounds the cumulative parameter-level interference across tasks. This provides principled mechanism for mitigating catastrophic forgetting, as it minimizes disruptions to the dominant weight structures that encode prior knowledge.
We emphasize that the above results are stated in parameter space and should be viewed as local stability condition rather than full function-level guarantee. Under standard smoothness assumptions, constraining updates the principal directions limits how much predictions on past tasks can change, while most adaptation is pushed into the approximate null space. Empirically, Sec. Sec. shows that such null-space- -pssspeuiiieeddd guided updates consistently reduce forgetting compared to alternative subspace choices and baselines. Deriving tighter, function-level forgetting bounds remains an interesting direction for future work.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Benchmarks. Our primary evaluation is conducted on the Multimodal Task Incremental Learn- ing (MTIL) benchmark (Zheng et al.) 2023), sequence of 11 diverse vision datasets designed to test model's ability to learn new tasks while preserving its core zero-shot capabilities. To assess long-sequence scalability, we also evaluate on the standard Jaass-nncrementll CIFARI00 bench- mark (Krizhevsky et al.] (200)). splitting 100 classes into sequences of 10, 20, and 50 tasks. We report three key metrics: Transfer, the zero-shot accuracy on unseen tasks; Avg., the average accuracy across all tasks during training; and Last, the final average accuracy, which measures forgetting.
5

---

Published as a conference paper at ICLR 2026
Table Performance and Computational Efficiency Analysis on the MTIL Benchmark. Boldface in- dicates the top storage-free performer. indicates methods re-implemented on the CLIP architecture.
Computation & Memory Cost
Performance (%)
Params
Additional Storage
Peak (GB)
GPU-Hours
Transfer
Avg.
Last
Storage based Modelss
ZSCL Zheng et 2023
149.6M
Model (10.5GB)
43.1
47.24
68.1
75.4
83.6
Adapters Yu
2024
59.8
Routers (4,SGB)
15.5
3.42
68.9
76.7
85.0
DIKI [Tang et 2029
1.M
Task Stats (159MB)
10.2
4.40
68.7
76.3
85.1
InfL Linng
2024
7.8M
Grad. Proj. Mem. (9MB)
6.6
4.29
66.2
74.2.
83.6
Storage-Free Free Models
Continual-FT
149.6M
None
12.76
44.6
55.9
77.3
LoRA! Hu et al. 2021
15.7M
None
6.7
1.21
63.9
70.1
79.9
MiLoRA Wang el 2025
15.7M
None
6.7
1.24
62.8
68.7
77.4
NoSA-CL (Owrs)
1.MM
None
6.6
1.21
68.6
75.1
82.8
Table 2: Transfer, Avg., and Last (%) for PEFT continual learning methods with a fixed backbone parameter budget on the 5-shot MTIL benchmark. LoRA, MiLoRA, and NuSA-CL are strictly storage-free, while InfLoRA uses small gradient projection memory; f indicates methods re- implemented on the CLIP architecture for fair comparison.
Aireraft
Cakeehi001
CIFAR100
DTD
EmroS AT
Flowers
Food
MNIST
OxfordPet
Cars
SUN397
AVG.
Zero-shot CLIP
24.3
88.4
68.2
44.6
54.9
71.0
88.5
59.4
89.0
64.7
65.2
65.3
Transfer
LoRAT
2021
85.9
63.0
42.3
39.5
55.9
80.5
63.7
76.0
49.5
57.1
60.4
Mil ORAT
Wang
2025
84.8
59.6
43.2
37.7
50.6
78.3
61.4
80.2
42.4
55.6
59.4
InfLoRA
Liang & 2024
87.2
65.9
44,5
52.1
64.3
85.3
63.4
83.8
58.9
62.5
66.8
NuSA-CL (Ours)
88.2
67.5
43.3
55.8
65.3
86.2
62.8
84.9
62.2
64.7
68.1
Avg.
LoRAT
Hu et 2021
15.9
90.6
68.1
54.1
69.1
74.3
81.8
73.6
81.9
58.3
62.0
66.8
MiLoRA
Wang ct 2025
13.6
89.4
66.8
53.8
66.4
73.5
81.4
71.9
83.5
56.2
62.5
66.0
InfLoRA1
Liang & 2024
18.7
91.0
73.0
55.4
67.8
78.2
86. 1
72.7
85
61.1
63.4
68.9
NuSA-CL (Ours)
28.9
90.5
73.2
56.1
71.9
76.9
87.2
72.9
86.8
63.5
65.3
70.3
Last
LoRAT
Hu et al 2021
21.3
89.6
65.3
58.0
76.8
83.9
83.2
90.4
85.6
67.4
72.1
72.2
MiLoRA
Wang al
2025
17.5
88.8
66.0
56.3
72.3
82.4
82.
87.3
87.7
68.2
72.0
71.0
InfLoRA
Liang & 2024
21.9
91.3
73.3
58.9
77.8
90.0
87.9
88.6
89.8
72.6
74.8
NuSA-CL (Ours)
92.0
75.4
Implementation and Baselines. All experiments the CLIP ViT-B/16 backbone. Our method, NuSA-CI, identifies the null space for each task using a cumulative energy cutoff and caps the LoRA update rank at mmax 128. We compare NuSA-CL against three categories of baselines. (1) Full Fine-Tuning Models (e.g.. Continual-FT, ZSCL) update all 150M parameters and require significantly more computational resources (e.g., 4 GPUs in our experiments) compared the single GPU usage of PEFT methods. (2) Storage-based PEFT Models require additional, often expanding, storage, including MoE-Adapters (Yu al.) 2024), DIKI (Tang et al.) (024). and InfLoRA (Liang & Li 2024). (3) Storage-Free PEFT Models, the most practical and challenging setting, includes standard LoRA (Hu al. 202T), MiLoRA (Wang et al. 2025), and our method. For methodological relevance, re-implemented the most comparable LoRA- LoRA-based methods within unified framework. applying adapters to both vision and text encoders with a consistent rank and merging them after each task.
5.2 RESULTS
NuSA-CL Demonstrates a Superior Efficiency-Performaace Tradeoff. Table presents our main results the full-shot MTIL benchmark. As shown, storage-based PEFT methods like MoE- Adapters YYu et al.] (022)) achieve the highest final accuracy. However, this comes at a significant cost: MoE-Adapters requires nearly 60M parameters and expanding router library, while 7SCL. (Zheng

---

Published as a conference paper at ICLR 2026
Table Class-incremental CIFARI00 results: Last and Avg. accuracies (%) for 10/20/50-step splits. Bold = best, underline = second best.
10 steps
20 steps
steps
Methods
Last
Avg
Last
Avg
Last
Avg
CLIP [Radford et al. 2021
65.92
74,47
65.74
75.20
65.94
75.67
Continual-FT
53.23
65,46
43.13
59.69
18.89
39.23
LwF (1.i & Hoiem. 2017
48.04
65.86
40.56
60.64
32.90
47.69
iCaRL DDng
2022
70.97
79.35
64.55
73.32
59.07
71.28
LwI-VR Ding
2022
70.75
78.81
63.54
74.54
59.45
71.02
ZSCL Zheng al.
73.65
82.15
69.58
80.39
67.36
79.92
NuSA CL (Ours)
74.51
80.25
73.84
81.03
71.85
80.19
et al. )023) incurs massive computational overhead of 47.24 GPU-Hours. In contrast, NuSA-CL establishes new state-of-the-an within the practical and challenging storage-free frragee-freeeeee setting, significantly outperforming other competitors like LoRA and MiLoRA. The key finding is that NuSA-CL achieves performance highly competitive with the storage-based based SOTA while being orders of magnitude more efficient. Specifically, compared to Adaaterss, NuSA-CL uses 40x fewer parameters (1.5M vs. 59.8M), zero additional storage, less than half the peak GPU memory, and is nearly 3x faster (1.21 vs. 3.42 GPU-Hours). This result highlights a vastly superior performance-to-cost tradeoff, positioning NuSA-CL as powerful and scalable solution.
NuSA-CL Excels in Data-Efficient, Few-Shot Learning. To further probe the robustness of our approach, we conduct a focused analysis on the challenging 5-shot MTIL benchmark, with results in Table 2 As summarized in Table we distinguish two axes of practicality: (i) whether the backbone parameter budget is fixed across tasks, and (ii) whether external storage (e.g.. replay buffers gradient memories) is required. focuees on PEFT continual learning methods that share the same CLIP backbone capacity and thus operate under fixed backbone parameter budget. Within this regime, LoRA, MiLoRA, and NuSA-CI are strictly storage-free, whereas InfLoRA A is storage-based maintaining an additional gradient projection memory on top of the backbone. This setting acts as a stress test, magnifying the fundamental differences between each approach.
The results clearly demonstrate the superiority of our null-space adaptation strategy. NuSA-CL achieves the best performance across all summary metrics, decisively outerforming InfLoRA, the strongest competitor in this group. This indicates that our persistent null-space constraint is fundamentally more robust and data-efficient strategy for mitigating forgetting than alternatives like subspace initialization (MiLoRA or gradient projection (Infl oRRA), validating the core mechanism of NuSA-CI.
Scalability in Long-Sequence Ineremental Learning. Finally, to address the critical question of long-sequence scalability, evaluate NuSA-CL on the Class- Incremenntt CIFAR 100 benchmark. Table shows that, although CLIP's zero-shot predictions already exceed naive fine-tuning (FT) and LWF LLi & Hoiem 2017). those baselines exhibit severe forgetting tasks increase. Even ZSCL (Zheng et al.) 2023) 's Last accuracy falls below 68% the 50-step split. The advantage of our method becomes increasingly pronounced the task sequence lengthens. In the most challenging 50-step scenario, NuSA-CL achieves final 'Last' accuracy of 71 71.85% significantly outperforming the strongest baseline, ZSCL, by a large margin of over 4.4%. This result provides strong empirical evidence that our dynamic, task-wise re-computation of the null space is an effective and scalable strategy for lifelong learning, confirming the longevity of our approach even after 50 sequential tasks.
6 ANALYSIS
In this section, we analyze the effectiveness of NuSA-CL from multiple angles. We first visual- ize and explain how NuSA-CL's learning dynamics knowledge accumulation versus overwrit- ing fugdammannttllll differ from conventional methods. We then establish why adapting within the null space is a superior strategy for continual learning. Finally, we experimentally validate our core mechanisms and justify the choice of key hyperparameters.
7

---

Published as a conference paper at ICLR 2026
-O-
FUlIFT
+
LoRA
-
NUSA-CL
Rank95 (left)
-C-- Null (right)
TEXT Encoder (d=511 (12)
VISION Encoder (d 768)
42.2
Effective rank (resfd. %)
58.8
59.0
42.0
Effectivv rank (restd,
52.6
48.2
41.8
Null Rablo 9%
Null (%)
58.6
58.4
41.6
52.2
47.8
41.4
58.2
0
52.0
47.6
41.2
58.0
51.8
47.4
41.0
57.8
0
1
4
8 10
2
3
5
6
9
10
Task
Task
Figure 2: Null-space dynamics across tasks. The plots show the evolution of the effective rank (solid lines, left axis) and the null ratio (dashed lines, right axis) for the text and vision encoders. While Full-FT and LoRA exhibit nearly static spectral behavior, NuSA-CL shows consistent increase in effective rank over tasks, indicating progressive utilization of previously underexplored spectral directions. This trend suggests that NuSA-CL integrates new knowledge by expanding into low-energy subspaces rather than overwriting dominant principal components.
6.1 NULL SPACE DYNAMICS: ACCUMULATION VS. OVERWRITING
Figure illustrates a fundamental divergence in the learning dynamics of NuSA-CL compared conventional fine-tuning approaches. The plots track the model's effective rank (solid lines), representing the capacity used to encode core knowledge, and the null ratio (dashed lines), the remaining underutilized capacity. The effective rank (%) defined as the minimum percentage of dimensions required to capture 95% of the weight matrix's 's total spectral energy (ras/d) /d). ros is computed per layer as the smallest integer satisfying Eq. [] The reported r9s values in figures and tables are averaged across layers, and therefore may appear as fractional values.
Conventional methods such as LoRA and Full-FT exhibit near-static spectral behavior. As shown in Figure and detailed in Appendix Table 11 their spectral properties remain almost static across all 11 tasks. For instance, the effective rank of LoRA's vision output projection layer barely changes, shifting trivially from an initial 447.42 to 447.58. This spectral inertia suggests that these methods do not exploit the model's underutilized capacity; instead, they primarily overwrite knowledge within the existing principal subspace, leaving the vast null space dormant.
In contrast, our method actively accumulates knowledge by progressively filling this underutilized space. For the same vision output layer, NuSA-CL's effective rank shows clear and consistent increase. This trend, observed across all attention layers, provides direct quantitative evidence that NuSA-CL dynamically reshapes the parameter space integrate new information. This additive learning process is the core mechanism behind NuSA-CL's ability to mitigate catastrophic forgetting and build a more informationally dense representation over time.
A natural question arises regarding the long-term viability of this approach: does the null space eventually become exhausted? Our analysis indicates that it does not. The "null space" low-energy spectral region, not a finite, empty container. The quantitative data in Appendix Table IT confirms this. Even after learning 10 diverse and challenging tasks, the number of available null directions in the most saturated layer (vision output projection) is still 3 313.58. This is more than double our empirically chosen update rank (Tmaz = 128), demonstrating that substantial, low-interference subspace persists for future adaptation.
To probe longer-horizon behavior under highly correlated task stream, we further analyze the 50-step CIFAR- 1000 CIL benchmark (Table 3 where all steps are constructed from single dataset and thus share highly correlated visual features. Our spectral analysis in Appendix Table 12 shows that the effective rank and null-space ratio measured at the beginning of Step and Step 50 remain essentially stable across all attention projection matrices. These results indicate that null-space- guideddggdddddddd updates do not induce spectral collapse even in highly correlated task streams.
8

---

Published as a conference paper at ICLR 2026
Forgetting (%) +better
6
Subspace
Tail
Transfer Average Last
Top
Random
better
82
4
80
78
3
76
74
Metric
70
68
66
32
64
128
Rank (r)
Rank
(a) Subspace Selection
(b) Maximum Update Rank (Tmax)
Figure 3: Ablations subspace choice and update rank. (Left) Subspace selection. Across all ranks, the Tail (null-like) subspace consistently yields lower forgetting than Top and Random, indicating that low-energy directions provide low-interference region for continual updates. (Right) Maximum update rank. Within the Tail subspace, overall continual learning performance is maximized Tmax 128 (measured by Transfer, Avg., and Last), suggesting that moderate rank balances retention (stability) and task adaptation (plasticity).
6.2 WHY THE NULL SPACE? SUBSPACE SELECTION STRATEGY
Tail outperforms Top and Random mitigating forgetting. To identify an update subspace that minimizes catastrophic forgetting, analyze how different spectral directions affect knowledge retention and adaptation. We compare three low-rank initialization strategies-Top (largest singular directions), Tail (smallest, null-like directions), and Random-using fixed rank 128 and 1000 training iterations per task on the 11 MTIL datasets. Forgetting (defined as the average drop from post-task to final performance) is reported Fig. Bal with detailed per-rank results in Appx. I0
Across all tested ranks, the Tail strategy consistently yields the lowest forgetting. For example, at r 128, Tail achieves 2.57% forgetting compared 4.44% for Top and 4.57% for Random. Although forgetting gradually increases with larger ranks (from 1.46% at = 32 to 5.1 5.11% 96 at 256). Tail remains superior at every rank. These results indicate that low-energy directions form safer region for continual updates, reducing interference with previously acquired knowledge.
Balancing stability and plasticity for continual learning. Does this imply that "the smaller (purer) the null space, the better" for continual learning? Not necessarily. As shown in Fig. 3b overall CL performance maximized around an update rank of Tmax=11 128, which balances stability (retention) and plasticity (on-task adaptation). Consistent with Table 10 observe stability-plastiiie trade- off at both the update-rank dimension and the subspace choice: increasing rank improves Target performance (accuracy on the currently learned task) but induces more forgetting, and Top attains marginally higher Target accuracy on the current task yet suffers substantially larger forgetting. Because continual learning ultimately constrained by retention, these results motivate our design in NuSA-CL: operate in the null space (Tail) and cap the update rank at r Tmax to maintain stability while preserving competitive adaptation.
6.3
VALIDATION OF NUSA-CL'S DESIGN PRINCIPLES
Core Mechanisms are Critical. As shown in Table 4a our core design choices are essential for success. The persistent constraint is vital; unfreezing the null space bases (U,, V/,) to make them trainable leads to significant drop in performance, confirming that strict, persistent constraint is necessary to prevent forgetting. Similarly, multimodal adaptation is superior, as jointly updating both text and vision encoders is key to maintaining cross-modal alignment.
Practicality and Robustness. A potential concern for our method is the overhead of SVD and sensitivity to hyperparameters. However, analysis shows NuSA-CL. is both practical and robust,

---

Published as a conference paper at ICLR 2026
Table Ablation studies validating NuSA-CL's design principles. (a) Core mechanisms such the persistent constraint and multimodal adaptation are shown to be critical. (b) The method is practical, with negligible initialization overhead, and robust hyperparameter choices.
(b) Robustness and Practicality
(a) Core Mechanism Ablations
Robustness Cutoff Threshold (p)
Persistent Constraint Ablation
threshold
Transfer
Avg.
Last
Condition
Transfer
Avg.
Last
0.80
68.29
74.87
82.28
Train only (Ours)
68.58
75.08
82.79
0.90
68.82
75.07
82.74
Train M & Vn
66.37
73.11
82.04
0.95
68.58
75.08
82.79
Train M. Un. Vn
62.60
68.12
77.32
0.99
68.49
74.89
82.70
Modality Ablation
0.999
68.11
72.89
79.16
Modality
Transfer
Avg.
Last
SVD Efficiency Analysis
Both (Ours)
68.58
75.08
82.79
Method
Init. Time
Train Time (hr)
Avg.
Text-only
68.47
72.62
79.09
InfLoRA
~11 min
4.29
74.2
Vision-only
14
70.49
77.86
Ours
min
1.21
75.1
shown in 4bb The SVD initialization exceptionally lightweight. While our data-agnostic SVD a one-time calculation per task with negligible overhead, competing methods like InfLoRA require heavy, data-dependent computations that scale poorly to design subspace using training dataset before learning. Furthermore, results show that performance remarkably stable across a wide range of energy cutoff thresholds, demonstrating that NuSA-CL does not require sensitive hyperparameter tuning.
Practical guidance for larger backbones. In practice, compute SVD once per task and per layer on the attention projection matrices (e.g., Wg. Wx. W.. W.). whose sizes remain moderate for CLiP-style backbones (e.g., up 512 512 for text and 768 768 for vision in VIT-B/16 experiments), making the spectral step inexpensive compared to standard forward-backward passes. While our empirical results focus on ViT-B/16, the same scaling behavior extends naturally to larger CLIP variants where attention projections typically have dimensionalities on the order of 768-1024. In such cases, practitioners may adopt truncated approximate SVD on selected layers further reduce overhead; these engineering choices are orthogonal to the core method and primarily affect runtime rather than the underlying learning dynamics.
CONCLUSION
This paper tackles the challenge of adapting vision-language models evolving tasks without catastrophic forgetting and without the unsustainable resource costs of methods whose storage or parameter counts grow linearly with the number tasks, rendering them impractical for lifelong learning. We introduce NuSA-CL, a memory-free framework based on intrinsic adaptation. NuSA- CL identifies underutilized null space directions and constrains low-rank updates to this subspace, integrating new knowledge while preserving pre-trained capabilities. The learned update is then merged into the base model, maintaining a fixed parameter budget. Across benchmarks, our method delivers superior performance-efficien rivals resource-intensive storage-based approaches at a fraction of the cost. Strong results on long task sequences validate its scalability and effectiveness for lifelong learning, positioning NuSA-C A-CL as a practical solution for deploying adaptable vision-language models in resource-constraing settings.
trade-off: it outperforms other storage free methods and
Limitations and future work. NuSA-CL remains robust on sequences of up to 50 tasks with ViT-B, but its capacity under extreme lifelong settings where the available null space directions may saturate warrants further study. We also note that the SVD step, while negligible with our reduced SVD on VIT-B, could become bottleneck for substantially larger models. Future work includes (i) quantifying sensitivity to task order and semantic relatedness, as highly correlated sequences may concentrate usage of specific null-space dimensions; and (ii) developing lightweight, more reversible integration strategies, enabling selective forgetting without relying on persistent external memory.
10

---

Published as a conference paper at ICLR 2026
Acknowledgements. This work was supported by Institute of Information & communications Technology Planning & Evaluation([IIPP) under the Leading Generative Al Human Resources Development(ITTP- S0-TT--222-RS---- 2-2---SS222--00970083
grant funded by the Korea governmen((MSIT). This work was also supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2024-00345889, Research AT Rohustness Against Distribution Shift in Real- World Scenarios; and No. RS- RS-2023-00222663, Center for Optimizing Hyperscale AI Models and Platforms).
REFERENCES
Lukas Bossard, Mauthieu Guillaumin, and Luc Van Gool. Food-101-mining discriminative compo- nents with random forests. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Purt VI 13, pp. 446-461. Springer, 2014.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip H. S. Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. 2019. URL https:: //arxvv.avivvvv/aa org/ abs 1 1902. 10486
Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describ- ing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3606-3613, 2014.
Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE signal processing magazine, 29(6): 14--12, 2012.
Yuxuan Ding, Lingqiao Liu, Chunna Tian, Jingyuan Yang, and Haoxuan Ding. Don't stop learning: Towards continual learning for the clip model. arXiv preprint arXiv:2207.09248, 2022.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pp. 178-178. IEEE, 2004.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2226, 2019.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, October 2021.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burch- fiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision- Manionn-nnuage--ate action model, 2024. URL http:::////// httpps///axxxv.crg///o/o 09246
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. object representations for fine ane-grrined categorization. In Proceedings of the IEEE international conference computer vision workshops, pp. 554-561, 2013.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2947 2017.
Yan-Shuo Liang and Wu-Jun Li. InfLoRA: Interference-FFree Low-Rank Adaptation for Continual Learning, April 2024.
Vijay Lingam, Atula Tejaswi Neerkaje, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Eunsol Choi, Alex Dimakis, Aleksandar Bojchevski, and Sujay Sanghavi. SVFT: Parameter-Efficient Fine- Mune-Tunigg with Singular Vectors. In The Thirty-eighth Annual Conference Newral Information Processing Systems, November 2024.
11

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL
https://arxiz: wwtp:/ii.org/.asoooeoe.oooo wttp:////////////e 2ttp://aa/2222..88588 08485
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction
tuning, 2024. URL https: org/ abs /2300. 03744
Yue Lu, Shizhou Zhang, De Cheng. Yinghui Xing, Nannan Wang. Peng Wang, and Yanning Zhang. Visual Prompt Tuning in Null Space for Continual Learning, October 2024.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5155/ 2013.
Fanxu Meng, Zhaohui Wang, and Muhan Zhang. PISSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, November 2024.
Maria-Flena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. 722-729. IEEE, 2008.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and patlern recognition, pp. 3498-3505. IEEE, 2012.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 9748-8763. PMLR, 2021.
Sylvestre-Alliss Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient Projection Memory for Continual Learning, March 2021.
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, and Remi Cadene. Smolvla: A vision-language-action model for affordable and efficient robotics, 2025. URL
Yuwen Tan, Qinhao Zhou, Xiang Xiang, Ke Wang, Yuchuan Wu, and Yongbin Li. Semantically- shifted incremental adapter-tuning is a continual vitransformer, 2024. https::/://aaxx
brg/abs/2403. 19979
Longxiang Tang, Zhuotao Tian, Kai Li, Chunming He, Hantao Zhou, Hengshuang Zhao, Xiu Li, and Jiaya Jia. Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models, July 2024.
Pengwei Tang, Yong Liu, Dongjie Zhang, Xing Wu, and Debing Zhang. LoRA-Null: Low-Rank Adaptation via Null Space for Large Language Models, March 2025.
Hanqing Wang, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning, March 2025.
Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training Networks in Null Space of Feature Covariance for Continual Learning, March 2021.
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision pp. 631 631-648. Springer, 2022a.
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy. and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 139-149, 2022b.
12

---

Published as a conference paper at ICLR 2026
Xiwen Wei, Guihong Li, and Radu Mareulescu. Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation, November 2024.
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, al. Robust fine-tuning of zero-shot models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7959-7971, 2022.
Yichen Wu, Hongming Piao, Long-Kai Mai Huang, Renzhen Wang, Wanhua Li, Hanspeter Pfister, Deyu Meng, Kede Ma, and Ying Wei. S-LoRA: Scalable Low-Rank Adaptation for Class Incremental Learning, January 2025.
Jianxiong Xiao, James Hays, Krista Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference computer vision and pattern recognition, pp. 3485-3492. IEEE, 2010.
Yibo Yang, Xiaojie Li. Zhongzhu Zhou, Shuaiwen Leon Song, Jianlong Wu, Liqiang Nie, and Bernard Ghanem. CorDA: Context- Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter- Fine-tuning, March 2025.
Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, and You He. Boosting Con- tinual Learning of Vision- Lasion-Language Models via Mixture-of Mfxterr-of-Experts Adapters. In 2024 IEEÉ/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 23219-23230, Seattle, WA, USA, June 2024. IEEE. ISBN 979-8-3503-5300-66 doi: 10.1 22211099CCPP5533..224400200
Maxime Zanella and Ismail Ben Ayed. Low-rank few-shot adaptation of vision-language models, 2024. htttttttt//xiiiii org hbceg//bs/24400
Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. Slea: Slow learner with classifier alignment for continual learning on a pre-trained model, 2023. URL https
//arxiv.org. abs http://bbb2333...03 2311.
Zhen Zhao, Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Rethinking Gradient Projection Continual Learning: Stability / Plasticity Feature Space Decoupling. In Proceedings of the IEEE/VVF Conference on Computer Vision and Pattern Recognition, pp.
3718-3727, 2023.
Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xiangyu Yue, and Yang You. Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models, August 2023.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A DATASET & IMPLEMENTATION DETAILS
A.1 BENCHMARKS AND METRICS.
Our primary evaluation is conducted on the Multimodal Task Incremental Learning (MTIL) bench- mark (Zhng et al. 2023), which requires model sequentially train 11 tasks while main- taining CLIP's zero-shot performance. The benchmark comprises Aircraft (Maji et al. 2013).
101 Fei al.
2004), CIFAR100 (Krizhevsky al. 2009). DTD (Cimpoi al. 2014).
EuroSAT (Helber et al.
2019). Flowers (Nilsback & Zisserman 2008), Food (Bossard et al. 2014).
MNIST (Deng] 2012). OxfordPet (Parkhi al., 2012), StanfordCars (Krause al. 2013). and
SUN397 (Xiao et al. 2010). Appendix Table 5 summarizes the number of classes, training/test
images, and recognition type for each task. highlighting the scale and heterogeneity of MTIL. We
report three metrics after completing the entire sequence:
Transfer: Measures zero-shot transfer capability. After training task I, we evaluate on the test sets of all future, unseen tasks /. + ++,,-... 1,...,, 11 and average the results.
Avg.: The mean test accuracy across all 11 datasets, recorded immediately after training on each task.
Last: The mean test accuracy of the final model on each task's test set, capturing perfor- mance degradation (forgetting).
Table 5: Dataset statistics for the MTIL benchmark (Zheng et al. 2023). The 11 tasks span diverse recognition problems and domains, from fine- ine-grained aircraft and series to satellite imagery and scene categorization.
Dataset
# classes
train
test
Recognition task
Aireraft (Maji al. 2013
100
3334
3333
aircraft series
Caltech101 Fer-Fer el al. 2004
101
6941
1736
real-world objects
CIFARIO0 (Knzhessky el al.
2009
100
50000
10000
real-world objects
DTD (Cimpor ct al. 2014
47
1880
1880
texture recognition
EuroSAT (Helber al. 2019
10
21600
5300
satellite scenes flower species food types digit recognition
Flowers [Nilsbak & Zisserman
2008
102
1020
6149
Food (Bossard el al. 2014)
101
75750
25250
MNIST 2012)
10
60000
10000
OxfordPet (Parkhi et al. 2012
37
3680
3669
animal species series scene categories
StanfordCars KKruse et al.
2013
196
8144
8041
SUN397 (iio et al.. 2010
397
87003
21751
Total
1201
319352
97109
To test for long-sequence scalability, also evaluate on the standard CIFARIO0 dlss-inccremenal splits (Krizhevsky et al.] 2009). To ensure comparability with prior work, we adopt the same protocol and report the baseline numbers in ZSCL (Zheng et al. 2023). The 100 classes are grouped into 10, 20, or 50 tasks (10, 5, or 2 classes per task respectively). As the number of tasks increases, catastrophic forgetting becomes more severe. Complementary comparisons to CLIP-based PEFT continual-tume -uniing methods such as DIKI and MoE-Adapte are provided on the MTIL benchmark, which is their primary evaluation setting, while CIFAR100 serves here as long- CIL stress test.
A.2 IMPLEMENTATION DETAILS.
All experiments use the CLIP ViT-B/16 backbone with adapters applied to every query, key, value, and output projection. For the methods re-implemented on the CLIP architecture, we follow the CLIP-LoRA implementation (Zanella & Ayed 2024). For a rigorous and fair comparison across all LoRA like methods (LoRA, MiLoRA, InfLoRA, and NuSA-CL),
implemented them within an identical framework: adapters were applied to both vision and text encoders with a consistent rank, and the updated weights merged into the backbone after each task.
14

---

Published as a conference paper at ICLR 2026
Table Accuracy (%) of our method on the MTIL benchmark (5-shot, 500 iterations). Metrics for the Transfer, Last and Avg. are shown in the rightmost column.
StanfordCan Cars
Aireralt
Caltee1101
CIFARIO0
EuroSAT
Flowers
MNIST
OxfordPet
SUN395
DED
Food
Metric
Transfer
88.2
67.5
43.3
55.8
65.3
86.2
62.8
84.9
62.2
64.7
68.1
Aircraft
33.4
88.2
68.2
44.4
56.2
66.1
87.4
57.3
85.9
63.9
65.8
Ca
28.6
91.8
41.9
53.1
82.9
72.6
62.1
CIFARI00
31.1
91.4
76.1
43.7
57.3
70.2
86.9
62.5
86.1
62.1
66.7
DTD
30.0
90.9
75.6
62.1
56.2
68.7
87.0
66.4
87.5
62.5
66.0
EuroSAT
29.2
90.8
75.1
62.0
82.4
68.2
86.3
66,2
86.6
62.2
65.2
Flowers
28.2
91.1
74.4
61.5
80.9
88.1
86.6
65.2
86.7
62.0
64.9
Food
28.1
90.8
60.9
86.7
88.6
86.6
64.3
MNIST
27.7
90.6
73.9
60.8
81.0
87.2
88.5
90.7
87.1
61.9
64.4
OxfordPet
60.1
StanfordCars
27.0
89.8
73.6
60.1
80.0
86.5
88.3
90.6
91.7
69.1
64,1
SUN397
27.2
90.2
74.0
59.7
81.3
85.9
88.9
90.2
92.0
69.1
71.2
75.4
Average
28.9
90.5
73.2
56.1
71.9
76.9
87.2
72.9
86.8
63.5
65.3
70,3
Table 7: Accuracy (%) of method on the MTIL benchmark (full dataset, 1000 iterations). The rightmost column shows the overall Transfer, Last, and Average metrics.
StanfordCars
Airenat
Caltchh011
CIFARIO0
ESroSAT
Flowers
MNISTE
OxfordPet
SUN391
DTD
Food
Metric
Transfer
88.3
66.8
44.0
55.5
67.9
85,8
66.7
84.8
60.7
65.2
68.6
Aircraft
49.7
88.3
67.5
44.3
53.6
69.9
88.3
58.0
87.7
63.2
65.5
Caltech101
42,9
96.7
66.0
42.3
53.2
64.5
85.2
83.9
63.8
CIFAR100
41.0
96.1
82.2
45.4
59.8
69.2
86.2
73.7
85.1
61.6
66.4
74.2
EuroSAT
39.9
95.6
81.0
73.6
97.0
67.9
85.0
69.7
84.3
60.6
65.1
Flowers
38.8
95.7
80.6
72.1
96.9
96.4
84.8
69.9
82.9
60.1
64.7
Food
38.6
95.5
80.7
73.5
96.9
95.9
91.1
68.7
85.4
60.3
65.7
MNIST
34.3
95.9
79.8
72.4
96.6
95.9
91.1
98.9
84.9
59.6
65.7
OxfordPet
33.2
95.9
79.6
72.3
96.6
95.2
90.6
98.9
94.8
59.0
64.4
StanfireCarr
33.2
95.6
79.7
71.8
96.6
94.6
90.6
98.9
95.1
79.2
64.9
SUN397
35.0
95.2
79.4
71.5
96.2
94.0
90.6
95.0
78.1
76.9
82.8
Average
38.9
95.2
78.0
64.9
81.7
82.9
88.1
78.4
87.6
64.0
66.3
75.1
We the AdamW optimizer (learning rate x 10-4, weight decay 10-2, 0.9, 92 0.999), with linear learning rate warmup for the first 5% of iterations followed by cosine decay schedule. For NuSA-CL, we identify the null-space using a cumulative energy ratio cutoff (99% for 5-shot, 95% for full-shot) and cap the maximum rank at
= 128.
In the 5-shot MTIL setting Yu et al. 2024), we sample five examples per class and train for 500 iterations with a batch size of 16. We scale each low-rank update by a /r with = 1, apply dropout rate 0.25 to the adapter branches, and use label smoothing of 0.2. In the full-shor setting, we use all training samples, train for 1000 iterations, and use a scaling factor of = 2, keeping all other hyperparameters identical, All experiments were conducted on a single NVIDIA RTX 3090 GPU. In the CIL setting, use learning rate of 3 x 10 maximum rank Tmax 256, dropout of 0.05, batch size 128, and no label smoothing.
A.3 BASELINES.
We evaluate NuSA-CL against baselines along two complementary axes: (i) whether additional storage is required, and (ii) whether the backbone parameter budget is fixed across tasks.
Under the storage criterion (Table []. methods are divided into storage-based and storage-free approaches. Storage-based methods include ZSCL (Zheng et al.] (023), which uses knowledge distillation, MoE-Adapters (Yu et al.) (224)). DIKI (Tang et al. 2024), and InfLoRA (Liang & Li 2024). Although originally proposed for ViTs, adapt infloR'ss gradient projection memory
15

---

Published as a conference paper at ICLR 2026
Table 8: Transfer, Avg., and Last (%) for continual learning methods the Order-2 sequence of the 5-shot MTIL benchmark. f indicates methods re-implemented on the CLIP architecture.
Cara
Food
MNIST
OxfordPet
Flowars
SUN397
Aircralt
Caltesh111
DTD
EumSAT
CIFAR100
AWG.
Zero-soot
64.7
88.5
59.4
89.0
71.0
65.2
24.3
88.4
44.6
54,9
68.2
65.3
Transfer
LoRA! HH
2021
MiL. Wang
2025
88.1
62.9
87.4
87.4
62.6
18.3
86.8
41.0
45.1
59.2
61.4
InfLoRA
Liung
2024
88.2
58.8
84.1
65.4
65.4
20.7
87.7
44.2
49.3
66.9
62.9
NuSA-CL (Ours)
87.6
60.0
86.3
65.8
63.8
21.9
88.3
43.6
53.8
68.3
63.9
Avg.
LoRAT et 2021
86.5
84.3
66.3
MiLoRAT Wang 2025
51.0
76.7
83.8
81.7
71.7
64.3
19.2
87.7
44.1
50.7
60.1
62.8
InfLoRA
Linng
2024
65.3
85,5
85.3
85.6
80.4
67.1
25.3
89.3
48.5
54.4
67.6
68.6
NuSA-CL (Ours)
66.3
87.6
84.1
89.9
78.5
67.3
27.1
89.4
47.5
58.5
68.9
69.6
Last
LoRAT
Hu et al.
2021
46.7
76.7
89.3
82.1
71.3
67.9
23.7
90.0
59.1
72.4
73.3
68.4
MiLoRAT
Wang
2025
28.6
63.1
69.5
73.0
58.2
61.6
14.0
87.7
49.1
71.6
69.0
58.7
InfLoRAt
Liang & 2024
60.4
83.3
90.0
86.6
87.1
69.6
30.7
91.2
59.4
76.7
73.7
73.5
NoSA-CI CL.
(Ours)
64.6
86.7
89.4
91.7
84.7
70.1
32.7
91.8
58.0
79.2
75.1
74.9
Table 9: Transfer, Avg., and Last (%) for continual learning methods the Order-2 sequence of the the full dataset MTIL benchmark. indicates methods re- re-implemented on the CLIP architecture.
Cars
Food
MNIST
OxfordPet
Flowars
SUN391
Aitctaft
Callech101
DTD
EumSAT
CIFAR100
AVG.
Zero-shot
64.7
88.5
59.4
89.0
71.0
65.2
24.3
88.4
44.6
54.9
68.2
65.3
Transfer
LoRA! Hu
2021
MiL.RAT Wang 2025
88.2
61.9
87.7
63.3
64.5
19.1
87.6
47.0
63.4
62.6
62.6
InfLoRA
Liang
2004
88.2
62.1
86.7
65.2
66.0
20.9
88.3
45.1
65.2
63.4
63.4
NuSA-CL (Ours)
88.1
57.7
87.0
66.2
64.8
21.9
89.2
49.4
66.9
63.4
63.4
Avg.
LoRA Hu et al.
2021
71.6
85.2
91.8
91.4
77.9
70.2
18.6
91.5
53.3
51.0
60.9
69.4
MiLoRAT Wang
2025
67.0
85.3
92.3
90.9
76.0
69.6
20.2
90.8
51.8
55.9
65.3
69.6
InfLoRA
Liang & 2024
80.0
89,2
92.4
92.2
82.8
71.9
30.8
91.5
53.5
55.4
67.0
73.3
NuSA-CL (Ours)
73.8
89.7
91.2
92.1
84.4
70.8
32.9
91.6
51.3
58.0
68.2
73.1
Last
LoRA!
Hu et al. 2021
52.5
78.1
96.8
91.9
72.5
71.6
3.9
97.0
73.4
84.7
84.3
73.3
Mi
Wang el al.
2025
42.5
77.1
97.7
87.5
65.6
69.6
13.2
96.1
72.5
93.2
84.2
72.6
InfLoRA
Liang & 2024
75.1
87.2
98.4
93.6
88.1
75.5
30.7
96.4
74,8
95.5
84.7
82.2
NoSA-CL CL (Ours)
64.6
98.4
93.4
93.9
75.8
44.8
95.8
72.5
96.4
80.9
82.9
mechanism to the multimodal CLIP architecture for conceptual alignment. Storage- methods include Continual-FH which naively fine-tunes all parameters, the standard LoRA (Hu et al. 2021), MiLoRA (Wang et al. (002). an LLM adaptation method we implement on CLIP due to the relevance of its subspace initialization strategy, and proposed NuSA-CL.
Under the backboebc PEFT setting (Table 2 we compare LoRA-like methods implemented within identical framework ensure controlled and fair comparison.
16

---

Published as a conference paper at ICLR 2026
Table 10: Per-rank results by subspace type. Lower is better for Forgetting: higher better for Target.
Rank
Subspace
Forgetting (%)
Target (%)
Tail
1.46
78.05
32
Top
3.30
79.66
Random
2.08
78.70
Tail
1.91
81.99
64
Top
3.69
82.96
Random
3.08
82.77
Tail
2.57
85.22
128
Top
4.44
85,50
Random
4.57
85.49
Tail
3.85
86.46
196
Top
4.65
86.76
Random
6.15
86.67
Tail
5.11
87.34
256
Top
5.36
87.46
Random
6.23
87.44
B ADDITIONAL EVALUATION RESULTS
B. MTIL COMPLETE RESULTS
Table 6 reports the results all eleven tasks in the 5-shot MTIL regime (500 iterations per task). Our method maintains strong zero-shot retention (Transfer = 68. 18.1%) while achieving an Average accuracy of 70.3% and Last accuracy of 75.4%. The per-dataset breakdown confirms that our method uniformly preserves performance: no task suffers dramatic collapse, and gains over baselines appear across both domain-shifted benchmarks (e.g.. EuroSAT, Flowers) and in-domain benchmarks (e.g.. CIFAR 100, MNIST). Table presents the analogous results in the full-shot regime (1000 iterations per task), showing the same pattern of robust Transfer, Avg., and Last scores.
B.2 MTIL ORDER-2 RESULTS
To assess sensitivity to task ordering, T Table report 5-shot and full-shot MTIL results, respectively, under different task sequence (Order-2). In both settings, our method again achieves the highest Transfer, Average, and Last metrics, matching the original ordering (Order-1), Crucially, Last accuracy remains above 80% even in this permuted protocol, confirming that our method mitigates forgetting regardless of task order. This order-agnostic stability underscores the general applicability of our approach.
C FURTHER ABLATION
This section provides detailed results for the analysis in Section 6 Table I0 presents the numerical data corresponding to the observations in Figure Table lists the specific dimension numbers plotted in Figure Table (2 further reports the effective ranks and null-space fractions of NuSA-CL. on the 50-step CIFAR- CIL benchmark, illustrating that the spectra remain stable even after long-horizon continual updates.
D LLM USAGE STATEMENT
We disclose that Large Language Models (LLMs) were utilized as an auxiliary tool in the preparation of this manuscript. The use of LLMs was limited to writing assistance, such improving the clarity of English expressions, correcting grammatical errors, and refining sentence structure for better readability. Specifically, LLMS employed to rephrase complex sentences for conciseness and to
17

---

Published as a conference paper at ICLR 2026
Table 11: Effecttive rank r95 and corresponding null directions (Null@95= r95) for CLIP, LoRA and NuSA-CL across encoders and projection parameters. For CLIP, we report spectra before any MTIL task is learned. For LoRA and NuSA-CL, we report spectra measured the beginning of the final MTIL task (Task 11), after sequentially training on all preceding tasks.
Method
Encoder
Param
Nu
q
279.67
232.33
k
284.83
227.17
TEXT
311.33
200.67
311.17
200.83
CLIP
q
354.08
413.92
k
358.42
409.58
VISION
432.75
335,25
447.42
320.58
q
279.75
232.25
k
284,92
227.08
TEXT
311.58
200.42
311.33
200.67
LoRA (Task 11)
354.17
413.83
358.75
409.25
VISION
432,92
335.08
447.58
320.42
q
282.58
229.42
288.25
223.75
TEXT
V
316.67
195.33
318.75
193.25
NuSA-CL (Task 11)
D 357.75
410.25
K 362.42
405.58
VISION
438.83
329.17
454.42
313.58
Table 12: Spectral stability of NuSA-CL on CIFAR-100 CIL (50 steps). We report the effective rank ras and the fraction of the approximate null space (Null ratio (d r/s)/d) measured the beginning of Step (before any continual updates) and at the beginning of Step 50 (after learning the first 49 steps).
Encoder
Param
T95 (Step
795 (Step 50)
Null ratio (Step
Null ratio (Step 50)
q
279.67
279.67
45.38%
45.38%
k
284.83
285.00
44.37%
44.34%
TEXT
V
311.33
311.83
39.19%
39. 39.10%
O
311.17
311.67
39.23%
39.13%
q
354.08
354.17
53.90%
53.88%
K
358.42
358.50
53.33%
53.32%
VISION
432.75
432.75
43.65%
43.65%
447.42
447.67
41.74%
41.71%
receive suggestions for appropriate vocabulary fitting an academic tone. All scientific contributions including the research ideas, experimental design, analysis of results, and drawing of conclusions were made entirely by the authors. The LLMs did not influence the originality or the core scientific content of this work.
18


Reviews:
[{"review_id": "XmpnAbgAFl", "reviewer": "Reviewer_JRHi", "summary": "This work proposes NuSA-CL, a lightweight memory-free continual learning framework to address distributional shifts/novel tasks during real-world CLIP usage. NuSA-CL employs low-rank adaptation and null space constraint via SVD to constrain the update of parameters in a way that has low influence on previous knowledge. The experiments demonstrate its effectiveness among memory-free methods.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The theoretical motivation is clear and reasonable.\n2. The experiments demonstrate a favorable result among memory-free methods. The analysis and discussions are comprehensive, showing the method's practicality and robustness.", "weaknesses": "1. Storage-free baseline models are not that strong. Then a question is: using null-space may constrain the model expressivity. Is it still able to trade off by increasing the memory cost to get a stronger performance? Or the null-space also constrain the performance upperbound?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Nf1oS3Wboa", "reviewer": "Reviewer_4o99", "summary": "The paper proposes NuSA-CL, a memory-free continual learning method for VLMs (e.g., CLIP). It identifies a low-energy “null” subspace via SVD before each task, constrains low-rank updates to this subspace during training, then merges updates into the backbone to avoid parameter growth. Experiments on MTIL (full/5-shot) and CIFAR-100 CIL show strong performance and efficiency; analyses include rank/null-space dynamics, subspace/rank ablations, and core mechanism ablations.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Simple, natural idea: SVD-based separation of principal vs. null-like directions; persistent constraint prevents drift and reduces interference.\n2. Truly memory-free and fixed-size: no replay, no task-specific modules; efficient and scalable.\n3. Strong empirical results with comprehensive cost reporting; SOTA among storage-free baselines and close to storage-based methods.\n4. Clear, informative ablations (Top/Tail/Random; rmax; persistent constraint; multimodal adaptation) and mechanism evidence (effective-rank, null-ratio trajectories).\n5. Practical and robust: negligible SVD overhead; stable across energy thresholds; single-GPU training.", "weaknesses": "1. Theory is mostly motivational; tighter links between parameter-space bounds and function-level forgetting would help.\n2. Long-horizon spectral drift and null-space quality: Although the method re-computes SVD per task, after many merges the spectrum will evolve. A more systematic study of whether low-energy subspaces gradually become “contaminated” (especially for highly correlated task sequences) and how to monitor/remedy this (e.g., periodic re-orthogonalization, spectral gating) would be valuable.\n3. Merged updates limit reversibility; lightweight selective rollback is not explored.\n4. Scalability guidance for larger backbones (ViT-L/H) would aid practitioners.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Lij0l9Amby", "reviewer": "Reviewer_SL2R", "summary": "NuSA-CL addresses continual learning (CL) by balancing stability (retaining old knowledge) and plasticity (adapting to new tasks). The core idea is a low-rank adaptation strategy that constrains task-specific weight updates to the model’s approximate null-space, minimizing interference with previously learned knowledge and preserving the model’s zero-shot learning ability. Unlike methods that rely on replay buffers or high-cost distillation, NuSA-CL requires minimal computation and memory, making it suitable for resource-constrained scenarios. Experimental results demonstrate that the framework not only maintains strong zero-shot transfer performance but also achieves competitive results on continual learning benchmarks.", "questions": "1. Q: In class-incremental tasks, NuSACL has not been compared with other PEFT approaches such as LPI, TAM, MoELora, or DIKI. How does it perform relative to these methods in terms of both accuracy and memory efficiency?\n2. Q: While NuSACL improves stability via orthogonal projection, catastrophic forgetting may still occur in challenging incremental protocols (e.g., many classes, highly overlapping features). Have the authors quantified forgetting under these extreme settings?\n3. Q: Although the method is described as lightweight, how does it scale to large models (e.g., ViT-L/14 or multimodal transformers) where repeated SVD computations might be expensive?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Introduces the novel idea of updating in null-space directions (Tail), which is conceptually different from previous approaches that focus on Top singular directions or random subspaces. Combines multimodal adaptation and rank-limited updates, which is a creative integration addressing the stability–plasticity trade-off. \n2. Significance Tackles catastrophic forgetting, a central challenge in continual learning. Demonstrates practical low-cost, robust implementation suitable for large-scale models and multimodal tasks. \n3. The paper is well-structured, with clear motivation, methodology, and empirical validation. Extensive ablation studies and visualizations (e.g., subspace and rank analysis) make the mechanisms and design choices transparent. \n4. Strong experimental evaluation, including comparisons with Top/Random subspaces and analysis of rank choices. Shows robustness to hyperparameters and demonstrates practical feasibility with low SVD overhead.", "weaknesses": "1. The method assumes that tasks have sufficiently distinct distributions, enabling interference reduction via orthogonalization or projection. However, when tasks are highly correlated or share overlapping features, the model may struggle to separate old and new knowledge, leading to reduced stability and adaptability.\n2. The experimental evaluation is limited to standard vision benchmarks (e.g., CIFAR, ImageNet subsets) with relatively few tasks, focusing only on class-incremental learning. The absence of experiments on more complex multimodal settings such as VQA or image-text retrieval limits the demonstration of generalizability.\n3. The method’s performance is sensitive to key hyperparameters, including projection dimension, update ratio, and orthogonalization strength. The lack of an adaptive tuning mechanism may hinder robustness and scalability in real-world, dynamic scenarios.\n4. In class-incremental tasks, the method has not been compared with other Parameter-Efficient Fine-Tuning (PEFT) approaches such as LPI, TAM, MoELora，or DIKI, limiting the completeness of its performance validation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hvtEqvW6sx", "reviewer": "Reviewer_TeKM", "summary": "This paper proposes NuSA-CL, a memory-free continual learning method that leverages orthogonal weight modulation to mitigate catastrophic forgetting. The approach updates model parameters only along directions orthogonal to previously learned subspaces, thus avoiding interference with past knowledge without storing any exemplars. Experiments on several vision and vision-language benchmarks show that NuSA-CL performs competitively among storage-free approaches but lags behind replay-based methods.", "questions": "1.\tWhat prevents NuSA-CL from surpassing any storage-based methods in performance, despite its theoretically clean orthogonal design?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tMemory-free design.\nThe method achieves continual learning without any replay or exemplar storage, offering a clean and theoretically motivated direction for efficient CL.\n2.\tStrong performance among storage-free models.\nWithin the memory-free setting, NuSA-CL demonstrates solid results and stable learning behavior across multiple benchmarks.", "weaknesses": "1.\tLimited performance.\nThe method performs worse than all storage-based models, showing limited competitiveness in practical CL scenarios.\n2.\tSimplicity vs. effectiveness.\nThe proposed approach is simple, but its results do not convincingly show that such a minimal mechanism can achieve strong continual learning.\n3.\tScalability concern.\nAs also noted by the authors, under extreme lifelong settings the null-space directions may saturate. Without a clear advantage in scalability over storage-based methods, the claimed benefit of being memory-free remains modest.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)