\textsc{PGO-BEN}: Proxy-Guided Orthogonalization and Beta Ensembling for Few-Shot Domain-Incremental Learning
Authors: Samrat Mukherjee
Thivyanth Venkateswaran
Eric Nuertey Coleman
Luigi Quarantiello
Julio Hurtado
Vincenzo Lomonaco
Gemma Roig
Subhasis Chaudhuri
Biplab Banerjee
Venue: Accepted by TMLR ()

Abstract: Continual adaptation to evolving domains with minimal supervision is essential for real-world deployment of machine learning systems. We formalize this objective as \textbf{Few-Shot Domain-Incremental Learning (FSDIL)}, where a model must adapt to each new domain using only a few labeled samples while retaining prior knowledge without access to previous data. This setting mirrors practical constraints in domains such as autonomous driving and medical imaging, where annotations are expensive and data retention is restricted by privacy regulations.
Pre-trained vision–language models such as CLIP provide a strong initialization for FSDIL due to their transferable multi-modal representations. However, adapting CLIP incrementally under domain shifts remains challenging: few-shot updates often trigger \emph{catastrophic forgetting} and insufficient \emph{plasticity} across evolving distributions.
To address these challenges, we introduce \textbf{\textsc{PGO-BEn}} (\textit{Proxy-Guided Orthogonalization and Beta Ensembling})—a rehearsal-free framework that leverages CLIP’s semantic priors via prompt learning while preserving prior domain knowledge through two key mechanisms. 
(1) \textbf{Proxy-Guided Orthogonalization (PGO):} identifies conflicts between current gradients and proxy representations of past knowledge, inferred from current samples, and projects conflicting updates into an orthogonal subspace to prevent knowledge degradation.  
(2) \textbf{Beta Ensembling (BEn):} introduces a Beta-function-based temporal ensembling strategy that adaptively balances stability and plasticity, outperforming conventional exponential moving average (EMA) approaches in retaining early-domain knowledge.
We extensively evaluate \textsc{PGO-BEn} on three diverse benchmarks—\textbf{DomainNet}, \textbf{CoRE50}, and \textbf{CDDB-Hard}—and demonstrate consistent improvements over state-of-the-art domain-incremental and few-shot learning methods across all supervision levels in this challenging setting.

Published in Transactions on Machine Learning Research (01/2026)
PGO-BEN: Proxy-Guided Orthogonalization and Beta En- sembling for Few-Shot Domain-Incrrmental Learning
Samrat Mukherjee
23d15990kttboc..
Centre for Machine Intelligence and Data Science, Indian Institute of Technology Bombay
Thivyanth Venkateswaran
thémyanth whith. in
Department of Engineering Physics, Indian Institute of Technology Bombay
Eric Nuertey Coleman
eric.colemonntthd.com unipi.
Computer Science Department, University of Pisa
Luigi Quarantiello
luigi. htt..-quuanntielo@@@dd.....ccc unipi.
Computer Science Department, University of Pisa
Julio Hurtado
julio.hurtado@uuavvic..ccc
Centre for Applications of Mathematical & Computing Sciences, University of Warwick
Vincenzo Lomonaco
vlomonacoBluiss. it
Department of AI, Data and Decision Sciences, LUISS
Gemma Roig
roignoguero@em wtw.:/mmmmiiiionffttttt.........
Deportment of Computer Science, Goethe University, Frankfurt The Hession Center for Artificial Intelligence (hessian.Al), Darmstadl, Germany
Subhasis Chaudhuri
sc ww..oe.iith.comm www.hitthhc....... in
Department of Electrical Engineering, Indion Institute of Technology Bombay
Biplab Banerjee
bbanerjee Qrith. in
Centre for Machine Intelligence and Data Science, Indian Institute of Technology Bombay
Reviewed on OpenReview: https: opw.reveee.eeeeeemmmme/ net/ forum? id= fLb27FbHLy
Abstract
Continual adaptation to evolving domains with minimal supervision essential for real- world deployment of machine learning systems. We formalize this objective Few-Shot Nomain-Incremernn Learning (FSDIL), where model must adapt to each new domain using only few labeled samples while retaining prior knowledge without access to previous data. This setting mirrors practical constraints in domains such as autonomous driving and medical imaging, where annotations are expensive and data retention is restricted by privacy regulations. Pre-trained lasion-l models such as CLIP provide a strong initialization for FSDIL due to their transferable multi-modal representations. However, adapting CLIP incrementally under domain shifts remains challenging: few-shot updates often trigger catastrophic forgetting and insufficient plasticity across evolving distributions. To address these challenges, introduce PGO-BEN (Prozy-Guided Guided Orthogonalization and Beta Ensembling)- rehearsal-free framework that leverages CLIP's semantic priors via prompt learning while preserving prior domain knowledge through two key mechanisms. (1) identifies conflicts between current gradients
Proxy-Guided Orthoogonalizatoon(PPC and proxy representations of past knowledge, inferred from current samples, and projects conflicting updates into an orthogonal subspace to prevent knowledge degradation. (2) Beta Ensembling (BEN): introduces Beta-function--asee temporal ensembling strategy

---

Published in Transactions on Machine Learning Research (01/2026)
that adaptively balances stability and plasticity, outperforming conventional exponential moving average (EMA) approaches in retaining early-domain knowledge. We extensively evaluate PGO-BEN on three diverse benchmarks DomainNet, CoRE50, and CDDB- Hard-and and demonstrate consistent improvements over state-of-tho-art domain-incremente and few-shot learning methods across all supervision levels in this challenging setting. Code:
https:/ htt.:///w.oom/aamms/////000000. /t//////////////////////////////
1 Introduction
Large-scale annotated datasets have catalyzed major advances in machine learning. However, collecting such datasets remains expensive, privacy-sensitive, and logistically challenging across many domains, especially those involving dynamic environments or regulated data (e.g., autonomous driving or clinical imaging). These limitations have spurred growing interest in data-efficient learning paradigms such as semi-supervised learning (SSL) van Engelen & Hoos ) Yang et al. (2022) and few-shot lcarning (FSL) Ravi & Larocholle (017) Song el al. 2023), which reduce reliance on exhaustive manual annotation. Yet, these paradigms are typically designed for static data distributions and struggle in scenarios where models must continuously adapt
evolving environments, necessitating the study of continual learning (CL) Castro et al. (0088) Rebuffi et al.
(2017 Riemer et al. [2019] Wang et al. (022a)).
Within CL. three principal paradigms have emerged: class-incremental learning (CIL), where new classes are introduced over time; task-incremental learning (TIL), where tasks differ and are explicitly identified; and iamain-incrementall learning (DIL), where input distributions shift across episodes while the label space remains constant. In this paper, we focus on DIL, but challenge a common assumption the literature that each incoming domain offers abundant labeled data overlooking the realistic scenario where new domains often provide only limited supervision due to data collection cost or privacy restrictions. In practical settings such autonomous driving, robotic vision, and clinical imaging, models encounter sequential domain shifts (c.g., changes in lighting, device, or geography) but only limited labeled data per domain due to high annolation costs. In healthcare, for example, shifts across hospitals or scanners necessitate continual adaptation, yet privacy constraints prevent storing past data, while annotation costs restrict label availability Zhou et al. (2021).. Likewise, deepfake detection requires identifying continually evolving fake content, and anomaly detection demands recognizing rare events while preserving knowledge of previously encountered anomalies. These scenarios motivate studying DIL under minimal supervision enperisionn-earrning from contimually evolving domains with fixed label space yet this problem has been largely overlooked in existing literature.
Table 1: Comparison of continual learning settings. Our proposed setting (FSDIL) is highlighted.
Setting
Space
Domain Shift
Label Budget in Incremental Scesion
Task/Domala Available
Clase Increnacntal Leurinng (CIL)
Expanciing
Nu
Pull supervision
No
Fon-Shot Cless Incrmmrral Teorning (FSCIT.)
Experrinng
No
Fru-shot
No
Domain Incremental Leorning (DIL)
Fined
Yes
Full supervision
No
Faw-Shhat Ennsiin-ncremnntall Learning (FSDIT.)
Fixed
You
Fow-shot
To bridge the gap between current benchmarks and woal-world applications, we introduce Few-Shot Domain- Incremental Learning (FSDIL) (Fig. 2a). In FSDIL, a model first trained on a well-annotated base domain and must continually adapt to stream of novel domains, each offering only a few labeled example
per class. FSDIL fundamentally differs from related paradigms (Table D]: unlike FSCIL Dong et al. (2021)
Tao et al. 2202 Sur et al. (2025), it assumess fixed label space with shifting domains; unlike Few-Shov Domain Adaptation (FSDA) Zhao et al. (2021). ) it requires continual rather than onc-time adaptation; and unlike unsupervised DIL variants Mukherjee et al. (2025); Rakshit et al. (2022). it leverages limited supervision in each new domain, better reflecting real-world constraints. The base session involves training with abundant data, mirroring the common industrial practice of pro-training on large datascts. To respect privacy constraints in applications such as clinical imaging, our proposed FSDIL setting prohibits storing exemplars from past sessions.
2

---

Published in Transactions on Machine Learning Research (01/2026)
Real
Zero-shot
(+3.18)
CLIP
Ours
Infogroohh
Painting (.5.7
(+5.39)
03.0
40.09
65.0
Quick Cef 66692 93)
75.85
Cliport
(+5.82)
68.12
Skatch (+3.75)
Figure 1: Comparison Zero-shol CLIP-VIT-B/16 with PGO-BEN model (1-shot) across all domains in the DomainNet dataset. Numbers in brackets indicate the performance gain of our method.
Although large-scale models like CLIP exhibit strong generalization, they are insufficient for all domains. We evaluate the frozen CLIP-ViT/B-16 model Radford et al. (2021) using the prompt "a photo of a on the Net dataset Peng et al. (0019), and compare with our model (Fig. []. The consistent. performance gap across domains highlights that large-scale pretrained models fail to adapt effectively to domain shifts, necossitating specialized strategies for continual adaptation.
FSDIL is particularly challenging for three reasons. First, supervision extremely limited: each new domain provides only a few labeled samples, making standard optimization prone to overfitting and variance collapse. Existing DIL methods like S-Prompt Wang et al. (2022a) and CP-Prompt Feng et al. (2024) fail under this regime, as their KNN-based domain prompt selection becomes biased with scarce data, leading to poor generalization. Second, sequential domain shifts occur without task boundary annotations, rendering exemplar- based or task-specific projection methods infeasible due to privacy, memory, or latency constraints. Third, large stylistic shifts across domains (c.g., photo - sketch) demand representations that remain label-consistent yet robust. visual variations.
Base Session
Beta
Exponential Moving Average (EMA)
R=B
Moving Average (BMA)
Abundaut samples per cliss
(Unnormalizee iandd HaamaaiiieeWWiigeeee Weightage of cach model during adaptation steps
Incremental Sessions:
before adapting
after adapting to domain
Training
Model Mr
Few-shht samples per class
domain
Reta distribution hased ensembling
Incromental session
better stability tradeotf
Almust forgets price domain info.
betwoon modills
Gradient distilliiiio
Training
Model
Model
M
Adaplation steps
Adaptation steps
(a) Few-shot
Dommin--Incrmmentl Learning task.
(b) Proposed 3-function-based temporal ensembling for to improve knowledge retention.
Figure 2: (a) FSDIL task. A well-labeled base domain is followed by sparsely labeled incremental domains. Combined domain shift and label sparsity risk overfitting and forgetting. (b) Beta Moving Average. BMA Beta distríbution to adaptively weight model states, retaining prior domain knowledge, whereas EMA overly discounts earlier states and risks forgetting prior knowledge.
3

---

Published in Transactions on Machine Learning Research (01/2026)
Given this setting, our work investigates three key questions: (Q1) How to design data-efficient adaptation strategy that generalizes well to new domains despite extreme supervision sparsity? (Q2) How con we mitigate catastrophic forgetting in a streaming setup without relying on task-specifee gradient estimation or data replay?
(Q3) Can we build a unified, robust representation that generalizes across diverse domains continually?
Our solution. We propose PGO-BEN, a unified framework for FSDIL grounded in the semantic strength of the pre-trained vision language model CLIP Radford al. 20011. motivated by its recent success in DIL Feng et al. 2024 Wang et al. (2022a). The framework introduces three key innovations.
(1) Efficient adaptation from few samples: leveraging CLIP's few-shot capacity, we design a multi-modal prompting strategy that departs from prior prompt-tuming methods [hoou et al. (2022b a Wang et al. (202aa) Feng et al. (2024). Learnable prompts are injected across layers in both CLIP encoders, with lext prompts conditioned on visual prompts to effectively capture domain shifts from minimal supervision while exploiting CLIP's prior knowledge. (2) Mitigating catastrophic forgetting: we introduce prozy- prozy-guided orthogonalization, aligning update directions with prior knowledge via cosine-based filtering. Unlike subspace projection methods Farajtabar et al. (2020); Liang & Li (2024); Lin et al. (2022): Saha et al. (2021). it requires storage or approximation of past gradients crucial when only few labeled samples are available thus improving scalability as domains evolve. To further stabilize learning, a ceta-function-ber based Moving Average (BMA) replaces the standard EMA Caron et al. (2021) ); Carta et al. (2023), adaptively weighting model states through a symmetric Beta distribution to preserve early domain knowledge often lost with EMA (Fig. 2b). (3) Learning domainn-enneralied representations: moving beyond prompt-pool methods Feng et al. (2024): Wang et al. (2022a[c) that rely on -ppeii- selection at inference, we condition text prompts vision prompt tokens across layers. This enables dynamic adaptation to evolving visual styles, yielding domsin-aguostic prompting and improved generalization without inference-time selection. Collectively, these designs make PGO-BEN an inference-efficient and scalable FSDIL tramework-fiee from prompt memory, task identifiers, or exemplar buffers. In summary, this paper:
Formalizes FSDIL a realistic continual learning problem involving sequential domain shifts with few-shot supervision, while enforcing privacy-aware constraints by avoiding storage of past data.
Proposes PGO-BEN, integrating multi-modal prompting, gradient-aligned distillation, and Beta- based ensembling to improve the space remains fixed.
tradc-off where domains evolve but the label
Demonstrates
state-of-tho-art performance on three benchmarks Domainnet |Png et al. (2019). CoRE50 Lomonaco & Maltoni (2017). and CDDB-Hard Li al. 2023 with supporting a ablations.
2 Related Works
Prompt Learning in CLIP. Prompt learning has emerged as lightweight alternative to full fine-tuning,
originally developed for NLP Lester et al. (2021 [i & Liang (2021) Mishra et al. (2023) and later extended to vision-language models like CLIP Radford et al. (2021). In this context, CoOp Zhou et al. (002bb)
introduces learnable prompts, while CoCoOp Zhou et al. (2022a) makes them input-conditioned to improve generalization. MaPLe Khattak et al. (2022) enriches CLIP's features by injecting hierarchical modality-
prompts, and StyLIP Bose et al. (2024) further integrates domain-speciffc cues. We introduce learnable prompts across encoders, conditioning text prompts on vision prompts to more effectively capture and adapt. to visual domain shifts.
Data-Efficient Continual Learning. CL aims to balance stability-plastiiity tradeoff Wang et al. (024)) methods like EWC Kirkpatrick et al. (0017), baply strategies like RL Rebuffi et al. (0177) and ER Riemer et al. (2019), and parameter-isolation techniques Wang et al. (2023) address this trade-off with varying overhead. In domain--nccemmnntal learning, where task IDs are absent, GEM Lopez-Paz & Ranzato (2017) and latent-replay Pellegrini et al.] (2020) help adapt to distribution shifts but rely on exemplar memory. Prompt-based continual learning methods such as S-Prompt |Wang let al. (2022a), CP-Prompt Feng et al. (2024) learn per-domain prompts and rely on domain-aware retrieval during inference, requiring abundant. labels and prompt selection mechanisms that increase fatency-making
Zhou et al. (2024). Regularization--base
4

---

Published in Transactions on Machine Learning Research (01/2026)
inference slower, quality that hinders application of FSDIL in real-world use cases. In contrast, FSDIL demands unified, domain-agnostic prompting strategy that adapts continuously without task-specific routing or per-domain memory. Its more restrictive setting lacking domain labels and extensive supervision -aalls calls for lightweight, generalizable prompting across evolving domains.
Gradient Projection in Continual Learning. To mitigate forgetting, methods such as OGD Faraj jabartaarr
iet al. 2 TRGP [inn et al. (0022), GPM Saha et al. (2021), and DualGPM Liang & Li (2023) constrain updates by projecting gradients onto orthogonal subspaces of prior tasks. While these leverage low-rank structures in gradient space, they require storing estimating task-specific subspaces an increasingly impractical demand as task count grows and data becomes scarce, as in FSDIL. In contrast, FSDIL necessitates scalable, memory-efficiet approaches that preserve knowledge without explicit gradient storage or subspace tracking.
3 Proposed Methodology
Problem Definition. Consider a sequence of N distinct domains, (Di.D2.... D.....,,.,.... where each domain D, during the training session t consists of tuples {æf, v/ Here, xf represents image, and y denotes the corresponding label, with vf E C = 02, 02,-.. .qe))- )- In particular, the set of classes C are identified from the beginning and will not change along the incremental learning stream Feng et al. (2024); Wang et al. (022a)), with |C] denoting number of distinct classes, and [Da] denoting number of samples in domain D,.
In the FSDIL setting, the initial domain Dj is characterized by a large number of training samples per class within the label space C. However, for each subsequent session t t> 1, the number of training samples is significantly less, following the incquality |D1] >> |Da] for all t 1. Specifically, we have Vt > 1, [D, [C] x n, where n is the number of samples per class in C.
During each incremental session, training is restricted only to samples from the current domain D,, with data from previous domains unavailable for reuse, adhering to an exemplar-free setup Wang et al. (2022a). We evaluate the model after every session, where in any given session t, the model's performance is tested across all domains encountered up to that point. The main objective is to train a model that adapts to each new domain with few labeled examples while retaining knowledge from previously seen domains.
3.1 PGO-BEN for Few-Shot Domain- Learning
To address the constraints of FSDIL generalizing with minimal supervision, adapting to distributional shifts across domains, and absence of task boundaries we build on the generalization capabilities of large-scale vision-language models. PGO-BEN is designed to exploit vision-language priors of CLIP-like models for FSDIL. Specifically, we leverage CLIP as the backbone for our proposed model M, formally defined as M = Jv. Pr, TOKr, TOKv), where Fr and Fv are the frozen text and vision encoders of CLIP, respectively. P P= ( ...., , denotes the Synergy module, a set of lightweight projector networks that modulate lcarnable tokens TOKT and TOKy at J intermediate layers in Text and Vision encoder respectively. To ensure semantic alignment of the text encoder representations under changing visual domains, we condition TOKr on corresponding TOKy via Encoder-Synerge module, enabling the text encoder to adapt to evolving visual domains and support domain-invariant representation learning. The unified learnable prompt representation Pr = [0a][12]- ... [fm][CLS) where [o:]'s are learnable tokens and [CLS] denotes the classification token, serves as input to Fr, avoiding manual domain-specific prompt tuning.
At each incremental session i, the model adapts to domain D. with updated parameters O, {Pr, Pr., TOKr,, TOKG, To effectively address FSDIL challenges, PGO-BEN comprises three syner- gistic components: (i) multi-modal prompting strategy that enables unified adaptation across visual and textual branches (Sec. 3.1.1): |); (ii) A proxy-guided orthogonal gradient update strategy, where the model trained till session - 1, M,-1 serves as a proxy for prior domains, ensuring that gradient updates for the current domain remain aligned to previously learned knowledge and do not cause forgetting. (Sec. (.1.2]; and (iii) a Beta function- basod temporal ensembling strategy that adaptively ensembles model states over time to enhance knowledge retention under domain shift without hampering the plasticity of the model (Sec. 3.1.3 Torether, these components allow PGO-BEN to retain prior knowledge while flexibly adapting
5

---

Published in Transactions on Machine Learning Research (01/2020)
Labels
Gradient directions
Mi
Text Encoder ( FT)
updation as per
Sppdato
Geart
Tokenizer
Enodder Lager X
Pr.
Uncoder Loper
Text fenture
C
Encader
Dot Prosbact
TOX
T0K1
Geurr
Ey Mileiee Prill
Naneconflicting uphles
madide
Few-shot samples
TOR
Image feature
IM Pr,, i
Gupdate = G
Eoooddr Lover &
Eroodde Layer
Geur
Encoder
Beta Moving Avenige
M
G
curt
Vision Encoder (Fy)
Conflicting pradient
Pr
Figure 3: Overview: PGO-BEN uses multi-modal prompt learning strategy to learn generalizable representation from few-shot examples observed in domain D.. To adapt to the current incremental domain Dr, > 1, PGO-BEN uses the labeled examples of current domain data guided by cross-entropy loss Lee- PGO-BEN compares the prediction of current model M, and previous model M on the current. data D; using CuI. PGO-BEN introduces an adaptive gradient direction selection strategy mitigate forgetting of knowledge of past domains guided by the proxy-knowledge of the past domains from Lui Since M has never seen Dr Beta-Moving Average ensembling technique is used to further enhance the slability-plasticity trade-off, while mitigating the risk of unreliable updates. Here P(yl-) denotes posterior prediction probability.
to new domains from limited samples. All learnable parameters are trained to obtain a domain-agnostic representation applicable for all seen domains, avoiding any domain-specific parameter selection steps prevalent in DIL. literature Wang et al. (2022a); Feng al. (2024 Fig. illustrates the process.
3.1.1 Multi-Modal Prompting for Domain-Invarian Representation Learning
Conventional prompt tuning for CLIP, such as CoOp and CoCoOp Zhou et al. (2022b a). adapts only the
text encoder, offering limited robustness to visual domain shifts. MaPLe Khattak et al. (002 enriches vision features with text representations, failing to capture evolving visual distributions in FSDIL, where supervision is sparse and domain relations across incremental sessions are undefined.
essential requirement
We propose a simple multi-modal prompting strategy, specifically for FSDIL, that conditions text tokens on visual cues across layers. Specifically, each transformer block CLIP's encoders includes learnable tokens TOK& and TOK,, linked via layer-specific light projector pi such that:
TOKY = PP
Since the feature representations evolve across transformer layers, we introduce layer-specific projectors rather than single shared projector, allowing flexible cross-modal alignment at different semantic depths. We conducted an ablation study on the number of layers the projector is needed to be introduced in Sup.Mat.. By propagating visual-domain cues into the text encoder, the model dynamically aligns its textual embedding space with evolving visual distributions, thereby enhancing domain invariance without explicit supervision (see Fig. Lap. We initialize this unified prompt space by training on the labeled base domain D1 using cross-entropy loss, enabling the model encode domain-agnostic semantics into the prompt tokens, supporting stable adaptation across subsequent domains with minimal supervision. Table X & Fig. 44] highlight the superiority of conditioning text prompts visual tokens which enables better adaptation to changing visual domains compared to other conditioning techniques Khattak et al. (2022) and baseline methods.
3.1.2 Retain While You Learn: Proxy-Guided Orthogonalization for Stability under Shift
Prior works Kirkpatrick et al. (2017): Saha CL. to gradient updates of the current task which overrides the knowledge of the past tasks in order to learn
(2021 9 Lin et al. 2022) attribute catastrophic forgetting in

---

Published in Transactions on Machine Learning Research (01/2026)
the current task Prior methods Liang & Li (2024); Saha al. (2021): p: Lin et al. (0022) Liang & Li 2023
address forgetting by projecting current task gradient updates orthogonally to pre-computed approximation of the gradient subspaces of the previous tasks. Such orthogonalization mitigates the risk of interference of knowledge of the past domains with gradient update of the current domain. Howover, approximating past gradient spaces scales poorly with an increasing number of lasks and require memory overhead Liang &
Li (2024): Lin et al. (2022). Under limited supervision constraint of FSDIL, such approximations become unreliable, leading to degraded performance. Motivated by Yu et al. (2020), we propose a memory-free ceasstrophicfforgettien by deriving proxy representation of prior domain knowledge from the few-shot samples of the current domain. This proxy knowledge is used to regularize the learning dynamics of the current domain, to preserve knowledge of previously seen domains without the need for explicitly storing pre-computed gradient spaces or approximaling gradients of past domains from limited data, unlike prior methods Liang & Li [0023] Saha et al. [2021]; Liang & Li (2024).
strategy that mitigates
At session > 1, we initialize the current model M₁ and prompt Pr. from the previous model M This provides an enriched initialization as compared to random
which has learned from (.,, initialization for every domain in Wang al. (2022a); Feng et al. (2024). While adapting Me to domain D. = ((1 v we use the frozen Ma-1 model as a functional proxy of knowledge of previously seen domains to regulate the direction of parameter updates. For each input x/, we compute:
D. "0....,,__11..
Cross-Entropy Loss for adaptation: Leo (il vt), where of Melad Pri)
KL Divergence to preserve past knowledge:
Mi(zi, Pr()
Cu = MM- ((e(,,PP-)))) Pri-1) log Mr-1(x;, Pre-1)
(1)
We compute gradients Geurr Vo, fee and Gprev Vo.-, Cxl, and calculate the angle between them:
( G Gprev Geurr [Grrov ||
cos
(2)
The angle & indicates whether the gradient update for adapting to the current domain is consistent with, conflicting against, the model's existing knowledge of previously seen domains. If the existing knowledge of the model align with the update direction (v E 1 }),, we hypothesize that updating the model following the update gradient Geure will not cause any forgetting (as, Glurr is along the direction of Gprev: see Fig. 3
In case of conflicting alignments(i.c. & É [- [1), updating the model parameters with Geurr will adapt the model on current domain D, at the cost of lörgetting the knowledge of the previously seen domains. In such scenario, we project Geure onto the orthogonal space of Gprev to obtain G and update the model using,
Gupdate
-1 Gouero otherwise
0, - 0. 17- Gupdate
Our adaptive-alignment strategy avoids storing the gradient-space information which has increasing memory overhead as we see more domains, we only maintain the prior model state M,-1, very standard practice in knowledge-distillatio literature. Unlike prior methods that constrain updates to fixed subspaces Liang & Li (2024); Lin et. al. (2022). we retain past. knowledge by dynamically adjusting update directions using Gprev as a proxy knowledge of prior domains. Updating the parameters in the orthogonal space of the previous gradient directions in conflicting scenarios, reduces the risk of forgetting prior domain knowledge. This implicit and scalable alignment maintains plasticity without. relying on unreliable gradient subspace approximations in low-data, regimes.
Benefits of our alignment strategy:
3.1.3 From EMA to BMA: Temporal Ensembling that Remembers
Since the model M,-1 has never been trained on domain Dt, relying solely on the gradient information maybe unreliable and could give sub-optimal performance in knowledge retention and may also hurt the
7

---

Published in Transactions on Machine Learning Research (01/2026)
plasticity. To avoid the ill-effects of unreliable gradient updates, temporal ensembling of model states. In CL. literature, FMA smoothing Carta et al. 2023);; Caron et al. (2021) b which is commonly used to stabilize
updates, as we observe in Table 3 underperforms under large, unconstrained domain shifts typical in FSDIL, due to the fixed decay nature (Fig. 2b)
We propose a more flexible Beta Moving Average (BMA) strategy that adaptively ensembles intermediate model states during training, based on the Beta distribution. Unlike EMA, which monotonically discounts early checkpoints, BMA assigns higher weight to both carly and late training phases. Early model states retain knowledge about the previously seen domains, where later model states have adapted to current domain, but have a risk of forgetting the prior knowledge. BMA improves stability and mitigating task-recency bias when under sparse labels and large domain shifts, better than EMA, thus being more suitable for FSDIL task, strengthening stability- plabtiity-paastcity dilemma (see Fig. 2 and Table. 3
Beta-function based Ensembling. During T" update steps on domain Dr, the model Mr, which is yet underfits Dz. In later iterations, the model adapts to D,, risking forgetting of prior knowledge. For better stability-plasticii trade-off, we treat the final model M, as weighted ensemble of intermediate models (Me) using A-function based ensembling
initialized with Mr-1, has knowledge about (Di,...; Dz-13 -
T"
aV
Me,
where ay (B))(
t' + 0.5
(4)
T +1
P=0
k= O
With 8 I, BMA highlights both early and late model states, enhancing memory of past domains while incorporating the current one (sce Fig. 2b
Efficient Online Implementation. In order to minimize memory overhead, rather than storing every intermediate state, we implement BMA as rumming average:
MBMA
k= an BMA
MBMA
Me.
(5)
Ek
Ex ax
This low-memory update requires only single auxiliary model state, offering temporal smoothing that complements gradient-base updates Shu et al. (2023). Each iteration of training applics proxy-guided orthogonal update followed by BMA integration to ensure stable adaptation across sessions.
During inference, we deploy the final BMA model MBMA for evaluations on all test samples across the domains (Di...., D and initialize M++1 for adaptation domain D441 with M and doesn't require any prompt-selection phase, resulting in efficient. inference. Pseudocode are provided in Sup. Mat.
3.2 On the Utility of PGO-BEN for FSDIL
To theoretically justify PGO-BEN in the FSDIL setting, employ PAC-Bayesian theory McAllester 11999 For model M, ~ (posterior over parameters 6,) adapted to domain D, with n examples, and a prior x (e.g,, CTIP-initialized 0n), the expected true risk Lo, (Mi) is bounded with probability 2 8 by:
(M. S (MMool((MMM) KL(pp (T) + log
2/z à
2n
This bound links true risk to empirical risk (M,,, the complexity term KL(pl(T), and sample size n. Contrastingly, the bound discussed in Shi & Wang (2023) relies on storing examples from previous domains in a memory buffer, contrasting our exemplar-free motivation. PGO-BEN's components aim to tighten this bound, achieving better adaptation to target domains:
CLIP Initialization as an Informative Prior x: CLIP initialization ensures π is centered in robust, generalizable region. For few-shot domains, the learned posterior p needs minimal deviation from æ minimize empirical risk, directly reducing the K (p)(TT term and tightening the bound.
8

---

Published in Transactions on Machine Learning Research (01/2026)
Gradient-Alignee Distillation for Posterior Stability: The gradient alignment mechanism (Sec 3.2.2) stabilizes p by constraining updates based on prior knowledge (MN-1). This prevents drastic parameter shifts, keeping closer to π and thus helping maintain a small (pllTT) This stability also limits sensitivity to few samples, reducing the appirical-to-expeetee risk gap and yiclding a more reliable (M())).
Beta Ensembling for Posterior Regularization: BMA (Sec 3.2.3) defines p as an implicit Beta-weighte mixture of intermediate model states. This averaging reduces estimator variance, leading to a more sta ble and potentially lower empirical risk E. (M,,)] Such ensembling also regularizes p. potentially finding flatter minima associated with better generalization and favorably impacting the complexity term.
Further formal and empirical discussions in this regard are provided in Sup. Mat.
4 Experimental Evaluations
Datasets. Following DIL literature Wang el al.
2022al:
Feng el al. (2024), we evaluate our method on three
DIL benchmarks: DomainNet Peng et al. (2019). CORe50 Lomonaco & Maltoni (2017), and EDDB-Hard Li
ct al. 2023). These datasets vary in both the number of classes and the domains the model must adapt to.
(022) for DomainNet and we follow the domain We follow the domain adaptation order of Rakshit et al.
order for CDDB-Hard and CoRE50 detailed in Wang et al.
(2022a); Feng et al. 2024). A detailed description
of each dataset and the implementation details are provided in Sup. Mat.
2025 we conduct few-shot 2023 Bendou et al. Evaluation Metrics. Following recent work Zhu et al.
experiments with 1, 4, and 8 shots to assess the effectiveness of our approach. We measure performance using two standard metrics: (i) Average Accuracy (AA) Rakshit et al. (2022): The mean classification accuracy across all domains seen so far, and (ii) Forgetting Alleviation (FA) Liu et al. (2023): The mean accuracy on a domain after the model adapts subsequent domains in the continual stream. We report the overall AA* and overall FA* averaged across all sessions like Mukherjee et al.] (2025). Detailed definitions of these metrics, per-domain results, are provided in Sup. Mat. Results are reported with average of three random seeds. Further results with five random seeds present in Sup. Mat.
Basclincs. We compare our method to multiple basclines adapted to the FSDIL setting. For regularization- based approaches, we incorporate EWC Kirkpatrick et al. (2017) and LwF Li & Hoiem (2017) into our backbone, updating only the learnable prompt. We also consider prompt-base based DIL methods such as L2P Wang et al.] (2022c); S-Prompt Wang et al. (2022a), CP-Prompt Feng et al. (2024), and Abased prior gradient approximation technique, InfLORA Liang & Li (022)). We re-run all basclines under the same experimental setup on every dataset to ensure fair comparisons and report the average over three runs with three random seed values. In contrast to the baselines which require a prompt pool set of prompts for each individual domains), our method lcarns and continually updates a fixed set of parameters, to avoid domain-prompl selection during inference prevalent in recently proposed parameter isolation based DIL strategies Wang et al. (2022a); Feng et al. (2024). Considering the closed-set nature of the FSDIL task, we choose not to compare our method against any FSCIL baselines. We used the following prompts for the Zero-shot CLIP experiments photo of a _. a photo of a image and there is a in this image., , for NomainNet, CDDB-Hard and CoRE50 respectively, comparison with other prompts in Sup.Mat.
4.1 Experimental Results
We use CLIP-VITB/16 as the backbone and re-run all baselines accordingly for fair comparison under DIL benchmarks. Wang et al. (2022a); Feng et al. (024).. Table 2 presents the average performance of PGO-BEN
across 1-, 2-, 4-, and 8-shot settings Net, CDDB-Hard, and CoRE50 datasets (per-shot results in Sup. Mat.). PGO-BEN consistently outperforms all basclines, including those that maintain domain-specific prompl pools for reducing forgetting and improving recognition. On Net, PGO-BEN achieves a gain of +1.31% in AA* and +1.27% in FA* over the strongest baseline. For CDDB-Hard and CoRE50, it surpasses prompt-pool basod methods by +6.66% and +2.53% in FA*, and by +6.56% and +4.86% in AA* popeetively-demonsttee
substantial improvements in balancing stability and plasticity. Importantly, PGO-BEN attains these results without relying on prompt pools or domain-specific routing at inference,
9

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: Comparison across DomainNet, CDDB-Hard, and CoRE50 averaged over 1, 2, 4, and 8-shot settings. Bold and underlined denote the best and second-best scores. PGO-BEN outperforms all baselines without using prompt pools, demonstrating its generalization strength. * indicates CLIP-ViTB/ based reimplementation. Results mcan + std over 3 seeds. Red font denotes least std method.
DomainNet
CDDB-Hard
CoItE50
Method
Average
Average
Average
Prompt
Backdone
(1)
PA*(1)
(1*(1)
PA*(1)
AA"(1)
PA*
DyTor Douillard al.
2021
VIT
30. 11.00
18.0220.4
37.1720000
53.17
46.5720.88
28.7220.00
Zero-sbot CLIP Radford et al.
2021
CLIP
69:05
56.32
12.67
LwFe
Holrmn 2017
CLIP
72.0611.82
60.70
68.2510.74
58.9910.15
64.41
67.7510.00
Kirisparick
2017
70.9210.90
38.8610.69
71.3010.81
62.2110.77
G3.4110.ca
65.0010.44
L2p*
Wanng es mi Muzze
+4.61
+0.52+0.94
65.8110.83
79.8850.76
78.366+
Dualf Walk
2022b
+3.45+t.85
63. 61.50+0.76
73.05+0.0
#6.51+0.50
35. 35.61+0.30
50.54+0.71
8: P.ommpp
Waing
67, 67.644t..7
56. 13...3
60.22
79 79.23
76.31#u.55
CODA-Pranpt
60.452:04r
InfL.RRA
V Ta
T1.93g985
60.41
66.6510.44
56.6310-17
65.3010.00
68.1011.77
CP-Prompt
Fong m4-
2024
71.8918882
60.8710.70
66.9510.36
62.10.0.22
81 81.57
70.99110.54
PGO (Ours)
80.09=0
66.4320.0
82.32
A
+1.81
+1.27
+4.56
+66.6
+ +4.86
+2..0
AA* (%)(()
FA (%)(t)
InfLORA
Average Cosine Starlacity
RWC M&PLE*
DunlProcupt CODA Prompt
DusllProcupt
0.85
CODA Prompt
73
PGO BEN
PGO BEX
0.3
74
0.25
78
72
Pointing
Cliport
Sketch
Quickdrww
Indograph
2-hot
4+hot
Sahott
Iodane
4-bect
5-hbst
(a) Learnt representation of PGO-BEN more aligned.
(b) PGO-BEN retains better with more data.
Figure (a) Encoder Reprosentation Alignment. We compare the representation similarity of the two encoder representations. Encoder representation of PGO-BEA have consistent higher cosine-similarity for all domains, indicating that Text-encoder is more aligned to the changing visual distribution in the Vision encoder as compared to other baselines. MaPLE* also fails to effectively capture the evolving visual distribution owing to a unsuitable encoder alignment direction. (b) Sample Efficiency. On DomainNet dataset, PGO-BEN consistently outperforms top-2 baselines across varying level of supervision.
offering a memory- and computationn-effiieeu solution compared methods that rely on prompt-pools (like Feng et al. (2024); Wang et al. (2022a), having a KNN based selection steps). Given that AA* and FA* * aggregate performance over all sessions (see Sup. Mat.), even moderate gains reflect meaningful improvements in continual learning dynamics. Our method achieves the lowest standard deviation across runs, indicating more consistent performance compared to all baseline methods.
These results highlight the effectiveness of our design: CLiP-based multi-modal prompting, with text-encoder prompt tokens (TOKr) conditioned on imagc-encoder prompts (TOKy) improves cross-domain adaptability with few labeled samples, while proxy-guided orthogonalivation and Reta-function--asse) temporal ensembling enhance long-term retention across evolving domains, without continually incrcasing memory overhead.
Representational alignment: We compare the representation similarity of the two CLIP encoder represen- tations by computing the average cosine similarity of the image embedding and text embedding of various basclines on the test datasct of the "Real" domain of the DomainNet dataset, as the models soquentially adapt to the new domains. As observed in Fig. 4a PGO-BEN maintains consistently higher image text embedding similarity than all baselines, indicating stronger representational alignment of and better retention under domain shift. In contrast, MaPLE* which conditions vision prompts on text prompts, shows weaker alignment. These results highlight the effectiveness of our conditioning strategy in both aligning the text encoder to the evolving visual distribution and preserving stability across domains.
10

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: Efficacy of proposed BMA compared to EMA on DomainNet dataset. BMA shows superiority
in retaining prior knowledge. Results are mean over 3 seeds.
I-shot
2-shots
4-shots
8-shots
Technique
AA*1
FA"1
AA"1
FA"1
AA
FA
AA*1
FA*
EMA (A = 0.98)
61.93
19.53
12.77
61.28
73.99
02.81
74.18
03:30
EMA (A- 0.99)
58.22
45.77
66.10
53.99
71.58
59.73
73.34
61.90
BMA (8 - (0.5.0.5))
74.27
63.76
74.36
63.92
74.58
64.17
74.61
64.28
Table 4: Robustness analysis of BMA (3 = 0.5)) over EMA (A 0.98). We compare the
cosine-similarity of the encoder representation on the test data of "Real" domain after adapting to "Clipart." domain (left) and "Sketch" domain (right), for PGO-BEN with EMA and PGO-BEN with BMA. BMA variant is observed to be supcrior, thus highlighting that it is stable even when domain gaps are large. Results are std over 3 seeds.
Real + Clipart Real + Sketch
PGO-BEN with EMA (A = 0.98)
32.50
31.95
PGO-BEN with BMA (3 0.5))
33.03
32.0610.r
A
+0,53
+0.11
4.2 Ablation Analysis
(a) Sensitivity to Training Sample Availability. We analyze the effect of supervision sparsity by varying labeled samples per class in DomainNet (Fig. 4b). PGO-BEN consistently outperforms the top baselines across all supervision lovels, achieving superior gencralization and retention in low-shot regimes (c.g.. +1.77% A and +1.38% FA* in 1-shot). This advantage further increases with more labels (e.g., +2.35% FA* over DualPrompt in 8-shot), confirming that PGO-BEN scales effectively with available supervision while maintaining cross-domain consistency.
(b) Comparison of EMA and BMA. We compare BMA with EMA across varying shots on DomainNet (Table 31. BMA consistently yields superior performance by more effectively balancing the trade-off. While EMA applies exponentially decaying weights that rapidly diminish the influence of early training states, it often leads aggressive forgetting of knowledge from prior domains. In contrast, BMA assigns symmetric Beta-function based weights (see Fig. 2b), preserving early domain knowledge while still adapting to the current task-rosulting in improved retention. Morcover, EMA is highly sensitive to the decay lyperparameter, requiring careful luning across domains. BMA exhibits greater robustness under hyperparameter variation (Table further supporting its suitability for dynamic FSDIL settings.
To assess the retention benefits of BMA over EMA, we compute the cosine similarity between the final CLIP embeddings and those obtained after learning the "Real" domain, evaluated on "Real" Lest samples after adaptation to Clipart and Sketch. Higher similarity indicates better knowledge preservation. As shown in Tab. 4 under the 4-shot sctting, the BMA variant of PGO-BEN consistently maintains higher similarity across both adaptation scenarios (Real + Clipart and Real + Sketch), demonstrating stronger stability against forgetting.
To further investigate the reasoning of such behavior in a continual adaptation scenario, in Tab. 5 we compare the variance the prediction vector of PGO-BEN with EMA and PGO-BEN with BMA on the test dataset of the "Real" domain of the DomainNet dataset, with respect to the prediction vector initially obtained after just training on the "Real" domain, as both the models sequentially adapt to the new domains. In the 4-shot scenario, observe that EMA variant of the model has relatively higher variance compared to the 1MA variant. This highlights why the performance of BMA remains more stable compared to EMA variant, highlighting the better stability thus achieved.
Clearly, these findings validate BMA as a more principled and stable ensembling strategy than EMA, enhancing long-term knowledge preservation in continual few-shot learning.
(c) Sensitivity to in Beta Ensembling. We evaluate the effect of different 8 values in the Beta distribution used to weigh intermediate model states (Me)T"s during adaptation to domain D. (Table
11

---

Published in Transactions on Machine Learning Research (01/2026)
Table 5: Comparison of prediction variance between EMA (A= 0.98) and BMA(3 = (0.5, 0.5)) ensembling on "Real" domain test set during continual domain adaptation. We compute the variance of the prediction logit vector of PGO-BEN with EMA and PGO-BEN with BMA, on all models which has adapted to the subsequent domains, on the test datasct Real" domain. PGO-BEN with BMA is observed to have less variance compared to PGO-BEN with EMA. Results are mean over 3 seeds.
Painting
Clipart
Sketch
Quickdraw
Infograph
PGO BEN with EMA (A = 0.98)
4.7
4.40gam
4.05gmm
2.30
2.
PGO-BEN with BMA (8 = (0.5, 0.5))
1.32
4.31 0.
2.24
o.80
A
-0.39
-0.09
-0.20
-0.06
+0.11
Table 6: Comparison of value of Superior stabllity-plasticity trade-off observed when B (0.5. Results are mean of seeds.
Shots
(0.3,0.3)
B (0.5.0.5)
(0.7.0.7)
B = (1.1)
AA
(1*
(1)
(1)
AA*((1 (1)
FA
(A
FA*(")
I-shot
73.96
63.40
74.27
63.76
73.97
63.38
74.17
63.77
2-sbot
74.32
63.79
74.36
63.92
74.29
63.74
74.32
63.86
4-shnt
74.46
63.80
74.53
64.17
74.42
63.84
74.47
64.02
8-shot 74.70
64.18
74.61
61.28
74.64
64.05
74.49
64.00
Average 74.36
63.81
74.44
64.03
74.33
63.75
74.36
63.92
The choice of B controls the temporal emphasis in the ensembling process: lower values emphasize early and late stages, while higher values favor mid-training checkpoints. Weighting curves are visualized in Sup. Mat. Empirically, 3 = (0.5, 0.5) yields the best average FA across all supervision levels, highlighting that moderately bi-modal weighting (3 < 1) offers better stability in BMA, validating its design choice as robust. temporal smoothing mechanism in PGO-BEN.
Table Analyzing the impact of the stability components. We experiment on the DomainNet dalasel. across 2, 4, and -shots training examples. Neglecting the BMA component results in significant forgetting of the knowledge of the previous domains. Results are mcan of 3 sceds.
Stabilily components
1-shot
2-shot
4-sbot
8-ebol
Gradienl Distill
BMA
(A*("))
FA*(1)
AA*(1))
FA)
AA*(1)
FA*(1)
AA*(1))
FA*(1)
73.75
62.89
74.07
63.11
74.05
73.97
62.38
72.80
61.87
73.74
63.05
76.44
61.52
74.42
61.33
74.27
63.76
74.36
63.92
74.53
64.17
74.61
64.28
Table 8: Prompting Configurations. We compare unimodal and multimodal setups, finding that condi- tioning text prompts on vision tokens improves generalization under domain shift. Results are mean of 3 soeds
Prompling
I-shot
2-ehots
4-shuts
8-chols
Teclniqqee
Encoder
AA*
PA*1
AA*1
FA
AA*;
FA*1
AA*1
FA*:
Unimedal
F;
70.34
59.33
71.11
59.68
72.41
61.92
72.96
62.21
72.02
62.35
71.85
02.15
72.27
62.21
[F. Fu)
72.52
62.13
72.98
02.51
73.13
62.62
13.43
62.23
Malsi-modal
Fe F.
70.99
61.28
72.50
02.79
F, FF, (Ours)
74.27
63.76
74.36
63.92
74.53
64.17
74.61
64.28
(d) Impact of Stability Components. We ablate the two stability components of PGO-BEN: Proxy
Guided Orthogonalization and BMA. As shown in Table 7 using Proxy Guided Orthogonalization alone may be suboptimal when D, differs significantly from previous domains, leading to unreliable Gprev- BMA, based solely on Geurr, adapts better to new data and reduces overfitting through temporal smoothing. Their combination consistently yields the best performance across all supervision settings, achieving a stronger balance between adaptation and retention, especially under label scarcity.
(c) Effectiveness of Multi-Modal Prompting. Table g compares PGO-BEN's vision-condittonned prompting with uni-modal and other multi-modal baselines: Independent Prompting and Text-conditioned Vision Prompting-MaPI. Multimodal prompting learns better representation as compared to Unimodal
12

---

Published in Transactions on Machine Learning Research (01/2026)
representation. MaPLE* conditions Fv's prompts Fr, thus performing sub-optimal in capturing visual domain-shift. PGO-BEN outperforms all variants across shots, with Fig. 4a showiing higher similarity between learnt embeddings and better retention of knowledge, supporting our choice of conditioning for better domain invariance and generalization.
Extended ablations, including prompt lengths, depth of encoder synergy, and observing novel classes during inference, further analysis on BMA, are mentioned in Sup. Mat.
5 Takeaways
We addressed the underexplored challenge of FSDIL, a setting critical for deploying continual learning systems in dynamic and low-supervision enviromments. We proposed PGO-BEN, unified framework integrating multi- modal prompting, proxy-guides orthogonalization, and Beta-based ensembling. Our method demonstrates strong resilience to catastrophic forgetting while enabling efficient adaptation under severe domain shifts. Comprehensive ivaluations across multiple benchmarks confirm the elfectiveness and generalization of PGO- BEN, highlighting the value of stabilizing gradient trajectories, and adaptively balancing stability and plasticity. Furthermore, the framework's 's ability to scale across domain variations positions it is a practical solution for real-world continual learning tasks. Future directions include extending PGO-BEN to handle unlabeled domain adaptation scenarios, and improving sample-efficinncy further via generative replay or self-supervised objectives.
Broader Impact and Limitation: This work enable AI to adapt in critical data-scarce fields (e.g., healthcare, robotics), though risks of flawed adaptation or model bias are data dependent. The method assumes fixed label spaces across domains, and its long-term scalability to numerous, highly diverse domains needs further study.
References
Yassir Bendou, Amine Ouasfi, Vincent Gripon, and Adnane Boukhayma. Proker: A kernel perspective on few-shot adaptation of large vision-language models. ar preprint arXiv:2501.11175, 2025.
Shirsha Bosc, Ankit Jha, Enrico Fini, Mainak Singha, Elisa Ricci, and Biplab Banerjee. Stylip: Multi-scale style-conditioned prompt learning for clip-based domain generalization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3542-5552, 2024.
Olivier Bousquet and André ElisseefT. Stability and generalization. Journal of machine learning research, 2 (Mar):499-3) 526, 2002.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégon, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021.
Autonio Carla, Joost Van de Weijer, et al. Improving online continual learning performance and stability with temporal ensembles. anXiv preprint orXiv:2306.1681 2023.
Francisco M Castro, Manuel J Marin-Jiméne", Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End- to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV), pp. 233-248, 2018.
Songlin Dong, Xiaopeng Hong, Xiaoyu Tao, Xinyuan Chang, Xing Wei, and Yihong Gong, Few-shot class- incremental learning via relation knowledge distillation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp. 1255 1255-1263, 2021.
Arthur Douillard, Alexandre Ram'e, Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognétion (CVPR), pp. 9275-9285, 2021. URL https: /taps:///pi.... semanticscholar wareloog/ooooo.o CCrrpurI
244478589
13

---

Published in Transactions on Machine Learning Research (01/2026)
Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochas- tic) neural networks with many more parameters than training data. arXiv preprint arXiu: 170:: 11008, 2017.
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Stafistics, pp. 3762-3773. PMLR, 2020.
Yu Feng, Zhen Tian, Yifan Zhu, Zongfu Han, Haoran Luo, Guangwei Zhang, and Meina Song. Cp-prompt: Composition-based cross-modal prompting for domain-incremental continual learning. In Proceedings of the 32nd ACM International Conference on Multimedia, MM '24, pp. 2729 2738, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400706868. doi: 04445//6666477.666 https://doi.org: 10. htt.///...rrrr...1///6666//66660
URL
3681481
Muhammad Uxair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. https:///rrti https://arzis..rr//bb/ wtggab///2000000 2220000119 2022.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinskee et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, (14((1):3521-3 3526, 2017.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficien prompt tuning. In Mario-Francins Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tan Yih (eds.), Proceedings of the 2021 Conference Empirical Methods in Natural Langwage Processing, pp. 3045 3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/ (t.18653/v1/2/ w1/2021..2mmmia emmll- www.lmmmin...... URL
emnlp-main main main.233/
Chuqiao Li, Zhiwu Huang, Danda Pani Paudel, Yabin Wang, Mohamad Shahbazi, Xiaopeng Hong, and Luc Van Gool. continual deepfake detoction benchmark: Datasct, methods, and esscntials. In Proceedings of the IEEE/CVF winter conference on applications of compuler vision, pp. 1339 1349, 2023.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582 4597, Online, August 2021. Association for Computational Linguistics. doi:
URL /ttps://aclaanthoooggg./ org/ www.oogg////2/.. 2021 ac1-1a.cclooogg 353/
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2944 2017.
Yan-Shuo Liang and Wu-Jun Li. Adaptive plasticity improvement for continual lcarning. In 2023 IEEE/CT /VF Conference on Compuler Vision and Pallern Recognition (CVPR), pp. 7816 7825, 2023. doi: 10.1109/ STPPR527992002.0002
Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-fres low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 23638 23647, 2024.
Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Trgp: Trust region gradient projection for continual learning. arXiv preprint arxin2202 N2200222.00011 2022.
Chenxi Liu, Lixu Wang, Lingjuan Lyu, Chen Sun, Xiao Wang, and Qi Zhu. Deja vu: Continual model generalization for unseen domains. arXiv preprint ar Xrx::23001.10418 2023.
Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. In Conference on robot learning, pp. 17-26. PMLR, 2017.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neuraa information processing systems, 30, 2017.
14

---

Published in Transactions on Machine Learning Research (01/2026)
David A. McAllester. Some pac-bayesian theorems. In Proceedings of the twelfth annual conference Compulational learning theory (COLTT), pp- 230-234. ACM, 1999.
Mayank Mishra, Prince Kumar, Riyaz Bhat, Rudra Murthy, Danish Contractor, and Srikanth Tamilselvam. Prompting with pseudo-code instructions. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 15178-15197, Singapore, URL
December 2023. Association for Computational Linguistios. doi: 10. http:///1/222.em//pp// https://aclanttollee wtw.///nnnooooooor/22233333
mainnlp-mmii...i. m
Samrat Mukherjee, Tanuj Sur, Saurish Seksaria, Subhasis Chaudhuri, Gemma Roig, and Biplab Banerjee. Uidaple: Unsupervised incremental domain adaptation through adaptive prompt learning. In ICASSP 2025- 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 1-5, 2025. doi:
Lorenzo Pellegrini, Gabriele Grafficti, Vincenzo Lomonaco, and Davide Maltoni. Latent replay for real-time continual learning. In 2020 TEFE/RSJ International Conference Intelligent Robots and Systems (TROS), pp. 10203 10209. IEEE, 2020.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1406 1415, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. $748-8763. PMLR, 2021.
Sayan Rakshit, Anwesh Mohanty, Ruchika Chavhan, Biplab Banerjee, Gemma Roig, and Subhasis Chaud- huri. Frida generrtivee feature replay for incremental domain adaptation. Computer Vision and Image Understanding. 217:108367, 2022.
Sachin Ravi and Hugo Larochelle. Optimization model for few-shot learning. In International Conference on Learning Representations, 2017. URL
Sylvestre-Alvis Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEFE conference on Computer Vision and Pottern Recognition, pp. 2001 2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International net Conference on Learning Represenlations, 2019. URL. htttps://openreiieeee http:////pppprreeeeeeet/oorrrrrr wtt.:e//fforrrriie/gg//shccc
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint ar oxXi:2213.00900 2021.
Haizhou Shi and Hao Wang. A unified approach to domain incremental learning with memory: Theory and algorithm. Advances in Neurol Information Processing Systems, 36:15027-15665 2023.
Yang Shu, Xingzhuo Guo, Jialong Wu, Ximei Wang, Jianmin Wang, and Mingsheng Long. Clipood: Generalizing clip to out-of-distributionn. In International Conference Machine Learning, pp. 31716- 31731. PMLR, 2023.
James Scale Smith, Loonid Karlinsky, Vyshnavi Gutta, Paola Cascanto-Bonille Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira, Coda-prompt: Continual decomposed attention- prompting for rehearsal-free continual learning. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 11909 11919, 2023.
Yisheng Song, Ting Wang, Puyu Cai, Subrota K Mondal, and Jyoti Prakash Sahoo. A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities. ACM Computing Surveys, 55 (13s):1-400 2023.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Tamuj Sur, Samrat Mukherjee, Kaizer Rahaman, Subhasis Chaudhuri, Muhammad Haris Khan, and Biplab Banerjee. Hyperbolic uncertainty-aware few-shot incremental point cloud segmentation. In 2025 TEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11810 11821, 2025. doi: 10.1109/
CVPR52734.2025.011 200.001
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few-shot class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pollern recognition, pp. -22192, 2020.
Jesper E. van Engelen and Holger H. Hoos. A survey on semi-supervised learning. Machine Learning, 109: 373 440, 2019. URL /tt.:ptoss/pppp...aatttccccooo wwwp.o/rrpss..222733388888
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu, A comprehensive survey of continual learning: theory, method and application. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers: An occam's razor for domain incremental learning. In S. Kovejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 5682-5695. 5695. Curran Associates, Inc., 2022a. URL Paper- Conference pdf
www.euriss.cpppppppeeee cc/paper files/ pt1ee/2/apr//2222/
Zhicheng Wang, Yufang Liu, Tao Ji, Xiaoling Wang, Yuanbin Wu, Congcong Jiang, Ye Chao, Zhencong Han, Ling Wang, Xu Shao, and Wenqiu Zeng. Rehearsal-free continual language learning via efficient parameter isolation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61sl Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10946, Toronto, Canada, July 2023. Association for Computational Linguisties. doi: 10.18653/ /////////222ac/// acl acc.oo.....oon....o long.612. URL httpp::/acclanthologg) .rg. ac1-1on.lllonn.. 612/
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su, Vincent. Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free ioutl learning. In European Conference on Computer Vision, pp. 631 648. Springer, 2022b.
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual Iearning. In Proceedings of the IEEE/CVE conference on compuler vision and paldern recognition, pp. 139 199-149, 2022c.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceediigg of the IEEE/CVF conference on computer vision and poltern recognition, pp. 374-382, 2019.
Xiangli Yang, Zixing Song, Irwin King, and ZZenllin Xu. A survey on deep semi-supervised learning. IEEE Transactions on Knowledge and Data Engineering, (599)8893448054 2022.
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsca Finn. Gradient surgery for multi-lask learning. Advances in neural information processing systems, 33:5824 5836, 2020.
An Zhao, Mingyu Ding, Zhiwu Lu, Tao Xiang, Yulci Niu, Jicchao Guan, and Ji-Rong Wen. Domain-adaptive few-shot learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1390-1399, 2021.
Da- Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De- Chuan Zhan. Continual learning with pre-trained models: A survey. arXio preprint Xrx::22401.0 101001111666666 16386, 2024.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 16816-16825, 2022a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for lasion-language models. International Journal of Computer Vision, 130(9):2337-22)) 2348, 2022b.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Zongwei Zhou, Jae Y Shin, Suryakanth R. Gurudu, Michael B Gotway, and Jianming Liang. Active, continual fine tuning of convolutional neural networks for reducing annotation efforts. Medical image analysis, 71: 101997, 2021.
Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 15659-15669, 2023.
17

---

Published in Transactions on Machine Learning Research (01/2026)
A
Appendix
We discuss the additional details like dataset details, evaluation metrics and additional results in the appendix.
B Contents in this Supplementary document.
This supplementary document provides detailed insights and analyses support the main paper. The contents are organized as follows:
Section Proxy-Guided Guided Orthogonalization (PGO)
Elaborates the Proxy Guided Orthogonalization mechanism, connecting it with existing orthogonality-bbseed continual learning literature. It also presents empirical observations that justify the chosen hyperparameters.
Theoretical Justification via PAC-Bayesian Framework Discusses the theoretical underpinnings of our proposed method, PGO-BEN, utilizing the PAC- Bayesian framework to provide generalization guarantees.
Section D
Section D Comparative Analysis of EMA and BMA
Offers thcoretical comparison between Exponential Moving Average (EMA) and Beta-based Moving Average (BMA). We analyze prediction variance across domain sequences and assess representation stability, especially under significant domain shifts.
Section F Distinction of our approach with Multi-Task Learning
Discusses the fundamental differences of approach with respect to the Multi-Task Learning from which method takes inspiration. We highlight the aspects where formulation differs from it, with experimental results highlighting that direct application of MTL method is sub-optimal.
Section G Dataset and Domain Order Details
Outlines the datasets used and the specific domain sequences followed our experiments.
Section H
Evaluation Metrics
Details the metrics employed for evaluation, providing formulas and explanations for clarity.
Section !
Implementation and Hardware Details
Discusses the implementation specifics of our method and the hardware configurations utilized during experimentation.
Scction Algorithm Pscudocode
Presents the pseudocode of our proposed algorithm, offering a step-by-step procedural understanding.
Section K Model agnostic nature of our methodology
Experimentally verifies that our method is model-agnostic and can be applied to any CLIP like architecture with Transformer based backbone in the encoders. Specifically, we re-run several baselines with ViTL/14 backbone of the Vision encoder of the CLIP model.
Section L Comparison with Zero-shot CLIP with manual prompt
Presents the results obtained by prompting CLIP-VITB/ model with various manual prompts for all the benchmark datascts. The results highlight that existing large-scale models require careful adaptation algorithms and pre-trained weights with manual prompts provide sub-optinal performance. This also highlight that manual prompting is very hard in fine- fine-grain datasets like CoRE50.
Section M Comprehensive Results
Provides detailed results for 1, 2, 4, and 8-shot settings across three benchmark datascts: DomainNet, CORe50, and CDDB-Hard. Additionally, we present results corresponding a seed value of 2 for all datasets.
18

---

Published in Transactions on Machine Learning Research (01/2026)
Section O
Encoder-Synergy Module Depth Analysis
Examines the performance implications of varying the depth of the Encoder-Synergy module.
Section P: Prompt Length Modulation
Analyzes how changes in prompt length affect performance, providing insights into optimal configu- rations.
Section Q
Novel Class Inference
Explores scenarios where novel classes are introduced during inference, demonstrating that PGO-BEN effectively recognizes new classes, attributed to the robust prior knowledge from CLIP.
Section R Experiments with 5 seeds
Explores the effect of using 5 seeds instead of seeds. We rerun all the baselines for all the datasets across all the shots and report the individual score along with the average.
C Further Discussions on Proxy-Guided Orthogonalization
We address concerns regarding the robustness, theoretical justification, and convergence properties of our Proxy-Guided Orthogonalization (PGO) strategy, designed enhance stability across soquential domain shifts in the Few-Shot Domain--ncrrmeent Learning (FSDIL) setting.
A primary concern is the reliability of predictions from the previous model M,- when adapting to new domain D,, especially when D, significantly differs from prior domains (DI...., D,-1}. To mitigate this, PGO employs a soft directional filter rather than a hard constraint. Specifically, when the cosine angle t between the current gradient Geurr and the previous gradient Gprev exceeds 90", indicating potential conflict, we project out the conflicting component and retain only the orthogonal component G (see Fig. 5). This approach ensures that adaptation to new domains does not adversely affect previously acquired knowledge, thus en enhancing stability.
19

---

Published in Transactions on Machine Learning Research (01/2026)
Gl
G update
G curr
G curr.
prev
G prev
uuddate
curr
curr
G curr
G along the direction of previous knowledge G (doesn't interfere with previous knowledge)
Gupdate
G.L. curr
G pree
G curg
G G
G Grre prev
Suddtte
G curr
G curr
Gil Geurr
Gll opposite the direction of previous knowledge Gyree (interfers with previous knowledge)
Figure 5: Illustration of conflicting and non-conflicting gradient update scenarios with respect to prior knowledge. We illustrate the scenarios where the current adaptation may or may not interfere with prior knowledge. When & E [- 21 31, there is conflict, allowing direct updates. In cases of conflict, updates are made in the orthogonal direction G
Unlike prior methods that rigidly project gradients into stored subspaces or freeze parameters Farajtabar et al. (2020); Saha al. (2021): Liang & Li (2024) 2023), our approach allows for flexible adaptation even under significant domain shifts. Empirical results demonstrate strong retention of prior knowledge despite domain divergence.
We adopt a fixed cosine angle threshold of 90" to detect conflicting gradients. This choice is simple, interpretable, and aligns with orthogonality-bassed continual learning literature Farajtabar et al. 2020 Saha
et al. 2002)); Liang Li (2024 2023). To assess sensitivity, we perform ablations by varying the threshold across multiple angular deviations. The results, detailed in Table g indicate that PGO's performance remains robust across range of threshold values, validating the evfectiveness of our chosen threshold.
Table 9: Using 90" conflicting knowledge among Gprev and Geur is observed to empirically also enhance the stability-plasticiy tradeoff.
cosine similarity threshold. The choice of 90" as the threshold to identify
v S 45"
1 2 315"
8 S 75"
& 1 285"
1 S 90" & v 270"
110%
2 250"
AA*1
FA*;
AA*t
FA
AA*1
FA*1
AA*1
FA*1
1-shot
73.77
62.89
73.89 74.54
62.95
74.27 74.93
63.76
74.11 74.46
63.01
4-shot
74.40
64.02
64.09
64.48
64.01
Our approach operates within the CLIP framework, where the vision and text encoders remain frozen, and only lightweight prompt and adapter parameters are updated. This low-dimensional setting mitigates
20

---

Published in Transactions on Machine Learning Research (01/2026)
optimization complexity. While formal convergence guarantees for non-convex losses are challenging, the cosine-based gating in PGO ensures that updates do not destructively interfere with previously optimized directions, leading to smoother loss trajectories. Moreover, PGO functions as directional regularizer, avoiding gradient interference without introducing projection errors common subspace- based continual learning.
In summary, StradiettAAligned Distillation lightweight, theoretically motivated, and empirically stable mechanism for continual adaptation under domain shifts. It effectively handles domain divergence, is robust to threshold variations, and supports convergence in practice, even in non-convex prompt learning setups.
D Theoretical Justification of PGO-BEN via PAC-Bayesian Framework
We formally justify the generalization behavior of PGO-BEN under the FSDIL setting using PAC-Bayesian analysis. The bound characterizes generalization performance for stochastic predictors trained on finite samples under a prior posterior distributional framework.
Let H. denote the hypothesis space parameterized by model weights 0, and let D, be the domain distribution at sossion t. Consider bounded loss E: H X
((, x4, v!), and [0, empirical risk LD, (h) n 1
expected risk LD, (h) ((xx)),, (((h), x.))
Theorem 1 (PAC-Bayer Generalization Bound McAllester 1999) Let be a prior distribution over H. and be dele-dependenl posterior. Then, for any 5 E (0, 1), with probability al least ō over the choice of sample S Di, we have:
KT(p|=) + log 2 8
Ba~p (h)) S En=plLD
2n
Prior т: We define the prior π as the parameter distribution of the pretrained CLIP model (0o) before adaptation begins. Due to CLIP's exposure to broad domain diversity, this prior is semantically rich and well-aligned with the hepothesis space for downstream domains, particularly useful the few-shot regime where p must stay close
Posterior via BMA: The posterior p is implicitly constructed mixture of model checkpoints across training steps:
('+0.5) (T441 ((tps))
oee 8M where ay x Beta(B, B)
This formulation yields smoothed, trajectory-aware posterior that reduces overfitting to the final iterate and aligus with posterior averaging methods shown to tighten PAC-Bayesian bounds Dziugaite & Roy 2017
Wu et al. 2019
Stability via Proxy- Orthogonalization: The cosine-based masking of gradient directions PGO prevents calastrophic drift from Mr11, enforcing update stability without requiring stored gradients. This aligns with algorithmic stability theory Bousquet Elisseeff (2002). which bounds the empirical empirical-expected risk gap and improves generalization.
Summarily, each component of PGO-BEN contributes to a tighter PAC-Bayesian bound:
strong prior æ via CLIP roduces the complexity term KL(p)(=);
The BMA-based posterior p smooths model updates, lowering variance in empirical loss;
Gradient-aligned updates stabilize training, narrowing the empirical-expectte loss gap.
While deriving exact closed-form bounds deep networks remains intractable Bousquet & Elissooff (202);
Dziugaite & Roy (2017), our formulation aligns with established PAC-Bayes theory and is supported by robust empirical generalization across non-i.i.d. domain streams.
21

---

Published in Transactions on Machine Learning Research (01/2026)
coch eoch mohel during adaytation stop
8(0.3,0.3) 0.3)
9(0.5, 0.5)
3(0.7,0.7)
A8,, 1)
8(2, 2)
(Unrommsieee Weghhageegggggeee of
Adaptation steps
Adaptation steps
Adaptation secps
Adaptation steps
Adapiation steps
Figure 6: Beta-distribution with different parameters. Variation of the Beta-distribution curve with different parameters.
E
Theoretical and Empirical Analysis of Beta-based Moving Average (BMA)
EMA is first-order recursive smoother of model paramcters:
OEMA 20; + (1 A
where X e [0, 1] is the decay parameter. Its recursive nature gives exponentially diminishing weights to early
iterates, rapidly discarding useful information from past domains.
BMA, in contrast, uses a non-recursive weighted sum of intermediate model checkpoints:
T"
t' + 0.5 T +1
BBMA_ Σ
Ocl ay x Beta(3,
This induces a non-monotonic, symmetric weighting over time, explicitly preserving early-stage information while still incorporating late-stage adaptation. Unlike Gaussian posterior smoothing or Bayesian ensembling, which require approximate posterior distributions over parameters (e-g-, Laplace or variational), BMA is implicit posterior smoother operating over deterministic iterates. This avoids costly uncertainty estimation sampling, while still capturing temporal uncertainty through weighting. This choice is deliberate: in FSDIL, labeled data is too sparse to fit full Bayesian posteriors per domain, and BMA provides an efficient surrogate.
Figure G describes the shape of the Beta-distribution curve. As we see, with 3 < 1, the models at cither end the adaplation stralegy are given more weightage. This aligus with the intuition that model state during the initial iterations of the adaptation step in domain D, likely to have more knowledge about domains (Di, Di-1), and hence should be given more weightage to ensure that even after adaptation, the model M,, adapted on domain D, preserves the knowledge of domains "2...",,y-13, B 1, gives equal weightage to all the intermediate states.
Variance Reduction: Let = Ev [wr]. The total variance under BMA is:
(ar, = [, -
which, for symmetric 3 1, gives more uniform support across the training trajectory, reducing the bias toward terminal points that plagues EMA. This stabilization is critical FSDIL, where prior domain knowledge must not be erased. In Fig. ?? (main paper), we show the prediction variance comparisons between BMA and EMA the prediction the test set of Real domain dataset the model keeps adapting to sequence of domains, in 4-shot scenario. BMA is found to reduce the prediction variance more effectively, thus maintaining steady performance as compared to EMA much better. We also compute the change in cosine-similarity of the output of lext and vision encoder, as the model has to adapt to a large domain-shift. After learning about the Real domain, the average cosine-similarity on the Real domain test set is 0.3480. As we observe, for both the adaptation scenarios (Real + Clipart and Real - Sketch), the drop the cosine similarity of the text and vision encoder representations for the EMA model higher compared to BMA, which indicates that the representations learnt by EMA is more stable than FMA.
22

---

Published in Transactions on Machine Learning Research (01/2026)
Ours
MTL.
1-shot
62.33
55.07
2-shot
62.42
55.16
8-shot
64.54
56.63
Table 10: Comparison of our method and MTL Yu et al. (2020) on DomainNet dataset, with respect to AAr- We observe that our method is significantly better than the applying MTL approach directly, which results more conflicting graident updates, hampering the learning.
The choice of B = (0.5, 0.5) corresponds to the arcsine distribution, which maximally weights both early and late checkpoints:
(et.(0.5.0.5) π [
x E (0,
This is particularly suitable in continual learning where: Early iterates capture prior domain knowledge, and Later iterates specialize to the current domain.
In summary, BMA offers principled, interpretable, and efficient strategy to smooth adaptation across domains in FSDIL. It reduces variance, preserves early domain knowledge, and improves generalization stability over EMA. While formal convergence bounds remain an open direction, our empirical and intuitive justification strongly supports its use over classical EMA or probabilistic smoothing methods the continual few-shot regime.
F Distinction with Multi-Task Learning
While method is inspired by Yu et al. (202)); it significantly departs from standard multi-task learning (MTL) due to the unique constraints of FSDIL.
In Multi- Learning Yu et al. (2020), all task data is available simultaneously, allowing per-task gradient computation and direct conffict resolution. In contrast, FSDIL restricts access to only the current domain D., with no replay or lask labels, and faces few-shot supervision and uncoustrained domain shifts conditions where direct gradient projection (as Liang & Li (2024) becomes unreliable.
Our key innovations as follows:
We use the frozen model Mi-1 as a proxy for prior domain knowledge. By passing current inputs D, through M we approximate prior gradients without accessing old data.
We compare gradients from M₁ (CE loss) and M,-1 (KL loss). When conflicting directions are detected, we project the current CE loss gradient orthogonally to preserve prior knowledge-achinn lorgetting mitigation without memory or subspace estimation.
Since M is ovaluated on an unseen domain, its gradients may be noisy. To stabilize updates, we introduce BMA, which adaptively ensembles model states and improves retention under shift.
Thus, while inspired by MTL conflict resolution, formulation is fundamentally adapted to FSDIL: exemplar-free, dommain-incremenntal, and few-shot. We will further clarify this in the revised version.
We conducted an experiment where data from all domains was introduced jointly and trained using Yu et al. (2020). treating each domain as a separate task-referred to as MTL* We compared this against our method, which observes each domain sequentially. Average accuracy (AA) across all domains reported below.
G Dataset details
We perform our experiments on three standard Domain Incremental Learning(DIL) benchmarks. The detailed descriptions and statistics These datasets follows:
23

---

Published in Transactions on Machine Learning Research (01/2026)
CDDB Li al. 2023) is dataset used for continuous deepfake detection, where the DIL objective involves recognizing hettentic and fake images across different domains. We adopted the Hard Setting from Wang et al. (022a)., requiring learning on 5 continuous deepfake detection domains: GauGAN, BigGAN, WeldDeep WhichFaceReal, and SAN. This entails approximately 27, 27,000 images. The
domain order followed aligus with Wang al. 2022a i.e. GauGAN + BigGAN - WildDeepfake
-> Waccceaa + SAN.
CORe50 Lomonaco & Maltoni 2017 designed for continuous object recognition, consisting of 11 domains, each with 50 categories. In DIL, perform incremental learning on the first eight domains, as sl + s2 + s3 + s4 + s5 + s6 + s7 + s8.
DomainNet Peng et al. (019)) is a domain adaptation dataset commonly used as a benchmark for DIL methods. It comprises 6 domains, each with 345 categories. The domain order is the same as Rakshit al. (2022), followed by Real + Painting + Clipart + Sketch - Quickdraw Infograph, which follows an incrementally more difficult domain learn.
H Evaluation metric details
After adapting to a domain D, we evaluate the performance on domains {D1 ... D.}. To measure the effectiveness of our method towards handling the stability-plasticity trade-off, we use two standard metrics, Average Accuracy (AA) and Forgetting Alleviation (FA). Average Accuracy for domain Dj defined as
AA ΣA.. Are
==1
where Ait denotes the accuracy obtained by the model on the i-th domain adapting to t-th domain. We used AA metric which is defined
N
1
AA
Σ
AA
(8)
N
t=1
where N denotes the number of domains. This metric provides more comprehensive measure of how the performance varies across all the training sessions and thus reduces the bias of only checking the performance the last session. As 4 is an average of average values, little improvement indicates much superior performance than the other counterpart.
Forgetting Alleviation for domain D, is defined as
FA
Σ A i=t+1
(9)
This measures the average performance the model on the domain Dj after being adapted to subsequent domains D .....--2 .Dv. We used FA as our metric which is defined as
1 N
FA
Σ FA
(10)
We present walthrough of the calculation of the metrics in Fig taking a toy example of three domains (D,, Da); with the domain sequence being (D, - D2 - Dz).
Implementation details
We maintained Raddord et al. (2021) ViT-B/16 as our backbone architecture for all the datasets, and all baselines (except in Sec. K). We used SGD as our optimizer with an initial learning rate of 0.002. The input images are resized (224 x 224) for all the baselines and our proposod method. We did our training on a single NVIDIA RIX A6000 48GB-GPU, and used Pytorch as our Deep learning framework, running the models for 20 epochs.
24

---

Published in Transactions on Machine Learning Research (01/2026)
Accuracy on seen domain as the domains observed
Average Accuracy and Overall
Forgetting Alleviation (FA) and
Avernge Accuracy (AA*)
Ovenill Forgelting Alleviation (FA*)
4A d1
FA (a21 + 031)
Accuracy Domain D3 after adapting Domsin D3
Q11
(a21 + a22)
Domain Sequence
AA2
2
FA₃ 032
D2
(asl + a32 + 433
AAg
D
031
0.32
4-
033
AA" (AA1 AA₁ + AAs)
FA" (FA1 + FAz)
3
D1
D22
D3
Figure 7: Metric calculation walkthrough: simple walkthrough using an example of three domains. We detail the individual steps to calculate AA and FA"
25

---

Published in Transactions on Machine Learning Research (01/2026)
J
Algorithm
We detail our learning process in the first session and in the incremental sessions with few-shot labeled examples in the form of a pseudo-code in Algorithm |
26

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm Training and inference procedure of PGO-BEn
Require: Dataset (2, D Model M, max epochs maz _poch, parameter / for Beta distribution. for 1 == 1 to N do
(x' )²)
De
if t===1 1 then
First domain
4:
for epoch = to max epoch do
5:
for j = 1 to Di do
Vpred t (z)
7:
Lie f- prrossenrropy(ypnrde W3)
8:
Lee backwack
optimizer.step()
Updates all the learnable porameters.
10:
end for end for
11:
12:
Use Mi for inference at
13:
else
Incremental domains with few-shot samples per class. Mr + MM-1 & Initializing model to be adapled to domain DD using the model which has been
14:
adapted to (Di .D-1)
15:
M t M
> Initializing the BMA model using the model which has been adapted to
{D1 D
16:
for epoch to epoch do iter
17:
18:
for j ==1 to Dr| do Vyred t Me(az) &> Prediction probability vector from the model we are adapting to domain
19:
De
20:
Myred t -M,-((0() & Prediction probability vector from the frozen model which has been adapted sequentially DI...D. -D-1
21:
Lee - CROSSENTTOPPY(J (preer v/))
22:
Lkd - KI Divergencel Wpredd Wpreed)
23:
Gearr < Gradient Dee
24:
Gyrev & Gradient of Lki
25:
Compute v. angle between Geurr & Gyrev for each learnable paramters of Mr. if 1 < 90 or 1h > 270" then
26:
27:
Gupdate Geurr
No conflict with knowledge of previous domains.
28:
else
Decompose Geury into Gearr and G which denote component of Geurr parallel to Gprev and perpendicular to Gprev respectively.
29:
30:
Gupdate + G
31:
end if
32:
optimizer.step()
33:
Obtain or from Equation 9 with t' iter. Compute Yt
34:
ΣV ax
Compute MBMA t (1 7MMBMA + +y Mr
35:
36:
iter iter +1
37:
end for end for
38:
39:
Use MBMA for inferencing on domains (D1 ... DD) seen so far.
40:
end if
41:
end for
27

---

Published in Transactions on Machine Learning Research (01/2026)
K Other backbone models (CLIP ViT-L)
To asses the model-agnostic effectiveness of our proposed method, conducted additional experiments using the CLIP ViT-L/11 backbone and compared with one regularizer based baseline and gradient approximation technique respectively. We evaluate our method across various levels of supervision, specifically in 1-shot, 2-shot and 4-shot scttings, in Table 11 We observed that our method consistently outperforms the baseline methods across all supervision levels.
Table 11: Performance with CLIP ViT-L/14 backbone. We replace the backbone of PGO-BEn and baseline methods with CLIP ViT-L/14 to assess generality. PGO-BEn maintains the best performance under supervision levels, validating its backbone-agnoste continual adaptation capability.
1-shot
2-shots
4-shots
Method
AA
FA*1
AA*
FA
AA*
FA*1
FeO
76.07
65.71
75:98
65.24
76.78
64.43
LwF
76.59
67.45
InfLORA
76.81
67.50
77:34
67.72
76.95
66.57
Ours
68.75
L Comparison with Zero-shot CLIP with various manual prompt
13 and 14 discuss the performance obtained upon changing the manual prompt. As we observe, the performance varies quite drastically for all the benchmarks. This highlights that, large-scale pretrained models like CL.IP fail to adapt to changing domains, or even fine-grain classification like identifying identity of individual objects. is thus required to design efficient methods to train the pre-trained models adapt this evolving domain scenarios.
In Table 12
Table 12: DomainNet.
Prompt
Real
Painting
Clipart
Sketeh
Quickdraw
Infograph
photo of
83.19
63.00
69.86
64.12
13.91
49.08
Ours (1-shot)
86.37
68.77
75.68
67.87
54.47
Table 13: CDDB-Hard
Prompt
GanGAN
BigGAN
Wild
Whichfaceis
SAN
photo of a
image
57.75
32.75
52.01
68.50
50.60
image
57.00
53.75
51.58
68.50
49.40
Ours (1-shot)
90.45
86.12
56.51
70.50
61.44
Table 14: CoRE50
Prompt
s1
s3
s6
s8
a photo a
11.67
12.20
9.20
11.33
10.53
8.20
10.20
9.83
there is a in image
14.13
13.73
11.97
11.37
12.77
10.50
12.90
13.73
image
13.33
11.80
11.38
11.10
9.27
11.37
Ours (1-shot)
88.67
77.23
75.26
80.33
40
68.53
73.30
83.96
M Result
In this scction we expand out the results on DomainNet, CDDB-Hard and CoRE50 dataset that we have shown in Table 2 of the main paper. We report the average AA* and FA* across 1, 2, 4, and shots in the main paper. Here detail them individually.
28

---

Published in Transactions on Machine Learning Research (01/2026)
Table 15 details results on NomainNet. Table 16 details obtained results on CDDB-Hard and Table 17 details obtained results on CoRE50 dataset.
The results, averaged over three seeds, are in the next page owing to the orientation.
29

---

Published in Transactions on Machine Learning Research (01/2026)
Table 15: Comparison with existing DIL benchmarks on Domain Net. dataset across 1, 2, 4 and 8-shots. We report the AA* and FA* values for comparison, which indicate Overall Average Accuracy' and 'Overall Forgetting Alleviation! The highest performance is shown in bold, with the socond highest inderlined. PGO BEN performs superior compared all the basclines across the varying levels of supervision, highlighting the
FA*(1)
18.5820mm ex
57.67 4.
55. 35.61e6.00
04. 04.58
59,00 39.05=m.2
64.24
58, 58.03. c..5
66.93 06.993...
+2.35
59.58
18.
8-shot
AA*(t)
=9 m
=0.57
68.14
74.44=0.4
74.47 =6.57
75.5 e0.17
+1.07
71.77
71.47
70
FA*(1)
22.57 20
57.85-1.2
54.82 2m
64.00=m.s
58 =m.
63.66 m0.00
60.04 0.05
64. 64.48=m..
60.17
+0.48
61. 12.82
4-shot
AA*(1)
35 10.
72. 02.08.um
70. 10.43
67 07.44an.2
73 23.81
00 80emaa
73.8 a3
71 1L833.c0
72. 22.52.na:
74.93
+1.12
FA*(1)
18 18.2020.0
59.04
54.92
63.18
65.81
63 13.10.
60 60.90...77
63.92
+0,74
61. 11.26
60 27
2-shot
AA*(1)
29.202 50
20.99
67.18
73.10
07.52
73.22 60.0
72.05
71.58
74.36
+1.26
72.26
73.22
70.00
effectiveness of the proposed methodology, * indicates reimplemented with CLIP.
F1*(1)
18.722909
61.51.1
60.87
53. 13.10
62. 62.25
50. 18
62.3 02.38
61 11.791m.
58 18.16.am
13.76x9.3
+1.11
I-shot
AA*(1)
29. 29.94gmm9
72.13
71.71
05.58
72. 72.50
62. 22.284mm
72. a2
72.39 10.00
74.27
+1.7
70.02
Backbone
VIT
CLIP
CLIP
Prompt pool
x
X
V
V
X
X
DualPrompt Wang et aL. 2022b
2023
EveC Kirkpatrick of aL 2017
S-Prompt Wang or aL 20
Dy Tox Douillard et al. 2021
CODA-PrompL Smith et AL.
InfLORA ORA Laangg & LA 2004
2027
L2P Wang ot aL 2022
LwF Li & Hoiem 2017
CP-Promp(ren Freng
PGO-BEn
Method
30

---

Published in Transactions on Machine Learning Research (01/2026)
Table 16: Comparison with existing DIL benchmarks on CDDB-Hard dataset across 1,2,4 4 and 8-shots. We report. the AA* and FA values for comparison, which indicate Overall Average Accuracy' and 'Overall Forgetting Alleviation. The highest performance is shown in bold, with the second highest underiind.. PGO- BEN performs superior compared to all the basolincs across the varying levels of supervision, highlighting the
FA*(1)
52.50
69.09 -0.00...02
65. -0.443mmm
67.33=ox)
61 =0.30
50.30 50.35=em
61.31 o.9
77.7 1 ee.1
+8.62
61.58
62.04.c2
8-shot:
AA*(1)
56. 1820.44
71.16.n.r
78. 25.09
73.27
75. 15
67.74 77.742m.2
09.34mm00
73,70 10.00
67.20 07.26
84. 84.22
+9.07
F1*(1)
51 11.5020a
65 00.52a0.0
67 77.44...00
60 79.02s
58. 1..00.....
76. 20.79
+6.16
70 11.63x.m
61. 26
61. 48
4-shot
AA*(1)
55 20.8229.0 26
78.10
73.24
73.75
20.5
70.33 70.33
68 18.6612.4
66.7 66.73.am1
83.69
+5.50
71.31
FA*(1)
53.3429
59.24
35.99
68 2
65. 05.98
80
60.87
55.05
62.95
71.84
+3.70
2-shot
AA*(1)
a1.9
65-32
74.84
72.4 72.47
04 23
66, in.9
77.78.0.
+2.04
67. 17.82.1m
61 11.58
65.
71
effectiveness of the proposed methodology * indicates reimplemented with CLIP.
FA*(1)
10.19
53.88 56
53. 13.1 36
08.00
05.3 65.32
23 ST
60 10.80
61. 11.03
66.3 a6.344..14
+1.02
52.21
55.
1-shot
AA*(1))
58, 79.792.m 6
62.71.666
64.54
72. 12.12=0
03 03.08=0.4
71.24 6
74.68
+2.56
62.69
06.87.6.
Backbone
ViT
CLIP
CLIP
Prompt pool
x
DualPrrmpt Wang al. (202bb)
2023
EwC BwCKKERS Kirkpariik et al 2017
S-Prompt Wang aL 2022
Dy Tbx Douillard et al 2021
CODA-Prouie Poopoo Smith cf aL.
InfLORA Liong & LA 2021
CP-Promp Peng 2022
L2P Wang et al 20222
LwF Li & Hoiem 2017
PGO-BEn
Method
31

---

Published in Transactions on Machine Learning Research (01/2026)
Table 17: Comparison with existing DIL benchmarks on CoRE50 dataset across 1, 2,2, 4 and 8-shots. We report the AA* and FA values for comparison, which indicate 'Overall Average Accuracy and 'Overall Forgetting Alleviation: The highest performance is shown in bold, with the second highest inderlined. PGO BEN performs superior compared all the basclines across the varying levels of supervision, highlighting the effectiveness of the proposed methodology, * indicates reimplemented with CLIP.
FA*(1)
30 20.
63.24
57. 07.62.c.m 4.
78.06
58. 38.1620.7
78.77 78.77=007
47. 17.89
66 14.10... m.7
82.79. c.
85.342.0.0
+2.55
8-shot
AA*(1)
48.33 20
65, =0 80
79.00
60. 73
13
50,0 50.97 m0
88, -3.43
+4.35 4.35
67. 62.62
73. 73.12.
81,08
FA*(()
27.30
54.80
77.98 77.98
52.39=m.x
77. 27.26=aass
44.95 mm.a
83.81 83.81
+2.55
57.20
58,27
81.26
4-shot
AA*(1)
45. 14goar
61.61.0.
63. 35.55.n.
79. 79.422nos
58 28
79 03
57.55 5m.77
63 00.00
87 97.38
82.27
+3.11
FA*(1)
29 20.83
18.1120.41
79.55
55.28
74.95
40.82
82.81 $0
57.48
57 57.69
78.68
+3.26
2-shot
AA*(1)
17.75
66.03
66.47
50.00
80.94
50.58
79.53 70.53
55 29.00
66.41 66.47.1.5
81.65
86. a3
+5.08
66
F1)(1)
20.9421.2 34
52.99 22
51.8920.0
77 87.85
30.35
74.262mm7
40.77 40.77
50.361
77.231aaa
78.1 4
+0.29
I-shot
AA*(1)
15.062o.0
59. 40.4
58. 434
80 +0
43 83.83
77. 47.03
53. a3.79
174441057
83. 83. 2
+3.00
78.28.m
Backbone
VIT
CLIP
CLIP
Prompt pool
x
X
V
X
X
DualPrompt Wang et aL. 2022b
2023
EveC Kirkpatrick of aL 2017
S-Prompt Wang or aL 20
Dy Tox Douillard et al. 2021
CODA-Prompp Smith et AL.
InfLORA Laang & LA 2024
CP-Promp(Peet Feng Feng 2027
L2P Wang ot aL 2022
LwF Li & Hoiem 2017
Method
Ours
32

---

Published in Transactions on Machine Learning Research (01/2026)
N Detailed results for all datasets
We detail the results of the performance of PGO-BEN on DomainNet Peng et al. (2019) in Table 18
CDDB-Hard Li et al. (0033 in Table 19 and about the CoRE50 dataset Lomonaco & Maltoni 2011 in
Table 20 across 1, 4, 8 and 16 shots.
Table 18: Performance change across varying levels of supervision for DomainNet dataset with seed 2.
(a) I-shot
Real
Painting
Clipart
Sketch
Quickdraw
Infograph
Real
88.52
Painting
87.
69.52
Clipart
87.57
69.46
75.15
Sketch
87.28
69.78
75.86
67.43
Quickdraw
85.45
67.66
75.62
66.34
21.09
Infograph
86.37
68.77
75.68
67.87
20.84
54.47
(b) 2-shot
Real
Painting
Clipart
Sketch
Quickdraw
Infograph
Real 88.52
Painting
87.98
70.16
Clipart
87.58
69.91
75.86
Sketch
87.59
69.83
75.87
67.55
Quickdraw
85.70
68.28
75.29
67.59
20.82
Infograph
86.64
69.64
75.48
67.28
20.53
54.98
(c) 4-shot
Real
Painting
Clipart
Sketch
Quickdraw
Infograph
Roal
88.52
Painting
86.93
72.48
Clipart
87.57
71.38
76.25
Sketch
86.92
70.98
76.24
68.74
Quickdraw
86.49
68.92
75.21
67.50
25.67
Infograph
86.94
69.42
75.72
67.49
22.98
55.73
(d) 8-shot
Real
Painting
Clipart
Sketch
Quickdraw
Infograph
Real
88.52
Painting
87.42
72.13
Clipart
87.11
72.90
76.36
Sketch
86.76
71.65
76.54
68.59
Quickdraw
85.82
69.41
76.38
68.29
29.53
Infograph
87.21
72.25
75.98
67.82
27.15
56.84
33

---

Published in Transactions on Machine Learning Research (01/2026)
Table 19: Performance change across varying levels of supervision for CDDB-Hard dataset, with seed value=2.
(a) 1-shot
GauGAN
BigGAN
Wild
WhichfaceisReal
SAN
GauGAN
98.90
BigGAN
89.15
66.75
Wild
69.35
68.12
45.95
WhichfaceisReal
51.00
53.00
51.04
50.25
SAN
53.75
56.87
52.30
50.25
63.85
(b) 2-shot
GauGAN
BigGAN
Wild
WhichfaceisReal SAN
GauGAN BigGAN
98.85
91.40
91.37
Wild
79.60
76.87
55.21
WhichfaceisReal
80.05
83.87
55.45
67.75
SAN
63.65
47.37
51.09
42.00
45.78
(c) 4-shot
GauGAN
BigGAN
Wild
WhichfaceisReal SAN
GauGAN BigGAN
98.80
96.15
79.62
Wild
90.70
84.37
64.03
WhichfaceisReal
85.60
86.00
64.42
80.25
SAN
90.45
86.12
56.51
70.50
61.44
(d) 8-shot
GauGAN
BigGAN
Wild WhichfaceisReal
SAN
GauGAN BigGAN
98.95
95.35
90.50
Wild
86.55
83.12
63.06
WhichfaceisReal
87.95
87.12
62.43
81.50
SAN
82.50
82.87
65.87
74.25
54.21
(e) 16-shot
GauGAN
BigGAN
Wild
WhichfaceisReal
SAN
GauGAN BigGAN
98.95
94.60
94.50
Wild
74.95
84.00
61.85
RhichfaceisReal
93.60
88.50
59.91
83.00
SAN
93.90
88.75
63.59
83.25
59.03
34

---

Published in Transactions on Machine Learning Research (01/2026)
Table 20: Performance change across varying levels of supervision for CoRE50 dataset with seed
value 2.
(a) 1-shot
s1
s2
83
84
85
86
87
88
98.00
#2
91.70
80.93
s3
92.50
81.83
80.86
92.30
80.76
79.67
82.93
s5
90.73
80.23
78.90
79.67
84.27
s6
89.16
75.76
74.93
78.33
78.
79.30
S7
88.33
74.43
72.87
78.56
74.53
71.70
77.13
s8
88.67
77.23
75.26
80.33
76.40
68.53
73.30
83.96
(b) 2-shot
s2
84
86
s8
s1
98.00
s2
95.00
82.20
s3
95.00
82.33
74.80
84
93.33
80.90
80.00
87.50
45
94.06
79.17
77.27
80.40
86.30
91.00
77.27
73.96
79.63
81.50
82.53
S7
91.23
80.57
75.97
81.37
79.63
80.86
80.76
s8
91.56
80.50
79.03
82.67
81.50
78.60
80.17
86.60
(c) 4-shot
s1
s2
s3
s4
s5
s6
87
88
98.00
#2
92.63
80.13
s3
93.20
81.33
84.03
s4
93.50
79.76
82.56
87.96
79.63
80.30
82.40
s6
93.43
78.13
78.00
82.80
80.70
83.20
47
94.13
81.36
78.60
84.93
83.13 13
80.00
82.90
93.77
82.80
80.00
83.66
83.36
13
80.43
88.60
(d) 8-shot
81
82
83
84
85
86
87
88
98.00
#2
92.13
87.27
92.87
85.67
87.57
94.07
84.50
84.40
90.27
85
94.20
84.20
83.10
86.50
91.20
s6
92.37
83.40
78.33
85.37
85.03
88.57
S7
93.40
84.47
82.47
85.80
85.33
83.33
s8
92.57
83.
82.87
86.27
85.20
80.37
82.87
91.23
(e) 16-shot
s4
86
s8
81
96.47
78.23
76.87
s3
87.13
81.03
84.63
s4
88.43
81.50
80.60
88.53
s5
88,23
82.10
79.33
84.90
90.80
81.33
75.40
81.70
85.77
87.30
87
88.80
82.20
77.57
81.77
83.97
82.07
86.07
48
89.60
83.03
80.20
85.47
85.67
79.73
81.90
90.47
35

---

Published in Transactions on Machine Learning Research (01/2026)
O
Prompt depth
We experiment with the depth of encoder blocks that we synergize with the proposed Encoder synergy module. We experiment with J = 1,3, 5,9 and 11. As we see in Table 21 as we increase the depth of the Encoder synergy module, we see the performance increases, which indicates that the model able to use CLIP pre-trained knowledge better, and hence, representation learned and better stability of performance. But we go to the last. block, where the features are already mature, see dip in performance, which aligns with observations made in Khattak et al. (0222
Table 21: Encoder synergy depth ablation
Shots
J=1
==
==
==
J=
AA*(")
(A*
(1)
(A*
(t)
FA*(1)
(A)
FA*(1)
(A)
FA*(1)
1-shot
73.22
62.58
72.32
60.71
73.10
62.23
74.27
63.76
73.34
63.12
2-shot
73.39
62.39
72.95
61.76
73.67
63.02
74.36
63.92
73.45
63.23
4-shot
73.58
62.65
73.53
62.56
73.83
63.30
74.93
64.48
73.49
63.45
8-shot
73.67
62.73
73.58
62.61
63.32
75.51
66.93
73.92
63.87
P Prompt Length ablation
Prompt length an important hyperparameter of PGO-BEn. We detail the changes the figure
CDDB-Hart
CoRE50
90
Avenage
Average FA*
Performanee %
85
Performunc
70
60
Average AA"
Average
Prompt tokeus
Prompe
Figure 8: Prompt length v/s AA* and FA* measure by taking average of 1, 1,2,4 4 and 8-shots.
As observe with very high prompt length, the performance dips.
Q Novel classes during inference
We experiment the scenario where the model encounters novel classes during inference time, and compare the results in Table 22 This a very practical scenario in different use cases. Following the experimental setup of Khattak al. (2022): Zhou et al. (2022bla) a), separate the of classes of every domain into two groups, Base and New. During training, the model observes base classes, and we evaluate the performance base classes and new classes.
We implemented the LwF, InILORA and our method in this experiment with DomainNet datasel in 1-shot and 4-shot, where we learn the context vectors in the base class and for inference, we change the [CLS] token in the prompt Pr, to perform inference on new classes. Our method achieves superior stability as compared to other basclines, highlighting the superiority of our method and the applicability our method in world scenarios where come across new classes after deployment, like autonomous driving.
36

---

Published in Transactions on Machine Learning Research (01/2026)
Table 22: Comparing performance of PGO-BEN with two baseline methods (LwF, InfLORA) towards recognizing novel classes during inference.
Base
New
Shots
Method
AA*(1)
FA*(1)
AA*(1)
LwF#
72.04
66.77
70.29
InfLORA
70.77
63.70
68.76
1-shol
PGO-Ben
77.74
68.51
76.00
LwFF
71.79
69.04
70.10
InLORA
68.87
64.93
67.51
4-shol
PGO-Ben
78.88
69.58
77.08
Experiments with more seed values
We discuss the results with more number of seed values in this section. The average 1, 2, 4 and 8-shot performance across five different seeds are detailed Table 23 The results for DomainNet dataset is Table 24 CDDB-Hard results in Table 25 and CoRE50 results in Table 26
Table 23: Comparison across NomainNet, CDDB-Hard, and CoRE50 averaged over 1, 2, 4, and 8-shot settings. Bold and underlined denote the best and recond-best scores. PGO-BEN outperforms all baselines without using prompt pools, demonstrating its generalization strength. indicates CLIP-ViTI CIIP-VIBB/II B/66- based reimplementation. Results are mean + std over 5 seeds. Red font denotes least std method.
DomainNet
CDDB-Hard
CoRE50
Method
Average
Average
Average
Prompt
AA*(1)
PAT(1)
AA*(1)
PA*(1)
AA*(1)
FA*(1)
Dy/Tox Douillard Al.
2021
VIT
31.601882
18.74g060
67.24g0000
53.541081
47.1920.09
29.48g1.0
Zero abot CLIP nadford et
2021
CLIP
69.05
56.32
12.67
LwF*
Holem
CLIP
12.211n866
60.7710.6
58.8510.62
64.8010.66
67.7610.66
70.8810.94
70.9990.91
62.2510.84
G1.57:0.72
63.54:10.48
L2p
Walg et ai. 20222
87.29+9.54
54. 70.7000.4
71.45+1.04
64.03+0.94
80.17+0.94
78.44+0.97
Duall Trmpt Waanc es
2022b
13.51#um4
63 50
72.85
66. 46
55.87=0.a9
50.89-00.4
d-Promps
Wain
67 87.8444
56.27
65.602...1
60 60.80
70. 70.4420.s4
76 29.22..
CODDA-Prampl
72.15.10.70
70..00...00
67.0310.55
57.4110.80
65.1510.07
68.15110.76
CP-Prompt
Folg mi.
4024
72.191.085
037777
66.88.0.41
62.21:0.25
11.68oo.77
PGO-BEN (Ours)
A
+1.18
+1.42
+6.84
+6.15
+4.70
+2.56
As see, there changes to the relative ordering of the baseline methods with us, with our method clearly superior across all scenarios. The previous table with 3 seeds is mentioned in Table 2
37

---

Published in Transactions on Machine Learning Research (01/2026)
Table 24: Comparison with existing DIL benchmarks on Domain Net. dataset across 1, 2, 4 and 8-shots. We report the AA* and FA* values for comparison, which indicate Overall Average Accuracy' and 'Overall Forgetting Alleviation! The highest performance is shown in bold, with the socond highest inderlined. PGO BEN performs superior compared all the basclines across the varying levels of supervision, highlighting the
FA*(1)
18. 18.5820mm er
57.71 45
55.4 05.41=6a:
04.10 44.19
50, 00.022...2
64, 14.72-c.m
59.30 6.72
66.90 26.90
+2.18
59.09
62. 82.87
8-shot
AA*(1)
01.01 =0
70.26
1:01
68.24
74.40
71.05=0.
74. 74.49 11.40=c. .0.5
75. 15.63
+1.14
72.17
72.03
73,07
70.
FA*(1)
22.50
60.17
57.80 $7.89
54.98
64.28
58 2m..0
63.4 =0.07
64.71 64.71=0034 20.34
+0.43
60.68
62.01
4-shot
effectiveness of the proposed methodology, * indicates reimplemented with CLIP. Results are mean I std of seeds.
AA*(1)
36 16
71 pm
70.41.000
67.97
73 73.67
00 23
73.6 f3.68
71 10.32
74.95
+1.27
72. 12.68.uu
(1)(1)
18 18.8020a1
61.31. 61.31007
50.80
55 85.26
63. 13.10
55.95
62.99
14..3
+0.08
61.00 00.00
60 86
2-shot
AA*(t)
29.6 29.
72.45
71.02
17.36
73.21
67 82
a3.55
72.11
71 71.281a8
74.68
+1.13
67.3
F1)(1)
18.9029.
61.3
61. 11.032000
53. 15
62.35
50. 40.55
62.91
61 01.6716m 16.00
63.9 03.98.9.2
+1.07
58 79,79.500
I-shot
AA*(1)
30.2 1077
72. 1220
71. 1.88
65.59
72.77
62. 22.064mm
72.9 a229
74.14
70 10.88100
+1.1
72.83 83
Backbone
VIT
CLIP
CLIP
Prompt pool
x
X
V
V
X
X
DualPrompt Wang et aL. 2022b
CODA-Prompl Smith eL aL. 2023
EvC* Kirkpatrick et al 2017
S-Prompt Wang or aL 2012a
Dy Tox Douillard et al. 2021
OnfLORA Liong & 2012
CP-Prompt Feng 2022
LwF" Li & Hotem 2017
L2P Wang ot aL 2022c
PGO-BEN (Ours)
Method
38

---

Published in Transactions on Machine Learning Research (01/2026)
Table 25: Comparison with existing DIL benchmarks on CDDB- Hard dataset across 1,2,4 4 and 8-shots. We report. the AA* and FA values for comparison, which indicate Overall Average Accuracy' and 'Overall Forgetting Alleviation. The highest performance is shown in bold, with the socond highest inderlined. PGO BEN performs superior compared all the basclines across the varying levels of supervision, highlighting the effectiveness of the proposed methodology, * indicates reimplemented with CLIP. Results are mean + std of 5 seeds.
FA*(()
53.11 1100
60.34 3d.....
08.0221.00
66.51
60.72
62 22.06.o..
58 38.82.00c
77.02
+9.00
62.48
63.01
8-shot:
AA*(1)
55 43.0.3
(7.33200.
72. 72.48811.0
74. 74.34amm
68 26.522m22
70.48
72.9110m
84.62
70.02
66. 16.48..m
+7.09
72.90
F1*(1)
51 12.02..00
62. 12.10.....
71. 71.48% 48.0.2
64 (4.88
08 23.20
.0.000
50 01 20.34
21.26 Lo.71
62 i0.30
+4.43
50.20
4-shot
AA*(1)
56.71 20
71.92
77.06s1..
74.11
22.49
66. 06.03
69.7 s9.u
60 60.48185
67.51
82.89
+5.83
72.
FA*(1)
54.1220.
55.21
67.31
65.01
14
61.414o.4
54.22
63 13
71.12.9.3
58.22
+3.81
2-shot
AA*(1)
57.11a1.5
a6.4
66. 1.34
73.92
73 79.19
04 04.924u73
72.01
12.
66. 10:3
77.14.0.5
+3.22
66.
FA*(1)
55.942 o7
51.74.c
54.31
57. 43.4
05. 15.82
59. 12.122
61.72 61.72 T7
53.71 53.71
60. ie.3
66.42 42
+0.60
1-shot
AA*(1))
M4
63.89
71.41=0
02.04
70.14
63.39 ..34
1
+2.70
67. 17.42.
Backbone
ViT
CLIP
CLIP
A
Prompt pool
x
DualPromp Wang al. 2022b
Campppssin Smith of aL. 2023
Kirkpatrick et al 2017
Dy Tox Douillard et al. 2021
Wang S-Prompt Wang aL 2022a
InfLORA Liang & LT 2027
CP-Prompt Feng ml. (2027
2017
L2P Wang et al 20222
LwF Li & Hoiem
PGO-BES BEN (Ours)
DODA-Prompll
Method
OwCO
39

---

Published in Transactions on Machine Learning Research (01/2026)
Table 26: Comparison with existing DIL benchmarks on CoRE50 dataset across 1, 1,2, 4 and 8-shots. We report the AA* and FA* values for comparison, which indicate 'Overall Average Accuracy and 'Overall Forgetting Alleviation: The highest performance is shown in bold, with the second highest inderlined. PGO BEN performs superior compared all the basclines across the varying levels of supervision, highlighting the
FA*(1)
31 31.4520mm es
63. 03.40....
$7.24 20000
78.06
58. 28.46=0.z
78,78 78.79-0.20
48. 28.1551..
$2.72. 6.30
=5.37
66 16.24.c..
+2.65
8-shot
AA*(1)
20
65.41 10
79.00
60 =0.
-0
60 s1.M
88, 18.4620.0
+4.29
67. 50
81.17
73.
FA*(()
27.98.1.00
54.83-1 20.3
77.98=1.s
52.72=0.s
77.34=mm
44,80 mm.00
$3.77
+2.58
57.21
58,21
81.19
4-shot
effectiveness of the proposed methodology, * indicates reimplemented with CLIP. Results are mean + std of 5 seeds.
AA*(1)
78.78g
64 ...92.m..
63.8 03.84.n.00
79. 79.32en.s
58 28.67
79. 70.72.m
57. 97.96sma0
63 78.78
82. 12.40
87 .7.34
+4.04
F1*(1)
30.67
57.31
18.11
79.88
55.62
74. 74.88
41 11.62
82, 25
+2.87 2.87
57.72
78.65
2-shot
AA*(1)
48 18
66. 23.7
66 20.38
20
50 20
70 20.94
55 a0
66, 16.4611111
81.78.06
86.64
+4.86
F1)(1)
27.802111 38
53 a9
51. 01
77 1
30. 80.78
74. 18
90.61am.2
50. 10.4
77.281
78.22.9.
+0.37
I-shot
AA*(1)
45. 2000
59.95
58 2030
81 05
44 44.10
77. 77.84
54.25
57.18
78. 10.4
83.11 +2.3
+1.06
Backbone
VIT
CLIP
CLIP
Prompt pool
x
X
V
X
X
DualPrompt Wang et aL. 2022b
CODA-Prompl Smith OL aL. 2023
EvC* Kirkpatrick et al 2017
S-Prompt Wang or aL 20122
Dy Tox Douillard et al. 2021
OnfLORA Liong & 2012
CP-Prompt Frng 2022
LwF" Li & Hotem 2017
L2P Wang ot aL 2022c
PGO-BEN (Ours)
Method
40


Reviews:
[{"review_id": "AVpvmgxChT", "reviewer": "Reviewer_XRrb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces PGO-BEn, a framework designed for FSDIL, which leverages pre-trained CLIP models via (i) a novel multi-modal prompting strategy that conditions text prompts on vision prompts, (ii) a proxy-guided orthogonalization mechanism for regularizing gradient updates to preserve prior knowledge, and (iii) a Beta-function-based temporal ensembling technique that adaptively weights intermediate model states to mitigate forgetting.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. Table 2 presents comprehensive results across three challenging datasets and four few-shot settings.\n2. The paper includes a PAC-Bayesian analysis justifying the framework's design, as well as detailed mathematical explanations for the orthogonalization and ensembling components.", "audience_yn": "Yes", "audience_explanation": "FSCIL is an important issue in the ML community.", "requested_changes": "1. Can the authors provide empirical or qualitative comparisons to the most recent prompt-based or language-augmented FSCIL works (e.g., PL-FSCIL, Language-Inspired Relation Transfer), and clarify what is gained versus such approaches? \n2. Non-prompt strategies, alternative ensembling, and distributional shift adaptation methods are omitted\n3. The computational cost for beta-ensembling and gradient computations across many domains is not quantified.\n4. What is the empirical/memory/computational overhead of PGO-BEn when N (the number of domains) or the prompt-token space grows much larger?", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "R7dKbUA0gs", "reviewer": "Reviewer_icEg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1.This paper addresses a realistic problem: pre-trained vision–language models often lack access to data from previous training tasks when performing few-shot domain-incremental learning (FSDIL), leading to catastrophic forgetting and limited plasticity. This challenge arises in multiple real-world applications, such as autonomous driving and medical imaging.\n\n2.The paper proposes PGO-BEn (Proxy-Guided Orthogonalization and Beta Ensembling), a rehearsal-free framework for FSDIL. It consists of two key components:\n\n(2.1) Proxy-Guided Orthogonalization (PGO) identifies conflicts between the current update and previously learned knowledge, and mitigates the conflicts by projecting the conflicting gradient into an orthogonal subspace, and\n\n(2.2) Beta Ensembling (BEn) stabilizes training by maintaining a Beta Moving Average (BMA) of previous intermediate models.\n\nPGO-BEn is supported by both theoretical and empirical evidence.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Overall, the claims are clear and generally convincing, though certain experimental design choices limit the accuracy of the empirical evidence.\n\n**Evidence of contribution 1:**\n\n(a) The paper defines the Few-Shot Domain-Incremental Learning (FSDIL) problem clearly. Section 1 (the second paragraph on page 2) introduces the concept, and Figure 2(a) provides an overview of the pipeline. A formal definition follows at the beginning of Section 3 (page 4).\n\n(b) The proposed model update procedure closely follows the FSDIL setting, relying solely on data from the current domain and previously learned model states, without requiring access to data from earlier domains. Equations 1–6 and the first equation on the top of page 6 confirm this property.\n\n**Evidence of contribution 2:**\n\n(a) The paper provides a step-by-step explanation of PGO-BEn in Section 3.1. The pseudocode is provided in Algorithm 1 in appendix J.\n\n(b) The method is supported by a PAC-Bayesian theoretical analysis in Section 3.2, which establishes a risk bound for PGO-BEn.\n\n(c) Figure 1 quantifies PGO-BEn’s performance across multiple domains, demonstrating improvements over a zero-shot CLIP baseline.\n\n(d) Table 2 and Figure 4(b) empirically supports the performance of PGO-BEn on three benchmarks, DomainNet, CoRE50, and CDDB-Hard. The performance is quantified using two metrics: average accuracy and forgetting alleviation. PGO-BEn consistently outperforms baseline methods on both. The detailed metric definitions are provided in Appendix H.\n(e) Figure 4(a) compares PGO-BEn’s representation alignment to baselines, highlighting that the proposed method learns more aligned representations than baselines.\n\n(e) The paper conducts comprehensive ablation studies to support each component’s contribution. \nFigure 5 demonstrates that BMA, a component of PGO-BEn, is more robust to large domain shifts compared to  exponential moving average (EMA).\n\nTable 3 demonstrates BMA’s higher average accuracy and forgetting alleviation compared to EMA.\n\nTable 5 suggests that PGO and BMA both contribute to the stability of the method. Using only one of them causes a performance degradation.\n\nTable 6 empirically shows the advantage of the proposed multimodal setup in prompting configuration.\n\n(f) Table 4 discusses sensitivity to a key parameter, analyzing performance under both lower and higher values.\n\n(g) Detailed implementation information is included in Appendix I.\n\n**Issue and limitation:**\n\nA notable weakness lies in the accuracy of experimental results. Table 2 reports results using only three random seeds, which is insufficient to provide statistically stable mean and standard deviation (STD) estimates. Increasing the number of seeds would improve the accuracy of the reported numbers. Similarly, Figure 6 omits information about the number of seeds used, making it difficult to assess the reliability of the reported variance.", "audience_yn": "Yes", "audience_explanation": "The problem addressed in this work is relevant to real-world applications, particularly when training domains contain sensitive data or when memory resources are limited.\n\nThe paper considers the practical applicability in design choices. The rehearsal-free setup aligns with realistic constraints where previously seen data cannot be reused due to privacy or storage limitations, broadening the potential impact of the method. Moreover, Equation (5) presents an efficient online implementation of Beta Moving Average (BMA) that reduces memory overhead. Additionally, the orthogonal projection step in Equation (3) adds minimal computational cost without introducing notable concern regarding the Big-O analysis.", "requested_changes": "The empirical evidence could be made more accurate and convincing by using additional random seeds, particularly when reporting the average accuracy and forgetting alleviation. Additionally, reporting the number of seeds used in all experiments would improve reproducibility.\n\nThe letter $n$ in PGO-BEn is inconsistent, appearing in both lowercase and uppercase. It would be clearer to adopt a consistent format.", "broader_impact_concerns": "N/A", "additional_comments": "I would appreciate if the authors could explain more on the learnable prompts introduced across encoders. A learnable token may cause the distribution shift of inputs even in the same task. Would this shift be a cause of instability during learning, and if yes, how does PGO-BEn mitigate that?", "extra_scores": {}, "extra_text": {}}, {"review_id": "cNFaMpx9Co", "reviewer": "Reviewer_R6Eh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a new problem: Few-Shot Domain-Incremental Learning (FSDIL). It differs from prior continual learning settings (such as DIL or FSCIL) as it addresses the practical challenge of adapting models to a sequence of new domains using only a few labeled samples, without access to past data.\n\nTo solve this problem, the paper proposes a novel learning method: PGO-BEn. Comparing to previous work, this method has the following novelties:\n1. Proxy-Guided Orthogonalization (PGO): Unlike prior gradient project methods that require storing past gradient/data, PGO uses the current model (from last session) as a proxy for past knowledge and projects conflicting updates into an orthogonal subspace to prevent forgetting.\n2. Beta Ensemble (BEn): Instead of the standard Exponential Moving Average (EMA), the paper proposes an ensemble strategy that leverages a Beta distribution to adaptively balance weights between early model states (preserving old knowledge) and latest model states (adapting to the new distribution).\n\nThe paper conducted extensive evaluations to show the efficacy of the proposed method PGO-BEn. Various ablation study are also conducted to validate the contribution of the proposed components.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. The introduced problem FSDIL is different from prior challenges such as DIL and FSCIL, and the authors clearly discussed their differences.\n2. The proposed method has two major contributions: PGO and BEn. The similarity and differences between prior arts are discussed in the paper. The efficacy of the proposed method are validated in the main results (Table 2) and the ablation study (4.2b, 4.2d, and 4.2e).", "audience_yn": "Yes", "audience_explanation": "The problem investigated in this paper, FSDIL, in which a model needs to adapt to a sequence of new domains using only a few labeled samples per class, with the constraint of being \"rehearsal-free\", is interesting and differs from prior work. The paper provides a benchmark datasets and the related metrics for evaluation.\n\nThe proposed PGO-BEn is a novel yet effective method for this specific problem. In particular, the Beta moving average is a simple but quite effective method to preserve prior (and the base) model knowledge, which might be very useful in similar settings (like generic CL).", "requested_changes": "The graphical representations in this paper contain non-standard scaling choices that potentially distort the interpretation of the data. In Figure 1, the radar graph's visual magnitude is compromised by a non-zero or inconsistent origin point, causing the result of 20.84 to appear disproportionately larger than 68.77. Furthermore, Figures 4, 5, and 6 employ a truncated Y-axis (non-zero baseline), which exaggerates the perceived magnitude of small differences. Specifically, Figure 5 (Real -> Sketch) displays values of 32.06 and 31.95 (a difference of 0.11), yet the Y-axis begins at 31.9, visually inflating the modest performance gap. \n\nTo maintain clarity and strengthen the communication, I would like the authors to update the figures to use standard axes.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)