Unlearning during Training: Domain-Specific Gradient Ascent for Domain Generalization
Authors: Di Zhao
Jingfeng Zhang
Hongsheng Hu
Philippe Fournier-Viger
Gillian Dobbie
Yun Sing Koh
Venue: ICLR 2026 Poster (2026)
Keywords: Unlearning
Transfer Learning
Domain Generalization
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Deep neural networks often exhibit degraded performance under domain shifts due to reliance on domain-specific features. Existing domain generalization (DG) methods attempt to mitigate this during training but lack mechanisms to adaptively correct domain-specific reliance once it emerges. We propose Identify and Unlearn (IU), a model-agnostic module that continually mitigates such reliance post-epoch. We introduce an unlearning score to identify training samples that disproportionately increase model complexity while contributing little to generalization, and an Inter-Domain Variance (IDV) metric to reliably identify domain-specific channels. To suppress the adverse influence of identified samples, IU employs a Domain-Specific Gradient-Ascent (DSGA) procedure that selectively removes domain-specific features while preserving domain-invariant features. Extensive experiments across seven benchmarks and fifteen DG baselines show that IU consistently improves out-of-distribution generalization, achieving average accuracy gains of up to 3.0\%.

Published as a conference paper at ICLR 2026
UNLEARNING DURING TRAINING: DOMAIN-SPECIRIC GRADIENT ASCENT FOR DOMAIN GENERALIZATION
Di Zhao', .Jinffeng Zhang', Hongsheng Hu², Philippe Fournier- Viger",
Gillian Dobbie' & Yun Sing Koh
University of Auckland, Auckland, New Zealand Shanghai Jiao Tong University, Shanghai, China
Shenzhen University, Shenzhen, China
{di zhao, jingfeng. zhang, g g.dobbie, y y.koheeaccklangg and ac hongsheng huês ftttsstt..duu.c.. edu.cn,
en
ABSTRACT
Deep neural networks often exhibit degraded performance under domain shifts due to reliance on domain-specific features. Existing domain generalization (DG) methods attempt to mitigate this during training but lack mechanisms to adap- tively correct domain-speciff reliance once emerges. We propose Identify and Unlearn (IU), model-agnostic module that continually mitigates such reliance post-epoch. We introduce an unlearning score to identify training samples that disproportionately increase model complexity while contributing little to general- ization, and an Inter-Domain Variance (IDV) metric to reliably identify domain- specific channels. To suppress the adverse influence of identified samples, IU employs Comain-Specifie Gradient- Ascent (DSGA) procedure that selectively removes Nomain--speifff features while preserving comain-invarian features. Ex- tensive experiments across seven benchmarks and 16 DG baselines show that IU consistently improves out-of-distribution generalization, achieving average accu- racy gains of up to 3.0%.
INTRODUCTION
The success of deep neural networks often relies on the assumption that training (source domain) and testing (target domain) data are drawn from the same distribution. However, this assumption is frequently violated in real-world scenarios due to domain shifts (Wang et al. 2022). Domain adap- tation (DA) methods address this issue by adapting knowledge from a source to target domains with limited labeled target data (Pan & Yang 2009). Unsupervised domain adaptation (UDA) further relaxes this requirement by leveraging only unlabeled data from the target domain (Xu et al. 2019). Despite their effectiveness, DA and UDA methods require access to target domain data during train- ing, limiting their practicality Yuue al. 2019). Domain generalization (DG) aims overcome this constraint by training models on multiple labeled source domains to generalize to unseen target domains. Existing methods can be divided into three categories: data augmentation (Zhou et al., 2020), representation learning (Wang et al. 2022), and training strategies (Zhao et al. 2025).
key limitation of existing DG approaches their reliance on training-time objectives prevent the learning of domain-speciffe features, without any mechanism to correct such reliance once it emerges. As result, when models inadvertently capture domain-specific features during training, these methods lack the capability to remove them. Importantly, -peaain-peccffi reliance may arise dynamically at different stages of training (Piratla et al.) 2020), suggesting the importance of an adaptive procedure that operates continually. This raises the central research problem of designing an adaptive mechanism that identifies training samples introducing domain-specific biases and then removes the associated features, while preserving comainn-nvvriian features.
To address this problem, we propose Identify and Unlearn (IU), a model aadel-agnostic module that con- tinually mitigates reliance domain-speent specific features they emerge during training. IU adaptively intervenes after each epoch to detect and remove domain -specffc features acquired by the model. The core idea builds influence functions (Koh & Liang 2017) and principled view of general-

---

Published as a conference paper at ICLR 2026
= 2.5
Vi(C)= = 1.8
(((A)= = 2.8
V((S) = 2.3 Vi(())
Invariant Texture Feature
=
VuCC 0.2
Va(A) 0.8
Specific Texture Feature
Variance for Texture- Channel
V 00 Variance for Baokground-Sensitive Channel
Variance
Variance
Painting Sexture-Sensitive Channel
Cartoon Painting Sketch Background-Sen SSenssiii--e Channel
Figure I: Illustration of Aggregated Variance (AV) versus Inter-Domain Variance (IDV). A texture- sensitive channel that responds to fur edges exhibits consistently high activation variance within each domain naturan fur in photos, bold outlines cartoons, brush-stroke textures paintings. and simplified contour strokes in sketches. AV, which pools variances across samples, incorrectly assigns this channel high score and mislabels as domain-specific, whereas IDV correctly identifies it as channel exhibits domain-dependen variability- in
A background-sensithe photos, moderate in paintings, lower in cartoons, and minimal in sketches. AV underestimates this cross-domain variability, while IDV accurately flags it as domain-specific.
ization (Zhang al. 2024) training samples that increase model complexity without contributing to generalization are likely to introduce domain-specific bias. IU identifies such samples and selec- tively suppresses their domain-specing features while preserving nomain-invvrian features. Specif- ically, IU introduces novel unlearning mechanism that (i) identifies an unlearning set of training samples that disproportionately increases model complexity while contributing little to generaliza- tion. (ii) pinpoints the domain-specine channels, and (iii) removes the domain- -senain- features of the unlearning set by selectively reversing their gradients on the comain--peccfff channels.
To identify the unlearming set, we introduce the unlearning score, which highlights samples that disproportionntely increase model complexity while contributing little to generalization. To identify domain-specifty channels, we propose Inter-Domain Variance (IDV), a domain- -ware metric that explicitly quantifies cross-domain variability. IDV first computes channel-wise activation variance within each domain, and then measures the variance of these variances across domains. Channels with high IDV values are therefore deemed domain-specific. This design makes IDV naturally robust to domain imbalance and less susceptible to conflating uniformly noisy channels with genuinely Nomain-speciff ones. contrast, existing approaches such as Aggregated Variance (AV) (Guo) al. 2023 Yu et al., 20244 suffer from two key limitations. First, AV is highly sensitive domain imbalance, often yielding biased estimates. Second, AV primarily captures within-domain variance rather than cross-domain variability, which can lead systematic misidentification. For example, a texture-sensitive channel that responds to fur edges exhibits consistently high variance across domains. Despite being domain-invariant, AV assigns this channel high score and incorrectly identifies it as comain-speciffe (Figure T. By directly capturing cross-domain variability, IDV offers more principled and reliable criterion for comainn-peecife channel identification.
Our contributions are threefold. (1) We introduce novel paradigm for DG that adaptively removes domain-specific features post-epoch, enhancing the robustness existing DG methods. (2) We pro- pose Inter-Domain Variance (IDV), a principled domain metric that quantifies cross-domain variability, enabling reliable identification of domain-specific channels. (3) We develop Identify and Unlearn (IU), model-agnostic module that adaptively identifies the unlearning set and removes their influence through Domain- -omaainSpeccfic Gradient-Ascent This selective unlearning mechanism mitigates domain-specife bias while preserving comain-invariant features. Extensive experiments across seven benchmarks and fifteen DG baselines demonstrate that IU consistently improves gen- eralization to unseen domains, achieving average accuracy gains of up to 3.0%.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Domain Shift refers to performance degradation caused by discrepancies between source (train- ing) and target (testing) domain distributions (Pan & Yang 2009). To mitigate this issue, Domain Adaptation (DA) methods align distributions between source and target domains (Baktashmotlagh
al.
2013
Luo 2020) or fine-tune models using annotated data from the target domain (Long
et al. 2013 Despite significant progress, DA methods are practically limited by their dependency
on annotated target domain data, which can be difficult to obtain (Yue et al. 2019). Unsupervised Domain Adaptation (UDA) addresses this limitation by transferring knowledge from labeled source domains to unlabeled target domains, eliminating the need for target domain annotations (L.uu al.) 2021p. However, UDA still necessitates data collection and adaptation for each target domain. This constraint motivates the development of approaches capable of generalizing effectively to unseen domains without relying on target domain data during training (Wang et al.
2022
Qiao al. 2026
Domain Generalization (DG) was initially introduced by (Blanchard et al.
2011 and subsequently formalized by (Muandet et al. 2013). Existing DG methods typically fall into three categories: data augmentation, representation learning, and training strategies. Data augmentation methods enhance dataset diversity through techniques such domain randomization (Yue al.] 2019), adversar- ial augmentation (Shankar et al. 2018), and feature interpolation (Sun et al., [022]] (Zhang et al.) 2022). Representation learning methods seek disentangle domain-invariant from domain-specifit
features via methods such as feature alignment (Jin et al. 2020), adversarial learning (Zhao et al.)
2020 or invariant risk minimization (Ahuja et al.J 2021a Training strategy methods optimize model generalizability through different training strategies, including ensemble learning (Dubey ct al. 2021), meta-learning (Balaji et al. 2018), self-supervised learning (Jeon et al., [021). and curriculum learning (Zhao et al. 2024). DG has proven effective across various domains, including re-identification (Li et al. 2025 Wu et al. 2026). A key limitation of cxisting DG approaches is their reliance training- time objectives to prevent the learning of domain-speciff features, without any mechanism to correct such reliance once emerges. To the best of our knowledge, IU is the first framework that explicitly removes dopein-specifie features after they have been learned.
Machine Unlearning (MU) was initially proposed to enable machine learning models to satisfy data deletion requests without requiring full retraining from scratch (Cao & Yang 2015 Bourtoule al.] 2021]. The concept of MU has subsequently extended across various machine learning domains, in- cluding image classification (Ti et al. 2023). image generation (Fan al. 2004), graph neural net- works (Wu et al. 2023), and large language models (Yao et al. (2024). Existing MU methodologies broadly encompass exact unlearning through retraining (Thudi al. 2022), differential privacy- based unlearning (Sekhari et al. 202T), and approximate unlearning via fine-tuning Jia et al. (0023).
Beyond data privacy, recent works have explored MU to enhance model performance in tasks such as continual learning (Wang al. 2024) and domain adaptation (Basak & Yin 2024). However, the potential MU to improve generalization to unseen target domains remains largely unexplored. Crucially, DG presents a fundamentally different challenge from traditional MU applications: rather than forgetting specific data points for safety, the objective is to selectively unlearn domain sppciffc features that impede generalization. This key distinction motivates our work, which, to the best of our Enowledge, is the first to explore machine unlearning as a tool for enhancing DG.
3 IDENTIFY AND UNLEARN FOR DOMAIN GENERALIZATION
We first describe the procedure for selecting the unlearning set (Section 3.1 We then introduce our approach for identifying domain-specific channels (Section 3.2). followed by the formulation of Domain-Specitf Gradient Ascent (Section 3.3). Section 3.4 presents a theoretical analysis of the proposed framework. An overview of the IC module within a single post-epoch is illustrated in
Z and the full algorithm is provided in Appendix D
3.1 UNLEARNING SET SELECTION
Motivated by the principle that, among models with comparable training performance, simpler mod- els tend to generalize better (Neyshabur et al.] 2017 Zhang al.) 2024), we aim to identify and unlearn training samples that disproportionately increase model complexity while contributing min- imally to generalization. To this end, adopt the norm of the model parameters (Neyshabur al.,
3

---

Published as a conference paper at ICLR 2026
Normal Training
Trining
A(-)
fo
Training
fo
Unlearning D.
fo
Demninn Specific Channels Cype
Selective Grudient Ascent
Influence Function
Validition
Inter-Demain
Mean Absolute
Deviation
Unicarning Set Ds
Mean Absolute
Deviation
Domain Domaii-Specific
Cape
(a) Unlearning Set Selection
(b) Domain-Speciffc Channel Selection
(c) Domain- Domain-Specific Gradient Ascent
Figure 2: Overview of the IU module within single post-epoch. (a) Unlearning Set Selection:
Following standard training, the influence function is applied to the training set to estimate each sample's impact model complexity and generalizability. Samples are then selected for unlearning via Median Absolute Deviation (MAD) thresholding, forming the unlearning set D.. (b) Domain- Specific Channel Selection: Inter- Domain Variance (IDV) is computed across the training set to quantify variability in within-domain activation variances. Domain-specific channels Cype are also identified using MAD thresholding.
(c) Domain-Speeific Gradient Ascent: Gradient ascent is selectively applied to domain-specific channels using samples from D., yielding the updated model fā with reduced domain-specific features and improved generalization capability.
2017) to quantify model complexity and leverage the influence function (Koh & Liang 2017 to estimate the impact of each training sample on both model parameters and validation performance. Specifically, we define two metrics: complexity score and generalization score. The complexity score, Cz. defined in Equation quantifies the impact of removing a single training sample x on the model's complexity. This is measured by the f2-norm of the resulting change in model parameters, serving as proxy for model complexity. A higher complexity score indicates a greater influence of the sample on model complexity. Here, 0 denotes the model parameters, H is the inverse Hessian matrix, and V VeL(x, 0) represents the gradient of the loss function with respect to 0. Since we are concerned only with the magnitude of parameter changes, not their direction, the fn-norm employed. This reduces the high-dimensional vector H2 V L(r, 0) to a scalar, enabling direct comparison of complexity contributions across training samples.
(I) The generalization score, G.. defined in Equation []] quantifies the influence of a training sample on the model's performance over the validation set D with higher values indicating a greater positive impact on generalization. Specifically, G is a scalar value computed by aggregating the influence of z across validation samples. Here, VV 0)T represents the transpose of the gradient of the loss function with respect to the model parameters f, evaluated at validation sample 2.
C₂ -1= -1 VoL(z, ||2
G Σ -VL((2,0) (H, oo
sEDw
After computing the complexity and generalization scores, we define an overall unlearning score Up for each training sample æ. shown in Equation 3 This score guides the selection of sam- ples for the unlearning set through a threshold baased criterion. A lower U2 indicates that a sample contributes little to generalization while substantially increasing model complexity, making it suit- able candidate for unlearning. The parameter a mitigates Score Equivalence Bias: without it, the unlearning score may assign equal importance to samples with markedly different generalization and complexity scores, thereby reducing the effectiveness of the selection process, A more detailed sensitivity analysis is provided in Appendix FF
(G2) U,
CN
We then compute threshold TV, selecting the unlearning set those samples with unlearning scores below TV. To determine TU, we adopt Median Absolute Deviation (MAD) thresholding (Equation In this formulation, U3 denotes the median of all unlearning scores, and i controls the strictness of the threshold: higher values of l result in fewer samples being selected for unlearning. Unlike alternative approaches such as mean absolute deviation, MAD does not assume any specific underlying distribution and is inherently more robust to outliers. Following standard practice in robust thresholding, we set & = 2 in all experiments.
TV Us k median( U2l
34 D
4

---

Published as a conference paper at ICLR 2026
3.2 DOMAIN-SPECIHIC CHANNEL SELECTION
After selecting the unlearning set, it is essential to selectively remove its impact on the model. A naive approach would be to unlearn all knowledge associated with the unlearning set; however, this is suboptimal, as it would also erase domain-invariant features crucial for generalization. To address this, we propose targeted strategy that first identifies domain-specific channels, which are predominantly responsible for capturing domain-specific features. By focusing on these channels, our method selectively removes nomain-speciffi features associated with the unlearning set, while preserving domain invariinn features that benefit generalization.
Several studies have investigated the relationship between channel activation variability and domain- specific features (Xu et al.) 2022, Guo et al. 2023) Yu et al. (022)). However, prior methods typically measure channel-wise variability by computing the aggregated variance across activations pooled from all source domains, process referred to as aggregated variance. These approaches implicitly assume that domain-specifit channels exhibit high overall variability across the dataset, regardless of domain boundaries. This assumption conflates within-domal dispersion with between-domain dif- ferences and is particularly susceptible to biases under domain imbalance, where dominant domains disproportionately influence the aggregated statistics. As result, channels may be erroneously classified as domain-invariant or domain-specific based solely on the magnitude of within-domain variation in the dominant domains.
In contrast to prior approaches, we propose a more principled definition of domain--peciffictty that more accurately reflects the structure of domain shifts. Specifically, we define a channel domain- specific if its activation variance within domains differs substantially across source domains. To quantify this, we introduce a novel metric, Inter- Inter-Domain Variance (IDV), which measures the vari- ance of a channel's within-domain variances across all source domains. Compared to aggregated variance, IDV offers two advantages: (1) It is domain-aware treating each domain as an indepen- dent unit of analysis, and (2) It is domain-size agnostic, assigning equal importance to each domain regardless of its size. Formally, the IDV for a given channel e defined in Equation [ Here, N denotes the number of source domains, and vsa) represents the variance of channel within domain d. The term o(d,s) represents the activation of the i-th sample in channel e from domain d. Play is the mean activation of channel within domain and N& is the number of samples in domain d.
Na L
(c) = Variance (foEn) (-11 eis
(5)
No
By focusing on across -cross-domain variation rather than within-domain variation, IDV offers a more precise characterization of domain-specific channels, mitigating biases introduced by imbalanced domain distributions. To identify domain- channels, we adopt MAD thresholding, consis- tent with the strategy used for unlearning set selection. Channels with IDV values exceeding this threshold are selected as domain-specifie.
3.3 DOMAIN-SPECIRE GRADIENT ASCENT
After identifying the unlearning set D,, and the set of domain-specifer channels Cspe. IU selectively removes domain-specific features via -SmaiinSpecific Gradient Ascent, as defined in Equation 6 Specifically, 0r denotes the parameters associated with domain-specifie channel c, and Vo- [Ix represents the gradient of the loss with respect to 0 for training sample x.
6 V g- L(x, 0), I E Du: CE C
By selectively applying gradient ascent domain-specifi channels using data from the unlearning set, IU removes domain-specifie features while preserving domain-invarian features.
3.4 THFORETICAL ANALYSIS
This section presents a theoretical analysis demonstrating that our approach reduces the model's predictive reliance on domain domain-specific features while preserving the domain-invarian invariant features.
Assumption (Feature Decomposition). Following standard assumptions in domain generalization and domain adaptation, assume that the learned representation can be decomposed into domain-
5

---

Published as a conference paper at ICLR 2026
invariant and domain specifc components:
f(x) - [inv (x), Sapc(a)]
where Slnv (x) denotes
features and (apc(x) denotes domain-specific features.
Definition 1 (Parameter Decomposition). Let (Oinv: Ospe) be the corresponding parameters, where Oie only influences (1) and Oppc only influences (tpc(t).
Definition 2 (Predictive Dependence). Let Dinv(0) = [(y; (i))(x) denote the mutual information between the model oulput and domain--nvariint features. Similarly, let sppe := T(y: Jspe denote the mutual information between the model output and domain- specifie features.
Theorem (DSGA Selectively Reduces Dependence Domain-Specific Features). Let 0e be the model parameters at step t. Then, the update:
gt spe
Vep (iNEDDL(((, 0)]
with Bittt = fi yields:
Dapc (B(+1) < D, Din (B == D
(9)
In other words. DSGA reduces the model's reliance domain-specifer features while preserving its dependence on domain-inwe invariant features.
Proof. Let the gradient ascent update be applied to domain-specific parameters:
0++1
0
Veeee Ex [Lxx 0)]
(10)
Under the chain rule:
Bage Lxx,0 0)
V fope ((g(simm(()) (t)((
(11)
30spe
the update affects only the influence of fape on the model output.
After this update, define the perturbed prediction:
g-+1 g(ffnv (x), (zp 0gtt2))) }}
(12)
We now assess how this update affects the mutual information Dspe 1(y fxpo
By properties of mutual information:
(y: fx)o 11 (1) H H2 (x))
(13)
where (y) is the entropy of labels and is fixed. So, reducing mutual information is equivalent to increasing the conditional entropy Hy fxpo(z)), i.e., making the prediction less certain given Sape- The gradient ascent update increases the expected loss on D.. thereby decreasing the model's predictive confidence (i.e., increasing entropy) conditioned on fapc- Formally, this means:
H(y fape (1+)) >
H(y fape(z);
(14)
implying:
11y fape (x))
(15)
Since Pny is not updated, the functional form and behavior of fine (z) and its contribution to the prediction remain unchanged, so:
Dimo (0(+1) Dinv (i')(0')
(16)
Therefore, the DSGA update reduces Dape (0), i.e., the model's predictive dependence on domain- specific features, while preserving its predictive dependence domain-invariant features.
n

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 EXPERIMENT SETTING
We evaluate our proposed module on the DomainBed benchmark (Gulrajann & Lopez-Paz, 2021) using five widely adopted datasets: PACS, OfficeHome, VLCS, Terra Incognita, and NomainNet. Additionally, we assess generalization performance on Dig following the protocol of (Zhou et al. 2020), and on NICO++, following the setup in (Zhang et al. 2023). To demonstrate both the effectiveness and -agnosia nature of our unlearning module, we integrate it with a diverse set of state-of-ther -fee------h-rtttt domain generalization baselines spanning multiple methodological categories. Consistent with previous studies (Gulrajani& & Lopez-Paz, 2021 Zhou et al. (2022), we adopt a leave- one-domain-our evaluation protocol. Details the baseline methods, evaluation metrics, validation set selection, and implementation settings are provided in Appendix C
4.2 EXPERIMENTAL. RESULTS
Table T the accuracy of the most up-to-date baselines across all benchmarks. Detailed results for additional baselines are available in Appendix [ The best-performing results highlighted in bold. We evaluate two versions of our proposed module: the standard Identify and Unlearn (IU), denoted as the baseline with the subscript IU, and variant incorporating Exponential Moving Average (EMA) smoothing to the unlearning score, denoted as the baseline with the subscript IUE. A detailed analysis of the effect of EMA smoothing is provided in Section 4.4
We draw three key observations from Table | (1) IU enhances the generalization performance of all baselines, regardless of their underlying methodology, underscoring both its model-agnostic na- ture and the effectiveness of removing Nomain-ssecinf features through post-epoch unlearning. (2) Applying EMA to smooth the unlearning score yields further gains, highlighting the importance of stabilizing the unlearning process. (3) Even for strong baselines, such as UDIM and VL2V, whose performance has plateaued on standard benchmarks, IU delivers modest yet consistent improve- ments, further demonstrating its ability to enhance the generalizability of existing DG methods.
4.3 ABLATION STUDY
Table[[presentt the results of an ablation study evaluating the individual contributions of Unlearning Set Selection (USS) and Domain-Speciffe Channel Selection (DSCS). The second row (ERMuss) reports the performance when only USS is applied: gradient ascent is performed over the entire pa- rameter set using the unlearning set, without distinguishing between domain-specific and domain- invariant features. As shown, indiscriminately reversing gradients for the unlearning set degrades generalization performance, as it inadvertently removes domain-invariant knowledge. The third row (ERMpscs) reports the performance when only DSCS is applied: gradient ascent is applied to domain-specific channels, but using the entire training set. This setting also results in degraded per- formance, as globally unlearning domain-specific features leads excessive simplification, thereby diminishing the model's 's representational capacity and impairing generalization to unseen domains. In contrast, the fourth row (ERM)u) demonstrates that combining USS and DSCS yields the best per- formance. These two components are complementary: removing one while retaining the other leads to suboptimal outcomes and compromises model effectiveness. We further compare Inter-Domain Variance with Aggregated Variance and evaluate the proposed Domain-Specific Gradient Ascent against other unlearning strategies. Detailed results and analyses are provided in Appendix
4.4 FURTHER ANALYSIS
Change in Unlearning Score w/o EMA. Figure shows the evolution of unlearning scores over training epochs (I to 50) for six randomly selected training samples, comparing results with and without Exponential Moving Average (EMA) smoothing. As shown, the unlearning scores both settings eventually converge. This convergence reflects the stability of each sample's influence on the model and offers insights into its learnability and memorization behavior, As illustrated in Figure 3b EMA smoothing significantly reduces noise and yields smoother score trajectories. Notably, it also improves the separability between samples; for instance, Data 2 and Data begin to diverge earlier, offering clearer distinctions in their contributions to model behavior.
7

---

Published as a conference paper at ICLR 2026
Table Leave-one-domain-ouu results on benchmarks (with 95% confidence intervals). OH denotes OfficeHome, Terra denotes Terra Incognita, DN denotes DomainNet, and NICO denotes NICO++
PACS
OH
VLCS
Terra
Digits
NICO
ERM (1999)
83.0 + 4
68.2 1: 6
77.2 + 5
41.7 1 66
40.7 + 4
79,4 1
79.8 +
ERMIU
85.7 = .3
69.8 + 6
80.0 = 4
44.2
+
42.2 +
82.1 .
81.2
ERMRE
86.0
70.0 + 55
80.6 =
44.5 1
43.1 +
82.9 -
81.5
MMD 20188)
83.2
67.7 1..6
77.2 1
46,6 1 6
31.7
79.9 .4
80.2
MMD
84.6 4
69.8 + .5
80.1 = .4
48.4 + .4
34.2
81.2
8
MMDIUE
84.9
70.4 + .4
80.7 +
48.9 +
34.6 + .6
81.9 + .4
83.0 +
IRM (2019)
81.5
66.9
76.4 +
43.1 +
36.0 +
79.2 4
79.3
IRMIU
82.8
69.7 1.
77.9
45.4 1
38.1
80.3 .5
81.9
IRMIUE
83.2
70.3
78.3
46.0
38.8
80.7
82.3 1
DDAIG (220)
83.2 +
69.9 + .3
76.7 + .3
45.2 + .2
41.5
80.2 + 3
81.4 81.4
DDAIGIU
85.1
71.6
78.8 1
46.6 1
42.9 1
81.7 1
82.6 1
DDAIGILE
85.5 -
71.8
79.6 - .4
47.3 + .2
43.6
81.9
82.7 =
MixStyle (2021)
82.3
77.5 =
49.0 + 3
42.8
81.
8
MixStyleju
84.2 -2
72.0 22.022
79.2 -
50.2 + 44
43.9
83.0
83.5
MixStyleroe
85.1 1 .2
72,9 19912 122
79.4 1 .4
50.9 1 .,4
44.6 1
83.6 1 .2
84.1
DomainMix (2022)
82.2
69.8
76.1
48.1 +
+
80.0
82.7
DomainMixiu
84.0
71.2
77.6
50.4 +
43.2
81.4
8..0
Domain MiXICE
84.3
71.6 +
77.9
50.6 + .4
43.6
82.2
8
EFDMix (022)
84.6
71.2
78.3 + .3
49.9 1 3
44.2 1
82.1
82.6 82.6
EFDMiX
86.1
72
79.4 -
51.1
45.3
83.7 22
8
RFDMix
86.6
73.1
51.5 +
45.6
84.8 +3
RISE (2023)
86.3 = 4
71.1
80.6 = 33
34.4
45.4
51.6
8..
RISE
87.4 1: .3
72,4
82.1 +
35.7 .2
46.6
53.0
...
RISEIUE
88.3 = .2
73.0
82.5 - .3
36.1 + .2
46.8 +
53.5 =
85.3 15
SSPL 2024
84.0
71,2 .2
77.9
48.5 +
42.8
81.1
8.3
SSPL
85.6 22
72.7 1.22
80.3 1 3
50.9 1
44.2
83.3
84.0
SSPLiue
85.8 -2
73.4
81.3 = .3
51.2 + .3
45.1
83.6
84.8 - .2
LFME (2024)
85.2 +
78.9 + 33
48.1 1
41.8 41.8
81.9 2
82.8 82.8 1
LFME,u
87.3
71.4
80.2 -
50.2 +
42.7
83.3 3
84.2
LFMEQUE
87.9
71.9
81.1
50.9
.2
UDIM (0024)
84.7
68.9 +
77.5 + .4
47.3 + 6
40.5 +
80.8 + .3
81.9
UDIM
86.6 + .3
70.1 +
80.2 + .3
50.9 +
44.3 +
83.5 +
84.4 ..3
UDIMUE
87.3 .4
70,4 .3
80.6 - .4
51.2 .7
44.7
84.4 + .3
84.6 14.644
VL2V (024)
85.5
69.8 +
79.2 + 22
48.6 + 8
42.1
82.3
83.2
VL2V
87.0 4
71.6 .4
80.5 - 22
50.2 1.
44.3 +
84.4
84.5 1
VL2VIQE
87.9 72.3 .3
80.7 = 50.5
.77
44.8 -
84.7 = 22
85.3
Table 2: Ablation Study for Unlearning Set Selection and Domain-Specifer Channel Selection.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0 + .4
68.2 + 6
77.2 = =5
41.7 6
40.7 + 4.4
79.4
+ .3
79.8 +
ERMuss
78.9 + 5
64.3 = 4
72.6 = -66
37.6 = 6
37.4 6
74.5
- 2
76.3 + .3
ER MRMssss
76.7 + 6
62.5 + 5
70.4 -5
36.3 4
34.6
+ .5
72.9
+
74.6 + .5
ERMu
85.7 13
69.8 1 6
80.0 .4
44.2 =3
42.2 1
82.1
1.4
81.2 5
Distribution of Unlearning Score w/o EMA. As shown in Figure applying EMA smoothing (Figure 4b) results broader distribution of unlearning scores compared to the non-EMA setting (Figure 4a This increased spread improves the signal-to-noise ratio, enhancing the separation between samples with high and low unlearning influence. The resulting distribution more clearly highlights influential outliers, enabling more accurate and confident selection of the unlearning set.
Distribution IDV. Figure F illustrates that IDV provides a strong discriminative signal for iden- tifying domain- domain-specifee specific channels. The majority of channels exhibit low IDV values and are thus regarded as
while a small subset forms a long-tail distribution with substantially higher IDV values, corresponding domain-specific channels. This clear bimodal pattern, charac-
8

---

Published as a conference paper at ICLR 2026
2000
EMA Un Un-wing Scote
Undeaminng Score
-2000
-1000
Data 0
Data 0
-6000
Data
Data 3
Data 4
3000
Data 4
-4000
Data
-10000
30
20
30
40
12
20
30
40
50
Tpoch
Epoch
(a) Unlearning Score
(b) Unlearning Score with EMA
Figure 3: Comparison of unlearning scores with and without Exponential Moving Average (EMA).
600
600
Freueenc
500
300
300
200
200
100
150
100
50
0
50
100
150
2500
2000
1500
10:00
500
0
500
1000
1500
Unleamiing Score Distribution
Unlearning Score Distribution
(a) Unlearning Score
(b) Unlearning Score with EMA
Figure 4: Unlearning Score w/o Exponential Moving Average
terized by a dense concentration of low-variance channels and sparse tail of high-variance ones, demonstrates the effectiveness IDV in separating coomain-invarianl and domain domain-specific channels.
10
Number of Dsta Bolcw Threshol
1100
10%
2000
600
Intee-Dmmiin Variance (EXV)
Epochs
Figure 5: Distribution of IDV
Figure 6: Number of Data Unlearned
Number of Samples Selected During Training. Figure 6 shows the number of training samples selected for unlearning during training in the PACS dataset. Fixed thresholding strategies, such as selecting the bottom 10% of samples based unlearning scores (green dashed linc), are inflex- scores during training. contrast, dynamic
ible and fail to adapt to the evolving distribution thresholding methods like MAD naturally adapt to score fluctuations, particularly in the early, un- stable training phases. This adaptability improves both the reliability and precision of unlearning set selection, reducing the likelihood of over-selection under-selection.
Others. We further provide supporting evidence through T-SNE visualizations, model complexity analyses, and computational cost evaluations. Details are presented in Appendices Gad H

---

Published as a conference paper at ICLR 2026
5 CONCLUSIONS
This paper introduced Identify and Unlearn (IU), model-agnostic module designed to continually mitigate domain speciiic reliance in deep neural networks. IU leverages the unlearning score to iden- tify the unlearning set, the Inter-Domain Variance (IDV) metric to detect domain-specins channels, and a Domain- Speecifcc Gradient-Ascem. (DSGA) procedure selectively remove domain specific features while preserving domain-invariant ones. Extensive experiments demonstrate the effective- ness of adaptive, post-epoch unlearning as a principled complement to existing DG approaches. Looking forward, future work will explore integrating IU with large models, such large multi- modal models, to evaluate its scalability and transferability.
ACKNOWLEDGEMENTS
This research was partially supported by the New Zealand MBIE Strategic Science Investment Fund (SSIF) Data Science platform Hime-Evolving Data Science Artificial Intelligence for Advanced Open Environmental Science (UOWXI910), We also gratefully acknowledge the Centre for Ma- chine Learning for Social Good and the Advanced Machine Learning and Data Analytics Research (MARS) Lab at the University of Auckland.
REFERENCES
Sravanti Addepalli, Ashish Ramayee Asokan, Lakshay Sharma, and Venkatesh Babu. Leveraging vision-language models for improving domain generalization in image classification. In CVPR, pp. 23922-23932, 2024.
Naman Agarwal, Brian Bullins, and Elad Hazan. Second-order stochastic optimization for machine learning in linear time. JMLR, 18(116):1-40, 2017.
Kartik Ahuja, Fthan Caballero, Dinghuai Zhang. Chrr-hhitttppheeeee Gagnon-Audet, Yoshua Bengio, Ioannis Mitliagkas, and Irina Rish. Invariance principle meets information bottleneck for out-of- distribution generalization. NeurIPS, 34:3438-3450, 2021a.
Kartik Ahuja, Jun Wang, Amit Dhurandhar, Karthikeyan Shanmugam, and Kush R. Varshney. Em- pirical or invariant risk minimization? a sample complexity perspective. ICLR, 2021b. URL
https: fittps://cpenrevveee wew.:wwweeeeeee/ee
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Pas Paz. Invariant risk minimization. ArXiv: Axxiv:990..02893, 2019.
Mahsa Baktashmotlagh, Mchrtash T Harandi, Brian Lovell, and Mathicu Salzmann. Unsupervised domain adaptation by domain invariant projection. In ICCV, pp. 769-776, 2013.
Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. Metareg: Towards domain gener-
alization using meta-regularizztion. NeurIPS, 31, 2018
Hritam Basak and Zhaozheng Yin. Forget more to learn more: Domain-specific feature unlearning for semi-supervised and unsupervised domain adaptation. In ECCV, pp. 130-148. Springer, 2024.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. 456-473, 2018.
ECCV, pp-
Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. NeurIPS, 24, 2011.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In S&P, pp. 141-159. IEEE, 2021.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unleamiigg. In S&P, pp. 463-480. IEFE, 2015.
10

---

Published as a conference paper at ICLR 2026
Liang Chen, Yong Zhang, Yibing Song, Zhiqiang Shen, and Lingqiao Liu. LFME: A simple frame- work for learning from multiple experts in domain generalization. In NeurIPS, 2024. URL
Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan. Adaptive methods for real-world domain generalization. In CVPR, pp. 14340-14349 2021.
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salun: Em- powering machine unlearning via gradient-based weight saliency in both image classification and generation. ICLR, 2024. URLhttpss htttttps:///peereeeeee
net/ forum ?1 www.rrmmmmiiiiiooooioooonnnnn dm TagnmmTTh@NM TQGNM
Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In ICCV, pp. 1657-1664, 2013.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In ICLR, 2021.
Jintao Guo, Lei Qi, and Yinghuan Shi. Domaindrop: Suppressing domain-sensitive channels for domain generalization. In ICCV, pp. 1911-9124, 2023.
Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves crosi- generalization. In ECCV, pp. 124-140. Springer, 2020.
Zeyi Huang, Andy Zhou, Zijian Ling, Mu Cai, Haohan Wang, and Yong Jae Lee. A sentence speaks thousand images: Domain generalization through distilling clip with language guidance. In ICCV, pp. 11685-11695, 2023.
Seogkyu Jeon, Kibeom Hong. Pilhyeon Lee, Jewook Lee, and Hyeran Byun. Feature stylization and domain-aware contrastive learning for domain generalization. In ACM MM, pp. 22-31, 2021.
Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. NeurIPS, 115-5555884-555 2023.
Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, and Li Zhang. Style normalization and restitution for generalizable person re-identification. In CVPR, pp. 3143-3152, 2020.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In ICML, pp. 1895-1894. PMLR, 2017.
Da Li, Yongxin Yang. Yi-Zhe Song. and Timothy M Hospedales. Deeper, broader and artier domain generalization. In ICCV, pp. 5542-5550, 2017.
La Yongxin Yang. Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI, volume 32, 2018a.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In CVPR, pp. 5400-5409, 2018b.
Yuzhuo Li, Di Zhao, Tingrui Qiao, Yihao Wu, Bo Pang, and Yun Sing Koh. Metawild: A multimodal dataset for animal re-identification with environmental metadata. In ACMMM, pp. 13009-13015, 2025.
Xiaofeng Liu, Xiongchang Liu, Bo Hu, Wenxuan Ji, Fangxu Xing, Jun Lu, Jane You, C-C Jay Kuo, Georges El Fakhri, and Jonghye Woo. Subtype-aware unsupervised domain adaptation for medical diagnosis. In AAAI, volume 35, pp. 2189-2197, 2021.
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In ICML, pp. 97-105. PMLR, 2015,
Yadan Luo, Zijian Wang, Zi Huang, and Mahsa Baktashmotlagh. Progressive graph learning for open-set domain adaptation. ICML, pp. 6468-6478. PMLR, 2020.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In ICML, pp. 10-18. PMLR, 2013.

---

Published as a conference paper at ICLR 2026
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring general- ization deep learning. NeurIPS, 30, 2017.
Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. TKDE, pp. 1345-1359, 2009.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In ICCV. pp. 1406-1415, 2019.
Vihari Piratla, Praneeth Netrapallii, and Sunita Sarawagi. Efficient domain generalization via common-specific low-rank decomposition. In ICML, pp. 7728-7738, PMLR, 2020.
Tingrui Qiao, Di Zhao, Caroline Walker, Chris Cunningham, and Yun Sing Koh. Zero-shot domain generalisation via prompt-driven feature refinement. WACV, pp. 6184-6193, 2026.
Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to forget: Algorithms for machine unlearning. Neur/PS, 34: 1805--800880 2021.
Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri. Preethi Jyothi, and Sunita Sarawagi. Generalizing across domains via gross-ggradiet training. ICLR, 2018.
Seungjae Shin, HeeSun Bae, Byeonghu Na, Yoon-Yeong Kim, and Il chul Moon. Unknown do- main inconsistency minimization for domain generalization. In ICLR, 2024. URL https:
Zhishu Sun, Zhifeng Shen, Luojun Lin. Yuanlong Yu, Zhifeng Yang, Shicai Yang, and Weijie Chen. Dynamic domain generalization. I/CAI, 2022.
Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Under- standing factors influencing machine unlearning. In EuroS&P, pp. 303-319, IEEE, 2022.
Vladimir N Vapnik. An overview of statistical learning theory. TNN, 10(5):988 1999.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In CVPR, pp. 5018-5027, 2017.
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip Yu. Generalizing to unseen domains: survey on domain generalization. TKDE, 2022.
Zhenyi Wang, Yan Li, Li Shen, and Heng Huang. A unified and general framework for continual learning. ICLR, 2024. URLhttpp:://peeeeviie www.:/.eee.oormmoeoomm dot.
Kun Wu, Jie Shen, Yue Ning, Ting Wang, and Wendy Hui Wang. Certified edge unlearning for graph neural networks. In SIGKDD, pp. 2606-2617, 2023.
Yihao Wu, Di Zhao, Yuzhuo Li, Matthew Alajas, Alistair S Glen, Jingfeng Zhang, Gillian Dobbie, Daniel Wilson, and Yun Sing Koh. Overcoming fine-grained visual challenges in animal re- identification via semantic feature alignment. In WACV, pp. 371-381. 2026.
Qi Xu, Liang Yao. Zhengkai Jiang, Guannan Jiang, Wenqing Chu, Wenhui Han, Wei Zhang, Chengjie Wang, and Ying Tai. Dirl: Domain-invariant representation learning for generalizable semantic segmentation. In AAAI, volume 36, pp. 2884-2892 2022.
Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In ICCV, pp. 1426-14286099 1436,143,.001 2019.
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. NeurlPS, 37:105425- 105475, 2024.
Xi Yu, Shinjae Yoo, and Yuewei Lin. Clipceil: Domain generalization through clip via channel refinement and image-text alignment. NewrIPS, 37:4267-4294, 2024.
Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni- Sungiovammi-Vnncetelll Kurt Keutzer, and Bo- qing Gong. Domain randomization and pyramid consistency: Simulation-to generalization without accessing target domain data. In ICCV, pp. 2100-2110, 2019.
12

---

Published as a conference paper at ICLR 2026
Xingxuan Zhang. Yue He, Renzhe Xu, Han Yu, Zheyan Shen, and Peng Cui. Nico+:: Towards better benchmarking for domain generalization. In CVPR, pp. 10036-16047,2 2023.
Yabin Zhang, Minghan Li, Ruihuang Li, Kui Jia, and Lei Zhang. Exact feature distribution matching for arbitrary style transfer and domain generalization. In CVPR, pp. 8035-8045, 2022.
Zheyuan Zhang, Bin Wang, Debesh Jha, Ugur Demir, and Ulas Bagci. Domain generalization with correlated style uncertainty. In WACV, pp. 2000-2009, 2024.
Di Zhao, Yun Sing Koh, Gillian Dobbie, Hongsheng Hu, and Philippe Fournier- Viger. Symmetric self-paced learning for domain generalization. In AAAI, volume 38, pp. 16961-16969, 2024.
Di Zhao, Jingfeng Zhang, Hongsheng Hu, Philippe Fournier- Fournier-Viger, Gillian Dobbie, and Yun Sing Koh. Balancing invariant and specific knowledge for domain generalization with online knowl- edge distillation. ICAII, pp. 2440-22420002 2400,2448.440 2025.
Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain generaliza- tion via entropy regularization. NeurlPS, 33:1606-161077 2020.
Kaiyang Zhou, Yongxin Yang. Timothy Hospedales, and Tao Xiang. Deep domain-adversaria im- age generation for domain generalisation. In AAAI, pp. 13025-13032, 2020.
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. ICLR, 2021.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. TPAMI, 2022.
APPENDICES
A ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. The study does not involve human subjects, per- sonally identifiable information, or sensitive data, and all datasets used are publicly available under appropriate licenses. No private or proprietary information was collected, and experiments pose risks of harm to individuals, groups, or the environment. The methods and findings are intended solely for advancing scientific understanding of domain generalization and not designed for harmful applications. We have taken care to ensure fairness and reproducibility by documenting our approach and following established research integrity standards.
B PRELIMINARY
Notation. Let X' denote an input feature space, with dimension d, and target class label space. A domain, D. is composed of data sampled from joint distribution PP(X, Y) on X' x V. where PX,, Y), E X C Rd, y E y C RR and the number of data in the domain. Here, X and Y denote the corresponding random variables (Wang et al. 2022; Zhou et al. 2022).
D (I,; y/)
Domain Generalization. For the task of domain generalization, the input N source domains (training set), S = {Di j 1,... .N,, where D = {(x H denotes the j-th domain and nj denotes the number of examples in the j-th domain. The joint distributions between each pair of domains are different: PXX, Y))) * PPX,, ()()), & k. The goal of domain generalization is to learn a robust and generalizable predictive function f X > y from the N source domains to unseen target domain T, where T cannot be accessed
achieve a minimum prediction error during training and P(X, (T) P(X,Y) for j E (1,...,NN)
Machine Unlearning. Consider a subset of training samples designated for removal, termed the unlearning set D.- A machine unlearning procedure (AA()), D. D.)) is formally defined mapping from a trained model A(D), the original training dataset D. and the unlcarning set D,,. to an updated model w., Here, A(-) represents arbitrary machine learning algorithm.
13

---

Published as a conference paper at ICLR 2026
C REPRODUCIBILITY STATEMENT
Datasets. We evaluate our module on the Domain benchmark (Gulrajani & Lapez-Paz
2021
using five widely adopted datasets: PACS (Li et al. (017). OfficeHome (Venkateswara al.
2017
VLCS (Fang et al. 2013). Terra Incognita (Beery et al. 2018), and DomainNet (Peng et al. In addition, we assess generalization performance on Digits-DG, following the protocol of (Zhou et al. 2020). and on NICO++ following the setup in (Zhang et al. 2023).
2019)
Baselines. We evaluate the effectiveness and model-agnostic nature of unlearning module by integrating it with a diverse set of state-of-the- domain generalization methods across multiple categories. Specifically, we consider CrossGrad (Shankar et al., (2018). MLDG (Li al.) 201 8a), MMD aLi ( IRM (Arjovsky et al. 2019), DDAIG (Zhou et al. 2020), RSC (Huang et al.,
2020 MixStyle (Zhou al. 2021). DomainMix (Sun al.] (022). EFDMix (Zhang et al. 2022). RISE Huang et al. 2003), DomainDrop (Guo et 2023), SSPL (Zhao et al. (024). LFME (Chen ct al. 2024), UDIM (Shin et al. 2024), and VL2V (Addepalli et al. 2024). In addition, we in- clude a standard Empirical Risk Minimization (ERM) baseline (Vapnik 1999), which trains on the combined data from all source domains without applying any domain generalization techniques.
Evaluation Metries. Following prior works (Gulrajani & Lopez- Paz 2021 Zhou et al. (022), adopt a leave-one-oomain-oo evaluation protocol. In each run, one domain is held out as the target domain for testing, while the remaining domains serve source domains for training. We report the top-1 1 classification accuracy (%) averaged over ten runs, along with the corresponding 95% confidence intervals.
Implementation Details. All input images across benchmarks are resized to 224 pixels. In our experiments, aligned with standard practices in the domain generalization literature, the validation set is constructed by uniformly sampling from all source domains, ensuring it is representative of the training distribution. Since we do not assume access any target domain data, the validation set does not introduce target-domain bias. For non-CLIP-base methods, we adopt pretrained ResNet-50 as the backbone. For CLIP-based methods, both ResNet-50 and ViT-B/32 are evaluated, with ViT-B/32 results reported in the supplementary material. All implementations are developed using the PyTorch library. Models are trained using Stochastic Gradient Descent (SGD) with a momentum of 0.9 and a weight decay of x 10-1 Training is conducted for 50 epochs with a batch size of 64 and an initial learning rate of 0.001, scheduled via the CosincAnnealingL.. scheduler. All experiments
on NVIDIA Tesla A100 GPUs (80GB) with CUDA version 12.1. The balancing parameter a is set to 1.1.
D ALGORITHM
E ADDITIONAL ABLATION STUDY
Comparing IDV with Aggregated Variance. Table presents evaluation results comparing Nomain--Secifii Gradient Ascent (DSGA) under different channel selection strategies. Here, AV refers to Aggregated Variance, and IDV refers to vations: (1) Even when domain-specific channels are identified using Aggregated Variance, IU still yields substantial performance improvements over the original baseline, demonstrating the effec- tiveness of the proposed unlearning strategy. (2) When Inter-Domain Variance is used for channel selection, performance gains further enhanced, highlighting the advantage of IDV over Aggre- gated Variance in accurately identifying domain-specific channels.
Variance. We draw two key obser-
Table 3: Comparing IDV with Aggregated Variance.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0 = .4
68.2 6
77.2 + 5
41.7 + 6
40.7
+ 4
79.4
+ 79.8 + .3
ERMAy
84.3 = .4
69.1
-
78.5 = 5
43.2 + 6
42.0
+ 6
81.2
3
80.8 = 3
ERMjpv
85.7 .3
69.8
80.0 + .4
44.2
42.2 +
82.1
81.2 .5
Comparing DSGA with Other Unlearning Methods. We do not compare Domain Somain-Specific Gra- dient Ascent (DSGA) against existing unlearning methods, as, to the best of our knowledge,
14

---

Published as a conference paper at ICLR 2026
Algorithm Identify and Unlearn (IU)
1: Input: D: training set: Dvat: validation set; C: set of channels; A(-): arbitrary DG method; k: MAD
threshold multiplier; a: parameter for unlearning score.
2: Output: fa=: unlearned model with reduced domain-specific features.
3: for each epoch do fo = A(D)
Train model with algorithm A on training set D.
do each training sample
6:
C2 = H VoL(x, 0) 12
P Compute complexity score.
7:
G. = E.p,, VoL(2, 8) Vo
Compute generalization score. Compute unlearning score.
8:
U-- [C,)
9:
end for
10:
Tu = U3 k mUU - U2|. 25 E Dimi)) > Compute threshold тu for selecting unlearning set.
11:
D. = {œ E D Up < TuJ
> Select unlearning set D...
12:
for each channel c do
(DV(c) Variance (fo d=1) 160
1 NN (E(a,) mis)
Compute
No
Variance for channel c. end for
15:
median([IV.. IDVal.e EC) domain-specific channels.
To IDV;
Compute threshold r. for selecting
16:
Coge = IDV(c) Te}
Select domain-specific channels Cype-
17:
for each x D., do
18:
for carcacheoo c E Capc do
19:
8 8t + Vo L(x,0)
Perform Domain-Specific Gradient Ascent.
20:
end for end for
21:
22:
end for
23:
Return fo-
learning strategy has been specifically designed for domain generalization. Unlearning techniques developed for generative models are not applicable this context due to fundamental differences in model architectures and objectives. Similarly, model-agnostic unlearning approaches such as standard Gradient Ascent (Thudi et al. (2022) are primarily designed for privacy preservation rather than performance improvement. These methods often degrade model performance, and their upper bound is typically equivalent to a retrained model. In contrast, our approach is explicitly designed to enhance model generalization by selectively unlearning features that impair performance on unseen domains. Finally, unlearning strategies tailored for domain adaptation are also inapplicable here, as they require access to the target domain, which is unavailable the domain generalization setting.
Removing the unlearning set from the training data and retraining the model. At specific training epochs, we remove the unlearning set entirely from the training data and continue training the model on the remaining samples. Table 4 presents the results for this experiment, where the unlearning set removed at epochs 5, 10, 15, and 20.
Table Removing the unlearning set from the training data and retraining the model.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0
68.2 66
77.2
41.7
40.7
79.4 1 .3
79.8
Epoch
80.1 +5
66.0 .6
74.1
39.1 =5
38.6 + 3
76.4 1.4+1 3
77.1 + 2
Epoch 10
81.1 + -4
66.8 =
75.4
+ .4
40.2 .4
39.5 = 4
77.8 - .4
78.3 4:
Epoch 15
81.6 - 4
67.1 -5
76.1
- 4
40.8 4
39.8 = 4
78.3
-
78.5 +
Epoch 20
82.0 1: .4
67.4 1
76.5
1
41.1 =55
40.0 1.
78.7
1
79.0 4:
As shown, this strategy consistently underperforms relative to the original ERM baseline. This is expected: completely removing the unlearning set is analogous to indiscriminately applying gradient ascent on all its associated features, both nomain-invarian and domain Nomain-specifie. This reduces the model's 's ability to learn useful information from these samples and decreases the overall training data size, ultimately leading degraded performance.
Moreover, we observe that the earlier the unlearning set is removed, the more significant the perfor- mance drop. This trend is consistent with our analysis in Figure which shows the unlearning set is
15

---

Published as a conference paper at ICLR 2026
Officettome
70.0
Accuracy
68.0
78
Terra
DomainNett
Digils
00.5
42.0
Accuracy
43.5
43.0
41.5
0.8
1.3
0.9
1.0
1.1
1.2
1.2
1.3
for Unlearning Score
a for Unlearning Score
for Unlearning Score
Figure 7: Sensitivity Analysis of a for Unlearning Score.
PACS
OfficeHame
VLCS
86
70.0
80
Accurscy
85
79
68.0
68.5
1.0
2.0
2.5
3.0
3.0
2.0
2.5
3.0
1.0
2.0 Digits
2.5
3.0
Terra
DomainNet
42.5
44
82
42.0
Accurcyy
81
41.5
80
41.0
2.5
for Set Selection
k for Set Selection
k: for Set Selection
Figure Sensitivity Analysis of for Unlearning Set Selection.
larger in the early stages of training. Thus, early removal eliminates more training data, amplifying the negative impact on generalization.
F SENSITIVITY ANALYSIS
Figures 7 present the sensitivity analysis of in the unlearning score; MAD thresholding for unlearning set selection; and l: in MAD thresholding for domain-specific channel selection. As shown in Figure except for extreme values of a, IU consistently outperforms baselines, indicating insensitivity to Figures 8 9 show that performance gains remain substantial even with relatively small or large values of k, demonstrating that IU similarly insensitive to the choice in MAD thresholding. These results highlight the effectiveness of the proposed unlearning score and Meer-Domain Variance in reliably identifying both the unlearning set and domain -spaiii channels. Detailed sensitivity results provided in Tables 5
16

---

Published as a conference paper at ICLR 2026
Officettome
VLCS
70.0
69.5
09.0
78
68.5
Terra
DomainNet
Digits
44.5
42.0
Accuracy
43.0
41.5
42.5
41.0
40.5
1.0
1.5
2.0
2,5
3.0
1.0
1.5
2.0
2.5
3.0
1.0
1.5
2.0
2.5
3.0
k for Channel Selection
for Channel Selection
k for Channel Selection
Figure 9: Sensitivity Analysis of k for Channel Selection.
Table 5: Sensitivity Analysis of for Unlearning Score.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0 4
68.2 + 6
77.2 + .5
41.7 + 6
40.7
.4
79.4 + 3
79.8 +
0.8
83.6 3
68.3 + 22
77.5 + .2
42.2 + 2
41.2 + 2
79.8 + 3
79.9 .3
0.9
84.0
68.6
78.3 +..3:1
42.9 1
41.6 22
79.9 1
80.1 .2
= 1.0
85.3 22
69.2
79.7 +
43.5
41.7 $1.7.3.
81.7
80.6 .2
a 1.1
85.7 1
698
80.0 +
44.2
42.2 12.2-.3
82.1
81.2
+
68.7
78.7
43.9
41.6
80.7
80.2
1.3
84.1
68.4
77.8
43.4
40.8
80.0
80.0
G FURTHER ANALYSIS
T-SNE Visualization. Figure 10 shows the T-SNE visualizations of embeddings produced by ERM and DomainDrop, with and without the IU module. For ERM (Figures I0a and [Ob]. incorporat- ing IU yields noticcably more separable embedding structure, suggesting that removing domain- specific features promotes learning domain-invariant representations and enhances generalization. For DomainDrop (Figures [0c and TOd), which already produces more separable embeddings than ERM, IU further improves cluster separation, demonstrating that IU complements existing DG meth- ods rather than merely compensating for weak baselines.
(Cimnce saraltee
Canoos
Biraffe
Canowa
-Camoon
Piratte
elephant
clephant
elephant
clephant
quitan t
guitaa
Lenitar
dog
dog
forse
hoSse
horse
horses
person
house
person
houge
eprroo
house
person
(a) ERM
(b) ERM
(c) DomainDrop
(d) Domain
Figure 10: T-SNE Visualization for ERM and DomainDrop w/o IU
Regularization versus Unlearning. We further investigate whether our domain speeiff channel selection method can serve a form of regularization during training. In this regularization setting, we not perform unlearning set identification or apply Domain-Specins Gradient Ascent. Instead, after identifying domain-specific channels, apply regularization to features from those channels
17

---

Published as a conference paper at ICLR 2026
Table 6: Sensitivity Analysis of & for Unlearning Set Selection.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0
77.2
41.7
40.7
79.8
h 1.0
83.4 +
69.1 .3
77.3 +
42.3 +..3..1
41.0
80.6 + .1
80.4 +
k 1.5
85.3
69.7 -
79.2 13
4..4
42.1 1
81.7 +
80.6 1:
A 2.0
85.7 =
69.8
80.0 =2
4
42.2
+3
82.1 -
81.2 -
2.5
84,6 1
69.2 -
79.1 +2
4.2
41,6 4:
81.4
+
80.9 +
è 3.0
83.3 =
68.9
-
77.6
41.9
41.1 1.2
79.5
80.3 -
Table 7: Sensitivity Analysis of k for Domain-Specific Channel Selection.
PACS
VLCS
Terra
DN
Digits
NICO
ERM
83.0 + .
68.2 + 66
77.2 + 55
41.7 + 66
40.7 + .4
79.4 + .3
79.8 + .3
k 1.0
85,4 + .2
69.3 1 .3
79.2 + .2
43.3 1
41.7 1: .22
80.5 + .2
80.5 1: .3
k ==155 1.5
85.5 = -2
69.4 - .3
79.7 -
43.6 .2
41.7 +
81.3 - 22
80.8 + .3
k 2.0
85.7 = .2
69.8 +
80.0 +
44.2 +
42.2
82.1 + +22
81.2 + .3
2.5
84,9 =
69.6
79.5
43.4
41.2
81.0
80.0 + .3
3.0
83.2 =
68.9
77.9 -
42.1 1.1:2
40.7
80.3 +
79.9 -
throughout training. As shown in Table 8 this regularization strategy also leads to performance improvements by suppressing the learning of domain-specifie features. However, remains less effective than unlearning strategy. We attribute this performance gap to the difference in gran- ularity: Unlearning leverages information across training epochs by identifying harmful samples and selectively applying gradient ascent between epochs, enabling more global perspective. In contrast, regularization operates within mini-batches, providing only a local view, which can lead suboptimal performance. This experiment highlights both the conceptual and practical differ- ences between unlearning and regularization. Importantly, the two strategies are orthogonal and be complementary. Our IU module is compatible with existing domain generalization methods, including those that employ regularization based techniques.
Table 8: Evaluating Domain-Speciin Channel Selection in the Regularization Setting.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0 + .4
68.2 +
77.2 + 5
41.7 +
40.7 + .4
79.4 + 3
79.8 + 3
ER ERMRes
83.7 + 4
69.3 + .4
78.8 + 5
42.1 +
40.9 + .4
80.2 + 3
80.5 +
ERMu
85.7 ..3
69.8 1 .6
80.0 1 ..4
44.2 1.3
42.2 1 82.1 1 .4
81.2 1
Model Complexity Change. Figure 11 illustrates the reduction in model complexity (measured on ResNet-50) across training epochs for all benchmarks. The solid lines represent the unlearned model complexity, while the shaded regions indicate the magnitude of complexity reduction intro- duced by IU. We draw three key observations: (1) IU consistently reduces model complexity across all benchmarks, highlighting its effectiveness in simplifying the model by removing domain-specific knowledge. (2) The extent of complexity reduction is benchmark-dependdnt. For example, bench- marks such as VLCS and PACS exhibit relatively modest reductions, while larger gains are observed for Terra and NICO++, where the original complexity decreases more substantially over training. (3) The degree of complexity reduction correlates with the size of the benchmark. Larger datasets tend result in a greater number of training samples selected for unlearning, thereby amplifying the impact of IU on model complexity.
H COMPUTATION COST ANALYSIS
Naive influence function estimation through direct inverse Hessian computation is prohibitively ex- pensive for large-scale models. Our method circumvents this limitation by employing Hessian- Vector Products (HVP) combined with the LiSSA algorithm to efficiently approximate inverse- Hessian vector products. Below, we provide both theoretical and empirical analyses demonstrating the scalability of our approach when leveraging LiSSA for efficient approximation.
18

---

Published as a conference paper at ICLR 2026
1000
800
Model Complexity
600
PACS
400
OfficeHome
VLCS
Teraa
200
Nemain
Digits
NICO+
10
20
30
40
50
Epochs
Figure 11: Model Complexity Change across Benchmarks.
Theoretical Discussion for Scalability. We compare three strategies (Table 9 for computing the in- verse Hessian matrix: Direct Hessian Inversion, Hessian- Vector Product + Conjugate Gradient, and Hessian-VVe Hessian-Vector Vector Product LiSSA, critical operation for influence function estimation. Each method differs in its computational efficiency, memory usage, and theoretical properties, particularly the context of high-ddmensiooal deep learning models.
Table Comparing Three Strategies for Computing the Inverse Hessian Matrix.
Method
Time
Memory
Explicit Hessian
Mini-batch
Direct Inversion
Old))
O
Yes
No
HVP + CG
OT d) (T iterations)
d
No (HIVP only)
No
HVP LiSSA
OTT d) (T recursion depth)
No (HIVP only)
Yes
Direct Hessian Inversion. Direct Hessian Inversion requires computing and storing the full Hessian matrix H E Rddd followed by matrix-vector multiplication. Time complexity: O(d²) for matrix inversion. Memory complexity: Odd)). required to store the full Hessian. Theoretical properties: Yields exact results when the full Hessian available; however, infeasible for large-scale models due cubic time and quadratic space complexity. Limitation: Not applicable in deep networks where d can exceed millions.
Dossian-Vector Product + Conjugate Gradient. The inverse Hessian-vector product is reformu- lated as solving the linear system Hx = v, which is solved iteratively using Conjugate Gradient (CG). This avoids explicit Hessian storage and uses Hessian-vector products (HVPss, efficiently computed via reverse-mode automatic differentiation. Time complexity: O(T d), where T is the number of CG iterations. Memory complexity:
Odd). storing only vectors and intermediate results.
Theoretical properties: Converges in at most dl iterations, faster convergence when H is symmetric positive definite and well-conditioned.
Hessian- Tessian-Vector Product + LiSSA. We leverage LiSSA (Agarwal et al.) 2017) in our implementa- tion, which provides stochastic, iterative approximation to - via a truncated Neumann series expansion. Unlike CG, LiSSA allows the of stochastic Hessian estimates computed mini- batches, improving scalability. Time complexity: O(T . d), where T is the number of recursion steps. Memory complexity: O(d), storing only vectors per iteration. Theoretical properties: Under strong convexity and smoothness assumptions, LiSSA exhibits geometric convergence. Con- vergence rate and stability improve with damping (i.e., computing (H -T)- Advantage:
19

---

Published as a conference paper at ICLR 2026
Highly scalable and amenable to mini-batch computation; preferred in harge-scale online settings where CG is unstable or too costly.
Empirical Discussion for Scalability, Table 10 reports the per-epoch runtime of ERM and the three components of IU: domain-specific channel selection (Selchamee), unlearning-set selection (Setunleam) ) and Domain-Specifer Gradient Ascent (DSGA). All experiments hardware described in the Appendix, with ResNet-18 as the backbone, and runtime measured in minutes. The cost of DSGA is marked as "_" because it simply adds the corresponding gradients back, and its overhead is negligible compared to the other steps.
the settings and
Including ERM in the table clarifies the relative overhead of IU. As shown, Setchannel adds only small cost over ERM, confirming that channel selection is lightweight. In contrast, the dominant cost arises from Setynleam which performs influence-function function-based scoring. The overhead scales with dataset size, reaching 354 minutes per epoch on the largest benchmark, DomainNet (= 580k: images). While non-trivial, this cost reflects the intrinsic difficulty of influence estimation at large scale and remains feasible given the substantial performance gains IU delivers.
Importantly, this overhead does not need to oceur every training epoch. As demonstrated in our accuracy-cost analysis (Table invoking IU every E [2,3, 5) epochs amortizes the running time by a factor of e while retaining most of the accuracy improvement. For example, at e = 2, the effective cost is reduced by 50%, yct performance drops by less than 0.5% on average. Thus, IU can be operated in significantly more lightweight configurations without sacrificing its effectiveness.
Finally, although all measurements were conducted on single A100 GPU, modern accelerators offer substantial additional speedups. Upgrading to an H200 reduces the Setynicum cost on Domain- Net from 354 minutes to approximately 106 minutes per epoch. With data-parallel execution 8xH200 GPUs, the cost can be further reduced to roughly 16 minutes per epoch, making IU increas- ingly practical with contemporary hardware.
Table 10: Running Time for IU (min)
PACS
OH
VLCCS
Terra
DN
Digits
NICO
ERM
0.3 + 0.1
0.5 0.1
0.2 0.1
0.9 0.2
11.7 + 0.6
0.9 + 0.1
5.3 0.3
Setchannel
0.4 0.1
0.6 0.1
0.2 0.1
1.1 0.2
29.5 + 0.6
0.8 0.1
6.1
Selusieum
39.6
45.4 3.3
9.8 1.3
131.4 6.8
354.5 + 9.4
75.0 + 6.1
184.6 9.9
DSGA
I ACCURACY-COST TRADE-OFF ANALYSIS
To evaluate the computational efficiency of IU, we vary the frequency of invoking the module by performing unlearning every e epochs (e E f1, 3, 5)). Table II the results across seven benchmarks. We observe clear and smooth trade-off between accuracy and cost. Executing IU every epoch (e 1) yields the highest overall performance, but reducing the frequency leads to only gradual degradation. Notably, e = retains more than 95% of the performance gain over ERM while reducing IU-related overhead by approximately 50%, making it a strong operating point in practice. Even with very sparse updates (e 2 4), IU continues to outperform ERM on all bench- marks, demonstrating that the module does not rely on per-epoch execution remain effective.
These results suggest that (i) the influence estimates and domain-speciff channel scores evolve smoothly over training, rumning IU intermittently is sufficient, and (ii) practitioners may select an appropriate update frequency depending on their computational budget. Overall, the findings confirm that IU provides consistent robustness improvements under wide range of operating costs, enabling flexible deployment without sacrificing its core benefits.
J LONG-TERM STABILITY OF ITERATIVE INFLUENCE- -AAEED UNLEARNING
A natural question is whether repeatedly applying cefuennee-asee unlearning after each epoch could lead to the accumulation of approximation error and potentially cause the model to become unsta-
20

---

Published as a conference paper at ICLR 2026
Table 11: Accuracy-Cost Trade-Off Analysis.
PACS
OH
VLCS
Terra
DN
Digits
NICO
ERM
83.0
68.2 =
77.2
41.7 -
40.7
79.4 + 3
79.8
85.7
69.8 + .6
80.0
44.2 + 3
42.2
82.1 + .4
81.2
84.9 .4
69.3 =
79.1 =
43.5
41.8 .6
81.5 1
80.9
84.3 5
69.1 =
78.6 -6
43.0
41.6 1
81.0
80.7 .5
83.8 .4
-.
78.1 =
42.6
80.4
80.6 + 66
5
83.4 = 5
68.4
77.9
66
42.1 1.5
41.0
80.0 1.5
80.1 = .5
ble. Although IU relics on a first-order influence approximation, several design choices ensure that approximation error remains controlled throughout training.
Localized interventions. Comain--Specife Gradient Ascent (DSGA) applies gradient reversal only to small subset of channels, those identified domain-specific, and only using samples in the unlearning set. This produces highly localized updates, preventing large parameter drifts that could amplify approximation inaccuracies. Unlike full-parameter unlearning, IU's channel-level selectiv- ity inherently constrains error propagation.
Robust re-identiffcatiin each epoch. Rather than repeatedly acting on fixed set of samples or channels (which could amplify early noise), IU re-identifies both the unlearning set and domain- specific channels every invocation using robust MAD-based thresholds. This adaptivity prevents unstable samples or transient fluctuations from being reinforced across epochs, effectively damping potential accumulation of approximation error.
Temporal smoothing via EMA. We incorporate an exponential moving average (EMA) over un- learning scores to suppress short-term noise. EMA produces smoother trajectories for sample-level statistics, reducing sensitivity to momentary fluctuations and ensuring that IU reacts only to persis- tent, high-confidence domain-specific signals. This further mitigates the risk instability caused by noisy influence estimates.
Collectively, these theoretical control mechanisms demonstrate that remains stable under iterative execution. While based on first-order approximations, IU's localized updates, robust adaptivity, and temporal smoothing prevent error accumulation and ensure reliable behavior throughout training.
K ADDITIONAL EXPERIMENTS RESULTS
Table 12: Leave-one-domain-out results on benchmarks (with 95% confidence intervals). OH denotes OfficeHome, Terra denotes Terra Incognita, DN denotes DomainNet, and NICO denotes NICO+
PACS
OH
VLCS
Terra
DN
Digits
NICO
CrossGrad 2018
81.7 1:
1 3 71.7
76.1
44.7
38.5
79.5
80.6
CrossGradau
84.2 + 2
79,0
45,9 +3
40.7
81.1
82.3
CrossGradjue
84.8 + .2
72.1 + .4
79.7 +
46.2 +
41.3 =111
81.6 +
82.9 3
MI.G (2018a)
82.8 + 3
68.6 + 4
77.2 + 44
46.2 46.2
41.0 +
79.7 +
79.7
MLDGGU
84.3 1 .4
70.8 1
80.1 1 .4
48.0
42,3
81.4
81.1 -
MLDGGUE
84.9 -
71.2 +
80.7 - .4
48.4
43.0 .3
81.7 +
81.6 .1
RSC (2020)
82.7 5
68.4 + 6
77.5 + .4
40.6
39.0 + .4
79.9 + .4
82.1 4
RSC
83,9 1 .7
70.5 + 66
80.4 1 .
41.0 - .3
81.3
83.7 = .
RSC
84.2 + .4
71.0 + .6
81.0 +
42.8 + .6
41.4 = .2
82.0 92.0 .6
83,9 .2
DomainDrop (2023)
+
71.4
4
4..
81.6
82.6
DomainDropii Dropiu
90.5 + 3
72.3 +
80.5 +
49.3
45.0
82.4
84.1
DomainDropur
90.9 1
72.9 1
80.9 1
49.9
45.5
83.1 1
84.4 .
21

---

Published as a conference paper at ICLR 2026
Number of Itunge
3000
PACS
OfficeHomee
VLCS
1000
Number of Imougs
1908
1900
200
Terra Incognita
NIC0+
DomsinNa
Figure 12: Dataset Distribution for PACS, OfficeHome, VLCS, Terra, NICO++, and NetainNet.
L ROBUSTNESS OF IDV UNDER DOMAIN IMBALANCE
All DG benchmarks used this paper exhibit substantial domain-size imbalance (Figure 12). PACS, VLCCS, Terralncognita, NICO++ OfficeHome, and especially DomainNet show sample-size differ- ences ranging from 2x to more than 4x across domains. This setting directly tests whether IDV is sensitive such imbalance. Methodologically. IDV is designed to be robust because it first com- putes within-domain variances and then measures the variance across domains, giving each domain equal weight regardless of its sample size. This avoids the dominant-domain bias inherent in Aggre- gated Variance (AV), where large domains can overshadow smaller ones. Empirically, IDV exhibits stable, long-tailed distribution across all six benchmarks and consistently identifies a small subset of high-IDV channels, despite large domain-size discrepancies. Combined with the strong perfor- mance gains observed across benchmarks, these results suggest that IDV remains effective and stable in the face of severe domain imbalance.
22


Reviews:
[{"review_id": "wF4QCMeHeg", "reviewer": "Reviewer_y5Ga", "summary": "This paper attempts to mitigate the influence of domain-specific futures for domain generalization. The authors proposed an unlearning strategy during training, named Identify and Unlearn (IU). IU firstly selects hard samples which contribute little to the generalization performance, then utilizes Inter-Domain Variance (IDV) to identify domain-specific channels. Subsequently, IU uses Domain-Specific Gradient-Ascent (DSGA) to mitigate the adverse effect of domain-specific features. Experiments demonstrates that IU could enhance the generalization performance across benchmarks.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "[+] The paper is easy to follow.\n\n[+] The detail of the method and experiment is well described.\n\n[+] The related work is detailed.", "weaknesses": "Major weakness:\n\n[-] The authors propose Inter-Domain Variance (IDV), which measures the variance of a channel’s within-domain variances across source domains. Since these within-domain variances are numerical values, what is the conceptual significance of taking their variance? How does this metric effectively identify domain-specific channels?\n\nFor example, consider a channel whose within-domain variances are [4, 4, 4] and within-domain means are [1, 5, 9] across three source domains. Would this channel be considered domain-invariant under the proposed IDV criterion? Please clarify the intuition behind this measure.\n\n[-] The idea of suppressing domain-specific features via channel manipulation is not novel; methods such as DomainDrop [1] and DMDA [2] also employ this strategy. However, these methods apply channel modulation globally, whereas the proposed IU method suffers from performance degradation when applied globally, as shown in Table 2.\n\nWhat is the underlying reason or mechanism that causes global channel manipulation to degrade performance in the proposed framework, in contrast to its success in prior works? Please explain this discrepancy.\n\n[-] The paper aims to preserve domain-invariant information while suppressing domain-specific features. To visually demonstrate the effectiveness of this objective, it would be helpful to include t-SNE visualizations that reflect the domain gap, ideally in comparison with state-of-the-art methods. \n\n[1]. DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization. ICCV, 2023.\n\n[2]. Rethinking Domain Generalization: Discriminability and Generalizability. TCSVT, 2024.\n\nMinor weakness:\n\n[-] The writing quality should be improved. For instance, punctuation marks are frequently missing after mathematical formulas, and the reference notation in Table 1 should be formalized to enhance readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1VAlzRmUaP", "reviewer": "Reviewer_etHq", "summary": "This paper proposes a model-agnostic post-epoch unlearning module (IU) that effectively improves domain generalization performance. While the idea is interesting and empirically validated, the methodological details (especially in identifying domain-specific channels and the theoretical analysis) remain insufficiently clear.", "questions": "Please refer to the weaknesses listed above.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The motivation for applying unlearning in domain generalization is logical and interesting.\n\n2. The experimental results based on improvements over existing methods are sufficient, including comparisons with recent approaches and evaluations on comprehensive datasets.", "weaknesses": "1. In the first part of the Introduction, the authors’ perspective that models should learn domain-invariant features while unlearning domain-specific features aligns with most prior DG studies. However, the details of how texture-sensitive channels and background-sensitive channels are identified remain unclear. It is not specified which layer of the model these channels belong to, nor how they are distinguished.\n\n2. In the motivation part of the Introduction (Figure 1), the phenomenon that texture-sensitive channels and background-sensitive channels correspond to domain-invariant and domain-specific features, respectively, is easy to understand. However, in the Method section, the authors do not explain how texture and background are located, but directly define low IDV values as domain-invariant features and high IDV values as domain-specific features. If, as the authors state, texture and background correspond to low and high IDV values, respectively, does that mean IDV values are used to determine texture and background? Is IDV the necessary condition for this distinction? This remains unclear.\n\n3. The calculation details of AV and IDV are not sufficiently compared, making it difficult to understand the improvements achieved by IDV.\n\n4. The comparison methods are somewhat incomplete. Table 1 only shows the performance improvement when the proposed method is combined with existing methods. However, it lacks a comparison of how existing methods alone improve the same baselines (e.g., DomainDrop [1]).\n\n5. The theoretical proof in Section 3.4 (“THEORETICAL ANALYSIS”) seems weakly related to the proposed DSGA method. It is unclear which encoders correspond to the two feature types $f_{spc}$ and $f_{inv}$. In the Method section 3.3, only one model \n$\\theta$ is introduced, and there are no separate encoders $\\theta_{spc}$ and $\\theta_{inv}$. Therefore, the theoretical section contributes little to supporting the main method.\n\n>[1] Guo J, Qi L, Shi Y. DomainDrop: Suppressing domain-sensitive channels for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023: 19114–19124.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qZN1F99y31", "reviewer": "Reviewer_oxVH", "summary": "This paper introduces Identify and Unlearn (IU), a model-agnostic module for domain generalization (DG). The core idea is to move beyond purely preventative DG strategies and introduce a corrective, post-epoch unlearning mechanism. The IU module operates in three steps: 1) It uses an unlearning score, derived from influence functions, to identify a set of training samples that increase model complexity without contributing significantly to generalization (Unlearning Set Selection). 2) It introduces a metric, Inter-Domain Variance (IDV), to identify feature channels that are domain-specific by measuring the cross-domain variance of their within-domain activation variances. 3) It performs Domain-Specific Gradient Ascent (DSGA) on the parameters of the identified domain-specific channels for the samples in the unlearning set, selectively removing domain-specific knowledge while preserving domain-invariant features. The experiments demonstrate that IU consistently improves the performance of different DG baselines.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of post-epoch unlearning for DG is a fresh perspective that addresses a limitation in the field, moving beyond static regularization techniques.\n2. The experimental evaluation includes a diverse set of baselines across seven different benchmarks. This thoroughness provides strong evidence for the method's model-agnostic nature and general effectiveness.\n3. The motivation, methods, and results are easy to follow. The ablation studies are clear in justifying the design choices of the proposed IU module.", "weaknesses": "1. The primary weakness of the proposed method is its extreme computational overhead. As reported in Table 11, the unlearning set selection step, which relies on influence function estimation, takes approximately 354 minutes (~6 hours) per epoch on the DomainNet dataset. This makes the method less practical for large-scale tasks. The lack of a baseline ERM training time in Table 11 also makes it difficult to fully grasp the relative overhead.\n2. The paper presents IDV as a principled, domain-aware metric for identifying domain-specific channels. However, the ablation study in Table 2 (ERM_DSCS) shows that applying gradient ascent to channels selected by IDV across the entire training set leads to a catastrophic drop in performance. This suggests that IDV is not a robust metric for channel selection on its own; its effectiveness appears to be entirely dependent on its application to the very small, targeted unlearning set.\n3. Table 1 presents results for both IU and IUE (IU with Exponential Moving Average). EMA is an orthogonal technique that is not part of the core contribution. Including IUE results in the main comparison table somewhat obfuscates the direct impact of the IU module and can give an inflated impression of the gains.\n4. While framed as a strength, performing the complex IU operation after every single epoch may not be optimal. The influence of samples and the set of domain-specific channels might not change dramatically from one epoch to the next. A more efficient approach could be to run the IU module intermittently (e.g., every 5 epochs), but this is not explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1uwFiFEkH8", "reviewer": "Reviewer_ZijE", "summary": "This paper proposes Identify and Unlearn (IU), a post-epoch correction module for Domain Generalization (DG). While most DG approaches aim to prevent the learning of domain-specific features, this paper argues that once such biases emerge, there is no mechanism to correct them during training. IU fills this gap by periodically identifying and unlearning domain-specific dependencies.\n \nThe method consists of three parts: (1) Unlearning Set Selection (USS) uses influence functions to find samples that increase model complexity but contribute little to generalization, (2) Inter-Domain Variance (IDV) measures the variance of per-domain activations to detect domain-specific channels, and (3) Domain-Specific Gradient Ascent (DSGA) reverses gradients of those channels to reduce their influence.\nIU is model-agnostic and improves performance by roughly ~3% across seven DG benchmarks and fifteen baselines (ERM, IRM, MixStyle, etc.).", "questions": "​1.​Have you tested whether computing unlearning scores less frequently (e.g., every few epochs) preserves performance while reducing runtime?\n​2.​How sensitive is the IDV measure to the number of domains or imbalanced domain sizes?\n​3.​Have you considered adding a normalization step or a bootstrapped confidence interval for IDV to make it more robust?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper clearly articulates an important limitation in DG research: existing methods lack any corrective mechanism once domain-specific bias has been learned. The post-epoch unlearning framework is conceptually elegant and well-motivated.\nThe combination of USS and IDV forms a coherent pipeline—sample-level influence and channel-level analysis complement each other effectively.\nThe experimental section is comprehensive, spanning multiple DG settings with consistent improvements, and the writing is clear and well-organized.", "weaknesses": "The computational overhead of IU remains a concern. The authors mention that the inverse-Hessian approximation uses LiSSA, which is efficient, but the main cost comes from recalculating unlearning scores for every sample after each epoch. It would be useful to know whether they have considered lowering the computation frequency (e.g., every K epochs) or adopting stochastic sampling for approximate score estimation.\nThe description of IDV is intuitive, but more discussion is needed on potential limitations. For instance, how stable is IDV when the number of domains is small or when within-domain noise dominates between-domain variance? Have the authors explored normalization or any other strategies to stabilize this measure?\nThe paper’s claim that DSGA performs “unlearning” could benefit from conceptual clarification. Gradient ascent here functions as a form of targeted regularization, not data or privacy unlearning. A short note distinguishing these definitions would prevent confusion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XoXS1AF88m", "reviewer": "Reviewer_YFJ2", "summary": "This paper introduces Identify and Unlearn, a novel, model-agnostic module designed to improve domain generalization. The core idea is to perform a post-epoch unlearning step to correct the model's reliance on domain-specific features as they emerge during training. The authors demonstrate through extensive experiments on numerous benchmarks and fifteen baseline methods that integrating the IU module consistently improves out-of-distribution accuracy.", "questions": "See Weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The central idea of treating DG as an iterative process of learning and adaptive unlearning is novel and compelling.\n- The proposed Inter-Domain Variance (IDV) metric is a principled and intuitive method for localizing domain-specific representations within the model. The ablation study provides clear evidence of its superiority over the more naive Aggregated Variance (AV) heuristic.\n- The paper's claims are supported by a remarkably thorough set of experiments and analysis.", "weaknesses": "- The primary practical weakness is the substantial computational cost, as honestly reported by the authors. The reliance on per-sample influence function estimation makes the training time per epoch an order of magnitude higher than standard training. While feasible for research, this presents a major barrier to practical adoption. Could the authors comment on potential strategies to mitigate this cost, such as performing the IU step less frequently (e.g., every 5 epochs)?\n- The core claim of the paper rests on the idea of surgically unlearning from the correct channels. While IDV is well-motivated, the analysis lacks a crucial control experiment: what would happen if DSGA were applied to the domain-invariant channels for the identified unlearning set?\n- The unlearning step relies on a first-order approximation of retraining via influence functions. When applied repeatedly after every epoch, it is unclear how this approximation error might accumulate. Could this repeated, approximate intervention push the model into an unstable or unforeseen state over the course of training, even if it appears beneficial in the short term? A discussion on the long-term stability of this iterative unlearning process would be valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RNZoNKbJSf", "reviewer": "Reviewer_CNti", "summary": "This paper presents \"Identify and Unlearn\" (IU), a model-agnostic module that addresses domain generalization by continually mitigating reliance on domain-specific features during neural network training. IU combines an unlearning score (to identify training samples that disproportionately increase model complexity while failing to improve generalization), an Inter-Domain Variance (IDV) metric (to identify domain-specific channels), and a domain-specific gradient ascent procedure to selectively remove these harmful features while preserving domain-invariant content. The method is thoroughly evaluated on seven benchmarks and multiple baselines, demonstrating consistent improvements in out-of-distribution performance.", "questions": "See in the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.\tThe proposed inter-domain variance metric is a novel identification of domain-specific channels.\n2.\tThe proposed UI module is model-agnostic, which could achieve performance improvements over multiple datasets and methods.", "weaknesses": "1.\tThe motivation is unclear. The authors claim that prior approaches (e.g., Aggregated Variance, AV) are highly sensitive to domain imbalance, whereas the proposed inter-domain variance is robust. However, this issue is neither revealed nor proved experimentally, as the datasets used do not appear to exhibit large domain imbalance. Besides, many DG methods already suppress domain-related features or channels during training, which seems to be in conflict with the existing methods that operate “without any mechanism to correct such reliance once it emerges.” Finally, the claim that AV ignores cross-domain variability also lacks direct evidence.\n2.\tThe details of Fig. 1 should be provided. The experimental description of Fig. 1 is vague. The paper does not specify how the fur-edge channel and background-sensitive channel are distinguished. It is also unclear how the analyzed features are extracted. \n3.\tThe contributions should be emphasized. The core contribution is the adaptive post-epoch removal of domain-specific features. Since this requires access to training data and adds computational overhead, what advantages does the post-epoch strategy offer over in-training suppression? How does the proposed procedure relate to prior domain-specific feature/channel suppression methods? Can the introduced IU module be applied on top of, or integrated into, those methods?\n4.\tMissing conceptual explanation. The key unlearning set selection relies on the measure of model complexity and a generalization score. Although citations and formulas are provided, the paper does not explain why these items capture “the impact of each training sample on both model parameters and validation performance.” \n5.\tWeak theory–method alignment. Theorem 1 crucially depends on Assumption 1, which decomposes features and parameters into domain-specific and invariant components, followed by separate analyses. The link between this decomposition and the proposed algorithmic steps is weak, and the result appears applicable to any domain-specific feature-suppression method. Please clarify which parts of the theorem the algorithm explicitly exploits or adjust the theoretical development to align more tightly with the implementation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)