Diversity Sampling Regularization for Multi-Domain Generalization
Authors: Lakpa Tamang
Mohamed Reda Bouadjenek
Sunil Aryal
Richard Dazeley
Venue: Accepted by TMLR ()

Abstract: Domain Generalization (DG) seeks to create models that can successfully generalize to new,
unseen target domains without the need for target domain data during training. Traditional
approaches often rely on data augmentation or feature mixing techniques, such as MixUp;
however, these methods may fall short in capturing the essential diversity within the feature
space, resulting in limited robustness against domain shifts. In this research, we revisit the
importance of diversity in DG tasks and propose a simple yet effective method to improve DG
performance through diversity-sampling regularization. Specifically, we calculate entropy
values for input data to assess their prediction uncertainty, and use these values to guide
sampling through Determinantal Point Process (DPP), which prioritizes selecting data sub-
sets with high diversity. By incorporating DPP-based diversity sampling as a regularization
strategy, our framework enhances the standard Empirical Risk Minimization (ERM) objec-
tive, promoting the learning of domain-agnostic features without relying on explicit data aug-
mentation. We empirically validate the effectiveness of our method on standard DG bench-
marks, including PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet, and through
extensive experiments show that it consistently improves generalization to unseen domains
and outperforms widely used baselines and S.O.T.A without relying on any task-specific
heuristics.

Published in Transactions on Machine Learning Research (02/2026)
Diversity Sampling Regularization for Multi-Domain Generalization
Lakpa Tamang Deakin University
Ltamang@ddakknneedee edu..on
Mohamed Reda Bouadjenck
reda wad.bbnuddjnnkkkkdkii.......
Deakin University
Richard Dazeley
Hichard.dazeeeyyddaaii..dddd
Deakin University
Sunil Aryal Deakin University
sunil.aryal@deakin. edw.siiiiiedee..
Reviewed on OpenReview: https: openreview. net/ forum? id- nXght7X2RX
Abstract
Domain Generalization (DG) seeks to create models that can successfully generalize to new, unseen target domains without the need for target domain data during training, Tradi- tional approaches often rely on data augmentation or feature mixing techniques, such as MixUp; however, these methods may fall short in capturing the essential diversity within the feature space, resulting in limited robustness against domain shifts. In this rescarch, revisit the importance of diversity in DG lasks and propose a simple yet effective method to improve DG performance through diversity-sampliin regularization. Specifi- cally, we calculate entropy values for input data to assess their prediction uncertainty, and use these values to guide sampling through Determinantal Point Process (DPP), which prioritizes selecting data subsets with high diversity. By incorporating DPP-based diver- sity sampling a regularization strategy, our framework enhances the standard Empirical Risk Minimization (ERM) objective, promoting the learning of domain-agnostic features without relying on explicit data augmentation ion. We empirically validate the effectiveness of our method on standard DG benchmarks, including PACS, VLCS, OfficeHome, Ter- raIneognita, and DomainNet, and through extensive experiments show that it consistently improves generalization to unseen domains and outperforms widely used baselines and S.O.T.A without relying on any task-specific heuristics. Our implementation is available at:
1 Introduction
Machine learning models have demonstrated tremendous progress across diverse applications, yet they con- timue to struggle when confronted with changes in data distribution. Due the inherent nature of training such models with independent and identical distribution (ii.d) hypothesis, their generalization performance unseen test data in a heterogencous distribution space is significantly impacted (Tamang. (024). This phenomenon in machine learning is termed as the Out-of-Distributtin (OOD) problem, where the change in cnatures/covarialt of the data occurs as a result of domain shift or environmental change. Figure Ia illustrates domain shift problem where semantic categories like dog, horse, giraffe, and house appear across multiple domains such as art, cartoons, photos, and sketches. Although the label space is unchanged, visual attributes such texture, color, and abstraction may vary significantly. This highlights two critical chal-

---

Published in Transactions on Machine Learning Research (02/2026)
Domain Shift
70
67.59
Dog
Target (%6
Aug. avgi 61:36
65
Labels
Horse
62.34
62.8
61,01
61.6
60.03
60
Girrafe
55
House
ERM
MisUp
MixStyle
MfoldMisUUp
CutMix
Ours
Art
Color
Photo
Sketch
Features/Covarrates
(a)
Figure 1: (a) Domain shift illustration. (b) DG target accuracy comparison averaged across five benchmarks.
lenges: (i) models trained on a single domain often fail to generalize to others, and (ii) successful generaliza- tion requires learning representations that ignore domain-specific cucs while preserving labol-discimmnatiive features. Domain Generalization (DG) [Zhou al. 2022) aims to address these challenges by leveraging multiple source domains to learn representations that remain stable across changes in the environment. The goal is to reduce reliance on domain-specifie cues and instead capture features that are both label- discriminative and domain-invarian, enabling robust generalization performance across new and previously unseen domains.
Empirical Risk Minimization (ERM) (Vapnik 1998 is considered decent baseline method to achieve rea- sonable In-Distribution (ID) performance. However, without proper regularization, ERM tends to overfit domain-specific spurious correlations, leading to poor generalization to unseen domains where such correla- tions may not hold. Another prevalent method in DG is achieved through data augmentation and synthesis (Li et al. 2018a Zhou et al. 2020). These approaches perform sugmentation by interpolating data points and their labels in imput space (Zhang et al.) 2017), hidden manifold space (Verma et al. 2019), and by mixing style statistical informations (eg: mcan, std) of features (Zhou et al. 2021), or replacing random patches of one input with another while mixing their corresponding label (Yun et al. 2019). Such techniques are known expose the model to a large variety of source domain samples, intrinsically learning more domain-invariant and hence generalizable representations. However, these methods are also prone to consuming generous amount of computing resources and require considerable engineering efforts to synthesize inputs to represent domain-agnostic features. Besides, some of these met utilize conventional augmentation methods (crop, rescale, rotate) and patch-based synthesis, limiting their versatility across other applications.
In this paper, we propose an alternative and relatively straightforward approach to enhance DG performance by regularizing the model to promote diverse feature learning. Our methodology comprises two steps: First, we calculate entropy provide an uncertainty measure of the input data. Sccond, we utilize this entropy as supplementary signals sample diverse features through Determinantal Point. Process (DPP) sampling [Kulesza al.) 2012). DPP offers class of precise probabilistic models for sample selection problems and is particularly effective for addressing subset selection problems with diversity constraints, such as video and document sunmarization (Launay et al. 2021). Our DPP sampling-guie guided regularization supports the original training objective of the existing ERM baseline to learn domain-agnostic classifier. Unlike conventional data augmentation methods that achieve diversity by generating or synthesizing additional samples, our mothod inherently incorporates diversity within the sampling process.
Our main contributions summarized follows:
2

---

Published in Transactions on Machine Learning Research (02/2026)
We propose DG method that enhances generalization across unseen domains by sampling diverse data representations near the decision boundary using uncertainty-guided DPP sampling.
Our approach provides an effective form of regularization that reduces reliance on complex data augmentation or synthesis, offering simpler alternative to augmentation--diien DG methods.
We demonstrate that our regularization technique is method-agnostic and can be seamlessly in- tegrated into existing DG frameworks by augmenting their final objectives, thereby promoting its applicability across wide range of DG problems.
Through empirical results, we show consistent improvements: (see Figure 1b about +5% over the ERM baseline, +6% over the average of popular augmentation-based DG benchmarks. We also beat several state-of-thee-rt DG techniques across multiple datasets.
2
Background and Related Works
Domain Generalization: DG Ye et al. 2022; Xu et al. 2021 Balaji et al. 2018 has been subject of extensive investigation, alongside related approaches such as transfer learning and domain adaptation. A typical approach in DG is through learning domain-invariant representations across multiple domains (Li et al. 2018a Muandet et al.) 2013) with an assumption that all domains share common features that are inherently agnostic and do not belong to a particular domain. Existing methods attempt to learn these fcatures through techniques such as invariant fcature learning (Li et al. 2018b), disentanglement lcarming (Zhang al.] p022,, causal inference (Mahajan et al. 2021 etc. Although the distillation of domain- agnostic features for classification remains promising, the precise determination of which features should be considered domain-specific remains significant question. Similarly, substantial portion of the literature in DG has been realized through adversarial learning such as GANs [Poursaced et al. 2021), and AutoEncoders Ghifary et al., 2015) with an objective of learning universal representations. However, the majority of these aforementioned methodologies are computationally expensive and involve complex conceptualizations. In fact, recent research (Gulrajani & Lopez-Paz, 2020) challenges the performance of most of these models through simple ERM framework that has undergone careful hyper-parameter tuning. Interestingly, it has been empirically demonstrated that simple model such ERM can achieve comparable results outperform complex approaches when evaluated in terms of OOD accuracy. Later studies (Cha et al. 2021 Wang et al.] 2023) have argued that, simply minimizing the empirical loss on a complex and non-convex loss landscape is typically not sufficient to arrive at a good generalization, thereby proposing to find flat minima on the loss landscape for improving DG performance. Despite the improvement, such a gain is usually altributed to heuristic approximations and these methods fail to strongly utilize domain-agnostic information.
Determinantal Point Process (DPP): DPP has proven to be an effective methodology for ensuring diversity in a wide range of applications. Historically, DPPs have found favor in information retrieval tasks, including text summarization (Perez- Beltrachini Lapata 2021), 1 image selection (Kulesza & Taskar 2011
recommendation systems (Chen al. 2018 Celis al.] 2018), and video summarization (Wilhelm et al. 2018). Beyond these fields, DPP sampling has demonstrated utility in various computer vision challenges including pose estimation (Kulesza & Taskar 2010), image processing for pixels patches sampling (Launay et al. 2021). transfer learning for generating diverse training subsets with enhanced transferability (Ly et al,
2022 multi-label classification (Xic et al.) 2017), and active lcarning (Biyik et al.) 2019). DOMI (Leng
et al. 2022), which is one of the closest work to ours uses DPP as a two level sampling process however, it necessitates features from two distinct networks and offers little to no information regarding scalability current DG benchmarks. Conversely, our approach can be scamlessly intograted as a straightforward plug-and-play technique with widely used existing methods, and we demonstrate its scalability across the current popular DG benchmarks.
3

---

Published in Transactions on Machine Learning Research (02/2026)
Dom, 0 Cls.
X
Dom, Cls.
Dom. Cls.
Dom. Cls.
Dom. Cls. 0
X
Dom. 2 Cls.
0.7
0.6
0.5
0.4
02-
w/o diversity
Target 83.25%
Target 94.75%
40
100
Epoch
(a)
(b)
(c)
Figure 2: Comparison of decision boundaries for (a) ERM and (b) ERM with diversity regularization. ERM alone produces a misaligned boundary with lower target accuracy (83.25%) Adding the DPP-basee diversity component yiclds a more robust boundary and substantially higher accuracy (94.75%), illustrating how diversity promotes domain-agnostic generalization. Source domains are [Dom 0, Dom 1), target domain Dom 2, with labels [Cls O, Cls 1]. (c) Comparison of the classification loss curve relative to the number of epochs.
3
The Proposed Methodology
3.1 Problem Formulation
Let X' denote the input space and y = (1...., C} the label space for C' class classification problem. We are given labeled samples from N source domains DI,..., DN, where each domain D; contains samples (x, y) (.( We consider a hypothesis class F defined as F= {fe fe(x) = g(h(x)), E H. g E g}, where h X' + Rdd a feature extractor and g TRd + RCC is classifier. The goal to learn a model fz X' RC that gencralizes well on a previously unscen target domain D, Pr Y), where Pr & (P.) In other words, the model is trained to minimize the empirical risk on source domains Rs(f) = Σ E (x,)) ((([(x)))) to obtain:
fö = arg min Rs(f)
FEF
with an assumption that få properly generalizes the target domain with true target risk given by:
RT(S) = Bx (
(2)
Then, under the standard domain shift assumption, the target risk can be upper bounded as:
RT(A) < Rs()) +
(3)
Here, C denotes generic domain discrepancy term that measures the distributional shift between the collection of source domains and the target domain, and can be instantiated by any divergence distance for which risk transfer bound of the form in 3 holds (e.g., integral probability metrics (IPM) or f divergence based measures), depending on the underlying assumptions on the data distributions, loss function, and hypothesis class. Note that, throughout this work, we treat abstractly and do not assume a specific form, as our method aims to roduce domain diserepancy implicitly via representation diversity rather than optimizing an explicit divergence.
3.2 Motivating Toy Example
We begin by motivating our problem through a simple toy example. Here, we construct a synthetic dataset using multivariate Gaussian distributions that comprises three domains and two classes; each class containing
4

---

Published in Transactions on Machine Learning Research (02/2026)
200 data points. The dataset was crafted to maintain consistent covariance across all domains while altering only the mean, hence emulating the domain shift nature in a real-world DG task, where object's s feature values change based on the environmental context. Figure 2aj shows the decision boundary learned by a standard ERM model trained across two source domains. Here's what we observe; although the model caplures some of the class separation, the boundary is misaligned with the data distribution. As a result, substantial portion of the clusters of the target domaín data are incorrectly classified, yielding a target accuracy of only 83.25%. This reflects a common limitation of ERM: the model tends to overfit to dominant patterns in the training set and fails to generalize well when class and domain structures overlap.
To this end, we introduce a diversity regularizer Raiv, which we integrate into the source risk minimization framework Eq. [ Specifically, instead of minimizing only the empirical source risk Rs(f), we consider the following objective:
f -ar min IEF
[s((J Rdiv):
where, Raiv is the diversity regularizer. By explicitly encouraging the model to altend to diverse samples dur- ing training, the decision boundary becomes more robust and better aligned with the data manifold as shown in Figure 2b This adjustment resolves much of the mis-classification seen in the ERM, especially around the overlapping regions of different domains. Consequently, the target accuracy increases substantially to 94.75%. Building on this motivating example, we formalize our method Diversity Sampling Regularization (DSR). As shown in the block diagram in Figure 3a, DSR leverages predictive uncertainty followed by DPP sampling to induce diversity constraint among feature vectors 2, where h(x) E Rd. In the following sections, outline how DSR is incorporated into the optimization objective alongside the source risk Rs(f).
3.3 Diversity in Feature Space
There are two key arguments how DPP supports to domain invariant feature learning. Firstly, DPP encourages inter- and intra-domain diversity, ensuring that the selected examples span larger volume in feature space, covering more modes of the data distribution. Secondly, unlike random sampling, there is less redundancy in selected features, which makes the probability of selecting the same or similar samples very DPP, which is a negative correlation between
low. This can be attributed to the pivotal characteristic binary variables, translating that the selection of one item reduces the probability of selecting others that are highly similar. As shown in Figure 3b] the larger the angular spread, the more diverse the items are. We utilize these correlations, which are determined by kernel matrix measuring item similarity. Consequently, more similar items are less likely to co-occur, resulting in DPPs favoring diverse subset of items.
3.4 Uncertainty-based Feature Modulation
In DG, not all samples contribute equally to improving generalization. In reality, confident predictions often correspond to redundant or well-understood features, whereas uncertain predictions may represent underexplored regions of the input space (Hüllermeier & Waegeman 2021). To exploit this, we compute the predictive entropy -- --=-b - de log We for each sample based on the softmax output of the classifier de = softmax(g(h(x))). This entropy is then used to scale each feature vector, effectively reweighting the feature matrix before passing for DPP sampling. We term this phenomenon as feature modulation and represent the byproduct as modulated feature given by:
z=z.u
(5)
The intuition here is that uncertain samples are likely to contain more informative ambiguous signals, and emphasizing them encourages the model to explore diverse and non-rodundant directions in feature space. This entrop modulation serves two primary purposes. First, it biases learning toward instances the model already deems informative, implicitly encouraging robustness across domains. Second, and more critically for our method, it alters the structure of the feature space on which we perform DPP-based diversity sampling. Empirically, we find that this strategy leads to more expressive representations and contributes significantly to performance under domain shift.
5

---

Published in Transactions on Machine Learning Research (02/2026)
High Entosy
Ennoay
Enraay
0.5
0.1
D1
Confidunce
Domains
Source
da
Littel
D2
2
9
U
Kernel Commination
DN
R divi
Sampling
DPP
Compuser Logedet
Intermediate Outcome
Network
Process
(a)
Cartoon
Sketch
Low similarity
High similarity
Cartoon
Picture
Low Diversity
High Diversity
(b)
Figure 3: (a) Schematic block diagram of DSR method. Based on the entropy scale, the high entropy values are prioritized for DPP sampling. (b) Diversity representation between data points. The larger the angular spread, the higher the diversity.
3.5 Diversity Kernel with DPP
Next, to promote diverse representation during training, we construct a kernel matrix L E BxB over the batch, B of the modulated feature vectors à using an RBF (Gaussian) similarity kernel.
Li = exp(->llž; 2, (b)
(6)
This kernel captures pairwise dissimilarities among feature vectors. In situations where two features are exactly same, 2, = ăj, Lij + 1. However, two independent features in a batch are never exactly the same, given they come from different input samples, which brings our Lij to span between and
Role of As the inverse bandwidth parameter, 7 0 determines the width of the RBF kernel. Since the mini-batch comprises samples from different domains that are shifted along the covariates, we use a dala-dependent which can capture the variances that are intrinsic to the shift dynamics. Specifically, we use a principled, data-dependent median heuristic measure (Garreau et al. 2017 Gretton et al. 2012) that is computed over pairwise distances of features in batch. Intuitively, the median value renders y to be robust outliers and automatically adapts to the scale of data in inadequately shifted distribution space. Mathematically, it is represented follows:

---

Published in Transactions on Machine Learning Research (02/2026)
7 median| 22 - 2 -
where e is a small positive constant that ensures numerical stability. Since the modulated fealures rellect. uncertainty, the diversity modeled by the DPP naturally emphasizes informative regions the feature space. This leads more mcaningful subset selection and loss computation, eventually improving generalization.
3.6 Reducing Domain Discrepancy through Diversity Regularizer Rdiv
Next, we utilize the previously obtained L, to enforce diversity in the learned feature representations via a log-determinant regularizer:
Raiv - log det(L + EZ),
(8)
To ensure that the kernel matrix L strictly positive definite and the log-determinant is well-defined, we regularize it by adding small multiple of the identity matrix I, with > 0 being small constant. This guarantees that all eigenvalues of L remain positive, avoiding numerical instability caused by near-zero negalive eigenvalues. Such a regularization is commonly practiced in determinant-basel objectives and ensures that the log det term remains finite and differentiable (Kulesza et al., 2012). This regularization encourages the feature vectors to be geometrically diverse, suppressing collapse into redundant subspaces.
Proposition 1. Mazimizing log de + el) promotes non-redundant, orthogonal features that span a high- volume subspace, thereby improving representation robustness.
Lemma 1. Kulesza el al., 2012 Let A Am be the eigenvalues of L. Then,
m
log det (Lee)(-++))) Σ log(Ae + +c),
which is marimized when the eigenvalues are uniiforml large and spread out, implying orthogonality and high entropy in the feature distribulion.
Since det(L+el) = II +c), maximizing its logarithm requires enlarging all eigenvalues of L. If the spectrum collapses so that only a few eigenvalues dominate, the determinant shrinks, reflecting concentration of variance in a low-dinensional subspace. Conversely, when eigenvalues are more uniformly distributed and nonzero, the determinant grows, corresponding to orthogonal features that span a higher- volume subspace. Thus, maximizing log det(L + el) promotes diversity by penalizing redundancy among features.
Proof sketch:
Discussion:
Diverse representations reduce the overlap between features of different domains, leading to more domain-invariant embeddings. Maximizing diversity thus implicitly aligns source domains in a shared high-volume subspace, shrinking the discrepancy term C in the generalization bound in Eq. 3
3.7 Interpretation of the log-det
In DPP sampling, the model is penalized if the mini-batch samples are redundant or lie in low-dimensional subspace. To illustrate the log-det procedure mathematically, let us consider the subset S containing two data points x,, and xj, then the probability of sampling can be calculated as:
Li
Lij
P(i.j E det(Ls)
Ljj
(9)
LaLj Lyg Lyi = Li
From Eqs. and we can deduce that, under ideal scenario if x,, and x, are dissimilar each other, then Ly + 0, and S will have high probability. Accordingly, the selected subset with the highest should contain set of highly diverse samples.

---

Published in Transactions on Machine Learning Research (02/2026)
It is important to note that in geometric intuition, the determinant det(Ls) represents volume of the par- allelepiped spanned by the feature vectors ă E S in the feature space Thus, by maximizing det(Ls), the selected subset S ensures that the feature vectors are linearly independent and spread out in representation space. This diversity implics that our method captures a wide variety of diverse features among 2, reducing the likelihood that the model overfits to any specific domain.
3.8 Overall Training Objective
With the regularization term formulated, DSR's final training objective corresponding to Eq. becomes follows:
N
E (xx)) (e([(x).))) +(1 - a:) Raiv where a E [0, balances the contribution of Ltotal = a
N
ERM
the diversity term.
4
Experimental Setup
In this section, we outline the experimental sctup of our method. This includes details regarding the datasets used, training configuration, evaluation metrics and selected baseline studies for comparison. All experiments were conducted multiple A100 GPU servers.
4.1 Datasets
The PACS dataset (Li al. 2017 comprises four distinct domains: Photo, Art Painting, Cartoon, and Sketch. Each domain contains images categorized into seven classes: Dog, Flephant, Giraffe, Guitar, House, Horse, and Person.
PACS:
VLCS: The VLCS dataset (Fang et al. 2013) combines four distinct image datasets: PASCAL VOC 2007 (V), LabelMe (L), Caltech-101 (C), and SUN09 (S). This consolidated dataset encompass five shared cate- gories: Bird, Car, Chair, Dog, and Person.
OfficeHome (OH): The OfficeHome dataset. (Venkateswar et al. 2017), multi-domain dataset, developed to facilitate research in domain adaptation and generalization. This dataset encompasses four distinct domains: Art, Clipart, Product, and Real-World.
TerraIncognita (TI): TerraIncoguita (Beery et al., 2018) comprises wildlife camera trap images collected from multiple geographical regions, referred to as "locations."
Net (DN): DomainNet (Peng al. 2019 prominent dataset in DG, comprising six distinct domains: Clipart, Infograph, Painting, Quickdraw, Real, and Sketch.
In Table. we provide detailed information regarding the distribution of samples in each domain of the DG datasets as discussed above. We can observe that the number of image samples in each domain vary significantly across all datasets.
4.2 Implementation Details
We trained our method on Resnet-50 architecture using the Stochastic Gradient Descent (SGD) optimizer for up to 100 epochs. The learning rate was set to 0.01 with decay rate of 0.75. The value of batch size was set to 64 and the value of a was set to 0.5 to balance the contributions of the loss term and the regularizer. We follow the training-validdttin
approach of (Wang & Lu 2021 where any one domain in the dataset is Our method's performance is compared Mixup (Yan al. 2020), CORAL
reserved for testing purposes while the rest are used for training.
against some of the widely recognized DG baselines: ERM (Vapnik]
1998
(Sun & Sacnko. 2016), MMD [Li ct al. 2018b), DANN (Li ct al.
2018b) GroupDRO (Sagawa et al. 2019
Inn DPP, the determinant viewed as of the parallelogram induced when unit square transformed by matrix.
8

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Summary of DG Benchmarks Used the Experiments.
Dataset
Domain
Images
Total
Art Painting
2,048
Cartoon
2,344
PACS
9,991
Photo
1,670
Sketch
3,929
Caltech101
1,415
LabelMe
2,656
VLCS
10,729
SUN09
3,282
VOC2007
3,376
Art
2,427
Clipart
4,365
OfficeHome
15,588
Product
4,439
Real-World
4,357
Location 100
3,381
Location 38
9,530
Terra Incognita
24,788
Location 43
6,293
Location 46
5,584
Clipart
48,129
Infograph
53,201
Painting
72,759
DomainNet
586,575
Quickdraw
172,500
Real
175,327
Sketch
70,386
and S.O.T.A methods such SWAD CCha al. 2021 SAGM WWng et al. 2023 GMDG (Tan et al.
2024h, and I3C (Zhou et al. 2025
We re-implement most of the methods and report the mean and standard
error across three random seeds.
METHODS
PACS
VLCS
OH
TI
DN
Avg.
Vapnik
1998
84.90 0:11
74.20 o.11
p.07
0.39
MixUp
2020
79.50
o.11
66.60
47.30
37,90
61.01
CORAL
Sun Smonlnn kn
2016
82.50 0.19
74.30 +.300--0557
66.80 +
39.50 0.47
36.40 0.48
59.9
GroupDRD
(Sagawa et all
2019
82.60 0.05
71.10
63.00 0.40
39.40 10:00:0000 0.24
32.40
57.7
MMD
al 2018b
84.70 0.17
75.70 0.00
67.80 0.19
11.80 0.20
38.50
SWAD
Cha
A721
88.19
78.77 +
70.50
47.50
46.40
66.27
SAGM
Wang of
2123
86.
77.13 0.14
69.26
48.43 13:43:000
44.60 1::00:000
65.22
GMDG:
Jan dl pli
2114
85.60
79.20
70.70
50.20
44.60
66.06
13C
al. 20/55
79.60
DSR_ (Owrs)
88.
09.10
70.00
51.11
45,60
67.54
DSR; (Ours)
88.70 1 0.09
80.31
0.34
70.24
52.93 0:47
45.25
67.59
Table DG target accuracy comparison DSR along with popular baselines across five different benchmark datasots. Best and socond-best results are bold and underlined, respectively. The results of methods with reported from their original implementation, therefore do not provide the standard error value for them.
5 Experimental Evaluation
In this section, we conduct exlensive experiments and analyses to evaluate the effectiveness of the proposed method on standard DG benchmarks and to provide extensive insights into its associated parameters, For ovaluation, we use target accuracy, which obtained by testing the model on the target domain that has no sample overlap with the source (training) domains. Note that the results are averaged over 3 independent trials, and represent the mean value and standard error across these runs.

---

Published in Transactions on Machine Learning Research (02/2026)
METHODS
Art:
Color
Picture
Sketch
Avg-
Caltech101
LabelMe
SUN09
VOC2007
Avg-
FRM
84.10
80.50
0.02
94.90 + 0.00
80.20
0.11
$4.90
96.40 0:10
62.70 0.16
68.40 0:31
69:20 6.03
74.20
MixUp
85.10 1
80,90
6.17
94,70 1 0.20
57.30
79.50
94.60
62.50 1
69.50
69.00 c.0
73.90
CORAL
82.30 + 0:25
80.20
+ 0.15
92.80
4: 0.00
71.99
= 0.34
82.50
91.60 = 0.70
62.
0.52
T1.50 = 0.30
69.20 = 0.71
71.30
GroupDRO
76.70 +: 0:20
78.00
0.10
90.00
85.00
0.00
82.00
93:90 0.1
60.40
06.90 0:27
64.20 0.21
71.10
MMD
83.50
80.10
1
93.50 1 0.00
$1.80
84.70
96.30
61.70
73.80
70.90
75.70
SWAD
88.59 12 0.14
88,58
0,51
97. 87.82-2 +: 0.22
82.76 0.10
88.
98. 94 + 0.14
63.71 + 0.11
71.10 0.20
78.35 = 0.22
78.77
SAGM
86.51 0.88
83:74 + 0.17
96, 0.87
79.93 = 0.18
86.72
98. = 0.10
61:27 +
72.31 0.18
76.37 0.15
77.13
GMDG
84.7
81.7
97.50
80.50
85.6
96.30
65.90
73.40
79:30
79.20
13C
89.20
80066
97.70
81. 40
87.10
98.80
67.50)
72.10
80..20
79.60
DSR
87.74 0:28
83 75 0.12
97.02 0.51
83,97 0:22
88,
98. = 0.10
//
74 63 0.79
76.26 0.48
79:10
DSR
$9.26 +: 0.09
84.04 0.02
97.25 4: 0.12
84.25 1: 0.15
88.70
988 0.18
68. 90 0.12
75.69 0.15
78.41
0:11
80.31
(a) PACS and VLOS
METHODS
Art
Clipart
Product
Real
Avg.
Loc38
Loc13
Loc46
Loc100
Avg.
ERM
62:20 0.04
53.80
76.00 +: 0.10
77. = 0.09
6T.30
16.10 = 0.11
53. (0
41.90
0.13
40.80 +: 0.05
45.50
MixUp
60.30 = 0.22
53.50
0:20
76 76.90 0.a1
76. 60 +: 0.14
66.60
44.30 = 0.18
52.50 0.10
33.50 0.00
58. 08.90-2 0.25
47.30
CORAL
50.60 0:21
55.90
0:36
75.00
76.50 0.17
39.20
0.37
41.60
32.20
1:0:1
45.00 0.39
39.50
GroupDRO
54.20
53.30
2:
72.00 0.45
72.00 1: 0.54
63.00
35.40
0.28
42.00
31.90 o.as
48.20
39.10
MMD
60.70 = 0.12
58. 70
0.16
75 60
+ 0.28
76.
+ 0.21
67. 80
10.40 = 0.31
47.10
= 0.18
31.40 0.15
45 0.14
41.80 10
SWAD
05.24 0.63
0:13 57.53
78.99
79.88 19.86-1 0.39
36.74
0.21
59.79 0.11
37.93 0:10
55.55 0.97
47.50
SAGM
64.26
55,92
78.04 0.07
78.83 1:
69.26
45.04 ::
57.96
40,26 o.sT
50,48 0.22
48.43
GMDG
68.90
36.20
79.90
82.00
70.70
59. 80
15.30
57.10
38,
50,20
13C
67.50
77.50
79.80
70.20
57.00
43.20
41.90
DSR,
86.05
56.85
1.
77.40 0.11
79.73 1.
70.00
48.22 1
55.76
43.63 L
58.05 1.
51.41
DSR,
66. + 0.28
37.21
+ 0.20
77.61
0.34
79.79 0.43
70.20
50.63 = 0.59
51.90 = 0.30
15.61
0.41
60.59 = 0.61
$2.93
(b) OfficeHome and Terralncognita
METHODS
Clipart
Infograph
Painting
Quickdraw
Real
Sketch
Avg.
ERM
58.00 +: 0.19
16.70
+ 0.21
45 15.30
+ 0.43
14.20
# 0.34
56. 70 + 0.65
48.10 + 0.18
39.80
MixUp
54.90
0.34
16.30
0.57
43.70
0.49
13.70
0.65
52.70
0.89
46.10
0.36
37.90
CORAL
37.10
0.87
14.60
0.37
39.80
0.31
10.70
0.75
50.20
46.40
0.29
36.40
GroupDRO
46.30 = 0.12
15.50
+: 0.09
34.60 + 0.17
10.80
0.43
46. + 0.09
41.10
+ 0.21
32.40
MMD
57.90
0.23
16.00
0.12
41.70 2 0.34
13.00
0.76
54.30 - 0.61
48.00
0.37
38.50
SWAD
65.75
0.12
22.
0.07
52.83
15.37
66.33 0.23
55.52
0.16
SAGM
63.13 0.31
20.56 0.21
51.16 1 0.56
14.22 0.51
64.06 + 0.39
53.47 0.26
44.60
GMDG
63.40
22.40
51.40
13.40
64.40
13C
63.00
21.40
51.40
13.60
63. G0
53.50
44.40
DSR₂
65.30 # 0.32
22.00 + 0.27
52.20 + 0.14
15.30 0.09
64.30 0.19
54.10 + 0.61
45.60
DSRg
65.34 0.14
22.40
0.33
52.31 0.56
15.70
0.21
64.50
54.27 0.06
45.75
(c) DomainNet
Table 3: A comprehensive comparison of DG performance of different methods across multiple benchmark datasets Rows highlighted in blue represent our methods. Best and second-best results are bold and under- lined respectively.
5.1 Performance Comparison
In Table 2 we compare the accuracies recent SOTA frameworks. Here, we present two variants of our method: DSR,, and DSR2. The former utilizes plain features, while the latter uses entropy modulated features for DPP sampling. We can see that through simple regularization, our method provides impressive boost to the ERM baseline, with almost 5% overall gain across the average of five different datasets. Both DSR2, and DSR, were able to outperform most of the methods, while only coming second to SAGM and SWAD by a marginal value across OH and DN benchmarks respectively. One thing to note here is that, SAGM and SWAD both are designed on complex heuristics of the loss landscapes and provide little information regarding the domain features. On the other hand, our mothod is complementary to the existing loss function and do not require modifying any of the original ERM objective. In fact, we show in later section that our approach is also agnostic to most of the models and can boost their accuracy by a comprehensive margin.
accuracies of the proposed method with popular DG benchmarks along with
Another insight to take from Table 2 is that MixUp, popular data augmentation technique known for promoting diversity in the representation space, performs considerably worse than both DSR variants. We further illustrate this difference through a t-SNE plot in Figure | which shows that DSR achieves more compact inter-class scparation on the target dataset compared ERM and MixUp. The comprehensive experimental results comparison of DSRg, and DSR; with the basclines is provided in Table B where we can witness that, in some domains (Loc100 of TerraIncognita), DSR brings upto 50% boost in the accuracy
10

---

Published in Transactions on Machine Learning Research (02/2026)
of vanilla ERM. These findings suggest that regularization markedly improves the DG performance.
(a) ERM.
(b) MixUp.
(c) DSR.
Figure 4: TSNE plot of targel domain of the PACS dalaset. Each colored cluster represents class. (a)
ERM (b) MixUp (c) DSR.
5.2 Model-Agnosticity of DSR
One of the key characteristies of DSR is that it is model agnostic, meaning that it can be coupled with any other method by augmenting their training objective. We show this phenomenon in Figure 5 where we plot the target accuracy of vanilla methods (ERM, DANN, CORAL, MMD, and GroupDRO) and their DSR counterparts averaged across all the available domains of five DG benchmark datasets as discussed in Section 4.1 Through the bar chart, it can be clearly seen that all methods have a certain degree of gain when they are trained with DSR. These gains starting from 1% reaching high as up to 13% reflect the versatility of DSR, promoting its utility among wide range of methods and frameworks.
Vanilla IDPP regularized
Tbaget (9))
15
41.03
00
PACE
VLCg
11
DN
PACB
VLGS
DN
PAOE
vLeg
T1
DN
(a) Mixup
(b) DANN
(c)
CORAL
64
Target (98)
022
70.02
80
80
52. D0
20
60
9004
60
40
40
PACS
VLes
on
T1
BN
PACS
VLOS
of
DN
PACS
vLes
OHl
11
EN
(d) MMD
GroupDRO
(f) SWAD
Figure 5: Comparison of target accuracy of five of the popular DG methods across five different benchmark datasets when trained on a vanilla and DSR regularized setting.
11

---

Published in Transactions on Machine Learning Research (02/2020)
0.6
Deroky
0.6
0.0
0.0
Eigenvatue
Eigenvalue
(b)
(c)
(d)
Figure 6: Analysis of features and eigenvalues after model training the PACS dataset. (a) t-SNE plot
with 2, (b) Histogram and KDE plot of eigen values using x, (c) t-SNE plot with 2, (d) Histogram and KDE plot of cigen values using 2. In (a), and (c) cach color represents a domain and each cluster represents a class.
5.3 Analysis on Feature Modulation of DSR
In Section 5.1 we presented two variants of DSR: DSR,, and DSRg: where the latter consistently provided better DG performance across all domains of all datasets. We attribute this improvement to an effective diversity regularizer driven by entropy-modulated features. Tn this section, by analyzing the feature values and distribution of kernel eigenvalues, clarify why this improvement persists. To better understand the impact of catropy-based feature modulation, first we analyzed both the training feature space structures of z and ă through t-SNE visualization as shown in Figures 6a and 6c respectively. Here, we notice two key observations: First, & shows tightly clustered inter-class samples and the features appear more dispersed across the embedding space. Second, the clustered features appear more domain-agnostic, with the three domains converging toward shared representational basis. From these findings, we deduce that modula- tion encourages the model to exploit more orthogonal directions in the feature space, thereby preventing rodundancy in the learned reprosentation.
Subsequently, we further substantiate this observation through an eigenvalue analysis of the kernel matrices constructed from these features. In Figures 6b and 6d we plot the density of eigenvalues obtained from these features. Here, we empirically validate Lemma as the cigenvalue spectrum kernel matrix of 2 appear more spread out, with a lower proportion of negligible (es 0) eigenvalues and larger support volume in the feature space. Such broader set of informative, non-redundant features benefits DPP sampling, reducing over-reliance on highly confident yet redundant features, thereby improving generalization in the context of domain shift.
5.4 Ablation Study: Effect of Y
In this ablation, we perform an experiment assess the impact of using fixed y values on generalization performance. We trained DSR; with static 7 values from the set {0.01, 0.03,.....0.1 aiming to mirror the empirical range noted during adaptive training. We plot the line-graph of target accuracies with respect to as shown in Figure 7a where we also append the result of adaptive at the end. Interestingly, we notice that not a single fixed value of 7 leads to optimal performance. This reveals that a fixed y in DSR, fails to capture the data-dependent nuances of the fcature space. Conversely, the adaptive y designed to fit the statistical structure of the features at each training slep, promotes elfective diversity regularization yielding better target accuracy, These results support our assertion that adaptively computing 7, such as through the median of pairwise distances enhances DG performance.
5.4.1 Training Different Alpha Values
Figure 7b ibllsstres the effect of varying the hyperparameter a on target-domain accuracy across the PACS, VLCS, and OH datasets. In the case of PACS, accuracy increases until a 0.5, where it attains a peak of 88.7%, before experiencing a slight decline at higher values. A similar pattern is observed for VLCS, with performance peaking at 0.5 (80.5%) and remaining relatively stable at other values. The OH
12

---

Published in Transactions on Machine Learning Research (02/2026)
Target Ace (%)
Targed Ace (%
90
95
80
70
0.01
0.03
0.05
0.08
0.1
adaptive
0.1 0.2 0.3
0.4 0.5 0.6 0.7 0.8 0.9
Art
Cartoon
Photo
Sketch
PACS
VLCS
OH
(a)
Figure 7: (a) Target accuracy on PACS dataset with static and adaptive y. (b) Ablation on alpha values for PACS dataset.
dataset also demonstrates sensitivity a, achicving maximum accuracy of 71.1% at = 0.5, followed decrease increases. Overall, the findings suggest that moderate values of (approxinately 0.5) consistently result in optimal performance across all datasets.
5.4.2 Training with Different Kernels
Figure illustrates kernel ablation study conducted the PACS, VLCS, and OH datasets, comparing the performance across different similarity kernels (Cho & Saul 2009 such RBF, Cosine, Linear, and Polynomial. The results demonstrate that, across all three datasets, the accuracies are relatively similar, with the RBF kernel exhibiting a slight advantage, achieving 88.7% on PACS, 80.3% on VLCS, and 70.2% on OH. These findings suggest that while the choice of kernel exerts only a minor influence, RBF kernel consistently yields the most notable improvements across the benchmarks. This consistent ad advantage motivates our choice of RBF the default kernel in the main experiment.
79
38.66 38.61
4.65
Thrgee Arc (%)
90
80 3
83
Thrget Anc (%)
80
60
36.6
50
533,6
24
10.56
10.2
20
10
70
PACS
VLCS
OH
+-90%
+800%
-90%
RBF
Cosine
Linear
Polynomial
ERM
MixUp
D
(a) Kernel ablation PACS, VLCS, and OH (larget
(b) ColoredMNIST: DG performance of DSR. base-
accuracy).
lines.
Figure 8: (a) Kernel ablation across datasets. (b) ColoredMNIST results.
5.4.3 Experiments on ColoredMNIST
In this experiment, utilize ColoredMNIST (Arjovsky et al. 2019), colored adaptation of the MNIST dataset, frequently employed standard benchmark for evaluating DG in the presence of controlled spuri- ous correlations. It includes three domains characterized by different label-color corrclations: +90%, +80%, and -9%. As depicted in Figure Sb) DSR, exhibits notable performance enhancements over Tradit ional ERM and MixUp techniques across all domain configurations. Remarkably, DSR surpasses baseline meth- ods by double margin even the highly challenging -90% sctting, where the correlation considered be adversarial. These results emphasize the statistical robustness of DSR in mitigating spurious features and improving generalization across diverse and non-i.i.d. domains.
13

---

Published in Transactions on Machine Learning Research (02/2026)
6 Conclusion and Future Works
In this work, we emphasize the undcrexplored rolc of diversity in improving DG performance. By utilizing entropy-based uncertainty to guide DPP sampling, we introduce principled diversity-awarg regularization strategy that augments the standard ERM objective. Our method promotes the learning of domain-invariant features without relying on explicit data augmentation or target domain information. Empirically, we demon- strate that our approach not only consistently outperforms both ERM and augmentation based basclines, but also secures SOTA results across most of the DG benchmark datasets. These results underscore the effectiveness and simplicity of diversity-driven sample selection in mitigating distribution shifts and improv- ing generalization to unseen domains. We acknowledge, however, that our current experiments are limited by the number of source domains, and increasing this number correspondingly raises the computational cost of DPP based sampling due to its dependence on kernel construction and determinant-basee selection. Addressing such a scalability challenge whether through approximate or low-rank DPP formulations, mini- batch level sampling, or more efficient diversily surrogates represents a crucial direction for future research. Overall, these results highlight the effectiveness of diversity-driven sample selection while clearly outlining its current computational limitations and future scalability potential.
References
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz, Invariant risk minimization. arXiv preprinl (xXi::1907.0889) 2019.
Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. Metareg: Towards domain generalization using metaareguularization. Advances in neural information processing systems, 31, 2018.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedliggs of the European conference on computer vision (ECCV), pp. 456 456-473, 2018.
Erdem Biyik, Kenneth Wang, Nima Anari, and Dorsa Sadigh. Batch active learning using determinantal point processes. Xr preprint arXiv:1906 8rX1v:1906.07975,, 2019.
Elisa Celis, Vijay Keswani, Damian Straszak, Amit Deshpande, Tarun Kathuria, and Nisheeth Vishnoi. Fair and diverse dap-based data summarization. In International conference on machine learning, pp. 716 725. PMLR, 2018
Junbun Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Newral Information Processing Systems, 34:22405 22418, 2021.
Laming Chen, Guoxin Zhang, and Eric Zhou. Fast greedy map inference for determinantal point process to improve rocommendation diversity. Advances in Neural Information Processing Systems, 31, 2018.
Youngmin Cho and Lawrence Saul. Kernel methods for deep learning. Advances in neural information processing systems, 22, 2009.
Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased metric learning: On the utilization of multiple dalasets and web images for softening bias. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1657-1664, 2013.
Damien Garreau, Wittawat Jitkrittum, and Motonobu Kanagawa, Large sample analysis of the median heuristic. arXiu preprint 1rXim:1707.07260
707.077.000 2017.
Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, and David Balduzi. Domain generalization for object recognition with multi-task autoencoders. In Proceedings of the IEEE international conference computer vision, pp. 2551 2559, 2015.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test.. The journal of machine learning research, 13(1):723-775 17727777,777 2012.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Ishaan Guirajani and David Lopez-Paz. Paz. In search of lost domain generalization. axXiv:200.01434, 2020.
arXiv preprint
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine learning, 110(3):457-566, 2021.
Alex Kulesza and Ben Taskar. Structured determinantal point processes. Advances in neural information processing systems, 23, 2010.
Alex Kulesza and Ben Taskar. k-dpps: Fixed-size doterminantal point processes. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 1198-1200, 2011.
Alex Kulesza, Ben Taskar, et al. Determinantal point processes for machine learning. Foundations and Trends@ in Machine Learning, 5(2 3):123 286, 2012.
Claire Launay, Agnès Desolneux, and Bruno Galerne. Determinantal point processes for image processing. SIAM Journal on Imaging Sciences, 14(1):304-344 348, 2021.
Xi Leng, Xiaoying Tang, and Yatao Bian. Diversity boosted learning for domain generalization with large number of domains. Xie preprint orXiv:2207.13865 2022.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain gener- alization. In Proceedings of the IEFE international conference on computer vision, pp. 5542-5550) 2017.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference computer vision and pottern recognition, pp. 5400-5409, 2018a.
Ya Li, Ximmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European conference on computer vision (ECCV), pp. 624-639, 2018b.
Ying Lv, Bofeng Zhang, Xiaodong Yue, and Zhikang Xu. A novel ensemble strategy based on determinantal point processes for transfer learning. Mathemalics, 10(23):4409, 2022.
Divyat Mahajan, Shruti Tople, and Amit Sharma. Domain generalization using causal matching. In Inter- national conference on machine learning, pp. 7313-7324. PMLR, 2021.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant. feature representation. In International conference machine learning, pp. 10 18. PMLR, 2013.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1406 1415, 2019.
Laura Perez-Beltrachin and Mirella. Lapata. Multi-docummet summarization with determinantal point pro- cess attention. Journal of Artificial Intelligence Research, 71:371-399 2021.
Omid Poursaced, Tianxing Jiang, Harry Yang, Serge Belongie, and Ser-Nam Lim. Robustness and general- ization via generative adversarial training. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 15711-15720, 2021.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.0879. 2019.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 10 and 15-16, 2016, Proceedings, Part III 14, pp. 443 450. Springer, 2016.
15

---

Published in Transactions on Machine Learning Research (02/2026)
Lakpa D Tamang. Towards making effective machine learning decisions against out-of-distribution data. In Proceedings of the 33vd ACM International Conference on Information and Knowledge Management, pp. 5479 5482, 2024.
Zhaorui Tan, Xi Yang, and Kaizhu Huang, Rethinking multi-domain generalization with general learning objective. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2024.
Vladimir Vapnik. Statistical learning theory. John Wiley & Sons google schola, 2:831-842, 1998.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on compuler vision and pattern recognition, pp. 5018 5027, 2017.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najañ, Ioannis Mitliagkas, David Lopez- Pazez and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In International conference machine learning, pp.
PMLR, 2019.
Jindong Wang and Wang Lu. Deepdg: Deep domain generalization toolkit.
https://github. com/
j aandongaang/branfeelleee httpa////eeeeeeaee/////////ee/ee
2021.
Pengfei Wang, Zhaoxiang Zhang, Zhen Lei, and Lei Zhang. Sharpness-aware gradient matching for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3769 3778, 2023.
Mark Wilhelm, Ajith Ramanathan, Alexander Bonomo, Sagar Jain, Ed H Chi, and Jennifer Gillenwater. Practical diversified recommendations youtube with determinantal point processes. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp- 2165-2173, 2018.
Pengtao Xie, Ruslan Salakhutdinov, Luntian Mou, and Eric P Xing. Deep determinantal point process for large-scale multi-label classification. In Proceedings of the IEEE International Conference on Computer Vision, pp. 173--844- 2017.
Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14383-14392, 2021.
Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training, arXiv preprint arXiv:2001.00677 2020.
Nanyang Ye, Kaican Li, Haoyue Bai, Rumpeng Yu, Lanqing Hong, Fengwei Zhou, Zhenguo Li, and Jun Zhu. Ood-bench: Quantifying and understanding two dimensions of out-of-distribution generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1947-7958. 2022.
Sangdoo Yun, Dongyoon Han, Scong Joon Oh, Sanghyuk Chun, Junsuk Choc, and Youngjoon Yoo. Cutmix: Regularivation strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/ /CVF international conference on computer vision, pp. 6023 6032, 2019.
Hanlin Zhang, Yi-Fan Zhang, Weiyang Liu, Adrian Weller, Bernhard Schölkopf, and Eric P Xing. Towards principled disentanglement for domain generalization. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 8024-8034, 2022.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 1tf:::771000412,, 2017.
Chaoyang Zhou, Zengmao Wang, and Bo Du. Learning intrinsic invariance within intra-class for domain generalization. IEFE Transactions on Multimedia, 2025.
16

---

Published in Transactions on Machine Learning Research (02/2026)
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In Computer Vision -annn--CVVVVV- August 23 28, 2020, Proceedings, Part XVI 16, pp. 561 578. Springer, 2020.
2020: 16th European Conference, Glasgow, UK,
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. arXiv preprint arXiv:2104.0200 2021.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Puttern Analysis and Machine Intelligence, 45(4):4396 4415, 2022.
A Appendix
A.1
Training DSR: Algorithm
The high-level implementation concept of DSR is illustrated in Appendix 1 We have implemented DSR an auxiliary training objective that functions on intermediate feature representations within a multi-source DG framework. During each training iteration, mini-batcher are sampled from all source domains and aggregated. Subsequently, features extracted by the encoder are utilized compute predictive probabilities and their corresponding entropy. This entropy acts as measure of model uncertainty and employed modulate the feature representations, highlighting samples that more ambiguous or likely to reside near domain boundaries. A DPP kernel is then constructed over the modulated features, and log-determinant regularizer is applied to encourage diversity by penalizing redundant representations in the feature space. The resulting diversity regularization term is optimized in conjunction with the standard classification loss in an end-to-end manner, without necessitating explicit domain alignment or additional supervision. This approach implicitly fosters robust and diverse representations that generalize across unseen domains.
Algorithm Training DSR method
Require: {Di) model fa, parameters 0, learning rate regularization weight a, epochs T, iterations
Initialize
for {1,...... (1....,T) do
for k = (1, (1...., K} do
for each D,, i {1...., ,N) do
5: Sample a mini-batch B, {(xij. ) from D, b E (1...., B}
6: end for
7:
end for
8:
Concatenate all mini-batches: B B=
Generate features z h(B), Predict logits ÿ = g(z), and probabilitics p softmax(i)
Compute entropy = E. ((g
10:
11:
Perform Feature Modulation: 2 =
12:
Compute pairwise kernel matrix L (xpp->11 2 12), where 7 (median -2;12++))
13:
Compute regularization term: Raiv - log det +c)
Compute loss: Liotal N E,=, ((((x)) U) + (1-a) a:) Raiv
14:
15:
Update parameters: 0 - 0 VoLooall
16:
end for
A.2 How Informative is Domain Sample Size and its Diversity in DSR?
As summarized in Table significant imbalance in sample sizes is common across DG benchmark datasets. Under such conditions, conventional minibatch sampling from a pooled dataset tends to disproportionately favor domains with larger sample sizes. Since random sampling does not account for feature redundancy, minibatches are often dominated by statistically frequent but highly correlated samples. Such a repeated exposure majority-style features reinforce spurious correlations. For instance, if images from 'Photo"
17

---

Published in Transactions on Machine Learning Research (02/2026)
domain consistently contain specifie background that is absent in 'Sketch' domain, then standard sampling may encourage the model to rely on this background as shortcut for prediction. Because such features appear frequently in majority-domnnated minibatches, the model overfits to domain-specific cues rather than learning domain-invariant reprosentations, resulting in degraded performance on unseen target domains.
In contrast, DSR operates on the pooled sample set but explicitly promotes informative and diverse sample selection through entropy-guided DPP sampling. Instead of ensuring balanced sampling across domains, DSR selects samples that maximize joint diversity in the learned feature space, conditioned predictive uncertainty. This mechanism implicitly mitigates domain imbalance: samples from underrepresentedd or more challenging domains tend to exhibit higher uncertainty or contribute novel feature variations, making them more likely to be selected by Raiv. Importantly, domain labels themselves are not required by our method. From the perspective of DSR, the objective not to sample from specific domains, but to select samples that collectively provide maximal information for learning domain-invariant representations. The key factor is not the domain of origin, but whether the samples provide complementary, non-rodundant information. In this sense, domain diversily emerges byproduct of diversity in the learned representation space, rather than an enforced structural prior.
A.3 Cost of DPP Kernel Computation
We conducted further analysis of the computational overhead introduced by our DPP-based diversity regularization. Specifically, we assessed the additional time needed to compute the DPP kernel and its log-determinant. during each training iteration. The observed overheads are reported as in Table 4 It is important to note that all datasets, except DomainNet, consist of four domains, whereas DomainNet con- tains six domains. Consequently, the computational cost. scales moderately with the number of domains involved, as the DPP kernel computation depends on the domain-wise feature dimensionality and pairwise similarity evaluations. Despite this, the additional cost remains relatively minor indicating that the pro- posed regularization introduces only small and manageable overhead while enhancing model diversity and generalization.
Table 4: Additional computational cost introduced by DPP kernel and log- log-determinant computation.
Dataset
# Domains
# Classes
Time (ms)
PACS
4
96.1
VLCS
5
91.6
OfficeHome
4
65
91.3
Terralncognita
4
10
92.4
DomainNet
345
171.3
A.4 Effect of Increasing Batch Size
In our experiment, we explored a variety of batch sizes, specifically 4, 8, 16, 32, 64, and 128. As illustrated in Figure. g there is a noticeable trend where increasing the batch size correlates with enhanced model accuracy. This improvement can be attributed to the fact that, with more samples per batch during each epoch, DPP is more likely to sample wider range of diverse representations from the feature pool.
A.5 In-Domain Generalization
In Figure. 10 we present a comparison of In-Domain (ID) accuracy with one of the most effective basclines, SWAD (Cha et al. 2021), from our study. To ensure a fair comparison, we used the original implementation of SWAD to replicate their results. The results for SWAD + DSR were obtained by training their method with DSR's objective.
clear that, with this synergy DSR effective in maintaining the average ID accuracy across all DG benchmarks. This shows that DSR not only promotes feature diversity but also preserves the underlying representation of domain structure. Such balanced regularization ensures that
18

---

Published in Transactions on Machine Learning Research (02/2026)
Art
Cartoon
Photo
Sketch
100
Accuracy
90
80
70
ND
16
32
128
Batch Size
Figure 9: Batch size vs Accuracy comparison of PACS dataset on Resnet-50 architecture.
feature learning remains robust and generalizable, allowing the model to retain competitive ID and OOD performance tradeoffs.
I SWAD+DDR
97.81
92.98
100
93
1.79
86.79
87.2
84.11
83,87
Accuracy
90
80
65.91 .1.91
70
66.55
PACS
VLCS
OfficeHome
Terralncognita
DomainNet
Figure 10: Comparison of In-domain performance of SWAD+DSR and SWAD.
A.6 Comparison with Diversity Sampling Method
In this section, we compare DSR with DOMI, a closely related approach that utilizes a two-level diversity sampling strategy for domain generalization across multiple domains. However, the original DOMT plementation offers limited information on reproducibility, and its scalability to standard DG benchmarks remains unexplored. Their experiments were confined to the MNIST dataset, the RotatedMNIST benchmark, where digits are rotated by 0,15,30,45,60,77 degrees to simulate multi- domain conditions. Employing a one-domain-out validation strategy, we trained on five rotation domains and evaluated on the remaining one. As shown in Table. 5 DSR consistently achieves higher accuracy across all test domains, with an average accuracy 97.41%, significantly outperforming both DOMIMMD (87.70%) This illustrates that DSR's DPP-basod diversity sampling effectively captures
replicated the setup
and (89.0.0...996600))) broader and more representative range of feature variations, allowing to surpass the existing diversity sampling techniques this benchmark.
19

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: Comparison of DSR with DOMI baselines on the RotatedMNIST dataset.
Train Domains (degrees)
Test Domain (degree)
Accuracy (%)
0, 15, 30, 45, 60
75
98.47
0, 15, 30, 45, 75
60
95.09
0, 15, 30, 60, 75
45
98.71
0, 15, 45, 60, 75
30
98.46
0, 30, 45, 60, 75
15
98.44
15, 30, 45, 60, 75
95.30
Average (DSR) DOMLUM DOMICORAL
97.41
87.70
89.60
A.7 Performance Comparison Across Different Network Architectures
We assess the performance of DSR across three commonly used lightweight neural network architectures: VGG16, ResNet-18, and ResNet-50. The hyperparameters 7=0.05 and a=0.5 remain constant throughout all experiments. As shown in Figure. [1] ResNet-50 consistently achieves the highest accuracy across all datasets, which corresponds to its enhanced representational capacity due to its deeper layers. The figure also underscores that different domains within each dataset exhibit significant variations in accuracy, reflecting the inherent complexity and diversity visual characteristics across domains. For instance, in VLCS, Caltech yields the highest performance while LabelMe records the lowest, whereas in PACS, Picture achieves the best results and Cartoon the weakest. Despite these differences, the relative performance trends remain consistent across architectures, confirming that the DPP-based diversity regularization in DSR enhances robustness and generalization without requiring additional data generation augmentation.
20

---

Published in Transactions on Machine Learning Research (02/2026)
(a) PACS
(b) VLCS
Accuracy (%)
100
95
90
90
80
85
70
80
60
Art
Cartoon
Picture
Sketch
Caltech
LabelMe
SUN09
VOC2007
(c) OfficeHome
(d) TerraIncognita
Accuracy (%)
80
60
70
55
60
50
45
50
Art
Clipart
Product
Real world
Loc38
Loc43
Loe46
Loc100
(c)
DomainNet
Accuracy (%)
60
40
20
Clipart
Infograph
Painting
Quickdraw
Real Sketch
VGG16
ResNet18 5e
Figure 11: Accuracy comparison across five DG benchmarks using different neural network backbones.
21


Reviews:
[{"review_id": "OK3dn8S7UR", "reviewer": "Reviewer_Xe9f", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides a method for domain generalization based on regularization via diversity sampling, which roughly works by prioritizes sampling data that has more uncertain prediction values. The authors provide extensive experimental validation to show the improvements of their method.\n\nStrengths:\n- The experiments show that the method beats state of the art methods\n\nWeaknesses:\n- The presentation is a little hard to follow, e.g. some details are missing.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, for the most part. The main theoretical claim is supported by evidence.\n\nOne minor issue is that I don't think tsne is a valid way to compare inter-class separation on datasets (Figure 4). Even rerunning tsne on the same data (with a different seed) will lead to vastly different results.", "audience_yn": "Yes", "audience_explanation": "Domain generalization is an important method in machine learning, as it doesn't require gathering additional data on unseen domains for (re)training.", "requested_changes": "It would strengthen the paper if an appendix describing how DPP works, to make the work self-contained. Additionally, with just reading the paper in its current state, I do not think I could implement the algorithm. It may be helpful to add an algorithm environment and/or reformat the discussion to make it easier to understand a high-level implementation of the method.", "broader_impact_concerns": "N/A", "additional_comments": "Typos:\n- Page 2: In this paper, we propose... \"could\" should be replaced with \"to\"?\n- Section 3.8: 4 -> (4) as eqref\n- 1e-2 -> 0.01?\n\nMinor:\n- Section 3.1: $\\mathcal{F}$ should be defined to be some function class\n- Section 3.2: 2a -> Figure 2a, similarly 2b -> Figure 2b (and other instances where figures and tables are referenced)\n\nQuestions:\n- $\\mathcal{C}$ is not defined in Eq. 3. Should we think of this as (average/max) TV/Wasserstein/KL bound?\n- $\\mathcal{R}_{div}$ is also not defined. It should be stated that this can be chosen by the practitioners, and also give some suggestions on how to choose it (e.g. in appendix).\n- What is z in Eq (5)? All I can find is based on Figure 3(a)\n- $\\alpha = 0.5$ seems to put quite a large fraction of the weight on the regularizer. I'm not sure how reasonable this is in practice, as one would expect regularization to be small (like between 0.1 and 0.2). Can the authors discuss good choices for $\\alpha$?", "extra_scores": {}, "extra_text": {}}, {"review_id": "g8v1QNKS6R", "reviewer": "Reviewer_ws11", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents DSR, a method for Domain Generalization designed to improve performance on unseen domains by addressing the limited feature diversity often observed in traditional augmentation techniques. The approach employs entropy values to assess input prediction uncertainty, using these scores to guide a Determinantal Point Process that selects diverse data subsets near the decision boundary. By integrating this sampling process as a regularization term within the ERM framework, the method seeks to learn domain-agnostic features without explicit data synthesis. Empirical evaluations across benchmarks such as PACS, VLCS, and DomainNet show that DSR yields improvements over baselines. \n\nStrengths:\n1. The method combines entropy-aware sampling and pairwise feature distances to encourage subset diversity, is lightweight to implement, and achieves leading or competitive results on several benchmarks. \n2. DSR is formulated as an architecture-agnostic, plug-and-play regularizer, making it straightforward to integrate into a variety of existing DG frameworks and backbone models.\n\nWeaknesses:\n1. Some design choices would benefit from stronger, more principled motivation. In particular, entropy-based feature scaling adopts heuristic priors that may limit interpretability and raise concerns about sensitivity to these assumptions.\n2. On benchmarks such as OfficeHome and TerraIncognita, DSR underperforms SWAD by a notable margin. This raises the question of whether certain inductive priors in DSR may be better aligned with specific datasets, and whether a more prior-free or adaptive formulation could improve robustness across a broader set of domains.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The evidence includes DSR consistently outperforming ERM and MixUp, demonstrating model agnosticity by boosting the performance of existing methods, and confirming the utility of its key components through targeted ablation studies.", "audience_yn": "Yes", "audience_explanation": "The topic of Domain Generalization is a classic and critical problem in machine learning. It addresses the challenge of building models that remain robust when faced with changes in data distribution or domain shifts, which is essential for successful deployment in diverse real-world application scenarios", "requested_changes": "1. It would be valuable to provide an analytical investigation into why the proposed DSR method underperforms compared to SWAD on the OfficeHome and DomainNet benchmarks. One potentially insightful direction would be to visualize and compare the feature distributions of the learned representations from DSR and SWAD, which may help reveal why DSR’s feature-space geometric constraints could be suboptimal for these datasets.\n\n2. Is the method limited to classification tasks, or can it be extended to more complex settings, such as segmentation and object detection?", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}, {"review_id": "TIOfNYYLWs", "reviewer": "Reviewer_Qhob", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Diversity Sampling Regularization (DSR), a method to improve domain generalization (DG) by promoting diversity in feature representations. DSR leverages predictive entropy to modulate features and applies Determinantal Point Processes (DPP) to sample diverse subsets during training. The approach is model-agnostic and does not rely on data augmentation. Extensive experiments on standard DG benchmarks show that DSR consistently outperforms ERM and several strong baselines, achieving competitive or state-of-the-art results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Accuracy: Results are reported as mean ± standard error over multiple runs on five standard DG benchmarks, against a wide range of strong baselines and SOTA methods.\n\nConvincing Evidence: The consistent performance gain across all datasets is compelling. Critical ablation studies prove the value of key components (entropy modulation, adaptive γ). The demonstration that DSR boosts the performance of other methods (model-agnosticity) strongly supports its utility as a general regulariz", "audience_yn": "Yes", "audience_explanation": "The paper tackles the critical problem of domain generalization, which is of central importance to the machine learning community. Its proposed method is both practical (as a simple, model-agnostic regularizer) and theoretically interesting (leveraging DPPs for diversity). The compelling results across standard benchmarks will attract a broad audience, including researchers focused on robustness, theory, and real-world applications.", "requested_changes": "How does DSR scale with the number of domains or classes? Is there a risk of performance saturation as diversity increases?\n\nCould DSR be applied to non-visual domains (e.g., NLP or tabular data)?\n\nHave you considered combining DSR with recent sharpness-aware or flatness-based methods (e.g., SWAD) to see if they are complementary?\n\nThe entropy modulation seems crucial. Did you experiment with other uncertainty measures (e.g., confidence scores, MC dropout)?", "broader_impact_concerns": "This work has predominantly positive broader impacts by contributing to more robust and reliable ML systems that can perform well under domain shift, a key challenge for real-world deployment. Potential negatives, such as dual-use or the method's inability to fully address social biases, are common to the field and not uniquely raised by this paper. The authors' approach is resource-conscious compared to augmentation-heavy alternatives, and they transparently report computational costs. The net impact is positive.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)