Minimal Semantic Sufficiency Meets Unsupervised Domain Generalization
Authors: Tan Pan
Kaiyu Guo
Dongli Xu
Zhaorui Tan
Chen Jiang
Deshu Chen
Xin Guo
Brian C. Lovell
LIMEI HAN
Yuan Cheng
Mahsa Baktashmotlagh
Venue: NeurIPS 2025 poster ()
Keywords: minimal sufficient representations
semantic learning
unsupervised domain generalization
Primary area: general_machine_learning

Abstract: The generalization ability of deep learning has been extensively studied in supervised settings, yet it remains less explored in unsupervised scenarios. Recently, the Unsupervised Domain Generalization (UDG) task has been proposed to enhance the generalization of models trained with prevalent unsupervised learning techniques, such as Self-Supervised Learning (SSL). UDG confronts the challenge of distinguishing semantics from variations without category labels. Although some recent methods have employed domain labels to tackle this issue, such domain labels are often unavailable in real-world contexts. In this paper, we address these limitations by formalizing UDG as the task of learning a Minimal Sufficient Semantic Representation: a representation that (i) preserves all semantic information shared across augmented views (sufficiency), and (ii) maximally removes information irrelevant to semantics (minimality). We theoretically ground these objectives from the perspective of information theory, demonstrating that optimizing representations to achieve sufficiency and minimality directly reduces out-of-distribution risk. Practically, we implement this optimization through Minimal-Sufficient UDG (MS-UDG), a learnable model by integrating (a) an InfoNCE-based objective to achieve sufficiency; (b) two complementary components to promote minimality: a novel semantic-variation disentanglement loss and a reconstruction-based mechanism for capturing adequate variation. Empirically, MS-UDG sets a new state-of-the-art on popular unsupervised domain-generalization benchmarks, consistently outperforming existing SSL and UDG methods, without category or domain labels during representation learning.

Minimal Semantic Sufficiency Meets Unsupervised Domain Generalization
Tan Pan 1.2g Kaiyu Guo 3:2; Dongli Xu², Zhaorui Tan', Chen Jiang 1.2 Deshu Chen
Xin Guo 12 Brian C. Lovell Limei Han 1. Yuan Cheng 12g Mahsa Baktashmotlaghi
A1S, Fudan University Shanghai Academy of Artificial Intelligence for Science The University Queensland
pant220m fudan. edu. en, jiangchen@sais wtwpcheenaais.or.ccrr.mm ong.cn, cheng_yuan edu
Abstract
The generalization ability of deep learning has been extensively studied in super- vised settings, yet it remains less explored in unsupervised scenarios. Recently, the Unsupervised Domain Generalization (UDG) task has been proposed to enhance the generalization of models trained with prevalent unsupervised learning techniques, such Self-Supervised Learning (SSL). UDG confronts the challenge of distin- guishing semantics from variations without category labels. Although some recent methods have employed domain labels tackle this issue, such domain labels are often unavailable in real-world contexts. In this paper, we address these limita- tions by formalizing UDG the task of learning Minimal Sufficient Semantic Representation: a representation that (i) preserves all semantic information shared across augmented views (sufficiency), and (ii) maximally removes information irrelevant to semantics (minimality). We theoretically ground these objectives from the perspective of information theory, demonstrating that optimizing representa- tions to achieve sufficiency and minimality directly reduces out-of -iit-----ittriutiinn risk. Practically, we implement this optimization through Minimal-Sufficien UDG (MS- UDG), learnable model by integrating (a) an InfoNCE-basee objective to achieve sufficiency; (b) two complementary components to promote minimality: a novel semantic-variation disentanglement loss and a reconstruction based mechanism for capturing adequate variation. Empirically, MS-UDG sets new popular unsupervised ramain-generalization benchmarks, consistently outper- forming existing SSL and UDG methods, without category or domain labels during representation learning.
Introduction
Generalization ability is a critical yet challenging problem deep learning. This challenge leads to the emergence of the task of Domain Generalization (DG) [55 53 42), which focuses generalizing
deep learning models to unseen distributions. Many works have been proposed on this topic under the supervised settings [4 25 I. However, for the scenario of unsupervised learning, e.g., self-supervised
learning (SSL), which is more prevalent and practical in the real world [3 41], limited works have
been proposed to address the generalization ability of the model. Recently, the task of Unsupervised Domain Generalization (UDG) (56) has been proposed to handle the generalization issue in self- supervised learning (SSL), and some methods have been proposed to improve the generalization
ability by learning or disentangling the fomain-invariann representations [54 56 51]
Equal contribution. This research was conducted during internship the Shanghai Academy Artificial Intelligence for Science. Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

29 often focus on disentangling semantics from variation factors (55], which becomes more challenging in UDG due the absence of category labels (e.g., cat, dog, and airplane). (ii) In order to disentangle meaningful semantics from domain-specific variations, existing UDG techniques predominantly depend on domain labels (e.g., painting, scratch, and clipart) that are inaccessible or expensive in real-world scenarios. Motivated by these challenges, our work addresses a critical research question: How can semantic information be effectively disentangled in UDG without relying domain labels?
However, two key challenges remain. (i) Traditional DG methods [3]29]]]]]
To address this research question, we explore learning semanties from an information therrioiiateeetticall perspective 2 48 Specifically, we argue that the shared information learned by contrastive learning adequate represent semantics [4]. However, may also encompass semantically irrelevant but confounding factors, such as shared style and texture, which can lead to suboptimal semantic representations. This issue might be exacerbated when unlabeled data exhibit significant semantic and covariate distribution shifts. Therefore, eliminating semantically irrelevant information is crucial to refining sufficient semantic representations for UDG.
Based on the above assumption, our intuition is to separate the semantic and variation information by minimizing semantically irrelevant information within sufficient representations. To achieve this, we formulate this non-trivial challenge as a constrained optimization problem, aiming to reduce the dependency between the sufficient representation and the input conditioned on the semantic space. Concretely, the optimization problem can be decomposed into two objectives: (1) minimizing the mutual information between the semantic and variation representations, and (2) maximizing the mutual information between the variations and the inputs conditioned on the semantic space. Objective ensures that semantics and variations are disentangled, promoting their independence and non-redundancy. Objective 2 enables the semantic representation to retain minimal semantically irrelevant information. By optimizing these two objectives, the system is able to learn approximately optimal semantics that is both disentangled and minimally sufficient.
To achieve these objectives, we introduce a novel algorithm, namely Minimal Sufficient UDG (MS-UDG). The model first employs traditional contrastive learning to ensure a sufficient semantic representation. Then, two modules, i.e., Information Disentanglement Module (IDM) and Semantic Representation Optimizing Module (SROM), are employed to disentangle the optimal semantic representation from the sufficient semantic representation. IDM separates the semantic and varia- tion representations from the sufficient semantic representation. SROM applies a mixed InfoNCE constraint to minimize the mutual information between semantics and variations. Simultaneously. maximizes the mutual information between variations and the inputs through reconstruction, thereby fulfilling our learning objectives.
Contributions. (1) To the best of our knowledge, we propose the first theoretical optimal semantic representation for UDG from an SSL perspectiv... Subsequently. we introduce a tractable estimation method for disentangling the optimal semantic representation through two optimization objectives. This approach offers a novel view for enhancing the generalization capabilities of SSL models to previously unseen domains.
(2) Based on dual optimization objectives, we introduce MS-UDG, algorithm that effectively removes semantically irrelevant information while preserving representative semantics, without relying on domain labels. We also provide theoretical analysis support the rationale and applicability of our framework.
(3) Experimental results popular benchmark UDG datasets demonstrate that our method achieves superior performance in downstream tasks compared to existing approaches.
2 Related Work
Unsupervised Domain Generalization. Unsupervised Domain Generalization (UDG) has been proposed handle the problem of domain generalization in unsupervised learning (56). Similar to supervised domain generalization [30 28 35 15), methods in UDG mostly rely on representation learning and data augmentation to conduct Nomain-invarian or domain-specific self-supervised learning (SSL). For data augmentation, FDA 550 and BSS [39 introduce Fourier-based methods to standardize the style of images to plug in native contrastive- generalizes the model by aligning the image unified style on the feature level. For representation
SSL. In addition, BrAD[[6]

---

learning, contrastive- based methods construct negative samples [56] or suppress intra-domaan domain connec- tivity by domain labels [27] Also, MAE [7] is utilized in UDG. DiMAE [52] and CycleMAE [5]] transform the original image into its style-mixed view and then decode different domain styles by several domain decoders. DisMAE [54] introduces a semantic-encode and variation-encom
to disentangle semantic attributes. Moreover, some methods use UDG methods to improve the gener- alization on specific tasks, such as face anti-spoofing (26). Unlike these methods, we theoretically build our method from the contrastive learning perspective.
a successful paradigm in SSL. Contrastive- based SSL methods aim to learn the shared information 32 between multi-view data. Recent methods [46] [44 [24 discuss relevnel and redundant representations of SSL based on information theory. Although there has been extensive exploration in the SSL field, the generalization of SSL to OOD data still requires further discussion.
Contrastive Learning. Contrastive learning 33
5
34
Disentangled Representation Learning. Disentangled representation learning is a promising approach for DG, aiming to separate semantics-relevant and irrelevant factors in data 49 56 21 577
I. Class labels help ensure the model learns class-specific features consistently across domains, avoiding reliance on domain-specife artifacts. Without class labels, the bisentanglement-baeeee methods in UDG often focus on domain style transfer [53] (55) to separate domain variations by domain labels. However, there is still a lack of theoretical support for disentangling ammain-invariant representations of SSL to obtain optimal semantics.
3 Preliminary and Motivation
In Sec. 3.1 we first introduce the setting of UDG, which aims to discard the covariate information in the representations (54). To address our research question, in Sec. 3.2, we propose new concept of minimal sufficient semantic representation, treating semantic information as a proxy for the unknown downstream task. Based this concept, we theoretically formulate two learning objectives optimized to disentangle representation into pure semantic information. Finally, based on these objectives, we present our algorithm and provide theoretical analysis in Sec. 4
3.1 Unsupervised Domain Generalization
Notation. Let X' be the input space and y be the label space. Considering supervised dataset D (XD, YD) with ND samples, where XD C X and YD C V, the distribution of is P(x.,F.)> define the distribution of SSL data, downstream supervised data, and test data as respectively. In SSL, for each input 22), we have P(x,..:)) P(Xpp) and P(Neeut.Yepd Xx Xaug is augmented from X The extracted features I1 Xost and 22 Xaug- where
Then
21 = h(()) and 2/ = h(x2), where h is the encoder and 21, 2g E Z, Z is the representation space. The parameterized latent space for semantic-relevant and -irrelevant, variate, factors are S and V, respectively. The intuitive relation is 2 = S ( V.
Problem setting, Following the setting of UDG (56), the test distribution should be kept unknown at any stage before inference. Then, we have the condition Support ((X::e:)))n(SSppppo (P (Patt(((Xaaaaa (((est))
0. In addition, to avoid the semantic shift in the downstream task, we need Support( (Prporr((((Yraaaa P(Ysup)) = (ppport((P(Yesddl))
Support(
The target of UDG is to obtain the optimized feature extractor h² which has the minimum risk on the test distribution h arg min, R Yeess) (h)
Motivation. Considering that UDG methods are built upon existing SSL methods with unknown downstream task targets T', we examine the goal of UDG from a contrastive learning (CL) perspective. Given an input Iı and its augmented counterpart T2. we adhere to the multi-view assumption [44), treating 21 and 22 as two corresponding views of the same data point. Under this assumption, the objective of CL is to extract task-relevane representations (44 44 posits compression gap [21 22 T)) under the task T. This implies that while optimizing for (1));)))) task-irrelevant information may inadvertently be retained in the learned representations. In datasets used for SSL, especially those from multiple domains that exhibit significant semantic and covariate distribution shifts, this compression gap cannot be overlooked if the aim is to achieve task-specific or semantic- specific representations. Based on this insight, we propose that semanties and variations can be separated by explicitly modeling and minimizing fask-irreeevant information.
3

---

3.2 Minimal Sufficient Semantic Representation
In this section, we first introduce the definition of sufficient representation and minimal sufficient representation [46] which aims to learn task-relevant representation optimally. In the supervised setting, with the given image x and label y. the information bottleneck [ can be applied to achieve optimal representations by minimizing mutual information [xx, z) and maximizing [zz y), where the representation of z. In self-supervised learning, the augmented image 42 plays a similar role as the supervised label 1 44). The definition is proposed as follows.
Definition 3.1 (Minimal Sufficient Representation in Contrastive Learning 46 ) Let ann and sprin
be the sufficient representation and minimal sufficient representation of æ in self-supervised learning, respectively. 22 the augmented data. Then the following conditions should be satisfied.
T( x2) (221; 21 arg min I (= ;.1)
This definition introduce the minimal sufficient representation with two stage: First, we define that 2q"d is sufficient if and only if ((2"";;2)) = [<11; which aligns with the target of contrastive loss maximize [=21 22) 32 Note that, since 2q"d is defined keep all the mutual information between In and Z2 instead of any prior labels, the loss or change of augmented information in I2 can also affect 2 Second, with 2/nh we define the minimal sufficient representation 2 by searching minimum / ;x1)) Symmetrically, we can also define the minimal sufficient representation of 22- More details of minimal sufficient representation can be found in Appendix.
However, the minimal sufficient representation in Definition 3.1 may still include task-irrelevant information, particularly in UDG datasets with significant covariate shifts.
;T) Proposition 3.2. [(2minn [22min T) + I(2min; 24 (T) 2 [(2 T), where i E {1,2), T is the downstream task.
The proposition 3.2 indicates that if I [24; 21 2m (T) is large, 2 may not be the optimal minimal sufficient feature. In practice, [ar;; 2yvón (T) can be affected by many factors in the shared informa- tion, including style, texture, and other factors between d2 and æj. In the multi-domain situation as the setting of UDG, the [xx;;; squin IT) will be large, which leads to performance degradation since there can be significant distribution shifts in all task-irrelevant distributions.
The downstream task remains unseen during SSL. However, in most cases, it should be related to the semantic information in Xuu. In this case, we can consider [[x1; T2; T) [x11 x2; S). Then, we argue that the sufficient representation should contain all semantic information between 21 and zz-
Definition 3.3 (Sufficient Semantic Representation in Contrastive Learning). z/"// is the disentangled
sufficient representation of xı if and only if I(=1">; ;22; S) = [(x1; :2: 5). S C S is the semantic information in Xsst
Definition 3.3 indicates that a representation containing all the shared semantic information between In and 22 is sufficient to capture the semantics of xn and Tz. However, the sufficient representation may not be the optimal representation as [[>*"f; () 2 O, where I[=*Mf, x)) > 0.
From the relation = S + V. every z=u can be decomposed as === = (s, v). where s is semantic- relevant, which is also sufficient semantic representation, and is semantic-irrelevant. That means S(;; = By this disentanglement, if (s;; S) can be minimized, then s the optimal sufficient representation expect. From this perspective, we propose the definition of the optimal sufficient semantic representation in Definition
Definition 3.4 (Minimal Sufficient Semantic Representation in Contrastive Learning). Let 2q be the minimal sufficient semantic representation of 11- Then, =zmin A arg min yas I(=;" ;2 Dy S), S C S is the semantic information in X,at, 22 is the augmented input.
Definition 3.4 suggests that the minimal sufficient semantic representation should contain the least semantic-irrelevant information. Furthermore, we can assume [ I2, SS) = 0. As mentioned above, we have the decomposition of sufficient semantic representations Vz*4f, =u = (=min v). Then, we theoretically illustrate how to effectively disentangle the z min which answers the research question we proposed.

---

where 7 is a temperature papee--prrmeter and the K is the number of negative samples. 2 swls- represents a negative representation from negative sample. Fourier-based augmentations (FA) (50), whose efficiency has been experimentally proved in UDG methods [38 (5) FA randomly alters the amplitudes one image (anchor) with the amplitudes from Proposition 3.5. [(21"/ 111 () = [21% 22,, 22 [2
;
Proposition 3.5 indicates that the semantic-irrlleaat information in 2] is derived from æ1, and we can obtain 2!_ as 21 & arg migging [ (),
Proposition 3.6. (s, u) the sufficient semantic representation of a and is semantic-relevant [(s; + [2) [(;;)(s)
representation. Then [s;;
we want to optimize as the minimal sufficient semantic representation, considering the Markov chain 5, we only optimize and v. So, propose to minimize [(5; v) and maximize SS to disentangle the optimal semantic representation in UDG.
4 Algorithm and Theory
4.1
Algorithm
Based on the objectives in Proposition 3.6 (1) minimize [(s; u) and (2) maximize [(u; x S),
we propose that objective enhances the independence and non-redundancy between semantic representation and variations, while objective 2 limits the inclusion of semantic-irelevant information semantic representation.
To achieve objectives, we propose the method Minimal Sufficient UDG (MS-UDG) as shown in Fig. After learning sufficient representations by contrastive learning loss InfoNCE [18), the model comprises two components: (1) Information Disentanglement Module (IDM), which aims disentangle semantic representation and variation v. (2) Semantic Representation Optimizing Module (SROM), which optimizes s towards minimal sufficient semantic representation.
S1
edf-erro:
attcotionn
Projectlan
X1
11
objective frein
MASK
Encuder
hoad
contrastive learning lesff
Fouff
Cynne
&
Lyax
F2z
abjecttve
x2
Z
perion hand
-MASK Encoder
Deceder
selforrns
stilantio
Modules
and
share weights
(4)
Figure 1: Our method's pipeline, where and 22 denote the input and its augmented counterpart. represents the sufficient representation, while s and v correspond the semantic representation and variation. (a) An illustration of our two objectives. First, we employ L to capture sufficient semanties. Subsequently, Lmin and Lmax are introduced to learn a minimal sufficient representation. We only describe L.in and Lmaz for æ1 in detail. The same procedure applies to (b) A detailed overview our network and the applied constraints.
Firstly, the InfoNCE loss is applied to constrain guf which aims to learn sufficient shared informa- tion between 2 and Ø2.
Lauf - llo exp(2 suff ousS /T) oull
22
(1)
Σk exp(211
2/ su11-/7)²
construct positive pairs by
another image (target). For our training, the target image is randomly selected from the dataset.
5

---

Then, based on the sufficient semantic representation, we introduce two components.
IDM. Following intuitive relation 2 = S V. assume that the representation can be disentangled into a semantic-relevant representation 8 and a semantic-irelevant representation v. After shared encoder extracts image features, the features processed through projection head obtain high-dimensional representation. Subsequently, two MLP modules are applied to 2 to separate semantic information from variation.
SROM. Corresponding to Proposition 3.6 we introduce a semantics and variation mixed InfoNCE Lmin loss minimize [s;; v) and translated reconstruction loss Lmax to maximize [(;; S).
exp(s1
Cmin log K
K
(2)
Σ exp(s1 $ )
Σ k=0
exp(s1.ug +
k=0
where is considered negative representation. Lmin is modified from InfoNCE loss, which not only maximizes the [s11; 82) but also minimizes e1)
Proposition 4.1. Lmin 2 ([s1;1,) ) 82)
According to Proposition 4.1 minimizing Lmin indicates decreasing I(s1; 01) and increasing [(s1: 82) synchronously. Minimizing Lmin satisfies target to minimize I(s; v). Meanwhile, same as InfoNCE, Lmin also keeps $1 as a sufficient semantic representation by maximizing [(s1: (2):
Lmaz D($1. 02) 22112 [ 01) 2111
2
Here, D the decoder to reconstruct images. Without loss of generality, we consider the object D(51, 02) to illustrate how Lmaz maximize (v in the following.
Proposition 4.2. [(22, $1: (21:22) I [ 22)))
Proposition 4.2 indicates that / (v2, 22) is related to the lower-bound of ((12; From [D((s,, U2)
I2 Lmaz aims to maximize the mutual information [ $1;22)) (1;251)22) Thus, Lmaz can help increase the mutual information I(1; 222 () by raising its lower bound
As discussed above, Lmin and Cmar meet our learning targets described in Section Therefore, the final loss
L == Lonfs + Lrmnn + Lman-
4.2 Theoretical Analysis of Generalization Upper Bound
As discussed in previous works 4 the generalization error bound based the multi-distribution learning theory BI the upper bound of the target risk can be summarized into three parts 4 source empirical risk, distribution discrepancy between source and target distributions, and confidence bound. Briefly, it can be presented as RP(xxsatXYeas)
(h) & X Y (h) + A = Csup + A, where Csup is the downstream error, A is the summation of domain discrepancy and confidence bound in downstream 4 However, on the error in the source domain to minimize the generalization bound in the downstream. In the following, demonstrate the relation between the minimal sufficient semantic representation and the downstream error.
not access the state of A in the SSL stage. Thus, focus
Theorem 4.3. For representation suf and zman, their Bayes error rates are min and cat re- spectively. min has the minimum upper bound compared with all minimal sufficient semantic representations. Specifically, given the downstream task T, we have
e suf < 1 - --xeep-((HH))) + I (2 (/25//// 2 []]]
mia 1 - 111-expl- eX) [(2 (2)min []]
Bayes eтrоr rate [12] is the minimum achievable error for any pngresentation-leamnee classifier. Following the previous work 46 we consider the downstream error. Theorem 4.3 indicates
that downstream error of sufficient semantic representation 2 is upper bounded by / [(2 f0 ;z1
According to Definition 3.4 the Bayes error rate of mmn has the minimum upper bound. Thus, mim provides the lowest upper bound for the risk R.Pp (h), which helps to improve the generalization ability of the unsupervised models.

---

Table 1: Performances of UDG and SSL models PACS dataset. All models trained on selected domains and tested on the left domains, which process is repeated for all domains. "avg." represents accuracc. We report the accuracy for every domain and the average accuracy for all domains. Best and second best are highlighted.
Label Fraction:
Label Fraction: 5%
Methods
Target domain
Target domain
photo
cartoon
sketch
avg.
photo
sketch
avg-
ERM
10.90
11.21
14.33
18.83
13.82
14.15
18.67
13.37
18.34
16.
MoCo
22.97
15.58
23.65
25.27
21.87
37.39
25.57
28.11
31.16
30.56
SimCLR V2
30.94
17.43
30.16
25.20
25.93
54.67
35.92
35.31
36.84
40.68
BYOL
11.20
14.53
16.21
10.01
12.99
26.55
17.79
21.87
19.65
21.46
AdCo
26.13
17.11
22.96
23.37
22.39
37.65
28.21
28.52
30.35
31.18
MAE
30.72
23.54
20.78
24.52
24.89
32.69
24.61
27.35
30.44
28.77
DARLING
27.78
19.82
27.51
29.54
26.16
44.61
39.25
36.41
36.53
39.20
DIMAE
48,86
31.73
25.83
32.50
34.73
50.00
41.25
34.40
38.00
40.91
BrAD
61.81
33.57
43.47
36.37
43.80
65.22
41.35
50.88
50.68
52.03
CycleMAE
52,63
36.25
35.53
34.85
39.82
63.24
39.96
42.15
36.35
45.43
BSS/SimCLR
43.31
38.96
48.61
48.76
44.91
58.16
46.37
55.69
65.63
56.40
MS-UDG
42.74
47.13
47.84
43.33
45.26
74.03
61.78
54.72
61.53
63.02
Label Fraction:
10%
1 Label Fraction:
100%
Methods
Target Domain
Target Domain
photo
art
cartoon
sketch
avg.
photo
aFt
cartoon
sketch
avg.
ERM
16.27
16.62
18.40
12.01
15.82
43.29
24.27
32.62
20.84
30.26
MoCo V2
44.19
25.85
35.53
24.97
32.64
59.86
28.58
48.89
34.79
43.03
SimCLR V2
54.65
37.65
46.00
28.25
41.64
67.45
43.6
54.48
34.73
50.06
BYOL
27.01
25.94
20.98
19.69
23.40
41.42
23.73
30.02
18.78
28.49
AdCo
46.51
30.31
31.45
22.96
32.81
58.59
29.81
50.19
30.45
42.26
MAE
35.89
25.59
33.28
32.39
31.79
36.84
25.24
32.25
34.45
32.20
DARLING
53.37
39.91
46.41
30.17
42.46
68,
41.53
56.89
37.51
51.20
DIMAE
77.87
59.77
57.72
39.25
78.99
63.23
59.44
55.89
64.39
BrAD
72.17
44.20
50.01
55.66
55.51
CycleMAE
85.94
67.93
59.34
62.87
90.72
75.43
69.33
50.24
71.41
RSS/SimCL.
63.29
51.37
59.43
60.04
79.50
62.73
65.67
73.02
70.23
MS-UDG
74.10
61.90
63.73
73.66
68.35
84,80
63.74
71.73
71.27
72.89
5
Experiments
5.1 Experimental Setup
Datasets. Following previous UDG work (56), PACS 23 and DomainNet (35]
evaluated for benchmarking UDG methods. Meanwhile, we also evaluate our methods on OfficeHome, Office31, and VLCS whose results and details can be found in the Appendix.
DomainNet has 345 categories and different domains (Clipart, Infograph, Quickdraw, Painting, Real, and Sketch). PACS has 9,991 images with 7 classes from 4 domains: Art, Cartoons, Photos, and Sketches. For DomainNet, the six domains are split into two sets: 1. Clipart, Infograph, and Quickdraw; 2. Painting, Real, and Sketch previous work did. 20 classes are selected both unlabeled and labeled data. We use one set for seen domains and the other one for unseen domains. We report the micro-accuracy and macro-ace -ccurrcyy six domains. For PACS, three domains selected for training, and the remaining domain is used for evaluation.
Experimental Protocol. We adopt the all-correlated setting, as proposed in DARLING [56 and DisMAE [54]. The overall process is divided into three main steps. First, a pre- pre-trained model is obtained using UDG methods the unlabeled data from the source domain. Second, this pre-trained model is fine-tuned on varying proportions labeled data from the source domain, with either the classifier the entire backbone network being adjusted. Finally, the model is tested the unseen domain. Following the benchmark [56] and general setting of SSL (i.e., linear probing and fine-tuning), adopt linear probing for 1% and 5% label fractions and fine-tuning for 10% and 100% label fractions in downstream tasks.
To ensure fair comparison with previous SSL. methods, including MoCo V2 7. SimCLR V2 5
BYOL (6, AdCo [9]. and MAE (17), as well as UDG methods such DARLING (56) DiMAE [2]] BIAD 16
BSS 39 and CycleMAE (51), the SSL models initialized using ImageNet pre-

---

Table 2: Performances of UDG and SSL models on the DomainNet subset, which contains six domains. The models are trained on source domains and evaluated on target domains. "Overall" and "avg." refer to micro-accuracy and macro-accuracy. Best and second best are highlighted.
Label Fraction:
Source
paintingLrreal Uskethh
Target
lquichdraw
painting Caantingguuealo Ureat
Isketch Useetskkethh
overall
avg.
BYOL
6,21
3,48
4.27
5.00
8.47
4.42
5.61
5.31
MoCo V2
18.85
10.57
6.32
11.38
14.97
15.28
12.12
12.9
AdCo
16.16
12.26
5.65
11.13
16.53
17.19
12.47
13.15
SimCLR V2
23.51
15.42
5.29
20.25
17.84
18.85
15,46
16.86
MAE
22.38
12.62
10.50
17.86
24.57
19.33
17.57
17.88
DARLING
18.53
10.62
12.65
14.45
21.68
21.30
16.56
16.54
DiMAE
26.52
15.47
15.47
20.18
30.77
20.03
21.85
21.41
BrAD
47.26
16.89
23.74
20.03
25.08
31.67
25.85
27.45
CycleMAE
37.54
17.13
22.85
30.38
22.31
24.08
24.71
BSS/SimCLR
61.94
27.40
31.55
41.49
32.27
34.82
MS-UDG
55.88
20.03
18.65
32.15
39.81
42.89
32.45
34.92
Label Fraction: 5%
Source
paintingUreal
clipart.infogrup Iunocruup..quicchraww paintingLreeal Lsketchh
Target
clipart infforuph lquichdraw
overull
avg.
BYOL
9.60
5.09
6.02
9.78
10.73
3.97
7.83
7.53
MoCo V2
28.13
13.79
9.67
20.8
24.91
21.44
18.99
19.79
AdCo
30.77
18.65
7.75
19.97
24.31
24.19
19.42
20.94
SimCLR V2
34.03
10.88
21.35
24.34
27.46
20.89
22.54
MAE
32.6
15.28
13.43
24.55
30.43
26.07
22.88
23.73
DARLING
39.32
21.09
30.51
28.49
23.31
24.83
DiMAE
42.31
18.87
15.00
27.02
39.92
26.50
27.85
28.27
BrAD
37.54
18.01
17.13
22.85
30.38
22.31
24.08
24.71
CycleMAE
55.14
20.87
19.62
27.64
40.24
28.71
30.80
32.04
BSS/SimCLR
71.21
20.93
32.42
36.68
41.49
52.75
39.73
42.58
MS-UDG
71.02
28.21
28.51
40.07
47.05
49.51
41,33
44.06
Label Fraction: 10%
Source
paintingUreall.e Usketch
Linforruin hwrp./onogrrrhhh.lhhhhaar
Targel
clipart Linforrrif sinforraphhquicchhaa
paintingU Creangurrelllii Leketttsehhhh
overall
avg.
BYOL
14.55
9.50
10.38
4.45
8.69
8.92
MoCo V2
32.46
18.54
25.35
29.91
23.71
21.87
23.00
AdCo
32.25
17.96
11.56
23.35
29.98
27.57
22.79
23.78
SimCLR V2
37.11
19.87
12.33
24.01
30.17
31.58
24.28
25.84
MAE
51.86
24.81
23.94
41.24
54.68
39.41
38.85
39.32
DARLING
35.15
20.88
15.69
25.90
33.29
30.77
26.09
26.95
DIMAE
70.78
38.06
27.39
50.73
64.89
55.41
49.49
51.21
BIAD
68.27
26.60
34.03
31.08
38.48
48.17
38.74
41.10
CycleMAE
74.87
38.42
28.32
52.81
67.13
56.37
50.78
52,98
BSS/SimCI.R
71.95
21.27
33.47
39.49
44.67
55.42
41.57
44.38
MS-UDG 79.70
30.01
40.11
53.73
63.77
65.82
53.37
55.52
trained model for the DomainNet and PACS datasets. Additionally, SSL training without ImageNet pre-trained model is conducted for comparison with DisMAE [54) on DomainNet. Further details of this experiment can be found in the Appendix.
Implementation Details. Following previous disentanglerment-basedd UDG methods 51 [2], we adopt VIT-S/16 [9] the backbone. The learning rate for pre- preitinin is set to 1 4 The weight decay is set to 0.05, and the batch size is 32. For 7 in Lgu/f. we set the temperature = 0.07 as previous work [1]]. During fine-tuning, all methods model is selected to test on unseen domain data, following the exact training schedule outlined in (54). The test domains also remain unseen during the model selection. Further details can be found in the Appendix.
trained for 50 epochs, and the best evaluation
5.2 Experimental Results
We present the experimental results in Tab. (PACS) and Tab. (DomainNet). Compared to contrastive-baser and generative-hased SSL. methods MoCo V2, SimCLR V2, BYOL, AdCo, and MAE, MS-UDG outperforms in most cases. Among contrastive-base methods, SimCLR v2, which

---

uses multi- -ielw augmented images as self-supervised signals, performs best. Compared with other SSL methods, MAE demonstrates better performance on the DomainNet dataset, second only to SimCLR V2 on the PACS dataset. From the results, these SSL methods perform poorly on downstream tasks with unseen domain data, as they ignore domain covariates.
Compared to the state-of-the-art (SOTA) SSL method SimCLR V2, MS-UDG improves the performance by +19.33%. +22.34 % 666.26.71* %2 and +22.83 % for label fractions 1%, 5%, 10% and 100%, respectively, on the PACS dataset. On the DomainNet dataset, MS-UDG improves the SOTA SSL method MAE by +17.04%, +18.45% %, and +14.52% in overall accu- racy for label fractions 1%,5 5%, and 10%.
From our results, MS-UDG outperforms most contrastive-based and disentanglemeent-basedd UDG methods across two datasets. Specifically, in linear evaluation with 1% and 5% label fractions, MS-UDG outperforms the SOTA method, BSS, by +0.35% and +6.62% % respectively, on the PACS dataset. On the DomainNet dataset, MS-UDG improves by +0.18 % and +1.6% in linear evaluation compared BSS. In full fine-tuning with 10% and 100% label fractions, MS-UDG surpasses the SOTA method CycleMAE by +5.48% and +1.48 %, respectively, on the PACS dataset. On the DomainNet dataset, MS-UDG shows an improvement of +2.59% overall accuracy for the 10% label fraction.
Figure 2: The reconstruction results produced by MS-UDG, Rows I and display input im- ages that preserve either semantic content or stylistic variations, derived from four distinct domains within the DomainNet dataset. In contrast, Rows 2 and 4 present reconstructed images using alternative variation representa- tions within the feature space.
5.3 Ablation Study
Effectiveness of Each Component of MS-UDG.
Table 3: Effectiveness of each component of MS-UDG on PACS dataset. All models are trained on 3 selected domains and tested on the remaining domains, which process is repeated for all domains. Best highlighted.
Since MS-UDG is based on two optimization objec- tives. we investigate the impact of each target. Our baseline is the native InfoNCE loss Czufs- Next, we introduce the IDM and SROM modules and evalu- ate Limin and Lmazs separately. Finally, we assess the performance of the entire method, incorporating fouff,, Ciin. and Lmaz.
Label Fraction 5%
Methods
1%
10%
Baseline (Loufe)
42.45
56.17
64.81
We evaluate the effectiveness on the PACS dataset. From Tab. 3 both applying Lmin and Lmaz improve the baseline, while the performance can be further improved by combining Emin and Lmax by +2.81% %
+ +Lman
42.85
57.91
65.32
+ +Lmax
44.77
57.72
67.40
+ + Cmoz
45.26
63.02
68.35
6,85% and +3.54% under 1%, 5%, and 10% label fractions. Combining the two losses further improves the results, aligning with our thcoretical analysis. Minimizing [(s; v) and maximizing T(o; ())) reduce the semantic-ireeleant information ((s:))
Visualization of Reconstruction. Fig. illustrates reconstructed images with consistent semantics and varying styles, showcasing MS-UDG's strong capability to separate features.
6 Conclusion and Limitation
In conclusion, we tackle the challenge of semantic disentanglement in UDG without domain labels by formulating it as a constrained optimization problem. Our framework, MS-UDG, integrates contrastive learning with novel modules to effectively separate semantics and variation, achieving otate-of-the-art performance. However, MS-UDG is primarily tailored for scenarios with pronounced domain discrepancies. This assumption may limit effectiveness in settings with subtle covariate shifts, where improvements in disentanglement are less evident.

---

References
[1] Albuquerque, J. Monteiro, M. Darvishi, H. Falk, and I. Mitliagkas. Generalizing unseen domains via distribution matching, arXiv preprint arXiv:1911.00804 2019.
[2] A. A. Alemi, I. Fischer, J. V. Dillon, and K. Murphy. Deep variational information bottleneck. arXiv preprint arXiv: ww....11..01100 2016.
[3] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. W. Vaughan. A theory of learning from different domains. Machine learning, 79:151-175, 2010.
[4] J. Cha, S. Chun, K. Lee, H.-C. Cho, S. Park, Y. Lee, and S. Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34:22405-2244 2021.
[5] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. simple framework for contrastive learning of visual representations. In International conference muchine learning, pages 1597-1607. PMLR, 2020.
[6] Chen, Kornblith, K. Swersky, M. Norouzi, and E. Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33:22243-22255. 2020.
[7] X. Chen, H. Fan, R. Girshick, and K. He. Improved baselines with momentum contrastive learning. arXiv preprint WoXiv:2003.04297, 2020.
[8] T. M. Cover. Elements of information theory. John Wiley & Sons, 1999.
[9] A. Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
[10] M. Feder and N. Merhav. Relations between entropy and error probability. IEEE Transactions on Information theory, (0(1):259-266 1994.
[11] M. Federici, A. Dutta, P. Forré, N. Kushman, and Z. Akata. Learning robust representations via multi-view information bottleneck. In International Conference on Learning Representations, 2020.
[12] K. Fukunaga. Introduction to statistical pattern recognition. Elsevier, 2013.
[13] J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo, and D. Tao. A survey on self self-supervised learning: Algorithms, applications, and future trends. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[14] I. Guirajani and D. Lopez-Paz. In search of lost domain generalization. arXiv preprint Xii::2007.01434, 2020.
[15] K. Guo and B. C. Lovell. Domain-aware triplet loss in domain generalization. Computer Vision and Image Understanding, 243:103979, 2024.
[16] S. Harary, E. Schwartz, A. Arbelle, P. Staar, S. Abu-Hussein, E. Amrani, R. Herzig, A. Alfassy, R. Giryes, H. Kuehne, al. Unsupervised domain generalization by learning bridge across domains. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 2280-5290, 2022.
[17] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick. Masked autoencoders scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[18] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729- 2020.
[19] Q Hu, X. Wang, W. Hu, and G.-J. Qi. Adco: Adversarial contrast for efficient learning of unsupervised representations from self-trained negative adversaries. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1074-1083, 2021.
10

---

[20] S. Hu, K. Zhang, Z. Chen, and L. Chan. Domain generalization via multidomain discriminant analysis. In Uncertainty in artificial intelligence, pages 292-302. PMLR, 2020.
[21] H. Kim and A. Mnih. Disentangling by factorising. In International conference on machine learning, pages 2649-2658. PMLR, 2018.
[22] B. O. Koopman. On distributions admitting a sufficient statistic, Transactions of the American Mathemalical society, 39(3):399-409 1936.
[23] D. Li, Y. Yang, Y.Z. Song, and T. M. Hospedales. Deeper, broader and artier domain gen- eralization. In Proceedings of the IEEE international conference on computer vision, pages 5542-55500 2017.
[24] P. Liang, Z. Deng, M. Q. Ma, J. Y. Zou, L.-P. Morency, and R. Salakhutdinov. Factorized contrastive learning: Going beyond multi-view redundancy. Advances in Neural Information Processing Systems, 36, 2024.
[25] T. Liu, Z. Tan, M. Chen, X. Yang, H. Jiang, and K. Huang. Medmap: Promoting incomplete multi-modal brain tumor segmentation with alignment. IEEE Journal of Biomedical and Health Informatics, 2025.
[26] Y. Liu, Y. Chen, M. Gou, C.-T. Huang, Y. Wang, W. Dai, and H. Xiong. Towards unsupervised domain generalization for face anti-spoofing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20654-20664, 2023.
[27] Y. Liu, Y. Wang, Y. Chen, W. Dai, C. Li, Zou, and H. Xiong. Promoting semantic connec- tivity: Dual nearest neighbors contrastive learning for unsupervised domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3510-3519, 2023.
[28] Mitrovic, B. McWilliams, C. Walker, L. H. Buesing, and C. Blundell. Representation learn- ing via invariant causal mechanisms. In International Conference on Learning Representations, 2021.
[29] H. Nam, H. Lee, J. Park, W. Yoon, and D. Yoo. Reducing domain gap by reducing style bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8690-8699, 2021.
[30] H. Nam, H. Lee, J. Park, W. Yoon, and D. Yoo. Reducing domain gap by reducing style bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8690 8699, 2021.
[31] L. NIu, W. Li, and D. Xu. Multi-view domain generalization for visual recognition. In Proceedings of the IEEE international conference on computer vision, pages 4201, 2015.
[32] A. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: a 1807.03748, 2018.
[33] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafranie, V. Khalidov, P. Fernandez, D. Haziza, Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
[34] Pan. Z. Tan, K. Guo, Xu, W. Xu, C. Jiang, X. Guo, Y. Qi, and Cheng. Structure-aware semantic discrepancy and consistency for 3d medical image self-supervised learning. arXiv preprint arXiv:2507.02580
2025.
[35] X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEF/CVF international conference computer vision, pages 1406-1455, 2019.
[36] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.

---

[37] K. Saenko, B. Kulis, M. Fritz, and T. Darrell. Adapting visual category models to new domains. In Computer Assion-ECCy 2010: 11th European conference on computer vision, Heraklion, Crete, Greece, September 5-11, 2010, proceedings, part iV 11, pages 213-226. Springer, 2010.
[38] M. Scalbert, M. Vakalopoulou, and F. Couzinié-Devy. Towards domain-invariane sef-ssueeviied learning with batch styles standardization. arXiv preprint arXiv:2303.06088 2023.
[39] M. Scalbert, M. Vakalopoulou, and F. Couzinie-Devy. Towards domain-invarinn self-supervised learning with batch styles standardization. In The Twelfth International Conference Learning Representations, 2024.
[40] R. Shwartz-Ziv and Y. LeCun. To compress or not to comprees--sef--supevvvsss learning and information theory: A review. arxiv 2023, arXiv preprint arXiv:2304.09355.
[41] 7. Tan, X. Yang. T. Pan, T. Liu, C. Jiang, X. Guo, Q Wang. A. Nguyen, Y. Qi, K. Huang, et al. Personalize to generalize: Towards a universal medical multi-modality generalization through personalization. arXiv preprint 2rXiv:2411.06/10 2024.
[42] Z. Tan, X. Yang. Q Wang. A. Nguyen, and K. Huang. Interpret your decision: Logical reasoning regularization for generalization in visual classification. Advances in Neural Information Processing Systems, 37:18166-18204 2024.
[43] N. Tishby, F. C. Percira, and W. Bialek. The information bottleneck method. arXiv preprint physics/0004057, 2000. 2000.
[44] Y.-H. H. Tsai, Y. Wu, R. Salakhutdinov, and L.-P. Morency. Self-supervised learning from multi-view perspective. In International Conference on Learning Representations, 2021.
[45] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2018-5027, 2017.
[46] H. Wang, X. Guo, Z.-H. Deng, and Y. Lu. Rethinking minimal sufficient representation in contrastive learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 16041-16050, 2022.
[47] J. Wang, C. Lan, C. Liu, Y. Ouyang. T. Qin, W. Lu, Y. Chen, W. Zeng, and P. S. Yu. Generalizing to unseen domains: A survey on domain generalization. IEEE transactions on knowledge and data engineering, (58))88052-807, 2022.
[48] T. Wang and P. Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning, pages 9929-9939 PMLR, 2020.
[49] X. Wang, H. Chen, Z. Wu, W. Zhu, al. Disentangled representation learning. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, 2024.
[50] Q. Xu, R. Zhang, Y. Zhang, Y. Wang, and Q. Tian. A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14383 14382-14392, 2021.
[51] H. Yang, X. Li, S. Tang, F. Zhu, Y. Wang, M. Chen, L. Bai, R. Zhao, and W. Ouyang. Cycle- consistent masked autoencoder for unsupervised domain generalization. In The Eleventh International Conference on Learning Representutions, 2022.
[52] H. Yang, Tang, M. Chen, Y. Wang, F. Zhu, L. Bai, R. Zhao, and W. Ouyang. Domain invariant masked autoencoders for self-supervised learning from multi-domains, In European Conference on Computer Vision, pages 151-168. Springer, 2022.
[53] X. Yu, H.-H. Tseng, S. Yoo, H. Ling, and Y. Lin. Insure: an information theory inspired disentanglement and purification model for domain generalization. IEFE Transactions on Image Processing, 2024.
12

---

[54] A. Zhang, H. Wang, X. Wang, and T.-S. Chua. Disentangling masked autoencoders for unsupervised domain generalization. In European Conference on Computer Vision, pages 126-151. Springer, 2025.
[55] H. Zhang, Y.FF Zhang, W. Liu, A. Weller, B. Schölkopf, and E. P. Xing. Towards principled disentanglement for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8024-8034 2022.
[56] X. Zhang. L. Zhou, R. Xu, P. Cui, Z. Shen, and H. Liu. Towards unsupervised domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4910-4920, 2022.
[57] K. Zhou, Z. Liu, Y. Qiao. T. Xiang, and C. C. Loy. Domain generalization: survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):43964444 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:
[Yes]
We describe the scope, assumptions and contributions in the Abstract and Introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discussed the limitations of multi-modality setting in Sec and further discussed the performance degradation when combining UDG and DG in Appendix
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: For assumptions, we give detailed statements and explanations in the main text For propositions and theorems, provide proofs in Appendix B
and Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed information of pre-training, linear probing, and fine-tuning settings in Sec. 5 and Appendix C. Meanwhile, for each component, we did ablation study
as shown in Sec. 5.3 Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [No]
Justification:
We will open the dataset splits and code upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //nips. lttaps.cccpublice wwtppbbiccgguueeeeeooooooccpcced
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide all the training and test details in Sec 5ndd Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We assess the proposed method alongside other SOTA approaches on five widely-used domain generalization benchmarks. All methods are subjected to identical training and evaluation settings, and the results are averaged across three different random seeds to ensure reliability and robustness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the information of computer resources in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
We follow the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We report broader impacts in Appendix K
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All data and code used in the paper are open-sourced, and follow their licenses and cite them in the paper. The data splits are also detailed in Sec. andd Appendix
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The code and documents will be released upon publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A Explanation of Concepts
A.1 Minimal Sufficient Representation
Minimal Sufficient Statistie:
The minimal sufficient statistic (MSS) proposes a concept effective statistic [8]. Given two sets of data X, YY (X,Y) ~ P(X, Y). F & (x( ) is statistic of X, if the Markov chain X + F + Y is satisfied, we can define the sufficient statistic, which captures the all information between and Y.
Definition A.1 (Sufficient Statistic 8 ) F is a sufficient statistic for Y if and only if I (),,Y)
=
T(X;Y)
However, the definition of sufficient statistic can not effectively extract the essential information 40 For instance, directly "copy" can satisfy the above definition. Thus, the minimal sufficient statistic is introduced.
Definition A.2 (Minimal Sufficient Statistic). A sufficient statistic F is minimal sufficient if and only if, any sufficient statistic S, 3g as a function, F g())
The MSS are sufficient statisties containing the least information about X 12 Based on the property
of MSS, many methods are proposed to obtain the effective representations, such as Information Bottleneck 43
Minimal Sufficient Representation: Motivated from the above discussion, the minimal sufficient representation is proposed for self-supervised learning 44 46). Definition introduce the definition of minimal sufficient representation, which is similar to the definition of minimal sufficient statistic. However, the minimal sufficient representation can still capture redundant information, especially when large distribution discrepancies exist in the dataset. For instance, the minimal sufficient representations from sketch images in PACS dataset may still capture the information of sketch domain. Therefore, we propose the definition of minimal sufficient semantic representation to handle this issue
A.2 OOD Generalization Bound
From the probably approximately correct (PAC) learning theory on multi-distribution B. the out-of- ditribution generalization bound has been proposed 4 Theorem A.3. Consider the risk on source domain D; is Rp(xD, No.) (h), then, we have
No
Reixiest (h) S Σ (o, div (D,, D.) +
Where div(D, De) is the divergence between the source distributions D2 and the unseen target distribution Dr, the confidence bound, and Na is the number of source domains.
For div(D., D.), Albuquerque al. define it the distance from unseen distribution De to the simplex supported by the source distributions D.. Cha al. 4 simply define it as 2N, Σ. 2 sup (A) (A) For X, Albuquerque et al. [I] consider as the shift of the label- ing function on the unseen distribution, and Cha er al. relate to the VC dimension of the model. In our paper, as we can not control these two items, we simply use A to present the summation of them. In addition, we intuitively consider the source risk Rp (p(x (h) Σi R .yx) (h). There- fore, we can utilize Theorem 4.3 to approximate the upper bound of the target risk RP (PPxxesn.Veaa) (h)
B Theoretical Proof
(ammin,77 :T) (e)min,T[z)) With lemma 1 in [46] I (6, T 21) and ( x2) should be zero. By the definition of interaction information, we have
Proof of Proposition 3.2 I(x;; T; 2mmi)) = I(2minn
1 (2 2mm)n) 1xx;;; T; smin) (xx;; smin (T)
1 (emin,T) + I [xx;; 2T amin (T)) 2 1 (
21

---

Proof of Proposition 3.5
Considering the Markov chain + 21, vf. f is arbitrary conditional
factor, we have (2) / x 0. This also the assumption in 46 so we have
(211)1 = [21;<]] ()) + (2 21)
[2 8)
Proof of Proposition 3.6
Following the assumption in Proof of Proposition 3.5 we have ((;;0
Then
[ S) ((
5) + [(u; x S,5)
= [(s: + [(x=][)) (u;
[ss:z|5) + ((v:=|S) [(s;u[S) + (s;e[S52))
2[) + T(u:=[S) I(s;e|S)
Since v(s; [(s; S). we have
[(s; I (2;x[5) [(s (0;
Proof of Proposition 4.1
According to 32 the log-bilinear model can be represented as density
ratio:
p(bla) a)
f(a, b) exp(a, b/T)
p(6)
a, are randomly given variables. In addition, The mutual information can be expressed as
p(bla)
I(a; b) Ex log(
p(b)
X is the given dataset. We set Zneg are representations from other images instead of æ1, which means
8g. Uk Zneg IZnegl = 2K Then, have
p(e2 01)
Lman Exx. log
p(az)
p(22(e1) p(22)
p(x- pe -E=-Enm
EZnog p(z-) + p(v))
Ex,,, log(1 2 p(82)
K p(32) p(u1 81) (
according 32
p(82 81)
p(82(51) p(1r)
Ex logl p(32) p(u1(s))
p(32 31) p(u1)
I(11;81) 82)
Proof of Proposition 4.2 Since 0 = I(u2; S), we have I(22:22|s1) < [(22;12) = I(22: (5)
Thus
T(U2,813 [(81;22) + [112 x2 (8))
< <I($) + [[2;; x2 SS)
From our assumption, 81 may not contain all the information the shared information between #1 and 22, thus 7(81; SS < (211222 (S). Then, with Definition 3.3 have
[(s1: 1(51;22; + I [(81: 22 5))
[ S) + (x S)
((x1;22)
22

---

Then.
81:22) (2) 1 (e2, 81: [ - [122; S)
Proof of theorem 4.3 According to [10]. the equality between e and H 21 (1 is
-In e - 2
Generally, we consider that sss is large enough cover the semantic information in Xupp thus we have ; ;S) = I(=1" ; :T). Then,
H(T =1")) H H(S) H((==1²))
So, we should consider the H (1²)) for the upper bound of enuf
H =( = [1 - uh (
[H(S (1) + 8
(S (1) + [11122] /u/ |S) - [<11=21 sw/) 1(x1; S)
S (S + ( 2//
|S) (
(()) (z1
H(S) + (
Thus, H( & H(T + [x11 no/ (). Then we have
cuf < expl - (H) [(21
((274/1 ;01 [S]]
Since T(=1rin; 2n (S) T(=1"1 mn ()), the upper bound of eminn is minimum in the sufficient repre- sentations.
C Implementation Details
For pre-training, we use ViT-S/16 as the backbone and the same for fine-tuning. The pre-training follows cosine decay schedule. All experiments were conducted on single Nvidia A100 80GB GPU, an 8-core CPU, and 250GB of memory. The details of experimental settings can be seen in Tab. 4 For all models, the input size is 224 224. When performing linear probing, we adopt the same normalization scheme as described in [17 by incorporating a batch normalization layer. Experiments are conducted three times with different random seeds, and the average accuracy from these runs is reported.
Table 4: Experimental Settings. Lp represents linear probing, and ft represents fine-tuning.
Tabel fraction
strategy
backbone
epech
batch weight_ decay
leaming
optimizer
betas
Pretraining (DomainNet and PACS)
VIT-S/16
(0.9,0.95)
Finetuning (DomainNet
1%
Ip
VIT-S/16
50
32
0.05
Sc-4
AdamW
(0.9.0.95)
58
Ip VIT-S/16
Adam
(0.9.0.95)
10%
ft VIT-S/16
50
128
0.05
5c-4
AdamW
(0.9.0.95)
100%
ft VIT-S/16
128
0.05
5e-4
AdamW
(0.9.0.95)
Finetuning (PACS)
1%
Ip VIT-S/16
50
16
0.05
5e44
AdamW
(0.9,0195)
5%
Ip VIT-S/16
0.05
5e-4
AdamW
(0.9.0.95)
10%
VIT-S/16
64
0.05
5e:5
AdamW
(0.9.0195)
100%
ft
VIT-S/16
50
64
0.05
5e-5
AdamW
(0.9.0.95)
D Effects of Fourier-based Augmentation
By comparing Tab. with Tab. [ we observe that the baseline, i.e., using only Lsuffs outperforms certain UDG methods. Prior work (56) has shown that Fourier-based Augmentation (FA) enhances
23

---

UDG performance. Here, we further investigate its impact our method. As presented in Tab, Gsufs without FA corresponds the native InfoNCE loss, aligning with MoCo V2 7 and achieves comparable average performance. However, incorporating FA significantly boosts the baseline by +21.22%. Our findings are consistent with BSS [39], where FA improves SimCLR V2's baseline performance by + +16.02% further underscoring the effectiveness of FA in self-supervised learning.
Table 5:
Results of detailed ablation study on PACS in label fraction 1%. "w/o FA" and "w/ FA" denote training without and with Fourier-based Augmentation, respectively. Best highlighted.
Methods
photo
cartoon
sketch
avg.
MoCo V2
22.97
15.58
23.65
25.27
21.87
Baseline (Cauff w/o FA)
25.02
23.75
16.85
19.31
21.23
Bascline (Couf w/ FA)
34.89
45.9
47.68
41.33
42.45
+ Lmin
35.55
44,93
47.84
43.08
42.85
+ +maa
39.82
46.28
47.04
45.94
44.77
+ +Lmin +mmar
42.74
47.13
47.84
43.33
45.26
E Learning without ImageNet Pre-trained Model
In this section, we discuss the performance of UDG methods learning without initializing with a pre-trained model ImageNet. To compare with previous work [54] fairly, we use ViT-B/16 as our backbone, and the depth of the decoder layer is 1, following the setting in the work [4 As shown in Tab. 6 MS-UDG outperforms SOTA methods in most settings.
F Effects of Decoder Depth
As previous studies explored the impact of decoder depth (the number of multi-head attention blocks, as in MAE), we present the results for different decoder depths on DomainNet in Tab, 7 In line with prior work 49 51), our findings confirm that lightweight decoder achieves superior performance.
G Visualization of Extracted Features Multi-domain Data
We present (-SNE visualization of six domain features from MS-UDG and MAE on the DomainNet datasets. From Fig. 3 MS-UDG separates variations well.
Table 6: Evaluation of UDG and SSL models which train without the initialization from the ImageNet pre-trained model. Most results are drawn from the work (21). We keep the same setting with DisMAE: pre-training 500 epochs and fine-tuning 50 epochs. Best is highlighted.
Label Fraction 1%
Label Fraction 5%
Target domuin
Target domain
real Isketch
overall
avg.
paint Jreal Isketch
overall
avg.
ERM
7,98
9.94
5.38
8.46
7.77
6,48
8.64
9.84
8.29
8.32
MAE
8.80
10.30
12.62
10.38
10.57
12.13
17.63
15.02
15.60
14.93
DARLING
8.59
9.01
11.10
9.32
9.57
9.31
12.00
13.72
11.61
11.68
CycleMAE
9.21
9.49
6.62
8.82
8.44
11.44
14.21
12.58
11.88
DisMAE
11.04
11.64
12.46
11.65
11.71
13.69
17.74
16.92
16.47
16.12
MS-UDG
12.38
12.41
12.77
12.52
12.48
16.87
21.87
17.52
19.59
18.76
Label Fraction
10%
Label Fraction
100%
Target domain
Target domain
paint maint.rrcallskee Jskethh
overall
avg.
paintl IreallJskethh
overalll
avg.
ERM
14.14
16.76
12.63
15.19
14.51
26.25
33.29
23.25
29.28
27.39
MAF
23.07
22.20
43.72
37.40
DARLING
13.72
19.76
16.43
17.40
16.64
25.87
37.60
26.67
32.11
30.05
CycleMAE
17.80
22.93
17.15
20.23
19.29
31.72
39.44
30.00
35.39
33.72
DisMAE
24.49
27.06
28.24
26.61
26.60
38.89
45.95
43.72
43.58
42.85
MS-UDG
29.31
27.09
26.53
51.60
45.08
46.99
45.50
24

---

Table 7: The results the different depths of the decoder layer DomainNet. The fine-tuning results are conducted on source domain set (clipart, infograph, quickdraw and target domain set (painting, real, sketch) with label fraction of 1%. Best is highlighted.
layer depth
overall avg.
27.63
31.02
2
25.83
26.63
25.33
26.55
24.05
24.94
(a) Semantics
b
Variation
(a) Features
MS-UDG
MAE
Figure 3: We present the t-SNE visualization six domain features from MS-UDG and MAE on the DomainNet dataset, with semantics and variation features of MS-UDG visualized separately.
H Effects of Cross Attention in SROM
In the SROM module, use the cross attention fuse 8 and 02 inspired by Stable Diffusion (36. which uses cross attention on prompts and image features. Here, we view the semantic representation 81 prompt to fuse semantics and variation. The ablation study of cross-attention is shown in Tab. Using cross attention to fuse v and s further enhances performance.
Table 8: Results with and without cross-attention. "w/o Cross Attn" and "w/ Cross Attn" denote training without and with cross-attention, respectively. Best is highlighted.
Label Fraction I 1%
Source
paintingUreall
Target
overall
avg.
w/o Cross Attn
54.82
19.47
18.94
26.36
31.08
w/ Cross Attn
49.44
20.90
22.71
27.63
31.02
Label Fraction 10%
Source
PeintinggUrealUssee
Target
clipart iigrattuiinngraahuuuuiuudaaaa
overall
avg.
w/o Cross Attn
79.03
32.89
36.31
44.02
49.41
w/ Cross Attn
79.70
30.01
40.11
45.33
49.94
I Results on More Benchmarks
We evaluate several SOTA methods on more benchmarks OfficeHome 45 VLCS D. and Of- fice31 [37] those datasets have limited images, making them less ideal for SSL, we used them downstream tasks by pre-training DomainNet and evaluating other datasets. To preserve
25

---

Table 9: Evaluation OfficeHome. Best is highlighted.
Label Fraction 1%
Pre- Pre-training Domain(Domainkl
Target Domain((OficeHomm
Art
Clipart
Product
avg.
MAE
13.41
24.63
23.21
20.42
DisMAE
22.89
18.07
43.79
28.25
BSS
24.39
27.16
35.53
29.03
MS-UDG
27.71
24.79
37.71
30.07
Label Fraction 5%
Pre-training (oomainnD
Painting RSketc
Target Domain(OfficcHomme)
Art
Clipart
Product
avg.
MAE
42.68
43.79
42.41
42.96
DisMAE
44.64
45.82
45.2
45.22
BSS
43.9
50.53
42.69
45.71
MS-UDG
49.4
58.16
56.78
54.78
the unseen domain setting, overlapping domains were excluded. Those benchmarks are all adopted leave-one-domain-our validation. The results, compared with SOTA UDG methods DisMAE and BSS, are summarized in Tab. 9
10 and II MS-UDG outperforms in those benchmarks as well.
OfficeHome consists of four domains with 15,500 images: Art, Clipart, Product, and Real-World. The Real-V World domain is excluded from the evaluation because of the overlap with the pre-train pre-training domain of DomainNet. VLCS consists of images from four distinct domains with around 10,000 images: Caltech101, LabelMe, SUN09, and VOC2007. Office31 consists of images from three distinct domains with 4110 images: Amazon, Webcam, and DSL. More details of datasets can be found in DomainBed [14]
J A Comparison of UDG and DG
Given the differences in scope and the available prior knowledge between DG and UDG, discussed in the following paragraph, making a direct and fair comparison between the two proves challenging. To further explore this, evaluate the performance of combining UDG with DG, i.e., pre-training with UDG methods followed by supervised learning using DG methods. We conducted some experiments with the UDG+DG approach as shown in Tab. 12 We observed some interesting results. When using 1% labeled data, SWAD negatively impacted the performance of all UDG models. However, with 100% labeled data, linear probing and fine-tuning produced different outcomes. Fine- tuning with SWAD improved the performance of DisMAE and MS-UDG, while slightly degrading the results for BSS and MAE. On the other hand, linear probing with SWAD led to slight performance degradation across all methods, except for MAE. This could be due to the limited number of tuned parameters in linear probing or the small training data few-shot settings, which might make difficult for SWAD to find flat minima. This is an intriguing phenomenon and presents an interesting avenue for future research in this area.
In the following, we discuss the detailed differences between UDG and DG. Unlike traditional DG methods, UDG leverages unlabeled data through self-supervised learning strategy to improve generalization on downstream tasks. In other words, UDG provides an unsupervised pre-trained model (upstream), whereas traditional DG methods train supervised model (downstream). Below are the differences from the setup and theory perspectives.
Firstly, from setup perspective, traditional DG methods typically train category labels to enhance generalization, whereas UDG uses only unlabeled data to train a pre-trained model. Given the differences in scope and the available prior knowledge between DG and UDG, believe it is challenging to make a direct and fair comparison between the two.
Secondly, from the theory perspective, traditional DG methods always focus on the generalization bound of the target risk since the methods are built on the available category labels. Thus, the original theory of DG is hard to adapt to the UDG area.
26

---

Table 10: Evaluation VLCS. Best is highlighted.
Label Fraction 1%
Pre-training Comain(Domainlle
Quirkkdrww
Target Domain(
1o
LabelMe
SUN09
VOC2007
MAE
62,79
46.85
39.19
43,93
48.19
DisMAR
68.17 17
57.24
41.66
45.74
53.20
BSS
58.20
44.31
40.25
42.04
46.20
MS-UDG
68.17
49.11
41.93
45.04
51.06
Label Fraction 3%
Pre-tree Pre-training Domaint DomainNet)
Quikkkrrrap
Target Domain(VI.CS,
Caltech101
LabelMe
SUN09
VOC2007
avg.
MAE
69.84
49.01
44.44
43.93
51.81
DisMAE
69.93
54.23
43.37
43.22
52.69
BSS
68.34
47.13
40.21
41.6
49.32
MS-UDG
79.01
52.12
54.65
49.78
Table 11: Evaluation on Office31. Best is highlighted.
Label Fraction 1%
Pre- treiniann Domain(DomainNer Target 1)
Amazon
DSLR
Wchcam
avg.
MAE
21.05
20.00
20.63
20.56
DisMAE
21.00
30.77
20.63
24.13
BSS
22.81
18.46
17.46
19.58
MS-UDG
24.56
41.54
19.05
28.38
Label Fraction 5%
Pre- Pre-train Pre-training Domain(Domain)
Infoorrphh [Quickdraw
Target 1)
Amazon
DSLR
Webcam
avg.
MAE
20.8
32.31
27.78
26.96
DisMAR
45.11
33.85
33.33
37.43
BSS
44.86
35.38
34.13
38.12
MS-UDG
58.40
52.31
50.00
53.57
K
Broader Impacts
This research seeks to advance the field of Unsupervised Domain Generalization (UDG) by addressing two core objectives: disentangling semanties and variation without domain labels and category labels. These efforts have the potential to make significant contributions across multiple domains, including healthcare, where the ability to generalize from limited labeled data of paramount importance. In fields where large-scale unlabeled data is scarce but robust generalization is essential, our approach to disentangling these factors in an unsupervised manner could provide a critical insight, enabling models to perform well in unseen environments without requiring extensive labeled data.
27

---

Table 12: Evaluation on UDG+DG(SWAD) methods.
Label Fraction 1% (Linear probing)
Pre Pre-trainn te-itrainng Domain(Domainneer
Painting RealUSSetch
Target (onain((fticcetomme))
Clipart
Product
arg.
MAE
13.41
24.63
23.21
20.42
MAF+SWAD
10.84
17.23
21.14
16.40
DisMAE
22.89
18.07
43.79
28.25
DisMAEASWAD
12.05
18.28
23.71
18.01
BSS
24.39
27.16
35.53
29.03
BSS+SWAD
13.86
23.95
27.14
21.65
MS-UDG
27.71
24.79
37.71
30.07
MS- SS-UDD+SWAA
21.08
26.89
38.29
28.75
Label Fraction (Fine tuning)
traitranning Domain(Doni) DemmainNt))
Painting RealUSketch
Target Domain(Officellonn))
Art
Clipart
Product
avg.
MAE
54.00
67.53
69.31
63.61
MAE+SWAD
48.00
67.13
71.43
62.19
DisMAE
55.50
67.13
75.4
66.01
DiSMABASWAD
59,00
74.3
76.98
70.09
BSS
51.00
63.35
65.87
60.07
BSS+SWAD
47.5
63.15
65.61
58.75
59.00
69.72
73.28
67.33
MS-UDG+SSAD
62.5
73.51
76.46
70.82
Label Fraction (Oonear probing)
Pre-training Domain (Domaine))
PaintingUReall
Skkethh
Target Domain(OfficeHomme)
Art
Clipart
Product
avg.
MAE
29.00
36.65
43.12
36.26
MAE+SWAD
51.00
35.86
65.34
50.73
DisMAE
40.00
58.76
53.17
50.64
DiSMAE+SWAD
40.00
60.36
44.18
48.18
BSS
40.00
55,58
53.97
49.85
BSS+SWAD
41.5
55.38
51.59
49,49
MS-UDG
50.00
67.33
63.76
63.76
MS-UDG+SWAD
52.00
67.73
65.34
61.69
28


Reviews:
[{"review_id": "az8KiHEmfO", "reviewer": "Reviewer_Uubv", "summary": "The paper addresses Unsupervised Domain Generalization (UDG) for image classification by formalizing the task through an information-theoretic framework. The authors propose learning Minimal Sufficient Semantic Representations that capture shared semantic information across multiple augmented views of the same image while eliminating semantically irrelevant information. The approach includes theoretical proof demonstrating that this formulation reduces out-of-distribution risk, which directly addresses the UDG challenge.", "questions": "Can you provide failure case analysis explaining why certain competing methods outperform MS-UDG in specific scenarios, and what insights does this offer about your method's limitations ?", "limitations": "yes.", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n- **Writing Quality** - The paper is exceptionally well-written with clear and well-motivated technical components.\n- **Comprehensive Evaluation** - The experimental comparison against existing methods is extensive and thorough, providing strong empirical validation.\n- **Complete Ablation** - The ablation studies effectively justify the use of both loss components, demonstrating that their combination yields superior performance.\n- **Theoretical Foundation** - The paper provides comprehensive theoretical analysis, introducing all necessary concepts regarding minimal sufficient semantic representations before presenting the method and its implementation.\n\n**Weaknesses:**\n\n- **Missing Failure Analysis** - The paper lacks a failure analysis section examining cases where competing methods outperform MS-UDG. Such analysis would provide valuable insights into the method's limitations and help readers understand when and why the approach may not be optimal.", "was_revised": true, "final_justification": "I appreciate the authors' response addressing my concerns regar sectionding the limitations of MS-UDG in some specific scenarios. The provided explanation has resolved my initial question, and I will therefore keep my original score.", "rebuttal": "We appreciate the reviewer for the positive recognition of our work, including the writing quality, comprehensive evaluation, complete ablation studies, and theoretical foundation. Below, we provide a detailed analysis of the failure cases and discuss the insights gained from competing methods that sometimes outperform MS-UDG in specific scenarios.\n\n**Missing Failure Analysis**\nWe conducted a comparative analysis with two advanced UDG methods, BSS and CycleMAE. The results highlight the strengths and limitations of each approach.\n\n* BSS (Fourier Augmentation Method): \nBSS benefits from mixing domain information through augmentation techniques, which sometimes enables it to outperform our method in domains with simpler and more uniform textures, such as sketches, quickdraw, and clipart. This improvement may not be suitable for rich-style domains, like art and real, since these domains contain richer low-frequency information compared with other domains. The Fourier augmentation in BSS may not significantly alter the style, which limits the effectiveness of data augmentation. Without disentanglement or other generalization techniques, the performance of BSS on rich-style domains is often inferior to that on other domains.\n\n* CycleMAE (Generation-Based Method):\nGeneration-based methods, such as CycleMAE, learn informative representations through reconstruction. However, domains like sketches often result in trivially low values of reconstruction losses because of simple backgrounds and domain styles. As a result, CycleMAE tends to focus on learning representations from richer-style domains, such as real images, art, and infographics. This bias toward more complex domains can cause CycleMAE to outperform MS-UDG in those specific areas, while in simpler-style domains, it underperforms compared to MS-UDG.. \n\nEven though these two methods occasionally perform better than our method through their own advantages, our MS-UDG method strikes a balance between these two approaches, which can enhance the generalization ability.  $L_{max}$ aims to learn the domain-related representation through reconstruction and $L_{min}$ helps disentangle the domain-related information from sufficient semantic representations.   As a result, our method achieves state-of-the-art performance on average, while still maintaining robustness across a diverse set of domains.\n\nWe will add these discussions to the revised manuscript. Thank you for the advice for better insights into those methods."}, {"review_id": "Q8JOwRkSMh", "reviewer": "Reviewer_sD9g", "summary": "This paper explores Unsupervised Domain Generalization (UDG), a task aimed at improving the generalization of unsupervised learning models like Self-Supervised Learning (SSL) without using category labels. While prior UDG methods rely on domain labels to separate semantics from variations, this approach is limited by the unavailability of such labels in real-world scenarios. The authors propose a new framework called Minimal-Sufficient UDG (MS-UDG), which learns semantic representations by optimizing for both sufficiency (retaining shared semantic information) and minimality (removing irrelevant variations). Based on information theory, MS-UDG combines an InfoNCE-based objective with novel components to disentangle semantics and variations, achieving state-of-the-art performance on UDG benchmarks without needing category or domain labels.", "questions": "Please refer to the Weaknesses.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. This paper is well motivated.\n\n2. The experimental results are good.\n\n3. Overall, the proposed method is simple and effetive.\n\nWeaknesses:\n\n1. Line 166 I(x_1;x_2;T) = I(x_1;x_2;S), what does S stands for? According to the following text, S may stand for semantic. But what does semantic information in X_{ssl} means? SSL should contain no label, so it hard to understand.\n\n2. Line 116~117 downstream supervised data P(X_{sup}, Y_{sup}) is used for fine-tuning? Is it the source domain ine Line 270?", "was_revised": true, "final_justification": "My issues are addressed. After reading all reviewers' comments and authors responses,  I prefer to maintain my initial rating.", "rebuttal": "We thank the reviewer for the thoughtful and positive feedback, including sound motivation, effectiveness of the method ,and promising experimental results. Below, we address each comment and question in detail.\n\n**W1: Explanation of semantic factor $\\mathcal{S}$**\n\n**W1-Ans.** While self-supervised models cannot anticipate specific downstream tasks during pretraining, we can systematically identify categories of information that consistently impede cross-domain generalization, such as task-irrelevant information. In UDG settings, task-irrelevant information typically encompasses domain-specific characteristics such as imaging style, background patterns, lighting conditions, and acquisition artifacts, collectively represented as $\\mathcal{V}$ in our framework. Given our decomposition $\\mathcal{Z}=\\mathcal{S}\\oplus\\mathcal{V}$, the semantic component $\\mathcal{S}$ captures the complementary information: fundamental visual structures, object relationships, and content-based features that remain consistent across domains and prove valuable for downstream recognition tasks.\n\nThis principled separation between semantic content and domain variations builds upon established theoretical foundations in UDG literature [1,2]. The practical significance becomes evident in real-world scenarios where training data inevitably spans multiple domains. For instance, medical datasets may aggregate scans from different institutions, each using varied imaging protocols, scanner manufacturers, and acquisition parameters. The resulting domain shifts introduce spurious correlations that can mislead models away from clinically relevant features. Our framework addresses these challenges by explicitly optimizing representations to retain semantic sufficiency while systematically reducing dependence on domain-specific information.\n\n**W2: Clarify the finetuning data**\n\n**W2-Ans.** Both the SSL pretraining data $X_{ssl}$ and supervised finetuning data $X_{sup}$ originate from source domains, following the standard UDG protocol where no target domain data is accessible during training. This setup mirrors classical domain generalization but operates without category labels during the representation learning phase. The key challenge lies in learning generalizable features from source domain variations that transfer effectively to unseen target distributions at test time. Our experimental setup strictly adheres to the benchmark protocols established in [1,2], ensuring fair comparison with existing methods. Specifically, we use the same train/validation/test splits and domain partitions as prior UDG works, where source domains provide both unlabeled data for SSL pretraining and labeled data for downstream task finetuning, while target domains remain completely held out until evaluation.\n\n\n[1] Zhang, An, et al. \"Disentangling Masked Autoencoders for Unsupervised Domain Generalization.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.\n\n[2] Zhang, Xingxuan, et al. \"Towards unsupervised domain generalization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022."}, {"review_id": "UQZgF2IGZx", "reviewer": "Reviewer_yW3M", "summary": "Authors propose an Unsupervised Domain Generalization (UDG) algorithm with theoretical analyses from the perspective of information theory. They also conduct extensive experiments to show its effectiveness.", "questions": "Please refer to weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- There are theoretical justifications for the proposed algorithm.\n- Experimental results clearly demonstrate the effectiveness of the proposed algorithm.\n\nWeaknesses:\n\n- Newer and more recent SSL methods could be added as baselines, e.g. MoCo-v3 [1], SMoG [2], ReLICv2 [3], BAM [4]\n\n\n\n[1] Chen X, Xie S, He K. An empirical study of training self-supervised vision transformers[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 9640-9649.\n\n[2] Pang B, Zhang Y, Li Y, et al. Unsupervised visual representation learning by synchronous momentum grouping[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 265-282.\n\n[3] Tomasev N, Bica I, McWilliams B, et al. Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet?[J]. arXiv preprint arXiv:2201.05119, 2022.\n\n[4] Shalam D, Korman S. Unsupervised Representation Learning by Balanced Self Attention Matching[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024: 269-285.", "was_revised": true, "final_justification": "My concerns are well addressed in rebuttal, so I decide to raise my score.", "rebuttal": "We thank the reviewer for the positive recognition of the theoretical justification and the effectiveness of our proposed algorithm based on the experimental results.\n\nRegarding the suggestion to include newer SSL methods such as MoCo-v3 [1], SMoG [2], ReLICv2 [3], and BAM [4] as additional baselines, we appreciate this valuable recommendation. We have conducted additional experiments incorporating SSL methods MoCo-v3, SMoG, and BAM within UDG settings, as outlined below. Our method demonstrates competitive performance when compared to these SSL approaches.\n\nRegarding ReLICv2, the publicly available implementation provides only linear evaluation components without the complete SSL pretraining pipeline. Despite our efforts to contact the original authors for implementation details, time constraints prevented us from developing a full reimplementation from scratch. We acknowledge this limitation in our current evaluation and commit to including ReLICv2 in future benchmark extensions once the implementation becomes available. \n\nWe plan to publicly release our UDG benchmark implementations for these SSL methods to facilitate future research and ensure reproducible comparisons within the community.\n\nAll reported experiments utilize the official codebases of the respective SSL methods, adapted to conform with standard UDG evaluation protocols while maintaining their core algorithmic components.\n\n| target domain | photo | art   | cartoon | sketch | avg   |\n|---------------|-------|-------|---------|--------|-------|\n| SMoG          | 70.81 | 48.72 | 45.76   | 48.71  | 53.50 |\n| BAM           | 52.16 | 54.33 | 46.72   | 58.89  | 53.03 |\n| MoCo-v3        | 70.61 | 64.10 | 54.03   | 52.21  | 60.24 |\n| Ours          | 74.10 | 61.90  | 63.73   | 73.66  | 68.35 |\n\nTable 1. The results on PACS under 10% label fraction. We report the accuracy for every domain and the average accuracy for all domains.\n\n| target domain | clipart | infograph | quickdraw | painting | real  | sketch | overall | avg.  |\n|---------------|---------|-----------|-----------|----------|-------|--------|---------|-------|\n| SMoG          | 41.75   | 17.41     | 17.15     | 24.22    | 35.50 | 26.52  | 26.37   | 27.09 |\n| BAM           | 74.96   | 30.88     | 25.66     | 37.07    | 46.24 | 47.67  | 43.98   | 43.75 |\n| MoCo-v3        | 74.90   | 30.56     | 30.98     | 52.54    | 62.31 | 52.85  | 49.33   | 50.69 |\n| Ours          | 79.70   | 30.01     | 40.11     | 53.73    | 63.77 | 65.82  | 53.37   | 55.52 |\n\nTable 2. The results on DomainNet under 10% label fraction. We report the accuracy for every domain and the average accuracy for all domains."}, {"review_id": "8hNXdxLkYj", "reviewer": "Reviewer_Ckt1", "summary": "This manuscript introduces MS-UDG, an unsupervised domain generalization method that learns minimal semantic representations by disentangling semantics from variations without requiring labels. While it shows improved performance over prior SSL and UDG methods, its contributions are largely incremental, with limited novelty. Additionally, inaccuracies in table annotations and weak theoretical insights diminish its overall contribution.", "questions": "Please refer to the weakness for more questions. I am open to changing the score if the concerns are well addressed.\n\nAdditional question\n\n1. The concepts of sufficiency and minimality are directly adopted from existing work. It remains unclear what novel theoretical or practical contributions this manuscript provides beyond these established definitions.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1. The experimental evaluation is comprehensive, including both quantitative benchmarks and qualitative visualizations. \n2. The paper attempts to provide theoretical foundations for the proposed contrastive learning framework.\n\nWeakness\n1. The manuscript offers limited theoretical novelty, as it relies on the well-established concept of Minimal Sufficient Semantic Representation (Line-145) without introducing new insights. Moreover, information-theoretic frameworks have already been extensively explored in domain generalization [#1,#2,#3,#4]. The proposed method exhibits little distinction from prior work, as it merely applies existing ideas without providing any theoretical advancements.\n2. Compared to existing work [#5], this manuscript primarily proposes incremental improvements, including introducing a mask, applying self-attention, and incorporating additional features for contrastive learning. Overall, the manuscript reads more like an extension or combination of [#5] and [#6] rather than presenting fundamentally new contributions.\n3. In Table 1, under the Label Fraction: 10% setting, MS-UDG achieves the second-best performance in the photo and art domains, yet the authors mark it as the best. Similar inconsistencies appear in other parts of the manuscript, where MS-UDG results are incorrectly highlighted as the top-performing method despite ranking second. These misannotations may mislead readers and should be carefully corrected.\n4. In the ablation study shown in Table 5 for the sketch domain, the performance when all losses are combined is worse than using each loss individually.\n5. In Figure 3, the t-SNE visualization shows that the semantic features from MS-UDG exhibit poor category separability.\n6. It is well known that self-supervised learning (SSL) benefits unsupervised tasks; this is not a novel contribution to the manuscript. The claim of operating without domain and category labels is also expected in the unsupervised setting and should not be regarded as a unique contribution.\n\n[#1] INSURE: an Information theory iNspired diSentanglement and pURification modEl for domain generalization[J]. IEEE Transactions on Image Processing, 2024. \n\n[#2] Rethinking domain generalization: Discriminability and generalizability[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024. \n\n[#3] How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis[J]. IEEE Transactions on Information Theory, 2025. \n\n[#4] Invariant information bottleneck for domain generalization[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022\n\n[#5] Towards unsupervised domain generalization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022\n\n[#6] Rethinking minimal sufficient representation in contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16041–16050, 2022", "was_revised": true, "final_justification": "Thank the authors for their response. While I appreciate the clarifications provided, using an existing theoretical solution in SSL for the first time in DG does not, in my view, constitute a strong theoretical/practical contribution, especially since SSL has already been justified in tackling DG problems in previous works. The contribution does not yet meet the bar expected for NeurIPS on my side, and I do not believe that this work will have a noticeable impact to the community.  Nonetheless, after carefully considering the completeness of this work and the ratings from other reviewers, I can increase my score to borderline acceptance.", "rebuttal": "We thank the reviewer for the detailed and insightful feedback on our manuscript. We reply on each point raised and address them as follows:\n\n**W1,W6,Q1: Theoretical novelty**\n\n**W1,W6,Q1-Ans.** First, we clarify that minimal sufficient representation builds on the established statistical concept of minimal sufficient statistics. Information bottleneck (IB) previously applied these principles to supervised learning, addressing the research problems of that time. Some methods extend this to the SSL scenario to discuss learning a minimal sufficient representation in unsupervised learning.\n\nOur framework adapts minimal sufficient statistics for learning domain-invariant representations in unsupervised domain generalization (UDG).  Crucially, UDG and supervised DG face fundamentally different theoretical challenges. PAC learning risk bounds for supervised domain generalization don't apply to self-supervised models where labels aren't available during training. This theoretical gap requires new frameworks tailored to the unsupervised setting. **Our work provides the first information-theoretic approach developed specifically for UDG.** Two learning objectives are induced from our theoretical framework, with empirical results validating this theoretical foundation across multiple benchmarks. In addition, **our framework first relates the representation in SSL model to the downstream generalization bounds in UDG.**\n\nIn the end, we appreciate the reviewer thinking about the theoretical improvement. Our theoretical discovery on UDG parallels how IB-based methods [7,8] use the same statistical foundation for domain generalization—these works appeared at top conferences and demonstrated how classical concepts can tackle modern problems. In addition, the analogy can extend to methodological relationships: ERM(empirical risk minimization,i.e., CrossEntropy for classification) provides the baseline for domain generalization methods, while SSL serves as the foundation for UDG approaches like ours. We will clarify and emphasize the unique contributions in the revised version.\n\n**W2: Technical improvements** \n\n**W2-Ans.** We thank the reviewer for the detailed technical discussion. Our paper's core contributions span two interconnected areas: 1) information-theoretically grounded optimization objectives for UDG; 2) practical algorithms that implement these theoretical insights. \n\nOur approach differs fundamentally from existing UDG methods. The UDG baseline [5] employs adversarial training with domain labels to achieve domain invariance. This strategy operates independently of information-theoretic principles and targets different learning objectives than our framework. Meanwhile, H. Wang et al. [6] focus on capturing non-shared task-relevant information across domains to avoid overfitting, which contrasts with our goal of disentangling semantic content within the shared representation space for domain generalization.\n\nThese methodological differences reflect deeper theoretical distinctions. While previous work addresses domain invariance through adversarial objectives or captures domain-specific information, our framework tackles the problem by formalizing what constitutes an optimal representation for generalization: one that preserves semantic sufficiency while achieving minimality. This information-theoretic perspective leads to novel optimization strategies that haven't been explored in prior UDG, DG, and SSL literatures.\n\n**W3,W4: 1. Misannotation and 2. table explaination**\n\n**W3,W4-Ans.** \n\nMisannotation: We thank the reviewer for identifying the annotation errors. These have been corrected in the revised manuscript.\n\nTable explaination: The tabulated results demonstrate our method's effectiveness across domains. Our loss design aims to improve generalization consistently across diverse domains rather than optimizing for individual domain performance. While the sketch domain shows a marginal decrease when using the complete loss formulation versus $\\mathcal{L}_{max}$ alone, the remaining domains (photo, art, cartoon) all demonstrate substantial improvements. This trade-off yields the highest overall average accuracy (45.26), confirming that our unified framework successfully balances performance across the full domain spectrum rather than overfitting to specific domain characteristics.\n\n**W5: Explanation of the t-SNE visualization**\n\n**W5-Ans.** The t-SNE plots examine domain invariance properties by showing how our semantic representations $\\mathcal{S}$ exhibit reduced domain clustering compared to baseline methods. The representation $v$ and vanilla MAE representations display clear domain separability, which validates our disentanglement approach, domain-specific information concentrates in the variation subspace, while semantic content becomes more domain-agnostic. This analysis targets the fundamental question of whether our method successfully separates domain-related variations from semantic content, rather than evaluating downstream classification performance. The visualization demonstrates that semantic representations avoid the domain-specific clustering patterns that typically hinder generalization, which aligns with our theoretical framework's emphasis on learning domain-invariant semantic features.\n\n\n[1] INSURE: an Information theory iNspired diSentanglement and pURification modEl for domain generalization[J]. IEEE Transactions on Image Processing, 2024.\n\n[2] Rethinking domain generalization: Discriminability and generalizability[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2024.\n\n[3] How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis[J]. IEEE Transactions on Information Theory, 2025.\n\n[4] Invariant information bottleneck for domain generalization[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022\n\n[5] Towards unsupervised domain generalization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022\n\n[6] Rethinking minimal sufficient representation in contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16041–16050, 2022\n\n\n[7] Ahuja, Kartik, et al. \"Invariance principle meets information bottleneck for out-of-distribution generalization.\" Advances in Neural Information Processing Systems 34 (2021): 3438-3450.\n\n[8] Du, Yingjun, et al. \"Learning to learn with variational information bottleneck for domain generalization.\" European conference on computer vision. Cham: Springer International Publishing, 2020."}]

Author rebuttal:

Meta-review: The paper introduces a new method for unsupervised domain generalization (UDG), building upon contrastive learning principles but proposing explicit loss components to achieve minimal sufficient semantic disentanglement. The authors presents their method from an information-theoretic perspective, distinguishing between semantic and variation components of the learned representation, and to optimize explicitly for minimal sufficient semantics. All reviewers agree that this is a solid paper with thorough experimental results and sufficiently well-motivated theory. The empirical evaluation is comprehensive, and the theoretical grounding connects well with the proposed training objectives. The ablation studies also demonstrate the usefulness of the loss components. The reviewer comments were also well addressed during the rebuttal. Overall, this is a good contribution for the important problem of semantic disentanglement and unsupervised domain generalization, a nice balance between reasonably-well motivated theory and training loss components that seem to demonstrate good experimental results.
Decision: Accept (poster)