Distributionally Robust Classification for Multi-source Unsupervised Domain Adaptation
Authors: Seonghwi Kim
Sung Ho Jo
Wooseok Ha
Minwoo Chae
Venue: ICLR 2026 Poster (2026)
Keywords: Unsupervised Domain Adaptation
Distributionally Robust Learning
Multi-source Learning
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Unsupervised domain adaptation (UDA) is a statistical learning problem when the distribution of training (source) data is different from that of test (target) data. In this setting, one has access to labeled data only from the source domain and unlabeled data from the target domain. The central objective is to leverage the source data and the unlabeled target data to build models that generalize to the target domain. Despite its potential, existing UDA approaches often struggle in practice, particularly in scenarios where the target domain offers only limited unlabeled data or spurious correlations dominate the source domain. To address these challenges, we propose a novel distributionally robust learning framework that models uncertainty in both the covariate distribution and the conditional label distribution. Our approach is motivated by the multi-source domain adaptation setting but is also directly applicable to the single-source scenario, making it versatile in practice. We develop an efficient learning algorithm that can be seamlessly integrated with existing UDA methods. Extensive experiments under various distribution shift scenarios show that our method consistently outperforms strong baselines, especially when target data are extremely scarce.

Published as a conference paper at ICLR 2026
DISTRIBUTIONALLY ROBUST CLASSIFICATION FOR MULTI-SOURCE UNSUPERVISED DOMAIN ADAPTA- TION
Seonghwi Kim
Sung Ho Jo
Pohang University of Science and Technology kshwi@postech.ac. ac.eechhhccc....................
Pohang University of Science and Technology t wig.psseehppp.ccccccoooo ac.
Wooseok Ha
Minwoo Chae
Korea Advanced Institute of Science and Technology haywse@kaist.ace ac. kr
Pohang University of Science and Technology mchae@postech. ac kr
ABSTRACT
Unsupervised domain adaptation (UDA) is statistical learning problem when the distribution of training (source) data is different from that of test (target) data. this setting, one has access to labeled data only from the source domain and unla- beled data from the target domain. The central objective is to leverage the source data and the unlabeled target data to build models that generalize to the target do- main. Despite its potential, existing UDA approaches often struggle in practice, particularly in scenarios where the target domain offers only limited unlabeled data or spurious correlations dominate the source domain. To address these chal- lenges, we propose a novel distributionally robust learning framework that models uncertainty in both the covariate distribution and the conditional label distribution. Our approach is motivated by the multi-source domain adaptation setting but is also directly applicable to the single-source scenario, making it versatile in prac- tice. We develop an efficient learning algorithm that can be seamlessly integrated with existing UDA methods. Extensive experiments under various distribution shift scenarios show that our method consistently outperforms strong baselines, especially when target data are extremely scarce.
INTRODUCTION
Many supervised learning algorithms rely on the assumption that the training and test data are drawn from the same underlying distribution. However, this assumption is often violated real-world applications due to various factors such as environmental changes, sampling biases, or temporal dy- namics, leading to distribution shifts between training and deployment environments (Beery et al., 2018; Zech et al., 2018; Volk et al., 2019). Such mismatches can result in significant performance degradation, posing a fundamental challenge to achieving reliable generalization in practice (Gulra- jani & Pazz 2020; Koh et al., 2021: Sagawa et al., 2021).
Among various challenges posed by distribution shifts, we focus on unsupervised domain adapta- tion (UDA; Ganin et al. (2016); Long al. (2015; 2018); Ben-David et al. (2010)). In UDA, given labeled data from a source domain and unlabeled data from a target domain, where the distri- related but not necessarily identical. The discrepancy may arise from
butions of the two domains differences in the marginal distribution of the input (Shimodaira, 2000; Sugiyama et al., 2007), the conditional distribution of the output given the input (Jin al., 2023; Cai et al., 2023), both (Ta- chet des Combes et al., 2020; Wu et al., 2025). The central objective is to build a predictive model that generalizes the target domain by effectively transferring knowledge from the source domain.
Two representative approaches in UDA are distribution alignment and pseudo-labeling methods. Distribution alignment reduces the discrepancy between source and target domains, either in the
"Corresponding author.

---

Published as a conference paper at ICLR 2026
data space (Courty al., 2017b) in the feature space (Ganin et al., 2016; Long et al., 2018). In contrast, pseudo-labeling methods refine target predictions using a source-trained model (Amini & Gallinari, 2003; Kumar et al., 2020; Wei et al., 2020), inspired by early work on self-training (Lee, 2013). Despite their popularity, both distribution alignment and pseudo-labeling methods often face practical limitations. Two common scenarios are when the amount of unlabeled target data is limited, or when the source data contains spurious correlations. When target data are scarce, alignment estimates can become unreliable and pseudo-labels may be noisy, which can lead poor generalization (Liang et al., 2020; Xie et al., 2020). When spurious correlations are present- for example, when irrelevant features like background or color are strongly associated with labels models often rely on these shortcuts, which do not transfer to the target domain and ultimately degrade performance (Arjovsky et al., 2019; Zhao et al., 2019; Liu et al., 2021).
In this paper, we propose a new method for UDA, with a particular focus on classification tasks, based on the framework of distributionally robust learning (also known as distributionally robust optimization; DRO'). Our approach is inspired by the maximin effect estimation method developed for regression problems with multi-source data (Meinshausen & Bühlmann, 2015). While originally motivated by the multi-soure setting, our method can also be applied in single-soure scenarios by simulating multiple pseudo-sources sources through random indexing.
The proposed method estimates the conditional distribution of the output given the input for each source domain and constructs an ambiguity set consisting of their mixtures. The mixing weights allowed to vary, enabling the model to adjust how much it trusts each source, while the input distribution is permitted to shift within small Wasserstein ball around the observed target inputs. By explicitly modeling both sources of uncertainty-40 which conditionals rely on and (ii) how the target inputs may vary-our method improves target prediction performance, particularly when target data are scarce or when source domains contain spurious correlations.
We empirically evaluate our approach on widely used benchmarks. including digit classification tasks (MNIST, SVHN, USPS) and spurious correlation datasets (Waterbirds, CelebA, Colored MNIST). Our experiments focus on two key scenarios: (1) standard UDA with limited unlabeled target data and (2) settings where spurious correlations hinder generalization. In both cases, our method substantially outperforms existing approaches in domain adaptation and robust learning.
Our main contributions can be summarized as follows:
We propose novel distributionally robust framework that jointly models uncertainty in the target covariate distribution and the conditional label distribution defined the feature space.
We develop tractable minimax optimization algorithm for the proposed method, which can be seamlessly combined with existing UDA methods.
We provide extensive experiments demonstrating substantial improvements over well-known baselines in two challenging scenarios: target data scarcity and spurious correlations.
In the following, we review key lines of related work, including modern approaches to UDA, DRO- based methods, and other robust methods addressing spurious correlations.
1.1 RELATED WORKS
Unsupervised domain adaptation UDA has been applied in a wide range of areas, including com- puter vision (Hoffman al., 2018), natural language processing (Gururangan al., 2020), and healthcare (Kamnitsas et al., 2017). Popular approaches in UDA learn domain-invariant features by aligning marginal or class-conditional distributions (Long al., 2015; Ganin et al., 2016; Sun & Saenko, 2016: Tzeng et al., 2017: Long et al., 2018), or by separating normalization statistics across domains (Chang et al., 2019). They also regularize decision boundaries (Shu et al.. 2018) source-free variants (Liang et al., 2020; Yang et al., 2021).
and exploit beuuo-labels--nnhhuee Optimal-transpon methods align either marginal input distributions (Courty al., 201 or joint distributions (Courty et al., 2017a; Damodaran et al., 2018). Recent work explores concept-based interpretability, diffusion-basee alignment, coormatioonthhoreee regularization, one-shot settings, and heterogeneous--ooda adaptation (Peng et al., 2024; Xu et al., 2025; Tan et al., 2025; Zhang
While DRO was originally studied in the context of optimization, in this paper we use the term more broadly to refer distributionally robust learning methods under distribution shift.
2

---

Published as a conference paper at ICLR 2026
et al., 2025; Yang et al., 2025). Causal-representation based approaches (Kong et al., 2023; Li et al., 2023) address molti-source settings by identifying invariant latent structures across heterogeneous environments, offering conceptually different direction within UDA. However, these methods can fail in the presence of spurious correlations.
To address this limitation, recent studies attempt to disentangle causal and spurious features using unlabeled target data. Examples include counterfactual consistency, generative interventions, and self-training refinements (Chen et al., 2020; Yue et al., 2023: Wei et al., 2025). Similar approaches have also been explored in multi-source settings (Li et al., 2023; Liu et al.. 2025). However, these methods often rely on strong structural assumptions or are highly sensitive to the quality of initial pseudo-labels, making them fragile under scarce or imbalanced target data.
Distributionally robust optimization Distributionally robust framework has recently gained atten- tion in various areas, including UDA. One line of work in DRO defines ambiguity sets via Wasser- stein balls or f-divergences. which provide robustness to small perturbations of the source distribu- tion (Ben-Tal et al., 2013; Duchi et al., 2021; Awad & Atia, 2023). Another line considers maximin- effect or GroupDRO formulations that address mixture shifts, where the target distribution differs in mixing proportions across subpopulations (Meinshausen & Bühlmann, 2015; Zhan et al., 2024; Wang et al., 2025; Guo, 2024; Sagawa et al., 2019). Maximin-effee approaches are closely related to our setting but have been mostly studied in regression tasks. We provide further discussion point in Section 3. Mixture-shift formulations, particularly GroupDRO. are also relevant when spu- rious correlations arise from changing group proportions. However, GroupDRO typically assumes access to group labels and does not explicitly use unlabeled target data for improving target predic- tion performance. (Guo et al., 2025) studies DRO for classification, but this paper mainly focuses on statistical convergence rates and the construction of bias-corrected estimators. More recently, Jo al. (2026) introduced a distributionally robust learning framework using hierarchical ambiguity sets to specifically address subgroup shifts.
this
Other robust methods under spurious correlation A closely related setting to UDA is domain generalization (DG), where unlike UDA, no unlabeled target data are available during training. Representative DG methods, such as IRM (Arjovsky et al., 2019), V-REx (Krueger et al., 2021), and Fish (Shi et al., 2021), aim to learn representations that are invariant across multiple training environments to mitigate spurious correlations. Beyond domain generalization, other lines of work address spurious correlations more directly. These include approaches such as sample reweighting, biased auxiliary models, and adversarial weighting without group labels (Nam et al., 2020; Lahoti al., 2020; Sohoni et al., 2020; Liu et al., 2021). Additional strategies suppress spurious features through contrastive objectives or mixup-based regularization (Zhang et al., 2022; Jin et al., 2024).
2 PRELIMINARIES
This section introduces basic notations, definitions and some preliminary setups. Let (X, Y) be the random variables of the input and output pair, where X X C Rd and Y E y. For a joint distribution P of (X, Y), its marginal distribution of X and conditional distribution of Y given X are denoted by Px and Pylx, respectively. Also, we use the lower case p for denoting the density of the corresponding distribution denoted by the upper case P. and vice versa. The expectation with respect to the distribution P is denoted by Ep.
Suppose for moment that there is single source domain. Let the population distributions of the source and target domains be denoted by per and pt respectively. For function f parameterized by 0 E θ C IR,, let { (X), Y) be the loss function, which is the cross-entropy loss in most cases. In UDA, the goal is to minimize the target risk Epu [f( (X), Y)] with the labeled source and unlabeled target data. The empirical risk minimizer (ERM) trained on the source data is the most naive baseline in UDA. However, when psc and p&g are different, ERM can exhibit poor predictive performance on the target domain.
An important alternative to ERM estimator under distributional shift is DRO. Let Q denote a class of joint distributions over (X, Y), often referred to as the ambiguity set. In general, the DRO estimator with ambiguity set Q can be obtained by solving the minimax optimization problem
minimize sup (x) AEO QEO
(1)
3

---

Published as a conference paper at ICLR 2026
Constructing a suitable ambiguity set Q is crucial and central component of DRO methods. A common approach is to define Q as a small (pseudo-)metrie ball around the empirical distribution of source data one of its variants. In practice, a key consideration is that the inner supremum must be not only well-defined but also computationally tractable, as it directly affects the overall optimization. For this reason, popular choices for the underlying metric include the Wasserstein distance (Gao et al., 2024; Gao & Kleywegt, 2023; Mohajerin Esfahani & Kuhn, 2018: Blanchet al., 2021) and f-divergences (Duchi & Namkoong, 2021; Namkoong & Duchi, 2016), which allow for efficient approximations or closed-form solutions in many settings.
While ambiguity sets around the source distribution provide robustness minor shifts, they are less suitable for structured shifts. A notable case is subpopulation shift, where the source is mixture of subpopulations and the target has the same subpopulations but different mixing weights. Here, the target can be far from the source in standard distances, making small-ball sets inadequate. A crucial example of an alternative formulation the ambiguity set used in GroupDRO (Sagawa al., 2019), which is specifically designed to address subpopulation shifts.
PROPOSED DISTRIBUTIONALI ROBUST LEARNING METHOD
In this section, we present our multi-source DRO framework together with the corresponding learn- ing algorithm. We assume access to labeled multi-source datasets (*) = {((::-) (k)) ) where D(a) denotes the dataset from the Ath source domain (k = 1,.... K), and an unlabeled target dataset D = )=()) Although the framework is motivated by the multi-source setting, it can also be applied to single-source problems, as discussed in Section 3.1. Section 3.2 provides a high-level formulation of the proposed ambiguity set, highlighting the core intuition behind our method. Sec- tion 3.3 specifies the components the ambiguity set in greater detail, and Section 3.4 presents the detailed learning algorithm.
3.1 MULTI-SOURCE FRAMEWORK FOR SINGLE SOURCE PROBLEMS
Suppose that the source dataset D {(=7" 2 Ji=i come from single domain. If natural grouping variable is available, such as an environment or a demographic attribute, it can be used to partition D into multiple sources. Otherwise, each dataset D can be formed by random sub- sampling with replacement from D This sub-sampling procedure can be justified when the source distribution is a mixture of heterogeneous subpopulations: repeated random sub-sampling with re- placement increases the chance that some sub-samples approximate individual subpopulation. Consequently, when the target distribution consists of the same subpopulations but with different mixing weights, treating the sub- samples distinct sources enables to develop procedures that are robust to changes in mixture proportions. This idea has been used in the maximin effect (Mein- shausen& & Bühlmann, 2015), and our approach builds on the same principle. Throughout the paper, even in the single-source setting, we treat each (*) as if it were drawn from distinct source do- main, thereby framing the problem as a multi-source domain adaptation task. For notational and modeling convenience, we refer to these sub-samples as "sources."
3.2 PROPOSED AMBIGUITY SET: HIGH-LEVEL FORMULATION
YIx denote the estimated conditional distribution based on the kthh source ((), and let P be an estimator of the target input distribution. Details on how these estimators are constructed will be provided in Section 3.3. Given 6, 2 O and 3 E AK- the proposed ambiguity set is defined as
Let ((R)
@ E(EL, Q= (Qx
Qrix = ΣK 3g
P(K) YIXI
BEAK-
} (2)
D1 Qxx P < D2 (8, <
where AK-1 = (S1...., ... 3K) Σ Ba = 1, Bx > 0} denotes the (K 1) dimeennsia simplex, and D2 and D2 are suitable divergence measures. Further details on the choices of D2 and D2 are provided in Section 3.3. The choices of Pyix and D, play crucial role in ensuring the computational tractability of the proposed method.
4

---

Published as a conference paper at ICLR 2026
The core idea behind the ambiguity set in (2) is that the target conditional distribution of the output given the input is expressed as a mixture of conditional distributions estimated from multiple source domains. To this end, we estimate different conditional distributions YII from sub-samples, and their mixtures form sufficiently diverse class capable of containing the target conditional distri- bution. The radii 61 and 62 control the size of the ambiguity set and, consequently, the degree of distributional uncertainty allowed. The mixing vector is centered around reference vector 3, which can encode prior knowledge about the relative importance of each source. If no prior infor- mation available, one can set 3 in proportion to the sample sizes of the source groups. In our experiments, each group was constructed with the same number of samples. Thus, we simply set B to the uniform vector.
In addition to modeling the conditional distribution of Y via mixtures, the ambiguity set (2) also accounts for uncertainty in the input distribution of the target domain through the radius parameter 61. This additional layer of robustness is particularly beneficial when the number N of target sam- ples is limited. In such cases, the empirical target input distribution P may be poor estimate of the true ptg and small perturbations in the input space lead to large variations in model per- formance. By allowing the input distribution to vary within a controlled neighborhood, the method hedges against this sampling variability and prevents overfitting to a small or biased target sample. Such robustness is especially useful in practice, as limited or imbalanced target samples often fail to capture the diversity of the true domain.
The proposed ambiguity set in (2) is inspired by recent work on maximin effect estimation in re- gression settings (Meinshausen & Bühlmann, 2015; Guo, 2024; Wang et al., 2025). These studies consider DRO formulations where the ambiguity set consists of mixtures of conditional distributions from multiple source domains. In contrast to our classification setting, their analysis primarily on regression, using negative explained variance or squared error loss the performance criterion. This choice offers computational benefits; for example, in linear regression it leads to a closed- form solution for the optimization problem (see Theorem 1 in Meinshausen & Bühhlann (2015)). In classification settings, however, developing a computationally feasible DRO algorithm under an ambiguity set of the form (2) presents new challenges. Addressing these challenges and designing an efficient algorithm for this setting constitutes one of our main technical contributions, which we describe in the following subsections.
3.3 DETAILED SPECIFICATION OF THE AMBIGUITY SET
To fully specify the ambiguity set Q in (2), we define the estimators Pts and PYxx as well as the two divergence measures Di and Dr. The reference vector 33 is treated as fixed, while the radii e and 62 serve as tuning parameters that control the size of the ambiguity set.
We simply define P as the empirical measure based on D The construction of each p(k) depends the available data. If sufficiently large datasets are available for each source, can estimate p(R) separately using only the data from that source. However, in many practical settings, the amount of data per source may be limited. Moreover, when the distributions of different sources similar, estimating them separately can be an inefficient use of data. To address this problem, in our experiments, we first train a single classification model using the entire source dataset. From this trained model, extract feature map X + Z by removing the final classification layer. Given the extracted features, we then estimate each p(k) Y|x (or the corresponding conditional density )) using simple linear logistic regression trained independently on each source, treating the softmax outputs practical probability estimates. This approach ensures that the proposed method remains efficient and scalable, even when the number of sources is large.
y
It is worth noting that the proposed method can be combined with existing UDA methods. For in- stance, after obtaining the feature map 2, one may employ well-known approaches such as CDAN or
STAR to construct p(k) Y using as the initial feature map, leading to further performance improve- ments. Further explanation and empirical results of this combination provided in Section 4.2.
For the divergence D2, we simply use the Euclidean distance. For D,. we adopt the infinite-order Wasserstein distance, chosen for its computational tractability; see Section 3.4 for algorithmic de-
5

---

Published as a conference paper at ICLR 2026
Algorithm Procedure for minimizing (3)
Input: step sizes Tjo, M18, Ves initial values ((0) g(0) g())
2:
repeat for = 2,
Sample _ Px and compute 2/ = =(rr"")
Compute y (3(t-1) x. as (4)
Step 1: Update x'
5: Update of using the gradient ascent and projection steps in (5)
D Step 2: Update
6: Update (dt) using the exponentiated gradient ascent and projection steps in (6)
7:
Compute ya (3(()) ,x(t),²²)) .,²))
Step 3: Update
8: Update g using the gradient descent in (7)
9:
until convergence
tails. Recall that the Wasserstein distance of order t E [1, (1, between two probability measures P and Q is defined WiQQ, P) inf, [(f c(xr x') dy) where the infimum is taken over all couplings y of Q and P, and c(.,) is cost function. The infinite-order Wasserstein distance is then given by Wx (Q, P) == lim,, (.QQP.) P).
In our case, we define the cost function as the Euclidean distance between feature representations, that is, c(x, x')= ||>(x) 2(2')ll2. where denotes the feature map introduced carlier. This choice often more effective than defining the cost function directly in the raw input space, particularly in high-dimenssonal settings (Ganin al., 2016; Zeiler & Fergus, 2014; Krizhevsky et al., 2017).
3.4 LEARNING ALGORITHM
The proposed DRO estimator is obtained by solving the general DRO problem (1) with the ambigu- ity set defined in (2). In this subsection, we describe the detailed learning algorithm. We assume that fd (X) depends X only through the fixed representation 2(X) as defined in Section 3.3. Thus, we denote the model by (=())) in the remainder of this section. For notational convenience, often treat the probability mass function pyix(- X) as a probability vector. Before present- ing the algorithm, we state the following proposition, which facilitates computationally efficient implementation.
Proposition 3.1 (Tractable surrogate reformulation). Suppose that the feature map 2 X + Z and the estimators ((() YIX are given. Let the divergence measures D and D2 be defined as in Section 3.3. For any 61, f2 > O and a fixed 0, the maximization objective the DRO problem (1) with the ambiguity set Q defined in (2), i.e., supeee Eole (f (X), Y)] is upper-bounded by the following surrogate objective
sup BEAx-1- [ Se2
Epi
sup {(
(3)
[X)
where
K (k) y (8.z)) = - Be ( for x e x
(4)
the soft pseudo-label vector defined as convex combination of source conditionals.
This reformulation admits an efficiently computable algorithm that leverages adversarial feature perturbation z' and soft pseudo-label y². The detailed rationale behind the approximation (3) is provided in Appendix A.1.
We now outline the procedure for minimizing (3). At high level, the algorithm alternates between updating z', 3, and / using minimax optimization scheme implemented via stochastic gradient descent, as summarized in Algorithm 1. The detailed update nules are described below.

---

Published as a conference paper at ICLR 2026
1. Update of 2'. Suppose that 0 and are fixed. Given sample z; from PX let 2, = =)x,"))
To maximize the mapping z' + (f2(2)), yº (3. we perform projected gradient ascent starting from 2/
2 - _ + V. (3)
(5)
2 + 41 1=: (z/),
where Vz denotes the gradient of + (z) 1f (8,2)))) me the step size, and IA(()) is the Euclidean projection onto the set A.
Update of f. Given and 2/., we update the mixture weights 3 via a projected exponentiated gradient ascent (Sagawa et al., 2019):
BR exp
noe (J2 (1)
Pyxx)
84
for k: I...., K,
Σj= 11=1 Bg exp nal / (=1), (=)))19xx1 (
(6)
8 -- 18.10--11 (8).
where 7/8 is the step size. See Appendix A.2 for further details on this update.
3. Update of 0. Given of and B3 we update using a stochastic gradient descent:
0 - 0 no Vol (J2 y (),
where V denotes the gradient of the map 0 + f(f2 (2(), (3,
Note that the role of is form adversarial mixture these estimated conditionals. Specifi- cally, the update rule in (6) assigns larger weights to conditional estimators that induce higher loss under the current classifier f The model parameters 0 are then updated in (7) to minimize the sulting adversarial objective. By iterating these updates, the classifier becomes robust to conditional uncertainty and potential mixture shifts. A detailed analysis of the stability of the learning algorithm is provided in Appendix A.3.
4 EXPERIMENTS
In this section, we evaluate the proposed method in two experimental settings using widely adopted benchmark datasets. The first experiment considers digit recognition tasks across three domains- MNIST. SVHN, and with particular emphasis on scenarios where the amount of aarget-domain data is limited. The second experiment examines the robustness of our method in the presence of spurious correlations, using popular benchmarks such Waterbirds, CelebA, and Colored MNIST, where distribution shifts are induced by non-causal attributes. In the first experiment, one of MNIST, SVHN, or USPS is used as the single source domain, and one of the remaining two serves as the target domain, resulting in three source-target pairs. In the second experiment, each benchmark dataset consists of single source domain and single target domain as described below. Thus, all experiments are conducted under the single-source UDA setting. See Appendix A.5 for more detailed descriptions of the datasets.
compares the proposed method with existing UDA approaches,
MNIST, SVHN and USPS (LeCun al., 2002: Netzer al., 2011; Hull, 2002): Three bench- mark datasets for handwritten digit recognition from different domains. MNIST contains grayscale images of digits from to SVHN consists of color images of house numbers obtained from Google Street View, and USPS includes grayscale images of handwritten digits collected from postal envelopes. For each dataset, we randomly sampled 102 and 10 unlabeled target samples per class.
Waterbirds, CelebA, and CMNIST (Sagawa et al. (2019); Liu et al. (2015); Arjovsky et al. (2019)): Three benchmark datasets widely used to study spurious correlations. Each dataset is di- vided into four groups, where spurious attributes (c.g., background in Waterbirds, gender in CelebA, or color in CMNIST) are correlated with labels. all cases, three majority groups are combined to form the single-souree domain, and the remaining minority group serves as the target domain. For Waterbirds, CelebA, and CMNIST, the unlabeled target sample sizes N are 56, 1,387, and 2,998, respectively.
7

---

Published as a conference paper at ICLR 2026
SVHN MNIST
MNIST USPS
USPS - MNIST
Method
10a
10
10a
10
10a
10
ERM (Sre-only)
59.61 1.8
63.41 1.4
60.41
DANN
66.0 4.9
61. 01.2+
82.0 3.9
74.3+
74.8 6.7
51.1+
CDAN
63.4+1.8
56.9
80.8
12.0+1.8
58.3+4.6
54.8+5.99
MK-MMD
50.013.0
18.3
63.311.1
41.3155.5
37.613.2
32.1133.9
ATDOC
83.3119.1
59.1
91.1
73.614.2
92.611.3
87.04:3.6
STAR
76.4+1.5
66.8+0.9
90.3+1.9
81.377.66
94.5=0077
85.2+:2.7
CORAL
75.4+2.7
03.6=0.9
90.4+0.7
85.4+0.5
75.9=1.9
64.7+33.5
MCD
79.111.0
61.3100.7
89.311.6
81.512.3
96.111.6
85.914.0
Ours (ERM)
92.011.6
87.010.9
92.1
87.113.6
90.312.2
86.412.55
Ours (CDAN)
92.5+2.3
87.0=2.0
93.5+1.3
87.8+1.5
91.5#1.9
87.0+11.
Ours (STAR)
94.4+1.7
91.3+11.
95. 65.6+110
91.2=14
97 34.88
93 03.0=2.8
*Ours (STAR)
92.912.1
85.111.2
93.611.5
90.5..0.6
96.611.4
87.6133.2
Table I: Comparison of test accuracies across different target sample sizes for three domain adapta- tion benchmarks: SVHN, MNIST, and USPS. Each block reports results under two target data sizes: 10a and 10 unlabeled samples class. Here, Ours(-) denotes our methods built upon different base classifiers, and *Ours(.) denotes the versions tuned via LODO-CV, Boldface indicates best performance.
4.1 EXPERIMENTAL SETUP
Following Meinshausen Bühlmann (2015), in all our experiments we set K and draw each sub-sample of size N N=//5 with replacement, which we denote by D {[<x,a) M, (k) J = I,..., 10. A detailed analysis regarding choice K is provided Appendix A.4. We then the entire source dataset learn the feature map 2. For constructing base classi- fier B(k)
Ji=
Y(X' we use three approaches: training simple linear logistic regression model (ERM) on
{<((x)))
(()(())) Mt (*) ) and training CDAN and STAR, both initialized with the learned feature map For the backbone, we use two different models. In the first setting, we adopt a deep neural net- work, following the architecture in (Ganin et al., 2016). In the second setting, we employ ResNet-50 following (Sagawa al., 2019).
For hyperparameter selection, we adopt validation-basee selection strategy to ensure fair com- parison across methods. Specifically, we assume the availability of small labeled target vali- dation set containing 10 samples per class, which is used to perform grid search over e {0, 0.2, 0.4, 0.6, 0.8, 13 and 62 E {0, 0,,, 0.4, 0.6, 1}. This setting is consistent with those used in prior domain adaptation studies (Yue al., 2023; Courty al., 2017b; Saito et al., 2018), and the same selection procedure is applied to all competing methods. This choice allows us to report target-domain performance without being confounded by differences in hyperparameter selection strategies. In addition, we report the performance of our method using maave-one-domaine validation (LODO-CV) (Gulrajani & Lopez-Paz, 2020), which does rely labeled target valida- tion data and provides a complementary evaluation. Results obtained under this setting are marked with asterisk (*Ours(-)). We compare our method with several baselines commonly used in do- main adaptation and robust learning: DANN (Ganin et al., 2016), CDAN (Long et al., 2018) 8), MK- MMD (Long et al., 2015), CORAL (Sun & Saenko, 2016), MCD (Saito et al., 2018), ATDOC (Tang et al., 2020), STAR (Lu et al., 2020), GroupDRO, ICON (Yue et al., 2023), DRUDA (Wang & Wang, 2024), and PDE (Deng et al., 2023).
cross
4.2 RESULTS ON DIGIT RECOGNITION TASKS
In the first experiment, we consider two target sample sizes: 102 and 10 per class. This setup al- lows us to evaluate whether our method can maintain robust performance even when only a very small number target samples are available for training. Table summarizes the results for three source- target domain pairs. Our method (Ours(-)) consistently achieves the best performance across all tasks and target sample sizes. Notably, the performance can further improved by combin- ing approach with existing UDA methods such as CDAN and STAR. For example, when
8

---

Published as a conference paper at ICLR 2026
Method
Group label
Waterbirds Test Acc
CelebA Test Ace
CMNIST
Test Acc
ERM (Sre-only)
48. 48.4
35.510066
0.9
DANN
X
35.
23.5+:2.1
0.9+1.8
CDAN
X
46.211.8
24.611.5
1.2.10.4
MK-MMD
X
45. 15.1
27.7+2.9
2.8+1.3
ATDOC
X
1..
31.8
3.140.9
STAR
X
49.8:5.6
24.412244
2.242.7
CORAL
X
50.9+2.9
31.7+1.9
1.7+0.4
MCD
X
59.013.1
30.712.5
1.9.199
DRST
X
37.1=6.0
29.5+:466
1.040.7
ICON
X
54.241.
31.1+2.7
4.443.2
GroupDRO
61.4+2.7
63 63.0+2.6
3.4+16
GroupDRO (with Tgt)
90.6100.2
1.3
73.1
PDE
57.1
35.0+5.5
1.341.
Ours (ERM)
87.3+2.1
85.0=4.1
7.5
*Ours (ERM)
83.311.7
76.013.8
4.910.7
Table 2: Comparison of test accuracies across Waterbirds, CelebA, and CMNIST. ERM (Src-only) and GroupDRO are trained solely on source domain data, without using any target domain samples. GroupDRO (with Tgt) denotes the setting where both source labeled data and target labeled data are available during training. Boldface indicates the best performance except for models that use labeled target data for training.
ing CDAN as the base classifier, our method improves upon classical CDAN by +29.1% on the task. Although *Ours(-) shows moderate drop compared to Ours(-), it remains consistently competitive and still outperforms baselines that rely on labeled target data for tuning.
These improvements highlight the twofold strength of our framework. First, the ambiguity set plicitly accounts for uncertainty in the target input distribution through the <1-radius. This design is particularly beneficial under extreme data scarcity, where the empirical distribution itg may be poor approximation of the true PVg By allowing controlled perturbations, the model hedges against sampling variability and mitigates overfitting to small or biased target samples. Second, incorporat- ing UDA methods such CDAN or STAR enables the construction of conditional estimators that generalize better to the target domain. The formulation of the ambiguity set as a mixture these refined conditionals produces tion. Consequently, the final model trained under this set achieves better generalization to target than mixtures based solely on ERM conditionals.
uncertainty set more closely aligned with the true target distribu-
4.3 RESULTS ON SPURIOUS CORRELATION BENCHMARKS
Table 2 reports the test accuracies on the target domain across the three spurious-correlati bench- marks. Overall, most baseline methods show limited generalization to the target domain, with some performing even worse than standard ERM. This underscores the difficulty of these benchmarks, where spurious correlations and target-data scarcity occur simultancously. classical UDA methods such as DANN, CDAN, and MK-MMD suffer from limited generalization when spurious correlations exist across domains, they mainly match marginal input distributions without preserving semantic invariance (Johansson et al., 2019; Zhao et al., 2019). This often aligns spurious attributes (e.g.. background, gender, or color) instead of causally related features, which can hurt generalization to the target domain. Pseudo-labeling methods such as STAR and ATDOC are also vulnerable, since spurious correlations can induce inaccurate pseudo-labels that propagate errors during training.
is well-known that
In contrast, our method does not rely on reducing the distance between marginal input distribu- tions. Instead, it explicitly considers a set of plausible target distributions through distributional ambiguity set and optimizes for the worst-case target risk within this set. By accounting for distribu- tional variations that may arise from spurious attributes, this formulation guides the model to focus on predictive features that are invariant across the plausible target distributions, thereby enhancing robustness to spurious correlations. As result, it significantly outperforms existing baselines, par-

---

Published as a conference paper at ICLR 2026
ticularly in scenarios where spurious correlations dominate and unlabeled target data is extremely limited. Compared to ERM, it improves target domain accuracy by +38.9% on Waterbirds, +49.5%% on CelebA, and +6.6% on CMNIST. Similar to the first experiment, *Ours(-) shows slight perfor- mance reduction compared to Ours(.): ) yet still achieves stronger performance than all baselines.
4.4
SENSITIVITY ANALYSIS OF THE HYPERPARAMETERS 61 AND E
Heatmap Test Accuracy scross and
Accuracy across Ej
79.7
08
68.2
96.4
76.1
71.2
67.5
62.4
0.2
66.3
89.9
82.8
81.2
51.5
15.7
03
0.2
67.7
85.5
84.5
05.0
05.6
61.3
47 0.4
71.0
90.5
83.3
70.7
50.9
24.2
0.4
by.77
86.5
04.
05.5
06.0
60.9
0.6
75.8
91.8
87.0
78.5
70.3
58.8
3.6
66.3
86.2
2
69.4
65.6
60.7
0.9
75.6
92.1
85.2
77.7
70.0
58.2
0.9
66.3
86.1
82.7
69.0
65.4
60.6
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.0
1.0
(a) MNIST -> USPS (10?)
(b) USPS
MNIST (10)
Figure 1: Heatmaps of average test accuracy across (c1,62).
In this subsection, we analyze how the hyperparameters 6 and E2 affect model performance through detailed sensitivity study. Figures report heatmaps of the average test accuracy in different combinations of 6 and 62- Figure ((a) corresponds to the MNIST-USSPS task with 10 unlabeled target samples per class, while Figure 1(b) shows results for the USPS +MNIST task with only 10 unlabeled target samples per class.
In Figure 1, the baseline setting (e1, (0, 0) performs noticeably worse than configurations that introduce moderate uncertainty. Increasing either or e2 generally improves accuracy, showing that controlled covariate or conditional perturbations enhance generalization. When the target sample size is moderately large, both hyperparameters exhibit a broad plateau of strong performance. As shown in Figure 1(a), accuracy remains high for 6 E {0.2, 0.4] and (2 2 0.2, demonstrating that the method is insensitive to small variations in these hyperparameters.
Under extreme target-data scarcity, the effect of 61 becomes more prominent. Figure 1(b) shows that accuracy changes sharply along the e axis, especially for 2 0.6, while remaining relatively stable across values of é2. This behavior reflects the setting in which 2 is estimated from very few samples, making robustness to covariate-level variability particularly important.
The appropriate value of depends on the scale of the learned embedding, choice applies across models. In practice, accuracy remains stable over a broad range of é1 values (Figure 1), suggesting that approximate heuristic selections work well in most cases. In contrast, <2 can be set to a relatively large value (e.g., oo) when no prior knowledge is available, as conditional mixing does not induce instability.
single universal
CONCLUSION
We presented a DRO framework for UDA that jointly models uncertainty in both the target covariate distribution and the conditional label distribution. By formulating an ambiguity set over mixtures of source conditionals and allowing controlled perturbations in target inputs, our method directly addresses two challenges that have gained increasing attention in recent research: the scarcity of unlabeled target data and the presence of spurious correlations in the source domain. Extensive experiments on digit recognition and spurious correlation benchmarks demonstrated consistent and substantial performance gains over strong baselines, highlighting the method's 's robustness and effec- tiveness. Although our experiments focus on vision benchmarks, extending the framework to other modalities such NL.P or time-series data is natural next step.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00240861. RS-2025-24523569, RS-2022-NR068758)
Korea Institute for Advancement of Technology (KIAT) grant funded by the Korea Government (MOTIE) 25-20224-0040009,, 2024 HRD Program for Industrial Innovation), and Institute of Infor- mation Communications Technology Planning Evaluation(IITP> -Gooball Data-X Leader HRD program grant funded by the Korea government (MSIT) (TTP-20224RS--222-0044111
REFERENCES
M Amini and Patrick Gallinari. Semi-supervised learning with an explicit label-error model for misclassified data. Proc. International Joint Conferences on Artificial Intelligence, pp. 555- 560, 2003.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. ArXiv:1907.02899 2019.
Akram S Awad and George K Atia. Distributionally robust domain adaptation. In Proc. International Workshop on Machine Learning for Signal Processing, pp. 1-6, 2023,
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition terra incognita. In Proc. European Conference on Computer Vision, pp. 456-473, 2018.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Machine Learning, 79:151-175, 2010.
Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. Management Sci- ence, 59(2):341 2013.
Jose Blanchet, Karthyek Murthy, and Viet Anh Nguyen. Statistical analysis of Wasserstein distribu- tionally robust estimators. In Tutorials in Operations Research: Emerging optimization methods and modeling techniques with applications, pp. 227-254, 2021.
Tiffany Tianhui Cai, Hongseok Namkoong, and Steve Yadlowsky. Diagnosing model performance under distribution shift. ArXiv:2303.0m 02001, 2023.
Woong-Gr Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. Domain -seaiffc batch normalization for unsupervised domain adaptation. In Proc. Computer Vision and Pattern Recognition, pp. 7354-7362, 2019.
Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. Self-training avoids using spurious features under domain shift. In Proc. Advances in Neural Information Processing Systems, vol- ume 33, pp. 21061-21071 2020.
Nicolas Courty, Rémi Flamary, Amaury Habrard, and Alain Rakotomamonjy. Joint distribution optimal transportation for domain adaptation. ArXiv:1705.08845 2017a.
Nicolas Courty, Rémi Flamary, Devis Tuia, and Alain Rakotomamoniy. Optimal transport for do-
main adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(9):1853- 1865, 2017b.
Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proc. European Conference on Computer Vision, pp. 447-463, 2018.
Yihe Deng, Yu Yang, Baharan Mirzasoleiman, and Quanquan Gu. Robust learning with progressive data expansion against spurious correlation. Proc. Advances in Neural Information Processing Systems, 36: 1900-140, 2023.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distribu- tionally robust optimization. The Annals of Statistics, 1973):1378-144) 2021.
11

---

Published as a conference paper at ICLR 2026
John C Duchi, Peter W Glynn, and Hongseok Namkoong. Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research, 46(3):946-900 999)99999, 2021.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17(59):1-35, 2016.
Rui Gao and Anton Kleywegt. Distributionally robust stochastic optimization with Wasserstein distance. Mathematics of Operations Research, 48(2):603-655, 2023.
Rui Gao, Xi Chen, and Anton Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research, 72(3)111771111,, 2024.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. ArXiv:2007.01434, 2020.
Zijian Guo. Statistical inference for maximin effects: Identifying stable associations across multiple studies. Journal of the American Statistical Association, 119(547):1968-19)) 2024.
Zijian Guo, Zhenyu Wang. Yifan Hu, and Francis Bach. Statistical inference for conditional group distributionally robust optimization with cross-entropy loss. ArXiv:2507.09905. 2025.
Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. Don't 't stop pretraining: Adapt language models to domains and tasks. ArXiv:2004.0 10964, 2020.
Judy Hoffman, Eric Tzeng, Taesung Park, Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In Proc. Interna- tional Conference on Machine Learning, pp. 1989-1998, 2018.
Jonathan J. Hull. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):550-554, 2002.
Xin Jin, Hongyu Zhu, Siyuan Li, Zedong Wang. Zicheng Liu, Juanxi Tian, Chang Yu, Huafeng Qin, and Stan Z Li. A survey mixup augmentations and beyond. ArXhv:2409005202. 2024.
Ying Jin, Kevin Guo, and Dominik Rothenhäusler. Diagnosing the role of observable distribution shift in scientific replications. ArXiv:2309.01056 2023.
Sung Ho Jo, Seonghwi Kim, and Minwoo Chae. Mitigating spurious correlation via distributionally robust learning with hierarchical ambiguity sets. In Proc. International Conference on Learning Representations, 2026.
Fredrik D Johansson, David Sontag, and Rajesh Ranganath. Support and invertibility in domain- invariant representations. In Proc. Artificial Intelligence and Statistics, 2019.
Konstantinos Kamnitsas, Christian Baumgartner, Christian Ledig, Virginia Newcombe, Joanna Simpson, Andrew Kane, David Menon, Aditya Nori, Antonio Criminisi, Daniel Rueckert, et al. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. Proc. International Conference Information Processing in Medical Imaging, pp. 597-609. Springer, 2017.
Jyrki Kivinen and Manfred K Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1):1-63, 1997.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, al. Wilds: A benchmark of in-the-wild distribution shifts. In Proc. International Conference on Machine Learning. pp. 5637-5664. PMLR, 2021.
Lingjing Kong, Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akin- wande, and Kun Zhang. Partial identifiability for domain adaptation. ArXiv:2306.065 10, 2023.
12

---

Published as a conference paper at ICLR 2026
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Communications of the ACM, 60(6):84 90, 2017.
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang. Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrap- olation (rex). In Proc. International Conference on Machine Learning, pp. 5815-5826. PMLR, 2021.
Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. In Proc. International Conference on Machine Learning, pp. 5468-5479, 2020.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed Chi. Fairness without demographics through adversarially reweighted learning, In Proc. Advances in Neural Information Processing Systems, volume 33, pp. 728-77 04307,0,20 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11))2278) 2324, 2002.
Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML Workshop Challenges in Representation Learning, pp. 896, 2013.
Zijian Li, Ruichu Cai, Guangyi Chen, Boyang Sun, Zhifeng Hao, and Kun Zhang. Subspace identi- fication for multi-source domain adaptation. In Proc. Advances in Neural Information Processing Systems, volume 36, pp. 34504-34518, 2023.
Jian Liang, Dapeng Hu, and Jiashi Feng- Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In Proc. International Conference on Machine Learning, pp. 6028-6039, 2020.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Proc. International Conference Machine Learning, pp. 6781-6792, 2021.
Yuhang Liu, Zhen Zhang, Dong Gong, Mingming Gong, Biwei Huang, Anton van den Hengel, Kun Zhang, and Javen Qinfeng Shi. Latent covariate shift: Unlocking partial identifiability for multi-source domain adaptation. Transactions Machine Learning Research, 2025.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proc. International Conference on Computer Vision, pp. 3730-3738, 2015.
Mingsheng Long, Yue Cao. Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In Proc. International Conference on Machine Learning, pp- 97-105, 2015.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. In Proc. Advances in Neural Information Processing Systems, volume 31, pp- 1640-1650, 2018
Zhihe Lu, Yongxin Yang, Xiatian Zhu, Cong Liu, Yi-Zhe Song, and Tao Xiang. Stochastic classifiers for unsupervised domain adaptation. In Proc. Computer Vision and Pattern Recognition, pp- 9111-9120, 2020.
Nicolai Meinshausen and Peter Bühlmann. Maximin effects in inhomogeneous large-scale data. The Annals of Statistics, 4314):1801 2015.
Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization us-
ing the Wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming. 171(1):115-16) 2018.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De- biasing classifier from biased classifier. In Proc. Advances in Neural Information Processing Systems, volume 33, pp. 00673-20684, 2020.
13

---

Published as a conference paper at ICLR 2026
Hongseok Namkoong and John C Duchi. Stochastic gradient methods for distributionally robust optimization with f-divergences. In Proc. Advances in Neural Information Processing Systems, volume 29, pp. 2208-2216, 2016.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Pp. 2011.
Duo Peng, Qiuhong Ke, ArulMurugan Ambikapathi, Yasin Yazici, Yinjie Lei, and Jun Liu. Unsuper- vised domain adaptation via domain -ddain- diffusion. IEEE Transactions on Image Processing, 2024.
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. ArXiv:1911.08731, 2019.
Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, et al. Extending the wilds benchmark for unsupervised adaptation. ArXiv:2112.05090 2021.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. Proc. Computer Vision and Pattern Recognition, pp. 3722-3732, 2018.
Yuge Shi, Jeffrey Seely, Philip HS Torr, Narayanaswamy Siddharth, Awni Hannun. Nicolas Usunier, and Gabriel Synnaeve. Gradient matching for domain generalization. ArXiv:2104.09337, 2021.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-
likelihood function. Journal of Statistical Planning and Inference, 90(2):227-244, 2000.
Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. A Dirt-t approach to unsupervised domain adaptation. WwXi::/802.0875, 2018.
Nimit Sohoni, Jared Dunnmon, Geoffrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse-grained classification problems. Proc. Advances in Neural Information Processing Systems, volume 33, pp. 19339-19352, 2020.
Matthew Staib and Stefanie Jegelka. Distributionally robust deep learning generalization of adversarial training. In NIPS Workshop on Machine Learning and Computer Security, volume 3, pp. 4, 2017.
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Proc. European Conference on Computer Vision, pp. 443-450, 2016.
Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoffrey Gordon. Domain adaptation with conditional distribution matching and generalized label shift. In Proc. Advances in Neural Information Processing Systems, volume 33, pp. 19276-19289, 2020.
Lianghao Tan, Zhuo Peng, Yongjia Song, Xiaoyi Liu, Huangqi Jiang, Shubing Liu, Weixi Wu, and Zhiyuan Xiang. Unsupervised domain adaptation method based relative entropy regularization and measure propagation. Entropy, 27(4):426, 2025.
Hui Tang, Ke Chen, and Kui Jia. Unsupervised domain adaptation via structurally regularized deep clustering. Proc. Computer Vision and Pattern Recognition, pp. 8725-8735, 2020.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proc. Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.
Georg Volk, Stefan Müller, Alexander Von Bernuth, Dennis Hospach, and Oliver Bringmann. To- wards robust enn- based object detection through augmentation with synthetic rain variations. In Proc. IEEE Intelligent Transportation Systems Conference, pp. 285-292. IEFF, 2019.
14

---

Published as a conference paper at ICLR 2026
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-uesd birds-200-2011 dataset. Technical Report CNS- TR-2011000-,,,, California Institute of Technology, 2011.
Yibin Wang and Haifeng Wang. Distributionally robust unsupervised domain adaptation. Journal of Computational and Applied Mathematics, 436:1 15369, 2024.
Zhenyu Wang, Peter Bühlmann, and Zijian Guo. Distributionally robust machine learning with multi-source data. The Annals of Statistics, 2025.
Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. Theoretical analysis of self-training with deep networks on unlabeled data. WwXiv:2010.03621 2020.
Xing Wei, Wenhao Jiang, Fan Yang, Chong Zhao, Yang Lu, Benhong Zhang, and Xiang Bi. Unsu- pervised domain adaptation via causal-contrastive learning. The Journal of Supercomputing, 81 (5):719, 2025,
Keru Wu, Yuansi Chen, Wooseok Ha, and Bin Yu. Prominent roles of conditionally invariant com- ponents in domain adaptation: Theory and algorithms. Journal of Machine Learning Research, 26(110):1 2025.
Qizhe Xie, Minh-Thang Thang Luong, Eduard Hovy, and Quoe V Le. Self-training with noisy student improves imagenet classification. In Proc. Computer Vision and Pattern Recognition, pp. 10687- 10698, 2020.
Xinyue Xu, Yueying Hu, Hui Tang, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. Concept-based unsupervised domain adaptation. In Proc. International Conference Machine Learning, 2025.
Jiawen Yang, Shuhao Chen, Yucong Duan, Ke Tang, and Yu Zhang. Heterogencous-modle unsuper- vised domain adaptation via latent space bridging. ArXiv:2506. 20t.:25066/99711, 2025.
Shiqi Yang, Joost Van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neigh- borhood structure for source-free domain adaptation. In Proc. Advances in Neural Information Processing Systems, volume 34, pp. 29393-29405, 2021.
Zhongqi Yue, Qianru Sun, and Hanwang Zhang. Make the in uda matter: Invariant consistency learning for unsupervised domain adaptation. In Proc. Advances in Neural Information Process- ing Systems, volume 36, pp. 26991-27004, 2023.
John R Zech, Marcus A Badgeley, Manway Liu, Anthony В Costa, Joseph J Titano, and Eric Karl Oermann. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross cross-nectional study. PLoS medicine, 15(1 (el1))ee00288 2018.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Proc. 2014.
European Conference on Computer Vision, pp.
Keyao Zhan, Xin Xiong, Zijian Guo, Tianxi Cai, and Molei Liu. Domain adaptation optimized for robustness in mixture populations. ArXiv:2407.200775 2024.
Michael Zhang, Nimit S Sohoni, Hongyang Zhang, Chelsea Finn, and Christopher Ré. Contect-n-contrast: A contrastive approach for improving robustness to spurious correlations.
ArXiv:2203.01517, 2022.
Yue Zhang, Mingyue Bin, Yuyang Zhang, Zhongyuan Wang. Zhen Han, and Chao Liang. Link- based contrastive learning for one-shot unsupervised domain adaptation. In Proc. Computer Vi- sion and Patter Recognition, pp. 4916-4926 2025.
Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In Proc. International Conference Machine Learning, 2019.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. In Proc. Pattern Analysis and Machine Intelligence, volume 40, pp. 1452-1464, 2017.
Chunting Zhou, Daniel Levy, Xian Li, Marjan Ghazvininejad, and Graham Neubig, Distributionally robust multilingual machine translation. 1rXiv:2109004000 2021.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A. PROOF OF PROPOSITION 3.1
In this subsection, we provide the detailed rationale of the tractable reformulation stated in Proposi- tion 3.1. With the ambiguity set Q defined in (2), the inner maximization in the DRO objective can be expressed as
sup Role (X), QEQ
sup 3xx QEe
Rrix le (X),
(8)
sup -
sup Wo (ox
EQN E rix
(x),
(9)
|| S
where
K
2 = Σ
3x
p(ke v|x
We recall the following lemma from Staib & Jegelka (2017), which will be useful in our reformula- tion.
Lemma A.1.1 (Staib & Jegelka (2017), Proposition 3.1). Suppose that the cost function c(.,) used define the Wasserstein distance is a metric on X. Let P be Borel probability measure on X, and let : X + R be measurable function. Then, for any e 2 0,
sup W
EQ [f(X)] = Ep
sup f(x') (xEB.(X)
where Be(x)= = fz' E X c(æ, x') S } and EEp denotes the expectation under X P.
By Lemma A.1.1, have
Eqx B [e(F((),))]]
sup W (Qx P Sc
=Epo
sup (=)==))=(X)11255
E O [e(()())))]]
Epg
sup
E O ((f((((
====))==(X]ll0s
Bag [((2(())))]]
S E8 pgs
sup
Therefore, (8) is upper-boundee by
oF [((2(1))))]
sup BEAR-1:
Epss
sup [X0]aa
| 18-8 25
By treating the probability mass function P (: X) as a probability vector, we define the soft pseudo-label vector
0 (B,
3i (k) Prlxx (: 2r).
(10)
k=1
Recall that the cross-entropy loss is defined by
(F2(2)), = > 1| = v) log 7 (y/ 1z'),
DEy
16

---

Published as a conference paper at ICLR 2026
where (- 2') denotes the predicted class probability vector obtained by applying the softmax
function to 12 (=') and [[[] is the indicator function. Therefore, we have
og x-=
[e (f%(()))
=
o d
Σ u| = log ( 2) 2
yEy
K
(11)
Σ
3R
Pyix(u1 log 22
(K)
yEY
= l (J2 (='), 8f (3,
Here, we slightly abuse the notation for 2( interpreting (f2(2'), (3,x)) as the cross-entropy between the two probability vectors (- 2/) and of (3, .).
Combining the above, obtain the following surrogate objective for (8):
sup BEAX-1:
Eps
sup [X0'=a
8 ((
(12)
18-82
A.1.1 ON THE TIGHTNESS OF THE RELAXATION GAP OF THE SURROGATE LOSS
Our surrogate objective (12) can be strictly smaller than the original objective. The gap arises from the inequality
sup =(1)=())=(((]I><a
S
sup =/:l| (X))==
BOg [((((2))
This inequality can be strict when the feature map z(-) does not cover the entire e ball. In many practical settings, however, embedding distributions produced by deep networks modeled under the assumption that the pushforward distribution of X through 2(-) admits a positive Lebesgue density on, or in a neighborhood of, its effective support.
Raw images often lie near a low-dimensional manifold in the input space and may not possess Lebesgue density. After several nonlinear transformations, internal representations typically exhibit different geometric behavior and are commonly treated as "thickened" manifolds with nonzero vol- ume in the ambient space. The widespread use of the Fréchet Inception Distance (FID) illustrates this modeling perspective, as the metric implicitly assumes non-degenerate covariance structure for deep embeddings.
Under the above modeling assumption, the image of z(-) generally overlaps substantially with the ball,, making the relaxation introduced after Lemma A.1.1 nearly tight and the resulting upper bound a close approximation of the original hierarchical objective.
A.2 EXPONENTIATED GRADIENT ASCENT FOR UPDATING /
We first review the exponentiated gradient method briefly and then describe its application to our 3-update procedure.
Exponentiated gradient ascent method The exponentiated gradient ascent method can be inter- preted as mirror ascent step over the simplex, obtained by maximizing the first-order approxima- tion of the objective with KL divergence regularization. Specifically, let L(3) be differentiable objective function. Then, the exponentiated gradient ascent update from (*)) is given by
{ (((30) + V)L(3 (B- ((--1) arg max BEAX
1
DKL 3
(31300
where Vg denotes the gradient of the mapping B + L(3), and ns > 0 is the step size. Since the terms involving ((t) are constant with respect to 33 this is equivalent to
g(++1) = arg max BEAR-1
VAL(B())
DKL. (xi((1) (81,310)
n
17

---

Published as a conference paper at ICLR 2026
By the KKT optimality conditions, the maximization above admits the following closed-form solu- tion:
na V [(((0))) Σラ= K 3/ exp 7s [ (3L((())1
8 exp
((++1) where [A]; denotes i-th component of A. See Kivinen & Warmuth (1997) for details.
k = 1, K,
(13)
Detailed procedure for updating / We now revisit our optimization problem in (3). Since the constraint || Bl/2 I eg is handled by projection after the ascent step, we focus only on updating over the simplex Ak-1- Suppose that x and 2/ are given. Then, we need to solve
[e(2 y (B,x
sup ØEAK-1
This problem can be expressed as:
sup ((f((=)) 1 (B, = sup
E
Qa
[((S2(2)))))
yEAx
BEAR
K
sup
3x E Ep()
(e(S(())) Y)]
DEAK k=
rIx-176
Σ
Ba el (f2(2), plix(( (1225))
sup
BEAK-1 Rm
where this expression follows from (4) and (11).
Setting
((B) Σ 3x (f((t)) (-
and applying the closed-form solution in (13), obtain the following update rule:
((0) exp 1/3 e ( Prix 12
g(++1)
h I=.,..,,K K.
ΣK exp na (=),, py) x (x
A.3
EMPIRICAL ANALYSIS OF THE STABILITY OF LEARNING ALGORITIIM
Trajectory of L2 norm of 9-gred ent
Trajectory of L2 norm of O-gradient
Lowinng ritto rutow0
Leaiiig mitedd Loantiy ralue0.00
L2 morm of graddeent
L2 horm of grredientt
22
keration
Iteration
(a) Trajectory Le norm 8-gradient
(b) Trajectory Le norm 0-gradient
Figure Gradient behavior of and under joint optimization.
In this subsection, we analyze the stability our learning algorithm presented in Section 3.4. To assess this stability, we further examine the gradient behavior of the joint optimization procedure
18

---

Published as a conference paper at ICLR 2026
by conducting additional controlled experiment. Specifically, we empirically track the gradient trajectories of both the 3-gradient (adversarial weights) and the I-gradient (model parameters).
Specifically, we fixed the hyperparameters (e1, E2) (0.6, 0.4) and performed a domain adaptation experiment on the SVHN-MNISS task using a mini-batch size of 128. At each iteration, we com- puted the L2 norm of the gradients with respect to / and 0. For panel (a), the learning rate for O was fixed at 10-3 while varying the learning rate for 3. For panel (b), the learning rate for 3 was fixed at 10-2 while varying the learning rate for 0. For each learning rate in {10-1, 10 10-2, 10 10-"), we repeated the procedure 20 times to assess stability and convergence across different update mag- nitudes.
Figure 2 demonstrates the resulting gradient trajectories of 8 and 0, respectively. The main obser- vations are summarized below.
Across all learning rates, both 3 and show monotonically decreasing and smooth gradient norms, indicating that the updates consistently move toward a stable point without sharp fluctu- ations.
Even under the most aggressive learning rate (10-1), the gradient norms remain well-controlled and do not exhibit any signs of gradient explosion, oscillation, or unstable plateauing, which are typical indicators of optimization instability in min-max procedures.
The variance bands across 20 random repetitions are uniformly narrow, suggesting that the opti- mization dynamics are highly reproducible and not sensitive to stochastic variation in initializa- tion or minibatch sampling.
These empirical findings demonstrate that the proposed joint optimization procedure is numerically stable, robust to learning-rate variation, and reliable across repeated runs.
The optimization structure follows the standard formulation of GroupDRO (Sagawa et al., 2019), whose convergence behavior has been analyzed in prior work. Similar min-max update rules have also been used in recent studies (Krueger et al., 2021; Zhou et al., 2021), indicating that this class of optimization procedures is generally well-behaved under practical settings.
A.4 SELECTION OF PREUDO-GROUPS
In this subsection, analyze practical strategies for selecting the number of gyeudo-groupg K and conduct a sensitivity analysis. Selecting the number of pseudo-groups K is critical in the single- source setting due to an inherent trade-off. If K is too small, the diverse conditional structures in the source distribution may not be fully captured, limiting the representation of target conditional uncertainty. Conversely, increasing K substantially raises computational and memory costs, as
each pscudo-group requires training separate conditional estimator p(k) Therefore, choosing an intermediate value of K is essential. We discuss two practical strategies for selecting K and analyze the sensitivity of the model this choice.
The first strategy utilizes a small labeled target validation set to guide the choice of K. In our experiments, we used 10 labeled samples per class. Figure 7(a) visualizes the validation accuracy for the MNIST-UUSP task. Accuracy increases stcadily as K grows from 2 to 7 across all eg. Beyond this point, performance stabilizes: when K 2 7 and c2 > 0.4, accuracy reaches plateau between 89% and 91% %. Although the highest accuracy is observed at K = 14 with c2 1, K = 10 serves as practical compromise, balancing performance with computational efficiency.
Alternatively, K can be selected via cross-validation the source data, leveraging the Maximin effect (Meinshausen Bühlmann, 2015). For each candidate K, the source data is partitioned into training and validation subsets. K pscudo-groups are formed from the training subset to train conditional estimators P(R) x- which are then evaluated on the validation subset. The K yielding the highest average score is selected. This provides a data-driven criterion independent of target labels. Figure 7(b) shows the validation aceuracy obtained from 5-fold cross validation. one can that validation performance improves with K and stabilizes for K 2 9 with c2 E [0.4, 0.8]. These results suggest that selecting K in the range of 9 to 14 is appropriate.
To assess robustness, Figure 7(c) summarizes test accuracy across varying K and 62. Performance varies smoothly across different 62, indicating low sensitivity to this parameter. Accuracy improves
19

---

Published as a conference paper at ICLR 2026
Validation Accuracy Heatmap
Validation Accuracy Heatmap
71
74
75
75
87
84
84
153
643
78
80
93
85
B0
80
84
BO
52
87
82
93
93
88
84
78
81
61
62
84
86
89
92
85
94
90
93
-31
91
95
80
85
88
89
89
89
71.
89
89
91
92
95
85
3
81
750
90
92
95
82
90
94
95
02
08
10
02
34
08
10
(a) Validation with small labeled target
(b) Validation using cross-validation
Test Accuracy Heatmap
73.0
73.8
74.6
74.2
75.6
80.6
8
81.7
81.8
81.5
81.0
82.9
80
81.4
81.7
82.0
82.6
80 2
81.0
79.9
60.5
83.9
01.5
84.1
85.7
05.0
05.3
82.7
82.1
83.0
85.5
85.6
06.0
06.0
0.6
0.4
0.6
18
(c) Test accuracy
Figure 3: Heatmaps of average accuracy for the MNIST-+USPS task with 102 unlabeled target samples per class, shown across different values of K and e2. Panels (a) and (b) show validation accuracy obtained using small labeled target and cross-validation, respectively. Panel (c) shows the corresponding test accuracy. All results are averaged over 5 independent runs.
up to K 7, after which plateaus within high-perte gighhperforming range (84%-86% for 2 8). Notably, both selection strategies converge consistent range of K [9, 12j. While larger values of may offer marginal gains, K = 10 is adopted as the default setting to balance accuracy with computational cost, which increases linearly with K.
20

---

Published as a conference paper at ICLR 2026
A.5 EXPERIMENTS DETAILS
In this section, we provide details regarding the datasets and baseline configurations. The source code is available at httph:tt//// htpp::///thhbbbb httm:/kswwi55000 for_U
A.5.1 DIGIT BENCHMARKS
(LeCun et al., 2002): A widely used benchmark dataset of handwritten digit recognition, consisting of grayscale images of digits from to Each image is of size 28 28 pixels, yielding 784-dimensional input vectors when flattened. The dataset contains a total of 70,000 images, with 60,000 images allocated for training and 10,000 images reserved for testing. The digit classes are balanced across 10 categories, corresponding the labels 0 through 9. Due to its simplicity and ac- cessibility, MNIST has been extensively used for evaluating classification algorithms, representation learning methods, and as a canonical testbed for new machine learning models.
MNIST
SVHN (Netzer et al., 2011): The Street View House Numbers dataset, designed for digit recog- nition in natural scenes. It consists of cropped color images of digits (32 x 32 pixels) obtained from Google Street View house numbers. SVHN provides 73,257 training images, 26,032 test im- ages, and an additional dataset covers 10 digit classes (0-9) and presents greater variability than MNIST due to cluttered backgrounds, varying illumination, and diverse font styles, making it challenging and widely used benchmark in digit classification and domain adaptation studies.
of 531,131 extra training images facilitate large-scale training. The
USPS (Hull, 2002): The U.S. Postal Service (USPS) dataset is benchmark collection of hand- written digits originally scanned from postal envelopes. Each image is grayscale 16 x 16 pixel rep- resentation of digits ranging from 0 9, resulting compact 256-dimensional input vectors when flattened. The dataset provides 7,291 training images and 2,007 test images, spanning 10 classes that are approximately balanced across digits. Compared to MNIST, USPS offers lower-resolution images and displays noticeable stylistic variations in handwriting, including slant, thickness, and shape differences, which make recognition more challenging. Due to these characteristics, USPS is often used alongside MNIST and SVHN complementary benchmark in digit classification tasks and as a target or source domain in domain adaptation studies.
504
/
9
2
3
#
8
0
7
0
1
5 3
6
2
8
d 9 /
4
2 7
3620
8
7
4
3 6
6 3 3
3 3 6
9
0
5
0 5 0
6
6
A
8 1 9
3
8
3
X
4
0
0
7
2
7
7
741
8
4 4 6
0
1 2
6
8
0
7
2
1
6
6
7
3
0
8
3
9
0 3
7 4
59
2
08
2
0
6
4
6
0
7
8
(a)
MNIST
(b) SVHN
(c) USPS
Figure 4: Example images from the CMNIST dataset.
A.5.2 SPURIOUS CORRELATION BENCHMARKS
Waterbirds (Sagawa et al. (2019)): The Waterbirds dataset designed to study the impact of spu- rious correlations in image classification. is constructed by overlaying bird images from the CUB- 200-2011 dataset (Wah et al. (2011)) onto background scenes from the Places dataset (Zhou et al. (2017)). This process induces a strong but spurious correlation between bird type and background. Specifically, the dataset is divided into four groups based on bird type (landbird vs. waterbird) and background (land vs. water): 91 [landbird, land), 92 {landbird, water). gs = {waterbird, land), and 94 = (waterbird, water). Most samples belong to the aligned groups (g1 and g1). while the misaligned groups (g2 and g3) are relatively rare. This imbalance creates minority groups that highlight the difficulty of learning invariant predictors under spurious correlations. In our experi- ments, we follow prior work by treating the three majority groups as the single-source domain and
21

---

Published as a conference paper at ICLR 2026
the minority group (gs, waterbirds with land background) the target domain. The training set contains 4,739 source images and 56 unlabeled target images, while the target test set includes 642 samples, providing controlled evaluation of out-of-distribution generalization.
Landbirds, Land (b) Landbirds, Water
Waterbirds, Land (d) Waterbirds,
Wa-
ter
Figure 5: Example images from the Waterbirds dataset.
CelebA (Liu et al. (2015)): The CelebA dataset contains large-scale celebrity face images anno- tated with multiple binary attributes. We consider the task of classifying blond vs. non-blond hair, where gender acts as spurious attribute correlated with hair color. This naturally induces four groups based on hair color (blond vs. non-blond) and gender (male vs. female): şI (non-blond, female), ). 92 (non-ble moond male), 93 (blond, female). and 94 {blond, male]. Most samples belong to 91, 92, and 93. while the minority group 91 (blond-haired males) is underrepresented, reflecting the spurious correlation between hair color and gender. The three majority groups combined to form the single-source domain, and the minority group is treated as the target domain. The dataset provides 161,383 labeled source training images, 1,387 unlabeled target training images, and 642 target test images for evaluation.
(a) Non-blond, Fe- (b) Non-blond, Male male
(c) Blond, Female
(d) Blond, Male
Figure Example images from the CelebA dataset.
Colored MNISTT (CMNIST) (Arjovsky et al. (2019)): The Colored MNIST dataset a synthetic variant of MNIST designed to study the effect of spurious correlations. We consider binary clas- sification task using digits and 1, which yields four groups based on digit identity and color: 9i {digit 0, red), 92 (digit 0, green), g3 (digit 1, red), and fa {digit green]. Most samples belong to the aligned groups (g1, 94. and ga), while the minority group 92 (digit O with green color) is underrepresented. capturing the spurious correlation between digit and color. In our experiments, the three majority groups are combined to form the single-souree domain, and the minority group is used the target domain. The dataset provides 26,00 labeled source training images, 2,998 unlabeled target training images, and 8,966 target test images.
A.5.3 BASELINE DETAILS
ERM (Sre-only) The standard empirical risk minimization approach that optimizes average ac- euracy the source training set. While simple and widely used, ERM does not incorporate any robust objective and cannot exploit unlabeled target data, making it insufficient under domain shift.
22

---

Published as a conference paper at ICLR 2026
3 -
3
9
9
(a) O, Green
(b) 1, Green
(c) O, Red
(d) 1, Red
Figure 7: Example images from the CMNIST dataset.
CDAN (Long et al., 2018): Extends DANN by conditioning the domain discriminator on both feature representations and classifier predictions. This coupling captures multimodal structures in the feature space, leading to more effective domain alignment.
MK-MMD (Long et al., 2015): Minimizes the maximum mean discrepaney (MMD) across mul- tiple kernels between source and target feature distributions. By leveraging multiple kernels, MK-MMD adapts flexibly to complex distribution shifts.
CORAL (Sun Saenko, 2016): Matches the second-order statistics (covariances) of source and target features to reduce domain discrepancy. This moment- mattienn approach is simple, efficient, and widely adopted in UDA.
MCD (Saito et 2018)) Employs two classifiers with shared feature extractor. By maximizing their prediction discrepancy on target samples and then minimizing it, MCD encourages the extractor to learn target-discrmminatie features.
ATDOC (Tang et al., 2020): Adapts classifiers by leveraging target-domain pscudo-labels in an adversarial manner. progressively refines pseudo-labels to improve alignment and robustness in the absence of ground -rout target labels.
STAR (Lu et al., 2020): Introduces stochastic classifiers to model diverse decision boundaries. By averaging predictions across multiple classifiers, STAR improves stability and robustness under domain shifts.
Group DRO (Sagawa et al., 2019): A robust optimization method that minimizes the worst-case loss across predefined groups. Since relies on explicit group labels the training data, Group DRO cannot leverage unlabeled target data, which limits its applicability in unsupervised domain adaptation.
Group DRO (with Tgt) (Sagawa al., 2019): A robust optimization method that minimizes the worst-case loss across predefined groups. In this variant, we assume access to group labels from both the source and target domains during training. This setup not realistic in unsupervised domain adaptation, since target group labels are unavailable in practice.
ICON (Yue et al., 2023): Focuses on unsupervised domain adaptation by enforcing invariant consistency across source and target predictions. ICON leverages consistency regularization to learn representations robust to distributional shifts, achieving state-of-the-at performance in UDA.
PDE (Deng et al., 2023): A method designed combat spurious correlations by progressively expanding the training data from easy samples to harder ones. It identifies samples where the model relies on invariant features rather than spurious ones, effectively improving generalization data without requiring full group labels for all samples.
on
DRUDA (Wang Wang, 2024): distributionally robust unsupervised domain adaptation (UDA) framework that addresses the distribution shift between source and target domains. By employing a distributionally robust optimization approach, it minimizes the worst-case risk over uncertainty around the target distribution, thereby enhaneing the model's stability and performance in unlabeled target environments.
A.6 DISCLOSURE ON LLM USAGE
used large language models solely to aid polishing writing of this paper.
23


Reviews:
[{"review_id": "jvp8VCyP9b", "reviewer": "Reviewer_nZt1", "summary": "The paper applies Distributionally Robust Optimization (DRO) to solve Unsupervised Domain Adaptation (UDA). It constructs ambiguity samples from the target data whose pseudolabel is the linear combination of the label from multiple source domain classifiers. The ambiguous sample is generated using a technique similar to adversarial training. The proposed method can also be viewed as a regularizer.", "questions": "In Eq 3. What is variable for the Expectation under \\hat{P}^{tg}_X. Is it z’ or z(X)? Likewise, in the expectation In lines 775-777 what is the difference between z(x) and z(X). How does it change to z’ in lines 781-782. There is inadequate explanation.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Novel extension of DRO to UDA\n2. Impressive results on target accuracy", "weaknesses": "1. The paper attempts to apply Distributionally Robust Optimization (DRO) to the problem of Unsupervised Domain Adaptation (UDA). The idea of applying DRO to UDA appears to be novel although some of the technical contributions are from pre-existing DRO work. The proposed approach is not technically convincing due to tedious notation and workflow. It was quite difficult and frustrating to follow the logic in explanation in the paper. \n2. Target accuracy is the only result made available. This does not give enough insight into the workings of the model.\n3. What is the intuition behind Eq. 6. Whichever subset (domain) k has the least predictive power (largest loss) it gets the largest \\beta_k. Why is this reasonable? Will this not result in negative transfer? \n4. In Eq. 5, How is Euclidean projection onto set A equivalent to minimizing the Wasserstein distance D1. What do you mean by Euclidean projection? How is the \\epsilon_1 constraint satisfied?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e1oPtV56DA", "reviewer": "Reviewer_Fp6T", "summary": "This paper proposes a novel distributionally robust optimization (DRO) framework for multi-source unsupervised domain adaptation (MS-UDA). The method models both input marginal distribution shifts and label conditional distribution shifts by constructing a mixture-based ambiguity set that combines conditional distributions from multiple source domains with adaptive weighting, while allowing controlled perturbations of the target input distribution. The authors further design a minimax optimization algorithm that alternately updates feature perturbations, mixture weights, and classifier parameters. Experimentally, results on several benchmark datasets (MNIST/SVHN/USPS, Waterbirds, CelebA, CMNIST) demonstrate that this approach achieves significantly better classification performance than mainstream UDA methods such as DANN, CDAN, and STAR, particularly in scenarios with scarce target data or spurious correlations.", "questions": "1. How does the reliance on labeled target data for hyperparameter selection (specifically for $\\epsilon_1$ and $\\epsilon_2$) affect the generalization of the method in unsupervised domain adaptation? Could this be considered as a shift towards a \"partially supervised\" or \"few-shot\" adaptation setting?\n\n2. How sensitive is the proposed method to these hyperparameters?\n\n3. How does the proposed method perform on data from other modalities?\n\n4. How critical is the choice of base classifier (ERM vs. CDAN/STAR) for the final performance?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a unified Distributionally Robust Optimization (DRO) framework that is applicable to both multi-source and single-source Unsupervised Domain Adaptation (UDA), offering flexibility across different scenarios.\n\n2. Unlike traditional approaches that typically model either input uncertainty or label distribution uncertainty in isolation, this method simultaneously accounts for both, providing a more comprehensive solution.\n\n3. The proposed algorithm is highly tractable, ensuring that it can be seamlessly integrated with existing UDA frameworks, making it easy to adopt and implement in real-world applications.\n\n4. The experimental results demonstrate the method's robust performance, especially in tasks where target data is scarce or where spurious correlations are prevalent, outperforming several strong baselines.", "weaknesses": "1. A reliance on labeled target data for model selection moves the problem into a \"partially supervised\" or \"few-shot\" adaptation setting. While the main training uses unlabeled target data, the crucial choice of hyperparameters $\\epsilon_1$ and $\\epsilon_2$ is supervised. \n\n2. The appendix shows heatmaps of performance vs. $(\\epsilon_1, \\epsilon_2)$, which is good. However, the main paper should discuss how sensitive the method is. Is there a broad range of \"good\" hyperparameters, or does the performance collapse without precise tuning? This context is crucial.\n\n3. Although multiple standard datasets are used for testing, there is a lack of evaluation in broader domains (such as NLP or time-series data) to demonstrate the generalizability of the method.\n\n4. The conclusion section does not sufficiently discuss the limitations of the method and potential future directions for improvement. Adding these aspects would help present a more comprehensive view of the research depth.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oZlSEu6W8H", "reviewer": "Reviewer_Lo6k", "summary": "The paper proposes a distributionally robust optimization (DRO) framework for unsupervised domain adaptation (UDA), particularly under multi-source and target-data-scarce settings. The method defines an ambiguity set over both the covariate (input) distribution and the conditional label distribution, allowing robustness against (i) uncertainty in target inputs and (ii) uncertainty in which source conditional distributions to rely on. The framework is applicable to both multi-source and single-source UDA, using pseudo-sources generated via sub-sampling. A tractable minimax algorithm is derived (Eqs. (3)–(7); Algorithm 1), optimizing over feature perturbations, mixture weights, and classifier parameters. Experiments on digit datasets (MNIST, SVHN, USPS) and spurious-correlation benchmarks (Waterbirds, CelebA, CMNIST) demonstrate consistent performance gains over UDA and robust-learning baselines (Tables 1–2).", "questions": "Please see the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper identifies two overlooked issues in UDA: scarce unlabeled target data and spurious source correlations.\n2.\tThis dual modeling is theoretically elegant and practically relevant for multi-source robustness.\n3.\tThe paper is well-written and easy to read.", "weaknesses": "1.\tD₁ (Wasserstein-∞) and D₂ (Euclidean) are chosen “for computational tractability” (Sec. 3.3) but without theoretical or empirical justification.\n2.\tHyperparameters ϵ₁, ϵ₂ are selected via a small labeled validation set (Sec. 4.1), partially violating the unsupervised setting.\n3.\tNo ablation compares using only conditional-mixing vs. only covariate-perturbation.\n4.\tThe relationship between pseudo-source construction (Sec. 3.1) and mixture weights β is not fully explained; readers may confuse stochastic sub-sampling with real domain partitioning.\n5.\tAbsence of gradient-stability discussion under joint optimization may raise reproducibility concerns.\n6.\tCompared to DRO literature [a, b], the paper’s theoretical contribution is weak.\n\n[a] Learning models with uniform performance via DRO.\n[b] Distributionally robust stochastic optimization with Wasserstein distance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FKkiOHJLre", "reviewer": "Reviewer_fB8c", "summary": "The paper introduces a novel method for unsupervised domain adaption based on the framework of distributional robust optimization. The key idea is to represent the conditiona distribution $P(Y|X)$ as the mixture of empiricial conditional distribution $\\hat{P}^{(k)}(Y|X)$ from multiple sources and allows the pertubation of target input distribution over move inside a small Wasserstein ball. This method is plug-and-play, can be effectively integrated with existing UDA methods. Experiments on digits (MNIST/SVHN/USPS) and spurious-correlation suites (Waterbirds, CelebA, Colored-MNIST) show consistent gains and strong performance when target data are very limited.", "questions": "Most questions have been proposed in the Weakness section.\n\nA minor question:\n\nDuring the analysis of impact of radius $(\\epsilon_1, \\epsilon_2)$, I do not quite understand the content from lines 1004-1007. The aurthors argue that $\\epsilon_2$ will play a critical role when the target data is quite scarce. However, according to the Figure 5(b), the $\\epsilon_2$ provides nearly no influence for a fixed $\\epsilon_1$. Can the authors elaborate more on their arugments, or am I missing anything?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is quite well-written. \n\n2. The proposed  method is quite intuitive and easy to understand, with tractable surrogate and a relatively simple algorithm.\n\n3. The algorithm integrates seamlessly with existing UDA frameworks since its neat design that the input of the algorithm is the feature mapping $z$. \n\n4. The empirical effectivness is validated on real-world datasets with non-trivial improvements.", "weaknesses": "Currently, the largest potential issue is the lack of discussion of the grouping strategy. Specifically, it is not clear for a single soruce distribution, how do we know the number of pseudo-sources $K$? The choice of $K$ should directly influence the performance given the main idea that the target conditional distribution is the mixture of empirical conditional distribution. It will be appreicated to provide a more detailed analysis regarding the choice of $K$ (e.g., heuristics, validation criteria, stability checks), plus a short sensitivity or ablation study.\n\nI’m not an expert in UDA, so I’m unsure whether the current baselines reflect the latest methods. A quick check search leads me to two recent papers [1], [2]. Could you comment on their relevance and, if appropriate, explain why they weren’t included (e.g., different setting, data requirements, or incompatibility)? I’m completely open to your explanation.\n\nAnother potential issue the optimization objective is surrogate-based, i.e., we are always apporacing a suboptimal result. In this case, an analysis regarding the tightness of the gap will be appreciated. However, I totally understand if there have not been any since this is not the contribution of the paper. I will not change my assessment of the paper regardless of the absent of such analysis.\n\n[1] Partial Identifiability for Domain Adaptation. \n\n[2] Subspace identification for multi-source domain adaptation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)