Revisiting Source-Free Domain Adaptation: a New Perspective via Uncertainty Control
Authors: Gezheng Xu
Hui Guo
Li Yi
Charles Ling
Boyu Wang
Grace Yi
Venue: ICLR 2025 Poster (2025)
Keywords: Source-Free Domain Adaptation
Unsupervised Domain Adaptation
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Source-Free Domain Adaptation (SFDA) seeks to adapt a pre-trained source model to the target domain using only unlabeled target data, without access to the original source data. While current state-of-the-art (SOTA) methods rely on leveraging weak supervision from the source model to extract reliable information for self-supervised adaptation, they often overlook the uncertainty that arises during the transfer process.  In this paper, we conduct a systematic and theoretical analysis of the uncertainty inherent in existing SFDA methods and demonstrate its impact on transfer performance through the lens of Distributionally Robust Optimization (DRO). Building upon the theoretical results, we propose a novel instance-dependent uncertainty control algorithm for SFDA.  Our method is designed to quantify and exploit the uncertainty during the adaptation process, significantly improving the model performance.  Extensive experiments on benchmark datasets and empirical analyses confirm the validity of our theoretical findings and the effectiveness of the proposed method. 
This work offers new insights into understanding and advancing SFDA performance.

Published as a conference paper at ICLR 2025
REVISITING SOURCE-FREE DOMAIN ADAPTATION: A NEW PERSPECTIVE VIA UNCERTAINTY CONTROL
Gezheng Xu Hui Guo Li Charles Ling! Boyu Wang 1.2 Grace Y. YI!.233
"Department of Computer Science. University of Western Ontario Vector Institute
3 Department of Statistical and Acturria Sciences, University of Western Ontario "Tiktok {gxu86, hguo288, charles. ling, gyi5]0uwo.cm li li. wwy.yyyyedaancee...mmm com, bwang&csd. uwo. ca
ABSTRACT
Source-Free Domain Adaptation (SFDA) seeks to adapt a pre-trained source model a target domain using only unlabeled target data, without access to the original source data. While current tatte-of-hheaasd art methods rely on leveraging weak super- vision from the source model to extract reliable information for self-supervised adaptation, they often overlook the uncertainty that arises during the transfer process. this paper, we conduct a systematic and theoretical analysis of the uncertainty inherent in existing SFDA methods and demonstrate its impact on transfer performance through the lens of Distributionally Robust Optimization. Building upon the theoretical results, we propose novel instance-dependent un- certainty control algorithm for SFDA. Our method quantifies and exploits the uncertainty during adaptation, significantly improving model performance. Ex- tensive experiments on benchmark datasets and empirical analyses confirm our theoretical findings and the effectiveness of the proposed method. This work offers new insights into understanding and advancing SFDA performance. We release our code at /https::/gitiiiii com wwwp///xggzeeegngggeee
SPDA
INTRODUCTION
Deep neural networks (DNNs) have achieved remarkable performance across wide range of tasks. However, their performance can degrade significantly when a domain shift occurs between training (source) and test (target) data. Traditional solutions rely on transferable knowledge from labeled source data to classify unlabeled target data, but access source data is often restricted due privacy concerns or proprietary constraints. To address this, Source-Free Domain Adaptation (SFDA) has emerged as solution, aiming adapt pre-trained source model to an unlabeled target domain without accessing the original source data (Liang et al. 2020 Yang et al. 2021b
Recent work has integrated self-supervised learning with transfer learning in SFDA, with contrastive learning-based supe-supereieed methods gaining widespread use and empirical support (Yang et al.) 2022 Karim al.) 2023; Chen et al.] 2022] Hwang al. 2024 Mitsuzumi al. 2024). A key challenge in applying contrastive learning methods to SFDA lies in selecting and utilizing positive and negative samples of target data using a
source model. Different from conventional contrastive learning methods using data augmentations as positive samples, in SFDA, the neighbors in the feature space can provide stronger supervision and usually be treated as positives, and the negative samples are the remaining data in the training mini-batch. However, due to the domain shift, these methods face severe uncertainty, as will be elaborated shortly.
In this paper, we systematically and theoretically examine the uncertainty present in SFDA through the lens of Distributionally Robust Optimization (DRO). Unlike previous studies that primarily focus on empirical strategies (Roy et al. 2022 Litrico et al. 2023 Pei et al. 2023 Lee et al.) 2022). our work offers a comprehensive analysis of two types of uncertainty arising from the use of negative and positive samples in existing SFDA methods, aiming to enhance SFDA performance by explicitly controlling the uncertainty. Specifically, on one hand, random sampling of negative samples in
"qqual contribution, Cooresspondigg Authors

---

Published as a conference paper at ICLR 2025
(181-)
Megesteee perDrra Pairt (8:kh) Ste.
Cansizznnyy
6.5
0.40
Gaera Sealhred clan Accurioy
Droertan fwik for olker lukel for decunee us the mattaael Aecurnyy Cortannbus
VIDA-RUST VaDA2017 Office
Officellome
Ar Ar-
Dataset
Office Home
VIsDA2017
Traliing Process
(a) Negative Sampling Uncertainty
(b) Inconsistency Prediction
(c) Positive Supervision Uncertainty
This figure summarizes key findings from our experiments, to be detailed in Section 4.4 (a) Presence of false negatives across different datasets; (b) Inconsistency between predictions for original images and their augmented views by the source model; (c) Illustration of varying predictive accuracies between certain and uncertain target data during adaptation on Office- Office-Home!! (Ar ++ CI).
Figure 1:
applications often introduces outliers, or "false negatives" - samples that belong to the same class as the considered target data point but are mistakenly selected as negatives (as shown in Figure Lap. This discrepancy leads to a deviation of the empirical negative distribution from the true distribution, thus introducing variability into the loss calculation. To address this sampling bias, we introduce a negative uncertainty set, which consists of distributions obtained by slightly perturbing the training negative distribution, and consider an outlier-robust worst-case risk within this set. We theoretically derive an informative upper bound for this risk, which motivates incorporating dispersion control term into the loss function. Moreover, to address the prediction inconsistency between a target image ingmentation-baseee dispersion
and its augmented view (as shown in Figure lb), we propose control approach to mitigate uncertainty introduced by noisy negative samples. On the other hand, domain shift causes models trained source data produce uncertain probability estimates for target data. In such cases, the supervisory information from positive examples may not fully align with the ground truth, making the use of neighboring predictions for supervision introduce additional uncertainty. Unlike existing methods that focus on mitigating uncertainty (Roy et al. 2022 Litrico al. 2023 Mitsuzumi et al. 2024), we aim to utilize this information more effectively. To better accommodate the uncertainty in the predicted probabilities of positive samples, we consider a positive uncertainty set centered around these probabilities and examine the worst-case risk within this set. We theoretically show that the optimal solution for target points involves partial label set. To make the most of this uncertain information, we propose novel criteria to identify uncertain data and use partial labels to relax supervision for these samples. As illustrated in Figure Ic leveraging such uncertainty information improves performance compared to relying only on certain data.
Our contributions are as follows: (1) We theoretically analyze two key sources of uncertainty in con- trastive learning- Sarning-bbssed based SFDA methods, identifying two types of worst-case risks under a unified DRO framework. This investigation explains why current contrastive learning methods can significantly improve SFDA performance (Section 4.2p while revealing the overlooked uncertain information in existing methods (Section 4.3). Our theoretical analysis also provides novel perspective in understanding the SFDA problem. (2) Based on our theoretical result, we design a novel Uncertainty Control algorithm for SFDA (UCon-SFDA) to minimize the negative effects of uncertainty nega- tive sample selection while leveraging the uncertain information from positive sample predictions to enhance the model's discriminability (Section 4..).. (3) We conduct extensive experiments to demonstrate the effectiveness of the proposed method.
RELATED WORK
Source-Free Domain Adaptation (SFDA). SFDA focuses on adapting well-trained source model
to a target domain with only unlabeled target data. Since source data are not accessible during adaptation, some methods extract source information through prototype generation (Qiu et al.) 2021), or minimization of reliance on source data through adversarial training (Li et al. 2020b). To address the lack of labels for target data, several methods aim to enhance supervision. For example, SHOT (Liang et al. 2020) employs deep clustering to create pseudo-labels, while NRC (Yang et al. 2021a) and G-SDD (Yang et al. 2021b) leverage neighboring predictions guide the
2

---

Published as a conference paper at ICLR 2025
adaptation process. Yi et al. (2023) formulates SFDA problem of learning from noisy labels. Recently, self--upervised learning has gained attention in SFDA, particularly contrastive learning- based sulf--uper methods. For instance, AaD (Yang et al. 2022) introduces positive and negative samples into SFDA and uses a simplified contrastive loss to enhance model discriminability while maintaining diversity; C-SFDA (Karim al. 2023) utilizes teacher-student framework to enhance the self-training in SFDA; methods like DaC (Zhang et al. 2022), AdaContrast (Chen et al. (022), and SF(DA)² (Hwang et al., 2024) explore explicit or implicit data augmentation further boost SFDA performance. I-SFDA (Mitsuzumi et al., 2024) offers a new perspective by approaching SFDA through self- training. Despite these advancements, a comprehensive theoretical framework explaining their effectiveness remains absent. Moreover, most existing methods do not fully account for the uncertainty inherent in the adaptation process, which can negatively impact SFDA performance.
Uncertainty in SFDA. Given the absence of both source data and target labels, handling uncertainty a key challenge in SFDA, particularly in the presence of domain shifts. Most existing research addresses prediction representation uncertainty by reweighting loss functions prioritizing more confident samples during training (Roy et al., 2022 Litrico et al. 2023 Pei et al. 2023] Lee et al. 2022). In contrast to these approaches, provide systematic and comprehensive analysis of various sources of uncertainty in contrastive learning- based SFDA from an instance- -attace-deeendant perspective. Building on this analysis, propose a novel algorithm that improves SFDA performance by effectively controlling variance during adaptation.
PRELIMINARIES
We use [k] to denote the set (1...., k} for any positive integer k. For a E R, we define a., max(a, (. 0}, and let [a] and [a] denote the floor and the ceiling of a, respectively. Let a A denote min(a, (a, b) for a, b R. For vector the jth element is denoted vy, and u indicates its transpose. For vectors U1 and 22, their inner product is denoted (u1, 12). Let 1 (.) represent the indicator function.
Problem Setup. For K-class classification problem, let X c Rdd represent the input space, and let y = [K] denote the label space, with d denoting the input dimension. Let X and Y denote the random input and label, respectively, and let x and y represent their realizations. In SFDA, the source domain distribution PS and the target domain distribution Pxy are unknown distributions over X x y that may be distinct. We factorize these distributions as products of the marginal and conditional distributions for the corresponding variables, indicated by the subscripts: P = P pr Px yix For the source domain, we have source model hig X + V, which, for example, is a DNN based predictor pre-trained with Ng labeled examples D3 _ (7), Nis drawn from P In the target domain, let D., A (x.) ) denote the unlabeled target domain data, consisting of N observations of independent and identically distributed (i.i.d) random variables drawn from PT, and is used as the training set. Given the source model hy and unlabeled target data Dr, our goal is to learn target model hr X + y that predicts labels in the target domain by adapting hig on Dr.
assume that
and pr xy
To facilitate our analysis in the context of deep learning, let f,(:;,)) : X - A-1 denote the network output for the target data, indexed by the parameter vector 0, associated with the DNN architecture, taking values in the parameter space O where A K denotes the K --simplexx and the jth component of the vector-valued function (x; Or), denoted fr(x; [i) models the predicted conditional probability IP(Y = j|X = x) for the target domain data. We then define the target model hy as h, 0.) = arg max,5e[]] (x; [,] for any x E X. Similarly, the source model hag along with fs(:;g)) X K- and g.. defined for the source data. In applications, the source and target models h: 0g)) and (:; fr), or more specifically f& (:; (,) and (r 0,),, are often specified with the same network architecture, with the target model's 's parameters initialized from the source model. Consequently, distinction in the following development, unless it is needed to distinguish them.
use the generic notation h(:; 0) or f(; 0) to represent these models without
Given an anchor point X = x from the target set (i.e., the data point used reference to determine positive and negative examples), a positive example refers to a sample in the target set D. that belongs to the same class x, and negative example refers to sample from different class. The latter is also called a "true negative", in contrast to the "false negative" mentioned in Section] I which refers to sample that belongs to the same class as the anchor point but is mistakenly selected as negative.
3

---

Published as a conference paper at ICLR 2025
4 THEORETICAL ANALYSIS AND ALGORITHM
4.1 MOTIVATION
Existing SFDA methods typically decompose their training loss into two components: (1) discrim- inability, which enhances the model's ability to distinguish between unlabeled target samples, and (2) diversity, which encourages predictions be distributed across different classes (Yang al. 2022 Mitsuzumi et al.) 2024 Cui et al. 2020). Among these approaches, contrastive learning methods are perhaps the most widely used, where the goal maximize the similarity between positive pairs to improve discriminability and minimize the similarity between negative pairs to ensure diversity. This can be formulated as the following expected risk with contrastive loss:
Reaxic ( = EpT p- (So(X* x)} + Ep- (X= x)]]]
where the function So(::) )) mapping from X' x X [0, 1]. represents the similarity measure between two instances, such as cosine similarity, as detailed in Section 3 The outer expectation Er is taken with respect to the imput distribution for in the target domam, and the inner expectations Ep+ and Ep- are evaluated under the conditional distributions of positive example X* and negative example X respectively, given X.
In contrastive braningg-based SFDA, for each target input xT in mini-batch B, the set of positive examples relative to x, denoted C,, consists of its k-nearest neighbours in the target domain data Dr, where K is typically chosen between 2 and 5. The negative set is taken B' {xx J. which, however, inevitably includes a fraction of false negatives, introducing sampling bias. While a well-trained source model helps to ensure that neighboring positive samples in the feature space provide effective supervision for most unlabeled target data, some highly uncertain samples persist due to domain shift. To address these issues, we propose robust strategy for managing uncertainty in SFDA using DRO.
4.2 NEGATIVE SAMPLING UNCERTAINTY AND DISPERSION CONTROL
To address sampling bias and distribution shift in negative examples, we formulate an expected DRO risk: for each given x E X' and 8 0,
sup (2 P , (TEra(PT)
[Eq- (Xx
where the expectation Eg- (So(X (o0xx;x) ;x) is evaluated under the conditional distribution Q' of X", given X = x. taken from the set (P) Here. I's(P") represents an uncertainty set of probability measures centered around the reference probability distribution P', with a radius ó >> that facilitates robustness. Commonly, (P" is defined as the distance uncertainty set:
[P(PP) (Q- E Px(X): d(Q" PT) 5}.
where (x)) denotes the class of Borel probability measures on X' with finite pth moment for some p > 1, and d is discrepancy metric of probability measures. Popular choices of d are i--iivergecces (including Kullback-Leibler (KL) divergence and x² divergence as special cases Gao et al. 2024 Blanchet & Murthy 2019).
Duchi 2016)) and Wasserstein distances (aoo 2023
In practice, negative samples are often drawn uniformly from the training data, which frequently leads to the inclusion of false negatives. Let Pri.in represent the distribution of these negative samples,
modeled using Huber's e-contamination method: Pi.inn - P + +eP- where (0. 1) is the contamination level, and P- represents an arbitrary contamination distribution (Huber 1992). For instance, consider some E X. Suppose collect negative samples, where fraction En
are i.i.d. false negatives drawn from P", and the rest are true negatives from P The resulting empirical distribution of the observed negative samples follows this model with a contamination level of [en]/n. To mitigate overfitting to worst-case instances that are likely to be outliers, we minimize a refined outlier-robust expected risk (Nietert et al. 2024a by Zhai et al. 2021):
(0; Prain: 8,c)
inf (x)
(R 8) 3P' E Px(X) s.t. Piain (1 -1-6h c)P' + cp) (4)
By definition, the minimizer of (4) is designed to ignore "hard" data points that contribute the most to the worst-case risk and instead focus the (1 e)-fraction of "easy" data points in the training set.
4

---

Published as a conference paper at ICLR 2025
Anchor point
True negative examples
Separation Dispersion control eflect
Psseudofalse negative examples
False negative examples
(a) No Dispersion Control
(b) Direct Dispersion Control
Dispersion Control with Pseudo- False Negatives
Figure 2: Illustrative visualization of the effect of dispersion control: (a) No dispersion control, (b) Direct dispersion control between the anchor and false-negative pairs, and (c) Dispersion control with pseudo-falss negatives.
This helps prevent overfitting to outliers, thereby reducing the risk of pushing the target data point away from others within the same class. For different choices of the discrepancy metric d in the uncertainty set 33. we establish a unified upper bound on the outlicr-robust risk (2; Pimin ó, c).
Specifically, for e-contaminated training distribution Pusin with 0 < < 1, let Pirain denote the associated density or mass function. For any given x E X. define the associated truncated distribution pa with density/mass function p": p p"(x")
- P- [x"] (So(x ;xxx S S for any negative example x of x, where s" is the 1 - c quantile satisfying Pimin (So(X S $*}
1 In contrastive SFDA, for each anchor point from the target set, the truncated version of urin denoted as P", concentrates all its mass on regions where the similarity falls below the (1 -c)-qunntill. In the following theorem, let R1 A 1-c So $ d.P.
train (So xx) S $}. R2 & 1 1o 8² .PPi.. (So(X";)) S s}. Ep- (Sx(Xxxx) = =R,, and Vp- (So(X x)] = R2 R??
Theorem 4.1. Suppose the similarity measure Sg satisfies the smoothness conditions in Lemma in Appendix AA.3orr all 0 E @ Then for a small and for different choices of the discrepancy metric d in (3). we have the following upper bounds the risk R.g (0; Pirai 8, in (4)
(i) If d is the
2 diverggen,, then
R= Pimino 8, S Ep- (So(Xxxx) + (V
(ii) If d is the KL-divergence, then
R2; Piruin 8,6) < Ep- [Sp + (Se 0(6).
(ii) If d. is the p- Wasserstein distance with [1, +o0) and the cost function of in Definition
A.1 is chosen as a norm || with the dual norm || I|,, then for q satisfying
=1
R2 Piruin 8, < Ep. (Sa(X xx) + [Ep. (Xx(XX 3/9 + ((62N))
Remark 4.1. Theorem 4.1 demonstrates that (0: Pi 8, is essentially upper bounded by the sum of two key terms regardless of a specific form of considered in the thcorem. The first term controls the average risk over potential true negatives, behaving similarly to the traditional negative sample loss (Yang et al. 2022). This implemented as the negative sample loss L., in (7). to be presented in Section 4.4 The second term facilitates the dispersion in similarity between these aechor-rrue-nggttii pairs from anchor-false-negatie ones; it encourages greater separation between prediction similarities for machor-true-negativv pairs and caloo--fale-negattii pairs, as shown by the wider gray area in Figure 2b than that in Figure 2a
true negatives, helping to distinguish
Remark 4.2. In practice, domain shift makes it challenging to distinguish between false negatives and true negatives. To address this, we propose to achieve dispersion control by manually constructing pseudo-false negatives using techniques such as data augmentation. As shown in Figure [b] for given anchor point x, the source model's prediction on its augmented version, denoted as AUG(x). may differ from the prediction for When this happens, AUG(x) is automatically treated as false
5

---

Published as a conference paper at ICLR 2025
negative for x. Motivated by the dispersion control term, we treat these augmentations pseudo-falss false negatives and minimize the negative similarity between the anchor point and its augmented prediction. As illustrated in Figure 2 this approach effectively amplifies the contrast between machor-false negative similarity and inceorr-rue-negative similarity, expanding the width of the gray region to improve the separation of the data points in the same class as the anchor point from the true negatives. This dispersion control effect is captured through the loss term Sc in as detailed in Section 4.4
4.3 POSITIVE SUPERVISION UNCERTAINTY AND PARTIAL LABELING
For each anchor point x in the target domain Dz, p 1 (pt....,, pk) & f(x; 0) E A K- denote the target model's predicted probabilities for where f(; 0) is described at the end of Section 3 and Py represents the jith component of f'x;; I). Similarly, for the positive example x* associated with x, let p* _ (pi..... .Pk) ) T fxx: 0) E A K represent the predicted probabilities for x". When using cosine similarity, the supervision information from the positive example x' encourages the model training to minimize the negative similarity, defined as (p',, .) = Σソ= PaPj.
In SFDA, leveraging a well-trained source model and the similarity between the source and target domain distributions, neighboring examples in the feature space are often treated as positive samples. While many of these positive samples provide effective supervision for unlabeled target data, there can still be highly uncertain examples due to domain shift. To better handle this uncertainty in model predictions. we explore the optimal prediction for an anchor point x by solving the following worst-case risk minimization problem based on DRO:
p*
inf RT(p;x", 5), with (p; A sup PEAK
(9'--); "*ETa(p")
(5)
where (''( is the uncertainty set centered around the reference distribution p', as defined in (3). Using the proof techniques in Guo et al. (2024), we derive closed-form expression for p" as follows.
Theorem 4.2. Let (pi....,Pik be arranged in decreasing order, denoted as P(a) ... 2 Piky with the corresponding indexes denoted as x(1)..... x(K). -a-ssssernttiin distance with the 0-1 cost function is used, then the optimal solution p* & (pt,..., -Pk) of
5 is given as follows:
(i) t + +
pby)
8 for all k* E [K -1, then p3 - for all E [K].
(ii) If there exists some ho E IK [k-1] 1] such that 1
No 1=1=1
10 5p > t and 1
To -i=Pim
J0 for all k* E [K 1]. then p(y) ku for j E [ko] and ply) = O for j - Ko + +,...,,K, K, where Přy) denotes the x(j)th element of p* corresponding to Play
L 80 2 F. P
Remark 4.3. Theorem 4.2 suggests that the optimal prediction for anchor point can be represented
by a set of instance-depended partial labels. The advantage of using partial labels, rather than the full predicted probabilities, the supervision signal is that it retains uncertain yet potentially more accurate label information, while eliminating interference from labels that are more likely to be incorrect. In the special case where P(1) 2 max + + 8², P(2) + j.), the optimal solution simplifies to P(U = and Pt) = 0 for j 2, .., K. That is, the optimal solution selects the label with the highest predicted probability for the anchor point, rather than a set of partial labels, when the gap between the top two probabilities exceeds given threshold. We refer to this scenario as certain label information; otherwise, we classify it as uncertain label information.
Remark 4.4. Motivated by Theorem 4.2 and Remark 4.3 we propose to leverage both certain and uncertain label information in distinct ways to effectively capture and utilize prediction uncertainty. Specifically. when an instance x receives certain label information, the optimal prediction for x corresponds to the label with the highest predicted probability. This certain supervision signal is incorporated through the positive supervision loss term L. in (8) When uncertain label information is present, the optimal prediction for x is expressed as a set of partial labels. Instead of relying solely on the estimated pseudo labels, we construct partial label set for x. This approach offers a more robust supervisory signal by accounting for multiple potential labels and reducing reliance on noisy single-lat label predictions. This information is captured through the partial label loss term Lt,, in (8). To distinguish between certain and uncertain label information in applications, we use the ratio of the two highest predicted probabilities, as detailed in Section 4.4

---

Published as a conference paper at ICLR 2025
4.4 IMPLEMENTATION
Our algorithm builds upon the conventional contrastive loss commonly adopted in previous works (Yang al. 2022 Mitsuzumi al.
2024): Ler. Cor + ferfeL
(6)
where Ler 1 EE -xxtee So(xtix))); for 1
Nr Li=1 Ex;eB (x.} (x(x;;x; with
Nr
B denoting the mini-batch; No. represents tuning parameter, and similarity is computed as (o xx) (x(x;; 0), [xx: 0)) or So(x = (xxx;0),
[( (xx; 0)). Here, positive samples the k-nearest neighbours in the feature space from the training set D,,, and negative samples the remaining data points in the same mini-batch B. Building on this simple yet widely adopted implementation in SFDA, our approach focuses on effectively controlling uncertainty during the adaptation process by refining both the negative and positive sample components.
Dispersion Control via Data Augmentation Alignment. To minimize the effect of false negatives, we introduce a dispersion control term Lic. which complements the conventional negative sample loss Ler. This leads to the following negative uncertainty control loss:
Nr
Cuoin Nor + AocLoo & Nor Ser + ddoe{
1
Σ de (AUG (xx), xx)
Nr
where de (AUG xx) (fxx; log f (AUG (x,) O)), which represents the cosine similarity between the network output of x, and the log probabilities of its augmented version AUG (x,). Often, AUG (x:) can be obtained through a series of stochastic transformations, including grayscale conversion, slight rotation, posterization, and Gaussian blur, as implemented in self-supervised learning (Chen et al. 2020). Similar to previous work (Yang al.) 2022). the decay coefficient Arz, is iter )9, where 3 and Aro hyperparameters, and "iter" and "max_iter" jia_ttee"
defined as Ao 11+10- marsitrr represent the current iteration value and the maximum number of adapting iterations, respectively, Ape regulates the minimization of the negative similarity between the anchor point and its augmented prediction, which is determined either through tuning hyperparameter or the inconsistency rate of data augmentation (shown in Figure lb Further details can be found in Appendices C.5 and D
Different from previous works that exclude false negative (Chen et al.) 2022 Litrico et al. 2023) or adjust the coefficient Xrx (Mitsuzumi et al., 2024). our proposed dispersion control term utilizes data augmentation to mimic false negatives without introducing additional uncertainty. This approach implicitly reduces the variability in prediction similarity between anchor points and noisy negative samples, while enhancing the model's prediction consistency.
Supervision Relaxation by Partial Label Training. As highlighted Theorem 4.2, partial labels help control uncertainty in positive sample predictions in SFDA. Our experimental findings (reported in Figure Ic) demonstrate that neighboring samples in the feature space can provide accurate label in- formation for initially confident target samples. However, highly uncertain samples require additional processing. To handle these uncertain samples, we propose an innovative approach to select uncertain samples during adaptation by tracking the ratio between the largest and second-largest predicted prob- f(x0))) < <T}.
abilities. Specifically, we maintain an uncertain data bank, defined as 2 = {x E Dr T(x:))a)
where (x; ())a is the ith largest predicted probabilities for x. The threshold - is typically set to a small value, usually between and 1.5, capture highly uncertain samples. Additionally,
store the historical TOP-Kpt predicted labels for each data x, to construct partial label set, denoted as y rI.i. which is then used to further supervise the training of uncertain data. The procedures for determining and Krt. are detailed in Appendices B and D Aftering incorporating the partial label
loss Eip,, the positive uncertainty control loss term Lucon is defined as:
Ciccnn & Cic. + AprCt.n & Lon. XPL
1
Σ
(ereen f(x;; 0)),
(8)
NT
EVLL
where lee is the smoothed cross-entropy loss, and ÅpL is a hyperparameter.
Unlike most uncertainty- based approaches SFDA, which focus on excluding or reducing the
negative impact of highly uncertain data during adaptation (Roy al. 2022 Litrico et al., 2023), our
method leverages uncertainty to extract additional label information from these samples, relaxing the training process and boosting the performance.
7

---

Published as a conference paper at ICLR 2025
Overall Uncertainty Control SFDA Loss. The final Uncertainty Control SFDA loss, Lucou SFDA defined as:
SECA = Lot + >pr + >xch
The pseudocode for the algorithm (Algorithm 1 and the training process can be found in Appendix B
5 EXPERIMENTS
5.1
EXPERIMENTAL SETUP
Datasets. To evaluate the proposed method, we conduct experiments on several SFDA benchmarks under three different domain shift scenarios: general SFDA, SFDA with severe label shift, and source- free partial set domain adaptation. For general SFDA, we test our method on the following datasets: Office-31 (Saenko et al., 2010). Office-Home Venkateswara et al.] 2017), VisDA2017 (Peng et al.) 2017), and DomainNet- (Litiico et al. 2023). VisDA2017 is relatively large-scale classification dataset with 12 classes, consisting of 152K synthetic images and 55K real-world object images. We use the synthetic images as the source domain and the real images as the target domain. Office-31 contains 4.652 images from three domains (Amazon, DSLR, and Webcam) across 31 categories, while Office-Home comprises 15,550 images from four domains (Real, Clipart, Art, and Product) with 65 classes. DomainNet-II 126 is subset of the larger DomainNet dataset that includes over 600K images across 345 categories and six domains (Clipart, Infograph, Painting, Quickdraw, Real, and Sketch) (Peng al., 2019). Following the setup of previous work (Litrico et al. (023). we use 126 selected classes from four of these sub-domains for our experiments.
We further evaluate our method on more complex SFDA tasks. For SFDA with label shift, we employ the ViSDA-RUST dataset, which presents a severe label imbalance in the target domain (Li et al.
2021). For source-free partial set domain adaptation, we follow the setup in Liang et al. (2020) for the Office-Home dataset, where only the first 24 classes are retained in the target domain. In addition, we use " "->" to indicate the adaptation direction from the source to the target domain.
Implementation Details. To ensure fair comparisons, we use the same DNN architectures and training schemes as previous state-of-the approaches (Liang et al.) 2020 Yang et al. 2022 Hwang et al. 2024). Specifically, we adopt ResNet-50 as the backbone model for the Office-31, Office-Home. and DomainNet--2e datasets, and ResNet-101 for VisDA. We replace the original fully connected layer in ResNet with bottleneck layer followed by batch normalization, and then add simple linear layer with weight normalization for the classification. For adaptation training on the target domain, we the SGD optimizer with the same learning rate scheduler as in Liang al. (020). For evaluation, we report the average accuracy for Office-31, Office-Home and Domain 126. For VisDA2017 and ViSDA-RUST, All experiments are run with three random seeds, and the average results are reported. Further implementation details, including hyperparameter selection, can be found in Appendix B
report both per-class top-1 accuracy and the overall average.
5.2 OVERALL EXPERIMENTAL RESULTS
The experimental results are summarized in Tables 1 and Table C2 in Appendix C.I with the best results highlighted in bold. Our proposed method consistently outperforms all baseline methods, especially on the large-scale datasets VisDA2017 (+1.2%) and DomainNet-126 (+1.9%). For VisDA2017, dataset with only 12 classes, conventional negative sample selection methods that treat the entire batch as negative samples often introduce significant noise and uncertainty. By incorporating the negative sample uncertainty loss, we investigate this issue and see a notable performance boost. Furthermore, our method excels in more challenging tasks, characterized by overall lower accuracy, such as Ar + CI and Pr + Cl on Office- Home (as shown in Table 3 and Table C2), and it consistently performs well across nearly all tasks on DomainNet- 126 (as demonstrated in Table 4
In more complex scenarios like VisDA-RUSE (with severe label imbalance), we observe a per- formance gain of +2.1%, while for the partial set Office-Home setup, our method shows +0.6% improvement. These results further confirm the robustness and generality of our proposed method, particularly in handling highly imbalanced target domain data and challenging SFDA tasks.
Additional experimental results and analyses, including self eeeficrediction accuracy, data augmentation consistency, variance control effect, hyperparameter sensitivity, performance under various similarity
8

---

Published as a conference paper at ICLR 2025
Table 1: Classwise accuracy (%) on the VisDA2017 dataset (ResNet- 101): synthetic (source) - real (target)
Method
plane
beyel
car
horse
knife
meyel
person
plant
sktbrd
train
truck
Per-class
3C-GAN (1i et
20/11b
94.8
73.4
68.8
74.8
93.1
95.4
88.6
84.7
89.1
84.7
83.5
48.1
81.6
SHOT [ Liang el al 2020
94.3
88.5
80.1
$7.3
93.1
94.9
80.7
80.3
91.5
89.1
86.3
58.2
82.9
Nett et al 2021
94.0
87.8
85.6
66.8
93.7
95.1
85,8
81.2
91.6
88.2
86.5
56.0
84.3
G-SFDA Yang
2021b
96.1
83.3
85.5
74.1
97.1
95.4
89.5
79.4
95.4
92.9
89.1
42.6
85.4
NRC Yang et d. 202
96.8
91.3
82.4
62.4
96.2
95.9
86.1
80.6
94,8
94.1
90.4
59.7
85.9
CPGA (Quu
95.6
89.0
75.4
64.9
91.7
97.5
89.7
83.8
93.9
87.7
69.0
86.0
AdaContrast (C Tien el
2022
97.0
84.7
84.0
77.3
96.7
93.8
91.9
84.8
94.1
93.1
94,1
47.9
86.8
CoWA-JMDS LLo
2022
96.2
89.7
83.9
73.8
96.4
97.4
89.3
86,8
94.6
92.1
88.7
53.8
86.9
DaC Zhang al.
96.6
86.8
86.4
78.4
96.4
96.2
93.6
83.8
96.8
95.1
89.6
50.0
87.3
AaD Yang al. 2012
97.4
90.5
80.8
76.2
97.3
96.1
89.8
82.9
95.5
93.0
92.0
64.7
88.0
C-SEDA Karim al.
2023
97.6
86.1
72.2
92.1
84.7
93.0
93.1
63.5
SRDDA Hwang al.
2024
96.8
89.3
82.9
81.4
96.8
95.7
90.4
81.3
95.5
93.7
88.5
64.7
88.1
I-SFDA
Mitszzuni et 2024
97.5
91.4
87.9
79.4
91.2
97.2
92.2
83.0
96.4
94.2
91.1
53.0
88.4
UCCon-PDD (Ours)
88.6
96.9
93.1
83.8
97.6
92.6
Table 2: Classwise accuracy (%) ) on the VisDA-RSUT dataset (ResNet- 101)
Method
plane
beyele
bus
horse
knife micccl
person
plant
sktbrd
train
truck Per-class
Source only (ee et al. 2016
79.9
15.7
40,6
77.2
66.8
11.1
85.1
12.9
48.3
14.3
64.6
3.3
43.3
SHOT (Liang et 2020
86.2
48.1
77.0
62.8
92.0
66.2
90.7
61.3
76.9
73.5
67.2
67.6
CoWA-JMDS
2022
63.8
32.9
69.5
59.9
93.2
95.4
92.3
69.4
85.1
68.4
64.9
32.3
68.9
NRC Yang et il. 2021a
86.2
47.6
66.7
68.1
94.7
76.6
93.7
63.6
87.3
89.0
83.6
20.0
73.1
AnD Yung 2022
73.9
33.3
56.6
71.4
97.0
91.9
70.8
88.1
81.2
39.4
73.4
SF(DA) (Hwang al.
2024
79.0
43.3
73.6
74.7
92.8
98.3
93.4
79.1
90.1
87.5
81.1
34.2
77.3
UCon-SFDA (Ours)
84.1
37.1
87.4
70.6
95.4
92.9
94.4
83.0
93.7
92.0
86.7
35.3
79.4
measures utilized in dispersion control term, and complexity analyses, are provided in Appendix C A further reduction in the number of hyperparameters within our algorithm, along with two enhanced automatic variants of UCon-SFDAA is detailed in the Appendix D
Table 3: Classification accuracy (%) on the Office-Home dataset (ResNet-50) under source-free partial-set domain adaptation
Method
|Ar-SCI Ar-Pt Ar-Rw CI-+Ar CI->Pr CI CI-RR
Pr-AAr
Pr-+CI
Pt- Rw-tAr Rw-sCI Rw Avg.
SHOT Liang et al. 2020
64.8
85.2
92.7
76.3
77.6
88.8
79.7
64.3
89.5
80.6
66.4
85.8
79.3
Yang
67.0
80.5
65.6
90.2
83.5
64.3
79.7
UCOn-SFDA (Ours)
65.6
87.8
91.0
78.6
79.3
87.6
80.2
65.9
87.3
83.2
69.1
88.7
80.3
5.3
ANALYSIS
Ablation Study. To evaluate the effectiveness and necessity of each component in our algorithm, we conduct an ablation study across four datasets. The results, shown in Table demonstrate that both partial label supervision training and dispersion control enhance the performance of the baseline approach (Ccr.). While Li., can better handle severe label shift scenarios, as seen in the VisDA-RUST dataset, Lio performs better on more difficult tasks. Notably, adding the dispersion control term alone improves or matches the performance of most negative sample denoising and umcertainty-basee methods, such as those from Roy al. (2022): Litrico et al. (2023); Chen al. (2022); Mitsuzumi et al. (2024), without requiring any additional networks. Combining both positive and negative uncertainty control can boost each other and enhance the performance.
Negative Sampling Dispersion Control. To further evaluate the effect of the dispersion control by Cie. we calculate the variance in prediction similarity between unchor-true-neer -reeaaatvvveee pairs during adaptation. Figure Belllustrstes that introducing Lh succesfully reduces this variance. Furthermore, the SF(DA)² method (Hwang et al. (2024) approaches the problem from graph-based perspective and introduces a quadratic regularized term the predicted probability similarity of anchor-negative pairs. This is equivalent to directly minimizing the variance. Our experimental results also demonstrate the effectiveness of our data augmentation-basee dispersion control.
Positive Supervision Uncertainty Relaxation, As shown in Figure Sa the top-1 self peeffcrrdicted label is more accurate for certain data (blue dot line) than for uncertain ones (yellow dot line), indicating that uncertain data require additional supervision during adaptation. To further validate the proposed partial label supervision for uncertain target data, we define neighbor label set that contains top-1

---

Published as a conference paper at ICLR 2025
Table 4: Classification accuracy (%) on Office-31 (left) and DomainNet-126 (right) using ResNet-50
Malled
J4 -
D
WD
W W-
DDD
----4
Axg.
Method
SS
C-55
P-C
P-
R-5
R-C
R
AAg
(Iane
Source colt
He.tt
2016
50.1
06.9
53.0
75,0
55.5
62.
55.6
SC-GANX
57.7
No.
76.7
76.1
30.1
67.7
77.3
89.4
63.5
Yiug maa01
100.0
76.5
59.9
Ads BasCrissa Letirr N00
669
53.0
68.6
90.5
782
698
578
93.9
77.9
GPUI [Jinsreen
67.5
64.0
68.8
76.5
63.
74.2
70.4
596
ISEDA
2005
SRDA [ianng
2014
67.7
99.6
67.8
83.5
602
688
70.5
883
UCan-SFDA (Ours)
94.8
95.4
88.9
10000
77.1
73.1
90.6
UCCon-S (Chrrs)
68.1
68.5
09.3
61.0
643
782
71.1
71.
Table 5: Ablation study results across different datasets and tasks
ViaDA2017
VisDA-RUST Real
DomainNet-1l 126
Officellome
Method
Real
Avg.
87.6
75.5
78.9
67.8
06.9
58.6
57.9
22.6
Cuo
Les
Eip
88.1
79.1
80.8
09.5
68.8
60.2
99.3
23.1
61.5
self-predicted labels of the neighbors. We compare the label information provided by this neighbor label set with our proposed partial label set. By comparing the two lines for neighbor label set accuracy marked with 'x' Figure [b] we observe that for uncertain data, the neighbor label set becomes increasingly unstable as training progresses, with accuracy sometimes even decreasing. This explains why we choose not to rely on neighbor labels in our algorithm. Instead, we use the sample's own TPP-Krt, predictions to form partial label set. A closer look the difference between the two blue lines and the two yellow lines in Figure 3b reveals that the partial label set provides a greater accuracy gain for uncertain data than for certain data. Interestingly, the accuracy of the neighbor's labels is consistently higher than the overall accuracy of the model's self-prediction which explains why we apply relaxed supervision through partial label loss only for uncertain data.
0:00-
Labe Sect Correct
Vartance Beealii
with
Ovars Sadeiia cias
Oreaman Lea secattaaaeee Accurt
Cartan Duti Sabeind Accartoy
Ortain aae Part Seckeoaney
com
withoul Diseesse
Theining Process
Thaining Process
Thining Process
(a) Self- Accuracies
(b) Partial Label Set Quality
(c) Similarity Dispersion Control
Figure 3: (a) Self-prediction accuracies across data with varying levels of predictive uncertainty on Office-Home (Ar Pr). (b) Comparison of the partial label set and neighbor label set quality across different uncertainty levels. (c) Comparison of prediction similarity variances between anchor-true negative pairs, with and without the dispersion control term Lpc on Office- Office-Home (Ar - Cl).
CONCLUSION
In this paper, we thoroughly analyze two types of uncertainty in SFDA arising from the use of positive and negative samples. By examining the uncertainty in the negative sample distribution during training, we construct an outlier-robust worst-case risk and derive an informative upper bound for it. This analysis not only explains why current contrastive learning methods significantly enhance dispersion control approach
SFDA performance but also leads to the design of an augmentation- to mitigate the uncertainty introduced by noisy negative samples. Furthermore, by investigating the prediction uncertainty of positive examples, we identify a partial label set as the optimal solution for the target data. This insight uncovers previously overlooked uncertain information in existing algorithms and motivates us to propose novel criteria for distinguishing uncertain data, thereby using partial labels to relax the supervision from positive examples.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
Yi is the Canada Research Chair in Data Science (Tier 1). Her research was supported by the Canada Research Chairs Program and the Natural Sciences and Engineering Research Council of Canada (NSERC).
REFERENCES
Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society: Series B, 28(1):131-42,, 1966.
Jose Blanchet and Karthyck Murthy. Quantifying distributional model risk via optimal transport. Mathematics of Operations Research, 44(2):565-600 6600, 2019.
Dian Chen, Dequan Wang. Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 295-305, 2022.
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXhv:2003.044: 2020.
Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In Proceedings of the IEEÉ/CVF Conference Computer Vision and Pattern Recognition, pp-
3941-3950, 2020.
John Duchi. Lecture notes for statistics 31 1/electrical engineering 377. URL: https://stanfond.
educclas//stats) I/Lectures/fflll notes. pdf. Last visited on, 2:23, 2016.
John Duchi. Information theory and statistics. Lecture Notes for Statistics, 2019
John C Duchi and Hongseok Namkoong. Learning models with uniform performance via distribu-
tionally robust optimization. The Annals of Statistics, 49(3):1378-1406 2021.
Rui Gao. simite-sample guarantees for wasserstein distributionally robust optimization: Breaking the curse of dimensionality. Operations Research, 7166):2211-206 2023.
Rui Gao, Xi Chen, and Anton J Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research, 177 2024.
Hui Guo, Boyu Wang, and Grace Y. Yi. Label correction of crowdsourced noisy annotations with an instance-dependent noise transition model. Advances in Neural Information Processing Systems, 36:347-386, 2023.
Hui Guo, Grace Y. Yi, and Boyu Wang. Learning from noisy labels via conditional distributionally robust optimization. Advances in Neural Information Processing Systems, 17:8267--82622 2024.
Lars Peter Hansen and Thomas J Sargent. Robustness. Princeton university press, 2008.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
pp. 770-778, 2016.
Peter J Huber. Robust estimation of location parameter. In Breakthroughs in statistics: Methodology and Distribution, pp. 492-518. Springer, 1992.
Uiwon Hwang, Jonghyun Lee, Juhyeon Shin, and Sungroh Yoon. Sf(da)?: Source-free domain adaptation through the lens of data augmentation. In International Conference Learning
Representations, 2024.
Nazmul Karim, Niluthpol Chowdhury Mithun, Abhinav Rajvanshi, Han-pang Chiu, Supun Sama- rasekera, and Nazanin Rahnavard. C-sfda: A curriculum learning aided self-training framework for efficient source free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 241 24120-24131, 2023.

---

Published as a conference paper at ICLR 2025
Henry Lam. Robust sensitivity analysis for stochastic systems. Mathematics of Operations Research, 41(4):1248-122) 2016.
Jonghyun Lee, Dahuin Jung, Junho Yim, and Sungroh Yoon. Confidence score for source-free unsupervised domain adaptation. In International Conference on Machine Learning, pp. 12365- 12377, 2022.
Junnan Li, Richard Socher, and Steven CH Hoi. Dividemix: Learning with noisy labels as semi- supervised learning. arXiv preprint arXiv:2002.07394 2020a.
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9641-9650, 2020b.
Xinhao Li, Jingjing Li, Lei Zhu, Guoqing Wang, and Zi Huang. Imbalanced source-free domain adaptation. In Proceedings of the 29th ACM International Conference on Multimedia, pp. 3330- 3339, 2021.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In Proceedings of the International Conference on Machine Learning, pp. 6028-6039, 2020.
Mattia Litrico. Alessio Del Bue, and Pietro Morerio. Guiding pseudo-labels with uncertainty
estimation for source-free unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 7640-7650, 2023.
Yu Mitsuzumi, Akisato Kimura, and Hisashi Kashima. Understanding and improving source-free domain adaptation from theoretical perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Sloan Nietert, Ziv Goldfeld, and Soroosh Shafiee. Outlier-robust wasserstein dro. Advances in Neural Information Processing Systems, 36, 2024a.
Sloan Nietert, Ziv Goldfeld, and Soroosh Shafiee. Robust distribution learning with local and global adversarial corruptions (extended abstract). In Shipra Agrawal and Aaron Roth (eds.). Proceedings of Thirty Seventh Conference on Learning Theory, volume 247, pp. 4007-4008, 2024b.
Jiangbo Pei, Zhuqing Jiang, Aidong Men, Liang Chen, Yang Liu, and Qingchao Chen. Uncertainty- induced transferability representation for source-free unsupervised domain adaptation. IEEE Transactions Image Processing, 32:2033-2046 2023.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint arXiv: 2017.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 1406-1415, 2019.
Ruizhi Pu, Xinyu Zhang, Ruofei Lai, Zikai Guo, Yinxia Zhang, Hao Jiang, Yongkang Wu, Yantao Jia, Zhicheng Dou, and Zhao Cao. Yes sir! optimizing semantic space of negatives with self- involvement ranker. arXiv preprint arXiv:2109.06436. 2021.
Zhen Qiu, Yifan Zhang, Hongbin Lin, Shuaicheng Niu, Yanxia Liu, Qing Du, and Mingkui Tan. Source-free domain adaptation via avatar prototype generation and adaptation. arXiv preprint arXiv:2106. 15326. 2021.
Subhankar Roy, Martin Trapp, Andrea Pilzer, Juho Kannala, Nicu Sebe, Elisa Ricci, and Amo Solin. Sincertainty-guidee source-free domain adaptation. In European Conference on Computer Vision, pp. 537-555, 2022.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European Conference Computer Vision, pp. 213-226, 2010.
12

---

Published as a conference paper at ICLR 2025
Kumiaki Saito, Donghyun Kim, Piotr Teterwak, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Tune the right way: Unsupervised validation of domain adaptation via soft neighborhood density, In Proceedings of the IEEE/CVP International Conference Computer Vision, pp. 9184-9193, 2021.
Cosma Rohilla Shalizi and Arych Kontorovich. Almost none of stochastic processss,, 2006. Lecture notes, available at: https: //www. stat cmu, wtt.p/eeee/eeeaaaaaaaaaaaottoooa 1/ 1/almost-nort paf
Alexander Shapiro. Distributionally robust stochastic programming. SIAM Journal on Optimization, 27(4):2258-222 -27-5 2017.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 5018-5027, 2017.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization, arXiv preprint arXiv:2006. 10726, 2020.
Haifeng Xia, Handong Zhao, and Zhengming Ding. Adaptive adversarial network for source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 9010-9019, 2021.
Gezheng Xu, Li Yi, Pengcheng Xu, Jiaqi Li, Ruizhi Pu, Changjian Shui, Charles I Ling, A. lan McL.ed, and Boyu Wang. Unraveling the mysteries of label noise in source-free domain adaptation: Theory and practice. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Shiqi Yang, Joost van de Weijer, Luis Herranz, and Shangling Jui. Exploiting the intrinsic neighbor- hood structure for source-free domain adaptation. Advances Neural Information Processing Systems, pp. 29393-29405, 2021a.
Shiqi Yang. Yaxing Wang, Joost Van De Weijer, Luis Herranz, and Shangling Jui. Generalized source-free domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8978-8987, 2021b.
Shiqi Yang, Yaxing Wang, Kai Wang, Shangling Jui, and Joost van de Weijer. Attracting and dispers- ing: A simple approach for source-free domain adaptation. In Advances in Neural Information Processing Systems, volume 35. pp. 5802-58 1802-5815, 2022.
Li Yi, Gezheng Xu, Pengcheng Xu, Jiaqi Li, Ruizhi Pu, Charles Ling, A. Ian McLeod, and Boyu Wang. When source-free domain adaptation meets learning with noisy labels. arXiv preprint 13X1v12301.1331. 2023.
Runtian Zhai, Chen Dan, Zico Kolter, and Pradeep Ravikumar. Doro: Distributional and outlier robust optimization. In International Conference on Machine Learning, pp. 12345-12355. PMLR, 2021.
Ziyi Zhang, Weikai Chen, Hui Cheng, Zhen Li, Siyuan Li, Liang Lin, and Guanbin Li. Divide and contrast: Source-free domain adaptation via adaptive contrastive learning. Advances in Neural Information Processing Systems, 35:5137-5149, 2022.
13

---

Published as a conference paper at ICLR 2025
APPENDICES TECHNICAL DETAILS AND ADDITIONAL EXPERIMENTS
A TECHNICAL DETAILS
We introduce key notations for the subsequent subsections. Let Ryo represent the set of all non- negative real values. Given (v1....,,7) up) and E [1, +oo], the 1 norm is defined as ||ا|| = (1 1/0 for I q < oc, and |/ = max, |tyl when g +o0 Let (2,g p)) represent probability measure space, where Ω is set, G is the o-algebra of subsets of Ω, and is the associated probability measure. For q > 0. let ( u),, or simply (L()), denote the space of Borel-measurable functions f Ω R such that f f[
oc. For random variable Z ~ μ., we may interchangeably write the expectation and variance of f(Z) respectively as (,(f(Z) and Va {f()), , (f) and (f). or Ez~u1 {Z() and Vznz {f(Z)). We use ((2) to denote the of Borel probability measures on ft, and let P (S() represent the subset of P(2)) with finite 9th moment
for q 0. That is, μ Pp )) if and only if Rz-p < co. Clearly, Pg( C P. if g 2
A.1 NOTATION TABLE
The notation table provides summary for the key notations used throughout the paper, with the symbols, descriptions, and the first appearance place included the first, second, an the third columns, respectively.
Notations
Descriptions
First appeurance
C Rd
d-dimensional input space
Section
y [A]
label space for K-clussification
Section
underlying distribution over X x y related to source domain
Pgy D:
unavailable source domain data D. è [x] vi,
Section
underlying distribution over X' x y related to target domain
P:: Dr
unlabeled target domain data D, A {x])
Section
fx(x: x: (x)/ 0 0 (x:
predicted probabilities of source/laaget/ggenrall model
Section
N'
1x [7h, 0)
source/targat/eenee classifier:
X X y
max [x; 1/1l (x: 0, (xx 0)]
Section
So(x
similarity between x' and x
Section 4.1. Eq.
e.g., 3x(x =< f(x': /)x [(x;0)
(empirical: P.)
distribution of input (target)
Section
("Ax)), or simply Pa
conditionul distribution for positive sample X, given
Section Eq.
(empirical: P*)
P'(xx), or simply p (empirical: P))
conditional distribation for negative sample X', given
Section 4.1. Eq.
L Cis
positive/negativee contrastive
Section 4.3.
Remark
Ct, I Live
partial label/dispersion control loss
Section 4.3.
Remark
Doem Eiow
overall positivennegative uncertuinty control loss
Section 4.4
Eq.
4
uncertainty control source-fre free domain adaptation loss
Section 4.4
Eq.
99
partial Label/dispersion lontro//negative contrastive
Nou Doo Nez
Section 4.4
Eq.
loss coefficient
number neighbors for each unchor point
Section 4.1
Kyn,
update number for partial label set
Section 4.4
Page 7)
uncertain sumple selection rutio
Section 4.4
Puge 7)
decay expoment of negative contrastive loss
Section 4.4
Page 7)
feature/predicted probabilities/
T /9/o/ /
Appendix B.
Algorithm
partial label set/uncertainty sample bank
AUG(x)
augmentation of input sample
Section
Remark
A.2 PRELIMINARIES ON DISCREPANCY METRICS AND LINEAR PROGRAMMING
We begin by presenting some definitions and optimization results of the p- Wasserstein distance and ---iivegeence, which can be chosen for the discrepancy metric in 3 These materials will be used in the proof of Theorem 4.1
14

---

Published as a conference paper at ICLR 2025
Definition A.1 (p- Wasserstein distance (Blanchet & Murthy 2019)
Let 52 denote Polish space (i.c., a complete separable metric space), endowed with a metric c: Ω2 x - Ry0, also called a cost function. Then, for p 2 1 and any P. QQ& Pp(S), the Wasserstein distance of order p for P and Q is defined as
Q) &
inf IEEC)((P.))
( (c (SI, 1/2
where CpI(P, Q). sometimes called the coupling set of Pnd Q, comprises all probability measures on the product space Ω x Ω such that their marginal measures are P(-) and Q(-). Here, c²(..-) represents ))
Duchi 2019)). Let P and be probability distributions on measurable space (92, 9), where g is the o-algebra of subsets of Ω. Let y R+ be convex function satisfying (0(1) and (t( = +oo for O. Without loss of generality, assume that P and Q are absolutely continuous with respect to the common dominating measure μ.. Let p and denote the density or mass functions of P and Q with respect to the measure μ. respectively; that is, ((dz) q(x))u((x) and P(dx) p(z)dp(x)). The --divergence between P
Definition A.2 (p-divergence (Ali & Silvey 1966
and Q is then defined as
(P(1))Q) = / q())
p(z)
dp(x) + ((0)))((q=)) 0},
(())
where (0o) represents limg-400 (t)/tt
Example A.1 (Duchi 2019 Chapter 2.2). Taking yo to be different forms gives popular examples of --divergeenes::
Kullback-Leibler (KL) divergence: taking stt) log t gives
D*(P)IQ) Dx.( / /lg(p//)ddz
The total variation distance: taking <t()) 1//t 1| yields
(p(()) 2 UP Q1
1
P _- 1 1/qd4 = sup PP(A) Q(A)).
2.
4
ACc
The Hellinger distance: taking sp(1) (Vt 1)2 tt- 1- 2/t + 1 leads to the squared Hellinger distance
(P(l1) & (²(P))Q) = ((VD///-- dp.
The x divergeee::: taking p(t) = (t 1)2 produces the x² -dvvergeee
(P((1) & xP(1) / - 1)d
Lemma 1 (Strong duality for robust risk based on p- Wasserstein distance; Gao et al. 2024 Lemma
1). Consider the p- Wasserstein distance W W,(..)) with E [1, oc) defined in Definition A. Given
an upper semi-continuous loss function ft Ω R. nominal distribution P PP))), and radius
8 > 0, the resulting robust risk based on the p- Wasserstein distance Wp(..-) ) is defined as
Ue
sup QEP(()
E 1 58..
and the dual problem is defined as
UD min 720
70" + Ezap
sup {f((2') 76 LeEn
Then, Dp tpp
15

---

Published as a conference paper at ICLR 2025
Lemma 2 (Strong duality for robust risk based on -pdiieegeenc;; Duchi & Namkoong 2021 Proposi- tion 1: Shapiro 2017, Section 3.2). Consider the vp-divergence D-1-11-) defined in Definition A.2
Given a loss function Q + R. nominal distribution P the measure space (S2, g), and radius > 0, the resulting robust risk based on the u--diverencce D (-ll.) is defined as
Up sup (Bz-o(((2)) (Q(( s 6] QEP
and the dual problem is defined as
[A(2) -1 7
} + + +
UD &
inf 20.HER
Ep
where (t) sup, & (ts - ((s)} for any t E IR is the Fenchel conjugate. Then, Vp = Moreover, the supremum in Up finite, then there exist finite 7 Z O and r) E E that attain the infimum in vn- Lemma 3 (Hansen & Sargent, 2008 Proposition 1.4.2). Let (S2, G, 10) represent a-finite measure space, where 32 is set, G is the o-algebra of subsets of 52, and p is the associated measure. Suppose f Ω + IR. is a bounded measurable function. Then the following results hold:
(i) The variational formula:
( log ja
{ (Dacellla) + A())du(w)) (AI)
(xp(-A([]3du(w)) inf VEP(S)
(ii) Suppose v² is the probability measure P which is absolutely continuous with respect to a and satisfies
du* dj (w) A exp(-A(ws)
expl [(w))du(()) for E Ω.
Then the infimum in the variational formula A7 is attained uniquely at v².
A.3 PROOF OF THROREM
Before presenting and proving the formal version of Theorem 4.1 we first examine the robust risk
given in 2 for different choices of the discrepancy metric d (3), including x -dvvevegee, KL- divergence, and p- Wasserstein distance with p E [1, +00). Proof techniques in Duchi & Namkoong (2021): Zhai al. (2021) Gao (202)); Gao al. (2024): Lam (2016); Guo et al. (2033) used.
Lemma 4. For different choices of the discrepancy metric d in 3 we have the following results on the robust risk R; P", ,8) given in 2
(i) If d the xi-divergence and 8 < (Sx (B-
then
P" Ep- (X + /
(ii) If d the KL-divergence, then,
(# P), = Fp = (X) + / 2- + 0(8).
with 8 > 0 being small.
(ii) Suppose d the p- Wasserstein distance with p E (00) and the cost function o(.;) ) in
Definition A. is chosen norm || with the dual norm ||.. Assume the following smoothness conditions:
a. For any X x X X. 3.4,, 1, My > and ( E [1. p). such that
V x))1= S Mi + +M2lix - x=1|5-1
b. There exists no > 0 and Ml, > O, such that for any X", x E X. if || xx'l S no.
then [/ (x ;x) V (x xxxxx=xxl>=+>>> S Malli - x x1
Let denote the Hölder number of p, that is 1 +
Then
(0; P" P)) < Ep- (X' (Ep- V |
16

---

R0 P08 S Ep- (S :x)] + / Epp (8x(X ( 8 Published as a conference paper at ICLR 2025
Proof of (i) with d set the X² -ivvergee::::
For the x -iveergeene, we have cpt) (t - 1)² for t 2 0 and (ptt) too for t < 0 by Example
Then the Fenchel conjugate of yp is given by
(s
t +
8
4 (t) = sup (ts - ((s)) = sup ts (s 1)2} = sup +20
+
+
sER
.20
4
12
t, for t 2 -2
(((++))+
4
(A2)
1, for t < -2
Step (i): Upper bound on the primal problem.
If the discrepancy metric d in (3) is chosen as the x2 diverrgge,, then the robust risk P", is expressed as
sup q-
[Ro
(A3)
The expectation BQ- (So(X";))) in A3
can be expressed as:
dQ- dP.
Eq- xx) Ep-- So (x((;;x)
(dp-
[QQ
dP. -1)
Ep- (S-( + Ep-
So(X
dQ-
}
= + Ep-
- Ep- (8p
dP.
where the first inequality holds via a change measure and the fact that Q < P
denotes the
Jp.
Radon- derivative, and the last equality is true since Ep- [QQ- - 1 0.
By the Cauchy-Schwarz inequality, we further obtain that
R2 P7, - -p- (X
- E, - (80(xx)x)) Ep- (dQ.
dP. -1) 2
)
Bpr-
Bp- [So(X:;x) - Ep- Ep- ;x) x2(1-11-))
Ep- [So(X::x) - Ep- - [Sx(
where the second equality is due to the definition of x -divergence given in Example A.l and the last step is due to the constraint in A3 Therefore, by A3 we obtain that
Ep + VoV,
(A4)
where , A Ep- (x x)) and V Ep- [So(X :x) Ep- (S x]
Step (ii): Attaining the equality in the upper bound using duality.
17

---

Published as a conference paper at ICLR 2025
Next, we prove that the equality in the upper bound in (A4) can be achieved by leveraging the strong duality result of the d-diieegencc based robust risk. Specifically, according to Lemma and (22).
Sx(Xx;x)= 17
}
R2 P", - inf
Ep
74p"
+ 78 + "
720
[x -1
}²
inf y20
7
7 +
- Y +
+ 78 +
inf
Ep { So(X - 1] + +2,
1²
-7 -7 70 +
>20
47
inf
44 Ep { (X ;x) xx) +11 877 + 11
>20,EER
inf >20.EE
(v; 7),
where the second last equality holds by taking î/ 2 17 1--22 27.
We now examine the minimum of ()); by fixing one argument. First, given îj, taking derivatives of (>; 7)) with respect to 7 gives that the optimal y infimize the preceding expression is given by:
Ep ;x 7/+
4(1+6)
Then substituting 7* into (111 7) gives
V (1+ + 8) (Sxx{(((Xx;))) + 1)
1
R2 P 5) inf
(A5)
HER
+
Next, g(î) 2 V (1++))Ep(Son (x () - T) + n. By taking
0)
V
(A6)
8
where μ and V are defined after (44.. we obtain that
2
-(7¹)
(1 + 5) p So (X ; x) 77 { is
(1 + S) So X xx) - -n² + +ē¹
1V12 2
(1 PD)EP So 1 +
+ +
(1+ 8)
Bp(So(X;5x) 11)
2
+ + 2
V
Ep So (xx; [1]
Ep
+ +
V (V+ 8 + V T
(1+ 6)
+ VOV.
where the first step holds since 77' = p - / < O, and the fifth step is due to the definitions of μa
and V.
Step (iii): Mean-dispersion form of the robust risk.
18

---

Published as a conference paper at ICLR 2025
With 17" = = p / in (A6), the dual objective (5)
in its infimum form achieves the equality in
A4), which the upper bound of the primal problem A3 in its supremum form. Consequently, obtain that
()) (So + V (Ep- (So(X)x) Ep- Ba---B--(--- (x)1
The proof is completed.
Proof of (ii) with d set as the Kl.-divegeence:
the discrepancy metric in expressed as
3 is chosen as the KL-divergence, then the robust risk (0; ,5,
R2 P" 8) = sup
Eq- (So(X)x))} Dxl(QTIPT) S 5d]
Q &P
Eq (So(X lB00 {
dQ
sup Q PP
log
dP-
8
(A7)
By a change of measure and denoting the likelihood ratio L(w) and the constraint in A7 can be expressed as
(Q"(w) aP"(w)
for w E X, the objective
dQ
Eq- So((X
Ep-
Sa(Xi;x)
Ep-
So(X (x)X
dP-
Eg- log
dQ dp-
}
Ep-
[
log
dQ-
dQ-1
Ep-- (Lx) log (L(X)))
dP-
dP-
Therefore, the expression of the robust risk R. (0; P 5) can be rewritten
max Ep- Sx (x) LEC
RT(0:PT,0) =
(A8)
s.t. Ep- L(X log (L(X')} 5
where -= (L Ep (L(X)) = =; (L)))))))))))))))-----))))))))))
0 a.s.).
Since (A8) is convex optimization problem with respect to L, by introducing the Lagrange multiplier 7 be further expressed as:
Ep- ( xx)(x))) - Ep [L(X') log (L(X))] -6] (A9)
R2 P
max LEC,200
Step (i): Optimal form of the likelihood ratio L".
Suppose
find 7- 0 and L" L such that L maximizes A9 for fixed y
y" and Ep- [(xx) ( log [L(X)) = 8. Then, for any L L satisfying the constraint
Ep- L(X)) (L(X-)) 8 in (8 we have that
Ep- Se(X xx)
} 7- { Ep- [L'(L"XX)) (L*(X))} - -6}
Ex-
>Ep (So(X -7" [L(X [ (L(X)))] - b)
2Ep-
and hence, L* is the optimal solution of (A8)
We first assume the existence of such 7* 2 0 and consider the form of the corresponding L*.
Let f(L:7) A Ep- (o-(Soo (xx(xxx))(XX)) m Ep- [L(X*) (L(X*))] - denote the objective
19

---

Published as a conference paper at ICLR 2025
function in A9 be expressed as
For a fixed y E R, consider the form of L* Le f(L:7)); which can
L' E arginax Ep- {p-{So(X (x( --" { -x log -6}
LEC
+ L' E arginax -7" (Bp-{ -So(X Ep- (08) })]
LEC
L'dP argmin
{ -o((x;))/+*) (07111111
Q (PE),(2)
By Lemma
3 we obtain that
{ (x((x)x)
Sx(X";; xx)
L'(X") exp
/Ep- exp
(A10)
?"
the unique optimal solution of L* e argmax f(L:?") for fixed y- since the similarity measure Sg is bounded function.
Step (ii) Existence of of
the y Step (i) exists, then the optimal L* is given in (A10), and the constraint and objective in A8 can be expressed as below:
8 Ep-
(*(X") log (L*(X)))
exp (So (X xx)//>>
So(X
(x((X;x))
Ep-
log Ep-
exp
Ep- lexp (So (X ;x); /
7"t
7*t
Ep- [So(X';x) exp (So (x"; xx)//"]] log Ep- exp
(X((X"x))
7"'
Ep- (exp (So(X X xx) /
Ep- [So (Xx;xxxx exp :0- So (X (xxx))]] log Ep- exp (ē So(X [X]x)}}
0.
Ep- (exp (0- (
A oh' (0) ((()-
(A11)
In addition.
Ep- [So(X":X) exp (So ;x)xx")))))
Ep- (xx(X
Er- lexp ;X=xxx x
=f'(@).
(A12)
where we let o e 1/7, ē 1/7", and (()) log Ep- |exp {e xx) Here A is the cumulant generating function of So(X"; :x), which is infinitely differentiable and strictly convex for non- constant " x), and passes through the origin (Shalizi & Kontorovich 2006). Moreover, using power series expansion, we obtain that:
((e)
Σ 1=1
(0) e'
where (4)) denotes the jth derivative of A, and (1) is referred as the jth cumulant. It can be verified that
(0) Ep- [Sx(X x));
A (2) (0) E- |So(X ix) - -P- xx) x)) 0;
())) Ep- |So(X -E [Ex
20

---

Published as a conference paper at ICLR 2025
By the strict convexity of h, we have that d (oh A (e) is strietly increasing in g. Moreover, by A111 using the Taylor series expansion, we obtain that
f((e)} /d (e) O, and hence (f)
8 = @ ((
4000
4000
0
Σ
10+ (0) 0" Σ
(0)
e
== i!
1=
400
+00
A (0) 0
Σ j!
(0)
d
1)
1
11
(0) d
(j-
j!
F=
1 f (0) 0² 1 1 (0) d² +00)
(A13)
2
Since ()) > 0 and the remainder is continuous in we conclude that there exists a small ø satisfying the equation (A13) for small enough 6, and that is the unique solution of (ATT). Corre- spondingly, for Y 1/0. the associated L* satisfies the constraint Ep- L L*(X*) log {L" (X') 8.
Hence, (0; P 5) Ep ;x))
Step (iii): Mean-dispersion form of the robust risk.
Now, we examine the form of the robust risk. By (A13). we have
28
2n (3)
26 0
= -²
00
+0 8²
0+ 0
(0)
(0
and further obtain that
28
1 1/(1+ 1+
20
0 + 0
0
(((2(())
()
28
24(3)
0+ 0+ (
((2)
3 (2) (0)
28
36 0 + 0(00
(02)
28
2n (0)
((A 8 0
(0)
Hence, by (12.. we have that
7 P 8) -Ep So(( ;x (x))
-h((0) (0)( (1(3) (
0² 0
-0 + / (0 066)
- Ep- -Ep- x)} + + 2Ep- -Bp 2| 6 0()).
Therefore, the proof is established.
Proof of (iii) with set as the p- Wasserstein distance:
21

---

Published as a conference paper at ICLR 2025
If the discrepancy metric d in (0; P 6) is expressed as
3 is chosen the p- p-Wasserstein distance, then the robust risk
(0; P 8)
sup Q (
(So-(So(X W,(Q", P
(A14)
Let AR; - R2 P 5) Bp ( denote the difference of the robust risk and the nominal
risk. By Lemma
|
we have that
sup {So(x);)) - > - x x(11]] Ep- ( (Xo(X;;)) ;x)
ARa = min 720
70th + Ep-
EE
[{
- so(X -71x8= 1115-xx1PP' - xIIP1)
min
y6P + Ep-
(A15)
720
En
Step (i): Upper bound on (x";xxx - ;x(x";x):
For any X x E X. by the mean value theorem, there exists x E X between x and x" such that
So(x - Sx(xx (V (VSa(X xx)) - xx),
which implies that
; - So(x - (x))xxxxx)xx' -x
(VS0(x;;> ;x) - VS0(x ;x), -x'I
[S](x;x)) V ; ;x)]|+ xx'll
VSg(X (
xx) :
|| -x'll
(A16)
where the inequality in the penultimate step is due to the Cauchy- Schwarz inequality.
If | x1 S no. by the smoothness condition (b), we have that |V x) - ( S Myllx - =1
(A17)
If | x1 2 no. by the smoothness condition (a), have that
(X ;x) (x xx)ll= M1 + M2ll x x111-11
(A18)
Combining (A16
A17 and A 8 we obtain that
- >> (x xx) So(x :x) V (x x"
-[[[X* -x S 1o) -M |ス
x 11 1(133 - x 2 n7) (Mi - x1I +22 x (19
2 I + Iz.
where
I A 1[|x - x < vo) Ma1 - x x1
Ig |[|| x 2 no) (Mi, || - xI M22 x 191
For I₁ if S S 2, we have
2-p
Z < 1 - x 5 110) Ma
170
|| - x1
{ - x
< Ma 2 2- - PP.
P.
If > 2, we have I₁ < |
112
For Iz, we have the following upper bound:
Ia <1([x x211210 no) { {.thh | (IR--x111 x=11 1-1
[K=x11)
1 -x11
x-c
|| -x"ll + M12
1|
170
17u
< <(oi1To (p-1) (p-1) M22 (0-50
(12 ||x - P
22

---

Published as a conference paper at ICLR 2025
Combining the discussions above, we have that
(x ix) - ;x VS -x -x M xx - -x pp
(A19)
M (I|x - -x - 11²) ifp> 2,
where M - MM 2-p Ms, (M + M2 - ).
Step (ii): Mean-dispersion form of the robust risk when p E [1, 2].
When p [1, 2]. by (A15) and (A19) we have that
(VS ;x), X M 1 | [1] >11 - xxPr]]
AR
min 120
Y0P + E
sup EEE
(VSo(X ;x), X (7 - _0) - x -x10)
min 720
70 + Ep-
sup
ER
min 120
78" + Ep-
sup
( | - xx - (7 - Mt) - -X1P))
EE
|| (X ; 11) 1 -10) }} + NP88
min 12
70" Ep-
sup
120
min 720
y08 + Ey-
sup
(x))llt -
700/ - +1.58
I4 + MMPP
(A20)
| - } and the third step is due to where I4 - min,20 (788 + Ep- supp20
the Cauchy-Schwarz inequality.
By taking the derivative of the function in I4 with respect to l and setting it to zero, we obtain the optimal value of I, given by
= (1 (
Let q denote the Hölder number of that is p 4 !
= 1. Then, q p- + 1
P p-
and 1
We have that
sup VSo :x)]:+=))++>) 7t"}
120
= =| (X"))xx :x)]l+t" 7tt")p
|X
|XVSo(Xx;x)
= =\\VSo(XX; :x)l.
-7.
YP
yp
V (X: :x)]]
(xp) - - (X x 7
= V (x)) = (1-1) 2
Thus, we further obtain that
Z4 = min 78"h + (1 (1- (
Ep ( : xx]]
(A21)
Similarly, by taking the derivative of the function in A21) with respect to ? and setting it zero we obtain the optimal value of 7, given by
6 (p- 1) p (X.;x)))
p
23

---

Published as a conference paper at ICLR 2025
Hence, by substituting y into the corresponding expression and simplifying, we further obtain that
1 p
8-(1-) [Bp- VS 8
I1
+
-
1
(p-1) 1) [Bp (V 13)1 :x)]1
p pD-1
p
p
1
8 PEp V xx)1
p P-1 p
6 [Ep-1 (X; 1/9
p
=6 Epp-
VX "1
(A22)
Combining A20 and
A22
we obtain that
A S 8 VS + MM80
Step (ii):: Mcan-dispersion form of the robust risk when p E (2, oc).
When p E (2)),), by A15) and
A19), similar to A20 in Step (ii), we have that
AR S min 70P + Ep-
sup ER
{
(VSo(X)))))
720
+ Ml |x - + | - x [x]11]] 7/3 7/8 -x1r1))
< min >20
700 + Ep-
sup E
| x - -XII
+ M01x - x 1 1 + 1 x-113 7/ x x11))) }}}
(X :x)] + Mr Mr gopP) }}}
min
708 + Ep-
sup
720
120
(x=xxxx))ll xl1 + Me - -100 }1} + Mo.0
< min
700 + Ep-
sup 120
720
||V (X ;x))lll Mltt - (12 + t 1880
min
(11 72)00 + Ep-
sup 120
21.1220
min
010" + Ep-
sup { [ -
120
.M -21²- } + MM50
+ min
7262 + sup
7220
100
1I + Io + N680
(A23)
where
A min I5
1100 + Ep-
sup
7200
120
Ig min
02
sup 120
Mtt (112-120))
2200
For Is. similar to the discussion on I4 with p E [1,2] as in A22 we obtain that, for p E (2, oo),
Is 5 5 (Ep- V 1/9
(A24)
24

---

Published as a conference paper at ICLR 2025
For Is. by taking the derivative of the function in Is with respect to and setting it to zero, obtain the optimal value of t, given by t-
} 1/(x-2) leading to
I6
min 7220
128" =tt)t)² (2)
min 7220
{
(228P + M
2.
p-2
72
2.
p-2
12p
72P
min 1220 + (22P)
F-2
M
7/2
p-2
p-2
P
7220
2
2
min 7220
126 +
P-2
2
Y2P
p-2
M
(A25)
p
By taking the derivative of the function 72, given by
(25
with respect to 72, we obtain the optimal value of
22
Mo.
(p
2
p
yielding
I6 02
2
12P
M F M..2
(A26)
p
2
Combining (A24).
A26)). and A226). we obtain
AR₃ 8 (Ep- [X + N082 - 10.80
(A27)
Hence, the proof is completed.
Proof of Theorem 4.1
To show the results, we examine the outlier robust risk (4) for different choices of the discrepancy metric d in (3). Proof techniques in Zhai et al. (021) are used.
Proof of (i) with d set the x² -'vivergeecc:
If the discrepancy metric d in is chosen as the diverrgeee by (4 and Lemma 4 we have that { [R; (',))) 8) apr Pp(l) s.t. Pimin = (1 c) + cP'
inf (0; Pimin: 5, (2))
inf P'E),(X))
Ep (So + (So
3P' Pp(x) s.t. Pimii = (1 c) + cpr)
(A28)
We consider the following quantity:
Ri A inf (2),
{ [Epr(So(X'
; 3P E Pp(X) s.l. Pinin = -) + EPr
[1 1/0000 P' (So X xx) < $} ds 38' E Pp(X)) s.t. Pimin = (1- + cpr }.
in inf (P)))(0))
(A29)
where in the second step, we use the fact that for nonnegative random variable Z with cumulative distribution function F, Er(2*) k fr k (1 F(u) if the kth moment EE (Z*) exists.
Since Piiii = (1 P)' + Pr, we have that for any & 2 0,
P' (S xx) $ 8)
min
Puian (So(X;;) S $s).1. . 1
(A30)
25

---

Published as a conference paper at ICLR 2025
As in Zhai et al. 2021), p. show that the equality in (A30) be achieved by some P E Pp(2). Specifically, since Prain and Sg are continuous, there exists an s such that
Pinii (So(X);)) s*}
Define
p" p"(x") & Prannni (x") if Sø(x 5
(A31)
0, if So(x";)] > 8"
where Pirain represents the density or mass function of Pimin Let p denote the associated measure of p". Then, have
J (x)
dP dr (x))
So "x)
Pizain (Sa(X S 5*}
1.
Therefore, P defined in A31 the probability distribution achieving the equality in (A30). Thus, and utilizing (A30), Ri can be written as:
by substituting P into A29
R1 =Ep (Sa (X }
+00
[1 p (Xo(X";)) S s
1
Prain (So(Xi;x) 5 1 P S S 1 e) ds
Pizin (So(X";x) S 8)
1(8 8* 8')ds
e)s² 10" g
Piiin (8e(Xiix) S $) ds
1
8 Prein (X:xx 5 80) - -f² Piain (Sx(X;x)) S 5.) ds }
1
1
dP" train (X ;x) S 8)
(A32)
For the variance term in A28 we consider the 2nd moment:
R2 A Ep.
[So(X {x)}?
[1 pr (So(X S 8))ds
28
1 - 1 Priin (Se(X S {s} 1[s s s² )ds
1
(1 - ()²) /0 2:P.... (So(X); s S ds
1 1-6
1
2 P (X S 88 <0)) - .( 2sP (So (X):))))) s 8) ds
e
1
2 dp Ira (8o(X :x) Ss).
(A33)
6 Jn
26

---

Published as a conference paper at ICLR 2025
Thus, obtain the upper bound on the outlier robust risk R2(0: Pimin: 6. given in (A28
(0; Piriin 5, 3Bp- (So (0xx(xx))) + (Sxxx))(X;;x)
R1 + 66 Ri),
where R2 and Rg given in (A32 and A33 respectively.
Proof of (ii) with d set the KL-ddiergencce:
If the discrepancy metric d in 33 is chosen the Kl-divergence, by (4 and Lemma 4 have that
R2; Prrain 6,e)
inf (2E)
{
()) 3P' E Pp(x) s.l Pirin = (1 - p) + eP')
inf ('EP((())
(So :x) + p. (So((xxx)
38¹ Pp(X) s.l. Pirain - c) + eP'
Similar to the proof of Theorem 4.1 (i) with the x² diveergge,,,, we construct the distribution P in
(A31) and obtain the following upper bound on the outlier robust risk R2 (0; Piminn 6,e):
(2; Priiin 8,c) SEp- (Xx(Xxx)) + 28V (So(X:;x))
Ri +
25(Bt2 9t9)
where R2 and R2 are given in (A32 and (A33). respectively.
Proof of (iii) with d the p- Wasserstein distance:
the discrepancy metric d in is chosen as the p- Wasserstein distance, by and Lemma we have that
R2 Pi d,t)=
(R.00PP :): 3e E Pp(X) s.t. Priin = (1 -1--ep e) + tePt
(xE
< inf P')
{ (Se( + (E
0(
3P E Pp(X) s.l. Pinin (1- }
Similar to the proof of Theorem 4.1 with the x² diveerggen we construct the distribution P* in A31 and obtain the following upper bound on the outlier robust risk R:(0: P 8, E):
(2; P 6, <Ep. (So X :x)] + (Ep [ x))
Ri 1 Sy (XT;))111 (T
where R is given in (A32)
A.4 PROOF OF THEOREM
We complete the proof following the deviations for Theorem 3.2 of Guo et al. (2024). By Lemma] [ when the p- Wasserstein distance with 0 cost is used to construct the uncertainty the robust
27

---

Published as a conference paper at ICLR 2025
risk (p;xx in
5 for positive example xt, be equivalently written
(x;p;"" ,0) sup
(
p)
K
Pk Pa = =k)
EY =sup
pp b)} -/168/ - (110'=1))') =11](}
inf 720
70th + Yxpp"
sup y'E[K]
k=1
K
inf 720
70P
Pj max
- -P1 - 7 :.j.:--jj-111 P3. Pj-1 ..... -P - -
Pj max { 1 -y....
inf y20
70P
PJ-1 7, pj. Pj-1 -7....
1 - PK - r7}
inf (h(7sp) 720
where h(75p) p- - 708h + - Pj max 1 - -,,.,, Pj-1 11,1-111--1 Pj. Pj- 7...,
PK - }. Consequently, the minimax problem (5) can be equivalently expressed as:
inf inf (h(r:))
PEAR-1 220
which is special case of the optimization problem in Theorem 3.2 of Guo et al. (0224) where the constant term -1 has no effect on the optimal solution. Thus, Theorem 4.2 follows directly from Theorem 3.2 Guo et al. (2024).
B EXPERIMENTAL DETAILS
Source Models. For the source models, we use those provided by [iang et al. (2020) and Yang et al.) (2021a) for the Office-Home and VisDA2017 datasets. Since open-source models are available for Office-31 31 and DomainNet- 126, we train the source models ourselves using the training methodologics from SHOT (Liang et al. 2020) and C-SFDA (Karim et al., 2023). respectively.
Target Adaptation Training, We train both the model backbone and classifier during the adaptation process, primarily following the SHOT (Liang et al.) (020)) and AaD (Yang et al. (022) setups. For optimization, we use SGD with momentum of 0.9 and a weight decay of 1e We also use the Nesterov update method. The initial learning rate for the bottleneck and classification layers is set to 0.001 across all datasets. For the backbone models, the initial learning rates are set as follows: 5e 5e-4 for Office-Homee - for DomainNet- 126 and Office-31, and 5e for VisDA2017. We use the same learning rate scheduler as Liang et al. (2020) for the Office- and DomainNet- 126 datasets. The batch size is 64 for all datasets. We train for 30 epochs on VisDA2017 and 45 epochs on Office-Home, Office-31, and DomainNet-126 All experiments are run single 32GB V100 or 40GB A100 GPU.
Hyperparameters Selection. In SFDA, hyperparameter selection presents significant challenge due to the lack labeled target data and the distribution shift between domains. In our experiments,
follow the common pipeline for hyperparameter tuning in the literature (c.g.. Yang et al. (2022): Hwang et al. (2024)), and employ the SND (Soft Neighborhood Density) score (Saito et al. 2021 and sensitivity analysis to guide the hyperparameter selection. Notably, most hyperparameters in method do not require intensive tuning, and their choices can be guided by our theoretical analysis outlined below.
Our (CCon-SFDA method consists of three main components: the basic contrastive loss Z the dispersion control term Lic, and the partial label term Lt.- Given the complexity of the parameter
28

---

Published as a conference paper at ICLR 2025
space, simplify the hyperparameter selection process by avoiding exhaustive consideration of all parameter combinations. Instead, we adopt a sequential, incremental approach to tune the parameters for the three loss terms, one at time.
First, for the hyperparameters in the Ler terms (first three columns in Table B BT) including the number of positive samples K, the decay exponent / for the negative term, and the negative sample loss coefficient A we largely follow the configurations used in Yang et al. (022) and Hwang al. (024). As in previous works, we directly set X to 1. For datasets with more classification categories, such as Office- less pronounced, we use a smaller decay exponent to enhance the impact of true- neg-ntettive samples during adaptation. In contrast, for VisDA, which contains only 12 classes with batch size of 64, we apply a faster decay rate to mitigate the influence of false-negative samples.
Office, and DomainNet-122 where noise in negative samples is
Next, we consider the hyperparameter associated with the dispersion term, Apc- In our initial experimental trials, we set this value to either 0.5 or 1, based on balance between the loss terms, Ce. and Lp and the sensitivity analysis of hyperparameters.
Finally, for the hyperparameters ApI. Krr., and 7 the partial label loss, we also perform the basic sequential tuning under the guidance of theoretical insights. According to the proposed algorithm, we use 7 to select uncertain data points and merge the top- pp predicted classes into the partial label set for each selected data point. Theoretically. smaller (yet naturally larger than 1) represents more uncertain set. As we want to apply the partial label loss only to uncertain data points and avoid the introduction of additional label uncertainty for more confident data points, we consider value in {1.1, 1.3, }.5} } for т. We find that 7 = 1.1 is sufficient for achieving promising performance, except for simpler tasks with high initial prediction accuracy, such as Office-31. Next, the value of the partial label number Kps. should be determined based on the algorithm and the number of categories the dataset. Generally, a small Kpt. is preferred, each epoch. A large KPL could result in an overly large partial label set, potentially introducing more uncertainty. Empirically, we evaluate Kr,, E {1, 2, 3}, and find that Kr,. datasets, except for VisDA2017, whose total number of classes is only 12 and Kys = 1 is sufficient. Finally, we tune Xpr. by considering ApL E f0.001, 0.01, 0.05, 0.1) and select the best- rest-performing value based on the guidance of the hyperparameter sensitivity analyses.
the partial label is gradually enlarged with
performs well for most
The final selected parameter values used our experiments are summarized Table B1 which are obtained by a relatively straightforward tuning process conducted subspace of hyperparameters. We note that more refined tuning over the full combinatorial hyperparameter space can further enhance the performance of our algorithm; additional analysis the sensitivity of these hyperparameters is provided in Appendix C.5
Table B1: Hypermaraters on different datasets
Dataset
k
N
β
Apc
ApL
KPL
T
Office-31
3
1
1
0.05
1.3
Office-Home
3
1
0.5
0.001
1.1
Office-Home (partial set)
5
1
0.75
1
0.1
2
1.1
VisDA2017
5
1
0.01
1.1
VISDA-RUST
3
1
5
0.5
0.1
2
1.1
DomainNet-122
2
0.75
0.5
0.1
1.1
Algorithm. The overall description of adaptation process with our UCon-SFDA method is shown in Algorithm
29

---

Published as a conference paper at ICLR 2025
Algorithm 1: UCon-SFDA Uncertainty-Contrlllee Source-Free Domain Adaptation
Input: Pre-Trained Source Model: fa( 0s) Target Data: D _ {x])=
Training Epochs: T
// Initialio aetttallzatiii i on Process
Initialize target model (x; fs
Construct feature bank I and predicted score bank 3R as described in Yang et al. (2022)
Initialize the partial label bank vv. and uncertainty sample bank W as proposed in Section 4.4
// Training/ Adaptation Process
for epock=/ to T do for iterations! 1.2.3.... do
Forward Propagation: obtain feature 2,, predicted probabilities (x; 0.) and frr (AUG 01) for each sample x; in mini-batch B
Bank Refresh: update and using zg and (xg: 02) as described in Yang al. 2022;; update Pz. and W proposed in Section 4.4
10
Compute Negative Uncertainty Control Loss Luc.a in Eq. (7) using (x f.) and
12
fr (AUGG(())))) (x)); 0,)
13
Compute Positive Uncertainty Control Loss Cieen in Eq. using I, 3F ye and U Compute the total Uncertainty Control Source-Free Domann Adaptation Loss
14
Lucon SFDDA Luco + Liccon
15
Update the parameters (r() via Lucon-sFDA
16
end for
end for
Output: Target Adapted Model (x;; 0,)
C ADDITIONAL EXPERIMENTAL RESULTS
C.I EXPERIMENTAL RESULT ON OFFICE-HOME
The experimental results on the Office-Hone dataset are reported in Table
Table C2: Classification accuracy (%) on the Office- Homee--Hme dataset (ResNet-50)
Method
Ar-CCI
PPr Ar RR CI Al
.rr CI -Rw
Pr-AAr
Pt-/CI
Pt P--Rw
Rw-tAr Rw Rw-xCI
Rw
Avg.
SHOT Liung
2020
57.1
78.1
81.5
68.0
78.2
78.1
67.4
54.9
82.2
73.3
58.8
84.3
71.8
No Ai 2021
58.4
79.0
82.4
67.5
79.3
78.9
56.2
82.9
74.1
60.5
85.0
72.8
G-SFD4 Yame
7021b
$7.9
78.6
81.0
66.7
71.2
77.2
65.6
56.0
82.2
72.0
57.8
83.4
71.3
20211
CPGA CJu gi 20M4
59.3
78.
79.8
63.4
75.5
76.4
65.7
58.0
81.0
72.0
64.4
83.3
71.6
CoWA-JNDS
2022
56.9
78.4
81.0
69.1
80.0
79.9
67.7
57.2
82.4
72.8
60.5
84.5
72.5
DaC Zhang otall 20222
59.1
79.5
81.2
69.3
78.9
79.2
67.4
56.4
82.4
74.0
61.4
84.4
72.8
C-SF (Knrin 2:1
222
60.3
80.2
82.9
69.3
80.1
78.8
67.3
58.1
83.4
73.6
61.3
86.3
73.5
Yarg
79.5
57.4
I-SFDX (Stienzuni eral. alll__2000 2024
60.7
78.9
82.0
69.9
79.5
79.7
61.1
58.8
82.3
74.2
61.3
86.4
73.4
UCon-SFDA (Ours)
61.5
80.5
82.1
69.3
$0.8
78.7
67.0
62.2
82.0
85.5
73.6
C.2 PARTIAL LABEL SET EVALUATION
We conduct the self self-prediction, partial label set, and neighbor label set evaluations across all 12 tasks on the office-hone home dataset. The results of self self prediciionn are shown in Figure CT Figure C4 and the results of partial label set and neighbor set comparison are shown in Figure C5 to Figure
30

---

Published as a conference paper at ICLR 2025
AccumI
Overi Ses unicy
Oxenal Sell Prodici ouryy
Orock Sel cittideennn Accericy
10
Takhing Process
Training
Takning Process
(a) Ar -
(b) Ar - Pr
(c) Ar Rw
Figure Cl: Self- celf-prediction accuracy among different data certainty levels on Office- Home dataset with source domain Ar
Uncarain Sall-ratcct
Uncartal slan Canain Data Sat hoake Acoussy
Uncaraln Data Sal-hreician
Catain bata Seltiuaaioon Accurety
Cutta Data Sell Fudiiie Accurayy Process
Traiiing Process
Training Process
CI
(b) CI Pr
(c)
Figure C2: Self-predicties accuracy among different data certainty levels on Office-Home dataset with source domain Cl
Ovnei Sea Protidion Accuacy Mecenskn Data scorny
Cueril Arodecie Acuuery Uecistaki Data wereere
Deaii Prodicien Arcency
Revenall bata Setireeiciin Scavity
Takking Procuss
Training Process
Takking Process
(a) Pr Ar
(b) Pr Cl
(c)
Pr Rw
Figure C3: Self prediceii accuracy among different data certainty levels on Office- Home dataset with source domain Pr
31

---

Published as a conference paper at ICLR 2025
SACEEII 150
Oveel Sei Prosiction unicy
Oxenl Sell Prodoo Borry
Orank Sellirot Protidienn Accertcy
15
Taknng Process
Training
Takning Process
(a) Rw + Ar
(b) Rw - CI
(c) Rw + Pr
Figure C4: Self- reff-prediction accuracy among different data certainty levels on Office- Home dataset with source domain Rw
Wemoore of Lobbet set with Connect Lo0ee
rorerrtiio of Labrl
Unerian Buta becetter Laad suanycy
Jecation Kityber Accetts Censa Date Pariil Acouneyy
Uecetain Dita he Ladd fet Gcoutey Oxrise Owno notse sccurrry
Conaa Dwn Ruttl Lold Acouccy
Training Process
Traiiin Process
Training Process
(a) -r
(b) Ar Pr
(c)
Figure C5: Label set correctness among different data certainty levels on Office-Home dataset with source domain Ar
Retce tagee (abe) Sent with Connect LAbr
Unceriin Bits Labe Set
Jecotonn Parill Libel fer Accurecy decartal kelp
Lecetail Dits Nema Accuniiy
Data he bertter Losd searnoy
Dats gabber Heaumoy
Conoe beigheer Noc
Cenaln bote hoiptter Labee ber Accurcy
Cense Own helghter Labe Nrc
Training Process
Trariin Pocoss
Training Process
(a) CI Ar
(b) CI + Pr
(c) CI Rw
Figure C6: Label set correctness among different data certainty levels on Office Home dataset with source domain Cl
32

---

Published as a conference paper at ICLR 2025
Fereemagg of Label Senc CoFFect Label
Unceria liin Buta fert Labe Set Accrac
Uecortan Data Partaa Label Ser Accurncy
Decetael Dite feria Lake Accunice
Laist Acarnoy
Cerran unod
Cortan Owa biig Labe Sat Accuraty
Cansin Data he hugtbar Labal Ser Acourccy
Cartan Dan heigh Labe Accusty
Traninng Process
Tairing hocoss
Training Process
(a) Pr + Ar
(b) Pr CI
(c) Pr + Rw
Figure C7: Label set correctness among different data certainty levels on Office- Home dataset with source domain Pr
perceragge of Laaee Seee with Coneect Lave)
Lance Cale with Connact
Unariaih Date Labe
Uncoten Tartal Label Accurss
Unostain Dota Nerre Accuncy
Uncericn Data peterer Laad Det Aetti lcarrry
vecorton Deta lelp foo usbel Acunnty
Uncetam Dite le peter Land Set Sel Reaursy
conte wwightr
Thininng Process
Training Pracess
Training Process
(a) Rw + Ar
(b) Rw CI
(c) Rw Pr
Figure C8: Label set correctness among different data certainty levels on Office-Home dataset with source domain Rw
C.3 DATA AUGMENTATION IN SFDA
Org Target Data Act Aug Target Data Acc Self-Pred Consistency
0.6
0:57
0.5
0.50
0.53
0,43
A7-43
Ar
C1-AM
C-4P
0 -RN
71-48
Tr-43
Pr.
-1
Ba
5yth Real
Offic-tteme
ViNDA2017
Figure C9: Inconsistency between the prediction results between the anchor image and its augmented view by source model
The data augmentation technique has been used in SFDA to improve prediction consistency, enhance the target model's generalizability, and control feature space variance (Karim et al.) 2023 Mitsuzumi ctt al., 2024; Xu et al.) 2025). However, these methods intuitively treat the augmented views as positive samples of the original image, without considering the model's 's initial representational or predictive capacity on these data. Moreover, they often overlook the fact that such data are more likely to be negative samples in terms of the self- self-prrdicted pseudo-label (Pu et al. 2021).
Here, we evaluate the prediction accuracy and consistency of the original target data and their augmented version by applying the source model to Office-Home and VisDA-2017. The consistency
33

---

Published as a conference paper at ICLR 2025
is defined as:
CONSISTENCY
(fa(x;; 0s) ( (A( 0s)]
As shown in Figure C9 the source model exhibits low accuracy in predicting the augmented data and demonstrates a high inconsistency between the predictions for the anchor data and its augmented versions. This experimental result is counterintuitive. It empirically explains why directly using the augmented predictions as additional labels or supervisory signals sometimes fails to effectively improve SFDA performance and may have negative impact.
C.4 VARIANCE CONTROL. EFFECT
We evaluate the dispersion control effect achieved by our augmentation-bain Lic across all 12 tasks on the office-home dataset. The results are shown in Figure CTO Figure CI3 dispersion reduction achieved validates the effectiveness of our proposed method.
The consistent
Vartencee trefiction Siritalty
0.21
With Brek Negeetae
Wighlees
0.57
2000
feniiee Voliine CVEIN
Nervve Veraaee Cwnie
Aemone sndienn In (oetrn) Oigeniion Contol
specsin
wirisc Contal
620
Taining Process
Taining Process
Taining Process
(a) Ar
(b) Ar Pr
(c) Rw
Figure C10: Dispersion control loss effect on Office- Office-Home dataset with source domain Ar
2031
2.23
Verdance Prodiction Siidarty
sentencor of Simdarky
with Thee tegeliee
Negsttege
4025
010-
varlanen witteut Di Drspeiion Centol
whete spenion Coviot
vitrect Dopesice Cantol
dnsrpe Cented
Average Vertere
Secrpe Vartunne
Triniin
Process
Training Process
(a) CI
(b) CI Pr
(c) + Rw
Figure C11: Dispersion control loss effect Office-Home dataset with source domain Cl
C.5 SENSITIVITY ANALYSES OF HYPERPARAMETERS
To further understand the performance of the proposed method, we conduct comprehensive experi- ments to study the sensitivity of our method to different choices of hyperparameters involved in our algorithm. While primarily use the hyperparameter configurations from previous works (Yang ct al. 2022: Hwang et al. 2024) for Ao K and 3, we also investigate the sensitivity our method relative to different choices of 3, KPL, T, >PI. and Xpe- The experimental results summarized in Figure C14 a), (b), (c), Figure C15 and Figure C16 respectively.
Specifically, in Figure C14 a)-(c), the solid lines represent the accuracy of different methods with respect to different values of 3, Kot, and r. Figure () we add the dashed horizontal lines to indicate the performance different datasets without the partial label loss for clear comparison.
34

---

Published as a conference paper at ICLR 2025
0.030
We Wwwnnce of Prodectton Siibrity
Tuee Neger antly
2:00
w/ Thua Tage
6025
Depers Oetrn
with Dhipe skn Corred wirout Olpariie Cental
with Dhous Contrel
Dipentio Certol
..s learge Vallanee IN Canveed
Aeerage vartaci Corteattortt
dearage varlanes thou Carvrod
Faining
Taining Process
Faining
(a) Pr + Ar
(b) Pr CI
(c) Pr + Rw
Figure C12: Dispersion control loss effect on Office- Home dataset with source domain
Viraa nop of Sroletinn Sintuity
0.274
Verance www.setttttttoooooooooooo Serteattt///aaarr Neggttuee
Eretuttin Tram Megal
2:00
e4th
641
with Dipprrs eT Outrrs legrage karlanste Semiocr volinee EI CarReE
with Zupe den Canrot
wif Dippra Centrol
Autrage Vertaros
legripe in femnor indiion Cantee Faining Process
Nerigg Velsiee www.eateeettttoooooooo
Trining Process
Traiiing Process
(a) Rw + At
(b) Rw -
(c) Rw + Pr
Figure C13: Dispersion control loss effect on Office- dataset with source domain Rw
In Figures C15 C16 the blue, red, and yellow lines represent the accuracy on the target dataset, the
accuracy on the small evaluation set, and the SND score, respectively. The shaded regions correspond to the results reported in the main text and the associated parameter values. For Figures C14. C16 except for the parameter values that vary along the x-axis, all other parameters are set according to Table B
333
746
560
733
733
62
6.2
65
6.75
(a) Decay Exponent
(b) Partial Label Num KPL
(c) Uncertainty Threshold
Figure C14: Sensitivity analysis of the proposed method relative to different values of hyperparame- ters B, Krt, and t. the legend, "wo" the abbreviation for "without".
Decay Exponent 3. Figure C14(a) a) reveals that the dispersion control term can help mitigate the sensitivity of 3 in contrastive learning based SFDA algorithms. Specifically, we compare the performance an SFDA task (R to P on DomainNet-1222 dataset) using our proposed method (UCon- SFDA) against the basic contrastive learning approach introduced in Yang et al. (2022). Beyond providing stable performance improvements, our method demonstrates reduced sensitivity to the hyperparameter 3, benefiting from the uncertainty-contrllingg regularizations.
35

---

Published as a conference paper at ICLR 2025
15 R S C C
75
Tagat DITA
6.001 6.00
620
635
(a) ViSDA-RUST
(b) DomainNet- 126
(c) Office-Home (Pr CI)
Figure C15: Sensitivity analysis of dispersion control loss coefficient XPL. Different colors represent various criteria for hyperparameter selection, while the shaded area indicates the parameter values chosen corresponding to the results reported in the main text.
75
Target
Smal Em
24 or Torget Dars
Smal Sea
540 TeTThrgeDDe
(a) VisDA RUST
(b) DomainNet (R P)
(c) OOffic-Hoom (Pr CI)
Figure C16: Sensitivity analysis of dispersion control loss coefficient Noe. Different colors represent various criteria for hyperparameter selection, while the shaded area indicates the parameter values chosen corresponding to the results reported in the main text.
Partial Label Number Kr, and Uncertainty Threshold т. Figure C14 b) and (c) illustrate the sensitivity our method to the partial label number Kr, and uncertainty threshold т. By comparing the performance variations on ViSDA-RUST, Office-31, and Office-Home (Pr to CI task) under different Kp,. and т, we observe that the accuracy our method is not significantly affected by varying values of Kp, and Moreover, the performance improvements by the partial label loss are both evident and stable, as shown by the comparison between the solid and dashed lines.
Partial Labeling Term Coefficient λcy and Dispersion Control Term Coefficient Ape- As shown we conduct an ablation study with finer-g emer-grained variations of XeL and Apc on three datasets access sensitivity of the experimental results. Relative to the blue lines, the adaptation performance remains stable and robust across different values of these two hyperparameters, with the regions of optimal performance being well-concentrated.
in Figures C15
C16
Additional Insights for Advanced and Practical Hyperparameter Selection Strategies. Hyper- parameter tuning in SFDA poses significant challenges due to the lack of target labels and substantial distribution shifts across domains. In our experiments, we find that SND scores often fail correlate consistently with performance on the full target dataset. Moreover, sensitivity analysis based on the full target data incurs high computational costs, making it less feasible for real-world applications. To overcome these limitations, we explore novel small evaluation set-based method. Specificallyy we randomly select subset (300 data points) from the full unlabeled target data (typically containing 5k-50k data points), manually label it, and create subsequently selected based on their performance on this small evaluation set. While this approach requires some manual annotation, the amount of labeled data needed is minimal, making it both practical and effective for real-world scenarios, while improving the accuracy of hyperparameter selection.
-aludd-vaiiitiinn set. Hyperparameters
Figures C15 and C16 demonstrate that the performance of our method on the small numan-Labelen evaluation set (red lines) aligns more closely with the desired model performance (blue lines). In
36

---

Published as a conference paper at ICLR 2025
contrast, the SND score (yellow lines), which is based feature space similarity and self self-prediction entropy, sometimes fails to identify the optimal hyperparameters.
Better Performance with Finer-Grained Hyperparameter Ranges. Refining the parameter selection range (as shown Figure (I)) or adopting a different tuning order (e.g.. tuning the partial label term first, followed by the dispersion control term, as shown in Figure C16 ab)) can achieve even better results, as indicated by the highest points on the blue lines. For instance, while we initially report the UCon-SFDA performance of 79.4 on VisDA-RUST (with LpI. 0.1 and DDC 0.5), we find that using slightly smaller LDC 0.1 improves its performance to 79.82. These findings demonstrate that satisfactory performance of our approach does not depend on excessive hyperparameter tuning, and further highlights the robustness and effectiveness of our algorithm.
C.6 DIFFERENT LOSSES FOR DISPERSION CONTROL TERM
We evaluate the performance of the dispersion control term under different similarity metrics between an anchor data point and its augmented version, de (AUG (x:) x:), in Eq. 7
Specifically, for Eq. (7 the main text, we define:
de (AUG (x:) xx) (f(x;; 0), log / (AUG (x;) :)).
To further validate the role of data augmentation from the perspective of negative sampling uncertainty, we experiment with different similarity metrics, including the direct dot product and the T.2 norm, respectively given by
de (AUG (xx) xx) A (f(x;; O). (AUG (x:) ;)))
and
d0 (AUG (xx) x4) A [x 0) - (AUG (x)
Additional experimental results, reported in Table C3 demonstrate the importance of treating data augmentations as negative samples well as the effectiveness of the proposed dispersion control term. Furthermore, while proposed al achieves the best performance across most datasets, other loss formulations also present comparable results. These experimental observations provide guidance on effectively leveraging data augmentations SFDA and verify the generalizability of our algorithm.
Table C3: Classification accuracy (%) under different distance measurements in dispersion control term. Bold text indicates the best results, and underlined text represents results that outperform the baseline.
Methods
Office- Home (Pr + Cl) VisDA-RUST DomainNet126 (R - P)
Ceu
57.90
75.50
67.80
Cet + Lie with de
59.70
78.90
70.30
Le, + Lpe with dlg,dor
60.21
78.02
70.08
Le. + L with d
59.14
77.77
69.34
C.7 TRAINING TIME AND RESOURCE USAGE ANALYSIS
To further validate the practical value our proposed methodology, we conduct the training time and resource usage analysis here.
Compared to the baseline model, AaD YYngg et al. 2022). a widely utilized contrastive learning and memory
SFDA method, our UCon-SFDA introduces explicit data augmentation and an additional partial label bank component. These additions increase both resource usage and computational complexity. However, such costs consistent with recent trends in the field (Hwang
37

---

Published as a conference paper at ICLR 2025
al. 2024 Karim et al. 2023 Mitsuzumi al. 2024). where enhanced resource utilization is commonly accepted to achieve significant performance improvements.
The computational complexity of our approach remains comparable to other modern techniques that leverage data augmentation or consistency regularization. For instance, compared to Karim et al. (2023) and Mitsuzumi al. 2024), which also incorporate explicit data augmentation during training, our DCon-SFDA avoids relying on additional network structures. Moreover, the partial label bank only incurs a small additional memory overhead that scales linearly with the size of the target domain data, making it practical for real-world SFDA applications. Importantly, our method demonstrates superior performance, as evidenced by the experimental results presented in main text.
Neverthless, we acknowledge that the explicit data augmentation employed in UCon-SFDD inevitably increases the GPU memory usage, which could present challenges in ressurce-constrained settings. Although our approach ensures that the additional overhead remains manageable, further algorithmic and implementation-lewe
optimizations could help mitigate this issue. For instance, future work could explore more memory-efficee augmentation techniques, optimize the computational graph during training, or incorporate mixed-precision training. These efforts hold promise for enhancing scalability while maintaining performance.
D THEORY-MOTIVATED HYPERPARAMETER DETERMINATION AND AUTOUCON-SEDA
In SFDA problems where neither target domain labels validation are available, minimizing the numbers of hyperparameters is crucial to ensuring the algorithm's practicality for new tasks. When designing the "Con-SFDA algorithm (as presented in the main paper), we prioritize engineering flexibility and ease of implementation, which lead us to introduce four hyperparameters: ADO, XPL. KrL and т. However, three of these hyperparameters have explicit expressions derived from our theoretical results or can be determined based on dataset and source model properties.
Here, we provide a detailed explanation of how theoretical insights can guide the direct selection or derivation of hyperparameters, thereby eliminating the need for manual tuning. Building on these theoretical principles, we propose two enhanced variants (autoUCon-SFDD) results demonstrate that directly using theoretically derived parameters not only simplifies the tuning process but also achieves promising-ame in some cases,
Additional experimental
across all benchmarks.
D.1 THEORETICAL GUIDANCE FOR HYPERPARAMETTE DETERMINATION
Based on our theoretical findings, the hyperparameters Ape in the dispersion control term and KPL, in the partial label term can be directly determined. Specifically,
Inconsistency Rate Ape- As suggested by Theorem 4.1 and Remark 4.2 the dispersion control effect can be achieved by minimizing the negative similarity between the anchor point and its augmented prediction. If the inconsistency rate between anchor points and their associated augmented predictions is high, it indicates greater uncertainty in negative sampling, thus requiring stronger dispersion control. Based on this observation, we propose directly using the model prediction inconsistency rate the coefficient for the dispersion control term.
Parameter KPL (ko in Theorem 4.2 By Theorem 4.2 when the uncertainty in Eq. (5 is defined using the 1- I-Wasserstein distance, the length of the partial label set, denoted by KpL. can be explicitly determined as Krt. = ko. where ko defined as follows:
If マ
4 P()
fō for all & [K - 1], then we take Kog = K.
Otherwise, we take the by E [K-1] 1] that satisfies Eee ply) 25 2 Σ for all k [K- 1].
E P( - 46
In the formulas above, K represents the number of classes, and pb) denotes the j-th largest predicted probability for the considered anchor point. Hence, the length of the partial label set, which can
38

---

Published as a conference paper at ICLR 2025
be directly calculated. is determined by the model's predictions for the anchor point well the specific classification task at hand.
Uncertainty Threshold 7. We propose two approaches distinguish between certain and uncertain label information and determine the uncertainty threshold T.
Statistical Insights Approach: This approach leverages the properties the source model and the target data, combined with statistical insights. Specifically, first use the source model to compute the predicted probabilities for each target data point. Next, we calculate the ratio of the highest predicted probabilities for all target data points and select the 10th percentile of these ratios as the value of т. This selection capitalizes on the information about the data distribution and identifies the 10% most uncertain data. The 10th percentile is chosen because it is a widely used measure in statistical research to highlight low-end values. This uncertainty threshold determination method leads to the development of the autoUCon-SFDAA (Stat.) algorithm.
Theoretical Criterion Approach: Alternatively, we can bypass the ratio of the two highest predicted probabilities and directly apply the criterion outlined Remark 4.3 to distinguish
between certain and uncertain label information. As discussed in Remark 4.3 in the special case where P(1y 2 max K + 6. 6.P(2) + 83, we refer to as certain label information. Conversely, this condition is not satisfied, the label information is deemed uncertain, and the corresponding data point is added to the uncertain data bank. This uncertainty threshold determination method leads to the development of the autoUCon-SFDDA (Theory) algorithm.
Building upon the preceding illustrations and different approaches to determining the uncertainty threshold we propose two automated versions of WCon-SFDA: AutoUCon-SFDD. (Statistics) and autoUCon-SFDA (Theory).
D.2 EXPERIMENTAL RESULTS OF AUTOUCON-SFDA
Compared with the original UCon-SFDA, autoUCon-SFDA (Statistics) and autoUCon-SFDA (The- ory) incorporate the following modifications in the implementation:
1. The original manually tuned hyperparameter Xpe (Orig. Apc) has been replaced by new Ape. which represents the inconsistency ratio between anchor points and their associated augmented predictions, derived by the source model.
2. The original fixed Kr.. (Orig. Krr.) has been replaced by the calculated ko- which is instance- and task-dependent (class category), self-adaptive during the training process, and computationally efficient.
3. We propose alternatives for the fixed parameter (Orig. 7):
In the statistical insights approach, autoUCCn-SSFIp (Stat.), т, is computed using the source model and fixed at the beginning of the adaptation process.
In the theoretical criterion approach, autoUCon-SFDD. (Theory), is dynamically calculated based on the uncertain data selected in each epoch.
Table D4:
Performance comparisons across different hyper- parameter selection (calculation) methods. Bold text indicates the best results.
autoUCon-SSDAA autoUCCon-SFDA (Stat.)
SOTA Method Performance
Dataset
UCon-SFDA
SOTA Method
(Theory)
Office31
90.6
90.6
90.2
90.5
C-SFDA
Hofe
73.6
73.6
73.8
73.5
C-SDDA
OfficeHome (partial set)
80.3
80.8
80.7
79.7
AaD
VisDA2017
89.6
89.3
89.2
88.4
I-SFDA
VISDA-RUST
79.4
79.2
79.5
77.3
SF(DA)
DomainNet126
71.5
71.5
71.6
69.6
GPUE
39

---

Published as a conference paper at ICLR 2025
Table DS: Hyperparameter values across different datasets. "Orig. Apc". "Orig. Kt. and "Orig. T" refer to the original values used in our paper, which are selected following the general hyyper-parametrr tuning pipeline the literature. Other hyperparameters are directly calculated with moooyymotivtted hyperparameter determination approaches, where "Init." and "Final" indicate the first and the last training epochs, respectively.
OfficeHome (partial set)
Metric
Office31
OfficeHome
VisDA2017
VisDA-RUST
DomainNet122
Orig. Ape
1.000
0.500
1.000
1.000
0.500
0.500
New Ape
0.390
0.520
0.476
0.494
0.461
0.553
Orig. Kyn
2.000
2,000
2.000
1.000
2,000
2,000
Tnit. Ky (Averaged)
1.320
1,535
1,513
1.341
1.348
1.644
Final Ko (Averaged)
1.028
1.079
Orig.
1,300
1.100
1.100
1.100
1.100
1.100
Init. Ti
1.308
1.265
1.238
1,790
1.674
1.232
Final 7
1.056
1,090
1.042
1,260
1,368
1.092
(10th percentile)
2.037
1.230
1.268
1.164
1.163
1.264
Table D6: Per source-target task configuration DomainNet126.. The metric notations the same as in Table DS
Ace. of Ucon-SFDA
Acc. of MutoUCon-SFDA. (Theory)
Acc. of autoUCon-SFDAA (Stat.)
Orig. Apo
New
Orig. Kyu
Init. (Averaged)
Final Kn (Averaged)
Aoo
Orig.
Init. n.
Final iz
T
C-8
66.5
64.5
66.0
0.50
0.52
1.70
1.08
1.1
1.20
1.08
1.23
P-C
69.3
70.3
70.0
0,50
0.59
2.33
1.11
1.1
1,30
1.11
1.17
P-R
81.0
81.4
81.4
0.50
0.45
1.64
1.04
1.1
1.28
1.08
1.36
R-+C
75.2
64.3
S-P
68.1
67.9
67.6
0.50
1.08
1.1
Avg.
71.5
71.5
71.6
0.50
0.55
1.64
1.08
1.1
1.23
1.09
1.26
first conduct comprehensive performance comparison of the original UCon-SFDA, auto- mated variants, and state-of-the-arr (SOTA) methods across all six benchmarks, shown in Table D4] Notably, our findings validate that directly using theoretically derived parameters can achieve promis- ing ang-and in some cases, superior auperior-perffrmancc performance across all benchmarks. (For the remaining three hyperparameters k, B and APL. we keep them the same as those used in UCon-SFDA.)
detailed parameter comparison is further provided in Table D5 For ko and Tr, report their values at the first and the last training epochs to illustrate their changing trend, denoted as "Init." and "Final" in the tables, respectively. It can be observed that the theoretically determined parameters largely aligned with the hyperparameters used in UCon-SFDA. However, they offer greater flexibility certain scenarios. For instance, based on the averaged values of ko the initial and final training epochs, the instance-dependant kn automatically adapts throughout the adaptation, unlike the fixed KrL, thereby better capturing uncertainty. similar self-adaptive behavior observed for 7t-
Additionally, we present the per-source-targe task configuration on DomainNet1 (Table D6 to clearly illustrate parameter variations and their impact. For instance, as shown in the sixth coloumn of Table D6 the new ADG is task-dependent, offering greater flexibility without requiring manual selection.
In summary, the automatic versions of UCon-SFIA demonstrate promising performance while signifi- cantly reducing the number of hyperparameters in the algorithm, retaining only three hyperparameters in autoUCon-SFDA of which only one is directly related to proposed methods. Furthermore, additional experimental results also highlight the effectiveness determination process.
seeeertintt-guudde parameter
40


Reviews:
[{"review_id": "W5Fz3cMnXq", "reviewer": "Reviewer_Bkmw", "summary": "This paper presents a novel approach for source-free domain adaptation (SFDA), grounded in an uncertainty-guided theoretical analysis of contrastive learning-based SFDA methods. We introduce a distributionally robust optimization framework to elucidate the role of uncertainty. Additionally, the method incorporates augmentation-driven dispersion control and an optimal solution for partial label sets within a contrastive learning-based SFDA approach. The proposed method is thoroughly evaluated across four benchmark datasets.", "questions": "See weaknesses", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This paper induces a new aspect to solve the source-free domain adaptation problem.\n- It induces an efficient method based on the guide of the proposed theoretical analysis.\n- The experimental results prove the advantage of the proposed method", "weaknesses": "- Intuitively, applying cross-entropy loss on uncertain target samples with estimated pseudo labels can be problematic and may lead to negative transfer due to noisy pseudo label information, as certain samples are more likely to have accurate pseudo labels than uncertain ones..\n- There are seven hyper-parameters, as shown in Table 6, which suggests that the proposed method requires specific parameter tuning to achieve promising results.\n- The data augmentation alignment in Eq. 7 is, in fact, a consistency regularization between the weak and strong augmentations of the same data, which is not a novel technique as it has been well studied.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nLJM3dwuif", "reviewer": "Reviewer_GdvL", "summary": "This paper studies source-fee domain adaptation, with a specific focus on the uncertainty in the contrastive learning-based source-free domain adaptation (SFDA) solution. The authors comprehensively analyze two types of uncertainty including both negative and positive uncertainty through the lens of Distributionally Robust Optimization. Based on the theoretical framework the authors propose an uncertainty-control SFDA method (UCon-SFDA). Extensive experiments demonstrate the advantages of UCon-SFDA over existing SFDA approaches.", "questions": "Please see the question in Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(i) Significance. This source-free domain adaptation problem studied in the submission is significant for real-world applications of deep learning models. The practical assumption of no source access is meaningful for privacy-sensitive scenarios such as medical data. \n\n(ii) Novelty. This paper provides an in-depth theoretical analysis of the overlooked uncertainty problem in previous SFDA methods within a unified DRO framework.\n\n(iii) Quality. Extensive experiments and comparisons demonstrate that the proposed approach UCon-SFDA outperforms existing SFDA methods.\n\n(iv) Clarity. The presentation is clear with detailed theoretical analysis and illustrative figures.", "weaknesses": "Although the approach is supported by in-depth theoretical analysis and impressive experimental results, I remain concerned about the hyperparameters setting.\n\nSince source-free domain adaptation only has unlabeled target-domain data, it is quite difficult to accurately tune hyperparameters for the SFDA approach. Usually, one solution is to avoid involving many hyperparameters in the SFDA approach. However, the proposed UCon-SFDA necessitates the tuning of many hyperparameters including different $\\lambda$ as shown in Table 6 in the Appendix. Moreover, Table 6 demonstrates that the authors adopt different hyperparameter values for different datasets. The challenge is that in practical applications, we would be given an unseen unlabeled target domain where existing hyperparameters can be worse. \n\nMy specific questions are:\n\n- How to tune various hyperparameters to ensure UCon-SFDA can work well in both this paper and other new SFDA tasks? \n\n- Hyperparameter sensitivity analysis is critical for understanding the method but missing in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FD7S8vZ1Kc", "reviewer": "Reviewer_1NWJ", "summary": "The current SFDA methods that use source models for self supervised adaptation neglects the uncertainties involved in the transfer process. The authors conduct a comprehensive analysis of the two types of uncertainties arising from both positive and negative samples, providing theoretical insights for both. Based on these theoretical insights, the authors propose a dispersion control method to mitigate the uncertainty introduced by noisy negative samples. For the uncertainty arising from positive samples, the authors suggest leveraging partial labels to fully utilize the predictive uncertainty. By managing these two types of uncertainties, the authors significantly improve the model's performance.", "questions": "How are the weights λ-CL, λDC, and λPL for the different loss terms (L-CL, L-DC, L-PL) determined? What principles or guidelines were used to set these hyperparameter values?\nHow are the threshold and the number of retained labels for the partial label method determined?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Overall, this could be an important theoretical and algorithmic contribution. The paper comprehensively analyzes the two types of uncertainties caused by positive and negative samples during the adaptation process, proves their impact on model performance, and provides effective solutions.", "weaknesses": "The method of the paper requires handling both positive and negative sample uncertainties, which increases the computational complexity and training difficulty of the model.The theoretical analysis section may be too complex and difficult for general readers to understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)