RrED: Black-box Unsupervised Domain Adaptation via Rectifying-reasoning Errors of Diffusion
Authors: Yuwu Lu
Chunzhi Liu
Venue: NeurIPS 2025 poster ()
Keywords: Black-box Unsupervised Domain Adaptation; Neuroscience; Diffusion Model; Prompt Learning
Primary area: general_machine_learning
TL;DR: A novel diffusion-based black-box unsupervised domain adaptation method.

Abstract: Black-box Unsupervised Domain Adaptation (BUDA) aims to transfer source domain knowledge to an unlabeled target domain, without accessing the source data or trained source model. Recent diffusion models have significantly advanced the ability to generate images from texts. While they can produce realistic visuals across diverse prompts and demonstrate impressive compositional generalization, these diffusion-based domain adaptation methods focus solely on composition, overlooking their sensitivity to textual nuances. In this work, we propose a novel diffusion-based method, called Rectifying-reasoning Errors of Diffusion (RrED) for BUDA. RrED is a two-stage learning strategy under diffusion supervision to effectively enhance the target model via the decomposed text and visual encoders from the diffusion model. Specifically, RrED consists of two stages: Diffusion-Target model Rectification (DTR) and Self-rectifying Reasoning Model (SRM). In DTR, we decouple the image and text encoders within the diffusion model: the visual encoder integrates our proposed feature-sensitive module to generate inferentially-enhanced visuals, while the text encoder enables multi-modal joint fine-tuning. In SRM, we prioritize the BUDA task itself, leveraging the target model's differential reasoning capability to rectify errors during learning. Extensive experiments confirm that RrED significantly outperforms other methods on four benchmark datasets, demonstrating its effectiveness in enhancing reasoning and generalization abilities.

RrED: Black-box Unsupervised Domain Adaptation via Rectifying-reasoning Errors of Diffusion
Yuwu Lu
Chunzhi Liut
School of Artificial Intelligence South China Normal University
School of Artificial Intelligence South China Normal University
Foshan, Guangdong, China 1uyusm20080163.comm
Foshan, Guangdong. China 2023024323 scnu, edu. cn
Abstract
Black-box Unsupervised Domain Adaptation (BUDA) aims to transfer source domain knowledge to an unlabeled target domain, without accessing the source data or trained source model. Recent diffusion models have significantly advanced the ability to generate images from texts. While they produce realistic visuals across diverse prompts and demonstrate impressive compositional generalization, these iittussoonnbbsse domain adaptation methods focus solely composition, overlooking their sensitivity to textual nuances. In this work, we propose a novel diffusion-based method, called ceetifying-reasoniie Errors of Diffusion (RrED) for BUDA. RrED is two-stage learning strategy under diffusion supervision to effectively enhance the target model via the decomposed text and visual encoders from the diffusion model. Specifically, RrED consists of two stages: Diffussio- Target model Rectification (DTR) and Self-rectifying Reasoning Model (SRM). In DTR, we decouple the image and text encoders within the diffusion model: the visual encoder integrates our proposed feature-sensitive module generate snferentially-enhaneee
visuals, while the text encoder enables multi-modes joint fine-tuning. In SRM, we prioritize the BUDA task itself, leveraging the target model's ss differential reasoning capability to rectify errors during learning. Extensive experiments confirm that RrED significantly outperforms other methods on four benchmark datasets, demonstrating its effectiveness in enhancing reasoning and generalization abilities.
1
Introduction
To address domain shift in training deep neural networks [1. domain generalization (DG) methods B only use source data for model learning to achieve generalization. However, in scenarios with accessible target samples, domain adaptation (DA) methods [4 show a significant performance advantage. Traditional unsupervised domain adaptation (UDA) methods [4-8) focus adapting models trained on a fully labeled source domain to an unlabeled target domain, aiming to alleviate the constraints of data collection and annotation. However, in scenarios like personal medical records, privacy-preeervig policies restrict access to source data, thus limiting the application of UDA techniques. To address this, source-free domain adaptation (SFDA) methods (9H12) have been recently introduced, assuming that only unlabeled target domain data and pre-trained source model are available during the adaptation process. Even though SFDA methods lower the possibility of privacy leaks by utilizing the pre-trained source model rather than source data, [13] found that certain generation techniques like [3 [4 have the potential to reconstruct the source data through
"Corresponding author.
Both authors contributed equally to this work.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

learning from the source model. In comparison to other UDA settings, black-box unsupervised domain adaptation (BUDA) offers enhanced data privacy protection along with greater flexibility in portability. BUDA adapts a model by leveraging the unlabeled target data and a black-box predictor trained on the source domain, egg. an API service in the cloud [5], to avoid privacy and safety problems caused by data and model leakage.
Recent mainstream BUDA methods [1H8 follow self-distillation pro- cess: distilling source knowledge and fine-tuning the model for the target domain. This process relies on high- reliability samples to suppress the neg- ative impact of low-reliability ones. However, AEM [19] observes that distillationn-asee methods selectively ignore those samples that classi- fied as low reliability, resulting un- derlying structural information from low-reliability samples not being uti- lized. To alleviate this problem, AEM introduces the multi-modal model external prompt to
extemal
Whar's 1187
prorrges?
relaforshig?
Kiglishrr
Task-spodie Question
Prompe Assosiaton
Knowledge Roason
Eror Racticcttoo
Memocy
Intuttvv Judgmentts
Memory
Repootorry
Reposlooy
Stage
Stago
Danag
CLIP 20 extract semantic knowledge. How- ever, AEM primarily forces the target model to align with CLIP, overlook- ing the further exploration of the tar- get model. Similar to CLIP, diffusion models 22 also use multi-modal techniques, which represent a new cat- egory of likelihood- based generative models that introduce iterative noise and denoising processes model the data distribution. Compared to other semantic knowledge but also can stably generate diverse images. Previous diffusion- DA methods (24]26] focus on enhancing sample distribution generalization via the image encoder, with limited exploration of the text encoder, and are restricted in scenarios where source samples are inaccessible due to protective policies. In the BUDA setting, such stable generation helps enhance model generalization by providing consistent image-level augmentation. How to effectively leverage diffusion models in BUDA tasks to guide the target model in enhancing its reasoning ability while preventing its potential negative effects? This is the key problem that needs to be addressed this research.
Taeal
Targen
Cample
Figure 1: Conceptual figure of our RiED. Above: our RrED simulates the human decision-making process under the guid- ance of external knowledge. Below: in stage 1, RrED aligns the decision boundaries of the diffusion model under guid- ance; in stage 2, RrED enhances the model's self self-reasoning ability by rectifying the errors among different versions.
multi-modal models, diffusion models not only have rich
In the field of neuroscience as an interaction between two stages: Stage unconsciously generates intuitive responses but tends exhibit cognitive biases and struggles with complex tasks like mathematical reasoning or weighing pros and cons; Stage 2 relies on domain knowledge for deliberate reasoning, handling complex problems more accurately but at a slower pace, Recent works 29 30
the human
process is typically regarded
have observed that discrepancies between multi-stage decision-making can introduce potential errors in reasoning. To address this issue, these works introduce a knowledge base to guide the intuitive learning process during the intuitive response process, leveraging domain knowledge to identify and correct potential errors in the neural network's output during the reasoning phase, thereby producing outputs consistent with the knowledge base.
Inspired by these works, we propose a novel diffusion- based method, named Rectifying -eaaoiniii- Errors of Diffusion (RrED), which is the first work that applies the diffusion model to high-security BUDA tasks innovatively. Specifically, RrED addresses the limitations of previous diffusion-basee methods by continually fine-tuning the text encoder while learning from the diffusion model. RrED composed of two stages: Diffusion- -Tareet model Rectification (DTR) that performs separation learning of the diffuusin image encoder and text encoder and task speciicccific fine-tuning of the text encoder, relf-rectifying Reasoning Model (SRM) that leverages the differential reasoning ability of the
2

---

Table 1: Comparison of different settings. Diffusion- DA relies on both labeled source and unlabeled target data, guided by an external diffusion model. Black-box DA only relies on the unlabeled target data and the predicted labels from a black-box predictor, thus offering better data privacy at the cost of partial performance. Our RrED follows Black-box DA setting for training with a diffusion model incorporated, achieving performance improvement while maintaining high-level
data privacy protection.
Setting
Source data
Source model
Predicted target labels
Target data
External prompt
Privacy risk
DG
X
X
Medium
Traditional DA
V
X
High
Source
X
Medium
Black-box DA
X
V
Low
Diffusion-based DA
V
V
High
Our RiED
x
target model and samples generated by the fine-tuned diffusion model to correct errors in the learning process. RrED introduces the diffusion model as a knowledge base to rectify the memory repository of BUDA. As shown in Figure 1, in DTR, the target model (modeled as the human brain) receives the guidance information from the diffusion model to make intuitive judgments about the task-specific information of BUDA and feeds back the discrepancies between predictions to fine-tune the diffusion model. In SRM, after being guided by the diffusion model, the target model conducts thoughtful comparative reasoning and error correction the task-specific information. The purpose of the two stages is to fine-tune the text encoder of the diffusion model and use the fine-tuned diffusion model to further improve the discriminative ability of the target model. Experimental results demonstrate that RrED significantly outperforms the previous SOTA methods on four benchmarks, confirming its effectiveness in enhancing the model's reasoning and generalization abilities.
Our contributions are summarized as follows:
We observe some weaknesses in existing DA methods and address them by proposing novel method, named RrED, which introduces the diffusion model into the BUDA setup and strengthens the target model's reasoning ability through our two-stage learning.
Inspired by the improved human decision- decision-making process, RrED is designed to consist of two stages, namely DTR and SRM. DTR guides the target model's learning process by rectifying diffusion model reasoning errors and leveraging its knowledge. SRM corrects errors in the learning process by leveraging the differential reasoning ability of the target model and samples generated by the fine-tuned diffusion model.
To evaluate the effectiveness of RrED, we conduct extensive experiments, achieving SOTA perfor- mance four benchmarks. Ablation studies further highlight the contributions of each component and provide a detailed analysis of the relationship among them.
2 Related Works
Domain Adaptation.
The challenge of unsupervised domain adaptation (UDA) resides in transferring the knowledge from the labeled source domain related yet distinct unlabeled target domain. Recently, UDA has been the subject of widespread research in diverse deep learning tasks, including
image classification [4] 31, semantic segmentation 7 32 33 object detection 6 R4 35) and
However, UDA relies on access to both the labeled source domain and the unlabeled target domain during training, which becomes restrictive under srevacy-preservngg policies that limit source data availability. To overcome this, source-free domain adaptation (SFDA) 38 39 [0] enhances privacy protection by requiring only the trained source model and unlabeled target data. Although SFDA methods mitigate privacy data breaches to some extent, recent works 40 highlight the risks of exposing the training white-box model in SFDA, as reverse generation can exploit this vulnerability.
time series forecasting g
36 37
techniques 13 [4]
Blx Unsupervised Domain Adaptation. BUDA has no need to access the source data or trained model, which enhances data privacy protection more effectively than other DA settings, reducing the risk of data breaches. Early work LNI.-KI. [1 proposes a noisy label learning approach using soft labels. Recent work DINE [15) first distills knowledge encourage touree-target
3

---

class alignment and then fine-tunes the distilled model to match the target distribution, using the reliable knowledge from distillation to cluster unreliable samples during fine-tuning, Building on the self-distillation process, recent methods [6 17 partition the target domain into high- and low-
reliability subdomains, and align their distribution discrepancies. BiMem (40] performs information
discrimination between useful and irrelevant information, emphasizing prioritized learning of useful samples while roughly aggregating irrelevant ones. RFC [18) further introduces neighborhood
clustering into [15 [6] to avoid minority class forgetting. Moreover, AEM [1]
first introduces multi-modal model CLIP [20] as an external prompt into BUDA, utilizing CLIP's s rich semantic knowledge conduct feature alignment of the target domain model.
Diffusion Models in Domain Adaptation. Diffusion models 22
42 use a parameterized Markov chain to transform noise from a common distribution to target distribution. Recently, diffusion models have been applied across various tasks like image generation (26), video generation 223. and text-to-image generation 43), due to their support for the interaction and creation of
text and image contents. In domain adaptation, some studies 22 24 26 44
have recognized that
[2 learns additional diffusion can be used to improve the target model's generalization ability. DAD
source-style target samples by continuously synthesizing source and target domain images, gradually transforming the data distribution. SDA 44] maps source and target domain images to a synthesis space, transforming domain transfer into sample aligmment in the synthesis space. However, we observe that current ceffusiio-basete UDA methods rely on both source and target data, limiting their application when policy restrictions prevent access to source data or models. Moreover, these methods are based on image encoder synthesis and do not contribute to the development of text encoder in diffusion models. To solve these problems, RrED integrates diffusion into the BUDA task and guides the diffusion model's generation by fine-tuning the text encoder. As shown in Table the respective processes and the differences among various settings are presented.
The Definition of Reasoning Ability. In human decision-making systems 22 the definition of reasoning ability refers to the capacity of individuals to make further judgments about target objects by leveraging prior knowledge and logical analysis based on partial observations when confronted with complex information and dynamic environments. Similarly, in computer vision, the model's reasoning process mirrors human decision-making by utilizing existing knowledge and feature similarity computations to determine whether targets in complex scenes meet the task requirements. This perspective aligns with the explanations provided by Grad-CAM (45), where models reason about predicted image classes based on convolutional feature maps, analogous to how humans reason about image categories through attention maps. While the reasoning ability of Large Language Models (LLMs) typically refers to abstract and logical inference, in computer vision it focuses on identifying the input regions that most influence the model's decision to infer the most probable target class. The reasoning definition is based on the well-known Grad-CAM technique (45) in computer vision, and our work further integrates the observations of the human decision-makinn system to refine this definition and enhance the model's 's reasoning capabilities.
3 Proposed Method
We first define an unlabeled target domain D, {x;;) where N, represents the number of unlabeled target domain data. In the BUDA setting, D; is uploaded to black-box predictor (i.e., cloud API service), which provides the hard predictions Ps using source model trained on the source domain D.- D, and D, share an identical label distribution over b classes, with common label set L = (1, 2, ... .b). Our goal is to enable the target model Ma to adapt in the target domain, parameterized by 0 and composed of feature extractor fo and a prediction classifier cp. The feature extractor is defined as Sa x; - 2; E Rdd, where d the feature space dimension and z; is the d- d-dimensional transitional output. The prediction classifier is defined as ca 24 + y4 E R&, where y, is the prediction output of the target samples. The complete training process is shown in Figure 2.
3.1 Black-box Learning and Diffusion Process
Task-specific Black-box Learning, Before the training period, the black-box predictor exposes only an open API, allowing external clients to request predictions by uploading data. The source model and source samples remain inaccessible throughout the process, effectively preventing potential data leakage. Additionally, batching requests through queue- based mechanism can improve response efficiency. Previous BUDA methods [15] [6 40] stored the predicted labels of target samples returned
4

---

training
Target model Receificatoon
Reettfylng Reasoning
Caro-Eitimy Loo
Seaoch Leart
Tagert Donain
Medel
Seooth Learing
Thel-eucite
Fase tarning Mulli modal Mode
Aciptation with Sclideatiiiigg
Source Deciain Mlddi
Arppiie Label Smoothin
Inooder
Image
SonpleVeniiie Invevcionn
Updast
Text
Acciiinnizne
Inoade
"Knglrie"
"Kingissee"
Synioiiea
Sysllcieed Dan
Souree Label
"Acdazz pl
Text Eassder
Maa
Featers- Sonaiive Cenentton
Donnlood
box predictor
Brage Easoder
Process
Predkoua Target Den
Smaudlh Label Reposhory
Diffeasue Model with Protistur
Melti-noda Madil
Generaaz Data
Taeget
Queee
Dats
Data Irput
Fester- Sardiiin
Bemove Specili
Irage
Mep
Dxfitoiis
Opes
Generation
Precpia
Repowioory
Security
API
Uplaad
Tagget
Epucch
Ejech
in
Wenkreing Data
Pronge
Tec
with
Figure 2: Overview of the whole training of RrED. According to the BUDA setting, (a) the source model is initially trained with standard procedures and transferred to a black-box predictor; (h) the black-box predictor then exposes restricted API, allowing external clients to query only batches of hard target predictions through iterative requests. In our RrED, (c) DTR guides the target model's s learning by correcting reasoning errors from the diffusion model and leveraging its semantic knowledge; (d) SRM corrects the reasoning error of the target model by leveraging the model's reasoning from predictive differences across versions.
by the open API into smooth label repository and employed adaptive label smoothing (ALS) [15] to filter out some redundant and noisy information. The ALS updating S(xz) can be expressed as:
Σ N₁ AdaLS(P"),
beginning
S(x:)
No (S((,)) + (1--))m
otherwise
where AdaLS(PP)) is function for initializing the smooth label repository in ALS [5]; PA represents the hard prediction of the i-th sample, obtained from the black-box predictor prior to training; and μ
is set to 0.6 following [1 [6 40), representing the static coefficient to stabilize the ALS updating.
During the learning process, the task-specific loss stably transfers knowledge from S(x;)) the target model to accomplish the BUDA task, ensuring efficient learning of target domain knowledge without forgetting source domain knowledge. The task-speeific loss can be expressed as:
Lrask min E, (Ma Me
where Dxz(-) (:) is the Kullback-Leible divergence.
Difffusion model for RrED. In this work, we introduce an image encoder with fixed weights from CLIP 20 and leverage the predictor along with the image encoder during the diffusion-target model rectification fine-tune the text encoder. Moreover, before the SRM period, the fine-tuned text encoder is fed back to the diffusion model to enable more controllable image generation and adapt the target model the BUDA task. The diffusion process and how the diffusion model applied to RrED are described in detail in Appendix A.
3.2 Diffusion-Targee model Rectification
Ceature-Sensitiee Generation (FSG). During the first-stage training, the target model stably learss from the images generated by the diffusion model under the control of FSG. Despite the constraints imposed by the text encoder the diffusion model, the mmage-generatton process remains uncon- trollable. To prevent the negative impacts of this uncontrollable factor, FSG needs to determine which regions should be composed of synthetic images, enhancing generalization without sacrificing discriminative ability. Specifically, FSG first evaluates the feature-sensitive regions of the model by evaluating the global image and comparing it with each local region, leveraging the local regions and their adjacent contextual information. The weight evaluation of each local area can be expressed as:
Oy.
weight...k = mar
),0},
1m1
Oz
5

---

where i is the index of the i-th sample; j E 11. = #1, h is the number obtained through dividing the image height H by the local block height u; k [1, 2I.,,...,w obtained through dividing the image width W by the local block width u; the patch size u of square block is set max (min (H, W) /32. 8); l is the index of the feature space dimension d, 2' is the I-th dimensional feature of the transitional output z;; n is the class index and y" is the classification Ovf" is the gradient information obtained by backpropagation of
W is the number u
prediction for the 1-th class. F2 (.C.A)
n-th class on the I-th dimensional feature. The operation of mar(Tanh() is employed to suppress the negative pixels belonging to other categories that the model does not focus In the absence of mar (Tanh(-)) }} the weight values sometimes do not effectively highlight the target class alone, leading worse performance in feature localization. Next, FSG retains the areas with model- interested features and replaces the areas that the model is not interested in with images generated by the diffusion model. By image fusion, target data with feature differences can be expressed as:
x z,(weights.,.t ra] Ta,(g) [weight,,, rr].
(4)
where r; is controlled by hyperparameter r' is to determine the ratio between areas of model interest and non-interest, r; xx Σソ= Σx= j. a @ is the element-wise multiplication symbol. (9)()) represents the i-th generated data output by the diffusion model; the generated fusion domain
is defined as D. = {(
Fine-tuning Multi-modal Model. To better align the diffusion model with the target domain style, RrED fine-tunes the text encoder from the diffusion model to rectify reasoning errors that arise during the inference process. Before fine-tuning, we introduce the diffusion- reliable semantic information from the diffusion model. Meanwhile, we introduce a matching image encoder from (20) to fine-tune the text encoder. In fine-tuning, we introduce the prompt learning to adapt BUDA by inserting learnable continuous vectors into the original text input, as follows:
predictor to leverage
Prompt Text =
[CL.S], [vy -1). [vm]],
(5)
where [v1]. ... [um] denote prompt word embeddings of the same dimensionality, is the number of context tokens, and [C'LS] is the class name. Then, we propose a task-specific prompt learning loss to accomplish the fine-tuning of multi-modal model, which can be expressed as:
fv = -iim E.D.D. (S(x;) or po (x:))) log Vo():
Vo
where Vi8 is defined as the multi-modas model; during the training period, only prompt word em- beddings [ri].... [Pm]]] are unlocked, while all other parameters are fixed: po(z;) is diffusion model prediction from the diffusion-basse based predictor: S(x;) is the ALS prediction for the i-th sample used as input from the smooth label repository and used for the initialization of the multi-modal model. As shown in Appendix G. when only source domain knowledge is available, domain discrepancy causes the model to fail in adapting well to the target domain. The diffusion model has more reliable seman- tic knowledge than the black-box predictor. Therefore, after initialization, Sxx;) in the fine-tuning process is replaced by pa (2:).
Adaptation Loss in DTR. During the early and middle stages of training, domain discrepancies often result in noisy and unreliable pseudo-labels for the target domain. Such discrepancies amplify erroneous gradients throughout the training process, thereby heightening the probability of compro- mised feature learning and detrimental knowledge transfer [9). Therefore, we propose a guidance correction to use the diffusion model rich in semantic knowledge to guide the learning of the target model, which can be expressed as:
LGc = min EE
Me (,, log Me(..).
where Lee is standard cross-entropy function. The entropy minimization process defined in Loe inherently biases predictions toward sample-dense areas in the feature distribution, consequently diminishing model generalization ability [46). To mitigate this effect, we introduce a conditional constraint loss to limit this impact while maintaining robust feature space consolidation:
Loc = -min
Σ 1=1
No
(G. Mo( (MoG))))MMM))) [M M,
Mo(,),
(8)
Me
j=1

---

Table Accuracies (%) on the Office-Home using ResNet-50 and the VisDA-17 using ResNet- 101. The setting of U, SF, and BP corresponds to UDA, SFDA, and BUDA, respectively. P and D indicate whether external prompts and diffusion model are utilized (v) or not (x), ""Surce-only" refers to using the black-box predictor to evaluate the predicted target samples. The lop-performing BUDA methods are highlighted in bold. The complete results on V1sDA-17 are in Appendix B.
Method
Setting
D
A-CC
A-P
A-R
C-A
C-P
C-R P-A
OfficeHoome
P-C P--
R-A
R-+C
R-P
Mean
Mean
VizM
Source-only
x
44.1
66.9
74.2
54.5
63.3
66.1
528
41.2
73.2
66.1
46.7
77.5
60.6
48.9
HMA
47
60.6
79.1
82.9
68.9
77,5
79.3
69.1
55.9
83,5
74,6
62.3
84,4
73.2
88.1
DAPL
48
54.1
84.3
84.4
74.4
83.7
85.0
74.5
54.6
84.8
75.2
54.7
83.8
74.5
86.9
PDA
49
55.4
85,1
85.8
75.2
85,2
85.2
74.2
55.2
85.8
74.7
55,8
86.3
75.3
89.7
DATUM
72.8
70.6
69.3
72.1
69.9
50.2
73.9
77.1
51.5
75.8
66.7
75.3
S-Fasion
k0
U
57.4
76.0
80.
67.7
76.5
77.6
67.9
56.4
81..2
75.6
62.1
86.4
72.1
86.
DACDM 24
60.4
78.8
82.7
69.6
80.5
79.6
65.2
58.3
83.1
75.8
64.2
85.6
73.6
86.8
DAD 25
62.5
78.6
83.0
70.4
79.2
79.8
70.2
58.3
83.1
76.3
63.5
88.2
74.4
90.0
PLUE
x
73.5
62.9
73.55
74.5
62.2
48.3
78.6
68.6
51.8
81.5
66.9
90.0
C-SDDA 100
SP
x 58.6
80.2
82.9
69.8
78.6
79.0
67.8
55.7
82.3
73.6
60.1
84.9
72.8
87.8
DIFO 32
SF
X
70.6
90.6
88.8
82.5
90.6
88.8
80.9
70.1
88.9
83.4
70.5
91.2
83.1
90.3
DINE 15
BP
X x
52.2
78.4
81.3
65.3
76.6
78.7
62.7
49.6
82.2
69.8
55.8
84.2
69.7
75.6
BiMem 4D
X
66.7
78.7
79.6
65.9
53.6
82.3
73.6
57.8
84.9
71.5
BETA
16
BP
57.2
78.5
82.1
68.0
78.6
79.7
67.5
56.0
83.0
71.9
58.9
84.2
72.1
85.1
RFC
BP
X
57.4
80.0
82.8
67.0
80.6
80.2
68.3
57.8
82.8
72.8
59.3
85.9
72.9
85.2
SEAL
17
BP
58.5
81.4
84.7
71.7
80.4
82.1
72.2
54.3
86.0
76.2
60.6
86.3
74.5
89.2
AEM 19
BP
65.4
88.3
89.5
80,1
90.7
89.7
78.9
61.4
89.9
79.2
63.6
90.8
80.6
89.3
RrED
BP
82.3
93.9
90.0
82.0
93.7
90.1
82.6
83.0
90.4
84.7
83.3
94.1
87.5
91.2
where [-] is the trace of a matrix. The prediction discriminability within mini-batch reaches its peak when the lower bound of Loe and the minimum of Lae are attained simultaneously, yielding fully determined prediction matrices. In DTR, the objective loss for the target model can be expressed as:
(DTRR) Lrask + LGC + yCcc,
where y is a hyperparameter to control the role of the conditional constraint loss Lco-
3.3
Self-Rectifying Reasoning Model
Sample-Version Interaction (SVI. Before the second phase begins, we substitute the original text encoder in the diffusion model for the fine-tuned text encoder with prompt word embeddings. In this regard, diffusion can generate images with more stable target domain styles. Then, we generate differentiated synthetic images according to the process of FSG, and perform interactive learning between the synthetic images and the target images in SVI. For this, our proposed interactive learning can divided into two parts: (1) the former term corrects model reasoning errors by measuring similarity between different versions of predictions, enforcing scattered data distribution boundaries to stabilize around the nearest feature cluster centers; (2) the latter term strengthens feature discrepancies between different versions of predictions to enhance model generalization and prevent overfitting. The interactive optimization can be expressed as:
Lsvr= = -min E Mg a.,..D.. w logg (. - log sim
Roctify reasoning crrors
Strengthen feature discrepancies
(10)
where sim(-) denotes the operation of calculating cosine similarity: w is employed to assign different weights according to the similarities between the features in the target data predictions Mo(x,) and the synthesized data predictions Mo (Tx). The similarity weight w can be formulated as:
w = cxp(- - sort(- ( log sim (Mo
(11)
where sort(.) denotes sorting in descending order and returning the corresponding indices of the samples.
Adaptation Loss in SRM. the SRM stage, our goal is to enable self-rectification of the target model by contrasting the reasoning discrepancies among different versions of the same sample, thereby enhancing its reasoning ability for better adaptation to the target domain. In SRM, the
objective loss for the target model can be expressed as:
[MR(S = Lrast + LSVI-
(12)
7

---

Ground-tree troth
Source-
only
tarning
Aac
mil Lee
Eap
RrED
Blcccle
Bus
Car
Person
Plant
Figure 3: Qualitative and quantitative ablation studies on VisDA-17 using Grad-CAM 45). Each
result is reported when the best accuracy is achieved. Zooming for a clearer view. The complete quantitative results of ablation are reported Appendix E.
To explain why our algorithm RIED works effectively and why it contributes to BUDA, we derive an error bound through theoretical analysis in Appendix C. Moreover, the whole training process of RrED is shown in Appendix D.
4 Experiments
Datasets. RrED is evaluated on four widely-used domain adaptation benchmarks. Office-31 [53] is a
Table 3: Accuracies (%) on the Office-31 using ResNet-50 backbone.
small-scale dataset with 4,110 im-
ages 31 categories from three do- mains: Amazon (A), Dsir (D), and
Method
|eetting
A-+ A-W D- D- W-+ W-D Mean
Source-ealy
x
79.9
76.6
56.4
92.8
60.9
98.5
77.5
Webcam (W). Office-Home [54] is
HMA
47
U
95.8
95.1 1
79.3
99.3
77.6
100
91.2
medium-scale dataset, containing 15.5K images across 65 categories from four domains: Real World (R),
DAPL
48
U
81.7
80.3
81.2
81.8
81.0
81.3
81.2
PDA 49
U
91.2
92.1
83.5
98.1
82.5
99.8
91.2
DATUM
U
81.7
97.3
86.8
Clipart (C), Art (A), and Product (P).
S-Fusion
K7
U
94.8
95.3
78.3
99.1 I
78.6
100
91.0
DACDM
U
97.5
96.9
79.8
98.9
77.7
97.5
91.8
VisDA-17 (55) is large-scale bench- mark, including 152K synthetic im-
DAD
25
U
95.6
98.5
$1.4
99.5
82.2
100
92.8
SF
69.6
97.9
ages (source) and 55K real-world im-
C-SFDA
10
SF
96.2
93.9
77.3
98.8 8
77.9
99.7
90.5
ages (target) across 12 categories, em- phasizing the synthetic-to
SFIDA 9
SF
95.8
92.1 1
75.7
99.0
76.8
99.8
89.9
DIFO
SF
98.8
domain gap. DomainNet [56] is the most ex- tensive benchmark, with about 600K images. Following previous methods the evaluation setup for adap- tation scenarios involves merely 4 do-
DINE
15
BP
X
91.6
$6.8
72.2
96.2
73.3
98.6
86.4
BiMem
46
BP
92.8
88.2
73.9
96.8
75.3
99.4
87.7
BETA 10
BP
93.6
88.3
76.1
95.5
76.5
99.0
88.2
RFC
BP
77.5
98.1
89.2
17 (22
SEAL 17
BP
95.1
88.3
77.6
96.0
76.7
99.3
88.8
AEM 19
BP
95.1
94.0
81.8
98.2
82.6
99.4
91.9
ReED
BP
97.8
95.9
83.7
99.1
84,5
99.8
93.5
mains with 126 categories, including Real (R), Clipart (C), Painting (P), and Sketch (S). There is need to overcome the domain gaps
among 12 subtasks with different adaptation scenarios.
Comparison Methods. We evaluate the performance of RrED by comparing it with several related methods across UDA, SFDA, and BUDA settings. For UDA, we conduct comparisons with HMA
477. DAPL [48), DATUM [50] S-Fusion 26) DACDM 24) DAD 25 PDA (9), and AD-CLIP
57
For SFDA, we compare with PLUE 511. C-SFDA 10 2
9 TPDS (58) and DIFO
52
For BUDA, we compare with previous SOTA methods, including DINE 15. BiMem 40).
BETA [66) RFC [1], SEAL [17], and AEM [19]. Among them, the previous methods include
DATUM, S-Fusion, DACDM, and DAD use the diffusion model external prompt; DAPL, PDA.

---

Table 4: Accuracies (%) on the DomainNer using ResNet-50 backbone.
Method
Setting
D|C=p
C-R
C-8$
P-C
P-R
P-4S
R-C
R-PP
R-8
S-C
S-PP
S-R
Mean
Source only
52.1
40.7
56.5
48.3
35.2
50.5
43.7
DAPL
48
U.
x
72.4
87.6
65.9
72.7
87.6
65.6
73.2
72.4
66.2
73.8
72.9
87.8
74,8
AD-CLIP
U
71.7
88.1
73.2
86.9
73.6
73.0
68.4
72.3
74.2
89.3
75.2
PLUE
SF
59.8
74.0
56.0
61.6
78.5
$7.9
61.6
65.9
53.8
67.5
64.3
76.0
64.7
TPDS
58
SI
62.9
77.1
59.8
65.6
79.0
61.5
66.4
67.0
58.2
68.6
64.3
75,3
67.1
DIFO
SF
76.6
87.2
74.9
80.0
87.4
75.6
80.8
77.3
75.5
80.5
76.7
87.3
80.0
DINE
13
BP
43.7
61.5
44.0
44.0
62.9
38.7
54.3
53.1
41.7
54.0
44.5
59.3
50.1
BETA
16
BP
48.3
64.7
49.2
-99.6
66.3
43.4
58.1
57.7
45.7
58.7
49.9
63.1
54.5
SEAL
17
BP
49.5
67.9
48.7
49.9
68.5
44.0
601.6
57.4
46.7
59.2
50.4
67.1
55.8
AEM
19
BP
66.4
77.8
72.1
80.0
86.7
69.1
79.5
76.6
67.8
78.1
72.6
77.6
75.4
RrED
BP
76.8
87.9
71.9
81.5
86.7
74.7
83.5
80.1
73.0
81.7
78.3
88.3
80.5
F Home
(2.3
$3.5
AEM VR: XIBR3
Zuntra
Number
humba
Passent
Tovert
(a) 7 in VisDA-17
(b) y in OfficeHomee
(c) in VisDA-17
(d) r Office-Home
Figure 4: The accuracy trends of predictions the VisDA-17 and Office-Home (A (A-C). controls the effect of Lee, as shown (a) and (b). r determines the ratio between regions of interest and non-interest, shown in (c) and (d).
AD-CLIP, DIFO, and AEM use the multi-modal model CLIP 20 as an external prompt. Specific implementation details are shown in Appendix F.
Results. As reported in Tables 2, 3, and 4, RrED achieves consistently superior performance over previous SOTA methods across all four benchmarks. We choose DINE 15] as the baseline. In terms of average accuracy, RrED surpasses the prior BUDA approach AEM 19 by 6.9%, 1.9%, 1.6%, and 5.1% on the Office- VisDA-17, Office-31, 31, and DomainNet, respectively. Furthermore, compared to the diffusion- on the Office-Home. These results demonstrate, by introducing the diffusion model into black-box learning through a two-stage strategy and fine-tune tuning the diffusion model's 's text encoder, RrED more effectively utilizes the diffusion model to enhance target model discriminability compared to previous methods that either generate semantically-rich additional samples or directly perform prediction using diffusion model. Furthermore, RrED exhibits significantly superior performance in scenarios with more stringent security protection constraints compared to previous diffusion-based DA methods.
UDA method DAD (25), RrED achieves a maximum improvement of 13. 1%
Ablation Study. Figure presents ablation studies on the VisDA-17 using Grad-CAM [45 visualizations. RrED is inspired by human decision docision-making systems and aims to refine model reasoning through a two-stage correction process. In computer vision, the model's reasoning process mirrors human decision-making by utilizing existing knowledge and feature similarity computations to determine whether targets in complex scenes meet the task requirements. As indicated in Figure RrED's two-stage correction focuses on improving the diffusion model's 's reasoning ability, thereby guiding and enhancing the reasoning of the target model. Therefore, verifying whether the model's reasoning ability improves during optimization is a central focus of our experiments. To verify the model's reasoning ability, we introduce Grad-CAM, which is well-known for validating the reasoning ability of models. For the functional, we employ Grad-CAM in Figure 3 to highlight the vital and irreplaceable roles play in the overall performance. As shown in Figure 3, Grad-CAM clearly demonstrates that our model exhibits stronger reasoning capabilities, better object recognition, and more precise capture of fine-grained features in target samples compared with existing SOTA methods. In DTR period, FSG is designed to prevent the diffusion- generated images from causing irreversible negative effects. When FSG is removed, the uncontrolled images solely generated by the diffusion model mislead the target model, resulting in significant performance drop. Lee and Lee act in a complementary manner: Loe enhances the model's 's discrininative capability via cross-entropy learning, while Loe mitigates the sample enrichment effect introduced by Loc to improve model's generalization. When Lce is removed, the target model exhibits the overfitting phenomenon prematurely. Only when both components work jointly can the full effectiveness be

---

realized. In SRM period, Lsvı performs self-rectifying inference learning by integrating interactive learning with the samples generated by the diffusion model whose text encoder has been fine-tuned. The combination of Levr with the fine-tuned diffusion model allows the overall model to capture key features more accurately. From the perspective of model reasoning, we observe from Figure that (1) FSG enhances generalization and helps the model attend to the correct class-discimminative regions; (2) LGc and Lce jointly determine the approximate region feature extraction from the target model; (3) the fine-tuned text encoder and Lsvi jointly optimize features for the region of interest of the model. More ablation studies are shown in Appendix E.
Parameter Analysis and Comparison. As shown in Figure 4, the effects under different values y and r are presented. 7 controls Lce modulate the distributional density of samples within the feature space. When À is equal to O, Lec is not effective; when À is equal to 5, excessive amplification of feature discrepancies severely impairs the model's ability to distinguish between samples. For the large-scale dataset VisDA-/7, appropriate À leads to notable improvements. For the medium-scale Office-Home, the differences between samples more significantly affect the model's discriminative ability. determines the ratio between regions of interest and non-interest. When O, FSG outputs the original target domain samples. When is 200%, FSG fails, and the outputs are entirely generated by the diffusion model. These results indicate that selecting an appropriate can effectively enhance the generalization ability of the target model, while also demonstrating that images generated solely by the diffusion model unreliable. More visual comparisons and further analysis are provided in Appendix G. The computational consumption is presented in Appendix H.
5 Conclusion
In this paper, we observe that existing methods have certain weaknesses. To tackle them, we propose diffusion-based algorithm, RrED, the first to introduce the diffusion model into the BUDA task and perform task-specific fine-tuning on its text encoder. Inspired by the human decision-making process, RrED is composed of DTR and SRM stages: DTR facilitates the target model's training by correcting reasoning errors from the diffusion model while harnessing its implicit knowledge: SRM refines the learning process by utilizing the target model's differential reasoning in combination with samples produced by the fine-tuned diffusion model. The experimental results show that RiED enhances class discrimination ability and model reasoning ability, ultimately achieving performance improvements far exceeding previous SOTA methods on all evaluated datasets.
Acknowledgment
This work was supported in part by the National Natural Science Foundation of China under Grant 62176162 and in part by Guangdong Basic and Applied Basic Research Foundation under Grant 2023A1515012875 and Grant 2022A1515140099.
References
[1] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition." in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.
[2] J. Cha, S. Chun, K. Lee, H.-C. Cho, S. Park, Y. Lee, and S. Park, "Swad: Domain generalization by seeking flat minima." " Proceedings of the 34th International Conference on Neural Information Processing Systems (NeurIPS), 2021, pp. 22 4054405200 418.
[3] K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy, "Domain generalization: A survey," IEEE Transactions Pattern Analysis and Machine Intelligence (TPAMI), vol. 45, no. 4, pp. 4396- 4415, 2023.
[4] P. O. Pinheiro, "Unsupervised domain adaptation with similarity learning." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp- 8004-8013.
10

---

[5] D. Hu, J. Liang, X. Wang, and C.-S. Foo, Dpseudo-calibattonn Improving predictive uncer- tainty estimation in unsupervised domain adaptation," in Proceedings of the 41th International Conference Machine Learning (ICML), 2024, pp. 19 1944-19933
[6] B. Pu, X. Lv, J. Yang. H. Guannan, X. Dong, Y. Lin, L. Shengli, T. Ying, L. Fei, M. Chen, Z. Jin, K. Li, and X. Li, "Unsupervised domain adaptation for anatomical structure detection in ultrasound images," " in Proceedings of the 41th International Conference on Machine Learning (ICML), 2024, pp. 41 204-41 220.
[7] Y. Zou, Z. Yu, V. Kumar. and J. Wang. "Unsupervised domain adaptation for semantic segmentation via fass-balanced self-training," in Proceedings of the 15th European Conference on Computer Vision (ECCV), 2018, pp. 289-305.
[8] X. Jin, Y. Park, D. Maddix, H. Wang, and Y. Wang. "Domain adaptation for time series forecasting via attention sharing." in Proceedings of the 39th International Conference on Machine Learning (ICML), 2022, pp. 10280-10297.
[9] U. Hwang. J. Lee, J. Shin, and S. Yoon, "SF(D))? Source-free domain adaptation through the lens of data augmentation," in International Conference on Learning Representations (ICLR), 2024. [Online]. Available:
[10] N. Karim, N. C. Mithun, A. Rajvanshi, H.-p. Chiu, S. Samarasekera, and N. Rahnavard, "C-sfda: curriculum learning aided self-training framework for efficient source free domain adaptation." " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 24 120-24 131.
[11] J. Liang, D. Hu, and J. Feng, "Do we really need to access the source data? Source hypoth- esis transfer for unsupervised domain adaptation." in Proceedings of the 37th International Conference on Machine Learning (ICML), 2020, pp. 6028-6039.
[12] Y. Wang, J. Liang, and Z. Zhang. "A curriculum-style self-training approach for source-free semantic segmentation." IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 46, no. 12, pp. 9890-9907, 2024.
[13] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," in Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS), 2014, pp. 2672-2680.
[14] Z. Fei, M. Fan, L. Zhu, J. Huang, X. Wei, and X. Wei, "Masked auto-encoders meet generative adversarial networks and beyond," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 24 4449..
[15] J. Liang, D. Hu, J. Feng, and R. He, "Dine: Domain adaptation from single and multiple black-box predictors." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 7993- 8003.
[16] J. Yang, X. Peng, K. Wang, Z. Zhu, J. Feng. L. Xie, and Y. You, "Divide to adapt: Mitigating confirmation bias for domain adaptation of black-box predictors," " in International Conference on Learning Representations (ICLR), 2023, [Online]. Available:
[17) M. Xia, J. Zhao, G. Lyu, Z. Huang, T. Hu, G. Chen, and H. Wang, "A separation and alignment framework for black-box domain adaptation." " in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2024, pp. 16 005-16013.
[18) Zhang, C. Shen, S. Lü, and Z. Zhang, "Reviewing the forgotten classes for domain adaptation of black-box predictors," in Proceedings of the AAAI Conference Artificial Intelligence (AAAI), 2024, pp. 16 830-16 837.
[19] S. Xiao, M. Ye, Q. He, S. Li, S. Tang, and X. Zhu, "Adversarial experts model for black-box domain adaptation," in Proceedings of the 32nd ACM International Conference on Multimedia (ACMMM), 2024, pp. 8982-8991.
11

---

[20] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, Mishkin, J. Clark, G. Krueger, and I. Sutskever, "Learning transferable visual models from natural language supervision." in Proceedings of the 38th International Conference on Machine Learning (ICML), 2021, pp. 8748-8763.
[21] J. Ho, A. Jain, and P. Abbeel, "Denoising diffusion probabilistic models," in Proceedings of the 33th International Conference on Neural Information Processing Systems (NeurIPS), 2020.
[22] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, "High-resolution image synthe- sis with latent diffusion models," in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10684-10695.
[23] Z. Xing, Q. Feng, H. Chen, Q. Dai, H. Hu, H. Xu, Z. Wu, and Y.-G. Jiang, "A survey on video diffusion models," ACM Computing Surveys (CSUR), vol. 57, no. pp. 1-42, 2024.
[24] Y. Zhang, S. Chen, W. Jiang, Y. Zhang, J. Lu, and J. T. Kwok, "Domain-guie gundid conditional diffusion model for unsupervised domain adaptation." Neural Networks (NN), vol. 184, p. 107031, 2025.
[25] D. Peng. Q. Ke, A. Ambikapathi, Y. Yazici, Y. Lei, and J. Liu, "Unsupervised domain adaptation via domain-adaptive diffusion," IEEE Trunsactions on Image Processing (TIP), vol. 33, pp. 4245-4260, 2024.
[26] K. Song, L. Han, B. Liu, D. Metaxas, and A. Elgammal, "Stylegan-fusion: Diffusion guided domain adaptation of image generators," in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 5463.
[27] "Cognitive reflection and decision making," Journal of Economic Perspectives (JEP), vol. 19, pp. 25 42, 2005.
[28] D. Kahneman, Thinking, Fast and Slow. Macmillan, 2011.
[29] W. Hu, W. Z. Dai, and 7. H. Zhou, "Efficient rectification of neuro-symbolic reasoning in- consistencies by abductive reflection," in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025, pp. 17 333-17341.
[30] L.-W. Cai, Z. Dai, Y.-. Huang. Y.F. Li, S. H. Muggleton, and Y. Jiang, "Abductive learning with ground knowledge base," in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), 2021, pp. 1815-1821.
[31] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, "Domain-ad aderrrriia training of neural networks," Journal of Machine Learning Research (JMLR), vol. 17, pp. 2096-220622 2030, 2016
[32] L. Hoyer, D. Dai, H. Wang, and L.. Van Gool, "Mic: Masked image consistency for context- enhanced domain adaptation." in Proceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 11 721-111117 732.
[33] I. Hoyer, D. Dai, and I.. Van Gool, "Hrda: Context-aware high-resolution domain-adaptive semantic segmentation," " in Proceedings of the 17th European Conference on Computer Vision (ECCV), 2022, pp. 372-391.
[34] M. Khodabandeh, A. Vahdat, M. Ranjbar, and W. G. Macready, "A robust learning approach to domain adaptive object detection," in Proceedings the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 480-490.
[35] Y. Chen, W. Li, C. Sakaridis, D. Dai, and L. Van Gool, "Domain adaptive faster r-cnn for object detection in the wild," in Proceedings the IEEF Conference on Computer Vision and Pattern Recognition ( (CVPR), 2018, pp. 3339-3348.
[36] R. Cai, J. Chen, Z. Li, W. Chen, K. Zhang, J. Ye, Z. Li, X. Yang, and Z. Zhang, "Time series domain adaptation via sparse associative structure alignment," " in Proceedings of the AAAI Conference Artificial Intelligence (AAAI), 2020, pp. 6859-6867.
12

---

[37] M. Ragab, E. Eldele, Z. Chen, M. Wu, C.-K. Kwoh, and X. Li, "Self-supprrised autoregressive domain adaptation for time series data," IEEE Transactions on Neural Networks and Learning Systems (TNNLSS,, vol. 35, no. 1, pp. 1341-1351, 2024.
[38] M. Jing, J. Li, K. Lu, L. Zhu, and H. T. Shen, "Visually source-free domain adaptation via adversarial style matching," IEEE Transactions Image Processing (TIP), vol. 33, pp. 1032- 1044, 2024.
[39] S. Yang, Y. Wang. K. Wang. S. Jui, and van de Weijer, "Attracting and dispersing: simple approach for source-free domain adaptation," in Proceedings of the 35th International Conference on Neural Information Processing Systems (NeurIPS), 2022, pp.
[40] J. Zhang, J. Huang, X. Jiang, and S. Lu, "Black-box unsupervised domain adaptation with bi-directional atkinson-shiffrin memory," in Proceedings of the IEEE/CVE International Con- ference on Computer Vision (ICCV), 2023, pp. 71771-11 782.
[41] H. Zhang, Y. Zhang, K. Jia, and L. Zhang, "Unsupervised domain adaptation of black-box source models," ArXiv, vol. abs/2101 000899, 2021.
[42] A. C. Li, M. Prabhudesai, S. Duggal, E. Brown, and D. Pathak, "Your diffusion model is secretly a zero-shot classifier," Proceedings of the IEEF/CVF International Conference on Computer Vision (ICCV), 2023, pp. 2206-2217.
[43] Y. Xu, Y. Zhao, Z. Xiao, and T. Hou, "Ufogen: You forward once large scale text-to-image generation via diffusion gans," in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 81 968-8206.
[44] Guo, J. Zhao, C. Du, Y. Wang, C. Ge, Z. Ni, S. Song. H. Shi, and G. Huang. "Everything to the synthetic: Diffusion-driven test-time adaptation via synthetic-domain alignment," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp.
[45] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp- 618-626.
[46] S.C, S. Wang. J. Zhuo, L. Q. Huang, and Q. Tian, "Towards discriminability and diversity: Batch nuclear-nore norm maximization under label insufficient situations," in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 3941- 3950.
[47] L. Zhou, M. Ye, X. Zhu, S. Xiao, X.-Q. Fan, and F. Neri, "Homeomorphism alignment for unsupervised domain adaptation." " Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), 2023, pp. 18 653-61111 18 664.
[48] C. Ge, R. Huang, M. Xie, Z. Lai, S. Song, S. Li, and G. Huang, "Domain adaptation via prompt learning," IEEE Transactions on Neural Networks and Learning Systems (TNNLS), vol. 36, no. 1, pp. 1160-1170, 2025.
[49] Bai, M. Zhang, W. Zhou, Huang, Z. Luan, D. Wang, and B. Chen, "Prompt-based distribu- tion alignment for unsupervised domain adaptation." in Proceedings of the AAAI Conference Artificial Intelligence (AAAI), 2024, pp. 729-737.
[50] Y. Benigmim, S. Roy, S. Essid, V. Kalogeiton, and S. Lathuilière, "One-shot unsupervised domain adaptation with personalized diffusion models." in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 698-708.
[51] M. Litrico, A. Del Bue, and P. Morerio, "Guiding pseudo-labels with uncertainty estimation for source-free unsupervised domain adaptation," in Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 7640-7650.
13

---

[52] S. Tang, W. Su, M. Ye, and X. Zhu, SSource-free domain adaptation with frozen multimodal foundation model," in Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2024, pp. 23 23711-23722 720.
[53] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, "Adapting visual category models new domains," in Proceedings of the 11th European Conference on Computer Vision (ECCV), 2010, pp. 213- 226.
[54] H. Venkateswara, Eusebio, S. Chakraborty, and S. Panchanathan, "Deep hashing network for unsupervised domain adaptation." in Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), 2017, pp. 5018-5027.
[55] X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and K. Saenko, "Visda: The visual domain adaptation challenge," ArXiv, vol. abs/1710.0691 2017.
[56) X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang. "Moment matching for multi- source domain adaptation." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 1406-1415.
[57] M. Singha, H. Pal, A. Jha, and B. Banerjee, "Ad-clip: Adapting domains in prompt space using clip," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 4355 4364.
[58] S. Tang, A. Chang, F. Zhang. X. Zhu, M. Ye, and C. Zhang, "Source-free domain adaptation via target prediction distribution searching," International Journal of Computer Vision (IJCV), vol. 132. no. pp. 654-672, 2023.
[59] Alquier, "User-friendly introduction to pac-bayes bounds," Foundations and Trends in Machine Learning Found (Trends Mach. Learn.), vol. 17, no. 2, pp. 174-303, 2024.
[60] P. Alquier, J. Ridgway, and N. Chopin, "On the properties of variational approximations of gibbs posteriors," " Journal of Machine Learning Research (JMLR), vol. 17, no. 236, pp. 1-41, 2016.
[61] S. Ben-David, J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Vaughan, "A theory of learning from different domains," Machine Learning, vol. 79, pp. 151-175, 2010.
[62] M. Long, Y. Cao, Z. Cao, J. Wang, and M. I. Jordan, "Transferable representation learning with deep adaptation networks." " IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 41, no. 12, pp. 3071-3085, 2019.
[63] M. Jing, J. Li, K. Lu, I.. Zhu, and H. T. Shen, "Visually source-free domain adaptation via adversarial style matching," " IEEE Transactions on Image Processing (TIP), vol. 33, pp. 1032- 1044, 2024.
[64] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2009, pp. 248-255.
[65] L. van der Maaten and G. Hinton, "Visualizing data using t-sne," Journal of Machine Learning Research (JMLR), vol. 9, no. 86, pp. 2579-2605, 2008
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims of this paper be found in the Abstract and Introduction sections, which accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We provide the limiting analysis in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
We provide the theoretical analysis in Appendix C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper fully discloses all necessary information for reproducibility. The whole training process shown in Algorithm 1. The implementation details in Appendix The experimental code and the main code are available in the Supplementary Materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The experimental code and the main code available in the Supplementary Materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.cc)
aublic/guudees/Codeetbbmsss
www.sssiinooiiillo for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wttps/cccppppclgguiuaeeeaeccuuea
Stteee///ooSSSeesssS.Si.a ssissssliccyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the details are described, including hyperparameters, experiments, and datasets.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the computational cost comparison in Table 7.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: httpp::///nurrige wtw.rpsu.p...// cc/ppullic/thiiccccueeeel......
Answer: [Yes]
Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide the broader impacts in Appendix I.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
In the Related Works and Introduction sections, we have explained that our aim is to provide better data privacy protection with more flexible portability to prevent the leakage of source data or the trained source model.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the code and dataset utilized in this work are publicly available and are only intended to compare the performances of different algorithms classification tasks.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This submission poses no such risks.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This submission poses no such risks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This submission poses no such risks.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
This submission poses no such risks.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htt.pp///earrrpcccc///ooeeeerrre for what should or should not be described.
21

---

Appendix
A. Diffusion Process for BUDA
standard diffusion model (c.g., DDPM (21]) consists of two core components: a forward diffusion operator q and reverse denoising operator p. In the forward process, DDPM diffuses the target data distribution by gradually injecting Gaussian noise into the data point x; over the total ST steps using fixed Markov chain. The forward diffusion operator q can be expressed as:
{<>|| = Naa); 1 -, ,,,1).
(13)
where j is defined as the diffusion timestep. E [0, 1..,,...TTJ ST]: 33 is the fixed variance scheduler that controls the scale of the Gaussian noise, 83 E [0. 11: represents the probability density function of the Gaussian distribution; I is unit vector; and B, is the covariance matrix. In the reverse denoising operator, DDPM concentrates the target data distribution by gradually generating a sequence of denoised images Zgg over the same T steps. The reverse denoising operator p can be expressed as:
p(an ( Nx
(
ak
To (x k2),
(14)
VOL
VI
where l is defined as the denoising timestep. E [ST, ST 1..., 0]; as = Be: a. -I o;; k) predicts the noise at the current timestep and denoises the corresponding input data zng. ( k) [0. 1]. To introduce the diffusion model into BUDA, we follow 422 to add
and on (igh
predictor pe with fixed weights to judge the category of the target data Ty. The judgment process of the predictor is as follows:
exp(-Exerlla - oo L;)1
(o(Tylaaa) Ei=1 exp(-Exer|lo on (x [))l33
(15)
where Lj is a low-dimensional text embedding corresponding to the j-th class of the i-th sample my b is the number of classes; follows the standard Gaussian distribution (O, 1). In this work, we introduce an image encoder with fixed weights from CLIP [20] and leverage the predictor along with the image encoder during the diffusion-targe model rectification to fine-tune the text encoder. Moreover, before the SRM period, the fine-tuned text encoder is fed back to the diffusion model to enable more controllable image generation and adapt the target model the BUDA task.
B. Supplement of Complete Experimental Results
As shown in Table 5, the comparison results demonstrate that our RrED effectively employs two-stage strategy guided by diffuusion model for target model optimization, achieving significantly
Table 5: The complete accuracies (%) ) on the VisDA-17 /7 using ResNet- 101 backbone.
Method
Setting
plane
hike
bus
car
horse
knife
meycle
person
plant
skthrd
train
truck
Mean
Source-only
64.3
24.6
47.9
75.3
69.6
8.5
79.0
31.6
64.4
31.0
81.4
9.2
48.9
HIMA 47
U
97.6
88.4
84.3
76.0
98.4
97.1
91.3
81.4
97.0
96.7
88.8
60.7
88.1
DAPL 4H
D
x
97.8
83.1
88.8
77.9
97.4 4
91.5
94.2
79.7
88.6
89.3
92.5
62.0
86.9
93.6
97.3
95.3
65.3
DATUM
50
U
85.7
76.4
79.7
75.4
84.1
82.3
80.4
76.7
81.9
82.6
78.4
20.2
75.3
Fusion 26
U
92.9
83.7
89.3
87.0
95.3
92.7
90.1
86.8
92.2
93.2
88.3
42.0
86.1
DACDM 24
U
96.2
84.8
83.2
73.3
94.8
96.6
91.0
88.2
93.0
93.4
87.5
59.7
86.8
96.9
93.7
42.5
PLUE 51
SF
X
97.3
96.2
90.5
91.8
90.0
94.2
87.4
87.7
97.0
84.3
93.0
81.0
90.0
CFDA 10
SF
97.6
88.8
86.1
72.2
97.2
94.4
92.1
84.7
93.0
90.7
93.1
63.5
87.8
SF(DAy2
SF
96.8
89.3
82.9
81.4
96.8
95.7
90.4
81.3
95.5
93.7
88.5
64.7
88.1
DIPO
97.7
90.5
83.6
96.7
95.8
74.1
92.4
93.8
92.9
65.5
DINE 15
BP
X
X
81.4
86.7
77.9
55.1
92.2
34.6
80.8
79.9
87.3
87.9
84.3
58.7
75.6
BETA 16
BP
94.9
61.1
93.1
92.9
91.9
55.0
RFC 18
BP
X
95.6
89.7
87.8
75.8
96.5
96.5
90.4
82.8
96.0
70.0
85.7
55.1
85.2
SEAL 17
BP
97.9
88.0
73.5
97.1
92.4
93.9
95.6
91.2
66.4
AEM 19
BP
98.6
88.1
89.7
74.8
98.0
93.9
93.0
89.3
90.1
97.2
95,2
63.5
89.3
ReED
BP
97.5
91.9
88.
88.0
98.1
96.9
94.3
88.8
96.6
96.6
94.1
63.8
91.2
22

---

101
Source-only: 48.9%
DINE 15 : 75.6%
BETA 16 85.1%
AEM 19
89.3%
SEAL 17
89.2%
RrED: 91.2%
Figure 5:
Classification results on VisDA-17 are visualized with a confusion matrix. Note that all these results are obtained through the evaluation which is conducted in the same experimental environment. (Zooming in for a clear view)
greater improvements on the large-scale benchmark VisDA-17 [55]. Furthermore, we observe that RrED does not outperform some comparison methods [7 [9] on certain classes. We attribute this to the fact that our target model is trained under the guidance a diffusion model, which tends to focus on broad class distinctions to enhance overall discriminative ability. In contrast, the distillation-ban method SEAL 177 exhibits slight overfitting to a few specific classes (e.g., the "bike" and "truck" class), resulting in higher recognition accuracy those classes but reduced performance The CLIP-based method AEM [9] demonstrates notable discriminative power on specific classes. Based on our analysis of the CLIP model [20], we find that these classes are often overrepresented during CLIP pretraining. For example, there is a strong similarity between the "person" class in the target domain and pretraining classes such "baseball player". "bridegroom", and "scuba diver" in CLIP model. In contrast, classes that are absent (e.g., the "knife" class) or rarely seen (e.g., the addition, we
others.
"plant" class) during pretraining tend to have much lower recognition accuracy. supplement the classification visualization of the VisDA-17 in Figure 5. For a fair comparison, all the classification visualizations are obtained in the same experimental environment. These results highlight that our RrED method substantially surpasses other BUDA approaches in improving class discrimination ability.
C. Theoretical Justifications
We provide theoretical justifications grounded in the generalization bound of reasoning to clarify the working mechanism of our algorithm.
First, we adopt PAC-Bayes theory [59] for the classification task to optimize the target model with the uncertainty estimation of the black-box predictor.
Theorem 660. Given a target data distribution Dr. hypothesis H, and a prior distribution т over the hypothesis space O. For any (0, 1] and À > O, with probability at least - over the target samples x, ~ Dr, for all posteriors we have:
BoCH) [[(H)] S Ep(H) (HI) +
+ (KE((Pl))
log v Wana (A,
(16)
where VZ,,((A, n) log (exp(A((TT))))) Z(H))))
Lemma [59]. The PAC-Bayes bound, involving constants and n, as introduced Theorem 1, minimized by the Bayesian posterior p(H), which represents the distribution over e.
23

---

Proof. The Donsker-Varee Varadhan's 's change of measure states that for any measurable function Φ θ - E. we have:
E & (KI(T loggE,rr)eexx(((())
(17)
Thus, with (H = M(CH - x()) and Vp over hypothesis space + we have:
Bp(th) [X (
- -Z(H, = A (Epa))((t)) (Ept)) ax))
S DKL(PllT) + log BA(H) exx (* (c(H)) [(H,
(18)
For the non-negative random variable (x+))) == (exm)ooxp(A(((1)) Z(H, xx))] we apply Markov's inequality on it, and have:
1 T (E 21-7. P(5
(19)
This implies that with probability at least 1 - T over the choice of de ~ Dr. we have Vp over hypothesis space O:
(B s EACH) (H))
(H)) + (DDr((l|T) + log _ + Vn (2,
2 21-7, (20)
where L a (X, n) = log (H) ( Be,,D, (exp(A(C((H) - [(H)), and we prove the statement of Theorem During target model training, as just as in Eq. (2), we utilize My as the prediction of posterior distribution and S(x;) as the prediction of prior distribution. Therefore, the upper bound of our target model can be expressed as:
1 N₂ i=1 [Cother + 1
X Dre (Mr
(21)
where N₂ is defined as the number of the target data x4. Following previous BUDA works 40
15
19
as presented Eq. (2), is set to 1 in the BUDA task. Moreover, Lother varies in different works. For example, in RrED, Lother = Loe + Loe in the first stage DTR, and Lother Loc + Lsvı in the second stage SRM. In summary, this proof fills the theoretical knowledge gap regarding the black-box predictor in previous BUDA works.
Generalization Bound. Since our target model is trained in unlabeled target domain data and further generates fusion data with feature differences based on the diffusion and our FSG module, we denote ~ D₂ as the real sample distribution of the target domain and T. D, as the generated fusion sample distribution of the target domain. And denote y/ as the predicted labels T,, For the corresponding generated fusion samples, Dt. are the predicted labels of the target domain. D. uploaded to a black-box predictor to obtain hard predictions from source model trained on the source domain D.. where I, D, as the sample distribution of the source domain. The pioncering study [61] on theoretical analysis for domain adaptation provide the generalization bound. Following [611 let H denote hypothesis, which can be expressed as:
c. (H, vi) S c. (H, vo) + dran (Di. D.) + 4,
(22)
where ip denotes the shared error of the ideal joint hypothesis, & = min(e, (H, Vo). ,,,,,, e (HI, yc)) dnan (D., D.)
2sup [En,-D, [H(x) + H*:)).) - En,-D, [H(x) 4 H' (H, y1) is the expected error of the target sample distribution; 6, (H, y,) is the expected error of the source sample distribution, which is obtained from the black-box predictor. In the BUDA setting, although we do not have the source domain data Zs, we can obtain hard predictions P. from the black-box predictor. Therefore, according to the theory [62 63) of source data absence, e, (H, ye) is small and can be ignored, so we do not need to obtain x, and ys in BUDA.
Then, we model a generated fusion domain distribution D, that is distributed similarly to the target distribution D,. To reduce the classification error on the target domain, the distributions D.,, Dr, and D, should be substantially similar to each other. Therefore, the generalization bound in RrED can be transformed into:
6 (H, y1) < E. (H, w.) + dnan (Dr,D. + P1
(23)
24

---

where (1 min(4 vo), domain distribution, which can be expressed as:
, E. (H, ū));; & (H. is the expected error of the generated fusion
Ex (H, (t) S cs (H, U.) + du (Dr, D.) + 42:
(24)
where 02 min(G (H, ū+) <x (H, y.)) Thus, our final generalization bound can be defined as:
Et (H, yr) f Es (H, y.) + dnan D.) + dnan(D;, Dz) + 91 + 92.
(25)
For Eq. (25), we analyze each component in detail in this paragraph:
's (H, is the expected error of the source sample distribution, During the training of the source model, the error between the source domain data and its true labels is minimized by cross-entropy loss. Thus, in the early stages of training, we can obtain good training results for the source samples through the black-box predictor. As the training progresses, the model gradually adapts to the distribution of the target domain with Adaptive Label Smoothing (ALS) [1]. The ALS maintains source domain knowledge, enabling the model to learn target domain knowledge while preventing the forgetting of source domain knowledge. Therefore, according to theories [62 63 C, (H, ya) is small in the whole training.
Instead of reducing duann (D., D.) in Eq. (16), our goal is to reduce dnan(D; D.) and dr (De, D.). For dzan (Dr, D.), it depends on the expected error of the disagreement be- tween two hypothesis on the target data and the generated fusion data distribution of the target domain. During the whole training, we design the FSG module to determine which regions should be composed of synthetic images. Dj is generated from D,, preserving key features of Dj while adding differential features generated by the diffusion model. Therefore, the distribu- tion divergence dr..n (D₁, De) is small. For Fnan(D; D,), we obtain that d,ann (D,, D,)
2 HH E BR (Z:) & H()))] - -E,,D, (1x2)) * Hzzz))] As the training progresses, DTR aligns I2 and ãN by continuously minimizing the cross- cross-entropy loss to facilitate the target model's training: SRM narrows the feature space distance between 2, and T5 by contrasting their differences. while enhancing the model's discriminative and generalization abilities by increasing dissimilarities with other samples. Therefore, R, H (w+) * H' [7:)) ER,,D (x+) # H' (xx4) and it is continuously reduced during training by minimizing LGo and Lrask- Meanwhile, Lee and Lsvi prevent overfitting of the target model. For Ez,~D, (xx,) * HH (zx)), according to the previous works [5 16), the ALS maintains a source knowledge base and use Lrosk to maintain the balance between source knowledge and target knowledge. Therefore, En.nD, (xx)) o H' (z.)] always maintains small value during the whole adaptation phase.
101 + 22 denotes the shared error of the ideal joint hypothesis, which is assumed to be a sufficiently small constant that reflects the complexity of the hypothesis space (62).
D. The Whole Training Process
Our pseudocode for the training process is shown in Algorithm In addition, our experimental and main code are available in the Supplementary Material.
E. Supplement of Complete Quantitative Ablation Experimental Results
As shown in Table 6, we report the complete quantitative results of ablation, and all the results include the task task-speeific loss. FSG is the key module of our work to prevent the cifffusion-gneeaale images from causing irreversible negative effects. When FSG and FT are not used, the results indicate that directly applying the default Stable Diffusion model, without adaptation leads to a sharp performance drop. In contrast, our method exhibits highly task-aware sensitivity to the structural characteristics of the Stable Diffusion model, enabling it to better leverage its semantic knowledge for downstream BUDA tasks. The effective knowledge learning of the target model through Lae and Lee can only be achieved when FSG utilized. When Lec is not used, the combined effect of Lrask and Lee enforces rapid sample clustering, which leads to overfitting of the target model. LCo mitigates the sample enrichment effect to improve target model's generalization. In this regard, both Lrosk and Lec can benefit from this process. Levi is to integrate interactive
the downstream task,
25

---

Algorithm RrED for BUDA task.
Target samples D: {(x black-box hard predictions Ps; diffusion model with the pre- dictor ps: multi-modal modal model Vo E {image encoder Is, text encoder Toh: and target model Me (feature extractor fo, prediction classifier ce). Parameter:
Input:
Training epoch learnable prompt emhedding model parameter 0; and hyperparameters
I: Initialize: initialize smooth label repository S with Ps; initialize Vo with L and diffusion model initializes generate data (a) corresponding to æ:;;
2:
Diffusion- Target model Rectification =
3: for t- to e/2 do
4: Get target sample æ; and the sample predictions y using Me: ys fo(co(z))).
Get generated fusion sample to fuse Fi,(s) z; using Eqs. (3)-(4);
6: Update the smooth label repository S using Eq. (1);
7: Get fusion sample predictions ii using Mo: Yj = (T:))
8:
Fine-tune Vg by minimizing Lv, with (o(),) using Eqs. (5)-(6) min max Lyz:
Io To
9:
Optimize Ma by minimizing (MTDDRR) with (2:) using Eq. (9): min max M
10:
end for
11:
Self-Rectifying Reasoning Model Initialize: Replace original text encoder the diffusion model with the fine-tuned text encoder with
prompt word embeddings;
13:
for 4- e/2 to do
14:
Get target sample and the sample predictions yu using MMg y fo(co(x:));
Get generated fusion sample 7, to fuse (i)(g) and 2, using Eqs. (3)-(4);
16:
Update the smooth label repository S using Eq. (1);
17:
Get fusion sample predictions Yi using Mg: Vi (T.)
18:
Assign different weights according to the similarities between and using Eq. (11);
19:
Optimize Mo by minimizing (SR(S with using Eq. (12): min max (SRRN))
f0
20: end for
Output: Target model MM.
Table 6: The complete quantitative results of ablation study on the Office-31 and VisDA-17. 17.
DM,
Office-31
VisDA-17
FSG
FT
Loc Loe LSvI
A-D
A-W
D
D
W-A
W-D
Mean
Mean
Source only
79.9
76.6
56.4
92.8
60.9
98.5
77.5
48.9
97.8
85.2
66.6
97.0
72.1
97.5
86.0
71.2
85.5
94.9
79.3
99.1
83.5
99.8
90.4
79.3
76.2
83.2
67.6
94.1
69.2
95.6
81.0
59.6
95.2
95.7
81.5
99.0
83.1
99.8
92.4
89.4
92.7
88.5
67.7
97.9
74.5
99.6
86.9
67.8
96.9
94.1
73.7
97.3
81.6
99.8
90.6
85.4
85.3
84.9
66.7
97.0
71.9
98.0
84.0
80.2
87.3
84.0
65.6
96.3
74.2
98.6
84.3
81.7
71.7
84.9
70.2
94.6
64.4
98.2
80.6
75.9
96.8
94.1
82.5
99.1
84.1
99.8
92.7
88.7
73.1
85.7
69.7
95.3
65.2
97.9
81.2
76.8
97.8
95.9
83.7
99.1
84.5
99.8
93.5
91.2
learning with the samples generated by the fine-tuned diffusion model. Lsvı becomes effective only when combined with fine-tuning. Experimental results show that this combination yields significant performance gains on large-scale dataset VisDA-17, while improvements small-scale dataset Office-31 are relatively limited. In summary, each component of our RrED contributes effectively to performance improvement and is indispensable.
26

---

Source-only: 44.1%
DINE 15 52.6%
BETA 16 57.0%
Diffusion
42 56.3%
RFC 18 57.5%
SEAL 17: 57.9%
AEM 19 63.7%
RrED: 82.3%
Figure 6: The feature visualization on the Office-Home (A-C) using the t-SNE 66 Herein, the points represent target samples and the different colors correspond their ground-truth classes RrED introduces diffusion into BUDA and fine-tunes it, ultimately achieving remarkable improvement.
F. Implementation Details.
We implement our RrED based on PyTorch and conduct all experiments using an NVIDIA GeForce RTX4090 GPU. For fair comparison, the backbone network is initialized following the protocol in [3], employing the ImageNet [64] pre-trained ResNet architectures: ResNet-50 for Office-31, Office- Home, and DomainNet, and ResNet-101 for VisDA-17. The optimization configuration employs SGD with a momentum of 0.9, a weight decay of Ie-3, and differentiated learning rates, where the learning rate is set to le-4 for the feature extractor fo and 1e-3 for the classifier co. Following [16 17..
set the bottleneck dimension to 256, the batch size to 64, the static momentum coefficient yu to 0.6, and the number of warm-up epochs to 3. To facilitate our joint multi mollil model CLIP 20 for fine-tuning, we choose Stable Diffusion v-1.5 [22] as the diffusion model. The strength of the noise addition is
0.6 in the diffusion model. For the diffusion predictor [42] and the fine-tuned text encoder we introduced, we keep their parameters frozen during the whole training. During the fine-tuning process, follow [52] 19 to the number of context tokens m 4. All the reported quantitative results are obtained by averaging multiple runs with seeds [2023, 2024, 2025].
G. More Visual Comparisons and Further Analysis
As shown in Figure 6, we use t-SNE 65 technique visualize the distribution of target samples in the feature space. Compared with previous methods, the discrimination ability of the target model for target samples with similar features has been significantly improved under the training of our RrED algorithm. Moreover, as can be clearly observed from the graph, due to the enhanced generalization ability of the model after being trained by RIED, the differences between different classes become more pronounced, and the distances between samples of the same class become more compact. Compared the previous method 42 that directly applies diffusion model for prediction, our RrED exhibits superior model generalization and class discrinination capabilities. Therefore, we conclude that the target model trained by RrED achieves significant performance improvement the high-security BUDA setting.
Next, we discuss our method's exploration of the diffusion model further demonstrate the superiority of our approach. As shown in Figure 7, we show the images that are generated by the diffusion model on the VisDA-17 under varying noise strengths. When the noise level is too low, the images generated by the diffusion model are too similar to the target domain images, providing limited benefit for enhancing the model's s reasoning ability. When the noise level is too high, the images generated by the diffusion model differ drastically from those in the target domain and may even contain unrelated objects. Directly using such images can irreversibly disrupt the discriminative ability of the target model. How can effectively utilize the diffusiin model to guide the target model in enhancing its reasoning ability while preventing its potential negative effects? This is the problem our work RiED aims solve. For this, FSG serves the key module to preclude the ceftusionnggenrrate
27

---

Ground-truth
Strength - 0.2
Strength 0.4
Strength = 0.6
Strengtt 0.8
Streagth =
FT
Acroplane
Bicycle
Motorcycle
Skateboard
LIEY
13
Truck
Figure 7: We present images generated by the diffusion model on the VisDA-17 under varying noise strengths, along with those produced when the noise strength is 0.6 after fine-tuning.
Table 7: Results of computational cost comparison on the VisDA-17 with the ResNet-101 backhone. The batch size is 64.
Method
|Space (MiB) Time (s/epoch) Accuracy (%)
DINE
18B
124s
75.6
BETA
20247MiB
1101s
85.1
SEAL
Over 24G
89,2
AEM
13747MiB
672s
89.3
RrED (Stage 1)
17654MiB
312s
89.4
RiED (Stage 2)
11721MiB
201s
91.2
images from bringing about irreversible adverse effects. FSG retains the regions of interest for the model, allowing the target model to maintain image discernibility even under higher noise levels in diffusion. Meanwhile, by fine-tuning the text encoder in the diffusion model, RrED enables to better understand the content to be generated while maintaining its generative capabilities. As shown in Figure 7, the images generated by fine-tuned diffusion model exhibit greater diversity, more distinct features, and fewer interfering objects. This allows the target model, in the second phase SRM, to first recognize the simpler generated images and then further distinguish the more challenging target images.
H. Computational Cost Comparison and Optimization Evolution
We supplement the computational cost comparison of the VisDA-17 (55 in Table 7. For a fair comparison, all the results are obtained in the same experimental environment. In Table 7, we document the maximum GPU space usage, the average runtime cost, and the best accuracy of each comparison method. When adapting the VisDA-17 dataset, it is worth noting that the comparison methods have consumption- stages: the first stage is the initialization, which requires initialization of the two models due to their mutually- disttilee network structures; the second stage is the two-step process, which requires distillation and fine-tuning for each epoch. SEAL [17 is highly resource-intensive,, and its official code cannot complete the adaptation task on VisDA-17 under the same conditions with 24GB GPU memory. During the training of AEM [19], two classifiers are required: one classifier processes the output of the target model, while the other aligns with the predictions of the ViL model. Moreover, in
limitations. BETA [16] operates in two computationally intensive
28

---

Source-only: 44.1%
Stage 1: 79.8%
Stage 2: 82.3%
Figure 8: The feature distribution evolutions of different stages on the Offfie--Home (A-CC) using t-SNE [65]. Herein, the points represent target samples and the different colors correspond to their ground-truth classes.
each iteration, the weights of the overall model and the classifier weights need to be updated separately, resulting in consuming a significant amount of time. Compared with the previous BUDA methods, although RrED introduced the diffusion model in stage 1 to guide the learning of the target model, still significantly reduced the time consumption by cutting out unnecessary calculation processes and optimizing loss functions. Moreover, in stage 2, after eliminating the resources consumed by fine-tuning and diffusion, RrED demonstrates extremely low overhead. These results demonstrate that RrED significantly outperforms other BUDA methods in enhancing class discrimination ability at relatively low cost.
In Figure 8, the optimization evolutions of feature distribution presented. The black-box predictor fails to effectively separate and cluster target sample features, with samples from different classes heavily entangled. This confusion introduces noisy signals during target model training, thereby hindering effective adaptation. After the first stage of training, the target model has learned the rich semantic knowledge in the diffusion model and significantly improved its class discrimination ability. After the second stage of training, the scattered data distribution boundaries stabilize around the nearest feature cluster centers, thus leading the samples with similar features exhibiting more compact behavior. These results demonstrate the superiority of the two-stage training in RrED and achieve the predefined objectives of each stage.
Broader Impacts and Limitations
Our work RrED focuses the problem of Black-box Unsupervised Domain Adaptation (BUDA), which provides better data privacy protection with more flexible portability compared with other Domain Adaptation (DA) settings. Meanwhile, RrED demonstrates extremely superior performance, significantly surpassing other DA methods. Inspired by research in neuroscience, RrED is specifically designed for the classification task. While its effectiveness has been demonstrated through extensive experiments and theoretical soundness established, its applicability to other tasks remains an open question. Therefore, plan to further explore the practical utility of this algorithm in broader range of task scenarios.
29


Reviews:
[{"review_id": "gt30ce6k31", "reviewer": "Reviewer_Wd9F", "summary": "This paper proposes RrED, a novel two-stage framework for black-box Unsupervised Domain Adaptation (BUDA), which, for the first time, integrates diffusion models as a semantic knowledge base to guide adaptation; the framework consists of Diffusion-Target model Rectification (DTR), which aligns the target model with diffusion-generated semantic features, and Self-Rectifying Reasoning Model (SRM), which enhances the target model's reasoning ability through self-corrective learning, collectively enabling effective reasoning error correction and achieving superior performance over existing state-of-the-art methods.", "questions": "See weaknesses", "limitations": "see weaknesses", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n    1:RrED is the first to apply the diffusion model to the highly secure BUDA task and innovatively fine-tuned the text encoder of the diffusion model. This resolves the issue that existing diffusion-based domain adaptation methods mainly focus on image encoders while neglecting the nuances of text.\n    2:RrED significantly outperforms previous state-of-the-art methods on all four benchmark datasets, demonstrating its effectiveness in enhancing model reasoning and generalization capabilities.\n\n\nWeakness:\n    1:It remains unclear how sensitive the proposed RrED framework is to the choice of diffusion backbone. For instance, would replacing the default diffusion model with alternatives such as Stable Diffusion or DALL·E yield comparable results, or introduce significant performance discrepancies? A systematic analysis of this dependency would strengthen the empirical foundation of the method and inform its generalizability.\n\n    2:RrED is tailored for the BUDA setting, but it remains unclear whether its core components or two-stage framework can generalize to other domain adaptation scenarios such as open-set, or partial adaptation. Exploring its applicability beyond BUDA would further demonstrate the method's versatility.\n\n    3：The paper lacks visualizations or quantitative analyses that assess the prediction quality of the black-box predictor.   In particular, it does not provide dedicated figures or experiments illustrating how varying levels of quality—such as low accuracy, biased predictions, or noisy outputs—affect the downstream performance of RrED.", "was_revised": true, "final_justification": "I read the replies and thanks for authors' hard work and detailed explanation, it's more clear now.", "rebuttal": "We sincerely appreciate the reviewer’s constructive suggestions and have answered them as follows with our highest respect.\n\nQ1: It remains unclear how sensitive the proposed RrED framework is to the choice of diffusion backbone. For instance, would replacing the default diffusion model with alternatives such as Stable Diffusion or DALL·E yield comparable results, or introduce significant performance discrepancies? A systematic analysis of this dependency would strengthen the empirical foundation of the method and inform its generalizability.\n\nA1: In our experiments, we have analyzed backbone sensitivity by conducting both qualitative and quantitative evaluations, where the default Stable Diffusion model was used in place of our fine-tuned version. In Figure 3, “w/o Fine-tuning” refers to using the default text encoder of the Stable Diffusion model without any fine-tuning, while “w/o FSG” indicates directly using the images generated by the default Stable Diffusion model without any further processing. As shown in Figure 3, we have performed qualitative and quantitative ablation evaluations on the VisDA-17 dataset to determine the sensitivity of each module to model improvements. The visualization also demonstrates that the model captures target features more accurately after improvements. Furthermore, based on Table 6, we supplement the results of using only the default Stable Diffusion backbone (w/o Fine-tuning and FSG) to further verify the system analysis of dependencies:\n\n| Office-31 | A→D | A→W | D→A | D→W | W→A | W→D | Mean | VisDA-17 (Mean) |\n| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| RrED w/o Fine-tuning | 96.8 | 94.1 | 82.5 | 99.1 | 84.1 | 99.8 | 92.7 | 88.7 |\n| RrED w/o FSG | 73.1 | 85.7 | 69.7 | 95.3 | 65.2 | 97.9 | 81.2 | 76.8 |\n| RrED w/o Fine-tuning and FSG | 71.7 | 84.9 | 70.2 | 94.6 | 64.4 | 98.2 | 80.6 | 75.9 |\n| RrED | 97.8 | 95.9 | 83.7 | 99.1 | 84.5 | 99.8 | 93.5 | 91.2 |\n\nIn the above table, we conducted a comparison in two different datasets Office-31 and VisDA-17. The results indicate that directly applying the default Stable Diffusion model, without adaptation to the downstream task, leads to a sharp performance drop. In contrast, our method exhibits highly task-aware sensitivity to the structural characteristics of the Stable Diffusion model, enabling it to better leverage its semantic knowledge for downstream BUDA tasks. We plan to include these analyses and experiments in the final version.\n\nQ2: RrED is tailored for the BUDA setting, but it remains unclear whether its core components or two-stage framework can generalize to other domain adaptation scenarios such as open-set, or partial adaptation. Exploring its applicability beyond BUDA would further demonstrate the method's versatility.\n\nA2: To explore the applicability of RrED in other tasks and further demonstrate its versatility, we conducted experiments in open-set black-box adaptation scenarios. We integrate our RrED into [1] to enable domain adaptation under the open-set black-box adaptation setting on Office-Home, and the results are as follows:\n\n| Office-Home | A→C | A→P | A→R | C→A | C→P | C→R | P→A | P→C | P→R | R→A | R→C | R→P | Mean |\n| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | \n| UB2DA | 60.9 | 69.6 | 76.3 | 74.4 | 69.2 | 76.5 | 74.5 | 60.3 | 76.2 | 74.1 | 62.0 | 71.1 | 70.4 |\n| ADU | 61.2 | 72.7 | 77.9 | 70.3 | 72.5 | 77.3 | 75.9 | 62.0 | 84.7 | 73.2 | 64.1 | 74.9 | 72.2 |\n| RrED | 69.1 | 81.6 | 84.3 | 77.9 | 78.2 | 83.1 | 81.8 | 67.3 | 87.6 | 79.6 | 70.9 | 80.4 | 78.5 |\n\nAs shown in the table above, even under a different setting, RrED continues to demonstrate strong performance and significantly outperforms open-set black-box adaptation SOTA methods [1, 2].\n\nQ3: The paper lacks visualizations or quantitative analyses that assess the prediction quality of the black-box predictor. In particular, it does not provide dedicated figures or experiments illustrating how varying levels of quality—such as low accuracy, biased predictions, or noisy outputs—affect the downstream performance of RrED.\n\nA3: In our paper, we have demonstrated through extensive quantitative and qualitative experiments that the black-box predictor has inherent limitations in prediction quality. In Tables 2-4 and Figure 3, \"Source-only\" refers to using the black-box predictor to evaluate the predicted target samples. In addition, we visualize the features of the black-box predictor using t-SNE in Figure 8. To enhance understanding, we provide the following additional explanations: As shown in Figure 8, the black-box predictor fails to effectively separate and cluster target sample features, with samples from different classes heavily entangled. This confusion introduces noisy signals during target model training, thereby hindering effective adaptation. We plan to include these explanations in the final version.\n\nWe appreciate the reviewer’s feedback and hope our response has addressed the reviewer’s concerns.\n\n[1] On universal black-box domain adaptation. arXiv, 2021. \n\n[2] ADU: Adaptive Detection of Unknown Categories in Black-Box Domain Adaptation. In CVPR, 2025."}, {"review_id": "Rj0qosuApJ", "reviewer": "Reviewer_PJJd", "summary": "This work introduces a novel diffusion-based method called Reifying-reasoning Errors of Diffusion (RrED) for Black-box Unsupervised Domain Adaptation (BUDA). RrED enhances target models by decomposing text and visual encoders, using a feature-sensitive module for visual enhancement and multi-modal joint fine-tuning. It prioritizes BUDA tasks and leverages differential reasoning to rectify errors, showing improved performance in reasoning and generalization across four benchmark datasets. The proposed method is relatively cumbersome and complex, but has achieved good performance.", "questions": "1) Please expain what is the mean of lines 177-182.  Meanwhile, it is evident that the classifier p of the diffusion model heavily relies on the pre-trained model on which dataset, therefore it is necessary to introduce the pre-trained diffusion model.\n\n2) Please explain Eq (6). Why can category related areas be selected.\n\n3) From my understanding, the first step is essentially using the diffusion model to update the multi model and local model. So, how can we ensure that the diffusion model is correct in this area, as the target domain samples may not have participated in training the diffusion model.\n\n4) This work is designed with many loss functions, and the number of parameters discussed is far less than the actual number of loss functions. Although the authors fixed some parameters to 1, they still need to be discussed. \n\n5) Please deeply explain Eq.(13). It is obvious that these two optimization items are conflicting.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1)  The proposed FSG block evaluates feature-sensitive regions by comparing global and local image regions, enhancing generalization without losing discriminative ability.\n\n2) The first to use diffusion model for black box domain adaptation problem.  SOTA performance.\n\nWeaknesses: \n1) Writing needs further improvement. For example, Figure 1 is too complex to express the core idea. Figure 2 is also too complex. At the same time, the diffusion model should not be introduced too much, and the contribution should be written in the main text.\n\n2) Introducing the diffusion model to assist in black box adaptation also brings in more knowledge. So in essence, there are unfairness in comparing with existing methods.", "was_revised": true, "final_justification": "I am generally satisfied with the responses, so I plan to maintain the previous score.", "rebuttal": "We sincerely appreciate the reviewer’s constructive suggestions and have answered them as follows with our highest respect.\n\nQ1: Writing needs further improvement. For example, Figure 1 is too complex to express the core idea. Figure 2 is also too complex. At the same time, the diffusion model should not be introduced too much, and the contribution should be written in the main text.\n\nA1: We sincerely thank the reviewer for the valuable feedback and will continue to improve our writing. To improve the clarity and structure of the paper, we have simplified Figures 1 and 2, moved the contribution summary into the main text, and relocated the diffusion process to the Appendix.\n\nQ2: Introducing the diffusion model to assist in black box adaptation also brings in more knowledge. So in essence, there are unfairness in comparing with existing methods.\n\nA2: Our RrED is the first work that applies the diffusion model to high-security BUDA tasks innovatively. Since no prior work shares exactly the same experimental configuration, a direct and perfectly fair comparison is not achievable. Accordingly, we perform comparisons with representative methods under both high- and low-challenge settings to indirectly demonstrate the fairness of our evaluation. Please refer to Table 1, compared to Diffusion-based DA, which has access to both source and target domains, our RrED offers stronger privacy protection by operating without any access to source knowledge. Compared to Black-box DA, which does not utilize diffusion models, RrED leverages knowledge from diffusion models to achieve significant performance improvements. Whether compared with the less challenging Diffusion-based DA or the more challenging Black-box DA, RrED has achieved significant improvements. In particular, compared to the less challenging Diffusion-based DA method DAD, RrED still achieved a maximum improvement of 13.1% on Office-Home without being able to obtain source knowledge. Therefore, our experiment achieved a fair comparison.\n\nQ3: Please explain what is the mean of lines 177-182. Meanwhile, it is evident that the classifier p of the diffusion model heavily relies on the pre-trained model on which dataset, therefore it is necessary to introduce the pre-trained diffusion model.\n\nA3: We provide detailed explanations for lines 177-182 from the perspectives of motivation, process, and result. \n\nFor the motivation of lines 177-182, as shown in Figure 7, when the noise level is too low, the images generated by the diffusion model are too similar to the target domain images, providing limited benefit for enhancing the model's reasoning ability. When the noise level is too high, the images generated by the diffusion model differ drastically from those in the target domain and may even contain unrelated objects. This leads us to a key question: how can we effectively utilize the diffusion model to guide the target model in enhancing its reasoning ability while preventing its potential negative effects? These motivations are described in lines 693-699.\n\nFor the process of lines 177-182, in our analysis of the Stable Diffusion model, we observed that it employs a CLIP-based enhanced text encoder to interpret textual inputs. Building upon this, we extracted the image encoder from CLIP and combined it with the enhanced text encoder from Stable Diffusion to construct a new text-image model. Inspired by recent advances in prompt learning (PL), as shown in Eq. (8), we insert learnable continuous vectors before the text encoder. During the first training stage, only these vectors are updated, while all other parameters in the new text-image model remain frozen. Since the learnable vectors are not yet well-optimized in the first stage and the Stable Diffusion model is not frequently invoked, we delay integrating the modified text encoder. Then, we replace the original text encoder in the Stable Diffusion model with the one incorporating the learnable vectors at the beginning of the second stage.\n\nFor the result of lines 177-182, as shown in Figure 7 and Table 7, this design significantly reduces computational overhead during training while enabling the diffusion model to better comprehend textual inputs, thereby generating higher-quality images and ultimately achieving substantial performance improvements.\n\nFor “it is evident that the classifier p of the diffusion model heavily relies on the pre-trained model on which dataset, therefore it is necessary to introduce the pre-trained diffusion model”, as shown in Figure 6, the classifier $p$ within the diffusion model contains richer semantic information compared to the black-box predictor. However, directly fine-tuning the diffusion model’s parameters is computationally expensive for our BUDA downstream tasks. Therefore, we uniformly adopt the default Stable Diffusion model as the diffusion module without additional pretraining, and employ a fixed diffusion classifier only during the first stage to provide guidance. As shown in Figure 8, by the end of the first stage, the target model already outperforms the diffusion classifier, making further use of it unnecessary. We then proceed to the second stage, where the target model is optimized independently. Our design achieves remarkable performance without any pretraining of the diffusion model on either the target or source domain samples. Due to the high computational cost of dataset-specific pretraining, we are currently unable to perform such pretraining for the Stable Diffusion model. We plan to further explore this direction once sufficient computational resources become available.\n\nQ4: Please explain Eq (6). Why can category related areas be selected.\n\nA4: In Eq. (6), FSG evaluates the feature-sensitive regions of the model by evaluating the global image and comparing it with each local region, leveraging the local regions and their adjacent contextual information. The category related areas are determined by the target model during training, and this selection process is obtained through evaluation calculation. As shown in Figure 3, the selection process is dynamic. The model evaluates local regions to identify feature-sensitive areas, and then leverages the features from these regions to determine class-relevant information.\n\nQ5: From my understanding, the first step is essentially using the diffusion model to update the multi model and local model. So, how can we ensure that the diffusion model is correct in this area, as the target domain samples may not have participated in training the diffusion model.\n\nA5: Due to limited computational resources, we are unable to fully train the diffusion model on the target domain, and thus cannot ensure the accuracy of its classification outputs. As shown in Figure 6, the diffusion model achieves a classification accuracy of 56.3%, higher than the black-box predictor but still far from sufficient for reliable BUDA tasks. To address this limitation, we reduce the noise interference of the diffusion model from two directions while ensuring that the diffusion model correctly guides the learning of the target model: (1) Our framework is designed in two stages: In the first stage, we leverage the rich semantic knowledge embedded in a fixed diffusion model to guide the initial learning of the target model. In the second stage, we discard the diffusion model’s predictions and instead use high-quality images generated by the diffusion model with a fine-tuned text encoder to further train the target model. (2) We propose Feature-Sensitive Generation, which integrates diffusion outputs through image fusion to serve as a buffer mechanism. This helps the model focus on class-discriminative regions more effectively.\n\nQ6: This work is designed with many loss functions, and the number of parameters discussed is far less than the actual number of loss functions. Although the authors fixed some parameters to 1, they still need to be discussed.\n\nA6: We sincerely thank the reviewer for the valuable feedback. As shown in lines 594-595, “Following previous BUDA works, as presented in Eq. (2), $\\lambda$ is set to 1 in the BUDA task”, Eq. (2) is a task-specific loss. To verify its effect on our method, we supplemented the following experiments on the VisDA-17 dataset:\n\n| $\\lambda$=0.2 | $\\lambda$=0.5 | $\\lambda$=1 | $\\lambda$=1.5 | $\\lambda$=2 | $\\lambda$=3 | \n| --- | --- | --- | --- | --- | --- |\n| 88.4 | 89.9 | 91.2 | 91.4 | 90.9 | 90.8 |\n\nExperimental results show that appropriately tuning the weight of the task-specific loss can improve model performance to some extent. \n\nQ7: Please deeply explain Eq.(13). It is obvious that these two optimization items are conflicting.\n\nA7: In Eq. (13), our proposed interactive optimization learning can be divided into two optimization items. Two items form a contrastive learning, the first term aims to correct model reasoning errors by aligning and clustering features based on the similarity between source and generated samples. Meanwhile, the second term enhances the feature discrepancies among samples to prevent overfitting caused by overly rapid alignment from the first term. The design principle of using one term for task execution and another to regulate its learning pace is common in contrastive learning, and our implementation in Eq. (2) presents an innovative extension of this principle.\n\nQ8:  Also, what is Lemma 1 in Appendix doing? I'm not quite sure, a more logical organization for this part is needed.\n\nA8: In our paper, we use Lemma 1 to demonstrate the rationality of the BUDA setting, clarify the common principles underlying BUDA methods, and explain why our approach and other BUDA methods are applicable in this context. Lemma 1 serves Eq. (18)-(21). As shown in lines 597-598, our proof fills the theoretical knowledge gap regarding the black-box predictor in previous BUDA works.\n\nWe appreciate the reviewer’s feedback and hope our response has addressed the reviewer’s concerns."}, {"review_id": "nJgeBLhuqp", "reviewer": "Reviewer_mM7d", "summary": "This paper aims to adapt a model in the setting of Black-box Unsupervised Domain Adaptation (BUDA), which doesn't allow to access source data and source model. It proposes a two-stage framework leveraging diffusion models as a \"knowledge base\" to enhance target model adaptation. In the first stage, it designed feature sensitive generation to fuse new samples and finetune the text encoder of diffusion model to generate more stable target samples. Then in the second stage, it contrasts predictions of original vs. diffusion-augmented samples to refine decision boundaries. Overall, the framework adapt a model achieve better classification results in several benchmarks.", "questions": "Just like items said in Weaknesses:\n1. How to effectively manage so many modules to make it stable in actual scenarios?\n2. Supplement analysis and intuitive examples for the immediate and innovative designs.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Clear and important problem setting. The Black-box Unsupervised Domain Adaptation pinpoint the significance of privact preservation. Later it might be very useful in the widespread usage of closed-source large models.\n2. Creative data-in-the-loop pipeline design. With the proposed fusion of samples and finetuning diffusion model, it integrates data manipulation into the adaptation process.\n3. Abundant empitical and theoretical analysis. It provides classification results of multiple benchmarks and proofs for error bound.\n\nWeaknesses:\n1. The organization and description are hard to follow. There are too many abbreviations for the problem setting, the method and each module or blocks, RrED, BUDA, FSG, DTR, SVI, .etc. It is hard to build connections among them. Besides, in section 3.2, the notations are also very intricate, which leads to barrier for understanding.\n2. Tanglesome design of many modules in the whole pipeline. FIgure 2 contains too many blocks, which is hard to filter key steps. Integration of diffusion model, multi-modal model, and at least 2 contrastive related losses, extremely increase the comlexity of the whole system.\n3. Hyperparameter Sensitivity. As the last point said, a lot of losses in the two stages meet challenges of balance. Just like Figure 4 shows, the same hyperparameter r leads to a large difference of performance across different datasets. It hinders further application in actual scenarios.\n4. Lack of quantitative and qualitative analysis of key innovative designs. After hard reading, it can find that the Feature Sensitive Generation, Finetuning diffusion models, and Sample-version Interaction are key innovative modules. Besides final classification performance, we don't know what intuitive and direct result these designs lead to. For example, what do the fused examples after Feature Sensitive Generation really look like? Although Figure 7 gives some samples generated by diffusion model, it is hard to know the difference before and after finetuning.", "was_revised": true, "final_justification": "After rebuttal and further response from authors, I decided to increase my rating from 3 to 4. The main reasons are:\n1. The task setting of BUDA is interesting and might have important application in widely used close-source large models.\n2. After rebuttal, although it is not perfect, most of my concerns about the whole pipeline and hyperparameter selection have been resolved.\n3. For the claim of connection between the method and visual reasoning, it is still not convincing. But the empirical results fundamentally verify the effectiveness.\nComprehensively, incorporating all the positive and negetive factors, I made such a decision.", "rebuttal": "We sincerely appreciate the reviewer’s constructive suggestions and have answered them as follows with our highest respect.\n\nQ1: The organization and description are hard to follow. There are too many abbreviations for the problem setting, the method and each module or blocks, RrED, BUDA, FSG, DTR, SVI, .etc. It is hard to build connections among them. Besides, in section 3.2, the notations are also very intricate, which leads to barrier for understanding.\n\nA1: Although our paper includes a number of abbreviations, each has been clearly defined and thoroughly introduced. For all equations, particularly those in Section 3.2, we have provided detailed explanations of their processes and their connections to our proposed method. Furthermore, all notations have been clearly interpreted following common practices in the machine learning community. To ensure the content is clear and easy to understand, we have carefully reviewed the content multiple times to avoid any omissions in the explanations of the final version.\n\nQ2: Tanglesome design of many modules in the whole pipeline. FIgure 2 contains too many blocks, which is hard to filter key steps. Integration of diffusion model, multi-modal model, and at least 2 contrastive related losses, extremely increase the comlexity of the whole system.\n\nA2: Although our method appears relatively complex, each step is specifically designed to address limitations inherent to the current task and is essential to the overall framework. We have modularized key components for better clarity and usability. The method can be divided into two core components: fine-tuning and integrating the diffusion model’s text encoder, and the two-stage learning pipeline for the target model. In the first stage, the diffusion model generates initial images and guides the target model using its classifier, while the text encoder is fine-tuned. In the second stage, the fine-tuned text encoder is reintegrated into the diffusion model to regenerate images for contrastive learning by the target model. We encapsulate the diffusion model operations into two functions, each corresponding to a learning stage, enabling high reusability and portability. As shown in Figure 2, every step of the process is clearly illustrated. To further improve clarity and structure, we have simplified Figure 2 by removing common operations in BUDA and optimizing the overall layout.\n\nQ3: Hyperparameter Sensitivity. As the last point said, a lot of losses in the two stages meet challenges of balance. Just like Figure 4 shows, the same hyperparameter r leads to a large difference of performance across different datasets. It hinders further application in actual scenarios.\n\nA3: For “As the last point said, a lot of losses in the two stages meet challenges of balance”, our method adopts a two-stage structure where, in addition to the task-specific loss, the first stage involves two losses and the second stage only one. In the first stage, $L_{CC}$ is a standard cross-entropy loss, while $L_{CC}$ is introduced to counteract the potential negative impact of cross-entropy minimization on generalization, with the hyperparameter $\\gamma$ controlling the contribution of $L_{CC}$. In the second stage, $L_{SVI}$ is a contrastive loss that achieves self-balancing by evaluating sample similarity through the similarity weight $w$. Therefore, the loss functions introduced in each stage are not overly complex and are carefully designed to ensure training stability and balance.\n\nFor “the same hyperparameter r leads to a large difference of performance across different datasets“, various factors influence performance across different datasets. In Figure 7, we compare VisDA-17 and Office-Home (A→C), where the number of target domain samples differs significantly: VisDA-17 contains around 55K samples, while Office-Home has only about 4K. As such, a large performance gap between the two is expected. In addition, the hyperparameter $r$, which determines the ratio between regions of interest and non-interest, plays a critical role in the effectiveness of our FSG module. Detailed discussion is provided in lines 330–334. Across different datasets, the optimal result is achieved when $r$ is around 100%, and this range has been verified in Figure 7. Our method involves only two key hyperparameters, $r$ and $\\gamma$, both of which have been thoroughly analyzed. To compare with existing methods, we have counted the number of hyperparameters that critically affect the performance of each method:\n\n| Method | BETA [1] | SEAL [2] | RFC [3] | ADU [4] | AEM [5] | RrED |\n| ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| Key hyperparameters | 4 | 2 | 2 | 3 | 3 | 2 |\n\nAccording to the statistics of key parameters, our method requires relatively fewer key hyperparameters, which makes it more adaptable to practical scenarios without extensive hyperparameter tuning. Furthermore, as shown in Table 7, our method improves performance while reducing time consumption compared to prior works, thus offering a more cost-effective solution and demonstrating greater suitability for practical applications:\n\n| Method | Space(MiB) | Time(s/epoch) | Accuracy(%) |\n| --- | --- | --- | --- | \n| DINE | 9881MiB | 124s | 75.6 |\n| BETA | 20247MiB | 1101s | 85.1 |\n| SEAL | Over 24G | - | 89.2 |\n| AEM | 13747MiB | 672s | 89.3 |\n| RrED (Stage 1) | 17654MiB | 312s | 89.4 |\n| RrED (Stage 2) | 11721MiB | 201s | 91.2 |\n\nQ4: Lack of quantitative and qualitative analysis of key innovative designs. After hard reading, it can find that the Feature Sensitive Generation, Finetuning diffusion models, and Sample-version Interaction are key innovative modules. Besides final classification performance, we don't know what intuitive and direct result these designs lead to. For example, what do the fused examples after Feature Sensitive Generation really look like? Although Figure 7 gives some samples generated by diffusion model, it is hard to know the difference before and after finetuning.\n\nA4: Please refer to Figure 3, Figure 7, and Table 6, We have conducted detailed quantitative and qualitative analyses on each innovation module. Figure 3 provides an intuitive visualization of the qualitative and quantitative impact of each module using Grad-CAM technique. Table 6 presents detailed quantitative ablation results, while Figure 7 illustrates the quality differences in images generated by the diffusion model before (Strength = 0.6) and after fine-tuning (FT). For the Feature Sensitive Generation (FSG) by reviewers, “w/o FSG” represents the impact on the model after the removal of FSG. In addition, considering the reviewer’s interest in the FSG, we plan to provide more detailed visualizations of images before and after FSG processing in the final version, to more clearly illustrate the contribution of this module.\n\nQ5: How to effectively manage so many modules to make it stable in actual scenarios?\n\nA5: Please refer to A2, we have described how to effectively manage our modules, while we have encapsulated the entire method into two functions, each corresponding to one learning stage, making it easier to deploy and adapt in actual scenarios. Please refer to A3, our method combines self-adaptation with minimal hyperparameter tuning, requiring only two hyperparameters to achieve stable performance in actual scenarios. Compared to previous BUDA methods, our method is relatively more transferable.\n\nQ6: Supplement analysis and intuitive examples for the immediate and innovative designs.\n\nA6: Please refer to A4, we sincerely thank the reviewer for the valuable feedback. To more clearly illustrate the contribution of this module, we plan to provide more detailed visualizations of images before and after FSG processing and more analysis and intuitive examples in the final version.\n\nWe appreciate the reviewer’s feedback and hope our response has addressed the reviewer’s concerns.\n\n[1] Divide to adapt: Mitigating confirmation bias for domain adaptation of black-box predictors. In ICLR, 2023.\n\n[2] A Separation and Alignment Framework for Black-Box Domain Adaptation. In AAAI, 2024.\n\n[3] Reviewing the forgotten classes for domain adaptation of black-box predictors. In AAAI, 2024.\n\n[4] ADU: Adaptive Detection of Unknown Categories in Black-Box Domain Adaptation. In CVPR, 2025.\n\n[5] Adversarial Experts Model for Black-box Domain Adaptation. In ACMMM, 2024."}, {"review_id": "gI5Q5uFhak", "reviewer": "Reviewer_H3mL", "summary": "This paper addresses the Black-box Unsupervised Domain Adaptation (BUDA) problem by introducing a novel diffusion-based framework, RrED (Rectifying-reasoning Errors of Diffusion). The method is structured in two stages: (1) Diffusion-Target model Rectification (DTR) and (2) Self-rectifying Reasoning Model (SRM). The first stage decouples and fine-tunes the diffusion model’s text encoder with a feature-sensitive generation (FSG) module, while the second stage uses the model’s prediction discrepancy across generated variations to refine its reasoning ability. Extensive experiments across four standard domain adaptation benchmarks show that RrED outperforms state-of-the-art (SOTA) BUDA methods.", "questions": "Please see the weakness above. Overall, this work presents a technically complex framework for black-box unsupervised domain adaptation, but its main weakness lies in the lack of clear motivation. The paper does not sufficiently justify the choice of diffusion models or explain the conceptual rationale behind the proposed design. In addition, the presentation should be improved to clearly convey what specific problem is being addressed in BUDA and how the proposed method aims to solve it.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strength**\n\n1. Employing diffusion model for tackling BUDA task is novel.\n\n2. The authors perform thorough evaluations on four well-established datasets (Office-31, Office-Home, VisDA-17, DomainNet), showing consistent improvements.\n\n3. Comprehensive ablation studies are conducted to validate the contributions of each component. The inclusion of qualitative visualizations (e.g., Grad-CAM) is helpful.\n\n**Weakness**\n\n1. This work presents a complex framework but lacks important and necessary analysis of the underlying intuition:\n\n(1) Insufficient Justification for Diffusion Models in BUDA: The paper lacks a detailed explanation of the motivation for using diffusion models to address the black-box unsupervised domain adaptation (BUDA) task. Specifically, it does not clearly explain why diffusion models are suitable for BUDA, what particular challenges in BUDA they aim to address, and in what aspects diffusion models are advantageous compared to other multimodal models such as CLIP.\n\n(2) Weak Theoretical Connection to Human Reasoning: The paper claims that the proposed method is inspired by the human decision-making process, specifically the dual-process theory involving System 1 and System 2. However, the connection between the method and this cognitive framework remains vague. The authors should provide more discussion on how the proposed components correspond to these cognitive systems. In particular, it is unclear what the term ‘reasoning’ specifically refers to in the context of BUDA. Does it involve rectifying predicted labels for target data?\n\n\n2. Lack of Qualitative Reasoning Examples: The model is claimed to enhance \"reasoning\" yet no explicit qualitative examples or visualizations are provided to show what these reasoning steps are. Without this, the claim that the model improves reasoning over past BUDA methods is difficult to assess.", "was_revised": true, "final_justification": "The authors’ response has addressed my concern, and I also hope the authors will take the reviewers’ comments into account during the revision. After considering the other reviewers’ comments, I will give a borderline accept.", "rebuttal": "We sincerely appreciate the reviewer’s constructive suggestions and have answered them as follows with our highest respect.\n\nQ1: Insufficient Justification for Diffusion Models in BUDA: The paper lacks a detailed explanation of the motivation for using diffusion models to address the black-box unsupervised domain adaptation (BUDA) task. Specifically, it does not clearly explain why diffusion models are suitable for BUDA, what particular challenges in BUDA they aim to address, and in what aspects diffusion models are advantageous compared to other multimodal models such as CLIP.\n\nA1: We have provided a detailed explanation of the motivation for using diffusion models to address the BUDA task. In Table 1 and lines 65-67, our motivation is to address the limitations of current diffusion-based UDA methods in terms of privacy protection and computational overhead. While diffusion models contain rich semantic knowledge that can guide BUDA learning, prior diffusion-UDA approaches focus solely on the image encoder, neglecting the role of the text encoder, thus failing to provide effective supervision for BUDA. Please refer to lines 78-91, our RrED is the first work that applies the diffusion model to BUDA tasks, which is a pioneering attempt. Please refer to lines 691-708, we have explained the limitations of directly applying diffusion models in BUDA tasks, and further clarified why our proposed method can achieve effective adaptation and performance improvement under this setting. Compared to other multimodal models, diffusion models offer the unique advantage of generating stable images. In addition, we qualitatively illustrate the differences between our approach and the CLIP-based method AEM [1] in Figure 3. In 562-567, We have provided a detailed explanation, from a class-level perspective, of why our diffusion model-based method holds advantages over other multimodal approaches. Furthermore, in lines 719–728, we have provided a detailed quantitative analysis demonstrating our advantages over AEM. Therefore, we have clearly explained the motivation for employing diffusion models in the BUDA setting and provided sufficient experimental evidence to validate its effectiveness and rationality.\n\nQ2:  Weak Theoretical Connection to Human Reasoning: The paper claims that the proposed method is inspired by the human decision-making process, specifically the dual-process theory involving System 1 and System 2. However, the connection between the method and this cognitive framework remains vague. The authors should provide more discussion on how the proposed components correspond to these cognitive systems. In particular, it is unclear what the term ‘reasoning’ specifically refers to in the context of BUDA. Does it involve rectifying predicted labels for target data?\n\nA2: As shown in 68-77 and Figure 1, we have provided a detailed description of the connection between our method and the human decision-making system. To address the reviewers' concerns, we provide more discussion on how our method corresponds to the cognitive system. For the term “reasoning”, in human decision-making systems, reasoning ability refers to the capacity of individuals to make further judgments about target objects by leveraging prior knowledge and logical analysis based on partial observations when confronted with complex information and dynamic environments. Similarly, in computer vision, the model’s reasoning process mirrors human decision-making by utilizing existing knowledge and feature similarity computations to determine whether targets in complex scenes meet the task requirements. The reasoning-related knowledge is relatively common. In neuroscience, human decision-making involves two stages: Stage 1 generates intuitive but sometimes biased responses unconsciously, while Stage 2 uses domain knowledge for careful, accurate reasoning. Recent studies add knowledge bases to guide this process and reduce reasoning errors. Our RrED models this by first aligning diffusion model decision boundaries in Stage 1, then correcting errors between versions in Stage 2 to improve self-reasoning. Meanwhile, inspired by previous approaches [3, 4] that enhance decision-making and reasoning, RrED introduce the diffusion model as a knowledge base. By leveraging its rich semantic representations, we guide the learning of the target model to identify and correct potential errors in its predictions. To improve the clarity and readability of the paper, we plan to supplement this knowledge in the Appendix.\n\nQ3: Lack of Qualitative Reasoning Examples: The model is claimed to enhance \"reasoning\" yet no explicit qualitative examples or visualizations are provided to show what these reasoning steps are. Without this, the claim that the model improves reasoning over past BUDA methods is difficult to assess.\n\nA3: In the field of computer vision, GAM-Grad [2] is a well-known technique for interpreting the reasoning behind model predictions through visualization. Leveraging its capabilities, we employ GAM-Grad in Figure 3 to highlight the vital and irreplaceable roles each component of our algorithm plays in the overall performance, with detailed explanations provided in lines 318-323. Furthermore, in Figure 3, we compare our approach with the previous SOTA method AEM, demonstrating that our model exhibits stronger reasoning ability, superior target discrimination, and more precise capture of feature details in target samples. To help reviewers more intuitively understand how our method improves the model’s reasoning ability during execution, we plan to include the initial and post-stage 1 Grad-CAM visualizations in the final version.\n\nWe appreciate the reviewer’s feedback and hope our response has addressed the reviewer’s concerns.\n\n[1] Adversarial Experts Model for Black-box Domain Adaptation. In ACMMM, 2024.\n\n[2] Grad-cam: Visual explanations from deep networks via gradient-based localization. In CVPR, 2017.\n\n[3] Efficient rectification of neuro-symbolic reasoning inconsistencies by abductive reflection. In AAAI, 2025.\n\n[4] Abductive learning with ground knowledge base, In IJCAI, 2021."}]

Author rebuttal:

Meta-review: This paper has received consistent positive ratings of 4, 4, 5, and 5. The reviewers raised concerns regarding backbone sensitivity, the specificity of the proposed design, fairness in comparisons, and issues with writing quality. The authors have addressed all of these points in their rebuttal, and the reviewers expressed satisfaction with the responses. While the writing issues can certainly be improved in the revised version, they currently hinder readability and limit the potential for a higher score. Given the consistent positive evaluations and the reviewers’ satisfaction with the rebuttal, I see no compelling reason to overturn their acceptance recommendations. Therefore, I recommend acceptance.
Decision: Accept (poster)