Title: RrED: Black-box Unsupervised Domain Adaptation via Rectifying-reasoning Errors of Diffusion
Venue: NeurIPS 2025
Accepted: yes
Keywords: Black-box Unsupervised Domain Adaptation; Neuroscience; Diffusion Model; Prompt Learning
Abstract: Black-box Unsupervised Domain Adaptation (BUDA) aims to transfer source domain knowledge to an unlabeled target domain, without accessing the source data or trained source model. Recent diffusion models have significantly advanced the ability to generate images from texts. While they can produce realistic visuals across diverse prompts and demonstrate impressive compositional generalization, these diffusion-based domain adaptation methods focus solely on composition, overlooking their sensitivity to textual nuances. In this work, we propose a novel diffusion-based method, called Rectifying-reasoning Errors of Diffusion (RrED) for BUDA. RrED is a two-stage learning strategy under diffusion supervision to effectively enhance the target model via the decomposed text and visual encoders from the diffusion model. Specifically, RrED consists of two stages: Diffusion-Target model Rectification (DTR) and Self-rectifying Reasoning Model (SRM). In DTR, we decouple the image and text encoders within the diffusion model: the visual encoder integrates our proposed feature-sensitive module to generate inferentially-enhanced visuals, while the text encoder enables multi-modal joint fine-tuning. In SRM, we prioritize the BUDA task itself, leveraging the target model's differential reasoning capability to rectify errors during learning. Extensive experiments confirm that RrED significantly outperforms other methods on four benchmark datasets, demonstrating its effectiveness in enhancing reasoning and generalization abilities.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the Black-box Unsupervised Domain Adaptation (BUDA) problem by introducing a novel diffusion-based framework, RrED (Rectifying-reasoning Errors of Diffusion). The method is structured in two stages: (1) Diffusion-Target model Rectification (DTR) and (2) Self-rectifying Reasoning Model (SRM). The first stage decouples and fine-tunes the diffusion model’s text encoder with a feature-sensitive generation (FSG) module, while the second stage uses the model’s prediction discrepancy across generated variations to refine its reasoning ability. Extensive experiments across four standard domain adaptation benchmarks show that RrED outperforms state-of-the-art (SOTA) BUDA methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n1. Employing diffusion model for tackling BUDA task is novel.\n\n2. The authors perform thorough evaluations on four well-established datasets (Office-31, Office-Home, VisDA-17, DomainNet), showing consistent improvements.\n\n3. Comprehensive ablation studies are conducted to validate the contributions of each component. The inclusion of qualitative visualizations (e.g., Grad-CAM) is helpful.\n\n**Weakness**\n\n1. This work presents a complex framework but lacks important and necessary analysis of the underlying intuition:\n\n(1) Insufficient Justification for Diffusion Models in BUDA: The paper lacks a detailed explanation of the motivation for using diffusion models to address the black-box unsupervised domain adaptation (BUDA) task. Specifically, it does not clearly explain why diffusion models are suitable for BUDA, what particular challenges in BUDA they aim to address, and in what aspects diffusion models are advantageous compared to other multimodal models such as CLIP.\n\n(2) Weak Theoretical Connection to Human Reasoning: The paper claims that the proposed method is inspired by the human decision-making process, specifically the dual-process theory involving System 1 and System 2. However, the connection between the method and this cognitive framework remains vague. The authors should provide more discussion on how the proposed components correspond to these cognitive systems. In particular, it is unclear what the term ‘reasoning’ specifically refers to in the context of BUDA. Does it involve rectifying predicted labels for target data?\n\n\n2. Lack of Qualitative Reasoning Examples: The model is claimed to enhance \"reasoning\" yet no explicit qualitative examples or visualizations are provided to show what these reasoning steps are. Without this, the claim that the model improves reasoning over past BUDA methods is difficult to assess.\n\nQUESTIONS:\nPlease see the weakness above. Overall, this work presents a technically complex framework for black-box unsupervised domain adaptation, but its main weakness lies in the lack of clear motivation. The paper does not sufficiently justify the choice of diffusion models or explain the conceptual rationale behind the proposed design. In addition, the presentation should be improved to clearly convey what specific problem is being addressed in BUDA and how the proposed method aims to solve it.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to adapt a model in the setting of Black-box Unsupervised Domain Adaptation (BUDA), which doesn't allow to access source data and source model. It proposes a two-stage framework leveraging diffusion models as a \"knowledge base\" to enhance target model adaptation. In the first stage, it designed feature sensitive generation to fuse new samples and finetune the text encoder of diffusion model to generate more stable target samples. Then in the second stage, it contrasts predictions of original vs. diffusion-augmented samples to refine decision boundaries. Overall, the framework adapt a model achieve better classification results in several benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Clear and important problem setting. The Black-box Unsupervised Domain Adaptation pinpoint the significance of privact preservation. Later it might be very useful in the widespread usage of closed-source large models.\n2. Creative data-in-the-loop pipeline design. With the proposed fusion of samples and finetuning diffusion model, it integrates data manipulation into the adaptation process.\n3. Abundant empitical and theoretical analysis. It provides classification results of multiple benchmarks and proofs for error bound.\n\nWeaknesses:\n1. The organization and description are hard to follow. There are too many abbreviations for the problem setting, the method and each module or blocks, RrED, BUDA, FSG, DTR, SVI, .etc. It is hard to build connections among them. Besides, in section 3.2, the notations are also very intricate, which leads to barrier for understanding.\n2. Tanglesome design of many modules in the whole pipeline. FIgure 2 contains too many blocks, which is hard to filter key steps. Integration of diffusion model, multi-modal model, and at least 2 contrastive related losses, extremely increase the comlexity of the whole system.\n3. Hyperparameter Sensitivity. As the last point said, a lot of losses in the two stages meet challenges of balance. Just like Figure 4 shows, the same hyperparameter r leads to a large difference of performance across different datasets. It hinders further application in actual scenarios.\n4. Lack of quantitative and qualitative analysis of key innovative designs. After hard reading, it can find that the Feature Sensitive Generation, Finetuning diffusion models, and Sample-version Interaction are key innovative modules. Besides final classification performance, we don't know what intuitive and direct result these designs lead to. For example, what do the fused examples after Feature Sensitive Generation really look like? Although Figure 7 gives some samples generated by diffusion model, it is hard to know the difference before and after finetuning.\n\nQUESTIONS:\nJust like items said in Weaknesses:\n1. How to effectively manage so many modules to make it stable in actual scenarios?\n2. Supplement analysis and intuitive examples for the immediate and innovative designs.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a novel diffusion-based method called Reifying-reasoning Errors of Diffusion (RrED) for Black-box Unsupervised Domain Adaptation (BUDA). RrED enhances target models by decomposing text and visual encoders, using a feature-sensitive module for visual enhancement and multi-modal joint fine-tuning. It prioritizes BUDA tasks and leverages differential reasoning to rectify errors, showing improved performance in reasoning and generalization across four benchmark datasets. The proposed method is relatively cumbersome and complex, but has achieved good performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1)  The proposed FSG block evaluates feature-sensitive regions by comparing global and local image regions, enhancing generalization without losing discriminative ability.\n\n2) The first to use diffusion model for black box domain adaptation problem.  SOTA performance.\n\nWeaknesses: \n1) Writing needs further improvement. For example, Figure 1 is too complex to express the core idea. Figure 2 is also too complex. At the same time, the diffusion model should not be introduced too much, and the contribution should be written in the main text.\n\n2) Introducing the diffusion model to assist in black box adaptation also brings in more knowledge. So in essence, there are unfairness in comparing with existing methods.\n\nQUESTIONS:\n1) Please expain what is the mean of lines 177-182.  Meanwhile, it is evident that the classifier p of the diffusion model heavily relies on the pre-trained model on which dataset, therefore it is necessary to introduce the pre-trained diffusion model.\n\n2) Please explain Eq (6). Why can category related areas be selected.\n\n3) From my understanding, the first step is essentially using the diffusion model to update the multi model and local model. So, how can we ensure that the diffusion model is correct in this area, as the target domain samples may not have participated in training the diffusion model.\n\n4) This work is designed with many loss functions, and the number of parameters discussed is far less than the actual number of loss functions. Although the authors fixed some parameters to 1, they still need to be discussed. \n\n5) Please deeply explain Eq.(13). It is obvious that these two optimization items are conflicting.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes RrED, a novel two-stage framework for black-box Unsupervised Domain Adaptation (BUDA), which, for the first time, integrates diffusion models as a semantic knowledge base to guide adaptation; the framework consists of Diffusion-Target model Rectification (DTR), which aligns the target model with diffusion-generated semantic features, and Self-Rectifying Reasoning Model (SRM), which enhances the target model's reasoning ability through self-corrective learning, collectively enabling effective reasoning error correction and achieving superior performance over existing state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n    1:RrED is the first to apply the diffusion model to the highly secure BUDA task and innovatively fine-tuned the text encoder of the diffusion model. This resolves the issue that existing diffusion-based domain adaptation methods mainly focus on image encoders while neglecting the nuances of text.\n    2:RrED significantly outperforms previous state-of-the-art methods on all four benchmark datasets, demonstrating its effectiveness in enhancing model reasoning and generalization capabilities.\n\n\nWeakness:\n    1:It remains unclear how sensitive the proposed RrED framework is to the choice of diffusion backbone. For instance, would replacing the default diffusion model with alternatives such as Stable Diffusion or DALL·E yield comparable results, or introduce significant performance discrepancies? A systematic analysis of this dependency would strengthen the empirical foundation of the method and inform its generalizability.\n\n    2:RrED is tailored for the BUDA setting, but it remains unclear whether its core components or two-stage framework can generalize to other domain adaptation scenarios such as open-set, or partial adaptation. Exploring its applicability beyond BUDA would further demonstrate the method's versatility.\n\n    3：The paper lacks visualizations or quantitative analyses that assess the prediction quality of the black-box predictor.   In particular, it does not provide dedicated figures or experiments illustrating how varying levels of quality—such as low accuracy, biased predictions, or noisy outputs—affect the downstream performance of RrED.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nsee weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}