Title: Diffusion-Based Probabilistic Uncertainty Estimation for Active Domain Adaptation
Venue: NeurIPS 2023
Accepted: yes
Keywords: diffusion-based models

active learning

domain adaptation

source-free domain adaptation

uncertainty estimation
Abstract: Active Domain Adaptation (ADA) has emerged as an attractive technique for assisting domain adaptation by actively annotating a small subset of target samples. Most ADA methods focus on measuring the target representativeness beyond traditional active learning criteria to handle the domain shift problem, while leaving the uncertainty estimation to be performed by an uncalibrated deterministic model. In this work, we introduce a probabilistic framework that captures both data-level and prediction-level uncertainties beyond a point estimate. Specifically, we use variational inference to approximate the joint posterior distribution of latent representation and model prediction. The variational objective of labeled data can be formulated by a variational autoencoder and a latent diffusion classifier, and the objective of unlabeled data can be implemented in a knowledge distillation framework. We utilize adversarial learning to ensure an invariant latent space. The resulting diffusion classifier enables efficient sampling of all possible predictions for each individual to recover the predictive distribution. We then leverage a t-test-based criterion upon the sampling and select informative unlabeled target samples based on the p-value, which encodes both prediction variability and cross-category ambiguity. Experiments on both ADA and Source-Free ADA settings show that our method provides more calibrated predictions than previous ADA methods and achieves favorable performance on three domain adaptation datasets.

Diffusion-Based Probabilistic Uncertainty Estimation for Active Domain Adaptation

Abstract

Active Domain Adaptation (ADA) has emerged as an attractive technique for assisting domain adaptation by actively annotating a small subset of target samples. Most ADA methods focus on measuring the target representativeness beyond traditional active learning criteria to handle the domain shift problem, while leaving the uncertainty estimation to be performed by an uncalibrated deterministic model. In this work, we introduce a probabilistic framework that captures both datalevel and prediction-level uncertainties beyond a point estimate. Specifically, we use variational inference to approximate the joint posterior distribution of latent representation and model prediction. The variational objective of labeled data can be formulated by a variational autoencoder and a latent diffusion classifier, and the objective of unlabeled data can be implemented in a knowledge distillation framework. We utilize adversarial learning to ensure an invariant latent space. The resulting diffusion classifier enables efficient sampling of all possible predictions for each individual to recover the predictive distribution. We then leverage a t-testbased criterion upon the sampling and select informative unlabeled target samples based on the p-value, which encodes both prediction variability and cross-category ambiguity. Experiments on both ADA and Source-Free ADA settings show that our method provides more calibrated predictions than previous ADA methods and achieves favorable performance on three domain adaptation datasets.

1 Introduction

Machine learning algorithms heavily depend on large amounts of labeled data. However, in many real-world scenarios, the distribution of data can change over time or across different domains, which challenges the training of models to perform consistently in all situations. Unsupervised Domain Adaptation (UDA) [1, 2, 3] addresses this problem by transferring knowledge learned from a labeled source domain to an unlabeled target domain. Despite impressive achievements, UDA struggles to bridge the performance gap with fully supervised methods due to the lack of target-supervised information. Recognizing this limitation, a promising alternative is Active Domain Adaptation (ADA) [4, 5, 6], which actively annotates a small subset of target data to greatly benefit the UDA model.

In ADA, the primary concern revolves around designing a criterion for selecting the most informative target samples that can have the maximum impact on performance once annotated. Existing ADA methods have been mainly enlightened by traditional Active Learning (AL) methods [7, 8, 9, 10] and rely on criteria that consider both predictive uncertainty and representativeness of the entire distribution. While they have shown empirical success, they usually handle the distribution shift by further measuring the representativeness of the target domain through various criteria (e.g., the output of a domain discriminator [11], free energy score [5], weighted clustering [4]). However, these

methods still estimate uncertainty based on metrics like confidence [12], distinctive margin [6], and entropy [4], obtained from a deterministic model. It is worth noting that distribution shift can easily lead to overconfident predictions for deep models [13], especially deterministic ones [14, 15], which results in poorly calibrated point estimates and unreliable uncertainty estimation in ADA.

In this work, we aim to accurately estimate the posterior distribution of the target variable y given the input x for target domain samples (i.e., p ( y | x ) ), by modeling the uncertainty in both data generation and model prediction processes. To achieve this, we utilize diffusion-based generative models [16, 17, 18] to explore the implicit predictive distribution beyond a point estimate for each target domain sample. Specifically, we harness the capabilities of diffusion models for classification [19], which involve forward and reverse diffusion chains to generate predictions from the conditional predictive distribution, without any assumptions on the parametric form of p ( y | x ) . Notably, another work [20] also tackles the miscalibration issue in ADA by considering the model prediction as a distribution on the probability simplex and introducing a Dirichlet prior over predictive distributions. However, such a constrained parametric form might not be effective if the prior distribution fails to accurately capture the predictive uncertainty [19]. Additionally, their approach only focuses on modeling the probability distribution in the model prediction space, neglecting the inherent uncertainty present in the data representation from a generative perspective.

To approximate the posterior distribution of both latent data representation and model prediction, we use variational inference by optimizing the evidence lower bound (ELBO) corresponding to the log-likelihood of all data points. Concretely, we show that for labeled data, the training objective can be formulated by Variational Autoencoder (VAE) [21] and a diffusion-based classifier [19] in the latent space. For unlabeled data, they can also be effective utilized by optimizing the ELBO, which can be implemented in a knowledge distillation framework. With the formulation, we establish a two-stage training procedure where we first learn a guided classifier for mean estimate and then train a diffusion probabilistic model guided by that for predictive distribution recovery. To ensure a shared latent embedding that can be used to solve tasks for both labeled and unlabeled data, we incorporate adversarial learning [1, 22], resulting in a Diffusion-based Adversarial Probabilistic Model (DAPM). With a collection of predictions generated by DAPM for each unlabeled target sample, we conduct t-test [23] between scores of the two most predicted classes and select the most informative samples based on the p-value, which generally takes into account the sampling scale, prediction variability and the cross-category ambiguity to estimate the prediction uncertainty for active annotation.

Our contributions: 1) We formulate ADA in a probabilistic framework for uncertainty estimate, which leverages the abilities of V AE and diffusion-based classification models to capture distributions of both data and prediction. 2) We conduct a two-stage training procedure and use adversarial learning to ensure an invariant latent space. A t-test-based criterion is utilized to estimate informativeness from multiple aspects. 3) We show that our method can naturally handle active learning for both UDA and Source-Free DA (SFDA). Experiments on three datasets show the effectiveness of our method.

2 Related Work

Active Learning [24, 25] aims to tap the fully supervised performance by only annotating a small subset of training data. The selection of informative samples can be made using a variety of criteria. Committee-based methods [26, 27, 28, 29] leverage diverse classifiers and evaluate the data informativeness based on their disagreement. Representativeness-based methods [30, 31] aim to select samples that are diverse or representative enough of the entire training distribution, typically through clustering [32] or core-set selection [33]. Uncertainty-based methods resort to various uncertainty heuristics such as entropy [34], confidence [35], and best-vs-second-best score [36], etc, to select the most uncertain instances. However, these AL methods will fail when there is a distribution shift between labeled and unlabeled data, limiting their applications in domain adaptation.

Active Domain Adaptation. Aseries of ADA works have been conducted to select the most informative target samples under domain shift. As an early work, AADA [37] applies a domain discriminator to evaluate the domainness of target samples and weights it with entropy-based uncertainty. TQS [11] designs a transferrable query criterion based on a classifier committee and a domain discriminator to handle domain shift. CLUE [4] performs an entropy-weighted clustering to select both uncertain and diverse samples. SDM [6] uses a distinctive margin loss for training and selects data lying in the margin. EADA [5] leverages the free-energy bias and uses an energy-based criterion to evaluate

the uncertainty and domainness of target samples. Despite the empirical success, these methods mainly focus on compensating for the domain gap by incorporating target-representativeness into the query function. However, the uncertainty criteria they used are still based on a point estimate from a deterministic model, which can be easily miscalibrated for out-of-distribution data [38]. To remedy this issue, DUC [20] places a Dirichlet prior on the class probability and evaluates the predictive uncertainty with a probabilistic view. However, they only consider the uncertainty in model outputs. Besides, the true predictive distribution may not be captured by such a restricted distribution form.

Diffusion-based generative models have been widely appreciated for their impressive capabilities of generating high quality and diverse samples [17, 16, 39, 18]. While most works focus on highdimensional generative tasks such as (conditional) image generation [39, 16], super-resolution [40, 41], image inpainting [42, 43], etc., some attempts have been made to apply diffusion models to categorical data [44] and discrete data [45], indicating their potential in classification tasks. Recently, Han et al. [19] utilize diffusion models to recover predictive distributions in regression and classification tasks, verifying their effectiveness for uncertainty estimation. Different from [19] that focuses on in-distribution scenarios, we consider a more challenging and realistic scenario of capturing predictive distribution in a cross-domain fashion. In addition, we also model the underlying variation of the data in low-dimensional latent space, which can be viewed as a data-level uncertainty and more favorable for downstream tasks compared to original image space [46].

3 Methodology

In ADA, we have a source domain S = { ( x s i , y s i ) } n s i =1 with n s labeled samples, and a target domain T = { ( x t i ) } n t i =1 with n t unlabeled samples, where x s i ( x t i ) is the input instance, and y s i ∈ Y is the corresponding label. It is assumed that both domains share a common label space Y but conform to different data distributions. Domain adaptation aims to train a model f Ω : X → Y parameterized by Ω that achieves good predictive performance on the target domain. To achieve this goal, the problem of ADA involves selecting a small subset of informative samples T l ⊂ T for annotation with a budget B , where B glyph[lessmuch] n t , such that the model performance on the target domain can be maximally improved. The selection process is typically completed in multiple rounds, where in each round, the model queries b samples from T u (the remaining unlabeled samples in T ) and adds them to T l . This process is repeated until the budget is exhausted. In this work, we examine active learning for both UDA and source-free UDA [47], where, besides using unlabeled target data, we can utilize the raw source data and a source-trained model, respectively, for the purposes of selection and adaptation.

3.1 Preliminary of Classification Diffusion Models for Uncertainty Estimation

In AL, a major concern in sample selection is predictive uncertainty. For a K -way classification problem, given an input variable x , the predictive uncertainty can be expressed as the posterior distribution over the predicted variable y 2 after observing the training set D , i.e., p ( y | x , D ) . Most AL methods merely pay attention to accuracy and adopt a non-Bayesian approach to train the model, e.g., Maximum Likelihood Estimation (MLE) or Maximum A Posteriori (MAP). As frequentist techniques, they are not capable of inferring the variance of predictive distribution p ( y | x ) , but merely provide a mean estimate E [ y | x ] under the assumption of additive noise [19]. Taking a Bayesian view, the predictive uncertainty can be modeled by assuming distributions over network parameters [48, 49], while it involves expensive computation and intractable posterior inference. Some alternative methods include modeling the output of a neural network as a probability distribution over possible outcomes [50, 5] or adding the noise term in the model outputs [51]. As explicit modeling methods, they all assume a specific form in p ( y | x ) (e.g., Gaussian or Dirichlet distribution).

In this work, we leverage the deep generative models to model the implicit predictive distribution. Specifically, we start with the classification diffusion models [19]. Re-denoted by y 0 ∈ R K the one-hot label, the diffusion model first applies a forward diffusion process q ( y 1: T | y 0 , x ) to original data by iteratively perturbing it to latent representations y 1: T with Gaussian noises. For each step t ,

where β t ∈ (0 , 1) is the noise variance and fixed as hyperparameter. Here the model output f Ω ( x ) (mean estimate) is injected in the forward process to act as the prior knowledge about the relationship

between x and y 0 , so that with a sufficiently large T and proper schedule { β t } T t =1 , we have

Note that the specific form of the forward diffusion process enables an efficient sampling for arbitrary steps in a closed form:

with α t := 1 -β t and ¯ α t := ∏ t s =0 α s . An appreciable feature of diffusion models is that if we know the posterior distribution q ( y t -1 | y t , x ) , we can sample y T ∼ N ( f Ω ( x ) , I ) and run the reverse diffusion process to gradually recover the original data from p ( y 0 | x ) under the guidance of f Ω ( x ) . Unfortunately, q ( y t -1 | y t , x ) is actually intractable, and a feasible alternative is to use a model p θ ( y t -1 | y t , x ) for approximation, which can be trained with the following ELBO:

where D KL is the Kullback-Leibler (KL) divergence. It is worth noting that scheduling small β t will make q ( y t -1 | y t , y 0 , x ) still a Gaussian [52], which can be explicitly formulated using Bayes theorem. Therefore L t -1 can be evaluated in a closed form. L T contains no optimizable parameter and is assumed to be sufficiently small and thus can be ignored. By adopting the appropriate form of p θ ( y t -1 | y t , x ) and reparameterization like DDPM [17], the model can be trained effectively by being tasked at predicting the forward noise glyph[epsilon1] for sampling y t , with glyph[epsilon1] θ ( x , y t , f Ω ( x ) , t ) .

3.2 Diffusion-Based Adversarial Probabilistic Model for ADA

The above conditional diffusion model admits the sampling of multiple predictions from p ( y | x , D ) for statistical analysis. However, it assumes that the evaluated sample x and the training set D comes from the same distribution, which is not the case in UDA. Aware of this, we aim to build a model that induces a common latent space Z , where the source and target samples share the same representations. We first consider the following generative process defined with the joint probability distributions (We omit the subscript i to represent arbitrary data point in the dataset):

where z s ( z t ) ∈ Z is the latent representation. To model the inference process of z s ( z t ) , we employ a shared encoder parameterized by ρ for both the source and target domains, supposing a domainshared embedding. And we use two separate decoders with parameters φ and ψ for the source and target reconstructions, respectively, since the generative process requires domain-specific information encoded in the parameter. Due to intractable posterior distributions, we then aim to learn latent variables for both domains using variational inference, with the following variational objectives:

Objective for Labeled Source Samples. For a labeled source sample ( x s , y s 0 ) , our goal is to learn both the low-dimensional embedding z s and latent class variable y s 1: T , which can be achieved by maximizing the ELBO of the marginal log-likelihood as follows:

Eq. (10) implies a standard V AE and a classification diffusion model in the latent space. We provide the detailed derivation in Appendix A.1. To ensure discriminative classification boundaries in Z , we additionally incorporate a deterministic classifier with parameter ω in the training objective. Then the overall objective for the labeled source data can be expressed as:

It is worth noting that the parameters of f Ω have been further decomposed into { ρ, ω } in our case.

Objective for Labeled Target Samples. For target samples with annotation, we treat them the same as source-labeled data in training. The only difference is that we use parameter ψ for target-specific reconstruction. Limited by space, we give the full expression of the training loss L l t in Appendix A.2.

Objective for Unlabeled Target Samples. The training objective for unlabeled target samples can not be carried out in a supervised fashion due to the absence of ground-truth labels. Therefore, we design the following variational lower bound for them by treating y t 0 and z t as latent variables:

Intuitively, it encourages to optimize a standard V AE and minimize the KL divergence between q ( y t 0 | x t ) and the approximation p ω ( y t 0 | z t ) . However, the ground-truth q ( y t 0 | x t ) is agnostic in practice. In this work, we implement it in a Knowledge Distillation (KD) framework [53], where q ( y t 0 | x t ) is substituted with the output of a teacher model f Ω ′ that is optimized by exponential moving average (EMA) from the weights of student model f Ω . We denote the KD loss as L KD and weight it by λ kd to absorb the influence of wrong predictions from the teacher model. And only those with confident predictions (larger than a pre-defined threshold α ) from the teacher model will be involved in this term. Meanwhile, we hope that the predictions of target samples to be individually confident and holistically diverse, which results in the following loss function:

where λ kl is the weighting coefficient. The derivation of Eq. (12) can be found in Appendix A.3.

Adversarial Learning between Labeled and Unlabeled data. The distribution shift between labeled and unlabeled data is a key obstacle to active learning. To extend the applicability of the diffusion model trained on labeled data to unlabeled ones, we apply a discriminator D τ parameterized by τ and conduct adversarial learning between labeled and unlabeled data, expecting to learn an invariant latent space. Inspired by CDAN [22], we also leverage the model prediction as the conditioning for joint distribution alignment, which gives the following training loss for the discriminator:

This minimax process is trained in an end-to-end mode using the gradient reversal layer [1].

3.3 Two-Stage Training Procedure

The whole model in our framework includes an encoder ρ , two decoders φ, ψ , a discriminator τ , a deterministic classifier ω and a diffusion-based classifier θ . As shown in Section 3.1, the diffusion model requires the i.i.d. assumption to hold and injects the mean estimate f Ω ( x ) into both forward and reverse diffusion process. Therefore, we schedule a two-stage training procedure in each active round. Specifically, the first stage is called the adaptation stage , which yields the following objective:

Encoder

𝜌

𝜙

𝜓

Deterministic Classifier

𝑧 𝑠 ~𝑞𝜌

𝑧 𝑠

𝑥 𝑠

𝑧 𝑡 ~𝑞𝜌

𝑧 𝑡

𝑥 𝑡

𝑝𝜙

𝑥 𝑠

𝑧 𝑠

𝑝𝜓

𝑥 𝑡

𝑧 𝑡

ω

EMA

τ

Domain Discriminator

KD

Source Decoder

Teacher Model

𝑦0

𝑓 Ω (𝑥)

𝑦𝑇

ො

𝑦0

𝑦𝑡

𝑧

𝜖𝜃 (𝑧, 𝑦

𝑡

, 𝑓 Ω

𝑥 , 𝑡)

Diffusion Classifier

…

DAPM

t-test

p-values

Sort

Annotation

Top-

b

Add

𝒮

𝒯 𝑙

𝒯 𝑢

𝒯 𝑢

𝒯 𝑙

unlabeled target data flow

labeled data flow (source and target)

labeled source data flow

labeled target data flow

target data flow (labeled and unlabeled)

Adaptation Stage

Diffusion Stage

Selection Stage

𝑓 Ω ′(𝑥)

prediction interval of the

diffusion classifier

prediction of the deterministic classifier

Repeated

sampling

𝜃

The first stage ensures a well-behaved mean estimator f Ω and a domain-invariant latent embedding. We then freeze all the trained parameters and train the diffusion model glyph[epsilon1] θ ( z , y t , f Ω ( x ) , t ) in the second stage, namely the diffusion stage , with all labeled data:

Having the diffusion model, we select target samples according to the strategy in Section 3.4 if there is a remaining budget. We depict the whole framework for ADA in Fig. 1.

Extension to Source-Free ADA (SFADA). We naturally extend our method to SFADA [47] based on the fact that the model f Ω := { ρ, ω } in the adaptation stage can be initialized (like in ADA) or source pre-trained. For the latter, we pre-train the source model with L s , and disable the source-related losses, i.e., L s and L adv in the adaptation stage. Besides, we use confident unlabeled target data pseudo-labeled by the teacher model to substitute the source samples in L s in the diffusion stage. The detailed procedure of the training process can be found in Appendix. B.

3.4 T-test-Based Selection Strategy

Existing ADA methods generally select both target-representative and uncertain samples. The former characteristic can be naturally absorbed within the adaptation stage, which makes our model operate in a domain-agnostic latent space. Therefore, we mainly focus on selecting samples with high predictive uncertainties. For each x t ∼ T u , we generate N predictions independently with the diffusion classifier, denoted by { ̂ y t n } N n =1 . Since the raw predictions can be arbitrary real vectors, we then convert them into the probability simplex by a softmax operation following [19], resulting in a set { ˜ y t n } N n =1 . Based on that, we identify the two most predicted classes among N votes for each instance, which gathers two groups of predicted scores g 1 = { ˜ y t n [ a ] } N n =1 and g 2 = { ˜ y t n [ b ] } N n =1 , where a, b denote two most voted class labels. We assume independent generation in each dimension and evaluate the uncertainty by conducting an independent two-sample t-test, where the t-value is:

here we use ¯ g 1 ( or ¯ g 2 ) and s 2 1 ( or s 2 2 ) to denote the mean and variance of the two groups, respectively. Conceptually, the t-value is formulated as the ratio of mean difference between groups to sampling variability . Compared to Best-versus-Second-Best (BvSB) [36] used in deterministic model-based AL methods, the t-test-based criterion can not only allow us to see if two means are different, but also tell us how significant the differences are. As a result, we use the p-value to express the significance of such a difference. Intuitively, uncertain samples are those with small mean differences between groups and high sampling variabilities, which leads to small t-values and high p-values. In practice, we choose unlabeled target samples with topb p-values for annotation in each active selection round.

4 Experiments

4.1 Experimental Setup

Datasets and Baselines. We evaluate our method on three widely used domain adaptation benchmarks, i.e., Office-31 [54], Office-Home [55] and VisDA [56]. We construct three groups of baselines

* For DUC and ELPT, we report the results on Office-31 and VisDA based on our own runs, respectively, according to the official code. † For SFADA, we implement several active learning methods upon our SFDA baseline and report the mean results over 3 runs.

for comparison. (i) Active learning: Random, Entropy [7], BvSB [36], CoreSet [33], BADGE [57]. (ii) Active Domain Adaptation: AADA [37], TQS [11], CLUE [4], EADA [5], DUC [20]. (iii) Source-Free ADA: ELPT [47], and our SFDA baseline with other AL selection methods: Random (DAPM-RD), BvSB (DAPM-BS), Entropy (DAPM-ET), CoreSet (DAPM-CS), BADGE (DAPMBG). We denote our method with the t-test-based selection criterion by DAPM-TT thereafter. It is worth noting that in the SFADA setting, we retain the abbreviation as DAPM for consistency, however we do not employ the adversarial learning loss.

Implementation. We implement our method on Pytorch and MindSpore 3 . The ResNet-50 [58] pre-trained on ImageNet [59] is adopted as the backbone, which constitutes the main body of the encoder. The decoder is simply a two-layer MLP. Following previous works [5, 20], we schedule 5 selection rounds, where in each round, the model selects b = 1% × n t samples for annotation, and therefore the total budget B = 5% × n t . We use SGD optimizer in the adaptation stage. The learning rate is set as 0.01 except for the VisDA dataset in SFADA, where we use 0.001 for training. The batch size is 32 for ADA and 64 for SFADA. We adopt the same learning rate scheduler as [22, 1]. For hyperparameters, we use λ kl = 0 . 1 , λ kd = 0 . 1 for ADA and λ kl = 1 . 0 , λ kd = 1 . 0 for SFADA. For all tasks, the confidence threshold α for knowledge distillation is set to 0.9. We set N = 100 , i.e., we generate 100 predictions for each individual sample for uncertainty estimation. More implementation details are provided in Appendix. C. Code is available at https://github.com/TL-UESTC/DAPM .

Evaluation Protocol. There are two classifiers involved in our method, i.e., a deterministic classifier parameterized by ω and a diffusion classifier parameterized by θ . For evaluating a target sample x t encoded by z t , we leverage the predictions generated from the diffusion classifier by calculating the expected class probability based on N independent votes, i.e., ¯ p θ ( y t | z t ) = 1 N ∑ N n =1 ˜ y t n . We show in Sec. 4.3 that ¯ p θ ( y t | z t ) provides a more calibrated prediction than the output of the deterministic classifier p ω ( y t | z t ) . On the other hand, we use the majority vote as the final predicted class.

4.2 Main Results

Active Domain Adaptation. We present our results on Office-31 and VisDA in Table 1. Notably, among traditional active selection strategies, those based on uncertainty (e.g., BvSB, Entropy, BADGE) tend to achieve higher performance compared to those based purely on representativeness (e.g., CoreSet). This is because a portion of the target domain is already well-aligned with the source domain, and selecting these easy samples would not provide informative feedback to the model. This highlights the importance of incorporating predictive uncertainty in the selection criterion. However, the accuracies of all traditional AL methods are significantly lower than those of ADA methods due

to the neglect of domain shift. Among ADA methods, our DAPM-TT outperforms other methods and achieves the best accuracies on both Office-31 and VisDA, surpassing the deterministic model EADA [5] by 0.8% and 0.3% on the two datasets, respectively. Compared to the recent method DUC [20], our method still achieves better performance, boosting the accuracy by 1.0% on Office-31 and achieving a slightly better performance on VisDA, showing the superiority of our probablistic framework. The results in Table 2 demonstrate that our method outperforms all deterministic ADA methods on the Office-Home dataset. Although it does not achieve the best accuracy in this case, it remains highly competitive with DUC. Moreover, it is worth noting that our probabilistic modeling is mainly designed to capture predictive uncertainty, rather than to optimize classification accuracy that is largely depended on domain adaptation methods. We will show in Sec. 4.3 that our method induces more calibrated predictions than DUC.

Source-Free Active Domain Adaptation. For SFADA, we mainly compare our DAPM-TT with other AL strategies using our SFDA baseline, where we conduct the adaptation stage and adopt some representative AL strategies based on the deterministic classifier. The results in Table 1 and Table 2 show that DAPM-TT consistently achieves the best performance on all datasets. In particular, we find that BvSB [36], which can be considered as the deterministic counterpart of DAPM-TT, achieves relatively good results on all datasets. However, our DAPM-TT outperforms BvSB by considering the full distribution of the predicted variable, providing a better measure of uncertainty. Compared to the recent SFADA method ELPT [47], our DAPM-TT is able to surpass it on all datasets, especially the VisDA dataset, where we achieve a 4.9% improvement in mean accuracy.

4.3 Analytical Experiments

Expected Calibration Error (ECE). To evaluate the calibration ability of our model, we plot the ECE curves for two tasks on Office-Home in Fig. 2a. The results demonstrate that, in both tasks, the output of our deterministic classifier is much more uncalibrated compared to that of the diffusion classifier, even though their accuracies remain very close (as shown in the Ablation Study). This confirms the well-known phenomenon that softmax classifiers based on point estimates tend to make overconfident predictions [60], which can be unsafe when dealing with uncertain estimation. Our DAPM aims to recover the full predictive distribution by modeling the uncertainty in both data

ADA

SF

ADA

(a) Accuracies under different budgets on VisDA. (b) Parameter sensitivity analysis on task D → A.

generation and model prediction, thereby effectively mitigating this issue. Compared to DUC [20], which only models the distribution in output space, our method achieves smaller ECE values in both tasks, demonstrating the benefits of our probabilistic framework.

Effect of Different Sampling Scales. We evaluate our model under varying sampling numbers N and report the performance on Office-Home in Fig. 2b. Our results demonstrate that the model performance initially improves as N increases. This is because when N is too small, the model suffers from sampling bias, which results in a performance degradation to a level similar to that of the point estimate. However, an excessively large number of samples can lead to increased storage and computing resources. We find that when N is around 100, the performance becomes stable.

Effect of Different Annotation Budgets. Fig. 3a shows how different budgets affect the model performance. For both ADA and SFADA, our method is able to select the most informative samples at the begining, resulting in superior performance even with a small budget. As the budget increases, the advantage of our method is slightly diluted. However, it is still able to maintain a leading accuracy.

Evaluation of Active Learning. We also provide the baseline domain adaptation performance (i.e., DAPM without active learning) in Table 1 and 2. It shows that without active selection and learning, our DAPM only achieves a moderate performance compared to modern methods that are specifically designed for UDA, given that the probabilistic backbone is mainly used for uncertainty modeling rather than accuracy. However, we are encouraged to observe that incorporating AL with our designed selection criterion significantly enhances the domain adaptation performance on all datasets. This outcome demonstrates the effectiveness of our diffusion-based uncertainty estimation, as it identifies informative samples that considerably improve overall performance.

Performance under Different Selection Strategies. To explore the superiority of our t-test-based selection strategy, we test our DAPM using other selection strategies for the ADA task. The results are presented in Table 3. In addition to strategies based on the deterministic classifier, we also evaluate three strategies based on the diffusion classifier: DAPM-VT, DAPM-IW and DAPM-AE. DAPM-VT, DAPM-IW select samples with the topB highest prediction variance and interval width on the majority voted class, respectively. DAPM-AE averages the entropy across the N predictions to get the final nonparametric entropy estimate for each data point, and select the ones with highest averaged entropies for annotation. As expected, DAPM achieved the best results. Our findings suggest that the t-test-based criterion is more suitable for the diffusion classifier since it takes into account both sampling variability and cross-category ambiguity.

T-SNE Visualization of Latent Representations. Wevisualize the latent representations of unlabeled target data and selected target data in Fig. 4 using t-SNE [61]. In this visualization experiment, we compared our DAPM-TT with BvSB [36] that is based on the deterministic classifier. It can be observed that BvSB tends to select samples from relatively ambiguous regions (the center region) since these samples often have ambiguity between different classes. However, many samples selected by BvSB are in areas where the model is able to make predictions accurately. Therefore, it will not help to correct the samples with wrong predictions, resulting in modest improvement on performance.

→

Cl)

(b) DAPM-TT (Ar

→

Cl)

→

Pr)

(d) DAPM-TT (Rw → Pr)

Our DAMP-TT, on the other hand, can select samples from both the regions where there is ambiguity between classes and the regions where a large number of samples are misclassified, which are exaclty the ones we want to select for annotation.

How Does T-test-based Selection Ensure Diversity? While our method mainly focuses on uncertainty estimation, we show that it has properties that help mitigate redundant selections. Firstly, the diffusion process introduces stochasticity by generating varied predictions for the same input. This creates more diversity in the uncertainty estimates across similar instances. In other words, duplicate inputs will not necessarily have identical uncertainty. Secondly, the t-test criterion accounts for both variability across predictions and similarity of top-2 classes. Highly variable samples with closer competing classes will be favored, which is naturally distributed around the classification boundary of every class, thus enabling diversity. In Fig. 4, we observe that our approach naturally selects a diverse range of sample classes, even though we did not explicitly impose a diversity constraint.

Ablation Study. We conduct extensive experiments to investigate the influence of different classifiers and losses in our method. The results are summarized in Table 4, and we observed the following: Firstly, in most cases, the diffusion classifier yields slightly better results than the deterministic one, indicating its superior tolerance for domain shift. Secondly, our approach still achieves much better results than other traditional AL methods without any loss function for domain adaptation, demonstrating the superiority of our selection method. Lastly, all the loss functions have a positive effect on the final result.

shown in Fig. 3b, our model exhibits low senand moderate sensitivity to λ kl in SFADA. Table 4: Ablation study of DAPM-TT on Office-31 and VisDA. Dif. and Det. are short for diffusion classifier and deterministic classifier, respectively. sourcethe training objective, the additional source do-

Hyperparameter Sensitivity. As sitivity to both λ kl and λ kd in ADA, We conjecture the reason is that the available scenario involves more loss terms in main data and distribution alignment objectives like L adv may make the optimization landscape more complex and susceptible to suboptimal solutions based on weighting hyperparameters. In contrast, in SFADA, the model solely relies on the target data and regularization losses like L KL for alignment, reducing dependence on precise weighting. As SFADA lacks sourcesupervised information, we recommend using a slightly larger λ kl to ensure good adaptation performance.

5 Conclusion

In this work, we propose a novel probabilistic framework for ADA that leverages the variability of both latent data representation and model prediction for better uncertainty estimation. Our approach combines a variational autoencoder, a diffusion probabilistic classifier, and an auxiliary deterministic classifier to guide training and ensure an invariant latent space. Our experiments on three domain adaptation benchmarks demonstrate the effectiveness of our approach in improving task performance and effectively handling uncertainty estimation for both ADA and SFADA.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 62173066, 62250061 and 62176042, and in part by Sichuan Science and Technology Program under Grant 2023NSFSC0483, and in part Sponsored by CAAI-Huawei MindSpore Open Fund.

References

Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML , pages 1180-1189. PMLR, 2015.

Zhekai Du, Jingjing Li, Hongzu Su, Lei Zhu, and Ke Lu. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In CVPR , pages 3937-3946, 2021.

Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR , pages 3723-3732, 2018.

Viraj Prabhu, Arjun Chandrasekaran, Kate Saenko, and Judy Hoffman. Active domain adaptation via clustering uncertainty-weighted embeddings. In ICCV , pages 8505-8514, 2021.

Binhui Xie, Longhui Yuan, Shuang Li, Chi Harold Liu, Xinjing Cheng, and Guoren Wang. Active learning for domain adaptation: An energy-based approach. In AAAI , volume 36, pages 8708-8716, 2022.

Ming Xie, Yuxi Li, Yabiao Wang, Zekun Luo, Zhenye Gan, Zhongyi Sun, Mingmin Chi, Chengjie Wang, and Pei Wang. Learning distinctive margin toward active domain adaptation. In CVPR , pages 7993-8002, 2022.

Dan Wang and Yi Shang. A new active labeling method for deep learning. In IJCNN , pages 112-119. IEEE, 2014.

Yazhou Yang and Marco Loog. Active learning using uncertainty information. In ICPR , pages 2646-2651. IEEE, 2016.

Zhenyu Lu and Josh Bongard. Exploiting multiple classifier types with active learning. In GECCO , pages 1905-1906, 2009.

Sheng-Jun Huang, Rong Jin, and Zhi-Hua Zhou. Active learning by querying informative and representative examples. In NeurIPS , 2010.

Bo Fu, Zhangjie Cao, Jianmin Wang, and Mingsheng Long. Transferable query selection for active domain adaptation. In CVPR , pages 7272-7281, 2021.

Avishek Saha, Piyush Rai, Hal Daumé, Suresh Venkatasubramanian, and Scott L DuVall. Active supervised domain adaptation. In ECML-PKDD , pages 97-112. Springer, 2011.

Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR , pages 427-436, 2015.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In NeurIPS , 2019.

Deng-Bao Wang, Lei Feng, and Min-Ling Zhang. Rethinking calibration of deep neural networks: Do not be afraid of overconfidence. In NeurIPS , pages 11809-11820, 2021.

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurIPS , pages 8780-8794, 2021.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS , pages 6840-6851, 2020.

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 , 2021.

Xizewen Han, Huangjie Zheng, and Mingyuan Zhou. Card: Classification and regression diffusion models. In NeurIPS , pages 18100-18115, 2022.

Mixue Xie, Shuang Li, Rui Zhang, and Chi Harold Liu. Dirichlet-based uncertainty calibration for active domain adaptation. In ICLR , 2023.

Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. Variational autoencoder for deep learning of images, labels and captions. In NeurIPS , pages 2360-2368, 2016.

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. In NeurIPS , pages 1640-1650, 2018.

Henry Hsu and Peter A Lachenbruch. Paired t test. Wiley StatsRef: statistics reference online , 2014.

Burr Settles. Active learning literature survey. 2009.

David A Cohn, Zoubin Ghahramani, and Michael I Jordan. Active learning with statistical models. JAIR , 4:129-145, 1996.

H Sebastian Seung, Manfred Opper, and Haim Sompolinsky. Query by committee. In COLT workshop , pages 287-294, 1992.

Yoav Freund, H Sebastian Seung, Eli Shamir, and Naftali Tishby. Selective sampling using the query by committee algorithm. Machine learning , 28(2-3):133, 1997.

Ido Dagan and Sean P Engelson. Committee-based sampling for training probabilistic classifiers. In Machine Learning Proceedings 1995 , pages 150-157. Elsevier, 1995.

Melanie Ducoffe and Frederic Precioso. Qbdc: query by dropout committee for training deep supervised architecture. arXiv preprint arXiv:1511.06412 , 2015.

Zhao Xu, Kai Yu, Volker Tresp, Xiaowei Xu, and Jizhi Wang. Representative sampling for text classification using support vector machines. In ECIR , pages 393-407. Springer, 2003.

Sanjoy Dasgupta and Daniel Hsu. Hierarchical sampling for active learning. In ICML , pages 208-215, 2008.

Fedor Zhdanov. Diverse mini-batch active learning. arXiv preprint arXiv:1901.05954 , 2019.

Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489 , 2017.

Xin Li and Yuhong Guo. Adaptive active learning for image classification. In CVPR , pages 859-866, 2013.

Keze Wang, Dongyu Zhang, Ya Li, Ruimao Zhang, and Liang Lin. Cost-effective active learning for deep image classification. TCSVT , 27(12):2591-2600, 2016.

Ajay J Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classification. In CVPR , pages 2372-2379. IEEE, 2009.

Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, and Manmohan Chandraker. Active adversarial domain adaptation. In WACV , pages 739-748, 2020.

Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshminarayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. In NeurIPS , pages 7498-7512, 2020.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In CVPR , pages 10684-10695, 2022.

Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. TPAMI , 2022.

Max Daniels, Tyler Maunu, and Paul Hand. Score-based generative neural networks for large-scale optimal transport. In NeurIPS , pages 12955-12965, 2021.

Patrick Esser, Robin Rombach, Andreas Blattmann, and Bjorn Ommer. Imagebart: Bidirectional context with multinomial diffusion for autoregressive image synthesis. In NeurIPS , pages 35183532, 2021.

Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR , pages 11461-11471, 2022.

Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. In NeurIPS , pages 1245412465, 2021.

Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In NeurIPS , pages 17981-17993, 2021.

Manuel Pérez-Carrasco, Guillermo Cabrera-Vives, Pavlos Protopapas, Nicolas Astorga, and Marouan Belhaj. Adversarial variational domain adaptation. CoRR , abs/1909.11651, 2019.

Xinyao Li, Zhekai Du, Jingjing Li, Lei Zhu, and Ke Lu. Source-free active domain adaptation via energy-based locality preserving transfer. In MM , pages 5802-5810, 2022.

Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In ICML , pages 1613-1622. PMLR, 2015.

José Miguel Hernández-Lobato and Ryan Adams. Probabilistic backpropagation for scalable learning of bayesian neural networks. In ICML , pages 1861-1869. PMLR, 2015.

Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In NeurIPS , 2018.

Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In NeurIPS , 2017.

William Feller. On the theory of stochastic processes, with particular reference to applications. In Selected Papers I , pages 769-798. Springer, 2015.

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015.

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In ECCV , pages 213-226. Springer, 2010.

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In CVPR , pages 5018-5027, 2017.

Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924 , 2017.

Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv:1906.03671 , 2019.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , pages 770-778, 2016.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR , pages 248-255. Ieee, 2009.

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In ICML , pages 1321-1330. PMLR, 2017.

Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR , 9(11), 2008.

Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi-supervised domain adaptation via minimax entropy. In ICCV , pages 8050-8058, 2019.

Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterization trick. In NeurIPS , 2015.

Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In ICML , pages 1050-1059. PMLR, 2016.

Audun Jsang. Subjective Logic: A formalism for reasoning under uncertainty . Springer Publishing Company, Incorporated, 2018.

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In NeurIPS , 2017.

Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, and Decebal Constantin Mocanu. Deep ensembling with no overhead for either training or testing: The all-round blessings of dynamic sparsity. arXiv preprint arXiv:2106.14568 , 2021.

Appendix Contents

Derivations

Algorithm

More Implementation Details

Additional Experimental Results

Further Discussion

Limitations and Broader Impacts

A Derivations

A.1 Derivation of the Labeled Source Objective

Given a labeled source sample ( x s , y s 0 ) , our goal is to inference the latent data representation z s and a sequence of latent class representations y s 1: T , which controls the generation process of data points and predictions, respectively. The log-likelihood of the labeled source data can thus be expressed by:

Since Eq. (17) is intractable to compute in practice, we then leverage variational inference to approximate the posterior distribution of unknown variables ( z s , y s 1: T ) and solve it by optimizing the following ELBO:

where q ( z s , y s 1: T | x s , y s 0 ) is the approximation of the ground-truth joint posterior p ( z s , y s 1: T | x s , y s 0 ) , which can be further factorized as:

With Eq. (19) and the generative process assumed in Eq. (8), we have the following derivation:

In GLYPH<192> , the forward diffusion process q ( y s 1: T | x s , z s , y s 0 ) and the reverse diffusion process p θ ( y s 0: T | x s , z s ) are still based on the original input x s . In this work, we make a simplification design to assume that the observed class variable y s 0 and latent ones y s 1: T are only conditioned

on the latent variable z s . We demonstrate the reasonability of this simplification in Appendix A.3. Consequently, we have

which has the same form of ELBO as the diffusion classifier in Eq. (4).

A.2 Expression of the Labeled Target Objective

For the labeled target data ( x t , y t 0 ) , the unknown latent variables are the same as the labeled source data, and therefore the ELBO is analogous to that of the labeled source data. The only difference is that we use the target-specific decoder p ψ ( x t | z t ) . We give the full expression as follows:

Analogously, we additionally impose the classifier p ω ( y t 0 | z t ) in the latent space to jointly train the source and target labeled data. The final training objective L l t for labeled target data is therefore:

A.3 Derivation of the Unlabeled Target Objective

For the unlabeled target data x t , our goal is to inference the low-dimensional latent embedding z t that induces a domain-invariant latent space Z and the class label y t 0 based on z t . Besides, we assume a meanfield distribution on q ( z t , y t 0 | x t ) , which can then be factorized as:

Therefore, we optimize the following ELBO by regarding z t and y t 0 as unknowns:

Since our deterministic classifier encodes the covariate-dependence between y t 0 and z t , therefore, y t 0 is not depended on x t in our formulation, i.e., p ω ( y t 0 | x t , z t ) = p ω ( y t 0 | z t ) . ` demonstrates that, to maximize L u ELBO , D KL ( q ( y t 0 | x t ) ‖ p ω ( y t 0 | z t ) ≡ 0 should always be satisfied. On the other hand, we empirically find that p ω ( y t 0 | z t ) can be a good approximation of q ( y t 0 | x t ) even when it is solely based on latent z t . Therefore, we assume that the model output y t 0 is only depended on the latent embedding z t , which supports the derivation in Eq. (20).

B Algorithm

B.1 Algorithm of DAPM-TT for Conventional Active Domain Adaptation

The overall trianing and section procedure of DAPM-TT for ADA is summarized in Algorithm 1.

Algorithm 1 Pseudo code of DAPM-TT for ADA

B.2 Algorithm of DAPM-TT for Source-Free Active Domain Adaptation

In SFADA, We do not use the domain classifier since the source domain data and target domain data cannot co-exist. In addition, we also use confident unlabeled target samples that are pseudo-labeled by the teacher model to substitute the source-labeled samples in L s in the diffusion stage. We denote the corresponding training loss by L p t .

The overall trianing and section procedure of DAPM-TT for SFADA is summarized in Algorithm 2.

Algorithm 2 Pseudo code of DAPM-TT for SFADA

Require: Labeled source dataset S for pre-training, whole target dataset T , unlabeled target dataset T u , labeled target dataset T l , total training rounds R , total annotation budget B , per round annotation budget b , step number per adaptation stage N a , step number per diffusion stage N d , step number of source pre-training N s .

Ensure: Optimal model parameters { θ, ρ, φ, ψ, ω } .

C More Implementation Details

C.1 Network Architecture

Variational Autoencoder The architecture of the V AE and the deterministic classifier is presented in detail in Fig. 5. The encoder comprises a pre-trained ResNet-50 backbone and three initialized linear layers with a ReLU activation following the first linear layer. We assume a Gaussian distribution for the latent embedding, and its mean and covariance are estimated by two separate linear layers based on the first linear layer. The decoder is a two-layer MLP that has the same output dimension as the backbone's output. This encourages the decoder to reconstruct the feature generated by the backbone.

Deterministic Classifier The deterministic is simply a single layer linear classifier. We adopt the weight normalization technique on the classifier to stablize the training.

Diffusion Classifier The diffusion classifier is conditioned on the latent embedding z , the groundtruth y , the guided information f Ω and the time step t . We adopt the same architecture as [19] for class variable diffusion, except for the dimension of the input variable. For clarity, we describe the detailed model structure in Fig. 6 (a).

(ResNet-50)

Backbone

2048

512

256

256

μ

σ

Encoder

𝜌

z

reparameterization

512

ReLU

2048

Decoder

𝜙(𝜓)

Classifier

ω

ReLU

Linear

Linear

Linear

Linear

Linear

Linear

BN

SoftPlus

Linear

𝑧

Linear

Linear

256

512

512

Linear

512

256

SoftPlus

SoftPlus

SoftPlus

Linear

BN

256

𝑦

256

𝑦 ′

256

Class num

C

Linear

Embedding

𝑡

1

256

⊙

256

BN

SoftPlus

⊙

256

Linear

𝑡

1

256

⊙

256

BN

SoftPlus

256

Embedding

𝑦 ′

guided information

C

concatenate

⊙

Hadamard product

256 * class num

1024

1024

1024

ReLU

Linear

Dropout

Linear

ReLU

Dropout

1024

Linear

1

Sigmoid

(a) Diffusion Classifier

(b) Discriminator

Class num

Class

num

Linear

𝑡

1

256

⊙

256

BN

SoftPlus

256

Embedding

Linear

𝑡

1

⊙

Embedding

Class num

ො 𝑦

Domain Discriminator. As shown in Fig. 6 (b). The domain discriminator we used is a three-layer MLP with a Dropout layer after the first and the second layers. And the output is a one-dimensional value with Sigmoid activation, which indicates the domainness of the sample.

C.2 Training Details

Conventional Active Domain Adaptation. In the adaptation stage, we utilize the SGD optimizer with a learning rate of 0.01, momentum of 0.9, and weight decay of 0.001. We set the EMA rate for the teacher model to 0.99. In the V AE objective, we assume a standard Gaussian distribution, N ( 0 , I ) , for the prior distribution of the latent variable z s ( z t ) . For Office-31 and VisDA, we use the features generated by a pre-trained ResNet-50 and freeze the backbone parameters to accelerate training and conserve memory. However, for Office-Home, which has more diverse categories, we jointly train the backbone with other modules to learn more specific category knowledge, and we set the learning rate to 0.001, which is 10 times lower than that of other models. For Office-31 and Office-Home, we conduct adaptation for 5 epochs and train the diffusion classifier for 10 epochs in each training round. The total number of training rounds is 20. For VisDA, we set the epoch number in each stage to 1, and the total number of training rounds is 10. To train the diffusion classifier, we use the Adam optimizer with a learning rate of 0.001 and epsilon of 1e-8. The batch size is the same as that in the adaptation stage, and we use an EMA strategy with a rate of 0.9999 to update the model parameters. All experiments are conducted on a single RTX 3090 GPU.

Source-Free Active Domain Adaptation. In SFADA, we use the SGD optimizer without momentum and weight decay for adaptation. The learning rate is set to 0.01 for Office-31 and Office-Home, and 0.001 for VisDA. As with ADA, we freeze the backbone for Office-31 and VisDA and open it for

Office-Home. We pre-train the source model for 10 epochs for VisDA and 30 epochs for the other datasets. In each active learning round, we set the epoch numbers for the adaptation stage and the diffusion stage to 5 and 10, respectively, for Office-31 and Office-Home, and both to one for VisDA. The training details for the V AE, teacher model, and diffusion classifier are the same as in ADA.

C.3 Hyperparameter Choices of the Diffusion Classifier

Following [19], the hyperparameters of the diffusion classifier are set in a standard DDPM [17] manner. Specifically, the number of diffusion timesteps T is set to 1000, and a linear noise schedule with β 1 = 0 . 0001 and β = 0 . 02 is adopted for the forward diffusion process.

C.4 Implementation of Compared Baseline Methods

Note that for conventional ADA, we cite the results of previous AL methods and ADA methods reproduced in [5] if the experimental settings are the same. For DUC [20] that does not report the result on Office-31 dataset, we report the resuls by our own runs based on the code from the official repository at https://github.com/BIT-DA/DUC . We have tuned some hyperparameters to ensure the best resuls we can achieve.

For SFADA, we implement compared baseline algorithms on our DAPM baseline with following details:

Random. We abondon the use of any selection strategy and randomly select samples from the unlabeled target dataset T u for annotation.

BvSB. We compute the best-versus-second-best score based on the output of the deterministic classifier for each unlabeled sample and select b samples with the lowest scores for annotation.

Entropy. We use the conditional entropy based on the output of the deterministic classifier to measure the prediction confidence. And samples with highest entropy values are selected for annotation.

CoreSet. We regard the sample selection in each round as a core-set cover problem and solve it with the code at https://github.com/ozansener/active_learning_coreset .

BADGE. We obtain the gradient vectors based on the pseudo labels generated by the deterministic classifier and utilize K-Means++ on the gradient vectors for diverse sampling. The algorithm is implemented based on the repository at https://github.com/JordanAsh/badge .

ELPT. We cite the resuls on Office-31 and Office-Home dataset from the original paper [47]. For VisDA, we run this method and report the resuls on ResNet-50 backbone based on the official code at https://github.com/TL-UESTC/ELPT .

D Additional Experimental Results

D.1 Accuracies of Confident Predictions under Different Thresholds

Fig. 7 illustrates the accuracies of the teacher model's predictions for confident samples across different threshold settings. At each training step, the teacher model is updated with the student model, and we have computed the accuracy of the current batch and presented it as a curve. As expected, increasing the threshold leads to an increase in the teacher model's accuracy. However, when the threshold is relatively high, only a small number of samples are considered confident at the beginning, leading to a higher accuracy initially and a subsequent drop. It is worth noting that the teacher model provided relatively reliable predictions at a threshold value of 0.9. Raising the threshold further would result in too few confident samples, making 0.9 a more appropriate choice.

100

300

500

700

900

1100

1300

1500

Step

50

60

70

80

90

Accuracy (%)

Synthetic -> Real (ADA)

0.9

0.7

0.5

0.3

0

(a) ADA

100

300

500

700

900

1100

1300

1500

Step

60

70

80

90

Accuracy (%)

Synthetic -> Real (SFADA)

0.9

0.7

0.5

0.3

0

(b) SFADA

D.2 Prediction Visualization of Different Classifiers

To investigate the contrasting behaviors of deterministic and diffusive classifiers, we randomly select two samples on task Ar → Cl, and visualize the predictions made by the deterministic classifier and the diffusion classifier ( N = 100 ). As shown in Fig. 8a, the deterministic classifier exhibits high confidence in predicting a refrigerator as a mug . This verifies the overconfident issue in traditional softmax-based deterministic model, making it challenging for the active learning methods to detect the error and select such hard samples. In contrast, the diffusion classifier produces an uncertain prediction, indicating confusion in its output with a p-value of 0.873. As depicted in Fig. 8b, the deterministic classifier displays high uncertainty and misclassified the sample, whereas the diffusion classifier correctly classifies the sample with a p-value of 0.024, which saves budget and resources that would have been wasted on correcting the misclassification.

p-value: 0.873

Scissors

Kettle

Fan

Sink

Bottle

Paper_Clip

Calculator

Refrigerator

Monitor

Push_Pin

Ruler

Oven

Candles

Radio

Speaker

Mop

Bike

Clipboards

Pen

Toys

Laptop

Webcam

Couch

Helmet

Trash_Can

Spoon

Postit_Notes

Sneakers

Bucket

TV

Curtains

Pan

Printer

ToothBrush

Alarm_Clock

Marker

Eraser

Backpack

Notebook

Chair

Knives

Flowers

Desk_Lamp

Telephone

Folder

Keyboard

Exit_Sign

Hammer

Bed

Table

Glasses

Batteries

Lamp_Shade

File_Cabinet

Fork

Shelf

Calendar

Screwdriver

Flipflops

Pencil

Computer

Soda

0.0

0.1

0.2

0.3

0.4

(a) Refrigerator

p-value: 0.024

Scissors

Kettle

Fan

Sink

Bottle

Paper_Clip

Calculator

Refrigerator

Monitor

Push_Pin

Ruler

Oven

Candles

Radio

Speaker

Mop

Bike

Clipboards

Pen

Toys

Laptop

Webcam

Couch

Helmet

Trash_Can

Spoon

Postit_Notes

Sneakers

Bucket

TV

Curtains

Pan

Printer

ToothBrush

Alarm_Clock

Marker

Eraser

Backpack

Notebook

Chair

Knives

Flowers

Desk_Lamp

Telephone

Folder

Keyboard

Exit_Sign

Hammer

Bed

Table

Glasses

Batteries

Lamp_Shade

File_Cabinet

Fork

Shelf

Calendar

Screwdriver

Flipflops

Pencil

Computer

Soda

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

D.3 Effect of the Modeling of the Latent Feature Distribution

In comparison to CARD [19] which utilizes a deterministic feature extraction network and a diffusion classifier to evaluate uncertainty based on the original image, our method employs an additional V AE to model data uncertainty in a low-dimensional latent space, and the diffusion classifier is based on latent variables. To investigate the benefits of this improvement for ADA tasks, we implement CARD in the ADA task and report the results of it on Office-31 and VisDA. Specifically, we use a deterministic ResNet-50 network as the feature extractor and train a deterministic classifier on top of it to guide the diffusion classifier. The diffusion classifier takes the original image x as one of the inputs and uses the same independent two-sample t-test-based criterion for sample selection. We denote this implementation by CARD-TT. As shown in Table 5, DAPM-TT significantly outperforms CARD-TT on both datasets. It is exciting to see that although V AE is mainly designed for modeling the uncertainty of the data generation process, it results in a significant improvement with respect to accuracy. We conjecture that the reason for this improvement is two-fold: firstly, domain shift leads to a significant distribution shift in the image space, and in such case, CARD fails to work as intended [19]. This effect is mitigated to a certain extent in the low-dimensional and less noisy latent space. Secondly, in VAE training, we use the same prior distribution, i.e., the standard Gaussian distribution, for the latent variables of data in both the source and target domains. This design draws the samples in both domains closer to the standard Gaussian distribution, thereby achieving an indirect distribution alignment.

D.4 Qualitative Analysis on Selected Samples

We present in Fig. 9 a list of all the selected samples by our approach to gain insight into which samples are chosen. Intuitively, our method tends to select samples that are challenging for the model, such as those with complex backgrounds or different styles from the other images in the dataset. Labeling these samples can help reduce the ambiguity in the model and enable it to better capture the semantic aspects of the category. Interestingly, we observe that our approach naturally selects a diverse range of sample classes, even though we did not explicitly impose a diversity constraint. Furthermore, we observe that the p-values of the selected samples gradually decrease with each

training round, and by the 5th round, the lowest p-value is 0.518. This indicates that selecting approximately 5% of the samples is sufficient to mitigate much of the ambiguity in the model.

D.5 Comparison between Different T-test Strategies

Based on the scores of the two most probable classes predicted by the diffusion classifier, we can use either paired two-sample t-test or independent two-sample t-test for selection, which correspond to different assumptions for the generation of predictions. The former assumes that the scores of different classes are generated in pairs, while the latter assumes that they are generated independently. For a paired t-test, the t-value of a target sample x t is calculated as follows:

where ¯ d = 1 N ∑ d i is the mean of sample differences d i = ˜ y t i [ a ] -˜ y t i [ b ] , µ d is the difference of the null hypothesis (usually set as 0), and s d = √∑ ( d i -¯ d ) 2 /N -1 is the standard deviation of the sample difference.

We denote our method with paired t-test-based criterion by DAPM-TT*, and report the resuls on Office-31 and VisDA in Table 5. Empirically, we find that independent two-sample t-test yields superior performance on both datasets. We conjecture the reason might be that the independent twosample t-test considers the internal variance of each group of samples. Therefore, when evaluating uncertainty, it considers an additional dimension compared to the paired t-test. In this work, we adopt independent two-sample t-test for all experiments.

D.6 Combine with other Semi-Supervised UDA methods

It is worth noting that our formulation provides certain flexibility for the implementation of the training objectives, which showcases the modular nature of the framework. For instance, we use a teacher-student framework to implement the second term in Eq. (12) and adopt conditional adversarial learning as the main adaptation technique, which may limit the baseline DA performance and then affect the performance after active learning. For ADA, he DA component and AL component can be decoupled. We have tried to incorporate a SSDA method MME [62] to implement the adaptation stage, which is also practiced by previous ADA works, e.g., [11, 4]. Specifically, the adversarial learning loss (i.e., L adv ) is replaced by the minimax entropy loss, and other losses remains unchanged. The results on Office-Home dataset can be found in Table 6, which shows the SSDA component further boosts the performance of our method.

D.7 Benefits of Data-level Uncertainty

Weconducted additional experiments ablating the VAE component, which can be simply implemented by replacing the variational encoder with a deterministic feature encoder. Through experiments, we found that the performance on VisDA-2017 is worse than modeling both the data-level and prediction-layer uncertainty, as shown in Table 7.

E Further Discussion

Related Uncertainty Estimation Works. In machine learning, there are primarily two kinds of uncertainties that are studied, i.e., epistemic uncertainty which arises due to a lack of knowledge or data and can be reduced with more data or improved models, and aleatoric uncertainty that stems from the inherent randomness in the data [51]. To model these uncertainties, the community has proposed many Bayesian deep learning methods. From a Bayesian perspective, these two uncertainties can be modeled by the posterior of model parameters W and outputs y , respectively, using the following formulation:

The family of Bayesian neural networks (BNNs) [48, 49, 63] is specifically designed to capture epistemic uncertainty by assuming a probability distribution over the network parameters. This involves estimating the posterior distribution over the parameters of the neural network given the observed data. However, due to the intractable form of the posterior, BNNs are often trained using appropriate approximations like Markov Chain Monte Carlo (MCMC) or Variational Inference (VI). Another approximation for BNNs is Monte Carlo Dropout [64], which assumes a Bernoulli distribution over network parameters. During inference, the output of the network is averaged over multiple stochastic forward passes with dropout enabled, resulting in a distribution over the predictions.

Evidential deep learning (EDL) [50, 20] is another method that models uncertainty associated with the output of the model based on evidential theory. In EDL, this is typically achieved by using a distributional output based on the theory of Subjective Logic [65], such as a Dirichlet distribution over class probabilities for classification tasks, instead of a point estimate.

For non-Bayesian methods, ensemble-based methods [66, 67] have been proposed to model predictive uncertainty by combining multiple deterministic neural networks with different initializations. However, all these methods are designed to capture either epistemic uncertainty or aleatoric uncertainty alone by modeling probability distributions over the model parameters and outputs, respectively. Moreover, they still impose a restricted form of distributions, such as Gaussian or Dirichlet, which limits their applicability in practice.

To capture both sources of uncertainties in a single model, Kendall et al. [51] propose modeling aleatoric uncertainty in the model outputs beyond model parameters by predicting the noise term for the output variable of each sample as part of the model output. However, the form of the noise is still assumed to be Gaussian.

Recently, Han [19] proposed modeling the implicit output distribution by leveraging the generative capability of the diffusion model. However, they only model aleatoric uncertainty in their formulation since the model they use is still a deterministic neural network, and the proposed method, CARD, only enables in-distribution generalization.

In addition to modeling aleatoric uncertainty with the diffusion classifier, we also incorporate a V AE to model the underlying data generation process. The V AE learns a probabilistic distribution over the latent space, which represents the model's uncertainty about the true underlying distribution of the data, given the limited amount of training data. As more data is provided during training, the learned distribution should converge to the true underlying distribution, reducing epistemic uncertainty. Therefore, our DAPM also offers a way to measure epistemic uncertainty. Furthermore, our diffusion classifier is conditioned on the latent variables in Z rather than the ones in the original image space X . We argue that the latent space contains less noise and is more suitable for cross-domain tasks.

The choice of Diffusion-based model for probabilistic modeling. Diffusion models have been shown to be equivalent to reversing stochastic differential equations (SDEs). Running the reverse process along the SDE trajectory to recover the original data distribution provides a principled way to recover the data distribution. Besides, compared to other generative models that complete generation in a single step, the multi-step noise schedule smooths the posterior and improves the exploration of the probability space, which will also result in more stable training and generation. In addition, the forward-reverse diffusion process allows effective modeling of high-dimensional complex distributions without parametric assumptions. Actually, the correlation between the covariates and the prediction variable is assumed to be high-dimensional and complex as well. Therefore, we regard diffusion models as a better choice to model the underlying predictive distribution.

F Limitations and Broader Impacts

Limitations. Our work presents a way to recover the predictive distribution of deep models by combining the power of diffusion models and variational autoencoders. However, like any research, our study may have some limitations that should be acknowledged. Firstly, the task we focus on is limited to image classification in this work. In our future study, we may extend the scope of research to other areas like image segmentation and object detection, etc. Secondly, probabilistic models are often more difficult to interpret compared to deterministic models. It is important to study insights in future research to help interpret probabilistic models.

Broader Impacts. Indeed, the impact of active domain adaptation is significant, especially in scenarios where labeled data is scarce in the target domain. The ability to adapt to new domains with limited labeled data can potentially reduce the time and cost required to gather labeled data for each specific task, thus making the deployment of machine learning models more accessible and costeffective. This can also facilitate the development of more robust and generalizable machine learning models that can be used across multiple domains, which is particularly important for applications that operate in dynamic and diverse environments. Overall, our work contributes to advancing the field of machine learning and promoting the development of more efficient and adaptive technologies.

(b) Round 2

(c) Round 3

Trash_Can 0.983

Monitao.sjS

Bike0974

Sink 0.968

trashicon

uter094

Sneakers 0.938

Candles 0.906

ttle0.902

》.

Bottle 0.865

Bucket 0.862

Spoon0.832

Table 0.83

ToothBrush 0.983

Note

96

V0.95B

Sink0.951

Classes 0.9

Bottle0.923

Radio0.92

0.908

Fork0.875

Monitor0.803

Spoon0.801

Pencil 0.786

Flipflops0.737

Spoon 0.889

Bike 0.879

Bucket0.845

Guns den't killpeople.

Spoons kill

people.

Notebo

0.793

0.777

RTAL

Refrigerator0.712

e0.668

Marker0.63

Kettle 0.97

Bottle 0.962

ToothBrush 0.889

Folder0.879

Rado 0.845

0.811

ami folders

Computer0.735

NESTUOCnG

Sink0.712

Mop9.63

(a) Round 5

Table0.915

Hammer

0855

Mop0.749

Pencil0.716

Soda 0.649

Fan0.638

Fork0.626

吧

Fork 0.564

Bed0.538

Bike0.536

Kettle0.518

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles active domain adaptation based on the diffusion probabilistic model to estimate the prediction uncertainty. The authors apply the variational inference to the joint distributions of input data, latent codes in feature and prediction spaces, and labels. To realize adaptation, KL loss, and adversarial training are employed. Based on the losses, the model is trained in a two-stage fashion. To select the informative samples, a t-test-based strategy is explored.\n\nSTRENGTHS:\n* Estimating the prediction uncertainty based on diffusion models seems to be more accurate as shown in experiments.\n* The results for active domain adaptation and source-free domain adaptation on three benchmarks are better than those of the baseline.\n* The paper is well-written and easy to follow.\n\nWEAKNESSES:\n* The main weakness is the combination nature. The proposed method seems to be a combination of VAE, diffusion-based uncertainty estimation [19], knowledge distillation, and adversarial training, which are common techniques in generative models and domain adaptation. Such a contribution is below the standard of NeurIPS. \n* The diffusion-based uncertainty estimation is borrowed from [19]. It would be better to develop it specified for DA with distribution shift.\n* Missing results on more challenging datasets such as DomainNet. From Table 1-2, DAPM-TT is marginally better or worse than DUC on VisDA and Office-Home. It would be better to evaluate DAPM-TT and DUC on more challenging datasets.\n* From Table 4, the results of the diffusion classifier and deterministic classifier are similar, so the effectiveness of the diffusion-based uncertainty estimation for ADA is limited.\n\nQUESTIONS:\n* The motivation for the design of the t-test-based strategy is not clear. Why did the authors choose the two most predicted classes? How about three or even more? Why define the t-value as Eq. (17)? Does it follow a t-distribution? \n* With Eq. (17), how to obtain the p-value. Please provide more details.\n* It would be better to add more explanation on the last two losses in Eq. (13).\n* In Line 40, the paper aims to accurately estimate the posterior distribution. However, a large margin exists between the estimated uncertainty and the true one, as in Fig. 2(a).\n\n\n* Minor: How about the performance on modern backbones, such as transformer or CLIP?\n\nLIMITATIONS:\nIt would be better to develop diffusion-based uncertainty estimation specified for DA with distribution shift.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a probabilistic framework for ADA, which incorporates both data-level and prediction-level uncertainties using variational inference. It employs a VAE and a diffusion-based classifier to approximate the joint posterior distribution. Adversarial learning is used to ensure an invariant latent space, and a t-test-based criterion is applied to select informative target samples. Experimental results demonstrate the effectiveness of the method on ADA and SFADA tasks.\n\nSTRENGTHS:\n1. This paper introduces a novel probabilistic framework for ADA, which goes beyond traditional deterministic model-based uncertainty estimation by capturing both data-level and prediction-level uncertainties. This approach is based on variational inference and incorporates a variational autoencoder and a latent diffusion classifier to approximate the joint posterior distribution.\n\n2. This paper highlights that the proposed method provides more calibrated predictions compared to previous ADA methods. Calibrated predictions are desirable in ADA tasks to ensure reliable and well-calibrated confidence estimates for informativeness assessment.\n\n3. The incorporation of t-test-based criterion for sample selection is inspiring and reasonable, as the t-value considers various factors, such as prediction variability, cross-category ambiguity, and sample size, in a unified manner, making it a comprehensive criterion.\n\n4. The proposed method is naturally compatible with SFADA setting, since the diffusion-based classifier is built on top of a transferable task-specific model, which can be trained in a standard UDA manner or in a source-free UDA manner.\n\nWEAKNESSES:\n1. This paper presents a comprehensive framework that combines variational inference, adversarial learning, and knowledge distillation. While these techniques can enhance the capabilities of the model, they may introduce increased complexity and computational cost, which could limit the scalability and practicality of the method.\n\n2. The introduced improvements of the proposed method over other methods appear to be limited. In particular, the proposed DAPM does not achieve the best performance on the Office-Home dataset, which necessitates further analysis or justification.\n\n3. In the ablation study, the comparison primarily focuses on highlighting the improvements of the diffusion classifier over the deterministic classifier, specifically emphasizing the advantages of prediction-level probabilistic modeling. However, the benefits resulting from data-level probabilistic modeling brought by the VAE are not demonstrated in this analysis.\n\nQUESTIONS:\n1. I like the design of this work. However, it still remains unclear that why the diffusion-based classifier can accurately recover the predictive distribution. Is there any theoretical guarantee or intuitive justification that the diffusion-based classifier is a better choice to model the predictive distribution?\n\n2. Does the use of the diffusion model and the two-stage training strategy significantly increase the training and inference time?\n\n3. Due to the impracticability of conducting explicit adversarial learning within the SFADA setting, the term \"DAPM\" is unsuitable in this context, necessitating a more appropriate name specifically for SFADA.\n\nLIMITATIONS:\nThe limitations have been discussed in the appendix. I do not find potential obvious negative social impact associated with this work.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to tackle the uncertainty estimation problem in ADA with a probabilistic framework, which captures two levels of uncertainties by a variational autoencoder and a diffusion-based classifier, respectively. An adversarial adaptation strategy and a statistical test-based criterion are further introduced. The experiments are conducted on both ADA and source-free ADA tasks, which verify the effectiveness of the proposed DAPM.\n\nSTRENGTHS:\n1. This paper revolves around the fundamental question in ADA, i.e., uncertainty estimation. Specifically, the authors establish a probabilistic model to assess the full predictive distribution beyond a point estimate, which is accomplished by a conditional diffusion-based probabilistic model.\n\n2. To tackle the domain shift problem, the authors propose to explore an invariant latent space with an adversarial learning strategy. The diffusion-based classifier is conditioned on latent variables, and the data uncertainty in this latent space is also explicitly modeled.\n\n3. This paper is well organized and technically solid. Specifically, the motivation of this work is well established, and the proposed method is plausible to address the uncertainty estimation problem. Besides, the proposed method is compatible with both conventional ADA and source-free ADA settings as presented in the manuscript. \n\n4. The experiments on three ADA datasets demonstrate that the proposed method exhibits not only high accuracy for classification but also low ECE for uncertainty calibration.\n\nWEAKNESSES:\n1. Despite the uncertainty estimation component (the second stage), the domain adaptation component (the first stage) is solely based on conditional domain adversarial networks (for ADA). This limited design could explain the marginal improvement observed when compared to other SOTA methods. I think the proposed method can benefit from incorporating other more advanced UDA or SSDA techniques for both ADA and SFADA.\n\n2. The proposed method differs from previous adversarial learning-based ADA methods by employing adversarial learning between labeled and unlabeled samples. However, the underlying motivation for this design choice is not explicitly stated, and the potential benefits of it are not adequately evaluated or demonstrated.\n\n3. Minor: The equation indexes in the appendix are missing.\n\nQUESTIONS:\n1. The diffusion model involves numerous steps of forward and inverse diffusion. How different are the model size and running time compared to baseline methods?\n\n2. The diffusion-based classifier is conditioned on the output scores of the deterministic classifier, which acts as prior knowledge regarding the relationship between x and y_0. How would the model perform without this prior?\n\nLIMITATIONS:\nThe authors have discussed the limitations and broader impacts in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors propose a new model for active domain adaptation. They train a VAE with domain-invariant latents (using adversarial alignment) and a supervised+teacher/student classifier head on top of learned embeddings to obtain point estimates of class probabilities, and then train a denoising diffusion model on predicted class labels to more accurately predict uncertainty of predictions. More specifically, for each unlabeled target sample, authors use denoising diffusion to sample from the distribution of predicted softmax vectors, then compute t-test statistic for the null hypothesis that two most likely output classes have equal probabilities, and label examples that fail to reject this hypothesis the most. Authors evaluate this (and an analogous source-free algorithm) on VisDA and Office-Home and show that in most cases their method achieves best performance (except Active DA on Office-Home where it archives 2nd best) and is calibrated better then the runner up (DUC). Authors evaluate the effect of the number of diffusion samples, the annotation budget, they propose and check several alternative deterministic and diffusion-based selection criteria, hyperparameter sensitivity, and effect of different VAE losses (e.g. adversarial alignment).\n\nSTRENGTHS:\nThe paper is well-written, technically sound, does a good job explaining the method, and the performance is good. I had several questions and ablation in mind while reading the paper, and most of them were addressed in the experiment section. The evaluation is thorough.\n\nWEAKNESSES:\nThe method uses every trick from the book to beat state of the art (eg student/teacher training, etc.), and combined with the proposed novel approach (diffusion-based selection criterion), it indeed achieves a small improvement over sota. I appreciate that authors provide this detailed ablation, but it reveals that the novel component provides only ~1pp improvement over a very strong “deterministic backbone method”.\n\nI understand that authors attempted to squeeze as much content as possible into the paper, but currently the paper is at times difficult to read because the table / figure margins are so thin.\n\nIn several places authors use a(b) to mean \"a (or b)\" which is usually clear from context but might be confusing for some readers. \n\nWhy in Figure 1 both green (labeled source) and pink (all labeled) go to \\tau? Isn't green included in pink?\n\nAlso, see questions below.\n\nQUESTIONS:\n1. Diffusion sampling might be time-consuming. How does the runtime/training time compares to other methods? Or at least the deterministic backbone? (+10%? 2x? 10x?)\n\n2. Why is DUC not shown in Figure 3a?\n\n3. I wanted to ask, given how thin margins are in Table 3, and given that “your” best deterministic strategy seems to beat most of SotA deterministic baselines, whether improvement should be attributed mostly to the proposed “diffusion uncertainty estimation” or a more powerful backbone. But then I realized that I cannot find number 93.5 from Table 3 in other tables. Where can I find corresponding baseline performances?\n\n4. On ADA method beats competing methods in low-data regime, but on SFADA it does not - any thoughts on why this is the case?\n\n5. It might be possible to obtain nonparametric estimates of entropy from sampled predictions.\n\nLIMITATIONS:\nAuthors did not explicitly address limitations and societal impact in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper concerns Active Domain Adaption leveraging a diffusion-based adversarial probabilistic model with T-statistics for uncertainty quantification. The paper combines many existing ideas such as variational auto-encoder, diffusion model, knowledge distillation, and T-statistics. Then it proposes a new framework to improve domain adaptation performance with active learning. It shows extensive experiments with superior performances compared to various benchmark methods.\n\nSTRENGTHS:\n1. The paper shows a good combination of existing methods and proposes a novel framework of active domain adaptation.\n2. The conducted experiments are pretty extensive and provide useful information as much as possible. It shows superior performance compared to other existing methods.\n\nSo I value the contribution of 1) proposing a new framework to combine many ideas, 2) extensive experiments.\n\nWEAKNESSES:\nThe proposed framework involves many existing ideas, so combining them is somewhat complex from another perspective. I have a generic question if this combination is ideal. I also have some concerns about the sensitivity of parameters, computational costs, and active learning. Please see Questions for more details.\n\nMinor Comments:\n- Algorithm 1 in Appendix (L44-45): Eq. (??) is missing\n\nQUESTIONS:\n1. The paper shows a good combination of many existing methods. Because the framework also leverages the knowledge-distillation step, I wondered if the framework can be generalized to a common domain adaptation scenario WITHOUT active learning. If the framework works well without active learning, i.e., producing pseudo-labels, I think the proposed combination would be more valuable. Otherwise, we may still conclude that most of the contribution to performance improvement might come from the active learning step, even if it shows a better performance.\n2. Figure 3-(b) for the parameter sensitivity shows that ADA is highly sensitive (non-convex or non-monotone) to the choice of parameters. This implies that the combined loss might conflict with each other when the source data is available. This result somewhat surprises me because more information makes the proposed framework more unstable. Could the authors reasonably explain the root of instability under a source-available scenario ADA?\n3. Because it uses a diffusion classifier, I wondered how the computational cost would differ from other methods. Similar, cheaper, or expensive?\n4. In active learning, diversified selection is critical. I was wondering how p-value with T-statistics acquisition can mitigate the redundant selections. For example, I imagine produced T-statistics would be very similar if two images are similar (or the same), even if it's with a diffusion classifier. Then how can we avoid the redundant selection, or how can we get a diversified selection?\n\nLIMITATIONS:\nThe authors provided the source code in the supplementary material, but I didn't have a chance to look at the details yet.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}