SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For SFADA, we implement compared baseline algorithms on our DAPM baseline with following details:
Random. We abandon the use of any selection strategy and randomly select samples from the unlabeled target dataset  $\mathcal{T}_u$  for annotation.
BvSB. We compute the best-versus-second-best score based on the output of the deterministic classifier for each unlabeled sample and select  $b$  samples with the lowest scores for annotation.
Entropy. We use the conditional entropy based on the output of the deterministic classifier to measure the prediction confidence. And samples with highest entropy values are selected for annotation.
CoreSet. We regard the sample selection in each round as a core-set cover problem and solve it with the code at https://github.com/ozansener/active_learning_coreset.
BADGE. We obtain the gradient vectors based on the pseudo labels generated by the deterministic classifier and utilize K-Means++ on the gradient vectors for diverse sampling. The algorithm is implemented based on the repository at https://github.com/JordanAsh/badge.
ELPT. We cite the results on Office-31 and Office-Home dataset from the original paper [47]. For VisDA, we run this method and report the results on ResNet-50 backbone based on the official code at https://github.com/TL-UESTC/ELPT.
D.1 Accuracies of Confident Predictions under Different Thresholds
Fig. 7 illustrates the accuracies of the teacher model's predictions for confident samples across different threshold settings. At each training step, the teacher model is updated with the student model, and we have computed the accuracy of the current batch and presented it as a curve. As expected, increasing the threshold leads to an increase in the teacher model's accuracy. However, when the threshold is relatively high, only a small number of samples are considered confident at the beginning, leading to a higher accuracy initially and a subsequent drop. It is worth noting that the teacher model provided relatively reliable predictions at a threshold value of 0.9. Raising the threshold further would result in too few confident samples, making 0.9 a more appropriate choice.
(a) ADA
(b) SFADA
Figure 7: Accuracies of confident predictions made by the teacher model under different threshold values on VisDA dataset.
D.2 Prediction Visualization of Different Classifiers
To investigate the contrasting behaviors of deterministic and diffusive classifiers, we randomly select two samples on task  $\mathrm{Ar} \rightarrow \mathrm{Cl}$ , and visualize the predictions made by the deterministic classifier and the diffusion classifier ( $N = 100$ ). As shown in Fig. 8a, the deterministic classifier exhibits high confidence in predicting a refrigerator as a mug. This verifies the overconfident issue in traditional softmax-based deterministic model, making it challenging for the active learning methods to detect the error and select such hard samples. In contrast, the diffusion classifier produces an uncertain prediction, indicating confusion in its output with a p-value of 0.873. As depicted in Fig. 8b, the deterministic classifier displays high uncertainty and misclassified the sample, whereas the diffusion classifier correctly classifies the sample with a p-value of 0.024, which saves budget and resources that would have been wasted on correcting the misclassification.
(a) Refrigerator
p-value: 0.024
(b) Drill
Figure 8: Visualization of predictions made by different classifiers for 2 randomly picked samples from refrigerator and drill, respectively. Red histograms represent the prediction of the deterministic classifier and green boxes denote the predictions of the diffusion classifier.
D.3 Effect of the Modeling of the Latent Feature Distribution
In comparison to CARD [19] which utilizes a deterministic feature extraction network and a diffusion classifier to evaluate uncertainty based on the original image, our method employs an additional VAE to model data uncertainty in a low-dimensional latent space, and the diffusion classifier is based on latent variables. To investigate the benefits of this improvement for ADA tasks, we implement CARD in the ADA task and report the results of it on Office-31 and VisDA. Specifically, we use a deterministic ResNet-50 network as the feature extractor and train a deterministic classifier on top of it to guide the diffusion classifier. The diffusion classifier takes the original image  $x$  as one of the inputs and uses the same independent two-sample t-test-based criterion for sample selection. We denote this implementation by CARD-TT. As shown in Table 5, DAPM-TT significantly outperforms CARD-TT on both datasets. It is exciting to see that although VAE is mainly designed for modeling the uncertainty of the data generation process, it results in a significant improvement with respect to accuracy. We conjecture that the reason for this improvement is two-fold: firstly, domain shift leads to a significant distribution shift in the image space, and in such case, CARD fails to work as intended [19]. This effect is mitigated to a certain extent in the low-dimensional and less noisy latent space. Secondly, in VAE training, we use the same prior distribution, i.e., the standard Gaussian distribution, for the latent variables of data in both the source and target domains. This design draws the samples in both domains closer to the standard Gaussian distribution, thereby achieving an indirect distribution alignment.
D.4 Qualitative Analysis on Selected Samples
We present in Fig. 9 a list of all the selected samples by our approach to gain insight into which samples are chosen. Intuitively, our method tends to select samples that are challenging for the model, such as those with complex backgrounds or different styles from the other images in the dataset. Labeling these samples can help reduce the ambiguity in the model and enable it to better capture the semantic aspects of the category. Interestingly, we observe that our approach naturally selects a diverse range of sample classes, even though we did not explicitly impose a diversity constraint. Furthermore, we observe that the p-values of the selected samples gradually decrease with each
Table 5: Comparison between probabilistic and deterministic feature extractors, and between different t-test strategies on ADA task (ResNet-50). w/o and w/ are short for without and with, respectively.
Category	Method	Office-Home	VisDA
A→D	A→W	D→A	D→W	W→A	W→D	Avg	Synthetic→Real
w/o adaptaion stage	CARD-TT	95.1	94.2	78.5	98.7	78.2	99.1	90.6	84.6
DAPM-TT	96.1	95.9	79.5	98.7	79.2	99.1	91.4	86.3
w/ adaptaion stage	DAPM-TT*	96.8	97.8	83.3	99.8	81.7	100	93.2	88.6
DAPM-TT	96.8	98.6	82.3	99.8	83.3	100	93.5	89.1
Table 6: Accuracy (%) of DAPM-TT combined with semi-supervised domain adaptation techniques on Office-Home with  $5\%$  annotation budget (ResNet-50).
Method	Ar→Cl	Ar→Pr	Al→Rw	Cl→Ar	Cl→Pr	Cl→Rw	Pr→Ar	Pr→Cl	Pr→Rw	Rw→Ar	Rw→Cl	Rw→Pr	Avg
DAPM-TT (w/ CDAN)	64.2	85.4	85.7	69.2	84.2	83.5	69.1	63.4	86.0	77.2	68.4	88.6	77.1
DAPM-TT (w/ MME)	65.1	85.5	84.9	72.5	84.6	83.7	72.4	64.7	86.3	77.9	70.3	88.8	78.1
training round, and by the 5th round, the lowest p-value is 0.518. This indicates that selecting approximately  $5\%$  of the samples is sufficient to mitigate much of the ambiguity in the model.
D.5 Comparison between Different T-test Strategies
Based on the scores of the two most probable classes predicted by the diffusion classifier, we can use either paired two-sample t-test or independent two-sample t-test for selection, which correspond to different assumptions for the generation of predictions. The former assumes that the scores of different classes are generated in pairs, while the latter assumes that they are generated independently. For a paired t-test, the t-value of a target sample  $x^t$  is calculated as follows:
$$
t = \left(\bar {d} - \mu_ {d}\right) / \left(s _ {d} / \sqrt {N}\right), \tag {26}
$$
where  $\bar{d} = \frac{1}{N}\sum d_{i}$  is the mean of sample differences  $d_{i} = \widetilde{\pmb{y}}_{i}^{t}[a] - \widetilde{\pmb{y}}_{i}^{t}[b]$ ,  $\mu_{d}$  is the difference of the null hypothesis (usually set as 0), and  $s_d = \sqrt{\sum(d_i - \bar{d})^2 / N - 1}$  is the standard deviation of the sample difference.
We denote our method with paired t-test-based criterion by DAPM-TT*, and report the results on Office-31 and VisDA in Table 5. Empirically, we find that independent two-sample t-test yields superior performance on both datasets. We conjecture the reason might be that the independent two-sample t-test considers the internal variance of each group of samples. Therefore, when evaluating uncertainty, it considers an additional dimension compared to the paired t-test. In this work, we adopt independent two-sample t-test for all experiments.
D.6 Combine with other Semi-Supervised UDA methods
It is worth noting that our formulation provides certain flexibility for the implementation of the training objectives, which showcases the modular nature of the framework. For instance, we use a teacher-student framework to implement the second term in Eq. (12) and adopt conditional adversarial learning as the main adaptation technique, which may limit the baseline DA performance and then affect the performance after active learning. For ADA, he DA component and AL component can be decoupled. We have tried to incorporate a SSDA method MME [62] to implement the adaptation stage, which is also practiced by previous ADA works, e.g., [11, 4]. Specifically, the adversarial learning loss (i.e.,  $\mathcal{L}_{adv}$ ) is replaced by the minimax entropy loss, and other losses remains unchanged. The results on Office-Home dataset can be found in Table 6, which shows the SSDA component further boosts the performance of our method.
D.7 Benefits of Data-level Uncertainty
We conducted additional experiments ablating the VAE component, which can be simply implemented by replacing the variational encoder with a deterministic feature encoder. Through experiments, we found that the performance on VisDA-2017 is worse than modeling both the data-level and prediction-layer uncertainty, as shown in Table 7.
Table 7: Comparison between the performance of DAPM-TT w/ and w/o data-level uncertainty on VisDA-2017 dataset with  $5\%$  annotation budget (Resnet-50).
Method	Synthetic → Real
DAPM-TT (w/ VAE)	89.1
DAPM-TT (w/o VAE)	88.3
E Further Discussion
Related Uncertainty Estimation Works. In machine learning, there are primarily two kinds of uncertainties that are studied, i.e., epistemic uncertainty which arises due to a lack of knowledge or data and can be reduced with more data or improved models, and aleatoric uncertainty that stems from the inherent randomness in the data [51]. To model these uncertainties, the community has proposed many Bayesian deep learning methods. From a Bayesian perspective, these two uncertainties can be modeled by the posterior of model parameters  $W$  and outputs  $y$ , respectively, using the following formulation:
$$
P (\boldsymbol {y} \mid \boldsymbol {x}, \mathcal {D}) = \int_ {\text {a l e a t o r i c u n c e r t a i n t y}} \underbrace {P (\boldsymbol {y} \mid \boldsymbol {x} , W)} _ {\text {e p i s t e m i c u n c e r t a i n t y}} \underbrace {P (W \mid \mathcal {D})} _ {\text {d} W.} \tag {27}
$$
The family of Bayesian neural networks (BNNs) [48, 49, 63] is specifically designed to capture epistemic uncertainty by assuming a probability distribution over the network parameters. This involves estimating the posterior distribution over the parameters of the neural network given the observed data. However, due to the intractable form of the posterior, BNNs are often trained using appropriate approximations like Markov Chain Monte Carlo (MCMC) or Variational Inference (VI). Another approximation for BNNs is Monte Carlo Dropout [64], which assumes a Bernoulli distribution over network parameters. During inference, the output of the network is averaged over multiple stochastic forward passes with dropout enabled, resulting in a distribution over the predictions.
Evidential deep learning (EDL) [50, 20] is another method that models uncertainty associated with the output of the model based on evidential theory. In EDL, this is typically achieved by using a distributional output based on the theory of Subjective Logic [65], such as a Dirichlet distribution over class probabilities for classification tasks, instead of a point estimate.
For non-Bayesian methods, ensemble-based methods [66, 67] have been proposed to model predictive uncertainty by combining multiple deterministic neural networks with different initializations. However, all these methods are designed to capture either epistemic uncertainty or aleatoric uncertainty alone by modeling probability distributions over the model parameters and outputs, respectively. Moreover, they still impose a restricted form of distributions, such as Gaussian or Dirichlet, which limits their applicability in practice.
To capture both sources of uncertainties in a single model, Kendall et al. [51] propose modeling aleatoric uncertainty in the model outputs beyond model parameters by predicting the noise term for the output variable of each sample as part of the model output. However, the form of the noise is still assumed to be Gaussian.
Recently, Han [19] proposed modeling the implicit output distribution by leveraging the generative capability of the diffusion model. However, they only model aleatoric uncertainty in their formulation since the model they use is still a deterministic neural network, and the proposed method, CARD, only enables in-distribution generalization.
In addition to modeling aleatoric uncertainty with the diffusion classifier, we also incorporate a VAE to model the underlying data generation process. The VAE learns a probabilistic distribution over the latent space, which represents the model's uncertainty about the true underlying distribution of the data, given the limited amount of training data. As more data is provided during training, the learned distribution should converge to the true underlying distribution, reducing epistemic uncertainty. Therefore, our DAPM also offers a way to measure epistemic uncertainty. Furthermore, our diffusion classifier is conditioned on the latent variables in  $\mathcal{Z}$  rather than the ones in the original image space  $\mathcal{X}$ . We argue that the latent space contains less noise and is more suitable for cross-domain tasks.
The choice of Diffusion-based model for probabilistic modeling. Diffusion models have been shown to be equivalent to reversing stochastic differential equations (SDEs). Running the reverse process along the SDE trajectory to recover the original data distribution provides a principled way to recover the data distribution. Besides, compared to other generative models that complete generation in a single step, the multi-step noise schedule smooths the posterior and improves the exploration of the probability space, which will also result in more stable training and generation. In addition, the forward-reverse diffusion process allows effective modeling of high-dimensional complex distributions without parametric assumptions. Actually, the correlation between the covariates and the prediction variable is assumed to be high-dimensional and complex as well. Therefore, we regard diffusion models as a better choice to model the underlying predictive distribution.
F Limitations and Broader Impacts
Limitations. Our work presents a way to recover the predictive distribution of deep models by combining the power of diffusion models and variational autoencoders. However, like any research, our study may have some limitations that should be acknowledged. Firstly, the task we focus on is limited to image classification in this work. In our future study, we may extend the scope of research to other areas like image segmentation and object detection, etc. Secondly, probabilistic models are often more difficult to interpret compared to deterministic models. It is important to study insights in future research to help interpret probabilistic models.
Broader Impacts. Indeed, the impact of active domain adaptation is significant, especially in scenarios where labeled data is scarce in the target domain. The ability to adapt to new domains with limited labeled data can potentially reduce the time and cost required to gather labeled data for each specific task, thus making the deployment of machine learning models more accessible and cost-effective. This can also facilitate the development of more robust and generalizable machine learning models that can be used across multiple domains, which is particularly important for applications that operate in dynamic and diverse environments. Overall, our work contributes to advancing the field of machine learning and promoting the development of more efficient and adaptive technologies.
(a) Round 1
(b) Round 2
(c) Round 3
(d) Round 4
(a) Round 5
Figure 9: All selected samples on task  $\mathrm{Cl} \rightarrow \mathrm{Ar}$  (ADA). For samples in each round, the priority (p-value) gradually decreases from top left to bottom right.