SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1 PyTorch-style pseudocode for MixVal.
# x: A batch of real target images with shuffled order.  
# lam: The mix ratio, a fixed scalar value between 0.5 and 1.0.  
# net: A trained UDA model in the evaluation mode.  
# model_list: A list containing candidate UDA models.  
# Calculate ICE scores for a mini-batch.  
def ice_score(x, lam, net):  
    # Random batch index.  
    randidx = torch.randomperm(x.shape[0])  
    inputs_a = x  
    inputs_b = x[rand IDX]  
    # Obtain model predictions and hard pseudo labels.  
    pred_a = net(input_a)  
    pl_a = pred_a.max(dim=1)[1]  
    pl_b = pl_a[rand IDX]  
    # Intra-cluster mixup.  
    same IDX = (pl_a == pl_b).nonzero(as_tuple=True)[0]  
    # Inter-cluster mixup.  
    diff IDX = (pl_a != pl_b).nonzero(as_tuple=True)[0]  
    # Mixup with images and hard pseudo labels.  
    mix Inputs = lam * inputs_a + (1 - lam) * inputs_b  
if lam > 0.5:  
    mix_labels = pl_a  
else:  
    mix_labels = pl_b  
# Obtain predictions for the mixed samples.  
mix_pred = net(mix Inputs)  
mix_pred_labels = mix_pred.max(dim=1)[1]  
# Calculate ICE scores for two-dimensional probing.  
ice_same = torch.sum(mix_pred_labels[same IDX] / same IDX.shape[0]  
    == mix_labels[same IDX]) / same IDX.shape[0]  
ice_diff = torch.sum(mix_pred_labels[diff IDX] / 
        == mix_labels[diff IDX]) / diff IDX.shape[0]  
return ice_same, ice_diff  
# Perform model selection based on ICE scores.  
def mixVal(model_list, x, lam):  
    # Calculate ICE scores for all candidate models.  
ice_same_list = []  
ice_diff_list = []  
for net in model_list:  
    ice_same, ice_diff = ice_score(x, lam, net)  
    ice_same_list.append(ice_same)  
    ice_diff_list.append(ice_diff)  
# Calculate the average rank of two types of ICE scores.  
ice_same_list = torch.tensor(ice_same_list)  
ice_diff_list = torch.tensor(ice_diff_list)  
ice_same_rank = torch.argsort(torch.argsort(ice_same_list))  
ice_diff_rank = torch.argsort(torch.argsort(ice_diff_list))  
average_rank = (ice_same_rank + ice_diff_rank) / 2  
# Choose the model with the highest average rank.  
return model_list[torch.argmax(average_rank)]
In our main experiments, we follow the methodology of previous model selection studies [18, 29]. For simplicity, we tune a single hyperparameter for various UDA methods. In the case of two-hyperparameter validation experiments, as in MCC [25] and MDD [22], we also tune the bottleneck dimension, choosing from the candidate values  $\{256, 512, 1024, 2048\}$ . This variation is due to observations that the bottleneck dimension varies across different datasets in the official code of these UDA methods. For the validation of semantic segmentation tasks, we also consider two hyperparameters with the training iteration as an additional hyperparameter selected from the set  $\{10,000,12,000,14,000,16,000,18,000,20,000,22,000,24,000,26,000,28,000,30,000\}$ , following SND [29]. Detailed hyperparameter settings are available in Table 11.
Table 11: Summary of the considered UDA methods and their corresponding hyperparameters.
UDA method	UDA Type	Hyperparameter	Search Space	Default Value
ATDOC [27]	closed-set self-training	loss coefficient λ	{0.02, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0}	0.2
BNM [24]	closed-set output regularization	loss coefficient λ	{0.02, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0}	1.0
CDAN [9]	closed-set feature alignment	loss coefficient λ	{0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0}	1.0
MCC [25]	closed-set output regularization	temperature T	{1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0}	2.5
MDD [22]	closed-set output alignment	margin factor γ	{0.5, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0}	4.0
SAFN [23]	closed/partial-set feature regularization	loss coefficient λ	{0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2}	0.05
PADA [15]	partial-set feature alignment	loss coefficient λ	{0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0}	1.0
DANCE [41]	open-partial-set self-supervision	loss coefficient η	{0.02, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0}	0.05
SHOT [20]	source-free hypothesis transfer	loss coefficient β	{0.03, 0.05, 0.1, 0.3, 0.5, 1.0, 3.0}	0.3
DMRL [50]	closed-set mixup training	loss coefficient λ	{0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0}	2.0
AdaptSeg [11]	closed-set output alignment	loss coefficient λ	{0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03}	0.0002
AdvEnt [12]	closed-set output alignment	loss coefficient λ	{0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03}	0.001
C Analysis of Intra-Cluster and Inter-Cluster Probing
In MixVal, we use mixed samples to directly probe the intra-cluster and inter-cluster structures within the target representations learned by a UDA model. This strategy shares similar spirits with the well-known Linear Discriminant Analysis (LDA) [66], which optimizes an objective focused on minimizing intra-cluster variance while maximizing inter-cluster variance. Drawing inspiration from the LDA optimization, we explore a straightforward validation baseline that employs an LDA-like metric. In this context, let's denote the number of target samples as  $N$ , the number of categories as  $K$ , and the feature of a sample encoded by a UDA model as  $f$ . The formulation for the LDA score is presented in Equation 1 below.
$$
\bar {f} = \frac {1}{N} \sum_ {m = 1} ^ {K} \sum_ {i = 1} ^ {N _ {m}} f _ {i} ^ {m} \quad \bar {f} _ {m} = \frac {1}{N _ {m}} \sum_ {i = 1} ^ {N _ {m}} f _ {i} ^ {m} \quad N = \sum_ {m = 1} ^ {K} N _ {m} \tag {1}
$$
$$
D _ {\mathrm {i n t r a}} = \frac {1}{N} \sum_ {m = 1} ^ {K} \sum_ {i = 1} ^ {N _ {m}} \left\| f _ {i} ^ {m} - \bar {f} _ {m} \right\| _ {2} ^ {2} D _ {\mathrm {i n t e r}} = \frac {1}{K} \sum_ {m = 1} ^ {K} \left\| \bar {f} _ {m} - \bar {f} \right\| _ {2} ^ {2} \quad \mathrm {L D A} = \frac {D _ {\mathrm {i n t e r}}}{D _ {\mathrm {i n t r a}}}
$$
In our implementation, we utilize the predicted logits as features and compute variance using the  $L_{2}$  distance. We assess the LDA baseline against all target-only validation methods in both closed-
set UDA (Table 12) and partial-set UDA (Table 13). In both UDA scenarios, LDA outperforms existing validation methods in terms of average validation accuracy. This highlights the advantage of considering two effective assumptions, in contrast to the single assumption used in SND and Entropy. It's worth noting that while LDA performs well, it still falls short of MixVal's performance, underscoring the benefits of MixVal's direct probing approach using mixed samples compared to the indirect probing via measurement of raw predictions used by SND, Entropy, and LDA.
Table 12: Validation accuracy (%) of closed-set UDA on Office-Home (Home). bold: Best value.
Method	ATDOC [27]	BNM [24]	CDAN [9]
→Ar	→Cl	→Pr	→Re	avg	→Ar	→Cl	→Pr	→Re	avg	→Ar	→Cl	→Pr	→Re	avg
SourceVal	66.63	52.54	78.57	76.61	68.59	62.44	50.74	77.53	74.76	66.37	55.00	42.65	69.50	68.81	58.99
Entropy [42]	63.67	55.83	76.54	78.36	68.60	66.28	54.49	74.15	77.64	68.14	67.66	57.56	76.37	77.45	69.76
InfoMax [28]	63.67	55.63	77.61	78.36	68.82	66.28	54.49	74.15	77.64	68.14	67.66	57.56	76.37	77.45	69.76
SND [29]	63.67	55.63	76.54	77.54	68.34	66.28	54.49	74.15	77.64	68.14	67.94	57.56	76.96	77.68	70.04
Corr-C [43]	63.51	50.39	73.89	73.88	65.42	58.10	45.37	68.97	70.59	60.76	53.84	41.21	64.96	67.65	56.91
LDA	63.67	55.63	76.54	77.54	68.34	66.28	54.49	74.15	77.64	68.14	67.66	57.56	76.37	77.45	69.76
MixVal	66.47	56.87	78.14	79.20	70.17	67.36	56.18	76.10	78.12	69.44	67.71	57.78	76.89	77.76	70.03
Worst	62.89	50.39	73.89	73.88	65.26	58.10	45.37	68.96	70.59	60.75	53.80	41.21	64.78	67.65	56.86
Best	68.97	58.35	80.27	80.58	72.04	68.93	57.51	78.43	79.57	71.11	68.19	57.90	77.44	78.19	70.43
Method	MCC [25]	MDD [22]	SAFN [23]
→Ar	→Cl	→Pr	→Re	avg	→Ar	→Cl	→Pr	→Re	avg	→Ar	→Cl	→Pr	→Re	avg
SourceVal	66.57	56.53	79.55	80.90	70.89	62.53	54.43	75.27	75.55	66.94	63.54	51.34	73.66	74.54	65.77
Entropy [42]	69.29	59.33	80.63	80.96	72.55	66.54	57.63	77.27	77.45	69.72	59.85	46.41	72.51	73.18	62.99
InfoMax [28]	66.58	58.48	79.12	80.81	71.25	66.54	57.74	77.27	77.45	69.75	64.56	49.71	73.77	73.18	65.31
SND [29]	69.05	55.61	79.72	79.10	70.87	51.34	38.01	77.61	68.46	58.86	57.90	46.41	67.04	68.18	59.88
Corr-C [43]	69.05	55.61	79.72	79.10	70.87	47.79	31.69	63.40	60.63	50.88	62.66	46.41	68.83	68.18	61.52
LDA	70.46	59.60	80.60	80.25	72.73	66.38	57.63	77.61	77.45	69.77	64.56	49.91	72.51	71.55	64.63
MixVal	69.79	59.24	80.47	80.74	72.56	65.73	58.01	77.36	76.91	69.50	65.98	53.14	74.76	75.40	67.32
Worst	62.72	54.63	76.19	78.19	67.93	47.79	31.69	63.40	60.63	50.88	57.90	46.41	67.04	68.18	59.88
Best	70.68	59.95	80.93	81.02	73.14	66.75	58.36	77.61	77.45	70.04	66.59	53.14	74.90	75.57	67.55
Table 13: Validation accuracy (%) of partial-set UDA on Office-Home.
Method	PADA [15]	avg	SAFN [23]	avg	Home AVG
→ Ar	→ Cl	→ Pr	→ Re	→ Ar	→ Cl	→ Pr	→ Re
SourceVal	57.21	41.90	64.48	71.89	58.87	66.82	54.71	74.41	76.48	68.11	63.49
Entropy [42]	60.08	46.51	53.16	62.47	55.56	71.75	55.62	76.36	76.59	70.08	62.82
InfoMax [28]	60.08	51.40	60.20	66.67	59.59	63.67	51.74	69.64	73.62	64.67	62.13
SND [29]	67.80	50.71	59.46	67.13	61.27	71.75	51.74	76.36	78.36	69.55	65.41
Corr-C [43]	61.34	45.65	54.90	62.25	56.04	71.23	55.70	76.94	79.13	70.75	63.40
LDA	64.52	46.51	69.47	72.67	63.29	71.75	54.39	73.93	77.10	69.29	66.29
MixVal	67.68	51.01	72.94	78.64	67.57	71.70	57.91	77.08	78.94	71.41	69.49
Worst	56.29	39.76	50.49	59.31	51.46	62.48	49.91	68.50	73.62	63.63	57.55
Best	69.33	55.86	74.55	79.59	69.83	73.37	58.09	77.35	79.33	72.03	70.93
D Full Validation Results
For classification tasks in our evaluation, we utilize the HOS score  $(\%)$  [62, 63] for open-partial-set UDA and accuracy  $(\%)$  for all other UDA tasks. In the case of segmentation tasks, we measure mIoU  $(\%)$  [11, 12]. To accommodate space constraints in this main text, we present the averaged validation results for UDA tasks that share the same target domain. For example,  $\rightarrow$  Ar' represents the averaged results of three tasks on the Office-Home benchmark, namely  $\mathrm{Cl}\rightarrow \mathrm{Ar}'$ ,  $\mathrm{Pr}\rightarrow \mathrm{Ar}'$ , and  $\mathrm{Re}\rightarrow \mathrm{Ar}'$ . Additionally, please note that the avg' row signifies the average of the rows preceding it within each UDA method, while the AVG' row represents the average results of avg' rows across all considered UDA methods. For more detailed validation results for each specific UDA task, please refer to the corresponding tables, from Table 14 to Table 29.
Table 14: Accuracy (%) of ATDOC [27], a closed-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	51.43	77.31	78.17	66.87	74.36	75.60	61.85	48.06	76.06	71.16	58.14	84.05	68.59
IWCV [30]	55.88	76.57	78.88	66.25	74.50	78.33	65.60	48.06	80.58	72.06	58.14	83.87	69.89
DEV [18]	51.43	76.55	78.88	66.25	74.36	77.67	64.77	51.29	81.62	71.16	59.98	82.43	69.70
RV [31]	56.38	76.12	80.01	66.25	76.80	78.33	67.82	55.62	80.58	71.98	56.40	83.87	70.85
Entropy [42]	55.88	74.14	78.88	59.25	74.52	77.67	64.19	54.39	78.54	67.57	57.23	80.96	68.60
InfoMax [28]	55.88	74.14	78.88	59.25	77.74	77.67	64.19	54.39	78.54	67.57	56.61	80.96	68.82
SND [29]	55.88	74.14	78.88	59.25	74.52	75.21	64.19	54.39	78.54	67.57	56.61	80.96	68.34
Corr-C [43]	51.41	72.00	76.04	59.37	69.36	69.54	61.85	48.04	76.06	69.30	51.71	80.31	65.42
MixVal	57.19	74.82	79.13	64.19	78.10	77.90	65.63	55.28	80.57	69.60	58.15	81.49	70.17
Worst	51.41	72.00	76.04	59.25	69.36	69.54	61.85	48.04	76.06	67.57	51.71	80.31	65.26
Best	58.01	77.31	81.04	66.91	79.48	78.52	67.94	57.07	82.17	72.06	59.98	84.03	72.04
Table 15: Accuracy (%) of BNM [24], a closed-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	56.93	77.00	77.74	57.64	73.33	69.36	56.45	42.38	77.19	73.22	52.90	82.26	66.37
IWCV [30]	46.46	77.00	79.30	63.86	61.34	62.54	63.95	42.38	78.01	71.86	55.65	83.92	65.52
DEV [18]	57.75	71.62	79.30	57.64	67.90	75.46	66.21	54.04	78.01	73.42	57.37	82.25	68.41
RV [31]	58.67	77.00	79.30	65.68	73.33	75.46	65.64	52.05	81.25	73.42	59.54	83.92	70.44
Entropy [42]	53.40	67.04	78.04	63.41	71.44	73.93	63.58	52.69	80.95	71.86	57.37	83.96	68.14
InfoMax [28]	53.40	67.04	78.04	63.41	71.44	73.93	63.58	52.69	80.95	71.86	57.37	83.96	68.14
SND [29]	53.40	67.04	78.04	63.41	71.44	73.93	63.58	52.69	80.95	71.86	57.37	83.96	68.14
Corr-C [43]	46.46	67.06	74.82	49.73	61.34	62.54	56.45	42.38	74.41	68.11	47.26	78.51	60.76
MixVal	56.30	72.90	78.75	64.66	71.44	74.52	64.32	54.04	81.09	73.09	58.19	83.96	69.44
Worst	46.46	67.04	74.82	49.73	61.34	62.54	56.45	42.38	74.41	68.11	47.26	78.51	60.75
Best	58.67	77.00	80.61	67.16	74.16	76.75	66.21	54.04	81.36	73.42	59.82	84.12	71.11
Table 16: Accuracy (%) of CDAN [9], a closed-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	43.41	62.51	75.51	43.96	61.59	57.70	53.75	37.53	73.22	67.28	47.01	84.39	58.99
IWCV [30]	43.18	62.51	77.81	44.71	54.61	56.14	65.14	37.53	81.85	74.08	43.02	84.39	60.41
DEV [18]	57.16	71.75	77.81	62.46	55.64	71.08	65.14	56.54	81.85	74.08	57.43	78.89	67.49
RV [31]	57.16	71.75	77.78	63.62	72.92	73.40	65.14	54.50	81.85	74.21	58.56	83.37	69.52
Entropy [42]	57.55	72.43	77.74	63.62	72.92	73.40	65.27	56.66	81.20	74.08	58.47	83.76	69.76
InfoMax [28]	57.55	72.43	77.74	63.62	72.92	73.40	65.27	56.66	81.20	74.08	58.47	83.76	69.76
SND [29]	57.55	72.43	77.78	64.61	73.73	73.40	65.14	56.66	81.85	74.08	58.47	84.73	70.04
Corr-C [43]	43.14	63.05	73.61	43.96	54.58	56.12	51.75	37.50	73.22	65.80	43.00	77.25	56.91
MixVal	57.55	73.71	77.75	63.95	73.19	74.06	65.23	56.59	81.47	73.95	59.19	83.76	70.03
Worst	43.14	62.51	73.61	43.96	54.58	56.12	51.63	37.50	73.22	65.80	43.00	77.25	56.86
Best	57.55	73.71	78.33	64.61	73.89	74.39	65.76	56.66	81.85	74.21	59.50	84.73	70.43
Table 17: Accuracy (%) of MCC [25], a closed-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	57.23	78.19	81.75	60.65	76.50	78.79	64.15	53.15	82.17	74.91	59.20	83.96	70.89
IWCV [30]	60.02	78.15	81.34	68.73	78.51	77.85	64.15	57.85	81.04	73.18	58.92	84.46	72.02
DEV [18]	57.16	78.15	81.34	69.10	73.01	76.80	64.15	57.85	82.17	73.18	59.20	84.46	71.38
RV [31]	59.34	78.53	80.70	69.10	77.83	78.22	67.20	57.85	82.24	74.91	59.20	85.54	72.56
Entropy [42]	59.31	78.53	81.59	66.87	77.83	78.79	67.20	57.85	82.51	73.79	60.82	85.54	72.55
InfoMax [28]	60.02	74.66	81.75	64.98	78.24	78.49	64.15	54.52	82.19	70.62	60.89	84.46	71.25
SND [29]	53.56	77.43	79.46	67.28	76.48	76.80	65.06	54.34	81.04	74.82	58.92	85.24	70.87
Corr-C [43]	53.56	77.43	79.46	67.28	76.48	76.80	65.06	54.34	81.04	74.82	58.92	85.24	70.87
MixVal	59.08	77.81	81.61	68.40	78.45	78.40	67.56	57.65	82.21	73.41	61.00	85.15	72.56
Worst	53.56	73.44	79.25	60.65	73.01	75.76	59.74	53.15	79.55	67.78	57.18	82.11	67.93
Best	60.02	78.53	81.75	69.22	78.51	78.79	67.90	58.49	82.51	74.91	61.35	85.74	73.14
Table 18: Accuracy (%) of MDD [22], a closed-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	54.85	73.35	77.05	58.76	69.95	72.23	60.03	51.02	77.36	68.81	57.42	82.50	66.94
IWCV [30]	56.40	69.52	76.59	58.76	67.40	69.43	61.89	56.43	76.82	71.94	56.68	84.43	67.19
DEV [18]	57.71	75.42	77.05	58.76	72.99	70.51	63.95	56.43	80.26	70.54	56.68	82.14	68.54
RV [31]	58.05	75.42	76.59	63.54	69.95	73.74	63.95	51.02	80.38	72.23	58.17	84.43	68.96
Entropy [42]	57.73	74.54	78.22	64.07	72.99	73.74	63.95	55.85	80.38	71.61	59.31	84.28	69.72
InfoMax [28]	58.05	74.54	78.22	64.07	72.99	73.74	63.95	55.85	80.38	71.61	59.31	84.28	69.75
SND [29]	58.05	75.42	77.05	44.99	72.99	48.06	37.08	21.60	80.26	71.94	34.39	84.43	58.86
Corr-C [43]	39.08	59.74	69.61	44.99	54.58	48.06	37.08	21.60	64.22	61.31	34.39	75.87	50.88
MixVal	57.39	75.13	78.15	63.45	72.67	72.69	63.91	56.63	79.90	69.82	60.02	84.27	69.50
Worst	39.08	59.74	69.61	44.99	54.58	48.06	37.08	21.60	64.22	61.31	34.39	75.87	50.88
Best	58.05	75.42	78.22	64.07	72.99	73.74	63.95	57.02	80.38	72.23	60.02	84.43	70.04
Table 19: Accuracy (%) of SAFN [23], a closed-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	50.78	69.72	76.06	59.66	70.29	69.86	60.90	46.07	77.71	70.05	57.16	80.96	65.77
IWCV [30]	50.24	69.72	77.28	62.63	67.24	69.86	58.84	49.69	75.72	71.45	57.16	79.97	65.82
DEV [18]	51.07	69.72	76.64	59.66	67.24	71.26	58.84	49.69	75.72	70.95	50.65	76.64	64.84
RV [31]	51.07	71.41	76.64	62.63	68.44	70.44	58.84	44.49	77.71	71.45	54.82	81.46	65.78
Entropy [42]	45.93	69.72	75.49	55.29	67.22	68.35	54.26	43.30	75.69	70.00	49.99	80.60	62.99
InfoMax [28]	50.47	69.72	75.49	62.46	70.98	68.35	61.23	43.30	75.69	70.00	55.37	80.60	65.31
SND [29]	45.93	64.36	70.60	55.29	60.13	62.50	54.26	43.30	71.43	64.15	49.99	76.64	59.88
Corr-C [43]	45.93	69.72	70.60	55.29	60.13	62.50	61.23	43.30	71.43	71.45	49.99	76.64	61.52
MixVal	51.73	72.14	77.08	63.92	70.98	71.55	62.71	50.52	77.56	71.30	57.16	81.17	67.32
Worst	45.93	64.36	70.60	55.29	60.13	62.50	54.26	43.30	71.43	64.15	49.99	76.64	59.88
Best	51.73	72.27	77.30	64.65	70.98	71.70	63.66	50.52	77.71	71.45	57.16	81.46	67.55
Table 20: Accuracy (%) of closed-set UDA methods on Office-31.
Method	ATDOC [27]	avg	BNM [24]	avg	CDAN [9]	avg
A→D	A→W	D→A	W→A	A→D	A→W	D→A	W→A	A→D	A→W	D→A	W→A
SourceVal	88.96	87.80	73.65	71.46	80.47	90.36	89.43	73.13	72.70	81.41	91.16	89.06	66.33	61.46	77.00
IWCV [30]	86.14	86.54	73.65	71.46	79.45	85.54	89.43	73.13	72.70	80.20	69.08	58.74	66.33	61.46	63.90
DEV [18]	86.14	86.54	73.65	71.46	79.45	85.54	89.43	73.13	72.70	80.20	91.16	88.30	66.33	61.46	76.81
RV [31]	89.96	87.23	74.28	75.58	81.76	88.55	89.43	74.90	66.52	79.85	91.16	88.30	76.18	70.36	81.50
Entropy [42]	86.14	87.80	73.87	72.70	80.13	85.54	83.14	71.07	74.26	78.50	91.16	89.06	72.88	70.36	80.87
InfoMax [28]	86.14	87.80	73.87	72.70	80.13	85.54	83.14	71.07	69.97	77.43	91.16	88.30	72.88	70.36	80.68
SND [29]	92.37	87.80	73.87	72.70	81.69	85.54	83.14	74.62	74.26	79.39	92.37	88.55	72.88	70.22	81.01
Corr-C [43]	90.96	84.40	71.88	70.22	79.37	84.34	78.99	67.80	66.52	74.41	67.67	59.62	58.15	58.43	60.97
MixVal	90.96	86.54	73.75	73.47	81.18	86.48	87.00	75.64	74.29	80.85	92.64	89.06	75.08	70.38	81.79
Worst	86.14	84.40	71.88	70.22	78.16	84.34	78.99	67.80	66.52	74.41	67.67	57.11	58.15	58.43	60.34
Best	92.37	87.80	75.04	75.58	82.70	90.36	89.43	75.75	75.29	82.71	92.77	89.06	76.18	70.57	82.15
Table 21: Accuracy (%) of closed-set UDA methods on Office-31.
Method	MCC [25]	avg	MDD [22]	avg	SAFN [23]	avg
A→D	A→W	D→A	W→A	A→D	A→W	D→A	W→A	A→D	A→W	D→A	W→A
SourceVal	90.96	91.07	73.33	72.89	82.06	91.06	86.23	76.68	74.76	82.18	83.73	87.17	68.96	69.44	77.33
IWCV [30]	91.16	88.55	73.33	72.89	81.48	91.16	89.18	76.68	74.30	82.83	86.55	80.38	68.96	69.68	76.39
DEV [18]	89.16	93.08	73.33	72.06	81.91	91.16	89.18	76.68	74.62	82.91	86.55	80.38	68.96	67.45	75.84
RV [31]	89.06	93.08	74.42	73.52	82.52	92.57	86.79	73.94	74.97	82.07	90.83	87.17	68.76	68.62	78.85
Entropy [42]	90.56	93.46	74.83	73.02	82.97	92.57	90.82	78.03	74.58	84.00	91.57	85.66	67.20	69.26	78.42
InfoMax [28]	89.16	88.55	74.16	73.70	81.39	92.57	90.82	78.03	74.97	84.10	91.57	87.42	67.20	69.26	78.86
SND [29]	91.97	93.46	74.83	73.02	83.32	92.17	90.82	78.03	74.97	84.00	89.96	85.66	67.20	69.26	78.02
Corr-C [43]	91.37	93.46	74.83	73.02	83.17	91.57	85.66	73.91	74.58	81.43	86.75	80.38	67.09	69.68	75.98
MixVal	91.77	93.21	74.74	73.44	83.29	91.77	91.74	77.35	74.58	83.86	89.96	86.83	69.91	69.31	79.00
Worst	86.75	87.17	71.18	69.93	78.76	87.35	85.66	73.91	72.20	79.78	83.73	80.38	67.09	67.45	74.66
Best	91.97	93.46	74.83	74.01	83.57	92.57	92.20	78.03	75.01	84.45	91.57	87.42	70.43	69.68	79.78
Table 22: Accuracy (%) of CDAN [9], a closed-set UDA method, on DomainNet.
Method	C → S	P → C	P → R	R → C	R → P	R → S	S → P	avg
Entropy [42]	58.04	64.78	74.42	69.39	68.65	60.63	62.94	65.55
InfoMax [28]	58.04	64.78	74.42	69.39	68.65	60.63	62.94	65.55
SND [29]	58.04	64.78	74.42	69.39	68.65	60.63	60.70	65.23
Corr-C [43]	58.04	57.73	74.42	56.98	65.07	51.23	60.70	60.60
MixVal	58.04	64.78	74.42	69.39	68.65	60.63	62.94	65.55
Worst	51.59	57.73	73.44	56.98	63.06	51.23	58.46	58.93
Best	58.04	64.78	74.44	69.39	68.65	60.63	62.94	65.55
Table 23: Accuracy (%) of BNM [24], a closed-set UDA method, on DomainNet.
Method	C → S	P → C	P → R	R → C	R → P	R → S	S → P	avg
Entropy [42]	56.42	61.57	74.31	65.15	65.15	40.95	63.42	61.00
InfoMax [28]	56.42	68.95	74.31	65.15	65.15	54.93	63.42	64.05
SND [29]	43.78	61.57	74.31	51.55	54.40	40.95	54.59	54.45
Corr-C [43]	43.78	60.03	77.62	59.47	67.19	40.95	59.64	58.38
MixVal	58.48	69.63	78.68	66.05	67.59	58.50	65.20	66.30
Worst	43.78	60.03	74.31	51.55	54.40	40.95	54.59	54.23
Best	58.48	69.63	78.68	66.10	67.79	58.50	65.20	66.34
Table 24: Accuracy (%) of ATDOC [27], a closed-set UDA method, on DomainNet.
Method	C → S	P → C	P → R	R → C	R → P	R → S	S → P	avg
Entropy [42]	46.43	65.98	79.60	61.52	64.24	57.92	59.46	62.16
InfoMax [28]	46.43	65.98	79.60	61.52	64.24	57.92	59.46	62.16
SND [29]	46.43	65.98	79.60	61.52	64.24	47.58	59.46	60.69
Corr-C [43]	54.71	60.63	74.42	59.33	64.58	52.66	59.95	60.90
MixVal	62.11	69.64	79.60	68.24	69.79	61.35	67.10	68.26
Worst	46.43	60.63	74.42	59.33	64.24	47.58	59.46	58.87
Best	63.12	71.14	80.38	69.45	69.79	61.35	67.10	68.90
Table 25: Accuracy (%) of PADA [15], a partial-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	45.03	68.85	81.89	43.25	46.83	57.26	57.12	36.42	76.53	71.26	44.24	77.76	58.87
IWCV [30]	55.58	65.10	84.54	51.42	61.29	53.01	57.02	35.16	81.34	70.52	60.78	74.12	62.49
DEV [18]	54.81	78.15	78.02	58.13	61.29	50.14	67.86	35.16	83.21	74.66	57.91	77.76	64.76
RV [31]	43.22	65.10	81.89	42.70	48.74	52.79	57.21	35.16	77.80	73.46	44.24	77.76	58.34
Entropy [42]	40.12	40.11	55.94	52.43	37.25	50.14	57.30	47.22	81.34	70.52	52.18	82.13	55.56
InfoMax [28]	54.81	69.24	78.02	52.43	37.25	50.14	57.30	47.22	71.84	70.52	52.18	74.12	59.59
SND [29]	40.12	40.11	55.94	58.13	56.13	64.11	70.62	51.22	81.34	74.66	60.78	82.13	61.27
Corr-C [43]	40.12	40.11	55.94	54.18	46.89	53.01	58.59	38.93	77.80	71.26	57.91	77.70	56.04
MixVal	45.02	78.15	83.69	56.23	57.85	68.19	71.41	47.22	84.04	75.39	60.78	82.82	67.57
Worst	40.12	40.11	55.94	41.41	37.25	50.14	56.93	34.87	71.84	70.52	44.30	74.12	51.46
Best	55.58	78.15	86.53	58.13	61.29	68.19	73.00	51.22	84.04	76.86	60.78	84.20	69.83
Table 26: Accuracy (%) of SAFN [23], a partial-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
SourceVal	59.40	77.14	81.34	63.97	67.00	71.29	65.60	46.21	76.81	70.89	58.51	79.10	68.11
IWCV [30]	52.24	74.45	82.16	70.98	62.41	70.18	63.45	53.49	76.81	73.65	56.00	78.49	67.86
DEV [18]	55.22	74.45	80.07	70.98	67.00	71.29	63.45	51.70	76.81	73.65	57.91	80.39	68.58
RV [31]	53.67	71.60	81.34	67.58	67.00	73.27	65.70	48.54	76.81	73.65	56.00	79.89	67.92
Entropy [42]	58.93	74.90	80.73	70.98	74.12	69.80	70.16	50.09	79.24	74.10	57.85	80.06	70.08
InfoMax [28]	51.82	67.62	76.97	64.65	65.77	69.80	59.69	50.09	74.10	66.67	53.31	75.52	64.67
SND [29]	51.82	74.90	80.73	70.98	74.12	75.10	70.16	50.09	79.24	74.10	53.31	80.06	69.55
Corr-C [43]	59.40	77.20	82.16	67.58	72.89	75.10	70.16	55.70	80.12	75.94	52.00	80.73	70.75
MixVal	59.24	76.90	81.28	68.96	73.71	74.82	70.19	55.34	80.73	75.94	59.16	80.62	71.41
Worst	51.52	67.62	76.97	61.07	62.35	69.80	59.69	46.21	74.10	66.67	52.00	75.52	63.63
Best	59.40	77.20	82.16	71.72	74.12	75.10	72.45	55.70	80.73	75.94	59.16	80.73	72.03
Table 27: HOS [62, 63] (%) of DANCE [41], an open-partial-set UDA method, on Office-Home.
Method	Ar → Cl	Ar → Pr	Ar → Re	Cl → Ar	Cl → Pr	Cl → Re	Pr → Ar	Pr → Cl	Pr → Re	Re → Ar	Re → Cl	Re → Pr	avg
Entropy [42]	38.29	26.08	36.51	32.92	17.10	32.19	37.69	46.40	45.53	25.39	33.75	39.37	34.27
InfoMax [28]	38.29	26.08	36.51	32.92	17.10	32.19	37.69	46.40	45.33	25.39	33.75	39.37	34.25
SND [29]	1.00	0.00	12.73	0.00	42.84	1.95	19.77	11.99	35.69	25.39	0.00	28.40	14.98
Corr-C [43]	1.00	0.00	12.73	0.00	42.84	1.95	19.77	11.99	35.69	69.02	0.00	28.40	18.62
MixVal	47.93	76.36	66.57	67.87	75.17	59.05	69.18	58.93	69.40	77.57	51.83	84.31	67.01
Worst	1.00	0.00	12.73	0.00	17.10	1.95	19.77	11.99	35.69	25.39	0.00	28.40	12.84
Best	67.00	76.96	66.57	71.76	75.17	69.99	77.42	64.32	72.87	81.84	67.54	84.31	72.98