Title: Style Adaptation and Uncertainty Estimation for Multi-Source Blended-Target Domain Adaptation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Domain adaptation

Transfer learning

Multi-source and blended-target

Style transfer

Uncertainty estimation
Abstract: Blended-target domain adaptation (BTDA), which implicitly mixes multiple sub-target domains into a fine domain, has attracted more attention in recent years. Most previously developed BTDA approaches focus on utilizing a single source domain, which makes it difficult to obtain sufficient feature information for learning domain-invariant representations. Furthermore, different feature distributions derived from different domains may increase the uncertainty of models. To overcome these issues, we propose a style adaptation and uncertainty estimation (SAUE) approach for multi-source blended-target domain adaptation (MBDA). Specifically, we exploit the extra knowledge acquired from the blended-target domain, where a similarity factor is adopted to select more useful target style information for augmenting the source features. \!Then, to mitigate the negative impact of the domain-specific attributes, we devise a function to estimate and mitigate uncertainty in category prediction. Finally, we construct a simple and lightweight adversarial learning strategy for MBDA, effectively aligning multi-source and blended-target domains without the requirements of domain labels of the target domains. Extensive experiments conducted on several challenging DA benchmarks, including the ImageCLEF-DA, Office-Home, VisDA 2017, and DomainNet datasets, demonstrate the superiority of our method over the state-of-the-art (SOTA) approaches.

Style Adaptation and Uncertainty Estimation for Multi-Source Blended-Target Domain Adaptation

Abstract

Blended-target domain adaptation (BTDA), which implicitly mixes multiple subtarget domains into a fine domain, has attracted more attention in recent years. Most previously developed BTDA approaches focus on utilizing a single source domain, which makes it difficult to obtain sufficient feature information for learning domaininvariant representations. Furthermore, different feature distributions derived from different domains may increase the uncertainty of models. To overcome these issues, we propose a style adaptation and uncertainty estimation (SAUE) approach for multi-source blended-target domain adaptation (MBDA). Specifically, we exploit the extra knowledge acquired from the blended-target domain, where a similarity factor is adopted to select more useful target style information for augmenting the source features. Then, to mitigate the negative impact of the domain-specific attributes, we devise a function to estimate and mitigate uncertainty in category prediction. Finally, we construct a simple and lightweight adversarial learning strategy for MBDA, effectively aligning multi-source and blended-target domains without the requirements of domain labels of the target domains. Extensive experiments conducted on several challenging DA benchmarks, including the ImageCLEF-DA, Office-Home, VisDA 2017, and DomainNet datasets, demonstrate the superiority of our method over the state-of-the-art (SOTA) approaches.

1 Introduction

Domain adaptation (DA), whose objective is to transfer knowledge from one or more well-labeled source domains to a non-labeled target domain, has been studied in recent years [1, 2, 3, 4, 5, 6], including object classification [1, 2], semantic segmentation [3, 4], and object detection [5]. However, most DA approaches are based on a setting that has single source domain and single target domain [1, 2]. In the real world, unlabeled target domains are usually drawn from different distributions. Therefore, most of the existing single target-based DA approaches may not be the best answer to address domain shifts in reality.

Fortunately, an increasing number of researchers have focused on the above-mentioned issues, and multi-target domain adaptation (MTDA) [7, 8, 9] has been studied. MTDA aims to learn a model that can simultaneously utilize information from single source domain and multiple target domains and then perform well in multiple target domains. For instance, HGAN [10] adopts a heterogeneous graph attention network to explore the relations among multiple target domain features. In [11], multiple expert models employed the corresponding source-target domain pair-groups and were then aligned by a student model. Although the existing MTDA approaches have made certain progress, the standard MTDA is still facing challenges because massive amounts of unlabeled data drawn from different distributions are commonly used in real-world settings. It is time-consuming and expensive to divide massive data into multiple corresponding distributions (target domains). For example, for

encrypted data stored in a cloud server, due to privacy protection, models cannot directly know the origins of these data (domain labels), which are drawn from different distributions and are blended into a large target domain. Based on the above-mentioned case, blended-target DA (BTDA), which is a more beneficial scenario in real-world settings, has been proposed [12].

Current BTDA approaches [12, 13] are mainly based on three points: 1) the adaptation process contains single source domain and multiple target domains. 2) During the adaptation process, the model cannot access both the domain labels and category labels of the target domains. 3) In blendedtarget domain, the category labels of each sub-target domain may not follow the same distribution. Therefore, simply utilizing MTDA or SSDA (single source domain adaptation) methods to handle the BTDA task may lead to negative transfer because the domain labels of target domains are unseen, and the category feature space is a hybrid space. As the first deep learning work focused on the BTDA scenario, AMEAN [12] employs two adversarial learning steps and utilizes meta-learning to minimize the domain gap between the source domain and the blended-target domain. However, insufficient information obtained from single source domain makes models difficult to align the distributions of multiple target domains. Moreover, the presence of different distributions in the blended-target domain may aggravate negative transfer. Recently, multi-source domain adaptation (MSDA) [14, 15, 16] has produced impressive results. MSDA approaches can utilize more feature information from extra source domains to learn domain-invariant representations, effectively solving negative transfer. However, as far as we know, no related works have been proposed to utilize more feature information from multiple source domains for BTDA.

In this paper, to further exploit feature information from multiple domains, we pay attention to the BTDA in the case of multiple source domains, i.e., Multi-Source Blended-Target Domain Adaptation (MBDA). The comparisons of different DA settings are illustrated in Table 1. At the same time, a style adaptation and uncertainty estimation (SAUE) method is proposed for MBDA. Different from previous works, we utilize the style information of the blended-target domain to enhance source domain features, thus building a better representation space. Specifically, we first simultaneously extract the source and target style information and then calculate the similarity factors between the source and target style information. The similarity factors are served as the weighted matrix during the feature augmentation process. Based on style adaptation, we further analyze the uncertainty in the classification model and adopt a loss function to eliminate the uncertainty introduced by the multi-source domains. In addition, as the domain labels of the blended-target domain are unavailable in MBDA, we construct an adversarial learning scheme for MBDA without the requirement of domain labels of the target domains.

The main contributions of this work are summarized as follows:

An approach SAUE is proposed to explore information from multiple source domains for BTDA. As far as we know, SAUE is the first work that was proposed for MBDA, which can utilize more feature information from extra source domains to learn domain-invariant representations.

To further exploit the style information in the blended-target domain, we propose a similaritybased style adaptation strategy for MBDA, which selects target styles to enhance the source representation space.

We propose an uncertainty estimation technique to enhance the robustness of our method, which exploits valuable knowledge from multiple source domains. In addition, we construct a specific adversarial learning strategy for MBDA, which aligns domains without the requirement of domain labels.

2 Related Works

Single-source and Single-target DA (SSDA). The objective of SSDA is to learn domain-invariant representations through the relations between the source and target domains. Based on this objective, researchers have carried out widespread researches and achieved certain progress [17, 18, 19, 20, 21, 22, 23]. The current SSDA methods are mainly divided into two types: distance metric-based approaches [17, 18, 19] and adversarial learning-based approaches [20, 21, 22, 23]. Distance metricbased methods learn domain-invariant representations through feature discrepancy matching by using a distance metric function. DAN [18] utilizes multi-kernel maximum mean discrepancy (MMD) to measure the discrepancy between the source and target domains and then minimizes the discrepancy to learn domain-invariant representations. CAF [19] utilizes sliced Wasserstein distance (SWD) [24] to measure domain discrepancy. Motivated by the GANs [25, 26], adversarial learning-based SSDA methods have also been widely researched [20, 21, 22, 23]. Adversarial learning methods perform min-max two-player games between the category classifier and domain discriminator to learn domain-invariant representations. DANN [20], which was the earliest work in adversarial learning-based SSDA, successfully achieves domain-level adaptation via a gradient reverse layer. Different from DANN, SCDA [22] and DALN [23] remove the discriminator from their networks and model the adversarial relation between the feature extractor and category classifier. Although the above-mentioned approaches have achieved great success, due to the limitations of single source domain features and single target domain features, current SSDA methods still face some challenges in real applications.

Multiple Domains DA. The motivation of multiple domains DA is to explore more useful knowledge from multiple domains for the tasks. Many researchers have focused on multiple-domain DA and proposed many excellent methods [13, 14, 15, 8], including MSDA [14, 15, 27, 28], MTDA [8], BTDA [13, 12], and MMDA (multi-source and multi-target DA) [29]. M3SDA [14] utilizes moment matching to align domain distribution. DCA [15] further extracts the multiview features from multiple source domains and then utilizes multiple classifiers and pseudo-label learning strategy to align distributions. Meanwhile, in MTDA, CGCT [8] utilizes feature aggregation and curriculum learning to learn the pseudo-labels of multiple target domains. AMDA [29] constructs multiple domain discriminators and utilizes attention mechanism to address the MMDA issue.

Recently, a more realistic DA scenario, BTDA, has been studied [12, 13]. For example, MCDA [13] utilizes the mutual condition to learn domain-invariant representations, which has achieved great progress in BTDA. However, single source domain in BTDA cannot provide sufficient feature information for aligning the source and blended-target domains. Furthermore, the unseen domain labels of the target domains also aggravate the challenges. Therefore, we consider multiple source domains in BTDA and utilize the style information of the target domains to optimize the representations of source features and minimize the model uncertainty, thereby obtaining a better transfer.

3 Method

3.1 Preliminary

In MBDA, we have M labeled source domains S = {S m } M m =1 that are drawn from distributions { P S m } M m =1 . S m = { x S m i , y S m i } |S m | i =1 , where x S m i ∈ R d denotes the i -th source sample from the m -th source domain and y S m i is the corresponding category label, and d denotes the number of dimensions. Meanwhile, we have an unlabeled blended-target domain T that consists of N sub-target domains {T n } N n =1 , and T = { x T j } |T | j =1 . The distributions of sub-target domains are { P T n } N n =1 . Therefore, the distribution of blended-target domain P T is the mixture of sub-target domains, i.e. , P T = ∑ N n =1 π n P T n , where π ∈ [0 , 1] and ∑ N n =1 π n = 1 . Each of the source and target domains shares the same category space. The objective of MBDA is to train a model that utilizes multiple source domain features and performs well on the blended-target domain. Different from MTDA, the target domain labels are unseen in MBDA. In addition, the analysis in [12] demonstrated that directly utilizing DA methods to address BTDA transfer tasks may cause increased uncertainty and negative transfer. Therefore, we utilize the style information of the blended-target domain to augment source features and minimize the uncertainty of the model. Furthermore, the adversarial learning strategy in our method without the requirement of domain labels of the sub-target domains is suitable for MBDA setting. Figure 1 illustrates the overall architecture of SAUE.

-

…

𝑆

!

𝑆

"

Source Domains

…

𝑇

!

𝑇

#

Blended

Target Domain

…

…

𝑧

$

%

!

𝑧

&

'

𝑧̃

$

%

!

𝑧

!

"

!

#

𝑧

&

'

ℒ

()*

ℒ

*+,

ℒ

)-.

&

#

&

#

𝐺𝑅𝐿

𝐷(𝑝

!

"

!

|𝛼 )

!

"

!

) 𝐷(𝑝

!

"

!

|

1, … , 1 )

KL divergence

Similarity Factors

𝝀

𝒊

𝑺

𝒎

Convolution Layer

Convolution Layer

Feature Extractor with Style Adaptation

Classifier and Uncertainty Estimation

Style Adaptation

𝝁

𝒊

𝑺

𝒎

,

𝝈

𝒊

𝑺

𝒎

𝝁

𝒋

𝑻

,

𝝈

𝒋

𝑻

Distance

Normalization

𝝁

𝒊

, 𝝈

𝒊

Adapt

Confuse

Source Dataflow

Backpropagation

𝐺𝑅𝐿

Gradient Reverse Layer

3.2 Style Adaptation from Blended-Target Domain

Since the principal parts of features from different domains remain domain-invariant, the domainspecific parts, which mainly contain style information, are the main discrepancies between different domains. In addition, the target feature distributions in MBDA are confused, which may cause model degradation. Therefore, we try to utilize the style information of blended-target domain to augment source features, which brings source features closer to target features. Previous work [13] has demonstrated that low-level features of deep neural networks (DNNs) mainly represent style information. Some works [13, 30] have utilized the channel-wise mean and variance of the low-level features to represent the style information of input samples. Thus, for sample x S m i which from the m -th source domain, let its low-level feature be z S m i ∈ R d × H S m × W S m , where d denotes the channel and H S m , W S m denote the height and width of sample x S m i . The channel-wise mean and variance of the low-level feature z S m i can be defined as follows:

Low-level features mainly represent style information, but different samples contain specific pieces of style information. Therefore, we adopt feature normalization technique to standardize the feature z S m i , and the normalized feature ˜ z S m i is defined as:

where ϵ is a small value used to avoid division by zero.

Then, the target features will be utilized to augment the normalized source features. Previous work [13] randomly augmented source features through target style information, which yielded limited performance. In our work, we select the target style information according to a weight factor. Specifically, we leverage the Wasserstein Distance [31] to measure the style distribution discrepancy w S m i between the source low-level feature z S m i and the target low-level feature z T j as follows:

Then, we utilize w S m i to calculate the weight factor as follows:

To ensure that the sum in Eq. (4) equals to 1, we utilize softmax operation to normalize each λ S m i . Then, we can obtain the weighted target style as follows:

Finally, the low-level source feature augmented by the blended-target style information can be calculated as follows:

Different from previously developed image augmentation method [13], our method directly utilizes target information with weight factor to augment source features instead of generating specific images. The low-level feature z S m T i augmented by diverse target styles further mitigates the impact of the domain-specific attributes.

3.3 Uncertainty Estimation and Elimination

Although multiple source domains provide additional supervised information for adaptation compared to a single source domain, they also introduce more domain-specific attributes. This can cause model degradation, especially when some source domains are significantly dissimilar to the blended-target domain due to the abundance of domain-specific attributes. Evidential model learning (EDL) [32] is an interpretable approach that fuses knowledge from multiple domains using the Dempster-Shafer Rule [33, 34], which is more beneficial to MBDA scenario. Thus, we utilize the Dirichlet-based evidential model [32] to estimate the uncertainty of our model during the training process. Specifically, for a sample x i , we have the predictions p i = C ( G ( x i )) = [ p i 1 , p i 2 , . . . , p iK ] , where C and G denote the classifier and feature generator, respectively. The probability density function of p i is defined as follows:

where U k = { p i | ∑ K k =1 p ik = 1 and 0 ≤ p i 1 , ..., p iK ≤ 1 } is the K -dimensional unit simplex and α i is the parameter of the Dirichlet distribution. B ( α i ) = Γ( ∑ K k =1 α ik ) ∏ K k =1 Γ( α ik ) is the K -dimensional multinomial beta function, and Γ( · ) denotes the gamma function.

Previous work [32] has demonstrated that DNNs can generate opinions for classification tasks as Dirichlet distributions. Therefore, given sample x i , for prediction of class c that generated by DNNs, the Dirichlet distributions connected with DNNs can be defined as follows:

The derivation of Eq. (8) is provided in Appendix B.

In this work, for source sample x S m i , we utilize the prediction of the category classifier as the evidence vector, and the parameters of the corresponding Dirichlet distribution can be defined as α S m i = C ( G ( x S m i )) + 1 . To eliminate the uncertainty, we force the total evidence to zero when the model generates an incorrect prediction for the source sample, and the corresponding uniform Dirichlet distribution is D ( p S m i | ⟨ 1 , . . . , 1 ⟩ ) . For implementation, we utilize the Kullback-Leibler (KL) divergence to reduce the impact of incorrectly classified source samples in our loss function, which is defined as follows:

where ˜ α S m i = y S m i +(1 -y S m i ) ⊙ α S m i denotes the Dirichlet parameter used to remove the true evidence from prediction α S m i . Specifically, the KL divergence is:

where Γ( · ) and Ψ ( · ) denotes the gamma function and digamma function, respectively.

Algorithm 1 SAUE for MBDA

Input: Source domains {S m } M m =1 and the corresponding data { x S m i , y S m i } |S m | i , blended-target domain data { x T j } , hyper-parameters λ e and λ d , maximum iteration I , and mini-batch size B .

Output: Optimal feature generator G and category classifier C

1: for i in 1: I do

2: Randomly sample a mini-batch of B source samples and target samples.

3: Augment the source features by utilizing style adaptation, i.e. , Eq. (6).

4: Minimize the parameters of the feature generator and category classifier by L cls .

5: Optimize the uncertainty of model through L unc

.

6: Perform the min-max game between feature generator and classifier with L d :

3.4 Domain Adversarial Alignment without Domain Labels

Existing works [20, 21, 22, 8] have demonstrated that adversarial learning strategy is beneficial in DA. However, most of the adversarial learning strategies in DA [20, 21] usually request the domain labels of the source and target domains to train their discriminators, which cannot satisfy MBDA. Inspired by the intra- and inter-class correlation [35], we design an adversarial learning strategy for MBDA without discriminator and domain label requirements. Specifically, the category classifier is reused to discriminate which domain a feature originates from, with the guidance of the Nuclear norm ∥·∥ ∗ . We first measure the distribution difference between the source and blended-target domains through the nuclear-norm 1-Wasserstein discrepancy (NWD) [23] and then utilize a gradient reverse layer (GRL) [20] to maximize the discriminative loss of the classifier. Simultaneously, we minimize the feature generator to play the min-max game with the classifier through the NWD. First, the NWD loss can be defined as:

Then, the adversarial learning strategy between feature generator and classifier is defined as follows:

3.5 Model Optimization and Theoretical Analysis

Overall Objective. The overall loss function that optimizes SAUE for MBDA is defined as:

where λ e = min(1 , e/λ ′ e ) ∈ [0 , 1] is the annealing coefficient, which prevents L unc from overpenalizing the neural network to a uniform distribution in the early training epochs, and e is the current number of epochs and λ ′ e = 40 . λ d is a hyper-parameter which is initially set to λ d = 1 as in [23]. L cls is the classification loss, which ensures that the category classifier can correctly classify samples. With the the cross-entropy loss L ce , the classification loss L cls is defined as follows:

After adversarial training, our model can effectively adapt the blended-target domain without the requirement of domain labels of sub-target domains. The concrete steps of SAUE are shown in Algorithm 1.

Theoretical Analysis. Here, we utilize PAC-Bayesian theory [36] to optimize our classification model with uncertainty estimation and elimination. The full-bound theorem motivated by previous work [37] is illustrated in Theorem 1 and Lemma 1, and the proofs are provided in Appendix C.

.

Theorem 1 [37]. Suppose we have given the m-th source data distribution P S m , a hypothesis set H , and a prior distribution π over the hypothesis space Θ . For any τ ∈ (0 , 1] and λ > 0 , with a probability at least 1 -τ over the source samples S m ∼ P S m , for all posteriors ρ , we have:

Lemma 1 [38]. The PAC-Bayes bound, involving constants τ and n , as introduced in Theorem 1, is minimized by the Bayesian posterior p ( H ) , which represents the distribution over Θ .

During uncertainty estimation and elimination, just as in Eq. (9), we utilize D ( p S m i | ˜ α S m i ) as the posterior distribution and D ( p S m i | ⟨ 1 , . . . , 1 ⟩ ) as the prior distribution. Therefore, the upper bound of the classification model can be expressed as:

Generalization Bound. In this part, we prove why SAUE performs well on the blended-target domain via Lemma 2 and Theorem 2. The proofs and derivations are provided in Appendix D.

Lemma 2 [39]. Suppose we have given the probability measures ν S m , ν T ∈ P ( F ) of the m -th source feature f S m and the blended-target domain feature f T , a hypothesis space Θ , and a subspace ˜ H ∈ Θ . Let F denote a fixed representation space and c ( f S m , f T ) denote the adaptation cost. For the ideal classifier h ′ ∈ ˜ H and any classifier h ∈ ˜ H with f S m ∼ ν S m and f T ∼ ν T , we have:

where ϵ S m and ϵ T denote the error on the m -th source domain and the error on the blended-target domain respectively, and ϵ T = 1 N ∑ N j =1 ϵ T j . d H ∆ H denotes the H ∆ H -distance.

Theorem 2. Based on Lemma 2, with the error of the ideal joint hypothesis η ′ = ϵ S m ( h ′ ) + ϵ T ( h ′ ) which is a sufficiently small constant, for any δ ∈ (0 , 1) , with probility at least 1 -δ , for every h ∈ H , ϵ T ( h ) is bounded by the following terms:

where η ′ = ϵ S m ( h ′ ) + ϵ T ( h ′ ) is the ideal error for the classifier, which is a sufficiently small constant. b ′ is the size of unlabeled samples.

Therefore, the final objective of the MBDA classification task is to reduce the joint domain discrepancy term ∑ M m =1 | ϵ S m ( h, h ′ ) -ϵ T ( h, h ′ ) | .

4 Experiments

4.1 Datasets and Implementation Details

Datasets. Four standard benchmark datasets are used to validate the effectiveness of our proposed method. The ImageCLEF-DA [40] contains 2,400 images and is divided into 4 domains: Bing (b), Caltech (c), ImageNet (i), and Pascal (p). Each domain has 12 categories, and every category has 50 images. The Office-Home [41] also consists of 4 domains and 15,588 images belonging to 65 categories from four subdomains: Art (Ar), Clipart (Cl), Products (Pr), and Real world (Rw). The DomainNet [14] is a large-scale dataset in DA that contains 0.6 million images of 345 categories from 6 domains: Clipart (C), Infograph (I), Painting (P), Quickdraw (Q), Real world (R), and Sketch (S). Following the protocol used in [29], we select 126 categories and 4 domains (C, P, R, and S) in our experiments. The VisDA 2017 [42] dataset is a challenging dataset consists of 2 domains (Syn. and Rel.) and 7 categories.

Implementation Details. We utilize PyTorch framework [43] to perform our experiments; the PyTorch version is 1.13.1 and CUDA version is 11.7. We use an ImageNet pre-trained ResNet [44], replacing the last FC layer with task-specific FC layers. All experiments are run on a single GeForce RTX-4090 GPU, and the batch size of both the source and blended-target domains are set to 32. The optimizer is Stochastic Gradient Descent (SGD) with a momentum parameter of 0.9 and a weight decay of 1e-3. The learning rate is 1e-3 and updated by the LambdaLR [43] during the training process.

4.2 Comparisons to State-of-the-Art

To evaluate the effectiveness of our proposed method, we conduct extensive experiments and compare our approach with the state-of-the-art (SOTA) methods in terms of DA classification. The comparison methods include SSDA approaches, i.e., MCD [45], DAN [18], TSA [46], DALN [23], BIWAA [47], and SCDA [22]; MSDA methods, i.e., MDAN [48], DCTN [49], and DIDA [50]. MTDA/BTDA methods: MTDA-ITA [7] and MCDA [13]; and Multi-source Multi-target DA (MMDA), i.e., AMDA [29] and HTA [51]. The comparison results are presented in Tables 2-4, in which we select two domains as source domains and combine other two domains to form the blended-target domain. Note that these approaches do not totally match the MBDA setting. Therefore, we utilize the following rule for our comparison. For SSDA setting, one column denotes one SSDA task, such as R → C in Table 2. For MSDA methods that contain more than two source domains, we implement those methods according to their released codes, reset the source domain into two domains, such as R+S → C, and mark them with '*'. Similarly, under the MTDA and BTDA settings, we reset the target into two domains and select the highest one in MTDA/BTDA task group that contains the same target domains, such as R → C+P and S → C+P in Table 2. For MMDA setting, two domains are sources, and the other domains are targets, such as R+S → C+P in Table 2. For better comparison, all results in Tables 2-4 are the averages of two target domains.

Results on the DomainNet are displayed in Table 2. Our SAUE method achieves SOTA performance in most of the experimental groups and attains the best performance in terms of average accuracy. Compared to the BTDA method MCDA in multi-source setting, our method achieves better performance because the style information of the target domain selected by the weight factor can enhance the source feature representations. Compared to MMDA method AMDA, although AMDA can access the domain labels of the target domains, our method still overpasses

'*' denotes that the results are obtained by the released code of the corresponding method. The best results are bolded.

AMDAin terms of average classification accuracy (overpass 7.5 %) and without the requirement of the domain labels of the target domains. Furthermore, both AMDA and our method are adversarial learning methods, and the comparison results further demonstrate the effectiveness of our adversarial learning strategy. These obtained improvements are mainly due to the uncertainty optimization process and the style information derived from target features.

'*' denotes that the results are obtained by the released code of the corresponding method. The best results are bolded.

Results on the Office-Home are shown in Table 3a. The experimental results are compared with those of the SOTA methods, illustrating that our proposed method achieves dramatic improvements in most comparison groups and achieves the highest average classification accuracy ( 73.7 %). Note that the Rw domain contains a total of 34,856 images, which is far more numerous than the other

three domains. Therefore, the adaptation task faces larger domain shifts and extremely unbalanced classes. The proposed method still achieves 2.8% improvements over AMDA and achieves dramatic improvements in the Rw+Pr → Ar+Cl and Rw+Ar → Pr+Cl tasks. These results occur because the proposed method decreases the impact of unbalanced classes by enhancing the feature representations and optimizing the prediction uncertainty.

Results on the ImageCLEF-DA are provided in Table 3b. Compared with the SOTA methods, our proposed method achieves an average accuracy of 84.3% , outperforming the existing approaches. Note that all four domains in ImageCLEF-DA contain 600 images. Therefore, the experimental results further demonstrate that our proposed method is effective when all the domains contain the same samples and classes.

Results on the Default Version of DomainNet and VisDA 2017 . To evaluate the effectiveness of SAUE in different numbers of the source and target domains. We perform comparisons on the default version of the DomainNet dataset and the VisDA 2017 dataset, respectively. The default version of the DomainNet dataset consists of 5 domains, leading to the division of transfer tasks for MBDA into two categories: C+P+Q → R+S and R+S → C+P+Q. As shown in Table 4, SAUE outperforms the comparison methods across all transfer tasks, achieving the highest average classification accuracy. These results from large-scale datasets further demonstrate the superiority and flexibility of SAUE.

4.3 Experiment Analysis

d

0.5

1.0

1.5

e

10

20

30

40

50

60

70

0.5

0.6

0.7

0.8

0.9

0.88

0.90

0.93

0.95

0.98

(a) b+i

→

c

d

0.5

1.0

1.5

e

10

20

30

40

50

60

70

0.3

0.4

0.5

0.6

0.7

0.74

0.76

0.78

(b) b+i

→

p

(a) b+i → c

(b) b+i

→

p

Sensitivity Analysis. We evaluate the model's performance under different hyperparameter choices. Note that the hyperparameters in our method are the adversarial learning balance parameter λ d and annealing parameter λ e . As shown in Figure 2, we test different parameter groups to analyze the parameter sensitivity of our method, where λ d = { 0 . 1 , 0 . 5 , 1 . 0 , 1 . 5 , 2 . 0 } and λ e = { 10 , 20 , 40 , 60 , 80 } . The corresponding numerical results of Figure 2 are illustrated in Table 5. In Figure 2, our method is not sensitive to λ d and the best parameter choice is λ d = 1 . 0 , but very sensitive to λ e (with λ e = 40

working best); if λ e too small, the model will suffer from tremendous degradation due to the model is over-penalized by uncertainty loss.

Ablation Study. As listed in Table 6, we conduct ablation experiments to demonstrate the effectiveness of the style adaptation module and the loss function of the uncertainty optimization process. We test three experimental groups in the DomainNet dataset, including that: 1) remove the style adaptation (SA) module, 2) remove the uncertainty loss L unc , and 3) remove both of the above-mentioned items.

The results illustrate that both style adaptation and prediction uncertainty optimization are useful for MBDA. Furthermore, we explore different style adaptation techniques. We directly change the Wasserstein Distance (WD) to randomly selected style information and report the obtained results in the fourth row of middle part of Table 6. Compared with the random augmentation methods, our proposed method can better select the style information through weight factors, which enhances the feature representations of the source domains and reduces the impact of domain shifts. More experiment analysis is provided in Appendix E.

Effectiveness of the SA Module with Different Backbones. We have analyzed the performance of our method using the ResNet-50 and ResNet-101 backbones in the default version of the DomainNet dataset in Table 7. The experimental results show that our method achieves significant performance gains when utilizing powerful backbones (ResNet-101). We compared the performance gains of the style adaptation module with standard backbone (ResNet-50) and powerful backbone (ResNet-101). The results indicate that the performance gains (+2.8% and +2.3%) of the style adaptation module when integrated into the ResNet-50 backbone surpass those (+1.6% and +1.3%) when integrated into the ResNet-101 backbone.

Effectiveness of the SAUE with ResNet and ViT backbones. We further evaluate the model's performance using different backbones, as shown in Table 8. When using ViT as the backbone, SAUE outperforms its performance with the ResNet-50 backbone. This

performance gain is primarily due to the larger number of tunable parameters in ViT-B/16, demonstrating that our method effectively leverages these additional parameters to exploit transferable knowledge from multiple domains.

5 Conclusion

In this paper, we propose a SAUE approach for MBDA, which utilizes information from multiple source domains to adapt a blended-target domain. In particular, the style adaptation process utilizes similarity factors to select target style information to enhance the representations of the source features. The uncertainty estimation procedure utilizes the Dirichlet distribution to estimate the uncertainty of the model and then adopts the KL divergence measure to optimize the prediction uncertainty. The discriminator-free adversarial learning strategy is beneficial for MBDA. Extensive experimental results demonstrate the superior performance of SAUE to that of the competing methods.

Acknowledgment

This work was supported by the National Natural Science Foundation of China (Grant No. 62176162) and the Guangdong Basic and Applied Basic Research Foundation (2023A1515012875, 2022A1515140099).

References

Rui Wang, Zuxuan Wu, Zejia Weng, Jingjing Chen, Guo-Jun Qi, and Yu-Gang Jiang. Cross-domain contrastive learning for unsupervised domain adaptation. IEEE TMM , 25:1665-1673, 2023.

Zhongyi Han, Haoliang Sun, and Yilong Yin. Learning transferable parameters for unsupervised domain adaptation. IEEE TIP , 31:6424-6439, 2022.

Yiting Cheng, Fangyun Wei, Jianmin Bao, Dong Chen, and Wenqiang Zhang. Adpl: Adaptive dual path learning for domain adaptation of semantic segmentation. IEEE TPAMI , 45(8):9339-9356, 2023.

Munan Ning, Donghuan Lu, Yujia Xie, Dongdong Chen, Dong Wei, Yefeng Zheng, Yonghong Tian, Shuicheng Yan, and Li Yuan. Madav2: Advanced multi-anchor based active domain adaptation segmentation. IEEE TPAMI , 45(8):13553-13566, 2023.

Poojan Oza, Vishwanath A. Sindagi, Vibashan Vishnukumar Sharmini, and Vishal M. Patel. Unsupervised domain adaptation of object detectors: A survey. IEEE TPAMI , 2023.

Jinjing Zhu, Haotian Bai, and Lin Wang. Patch-mix transformer for unsupervised domain adaptation: A game perspective. In CVPR , pages 3561-3571, 2023.

Behnam Gholami, Pritish Sahu, Ognjen Rudovic, Konstantinos Bousmalis, and Vladimir Pavlovic. Unsupervised multi-target domain adaptation: An information theoretic approach. IEEE TIP , 29:3993-4002, 2020.

Subhankar Roy, Evgeny Krivosheev, Zhun Zhong, Nicu Sebe, and Elisa Ricci. Curriculum graph coteaching for multi-target domain adaptation. In CVPR , pages 5347-5357, 2021.

Jiazhong Zhou, Qing Tian, and Zhanghu Lu. Progressive decoupled target-into-source multi-target domain adaptation. INS , 634:140-156, 2023.

Xu Yang, Cheng Deng, Tongliang Liu, and Dacheng Tao. Heterogeneous graph attention network for unsupervised multiple-target domain adaptation. IEEE TPAMI , 44(4):1992-2003, 2022.

Takashi Isobe, Xu Jia, Shuaijun Chen, Jianzhong He, Yongjie Shi, Jianzhuang Liu, Huchuan Lu, and Shengjin Wang. Multi-target domain adaptation with collaborative consistency learning. In CVPR , pages 8183-8193, 2021.

Ziliang Chen, Jingyu Zhuang, Xiaodan Liang, and Liang Lin. Blending-target domain adaptation by adversarial meta-adaptation networks. In CVPR , pages 2248-2258, 2019.

Pengcheng Xu, Boyu Wang, and Charles Ling. Class overwhelms: Mutual conditional blended-target domain adaptation. In AAAI , pages 3036-3044, 2023.

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In CVPR , pages 1406-1415, 2019.

Keqiuyin Li, Jie Lu, Hua Zuo, and Guangquan Zhang. Dynamic classifier alignment for unsupervised multi-source domain adaptation. IEEE TKDE , 35(5):4727-4740, 2023.

Keqiuyin Li, Jie Lu, Hua Zuo, and Guangquan Zhang. Multi-source contribution learning for domain adaptation. IEEE TNNLS , 33(10):5293-5307, 2021.

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. Unsupervised domain adaptation with residual transfer networks. In NeurIPS , pages 136-144, 2016.

Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Transferable representation learning with deep adaptation networks. IEEE TPAMI , 41(12):3071-3085, 2019.

Binhui Xie, Shuang Li, Fangrui Lv, Chi Harold Liu, Guoren Wang, and Dapeng Wu. A collaborative alignment framework of transferable knowledge extraction for unsupervised domain adaptation. IEEE TKDE , 35(7):6518-6533, 2023.

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francois Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMLR , 17(1):2096-2030, 2016.

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Conditional adversarial domain adaptation. In NeurIPS , pages 1640-1650, 2018.

Shuang Li, Mixue Xie, Fangrui Lv, Chi Harold Liu, Jian Liang, Chen Qin, and Wei Li. Semantic concentration for domain adaptation. In ICCV , pages 9102-9112, 2021.

Lin Chen, Huaian Chen, Zhixiang Wei, Xin Jin, Xiao Tan, Yi Jin, and Enhong Chen. Reusing the taskspecific classifier as a discriminator: discriminator-free adversarial domain adaptation. In CVPR , pages 7171-7181, 2022.

Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht. Sliced wasserstein discrepancy for unsupervised domain adaptation. In CVPR , pages 10285-10295, 2019.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NeurIPS , pages 2672-2680, 2014.

Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A. Bharath. Generative adversarial networks: An overview. IEEE MSP , 35(1):53-65, 2018.

Yongchun Zhu, Fuzhen Zhuang, and Deqing Wang. Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. In AAAI , pages 5989-5996, 2019.

Haozhe Feng, Zhaoyang You, Minghao Chen, Tianye Zhang, Minfeng Zhu, Fei Wu, Chao Wu, and Wei Chen. Kd3a: Unsupervised multi-source decentralized domain adaptation via knowledge distillation. In ICML , pages 3274-3283, 2021.

Yuxi Wang, Zhaoxiang Zhang, Wangli Hao, and Chunfeng Song. Attention guided multiple source and target domain adaptation. IEEE TIP , 30:892-906, 2021.

Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Mixstyle neural networks for domain generalization and adaptation. IJCV , 132(3):822-836, 2024.

SS Vallender. Calculation of the wasserstein distance between probability distributions on the line. Theory Probab. its Appl. , 18(4):784-786, 1974.

Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In NeurIPS , pages 3179-3189, 2018.

Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view classification with dynamic evidential fusion. IEEE TPAMI , 45(2):2551-2566, 2022.

Zheyao Gao, Yuanye Liu, Fuping Wu, Nannan Shi, Yuxin Shi, and Xiahai Zhuang. A reliable and interpretable framework of multi-view learning for liver fibrosis staging. In MICCAI , pages 178-188. Springer, 2023.

Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion for versatile domain adaptation. In ECCV , pages 464-480. Springer, 2020.

David A. McAllester. Some pac-bayesian theorems. Mach. Learn. , 37(3):355-363, 1999.

Pierre Alquier, James Ridgway, and Nicolas Chopin. On the properties of variational approximations of gibbs posteriors. JMLR , 17(1):8374-8414, 2016.

Pascal Germain, Francis Bach, Alexandre Lacoste, and Simon Lacoste-Julien. Pac-bayesian theory meets bayesian inference. In NeurIPS , pages 1884-1892, 2016.

Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In NeurIPS , pages 137-144, 2007.

Barbara Caputo, Henning Müller, Jesus Martinez-Gomez, Mauricio Villegas, Burak Acar, Novi Patricia, Neda Marvasti, Suzan Üsküdarlı, Roberto Paredes, and Miguel Cazorla. Imageclef 2014: Overview and analysis of the results. In ICCLEF , pages 192-211, 2014.

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In CVPR , pages 5018-5027, 2017.

Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint , 2017. arXiv:1710.06924.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, and Luca Antiga. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS , pages 8026-8037, 2019.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR , pages 770-778, 2016.

Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR , pages 3723-3732, 2018.

Shuang Li, Mixue Xie, Kaixiong Gong, Chi Harold Liu, Yulin Wang, and Wei Li. Transferable semantic augmentation for domain adaptation. In CVPR , pages 11516-11525, 2021.

Thomas Westfechtel, Hao-Wei Yeh, Qier Meng, Yusuke Mukuta, and Tatsuya Harada. Backprop induced feature weighting for adversarial domain adaptation with iterative label distribution alignment. In WACV , pages 392-401, 2023.

Han Zhao, Shanghang Zhang, Guanhang Wu, José M. F. Moura, Joao P. Costeira, and Geoffrey J. Gordon. Adversarial multiple source domain adaptation. In NeurIPS , pages 8559-8570, 2018.

Ruijia Xu, Ziliang Chen, Wangmeng Zuo, Junjie Yan, and Liang Lin. Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In CVPR , pages 3964-3973, 2018.

Zhongying Deng, Kaiyang Zhou, Da Li, Junjun He, Yi-Zhe Song, and Tao Xiang. Dynamic instance domain adaptation. IEEE TIP , 31:4585-4597, 2022.

Zhuanghui Wu, Min Meng, Tianyou Liang, and Jigang Wu. Hierarchical triple-level alignment for multiple source and target domain adaptation. Appl. Intell. , 53:3766-3782, 2023.

Yuwu Lu, Haoyu Huang, Biqing Zeng, Zhihui Lai, and Xuelong Li. Multi-source and multi-target domain adaptation based on dynamic generator with attention. IEEE TMM , 26:6891-6905, 2024.

Mixue Xie, Shuang Li, Rui Zhang, and Chi Harold Liu. Dirichlet-based uncertainty calibration for active domain adaptation. In ICLR , 2023.

Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR , 9:2579-2605, 2008.

Appendix Contents

This supplementary material provides more details that are not presented in the main paper due to space limitations. The organization is as follows:

A Broader Impacts & Limitations

B provides the derivations of P ( y = c | x i ) = E [ D ( p ic | α i )] .

C provides the proof and derivations of Theorem 1.

D provides the proof and derivations of the generalization bound.

E provides additional experiment analysis on the ImageCLEF-DA dataset.

A Broader Impacts & Limitations

Our work focuses on the problem of multi-source blended-target domain adaptation (MBDA), which aims to train a model that generalizes well on an unlabeled and distribution-confused target domain by leveraging multiple labeled source domains. The effectiveness of our method on several real-world datasets suggests that it can benefit relevant applications and communities dealing with domain shifts, such as encrypted data analysis, medical imaging, and autonomous driving. Nevertheless, we should also be cautious about potential failures of our method when encountering more significant distribution shifts, such as an increased number of source or target domains. In the future, we plan to incorporate more sub-domains into our experiments for further verifying the performance of our method.

Given sample x i , for prediction of class c that generated by DNNs can be calculated as:

where p i = C ( G ( x i )) = [ p i 1 , p i 2 , . . . , p iK ] and p ic is the c -th element of p i . Then, given ρ ( p i | x i ) ∼ D ( p i | α i ) , we have ρ ( p ic | x i ) ∼ Beta ( p ic | α ic , α i 0 -α ic ) , where α i 0 = ∑ K k =1 α ik . Thus, we further have:

where B ( · , · ) is the K -dimensional multinomial beta function, and B ( α ic , α i 0 -α ic ) = Γ( α ic )Γ( α i 0 -α ic ) Γ( α ic + α i 0 -α ic ) , Γ( · ) denotes the gamma function. Based on this, we can further obtain the following

derivation:

In our work, the outputs of classifier are adopting the exponential function. Thus, following [53], the outputs of DNNs in SAUE can be viewed as the expectation of Dirichlet distribution.

C Proof of Theorem 1

Theorem 1 [37]. Suppose we have given the m-th source data distribution P S m , a hypothesis set H , and a prior distribution π over the hypothesis space Θ . For any τ ∈ (0 , 1] and λ > 0 , with a probability at least 1 -τ over the source samples S m ∼ P n S m , for all posteriors ρ , we have:

Lemma 1 [38]. The PAC-Bayes bound, involving constants τ and n , as introduced in Theorem 1, is minimized by the Bayesian posterior p ( H ) , which represents the distribution over Θ .

Proof. The Donsker-Varadhan's change of measure states that for any measurable function ϕ : Θ - → R , we have:

Thus, with ϕ ( H ) := λ ( L ( Hˆ L ( θ, S m ) and ∀ ρ over hypothesis space Θ , we have:

For the non-negative random variable ζ π ( S m ) := E π ( H ) [ e λ ( L ( H ) -ˆ L ( H , S m )) ] , we apply Markov's inequality on it, and have:

This impiles that with probability at least 1 -τ over the choice of S m ∼ P n S m , we have ∀ ρ over hypothesis space Θ :

where Ψ S m ,π ( λ, n ) = log E π ( H ) E S m ∼ P n S m [ e λ ( L ( H ) -˜ L ( H )) ] , and we prove the statement of the Theorem 1.

D Generalization Bound

Lemma 2 [39]. Suppose we have given the probability measures ν S m , ν T ∈ P ( F ) of the m -th source feature f S m and the blended-target domain feature f T , a hypothesis space Θ , and a subspace ˜ H ∈ Θ . Let F denote a fixed representation space and c ( f S m , f T ) denote the adaptation cost. For the ideal classifier h ′ ∈ ˜ H and any classifier h ∈ ˜ H with f S m ∼ ν S m and f T ∼ ν T , we have:

where ϵ S m and ϵ T denote the error on the m -th source domain and the error on the blended-target domain respectively, and ϵ T = 1 N ∑ N j =1 ϵ T j . d H ∆ H denotes the H ∆ H -distance.

Proof. By the definition of H ∆ H -distance, we have:

̸

̸

Theorem 2. Based on Lemma 2, with the error of the ideal joint hypothesis η ′ = ϵ S m ( h ′ ) + ϵ T ( h ′ ) which is a sufficiently small constant, for any δ ∈ (0 , 1) , with probility at least 1 -δ , for every h ∈ H , ϵ T ( h ) is bounded by the following terms:

where η ′ = ϵ S m ( h ′ ) + ϵ T ( h ′ ) is the ideal error for the classifier, which is a sufficiently small constant. b ′ is the size of unlabeled samples.

Proof. From Lemma 2, we can obtain the following terms:

Finally, the expected error on the blended-target domain can be bounded by utilizing the expected measures of NWD on the joint distribution of multiple source and blended-target domains.

E Additional Experiment Analysis

In this part, we present more visualization results with extra comparison methods including DANN [20] and AMDA [29]. All experiments are performed on task b+c - → i+p of the ImageCLEF-DA [40].

Distribution Analysis. The t-SNE [54] feature visualization results of extra comparison methods are illustrated in Figure 1. Note that different color dots denote different domains. Compared to ResNet-50, due to the domain adversarial learning, DANN can better align the source and target domains. Furthermore, AMDA achieves better performance through multi-source and multi-target domain features. Due to style adaptation and uncertainty estimation and elimination, SAUE achieves the best performance. The features from the same class generated by SAUE are better clustered while those belonging to different classes are better separated.

Confusion Matrix. The comparison of confusion matrices with extra methods are illustrated in Figure 2. Although DANN and AMDA achieve significantly progress compared to ResNet-50,

(a) ResNet-50

(b) DANN

(d) SAUE

(c) AMDA

Sourcel(b)

Source1(b)

Target1(c)

Target1(c)

Target2(i)

Target2(i)

Source1(b)

Source1(b)

Target1(c)

Target1(c)

Target2(i)

Target2(i)

(a) ResNet-50

(b) DANN

(d) SAUE

(c) AMDA

people

people

people

people

aeroplane

aeroplane

aeroplane

aeroplane

monitor

monitor

monitor

monitor

car

car

car

car

bus

bus

bus

bus

bird

bird

bird

bird

boat

boat

boat

boat

horse

horse

horse

horse

motorbike

motorbike

motorbike

motorbike

bike

bike

bike

bike

6op

6op

6op

6op

bottle

bottle

bottle

bottle

people

aeroplane

monitor

car

snq

bird

horse

bike

6op

bottle

people

aeroplane

monitor

car

snq

horse

bike

6op

bottle

people

aeroplane

monitor

car

snq

bird

horse

bike

6op

bottle

people

aeroplane

monitor

car

snq

horse

bike

6op

bottle

5

class_loss

d_loss

total_loss

3

0

0

2500

5000

7500

10000

01250015000

01750020000

they still misclassified some classes, e.g., class 'bike' is misclassified into class 'motobike'. In contrast, benefiting from the style adaptation and uncertainty estimation, SAUE generates more correct predictions which located on the main diagonal elements of confusion matrix.

Convergence. We further analyze the evolution of different loss functions with increasing iterations. The results are shown in Figure 3. The graphical representation illustrates that all the loss functions in our approach effectively converge as the training iterations increased. This convergence showcases the adaptability and reliability of our approach in MBDA tasks.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims of this paper can be found in the abstract and introduction, which accurately reflect the paper's contributions and scope.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of the work are discussed in the Appendix A.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the theoretical analysis of the proposed method in Section 3.5 and the related proofs on Appendices B and C.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The source code of SAUE is provide in the Supplementary Material.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The source code of SAUE is provide in the Supplementary Material.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The paper specifies all the training and test details, please refereed to Section

4.1.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The paper reports error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments, please refer to Section 4.2.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The paper provides sufficient information on the computer resources, which are included in the Section 4.1.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics in every respect.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The paper discusses both potential positive societal impacts and negative societal impacts of the work in Appendix A.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This submission poses no such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All the code and dataset utilized in this work are publicly available and are only intended to compare the performances of different algorithms on classification tasks.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This submission poses no such risks.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This submission poses no such risks.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This submission poses no such risks.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel challenge in domain adaptation: the problem of Multi-Source Blended Target Domain Adaptation (MBDA). In MBDA, target domain distributions are blended, necessitating a model that can leverage features from multiple source domains to perform effectively on the blended-target domain. To address this problem, a Style Adaptation and Uncertainty Estimation (SAUE) approach is proposed. SAUE utilizes a similarity factor to select style information from the blended-target domain, creating a better representation space. Additionally, it enhances model robustness by employing a Dirichlet-based uncertainty estimation model to handle the diverse distributions introduced by multiple source domains. The effectiveness of the proposed method is validated through extensive image classification experiments. A theoretical analysis of SAUE underscores its potential in solving the MBDA problem.\n\nSTRENGTHS:\n1. The proposed MBDA problem is highly relevant for real-world applications. Unlike existing DA problems, MBDA incorporates confusing domain scenarios that mimic real-world situations where the source domain distributions are various, and the target domain distributions are blended. This assumption adds an additional challenge for DA, as the model needs to align domains without domain labels of the target domain.\n2. The proposed method SAUE is innovative. The inclusion of style adaptation to enhance the source domain features is intriguing, where the similarity-based weighted matrix is cleverly used to select more useful target style information for constructing a better representation space.\n3. The uncertainty estimation component effectively addresses the distribution discrepancy issue introduced by multi-source domains. In addition, the authors construct an adversarial learning strategy that aligns domains without the requirement of domain labels.\n4. The paper is well-written with the theoretical analysis clearly articulated and thoroughly explained.\n\nWEAKNESSES:\n1. The paper mentions various terms related to domain adaptation in Sections 1 and 2, such as UDA, SSDA, MSDA, MMDA, MTDA, and BTDA. What are the main differences and relationships among these terms? The authors are encouraged to provide a table summarizing the main characteristics to distinguish these terms.\n2. In the ablation study, the comparison primarily focuses on highlighting the improvements of each proposed module. It is suggested that the authors compare different feature augmentation methods to further emphasize the superiority of the proposed Style Adaptation module.\n3. A minor point of interest is how the proposed method performs on other backbones, such as the Vision Transformer.\n\nQUESTIONS:\nPlease refer to the Weaknesses part.\n\nLIMITATIONS:\nThe authors have adequately claimed their limitations in Appendix B.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-22"}

comments: SUMMARY:
This paper works on Multi-source Blended-target Domain Adaptation setting, which learns a model from multiple source domain and evaluates the model in a mixed multi-target domain without access to the domain labels of target data. This paper utilize the style information of the blended-target domain with weight factor to enhance source domain features for feature augmentation. To avoid the negative impact of domain-specific information in multi-source data, this paper also uses KL loss to reduce the impact of incorrectly classified source samples. This paper also proposes a domain adversarial loss with the help of nuclear-norm 1-Wasserstein discrepancy to reduce the domain gap between source domain and target domain by reusing the classifier of task network. Extensive experiments on various benchmark and comprehensive analysis prove the effectiveness of the proposed method.

STRENGTHS:
1. This paper is well motivated.
2. The proposed method is evaluated with various benchmark and analyzed extensively.

WEAKNESSES:
1. The style adaptation method is not compared with prior works proposed for the same purpose like MixStyle.
2. Whether Domain Adversarial Alignment without Domain Labels outperform domain adversarial learning by treating multi-source data as one single source domain and multi-target data as one target domain is not discussed.

QUESTIONS:
Please see weakness part.

LIMITATIONS:
No limitation is discussed in the paper.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a \"Style Adaptation and Uncertainty Estimation (SAUE)\" method for Multi-source Blended-Target Domain Adaptation (MBDA). The core objective of the SAUE method is to enhance the source domain features by leveraging style information from a blended-target domain, which is a mixture of multiple sub-target domains. The method employs a similarity factor to select beneficial target style information and integrates an uncertainty estimation technique to fortify the model's robustness. The proposed SAUE is evaluated on several domain adaptation benchmarks, including ImageCLEF-DA, Office-Home, and DomainNet, demonstrating superior performance over existing state-of-the-art techniques.\n\nSTRENGTHS:\nIt is demonstrated an impressive results on several domain adaptation benchmarks with a significant improvement.\nThe paper is well-organized and easy to follow.\n\nWEAKNESSES:\n1. Although the blended-target domain adaptation has attracted much attention, it is similar to the open-compound domain adaptation that has no annotation of the target domain.  Authors should compare and discuss the difference between the open-compound domain adaptation works.\n\n2. The proposed style adaptation and uncertainty estimation are motivated and effective ways to handle domain shift problems. However, the idea has been well studied in conventional domain adaptation [a], domain generation [b], and open-compound domain adaptation fields. Therefore, the proposed method seems to lack innovation. \n\n    [a] Informative Data Mining for One-shot Cross-Domain Semantic Segmentation\n\n    [b] Uncertainty modeling for out-of-distribution generalization\n\n    [c] Open Compound Domain Adaptation with Object Style Compensation for Semantic Segmentation\n\n3. In my opinion, the proposed style adaptation technique is not tailored for multi-source BTDA as it also works for single-source BTDA. Therefore, the statement \"the first work for MBDA\" seems to be weak.\n\nQUESTIONS:\nDetailed questions refer to the weaknesses.\n\nLIMITATIONS:\nThe limitations of the work have been discussed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the setting of Blended target domain adaptation. The paper proposes style adaptation and uncertainty estimation for multi-source blended target domain adaptation. They propose to utilize the extra knowledge acquired from the blended target domain. Where a similarity factor is utilized to obtain target style information to augment the source features. During training, they utilize the information from the target domain with a scaling factor and Wasserstein distance between the source and target features. Moreover, they do not utilize the domain labels without which they claim to align the multi-source and blended target domains. They utilize common domain adaptation datasets to demonstrate the effectiveness of their method.\n\nSTRENGTHS:\n– The problem setting of addressing multi-target distribution shifts with a source trained model is useful \n\n– Performance improvement is significant compared to the baseline methods.\n\nWEAKNESSES:\nThe paper lacks extensive validation, clarity, and novelty. Elaborated below.\n\nQUESTIONS:\n– Can the authors demonstrate with the default version of DomainNet instead of utilizing a subset of it? Moreover, the inclusion of the VisDA dataset would also be useful. \n\n– Could the authors also provide detailed numerical results of Figures C and D for better clarity? \n\n– The idea of using the target and source features during the training process with a scaling factor and Wasserstein distance doesn’t seem novel. Moreover the paper doesn't cite [1][2] which makes the related work quite limiting. \n\n– The idea of selecting style information is not new. Moreover, it is not clear which layer has been utilized for the style information and what the motivation is to utilize the style content at a particular layer. \n\n– The distribution analysis can be moved to the appendix, which doesn’t offer significant insights of the method. \n\n– Akin to MCDA [12], could the authors utilize ResNet 101 instead of ResNet50 and compare the results for the default version of the DomainNet dataset? With powerful backbones, it would be interesting how the style features are still preserved. \n\n– Extensive implementation details have not been provided; the batch size of the blended target domain, learning rates, and which layers from the ResNet model have been utilized for this if their code will be released. Does the method necessitate a high batch size for the target domain? In this case, it might be a limitation.\n\nReferences:\n[1] Shen, Jian, et al. \"Wasserstein distance guided representation learning for domain adaptation.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 32. No. 1. 2018.\n[2] She, Qingshan, et al. \"Improved domain adaptation network based on Wasserstein distance for motor imagery EEG classification.\" IEEE Transactions on Neural Systems and Rehabilitation Engineering 31 (2023): 1137-1148.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-14"}