Augmented Self-Labeling for Source-Free Unsupervised Domain Adaptation
Abstract
Unsupervised domain adaptation aims to learn a prediction model that generalizes well on a target domain given labeled source data and unlabeled target data.
However, source data sometimes can be unavailable due to data privacy or decentralized learning architectures.
In this paper, we address the source-free unsupervised domain adaptation problem where only the pretrained source model and unlabeled target data are given.
To this end, we propose an Augmented Self-Labeling (ASL) method that jointly optimizes the prediction model and the pseudo-labels for the target data starting from the initial source model.
It involves two alternating steps: augmented self-labeling improves pseudo-labels by solving an optimal transport problem with the Sinkhorn-Knopp algorithm, and model re-training trains the model with the supervision of improved pseudo-labels.
We further introduce model regularization terms to improve the model re-training.
Experiments show that our method achieves comparable or better results than the state-of-the-art methods on the standard benchmarks.
Introduction
Unsupervised domain adaptation tackles the setting where labeled source data and unlabeled target data are available when adapting to target domains.
However, source domain data might be inaccessible in some privacy-sensitive applications.
For example, federated learning collaboratively trains a model using decentralized data on mobile devices without fetching data into a centralized machine [2]. When adapting the model trained via federated learning, we have no access to the source data. This induces the source-free unsupervised domain adaptation setting, where only the trained source model and the unlabeled target data are given. Traditional unsupervised domain adaptation methods are not applicable to this setting because they usually seek to align distributions of source and target domains with data samples from both domains.
A few methods that tackle this source-free unsupervised domain adaptation has recently been proposed in the literature. For example, SHOT [16] alternately reﬁnes the pseudo-labels of target data with a prototype classiﬁer and ﬁne-tunes the feature extractor together with a model regularization term to maximize the mutual information between model inputs and outputs. PPDA [11] assigns pseudo-labels based on a prototype classiﬁer and a sample-level re-weighting scheme. 3C-GAN [15] and SDDA [12] utilize the conditional GAN to generate labeled target data through input-level adversarial training.
In this paper, we propose a new Augmented Self-Labeling (ASL) method for the source-free unsupervised domain adaptation problem.
It involves two alternating steps: the augmented self-labeling step aims to improve the pseudo-labels of target data and the model re-training step retrains the target model with the self-labeled target data. These two steps work together to gradually adapt the prediction model by incorporating the target data into training.
In particular, we propose to ensemble the predicted probabilities corresponding to multiple randomly augmented versions of the same sample for self-labeling.
We derive this problem into an instance of the optimal transport problem.
In order to avoid the degenerate solution, we add the equipartition constraint on the labels and solve the problem efﬁciently via a fast version of the Sinkhorn-Knopp algorithm [5].
Moreover, we also introduce several model regularization terms to improve the model re-training.
We apply the proposed ASL method to the source-free unsupervised domain adaptation tasks.
Experiments show that our method can achieve comparable or even better results than the state-of-the-art methods on the standard benchmarks.
Augmented Self-Labeling (ASL) for Source-Free Domain Adaptation
This paper tackles the source-free unsupervised domain adaptation problem where only the pretrained source model and the unlabeled target data are available.
Speciﬁcally, we assume the source prediction model f s :
X s → Y s trained in the source domain and the unlabeled target data { x i } Ni =1 are given.
The goal is to produce a prediction model f that works well on the target data.
We propose to adapt the source model into the target domain to induce a good target prediction model f by alternatively performing pseudo-label reﬁnement on the target data through augmented self-labeling and conducting model retraining with the reﬁned pseudo-labels.
2.1 Augmented Self-Labeling
We initialize the target model f with the parameters of the source model f s .
Given the unlabeled target data { x i } Ni =1 , their pseudo-labels can be obtained by choosing the highly conﬁdent predictions based on the current model f , which are further used to ﬁne-tune the model in an alternative manner.
Due to the existence of domain discrepancy, pseudo-labels for target data can be noisy which may lead to error accumulation in the target model.
We thus propose an Augmented Self-Labeling method to optimize the target labels from the weighted average of multiple output predictions corresponding to samples with random data augmentations.
Speciﬁcally, M different augmented version of samples { x mi } Mm =1 can be obtained from the original sample x i by independently applying random data augmentations M times, i.e.
[Equation]
where RandAugment ( · ) denotes a combination of multiple random data augmentations.
The data augmentations we used include random resized crop, random auto-contrast and random color distortion [3].
In order to reduce the noise in the predicted probability, we take the ensemble of the M + 1 probabilities corresponding to the M augmented version and the unaugmented version of each sample x i to get the following average prediction outcome, which indicates the probability of the sample x i belonging to class y ,
[Equation]
Here half weight is assigned to the predicted probability on the unaugmented version of the sample. The reason is that the original data samples still carry most of the useful information and higher weights assigned to the original samples can make the prediction more stable and reliable.
Next, we propose to improve the pseudo-labels through the following augmented self-labeling:
[Equation]
where p iy is the augmented prediction term in Eq.(2), K is the number of classes, and { q iy } denote the soft-labels we aim to produce after taking the overall prediction situation into consideration through constraints.
The negative entropy term in the objective is added to get smoothed soft-labels while the hyperparameter λ controls the degree of smoothness.
Following [25], the equipartition constraints, { (cid:80) i q iy = N/K, ∀ y } , are added to avoid degenerate solutions where the same arbitrary label is assigned to all the samples.
The constraints enforce that each category contains similar number of samples, which is reasonable in class-balanced datasets.
In general, it can be explained as enforcing maximization over the mutual information between the sample indices and labels [25].
This augmented self-labeling problem for pseudo-label reﬁnement is actually an instance of the optimal transport problem [5].
To make it more clear, we convert the notations in Eq.(3) to matrix form by introducing Q with [ Q ] iy = q iy as the label matrix with dimension of N × K and P with [ P ] iy = p iy as the predicted probability matrix with dimension of N × K .
The objective in Eq.
(3) can be rewritten as:
[Equation]
where (cid:104)·(cid:105) denotes the Frobenius inner product, i.e. the sum of element-wise product between two matrices.
The matrix Q is thus constrained to be an element of the transport polytope [5],
[Equation]
where r = 1 N , and c = N K 1 K ; 1 d denotes a column vector with all 1 values and length d .
This is equivalent to the constraints of labels in Eq.
This problem can be solved via the fast version of Sinkhorn-Knopp algorithm [5], which has the following form of solution:
[Equation]
where u, v are vectors guaranteeing the constraints in Eq.
(5) and can be computed with the Sinkhorn’s ﬁxed point iteration until convergence:
[Equation]
Speciﬁcally, we initialize v as normalized unit vector and then iteratively compute u and v until v converges.
In practice, this iteration can converge in a few steps.
After pseudo-label reﬁnement through the augmented self-labeling procedure above, we convert the soft-labels { q iy ∈ [0 , 1] } into hard-labels { ˆ q iy ∈ { 0 , 1 }} by setting the pseudo-label of each instance as the class label with the maximum probability.
Then we perform model re-training to update the target model f ( · ; θ ) by minimizing the following cross-entropy loss:
[Equation]
In order to learn a target prediction model that generalizes well, we further incorporate several model regularization terms into the model re-training process.
First, we adopt the following conditional entropy minimization [8] loss as a regularization term:
[Equation]
This conditional entropy regularization term is suitable for exploiting unlabeled data in a semisupervised manner, and it pushes the decision boundary far away from the data dense regions to support the cluster assumption [22].
However, the conditional entropy above is empirically estimated using the available target data.
According to [8, 22], this approximation holds only if the model is locally-Lipschitz.
Therefore, we further add the following virtual adversarial loss [18] as a regularization term to guarantee the locally-Lipschitz constraint:
[Equation]
where D KL ( ·(cid:107)· ) is the Kullback-Leibler Divergence and f ( x ) = [ p (1 | x ; θ ) , · · · , p ( K | x ; θ )] . [Table]
Moreover [Table] as the optimal classiﬁer would generalize well on both domains according to the theoretical analysis in [1]
[Table]
we add the following weight regularization term to prevent excessive deviation from the source model parameters θ s :
[Equation]
This weight regularization not only prevents the target hypothesis from getting far away from the initial source model and helps to preserve the source knowledge in the target model, but also stables the re-training of the target model during alternative updates.
Finally, by integrating these regularization terms into the cross-entropy loss in Eq.
(8), we have the following overall loss function for model re-training:
[Equation]
where λ 1 and λ 2 are trade-off parameters.
The entropy loss and virtual adversarial loss empirically share the same trade-off parameter [22, 15].
Experiments
We conduct experiments on the standard benchmarks under the source-free unsupervised domain adaptation setting.
We compare our ASL method 1 with the state-of-the-art source-free domain adaptation methods and report the comparison results.
Datasets We evaluate our method on the following benchmark datasets.
Ofﬁce-31 [20] is a standard small-sized visual domain adaptation benchmark which contains images of 31 categories from three domains: Amazon ( A ), DSLR ( D ) and
Webcam ( W ), each containing 2817, 498 and 795 images respectively.
VisDA-2017 [19] is a large-scale synthetic-to-real dataset with images in 12 categories from two domains, Synthetic and Real , each consists of 152,397 and 55,388 images respectively.
Implementation Details We use the same network architectures as the previous methods to achieve fair comparisons.
On Ofﬁce-31 , we use ResNet-50 [9] as the backbone network.
ResNet- [9] is utilized as the backbone module on VisDA-2017 .
The target model is optimized using the mini-batch SGD algorithm where batch size is set to be 32.
The learning rate is ﬁxed to be 10 −for the backbone, and 10 − 3 for the bottleneck and FC layers.
For trade-off parameters, we set λ = 2 , λ 1 = 1 , λ 2 = 0 .
1 , M = 4 for the Ofﬁce-31 dataset and λ = 100 , λ 1 = 1 , λ 2 = 0 .
01 , M =for the VisDA-2017 dataset.
Results on Ofﬁce-31 Table 1 reports the results of different methods on the six domain adaptation tasks of this small sized dataset, where the top section includes results of the source-only and unsupervised domain adaptation (UDA) methods that use the source domain data and the bottom section presents the comparison results of the source-free unsupervised domain adaptation methods. We can see that our proposed ASL method outperforms most previous UDA methods that exploit the source data.
Among the source-free UDA methods, our method achieves better performance than
SHOT [16] and SDDA [12], and achieves similar performance as 3C-GAN [15], while 3C-GAN uses generative models to generate lots of labeled target data and is time-consuming and resourceextensive.
In particular, our ASL method achieves the state-of-the-art performance on the ﬁrst three [Table] [Table] tasks, i.e. A → D, A → W and D → A, under the source-free unsupervised domain adaptation setting.
These results demonstrate the effectiveness of the proposed augmented self-labeling method. Results on VisDA-2017 Table 2 reports the test accuracy results for different methods on this large scale benchmark.
Again the top section reports the results of source-only and UDA methods and the bottom section reports the results of source-free UDA methods. We can see our ASL method achieves the state-of-the-art performance under the source-free unsupervised domain adaptation setting: it outperforms PPDA [11], SHOT [16] and 3C-GAN [15] in most classes and produces the best per-class average result.
It also outperforms most previous unsupervised domain adaptation methods that exploit source data.
This again validates the efﬁcacy of the proposed ASL method. Ablation Study As shown in Table
[Table]
we investigated the contribution of different components of the proposed method and compared the variants with the naive pseudo-labeling (PL) method [14], which directly ﬁne-tunes the source model with the pseudo-labeled target data. We can see that both self-labeling (SL) and augmented self-labeling (Asl) can easily outperform the naive PL method with or without the model regularization terms. The model regularization terms can improve the performance of all the methods: naive PL, SL, and Asl.
Nevertheless, our proposed augmented self-labeling can promote the results by a large margin comparing with the self-labeling.
This demonstrates the effectiveness of all the components of our overall ASL method for source-free unsupervised domain adaptation.  Conclusion
In this paper, we proposed a new Augmented Self-Labeling method for the source-free unsupervised domain adaptation, where only the source model and the unlabeled target data are available.
We treated this problem as a joint optimization over the pseudo-labels and the prediction model, and solved it with two alternating steps, where augmented self-labeling improves the pseudo-labels and re-training improves the model with the self-labeled target data.
We exploited data augmentation to improve the self-labeling quality by ensembling multiple prediction probability matrices corresponding to the augmented versions of samples. We also deployed model regularization terms to help model re-training.
Experiments on the benchmarks validated the effectiveness of our proposed method for source-free unsupervised domain adaptation.