Cycle Self-Training for Domain Adaptation
Authors: Hong Liu
Jianmin Wang
Mingsheng Long
Venue: NeurIPS 2021 Poster (2021)
Keywords: Domain Adaptation
Self-Training
Theoretical Analysis

Abstract: Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift, which are empirically effective but theoretically challenged by the hardness or impossibility theorems. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. In this paper, we propose Cycle Self-Training (CST), a principled self-training algorithm that explicitly enforces pseudo-labels to generalize across domains. CST cycles between a forward step and a reverse step until convergence. In the forward step, CST generates target pseudo-labels with a source-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations to make the target classifier perform well on the source data. We introduce the Tsallis entropy as a confidence-friendly regularization to improve the quality of target pseudo-labels. We analyze CST theoretically under realistic assumptions, and provide hard cases where CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over the state-of-the-arts on visual recognition and sentiment analysis benchmarks.

Cycle Self-Training for Domain Adaptation
Hong Liu Dept of Electronic Engineering
Jianmin Wang
Mingsheng Long"
School of Software, BNRist Tsinghua University
School of Software, BNRist Tsinghua University
Tsinghua University
honglin9903@gmail.com
j jimeengtteinghua.. edu.
mingsheng@tsiinguu. edu.
Abstract
Mainstream approaches for unsupervised domain adaptation (UDA) learn domain- invariant representations to narrow the domain shift, which are empirically effective but theoretically challenged by the hardness or impossibility theorems. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. In this paper, we propose Cycle Self- Training (CST), principled self-training algorithm that explicitly enforces pscudo-label labelss to generalize across domains. CST cycles between forward step and a reverse step until convergence. In the forward step, CST generates target pscudo-labels with a source surrce-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations make the target classifier perform well on the source data. We introduce the Tsallis entropy as a manfidence-friendee regularization improve the quality of target pseudo-labels. We analyze CST theoretically under realistic assumptions, and provide hard cases where CST recovers target ground truth, while both invariant feature learning and vanilla self- self-training fail. Empirical results indicate that CST significantly improves over the state-of-the-arts on visual recognition and sentiment analysis benchmarks.
Introduction
Transferring knowledge from source domain with rich supervision to unlabeled target domain is an important yet challenging problem. Since deep neural networks are known to be sensitive to subtle change in underlying distributions [70], models trained one labeled dataset often fail to generalize to another unlabeled dataset [58, 1]. Unsupervised domain adaptation (UDA) addresses the challenge of distributional shift by adapting the source model the unlabeled target data [50, 43).
The mainstream paradigm for UDA is feature adaptation, a.k.a. domain alignmens. By reducing the distance of the source and target feature distributions, these methods learn invariant representations' to facilitate knowledge transfer between domains [34, 22, 36, 54, 37, 73], with successful applications in various areas such as computer vision [63, 27, 77] and natural language processing [75, 49). Despite their popularity, the impossibility theories [6] uncovered intrinsic limitations of learning invariant representations when it comes to label shift [74, 32] and shift in the support of domains [29].
Recently, self-training (a.k.a. pseudo-labeling) [21, 78, 30, 32, 47, 68] has been gaining momentum as a promising alternative to feature adaptation. Originally tailored to semi-supervised learning, self-training generates pseudo-labels of unlabeled data, and jointly trains the model with source labels and target pseudo-labele [31, 39, 30]. However, the distributional shift in UDA makes pseudo-labe more difficult. Directly using all pseudo-labels risky due to accumulated error and even trivial solution [14]. Thus previous works tailor self-training to UDA by selecting trustworthy pseudo-labels. Using confidence threshold or reweighting, recent works try to alleviate negative effect of domain
"Corresponding author: Mingsheng Long (mingshenggtainnguua. edu. cn)
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Standard Self-Treee -rainirin
Cycle Self- Self-Training
Target
Target Classifer
Loss
0 (x/
Targor Lass
@ [xf]
Jeesy (0..
he
go
Rep
Loss
Source
- (4000
Loss
Reatures Cheotfier
Shared
Source
L2 +)
Source
Pratures
Classffer
Loss
Forward
Updatte
25
Label
Forward
looge
Outer looge
Label Sharpening
Sharpenking
updatte R2
update andd 6,
Figure 1: Standard self- self-training vs. cycle self-training In standard self training, we generate target pseudo- labels with source model, and then train the model with both source ground-truths and target pseudo-labels. In cycle self-training. we train a target classifier with target pseudo-labels in the inner loop, and make the target classifier perform well on the source domain by updating the shared representations in the outer loop.
shift in standard self-training [78, 47], but they can be brittle and require expensive tweaking of the threshold weight for different tasks, and their performance gain is still inconsistent.
In this work, we first analyze the quality of pseudo-labels with or without domain shift to delve deeper into the difficulty of standard self-training in UDA. On popular benchmark datasets, when the source and target are the same, our analysis indicates that the pseudo-label distribution is almost identical to the ground-truth distribution. However, with distributional shift, their discrepancy very large with examples of several classes mostly misclassified into other classes. We also study the difficulty of selecting correct pseudo-labels with popular criteria under domain shift. Although entropy and confidence are reasonable selection criteria for correct pseudo-labels without domain shift, the domain shift makes their accuracy decrease sharply.
be
Our analysis shows that domain shift makes pseudo-labels unreliable and that self self-training on selected target instances with accurate pseudo-labels is less successful. Thereby, more principled improvement of standard self-training should be tailored to UDA and address the domain shift explicitly. this work, we propose Cycle Self-Training (CST), principled self-training approach to UDA, which overcomes the limitations of standard self-training (see Figure 1). Different from previous works to select target pseudo-labed with hard-to-twedt protocols, CST learns to generalize the pseudo-labels across domains. Specifically, CST cycles between the use of target pseudo-labell to train a target classifier, and the update of shared representations make the target classifier perform well on the source data. In contrast to the standard Gibbs entropy that makes the target predictions over-confident, we propose a wwaffddence-rriendll uncertainty measure based on the 7 Tsallis entropy in information theory, which adaptively minimizes the uncertainty without manually tuning or setting thresholds. Our method is simple and generally applicable vision and language tasks with various backbones.
We empirically evaluate our method on series standard UDA benchmarks. Results indicate that CST outperforms previous state-of- of-the-art methods in 21 out of 25 tasks for object recognition and sentiment classification. Theoretically, we prove that the minimizer of CST objective is endowed with general guarantees of target performance. We also study hard cases specific distributions, showing that CST recovers target ground-truths while both feature adaptation and standard self-training fail.
Preliminaries
We study unsupervised domain adaptation (UDA). Consider a source distribution P' and target distribution Q over the input-label space X' x y. We have access to n, labeled i. samples P = [=f, yf ) from P and n, unlabeled i..dd samples Q = {x{ from Q. The model f comprises a feature extractor ha parametrized by & and head (linear classifier) ge parametrized by ie. (x) ge (h)(z)). The loss function is f(..). .). Denote by (0 o) E(s.a)-r&
v)
the expected error on P. Similarly, we use p Φ) to denote the empirical error on dataset P.
We discuss two mainstream UDA methods and their formulations: feature adaptation and self-training.
Feature Adaptation trains the model f on the source dataset P, and simultaneously matches the source and target distributions in the representation space Z h(x):
min Lp(0, Φ) + 8,0
(1)

---

Teap gradite
Pscudo labels
Total Variation Belween
Pseudo labels and Ground Truth
Epochs
Analysis of pseudo-labels under domain shift on VisDA-2017. Left: Pseudo-label distributions with and without domain shift. Middle: Changes of pseudo-label distributions throughout training. Right: Quality of pseudo-labels under different pseudo-label selection criteria.
Figure 2:
Here, hyP denotes the pushforward distribution of P, and d(: is some distribution distance. For instance, Long et al. [34] used maximum mean discrepancy dMMD, and Ganin et al. [22] approximated the H-AA-disttnce dHAH [7] with adversarial training. Despite its pervasiveness, recent works have shown the intrinsic limitations of feature adaptation under real-world situations [6, 74, 33, 32, 29].
Seff-Trainnn considered promising alternative to feature adaptation. In this work we mainly focus on pscudo-labeling [31, 30J. Stemming from semi-supervised learning, standard self-training trains a source model fa the source dataset P: mine, ,.. (p(... o.). The target pseudo-labels are then generated by fa on the target dataset Q. To leverage unlabeled target data, self- self-training trains the model on the source and target datasets together with source ground-truths and target pseudo-labels:
min (p + E (f arg max 0,0
(z)1g})
(2)
Self-trand belf-training also uses label-sharpening as standard protocol [31, 57]. Another popular variant of pseudo-labeling is the teacher-student model [4, 61], which iteratively improves the quality of pseudo-labels via alternatively replacing 0, and Φ, with 0 and & of the previous iteration.
2.1 Limitations of Standard Self-Training
Standard self-training with pseudo-labels uses unlabeled data efficiently for semi- semi-supervised learn- ing [31, 39, 57J. Here we carry out exploratory studies on the popular VisDA-2017 [45] datasct using ResNet-50 backbones. We find that domain shift makes the pseudo-labels biased towards several classes and thereby unreliable in UDA. See Appendix C.1 for details and results on more datasets.
Pseudo-label distributions with or without domain shift. We resample the original VisDA-200 2017 to simulate different relationship between source and target domains: 1) i.i.d., 2) covariate shift, and 3) label shift. We train the model on the three variants of source dataset and use it to generate target pseudo-labels. We show the distributions of target ground-truths and pseudo-labels in Figure (Left). When the source and target distributions are identical, the distribution of pseudo-labels is almost the same as ground- truttss, indicating the reliability of pseudo-labels. In contrast, when exposed to label shift or covariate shift, the distribution of pseudo-labels is significantly different from target ground bround-truths. Note that classes 2, 7, 8 and 12 appear rarely in the target pseudo-labels in the covariate shift setting, indicating that the pseudo-labed Self- Self-training with these pseudo-labela is risky since it may lead to misalignment of distributions and misclassify many examples of classes 2, 7, 8 and 12.
are biased towards several classes due to domain shift.
Change of pseudo-label distributions throughout training. To further study the change of pseudo- labels in standard self-training, we compute the total variation (TV) distance between target ground- truths and target pscudo-labels: drv(c, c') 1/2 Σ, ||q. - d1l, where e, is the ratio of class We plot its change during training in Figure 2 (Middle). Although the error rate of ascudo-labels continues to decrease, dry remains almost unchanged at 0.26 throughout training. Note that dry is the lower bound of the error rate of the pseudo-labels (shown in Appendix C) If dry converges to 0.26, then the accuracy of pseudo-labels is upper-bounded by 0.74. This indicates that the important denoising ability [66] of pseudo-labes labels in standard self-training is hindered by domain shift.
Difficulty selecting reliable pseudo-labels under domain shift. To mitigate the negative effect of false pseudo-labels, recent works proposed to select correct pseudo-labels based on thresholding the entropy or confidence criteria [35, 21, 37, 57]. However, it remains unclear whether these strategies still effective under domain shift. Here compare the quality of labels selected by
3

---

different strategies with or without domain shift. For each strategy, we compute False Positive Rate and True Positive Rate for different thresholds and plot its ROC curve Figure (Right.. When the source and target distributions are identical, both entropy and confidence are reasonable strategies for selecting correct pseudo-labels (AUC=0.89). However, when the target pseudo-labels are generated by the source model, the quality of pseudo-labels decreases sharply under domain shift (AUC=0.78).
3 Approach
We present Cycle Self-Training (CST) to improve pscudo-labels under domain shift. An overview of our method is given in Figure 1. Cycle Self-Training iterates between a forward step and reverse step to make self-trained classifiers generalize well on both target and source domains.
3.1 Cycle Self-Training
Forward Step. Similar to standard self-training, we have a source classifier O, trained on top of the shared representations & on the labeled source domain, and use it to generate target pseudo-labels as
y' arg max (
(3)
for each in the target dataset @. Traditional self-training methods use confidence thresholding or reweighting to select reliable pseudo-labels. For example, Sohn et al. [57] select pseudo-labels with softmax value and Long et al. [37] add entropy reweighting to rely on examples with more confidence prediction. However, the output of deep networks is usually miscalibrated [25], and is not necessarily related to the ground-truth confidence even the same distribution. In domain adaptation, as shown in Section 2.1, the discrepancy between the source and target domains makes pseudo-labels even more unreliable, and the performance of commonly used selection strategies is also unsatisfactory. Another drawback is the expensive tweaking in order to find the optimal confidence threshold for new tasks. To better apply self-training to domain adaptation, we expect that the model can gradually refine the pseudo-labels by itself without the cumbersome selection or thresholding.
Reverse Step. We design a complementary step with the following insights to improve self-training. Intuitively, the labels on the source domaín contain both useful information that can transfer to the target domain and harmful information that can make pseudo-labels incorrect. Similarly, reliable pseudo-labels on the target domain can transfer to the source domain in turn, while models trained with incorrect pseudo-labels on the target domain cannot transfer to the source domain. In this sense, we explicitly train the model to make target pseudo-labels informative of the source domain, we can gradually make the pseudo-labels more accurate and learn generalize to the target domain.
Specifically, with the pseudo-labels y' generated by the source classifier 9₂ at hand as in equation 3. we train a target head 0r (d) on top of the representation e with pseudo-labels on the target domain Q.
(b))) = arg min E. (
(4)
We wish to make the target pseudo-labels informative of the source domain and gradually refine them. To this end, we update the shared feature extractor 6 to predict accurately on the source domain and jointly enforce the target classifier (() to perform well on the source domain. This naturally leads to the objective of Cycle Self-nrainin:
minimize Lcycte(0...) (). == Lp(0.. () + L) ()(
(5)
0g.6
Bi-level Optimization. The objective in equation 5 relies on the solution ( to the objective in equation 4. Thus, CST formulates a bi-level optimization problem. In the inner loop we generate target pseudo-labels with the source classifier (equation 3), and train a target classifier with target pseudo-labels (equation 4). After each inner loop, we update the feature extractor o for one step in the outer loop (equation 5), and start a new inner loop again. However, since the inner loop of the optimization in equation 4 only involves the light-weight linear head 0,, we propose to calculate the analytical form of (o) and directly back- propagate to the feature extractor ó instead of calculating the second-ords -odder derivatives as in MAML [18]. The resulting framework is as fast as training two
heads jointly. Also note that the solution (d) relies on 0, implicitly through y/. However, both standard self-training and our implementation use label sharpening, making s/ not differentiable. Thus we follow vanilla self-training and do not consider the gradient of (Ø) w.r.t. y in the outer loop optimization. We defer the derivation and implementation of bi-level optimization to Appendix B.2.

---

3.2 Tsallis Entropy Minimization
Gibbs entropy is widely used by existing semi-supervised learning methods to regularize the model output and minimize the uncertainty of predictions on unlabeled data [24]. In this work, we generalize Gibbs entropy to Tsallis entropy [62] in information theory. Suppose the softmax output of a model y E IR then the a-Tsallis entropy defined as
Sa(v) a (1- Mv)
(6)
where > 0 is the entropic-index. Note that lima-11 Sa = =: -Vyloo(()))) which exactly recovers the Gibbs entropy. When 2, (1) becomes the Gini impurity - Vi
We propose to control the uncertainty of target pseudo-labels based on Tsallis entropy minimization:
Q Tsalliss (0, B 2 (ao,
Figure 3 shows the change of Tsallis entropy with differ- ent entropic-indices a for binary problems. Intuitively, smaller a exerts more penalization dictions and larger a allows several scores y/ 's to be similar. This is critical in self-training since overly small a (as in Gibbs entropy) will make the incorrect di- mension of pseudo-labels close and have no chance to be corrected throughout training. In Section 5.4, we further verify this property with experiments.
uncertain pre-
Paa
d=1 (ttaddad Entrppy)
a=12
a=13 13
3.
am 2 (5 Impurtt
0.0
0.2
56
08
10
Figure 3: Tsallis entropy vs, entropic-index a.
An important improvement of the Tsallis entropy over Gibbs entropy is that can choose the suitable measure of uncertainty for different systems to avoid over-confidence caused by overly penalizing the uncertain pseudo-labels, To automatically find the suitable a, we adopt similar strategy as train the source classifier
Section 1. The intuition is that if use the suitable entropic-index Os.a. the target pseudo-labels generated by 0s.a will contain desirable knowledge of the source dataset, ie. a target classifier 6t, trained with these pseudo-labels will perform well on the source domain. Therefore, we semi- cuni-supervisedly train a classifier ê on the source domain with the a- --aalllis entropy regularization 1-2 Tsallsss on the target domain as: 0z.a arg ming p T.p (0,, Φ) + ( @TTalllssss (0, Φ). from which we obtain the target pseudo-labels. Then we train another head θ,,c with target pseudo-labels. We automatically find by minimizing the loss of Ör, the source data:
à arg min (O,(0,,
(10)
aE[1,2]
To solve equation 10, we discretize the feasible region [1. 2] of and use discrete optimization to lower computational cost. We also update the start of each epoch, since we found more frequent
Algorithm 1 Cycle Self Self-Training (CST)
1: Input: source dataset P and target dataset Q.
2: for epoch
to MaxEpoch do
3: Select & as equation 10 at the start of each epoch.
4: for = 0 to MaxIter do
5: Forward Step
6: Generate pscudo-labels on the target domain with & and 0,: y/ arg max, (z))a)
7: Reverse Step
8: Train a target head 0, (0)) with target pseudo-labels y' on the feature extractor di
(,()) arg min (z-g(([o,ss
9: Update the feature extractor & and the source head 0, to make ((()) perform well on the source dataset and minimize the á-T'slllis entropy the target dataset:
v/ (0. Φ)) + Lp(0, ())))(())))) Φ) + @. (0., ()]
(7)
02 - V, (0.. Φ) + LQ (0., Φ)].
(8)
10: end for
11: end for
5

---

update leads to no performance gain. Details are deferred to Appendix B.3. Finally, with the optimal found, we add the â-Tsallis entropy minimization term L T the overall objective:
minimize Lcyete (0x, x) + L T (0s, +).
(11)
0,.9
In summary, Algorithm depicts the complete training procedure of Cycle Self-Training (CST).
Theoretical Analysis
We analyze the properties of CST theoretically, First, we prove that the minimizer of the CST loss (fsr(fa. fe) will lead to small target loss Eng (fa) under simple but realistic expansion assumption. Then, we further demonstrate a concrete instantiation where cycle self-training provably recovers the target ground truth, but both feature adaptation and standard self-training fail. Due to space limit, we state the main results here and defer all proof details to Appendix A.
4.1 CST Provably Works under the Expansion Assumption
We start from a K K-way classification model, f X - [0, 1] E F and f(x) == arg max; f(x) ja] denotes the prediction. Denote by P. the conditional distribution of P given y = Assume the supports of P, and P3 are disjoint for i * j. The definition is similar for Qi. We further Assume = P) = Qyy = i). For any x X. N(xx) is defined as the neighboring set of a with a proper metric ,, = {x': : d(æ, x²) I E). ((A) == (z)). Denote the expected error on the target domain by (fr := ((xxt))) * y).
We study the CST algorithm under the expansion assumption of the mixture distribution [66, 11]. Intuitively, this assumption indicates that the conditional distributions P. and Q; are closely located and regularly shaped, enabling knowledge transfer from the source domain to the target domain.
Definition ((4, -)oontttn expansion [66]). We say P and Q satisfy (q, cooonstant expansion for some constant 9, E (0, 1). for any set E X and any i E [K] with > ()(P)+Q.)(A) > q, we have P4 (()
> minfe, (A)))
Based on this expansion assumption, consider robustness-consstraineee version of CST. Later we will show that the robustness is closely related to the uncertainty. Denote by fo the source model and ft the model trained on the target with pseudo-labels. Let R(fi) == P)(P+Q) ((x 3x' e
f.)) + fi(x')) represent the robustness [66] of fe on P and Q. Suppose ((a.))-"((..()) + fx)))) S e and R(fe)= S p. The following theorem states that when Sss and ft behave similarly on the target domain Q and fe is robust to local changes in input, the minimizer of the cycle source error (fry(ft) will guarantee low error of fs on the target domain Q.
Theorem 1.
Suppose Definition I holds for P and Q. For any far Se satisfying (((f()) + fe(x:) S cnd R(fi) S p. the expected error of Is on the target domain Q is bounded,
Efr((fs) S (fry(ft) + +c+ 2g + min(c,g]
P
(12)
To further relate the expected error with the CST training objective and obtain finite-sample guar- antee, we use the multi-class margin loss: 4(f(z), y) == (y(-M(((z) y)), where M(v. y) = P/o - ma Ply]] and b. the ramp function. We then extend the margin loss: M(U)
max,("()) - maxy'sy 1/y1) (The difference between the largest and the second largest scores in v). and (((()) = wh (-M((((())) [x))). (.())) Further suppose /(a is L.y-siisehitz w.r.t. the metric .(..)) and T = 1 1 2 min(e, q} > 0. Consider the following training objective for CST, denoted by LCsT(fa, ft), where Lp (ft) := E ( phy (ft(x), y) corresponds to the cycle source loss in equation 5, T, (Jr, == E (U fx(x)) consistent with the target loss in equation 4, and M(f((z)) is closely related to the uncertainty of predictions in equation 11.
E R 4(P
min LesT f() = (1) L (c. +
(13)
The following theorem shows that the minimizer of the training objective Tess([s; St) guarantees low population error of fa the target domain Q.

---

Theorem 2. R(Flp) denotes the empirical Rademacher complexity function class F on dataset
P. For any solution of equation 13 and y > O, with probability larger than 8,
Ereo (fa) I LesT(fe, +2
4K
7 R(F())
R(F x(Fx
2
((F(()) R(Fla)) + S.
where Ç = 0 (Vlog(1/8)) class (x: + (()) E F)-
Vlog /ogg(1/0//m)) a low-order term. F. refers to the function
Main insights. Theorem 2 justifies CST under the expansion assumption. The generalization error of the classifier fa on the target domain is bounded with the CST loss objective (fs: the intrinsic property of the data distribution q. and the complexity of the function classes. In our algorithm, LeST(J ft) minimized by the neural networks and q is a constant. The complexity of the function class can be controlled with proper regularization.
4.2 Hard Case for Feature Adaptation and Standard Self-Training
To gain more insight, we study UDA in a quadratic neural network (e. = 0 (Txx)))))))))))))))))) where @ is element-wise power. In UDA, the source can have multiple solutions but we aim to learn the one working on the target [34]. We design the underlying distributions p and g in Table
reflect this.
Consider the following P and Q. 2[1 and 2[2] are sampled i.i.d. from distribution p on P. and from
q on Q. For i E [3. -h = ],t122] on P and T[ 0,4111] on Q.
Table 1: The design of p and g.
0; E [=1] are i.i.d. and uniform. We also assume realizability:
Distribution -1
+1
0
2 "[[] oP6] for both source and target. Note that =fn I[
Source
0.05
0.05
0.90
Q. We visualize this specialized setting in Figure [2]
for all E [2, d) are solutions to P but only y = =
works on
Target
0.25
0.25
0.50
Source
Source
Targat
Target
(a)
(b)
M011
(c)
Xii
(d)
Pass
Figure 4: The hard where d Green dots for y 1, red dots for y 0, and blue dots for y -1. The grey curve is the classification boundary of different features. The good feature =mij =iu works on larget domain (shown in (a) and (c)), whereas spurious feature =Lu =Moo only works on the source domain (shown (b) and (d)). In Section 4.2, we show feature adaptation and standard self-training learn zi1) =fg) while CST learns =m1
=
To make the features more tractable, we study the norm-constraines version of the algorithms (details are deferred to Section A.3.2). We compare the features learned by feature adaptation, standard self- training, and CST. Intuitively. feature adaptation fails because the ideal target solution y = 2² 2
has larger distance in the feature space than other spurious solutions y = =²1) z Standard self- training also fails since it will choose randomly among all solutions. In comparison, CST can recover the ground truth, because it can distinguish the spurious solution resulting in bad pseudo-labels A classifier trained with those pseudo-label, cannot work on the source domain turn. This intuition is rigorously justified in the following theorems.
Theorem 3. For E (0, 0.5), the following statements hold for feature adaptation and self-training:
For failure rate & >> O, and target dataset size n, > O(log } with probability at least 1 & over the sampling of target data, the solution (0EA, δea) found by feature adaptation satisfies
Eriq (rrg (Px))
(14)
With probability at least 2-1 the solution (ōsr, Gst) of standard self-training satisfies
Erno (OST, @ST)
(15)
7

---

Theorem 4. For failure rate { >>>000000 0, and target dataset size > ( }), with probability at least
1 - 5, the solution of CST (δeST: fCTT) recovers the ground truth of the target dataset:
EntQ(OcsT,écst) = 0.
(16)
5
Experiments
We test the performance of the proposed method on both vision and language datasets. Cycle Self- Training (CST) consistently outperforms sfate-of-the-art feature adaptation and self-training methods. Code is available https:// thub. comm//iuoong9//S/0
5.1 Setup
Datasets. We experiment on visual object recognition and linguistic sentiment classification tasks: Office-Home [64] has 65 classes from four kinds of environment with large domain gap: Artistic (Ar), Clip Art (CI), Product (Pr), and Real-World (Rw); VisDA-2017 [45] is a large-scale UDA dataset with two domains named Synthetic and Real. The datasets consist of over 200k images from 12 categories of objects; Amazon Review [10] is linguistic sentiment classification dataset of product reviews in four products: Books (B), DVDs (D), Electronics (E), and Kitchen (K).
Implementation. We use ResNet-50 [26] (pretrained ImageNet [53]) feature extractors for vision tasks, and BERT [16] for linguistic tasks. On VisDA-2017, we also provide results of ResNet- 101 to include more baselines. We use cross-entropy loss for classification the source domain. When training the target head Ó, and updating the feature extractor with CST, use squared loss to get the analytical solution of 0g directly and avoid calculating second order derivatives as meta- learning [18]. Details on adapting squared loss to multi-class classification are deferred to Appendix B. We adopt SGD with initial learning rate ?/u 2c 3 for image classification and fj = 5e 4 for sentiment classification. Following standard protocol in [26]. we decay the learning rate by 0.1 each 50 epochs until 150 epochs. We run all the tasks times and report mean and deviation in top-1 accuracy. For VisDA-2017, we report the mean class accuracy. Following Theorem 2, we also enhance CST with sharpness-aware regularization [19] (CST+SAM), which help regularize the I. Lipschitzness of the function class. Due to space limit, we report mean accuracies in Tables 2 and 3 and defer standard deviation to Appendix C.
5.2 Baselines
We compare with two lines of works domain adaptation: feature adaptation and self-training, We also compare with more complex state-of-the-aryy and create stronger baselines by combining feature adaptation and self-training.
DANN [22]. MCD [54], CDAN [37] (which improves DANN with pseudo- label conditioning), MDD [73] (which improves previous domain adaptation with margin theory),
Feature Adaptation:
Implicit Aligmment (IA) [28] (which improves MDD to deal with label shift).
Self-Training: We include VAT [40], MixMatch [8] and FixMatch [57] in the semi-supervised learning literature as self-training methods. We also compare with self-training methods for UDA: CBST [77], which considers class imbalance in standard self-training, and KLD [78], which improves CBST with label regularization. However, these methods involve tricks specified for convolutional networks. Thus, in sentiment classification tasks where we use BERT backbones, we compare with other consistency regularization baselines: VAT [40], VAT+Entropy Minimization.
Feature Adaptation + Telfnirain DIRT-T [56] combines DANN, VAT, and entropy minimiza- tion. We also create more powerful baselines: CDAN+VAT+Entrppy and MDD+Fixmatch.
Other SOTA. AFN [69] boosts transferability by large norm. STAR [38] aligns domains with stochas- tic classifiers. SENTRY [48] selects confident examples with a committee of random augmentations.
5.3 Results
Results on 12 pairs of Office-Home tasks are shown in Table 2. When domain shift is large, standard self-training methods such as VAT and FixMatch suffer from the decay in pseudo-label quality. CST outperforms feature adaptation and self-training methods significantly in out of 12 tasks. Note that CST does not involve manually setting confidence threshold or reweighting.
8

---

Table 2: Accuracy (%) on Offece-Home for unsupervised domain adaptation (ResNet-50).
Method
Ar-CI Ar-Pr Ar-Rw Cl-Ar CI-Pr Cl-Rw Pr-Ar Pr-CI Pr-Rw Rw-Ar Rw-CI Rw-Pr Avg.
DANN [22]
45.6
59.3
70.1
47.0
58.5
60.9
46.1
43.7
68.5
63.2
51.8
76,8
57.6
CDAN [37]
50.7
70.6
76.0
57.6
70.0
70.0
57.4
50.9
77.3
70.9
56.7
81.6
65.8
52.2
71.5
76.4
61.1
70.3
67.8
59.5
54.4
78.6
73.2
59.0
82.7
67.3
FixMatch [57]
51.8
74.2
80.1
63.5
73.8
61.3
64.7
51.4
80.0
73.3
56.8
81.7
67.7
MDD [73]
54.9
73.7
77.8
60.0
71.4
71.8
61.2
53.6
78.1
72.5
60.2
82.3
68.1
MDD+IA [28]
56.2
77.9
79.2
64.4
73.1
74.4
64.2
54.2
79.9
71.2
58.1
83.1
69.5
SENTRY [48]
61.8
77.4
80.1
66.3
71.6
74.7
66.8
63.0
80.9
74.0
66.3
84.1
72.2
CST
59.0
79.6
83.4
68.4
77.1
76.7
68.9
56.4
83.0
75.3
62.2
85.1
73.0
Table 3: Accuracy (%)
Sentiment Dataset for domain adaptation with BERT.
Method
B-D
B-E
B-K
D-B
D-E
D-K
E-B
E-D
E-K
K-B
K-D
K-E
Avg.
Source-only
89.7
88.4
90.9
90.1
88.5
90.2
86.9
88.5
91.5
87.6
87.3
91.2
89.2
DANN [22]
90.2
89.5
90.9
91.0
90.6
90.2
87.1
87.5
92.8
87.8
87.6
93.2
89.9
VAT [40]
90.6
91.0
91.7
90.8
90.8
92.0
87.2
86.9
92.6
86.9
87.7
92.9
90.1
VnT+Entropy
90.4
91.3
91.5
91.0
91.1
92.4
87.5
86.3
92.4
86.5
87.5
93.1
90.1
MDD [73]
90.4
90.4
91.8
90.2
90.9
91.0
87.5
86.3
92.5
89.0
87.9
92.1
90.0
CST
91.5
92.9
92.6
91.9
92.6
93.5
90.2
89.4
93.8
87.9
88.3
93.5
91.5
Table 4 shows the results on VisDA-2017 CST surpasses state-of-the sofftteearte arts with ResNet-50 and ResNet- 101 backbones. We also combine feature adaptation and self self-training (DIRT-T, CDAN+ VATN+VAT+nntrop and MDD+FixMatch) to test if feature adaptation alleviates the negative effect of domain shift in standard self-training. Results indicate that CST a better solution than simple combination.
While most traditional self-training methods include techniques specified for ConvNets such as Mixup [72], CST is universal method and can directly work sentiment classification by simply replacing the head and training objective of BERT [16]. In Table 3, most feature adaptation baselines improve over source only marginally, but CST outperforms all baselines on most tasks significantly.
5.4 Analysis
Ablation Study. We study the role each part of CST in self-training. CST w/o Tsallis removes the Tsallis en- tropy ITsallis, Ent+en replaces the Tsallis entropy with standard entropy. FixMatch+Tsalll adds Lssalss, to standard self-training. Observations are shown in Table CST+Entropy performs 3.7% worse than CST. indicating that Tsallis entropy a better regularization for pseudo- labels than standard entropy. CST performs 5.4% better
Table 5: Ablation on VisDA-2017.
Method
Accuracy f
div
FixMatch [57]
74.5 1 0.2
0.22
Fixmatch+ Tsallis
76.3 0.8
0.15
CST w/o Tsallis
72.0
0.16
CST+Entropy
76.2 0.6
0.20
CST
79.9 + 0.5
0.12
than FixMatch, indicating that CST better adapted to domain shift than standard self-training. While FixMatch+Tssal outperforms FixMatch, it is still 3.6% behind CST, with much larger total variation distance drv between pseudo-labele and ground-trume truths indicating that CST makes pseudo-labels more reliable than standard self- self-training under domain shift.
Quality of Pseudo-labels. We visualize the error of pseudo-labels during training on VisDA-2017 in Figure 5 (Left). The error of target classifier f2 on the source domain decreases quickly in training, when both the error pseudo-labels (error of f, on Q) and the total variation (TV) distance between pseudo-labels and ground- truthss continue to decay, indicating that CST gradually refines pseudo- labels. This forms a clear contrast to standard self-training as visualized in Figure 2 (Middle), where the distance dry remains nearly unchanged throughout training.
Comparison of Gibbs entropy and Tsallis entropy. We compare the pseudo-labels learned with standard Gibbs entropy and Tsallis entropy on Ar-CC with ResNet-50 at epoch 40. We compute the difference between the largest and the second largest softmax scores of each target example and plot the histogram in Figure 5 (Right). Gibbs entropy makes the largest softmax output close to 1, indicating over-confidence. In this case, if the prediction wrong, it can be hard to correct using self- self-trainin foff-traiing. In contrast, Tsallis entropy allows the largest and the second largest scores to be similar.

---

Table Mean Class Accuracy (%) for unsupervised domain adaptation on VisDA-2017.
Method
ResNet-50
Resteet1101
Method
Realet-500
ResNet-101
DANN [22]
69.3
79.5
CBST [77]
76.4
VAT [40]
68.0 = 0.3
73.4 + 0.5
KLD [78]
78.1 1 + 0.2
DIRT-T [56]
68.2 = 0.3
77.2 + 0.5
MDD [73]
74.6
81.6 + 0,3
MCD [54]
69.2
77.7
AFN [69]
76.1
CDAN [37]
70.0
80.1
MDD+IA [28]
75.8
CDAN+ Enrooy
76.5 = 0.5
80.4 1: 0.7
MDD+FixMatch
77.8
0.3
82.4
MixMatch
69.3 = 0.4
77.0 1 0.5
STAR [38]
82.7
FixMatch [57]
74.5 = 0.2
79.5 = 0.3
SENTRY [48]
76.7
CST
79.9 = 0.5
84.8 + 0.6
CST+SAM
80.6 0.5
86.5 + 0.7
Error Rate of of 8, on target
Gibbs Entropy
Error Rate of 6 an source P
Tsal is Entropy
05
Total Variation Between Pseudo labels and Ground Truth
Density
Epochs
303
140
1.4 2.6
1.0
Figure 5: Analysis. Left: Error of pseudo- labels and reverse pseudo-labels labels. The error of target classifier d on the source domain decreases, indicating the quality of pseudo-labels is refined. Right: Histograms of the difference between the largest and the second largest softmax scores. Tsallis entropy avoids over-confidence.
Related Work
Self-Training Self-training is a mainstream technique for semi-supervised learning [13]. In this work, we focus on pseudo-labeling [52, 31, 2], which uses unlabeled data by training on pseudo-labels generated by a source model. Other lines of work study consistency regularization [4, 51, 55, 40].
Recent works demonstrate the power of such methods [67, 57, 23]. Equipped with proper training techniques, these methods can achieve comparable results as standard training that uses much more labeled examples [17]. Zoph et al. [76] compare self-training to pre-training and joint training. Vu al. [65], Mukherjee & Awadallah [42] show that task-level self-training works well in few-shot learning. These methods are tailored to semi-supervised learning or general representation learning and do not take domain shift into consideration explicitly. Wei et [66], Frei et al. [20] provide the first nice theoretical analysis of self-training based on the expansion assumption.
Domain Adaptation. Inspired by the generalization error bound of Ben-David et al. [7], Long et al. [34], Zellinger et al. [71] minimize distance measures between source and target distributions to learn features. Ganin al. [22] (DANN) proposed approximate the domain distance by adversarial learning. Follow-up works proposed various improvement upon DANN [63, 54, 37, 73, 28]. Popular as they are, failure cases exist in situation like label shift [74, 32], shift in support of domains [29], and large discrepancy between source and target [33]. Another line of works try to address domain adaptation with self-training. Shu et al. [56] improves DANN with VAT and entropy minimization. French et al. [21], Zou et al. [78], Li et al. [32] incorporated various semi-supervised learning techniques boost domain adaptation performance. Kumar al. [30], Chen et al. [15] and Cai et al. [1I] showed self-training provably works in domain adaptation under certain assumptions.
7 Conclusion
We propose cycle self-training in place of standard self self-training to explicitly address the distribution shift in domain adaptation. We show that our method provably works under the expansion assumption and demonstrate hard cases for feature adaptation and standard self-training. Self-trinnig (or pseudo- labeling) is only one line of works in the semi-supervised learning literature. Future work can delve into the behaviors of other semi-supervised learning techniques including consistency regularization and data augmentation under distribution shift, and exploit them extensively for domain adaptation.
A Acknowledgements
This work was supported by the National Natural Science Foundation of China under Grants 62022050 and 62021002, Beijing Nova Program under Grant 7201 100006820041, China's Ministry of Industry and Information Technology, the MOE Innovation Plan and the BNRist Innovation Fund.
10

---

References
[1] Albadawy, E. A., Saha, A., and Mazurowski, M. A. Deep learning for segmentation brain tumors: Impact of noss-institutionall training and testing. Medical Physics, 45(3), 2018.
[2] Arazo, E., Ortego, D., Albert, P., O'Connor N. E.,, and McGuinness, K. Pseudo-labell -aeudi-labeling and confirmation bias in deep semi-supervised learning. CoRR, abs/1908.02283. 2019.
[3] Arora, S., Du, S., Hu, W., Li, Z., Salakhutdinov, R., and Wang. On exact computation with an infinitely wide neural net. In NeurlPS, pp. 8141-8150. 2019.
[4] Bachman, P., Alsharif, O., and Precup, D. Learning with pseudo-ensemble In NeurIPS, volume 27, pp. 3365-3373, 2014.
[5] Bartlett, P. L. and Mendelson, S. Rademacher and gaussian complexities: Risk bounds and structural results. JMLR, 3(Nov):463 482246646422.. 2002.
[6] Ben-David, S. and Umerr R. On the hardness of domain adaptation and the utility of unlabeled target samples. In ALT, pp. 139-153, 2012.
[7] Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F,, and Vaughan, J. W. A theory of learning from different domains. Machine Learning, 79(1-2):151-175, 2010.
[8] Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C. Mixmatch: holistic approach semi-supervised learning, arXiv preprint 1rXiiv:905.02200 2019.
[9] Bertinetto, L., Henriques, F., Torr, P., and Vedaldi, A. Meta-learning with differentiable closed-form solvers. In ICLR, 2019.
[10] Blitzer, J., Dredze, M., and Pereira, F. Biographies, Bollywood, boom-boxes and blenders:
Domain adaptation for sentiment classification. In ACL, pp. 440-447, 2007.
[11] Cai, T., Gao, R., Lee, J. D., and Lei, Q. A theory of label propagation for subpopulation shift, 2021.
[12] Carlini, N. Poisoning the unlabeled dataset of semi-supervised learning, 2021.
[13] Chapelle, O., Schölkopf, B., and Zien, A. Semi-supervised learning. MIT press Cambridge, 2006.
[14] Chen, C., Xie, W., Huang, W., Rong. Y., Ding, X., Huang, Y., Xu, T., and Huang, J. Progressive feature alignment for unsupervised domain adaptation. In CVPR, pp. 627-636, 2019.
[15] Chen, Y., Wei, C., Kumar, A., and Ma, T. Self celf-training avoids using spurious features under domain shift. In Neur/PS, pp.
2020.
[16] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, pp. 4171-4186, 2019.
[17] Du, J., Grave, E., Gunel, B., Chaudhary, V., Celebi, O., Auli, M., Stoyanov, V., and Conneau, A. Self-training improves pre-training for natural language understanding. In NAACL, pp. 5408 0011,4488,220 2021.
[18] Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. ICML, pp. 1 1126-1 135, 2017.
[19] Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware minimization for efficiently improving generalization. In ICLR, 2021.
[20] Frei, S., Zou, D., Chen, Z., and Gu, Q. Self-training converts weak learners to strong learners in mixture models. arXiv preprint arXiv:2106. 200.://50..305,, 2021.
[21] French, G., Mackiewicz, M., and Fisher, M. Self-ensembling for visual domain adaptation, In ICLR, 2018

---

[22] Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle. H., Marchand, M., and Lempitsky,
V. Domain aovain-adversarial training of neural networks. JMLR, 17(1):2096-2300, 2016.
[23] Ghiasi, G., Zoph. B., Cubuk, E. D., Le, Q. V., and Lin, T.-.. Multi-task self-training for learning general representations. ICCV, pp. 8856-8865, 2021.
[24] Grandvalet, Y. and Bengio, Y. Semi-supervised learning by entropy minimization. In NeurIPS, pp. 529-536,, 2004.
[25] Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In ICML, pp. 1321-1330, 2017.
[26] He, K., Zhang. X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, pp. 770-778, 2016.
[27] Hoffman, J., Tzeng, E., Park, T., Zhu, J., Isola, P., Saenko, K., Efros, A. A., and Darrell, T. Cycada: Cycle-consistent adversarial domain adaptation. In ICML, pp. 1994-2003, 2018.
[28] Jiang, X., Lao, Q.. Matwin, S., and Havaei, M. Implicit class-conditioned domain alignment for unsupervised domain adaptation. In ICML, pp. 4816-48211 2020.
[29] Johansson, F. D., Sontag, D., and Ranganath, R. Support and invertibility in domain-invariant representations. In AISTATS, pp. 527-536, 2019.
[30] Kumar, A., Ma, T., and Liang, P. Understanding self-training for gradual domain adaptation. In ICML, pp. 5468-5479, 2020.
[31] Lee, D.-H. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. ICML Workshop: Challenges in Representation Learning (WREPL), 2013.
[32] Li, B., Wang, Y., Che, T., Zhang, S., Zhao, S., Xu, P., Zhou, W., Bengio, Y., and Keutzer, K. Rethinking distributional matching based domain adaptation. ArXiv, abs/2006. 13352, 2020.
[33] Liu, H., Long, M., Wang, J., and Jordan, M. Transferable adversarial training: general approach adapting deep classifiers. In ICML, volume 97, pp. 4013-4022, 2019..
[34] Long, M., Cao, Y., Wang. J., and Jordan, M. I. Learning transferable features with deep adaptation networks. In ICML, pp. 97-105, 2015.
[35] Long, M., Zhu, H., Wang, J., and Jordan, M. I. Unsupervised domain adaptation with residual transfer networks. In NeurlPS, pp.
2016.
[36] Long, M., Zhu, H., Wang, J., and Jordan, M. I. Deep transfer learning with joint adaptation networks. In ICML, pp. 2208-2217, 2017.
[37] Long, M., Cao, Z., Wang, J., and Jordan, M. Conditional adversarial domain adaptation. In NeurlPS, pp. 1640-1651 2018.
[38] Lu, Z., Yang, Y., Zhu, X., Liu, C., Song, Y.-.,, and Xiang, T. Stochastic classifiers for unsupervised domain adaptation. In CVPR, pp. 9111-9120, 2020.
[39] Mey, A. and Loog, M. soft-labeled self-training approach. In ICPR, 2016.
[40] Miyato, T., Maeda, S., Ishii, S., and Koyama, M. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. TPAMI, 2018.
[41] Mohri, M., Rostamizadch, A., and Talwalkar, A. Foundations of machine learning. MIT press, 2018.
[42] Mukherjee, S. and Awadallah, A. Uncertainty-aware self-training for few-shot text classification. In NeurIPS, volume 33, pp. 21199-21212, 2020.
[43] Pan, S. J. and Yang, Q. A survey on transfer learning, TKDE,
12

---

[44] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang. E.. Di Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang. I.., Bai, J., and Chintala, S. Pytorch: An imperative style, performeree deep learning library. In NeurIPS, volume 32, pp. 8026-8037, 2019.
[45] Peng. X., Usman, B., Kaushik, N., Hoffmann, J., Wang, D., and Saenko, K. Visda: The visual domain adaptation challenge. CoRR, abs/1710.06922 2017.
[46] Peng, X., Bai, Q. Xia, X., Huang, Z., Saenko, K., and Wang, B. Moment matching for multi-source domain adaptation. In ICCV, pp. 1406-1415, 2019.
[47] Prabhu, V. Khare, S., Kartik, D., and Hoffman, J. Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation, 2020.
[48] Prabhu, V., Khare, S., Kartik, D., and Hoffman, J. Sentry: Selective entropy optimization committee consistency for unsupervised domain adaptation. In ICCV, pp. 8558-8567, October 2021.
[49] Qu, X., Zou, Z., Cheng, Y., Yang, Y., and Zhou, P. Adversarial category alignment network for cross cross-domain sentiment classification. In NAACL, 2019.
[50] Quionero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset Shift in Machine Learning. The MIT Press, 2009.
[51] Rasmus, A., Berglund, M., Honkala, M., Valpola, H., and Raiko, T. Semi-supervised learning with ladder networks. In NeurlPS, volume 28, pp. 3546-3554, 2015.
[52] Rosenberg, C., Hebert, M., and Schneiderman, H. Semi-supervised self-training of object detection models. In WACV, volume 1, pp. 29-36, 2005.
[53] Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L. ImageNet Large Scale Visual Recognition Challenge. IJCV, 115()):11-222,, 2015.
[54] Saito, K., Watanabe, K., Ushiku, Y., and Harada, T. Maximum classifier discrepancy for unsupervised domain adaptation. CVPR, pp. 3723-3732, 2018.
[55] Sajjadi, M., Javanmardi, M., and Tasdizen, T. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In NeurIPS, volume 29, pp. 1163-1171, 2016.
[56] Shu, R., Bui, H., Narui, H., and Ermon, S. A DIRT-t approach unsupervised domain adaptation. In ICLR, 2018.
[57] Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C. A., Cubuk, E. D., Kurakin, A., and Li, C.-L. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In NewrlPS, 2020.
[58] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, and Fergus, Intriguing properties of neural networks. In ICLR, 2014.
[59] Talagrand, M. Upper and lower bounds for stochastic processes: modern methods and classical problems, volume 60. Springer Science & Business Media, 2014.
[60] Tan, S., Peng. X., and Saenko, K. Class-imbalanced domain adaptation: An empirical odyssey. In ECCV Workshop, 2020.
[61] Tarvainen, A. and Valpola, H. Mean teachers are better role models: Weight-averaged con- sistency targets improve suni-super deep learning results. In NeurIPS, volume 30, pp. 1 1195-1204, 2017.
(62] Tsallis, C. Possible generalization of boltzmann-ginn giibss statistics. Journal of Statistical Physics, 52(1-2)4479-483 1988.
13

---

[63] Tzeng, E., Hoffman, J.. Saenko, K., and Darrell, T. Adversarial discriminative domain adaptation. In CVPR. pp. 7167-7176, 2017.
[64] Venkateswara, H., Eusebio, J., Chakraborty, S., and Panchanathan, S. Deep hashing network for unsupervised domain adaptation. In CVPR, pp. 5018-5027, 2017.
[65] Vu, T., Luong, M.-T., Le, Q. V., Simon, G., and lyyer, M. Strata: Self-training with task augmentation for better few-shot learning. arXiv preprint arXiv:2 2021.
[66] Wei, C., Shen, K., Yining, C., and Ma, T. Theoretical analysis of self-training with deep networks on unlabeled data. In ICLR, 2021.
[67] Xie, Q., Luong, M. T., Hovy, E., and Le, Q. Self-training with noisy student improves imagenet classification. In CVPR, 2020.
[68] Xie, S. M., Kumar, A., Jones, R., Khani, F., Ma, T., and Liang, P. In-n-out: Pre-training and self-training using auxiliary information for out-of-distribution robustness. In ICLR, 2021.
[69] Xu, R., Li, G., Yang, J., and Lin, L. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In ICCV, 2019.
[70] Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. How transferable features in deep neural networks? In NeurlPS, pp. 3320-3328. 2014.
[71] Zellinger, W. Grubinger. T., Lughofer, E., Natschläger, T., and Saminger-Platee S. Central moment discrepancy (CMD) for domain-invariant representation learning. In ICLR, 2017.
[72] Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D. mixup: Beyond empirical risk minimization. In ICLR, 2018.
[73] Zhang, Y., Liu, T., Long, M., and Jordan, M. Bridging theory and algorithm for domain adaptation. In ICML, pp. 7404-7413, 2019.
[74] Zhao, H., Combes, R. 'T. D., Zhang, K., and Gordon, G. On learning invariant representations for domain adaptation. In ICML, volume 97, pp. 7523-7532, 2019.
[75] Ziser, and Reichart, R. Pivot based language modeling for improved neural domain adaptation. In NAACL, pp. 1241-1251, 2018.
[76] Zoph, B., Ghiasi, G., Lin, T.-Y,, Cui, Y., Liu, H., Cubuk, E. D., and Le, Q. Rethinking pre-training and self-training. In NeurlPS, volume 33, pp. 3833-3845, 2020.
[77] Zou, Y., Yu, Z. Vijaya Kumar, B. V. K., and Wang, J. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In ECCV, pp. 297-313, 2018.
[78] Zou, Y., Yu, Z., Liu, X., Kumar, B. V., and Wang. J. Confidence regularized self-training In ICCV, October 2019.
14


Reviews:
[{"review_id": "uXdXeH1rf8f", "reviewer": "Reviewer_PLkj", "summary": "This paper presents a novel approach for unsupervised domain adaptation (UDA) based on cycle self-training. Instead of conventional self-training, there are a generated and two attached classifiers with different purposes. The source classifier, mainly trained on the source loss, is applied to infer pseudo target labels. The target classifier is trained by the self-training loss based on the pseudo labels provided by the source classifier.  The authors also propose the Tsallis entropy as the regularization term, which can be seen as a side contribution.", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n+ The paper is organized in clear logic, which is easier to read. The discussion of related works is comprehensive enough to cover most of the topics in the field.\n\n+ There is a detailed analysis of self-training in different cases: with or without domain shift, distribution change, etc. It is helpful to deeply understand the effects of pseudo labels in the DA setting.\n\n+ Tsallis entropy is an interesting idea that seems to fit better with the DA problem than the conventional entropy loss. And the discussion of its motivation is comprehensive and clear.\n\nWeaknesses:\n\n-The cycle self-training seems not novel and too simple. Some previous works on the tri-training-based DA have tried similar ideas where a fixing classifier is used to help another classifier. And I am not quite convinced by the design of the source classifier, which seems not to be trained by the target samples. If so, it is hard for me to believe that it can infer the accurate target pseudo labels without seeing them. Will the pseudo labels have biased towards the source domain? How to address such a problem.\n\n-The experimental results are not strong enough. Although it outperforms the SOTA SSL methods and some baseline UDA methods, some ignored UDA works such as [1] and [2] can beat the proposed method by a large margin. \n\n\n\nOther Comments:\n\n1. The right subfigure of Fig2 is too small to read. It will be much better if enlarging it.\n\n2. Please provide which dataset is used in the caption of Fig 2.\n\n3. In lines 36 – 42, the motivation of the reverse step is not convincing enough. The target pseudo labels informative to the source domain may also be harmful. The authors should provide more solid experimental results to support such a claim.\n\n[1] Contrastive Adaptation Network for Unsupervised Domain Adaptation. CVPR 2019.\n[2] FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation. CVPR 2021.\n\nAfter rebuttal:\nMy main concerns are addressed. I'd like to increase my rating score. Thanks for the detailed feedback.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes. The authors provide adequate discussion of limitations and potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "14"}, "extra_text": {}}, {"review_id": "dOn-Y6qbfB6", "reviewer": "Reviewer_wNMh", "summary": "This paper proposes an unsupervised domain adaptation method based on cyclical self-training. There are two main components to the method: 1. a cyclical self-training algorithm that solves a bilevel optimization problem based on an in the inner loop which trains a target classifier with target pseudo-labels  and an outer loop that makes the target classifier perform well on the source domain by updating the shared representations in the outer loop. 2. an uncertainty measure based on the Tsallis entropy, which adaptively minimizes the uncertainty in target pseudo-labels, which replaces the standard Gibbs entropy used by self-training methods. Experiments on computer vision and NLP datasets show that proposed method outperforms baseline methods by a decent margin. Ablation studies show that both components of the algorithm---cyclical self-training and Tsallis entropy---meaningfully contribute to performance improvement. Qualitative and quantitative studies show that target pseudo-label quality is improved by the proposed approach. ", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The paper is well-written and organized. The introduction section motivates the problem with current self-training methods using qualitative and quantitative experiments. The proposed approach is supported by theoretical understanding. Experiments and ablation studies help understand how the algorithm works and quantify performance improvement. \n\n2. Experiments across data domains show good perfomance improvement over state-of-the-art methods, including those tailored to specific domains or dataset types. \n\nWeakness:\nThis is not a weakness per se, but the paper would benefit from more analysis of what type of performance improvement is obtained by using this method. Understanding how label shift and covariate shift is tackled and what type of examples are classified better by this method can help spur future research in this problem space. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See above. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "VfJ8n-wpPVd", "reviewer": "Reviewer_txSf", "summary": "Proposes cycle self-training (CST) an unsupervised domain adaptation (UDA) algorithm that cycles between standard self-training of a target classifier on (source model generated) target pseudolabels, and a new cycle self-training step that updates backbone representations so as to generalize the target classifier to the source domain. An additional self-training objective based on the Tsallis entropy is introduced. Theoretical and empirical results are presented on several standard UDA benchmarks.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality\n\nThe proposed approach is novel and original.\n\nQuality\n\n– The paper does well to analyze and clearly describe some of the shortcomings of self-training (Sec 2.1). \n\n– The proposed approach, although slightly complex, is well-designed, explained, and well-supported by theoretical analysis (Sec 4).\n\n– The performance gains of the proposed approach are consistent and promising, and the paper does well to present results on several benchmarks (including additional ones in the supplement) across vision and language tasks, and compare to a wide range of baselines, including strong baselines derived from combining existing methods. \n\nClarity\n\n– The paper is well-written and easy to follow.\n\n– [minor] The flow of logic in Figure 1 is somewhat difficult to follow and it took me multiple reads. It would be good to try and simplify it.\n\n– [minor] In Eq 3-5, it would be good to include what instances (source / target) the objective operates on\n\nSignificance\n\nThe paper focuses on the UDA problem that is of considerable research interest and the proposed approach improves upon prior work in a wide variety of settings.\n\nOverall\n\nThis is a well-written paper that proposes an interesting, effective, and principled UDA approach. I recommend acceptance.\n\n---Post-Rebuttal----\n\nI have read the author response and other reviews, and would like to keep my accept rating.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors adequately addressed the limitations and potential negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "9kOY3scXbYw", "reviewer": "Reviewer_Yv3B", "summary": "The paper tackles the problem of domain adaptation with a newly proposed cycle self-training algorithm. Given observed that the pseudo labels are noisy and existing de-noising methods require ad-hoc hyper-parameters for specific tasks, the authors attempt to progressively refine the pseudo labels by the capability of the network itself. With the intuition of transferring the source-domain knowledge to the target domain, the authors propose to regularize the pseudo labels with a cyclic training pipeline. Tsallis Entropy is further introduced to improve the label quality. Experiments on both visual classification and linguistic sentiment classification indicate the effectiveness of the introduced method.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n+ The paper is well written and easy to follow. The figures and charts are nice and help to understand.\n\n+ The analysis of limitations of existing self-training methods in Sec. 2.1 well motivates the introduce method.\n\n+ The authors carefully analyze the introduced method both theoretically and empirically. \n\n+ The introduced method is simple but effective. It seems easy to implement and I believe the results can be well reproduced. \n\n+ The introduced Tsallis Entropy can be seamlessly integrated into existing methods, e.g., FixMatch, and achieved significant performance gains as shown in Table 5.\n\n\nI am overall satisfied with this paper and would like to give some comments that may make this paper better.\n\n1. The notations are a bit complex.\n\n2. Despite that the introduced method is very simple, it is also necessary to compare with more SoTA methods (in CVPR2020/ECCV2020/…) in the experimental part (Table 2/3/…). \n\n3. Generalizing this method to open-set domain adaptation would make it more applicable in real-world scenarios.\n\n\nAfter rebuttal: My concerns have been well addressed. I recommend accepting this paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The introduced cycle self-training could only be applied when two domains share the same label set. How to properly generalize this method to real-world scenarios (e.g., open-set) is important.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers unanimously agree that the paper should be accepted. I find the idea of the paper interesting and the empirical results seem encouraging. I am not convinced that the presented algorithm in its full complexity can be theoretically studied. The experimental results are encouraging but still falling behind the state-of-the-art. Notation is cluttered and exposition is not as clear as it should be. I recommend acceptance as a poster, but please incorporate reviewers' feedback to improve the paper.
Decision: Accept (Poster)