Unsupervised Domain Adaptation via Minimized Joint Error

Abstract: Unsupervised domain adaptation transfers knowledge from learned source domain to a different (but related) target distribution, for which only few or no labeled data is available. Some researchers proposed upper bounds for the target error when transferring the knowledge, i.e.,Ben-David et al. (2010) established a theory based on minimizing the source error and distance between marginal distributions simultaneously. However, in most works the joint error is usually ignored. In this paper, we argue that the joint error is essential for the domain adaptation problem, in particular if the samples from different classes in source/target are closely aligned when matching the marginal distributions. To tackle this problem, we propose a novel upper bound that includes the joint error. Moreover, we utilize a constrained hypothesis space to further tighten up this bound. Furthermore, we propose a novel cross margin discrepancy to measure the dissimilarity between hypotheses. We show in this paper that the new cross margin discrepancy is able to alleviate instability during adversarial learning. In addition, we present extensive empirical evidence that shows that our proposal outperforms related approaches in image classification error rates on standard domain adaptation benchmarks.

Reviews and author responses:

1. In practice, we can always add the validation set into the training set after we determine the hyper-parameters based on the performance on the validation set. However, in our experiments, we did not do that because we have to provide the comparisons under a fair condition.

    Besides, previous works like [1,2,3,4,5,6] also leave a proportion of target data to be held out for model selection and the rest target data for  training.

    Since the validation performance is unbiased with controlled variance, we believe it is more suitable for practical cases (where accuracy is more important than labeling cost) compared to the methods with unbounded variance like IWCW.

    Some works show robust performance when varying hyper-parameters in a specific range. However, it remains unclear how this range is selected in the first place. One way or another, we need to deal with the model selection problem if we want to achieve the best performance on a brand new task in practice, no matter how insensitive the model seems to be.

    [1]Hoffman et al. Cycada: Cycle-consistent adversarial domain adaptation. ICML, 2018.
    
    [2]Saito et al. Asymmetric Tri-training for Unsupervised Domain Adaptation. ICML, 2017.

    [3]Long et al. Learning transferable features with deep adaptation networks. ICML, 2015.

    [4]Ganin and Lempitsky. Unsupervised domain adaptation by backpropagation. ICML, 2014

    [5]Bousmalis et al. Domain separation networks. NIPS, 2016

    [6]Saito et al. Semi-supervised Domain Adaptation via Minimax Entropy. ICCV, 2019


2. $\lambda$ is not even close to $\theta$.

    As explained in A.8,

    $\theta=\min_{f^{\ast}_S,f^{\ast}_T \in H}2\epsilon_T(f_S,f^{\ast}_S)+\epsilon_S(f_S,f^{\ast}_S)+2\epsilon_S(f^{\ast}_T,f_T)+\epsilon_T(f^{\ast}_T,f_T)$

    On the other hand,

    $\lambda=\min_{h \in H}\epsilon_S(h,f_S)+\epsilon_T(h,f_T)$

    If $f_S$ and $f_T$ contradicts (we prove this can happen by simply matching the marginal distributions in A.10), no $h$ can minimize both of the terms in $\lambda$.

    However, we can always choose $f^{\ast}_S,f^{\ast}_T$ that could approximate $f_S,f_T$ respectively (even if $f_S$ and $f_T$ contradicts) to reduce $\theta$ given a hypothesis space with sufficient complexity.  By assuming:
    
    $arg\min_{f \in H}\epsilon_{S\bigcup 2T}(f,f_S)=f^{\ast}_S$
    
    $arg\min_{f \in H}\epsilon_{2S\bigcup T}(f,f_T)=f^{\ast}_T$
   
    We can ignore $\theta$ and the bound becomes:

    $\epsilon_T(h) \leq \epsilon_S(h) + C_{S,T}(f^{\ast}_S,f^{\ast}_T,h)$

    Here, the unaccessible target labels do not interfere the assumption about $f^{\ast}_S,f^{\ast}_T$ because we do not need  to actually compute them. By further assuming $f^{\ast}_S \in H_1\leq H, f^{\ast}_T \in H_2\leq H$, we can relax the bound to:

    $\epsilon_T(h) \leq \epsilon_S(h) + \sup_{f_1\in H_1, f_2\in H_2}C_{S,T}(f_1,f_2,h)$

    This bound is consistent with the one under $f_S,f_T\in H$ in Eq.4.

    

3. We never claim any rigorous theoretical results about pseudo labeling.

    The Rademacher Complexity we provide in A.7 shows an upper bound (the worst case) of the generalized target error under nearly perfect pseudo labeling. It means if the pseudo labels are reliable, by learning our upper bound, the target error is bounded by the empirical values showed in Eq.24.

    We cannot prove any theoretical results if the pseudo labels are totally wrong. And we are not aware of any existing work that provides a generalization error bound when introducing pseudo labeling.

    What we can do is to verify it on multiple benchmarks. And we empirically demonstrate that pseudo labeling can generally improve the performance under our proposal and MCD.

    As explained above, Domainnet is no more than a dataset that contains multiple domains and  it does not increase the complexity of the UDA tasks. Our proposal is tested on Office-Home which contains 4 domains in Domainnet and shows improvement.

    We set $\eta=0.9$ in VisDA, Office-31, Office-Home and set $\eta=0$ in Digits. However, actually, as stated in the paper, the performance for different value of $\eta$ varies subtly in Digits dataset. Our proposal is still superior to the others even if we set $\eta=0.9$ in Digits dataset.


*Summary: This paper tries to study unsupervised domain adaptation problem via minimized joint error. The authors propose an upper bound including joint error term, and design algorithm on the basis of the theoretical results. Good experimental results show the effectiveness of the proposed method.


*Strengths:

+This paper studies unsupervised domain adaptation in the perspective of joint error, which I think is one of the most important issues in the deep regime. I really like the topic of the paper.

+The proposed method works well in several benchmarks.

+This paper conveys some interesting conclusions by showing some interesting examples in figures. Hence this paper is very interesting and easy to follow.

*Weaknesses:

I like the topic of the paper, nevertheless, I'm not convinced by the theoretical results, and some claims are not well supported. This is my main scoring basis. 
Also, I think the practicality of the method needs further verification. The specific comments are as follows:

-Unreliable theoretical analysis. I don't think introducing true labeling functions is a good choice for minimized joint error. We can see that $C_{S, T}(f_S, f_T, h)$ is actually an upper bound of joint error in inequality (2). Nevertheless, I don't think it makes any sense and it is just an upper bound. On the one hand, the bound is not tighter; on the other hand, without assumptions about $f_S, f_T$, the bound is pointless. For example, we can set $f_1$ and $f_2$ predict opposite results and then $C_{S, T}(f_1, f_2, h)$ is larger than $M$ for any $h$. In this point of view, the theory actually relies on assumptions about the $f_S$ and $f_T$.

-$f_S, f_T$ are contained in hypothesis space is a very strong assumption. In some real scenarios with high-dimensional input spaces, the true labeling function may be very complex and can not be contained in the hypothesis space. Additionally, in order to ensure that the bound is not trivial, authors even need to restrict $f_S, f_T$ in subsets of hypothesis. 

-The assumption that $f_T$ can be found by source risk minimization with a smaller weight may not be reliable. By the result of [i], neural networks could memorize any labels. So, weights may not impact the memorization of source data, and the claim ‘$\gamma$  is accuary and used as weighted source error’ is very confusing. Do you mean a small learning rate for source loss will lead to a low accuary, and their relationship is approximately linear? I think this claim is clearly refuted by [i].

-The assumption that $f_T$ could be achieved by source loss minimization and pseudo label training needs theoretical guarantees. I think the method of using pseudo label is difficult to guarantee performance. When the two domains are distant, pseudo label may be unreliable. Using pseudo label may be okay for relatively simple benchmark. Nevertheless, I doubt whether it is helpful for solving complex problems, especially many practical problems. I am very interested in the performance of the proposed method on domainnet[ii], which is a difficult benchmark for domain adaptation.

-The experimental results are very sensitive to hyperparameters and may not be applicable in real problem. The proposed method achieves best results on different datasets with different choices of $\eta$ and $\gamma$. However, how to select a proper hyperparameter is not well stated.

-The motivation of the paper is similar to [iii], which shows some interesting empirical results. This paper can give the value of joint error empirically, similar to [iii].

[i] Zhang et al. Understanding deep learning requires rethinking generalization.

[ii] Peng et al. Moment matching for multi-source domain adaptation.

[iii] Chen et al. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation.

--------------------------------------------------------------------------------------------
The authors' response addresses a small part of my concerns, so I change my score. However, I still recommend rejecting this submission (Details are shown in my response).




I read the authors' response and I do not think the authors address most of my concerns. I will state my opinions as follows:

1. The authors state that they choose hyperparameters based on the validation performance on a separate dataset. This is an incredible answer. Why not add this dataset to the trainset? It can boost performance. Also, the authors admit that the results are indeed sensitive to the hyper-parameters. In my experience, hyperparameter selection is an effective procedure for a supervised learning framework, but it is a very difficult procedure for unsupervised domain adaptation without target labels.  We can see most pioneer works are not sensitive to the hyper-parameters. Thus, the practicability of this algorithm can not be guaranteed.

2. The authors argue that their proposal holds even if $f_S, f_T \notin H$, which is proved in A.8. It seems like that they replace $\lambda$ with $\theta$ in equation 25 in A.8. Nevertheless, both $\lambda$ and $\theta$ are unobservable for the same reason (target labels are unaccessible). They assume that $\theta$ can be ignored, while prior works assume that $\lambda$ can be ignored. With this unobservable $\theta$, the theoretical guarantee of the algorithm is not complete, let alone the use of pseudo labels in the algorithm. 

3. It makes sense that we can find a hypothesis space containing $f_S$ (small source error). However, when we try to find a hypothesis space containing $f_T$, using pseudo labels still lacks theoretical guarantees and make the algorithm unreliable.  The authors state that this concern applies to every method that utilizes pseudo labels. However, those works just try to establish an empirical framework for domain adaptation application, they never claim rigorous theoretical results. I'm not sure if they are likely to perform poorly on difficult datasets. But intuitively, there are more challenges when utilizing pseudo labels on difficult datasets (domainnet). In my opinion, when we try to establish a theoretical framework, we should make every result as rigorous as possible. Furthermore, $\eta$ have different values in experiments, which also shows that utilizing pseudo labels may be unreliable.

The authors' response addresses a small part of my concerns, so I change my score. However, for the above reasons, I still recommend rejecting this submission.





Thanks for your comments.
We response to your concerns as follows.

1. We add a section in Appendix to explain the pipeline as well as the final objective.

2. We add some comparisons with "Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment" in Related Work.   
     Basically, their approach focus on the problem caused by exact distribution matching under shifting label scenario. Instead of the exact match, they deliberately relax the bound by replacing the H-divergence with $\beta$-admissible distance between the source and target. By optimizing this relaxed match and source error, they prove that under some circumstances (the target domain consists of clusters and each of them overlaps with the source domain), there exist alignments perfect for prediction. However, there also exist other alignments that satisfy the objective but predict poorly on target data if the conditions are not met, like the case (samples from different classes get mixed together) showed in Fig.1(b) (our paper) and Fig.2(b) (their paper) where our proposal can penalize but theirs can not. Besides, from their experiments, we can see that they mainly focusing on verifying the effectiveness for shifting label distributions. When we compare the normal situation (no label shifting), our proposal outperforms theirs by a lot (over 15%). In addition, they only test their method in simple cases (MNIST,USPS) where their conditions about the overlap between the source and target can be easily met. Therefore, we doubt whether their proposal can work in real world problem where our proposal shows reliable performance. 

    In short: a relaxed match does not necessarily reduce the joint error (ours does because the proposal is an upper bound of the joint error) but may be effective to the label shifting problem; our proposal works even if the domain gap is huge but we doubt about theirs.


3. As explained in A.9, our proposal is a more general upper bound compared to MCD. Because, when we set $\gamma=1$ or $\eta=1$ and $h=f_1$, our bound reduces to the MCD objective. The problem of MCD is that:
Our upper bound is optimized when $h = f_S$. However, it no longer holds after taking supreme, i.e.  setting $h = f_1$ (MCD) does not necessarily minimize the objective which includes joint error. Besides, MCD still tries to follow the upper bound proposed by Ben-David and ignore the joint error, while ours is motivated by reducing it.

    Smaller $\gamma$ leads to a more relaxed constraint on hypothesis space. However, it is expected to work only when the domain shift is so huge that we totally have no idea about the location of $f_T$, like  MNIST $\to$ SVHN (a classifier based on MINST can hardly classify SVHN). If the domain shift is small and we can approximate $f_T$ by learning source samples, then it would be better to choose a larger $\gamma$. 

    In VisDA experiments, the optimal value of $\eta=0.9$ achieves 79.7% accuracy. When we set $\eta=1.0$ and discard the information from the approximate target domain, the accuracy drops to 76.4%. Therefore, the THS constraint that allow us to leverage the information from pseudo labeled target domain is apparently useful. 

    We add a section in Appendix to explain how to choose hyper-parameters.

4. We never claim that our proposal achieves SOTA performance. 
The major contribution of this paper is the proposal of a target error upper bound that includes the joint error and a constrained hypothesis space to optimize it.

    Contrastive Adaptation Network only report the results on 2 benchmarks. We don't know their performance especially on some complicated tasks like Office-Home. Besides, their approach lacks theoretical guarantee of the generalization error even under a perfect pseudo labeling.


Thanks for your comments.

We are always ready to address your further concerns.

Thanks for your comments.
We response to your concerns as follows.

1. $C_{S,T}(f_S,f_T),h$ is not an upper bound of the joint error. The minimum (achieved when $h=f_S$) of the entire bound is.

    Our bound is not loose. Because if the other upper bounds are really "so tight",  we won't be able to improve the performance on so many different tasks.The methods based on Ben-David's bound only care about the source error and H-divergence which can be easily minimized given enough samples, an appropriate hypothesis space and modern techniques like GAN to match the distribution.
In that case, the corresponding accuracy should be close to 100%. However, in real world problems, not a single method could achieve that. We believe that is due to the increasing joint error during matching the marginal distribution which has also been pointed out by [1].
Motivated by this, we proposed a method that could potentially reduce the joint error without ignoring any term during the optimization. We think it is meaningless to emphasize how tight the bound is when you actually dropping a term that you could not handle during optimization. 

    The tightness of the proposal indeed relies on the constraint of hypothesis space. It is the reason we introduce SHS and THS in the following section so that we can build appropriate constraint to ease the optimization. We won't arbitrarily choose $f_1,f_2$ and let them contradict without any constraint. Even if they do contradict on every sample, $C_{S,T}(f_1,f_2)$ is still bounded. As defined in A.7, the loss function $\epsilon$ is bounded by M:$\epsilon(h,h')\leq M$, for all $h,h'\in H$, which follows the well-established theory proposed by [2].


2. Our proposal holds even if $f_S,f_T \notin H$, which is proved in A.8.

3. We do agree that the explanation of this part is a little confusing. 

    We add some explanations in Sec.3.2.1 to make it easier to understand.

    We state in the paper that it is difficult to actually build a classifier with a specific accuracy. And a weighted source error wighted by $\gamma$ does not give us a classifier with an accuracy of $\gamma$. However, as we explain in the training procedure (Fig.13(b)), a small $\gamma$ allows $f_2$ to misclassify more source samples because the source error is together trained with the CMD which $f_2$ has to maximize.

4. Indeed the training process lacks theoretical guarantee when introducing pseudo labels. However, this concern applies to every method that utilize pseudo labels, and deriving a generalization error bound for pseudo label training is not the main theme of this paper.The major contribution of this paper is the proposal of a target error upper bound that includes joint error and a constrained hypothesis space to optimize it.We choose pseudo labeling only because it is a naive way to find a feasible constraint for the hypothesis space. Besides, the experiments show that, even for the case where the two domains are distant (VisDA, Office-Home), the proposal using pseudo labels always improves the performance.

    Domainnet is just a dataset contains more domains, and it does not increase the complexity of the task (because the adaptation is always made from one source to one target). Besides, Office-Home contains 4 domains (art, clipart, product, real-world) which are also included in Domainnet. Our proposal achieves reliable performance on this dataset, which means our proposal is effective for the practical problems.

5. We add a section in Appendix to explain how to choose hyper-parameters.
    In all experiments, the hyper-parameters $\gamma,\eta$ are set based on the validation performance on a separate dataset. The results are indeed sensitive to the hyper-parameters, however, it also suggests that an appropriate choice of the hyper-parameters can significantly boost the performance, which allows us to select the hyper-parameters via a validation dataset efficiently.

6. We do not think our research and [iii] share a similar motivation. [iii] believe that feature discriminability will enhance transferability and achieve this by penalizing the largest singular values so that other eigenvectors can be relatively strengthened to boost the feature discriminability. We did not mention anything about the discriminability. On the contrary, we address the problem faced by the traditional methods: simply matching the distribution may increase the joint error.

    [iii] claims that their method can control $\lambda$ but it lacks theoretical proof. Besides, [iii] does not provide any estimation of the joint error. In their experiments, the ideal joint hypothesis is found by training a classifier on all source and target data with labels.

[1] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In ICML, 2019.
[2] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In COLT, 2009.

Thanks for your comments.
We response to your concerns as follows.

1. We add a section in Appendix to explain how to choose hyper-parameters. 

    In all experiments, the hyper-parameters $\gamma,\eta$ are set based on the validation performance on a separate dataset.

    The sensitivity analysis of the hyper-parameters is showed in Fig.4(a), Fig.5(a),Fig.5(b).

    The intuition of $\eta$ is explained in Sec.3.2.3. It allows us to leverage the information from pseudo labeled target samples so that we can build a more reliable constraint for $f_2$. If we are confident about the pseudo labels, then we tend to choose a smaller $\eta$. If not, it would be better to use a larger $\eta$

2. We add an ablation study on VisDA dataset in Appendix.

We revise our paper as follows:
1. We add a section A.12 to explain how to choose the hyper-parameters.
2. We add a section A.13 to explain the detail of the training procedure.
3. We add an ablation study on VisDA in A.6.
4. We add a comparison with "Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment" in Sec.2.
5. We add some explanations for the role of $\gamma$ in Sec.3.2.1.

This paper studies the problem of unsupervised domain adaption, giving a theoretical analysis that yields a new upper bound on the target error.  In particular, in contrast to much existing work, this paper focuses on the role of the ideal joint source/target error in the upper bound.  The paper proposes two methods for improved results within the framework of the upper bound - constraining the searched hypothesis space in two possible manners (source-driven and target driven), and proposing a a margin-based measure of dissimilarity between hypotheses.  

Positives:
+ derivation of novel upper bound for target error
+ demonstrates that two existing methods for unsupervised domain adaptation can be derived as special cases of the proposed framework
+ ablation study demonstrates the relative improvements from each proposed component
+ good empirical results relative to existing work

Negatives:
- one main concern I have is with the role of $\eta$ in the target-drive hypothesis space constraint.  First, it would ideally be nice to have some intuition about its role and how it might be set.  Second, it appears different values are optimal depending on the data set, sometimes being set to 0, and other times 0.9.  I would have liked to see some sensitivity analysis to this value.  More importantly, it appears that the value is being set to directly optimize the test performance?  If so, it should instead be set based on some separate validation data, and the accuracy numbers updated to reflect this.
- would have been nice if the ablation study had been applied to more than just one specific data set/direction

Overall, I found this to be a nice paper, but would like to see the concern above about $\eta$ addressed.

This paper proposes a new algorithm for unsupervised domain adaptation: taking the adaptability term (the joint error) into consideration instead of minimizing only the domain discrepancy. The authors also justify their method with some theoretical intuitions. Empirical results show that their method achieves state-of-the-art performance.

The paper is overall clearly organized and easy to follow. Minimizing the joint error along with the domain discrepancy is a novel idea.

I note that this paper appeared in iclr 2020 (paper 559). For the previous version, reviewers raised concerns about
1)	The realizability assumption: f_1 and f_2 are not in H.
2)	The derivation of the general bound can be confusing.
3)	Lack of ablation study to probe into where the improvement of performance comes.

In the current manuscript, the authors deal with the problems above:
1)	Add the analysis where the realizability assumption does not hold.
2)	Reorganize the derivation to make their intuition clear.
3)	Also add some part of the ablation study.

I believe the authors have addressed most of the concerns in iclr 2020 (paper 559). Thus this work can be more ready for publication. 


This paper presents a novel approach to tackle the problem of unconstrained joint error in previous unsupervised domain adaptation theory. It builds up a new upper bound that, under certain circumstances, reduces to an upper bound of optimal joint error. In practice it instantiates two methods SHS and THS to approximate a hypothesis space that most likely contains f_T. Furthermore, it proposes a novel cross margin discrepancy to alleviate instability during adversarial learning.

The main weaknesses of this paper include the following aspects:
1.	This paper is not well written and some parts are hard to follow. It lacks necessary logical transition and important figures. For example, it lacks explanations to support the connection between the proposed training objective and the Cross Margin Discrepancy. Also, it should at least contain one figure to explain the overall architecture or training pipeline.
2.	The authors claim that there is still no research focusing on the joint error for UDA. But this problem of arbitrarily increased joint error has already been studied in previous works like “Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment”, in ICML2019. The authors should discuss on that work and directly illustrate the relationship between that work and the proposed one, and why the proposed method is better.
3.	Although the joint error is indeed included in the proposed upper bound, in practice the authors have to use Source-driven Hypothesis Space and Target-driven Hypothesis Space to obtain approximation of f_{S} and f_{T}. To me, in practice the use of three classifiers h, f_{1}, f_{2} is just like an improvement over MCD. Hence, I doubt whether the proposed method can still simultaneously minimize the domain discrepancy and the joint error. For example, as shown in the Digit experiments, the performance is highly sensitive to the choice of \gamma in SHS, and sometimes the optimal \gamma value is conflicting for different domains in the same dataset, which is strange since according to the paper’s theorem, smaller \gamma only means more relaxed constraint on hypothesis space. Also, as shown in the VisDA experiments, the optimal value of \eta is close to 1, which means classification error from the approximate target domain is basically useless.
4.	The benchmark results are inferior to the state-of-the-art methods. For instance, the Contrastive Adaptation Network achieves an average of 87.2 on VisDA2017, which is much higher than 79.7 achieved by the proposed method. And the same goes with Digit, Office31, and Office-Home dataset.


Meta-review: This submission provides a new bound and derived method for unsupervised domain adaptation, based on adversarial training. 
The method is then extensively evaluated empirically.

Pro:
- the proposed method seems empirically successful

Con:
- I agree with one of the reviewers that the presented theoretical justification is not convincing. There should be assumptions on how f_S and and f_T relate in order to drive meaningful guarantees in the proposed framework. The presented results here seem to be a case of a method that was motivated by some theoretical consideration, and worked well after some heuristic approximations were made. It would be nice to see a cleaner analysis of the conditions under which the method will be successful.

Decision: withdrawn-rejected-submissions