  Unsupervised domain adaptation aims to learn a task classifier that performs
well on the unlabeled target domain, by utilizing the labeled source domain.
Inspiring results have been acquired by learning domain-invariant deep features
via domain-adversarial training. However, its parallel design of task and
domain classifiers limits the ability to achieve a finer category-level domain
alignment. To promote categorical domain adaptation (CatDA), based on a joint
category-domain classifier, we propose novel losses of adversarial training at
both domain and category levels. Since the joint classifier can be regarded as
a concatenation of individual task classifiers respectively for the two
domains, our design principle is to enforce consistency of category predictions
between the two task classifiers. Moreover, we propose a concept of vicinal
domains whose instances are produced by a convex combination of pairs of
instances respectively from the two domains. Intuitively, alignment of the
possibly infinite number of vicinal domains enhances that of original domains.
We propose novel adversarial losses for vicinal domain adaptation (VicDA) based
on CatDA, leading to Vicinal and Categorical Domain Adaptation (ViCatDA). We
also propose Target Discriminative Structure Recovery (TDSR) to recover the
intrinsic target discrimination damaged by adversarial feature alignment. We
also analyze the principles underlying the ability of our key designs to align
the joint distributions. Extensive experiments on several benchmark datasets
demonstrate that we achieve the new state of the art.
