  Domain adaptation has been widely explored by transferring the knowledge from
a label-rich source domain to a related but unlabeled target domain. Most
existing domain adaptation algorithms attend to adapting feature
representations across two domains with the guidance of a shared
source-supervised classifier. However, such classifier limits the
generalization ability towards unlabeled target recognition. To remedy this, we
propose a Transferable Semantic Augmentation (TSA) approach to enhance the
classifier adaptation ability through implicitly generating source features
towards target semantics. Specifically, TSA is inspired by the fact that deep
feature transformation towards a certain direction can be represented as
meaningful semantic altering in the original input space. Thus, source features
can be augmented to effectively equip with target semantics to train a more
transferable classifier. To achieve this, for each class, we first use the
inter-domain feature mean difference and target intra-class feature covariance
to construct a multivariate normal distribution. Then we augment source
features with random directions sampled from the distribution class-wisely.
Interestingly, such source augmentation is implicitly implemented through an
expected transferable cross-entropy loss over the augmented source
distribution, where an upper bound of the expected loss is derived and
minimized, introducing negligible computational overhead. As a light-weight and
general technique, TSA can be easily plugged into various domain adaptation
methods, bringing remarkable improvements. Comprehensive experiments on
cross-domain benchmarks validate the efficacy of TSA.
