  Visual domain adaptation involves learning to classify images from a target
visual domain using labels available in a different source domain. A range of
prior work uses adversarial domain alignment to try and learn a domain
invariant feature space, where a good source classifier can perform well on
target data. This however, can lead to errors where class A features in the
target domain get aligned to class B features in source. We show that in the
presence of a few target labels, simple techniques like self-supervision (via
rotation prediction) and consistency regularization can be effective without
any adversarial alignment to learn a good target classifier. Our Pretraining
and Consistency (PAC) approach, can achieve state of the art accuracy on this
semi-supervised domain adaptation task, surpassing multiple adversarial domain
alignment methods, across multiple datasets. Notably, it outperforms all recent
approaches by 3-5% on the large and challenging DomainNet benchmark, showing
the strength of these simple techniques in fixing errors made by adversarial
alignment.
