  We study the problem of robust domain adaptation in the context of
unavailable target labels and source data. The considered robustness is against
adversarial perturbations. This paper aims at answering the question of finding
the right strategy to make the target model robust and accurate in the setting
of unsupervised domain adaptation without source data. The major findings of
this paper are: (i) robust source models can be transferred robustly to the
target; (ii) robust domain adaptation can greatly benefit from non-robust
pseudo-labels and the pair-wise contrastive loss. The proposed method of using
non-robust pseudo-labels performs surprisingly well on both clean and
adversarial samples, for the task of image classification. We show a consistent
performance improvement of over $10\%$ in accuracy against the tested baselines
on four benchmark datasets.
