  Object recognition is a key enabler across industry and defense. As
technology changes, algorithms must keep pace with new requirements and data.
New modalities and higher resolution sensors should allow for increased
algorithm robustness. Unfortunately, algorithms trained on existing labeled
datasets do not directly generalize to new data because the data distributions
do not match. Transfer learning (TL) or domain adaptation (DA) methods have
established the groundwork for transferring knowledge from existing labeled
source data to new unlabeled target datasets. However, current DA approaches
assume similar source and target feature spaces and suffer in the case of
massive domain shifts or changes in the feature space. Existing methods assume
the data are either the same modality, or can be aligned to a common feature
space. Therefore, most methods are not designed to support a fundamental domain
change such as visual to auditory data. We propose a novel deep learning
framework that overcomes this limitation by learning separate feature
extractions for each domain while minimizing the distance between the domains
in a latent lower-dimensional space. The alignment is achieved by considering
the data manifold along with an adversarial training procedure. We demonstrate
the effectiveness of the approach versus traditional methods with several
ablation experiments on synthetic, measured, and satellite image datasets. We
also provide practical guidelines for training the network while overcoming
vanishing gradients which inhibit learning in some adversarial training
settings.
