Open-set domain adaptation for scene classification using multi-adversarial learning

Authors:
raw_author_name: Juepeng Zheng
raw_affiliation_string: School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China
raw_author_name: Yibin Wen
raw_affiliation_string: School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China
raw_author_name: Mengxuan Chen
raw_affiliation_string: Tsinghua University (Department of Earth System Science)- Xi'an Institute of Surveying and Mapping Joint Research Center for Next-Generation Smart Mapping, Beijing, China
raw_author_name: Shuai Yuan
raw_affiliation_string: Department of Geography, The University of Hong Kong, Hong Kong, China
raw_author_name: Weijia Li
raw_affiliation_string: School of Geospatial Engineering and Science, Sun Yat-Sen University, Zhuhai, China
raw_author_name: Yi Zhao
raw_affiliation_string: Tsinghua University (Department of Earth System Science)- Xi'an Institute of Surveying and Mapping Joint Research Center for Next-Generation Smart Mapping, Beijing, China
raw_author_name: Wenzhao Wu
raw_affiliation_string: National Supercomputing Center in Wuxi, Wuxi, China
raw_author_name: Lixian Zhang
raw_affiliation_string: Tsinghua University (Department of Earth System Science)- Xi'an Institute of Surveying and Mapping Joint Research Center for Next-Generation Smart Mapping, Beijing, China
raw_author_name: Runmin Dong
raw_affiliation_string: Tsinghua University (Department of Earth System Science)- Xi'an Institute of Surveying and Mapping Joint Research Center for Next-Generation Smart Mapping, Beijing, China
raw_author_name: Haohuan Fu
raw_affiliation_string: National Supercomputing Center in Wuxi, Wuxi, China; Tsinghua University (Department of Earth System Science)- Xi'an Institute of Surveying and Mapping Joint Research Center for Next-Generation Smart Mapping, Beijing, China

Abstract: Domain adaptation methods are able to transfer knowledge across different domains, tackling multi-sensor, multi-temporal or cross-regional remote sensing scenarios as they do not rely on labels or annotations in the target domain. However, most of the previous studies have focused on closed-set domain adaptation, based on the assumption that the source and target domains share identical class labels. Real-world scenarios are typically more complex, and the model could potentially encounter novel classes that are not previously included in the source domain, commonly referred to as "unknown" classes. Here we investigate the open-set domain adaptation scenario in the field of remote sensing scene classification, where there is a partial overlap between the label space of the target domain and that of the source domain. To deal with this problem, we propose a novel open-set domain adaptation method for scene classification using remote sensing images, which is named Multi-Adversarial Open-Set Domain Adaptation Network (MAOSDAN). Our proposed MAOSDAN consists of three main components. First, we employ an attention-aware Open Set BackPropagation (OSBP) to better distinguish the "unknown" and "known" samples for the target domain. Then, an auxiliary adversarial learning is designed for mitigating the negative transfer effect that arises from forcefully aligning the "unknown" target sample in network training. Finally, we adopt an adaptive entropy suppression to increase the probability of samples and prevent some samples from being mistakenly classified. Our proposed MAOSDAN achieves an average score of 75.07% in three publicly available remote sensing datasets, which significantly outperforms other open-set domain adaptation algorithms by attaining 4.52∼17.15%. In addition, MAOSDAN surpasses the baseline deep learning model with 18.12% improvement. A comprehensive experimental evaluation demonstrates that our MAOSDAN shows promising prospects in addressing practical and general domain adaptation scenarios, especially in scenarios where the label set of the source domain is a subset of the target domain.