###### Abstract  
Deep learning-based building extraction methods have achieved a high accuracy in closed remote sensing datasets. In fact, the distribution bias between the source and target domains can lead to a dramatic decrease in their building extraction effect in the target domain. However, the mainstream domain adaptation methods that specifically address this domain bias problem require the reselection of many unlabeled samples and retraining in other target domains. This is time-consuming and laborious and even impossible at small regions. To address this problem, a novel domain adaptation framework for cross-domain building extraction is proposed from a perspective of anomaly detection. First, the initial extraction results of images in the target domain are obtained by a source domain-based pre-trained model, and then these results are classified into building mixed and non-building layers according to the predicted probability. Second, anomalous objects in the building layer are detected using the isolation forest method. Subsequently, the remaining objects in the building layer and the objects in the non-building layer are used as positive and negative samples, respectively, to reclassify the mixed layer using the random forest classifier. The newly extracted objects are fused with the remaining objects in the building layer as the final result. Four different experiments are performed on different semantic segmentation models and target domains. Some experimental results indicate that our framework can improve cross-domain building extraction compared to the pre-trained model, with an 8.7% improvement in the F1 metric when migrating from the Inria Aerial Image Labeling dataset to the Wuhan University dataset. Furthermore, experimental results show that our framework can be applied to multiple target domains without retraining and can achieve similar results to domain adaptation models based on adversarial learning. building extraction; remote sensing image; domain adaptation; semantic segmentation; anomaly detection +
Footnote †: journal: Medical  
dataset (target domain), it is necessary to reselect the training samples and carry out the training again, which is often time-consuming and laborious [10,11]. Currently, the mainstream approach to solve the above-mentioned model migration problem is domain adaptation [12, 13, 14]. The domain adaptation methods aim to transfer the knowledge learned from a source domain to a new target domain [15,16]. For example, [PERSON] et al. [17] presented a DAugNet for multisource, multitarget, and life-long domain adaptation problems by using only one encoder, one decoder, and one discriminator. [PERSON] et al. [18] proposed a joint pixel- and representation-level network based on a cycle generative-adversarial network and adversarial learning. These domain adaptation methods mainly use the idea of adversarial learning and do not require labeled data in the target domain [19]. This indeed improves the performance of cross-domain building extraction while reducing the labeling workload. However, these methods still require a large number of unlabeled samples in the target domain to complete the training process with the labeled samples in the source domain and, even worse, they need to be retrained whenever the target domain changes. This condition is difficult to be satisfied for a small scope of cross-domain building extraction scenarios. Moreover, it should be noted that the research on domain adaptation is still in its initial stage. To address the abovementioned issues, we attempt to investigate the domain adaptation method from a novel perspective of anomaly detection and propose an anomaly detection-based domain adaptation framework (AD-DAF) for cross-domain building extraction from remote sensing images in this paper. The framework is designed with anomaly detection and reclassification processes based on a conventional pre-trained model.