###### Abstract  
Building extraction from high spatial resolution imagery (HSRI) plays an important role in the remotely sensed imagery application fields. However, automatically extracting buildings from HSRI is still a challenging task due to such factors as large size variations of buildings, background complexity, variations in appearance, etc. Especially, it is difficult to extract both crowded small buildings and large buildings with accurate boundaries. To address these challenges, this paper presents an end-to-end encoder-decoder model to automatically extract buildings from HSRI. The designed network, called AEUNet++, is based on UNet++, attention mechanism and multi-task learning. Specifically, the AEUNet++ introduces the UNet++ as the backbone to extract multiscale features. Then, the attention block is used to effectively fuse different-layer feature maps instead of direct concatenation in the output of traditional UNet++, which can assign adaptive weights to different-layer feature maps as their relative importance to enhance the sensitivity of the mode and suppress the background influence of irrelevant features. To further improve the boundary accuracy of the extracted buildings, the boundary geometric information of buildings is integrated into the proposed model by a multi-task loss using a proposed distance class map during training of the network, which simultaneously learns the extraction of buildings and boundaries and only outputs extracted buildings while testing. Two different data sets are utilized for evaluating the performance of AEUNet++. The experimental results indicate that AEUNet++ produces greater accuracy than U-Net and the original UNet++ architectures and, hence, provides an effective method for building extraction from HSRI. building extraction; high spatial resolution imagery (HSRI); UNet++ +
Footnote †: journal:  
Recently, deep learning technologies, especially the convolutional neural network (CNN), have shown potential in remote sensing applications [12]. The use of deep learning for building extraction has attracted considerable attention, because it has good feature representation ability [13; 14; 15]. However, the typical CNN models, such as AlexNet [16] and VGG networks [17], are usually used to mark the entire image, not to label each pixel to a class. Evolving from CNN, the fully convolutional network (FCN) contributes greatly to semantic segmentation, which can assign a label to each pixel. Inspired by the great success of FCN in semantic segmentation, building extraction can be completed by semantic segmentation. Thus, on the basis of FCN architecture, many FCN-based models have been proposed for extracting buildings [18; 19; 20]. However, due to pooling operators in the CNN or the availability of only a portion of features to generate the final feature map, the problems of missing image content details and poor boundary accuracy can arise. Furthermore, due to large intra class variance and small inter class variance in the pixel values and similarities between buildings and their backgrounds, as well as the materials, proportions and illumination of buildings, extensive salt-and-pepper noise and boundary ambiguities exist in the building extraction results produced by the FCN-based methods. To address the problem of spatial location and content details induced by the downsampling operations in the CNN, many improved methods have been presented for achieving more accurate edges by using more boundary geometric information or post-processing operators. [PERSON] et al. [18] designed a new FCN for extraction of buildings from images, in which post-processing conditional random fields (CRFs) are added at the end of the network to refine the coarse pixel level label predictions to produce fine-grained segmentation results. The principle is to transform the pixel classification problem into a probabilistic reasoning problem. [PERSON] et al.