# An Attention-Based Full-Scale Fusion Network for Segmenting Roof Mask from Satellite Images
## 2 Related Work
### Deep Learning Methods for Semantic Segmentation of Remote Sensing Imagery  
On the one hand, semantic segmentation of remote sensing imagery is an important prerequisite for many applications of remote sensing images. On the other hand, in the past decade, deep learning has surpassed traditional algorithms in performance and become the mainstream technology in computer vision applications such as object detection [28], segmentation, and recognition. Intuitively, several classic deep learning-based semantic segmentation algorithms have been transferred to the semantic segmentation of remote sensing images [29]. A Fully Convolutional Network (FCN) [30] can accept input images of any size and classify them at the pixel level, thus solving the problem of semantic-level image segmentation. Several works extend FCN to the task of semantic segmentation of remote sensing imagery [19; 31; 32]. SegNet [33], consisting of an encoder and decoder, assists in restoring spatial information and improving the accuracy of boundary segmentation by reusing pooling position indices, and is applied to remote sensing image segmentation tasks [34]. DeepLab [35] is an extension of FCN but employs atrous convolution to enlarge the scope of filters. In [36], authors develop models based on DeepLab for land cover segmentation and road segmentation, respectively. UNet [37] shares a similar architecture to SegNet. The difference between UNet and SegNet is that UNet transfers the features extracted by the encoders to the corresponding decoders then concatenates them into upsampled feature maps. Several works [38; 39] extend UNet to the remote sensing image segmentation task. DIResUNet [40] integrates an inception module and a dense global spatial pyramid pooling module in UNet to simultaneously extract multi-level features and contextual features. The authors in [41] introduce attention operation in the fusion of hierarchical features corresponding to the encoder and decoder of UNet network in order to achieve better multi-level feature fusion and improve the model's ability to detect deforestation. The multi-attention-based semantic segmentation network proposed in [42] uses a coordinate attention-based residual network in the encoder and a content-aware reorganization module in the decoder to improve the extraction capability of the network. To solve the multi-scale problem, a fused attention module is adopted for feature map fusion. In [43], Depth-wise Pyramid Pooling (DPP) block and a dense block with multi-dilated depth-wise residual connections are designed to form the DPPNet for land cover segmentation from high-resolution satellite images, which takes full advantage of pyramid structure and features of different levels.  
To further improve the accuracy of roof segmentation, we simultaneously introduce attention mechanisms in each basic feature extraction module of U2 Net [44] and the operation of fusing features of different scales (full-scale feature fusion strategy based on attention mechanism). In fact, U2 Net is an improved version of the UNet and has a two-level nested U structure. To the best of our knowledge, the full-scale feature fusion strategy based on attention mechanism is the first work which can fully integrate the features of all levels in the U2 Net (both encoder and decoder) through attention method, which is useful for semantic segmentation tasks. Furthermore, we add another branch to the network to further enhance the sensitivity of the model to building roofs by utilizing weakly labeled data.