# An Attention-Based Full-Scale Fusion Network for Segmenting Roof Mask from Satellite Images
Despite the fact that these public datasets provide high-precision building masks, manually annotated, we found that the building masks are generally larger than or equal to the mask area of the building roof (Figure 1). When we use building masks to evaluate the photovoltaic capacity that can be installed on building roofs, we receive an exaggerated evaluation result. Taking the three buildings in Figure 1 as an example, from the satellite image, we can calculate that their roof areas are 1184 m\({}^{2}\), 1173 m\({}^{2}\), and 1140 m\({}^{2}\), respectively, with corresponding building areas of 2551 m\({}^{2}\), 2750 m\({}^{2}\), and 2549 m\({}^{2}\). This means that the estimated roof photovoltaic power generation and carbon emission reduction potential calculated from the building area will be more than twice the actual value. One may argue that this issue is due to the use of heavily tilted remote sensing images. However, considering the large surface area of the Earth, remote sensing satellites cannot continuously adjust their positions to capture vertical images of each region, which is too inefficient and costly. Therefore, we cannot obtain vertical images of all regions. Additionally, accurately labeling the contours of building roofs on satellite images is a time-consuming, labor-intensive, and extremely challenging task. However, it is much simpler to only annotate whether there are buildings (roofs) in the image. In this paper, we refer to data that only annotate whether there are buildings (roofs) in the image as weakly labeled data. Therefore, determining how to fully utilize weakly labeled data to assist in training networks for roof segmentation is a worthwhile research objective. Researchers applied machine learning algorithms such as agent-based modeling [23], the statistical method [24], CNN [25], FCN [19], Mask-RCNN [26], and UNet [27] in order to segment buildings from satellite images. Due to the use of early-developed computer vision algorithms with limited performance, the error in estimating the roof photovoltaic potential based on these algorithms is relatively large. In this study, we attempted to solve the above limitations by improving current roof area estimation algorithms and developing a new building roof segmentation dataset. The accuracy of city-scale roof area estimation can be significantly improved. The main contributions of this study can be summarized as the following four aspects:  
(1) We design an attention-based residual ulback, which is the basic block of the proposed network, to model the channel relationship. (2) We propose a full-scale feature fusion strategy based on an attention mechanism to achieve full fusion and utilization of features in various stages of the network. Specifically, when fusing features at different scales, we use attention mechanisms to learn different weights for features at different scales. (3) We develop a network with a ladder-like structure to enhance the sensitivity of the model to roofs using weakly labeled data. Specifically, we use image-level classification tasks to assist in training pixel-level semantic segmentation network. (4) We contribute a new dataset for the study of building roof segmentation algorithms. Unlike existing datasets, the new dataset calculates roof area based on building roof boundaries rather than building boundaries. We verify the effectiveness of the proposed network on the new dataset and WHU dataset through experiments. We organize the rest of this study as follows. In Section 2, we briefly review the works that are most closely related to this paper. The details of the proposed attention-based full-scale fusion (AFSF) network are presented in Section 3. The new roof segmentation dataset is described in Section 4. The experimental results are provided in Section 5 to show the effectiveness of the proposed approach. Finally, some concluding remarks are given in Section 7. Figure 1: The difference in area between buildings and their roofs in satellite images.