# Dual Hybrid Attention Mechanism-Based U-Net for Building Segmentation in Remote Sensing Images
## 2 Methodology
### DHAU-Net Structure  
A notable advantage of the U-Net architecture is the incorporation of skip connections between the encoder and decoder. This effectively addresses the problem of the loss of detailed information during the compression and transmission from the encoder to the decoder. However, in the traditional skip connections, all transmitted shallow vector information is treated equally. No useful information that is important to the target task is further screened in the shallow information, and no disturbance information, such as noise, is weakened. As a result, it leads to insufficient segmentation accuracy and significant noise interference in the final results. An attention mechanism can effectively imitate the human visual information processing system, focusing on processing the commands given by objects stimulated by the neural system of the brain, which ignores irrelevant background information, and improves the efficiency of information extraction, allowing for more efficient feature extraction. Therefore, based on U-Net, this paper assigns corresponding weights to different shallow information, which is denoted by replacing the skip connections of U-Net with channel and spatial attention mechanisms, in order to identify the most relevant aspects of the target segmentation task. The proposed approach enables one to focus on extracting significant and useful information from the input data that is directly related to the current output, thereby improving the quality of the segmentation results. Additionally, the attention mechanism helps disregard noise and redundancy in the input, further enhancing the segmentation accuracy and reducing interference from irrelevant information. DHAU-Net is an end-to-end deep CNN (DCNN) based on the U-Net architecture. DHAU-Net contains two subnetworks and an attention model: a downsampling network for extracting building feature maps from RGB images, an upsampling network is employed to reconstruct the extracted feature maps, generating an output image of the same size as the input image. And, a DHA model structure connected between the two network architectures. At the end of the network, we use a convolutional layer with a kernel size of \(1\times 1\) to obtain the final segmentation result. Figure 2 shows the detailed architecture of the DHAU-Net network. In the downsampling network, we choose RGB remote sensing images of size \(3\times H\times W\) as the input feature images. First, the images pass through two convolutional layers; the padding is 1, the stride is \(1\times 1\), and the filter size is \(3\times 3\). On one hand, the output feature image is input into the DHA feature extraction module and then transmitted to the corresponding upsampling block. On the other hand, the feature map undergoes a max  
Figure 2: Net architecture. (**a**) U-Net; (**b**) DHAU-Net; (**c**) DHA extraction module. pooling operation, reducing its size by half. This combined convolution and max-pooling operation is reiterated four times, resulting in a final layer count increased from the initial value of 3 to 512. The image size is finally \(\frac{1}{16}\) of the original size. Finally, at the bottom layer of the network, after the feature images pass through a convolutional layer with a filter size of \(3\times 3\), the number of feature image layers is increased to 1024, and then the images are processed through the DHA extraction module as the inputs of the upsampling block. Four consecutive downsampling blocks are used to extract deeper image features for model training and generate more accurate classifications. In the upsampling block of DHAU-Net, each feature map is first passed through a convolutional layer with a filter size of \(3\times 3\) to reduce the feature maps from 2048 layers to 512 layers. The 512-layer feature maps are then processed through a deconvolution layer with a filter size of \(2\times 2\), keeping the number of feature layers the same while doubling the image size.