# MFFNet: A Building Extraction Network for Multi-Source High-Resolution Remote Sensing Data
## 3 Model and Evaluation Metrics
### MFFNet
After q processing, the dimensions become (B,H*W,C/8) for \(W^{q}\), (B,C/8,H*W) for \(W^{k}\) after \(k\) processing, and (B,C,H*W) for \(W^{v}\) after v processing. \(W^{q}\) and \(W^{k}\) are matrix-multiplied on the channel dimension to obtain the matrix qk. After the matrix qk goes through the softmax layer to obtain the weight value between the two feature maps, it is matrix-multiplied with \(W^{v}\) on the channel dimension and returns to the original batch feature map size to obtain the matrix qkv. Finally, the matrix qkv is multiplied by the learnable parameter gamma and the product is added to the original input feature map after the self-attention learning module to obtain the result. The initial value of the learnable parameter gamma is 1. As training progresses, the gamma parameter adaptively obtains the optimal gamma value within the learnable range (Equation (2)). \[Output=sigmoid(conv3\times 3(concatenate(mean(X),max(X))))\times X \tag{1}\]  
where \(Output\) is the output feature map, \(conv3\times 3\) is a \(3\times 3\) convolution, and _concatenate_ represents splicing in the same dimension. \[Attention=gamma\times softmax\bigg{(}W^{q}\times\bigg{(}W^{k}\bigg{)}^{T} \bigg{)}\times W^{v}+X \tag{2}\]  
where _Attention_ represents the output feature map, \(gamma\) is a learnable parameter, \(X\) represents the input feature map, and \(W^{q}\), \(W^{v}\), and \(W^{k}\) are conditional feature maps generated through deformation. Figure 3: Residual block. The multi-scale spatial semantic extraction module differs from the deep semantic information extraction module in that the latter does not downsample and will not cause changes in the feature map size. The deep semantic information extraction module branches the feature map before the spatial attention block of the first operation of the multi-scale spatial semantic extraction module, obtaining a feature map parallel to the deep semantic information extraction module. This feature map will go through two consecutive residual convolution blocks and then be connected to an atrous spatial pyramid pooling (ASPP) block. The specific structure of ASPP blocks is shown in Figure 6. The feature map entering ASPP blocks will undergo five parallel operations. These five operations are a \(1\times 1\) convolution, three \(3\times 3\) convolutions with dilation rates of 6, 12, and 18, respectively, and a global average pooling followed by a \(1\times 1\) convolution and upsampling. The results of these five parallel operations are concatenated and then passed through a \(3\times 3\) convolution, after which they are subjected to batch normalization and a ReLU activation function. Figure 4: Spatial attention module. Figure 5: Learnable self-attention module. The decoder part of MFFNet consists of an upsampling layer, convolutional layer, multi-feature concatenation layer, and segmentation output layer. The upsampling layer, convolutional layer, and multi-feature concatenation layer together form a group. To integrate multi-scale features, it is ensured that the number of groups in the decoder is the same as in the encoder. The upsampling layer in the decoder uses a 2 \(\times\) 2 transposed convolution, followed by batch normalization and a ReLU activation function. The convolutional layer includes a 3 \(\times\) 3 convolution operation, batch normalization, and a ReLU activation function. Before ReLU activation, a dropout layer with a rate of 0.5 is added to prevent overfitting. After this convolutional layer, the image size remains unchanged, but the number of feature map channels is reduced to half of the input. The multi-feature concatenation layer concatenates the feature map from the corresponding spatial attention extraction block in the encoder with the feature map after upsampling and convolution. Notably, the fourth group of the multi-feature concatenation layer concatenates the feature map extracted from the encoder of the corresponding size, the feature map after multi-scale spatial semantic extraction, and the feature map after upsampling and convolution.