## 3 Experiments and Results
### Experiments Analysis  
As shown in Table 2, a total of Transform-based and CNN-based models are compared in the same experimental environment. TransUNet [36] and SwinUNet [37] are improved models based on the Transformer structure. TransUNet is a network combined with the Transformer module. The Transformer module is used to establish global dependencies, and the detailed information is retained through the U-Net structure. Among them, a Transformer is a neural network structure, which captures long-distance dependencies through the global self-attention mechanism, so as to better understand the context information of the input sequence. The multi-head soft attention mechanism performs self-attention and weight calculation on different positions of the input, allowing the model to strengthen or weaken the degree of attention according to different positions. TransUNet may perform better for small-size images or low-resolution images, but requires enormous data for training, and has a poor ability to extract local detail information. SwinUNet is a network structure that combines Swin Transformer and U-Net. Swin Transformer is a network structure based on Transformer that introduces Swin Block and window attention mechanism. Swin Block as a basic building block includes a displacement layer and a window attention layer. The traditional self-attention mechanism requires the entire input sequence when calculating the attention weight. Perform global operations, which is computationally expensive for large-scale images. Swin Transformer divides large-scale image input into multiple fixed-size image blocks (called windows), and calculates self-attention weights at each window level to reduce computational complexity. SwinUNet may encounter resource constraints when processing high-resolution images or large-scale datasets. In order to compare with the traditional cloud extraction model based on CNN, the added CNN-based includes the combination network of U-Net, and DeeplabV3+ for comparative experiments. U-Net is a classic structure for image semantic segmentation. Owing to the limitations of the local receptive field and upsampling layer of the network structure, U-Net often performs poorly in the segmentation of fine boundaries, including important global context information. For more demanding tasks, the performance of U-Net is limited. And DeeplabV3+ [31] is a convolutional neural network image segmentation model based on pyramid pooling. This model improves the traditional DeepLabV3+ by introducing atrous convolution and decoder modules, but for small-sized target objects, its detection and segmentation accuracy may be reduced. MSACN-small is a deformation of the proposed method. MSACN is the main network proposed in this paper. The number of multi-head modules of MSACN in Table 2 is 8. In the ablation experiment in Section 3.3, the relationship between the heads of soft attention modules and the model accuracy will be further explored. In Table 3, IoU and accuracy take the optimal values in all rounds, while Precision, Recall and F1 take the values in the round where the IoU optimal value is located (hereinafter MSACN refers to MSACN-8 head). Figure 6 is the trend chart of the five evaluation indicators in 200 epochs. As illustrated in Table 3, the MSACN-8 head performs outstandingly in the ZY-3 satellite image dataset. The accuracy is 2.03% higher than the Transformer \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Accuracy** & **IoU** & **Precision** & **Recall** & **F1** \\ \hline TransUNet & 0.9653 & 0.8988 & 0.9528 & 0.9409 & 0.9464 \\ SwinUNet & 0.9189 & 0.8147 & 0.8908 & 0.9011 & 0.8941 \\ U-Net & 0.9704 & 0.9285 & 0.9595 & 0.9665 & 0.9628 \\ DeeplabV3+ & 0.9714 & 0.9218 & 0.9589 & 0.9559 & 0.9574 \\ MSACN-small & 0.9746 & 0.9403 & 0.9703 & 0.9689 & 0.9699 \\ MSACN & **0.9857** & **0.9510** & **0.9761** & **0.9737** & **0.9748** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Various verification indicators of MSACN and various models.