## 2 Related Work  
### Swin Transformer  
The Transformer [10] was first proposed in the field of natural language processing (NLP) and was published by Google in Computation and Language. The emergence of the Transformer solves problems in the NLP field, such as the inability to be parallelized, limited memory length, etc., previously faced by RNN, LSTM, and other networks. Inspired by the success of the Transformer in the field of NLP, ref. [5] proposed the Vision Transformer, which applies the standard Transformer model to the field of vision with minimal modifications. While keeping the core structure of the Transformer model unchanged, necessary adjustments were made to adapt it to image-processing tasks. The images are divided into patch blocks, which are then used as input token sequences for the model. Through the Vision Transformer's unique image patch processing, token sequence input, self-attention mechanism, and global information modeling capabilities, it effectively addresses the shortcomings of previous CNNs in capturing global semantic information. Therefore, the Vision Transformer has been highly sought after by researchers in the vision field since it came out. For example, in reference [11], researchers utilized the powerful feature-extraction and sequence-modeling capabilities of the Transformer architecture to capture the spatial-temporal relationships between multi-view images, proposing a 3D human pose estimation method based on a Transformer that incorporates multi-view spatial-temporal relationships. However, as research has progressed, researchers have found that, limited by hardware conditions, the Vision Transformer has been shown to have a poor application impact in the field of image segmentation and target detection. Then, in 2021, Microsoft Research published a paper on the Swin Transformer [12] in ICCV, which performed well on multiple visual tasks once published. Compared to the Vision Transformer, the Swin Transformer adopts a hierarchical structure, where the size of feature maps decreases gradually as the network depth increases while increasing the level of feature abstraction. This design is akin to the downsampling operation in convolutional neural networks (CNNs), but the Swin Transformer achieves this process through the self-attention mechanism. Given that computing global self-attention is very computationally intensive, the Swin Transformer uses local windows, and within each window, the Swin Transformer employs the self-attention mechanism to model relationships between pixels. This mechanism allows the model to reference other pixels within the window when processing a pixel, capturing global contextual information. Another key feature of the Swin Transformer is the use of Shifted Windows for computing self-attention, which confines the self-attention computation within non-overlapping local windows, thus improving computational efficiency. Additionally, by moving windows within consecutive blocks, the model can transmit information between adjacent windows, maintaining the communication of global information and effectively alleviating the computational burden and hardware limitations that arise when using the Vision Transformer for image segmentation. However, with the training and application of large-scale visual tasks, the Swin Transformer suffers from problems, such as training instability and a resolution gap between pre-training and fine-tuning. In [13], the Swin Transformer V2 model is proposed, which effectively solves the problem of training instability by using the scaled cosine attention module.