# Swin Transformer Assisted Prior Attention Network for Medical Image Segmentation
###### Abstract  
Transformer complements convolutional neural network (CNN) has achieved better performance than improved CNN-based methods. Specially, Transformer is utilized to be combined with U-shaped structure, skip-connections, encoder, and even them all together. However, the intermediate supervision network based on the coarse-to-fine strategy has not been combined with Transformer to improve the generalization of CNN-based methods. In this paper, we propose Swin-PANet, which is applying a window-based self-attention mechanism by Swin Transformer in the intermediate supervision network, called prior attention network. A new enhanced attention block based on CCA is also proposed to aggregate the features from skip-connections and prior attention network, and further refine details of boundaries. Swin-PANet can address the dilemma that traditional Transformer network has poor interpretability in the process of attention calculation and Swin-PANet can insert its attention predictions into prior attention network for intermediate supervision learning which is humanly interpretable and controllable. Hence, the intermediate supervision network assisted by Swin Transformer provides better attention learning and interpretability in network for accurate and automatic medical image segmentation. The experimental results evaluate the effectiveness of Swin-PANet which outperforms state-of-the-art methods in some famous medical segmentation tasks including cell and skin lesion segmentation. prior attention network; medical image segmentation; Swin Transformer; convolutional neural networks +
Footnote †: journal: Applied  
for more accurate and effective segmentation models. Hence, accurate medical image segmentation is of great significance for computer-aided diagnosis and image-guided clinical surgery [1; 2; 3]. Convolutional neural networks based on U-shaped topology are widely popular in medical image segmentation. However, despite great breakthroughs in this field, CNNs-based methods mostly demonstrate limitations in capturing features to model global dependency, caused by the inner locality of convolution operation. Attention mechanism is a useful solution to settle the above problem and it is inspired by human's visual cognition and perception. Recently, there is an excellent work called prior attention network [4] which proposes a novel structure consisting of U-shaped convolutional neural network and an intermediate supervision network between encoder and decoder. It follows the coarse-to-fine strategy and intermediate supervision strategy. Particularly, they propose an attention-based method called attention guiding decoder in intermediate supervision network, and it takes the rich information of multi-scale features from the encoder to generate spatial attention maps for guiding the final segmentation of decoder in CNNs. Further, the process of traditional attention learning is often not humanly interpretable, and the regions focused by the network are usually different from the regions that we pay attention to. In prior attention network, the intermediate supervision learning that guides the next step of segmentation can provide the interpretability of the network in a certain manner. However, this non-local attention mechanism has a poor capability for aggregating multi-scale features from different modules and extracting boundary information. Different from the non-local attention-based methods, Transformer has been proposed as an alternative method for modeling long-range dependency by its self-attention mechanism. The current research direction of Transformer on medical image segmentation are pure Transformer network and hybrid Transformer network. An interesting work called UCTransNet [5] investigates the potential limitation and semantic gap of the skip-connection between the encoder and decoder in U-Net, and proposes a novel design called channel-wise Transformer module to replace the skip-connection for better fusing multi-scale features from encoder and reducing the semantic gap.