# Swin Transformer Assisted Prior Attention Network for Medical Image Segmentation
## 3 Modeling, Methods, and Design
### Hybrid Transformer Network  
To better perform the dual supervision strategy and fuse multi-scale features of inconsistent semantics between prior attention network and hybrid Transformer network, enhanced attention block is proposed for guiding the information filtration and cross attention of the Transformer attention features along channel-wise. Enhanced attention block is equipped in decoder layer of the hybrid Transformer network, which is to utilize a feature fusion between global and local contexts along channel-wise for achieving better performance of attention learning. In the enhanced attention block, compared with the traditional CAA module, the difference is that it receives multiple features from previous level of enhanced attention block, corresponding with skip-connection between encoder and decoder, prior attention network assisted by Swin Transformer. As shown in Figure 4, we take the \(i_{th}\) level output of enhanced attention block \(D_{i}\in\mathbb{R}^{C\times H\times W}\), the attention prediction \(M\), and the \(i_{th}\) level output of skip-connections \(\hat{O}_{i}\) as the input of next level enhanced attention block. \(C\), \(H\), \(W\) denotes the number of channels, and the height and width of features, respectively. Compared with traditional residual blocks, we set a learnable parameter \(\alpha_{i},i\in(1,4)\) in the residual paths, which can be updated along with the back propagation, and plays a role in retaining effective features and adding non-linearity to the process of integrating and refining the features. On account of the channel-axis attention of the hybrid Transformer network, \(\alpha_{i},i\in(1,4)\) can activate the effective channels and restrain the useless channels so as to increase the convergence speed of network while ensuring segmentation performance. The refined feature map \(R_{i},i\in(1,4)\) after the residual module is computed as follows:  
\[R_{i}=\alpha_{i}\cdot\hat{O}_{i}+M\cdot\hat{O}_{i},i\in(1,4) \tag{10}\]  
The operation of spatial squeeze is performed through global average pooling (GAP) [5] which produces a vector \(\kappa(X)\in\mathbb{R}^{C\times 1\times 1}\), and the \(l_{th}\) channel of vector can be formulated as:  
\[\kappa(X)=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}X^{k}(i,j) \tag{11}\]  
Through this operation, we can embed global spatial information to generate the attention map:  
\[ \begin{array}{ll}\hat{M}&=L_{1}\cdot\kappa(R_{i})+L_{2}\cdot\kappa(D_{i})\\ &=L_{1}\cdot\kappa(\alpha_{i}\cdot\hat{O}_{i}+M\cdot\hat{O}_{i})+L_{2}\cdot \kappa(D_{i})\end{array}  \tag{12}\]  
where \(L_{1}\) and \(L_{2}\) represent the weight matrices of linear layers and the ReLU layers. The operation in Equation (12) plays a role of encoding the dependency in channel dimension. Following ECA-Net [37], which experimentally exhibits the importance of avoiding dimensionality reduction for better learning attention among patches. Hence, there is only one Linear layer and Sigmoid function before the element-wise multiplication. The output \(D_{i+1}\) can be formulated as:  
\[D_{i+1}=\sigma(\hat{M})\cdot\hat{O}_{i} \tag{13}\]  
where the \(\sigma(\cdot)\) represents the function of activation, and the result \(\sigma(\hat{M})\) calculates the attention extent of each channel. Whereafter, the predicted mask \(D_{i+1}\) will be fed into next enhanced attention block with the attention prediction and next level output of multi-head cross fusion Transformer. After the first step of attention learning in Swin Transformer and attention guiding decoder, enhanced attention block receives the attention prediction and fuses it with other multiple features. It can acquire semantic-rich features and recover the origin image information lost by the attention calculation. On the other hand, the second step of attention learning in enhanced attention block guides the information filtration and cross attention of the attention features along channel-wise, which enhance the ability of extracting global context features and modeling long-range dependency along channel-wise.