# Attention-Based Two-Branch Hybrid Fusion Network for Medical Image Segmentation
## 3 Method
### Attentional Feature Fusion Module  
We introduce an innovative Attention feature fusion (AtFF) module, illustrated in Figure 4, which amalgamates an attention mechanism with a multimodal fusion mechanism to effectively blend the hierarchical features from CNN and Transformer. To counteract the spatial information loss due to downsampling, we employ a dual-branch skip link (depicted in red) to integrate the high-level global context with the low-level precise details from both global and local encoders. The AtFF module is linked to the preceding fusion module through an additive process, enabling the fusion of local and global features at the same stage, as observed in Figure 4. Therefore, the skip connection module (SCM) process can be expressed as Equations (5)-(8):  
\[\hat{t}_{i}=Channel\,Attention(t_{i}),i=1,2,3,4 \tag{5}\]  
\[\hat{r}_{4}=Spatial\,Attention(r_{4}) \tag{6}\]  
\[\hat{r}_{i}=GAU(r_{i},\hat{r}_{i+1}),i=1,2,3 \tag{7}\]  
\[l_{scm}=SCM(\hat{t}_{i}+\hat{r}_{i},l_{i+1}) \tag{8}\]  
where GAU stands for the Global Attention Up-Sampling module calculation and ChannelAttention and SpatialAttention, respectively, indicate the Channel Attention and Spatial Attention computations. \(t_{i}\) and \(r_{i}\) are the features in the Transformer branch and CNN branch in stage \(i\), respectively, and \(l_{i}\) is the output of the AtFF module. In order to improve the feature representation capability, Transformer branch features use channel attention to determine which global key information on the feature graph contains important information. Channels carrying less crucial information receive diminished focus. The SE-Block is employed to enact channel attention, thereby enhancing the dissemination of global information from the Transformer branch. SE-Block is composed of two components, namely Squeeze and Excitation. After collecting several feature graphs (t) by a global average pooling operation, the Squeeze operation compresses each feature graph, resulting in a \(1\times 1\times\) C sequence of real values from the many feature maps. The specific calculation is shown in Equation (9):  
\[z=F_{squeeze}(t_{i})=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}t_{i}(i,j) \tag{9}\]  
Figure 4: The diagram of Attention Feature Fusion Module (AtFF). To accurately represent channel dependencies, the subsequent Excitation operation is essential to utilize the insights obtained from the Squeeze stage. We employ a straightforward gating method with sigmoid activation in our work. The specific computational representations are as in Equations (10) and (11):  
\[s=F_{excitation}(z,W)=\sigma(W_{2}\cdot\delta(W_{1}\cdot z)) \tag{10}\]  
\[\hat{t}_{i}=F_{scale}(t_{i},s)=t_{i}s \tag{11}\]  
where \(\sigma\) represents the sigmoid activation function, \(\delta\) represents the relu activation function, \(W_{1}\in\mathbb{R}^{\hat{\tau}\times c}\), \(W_{2}\in\mathbb{R}^{c\times\hat{\tau}}\). \(F_{scale}\) is the product over channels. Initially, the CBAM (Convolutional Block Attention Module) block serves as a spatial filter for the convolutional neural network (CNN) branch's most reduced-scale features. It enhances local details and suppresses irrelevant regions to address the issue of noisy low-level CNN features. The feature maps are transformed from C \(\times\) H \(\times\) W to C \(\times\) 1 \(\times\) 1 by passing them through two parallel MaxPool and AvgPool layers. Subsequently, the MLP module generates channel attention maps, and the ReLU activation function extracts two activated results. Following an element-wise summation of the two outputs, a sigmoid activation function is applied to derive the Channel Attention output. This output is then subjected to Maximum Pooling and Average Pooling operations, resulting in two 1 \(\times\) H \(\times\) W feature maps. These maps are separated using the Concat operation, converted into 1-channel feature maps via 7 \(\times\) 7 convolution, and ultimately, a sigmoid function is used to obtain the Spatial Attention feature map.