# Attention-Based Two-Branch Hybrid Fusion Network for Medical Image Segmentation
## 2 Related Work
### CNN and Transformer  
Convolutional Neural Networks (CNNs) are a cornerstone of deep learning, particularly adept at processing grid-structured data like images. The architecture comprises various layers: convolutional, activation, pooling, and fully connected. In the convolutional layer, an array of trainable filters extract features from the input by scanning it, thus reducing dimensionality. The activation layer introduces nonlinearity, allowing the network to learn complex patterns. The pooling layer downsamples the data, aiding in computational efficiency and preventing overfitting. The fully connected layer connects every neuron in one layer to every neuron in another, mapping the abstracted features to output classes. CNNs are efficient due to weight sharing and local receptive fields, which minimize the number of parameters. Their multi-layered design captures the hierarchical nature of data, making them pivotal in applications such as medical image segmentation. Notably, architectures like the U-Net encoder-decoder [7] and its offshoots have demonstrated superior performance in this domain. The UNet++ [8] and UNet 3+ [14] networks proposed by [PERSON] and [PERSON] et al. design a series of nested and dense skip paths to reduce semantic gaps. A unique Attention Gate (AG) mechanism model, proposed by Attention U-Net [10], focuses on targets with varying sizes and forms. In order to enhance retinal blood vessel segmentation performance, ResUNet [9] incorporates a weighted attention method. DenseUNet [11] substitutes dense connections for skip connections in U-Net. The R2U-Net [12] optimizes feature representation by combining the benefits of residual networks. KiU-Net [13] presents a unique design that makes use of incomplete and overcomplete features to enhance the segmentation of tiny anatomical components. UNeXt [24] presents a merger of MLP with UNet, substantially cutting the parameter count without compromising segmentation efficacy. These techniques are all still grounded in CNN. An increasing number of transformer-based techniques are showing up in CV tasks, spurred by Transformer's [19] success in a variety of NLP tasks. Central to Transformer's operation is the concept of Self-Attention, a process that enables the model to dynamically weigh the significance of various elements in a sequence relative to one another. This approach enables parallel processing of the full input sequence, circumventing the issue of long-term dependencies that traditional Recurrent Neural Networks (RNNs) struggle with and their inability to process information concurrently. The Transformer architecture also encompasses positional encoding to maintain the sequential order of data, a multi-head attention setup to enhance the model's representation capabilities, and incorporates both a feed-forward network and layer normalization to refine the feature extraction process. With its versatile and modular design, Transformers have not only revolutionized tasks such as language modeling and machine translation but have also been widely adopted in image recognition, generative modeling, and other areas, thus expanding the scope of research and application possibilities. ViT [20] is among the newer vision transformers and initially showed that Transformer-exclusive architectures can hit state-of-the-art performance in image recognition when pre-trained on extensive datasets. To train on the smaller ImageNet-1K dataset, DeiT [25] proposes data-efficient training techniques and knowledge distillation. Swin Transformer [22] with a hierarchical structure reduces the computational complexity and achieves SOTA performance attains state-of-the-art results in a wide range of tasks by the proposed shift-window based self-attention. Meanwhile, SETR [21] treats semantic segmentation as a sequence prediction problem by employing the Transformer as an encoding component. Although the Transformer excels in global context modeling, it struggles with capturing fine-grained details, particularly in medical imaging.