# Automatic Medical Image Segmentation with Vision Transformer
## 1 Introduction
Additional difficulties arise from subtle visual differences between normal and infected tissues, as well as low-intensity contrast. Moreover, collecting extensive labeled infection data is arduous, as acquiring precise pixel-level annotations to train deep models is expensive and time-consuming [15]. Infection detection in medical images is difficult due to infections' heterogeneous appearances, their visual similarity to normal tissue, and the lack of abundant annotated data. Overcoming these obstacles is key to improving automated infection detection from medical images. Here, we develop a novel medical segmentation network that establishes self-attention mechanisms for sequence-to-sequence prediction. In order to compensate for transformers' need to sacrifice fine-grained feature resolution, we employ a hybrid CNN-Transformer structure. This combines the advantages of high-resolution spatial details from CNN representations with the overall context modeling capacities of transformers. Transformers are better at modeling long-range dependencies in images, more flexible at handling varying input sizes, and they provide greater model interpretability, offering a promising alternative to CNNs for medical image segmentation. We integrate transformers in medical segmentation, providing both locally precision segmentation and global contextual modeling for accurate analysis. Inspired by parallel structure designs, the self-attentive features from transformers are up-sampled and combined with CNN features of high-resolution skipped from the encoding path. This fusion strategy enables the precise delineation of boundaries while preserving the transformers' self-attention mechanism advantages for medical segmentation. Empirical experimental results suggest our transformer framework better leverages self-attention compared to existing self-attention CNN-based models. Simultaneously, incorporating more intensive low-level feature details generally improves segmentation accuracy. Extensive experiments across various medical image modalities demonstrate our method superiority over compared approaches. Our parallel integration of transformer global context modeling with CNN local detail extraction, coupled with multi-level feature fusion, allows transformers to significantly enhance the performance of medical image segmentation tasks. In summary, our framework seamlessly combines the strengths of transformers and CNNs, facilitating accurate and robust segmentation of medical images by effectively capturing both global and local contextual information. In summary, our main contributions in this work are as follows:  
* We develop the attempt to investigate the integration of transformer architectures into general medical image segmentation tasks, demonstrating their applicability beyond NLP. * We introduce a novel transformer-based network tailored for infection segmentation, designed to mitigate the challenges posed by limited labeled data availability. Our system exhibits superior learning capabilities through random propagation and self-supervised pre-training on unlabeled data. * Our proposed framework achieves high-performance segmentation results in a self-supervision manner, highlighting the significant potential of pre-training transformers on large unlabeled medical image datasets to learn rich representations that can be effectively transferred to the segmentation task. The remainder of this paper is organized as follows. Section 2 provides a comprehensive review of related work, including existing segmentation methods and relevant techniques. Section 3 details our conceptual framework and the methodologies employed in our automatic segmentation network. Section 4 reports the extensive experimental results and provides in-depth analysis. Finally, Section 5 concludes and discusses future research directions.