# Automatic Medical Image Segmentation with Vision Transformer
## 1 Introduction  
Medical image segmentation using automated techniques is vital for many clinical tasks, including discovering new biomarkers, monitoring disease progression, and facilitating computer-assisted determination and treatment direction [1], particularly in the labeling of lesions or infected areas during public health crises. The recent advances in deep neural network structures have led to major enhancements in the performance of medical image segmentation models. However, medical image segmentation tasks exhibit immense diversity, as different imaging modalities produce images with distinct characteristics and appearances while the presentation of diseases also varies considerably. This heterogeneity makes the direct application of even a successful architecture such as U-Net [2], a convolutional network for biomedical image segmentation, to a new task unlikely to achieve optimal results. Since the manual annotation of infected areas is alaborious and time-consuming task, and radiologists' annotations are highly subjective and prone to personal bias informed by past clinical encounters, it is an urgent requirement to explore automated and robust segmentation techniques for medical images [3]. Segmenting organs and lesions in computed tomography (CT) slides provides critical clinical data for disease identification and quantification [4]. Recently, many algorithms using feature extraction and classification techniques have been developed for lung nodule segmentation, achieving promising performance. Nevertheless, visual similarities between nodules and the background make isolating them challenging. To address these challenges, researchers have developed deep learning approaches to learn more discriminative visual representations for segmentation [5; 6; 7; 8; 9]. For instance, from heterogeneous CT slices, the authors of [5] introduced a centrally focused convolutional neural network (CNN) to isolate lung nodules. Meanwhile, in generative adversarial networks, GAN-synthesized information is utilized to enhance the discriminative training model for lung pathology segmentation. A dual deep network that incorporates multiple residual streams at varying resolutions [9] was designed to isolate lung tumors in CT images [8]. In summary, deep learning shows promise for improving the segmentation of lung nodules and pathology markers in CT scans via learning robust features to overcome the difficulty of differentiating nodules from image backgrounds. Artificial intelligence technologies have been widely applied to combat COVID-19 (coronavirus disease 2019), with many deep learning systems having been proposed for detecting infected patients through radiological imaging. However, labeled data are still limited. In this context, semi-supervised models can identify target regions from other anomalies given minimal labeling, making them well-suited for COVID-19 assessments. Additionally, transformer learning methods constitute another promising approach for situations with limited data [10; 11]. Transformers excel at modeling global contexts and, with large-scale pre-training, demonstrate superior transfer learning ability on downstream tasks. This has been evidenced by their success in natural language processing (NLP) and machine translation tasks [12; 13]. Recently, transformers have achieved or surpassed cutting-edge performance on various image recognition challenges [6; 14]. In summary, semi-supervised learning and transformers are two effective artificial intelligence (AI) techniques which can address the lack of labeled medical imaging data, capitalizing on their abilities to learn from limited supervision and leverage pre-training. Their adoption could enable accurate screening from radiological images despite data constraints. Detecting infected regions and affected areas in medical images remains challenging due to the variance in morphological characteristics like texture, size, and positioning across different medical image modalities.