# A Self-Supervised Detail-Sensitive ViT-Based Model for COVID-19 X-ray Image Diagnosis: SDViT
## 2 Related Work
\begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**Model** & **Architecture** & **Contribution** & **Advances** & **Disadvantages** \\ \hline \multirow{3}{*}{SETR} & Semantic segmentation with transformers & Using semantic segmentation as a sequence-to-sequence predictor task-using a given transformer as another module, using the object network at each layer and designing three different decoders for pixel-level segmentation. & Can learn superior feature representations compared to different FCNs; removes the reliance on FCNs and addresses the challenge of a limited receptive field. & Requires huge (Q0M+) parameters and computation; does not work well for small objects; increasing the input image reduces the path size, increasing computation quadratically. \\ \hline \multirow{3}{*}{DETR} & Object detection with transformers & Propose a deformable Attention Module to replace the self-attention module in the encoder, greatly reducing computations. & Can be extended to complex tasks easily; better than fast R-CNN on large objects; also not need for NAS post processing; shoes for anchor. \\ \hline \multirow{3}{*}{VIT} & Transformers for image recognition & The first vision model to use Transformer. & The MSA (Multihead Self-Attention) block complements the pretrained spatial smoothing of convolutional networks. MSA integrates feature maps and smoothing loss planes, which highly neural networks learn more robust feature representations. \\ \hline \multirow{3}{*}{DefT} & A VIT model with a teacher student strategy and distillation token & Better performance than Transformer was achieved by using Transformer to build a competitive convolutions-mode model to solve dilation to outperform traditional distillation, and using current as a Teacher. & To achieve the same effect as VIT, DET requires a smaller training dataset and a shorter training period. & Consistent with VTS issues. \\ \hline \multirow{3}{*}{Attention VGG-16} & An attention-based deep learning model for image classification & A branch is added to the convolutional neural network with a spatial attention module to better obtain the associations and interactions of different spatial regions and to achieve the best model for the current domain. & It compares the spatial relationship between ROIs in CXR images, and is efficient in both classification accuracy and training parameter. & It does not deal with information such as edges and other details, and does not consider the difference in distribution between the training and test sets, which can lead to a certain degree of degradation in generalization performance. It does not consider the use of self-supervision to further improve accuracy. \\ \hline \multirow{3}{*}{CovidNet} & A deep learning-based approach for automatic diagnosis of COVID-19 using chest radiographs & Chest radiographs are used for early and accurate diagnosis of COVID-19 through a model with semantic labels of COVID-19 using chest radiographs & Automatic detection and early diagnosis of COVID-19 while reducing diagnosis workload, reducing the workload of radiologists, and avoidance from human baseline models. & Advantage over other models are unclear, though these can be obtained through comparative study. \\ \hline \multirow{3}{*}{SDMIT} & A Self-supervised Dealal-sensitive VIT-Based Model for Medical Diagnosis & A self-supervised strategy that allows the model to repair ambiguous and defective parts using information from other parts are active, the Dealal Correction Path, to repair the possible loss of applied key features caused by MagNet & A shorter training period, stronger resistance to noise, and missing interference on a small number of sample datasets & Same as VIT \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of our proposed model with previous models.