# A Self-Supervised Detail-Sensitive ViT-Based Model for COVID-19 X-ray Image Diagnosis: SDViT
## 4 Experiment
### Results  
The performance of different model configurations is shown in Tables 6-9. The best performance in each column is marked in bold. To further compare the model effects and reduce bias for the data, we use k-Fold cross validation, with k = 7. Table 6 provides the accuracy of different model configurations on the X-ray Image dataset. The current SOTA, Attention-VGG-16, is assessed as well. We start with the ViT structure alone, which is used as a comparative classifier in our study. Compared to ResNet101, ViT obtains a similar, though slightly better, performance in accuracy of 88.0952% (ResNet101) and 88.6905% (ViT). When we add our proposed self-supervised learning strategy and Detail Correction Path (DCP) to the ViT structure, the classification effect of the model is substantially improved. The improvement is further extended when adding both, which fully demonstrates the effectiveness of our proposed method. The best performance is achieved after the domain transfer strategy is added, with an accuracy of 95.2381%. We call this model SDViT. The current SOTA model, Attention-VGG16, achieves higher accuracy than native ViT. With our proposed innovations added to native ViT one by one, however, the new models outperform the Attention-VGG16 model. SDViT achieves the best performance among all the models. When examining the sevenfold K-Fold accuracy, we observe similar patterns of improvement among the models. Tables 7 provide the same set of the accuracy of different model configurations on the synthetic-covid-cxr dataset. We continue to conduct sevenfold cross-validation. ViT has better performance in terms of accuracy (test set accuracy: 81.2667% and K-Fold accuracy: 81.3333%) compared to ResNet101 (test set accuracy: 80.4333% and K-Fold accuracy: 80.4733%). With either one of our proposed self-supervised learning strategy and Detail Correction Path (DCP) additions to the ViT structure, the classification effect of the model is substantially improved. The improvement is further extended when adding both, which fully demonstrates the effectiveness of our proposed method. The best performance is achieved with all three innovations added, that is, the proposed SDViT model. It achieves a test set accuracy of 86.9% and K-Fold accuracy of 86.93333%. The current SOTA model, Attention-VGG16, is outperformed by the models with our innovations added. Tables 8 and 9 list the statistical metrics (precision, recall, and F1-score) for the models. Table 8 shows the results on the X-ray Image dataset, while Table 9 shows the results on the synthetic-covid-cxr dataset. On both datasets, the SOTA model is outperformed by the models with our innovations added, and our proposed model, SDViT, achieves the best performance for all metrics. Comparing the number of parameters, the inference speed, and the required computation, we find that SDViT is able to improve the classification effect while greatly reducing both the computation and time complexity, as shown in Table 10.  
In addition, we build up confusion matrices for ViT, VGG-16, and SDViT on both the X-ray Image and synthetic-covid-cxr datasets. Tables 11-13 list the confusion matrices \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Attention** & **Self-Supervised Learning** & **DCP** & **Domain Transfer** & **Test Set Accuracy** & **K-Fold Accuracy** \\ \hline ResNet101(Baseline) & \(\times\) & \(\times\) & \(\times\) & \(\times\) & 88.0952\% & 87.0833\% \\ Attention-VGG (SOTA) & \(\times\) & \(\times\) & \(\times\) & 90.4761\% & 89.9404\% \\ ViT + Self-supervised learning & ✓ & ✓ & \(\times\) & 88.6905\% & 89.4048\% \\ ViT + DCP & ✓ & ✓ & ✓ & \(\times\) & 93.4524\% & 93.9286\% \\ DCP + Self-supervised learning + ViT & ✓ & ✓ & ✓ & \(\times\) & 94.0476\% & 95.3571\% \\ SDViT(Our Model) & ✓ & ✓ & ✓ & \(\times\) & 94.6428\% & 95.7143\% \\ SDViT(Our Model) & ✓ & ✓ & ✓ & ✓ & 95.2381\% & 96.13095\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accuracy of different model configurations on X-ray Image dataset.