# Detecting COVID-19 Effectively with Transformers and CNN-Based Deep Learning Mechanisms
## 4 Discussion  
[PERSON] and [PERSON]'s 2020 study [17] employed fine-tuning techniques on a modified version of ResNet50 architecture, resulting in the creation of Covid-ResNet, a model designed to identify COVID-19 from CT scans. Similarly, in 2021, [PERSON] and [PERSON] [18] utilized ResNet50 architecture with a chest X-ray dataset to construct a machine learning model capable of detecting COVID-19. Researchers have also investigated the use of EfficientNet, a highly successful convolutional neural network architecture, either independently or as part of an ensemble network, due to its flexibility and versatility in customization to meet specific research and practitioner needs [19]. Other researchers have utilized varying forms of VGG16 and VGG19 models, either independently or in an ensemble, to create automated COVID-19 detection systems [20; 21]. DenseNet, another deep neural network that employs a feed-forward model to address vanishing gradients, has also demonstrated exceptional performance in various studies [22; 23]. Using [PERSON] et al. [24] comprehensive analysis of a range of popular models suggested that the VGG-19 model achieved the highest accuracy and F1 score of 0.905. Additionally, we referenced [PERSON] et al. [16], where the models achieved an F1 score of 0.9423 and an accuracy of 0.9531. We evaluated several models for COVID-19 image classification, including VGG-19, DenseNet-121, Maftouni Ensemble with FC, and Maftouni Ensemble with FC+ SVM. We found that the performance of all the trained models met or exceeded our expectations, demonstrating the effectiveness of our curated dataset for model training.  
Based on the suggestion that CNN-based models may struggle with the long-range relationship problem resulting from regional locality in convolution operations [25], we developed a model using Vision Transformer, an image classification model inspired by NLP transformers, which has been pre-trained on large datasets to address this issue. Several studies have indicated that transformers show promise as an alternative to traditional CNN-based neural networks [26; 27; 28; 29; 30]. Our model produced impressive results, achieving an F1 score of over 0.96 for the one-step classifier and over 0.97 for the two-step classifier. These findings demonstrate the high accuracy of our developed model for COVID-19 image classification and support the use of artificial intelligence in CT scans for COVID-19 detection.  
As a future task, we suggest implementing multitask learning to identify not only whether a person has COVID-19, but also other potential issues. Multitask learning is a promising approach that allows for the simultaneous detection of multiple diseases or medical conditions. By leveraging the vast amounts of medical data available, AI can help clinicians make more comprehensive and accurate diagnoses, ultimately improving patient outcomes. Our study provides important insights into the potential of AI in healthcare and demonstrates the effectiveness of our model design for COVID-19 image classification. The use of AI in medical diagnosis has the potential to revolutionize healthcare, and we believe that our findings could inspire further research and development in this field. With the ongoing COVID-19 pandemic and the increasing demand for accurate and efficient medical diagnosis, the development and implementation of AI in healthcare is more important than ever.