# On-Edge Deployment of Vision Transformers for Medical Diagnostics Using the Kvasir-Capsule Dataset
## 2 Literature Review
### On-Edge Deployment for Medical Diagnostics  
When exploring the medical field, we can look at the advancements and incorporations of lightweight models (such as those developed with TFLite) to the Internet of Medical Things (IoMT)--an emerging field with significant benefits both in terms of inference efficiency and diagnosis [20]. In [21], [PERSON] and colleagues worked on developing a lightweight TFLite-based model for auto-classifying histopathological images [22]. The authors use post-training quantization on their model, ReducedFireNet, which compresses the weights of the network after it has been trained. ReducedFireNet has been designed for implementation on IoMT devices, posing several advantages compared to larger models. The key benefits are the high F1-score (reporting an average F1-score of 0.9680) and the model's significantly smaller size. Comprising only around 21,000 parameters, the model comes in at a size of 0.391 MB, highlighting its versatile deployability. [PERSON] et al. [23] worked on detecting COVID-19 from chest X-ray images [24]. Deploying their model on an ARM Mali GPU and developing an Android application to supplement their propositions, the authors introduced the DDSM++ model, inspired by the DenseNet121 architecture. Their TFLite-quantized models, F16, were reported to have a 30% reduction in inference time post-quantization, whilst their accuracies when trained on a COVID-19 X-ray image dataset dropped by only around 5%. The accuracy of the full model, F632, was reported as  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Paper** & **Images Used** & **Best Model** & **Performance** & **Model Size (MB)** \\ \hline [PERSON] et al. [16] & Top 9 classes: 4500 & ViT & F1-score: 0.97 & Not reported \\ [PERSON] et al. [18] & 12 classes: 6990 & ViTCA-Net & F1-score: 0.97 & Not reported \\ [PERSON] et al. [19] & Top 10 classes: 47,002 & TNCB & F1-score: 0.95 & Not reported \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of existing work on Kvasir-Capsule.  
0.9847, with FG16 clocking in at 0.93556. Despite the drop, it is evident that the model still performs comparatively well in post-quantization.  
More recently, [PERSON] et al. [25] proposed a mobile application called \"LungVision\", aiming to introduce the classification of respiratory diseases to on-edge applications using a publicly available dataset of lung X-rays [26]. The authors experimented with six different neural networks and four different quantization techniques. They showed how QAT is the best optimization approach, with an average of 75.59% reduction in model size. Integer quantization, however, improved inference time by 1.4 s more than other quantization methods. For their mobile application, the authors chose to deploy EfficientNetV2B2, trained using QAT, as it outperformed all other experiments with an F1-score of 97.51% in classifying respiratory diseases and a model size of 9.9 MB. These results are summarized in Table 2.  
The previous few papers work towards displaying the merit of using TFLite for image classification in the medical domain. We can see the benefits of these lighter-weight models, which can perform comparably to full-scaled models. Since no research has been conducted specifically on our proposed datasets, this leaves us with an avenue for conducting further research.