# On-Edge Deployment of Vision Transformers for Medical Diagnostics Using the Kvasir-Capsule Dataset
## 4 Results and Discussion
### On-Edge Testing  
Given the metrics presented here, we will be selecting the following two quantized models to analyze further through deploying on a CPU: MobileViT_V2_175 with F16 quantization (selected due to having the highest F1-score across all models and quantization types) and MobileViT_V2_050 with F16 quantization (selected due to having the best  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Paper** & **Model** & **F1-Score** & **Params** & **Quantization** \\ \hline
[16], 2023 & ViT & 0.9700 & \(\approx\)86 M & None \\
[18], 2024 & ViTCA-Net & 0.9700 & \(\approx\)86 M & None \\
[19], 2024 & TNCB & 0.9500 & 12.2 M & None \\ Proposed solution & MobileViT\_V2\_175 & 0.9534 & 14.3 M & F16 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Comparison of our proposed model with previous works.  
performance-to-size ratio with an F1-score above 0.90), as illustrated in Table 11. Once again, the models are tested on a CPU to mimic their performance on an on-edge device without GPU capabilities. Experiments were carried out on a CPU instead of an edge device due to the absence of Flutter packages compatible with vision transformers.  
The two selected models were then tested using the CPU resources available on Google Colab, which uses an Intel(R) Xeon(R) CPU @ 2.20 GHz (Intel, Santa Clara, CA, USA), to mimic how they would perform on edge devices. Furthermore, mobile devices such as the Samsung Galaxy S24 use a Qualcomm Snapdragon 8 Gen. 3 processor (Samsung, Suwon-si, Republic of Korea), which is loosely comparable to the performance level of an Intel Core i7 processor, justifying why the models were tested on a CPU rather than a mobile device [11]. We will be considering the F1-score, average inference time per image, and the confusion matrix, giving us insight into how the model performs when considering the speed at which it predicts classes and how accurately it does so.  
The models were tested on the same 45 images from the testing subset of Kvasir-Capsule, with 5 images taken from each class. F1-scores and average inference time per image were also generated for both models. Looking at Table 12, a significant difference in performance can be observed between the two F1-scores and inference times. The differences in performance can be better visualized in the confusion matrices generated in Figure 8, where MobileViT_V2_175 does a much better job at producing reliable results. MobileViT_V2_050, however, is able run inference that is around 9 times faster than MobileViT_V2_175. Considering the domain that this paper is tackling, accurate results would outweigh model size and latency. MobileViT_V2_175 is able to achieve that while maintaining a reasonable model size and inference time that would not be detrimental to the target edge device.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Quantization** & **F1-Score** & **Size (MB)** & **Performance-to-Size Ratio** \\ \hline MobileViT\_V2\_175 & F16 & 0.9534 & 27.47 & 3.47 \\ MobileViT\_V2\_050 & F16 & 0.9033 & 2.89 & 31.30 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Chosen models to test on a CPU.  
Figure 8: Confusion matrices generated from running inference on a CPU.