# On-Edge Deployment of Vision Transformers for Medical Diagnostics Using the Kvasir-Capsule Dataset
## 4 Results and Discussion
### Model Evaluation
#### 4.1.3 Model Performance-to-Size Ratios  
To combine the results of the previous two subsections, we can now evaluate the models in terms of the ratio between their F1-scores and sizes, presented in Table 9. This will give us insight into how well a model performs despite the size reduction and allow us to choose which model would be most optimal for deployment on the mobile device. Higher F1-scores and lower model size provide a higher ratio, which is preferred in the context of this work. Figure 7 indicates that the model and quantization technique with the best ratio is the MobileViT_V2_050 model using F16 quantization. Upon further investigation, we can see that the model's size is only 2.89 MB (as opposed to the original 5.50 MB), which is an extremely small model given its function and complexity. This points us in the direction of model efficiency, given that the model can classify images despite using a significantly smaller memory allocation. It is quite quickly evident, however, that the model's F1-score of only 0.9033 makes it relatively unreliable, especially given the medical domain that our study delves into. \begin{table}
\begin{tabular}{l c c c} \hline \hline  & \multicolumn{3}{c}{**Quantization Technique**} \\ \cline{2-4}
**Model** & **F32** & **F16** & **QAT** \\ \hline EfficientFormerV2S2 & 1.72 & 3.41 & 1.83 \\ EfficientViT\_B0 & 12.52 & 24.90 & 7.91 \\ EfficientViT\_M4 & 2.72 & 5.39 & 9.84 \\ MobileViT\_V2\_050 & 16.43 & 31.30 & 41.25 \\ MobileViT\_V2\_100 & 4.95 & 9.76 & 16.42 \\ MobileViT\_V2\_175 & 1.74 & 3.47 & 6.58 \\ RepViT\_M11 & 2.89 & 5.75 & 11.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance-to-size ratio comparison for different ViT models and quantization techniques. Figure 7: Performance-to-size ratio for each ViT model. Models with an F1-score below 0.90 were omitted from the graph. One observation to be made is that the more complex the model is, the better it can classify between each of the nine classes extracted from the dataset. In particular, MobileViT_V2_175 is the best-performing model on average across all quantization types, with a peak performance of 0.9534 for the F16 quantized model. This happens to also be the biggest model of all F16 quantizations, further solidifying the fact that model size generally corresponds to performance in ViT-based classifiers. Regardless, we can also see that the F16 quantized model performs comparably to the F32 default model despite being around half the size. Given that the F16 quantized MobileViT_V2_175 is still only around 27 MB, this makes it a reasonable candidate for deployment on edge devices, although the inference time is expected to be higher given this size difference to other models. Given that there are currently no implementations of quantized ViT-based classifiers for the Kvasir-Capsule dataset in the literature, these results indicate a promising area of exploration pertaining to small-scaled transformer-based models for medical image classification. It should also be noted that the images used for almost all models were rescaled to be \(224\times 224\) pixels, although in the case of all versions of MobileViT_V2 they were set to \(256\times 256\) as per the implementation by Apple [31]. This means that images would have to be pre-processed in a different manner altogether should we implement it on a hand-held device, which can be performed by adding zero paddings to the images with a depth of 16 pixels. The degradation in performance across different quantization techniques can be credited to the reduced model complexity. However, QAT--through re-training--on average displays the lowest drop in performance. Since it is not a post-training quantization technique, we can better understand the impact of losing information in a model in terms of the weight data. In terms of QAT, MobileViT_V2_175 performed best with an F1-score of 0.9442 at 14.36 MB, and RepViT_M11 performed second best with an F1-score of 0.9311 at 8.46 MB.