## 4 Experiments
### Performance Metrics  
When comparing the two ViT models trained on the low-resolution leather dataset, it is evident that both models displayed similar training and validation accuracy ROC curves. However, by optimizing training efficiency with the 'early stopping' callback method and a patience of 10, there is a notable difference in their accuracy plots (shown in Figure 11). The second ViT model (ViT-02) achieves a significantly higher validation accuracy of 93.89% in just 68 epochs compared to the first (ViT-01), with an average validation accuracy of 89.86% after stopping at 75 epochs. This indicates that ViT-02 performed better in terms of validation accuracy, suggesting improved classification or prediction capability compared to ViT-01 within a shorter training duration. Figure 11: Receiver operating characteristic (ROC) curves for training accuracy versus validation accuracy: (**a**) ViT-01; (**b**) ViT-02. In Figure 12, the validation loss comparison between the two ViT network configurations indicates a similar result; however, ViT-02 reaches a lower average validation loss of 0.1846 after stopping at 68 epochs compared to ViT-01's average validation loss of 0.1970 at 75 epochs. This indicates that ViT-02 achieved an improved accuracy or generalization compared to ViT-01 within a shorter training duration. Also noted in each of these figures is that ViT-02 has a much steeper gradient and therefore reaches its optimal range more efficiently than ViT-01. Figure 13 depicts the heatmap confusion matrices for the categorical classification performance during training of each vision transformer. It is evident that although trained on a small number of low-resolution images, both models perform similarly, with a relatively high number of true positives (TP) in each category of the leather dataset. It can also be seen in Figure 14 that by comparing the average precision, recall, and F1-scores for each leather category, there is relatively little difference between the two models' training performances; however, ViT-02 performs equally well as or outperforms the original ViT model in all leather categories. Table 1 displays the average training metrics of each tested architecture trained on the Leather Defect detection and Classification dataset. From this comparative analysis, it is evident that the ResNet-50 model exhibited subpar performance and also takes the longest time to train, further emphasizing its inefficiency. Conversely, the EfficientNet-B0 and Inception-V3 models demonstrated superior results in terms of average validation accuracy and precision when compared to ViT-01. Between Inception-V3 and EfficientNet-B0, the former shows a slightly higher precision but a slightly lower recall than the latter. Inception-V3 not only has a higher AUC, indicating that it performs slightly better across various classification thresholds, but it also takes significantly less time to train than EfficientNet-B0, making it the more efficient model between the two. Of the five models in Table 1, ViT-02 demonstrated superior overall performance and generalization ability reached in almost half the training time of ViT-01, making it a more robust and effective model for the categorical classification of leather surface defects. Additionally, ViT-02 has the shortest training time (176.27 s), suggesting that it is the most computationally efficient model. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **Accuracy** & **Loss** & **Precision** & **Recall** & **F1-Score** & **AUC** & **Training Time** \\ \hline ResNet-50 & 0.3983 & 1.4732 & 0.6806 & 0.0169 & 0.0328 & 0.7790 & 769.69 s \\ Inception-V3 & 0.9152 & 0.2347 & 0.9248 & 0.8737 & 0.8984 & 0.9944 & 292.34 s \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Training time and average performance metrics for the ResNet-50, Inception-V3, EfficientNet-B0, ViT-01, and ViT-02 models trained on the low-resolution leather dataset.