## 3 Materials and Methods
### Transfer Learning (TL)
#### 3.3.4 Evaluation Metrics  
The comparison of the ImageNet models and Google ViT was based on two types of learning curves: optimization and performance.  
Optimization curves are a type of learning curve calculated based on the metric by which the model's parameters are being optimized, such as loss or mean squared error (MSE).  
In this work, categorical cross-entropy [23] was used as a loss function. Cross-entropy loss is used when adjusting model weights during training aiming to minimize the loss. This means that the smaller the loss, the better the model. A perfect model has a cross-entropy loss of zero. Cross-entropy [23] is defined in Equation (1) as follows:  
\[L_{CE}=\sum_{i=0}^{n}t_{i}\log(p_{i}) \tag{1}\]  
where \(t_{i}\) is the truth label and \(p_{i}\) is the Softmax probability for the ith class. Moreover, there are also two essential correlated terms associated with optimization curves: variance and overfitting.  
Variance [24; 25; 26] is the difference in fit between the training and validating datasets. A high variance typically occurs when the model is too complex and does not reflect the simpler real patterns existing in the data. Variance is calculated using Equation (2):  
\[\sigma^{2}=\frac{1}{N}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2} \tag{2}\]  
where \(x_{i}\) equals each value in the dataset, \(\bar{x}\) is the mean of all values in the dataset, and \(N\) is the number of values in the dataset.  
The training loss (_TL_) indicates how well the model fits the training data, while the validation loss (_VL_) indicates how well the model fits the new data. Variance is correlated with the loss difference (_LD_), which is the difference between _VL_ and _TL_. The _LD_ is calculated using the Equation (3):  
\[LD=\textit{VL}-\textit{TL} \tag{3}\]  
Figure 7[27] explains the underfitting and overfitting problems in relation training and validation losses. When the deep learning algorithm effectively captures the training data but performs poorly on the new data, it is unable to generalize, and this is known as overfitting. The greater the variance of a model, the more it overfits the training data.  
As for performance learning, accuracy represents the ratio of true predicted classes to the total number of samples evaluated [28]. Equation (4) [28] demonstrates this computational process:  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{4}\]  
where _TN_ and _TP_ account for successfully classified negative and positive cases, respectively. Additionally, _FN_ and _FP_ report the number of misclassified positive and negative cases, respectively.  
Another parameter used for performance assessment was the accuracy difference (\(AD\)) (5), i.e., the difference between the training accuracy (\(TA\)) and the validating accuracy (\(VA\)):  
\[AD=VA-TA \tag{5}\]  
Apart from the previously mentioned metrics for model evaluation, another valuable tool employed for enhancing the interpretability of the models was Score-CAM [29]. Score-CAM is a visual explanation technique that utilizes class activation mapping (CAM) to assign scores to different regions in convolutional neural network (CNN) models, facilitating a deeper understanding of their internal workings.