## 4 Results
### Result Analysis  
The evaluation process employs various evaluation metrics, including accuracy, precision, recall, F1 score, and the Matthews correlation coefficient (MCC). Among these metrics, MCC is the most informative one since it accommodates all the coordinates [63]. The metrics can be calculated using the following equations. \[Accuracy=\frac{TP+TN}{TP+PP+TN+FN} \tag{12}\]  
\[Precision=\frac{TP}{TP+FP} \tag{13}\]  
\[Recall=\frac{TP}{TP+FN} \tag{14}\]  
\[F1-score=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{15}\]  
\[MCC=\frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}} \tag{16}\]  
For further evaluation, we present the confusion matrix and the receiver operating characteristic (ROC) curve. Various models have been tested for the teacher model. Table 4 presents a short description of the teacher models. While the proposed teacher model has over 254 million parameters, its only responsibility is to train the student model. Given that the student model is intended for use in real-world applications, it is essential that its architecture be lightweight, emphasizing efficiency and practicality in operational contexts. Tables 5 and 6 present a comparison of different teacher models on the APTOS and IDRiD datasets, respectively. Figures 9 and 10 present graphical overviews of the comparison on the APTOS and IDRiD datasets, respectively. Among the eight models, the proposed fusion model based on Swin Transformer and ResNet152V2 performs the best. Traditionally, fusion models perform better than single-model classifiers. Moreover, the suggested teacher model incorporates two different feature extractors (CNN and transformer), which have totally different methods for feature extraction. Therefore, the extracted features only have \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Hyperparameter** & **Value** & **Accuracy on APTOS** & **Accuracy on IDRiD** \\ \hline \multirow{3}{*}{Optimizer} & Adam & 0.9843 & 0.9270 \\  & Adamax & 0.9861 & 0.9236 \\  & RMSProp & 0.9702 & 0.9061 \\ \hline \multirow{3}{*}{Learning Rate} & 0.1 & 0.9636 & 0.9346 \\  & 0.005 & 0.9690 & 0.9308 \\  & 0.001 & 0.9781 & 0.9383 \\ \hline \multirow{3}{*}{Epoch} & 20 & 0.8256 & 0.8642 \\  & 50 & 0.9411 & 0.9259 \\ \cline{1-1}  & 100 & 0.9904 & 0.9417 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Investigation of the hyperparameters on the student model. For the student model, nine lightweight state-of-the-art image classifiers were considered. Table 7 holds the details of the classifiers. Among them, Xception performed the best and EfficientNetV2B0 performed the worst. Three versions of the EfficientNet models were considered. EfficientNet architecture achieved high performance in various image classification tasks; however, the models failed to achieve acceptable results in DR classification. Although Xception did not have the fewest parameters and was relatively larger in size, it outperformed all other models, taking into account the comparative analysis, with a remarkable accuracy of 86.23% on the APTOS dataset, with an acceptable size. With knowledge distillation, the accuracy increased to 87.99%. The multi-class accuracy of the student model on the two datasets is presented in Figure 11. Tables 8 and 9 present the comparisons of different student modes with and without knowledge distillation, respectively, on the APTOS dataset. Likewise, Table 10 presents a comparison on the IDRiD dataset. The knowledge distillation process also improved its accuracy by 2.38%.