## 6 Results and Discussion
### Experimental Results
#### 6.2.2 Classification Results

Table 3: Results of 10 different experimental runs by random bootstrapping selection of 1000 images as testing set out of the 109,312 images on Large Dataset of Labeled Optical Coherence Tomography (OCT) v3 [25]. Figure 13: Training and Testing Curves for Norm-VGG16 (kernel regularized) and ResNet-50 (kernel regularized) (**A**) Training accuracy curves (**B**) Testing accuracy curve (**C**) Training loss curves (**D**) Testing loss curve. The performance of the models is summarized and compared in Table 4. From the shown results, it is determined that the best performing model is HyCAD architecture according to its confusion matrix presented in Figure 14A, scoring an accuracy of 98.8%. In Urgent referrals group, the HyCAD architecture only misclassified 3 images out of 500 and achieves a sensitivity of 99.4% as shown in Table 4. In the Nonurgent referrals group, the same model misclassified only 9 images out of 500 images and achieves 98.2%. It is worth mentioning that the Norm-VGG16 architecture without Kernel regularization attained a testing accuracy of 96.7% when trained on the dataset from scratch. 2. HyCAD Performance Compared to Pure CNNs Performance and State of the Art  
The accuracy per class is calculated and compared with previous models of [PERSON] et al. [4] and [PERSON] et al. [20] in Table 5. The per class accuracy is given by dividing the truly classified image for each class by their total number. It is noticeable from the accuracies in Table 5 that the lowest accuracy scored by HyCAD is of DRUSEN class, which is explainable due to the fact that it has the lowest percentage in the training set. Nevertheless, HyCAD with the fused hand-crafted features managed to surpass its CNN counterpart Norm-VGG16 in separating the DRUSEN minority class with a difference of 3.2%. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Accuracy** & **Sensitivity (UR) (Recall)** & **Specificity (UR)** \\ \hline ResNet-50 (kernel regularized) & 97\% & 98.6\% & 95.4\% \\ \hline Norm-VGG16 (kernel regularized) & 97.3\% & 98\% & 96.6\% \\ \hline
**HyCAD architecture** & **98.8\%** & **99.4\%** & **98.2\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison between different architectures. Figure 14: Confusion matrix (**A**) HyCAD confusion matrix (**B**) Norm-VGG16 (kernel regularized) (**C**) ResNet-50 (kernel regularized). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Accuracy/Class Models** & \multicolumn{2}{c}{**Urgent Referrals Group**} & \multicolumn{2}{c}{**Nonurgent Referrals Group**} \\ \cline{2-5}  & **CNV** & **DME** & **DRUSEN** & **NORMAL** \\ \hline [PERSON] et al. 2018 [4] & 94.8\% & 96.8\% & 94.4\% & 98.4\% \\ \hline [PERSON] et al. 2019 [20] & 96.4\% & 99.2\% & 99.2\% & 99.6\% \\ \hline HyCAD model & 99.6\% & 99.2\% & 97.2\% & 99.2\% \\ \hline Norm-VGG16 (kernel regularized) & 96.8\% & 99.2\% & 94\% & 99.2\% \\ \hline ResNet-50(kernel regularized) & 99.6\% & 97.6\% & 90.8\% & 100\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Accuracy per class comparison between proposed models, [PERSON] et al. and [PERSON] et al. models on Large Dataset of Labeled Optical Coherence Tomography (OCT) v3 [25]. As shown in Table 5, the proposed HyCAD model achieves a significant increase in accuracy of CNV, DME, DRUSEN by 4.8%, 2.4%, 2.8%, while it attains similar NORMAL accuracy compared to [PERSON] et al. [4]. The proposed HyCAD model achieves an increase in accuracies of CNV class by 3.2% compared to [PERSON] et al. [20]. In order to further examine the performance of our best performing model HyCAD architecture relative to the state of the art, its overall performance is compared to the work of [PERSON] et al. [4] and [PERSON] et al. [20]. The comparison is depicted in Table 6 and Figure 15. The values for [PERSON] et al. [4] and [PERSON] et al. [20] are calculated from the best reported confusion matrices as shown in Figure 16. The HyCAD architecture outperforms the work of [PERSON] et al.