## 2 Novel Diabetic Retinopathy (DR) Detection Framework and Experiment
### Methodology
#### 2.2.4 Early DR Detection Algorithms  
The architecture used for DR detection was based on a transfer learning-based approach using known DL networks. This has been shown to be effective for detection in medical imaging applications. Eight pre-trained DNNs were used. The DNNs were kept in their original states, with the exception of the last layer, which was replaced with a fully connected layer with two units for DR detection. This was followed by a SoftMax function as the activation function, as well as a classification layer. DR detection was performed by means of pre-trained DNNs and convolutional neural networks (CNNs) such as AlexNet, ResNet, Inception-v3, and GoogleNet. The performance of each of these networks was studied and compared by means of the sensitivity, specificity, and accuracy, and precision measures, using the same testing data. For the detection, all images of the retinas that were marked as mild DR, moderate DR, severe DR, and proliferative DR were combined into a single 'Yes' category, and the retinal images without DR comprised the 'No' category. Of the available dataset, with N = 3662 (100%), 80% of the data were used for training and validation, and the other 20% were used for testing. This is illustrated in Table 3, showing the distribution of the training, testing, and validation datasets for DR detection; the total number of images without DR was 1805, and the total number of images with DR was 1857. Hence, the normal images constituted 49.3% and the DR images constituted 50.7%. These two classes/sets were considered approximately uniform, with a tolerance range of 1-2%. As described earlier, different pre-trained CNNs were used for DR detection, and their details, including the network name, depth, size, parameters, and image input size, are shown in Table 4. The selected pre-trained networks were trained on ImageNet, and their main properties are listed in Table 4. The network depth was defined as the largest number of sequential convolutional or fully connected layers on a path from the input layer to the output layer. The networks were Alexnet, Googlenet, DarkNet-53, EfficientNet-b0, SqueezeNet, Inception-v3, MobileNet-v2, and ResNet-50. Deep residual networks (ResNets), such as the popular ResNet-50 model, are CNNs with a depth of 50 layers. ResNet is an artificial neural network (ANN) that stacks residual blocks on top of each other to form a network. The inputs to the frameworks were the RGB images. In addition to the DNNs' properties, the models' learning rates, hyperparameters, and hardware requirements are reported in Table 5. The learning rate ranged between 0.001 and 0.01, and the decay strategy was the same for all networks. Concerning the hyperparameters, the learning rate for Alexnet, GoogleNet, ResNet-50, EfficientNet-b0, and SqueezeNet was 0.01; that of DarkNet-53 was 0.001; and that of Inception-v3 and MobileNet-v2 was 0.045. Regarding the decay, step decay was applied for all networks, as reported in Table 5.  
Regarding the CPU and GPU requirements, AlexNet required a moderate CPU/moderate GPU, GoogleNet required a high CPU/moderate GPU, ResNet-50 required a high CPU/high GPU, DarkNet-53 required a high CPU/high GPU, EfficientNet-b0 required a \begin{table}
\begin{tabular}{p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline \hline
**DR Label** & **\# Total Images** & **\# Training Images** & **\# Validation Images** & **\# Test Images** \\ \hline No & 1805 & 1300 & 144 & 361 \\ \hline Yes & 1857 & 1337 & 149 & 371 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The APTOS 2019 dataset: binary-class image distribution with the total number of images, and the number of trained, validated and tested images. moderate CPU/moderate GPU, SqueezeNet required a low CPU/low GPU, Inception-v3 required a high CPU/high GPU, and MobileNet-v2 required a low CPU/moderate GPU, as shown in Table 5.  
Finally, the Adam optimization technique was used for all networks, with a maximum number of epochs of six and with 274 iterations per epoch, based on empirical inference.