## 2 Materials and Methods
### Enhanced CNN Models  
Initially, the architectures of VGG16, ResNet50 and DenseNet169 were applied to the APTOS dataset. To be able to use these architectures in transfer learning and for classifying the diabetic retinopathy images into five classes, fully connected layers were added. The 3-dimensional feature map obtained from the last convolutional layer was converted to one dimension by using global average pooling 2D and passed to a series of a dropout layer, a dense layer and a dropout layer and finally to a dense layer with five nodes, representing the normal and the DR grades. The fully connected layers were selected as in the ResNet model in [PERSON] et al. (2019), and [PERSON] et al. (2020) shows that adding fully connected layers yields better results. The activation function used in the last dense layer was Softmax, as used in [PERSON] et al. (2019) for face detection classification. The Adam optimizer was applied to the 3 models with a learning rate of \(1\times 10^{-3}\), and the loss entropy used was the categorical cross entropy. In this work, data balancing was performed using basic image manipulation techniques ([PERSON] et al., 2020). In the deep neural network, the Adam optimizer was used instead of the stochastic gradient descent (SGD) since the former is computationally more efficient. The Adam optimizer has been found to be faster in converging the algorithm to the minima, hence reducing the training time ([PERSON] et al., 2020). The use of the SGD and other approaches will be explored in future works. Here, only the last 5 layers, namely the global average pooling 2D, dropout, dense, dropout and dense layers, were trained. The other layers were frozen as we were only extracting the features from the base model. These steps resulted in the models producing the relevant learnable parameters during the training process. For example, for the ResNet model, out of the 27,794,309 parameters, 4,206,597 were trainable. In this work, the sequential modelling approach was adopted for adding and customizing the convolution, dropout, dense and optimizer layers. The sequential model is appropriate for a plain stack of layers whereby each layer has exactly one input tensor and one output tensor, which was the case in this application.  
To improve the performance of the models and cater for underfitting/overfitting, the 3 models were fine-tuned. The Adam optimizer was again used but this time with a learning rate of \(10^{-4}\). The learning rate was decremented by 10 as this has been shown to both reduce the risk of overfitting ([PERSON] et al., 2020) and to improve classification ([PERSON] et al., 2020). When the validation loss metric stopped improving, the learning rate was halved as in ([PERSON] et al., 2020). Several parameters were changed and added to the models for fine-tuning. Firstly, the loss function was changed to binary cross entropy. Using the latter along with a SoftMax classifier helped the model in reducing the cross entropy loss of each iteration in multiclass classification ([PERSON] et al., 2020). Afterwards, an early stopping feature was added to end training when the network began to overfit the data according to the validation loss ([PERSON] et al., 2020). Eventually, all the convolutional layers were unfrozen, and the models were set to be trained.  
The enhanced transfer learning model that was trained on the augmented APTOS dataset was tested on APTOS test data and on a blind Mauritian test datum annotated by a medical practitioner.  
Figure 7: Architecture of DenseNet169 model.