# Diabetic Retinopathy Improved Detection Using Deep Learning
## 4 Experimental Results  
The training of our proposal was optimized using the Adam algorithm with a learning rate of \(1\times 10^{-5}\) with the categorical cross-entropy loss function. In addition, all the model parameters were updated during 50 epochs using the early stop method to prevent overfitting. Both datasets were used for training, validation, and testing purpose implemented as cross-testing. Hence, if the APTOS dataset is used to optimize the model, 80% is used for training and 20% for validation, leaving the Messidor dataset for testing. Equivalently, if Messidor is used to optimize the model, the same proportion is used for training and validation, leaving the APTOS dataset for testing. To compare the results between each dataset used for training, the accuracy, precision, recall, f1-score, the receiver operating characteristic (ROC) curve, and the area under the ROC curve are used. Tables 2 and 3 present the metrics for validation and testing results, respectively. In both Tables, DS means dataset, and in Table 3 only, the APTOSMSs and MessAPTOS are the testing metrics for the Messidor dataset with the model trained over APTOS and vice versa, respectively. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Dataset** & **No DR** & \begin{tabular}{c} **Slight** \\ **NPDR** \\ \end{tabular} & \begin{tabular}{c} **Medium** \\ **NPDR** \\ \end{tabular} & \begin{tabular}{c} **Severe** \\ **NPDR** \\ \end{tabular} & \
\begin{tabular}{c} **PDR** \\ \end{tabular} &
\begin{tabular}{c} **Total** \\ \end{tabular} \\ \hline
**APTOS** & 1805 & 370 & 999 & 193 & 295 & 3662 \\
**Messidor** & 1017 & 270 & 347 & 75 & 35 & 1744 \\ \hline
**Total** & 2822 & 640 & 1346 & 268 & 330 & 5406 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Quantity of images per class present in each dataset. The model presents a higher accuracy trained with the APTOS dataset than Messidor in the validation subset with 81% and 64%, respectively. Considering the cross-testing, the model trained over APTOS reached a 59% accuracy recognizing the classes in the Messidor dataset, being better than the training over Messidor with 33% of testing accuracy in the APTOS dataset. Observing the metrics during validation, the model learns a better precision, recall, and F1-score from the APTOS than Messidor datasets for all the classes. However, the Severe NPDR class presented better results from Messidor, and the PDR label classification was not learned from the Messidor dataset. In terms of testing results, the obtained metrics show that the model reaches a better precision trained over Messidor. Nevertheless, a better recall and F1-score is obtained for the model trained over APTOS. From the Figure 2 with the ROC curves, it can be appreciated that the model trained over APTOS learned more useful features for most of the classes than the trained over the Messidor dataset. The most difficult class was the Slight NPDR because the area under the ROC curve was almost the same for both datasets. A special issue in this classification task is the lack of a standardized benchmark dataset, establishing baseline results that must be used as a reference for new proposals. The Messidor dataset was previously used and presents a challenging data distribution, making it suitable as a benchmark dataset. In counterpart, the APTOS dataset presents a more variate data distribution suitable to optimize the model's parameters. Considering the images from Figure 3 and the values in Table 3, the standardized images from the Messidor dataset expose better features for each different 4 DR-related outputs. However, the data distribution from Messidor is not enough to generalize in different image representations.