# A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading
## 5 Conclusions  
In this paper, a joint grading model of DR and DME based on multi-tasking and multi-branching MaMNet is proposed. Thanks to the multi-task learning strategy, MaMNet increases the grading accuracy by tapping more correlated information of DR and DME. A multi-branch network is designed to extract the underlying features of DR and DME in a targeted manner. The feature fusion module is constructed to obtain more comprehensive \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Input Image** & **True Label** & **Predicted Label** & **Various Grading Scores** & & & \\ \hline DR3 & DR3 & DR 0.000 & 0.000 & 0.093 & **0.697** & 0.209 \\ DME2 & DME2 & DME 0.000 & 0.052 & **0.947** & & \\ \hline DR2 & DR2 & DR 0.010 & 0.008 & **0.889** & 0.072 & 0.021 \\ DME2 & DME2 & DME 0.002 & 0.026 & **0.972** & & \\ \hline DR0 & DR0 & DR **0.912** & 0.022 & 0.043 & 0.016 & 0.007 \\ DME0 & DME0 & DME **0.902** & 0.065 & 0.033 & & \\ \hline DR2 & DR2 & DR 0.006 & 0.005 & **0.924** & 0.047 & 0.017 \\ DME2 & DME2 & DME 0.000 & 0.008 & **0.991** & & \\ \hline DR0 & DR0 & DR **0.991** & 0.003 & 0.004 & 0.001 & 0.000 \\ DME0 & DME0 & DME **0.970** & 0.018 & 0.012 & & \\ \hline DR4 & DR4 & DR 0.000 & 0.000 & 0.017 & 0.297 & **0.686** \\ DME2 & DME2 & DME 0.000 & 0.000 & **1.000** & & \\ \hline \hline \end{tabular}  
* The bold in the table represent the grading scores of the correctly classified category.  
\end{table} 
Table 7: Visualization of the best model prediction results. Figure 9: ROC curve and AUCPR curves of DR and DME. (**a**) ROC Curve; (**b**) DR’s AUPR Curve; (**c**) DME’s AUPR Curve. and disease-specific discriminative features. To validate the performance of MaMNet, experiments are conducted on the challenging IDRiD dataset. The experimental results show that compared with the other 11 advanced methods, the proposed method has better grading performance, and its joint accuracy, DR accuracy, and DME accuracy can reach 61.2%, 64.1%, and 79.6%, respectively. The dataset IDRiD used for model training is a public dataset with both DR and DME classification labels, only 413 images are used for training. To further improve the grading accuracy, the fundus images of diabetic patients with labels of both DR and DME disease categories will be collected subsequently to build a large dataset for joint grading and to train the model. In addition, we also want to explore the relationship between multiple diseases that occur in one fundus image in the future. At the same time, considering the existing issue of low accuracy, in future research directions, it may be beneficial to utilize classifiers such as Support Vector Machines (SVM), Naive Bayes (NB), and k-Nearest Neighbors (KNN) for the classification of DR and DME using features extracted through MaMNet [47]. In the last, for a new researcher, applying existing network architectures to new cases can be challenging. Here, a procedural guide is presented. The initial step entails inputting training images and labels for model training. Upon achieving a certain accuracy threshold, the trained model is stored for subsequent testing. In the subsequent step, the qualified model from the initial phase is employed on the test set to acquire corresponding labels for the test set images. This facilitates an assessment of the accuracy of the test results. Once a specific precision level is reached, the third step can be undertaken, involving the application of the final qualified model to a grading task with unknown labels. The final grading prediction results can assist physicians in conducting preliminary diagnosis and treatment for patients. Conceptualization, X.X. and M.Y. ; methodology, X.X. and M.Y. ; software, S.M., M.Y. and H.Y. ; validation, S.M., D.Y., J.Z. and C.Z. ([PERSON]); formal analysis, S.M. and M.Y. ; investigation, C.Z. ([PERSON]); writing--original draft preparation, M.Y. ; writing--review and editing, X.X., M.Y. and S.M. ; supervision, T.X. All authors have read and agreed to the published version of the manuscript.