# A Multi-Task Learning and Multi-Branch Network for DR and DME Joint Grading  
[PERSON]  
1 College of Electronic Information Engineering, Changchun University, Changchun 130012, China; [EMAIL_ADDRESS] (S.M. ); [EMAIL_ADDRESS] (M.T. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z.) [PERSON]  
1 College of Electronic Information Engineering, Changchun University, Changchun 130012, China; [EMAIL_ADDRESS] (S.M. ); [EMAIL_ADDRESS] (M.T. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z.) [PERSON]  
1 College of Electronic Information Engineering, Changchun University, Changchun 130012, China; [EMAIL_ADDRESS] (S.M. ); [EMAIL_ADDRESS] (M.T. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z.) [PERSON]  
1 College of Electronic Information Engineering, Changchun University, Changchun 130012, China; [EMAIL_ADDRESS] (S.M. ); [EMAIL_ADDRESS] (M.T. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z. ); [EMAIL_ADDRESS] (C.Z.) [PERSON]  
1 College of Electronic Information Engineering, Changchun University, Changchun 130012, China; [EMAIL_ADDRESS] (S.M. ); [EMAIL_ADDRESS] (M.T. ); [EMAIL_ADDRESS] (H.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (D.Y. ); [EMAIL_ADDRESS] (C.Z. DME is a concomitant symptom of DR and is the most common cause of visual impairment or loss. It refers to retinal thickening or hard exudative deposits caused by the accumulation of extracellular fluid within the central macular sulcus, which can be classified into 3 levels by the distance of exudate from the macular center [4]: 0 (normal, no obvious hard exudate), 1 (mild, hard exudate outside the circle with a radius of one optic disc diameter from the macular center), and 2 (severe, hard exudate within a circle with a radius of one optic disc diameter from the macular center). DR and DME have a more complex relationship, as shown in Figure 2. As shown in Figure 2a,b, there is a small amount of hard exudate in the macular area of the fundus image, and its DR and DME labels are 2. The DR grade of 2 means that the degree of DR is moderate. The DME grade of 2 means the lesion level of DME is severe, that likely causes the patient to go blind. So, in this circumstance, if only DR grading is done, the extent of the patient's lesion will be misjudged. As seen in Figure 2c,d, there is no exudate in the macular area, which has DR and DME grades of 4 and 0. It means normal with a grading label of 0 for DME, but a grading label of 4 for DR means proliferative, which can also lead to vision loss. In such a case, if only DME grading is done, the degree of the patient's lesion will also be misjudged. Therefore, it is important to automatically grade both DR and DME to assist physicians in taking the appropriate therapeutic options. The study of DR and DME auto-grading based on deep learning has also evolved with the development of convolutional neural networks(CNN). A CNN-based model was first proposed by [PERSON] et al. [5] to classify the five-level of DR, where they used a class-weighting strategy to update the parameters of each batch during backpropagation to compensate for the class imbalance in the dataset and reduce overfitting. The model designed by [PERSON] and [PERSON] [6] had five residual blocks, which first extracted fundus lesion features and then fed the extracted features into a decision tree for secondary classification with and without  
Figure 1: Fundus images from the IDRiD dataset with hard exudates, soft exudates, hemorrhage, microaneuryss, and other lesions.The severity of DR is related to microaneuryss, hemorrhage, soft exudates and hard exudates. The severity of DME is determined by the distance from hard exudates to the macular area.