# Deep Learning Based Cardiac MRI Segmentation: Do We Need Experts?
## 2 Methods and Data  
As mentioned before, medical data annotation requires rightful expertise so that the labeling can be used with full confidence. Expert annotators are typically medical doctors or medical specialists whose training and experience are reliable sources of truth for the problem at hand. These experts often have close collaborators working daily with medical data, typically computer scientists, technicians, biophysicist, etc. While their understanding of the data is real, these non-experts are typically not considered a reliable source of truth. Non-experts are thus considered as people who can manually label data but whose annotations are biased and/or noisy and thus unreliable.  
In this study, two non-experts were asked to label 1902 cardiac images. We defined a non-expert as someone with no professional expertise on cardiac anatomy nor on cine-MR images. Non-Expert 1 is a technician in biotechnology who received 30 min of training by a medical expert on how to recognize and outline cardiac structures. The training was done on a few examples where the expert showed what the regions of interest in the image look like and where their boundaries lie. Training also came with an introduction to the cardiac anatomy and its temporal dynamics. Non-Expert 2 is a computer scientist with 4 years of active research in cardiac cine-MRI with several months of training. In the case of Non-Expert 2, the training spanned several months, where directions about the imaging modality as well as the anatomy and pathologies were thoroughly explained. In addition,fine delineation guidelines were provided to disambiguate good from poor annotations. In this study, we both gauge the effect of training a neural network on non-expert data and also verify how the level of training of the non-experts impact the overall results.  
The non-experts were asked to delineate three structures of the heart, namely, the left ventricular cavity (endocardial border), the left ventricle myocardium (epicardial border) and the endocardial border of the right ventricle. No further quality control was done to validate the non-expert annotations. Segmentations were used as is for the subsequent tasks.  
We used the gold standard for medical image segmentation U-Net [15] as the baseline network. In addition, the well-known Attention U-Net [16] and ENet [17] networks were trained in order to ensure that the results are affected by the differences in annotations and not the network architecture. We first trained the the segmentation models (U-Net, Attention U-Net and ENet) on the original ACDC data set (Automated Cardiac Diagnosis Challenge) [2] with its associated ground truth, using a classical supervised training scheme, with a combined cross-entropy and Dice loss:  
\[L_{ce}=-\frac{1}{N}\sum_{i=1}^{N}\sum_{k=1}^{3}y_{ki}\log\hat{y}_{ki} \tag{1}\]  
\[L_{dice}=1-\frac{1}{N}\bigg{[}\sum_{i=1}^{N}\frac{2\times\sum_{k}\hat{y}_{ki}y _{ki}}{\sum_{k}\hat{y}_{ki}+\sum_{k}y_{ki}}\bigg{]} \tag{2}\]  
where \(\hat{y}_{ki}\) is the probabilistic output for image \(i\in N\) (\(N\) is the number of images in the batch) and class \(k\in\{1,2,3\}\) (3 is the number of classes). \(\hat{y}\) is the predicted output of the network, \(y\) is a one-hot encoding of the ground truth segmentation map.  
We then re-trained the neural networks with the non-expert labels, using the same training configuration. Furthermore, considering that the non-expert annotations can be seen as noisy versions of the true annotation (i.e., \(y^{\prime}=y+\epsilon\) where \(y^{\prime}\) is the non-expert annotation, \(y\) is the ground truth and \(\epsilon\) a random variable), we also trained the networks with a mean absolute error loss which, as shown by [PERSON] et al. [18], has the solve property of compensating for labeling inaccuracies.