# Deep Learning Based Cardiac MRI Segmentation: Do We Need Experts?
## 1 Introduction  
Deep neural networks (more specifically, _convolutional neural networks_) have deeply percolated through healthcare R&D, addressing various problems, such as survival prediction, disease diagnostics, image registration, anomaly detection, and the segmentation of images, be it magnetic resonance images (MRI), computed tomography (CT) or ultrasound (US), to name a few [1]. The roaring success of deep learning methods is rightly attributed to the unprecedented amount of annotated data across domains. However, ironically, while solutions to decade-long medical problems are at hand [2], the use of neural networks in day-to-day practice is still pending. This can be explained in part by the following two observations. First, while being accurate _on average_, neural networks can nonetheless be sometimes wrong [3], as they provide no strict clinical guarantees. In other words, any neural network within the intra-expert variability is excellent _on average_ but not immune to sparse erroneous (yet degenerated) results, which is problematic in clinical practice [2]. Second, machine learning methods are known to suffer from domain adaptation problems, one of the most glaring medical imaging issue of our times [4]. As such, clinically accurate machine learning methods trained on a specific set of data almost always see their performance drop when tested on a data set acquired by following a different protocol. These problems are derived in good part from the fact that current data sets are still relativelysmall. According to [PERSON] et al. [5] most medical imaging challenges organized so far contain less than 100 training and testing cases. This shows that medical applications cannot rely on a _very_ large medical data set encompassing tens of thousands of annotated data acquired in various conditions, with machines of various vendors showing clinical conditions and anatomical configurations of all kinds. This is unlike non-medical computer vision problems, which have had access for a long time to large and varied data sets, such as ImageNet, Coco, PascalVOC, ADE20k, and Youtube-8M, to name a few [6]. The annotation of these data sets relies on non-experts, often through online services, such as Mechanical Turk [7]. Unfortunately, obtaining similarly large annotated data sets in medical imaging is difficult. The challenge stems from the nature of the data, which is sensitive and requires navigating a complicated regulatory framework and privacy safeguards. Furthermore, labeling medical data sets is quite resource intensive and prohibitively costly, as it requires a domain expertise. For these reasons, the medical imaging literature has had an increasing number of publications, whose goal is to compensate for the lack of expert annotations [8]. While some methods leverage partly-annotated data sets [9], others use domain adaptation strategies to compensate for small training data sets [10]. Some other approaches artificially increase the number of annotated data with generative adversarial networks (GANs) [11; 12], while others use third-party neural networks to help experts annotate images more rapidly [13]. While these methods have been shown to be effective for their specific test cases, it is widely accepted that large manually-annotated data sets bring indisputable benefits [14]. In this work, we depart from trying to improve the segmentation methods and focus on the data sets as we challenge the idea that medical data, cardiac cine MRI specifically, needs to be labeled by experts only, and explore the consequences of using non-expert annotations on the generalization capabilities of a neural network. Non-expert here refers to a non-physician who cannot be regarded as a reference in the field. While non-expert annotations are easier and cheaper to obtain, they could be used to build larger data sets faster and at a reduced cost. This idea was tested on cardiac cine-MRI segmentation.