Can AI Help Make Screening Mammography "Lean"?

Although it is well known that mammographic screening can reduce breast cancer mortality, there is substantial variability in mammography reader performance (1). This variability, coupled with the increasing volume of screening studies, is stressing an already limited workforce. Methods to improve the sensitivity and specificity of readers have included computer-aided detection and the double reading of cases. Computer-aided detection, which is generally focused on improving detection, has had mixed results with little impact on sensitivity in large population studies (2). Double reading, which is frequently performed in service screening programs, has also been shown to have variable impact on screening outcomes and has only further strained an already limited workforce, even when performed with technologist assistants (3). Therefore, finding ways to maximize screening efficiency by eliminating wasteful activity while also maintaining or improving outcomes, a so-called lean approach to screening mammography, is necessary (4). Deep learning applications are rapidly gaining momentum with advances in hardware technology and have been shown to outperform conventional computer-aided detection algorithms (5). Most deep learning applications in breast cancer screening have focused on classifying suspicious findings (6). In this issue of Radiology, Yala et al (7) present an artificial intelligence, or AI, application that focuses on efficiency. Their goal was to reliably identify cancer-free mammograms (ie, true-negative findings) for triaging screening examinations at high confidence of being cancer free and, therefore, reduce the radiologist’s caseload to improve the efficiency while not compromising detection. In this study, a deep convolutional neural network, or CNN (ResNet18), was implemented. The CNN was trained, validated, and tested on a retrospective sample of a total 223 109 screening mammograms from 66 661 women. These mammograms were originally read by one of 23 breast imaging radiologists as part of routine clinical screening between 2009 and 2016 at the authors’ institution. The CNN incorporated pretraining from the ImageNet competition with data augmentation further refined with the authors’ training set, where the entire image, resized for optimal graphics processing unit processing, was fed as the network input. This model was previously trained by the authors to predict diagnosis of breast cancer within 1 year based on all the routine mammographic screening views for each patient (8). In the new application, Yala et al specifically chose the model’s threshold on the validation set to be that of the minimum score of a true-positive case, therefore maximizing the likelihood of triaging out true-negative mammograms while, ideally, maintaining sensitivity for cancer detection. To evaluate a clinical scenario where the developed model would be used for triage of true-negative assessments, the authors compared the performance of the radiologists during their original screening assessment to that of a retrospectively simulated scenario where the radiologists did not read any of the mammogram scored below the model’s “cancer-free” threshold, while reading the remainder of the nontriaged mammograms as before. The model showed good discriminatory ability, with an overall area under the curve of 0.82. The model had a similar performance across all age groups and races. The model was also discriminative across women with different breast density categories, with reported areas under the curve of 0.82, 0.81, 0.85, and 0.71 for women with fatty, scattered, heterogeneously, and extremely dense breasts, respectively. The model triaged a total of 5120 mammograms as cancer free, representing a reduction in caseload of 19%. In this simulated scenario, radiologists achieved a sensitivity and specificity of 90% and 94% (24 814 of 26 349; 95% confidence interval: 94.0%, 94.6%), respectively. The increase in specificity was significant (P = .002), and the sensitivity was noninferior by a margin of 5% (P < .001) compared with the complete caseload read according to standard of care. Two cases of cancer were missed with the new AI-driven model, one that was originally deemed as a false-negative finding by a radiologist and one deemed a true-positive finding and detected by a radiologist. Overall, the results reported by Yala et al are in agreement with data published by Rodriguez-Ruiz et al (8,9), who also suggest a 17% reduction in caseload and a 1% drop in sensitivity in a simulated enriched data set when applying a computer-aided model. Kyono et al (10) also recently showed a CNN model was able to maintain a negative predictive value of 99% while identifying 34% and up to 91% of negative mammograms for test sets with cancer prevalence of 15% and 1%, respectively. The findings of the study by Yala et al, therefore, add to the body of evidence that AI, when incorporated into the clinical workflow of mammographic screening interpretations, has potential advantages, including reducing the overall number of cases that require interpretation by a breast imaging radiologist. However, it still must be shown clinically whether this reduction of caseload leads to a reduction in overall time to read one’s “workload of cases.” It could be that with fewer cases in a radiologist’s worklist, the radiologist may spend more time on complex cases when the simple cases no longer require review. This approach would, ideally, improve the sensitivity of screening without necessarily decreasing total time of interpretation. This, nevertheless, remains to be seen. Most likely, a decrease in screening caseload will allow the radiologist to spend more time on other tasks such as biopsy procedures, diagnostic imaging, and multimodality imaging correlations. Although Yala et al report promising results, limitations of the study must be noted. As acknowledged by the authors, their retrospective study represented a simulated scenario. It is therefore unclear how a radiologist would not only interpret but also pace the reading of the remaining cases in the presence of the proposed reduced caseload. In addition, no tomosynthesis studies were included in the training of the CNN model. It is estimated that tomosynthesis images, which are rapidly becoming the standard for screening, take approximately twice the time to interpret due to the large data set. Thus, it will be important to develop similar algorithms to triage such cases to determine whether similar or potentially greater gains in efficiency could be achieved. From a technical implementation perspective, the training of the CNN model was done on “For Presentation” images and it is difficult to appreciate how much the characteristics captured may have been impacted by vendor-specific postprocessing. A tomosynthesis implementation could further compound computational efficiency of the CNN model due to the larger size and three-dimensional format of the imaging data. Finally, although the sensitivity of reading dropped slightly with the theoretical CNN model, it is unclear what type of cancers these were (eg, advanced invasive cancers vs in-situ lesions) and how such a reduction in sensitivity would translate into clinically significant declines in patient outcomes when scaled up to a population level. Ultimately, larger and more ethnically diverse studies (potentially incorporating additional risk factors) are needed to reliably triage true-negative screening mammograms while not compromising cancer detection and most importantly, patient outcomes. Toward this end, Yala et al made their convolutional neural network model publicly available. This can accelerate independent validation by other studies. Ultimately, this innovative application of artificial intelligence may prove more effective and reliable than conventional computer-aided detection in advancing a so-called lean approach to mammographic screening.