Deep learning in glaucoma: progress, but still lots to do
Authors: Felipe A Medeiros
Journal: The Lancet Digital Health (2019)

In the past 5 years, the applications of artificial intelligence—and particularly of deep-learning networks—have increased rapidly in medicine. Convolutional neural networks, a type of deep-learning network, have become the go-to algorithm in computer vision tasks such as image classification, with performance sometimes surpassing even that of humans.1He K Zhang X Ren S Sun K Delving deep into rectifiers: surpassing human-level performance on ImageNet classification.arXiv. 2015; (DOI:1502.01852v1 published online Feb 15.)Google Scholar As a medical specialty that is highly dependent on ancillary imaging tests, ophthalmology has been a prime area for application of convolutional neural network algorithms to help to analyse these imaging data. In The Lancet Digital Health, An Ran Ran and colleagues2Ran AR Cheung CY Wang X et al.Detection of glaucomatous optic neuropathy with spectral-domain optical coherence tomography: a retrospective training and validation deep-learning analysis.Lancet Digital Health. 2019; 1: e172-e182Summary Full Text Full Text PDF PubMed Scopus (69) Google Scholar apply a deep-learning algorithm that uses spectral-domain optical coherence tomography (SDOCT) scans of the optic nerve head and retinal nerve fiber layer to attempt to detect glaucoma, a leading cause of irreversible blindness worldwide. Damage to the optic nerve is the defining feature of glaucoma, and SDOCT of the optic nerve head can provide a wealth of data that could be interpreted by convolutional neural network approaches for improved diagnosis of the disease. By contrast with previous deep-learning approaches that have focused on analysing two-dimensional information from SDOCT scans or fundus photographs,3Medeiros FA Jammal AA Thompson AC From machine to machine: an OCT-trained deep learning algorithm for objective quantification of glaucomatous damage in fundus photographs.Ophthalmology. 2019; 126: 513-521Summary Full Text Full Text PDF PubMed Scopus (112) Google Scholar, 4Li Z He Y Keel S Meng W Chang RT He M Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs.Ophthalmology. 2018; 125: 1199-1206Summary Full Text Full Text PDF PubMed Scopus (380) Google Scholar, 5Christopher M Belghith A Bowd C et al.Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs.Sci Rep. 2018; 816685Crossref PubMed Scopus (177) Google Scholar Ran and colleagues derived a convolutional neural network that used three-dimensional information from SDOCT scans of the optic nerve head. In theory, such an approach could lead to better assessment of damage by taking into account the whole topography of the optic nerve. To provide the ground-truth or reference standard to train the deep-learning network in Ran and colleagues' study, human experts classified the same SDOCT images as having or not having glaucomatous optic neuropathy, but also had access to visual field tests from the same eyes. The performance of the deep-learning algorithm was then assessed in three external validation datasets from different institutions. The diagnostic accuracy was similar in all three external validation datasets, with areas under the receiver operating characteristic curves (AUROCs) of approximately 0·89 and sensitivities ranging from 78% to 90% for specificities of 79–86%. Ran and colleagues' work2Ran AR Cheung CY Wang X et al.Detection of glaucomatous optic neuropathy with spectral-domain optical coherence tomography: a retrospective training and validation deep-learning analysis.Lancet Digital Health. 2019; 1: e172-e182Summary Full Text Full Text PDF PubMed Scopus (69) Google Scholar is a welcome addition to the literature, and the authors are to be congratulated for the laborious work of carefully assembling large datasets for training and testing of the deep-learning algorithm. However, although the potential for modern deep learning applied to SDOCT images in ophthalmology is vast and promising, old truths related to how medical tests need to be assessed and applied in clinical practice still stand strong. One of the most important aspects in assessments of the accuracy of any medical test is how the reference standard is defined and constructed. In the case of deep-learning networks, the reference standard could also serve as a target that the network is trained to predict. In this study, the specific criteria used by the graders to classify the presence of absence of glaucomatous optic neuropathy in SDOCT images are unknown. A plethora of parameters are provided by standard SDOCT software, and how graders created the ground-truth labelling from these parameters is unknown. Irrespective of how well a deep-learning network is built and trained, it will only perform as well as the ground-truth that it was trained to replicate. A poorly constructed ground-truth will produce a deep-learning network that will replicate the same mistakes made by humans.3Medeiros FA Jammal AA Thompson AC From machine to machine: an OCT-trained deep learning algorithm for objective quantification of glaucomatous damage in fundus photographs.Ophthalmology. 2019; 126: 513-521Summary Full Text Full Text PDF PubMed Scopus (112) Google Scholar Because glaucoma has a low prevalence, an algorithm would need to have high specificity and sensitivity (higher than 95%) before being used for disease screening.6Vaahtoranta-Lehtonen H Tuulonen A Aronen P et al.Cost effectiveness and cost utility of an organized screening programme for glaucoma.Acta Ophthalmol Scand. 2007; 85: 508-518Crossref PubMed Scopus (69) Google Scholar For example, if the glaucoma prevalence is 4%, the positive predictive value of the deep-learning network developed by Ran and colleagues would be around 20%—ie, 80% of the people whose scans tested positive would actually not have glaucomatous optic neuropathy. The low accuracy in the validation datasets is surprising given that the reference standard was based on human gradings of the same SDOCT images. Although the authors claim that the three-dimensional SDOCT deep-learning model performed better than a two-dimensional model based on the fundus image, this finding would be expected given that the reference standard (grading by glaucoma specialists) used the SDOCT images rather than fundus images. If the purpose of Ran and colleagues' algorithm were to assist in the interpretation of SDOCT images in clinical practice, rather than population-based screening, performance might still be considered suboptimal. For example, the average mean deviation of eyes in external validation dataset 3 was −9 dB, suggesting that most eyes had severe glaucoma. But in this dataset, the deep-learning system had 78% sensitivity and 86% specificity. Importantly, in clinical practice, diagnostic tests are used to assist with assessment of patients suspected of having the disease, not those who already have a confirmed diagnosis. Therefore, from a practical standpoint, a new diagnostic test that is able to detect glaucoma in a patient with moderate or severe visual field loss might not be useful because such patients would already have a clear diagnosis established by perimetry. It will be important in the future to see how Ran and colleagues' proposed deep-learning algorithm could help to diagnose glaucomatous damage in people suspected of having glaucoma. The performance of the deep-learning algorithm was not compared with average peripapillary retinal nerve fibre layer thickness—a summary parameter automatically provided by all SDOCT scanners that discriminates well between glaucomatous eyes and healthy eyes, with most studies showing AUROCs that surpass 0·90 and the ability to detect damage in people with suspected glaucoma.7Kuang TM Zhang C Zangwill LM Weinreb RN Medeiros FA Estimating lead time gained by optical coherence tomography in detecting glaucoma before development of visual field defects.Ophthalmology. 2015; 122: 2002-2009Summary Full Text Full Text PDF PubMed Scopus (97) Google Scholar Comparisons to tried-and-tested summary SDOCT parameters should be done to show the advantage of a newly proposed algorithm analysing SDOCT data. There are several exciting applications of deep-learning analysis of SDOCT images. SDOCT devices can image not only the optic nerve, but also areas such as the macula, which is also important in glaucoma.8Kim KE Park KH Macular imaging by optical coherence tomography in the diagnosis and management of glaucoma.Br J Ophthalmol. 2018; 102: 718-724Crossref PubMed Scopus (39) Google Scholar Deep-learning algorithms could be used to integrate the vast information derived from imaging multiple regions of the eye to improve detection of glaucomatous damage. It will also be important to assess whether deep-learning algorithms can help to detect progressive glaucomatous changes over time—a task that has proven a lot harder than discrimination of glaucomatous eyes from healthy eyes. Additionally, cost-effectiveness studies will be important to identify the benefits of incorporating new deep-learning based approaches in clinical practice and in screening for glaucoma. In conclusion, the future of deep-learning applications in glaucoma and ophthalmology is bright. However, no matter how exciting new technologies can be, validation of new diagnostic tests should be based on rigorous methods with particular attention paid to how the reference standards are defined and the settings where the tests are going to be applied in practice. These points are especially important in glaucoma, a disease that does not have a litmus test for diagnosis. For deep learning in glaucoma, Ran and colleagues certainly help to move us forward, but there is still a lot of work to do. I have received research support from Carl-Zeiss Meditec and Heidelberg Engineering. Detection of glaucomatous optic neuropathy with spectral-domain optical coherence tomography: a retrospective training and validation deep-learning analysisThe proposed 3D deep-learning system performed well in detection of glaucomatous optic neuropathy in both primary and external validations. Further prospective studies are needed to estimate the incremental cost-effectiveness of incorporation of an artificial intelligence-based model for glaucoma screening. Full-Text PDF Open Access