AI-automated referral for patients with visual impairment
Authors: Minhaj Alam, Joelle A Hallak
Journal: The Lancet Digital Health (2020)

In the past 5 years, deep neural networks have been applied in several medical applications such as detecting pneumonia and lung consolidation from chest x-rays and CT scans,1Zhang K Liu X Shen J et al.Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography.Cell. 2020; 1821360Summary Full Text Full Text PDF PubMed Scopus (19) Google Scholar and melanoma from skin photographs.2Esteva A Kuprel B Novoa RA et al.Dermatologist-level classification of skin cancer with deep neural networks.Nature. 2017; 542: 115-118Crossref PubMed Scopus (54) Google Scholar Deep learning has also been integrated into ophthalmic applications, especially with use of fundus and optical coherence tomography images because of their widespread use in clinical settings. Deep learning has been shown to detect diabetic retinopathy, glaucoma, age-related macular degeneration, diabetic macular oedema, and retinopathy of prematurity, and even to predict several systemic cardiovascular risk factors.3Rim TH Lee G Kim Y et al.Prediction of systemic biomarkers from retinal photographs: development and validation of deep-learning algorithms.Lancet Digit Health. 2020; 2: e526-e536Summary Full Text Full Text PDF Scopus (24) Google Scholar Additionally, AI models have been shown to be effective for clinical referrals to tertiary care centres.4Dunnmon JA Ratner AJ Saab K et al.Cross-modal data programming enables rapid medical machine learning.Patterns (N Y). 2020; 8100019Summary Full Text Full Text PDF Scopus (9) Google Scholar, 5Bellemo V Lim ZW Lim G et al.Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study.Lancet Digit Health. 2019; 1: e35-e44Summary Full Text Full Text PDF Scopus (102) Google Scholar, 6Wu X Huang Y Liu Z et al.Universal artificial intelligence platform for collaborative management of cataracts.Br J Ophthalmol. 2019; 103: 1553-1560Crossref Scopus (41) Google Scholar, 7Brown JM Campbell JP Beers A et al.Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.JAMA Ophthalmol. 2018; 136: 803-810Crossref PubMed Scopus (244) Google Scholar Studies have shown multiple models and pipelines for triage of pneumonia using chest x-rays;4Dunnmon JA Ratner AJ Saab K et al.Cross-modal data programming enables rapid medical machine learning.Patterns (N Y). 2020; 8100019Summary Full Text Full Text PDF Scopus (9) Google Scholar for referral for diabetic retinopathy (vision threatening),5Bellemo V Lim ZW Lim G et al.Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study.Lancet Digit Health. 2019; 1: e35-e44Summary Full Text Full Text PDF Scopus (102) Google Scholar cataract,6Wu X Huang Y Liu Z et al.Universal artificial intelligence platform for collaborative management of cataracts.Br J Ophthalmol. 2019; 103: 1553-1560Crossref Scopus (41) Google Scholar and retinopathy of prematurity;7Brown JM Campbell JP Beers A et al.Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks.JAMA Ophthalmol. 2018; 136: 803-810Crossref PubMed Scopus (244) Google Scholar and telemedicine programmes using fundus photographs. In April, 2018, the US Food and Drug Administration approved the first AI-based referral device for diabetic retinopathy.8Abràmoff MD Lavin PT Birch M Shah N Folk JC Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices.NPJ Digit Med. 2018; 1: 39Crossref PubMed Google Scholar In The Lancet Digital Health, Yih-Chung Tham and colleagues9Tham Y-C Anees A Zhang L et al.Referral for disease-related visual impairment using retinal photograph-based deep learning: a proof-of-concept, model development study.Lancet Digit Health. 2020; 3: e29-e40Google Scholar report development of a single modality (fundus photograph) deep learning algorithm for the detection of disease-related visual impairment with the goal of creating a referral system in vision care. Following the recommendation of WHO, some high-income countries have implemented annual vision screening, especially for older populations.9Tham Y-C Anees A Zhang L et al.Referral for disease-related visual impairment using retinal photograph-based deep learning: a proof-of-concept, model development study.Lancet Digit Health. 2020; 3: e29-e40Google Scholar Some screening systems rely on visual acuity checks whereas others use retinal photography to identify disease for referral to tertiary care. To incorporate visual acuity and retinal images for tertiary referral, large-scale resources and trained personnel are required. However, a robustly trained AI system could potentially automate this referral process. To develop the algorithm, Tham and colleagues used data from the Singapore Epidemiology of Eye Disease Study (SEED) to train the model, and they validated the model using five external datasets. The diverse datasets contained images of eyes from Malay, Indian, Chinese and some White participants, with the mean age across the external validation datasets ranging 48·2 years (SD 13·4) to 76·6 years (6·8). The system is two-tiered, with two separate algorithms used for prediction of the presence of visual impairment (classification task), and prediction of best-correlated visual acuity level (regression task). The authors used a standard neural network model (Residual Neural Network [ResNet]-50) for classification and regression. However, the use of disease-related visual impairment as a predictor for tertiary referral is new. The ResNet model did well in predicting the presence of any disease-related visual impairment, with an area under receiver operating characteristic curve (AUC) range varying from 86·6% (95% CI 83·4–89·7) to 93·6% (92·4–94·8) across the external validation datasets.9Tham Y-C Anees A Zhang L et al.Referral for disease-related visual impairment using retinal photograph-based deep learning: a proof-of-concept, model development study.Lancet Digit Health. 2020; 3: e29-e40Google Scholar In external datasets, the AUC range for predicting mild disease-related visual impairment was 81·9% (77·2–86·6) to 92·5% (90·8–94·2) and for moderate disease-related visual impairment was 85·9% (81·8–90·1) to 93·5% (91·7–95·3). According to these AUC data, the model did quite well to identify disease-related visual impairment in the external datasets, indicating potential generalisability, which is a crucial element for widespread deployment and clinical utility. Additionally, to visualise the regions of interest for the trained model, the authors generated saliency maps using the GradCAM method. They found that saliency maps were congruent with the pathological signs correlated with disease-related visual impairment. Interestingly, when more than one disease was present in the image, the algorithm had the ability to identify all features with different contributory weights. One of the limitations of the algorithm developed by Tham and colleagues is the paucity of representation for different populations. Depending on where the referral system is to be deployed, it needs to be trained and validated further on data that are representative of the population. Another limitation is the high class imbalance (approximately 10% positive visual impairment labels). The authors mentioned that they used a class weight when training the model, but a better method might be to train the model on more eyes with visual impairment to encompass a more representative sample of the general population. Furthermore, similar to other AI-based systems, this model is susceptible to bias towards ground labels and training datasets. The model has learned only what was presented in the training images, which might not represent real-world data. The authors' approach to develop a deep learning algorithm for tertiary eye-care referral might enable diagnosis of any eye disease with substantial visual impairment. Subsequent studies could include the development of referral systems that could identify specific stages of disease causing visual impairment and visual loss. These algorithms might be more valuable from a clinical standpoint, serving as a tool for both screening and referral. We have observed that AI models are robust in identifying diseases and their specific stages. For example, studies have shown how deep learning algorithms can identify control eyes versus eyes with diabetic retinopathy and whether the eyes have mild, moderate, or severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy.10Gulshan V Peng L Coram M et al.Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.JAMA. 2016; 316: 2402-2410Crossref PubMed Scopus (3079) Google Scholar In a diabetic retinopathy example, an AI system could be trained to only flag eyes that have severe non-proliferative diabetic retinopathy or proliferative diabetic retinopathy causing visual impairment in either eye for referral. We can be hopeful that a future robust multidisease system that might be able to identify control eyes versus eyes with early stage disease for screening, and then identify specific disease and stages for specific referral. Despite the exponential growth of AI applications in medicine and ophthalmology, translations into clinical practice remain challenging. Algorithms need to be generalisable to different populations and explainable to ensure targeted performance, population representation, and identification of potential problems and bias in training data. To implement a referral tool for patients with disease-related visual impairment, collaboration between researchers and health-care workers in primary and tertiary centres, as well as regulatory agencies, need to be forged for successful integration within the clinical workflow. JAH has received grant funding support from the BrightFocus Foundation outside of the submitted work. We declare no competing interests. Referral for disease-related visual impairment using retinal photograph-based deep learning: a proof-of-concept, model development studyThis proof-of-concept study shows the potential of a single-modality, function-focused tool in identifying visual impairment related to major eye diseases, providing more timely and pinpointed referral of patients with disease-related visual impairment from the community to tertiary eye hospitals. Full-Text PDF Open Access