Artificial Intelligence, Machine Learning and Deep Learning in Ophthalmology: Current Clinical Relevance

With the advent of computer graphic processing units, improvement in mathematical models and availability of big data, artificial intelligence (AI) using machine learning (ML) and deep learning (DL) techniques have achieved robust performance for potential application across many industries, including social-media, the internet of things, the automotive industry and healthcare. DL systems provide capability in image, speech and motion recognition as well as in natural language processing. In medicine, most of the progress of AI, ML and DL systems has been demonstrated in image-centric specialties such as radiology, dermatology and pathology. There is increasing interest in AI in ophthalmology. New studies, including pre-registered prospective clinical trials, have shown DL systems are effective in detecting diabetic retinopathy (DR), glaucoma, age-related macular degeneration, retinopathy of prematurity, refractive error and in identifying cardiovascular risk factors and diseases, using image based data such as fundus photographs and optical coherence tomography. Additionally, the application of ML to Humphrey visual fields may be useful in detecting glaucoma progression. There are fewer studies that incorporate clinical data in AL algorithms and no prospective studies to demonstrate that AI algorithms can predict the development of eye disease. This article describes the current global eye disease burden, clinical unmet needs and selected common ophthalmic conditions of public health importance for which AI and DL systems may be applicable. Technical and clinical aspects to build a DL system to address those gaps, and the potential challenges for clinical adoption are discussed. AI, ML and DL likely will play a crucial role in clinical ophthalmology practice, with implications for screening, diagnosis and follow up of the major causes of vision impairment, in the setting of the ageing population globally. Introduction With the advent of graphic processing units (GPUs), advances in mathematical models, the availability of big datasets and low cost sensors, artificial intelligence (AI) using machine learning (ML) techniques initially and deep learning (DL) techniques subsequently, has sparked tremendous interest in many industries. These include application of AI in social-media, the internet of things, finance and banking, the automotive industry and healthcare. AI systems can be designed not only for image, speech and motion recognition, but also in natural language processing. In medicine, the most robust AI algorithms have been demonstrated in image-centric specialties, including radiology, dermatology, pathology and increasingly so in ophthalmology. For example, Lakhani et al demonstrated excellent performance in detecting pulmonary tuberculosis from chest radiographs, while Esteva et al was able to differentiate malignant melanoma from benign lesions on skin photographs. In ophthalmology, there have been two major areas in which AI and new DL systems have been applied. First, AI systems have been shown in new studies, including preregistered prospective clinical trials, to accurately detect diabetic retinopathy (DR), 13 glaucoma, age-related macular degeneration (AMD), retinopathy of prematurity (ROP), and refractive error, from digital fundus photographs. A range of cardiovascular risk factors have also been accurately predicted from fundus photographs. Second, several retinal conditions [e.g., neovascular AMD, earlier stages of AMD, and diabetic macular edema (DME)] has also be detected accurately using optical coherence tomography (OCT). There are relatively fewer AI studies using other data, such as studies which show good performance in detecting glaucoma progression from serial Humphrey visual fields (HVFs). However, there are fewer studies that incorporate clinical and imaging data in AL algorithms, and no prospective studies to demonstrate that AI algorithms can predict the development of eye diseases over time. Furthermore, the implementation and adoption of AI into routine clinical care remains extremely challenging. These remain significant goals of AI research in ophthalmology This article describes basic concepts of AI, ML and DL and how such systems might address some of the global burdens created by common eye conditions. Furthermore, the technical and clinical aspects of developing and validating an AI/DL system, potential challenges and future directions are also discussed in this article. Artificial Intelligence, Machine Learning and Deep Learning AI was conceptualized in 1956, after a workshop at Dartmouth College (Figure 1). In the workshop, many AI groups showed promising results in computer learning of checkers strategies, solving word problems in algebra and proving logical theorems. These tasks involved mostly pattern recognition and computational learning. All AI systems were designed to execute and maximise its chance of ‘winning’ within a constructed environment. The term ‘machine learning’ (ML) was subsequently coined by Arthur Samuel in 1959 and stated that “the computer should have the ability to learn using various statistical techniques, without being explicitly programmed”. Using ML, the algorithm can learn and make predictions based on the data that has been fed into the training phase, using either a supervised or unsupervised approach. ML has been widely adopted in applications such as computer vision and predictive analytics using complex mathematical models. In supervised learning, the computer is trained with labelled examples, also known as ground truth, whereas for unsupervised learning, no labelling is required for the algorithm to find its own structure in the input. The majority of AI application in biomedical research uses supervised learning. DL utilizes multiple processing layers to learn representation of data with multiple levels of abstraction. Although some forms of deep neural networks have already been investigated in the past, the advent of graphic processing units (GPU) with improved processing power, larger annotated datasets, and other factors, have recently boosted its diagnostic performance in many domains. Using learning approaches such as backpropagation, a ML or DL system is able to discover intricate structure in large data sets, then changing its internal parameters that are used to compute the representation in each layer from the previous one. These approaches permit the use of regional samples to allow the network to learn to detect biomarkers; furthermore these approaches use complete images, and associate the entire image with a diagnostic output, thereby eliminating the use of “hand-engineered” image features. Given the much improved performance, DL has been widely adopted in image recognition, speech recognition and natural language processing. General Approach in Building a Robust AI system This section explains some common terminologies, software framework, network architectures, datasets selection, assistive vs. autonomous AI system, consideration factors to ensure the robustness of these algorithms (Table 1). In order to build a robust DL system, it is important to have 2 main components – the ‘brain’ (technical networks – Convolutional Neural Network (CNN) and the ‘dictionary’ (the datasets). 1. What is a CNN? A CNN is a deep neural network consisting of a cascade of processing layers that resemble the biological processes of the animal visual cortex. It transforms the input volume into an output volume via a differentiable function. Inspired by Hubel and Weisel, each neuron in the visual cortex will respond to the stimulus that is specific to a region within an image, similar to how the brain neuron would respond to the visual stimuli, that will activate a particular region of the visual space, known as the receptive field. These receptive fields are tiled together to cover the entire visual field. Two classes of cells are found in this region – simple vs complex cells. The simple cells active when they detect edge-like patterns, while the more complex cells activate when they have a larger receptive field and are invariant to the position of the pattern. Broadly, the CNN can be divided into the input, hidden (also known as featureextraction layers) and output layers (Figure 2A). The hidden layers usually consist of convolutional, pooling, fully connected and normalization layers, and the number of hidden layers will differ for different CNNs. The input layer specifies the width, height and the number of channels (usually 3 channels – red, green and blue). The convolutional layer is the core building block of a CNN, transforming the input data by applying a set of filters (also known as kernels) that acts as the feature detectors. The filter will slide over the input image to produce a feature map (as the output). A CNN learns the values of these filters weights on its own during the training process, although the specific parameters such as number of filters, filter size, network architecture still need to be set prior to that. Additional operations called activations (for example ReLU or Rectified Linear Unit) are used after every convolution operation. For pooling, the aim is to reduce the dimensionality of each feature map and make it somewhat spatially invariant, and retain the most important information. Pooling can be divided into different types: maximum, average and minimum. In the case of maximum pooling, the largest element from the rectified feature map will be taken (Figure 2B). The output from the convolutional and pooling layers represent the high-level features of the input image. The purpose of the fully connected layer is to use these high-level features to classify the input image into various classes based on the training dataset. Following which, backpropagation is conducted to compute the network weights and uses the gradient descent to update all filters and parameter values to minimize the output error. T