Is the future of medical diagnosis in computer algorithms?
Authors: Karl Gruber
Journal: The Lancet Digital Health (2019)

In a recent study, Chinese and American researchers developed an artificial intelligence (AI) system using 1·3 million electronic health records from across China. Their system was capable of diagnosing diseases as accurately as an experienced paediatrician. This was an amazing feat, considering the challenges involved. The human body is a complex piece of biological machinery and trying to diagnose our ailments is a monumental task, even for the most experienced doctors. To improve their diagnoses, doctors gather as much information as they can from their patients. This information comes from a wide range of sources, including genome-wide studies, demographics, doctor's notes, clinical images, laboratory results, genetic tests, or even from wearable sensors. Today, databases of electronic health records from around the world hold such information for billions of patients. But, despite all the data and the best intentions, misdiagnoses happen more often than you might think. “Clinicians think and act in chaotic, time-pressured, clinical environments while dealing with uncertainty, which is integral to the diagnostic process”, Dr Hardeep Singh, a patient safety researcher at the Michael E DeBakey VA Medical Center and Baylor College of Medicine, in Houston, TX, USA, tells The Lancet Digital Health. All this pressure in the fast-paced primary care environment can translate into diagnostic errors, especially when a doctor runs into problems gathering or interpreting information related to patient history, physical examinations, or tests, Singh explains. He adds, “Things are not always black-and-white when patients' symptoms and disease conditions are unfolding and dynamically evolving over time. Now add to this the currently immature and disconnected data information systems and you get the perfect storm of risk”. A 2014 study led by Singh found that at least one in 20 adults in the USA left the doctor's office with a misdiagnosis, which equates to 12 million people per year. Half of these misdiagnoses, the authors estimated, are potentially harmful. One way to help physicians improve their diagnostic accuracy is by helping them analyse data more efficiently. With this goal in mind, and as computers become more powerful, scientists are looking at AI to design systems that can think like humans, only faster and more efficiently. AI approaches, such as machine learning, natural language processing, representation learning, and deep learning, aim to identify patterns in words, images, voice and video recordings, and other data in record time. To develop an AI system to diagnose paediatric diseases, the Chinese and American team first created a dictionary with key words linking symptoms to a disease. This dictionary, which holds more than 1 million such key terms, was then used to train the AI system. “You can imagine that this requires both a huge dataset and a lot of human physician manpower to curate data—but once it is made, it can save a huge amount of work and has enormous impact to health-care delivery”, Prof Kang Zhang, lead author of the study, at the University of California, San Diego, CA, USA, tells The Lancet Digital Health. This AI system was then used to analyse plain language inputs that would refer to symptoms with the goal of finding patterns that linked to a disease. To test their system, Zhang and his team used data from retrospective electronic medical records. “We tested diagnoses in 50 000 patients' EMR data as a validated cohort and additional 10 000 patients were tested when we compared accuracy of AI vs physicians” Zhang explains. The results were nothing short of amazing, with a diagnostic accuracy of 95% for common ailments such as acute upper respiratory infection and sinusitis. The key to their success was in the amount and quality of the data available to train the AI algorithm and in the target population in whom it was tested; and here lays a potential limitation of this AI system. “AI can only work with diseases it has seen and trained before” Zhang says. So, the more data available for training, the more accurate the prediction the AI system can make, Zhang explains. So far, the AI system designed by Zhang and his teams remains a proof of concept. A promising research experiment, not yet in use with actual patients. But, there are other AI systems that are already interacting with patients, such as IBM's Watson or Babylon's AI chatbot. Babylon's AI system, for example, is currently being used by over 3 000 000 users from the UK and Rwanda. The system provides quick and easy-to-access advice for common ailments. But, how accurate is this advice? In a non-peer-reviewed 2018 study, Babylon researchers compared the performance of human physicians with Babylon's AI system at evaluating identical medical cases. According to their results, the AI system outperformed the average human doctor. However, in a 2018 correspondence in The Lancet, Hamish Fraser, Associate Professor of Medical Science at Brown University, Providence, RI, USA, argued that Babylon's AI system still needed more testing before being able to prove it can perform well in real-world settings. For instance, Babylon's study did not use real patient data, but so-called patient vignettes, clinical examples of patient-related cases that are normally used for educational purposes. Other issues with Babylon's study, Fraser says, involves data input into the AI system, which was done by doctors, not patients (lay users), and that few statistical tests were done to assess the significance of their findings. “The biggest problem at present with diagnosis programs for patients is the lack of rigorous, clinical evaluation. The studies to date have almost all been small and have not used real patient data so we don't know if a system like Babylon performs well with typical patients presenting with a wide range of clinical problems”, Fraser says. Saurabh Johri, Chief Scientific Officer at Babylon, commented on this lack of rigorous clinical evaluations and says that studies are on their way. “We are keen to run robust, real-world studies on Babylon's AI that can be published in peer-reviewed journals”, he says. According to Johri, Babylon is currently working with academics to validate their AI technology. He added, “…we know new technology needs high quality evidence to earn people's trust”. The question in people's mind is will doctors ever be replaced by computers? Experts agree that this is unlikely, at least under most circumstances. AI systems could be particularly useful in resource-limited areas or in emergency situations when a human doctor does not have the time or resources to reach a diagnosis. “I would envision AI can be used as a primary deployment to diagnose and triage diseases in rural or resource poor areas, for example, as an online system”, Zhang says. But even under these circumstances, Zhang argues that human doctors should still be involved and that “AI should be used primarily as a physician assistant to reduce overall burdens of physicians and it should not replace physicians”. Taken together, the use of AI in health care heralds an exciting and promising new era in diagnosis, in which meaningful medical guidance can be made available to more people in need.