Towards Reducing Diagnostic Errors with Interpretable Risk Prediction (2024)

Abstract: Many diagnostic errors occur because clinicians cannot easily access relevant information in patient Electronic Health Records (EHRs). In this work we propose a method to use LLMs to identify pieces of evidence in patient EHR data that indicate increased or decreased risk of specific diagnoses; our ultimate aim is to increase access to evidence and reduce diagnostic errors. In particular, we propose a Neural Additive Model to make predictions backed by evidence with individualized risk estimates at time-points where clinicians are still uncertain, aiming to specifically mitigate delays in diagnosis and errors stemming from an incomplete differential. To train such a model, it is necessary to infer temporally fine-grained retrospective labels of eventual “true” diagnoses. We do so with LLMs, to ensure that the input text is from before a confident diagnosis can be made. We use an LLM to retrieve an initial pool of evidence, but then refine this set of evidence according to correlations learned by the model. We conduct an in-depth evaluation of the usefulness of our approach by simulating how it might be used by a clinician to decide between a pre-defined list of differential diagnoses.

## introduction a major source of poor patient outcomes and unnecessary costs in healthcare are missed or delayed diagnoses. a recent report estimated that diagnostic errors result in around 795,000 serious 1 we make our code publicly available for: 1) retrieving evidence and target diagnoses from ehr text in the form of a gym environmenthttps://github.com/dmcinerney/ ehr-diagnosis-env , 2) training agentshttps://github. com/dmcinerney/ehr-diagnosis-agent , and 3) visualizing and annotating predictionshttps://github.com/ dmcinerney/ehr-diagnosis-env-interface . ## dataset we use mimic-iii @xcite , an open-source dataset of ehrs from icu patients. the icu is one of the hospital settings (along with, e.g., the er and radiology) where misdiagnosis or delayed diagnosis are often caused by incomplete information, since clinicians typically do not have enough time to fully examine a patient's ehr. in healthcare, cancer, infection, and vascular dysfunction (termed the "big three") account for about 75% of all mis-diagnosis-related harms @xcite . within the icu, the latter two categories mostly manifest as pneumonia, and pulmonary edema (which in this paper we treat as interchangeable with congestive heart failure). for this reason, we will focus on predicting the risk of icu patients for cancer, pneumonia, and pulmonary edema. these are also conditions for which clinical correlation with notes from the past ehr is important for diagnosis. we use all patients in the mimic dataset so that we have both negative and positive examples of the conditions. we include additional details regarding the dataset and preprocessing in appendix section a. ## an interpretable risk prediction model we propose a multi-stage approach to risk prediction, capitalizing on a modern llm, flan-t5-xxl @xcite in this case, to implement each of the following steps. retrieval (section 3.1). we generate abstractive evidence from free text notes by prompting an llm with appropriate queries. the evidence snippets provide a form of interpretability, in that they can be inspected directly to verify predictions. ## certain diagnosis extraction we make an assumption about the ehr of patients that eventually receive a diagnosis that there is some period of time in the record where a diagnosis is "uncertain" before it becomes "certain", and the eventual "certain" diagnosis is correct. of course just because a diagnosis is definitive as noted by clinician in the record does not necessarily mean that it is correct-sometimes clinicians are wrong. however, it is hard to detect such cases, so here we focus on reducing delayed diagnosis errors where we assume some evidence in the medical record from that "uncertain" period could have influenced a clinician to make a diagnosis or order a certain kind of test sooner than they did, or keep a diagnosis in the running list of differentials for longer. if notes are incorporated into the input where the diagnosis is already certain, the prediction problem becomes too easy, which is why a time-wise fine-grained label is necessary-such a label could more accurately weed out all of this obvious evidence. to extract these certain diagnoses with an llm, we use three sequential prompts and a normalization step. ## evaluation because our targets are synthetically generated using an lm, we first evaluate how well our labels align with the "ground truth" (section 5.1). next, we aim to evaluate how well the model can realistically help with risk prediction. though it is straightforward to assess the accuracy of the risk prediction itself-we use the standard metrics of precision, recall, f1 and auroc scores to compare to various uninterpretable baselines-it is not as easy to assess what we really care about: how helpful is the interpretability offered by the proposed model to clinicians (section 5.2)? for this we resort to manual evaluation by our clinical coauthors and develop bespoke interfaces to facilitate annotation. ## results and discussion the majority of our results are based on annotations from 4 annotators on 24 instances and 3 models. each instance has a maximum of 3 annotators, each annotating different models (assigned randomly). our main goal is to understand if our approach can retrieve better evidence. to this end, we plot the percentage of evidence annotated in each category of usefulness for each model in figure 3 . though we record usefulness for each condition individually, here we combine these annotations by taking the maximum score across the conditions for each piece of evidence. to identify hallucinated evidence, we conducted post-hoc annotations with only the annotated llm-generated evidence that was abstractive (42 of 108). 6 the results highlight the necessity of both the "llm evidence" retrieval component and the "log odds sorting" method, as both other variants retrieve significantly less "useful" and "very useful" evidence and more "weakly correlated" and "not relevant" evidence. we also find a relatively small number of hallucinations (5) and note where the hallucinated evidence was originally ranked in compared to the previous tracing atrial fibrillation is seen. other findings are similar. the patient is at risk of pulmonary edema." in this case the annotator changed their estimate of the likelihood of pulmonary edema from unlikely to somewhat likely, and it turns out that pulmonary edema did appear in a future report. we show all 7 instances where annotators changed their mind after viewing evidence in appendix given that we are using synthetic labels of future diagnoses for both training and evaluation for risk prediction (discussed next), it is important to evaluate how well our labels align with ground truth. given that icd codes are not fine-grained enough and are not always accurate, we turn to manual annotations of precision for this evaluation. in figure 5 , we report the precision of these labels for being correct or for being "correct and on time". this second category is a stronger correctness in which the annotator also noted that the note where the label was detected subjectively seems to be the first note where that label should have been given as judged using the phrasing in the note. 7 we see reasonable precision when using automatic labeling with the llm pipeline (about 80 percent and above for all conditions). we also compute inter-annotator agreement for these annotations of precision across the 4 annotators by enforcing that 8 annotated predictions overlap for all the annotators. the fleiss' kappa score for these synthetic label annotations was .68 for the 3-category classification shown in figure 5 and .86 for the 2-category classification obtained by simplifying the labels into just "correct" or "incorrect". we would also like to assess how well our models' risk estimates aligns with the intuitions of clinicians with respect to the aggregated and individual predictions. though for the aggregated prediction for an instance, we ask annotators to take the magnitude of the risk, not just the direction (i.e. increased compared to baseline or decreased compared to baseline) into account, for evidence-level predictions, we ask annotators to take the magnitude with a grain of salt and mostly judge based on the direction. this is because the magnitudes appeared to be somewhat artificially inflated potentially either due to the strong evidence trying to "compensate" for the evidence that does not actively contribute to the log odds (see figure 12 ) or because of the sorting method. 8 figure 6 shows that both models do reasonably well with respect to the aggregated and evidence-wise predictions, and both do slightly better on evidence-wise as opposed to aggregated predictions. finally, it is important to evaluate the actual prediction performance of our models on our synthetic labels. here we also compare against baseline models that are not interpretable: bert and longformer. these black-box models are trained on both the all ehr and the concatenated retrieved llm evidence. figure 7 shows that including all evidence usually helps prediction performance, but using the blackbox vs interpretable models on the same input does not effect performance. ## conclusions clinicians should have access to all the pertinent information to make well-grounded decisions for diagnosing a patient, but currently they are inundated with (unstructured) information from the ehr. this is exacerbated by the time constraints faced by practitioners. we have proposed an approach that aims to facilitate efficient access to potentially important data within ehr; our method capitalizes on the capabilities of llms to produce digestable, abstractively generated text evidence, which is then consumed by a neural additive model (nam) to yield a prediction. we find that using nams does not sacrifice predictive quality, but does enable models to surface useful evidence to clinicians. using the llm to create the starting set of evidence to feed into the nam does sacrifice some performance, but it also significantly increases the usefulness of the evidence in comparison with using the raw sentences from ehr notes as evidence. further, we find that in some cases the surfaced evidence is able to change a clinician's mind, increasing the clinician's recall though decreasing precision, which warrants future work to improve on this system. one major concern is that this type of system could increase clinician's workload rather than decrease it. future work should assess exactly how and when it might be beneficial to show snippets to clinicians.