DAVE : Differential Diagnostic Analysis Automation and Visualization from Clinical Notes (2023)

Abstract: The Differential Analysis Visualizer for Electronic Medical Records (DAVE) is a tool that utilizes natural language processing and machine learning to help visualize diagnostic algorithms in real-time to help support medical professionals in their clinical decision-making process

## introduction information reported in electronic medical records (emrs) revolutionized medical language research. healthcare providers follow specific procedures in the process of caring for and managing a patient. the clinical assessment starts by noting the chief complaints of the patient and the purpose of the visit. this step is followed by a review of family history and symptoms. the healthcare providers document the aforementioned information in clinical notes embedded in an emr. they proceed with differential analysis leading to a diagnosis of the case on hand and a declaration of the future actions to be taken. the adequate decision comes from following a specific set of evidence-based diagnostic algorithms that medical professionals learn during their education and training. these algorithms are decision diagrams whose top nodes are labeled with markers and identifiers that occur typically in the chief complaints. the algorithms lead the healthcare providers with a series of choice nodes reaching to the leaves which typically dictate the diagnosis and the treatment plan. intermediary nodes in the algorithms describe lab tests, medication prescriptions, special treatments, and life-style changes among other actions. the healthcare providers select the algorithms to follow after assessing the situation from discussions with the patient, initial clinical tests and documentation in the clinical notes and emrs. they select and follow the algorithm that best matches the encountered case. they traverse the decision diagram of the algorithm, come up with an adequate diagnosis, and declare further actions to be taken. the healthcare provider documents the information collecting during the encounter with the patient in a clinical note summarizing the process from the initial complaint to the actual outcomes. our aim is to assist the healthcare professionals in the process of coming up with adequate diagnosis by filtering out the top three diagnostic algorithms matching their notes and visualizing them. in this paper, we present dave (differential analysis visualizer for emrs), a system that automates the selection and visualization process of the diagnostic algorithms. dave alleviates healthcare providers from the tedious tasks that require remembering and visualizing the graphs. it also simplifies the decision-making process involved in the diagnostic differential analysis (dda) phase. dda consists of analyzing in real time the input notes and selecting the most suitable algorithms. we developed and fine-tuned dave via training on pre-existing curated datasets. we leveraged an annotated corpora of electronic medical records collected from aubmc and the hariri medical center in beirut, lebanon. the corpora consist of 151,930 total medical notes focused on family medicine. all the notes are annotated with diagnostic codes and treatment plans that annotate the whole record. a subset of 3,616 of the notes are richly annotated with textual annotations referring to textual elements in the note itself. dave also leverages digitized clinical diagnostic algorithms extracted from medical books: "the patient history: evidence-based approach" @xcite and "symptom to diagnosis: an evidence-based guide" @xcite . these textbook algorithms represent the steps that need to be taken to determine and treat a specific healthcare condition. our method involves crossdocument analysis based on cross-referencing electronic medical record entities. this aims at extracting diagnosis indicators from the notes using natural language processing (nlp) and computational linguistics (cl). we presented dave to a number of medical professionals, physicians, and medical it experts and received overwhelmingly positive feedback. we demonstrated the project to three physicians and interviewed them about usability and functionality. after using dave, they provided positive feedback and emphasized its usefulness, particularly for students, nurses, and young physicians. we also interviewed the head of it at aubmc, a wellestablished medical center, and her team. she also praised the software and encouraged its integration within the medical center for professional use. ## related work information extraction (ie) from emr has become a crucial tool to progress medical and clinical practice research since the emergence of digital records. the understanding gained from this data has a significant positive impact on current medical research. for this reason , a number of studies and initiatives have looked into the best ways of extracting data from medical records. rule-based algorithms, machine learning (ml) models and keyword-based search are the major methodologies used in ie from emrs, with rule-based models being the most accurate but most time consuming to build. this is due to the fact that the rules express accurately the direct knowledge and experience of healthcare professionals @xcite . several reviews discussed the different approaches used for ie in the medical field @xcite @xcite . dave will be built using ml as its core. it will mimic the behavior of rule-based methods by capturing insight from clinical diagnostic books. it avoids the cost of rule-based approaches since healthcare professionals will not have to manually provide rules. the work (b.sharafeddin, 2020) introduced a model for the automation of the process of diagnostic extraction from clinical notes. it matches unstructured de-identified medical notes to medical diagnostic algorithms using a cross-document analysis method. the model uses the diagnostic algorithms @xcite ) from medical books to build bayesian networks corresponding to every diagnosis case. in these networks, nodes are interrelated by informational dependencies.in other words, each node is given a conditional probability depending on the probability of its parent @xcite . afterwards, it calculates the distributional similarity for the words in the electronic medical notes and united states medical licensing examination (usmle) questions using disco (extracting distributionally related words using co-occurrences) @xcite . this method allows the extraction of diagnosis indicators by retrieving the semantic similarity between words and phrases in large text fields to create sets of similar words for every word (b. @xcite . in order to compute semantic similarity, we also used disco, also known as the kolb (after peter kolb) vector similarity. disco retrieves the most semantically similar words for an input word @xcite . it is accompanied with the disco builder tool (dbuilder). the latter creates a database of contextually similar words, given a text corpus. prior to the actual build, dbuilder recommends configuring the corpus to the given format to ensure the best possible outcome. dbuilder accepts lemmatized or tokenized input in single or multiple files. larger contexts (or paragraphs as referred to by the documentation) should be specified using tags. dbuilder takes care of excluding stop-words from the context as they can contribute to noise. it is also important to specify the size of the context each word should be taken in when configuring the builder instance. @xcite . giving @xmath0. subsequently, dbuilder creates a matrix of co-occurrences in which each row describes a specific word. the size of the resulting matrix would be n × m × r where n denotes the number of words in the corpus, m denotes the number of words used as features and r is the window size. equation 1 is used to provide meaningful weights for the features in which w, and w ′ stand for words, r stands for the window size, -within functional parameters stands for dependency relation and f is the frequency of occurrence. the information theory based lin's measure is then used for the comparison of every word vector with all other word vectors to create a distributional similarity scheme between all the words @xcite . negated words present in the medical notes lead to faulty results when matching them to the corresponding diagnostic algorithm in our work. the context algorithm provides an approach to deal with negated words by employing a specific scope of its trigger terms. once medical conditions are indexed, the algorithm assigns three contextual properties to each condition being (i) negation, (ii) temporality and (iii) experiencer. negation can be either negated or affirmed. temporality can be recent, historical, or hypothetical. finally, experiencer can be either patient or other. a set of trigger words is assigned to each non-default status @xcite . the status of the contextual properties is updated when the indexed medical condition term falls within the scope of one of the trigger terms. moreover, the algorithm contains pseudo-trigger words for each non-default status of the contextual properties. the scope of the trigger or pseudotrigger terms includes all the clinical conditions following them till the termination word or till the end of the sentence if no termination word was present. the termination words are assembled in the algorithm following conceptual groups. in other words, the algorithm takes as input the sentence, indexes the clinical conditions, and assigns each of the contextual properties a default value. then, it marks all the trigger, pseudo-trigger and termination words and iterates through them to determine the scope of each trigger term and update the corresponding contextual property of the indexed medical case. ## methods this section discusses the preprocessing of emr notes, building and utilising disco models, aggregating scores from different models, capturing user input, and visualizing the graphs. ## user study and feedback during its development phases, dr. lama sharafeddine and dr. rabiaa algeboury provided valuable feedback on dave, praising its usability and effectiveness in its final version. as medical professionals themselves, they recommended the tool as a learning aid for up-and-coming physicians and a support tool for note completion during patient visits, as well as for the visualization of more complex cases. additionally, they expressed interest in training dave with notes and algorithms in specialty medicine so that medical doctor residents may also benefit from its use. we conducted two sets of interviews with medical it experts, specifically the head of it of aubmc, ms. rola antoun, and her team. in the first round, we presented the idea of dave and our preliminary implementation plans. they commended our design and helped us brainstorm potential use cases among physicians. they also provided us with insight on what to avoid so that physicians don't refrain from using the software. for example, because it appears complex or requires any additional load like navigating through multiple pages and pressing multiple buttons. finally, they showcased a few of the more popular software tools and applications among physicians, in hopes of providing further inspiration for dave. after completing the project, we met for a second round to demonstrate our work and get usability feedback. they praised the project, were fascinated by the results, and thought it was mature enough to be deployed in the ehr of the medical center for professional use. furthermore, we had three physicians try out the software and discuss its usability and practicality. we first asked them if this software would benefit them. two of three said they find the visualization very helpful and beneficiary and would assist the complex task of differential analysis visualization in memory. they also thought that expanding beyond the top three matches is helpful. when asked about user-friendliness, all physicians found the user experience to be simple, fast, and non-tasking. finally, we received some ideas to make dave better. for example, adding specialty diagnostic books to target specific areas of medicine would make the software more useful for more experienced physicians. ## results dave's model presented successful and promising results. considering the difficulty and challenges of nlp in the medical field and case detection algorithms in general, dave's model achieved a fairly high accuracy while maintaining a small computation overhead since the results are required to be displayed instantaneously. it also has no problem supporting large text and does not require gpu resources. the best version of the model with all the processing and word augmentations achieved a 74.3% rate of displaying a related diagnosis in the top three scored algorithms. we opted for precision at the best three results to eliminate any chance of bias from headache and fatigue diagnoses which are very general making their symptoms usually present in all diagnoses. this accuracy increased significantly with the addition of pubmed medical corpora as shown in to conclude, dave achieved accurate results and was deemed user-friendly and very convenient for professional use. it offloads a tedious task from physicians and supports their decision-making during patient visits. it was also regarded as ready for professional use by medical professionals and medical it experts. several medical information outlets such as hospitals, and medical insurance companies provide text to diagnosis portals. however, they do not attempt to fully automate the differential diagnosis process. up to our knowledge no dave alternative systems exist so that we can perform a one to one comparison. ## limitations the main limitation of dave is testing it in real time situations. since the involvement of physicians is required to test dave in different notetaking stages, exhaustive realtime testing proved to be difficult. dave's objective is to guide and support the physician during the diagnosis process, hence we are interested in the accuracy of the program during different stages of completion of the clinical note. we tested dave with volunteer physicians, however, we consider the sample tests as initial and we think that full deployment requires more systematic testing. ## conclusion this paper details a novel model that can suggest diagnostic algorithms to medical professionals based on their clinical notes in real-time. the model uses directed acyclic graphs and semantic similarity metrics to rank diagnostic algorithm graphs taken from digitized medical textbooks and suggests the top three for consideration. the model achieves 74.3% success rate and high acceptance for usability. this model is a significant step forward in improving the accuracy and efficiency of the differential analysis process, which is crucial in making timely and accurate diagnoses and developing effective treatment plans for patients. you can find dave online: www.davemr.com . the following is a dave system demonstration video. you could also find dave's source code on github