CLD - MEC at MEDIQA - CORR 2024 Task: GPT -4 Multi-Stage Clinical Chain of Thought Prompting for Medical Errors Detection and Correction (2024)

Abstract: This paper demonstrates CLD-MEC team submission to the MEDIQA-CORR 2024 shared task for identifying and correcting medical errors from clinical notes. We developed a framework to track two main types of medical errors: diagnostics and medical management-related errors. The tracking framework is implied utilizing a GPT-4 multi-stage prompting-based pipeline that ends with the three downstream tasks: classification of medical error existence (Task 1), identification of error location (Task 2), and correction error (Task 3). Throughout the pipeline, we employed clinical Chain of Thought (CoT) and Chain-of-Verification (CoVe) techniques to mitigate the hallucination and enforce the clinical context learning. The model performance is acceptable, given it is based on zero-shot learning. In addition, we developed a RAG system injected with clinical practice guidelines as an external knowledge datastore. Our RAG is based on the Bio_ClinicalBERT as a vector embedding model. However, our RAG system failed to get the desired results. We proposed recommendations to be investigated in future research work to overcome the limitations of our approach.

## introduction medical errors identification and handling in clinical practice is paramount to ensuring optimized patient safety and efficient healthcare delivery. yet, tracking medical errors is complex to achieve. the umbrella of medical error instances is wide, along with the different phases of the patient journey from assessment to diagnosis, followed by medical management. in addition to the various medical error types that occur throughout these phases. the most common types are diagnostics errors and clinical management errors. understanding the patient journey-related phases and types of medical errors is crucial in modeling a tracking system framework. clinical notes of electronic health records (ehr) are considered as documented references of the entire patient's medical journey, from the first point of care to post-medical care plan follow-up. developing technologies that work on processing clinical notes and notifying healthcare providers with realtime medical error signaling and correction will move healthcare to its next level with a new leveraged paradigm in patient safety. mediqa-corr 2024 shared tasks covers three tasks related to medical error detection and correction from clinical notes @xcite . this competition is to establish state-of-the-art techniques (sota) to formulate a reliable clinical task of this kind. in this paper, we demonstrate our participation in addressing these three challenges. the complexity of designing a tool that addresses all incidents of medical errors from clinical notes comes from the variety of clinical notes' architecture/context. this variation occurs across: considering the stage of patient care and the level of clinical note details when mapping case scenarios of medical error incidents within the solution framework is a functional step in building a sustainable medical error tracking system. for instance, 1 537 detailed documentation of clinical findings and diagnostic tests may facilitate the identification of diagnostic errors during the assessment or diagnosis stage. on the other hand, comprehensive documentation of treatment plans and medication history may aid in identifying medical management errors during the medical management and planning phase. our approach to handling this challenge involves designing a framework that consistently addresses the most common two types of medical errors: diagnostics and medical management errors. to detect these types, we propose a tracking algorithm based on classifying the context of clinical notes to map them with their related medical error case scenarios, ending with detecting medical error incidents of one of these two types. to formulate this in the framework, each clinical note should be screened for three case scenarios of medical error incidents. we categorize clinical notes into two levels of contextual architecture. level 1 (l1) addresses the first case scenario of medical error. while level two (l2) helps us track the second and third case scenarios of medical error instances. the details of these case scenarios and clinical note levels are demonstrated in section 5. we implied our tacking approach in a gpt-4 multistage prompting-based pipeline that ends with the three downstream tasks: classification of medical error existence (task 1), identification of error location (task 2), and correction of error (task 3). the pipeline is composed of four main stages, as illustrated in figure 1 . throughout the pipeline, we applied clinical chain of thought (cot) and chainof-verification (cove) to mitigate the hallucination and enforce the model to reference its reasoning rational response according to its evidence-based medicine (ebm) clinical practice guidelines attributed knowledge of gpt-4 acquired during training. as a side work, we developed a rag system injected with clinical practice guidelines as an external knowledge datastore. large language models (llms) have proven their potential in various domains, including finance, marketing, and education. healthcare is a wide area with many horizons (medical education, translational medicine, clinical practice, domain-specific clinical specialty), and the efficiency of llms varies within each horizon area. in some instances, pretrained language models (plms) show an efficient performance on specific basic nlp clinical tasks such as named entity recognition (ner), classification, and relationship extraction (re). however, efficient performance is yet to be reliable and implemented on generative advanced clinical nlp tasks, including clinical text generation, medical question answering, and clinical text summarization. thus, there is a considerable area for optimizing and leveraging the state-of-the-art in the area of applied generative clinical nlp. hallucination and out-of-source generation are some of the main limitations that llms and plms face, especially with up-to-date and niche-focused domain-related tasks. rag and cot are leading techniques that have been shown to mitigate the limitations mentioned above @xcite . rag framework works on optimizing the output of llms by appending llms with an external up-todate knowledge/ data store to be attributed/injected in the generative process through a retrieval and query process @xcite . rag contextualizes the model to be more aligned with domain-specific downstream tasks, ending with a more accurate, customized, and specific evidencegrounded response with its data source to be more valid. it encompasses 3 main components: the retrieval, the generation, and the augmentation techniques. pretrained llms performance is comparable with llms with rag. rag can overcome the need to retrain/finetune llms on up-to-date or domain-specific information. instead, it augments the knowledge with llms without retraining the model and results with applicable performance (gupta et al., 2024). rag with llms has shown its potential to drastically advance llms' usability and reliability. in the healthcare domain, integrating rag with llms has been applied with notable enhancements in the generated responses of llms to make them more accurate, informative, and reliable. llms output aligned remarkably with the augmented rag case-specific medical knowledge. (zakka et al., 2024; ge et al., 2023) incorporated rag into llms with a medical knowledge database for medical guidelines and treatment recommendations. these llms with rag outperform standard llms significantly on the level of accuracy, user satisfaction and consistency. another study illuminates the impact of appending clinical trials related 2 538 to medical knowledge to llm with rag on an exceeding performance of this framework compared to experts in clinical trial screening @xcite . from the perspective of our shared task, medical error correction is one of its downstream tasks. which needs for techniques to support and enhance formulating this task. factual error detection and correction with evidence retrieved (fleek) @xcite , is an innovative solution that overcomes hallucinations. it performs two tasks: fact verification and fact revision. it splits an input passage into sentences and uses a sequential pipeline to verify each sentence and correct it so it reduces hallucinations with unstructured knowledge, such as web-based and structured knowledge graphs. facts are defined as units of information that describe entities, relations, or events and are represented using a semi-structured triple format. fleek's performance is evaluated using benchmarks and preliminary experiments using manually created evaluation data. @xcite introduced chain-of-verification (cove), a method to reduce hallucinations in large language models by breaking down verifications into more straightforward questions and self-correcting them. factored cove helps alleviate copying hallucinations and provides performance gains over original responses. for accurate clinical diagnosis, @xcite explored llms in medicine to imitate the clinical reasoning rationale (crr) as a cot approach to perform differential diagnosis steps during the medical diagnosis process. they created a diagnostic prompting method that allows llms to construct diagnosis while accurately mimicking clinical reasoning using cot prompts. this led to gpt-4 being prompted to imitate the thought processes of clinicians, giving doctors a comprehensible justification for assessing the precision of llm replies. these techniques are utilised in order to enhance our model's ability to identify medical errors in the clinical context and consider potential corrections. rred @xcite is a deep learning framework designed to detect errors in radiology reports. the system creates artificial existing errors using an error generator and supervised learning techniques. the method addresses error detection in radiology reports using a deep learning framework with a rich contextual and medical understanding. the error generator generates realistic errors from existing radiology reports, creating synthesized datasets for training the error detector. the error detector employs a bert-based architecture to detect errors based on a semantic understanding of radiology reports. ## description of the three tasks in this research, we worked on three tasks; the description of each task is as follows: binary classification (detecting medical errors): in this task, we had to determine whether the text contained a medical error or not. this task involved binary classification (0/1) regarding the existence of the medical error in the text. ## methods we build a gpt-4 prompting-based pipeline that processes the text of clinical notes and endeavors to detect medical errors and correct them if they exist. the model pipeline addresses medical error incidents using three types of clinical note context case scenarios. these case scenarios are classi-4 540 fied according to the types of medical errors we aim to track, including diagnostic and clinical management errors. diagnostic errors occur when a patient's medical condition is attributed to an incorrect cause (pathogen, poison, etc.) or misdiagnosis. while medical management types of errors involve an incorrect medication, lab test, procedure, or medical image for a patient's medical condition. these errors occur due to an incorrect interpretation of findings to the correct cause and diagnosis or an incorrect interpretation of the cause and diagnosis to the most appropriate medical management. these consequences of incorrect interpretations that lead to medical errors are addressed through our solving approach, which is systematically employed throughout the pipeline to track incidents of these types of medical errors. in a brief overview, the pipeline involves removing the cause or diagnosis from the clinical note and subsequently generating a cot process. this process is tailored to find the most probable cause, findings, and clinical management of a patient's clinical case. then, assess for discrepancy between the generated cot and original clinical note, thus indicating if an error exists and correcting it. the output of each prompt is formulated to be unified across all dataset. the pipeline is composed of four main processing phases. in the following subsec-5 541 tions 5.1, 5.2, 5.3, 5.4, we elucidate these phases with the details of the techniques used within each phase. clinical notes can be articulated in various architectures, depending on the timeline and stages of the patient's medical journey. we categorized clinical notes into two levels of contextual architecture. level one (l1) addresses the first case scenario of medical error, while level two (l2) helps us track the second and third case scenarios of medical error instances. this facilitates formulating a clear link between the context of clinical note and the most suspected medial error case scenario to be signaled as the following: l1: documented hpi or pmh without clinical management actions related to the cause or the diagnosis. at this level, the clinical management would be directly moved to detect the suspected cause and diagnosis of the medical condition upon the existing clinical findings without thinking of any further needed medical management actions to confirm the diagnosis or to manage the current patient's medical status. l2: documented hpi or pmh with clinical management actions related to the cause or the diagnosis. at this level, the patient is already known with the diagnosis and related cause. a wider margin of settings. this level will track the second and third case scenarios of medical error instances. medical errors at this level mostly will be related to signal the second case scenario. the third case scenario is related to the previous step of the patient medical journey, at the diagnosis step. specifically when the diagnosis and clinical findings are not directly connected to each other in most common clinical contexts, yet the note contains a clinical management action related to the incorrect patient's diagnosis. refer to figure 2 of appendix b that demonstrates an example of each case scenario. ## experiments and results here, we report the experimental findings demonstrating our model's effectiveness on the shared tasks. our model was performed on the three tasks utilizing a zero-shot learning approach and a gpt-4 prompting-based pipeline with cot and cove methods and structured output. since our approach mainly focuses on zero-shot learning. thus, we only used the training and validation for prompt optimization until we reached a reasonable output in tracking the three case scenarios. for a comprehensive show of our approach's functionality in tracking the three case scenarios, please refer to appendix b. it provides an example experiment of tracking each one of the case scenarios, illustrating its input/output at each prompt through the entire process. the function calling feature of openai api and json mode is utilized to get the aimed structured format and ensure consistent output throughout the dataset. the results of our approach performance on the testing dataset show that the accuracy of the first and second tasks is 0.566 and 0.49, respectively, without using external knowledge sources, finetuning methods, or group learning. while for task 3 and the main results, performance metrics yielded the following scores: rouge-1-f of 0.427, bertscore of 0.48, bleurt of 0.53, their aggregate-score (mean of rouge-1-f, bertscore, bleurt-20) of 0.48, and their composite scores of 0.34. these metrics assess the model's ability to produce contextually appropriate corrections for clinical errors identified in clinical text. the performance of our model is somehow acceptable but unreliable when applied to the testing dataset. the algorithmic approach of following three case scenarios of medical events based on clinical note contextual architecture might undertrack other medical error incidents case scenarios. for the llm we have used, gpt-4, we have aimed to be built based on a rag framework incorporated with clinical practice guidelines. our hypothesis was to optimize the output of un pretrained generative model in the arena of clinical practice (niche-focused) to get reliable, inferential and ground truth knowledge without hallucinations. rag framework is the best to be employed with a massive llm such as gpt, bart, or t5. for limited hardware resources, we took gpt-4, as an open llm model. our rag system failed in retrieving relevant queries. it was supposed to be connected to our pipeline at phase 4, "verify crr-cot with the ebm", but for irrelevant retrieved chunks, we continued the work without it. this shortcoming performance could be one of the following: along the pipeline, we used crr-cot and cove as prompt optimization techniques. future work should investigate the performance of public llms, to unlock their known capabilities for these downstream tasks within hardware accessible facilities. in addition to exploring advanced chunking strategies such as semantic chunking and finetuning a domain-specific model such as bio_clinicalbert for retrieval. ## limitations our work is limited by two points. the first one is our inability to produce a reliable rag system due to time constraints. we could not explore how our approach would perform with a successful rag system implementation; specifically, we utilized rich, niche-focused external knowledge to boost the reliability and applicability of the generated output. secondly, with our limited computing and financial capacity, we would not be able to experiment with other massive llms, whether they are general, clinically fine-tuned, or pre-trained ones. asma ben abacha, wen wai yim, velvin fu, zhaoyi sun, meliha yetisgen, fei xia, and thomas lin. zeming chen, alejandro hernández, cano angelika, romanou antoine, kyle matoba, francesco salvi, matteo pagliardini, simin fan, andreas köpf, amirkeivan mohtashami, alexandre sallinen, alireza sakhaeirad, and vinitra swamy. 2023. med-itron -70b: scaling medical pretraining for large language models. arxiv preprint arxiv:2311.16079. shehzaad dhuliawala, mojtaba komeili, jing xu, roberta raileanu, xian li, asli celikyilmaz, and jason weston. 2023. chain-of-verification reduces hallucination in large language models. arxiv preprint arxiv:2309.11495. jin ge, steve sun, joseph owens, victor galvez, oksana gologorskaya, jennifer c lai, mark j pletcher, and ki lai. 2023. development of a liver diseasespecific large language model chat interface using retrieval augmented generation. medrxiv. aman gupta, anup shirgaonkar, angels de luis balaguer, bruno silva, daniel holstein, dawei li, jennifer marsman, leonardo o nunes, mahsa rouzbahman, and morris sharp. 2024. rag vs fine-tuning: pipelines, tradeoffs, and a case study on agriculture. arxiv preprint arxiv:2401.08406. omar khattab and matei zaharia. 2020. colbert : efficient and effective passage search via contextualized late interaction over bert. in proceedings of the 43rd international acm sigir conference on research and development in information retrieval, pages 39-48. dabin min, kaeun kim, jong hyuk lee, yisak kim, and chang min park. 2022. rred: a radiology report error detector based on deep learning framework. in proceedings of the 4th clinical natural language processing workshop, pages 41-52. thomas savage, ashwin nayak, robert gallo, ekanath rangan, and jonathan h chen. 2024. diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. npj digital medicine, 7(1):20. kurt shuster, spencer poff, moya chen, douwe kiela, and jason weston. 2021. retrieval augmentation reduces hallucination in conversation. in findings of the association for computational linguistics: emnlp 2021, pages 3784-3803. s m towhidul islam tonmoy, s m mehedi zaman, vinija jain, anku rani, vipula rawte, aman chadha, and amitava das. 2024. a comprehensive survey of hallucination mitigation techniques in large language models. arxiv e-prints, pages arxiv-2401. ozan unlu, jiyeon shin, charlotte j mailly, michael f oates, michela r tucci, matthew varugheese, kavishwar wagholikar, fei wang, benjamin m scirica, and alexander j blood. 2024. retrieval augmented generation enabled generative pre-trained transformer 4 (gpt-4) performance for clinical trial screening. medrxiv, pages 2002-2024. cyril zakka, rohan shad, akash chaurasia, alex r dalal, jennifer l kim, michael moor, robyn fong, curran phillips, kevin alexander, and euan ashley. 2024. almanac-retrieval-augmented language models for clinical medicine. nejm ai, 1(2):aioa2300068. ## 550 verified crr: {'verification queries': {'query 1': 'what are the clinical practice guidelines for diagnosing enterobius vermicularis (pinworm) infestation?', 'query 2': 'what are the first-line treatments for enterobius vermicularis (pinworm) infestation according to current clinical guidelines?', 'query 3': 'what hygiene measures are recommended in clinical practice guidelines to prevent pinworm reinfection and spread?', 'query 4': "is the 'tape test' recommended by clinical practice guidelines for the diagnosis of pinworm infestation?", 'query 5': 'are there any guidelines that recommend treating all household members when one person is diagnosed with pinworm infestation?'}, 'retrieved information': {'response 1': "clinical practice guidelines recommend the diagnosis of enterobius vermicularis (pinworm) infestation to be confirmed through visualization of worms or by identifying eggs using the 'tape test'.", 'response 2': 'guidelines confirm that the first-line treatment for pinworm infestation is either albendazole or mebendazole, given as a single dose and repeated in two weeks.', 'response 3': 'to prevent reinfection and spread, guidelines recommend stringent hygiene measures including washing hands, changing bed linens and underwear frequently, and maintaining clean living environments.', 'response 4': "yes, the 'tape test' is specifically recommended for diagnosing enterobius vermicularis infestation.", 'response 5': 'clinical practice guidelines do recommend considering the treatment of all household members simultaneously to prevent reinfection, reflecting the high contagion rate of pinworm among individuals living in close contact.'}, 'comparison': {'clinical findings': "the clinical findings and the use of the 'tape test' for diagnosis align with clinical practice guidelines.", 'causes': 'the correlations between symptoms and enterobius vermicularis as the cause are consistent with the expected clinical presentation.', 'treatment': 'the treatment plan including the use of albendazole or mebendazole and the emphasis on hygiene measures, matches the recommendations from the clinical practice guidelines.'}, 'discrepancy': none, 'final cot': {'differential diagnosis process': {'step 1': 'given the symptoms of genital itching, particularly noted to be worse at night, and the presence of excoriations over the anus and near the vagina, parasitic infections are highly suspected.', 'step 2': 'the recent history of diarrhea could suggest a gastrointestinal infection or infestation, which may be related to the itching if the pathogen has a cutaneous manifestation or provokes such a reaction.', ' step 3 final note: {"error_flag":1, "error_location":10, "sentence_correction":"suspected of enterobius vermicularis (pinworm) infestation."} medical errors detection and correction prompt 5 1st case scenario figure 6: the last structured output includes: "error flag" to identify if an error exists. and "error location" and "sentence correction" if an error within a clinical note corresponds to the first case scenario and level one notes. at stage 4, the medical errors detection and correction, using prompt 5. deleted part: {} preprocessing 2nd case scenario preprocessed note: a 31-year-old woman comes to the physician because of intermittent episodes of stabbing right lower jaw pain for 6 weeks. the pain is severe, sharp, and lasts for a few seconds. these episodes commonly occur when she washes her face, brushes her teeth, or eats a meal. she does not have visual disturbances, weakness of her facial muscles, or hearing loss. five weeks ago, she had an episode of acute bacterial sinusitis, which was treated with antibiotics. treatment with doxepin is initiated. prompt 1 figure 7: the outputs of clinical notes with the second case scenario, level two clinical notes. at stage 1, the preprocessing, using prompt 1. 16 552 crr round 1: {'differential diagnosis step by step': {'step 1': 'given the location of the pain (right lower jaw) and the trigger factors (washing face, brushing teeth, eating), conditions involving the trigeminal nerve should be considered, as well as dental causes.', 'step 2': 'the absence of visual disturbances, weakness of facial muscles, or hearing loss rules out conditions that involve other cranial nerves or central nervous system disorders.', 'step 3': 'the recent history of acute bacterial sinusitis suggests a possible link between sinus inflammation and the jaw pain, but the nature of the pain described is not typical for sinusitis radiating pain.', 'step 4': 'the nature of the pain (severe, sharp, and lasts for a few seconds) is highly suggestive of trigeminal neuralgia, a disorder of the trigeminal nerve.', 'step 5': 'considering the treatment initiated (doxepin), which is an antidepressant used off-label for chronic pain management, further supports the neuralgia hypothesis over other dental or sinus-related causes.'}, 'differential diagnosis': {'most likely cause': 'trigeminal neuralgia', 'explanation': "the patient's symptoms of intermittent episodes of stabbing pain in the right lower jaw triggered by facial movements or activities, along with her recent sinusitis episode and the absence of other neurological signs, strongly suggest trigeminal neuralgia. this condition involves the trigeminal nerve and is characterized by sudden, severe, brief, stabbing recurrent episodes of facial pain in one or more divisions of the nerve."}, 'treatment step by step': {'step 1': 'confirm the diagnosis of trigeminal neuralgia through a detailed patient history and physical examination to rule out other causes of facial pain.', 'step 2': 'initiate pharmacological treatment with carbamazepine or oxcarbazepine as the first-line treatment for trigeminal neuralgia, adjusting the dosage according to patient response and tolerance.', 'step 3': 'monitor for side effects of medication, such as dizziness, nausea, and diplopia.', 'step 4': 'if the patient does not respond to or cannot tolerate the first-line medications, consider alternative treatments such as gabapentin, pregabalin, or low doses of tricyclic antidepressants like amitriptyline or nortriptyline.', 'step 5': 'in cases where medication does not provide adequate relief, or for individuals who prefer not to take medication long term, refer the patient to a neurosurgeon for consideration of surgical options such as microvascular decompression or gamma knife radiosurgery.', 'step 6': 'provide patient education regarding the nature of the disease, the importance of medication adherence, and possible side effects.'}, 'definitive diagnosis': 'trigeminal neuralgia', 'treatment': {'definitive treatment': 'first-line pharmacological treatment with carbamazepine or oxcarbazepine, with consideration for surgical intervention in refractory cases.'}} prompt 2 crr of diagnosis and clinical management clinical note (deleted diagnosis with 0): a 31-year-old woman comes to the physician because of intermittent episodes of stabbing right lower jaw pain for 6 weeks. the pain is severe, sharp, and lasts for a few seconds. these episodes commonly occur when she washes her face, brushes her teeth, or eats a meal. she does not have visual disturbances, weakness of her facial muscles, or hearing loss. five weeks ago, she had an episode of acute bacterial sinusitis, which was treated with antibiotics. treatment with doxepin is initiated. prompt 3 crr round 2: {'differential diagnosis step by step': {'step 1': 'given the location of pain in the right lower jaw and its triggering by mechanical stimuli such as washing the face and brushing teeth, the differential diagnosis initially includes dental causes like dental abscess, temporomandibular joint disorders, and neuralgias.', 'step 2': 'the absence of dental pathology signs (e.g., toothache, swelling), temporomandibular joint dysfunction symptoms (e.g., jaw clicking, difficulty chewing), and the description of the pain as intermittent, stabbing, and severe, directs the differential towards neuralgias.', 'step 3': "excluding facial muscle weakness, hearing loss, and visual disturbances rules out conditions such as bell's palsy, acoustic neuroma, and multiple sclerosis.", 'step 4': 'the history of acute bacterial sinusitis treated with antibiotics suggests a possible relationship; however, the lack of ongoing sinusitis symptoms decreases the likelihood of this being the direct cause.', 'step 5': 'the pain experienced during common stimuli like eating and brushing teeth specifically implicates triggers typical for trigeminal neuralgia.'}, 'differential diagnosis': {'most likely cause': 'trigeminal neuralgia', 'explanation': 'given the characteristics of the pain (severe, sharp, and lasts for a few seconds) occurring in the distribution of the trigeminal nerve, and triggered by touch, combined with the exclusion of other facial or dental pathologies, trigeminal neuralgia is the most likely diagnosis.'}, 'treatment step by step': {'step 1': 'initiate pharmacologic treatment with anticonvulsants, considering carbamazepine as the first-line treatment due to its effectiveness in relieving neuralgic pain.', 'step 2': 'if carbamazepine is not tolerated or contraindicated, other medications such as gabapentin, lamotrigine, or oxcarbazepine may be considered as alternatives.', 'step 3': 'monitor for side effects of the medication and adjust dosage accordingly to optimize pain control while minimizing adverse effects.', 'step 4': 'in cases where medication is ineffective or side effects are intolerable, referral to a neurosurgeon for evaluation for surgical intervention, such as microvascular decompression, may be considered.', 'step 5': 'provide patient education on trigeminal neuralgia, including potential triggers to avoid and the importance of adherence to medication for pain management.'}, 'definitive diagnosis': 'trigeminal neuralgia', 'treatment': neuralgic conditions, but the first-line treatment for trigeminal neuralgia should be carbamazepine, due to its proven efficacy in this condition. other options include gabapentin and surgical interventions for refractory cases.'}} prompt 2/ 2nd time 2nd case scenario