M ed- C o DE : Medical Critique based Disagreement Evaluation Framework (2025)

Abstract: The emergence of large language models (LLMs) has significantly influenced numerous fields, including healthcare, by enhancing the capabilities of automated systems to process and generate human-like text. However, despite their advancements, the reliability and accuracy of LLMs in medical contexts remain critical concerns. Current evaluation methods often lack robustness and fail to provide a comprehensive assessment of LLM performance, leading to potential risks in clinical settings. In this work, we propose Med-CoDE, a specifically designed evaluation framework for medical LLMs to address these challenges. The framework leverages a critique-based approach to quantitatively measure the degree of disagreement between model-generated responses and established medical ground truths. This framework captures both accuracy and reliability in medical settings. The proposed evaluation framework aims to fill the existing gap in LLM assessment by offering a systematic method to evaluate the quality and trustworthiness of medical LLMs. Through extensive experiments and case studies, we illustrate the practicality of our framework in providing a comprehensive and reliable evaluation of medical LLMs.

## introduction medical question answering systems based on large language models represent a significant leap in leveraging artificial intelligence for healthcare. these systems are designed to process and respond to medical queries. the primary aim of medical qa llms is to provide accurate, reliable, and timely information to support clinicians, researchers, and patients. evaluating the performance of these llms is crucial to ensure their reliability and effectiveness in real-world medical applications. performance evaluation typically involves assessing the accuracy, relevance, and co-none low moderate high degree of disagreement herence of the generated responses compared to established medical standards or expert opinions. traditional methods for evaluating text generation, such as string similarity metrics (e.g., me-teor, bleu, rouge), have been used widely across various domains. these metrics compare the overlap between generated and reference textbased on the n-gram matching, synonymy, and paraphrasing. while effective in general text generation tasks, these metrics pose significant limitations in the medical qa domain. medical texts often require precise and contextually accurate responses where minor discrepancies can lead to substantial misunderstandings or clinical errors. traditional metrics fail to capture the nuanced medical context, thereby providing an inadequate measure of llm performance in this sensitive field. to address the shortcomings of traditional evaluation methods, researchers have started exploring the use of llms themselves for evaluating other llms. frameworks such as harness @xcite , deepeval foot_0 , mlflow foot_1 represent this shift towards llm-assisted evaluation. these frameworks aim to provide more contextual and comprehensive evaluations by leveraging the advanced capabilities of llms to understand the generated responses. despite these advancements, the current llm-assisted evaluation methods still lack a structured approach to quantifying disagreement and assessing reliability. this research paper presents an reliable evaluation framework tailored for medical qa llms. drawing inspiration from the work of @xcite , our framework introduces a critique-based methodology that quantitatively assesses the discrepancies between model-generated responses and established medical ground truths. by employing a critique model, we analyzed the differences in llm outputs and provide a comprehensive evaluation of their accuracy and reliability. the visual representation of med-code framework is shown in fig. 1 . the contributions of this work are as follows. ## related work this section discusses related work in the field of evaluation, highlighting previous contributions. our motivation stems from the shepherd model @xcite , which introduces a large language model designed to generate critiques of model responses to given prompts. we extend this work by using critiques to evaluate discrepancies between model responses and ground truth. recent studies have shown that traditional metrics such as meteor @xcite , rouge @xcite and bleuscore @xcite are inadequate for accurately evaluating open-ended generation tasks due to their reliance on reference text @xcite @xcite @xcite . advances have led to new research using llms as evaluators, demonstrating their potential to overcome these limitations @xcite @xcite . notably, approaches employing powerful llms like gpt-4 have achieved remarkable performance @xcite . however, current llm-based evaluators exhibit shortcomings in robustness, as their performance is highly sensitive to prompts, leading to instability in the evaluation process. recent studies have sought to address these challenges by generating explanations for evaluation outputs @xcite , but this approach does not inherently improve robustness or reliability due to issues such as hallucinations @xcite . in the context of medical ai, where accuracy and reliability are crucial, several research efforts propose strategies to evaluate llm responses. an automatic evaluation metric and algorithm for llms' clinical capabilities is proposed in @xcite , featuring a multi-agent framework with retrieval-augmented evaluation (rae) to assess the behaviors of a doctor agent. @xcite propose a structured method for comprehensive human evaluation of llm outputs, introducing the humanely guidance and tool. @xcite introduce the automated interactive evaluation (aie) framework, which provides a dynamic, realistic platform for assessing llms through multi-turn doctor-patient simulations. ## experiments in this section, we will delve into the experiment setup we have used for building this framework. it is divided into two subsections, first is for the critique generation model, and second is for the critique classification model. ## results & analysis to assess the effectiveness of evaluating responses from large language models, we conducted experiments on four medical benchmark datasets: medqa @xcite , medmcqa @xcite , pubmedqa @xcite , and mmlu @xcite . these datasets are widely used in medical benchmarking and consist of objective-type questions. our analysis focused on the test sets of these datasets using three llms: llama-3 (ai@meta, 2024), mistral @xcite , and biomistral @xcite . we selected these llms due to their demonstrated superior performance on general tasks and medical benchmarks. we utilized meteor and rouge-l scores for automatic evaluation, the llama-3 model for llmassisted evaluation, and our med-code framework to analyze llm performance comprehensively. med-code categorizes responses into four degrees of disagreement, where an ideal model would show the highest average probability for "none" disagreement and the lowest for "high" disagreement. detailed descriptions of each disagreement label are provided in the section 3.1. in ## conclusion in this work, we introduce med-code, an evaluation framework designed to assess the performance of medical llms using critiques and degrees of disagreement. med-code excels in identifying subtle discrepancies between ground-truth answers and model predictions, offering a nuanced evaluation with four levels of disagreement. these levels provide insights into the model's behavior, such as hallucinations, accuracy, and adherence to the question. our framework aids researchers in pinpointing areas where llms fall short, enabling targeted improvements. extensive experiments on standard medical benchmark datasets demonstrate med-code's effectiveness in thoroughly and efficiently analyzing model behavior. this robust evaluation method is crucial for advancing the reliability and safety of ai-driven healthcare solutions. this evaluation framework is adaptable for assessing large language models across various domainspecific tasks as well as general tasks, simply by modifying the critique dataset.