M ed E thic E val: Evaluating Large Language Models Based on C hinese Medical Ethics (2025)

Abstract: Large language models (LLMs) demonstrate significant potential in advancing medical applications, yet their capabilities in addressing medical ethics challenges remain underexplored. This paper introduces MedEthicEval, a novel benchmark designed to systematically evaluate LLMs in the domain of medical ethics. Our framework encompasses two key components: knowledge, assessing the models’ grasp of medical ethics principles, and application, focusing on their ability to apply these principles across diverse scenarios. To support this benchmark, we consulted with medical ethics researchers and developed three datasets addressing distinct ethical challenges: blatant violations of medical ethics, priority dilemmas with clear inclinations, and equilibrium dilemmas without obvious resolutions. MedEthicEval serves as a critical tool for understanding LLMs’ ethical reasoning in healthcare, paving the way for their responsible and effective use in medical contexts.

## introduction the rapid advancement of large language models (llms) has enabled their application across various domains @xcite , including healthcare @xcite . llms are now being used in clinical decision support @xcite , medical education @xcite , and patient communication @xcite . however, their deployment in medicine raises critical concerns about their understanding of medical ethics and the safety of their recommendations @xcite . unlike other domains where factual accuracy might suffice, the field of medical ethics requires models to navigate complex, often ambiguous, ethical principles @xcite , where decisions can have significant real-world consequences. medical ethics is commonly guided by figure 1 : overview of the medethiceval four fundamental principles: respect for autonomy, beneficence, non-maleficence, and justice @xcite . these principles have historically guided human decision-making in medical ethics, playing crucial roles in scenarios like end-of-life care, reproductive ethics and organ donation. however, in the era of llms, these principles are often not sufficiently specific or comprehensive to address the complexities posed by ai-driven decisionmaking @xcite . meanwhile, llms have demonstrated competence in understanding and generating medical knowledge, their ability to handle ethical challenges, especially in nuanced scenarios, remains inadequately assessed. current datasets, such as medsafetybench @xcite and the ethics subset of medbench @xcite , though pioneering this research domain, have certain limitations. first, they fail to account for the multidimensional nature of medical ethics, which includes scenarios involving blatant ethical violations as well as complex ethical dilemmas. these distinct categories require different evaluation criteria, yet existing benchmarks do not make such distinctions. second, they lack differentiation across various medical contexts, despite the fact that ethical principles and their prioritization can vary significantly depending on the specific scenario, such as emergency care, end-of-life decisions, or public health interventions. as a result, there is a pressing need for a more detailed evaluation framework that can rigorously assess llms' capabilities in making ethical decisions. in this work, we propose medethiceval, an evaluation framework designed to assess the capabilities of llms in the domain of chinese medical ethics. following current practice on modern medical ethics @xcite , our framework similarly compromises two main components: ethical knowledge capacity and applying ethical principles to real scenarios, depicted in fig. 1 . knowledge component evaluates the model's understanding and retention of core medical ethics principles and concepts. application component assesses the model's ability to apply this knowledge, where we creatively crafted three scenarios which can be metaphored through a mass balance: (1) detecting violation, which tests the model's ability to recognize and appropriately reject queries that blatantly violate medical ethics; (2) priority dilemma, which examines the model's decision-making in ethically charged dilemmas with clear priorities or inclinations; and (3) equilibrium dilemma, which focuses on the model's responses to ethically neutral or balanced dilemmas without an obvious resolution. fig. 4 provides a more vivid illustration of the three dimensions evaluated in the application component. together, these components provide a holistic view of the model's medical ethics proficiency, both in theory and in practice. for the knowledge component, we utilize existing open-source datasets. in contrast, for the application component, we developed three entirely new datasets foot_0 , each tailored to assess one of the three evaluation dimensions. to construct these datasets, we compiled a collection of medical scenarios and their corresponding ethical guidelines, as shown in fig. 2 . our contributions are threefold: although we currently focus on chinese medical ethics, the criteria, dimensions, scenario classification, and attacking prompts can all serve as guidance for constructing medical ethics benchmarks in other cultures and languages. an example of a single data entry from our datasets is illustrated in fig. 3 . ethics in ai the intersection of artificial intelligence and ethics has attracted considerable attention in recent years. in the context of healthcare, ethical principles such as autonomy, beneficence, non-maleficence, and justice are critical @xcite . prior research has explored the application of these principles in ai systems, focusing on areas such as transparency, bias reduction, and fairness @xcite . however, the evaluation of llms specifically on medical ethics-how well they adhere to these ethical principles in clinical settings-remains underdeveloped. existing ethical evaluations often lack the depth required to assess nuanced scenarios that arise in medical practice. current benchmarks for medical ethics two benchmarks have been developed to evaluate ai systems on medical and ethical considerations. medsafetybench @xcite ) is one such dataset that uses the american medical association (ama) guidelines @xcite to assess ai's compliance with medical ethics. similarly, the medbench @xcite dataset includes a subset focused on ethical decision-making. however, these resources have limitations, such as a narrow focus on specific guidelines or a lack of coverage across diverse clinical scenarios. they fail to address complex ethical dilemmas where multiple principles may conflict, which is crucial for a thorough assessment of llms' capabilities in real-world applications. gaps in existing research while the above efforts provide valuable insights, there remain significant gaps in the current evaluation of llms in medical ethics. existing benchmarks either do not capture the full range of ethical considerations involved in diverse medical scenarios or lack the gran-ularity needed to assess how llms balance conflicting principles. our work aims to fill these gaps by introducing a more comprehensive benchmark that evaluates llms across a wide range of medical scenarios, integrating nuanced ethical dilemmas and aligning with international standards. ## medethiceval construction the benchmark comprises four datasets, three of which are original contributions. the distribution and size of these datasets are presented in we selected the qwen2.5 (qwen team, 2024) to generate our datasets based on several key reasons. firstly, qwen2.5 has demonstrated strong performance in most chinese automatic evaluation benchmarks (qwen team, 2024), outperforming other notable models, such as gpt-4 (openai, 2023). additionally, the quality of the generated questions, as evaluated by experts, surpasses that of other models, as shown in fig. 5 . furthermore, qwen2.5 offers the advantage of being deployable locally, ensuring that data generation does not pose a risk of data leakage. once generated, the questions are also subject to expert review and refinement to ensure quality. ## evaluating criteria in this section, we present the evaluation criteria used for each part of the medethiceval benchmark. knowledge since all the questions in this dataset are multiple-choice, we use accuracy as the evaluation metric, measuring the proportion of correctly answered questions. application we customized evaluation criteria for each assessment section, which were established by a panel of five medical ethics experts. based on their review of how students typically respond to such questions-focusing on the "what" (facts), "why" (reasons), and "how" (applications) (stufflebeam and coryn, 2014)-they developed the evaluation standards for large language models. these criteria, rooted in their professional consensus, guided the evaluation process to ensure rigor and relevance to real-world ethical decisionmaking in medical practice. after observing different models' output and in adherence to existing ethical measurement practice, 5 senior experts in medical ethics research have agreed on the following criteria, which we believe offers a more in-depth perspective to understand and evaluate models' ethical awareness. the evaluation of the model's performance in detecting violations of medical ethics is based on a fourpoint scale, as outlined in ## experimental results and analysis we evaluated the medethiceval benchmark across six llms 3 . ha (health assistant) is fine-tuned on medical text data and uses retrieval-augmented generation (rag) @xcite to incorporate external medical knowledge, enhancing its domain-specific understanding. the evaluation was conducted using a human annotation process. each 3 for detailed information about the evaluated models and model evaluation examples, please refer to appendix c and appendix d. ## conclusion this paper presents medethiceval, a benchmark for evaluating the medical ethics capabilities of llms. through four datasets-knowledge, violation detection, priority dilemma and equilibrium dilemma-we provide a framework for assessing llms' ability to address complex medical ethics challenges. our findings show that qwen2.5 excels in most tasks, while llama3-8b, despite its smaller size, demonstrates impressive performance in both knowledge and ethical reasoning, potentially offering insights for future models focused on safety and ethics. notably, the "post-hoc justification" attack prompt proved to be particularly effective in eliciting unethical behaviors from the models. overall, medethiceval offers valuable insights into llms' medical ethics capabilities and helps guide the responsible deployment of ai in healthcare. limitations cultural and regional variations in ethical norms ethical standards can vary significantly across different countries, cultures, and religious contexts. concepts such as patient autonomy, endof-life care, and privacy protections may be interpreted and implemented differently in various regions. our current benchmark primarily focuses on universal ethical principles and may not fully capture these cultural and regional variations. as a result, models that perform well on this benchmark might still face challenges when applied in contexts with distinct ethical expectations.