A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare (2025)

Abstract: The application of large language models (LLMs) in healthcare holds significant promise for enhancing clinical decision-making, medical research, and patient care. However, their integration into real-world clinical settings raises critical concerns around trustworthiness, particularly around dimensions of truthfulness, privacy, safety, robustness, fairness, and explainability. These dimensions are essential for ensuring that LLMs generate reliable, unbiased, and ethically sound outputs. While researchers have recently begun developing benchmarks and evaluation frameworks to assess LLM trustworthiness, the trustworthiness of LLMs in healthcare remains underexplored, lacking a systematic review that provides a comprehensive understanding and future insights. This survey addresses that gap by providing a comprehensive review of current methodologies and solutions aimed at mitigating risks across key trust dimensions. We analyze how each dimension affects the reliability and ethical deployment of healthcare LLMs, synthesize ongoing research efforts and identify critical gaps in existing approaches. We also identify emerging challenges posed by evolving paradigms, such as multi-agent collaboration, multi-modal reasoning, and the development of small open-source medical models. Our goal is to guide future research toward more trustworthy, transparent, and clinically viable LLMs.

## introduction the application of llms in healthcare is advancing rapidly, with the potential to transform clinical decision-making, medical research, and patient care. however, incorporating them into healthcare systems poses several key challenges that need to be addressed to ensure their reliable and ethical use. as highlighted in @xcite , a major concern is the trustworthiness of ai-enhanced biomedical insights. this encompasses improving model explainability and interpretability, enhancing robustness against adversarial attacks, mitigating biases across diverse populations, and ensuring strong data privacy protections. key concerns include truthfulness, privacy, safety, robustness, fairness, and explainability, each of which plays a vital role in the reliability and trustworthiness of ai-driven healthcare solutions. truthfulness, defined as "the accurate representation of information, facts, and results by an ai system" @xcite , is critical in healthcare, as inaccuracies can lead to misdiagnoses or inappropriate treatment recommendations. ensuring that generated information is both accurate and aligned with verified medical knowledge is essential. additionally, privacy concerns arise from the risk of exposing sensitive patient data during model training and usage, potentially leading to breaches or violations of regulations such as hipaa (health insurance portability and accountability act) and gdpr (general data protection regulation). ensuring patient confidentiality while leveraging llms for diagnostics and treatment recommendations is a critical challenge. safety, defined as "ensuring that llms do not answer questions that can harm patients or healthcare providers in healthcare settings" @xcite , further underscores the necessity of implementing stringent safeguards to mitigate harm. robustness refers to an llm's ability to consistently generate accurate, reliable, and unbiased outputs across diverse clinical scenarios while minimizing errors, hallucinations, and biases. it also encompasses the model's resilience against adversarial attacks, ensuring that external manipulations do not compromise its integrity. a truly robust llm in healthcare must demonstrate stability, reliability, and fairness, even when faced with noisy, ambiguous, or adversarial inputs. similarly, fairness and bias must be addressed to prevent discriminatory patterns in model outputs, which could lead to unequal treatment recommendations and exacerbate healthcare disparities. furthermore, the explainability of llms, which ensures that model outputs are interpretable and transparent, plays a vital role in fostering trust and allowing informed decisionmaking by healthcare professionals. lack of transparency in model reasoning complicates clinical adoption and raises accountability concerns. clinical deployments of llms expose trust gaps across dimensions. med-palm and med-palm 2 show truthfulness and safety issues, with hallucinated that could misguide care @xcite . integrating llms with ehrs in cloud settings risks hipaa/gdpr violations, prompting on-prem deployment and stronger de-identification @xcite . robustness remains problematic; frameworks like medic and creola assess hallucination severity and clinical safety @xcite . fairness issues persist, with studies showing that llms can perpetuate racial biases in medical recommendations @xcite . finally, explainability challenges were evident in amie-a conversational diagnostic agent evaluated in oscestyle clinical exams-demonstrated strong diagnostic reasoning but lacked transparency compared to human doctors @xcite . tackling these challenges is essential for the trustworthy and ethical implementation of llms in healthcare. recently, researchers have begun developing benchmarks and evaluation frameworks to systematically assess the trustworthiness of llms @xcite . the trustworthiness of llms in healthcare is gaining increasing attention due to its significant social impact. however, there is currently no systematic review that provides a comprehensive understanding and future in-sights into this area. to bridge this gap, we present a comprehensive survey that explores these trustrelated dimensions in detail, reviewing existing benchmarks and methodologies aimed at improving the trustworthiness of llms in healthcare. ## trustworthiness of llms in healthcare we examine the challenges related to the trustworthiness of llms in healthcare, outlining key strategies for identifying and mitigating these concerns. from our literature review screening, we identified truthfulness, privacy, safety, robustness, fairness and bias, and explainability as key trustworthiness dimensions of llms as highlighted in trustllm @xcite , particularly in healthcare. figure 2 provides a summary of the recent research on trust in llms for healthcare across key dimensions of trustworthiness. ## future directions while core trust dimensions, truthfulness, privacy, robustness, fairness, explainability, and safety, have been the focus of recent work, emerging model paradigms such as multi-agent systems, multimodal models, and small open-source llms introduce new trust challenges underexplored. multi-agent llms multi-agent llms enable distributed reasoning through collaboration between specialized agents, offering improved robustness and self-correction. however, they also raise concerns around coordination, error propagation, and the interpretability of inter-agent communication. trustworthy multi-agent systems will require protocols for communication, verification, and evaluation that ensure factual alignment and fairness. for example, @xcite introduce triageagent, a clinical multi-agent framework with role-specific llms for diagnosis and decisionmaking. while it shows benefits like structured collaboration and early stopping, it also reveals trust challenges, including inconsistent agent confidence, limited transparency, and error propagation-highlighting the need for stronger verification and alignment in high-stakes settings. ## conclusion as large language models continue to expand their role in healthcare, ensuring their trustworthiness remains a critical challenge. this survey reviewed six core dimensions-truthfulness, privacy, safety, robustness, fairness, and explainability-highlighting key methods, benchmarks, and limitations in current research. while recent advances have laid important groundwork, most existing solutions remain narrowly scoped and lack integration across dimensions, limiting their effectiveness in realworld clinical settings.