Clinical Text Summarization: Adapting Large Language Models Can
  Outperform Human Experts
Authors: name: Dave Van Veen; name: Cara Van Uden; name: Louis Blankemeier; name: Jean-Benoit Delbrouck; name: Asad Aali; name: Christian Bluethgen; name: Anuj Pareek; name: Malgorzata Polacin; name: William Collins; name: Neera Ahuja; name: Curtis P. Langlotz; name: Jason Hom; name: Sergios Gatidis; name: John Pauly; name: Akshay S. Chaudhari
Published: 2023-09-14 05:15:01

Abstract: Sifting through vast textual data and summarizing key information imposes a
substantial burden on how clinicians allocate their time. Although large
language models (LLMs) have shown immense promise in natural language
processing (NLP) tasks, their efficacy across diverse clinical summarization
tasks has not yet been rigorously examined. In this work, we employ domain
adaptation methods on eight LLMs, spanning six datasets and four distinct
summarization tasks: radiology reports, patient questions, progress notes, and
doctor-patient dialogue. Our thorough quantitative assessment reveals
trade-offs between models and adaptation methods in addition to instances where
recent advances in LLMs may not lead to improved results. Further, in a
clinical reader study with six physicians, we depict that summaries from the
best adapted LLM are preferable to human summaries in terms of completeness and
correctness. Our ensuing qualitative analysis delineates mutual challenges
faced by both LLMs and human experts. Lastly, we correlate traditional
quantitative NLP metrics with reader study scores to enhance our understanding
of how these metrics align with physician preferences. Our research marks the
first evidence of LLMs outperforming human experts in clinical text
summarization across multiple tasks. This implies that integrating LLMs into
clinical workflows could alleviate documentation burden, empowering clinicians
to focus more on personalized patient care and other irreplaceable human
aspects of medicine.