Overview of the First Shared Task on Clinical Text Generation: RRG 24 and “Discharge Me!” (2024)

Abstract: Recent developments in natural language generation have tremendous implications for healthcare. For instance, state-of-the-art systems could automate the generation of sections in clinical reports to alleviate physician workload and streamline hospital documentation. To explore these applications, we present a shared task consisting of two subtasks: (1) Radiology Report Generation (RRG24) and (2) Discharge Summary Generation (“Discharge Me!”). RRG24 involves generating the ‘Findings’ and ‘Impression’ sections of radiology reports given chest X-rays. “Discharge Me!” involves generating the ‘Brief Hospital Course’ and '‘Discharge Instructions’ sections of discharge summaries for patients admitted through the emergency department. “Discharge Me!” submissions were subsequently reviewed by a team of clinicians. Both tasks emphasize the goal of reducing clinician burnout and repetitive workloads by generating documentation. We received 201 submissions from across 8 teams for RRG24, and 211 submissions from across 16 teams for “Discharge Me!”.

## introduction an important application of natural language generation (nlg) in medical artificial intelligence (ai) is radiology report generation (rrg). specifically, an rrg system can be designed to accept radiology images (e.g., * equal contribution chest x-rays) of a patient and generate a textual report describing the clinical observations in the images. this is a clinically important task, and offers the potential to reduce the repetitive work of radiologists and generally improve clinical communication @xcite . existing studies have been conducted using a single dataset, which limits the scale and diversity of the data and results. therefore, we introduce our first subtask, rrg24, where we curate interpret-cxr, a large-scale collection of rrg datasets from a variety of different sources (i.e., mimic-cxr @xcite , chexpert @xcite , padchest @xcite , bimcv-covid19 @xcite , and openi @xcite ). in rrg24, participants generate the findings and impression sections from chest x-rays. we then evaluate the generations on common leaderboards with standard and recently proposed metrics. ultimately, this task aims to benchmark recent progress using common data splits and evaluation implementations. nlg can also impact discharge documentation by playing a role in generating discharge summaries. hence, we introduce our second subtask, "discharge me!", with the primary objective of encouraging nlg systems that alleviate clinician burden when writing detailed discharge summaries. clinicians play a crucial role in documenting patient progress after a hospital stay, but the creation of concise yet comprehensive brief hospital course (bhc) sections and discharge instructions often demands a significant investment of time @xcite . these two sections in particular cannot be readily copied from prior notes, and thus must be written from scratch by clinicians who synthesize information from across the patient record @xcite . this process contributes to clinician burnout and poses operational inefficiencies within hospital workflows @xcite . we hypothesize that computer-generated clinical documentation has the potential to more accurately and completely capture a patient's hospital course while reducing the administrative burden on clinicians, which, in turn, mitigates burnout, streamlines hospital operations, and ultimately improves the quality of care. thus, in "discharge me!", participants submit generations of both target sections @xcite . we evaluate submissions on a common leaderboard and conduct a subsequent manual clinician review to measure clinical alignment of the outputs. ## rrg24: radiology report generation rrg24 was hosted on vilmedic @xcite , a modular framework for vision-language multimodal research in medicine. the library contains reference implementations for state-of-the-art visionlanguage architectures for medicine and also hosts shared challenges in ai. a total of 201 submissions were received from across 8 teams.