BDKG at MEDIQA 2021: System Report for the Radiology Report Summarization Task (2021)

Abstract: This paper presents our winning system at the Radiology Report Summarization track of the MEDIQA 2021 shared task. Radiology report summarization automatically summarizes radiology findings into free-text impressions. This year’s task emphasizes the generalization and transfer ability of participating systems. Our system is built upon a pre-trained Transformer encoder-decoder architecture, i.e., PEGASUS, deployed with an additional domain adaptation module to particularly handle the transfer and generalization issue. Heuristics like ensemble and text normalization are also used. Our system is conceptually simple yet highly effective, achieving a ROUGE-2 score of 0.436 on test set and ranked the 1st place among all participating systems.

## introduction radiology reports are documents that record and interpret radiological examinations. a typical radiology report usually consists of three sections: (1) a background section that describes general information about the patient and exam, (2) a findings section that presents details of the examination, and (3) an impression section that summarizes the findings against the background @xcite . figure 1 provides an example of such a radiology report. in a standard radiology reporting process, a radiologist first dictates detailed findings into the report, and then summarizes the findings into a concise impression based also on general background of the patient @xcite . the impression section, which provides the most valuable information to make clinical decisions, is the most crucial part of a radiology report for both doctors and patients. however, manually summarizing radiology findings into impressions are time-consuming and error-prone @xcite , which necessitates the need to automatically generate radiology impressions. background: examination: chest (portable ap) indication: history: ___m with acute coronary syndrome technique: upright ap view of the chest comparison: chest radiograph ___ findings: patient is status post median sternotomy and cabg. heart size remains mildly enlarged. the aorta is tortuous. mild pulmonary edema is new in the interval. small bilateral pleural effusions are present. patchy bibasilar airspace opacities likely reflect areas of atelectasis ... impression: mild pulmonary edema and trace bilateral pleural effusions. the mediqa 2021 shared task @xcite at the naacl-bionlp workshop sets up a radiology report summarization subtask, the aim of which is to build advanced systems to automatically summarize radiology findings (along with the background) into concise impressions. a key feature of this task is that radiology reports used for training and evaluation are collected from different sources, e.g., training instances are sampled from the mimic-cxr database @xcite and some evaluation instances come from the indiana chest x-ray collection @xcite . this inevitably results in significant discrepancies between training and evaluation, posing new challenges to the generalization and transfer ability of participating systems. @xcite presented the first sequenceto-sequence attempt at automatic summarization of radiology findings into natural language impressions. after that, several extensions and improvements have been proposed, e.g., to take into account the factual correctness @xcite or the ontologies (macavaney et al., 2019; gharebagh transformer encoder-decoder mimic-cxr train set transformer encoder-decoder validation set test set ensemble selected impressions pegasus pegasus transformer encoder-decoder 1. training 2. domain adaptation (used for indiana subset) 3. ensemble negative impression normalization 4. normalization (used for stanford subset) 0.3 0.1 -0.1 -0.5 mutual similarity score -0.5 0.3 ❌ ❌ ✔ impression 0 impression 1 impression 2 select max overall similarity 0.4 0.6 0.8 selected impressions normalized impressions im pr es sio n 0 im pr es sio n 1 im pr es sio n 2 figure 2: an overview of our system, which consists of (1) a transformer encoder-decoder tuning module, (2) a domain adaptation module, (3) an ensemble module, (4) a negative impression normalization module. the domain adaptation module is activated only for test instances in the indiana subset, and the final normalization module is activated only for test instances in the stanford subset. in the past few years, pre-training transformerbased encoder-decoder architectures from largescale text corpora has been proposed and quickly received massive attention @xcite @xcite . quite a number of such pre-trained models, e.g., mass @xcite , bart @xcite , and t5 @xcite , have been devised and proved extremely effective in various language generation tasks. against this background, we choose pega-sus @xcite , a pre-trained model that reports state-of-the-art performance on abstractive text summarization, as the backbone of our system. since radiology report summarization is a special form of abstractive text summarization, we expect this choice to yield optimal performance. besides, we employ a simple yet effective domain adaptation strategy, by further fine-tuning on a small amount of in-domain data to improve generalization and transfer abilities. we also use model ensemble and negative impression normalization strategies to further enhance the performance. figure 2 provides an overview of our system. with all these strategies, our system achieves an overall rouge-2 score of 0.436 on the whole test set, ranked at the 1st place among all participating systems. we will discuss later in the experimental section the performance of different pre-trained models and the effect of each individual strategy. ## task description this section gives a formal definition of the radiology report summarization task, and introduces data and evaluation metrics used for the task. ## our approach we employ a transformer-based encoder-decoder architecture for radiology report summarization. our system, as illustrated in figure 2 , consists of four consecutive modules: our system is simple yet highly effective, ranked at the 1st place among all participating systems. in the rest of this section, we detail key modules of the system. ## experiments and results this section presents experiments and results of our system on the official data. ## conclusion this paper presents our winning system at the radiology report summarization track of the mediqa 2021 shared task. participating systems in this track are required to summarize radiology findings into natural language impressions, and be able to generalize or transfer to reports collected from previously unseen hospitals. we build our system on the basis of a pre-trained transformer encoder-decoder architecture, namely pegasus. we further employ a domain adaptation module to enhance generalization and transfer ability. heuristics such as ensemble and negative impression normalization are also used. our system finally achieves a rouge-2 score of 0.436 on the test set, ranked the 1st place among all participating systems.