e-Health CSIRO at R ad S um23: Adapting a Chest X -Ray Report Generator to Multimodal Radiology Report Summarisation (2023)

Abstract: We describe the participation of team e-Health CSIRO in the BioNLP RadSum task of 2023. This task aims to develop automatic summarisation methods for radiology. The subtask that we participated in was multimodal; the impression section of a report was to be summarised from a given findings section and set of Chest X-rays (CXRs) of a subject’s study. For our method, we adapted an encoder-to-decoder model for CXR report generation to the subtask. e-Health CSIRO placed seventh amongst the participating teams with a RadGraph ER F1 score of 23.9.

## introduction the impression section of a radiology report provides an overview of the key findings from an imaging study. it is meant to convey important information in a concise manner that can be interpreted by the referring clinician. the impression is often written after the findings section of a report, which may include more detailed descriptions, measurements, and technical terms. the impression section should be structured consistently across reports, allowing for a comparison of studies over time on the same subject or subjects with similar conditions. automated report summarisation is gaining more interest as it could reduce the amount of clinical documentation that clinicians must undertake @xcite . this could be especially important in emergency, where it is predicted that the automation of clinical documentation could reduce the burden placed on clinicians @xcite . automated report summarisation is the focus of radsum23 (task 1b of bionlp 2023), a challenge that requires participants to summarise a given findings section via the impression section @xcite . radsum23 consisted of two subtasks: the first focused on summarising the findings sections of mimic-iii @xcite , while the second was a multimodal summarisation task that required participants to summarise the findings section and chest x-rays (cxrs) of a subject's study. the multimodal summarisation task involved two datasets: mimic-cxr for model development and a hidden test set of chexpert. evaluation of the submissions was based on the factual correctness between the predicted and ground truth impression sections. participants were also permitted to use external data sources during model development. methods that could be considered for both subtasks include large language models (llms) @xcite , or retrieval-based methods @xcite . this paper details the participation of the e-health csiro team in radsum23. in particular, we participate in the multimodal summarisation subtask, and not the mimic-iii subtask. our background is in cxr report generation, where the the findings and impression sections for a subject's study are generated given the cxrs for the study. hence, our aim was to adapt a cxr report generator to the task of multimodal summarisation and to determine if it could generalise. ## results & discussion the results for each of the different models on the mimic-cxr test set and the chexpert hidden test set are given in tables 1 and 2 , respectively. mimic-cxr tf produced better scores than tf and mimic-cxr on both test sets, showing that the pre-training task of cxr report generation, fol ## conclusion for bionlp radsum 2023, the performance of our best submission placed the aehrc csiro team seventh, with a radgraph er f1 score of 23.9. even when optimising for the metric used to rank each participant with scst, our encoder-todecoder model developed for cxr report generation and adapted to the summarisation task could not produce impressions that were as factually correct as those produced by the other participants. we will be investigating more appropriate methods for multimodal radiology summarisation in the future, such as llms specifically trained for summarisation and information retrieval-based methods.