e-Health CSIRO at RRG 24: Entropy-Augmented Self-Critical Sequence Training for Radiology Report Generation (2024)

Abstract: The core novelty of our approach lies in the addition of entropy regularisation to self-critical sequence training. This helps maintain a higher entropy in the token distribution, preventing overfitting to common phrases and ensuring a broader exploration of the vocabulary during training, which is essential for handling the diversity of the radiology reports in the RRG24 datasets. We apply this to a multimodal language model with RadGraph as the reward. Additionally, our model incorporates several other aspects. We use token type embeddings to differentiate between findings and impression section tokens, as well as image embeddings. To handle missing sections, we employ special tokens. We also utilise an attention mask with non-causal masking for the image embeddings and a causal mask for the report token embeddings.

## introduction machine learning holds the potential to significantly enhance diagnostic processes and clinical reporting, particularly within the field of radiology -a discipline characterised by high volumes of imaging data. radiologists are often tasked with interpreting and reporting on hundreds of imaging studies daily, a repetitive process that is susceptible to fatigue and error. automated systems capable of generating radiology reports from chest x-rays (cxrs) could greatly alleviate this burden by ensuring consistency and potentially reducing diagnostic turnaround times. the shared task on large-scale radiology report generation (rrg24) challenges participants to develop automated systems for producing textual reports from cxr images, with a particular focus on the findings and impression sections @xcite . these sec-tions are crucial as they convey the diagnostic interpretation and clinical significance of a patient's study. the challenge provides a means to benchmark the various models under uniform conditions, offering insights into which approaches are most effective for cxr report generation. participants were to train and evaluate their submissions on a dataset formed from five different sources, including mimic-cxr @xcite , chexpert @xcite , padchest @xcite , @xcite , and open-i iu x-ray @xcite . this dataset consisted of four subsets, including the training, validation, public-test, and hidden-test, where the radiology reports were available for all except the hidden-test set. finally, rrg24 presents participants with unique challenges to overcome, such as handling studies with missing sections and deciding whether to use a single model or separate models for each section. this paper outlines the approach taken by team e-health csiro in the rrg24 challenge. for this, we developed a multimodal language model that conditions report generation not only on previously generated words (or subwords), but also on the image embeddings of all the cxrs of a patient's study. we utilised a single model to generate both sections and incorporated special tokens to signify the absence of a section during training. these special tokens were also used to guide the model to generate specific sections during testing. a key factor to the performance of our submissions was our modification to the self-critical sequence training (scst) reinforcement learning (rl) algorithm @xcite . a widelyused technique to enhance rl is to add entropy regularisation into the objective function. this approach boosts exploration and prevents the model from prematurely settling on less optimal actions @xcite . hence, we add entropy regularisation to scst, forming entropy-augmented self- generated tokens t [bos] p 433 s 1 t [sep] s 1 image position s 1 p 432 t frontal + p 434 s 1 t and • • • s 2 t subtle p 573 s 2 t left • • • frontal and [sep] subtle left [eos] • • • • • • findings impression p 572 p 0 s 0 p 1 s 0 p 2 s 0 • • • t [bos] t frontal + + + + + + + + + + + + + + + + + prompt: image embeddings embedding type + + + + + + s 0 s 0 s 0 p 429 p 430 p 431 p 571 generation: report token embeddings decode sample/baseline report radiologist report/ground truth reward llama decoder autoregression projection uniformer token position embedding type projection uniformer projection uniformer type type image embeddings non-causal masking report token embeddings causal masking [bos] frontal and [bos] frontal and attention mask east: entropy-augmented self-critical sequence training figure 1: e-health csiro's submission into rrg24, named cxrmate-rrg24. [bos] denotes the beginning-ofsentence special token, [sep] denotes the separator special token, and [eos] denotes the end-of-sentence special token. e k [i] is the i th output of the projected last hidden state of the encoder for the k th image of the study. ## results and discussion the results for our key submissions on the public and hidden test sets are shown in tables 1 and 2 , respectively. the metrics utilised for rrg24 include bleu-4 @xcite , rouge-l @xcite , bertscore @xcite , chexbert-f1 @xcite , and radgraph-f1 @xcite , the later of which is the primary metric used to rank the teams. here, we compare tf, to scst, and to our proposed method, east. east attained a higher score than tf for each metric, something scst was not able to do (tf attained a higher bleu-4 score than scst for 2 https://huggingface.co/blog/poedator/4d-masks the findings section of both test datasets). comparing east to scst, scst attained a higher rouge-l score on the public-test findings sections, and a higher bertscore on the publictest findings and impression sections, as well as the hidden-test findings sections. for all other cases, east demonstrated an improvement over scst. policies trained with entropy regularisation often have improved generalisation, as they have learnt to consider a broader set of possible actions. this may have led east to be more robust to the differing characteristics of each of the datasets used in the public and hidden test sets. with east, team e-health csiro achieved a first-place finish amongst participants for the public-test impression sections and the hidden-test findings and impression sections. we also also achieved a second-place finish for the public-test findings sections. for a comparison of cxrmate-rrg24 to state-of-theart methods in the literature, please see @xcite .