Building blocks for complex tasks: Robust generative event extraction for radiology reports under domain shifts (2023)

Abstract: This paper explores methods for extracting information from radiology reports that generalize across exam modalities to reduce requirements for annotated data. We demonstrate that multi-pass T5-based text-to-text generative models exhibit better generalization across exam modalities compared to approaches that employ BERT-based task-specific classification layers. We then develop methods that reduce the inference cost of the model, making large-scale corpus processing more feasible for clinical applications. Specifically, we introduce a generative technique that decomposes complex tasks into smaller subtask blocks, which improves a single-pass model when combined with multitask training. In addition, we leverage target-domain contexts during inference to enhance domain adaptation, enabling use of smaller models. Analyses offer insights into the benefits of different cost reduction strategies.

## introduction radiology reports contain a diverse and rich set of clinical abnormalities documented by radiologists during their interpretation of the images. automatic extraction of radiological findings would enable a wide range of secondary use applications to support diagnosis, triage, outcomes prediction, and clinical research @xcite . we adopt an event-based schema to capture both indications, the reason for radiology exams, and abnormal findings documented in radiology reports. we use an annotated a corpus of reports from three distinct radiology examination modalities @xcite : magnetic resonance imaging (mri), positron emission tomography (pet), and computed tomography (ct). each event consists of a trigger, words that indicate a particular indication or finding (e.g., lesion), and a set of attributes (assertion, anatomy, characteristics, size, size trend, size count) that describe this indication or finding. manual annotation of radiology reports is costly, therefore we hope models can generalize across different exam modalities. in this work, we define each modality in our annotated corpus as a domain and study cross-domain generalization among different modalities for the task of event extraction. event extraction can be conceptualized as a series of subtasks, which include entity detection (trigger and attribute spans), relation detection (between triggers and attributes), and entity normalization (fine-grained labels on spans). in our experiments, we focus on trigger detection and anatomy attribute extraction with normalized labels. to enhance generalization capabilities, some studies employ generative models and formulate tasks as question answering and using texts to represent both inputs and outputs @xcite , as opposed to allowing the model to solely learn task intent from training data @xcite . the exceptional performance of generative models often rely on large model size; however, in real-time inference for processing large-scale clinical notes, reducing inference costs is crucial. to address this need, for task inference, we want to reduce the number of decoding passes and employ smaller models. due to the high inference costs, there is a desire to merge these subtasks and decode them in a single step. however, the generative approach has been reported to perform better on solving subtasks individually but worsen when combined, a phenomenon referred to as the compositionality gap @xcite . this gap can be exacerbated under domain shifts when models learn subtasks jointly, as interdependence of subtasks may vary across domains. while large language models (llms) mitigate the compositionality gap using reasoning steps @xcite to solve complex questions by decomposing them into smaller ones, there is limited work on reasoning for highly specialized domains (such as medical event extrac-tion) or with smaller models. in this paper, we reduce the compositionality gap for smaller models through formatting of complex tasks into easier subtasks as blocks. this approach teaches models how to solve individual subtasks independently and how to assemble them for solving more complex tasks. the generative model enables seamless integration of supplementary contexts into the prompt, which compensates for the knowledge gap to larger models and reduces inference costs. to aid in domain adaptation, we extract target domain contexts that are likely to be helpful for the task, instead of retrieving similar contexts for general purpose. specifically, to assist with anatomy normalization tasks, we employ an unsupervised extractor to acquire pertinent contexts that likely contain anatomical information from the same document and/or unannotated text from the same domain. this process can either disambiguate the original singlesentence input or provide anatomy-related hints that the model can utilize. to avoid introducing source-domain-specific reliance on the contexts, we incorporate the contexts only at the inference stage. in our experiments, we first study domain shift for extracting radiology finding events and observe that cross-domain performance decline is more pronounced for knowledge-intensive anatomy normalization tasks, while detecting entity spans exhibits relatively stable performance. we demonstrate that building subtask blocks and assembling them as sequences to solve complex tasks can reduce the compositionality gap in smaller models. we show that incorporating target-domain contexts in domain adaptation can compensate for reduced model sizes, enabling good performance with smaller models. ## results as described earlier, target-domain contexts are added to prompts during a second step of t5 decoding to help anatomy normalization, after the 1-step subtask block decoding with t5-base. ## analysis in this section, we analyze results to better understand performance improvements associated with the subtask block format and retrieved context in prompts. ## conclusion in conclusion, we present generative event extraction methods for radiology findings that improve generalization under domain shifts and reduce the inference costs. by decomposing complex tasks into simpler subtask blocks and incorporating target-domain context during the inference process, our approach enables smaller models to achieve performance similar to or better than those obtained with more decoding passes, and comparable to larger models on anatomy normalization. our methods make efficient inference for extensive clinical notes more feasible. this work offers insights into reasoning with smaller models and using context to compensate the reduced model size.