RECAP : Towards Precise Radiology Report Generation via Dynamic Disease Progression Reasoning (2023)

Abstract: Automating radiology report generation can significantly alleviate radiologists’ workloads. Previous research has primarily focused on realizing highly concise observations while neglecting the precise attributes that determine the severity of diseases (e.g., small pleural effusion). Since incorrect attributes will lead to imprecise radiology reports, strengthening the generation process with precise attribute modeling becomes necessary. Additionally, the temporal information contained in the historical records, which is crucial in evaluating a patient’s current condition (e.g., heart size is unchanged), has also been largely disregarded. To address these issues, we propose RECAP, which generates precise and accurate radiology reports via dynamic disease progression reasoning. Specifically, RECAP first predicts the observations and progressions (i.e., spatiotemporal information) given two consecutive radiographs. It then combines the historical records, spatiotemporal information, and radiographs for report generation, where a disease progression graph and dynamic progression reasoning mechanism are devised to accurately select the attributes of each observation and progression. Extensive experiments on two publicly available datasets demonstrate the effectiveness of our model.

## introduction radiology report generation @xcite @xcite , aiming to generate clinically coherent and factually accurate free-text reports, has received increasing attention from the research community due to its large potential to alleviate radiologists' workloads. recent research works @xcite @xcite @xcite have made significant efforts in improving the clinical factuality of generated reports. despite their progress, these methods still struggle to produce precise and accurate free-text reports. one significant problem within these methods is that although they successfully captured the semantic information of observations, their attributes still remain imprecise. they either ignored historical records (i.e., temporal information) that are required for assessing patients' current conditions or omitted the fine-grained attributes of observations (i.e., spatial information) that are crucial in quantifying the severity of diseases, which are far from adequate and often lead to imprecise reports. both temporal and spatial information are crucial for generating precise and accurate reports. for instance, as illustrated in figure 1 , the patient's conditions can change from time to time, and the observations become better and stable. only if accessing the historical records, the overall conditions could be estimated. in addition, different attributes reflect the severity of an observation, such as normal and top-normal for cardiomegaly. in order to produce precise and accurate free-text reports, we must consider both kinds of information and apply stronger reasoning to strengthen the generation process with precise attribute modeling. in this paper, we propose recap, which captures both temporal and spatial information for radiology report generation via dynamic disease progression reasoning. specifically, recap first predicts observations and progressions given two consecutive radiographs. it then combines them with the historical records and the current radiograph for report generation. to achieve precise attribute modeling, we construct a disease progression graph, which contains the prior and current observations, the progressions, and the precise attributes. we then devise a dynamic progression reasoning (prr) mechanism that aggregates information in the graph to select observation-relevant attributes. in conclusion, our contributions can be summarized as follows: ## conclusion in this paper, we propose recap, which can capture both spatial and temporal information for generating precise and accurate radiology reports. to achieve precise attribute modeling in the generation process, we construct a disease progression graph containing both observations and fined-grained attributes which quantify the severity of diseases and devise a dynamic disease progression reasoning (prr) mechanism to select observation-relevant attributes. experimental results demonstrate the effectiveness of our proposed model in terms of generating precise and accurate radiology reports.