QuantRad: Advancing Quantitative Reliability in Radiology Report Generation with Cascaded Decoders

Authors: Ying Jin, Noel C Codella, Yanbo Xu, Haoquan Fang, Yu Gu, Mu Wei, Jenq-Neng Hwang
Primary area: generative models
Keywords: radiology report generation, image captioning, medical imaging

Abstract:
Radiology report generation using artificial intelligence has shown promise in enhancing clinical workflows. However, due to limitations of language modeling loss, existing approaches struggle with quantitative accuracy (e.g., measuring the size of nodules), and lack the ability to produce confidence scores for medical findings, which is crucial for quantitative metrics required by regulatory approval. This paper introduces QuantRad, a novel approach utilizing cascaded decoders to address these challenges in radiology report generation. QuantRad pairs a vision encoder with three decoders that operate sequentially: the first conducts sentence-level topic planning by generating a series of questions, the second recognizes abnormal targets and their quantitative and categorical attributes, and the third generates the final report by answering each question based on the recognized targets. With the dedicated target recognition step, our method integrates the quantitative strength of a perception model to text generation. Specifically, QuantRad recognizes abnormal targets without being biased by language priors, and produces probability scores along with each finding, allowing adjustments of sensitivity for clinical adoption and producing ROC curves for regulatory compliance. Besides, the disentangled topic planning captures the uncertainties in the omission of medical findings and their presentation order, allowing the report generation decoder to be trained with less ambiguity. Our method advances the accuracy and reliability of radiology report generation, offering a promising path for clinical applications and regulatory validation.

Decision: withdrawn
Reviewer scores: 0

Reviews and discussion:

Metareview: