Reinforcement Learning for Improved Alignment in Radiology Reports Generation
Authors: Yongqi Shao, Renxin Xu, Cong Tan, Tao Fang, Hong Huo


Reviewer paper summaries: This paper proposes a reinforcement learning (RL) framework for improving the alignment between chest X-ray images and automatically generated radiology reports. The key components include: (1) Using high-level concepts extracted by the CheXbert tool as an intermediate representation. (2) A double-staged model that first generates these high-level concepts from the image using a Meshed-Memory Transformer, and then generates the full report from the concepts using BART. (3) Fine-tuning the full model end-to-end with RL, using rewards based on NLG metrics like BLEU, METEOR and ROUGE to encourage better alignment between the image and text. Experiments on the IU X-Ray and MIMIC-CXR datasets demonstrate improved performance over prior methods.
Summary of strengths: 1. Incorporating RL with rewards based on NLG metrics is a clever way to encourage alignment between the visual and textual information without requiring expensive manual annotations. The ablations validate the potential effectiveness of the RL fine-tuning.
2. The paper includes a thorough set of ablation experiments, systematically evaluating the contributions of different components like the progressive generation and RL fine-tuning. These ablations provide valuable insight into the relative importance of each design choice and help justify the final model architecture. This level of empirical analysis strengthens the paper's technical contributions.
2. The implementation details in the appendix are quite thorough, supporting reproducibility. 
3. The paper is well written and easy to follow.
Summary of weaknesses: 1. While the proposed approach combines several techniques in a new way, the overall novelty appears somewhat incremental. Recent works have explored similar ideas, such as using RL to improve radiology report generation [1,3] and progressive generation with high-level concepts [2]. Moreover, the results shown in Table 3 has marginal improvements. The authors could do more to clarify how their specific design choices and technical contributions provide benefits over these closely related methods.

[1] Zhou, Zijian, et al. "Large Model driven Radiology Report Generation with Clinical Quality Reinforcement Learning." arXiv preprint arXiv:2403.06728 (2024).

[2] Parres, Daniel, Alberto Albiol, and Roberto Paredes. "Improving Radiology Report Generation Quality and Diversity through Reinforcement Learning and Text Augmentation." Bioengineering 11.4 (2024): 351.

[3] Qin, Han, and Yan Song. "Reinforced cross-modal alignment for radiology report generation." Findings of the Association for Computational Linguistics: ACL 2022. 2022.

While the individual model components are clearly described, it would be helpful to provide a more detailed walk-through of how they are trained. For example, is the ViLM component pre-trained separately first, or is everything trained end-to-end from scratch? Are the NLG-based rewards only used in the final RL stage or also in earlier training? Making the training process more explicit would improve clarity.

2. The analysis of the qualitative examples is somewhat brief. Add more non cherry-picked examples to the appendix that show better results. A more in-depth discussion of specific patterns in the model's errors and where it improves over the baselines would give additional insight.
Comments, suggestions and typos: 1. In Figure 2, consider making the "Visual Extractor" and "Encoder-Decoder" labels more prominent, as they get a bit lost in the details of the diagram.
2. The last paragraph of Section 2.3 has some grammatical issues, e.g., "the baseline b is calculated by assessing the NLG metric of the report produced through greedy sampling during the inference phase of training" is a bit hard to follow. Simplifying the phrasing would help.
3. There are a few typos and consistency issues in the references, e.g., missing italics for "et al.", inconsistent capitalization of titles. A careful proofreading would help polish these.
Overall assessment scores: 2