Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation (2020)

Abstract: Automated generation of medical reports that describe the findings in the medical images helps radiologists by alleviating their workload. Medical report generation system should generate correct and concise reports. However, data imbalance makes it difficult to train models accurately. Medical datasets are commonly imbalanced in their finding labels because incidence rates differ among diseases; moreover, the ratios of abnormalities to normalities are significantly imbalanced. We propose a novel reinforcement learning method with a reconstructor to improve the clinical correctness of generated reports to train the data-to-text module with a highly imbalanced dataset. Moreover, we introduce a novel data augmentation strategy for reinforcement learning to additionally train the model on infrequent findings. From the perspective of a practical use, we employ a Two-Stage Medical Report Generator (TS-MRGen) for controllable report generation from input images. TS-MRGen consists of two separated stages: an image diagnosis module and a data-to-text module. Radiologists can modify the image diagnosis module results to control the reports that the data-to-text module generates. We conduct an experiment with two medical datasets to assess the data-to-text module and the entire two-stage model. Results demonstrate that the reports generated by our model describe the findings in the input image more correctly.

## introduction writing medical reports manually from medical images is a time-consuming task for radiologists. to write reports, radiologists first recognize what findings are included in medical images, such as computed tomography (ct) and x-ray images. then radiologists compose reports that describe the recognized findings correctly without omission. doctors prefer radiology reports written in natural language. other types of radiology reports, such as tabular reports, are difficult to understand because of their complexity. the purpose of our work is to build an automated medical report generation system to reduce the workload of radiologists. as shown in figure 1 , the medical report generation system should generate correct and concise reports for the input images. however, data imbalance may reduce the quality of automatically generated reports. medical datasets are commonly imbalanced in their finding labels because incidence rates differ among diseases; moreover, the ratios of abnormalities to normalities are also significantly imbalanced. figure 2 shows an imbalanced distribution of finding labels in the mimic-cxr dataset @xcite . for example, the finding label "enlarged cardiomediastinum.negative" appears approximately 70 times more frequently than the finding label "atelectasis.negative". as a result of that imbalance, the generation model tends to train only the frequent finding labels, and tends to omit descriptions of the infrequent labels. this tendency increases incorrectness of generated reports. to improve the correctness of generated reports, we propose a novel reinforcement learning (rl) strategy for a data-to-text generation module with a reconstructor. we introduce a new reward, clinical reconstruction score (crs), to quantify how much information the generated reports retain about the input findings. the reconstructor calculates crs and uses it as a reward for rl to train the model to generate a greater number of correct reports. additionally, we introduce a new reinforcement learning with data augmentation method (rl-da) to alleviate data imbalance problems that arise from infrequent findings. to replace the entire workflow of radiologists, end-to-end image captioning approach is primarily considered @xcite . they generate reports solely from input medical images. however, such approaches are difficult to apply to the real medical field for the following two reasons. first, the quality of generated reports is adversely affected by the insufficient accuracy of image diagnosis systems. to generate correct reports, radiologists must be able to correct wrong image diagnosis results. second, end-to-end models cannot reflect the intentions of radiologists to reports. in contrast to abnormalities, normalities are less important but frequently appear in the images. radiologists sometimes deliberately omit the descriptions of some normalities to write concise reports, especially at return visits. to generate concise reports, radiologists should be able to select which findings the system should include in the reports. we employed the two-stage medical report generator (ts-mrgen), a novel framework for controllable report generation. figure 1 presents an overview of ts-mrgen. ts-mrgen consists of two separate stages: an image diagnosis module and a data-to-text generation module. the image diagnosis module recognizes the findings in the image. subsequently, reports are generated by the data-to-text module. radiologists can modify the wrong or unintended results of the image diagnosis module. next, the modified findings are used as the input to the data-to-text module. this approach greatly improves the correctness and conciseness of generated reports. overall, the main contributions of this study are as follows: we evaluate the proposed approach on two datasets: the japanese computed tomography (jct) dataset and the mimic-cxr dataset. automatic and manual evaluations on the jct dataset show that our crs and rl-da improve the correctness of generated reports. an experiment conducted on the mimic-cxr dataset shows the generality of crs and rl-da; moreover, the experiment on the mimic-cxr dataset demonstrates that ts-mrgen with the modification process generates more correct reports than the two-stage model without a modification process. ## method medical report generation is a task to generate reports consisting of a sequence of words @xmath0, y 2 , ...y n } from a set of images @xmath1. most cases y include more than one sentence. we annotated a set of finding labels @xmath2, f 2 , ...f t } for each set of images. the finding labels include abnormalities (indicated as .positive), normalities (indicated as .negative) and uncertain findings (indicated as .uncertain) . each finding label can be disassembled into a sequence of words as f @xmath3, w t2 , ...w tk }. for example, an abnormality "airspace opacity.positive" label is divided into a sequence of {airspace, opacity, positive}. ## experiment first, to evaluate the effects of our proposed crs and rl-da on the data-to-text module, we conduct an experiment with the japanese computed tomography (jct) dataset. moreover, to evaluate the generality of crs and rl-da and the effects of the modification process on ts-mrgen, we conduct an experiment with the mimic-cxr dataset. ## conclusion we proposed a novel clinical reconstruction score (crs) and reinforcement learning and data augmentation (rl-da) methods to train a data-to-text model for an imbalanced dataset. additionally, we employed a two-stage medical report generator (ts-mrgen) for controllable medical report generation from input medical images. an evaluation of the data-to-text module revealed that our proposed crs and rl-da methods improved the clinical correctness of generated reports, especially for infrequent finding labels. an evaluation of the entire medical report generation system revealed that our ts-mrgen generated more correct reports than an end-to-end generation model. in future work, we would like to explore whether our method is applicable to other domain tasks in data-to-text generation, such as sports summary generation and biography generation tasks. steven j rennie, etienne marcheret, youssef mroueh, jerret ross, and vaibhava goel. 2017. self-critical sequence training for image captioning. in proceedings of 2017 ieee conference on computer vision and pattern recognition. toshinori sato, taiichi hashimoto, and manabu okumura. 2017. implementation of a word segmentation dictionary called mecab-ipadic-neologd and study on how to use it effectively for information retrieval. in proceedings of the 23th annual meeting of the association for natural language processing. mingxing tan and quoc le. 2019. efficientnet: rethinking model scaling for convolutional neural networks. in proceedings of the 36th international conference on machine learning. sam wiseman, stuart m shieber, and alexander m rush. 2017. challenges in data-to-document generation. in proceedings of the 2017 conference on empirical methods in natural language processing. yixiao zhang, xiaosong wang, ziyue xu, qihang yu, alan yuille, and daguang xu. 2020a. when radiology report generation meets knowledge graph. in thirty-fourth aaai conference on artificial intelligence. yuhao zhang, derek merck, emily bao tsai, christopher d manning, and curtis p langlotz. 2020b. optimizing the factual correctness of a summary: a study of summarizing radiology reports. in proceedings of the 58th annual meeting of the association for computational linguistics.