Graph Enhanced Contrastive Learning for Radiology Findings Summarization (2022)

Abstract: The impression section of a radiology report summarizes the most prominent observation from the findings section and is the most important section for radiologists to communicate to physicians. Summarizing findings is time-consuming and can be prone to error for inexperienced radiologists, and thus automatic impression generation has attracted substantial attention. With the encoder-decoder framework, most previous studies explore incorporating extra knowledge (e.g., static pre-defined clinical ontologies or extra background information). Yet, they encode such knowledge by a separate encoder to treat it as an extra input to their models, which is limited in leveraging their relations with the original findings. To address the limitation, we propose a unified framework for exploiting both extra knowledge and the original findings in an integrated way so that the critical information (i.e., key words and their relations) can be extracted in an appropriate way to facilitate impression generation. In detail, for each input findings, it is encoded by a text encoder and a graph is constructed through its entities and dependency tree. Then, a graph encoder (e.g., graph neural networks (GNNs)) is adopted to model relation information in the constructed graph. Finally, to emphasize the key words in the findings, contrastive learning is introduced to map positive samples (constructed by masking non-key words) closer and push apart negative ones (constructed by masking key words). The experimental results on two datasets, OpenI and MIMIC-CXR, confirm the effectiveness of our proposed method, where the state-of-the-art results are achieved.

## introduction radiology reports document critical observation in a radiology study and play a vital role in communication between radiologists and physicians. a radiology report usually consists of a findings section describing the details of medical observation and an impression section summarizing the most prominent observation. the impression is the most critical part of a radiology report, but the process of summarizing findings is normally time-consuming and could be prone to errors for inexperienced radiologists. therefore, automatic impression generation (aig) has drawn substantial attention in recent years, and there are many methods proposed in this area @xcite @xcite . most existing studies focus on incorporating extra knowledge on the general encoder-decoder framework. for example, @xcite utilized the background section in the radiology report through a separate encoder and then used it to guide the decoding process to enhance impression generation. similarly, @xcite and @xcite proposed to extract the ontology information from findings and used an encoder to encode such information to promote the decoding process. although these approaches have brought significant improvements, they only leverage extra knowledge and findings separately (i.e., through an extra encoder). thus, their performance relies heavily on the quality of extra knowledge, and the further relationships between extra knowledge and findings are not explored. in this paper, we propose a unified framework to exploit both findings and extra knowledge in an integrated way so that the critical information (i.e., key words and their relations in our paper) can be leveraged in an appropriate way. in detail, for each input findings, we construct a word graph through the automatically extracted entities and dependency tree, with its embeddings, which are from a text encoder. then, we model the relation information among key words through a graph encoder (e.g., graph neural networks (gnns)). finally, contrastive learning is introduced to emphasize key words in findings to map positive samples (constructed by masking non-key words) closer and push apart negative ones (constructed by masking key words), as shown in figure 1 . in such a way, key words and their relations are leveraged in an integrated way through the above two modules (i.e., contrastive learning and the graph encoder) to promote aig. experimental results on two prevailing datasets (i.e., openi and mimic-cxr) show that our proposed approach achieves state-of-the-art results compared to existing studies. ## method we follow the standard sequence-to-sequence paradigm for aig. first, we utilize wordpiece @xcite to tokenize original findings and obtain the source input sequence @xmath0, x 2 , • • • , x n , where n is the number of tokens in x . the goal is to find a sequence @xmath1, ...y i , ..., y l } that summarizes the most critical observations in findings, where l is the length of impression and y @xmath2. the generation process can be formalized as: the model is then trained to maximize the negative conditional log-likelihood of y given the x : ## related work recently, nlp technology has broadly applied in the medical domain, such as medical entity recognition @xcite , radiology report generation @xcite @xcite , aig, etc. impression generation can be regarded as a type of summarization task that has drawn substantial attention in recent years, and there are many studies for addressing general abstractive summarization @xcite @xcite . you et al. (2019) designed a novel focus-attention mechanism and saliencyselection network, equipped in the encoder and decoder to enhance summary generation. li et al. (2020) proposed an abstractive sentence summarization method guided by the key words, which utilized a dual-attention and a dual-copy mechanism to integrate the semantics of both original sequence and key words. many methods propose to introduce specific designs on the general summarization model to address radiology impression generation (zhang et al., 2018; gharebagh et al., 2020; macavaney et al., 2019; hu et al., 2021; abacha et al., 2021). macavaney et al. (2019); gharebagh et al. (2020) extracted the salient clinical ontology terms from findings and then incorporated them into the summarizer through a separate encoder for enhancing aig. hu et al. (2021) further introduced pre-defined word graphs to record salient words as well as their internal relation and then employed two separate graph encoders to leverage graphs for guiding the decoding process. most of these approaches exploit separate encoders to encode predefined knowledge (e.g., ontology terms and word graph), which are then utilized to enhance impression generation. however, they tend to over-rely on the quality of pre-extracted ontologies and word graphs and lack sensitivity to vital information of findings themselves. compared to these models, our method offers an alternative solution to robustly improve key information extraction with the help of both graphs and contrastive learning. ## conclusion in this paper, we propose to combine graphs and contrastive learning to better incorporate valuable features for promoting impression generation. specifically, we utilize the graph encoder to extract relation information from the graph, constructed by medical entities and the dependence tree, for enhancing the representation from the pre-trained text encoder. in addition, we employ contrastive learning to assist the model in distinguishing between critical and secondary information, simultaneously improving sensitivity to important word represen-tation by comparing positive and negative examples. furthermore, we conduct experiments on two benchmark datasets, and the results illustrate the effectiveness of our proposed method, where new state-of-the-art results are achieved.