Event Graph based Sentence Fusion (2021)

Abstract: Sentence fusion is a conditional generation task that merges several related sentences into a coherent one, which can be deemed as a summary sentence. The importance of sentence fusion has long been recognized by communities in natural language generation, especially in text summarization. It remains challenging for a state-of-the-art neural abstractive summarization model to generate a well-integrated summary sentence. In this paper, we explore the effective sentence fusion method in the context of text summarization. We propose to build an event graph from the input sentences to effectively capture and organize related events in a structured way and use the constructed event graph to guide sentence fusion. In addition to make use of the attention over the content of sentences and graph nodes, we further develop a graph flow attention mechanism to control the fusion process via the graph structure. When evaluated on sentence fusion data built from two summarization datasets, CNN/DaliyMail and Multi-News, our model shows to achieve state-of-the-art performance in terms of Rouge and other metrics like fusion rate and faithfulness.

## introduction sentence fusion aims to combine several related sentences into a single coherent text. it is important in many nlp tasks such as text summarization, question answering and retrieval-based dialogue. in text summarization, it is a common practice for a proficient editor to fuse the information from several related sentences, however, it remains challenging for a state-of-the-art neural abstractive summarization model to achieve effective sentence fusion. as pointed out in @xcite , the human-written summaries contain 32% fusion sentences on the cnn/dailymail dataset, 1 these authors contributed equally to this work. while only 6% of the summary sentences generated by the pointer-generator model @xcite are shown to fuse the information spread over sentences. besides, without proper guidance, many sentences generated by fusion contain factual errors. therefore, it is worthwhile to explore effective sentence fusion methods in the context of text summarization. in fact, the importance of sentence fusion has long been recognized by researchers in the text summarization community. as shown in figure 1 , the researchers have been concerned with two types of sentence fusion task in the past. one is similar sentence fusion and the other one is disparate sentence fusion. for similar sentence fusion, a word graph or a dependency tree is often explored to find a coherent fusion path @xcite @xcite . for disparate sentence fusion, the coreference relations are typically considered as the key to tie the sentences together @xcite . although both types of sentence fusion benefit text summarization, especially multidocument summarization, the solutions are rarely proposed to deal with the two types together. in this paper, we propose to apply the structured event information to guide the two types of sentence fusion in a unified framework. we address the challenge of sentence fusion by building an event graph to capture the semantic relationships among the input sentences. the event graph is a directed graph composed of the nodes representing the predicate and event arguments and the edges that connect these event components together. compared to the word graph or the dependency tree, the event graph provides more informative event-level (or to say entity-level) information. meanwhile, it maintains the semantic integrity of each node, which allows us to add additional edges to represent some crucial relationships in disparate sentence fusion like co-reference. such a structured representation is capable of preserving inherent event information and meanwhile formulating cross-sentence information such as entity interactions and proximity of relevant concepts. with the target to guide sentence fusion, we develop a decoder that utilizes the information from both the sentence sequence and the event graph equipped with different attention mechanisms. we employ sequence attention and graph attention to determine what information is important to be select to generate the appropriate word token at each decoding step. note that sentence fusion requires not only selecting the right salient information but also organizing the selected information logically and orderly. otherwise, the models may tend to randomly combine the key event components or simply copy the most important text span. to this end, we develop a graph flow attention to explore potential fusion paths via the graph structure and control the fusion process. moreover, how to avoid factual errors in a fused sentence is also a critical issue in sentence fusion. inspired by @xcite , we incorporate faithful beam search at the inference stage to reduce possible factual errors. this allows the model to remove the unfaithful candidate output sequence during the generation process by refining the generation probability with a faithful score. since there is no available dataset to evaluate the effectiveness of the sentence fusion models in the context of text summarization, following previous work @xcite , we automatically generate sentence fusion data from summarization datasets including cnn/daliymail @xcite and @xcite . the experiments show that our proposed model indeed improves rouges and the other metrics like faithfulness and the fusion rate. the contribution of our work can be summarized as follows: ## method our sentence fusion model follows the typical encoder-decoder architecture, as shown in figure 2 . it is composed of a joint encoder that produces both source sentences and event graph representations, and a decoder that incorporates the information from the source sentences and the event graph to generate a fused sentence. ## conclusion in this paper, we investigate the sentence fusion problem in the context of text summarization by exploring the event graph. our model captures both node representations and the structural information embodied in the event graph to guide the fusion. we further propose a faithful beam search to reduce the possible faithful errors. the experiment results suggest that event graph is crucial for effective sentence fusion and both node representations and graph structure play important roles in sentence fusion. in the future, we would like to further explore the direct incorporation of event information and the sentence fusion model to text summarization.