Question: When compared with the state-of-the-art models for each dataset, how does the model perform?
Evidence: Text summarization is a task to generate a shorter and concise version of a text while preserving the meaning of the original text. The task can be divided into two subtask based on the approach: extractive and abstractive summarization. Extractive summarization is a task to create summaries by pulling out snippets of text form the original text and combining them to form a summary. Abstractive summarization asks to generate summaries from scratch without the restriction to use the available words from the original text. Due to the limitations of extractive summarization on incoherent texts and unnatural methodology BIBREF0 , the research trend has shifted towards abstractive summarization.
Sequence-to-sequence models BIBREF1 with attention mechanism BIBREF2 have found great success in generating abstractive summaries, both from a single sentence BIBREF3 and from a long document with multiple sentences BIBREF4 . However, when generating summaries, it is necessary to determine the main topic and to sift out unnecessary information that can be omitted. Sequence-to-sequence models have the tendency to include all the information, relevant or not, that are found in the original text. This may result to unconcise summaries that concentrates wrongly on irrelevant topics. The problem is especially severe when summarizing longer texts.
In this paper, we propose to use entities found in the original text to infer the summary topic, mitigating the aforementioned problem. Specifically, we leverage on linked entities extracted by employing a readily available entity linking system. The importance of using linked entities in summarization is intuitive and can be explained by looking at Figure 1 as an example. First (O1 in the Figure), aside from auxiliary words to construct a sentence, a summary is mainly composed of linked entities extracted from the original text. Second (O2), we can depict the main topic of the summary as a probability distribution of relevant entities from the list of entities. Finally (O3), we can leverage on entity commonsense learned from a separate large knowledge base such as Wikipedia.
To this end, we present a method to effectively apply linked entities in sequence-to-sequence models, called Entity2Topic (E2T). E2T is a module that can be easily attached to any sequence-to-sequence based summarization model. The module encodes the entities extracted from the original text by an entity linking system (ELS), constructs a vector representing the topic of the summary to be generated, and informs the decoder about the constructed topic vector. Due to the imperfections of current ELS's, the extracted linked entities may be too ambiguous and coarse to be considered relevant to the summary. We solve this issue by using entity encoders with selective disambiguation and by constructing topic vectors using firm attention.
We experiment on two datasets, Gigaword and CNN, with varying lengths. We show that applying our module to a sequence-to-sequence model with attention mechanism significantly increases its performance on both datasets. Moreover, when compared with the state-of-the-art models for each dataset, the model obtains a comparable performance on the Gigaword dataset where the texts are short, and outperforms all competing models on the CNN dataset where the texts are longer. Furthermore, we provide analysis on how our model effectively uses the extracted linked entities to produce concise and better summaries.
Moreover, when compared with the state-of-the-art models for each dataset, the model obtains a comparable performance on the Gigaword dataset where the texts are short, and outperforms all competing models on the CNN dataset where the texts are longer.
Answer: The model obtains a comparable performance on the Gigaword dataset where the texts are short, and outperforms all competing models on the CNN dataset where the texts are longer.