Entity-based S pan C opy for Abstractive Summarization to Improve the Factual Consistency (2023)

Abstract: Discourse-aware techniques, including entity-aware approaches, play a crucial role in summarization. In this paper, we propose an entity-based SpanCopy mechanism to tackle the entity-level factual inconsistency problem in abstractive summarization, i.e. reducing the mismatched entities between the generated summaries and the source documents. Complemented by a Global Relevance component to identify summary-worthy entities, our approach demonstrates improved factual consistency while preserving saliency on four summarization datasets, contributing to the effective application of discourse-aware methods summarization tasks.

## introduction discourse-aware models play a crucial role in natural language processing applications, including machine translation @xcite ) and text summarization @xcite . among these applications, abstractive text summarization, the task of generating informative and fluent summaries of the given document(s), has attracted much attention in the nlp community. while early neural approaches focused more on designing customized architectures or training schema @xcite @xcite , recent works have shown that both pre-trained generation models fine-tuned on in-domain datasets and zeroshot gpt-like decoder-only models generally have better performance @xcite @xcite . however, even with state-of-the-art performance on standard automatic evaluation metrics such as rouge @xcite and bertscore @xcite , the generated summaries still suffer from the problem of factual inconsistency, which entities in source doc: royal marine, falklands, portsmouth, falklands war memorial.... ## conclusion and future work in this paper, we tackle the problem of entity-level factual consistency for abstractive summarization through a discourse-aware approach, by guiding the model to directly copy the summary-worthy entities from the source document, through the novel spancopy mechanism (with the optional gr component). this mechanism can be integrated into entities in the source document: yemen(0.28), americans(0.25), saudi arabia(0.23), the state department(0.23), cnn(0.20),..., u.s.(0.15), ... ground-truth summary: no official way out for americans stranded amid fighting in yemen. u.s. deputy chief of mission says situation is very dangerous so no mass evacuation is planned . pegasus: cnn's ivan watson joins a mother and her grandchildren waiting to be evacuated from yemen. the state department has said it is too risky to evacuate americans from the area. watson meets americans who were on a cnn ship that docked at a yemeni port. spancopy: dozens of americans are trapped in yemen. the u.s. has said it is too dangerous to evacuate americans. spancopy+gr: the u.s. has said it is too dangerous to evacuate americans from yemen. the state department said it is too risky to conduct an evacuation of citizens. a group of u.s. organizations have filed a lawsuit against the government's stance on evacuations. to validate the effectiveness of our approach, we conducted experiments on four diverse summarization datasets, including a sanity check on arguably easier subsets. the results confirmed that span-copy with gr performs better on both entity-level factual consistency and saliency. notably, experiments on the original test sets demonstrated that the spancopy mechanism can effectively improve entity-level factual consistency while maintaining word-level and token-level saliency. despite the recent success of gpt-like decoderonly systems on the summarization task @xcite , they still appear to suffer from hallucinations and inconsistencies in the generated text @xcite . as mentioned in section 3, our method can be easily extended to the decoder-only models, we intend to investigate how the mechanism works with the models for addressing these limitations. more long term, we plan to extend our discourseaware approach towards controllable generation with given entities. specifically, instead of using the learned gr scores, the model could generate summaries with desired entities provided by human users.