Event Oriented Abstractive Summarization (2022)

Abstract: Abstractive Summarization models are generally conditioned on the source article. This would generate a summary with the central theme of the article. However, it would not be possible to generate a summary focusing on specific key areas of the article. To solve this problem, we introduce a novel method for abstractive summarization. We aim to use a transformer to generate summaries which are more tailored to the events in the text by using event information. We extract events from text, perform generalized pooling to get a representation for these events and add an event attention block in the decoder to aid the transformer model in summarization. We carried out experiments on CNN / Daily Mail dataset and the BBC Extreme Summarization dataset. We achieve comparable results on both these datasets, with less training and better inclusion of event information in the summaries as shown by human evaluation scores.

## introduction summarization is the process of giving an overview of a piece of text. this is done to reduce the amount of time required to understand a topic by eliminating information that is not as relevant to the topic. in abstractive summarization, the model tries to grasp the source text and produce a summary that consists of novel words and phrases. as the sentences produced are generated by the model, the redundancy in the final summary is significantly reduced as compared to extractive text summarization. this task of summarization is a complex one for humans as well. the difficulty of this task is due to the fact that summarization is fairly subjective. people may assign importance to parts of text differently. thus, the main focus of one person's summary may be just a passing mention in someone else's summary. another reason for this difficulty is that there has to be a balance between novel text and text taken from the source article. for abstractive summaries we want the model to understand the source text, and then represent it in a concise manner. this is a tricky balance to maintain as we want to achieve saliency, but we also want to avoid direct copying from the source text. we use transformers to carry out abstractive text summarization on the cnn / daily mail dataset @xcite and extreme summarization dataset @xcite . the transformer architecture we have used is bart @xcite . in this paper, we propose a system, which modifies the existing bart @xcite architecture by adding an additional event attention block. events can be described as the sub-topics around which the news articles revolve. identifying these events and adding them separately, along with source text, prompts the model to focus the summaries around these events. we perform keyphrase extraction using keybert @xcite to extract important events from the source text and use these events for prompting our model to generate event-oriented summaries. we achieve comparable results for rouge @xcite and bertscore @xcite metrics for cnn / daily mail and xsum datasets, with the base variant of the bart model. moreover, with the help of human evaluation, we quantify the extent to which our generated summaries are influenced by the events input to the model. ## conclusion and future work we introduce event prompting and an additional event attention block in the existing bart-base architecture to enable the model to generate summaries related to the events identified in the source text. our model achieves comparable rouge and bert scores as the larger versions of bart and pegasus @xcite . we also carry out human evaluation for our trained model, and achieve higher scores for event information inclusiveness as compared to the other transformer based models.