# Enhancing Abstractive Summarization with Extracted Knowledge Graphs and Multi-Source Transformers
## 1 Introduction  
Abstractive summarization is one of the most challenging tasks in natural language processing. A model is required that can learn the context of long documents while generating summaries with the key information preserved. Essentially, models completing this task will extract text patterns, match them to well-learned patterns in the decoder, and regenerate a summary with their own language encoder. Multiple sequence-to-sequence models using attention mechanisms have been proposed for abstractive summarization and perform well [1, 2]. Since the advent of large language models like BART (Bidirectional and Auto-Regressive Transformers) and BERT (Bidirectional Encoder Representations from Transformers) [3, 4], we have seen significantly more progress in the field of text summarization, and with the availability of GPT (Generative Pre-trained Transformers) and its commercialized version, ChatGPT, introduced by OpenAI [5], large language models and their excellent performance in extracting key information from a multitude of text sources have sparked public and academic interest in LLMs. Although most of the existing state-of-art models are able to generate smooth, informative summaries both on par with human performance and achieving high scores on automatic evaluation metrics like ROUGE [6], they might not understand puns and productnames, which leads to misunderstanding. Furthermore, since summary text is generated based on the possibility of a word's existence in a certain context, they may not be able to consider the factual information and entity correlation in the source document, which leads to fabricated content [7]. We have seen many cases on social media recently where ChatGPT/New Bing has fabricated plausible but fake records based on the user's request. This poses a huge information threat to users who cannot easily verify content truthfulness, which can also be used by hostile groups to spread misinformation and pollute the online literature. This study aims to utilize information from knowledge graphs to generate more accurate and coherent summaries with large language models like BART. Furthermore, the knowledge graph provides us with a semantic interpretation of the input, which is critical for explainable summarization. Compared to concise and information-dense news articles such as those from the NYT (_New York Times_) Corpus and the CNN/DailyMail dataset, generating abstracts from information-sparse crowdsourcing articles, like those on Wikipedia, is quite challenging. In order to handle such information sparsity in Wikipedia articles, we argue that the internal semantics can be extracted to focus on entities and relations. We chose to use knowledge graphs to build a structural representation to connect relevant subjects between the sentences and paragraphs. As for more natural but less formalized texts like Wikipedia articles, complex events related to the same entity or subject are spread over several paragraphs. If we examine the center entity (or essentially, the Wikipedia article's topic subject), it would normally be discussed in the majority of paragraphs, potentially with different names. Therefore, if we can capture this latent semantics structure, it will enable our model to focus on what entities the story is based upon. As a result, we can improve both the readability and the information density of our generated abstract. At the same time, this method should also work well for summarizing abstracts of more concise and formatted text like news and gazette articles. To this end, our main contributions in this work include:  
* Wiki-Sum, a dataset that is extracted from the Wikipedia textural dump, then tokenized and extracted into an abstract summary form;
* MultiBART-GAT, a framework for abstractive text summarization incorporating transformers with graph representation augmentation.