# Enhancing Abstractive Summarization with Extracted Knowledge Graphs and Multi-Source Transformers
## 8 Conclusions  
In summary, our research presents a novel approach to abstractive summarization that addresses limitations in current large language models (LLMs). By incorporating knowledge graph information and structured semantics, we enhance the factual correctness. The MultiBART-GAT model, based on BART, serves as an encoder for textual and graphical inputs, improving the summary quality. An evaluation on the Wiki-Sum dataset, derived from Wikipedia, demonstrates its superior performance compared to baseline models in generating informative summaries. As shown in Tables 6 and 7, our research gives insights into leveraging knowledge graphs for more accurate and verified LLM-generated summaries. We also point out potential directions for improvement:  
* **Dataset cleaning**. To make the dataset more appropriate, it would be better to conduct data pruning by hand or through mechanical turks to form more concise versions of long Wikipedia articles that preserve the majority of information. Stubs still included in the dataset or special articles without an introduction should also be removed. Furthermore, it is worth mentioning that abstracts for Wikipedia articles describing names and places usually contain information (e.g., full names and birth and death dates) that is exclusively saved in the infobox on the side. We think it would be more appropriate for these facts to be included in the source article in some form, such as in automatically generated sentences.
* **Model design**. On the one hand, other graph encoders like GCN or GGNN should also be considered as concatenated structural inputs. On the other hand, inputs to the encoder can also include extracted entities in the form of emphasized tags. This is also a way to include internal semantics as input enrichment.
* **Training**. We think that there is much room for improvement regarding the speed of model convergence; one way to do this is to use a loss that focuses more on node selection.
* **Future directions**. In the final design of our model, we wish to incorporate a multi-hop generation that controls the story flow coupled with regularly trained language models that have been converted for text generation. This will lead to an LLM that is able to align to verified true storylines and make fewer factual errors.  
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Ground Truth** & **Summary by BART** & **Summary by ASGARD** \\ \hline Georgia is a transcontinental country in the Caucasus group of Eurasia. Located at the crossroads of Western Asia and Eastern Europe, it is bounded to the west by the Black Sea, to the north by Russia, to the east by Azerbaijan, and to the south by Armenia and Turkey. The capital and largest city is Tbilisi. & The Roman Empire, also known as the Roman Republic, was a Roman catholic church in United States. It was founded in 1848 by the Russian Empire, and was the second largest in the world. It is one of the most populous city of the world’s most populous state. & Georgia, also known as the Georgia, is a feudal region of the Persian peninsula. It is the largest of the world’s largest city in the world. \\ \hline Blues Brothers 2000 is a 1998 American musical comedy film that is a sequel to the 1980 film Blues Brothers, written and produced by [PERSON] and [PERSON]. Directed by [PERSON], the film stars [PERSON] and [PERSON], with came appearances by various musicians. The film is dedicated to [PERSON], [PERSON], and [PERSON], cast members from the original film who had died prior to the sequel’s production, as well as [PERSON], who died one month before it was released. & Y Blues Brothers 2000 is a 1998 American musical film directed by [PERSON], starring [PERSON], and [PERSON]. The film was released on 14 October 2000. \\ \hline \hline \end{tabular}
\end{table}
Table 6: Examples generated from the Wiki-Sum dataset. We use BART and ASGARD samples to show the dataset’s quality.