SKGS um: Structured Knowledge-Guided Document Summarization (2024)

Abstract: A summary structure is inherent to certain types of texts according to the Genre Theory of Linguistics. Such structures aid readers in efficiently locating information within summaries. However, most existing automatic summarization methods overlook the importance of summary structure, resulting in summaries that emphasize the most prominent information while omitting essential details from other sections. While a few summarizers recognize the importance of summary structure, they rely heavily on the predefined labels of summary structures in the source document and ground truth summaries. To address these shortcomings, we developed a Structured Knowledge-Guided Summarization (SKGSum) and its variant, SKGSum-W, which do not require structure labels. Instead, these methods rely on a set of automatically extracted summary points to generate summaries. We evaluate the proposed methods using three real-world datasets. The results indicate that our methods not only improve the quality of summaries, in terms of ROUGE and BERTScore, but also broaden the types of documents that can be effectively summarized.

## introduction automatic summarization is one of the most effective solutions to help people read lengthy content quickly and comprehend key information that interests them. furthermore, well-structured summaries can significantly improve the accessibility of information by offering organized key points. in linguistics, the notion that texts of the same type display similar structural features is widely accepted in discourse structure @xcite and genre theory @xcite . texts of the same genre share similar structures to meet specific communicative objectives. these structures have evolved based on cultural norms and the shared expectations of both the writer and reader @xcite . building on genre theory, numerous educational research studies strongly advise students or new writers to write structurally articulated summaries to enhance their readability, such as those related to news articles @xcite , government reports @xcite , legal documents @xcite , medical reports @xcite , and research articles @xcite . typically, a structured summary comprises several summary points, each representing a specific piece of information from the source document. we find that various open-source long document summaries exhibit certain structures @xcite @xcite , as elaborated in section 2.1. however, most existing text summarization methods @xcite @xcite @xcite neglect summary structures. while many researchers agree that structured information is essential to summarization, most of the current models focus on the source document structures or the relations between source sentences instead of the structure of summaries @xcite @xcite . when implementing these models, a significant concern arises due to their tendency to overly emphasize certain sections while neglecting others. for instance, most news articles follow the inverted pyramid model under which the most crucial details (introduction) are presented first, followed by supporting information (news details), and then additional background or related commentary (reactions). as observed in human: tong shao, 20, was an international student from china attending iowa state university. her body was found in the trunk of her car in iowa city on september 26. police believe it had been for three weeks. she died of blunt force trauma and asphyxiation. her boyfriend, xiangnan li, 23, was the last to see her, but flew to china on september 8, before shao was officially missing. according to tong's father, an arrest warrant has now been issued. however, li has disappeared. abstractive (bart): tong shao, 20, was found dead in her toyota camry in iowa city on september 26. her boyfriend, xiangnan li, 23, was listed as a person of interest in the case. the two had met in china in 2011 and li was the last to see her. guiding knowledge (gsum): tong shao, a chemical engineering student at iowa state university, went missing in september 2014. after a three week search, police found her body stuffed in the trunk of her toyota camry in iowa city. xiangnan li, 23, was listed as a person of interest in the case. gpt-4: tong shao, a 20-year-old chemical engineering student at iowa state university, was found murdered and stuffed in the trunk of her car in iowa in 2014. her boyfriend, xiangnan li, was the last person to see her alive and is considered a person of interest in the case. he bought a one-way ticket to china soon after shao's disappearance. llama2: tong shao, a 20-year-old chinese student at the university of iowa, was found murdered in her car in iowa city in september 2014.her boyfriend, xiangnan li, a chemical engineering student at iowa state university, went missing and an arrest warrant has been issued for him. li is wanted for the first-degree murder of tong. tong's father, chunsheng shao, has pleaded with u.s. authorities to share evidence gathered in the case with chinese authorities, as there is no extradition treaty between the two countries. the case is ongoing, and police have not confirmed the arrest warrant. skgsum-w: tong shao, 20, was found dead in her toyota camry on september 26. her body was found wrapped in a towel and in the trunk for three weeks. xiangnan li is believed to have been the last person to see tong alive. he bought a one-way ticket to china in the days after her disappearance. police have issued an arrest warrant for her boyfriend, xiangnan li, 23. the father of tong shao says he wants more to be done in solving her murder. mail foot_0 , existing state-of-the-art (sota) summarizers fall short in adequately covering the three crucial summary points: "introduction", "news details", and "reactions", when compared to humangenerated summaries. more specifically, abstractive summarizers omit information regarding "reactions" and place lesser emphasis on "news details". this is also observed in the knowledgeguided method @xcite , which similarly lacks coverage of "reactions" and under-represents "news details". the challenge in summarizing lengthy documents often arises from adopting an appropriate summary structure during the generation process. we argue that an effective document summarizer should generate structured summaries that cover diverse summary points, such as 'introduction', 'news details', and 'reactions', which originate from the source documents. while some studies recognize the importance of summary structure, they require manual assignment of each sentence in the source document to a predefined summary struc-ture label @xcite . alternatively, other research relies on human input to create summary structures and manually generate summary structure labels @xcite . obviously, such a method is practical only for documents that have a clear, consistent, and fixed summary structure, such as legal documents. to overcome the above challenge, we propose a structured knowledge-guided summarization method (skgsum). skgsum employs a set of key phrases, automatically extracted from humanwritten summaries (as detailed in section 4.1), to represent each summary point. these key phrases are utilized to identify relevant words or sentences in the source document through a summary pointdocument alignment layer. this method enables skgsum to effectively capture pertinent information for each summary point, thus addressing the issue of information loss. however, training skg-sum still requires summary structure labels. to remove the need for these labels during training and to broaden the application scope, we introduce a variant of skgsum, named skgsum-w. this variant synthesizes information from all summary points to generate a comprehensive summary. our contributions are summarized as follows: (1) we propose a structured knowledge-guided summarization (skgsum) to identify essential information for each summary point from the source documents. (2) we propose skgsum-w, a vari2 motivation and problem formulation ## methodology most summarizers employ a single document encoder and a single decoder, as illustrated at the top of figure 1 . such an architecture cannot generate content regarding different summary points. alternatively, one can train separate decoders to decode different parts of a summary as illustrated in the second part of figure 1 . to address the limitations of previous works, we propose a structured knowledge-guided summarization method (skgsum) in section 3.1. we further discuss a variant of skgsum, named skgsum-w, for datasets without gold part summaries in section 3.2. ## related work extractive and abstractive summarizers. existing summarizers can be roughly divided into two types: extractive and abstractive summarizers. extractive summarization classifies the sentences in the source document and determines whether a sentence should be included in the summary. neuralsum @xcite is one of the earliest methods using encoder-decoder structure. recent research like extsum-lg @xcite uses the rnn-based encoder for multi-level representation of sentences by considering document and sentence structure. then, bert encoder @xcite @xcite and transformer encoder @xcite @xcite are introduced to replace rnn-related encoders. besides, mem-sum @xcite introduces reinforcement learning to reduce redundancy in generating summaries by considering selection histories. abstractive methods generate summaries word by word. bert @xcite and bart @xcite are representative models in this type. based on these models, recent studies introduce new knowledge to improve the summarization performance, such as document segmentation @xcite , role labeling detection @xcite , entity aggregation @xcite , key phrases detection @xcite , and guidance signals @xcite . ## conclusion in this paper, we analyze summaries of lengthy documents and observe that their summaries consistently adhere to a distinct structure. to harness this structural information, we introduce two methods, namely skgsum and skgsum-w. our methods align their generated summaries closely with automatically extracted summary points. experiments on three real-world open-source datasets have demonstrated the superiority of our method over state-of-the-art methods in summarizing news articles, legal judgments, and government reports.