TWAG : A Topic-Guided W ikipedia Abstract Generator (2021)

Abstract: Wikipedia abstract generation aims to distill a Wikipedia abstract from web sources and has met significant success by adopting multi-document summarization techniques. However, previous works generally view the abstract as plain text, ignoring the fact that it is a description of a certain entity and can be decomposed into different topics. In this paper, we propose a two-stage model TWAG that guides the abstract generation with topical information. First, we detect the topic of each input paragraph with a classifier trained on existing Wikipedia articles to divide input documents into different topics. Then, we predict the topic distribution of each abstract sentence, and decode the sentence from topic-aware representations with a Pointer-Generator network. We evaluate our model on the WikiCatSum dataset, and the results show that TWAG outperforms various existing baselines and is capable of generating comprehensive abstracts.

## introduction wikipedia, one of the most popular crowd-sourced online knowledge bases, has been widely used as the valuable resources in natural language processing tasks such as knowledge acquisition @xcite and question answering @xcite due to its high quality and wide coverage. within a wikipedia article, its abstract is the overview of the whole content, and thus becomes the most frequently used part in various tasks. however, the abstract is often contributed by experts, which is labor-intensive and prone to be incomplete. in this paper, we aim to automatically generate wikipedia abstracts based on the related documents * corresponding author collected from referred websites or search engines, which is essentially a multi-document summarization problem. this problem is studied in both extractive and abstractive manners. the extractive models attempt to select relevant textual units from input documents and combine them into a summary. graph-based representations are widely exploited to capture the most salient textual units and enhance the quality of the final summary @xcite @xcite . recently, there also emerge neural extractive models @xcite utilizing the graph convolutional network @xcite to better capture inter-document relations. however, these models are not suitable for wikipedia abstract generation. the reason is that the input documents collected from various sources are often noisy and lack intrinsic relations @xcite , which makes the relation graph hard to build. the abstractive models aim to distill an informative and coherent summary via sentence-fusion and paraphrasing @xcite @xcite , but achieve little success due to the limited scale of datasets. @xcite proposes an extractive-then-abstractive model and contributes wikisum, a large-scale dataset for wikipedia abstract generation, inspiring a branch of further studies @xcite @xcite . the above models generally view the abstract as plain text, ignoring the fact that wikipedia abstracts describe certain entities, and the structure of wikipedia articles could help generate comprehensive abstracts. we observe that humans tend to describe entities in a certain domain from several topics when writing wikipedia abstracts. as illustrated in figure 1 , the abstract of the arctic fox contains its adaption, biology taxonomy and geographical distribution, which is consistent with ## problem definition definition 1 wikipedia abstract generation accepts a set of paragraphs foot_0 @xmath0, d 2 , . . . , d n } of size n as input, and outputs a wikipedia abstract @xmath1, s 2 , . . . , s m ) with m sentences. the goal is to find an optimal abstract s * that best concludes the input, i.e., previous works generally view s as plain text, ignoring the semantics in wikipedia articles. before introducing our idea, let's review how wikipedia organizes articles. wikipedia employs a hierarchical open category system to organize millions of articles, and we name the top-level category as domain. as for a wikipedia article, we concern three parts, i.e., the abstract, the content table, and textual contents. note that the content table is composed of several section labels {l}, pairing with corresponding textual contents {p}. as illustrated in figure 1 , the content table indicates different aspects (we call them topics) of the article, and the abstract semantically corresponds to these topics, telling us that topics could benefit the abstract generation. however, general domains like person or animal consist millions of articles with diverse content tables, making it not feasible to simply treat section labels as topics. considering that articles in specific domains often share several salient topics, we manually merge similar section labels to convert the sections titles to a set of topics. formally, the topic set is denoted as @xmath2, t 2 , ..., t nt } of size n t , where each topic t @xmath3, l 2 i , . . . , l m i }. now, our task can be expressed with a topical objective, i.e., definition 2 given the input paragraphs d, we introduce the latent topics @xmath4, z 2 , . . . , z n }, where z @xmath5, and our objective of wikipedia abstract generation is re-written as therefore, the abstract generation could be completed with two sub-tasks, i.e., topic detection to optimize arg max z p (z|d) and topic-aware abstract generation to optimize arg max s p (s|d, z). ## the proposed method as shown in figure 2 , our proposed twag adopts a two-stage structure. first, we train a topic detector based on existing wikipedia articles to predict the topic of input paragraphs. second, we group the with a broader contour , strong keel and dextral coil … endemic to the remaining wet forests on the hawaiian island of lanai … the twag framework. we use an example domain with 3 topics for illustration. the left half is the topic detector which attempts to find a topic for each input paragraph, and the right half is the topic-aware abstract generator to generate the abstract by sentence based on input paragraphs and their predicted topics. ## conclusion in this paper, we propose a novel topic-guided abstractive summarization model twag for generating wikipedia abstracts. it investigates the section labels of wikipedia, dividing the input document into different topics to improve the quality of generated abstract. this approach simulates the way how human recognize entities, and experimental results show that our model obviously outperforms existing state-of-the-art models which view wikipedia abstracts as plain text. our model also demonstrates its high data efficiency. in the future, we will try to incorporate pretrained language models into the topic-aware abstract generator module, and apply the topic-aware model to other texts rich in topical information like sports match reports. bart-base. we infer that bart-large may overfit on training data, and bart-base is more competent to be the baseline.