L ip K ey: A Large-Scale News Dataset for Absent Keyphrases Generation and Abstractive Summarization (2022)

Abstract: Summaries, keyphrases, and titles are different ways of concisely capturing the content of a document. While most previous work has released the datasets of keyphrases and summarization separately, in this work, we introduce LipKey, the largest news corpus with human-written abstractive summaries, absent keyphrases, and titles. We jointly use the three elements via multi-task training and training as joint structured inputs, in the context of document summarization. We find that including absent keyphrases and titles as additional context to the source document improves transformer-based summarization models.

## introduction key content of an article can be presented in different ways, including summaries, keyphrases, and titles. while most previous research has addressed each element individually (e.g. news summarization @xcite @xcite and keyphrase generation in the scientific domain @xcite ), in this work we release a novel news dataset that consists of highly absent keyphrases (i.e. keyphrases which abstract over the content of the document), abstractive summaries, and titles to investigate the three elements in the context of single-document abstractive summarization. previous work has mainly utilized present keyphrases (i.e. keyphrases that are directly drawn from the source text) through unsupervised and supervised methods for summarization. for instance, traditional summarization models @xcite @xcite @xcite and modern neural models (müngen and kaya, 2018; nallapati et al., 2016; liu et al., 1 data and code used is available at: https://github.com/ fajri91/lipkey 2021) have been combined with the top-k frequent words, @xcite to obtain keyphrases. elsewhere, @xcite ; @xcite used words contained in both the summary and article as keyphrases to improve summarization. this paper aims to study how absent keyphrases (i.e. keyphrases that do not match any words in the source text) can be incorporated into summarization systems. compared to present keyphrases used in previous work, absent keyphrases potentially better complement abstractive summarization methods. previous work has been hindered by the unavailability of a large annotated dataset with gold-standard summaries and keyphrases, thus opting for present keyphrase extraction @xcite . we additionally study the utility of titles in summarization. the underlying hypothesis is that titles and keyphrases are concise, complementary representations of an article, and provide relevant clues for summarization. while previous summarization datasets such as cnndm @xcite , nyt @xcite , and xsum @xcite do not include keyphrases and titles, we present a novel large-scale dataset containing both. following @xcite , we crawl liputan6 foot_0 -an indonesian news portal -to obtain 105k news articles with titles, abstractive summaries, and absent keyphrases, all authored by journalists. note that the dataset of @xcite is based on the time period 2000-2010, at which point liputan6 did not include keyphrases, while our dataset is based on the time period 2019-2021. 3 furthermore, the fact that the dataset is in indonesian contributes to language diversity in nlp @xcite . to summarize our contributions: (1) we release lipkey, the largest news corpus containing human-written abstractive summaries and absent keyphrases, as well as being the first large-scale indonesian keyphrase dataset; (2) through extensive experimentation, we benchmark multi-task training and structured input methods using keyphrases and titles for indonesian text summarization over different pretrained language models. we find that incorporating keyphrases and titles as structured inputs performs better than multi-task training, and consistently improves summary quality. ## related work most keyphrase datasets are in the domain of english scientific publications @xcite @xcite . in incorporating keyphrases into summarization has been explored in other languages such as chinese @xcite ), but using present keyphrases. this is the first work to combine the two tasks in the indonesian language, with previous work separately tackling: (1) keyphrase extraction, over twitter @xcite , consumer-health questions (saputra ## data construction liputan6 is one of the largest indonesian news portals, containing news on topics such as politics, health, business, and popular culture. 5 koto et al. (2020a) found that liputan6 summaries are highly abstractive, written by journalists, and suitable for indonesian text summarization research. the summary and keyphrases are encapsulated in javascript variables window.kmklabs.article with the keys shortdescription and keywords, respectively. 6 in crawling liputan6, we use article id to ensure there is no redundancy in the dataset. lipkey articles span the period december 2019 to march 2021, and each article is associated with a summary, title, and keyphrase(s). 7 in summarization datasets: indosum (kurniawan and louvan, 2018) and liputan6 @xcite . we observe that summaries in lipkey are more abstractive than indosum in terms of novel n-grams (computed relatively to the article). interestingly, we found that lipkey's titles are even more abstractive than the summaries in all datasets. note that the median summary length in lipkey is one sentence, and shorter than liputan6 @xcite at two sentences, despite both datasets being crawled from the same news portal. in constructing lipkey, we discard instances where: (1) one of the keyphrases has more than 6 words (which tends to be noise); (2) the article has less than 15 words; or (3) the summary has less than 5 words. this results in 105,537 instances that we split into 96,541/4,154/4,842 for train/development/test. in terms of the number of words, 33% and 43% of keyphrases consist of 1 and 2 words, respectively, with the remainder being 3-6 words (see we also perform manual analysis over 100 random samples to examine why the keyphrases are absent (i.e. do not occur) in the source article. we find that 80% of keyphrases partially match the article or are word-order variants (see lipkey consists of diverse news genres as shown in ## conclusion in this paper, we release lipkey, the largest news corpus with human-written keyphrases, summaries and titles which is also the first-large scale indonesian keyphrase dataset. we experimented with incorporating keyphrases (and titles) into summarization training via multi-task training or as structured inputs, and found that the latter works better. in this preliminary results, we show that absent keyphrases benefit summarization systems more than present keyphrases extracted by rake.