Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020)

Abstract: News headline generation aims to produce a short sentence to attract readers to read the news. One news article often contains multiple keyphrases that are of interest to different users, which can naturally have multiple reasonable headlines. However, most existing methods focus on the single headline generation. In this paper, we propose generating multiple headlines with keyphrases of user interests, whose main idea is to generate multiple keyphrases of interest to users for the news first, and then generate multiple keyphrase-relevant headlines. We propose a multi-source Transformer decoder, which takes three sources as inputs: (a) keyphrase, (b) keyphrase-filtered article, and (c) original article to generate keyphrase-relevant, high-quality, and diverse headlines. Furthermore, we propose a simple and effective method to mine the keyphrases of interest in the news article and build a first large-scale keyphrase-aware news headline corpus, which contains over 180K aligned triples of <news article, headline, keyphrase>. Extensive experimental comparisons on the real-world dataset show that the proposed method achieves state-of-the-art results in terms of quality and diversity.

## introduction news headline generation is an under-explored subtask of text summarization @xcite @xcite . unlike text summaries that contain multiple contextrelated sentences to cover the main ideas of a document, news headlines often contain a single short sentence to encourage users to read the news. since one news article typically contains multiple keyphrases or topics of interest to different users, it is useful to generate multiple headlines covering different keyphrases for the news article. multiheadline generation aims to generate multiple independent headlines, which allows us to recommend news with different news headlines based on the interests of users. besides, multi-headline generation can provide multiple hints for human news editors to assist them in writing news headlines. however, most existing methods @xcite @xcite @xcite focus on single-headline generation. the headline generation process is treated as an one-to-one mapping (the input is an article and the output is a headline), which trains and tests the models without any additional guiding information or constraints. we argue that this may lead to two problems. firstly, since it is reasonable to generate multiple headlines for the news, training to generate the single ground-truth might result in a lack of more detailed guidance. even worse, a single ground-truth without any constraint or guidance is often not enough to measure the quality of the generated headline for model testing. for example, even if a generated headline is considered reasonable by humans, it can get a low score in rouge @xcite , because it might focus on the keyphrases or aspects that are not consistent with the ground-truth. in this paper, we incorporate the keyphrase information into the headline generation as additional guidance. unlike one-to-one mapping employed in previous works, we treat the headline generation process as a two-to-one mapping, where the inputs are news articles and keyphrases, and the output is a headline. we propose a keyphrase-aware news multi-headline generation method, which contains two modules: (a) keyphrase generation model, which aims to generate multiple keyphrases of interest to users for the news article. (b) keyphrase-aware multi-headline generation model, which takes the news article and a keyphrase as input and generates a keyphrase-relevant news headline. for training models, we build a first large-scale news keyphrase-aware headline corpus that contains 180k aligned triples of news article, headline, keyphrase . as in years past, a lot of the food trends of the year were based on creating perfectly photogenic dishes. an aesthetically pleasing dish, however, doesn't mean it will stand the test of time. in fact, it's not uncommon for food trends to be all the hype one year and die out the next. from broccoli coffee to "bowl food," here are 10 food trends that you likely won't see in 2019. in 2018, restaurants all over the us decided it was a good idea to place gold foil on everything from ice cream to chicken wings to pizza resulting in an expensive food trend. for example, the ainsworth in new york city sells $1,000 worth of gold covered chicken wings. it seems everyone can agree that this is a food trend that might soon disappear. ## keyphrase-aware headline corpus our keyphrase-aware news headline corpus called keyaware news is built by the following steps: ## related works news headline generation is a subtask of summarization which has been extensively studied recently @xcite @xcite @xcite @xcite . with the rapid development of neural networks, various neural models have been successfully used in the headline generation task. rush et al. (2015) propose an attention-based neural network for headline generation. takase et al. (2016) propose a method based on encoder-decoder architecture and design an amr encoder for headline generation. to take evaluation metrics into consideration, ayana et al. (2016) apply minimum risk training method to the generation model. tan et al. (2017) propose a coarse-to-fine method, which first extracts the salient sentences and then generates the headline based on these sentences. zhou et al. (2017b) propose a method which divides the process of headline generation into three phases: a sentence encoder, a selective gate network for sen-tence selection, and a headline decoder. higurashi et al. (2018) propose an extractive headline generation method, different from previous works that target the headline generation for the articles, this work focus on the task of headline generation for the community question answering forums. due to lack of supervised training data, they propose a learning-to-rank based method to extract the essential substring from a question and use this substring as the headline of the forums. zhang et al. (2018) propose a method for question headline generation, which designs a dual-attention seq2seq model. however, most previous headline generation methods focus on one-to-one mapping, and the headline generation process is not controllable. in this work, we focus on the news multi-headline generation problem and design a keyphrase-aware headline generation method. different information aware methods have been successfully used in natural language generation tasks @xcite @xcite , such as responses generation in the dialogue system. similar to our task, responses generation in a dialogue system is also a one-to-many problem, @xcite propose a mechanism-aware seq2seq model for controllable response generation. they model different mechanisms as latent embeddings and learn the latent embeddings in their seq2seq model. incorporating these mechanisms, their model can generate controllable responses. @xcite propose a commonsense knowledge aware conversation generation method. more concretely, in the first stage, their model retrieves subgraphs from a knowledge base, and the model encodes the subgraphs using a dynamic graph neural network to facilitate better conversation generation in the second stage. @xcite propose an encoder-decoder based neural network for response generation. to our best knowledge, we are the first to consider keyphraseaware mechanism on news headline generation and build the first keyphrase-aware news headline corpus. ## conclusion in this paper, we demonstrate how to enable news headline generation systems to be aware of keyphrases such that the model can generate diverse news headlines in a controlled manner. we also build a first large-scale keyphrase-aware news headline corpus, which is based on mining the keyphrases of users' interests in news articles with user queries. moreover, we propose a keyphraseaware news multi-headline generation model that contains a multi-source transformer decoder with three variants of attention-based fusing mechanisms. extensive experiments on the real-world dataset show that our approach can generate highquality, keyphrase-relevant, and diverse news headlines, which outperforms many strong baselines.