SGG : Learning to Select, Guide, and Generate for Keyphrase Generation (2021)

Abstract: Keyphrases, that concisely summarize the high-level topics discussed in a document, can be categorized into present keyphrase which explicitly appears in the source text and absent keyphrase which does not match any contiguous subsequence but is highly semantically related to the source. Most existing keyphrase generation approaches synchronously generate present and absent keyphrases without explicitly distinguishing these two categories. In this paper, a Select-Guide-Generate (SGG) approach is proposed to deal with present and absent keyphrases generation separately with different mechanisms. Specifically, SGG is a hierarchical neural network which consists of a pointing-based selector at low layer concentrated on present keyphrase generation, a selection-guided generator at high layer dedicated to absent keyphrase generation, and a guider in the middle to transfer information from selector to generator. Experimental results on four keyphrase generation benchmarks demonstrate the effectiveness of our model, which significantly outperforms the strong baselines for both present and absent keyphrases generation. Furthermore, we extend SGG to a title generation task which indicates its extensibility in natural language generation tasks.

## introduction automatic keyphrase prediction recommends a set of representative phrases that are related to the main topics discussed in a document @xcite . since keyphrases can provide a high-level topic description of a document, they are beneficial for a wide range of natural language processing (nlp) tasks, such as information extraction @xcite ), text summarization @xcite and question generation @xcite . existing methods for keyphrase prediction can be categorized into extraction and generation approaches. specifically, keyphrase extraction methods identify important consecutive words from a given document as keyphrases, which means that the extracted keyphrases (denoted as present keyphrases) must exactly come from the given document. however, some keyphrases (denoted as absent keyphrases) of a given document do not match any contiguous subsequence but are highly semantically related to the source text. the extraction methods fail to predict these absent keyphrases. therefore, generation methods have been proposed to produce a keyphrase verbatim from a predefined vocabulary, no matter whether the generated keyphrase appears in the source text. compared with conventional extraction methods, generation methods have the ability of generating absent keyphrases as well as present keyphrases. copyrnn @xcite is the first to employ the sequence-to-sequence (seq2seq) framework @xcite with the copying mechanism @xcite to generate keyphrases for the given documents. following the copy-rnn, several seq2seq-based keyphrase generation approaches have been proposed to improve the generation performance @xcite @xcite @xcite . all these existing methods generate present and absent keyphrases synchronously without explicitly distinguishing these two different categories of keyphrases, which leads to two problems: to address the aforementioned problems, we propose a select-guide-generate (sgg) approach, which deals with present and absent keyphrase generation separately with different stages based on different mechanisms. figure 1 illustrates an example of keyphrase prediction by sgg. the motivation behind is to solve keyphrase generation problem from selecting to generating, and use the selected results to guide the generation. specifically, our sgg is implemented with a hierarchical neural network which performs seq2seq learning by applying a multi-task learning strategy. this network consists of a selector at low layer, a generator at high layer, and a guider at middle layer for information transfer. the selector generates present keyphrases through a pointing mechanism @xcite , which adopts attention distributions to select a sequence of words from the source text as output. the generator further generates the absent keyphrases through a pointing-generating (pg) mechanism @xcite . since present keyphrases have already been generated by the selector, they should not be generated again by the generator. therefore, a guider is designed to memorize the generated present keyphrases from the selector, and then fed into the attention module of the generator to constrain it to focus on generating absent keyphrases. we summarize our main contributions as follows: ## related work as mentioned in section 1, the extraction and generation methods are two different research directions in the field of keyphrase prediction. the existing extraction methods can be broadly classified into supervised and unsupervised approaches. the supervised approaches treat keyphrase extraction as a binary classification task, which train the models with the features of labeled keyphrases to determine whether a candidate phrase is a keyphrase @xcite @xcite . in contrast, the unsupervised approaches treat keyphrase extraction as a ranking task, scoring each candidate using some different ranking metrics, such as clustering @xcite , or graph-based ranking @xcite @xcite . this work is mainly related to keyphrase generation approaches which have demonstrated good performance on keyphrase prediction task. following copyrnn @xcite , several extensions have been proposed to boost the generation capability. in copyrnn, model training heavily relies on large amount of labeled data, which is often unavailable especially for the new domains. to address this problem, @xcite proposed a semi-supervised keyphrase generation model that utilizes both abundant unlabeled data and limited labeled data. copyrnn uses the concatenation of article title and abstract as input, ignoring the leading role of the title. to address this deficiency, @xcite proposed a title-guided seq2seq network to sufficiently utilize the already summarized information in title. in addition, some research attempts to introduce external knowledge into keyphrase generation, such as syntactic constraints @xcite and latent topics @xcite . these approaches do not consider the one-tomany relationship between the input text and target keyphrases, and thus fail to model the correlation among the multiple target keyphrases. to overcome this drawback, @xcite incorporated the review mechanism into keyphrase generation and proposed a model corrrnn with correlation constraints. similarly, sgg separately models one-to-many relationship between the input text and present keyphrases and absent keyphrases. to avoid generating duplicate keyphrases, @xcite proposed an exclusive hierarchical decoding framework that includes a hierarchical decoding process and either a soft or a hard exclusion mechanism. for the same purpose, our method deploys a guider to avoid the generator generating duplicate present keyphrases. last but most important, all these methods do not consider the difference between present and absent keyphrases. we are the first to discriminately treat present and absent keyphrases in keyphrase generation task. ## conclusion in this paper, a select-guide-generate (sgg) approach is proposed and implemented with a hierarchical neural model for keyphrase generation, which separately deals with the generation of present and absent keyphrases. comprehensive empirical studies demonstrate the effectiveness of sgg. furthermore, a title generation task indicates the extensibility of sgg in other generation tasks.