Unsupervised Open-domain Keyphrase Generation (2023)

Abstract: In this work, we study the problem of unsupervised open-domain keyphrase generation, where the objective is a keyphrase generation model that can be built without using human-labeled data and can perform consistently across domains. To solve this problem, we propose a seq2seq model that consists of two modules, namely phraseness and informativeness module, both of which can be built in an unsupervised and open-domain fashion. The phraseness module generates phrases, while the informativeness module guides the generation towards those that represent the core concepts of the text. We thoroughly evaluate our proposed method using eight benchmark datasets from different domains. Results on in-domain datasets show that our approach achieves state-of-the-art results compared with existing unsupervised models, and overall narrows the gap between supervised and unsupervised methods down to about 16%. Furthermore, we demonstrate that our model performs consistently across domains, as it surpasses the baselines on out-of-domain datasets.

## introduction keyphrases are short word sequences that describe the core concepts of the text. the prediction of keyphrases for a text is a task that has received much attention recently. it is a crucial problem as its outputs can be useful for a variety of downstream tasks such as building digital libraries @xcite , document summarization @xcite , document visualization @xcite and so on. there are mainly two approaches to keyphrase prediction, namely keyphrase extraction @xcite @xcite and keyphrase generation @xcite @xcite . keyphrase extraction highlights keyphrases that appear within the text. on the other hand, keyphrase generation generates keyphrases based on the understanding of the given text and therefore allows predicting absent keyphrases alongside present ones @xcite . this ability has made keyphrase generation receive more attention than keyphrase extraction in recent years, as human also tend to use keyphrases that are absent from the text. most of the existing keyphrase generation models use manually labeled data for training @xcite @xcite @xcite . however, obtaining labeled data is often the most expensive component of any machine learning model, and this is the same for keyphrase generation. compared to labeled data, access to unlabeled data is easier and mostly available. for example, the arxiv dataset @xcite containing metadata (e.g., title, abstract) of 1.7 million research articles is readily available on kaggle. therefore, it is more desirable to construct a keyphrase generation model in an unsupervised fashion. furthermore, in practice, the model may have to encounter texts that come from various domains or even unseen ones. therefore, another attractive property of a keyphrase generation model is the ability to handle open-domain documents. considering the above scenario, we propose a new problem called unsupervised opendomain keyphrase generation. similar to every keyphrase generation methods, the model of our objective is given a text x as input, and as output, it generates a set of keyphrases {y}. both x and y are word sequences. furthermore, the model of our objective should satisfy two requirements: 1) it can be built using only an unlabeled corpus, denoted as d; 2) it can effectively handle inputs from across domains. this is a challenging task because we do not have access to labeled data from which to learn the patterns for keyphrases. additionally, we also need our model to work across domains. this is difficult because there might exist different patterns for keyphrases for different domains. none of the existing work addresses these challenges. for instance, supervised keyphrase generation models @xcite @xcite @xcite not only require manually labeled data for training but are also known to perform poorly when being moved out-of-domain. on the other hand, @xcite propose au-tokeygen, which uses pseudo-labeled data to train a seq2seq model in a weakly-supervised fashion, thereby removing the need for human annotation effort. however, similar to supervised models, the weakly-supervised approach taken by autokeygen does not enable it to maintain performance in unseen domains. therefore, to solve our problem, we propose an unsupervised keyphrase generation model that can work across domains. the key idea is to modularize a seq2seq model into two modules. the motivation for modularizing is to decompose keyphrase generation into two simpler problems where each of which can be addressed in an unsupervised and open-domain setting. the first module, named the phraseness module, is responsible for generating phrases, while the second module, named the informativeness module, guides the generation toward the phrases that represent the most crucial concepts of the text. the phraseness module is a retrieval-augmented seq2seq model, where the retriever assists the seq2seq component in generating absent phrases alongside present ones. this module can be built in an unsupervised fashion because it leverages noun phrases to index the retriever and to train the seq2seq model, which can easily be obtained using open-sourced software libraries such as nltk @xcite , and therefore does not require human annotation effort. furthermore, the phraseness module can also be built in an open-domain fashion, thanks to 1) the part-of-speech information incorporated into the seq2seq model, which allows copying words to form grammatically correct noun phrases regardless of domains; 2) the fact that the retriever can be further indexed with domain-relevant information, to provide reliable references. the informativeness module is another seq2seq model, where a phrase is likely to be generated if it contains words that are informative to the given text. inspired by embedding-based unsupervised keyphrase extraction (uke) methods, we quantify informativeness of a word and a text based on their closeness in meaning, which is measured via the similarity between their embeddings. we choose this method of evaluating informativeness over other uke methods (e.g. graph-based, statistics based) since it supports not only present phrases, but also absent ones. similar to the phraseness module, the informativeness module can also be built in an unsupervised and open-domain fashion. this is obtained by using a domain-general, unsupervised text embedding model (e.g. sent2vec @xcite ). we summarize the contributions of our paper. firstly, we propose a new problem called unsupervised open-domain keyphrase generation. secondly, we design a model for solving the problem. our proposed model is a seq2seq model that consists of two modules, one is responsible for generating phrases and the other guides the generation towards the phrases that represent the core concepts of the text. finally, we conduct extensive experiments on multiple datasets across domains to demonstrate the effectiveness of our model as we contrast it against multiple strong baselines. ## proposed method figure 1 illustrates our proposed framework. we propose a seq2seq model that consists of two modules, namely phraseness and informativeness module. we adopt the two terms phraseness and informativeness from @xcite , to describe the desirable criteria a keyphrase should satisfy. phraseness refers to the degree to which a word sequence is considered a phrase, and informativeness refers to how well the phrase illustrates the core concepts of the text. each of the two modules guarantees a criterion mentioned above. in particular, the phraseness module generates (present and absent) phrases, while the informativeness module guides the generation toward phrases that describe the core concepts of the text. in the following sections, we will describe in detail the two modules, as well as how they are combined to generate keyphrases. ## case study we display two examples of generated keyphrases from autokeygen and our proposed model in figure 4. the first example is from krapivin, an indomain dataset, while the second one is from kp-times, an out-of-domain dataset. for the first example, we observe that both the proposed model and autokeygen correctly predict the groundtruth (present and absent) keyphrases. however, it can be seen that, for generating absent keyphrases, au-tokeygen only reorders words that are present in the given text. on the other hand, our model can generate keyphrases whose component words are absent, such as "relational models", "categorical models" and "kripke models". in the second example, it is clear that our model predicts more correct keyphrases. we observe that the absent keyphrases generated by autokeygen are highly irrelevant. on the other hand, our model successfully predicts "storms" and also outputs other absent keyphrases that are relevant, although not being within the ground truth keyphrases. this example help shows that our model is better at handling documents from different domains.