Unsupervised Keyphrase Extraction by Learning Neural Keyphrase Set Function (2023)

Abstract: We create a paradigm shift concerning building unsupervised keyphrase extraction systems in this paper. Instead of modeling the relevance between an individual candidate phrase and the document as in the commonly used framework, we formulate the unsupervised keyphrase extraction task as a document-set matching problem from a set-wise perspective , in which the document and the candidate set are globally matched in the semantic space to particularly take into account the interactions among all candidate phrases. Since it is intractable to exactly extract the keyphrase set by the matching function during the inference, we propose an approximate approach, which obtains the candidate subsets via a set extractor agent learned by reinforcement learning. Exhaustive experimental results demonstrate the effectiveness of our model, which outperforms the recent state-of-the-art unsupervised keyphrase extraction baselines by a large margin.

## introduction keyphrase extraction (ke) is the task of extracting a keyphrase set that provides readers with highlevel information about the key ideas or important topics described in the document. ke methods can be divided into supervised @xcite @xcite or unsupervised @xcite . the former requires large-scale annotated training data and is often domain-specific, whereas unsupervised methods do not need annotated data @xcite . therefore, in this paper, we focus on unsupervised keyphrase extraction (uke). currently, most uke methods mainly consist of two components: candidate set generation and keyphrase importance estimation. the former uses heuristic rules to obtain a candidate set for a given document. the latter scores individual phrase from the candidate set with respect to a document, and then selects top-ranked phrases to form a keyphrase set. for example, bennani-smires et al. (2018); sun et al. (2020); liang et al. (2021); song et al. (2022b); zhang et al. (2022) address it with the pre-trained embeddings @xcite . these methods independently estimate the relevance between each phrase in the candidate set and the document as the importance of the phrase from a point-wise perspective, as illustrated in figure 1(a) . unfortunately, the above point-wise models are essentially phrase-level uke approaches, and they can not take into account the interactions among all candidate phrases and fails to consider the semantics of the complete candidate set. this makes them more inclined to select keyphrases with highfrequency words while ignoring the coupling of multiple phrases. as a result, the diversity of the selected keyphrases suffers as quantified in our experiments (as shown in to address the above issue, we investigate extracting keyphrases globally from a set-wise perspective (as illustrated in figure 1 ## methodology overview in this paper, keyphrases are globally selected from a set-wise perspective. more formally, consider a ke system: given the document d, generate its candidate set first. and then, an optimal subset s * ⊆ c is selected from the candidate set c. to achieve this goal, we propose a two-stage model (setmatch), including candidate set generation and neural keyphrase set function f s . first, candidate set generation aims to generate a candidate set c from the document d with a higher recall to cover more ground-truth keyphrases (sec 2.1). second, a neural keyphrase set function f s is learned to estimate the document-set matching score (sec 2.2), which is used to guide the keyphrase set extractor agent to search an optimal subset (sec 2.3). ## neural keyphrase set function (f s ) there are many ways to judge whether a keyphrase set is good or bad under the document d. one intuitive way is through a matching framework. therefore, we formulate the neural keyphrase set function f s as a document-set matching task in which the document d and the candidate set c will be matched in a semantic space, as shown in figure 2 . then, we propose a margin-based triplet loss with multiple perspectives orthogonal regularization l e to optimize the siamese-bert auto-encoder architecture. the following section details how we instantiate our neural keyphrase set function f s using a simple siamese-based architecture. ## keyphrase set extractor agent as mentioned before, it is intractable to search the optimal subset by the set function precisely. therefore, we propose a keyphrase set extractor agent to efficiently search an optimal subset. we first exploit a pre-trained bert model to obtain representations of phrases in the candidate set c and the document d, and then learn a subset sampling network to sample a subset s from the candidate set c based on their representations. after obtaining the candidate subset s, we use the keyphrase set function f s to calculate the document-set matching score f s (s, d) as the reward r(s, d) to optimize the keyphrase set extractor agent for extracting an optimal subset s * via reinforcement learning. ## related work unsupervised keyphrase extraction approaches can be mainly categorized into statistics-, graph-, and embedding-based methods @xcite @xcite . the statistics-based methods @xcite exploit to use various features (e.g., word frequency, position, linguistic, etc.) for capturing context information. graph-based methods @xcite @xcite usually convert the document into a graph and rank candidate phrases in the graph. ## conclusion and future work we formulate the unsupervised keyphrase extraction task as a document-set matching problem and propose a novel set-wise framework to match the document and candidate subsets sampled in the candidate set. it is intractable to exactly search the optimal subset by the document-set matching function, and we thereby propose an approximate algorithm for efficient search which learns a keyphrase set extractor agent via reinforcement learning. extensive experimental results show setmatch outperforms the current state-of-the-art unsupervised keyphrase extraction baselines on three benchmark keyphrase extraction datasets, which demonstrates the effectiveness of our proposed paradigm. lately, the emergence of large language models (llms) has garnered significant attention from the computational linguistics community. for future research, exploring effectively utilizing llms to generate candidates and rank candidates to extract keyphrases may be an exciting and valuable direction (i.e., exploring llm-based uke).