Improving Embedding-based Unsupervised Keyphrase Extraction by Incorporating Structural Information (2023)

Abstract: Keyphrase extraction aims to extract a set of phrases with the central idea of the source document. In a structured document, there are certain locations (e.g., the title or the first sentence) where a keyphrase is most likely to appear. However, when extracting keyphrases from the document, most existing embedding-based unsupervised keyphrase extraction models ignore the indicative role of the highlights in certain locations, leading to wrong keyphrases extraction. In this paper, we propose a new Highlight-Guided Unsupervised Keyphrase Extraction model (HGUKE) to address the above issue. Specifically, HGUKE first models the phrase-document relevance via the highlights of the documents. Next, HGUKE calculates the cross-phrase relevance between all candidate phrases. Finally, HGUKE aggregates the above two relevance as the importance score of each candidate phrase to rank and extract keyphrases. The experimental results on three benchmarks demonstrate that HGUKE outperforms the state-of-the-art unsupervised keyphrase extraction baselines.

## introduction keyphrase extraction is the fundamental task of automatically extracting a set of salient phrases from a document that concisely describes its primary content @xcite . figure 1 shows an example of the source document and its corresponding keyphrases. recent developments in pre-trained language models @xcite have heightened the need for utilizing pre-trained embeddings on natural language processing tasks, which significantly improves the performance of embedding-based unsupervised keyphrase extraction models @xcite @xcite . existing embedding-based models mainly consist of two components: candidate keyphrase extraction and keyphrase importance estimation (hasan and figure 1 : randomly sampled document with its corresponding keyphrases from the benchmark keyphrase extraction dataset inspec. bold orange represents the content related to the title, and underlined indicates the content related to the first sentence. @xcite @xcite . the former extracts continuous words from the document as candidate keyphrases through heuristic rules, and the latter estimates the importance of candidate phrases by matching similarity with their corresponding document. generally, the source document has both salient information and noises (redundant content). hence, there may be a deviation when directly using the phrase-document relevance as the importance score of each candidate to select keyphrases. for many specific-domain documents (e.g., news or scientific articles), the highlights (the title or the first sentence) typically contains the central information of the source document (as shown in figure 1 ), which has more significant guidance for extracting keyphrases. however, the recent embedding-based unsupervised keyphrase extraction models ignore the effect of the highlight information, leading to extract wrong keyphrases. motivated by the above issues, we propose a new highlight-guided unsupervised keyphrase extraction model (hguke), which estimates the importance score of each candidate phrase by jointly considering the global and local relevance between candidate phrases and their corresponding document. concretely, hguke first calculates the global relevance by leveraging the highlights rather than the whole document and then locally computes the cross-phrase relevance between all candidate keyphrases, as illustrated in figure 2 . finally, hguke aggregates the global and local relevance as the importance score of each candidate keyphrase to rank and extract keyphrases. experimental results demonstrate that the proposed model hguke outperforms the recent state-ofthe-art embedding-based unsupervised keyphrase extraction baselines on three benchmark keyphrase extraction datasets. ## related work most existing unsupervised keyphrase extraction methods can be mainly divided into four categories: statistics-based, topic-based, graph-based, and embedding-based models. specifically, statisticsbased models @xcite usually extract keyphrases by estimating the importance of candidate phrases with different statistic features, such as word frequency feature, phrase position feature, linguistic features of natural language, etc. topic-based models @xcite typically utilize topic information to determine whether a candidate phrase is a keyphrase. graph-based models @xcite represent the document as a graph and rank candidate phrases by graph-based similarities. embedding-based models usually adopt the pretrained embeddings to obtain document and candidate phrase representations and calculate the importance score of each candidate depending on the obtained representations. benefiting from the development of transformer-based pre-trained language models @xcite in the natural language processing field, embedding-based models @xcite @xcite have achieved outstanding performance. concretely, embedding-based models mainly consist of two procedures: candidate keyphrase representation and keyphrase importance estimation @xcite . the first procedure utilizes natural language linguistic features to construct candidate keyphrases and represents them by pre-trained embedding approaches (e.g., bert @xcite and roberta @xcite ). the second procedure estimates the importance of candidate phrases from different perspectives to determine whether a candidate phrase is a keyphrase. unlike the existing unsupervised keyphrase extraction models, we use the highlight information of the document to calculate the phrase-document relevance instead the whole document. ## conclusion and future work in this paper, we incorporate structural information to improve the performance of embedding-based unsupervised keyphrase extraction. specifically, in this paper, we propose a new highlight-guided unsupervised keyphrase extraction model (hguke), which calculates the phrase-document relevance via the highlight information instead of the whole document to select relevant candidate phrases. extensive experiments demonstrate that hguke outperforms the state-of-the-art unsupervised baselines. future research may investigate adopting different structural information of the source document to improve the performance of unsupervised keyphrase extraction.