Keyphrase Extraction from Scientific Articles via Extractive Summarization (2021)

Abstract: Automatically extracting keyphrases from scholarly documents leads to a valuable concise representation that humans can understand and machines can process for tasks, such as information retrieval, article clustering and article classification. This paper is concerned with the parts of a scientific article that should be given as input to keyphrase extraction methods. Recent deep learning methods take titles and abstracts as input due to the increased computational complexity in processing long sequences, whereas traditional approaches can also work with full-texts. Titles and abstracts are dense in keyphrases, but often miss important aspects of the articles, while full-texts on the other hand are richer in keyphrases but much noisier. To address this trade-off, we propose the use of extractive summarization models on the full-texts of scholarly documents. Our empirical study on 3 article collections using 3 keyphrase extraction methods shows promising results.

## introduction automatic keyphrase extraction is the process of identifying representative phrases in a document that summarize its content. keyphrases are important pieces of information for many applications, including information retrieval @xcite ), text classification @xcite ), text summarization @xcite , entity recognition @xcite and event detection @xcite . this work focuses on keyphrase extraction from scholarly documents. in particular, we consider an interesting issue in this domain, which concerns the part of a scientific article that should be given as input to keyphrase extraction methods. we can see that recent deep learning keyphrase extraction and generation methods take titles and abstracts as input, due to the complexity in pro-cessing larger sequences. traditional supervised learning methods, as well as unsupervised ones can handle full-texts, but this does not necessarily lead to better results compared to using just titles and abstracts. @xcite show that graph-based methods achieve better accuracy when titles and abstracts are used, while the strong baseline tfidf works best with full-text. @xcite and @xcite show that keyphrases generally occur in positions very close to the beginning of a scholarly document. nguyen and luong (2010) show that title and abstracts have the highest density of keyphrases, followed by the conclusions, introduction and related work sections. it appears that there is a trade-off between using titles and abstracts versus using full-texts of academic papers as input to keyphrase extraction methods. full-texts provide richer information, including more keyphrases, but at the same time they are much more noisy compared to the titles and abstracts. motivated from this observation, our scientific question is whether using automated summarization models on the full-text of a scientific article can lead to textual information that is richer than titles and abstracts, yet less noisy than fulltexts. towards answering this question, we present some first steps employing extractive summarization. our main goals are to: a) investigate the dynamics of summarization in keyphrase extraction, paving the way for the research community to develop approaches combining techniques from both tasks (e.g., via multi-task learning) and b) provide some guidelines to practitioners of the field suggesting better utilization of the full-texts. our empirical study provides strong evidence that the full-text extractive summaries manage to capture keyphrases, which in most cases improve the performance of state-of-the-art supervised and unsupervised keyphrase extraction methods (regarding the f 1 score) on three datasets compared to the conventional use of abstracts and full-texts. ## our approach and alternatives we are interested in finding out whether we can improve the signal-to-noise ratio of the input given to keyphrase extraction approaches by applying automated summarization on the full-text of scientific articles. as a first step towards investigating this hypothesis, we focus on extractive summarization models. we generate extractive summaries from the corresponding full-texts using the pre-trained distillated roberta model distilroberta-base-ext-sum from the transformersum foot_0 library. distillated roberta is a version of roberta @xcite , which is based on distilbert @xcite . it is a lighter, faster and smaller variant of the original roberta, that achieves a time speedup of 50%, while retaining 95% performance of the original model. furthermore, we investigate the utility of alternative input types, such as the first three paragraphs of the document that include the title, the abstract and a part of the document's introduction. we experiment with two different paragraph lengths in words, i.e., 220 and 400. our investigation includes the standard input types, i.e., title+abstract and full-text, too. for deep learning methods, we split full-texts into sentences and paragraphs, as they cannot handle their whole length at once due to memory limitations. finally, we explore an ensemble approach to keyphrase extraction, which involves the late fusion of two input types: the standard title plus abstract and the title plus the extractive summary. we apply keyphrase extraction methods to these two input types independently and then consider the union of the extracted keyphrases. abbr. description ta abstract abse abstract in sentences f full-text fp full-text in paragraphs fs full-text in sentences ts extractive summary as abstract ∪ extractive summary 3p 220 first 3 paragraphs -length in words: 220 3p 400 first 3 paragraphs -length in words: 400 ## experimental setup our empirical study includes three keyphrase extraction methods: tfidf, as a baseline method, mul-tipartiterank (mr) @xcite , as a strong graph-based method, and bi-lstm-crf (blc) @xcite , as a strong neural model. due to the lack of publicly available code for a blc model tailored to keyphrase extraction, we proceeded to our own implementation, which we make publicly available along with all experiments in this paper foot_1 . blc is trained using the train and validation sets from @xcite . specifically, we trained both models described in @xcite , i.e., the blc t a on the documents' abstracts and the blc abse on the abstracts' sentences (used only with test datasets that their text is split in sentences and only for the model comparison). experiments were performed on a ryzen 5 3600 cpu with 16gb ram. training the model on title and abstract takes approximately 24 hours for a total of 5 epochs, while training on title and abstract split in sentences takes about 5 hours to complete. these keyphrase extraction methods are evaluated on three well-known datasets that contain fulltext articles from the computer science domain: se-meval @xcite , nus (nguyen and @xcite , and acm @xcite . these datasets contain 244, 211, and 2304 documents, respectively (we merged the train and test sets of the semeval dataset). we compute f 1 (f 1 @10 for unsupervised methods) according to both the exact (e) and partial (p) @xcite string match to determine the number of correctly matched phrases with the golden ones for a document. we also apply stemming to the methods' output and the article's golden phrases as a pre-processing step before the evaluation process. we employ the authors' and readers' (in case they are available) keyphrases as a gold evaluation standard for all dataset collections. finally, we use a two-sided wilcoxon signedrank test to check the statistical significance of the results in terms of the most popular exact match evaluation between the proposed input types and the conventional ones, at a significance level of 0.05. we denote with a "*" the statistical significance with ta and with a " †" the statistical significance with abse or f (in cases there is an improvement). ## results and discussion one disadvantage of extractive summaries, is that they require an additional pre-processing step compared to the rest pre-existing textual content types. the average time to generate the extractive summary per document in the machine used for the experiments is 2.21, 2.13, and 2.34 seconds for the semeval, nus, and acm datasets, respectively. this is not high for offline applications, while for online ones, higher scale hardware and/or more efficient architectures could be employed. ## conclusions and future work our work set out to investigate whether using automated summarization, as a pre-processing step, can lead to improved results in the task of keyphrase extraction from scholarly documents. our empirical study shows that unsupervised approaches improve their accuracy using extractive summaries as input, highlighting the full-text's useful information for the task and showing a positive relationship between the tasks of extractive summarization and keyphrase extraction. it is worth noting that even though the gains on the exact match f 1 -scores seem to be moderate, this does not necessarily reflect the actual performance gain. considering that exact match scores are generally low due to the strict nature of the method, a moderate increase in performance leads to considerable percentage gain over the initial performance. as future work, an interesting direction would be to experiment with additional summarization methods, including abstractive ones as well as their combination with extractive ones. in addition, we could experiment with additional recent and stateof-the-art keyphrase extraction methods, including methods building on top of contextual embeddings @xcite .