Automatic Keyphrase Generation by Incorporating Dual Copy Mechanisms in Sequence-to-Sequence Learning (2022)

Abstract: The keyphrase generation task is a challenging work that aims to generate a set of keyphrases for a piece of text. Many previous studies based on the sequence-to-sequence model were used to generate keyphrases, and they introduce a copy mechanism to achieve good results. However, we observed that most of the keyphrases are composed of some important words (seed words) in the source text, and if these words can be identified accurately and copied to create more keyphrases, the performance of the model might be improved. To address this challenge, we propose a DualCopyNet model, which introduces an additional sequence labeling layer for identifying seed words, and further copies the words for generating new keyphrases by dual copy mechanisms. Experimental results demonstrate that our model outperforms the baseline models and achieves an obvious performance improvement.

## introduction a keyphrase is a short piece of text that summarizes and abstracts the main semantics of a long text (named "document" or "source text" in this study). high-quality keyphrase promotes readers to efficiently understand, summarize and access documents' content @xcite . not only that, extracting high-quality keyphrase had been widely applied to many downstream tasks in natural language processing and data mining, such as information retrieval @xcite ), text summarization @xcite ), text categorization @xcite and opinion expression mining @xcite . thus, how to automatically extract high-quality keyphrases has become popular research topics in recent decades @xcite . due to accessibility of text data, many datasets from scientific articles are used as benchmarks for keyphrase extraction algorithms. thus, our study also focuses on extracting keyphrases from the scientific articles. generally speaking, keyphrases can be divided into two categories: present keyphrase and absent keyphrase. present keyphrases are the explicit words that appear directly in source text, and vice versa for absent keyphrase. many previous studies have focused on how to extract present keyphrases from documents. these studies @xcite @xcite @xcite consider the keyphrases extraction as a ranking task, which extracts a set of candidate phrases from the source text, and then selects keyphrases from the sorted candidates with the higher importance score. in recent years, some studies have also attempted to use deep learning methods for present keyphrase extraction. for example, @xcite considered the present keyphrases extraction as a sequence labeling task by using lstm-crf model to label sequence, and obtained a better performance. @xcite used popular bert @xcite model to extract present keyphrases. however, these methods were not expert in extracting absent keyphrases because the source text has no absent keyphrase. to solve this problem, some sequence-to-sequence @xcite based models were used to generate present/absent keyphrases, such as @xcite @xcite @xcite . they first encoded the source text and then dynamically outputted corresponding present/absent keyphrases through a decoder. however, the decoder usually generates high-frequency words and ignores many out-ofvocabulary words, so it is unsatisfied in the task. to address this problem, @xcite incorporated a copy mechanism @xcite in decoder to successfully predict out-of-vocabulary (oov) words, namely oov words copy mechanism. besides, in many scientific articles, we observe that some words repeatedly appear in many keyphrases. for example, as shown in figure 1 , such as "index" appears in keyphrases "nextword index, index representation, inverted index", and "query" appears in keyphrases "query evaluation, phrase query" (these words are called seed words in this study). therefore, if these seed words from source text can be identified and applied by the decoder to generate keyphrases as much as possible, the performance of the model will be greatly improved. to address this challenge, we propose a novel sequence-to-sequence model (named dualcopy-net) to incorporate dual copy mechanisms for generating present/absent keyphrases. since there is no labeled data, it is very difficult to directly extract the seed words from the source text. thus, we try to extract present keyphrases as seed words. specifically, besides a canonical encoder layer in dualcopynet, we also introduce a sequence labeling layer for extracting present keyphrases (seed words). in addition, in the decoder layer, we introduce two kinds of copy mechanism. seed words copy mechanism, it enables the decoder to generate phrases by selecting appropriate words from seed words. oov words copy mechanism, it is a feasible solution that enables the decoder to predict oov words by selecting appropriate words from the source text @xcite . moreover, the decoder of dualcopynet softly fuses the dual copy probability and generation probability through a gate mechanism to copy words (seed or oov words) from the source text and generate words from the vocabulary. when training the model, we use a multi-task learning approach to op-timize the primary task (generating keyphrases) and the auxiliary task (predicting seed words). finally, we conduct experiments on four datasets. the results show that dualcopynet has an obvious performance improvement in predicting present/absent keyphrases. the contributions of our paper are as follows: ## experimental settings in this section, we will describe the training and testing sets used for the experiments, then introduce the baseline models and evaluation metrics. ## conclusions in this paper, we propose a novel dualcopynet for keyphrases generation. based on the phenomenon of that many keyphrases are composed of seed words in the source text, we design dual copy mechanisms to precisely copy seed words and oov words from the source text. furthermore, aiming to obtain seed words, we introduce an additional sequence labeling layer and train the model with a multi-task learning. finally, the experiments conducted on multiple datasets show our model's achievements are higher than most of baselines. meanwhile, ablation experiments show a positive effect of the seed word copy mechanism and multitask learning.