Keyphrase Extraction with Incomplete Annotated Training Data (2021)

Abstract: Extracting keyphrases that summarize the main points of a document is a fundamental task in natural language processing. Supervised approaches to keyphrase extraction(KPE) are largely developed based on the assumption that the training data is fully annotated. However, due to the difficulty of keyphrase annotating, KPE models severely suffer from incomplete annotated problem in many scenarios. To this end, we propose a more robust training method that learns to mitigate the misguidance brought by unlabeled keyphrases. We introduce negative sampling to adjust training loss, and conduct experiments under different scenarios. Empirical studies on synthetic datasets and open domain dataset show that our model is robust to incomplete annotated problem and surpasses prior baselines. Extensive experiments on five scientific domain datasets of different scales demonstrate that our model is competitive with the state-of-the-art method.

## introduction keyphrase extraction is the task of automatically extracting a set of phrases that provide a concise summary of a document content. an effective keyphrase extraction (kpe) system can benefit a wide range of natural language processing and information retrieval tasks @xcite , such as text categorization @xcite , opinion mining @xcite and recommendation @xcite . recent supervised neural methods typically treat keyphrase extraction as a classification problem @xcite @xcite , where given phrases are classified as keyphrases or nonkeyphrases. although supervised methods perform well in this task, it requires a large amount of labeled data which is extremely expensive and timeconsuming to collect in many application scenarios @xcite . the latest advances in neural kpe are mainly carried out on data sets in scientific domain datasets. @xcite . because scientific domain has sufficient and high-quality training data for neural methods: some training data are annotated by authors, and the author is the person who is most familiar with their articles, the keyphrases annotated by author are high-quality. however, due to the requirements for professional knowledge in scientific domain, some keyphrases annotated by readers are incomplete. it has been suggested that keyphrase annotation is highly subjective @xcite . in real world scenarios, most potential applications of kpe deal with diverse documents originating from sparse sources that are rather different from scientific papers @xcite . they often involve various domains whose contents target much wider audiences than scientists and require a large amount of high-quality labeled data which is extremely expensive to collect @xcite . there are several prior works focused on data quality and insufficient labeled data issues, sterckx et al. @xcite treat supervised keyphrase extraction as positive unlabeled learning @xcite by reweighting importance of training samples. wang et al. @xcite used the idea of transfer learning and proposed topic-based adversarial neural network (tann). they exploited unlabeled data in the target domain and data in the resource-rich source domain to alleviate incomplete annotated problem. however, ## related work in this section, we briefly review two classes of closely related works: keyphrase extraction approaches and learning with noisy data. ## methodology inspired by the effectiveness of negative sampling in the field of ner @xcite , we introduce negative sampling to keyphrase extraction field to solve incomplete annotated problem. cnn-based keyphrase extraction model will be presented firstly, and then we will introduce how to use negative sampling to improve the training process to make the model more robust. ## conclusion and future work in this work, we introduce negative sampling to keyphrase extraction model, to alleviate the misleading of the incomplete annotated problem. according to the length characteristics of the keyphrase, we use sliding cnn to extract the representation of the candidate gram, turn the keyphrase extraction problem into binary classification. in different scenarios and different scales datasets, we have confirmed the effectiveness of our model. our work can be extended in many directions. to begin with, currently binary classifier in our model treats each candidate gram individually, but there is usually a relationship between different grams. we could leverage attention mechanism to model the relationship between grams. moreover, we would like to further explore utilizing smart gating units or designing a network layer to filter unlabeled keyphrases.