KEPL : Knowledge Enhanced Prompt Learning for C hinese Hypernym-Hyponym Extraction (2023)

Abstract: Modeling hypernym-hyponym (“is-a”) relations is very important for many natural language processing (NLP) tasks, such as classification, natural language inference and relation extraction. Existing work on is-a relation extraction is mostly in the English language environment. Due to the flexibility of language expression and the lack of high-quality Chinese annotation datasets, it is still a challenge to accurately identify such relations from Chinese unstructured texts. To tackle this problem, we propose a Knowledge Enhanced Prompt Learning (KEPL) method for Chinese hypernym-hyponym relation extraction. Our model uses the Hearst-like patterns as the prior knowledge. By exploiting a Dynamic Adaptor Architecture to select the matching pattern for the text into prompt, our model embeds patterns and text simultaneously. Additionally, we construct a Chinese hypernym-hyponym relation extraction dataset, which contains three typical scenarios, as baike, news and We-media. The experimental results on the dataset demonstrate the efficiency and effectiveness of our proposed model.

## introduction hypernym discovery is a core work in taxonomy construction @xcite . hypernym relation is a semantic relation that exists between a term (hyponym) and a more general or abstract term (hypernym). due to its capacity for representing semantic relations, hypernym becomes an essential concept in modern natural-language research, and is a fundamental component in many natural language processing (nlp) tasks, such as question answering @xcite , taxonomy construction @xcite and personalized recommendation @xcite . typical efforts in hypernym discovery can be roughly classified into two main types: rule-based methods and detection-based methods. rulebased methods @xcite @xcite @xcite rely on predefined linguistic rules or patterns to extract hyponym-hypernym relations. these rule-based methods capture specific syntactic or semantic structures that indicate a hierarchical association between terms, however, rule-based methods lack sufficient capability to discover the hyponym-hypernym relation embedded in the semantic content of sentences. for example, in the sentence 'jay chou, an acclaimed singer, mesmerizes audiences with his soulful performances.' the rule-based approach might fail to recognize the hyponym-hypernym relation between 'jay chou' and 'singer'. another effective way is to model this problem as a hypernym detection task @xcite @xcite . these methods handle the hypernym detection task in a pipelined manner, i.e., extracting the entities first and then recognizing hypernym relations. this separate framework makes each component more flexible, but it neglects the relevance between two sub-entities. identifying hyponym-hypernym relation by modeling the interaction of sparse attributes, many studies are dedicated to constructing english hypernym-hyponym relationship datasets @xcite . meanwhile, chinese language has unique linguistic characteristics and categories that need to be considered @xcite . the lack of such datasets has hampered the progress of chinese hyponymy extraction research. even though chinese speakers account for a quarter of the world population, there has been no existing high-quality dataset for chinese hypernym relation extraction. in this paper, we propose the knowledge enhanced prompt learning (kepl) method which utilizes both knowledge and latent semantics for end-to-end chinese hypernym relation extraction. specifically, our method employs a dynamic adaptor for knowledge which can adaptively construct a unified representation for both the structured prior knowledge and the unstructured text context. to facilitate a more coherent integration of the structured prompts and unstructured text, we employ a mechanism that learns a unified representation of context through specific attention. for span selection, we use the focal loss to counteract the issue of sample imbalance, a commonly observed phenomenon in extractive tasks. the lack of specific hypernym relation datasets also leads to deficiencies in current works @xcite . to address this challenge, we propose the chr dataset that aims to improve the coverage and accuracy of hypernym relations in taxonomies. we believe that our dataset can contribute to the development of more accurate and comprehensive taxonomies. the main contributions of our work are summarized as follows: ## related work hypernym dection research into hypernym relation extraction has mainly used unsupervised methods, falling into two categories: patternbased and distributional approaches. the pattern-based approach @xcite @xcite , established by @xcite , employs specific predefined linguistic patterns, such as 'isa' and 'including', to detect hypernym relations. while this approach is simple and widely applicable, it is constrained by its reliance on predefined patterns, sensitivity to sentence structure, and the necessity for manual resource curation. distributional approaches like @xcite use a distant supervision method for extracting hypernyms from various sources. their models produce a list of hypernyms for a given entity. subsequently, @xcite highlights the unique performance of the baroni dataset in providing consistent results, attributing its effectiveness to its alignment with specific dimensions of hypernymy: generality and similarity additionally, hybrid methods (bernier-colborne and @xcite @xcite ) that amalgamate different techniques have been explored. for instance, @xcite proposed a method that merges hyperbolic embeddings with hearst-like patterns, resulting in better performance on various benchmark datasets. prompt learning prompt-based learning, a novel paradigm in pretrained language models @xcite , restructures downstream tasks to better align with pre-training tasks, enhancing the model's performance. a notable application of this approach is demonstrated by @xcite , where classification problems are transformed into cloze tasks. this is achieved by creating relevant prompts with blanks and establishing a mapping from specific filled words to the corresponding predicted categories. this method effectively bridges the gap between the task and the model's training. furthermore, @xcite introduces a model named paie, which leverages prompts for event argument extraction (eae) at both sentence and document levels. this innovative use of prompts in eae tasks showcases the versatility and efficiency of the prompt-based learning approach. ## data we introduce the chr (chinese hypernym recognition) dataset -an innovative resource that specifically addresses the current shortcomings in chinese hypernym discovery. the key idea of constructing the chr dataset is to enhance the quality and diversity across various domains, which are currently insufficient in existing resources. ## method in this section, we describe our knowledge enhanced prompt learning (kepl) framework, specifically designed to extract informative hypernym arguments from text documents. the overall structure of kepl is depicted in figure 2 . ## experiment in this section, we evaluate the efficacy of the proposed kepl model. the evaluation experiments are presented in section 5.1 and the results are discussed in section 5.2. furthermore, we assess the effect of varying the number of knowledge prompts in section 5.3. an ablation study can be found in section 5.4. ## conclusion in this paper, we introduce knowledge enhanced prompt learning (kepl) for extracting hypernym-hyponym relations in chinese language. kepl utilizes the concept of prompt learning to incorporate prior knowledge in the form of patterns into the model, which simultaneously embeds both the pattern and text. the prompt in the framework uses hearstlike patterns, specifically for extracting hypernymhyponym relations. additionally, we have created a chinese hypernym-hyponym relation extraction dataset, which includes three different types of scenarios: wikipedia, news articles, and we-media. the results of our experiments using this dataset show that our proposed model is both efficient and effective.