KEPL: Knowledge Enhanced Prompt Learning for Chinese Hypernym-Hyponym Extraction
Authors: Ningchen Ma
Dong Wang
Hongyun Bao
Lei He
Suncong Zheng
Venue: EMNLP 2023 Main (2023)
Keywords: Knowledge Base
prompt
Hypernym discovery

Abstract: Modeling hypernym-hyponym ("is-a") relations is very important for many natural language processing (NLP) tasks, such as classification, natural language inference and relation extraction. Existing work on is-a relation extraction is mostly in the English language environment. Due to the flexibility of language expression and the lack of high-quality Chinese annotation datasets, it is still a challenge to accurately identify such relations from Chinese unstructured texts. To tackle this problem, we propose a Knowledge Enhanced Prompt Learning (KEPL) method for Chinese hypernym-hyponym relation extraction. Our model uses the Hearst-like patterns as the prior knowledge. By exploiting a Dynamic Adaptor Architecture to select the matching pattern for the text into prompt, our model embeds patterns and text simultaneously. Additionally, we construct a Chinese hypernym-hyponym relation extraction dataset, which contains three typical scenarios, as baike, news and We-media. The experimental results on the dataset demonstrate the efficiency and effectiveness of our proposed model.

KEPL:
Knowledge Enhanced Prompt Learning for Chinese HnteenyymHyponym Extraction
Ningchen Ma Dong Wang"; Hongyun Bao"; Lei He', Suncong Zheng²
"China University of Mining & Technology, Beijing
Tencent Machine Learning Platform, Tencent, Beijing, China
Institute of Automation, Chinese Academy of Sciences, Beijing, China
maningchene901@@mee com, hongyun.bao@ia.c bao@ia.aaaaaaa....o ac (thudongwang, bettyleihe Ctgg congeeeeggeeeeeeeeeenttc..
Abstract
Typical efforts in Hypemyy Discovery can be roughly classified into two main types: rule-based methods and detection- based methods. Rule- based methods (Auger and Barrière, 2008; Kliegr et al., 2008; Seitner et al., 2016; Snow et al., 2004; Wang et al., 2017) rely on predefined linguistic rules or patterns to extract hyponym-hypernym re- lations. These rule-based methods capture spe- cific syntactic semantic structures that indicate hierarchical association between terms, however, rule-based methods lack sufficient capability to discover the hyponym-hypernyy relation embed- ded in the semantic content of sentences. For ex- ample, in the sentence 'Jay Chou, an acclaimed singer, mesmerizes audiences with his soulful per- formances.' the rule-based approach might fail recognize the hyponym-hhpeenyym relation be- tween 'Jay Chou' and 'singer'.
Modeling hypprnym-hyponym ("is-a") rela- tions is important for many natural language processing (NLP) tasks, such as classification, natural language inference and relation extrac- tion. Existing work on is-a relation extraction is mostly in the English language environment. Due to the flexibility of language expression and the lack of high-quality Chinese annota- tion datasets, is still challenge to accurately identify such relations from Chinese unstruc- tured texts. To tackle this problem, we pro- pose Knowledge Enhanced Prompt Learn- ing (KEPL) method for Chinese hypernym- hyponym relation extraction. Our model uses the Hearst-like patterns the prior knowl- edge. By exploiting a Dynamic Adaptor to se- the matching pattern for the text into the prompt, method simultaneously embed- ding patterns and text. Additionally, we con- struct Chinese hypemym-hyponym relation extraction dataset, which contains three typ- ical scenarios, as Baidu Encyclopedia, news and We-media. The experimental results the dataset demonstrate the efficiency and ef- fectiveness of our proposed model.
Another effective way is to model this problem as a hypernym detection task (Dash al., 2020a; Roller et al., 2018; Yamane et al., 2016). These methods handle the hypernym detection task in a pipelined manner, i.e., extracting the entities first and then recognizing hypernym relations. This separate framework makes each component more flexible, but it neglects the relevance between two sub-entities.
1 Introduction
Hypernym Discovery is core work in laxonomy construction (Wang al., 2019). Hypernym rela- tion a semantic relation that exists between a term (hyponym) and more general or abstract term (hypernym). Due to its capacity for repre- senting semantic relations, hypernym becomes an essential concept in modern natural-language re- search, and is fundamental component in many natural language processing (NLP) tasks, such as question answering (Yang al., 2017: Yu et al., 2021), taxonomy construction (Chen al.. 2019; Luo et al., 2020) and personalized recommenda- tion (Huang et al., 2019).
Identifying hyponym-hypernyne relation by modeling the interaction of sparse attributes, many studies are dedicated to constructing English hypernym-hyponym relationship datasets (Berend et al., 2018; BeBnniee-rroooornee and Barrière, 2018). Meanwhile, Chinese language has unique linguis- tic characteristies and categories that need be considered (Zhang et al., 2022b). The lack of such datasets has hampered the progress Chinese hy- ponymy extraction research. Even though Chinese speakers account for a quarter of the world pop- ulation, there has been no existing high-quality dataset for Chinese hypernym relation extraction. In this paper, we propose the Knowledge En-
"These authors contributed equally to this work.
"Corresponding Author

---

INSTANCE FROM NEWS
Nie Hivogiis,
Zufong Advisess, Secerities Daiyy
IT 1R 3e 28 3000/1000 Rbly 1E 10 Sp TILE
contanaees high Ieeperature mony places las put grent peesare or the power suppls and cosi "bellase" Su sabilelee 7t,
pannes HiW BREE S:58 scmec 96 2b 10 Bis W1th
asage
of
saccaae lhe satnita and supple ofuoed, and kiak sumperatue tn vesance will unaress dae suc cuade sel sntural gas. aon
7F 6SFF.
The Input Sample
moutaees
pocupptoon
Hypernym- m- m-Ilypong m Tuple
senter incestevntt convrilont hiflnng Invesmmet Adviowyy E3: Sooi St:2
SAcRice amrorrant hafony imvotmar Adbricry
baskethall puancr
Xie Honqis
You Stas
Figure 1: Hypernym discovery for taxonomy construction
hanced Prompt Learning (KEPL) method which utilizes both knowledge and latent semantics for end-to-end Chinese hypernym relation extraction. Specifically, our method employs Dynamic Adaptor for Knowledge which can adaptively con- struct unified representation for both the struc- tured prior knowledge and the unstructured text context. To facilitate a more coherent integration of the structured prompts and unstructured text, we employ mechanism that learns unified rep- resentation of context through specific attention. For span selection, use the focal loss to coun- teract the issue of sample imbalance, commonly observed phenomenon in extractive tasks.
leverages prior knowledge as prompts and trans- fers prior knowledge into extraction task. Our framework learns unified representation of con- text through specific attention, proving effective for various natural language processing tasks, cluding but not limited to taxonomy construction and semantic search.
3. Our extensive experiments on the proposed dataset have shown that the KEPL framework achieves 2.3% improvement in F1 over the best method, demonstrating the effectiveness of our ap- proach. Further, we show the results of individu- ally removing components from the trained KEPL on CHR dataset and proved the effectiveness of each component.
The lack of specific hypernym relation datasets also leads to deficiencies in current works (Chen et al., 2019; Luo al., 2020). To address this chal- lenge, we propose the CHR dataset that aims to im- prove the coverage and accuracy of hypernym re- lations in taxonomies. We believe that our dataset contribute to the development of more accu- rate and comprehensive taxonomies.
Related Work
Hypernym Dection Research into hypernym re- lation extraction has mainly used unsupervised methods, falling into two categories: pattern- based and distributional approaches.
The pattern-based approach (Navigli and Ve- lardi, 2010; Boella and Di Caro, 2013; Vyas and Carpuat, 2017: Bott et al., 2021), established by (Hearst, 1992; Wang and He, 2020), employs spe- cific predefined linguistic patterns, such as 'is- a' and 'including', to detect hypernym relations. While this approach is simple and widely applica- ble, it is constrained by its reliance predefined patterns, sensitivity sentence structure, and the necessity for manual resource curation. Distribu- tional approaches like (Fu et al., 2014) use dis- tant supervision method for extracting hypernyms from various sources. Their models produce a list
The main contributions of our work are summa- rized as follows:
1. To the best of our knowledge, there cur- rently cxists commonly used dataset for Chi- nese hypemnym-hyponym discovery. We construct Chinese hypernym relation extraction dataset, which contains three typical scenarios, as follows as Baike, news and We-media. The proposed dataset with multiple data sources can well cover specific expressions in various corpus.
We propose a novel framework, the Knowl- edge Enhanced Prompt Learning (KEPL) which

---

of hypernyms for given entity. Subsequently, (Sanchez and Riedel, 2017) highlights the unique performance of the Baroni dataset in providing consistent results, attributing its effectiveness to its alignment with specific dimensions hypernymy: generality and similarity
We gather the Baike data from variety of rep- utable Baidu Baike online encyclopedias. How- ever, we strategically omitted entries that were too short, lacked contextual richness, or had content outside the scope of our study, such as stub en- tries and disambiguation pages. The We-Media data was gathered from a wide range of accounts as lifestyle, entertainment, technology, and educa- tion. Our scraper was programmed regularly check these accounts for the latest articles and re- trieve historical articles where possible. Advertise- ments and unrelated links were excluded from our dataset; we focused only on preserving the main content of the articles.
Additionally, hybrid methods (Bernier- Colborne and Barrière, 2018; Dash al., 2020b; Yu et al., 2020) that amalgamate different techniques have been explored. For instance, (Held and Habash, 2019) proposed a method that merges hyperbolic embeddings with Hearst-like patterns, resulting in better performance on various benchmark datasets.
Our dataset includes news data from multiple high-profile news platforms such as Xinhua News Agency, Tencent News, and Sina News. To elim- inate domain bias, we selected articles from broad spectrum of categories including politics, economies, sports, culture, and science, and we eliminated articles with insufficient text, duplicate articles, and those not pertaining to our research from the dataset.
Prompt learning Prompt-based learning, novel paradigm in pretrained language models (Zhang et al., 2022a), restructures downstream tasks to better align with pre-training tasks, en- hancing the model's performance. A notable appli- cation of this approach is demonstrated by (Schick and Schütze, 2021), where classification problems are transformed into cloze tasks. This is achieved by creating relevant prompts with blanks and es- tablishing mapping from specific filled words to the corresponding predicted categories. This method effectively bridges the gap between the task and the model's training. Furthermore, (Ma et al., 2022) introduces model named PAIE, which leverages prompts for Event Argument Ex- traction (EAE) at both sentence and document levels. This innovative use of prompts in EAE tasks showcases the versatility and efficiency of the prompt-based learning approach.
3.2 Data construction
Pre-processing After gathering data from the various sources, remove irrelevant features which contain HTML tags, special characters, and formatting. We also performed tokenization, con- verting sentences into tokens for further process- ing.
Sentences with a character count falling below the established threshold of 10 characters were dis- regarded. Those exhibiting an overuse of punctua- tion marks such as commas, colons, and periods were excised. Additionally, sentences embody- ing numerals or particular non-Chinese characters were filtered out.
3 Data
We introduce the CHR (Chinese Hypernym Recog- nition) Dataset - an innovative resource that specifically addresses the current shortcomings in Chinese hypernym discovery. The key idea of con- structing the CHR dataset is to enhance the quality and diversity across various domains, which are currently insufficient in existing resources.
The sentences that have passed through these filters are then manually labeled with the appro- priate hyperymm-hyponym relation by our team. We mark the entity position by determining the start entitystart and end entity.nd positions of the answer in the text, aiming for accurate iden- tification of the relation scope. If sentence does not contain explicit hypernym-hyyonynn entities, the corresponding positions are marked as NONE. Following the above extraction, the potential pairs were presented to a team trained individuals for review.
3.1
Data source
To address the existing limitations in Chinese hy- pernym discovery, particularly the lack of diver- sity, we constructed the CHR dataset.
Our dataset constructed by incorporating data from three distinct sources: encyclopedic knowl- edge, We-Media public accounts and news.

---

Final Format We streamline and structure our data into a uniform and easily accessible format, facilitating subsequent analysis and modeling. We use the data consisting of set of (data, spann,)pan2), Which spanl represents the latent position of hypernym, span2 represents the latent position of the hyponym. An excerpt for one ample: 4e fo fir
being selected as the start or end of each role fea- ture in Section 4.4.
4.2 Dynamic Adaptor for Knowledge
In the proposed KEPL model, our aim is to con- struct a unified representation for both the struc- tured prior knowledge and the unstructured text context. To achieve this, we use the Hearst method which facilitates the identification of hypernym ("is-a") relations within sentence by exploit- ing certain lexico-syntactic patterns. By introduc- ing this prior knowledge in the form lexico- syntactic patterns, we aim to increase the model's s capability to precisely identify hypernym and hy- ponym relations.
(link checker) A Hi A P4 3hi0 DI IiI I x
Ke Pi Ff.
this example, HiJ the hypernym of the i Ba
The format of the output is set as < #F i to D. We conducted a thorough exami- nation of the data and implemented a meticulous annotation process to ensure the quality and relia- bility of the dataset, the detail can be seen in Table 1.
Firstly, we represent each structured prior knowledge (the lexico-syntactic patterns) in the form of prompts, and use the corresponding prompt representations, Ep as follows:
Table 1: The detail of the proposed CHR dataset
Context
Hypernym
Train 21376
12006
Ep = Late (pmi)
(I)
Dev 2672
1431
Test
2672
1475
Here, pmi represents a set of structured prompts with Hearst patterns, and E,, denotes the corre- sponding embeddings obtained from the decoder module Liec of the pre-trained language model.
4 Method
this section, we describe our Knowledge Enhanced Prompt Learning (KEPL) framework, specifically designed to extract informative hyper- nym arguments from text documents. The overall structure of KEPI. is depicted in Figure 2.
Given prompt representation Ep and a set of prompt pmi, for different input 8j, we generate a scoring matrix w E R!xx, to select the suit- able template combination under specific seman- tics, this progress can be expressed by:
4.1 KEPL framework
exp exp WxiT +b) erp(We T + b)
Preliminary We focus on the problem of hyper- nym discovery. Each document is represented as doc (di,.... dN} and is mapped to a set of spans, {S1, S2. ... Sg}. using specific prompt Pm; Here, each span consists of pair HugHg where Ha denotes hyponym and Hg rep- resents a hypernym..
= = EM
Ppp
(2)
where Epp is the set of available prompts embed- dings. Ey C RMXL*H is a weighted representa- tion that blends the adapted prompt embeddings, incorporating both the input sentence and the en- tire adjusted prompt set's semantic information, and W RL*H is learned weight matrix that is used adapt the selected prompt match the semantics of the input sentence 8;.
KEPL Structure Our KEPL structure is based on sequence-to-sequeece pre-trained language model, primarily used for text generation. We first explain how we inject different knowledge into given sentence (sec 4.2). Subsequently, attention mechanism learn Unified Repre- sentation for Templates and Context, allowing us to capture valuable information from both struc- tured prompts and unstructured text in Section 4.3. Lastly, we calculate the distribution of each token
The Hearst method (Hearst, 1992) is an effi- cient method for identifying hypernym (is-a) lation from given sentence with exploiting cer- tain lexico-syntactic patterns. To attenuate the impaction from the specific prompt, we use the Hearst pattern as a representation of knowledge. We give examples of these prompts in Table 2.

---

Knowledge Attention Layer
Dynomic Knowlodge Adapio
Contact
Scaled ProdPrt Atlention
Encoder
Decuder
Temty!
S
Pm
Figure 2: The overall framework of KEPL
Table Examples prompt used KEPL
learn a unified representation through specific tention. Specifically, shown in Figure 2, we use semg, semy, Eg to compute an attention out- put Tott E RL*H This process can be expressed as follows:
Prompt
ft 1 +
Object represents a classification サ 1
semq: sem," ' ddg
The object signifies a classification
Falt = softmax
Temy (4)
Objects belong to the same classification
x -
where semg, semy are the linear projection the L(si). Temy E RL*H represent the Ep E RLxH through linear layer. This advancement is leveraged to facilitate enhanced interplay between templates and input information, promoting more effective integration of structured prompts and un- structured text.
The object type of classification
4.3
Unified representation for templates and context
To facilitate a more coherent integration of the structured prompts and unstructured text, we em- ploy attention mechanism, we learn unified representation of context:
4.4 Knowledge-guided extraction
In the process of knowledge-guided extraction, we aim to calculate the distribution each token for the start or end of given role based on the unified context- context-prompt prompt representation. The logits reflect the unnormalized log probabilities of each token being the starting or ending positions of target hypernym hyponym. They are calculated via the linear projection from Eatt follows:
(C C (s:)
(3)
where ((()) E RLXH denotes the context repre- sentation. L now refers the Language Model (LLM) and H is the dimension of the contexts. This progression in the approach leveraged to effectively integrate context and prompts.
To capture useful information and uncertainty from each view (knowledge and context), we
start/end start/ead Erj"
logits, x
(5)
atd

---

stort//nd att'' are the linear projections of Fall
Here, x that are computed separately for the start and end positions of a target feature (either hypernym or hyponym). Fatt' E RI'H represents the linear projection of Tatt: Fyy" E R and logits; E R symbolizes the contextual token distributions. Noted that different span Spanpi will result in dif- ferent corresponding Tott.
and various news outlets. This CHR dataset pro- vides more practical perspective on the applica- bility of our KEPL framework, as encompasses a wide range of language styles and topics.
Our experiments have been em- ployed on CHR datasets. We briefly describe each below. In order to evaluate the performance of our experiments on CHR datasets, we employ preci- sion, recall and Fl our primary evaluation indi- cators.
Evaluation
The logits are the scores assigned to each token in the input sentence and they measure the likeli- hood of each token being the start or end token of the span of interest. Once these logits are trans- formed into probabilities through softmax oper- ation, these values are utilized to determine the start and end positions for hypernym and hyponym spans the text.
Baselines We perform a comparison with the current state-of-the-art and also with some classi- cal models show the efficiency and effectiveness of the proposed KEPL, model.
We observed sample imbalance in the extractive task, hence we utilize the focal loss function as follows:
W2NER(Li et al., 2022) which discontinuous NER (8 English and 6 Chinese datasets) top- performing baselines
FL(P;) = (1 p:)7 log (p:)
(6)
ESuie3(Sun et al., 2021) is a SOTA pretrained Chinese model outperforms the state-of-the- art models on 54 Chinese NLP tasks
Pi = logits;
Σ logils)
UlE(Lu et al., 2022) excels in Chinese infor- mation extraction tasks and is wildly used.
In these equations, represents the predicted probability, and to avoid the exhaustive threshold tuning, we use the softmax function compute these probabilities.
Implement Details The CHR Dataset em- ployed in our experiments. divided into training, validation, and test sets in a 0.7:0.15:0.. 15 ratio. The model was trained with a batch size of 32 and a learning rate of 0.0001 using the Adam op- timizer. The models were trained for five epochs, with early stopping implemented to prevent over- fitting.
The connotation for starting and ending posi- tions of the target span & is illustrated as:
(86) set (1, E mag score: (i,5) (8)
The pair (i.j) that maximizes score (i, which gives us the span position of hypernym or hyponym in the sentence.
5.2 Experiment Results
We report the results of different methods in Ta- ble 3. A more visual presentation of the direct sults from our experiment can be appreciated in Table 4. It can be seen that our method outper- forms all other methods in Fl score and achieves a 2.3% improvement in Fl over the best method UIE (Lu et al., 2022). It shows the effectiveness of our proposed method. Furthermore, from Table I, we also can see, unlike other methods that predom- inantly rely localized features word-level lationships, KEPL models effectively account for the broader semantic context. This approach en- hances the accuracy of inferring hypernym rela- tionships, even in the absence of explicit markers for hierarchical relationships.
5 Experiment
In this section, we evaluate the efficacy of the pro- posed KEPL model. The evaluation experiments are presented in Section 5.1 and the results discussed in Section 5.2. Furthermore, the effect of varying the number of knowledge prompts in Section 5.3. An ablation study can be found in Section 5.4.
assess
5.1 Evaluation and Settings
Dataset Our KEPL framework further evalu- ated using the CHR Dataset, compiled from three diverse sources: We-Media, Baidu Encyclopedia,

---

Table 3: Different Performance of KEPL on Different Data Sources
Method
Baike
We-Media
News
R
P
F1
R
P
F1
R
P
F1
W2NER
0.9056
0.5655
0.8703
0.5028
0.5655
0.5323
0.7185
0.6841
0.7009
Ernie3
0.9773
0.8356
0.9009
0.7926
0.3987
0.5306
0.9642
0.5844
0.7277
UIE 0.9769
0.8208
0.8921
0.7407
0.6348
0.7273
0.9574
0.6348
0.7634
KEPL-Bart
0.9785
0.8847
0.9293
0.7466
0.6728
0.7078
0.9067
0.8861
0.8294
KEPL-Emicc
0.9861
0.8861
0.9334
0.8613
0.6327
0.7295
0.9543
0.7717
0.8534
Table 4: Different model results for Chinese sentence
Instance
2  SINGG 2
Chatgpt
Lexcial
Pattern
X L M M
Baseline model
Our Method
Adaptor prompt
5.2.1 Results on CHR Dataset
1. Unstructured text lacks a predefined struc- ture explicit markers for hierarchical rela- tionships, making it difficutt to discern the underlying organization. Hierarchies are of- ten implied through contextual cues, such as sentence structure, semantic relationships, or proximity of concepts, requiring sophisti- cated natural language processing techniques to identify and extract these relationships ac- curately.
The KEPL-Bart and KEPL-Emie3 methods achieved higher rates when compared to other models such as W2NER, Ernie3, and UIE. This demonstrates the efficacy of the KEPL approach, particularly the effectiveness of incorporating Hcarst-like patterns as prompts and embedding patterns and text simultaneously.
The performance enhancements of our KEPL. models are largely attributed to the implemen- tation of the Dynamic Adaptor for Knowledge and knowledge attention modules, which effec- tively mediate the interaction between structured prompts and unstructured text, contributing to su- perior hypemym-hhyonym extraction.
2. The other methods are unable recognize the<Hypernym. Hyponym> appears pairs, they predominantly rely on localized fea- tures or word-level relationships, thereby dis- regarding the encompassing semantic context required for accurate inference of hypernym relations.
Table 5: Metrics on the test set for the CHR
5.2.2 Results on specific scene
R
Micro Fl
Fl
We compare KEPL, performance with the baseline model on each scene in Table 3, observing the re- sults on the Baidu, We-Media, and News datasets, the performance of the KEPL models varies.
W2NER
0.5958
0.5488
0.6743
Ernie3
0.8090
0.7164
0.7826
UIE
0.8846
0.7992
0.8432
KEPL-Bart 0.9138
0.8324
0.8665
Compared to other models, KEPL models show higher scores in almost all metrics across differ- datasets. This can be attributed to the Hearst pattern selector, which adapts different sentence semantics, and the unified representation for tem- plates and context, which effectively captures the
KEPL-Ernie3
0.9413
0.8471
0.8701
The evaluation results of different methods are shown in Table 5, from which we have several ob- servations:

---

useful information from both the pattern and the text.
enabling it capture and utilize the rich hierarchi- cal information present in the text.
The results highlight the effectiveness of the KEPL approach in handling different text com- plexities and styles, thereby exhibiting promising potential for real-world hypernym-hyponym ex- traction tasks. Nevertheless, improvements could be made better handle datasets with informal and context-specifie language like the We-Media data.
In the atm w/o experiment, we deliberately trained KEPL KKnowledge-Enhang Learning) without sign choice resulted in reduction in performance, primarily due to the model's compromised ability to effectively integrate context and prompts. Atten- tion mechanisms play crucial role in enabling the model to attend to relevant parts of the input and appropriately align them with the given prompts. Therefore, the absence of attention mechanisms in our experiment negatively impacted the model's capacity to fully exploit contextual information and prompts, ultimately affecting its ability to curately represent and extract hyponym-hyperny relationships from unstructured text.
Prompt
wwwwledge-attention This de-
5.3 Effect on Knowledge Number
this section, we evaluate the effect of varying the number of knowledge prompts on the perfor- mance of our KEPL. model in Table We vary the number of prompts from 50 to 300 in increments of 50 and report the Recall (R), Precision (P), and Fl score in each case.
In the Knowledge-guie extraction w/o experi- ment, we trained KEPL logits with Linear MLP. We found that performance decreases by 4.97% in Fl score. This highlights the importance of in- corporating prompts before directly merging them with the context, particularly in tasks related to quiring hypernyms and hyponyms. The prompt serves as guidance signal, providing explicit in- structions to the model and aiding it in understand- ing the desired relationship or task. By incorpo- rating prompts, the model gains a clearer direction and context- -ppeccfcc cues, which are crucial for ac- curately capturing and representing hierarchical re- lationships.
Table 6: Effect on Knowledge number for KEPL
Prompt number
R
F1
50
0.8938
0.8210
0.8495
100
0.8999
0.8204
0.8523
200
0.9043
0.8293
0,8617
300
0.9138
0.8238
0.8665
As seen in Table 6, there is clear improvement in the KEPL model's performance as the num- ber of prompts increases. This confirms our hy- pothesis that increasing the volume of knowledge prompts can enhance the model's effectiveness.
5.4 Ablation Study
Conclusion
this section, we further provide more insights with qualitative analysis and error analysisto ad- dress the remaining challenges. In Table 7, we dis- play the results of individually removing compo- nents from the trained KEPL model the CHR dataset.
In this paper, we introduce Knowledge En- hanced Prompt Learning (KEPL. for extract- ing hypernym-hyponym relations in Chinese lan- guage. KEPL utilizes the concept of prompt learn- ing to incorporate prior knowledge in the form of patterns into the model, which simultaneously em- beds both the pattern and text.
In the prompt w/o experiment, we use a Random matrix to replace the Dynamic Adaptor, which leads to a drop of 6.73% in Fl score, showing the importance of using the prompt as the connection of hypernym and hyponym. The prompt instructs the model to consider the hierarchical structure between concepts and to accurately identify hy- ponyms and hypernyms. This approach provides a structured framework for the model to leverage contextual cues, linguistic patterns, and semantic associations related to hyponymy and hypernymy,
The prompt in the framework uses Hearst- like patterns, specifically for extracting hypernym- hyponym relations. Additionally, we have created Chinese hypernym-hyponym relation extraction dataset, which includes three different types of sce- narios: Wikipedia, news articles, and We-media. The results of our experiments using this dataset show that our proposed model is both efficient and effective.

---

Table 7: Ablation Study with for Konwledge atten- tion, D for Dynamic Adaptor, for Knowledge-guided extraction
for Computational Linguistics: ACL-IJCNLP 2021, pages 186-192, Online. Association for Computa- tional Linguisties.
Jindong Chen, Ao Wang, Jiangjie Chen, Yanghua Xiao, Zhendong Chu, Jingping Liu, Jiaqing Liang. and Wei Wang. 2019. Cn-probase: A data-driven ap- proach for large-scale chinese taxonomy construc- tion.
R
P
F1
KEPL w/o A 0.8667
0.8013
0.8328
KEPL w/o D 0.8201
0.7965
0,8082
KEPL. w/o E
0.8295
0.8174
0.8234
KEPL
0.9138
0.8238
0.8665
Sarthak Dash, Md Faisal Mahbub Chowdhury. Alfio Gliozzo, Nandana Mihindukulasooriya, and Nico- las Rodolfo Fauceglia. 2020a. Hypernym detection using strict partial order networks. In Proceedings of the AAAI Conference Artificial Intelligence, vol- ume 34, pages 7626-7633.
Limitations
Domain Adaptability While the KEPL model has demonstrated effective performance in scenar- ios such as Baike, News, and We-media, its appli- cability in other domains contexts is yet to be confirmed. For example, the model may require further tuning and optimization for specialized do- mains such as technology, law, or medicine.
Sarthak Dash, Md Faisal Mahbub Chowdhury, Alfio Gliozzo, Nandana Mihindukulasooriya, and Nico- las Rodolfo Fauceglia. 2020b. Hypernym detection using strict partial order networks. In Proceedings of the AAAI Conference Artificial Intelligence, vol- ume 34, pages 7626-7633.
Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Learning semantic hier- archies via word embeddings. 52nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 1199 1209. Association for Computational Linguis- tics.
Data Dependency The performance of the KEPL model is to a significant extent dependent on the quality and quantity of available data. In cases where data is scarce, particularly in certain domains or for specific tasks, the model may re- quire larger datasets for efficient training.
Proceedings of the
Marti A. Hearst. 1992. Automatic acquisition of hy- ponyms from large corpora. COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics.
References
Alain Auger and Caroline Barrière. 2008. Pattern- based approaches to semantic relation extraction: A state-of-the-art.
William Held and Nizar Habash. 2019. The effective- ness of simple hybrid systems for hypernym dis- covery. In Proceedings of the 57th Anmual Meet- ing of the Association for Computational Linguis- tics, pages 3362-3367. Association for Computa- tional Linguisties.
14:1-19.
Gábor Berend, Márton Makrai, and Péter Földiák. 2018. 300-sparsans at semeval-2018 8 task 9: Hyper- nymy as interaction of sparse attributes. In Proceed- ings of the 12th International Workshop on Semantic Evaluation, pages 928-934, Association for Compu- tational Linguistics.
Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji- Ro-n Wen, and Daxiang Dong. 2019. Taxonomy- aware multi-hop reasoning networks for sequential recommendation. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pages 573-581.
Gabriel Bernier-Colbone and Caroline Barrière. 2018. Crim at semeval-2018 task 9: A hybrid approach to hypernym discovery. In Proceedings of the 12th International Workshop on Semantic Evalua- tion, pages 725-731. Association for Computational Linguistics.
Tomas Kliegr. Vojtech Svatek, Krishna Chandramouli, Jan Nemrava, and Ebroul Izquierdo. 2008. Wikipedia as the premiere source for targeted hypernym discovery.
Guido Boella and Luigi Di Caro. 2013. Supervised
learning of syntactic contexts for uncovering def- initions and extracting hypernym relations in text databases. In Camille Salinesi, Moira C. Norrie, and Óscar Pastor, editors, Advanced Information Systems Engineering, volume 7908, pages 64-79. Springer Berlin Heidelberg.
Jingye Li, Hao Fei, Jiang Liu, Shengqiong Wu, Meis- han Zhang, Chong Teng, Donghong Ji, and Fei Li. 2022. Unified named entity recognition word- word relation classification. 36(10):09655-0997
Thomas Bott, Dominik Schlechtweg, and Sabine Schulte im Walde. 2021. More than just fre- quency? demasking unsupervised hypernymy pre- diction methods. In Findings of the Association
Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022. Uni- fied structure generation for universal information extraction.

---

Xusheng Luo, Luxin Liu, Yonghua Yang, Le Bo, Yuan- peng Cao, Jinhang Wu, Qiang Li, Keping Yang, and Kenny Q. Zhu. 2020. Alicoco: Alibaba e-commerce cognitive concept net.
Chengyu Wang and Xiaofeng He. 2020. BiRRE: Learning bidirectional residual relation embeddings for supervised hypernymy detection. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 3630-3640, On- line. Association for Computational Linguistics.
Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, and Jing Shao. 2022. Prompt for extraction? paie: Prompting argument interaction event argument extraction.
Chengyu Wang, Xiaofeng He, and Aoying Zhou. 2017. A short survey on taxonomy learning from text cor- pora: Issues, resources and recent advances. In Pro- ceedings of the 2017 Conference on Empirical Meth- ods Natural Language Processing, pages 1190- 1203, Copenhagen, Denmark. Association for Com- putational Linguistics.
Roberto Navigli and Paola Velardi. 2010. Learning word-class lattices for definition and hypernym ex- traction. In Proceedings of the 48th Annual Meet- ing of the Association for Computational Linguis- Association for Computa-
tics, pages tional Linguisties.
Josuke Yamane Tomoya Takatani, Hitoshi Yamada, Makoto Miwa, and Yutaka Sasaki. 2016. Distribu- tional hypernym generation by jointly learning clus- ters and projections. In Proceedings of COLING 2016, the 26th International Conference on Compu- tational Linguistics: Technical Papers, pages 1871- 1879. The COLING 2016 Organizing Committee.
Stephen Roller, Douwe Kiela, and Maximilian Nickel. 2018. Hearst patterns revisited: Automatic hyper- nym detection from large text corpora.
Ivan Sanchez and Sebastian Riedel. 2017. How well can we predict hypernyms from word embeddings? a dataset-centric analysis. In Proceedings of the 15th Conference of the European Chapter of the As- sociation for Computational Linguistics: Volume 2, Short Papers, pages 401-407. Association for Com- putational Linguistics.
Shuo Yang, Lei Zou, Zhongyuan Wang, Jun Yan, and Ji-Rong Wen. 2017. Efficiently answering technical questions a knowledge graph approach. 31(1).
Changlong Yu. Jialong Han, Haisong Zhang, and Wil- fred Ng. 2020. Hypernymy detection for low- resource languages via meta learning. In Proceed- ings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 3651-3656, Online. Association for Computational Linguistics.
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze questions for few shot text classification and natural language inference.
Julian Seitner, Christian Bizer, Kai Eckert, Stefano Faralli, Robert Meusel, Heiko Paulheim, and Si- mone Paolo Ponzetto. 2016. A large database of hypernymy relations extracted from the web. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC' 16), pages 360-367. European Language Resources As- sociation (ELRA).
Wenhao Yu, Lingfei Wu, Yu Deng, Qingkai Zeng. Ruchi Mahindru, Sinem Guven, and Meng Jiang. 2021. Technical question answering across tasks and domains.
Hanqing Zhang. Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2022a. A survey of controllable text generation using transformer-based pre-trained language models.
Rion Snow, Daniel Jurafsky, and Andrew Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In Advances in Neural Information Pro- cessing Systems, volume 17. MIT Press.
Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang. Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang. Hui Zong, Zheng Yuan. Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, and Qingeai Chen. 2022b. Cblue: A chinese biomedical language understand- ing evaluation benchmark.
Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanhin Zhao, Yuxiang Lu, Weixin Liu, Zhi- hua Wu, Weibao Gong, Jianzhong Liang, Zhizhou Shang. Peng Sun, Wei Liu, Xuan Ouyang. Dian- hai Yu, Hao Tian, Hua Wu, and Haifeng Wang- 2021. Ernie 3.0: Large-scale knowledge enhanced pre- pre-training for language understanding and genera- tion.
Yogarshi Vyas and Marine Carpuat. 2017. Detecting asymmetric semantic relations in context: A case- study on hypernymy detection. Proceedings of the 6th Joint Conference on Lexical and Computa- tional Semantics (*SEM 2017), pages 33-43. Asso- ciation for Computational Linguistics.
Chengyu Wang, Yan Fan, Xiaofeng He, and Aoying Zhou. 2019. Predicting Sypernym-hyponym rela- tions for chinese taxonomy learning. 58:585-610.


Reviews:
[{"review_id": "kGsFAaWSBv", "reviewer": "Reviewer_mrHG", "summary": "", "questions": "- A: The reviewer needs to understand the explanation of the proposed framework.\nCould the authors explain the framework, including the shape of tensors and parameters and a specific example?\n\n- B: How many and what lexico-syntactic patterns were used to create the dataset?\n\n- C: How were the 300 prompts made, and what are they like?\n\n- D: Is the proposed framework based on the assumption that the data must contain hypernym-hyponym relation instances? \nThe reviewer thinks the prompts of this paper may harm precision if the data does not have hypernym-hyponym relation instances.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your valuable feedback! We appreciate your time and effort in providing feedback. We realize there are some areas of confusion and we’d like to address your concerns as follows:\n\n- **Construction of Dataset**: \nWe acknowledge the feedback regarding the need for greater transparency in dataset construction. We employ certain specific rules to filter short sentences. We will provide a detailed section showing Construction of Dataset in the revised manuscript.These rules include the following:\n  - We disregard any sentences with a character count less than a given threshold (10 characters).\n  - We eliminate sentences with an excess of punctuation marks (e.g., commas, colons, periods).\n  - We filter sentences with numerals or certain non-Chinese characters.\n  - The sentences that pass these filters are then manually labeled with appropriate hypernym-hyponym relation by our team. We mark the entity position: Determine the start (entity _start) and end (entity _end) positions of the answer in the text, ensuring the scope of the answer is accurate. If there is no explicit hypernym-hyponym entity, mark the corresponding position as NONE.\n\n\n- **Dataset Format**:\n Here, the 'latent position' refers to the position of words in the sentence that the spans denote. For example, in a sentence \"Apple is a type of fruit\", {span1} = {0,1} would denote the position of \"fruit\" (hypernym), and {span2} = {6,7} would denote the position of \"Apple\" (hyponym).\n\n- **Explanation in Section 4.2**: \nThe knowledge Ψ refers to the dynamic adaptor for knowledge. It is a component that dynamically selects and applies the most appropriate prompts (patterns) based on sentence semantics. $L$ is the LLM and $M$ represents the number of prompt candidates available. The scoring matrix $w$ is used to generate a set of weights for each pattern, which indicates its relevance to the given sentence. Each sentence $s_i$ is an input of context which might contain hypernym-hyponym relation(like 'Apple is a type of fruit'). Regarding your question on the function $L_{dec}(pmi)$, it is used to generate prompt embeddings $E_p$, which have a shape of $N\\times L\\times H$ where $N$ is the number of prompts, $L$ is the length of prompts, and $H$ is the dimension of the contexts.\nFor the parameter $pmi$, each component corresponds to a prompt and its shape is $N\\times L$, where $N$ is the number of prompts and $L$ is their length. We apologize for the typographical error where the bias vector, '$b$', was mistakenly denoted as '$B$'. We assure you that it will be corrected in the revised manuscript.\n\n- **Parameters in Equation (6)**: \nThe equation (6) does not directly calculate the distribution for start/end of hypernym and hyponym. It calculates the logits (unnormalized log probabilities) for each word in a sentence to be part of a relation. By applying softmax to these logits, we can obtain the probabilities which can then be used to infer start/end positions for hypernym and hyponym spans.\n\n- **Switching Corresponding Parameters**: \nRegarding the process of switching corresponding parameters for each within the proposed framework, it is achieved through the attention mechanism. Depending on the attention score, the model learns to focus more on certain words that are likely to be involved in the hypernym-hyponym relation.\n\n- **Equation (9)**: \nThe score_k represents the \"strength\" of a pair (i, j) being the target span (hypernym and hyponym). It is obtained by running the unified representation for templates and context through several Transformer layers followed by the application of a softmax function.\n\nWe hope this clarification resolves your concerns. As for the specific points:\n\n- **A.Explain framework**: \nThank you for your advice. We agree that more illustrative examples would be beneficial. We will provide a detailed example showing the shape of tensors and parameters in the revised manuscript.\n\n- **B.Number of lexico-syntactic patterns**: \nFor the question about the lexico-syntactic patterns used in building the dataset (question B), we did not rely on specific patterns to avoid potential biases. Our construction process involved manual annotation of the sentences that passed our filters, marking appropriate hypernym-hyponym relationships. This in turn helps our Knowledge Enhanced Prompt Learning (KEPL) model train more accurately on a broad range of hypernym-hyponym relations, rather than those limited to specific patterns only.\n\n- **C.Creation of 300 prompts**: \nThe 300 prompts were manually created by our team. We constructed phrases that capture the 'is-a' relationship between two entities in diverse ways, an example can be seen in Table 2 of our paper. The 300 prompts were manually created by our team. We formulated phrases that encompass the 'is-a' relationship between two entities in a variety of ways, as demonstrated in Table 2 of our paper. It's important to note that these prompts were used only for knowledge purposes and were not utilized in the data construction process, as mentioned in question B.\n\n- **D.Assumption and potential precision harm**: \nWhile our method does aim to extract hypernym-hyponym relations, it is not reliant on the assumption that each input sentence must contain such relations. When a sentence does not contain targeted relations, our model can predict a null output.\n\nWe will ensure that the typographical errors you highlighted will be corrected in our revised manuscript. We greatly appreciate your constructive feedback and thank you for considering our work. Thank you once again for your valuable feedback. We look forward to addressing these points as we revise our manuscript.\n\nSincerely,\n\nThe Authors", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- A new dataset for Chinese hypernym-hyponym relation extraction was built.\n- The experiment results showed the effectiveness of the proposed framework.", "reasons_to_reject": "- The method of dataset construction could be more transparent, including the specific rules and annotation guidelines used to extract candidates.\n- It is difficult for the reviewer to know the dataset format (data, span1, span2). What is the \"latent position\" that span represents?\n- The reviewer needs help understanding the explanation in section 4.2.\n  - What is knowledge Ψ, specifically?\n  - What is L and M?\n  - Where and how is the scoring matrix w used?\n  - What are s_{i} and s_{i,j}? (What is the shape of s_{i} and s_{i,j}?)\n  - The reviewer needs to learn the specific calculation of equation (2).\n  - What is the shape of pmi? How to obtain pmi?\n  - What is B^{HxH}?\n- This method may need the distribution of the start and end for each hypernym and hyponym, but the reviewer does not know how to calculate it from equation (6).\n- Where does the framework switch the corresponding parameter for each?\n- The reviewer needs to learn the score_k of equation (9).", "questions_to_authors": "- A: The reviewer needs to understand the explanation of the proposed framework.\nCould the authors explain the framework, including the shape of tensors and parameters and a specific example?\n\n- B: How many and what lexico-syntactic patterns were used to create the dataset?\n\n- C: How were the 300 prompts made, and what are they like?\n\n- D: Is the proposed framework based on the assumption that the data must contain hypernym-hyponym relation instances? \nThe reviewer thinks the prompts of this paper may harm precision if the data does not have hypernym-hyponym relation instances.", "paper_topic_and_main_contributions": "In this paper, the authors built a Chinese hypernym relation extraction dataset, proposed a framework that learns a unified representation of context through specific attention, and evaluated the effectiveness of the proposed framework on the dataset.\n\nThe dataset is to amplify the quality and diversity across various domains against existing resources and is built from Baidu Baike, We-Media, and several news sources.\nThe proposed framework can inject knowledge for a given sentence and calculate the distribution of each token selected as start/end for each hypernym and hyponym using the unified representation of templates and contexts.\n\nThe experimental results using the dataset showed that the F1-measure of the proposed framework is higher than the ones of other baselines.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "In Table 1\nCntext  ->  Context\n\nIn section 5.4\nPattern-guided extraction -> Knowledge-guided extraction"}, "extra_text": {}}, {"review_id": "LN4EjjAiaS", "reviewer": "Reviewer_rkKb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We value your insightful comments and thank you for raising these concerns. We would like to address your feedback in the following ways:\n\n- **Hypernym-Hyponym Pairs Extraction:** We appreciate the reviewer's comments and would like to clarify a potential misunderstanding regarding the content. It should ne noticed that KEPL dataset does contain many instances like (053-054) that cannot be captured by rule-based approaches, which is one of the major reasons we deem it necessary to use multi-source datasets. The sentences that pass these filters are then manually labeled with appropriate hypernym-hyponym relation by our team. Please refer our response to mrHG in **Construction of Dataset** for more detail.  We will provide a detailed section showing Construction of Dataset in the revised manuscript.\n\n- **Rule-based Approach as a Baseline:** Thank you for raising concerns about the rule-based method. It's worth noting that we did not follow a specific pattern in data prepossessing. The rule-based method does boast high precision rather than recall, as shown in the additional experiment. Indicative performance metrics for a rule-based approach are as follows：\n\n|         | Precision | Recall | F1 Score |\n|---------|-----------|--------|----------|\n| Pattern |    0.409  |  0.251 |   0.311  |\n| Lexical |    0.471  |  0.281 |   0.352  |\n\n\n Our findings show that both precision and recall rates are relatively low when rule-based method applied to Chinese contexts.Noticed that the rule-based approach[1] used here in our experiment for comparison is different from the rule used during our dataset construction.\n- **Hypernym Detection Method:** Before clarifying the difference between our paper and the hypernym detection method, we should point out that we have compared with the recent SOTA methods to ensure the effectiveness and robustness of our proposed method in identifying hypernym-hyponym relationships in various contexts. The hypernym detection task is fundamentally interested in whether a given term pair \"is\" or \"is not\" in hypernymy relation. As a pipeline approach, Hypernym Detection could be prone to error propagation issues, supporting our assertions that a hypernym detection method may not be suitable baseline for this task[2].\n\n- **ChatGPT Evaluations:** We agree with your suggestion about including results from ChatGPT for a fair comparison. We are currently working on more elaborative tests in this regard and appreciate your patience. A crucial preliminary finding, as evidenced in Table 5, is the inability of ChatGPT to accurately acknowledge hypernymy relationships. For instance, in the example provided concerning Omaha Beach, one of the four primary landing points during World War II's Normandy Invasion, ChatGPT struggled to recognize \"Omaha Beach\" as a code name.\n\n- **Missing References:**  In comparing our work to \"BigCilin\", we believe that while it is a notable work in hypernym detection, it focuses on building a linguistic resource, the BigCilin, as an open-domain Chinese knowledge graph. In contrast, KEPL primarily targets the improvement of hypernym discovery (Given an input term, the hypernym discovery task retrieves a ranked list of suitable hypernyms from a large corpus) [2] by constructing a dataset specifically for this task.\n\n- **Data constructions:** \n\n  - We appreciate the input regarding the necessity to enhance the clarity of how the dataset was assembled. We hope this clarification proves helpful. We are grateful for your valuable feedback and will take it into account as we revise our manuscript.  We are also committed to making our dataset open-source, and we plan to release it in the near future for the benefit of the research community. Please refer to \"Construction of Dataset\" in response for mrHG for more detail. \n\n  - In order to establish a balanced representation, it is essential to maintain a 1:1 ratio of positive and negative pairs among the candidate sample as outlined in Table 1. This approach will counter any potential bias and provide a more comprehensive scope for rebuttal analysis.\n\n  - The meaning of \"the most appropriate pair\" is to select the most accurate or most appropriate combination of entities or elements in a specific context. In the example {Xie Houqin, a senior investment consultant of Jufeng Investment Advisory, told the Securities Daily that since July this year}, 'the most appropriate pairing' is \"Xie Houqin, senior investment consultant of Jufeng Investment\", not \"Xie Houqin, senior investment consultant\".\n\nWe are committed to rectifying the typographical errors that you've pointed out in our revised manuscript. Your helpful critique is highly appreciated; thank you for giving our work your attention. Once again, we're grateful for your insightful commentary. We're eager to tackle these issues as we refine our manuscript.\n\n\n\nSincerely,\n\nThe Authors\n\n[1]https://github.com/HillZhang1999/Chinese-Hypernym-Hyponym-Relation-Extraction\n\n[2]Bai Y, Zhang R, Kong F, et al. Hypernym discovery via a recurrent mapping model[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. 2021: 2912-2921.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. It constructs a new hypernym dataset in Chinese, which covers several data sources.\n2. Some baselines are appropriate and it has some ablation studies.", "reasons_to_reject": "1. One main problem of the dataset is that the authors use the rule-based approach to identify potential hypernym-hyponym pairs. That is, the dataset does not contain the hypernym-hyponym pairs that cannot be captured by the rules (by the way, what rules you use?). The examples in the paper in line 247-248) and Table 4 share a pattern. Thus, the dataset would favor the model that learns those patterns and I believe you can improve the performance on this dataset by simply using the same rule-based approach you used plus a trained model to verify the candidates. \n\n2. It is unclear why we need to have a special architecture in Figure 2 for the hypernym detection task. If prompts are effective, couldn't we just use the typical prompt approach such as append XXX is a <mask> after the GPT or RoBERTa? You should compare your methods with these baselines to justify your special architecture design. There are many existing hypernym detection methods such as Dash et al., 2020 and Roller et al., 2018. You should also compare your methods with them. It's also good to see how well ChatGPT could do in this task.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors build a Chinese hypernym detection dataset by first using a rule-based approach to extract the candidate hypernym-hyponym pairs and next ask humans to verify. Then, the authors propose a new architecture to improve its performance on the proposed dataset over several baselines, including W2NER (state-of-the-art Chinese NER), Ernie3 (state-of-the-art pretrained LM), and UIE (widely used Chinese information extraction tool).", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] also builds a Chinese hypernym dataset. You should compare your dataset with theirs and explain why you want to build another one.\n\n[2] Liu, Ming, Yaojia LV, Jingrun Zhang, Ruiji Fu, and Bing Qin. \"BigCilin: An Automatic Chinese Open-domain Knowledge Graph with Fine-grained Hypernym-Hyponym Relations.\" arXiv preprint arXiv:2211.03612 (2022).", "Typos_Grammar_Style_and_Presentation_Improvements": "The data construction process could be more clear. Who label the hypernym? What's the ratio of positive and negative pairs among those candidates from rule-based approach. What approach do you use for disambiguation and what do you mean the most appropriate pair?\n\nTypos:\nline 66: sub-entities\nline 244 and 265: space after ,\nline 318 and 286: the uppercase rule should be the same in your titles\nline 329: pic 2 -> Figure 2\nline 390, 393, 396: space before (\nline 398: wildly used -> widely used\nDash 2020a is the same as Dash 2020b"}}, {"review_id": "j5OjpIytjr", "reviewer": "Reviewer_3dDj", "summary": "", "questions": "Table 2 and Table 4: hard to understand without English translations.\n\nThe presentation quality should be improved by proofreading.\n\nFigure 1 is hard to read.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are sincerely thankful for providing us with constructive feedback and detailed suggestions about our paper! We would like to address your feedback in the following ways:\n\n- **Missing references:** We appreciate the reviewer's suggestions on the missing references and will update our bibliography to include the recommended papers[1][2][3][4].\n\n- **Lack of English translations in case study (Table 2 and Table 4):** We apologize for this oversight and will provide English translations of the examples presented in Table 2 and Table 4 to ensure the content is clear and accessible to all readers.We agree with your suggestion about lacking of English translations.\n\n- **Presentation quality and proofreading:** We acknowledge that the presentation quality of the paper can be improved. We will thoroughly proofread the manuscript, address any grammatical errors, and ensure that the language is clear and concise. In addition, we will enhance the readability of Figure 1 to ensure that the information presented is easy to understand. \n\n- **Domain adaptability:** The reviewer raises a valid concern regarding the domain adaptability of our KEPL model. We recognize that the model has demonstrated effective performance in certain domains, but its applicability in other domains is yet to be confirmed. In future work, we will explore the extension of our model to other specialized domains, such as technology, law, or medicine.\n\n- **Data dependency:** We acknowledge that our KEPL model's performance heavily relies on the quality and quantity of the available data. In cases where data is scarce, particularly for certain domains or specific tasks, we will work on improving the model by creating or collecting larger datasets for efficient training.Furthermore, we are also committed to making our dataset open-source, and we plan to release it in the near future for the benefit of the research community.\n\nWe affirm our commitment to rectify the errors you outlined in our revised manuscript. Your constructive feedback is deeply appreciated and we are grateful for your consideration of our work.\n\nSincerely,\n\nThe Authors\n\n[1]A Short Survey on Taxonomy Learning from Text Corpora: Issues, Resources and Recent Advances. EMNLP 2017: 1190-1203\n\n[2]More than just Frequency? Demasking Unsupervised Hypernymy Prediction Methods. ACL/IJCNLP (Findings) 2021: 186-192\n\n[3]BiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection. ACL 2020: 3630-3640\n\n[4]Hypernymy Detection for Low-Resource Languages via Meta Learning. ACL 2020: 3651-3656", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Using prompt learning for Chinese hypernym-hyponym extraction is novel.\n\n2. Good performance compared to previous methods.\n\n3. Good experimental analysis.", "reasons_to_reject": "1. Missing references.\n\n2. Lack of English translations in case study.", "questions_to_authors": "Table 2 and Table 4: hard to understand without English translations.\n\nThe presentation quality should be improved by proofreading.\n\nFigure 1 is hard to read.", "paper_topic_and_main_contributions": "This paper presents a  Knowledge Enhanced Prompt Learning for Chinese hypernym-hyponym extraction.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "A Short Survey on Taxonomy Learning from Text Corpora: Issues, Resources and Recent Advances. EMNLP 2017: 1190-1203\n\nMore than just Frequency? Demasking Unsupervised Hypernymy Prediction Methods. ACL/IJCNLP (Findings) 2021: 186-192\n\nBiRRE: Learning Bidirectional Residual Relation Embeddings for Supervised Hypernymy Detection. ACL 2020: 3630-3640\n\nHypernymy Detection for Low-Resource Languages via Meta Learning. ACL 2020: 3651-3656"}}]

Author rebuttal:

Meta-review: The paper presents a new approach for hypernym discovery in Chinese and a new dataset for evaluation. 

Reviewers mrHG and rkKb call out the dataset as an important contribution and all three reviewers acknowledge the performance improvements that the proposed method achieves.

The majority of issues raised by the reviewers come down to missing details in the manuscript, and other presentation issues. From the extensive and constructive discussions (especially with Reviewers mrHG and rkKb) it appears that the authors were able to address all these concerns, either with additional experiments or with clarifications. An additional issue raised by Reviewer rkKb regarding a missing simpler baseline was similarly addressed by the authors. 

While the revisions required by the authors would be relatively extensive, the underlying soundness of the work is high, as evidenced by the increased scores by the reviewers.
Decision: Accept-Main