TacoPrompt: A Collaborative Multi-Task Prompt Learning Method for Self-Supervised Taxonomy Completion
Authors: Hongyuan Xu
Ciyi Liu
Yuhang Niu
Yunong Chen
Xiangrui Cai
Yanlong Wen
Xiaojie Yuan
Venue: EMNLP 2023 Main (2023)
Keywords: Taxonomy completion
prompt learning
self-supervised learning
multi-task learning

Abstract: Automatic taxonomy completion aims to attach the emerging concept to an appropriate pair of hypernym and hyponym in the existing taxonomy. Existing methods suffer from the overfitting to leaf-only problem caused by imbalanced leaf and non-leaf samples when training the newly initialized classification head. Besides, they only leverage subtasks, namely attaching the concept to its hypernym or hyponym, as auxiliary supervision for representation learning yet neglect the effects of subtask results on the final prediction. To address the aforementioned limitations, we propose TacoPrompt, a Collaborative Multi-Task Prompt Learning Method for Self-Supervised Taxonomy Completion. First, we perform triplet semantic matching using the prompt learning paradigm to effectively learn non-leaf attachment ability from imbalanced training samples. Second, we design the result context to relate the final prediction to the subtask results by a contextual approach, enhancing prompt-based multi-task learning. Third, we leverage a two-stage retrieval and re-ranking approach to improve the inference efficiency. Experimental results on three datasets show that TacoPrompt achieves state-of-the-art taxonomy completion performance. Codes are available at https://github.com/cyclexu/TacoPrompt.

TacoPrompt: A Collaborative Multi-Task Prompt Learning Method for Self-Supervised Taxonomy Completion
Hongyuan Xu, Ciyi Liu, Yuhang Niu, Yunong Chen, Xiangrui Cai, Yanlong Wen; Xiaojie Yuan College of Computer Science, Nankai University, Tianjin, China (xuhongyuan, liuciyi, niuyuhang, chenyunong @dbis mankai edu. cn (caixr, wenyl, yuanxj T @nankai. edu, cn
Abstract
Existing Taxonomy
Computer
New Concepts
Automatic taxonomy complction aims to at- tach the emerging concept to appropriate pair of hypernym and hyponym in the existing laxonomy. Existing methods suffer from the overfitting to leaf-only problem caused by im- balanced leaf and non-leaf samples when train- ing the newly initialized classification head. Besides, they only leverage subtasks, namely attaching the concept to its hypernym or by- ponym, auxiliary supervision for represen- tation learning yet neglect the effects of sub- task results on the final prediction. To dress the aforementioned limitations, pro- pose TacoPrompt, Collaborative Multi-Task Prompt Learning Method for Self-Super Taxonomy Completion. First, we perform triplet semantic matching using the prompt learning paradigm to effectively learn non-leaf attachment ability from imbalanced training samples. Second, we design the result con- text relate the final prediction to the subtask results by contextual approach, enhancing prompt-based multi-task learning. Third, we leverage two-stage retrieval and re-ranking ap- proach to improve the inference efficiency. Ex- perimental results on three datasets show that TacoPrompt achieves state-of-ther thee-of-the-rtt taxonomy completion performance. Codes are available http://ghhtbb..oo/c/c/////////oe Proooppp
Computer Hardware
Data Minig
NLP
ChatGiT
CPU
GPU
Langunge Model
Text Gemeration
Integratad Cirsuit
Prompt Learning
If all MA SK Parent: Hardware Child. GPU [SEP]Query: SSD
Probabolity
nnchor Description
Query Description
Computer
Completed Taxonomy
Scieace
Computer Hardware
Data Minig
Integgatee Cinuuit
Language Model
Text Generation
SSD
CPU
GPU
Prompt Leaminng
ChatGPT
Figure 1: An example attaching new concepts to the existing "Computer Science" taxonomy by triplet semantic matching using prompt learning paradigm.
continuously emerge, is infcasible to manage the overwhelming amount of new content in online streaming (Zhang et al., 2021). To this end, con- siderable effort has been devoted to automatic tax- onomy expansion task (Shen et al., 2020; Yu et al.. 2020; Wang al., 2021 where (query) is only attached to the most appropriate hypernym (parent) leaf node while ignoring its potential hyponym (child) in the existing tax- onomy. However, such a "leaf-only" assumption is inappropriate (Zhang et al., 2021) and results in large limitations (Wang et al., 2022) in real-worid applications. Therefore, Zhang et al. (2021) pro- poses the taxonomy completion task, which aims to find appropriate hypernym and hyponym pair (anchor or position) for the new concept, satisfy the non-leaf attachment requirement. For example, the query "Integrated Circuit" is attached to the parent "Computer Hardware" and the child "GPU"
concept
1 Introduction
A taxonomy a tree-like hierarchical structure organized around hypernym-hyponym ("is-a") re- lations between concepts. It has been widely used in downstream natural language processing (NLP) tasks such named entity recognition (Wang et al., 2021b), transfer learning (Luo al., 2022) and lan- guage models (Bai et al., 2022).
Most the existing taxonomies are maintained by domain experts manually, which labour- intensive and time-consuming. As new concepts
Coorreponniigg author.

---

instead of just to the parent as shown in Figure 1.
leverage two-stage retrieval and re-ranking approach to reduce the cross-encoder's expensive inference computation costs, increasing the infer- ence efficiency by up to 1,000 times.
Several recent studies have achieved promising progress in taxonomy completion. Typically, they first leverage semantic (Wang al., 2022: Arous et al., 2023) or structural (Jiang et al., 2022) in- formation to generate the concept and the position representation using multiple tasks as supervision (Zhang et al., 2021). Then, these methods utilize the matching module to identify the attachment pre- diction for the given query based on the generated representation. Although achieving remarkable per- formance, these representation- two main limitations. First, they primarily lever- age a newly initialized classification head as the matching module, whose ability overfits to leaf- only, i.c., the taxonomy completion task degrades to the expansion task (Wang et al., 2022), when learning from imbalanced leaf and non-leaf train- ing samples. Second, these methods only leverage subtasks, namely attaching the concept to its hy- pernym or hyponym, as auxiliary supervision for representation learning (Zhang et al., 2021; Wang et al., 2022; Jiang et al., 2022) while neglecting the results of subtasks, which are proven effective in the final prediction (Wei et al., 2022).
We conduct extensive experiments on three tax- onomy datasets to compare TacoPrompt with state- of-the-art taxonomy expansion and completion methods. The comparative results show that Taco- Prompt outperforms previous state-of-the-art meth- ods by a large margin, such as by 11.1 % in hit@] and 9.1 % in recall @ 10 on average.
Our contributions can be summarized as follows:
methods have
We propose TacoPrompt, a self-supervised taxonomy completion framework that lever- ages the cross-encoder by the prompt learning paradigm to effectively learn from imbalanced training samples and multiple tasks.
We design the hard and soft result context relate the final prediction to the results of subtasks by a contextual approach to enhance prompt-based multi-task learning.
We leverage a retrieval and re-ranking ap- proach to improve the inference efficiency. Ex- perimental results on three taxonomy datasets show that TacoPrompt outperforms previous methods by large margin in the taxonomy completion task.
To effectively address the aforementioned limita- tions, we propose TacoPrompt for self-supervised taxonomy completion in this paper. First, we lever- age the pre-traine tre-trained language model (PLM) by the prompt learning paradigm to perform triplet seman- tic matching for the hierarchical "is-a" relations prediction between the query, candidate hypernym and hyponym concepts, as illustrated Figure 1. Prompt learning paradigm (Schick and Schütze, 2021; Liu et al., 2023a) reuses the PLM's masked language modelling (MLM) head instead of intro- ducing the randomly initialized classification head, requiring fewer training samples to learn the target ability. Thus, we leverage ting problem caused by imbalanced training sam- ples. Second, we extend the multi-task learning from representation--asse methods to the cross- encoder-based framework by taking advantage of prompt learning's effectiveness in multi-task learn- ing scenarios (Fu et al., 2022; Sanh et al., 2022). Morcover, we design the result context to relate the final prediction to the subtask results by contex- tual approach to enhance prompt-based multi-task learning. Specifically, we design the hard result context using explicit answer tokens and the soft re- sult context using learnable hidden vectors. Third,
Related Work
2.1 Taxonomy Expansion and Completion
Numerous investigations have been conducted on automatically updating the existing taxonomies. Based on the assumption of the problem, the stud- ies can be divided into two main categories: tax- onomy expansion and completion. In taxonomy expansion, researchers (Shen al., 2020; Yu et al., 2020; Ma al., 2021; Wang et al., 2021a; Liu et al., 2021: Takeoka et al., 2021; Zhai et al., 2023; Jiang et al., 2023; Cheng et al., 2022: Phukon et al., 2022) suggest that new concepts can only be added as leaf nodes. However, this assumption is inappro- priate for real-world applications: the query nodes can also be added as non-leaf nodes to the taxon- omy (Zhang et al., 2021). To address this issue, Zhang al. (2021) defines new task named tax- onomy completion, where new concepts can be added at any position within the taxonomy and proposes to use multiple score functions to find the most appropriate position for the new concept.
address the overfit-

---

Wang et al. (2022) and Jiang et al. (2022) further develop it by incorporating important sibling rela- tions to enhance the nodes' representations. Arous et al. (2023) observes that distant nodes can also improve the representation and proposes a position- enhanced node representation learning method. Be- sides, Zeng et al. (2021) generates potential con- cepts before completing the taxonomy. Xia et al. (2023) proposes new two-stage completion prob- lem to find the parent first and then all children under the fixed parent. Note that this newly defined problem is out of the scope of this paper.
and Schütze, 2021) and multi-task scenarios (Sanh et al., 2022: Sun et al., 2022: Asai et al., 2022; Wang et al., 2023).
In the taxonomy expansion task, Xu et al. (2022) leverages prompt learning as the generation tech- nique to generate candidate hypernyms directly based on the query description. In contrast, we aim to leverage prompt learning to address the imbal- anced training samples and better multi-task learn- ing challenges in the taxonomy completion task. We study the utilization of logic-related tasks' re- sults by a contextual approach to enhance prompt- based multi-tas! learning.
We observe that all previous completion meth- ods focus on better representation learning with semantic information (Wang et al., 2022; Arous et al., 2023) or structural information (Zeng et al.. 2021; Jiang et al., 2022; Arous et al., 2023). In this paper, we develop a cross-encoder-based method to perform triplet semantic matching between con- cept descriptions along with a two-stage retrieval and re-ranking approach to improve the inference efficiency of the cross-encoder-base method.
3 Preliminaries
3.1 Prompt Learning Pipeline
The core idea of prompt learning is to convert the downstrcam task into the MLM task form. As- suming that M,F denote the PLM and the tem- plate function respectively. Given sentence input (mo, 21, ................................ Fn)) the template function F gen- erates the task-specific template å = F(x) as:
2.2 Prompt Learning Paradigm
F(x) [CLS] ((x) [MASK] R [SEP], (1)
Traditionally, the fine-tuning paradigm is lever- aged adapt the PLM to downstream tasks. The paradigm typically decodes the last layer's hid- den state the special [CLS] token with a task- specific classification head (Liu et al., 2021; Devlin et al., 2019). However, training such a head from scratch is prone to the overfitting problem (Dong et al, 2023) and requires large number of labelled training samples, making fine-tuning paradigm in- feasible in few-shot scenarios (Liu et al., 2023a).
where L(x), R(x) refer to the left and right task- specific context. Next, contextual hidden vectors over & are produced sequentially by M's trans- former layers. The hidden vector of the [MASK] token is h [MASK] Rd, where is the hidden di- mension of M. Then, the MLM head of M gener- ates logits to calculate each word v's probability be- ing filled in [MASK] position as PM [MASK] = h [xxAs)) ). Given a verbalizer f : y H+ V that defines the mapping from the label space y' to the label token space V. the probability distribution
Prompt learning paradigm has gained consider- able attention since the birth of GPT-3 (Brown et al., 2020; Chen et al.. 2022). The paradigm fills the gap between the learning objectives of downstream tasks and the PLM by reformulating downstream tasks into the MLM task form (Liu al., 2023a) uti- lizing hard prompts (Chen et al., 2022: Schick and Schütze, 2021) or soft prompts (Hambardzumyan et al., 2021; Liu et al., 2023b; Cai et al., 2022). Instead of training newly initialized classification head, the paradigm reuses the MLM head, which is well-trained in the PLM's pre-training stage, the decoder for downstream tasks. Prompt learning has achieved impressive performance in text-related tasks (Brown et al., 2020; Dong et al., 2023; Schick and Schütze, 2021), especially in sew-shot scenar- ios (Cai et al., 2022; Zhang et al., 2022; Schick
over the label y is formulated as:
Pyy x) (PM [MASK]
F(x)), v V2y;,
(2)
where Vy represents the subset of V and refers to a function converting the probability of label words to that of label y.
3.2 Problem Formulation
In this section, we formally define the taxonomy, the taxonomy completion task and the multiple tasks studied in this paper.
Taxonomy, We follow Shen al. (2020); Zhang et al. (2021) and define a taxonomy as a directed acyclic graph T = (N, E), where each node n E Af represents a concept and each edge (n,, ne) E E

---

indicates a hypemymnhyyoony relation expressing that ny is the most specific concept that is more general than ne. Following Wang et al. (2021a), we assume there exists a corpus resource D, from which each concept n 's corresponding description dn can be retrieved.
Arous al., 2023). However, prior taxonomy completion methods focus on leveraging the PLM to generate semantic representations (Wang et al., 2022; Arous et al., 2023) or structural representa- tions (Jiang et al., 2022) for the query and candidate positions while neglecting the powerful contextual dependency capture ability of the PLM. Thus, we utilize the PLM in a cross-encoder manner to dis- tinguish hierarchical semantics between concepts. Specifically, given an input x (q. P. c) and their corresponding description dy. dp. de D, we first construct the triplct semantic context as:
Problem Definition. The imput of the taxonomy completion task includes two parts: (1) existing taxonomy T (NV, E)) and (2) a set of new concepts C. The overall goal is complete To by attaching each new concept q E to the most appropriate candidate position a. The valid candi- date position is defined by Zhang et al. (2021) as a concept pair = (p, c), where p E A and cs one of the descendants of p in the existing taxonomy. Note that p could be "pscudo node" acting as placeholder represent that q is semantically the most general or specific concept for the existing taxonomy. The existing taxonomy is completed to T = (~ U {q}, 0 {(p, q), (g. e) after at- taching q. Following Shen et al. (2020y; Manzoor et al. (2020); Zhang et al. (2021), we assume the new concepts in C are independent of each other and reduce the taxonomy completion task into |C] independent attachment optimization problems:
S(x) = d [SEP] (Query)
where utilize delimiter tokens, c.g., "(Parent)" " to identify the different types of concept semantic context. Then, we leverage the template function FF to generate the task-specific template a = F(x) for the taxonomy completion task as:
F(x) = [CLS] All [MASK]((1) [SEP]. (5)
Finally, we perform the prompt learning pipeline in Section 3.1 to calculate the probability of the query concept q attaching to the candidate position in
Equation 3 as:
0 = arg max log P (q/ a; +);
(3)
a.EAA
PM( [M] yes) PM[[M] =no) h h (6)
where Vi E 21, ..... .CI], A and & represent the set of valid candidate positions and model parame- ters, respectively.
where [M] is short for [MASK] and we address the multi-classsfication problem through binary classi- fication (Zhang et al., 2021; Shen et al., 2020). We denote the probability as the score sa(g, a).
Multiple Tasks. Following Zhang et al. (2021), we focus on final task denoted as T., which aims to attach the query to the anchor a, along with two related subtasks Tn and T. that aims to attach g to the parent and child c, respectively.
4.2 Collaborative Multi-Task Learning
Multi-task learning has been a typical approach in previous concept representation--ased taxonomy completion methods (Wang et al., 2022; Jiang al.. 2022) since subtasks have been proven effective as an inductive bias for representation learning (Zhang et al., 2021). In contrast, we introduce our cross-encoder-bast based collaborative multi-task learning method in this section.
4 Methodology
In this section, we first introduce our triplet seman- tic matching pipeline using the prompt learning for taxonomy completion in $4.1. Then, enhance it by collaborative multi-task learning via the con- textual approach in $4.2. Furthermore, we discuss the efficient inference process based on a two-stage approach in $4.3 and describe the self-supervised training details in $4.4.
Following prior representation--based methods, the most straightforward multi-task learning ap- proach is co-training the final task with subtasks. Specifically, we utilize the same semantic match- ing pipeline in Section 4.1 for subtasks T, and Te, except that the semantic context only includes " "(Parent)d,"" " and "(Child)d,", respectively, and the delimiter token before [MAAK]] " in Equaition 5 is changed from "All" to "Parent" or "Child".
4.1 Triplet Semantic Matching
As a potential knowledge base, the PLM has been proven effective in hypemym--yyonyym relations learning (Wang et al., 2022; Jiang et al., 2022;

---

(a) TacoPrompt-Chain
Subtask Attach query parent
Subtask Attach query child
Final Task: Attach query anchor
Prompt Teguplate Semantic Contexti
T2:
Result Contegts Semantic Context2
T3:
Result Context2
Somantic3
[MASK] <Posiop (SEP) <Qrnup [SEP)
[MASK]
Parent: <C> [SEP] <Q>
[MASK] Parent: Child <C>
Transformer I Layens
Trrmer Layers
Transformer Layers
MLM Head
MLM Head
MLM Head
Score(g.p) Explicit answer iParent: Yes
Score(q.c) Explicit
'Child:
Score(q. Final answer Attach? Yes"
(b) TacoPrompt-Hidden
Tasks: Subtask
Subtask and Final Task
Prompt Template
Sementic
Al:: [MASK]] Parent [MASK]2 Child: [MASK]3 <aarntt> yleap usfoy <Child> deep lighty (SEP) <Quaryp toke [SEP)
Transfomeer Layens
MMASK]]
Self-Attention
MMMASK]2
Self-Attention
M[MASKJ3
Soft
MLM Head
Score(q.a)
Collaboration
Score(q.p)
Collaboration
Score(q.ee)
Figure 2: Illustration of TacoPrompt framework.
To further enhance the prompt-based multi-task learning, we incorporate the subtask results as con- texts into the template function defined in Equation 5. enabling the backbone LM to learn the depen- dency between the prediction for T, and the results of subtasks. Following this core idea, TacoPrompt has two variants named -Chain and -Hiden, that utilize the hard and the soft result context, respec- tively, as illustrated in Figure
where C(x) is the result context of Te. We leverage Sa (a a) as the final attachment score. The effects of different score function combinations will be discussed in Section 5.1.2.
TacoPrompt--iideen. This variant utilizes hid- den vectors as the soft result context. Specifi- cally, we replace the hard result context (zx) and Czz) in Equation to learnable "Parent: [MASK] and "Child: [MASK]" respectively. TacoPrompt- Hidden utilizes multiple transformer layers with the self-attention mechanism facilitate information interaction between hidden vectors of mask tokens, which are decoded for the result predictions of the corresponding tasks. Therefore, the results sub- tasks are shown to the prediction of the final task through hidden vectors. Likewise, we also utilize sa (g, a) as the final score in this variant.
WoomttChhiin This variant utilizes ex- plicit answers as the hard result context. Inspired by Choun-of-Thought Prompting (Wei et al., 2022), we address the task in step-by-step approach. Specifically, we first utilize our semantic match- ing pipeline for the task Tn for the corresponding result context denoted as P(x):
Parent: yes, if sp(p 2
P(x)
Parent: no, otherwise,
4.3 Efficient Inference
In this section, we analyze the inference time com- plexity of our method. We introduce a two-stage retrieval and re-ranking approach to improve the efficiency of the inference process.
where sp(g.p) p) is the score of the query attaching to the parent calculated by Equation 6 in the Tp. Then, the template function of T, leverages the subtask result the additional context:
Inference Time Complexity. Assuming that the number of queries is |C] and the number of candi- date positions is [A]. Our method utilizes the back- bone LM to predict the probability for each candi- date position, and the time complexity of the back- bone LM is O x dxix), where θ and represent the parameter number and hidden dimension of the backbone LM, and / is the average length of the
Fz(z) = Child [MASK]]()]]))) Sc) (8)
where o is the string concatenation operation. We repeat this process for T. and generate the template function of T as:
F. (x) = All [MASM]] C(x) o (.)x), (9)

---

input sequence. Therefore, the inference time com- plexity of our method is ( |A]] θ d (²). We observe that [C], I are related to the task, and the choice of LM determines d, θ, the effects of which will be discussed in Section 5.1.3. Actually, the ex- pensive computation, which is the main drawback of the cross-encoder-based method, mainly comes from the need to recompute |A| different triplets of sentences (Almagro al., 2023).
5 Experiments
Detailed experimental settings, including the intro- duction of datasets, evaluation metrics and base- lines, are described in Appendix A.
5.1 Experimental Results
5.1.1 Comparison With Baselines.
The performance of all methods on SemEval-Food, McSH, and WordNet- Verb datasets are summarized in Table 1. We discuss the questions below based on the comparative results.
TacoPrompt++ Following the retrieval and re- ranking approach in Information Retrieval (IR), we first leverage a lightweight representation-bee -based method to recall the top 50 relevant candidate posi- tions and then re-rank them using our method since the automatic taxonomy completion focuses more top predictions. This two-stage approach im- proves the inference efficiency by |A|110 times since the representation in the first stage can be generated offline. More specifically, it theo- retically 146x, 858x and 1023x more efficient on the SemEval- MeSH and WordNet- Verb datasets. In this paper, we leverage the state-of-the. art representation-based method, TaxoComplete (Arous et al., 2023), as the retrieval stage method.
1. Which method achieves the most competitive performance in taxonomy completion? Ours. As can see in Table 1, both variants of our proposed method achieve statistically significantly better per- formance than the second-best method on three datasets. In particular, the two variants averagely in- crease MRR/H @ I/R @ 10 by 0. 1344/..6%//1..... 0.22, 0.064/8.80 0444/88%%/..2% and O. 0.102/7..%/8.4499 over base- lines on three datasets, respectively. First, we observe that all methods that train classification head from scratch perform relatively poorly the non-leaf attachment scenario. Even the strongest baseline, TEMP, is defeated by Taxo- Complete, which leverages the cosine similarity for semantic matching, in attaching non-leaf queries on the MeSH and WordNet- Verb dataset. This il- lustrates that imbalanced leaf and non-leaf training samples will result in the overfitting to leaf-only problem of the newly initialized head. In contrast, our method leverages prompt learning to tune the pre-trained MLM head with imbalanced training samples, achieving advantageous performance non-leaf attachment than other methods.
4.4 Self-supervised Training
In this section, we introduce the self-supervised training details of our framework.
Self-supervision Generation. Given one node g E 2 in the existing taxonomy To = (No, query, we sample 1 + N anchors the q's an- chor set Ag = (60 e)), (pt, c.)................,.. ( ) including a positive anchor that (p", g) and (q, e are edges that exist in 8d and negative anchors which at least one end does not exist in ε We label each of these anchors by y = {Ma, Up, Nch. where Up = Igo ((p, 9)),, Ve = Ieo ((g. c)). ya = Up A Me and (.) is the indicator function. The label set of q is denoted Vg , .'.....,''. Finally, training instance X (g. Agg Wa) corre- sponds the q is created. By repeating the above process for each node q E A we obtain the full training data X = {Xx, X , XIv').,
Besides, we notice that representation- based methods supervised with multiple tasks perform better than those not, as TMN, QEN, and TaxoEn- rich outperform BERT+MLP, TaxoExpan and Ar- borist in most cases. On the other hand, we find that cross-encoder--asee methods are more power- ful than representation-batt ones in the taxonomy completion, as TEMP has the best scores among baselines in 22 out of 24 columns in the total at- tachment scenario. Our improvement over TEMP confirms our intuition about prompt learning: it typ- ically enables the LM to learn from multiple tasks' supervision, and we take advantage of the cross- encoder's contextual dependency capture ability enhance multi-task learning via the result context. We will analyze more about the superiority of our prompt learning framework in Section 5.1.2.
Learning Objective. We learn our model on X using the following objective:
((0) - XoLa + ApCp + AcCo.
(10)
where La. Lp. and Le represent loss functions for different tasks, and we calculate them by the BCELoss. The parameersssserrs λo, Ap and Ae are weights to adjust the effect of each task.

---

Total
Leaf
Non leaf
Datesets
Methods
MRJ
MRR
R@1
R@5
R@
10
11@1
1
110:1
MRR
11@.5
R::10
MRR
I[@5
R@
BERT-MIP
702,980
0.279
5.5
15.1
20.9
11.5
28.4
40.5
0.565
24.0
TasoExpun
371.291
0.286
5.7
13.3
18.0
11.5
26.4
34.5
0.477
30.1
35.6
0.130
8.0
3.6
Athorist
256.491
0.290
13.0
18.0
21.0
26.4
34.5
38.5
0.466
39.0
38.5
0.146
12.0
6.7
173.516
0.332
22.0
21.6
36.5
39.9
0.538
12.0
TasoEnrih
230,424
0:408
11.7
26.7
31.7
23.6
49.3
58.1
0.723
58.5
66.7
0.149
4,0
3.0
QEN
336.554
0.439
21.2
30.9
35.0
45.9
58.8
64.9
0.732
64.2
68.9
0.209
32.0
9.1
Semeval-Rood
TaxoComplte
296.072
0.489
38.0
29.7
53.4
0.702
32.0
15.8
TEMP
51.374
0.579
20.3
41.2
47.9
42.6
76.4
81.1
0.881
81.3
83.0
0.348
52.0
21.0
Ours-Chain
51.643
0.717
28.6
51.4
60.8
60.1
85.8
89.2
0.886
87.8
86.7
0.587
76.0
40,9
Chain++
49.5
56.8
75.0
76.4
60.0
30.7
Ours-Hidden
47,423
0.708
30.9
51.1
60.1
64,9
85.8
86.5
0.899
87.8
87.4
0.561
76.0
39,2
Ours: Hidden++
28.9
43.4
49.2
60.8
75.7
77.0
78.0
74.8
64.0
29.5
BERT-MIP
9152.507
0.045
0.2
1.2
0.102
3.6
4.5
TasoExpun
1029.344
0.233
2.7
6.2
12.2
6.0
12.7
23.9
0.381
16.3
24.3
0.137
5.0
4.3
Arhorist
843.199
0.337
5.0
13.6
21.8
11.0
25.8
37.4
0.437
26.7
30.6
0.271
23.8
16.0
567.831
0.372
1.2
24.6
15.9
33.6
43.8
0.525
23.4
TasoEurih
393,062
0:424
7.4
22.4
31,0
16.2
42.6
52.5
0.619
51.3
54.1
0.296
24.1
15.9
QRN
451.253
0.438
7.5
21.3
30.8
17.1
43.1
55.9
0.611
51.1
51.8
0.332
26.1
17.9
McSH
TeoCComllte
357.494
0.540
29.3
41.1
24.5
54.1
63.9
0.605
54.8
34.1
TEMP
80,291
0.612
13.8
35.3
48.0
31.4
66.5
77.5
0.839
75.4
77.6
0.471
47.5
29.8
Ours-Chain
76.579
0.678
17.4
41.8
56.4
39.6
72.9
82.4
0.833
77.4
78.2
0.583
63.2
43.0
Chain++
42.1
53.7
40.8
71.2
76.4
69.7
46.1
Ours-Hidden
49,140
0.674
17.9
42.4
55,9
40.7
74.6
84.6
0.868
79.0
81.5
0.554
65.1
40.2
Ours-Hidden+++
18.3
40.8
$3.0
41.5
71.8
78.5
75.4
68.9
64.0
43.2
BERT-MPP
5858 466
0.113
2.6
5.7
11:7
0.202
0.015
TasoExpun
1752.271
0.215
4.1
11.4
15.1
17.1
22.5
0.354
20.5
26.7
0.057
3.1
Atborist
1455.251
0.246
3.8
11.0
15.5
5.7
15.5
21.6
0.331
16.2
21.8
0.148
12.8
8.4
1813.634
0.290
5.4
20.7
21.2
29.1
0.425
10:7
TasoEnrich
5462.075
0.179
3.9
9.0
12.3
5.8
13.6
18.4
0.313
16.8
22.6
0.025
0.4
QRN
1730.755
0.404
9.1
23.3
31.0
13.9
34.0
43.9
0.568
38.6
48.4
0.224
15.3
11.8
WordNet- Verb
TaxoComplete
2661 .488
0.407
9.0
30.9
13.6
31.7
40.8
0.487
0.315
27.6
19.1
TEMP
960.536
0.450
13.3
30.6
37.5
20.3
45.9
55.0
0.692
53.4
62.8
0.182
15.3
9.5
Ours-Chain
597.098
0.546
18.5
35.5
45.2
28.3
49.3
$8.9
0.692
$2.9
61.9
0.385
34.7
26,8
Chain++
42,9
30.2
48.8
53.4
42.9
30.2
Ours-Hidden
436.799
0.557
18.3
36.9
46,5
28.0
52.3
62.5
0.762
56.5
65.8
0.370
35.2
25,3
Ours-Hiddenn++
17.2
35.4
42.1
26.3
48.5
53.1
50.9
54.5
38.8
28.4
Table 1: Overall results on all three datasets. All baseline methods are reproduced for non-leaf attachment results. The best results of our and baseline methods are in bold and underlined respectively for comparison.
Datasets
Chain
Chain++
Hidden
Hidden++
one semantic matching process.
Semfival-Fool
40.716s
0.284s
12.811s
0.087s
3. Is the two-stage retrieval and re-ranking ap- proach effective in the inference stage? Yes. TacoPrompt+++ still achieves state-of-the-art per- formance while significantly decreasing the aver- age inference time, as shown in Table 2, suggesting that the two-stage retrieval and re-ranking approach is feasible to improve the inference efficiency of the cross-encoder method in the taxonomy comple- tion task. Interestingly, TacoPrompt++ even out- performs TacoPrompt in some cases. For example, the Chain++ increases H@) 1 by 1.9% on the Verb dataset. Therefore, developing better method for the retrieval stage is expected in future studies due to the impressive effectiveness of the cross-encoder method in the taxonomy completion task.
McSH
241.971s
0.278s
81.145s
0.094s
WordNet- Verb
287.926s
0,281s
95.970s
0.093s
Table 2: Average inference time per query using a single NVIDIA RTX 3090 GPU all three datasets.
2. What is the performance comparison between the two variants? They achieve overall compa- rable performance to each other. Specifically, the Hidden variant exhibits superior performance for leaf attachment, whereas the Chain variant excels in the leaf attachment scenario. This indicates that the hard result context provides better resis- tance to the overfitting problem than the Iearnable soft result context. For efficiency, Hidden around 3x more efficient than Chain as shown in Table since Chain has to perform cross-encoder-basee semantic matching three times for all tasks. In con- trast, Hidden tackles three tasks concurrently in
5.1.2 Ablation Studies
Table 3 presents the ablation results under different settings. Assuming that re, mt and pl represent the result context, multi-task learning and prompt

---

learning, respectively and the "-NL" short for the non-leaf scenario, we discuss the questions below.
Datasets
Settings
MRR
H&1
R810
MRR-NL
RG10NNL
68.8
40.9
ChrsHHdden
0.708
64.1
60.1
03561
39.2
1. Is prompt learning effective in addressing the taxonomy completion? Yes, methods w/t pl out- perform their counterparts w/o pl under the same re and mt settings on three datasets. For example, they averagely increase MRR-NL/R @ 1O-NL on three datasets by 0.036/2.7% under the multi-task setting and 0.033/4.99% under the single-task set- ting. It indicates that leveraging the MLM head instead of newly initialized classification head keeps the backhone LM from overfitting to the leaf attachment, the training sample number of which surpasses that of the non-leaf attachment as shown in Table 7. Besides, the improvement under the multi-task setting illustrates the effectiveness of prompt learning learning from multiple tasks in the eross-encoder- completion framework.
SenoFval-Food
0.78
55.4
62.2
57.2
55.9
0.498
34,1
324
wlo
0.51
53.4
55.0
02441
28.4
0.445
48.6
54.0
0.443
28.4
0.678
39.6
5644
43.0
Chus-Hidden
0.674
40.7
55.9
0.534
40.2
McSII
0.677
38.1
56.1
0,558
41.5
41.8
54.1
37.6
w/o mc.
0.650
34,3
53.7
0,536
36,9
0.627
33.7
50.2
0.450
31,0
28.3
45.2
26.8
Our- Miid-iddenn
0.557
28.0
46.5
0,370
25.3
WordNe: Verb
0.536
27.4
44.2
0.357
23.2
nnd
0.549
25.0
45.1
23.2
w/o mc. 0.539
24.9
64.6
0.337
21.8
0.552
26.1
45.8
0.337
20,9
Table 3: Ablation studies on all three datasets. Note that re, mt and pl represent the result context, multi-task learning and prompt learning. respectively, and "NI." is short for the non-leaf scenario. In w/o pl experiments. we utilize two-layer MLP as the classification head decode the hidden vector of the [CLS] token for attach- ment probability prediction.
2. Can the result context enhance prompt- based multi-task learning? Yes, the performance of method w/o re drops by different extents, es- pecially in the most important non-leaf attach- ment scenario in the taxonomy completion task (Wang et al., 2022). For example, R@ 10-NL drops 8.5%, 1.5% and 3.6% compared to the method w/t the hard result context on the Food, MeSH and Verb datasets, respectively. In addition, the total H@1 drops 2.7%, 2.6% and 0.6% compared the method w/t the soft result context on three datasets. Together, the result context effectively relates the prediction of different tasks in a contex- tual approach and is proven effective in enhancing prompt-based multi-task learning.
T.
MRR
R0
10-16-
0.662
55.4
67.4
34,1
Aulo.p) sylc.4)
75.6
Helg.p) Adle.))
0.681
55.4
64.4
38.6
V
0.704
60.1
86.7
39.2
Au
0.685
53.1
88.9
(1.71%
64.9
87.4
39.2
0.609
66.9
85.9
39.8
Au(c-.)
0.674
86.7
35.2
Table 4: Ablation studies of different task supervision and score function combinations on SemEval- Food. Note that the score function s, y) refers the prob- ability of attaching to y in the task T, "L" and "NL" are short for the leaf and non-leaf scenarios.
5.1.3 Further Discussions
We manually construct the pseudo semantic context by reversing descriptions of parent and child and the pseudo result context by replacing the hard result context with the ground truth or the fake truth. The inference performance of the trained model provided with different pseudo contexts SemEval-FFoo is shown in Figure 3. We discuss the questions below.
Below, we discuss the results using different tasks and score function combinations on SemEval- Food, shown in Table 4.
3. Which combo choices of task supervision and score functions perform better? Methods supervised by all tasks T., Tn and TV perform bet- ter. First, each task provides supervision for tax- onomy completion as shown lines 1-3. Second, co-training two or all tasks can introduce bias for better performance in our proposed cross-encoder- based framework, and methods leveraging all tasks during training in lines 6-7 achieve the overall best performance. Compared to utilizing the predic- tion probability of T. as the final score, combining that of T and T. shows the potential to improve automatic completion performance since H@1 is further improved 2% as shown in line 7.
1. Does the language model predict the final attachment probability based on the result con- text? Yes, our method's performance fluctuates with different pseudo hard result contexts. Specifi- cally, the method performs better when the result context is closer ground-truth and vice versa. For example, the method provided with all ground-truth result contexts beats that provided with all fake ones almost twice on MRR on the SemEval-Fau dataset. This further illustrates that our method

---

Granarzhh
Grezzamn Clbte
Origiaal Contatt
Greeshro Plkk
KIP
SERT
ALBERT
60
60
EUEERA
14M
50.5
17.5
9.5%
63.7
RISTT
130M
0.77
011
101
404
00334
RoBERTS
125M
0.76
061
40.
Chan
ALBERT
125M1
0.713
39.
35.2
50,555
ELECTRA
14M
0667
19.2
30676
D6NIBERT
0.690
203
SomEral-Food
Mclll
WeedSesVee
SumboniFFood
Mclii
WeedSesVer
Table 5: Completion performance comparison of differ- ent backbone LMs on SemEval-Food. Tr. Param. refers to the number of trainable parameters. The average in- ference time is tested under the maximum batch size.
Figure 3: Inference performance with manually con- structed pseudo semantic and result contexts. Note that "RD" is short for the "Reversed Description". We report the H@1 and &@ 10 results in (a) and (b).
Dataset
Backbone LM
R@1
RG5
R
10
IGG
Its
H&
Llumn-78
1.6
5.8
12.9
3.4
10.8
22.3
relates the subtask results to the final prediction. 2. What is the relationship between semantic and result context? They provide mutual comple- mentary information. First, the method provided with all fake result contexts can still predict cor- rect final results, suggesting that it is robust to the noise results provided by subtasks. Conversely, the method provided with the reversed semantic context be improved by ground-truth result contexts. Overall, our method can cross-consider semantic and result contexts for final prediction. In addition, we discuss the effects of the back- bone LM choice, namely the bi-directional LM as shown in Table 5 and the LLM as shown in Table 6. We discuss the questions below.
SemEival Food
Limma 2-7B
42
2.7
7.4
21.6
Oars-Clainee
27.0
5688
76.4
77.7
Lhma-78
0.6
54
12.0
1.5
11.6
22.5
McSH
Lluma-2-78
12.1
Ours Chaine+
27.0
453
53.1
56.8
76.4
77.7
[.mma78
4.1
9.6
0.9
6.2
13.5
WoedNer:
Llama-2-78
Ours-Chains
27.0
45.3
53.1
56.8
76.4
77.7
Table 6: Completion performance of few-shot prompt- ing the LLM with hard result context. We compare the performance using the two-stage retrieval and re- ranking approach due to the LLM's expensive inference computation costs.
shot prompting approach with hard result context. The implementation details are described in Ap- pendix B.2. We can observe large margin ex- ists between few-shot prompting LLMs and our prompt learning framework under the same attach- ment probability prediction pipeline. We analyze the possible reason accounting for the LLM's poor performance is that they are unaware of the struc- tural distribution of existing taxonomies. In con- trast, our trainable framework is more feasible for the taxonomy completion.
3. Which backbone LM is the best choice to balance effectiveness and efficiency? ELEC- TRA. In Table 5, study the influence of back- bone LMs, including RoBERTa (roberta-base) (Liu et al., 2019), ALBERT (albert-base-v2) (Lan et al., 2020), ELECTRA (google/cleetra-smaal)) -generaor) (Clark et al., 2020) and DistilBERT (distilbert-bass uncased) (Sanh et al., 2019), on the taxonomy com- pletion performance. The results show that the LMs with more parameters, namely ALBERT and RoBERTa, have yet show obvious performance advantages over BERT. On the other hand, Distil- BERT consistently performs poorly attaching lon-leaf queries. In contrast, ELECTRA achieves comparable results to BERT in the total and non- leaf scenarios with nearly one-eighth of parame- ters, indicating that it is the possible backbone LM choice for balancing effectiveness and efficiency in our framework.
Conclusion
In this paper, we propose TacoPrompt for self- supervised taxonomy completion. Specifically, we leverage the prompt learning paradigm to perform triplet semantic matching, addressing the overfit- ting to the leaf-only problem caused by imbalanced training samples. Then, we perform collaborative multi-task learning by designing the result con- text to enhance the prompt learning framework. Moreover, we propose two-stage retrieval and re-ranking approach for efficient inference. Exper- imental results on three taxonomy datasets show that TacoPrompt outperforms the state-of-the-an methods by large margin.
4. What is the performance of few-shot prompt- ing LLMs with result context in taxonomy com- pletion? Poor. Table 6, we utilize Llama-7B (Touvron et al., 2023a) and Llama-2-7B (Touvron et al., 2023b) to complete the taxonomy in the few-

---

Limitations
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Tiric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish. Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot Iearners. In NeurIPS, pages 1877-1901.
We consider that the proposed method has the fol- lowing three limitations. (1) It has poor scalability to process long description texts. We truncate the long concept description to the fixed token length to satisfy the max sequence length limitation of the backhone LMM. However, this approach may cause information distortion and result in performance degradation. (2) We ignore the sibling information, which is proven effective in laxonomy complction, in our framework. Potential future studies include incorporating more subtasks, e.g.. finding the ap- propriate sibling concept for the query, in the pro- posed collaborative multi-task learning framework. (3) The proposed two variants perform taxonomy completion separately. We will explore their coop- eration approach for better taxonomy completion performance in the future.
Xiangrui Cai, Haidong Xu, Sihan Xu, Ying Zhang, al. 2022. Badprompt: Backdoor attacks on continuous prompts. In NeurlPS, pages -7768-37080.
Xiang Chen, Lei Li, Ningyu Zhang. Xiaozhuan Liang, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022. Decoupling knowledge from memorization: Retrieval-auge -aggmnted prompt learning. In NeurlPS, pages 23908-23922
Sijie Cheng. Zhouhong Gu, Bang Liu, Rui Xie, Wei Wu, and Yanghua Xiao. 2022. Learning what you need from what you did: Product taxonomy expansion with user behaviors supervision. In ICDE, pages 3280-3293.
Acknowledgements
We thank anonymous reviewers for their valuable comments. We also thank Yuxun Qu and Yuxiao Liu for their insightful suggestions. This research is supported by Chinese Scientific and Technical Innovation Project 2030 (No.2018AAA100IIU NSFC-General Technology Joint Fund for Ba- sic Research (No.U1936206), National Natu- ral Science Foundation of China (No.62077031,
Kevin Clark, Minh-Thang Luong, Quoe Le, and Christopher D. Manning. 2020. ELECTRA: pre- training text encoders as discriminators rather than generators. In ICLR, pages 1-18.
Jacob Devlin, Ming-Wei Chang, Kenton Lec, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL, pages 4171-4186.
62372252, 62002178
Hongyuan Dong, Weinan Zhang, and Wanxiang Che. 2023. Metricprompt: Prompting model as a rel- evance metric for few-shot text classification. In SIGKDD, pages 426-436.
References
Mario Almagro, Emilio J. Almazán, Diego Ortego, and David Jiménez. 2023. LEA: improving sentence similarity robustness typos using lexical attention bias. SIGKDD, pages 36-46.
Jinlan Fu, See-Kiong Ng, and Pengfei Liu. 2022. Poly- glot prompt: Multilingual multitask prompt training. In EMNLP, pages 9919-9935.
Ines Arous, Ljiljana Dolamic, and Philippe Cudré- Mauroux. 2023. Taxocomplete: Self saff-suerviied tax- onomy completion leveraging position-enhanced se- mantic matching. In WWW, pages 2509-2518.
Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021. WARP: word-level adversarial reprogramming, In ACL/JCNLP, pages 4921-4933.
Akari Asai, Mohammadreza Salehi, Matthew E. Pe- ters, and Hannaneh Hajishirzi. 2022. ATTEMPT: parameter-effiient multi-task tuning via attentional mixtures of soft prompts. In EMNLP, pages 6655- 6672.
Minhao Jiang, Xiangchen Song, Jieyu Zhang, and Ji- awei Han. 2022. Taxoenrich: Self-supervised taxon- omy completion via structure- structure-semantic representa- tions. In WWW, pages 925-9993--0
He Bai, Tong Wang, Alessandro Sordoni, and Peng Shi. 2022. Better language model with hypernym class prediction. In ACL, pages 1352-1362.
Song Jiang, Qiyue Yao, Qifan Wang, and Yizhou Sun. 2023. A single vector is not enough: Taxonomy expansion via box embeddings. In WWW, pages 2467-2476.
Georgeta Bordea, Paul Buitelaar, Stefano Faralli, and Roberto Navigli. 2015. SemEval-2015 task 17: Taxonomy extraction evaluation (TExEval). In ComEvvl@NAALL-HHH 2015, pages 902-910.
David Jurgens and Mohammad Taher Pilehvar. 2016. Semeval-2016 task 14: Semantie taxonomy enrich- ment. In SemEval-2016, pages 1092-1102.

---

Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervieed learning of language representations. In ICLR, pages 1-17.
Techan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In ICLR, pages 1-216.
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In EACL, pages 255-269,
Carolyn E Lipscomb. 2000. Medical subject headings (mesh). Bulletin of the Medical Library Association, page 265.
Timo Schick and Hinrich Schütze. 2021. Few-shot text generation with natural language instructions. EMNLP, pages 390-402
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang. Hiroaki Hayashi, and Graham Neubig. 2023a. Pre- train, prompt, and predict: A systematic survey of prompting methods natural language processing. ACM Comput. Surv. pages 1-35.
Timo Schick and Hinrich Schütze. 2021. It's not just size that matters: Small language models are also few-shot learners. In NAACL, pages
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023b. Gpt understands, too. Open, pages 1-10.
Jiaming Shen, Zhihong Shen, Chenyan Xiong, Chi Wang, Kuansan Wang, and Jiawei Han. 2020. Tax- oexpan: supersuper taxonomy expansion with position-enhanced graph neural network. In WWW, pages 486-497.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du. Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized bert pretraining ap- proach. CoRR, abs/ Nos/1907.11692.
Tianxiang Sun, Zhengfu He, Qin Zhu, Xipeng Qiu, and Xuanjing Huang. 2022. Multi-task pre-training of modular prompt for few-shot learning. CoRR. abs/2210.07565
Zichen Liu, Hongyuan Xu, Yanlong Wen, Ning Jiang, Haiying Wu, and Xiaojie Yuan. 2021. TEMP: tax- onomy expansion with dynamic margin loss through faxonomy-pathe In EMNLP. pages 3854-3863.
Kunihiro Takcoka, Kosuke Akimoto, and Masafumi Oyamada. 2021. Low-resource taxonomy enrich- ment with pretrained language models. In F.MNIP, pages 2747-2758.
Yifei Luo, Minghui Xu, and Deyi Xiong. 2022. Cog-
taskonomy: Cognitively inspired task taxonomy is beneficial to transfer learning in NLP, In ACL, pages 904-920.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez. Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. Llama: Open and efficient foundation language models. CORR, abs/2302.13977
Mingyu Derek Ma, Muhao Chen, Te-Lin Wu, and Nanyun Peng. 2021. Hyperexpan: Taxonomy ex- pansion with hyperbolic representation learning. In EMNLP, pages 4182-4194.
Emaad Manzoor, Rui Li, Dhananjay Shrouty, and Jure Leskovee. 2020. Expanding taxonomies with im- plicit edge semanties. In WWW, pages 2044 2054.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton- Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, An- thony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura. Maric-Anme Lachaux, Thibaut Lavril, Jenya Lee, Di- ana Liskovich, Yinghai Lu, Yuning Mao, Xavier Mar- tinet, Todor Mihaylov, Pushkar Mishra, Igor Moly- bog, Yixin Nie, Andrew Poulton, Jeremy Reizen- stein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subrama- nian, Xiaoqing Ellen Tan, Binh Tang, Ross Tay- lor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan. Iliyan Zarov, Yuchen Zhang, Angela Fan. Melanie Kambadur, Sharan Narang, Aurélien Ro- driguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models. CoRR, 025/2337709288
Bornali Phukon, Anasua Mitra, Sanasam Ranbir Singh, and Priyankoo Sarmah. 2022. TEAM: A multitask learning based taxonomy expansion approach for at- tach and merge. NAACL Findings. pages 366-378
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108..
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj, Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Févry, Jason Alan Fries, Ryan

---

Suyuchen Wang, Ruihui Zhao, Xi Chen, Yefeng Zheng. and Bang Liu. 2021a. Enquire one's s parent and child before decision: Fully exploit hierarchical structure for self-supervised laxonomy expansion. In WwW, pages 3291-3304.
Dataset
N MV/pirrun
[8]
#depch
#lcol rate
+candidates
SemEEval-Fod
1486/1190
McSH
97108002
10,498
0,708
42470
WordNet- Verb
13936/11933
13,407
0.771
51139
Table The dataset statistics. [M], [ the total number of nodes and edges. We count the proportion of leaf nodes in the train nodes and the number of candidate positions.
Suyuchen Wang. Ruihui Zhao, Yefeng Zheng, and Bang Liu. 2022. Qen: Applicable laxonomy completion via evaluating full taxonomic relations. In www. pages 1008-1017.
Xuan Wang. Vivian Hu, Xiangchen Song, Shweta Garg, Jinfeng Xiao, and Jiawei Han. 2021b. Chem- ner: Fine- grained chemistry named entity recogni- tion with ontology-guides distant supervision. In EMNLP, pages 5227-5240.
A Experimental Setup
A.1 Datasets.
Following Wang et al. (2022), we evaluate Taco- Prompt on three taxonomy completion datasets. The SemEval-Food is a food taxonomy based on the largest taxonomy of SemEival22016 Task 13 (Bordea et al., 2015). The Medical Subject Headings (MeSH) is hicrarchically organized vocabulary of medicine (Lipscomb, 2000). The WordNet-Verb based on WordNet 3.0 has fully developed taxonomy in SemEval-2016 Task 14 (Jurgens and Pilehvar, 2016). For each of these taxonomies T = (W,E), we follow Wang et al. (2022) and divide A into non-overlapping train nodes Nirran, validation nodes Wellidation and test nodes Neest. Specifically, we randomly sample 1,000 nodes validation nodes Vivaidation and an- other 1,000 nodes as nodes Wiest for WordNet- Verb, and we sample 10% nodes as validation nodes and other 10% nodes as test nodes for SemEval- Food and McSH. The left nodes are leveraged as train nodes Mirain Table 7 presents the statistical information of the three datasets.
Zhen Wang, Rameswar Panda, Leonid Karlinsky. Rogério Feris, Huan Sun, and Yoon Kim. 2023. Multitask prompt tuning enables parameter-effict transfer learning. In ICLR, pages 1-16.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian lchter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. 2022. Chain-of fhain-of-hoogghe thought prompt- ing elicits reasoning in large language models. In NeurlPS, pages 24824-24837.
Fei Xia, Yixuan Weng, Shizhu He, Kang Liu, and Jun Zhao. 2023. Find parent then label children: two- stage taxonomy completion method with pre-trained language model. In EACL, pages 1042-1042.
Hongyuan Xu, Yunong Chen, Zichen Liu, Yanlong Wen, and Xiaojie Yuan. 2022. Taxoprompt: A prompt- based generation method with taxonomic context for self self-supervised taxonomy expansion. In IJCAI, pages 4432-4438.
Yue Yu, Yinghao Li, Jiaming Shen, Hao Feng, Ji- meng Sun, and Chao Zhang. 2020. STEAM: self- supervised taxonomy expansion with mini-paths. In SIGKDD, pages 1026-1035.
A.2 Evaluation Metries.
Since the model outputs a ranking list of the didate positions for each query concept, we utilize the following rank-based metrics for evaluation.
Qingkai Zeng. Jinfeng Lin, Wenhao Yu, Jane Cleland- Huang, and Meng Jiang. 2021. Enhancing taxon- omy completion with concept generation via fusing relational representations. In SIGKDD, pages 2104- 2113.
Mean Rank (MR) calculates the macro av- erage ranking of the query concept's 's ground- truth among all candidate positions. Smaller MR indicates better performance.
Songlin Zhai, Weiqing Wang, Yuan-Fang Li, and Yuan Meng. 2023. DNG: taxonomy expansion by explor- ing the intrinsic directed structure on non-gaussian space. In AAAI, pages 6593-6601.
Mean Reciprocal Rank (MRR) computes the macro average reciprocal rank of the truc positions. Following Shen al. (2020), we scale the original MRR by 10 to expand the performance gap between different models.
Jieyu Zhang, Xiangehen Song, Ying Zeng, Jiaze Chen, Jiaming Shen, Yuning Mao, and Lei 2021. Tax- onomy completion via triplet matching network. In AAAL, pages 4662 4670.
Ningyu Zhang, Luoqiu Li. Xiang Chen, Shumin Deng. Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. 2022. Differentiable prompt makes pre-trained lan- guage models better few-shot learners. In ICLR, pages 1-19.
Recall @k counts the ratio of the true posi- tions ranked in the top k to the total true posi- tions for all query concepts.

---

Hit@k measures the number of query con- cepts with at least a ground-truth position
B Implementation Details
B.1 Comparison with Baselines
within the top & predictions, divided by the total number of query concepts.
(Devlin et al., 2019) our backbone LM. For our method, we truncate the long concept description to tokens to satisfy the max sequence length limitation of the backhone LM. We utilize the Adam optimizer and set the learning rate to 3e-5, batch size to 6 and accumu- lation step to 2 for training the model. The hyper- parameters C., Pps Le are set to 1.0 except L. for MeSH -Hidden is set to 0.3. We sample 15 neg- ative positions for each training instance. During the training process, we first train the model with self-supervised training data for 40 epochs and test on the validation set. According to the early stop- ping strategy, the training process ends when the MRR score on the validation set does not increase within 5 epochs. We evaluate the best model the test set. To fairly compare the proposed model with all baselines, we replace the backbone LM of QEN from distillBERT BERT to align with our set- tings. We adapt the taxonomy expansion method, TEMP, to the taxonomy completion task by adding the candidate child concept to its taxonomy-pati All the experiments are accelerated by NVIDIA RTX 3090 GPU devices.
We utilize BERT
It is worth noticing that the metric Precision @ k. which is widely used in previous work (Zhang et al., 2021; Wang al., 2022: Jiang et al., 2022), is proportional Recall@k.
A.3 Baseline Methods.
We compare our method with the following state- of-the-art taxonomy completion techniques.
TMN (Zhang et al., 2021) proposes the taxon- omy completion task. This framework lever- ages subtasks as auxiliary supervision signals for concept representation learning.
TaxoEnrich (Jiang et al., 2022) leverages structural information via taxonomy contex- tualized embedding and laxonomy-awar se- quential encoders. This method utilizes query-aware sibling aggregator augment position representations.
QEN (Wang al., 2022) generates semantic concept representation by pretrained language model. This framework evaluates sibling rela- tions to reduce the pseudo-leaf noise.
B.2 Few-shot Prompting Llama
When few-shot prompting Llama, we construct examples depending on the bi-encoder-based se- mantic similarity. Specifically, we first leverage the query concept's most semantically similar concept in the training set to construct the example prompt by filling in ground-truth results of subtasks in Equation 9, Similarly, we add a negative example to the prompt. Then, we utilize Llama to perform the same inference pipeline of TacoPromppt-haan except that the result context is autoregressively generated, to calculate the final prediction score. We accelerate few-shot prompting Llama by a sin- gle A A100 GPU device.
TaxoComplete (Arous et al., 2023) leverages semantic similarity through bi-encoders. This method utilizes direction-aware propagation to learn position-enhanced node representa- tion.
Following Zhang al. (2021), we adapt tax- onomy expansion baselines, namely BERT+MLP, TaxoExpan (Shen et al., 2020) and Arborist (Man- zoor et al., 2020), to the taxonomy completion task by concatenating the representation of the parent and the child as the representation of the corre- sponding candidate position. Note that all meth- ods mentioned above focus on better concept or candidate position representation. We adapt the state-of-the-art cross-encoder-
C Effects of Delimiter Contexts
We have discussed the effects of the semantic con- text (xx) and the result context P P(x) and C(x) in Section 5.1.3, ., but the effects of the delimiter con- text, e.g., "(Parent)ds d," the semantic context and "Parent: [MASK]" " in the result context, are ex- posed, which essential in prompt learning (Liu
taxonomy ex- pansion method TEMP (Liu et al., 2021) to the completion since our model also based on the powerful cross-encoder.

---

Chain
Hidden
Dataset
Backhone LM
R@1
Res
R:100
He1
11:5
10
Contest
MRR
H661
R@10
MRR
H611
Lamu-78
1.6
5.8
3,4
12mm-2277
1.3
4.2
2.7
7.4
Answers
0.714
62.2
6011
0.692
60.1
59.5
SemFval--Fod
Lama 2-700 8ibi
1.0
4.2
9.6
2.0
8.1
16.2
Unmatch
56.1
6015
64.2
Dur Chaitere
53.1
None
0.677
54.1
55.9
0.068
55.4
55.0
0.717
60.1
60.8
0.708
64.9
60.1
WondNee-Vort Verb
Llana-2776
0.9
5.0
10.2
1.4
7.3
13.9
12mm-22700B588
0.6
3.1
84
1.0
4.3
11.5
Table 8: Results of our method on SemEval-Foox with different delimiter contexts the semantic and the result context.
Table 9: Performance of few-shot prompting the Llama the taxonomy completion task.
al., 2023a). In this section, we conduct experi- ments with different delimiter context designs. In Table 8, "Answers" and "Unmatch" indi- cate that we replace the delimiter tokens in (xx) by "Parent Answer:" and "P:" respectively while keeping the delimiter tokens in semantic context unchanged "(Parent)". We replace the delimiter tokens in C(x) in the same way. "None" repre- sents that we remove delimiter tokens in both the semantic and result contexts.
tribution of the existing taxonomy when reasoning the query's s attachment answers. From another per- spective, the LLM urges laxonomy completion researchers to construct up-to-date datasets con- sisting of sufficiently new concepts for emergent concept attachment performance evaluation.
Firstly, we observe that the delimiter context significantly helps the language model distinguish between different concepts. Without the delimiter context, our method experiences a substantial per- formance drop. Secondly, the best performance is achieved when the delimiter tokens preceding the [MASK] token align with those in the seman- tic context, such as both being set as "Parent". In contrast, the model's performance drops by differ- ent extents when the delimiter context in the result context does not match that in the semantic context ("Unmatch")) or when it involves repetitive interfer- ence information ("Answers").
D More Analysis of Performing LLMs
We have compared few-shot prompting Llama-7B and Llama-2-7B with our trainable framework in Section 5.1.3. We further study the taxonomy com- pletion performance of larger Llama SemEval- Food and WordNet-Vert Verb shown in Table 9. Sur- prisingly, the Llama-2-70B-871 consistently un- derperforms the smaller Llama in the taxonomy completion task. We analyze the possible reason behind this phenomenon, which is that LLM con- siders a larger concept set instead of understanding the specific distribution of the existing taxonomy. More specifically, LLMs may try find the query's most specific hypernyms hyponyms, which may not exist in the existing taxonomy.
Such observation can motivate further studies from two aspects. From one perspective, is chal- lenging to make LLMs understand the specific dis-


Reviews:
[{"review_id": "NNIdpFd5Xg", "reviewer": "Reviewer_j9Kp", "summary": "", "questions": "According to Table 2, it’s interesting that two versions of the proposed method are good at the different subtasks. Specifically, TacoPrompt-Hidden is better at leaf attachment while TacoPrompt-Hidden is more suitable for Non-leaf attachment. Is there any analysis on this question?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and valuable suggestions. Below, we provide our response to the questions and concerns.\n\n**Q1. Missing Reference: IJCAI 2022 TaxoPrompt**\n\nA1. We must argue that the taxonomy completion (TC) task is more than a ‘similar task’ to the taxonomy expansion (TE) task. \nFirst, the task input is different. TC inputs candidate pairs of <parent, child> for each query whereas TE only inputs a single candidate parent. \nThis input difference brings the non-leaf attachment challenge to TC: the TC task may degenerate into the TE task with limited non-leaf training samples [1].\nSecond, TC naturally consists of multiple tasks compared to composed to TE, which only consists of a single task. Thus, only TC is concerned with better multi-task learning methods.\n\nWe propose a prompt-based multi-task learning framework to address TC since the prompt learning paradigm has been proven effective in learning with few labelled training samples [2] and multi-task learning [3]. To further improve prompt-based multi-task learning for TC, we design the result context to consider the results of subtasks in the final prediction.\n**Overall**, we aim to address research problems that only exist in TC and that’s why we focus on introducing the related studies of TC instead of TE.\n\nTaxoPrompt is proposed to address the TE task by generating the candidate hypernym of the query concept through the prompt learning paradigm. However, it cannot generate a pair of candidate hypernym and hyponym to address the TC task. Hence, we claimed that we propose the first prompt learning-based framework for the TC task. \n\n**Q2. Unconvincing Results**\n\nA2. The base models **utilize the same backbone LM in different ways**. Our simplest base model FT-ST feeds description sentences of the query, parent and child as a triplet to the backbone LM to directly perform TC. The base models of previous TC methods (including the strongest baseline QEN) input individual description sentences into the backbone LM to derive node-level embeddings of the query, parent and child, which are concatenated and then decoded for TC. The experimental results show that our base model is more effective than those in previous TC methods. \n\n**However, that doesn't mean our contributions mainly come from such a stronger base model.** According to Table 3 in the paper, the effectiveness of our proposed framework on claimed (1) better non-leaf attachment by addressing TC with prompt-based learning paradigm, and (2) task collaboration by the result context to improve prompt-based multi-task learning for TC is demonstrated. For example, our framework averagely increases R@10 of non-leaf queries by 11.65% over FT-ST on the Food dataset. \n\n**Besides, we supplement the reproduction results of the latest published work TaxoComplete [4]**, which could be seen as contemporaneous work to our submission according to the Paper Submission Information of EMNLP 2023. I hope these supplementary results can provide more convincing comparisons and address your concerns. Note that  '#' and '*' refer to leaf and non-leaf queries, respectively.\n\n\n| Food      | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----  | ---- | ---- | ---- | ---- |\n| TaxoComplete | 96.039  | 0.507 | 13.9 | 30.0 | 38.3 | 28.4 | 60.1 | 71.6 | 0.764 | 65.6 | 70.9 | 0.322 | 34.6 | 14.8 |\n| Ours-Chain  | 51.643 | 0.717 | 28.6 | 51.4 | 60.8 | 60.1 | 85.8 | 89.2 | 0.886 | 87.8 | 86.7 | 0.587 | 76.0 | 40.9 |\n| Ours-Hidden | 47.423 | 0.708 | 30.9 | 51.1 | 60.1 | 64.9 | 85.8 | 86.5 | 0.899 | 87.8 | 87.4 | 0.561 | 76.0 | 39.2 |\n\n| MeSH     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 93.741 | 0.565 | 10.7 | 30.5 | 43.6 | 20.6 | 50.2 | 64.3 | 0.616 | 49.4 | 51.5 | 0.526 | 52.0 | 37.5 |\n| Ours-Chain |  76.579 | 0.678 | 17.4 | 41.8 | 56.4 | 39.6 | 72.9 | 82.4 | 0.833 | 77.4 | 78.2 | 0.583 | 63.2 | 43.0 |\n| Ours-Hidden |  49.140 | 0.674 | 17.9 | 42.4 | 55.9 | 40.7 | 74.6 | 84.6 | 0.868 | 79.0 | 81.5 | 0.554 | 65.1 | 40.2 |\n\n| Verb     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 596.347 | 0.424 | 7.7 | 21.1 | 30.4 | 12.2 | 31.6 | 42.6 | 0.529 | 33.2 | 44.4 | 0.319 | 25.4 | 16.4 |\n| Ours-Chain   | 597.098 | 0.546 | 18.5 | 35.5 | 45.2 | 28.3 | 49.3 | 58.9 | 0.692 | 52.9 | 61.9 | 0.385 | 34.7 | 26.8 |\n| Ours-Hidden  |  436.799 | 0.557 | 18.3 | 36.9 | 46.5 | 28.0 | 52.3 | 62.5 | 0.762 | 56.5 | 65.8 | 0.370 | 35.2 | 25.3 |\n\n\n**Q3. Analysis of Two Variants**\n\nA3. The results of subtasks are represented to the LM as explicit ‘yes’ or ‘no’ tokens and hidden vectors respectively in TacoPrompt-Chain and in TacoPrompt-Hidden. The difference in how LM understands these two result representations could lead to respective advantages of these two variants in non-leaf and leaf attachment scenarios.\n\nThank you for the valuable suggestions. Please let us know if you have any other comments/questions.\n\nBest,\n\nAuthors\n\n**References**\n\n[1] Wang S, Zhao R, Zheng Y, et al. Qen: Applicable taxonomy completion via evaluating full taxonomic relations[C]//Proceedings of the ACM Web Conference 2022. 2022: 1008-1017.\n\n[2] Brown T, Mann B, Ryder N, et al. Language models are few-shot learners[J]. Advances in neural information processing systems, 2020, 33: 1877-1901.\n\n[3] Fu J, Ng S K, Liu P. Polyglot Prompt: Multilingual Multitask Prompt Training[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022: 9919-9935.\n\n[4] Arous I, Dolamic L, Cudré-Mauroux P. TaxoComplete: Self-Supervised Taxonomy Completion Leveraging Position-Enhanced Semantic Matching[C]//Proceedings of the ACM Web Conference 2023. 2023: 2509-2518.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) Interesting Method: The proposed approach formulates the task as a multi-task prompt learning problem, which seems a interesting perspective for taxonomy completion.\n\n(2) Robust Experimental Validation: The paper conducts comprehensive experiments to study diffierent aspects of the targeted task and the proposed method based on three benchmark datasets.\n\n(3) Clear Writing: The paper is well-written and concise, allowing for easy understanding of the main concepts and contributions of the paper.", "reasons_to_reject": "(1) Missing Important Reference: This paper misses the most related reference from IJCAI22, titled “TaxoPrompt: A Prompt-based Generation Method with Taxonomic Context for Self-Supervised Taxonomy Expansion”. This missed reference is the first to propose the prompt based method to solve the similar task (although the transcript under the review claims itself is the first one). \n\n(2) Unconvincing Results: According to Table 2, the proposed model greatly exceeds the baseline in different evaluation indicators in the three datasets. For example, the strongest baseline, QEN, 25% is exceeded on Hits@10 on the SemEval-Food dataset. But from the design of the method, I don't see enough advantages to support such an obvious improvement. Additionally, according to the ablation experiments in Table 3, the simplest base models in this paper, FT-ST, and the strongest baseline, QEN, use the same backbone LM (Bert). But there is a significant gap between them in the experimental results. Specifically, FT-ST is 19% higher than QEN on R@10 on the Food dataset. I think this result needs more explanation to make it convincing.", "questions_to_authors": "According to Table 2, it’s interesting that two versions of the proposed method are good at the different subtasks. Specifically, TacoPrompt-Hidden is better at leaf attachment while TacoPrompt-Hidden is more suitable for Non-leaf attachment. Is there any analysis on this question?", "paper_topic_and_main_contributions": "The paper introduces a multi-task prompt learning method for tackling the problem of automatic taxonomy completion. This approach is characterized by its collaboration between two subtasks, specifically attaching concepts to the hypernym and hyponym within existing taxonomies. The proposed method formulates the completion task in a masked language model form, enabling the system to learn how to attach non-leaf concepts with limited labeled data. By integrating the results of subtasks in a contextual manner, the system allows collaborative completion. The paper also provides experimental evidence to demonstrate the superiority of TacoPrompt over existing methods on three benchmark datasets.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "@inproceedings{xu2022taxoprompt,\n\n  title={TaxoPrompt: A Prompt-based Generation Method with Taxonomic Context for Self-Supervised Taxonomy Expansion.},\n\n  author={Xu, Hongyuan and Chen, Yunong and Liu, Zichen and Wen, Yanlong and Yuan, Xiaojie},\n\n  booktitle={IJCAI},\n\n  pages={4432--4438},\n\n  year={2022}\n}"}}, {"review_id": "i4bODoyAaN", "reviewer": "Reviewer_tsdR", "summary": "", "questions": "A. The baseline methods used are not the most recent state-of-the-art. Could you compare against methods from [1] and [2] to better situate the improvements?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your thoughtful comments and valuable suggestions. Below, we provide our response to the questions and concerns.\n\n**Q1. Missing Reference & The Choice of Baselines**\n\nA1. First, we actually have cited EMNLP 2021 TEMP and EACL 2023 ATTEMPT.\n\nSecond, we want to clarify why we didn’t compare our method with TEMP or ATTEMPT on the taxonomy completion (TC) task.\n\n1\\) TEMP is designed for the taxonomy expansion (TE) task, whose input is limited to a query and a candidate parent, while the input of the TC task includes a candidate child additionally. That’s possibly why all TC studies (including the latest published TaxoComplete [7]) haven’t compared with TEMP on the TC task.\n\n2\\) ATTEMPT aims to solve a quite different task from the TC task. Specifically, it firstly finds the <parent, query> and secondly tries to find all children for the query by only considering the query's siblings under the fixed parent (instead of all candidate parents). ATTEMPT has neither published its code nor provided enough details on how to score all candidate <parent, child> pairs to perform the TC task.\n\nAdditionally, **we supplement the reproduction results of the latest published work TaxoComplete [7]**, which could be seen as contemporaneous work to our submission according to the Paper Submission Information of EMNLP 2023. I hope these supplementary results can address your concerns about the choice of baselines and can provide more convincing comparisons. Note that  '#' and '*' refer to leaf and non-leaf queries, respectively.\n\n\n| Food      | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----  | ---- | ---- | ---- | ---- |\n| TaxoComplete | 96.039  | 0.507 | 13.9 | 30.0 | 38.3 | 28.4 | 60.1 | 71.6 | 0.764 | 65.6 | 70.9 | 0.322 | 34.6 | 14.8 |\n| Ours-Chain  | 51.643 | 0.717 | 28.6 | 51.4 | 60.8 | 60.1 | 85.8 | 89.2 | 0.886 | 87.8 | 86.7 | 0.587 | 76.0 | 40.9 |\n| Ours-Hidden | 47.423 | 0.708 | 30.9 | 51.1 | 60.1 | 64.9 | 85.8 | 86.5 | 0.899 | 87.8 | 87.4 | 0.561 | 76.0 | 39.2 |\n\n| MeSH     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 93.741 | 0.565 | 10.7 | 30.5 | 43.6 | 20.6 | 50.2 | 64.3 | 0.616 | 49.4 | 51.5 | 0.526 | 52.0 | 37.5 |\n| Ours-Chain |  76.579 | 0.678 | 17.4 | 41.8 | 56.4 | 39.6 | 72.9 | 82.4 | 0.833 | 77.4 | 78.2 | 0.583 | 63.2 | 43.0 |\n| Ours-Hidden |  49.140 | 0.674 | 17.9 | 42.4 | 55.9 | 40.7 | 74.6 | 84.6 | 0.868 | 79.0 | 81.5 | 0.554 | 65.1 | 40.2 |\n\n| Verb     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 596.347 | 0.424 | 7.7 | 21.1 | 30.4 | 12.2 | 31.6 | 42.6 | 0.529 | 33.2 | 44.4 | 0.319 | 25.4 | 16.4 |\n| Ours-Chain   | 597.098 | 0.546 | 18.5 | 35.5 | 45.2 | 28.3 | 49.3 | 58.9 | 0.692 | 52.9 | 61.9 | 0.385 | 34.7 | 26.8 |\n| Ours-Hidden  |  436.799 | 0.557 | 18.3 | 36.9 | 46.5 | 28.0 | 52.3 | 62.5 | 0.762 | 56.5 | 65.8 | 0.370 | 35.2 | 25.3 |\n\n**Q2. The Novelty of Our Framework**\n\nA2. We've already explained why TEMP and ATTEMPT are not TC methods in the Q1 reply. It's also worth noting that **TEMP and ATTEMPT haven’t leveraged the prompt learning paradigm to solve TC tasks**. These methods focus on providing LMs with the structural information of taxonomies by describing path-like structures with natural language, which is totally different from our contributions.\n\nIn this paper, we leverage the prompt-based learning paradigm to address two problems that exist only in TC (not in TE): (1) learning to attach non-leaf queries with limited training samples and (2) designing a better multi-task learning TC method.\n\nThe core idea of the prompt learning paradigm is to reformulate the downstream tasks into the form of the [MASK] prediction to align with the PLM’s pre-training task [1, 2], which for our backbone LM is the MLM task [3]. \nSuch a paradigm has been proven effective in learning with limited labelled training samples [4]. At the same time, we observe that the TC task suffers from task degradation caused by an unbalanced training data distribution (few non-leaf training samples). Therefore, **our first contribution is to reformulate the TC task into the MLM form based on the core idea of the prompt learning paradigm, improving the performance of the non-leaf query attachment with limited training samples.**\n\nBesides, the prompt learning paradigm has shown superiority in PLM-based multi-task learning [5] while TC task naturally consists of multiple tasks. This is the second reason why we leverage the prompt learning paradigm to solve the TC task. \nWe found that previous TC methods only leverage subtasks as auxiliary supervision for training representations, which are used only for final prediction. They ignore the potential of prediction results of subtasks to improve the TC task while recent studies have found that the step-by-step prediction method performs better than directly predicting the final outputs [6]. Thus, **our second contribution is to further improve prompt-based multi-task learning for TC by task collaboration through the design of the result context.**\n\n\n**Q3. The Choice of the Backbone LM**\n\nA3. Our main contributions are not limited to the choice of the backbone LM. The choice of BERT mainly aims to align with the experimental settings of previous works for fair comparison.\nWe appreciate the interesting idea of leveraging causal LMs or LLMs to perform the TC task, which is inspirational for our future work.\n\nThank you for the valuable suggestions. Please let us know if you have any other comments/questions.\n\nBest,\n\nAuthors\n\n**References**\n\n[1] Cai X, Xu H, Xu S, et al. Badprompt: Backdoor attacks on continuous prompts[J]. Advances in Neural Information Processing Systems, 2022, 35: 37068-37080.\n\n[2] Xu S, Pang L, Shen H, et al. Match-Prompt: Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning[C]//Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022: 2290-2300.\n\n[3] Chen X, Li L, Zhang N, et al. Decoupling knowledge from memorization: Retrieval-augmented prompt learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 23908-23922.\n\n[4] Brown T, Mann B, Ryder N, et al. Language models are few-shot learners[J]. Advances in neural information processing systems, 2020, 33: 1877-1901.\n\n[5] Fu J, Ng S K, Liu P. Polyglot Prompt: Multilingual Multitask Prompt Training[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022: 9919-9935.\n\n[6] Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[J]. Advances in Neural Information Processing Systems, 2022, 35: 24824-24837.\n\n[7] Arous I, Dolamic L, Cudré-Mauroux P. TaxoComplete: Self-Supervised Taxonomy Completion Leveraging Position-Enhanced Semantic Matching[C]//Proceedings of the ACM Web Conference 2023. 2023: 2509-2518.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Contributions are clearly articulated and convincingly demonstrated.\n- Addresses limitations of prior work in non-leaf concept attachment.\n- Strong empirical results, outperforming prior methods substantially.\n\n---\n\nOverall, the paper has solid experimentation, the results are relevant to the claims, and it demonstrates that using this method is reasonable. Therefore, I recommend a **Soundness=4.5**", "reasons_to_reject": "- While the author claims to be the first to propose the MLM format for solving the taxonomy completion (TC) task, utilizing prompts to help BERT solve the TC task is not novel [1][2]. The author attributes the inspiration of TacoPrompt-Chain to Chain of Thought, but the author did not choose causal language models like GPT/OPT/Bloom/Llama to perform the TC task (to my knowledge, placing \"Yes\"/\"No\" at the end would allow a similar approach with TacoPrompt). We also do not know how well Large Language Models would perform.\n\n---\n\nTherefore, I still have doubts regarding Excitement, and if the author can adequately address my concerns, I am willing to reconsider the rating.", "questions_to_authors": "A. The baseline methods used are not the most recent state-of-the-art. Could you compare against methods from [1] and [2] to better situate the improvements?", "paper_topic_and_main_contributions": "This paper proposes a new method called TacoPrompt for taxonomy completion. Taxonomy completion aims to attach new concepts to appropriate hypernym-hyponym pairs in an existing taxonomy. The key ideas are: Formulate taxonomy completion as a masked language modeling task, which allows efficiently learning to attach non-leaf concepts with few labeled examples. Use a multi-task learning approach with two subtasks: attaching the concept to the hypernym, and attaching it to the hyponym. Introduce a contextual approach to integrate the subtask results, showing them as contexts to enable collaboration between tasks. Propose two variants: TacoPrompt-Chain uses explicit subtask answers as context, while TacoPrompt-Hidden uses implicit vector representations. Experiments on 3 datasets show significant improvements over prior state-of-the-art methods, especially for attaching non-leaf concepts.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "None"}, "extra_text": {"Missing_References": "[1] Liu Z, Xu H, Wen Y, et al. TEMP: taxonomy expansion with dynamic margin loss through taxonomy-paths[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021: 3854-3863.\n\n[2] Xia F, Weng Y, He S, et al. Find Parent then Label Children: A Two-stage Taxonomy Completion Method with Pre-trained Language Model[C]//Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023: 1032-1042."}}, {"review_id": "UTxweKfd4U", "reviewer": "Reviewer_6PQe", "summary": "", "questions": "A. Why is the current prompting template chosen in the first place? Would be good to see some ablation studies or analysis on the prompting template.\n\nB. In some of the performance of the baselines reported in the main table, it seems that many methods performed worse than the baselines before them, which do not follow the patterns studied in the previous papers. Could you explain why these methods (TMN is better than TaxoEnrich and QEN in Food dataset, TMN is worse than Arborist in MeSH dataset, etc.) became worse in the comparisons?\n\nC. Can you report the running time and the efficiency of the proposed method compared to other baselines? It seems to me that the framework is complex when the number of nodes becomes very large.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and valuable suggestions. Below, we provide our response to the questions and concerns.\n\n**Q1. The Novelty of Our Framework**\n\nA1. \n**GenTaxo and TaxoEnrich haven’t leveraged the prompt learning paradigm to solve the taxonomy completion (TC) task**. These methods focus on utilizing LMs to learn structural information of taxonomies by describing tree-like or path-like structures with natural language and feeding them into LMs for better structural representations, which is totally different from our main contributions that (1) leveraging prompt-based learning paradigm for better non-leaf attachment with limited labelled training samples; (2) designing the result context to improve the prompt-based multi-task learning for TC.\n\nIt is also worth noting that **the multi-task learning objective is not claimed as the main contribution or novelty of our framework**. We focus more on the effectiveness of result context in realizing task collaboration to improve prompt-based multi-task learning for TC, which has not been discussed in previous TC methods.\n\nPlease refer to the response to Reviewer tsdR#Q2 for more discussions about the novelty of our framework. \n\n**Q2. The Design Reasons and Ablation Studies of Our Prompt Template**\n\nA2. Our prompt template includes three different parts: \n\n1\\) delimiter context which aims to help the LM understand different sentences, for example, ''<Parent> sleep is to be asleep <Child>'' aims to help the LM understand that the description of the parent concept ''sleep'' is between ''<Parent>'' and ''<Child>''; \n\n2\\) task-specific context which aims to provide more semantical information, which refers to descriptions of the parent, child and query; \n\n3\\) result context that aims to relate the results of subtasks to the final prediction, which is one of main our contributions.\n\nWe have performed ablation studies of 3\\) in Table 3, discussed the effectiveness of 2\\) and 3\\) in Section 4.3 ''Effects of Different Contexts'' and studied the influence of different designs of 1\\) in Appendix B ''Effects of Delimiter Context''.\n\n**Q3. The Performance of Baselines**\n\nQ3.1 Why does TMN perform better than TaxoEnrich on the Food dataset?\n\nA3.1 First, we found that all the results on the food dataset reported by QEN [3] are unconverged. Thus, we carefully tuned the parameters for each method and trained them to converge.\nThe results show that TMN is better than TaxoEnrich (but not than QEN) on the Food dataset. Such results can be attributed to the sensitivity of TaxoEnrich to the scarcity of taxonomies. SemEval-Food is a low-resource dataset and its non-connectivity may cause the representation generation module of TaxoEnrich difficult to extract the structure information during the training.\n\nSecond, description resources have been proven effective in providing extra semantic information in taxonomy-related tasks [1, 2, 3]. We leveraged descriptions to generate fasttext embeddings for TMN (it originally uses concepts’ surface names to generate embeddings), **aiming to reduce the impact of the use of extra semantic resources when comparing different methods.** The performance of TMN using different embedding generation resources is presented below. We can observe that **TaxoEnrich is better than TMN-Name, which follows the patterns studied in the previous papers.** Note that  '#' and '*' refer to leaf and non-leaf queries, respectively.\n\n| SemEval-Food | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----  | ---- | ---- | ---- | ---- |\n| TMN-Desc  | 153.963 | 0.423 | 18.0 | 31.0 | 34.0 | 36.5 | 58.8 | 64.2 | 0.809 | 70.7 | 74.8 | 0.107 | 0.0 | 0.6 |\n| TaxoEnrich | 230.424 | 0.408 | 11.7 | 26.7 | 31.7 | 23.6 | 49.3 | 58.1 | 0.723 | 58.5 | 66.7 | 0.149 | 4.0 | 3.0 |\n| TMN-Name | 140.961 | 0.358 | 9.0 | 18.7 | 24.0 | 18.2 | 37.2 | 41.9 | 0.541 | 41.5 | 43.0 | 0.208 | 16.0 | 8.5 |\n\nQ3.2 Why does TMN perform worse than Arborist on the MeSH dataset?\n\nA3.2 TMN actually performs better than Arborist on the MeSH dataset as reported in Table 2 in the paper. We also experiment with surface name-based embeddings for TMN on the MeSH dataset and find that the performance of TMN is improved as shown in the table below. Such improvement can be due to the over-long average description length in MeSH, resulting in low-quality concept embeddings generated by averaging token embeddings.\n\n| MeSH | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----  | ---- | ---- | ---- | ---- |\n| TMN-Desc  | 1342.775 | 0.195 | 2.7 | 7.4 | 11.0 | 6.0 | 15.5 | 23.1 | 0.350 | 19.7 | 22.7 | 0.094 | 6.5 | 3.4 |\n| Arborist | 1045.969 | 0.198 | 1.1 | 5.1 | 9.6 | 2.4 | 9.2 | 16.6 | 0.235 | 7.0 | 11.4 | 0.173 |13.8 |8.4 |\n| TMN-Name | 548.434 | 0.391 | 7.4 | 17.4 | 26.6 | 16.4 | 34.1 | 46.8 | 0.548 | 40.0 | 43.8 | 0.289 | 21.5 | 15.3 |\n\nSupplementary experiments illustrate that the performance of TMN is related to the representation generation resources, the effectiveness of which varies across datasets. Thank you for pointing out this issue and helping us to find that the description-based representation is not always the best for TMN.\n\n**Q4. The Efficiency of the Proposed Method**\n\nA4. Assuming that the number of queries is $|C|$ and the number of candidate positions is $|A|$, the inference time complexity of baselines is $O(|C|+|A|)$, while that of our method is $O(|C|*|A|)$. Such inference time complexity difference comes from the different base models (please refer to the response to Reviewer j9Kp#Q2 for more discussions about base models). \nThe efficiency is indeed one of our method’s limitations as discussed in the paper. We will study how to improve efficiency in our future work.\n\n**Q5. Missing Reference: KDD 2021 GenTaxo**\n\nQ5.1 Why didn't we cite GenTaxo?\n\nA5.1 GenTaxo changes the problem assumption and proposes a new task, \"taxonomy generation\". It is different from the common definition of the TC task in that new concepts are generated instead of being given when completing the taxonomy. It focuses more on the new concept generation than on the TC.\n\nQ5.2 Why didn't we make GenTaxo our baseline?\n\nA5.2 As mentioned above, GenTaxo focuses on a different task instead of the TC task. Although the author claims that GenTaxo++ is suitable for the common TC task, its \"completion\" stage uses the existing TC method—TaxoExpan, which has been our baseline model.\nBesides, GenTaxo's output is unsuitable for our ranking-based metrics. GenTaxo generates the concept for the candidate position instead of a ranking list for potential parents.\n\nTo provide more comparisons,  **we supplement the reproduction results of the latest published work TaxoComplete [7]**, which could be seen as contemporaneous work to our submission according to the Paper Submission Information of EMNLP 2023. Note that  '#' and '*' refer to leaf and non-leaf queries, respectively.\n\n| Food      | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----  | ---- | ---- | ---- | ---- |\n| TaxoComplete | 96.039  | 0.507 | 13.9 | 30.0 | 38.3 | 28.4 | 60.1 | 71.6 | 0.764 | 65.6 | 70.9 | 0.322 | 34.6 | 14.8 |\n| Ours-Chain  | 51.643 | 0.717 | 28.6 | 51.4 | 60.8 | 60.1 | 85.8 | 89.2 | 0.886 | 87.8 | 86.7 | 0.587 | 76.0 | 40.9 |\n| Ours-Hidden | 47.423 | 0.708 | 30.9 | 51.1 | 60.1 | 64.9 | 85.8 | 86.5 | 0.899 | 87.8 | 87.4 | 0.561 | 76.0 | 39.2 |\n\n| MeSH     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 93.741 | 0.565 | 10.7 | 30.5 | 43.6 | 20.6 | 50.2 | 64.3 | 0.616 | 49.4 | 51.5 | 0.526 | 52.0 | 37.5 |\n| Ours-Chain |  76.579 | 0.678 | 17.4 | 41.8 | 56.4 | 39.6 | 72.9 | 82.4 | 0.833 | 77.4 | 78.2 | 0.583 | 63.2 | 43.0 |\n| Ours-Hidden |  49.140 | 0.674 | 17.9 | 42.4 | 55.9 | 40.7 | 74.6 | 84.6 | 0.868 | 79.0 | 81.5 | 0.554 | 65.1 | 40.2 |\n\n| Verb     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 596.347 | 0.424 | 7.7 | 21.1 | 30.4 | 12.2 | 31.6 | 42.6 | 0.529 | 33.2 | 44.4 | 0.319 | 25.4 | 16.4 |\n| Ours-Chain   | 597.098 | 0.546 | 18.5 | 35.5 | 45.2 | 28.3 | 49.3 | 58.9 | 0.692 | 52.9 | 61.9 | 0.385 | 34.7 | 26.8 |\n| Ours-Hidden  |  436.799 | 0.557 | 18.3 | 36.9 | 46.5 | 28.0 | 52.3 | 62.5 | 0.762 | 56.5 | 65.8 | 0.370 | 35.2 | 25.3 |\n\nThank you for the valuable suggestions. Please let us know if you have any other comments/questions.\n\nBest,\n\nAuthors\n\n**References**\n\n[1] Liu Z, Xu H, Wen Y, et al. TEMP: taxonomy expansion with dynamic margin loss through taxonomy-paths[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021: 3854-3863.\n\n[2] Wang S, Zhao R, Chen X, et al. Enquire one’s parent and child before decision: Fully exploit hierarchical structure for self-supervised taxonomy expansion[C]//Proceedings of the Web Conference 2021. 2021: 3291-3304.\n\n[3] Wang S, Zhao R, Zheng Y, et al. Qen: Applicable taxonomy completion via evaluating full taxonomic relations[C]//Proceedings of the ACM Web Conference 2022. 2022: 1008-1017.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed method of using prompts in the taxonomy completion task is interesting and the motivation is clear.\n2. The writing is well-structured and easy to follow overall.\n3. The experimental results show that the proposed framework is very effective.\n4. The framework was extended to TacoPrompt-Chain and Hidden. These two variants are well-motivated and shown to be effective.", "reasons_to_reject": "1. The overall novelty and contribution of the paper are limited. The prompting template is similar to what is described GenTaxo (Zeng et al., 2021) and TaxoEnrich (Jiang et al., 2022) where these two papers used pseudo sentences to generate the embeddings for the concepts. But the prompting template used in this paper is definitely more natural and effective. The multi-task learning objective is almost the same as what was used in TMN (Zhang et al., 2021), which also considers the relationships between query node and parent, query node and child respectively.\n2. See the questions below.", "questions_to_authors": "A. Why is the current prompting template chosen in the first place? Would be good to see some ablation studies or analysis on the prompting template.\n\nB. In some of the performance of the baselines reported in the main table, it seems that many methods performed worse than the baselines before them, which do not follow the patterns studied in the previous papers. Could you explain why these methods (TMN is better than TaxoEnrich and QEN in Food dataset, TMN is worse than Arborist in MeSH dataset, etc.) became worse in the comparisons?\n\nC. Can you report the running time and the efficiency of the proposed method compared to other baselines? It seems to me that the framework is complex when the number of nodes becomes very large.", "paper_topic_and_main_contributions": "The paper proposes TacoPrompt, which utilizes prompting and multi-task objectives to address the taxonomy completion task. It leverages the prompt-based template to compute the probability of the query concept being attached to the candidate position. And then the framework divides the learning objective into two subtasks to learn the relationships between query nodes and the candidate positions respectively. And the experiments show that the proposed method outperforms all compared baselines by a large margin.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "N/A"}, "extra_text": {"Missing_References": "Zeng, Qingkai, et al. \"Enhancing taxonomy completion with concept generation via fusing relational representations.\" Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021.\nHope to see some discussions and comparisons maybe with this paper since it's also one major baseline in other previous papers."}}, {"review_id": "tLDB8pIbab", "reviewer": "Reviewer_Z6sr", "summary": "", "questions": "1. I am curious about why the framework performs poorly on the WordNet dataset compared to the other datasets.?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful comments and valuable suggestions. Below, we provide our response to the questions and concerns.\n\n**Q1. The Novelty of Our Framework**\n\nA1. We have to emphasize that **reformulating the downstream task to a Masked Language Model (MLM) form is actually the core idea of the prompt learning paradigm [1]**. Please refer to the response to Reviewer tsdR#Q2 for more detailed discussions about the novelty of our framework.\n\n**Q2. The Lack of Network Information**\n\nA2. We agree with you that the network information contributes to the taxonomy expansion (TE) task and taxonomy completion (TC) task and many previous works have focused on utilizing such information [2, 3, 4]. **However, our work is not concentrated on better network information utilization**. In fact, we aim to address two issues that only exist in the TC task (and not in the TE task): \n(1) learning to attach non-leaf queries with limited training samples and (2) designing a better multi-task learning TC method.\nOur experiments have demonstrated that the prompt-based learning TC framework and collaborative multi-task learning using the result context can address the above two issues, which are the actual impacts of our proposed method.\n\n**Besides, we have compared many TC baselines that leverage network information**, including QEN and TaxoEnrich, as shown in Table 2 in the paper. **Moreover, we supplement the reproduction results of the latest published work TaxoComplete [7]**, which could be seen as contemporaneous work to our submission according to the Paper Submission Information of EMNLP 2023.\n\nTaxoComplete leverages the distance information in the network as self-supervision when performing TC. I hope these supplementary results can address your concerns and provide more comparisons. Note that '#' and '*' refer to leaf and non-leaf queries, respectively.\n\n\n| Food      | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ----  | ---- | ---- | ---- | ---- |\n| TaxoComplete | 96.039  | 0.507 | 13.9 | 30.0 | 38.3 | 28.4 | 60.1 | 71.6 | 0.764 | 65.6 | 70.9 | 0.322 | 34.6 | 14.8 |\n| Ours-Chain  | 51.643 | 0.717 | 28.6 | 51.4 | 60.8 | 60.1 | 85.8 | 89.2 | 0.886 | 87.8 | 86.7 | 0.587 | 76.0 | 40.9 |\n| Ours-Hidden | 47.423 | 0.708 | 30.9 | 51.1 | 60.1 | 64.9 | 85.8 | 86.5 | 0.899 | 87.8 | 87.4 | 0.561 | 76.0 | 39.2 |\n\n| MeSH     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 93.741 | 0.565 | 10.7 | 30.5 | 43.6 | 20.6 | 50.2 | 64.3 | 0.616 | 49.4 | 51.5 | 0.526 | 52.0 | 37.5 |\n| Ours-Chain |  76.579 | 0.678 | 17.4 | 41.8 | 56.4 | 39.6 | 72.9 | 82.4 | 0.833 | 77.4 | 78.2 | 0.583 | 63.2 | 43.0 |\n| Ours-Hidden |  49.140 | 0.674 | 17.9 | 42.4 | 55.9 | 40.7 | 74.6 | 84.6 | 0.868 | 79.0 | 81.5 | 0.554 | 65.1 | 40.2 |\n\n| Verb     | MR↓  | MRR  | R@1  | R@5  | R@10 | H@1  | H@5  | H@10 | MRR# | H#5  | R#10 | MRR* | H*5  | R*10 |\n| ------------ | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |\n| TaxoComplete | 596.347 | 0.424 | 7.7 | 21.1 | 30.4 | 12.2 | 31.6 | 42.6 | 0.529 | 33.2 | 44.4 | 0.319 | 25.4 | 16.4 |\n| Ours-Chain   | 597.098 | 0.546 | 18.5 | 35.5 | 45.2 | 28.3 | 49.3 | 58.9 | 0.692 | 52.9 | 61.9 | 0.385 | 34.7 | 26.8 |\n| Ours-Hidden  |  436.799 | 0.557 | 18.3 | 36.9 | 46.5 | 28.0 | 52.3 | 62.5 | 0.762 | 56.5 | 65.8 | 0.370 | 35.2 | 25.3 |\n\n**Q3. The Equation Problem**\n\nQ3.1 What is the difference between PM ([MASK] and PM ([M].\n\nA4.1 ‘[M]’ is short for ‘[MASK]’ and such an abbreviation is first denoted in line 234 in the paper.\n\nQ3.2 Equation 4 is difficult to understand.\n\nA3.2 Equation 4 indicates that the final prediction score is calculated by the difference between the probabilities of the ‘[MASK]’ token being predicted as ‘Yes’ and ‘No’.\n\n**Q4. The Performance on the WordNet-Verb Dataset**\n\nA4. Actually, not only our framework but also the previous SOTA method QEN performs relatively poorly on the WordNet-Verb dataset. Such relatively poor performance reflects the challenges of performing the TC task on this dataset.  We can conclude two main challenges of this dataset. First, the verb concept is semantically more abstract than the noun concept, making it harder to distinguish the hierarchical semantics of <parent, query, child>. Second, the WordNet-Verb dataset consists of several subtrees. The differences in the hierarchical distribution of these subtrees will increase the difficulty of distinguishing hierarchical semantics during training and inference.\n\n**Q5. Missing References**\n\nA5. TEAM is proposed to solve the TE task. We focus on introducing and comparing with the previous studies of the TC task instead of the TE task. Please refer to our response to Reviewer j9Kp#Q1 for more discussions about the difference between TC and TE.\n\nThank you for the valuable suggestions. Please let us know if you have any other comments/questions.\n\nBest,\n\nAuthors\n\n**References**\n\n[1] Chen X, Li L, Zhang N, et al. Decoupling knowledge from memorization: Retrieval-augmented prompt learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 23908-23922.\n\n[2] Yu Y, Li Y, Shen J, et al. Steam: Self-supervised taxonomy expansion with mini-paths[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020: 1026-1035.\n\n[3] Liu Z, Xu H, Wen Y, et al. TEMP: taxonomy expansion with dynamic margin loss through taxonomy-paths[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021: 3854-3863.\n\n[4] Jiang M, Song X, Zhang J, et al. Taxoenrich: Self-supervised taxonomy completion via structure-semantic representations[C]//Proceedings of the ACM Web Conference 2022. 2022: 925-934.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. As claimed by author, The paper proposes a novel prompt-based framework, TacoPrompt, for the challenging task of taxonomy completion.  While the concept of attaching new concepts to hypernyms and considering hyponyms is not new and has been previously studied, the novelty of TacoPrompt lies in its application of prompt learning to tackle the taxonomy completion task. Prompt learning is a recent and innovative technique that converts the task into a masked language model (MLM) form, effectively transforming it into a fill-in-the-blank problem.\n\n2. The authors conduct extensive experiments on three benchmark taxonomy completion datasets, comparing TacoPrompt with state-of-the-art methods and taxonomy expansion baselines. The comprehensive evaluation, along with ablation studies, demonstrates the effectiveness of the proposed approach in different scenarios and highlights its superior performance, especially for attaching non-leaf concepts.", "reasons_to_reject": "1. The paper claims its novelty in utilizing prompt learning for taxonomy completion. However, it seems that the approach primarily revolves around applying a Masked Language Model (MLM) in a taxonomy sequence. This raises questions about the true novelty of the contribution.\n\n2.  Moreover, The reported results do demonstrate improvements over baselines, including \"TaxoExpan\". However, considering that \"TaxoExpan\" already incorporates attention mechanisms and network information, it raises questions about the true significance of the proposed method's performance enhancement. It is worth noting that the current approach appears to lack the utilization of network information, which could be essential in the context of taxonomy expansion tasks. Additional clarification and analysis from the authors would be valuable to better comprehend the actual impact of their proposed approach, especially in comparison to methods that leverage network information.\n\n3. The authors have introduced a relatively straightforward approach, but the presentation and explanation of the method tend to be  complex.\n\n4. It seems that there are some inconsistencies and issues with the equations presented in the paper. For example... (1) What is the difference between PM ([MASK] and PM ([M]. (2). Equation 4 is difficult to understand.", "questions_to_authors": "1. I am curious about why the framework performs poorly on the WordNet dataset compared to the other datasets.?", "paper_topic_and_main_contributions": "This paper proposed a frame work TacoPrompt, a prompt-based framework for self-supervised taxonomy completion. The taxonomy completion task involves attaching new concepts to the most suitable positions in an existing taxonomy.  \n\nThe paper highlight the limitations of simply attaching new concepts to hypernyms without considering hyponyms. Instead, they emphasize the completion task, where new concepts are added as parent-child pairs in existing taxonomies. \n\nThe paper suggests incorporating two subtasks - hypernym and hyponym matching  to enhance concept representation learning. However, the existing methods lack a comprehensive understanding of how these subtasks contribute to the main task and ignore their interdependencies. Moreover, paper mentioned that difficulties arise in attaching non-leaf queries, especially in low-resource or imbalanced taxonomies. The paper aims to address these issues and provide insights into improving completion methods for taxonomies.\n\nTacoPrompt utilizes two variants of result context, explicit answers and implicit hidden vectors, to facilitate task collaboration. Experimental results demonstrate that TacoPrompt outperforms state-of-the-art methods on benchmark taxonomy datasets. The paper also analyzes the effectiveness of different tasks, contexts, and backbone language models in the proposed framework. Overall, author claims that TacoPrompt offers a promising approach for efficient and accurate taxonomy completion in natural language processing tasks.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "1.While there may not be any prior work specifically on prompt learning for taxonomy completion, I came across a relevant study titled \"TEAM: A multitask learning based Taxonomy Expansion approach for Attach and Merge\". This paper delves into multi-task scenarios in the domain of taxonomy expansion. Considering the similarities in the research context, I believe that the reference to this paper is missing in this paper."}}]

Author rebuttal:

Meta-review: The reviewers express mixed views towards the paper that proposes a new prompt-based framework for taxonomy completion. 

Collectively, the strengths of the paper include a sound approach to the taxonomy completion task using prompt learning. Reviewers appreciated the extensive experiments on benchmark taxonomy datasets, the presentation of two innovative variants of TacoPrompt, and the solid empirical results, with significant improvement over previous methods.

However, questions have been raised about the novelty of using the Masked Language Model (MLM) and prompts in the taxonomy completion task, as the MLM/generation formulation has been studied for other IE tasks. Also, further comparisons with other techniques is required for a comprehensive evaluation
Decision: Accept-Main