Event Ontology Completion with Hierarchical Structure Evolution Networks
Authors: Pengfei Cao
Yupu Hao
Yubo Chen
Kang Liu
Jiexin Xu
Huaijun Li
Xiaojian Jiang
Jun Zhao
Venue: EMNLP 2023 Main (2023)
Keywords: Event Ontology Completion
Event Type Induction
Hierarchy Expansion
Type Naming

Abstract: Traditional event detection methods require predefined event schemas. However, manually defining event schemas is expensive and the coverage of schemas is limited. To this end, some works study the event type induction (ETI) task, which discovers new event types via clustering. However, the setting of ETI suffers from two limitations: event types are not linked into the existing hierarchy and have no semantic names. In this paper, we propose a new research task named Event Ontology Completion (EOC), which aims to simultaneously achieve event clustering, hierarchy expansion and type naming. Furthermore, we develop a Hierarchical Structure Evolution Network (HalTon) for this new task. Specifically, we first devise a Neighborhood Contrastive Clustering module to cluster unlabeled event instances. Then, we propose a Hierarchy-Aware Linking module to incorporate the hierarchical information for event expansion. Finally, we generate meaningful names for new types via an In-Context Learning-based Naming module. Extensive experiments indicate that our method achieves the best performance, outperforming the baselines by 8.23%, 8.79% and 8.10% of ARI score on three datasets.

Event Ontology Completion with Hierarchical Structure Evolution Networks
Pengfei Cao 1:2 Yupu Hao 1.2 Yubo Chen 1.2 Kang Liu Jiexin Xu', Huaijun Li", Xiaojian J Zhao
The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China China Merchants Bank (pengfei.cao, cao, yubo.chen kliu, jzhao}@nlpp pr ia
ia.ac. ac cn, haoyupu2023eia. ac. cn
Abstract
Type
Traditional event detection methods require predefined event schemas. However, manu- ally defining event schemas is expensive and the coverage of schemas limited. To this end, some works study the event type induction (ETI) task, which discovers new event types via clustering. However, the setting of ETI suffers from two limitations: event types are not linked into the existing hierarchy and have no seman- tic names. In this paper, we propose new research task named Event Ontology Com- pletion (EOC), which aims to simultaneously achieve event clustering, hierarchy expansion and type naming. Furthermore, we develop a HierarchicAL STructure EvOlution Network (HAlTon) for this new task. Specifically, first devise a Neighborhood Contrastive Clus- tering module cluster unlabeled event in- stances. Then, we propose a Hierarchy-Awar Linking module to incorporate the hicrarchical information for event expansion. Finally, generate meaningful names for new types via an In- Context Learning-bor -based Naming module. Extensive experiments indicate that method achieves the best performance, outperforming the baselines by 8.23%, 8.7 8.79% and 8. 8.10% of ARI score on three datasets!.
Model
ETT
Type
Type 3
Uniabeled
An illustration of event type induction
Uniabeled Instances
Event Chasters
Hierarchy Expansion
Event Roo:
Life
Eveul Clustering
Justic
Contact
Nomiig
EOC Model
Sue
Sextrace
Keet
Aneet
Event Ostology
(b) An illustraiion of event entology completion
Figure 1: (a) Event type induction only clusters un- labeled event instances into several groups. (b) Event ontology completion not only discovers new event types, but also adds them into the existing event hierarchy and generates meaningful names for them.
examine amounts of raw data in advance to specify potential event types. Besides, happening every day (Cao et al., 2020; Yu et al., 2021; Liu et al., 2022a), is neither realistic nor scalable define all event schemas in advance.
events are
To get rid of the above problems, some re- searchers study the task of event type induction (ETI), which aims to discover new event types from an input corpus (Yuan al., 2018; Huang and Ji, 2020; Shen et al., 2021). The task generally for- mulated as a clustering problem, where each cluster represents an event type (cf. Figure 1(a)). Existing methods typically utilize probabilistic generative models (Chambers, 2013; Nguyen et al., 2015), ad- hoc clustering algorithms (Sekine, 2006; Huang et al., 2016) neural networks (Huang and Ji, 2020; Shen et al., 2021; Li al., 2022) to induce event clusters. Despite these successful efforts for clustering, the ETI setting inevitably suffers from two limitations in real applications:
1 Introduction
Automated real-world event detection is a crucial task towards mining fast-evolving event knowledge. Existing methods (Ji and Grishman, 2008; Chen et al., 2015; Du and Cardie, 2020; Wang et al., 2022) typically require pre-defined event schema along with massive human-labeled data for model learning. Despite the tremendous success, manu- ally defining event schema is especially expen- sive and labor-intensive, which requires experts to
*Corresponding author.
Code is available at https: wtwggttthu.cccco cOm.cb/-LC///-NL
HOLTON.

---

Event types are not linked into the existing hierarchy: These methods only divide unlabeled event instances into several isolated clusters, with- out linking newly discovered types to an existing event ontology (i.e., an event hierarchy) Some studies about human cognition find that people tend to organize real-world events in a hierarchical way (Burt et al., 2003; Tenenbaum et al., 2011), ranging from coarse-grained (i.e., top-level) events to fine- grained (i.e., bottom-level) events. Moreover, the ontologies of most knowledge bases also adopt hi- erarchical organization forms of event types (Baker et al., 1 1998; Kingsbury and Palmer, 2003). The hi- erarchical forms represent events at different gran- ularity and abstraction levels, which helps people quickly understand related scenarios. For example, according to the event hierarchy in Figure 1(b), can easily gain the overall picture of the Justice scenario, which may involve multiple events, such very
the task requires that the model simultancously achieves the following goals: (1) Event Clustering: dividing the unlabeled instances into several clus- ters; (2) Hierarchy Expansion: linking new event types (i.e., predicted clusters) into an existing event hierarchy; and (3) Type Naming: generating se- mantically meaningful names for new event types. As shown in Figure 1(b), the EOC model aims to divide the unlabeled instances into three clusters, and link the clusters to the Root and Justice node of the event hierarchy. Meanwhile, the three new event types are named Life, Sue and Arrest, respec- tively. Compared to ETI, EOC requires models to complete the event ontology. instead of only event clustering. Therefore, the proposed task is more useful and practical, but it is also more challenging.
To this end, we propose novel method named HierarchicAL STructure EvOlution Network (HalTon) for this new task. Concretely, we first devise a Neighborhood Contrastive Clustering module for event clustering. The module utilizes a neighborhood contrastive loss to boost clustering for both supervised and unsupervised data. Intu- itively, in a semantic feature space, neighboring instances should have a similar type, and pulling them together makes clusters more compact. Then, propose a Hierarchy-Aware Linking module for hierarchy expansion. The module uses dynamic path-based margin loss integrate the hierarchical information into event representations. Compared with the static margin, the dynamic margin can cap- ture the semantic similarities of event types in the hierarchy, which is conducive to hierarchy expan- sion. Finally, we design an In- Cn-Ceotext Learning- based Naming module for type naming. The mod- ule clicits the abstraction ability of large language models (LLMs) via in-context learning to generate human readabee names for discovered event types. Extensive experiments on three datasets show that our proposed method brings significant improve- ments over baselines.
as Sue, Arrest and Sentence. Therefore, necessary to establish and maintain the event hier- archy. However, since new events emerge rapidly and incessantly, it is impractical to manually add newly discovered types into the event ontology. Therefore, how to automatically expand the ex- isting event hierarchy with new event types is important problem.
Event types have no semantic names: Most ETI methods only assign numbers (i.e., type num- ber) the new event types, and lack the ability to generate human-readable type names. To enable new event types to be used in downstream tasks, is inevitable to assign meaningful names for them in advance. For example, the event type name is required for training event extraction models (Li et al., 2021b) and constructing event knowledge graphs (Ma et al., 2022). Although the event type name is important, previous studies only focus on event clustering and ignore the type naming (Huang et al., 2016; Huang and Ji, 2020; Shen et al.. 2021). As a result, the discovered event types cannot be di- rectly applied to downstream applications, and ex- tra human efforts are needed to conduct secondary labeling for the new types. Thus, how to automat- ically generate meaningful names for new event types is also a problem worth exploring.
To summarize, our contributions are: (1) As a seminal study, we propose a new research task named event ontology completion, and introduce baselines and evaluation metrics for three task set- tings, including event clustering, hierarchy expan- sion and type naming. (2) We devise novel method named Hierarchical Structure Evolution Network (HALTON)), which achieves task goals via the collaboration of three components, namely neighborhood contrastive clustering, hierarchy-
In the light of the above restrictions, we propose a new task named Event Ontology Completion (EOC). Given a set of unlabeled event instances,
Fvvntt ontology denotes the hicrarchical organization structure of known event types, which is usually incomplete.

---

aware linking and in-context learningbbased nam-
BERT outputs obtain the event representation:
ing. It can serve as strong baseline for the re- search on the task. (3) Experimental results in- dicate that our method substantially outperforms baselines, achieving 8.23%, 8.79% and 8. 10% im- provements of ARI score on three datasets.
h...... ,, h. = BERT(a))
(I)
h Moxx-Poolinghh....... hh.).
where x denotes the input sentence. n is the length of the input sentence. and represent the start and end positions of the trigger, respectively.
2 Task Formulation
Base Losses In this way, we obtain the event representations of labeled and unlabeled instances, denoted as {h(),= and (h f==1- respectively. We feed the representations of labeled instances into softmax function for prediction, and utilize the cross-entropy loss to train the model:
The EOC task assumes that there is an incomplete event ontology T, which is constructed by experts in advance. The ontology is tree-like structure, where leaf nodes denote known event types. Given an unlabeled dataset T = {. h= and an esti- mated number unknown types Mn, the goals of EOC include: (1) Event Clustering, dividing the unlabeled instances into Ma groups; (2) Hi- erarchy Expansion, linking each cluster C to the corresponding position of the hierarchy T; and (3)
Lee N Σ (olggssoftmax(()))
where y is a one-hot vector representing the golden label of the instance z¹. For unlabeled in- stances, we use the K-means algorithm to obtain their pscudo labels:
Type Naming, generating human-readable name for each cluster C. Following Li al. (2022), we use golden triggers for event clustering?. To enable the model to achieve the above goals, we leverage a labeled dataset D' = [( ) to assist model learning. The types set of the labeled dataset is denoted as y The event types in y belong to known types, which correspond to the leaf nodes of the event ontology T.
g" K-means(h") E (l,..., M...
Since the order of clusters often changes in mul- tiple clustering, is not readily to use cross-entropy loss for training the model on unlabeled instances. Instead, we compute pair-wise pseudo labels, ac- cording to the clustering result:
3 Methodology
q: 1 07}.
(4)
Figure 2 shows the overall architecture of HAL- TON, which consists of three major components: (1) Neighborhood Contrastive Clustering ($3.1), which learns discriminative representations for event clustering; (2) Hierarchy-Aware Linking (63.2), which attaches newly discovered event types to the existing event hierarchy; and (3) In-Context Learning-bar based Naming ($3.3), which generates event type names via in-context learning. We will illustrate each component in detail.
where dz denotes whether xa and z belong the same cluster. We input the representations of unlabeled instances into a classifier obtain pre- dicted distributions {p!) M Intuitively, if a pair of instances output similar distributions, can be assumed that they are from the same cluster. There- fore, we use the pair-wise Kullback-Leibler (KI.) divergence to evaluate the distance between two unlabeled instances:
de KL(p"Ile") + KL(p
3.1 Neighborhood Contrastive Clustering
If 2 and x belong different clusters, their predicted distributions are expected be different. Thus, we modify standard binary cross-entropy loss by incorporating the hinge-loss function (Zhao et al., 2021):
Encoding Instances Given the impressive perfor- mance of pre-trained language models on various NLP tasks (Sun al., 2022; Zhao al., 2023), we utilize BERT (Devlin al., 2019) to encode input sentences. Since the trigger may contain multiple tokens, we conduct a max- max-pooling operation over
Lace CH (ggdd (1- (:g)max(00 dıs)), (6)
"How to identify event triggers is not our focus in this paper. Actually, our method can he combined with any event detection model to extract event triggers.
where a is a hhper--prannnn for the hinge loss. C denotes the number of combinations.

---

Event Clusters
Unkmown Type Instances
Neighborhood Contrastive Clustering
Type
Hierarchy-
Tyge
Aware Linking
Type
BERT
Event Outology
Traiingg
Type Names
In-Context
Evest Root
Known Lastaances
Classifler Known Type
CE
Learning-based baanng Naming
IPM
Justice
Contact
NC Loss
Demonntrtions
Sun
Classifier Unknown Type
BCE Loss
Amest
Senternce
Meet
Figure 2: The architecture of the proposed Hierarchical Structure Evolution Network (HALTON) for the event ontology completion task. CE: cross-entropy, DPM: dynamic path-based blseb margin, NC: neighborhood contrastive. and BCE: binary cross-entropy.
Neighborhood Contrastive Loss Since con- trastive learning is very effective representation learning technique (He al., 2020; Zhong et al., 2021;Zu0 et al., 2021), we propose a neighborhood contrastive loss to learn more discriminative repre- sentations from both the labeled and unlabeled data. Concretely, for each instance Hı, select its top-K nearest neighbors in the embedding space to form a neighborhood Ng. The instances in N; should share a similar type as Tú, which are regarded as its positives. The neighborhood contrastive loss for unlabeled instances is defined as follows:
from type v/. which serve as anchor instance and positive instance p, respectively. We also randomly sample an instance from type y/ as negative in- stance n. The loss encourages a dynamic margin between the positive pair (a, p) and the negative pair (a, n), which is computed as follows:
Eapm
Σ max(0, sim(ha, hn) (olofe
+ + v/) sim hp)):
where S denotes the set of the combination of any two known event types. To more accurately reflect the similarity between two types in hierarchy, the margin (Vl. v/)) is computed based on the paths:
1 M
exp(sim(h") (h",, h.)rr/
Luen
log lep exp(sim(hjj hr)/r)
(7)
where sim(:.-) is the similarity function (e.g., dot product). is the temperature scalar. For labeled instances, the positives set is expanded with the instances having the same event type. Thus, the neighborhood contrastive loss for labeled instances is written follows:
(ATH( PATH( PATH of PATH
1, (10)
where PATH(y!) represents the set containing the nodes on the path from the root event the type 1/ If the intersection (i.e., less common super-classes), the margin will become larger. Therefore, compared with the static margin, the dynamic margin can capture the se- mantic similarities of event types in the hicrarchy. which is effective for event clustering and hierar- chy expansion. We reach the final loss function by combining the above terms:
the two paths smaller
exp(sim(Hi, (ilsmm,,,,,a 1(//T)
Lus
log Eide exp(sim(h(, (4,4)
M
(8)
where No denotes the positives for the labeled
instance 2!.
3.2 Hierarchy-Aware Linking
L = Las + Lice + Luen + Lost + Ldpm-
(11)
Dynamic Path-based Margin Loss To better ac- complish the hierarchy expansion, use the mar- gin loss (Schroff et al., 2015; Liu al., 2021) to integrate hierarchy information into event repre- sentations. To this end, we devise dynamic path- based margin loss. In detail, given two known event types yf and v/S. we randomly sample two instances
Greedy Expansion Strategy After training the model using the final loss function, can discover new event types and link them to the existing on- tology via a greedy expansion algorithm (Zhang et al., 2021). Specifically, for each new event type (i.e., predicted cluster), starting from the root node,

---

Prompt
parent node of the type. The question is "According to this, what is the type name?".
Task Description
Generate the type name according to the given text and event trigger The generated should one clear and brief word.
Incomplete Entry is filled by LLMs, whose composition is similar to the in-context examples. Intuitively, if the text provides more relevant in- formation about the event, the model will give more accurate predictions. Thus, select the instance closest to the cluster centroid the sen- tence. The path information is obtained via the hierarchy-aware linking module. As for the answer part, we leave it blank for LLMs to complete.
In-Context Examples
Sentence: British Chanctlor named head of the energy regullatrr as the new chairman of finance watchdog the FSA.
Event' Trigger:: named
Path:
Root:Ponsonnel
Question: According to this, what is the type name?
Answer: Nominale
Incomplete Entry
Sentence: As well as previously holding position at B Barclays
Bank, Smith was formerly servant the Department Industry.
Then, the constructed prompt is input into the LLMs (i.e., ChatGPT) for type name generation. This overall training and inference procedure is detailed in Appendix A.
Trigger::
previcusly
Path:
Root:Personnel
Question:
According to this, what is the type name?
Answer:
Figure 3: An example of prompt, including task descrip- tion, in-context examples and incomplete entry.
4 Experiments
4.1 Datasets
we compute the similarity between the new event type and its children nodes. Then, we select the event type (i.e., node) with the highest similarity to repeat the above process. The search process terminates if the similarity does not increase com- pared to the previous layer. The similarity between the new event type and existing event type is computed follows:
So far, there is no benchmark for evaluating EOC models. Based on three widely used event detection datasets, namely ACE (Doddington et al., 2004), ERE (Song et al., 2015), and MAVEN (Wang et al., 2020), we devise the following construc- tion method: for the ACE dataset, regard the top 10 most popular types are regarded known types and the remaining 23 event types as unknown types. For the ERE dataset, we also set the top 10 most popular types as seen and the remaining 28 types as unseen. For the MAVEN dataset, we select the top 60 most frequent types to alleviate long-tail problem, where the top 20 most popular event types serve as known types and the remain- ing 40 types regarded unknown types. For the three datasets, the event hierarchy is tree- like structure constructed by known types. We list known and unknown types in Appendix B.
S(yas Ve) = Exvep, Exeer, sim(le,,
(12)
[Pa][Pa]
where yn is a new event type (i.e., event cluster)
and ye is an existing event type. Pn, and Pe denote the sets of event instances belonging to yn and ye. respectively.
3.3 In-Context Learning-basee Naming
To obtain human-readable name for each pre- dicted cluster, we propose an in-context learning- based naming technique, which elicits the naming ability of LLMs by providing few demonstrative instances (Li et al., 2023). We first construct the prompt for LLMs. Figure shows an example of the prompt, which includes three parts:
4.2 Event Clustering Evaluations
We compare our HALTON with the fol- lowing methods: (1) SS-VQ-VAE (Huang and Ji, 2020) utilizes vector quantized variational autocn- coder to learn discrete latent representations for seen and unseen types. (2) ETYPECLUS (Shen et al., 2021) jointly embeds and clusters predicate- object pairs in latent space. (3) TABS (Li et al., 2022) designs co-training framework that com- bines the advantage of type abstraction and token- based representations.
Baselines
Task Description is a short description of the task. We devise a simple and effective version, i.e., "Generate the type name according to the given text and event trigger. The generated name should be one clear and brief word."
In- Context Examples consist of the sentence, event trigger, path, question and answer, As shown in Figure 3, the starting point of the path is the root node of the hierarchy, and the ending point is the
Evaluation Metrics Following previous ETI works (Huang and Ji, 2020; Li et al., 2022), we

---

Datasets
Methods
ART (%)
NMT (%)
Accuracy (%)
BCuhed-F1 (%)
SS-VOL -S-VQ-VAE
8.53
33.81
29.95
27.60
ETYPECTUS
26.17
53.91
40.70
38.69
ACE
TABS
59.18
79.36
71.42
69.44
HALTON (Ours)
67.41 (1 8.23)
84.29 (1 4.93)
77.26 (1 5.84)
75.06 (1 5.62)
SS-VQ-VAE ETYPECLUS
13.46
40.45
29.96
26.69
15.89
46,80 86
34.55
29.13
ERE
TABS
47.22
71.26
60.24
55.82
HALTON (Ours)
56.01 (1 8.79)
78.13 (1 6.87)
67.72 (1 7.48)
64.66 (1 8,84)
SS- SS-VQ-VAF
3.06
17.57
12.29
11.14
ETYPECLUS
11.27
30.79
20.82
14.73
MAVEN
27.93
39.38
31.52
HALTON (Ours) 36.03 (1 8.10)
60.34 (1 6.50) 52.70 (1 13.32)
39.35 7.83)
Table 1: Event clustering results the ACE, ERE and MAVEN datasets, respectively. The performance of our method is followed by the improvements (f) over the second best-perf model.
Predicted Cluster
Golden Cluster
Datasets
Methods
Taxo P (()
Taxo R (%)
Taxo Fl (%)
Taxo P (%)
Taxo R (%)
Taxo_ Fl (%)
-S-VQ-VAE+GE
9.12
10.14
9.60
9.52
13.04
11.01
ETYPECLUS+GE
30.70
23.46
26.59
34.14
33.33
33.73
TABS+GE
34.31
30.43
32.25
33.33
37.68
35.37
ACE
Type_Simme Simmlarry
31.79
40.58
35.65
33.33
36.51
LLMs_ LLMMsPrrompt
34.09
34.78
34.43
42.85
43.47
43.16
HALTON (Ours)
37.00
39.13
38.04
2.39)
44.44
44.92
44.68 (1 1.52)
SS-VQ- VQ-VAFFGR
16.38
14.28
15.26
26.00
25.00
25.49
ETYPECLUS+GE
9.52
9.68
18.00
16.66
17.30
TABS+GE
23.68
17.85
20.36
26.00
25.00
25.49
ERE
Type Similarity
20.37
21.49
20.88
22.00
21.42
21.71
Promppt
20.68
20.43
20.55
24.00
21.49
22,64
HALTON (Ours)
22.54
23.60
23.06 2.18)
26.80
25.73
26.25 (1 0.76)
SS-VO-VAE+GE
19.45
20.14
19.79
26.94
43,00
33.13
CYYPECLUS+GG
15.83
17.50
16.62
23.75
28.75
26.01
TABS+GE
27.82
32.03
29.78
27.53
40.42
32,75
MAVEN
Type_Similarii
22.50
27.50
24.75
27.91
32.50
30.03
LLMs Prompt
12.50
10.00
11.11
27.50
21.50
23.97
HALTON (Ours)
34.79
52.50
41.85 (+ 12.07)
39.38
59.38
47.35 (f 14.60)
Table 2: Hierarchy expansion results on the ACE, ERE and MAVEN datasets, respectively. Predicted (Golden)
cluster refers linking predicted (golden) clusters to the ontology. "GE" denotes the greedy expansion algorithm.
adopt several standard metrics to evaluate event clustering results, including Adjusted Rand In- dex (ART) (Hubert and Arabie, 1985), BCubed-F1 (Bagga and Baldwin, 1998), Normalized Mutual Information (NMI) and Accuracy. The detailed descriptions are in Appendix C.2.
our method achieves 8.23%, 8.79% and 8.10% im- provements of ARI score the three datasets, spectively. The significant performance gain over the baselines demonstrates that the HALTON is very effective for event clustering. We attribute it to that our method can learn discriminative repre- sentations via the neighborhood contrastive loss.
Results Table shows the event clustering results the three datasets, from which can observe that our method HALTON outperforms all the base- lines by a large margin, and achieves new state- of tf-th performance. For example, compared with the strong baseline TABS (Li et al., 2022),
4.3 Hierarchy Expansion Evaluations
Baselines Since the ETI methods cannot tackle the hierarchy expansion, we augment ETI base- lines with the greedy expansion (GE) algorithm,

---

Datasets
Methods
Rouge-I.
BERTSSore (Sr
Methods
ARI (%)
NMI (%)
Taxo_ (%)
40
HALTON
67.41
84.29
38.04
ACE:
TS_ Template
18.66
35.25
wo NC Loss
61.08
80,94
37.69
Trigger_See Sel
42.46
w/o DPM Loss
67.18
83,80
37.69
HOLToN (Ours)
24.09 (? 3.23)
46.24 (1 3.78)
w/o BCE
57.05
79.97
22.11
TABS
11.90
28.03
w/o CE Loss
63.30
82.47
37.56
T5_ Templpltee Trigger_See Sel
13,46
32.51
ERE
12.59
35.07
Table 4: Ablation study by removing main components on the ACE dataset.
HALTON (Ours)
16.20 (? 2,74)
39.32 (1 4.25)
TABS
16.02
36.24
TS. Template
24.94
38.20
MAVEN
4.4 Type Naming Evaluations
Trigger_Ser Sel
27.30
40.70
HALTON (Ours)
30.89 (1 3.59)
41.14 (1 0.44)
Baselines We compare our method with the TABS model that uses the abstraction mechanism to generate type names. In addition, we also de- velop two competitive baselines: (1) T5 TT_meellaee, which designs the template and uses T5 (Raffel et al., 2020) to fill it. (2) Trigger_ which ran- domly selects a trigger from clusters as the type name. Appendix E. describes more details.
Table 3: Type naming results on the ACE, ERE and MAVEN datasets, respectively.
namely X+GE, where is the ETI method. Be- sides, we also devise two representative baselines: (1) Srpi__imilarttt, which links new types based on the similarity between representations of new types and known type names. (2) LLMs Prompt, which devises prompts leverage LLMs for link- ing. We describe more details in Appendix D.I.
Evaluation Metries To our best knowledge, there no evaluation metries designed for event type name generation. We adopt two metrics: (1) Rouge-L (Lin, 2004), which measures the degree of matching between generated names and ground- truth names (i.e., hard matching). (2) BERTScore (Zhang et al., 2020), which computes the semantic similarity between generated name and the ground- truth (i.c., soft matching). The math formulas of Rouge-L and BERTScore in Appendix E.2.
Evaluation Metries To measure hierarchy expan- sion performance, we utilize the taxonomy metric (Dellschaft and Staab, 2006), which is originally proposed to evaluate taxonomy structure. For each cluster, the metric compares the predicted position and the golden position in the existing ontology. We report the taxonomy precision (Taxo_P), recall (Taxo_ R) and Fl-score (Taxo FI). More detailed descriptions about the metric are in Appendix D.2.
We present the type naming results Table From the results, we can observe that our method HALTON significantly outperforms all the baselines on the three datasets. For example, compared with the second best-performing model Trigger_Set our method achieves 3.23%, 2.74% and 3.59% improvements of Rouge-L score the three datasets, respectively. It indicates that our method can generate type names that are more sim- ilar to ground-trute names. The reason is that the proposed in-context learning-based naming tech- nique can better elicit the abstraction abilities in LLMs for type naming.
Results
Results The hierarchy expansion results are shown in Table 2, with the following observa- tions: (1) Our method HALTON has great ad- vantage over the baselines. For example, com- pared with the TABS+GE, 12.07% improvements of Taxo Fl with predicted clusters on the MAVEN dataset. Even given golden clusters (i.e., same clustering results), our method still outperforms the baselines. It indicates that the hierarchical information captured by the dy- namic based margin loss can provide guid- ance for hierarchy expansion. (2) Our method outperforms Hype__Smillritt, which proves that the greedy expansion algorithm is effective. Be- sides, our method improves more significantly on the MAVEN dataset. We guess that hierarchical information is more useful for hierarchy expansion in more complex scenarios.
method achieves
4.5 Ablation Study
To demonstrate the effectiveness of each compo- nent, we conduct ablation studies the ACE dataset, which shown in Table 4. We observe that the performance drops significantly if we re- move the neighborhood contrastive (NC) loss. It indicates the NC loss plays a key role in event clus- tering. Without the dynamic path- based margin

---

Event Instances and Type Names
Instancel: Ahmadi-Nejad, repurted to be hardliner, was appointed mayor and change Hamshahri's 's management has been considered inevitable. TABS: appointed T3 T5_Template: new mayor
Trigger Sel: appointed HALTON: appoint Golden type name: Nominate
(a) TABS
(b) HALTON (Ours)
Instance2: The meeting was Shalom's 's first encounter with counterpart since took office Is-
Figure 4: The visualization of features for event cluster- ing after t-SNE dimension reduction.
rael's s foreign minister on February 27.
TABS: new T5_ "e_ppateee: meeting
Trigger Sel: becoming HALTON: assume-position Golden type name: Start-Position
Score 0.019
Event Root
Table 5: Examples of generating names for new types.
Some: (L113
Soore: (1:33)
Sport:
Life
Justice
Personal
4.7
Case Study of Type Naming
Soore:0.300
Table 5 shows case studies, where our method and baselines generate event type names for the unla- beled instances. For the first example, the event trigger is similar to the golden type name. Our method and the baselines can produce type names that are semantically similar to golden names. For the second example, it is more challenging. All the baselines fail to generate correct type names. By contrast, our method successfully generates the type name that is almost identical to the ground truth. It demonstrates that the in-context learning- based naming module very effective.
Die
Injuure
Tral-- Heall-Heringg
Sentence
Figure 5: The process of expanding the existing event hierarchy with the new event type Sentence.
(DPM) loss, the performance is also degraded, sug- gesting the hierarchical information can provide guidance for hierarchy expansion. In addition, the cross-entropy (CE) and binary cross-entropy (BCE) losses are also useful, which is conducive train- ing the model by using labeled and unlabeled data,
4.6 Visualization
Related Work
Event Clustering To better understand our method, we visualize the features for event clus- tering using t-SNE (Van Der Maaten, 2014) the ERE dataset. The results are shown in Figure 4. Although TABS can learn separated features to some degree, it divides the instances with red colors into two clusters. By contrast, our method can generate more discriminative representations, which proves the effecttiveness of our method for event clustering.
Although event extraction has met with remark- able success (Ji and Grishman, 2008; Liu et al., 2018; Nguyen and Nguyen, 2019; Liu et al., 2020, 2022b; Cao et al., 2023), it usually requires that hand-crafted event schemas and annotations are given in advance. Since manually defining event schemas is labor-intensive and fails to generalize to new scenarios, some rescarchers have attempted to explore the ETI task (Chambers, 2013; Huang et al., 2016; et al., 2020, 202 la, 2022: Jin et al., 2022; Xu et al., 2023; Edwards and Ji, 2023). Typi- cal approaches utilize probabilistic generative mod- els (Chambers, 2013; Nguyen et al., 2015), ad- hoc clustering techniques (Chambers and Jurafsky, 2011) and neural networks (Huang and Ji, 2020; Shen al., 2021) to induce event clusters. Yuan et al. (2018) study the event profiling task and utilizes Bayesian generative model to obtain clus- ters. Shen et al. (2021) design an unsupervised method to generate salient event types by clus-
Hierarchy Expansion To intuitively show the process of hierarchy expansion, we visualize the workflow of linking the new type Sentence to the ex- isting event hierarchy via our method, as shown in Figure 5. As similarity between the new type and known types in top-down manner, and links the new event type to the correct position in the existing event ontology. addition, the greedy expansion strategy provides better interpretability for the expansion process.
see, method computes the

---

tering predicate-object pairs. Recently, Li et al. (2022) propose a co-training framework combine abstraction-baset and token-based representations for the task.
References
Amit Bagga and Breck Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In COLING 1998 Volume 1: The 17th Inter- national Conference on Computational Linguistics.
Despite these successful efforts, existing meth- ods cannot link new event types to the existing on- tology, and lack the ability to generate meaningful names for new event types.
Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In COLING 1998 Volume 1: The 17th International Conference on Computational Linguistics.
6 Conclusion
Christopher DB Burt, Simon Kemp, and Martin Con- way. 2003. Themes, events, and episodes in autobi- ographical memory. Memory & Cognition, 31:317- 325.
In this paper, we define a new event ontology com- pletion task, aiming at simultaneously achieving event clustering, hierarchy expansion and type nam- ing. Furthermore, we propose hierarchical struc- ture evolution network (HALTON), which achieves the goals via collaboration between neighborhood contrastive clustering, awwaee
Pengfei Cao, Yubo Chen, Jun Zhao, and Taifeng Wang. 2020. Incremental event detection knowledge consolidation networks. In Proceedings of the 2020 Conference Empirical Methods in Natural Lan- guage Processing, pages 707-717, Association for Computational Linguistics.
aware linking and in-context learning-base based naming. Experimental re- sults on three datasets show that our method brings significant improvements over baselines.
Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao. 2023. Zero-shot cross-lingual event argu- ment extraction with language-oriente prefix-tuning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12589-12599)
Limitations
In this paper, the size of used datasets is relatively small and the datasets are most in the newswire genre. To facilitate further research on this task, constructing a large-scale and high-quality dataset is an important research problem. In addition, sim- ilar to the event type induction, the proposed event ontology completion task also requires labeled in- stances for training models and constructing the existing event ontology. The ultimate goal of the event ontology completion task is to automatically construct the event ontology structure from scratch. We plan to address the event ontology completion task in the unsupervised scenario.
Nathanael Chambers. 2013. Event schema induction with prohabilistic entity-driven model. In Proceed- ings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807. Association for Computational Linguisties.
Nathanael Chambers and Dan Jurafsky, 2011. Template- based information extraction without the templates. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Longuage Technologies, pages 976-986. Association for Computational Linguistics.
Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi- pooling convolutional neural networks. In Proceed- ings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Interna- tional Joint Conference Natural Language Pro- cessing (Volume Long Papers), pages 167-176. Association for Computational Linguistics.
Acknowledgments
We thank anonymous reviewers for their insight- ful comments and suggestions. This work is sup- ported by the National Key Research and Develop- ment Program of China (N..2020AAA010GA)) the National Natural Science Foundation of China (No.62176257, No.6197621 I), the Strategic Pri- ority Research Program of Chinese Academy of Sciences (Grant No. No.XDA27020100 ), the Youth In- novation Promotion Association CAS, and Yunnan Provincial Major Science and Technology Special Plan Projects (N.2202202AD080000))
Klaas Dellschaft and Steffen Staab. 2006. On how perform gold standard based evaluation of ontol- ogy learning. In Proceedings of the International Semantic Web Conference, pages 228-241 Springer.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I, pages 4171-4186. Association for Computational Linguistics.

---

George Doddington, Alexis Mitchell, Mark Przybocki. Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extrac- tion (ACE) program - tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation. European Language Resources Association.
Manling Li, Sha Li, Zhenhailong Wang. Lifu Huang. Kyunghyun Cho, Heng Ji, Jiawei Han, and Clare Voss. 2021a. The future is not one-dimensional: Complex event schema induction by graph model- ing for event prediction. In Proceedings of the 2021 Conference Empirical Methods in Natural Lan- guage Processing, pages 5203-5215. Association for Computational Linguistics.
Xinya Du and Claire Cardie. 2020. Event extraction by answering (almost) natural questions. In Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 671-683. As- sociation for Computational Linguistics.
Manling Li, Qi Zeng. Ying Lin, Kyunghyun Cho, Heng Ji, Jonathan May, Nathanael Chambers. and Clare Voss. 2020. Connecting the dots: Event graph schema induction with path language modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 684- 695. Association for Computational Linguistics.
Carl Edwards and Heng Ji. 2023. Supi-supe new event type induction and description via contrastive loss-enforced batch attention. In Proceedings of the 17th Conference of the European Chapter of the As- sociation for Computational Linguistics, pages 3805- 3827. Association for Computational Linguistics.
Sha Li, Heng Ji, and Jiawei Han. 2021b. Document- level event argument extraction by conditional gener- ation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, pages Linguistics.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020. Momentum contrast for un- supervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 9726-9735. IEFE
Association for Computational
Sha Li, Heng Ji, and Jiawei Han. 2022. Open rela- tion and event type discovery with type abstraction. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, pages 6864-6877. Association for Computational Linguis- tics.
Lifu Huang, Taylor Cassidy, Xiaocheng Feng, Heng Ji, Clare R. Voss. Jiawei Han, and Avirup Sil. 2016. Liberal event extraction and event schema induction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pages 258-268. Association for Computational Linguistics.
Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, and Wenhu Chen. 2023, Few-shot in-context learning on knowledge base question answering. In Proceed- ings of the st Annual Meeting of the Association for Computational Linguistics (Volune 1: Long Papers). pages 6966-6966 6980. Association for Computational Linguisties.
Lifu Huang and Heng 2020. Sepi-super new event type induction and event detection. In Proceed- ings of the 2020 Conference Empirical Methods Natural Language Processing, pages 718-724, As- sociation for Computational Linguistics.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81. Association for Compu- tational Linguistics.
Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of classification, pages 193-218.
Heng Ji and Ralph Grishman. 2008 Refining event extraction through cross-document inference. In Pro- ceedings of ACI-08: HI.. pages 254-262. Associa- tion for Computational Linguistics.
Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xiao- jiang Liu. 2020. Event extraction as machine reading comprehension. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 1641-1651. Association for Computational Linguistics.
Xiaomeng Jin, Manling Li. and Heng Ji. 2022. Event schema induction with double graph autoencoders. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 2013-2025, Association for Computational Linguistics.
Minqian Liu, Shiyu Chang, and Lifu Huang. 2022a. In- cremental prompting: Episodic memory prompt for lifelong event detection. In Proceedings of the 29th International Conference on Computational Linguis- rics, pages 2157-2165. International Committee on Computational Linguistics.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69800
Xiao Liu, Heyan Huang, Ge Shi. and Bo Wang. 20226. Dynamic prefix- tuning for generative template- event extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 5216-5228. Association for Computational Linguistics.
Paul Kingsbury and Martha Palmer. 2003. Propbank: the next level treebank. In Proceedings of Tree- banks and lexical Theories, volume 3.

---

Xiao Liu, Zhunchen Luo, and Heyan Huang. 2018. Jointly multiple events extraction via attention-basee graph information aggregation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1247-1256. Association for Computational Linguistics.
Zhiyi Song, Ann Bies, Stephanie Strassel, Tom Riese, Justin Mott. Joe Ellis, Jonathan Wright, Seth Kulick, Neville Ryant, and Xiaoyi Ma. 2015. From light to rich ERE: Annotation of entities, relations, and events. In Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, pages 89-98. Association for Com- putational Linguistics.
Zichen Liu, Hongyuan Xu, Yanlong Wen, Ning Jiang, HiiY Wu, and Xiaojie Yuan. 2021. TEMP: Tax- onomy expansion with dynamic margin loss through faxonomy-pathe In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing, pages 3854-3863, Association for Com- putational Linguistics.
Tian-X Miann Sun, Xiang Yang Liu, Xi-Peng Qiu, and Xuan-Jing Huang. 2022. Paradigm shift in natural language processing, Machine Intelligence Research, 19())169-113:)
Joshua B Tenenbaum, Charles Kemp. Thomas Grif- fiths, and Noah D Goodman. 2011. How to grow a mind: Statisties, structure, and abstraction. science, (51(6022)1279-1111
Yubo Ma, Zehao Wang. Mukai Li, Yixin Cao, Meiqi Chen, Xinze Li, Wenqi Sun, Kunquan Deng, Kun Wang, Aixin Sun, and Jing Shao. 2022. MMEKG: Multi-modal event knowledge graph towards univer- sal representation across modalities. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 231-239. Association for Computational Lin- guistics.
Laurens Van Der Maaten. 2014. Accelerating t-sne using based algorithms. The Journal of Machine Learning Research, 15(1):221133255
Sijia Wang. Mo Yu, Shiyu Chang, Lichao Sun, and Lifu Huang. 2022. Query and extract: Refining event extraction as type-oriented binary decoding. In Find- ings of the Association for Computaaional Linguistics: ACL 2022, pages 169-182. Association for Computa- tional Linguistics.
Kiem-Hieu Nguyen, Xavier Tannier, Olivier Ferret, and Romaric Besançon. 2015. Generative event schema induction with entity disambiguation. In Proceed- ings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Interna- tional Joint Conference on Natural Language Pro- cessing (Volume 1: Long Papers), pages 188 1877 Association for Computational Linguistics.
Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, and Jie Zhou. 2020. MAVEN: A Massive General Domain Event Detection Datasct. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing, pages 1652-1671, As- sociation for Computational Linguistics.
Trung Minh Nguyen and Thien Huu Nguyen. 2019. One for all: Neural joint modeling of entities and events. In The Thirty-Third Third AAAI Conference on Artificial Intelligence, AAAI 2019, pages 6851-6858 AAAI Press.
Nan Xu, Hongming Zhang, and Jianshu Chen. 2023, Ceo: Corpus-based open-domain event ontology in- duction. arXiv preprint urXiv:2305.1352
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn, Res., 21:140:1-140:61
Pengfei Yu, Heng Ji, and Prem Natarajan. 2021. Life- long event detection with knowledge transfer. In Pro- ceedings of the 2021 Conference Empirical Meth- ods in Natural I Lnguage Processing, pages 5278- 5290. Association for Computational Linguistics.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 815- 823. IEEE Computer Society.
Quan Yuan, Xiang Ren, Wenqi He, Chao Zhang, Xinhe Geng, Lifu Huang, Heng Ji, Chin- Yew Lin, and Ji- awei Han. 2018 Open-schema event protiling for massive news corpora. In Proceedings of the 27th ACM International Conference Information and Knowledge Management, CIKM 2018, Torino, Italy, October 22-26, 2018, pages 587-596. ACM.
Satoshi Sekine. 2006. On-demand information extrac- tion. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pages 131-738. Associ- ation for Computational I. Linguistics.
Kai Zhang, Yuan Yao, Ruobing Xie, Xu Han, Zhiyuan Liu, Fen Lin, Leyu Lin, and Maosong Sun. 2021. Open hierarchical relation extraction. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages $682-5693. Association for Computational Linguistics.
Jiaming Shen, Yunyi Zhang, Heng Ji, and Jiawei Han. 2021. Corpus-based open-domane event type induc- tion. In Proceedings of the 2021 Conference on Em- pirical Methods in Natural Language Processing, pages 5427-5440. Association for Computational Linguistics.

---

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evalu- ating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020.
A Training and Inference Procedure
Algorithm 1 The HALTON Method
Require:
Labeled dataset D { v/) and un- labeled dataset n {a() for training, an- other unlabeled instances D {a¹) } for in-
Jun Zhao, Tao Gui, Qi Zhang, and Yaqian Zhou. 2021. A relation-oriented clustering method for open re- lation extraction. In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Lunguage Processing, pages 9707-9718 Association for Com- putational Linguistics.
ference, existing event ontology T, model pa-
rameters O learning rate
Ensure:
Optimized model parameters, and com- pleted event hierarchy.
Yang Zhao, Jiajun Zhang, and Chengqing Zong. 2023, Transformer: A general framework from machine translation to others. Machine Intelligence Research, pages 1-25.
for epoch + 1 to L do
Compute the C, on DD and D Optimize model parameters via gradient de- scent = V
Zhun Zhong, Enrico Fini, Subhankar Roy. Zhiming Luo, Elisa Ricci, and Nicu Sebe. 2021. Neighborhood contrastive learning for novel class discovery. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 10867-10875.
end for
5: Cluster unlabeled data D via trained model;
6: Link cach cluster to T using the greedy expan- sion algorithm;
Generate type names via in-context learning- based naming module.
Xinyu Zuo, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Weihua Peng, Yuguang Chen. 2021. Improving event causality identification via self- supervised representation learning external causal statement. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pages 2162-2172.
Known and Unknown Types
B.1 ACE Dataset
The known event types include:
Trial-Hearing, Die,
Transfer-Moncy, Injure, End-Position, Elect, Meet,
Phone-Write, Transport, and Attack.
The unknown event types include: Merge-Org. Start-Org. Baclare-Bankruptee, End-Org, Pardon, Extradite, Execute, Fine, Sentence, Appeal, Con- vict, Sue, Release-Parole, Arrest-Jail, Charge- Indict, Acquit, Demonstrate, Start-Position Nom- inate, Transfer-Ownertee Marry, Divorce, and Be-Born.
B.2 ERE Dataset
The known event types include: Attack, Transport- Person, Transfer-Money. Contact, Die, Broadcast, Transfer-Ownershhp.. Meet, End-Position, and Cor- respondence.
The unknown event types include: Arrest- Jail, Start-Position, Trial-Hearing, Elect, Charge- Indict, Artifact, Transaction, Demonstrate, Sen- tence, Marry, Convict, Transport-Artiffcc, Be-Born, Release-Parole, Injure, Sue, Pardon, Nominate, Ex- ecute, Start-Org, End-Org, Divorce, Acquit, Ex- tradite, Merge-Org, Appeal, Fine, and Declare- Bankruptcy.

---

B.3 MAVEN Dataset
where Ne is the total number of instances. E(RI) is the expectation of the RI.
The known event types include: Causation, Pro- ress_stare Attack, Hostile_encountee, Motion, Competition, Killing, Process end, Docial_event, Conquering, Statement, Self_motion: Arriving, Destroying, Coming to to_be, Bod- ily harm, Death, Creating, and -ipettry_operaaion.. The unknown event types include: Damaging,
encounter, Catastrophe,
NMI is the normalized mutual information score,
which is calculated follows:
2 x MI(Y;C) NMI = H(Y) + H(C))
where and denote the ground truth and pre- dicted clusters, respectively. H(-) is the entropy function. MI(Y; C)) denotes the mutual informa- tion between Y and C.
Cause cause_change__e of strength, Cause change__cceteeeeee of oo__pootioon_te an__o________oooo a scale, Hold, Control, Earnings_ant ing. Arranging, Know, Preventing_oo or le__ttnnngg Presence, Escaping, Defending, Action, Mo- tion directiicii,,, Cause to be je_iincldddd, Change, Traveling, Placing. Participation, Influence, Change_of Je_aee_s_p.a ludementt_communee communication, Expressing_publicky, Name coneeconf,rral, Request, Giving, Supporting, Recording, Removing, ggree_or_refuse___ _ttacct Using, Supply, Commu- nication, Reporting, Choosing, Sending, Bringing, and Departing.
Getting, Becom-
BCubed (Bagga and Baldwin, 1998) averages the precision and recall of each instance. The B-Cubed precision is defined as follows:
Ne
((c)) [C(c)]
BCubed-P
N
where (-) is the mapping function from an in- stance to its ground truth cluster. Similarly, we can compute the B-Cubed recall. The B- B-Cubed Fl is calculated by their harmonic average.
C Baselines and Evaluation Metries for Event Clustering
Accuracy estimates the quality of clustering by finding permutation from predicted cluster la- bels to the ground-truth that gives the highest accuracy:
C.1 Baselines
SS-VQ-VAE (Huang and Ji, 2020) first the BERT to encode the event trigger, and then pre- dicts the type by looking up a dictionary of dis- crete latent representations. also utilizes a vari- ational autoencoder to avoid overfitting problem.
N
Accuracy =
max oEPerm(k)
[
=(ys)),
N
ETYPECLUS (Shen et al., 2021) first selects salient predicates and object to represent events. Then, it leverages dictionary to disambiguate predicate senses. Finally, it embeds and clusters the events in latent spherical space.
where k is the number of clusters. Perm(k) denote all permutation functions.
D Baselines and Evaluation Metrics for Hierarchy Expansion
TABS (Li al., 2022) proposes abstraction- based representation, which is complementary to the boken-based representation of events. devises a prompt to elicit semantic knowledge in pre-trained language models for clustering.
D.1 Baselines
Type Similarity first computes the prototype for the new type by averaging all instance represen- tations belonging to the type. Then, it uses the BERT (Devlin et al., 2019) to encode known type names. Finally, it links the new type to the cxist- ing ontology based on the similarity between the prototype and known type representations.
C.2 Evaluation Metrics
ARI (Hubert and Arabie, 1985) measures the similarity between two cluster assignments. The number pairs the same (different) clusters is denoted as TP (TN). The ART computed as follows:
LLMs Prompt first devises a prompt, and then utilizes the LLMs (i.e., ChatGPT) to fill it. The prompt is defined follows:
RI E(RI)
TP + TN
The existing event ontology consists of these event types, including T1, T2, TN. Please link the
ARI
RI
max RI E(Ri)
N

---

new event type the correct position of the event ontology. The answer should be one of these existing event type names. The following is an example:
Datasets
Methods
Rouge-T.
BERTScore
SS- AS-VO-VAAHHAE VAE+ICLN
9.62
34.27
ETYPECLUS+ICLN
14.31
31.39
ACE
TABS+ICLN
16.78
33.50
TABS
17.49
29.40
Trigger: triggerl, Sentence: sl
HALTON (Ours)
24.09
46.24
Answer: one known type
SS- SS-VQ-VAE+CLNN
10.86
26.81
Trigger: trigger2, Sentence: s2, Answer:
ETYPECLUS+ICLN
7.19
27.32
ERE
TABS+ICLN
12.69
31.12
D.2
Evaluation Metries
TABS
11.90
28.03
Taxonomy metric (Dellschaft and Staab, 2006) compares the predicted position of the clusters and the golden position in the hierarchy. The taxonomy precision (Taxo P) is formulated as follows:
HALTON (Ours)
16.20
39.32
SS-VQ-VAF+ICIN
15.86
32.67
ETYPECLUS+ICIN
13.31
28.85
MAVEN
TABS+ICLN
24.28
36.41
TABS
16.02
36.24
HALTON (Ours)
30.89
41.14
1 |C|
lu(t,) n u(tt)) [u(tp)]
Taxo
Table 6: Type naming results of augmented baselines the ACE, ERE and MAVEN datasets, respectively. "ICLN" denotes the in-context learning-based naming module.
IEC
where C' are predicted clusters. l, and lg denote the predicted and golden positions of the event type f, respectively. u((p) is the union of all the ancestors and itself of the predicted position 4p- We can compute the recall (Taxo_R) in similar way.
where is the golden type name, and Y denotes the generated name. m and n denote the length of X and Y. respectively, LCS(X, Y) is the longest common subsequence between X and Y. 3 is a hyper-parameter.
E Baselines and Evaluation Metrics for Type Naming
E.1 Baselines
BERTScore (Zhang et al., 2020) computes the semantic similarity between generated names and ground-truth labels by using BERT to obtain con- textual representations. The precision is formu- lated as follows:
TS_ 15_Template devises a template and utilizes T5 (Raffel et al., 2020) to fill it. The template is defined as follows:
(Context). According to this, the trigger word of this [MASK] (Trigger).
Σ max ai yo.
P
Y BEY 2,xX
In the template, (Contexi) represents the text that deseribes the event. (Trigger) is a placeholder that is replaced by the actual trigger in the pro- totype instance. [MASK] is expected to be filled with the type name.
where X and Y denote the the ground-truth la- bel and generated type names, respectively. V: is the embedding of i-th token in Y. After sym- metrically calculating the recall, we can get the BERTScore based on their harmonic average.
Trigger_See Sel randomly selects an event trigger from clusters as the new type name.
For the evaluation metrics, the generated type names and golden type names are both com- posed of node names from the root to the leaf the ontology tree.
E.2 Evaluation Metrics
Rouge-L (Lin, 2004) measures the degree of matching based on the longest common subse- quence between generated names and golden type names, which can be computed as follows:
F Augment Baselines with In-Context Learning- Naming
Pie. LCS(X.Y)
We augment the event clustering baselines with the proposed in-context learning-basee naming mod- ule. The results are shown Table From the table, we can observe that the three baselines with
Rica LCS(X,Y)
m
(Pec Pies FiL. Rice + Pre

---

(a) ETYPECLUS
(b) HALTON (Ours)
Figure 6: The feature visualization of ETYPECLUS and our method for event clustering after t-SNE dimension reduction.
the type naming technique can achieve better or comparable performance than the original TABS. It indicates that the proposed in-context learning- based naming module is very effective.
G Visualization of Event Clustering
In section 4.6, we show the feature visualization of TABS and our method for event clustering. In this section, present the visualization result of the ETYPECLUS, which is shown in Figure 6. From the result, can see that the ETYPECLUS fails distinguish the unlabeled instances. By con- trast, our method can learn discriminative features, which proves the effectiveness of our method.
Implementation Details
In our implementations, method uses the Hug- gingFace's Transformers library" implement the the uncased BERT base and T5 base models. The learning rate is initialized as le-4 with a linear decay. We utilize the Adam algorithm (Kingma and Ba, 2014) to optimize model parameters. The batch size is set to 128. The hyper-paramete for the hinge loss in BCE loss is set to 2. The number of neighbors K is set to 3. The number of training epochs is 100. Each experiment is conducted on NVIDIA RTX A6000 GPUs.


Reviews:
[{"review_id": "TijSzzQpzm", "reviewer": "Reviewer_GXuj", "summary": "", "questions": "* A. [§3.1] is the assumption to know in advance M_u, i.e., the number of unknown event types, a practical limitation of the approach? While I acknowledge this is consistent, e.g., with Li et al 2022, as well as the way datasets are derived (there, M_u is known), estimating M_u in a practical context seems to me difficult and the paper may benefit of some discussion about how to deal with that, and/or of additional experimental setting(s) showing the impact on sub-tasks performance of using a 'wrong' estimate of M_u.\n\n* B. [§3.1] when feeding the representations h_j^u of unlabeled instances \"into a classifier to obtain predicted distributions\" (later assessed via loss L_bce), what classes is this distribution over? Y^l types? M_u clusters? And what classifier is used here? is it the same of the one for labeled instances (the one assessed via loss L_ce)?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your kind feedback and valuable reviews. Below we would like to give detailed responses to each of your comments.\n\n***Q1:*** *Reproducibility could be improved (the paper promises to release the code). Lacking availability of an implementation or extensive supplemental material, there are details of the approach that remain unclear in the text of the paper (e.g., see question B).*\n\n**A1:** Thanks for your comments. We have illustrated the overall training and inference procedure in Section 3, and introduced the implementation details in Appendix H. Besides, we will release the source code to improve reproducibility and facilitate further research.\n\n---\n***Q2:*** *Incremental nature of proposed task and method. Clustering of un-typed event mentions is already considered in ETI, and some approaches addressing ETI also address naming of new types (e.g., Li et al, 2022, although this sub-task is not thoroughly as in this submission).*\n\n**A2:** Although our work is related to ETI, it still makes significant contributions to the NLP community: (1) From the task perspective, unlike the ETI task, this newly proposed EOC task not only requires the model to achieve event clustering, but also to achieve hierarchy expansion and type naming. Besides, we also introduce corresponding baselines and evaluation metrics for the three task settings, respectively. (2) From the method perspective, although the method proposed by Li et al.2022 can tackle the event clustering and type naming to some degree, our method can achieve better performance, due to the novel neighborhood contrastive loss and in-context learning-based naming technique. Moreover, our method devises a dynamic path-based margin loss to integrate hierarchy information into event representations, and utilizes the greedy expansion strategy to address the expansion of event hierarchy. The proposed method can serve as a strong baseline for the research on the new task.\n\n---\n\n***Q3:*** *[§3.1] is the assumption to know in advance M_u, i.e., the number of unknown event types, a practical limitation of the approach? While I acknowledge this is consistent, e.g., with Li et al 2022, as well as the way datasets are derived (there, M_u is known), estimating M_u in a practical context seems to me difficult and the paper may benefit of some discussion about how to deal with that, and/or of additional experimental setting(s) showing the impact on sub-tasks performance of using a 'wrong' estimate of M_u.*\n\n**A3:** Thanks for your valuable suggestions. To maintain consistency with previous work, we also assume that the value of M_u is known. We research some papers about clustering, and find that many methods have been proposed to estimate the number of clusters [1][2][3]. We think your suggestion is well, and we will add the experiments about the estimation of M_u in the revised version.\n\n---\n\n***Q4:*** *[§3.1] when feeding the representations h_j^u of unlabeled instances \"into a classifier to obtain predicted distributions\" (later assessed via loss L_bce), what classes is this distribution over? Y^l types? M_u clusters? And what classifier is used here? is it the same of the one for labeled instances (the one assessed via loss L_ce)?*\n\n**A4:** The predicted distributions of unlabeled instances are over M_u clusters. The classifier is the softmax function. As shown in Figure 2, the classifiers for labeled and unlabeled instances are different. We will add the detailed descriptions to make our paper clearer.\n\n---\n\n***Q5:*** *[Eq. 2] I assume representation h_i^l goes through a classification layer before applying softmax.*\n\n**A5:** We apologize that we don’t illustrate it clearly. We omit the dimension transformation matrix. Thanks for pointing out the issue, and we will add the descriptions in the revised version.\n\n---\n\n***Q6:*** *[§3.1] \"If x+i^u and x_j^u belong to different classes\" -> could it be \"clusters\", instead of \"classes\"?*\n\n**A6:** We think your suggestion is well, and we will revise it in the revised version.\n\n---\n\n***Q7:*** *[Fig 4] On which dataset is the t-SNE visualization being generated? ACE?*\n\n**A7:** We apologize that we don’t illustrate it clearly. The t-SNE visualization is conducted on the ERE dataset. We will also add the visualization results on the ACE and MAVEN datasets in the revised version.\n\n---\n\n***Q8:*** *[§B] (minor) Suggest using a more compact tabular representation for known/unknown event types in the three datasets.*\n\n**A8:** We think your suggestion is well, and we will use a more compact tabular representation for known/unknown event types in the revised version.\n\n---\n\n***Q9:*** *[§D.1] \"type1\" -> \"types\".*\n\n**A9:** Thanks for pointing out the typo. We will correct it in the revised version.\n\n---\n\n### Reference\n[1] Khan, Imran, et al. \"Variable weighting in fuzzy k-means clustering to determine the number of clusters.\" IEEE Transactions on Knowledge and Data Engineering 32.9 (2019): 1838-1853.\n\n[2] Tapaswi, Makarand, Marc T. Law, and Sanja Fidler. \"Video face clustering with unknown number of clusters.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.\n\n[3] He, Zhaoshui, et al. \"Detecting the number of clusters in n-way probabilistic clustering.\" IEEE Transactions on Pattern Analysis and Machine Intelligence 32.11 (2010): 2006-2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* S1. By explicitly accounting for type naming and hierarchy expansion, the proposed EOC task appears more close to the needs of practical applications than the ETI task it derives from.\n* S2. Substantial performance improvements of HALTON over considered baselines / applicable state-of-the-art systems.\n* S3. Solid experimental evaluation providing insights both on HALTON design aspects and the considered EOC sub-tasks themselves.\n* S4. Overall well written paper.", "reasons_to_reject": "* W1. Reproducibility could be improved (the paper promises to release the code). Lacking availability of an implementation or extensive supplemental material, there are details of the approach that remain unclear in the text of the paper (e.g., see question B).\n* W2. Incremental nature of proposed task and method. Clustering of un-typed event mentions is already considered in ETI, and some approaches addressing ETI also address naming of new types (e.g., Li et al, 2022, although this sub-task is not thoroughly addressed as in this submission).", "questions_to_authors": "* A. [§3.1] is the assumption to know in advance M_u, i.e., the number of unknown event types, a practical limitation of the approach? While I acknowledge this is consistent, e.g., with Li et al 2022, as well as the way datasets are derived (there, M_u is known), estimating M_u in a practical context seems to me difficult and the paper may benefit of some discussion about how to deal with that, and/or of additional experimental setting(s) showing the impact on sub-tasks performance of using a 'wrong' estimate of M_u.\n\n* B. [§3.1] when feeding the representations h_j^u of unlabeled instances \"into a classifier to obtain predicted distributions\" (later assessed via loss L_bce), what classes is this distribution over? Y^l types? M_u clusters? And what classifier is used here? is it the same of the one for labeled instances (the one assessed via loss L_ce)?", "paper_topic_and_main_contributions": "SUMMARY: The paper extends the task of Event Type Induction (ETI), which aims at assigning untyped event mentions to clusters denoting new event types, into a newly proposed task of Event Ontology Completion (EOC), which also includes naming new types and inserting them into an existing (incomplete) event hierarchy/ontology. After formulating the EOC task, authors propose a new neural method \"HierarchicAL STructure EvOlution Network (HALTON)\" to perform the three clustering/naming/hierarchy expansion sub-tasks, by (i) combining different loss functions to learn effective event mention representations for clustering, (ii) leveraging a LLM (T5) for naming, and (iii) introducing a similarity-based greedy approach for hierarchy expansion. HALTON is evaluated it on three EOC datasets derived from ACE, ERE and MAVEN, demonstrating superiority over reasonable baselines and existing methods (SS-VQ-VAE, ETYPECLUS, TABS) that can be either reused as is or adapted to these sub-tasks. The paper includes an appendix but no code or other supplemental material.\n\nCONTRIBUTIONS:\n* C1. Formalization of EOC task and its sub-tasks, as incremental contribution extending the ETI task.\n* C2. Provision of EOC benchmarks by defining a methodology for deriving EOC datasets out of existing event detection datasets (e.g., ACE) and by proposing the metrics to evaluate EOC sub-tasks.\n* C3. Design of the HALTON model for the EOC task.\n* C4. Experiments to assess the performance of HALTON and other baselines/systems on three derived EOC datasets, also providing insights on the difficulty of individual EOC sub-tasks.\n\nPOST-REBUTTAL UPDATE. I thank the authors for the clarifications in their rebuttal. After reading it and considering the other reviews as well, I increased the excitement score from 3 to 4 to better reflect my leaning positively towards this submission.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "* T1.  [Eq. 2] I assume representation h_i^l goes through a classification layer before applying softmax\n* T2.  [§3.1] \"If x+i^u and x_j^u belong to different classes\" -> could it be \"clusters\", instead of \"classes\"?\n* T3.  [Fig 4] On which dataset is the t-SNE visualization being generated? ACE?\n* T4.  [§B] (minor) Suggest using a more compact tabular representation for known/unknown event types in the three datasets\n* T5.  [§D.1] \"type1\" -> \"types\""}}, {"review_id": "Qj2IJqUhSj", "reviewer": "Reviewer_ktxz", "summary": "", "questions": "- Why the authors are calling it ontology completion, should not it come under hierarchy completion or taxonomy completion?\n- How are authors extracting the events from the text?\n- Is equation 1 needed?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation of our work and careful reviews. Below we would like to give detailed responses to each of your questions.\n\n***Q1:*** *Why the authors are calling it ontology completion, should not it come under hierarchy completion or taxonomy completion?*\n\n**A1:** Since event ontology denotes the hierarchical organization structure of known event types and we aim to link new types to existing event hierarchy, the proposed task is named event ontology completion. The event ontology is a hierarchical structure, thus we think that the task named hierarchy completion or taxonomy completion is also reasonable.\n\n----\n\n***Q2:*** *How are authors extracting the events from the text?*\n\n**A2:** We apologize that we don’t illustrate it clearly. Many studies indicate that using fine-tuned pre-trained language models to extract triggers can achieve very competitive performance [1][2]. Trigger extraction is not the focus of this paper. Therefore, following previous work [3], we use the golden trigger for event clustering. Actually, our method can be combined with any trigger extraction model. That is to say, we can first use a trigger extraction model to obtain event triggers, and then utilize our proposed method to accomplish the EOC task. We leave this idea as a future work to explore.\n\n---\n\n***Q3:*** *Is equation 1 needed?*\n\n**A3:** Equation 1 explains how to obtain an event representation. To make our paper clearer, we add the equation 1 in the paper.\n\n---\n### Reference\n[1] Wang, Xiaozhi, et al. \"Adversarial training for weakly supervised event detection.\" Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019.\n\n[2] Veyseh, Amir Pouran Ben, et al. \"Unleash GPT-2 power for event detection.\" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021.\n\n[3] Li, Sha, Heng Ji, and Jiawei Han. \"Open Relation and Event Type Discovery with Type Abstraction.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper targets an interesting problem.\n- The proposed solution is very interesting and novel.\n- The paper is written very clearly and the approach is thoroughly experimented. \n", "reasons_to_reject": "- I see no reason to reject.", "questions_to_authors": "- Why the authors are calling it ontology completion, should not it come under hierarchy completion or taxonomy completion?\n- How are authors extracting the events from the text?\n- Is equation 1 needed?", "paper_topic_and_main_contributions": "The paper targets the problem of event ontology completion with the help of clustering, type naming, and hierarchy extension. The clustering is based on the neighborhood contrastive clustering, in context learning based naming and hierarchy aware linking.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "- The authors have covered the important references.", "Typos_Grammar_Style_and_Presentation_Improvements": "The paper is clearly written."}, "extra_text": {}}, {"review_id": "8NuI0dKNIh", "reviewer": "Reviewer_D6mY", "summary": "", "questions": "Q1: How were the triggers selected in advance?\nQ2: How can this be visualized at scale? (MAVEN has 118k examples, therefore doing error analysis at that scale is impossible)", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your kind feedback and valuable reviews. Below we would like to give detailed responses to each of your comments.\n\n***Q1:*** *They do not follow the classic structure of scientific papers - adding Related Work only before conclusion makes the whole paper difficult to read for casual readers (I am familiar with the domain so it was not an issue for me).*\n\n**A1:** Thanks for pointing out this issue. We will adjust the placement of Related Work section in the revised version.\n\n---\n\n***Q2:*** *I would have preferred a longer conversation in plain English about trigger selection (such details should not be hidden by equations, but rather clearly explained).*\n\n**A2:** We apologize that we don’t illustrate it clearly. Many studies indicate that using fine-tuned pre-trained language models to extract triggers can achieve very competitive performance [1][2]. Trigger extraction is not the focus of this paper. Therefore, following previous work [3], we use the golden trigger for event clustering. Actually, our method can be combined with any trigger extraction model. That is to say, we can first use a trigger extraction model to obtain event triggers, and then utilize our proposed method to accomplish the EOC task. We leave this idea as a future work to explore.\n\n---\n\n***Q3:*** *How were the triggers selected in advance?*\n\n**A3:** Please refer to the response A2.\n\n---\n\n***Q4:*** *How can this be visualized at scale? (MAVEN has 118k examples, therefore doing error analysis at that scale is impossible).*\n\n**A4:** Thank you for raising the question about visualizing the dataset at scale, particularly given the substantial size of the MAVEN dataset. We understand that attempting to visualize every instance might not be feasible. Instead, we employ a sampling approach to select a representative subset of instances for detailed error analysis. This can also provide valuable insights into the overall performance. We will add the detailed descriptions in the revised version to make our paper clearer.\n\n---\n\n### Reference\n[1] Wang, Xiaozhi, et al. \"Adversarial training for weakly supervised event detection.\" Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019.\n\n[2] Veyseh, Amir Pouran Ben, et al. \"Unleash GPT-2 power for event detection.\" Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021.\n\n[3] Li, Sha, Heng Ji, and Jiawei Han. \"Open Relation and Event Type Discovery with Type Abstraction.\" Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- good overall experimental design\n- great presentation\n- good results\n- innovative approach\n- code and results will be shared at a later stage", "reasons_to_reject": "- they do not follow the classic structure of scientific papers - adding Related Work only before conclusion makes the whole paper difficult to read for casual readers (I am familiar with the domain so it was not an issue for me)\n- I would have preferred a longer conversation in plain English about trigger selection (such details should not be hidden by equations, but rather clearly explained)", "questions_to_authors": "Q1: How were the triggers selected in advance?\nQ2: How can this be visualized at scale? (MAVEN has 118k examples, therefore doing error analysis at that scale is impossible)", "paper_topic_and_main_contributions": "The paper provides an original method to create event types that can be applied to downstream applications. Previous studies focused only on clustering, and therefore the results were difficult to apply in other contexts.\n\nThe HALTON network achieves good results via 3 tasks: i) neighborhood contrastive clustering; ii) hierarchy-aware linking and iii) in-context learning (aka prompt engineering). Showcased experiments display an improvement of over 8% compared to traditional methods.\n\nGenerally the text is well-written and well illustrated. Figure 2 showcased the architecture really well. The only major criticism is that the Related Work section is placed before conclusion. This should not happen as new work builds on previous work and doesn't really appear out of nowhere. Also placing Related Work as the second section serves as extending the introduction a little bit and helping the casual reader get comfortable with the domain.\n\nThe formalization and methodology section work really well and build the mathematical foundation needed to understand the contribution. The experimental section is detailed and provides not only the scores, but also the improvements compared to the baselines which makes it rather easy to follow. The experiments are also doubled by an ablation study which shows that removing the contrastive clustering leads to performance drops. A visualization is also included, therefore readers are provided various mechanisms through which to understand the end results.\n\nI have read the rebuttal and I am generally okay with their changes.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "I have not found missing references.", "Typos_Grammar_Style_and_Presentation_Improvements": "Very few typos and presentation issues. The only major issue is the placement of the Related Work section."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper targets the problem of event ontology completion with the help of clustering, type naming, and hierarchy extension, that can be applied to downstream applications.

The proposed EOC task appears closer to the needs of practical applications than the ETI task it derives from. Solid experimental evaluation providing insights both on HALTON design aspects and the considered EOC sub-tasks themselves.

However, some concerns might regard the incremental nature of clustering technique, which is yet considered in ETI.
Decision: Accept-Main