Schema-adaptable Knowledge Graph Construction
Authors: Hongbin Ye
Honghao Gui
Xin Xu
Xi Chen
Huajun Chen
Ningyu Zhang
Venue: EMNLP 2023 Findings (2023)
Keywords: Knowledge Graph Construction
Information Extraction
TL;DR: We propose a new task called schema-adaptable KG Construction, which aims to continually extract knowledge based on a dynamically changing schema graph without re-training.

Abstract: Conventional Knowledge Graph Construction (KGC) approaches typically follow the static information extraction paradigm with a closed set of pre-defined schema. As a result, such approaches fall short when applied to dynamic scenarios or domains, whereas a new type of knowledge emerges. This necessitates a system that can handle evolving schema automatically to extract information for KGC. To address this need, we propose a new task called schema-adaptable KGC, which aims to continually extract entity, relation, and event based on a dynamically changing schema graph without re-training. We first split and convert existing datasets based on three principles to build a benchmark, i.e., horizontal schema expansion, vertical schema expansion, and hybrid schema expansion; then investigate the schema-adaptable performance of several well-known approaches such as Text2Event, TANL, UIE and GPT-3.5. We further propose a simple yet effective baseline dubbed AdaKGC, which contains schema-enriched prefix instructor and schema-conditioned dynamic decoding to better handle evolving schema. Comprehensive experimental results illustrate that AdaKGC can outperform baselines but still have room for improvement. We hope the proposed work can deliver benefits to the community.

Schema-adaptable Knowledge Graph Construction
Hongbin Yel Honghao Gui Xin Xul Xi Chen', Huajun Chen 14, Ningyu Zhang
Zhejiang University 2 Zhejiang Lab 3 Platform and Content Group, Tencent
Zhejiang University Ant Group Joint Laboratory of Knowledge Graph
yehongbin@zhejiine huajuunssrrraannnnneeeeeee... edwp:u/uejuuuuuu.cco.... {guihonghao, xxucs, huajunsir, zhangningyujezju..
com,
Abstract
Conventional Knowledge Graph Construction (KGC) approaches typically follow the static information extraction paradigm with closed set of pre-defined schema. As a result, such approaches short when applied to dynamic scenarios or domains, whereas a new type of knowledge emerges. This necessitates a sys- tem that can handle evolving schema automat- ically to extract information for KGC. To ad- dress this need, we propose a new task called aitemma-adaptbble KGC, which aims contin- ually extract entity, relation, and event based on a dynamically changing schema graph without re-training. We first split and convert exist- ing datasets hased on three principles to build a benchmark, ie.. horizontal schema expan- sion, vertical schema expansion, and hybrid schema expansion; then investigate the schema- adaptable performance of several well-known approaches such as Text2Event, TANL, UIE and GPT-3.5. further propose a simple yet effective baseline dubbed ADAKGC, which contains schema- enriched prefix instructor and schema-conditioned dynamic decoding to bet- ter handle evolving schema. Comprehensive experimental results illustrate that ADAKGC can outperform baselines but still have room for improvement. We hope the proposed work can deliver benefits to the community'.
Instance
Instance 2
Instonce
Knowledge Graph
Ontology Graph
Task: Documents
Schenaa Change
Schemna
Schemn
Schema 3
Figure Knowledge Graph Construction (KGC) with dynamic updates of schema.
relation extraction (RE) (Chen et al., 2022a: Zheng et al., 2021; Yao et al., 2022) and event extraction (EE) (Huang et al., 2018; Liu et al., 2020; Lu et al.. 2021; Lou et al., 2023) are "reactive", relying on static pre-defined schema from end-users. How- ever, as shown in Figure the schema may evolve along with scenario adaptation, making previous models challenging utilize without re-training.
Note that cxisting information extraction sys- tems can only handle a fixed number classes by pre-defined schema and performing once-and-for all training fixed framework. It is desirable to respond to changes (e.g.. evolving schema) to exist- ing KGs, making the system act more "proactively" like humans who can handle flexible knowledge updates. Early, several approaches introduce in- cremental learning (Cao et al., 2020; Wang et al.. 2019; Shen et al., 2020; Cui et al., 2021b) to learn new classes continually. In this case, the extrac- tion system learns from the class incremental data stream but usually suffers significant performance degradation the old class when adapting to the new class. Stated differently, previous studies put emphasis on struggling against catastrophic for- getting (Thrun, 1998). However, for the schema- evolving scenarios, the dynamic generalizability extraction models plays a vital role and needs to be
1 Introduction
Knowledge Graph Construction (KGC), typi- cally through information extraction, has enjoyed widespread empirical success and can provide back- end support for various NLP tasks, such as question answering (Saxena et al., 2020; Shang al., 2022; Zhang et al., 2022a), commonsense reasoning (Ya- sunaga et al., 2021; Zhang et al.. 2022d) etc. Tra- ditional KGC tasks, including named entity recog- nition (NER) (Liu et al., 2021; 1; Wang et al., 2021),
Corresponding author. 'Code and datasets available at https: http:/ggttthhbhhhee com/ ejunlp/AdaKGC.

---

inspected from an ontology evolution perspective. Therefore, we propose a novel KGC task dubbed acapaaled KGC, where the models are re- quired to have the ability to represent and adapt to complement knowledge extraction. We first con- struct datasets according to three principles of evo- lutionary schema directions (Horizontal Schema Expansion, Vertical Schema Expansion, and Hybrid Schema Expansion) on three tasks of NER, RE², and EE. Through empirical analysis, we notice that approaches of Text2Event (Lu et al., 2021), TANL (Paolini et al., 2021), UIE (Lu et al., 2022), and GPT-3.5(Ouyang et al., 2022) cannot effectively extract the information given complex evolving schema. We argue that the major issues lie in the following: 1) How to learn dynamic and general- izable schema representations conditions for ex- traction; 2) How to precisely extract new instances constrained with newly updated schema.
presents new research opportunities for the NLP community.
2 Problem Statement and Overview
2.1 Background of KGC
KGC has been promising research challenge (Lu et al., 2022: Zhang et al., 2022b), and existing benchmarks utilize a well-defined schema for di- recting knowledge graph construction, focusing on generating domain-specific knowledge graphs or aggregating heterogeneous structured databases. For example, FEW-NERD (Ding et al., 2021) con- sists of coarse-grained and fine-grained entity type definitions to locate and classify named entities from unstructured natural language. NYT (Riedel et al., 2010a) extracts relational triple instances specifically from textual data sources according a specific taxonomy structure. ACE2005 (Ntroduc- tion) identifies triggers and event types based on context, and each has its own event arguments, de- scribed in slot-filling way. In addition, TAC-KBP (Ellis et al., 2014) is designed to leverage existing generie domain structured data sources and extend entity links employing descriptive text as additional information. OAEI (Euzenat al., 2011) creates an integrated ontology based on an alignment be- tween two or more existing ontologies or knowl- edge graphs. In this paper, we focus on the work of extracting knowledge instances from unstructured text, which is regarded as the schema- constrann prediction (structure prediction) task.
To this end, we propose a simple baseline dubbed ADAKGC, which introduces Schema- enriched Prefix Instructors (SPI) to represent and transfer the learnable schema-specifie knowledge. At each iteration stage, we linearly convert from the current schema graph learnable prompts, initial- ized with the ontology name and connected task- specific prefixes. encourage the decoder to un- derstand the dynamic schema, we utilize a Schema- conditioned Dynamic Decoding (SDD) strategy that constructs a decoding path of schema-specific vocabulary to the output space. When the schema changes, we dynamically construct new trie-tree adjust the output space. Note that ADAKGC is model agnostic and can handle variety of chal- lenging schema evolution scenarios. We summa- rize the contribution of this work as:
2.2 Definition of Schema-adaptable KGC
In the real world, the KGC system extracts struc- tured knowledge from unstructured text and nor- malizes it to the instance graph according to a fre- quently adjusted schema. Given set of schema graphs S {81, 82, ...; 8n), the task of schema- adaptable KGC is generate a set of schema- constraint instances G {gi, 92,
We introduce adaptable KGC to meet the schema evolu- tion requirements, which is a new branch that has not been well-explored to the best of our knowledge.
task of the schema-
... 12.....9... 9n for each iteration. Suppose there a model Ma = I
LM (D) (S1) trained on the initial training set, after which labeled data for updated schema are not available. A schema schema-adaptable data stream (D(1), D(2). the adaptability of model for the dynamic up- dates of schema. Each D(k) contains dev/test data (Dk dev. Dí and schema graph sk. Note that the model will not be re-trained but hope to pick up on the ability of information extraction with evolv- ing schema. The challenge is that the model is
We propose a new baseline ADAKGC, which includes schema-enriched prefix instructors and schema-conditioneed dynamic decoding strategy, and experimentally demonstrate the adaptability.
...,, D(M) is provided to evaluate
We release the aademaa-daapable KGC bench- mark, which imposes new challenges and
"We regard RE as relational triple extraction in this paper.

---

Input Professor Dr. Weldon with his sou Charles traveled throughout the world, living in Paris, France.
(peopie peeole.location hwwp://cccoooccllliioe localion)
location,contains, contains, contains,
(location nocation.contains, location)
(people, children.persory
pesople.children.errea
(people, wwtp:e.essssso.ppoooom
(Dr: Weldon, place Riedd Paris)
Graph
(Frunce.cunuuing Paris)
location. Puris))
(Dr. Weldon childdee Charles)
(Prance,consinn, Paris)
(Dr. Weldon,chiddonn children))
Horizental Schema Expansion
Weldon profession, Prafessor)
(pesple.facation.eee location,location
Vertical Schemn Expansion
[occttioncom contains, contains,/ccatinn)
(people.plase place lived location
(people, place lnvatioccation) location.capital.g location)
(location, contains, location)
Instance Extraction:
Weldonn.pace lived,Puris)
Weldon.place lives,, Paris)
(Dr. Weldon Cariiionoooaiin.n.aa
(Frunce, Paris)
(Frunce.coniaing Paris)
FFrunce.capiall Paris)
Illustration of schema-adaptable KGC with iterative schema evolution. Schema graph nodes (colored are evolved adding class node (colored in horizontal schema expansion while is inherited by subclass node (colored in vertical schema expansion.
Figure 2:
expected perform well in each iteration of the Dis which contains the golden instances changed for the updated schema.
Algorithm 1 Dataset Construction Process.
Input: iteration N, raw schema Sraws and dataset (Ptran: Dew Diegy
Output: Schema SD.(Dirain DN D.V
dev' teat)
2.3 Dataset Construction Process
Randomly initialize Rinit nodes in Sraw S Pick out the instance associated with schema as ( = {D) (rimin D Deww: Dlest)
As shown Figure 2, we design three principles regarding different types of schema evolution and apply Algorithm to build the dataset for evalua- tion: (1) Horizontal Schema Expansion requires the schema to add new class nodes of the same level, which be considered form of class- incremental learning without new classe instances training data. Based on the generalization ef- fect on the neighboring new classes, we can assess the transfer capabilities of the schema feature. (2) Vertical Schema Expansion requires the schema to add subclasses of father classes. Based on the generalization effect on subclasses, we can assess the inheritance and derivation capabilities of the schema feature. (3) Hybrid Schema Expansion requires the schema to randomly expands nodes horizontally or vertically at each iteration, which summarizes schema graphs and represents their po- tential co-cvolutionary pattern. More details are in Appendix A. 1, besides the above structural exten- sions, we further explore analogous node replace- ment from the perspective of semantics.
3: for iteration 2,...., 72 do
4: Horizontal Schema Expansion: Compute Pneighhor (W.. Ws) )) for candidate schema S
5: Vertical Schma Expansion: Select Nater
sub node whose father node belongs to S and update
Hybrid Schema Expansion: Combine ex- tension Steps 4 and 5
7: Ouput iteration i dataset schema S, S, instance D(i) = [D) Dirst))
8: end for
adjusted according to the schema. We detail sev- cral vanilla basclines as follows and introduce the proposed ADAKGC in $3.
Schema-adaptable KGC can be thought of as a structured prediction language task that transfers information between class nodes through the generalizability of the structure. TANL (Paolini et al., 2021) introduces augmented nat- ural language translation task from which infor- mation related to the schema can be implicitly ex- tracted. TEXT2EVENT (Lu al., 2021) is uni- fied sequence-to-stee -oreencett--stuetuuee architecture for event extraction with constrained decoding algorithm for event schema knowledge injection during in-
Vanilla Baselines:
2.4 Schemaaadaptabbe KGC Benchmark
There two challenges for schema-adaptabbe KGC. Firstly, since the schema is updated in each it- eration, the schema evolution information needs to be dynamically injected into the model. Secondly, since the output target of KGC is often demand- specific, the extraction results should be adaptively

---

Ixper Teat
Ienation i Clspur Instence:
Becrsiie a selensa
Schima-pecific
Subroquent lis retactinnt, Professi DeWridon with his Charles traveled throughon the world, Paris,
FFusse.coiitiin (Dr. Wellos,lase livad Panvej (Th Wolken plose lived, Frnnce) [Dt. Wellon,chikreen, Chirkrenn,,uurees
Instanes Databsss Upddare
ipo; tpc; 1pe, 194 1po wwtt-atttt
Inveaionn a Ouipur fnsemce: O'nnce porrats. Wt. Waldonplaze
licrstion ard ydlctna
text embodding
Weldonglace Sydney? en (Paris administra divitios Prance)
Schemaz-nnicceed Prefix Instructor
tpc; tpe, tpc: tpc, 1pts 100--
Dacoder
Schemm--onditioned Denamic Decoding
liersiiinn trie trme (peaple Ioortoon
Ineration a+1 trie tree (poolkysloer livetlocutio)
Saceuun Tarte qulmiiaaiie
Iteaaion
Akcadionucoeereen Jocation) persooo) person)
pocatton,owssite person)
7
Congtutladeetiiorstte disiisss,aannttyy
Figure 3: The overview of ADAptive Knowledge Graph Construction ADAKGC.
ference. UIE (Lu et al., 2022) is unified text-to- structure generation framework that enables unified modeling different IE tasks and adaptively gener- ates target sequences by schema- based prompting mechanism. GPT-3 (Brown et al., 2020), large- scale language model (LLM), can serve as a base- line for adaema-adaptable KGC. Although current works focusing on structured extraction can achieve excellent performance with static types of knowl- edge, they typically unaware of schema evolu- tion. To clarify this issue, we introduce simple yet effective baseline dubbed schema-ADAptiv Knowledge Graph Construction ADAKGC.
max log Po sk)
8
(2)
max
log po het)
0
LE Videx
where he is the activation vector at decoding time
step l. he [ ;h,"m (m)] is concatenation of all activation layers, and n,()) is the activation vector of the j-th layer at time step t.
3.2 Schema-enriched Prefix Instructor
Inspired by prefix-tuning (Li and Liang, 2021), use task-specific prefix instructors to indicate task information, which are pairs of transformer- activated differentiable sequences (stens side), ) each containing p consecutive D-dim vectors for encoder and decoder. Since using discrete natural language task instruction the context (c.g.. "The schema used for the task is:") may guide the LM to produce a sub-optimal generated sequence, we op- timize the instructions as continuous soft prompt, propagating upward to all transformer activation layers and rightward to subsequent tokens.
3 The Proposed Baseline: ADAKGC
3.1 Overview
As shown in Figure 3, ADAKGC utilizes pre- trained encoder-decoder language model (LM) T5 (Raffel al., 2020a) the basic architecture for the schema-adaptable KGC task. Specifically, let encoder input X.n = [S; X] be the concatenation schema and input X. the decoding process, the LM calculates the conditioned probability of generating new token given the previous token Ver
Due to schema changes with iterations, we present schema-specific prefix instructors to in- struct the encoding process. Specifically, we for- malize the schema graph as linearized text. Assume given the constrained schema of RE task 8 {(11 ri, (h.,,,,, Tas }, and tpc, (h4, Ti, (x) denotes the i-th triple prefix constraint. By concate- nating these schema prefix constraints initialized by word embedding, spe can dynamically adjusted as the schema evolves, and added padding tokens to a fixed length when instructing the LM:
| I p (ve Ueee S, X)
pYde X
(1)
We initialize the model using the pre-trained parameter 0. Here, pa is trainable language model distribution. In the k-th iteration, we perform a gradient update on the following log-likelln bigeliood objective:

---

In the schema-conditiones decoding process, we apply a trie-based decoding mechanism that dy- namically constructs a trie-tree by leveraging the latest schema. An intuitive interpretation is that the schema contains rich semantic information (e.g., instance types) and structural information (e.g., re- lational edges between instance types) that the decoding process can be constrained to ensure that the generated token is valid. Specifically, strain the model to generate the type tokens consis- tently with the existing schema at the type location, We pursue the LM output to be a sequence of RE following pattern and optimized using the standard seq2seq objective function:
spe- Concat (tpe1...... tpen, PAD)
(3)
Thus, the schema-enriched prefix instructor provides a two-part prefix combination Z {sien, spee; }. where ":" separates the respec- tive prefix instructors for encoder and decoder. We recursively activate the decoder transformer activa- tion vector he which is the connection of all layers, at time step l in the LMM
con-
if S p [M (ye, het X), otherwise
sig,
h2
(4)
[bos] ...Thh"h" _eeh" R((",, T,(")
E c(")..... [co[s
The training parameters of our model con- tain the LM parameters 8, the encoder-decoder task-specific prefix instructor(sian si and the schema-specifie constraint instructor spc. For stable optimization, we follow Li and Liang (2021) to reparameterize the matrix Molt, :| (MM with smaller matrix My consist- ing of large feedforward neural network MLP, which can alleviate the optimization instability caused by directly updating the prefix parameters and is applied to {sten, side; spc). We train the parameters of the model in the following steps: (1) First, freeze other parameters, fine-tune the prefix instructor {sien, } to learn task-specific prompts; (2) Secondly, freeze (sien; side), opti- mize the schema-speccfic instructor spc for the given schema graph; (3) Finally, we unfreeze the LM parameter and collaboratively optimize all parameters to capture the association between the prefix instructor and model parameters.
where E(7), Th(m), g(/), Ef(T) refer the n-th generated head entity, tail entity, and their respec-
tive types while R(a)) refer to relation.
4 Experiments
4.1 Experimental Settings
Datasets. We conduct experiments KGC tasks, including NER, RE and EE. The used datasets includes FEW-NERD (Ding et al., 2021), NYT (Riedel et al., 2010b) and ACE2005 (Walker et al., 2006). In our work, we need to construct schema as well as golden validation/test sets dynamically. For each dataset, we build three types of evaluation set- tings based on $2.3. Therefore for original datasets, we use certain proportion of the schema as ini- tialization to conduct schema expansion regarding three schema evolution categories in Appendix A.1. Evaluation. We use span-based Micro-F1 as the primary metric. Rel-S means that the relation cor- rect if the relation type is correct and the string and entity types of the related entity mentions are cor- rect. For each iteration experiment, we report the average performance over 3 random seeds. UIE is implemented without pre-training by directly using T5-v1.1-base as the backbone for a fair comparison. More details are in Appendix A.2.
3.3 Schema-conditioned Dynamic Decoding
Previous works leverage a greedy decoding algo- rithm to generate linearized instance predictions to- ken by token for the hidden sequence he which se- lects the token with the highest prediction probabil- ity p (yr Vec,, S, X) at each vanilla decoding step t. Unfortunately, when the schema changes, this de- coding algorithm does not guarantee the generated instances are consistent with the latest schema. In other words, it may result in out-of-date or invalid types being generated due to the lack of labeled data fine-tuning the model to adapt the probability distribution to the current schema constraints. In addition, the greedy decoding algorithm neglects useful schema knowledge that can effectively guide the decoding process.
4.2 Main Results
We report empirical results regarding horizontal schema expansion, vertical schema expansion and hybrid schema expansion settings to compare our proposed methods with the baselines. The perfor- mance over all iterations during the whole schema- adaptable KGC process is presented in Table 1-3. From the results, we can observe that:

---

Model
Iter
Iter 2
Iter 3
Iter 4
Iter 5
Iter
Iter 7
AVE
Entity
TANL
71.52
65.21
60.38
56.37
53.21
49.66
46.55
57.56
(NERD)
UIE
72.72
66.78
62.24
58.29
55.08
51.42
48.04
59.22
Ent-F1
AdaKGC
72.91
66.95
62.37
58.51
55.38
51.81
48.54
59.50
Relation
TANL
89.92
81,81
78.34
72.71
68.42
65.19
62.94
74.19
(NYT)
UIE
90.17
82.09
78.74
73.12
69.03
63.53
63.30
74.57
Rel-S
AdaKGC
90.34
82.33
79.03
73.34
69.19
65.87
63.58
74.81
Event Trigger
TEXT2EVENT
69.23
68.05
65.45
61.37
60.15
59.34
54.42
62.57
(ACE2005)
UIE
70.75
69.13
66.20
62.19
60.90
59.83
54.74
63.39
Evt Tri F1
AdaKGC
72,43
70.90
68.14
63.49
61.97
61.33
55.73
64.86
Event Argument (ACE2005)
TEXT2EVENT
46.15
44.40
42.58
39.73
39.17
38.77
35.31
40.87
UIE
49.14
47.90
45.75
42.32
41.83
41.27
37.60
43.69
Evt Arg F1
AdaKGC
49.18
48.14
47.08
43.85
43.17
43.10
38.79
44.76
Table Horizontal schema expansion results schema-adaptable knowledge graph construction.
Model
Iter
Iter 2
Iter 3
Iter 4
Iter 5
Iter 6
Iter 7
AVE
Entity
TANL.
73.49
64.83
$7.05
51.84
47.16
42.06
35.96
53.20
(NERD)
UIE
74.45
66.09
58.24
53.01
48.24
45.77
48.41
56.32
Ent-F1
AdaKGC
74.45
66.05
58.26
53.06
48.39
45.92
48.57
56.39
Relation
TANL
90.13
83.11
76.63
73.03
67.03
63.40
47.13
71.49
(NYT)
UIE
90.38
81.76
74.27
71.61
65.30
62.95
59.81
72.30
R1
AdaKGC
90.18
81,65
74.13
71.48
65.10
62.75
59.77
72.15
Event Trigger
TEXT2EVENT
67.10
55.81
50.38
52.99
45.82
41.39
41.65
50.73
(ACE2005)
UIE
70.94
60.00
$7.01
62.52
60.39
54.90
53.62
59.91
Evt Tri F1
AdaKGC
70.57
59.75
56.50
62.31
60.57
55.49
54.32
59.93
Event Argument (ACE2005)
TEXT2EVENT
49.32
37.83
33.43
35.49
30.67
27.49
27.79
34.57
UIE
50.70
41.45
39.66
44.14
43.65
38.86
37.77
42.32
Evt Arg F1
AdaKGC
51.87
42.74
40.68
45.16
43.97
40.10
39.05
43.37
Table 2: Vertical schema expansion results in schema-adaptable knowledge graph construction.
Schema adaptive generalization challenge. On all three expansion categories, the model per- formances tend to decrease as the iterations in- crease. TANL achieves lower performance which employs an augmented language and implicitly trains the model to learn schema information. TEXT2EVENT utilizes schema as constraint information the decoding side and outper- forms other models in some iterations. Although ADAKGC and UIE obtain optimal suboptimal performance, the performance of itcration and iteration 7 has a significant drop. We believe that the implicit schema evolution rules can help future work to develop adaptive generalization capabili- ties for schema-adaptable KGC.
Hetturaa//ri Vintol-GPTTS
BetrrmasAdko Vetol AaKoo -X- TbAeSS-AKKG
12
1ter1
Iter3
Iter3
lter?
Figure 4: GPT-3.5 results on schema expansion dataset.
ADAKGC improves 0.71% on trigger extraction and 3.65% on event argument extraction, indicat- ing that ADAKGC can capture schema-specific information under evolutionary schema.
Schema-enhanced modules boost the perfor- mances. Compared other models, ADAKGC is improved with schema-enhanced modules on both the encoder and decoder, which allows it to achieve the best performance in most settings. On the ACE2005 hybrid schema expansion dataset,
LLMs can understand schema adaption pat- terns better. To explore the performance LLMs Qiao et al., 2023) on the proposed tasks, we per-

---

Model
Iter
Iter 2
Iter 3
Iter 4
Iter 5
Iter 6
Iter 7
AVE
Entity
TANL
68.37
58,50
54.51
52,46
48.67
44.13
41.51
52.59
(NERD)
UIE
69.26
59.80
55.71
53.66
49.55
46.97
48.11
54.72
Ent-F1
AdaKGC
69.48
59.97
55.94
53.89
49.92
47.44
48.43
55.01
Relation
TANL
88.67
81,59
76.48
72.22
72.35
61.76
57.87
72.99
(NYT)
UIE
90.17
83.14
77.81
71.45
72.97
65.57
63.12
74.89
Rel-S F1
AdaKGC
90.07
83.06
77.68
71.38
72.94
65.73
63.32
74.88
Event Trigger
TEXT2EVENT
69.26
56.99
53.32
46.03
40.44
56.86
48.84
53.11
(ACE2005)
UIE
74.69
66.34
63.12
63.21
59.86
53.18
53,26
61.95
Evt Tri F1
AdaKGC
74.84
66.99
63.28
63.07
60.94
54.72
54.80
62.66
Event Argument (ACE2005)
TEXT2EVENT
50.32
38.14
35.79
31.94
28.79
37.75
33.54
36.61
UIE
51.94
45.22
42.97
43.12
40.66
36.66
36.17
42.39
Evt Arg F1
AdaKGC
55.08
48.58
46.19
46.27
45.22
40.52
40.41
46.04
Table 3: Hybrid schema expansion results schema-adaapablle knowledge graph construction.
5A12
Iteration Iteration:
Average SEOrES
23.53
F1 scores
30-
10
Elorizontal scherna expansion
Vertical schema expunsion
elecadin
Bral
possoe <bor
(ner) buskinn ober boin
exyitte [sce]
Atakoc
Deoadinx
w/o Schara lesoruter
Laimroter&& Decoding
Figure 6: Case study on NERD vertical schema expan- sion dataset, Iteration vs. Iteration 7.
Figure 5: Ablation study NERD horizontal schema expansion dataset, with the average result 7 iterations.
ing, we conduct the ablation study, and the results are shown in Figure 5. From two evolutionary cate- gories, we observe that: (1) Both schema-enrichan prefix instructor and schema-conditioned dynamic decoding can help the schema-adaptabe learning process; (2) Efficiently encoding schema evolu- tion information more important, which achieves improvements of 0.77% on horizontal schema ex- pansion and 0.36% on vertical schema expansion.
form comparative experiments with GPT-3.5 on NYT. Since we cannot utilize all training instances, we report in-context learning performance given 20-shot demonstrations as shown in Appendix A.5. From Figure 4, we notice that GPT-3.5 capable producing instances that conform to the dynam- ically changing schema but still yield low perfor- mance due to the low-shot issue. Likewise, we sample several cases and use ChatGPT3 to evalu- ate schema-ate addatable KGC (See Figure 7 and 8 in Appendix A.6), which surprisingly demonstrates stable generalization ability with evolving schema. These findings indicate a promising future work of adhema-adaptable KGC to develop alignment prompts with LLMs.
4.4 Case Study
As shown in Figure 6, we randomly select 8 types and observe that: (1) The types that appear in the initial schema mostly degrade performance, indi- cating that the model causes slight confusion as the schema expands. (2) Due to the structural in- heritance relationship in the vertical expansion of the schema, our model can effectively transfer the labels of the father node to the child nodes when new child nodes are added.
4.3 Ablation Study on ADAKGC
To prove the effects of the schema-enriched prefix instructor and schema- conditinne dynamic decod-
chatgpt/
To further analyze the drawbacks of our model

---

Ervor Analysis
Input Example
Gold Typo(s)
Predicted Typo(s)
Proportion
Weak
Kelly, who declined to lalk to reporters here, travel to Tokyo Sunday
Meet[trave]]
Transpor[[rrvel]]
38%
Transfer
for talks with Japanese officials.
Inheritance
started his entertainment career ABC. where credited
Start-
Personnel(staried
24%
Deficiency
with creating the 'movie of the week' concepl.
Pusition[
Relevance
Kelly, the US assisiant secrelary for East Asia and Pacilic Allairs, arrived Scoul from Beijing Friday brief Yoon, the forcign minisles.
Transoo[(arived] Meet[brief]
Neglect
Transpor[arrived]
21%
Within weeks he was arrested and charged with sodomising official driver several years previously and with abusing his powers to up the offence.
Arrest-
Class Imbalance
Jail arrestrre)
Transpon[arrested]
13%
Charge-
[ndic(fcharge]]
Anne-Maric will get the couple's 19-room home New York which was on the market last year for 21.5 million dollars, as well as their fine art collection.
Potential
Transfer- ownershiplget]
None Event
4%
Annolation
Table Error analysis on all ACE2005 schema expansion datasets.
and promote future works of schema-adaptule KGC, we count incorrect instances and classify them into five categories below, as shown in Table 4: (1) Weak Transfer: Despite schema expansion, the model is prevented from updating labels by old model parameters. (2) Inheritance Deficiency. The label not inherited in time when subdividing the father node. (3) Relevance Neglect. The lack on- tology relevance leads to the absence of correlated event extraction.
ited training instances available. As opposed to this instance-driven KGC approach, we argue that the aarive approach can leverage evolutionary instructions provided with richer ontological asso- ciations, resulting new challenges and research opportunities.
(4) Class Imbalance. Models suffering from unbalanced class learning problems tend to depend on similarly in-context sentences judge high-frequency labels. (5) Potential An- notation. Some example outputs suggest potential errors or omitted annotation.
5.2 Lifelong Learning
Lifelong learning is aimed at training new classes online without catastrophic forgetting. Gener- ally, lifelong learning mainly falls into four cat- egories: iecllarization-baseee (Kirkpatrick et al.. 2017; Zenke et al., 2017; Aljundi et al., 2018), replay-based (Lopez-Paz and Ranzato, 2017; Shin et al., 2017), architecture- based (Mallya and Lazeb- nik, 2018; Yoon et al., 2018) and knowledge distil- lation (Chuang et al., 2020; Cao et al., 2020). To study ctass-incremental learning, Monaikul et al. (2021) builds unified NER classifier for all the classes encountered over time, while Wang et al. (2022a) develops framework to reconstruct syn- thetic training data of the old classes. Recently, (Wang et al., 2019) proposes lifelong RE method that employs an explicit alignment model over- come forgetting, while (Shen et al., 2020) presents a adlf-adaetive dynamic regularization method. To address class incremental learning in event detec- tion (Cao et al., 2020), Yu et al. (2021) takes ad- vantage of rich correlations among ontology types, and Liu et al. (2022) adopts continuous prompts to learn event-specific representation for prediction. Compared with previous work that focused only on class increments, we discuss three principles of schema expansion from the potential demand for schema adaptation.
5 Related Work
5.1 Knowledge Graph Construction
Automatic construction of knowledge graphs from textual or structured data has attracted extensive research in recent years, including tasks such as NER (Paolini et al., 2021: Cui et al., 2021a), RE (Lin et al., 2020; Joshi et al., 2020; Ye et al., 2021), I), EE (Ramponi et al., 2020; Liu et al.. 2020; Lu et al., 2021), etc. In contrast to closed-domain knowledge extraction, open knowledge extraction (Kolluru al., 2020; Zhan and Zhao, 2020; Kot- nis et al., 2022: Wang et al., 2022b) is oriented toward the absence of schema constraints and can quickly generate extensive and meaningful knowl- edge. However, the ignoring of schema introduces uncertainty and ambiguity in output control, and we believe that clear setting can be chosen track the realignment of instances. Besides, KGC in low-resource scenarios (Huang et al., 2018; Zhang et al., 2020; Schick and Schütze, 2021; Chen et al., 2022b; Ye et al., 2022; Zhang et al., 2022c) requires the model to predict new instances with only lim-

---

6 Conclusion
Ethical Considerations
This paper introduces a new task of schema- adaptable KGC with benchmark datasets and new baseline ADAKGC. We illustrate the task dif- ficulties with previous baselines on three principles of schema expansion patterns (horizontal, vertical, hybrid) and demonstrate the effectiveness of the proposed ADAKGC.
Intended use. The dataset and model in this pa- per indented be used for exploratory analysis of addptabeee KGC.
Biases. We collect data from existing datasets (e.g., Few-NERD: CC BY-SA 4.0 license.), which may contain some data with offensive language or discriminatory.
Acknowledgment
We would like to express gratitude to the anony- mous reviewers for their kind comments. This work supported by the National Natural Sci- ence Foundation of China (No.62202246), Zhe- jiang Provincial Natural Science Foundation of China (No. LGG22F03001 Ningbo Natural Science Foundation (2021J190), Yongjiang Tal- ent Introduction Programme (2021A-156-G), CCF- Baidu Open Fund, and Information Technology Center and State Key Lab of CAD&CG, Zhejiang University.
References
Rahaf Aljundi, Francesca Babiloni, Mohamed Elho- seiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018. Memory aware synapses: Leamning what (not) to forget. In Computer Vision ECCV 2018 15th Eu- ropean Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part III, volume 11207 of Lecture Notes Computer Science, pages 144-161. Springer.
Collin F. Baker and Hiroaki Sato. 2003. The framenet data and software. In ACL 2003, 41st Annual Meet- ing of the Association for Computational Linguistics, Companion Volume to the Proceedings. 7-12 July 2003, Sapporo Convention Center, Sapporo, Japan, pages 161-164. The Association for Computer Lin- guistics.
Limitations
The proposed work still contains several limita- tions, as follows:
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Verbert-Voss. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learnrrs In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Datasets: Note that several datasets, such as ACE2005, cannot be released due to LICENCE issues; we release the code to build the datasets and provide all the pre-processed publicly avail- able datasets (e.g., Few-NERD, NYT) We use sev- eral existing datasets to construct addetaa-adaptable benchmarks; however, previous datasets may have limited schema structures (the schema pattern in some datasets is very simple). We plan to build more datasets via crowdsoureing for comprehen- sive evaluation. In addition, we will continue to promote the construction of multimodal schema adaptive graphs, which leverage the dynamic evo- lution of schema to integrate visual and textual knowledge into self-learning graph extraction system.
Pengfei Cao, Yubo Chen, Jun Zhao, and Taifeng Wang. 2020. Incremental event detection via knowledge consolidation networks. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2020, Online, November 16-20, 2020, pages 707-717. Association for Com- putational Linguistics.
Baselines and Proposed ADAKGC: Note that the proposed one, although better than previous approaches, including Text2Event (Lu et al., 2021), TANL (Paolini et al., 2021), UIE (Lu al., 2022), still suffers from poor generalization ability. How- ever, we notice very stable performance with
Xiang Chen, Lei Li, Ningyu Zhang, Chuanqi Tan, Fei Huang. Luo Si, and Huajun Chen. 2022a. Rela- tion extraction as open-book examination: Retrieval- enhanced prompt tuning. In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development Information Retrieval, Madrid, Spain, July 11 15, 2022, pages 2443-2448 ACM.
LLM (though deficient performance). indicating a new promising solution for schema- schema-adatable KGC.

---

Xiang Chen, Ningyu Zhang, Xin Xie. Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022b. Knowprompt: Knowledge- aware prompt-tuning with synergistic optimization for relation extraction. In WWW '22: The ACM Web Conference 2022, Virtual Event, Lyon, Frunce, April 25 29, 2022, pages 2778-2788. ACM.
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel Weld, Luke Zettlemoyer, and Oner Levy. 2020. Spanbert: Improving pre-training by representing and predict- ing spans. Trans. Assoc. Comput. Linguistics, 8:64- 77.
Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer. 2008. A large-scale classification of english verbs. Lang. Resour, Evaluation, 42(1):21- 40.
Yung-Sung Chuang, Shang- Yu Su, and Yun-Nung Chen. 2020. Lifelong language knowledge distillation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. EMNLP 2020, Online, November 16-20, 2020, pages 2914- 2924, Association for Computational Linguistics.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Grabska- Barwinska, et al. 2017. Over- coming catastrophic forgetting neural networks. Proceedings of the national academy of sciences. 113333333233332
Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021a. Template-basee based named entity recognition us- ing BART. In Findings of the Association for Com- putational Linguisties: ACI/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of Findings of ACL. pages 1835- 1845. Association for Computational L Linguistics.
Keshav Kolluru, Samarth Aggarwal, Vipul Rathore. Mausam, and Soumen Chakrabarti. 2020. Imojie: Iterative memory-based joint open information ex- traction. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 3871-5886 Association for Computational Linguisties.
Li Cui, Deqing Yang. Jiaxin Yu, Chengwei Hu, Jiayang Cheng, Jingjie Yi, and Yanghua Xiao. 202 lb. Refin- ing sample embeddings with relation prototypes to enhance continual relation extraction. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/JJCNLP 2021. (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 232-243. Association for Computational Linguistics.
Bhushan Kotnis. Kiril Gashteovski, Daniel Rubio, Am- mar Shaker, Vanesa Sodtriguez-Tembran Makoto Takamoto, Mathias Niepert, and Carolin Lawrence. 2022. MILIE: modular iterative multilingual open information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 6939-6950. Association for Computational Linguisties.
Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang. Xu Han, Pengjun Xie, Haitao Zheng, and Zhiyuan Liu. 2021. Few-nerd: A few-shot named entity recog- nition dataset. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing, ACI/JCCNP 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pages 3198-3213. Association for Computa- tional Linguistics.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tniig: Optimizing continuous prompts for generation. Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing, ACL/UJCNLP 2021, (Volume I: Long Papers). Virtual Event, August 1-6, 2021, pages 4582- 4597. Association for Computational I Linguistics.
Joe Ellis, Jeremy Getman, and Stephanie M Strassel. 2014. Overview of linguistic resources for the tac kbp 2014 evaluations: Planning, execution, and re- sults. In Proceedings of TAC KBP 2014 Work- shop, National Institute of Standards and Technology, pages 17-18.
Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features.
Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guisties, ACL 2020, Online, July 5-10, 2020, pages 7999 8009. Association for Computational Linguis- tics.
Jérôme Fuzenat, Christian Meilicke, Heiner Stucken- schmidtt, Pavel Shvaiko, and Cássia Trojahn dos San- tos. 2011. Ontology alignment evaluation initiative: Six years of experience. J. Data Semant., 15:158- 192.
Jian Liu, Yubo Chen, Kang Liu, Wei Bi, and Xiaojiang Liu. 2020. Event extraction as machine reading com- prehension. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2020, Online, November 16-20, 2020, pages 1641-1651. Association for Computational Linguistics.
Lifu Huang, Heng Ji, Kyunghyun Cho, Ido Dagan, Se- bastian Riedel, and Clare R. Voss. 2018. Zero-shot transfer learning for event extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume I: Long Papers, pages 2170-2170. Association for Computational Linguistics.
Kun Liu, Yao Fu, Chuanqi Tan, Mosha Chen, Ningyu Zhang, Songfang Huang, and Sheng Gao. 2021. Noisy-labeled NER with confidence estimation. In

---

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional L Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 3437-3445. Association for Computational Linguis- tics.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida. Carroll L. Wainwright, Pamela Mishkin. Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der. Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instruc- tions with human feedback. In NeurlPS.
Minqian Liu, Shiyu Chang, and Lifu Huang. 2022. In- cremental prompting: Episodic memory prompt for lifelong event detection. CoRR, abs/2204.072 07275.
Martha Palmer, Paul R. Kingsbury, and Daniel Gildea. 2005. The proposition bank: An annotated corpus of semantic roles. Comput. Linguistics, 3 (11):711-00..
David Lopez-Paz and Mare' Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Pro- cessing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6467-6476
Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma. Alessandro Achille, Rishita Anubhai, Cícero Nogueira dos Santos, Bing Xiang. and Ste- fano Soatto. 2021. Structured prediction as transla- tion between augmented natural languages. In 9th International Conference on Learning Representa- rions, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Jie Lou, Yaojie Lu, Dai Dai, Wei Jia, Hongyu Lin. Xi- anpei Han, Le Sun, and Hua Wu. 2023. Universal information extraction as unified semantic matching. CoRR, abs/2301.033 1.02822.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary Vito,, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch.
Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, and Shaoyi Chen. 2021. Text2event: Controllable sequence-to- structure generation for end-to-end event extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Lan- guage Processing, ACI/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 2795 2806. Association for Computational Linguis- tics.
Sameer S. Pradhan, Eduard H. Hovy, Mitchell P. Marcus, Martha Palmer, Lance A. Ramshaw, and Ralph M. Weischedel. 2007. Ontonotes: unified relational semantic representation. In Proceedings of the First IEEE International Conference on Se- mantic Computing (ICSC 2007), September 17-19, 2007, Irvine, California, USA, pages 517-526. IEEE Computer Society.
Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022. Uni- fied structure generation for universal information extraction. Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 5755-5772, Association for Computational Linguistics.
Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao. Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2023. Reasoning with language model prompting: A survey. In ACL. The Associa- tion for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang. Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020a. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 21:140:1-140:67.
Arun Mallya and Svetlana Lazebnik. 2018 Packnet: Adding multiple tasks to a single network by iterative pruning. In 2018 IEEF Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. pages 7765- 7773. Compuerr Vision Foundation IEEE Computer Society.
Colin Raffel, Noam Shazcer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020b. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of Machine Learning Research, 2 21(140):1-67.
Natawut Monaikul, Giuseppe Castellucci. Simone Fil- ice, and Oleg Rokhlenko. 2021. Continual learning for named entity recognition. In Thirty-Fifit Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Fleventh Symposium on Educational Advances in Artificial In- telligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 13570-13577. AAAI Press.
Alan Ramponi, Rob van der Goot, Rosario Lombardo, and Barbara Plank. 2020. Biomedical event extrac- tion as sequence labeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2020, Online, November 16-20, 2020, pages putational Linguistics.
Association for Com-
Ti. I Ntroduction. The ace 2005 ace 05) ) evaluation plan evaluation of the detection and recognition of ace entities, values, temporal expressions, relations and events I.
Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010a. Modeling relations and their mentions with- out labeled text. In Machine Learning and Knowl- edge Discovery in Databases, European Conference,

---

ECML PKDD 2010. Barcelona, Spain, September 20-24, 2010, Proceedings, Part III, volume 6323 of Lecture Notes in Computer Science, pages 148-163, Springer.
Computational Linguisties: Human Language Tech- nologies, NAACL-HLI 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 796-806 Association for Computational Lin- guistics.
Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010b. Modeling relations and their mentions with- out labeled text. In Machine Learning and Knowl- edge Discovery in Databases, pages 148-163, Berlin, Heidelberg. Springer Berlin Heidelberg.
Rui Wang, Tong Yu, Handong Zhao, Sungchul Kim, Subrata Mitra, Ruiyi Zhang, and Ricardo Henao. 2022a. Few-shot class-incremental learning for named entity recognition. In Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 571 1- 582. Association for Computational Linguistics.
Apoory Saxena, Aditay Tripathi, and Partha P. Taluk- dar. 2020. Improving multi-hop question answering over knowledge graphs using knowledge base embed- dings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 4498-4507. Association for Computational Linguistics.
Xin Wang, Minlong Peng. Mingming Sun, and Ping Li. 2022b. Oie@oia: an adaptable and efficient open information extraction framework. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume Long Pupers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 6213 6226. Association for Computational Linguis- tics.
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questinn for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the As- sociation for Computational Linguistics: Main Vol- une, EACI, 2021, Online, April 19 23, 2021, pages 255 269. Association for Computational Linguistics.
Xinyu Wang. Yong Jiang, Nguyen Bach, Tao Wang. Zhongqiang Huang, Fei Huang, and Kewei Tu. 2021. Automated concatenation of embeddings for struc- tured prediction. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference Natural Longuage Processing, ACL/IJCNLP 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pages 2643-26660 Association for Computa- tional Linguisties.
Chao Shang, Guangtao Wang. Peng Qi, and Jing Huang. 2022. Improving time sensitivity for question answer- ing over temporal knowledge graphs. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Vooume 1: Long Papers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pages 8017-8026. Association for Computational Linguis- tics.
Yunzhi Yao, Shengyu Mao, Xiang Chen, Ningyu Zhang, Shumin Deng, and Huajun Chen. 2022. Schema-aware reference as prompt improves data- efficient relational triple and event extraction. CoRR, 10./2210.10709
Hangjie Shen, Shenggen Ju, Jieping Sun, Run Chen, and Yuezhong Liu. 2020. Efficient lifelong relation ex- traction with dynamic regularization. In Natural Lan- guage Processing and Chinese Computing- 9th CCF International Conference, NLPCC 2020, Zhengzhou, China, October 14-18, 2020, Proceedings, Part II, volume 12431 of Lecture Notes Computer Science, pages 181 1192. Springer.
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021. QA-GNN: reasoning with language models and knowledge graphs for question answering. In Proceedings of the 2021 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 535-546. Association for Computational Linguistics.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Advances in Neural Information Process- ing Systems 30: Annual Conference on Neural In- formation Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. pages 2990-2999.
Sebastian Thrun. 1998. Lifelong learning algorithms. In Sebastian Thrun and Lorien Y. Pratt, editors, Learning Learn, pages 181-209. Springer.
Hongbin Ye, Ningyu Zhang, Shumin Deng, Mosha Chen, Chuanqi Tan, Fei Huang, and Huajun Chen. 2021. Contrastive triple extraction with genera- tive transformer. In Thirty- AAAI Conference Artificial Intelligence, AAAI 2021, Thirty-Thire Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, FAAI 2021, Virtual Event, February 2-9, 2021, pages 14257-14265. AAAI Press.
Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006. Ace 2005 multilingual training corpus.
Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. 2019. Sen- tence embedding alignment for lifelong relation ex- traction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for
Hongbin Ye, Ningyu Zhang, Shumin Deng. Xiang Chen, Hui Chen, Feiyu Xiong, Xi Chen, and Huajun Chen.

---

2022. Ontology-enhan enhanced shot learning. In WWW '22: The ACM Web Confer- ence 2022, Virtual Event, Lyon, France, April 25 29, 2022, pages 778-787. ACM.
for few-
graph completion. Journal of Software, 33(10):3531- 3545.
Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, and Jure Leskovee. 2022d. Greaselm: Graph reason- ing enhanced language models for question answer- ing, ArXiv, abs/2201.08860.
Jachong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. 2018. Lifelong learning with dynamically expandable networks. In 6th International Confer- ence on Learning Representations, ICLR 2018, Van- couver. BC, Canada, April 30 May 3, 2018, Confer- ence Track Proceedings. OpenReview.net.
Hengyi Zheng, Rui Wen, Xi Chen, Yifan Yang. Yun- yan Zhang. Ziheng Zhang, Ningyu Zhang, Bin Qin, Xu Ming, and Yefeng Zheng. 2021. PRGC: potential relation and global correspondence based joint rela- tional triple extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing. ACL/JCCLP 202 ( (Volume I: Long Papers), Virtual Event, Au- gust 1-6, 2021, pages 6225 0225-6235. Association for Computational Linguistics.
Pengfei Yu, Heng Ji, and Prem Natarajan. 2021. Life- long event detection with knowledge transfer. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, EMNLP 2021. Virtual Event / Punta Cana, Dominican Repub- lic, 7-11 November, 2021, pages 5278-5290. Associ- ation for Computational Linguistics.
Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelli- gence. In Proceedings of the 34th International Con- ference on Machine Learning, ICMI. 2017, Sydney, NSW. Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3987-3995, PMLR.
A Appendix
This section describes the details of experiments, including dataset construction and evaluation on downstream tasks.
Junlang Zhan and Hai Zhao. 2020. Span model for open information extraction accurate corpus. In The Thirty-Four Fourth AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty-Second Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 9523 9530. AAAI Press.
A.1 Dataset Construction
A.1.1 Construction Process
In each task, we execute three schema evolution strategies. The raw dataset statistics are shown in Table 5, where it can be seen that they have a two-level schema structure, leaving the research of a more hierarchical schema structure for future work. As shown in Algorithm 2-4, we describe in detail the specific construction process of horizon- tal schema extension, vertical schema extension and hybrid schema extension.
Jing Zhang, Xiaokang Zhang, Jifan Yu, Jian Tang, Jie Tang, Cuiping Li, and Hong Chen. 2022a. Subgraph retrieval enhanced model for multi-hop knowledge base question answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Lànguistics (Volume 1: Long Popers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pages 5773-5784, Association for Computational Linguistics.
In particular, we also release additional datasets from a semantic substitution perspective. As shown in Algorithm 5, analogous schema expansion re- quires schema replacement for semantically similar new nodes. Based on the performance of the old class transfer to the new semantic class, we can evaluate the semantic invariance capability.
Minhao Zhang, Ruoyu Zhang, Yanzeng Li, and Lei Zou. 2022b. Crake: Causal-enhancee table-filler for question answering over large scale knowledge base. In Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 1787-1798. Association for Computational Linguistics.
#Maj
#Sub
# #Train
#Val
#Test
NERD NYT
8
66
131,767
18,824
37,648
Ningyu Zhang, Shumin Deng, Zhanlin Sun, Jiaoyan Chen, Wei Zhang, and Huajun Chen. 2020. Rela- tion adversarial network for low resource knowledge graph completion. In WWW '20: The Web Confer- ence 2020, Taipei, Taiwan, April 20-24, 2020, pages 1-12. ACM/ IW3C2.
4
24
56.196
5,000
5,000
ACE-2005
8
33
19,216
901
676
Table 5: Raw datasets statistics. #Maj indicates the number of major classes, #Sub is the number of sub- classes, and #Val #Test is the number of sentences.
Ningyu Zhang, Xin Xie, Xiang Chen, Shumin Deng, Hongbin Ye, and Huajun Chen. 2022c. Knowledge collaborative fine-tuning for low-resource knowledge
Horizontal Schema Expansion. Neighboring nodes of the specified type that have high-level

---

similarity values in the same framework are also adjacent when projected into the semantic space (Huang et al., 2018). Existing research eftorts have developed many rich libraries of ontologies (e.g. Frammeet (Baker and Sato, 2003), VerbNet (Kipper et al., 2008), Propbank (Palmer et al., 2005), and OntoNotes (Pradhan et al.. 2007)), where each ontology type is associated with set of pre-defined neighboring ontologies. (1) Search- ing the ontology library to retrieve candidate nodes W associated with target nodes W, at the same hierarchy, (2) The similarity metric is obtained by calculating the cosine vector similarity of all candi- date nodes y to the target node Wa (Eq. 5). (3) Selecting the appropriate threshold of node pairs to confirm the sorted addition of horizontal nodes. (4) Updating the schema with horizontal nodes and adding the golden validation set and test set in the dataset.
replace similar semantic expressions for the nodes, (I) Random selection of candidate nodes obtain word expressions Wo (2) Candidate nodes are created by pairing Wc with all words in the corpus word list Wt. The consistency between individual words is calculated by the normalized point-by- point mutual information (NPMI) of w; and wy (Bgq. 6), where adding smooth and 7 controls for log p (w,, wj) weights for higher NPMI values (Eq. 7). (3) Adopting candidate nodes that exceed the threshold replace the schema and updating the golden validation set and test set in the dataset.
J(W
Σ WeWWe
NMPI (1og,
j=1,.... W (6)
pfw,, log p(u)).p(ws)
IW, WE Wf
@sim (W,, W.)
(5)
(MPI(W; wy)²
[Wyl2 ||W,1l2
logp (wi, Wj) +
Vertical Schema Expansion. Structural similarity needs to be exploited when adding schema hier- archy nodes as new classes. (1) For search con- venience, we link the hypernym ontology under a root node so that the schema forms tree struc- ture. (2) Starting at the root node, we utilize child selection strategy by recursively applying through the tree until reaching the deepest node. A node could be expandable when represents a non-terminal state or has hyponyms in seman- ties (c.g., location->country). (3) According to the available hyponyms, one (or more) child nodes are added to expand the current schema tree. (4) Up- dating the schema with vertical nodes and adding the golden validation set and test set in the dataset. Hybrid Schema Expansion. It is necessary hy- brid horizontal and vertical expansion form comprehensive structural topology, which more consistent with real scenarios. (1) Setting the threshold a for random selection. (2) Executing horizontal expansion iteration below the threshold or vertical node expansion above the thresh- old a. Note that when the father node of added nodes does not exist, we also add the father node maintain the schema hierarchy. (3) Updating the schema with the corresponding nodes and adding the golden validation set and test set in the dataset. Analogous Schema Expansion. To detect the se- mantic node sensitivity of the schema, we randomly
A.1.2 Schema-adaptable Datasets Statistic
We set the number A of total iterations, and initial- ize the original number of schema nodes. We show the statistics of schema- adantable datasets for each task in Table
A.2 Evaluation
We span-based Micro-F1 the major metric evaluate the model and adopt the same evaluation metrics as previous work:
* Named Entity Recognition: an entity men- tion correct if its strings and type match a reference entity.
Relation Strict: a relation is correct if its re- lation type is correct and the string and entity types of the related entity mentions are cor- rect.
* Event Trigger: event trigger is correct if its strings and event type match reference trigger.
* Event Argument: an event argument is cor- rect its strings, role type, and event type match a reference argument mentioned.

---

Horizontal
Vertical
Hybrid
Analogous
Taxonomy
#init
#Add
#N
#lnit
#Add
#N
#init
#Add
#N
#lnit
#Add
#N
NERD
30
7
6
30
66
7
NYT
10
8
10
2
8
10
2
8
24
7
ACE-2005
15
Table 6: Schema-adaptable datasets statistics. #Init indicates the number of initial subclasses, #Add the number of subclasses added per iteration, and #N the total number of iterations.
A.3 Hyper-parameters
A.4 Analogous Schema Expansion Experiment
We adopt T5-v1.1-base (Raffel et al., 2020b), which has 12 layers of the encoder, 12 layers of the decoder, 768 hidden units, and 12 attention heads the backbone. Specifically, we utilize Pytorch (Paszke al., 2017) to conduct experiments with batch size 16 one NVIDIA 3090 GPU. We detail the syper-parameters for each dataset as follows: NERD. The hyper-parameter search space is:
As shown in Figure 7, our ADAKGC also has powerful semantic transplantation capabilities, which achieves competitive performance with base- lines. With the schema-enrichee prefix instructor, ADAKGC achieves an improvement of 7.70% on average TEXT2EVENT
the event trigger extraction task and 4.87% the event argument extraction task. This verifies the proposed schema- enriched prefix instructor and decoding modules can learn general schema-adaptable ability even the schema evolution knowledge is rarely the pre- training stage. Note that TANL achieves the best performance on the NYT dataset, indicating that language models have the ability to learn schema semantic transfer implicitly ral language prediction task. Therefore we believe that in addition to the schema structure perception modules, semantic robustness modules for analo- gous node expansion scenarios are also essential.
epoch: 15
batch size: 16
accumulate:
learning rate: [1e-4, 3e-4, 5e-4]
augmented natu-
warmup rate: 0.06
NYT. The hyper-parameter search space is:
epoch: 20
A.5 GPT-3.5 Experiment Details
GPT-3.5 is large autoregressive language model with 175 billion parameters. To explore the perfor- mance of GPT-3.5 the schema-adaptive KGC task, follow the input format of few-shot learn- ing using OpenAl API As shown in Table 12, we utilize a fixed manual template to generate a con- textual window suitable for the model, including natural language task descriptions (text in blue), linearized schemas (text in purple), 20 examples in the model's 's context, and task prompts (text in red).
batch size: 16
accumulate:
learning rate: [1e-4, 3e-4, 5e-4]
warmup rate: 0.06
ACE2005. The hyper-parameter search space is:
epoch: 30
A.6 ChatGPT Results
ChatGPT trains an initial model using supervised fine fine-tuning and further utilizes reinforcement learn- ing systems to rank by quality for human feedback rewards. We handle Schema adaptabe KGC tasks
batch size: 16
accumulate:
learning rate: [le-4, 3c-4, 5c-4]
"https://platfrmn openaa..m.ppp.....mmmmo.m..m.... wwwpaa..com/.eeeeeeeeoooo.m..oet http://cc/o///////eeeeee//// gpt-3-5 hhttps://chat.cm openai. com/chat
warmup rate: 0.06

---

Algorithm 2 The construction process of horizon- tal schema expansion.
I: Set sampling seed 8, total iteration
N, raw schema Syaw- and raw dataset (Derain dew Diest)
2: Initialize blank schema S. blank dataset
(Dtrain; Dac, Diest) and initial node number Rinit, node number niter per iteration Randomly select Pimii nodes in Sram, = SU init node, S
3:
S,Sraw Sraw S the instance associated
4: Pick with S. (D) trainn Daews Dleat
{Drrain, Daeos Dreat)
5: for iteration N do
Eni Wel wewy 1|W Walla
Algorithm 3 The construction process of vertical schema expansion.
Calculate @ (W Wo)
for node between Srow and 5
Set sampling seed 0, total iteration N, raw schema Srow, and raw dataset
7: Pick out top nater schema node, S SS Srawl: Riterl, Sraw Sraw - Srawl: niterl Iteration i dataset S; = S,(Di dovt Diest) (Deevs Dreat]
{ (Dras Drew Drea)
8:
2: Initialize blank schemn S, blank dataset
(Droain Deev, Diest) and initial node number Rinit, node number Niter per iteration for major node in Sraw do
9: end for
S SU major node end for Randomly select rinst nodes Srows S = S init node, S S,S SSrw S 7: Pick out the instance associated with S. [D D Dlest)
by asking questions to the chatbot in a conversa- tional mode. First, we present the task description and the 20 demonstrations as shown in Figure 7. Then give paragraph text to test whether the chatbot can extract the corresponding triples based on the same schema as the demonstration exam- ples comply with. From Figure we can find that some of the facts are well extracted, indicating that ChatGPT can understand the task and perform ex- traction consistent with the schema. Finally, we add three new nodes "profession" "place founded" "founders" to the previous schema under hori- zontal schema expansion iteration. Output results in Figure 8 show that ChatGPT only adapts the output the updated schema but also deduces reasonable facts by a chain-of of-thought approach.
(Dtrain, Daevs Drest) for iteration N do
Randomly select natrr sub node, whose par- belongs to S
10: S Srawl: Niter], Sraw Sraw
Srawl: Niterr]
11: Iteration dataset S₁ = S,(Di dens Diest)
DDev Dzest}
12: end for

---

Algorithm 4 The construction process of hybrid schema expansion.
1: Set sampling seed 0, hybrid ratio o,total iter- ation N, raw schema Sraw, and raw dataset (Dtrain dev Dras)
Algorithm The construction process of analo- gous schema expansion.
Set sampling seed 0, total iteration N, raw schema Srow, and raw dataset
2: Initialize blank schema S, blank dataset
(Dtrain, Daeu, Dreat) and initial node number Rinit, node number Niter per iteration
{ (Dtran Dew Deerr
2: Initialize blank schema S, blank dataset
3: Randomly select Rimit nodes in Syaw, init node, S₁ = S,Smmw = Sram S S
{ (Draai, Deevy Diest) and node number nater per iteration
4: Pick out the instance associated with S. (D Dlene Dlest)
3: Initialize
Srow. S1 (Drrain Ddev, D Dree Dres I Pick out the instance associated with S. (Dirasno D Dleat)
$
S,
(Derains Daev, Dreat)
5: for iteration N do
6: if random(0,11< a then
(Drain Ddevs Drest} for iteration N do Randomly
Calculate @sim (W), Σ D. wayy 7:
[Wyll--1Wal1 for node between Sraw and S 8: Pick out top Niter schema node, S S
select Niiter node in S. calculate v(W) {Ew,ew NMPI (w,, ",,") )=,.. W,. for each node, create candidate nodes by pairing W, with all words Wy in the corpus word list
schema
U Sraml: Nster], Sraw Srawr Srowl: Nittrl
9:
else
10:
Randomly select niter sub node, whose
parent belongs to 5
7: Replace Sl: niter] with candidate nodes with analogous semantics
11:
S=S U Syowl: |: nitorl, Sraw Sraw
8: Iteration dataset S S.(D Diest)
Srawl: n4tee
12:
end if
Dievs Dteat) end for
13:
Itcration i dataset S, = S,Di dent D
(Ddev, Dreat)
end for

---

Model
Iter
Iter
3
Iter
Iter
Iter
AVE
Entity
TANL
68.34
62.70
57.22
52.94
47.73
42.24
36.09
$2,47
(NERD)
UIE
67.58
63.39
60.37
58.00
58.17
$4.26
49.53
58.76
Ent-F1
AdaKGC
68.22
63.87
61.01
58.18
58.76
54.58
49.81
59.20
Relation
TANL
89.80
86.82
83,51
77.99
73,61
73.32
66.37
78.77
(NYT)
UIE
89.66
86.56
83.27
77.80
73.30
73.27
66.33
78.60
Rel-S F1
AdaKGC
89.17
86.12
82.76
77.33
72.85
72.91
65.84
78.14
Event Trigger
TEXT2EVENT
64.40
60.39
53.96
52.64
40.83
32.45
29.58
47.75
(ACE2005)
CIE
69.69
64.53
62.10
58.47
49.76
42.51
39.31
55.20
Evt Tri F1
AdaKGC
69.63
64.77
61.86
58.31
50.08
43.07
40,40
55.45
Event Argument (ACE2005)
TEXTZEVENT
45.88
40,96
34.21
33.40
24.49
19.62
16.77
30.76
UIF
49.96
42.67
40.50
36.56
28.70
24.06
20.47
34.70
Evt Arg F1
AdaKGC
51.74
44.56
42.48
32.61
30.52
25.41
22.10
35.63
Table 7: Analogous schema replacement results in schema adaatable knowledge graph construction.
Input: The Belgrade district court said that Markovic will be tried along with Labels other Milosevic-era officials who face similar charges of 'nappropriate use of state property' that carry sentence of up to five years in jail.
"attack", "start position", "transfer ownership". "be born", Sentence[sentence] "sentence", "die", "arrest jail", "transport", "elect", "phone write", "end organization", "sue", "acquit", "marry", "extradite"
Iteration schema:
"attack", "start position", "transfer ownership", "be born", Sentence[sentence) "sentence", " "dic", "arrest jail". "transport". "elect", "injure", "phone write", "fine". "convict", "end organization", "sue". "acquit", "marry". "extradite"
Iteration 2 schema:
Ieeaaiin 3 schema: "attack", "start position", "transfer money", "transfer ownership", Sentence[sentence) "be bom",, "sentence", "dic", "demonstrate", "arrest jail", "transport", "elect", "injure". "phone write", "fine", "convict", "end organization", "sue". "acquit". "execute", "marry", "extradite"
Iteration 4 schema: "end position", "attack", "start position", "transfer money", Sentence[sentence) "transfer ownership", "be born", "sentence", "dic", "demonstrate", "arrest jail", "transport". "elect", "start organization", "injure", "phone write", "fine", "conviet", "end organization" "sue", "acquit", "execute", "marry", "extradite", "pardon"
Iteration 5 schema: "end position". "attack", "start position", "transfer money", "transfer ownership", "be born", "sentence", "die", "demonstrate", "arrest jail", "trans- port". "clect", "start orgamization", "injure", "phone write", "declare bankruptcy", "trial hearing", "fine", "convict", "end organization", "sue", "acquit", "appeal", "exe- cute", "marry", "extradite", "pardon"
Sentence[sentence]] Trial hearing[tried]
Iteration 6 schema: "end position", "attack", "start position", "charge indict", "trans- fer money", "transfer ownership", "release parole", "be born", "sentence", "die", "demonstrate", jail", "transport", "elect", "start organization",, "injure", "phone write", "merge organization", "declare bankruptcy". "trial hearing", "fine", "convict". "end organization", "sue", "acquit", "appeal", "execute", "marry", "extradite", "par- don"
Sentence(sentenne)]
Trial hearing [tried]
Charge-
Indict[charges]
Iteration 7 schema: "end position", "attack", "start position". "nominate", "charge indict", "transfer money", "transfer ownership". "release parole", "be born", "sen- tence", "die". "demonstrate", "arrest jail", "transport", "elect", "start organization", "meet", "injure". "phone write", "merge organization", "declare bankruptcy", "trial hearing", "fine", "conviet", "end organization", "sue", "divorce", "acquit", "appeal", "execute", "marry, "extradite", "pardon"
Sentence[sentencel
Trial hearing
Charge-
Indict[charges]
Table 8: Adaptive evolution of horizontal schema expansion on ACE2005 dataset.

---

Input: Kelly, the US assistant secretary for East Asia and Pacific Affairs, arrived Labels Scoul from Beijing Friday brief Yoon, the foreign minister.
Iteration 1 schema:
"personnel", "attack", "justice", "transfer money", "transfer Transpor[arrived]
ownership". "release parole", "be born", "sentence", "die", "demonstrate", "transport", Contact[brief] "business", "contact". "life", "fine", "sue", "execute", "marry", "extradite", "pardon"
Iteration schema: "personnel", "attack", "justice", "transfer money", "trans- [ranspor[[arrive]]
fer ownership", "release parole", "be born", "sentence", "dic". "demonstrate", Mcet[brief] "transport", "business", "meet", "life", "contact", "fine", "sue", "acquit", "appeal", "execute", "marr", "extradite", "pardon"
Ieraaiio 3 schema:
"personnel", "attack", "justice", "transfer money", "transfer Transpor(arrived]
ownership". "release parole", "be born", "sentence", "die", "demonstrate", "transport", "start organization", "meet", "life", "contact", merge organization", "business", "trial hearing". "fine", "suc", "acquit", "appeal", "execute", "marry", "extradite"; "pardon"
Meet[brief]
Iteration 4 schema: "personnel", "attack", "justice", "transfer money", "transfer ownership", "release parole", "be born", "sentence", "die", "demonstrate", "transport", Meet[brief] "start organization", "meet", "life", "phone write", "merge organization", "declare bankruptcy", "trial hearing", "fine", "end organization", "sue", "acquit", "appeal", "execute", "marry", "extradite", "pardon"
Transport(arrive]]
Iteration 5 schema: "personnel", "attack", "charge indict", "transfer money", "trans- Transport(arrived] fer ownership", "release parole", "be born", "sentence", "die", "demonstrate", "arrest Mee([brief] jail", "transport", "start organization", "meet". "life". "phone write", "merge organi- zation", "declare bankruptcy", "trial hearing", "fine", "convict", "end organization", "sue", "acquit", "appeal", "execute", "marry,, "extradite", "pardon"
"end position", "attack", "personnel", "charge indict", "transfer Transport[arrived] money". "transfer ownership". "release parole", "be born", "sentence", "die". "demon- strate", "arrest jail", "transport", "start organization", "meet", "injure", "phone write", "merge organization", "declare bankruptcy", "trial hearing", "fine", "convict", "end organization", "sue", "divorce", "acquit", "appeal", "execute", "marry", "extradite". "pardon"
Iteration 6 schema:
Meet[brief]
Iteration schema: "end position", "attack", "start position", "nominate", "charge indict", "transfer money". "transfer ownership". "relcase parole", "be born", "sen- tence". "die". "demonstrate", "arrest jail", "transport", "elect", "start organization", "meet", "injure", "phone write", "merge organization", "declare bankruptcy", "trial hearing", "fine", "conviet", "end organization", "sue", "divorce", "acquit", "appeal". "execute", "marry". "extradite", "pardon"
Transpor[[arrived] Mcet[brief]
Table Adaptive evolution of vertical schema expansion on ACE2005 dataset. Underlined classes refer to major classes, which will be covered by refined sub classes.

---

Imput: The charismatic leader of Turkey's governing party was named prime minister Labels Tuesday, a step that probably boosts chances that the United States will get permission deploy troops in the country along Iraq's northern border.
Iteration 1 schema: "attack", "justice", "transfer money". "transfer ownership", Transpor(deploy] "release parole", "be born", "sentence". "die". "demonstrate", "transport", "life", "fine", "sue", "execute", "marry", "extradite,, "pardon"
Iteration 2 schema:
"attack", "justice", "transfer money". "transfer ownership", Transport[deploy] "release parole", "be born", "sentence", "dic". "demonstrate". "transport", "meet". "life". "contact", "fine", "sue", "acquit", "appeal", "execute", "marry", "extradite", "pardon"
Iteration 3 schema: "attack", "justice", "transfer money". "transfer ownership", Transport|deploy] "release parole", "be born", "sentence", "die", "demonstrate", "transport", "start organization", "meet", "life", "contact", "merge organization", "business", "trial hearing", "fine", "sue", "acquit", "appeal", "execute", "marry" "extradite", "pardon"
Iteration 4 schema:
"attack", "justice", "transfer money", "transfer ownership". Transport(deploy] "release parole", "be born", "sentence", "die", "demonstrate", "transport", "start orga- nization", "meet", "life", "phone write", "merge organization", "declare bankruptcy", "trial hearing", "fine", "end organization", "sue", "acquit", "appeal", "execute". "marry" "extradite", "pardon"
Iteration 5 schema:
"attack", "charge indict", "transfer money", "transfer ownership", Transpor([deploy] "release parole", "be bom",, "sentence", "die". "demonstrate", "arrest jail", "transport", "start organization", "meet", "life", "phone write", "merge organization", "declare bankruptey". "trial hearing", . "fine", "convict", "end orgamization", "suc", "acquit", "appeal". "execute", "marry", "extradite", "pardon"
"end position". "attack", "personnel", "charge indict", "transfer Transpor[(deploy] money", "transfer ownership". "release parole", "be born", "sentence", "die", "demon- strate", "arrest jail", "transport", "start organization", "meet", "injure", "phone write", "merge organization", "declare bankruptcy", "trial hearing", "finc", "convict", "end organization", "sue". "divorce", "acquit", "appeal", "execute", "marry", "extradite", "pardon"
Iteration 6 schema:
Personnellnamed]
Ieeraiion schema: "end position", "attack", "start position", "nominate", "charge Transpor[deploy] indict". "transfer money", "transfer ownership", "release parole", "be born", "sen- tence", "die", "demonstrate", "arrest jail", "transport", "elect", "start organization", "meet", "injure", "phone write", "merge organization", "declare bankruptey", "trial hearing", "fine", "convict", "end organization", "sue", "divorce", "acquit", "appeal", "execute", "marry", "extradite", "pardon"
nlmed]
Table 10: Adaptive evolution of hybrid schema expansion on ACE2005 dataset. Underlined classes refer to father classes, which occurs when directly adding sub classes that corresponding major class not exists.

---

Webb also said details of the breakdowns of the Welches' previous marriages were likely to come up. and cited reports of alleged extramarital affairs by both.
Input:
Labels
"end position". "attack", "start position". "nominate", "charge indict", "transfer money", "transfer ownership", "release parole", "be born", "sen- tence", "die", "demonstrate", "arrest jail", "transport", "elect", "start organization",, "meet". "injure", "phone write", "merge organization", "declare bankruptcy", "trial hearing". "fine", "convict", "end organization", "sue", "divorce", "acquit", "appeal", "execute", "marry,, "extradite", "pardon"
Iteration schema:
Divoreo(breakdowns)
Marrymmarriags]
Iteration schema: "end position". "attack", "begin", "nominate", "charge indict", "transfer money", "transfer ownership", "release parole", "be born", "sentence", "die", "demonstrate", "arrest jail", "carry", "elect", "start organization", "meet", "injure". "phone write", "merge organization". "declare bankruptcy", "trial hearing". "fine", "convict", "end organization", "sue", "separate", "acquit", "appeal", "execute", "marry", "extradite", "pardon"
Separace(brcuklowns))
Marrymmarriags]
Iteration schema: "end", "attack", "begin", "nominate", "prosecute", "remittance", "transfer ownership", "release parole", "be born", "sentence", "die", "demonstrate", "arrest jail", "canyy", "clect", "start organization", "meet", "injure", "phone write". "merge organization", "declare bankruptcy", "trial hearing", "fine", "convict", "end organization", "sue", "separate", "acquit", "appeal", "execute", "marry", "extradite", "pardon"
Separate(breakdoons]] Marry[marriags]
Iteration 4 schema: "end",, "attack", "begin", "nominate", "prosecute", "remittance", Separate(breakklowss] "transfer ownership", "release parole", "be born". "sentence", "pass away", "demon- [mariaae] strate", "arrest jail", "carry". "elect", "start organization", "meet", "injure", "phone write", "merge organization", "declare bankruptcy", "attend the trial", "fine", "con- vict", "end organization", "sue", "separate", "acquit", "appeal", "perform". "marry". "extradite", "pardon"
Iteration schema: "end", "attack", "begin", "nominate", "prosecute", "remit-
Separate(rreuklowns] Marry[marriags]
tance". "tranffer ownership", "release parole", "be born", "condemn", "pass away", "demonstrate", "arrest jail", "carry", "elect", "start organization", "encounter", "in- jure", "phone write", "merge organization", "go out of business", "attend the trial". "fine", "convict", "end organization", "sue", "separate", "acquit". "appeal", "perform". "marry", "extradite", "pardon"
"end", "attack", "begin", "nominate", "prosecute", "remittance", Separate(breakkdonns] "giveaway", "release parole". "be born", "condemn", "pass away", "parade", "arrest jail", "carry", "vote", "start orgamization", "encounter", "injure", "phone write". "merge organization", "go out of business", "attend the trial", "fine", "convict", "end organization", "sue", "separate", "acquit", "appeal", "perform", "marry", "extradite", "pardon"
Iteration 6 schema:
Marry(marriage])
Iteration 7 schema: "end", "attack", "begin", "nominate", "prosecute", "remittance", Separace(breakklonss) "giveaway", "release parole", "be born", "condemn", "pass away". "parade", "arrest jail", "carry", "vote", "start organization", "encounter", "hurt", "communication", "merge organization", "go out of business", "attend the trial", "fine", "convict", "end organization", "sue", "separate", "acquit", "appeal", "perform", "wed", "extradite". "pardon"
[ed[maris
Table I: Adaptive evolution of analogous schema expansion on ACE2005 dataset.

---

GPT-3.5 Input Example:
There are some relation extraction samples, relation must be taken from schema, head entity and tail entity must be taken from context. Relation, head entity and tail entity may have multiple. schema:
l"people", "country", "religion", "major sharcholder of", "industryy", "contains", "brith place", "location", "nationality", "advisors", "neighborhood of", "place lived", "capital", "geo- graphic distribution", "teams", "major shareholders", "place of death", "children", "company", "profession", "place founded", "founders"
Context: In Queens, North Shore Towers, near the Nassau border, supplanted a golf course, and housing replaced a gravel quarry in Douglaston.
The relation involved in the above sentence 1. The head entity is Douglaston, relation is neighborhood of, tail entity Queens; 2. The head entity is Queens, relation is contains, tail entity is Douglaston.
Context: Martin, the district attorne for Lehigh County in Pennsylvania, said that after his office's review of the records, was satisfied with Mr. Cullen's denials.
The relation involved in the above sentence are: 1. The head entity is Pennsylvania, relation is contains, tail entity is Lehigh County.
Context: Mr.Brown has demeaned Mr.Bush as "a cheerleader," " declared that Homeland Security Secretary Michael Chertoff did not know "the first thing about running disaster," and called critics like Representative Gene Taylor, Democrat of Mississippi, "a little twerp" and Senator Norm Coleman, Republican of Minnesota, an unprintable vulgarity (both in Playboy).
The relation involved in the above sentence are: 1. The head entity is Gene Taylor, relation is place lived, tail entity is Mississippi.
Do you understand how to do relation extraction based on schema? Now it's your turn to do relation extraction.
schema:
l"people", "country", "religion", "major shareholder of", "industry", "contains", "birth place", "location", "nationality", "advisors", "neighborhood of", "place lived", "capital", "geo- graphic distribution", "teams", "major sharcholders", "place of death", "children", "company", "profession". "place founded", "founders"]
Context: But that spasm of irritation by a master intimidator was minor compared with what Bobby Fischer, the erratic former world chess champion, dished out in March at a news conference in Reykjavik, Iceland.
The relation involved in the above sentence are:
GPT-3.5 Output Example: The relation involved in the above sentence are: The head entity is Bobby Fischer, relation is place lived, tail entity is Iceland; 2. The head entity is Iceland, relation is contains, tail entity is Reykjavik; 3. The head entity is Iceland, relation is capital, tail entity Reykjavik.
Golden Output Example: The relation involved in the above sentence are: 1. The head entity is Bobby Fischer, relation is nationality, tail entity is Iceland; 2. The head entity is Iceland, relation is capital, tail entity is Reykjavik; 3. The head entity is Iceland, relation is contains, tail entity Reykjavik; The head entity Bobby Fischer, relation is place of death, tail entity is Reykjavik.
Table 12: Examples of GPT-335 experiment on NYT dataset. A total of 20 demonstrations are given to the model.

---

HU
There are some relation extraction samples, relation must taken from schema, head entity and tail entity must be taken from Context. relation, head entity and tail entity may have multiple.
schema: ("people", "country", "religion", "major shareholder of", "industry", "contains", "brith place", "location", "nationality", "advisors", "neighborhood of", "place lived", "capital", "geographic distribution", "teams", "major shareholders", "place of death", "children", "company"]
Context: Queens, North Shore Towers, near the Nassau border, supplanted a golf course, and housing replaced gravel quarry in Douglaston.
The relation involved in the above sentence are: 1.The head entity is Douglaston, relation is neighborhood of, tail entity Queens;2.The head entity Queens, relation contains, tail entity is Douglaston;
Context: his authoritative and tough-minded new book, The Assassins Gate America in Iraq, the New Yorker writer George Packer reminds us that the decision of the Bush administration to go to war against Iraq and its increasingly embattled handling of the occupation were both predicated upon large, abstract ideas about the role of America in the post-cold war world most notably, belief pre-emptive and unilateral action, the viability of exporting democracy abroad, the urge streamline the military and the dream of remaking the Middle East
The relation involved in the above sentence are: 1. head entity is Middle East, relation is contains, tail entity is 1raq;2.The head entity is Middle East, relation is contains, tail entity is Iraq;
Context: Martin, the district attorney for Lehigh County Pennsylvania, said that after records, was satisfied with Mr. Cullen denials. The relation involved in the above sentence are: 1.The head entity is Pennsylvania, relation is contains, tail entity is Lehigh County;
his office review
Context: Mr. Brown has demeaned Mr. Bush as cheerleader, "declared that Homeland Security Secretary Michael Chertoff did not know the first thing about running disaster and called critics like Representative Gene Taylor, Democrat of Mississippi, little twerp and Senator Norm Coleman, Republican of Minnesota, an unprintable vulgarity both in Playboy )
The relation involved in the above sentence are: 1.The head entity is Gene Taylor, relation is place lived, tail entity is Mississippi;
Context: Pakistan 's president, Pervez Musharraf, and President Hamid Karzai talked by telephone for an hour on Tuesday, a day after the Afghan intelligence service said arrested three Pakistanis who were trying to assassinate the American ambassador, Zalmay Khalilzad... "
The relation involved in the above sentence are: 1.The head entity is Pervez Musharraf, relation is nationality, tail entity is Pakistan;
Figure 7: ChatGPT input example NYT dataset. A total of 20 demonstrations are given the model.

---

HU
Do you understand do relation extraction based on schema?
Now it's your do relation extraction.
("people", "country", "religion", "major shareholder of", "industry", "contains", "brith place", "location", "nationality", "advisors", "neighborhood of", "place lived", "capital", "geographic distribution", "teams", "major shareholders", "place of death", "children", "company"]
schema:
Context: But that spasm irritation master intimidator minor compared with what Bobby Fischer, the erratic former world chess champion, dished out in March at news conference in Reykjavik, Iceland
The relation involved in the above sentence are:
The head entity Bobby Fischer, relation is people, entity former world chess champion
The head entity Reykjavik, relation location, tail entity Iceland.
Do you understand how do relation extraction based schema? Now it's your turn to do relation extraction.
["people", "country"; "religion", "major shareholder of", "industry", "contains", "brith place", "location", "nationality", "advisors", "neighborhood of", "place lived", "capital", "geographic distribution", "teams", "major shareholders "place of death", "children", "company", "profession", "place founded", "founders']
schema:
Context: But that spasm of irritation by master intimidator was minor compared with what Bobby Fischer, the erratic former world chess champion, dished out March at news conference in Reykjavik, Iceland..
The relation involved in
sentence are:
The head entity Bobby Fischer, relation is people, tail entity is former world chess champion.
2. The head entity is Bobby Fischer, relation is profession, tail entity is chess player.
The head entity Reykjavik, relation location, tail entity Iceland.
4. The head entity is Bobby Fischer, relation is place lived, tail entity is Iceland (assuming
Bobby Fischer lived Iceland at time of news conference)
The head entity is Bobby Fischer, relation is place founded, tail entity is United States (assuming Bobby Fischer was born in the United States)
Figure 8: ChatGPT output results under horizontal schema expansion iteration.


Reviews:
[{"review_id": "00uN6JvXEk", "reviewer": "Reviewer_hsKx", "summary": "", "questions": "1) What is $S$ exactly?\n2) Are your results statistically significant?\n3) What are the runtimes of your approaches, especially compared to that of the reference approaches?\n4) Why do you only compare with those three baselines?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for vour comments and feedback, In addition to the general updates. we address vour concerns here:\n> Q1: What is $S$ exactly?\n\nIn the context of the paper, the term \"schema\" represented by the $S$ typically refers to the structure and organization of the knowledge graph, including the types of entities, relations, and events that are part of the graph. It encompasses the underlying framework that defines how different pieces of information are connected in the graph. Specifically, we adopt the same definition as UIE, which utilizes a specific pre-defined schema as the structural schema instructor. An innovation is that the schema we propose is dynamically constructed, demonstrated on Table(8-11) with 4 types of expansion patterns. We will provide a formal definition or explanation for the schema denoted by  $S$. We are sorry for the ambiguity of the expression and will correct these defects.\n\n> Q2: Are your results statistically significant?\n\nFor each iteration experiment, we report the average performance over 3 random seeds. We conduct a pair-wise t-test and the p-values are less than 0.005\n\n> Q3: What are the runtimes of your approaches, especially compared to that of the reference approaches?\n\nThe paper does not explicitly mention the specific runtimes of the approaches mentioned (Text2Event, TANL, UIE, GPT-3.5, and ADAKGC) or provide a direct comparison of their runtimes. Since real-time capability is not the highlight that we want to highlight in our paper. Intuitively, AdaKGC's runtime does not differ significantly from those baseline.\n\n> Q4: Why do you only compare with those three baselines?\n\nText2Event, TANL, UIE, and GPT-3.5 are chosen due to their relevance to knowledge graph construction tasks. Additionally, these methods could have been widely discussed and benchmarked in previous literature, making them suitable for evaluation against the proposed schema-adaptable KGC approach, AdaKGC. A detailed description of them is shown in Section 2.4 Vanilla Baselines (pages 3-4).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The problem is highly relevant to real-life applications. The paper is easy to understand (up to some details in the equations) even if it does contain a number of typos. The evaluation is also clear.", "reasons_to_reject": "The authors use the schema $S$ in equations without formally defining what that is. Note that the definition of schema uses a cursive S and small letters for instances. It is really unclear how that is to be understood but I guess it's an easy fix. There is no statistical test, hence it is unclear whether the performance of AdaKGC is really superior to that of the state of the art. I would assume that it is the case but would really like to see a corresponding test carried out.", "questions_to_authors": "1) What is $S$ exactly?\n2) Are your results statistically significant?\n3) What are the runtimes of your approaches, especially compared to that of the reference approaches?\n4) Why do you only compare with those three baselines?", "paper_topic_and_main_contributions": "The paper addresses the challenge of schema-adaptable knowledge graphc completion (which is really knowledge extraction in this context). The fundamental premise of the paper is that the ontology of a domain might change over time. This entails that being able to adapt to changes in the ontology is necessary for the application of knowledge extraction in the real world. The authors presents novel datasets to evaluate the performance of existing extraction approaches and present a novel approach dubbed AdaKGC. In their evaluation setting, the authors refrain from retraining the underlying models. The authors compare their approach with UIE, TANL and TEXT2EVENT. The results suggest that their approach perform at least as well as the state of the art. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "EUudiHsyeu", "reviewer": "Reviewer_rg33", "summary": "", "questions": "1. If I understand the proposed task \"schema-adaptable KGC\" correctly, the model is only trained once with the initial schema and then evaluated on the evolving schema over different iterations. In lines 289-298, the authors introduce a three-step training strategy. May the authors confirm that this strategy is only used for the initial training? Also, the authors do not explain the rationale behind this strategy nor ablate its effect. It would be better if the authors can provide some analysis of this strategy.\n2. In lines 323-325, the authors mention that \"we constrain the model to generate the type tokens consistently with the existing schema at the type location.\" I am wondering how this is implemented. Is this something as simple as constraining the decoding vocabulary? If yes, it would be better to explicitly state it in the paper, and it is a common practice and not a novel contribution.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for vour comments and feedback, In addition to the general updates. we address vour concerns here:\n\n> Q1: If I understand the proposed task \"schema-adaptable KGC\" correctly, the model is only trained once with the initial schema and then evaluated on the evolving schema over different iterations. In lines 289-298, the authors introduce a three-step training strategy. May the authors confirm that this strategy is only used for the initial training? Also, the authors do not explain the rationale behind this strategy nor ablate its effect. It would be better if the authors can provide some analysis of this strategy.\n\nYes, that's correct.  In the proposed task of \"schema-adaptable KGC\", the model is trained initially with a initial schema.  However, the novelty lies in its ability to adapt to dynamically changing schemas during evaluation without requiring re-training.  This means that the model is evaluated on new or evolving schemas in different iterations without undergoing a full training process again.  The focus is on the model's capability to extract entity, relation, and event information from the changing schema over time.\n\nThe three-step training strategy introduced in lines 289-298 is indeed applied only during the initial training phase.  This strategy is designed to facilitate the convergence of our model's parameters by progressively incorporating different types of objectives( Prefix instructor, Schema-specific instructor, LM parameters). This strategy comes from our intuition that along the stepwise task learning or curriculum learning, task prefix, schema prefix, and full parameter training need to have a reasonable training sequence.\n\n> Q2: In lines 323-325, the authors mention that \"we constrain the model to generate the type tokens consistently with the existing schema at the type location.\" I am wondering how this is implemented. Is this something as simple as constraining the decoding vocabulary? If yes, it would be better to explicitly state it in the paper, and it is a common practice and not a novel contribution.\n\nEnsuring the model can generate type tokens consistently with the existing schema involves constraining the decoding process. Thus, constraining the decoding vocabulary is an effective method where we construct vocabulary based on the dynamically changing Tire-Tree. It is indeed simple in terms of innovation and we will state it in the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper studies an interesting and important task, which has great practical value.\n2. The authors conduct extensive experiments and provide necessary analyses to demonstrate the effectiveness of the proposed model.", "reasons_to_reject": "1. The writing of the paper can be improved, for example, the description of the experimental setting and model details is not clear enough, which leads to some confusion (see the questions below).\n2. The technical novelty of the paper is limited as the two proposed components seem to be very straightforward.", "questions_to_authors": "1. If I understand the proposed task \"schema-adaptable KGC\" correctly, the model is only trained once with the initial schema and then evaluated on the evolving schema over different iterations. In lines 289-298, the authors introduce a three-step training strategy. May the authors confirm that this strategy is only used for the initial training? Also, the authors do not explain the rationale behind this strategy nor ablate its effect. It would be better if the authors can provide some analysis of this strategy.\n2. In lines 323-325, the authors mention that \"we constrain the model to generate the type tokens consistently with the existing schema at the type location.\" I am wondering how this is implemented. Is this something as simple as constraining the decoding vocabulary? If yes, it would be better to explicitly state it in the paper, and it is a common practice and not a novel contribution.", "paper_topic_and_main_contributions": "This paper proposes a new task named schema-adaptable knowledge graph construction, whose goal is to perform information extraction with an evolving schema without re-training the model. The authors design three types of schema evolution (horizontal, vertical, and hybrid) and build task-specific benchmarks using existing datasets. They also propose a new model, named AdaKGC, which includes two new components: schema-enriched prefix instructors and a schema-conditioned dynamic decoding strategy, addressing the challenges of schema evolution. The empirical results on three IE tasks (NER, RE, and EE) across three datasets demonstrate the effectiveness of the proposed model.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "fyH9RR7oIO", "reviewer": "Reviewer_n6r2", "summary": "", "questions": "* A. How large a schema can be reasonably processed via ADAKGC, considering it has to be supplied in input to the LM? Is this a limitation to discuss in the corresponding section of the paper?\n\n* B. For vertical expansion, when selecting a super-class C1 but not one of its sub-classes C2, are train/dev/test examples for C2 (e.g., mentions of C2 in the text, relations involving mentions of C2, etc) included among corresponding train/dev/test examples to the super-class C1? I would expect so, in order to guarantee completeness of text annotations w.r.t. class C1 (the rationale is that an occurrence in text of a mention/relation/etc for class C2 is *also* an occurrence for class C1).", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for vour comments and feedback, In addition to the general updates. we address vour concerns here:\n\n> Question A: How large a schema can be reasonably processed via ADAKGC, considering it has to be supplied in input to the LM? Is this a limitation to discuss in the corresponding section of the paper?\n\nThe size of the schema that can be reasonably processed via ADAKGC depends on the maximum input length that LLMs, such as GPT-3.5, can handle.  GPT-3.5 has a maximum token limit, and both the schema-enriched prefix instructor and schema-conditioned dynamic decoding need to fit within this limit along with the rest of the input text.   In the corresponding section of the paper, we will acknowledge this limitation and discuss it as a consideration for the practical application of ADAKGC.  We will mention that the maximum token limit of the LM imposes a constraint on the complexity and size of the schema that can be utilized effectively.  Additionally, we will discuss potential strategies for handling larger schemas, such as prioritizing key schema elements or implementing techniques for schema summarization.   It's important to highlight that while there is a limitation on the schema size, ADAKGC's effectiveness in handling evolving schemas is still significant and relevant for scenarios where the schema is dynamically changing or evolving over time.\n\n> Question B: For vertical expansion, when selecting a super-class C1 but not one of its sub-classes C2, are train/dev/test examples for C2 (e.g., mentions of C2 in the text, relations involving mentions of C2, etc) included among corresponding train/dev/test examples to the super-class C1? I would expect so, in order to guarantee completeness of text annotations w.r.t. class C1 (the rationale is that an occurrence in text of a mention/relation/etc for class C2 is also an occurrence for class C1).\n\nYou are correct in your expectation.  In the vertical schema expansion, when selecting a super-class C1, train/dev/test examples for its sub-classes (like C2) are indeed included among the corresponding train/dev/test examples for the super-class C1. Since an occurrence in the text of a mention, relation, or other relevant information for class C2 can also be considered as an occurrence for class C1, including these examples in the context of C1 helps capture the broader context and potential relationships between different classes in the schema hierarchy.\n\n> Typos Grammar Style And Presentation Improvements\n\nWe appreciate your attention to detail about typos, grammar, style, and presentation of the paper. We will thoroughly review the paper again to address the issues you've highlighted and ensure that it meets the standards of clarity and professionalism.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* S1. Interesting and adequately motivated new task that, along with the supplied framework / benchmark / baseline, may spark interest in further work in this direction\n\n* S2. Good reproducibility, with involved code/data resources made available to reviewers as supplemental material\n\n* S3. Sufficient experimental validation, considering multiple KG construction sub-tasks and settings (Tab. 1,2,3) and including ablation tests (Fig 5), error analysis (Tab. 4), check for catastrophic forgetting (Fig. 6) and additional tests of GPT-3.5 / ChatGPT and of a different \"analogous\" schema expansion form in the Appendix.", "reasons_to_reject": "* W1. Improvements of ADAKGC over UIE are mostly marginal, this happening also after several schema evolution iterations, i.e., the setting for which ADAKGC is specifically designed for (differently from UIE); besides this, the currently assessed performance loss incurred with schema evolution appear substantial, raising some concern about the practical feasibility of the task.\n\n* W2. Except for schema-conditioned decoding, the architecture of ADAKGC appears to me close to the one of UIE and thus not particularly novel; besides, I wonder about the feasibility of encoding the whole schema in the input of the language model, which seems to me may pose a limit to the size of the schema that can be used in practice (see question A).\n\n* W3. Definitions and examples of horizontal and vertical expansions appear a bit unclear. Can sub-relations be introduced as well for vertical expansion? The paper mentions and appear to focus on sub-classes, but the displayed example (Fig. 2) is actually about a sub-relation (\"capital\", refining \"contains\") and this kind of evolution would seem natural to me as well. Similarly, for horizontal expansion, what does it mean exactly for the new class to be \"at the same level\"? It cannot have a super-class as otherwise it would be a vertical expansion, so I imagine it has to be a new top-level class. And when adding it, can it be related to existing classes through some existing relation? All these details, along with what to do with train/test/dev examples of super-classes(/relations?) when doing vertical expansions, should be rigorously defined in the paper in order to delimit the kind of expansions considered and tested in this work.", "questions_to_authors": "* A. How large a schema can be reasonably processed via ADAKGC, considering it has to be supplied in input to the LM? Is this a limitation to discuss in the corresponding section of the paper?\n\n* B. For vertical expansion, when selecting a super-class C1 but not one of its sub-classes C2, are train/dev/test examples for C2 (e.g., mentions of C2 in the text, relations involving mentions of C2, etc) included among corresponding train/dev/test examples to the super-class C1? I would expect so, in order to guarantee completeness of text annotations w.r.t. class C1 (the rationale is that an occurrence in text of a mention/relation/etc for class C2 is *also* an occurrence for class C1).", "paper_topic_and_main_contributions": "SUMMARY: The paper defines the task of \"schema-adaptable knowledge graph (KG) construction\", whose aim is to extract <typed head entity, relation, typed tail entity> KG triplets from natural language text when the target schema (ontology TBox) changes in time with the addition of new classes/properties lacking training instances, and without re-training/tuning of the extraction model. A methodology to derive datasets for this task is provided along with datasets derived from FEW-NERD, NYT, ACE2005. A dedicated 'baseline' model for the task, named ADAptive Knowledge Graph Construction (ADAKGC) and based on pre-trained T5 language model fed with <continuous soft prompt, current schema encoded, input text>, is proposed and evaluated over such datasets against other state-of-the-art approaches (TANL, TEXT2EVENT, UIE) that might be used for the task, showing marginal improvements over UIE in most settings.\n\nCONTRIBUTIONS:\n\n* C1. Definition of \"schema-adaptable KG construction\" task along with related concepts (e.g., horizontal/vertical/hybrid schema expansion), dataset derivation and evaluation methodologies;\n\n* C2. Provision of a concrete evaluation benchmark for the new task derived from FEW-NERD, NYT, ACE2005, coming with data and/or code where data cannot be shared (e.g., ACE2005);\n\n* C3. Proposal of the ADAKGC extraction model tailored to the new task for which it sets strong baseline performance, coming along with code ensuring reproducibility.\n\nPOST-REBUTTAL UPDATE. I thank the authors for the clarifications in their rebuttal. After reading it and considering the other reviews as well, I confirm the original scores in my review.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Schema expansion in §2.3 and Fig. 2 should be better introduced: in Fig. 2, \"(people, profession, person)\" looks weird (replace class \"person\" with class \"job\"?) and it's also unclear the difference between classes \"people\" and \"person\"; in §2.3 the addition of sub-class nodes is mentioned (e.g., \"father\") but Fig. 2 shows the addition of sub-relations (e.g., \"capital\").\n\n* T1.  [§1] \"adapt dynamic schema\" -> \"adapt to\"\n* T2.  [§2.1, references] check entry \"(Expressions, 2005)\" (wrong author)\n* T3.  [§2.2] \"training paramerer on\" -> \"trained on\"\n* T4.  [§2.3] \"without new classes as training data\" -> \"instances\"\n* T5.  [§2.3] \"and except for\" -> \"besides\"\n* T6.  [Alg. 1] please introduce the meaning of \"raw\" (whole original dataset?), of \"W_c\" and \"W_s\" (clear only after checking the appendix)\n* T7.  [§3.1] \"object\" -> \"objective\"\n* T8.  [Eq. 2] what is \"Y_index\"?\n* T9.  [§3.3] \"token by the token\" -> remove \"the\"\n* T10. [§4.4] \"bellows\" -> \"below\"\n* T11. [Tab. 4] \"Meet[visited]\" appears to refer to mention \"visited\" that do not appear in the corresponding \"Input Example\"\n* T12. [Limitations] \"the proposed\", \"suffer\", \"schema-datable\" -> \"the proposed one\", \"suffers\", \"schema-adaptable\"\n* T13. [Alg. 2,3,4 in appendix] seems to me that selection of \"n_init\" nodes cannot simply be \"random\", if a proper, connected hierarchy of classes has to be extracted to start with"}}]

Author rebuttal:

Meta-review: The paper proposes a new task named schema-adaptable knowledge graph construction to enable the model to handle emerging knowledge without retraining the IE models. This work is applicable in real-world scenarios. The authors have studied changes in the schema in three dimensions: horizontal, vertical, and hybrid, and they have built task-specific benchmarks using existing datasets.

The proposed model is evaluated on three IE tasks (NER, RE, and EE) across three datasets and compared to four baselines, achieving similar scores as the baselines. However, the t-test indicates that the results are statistically significant with a p-value < 5%.

Paper is easy to follow with typos and grammar style errors.
Decision: Accept-Findings