Text2NKG: Fine-Grained N-ary Relation Extraction for N-ary relational Knowledge Graph Construction
Authors: Haoran Luo
Haihong E
Yuhao Yang
Tianyu Yao
Yikai Guo
Zichen Tang
Wentai Zhang
Shiyao Peng
Kaiyang Wan
Meina Song
Wei Lin
Yifan Zhu
Anh Tuan Luu
Venue: NeurIPS 2024 poster (2024)
Keywords: N-ary Relation Extraction
N-ary relational Knowledge Graph
Knowledge Graph Construction
Primary area: natural_language_processing
TL;DR: We introduce Text2NKG, a novel fine-grained n-ary relation extraction framework for n-ary relational knowledge graph construction.

Abstract: Beyond traditional binary relational facts, n-ary relational knowledge graphs (NKGs) are comprised of n-ary relational facts containing more than two entities, which are closer to real-world facts with broader applications. However, the construction of NKGs remains at a coarse-grained level, which is always in a single schema, ignoring the order and variable arity of entities. To address these restrictions, we propose Text2NKG, a novel fine-grained n-ary relation extraction framework for n-ary relational knowledge graph construction. We introduce a span-tuple classification approach with hetero-ordered merging and output merging to accomplish fine-grained n-ary relation extraction in different arity. Furthermore, Text2NKG supports four typical NKG schemas: hyper-relational schema, event-based schema, role-based schema, and hypergraph-based schema, with high flexibility and practicality. The experimental results demonstrate that Text2NKG achieves state-of-the-art performance in F1 scores on the fine-grained n-ary relation extraction benchmark. Our code and datasets are publicly available.

Text2NKG: Fine-Grained N-ary Relation Extraction for N-ary relational Knowledge Graph Construction
Haoran Luo', Haihong E Yuhao Yang', Tianyu Yao', Yikai Guo Zichen Tang', Wentai Zhang', Shiyao Peng', Kaiyang Wan', Meina Song', Wei Lin", Yifan Zhu', Luu Anh Tuan
School of Computer Science, Beijing University of Posts and Telecommunications, China School of Automation Science and Electrical Engineering, Beihang University, China Beijjng Institute of Computer Technology and Application "Inspur Group Co., Ltd., China CColleee of Computing and Data Science, Nanyang Technological University, Singapore (ouhaoran, shaihong, yi fan_zhu @bupt. edu. cn, anhtuan. luuênt wdu..eeduus.....
Abstract
Beyond traditional binary relational facts, n-ary relational knowledge graphs (NKGs) are comprised of n-ary relational facts containing more than two enti- ties, which are closer real-world facts with broader applications. However, the construction of NKGs remains at coarse-grained level, which is always in a single schema, ignoring the order and variable arity of entities. To address these restrictions, we propose Tex12NKG, a novel fine-grained n-ary relation extraction framework for n-ary relational knowledge graph construction. We introduce a span-tuple classification approach with hetero-ordered merging and output merging accomplish fine-gras grgiied n-ary relation extraction in different arity. Further- more, Text2NKG supports four typical NKG schemas: ttyper-relatonal schema, event-based schema, role-based schema, and hypergraph--brr -assed based schema, with high flexibility and practicality. The experimental results demonstrate that Text2NKG achieves state-of-the-arr performance in Fi scores on the fine- fineggrineed n-ary relation extraction benchmark. Our code and datasets are publicly available
Introduction
Modern knowledge graphs (KGs), such as Free- base Google Knowledge Vault [7], and Wiki- data 21 utilize multi-relational graph struc- ture to represent knowledge. Because of the
Natural Language Text
N-ary relational Knowledge Graph relational fact
liaeid condred Dactaric Augrne Plysis fixe the to 1911, basis was Feefnoor bor Theeriteal Plosio
fee-graned
advantage of intuitiveness and interpretability. KGs find various applications in question an-
cxtroctie
Enerntiontooie
Baande
Prioe is Payoecia Sand
swering (28), query response []], logical rea-
1911
soning 4.. and recommendation systems [29] Traditional KGs mostly composed of binary relational facts (subject, relation, object), which represent the relationship between two
and expeaaly taraa thersrr
ae tan of Paseedecttcc
Poris
Photetate Sikel
n-ary
Sehi Dines n-ary relational fact_3
entities 3 However, it has been observed (20] that over 30% of real-world facts involve n-ary
Figure An example of NKG construction.
relation facts with more than two entities (n > 2). As shown in Figure | an n-ary relational knowl- edge graph (NKG) is composed of many n-ary relation facts, offering richer knowledge expression
Conresooningg author.
htttp:://itthb. wwwpLHRLAB/Tee......
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Natural Language Sentence:
Spon-tuple for Entities:
Einstain recuived his Ductorate dagree in Plysics from the Univensly of Zurich,
(Einstein, Daciorati, Physies, University of Zurich)
Answer Lobel- list for Relations:
Infocasion, pagoraatnes, pasloni,_fopeee
prove,, rollyys
Jelecation, olhosber.
Inhocalied
NKG schemos:
hyper-ndaaional schoma
event based schema
rolo- basedaaedd schoma
hyporgraph based schoma
Dinids
Unirenlly Zurkk
Foral Type
edhnnber Jeve
olecalive
mape
Uisherrty
Plyto
&laleesity of Zuric
saaj Physies
4ryee
Plysie
Dacteaate
scadaal Jegne
Bocturate
Plylle
Decisrate
Figure 2: Taking a real-world textual fact as an example, we can extract a four-arity structured span-tuple for entities (Einstein, University of Zurich, Doctorate, Physics) with an answer label-list for relations accordingly as a 4-ary relational fact from the sentence through n-ary relation extraction.
and wider application capabilities. As a key step of constructing NKGs, n-ary relation extraction (n-ary RE) is task of identifying n-ary relations among entities in natural language texts. Compared to binary relational facts, n-ary relational facts in NKGs have more diverse schemas for different sce- narios. For example, Wikidata utilizes n-ary relational facts in a hyper-relational schema 20 10 23
i... (s, r, o, {k,, 24) ? which adds (n 2) key-valus pairs to the main triple to represent aux- iliary information. In addition to the hyper-relational schema, the existing NKG schemas also
include wrent-based schema (r, {(k,, 0;) ] )) [6 role-based schema (((k,, 2;)
12 13)
and hypergraph-basee based schema (r, {u: 1) 226 8, as shown in Figure 2
Currently, most existing NKGs in four schemas, such as JF17K 126. Wikipeople 12]
WD50K [00
and EventKG [1I, are manually constructed. Previous n-ary RE methods [33 31 focus on extraction
with fixed number of entities in hypergraph-based schema or role-based schema. Existing event
extraction methods [16] 17 can achieve n-ary RE in event-based schema. Recently, CubeRE [5]
introduce a cube-filling method, which is the only n-ary RE method in hyper-relational schema.
However, there are still three main challenges in automated n-ary RE for NKG construction, which remains at a coarse-grained level: (1) Diversity of NKG schemas. Previous methods could only perform N-ary RE based on a specific schema, but eurrently, there is no flexible method that can perform n-ary RE for arbitrary schema with different number of relations. (2) Determination of
the order of entities. N-ary RE involves more possible entity orders than binary RE, for example, as shown in F2 in a hyper-relational schema, there is an order issue regarding which entity the head entity, tail entity, or auxiliary entity. Previous methods often ignored the joint impact of different entity orders, leading to inaccurate extraction. (3) Variability of the arity of n-ary RE. Previous methods usually output a fixed number of entities and are not adept at determining the variable number of entities forming n-ary relational fact.
To tackle these challenges, we introduce Tex12NKG, a novel fine- -raiigr n-ary RE framework designed to automate the generation of n-ary relational facts from natural language text for NKG construction. Tex12NKG employs a span-tuple multi-label classification method, which transforms n-ary RE into multi-label classification task for span-tuples, including all combinations of entities in the text. Because the number of predicted relation labels corresponds to the chosen NKG schema, Tex12NKG is adaptable to all NKG schemas, offering examples with hyper-relational schema, event-based schema, role-based schema, and hypergraph--asse schema, all of which have broad applications. Moreover, Text2NKG introduces hetero-ordered merging method, considering the probabilities of predicted labels for different entity orders to determine the final entity order. Finally, TexI2NKG proposes output merging method, which is used to unsupervisedly derive n-ary relational facts of any number of entities for NKG construction.
In addition, we extend the only n-ary RE benchmark for NKG construction, HyperRED []. which is in the hyper-relational schema, to four NKG schemas. We've done sufficient n-ary RE experiments on HyperRED, and the experimental results show that Tex12NKG achieves state-of-the--rt performance in Fj scores of hyper-relational extraction. We also compared the results of Text2NKG in the other three schemas to verify applications.
2

---

2 Related Work
2.1 N-ary relational Knowledge Graph
An n-ary relational knowledge graph (NKG) consists of n-ary relational facts, which contain entities (n 2 2) and several relations. The n-ary relational facts are necessary and cannot be replaced by combinations of some binary relational facts because cannot distinguish which binary relations are combined to represent the n-ary relational fact in the whole KG. Therefore, NKG utilizes a schema in every n-ary relational fact locally and a hypergraph representation globally [1]]
Firstly, the simplest NKG schema is hypergraph- -aagad...-addd..... (26) found that over 30% of Freebase 2 entities participate facts with more than two entities, first defined n-ary relations mathematically
and used star-to-clique conversion to convert triple-based facts representing n-ary relational facts into the first NKG dataset JFI7K in Hypergraph-bbusee schema (r, {vx y 1) [8] proposed FB-AUTO and M-KB15K with the same hypergraph-buce for n-ary relational facts and extracted Wikipeople, the first NKG dataset in role-based schema (E(k, u/) H==)), 2). composed of rolu-value pairs. Thirdly, Wikidata (21) the largest knowledge base, utilizes an NKG schema based on hyper-relation (s, 0, {k;, vi );));=??, which adds auxiliary key- value pairs to the main triple. [0] first proposed an NKG dataset in hyper-relational schema WD50K. Fourthly, as M pointed out, events are also n-ary relational facts. One basic event representation has event type, trigger, and several key-value pairs [16]. Regarding the event type as the main relation, the (trigger: value) as one of the key-value pairs, and the arguments as the rest key-value pairs, can obtain event-based NKG schema (r, {(ki, 02) )=)]==)
schema. Secondly, [12] introduced role information
Based on four common NKG schemas, we propose Tex12NKG, the first method for extraction of structured n-ary relational facts from natural language text, which improves NKG representation and application.
2.2 N-ary Relation Extraction
Relation extraction (RE) an important step of KG construction, directly affecting the quality, scale, and application of KGs. While most of the current n-ary relation extraction (n-ary RE) for NKG construction depends on manual construction 266 12 [0] but not automated methods. Most automated RE methods target the extraction of traditional binary relational facts. For example. [22] proposes a fable-filling method for binary RE, and 30 27 propose span-based RE methods with levitated marker and packed levitated marker, respectively.
For automated n-ary RE, some approaches role-based schemu as a binary classification problem and predict whether the composition of n-ary information a document is valid or not. However, these methods extract n-ary information in fixed arity, which are not flexible. Moreover, some event extraction methods [16] 17 9 propose different event trigger and argument extraction techniques, which can achieve n-ary RE in event-based schema. Recently, CubeRE (55) proposes automated n-ary RE method in hyper-relational schema, which extends the table-filling extraction method to n-ary RE with cube-filling. However, these methods only model one of the useful NKG schemas with limited extraction accuracy.
13 treat n-ary RE hypergraph-buxed schema or
In this paper, we propose the first fine- aineggrineed n-ary RE framework Tex12NKG for NKG construction in four example schemas, proposing a span-tuple multi-label classification method with hetero- ordered merging and output merging to improve the accuracy fine-grained n-ary RE extraction in all NKG schemas substantially.
3 Preliminaries
Formulation of NKG. NKG G = (E, R, F} consists of an entity set E, a relation set R, and an n-ary fact (n>2) set F. Each n-ary fact fr E F consists of entities E and relations E R. For
hyper-relational schema (20:
(r where {er)% E E, {ri) 1- E R. f% = (ri, (rit1, )1) where {ei] E E, (ra)+t E R. For role-
f = (,1,
For event-based schema [6]
based schema [12] f
((ri, (1)) where {ei) E (re] E R. For hypergraph-basee
schema [26]: fhg (F1, |2 )2 where {e,) E.rre R.
3

---

Natural Language Sentence:
Einstein received his Doctorate degree Plysics fress the University of Zuriih.
Input:
10
Elnstein
[S]
received his DDctorate degree
PPhysics from University Zurich
Bert-based Encoder
Entity_ I
Enclty __
Entity_
Eniliy: .4
Suan-tuple for Entities:
Answer Label-
for Relations:
3 3-ory relational focts:
Eattiy I
Eaiity_2
Eniliy_3
acadomic degrev
acualemic majjor
[Einsicis, othontte Las University aosdewic dorns Dachorad)
(Fantteis, odvensed on University of Zarkh. ncsdenic major, Plysicl
Euriit
Eailey
Eatlity
ecadomic majio
acudemiic deggioo
L Output Merging
Endity
Eatley
Entity
acadomic major
olncated
Mutti-label
Hetero-cree eeter--ordered
Earity_ _I
Fatity 4
Emiity 2
adurated__r at Classification
prademic degres
Merging
n-ary relational facts:
University Zarich
Emiity_!
Earitt
Farity
acadomic mujir
rducaated
Eatity 1
Entity
Eucity
educatiol at
acudeanic major
aradenn degiee Dactonale
Figure 3: An overview of Tex12NKG extracting n-ary relation facts from natural language sentence in hyper-relational NKG schema for an example.
Problem Definition. Given input sentence with 1 words {ww,, u2,..., Wt entity is consecutive span of words: = [wpp Wp+1.-.., W³y E &g, where p, q E (1....1), and E, = {cg 1 is the entity set of all m entities in the sentence. The output of n-ary relation extraction, RO,, is a set of n-ary relational facts F, in given NKG schema in (/hr Le0> fīg) Specifically, each n-ary relational fact f F3 is extracted by multi-label classification of one of the ordered span-tuple for n entities [e]]]- E Es, forming an answer label-list for n, relations [r:] E R, where n is the arity of the extracted n-ary relational fact, and n, is the number of answer relations in the fact, which is determined by the given NKG schema: R[(e]]] = [r when f fhy R8[ [[(es]]==))
|ri] when f = Sev- [((e,)== = [ra]==] when f = fro. and R((e;)]=1) [ri] when f Shhg
4 Methodology
In this section, we first introduce the overview of the Tex12NKG framework, followed by the span- tpple multi-label classification, training strategy, hetero-ordered merging, and output merging.
4.1 Overview of Text2NKG
Text2NKG is fine-grained n-ary relation extraction framework built for n-ary relational knowledge graph (NKG) construction. The input Text2NKG natural language text tokens labeled with entity span in sentence units. First, inspired by [27). Tex12NKG encodes the entities using BERT-baa Encoder [6] with a packaged levitated marker for embedding. Then each arrangement of ordered span-tuple with three entity embeddings will be classified with multiple labels, and the framework will be learned by the weighted cross-entropy with a null-label bias. In the decoding stage, in order filter the n-ary relational facts whose entity compositions have isomorphic hetero-ordered characteristics. Tex2NKG proposes a cederr- merging strategy to merge the label probabilities of 31 arrangement cases of span-tuples composed of the same entities and filter out the output 3-ary relational facts existing non-conforming relations. Finally, Tex12NKG combines the output 3-ary relational facts to form the final n-ary relational facts with output merging.
4.2 Span-tuple Multi-label Classification
For the given sentence token {wi, W2, .... wr and the set of entities E., in order to perform fine- grained n-ary RE, we need first to encode a span-tuple (e1, e2, e3) consisting of every arrangement of three ordered entities, where e1, E E.. Due to the high time complexity of training every

---

span-tuple as one training item, inspired by (27]. we achieve the reduction of training items by using packed levitated markers that pack one training item with each entity in &, separately. Specifically, in each packed training item, pair of solid tokens, [S] and [/S], are added before and after the packed entity es = { Way ) and ([E,) 1) pairs of levitated markers, [1.] and [/L.], according to other entities in Es. are added with the same position embeddings as the beginning and end of their
corresponding entities span EL, = {"p,,,".. 1z } to form the input token X:
= [S], pss Wys 1/S], ..... Wpz; [4], ... Waz, U / .... wr).
We encode such token by the BERT-hased pre-trained model encoder [6]
(h1:12,- he) BERT(X);
where / X is the input token length, (me E Rd, and d is embedding size.
There are several span-tuples (A, B, C) in a training item. The embedding of first entity h E Rdd in the span-tuple is obtained by concat embedding of the solid markers, [S] and [/S], and the embeddings of second and third entities hB. hc E R2d are obtained by concat embeddings of levitated markers, [L] and [/L] with all - arrangement of any other two entities in & Thus, obtain the embedding representation of the three entities form A A span-tuples in one training item. Therefore, every input sentence contains training items with m.A m-1 A span-tuples for any ordered arrangement of three entities.
We then define πt linear classifiers, each of which consists of 3 feedforward neural networks {FNN In, Ji=1, k = 1, 2, 3, to classify the span-tuples for multiple-label classification. Each classifier targets the prediction of one relation rg, thus obtaining probability lists (P,) with all relations in given relation set R plus a null-label:
P. FNN! (hA) (hB) + FNN,
where FNN E (22x((R +1), and P; E R(R[+1)
4.3 Training Strategy
To train the n, classifiers for each relation prediction more accurately, we propose data augmentation strategy for span-tuples. Taking the hyper-relational schema as an example, given hyper-relational fact (A, ; B, r2, C),, consider swapping the head and tail entities, and changing the main relation to its inverse (B, ri A. r2, C), as well as swapping the tail entities with auxiliary values, and the main relation with the auxiliary key (A, r2, C,ri, B), also as labeled training span-tuple cases. Thus Ra (A, B, C) (2), can be augmented with 3!- 6 orders of span-tuples:
RAy [A,
B, C*)
(r1,
RAy (B,A, (rī), r2).
Rer (A,C, B) (12,71).
RAr R (B,C,A) = =(r2,TT))
Rer AC, = (r2),m)
Rar (C, (, (r1,12)).
For other schemas, we can also obtain 6 fully-arranged cases of labeled span-tuples in a similar way, described in Appendix [A] If no n-ary relational fact exists between the three entities of span-tuples, then relation labels are set as null-label.
Since most cases of span-tuple are null-label, we set weight hyperparameter E (0, 1] between the null-label and other labels to balance the learning of the null-label. We jointly trained the n, classifiers for each relations by cross-entropy loss L with null-label weight bias W.:
exp (P.
C
Wa log
(5)
R| exp (Pt))
1=1
where W. [a, 1.0, 1.0, 1.0) E ( +1),
5

---

All
Train
Dev
Text
Dataset
#Ent
#R_hr hr
#R_ev
#RR_ ro
#R__gg bg
#SSentence
#Fact
VSentence
#Flact
#Sentence
#Faett
#Sentence
#Fact
HyperRED
40,293
106
232
168
62
44,840
45,994
39,840
39,978
1,000
1,220
4,000
4.796
Table 1: Dataset statistics, where the columns indicate the number of entities, relations with four schema, sentences and n-ary relational facts in all sets, train set, dev set, and test set, respectively.
4.4 Hetero-ordered Merging
In the decoding stage, since Tex12NKG labels all different arrangement of the same entity com- position, design a hetero-ordered merging strategy merge the corresponding labels of these 6 hetero-ordered span-tuples into one to generate non-repetitive n-ary relational facts unsupervisedly. For eoyee-relationa schema (n, 2), combine the predicted probabilities of two labels P1, P2 in orders to (A, B, C)) order as follows:
Pi P (ABC) + [(P1 (BAC)) + P (ACB)
(BA)) P (CAB) + (CBA)
(6)
P (ABC) ((BAC) + +P(ACB)
P2
+P (BCA) [[P
+ 1 (CBA)) (C
where I() is a function for swapping the predicted probability of relations and the corresponding inverse relations. Then, we take the maximum probability to obtain labels Ti, 12, forming a 3-ary relational fact (A, rı, B, r2, C) and filter it out if there are null-label in (r1, 72). If there are inverse relation labels (r1,72) we can also transform brent-based schema, role-based schema, and hapergraph-baseer schema, all can be generated by hetero-ordered merging according to this idea, as shown in Appendix
order of entities and relations as equation [ For
4.5
Output Merging
After hetero-ordered merging, we merge the output 3-ary relational facts to form higher-arity facts, with hyper-relational schema based the same main triple, event-based schema based on the same main relation (event type), role-based schemu based on the same key-value pairs, and hypergraph- based schema based on the same hyperedge relation. This way, we can unsupervisedly obtain n-ary relational facts with dynamic number of arity numbers for NKG construction. More details are discussed in Appendix G.2 and Appendix G.3
Experiments
This section presents the experimental setup, results, and analysis. We answer the following research questions (RQs): RQ1: Does Text2NKG outperform other n-ary RE methods? RQ2: Whether Tex12NKG can cover NKG construction for various schemas? RQ3: Does the main components of Text2NKG work? RQ4: How does the null-label bias hyperparameter in Tex12NKG affect performance? RQ5: Can Text2NKG get complete n-ary relational facts in different arity? RQ6: How is Tex12NKG's computational efficiency? RQ7: How does Tex12NGG perform in specific case study? RQ8: What is the future development of Text2NKG in the era of large language models?
5.1 Experimental Setup
Datasets. The existing fine-grained n-ary RE dataset HyperRED (5) only in hyper-relational schema with annotated extracted entities. Therefore, expand the HyperRED dataset four schemas as standard fine-grained n-ary RE benchmarks and conduct experiments on them. The statistics of the HyperRED with four schemas shown in Table TalleTann and the construction detail is in Appendix[C]
Pipeline Baseline 24 and CubeRE (5] fine-grained n-ary RE task of hyper-relational schema. For n-ary the other three schemas, compared Tex1NKGG with event extraction models such as vext2Event [16), UIE [17].
Baselines. We compare Text2NKG against Generative Baseline 14

---

hyper-relational schema / Dev
lyper-relational schema / TTes Recall
Model
PLM
Precision
Recall
Precision
Unsupervised Method
CharGPT
gpi 3.5 turbo gpt-4
12.0583
11,2764
11,6522
11,021
10,9134 15,4824
11.1524
GPT-4
15.7324
15.2377
15.4811
15.8187
15.6487
Supervised Methodl
Generative Baseline
63.79:0.22
59.94 2.0.
61.80 2037
64.60 2047
59.67 20.35
62.05 2021
Pipelinge Baselime
0:10
58,21 10.57
63.24 2.44
57.55 20.19
62.75
CabeRE
66.14 a.
64.39 1:21
65.23 0082
65.82 1s8
64.28 102
65.04 10:29
TexIZNKG wio DA
BERT-base (110M)
76.02 0.50
72.28 20:6
74.10 2
73.55 1051
70.63 1::0
72.06 103
TexI2NKG w/o
88.77 aas5
78.39
81.26 0.30
88.09
76.64
81.97 10:58
Tex22KKG w/o HM
G1.74+033
76.97 -0:41
68.52 n.
61 07 u:73
76. 16:16
67.72 0:48
TeX22KG (ours)
91.26
79.36 2.51
84,89
90.77
77.53
83.63
Generative Baseline
67.08 4.44
65,73 -077
66.40 0.47
67.17 0.40
6..5
65.84 1025
Pipelinge Baseline
70.58 0.78
66.58 000
68.52
69.3 20.55
64.27 =0.24
66.65 20
larg- (340M)
CabeRE
68.75
68.88
68.81
66.39
62.12.0m
66.75 10
Tex12NKG (ours)
91.90 8.79
79.43 9,42
85.21 0.00
91.06 1881
77.64
83.81 10
Table 2: Comparison of Tex12NKG with other baselines in the hyper-relational extraction on HyperRED. Results of the supervised baseline models are mainly taken from the original paper [5] The best results in each metric are in bold.
event-based schema
role-based schema
hypergraph- schema Rocall
Model
PLM
Precision
Rocall
Fj
Procision
Recall
F.
Precision
Fi
Unsupervised Method
ChaiGPT
g-t-3.5-murbo gpt-4
10.4678
11,1628
10.8041
114487
10.4203
1039058
11,2998
11,7852
11,5373
GPT4
13.3681
14.6701
13.9888
13.6397
12,5355
13.0643
13.0907
13,6701
13,3741
Supervised Method
Text2Fivett
73.94 1.7 70.56
10.8
1:25
72.73 1.
1.34
70.52 nss
73.68 1ass
70.37
71,98 1.90
UIE
TS-base (220M)
76.51 +028 73.02 2000
74.72 -.11
72.17 20.9 69.84
70.98 n.31
-2.03
68.74
2013 70.34
LasUIE
79.62=
=027 78.04
2075
78.82
20.36
77.01 20220 74.26=025
75.61 +0.24
76.21 20.07
73.75
20.77
=4.9
Tex12NKG BERT-base (I (0M)
86.20
79.25
82,58
0.23
86.72
78.94
82.64
83.53
86.59
85.03
Texi2Event
75.58 0.5 72.39
1082 73.97
1.19
73.21
1...5 70.85
266: 72.01 20.31
75.28 a.89 72.73
am 73.98 1...0
UIR
Targ (770M)
79.38 +.76 74.69
-m6t 76.96
74.47 1.44 71.84
-0m 73.14-0..
74.57
4.64 71.93 -0. 73.22
LasUIB
81.29 =083 79.54
0.266 80.40
=:065
79.37
0002 76.63
-044
77.97
0.77%
77.49
0.35 74.96
20000 76.20
TeX2NKGG BERT-large (340M) 88.47 20.5
80.30
2.75 84.19
86.87
2.87 2.229
83.76 1117
85.06 2.33 86.72 29.66 85.89 2.09
Table 3: Comparison of TeX2NKGG with other baselines in the n-ary RE in event-based, role-based, and hypergraph- based schemas on HyperRED. The best results in each metric are in bold.
and LasUIE [9. Furthermore, utilized different prompts to test the currently most advanced large-scale pre-trained language models ChatGPT (25] and GPT-4 [19] in an unsupervised manner, specifically for the extraction performance across the four schemas. The detailed baseline settings can be found in Appendix D
Ablations. To evaluate the significance of Tex12NKG's three main components, data augmentation (DA), null-label weight hyperparameter (ar), and hetero-ordered merging (HM), we obtain three simplified model variants by removing any one component (Text2NKG w/o DA, Text2NKG w/o o, and TeXt2NKG w/o HM) for comparison.
Evaluation Metries. We use the F1 score with precision and recall to evaluate the dev set and the test set. For a predicted n-ary relational fact to be considered correct, the entire fact must match the ground facts completely.
Hyperparameters and Enviroment. We train 10 epochs HyperRED using the Adam optimizer. All experiments were done on a single NVIDIA A100 GPU, and all experimental results were derived by averaging 5 random seed experiments. Appendix B shows Tex12NKG's optimal hyperparameter settings. Appendix F shows training details.
5.2 Main Results (RQ1)
The experimental results of proposed Tex12NKG and other baselines with both BERT-base and BERT-large encoders can be found in Table 2 for the fine-grained n-ary RE in hyper-relational schema. We can observe that Tex12NKG shows a significant improvement over the existing optimal model CubeRE on both the dev and test datasets of HyperRED. The F1 score is improved by 19.66 percentage points in the dev set and 18.60 percentage points in the test set with the same BERT-base encoder, and 16.40 percentage points in the dev set and 17.06 percentage points in the test set with
7

---

Rocall
Accuracy (%)
Namber
Accuracy
Preciiion
Recall
390
takI
Epoch
Epoch
Hyperpurameter
(a)
(b)
(C)
Figure 4: (a) Precision, Recall, and P1 changes the dev set during the training of Tex12NKG. (b)
The changes of the number of true facts, the number of predicted facts, and the number of predicted accurate facts during the training of Tex22KGG (c) Precision, Recall, and F results on different null-label hyperparameter (a) settings.
the same BERT-large encoder, reflecting Tex12NKG's excellent performance. Figure 4(a) and (bb intuitively show the changes of evaluation metries and answers of facts in the dev set during the training of Text2NKG. It is worth noting that Tex12NKG exceeds 90% in precision accuracy. which proves that the model can obtain very accurate n-ary relational facts and provides a good guarantee for the quality of fine-grained NKG construction.
5.3 Results on Various NKG Schemas (RQ2)
As shown in Table 3, besides hyper-relational schema, Tex12NKG also accomplishes the tasks of fine-grained n-ary RE in three other different NKG schemas on HyperRED, which demonstrates good utility. In the added tasks of n-ary RE for event-based role-based and stypergraph-bate saahh schemas, since no model has done similar experiments at present, we used event extraction or unified extraction methods such as Text2Event [16]. UIE [17), and LasUIE [9] for comparison. Text2NKG still works best these schemas, which demonstrates good versatility.
5.4 Ablation Study (RQ3)
Data augmentation (DA), null-label weight hyperparameter (a), and hetero-ordered merging (HM) are the three main components of Text2NKG. For the different Text2NKG variants as shown in Table 2 DA, a, and HM all contribute the accurate results our complete model. By comparing the differences, we find that HM is most effective by combining the probabilities of labels of different orders, followed by DA and
5.5 Analysis of Null-label Weight Hyperparameters (RQ4)
We compared the effect for different null-label weight hyperparameters (a). As shown in Figure [[c the larger the a, the greater the learning weight of null-label compared with other lables, the more relations are predicted as null-label. After filtering out the facts having null-label, fewer facts are extracted, so the precision is generally higher, and the recall is generally lower. The smaller the more relations are predicted as non-null labels, thus extracting more n-ary relation facts, so the recall is generally higher, and the precision is generally lower. Comparing the results of Fi values different a, is found that a 0.01 works best, which can be adjusted in practice according to specific needs to obtain the best results.
5.6 Analysis N-ary Relation Extraction in Different Arity (RQ5)
Figure 5(a) shows the number of n-ary relational facts extracted after output merging and the number of the answer facts different arity during training of Text2NKG on the dev set. We find that, as the training proceeds, the final output of Tex12NGG converges to the correct answer in terms of the number of complete n-ary relational facts each arity, achieving implementation of n-ary RE in indefinite arity unsupervised, with good scalability.

---

3od
(Ro, cdlontedl as, Pecklington Schael,
Number
1943)]
He was born Skärpenheck, York Pocklington School from
jolerated at, [edlsoteed ot efocated at Pook Tockiliggis Srihod, rinus 1963 diar 1980]
18'
Text2NKG
schema
Hedocated timic Pockington start
hypergraph based schema
Epoch
1943)]
(h)
Figure 5: (a) The changes of the number of extracted n-ary RE in different arity, where "pred nn" represents the number extracted n-ary facts with different arities by Tex12NKG, and "ans aan_n" represents the ground truth. (b) Case study of Text2NG''s n-ary relation extraction in four schemas on HyperRED.
5.7 Computational Efficiency (RQ6)
As mentioned in Section 4.2 the main computational consumption of Tex12NKG is selecting every span-tuple of three ordered entities to encode them and get the classified labels in multiple-label classification part. If we adopt an traversal approach with each span-tuple in one training items, the time complexity will O(m²). ). To reduce the high time complexity of training every opan-tuple as one training item, Tex22NKG uses packed levitated markers that pack one training item with each entity in & separately. We obtain the embedding representation of the three entities to form A span-tuples one training item. Every input sentence contains m training items with m m-1 A span-tupy ppan-tuples for any ordered arrangement of three entities for multiple-label classification. Therefore, the time complexity decreased from O(n²)) to O(m).
5.8 Case Study (RQ7)
Figure 5(b) shows case study of n-ary RE by trained Tex12NKG. For sentence, "He was born in Sk rpenbeck, near York and attended Pock lin.", four structured n- ary RE can be obtained by Tex12NKG according to the requirements. Taking the hyper-relational schema for example, Tex12NKG can successfully extract one n-ary relational fact consisting of main triple [He, educated at, Pocklington, and two auxiliary key-value pairs (start time: 1 936), (end time: 1943). This intuitively validates the practical performance of Tex12NKG on fine-grained n-ary RE to better contribute to NKG construction.
5.9 Comparison with ChatGPT (RQ8)
As shown in Table 2 and Table 3 we compared the extraction effects under four NKG schemas of the supervised TexI2NKG with the unsupervised ChatGPT and GPT-4. We found that these large language models cannot accurately distinguish the closely related relations relation repository, resulting in their Fl scores ranging around 10%-1555 15%. which is much lower than the performance of Text2NKG. On the other hand, the limitation of Tcxt2NKG that its performance is confined within the realm of supervised training. Therefore, in future improvements and practical applications, we suggest combining small supervised models with large unsupervised models to balance solving the cold-start and fine-grained extraction, which is detailed in Appendix G.1 G.1
fine-grained NKG
Conclusion
In this paper, we introduce Tex12NKG, novel framework designed for fine- ome-grained n-ary relation extraction (RE) aimed at constructing N-ary Knowledge Graphs (NKGs). Our extensive experiments demonstrate that Text2NKG outperforms all existing baseline models across wide range of fine- grained n-ary RE tasks. Notably, it excels in four distinct schema types: hyper-relational, event- based, role-based, and hypergraph-based. Furthermore, we have extended the HyperRED dataset, transforming into comprehensive fine-graines n-ary RE benchmark that supports all four schemas.

---

Acknowledgments
This work supported by the National Science Foundation of China (Grant No. 62176026, Grant No. 62406036, Grant No. 62473271, and Grant No. 62076035). This work is also supported by the SMP-Zhipu.Al Large Model Disspppicillliaay Fund, the BUPT Excellent Ph.D. Students Foundation (No. CX2023133) the BUPT Innovation and Entrepreneurship Support Program (No. 2024-YC-A091 and No. 2024-YC-T022), and the Engineering Research Center of Information Networks, Ministry of Education.
References
[1] Erik Arakelyan, Daniel Daza, Pasquale Minervini, and Michael Cochez. Complex query answering with neural link predictors. In International Conference on Learning Representations, 2021.
[2] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD '08, page 1247-1250, New York, NY, USA, 2008. Association for Computing Machinery.
[3] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
[4] Xuelu Chen, Ziniu Hu, and Yizhou Sun. Fuzzy logic based logical query answering on knowledge graphs. Proceedings of the AAAI Conference on Artificial Intelligence, 36(4):3939- 3948, Jun. 2022.
[5] Yew Ken Chia, Lidong Bing. Sharifah Mahani Aljunied, Luo Si, and Soujanya Poria. dataset for hyper-relational extraction and sube-filling approach. In Yoav Goldberg. Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing, pages 10114-10133, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
[6] Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[7] Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to proba- bilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '14, page 601-610, New York, NY, USA, 2014. Association for Computing Machinery.
[8] Bahare Fatemi, Perouz Taslakian, David Vazquez, and David Poole. Knowledge hypergraphs: Prediction beyond binary relations. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAP'00, 2021.
[9] Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Fei Li, Libo Qin, Meishan Zhang, Min Zhang, and Tat-Seng Chua. Lasuie: Unifying information extraction with latent adaptive structure-awar generative language model. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 15460-15475. Curran Associates, Inc., 2022.
[10] Mikhail Galkin, Priyansh Trivedi, Gaurav Maheshwari, Ricardo Usbeck, and Jens Lehmann. Message passing for hyper-relational knowledge graphs. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNIP)), pages 7346-7359, Online, November 2020. Association for Computational Linguistics.
10

---

[11]
Saiping Guan, Xueqi Cheng, Long Bai, Fujun Zhang, Zixuan Li, Yutao Zeng, Xiaolong Jin, and Jiafeng Guo. What event knowledge graph: survey. IEEE Transactions on Knowledge and Data Engineering, pages 1-20, 2022.
[12] Saiping Guan, Xiaolong Jin, Yuanzhuo Wang, and Xueqi Cheng. Link prediction on n-ary relational data. In The World Wide Web Conference, WWW '19, page 583-593, New York, NY, USA, 2019. Association for Computing Machinery.
[13] Robin Jia, Cliff Wong, and Hoifung Poon. Document-level n-ary relation extraction with multiscale representation learning. In Proceedings of the 2019 Conference the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Papers), pages 3693-3704, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[14] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequee pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7886 Online, July 2020. Association for Computational Linguistics.
[15] Yu Liu, Quanming Yao, and Yong Li. Role-aware modeling for n-ary relational knowledge bases. In Proceedings of the Web Conference 2021, WWW 221, page 2660-2671, New York, NY, USA, 2021. Association for Computing Machinery.
[16] Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, and Shaoyi Chen. Text2Event: Controllable sequence-to--trruture generation for end-to- end event extraction. In Proceedings of the 59th Anmual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ( (Volume I: Long Papers), pages 2795 2806, Online, August 2021. Association for Computational Linguistics.
[17] Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. Unified structure generation for universal information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 5755-5772, Dublin, Ireland, May 2022. Association for Computational Linguisties.
[18] Haoran Luo, Haihong E, Yuhao Yang, Yikai Guo, Mingzhi Sun, Tianyu Yao, Zichen Tang, Kaiyang Wan, Meina Song, and Wei Lin. HAHE: Hierarchical attention for reletonllationa knowledge graphs in global and local level. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computa- tional Linguistics (Volume I: Long Papers), pages 8095-8107, Toronto, Canada, July 2023. Association for Computational Linguistics.
[19] OpenAl. Gpt-4 technical report, 2023.
[20] Paolo Rosso. Dingqi Yang, and Philippe knowledge graph embedding for link prediction. In Proceedings of The Web Conference 2020, WWW '20, page 1885-1896, New York, NY, USA, 2020. Association for Computing Machinery.
Beyond triplets: Hyper-relational
[21] Denny Vrandečić and Markus Krötzsch. Wikidata: A free collaborative knowledgebase. Commun. ACM, 57(10))78-88 85, sep 2014.
[22] Jue Wang and Wei Lu. Two are better than one: Joint entity and relation extraction with table- sequence encoders. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1706-1721, Online, November 2020. Association for Computational Linguisties.
[23] Quan Wang, Haifeng Wang, Yajuan Lyu, and Yong Zhu. Link prediction on n-ary relational facts: A graph-based approach. In Findings of the Association for Computational Linguistics: ACI-IJCNIPP 2021, pages 396-407, Online, August 2021. Association for Computational Linguistics.
11

---

[24] Yijun Wang, Changzhi Sun, Yuanbin Wu, Hao Zhou, Lei Li, and Junchi Yan. UniRE: A unified label space for entity relation extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 220-231, Online, August 2021. Association for Computational Linguistics.
[25] Xiang Wei, Xingyu Cui. Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, Yong Jiang, and Wenjuan Han. Zero-shot information extraction via chatting with chatgpt, 2023.
[26] Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, and Richong Zhang. On the representation and embedding of knowledge bases beyond binary relations. In Subbarao Kambhampati, editor, Proceedings of the Twenty-Fisth International Join! Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pages 1300-1307. ICCALAAAI Press, 2016.
[27] Deming Ye, Yankai Lin, Peng Li, and Maosong Sun. Packed levitated marker for entity and relation extraction. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 4904-4917, Dublin, Ireland, May 2022. Association for Computational Linguistics.
[28] Scott Wen-tau Yih, Ming- Wei Chang, Xiaodong He, and Jianfeng Gao. Semantic parsing via staged query graph generation: Question answering with knowledge base. In Proceedings of the Joint Conference of the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing of the AFNLP. ACL Association for Computational Linguistics, July 2015. Outstanding Paper Award.
[29] Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, page 353-362, New York, NY, USA, 2016. Association for Computing Machinery.
[30] Zexuan Zhong and Danqi Chen. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 50-61, Online, June 2021. Association for Computational Linguistics.
[31] Yuchen Zhuang, Yinghao Li, Junyang Zhang, Yue Yu, Yingjun Mou, Xiang Chen, Le Song, and Chao Zhang. ReSel: N-ary relation extraction from scientific text and tables by learning to retrieve and select. In Yoav Goldberg, Zomitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 730-744, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
12

---

Appendix
A Supplement to Data Augmentation
In addition to the hyper-relational schema, the data augmentation strategies for other schemas are as follows:
For event-based schema, given an event-based fact (r1, T2, A, r3. B, r4, C), we consider keeping the main relation J1 unchanged, and swapping other key-value pairs, {r2, A}, (ra, B3, and {ra, C), positionally, also as labeled training span-tuple cases. Thus Re ( (A, B, C*) (T1, T2, Y3, T4) can be augmented with 6 orders of span-tuples:
(A, = (21,72,73,,0 ra),
Reo (B,,A, (11,73.72,74),
(e,(A,C,B) = (1i.72,Ta,73);
Reo (B, A) (11773)
(C, = (2,
Ree B, A) (1,774,73,722.
For role-based schema, given role-based fact (r1, A, r2, B, Ta. C), consider swapping key-value pairs, {r1, A), (r2, B), and (r3, C)), positionally, also as labeled training span-tuple cases. Thus Rro A, B, C) (ri, 12, ra) can be augmented with 6 orders of span-tuples:
(r, [, = (r1, r3);
A, C) (B,
(12,T1,73),
[A, C, B) = (T1,Ta,72),
(8)
( C, (r2,T3,T1):
(C, A, B) (ra,Ti,T2).
Rro B, A) (13,72.71).
For hypergraph-basee schema, given hypergraph- based fact (r1, A, C), consider keeping the main relation r unchanged, and swapping entities, A, B, and C, positionally, also as labeled training span-tuple cases. Thus Rag B, C) (ri) be augmented with 6 orders of span-tuples:
Rhg [A, A, = (n))
Rag (B,ACC (r)),
Rhg A,C,B) (r1),
(9)
Rhg (B (r1),
Rog (C,A, B) = (ri),
Rhg (C, B, (r1).
B Supplement to Hetero-ordered Merging
In addition the hyper-relational schema, the hetero-ordered merging strategies for other schemas
follows:
13

---

For event-baseds schema (nr
4), we combine the predicted probabilities of four labels
P1, P2, P3. P4 orders (A, B, C) order follows:
P P ((BC)) + ((BAC) + + (ACB)
+ +P +P(CA) + | (CAB + +P (CBA)
P2 P (ABC) +P (BAC)
+ (ACB)
+P4 (BCA) P (CAB) -P(BA)
(10)
P3 = (ABC)
3 +P2 (BAC)
+ P4 (ACB)
+ +P2 (BCA)))
+ +P1 (CAB)
+ +P3 (CBA))
P4 = PP=P(ABC)) +p(BAC)
(BAC) P (ACB)
+ +P (BCA)
P (CBB) P (CBA)
Then, we take the maximum probability to obtain labels T1, 22, 13, T4, forming a 3-ary relational fact (r1, 72; A, B, T4: () and filter it out if there are null-label in (r1, r2, ra, 14).
For role-based schema (n, = 3), we combine the predicted probabilities of three labels P1, P2, Ps in orders to (A, B, C) order follows:
P1 P (ABC)
+ + (BAC)
+ -PP. (ACB)
+ +P(CAA) + (CAB)
+P + +P3 +P (CBA)
P2 P PlABO (ABC) + (BAC) + +P (ACB)
(11)
+P (BCA) + +PS (CAB)
3 + +P (CBA))
P3 P (ABC)) H (BAC)
(ACB)
+ +P( (BCA)
P (CAB) (CBA)
Then, we take the maximum probability to obtain labels v1, Ts, forming a 3-ary relational fact (r',, A, T2, B,ra, ra, C) and filter it out there are null-label (2,,72,7 ra).
For hyyeegraahh-bsee schema (n, = 1), we combine the predicted probabilities of one label P1 in orders (A, B, C") order as follows:
P = P( (ABC). + (B(AAC) + (ACB))
(12)
+P(BCA) (CAB) + +p (CBA).
Then, take the maximum probability obtain labels r1. forming a 3-ary relational fact (r1, A, B, C) and filter it out if rı is null-label.
C Construction of Dataset
Based on the original hyper-relational schema HyperRED dataset (5). we construct other three schemas (event-based, role-based, and hypergraph-based for fine-grained n-ary RF. Firstly, we view the main relation in the hyper-relational schema the event type in the event-based schema, combine the head entity and tail entity with two extra head key and tail key to convert them into two key-value pairs, and remain the auxiliary key-value pairs in the hyper-relational schema. Taking "Einstein received his Doctorate degree in Physics from the University of Zurich. as an example, can be represented as (Einstein, educated, University of Zurich, {academic iaciiirmaarrraa Physics), facademic_ degree, Doctorate }) in the hyper-relational schema and (education, {trigger, received), (person, Einstein), (college, University of Zurich), (acudemic_major Physics), Physics).(ccuddmmic (academic_dess _degree, Doctorate)) in the event-based schema. Secondly, we remove the event type in the event-hased schema obtain the role-based schema. Thirdly, we remove the keys key-value pairs and remain the relation to build the hypergraph-basse schema.
14

---

D Baseline Settings
Firstly, for the original hyper-relational schema of HyperRED, we adopted the same baselines as in the CubeRE paper [ to compare with Tex1NKG:
Generative Bascline: Generative Baseline uses BART
14 sequence-to-sequenee model, to
transform input sentences into a structured text sequence.
to extract relation triplets in the first stage and a span extraction model based on BERT-Tagger extract value entities and corresponding qualifier labels in the second stage.
Pipeline Baseline: Pipeline Baseline uses UniRE 24
CubeRE: CubeRE 5 is the only hyper-relational extraction model that cube-filling model inspired by table- fillingiiingg approaches and explicitly considers the interaction between relation triplets
and qualifiers.
Secondly, for the event-based schema, role-based schema, and hypergraph-based schema, we added the following baselines to further validate the effect of Text2NKG on the fine ine-grained N-ary relation fact extraction task in the HyperRED dataset:
Text2Event: Text2Event [5] is a classic model in the Event extraction domain. However, it is not applicable to extractions the hyper-relational schema. For the role-based schema extraction, we retained the key without referring to the main relation, while for the hypergraph-based schema extraction, we retained the main relation without referring the key to get the final result for comparison.
UIE/ LasUEE: UIE and LasUIE [9] are unified information extraction models that can handle most tasks like NER, RE, EE, etc. However, they are still only suitable for event extraction in the multi-relational extraction domain and are not applicable extractions of the hnper-relational schema. Therefore, we adopted the same approach as with Text2Event to compare with Text2NKG.
Thirdly, under the impact of the wave of large-scale language models brought about by ChatGPT traditional natural language processing tasks, we added unsupervised large models as baselines to compare with Text2NKG in the n-ary RE tasks of the four schemas.
ChatGPT GPT4:
Using different prompts, we tested the latest state-of-the-an large-scale pre- trained language models ChatGPT [25] and GPT-4 [19] in unsupervised manner, evaluating their performance on the extraction of the four schemas.
E
Hyperparameter Settings
We use the grid search method to select the optimal hyperparameter settings for both Tex12NKG with Bert-base and Bert-large. We use the same hyperparameter settings in Tex12NKG with different encoders. The hyperparameters that we can adjust and the possible values of the hyperparameters are first determined according to the structure of our model in Table Afterward, the optimal hyperparameters shown in bold.
Hyperparameter
HyperRED
a Train batch size Eval batch size Learning rate
{1.0, 0.01, 0.001}
{2,4. 8, 16} {1)
1e - 5. 2e-5, 5e -5 (128, 256, 512, 1024) (0.0. 0.1, 0.2, 0.3)
Max sequence length
Weight decay
Table Hyperparameter Selection.
F Model Training Details
We train 10 epochs on HyperRED with the optimal combination of hyperparameters. Text2NKG and all its variants have been trained on single NVIDIA A100 GPU. Using our optimal hyperparameter settings, the time required complete the training on HyperRED 4h with BERT-base encoder and 10h with BERT-large encoder.
15

---

G Further Discussions
G.1 How does ChatGPT perform in Fine-grained N-ary RE tasks?
We have tried to use LLM APIs such as ChatGPT and GPT to do similar n-ary RE tasks, i.c., prompting model input and output formats for extraction. The advantage of ChatGPT is that it can perform similar tasks in a few-shot situation, however, for building high-quality knowledge graphs, the performance and the fineness of the n-ary RE are much lower than Text2NKG. This is because ChatGPT is not good at multi-label classification tasks that contain less semantic interpretation. When the number of labels of relations in our relation collection is very large, we need to write a very long prompt to tell the LLM about our label candidate collection, which again leads to the problem of forgetting. Therefore, we have tried numerous prompt templates to enhance the extraction effect of ChatGPT, however, on fine-grained n-ary RE task, the best result of ChatGPT can only reach about 10% of Fi value on HyperRED, which is much lower than the result of 80%+ F1 value of Tex12NKG.
However, advanced LLMs such as ChatGPT are a good idea for training dataset generation for Tex12KGG in such tasks to save some manual labor only verify and correct the training items generated. For future work, we will continue our research in this direction and try to combine large language models with Text2NKG-like supervised models for automated fine- mee-grained n-ary RE for n-ary relational knowledge graph construction.
G.2
Why first Extracting 3-ary facts and then Merging them into N-ary Facts? ?
We use output merging to address the dynamic changes in the number of elements in n-ary relational facts. The atomic unit of an n-ary fact includes a 3-ary fact with three entities. For instance, in the hyper-relational fact (Einstein, educated Physics), the Text2NKG algorithm allows us to extract two 3-ary atomic facts: (Einstein, educated at, University of Zurich, degree: Doctorate degree) and (Einstein, educated at, University of Zurich, major: Physics). These are then merged based on the same primary triple (Einstein, educated at, University of Zuricc) to form a 4-ary fact. The same principle applies to facts of higher arities.
University of Zurich, degree: Doctorate degree, major:
As another example demonstrating the problem with merging binary relations: consider the statement "Einstein received his Bachelor's degree in Mathematics and his Doctorate degree in Physics." " When represented as binary relations, the facts become (Einstein, degree, Doctorate degree), (Einstein, major, Physics), (Einstein, degree, Bachelor), and (Einstein, major, Mathematics). With this representation, we cannot merge these binary relation facts effectively because there's 's no way to determine whether Einstein's doctoral major was Physies or Mathematics. This necessitates the use of NKG's n-ary relationship facts to represent this information, as seen in (Einstein, degree, Doctorate degree, major, Physics).
Therefore, using binary facts, we can't merge them into n-ary facts based on shared elements within these facts. On the other hand, using facts with four entities more makes it challenging effectively extract 3-ary atomic facts.
In Section 5.6 Figure 5()) we also analyzed the effects and detailed insights of unsupervised extraction of arbitrary-arity facts.
G.3 How Tex12NKG can address Long Contexts with Relations spread across Various Sentences ?
As long as the text to be extracted is lengthy piece with entities annotated, it can undergo long-form n-ary relation extraction. The maximum text segment size for our proposed method depends on the maximum text length that a transformer-based encoder can accept, such as Bert-base and Bert-large, which have maximum limit of 512. To extract from larger documents, we simply need to switch to encoders with larger context length, which all serve as the encoder portion of Tex12NKG and are entirely decoupled from the n-ary relation extraction technique we propose. This is one of the advantages of TeXt2NKG. Its primary focus is to address the order and combination issues of multi-ary relationships. We can scamlessly combine transformer encoder that supports long texts with Span-tuple Multi-label Classification to process n-ary relation extraction in long chapters.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction accurately reflect the paper's contributions and scope. The claims made are supported by the detailed methodology and experimental results sections. Tex12NGG demonstrates improvements in fine-grained n-ary relation extraction, supports multiple NKG schemas, and achieves state-of-the-art performance as claimed. The public availability of code and datasets further supports the paper's transparency and reproducibility.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses several limitations, including the constraint of supervised training, the suggestion to combine supervised and unsupervised models, and the handling of long contexts with appropriate encoders. These points are addressed in the comparison with large language models and in discussions about future work and handling long texts.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
17

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
The paper provides comprehensive set of assumptions and proofs for its theoretical results. These are detailed in both the main sections and the appendices. For instance, the hetero-ordered merging strategy and the output merging methodology explained with equations and detailed descriptions of the processes involved. Additionally, the assumptions for the span-tuple multi-label classification method and the handling of null-label weights are clearly stated, with mathematical formulations provided to support the theoretical claims.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper includes detailed information necessary to reproduce the main experimental results. It provides a comprehensive description of the datasets used, the baselines for comparison, the experimental setup, and the evaluation metrics. Additionally, the paper mentions the use of a single NVIDIA A A100 GPU and details the hyperparameters and training environment in the appendices. The inclusion of the link to the anonymous GitHub repository ensures that the code and datasets are accessible for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
18

---

reproducible, they While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provides open access to the data and code, including a link to an anonymous GitHub repository. It also includes detailed instructions how to reproduce the main experimental results, covering data access, preparation, and the specific commands and environment needed to run the experiments. These details are described in the supplemental material and appendices.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
pub c / gu
ss onPoli for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https? / //nips cc/ httpps..ccppubleee httppppbllllggcpp wttp//ui/////eeeee Cooe/SudeeSSnn Subeiibbm... ss i onoollloiee lcyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper specifies all the necessary training and test details, including data splits, hyperparameters, and the type of optimizer used. This information is provided in the main text and further elaborated the appendices, ensuring that the experimental setup fully transparent and reproducible.
Justification:
19

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The paper reports error bars and provides appropriate information about the statistical significance of the experiments. The factors of variability captured by the error bars are clearly stated, and the method for caleulating the error bars explained. The assumptions made and whether the enror bar represents the standard deviation or the standard error the mean are also clearly mentioned.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of eтrоrs is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides detailed information on the compute resources used for the experiments. It specifies that all experiments were conducted on single NVIDIA A100 GPU. The training details include the time required for training on HyperRED, which is hours with the BERT-base encoder and 10 hours with the BERT-large encoder. The optimal hyperparameter settings and the use of the Adam optimizer are also described, along with the number of epochs and batch sizes.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
20

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
The research described in the paper conforms to the NeurlPS Code of Ethics. We have ensured transparency, reproducibility, and ethical use the data and models. We have also made the code and datasets publicly available to support open science and reproducibility. There is no indication of any ethical violations or concerns regarding the research process outcomes.
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e-g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses potential positive societal impacts, such improving the quality of knowledge graphs and enabling more accurate information extraction from text. It also considers potential negative impacts, such the possibility of misuse of the technology for disinformation or unfair dccision-making We suggest that combining small supervised models with large unsupervised models could help mitigate some of these risks by improving the accuracy and robustness of the extraction process.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymeetn of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
21

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
The paper addresses the potential misuse of the proposed methods and models. It emphasizes the importance of responsible use and includes strategies to mitigate risks. We suggest combining small supervised models with large unsupervised models to balance solving the cold-start problem and fine-gratd seee-graied extraction. They highlight that unsupervised large language models like ChatGPT and GPT-4 cannot accurately distinguish closely related relations, which implies careful consideration for controlled use. They also recommend specific usage guidelines and restrictions to ensure safe and ethical deployment.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The paper properly credits the creators and original owners of the assets used. It cites the relevant datasets and models, including their sources and versions. We ensure that the licenses and terms of use explicitly mentioned and respected. This information is detailed in the references and the supplemental material, where the datasets and models used are listed along with their corresponding licenses.
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswithee curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
com/ dat aase has
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
22

---

Justification: The new assets introduced in the paper are well documented. The documen- tation includes details about the training data, model architecture, and usage instructions. We provide structured templates for communicating the dataset/code/mmodl details, includ- ing training procedures, licenses, limitations, and consent obtained from people whose data is used. This comprehensive documentation is available alongside the assets in the supplemental material and the anonymous GitHub repository. Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments research with human subjects. Therefore, this question is not applicable to the current work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing experiments or research with human subjects. Therefore, IRB approvals equivalent reviews are not applicable to the current work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "yfSk5rA5Tc", "reviewer": "Reviewer_rUhA", "summary": "The paper presents Text2NKG, a new framework designed for fine-grained n-ary relation extraction aimed at constructing n-ary relational knowledge graphs (NKGs). By introducing a span-tuple classification method combined with hetero-ordered merging and output merging strategies, it achieves extraction across varying entity arities while preserving their order, enhancing the granularity of NKG construction. Moreover, Text2NKG is adaptable to four prevalent NKG schemas.", "questions": "Please refer to the Weaknesses", "limitations": "The authors have addressed the limitations.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "**Validated Effectiveness**: Experimental results demonstrate the capability of Text2NKG in n-ary relation extraction.\n\n**Relevance and Significance**: This work focuses on a related challenge in knowledge graph construction. Its consistency with practical application requirements enhances the importance of research.", "weaknesses": "**Lack of Implementation Detail**: Vital specifics regarding input prompts to language models like GPT-4, and whether the Generative Baseline and Pipelinge Baseline underwent supervised fine-tuning, are omitted. These details are crucial for reproducibility and assessing the thoroughness of comparative analysis.\n\n**Inherent Limitations of BERT-style Architectures**: The reliance on BERT-based architectures may restrict the handling of long texts. This could limit the scalability and applicability of Text2NKG in domains with extensive contextual requirements.\n\n**Limited application scope**: Although Text2NKG supports 4 typical NKG schemas, the framework seems to be specifically designed for these specific structures, which may limit its generality in practical applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for a detailed review. We appreciate that you find our work effective for n-ary relation extraction and consist with practical applications. We answer your concerns below.\n\n---\n**For Weakness 1(i) (Vital specifics regarding input prompts to language models like GPT-4.):**\n\nOur ChatGPT and GPT-4 prompts both adopted a 1-shot unsupervised design. The specific input prompts (which we will add to the Appendix of the final paper) are as follows: \n\n```\nTask: Based on the relation_list and qualifier_list and the given input sentence and entity_list, output n-ary relational facts in hyper-relational schema.\n\nrelation_list=['adjacent station', 'award received', 'candidacy in election', 'capital of', 'cast member', 'chairperson', 'child', 'coach of sports team', 'connecting line', 'country', 'country of citizenship', 'director / manager', 'educated at', 'employer', 'followed by', 'head of government', 'head of state', 'headquarters location', 'home venue', 'incarnation of', 'instance of', 'league', 'legislative body', 'located in the administrative territorial entity', 'located on street', 'location', 'manufacturer', 'member of', 'member of political party', 'member of sports team', 'military branch', 'narrative role', 'noble title', 'nominated for', 'notable work', 'occupant', 'occupation', 'operator', 'original broadcaster', 'owned by', 'parent organization', 'part of', 'part of the series', 'participant', 'participating team', 'partner in business or sport', 'performer', 'place of birth', 'position held', 'present in work', 'replaces', 'residence', 'shares border with', 'significant event', 'sport', 'sports season of league or competition', 'spouse', 'stock exchange', 'subclass of', 'used by', 'voice actor', 'winner'] \n \nqualifier_list=['academic degree', 'academic major', 'adjacent station', 'affiliation', 'applies to part', 'character role', 'connecting line', 'country', 'diocese', 'electoral district', 'end time', 'follows', 'for work', 'has part', 'instance of', 'located in the administrative territorial entity', 'location', 'member of political party', 'mother', 'national team appearances', 'nominee', 'number of matches played/races/starts', 'number of points/goals/set scored', 'object has role', 'of', 'performer', 'point in time', 'position held', 'position played on team / speciality', 'publication date', 'quantity', 'ranking', 'replaces', 'series ordinal', 'sports league level', 'start time', 'statement disputed by', 'statement is subject of', 'street number', 'subject has role', 'ticker symbol', 'together with', 'towards', 'winner'] \n\nExample: \nInput: \nsentence='The current Leader of the National Party in the Parliament of Australia is Barnaby Joyce , and Deputy Leader is Fiona Nash , both elected on 11 February 2016 following the retirement of Warren Truss as Leader .' \nentity_list=['National Party', 'Barnaby Joyce', '11 February 2016', 'Warren Truss'] \nOutput: \n[(entity1='National Party', relation='chairperson', entity2='Warren Truss', qualifier1='end time', entity3='11 February 2016'), (entity1='National Party', relation='chairperson', entity2='Barnaby Joyce', qualifier1='start time', entity3='11 February 2016')] \n\nThen: \nInput: \nsentence='Apple was founded by Steve Jobs , Steve Wozniak , and Ronald Wayne on April 1 , 1976 , to develop and sell personal computers .' entity_list=['Apple', 'Steve Jobs', 'April 1 , 1976'] \nOutput: \n``` \n\n**For Weakness 1(ii) (Whether the Generative Baseline and Pipeline Baseline underwent supervised fine-tuning, are omitted.):**\n\nAs **Section 5.1** and **Appendix D** show, we adopted the same baselines as in the CubeRE paper to compare with Text2NKG. The settings and results of the Generative Baseline and Pipeline Baseline are both from that paper. Similar to CubeRE, Text2Event, UIE, and LasUIE, the Generative Baseline and Pipeline Baseline used supervised training methods. Additionally, as shown in **Table 2**, unsupervised and supervised methods are obviously distinguished.\n\n---\n**For Weakness 2 (The reliance on BERT-based architectures may restrict the handling of long texts.):**\n\nIn **Appendix G.3**, we have analyzed how Text2NKG can address long contexts with relations spread across various sentences. If the text to be extracted is a lengthy piece, it can undergo long-form n-ary relation extraction. The maximum text segment size for our proposed method depends on the maximum text length that a transformer-based encoder can accept, such as Bert-base and Bert-large, which have a maximum limit of 512. To extract from larger documents, we simply need to switch to encoders with larger context lengths, which all serve as the encoder portion of Text2NKG and are entirely decoupled from the n-ary relation extraction technique we propose. This is one of the advantages of Text2NKG. Its primary focus is to address the order and combination issues of multi-ary relationships. We can seamlessly combine a transformer encoder that supports long texts with Span-tuple Multi-label Classification to process n-ary relation extraction in long chapters.\n\n---\n**For Weakness 3 (Although Text2NKG supports 4 typical NKG schemas, the framework seems to be specifically designed for these specific structures.):**\n\nText2NKG's multi-label classification strategy and hetero-ordered merging strategy allow users to freely replace custom n-ary relation schemas. When users define an n-ary relational fact composed of n entities and m relations, they only need to set the corresponding number of FNNs for the output of m categories. Then, by using user-defined rules for hetero-ordered merging, custom fine-grained n-ary relation extraction can be quickly and flexibly implemented.\n\nPlus, the hyper-relational schema, event-based schema, role-based schema, and hypergraph-based schema cover nearly all schemas of n-ary relational knowledge graphs (NKGs). In **Section 2.1**, we provide a detailed summary of existing instances of NKGs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5yOUoxBFRv", "reviewer": "Reviewer_ED7F", "summary": "The paper introduces a novel framework for fine-grained n-ary relation extraction aimed at constructing n-ary relational knowledge graphs (NKGs). Traditional KGs primarily focus on binary relations, but this work targets n-ary relations which involve more than two entities, aligning more closely with real-world facts.", "questions": "1. Can you provide an example of extracting N-ary relations in real-world information extraction?\n2. How should NKG be stored and utilized, and can n-ary extraction be applied in RAG or Agent in the era of LLMs?", "limitations": "See the Weakness.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The writing of the paper is clear, well-structured. The logical flow from the problem definition to the proposed methodology and experimental results makes it easy to follow.\n2. The span-tuple classification and hetero-ordered merging approach are novel and effective, enabling the extraction of fine-grained n-ary relations which are more representative of real-world facts compared to traditional binary relations.\n3. Proposed method achieves state-of-the-art performance on the HyperRED benchmark, with significant improvements in F1 scores over existing methods.\n4. The paper compares Text2NKG with ChatGPT (gpt-3.5-turbo) and GPT-4 for N-ary extraction performance across the four schemas, and analyzes the advantages and disadvantages between large and small models for NKG construction.", "weaknesses": "1. Although the framework shows potential for scalability, the paper does not thoroughly address the computational efficiency and scalability when applied to real-time applications.\n2. While the paper includes ablation studies, more detailed analysis and discussion on the impact of each component (e.g., data augmentation, null-label weight hyperparameter) on different types of NKG schemas would provide deeper insights into their contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful evaluation of our paper. We are happy that you find our paper clear to follow and the results impressive. We respond to your concerns and questions below.\n\n---\n**For Weakness 1 (The paper does not thoroughly address the computational efficiency and scalability.):**\n\nAs described in **Section 3** and **Section 4.2**, given an input sentence with $l$ words $s=\\{w_1,w_2,...,w_l\\}$, an entity $e$ is a consecutive span of words: $e=\\{w_p,w_{p+1},...,w_q\\}\\in \\mathcal{E}_s$, where $p,q\\in\\{1,...,l\\}$, and $\\mathcal{E}_s=\\{e_1,e_2,...,e_m\\}$ is the entity set of all $m$ entities in the sentence. In order to perform fine-grained n-ary RE, we need first to encode a span-tuple ($e_1,e_2,e_3$) consisting of every arrangement of three ordered entities, where $e_1,e_2,e_3\\in \\mathcal{E}_s$. The main computational consumption of Text2NKG is selecting every span-tuple of three ordered entities to encode them and get the classified labels in multiple-label classification part. If we adopt an traversal approach with each span-tuple in one training items, the time complexity will be O($m^3$).\n\nTo reduce the high time complexity of training every span-tuple as one training item, Text2NKG uses packed levitated markers that pack one training item with each entity in $\\mathcal{E}_s$ separately. Every input sentence contains $m$ training items with span-tuples for any ordered arrangement of three entities for multiple-label classification. Therefore, the time complexity decreased from O($m^3$) to O($m$).\n\n---\n**For Weakness 2 (More detailed analysis and discussion on the impact of each component on different types of NKG schemas would provide deeper insights into their contributions.):**\n\nThanks for your suggestions. In the setup of the ablation experiments, we separately removed the three main components of Text2NKG: data augmentation (DA), the null-label weight hyperparameter (α), and hetero-ordered merging (HM). DA and α are explicitly discussed in **Section 4.3 (Training Strategy)** and are part of the Multi-label Classification in **Figure 3**, representing two main training strategies. HM is part of the Hetero-ordered Merging in **Figure 3**. The ablation experiment results show that all three components contribute to the final outcome, validating the rational design of these modules.\n\nSpecifically, in the **Text2NKG w/o DA** setting, we did not perform augmented training with all 6 permutations of span-tuples, only training with the original order. This led to uneven training samples, causing some relational representations to be under-trained and reducing effectiveness. In the **Text2NKG w/o α** setting, we set α to 1.0, giving the null-label the same weight as other labels. Since the null-label appears far more frequently than other labels in training, this caused the model to favor classifying as null-label, resulting in fewer extracted n-ary relational facts and a drop in recall, thus decreasing performance. In the **Text2NKG w/o HM** setting, we didn't combine the probability values of different permutations during the decoding phase, preventing the model from fully considering different ordering, leading to a decrease in performance.\n\n---\n**For Question 1 (Can you provide an example of extracting N-ary relations in real-world information extraction?):**\n\nTake medical diagnostic decision-making applications as an example. Medical knowledge is more complex than general domain facts, with a higher proportion of n-ary relational facts. The medical fact **\"A male hypertensive patient is diagnosed with mild creatinine elevation when serum creatinine is between 115-133μmol/L\"** is represented as the main triplet with gender and two creatinine value indicators as auxiliary keys: **(Hypertensive patient, diagnosis, mild increase in creatinine | gender: male, numerical indicator: blood creatinine (μmol/L) ≥ 115, numerical indicator: blood creatinine (μmol/L) ≤ 133)**, forming a hyper-relational fact with five entities, more completely representing complex hypertension medical knowledge.\n\nWith the Text2NKG framework, we can more easily extract NKG in hyper-relational schema from medical guidelines to model medication and treatment rules, a process previously often entirely manual. With Text2NKG's help, this process can be simplified to automated extraction and manual review. The extracted n-ary relational facts comprising the NKG can be used for subsequent tasks through link prediction, hierarchical graph, multi-hop logical queries, etc., realizing the practical application of NKG.\n\n---\n**For Question 2 (How should NKG be stored and utilized, and can n-ary extraction be applied in RAG or Agent in the era of LLMs?):**\n\nNKGs should be stored in structured and efficient formats to facilitate easy retrieval and manipulation. Graph databases like Neo4j and Tugraph are ideal for managing n-ary relations due to their optimization for graph data. This n-ary relational fact can be represented using a special node to denote the n-ary relation, and then stored using traditional binary relational RDF. Efficient indexing and querying mechanisms, including SPARQL for RDF-based graphs, are crucial for quick data access. \n\nN-ary extraction can significantly benefit RAG models and intelligent agents in the era of LLMs. For RAG models, n-ary extraction enhances contextual understanding and enables dynamic content generation by providing richer contexts and retrieving relevant information from knowledge bases, improving GraphRAG. In LLM agents, n-ary extraction improves decision-making and interaction quality by enabling better comprehension of complex relationships and contextual reasoning. The scalability of n-ary extraction allows agents to adapt to various domains, offering versatility through custom n-ary relation schemas.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VdEljI0C2K", "reviewer": "Reviewer_GMbk", "summary": "Text2NKG is a novel framework for fine-grained n-ary relation extraction aimed at constructing n-ary relational knowledge graphs (NKGs). Unlike traditional binary relational knowledge graphs, NKGs encompass relations involving more than two entities, making them more reflective of real-world scenarios. Text2NKG employs a span-tuple classification approach along with hetero-ordered merging and output merging techniques. It supports multiple NKG schemas—hyper-relational, event-based, role-based, and hypergraph-based—and achieves state-of-the-art performance in fine-grained n-ary relation extraction benchmarks.", "questions": "What is the connection and difference between n-ary relation extraction and event extraction?", "limitations": "The authors have listed the limitations in the Checklist.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1) Traditional triplet representations of knowledge cannot fully express complex relationships. However, there is limited research on the extraction of multi-ary and hyper-relations. This study on fine-grained n-ary relations has significant potential applications.\n2) The method proposed in the article improved extraction effectiveness by 20% in n-ary extraction.\n3) The author considered many additional technical issues, such as comparisons of LLMs and handling long texts, and conducted supplementary analyses.\n4) A clever method was used to solve the entity order problem in n-ary relation extracion, and the experiments demonstrated that it is possible to extract facts with varying numbers of arity in an unsupervised manner.\n5) The code and datasets used in the experiments are made publicly available, promoting transparency, reproducibility, and further research in this area.", "weaknesses": "1) The comparison settings with the large language models need to be clearer.\n2) Line 159: imput -> input.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for a detailed review. We are delighted that you think our work have significant potential applications and technical effectiveness. We answer your questions below.\n\n---\n**For Weakness 1 (The comparison settings with the large language models need to be clearer.):**\n\nThanks for your suggestions. Our ChatGPT and GPT-4 prompts both adopted a 1-shot unsupervised design. The specific input prompts (which we will add to the Appendix of the final paper) can be referenced from my response to the **Weakness 1(i)** mentioned by **Reviewer rUhA**.\n\n---\n**For Weakness 2 (Line 159: imput -> input.):**\n\nThanks for your carefully reading. We will correct this typo in the final paper.\n\n---\n**For Question (What is the connection and difference between n-ary relation extraction and event extraction?):**\n\nN-ary relation extraction and event extraction are closely related tasks in natural language processing that aim to extract structured information from text. N-ary relation extraction focuses on identifying relationships involving more than two entities. Event extraction, as one of specific schema in n-ary relation extraction, identifies events described in the text and extracts participants and attributes associated with these events. An event consists of a trigger word indicating the occurrence and arguments that represent the entities involved.\n\nThe primary difference between the two lies in their focus and output. N-ary relation extraction captures static relationships among multiple entities, resulting in tuples that represent complex interactions. Event extraction, however, focuses on dynamic occurrences, identifying events, triggers, and arguments to produce structured event representations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kbKZg49PN4", "reviewer": "Reviewer_XsBb", "summary": "This work introduces Text2NKG. A proclaimed first method of extracting n-ary facts from text for construction a KG. The focus on n-ary is a more complex task than standard RE as an n-ary fact can hold more entities than the standard RDF  {subject, relation, object}.\n\nCandidate 3-ary span tuples are formed from entities combinations in a sentence. This is embedded using BERT, and put through linear classifiers (Feed-Forward-Networks). The resulting 3-ary facts are then combined for n-ary facts using \"hetero-ordered merging\". The fine-tuning process makes use of a data augmentation technique, a balancing parameter to compensate for the large amount of \"no relation\" labels, and without merging.\n\nThe method is verified on HyperRED (the only public hyperrelational dataset currently available), and beats several SOTA methods.", "questions": "Questions:\n\t- Line 40: Several more benchmarks for NKG are mentioned (JF17K, Wikipeople, WD50K, etc.), but not considered (or no argumentation is provided). Why were these not considered?\n\t- From inspection, what are cases where the model fails (e.g. seemingly difficult cases where it struggles) or cases where it consistenly succeeds?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The motivation and contributions of the paper are clear and concrete. The previous limitations of research (Diversity, determination of order, and variability) are clearly mentioned and addressed.\n\t- The authors compare to a variety of different baselines, including SOTA LLM (GPT).\n\t- For this, a number of prompt templates have been considered\n\t- Despite the sometimes overwhelming amount of notations, the work is generally pleasant to read.", "weaknesses": "- The provided code is complex with low readibility (>5 nested for/while statements with little/no comments and no documentation).\n\t- Code makes mention of ACE dataset, but this does not seem to have made it into the end-work, adding to the confusion.\n\t- Other than the case study in Figure 5, the error analysis is fairly weak and generally performance based. There is no inspection provided as to what explains (for example) the discrepancy between \"Text2NKG\" and \"Text2NKG w/o HM\".\n\t- Prompt templates for GPT that were considered (or eventually used for best result) are not included\n\t- BERT was trained on a copy of Wikipedia in 2018. There seems to be no consideration that the data in HyperRED may overlap with the pre-training data of BERT.\n\t- The work hyperfixates on a single benchmark: HyperRED, despite others being mentioned.\n\n\n\nComments:\nFigure 2: Typo \"hyper-relaional\"\nLine 159: typo\nLine 214: Broken sentence.\nLine 219: Why was it decided to name these Generative and Pipeline and not specific to the previous works? (BART and UniRE)\nTable 2: The GPT results dont need 4 decimal places\nTable 2: typo (pipelinge)\n\nEquation 6: It is unclear to me how this is derived, and it is not explained why (the how is there) this specific formulation works.\nFigure 5a: Figure is hard to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for a careful review of our work. We appreciate that you find our work concrete in motivation and pleasant to read. We respond to your specific concerns and questions below.\n\n---\n**For Weakness 1 (Code makes mention of ACE dataset, but this does not seem to have made it into the end-work.):**\n\nWe are pleased for your interest in our code. We will optimize it for better contribution to the open-source community. Previously, we converted the ACE dataset as a benchmark for n-ary relation extraction, but reviewers found it contributed little as a benchmark, so we removed it from the paper. Any remaining mentions in the code will be corrected in the final release. Thank you for your feedback.\n\n---\n**For Weakness 2 (There is no inspection provided as to what explains the discrepancy between \"Text2NKG\" and \"Text2NKG w/o HM\".):**\n\nThanks for your suggestions. As described in **Section 3**, n-ary RE involves more entity orders than binary RE. Text2NKG uses a hetero-ordered merging method to consider probabilities of different entity orders.\n\nIn the **Text2NKG w/o HM** setting, we didn't combine probabilities of different arrangements during decoding, leading to a decrease in performance.\n\n---\n**For Weakness 3 (Prompt templates for GPT that were considered are not included):**\n\nThanks for your suggestions. Our ChatGPT and GPT-4 prompts both adopted a 1-shot unsupervised design. The specific input prompts (which we will add to the Appendix of the final paper) can be referenced from our response to the **Weakness 1(i)** mentioned by **Reviewer rUhA**. \n\n---\n**For Weakness 4 (There seems to be no consideration that the data in HyperRED may overlap with the pre-training data of BERT.):**\n\nBERT is widely used for downstream tasks, including n-ary relation extraction. Baseline models like CubeRE and those from the HyperRED dataset paper are also fine-tuned on BERT-based pre-trained methods for fair comparison. Plus, since Wikipedia is not originally structured with n-ary relational schemas, HyperRED focuses on extracting these structured relations from unstructured text, making it a more challenging task that involves entity count, order, and evaluation criteria.\n\n---\n**For Weakness 5 (The work hyperfixates on a single benchmark: HyperRED, despite others being mentioned.):**\n\nThere is a scarcity of work on automatically constructing NKGs from natural texts. HyperRED is currently the only hyper-relational extraction dataset. Text2NKG is the first method to unify n-ary RE extraction for four types of NKG schemas and can support more NKG schemas.\n\n---\n**For Comment (1) (Typos in Figure 2, Line 159, Line 214, and Table 2.):**\n\nThanks for your carefully reading. We will correct \"hyper-relaional\" in **Figure 2** to \"hyper-relational\", \"imput\" in **Line 159** to \"input\", 4 decimal places in **Table 2** to 2 decimal places, \"Pipelinge\" in **Table 2** to \"Pipeline\". Your proposal greatly enhances the presentation of the paper and will be all implemented.\n\n---\n**For Comment (2) (Why was it decided to name these Generative and Pipeline?):**\n\nAs stated in **Table 2**, the results of the supervised baseline models are primarily taken from the original paper of CubeRE. For consistency, we retained the baseline names from the CubeRE paper, using the same terms: Generative and Pipeline instead of BART and UniRE.\n\n---\n**For Comment (3) (Equation 6: It is unclear to me how this is derived, and it is not explained why this specific formulation works.):**\n\nIn the data augmentation section in **Section 4.3**, we trained on all 6 arrangements of A, B, C. Therefore, in Hetro-ordered Merging (prediction phase), we also need to consider all 6 arrangements to accurately evaluate (A, B, C). However, we must convert the 6 different orders back to $(A, r_1, B, r_2, C)$ to obtain the corresponding probability $P_i$ for each $r_i$ and sum them up. Let's take the calculation of $\\mathbf{P}_{1}$ as an example, which is derived from the sum of six terms. The first term $\\mathbf{P}_1^{(ABC)}$ represents the probability of $r_1$ in $(A, r_1, B, r_2, C)$ without any operation. The second term $I(\\mathbf{P}_1^{(BAC)})$ represents the probability of $r_1'=r_1^{-1}$ in $(B, r_1^{-1}, A, r_2, C)$, where $r_1'$ needs to be inverted to transform to (A, B, C). The remaining 4 terms in the equation follow the similar logic. Finally, we calculate the combined probability of the two $r_1,r_2$ in (A, B, C) to get Equation 6.\n\n---\n**For Comment (4) (Figure 5a is hard to read.):**\n\nIn **Figure 5a**, \"pred_n\" represents the number of extracted n-ary facts with different arities $n$ by Text2NKG, and \"ans_n\" represents the ground truth. As training progresses, Text2NKG's output converges to the correct number of n-ary facts for each arity, demonstrating its ability to handle n-ary RE with arbitrary arity and good scalability.\n\n---\n**For Question 1 (Several more benchmarks for NKG are mentioned (JF17K, Wikipeople, WD50K, etc.). Why were these not considered?):**\n\nDatasets like JF17K, Wikipeople, and WD50K, used for link prediction in n-ary relational knowledge graphs (NKGs), lack original text. Practical fields like medicine, finance, and law require extracting n-ary relational facts from natural language. HyperRED is the only high-quality dataset for this. Mentioning other NKGs helps design Text2NKG's method to cover all NKG schemas.\n\n---\n**For Question 2 (What are cases where the model fails or cases where it consistenly succeeds?):**\n\nFrom **Table 2** and **Table 3**, Text2NKG shows higher Precision than Recall, indicating it accurately extracts n-ary facts but often misses some. \n\nTo balance this, we introduced a null-label weight hyperparameter ($\\alpha$). As shown in Figure 4(c), increasing $\\alpha$ raises Precision but lowers Recall by predicting more null-labels. Decreasing $\\alpha$ does the opposite. This adjustability allows Text2NKG to be optimized for different scenarios.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all four reviewers for carefully reading our paper and providing constructive feedback. We appreciate the recognition of our work's strengths, which are summarized as follows.

**1. Novelty in Motivation.** 
To address challenges such as the diversity of NKG schemas, determination of the order of entities, and variability of the arity of n-ary RE, we propose a novel fine-grained n-ary relation extraction framework, Text2NKG, for NKG construction. (Supported by Reviewers: ED7F: Strength 1, GMbk: Strength 1, XsBb: Strength 1)

**2. Practicality and Flexibility.** 
Over 30% of real-world facts contain more than two entities, with multiple n-ary schemas. Text2NKG supports any schema of n-ary relation extraction flexibly. (Supported by Reviewers: rUhA: Strength 2, ED7F: Strength 2, GMbk: Strength 4, XsBb: Strength 3)

**3. SOTA Performance and Effectiveness.** Experimental results show that Text2NKG achieves state-of-the-art performance in fine-grained n-ary relation extraction tasks, showing its effectiveness. (Supported by Reviewers: rUhA: Strength 1, ED7F: Strength 3, GMbk: Strength 2)

**4. Extensive Verification and Analysis.**
We conducted extensive experiments to demonstrate the effectiveness of Text2NKG and each of its modules, and compare it further with advanced LLMs like GPT. We also analyzed the advantages and disadvantages of large and small models for this task, and considerations for long texts. (Supported by Reviewers: ED7F: Strength 4, GMbk: Strength 3, XsBb: Strength 2)

We believe we have addressed all points mentioned and substantially improved the paper. We responded to the four reviewers, respectively.

Meta-review: The paper focuses on the extraction of n-ary relations from text and proposes a method based on a linear multi-label classification of all possible entity encoding $3$-combinations to build $3$-ary facts. They also use data augmentation (inverse relations, reordering of hyperrelational facts) and use maximum aggregation to obtain a prediction. The $3$-ary facts are combined via a fixed rule for common $(n-1)$-ary prefixes.
The paper's strengths include (1) the motivation and relevance of the task (`rUhA`, `XsBb`), (2) the writing (`ED7F`, `XsBb`), (3) the experimental results (`rUhA`, `ED7F`, `GMbk`), and (4) the number of baselines and ablations considered (`ED7F`, `GMbk`, `XsBb`).
The following shortcomings are present: (1) the scalability of the approach (`ED7F`), (2) a lack of details regarding the use of LLM baselines (`rUhA`, `GMbk`, `XsBb`; these were added during the rebuttal), (3) a lack of detailed analysis of the individual components presented, and (4) a heuristic rule-based aggregation to higher arity.
The reviewers and authors generally participated in the discussions, and some issues were resolved during the discussions.
Decision: Accept (poster)