Two Birds, One Stone: An Equivalent Transformation for Hyper-relational Knowledge Graph Modeling
Authors: Yu Liu
Shu Yang
Jingtao Ding
quanming yao
Yong Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: Hyper-relational knowledge graph
hyperedge expansion
graph neural network

Abstract: By representing knowledge in a primary triple associated with additional attribute value qualifiers, hyper-relational knowledge graph (HKG) that generalizes triple based knowledge graph (KG) has been attracting research attention recently. Compared with KG, HKG is enriched with the semantic difference between the primary triple and additional qualifiers as well as the structural connection between entities in hyper-relational graph structure. However, to model HKG, existing studies mainly focus on either semantic information or structural information therein, fail to capture both simultaneously. To tackle this issue, in this paper, we propose an equivalent transformation for HKG modeling, referred to as TransEQ. Specifically, the equivalent transformation transforms a HKG to a KG, which considers both semantic and structural characteristics. Then a generalized encoder-decoder framework is developed to bridge the modeling research between KG and HKG. In the encoder part, KG-based graph neural networks are leveraged for structural modeling; while in the decoder part, various HKG-based scoring functions are exploited for semantic modeling. Especially, we design the sharing embedding mechanism in the encoder-decoder framework with semantic relatedness captured. We further theoretically prove that TransEQ preserves complete information in the equivalent transformation, and also achieves full expressivity. Finally, extensive experiments on three benchmarks demonstrate the superior performance of TransEQ in terms of both effectiveness and efficiency. On the largest benchmark WikiPeople, TransEQ significantly improves the state-of-the-art models by 15% on MRR.

Under review as a conference paper at ICLR 2023
TWo BIRDS, ONE STONE: AN EQUIVALENT TRANS- FORMATION FOR HYPER-RELATIONAL KNOWLEDGE GRAPH MODELING
Anonymous authors Paper under double-blind review
ABSTRACT
By representing knowledge in a primary triple associated with additional attribute- value qualifiers, hyper-relational knowledge graph (HKG) that generalizes triple- based knowledge graph (KG) has been attracting research attention recently. Com- pared with KG, HKG is enriched with the semantic difference between the primary triple and additional qualifiers as well as the structural connection between entities in repeerrelational graph structure. However, model HKG, existing studies mainly focus on either semantic information or structural information therein, fail capture both simultaneously. To tackle this issue, in this paper, we propose equivalent transformation for HKG modeling, referred to as TransEQ. Specifically, the equivalent transformation transforms a HKG to KG, which considers both semantic and structural characteristics. Then a generalized encoder-decoder frame- work developed bridge the modeling research between KG and HKG. In the encoder part, KG-based graph neural networks are leveraged for structural model- ing; while in the decoder part, various HKG-based scoring functions are exploited for semantic modeling. Especially, we design the sharing embedding mechanism in the encoder-decoder framework with semantic relatedness captured. We further theoretically prove that TransEQ preserves complete information in the equivalent transformation, and also achieves full expressivity. Finally, extensive experiments three benchmarks demonstrate the superior performance of TransEQ in terms of both effectiveness and efficiency. On the largest benchmark WikiPeople, TransEQ significantly improves the state-of-the-ard models by 15% MRR.
INTRODUCTION
In the past decade, knowledge graph (KG) has been widely studied in artificial intelligence area (Ji et al., 2021). By representing facts into triple of rs, o) with subject entity s, object entity and relation r, KG stores real-world knowledge in a graph structure. However, recent studies find that KG with simple triples provides incomplete information (Galkin et al., 2020; Rosso al., 2020). For example, both (Alan Turing, educated at, Cambridge) and (Alan Turing, educated at, Princeton) are true facts in KG, which might be ambiguous when the degree matters.
Hence, the hyper-relational KG (HKG) (Galkin et al., 2020; Rosso et al., 2020; Yu & Yang, 2021), a.k.a., knowledge hypergraph (Fatemi al., 2020; 2021) and n-ary knowledge base (Guan al., 2019; Liu et al., 2021), is proposed for more generalized knowledge representation. Formally, in HKG, a primary triple is augmented with additional strribueevaalus qualifiers for rich semanties. called the hyper-relational fact (Guan et al., 2020). Note that the triple without qualifiers is a special case of hyper-relational facts. Taking Figure 1 as an example, both (Alan Turing, educated a at, Cambridge, (degree, Bachelor)) and (Alan Turing, educated at, Princeton, (degree, PhD)) are hyper-relational facts, where (degree, Bachelor) and (degree, PhD) are qualifiers with the degree attribute considered. Such hyper-relational facts ubiquitous that over 1/3 of the entities in Freebase (Bollacker et al., 2008) involve in them (Wen et al., 2016).
To learn from HKG and further benefit the downstream tasks, HKG modeling learns diw-dimensional vector representations (embeddings) of entities and relations (Wang et al., 2021), which designs a scoring function (SF) based on the embeddings measure the hyper-relational fact plausibility such

---

Under review as a conference paper at ICLR 2023
that valid ones obtain higher scores than invalid ones. Especially, existing studies mainly consider two aspects of semantic information and structural information in HKG for modeling.
The semantic information emphasizes the interaction between entities and relations in a hyper-relational fact. Especially, there is distinction, a.k.a., semantic differ- ence (Galkin et al., 2020) between the primary triple and attribute-vall value qualifiers, e.g.. the primary triple (Alan Turing, educa
primary triples
educated
autucated at
Alan Baning alumns
at, Cambridge) serves as the fun- damental part and preserves the essential knowledge Alan Turing 's education experience at Cambridge, while the attribute-value qualifier (degree, Bachelor) serves as the auxiliary part and enriches the primary triple. To model the semantic information, early studies treat the primary relation and qualifier relations as an n-ary (n22) composed relation (Abboud et al., 2020) or multiple semantically equal attributes (Guan et al., 2019; Liu et al., 2021), largely ignoring the semantic difference. Various SFs are further developed in recent studies (Galkin et al., 2020: Rosso et al., 2020; Yu & Yang, 2021) with semantic difference considered.
Mirile edurated Sf
Princean
Cenbrage
PhD
dogree
dagroo
RHO
attribute-value qualifiers
Bachelir
Figure 1: An example of HKG in- cluding primary triples and attribute- value qualifiers. The entities/relation in the triple are called as primary en- tities/relation, and attributes/values in qualifiers are called as qualifier enti- ties/relations.
On the other hand, the structural information focuses the topologiaal connection between entities in the hyper-relational graph structure, like an entity's s neighboring entities under various hyper- relational links, e.g., in Figure 1 Bachelor and Michelle Obama neighbors of Alan Turing via degree and a alumii, respectively. Only few studies (Galkin et al., 2020; Yadati, 2020) extend hypergraph neural network (HGNN) based modules capture the structural information in HKG, however, empirical results in (Yu & Yang. 2021) demonstrate that removing such modules will not bring performance degradation, i.e., the direct extensions are quite immature for effective structural information capture. Hence, to the best of our knowledge, none of existing studies achieve HKG modeling with both semantic information and structural information completely captured, and is still an open problem to be addressed.
Targeting on this open problem, we look back to KG modeling with an interesting observation that, recent studies (Vashishth et al., 2019; Yu et al., 2021) leverage an encoder-decoder framework for KG modeling, i.e., a powerful graph neural network (GNN) based encoder and an expressive SF-based decoder on triples are leveraged for structural information and semantic information, respectively. Inspired by this, in this paper. we propose an EQuivalent Transformation for HKG modeling, termed as TransEQ. Specifically, TransEQ designs equivalent transformation the hyper-relational graph structure, transforming a HKG to a KG with semantic difference considered, based on which generalized encoder-decoder framework is further developed capture information. For structural information, TransEQ introduces a GNN-based encoder on transformed KG with transformation characteristics combined. As for semantic information, to measure the plausibility of a hyper-relational fact, TransEQ exploits various SFs in existing HKG modeling studies as the decoder. The sharing embedding mechanism is further designed to capture the semantic relatedness between hyper-relational facts. In this way, with the equivalent transformation, the encoder-decoder framework in TransEQ captures not only structural information but also semantic information, which is the very innovation of this work, just like killing two birds with one stone. Besides, the flexible choice of SF in decoder ensures the full expressivity of TransF:Q, representing types of relations. We further theoretically prove that the proposed transformation is equivalent between a HKG and KG without information loss. Extensive experiments show that TransEQ achieves the state-of-the-nn results, obtaining 15% relative increase of MRR on the largest benchmark WikiPeople.
RELATED WORK
As described before. related studies mainly exploit two aspects of semantic information and structural information for HKG modeling, considering -kGG-bseed SF design and hyper-relational graph structure, respectively.
2

---

Under review as a conference paper at ICLR 2023
Semantic Modeling Studies. Given hyper-relational fact (Alan Turing, educated at, Cam bridge, (degree, Bachelor),, some studies treat all involved relations as an n-ary composed relation educated at at_ degree (here is 3) with the fact (educated at degree, Alan Turing, Cam- bridge, Bachelor). For example, both m- m-TransH (Wen et al., 2016) and RAE (Zhang et al., 2018) extend the SF of TransH (Wang al., 2014) the hyper-relational case. BoxE (Abboud al., 2020) combines translational idea with box embeddings. Moreover, GETD (Liu et al., 2020) and S2S (Di et al., 2021) are both generalized from TuckER (Balazevic et al., 2019), where GETD further introduces tensor ring decomposition while S2S applies neural architecture search techniques. The bilinear product is also extended to multilinear product with symmetric embeddings in m- Mi-Dittullt (Yang et al., 2015), convolutional filters in HypE (Fatemi et al., 2020), and relational algebra op- erations in ReAlE (Fatemi et al., 2021). These studies are directly extended from KG modeling methods without multiple relational semantics considered. On the other hand, NaLP (Guan et al., 2019) and RAM (Liu et al., 2021) decompose all involved relations into semantically equal attributes, and treat the example fact into collection of sitribute-value qualifiers, (educated at headd Alan Turing, educated at at_tail, Cambridge, degree, Bachelor). Nevertheless, models above largely ignore the semantic difference in hyper- seper-relational facts. To capture such semantic difference between the primary triple and attribute-value qualifiers, NeuInfer (Guan et al., 2020) and HINGE (Rosso et al., 2020) design two sub-modules for HKG modeling, i.e., one for triple modeling and the other one for qualifier modeling, where NeuInfer mainly adopts fully connected layers while HINGE resorts to convolutional neural networks. Besides, recent studies of GRAN (Wang et al., 2021) and Hy-Transformer (Yu & Yang, 2021) leverage transformer and embedding processing techniques for HKG modeling. However, these neural network based models rely on tremendous parameters for expressivity and are prone to overfitting.
Structural Modeling Studies. G-MPNN (Yadati, 2020) ignores attribute information and treats HKG as multi-relations ordered hypergraph with n-ary composed relations, and further proposes multi-relational HGNN for modeling. The rough design makes G-MPNN less competitive in practice. StarE (Galkin et al., 2020) firstly introduces GNN for HKG modeling with relation-speciff message passing mechanism developed. However, StarE aggregates hyper-relational fact messages for a specific entity only when the entity involves with the primary triple, but ignores the ones when the entity is in attribute-value qualifiers, i.e., StarE only captures connections among primary triples (Yu & Yang, 2021). Thus, capturing structural information for HKG modeling is still immature and needs further investigation.
Overall, existing HKG modeling studies are affected by various limitations from semanties and structure, while our proposed TransEQ elegantly models both aspects with full expressivity achieved, which quite important property for learning capacity in both KG modeling (Balazevic al., 2019; Sun et al.. 2019) and HKG modeling (Abboud et al., 2020; Liu et al., 2020). Besides, the inductive link prediction and logical query for HKG are investigated in recent studies (Ali et al., 2021; Alivanistos et al., 2022), which are beyond the scope of this paper.
3 METHOD
Here we first introduce the mathematical definition of HKG as well as the investigated problem.
Definition 1 Hyper-relational Knowledge Graph. A HKG is defined as G = (E, R,, JH), where & and are the sets of entities and relations, respectively. A hyper-relational fact can be expressed as (s,T, o, {(a,, v.) )) where (s, o) is the primary triple and (a,, v:) aa E R, E is the attribute-value qualifier set. Moreover, F" CE x R x E x P denotes the fact set and P denotes all possible combinations of altribute-value qualifiers.
Note that the number of qualifiers can be zero for hyper-relational fact, i.e., HKG reduces to KG with an empty set P. In practice, attributes and values are also described by relations and entities. respectively (Galkin et al., 2020; Yu & Yang, 2021). Then we state our research problem.
Problem HKG Modeling Problem. Given a HKG GH (E, FH), the HKG modeling problem aims to learn representations for entities and relations in E and R, respectively.
Especially, the HKG is always incomplete, which specifies the research problem as HKG completion problem in practice, i.e., given incomplete hyper-relational fact with an entity missing at triple or
3

---

Under review as a conference paper at ICLR 2023
qualifiers, inferring the missing entity from & with observable facts F Moreover, HKG involves semantic information of primary triple and attribute-value qualifiers as well as structural information of hyper-relational graph structure, which should be elegantly considered in modeling.
As described before, the encoder-decode framework has shown superior performance to capture both structural information and semantic information in KG (Yu et al., 2021; Vashishth et al., 2019), and thus a natural idea is to explore it for HKG modeling. Besides, standard RDF reification in semantic web (Frey al., 2019) well compound value type in Freebase (Bollacker al., 2008) are investigated to describe triple with metadata by transformation. These works provide motivation our work transforming HKG to KG with the encoder-decoder framework combined. Hence, we build TransEQ with such points in mind, which is presented in the following.
3.1 THE TRANSEQ MODEL
We now come the details TransEQ, the architecture of which is illustrated Figure 2. TransEQ first introduces the equivalent transformation with a HKG transformed to KG. and then develops generalized encoder-decoder framework, where GNN-based encoder and a SF-based decoder leveraged for modeling structural information and semantic information, respectively.
Equivalent Transformation
Topology
Embedding
Brsc
medistor
entity
MSC
AGE
UPD
+
Original HKG
Transformed KG
GNN-based Encoder
SF-based Decoder
Figure 2: The architecture of our proposed HKG modeling model TransEQ.
3.1.1 ONE STONE: EQUIVALENT TRANSFORMATION
To identify the importance of transformation between HKG and KG, here we first introduce the definition of equivalent transformation.
Definition 2 Equivalent Transformation. A transformation between HKG and KG is equivalent, if the transformation preserves the complete information, i.e., given any HKG and its transformed KG via the transformation, they can be retrieved from each other.
Moreover, a hyper-relational fact (s,r, r, (aa, v;)) can be viewed as a hyper-relational edge, which connects entities of 8, 0, ]= with heterogeneous semanties of primary relation and attributes {a,)= as shown in Figure 3(a) with the k-th fact in HKG. Thus, motivated by star expansion, we propose an equivalent transformation for hyper-relational edges such that entities and relations in the original HKG are reorganized to the transformed KG with both structural information and semantic information preserved.
Specifically, the equivalent transformation in Figure 3(b) introduces mediator entity by to identify the fact, and the primary relation is extended with two relations suhh and nh for the relational edges between by and subject entity and object entity respectively. The attribute information in original hyper-relational fact preserved by the attributed-based edges between bę and value entities. Moreover, relational edge connects entities and for semantic difference, i.c., such operation leads to a three-node clique motif (Milo et al., 2002), reflecting the primary role the triple.
For better understanding, Algorithm 1. Especially, in lines 7-8, TransEQ utilizes different transformation operations to model the semantic difference. Besides, the original structure of the triple fact, i.e., hyper-relational fact without qualifiers, is kept to avoid redundancy. As proved later, such transformation brings no information loss, and provides good basis for the following encoder-decoder framework.
present the execution process of the equivalent transformation in
4

---

Under review as a conference paper at ICLR 2023
Algorithm 1: The algorithm for equivalent transformation.
Input: HKG G (E, R, F");
Init: Transformed KG g (E, R. F) with F +- 0; Obtain the set of primary relations for facts in FH Ren;
a22
2 for r E Red do
Define new relations Ts. and R < RU frouth yob }: end
(n) k-th hyper-rclational fact
5 for k-th fact (s, r, o, {(a,, u.)
F do
S
sub
V1
Define mediator entity bk_ and & t & U {ba}:
Dy
7
FiF U {(s, (,, o), (bk.r youn s), (bk. o)}:
pob
F+ FU {(bk, at 24 1=
// semantic di difffe ffrence modeling in
(b) equivalent transformation
lines 7-8.
end
Figure 3: The illustration of the equivalent transformation.
Output: Transformed KG G = (E, F).
3.1.2 Two BIRDS: ENCODER-DECODEE FRAMEWORK
To model both structural information and semantic information in the original HKG, TransEQ further introduces a generalized encoder-decoder framework on the transformed KG.
GNN-based Encoder. The powerful GNN is developed to capture structural information, where the semantic relatedness in HKG and the mediator entities in equivalent transformation are also incorporated therein. Especially, the semantic relatedness explicitly lies in the shared primary relations across hyper-relational facts. For example, the hyper-relational facts of (Alan Turing, educat ed at, Cambridge, (degree. Bachelor)) and (Alan Turing, educated at, Princeton, (degree, PhD)) share the same primary relation, which indicates strong semantic relatedness. On the other hand, in our proposed equivalent transformation, each mediator entity plays an important role of relaying connections among the entities in an original hyper-relational fact, and thus mediator entities aggregate the semantics of corresponding facts.
Hence, we introduce the sharing embedding for mediator entities to capture the semantic relatedness, and further combine it with three steps of unified multi-relational message passing mechanism in KG-based GNN (Schlichtkrull et al., 2018; Vashishth et al., 2019).
Initialized embedding. Given the embedding dimension d, for the mediator entity b, we denote (bb) the mapping from b to its involved primary relation, and initialize its representation as hg - (c) es), where (w( E LBLa-d] and e E Ld-dd are sharing and independent embeddings, respectively. a is the hyperparameter to tune the sharing embedding ratio. Thus, mediator entities involved with the same primary relation (b)) share part of embedding (v())-
Message calculation. Considering the stacking layers of GNN, we denote m 1+1,ent and m 1+1.ll the messages from triple (u, r, t) for target entity and relation at the (1 + 1)-th layer, respectively, which are calculated as follows,
++1,eat MSG (h',, 1,,,,,)) h2),, 1+1.ml M 1,,,,))))
m urt
where h1 h/, N e are the embeddings of entities and relation at the 1-th layer, while MSG and MS can be composition function in CompGCN (Vashishth al., 2019), relation-specific projection in R-GCN (Schlichtkrull et al., 2018) and etc. Besides, the entity representations at the input layer expressed as,
[ev(a ez]
if is mediator entity if a is original entity
0 x
for x E (u,t),
e C
Message aggregation. Then neighborhood messages of M and M are aggregated as follows,
(m|= r uENT), M AGG (41.el (u, t) eN)), where N/ denotes the entities linked to via relation and Mr denotes the entity pair linked by relation AGG and AGG aggregation functions like mean/sum pooling function.
5

---

Under review as a conference paper at ICLR 2023
Representation update. Finally, the representations the (1 1)-th layer updated with aggregated messages and former layer representations:
h h UPD UP, h(), hit = (M, .,,)).
where UPD and UPD can be nonlinear activation functions.
Owing to above encoding process, TransEQ fully exploits the topological connections between entities for structural information.
SF-based Decoder. The decoder part exploits various SFs to model semantic information. For each hyper-relational fact, the encoder part feeds the representations corresponding entities and relations into the SF-based decoder to model the interaction between entities and relations therein. Especially, the choice SF is orthogonal to the encoder, and most existing SFs on HKG modeling can be modified in the decoder. For example of the hyper-relational fact x == (8, r,0, {(a,, 0z) oz)] we rewrite m-DistMult's ' SF as:
((x) = (h h ... h/,,) ), h h. h h:
where ((x) is plausibility score measured by TransEQ. (-) denotes the multilinear product', and L denotes the number of GNN layers in encoder part. Since m-DistMult adopts a composed relation for SF, we introduce the function ip to aggregate the embeddings of involved primary relation and attributes for the composed relation embedding, such as mean/sum pooling function. Note that the semantic difference in HKG also modeled by the SF in decoder. Various SFs further investigated in experiments later.
Model Training. To learn the model parameters, we adopt the cross-entropy loss for training. For the hyper-relational fact x E FH with o(x), the practical loss can be written as:
ed(a)
Ec)(0==
Σ log e + EM'EN, eP(x')
-log
#EF"
ZEFI
where Nr, denotes the negative samples, i.e., entities in triple and attribute-value qualifiers of x are replaced by other entities ε. The training algorithm TransEQ is presented in Appendix for better understanding. The overall model is trained in a mini-batch way with batch normalization and dropout utilized for regularization.
Overall, the proposed TransEQ develops an equivalent transformation that transforms HKG to KG. Then a generalized encoder-decoder framework associates KG modeling research with HKG ones, where KG-based GNN encodes structural information while HKG-based SF in decoder focuses on semantic information.
Table 1:
comparison representative HKG modeling studies. fig. n,, and ni denote the numbers of entities, relations and primary relations. d is the embedding dimension. n. is the maximum number of attribute-valus qualifiers for facts, and N = | is the total number of facts in HKG. Neural: neural network based SF, Multilinear: multilinear product based SF.
Structure Modeling
Semantic Scoring
Model
Difference
Function
Expressive
time
space
NaL.P
Neural
O
O + nn,d)
m-DistMult
Multilinear
X
Odd)
O (m + n.
HypE
X
X
Multilincar
(d)
O + n d)
HINGE
X
Neural
X
O(d))
On + n,d)
G-MPNN
HGNN
X
Multilincar
O(Nd))
Onned + nffd + nod) O(n,d + +n,d)
StarE
GNN
Neural
O(Nd + +n.dd)
TransEQ
Transformation & GNN
Arbitrary SF
O(Nd))
O(ned + n,d Nd)
3.2 THEORETICAL UNDERSTANDING
Complexity Analysis. To distinguish our proposed TransEQ model design, in Table 1, we present comparison HKG modeling studies with structural modeling, semantic modeling, full expressivity
(hi, h2 hnd) E, h1 hgli] hh,

---

Under review as a conference paper at ICLR 2023
well time and space complexity. According to the table, structural information is rarely explored in existing studies, while HGNN in G-MPNN is at its early stage and thus fails to model attribute semantics. StarE only captures triple-base bille- connections (Yu & Yang, 2021), while TransEQ combines the equivalent transformation with GNN for structural information. As for semantic modeling, TransEQ not only captures the semantic difference between the primary triple and attribute- value qualifiers, but also applies for arbitrary SFs. Compared with the weak expressive power of most studies, the flexible choice of SF guarantees the full expressivity of TransEQ to model various HKGs, and brings performance improvement. Besides, the message passing mechanism in modeling structural information leads to the time complexity of ((N)). and Transformer module in StarE brings an additional complexity of dn,,d). Since the equivalent transformation introduces mediator entity for each hyper-relational fact, TransEQ builds the space complexity of Om, nn, Nd) with original entities and relations considered. Owing to parallel implementation and GPU accelcration, TransEQ obtains comparable efficiency to the fastest current studies in experiments. In this way, TransEQ achieves efficient and expressive HKG modeling with both structural information and semantic information captured.
Information Preserving Transformation. Following the structural information loss concern in hyperedge expansion (Arya et al., 2021; Dong et al., 2020; Zhou et al., 2006), here we investigate the information loss problem for our proposed transformation on HKG, which emphasizes preserving both structural information and semantic information. Based on the equivalent transformation in Definition 2 and the proposed transformation in TransEQ, we identify the property with following theorem.
Theorem In the conversion from a HKG to a KG, the proposed transformation in TransEQ is an equivalent transformation and preserves the complete information.
Full Expressivity. To demonstrate the expressivity of TransEQ, here we introduce the full expressivity property (Abboud et al., 2020; Fatemi et al., 2020; Liu et al., 2021). HKG modeling model is fully expressive if, for any given HKG, the model can separate valid hyper-relational facts from invalid ones by appropriate parameter configuration. Considering the encoder-decoder framework in TransEQ, such property is mainly determined by the SF in decoder part, thus we establish the expressivity of TransEQ with the following theorem.
Theorem
With encoder parameters configured appropriately, the expressivity of TransEQ is in accord with that of the scoring function it uses in decoder, i.e., TransEQ fully expressive if the scoring function used in decoder is fully expressive.
Thus, with appropriate choice of SF like HypE (Fatemi et al., 2020) as well as model parameters, a fully expressive TransEQ model has the potential to represent all types of relations in HKG including symmetric relations, inverse relations, etc. (Liu et al., 2021; Sun et al., 2019), which generally outperforms the weak ones in practice, validated in Section 4.3.
The proofs of above theorem are provided in Appendix C
Manually-designe v.s. Learnable Transformations. According to the TransEQ model design, with theoretical guarantee on preserving information, the manually-designel equivalent transformation paves the way for capturing both semantic information and structural information in HKG. Although such transformation design can be learnable, the learning process is over complex without theoretical guarantee, while TransEQ with the manually-designed transformation has achieved the state-of-the-ar performance, as validated by results in Section 4.2. Moreover, the simple yet effective manually- designed transformation takes the semantic difference into consideration, which offers valuable insights and rethinking discussion to the HKG modeling research.
EXPERIMENTS AND RESULTS
4.1 EXPERIMENTAL SETUP
Datasets. The experiments conducted on three benchmark HKG datasets, i.e., WikiPeople (Guan et al., 2019). JF17K (Zhang et al., 2018) and FB-AUTO (Fatemi et al., 2020). We follow the standard splits (Guan et al., 2019) of these datasets. Detailed statistics can be found in Appendix D.
7

---

Under review as a conference paper at ICLR 2023
Baselines. As for performance comparison, compare with several sfate-of-the-ee HKG modeling approaches, including semantic modeling ones of BoxE (Abboud et al., 2020) S2S (Di et al., 2021), 1), HypE (Fatemi et al., 2020), NeuInfer (Guan et al., 2020), RAM (Liu et al., 2021), HINGE (Rosso et al., 2020), m-TrransH (Wen et al., 2016) as well as structural modeling ones of StarE (Galkin et al., 2020) and G-MPNN (Yadati, 2020). Besides, in Tra et al., 2019) as encoder and m-DistMull (Fatemi et al., 2020) as decoder.
mainly adopt CompGCN (Vashishth
Task and Evaluation Metries. Following typical settings (Abboud et al., 2020; Fatemi et al., 2020; Guan et al., 2019; Liu et al., 2020; Wang et al., 2021), we evaluate HKG modeling approaches on HKG completion task in transductive setting. and predict the missing entity at each position including triple and qualifier parts. Note that this task is more generalized than only predicting positions in triple part (Galkin et al., 2020; Rosso et al., 2020; Yu & Yang. 2021). As for evaluation metries, the standard mean reciprocal ranking (MRR) and Hit @ 1,3110 are utilized in filtered setting (Bordes et al., 2013; Guan et al., 2019). Code and data available: https: / /nonymouus 4open. sci /ec I/TransEQ_Impp
03FB.
4.2 HKG COMPLETION RESULTS
Table 2: Results of HKG completion on all datasets. Results of baselines collected from original papers and (Di & Chen, 2022; Fatemi et al., 2020; Liu et al., 2021). Best results are highlighted in bold, and second best results are highlighted with underlines. "_" denotes missing results.
WikiPeople Hit@1
JF17K
FB-AUTO
Model
MRR
@it@ 10
MRR
Hit@1 1
Hit@ 10
MRR
Hit@! I
Hit@ 10
m- TransH
0.444
0.370
0.581
0.727
0.728
0.728
HINGE
0.333
0.259
0.477
0.473
0.397
0.618
0.678
0.630
0.765
NeuInfer
0.350
0,282
0.467
0.517
0.436
0.675
0.737
0.700
0.805
m- DistMult
0.318
0.213
0.478
0.452
0.375
0.599
0.784
0.745
0.845
HypE
0.292
0.162
0.502
0.507
0.421
0.669
0.804
0.774
0.856
RAM
0.370
0.293
0.507
0.539
0.463
0.690
0.830
0.803
0.876
S2S
0.372
0.277
0.533
0.528
0.457
0.690
BoxE
0.395
0.293
0,503
0.560
0.472
0.722
0.844
0.814
0.898
G-MPNN
0.367
0.258
0.526
0.530
0.459
0.688
0.763
0.724
0.838
StarF: 0.378
0.265
0.542
0.542
0.454
0.685
0.764
0.725
0.838
TransEQ 0.454
0.373
0.593
0.569
0.489
0.722
0.870
0.842
0.909
We present the benchmark comparison of HKG completion in Table 2. According to the results, our proposed TransEQ model achieves the state-of-the-an performance on all benchmarks. On the hardest dataset WikiPeople with the most entities and relations, TransEQ significantly improves the best baseline (BoxE) by 27% and 15% on Hit@1 and MRR, respectively. Considering hyper-relational connections provided WikiPeople, this improvement demonstrates that our proposed equivalent transformation preserves complete HKG information. Besides, TransEQ significantly outperforms m- DistMult, its original decoder model without GNN-based encoder, which indicates the effectiveness and necessity consider structural information in HKG modeling. Such results also imply that with powerful SFs like BoxE, TransEQ can obtain even better performance. Moreover, compared with structural modeling approaches of G-MPNN and StarE, the substantial improvement of Trans owes to subtle design of the equivalent transformation as well as the semantic information captured in the decoder part.
4.3 ENCODER-DECCE DECODER CHOICE COMPARISON
To further investigate the effects of different GNN-based encoders along with HKG-based SFs as decoders, compare the performance of different encoder-decoder choices in Table 3. In the table, each result corresponds to the TransEQ model with X as encoder and as decoder.
According the results of each row in Table 3, compared with original models (X=No Encoder), TransEQ models with various GNN-based encoders bring substantial improvement, which again demonstrates the effectiveness of structural information encoding. Since neural network models
8

---

Under review as a conference paper at ICLR 2023
Table 3: Performance comparison of different encoder-decoder choices JF17K. "OOM" indicates out of memory. Best results for each encoder are highlighted in bold.
JF17K
Encoder X
+
CompGCN
R R-GON
No Encoder
Decoder +
MRR
Hit@ 10
MRR
Hit@10
MRR
Hit 10
m-TransH
0.542
0.694
OOM
0.444
0.581
Transformer
0.526
0.677
OOM
0.504
0.648
m mn-DitMMult HypB 0.572
0,569
0.722
0,483
0.632
0,452
0,599
0.715
0.550
0.702
0.507
0.669
with tremendous parameters easily overfit, the performance improvement of ANN-based encoder for Transformer is much lower than that for other models. As for the encoder in each column, the decoder choices of HypE achieve the best performance, mainly attributed the linear complexity and full expressivity property. Benefited from the generalized encoder-decoder framework, TransEQ can flexibly adapt various GNNs and SFs for both superior performance and full expressivity.
4.4 INFORMATION SHARING STUDY
To validate whether the semantic relatedness in HKG is captured by sharing embedding mediator entities, we obtain hyper-relational facts of top ten primary relations and visualize their mediator entity embeddings via I-SNE (Maaten & Hinton, 2008), shown in Figure 4(a). We select WikiPeople for visualization considering explicit attribute information therein, and mediator entities belonging to the primary relation marked in the same color. From the figure, observe that mediator entities are neatly clustered according to their mapping primary relations, which accord with our sharing embedding design in GNN-based encoder.
0.560
MRR
0.55
0.50
(a) Visualization
(b) JF17K
(c) FR-AUTO
Figure 4: (a) The visualization of mediator entity embeddings with top ten primary relations in
WikiPeople via t-SNE; The effects of share embedding ratio a on (b) JF17K and (c) FB-AUTO.
We further investigate the effect of sharing hyperparameter in Figure 4(b) and (c). An extreme point can be observed in both datasets, which estimates the semantic relatedness corresponding datasets. Moreover, higher sharing ratio brings fewer model parameters, e.g., to the case that each mediator entity has independent embedding while a = means all mediator entities with the same primary relation own the same representation. Thus, tradeoff between model parameter complexity and practical performance can be achieved.
corresponds
5 CONCLUSION
In this paper, we propose TransEQ for HKG modeling. With the equivalent transformation developed, TransEQ successfully transforms HKG KG without information loss. Especially, TransEQ builds the generalized encoder-decoder framework, which firstly captures both structural information and semantic information for HKG. Experiment results show that TranSEQ obtains the state-of- the-art results on benchmark datasets. For future work, we would like to make the transformation design automated, such that each hyper-relational fact can be automatically transformed into multi-relational subgraph following relation-speciff transformation. Moreover, we plan to introduce specific GNN modules on the transformed KG to process the attribute information as well as primary information attached the mediator entities.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Ralph Abboud, ismail Ilkan Ceylan, Thomas Lukasiewicz, and Tommaso Salvatori. BoxE; box embedding model for knowledge base completion. In NeurIPS, 2020.
Sameer Agarwal, Kristin Branson, and Serge Belongie. Higher order learning with graphs. In ICML, 2006.
Mehdi Ali, Max Berrendorf, Mikhail Galkin, Veronika Thost, Tengfei Ma, Volker Tresp, and Jens Lehmann. Improving inductive link prediction using hyper-relational facts. In ISWC, pp. 74-92, 2021.
Dimitrios Alivanistos, Max Berrendorf, Michael Cochez, and Mikhail Galkin. Query embedding on hyper-relational knowledge graphs. In ICLR, 2022.
Devanshu Arya, Deepak K Gupta, Stevan Rudinac, and Marcel Worring. Adaptive neural message passing for inductive learning on hypergraphs. arXiv preprint arXiv:2109. 10683, 2021.
Ivana Balazevic, Carl Allen, and Timothy Hospedales. Tucker: Tensor factorization for knowledge graph completion. In EMNLP, pp. 5188-5197, 2019.
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: A collabo- ratively created graph database for structuring human knowledge. In SIGMOD, pp. 1247-1250, 2008.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In NeurlPS, pp. 2787-2795, 2013.
Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, and Qingming Huang. Dual quaternion knowledge graph embeddings. In AAAI, number 8, 2021.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. In AAAI, 2018.
Shimin Di and Lei Chen. Message function search for hyper-relational knowledge graph. 2022. URL https: / wpwpreweee.ee.eee//////.. net fowumeid--0zisseeeeee
Shimin Di, Quanming Yao, and Lei Chen. Searching to sparsify tensor decomposition for n-ary relational data. In WWW, pp. 4043-4054, 2021.
Yihe Dong, Will Sawin, and Yoshua Bengio. Hnhn: Hypergraph networks with hyperedge neurons. In ICML GRL Workshop, 2020.
Bahare Fatemi, Perouz Taslakian, David Vazquez, and David Poole. Knowledge hypergraphs: Prediction beyond binary relations. In IJCAI, 2020.
Bahare Fatemi, Perouz Taslakian, David Vazquez, and David Poole. Knowledge hypergraph embed- ding meets relational algebra. arXiv preprint
2021.
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In AAAI, number 01, pp. 3558-3565, 2019.
Johannes Frey, Kay Müller, Sebastian Hellmann, Erhard Rahm, and Maria-Esther Vidal. Evaluation of metadata representations in rdf stores. Semantic Web, 10(2):205-229, 2019.
Mikhail Galkin, Priyansh Trivedi, Gaurav Maheshwari, Ricardo Usbeck, and Jens Lehmann. Message passing for hyper-relational knowledge graphs. In EMNLP, 2020.
Saiping Guan, Xiaolong Jin, Yuanzhuo Wang, and Xueqi Cheng. Link prediction on n-ary relational data. In WWW. pp. 583-593, 2019.
Saiping Guan, Xiaolong Jin, Jiafeng Guo, Yuanzhuo Wang, and Xueqi Cheng. NeuInfer: Knowledge inference on n-ary facts. In ACL, pp. 6141-6151, 2020.
Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip. A survey on knowledge graphs: Representation, acquisition, and applications. TNNLS, 2021.
10

---

Under review as a conference paper at ICLR 2023
Yu Liu, Quanming Yao, and Yong Li. Generalizing tensor decomposition for n-ary relational knowledge bases. In WWW. pp. 1104-1114, 2020.
Yu Liu, Quanming Yao, and Yong Li. Role-aware modeling for n-ary relational knowledge bases, In WWW, pp. 2660-2671, 2021.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. JMLR, 9NNov):5579-2605 2008.
Ron Milo, Shai Shen-Orr, Shalev Itzkovitz, Nadav Kashtan, Dmitri Chklovskii, and Uri Alon. Network motifs: Simple building blocks of complex networks. Science, 298(5594):824-827, 2002.
Xavier Ouvrard. Hypergraphs: An introduction and review. arXiv preprint arXiv:2002.05000 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, pigh-performance deep learning library. In NeurlPS, volume 32, pp. 8026-8037, 2019.
Paolo Rosso, Dingqi Yang, and Philippe edge graph embedding for link prediction. In WWW, 2020.
Beyond triplets: Hyper-relational knowl-
Michael Schlichtkrull, Thomas N Kipf. Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, pp. 593-607, 2018.
Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, and Bowen Zhou. End-to-end structure- aware convolutional networks for knowledge base completion. AAAI, number 01, pp. 3060-3067, 2019.
Xiangguo Sun, Hongzhi Yin, Bo Liu, Hongxu Chen, Qing Meng, Wang Han, and Jiuxin Cao. Multi- level hyperedge distillation for social linking prediction on sparsely observed networks. In WWW, pp. 2934-2945, 2021.
Zhiqing Sun, Zhi-Hong Deng, Jian- Yun Nie, and Jian Tang. RotatE: Knowledge graph embedding by relational rotation in complex space. In ICLR, 2019.
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In ICML, pp. 2071-2080, 2016.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition-base multi- relational graph convolutional networks. In ICLR, 2019.
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. Knowledge graph embedding: A survey of approaches and applications. TKDE, 29(12):2244-2743 2017.
Quan Wang, Haifeng Wang, Yajuan Lyu, and Yong Zhu. Link prediction on n-ary relational facts: A graph-based approach. ACL Findings, 2021.
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, 2014.
Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, and Richong Zhang. On the representation and embedding of knowledge bases beyond binary relations. In IJCAI, pp. 1300-1307, 2016.
Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty. Tao Yu, and Kilian Weinberger. Simplify- ing graph convolutional networks. In ICML, 2019.
Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. Self- supervised hypergraph convolutional networks for session-based recommendation. In AAAI, number pp. 4503-4511, 2021.
Naganand Yadati. Neural message passing for multi relational ordered and recursive hypergraphs. In NeurIPS, volume 33, 2020.
11

---

Under review as a conference paper at ICLR 2023
Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergen: A new method for training graph convolutional networks on hypergraphs. In NeurIPS, volume 32, pp. 1511-1522, 2019.
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In ICLR, 2015.
Rui Ye, Xin Li, Yujie Fang, Hongyu Zang, and Mingzhong Wang. A vectorized relational graph convolutional network for multi-relational network alignment. In IJCAI, pp. 4135-4144 2019.
Donghan Yu and Yiming Yang. Improving hyper-relational knowledge graph completion. arXiv
preprint (ex.x:2204.08.00 2021.
Donghan Yu, Yiming Yang. Ruohong Zhang, and Yuexin Wu. Knowledge embedding based graph convolutional network. In WWW. 1619-1628, 2021.
Richong Zhang, Junpeng Li, and Yongyi Mao. Scalable instance reconstruction in knowledge bases via relatedness affiliated embedding. In wwW, pp. 1185-1994, 2018.
Yongqi Zhang and Quanming Yao. Knowledge graph reasoning with relational digraph. wwW, pp. 912-924, 2022.
Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clustering, classification, and embedding. In NIPS, pp. 1601-1608, 2006.
Zhaocheng Zhu, Zuobai Zhang, Louis-Pascal Xhonneux, and Jian Tang. Neural bellman-ford networks: general graph neural network framework for link prediction. NeurlPS, 2021.
12

---

Under review as a conference paper at ICLR 2023
CONTENTS
Introduction
2 Related Work
3 Method
3.1
The TranSEQ Model
3.1.1
One Stone: Equivalent Transformation
3.1.2
Two Birds: Encoder-Decodes Framework
3.2
Theoretical Understanding
Experiments and Results
4.1
Experimental Setup
4.2
HKG Completion Results
4.3
Encoder-Decoder Choice Comparison
8
4.4
Information Sharing Study
5
Conclusion
A Other Related Work
14
A.1 Knowledge Graph (KG) Modeling
14
A.2 Hypergraph & Hyperedge Expansion
14
B Method Details
15
B.1 Other Variants of Transformations
15
B.2 Training Procedure
15
Theoretical Details
16
C.1 Scoring Function Comparison
16
C.2 Complexity Analysis on Transformations
16
C.3 Proof of Information Preservation
16
C,4 Proof of Full Expressivity.
17
Experiment Details
18
D.I Dataset Details
18
D.2
Implementation Details
................................
18
D.3 Efficiency Comparison
18
D.4 Transformation Comparison
19
D.5 Encoder-Decoder Choice Comparison
19
D.6 Additional HKG Completion Results
19
D.7 Additional Visualization Results
20
13

---

Under review as a conference paper at ICLR 2023
A
OTHER RELATED WORK
A.1 KNOWLEDGE GRAPH (KG) MODELING
Learning representations for entities and relations in KGs has been investigated thoroughly (Ji et al., 2021; Wang al., 2017), which designs various SFs model the semanties in triple knowledge (8, os) Based on translational thought, TransE (Bordes et al., 2013), TransH (Wang et al., 2014) and RotatE (Sun al., 2019) measure the distance between subject and object entities in relation-specific latent space. Besides. ConvE (Dettmers et al., 2018) adopts the convolutional neural networks for SF design. TuckER (Balazevic al., 2019) employs Tucker decomposition for SF design. Furthermore, several models combine the bilinear product with various types of embeddings (Cao et al., 2021; Trouillon al., 2016; Yang et al., 2015). For example, ComplEx (Trouillon et al., 2016) and DulE' (Cao et al., 2021) employ complex-valued embeddings and dual quaternion embeddings, respectively, However, models above ignore the multi-relational graph structure of KGs.
Until the emergence of message passing mechanism with GNN, structural information capture becomes an important topic in KG modeling. An encoder-decoder framework is developed recent KG-based GNN studies, where GNNs encode structural information of KG and various SFs are combined for semantic information. Specifically, both R-GCN (Schlichtkrull et al., 2018) and SACN (Shang et al., 2019) treat the multi-relational KG multiple single-relational graphs, and apply relational graph convolutional network (GCN) for entity representations. Moreover, VR-GCN (Ye al., 2019) combines the translational idea with GNN learn both entity and relation representations. CompGCN (Vashishth et al., 2019) develops three entity-relation composition operators to update (Yu et al., 2021) further incorporates the composition
entity representations in GCN, and with relation update. NBFNet (Zhu et al., 2021) and RED-GNN (Zhang & Yao, 2022) also explore GNN with subgraph for KG completion. Overall, GNN-based models achieve promising results in KG modeling, which demonstrates the importance of capturing structural information.
A.2 HYPERGRAPH & HYPEREDGE EXPANSION
hypergraph generalization of graph, where hyperedge can join any number of nodes (Ouvrard, 2020). Especially, hyperedge expansion (Agarwal et al., 2006; Dong et al., 2020; Zhou et al., 2006) introduced transform hypergraph can work on hypergraphs (Feng et al., 2019; Yadati et al., 2019). Since HKG is viewed as multi- relational ordered hypergraph (Yadati, 2020), here we investigate the representative expansion strategy of star expansion for additional insights.
homogeneous graph, such that graph learning methods
V1
V1
S
12
(a) a hyperedge
(b) star expansion
Figure 5: The illustration of star expansion on a hyperedge.
Figure 5 presents an example of hyperedge with four nodes and its transformed graph by star expansion?. Specifically, for a hyperedge, the star expansion introduces a mediator node (like the blank node the center of Figure 5(b)), which then connected with all original nodes in the hyperedge. With the elegant transformation, hyperedge expansion has been widely applied in recommender systems (Xia al., 2021), link prediction (Sun et al., 2021), etc.
On the other hand, the structural information loss has always been a concerned issue with hyperedge expansion strategy (Arya et al., 2021: Dong et al., 2020: Zhou et al., 2006). To be specific, an expansion strategy on hypergraph suffers from structural information loss, if there can be two distinct hypergraphs on the same node set reduced to the same graph by the expansion (Dong et al., 2020).
"The expansion strategy is named according its graph illustration.
14

---

Under review as a conference paper at ICLR 2023
According (Arya et al., 2021; Dong et al., 2020), the star expansion preserves the complete structural information. However, such traditional hyperedge expansion strategy cannot handle the HKG with hyper-relational semantics considered, which also guides research that both structural and semantic information loss should be concerned transforming a HKG to a KG.
B
METHOD DETAILS
B.1 OTHER VARIANTS OF TRANSFORMATIONS
To demonstrate the effectiveness of our proposed equivalent transformation in Scction 3.1.1, here we further show other variants of transformations in Figure 6. Especially, the plain transformation in Figure 6(a) follows star expansion without attributes considered.
V1
0
by
sub 2g
Fag
obj
6bj
1/2
Ta 02
(a) plain
(b) clique- based plain
(c) clique-based semantic
Figure 6:
The illustration of other variants of transformations.
In comparison to the star expansion, clique expansion also a popular hyperedge expansion strategy (Dong al., 2020; Zhou al., 2006), which transforms the hyperedge into a clique subgraph, i.e., each pair of nodes in the hyperedge are connected in the transformed graph. Thus, we also extend clique expansion into the HKG case, e.g., the clique-based based plain transformation in Figure 6(b) with only primary relations considered. To model the attribute information, in Figure 6(c), for each attribute o,, the clique-based semantic transformation decomposes it into two relations of ra, and .bbj
Ya, which connect the value entity with subject and object entities, respectively. Each pair of value entities are also connected by devised relations berween attributes to satisfy the clique structure. However, these variants of transformations bring information loss while proposed equivalent one preserves complete information, as validated by both theoretical proof and experimental performance later.
B.2 TRAINING PROCEDURE
Algorithm 2: TransEQ training algorithm.
Input:
HKG G (E, R, JH);
Init: E for E E, R for R, Ogn for GNN-based encoder, P for SF- decoder; Build encoder module Enc() with Ogne;
Build decoder module Dec() with Opec
Transform HKG g to KG G with Algorithm 1;
for (. 1.... /iiter do
Sample mini-batch Foath E F.H of size ma. L e 0:
E, R Ene(g, E, R, OEne);
for x (s, r, (ai, vz) {²))=n) E Frunch do
Construct negative samples Ng:
6(x) Dec(x. R, E. R, ODw);
o(x') Dec(x', E, R. ODO)), Yx' E Na:
10
11
Update loss L+ L+ with C, in equation 1;
end
Update learable parameters w.r.t. the gradients VL:
14 end
Output: Embeddings E, R and parameters O Opec
15

---

Under review as a conference paper at ICLR 2023
C THEORETICAL DETAILS
C.I SCORING FUNCTION COMPARISON
In Table 4, we present a comparison of representative HKG modeling studies with SF. Especially, TransEQ can leverage any SF therein for SF-based decoder.
Table 4: A comparison of representative HKG modeling studies with scoring function. pj is the position embedding in G-MPNN. Conv: convolutional neural network, FCN: fully connected network, Trf: Transformer, [: vector concatenation, Mean(.): element-wise average. min(-): element-wise minimization.
Model
Scoring Function
NaLP (Guan et al., 2019)
FCN(min, (Covv ([h... ha]i (hros hols h kv,)))))
m-
DistMult (Fatemi et al., 2020)
(hey h., Noy h ... hu..?
HypE (Fatemi et al., 2020)
(hr Conv Conv(l,), ft,), Conv Conv(h.) Conv (
Conv }}
HINGE (Rosso et al., 2020)
(CN( [Conv( (hr; hx; holl: 3; Conv [hr; has ho; has hor bo, DI)) (hr P1, Po+2, hx, Boy hug hon))
G-MPNN (Yadati, 2020)
StarE (Galkin et al., 2020)
h, FCN(Mean( (Trfeann(T(((ee (,,, bais"" .*. .aa,. Re, Hoys I D)
TransEQ
Arbitrary SF
C.2 COMPLEXITY ANALYSIS ON TRANSFORMATIONS
Based on the description in Section B.1, we analyze the parameter complexity of different transfor- mations in Table 5.
Table 5: The parameter complexity of different transformations, in terms of entity/node, relation and edge. nee |8| and [R] are the number of entities and relations in HKG. n² and n are the numbers of primary and qualifier relations, respectively. na is the maximum number of attribute-valee qualifiers for facts. Aqua and NP are the number of hyper-relational facts with and without attribut-value qualifiers, such that P Nuu
Fl.
Transformation
O eent/C bode
odgo
rel
plain
On, (
O(Ne Nqwa +2)
(
basqu plain
O(ne)
+. + 2)²)
(ne"))
clique-hased
((ne)
O(NN (n, + 2)²)
On - qua)
semantic
equivalent
+Nquu)
O(Nri+N'' (na +3)) O3n +n.
According to the transformation design, clique-based transformations introduce pairwise edges for relatedness while star-based ones of plain transformation and equivalent transformation rely on additional mediator entities. Therefore, clique-based transformations keep the node complexity of (ne) while star-based ones build Om,,+ + Nqux) nodes. On the other hand, the plain transformation keeps the same edge complexity with the original HKG structure, while a relational edge between subject and object entities is added in equivalent transformation for semantic difference, bringing the complexity increase of ((Nuu). Compared with the relation complexity of about On, + +n, the original HKG, the equivalent transformation introduces O(n")) relations to distinguish links between subject and object entities, which are acceptable in practice.
C.3 PROOF OF INFORMATION PRESERVATION
To demonstrate the zero information loss in the equivalent transformation, in Algorithm 3, we present the process that equivalently recover the original HKG from the transformed KG.
Note that N₁, in line is a subgraph, and attribute-vall value qualifiers can be extracted from direct relational links mediator by in line 5. Here we consider hyper-relational fact with least one
16

---

Under review as a conference paper at ICLR 2023
qualifier, while triple facts be directly added in recovered HKG due to mediator. Thus, Algorithm I and Algorithm 3 form an equivalent conversion between HKG and KG.ie. the equivalent transformation preserves the complete information. In comparison, the plain transformation and clique-based plain transformation only keep primary relations in conversion, which cannot be recovered due attribute loss. Besides, clique-based based semantic transformation inherits the structural information loss of clique expansion in hyperedge expansion (Dong et al., 2020).
Algorithm 3: The algorithm for recovering HKG from the transformed KG by equivalent transformation.
Input: Transformed KG G (E, F);;; Init: Recovered HKG G (E" R" ,F) with E + R t =, 0: Obtain the set of mediator entities from &, gmed;
for by E gmned do Find by's 's neighbor entities and their connected relations from F, No., eil= Extract (8, o) from No, via motif-structur discovery; Extract {(a,, u)) from left parts of No
// part of fr corresponds to (aa,)))
EH u R R Ru u {ad?::
F - - F (,,,,,,, o, fla, =));>)))
end
Output: Recovered HKG g" = (E", (C",R".. (").
C.4 PROOF OF FULL EXPRESSIVITY
Our proposed TransEQ firstly transforms a HKG to KG, then develops GNN-based encoder for representation encoding, and calculates plausibility scores based on existing SFs in HKG modeling studies with entity and relation embeddings from encoder part. Mcanwhile, several SFs from HypE (Fatemi et al., 2020), BoxE (Abboud et al., 2020), RAM (Liu et al., 2021), etc., have been proved to be fully expressive with assignment of entity and relation embeddings in their original papers. Hence, with a fully expressive SF in decoder, the TransEQ model is fully expressive if the output embeddings from encoder part follow corresponding assignment required by SF, which is proved as follows, Proof. For 1 E E,r E R, let h. h denote their initialized representations, while h,",, denote corresponding embeddings outputted from encoder part. We also denote h h the required input embeddings of SF in decoder. Then, in mathematical. with h = Enc(h Ogne) and h = Enn(h:: On we should prove h. h.F and h = hS can be achieved with appropriate choice of encoder parameters O and initialized embeddings h. h..
Taking the example of R- R-GCN (Schlichtkrull et al., 2018) as encoder, the message passing process of each GCN layer can be written as,
h
E
[NT1 W'n,, + Wyhh),
PERMENT
where a denotes nonlinear activation function like ReLU, which unnecessary and can be removed (Wu et al., 2019).
Now, we describe a feasible assignment of encoder parameters: For each layer E (1,.... L. and E RR relation-specing matrix W/ is set to null matrix, while W! is set to identity matrix, where both W and W belong to encoder parameters OEnc-
Following the assignment above, we have h h!, i.e., h4 h Hence, we can set the values of fef according to her". In R-GCN, h. is directly initialized and can be set to h Overall, the encoder's output embeddings follow the required embedding assignment SF with above assignment on O hi and h. Thus, the expressivity of TransEQ is proved to be in accord with that of the SF it uses in decoder. Finally, we note that the proof can be trivially extended to other GNN-based encoders like CompGCN (Vashishth et al., 2019) by introducing extra assignments
encoder parameters. D
Note that here simplify the expression of encoder module, which is still in accord with the form in Algorithm 2.
17

---

Under review as a conference paper at ICLR 2023
D EXPERIMENT DETAILS
Here we provide more experiment details to support our claim. Moreover, we further perform experiments five datasets validate the robustness of our proposed TransEQ model.
D.1 DATASET DETAILS
We detail the dataset statistics in Table 6.
Table Dataset statisties.
Dataset
[3]
[R]
#Train
#Valid
#Test
WikiPeople
47,765
707
305,725
38,223
38,281
JF17K
28.645
322
61,104
15,275
24,568
FB-AUTO 3,388
6,778
2,255
2,180
To validate the robustness, we further consider a recently developed dataset WD50K and its variant WD50K (Galkin et al., 2020), where all facts contain qualifiers, i.c., no simple triple facts therein. We also consider four datasets of WikiPeople-3,, JFI7K-3, WikiPcople-4, JF17K-4, developed from (Liu et al., 2020), where facts have a fixed number of qualifiers in accord with the dataset name. Table 7 presents dataset statistics.
Table 7: Dataset statistics.
Dataset
|E
|R|
#Train
#Valid
#Test
WD50K
47,156
532
166,435
23,913
46,159
WD50K(100)
18,792
279
22,738
3,279
5,297
WikiPeople-3
12,270
66
20,656
2,582
2,582
JFI7K-3
11,541
104
27,635
3,454
3,455
WikiPeople-4
9,528
50
12,150
1,519
1,519
JF17K-4
6,536
23
7,607
951
951
D.2 IMPLEMENTATION DETAILS
We implement TransEQ in PyTorch (Paszke et al., 2019) with Adam optimizer. The embedding dimension d is set to the typical size 200 (Abboud et al., 2020; Fatemi et al., 2020; Galkin et al., 2020; Wang al., 2021: Yu & Yang. 2021). The batch size, learning rate and dropout chosen from (64, 128), (0.0001, 0.0005, 0.001, 0.005 ) and [0.1, 0.5] with step 0.1, respectively. Besides, we mainly adopt CompGCN (Vashishth al., 2019) encoder and m-DistMult (Fatemi et al., 2020) as decoder. For the encoder part, the number of GNN layers and sharing ratio a are chosen from (1, 2, 4] and [0.0, 1.0] with step 0.2, respectively. The composition operation in encoder set to rotate function (Sun et al., 2019). We tune hyperparameters over the validation set with early stopping strategy employed. All experiments are run on RTX 2080 Ti GPU.
D.3
EFFICIENCY COMPARISON
0.5
0.6
0.9
0.4
0.5
0.8
0.3
0.4
0.7
0.2
0.3
0.6
0.1
HypE
0.2
C-MPPN
HypE
GMMPNN
HypE
0
4
12
16
20
24
28
6
10
15
30
45
60
Time
Time (hours)
Time (minutes)
(a) WikiPeople
(b) JF17K
(c) FB-AUTO
Figure 7: Comparison on clock time of model training vs. testing MRR.
18

---

Under review as a conference paper at ICLR 2023
Furthermore, compare the learning processes of TransEQ with structural modeling approaches on three datasets in Figure 7. The learning curve of HypE with linear time complexity is also plotted for comparison. It can be observed that TransEQ achieves similar convergence speed with HypE in practice, which owes to the multilinear product based SF (Liu et al., 2021) and efficient implementation. With similar form of SF adopted, G-MPNN achieves a close convergence rate but inferior performance, which demonstrates the strength of GNN-based encoder compared with HGNN, As for StarE with Transformer-basee SF, tremendous parameters lead to time-consuming training on all datasets.
D.4 TRANSFORMATION COMPARISON
To analyze the effects of various transformations, we present the performance comparison in Table Due to space limitation, results with Hit@3 are omitted, which are in accord with other metrics. As described in Section 3.1.1, our proposed equivalent transformation connects subject and object entities via a relational edge to form the motif for semantic difference. Thus, we investigate the effectiveness of such operation by removing the edge in the transformation, referred to as w/o distinction transformation.
Table 8: Performance comparison different transformations. Best results are highlighted bold.
JF17K
FB-AUTO
Transformation
MRR
Hit@
Hit@ 10
MRR
Hit@]
Hit 10
plain
0.504
0.422
0.669
0.836
0.809
0.884
clique-based plain
0.557
0.478
0.706
0.824
0.804
0.859
clique-base bassud semantic
0,552
0.474
0.706
0.831
0.809
0.869
w/o distinction equivalent
0.536
0.456
0.695
0.849
0.810
0.884
0.569
0.489
0.722
0.870
0.842
0.909
From the table, observe that the equivalent transformation outperforms other variants of transformations, which in accord with the information loss analysis in Section 3.2, i.c., only equiv- alent transformation preserves complete information. Moreover, removing the relational edge in the equivalent transformation leads to Hit@1 performance drop of 7% on JF17K, which demonstrates the effectiveness and necessity of considering semantic difference in the transformation. Besides, since any two entities are connected in two clique-based transformations, the relatedness between entities is largely captured and thus they obtain close performance, i.e., the clique structure makes these transformations insensitive to semantic information. In comparison, the star structure in plain transformation and equivalent transformation is quite simple, and additional information including hyper-relational semantics and semantic difference should be incorporated in transformation, which also accounts for the obvious gap between these two transformations. Considering the zero informa- tion loss and experimental performance, the equivalent transformation becomes the best choice for TransBQ in HKG modeling.
D.5 ENCODER-DECOREE CHOICE COMPARISON
We also compare the performance of different encoder-decoder choices of TransEQ models on FB-AUTO in Table 9. These results further validate the observations in Section 4.3.
D.6 ADDITIONAL HKG COMPLETION RESULTS
Since on WD50K and WD50K(100) former studies (Galkin et al., 2020; Yu & Yang, 2021) only predict missing entities at primary triple, not comparable to HKG completion task in Section 4. select competitive baselines in Table 2 and report their performance in Table 10. Here we evaluate TransEQ models with m-DistMult and Transformer as decoders, denoted by TransEQ-DM and TransEQ-Tri respectively. According the table, TransEQ model with Transformer- baasdd decoder generally performs well on both datasets, which again demonstrates the effectiveness of model design.
these datasets, as shown
In Table 11, with m- DistMult and HypE(HP) as decoders, we further investigate TransEQ's perfor- mance on HKG datasets with fixed number of qualifiers, compared with HINGE (Rosso et al., 2020),
19

---

Under review as a conference paper at ICLR 2023
Table 9: Performance comparison of different encoder-decoder choices FB-AUTO. "OOM" indicates out of memory. Best results for each encoder are highlighted in bold.
FB-AUTO
Encoder +
CompGCN
R-GCN
No Encoder
Decoder Y 4
MRR
Hit@ 10
MRR
Hit@ 10
MRR
Hit@ 10
m-TransH
0.825
0.873
OOM
0.728
0.728
Transformer
0.846
0.899
OOM
0.834
0.897
m-DistMult
0.870
0.909
0.834
0.892
0.784
0.845
HypE 0.860
0,902
0.840
0.892
0.804
0.856
Table 10: Results of HKG completion WD50K(1000 and WD50K. Best results are highlighted in bold. "_" denotes exceeding time limit.
WD50K(100)
WDS0K
Model
MRR
Hit@1
Hit@ 10
MRR
@it@1 1
Hit@ 10
Neulnfer
0.289
0.252
0.358
0.179 0,233
0.141 0.158
0.250
HypE
0.436
0.339
0.617
0.377
BoxE
0.672
0.609
0.747
TransEQ-DM
0.553
0.516
0.621
0.290
0.232
0.400
TransEQ-Th Trf
0.661
0.608
0.756
0.343
0.278
0.465
NeuInfer (Guan al., 2020), n-TuckER (Liu al., 2020) and GETD (Liu et al., 2020). Note that n- n-TuckER and GETD can only handle datasets with a fixed number of qualifiers with competitive performance. According the results, TransEQ models obtain the state-of-the-al performance most datasets, indicating the robustness and effectiveness of the proposed equivalent transformation well the generalized encoder-decoder framework.
Table 11: Results of HKG completion datasets with fixed number of attribute value qualifiers. Results of baselines are collected from original papers and (Di & Chen, 2022; Liu et al., 2020).
WikiPeople-3
JF17K-3
WikiPeople-4
JF17K-4
Model
MRR H@ 10
MRR
H@ 10
MRR H@ 10
MRR
H@ 10
HINGE
0.338
0.508
0.587
0.738
0.352
0.557
0.745
0.842
NeuInfer
0.355
0.521
0.622
0.770
0.361
0.566
0.765
0.871
n-TuckER
0.373
0.558
0.727
0.852
0.362
0.570
0.804
0.902
GETD
0.373
0.558
0.732
0.856
0,386
0.596
0,810
0.910
TransEQ-DM 0.382
0.557
0.685
0.827
0.378
0.614
0.820
0.923
TransEQ-HP
0.370
0.557
0.726
0.847
0.394
0.602
0.805
0.908
D.7 ADDITIONAL VISUALIZATION RESULTS
Following the settings Section 4.4, in Figure 8, we compare the visualization results of utilizing independent embedding (a 0.0) and sharing embedding (the best setting with 0.8), which further validates the effectiveness of TransEQ capturing semantic relatedness for mediator entities.
20

---

Under review as a conference paper at ICLR 2023
30
440
(a) a 0.0
(b)
0.8
Figure 8: The visualization of mediator entity embeddings with top ten primary relations in WikiPco- ple via t-SNE share embedding ratio (a) 0.0 and (b) = 0.8.
21


Reviews:
[{"review_id": "uxDmBjgEel4", "reviewer": "Reviewer_94ho", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Studies link prediction for hyper-relational knowledge graphs (in which edges are associated with have attribute/entity pairs). Transforms the hyper-relational KG into a standard KG using reification, then runs a relational GNN for link prediction.\n", "summary_of_the_review": "Simple, potentially useful approach, but little novelty and not insightful.", "strength_and_weaknesses": "Strengths:\n\nS1. Simple approach (but see W1).\n\nS2. Good experimental results (but see W2).\n\nWeaknesses:\n\nW1. Little novelty. The proposed transformation is equivalent to reification (e.g, as used in RDF, not discussed in paper) + keeping the original fact. The introduced blank nodes share (partial) embeddings with their subject entity, which is a good idea but also the immediate approach after a minute of thought. The GNNs being used are out-of-the-box relational GNNs.\n\nW2. Experimental study not insightful. (i) The experimental study does not describe the experimental setup, most notably hyperparameter optimization. (ii) The proposed methods are sometimes close, sometimes further away in quality compared to related work, but it's not explored why. (iii) In the datasets used in the appendix, the proposed method seems to provide little benefits compared to prior work. (iv) The protocol for evaluating \"link prediction\" in the experimental study is not convincing: It uses a ranking protocol where only one of the entities in a hyperfact is hidden (e.g., degree) but the others are specified (e.g., university). This problem statement does not appear to be model a practical use case. How would one use such a method to predict a new fact and how good would it be? (v) There are no ablation studies to justify the modelling choices (e.g., just make everything binary, or do not keep the original triple after reification).\n\nW3. Arguments about \"semantic difference\" unclear. For a fact such as \"A graduated at B obtaining degree C\", the paper argues that the \"graduated at\" relation is a primary relation, whereas the degree is auxiliary. This argument is not convincing (one may argue the other way around) to me, and, in fact, the is little justification of the need to capture \"semantic difference\". Moreover, even if reification is used (i.e., without keeping the original fact) the HKG is still reconstructable.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "h8gZQ-vWUz", "reviewer": "Reviewer_VWAf", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the authors proposed a model named TransEQ for hyper-relational knowledge graph modeling. In the proposed pipeline, they first convert the hyper-relational knowledge graph into normal knowledge graphs by transforming hyper-relations into mediator entities and relations while keeping the structural and semantic information; then, a graph neural network based model is used for knowledge graph encoding, and hyper-relational knowledge graph scoring functions are used for prediction.  Experiments on three datasets demonstrate the effectiveness of TransEQ.\n", "summary_of_the_review": "To summarize, the paper is well-written and easy to follow. The HKG to KG transformation method is intuitive and easy to implement. However, the weaknesses of the paper cannot be ignored. The reviewer is concerned about the complexity, generalization capability, and correctness of the evaluation, which makes the reviewer give a negative recommendation.", "strength_and_weaknesses": "### Strength \n- The transformation idea is straightforward and well-motivated. The authors provide brief proof of the full expressivity yet it can be done in a more detailed way.\n- They conducted thorough model analyses (complex analysis, comparison of model properties, hyperparameters, etc.)\n\n### Weaknesses\n- The expressiveness of the hyper-relational knowledge graph transformation is highly dependent on the choice of scoring function, which may limit the generalization capability.\n- The transformation will produce a large number of additional edges and entities, making the learning and inference more expensive, as pointed out in section 3.2, they have to use parallel implementation to accelerate the process. \n-  Some baselines are missing; please refer to [a][b][c[. Regarding the baselines, The reviewer is confused about the performance of StarE; according to Table 2 in the StarE paper, the best performance of StarE is higher than the reported one in this paper. For example, the best performance on WikiPeople is (MRR: 0.491; H@1:0.415, H@10: 0.648); The best on JF17K are (MRR: 0.574; H@1:0.496, H@10: 0.725). It is noted that StarE outperforms the proposed model on all metrics.\n\n\nReference:\n[a] Generalizing Tensor Decomposition for N-ary Relational Knowledge Bases, WWW 2020\n[b] Role-Aware Modeling for N-ary Relational Knowledge Bases, WWW 2021\n[c] Searching to Sparsify Tensor Decomposition for N-ary Relational Data, WWW 2021", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "I6ERbmdr2be", "reviewer": "Reviewer_cp9m", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a comprehensive method to model the hyper-relational knowledge graph. The authors transform a HKG into a KG by introducing extra entities and relations which constructing a one-to-one mapping between the constructed KG and the previous HKG. The transformation is information preserving which is intuitive and widely used in other research domain such as graph theory. The author then design cured  function to aggregate information for newly created entities. And feed into an Encoder-Decoder framework for model training. The experiment result for three datasets shows the improvement for such graph transformation.", "summary_of_the_review": "The proposed method is simple yet effective. The technical contribution is limited. The experiment settings omit a lot of information which prevent fair comparison. Hence I would recommend \"Discussion\" to get feedback from the authors.", "strength_and_weaknesses": "Strength:\n1. The proposed method is intuitive and effective\n2. The proposed graph transformation bridges the algorithms between KG modeling and HKG modeling.\n\nWeaknesses:\n1. In Sec 3.1.2 under the GNN-based Encoder. Initialized embedding alpha setting is kind of ad-hoc (grid search in the experiment). An insightful guidance for setting such hyper-params for new dataset will be helpful.\n2. In Model Training part. The negative sampling procedure is unclear since the graph itself is not the graph original HKG task. The negative sampling procedure may influence the result.\n3. Possibly unfair comparison in the experiment. In Sec 4.2, the author indicated the task in their setting is different from StarE, which include more predicting task. However, important detailed information is missing such as how they handle facts with different number of qualifiers, how they adapt the baseline which doesn't support predicting entities in qualifiers into their setting.\n4. A lot of explanations are in appendix, which block the reader from following the main stream of the paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5SsGflDDyD", "reviewer": "Reviewer_FnvL", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes TransEQ: a method for hyper-relational knowledge graph embedding. The motivation behind TransEQ is considering both semantic and structural information in a hyper-relational knowledge graph. The proposed method achieves state-of-the-art results on established datasets and strong theoretical properties.\n", "summary_of_the_review": "The method used in the paper to transform an HKG into a KG is almost identical to the reification method discussed in [1]. Fatemi et al [1] showed that transforming the HKG into a KG (conversion approaches) using reification (or S2C) and applying SFs does not work. It is exciting to see that transforming an HKG into a KG and using GNNs+FSs yield better results. I appreciate it if the authors can add detail about this to their paper.\n\nI do not agree with the paper on the importance of semantic information in an HKG. The paper implies that for a hyper-relation educated at, the school is more important than the degree that the corresponding person is educated. This is, however, not given in many datasets and might vary from one dataset to another. Regarding this, can the authors elaborate on how the primary triple in a hyper-relation is identified? Especially for the datasets used in this paper. \n\n[1] Fatemi, Bahare, Perouz Taslakian, David Vazquez, and David Poole. \"Knowledge hypergraphs: Prediction beyond binary relations.\"", "strength_and_weaknesses": "This paper studies the important problem of modeling hyper-relational knowledge graphs. \n\nDespite the simplicity, the proposed approach in the paper achieves state-of-the-art experimental results and strong theoretical properties.\n\nThe experimental results are reported on various datasets with different statistics.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject