Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
Authors: Jaejun Lee
Joyce Jiyoung Whang
Venue: ICML 2025 poster (2025)
Keywords: Hyper-Relational Knowledge Graph
Knowledge Graph Representation Learning
Knowledge Graph Completion
Graph Neural Networks
Inductive Link Prediction
Primary area: deep_learning->graph_neural_networks
TL;DR: Thoroughly leveraging the structure of a hyper-relational knowledge graph (HKG) is crucial for reasoning on HKGs, and a purely structure-based representation learning method can achieve state-of-the-art performance on various link prediction tasks.

Abstract: Hyper-relational knowledge graphs (HKGs) enrich knowledge graphs by extending a triplet to a hyper-relational fact, where a set of qualifiers adds auxiliary information to a triplet. While many HKG representation learning methods have been proposed, they often fail to effectively utilize the HKG's structure. This paper demonstrates that thoroughly leveraging the structure of an HKG is crucial for reasoning on HKGs, and a purely structure-based representation learning method can achieve state-of-the-art performance on various link prediction tasks. We propose MAYPL, which learns to initialize representation vectors based on the structure of an HKG and employs an attentive neural message passing consisting of fact-level message computation and entity-centric and relation-centric aggregations, thereby computing the representations based solely on the structure. Due to its structure-driven learning, MAYPL can conduct inductive inferences on new entities and relations. MAYPL outperforms 40 knowledge graph completion methods in 10 datasets, compared with different baseline methods on different datasets to be tested from diverse perspectives.

Structure Is All You Need: Hyper-Relational Knowledge Graphs
Structural Representation Learning on
Jaejun Lee!
Joyce Jiyoung Whang'
Abstract
et al., 2020), where a qualifier consists of a relation and an entity, e.g., (country, Australia). In HKGs, each piece of information is represented as a hyper-relational fact defined as a triplet and its qualifiers, c.g. ((Finding Nemo, set in, Sydney), Australia), (state, New South Wales))).
Hyper-relational knowledge graphs (HKGs) en- rich knowledge graphs by extending a triplet to hyper-relational fact, where set of qualifiers adds auxiliary information to triplet. While many HKG representation learning methods have been proposed, they often fail to effectively utilize the HKG's structure. This paper demonstrates that thoroughly leveraging the structure of an HKG is crucial for reasoning on HKGs, and a purely structure-based based representation learning method can achieve state-of- the-art performance var- ious link prediction tasks. We propose MAYPL,, which learns to initialize representation vectors based on the structure of an HKG and employs an attentive neural message passing consisting of fact-level message computation and entity-centric and relation-centric aggregations, thereby comput- ing the representations based solely the struc- ture. Due to its siructure-rrivn learning, MAYPL can conduct inductive inferences on new entities and relations. MAYPL outperfors 40 knowledge graph completion methods in 10 datasets, com- pared with different baseline methods on different datasets to be tested from diverse perspectives.
Reasoning and inferring new knowledge HKGs neces- sitates understanding and exploiting intricate relationships among entities and relations. However, most existing stud- ies on HKGs do not sufficiently utilize the graph structure of HKGs. For example, transformer-based methods (Wang et al., 2021; Yu & Yang. 2021; Chung et al., 2023) process each hyper-relational fact (referred to as a 'fact" for brevity) individually, failing to capture the interconnectednnss be- tween different facts. Some other recent methods (Hu et al., 2023) use simple one-hop neighbourhood information to learn representations on HKGs, While StarE (Galkin et al., 2020) proposes a graph neural network (GNN) layer for encoding a fact, it turns out that the proposed GNN encod- ing is redundant and does not critically affect the overall performance (Yu & Yang, 2021). Recently, HAHE (Luo et al., 2023) introduces another GNN-based encoding, but it does not consider relations and the positions of entities within each fact. Like this, existing HKG methods incor- porate only limited structural information and fail to utilize the structures of HKGs effectively.
1. Introduction
In this paper, we demonstrate that thoroughly leveraging HKG's strueture is crucial for successful reasoning on HKGs, and purely structure-based representation learn- ing can achieve state-of-the tett-hh-rr art performance on a range of transductive and inductive link prediction tasks. We propose a structure-driven representation learning method, named MAYPL (Message pAssing framework for byper-relational knowledge graph rePresentation Learning), that learns to ini- tialize the representations of entities and relations based on their interconnections, co-occurrence, and positions within fact. By compressing the information of which entities and relations are included and how they are connected within each fact, MAYPL computes fact-level messages, which are attentively aggregated to entities and relations. Along with the fact-level structure, the component-level connectivity is also considered in MAYPL's attentive neural message
Knowledge graphs (KGs) provide structured representa- tion of human knowledge, which eases organizing the re- lationships between concepts or objects (Ji et al., 2022).
In KGs, information is represented in the form of a triplet (Toutanova & Chen, 2015; Bordes et al., 2013), e.g., (Finding Nemo, set in, Sydney). Since the sim- ple triplet form oversimplifies information, hyper-relational knowledge graphs (HKGs) have been recently proposed to add auxiliary details by adding qualifiers triplet (Rosso
'School of Computing, KAIST, Daejcon. South Korea, Corre- spondence to: Joyce Jiyoung Whang <jjwhang@ kr.>
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Structure Is All You Need: Structural Representation Learning on Flyper-Relationa. Knowledge Graphs
Table 1: Comparison of MAYPL and existing methods.
heads F1 (1i talle P2))
(1% P4)
Fact #2
D5
(/2 head F1)
(f2 tails Pi))
(/2, Pt))
Know. Rep. Ind. Inf.
72
or
12
(/' heads F2)
(hi, P2l
(r3, P4)
KG KHG NRR HKG ent. rel.
Pi
InGram (Lee et al., 2023b)
V'
h/ F1)
(/2
(/2, v/))
RMPI (Geng ct al., 2023)
V
72
(b) N-ary Relational Representation (NRR)
G-MPNN (Yadati, 2020)
HCNet (Huang et al., 2024)
13
17
2
NaLP (Guan al., 2019)
RAM (Liu et al., 2021b)
Fact #1
StarE (Galkin et al., 2020)
HAHE (Luo et al., 2023)
(a) Hyper-rclational
Juf
HyNT (Chung al., 2023)
Knowledge Graph (HKG). (c) Knowledge Hypergraph (KHG)
QBLP (Ali et al., 2021)
Figure I: HKGs cannot be transformed into NRRs or KHGs
MAYPL
without losing information. Details are in Section 2.
passing mechanism. From initialization to link prediction, MAYPL utilizes only the structure of given HKG. Since MAYPL learns how to compute, propagate, and aggregate messages over a given HKG's s structure, MAYPL can also handle a new HKG that is different from a training HKG.
MAYPL outperforms 40 different methods on 10 benchmark datasets, where each benchmark is intro- duced to test a different perspective of a method."
2. Related Work
While real-world knowledge bases, such as Wiki- data (Vrandečié & Krötzsch, 2014) and YAGO (Suchanek al.,2007 2007), take the form of HKGs, and many recently pro- posed methods directly handle HKGs (Galkin et al., 2020; Chung et al., 2023; Luo et al., 2023), some transform the original HKGs into knowledge hypergraphs (KHGs) (Ya- dati, 2020) or n-ary relational representations (NRRs) (Guan et al., 2019; Liu et al., 2021b), putting up with losing some information provided in the original HKGs. Table com- pares MAYPL and other existing methods in terms of (i) the form of knowledge representations (Know. Rep.) and (ii) the capability of inductive inference (Ind. Inf.). For the inductive inference, we consider whether a method can han- dle new entities (ent.) and new relations (rel.). We provide the details of the compared methods in Section 2 and com- plete comparison including more methods in Appendix A. MAYPL is the only HKG-handling method that can conduct inductive inference with new entities and relations. Since MAYPL learns how to compute the representations using an HKG's structure, what it learned can be seamlessly applied to new HKG. Our main findings and contributions are:
N-ary Relational Representations and Knowledge Hy- pergraphs NRRs (Guan et al., 2019; Liu et al., 202 and KHGs (Fatemi et al., 2020; Abboud et al., 2020) have been studied to add supplementary information to KGs. While each fact is represented using a primary triplet and a set of qualifiers in HKGs, NRRs (Wen et al., 2016) express cach fact using set of role-value pairs. On the other hand, in KHGs, each fact is converted into a tuple in the form of r(v,,-..,, where the order of entities matters. Figure 1 shows the conversions from an HKG to an NRR or KHG, In the original HKG shown in Figure 1(a), itss and Us are both incident to due to the facts and ((01.)2,08), {(r2, ug)]). However, on the NRR, shown in Figure 1(b), (U1.22,u5) is decomposed into (r2 (1) and (r2 Us), and thus, Vt is now incident to different re- lation from 13. Alternatively, one can decompose the triplet into (F2_ ) and (?2,%s), in which case, one cannot iden- tify the tail entity of r2 since (r2,0z) also exists. In the KHG, shown in Figure 1(c), much information is also lost, e.g.. the information that 03 is incident to r'2 as a qualifier in both facts is lost. Therefore, an HKG cannot be transformed into an NRR or KHG without losing information, which indicates that it is desirable to handle HKGs directly instead of converting them into other forms.
Employing an HKG's structure is crucial for HKG reasoning, and purely structure- based learning can suc- cessfully solve various link prediction problems. MAYPL is the first structure-oriented representation learning method for HKGs that can be applied in both transductive and inductive learning settings.
NaLP (Guan et al., 2019) and RAM (Liu et al., 2021b) have been proposed for NRRs, considering the relatedness between the pairs (Guan et al., 2023; 2019) or the role-aware modeling (Liu et al., 2021b). For KHGs, G-MPNN (Yadati,
By learning how to compute representations based solely on the structure of a given HKG, MAYPL can make inductive inferences where new entities and rela- tions appear at inference time.
'Our codes are available at https: htips:///ittub.com ctmbb.ccomm bdi- oddilab//ee lab/MAYPL.
2

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
2020) and HCNet (Huang et al., 2024) have been proposed, which can partially solve inductive link prediction. While G- MPNN can handle new entities only one-hop distant from observed entities, HCNet can handle the case where all entities are new at inference time. Both G-MPNN and HCNet cannot handle new relations because they assume that all relations are observed during training time and learn representations specific to the observed relations.
In hyper-relational fact ((11, "1,7), 02), (12, 13))), where (11,71, 02) is primary triplet and (r2, v2) is qualifier, we indicate the position of each entity as follows: di is a head entity, U2 is tail entity, and V3 is qualifier entity. Similarly, the position of a relation is specified as follows: ri1 is a primary relation and r2 is a qualifier relation. Transductive link prediction on HKGs are described in Definition 3.2.
Definition 3.2 (Transductive Link Prediction on HKGs). Given an HKG, G = (V. R, H). H is decomposed into three pairwise disjoint sets, such as = Hrr U Heal U Hiss where Hir is a training set, Hoat is validation set, and Har is test set. A transductive link prediction on an HKG is to predict missing entity in a hyper-relational fact h e Hus-
Representation Learning on HKGs
Many recently pro-
posed HKG methods utilize transformers ( (Wang et al., 2021; Yu & Yang, 2021; Chung et al., 2023; Galkin et al., 2020), including StarE (Galkin et al., 2020) and HyNT (Chung et al., 2023). While some methods rarely take into account the graph structure of HKGs and focus on processing facts individually (Rosso et al., 2020; Guan et al., 2020; Yan et al., 2022; Xiong et al., 2023), others consider direct neighbors of entities (Hu et al., 2023) or a GNN-based encoding (Luo et al., 2023). For example, HAHE (Luo et al., 2023) updates entity representations using a GNN layer, though it ignores the information about relations and the positions of compo- nents. All these methods rely on learning representations of fixed entities and relations, which limits handling new entities or relations at inference time. Unlike these, MAYPL utilizes the fact-level, entity-level and relation-level connec- tivity structures along with their compositional and posi- tional information, which enables inductive link prediction on an entirely new HKG with all new entities and relations.
On the other hand, in inductive link prediction, model is trained on training HKG and makes predictions on an inference HKG, as detailed in Definition 3.3.
Definition 3.3 (Inductive Link Prediction HKGs), Con- sider a training HKG, G = (V, R. H) and an inference HKG, G - (². RR H),, where V ae V or R. th R. Note that H is a training set, and H. is decomposed into three pairwise disjoint sets, such as H. Hea U Haal HAa.. where Hia is a set of observed facts, Hval is a validation set, and Hut is test set. An inductive link prediction on HKG is to predict missing entity in h Hest using Hes.
4. Structural Representation Learning on Hyper-Relational Knowledge Graphs
Inductive KG Completion Inductive KG completion methods have been proposed to predict missing links in a new KG different from a training KG (Teru et al., 2020; Lee al., 2023b; Geng et al., 2023). While most existing inductive KG methods assume that only entities are new and the relations are identical to the training set (Teru et al., 2020; Zhu et al., 2021), a few recent methods, such as In- Gram (Lee et al., 2023b) and RMPI (Geng et al., 2023), allow both entities and relations to differ from the training set, though they cannot handle HKGs. While QBLP (Ali
Figure 2 shows an overview of MAYPL consisting of a structure-drive driuen initializer, attentive neural message passing, and link prediction. While the structure-dis -riiven initializer computes the initial representations of entities and relations based on their interconnected structure, the attentive neural message passing module encodes fact-level messages using the entity and relation representations based on the facts' composition information, which are attentively aggregated to entities and relations considering their positions. MAYPL predicts a missing query entity in an incomplete fact by selecting the entity with the highest similarity to the query.
et al., 2021) assumes new entities but not new relations in HKGs, it requires external text features of entities, and its inductive capability comes from these external features. In contrast, MAYPL can make inductive inferences on new entities and relations in HKGs without any extra features.
4.1. Structure-driven Initializer of MAYPL
A simple message-passinng passing- based initializer, referred to as a structure-driven initializer, is proposed, which exploits the interconnections, co-occurrence, and positions of entities and relations. In the initializer, the representation of an en- tity u computed by considering the entities that co-occur with e in the same facts and the relations incident to Sim- ilarly, the representation of relation is computed based on the relations that co-occur with r in the same facts and the entities incident to r. In these processes, the positions of entities and relations within the facts and the positions of entities for their incident relations are considered. The
3. Problem Definitions
Definition 3.1 (Hyper-relational Knowledge Graph). A hyper-relational knowledge graph is defined as G (V, R. H) where V is a set of entities, R. is set of rela- tions, and H. is a set of hyper-relational facts. A hyper- relational fact h. e H is defined as h E T x P(Q) where
T V x R x V is a set of triplets, Q S R x V is a set of qualifiers, and P()) is the power set of Q
3

---

Structure Is All You Need: Structural Representation Learning on Elyper-Relationa Knowledge Graphs
1. Structure-Driven Initializer
Attentive Neural Message Passing
Updating Entities
Computing Facts
Updating Entities
P1
r2
72
T2
2
13
T33 Messages from Relations
I'' Message Computation
reper-relational Knowledge Graph
Messages from Entities
Message Aggregation
Il Link Prediction
Updating Relations
Updating Relations
22
Dot Product Similarity
MAYPL
12
T2
Message PAssing framework for hY perr-relaioonal knowledge graph rePresentation Learning
Messages from Relations
Messages from Entities
Message Aggregation
Figure 2: MAYPL consists of a structure- driven initializer, an attentive neural message passing, and link prediction. The structure-die -rriver initializer initializes the entity and relation representations by exchanging messages among co-occurring entities and relations within the same facts and considering their positions. In attentive neural message passing, fact's message is computed based on its entities and relations, and the entity and relation representations are updated by attentively aggregating the facts' messages to which they belong. Link prediction is performed using the dot product similarity.
initializer outputs the initial representations of entities de- noted by (0)) E Redo for e e V and of relations denoted by (())) e Edd for r E R, where do is the dimension of the initial representation.
and R. is the set of relations incident to v. The message aggregation function for entities, AGGR!! ent is defined by
1 > An(o) W An(w)
AGGR
Σ
MHnnHo
Let the superscript (0) of a vector or a matrix indicate the 7-th layer, where 7 1,... L and is the total num- ber of layers in the initializer. We set learnable vectors Vinit, Pinit E ido that are shared across all entities and re- lations, respectively. Let v()) E Rd denote an entity rep- resentation vector of in the initializer, and (0) = Vinit- Similarly, let (()) de denote relation representation vector of r in the initializer, and ((()) = Finit- To indicate the position of an entity or a relation a fact h, we define a ansition-indicator function (z) that returns the position of 2 in h. If is an entity, (z) returns head entity, tail entity, or qualifier entity. If = is relation, (z) returns primary relation qualifier relation. Similarly, (+()) is defined to return the position of an entity v for a relation r: a head entity, tail entity, or qualifier entity.²
we he Ao
Σ (0)))))))))))))))))))))))))) [R.. reR.
where Hy is the set of facts that entity v belongs to, ( (n))) W (u) E do are learnable matrices that are learned depending on the positions of the entities v and u (
in h, respectively, and A. A,,(v) is learnable matrix that is learned depending on the position of with respect to r.
Similarly, relation r's representation is computed by ag- gregating the messages of relations that co-occur with in facts and the entities that are incident to r:
pD A (15(111) E R.), (M0 v.))
where R, is the set of relations that co-occur with r in facts and V, the set of entities that are incident to r. The message aggregation function for relations, AGGR ( 7011 is
In the initializer, an entity v's representation is computed by aggregating the messages the entities that co-occur with in facts and the relations that are incident to e:
@ AGGR HantH
Σ
Σ
ih W An(r)
An An
!) sat ( @
pu Vel, (x( Ir R..))
201
yeRe ho
vER.
Σ
where V, is the set of entities that co-occur with v in facts
Wev,
WWhile we assume that is unique in h in >x(2) for brevity, implementation correctly returns the position of 2 even if is not unique; the same also applics to Ty (T) Refer to our codes.
where H, is the set of facts that relation r belongs to,
(a(r); W An(a) E Rdoxdo are learnable matrices that are

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
learned depending on the positions of the relations and y (i)
where p() (v) E RRdd W U
(n)s As (t) R are
in h, respectively, and A A.(()) is a learnable matrix that is learned depending on the position of o with respect to r,
learnable projection matrices which are learned depending on the position of v in h. and O is the Hadamard product. By considering pd) for Vp e (R x V),, a message of is computed as follows:
Motivation and Principles Our structure-drived initial- izer is designed to compute the initial representations of entities and relations purely using the structure of an HKG, contributing to MAYPL's inductive inference capability. From a single learnable vector Vin shared across all entities the beginning, (1) is updated for u V using AGGR ant what is learned during this process is how aggregate mes- sages from the co-occurred entities and incident relations of each entity. By introducing different learnable projection matrices for different positions of entities and relations, the position information is also considered in this aggregation.
h h I(R X V)al
Σ
D
(I)
pE(RXV),,
which implies that a message of h is computed by the position-aware aggregation of its relation-entity pairs.
UPDATE OF ENTITY REPRESENTATIONS
For an entity e E V, let us consider a pair of its incident relation and the corresponding fact. Let q (r, h) (R H), denote a pair of relation r' and fact h where is incident to r in h and (R H)) is the set of relation-fact pairs for v. A message q of q is computed by
Similarly, for relations, AGGR!! (i)
rel learns to aggregate mes- sages from the co-occurred relations and incident entities of each relation. Both AGGR ent and AGGR (1) can be di- rectly applied to new HKG that is different from training HKG since AGGR! (1)
(1)
q = -000000 An(
(1) + Anto)
(A
("-
(2)
ent and AGGR rel have learned how to aggregate messages on an HKG using its structure. After L layers, ((L) is computed for e V and p(( computed for R, which are fed into the following attentive neural message passing module.
(v) E Rddi are learnable matrices learned depending on the position of e in h. Note that qd)) is a message for v, which reflects the represen- tations of its incident relation and the corresponding fact. Finally, an entity representation v(t) of v is computed by an attentive aggregation of q()) for Vig E (R. x H), follows:
where Qx)(o) E RRdi A (1)
4.2. Attentive Neural Message Passing
After initializing the representations of entities and rela- tions, MAYPL updates the entity and relation representa- tions by message computation and aggregation. Specifically, MAYPL considers (i) which entities and relations comprise a fact when computing a fact's message, (ii) which facts an entity belongs to when updating an entity representation, and (iii) which facts a relation belongs to when updating a relation representation. In this process, not only these com- position and connectivity information but also the positions of the entities and relations within each fact are considered. Let (!) e dh denote an entity representation of v, where di is the dimension at the 1-th layer, = I,... I., and T, is the total number of layers. Let r(O) e d denote a relation representation of r. Note that (0) (L) and (())) = x(8)
Σ qE(Rx
(1) 9 q,0
!I
(3)
H),
(0) a (Q"V K kOI"))
exp
9,0
Σ exp (al). (Q()((-1)) + K k(O(())
Ar (R(X)))
where Q (2) Rdixd-1 K(), B(0 e Rddx are learnable projection matrices, a E Rdd is a learnable vector, and
a-) is an activation function. Note that ag is the relative importance of to v among all relation-fact pairs of v.
UPDATE OF RELATION REPRESENTATIONS
Similarly, for relation r E R, we consider pair of its incident entity and the corresponding fact. Let = (v, h), (V x H), denote a pair of an entity and fact h where r is incident to u in h and (V x H), is the set of entity-fact pairs for r. Then, a message q1) of 7 is computed by
COMPUTING FACTS' MESSAGES
Given a fact h E H. let us decompose as a set of its relation-entity pairs. For example, ((m,, ri, u2). (122, o2))) is decomposed by {(11, 01). (11, 02), (r2, 0z)]. Let p (r, ()) E (RR x V), denote pair of relation and an entity in h, where is incident to and (R V), is the set of relation-entity pairs for h. Then, message (!)) of is computed by
q QQ
hh"
(A ((-)))
(4)
where @ (x) E Ri is learnable matrix learned de- pending on the position of r in h, and A Xa (v) E dd is learnable matrix learned depending on the position of
( = p()) (o) (WWM) An(os v((-1) o (Ur))) As (()())((111))
5

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
in h. Note that q is a message for r, which reflects the representations of its incident entity and the corresponding fact. Finally, relation representation r(0) of r is computed by an attentive aggregation of q for Vq E (V> x H),
ing HKG, G (Definition 3.3). At inference time, MAYPL utilizes the fact set His of & to initialize the representations of entities and relations (Section 4.1), update them (Sec- tion 4.2), and perform link prediction (Section 4.3). Note that all the model parameters needed in this process are learned during training; the learned parameters can be di- rectly applied to the inference HKG that can be different from a training HKG because, from initialization to link prediction, what MAYPL learns is how compute and ag- gregate messages purely based on the structure of an HKG.
r())
Σ
a()). o Q (1)
((11)) + K (1 q()))
exp
(1)
0g.r
Σ exp all)) (E(VxHH)
Q ((-1) + K k((((())
(1)
where Q") E Rd K (1) ( Rdi are learnable projection matrices, a()) E Rd is learnable vector, and (-)) is an activation function. Note that ag. is the relative importance of q to among all entity-fact pairs r.
5. Experiments
We use diverse baseline methods presented in Appendix and different benchmark datasets detailed in Appendix B. All results of baselines are obtained from the baselines' original papers or from the papers that introduced the datasets (Galkin et al., 2020; Guan et al., 2019; Ali et al., 2021; Yadati, 2020; Lee et al., 2023b). In Appendix C, we present which methods are run on which datasets and which results are obtained from which papers. In all tables of ex- perimental results, ''' indicates that no result is available for model on a benchmark from either the model's original paper or the benchmark's ' paper; the best result is boldfaced, the second-best result is underlined, and the third-best result is italic. The implementation and experimental details of MAYPL are described in Appendix D. We measure link prediction performance using three standard metrics: Mean Reciprocal Rank (MRR), Hit10, and Hitl.
(!)
Summary When encoding a fact in EI,()), each fact is decomposed into a set of relation-entity pairs that comprise the fact, and the fact's message is computed by aggregat- ing the pairs' representations. Each entity's ' representation is updated by attentively aggregating the facts' messages to which the entity belongs and the representations of the corresponding relations, shown in (2) and (3). Rach relation's 's representation is updated by attentively aggregat- ing the facts' messages to which the relation belongs and the representations of the corresponding entities, shown in (4) and (5) In all these computations and updates, the positions of entities and relations are considered.
4.3. Link Prediction on HKGs
5.1. Transductive Link Prediction on HKGs
For link prediction during training, an incomplete fact is made by dropping an entity from a complete fact in a train- ing set. For an incomplete fact h, let z denote the missing en- tity, also called a query entity, and assume that x is incident to relation The initial representation of x is computed by the structure-driven initializer: starting from x0) = Vinit, it is updated by x = AGGR ent ((v(ī))) V2), ()) where ī = 1,... T. and V. is the set of entities that co- occur with æ in h. Then, given x(") = x(2), the attentive neural message passing applied by computing the mes- sage of h by h())
Table shows the transductive link prediction results on HKGs using WD50K (Galkin et al., 2020) and WikiPeople" (Wang al., 2021) datasets, where all meth- ods are trained using the training set and evaluated on the validation set for hyperparameter tuning and then trained again using the union of the training and validation sets with the selected hyperparameters and tested to predict missing entities on the facts in the test set. Note that this is the standard evaluation protocol for these datasets (Galkin et al., 2020; Wang et al., 2021), and we also follow this protocol. Models predict entities in the primary triplets (Tri) or all entities in facts (All). Also, Table 3 shows the results on WikiPeople (Guan et al., 2019), where the models predict all entities. Overall, MAYPL outperforms the basclines on these transductive link prediction tasks.
[TRXD]aT -))(R ((), followed where 1, ................................ L and all the model parameters are explained in Section 4.2. By computing the dot product similarity be- tween (z) and each entity representation. MAYPL predicts the missing entity as the entity with the highest similarity; the cross-entropy is used for the training loss.
(1) (1)
(()) (Qt) (x)))) (h") O (A As
5.2. Inductive Link Prediction on KGs
Table 4 shows inductive link prediction results on KGs us- ing NI-50, WK-50, and FB-50 datasets, where all entities are new, and half of the triplets include new relations at an inference KG (Lee et al., 2023b). We observe that MAYPL outperforms the baseline methods with a considerable gap
The process of link prediction at inference time is identical to what is described above, except each incomplete fact is made from a test set instead of a training set. In inductive link prediction, an inference HKG, G, differs from a train-
6

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
Table 2:
Transductive link prediction on WD50K and WikiPeople Mod- Table 3:
Transductive link prediction on all entities in facts in WikiPeople.
els predict entities in the primary triplets (Tri) or all entities in facts (All).
WDSOK
WikiPeople
WikiPeople
MRR
Hit10
Hitl
MRR
Hit10
Hidl
MRR
Hitl0
Hitl
NaLP
0.177
0.264
0,131
RAE
0.172
0.320
0.102
HINGE
0.243
0.377
0.176
NaLP
0.338
0.466
0.272
StarE
0.349
0.496
0.271
0.491
0.648
0.398
HypE
0.282
0.487
0.148
GRAN
0.503
0.620
0.438
NeuInfer
0.350
0.467
0.282
Transsrnffrmmer
0.356
0.498
0.281
0.501
0.634
0.426
RAM
0.380
0.539
0.279
Tri
HyperFormer
0.366
0.514
0.288
0.473
0.646
0.361
$2$
0.372
0.533
0.277
HAHE
0.368
0.516
0.291
0.509
0.639
0.447
PolygonE
0.431
0.568
0.334
ShrinkE
0.345
0.482
0.275
0.485
0.601
0.431
HyConvE
0.362
0.501
0.275
HyNT
0.357
0.501
0.281
MSeaKG
0.392
0.553
0.290
tNaLP
0.350
0.471
0.288
MAYPL
0.381
0.544
0.297
0.519
0.657
0.444
HCNet
0.421
0.565
0.344
HAHE HyNT 0,383
0.402
0.546
0.327
0.495
0.631
0.420
HyCubE
0.448
0,592
0,368
0.527
0.308
HJE
0.450
0.582
0.375
All
MAYPL
0.411
0.572
0.326
0.521
0.659
0.446
MAYPL
0.488
0.635
0.405
Table 4: Inductive link prediction performance on KGs using NL-50, WK-50, and FB-50 datasets.
NL-50
WK-50
FB-50
MRR
Hit10
Hitl
MRR
Hit10
Hitl
MRR
Hitl0
Hitl
GralL (Teru ct al., 2020)
0.162
0.288
0.104
COMPILE (Mai et al., 2021)
0.194
0.330
0.125
SNRI (Xu al., 2022)
0.130
0.187
0.095
INDIGO (Liu et al., 2021a)
0.167
0.217
0.134
RMPI (Geng et al., 2023)
0.185
0.307
0.109
CompGCN (Vashishth et al., 2020)
0.003
0.005
0.000
0.003
0.002
0.001
0.004
0.006
0.002
NodePiece (Galkin al., 2022)
0.037
0.079
0.013
0.008
0.013
0.002
0.021
0,048
0.006
NeuralLP (Yang et al., 2017)
0.101
0.190
0.064
0.025
0.054
0.007
0.088
0,184
0.043
DRUM (Sadeghian et al., 2019)
0.107
0.193
0.070
0.017
0.046
0.002
0.101
0.191
0.061
BLP (Daza et al., 2021)
0.041
0.093
0.011
0.041
0.092
0.013
0.078
0.156
0.037
QBLP (Ali et al., 2021)
0.048
0.097
0.020
0.035
0.080
0.011
0.071
0.147
0.030
NBFNet (Zhu al., 2021)
0.225
0.346
0.161
0.062
0.105
0.036
0.130
0.259
0.071
RED-GNN (Zhang Yao, 2022)
0.179
0.280
0.115
0.058
0.093
0.033
0.129
0,251
0.072
InGram (Lee et al., 2023b)
0.281
0.453
0.193
0.068
0.135
0.034
0.117
0.218
0.067
HyRel (Yang et al., 2024)
0.321
0.520
0.222
0.068
0.138
0.036
0.178
0.333
0.1
AAGGE-EIPR (Mu et al., 2024)
0.315
0.529
0.217
0.076
0.164
0.038
0.204
0.376
0.123
MAYPL
0.343
0.508
0.262
0.109
0.230
0.054
0.205
0.361
0.129
on NL-50 and WK-50 and is comparable the best baseline in FB-50. The full results on all 12 datasets from (Lee et al., 2023b) are reported in Table 18 in Appendix E. The sults show that MAYPL's stracture-oriented representation learning is effective in handling new entities and relations.
Hitl0 in (Yadati, 2020), and thus, MAYPL's performance is also measured using Hit3. MAYPL significantly outper- forms all methods in all metries on these datasets, showing MAYPL's remarkable inductive capability on HKGs.
5.4. Ablation Studies and Qualitative Analysis
5.3. Inductive Link Prediction on HKGs
Table 7 shows ablation studies of MAYPL on WikiPeople (WP-). WK-50, and WD20K(100)y2 (WDv2), where
Table 5 shows inductive link prediction results on HKGs using WD20K(100)r1 (Ali et al., 2021) and (120K(100)y2 (Ali et al., 2021) datasets, where models predict entities the primary triplets. Also, Table shows the results on WP-IND (Yadati, 2020) and MFB-IND (Ya- dati, 2020), where models predict all entities in facts; the baselines' performances reported using Hit3 instead of
MRRs are reported. We consider the following variations: (i) introduce learnable vectors for individual entities and relations instead of using the structure-driven initializer in Section 4.1, (ii) remove the attentive neural message pass- ing in Section 4.2, (iii) let all facts equally contribute the update of entities and relations without considering their

---

Structure Is All You Need: Structural Representation Learning on
Knowledge Graphs
Table 5:
Inductive link prediction performance on the primary triplets in WD20K(100)y1 and (1002
Table 6: Inductive link prediction performance on all entities of facts in WP-IND and MFB-IND.
(120K(100)w1
WD20K(100)v2
WP-IND
MFB-IND
MRR
Hitl0
Hitl
MRR
Hit10
Hitl
MRR
Hit3
Hitl
MRR
Hit3
Hitl
BLP
0.057
0.123
0.019
0.040
0.092
0.015
HGNN
0.072
0.112
0.045
0.121
0.114
0.076
CompGCN
0.104
0.184
0.058
0.026
0.053
0.007
HyperGCN
0.075
0.111
0.049
0.118
0.117
0.074
StarF
0.113
0.213
0.062
0.051
0.129
0.014
G-MPNN-sum
0.177
0.191
0.108
0.124
0.123
0.071
QBIP (w/o qual.)
0.055
0.126
0.015
0.049
0.097
0,026
G-MPNN-max
0.200
0.214
0,125
0.268
0.283
0.191
QBLP (w/ qual.)
0.107
0.245
0.040
0.067
0.120
0.035
HCNet
0.414
0.451
0.352
0.368
0.417
0.223
MAYPL
0.486
0.662
0.396
0.298
0.518
0.195
MAYPL
0.468
0.529
0.363
0.550
0.582
0.469
Table 7: Ablation Studies of MAYPL. MRRs are reported.
Table 8 shows the top 3 most similar entities relations to a target in WP where it shows similar entities for en- tities, i.e., Vancouver computer scientist and similar relations for a relation, i.e., fami ly. Given target, the top similar entitiesfrelations are selected by computing the dot-products between the target and entities/relations using the initial representations returned by the structure- driven initializer (Initializer) or the final representations (Fi- nal Rep.) of MAYPL. While the initializer enables roughly selecting entities/relations of a similar type or conceptually relevant ones to the target, the ones chosen by the final repre- sentations hecome more relevant and semantically closer to the target. This shows that our initializer computes reason- able initial representations, which are effectively refined by the following attentive neural message passing in MAYPL..
WP
WK-50
WDv2
Learn v (0)) (0)))(()) 1= 0
0.388
0.061
0.112
(
0.109
0.029
0.144
(iii)
a = ( =0
0,504
0,086
0,210
(iv)
( = A (1-1)
0,519
0.062
0.248
Rep. Hadamard Prod. w/ Concat.
0.518
0.090
0.265
(vi) Diff. Para. for x and
0.520
0.081
0.179
MAYPL
0.521
0.109
0.298
Table 8: Top 3 similar entities/relations to a target in WP based on the initial representations returned by the structure- driven initializer and the final representations of MAYPL.
Target
Initializer
Final Rep.
Venice
Toronto
Vancouver
Budapest
Victoria
6. Conclusion and Discussion
3
Gothenburg
Ottawa
MAYPL learns representations based on how facts, entities, and relations are connected, positioned, and organized in HKGs. Due to its structure-driven learning, MAYPL can effectively compute representations on a new HKG consist- ing of new entities and relations unobserved during training time. Experimental results show that MAYPL outperforms 40 different baseline methods on 10 benchmark datasets in varied settings, i.e., transductive link prediction on HKGs, inductive link prediction on KGs, and inductive link predic- tion on HKGs. MAYPL's success implies that thoroughly learning and exploiting the structure of an HKG necessary and sufficient for learning representations on HKGs.
psychologist
mathematician
computer scientist
professeur des universités
programmer
inventor
AI researcher
death type citizen of ethnic group
sibling family ly name father
family
relative importance in Section 4.2, (iv) compute relation representations by a simple projection instead of the relation aggregation in Section 4.2, (v) replace the Hadamard prod- ucts in computing messages of facts, entities, and relations with concatenation in Section 4.2 and Section 4.3, (vi) use different parameters to compute the query representation and the other entity representations in Section 4.3. While varying some modules more critically affects performance in inductive settings (WK-50 and WDv2) than the transduc- tive setting (WP-). the variations degrade the performance, validating each module's contribution in MAYPL.3
Limitations and Future Work Within MAYPL, the pro- cedure of message computation and aggregation requires enough computing resources as described in Appendix D. We plan to further optimize our implementations and apply cost-effeetive attention computation, e.g., replace Bahdanau- style attention (Bahdanau et al., 2015) with Luong-style attention (Luong et al., 2015), for increasing MAYPL's scal- ability. In Table 9, we compare MAYPL and other methods on WikiPeople" regarding the training time, the number of parameters, and MRR, where we present all baselines
More detailed explanations about these ablation studies are available in Appendix F. Also, Appendix G provides additional case studies about MAYPI..
8

---

Structure Is All You Need: Structural Representation Learning on Knowledge Graphs
Table 9: The training time, the number of parameters, and MRR of MAYPL and other methods on WikiPeople".
completion. In Proceedings of the 34th Conference on Neural Information Processing Systems, pp. 9649-9661, 2020.
Training Time parameters
MRR
Ali, M., Berrendorf, M., Galkin, M., Thost, V., Ma, T., Tresp, V., and Lehmann, J. Improving inductive link prediction using hyper-relational facts. In Proceedings of the 20th International Semantic Web Conference, pp. 92-92, 2021.
StarE GRAN
days
8.2M
0.491
10h
>8.9M
0.503
Hy-Transformer
5h
7.8M
0.501
HyperFormer
N/A
67.0M
0.473
HAHE
12h
30.2M
0.509
MAYPL
20h
10.5M
0.519
Ba, J. L.,, Kiros, J. R., and Hinton, G. E. Layer normalization. In arXiv preprint, 2016.
whose training times and parameter sizes are available. We believe MAYPL can be applied question answering (QA) tasks (Chen al., 2021), serving knowledge for pretrained language models. The theoreti- cal analysis of MAYPL (Lee et al., 2024) and its extension to various forms of knowledge graphs (Lee et al., 2023a; Chung Whang, 2023) remain as future work. Further- more, by extending MAYPL, we plan to develop foundation models for KG reasoning (Galkin et al., 2024) applicable to wide range of real-world cases.
Bahdanau, D., Cho, K., and Bengio, Y. Neural machine translation by jointly learning to align and translate. Proceedings of the 3rd International Conference on Learning Representations, 2015.
additional source of
Bordes, A., Usunier, N., Garcia-Durán, A., Weston, J., and Yakhnenko, O. Translating embeddings for modeling multi-relational data. In Proceedings of the 27th Con- ference Neural Information Processing Systems, pp. 2787-2795, 2013.
Brody, S., Alon, U., and Yahav, E. How attentive graph attention networks? In Proceedings of the 10th Interna- tional Conference on Learning Representations, 2022.
Impact Statement
Knowledge representation learning is one the key tech- niques of many modern Al applications. We believe our work can be applied various applications, such as gen- erative models, natural language processing (Chen et al., 2021), robotics (Kwak et al., 2022), and information trieval, and advance the machine learning fields requiring representation grounding knowledge and structure-infusee vectors. Given that our work eases the automatic comple- tion of missing knowledge, possible undesirable misuses of our technique include attempts to predict and disclose others' private information with malicious intentions. Such risks should be monitored, and the readers of this paper are advised to be aware that our research should not be applied to such activities.
Chen, S., Liu, X., Gao, J., Jiao, J., Zhang, R., and Ji, Y. HiuER: Hierarchical transformers for knowledge graph embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 10395-10407, 2021.
Chung, C. and Whang, J. J. Learning representations of bi-level knowledge graphs for reasoning beyond link pre- diction. In Proceedings the 37th AAAI Conference on Artificial Intelligence, pp. 8008-4216, 2023.
Chung, C., Lee, J., and Whang. J. Representation learning hyper-relational and numeric knowledge graphs with transformers. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 310-322, 2023.
Acknowledgements
This work was supported by the NRF of Korea grant funded by MSIT (70% from RS-2025-00559066, Responsible Mul- timodal Graph AD) and the IITP grant funded by MSIT (30% from RS-2022-11220369, (Part 4) Development of Al Tech- nology to support Expert Decision-making that can Explain the Reasons/Grounds for Judgment Results based on Expert Knowledge).
Daza, D., Cochez, M., and Groth, P. Inductive entity repre- sentations from text via link prediction. In Proceedings of The Web Conference 2021, pp. 798-8088 0888, 2021.
Di, S. and Chen, L. Message function search for knowl- edge graph embedding. In Proceedings of the ACM Web Conference 2023, pp. 2633-2644, 2023.
References
Di, S., Yao, Q., and Chen, L. Searching to sparsify tensor decomposition for n-ary relational data. In Proceedings of the Web Conference 2021, pp. 4043 4054, 2021.
Abboud, R., ismail ilkan Ceylan, Lukasiewicz, T., and Sal- vatori, T. A box embedding model for knowledge base

---

Structure Is All You Need: Structural Representation Learning on Alyper-Relationa Knowledge Graphs
Fatemi, B., Taslakian, P., Vazquez, D., and Poole, D. Knowl- edge hypergraphs: Prediction beyond binary relations. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, pp. 2191-2197, 2020.
He, K., Zhang, X., Ren, S., and Sun, Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Feng. Y., You, H., Zhang, Z., Ji, R., and Gao. Y. Hypergraph neural networks. In Proceedings of the 33rd AAAI Con- ference on Artificial Intelligence, pp. 3558-3565, 2019,
Hu, Z., Butiérrez-Bsullto, V., Xiang, Z., Li, R., and Pan, J. Z. HyperFormer: Enhancing entity and relation interaction for hyper-relational knowledge graph completion. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pp. 803- 812. 2023.
Galkin, M., Trivedi, P., Maheshwari, G., Usbeck, R., and Lehmann, J. Message passing for hyper-relational knowl- edge graphs. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 7346-7359, 2020.
Huang, X., Orth, M. R., Barceló, P., Bronstein, M. M., and ismail ilkan Ceylan. Link prediction with relational hypergraphs. In arXiv preprint, 2024.
Galkin, M., Denis, E., Wu, J., and Hamilton, W. L. Node- Piece: Compositional and parameter-efficiee representa- tions of large knowledge graphs. In Proceedings of the 10th International Conference on Learning Representa- tions, 2022.
Ji, S., Pan, S., Cambria, E., Marttinen, P., and Yu, P. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems, 33(2):494-514, 2022.
Kingma, D. P. and Ba, L. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, 2015.
Galkin, M., Yuan, X., Mostafa, H., Tang, J., and Zhu, Z. To- wards foundation models for knowledge graph reasoning- In Proceedings of the 12th International Conference Learning Representationss 2024.
Kwak, J. H., Lee, J., Whang, J. J., and Jo, S. Semantic grasping via a knowledge graph of robotic manipulation: A graph representation learning approach. IEEE Robotics and Automation Letters, 7(4):9397-9404, 2022.
Geng, Y., Chen, J., Pan, J. Z., Chen, M., Jiang, S., Zhang, W., and Chen, H. Relational message passing for fully inductive knowledge graph completion. In Proceedings of the 2023 IEEE 39th International Conference on Data Engineering, pp. 1221-1233 2023.
Lee, J., Chung, C., Lee, H., Jo, S., and Whang. J. VISTA: Visual-textual knowledge graph representation learning, In Findings of the Association for Computational Linguis- tics: EMNLP 2023, pp. 731 7314-7328, 2023a.
Gesese, G. A., Sack, H., and Alam, M. RAILD: Towards leveraging relation features for inductive link prediction in knowledge graphs. In Proceedings of the I 11th Interna- tional Joint Conference on Knowledge Graphs, pp. 82-90, 2022.
Lee, J., Chung, C.,, and Whang, J. J. InGram: Inductive knowledge graph embedding via relation graphs. In Pro- ceedings of the 40th International Conference on Ma- chine Learning, pp. 18796-18809, 2023b.
Guan, S., Jin, X., Wang, Y., and Cheng, X. Link prediction on N-ary relational data. In Proceedings of The Web Conference 2019, pp. 583-593, 2019.
Lee, J., Hwang, M., and Whang, J. J. PAC-Bayesian gen- eralization bounds for knowledge graph representation learning. In Proceedings of the 41st International Con- ference on Machine Learning, pp. (6589-26622) 2024.
Guan, S., Jin, X., Guo. J., Wang, Y., and Cheng, X. Neuln- fer: Knowledge inference on N-ary facts. In Proceedings of the 58th Annual Meeting of the Association for Com- putational Linguistics, pp. 6141-6151 2020.
Li, Z Wang, C., Wang, X., Chen, 7., and Li, J. HJE: Joint convolutional representation learning for knowledge hy- pergraph completion. IEEE Transactions Knowledge and Data Engineering, 36(8):3879-333 3892, 2024a.
Guan, S., Jin, X., Guo, J., Wang, Y., and Cheng, X. Link prediction on N-ary relational data based on relatedness evaluation. IEEE Transactions on Knowledge and Data Engineering, 35(00))672-6855 2023.
Li, Z., Wang, X., Li, J., Guo, W., and Zhao, J. HyCubE: Efficient knowledge hypergraph 3d circular convolutional embedding. In arXiv preprint, 2024b.
He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing huvan-level performance on imagenet classification. In Proceedings of the 2015 IEEE International Conference on Computer Vision, pp. 1026- 1034, 2015.
Liu, S., Grau, B., Horrocks, I., and Kostylev, E. INDIGO: Gan-based inductive knowledge graph completion using piir-wise encoding. In Proceedings of the 35th Con- ference on Neural Information Processing Systems, pp. 2045, 2021a.
10

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
Liu, Y., Yao, Q. and Li, Y. Role-aware modeling for N-ary relational knowledge bases. In Proceedings of the Web Conference 2021, pp. 2660-2671, 2021b.
Toutanova, K. and Chen, D. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57-66, 2015.
Luo, H., E, H., Yang, Y., Guo, Y., Sun, M., Yao, T., Tang, Z., Wan, K., Song, M., and Lin, W. HAHE: Hierarchi- cal attention for hyper-relational knowledge graphs in global and local level. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pp. 8095-8107, 2023.
Vashishth, S., Sanyal, S., Nitin, V., and Talukdar, P. Composition based multi-relational graph convolutional networks. In Proceedings of the 8th International Con- ference on Learning Representations, 2020.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, I.,, Gomez, A. N., Kaiser, t.., and Polosukhin, I. Attention is all you need. In Proceedings of the 31st Conference on Neural Information Processing Systems, pp. 1998-6008, 2017.
Luong, T., Pham, H., and Manning, C. D. Effective ap- proaches to attention based neural machine translation. In Proceedings of the 2015 Conference on Empirical Meth- ods in Natural Language Processing, pp. 1412-1421, 2015.
Vrandečic, D. and Krötzsch, M. Wikidata: a free collabo- rative knowledgebase. Communications of the ACM, 57 (10):78-85, 2014.
Mai, S., Zheng. S., Yang, Y., and Hu, H. Communicative message passing for inductive relation reasoning. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 4294 4302, 2021.
Wang, C., Wang. X., Li, Z., Chen, Z., and Li, J. HyConvE: A novel embedding model for knowledge hypergraph link prediction with convolutional neural networks. In Proceedings of the ACM Web Conference 2023, pp. 188- 198, 2023.
Mu, C., Zhang, L.., Zhang, J., Huang, Q., and Wang. Z. Inductive knowledge graph embedding via exploring in- teraction patterns of relations. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 1722-1731, 2024.
Wang, Q., Wang, H., Lyu, Y., and Zhu, Y. Link prediction on N-ary relational facts: A graph- based approach. In Find- ings of the Association for Computational Linguisties: ACL-IJCNLP 2021, pp. 396 196-407,- 2021.
Rosso, P., Yang, D., and Cudré-Mauroux, P. Beyond triplets: Hyper-relational knowledge graph embedding for link prediction. In Proceedings of The Web Conference 2020, pp. 1885-1896, 2020.
Wen, J., Li, J., Mao, Y., Chen, S., and Zhang, R. On the representation and embedding of knowledge bases beyond binary relations. In Proceedings of the 25th Inter- national Joint Conference on Artificial Intelligence, pp. 1300-1307, 2016
Sadeghian, A., Armandpour, M., Ding, P., and Wang. D. Z. DRUM: End-to-end differentiable rule mining on knowl- edge graphs. In Proceedings of the 33rd Conference on Neural Information Processing Systems, pp. 15347- 15357, 2019.
Xiong, B., Nayyeri, M., Pan, S., and Staab, S. Shrinking embeddings for hyper-relational knowledge graphs. In Proceedings of the 61st Annual Meeting of the Associ- ation for Computational Linguistics (Volume 1: Long Papers), pp. 13306-13320, 2023.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research,
2014.
Xiong, W., Hoang, T., and Wang, W. Y. Deeppath: A rein- forcement learning method for knowledge graph reason- ing. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. pp.564-511 564-573, 2017.
Suchanek, F. M., Kasneci, G., and Weikum, G. Yago: A core of semantic knowledge. In Proceedings of the 16th International Conference on World Wide Web, pp. 697- 706, 2007.
Szegedy, C., Vanhoucke, V., loffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the 2016 IEEE Conference Computer Vision and Pattern Recognition, pp. 2818- 2826, 2016.
Xu, X., Zhang, P., He, Y., Chao, C., and Yan, C. Subgraph neighboring relations infomax for inductive link predic- tion on knowledge graphs. In Proceedings of the 31st International Joint Conference on Artificial Intelligence. pp. 2341-2344 2022.
Teru, K., Denis, E., and Hamilton, W. Inductive relation prediction by subgraph reasoning. In Proceedings of the 37th International Conference on Machine Learning. pp. 9448 9488-947, 2020.
Yadati, N. Neural message passing for multi-relational ordered and recursive hypergraphs. In Proceedings of the 34th Conference on Neural Information Processing Systems, pp.
2020.

---

Structure Is All You Need: Structural Representation Learning on Relational Knowledge Graphs
Yadati, N., Nimishakavi, M., Yadav, P., Nitin, V., Louis, A., and Talukdar, P. HyperGCN: A method for training graph convolutional networks hypergraphs. In Pro- ceedings of the 33rd Conference on Neural Information Processing Systems, pp. 1511-1522, 2019.
Yan, S., Zhang, Z.., Sun, X., Xu, G., Li, S., Liu, Q., Liu, N., and Wang, S. PolygonE: Modeling n-ary relational data as gyro- poroopolygons.oooo in hyperbolic space. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, pp. 4308 4318, 2022.
Yang, F., Yang. Z., and Cohen, W. W. Differentiable Iearn- ing of logical rules for knowledge base reasoning. In Proceedings of the sl Conference on Neural Informa- tion Processing Systems, pp. 2328, 2017.
Yang. J., Jiang, X., Gao, Y., Yang, L. T., and Yang. J. Gen- eralize to fully unseen graphs: Learn transferable hyper- relation structures for inductive link prediction. In Pro- ceedings of the 32nd ACM International Conference on Multimedia, pp. 1274-1282, 2024.
Yu, D. and Yang, Y. Improving hyper-relational knowledge graph completion. In arXiv preprint, 2021.
Zhang, R., J., Mei, J., and Mao, Scalable instance re- construction in knowledge bases via relatedness affiliated embedding. In Proceedings of the 2018 World Wide Web Conference, pp. 1185-119 1194, 2018.
Zhang, Y. and Yao, Q. Knowledge graph reasoning with relational digraph. In Proceedings of The ACM Web Conference 2022 pp. 912-924, 2022.
Zhu, Z., Zhang, Z., Xhonneux, L.-.., and Tang, Neural Bellman-Ford networks: A general graph neural network framework for link prediction. In Proceedings of the 35th Conference on Neural Information Processing Systems, pp. 19476-29499 2021.
12

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
A. Comparison of MAYPL with Existing Methods
Table 10: Comparison of MAYPL and all baseline methods regarding their ways of representing knowledge and their abilities to handle new entities and relations in inductive inference.
Knowledge Representation
Inductive Inference
KG
KHG
NRR
HKG
new entities
new relations
CompGCN (Vashishth al., 2020)
GralL (Teru et al., 2020)
COMPILE (Mai ct al., 2021)
SNRI (Xu al., 2022)
INDIGO (Liu et al., 2021a)
NeuralLP (Yang et al., 2017)
DRUM (Sadeghian et al., 2019)
BLP (Daza et al., 2021)
NodePiece (Galkin al., 2022)
RED-GNN (Zhang Yao. 2022)
NBFNet (Zhu al., 2021)
RAILD (Gesese et al., 2022)
RMPI (Geng et al., 2023)
InGram (Lee al., 2023b)
HyRel (Yang et al., 2024)
InKGE-EIPR (Mu et al., 2024)
HGNN (Feng al., 2019)
HyperGCN (Yadati et al., 2019)
HypE (Fatemi et al., 2020)
G-MPNN (Yadati, 2020)
S2S (Di et al., 2021)
HyConvE (Wang al., 2023)
MSeaKG (Di & Chen, 2023)
HIE (Li al., 2024a)
HCNet (Huang et al., 2024)
HyCubE (Li et al., 2024b)
RAE (Zhang et al., 2018)
NaLP (Guan et al., 2019)
RAM (Liu et al., 2021b)
INaLP (Guan et al., 2023)
Neulnfer (Guan et al., 2020)
HINGE (Rosso al., 2020)
StarE (Galkin al., 2020)
GRAN (Wang ct al., 2021)
Hy- Transformer (Yu & Yang. 2021)
PolygonF: (Yan al., 2022)
HyperFormer (Hu et al., 2023)
HAHE (I.uo et al., 2023)
ShrinkE (Xiong et al., 2023)
HyNT (Chung et al., 2023)
QBLP (Ali et al., 2021)
MAYPL
Table 10 compares MAYPL and other existing methods in terms of (i) the form of knowledge representations and (ii) the capability inductive inference. While KGs only contain triplets, KHGs, NRRs, and HKGs provide more enriched information by providing additional details. Comparisons between KHGs, NRRs, and HKGs are described in Section where show that HKGs are better forms than KHGs and NRRs in preserving the original information of real-world knowledge bases. For the inductive inference, we consider whether method can handle new entities and new relations.
Some methods in Table 10 are explained in Section I and Section 2. While HGNN (Feng et al., 2019) and HyperGCN (Yadati et al., 2019) are not originally designed for KGs, they are considered (Yadati, 2020) since they handle hyperedges. Also, G-MPNN (Yadati, 2020) can handle new entities only one-hop distant from observed entities, having limited inductive capability. Thus, these methods are denoted by A Table 10.
13

---

Structure Is All You Need: Structural Representation Learning on Elyper-Relationa Knowledge Graphs
Table 11: Datasets for Transductive Link Prediction on HKGs.
[V]
[
[Hu]
[Haal
[Hae
WDS0K (Galkin et al., 2020)
47,155
531
166,435
23,913
46,159
WikiPeople (Wang et al., 2021)
34,825
178
294,439
37,715
37,712
WikiPeople (Guan et al., 2019)
47,765
193
305,725
38,223
38,281
Table 12: Datasets for Inductive Link Prediction on KGs.
|V]
[R]]
|>]
[T[]
|Phtel
[Haai
[as
NI.-100 al,,
1,258
55
7,832
1,709
53
2,378
793
793
NL-75 (Lee 202,))
2,607
96
11,058
1,578
116
1,818
606
607
NL-50 (Lee et. a0,, 2023b)
4.396
106
17,578
2,335
119
2,576
859
859
NL-25 et al,
4,396
106
17,578
2,146
120
2,230
743
744
WK-100 (Lec ct al., 2023b)
9,784
67
49,875
12,136
37
13,487
4,496
4,496
WK-75 (Lec et 2023b)
6,853
52
28,741
2,722
65
3,430
1,143
1,144
WK-50 (Lee al., 2023b)
12,022
72
82,481
9,328
93
9,672
3,224
3,225
WK-25 (Lee al., 2023b)
12,659
47
41,873
3,228
74
3,391
1,130
1,131
FB-100 (Lee et al., 2023b)
4,659
134
62,809
2,624
77
6,987
2,329
2,329
FB-75 (Lee et al., 2023b)
4.659
134
62,809
2,792
186
9,316
3,106
3,106
FB-50 (Lee et al., 2023b)
5,190
153
85,375
4,445
205
11,636
3,879
3,879
FB-23 (Lee et al., 2023b)
5,190
163
91,571
4.097
216
17.147
5,716
5,716
B. Details of Datasets
Table 11 shows the statistics of datasets for transductive link prediction on HKGs: WD50K (Galkin et al., 2020), WikiPeople (Wang et al., 2021), and WikiPeople (Guan al., 2019), where WikiPeople was generated by remov- ing the literals from WikiPeople. Although WikiPeople was first used by (Rosso et al., 2020), we refer to (Wang et al., 2021) because it was named WikiPeople by (Wang et al., 2021). We noticed that WikiPeople was denoted as WikiPeople in some baselines' papers, including (Rosso et al., 2020; Galkin et al., 2020; Luo et al., 2023); by examining the statistics of the datasets used in their official GitHub or reported in the paper, we separate the existing experimental results for WikiPeople" and WikiPeople. We used both datasets because these datasets are used in different litcratures. WikiPeople" is used compare methods for HKGs such as StarE (Galkin al., 2020) or HAHE (Luo et al., 2023), while WikiPeople used compare methods for NRRs KHGs, such NaLP (Guan et al., 2019) HCNet (Huang et al., 2024).
For WD50K and WikiPeople we adopt the widely-used experimental setting of (Galkin al., 2020), where model is tuned using validation set and re-trained using both training and validation sets. Since there is no official experimental results for HINGE (Rosso et al., 2020) using this setting on WikiPeople we do not report its results; the results in (Rosso et al., 2020) are the ones reported after training only using the training set.
The statistics of WD50K in Table 1I differs from that (Galkin et al., 2020) because (Galkin et al., 2020) also counts some extra tokens. Also, the number of relations of WikiPeople differs from (Guan et al., 2019) due to the different forms of knowledge representations. We note that the JF17K dataset (Wen al., 2016) is another well-known dataset for HKGs, but the authors who released it (Wen et al., 2016) suggest not using it due redundant entries. Therefore, we did not use it. Refer to ht www. site. uot wowttaw..... ca/ yymao/ / JF17K/
Table 12 shows the statistics of all 12 datasets for inductive link prediction on KGs: NL-100 (Lee et al., 2023b), NL-75 (Lee et al., 2023b), NL-50 (Lee et al., 2023b), NL-25 (Lee et al., 2023b), WK-100 (Lee et al., 2023b), WK-75 (Lee et al., 2023b), WK-50 (Lee et al., 2023b), WK-25 (Lee al., 2023b), FB-100 (Lee et al., 2023b), FB-75 (Lee et al., 2023b), FB-50 (Lee al., 2023b), and FB-25 (Lee al., 2023b). The full experimental results on these datasets are presented Appendix E.
Also, Table 13 shows the statistics of datasets for inductive link prediction on HKGs: (120K (Ali al., 2021). WD20K (100) (Ali et al., 2021), WP-IND (Yadati, 2020), and MFB-IND (Yadati, 2020). The statistics of WD20K(100)v1 and WD20K(100)y2 differr from the original paper (Ali et al., 2021) because (Ali et al., 2021) reported the statistics by only considering at most six qualifiers per fact, whereas we removed this constraint. In addition, (Ali et al., 2021) also counts some extra tokens, as in (Galkin et al., 2020). On the other hand, in (Yadati, 2020), unseen entities are assumed to be predefined in WP-IND and MFB-IND, and all other entities are considered as seen entities, regardless of whether they
14

---

Structure Is All You Need: Structural Representation Learning on Plyper-Relationa. Knowledge Graphs
Table 13: Datasets for Inductive Link Prediction on HKGs.
[R]]
[7c]
|>]
[
[
|7
[uul
WD20K(1000vl (Ali et al., 2021)
5,785
91
7,785
4,286
74
2,667
295
365
WD20K(100)V2 (Ali et al., 2021)
3,226
56
4,146
5,593
53
4,274
539
678
WP-IND (Yadati, 2020)
4.017
35
4,139
4,463
4,690
275
275
MFB-IND (Yadati, 2020)
3,223
22
336,733
3,783
22
344,260
3,763
3,764
actually appear in a training set. However, in our definition, unseen entities are those that do not appear in the training set, which reduces the number of seen entities [)]. Additionally, we transform hyperedges into hyper-relational facts, which increases the number of relation labels [R]. Therefore, the statistics we report for WP-IND and MFB-IND differ from those reported in (Yadati, 2020). Note that although WD20K(1000w1, WD20K(100ww, WP-IND, and MFB-IND include node features, MAYPL was trained and evaluated purely based on the graph structure, without utilizing the node features.
C. Coverage of Methods Across Datasets
In Section 5, we compare MAYPL with 40 different baseline methods using 10 benchmark datasets. Also, Appendix E shows additional experimental results of inductive link prediction on KGs, which add 9 more datasets (NL-100, NL-75, NL-25, WK-100, WK-75, WK-25, FBB100, FB-75, FB-25) and one more baseline (RAILD ran on N1.100,, WK-100, and FB-100). Thus, MAYPL is compared with 41 baseline methods on 19 datasets in total. Table 14 shows which methods are ran on which datasets. The baselines' results are obtained from the baseline's original paper (marked as O) or the paper that proposed the dataset (marked as )) For brevity, NL-75, NL-50, and NL-25 are denoted by NLs, WK-75, WK-50, and WK-25 are denoted by WKs, and FB-75, FB-50, and FB-25 are denoted by FBs.
D. Implementation and Experimental Details of MAYPL
We provide implementation details of MAYPL. Also, we describe how we ran MAYPL and present the runtime and memory usage of MAYPL. Our codes are available at https: //ttthgbttttttt///// com/coo///////oo comm/dii-a llablaaaaaaa MAYYL.
D.1. Implementation Details of MAYPL
In our implementation of MAYPL, we use the Adam optimizer (Kingma & Ba, 2015), PReI.U (He et al., 2015) activation function, dropout (Srivastava et al., 2014), residual connection (He et al., 2016), label smoothing (Szegedy et al., 2016), and layer normalization (Ba et al., 2016). All linear projections are followed by a bias term omitted for brevity in the formulas of Section 4. In the attentive neural message passing, we use multi-heads (Vaswani et al., 2017: Brody al., 2022). We use the dynamic split strategy (Lee al., 2023b) with modifications: instead of creating a spanning tree, we simply use some portion of training graph as a message set even if it is not connected and use the remaining facts consisting of entities and relations in the chosen message set for loss calculation. We use Python 3.9, and PyTorch 2.0.1 with cuda version 1 11.7.
D.2. Experimental Details, Runtime, and Memory Usage of MAYPL
We provide experimental details of each experiment along with MAYPL's runtime and memory usage. For runtime, we report the time required to reproduce the reported results from scratch, including validation and test time. We report the GPU memory using torch. cuda mnxx_meorry.. aa lcated () function in PyTorch. The time and memory are rounded for brevity. Note that is the dimension of all layers (d = do = d1 = ... = di), c is the amount of label smoothing, ebest is the best epoch, is the learning rate, Thend is the number of attention heads, Thatch is the number of batches, 8, is the train graph ratio, and ōdrop is the dropout ratio.
Table 15 shows the best hyperparameters, runtime, and memory usage of MAYPL for WD50K, WikiPeople and WikiPeople. We trained MAYPI. on the training set of WikiPeople- to find the best hyperparameters for both WikiPeople and WikiPeople We chose the best hyperparameters based on the validation performance. After finding the best hyperparameters, we fixed the best epoch for WikiPeople" and re-trained MAYPL on WikiPeople with training+validation sets. For WikiPeople, we used the training set of WikiPeople to re-train MAYPL, and tuned the epoch while fixing all other parameters. For WD50K, we tuned MAYPL on the training set of WD50K, fixed all hyperparameters, and re-trained
15

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
Table 14: Coverage of Methods Across Datasets and the Sources of Baselines' Results. The baselines' results are obtained
from the baseline's 's original paper (marked as O) or the paper that proposed the dataset (marked as ). NLs denote NL-75, N1,-00, and NI,-55, WKs denote WK-75, WK-50, and WK-25, and FBs denote FB-75, FB-50, and FB-25. In Section 5 and Appendix E, MAYPL is compared with baseline methods on 19 datasets total.
NI. NT. WK-100 WKs FR-100 FRs
WP WP
WDS0K WP-IND MER-IND (D20K (1220
CompGCN (Vashishila et 2020)
GraiL (Teru et al., 2020)
CoMPILE (Mai et al., 1021)
SNRI (Xu al. 2022)
INDIGO (Liu et al., 2021a)
NeuralLP (Yang at., 2017)
DRUM (Sadeghian al., 2019)
BLP (Duza al., 2021)
NodePiece (Galkin et al., 2022)
RED-GNN (Zhang Yao, 2022)
NBFNet (Zhu et al., 2021)
RAILD (Gesese et 2022)
RMPI (Geng al. 2023)
000
InCiram (Lee et 2U23b)
HyRel ol., 2024)
0.00
0.00
0,00
000
EIPE (Mu et al., 2024)
HGNN (Feng et al., 2019)
HyperGCN (Yadati al., 2019)
G-MPNN (Yadati, 2020)
RAE (Zhung al., 2018)
HypE (Fatemi et al., 2020)
S2S (Di et al., 2021)
HyConvE (Wang al., 2023)
MSeaKG (Di & Chen, 2023)
HIB (Li al. 2024a)
HCNet (Huang et al., 2024)
HyCubE (Li al., 2024b)
Nat.P (Guan al., 2019)
RAM(Lin al., 2021b)
INaL.P (Guan al., 2023)
Neulnfer
2020)
HINGE (Rosso al., 2020)
StarE (Galkin et al., 2020)
GRAN (Wang et al., 2021)
Hy- Tranffrfrrrrrr (Ytu& & Yang. 2021)
PolygonE (Yan al., 2022)
HyperFormer (Hu
2023)
0000
HAHE (Luo et al., 2023)
ShrinkE (Niong
2023)
HyNT (Chung et al., 2023)
QBLP (Ali al., 2021)
MAYPL
0
0
MAYPL on WD50K with training+validation sets. We performed validation every 50 epoch, and stopped training if the validation performance did not increase for 250 epochs. We ran MAYPL on NVIDIA RTX A6000 with d = 256.
Table 16 shows the best hyperparameters, runtime, and memory usage of MAYPL. for NI.-100, NL.-75, NL-50, NL.25,, WK-100, WK-75, WK-50, WK-25, FB-100, FB-75, FB-50, and FB-25. MAYPL was trained training graph, validated and tested on an inference graph without re-training. We chose the best hyperparameters based on the validation performance. We did not adopt the early stop strategy and ran MAYPL for 500 epochs, performing validation every 10 epochs. One exception is NI.75,, which we ran MAYPL for 1000 epochs. We ran MAYPL on NVIDIA RTX 2080 Ti with 32.
Table shows best hyperparameters, runtime, and memory usage MAYPL for WD20K WD20K<100)>> WP-IND, and MFB-IND. MAYPL was trained on training graph, validated and tested on an inference graph without re-training. We chose the best hyperparameters based on the validation performance. We did not adopt the early stop strategy for WD20K(100)vl and WD20K(100)y2 and ran MAYPL for 1000 epochs and 500 epochs, respectively, while performing validation every 10 epochs. For WP-IND, we did not adopt the early stop strategy and ran MAYPL for 2000 epochs while performing validation every 20 epochs. For MFB-IND, we performed validation every 50 epochs, and stopped training if the validation performance did not increase for 250 epochs. We ran MAYPL on NVIDIA RTX 2080 Ti with 256 for WD20K(100y) and WD20K(100)y,, and 128 for WP-IND. For MFB-IND, we ran MAYPL on NVIDIA RTX A6000 with 128.
16

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
Table 15: Best hyperparameters, runtime, and memory usage of MAYPL for WDSOK, WikiPeople and WikiPeople.
Time
Memory
Cbest
Ir
L
L
Thead
Tbbacch
6
ddeup
WD50K WikiPeople
21h
42.8GB
0.05
3000
0.0001
4
6
16
20
0.7
0.2
20h
43.8GB
0.0
2900
0.0001
20
0.7
0.1
WikiPeople
21h
45.8GB
0.0
2400
0.0001
3
4
32
20
0.7
0.1
Table 16: Best hyperparameters, runtime (m: minutes, h: hours), and memory usage of MAYPL for NL-100, NL-75, NL-50, NL-25, WK-100, WK-75, WK-50, WK-25, FB-100, FB-75, FB-50, and FB-25.
Time
Memory
Cbes
Ir
L
L
7head
Neatch
8
òdrop
NL-100
15m
0.1GB
0.0
160
0.0002
4
4
4
30
0.35
0.05
NL-75
30m
0.IGB
0.0
660
0.0002
0.15
0.0
NL-50
0.4GB
0.0
390
0.001
0.4
0.1
NL-25
40m
0.5GB
0.0
260
0.002
2
10
2
30
0.5
0.05
WK-100
1h
0.6GB
0.0
410
0.0005
3
5
2
30
0.3
0.05
WK-75
25m
0.5GB
0.0
250
0.0005
4
30
0.5
0.05
WK-50
35m
1.2GB
0.0
230
0.0002
0.3
0.1
WK-25
30m
0.8GB
0.0
300
0.001
3
5
8
30
0.5
0.05
FB-100
50m
0.7GB
0.0
310
0.00015
4
30
0.3
0.1
FB-75
20m
0.7GB
0.0
140
0.0005
5
8
30
0.3
0.1
FB-50
45m
1.1GB
0.0
320
0.001
0.35
0.1
FB-25
1.6GB
0.0
390
0.001
0.6
0.1
Table 17: Best hyperparameters, runtime (m: minutes, h: hours), and memory usage of MAYPL for WD20K(100))1, WD20K(100)y2, WP-IND, and MFB-IND.
Time
Memory
Obest
T.
Thha
7bbuch
8g
6aup
WD20K(100Jvl
2h
3.1GB
0.0
930
0.0001
6
32
30
0.5
0.05
WD20K<100Jv2
55m
1.2GB
0.0
490
0.0002
8
0.4
0.05
WP-IND
8h
1,4GB
0.0
1800
0.0001
4
10
4
50
0.7
0.1
MFB-IND
7h
27GB
0.0
1550
0.001
3
3
16
20
0.7
0.1
E. Additional Experimental Results of Inductive Link Prediction on KGs
Table 18 shows inductive link prediction performance of MAYPL and other baseline methods on all 12 different datasets provided in (Lee et al., 2023b). The prefixes in the dataset names denote their origins (WK for Wikidata68K (Gesese et al., 2022), FB for FB15K237 (Toutanova & Chen, 2015), and NL for NELL-995 (Xiong al., 2017)), while the suffixes indicate relations (Lee al., 2023b). For example, the WK-75 dataset is
the ratio of triplets in inference graph that contains extracted from Wikidata68K, and its 75% triplets in the inference graph involve new relations. In Table 6, the best result is boldfaced, the second-best result is underlined, and the third-best result is italic. Among the baseline methods, RAILD only can be run on WK-100, FB-100, and NL-100. Also, GralL, CoMPILE, SNRI, INDIGO, and RMPI are only run on datasets extracted from NELL-995 due to their scalability issues (Lee et al., 2023b).
On each dataset, we use three standard link prediction metries, MRR, Hit and Hitl. To assess the relative performance of each model in comparison to the pest-performing model for each metric on given dataset, we compute the loss follows:
lossmetrie.modt SCOTemetric.bes scoremeriic.mate
scoremetric.eo
where mcoremetric.eett denotes the performance of the best-performing model using the given metric, and scoremetri..moeel denotes the performance of a target model. Since each metric presents different perspective of model's performance, and model's performance varies depending metrics and ratios of triplets with new relations, introduce holistic measure called the total loss, providing comprehensive evaluation of model's performance across multiple metrics and varying ratios of triplets with new relations. The total loss of a model is obtained by summing its losses across all metrics for datasets with the same origin. A lower total loss indicates superior performance.
17

---

Structure Is All You Need: Structural Representation Learning on Hyper- Relatinnllaional Knowledge Graphs
Table 18: Inductive Link Prediction Performance on 12 Different Datasets in (Lee et al., 2023b).
WK-100
WK-75
WK-50
WK-25
Total
MRR
Hitl0
Hitl
MRR
Hit10
Hitl
MRR
Hit10
Hitl
MRR
Hitl0
Hitl
Loss
CompGCN
0.003
0.009
0.000
0.015
0.028
0.003
0.003
0.002
0.001
0.009
0.020
0.000
11.679
NodePiece
0.007
0.018
0.002
0.021
0.052
0.003
0.008
0.013
0.002
0.053
0.122
0.019
10.917
NeuralLP
0.009
0.016
0.005
0.020
0.054
0.004
0.025
0.054
0.007
0.068
0.104
0.046
10.269
DRUM
0.010
0.019
0.004
0.020
0.043
0.007
0.017
0.046
0.002
0.064
0.116
0.035
10.542
BLP
0,012
0,025
0,003
0,043
0.089
0.016
0.041
0.092
0.013
0.125
0.283
0.055
9,003
QBLP
0.012
0.025
0.003
0.044
0.091
0.016
0.035
0.080
0.011
0.116
0.294
0,042
9.222
NBFNet
0.014
0.026
0.005
0.072
0.172
0.028
0.062
0.105
0.036
0.154
0.301
0.092
7.571
RED-GNN
0.096
0.136
0.070
0.172
0.290
0.110
0.058
0.093
0.033
0.170
0.263
0.111
4.864
RAILD
0.026
0.052
0.010
InGram
0.107
0.169
0.072
0.247
0.362
0.179
0.068
0.135
0.034
0.186
0.309
0.124
3.342
HyRel
0.091
0.165
0.059
0.255
0.389
0.187
0.068
0.138
0.036
0.191
0.316
0.125
3.403
InKGE-EIPR
0.125
0.209
0.086
0.262
0,380
0.188
0.076
0.164
0.038
0.203
0.330
0.137
2.300
MAYPL
0.139
0.236
0.091
0.290
0.441
0.213
0.109
0.230
0.054
0.275
0.501
0.169
0.000
FB-100
FB-75
FB-50
FB-25
Total
MRR
Hit10
Hitl
MRR
Hit10
Hitl
MRR
Hit10
Hitl
MRR
Hit10
Hitl
Loss
CompGCN
0.015
0.025
0.008
0.013
0.026
0.000
0.004
0.006
0.002
0.003
0.004
0.000
11,698
NodePiece
0.006
0.009
0.001
0.016
0.029
0.007
0.021
0.048
0.006
0.044
0.114
0.011
11.007
Neurall.P PP
0.026
0.057
0.007
0.056
0.099
0.030
0.088
0.184
0.043
0.164
0.309
0.098
7.897
DRUM
0.034
0.077
0.011
0.065
0.121
0.034
0.101
0.191
0.061
0.175
0.320
0.109
7.350
BLP
0.017
0.035
0.004
0.047
0.085
0.024
0.078
0.156
0.037
0.107
0.212
0.053
9.001
QBLP
0,013
0,026
0,003
0,041
0.084
0.017
0.071
0.147
0.030
0.104
0.226
0.043
9.268
NBFNet
0.072
0.154
0.026
0.089
0.166
0.048
0.130
0.259
0.071
0.224
0.410
0.137
5.768
RED-GNN
0.121
0.263
0.053
0.107
0.201
0.057
0.129
0.251
0.072
0.145
0.284
0.077
6.168
RAILD
0.031
0.048
0.016
InGram
0.223
0.371
0.146
0.189
0.325
0.119
0.117
0,218
0.067
0.133
0.271
0.067
4.761
HyRel
0.282
0.463
0.188
0.277
0.433
0.196
0.178
0.333
0.101
0.210
0.420
0.114
1.451
InKGE-EIPR
0.333
0.509
0.245
0.285
0.462
0.195
0.204
0.376
0.123
0.227
0.430
0.125
0.344
MAYPL
0.341
0.512
0.254
0.282
0.436
0.197
0.205
0.361
0.129
0.238
0.438
0.144
0.107
NI-100
NI.-75
NI.50
NL-25
Total
MRR
Hitl0
Hitl
MRR
1it
Hitl
MRR
Hit10
Hit1
MRR
Hit10
Hitl
Loss
Grall.
0.135
0.173
0.114
0.096
0.205
0.036
0.162
0.288
0.104
0.216
0.366
0.160
6.830
COMPILE
0.123
0.209
0.071
0.178
0.361
0.093
0.194
0.330
0.125
0.189
0.324
0.115
6.180
SNRI
0.042
0.064
0.029
0.088
0.177
0.040
0.130
0.187
0.095
0.190
0.270
0.140
8,226
INDIGO
0.160
0.247
0.109
0.121
0.156
0.098
0.167
0.217
0.134
0.166
0.206
0.134
6,907
RMPI
0.220
0.376
0.136
0.138
0.275
0.061
0.185
0.307
0.109
0.213
0.329
0.130
5,870
CompGCN
0.008
0.014
0.001
0.014
0.025
0.003
0.003
0.005
0.000
0.006
0.010
0.000
11.793
NodePicce
0,012
0.018
0.004
0.042
0,081
0.020
0.037
0.079
0.013
0.098
0.166
0.057
10.446
NeuralLP
0.084
0.181
0.035
0.117
0.273
0.048
0.101
0.190
0.064
0.148
0.271
0.101
8,058
DRUM
0.076
0.138
0.044
0.152
0.313
0.072
0.107
0.193
0.070
0.161
0.264
0.119
7.683
BLP
0.019
0.037
0.006
0.051
0.120
0.012
0.041
0.093
0.011
0.049
0.095
0.024
10,689
QBLP
0.004
0.003
0.000
0.040
0.095
0.007
0.048
0.097
0.020
0.073
0.151
0.027
10.666
NBFNet
0.096
0.199
0.032
0.137
0.255
0.077
0.225
0.346
0.161
0.283
0.417
0.224
5.700
RED-GNN
0.212
0.385
0.114
0.203
0.353
0.129
0.179
0.280
0.115
0.214
0.266
0.166
5.292
RAILD
0.018
0.037
0.005
InGram
0.309
0.506
0.212
0.261
0.464
0.167
0,281
0.453
0.193
0.334
0.501
0.241
1.965
HyRel
0.394
0.574
0.299
0.305
0.502
0.205
0.321
0.520
0.222
0.348
0.541
0.263
0.403
InGGE-EIPR
0.333
0.533
0.247
0.318
0.525
0.211
0.315
0.529
0.217
0.350
0.554
0.267
0.684
MAYPL
0.374
0.523
0.301
0.289
0.479
0.201
0.343
0.508
0.262
0.352
0.532
0.262
0.465
18

---

Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs
For datasets derived from Wikidata68K (i.e., WK-100, WK-75, WK-50, WK- 25), MAYPL. achieves the best performance across all metrics. Similarly, for datasets from FB 15K237 (i.e., FB-100, FB-75, FB-50, FB-25), MAYPL sig-
Table 19: Sum of total losses for all 12 datasets provided in (Lee et al., 2023b).
nificantly outperforms the baseline models, showing a considerable gap to the best baseline in the total loss. For datasets extracted from NEIL--995 (i.e., NI- 100, NL-75, NL-50, NL-25), MAYPL demonstrates competitive performance compared to the best-performing baseline, HyRel, while surpassing all the other baseline methods. Note that HyRel is not as competitive as MAYPL in the other Wikidata68K and FB 15K237 datasets. Table 19 shows the sum of total losses, i.e., the sum of losses for all metrics over all 12 datasets provided in (Lee et al., 2023b). This comprehensive evaluation shows that MAYPI substantially outperforms the baseline methods in inductive link prediction on KGs.
Sum of Total Losses
CompGCN
35.170
NodePiece
32.370
NeuralLP
26.224
DRUM
25.575
BLP
28.692
QBLP
29.155
NBFNet
19.039
RED-GNN
16.323
InGram
10.068
HyRel
5.257
F. More Detailed Explanations about Ablation Studies
3.328
In Table 7, we consider six different variations of MAYPL. (i) introduces learn- able initial representation vectors for individual entities and relations instead of computing them using the structure-driven initializer described in Section 4.1. When the initializer is removed, performance significantly drops in both trans- ductive and inductive settings, which shows the critical role of the structure-driven initializer. (ii) removes the attentive neural message passing module proposed in Section 4.2 and the initial entity and relation representations are directly used for link prediction. The considerable performance drop explains the importance of attentive neural message passing. (iii) does not apply the attention-based aggregation and uses simple mean aggregation when updating entity and relation representations in (3) & (5). We see that attentively aggregating representations leads to better performance than the simple mean aggregation. (iv) does not apply the relation aggregation process of Eq.(5) and updates relation representation using simple learnable projection matrix. The performance drop shows the effectiveness of our relation aggregation module. (v) replaces all Hadamard products with concatenations in computing messages of facts, entities, and relations in Section 4.2 and Section 4.3. With these changes, the sizes of the corresponding projection matrices are also appropriately modified. The original design using the Hadamard product is more effective than the concatenation. (vi) separates parameters for computing entity representations Section 4.1 and Section 4.2 and computing a query's representation in link prediction in Section 4.3, while the original MAYPL uses the parameters computed in Section 4.1 and Section 4.2 to perform link prediction in Section 4.3. The performance drop shows the advantage of the parameter sharing in MAYPL.
MAYPI
0.572
G. A Additoona Case Studies about MAYPL
Table 20 shows MAYPL's top 3 predictions on problems in WikiPeople where the ground-truth answers are boldfaced. Note that MAYPL. correctly predicts the answers to these problems, and MAYPL'ss top predictions each problem include entities of the same type the answer. For example, Problem #4, when the answer is Ron Howard, the second and the third predictions primary triplets are identical: (?, awa r rded, Oscar for Best Di rictorr). However, their answers are different due to their different qualifiers. MAYPL accurately predicts the answers by appropriately considering the qualifiers in facts.
also directors, James Cameron and Steven Spielberg. In Problem #3 and Problem #4, their
Table 20: Top 3 predictions made by MAYPL for problems in WikiPeople The answer is boldfaced.
(Marilyy Monsoe,
Angeles). ((ccuntry, USA), located in, (n))
Prediction
California, New York, New York City
Problem 42
([steven spielberg.nomiin nominated for, uscar for Best Director), ((aubject of, (far work, schindler's List)) 66th Oscars. 71ss Oscazs, 47th Oscars
Prodiction
#3
(1?, awarded, Oscar sest Director), ((subject of, 60th (scars)
Last emperor))
Prediction
Bernardo Mertolucci.Mi.. Forman, David Byrne
Problem #4
(13, awarded, Oscar for sest pirector), {(For Work, seautiful Xind))
Prodiction
Ron Howard, James Camezon, Steven Spielberg
19


Reviews:
[{"review_id": "d0L7l4DuJQ", "reviewer": "Reviewer_iapV", "summary": "The paper proposes a structure-driven representation learning method for hyper-relational knowledge graphs (HKGs). Traditional knowledge graph models extend simple triplets into hyper-relational facts by incorporating qualifiers, but many existing methods fail to effectively utilize the structure of HKGs. The authors introduce a novel representation learning approach MAYPL (Message pAssing framework for hYper-relational knowledge graph rePresentation Learning). The study concludes that emphasizing HKG structure can lead to superior performance on the hyper-relational knowledge graph completion.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**C1**\\\nWhile the input of many HKG methods is only an HKG itself without extra features, most existing HKG methods (e.g., GRAN, Hy-Transformer, and HyNT) consider an HKG as a set of individual hyper-relational facts and feed each fact into a transformer independently, disregarding the interconnectedness between different facts; they do not effectively and sufficiently consider the HKG’s structure. While StarE and HAHE introduced a GNN-based encoder, they are limited (e.g., ignoring relations) and still depend on a transformer-based decoder for link prediction. None of existing HKG methods comprehensively utilize the graph structure of an HKG by considering the multi-hop structural information between entities, relations, and facts along with their composition, co-occurrence, connectivity, and positional information, as MAYPL does. Furthermore, MAYPL does not employ a non-GNN decoder and only involves structure-based learning and prediction. MAYPL’s superior performance on a range of link prediction tasks demonstrates the importance of thoroughly leveraging the structural information of HKGs, and our paper title intends to emphasize this point.\n\n**C2**\\\nMAYPL is a standalone, end-to-end GNN model, not just a more advanced GNN-based encoder. Note that MAYPL does not require any non-GNN decoder, and this distinguishes it from existing HKG approaches. Unlike existing HKG methods that employ GNN-based encoders but rely on non-GNN decoders, such as transformers, for link prediction, MAYPL directly leverages the parameters learned in the structure-driven initializer and the attentive neural message passing, thereby eliminating the need of an external decoder for link prediction. MAYPL only utilizes the structure of HKGs from initialization to link prediction, which turns out to be crucial for reasoning on HKGs; this is the claim we make in our paper.\n\n**C3**\\\nOn WikiPeople-, we compare MAYPL and other methods regarding the training time, the number of parameters, and MRR, where we present all baselines whose training times and parameter sizes are available.\n||Training time|# parameters|MRR|\n|-|-|-|-|\n|StarE|4 days|8.2M|0.491|\n|GRAN|10h|>8.9M|0.503|\n|Hy-Transformer|5h|7.8M|0.501|\n|HyperFormer|N/A|67.0M|0.473|\n|HAHE|12h|30.2M|0.509|\n|MAYPL|20h|10.5M|0.519|\n\nMAYPL requires 20 hours of training, less than a quarter of StarE and ~1.7 times of HAHE. Also, MAYPL requires one-third of the parameters of HAHE and one-sixth of HyperFormer while using ~1.3 times the parameters of StarE and Hy-Transformer. In MRR, MAYPL outperforms all baselines. While we briefly mentioned our plans for improving the scalability of MAYPL in “Limitations and Future Work”, we can expand them and include the above comparison in the camera-ready version with the extra page.\\\n*[Details]* HyperFormer didn’t report its training time. While we can’t report the exact parameter count of GRAN due to its outdated environment, we know it requires at least 8.9M parameters since it learns individual entity/relation representations.\n\n**C4**\\\nWe believe you meant $\\tilde{L}$. We provide an ablation study on the impact of $\\tilde{L}$. The table below shows MAYPL’s MRRs regarding different values of $\\tilde{L}$ on WK-50 and WD20K(100)v2. The last row is the best baseline methods' MRRs.\n|$\\tilde{L}$|WK-50|WDv2|\n|-|-|-|\n|1|0.089|0.277|\n|2|0.086|0.272|\n|3|0.109|0.298|\n|4|0.088|0.281|\n|5|0.095|0.262|\n|6|0.090|0.248|\n|7|0.096|0.235|\n|8|0.097|0.263|\n|9|0.095|0.256|\n|10|0.095|0.249|\n|Best baseline|0.076|0.067|\n\nWhile $\\tilde{L}$ affects the performance, there is no evidence of over-smoothing since there is no tendency that a larger $\\tilde{L}$ continuously degrades the performance. While $\\tilde{L}$ is a tunable hyperparameter, MAYPL consistently outperforms the best-performing baselines across all values of $\\tilde{L}$.\\\n&nbsp;&nbsp;&nbsp;&nbsp;For deeper analysis, we analyzed the Dirichlet energy[1] of MAYPL trained with $\\tilde{L}=10$, computed at each layer of our structure-driven initializer. Following Equation (3) of [1] and applying it to HKGs, we computed the Dirichlet energy values across different layers, shown in the table below:\n|$\\tilde{l}$|1|2|3|4|5|6|7|8|9|10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|WK-50|23.6|25.7|25.4|26.8|26.6|27.6|23.6|24.9|22.9|26.7|\n|WDv2|54.4|57.6|58.3|59.5|60.8|68.4|68.2|67.3|67.2|55.8|\n\nAccording to [1], the Dirichlet energy should constantly decrease as the depth increases when over-smoothing occurs. In our structure-driven initializer, the Dirichlet energy does not consistently decrease with increasing depth, indicating that over-smoothing does not occur. Additionally, as discussed in Appendix D.1, we employ residual connections and layer normalization in all layers of MAYPL, which are commonly-used strategies for preventing over-smoothing in GNNs [2].\n\n[1] “A Survey on Oversmoothing in Graph Neural Networks”, arXiv 2023.\\\n[2] “Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs”, ICLR 2025.", "claims_and_evidence": "The major claims are well supported by the empirical studies.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are appropriate.", "theoretical_claims": "N.A. This paper does not include the theoretical analysis.", "experimental_designs_or_analyses": "Checked. The experiments mainly follow standard procedures.", "relation_to_broader_scientific_literature": "This paper belongs to a subfield of graph learning—hyper-relational graphs—which may have some influence on other areas of graph learning, especially those that focus purely on graph structures without emphasizing node and edge features.", "essential_references_not_discussed": "Overall, the paper provides a well-conducted literature review, covering well-known works on hyper-relational knowledge graphs, knowledge hypergraphs, and N-ary relational data.", "other_strengths_and_weaknesses": "**Strengths**\n\nS1. The paper presents a clear and well-structured approach with strong logical coherence.\n\nS2. The experimental evaluation is comprehensive, covering major baselines, most commonly used datasets in recent years, and various experimental settings.\n\n**Concerns**\n\nC1. The title, “Structure Is All You Need: Structural Representation Learning on Hyper-Relational Knowledge Graphs”, may need reconsideration. In the standard knowledge graph setting (including hyper-relational knowledge graphs), the input typically consists only of the graph structure, while node and edge features are often ignored (as they are generally assumed to be absent). In other words, the input itself is already purely structural, meaning that the phrase “Structure Is All You Need” may be meaningless since “Structure Is All You Have”.\n\nC2. Compared to Transformer-based models, as well as methods in KHG and NNR, GNNs naturally preserve the topological structure of graphs without losing feature information. From this perspective, the paper primarily upgrades GNN-based methods by hierarchically modeling the main triple graph, the qualifier relationships, and the intrinsic connections between complex facts—essentially serving as a more advanced GNN-based encoder. Therefore, there is a risk that the paper overclaims its core contributions.\n\nC3. A crucial aspect not discussed in this paper is the efficiency comparison between methods, including training complexity, inference complexity, and # parameters. Intuitively, this approach might have one of the highest training times and parameter sizes among all methods. Besides Tables 14–16, the authors should objectively discuss and present any efficiency limitations (if they exist), such as training time comparisons, parameter size comparisons, and learning curve comparisons.\n\nC4. The entity and relation learning approach in Section 4.1 is intuitive and reasonable. However, does this learning strategy potentially introduce or exacerbate the over-smoothing issue? Beyond the results in Tables 14–16, the authors should provide an ablation study on the impact of the $\\hat{L}$ setting.\n\nC5. Figure 1 can be further improved to be clearer and more effectively illustrate the core design ideas of the paper. The current version still appears somewhat messy.", "other_comments_or_suggestions": "N.A.", "questions_for_authors": "Please focus your response on C1, C2, C3, and C4.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OKDT103TEz", "reviewer": "Reviewer_ZDZW", "summary": "This paper proposes MAYPL, a GNN-based method designed for inductive reasoning on hyper-relational knowledge graphs (HKGs), a specific variant of knowledge graphs. MAYPL initializes representation vectors based on the HKG structure and utilizes a structure-driven message-passing mechanism, enabling it to perform inductive inference on unseen entities and relations. Extensive experimental results demonstrate its robust performance, outperforming existing methods across multiple benchmarks.\n\n## update after rebuttal:\n\nThank the authors for their responses. They state that MAYPL is not a foundation model, yet assert its claimed inductive capability on entirely new HKGs. Avoiding direct comparison with ULTRA remains unconvincing. Besides, no concrete solutions were provided to address the model's architectural complexity and inefficiency. Therefore, I would like to keep my score.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">The evaluation of the proposed method(designed for HKGs) on normal KG inductive datasets raises concerns about fairness. Baselines like NBFNet and RED-GNN are not specifically designed for relation inductive reasoning, making the comparison less equitable. To better demonstrate superiority in inductive reasoning, a comparison with ULTRA under the same pre-training KG conditions would be more appropriate.\n\n**A1.** The baselines in our paper, e.g., NBFNet and RED-GNN, have been widely adopted for inductive KG completion literature, including relation inductive reasoning; our experiments also include all available state-of-the-art relation inductive reasoning models[1,2,3]. ULTRA is a foundation model, whereas MAYPL is not a foundation model. While foundation models are pre-trained on multiple KGs and perform link prediction on various KGs, non-foundation models like MAYPL are trained on a single training graph and usually tested on a single inference graph. These fundamentally different training and evaluation settings make a direct comparison between ULTRA and MAYPL inappropriate. As mentioned in “Limitations and Future Work”, we plan to extend MAYPL into a foundation model as future work.\n\n[1] InGram: Inductive Knowledge Graph Embedding via Relation Graphs, ICML 2023\\\n[2] Generalize to Fully Unseen Graphs: Learn Transferable Hyper-Relation Structures for Inductive Link Prediction, MM 2024\\\n[3] Inductive Knowledge Graph Embedding via Exploring Interaction Patterns of Relations, CIKM 2024\n\n>This design is tailored for the specific structure of facts in HKGs, but lack significant novelty. The initilization and message-passing encoding design are complex and somewhat redundant, leading to inefficiency. For technical design, it would be expected to adhere to Occam's Razor and pursue a more lightweight yet equally effective approach. Given that recent work has already proposed full inductive reasoning models for KGs(such as ULTRA), the technical contribution of this paper appears incremental.\n\n**A2.** Our structure-driven initializer and the attentive neural message passing are not redundant, as our ablation studies indicate in Table 7, where (i) and (ii) indicate the cases of removing the initializer and the attentive message passing, respectively. Removing one of these modules degrades performance drastically, indicating both are critical in MAYPL. Indeed, these modules serve distinct roles; the structure-driven initializer considers the interconnections and co-occurrences between entities and relations, whereas the attentive neural message passing computes fact-level messages using compositional information to update the entity/relation representations. This stepwise approach enables MAYPL to effectively capture the diverse structural information of HKGs.\\\n&nbsp;&nbsp;&nbsp;&nbsp;Existing full inductive reasoning models, e.g., ULTRA and InGram, can handle only vanilla KGs but not HKGs, whereas MAYPL can handle both KGs and HKGs. Even if we scope down to KGs, MAYPL substantially differs from any existing KG methods. For example, while existing full inductive KG methods construct separate relation graphs that consider only simple incidence relationships between relations, MAYPL directly utilizes the given KG structure and incorporates the interconnections between entities and relations along with the triplets’ composition information. MAYPL’s performance on inductive link prediction on KGs shows its superiority in modeling KGs(Table 4). In addition to inductive reasoning, MAYPL also achieves the state-of-the-art performance on transductive HKG reasoning tasks. Please read our response to **C1** of **Reviewer iapV** for how MAYPL is distinguished from prior HKG methods. MAYPL is the first structure-oriented method for HKGs that only utilizes the structure of HKGs from initialization to link prediction.\n>In Tables 14 and 15, the authors present the runtime and memory costs for each dataset. However, these costs are notably high for the small-scale KGs used(with only thousands of entities), which raises concerns about the scalability and practicality of the method.\n\n**A3.** According to Tables 14-16, MAYPL needs 35 minutes and 1.2GB of memory to process WK-50(12K entities, 82K facts). On MFB-IND(3K entities, 337K facts), MAYPL needs 7 hours and 27GB. On WikiPeople(48K entities, 306K facts), the largest dataset in our paper, MAYPL needs 21 hours and 45.8GB. While these demonstrate a reasonable cost relative to the HKGs' scale, one exception is WP-IND. Despite containing only 4K entities and 4K facts, MAYPL needs 8 hours and 1.4GB on WP-IND. This is the only dataset that exhibits high training time compared to its scale, which we attribute to its sparsity, requiring more epochs for training. For a comparison to other methods on scalability, please read our response to **C3** of **Reviewer iapV**, showing that MAYPL is comparable to the baseline methods in scalability while achieving the best prediction performance.", "claims_and_evidence": "The claim about model superior performance on inductive reasoning is not convincing enough, please check the comments in  Experiments part.", "methods_and_evaluation_criteria": "The structure-driven representation learning in the  proposed method makes sense. This design is tailored for the specific structure of facts in HKGs, but lack significant novelty. The initilization and message-passing encoding design are complex and somewhat redundant, leading to inefficiency. For technical design, it would be expected to adhere to Occam's Razor and pursue a more lightweight yet equally effective approach. Given that recent work has already proposed full inductive reasoning models for KGs (such as ULTRA), the technical contribution of this paper appears incremental.", "theoretical_claims": "The paper lacks theoretical analysis to substantiate the proposed design.", "experimental_designs_or_analyses": "The evaluation of the proposed method (designed for HKGs) on normal KG inductive datasets raises concerns about fairness. Baselines like NBFNet and RED-GNN are not specifically designed for relation inductive reasoning, making the comparison less equitable. To better demonstrate superiority in inductive reasoning, a comparison with ULTRA under the same pre-training KG conditions would be more appropriate.", "relation_to_broader_scientific_literature": "Please check the above suggestions.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "The draft is well-organized and easy to follow.\n\nIn Tables 14 and 15, the authors present the runtime and memory costs for each dataset. However, these costs are notably high for the small-scale KGs used (with only thousands of entities), which raises concerns about the scalability and practicality of the method.", "other_comments_or_suggestions": "Please check the above suggestions.", "questions_for_authors": "Please check the above suggestions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o4R7Q91nBA", "reviewer": "Reviewer_KUd9", "summary": "This paper presents MAYPL, a structure-driven representation learning method for hyper-relational knowledge graphs. MAYPL contributes a structure-driven initializer and attentive neural message passing to learn entity and relation representations. The method is designed to handle transudative and inductive inference settings and reports extensive experimental results across multiple benchmark datasets, showing superior performance compared to existing SOTA methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1**\\\nMessage passing is a fundamental concept underlying most existing GNNs for structural learning. Therefore, GNNs are not distinguished by their adherence to the message passing paradigm but by how they compute messages and update representations. MAYPL presents its unique way of encoding messages and updating representations to thoroughly leverage the structural information of HKGs. Even if we scope down to KGs, MAYPL substantially differs from any existing KG methods. For example, while existing fully inductive KG methods construct separate relation graphs that consider only simple incidence relationships between relations, MAYPL directly utilizes the given KG structure and incorporates the interconnections between entities and relations along with the triplets’ composition information. MAYPL’s performance on inductive link prediction on KGs shows its superiority in modeling KGs(Table 4).\\\n&nbsp;&nbsp;&nbsp;&nbsp;While there have been various structure-based learning approaches for KGs, designing a purely structure-oriented representation learning method for HKGs has remained challenging since the structure of HKGs is far more complicated than KGs. As a result, most existing HKG methods(e.g., GRAN, Hy-Transformer, and HyNT) consider an HKG as a set of individual hyper-relational facts instead of considering an HKG as a graph itself. While StarE and HAHE introduced a GNN-based encoder, they are limited(e.g., ignoring relations) and still rely on a non-GNN decoder for link prediction.\\\n&nbsp;&nbsp;&nbsp;&nbsp;None of existing HKG methods comprehensively utilize the graph structure of an HKG by considering the multi-hop structural information between entities, relations, and facts along with their composition, co-occurrence, connectivity, and positional information, as MAYPL does. Furthermore, MAYPL does not employ a non-GNN decoder and only involves structure-based learning and prediction. **We emphasize that MAYPL is the first structure-oriented method for HKGs that can be applied in both transductive and inductive learning settings.**\n\n**Q2**\\\nThe ICML submission deadline was Jan. 30th, while the preprints of WWW 2025 papers were made public on OpenReview on Jan. 29th. According to the reviewer guidelines, *authors cannot expect to discuss other papers that have only been made publicly available within four months of the submission deadline*. We believe **our paper should neither be evaluated nor penalized by WWW 2025 papers**[1,2]. Besides, we found [1]’s results questionable due to potential test leakage in their code: the model uses other facts from the test set during prediction (line 113 in main.py). But, since the current version is not final yet (WWW 2025 starts on Apr. 28), it can be fixed in the final version. Thus, comparing the not-final results of [1] with MAYPL is inappropriate.\\\n&nbsp;&nbsp;&nbsp;&nbsp;HyperMono[3] is an arXiv paper that has not been peer-reviewed; the table below shows that MAYPL outperforms HyperMono. Also, we have made every effort to cover all relevant HKG papers published at top-tier venues at the time of submission.\n|||WD50K|||WP-||\n|-|-|:-:|-|-|:-:|-|\n||MRR|Hit10|Hit1|MRR|Hit10|Hit1|\n|HyperMono|0.375|0.522|**0.298**|0.494|**0.657**|0.390|\n|MAYPL|**0.381**|**0.544**|0.297|**0.519**|**0.657**|**0.444**|\n\n[1] Hyper-Relational Knowledge Representation Learning with Multi-Hypergraph Disentanglement, WWW 2025\\\n[2] HySAE: An Efficient Semantic-Enhanced Representation Learning Model for Knowledge Hypergraph Link Prediction, WWW 2025\\\n[3] HyperMono: A Monotonicity-aware Approach to Hyper-Relational Knowledge Representation, arXiv\n\n**Q3 & W1**\\\nOur structure-driven initializer **learns the parameters described in Section 4.1, which are used to compute the initial representations of entities and relations**. It is not designed to accelerate convergence or improve computational efficiency. To assess its impact, we conducted an ablation study by removing the structure-driven initializer (check (i) in Table 7), which led to a significant degradation in performance. This confirms the importance of our structure-driven initializer. Regarding scalability, we provided the costs of MAYPL in Tables 14-16. For additional scalability comparison between MAYPL and other methods, please read our response to **C3** of **Reviewer iapV**.\n\n**W2**\\\nHKGs used in our paper, WD50K, WikiPeople, and WikiPeople, are the largest among the most commonly used HKGs available in the literature.\n\n**W3**\\\nWe provided case studies in Table 8 and Table 19 to help readers understand how MAYPL works. Table 8 shows that MAYPL operates by first computing reasonable initial representations using the structure-driven initializer and then refining these representations through attentive neural message passing. Table 19 shows that the top 3 predictions of MAYPL include entities of the same type as the answer, and MAYPL accurately predicts answers by appropriately considering the qualifiers in facts.", "claims_and_evidence": "The claims made in the submission are generally well-supported by evidence. The authors demonstrate through extensive experiments that their structure-driven approach outperforms existing methods on various link prediction tasks. The results across multiple datasets (WD50K, WikiPeople, NL-50, etc.) consistently show that MAYPL achieves superior performance in MRR, Hit10, and Hit1 metrics.", "methods_and_evaluation_criteria": "The methods make sense for the HKGs problem. The structure-driven initializer and attentive neural message-passing mechanism are effective for performance.  \n\nHowever, I have seen too many papers based on RGCN that still make trivial modifications in message passing. This paper's method only shows unique effectiveness in the context of HKG but does not propose a new paradigm. \nMoreover, the comparison does not include some of the latest state-of-the-art baselines [1][2], which undermines the value of the intensive experimental results.\n\n- [1] Hyper-Relational Knowledge Representation Learning with Multi-Hypergraph Disentanglement. WWW2025.\n- [2] HySAE: An Efficient Semantic-Enhanced Representation Learning Model for Knowledge Hypergraph Link Prediction. WWW2025.\n- [3] HyperMono: A Monotonicity-aware Approach to Hyper-Relational Knowledge Representation. arxiv", "theoretical_claims": "Neither is proof for the design of Implant's an experiment-oriented paper.", "experimental_designs_or_analyses": "No, there is no proof of the design of MAYPL;  it's an experiment-oriented paper.", "relation_to_broader_scientific_literature": "The paper's contributions are well-situated within the broader literature on KGRL. It builds upon previous work on HKGs while addressing limitations in utilizing structural information. It references prior works in KGC(ind. / trans. setting) and structural representation learning.", "essential_references_not_discussed": "I think the key comparison of recent sota is necessary.\nAnd they are highly related to HKG completion tasks. \n- [1] Hyper-Relational Knowledge Representation Learning with Multi-Hypergraph Disentanglement. WWW2025\n- [2] HySAE: An Efficient Semantic-Enhanced Representation Learning Model for Knowledge Hypergraph Link Prediction. WWW2025\n- [3] HyperMono: A Monotonicity-aware Approach to Hyper-Relational Knowledge Representation. arxiv\n\nThere must be others that have escaped my attention, in addition to the three I mentioned.", "other_strengths_and_weaknesses": "## Strengths\n- Comprehensive introduction of structural representation learning from traditional RGCN into HKG\n- Effective test setting handing of ind. Completion\n\n## Weaknesses\n- Limited discussion of computational complexity and scalability, which decides on real-world applications. A better initializer should contribute to faster convergence and better efficiency, but there is a lack of appropriate discussion.\n- It remains a question of how MAYPL performs on larger HKGs.\n- A case study or a more in-depth analysis of how MAYPL works could help readers better understand the method despite its promising but not very informative experimental performance.", "other_comments_or_suggestions": "Providing more detailed visualizations of the attention mechanisms and how they capture structural patterns would enhance the interpretability of the results. Also, a more comprehensive comparison of recent sota is needed; HKRL seems stronger than MAYPL.", "questions_for_authors": "1. Structural learning on knowledge graphs (KGs) has already been explored on various KGs. I actually don't see much new from the formulas. I need stronger clarification to demonstrate that this paper's method is a unique structural learning characteristic in the context of HKG. This affects my overall impression of whether the paper is a creative work or simply a modification of the message-passing mechanism, which influences my perception of the upper limit of the score I would assign to this paper.\n2. The paper has sufficient experiments, which is a major strength. However, I need to see comparisons with some of the new methods I mentioned that have been overlooked in terms of performance or efficiency to convince me that this is truly a state-of-the-art approach.\n3. The training process could achieve the same convergence effect without the proposed initialization, albeit requiring a longer time.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kxym3e61qb", "reviewer": "Reviewer_wV5x", "summary": "This paper presents MAYPL, a novel structure-driven representation learning method for hyper-relational knowledge graphs (HKGs). Unlike existing methods that rely on transformers or GNNs with limited structural utilization, MAYPL fully exploits the structural properties of HKGs to achieve state-of-the-art link prediction performance. The core idea of MAYPL is to initialize entity and relation representations based on their connectivity patterns, followed by an attentive neural message-passing mechanism that aggregates fact-level, entity-level, and relation-level features. This purely structure-based approach enables MAYPL to perform both transductive and inductive link prediction, making it capable of generalizing to entirely new entities and relations at inference time. The paper demonstrates that MAYPL outperforms 40 baseline methods on 10 benchmark datasets, covering both standard and inductive link prediction tasks. The results highlight that leveraging structure alone is sufficient for effective knowledge graph reasoning, challenging the necessity of complex embedding-based models.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">I believe HittER is also quite relevant to this work. Hitter (Hierarchical Transformers for Knowledge Graph Embeddings) (Chen et al, EMNLP 2021) is a transformer-based model specifically designed for hyper-relational knowledge graphs (HKGs). Instead of relying on GNNs or embeddings, HittER models hyper-relational facts as a sequence of tokens and applies attention mechanisms to capture dependencies among entities, relations, and attributes. Both HittER and MAYPL focus on hyper-relational knowledge graphs. While HittER uses transformers for relational reasoning, MAYPL relies on a multi-level message-passing framework to capture structural dependencies. I notice this paper already includes discussion on transformer-based methods, but HittER is one of the SOTA works, so it would be better to include it as well.\n\n**R1.** Since HittER[1] is designed for handling vanilla knowledge graphs, not hyper-relational knowledge graphs, a direct comparison of HittER and MAYPL may not be feasible. However, we believe extending MAYPL to QA tasks, similar to how HittER is applied to those tasks, can be an interesting future research. We will discuss this point and cite HittER in this context in the camera-ready version (if this paper is accepted).\n\n[1] Chen et al., “HittER: Hierarchical Transformers for Knowledge Graph Embeddings”, EMNLP 2021.\n\n>In the introduction, the authors mention some previous works use simple one-hop neighborhood information to learn representations on HKGs which is not effective enough. But in the design of MAYPL, the definition of message passing and structural initialization only involves the co-occur entities/relations, facts and incident entities, right? Those information are also with-in one-hop in the HKG. Could you explain more about how you can guarantee the structural information beyond one-hop are also well-captured?\n\n**R2.** As discussed in our introduction, some prior works (e.g., HyperFormer[2]) use simple one-hop neighborhood information. For example, when predicting a missing tail entity in an incomplete fact $((v_1,r_1,?), \\\\{(r_2,v_2), \\cdots, (r_k, v_k)\\\\})$, HyperFormer collects all facts whose head entity is $v_1$, and each fact is individually encoded using a transformer. Those encoded facts are then averaged to be used for link prediction. Since each fact is independently fed into a transformer and only facts that include $v_1$ are considered, the structural information in this approach is restricted to the one-hop neighbors of the head entity $v_1$.\\\n&nbsp;&nbsp;&nbsp;&nbsp;In contrast, MAYPL captures multi-hop structural information through its attentive neural message passing mechanism that employs $L$ layers to consider $L$-hop distant neighboring information. In the attentive neural message passing, the message of a fact is computed based on the composition and connectivity information of its entities and relations, as well as their positions. These messages are then attentively aggregated to update the representations of entities and relations. When an entity’s representation is updated in the first layer, it incorporates information from its one-hop neighbors by aggregating facts’ messages to which it belongs. Then, since each entity already encodes its one-hop structural information, the subsequent update of an entity representation facilitates the information of the neighbors of its direct neighbors, effectively capturing two-hop structural information. After $L$ layers, the entity representations are updated by considering their $L$-hop distant neighbors. The same principle also applies to relations. In this way, MAYPL captures the structural information up to $L$ hops.\n\n[2] Hu et al., “HyperFormer: Enhancing Entity and Relation Interaction for Hyper-Relational Knowledge Graph Completion”, CIKM 2023.", "claims_and_evidence": "The major claims in the paper are largely supported by empirical results and theoretical reasoning:\n1.  MAYPL fully exploits the structural properties of hyper-relational knowledge graphs (HKGs) for representation learning\n- The proposed method uses only structural information (without textual embeddings or external node features) and achieves strong results.\n- The architecture includes fact-level, entity-level, and relation-level message passing, which aligns with the claim of capturing hierarchical structural properties.\n2. MAYPL generalizes to unseen entities and relations (inductive reasoning) effectively\n- Mainly supported by evaluation result.", "methods_and_evaluation_criteria": "Mainly yes, \nOn the method side, MAYPL is designed to fully leverage graph structure rather than relying on embedding-based models, which aligns well with the need for interpretable, structure-driven reasoning in HKGs. The message-passing mechanism across fact-level, entity-level, and relation-level information is a strong design choice for capturing hyper-relational dependencies. Also initializing the embeddings with just structural data can help in inductive setting is reasonable, since that is the only information we have about the new entities or relations. \n\nOn the evaluation side, the evaluation covers a diverse range of transductive and inductive link prediction datasets, ensures MAYPL’s results are not dataset-specific. The result also proves that MAYPL is effective in both settings. The evaluation metrics are standard in KGC tasks. The case study on most similar relations and entities at the start and end of the training also supports the claims.", "theoretical_claims": "The work doesn't have proof for theoretical claims. The claim about MAYPL effectively capturing hyper-relational knowledge graph structure through message passing and aggregation, is supported in the model design, and the formulations for message passing across different levels of the graph aligns with standard GNN models.", "experimental_designs_or_analyses": "- Use of Multiple Benchmark Datasets for a Comprehensive Evaluation\n  - The paper evaluates MAYPL on 10 benchmark datasets, covering both standard and inductive link prediction tasks.\n  - The datasets include:\n    - Traditional KG benchmarks (FB15k-237, WN18RR).\n    - Hyper-relational datasets (JF17K, WikiPeople).\n    - Inductive reasoning datasets (New unseen entities in test).\n  - This evaluation ensures MAYPL is tested on a variety of knowledge graph structures. The inclusion of inductive datasets helps validate generalization capabilities.\n- Evaluation Metrics (MRR, Hits@K) Are Standard in KG Research\n  - The paper uses Mean Reciprocal Rank (MRR) and Hits@K as primary evaluation metrics.\n  - These metrics align with standard KG completion benchmarks, ensuring results are comparable to prior work.\n- Inclusion of Inductive Link Prediction Tasks\n  - Many KG models focus only on transductive reasoning, meaning they struggle with new entities.\n  - MAYPL is tested in an inductive setting, showing its ability to generalize beyond the training set.\n- Component Ablation Study is Useful\n  - The paper includes ablation studies to test the contribution of different MAYPL components. It helps identify which structural properties contribute most to performance and improves interpretability of the model’s reasoning process.", "relation_to_broader_scientific_literature": "he paper presents MAYPL (Message pAssing framework for hYper-relational knowledge graph rePresentation Learning), a novel structure-driven representation learning method for hyper-relational knowledge graphs (HKGs). It builds upon several existing lines of research, including graph-based reasoning for knowledge graphs, hyper-relational embedding methods, message-passing networks, and inductive KG reasoning. Its novelty lies in leveraging multi-level message passing to capture hyper-relational structure without relying on embeddings. The experiments on general KGC, inductive KGC and ablation studies present the effectiveness of learning structural information in improving the quality of representations of hyper knowledge graphs.", "essential_references_not_discussed": "1. I believe HittER is also quite relevant to this work. Hitter (Hierarchical Transformers for Knowledge Graph Embeddings) (Chen et al, EMNLP 2021) is a transformer-based model specifically designed for hyper-relational knowledge graphs (HKGs). Instead of relying on GNNs or embeddings, HittER models hyper-relational facts as a sequence of tokens and applies attention mechanisms to capture dependencies among entities, relations, and attributes. Both HittER and MAYPL focus on hyper-relational knowledge graphs. While HittER uses transformers for relational reasoning, MAYPL relies on a multi-level message-passing framework to capture structural dependencies. I notice this paper already includes discussion on transformer-based methods, but HittER is one of the SOTA works, so it would be better to include it as well.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. In the introduction, the authors mention some previous works use simple one-hop neighborhood information to learn representations on HKGs which is not effective enough. But in the design of MAYPL, the definition of message passing and structural initialization only involves the co-occur entities/relations, facts and incident entities, right? Those information are also with-in one-hop in the HKG. Could you explain more about how you can guarantee the structural information beyond one-hop are also well-captured?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes MAYPL, a structure-driven representation learning method for hyper-relational knowledge graphs (HKGs). MAYPL leverages the HKG’s structure through a structure-driven initializer and attentive neural message passing, achieving state-of-the-art performance in both transductive and inductive link prediction.

Strengths:

-	The paper presents a novel approach to hyper-relational knowledge graph representation learning by fully utilizing the structural information of HKGs.
-	The comprehensive experimental evaluation and ablation studies provide strong support for the claims made in the paper.

Weakness:

-	The paper lacks a detailed discussion on computational complexity and scalability, raising concerns about practical deployment on larger-scale HKGs.

The reviewers generally agree that MAYPL is a well-executed contribution with strong empirical results. The authors addressed some key concerns of reviewers in rebuttal with additional experiments and explanations.
Decision: Accept (poster)