Shrinking Embeddings for Hyper-Relational Knowledge Graphs
Year: 2023

Abstract:
Link prediction on knowledge graphs (KGs) has been extensively studied on binary relational KGs, wherein each fact is represented by a triple. A significant amount of important knowledge, however, is represented by hyperrelational facts where each fact is composed of a primal triple and a set of qualifiers comprising a key-value pair that allows for expressing more complicated semantics. Although some recent works have proposed to embed hyper-relational KGs, these methods fail to capture essential inference patterns of hyperrelational facts such as qualifier monotonicity, qualifier implication, and qualifier mutual exclusion, limiting their generalization capability. To unlock this, we present ShrinkE, a geometric hyper-relational KG embedding method aiming to explicitly model these patterns. ShrinkE models the primal triple as a spatial-functional transformation from the head into a relation-specific box. Each qualifier “shrinks” the box to narrow down the possible answer set and, thus, realizes qualifier monotonicity. The spatial relationships between the qualifier boxes allow for modeling core inference patterns of qualifiers such as implication and mutual exclusion. Experimental results demonstrate ShrinkE’s superiority on three benchmarks of hyper-relational KGs.

heading: 1 Introduction
text: Link prediction on knowledge graphs (KGs) is a central problem for many KG-based applications (Zhang et al., 2016; Lukovnikov et al., 2017; Lu et al., 2023; Xiong et al., 2022b; Chen et al., 2022). Existing works (Sun et al., 2019; Bordes et al., 2013) have mostly studied link prediction on binary relational KGs, wherein each fact is represented by a triple, e.g., (Einstein, educated_at, University of Zurich). In many popular KGs such as Freebase (Bollacker et al., 2007), however, a lot of important knowledge is not only expressed in tripleshaped facts, but also via facts about facts, which
taken together are called hyper-relational facts. For example, ((Einstein, educated_at, University of Zurich), {(major:physics), (degree:PhD)}) is a hyper-relational fact, where the primary triple (Einstein, educated_at, University of Zurich) is contextualized by a set of key-value pairs {(major:physics),(degree:PhD)}. Like much other related work, we follow the terminology established for Wikidata (Vrandečić and Krötzsch, 2014) and use the term qualifiers to refer to the key-value pairs.1 The qualifiers play crucial roles in avoiding ambiguity issues. For instance, Einstein was educated_at several universities and the qualifiers for degree and major help distinguish them.
In order to predict links in hyper-relational KGs, pioneering works represent each hyperrelational fact as either an n-tuple in the form of r(e1, e2, · · · , en) (Wen et al., 2016; Zhang et al., 2018; Fatemi et al., 2020; Liu et al., 2020; Abboud et al., 2020) or a set of key-value pairs in the form of {(ki : vi)}mi=1 (Guan et al., 2019, 2021; Liu et al., 2021a). However, these modelings lose key structure information and are incompatible with the RDF-star schema (Arndt et al., 2021) used by modern KGs, where both primal triples and qualifiers constitute the fundamental data structure. Recent works (Guan et al., 2020; Rosso et al., 2020) represent each hyper-relational fact as a primary triple coupled with a set of qualifiers that are compatible with RDF-star standards (Arndt et al., 2021). Link prediction is then achieved by modeling the validity of the primary triple and its compatibility with each annotated qualifier (Guan et al., 2020; Rosso et al., 2020). More complicated graph encoders and decoders (Galkin et al., 2020; Yu and Yang, 2021; Wang et al., 2021; Shomer et al., 2022) are proposed to further boost the performance. However, they require a relatively huge number of parameters
1Synonyms include statement-level metadata in RDF-star (Arndt et al., 2021) and triple annotation in provenance communities (Green et al., 2007).
13306
that make them prone to overfitting.
To encourage generalization capability, KG embeddings should be able to model inference patterns, i.e., specifications of logical properties that may exist in KGs, which, if learned, empowers further principled inferences (Abboud et al., 2020). This has been extensively studied for binary relational KG embeddings (Trouillon et al., 2016; Sun et al., 2019) but ignored for hyper-relational KGs in which not only primal triples but also qualifiers matter. One of the most important properties is qualifier monotonicity. Given a query, the answer set shrinks or at least does not expand as more qualifiers are added to the query expression. For example, a query (Einstein, educated_at, ?x) with a variable ?x corresponds to two answers {University of Zurich,ETH Zurich}, but a query ((Einstein, educated_at, ?x) , {(degree : B.Sc.)}) extended by a qualifier for degree will only respond with {ETH Zurich}. Besides, different qualifiers might form logical relationships that the model must respect during inference including qualifier implication (e.g., adding a qualifier that is implicitly implied in the existing qualifiers does not change the truth of a fact) and qualifier mutual exclusion (e.g., adding any two mutually exclusive qualifiers to a fact leads to a contradiction).
In light of this, we propose ShrinkE, a hyperrelational embedding model that allows for modeling these inference patterns. ShrinkE embeds each entity as a point and models a primal triple as a spatio-functional transformation from the head entity to a relation-specific box that entails the possible tails. Each qualifier is modeled as a shrinking of the primal box to a qualifier box. The shrinking of boxes simulates the “monotonicity” of hyperrelational qualifiers, i.e., attaching qualifiers to a primal triple may only narrow down but never enlarges the answer set. The plausibility of a given fact is measured by a point-to-box function that judges whether the tail entity is inside the intersection of all qualifier boxes. Moreover, since each qualifier is associated with a box, the spatial relationships between the qualifier boxes allow for modeling core inference patterns such as qualifier implication and mutual exclusion. We theoretically show the capability of ShrinkE on modeling various inference patterns including (fact-level) monotonicity, triple-level, and qualifier-level inference patterns. Empirically, ShrinkE achieves competitive performance on three benchmarks.

heading: 2 Related Work
text: Related works on hyper-relational KG embeddings can be categorized by their representations of facts. Prominent representations include tuple, key-value pairs, and triple+key-value pairs.
Tuple based Pioneering works view a hyperrelational fact as an n-tuple, a.k.a. n-ary fact, consisting of a single abstract relation r and its n values, i.e., r(v1, v2, · · · , vn). Functional models represent the tuple-based facts by functional mapping. For example, m-TransH (Wen et al., 2016), a generalization of TransH (Wang et al., 2014) to hyperrelational facts, projects all entities onto a relationspecific hyperplane and measures the plausibility as the weighted sum of projected embeddings. RAE (Zhang et al., 2018) improves m-TransH by further modeling the relatedness of values. Multilinear models generalize bilinear models to hyperrelational facts via multi-linear products. For example, HsimplE (Fatemi et al., 2020), m-CP (Fatemi et al., 2020), and GETD (Liu et al., 2020) generalize SimplE (Kazemi and Poole, 2018), Canonical Polyadic (CP) decomposition (Trouillon et al., 2017), and TuckER (Balazevic et al., 2019), respectively. GETD only applies to KGs with single-arity relations (Liu et al., 2021b) and S2S (Di et al., 2021) extends it to support mixed arity facts. HypE (Fatemi et al., 2020) encodes hyper-relational facts by positional convolutional filters and evaluates the facts’ plausibility using the multilinear product. However, these models ignore the semantics of relations and loosely represent a combination of all relations of the original fact (Galkin et al., 2020).
Key-value pairs NaLP (Guan et al., 2019) view each hyper-relational fact as a set of key-value pairs, i.e., {(ki : vi)}mi=1. Convolutional networks are employed to encode the key-value pairs, followed by a multi-layer perceptron (MLP) that measures the compatibility between the key and its values. RAM (Liu et al., 2021b) further models the relatedness between different keys and the relatedness between a key and all involved values. NaLP+ (Guan et al., 2021) improves NaLP by considering type information. However, the key-value-based modeling treats all key-value pairs equally and does not distinguish primal triples from qualifiers.
Triple+key-value pairs NeuInfer (Guan et al., 2020) and HINGE (Rosso et al., 2020) represent a hyper-relational fact as a primary triple combined
with a set of the key-value form of qualifiers, i.e., ((h, r, t) , {(ki : vi)}mi=1), which is compatible with the RDF-star standard (Delva et al., 2021) used in modern KGs. Both methods adopt neural networks to obtain the fact validity by measuring the validity of the primary triple and its compatibility with each qualifier. NeuInfer applies MLP while HINGE uses a convolutional network as an encoder. StarE (Galkin et al., 2020) leverages a message passing network, CompGCN (Vashishth et al., 2020), as an encoder to obtain the relation and entity embeddings, which are then fed into a transformer decoder to obtain the validity of facts. HyTransformer (Yu and Yang, 2021), GRAN (Wang et al., 2021) and QUAD (Shomer et al., 2022) further improve it with alternative designs of encoders and via auxiliary training tasks. Relatively, these models, though useful, require a large number of parameters and are prone to overfitting.

heading: 3 Preliminaries
text: We view a hyper-relational fact in the form of a primal triple coupled with a set of qualifiers. Definition 1 (Hyper-relational fact). Let E andR denote the sets of entities and relations, respectively. A hyper-relational fact F is a tuple (T ,Q), where T = (h, r, t), h, t ∈ E , r ∈ R is a primal triple and Q = {(ki : vi)}mi=1 ki ∈ R, vi ∈ E is a set of qualifiers. We call the number of involved entities in F , i.e., (m+ 2), the arity of the fact.
A hyper-relational fact reduces to a triple/binary fact when m = 0. When m > 0, each qualifier can be viewed as an auxiliary description that contextualizes or specializes the semantics of the primal
triple. In typical open-world settings, facts with the same primal triple might have different numbers of qualifiers. To characterize this property, we introduce the concepts of partial fact and qualifier monotonicity in hyper-relational KGs. Definition 2 (Partial fact (Guan et al., 2020)). Given two facts F1 = (T ,Q1) and F2 = (T ,Q2) that share the same primal triple. We call F1 a partial fact of F2 iff Q1 ⊆ Q2.
In this paper, we follow the monotonicity assumption by restricting the model to respect the monotonicity property.2 For this purpose, we consider the monotonicity of query and inference. Definition 3 (Qualifier monotonicity). Let QA(·) denote a query answering model taking a query and a KG as input and outputting the set of answer entities. Given any pair of queries q1 = ((h, r, x?) ,Q1) and q2 = ((h, r, x?) ,Q2) that share the same primal triple and Q1 ⊆ Q2, qualifier monotonicity is given iff,
QA(q2; KG) ⊆ QA(q1; KG). (1) Qualifier monotonicity implies that attaching any qualifiers to a query does not enlarge the answer set of the possible tail entities, and inversely, removing the qualifiers from a query can only return more possible tail entities. This implies that if a fact is true, then all its partial facts must also be true (a.k.a. weakening of inference rule), i.e.,
(T ,Q1) ∧ (Q2 ⊆ Q1)→ (T ,Q2) . (2) 2Some kinds of qualifiers may represent semantically opaque contexts. For instance, ((Crimea, belongs_to, Russia), {(said_by, Putin)}) does not imply the primary triple and should therefore be excluded.

heading: 4 Shrinking Embeddings for Hyper-Relational KGs
text: We aim to design a scoring function f(·) taking the embeddings of facts as input so that the output values respect desired logical properties. To this end, we introduce primal triple embedding and qualifier embedding, respectively.

heading: 4.1 Primal Triple Embedding
text: We represent each entity as a point e ∈ Rd. Each primal relation r is modeled as a spatio-functional transformation Br : Rd → Box(d) that maps the head eh ∈ Rd to a d-dimensional box in Box(d) with Box(d) being the set of boxes in Rd. Each box can be parameterized by a lower left point m ∈ Rd and an upper right point M ∈ Rd, given by
Boxd(m,M) = {x ∈ Rd |mi ≤ xi ≤Mi, i = 1, · · · , d}. (3)
We leave the superscript of Boxd away if it is clear from context. and call the transformed box a query box. Intuitively, all points in the query box correspond to the possible answer tail entities. Hence, the query box can be viewed as a geometric embedding of the answer set. Note that a query could result in an empty answer set. In order to capture such property, we do not exclude empty boxes that correspond to queries with empty answer set. Empty boxes are covered by the cases where there exists a dimension i such that mi ≥Mi. Point-to-box transform The spatio-functional point-to-box transformation B is composed of a relation-specific point transformation Hr : Rd → Rd that transforms the head point eh to a new point, and a relation-specific spanning that spans the transformed point to a box, formally given by
Br(eh) = Box(Hr(eh)−τ(δr),Hr(eh)+τ(δr)), (4) where δr ∈ Rn is a relation-specific spanning/offset vector, and τt(x) = t log ( 1 + ex/t ) with t being a temperature hyperparameter, is a softplus function that enforces the spanned box to be non-empty.
The point transformation functionHr could be any functions that are used in other KG embedding models such as translation used in TransE (Bordes et al., 2013) and rotations used in RotatE (Sun et al., 2019). Hence, our model is highly flexible and effective at embedding primal triples. To allow for
capturing multiple triple-level inference patterns such as symmetry, inversion, and composition, we combine translation and rotation, and formulateHr as Hr(eh) = Θreh + br (5) where Θr is a rotation matrix and br is a translation vector. We parameterize the rotation matrix by a block diagonal matrix Θr = diag ( G (θr,1) , . . . ,G ( θr, d
2
)) , where
G(θ) = [ cos(θ) sin(θ) sin(θ) cos(θ) ] . (6)
Point-to-box distance The validity of a primal triple (h, r, t) is then measured by judging whether the tail entity point et is geometrically inside of the query box. Given a query box Boxn(m,M) and an entity point e ∈ Rd, we denote the center point as c = m+M2 . Let | · | denote the L1 norm and max() denote an element-wise maximum operation. The point-to-box distance is given by
D(e,Box(m,M)) = |e− c|1
|max(0,M−m)|1 + (|e−m|1 + |e−M|1 − |max(0,M−m)|1)2 . (7)
Fig. 2 visualizes the distance function. Intuitively, in cases where the point is in the query box, the distance grows relatively slowly and inversely correlates with the box size. In cases where the point is outside the box, the distance grows fast.

heading: 4.2 Qualifier Embedding
text: Conceptually, qualifiers add information to given primary facts potentially allowing for additional inferences, but never for the retraction of inferences, reflecting the monotonicity of the representational paradigm. Corresponding to the non-declining number of inferences, the number of possible models for this representation shrinks, which can be intuitively reflected by a reduced size of boxes incurred by adding qualifiers.
Box Shrinking To geometrically mimic this property in the embedding space, we model each qualifier (k : v) as a "shrinking" of the query box. Given a box Box(m,M), a shrinking is defined as a box-to-box transformation S : Box→ Box that potentially shrinks the volume of the box while not moving the resulting box outside of the source box. Let L = (M−m) denote the side length vector, box shrinking is defined by
Sr,k,v (Box (m,M)) = Box (m + σ (sr,k,v) L,M− σ (Sr,k,v) L) ,
(8)
where sr,k,v ∈ Rn and Sr,k,v ∈ Rn are the "shrinking" vectors for the lower left corner and the upper right corner, respectively. σ is a sigmoid function and is element-wise vector multiplication. The resulting box, including the case of empty box, is always inside the query box, i.e., S(Boxn(m,M)) ⊆ Boxn(m,M), which exactly resembles the qualifier monotonicity.
We use r, k, v as the indices of the shrinking vectors because the shrinking of the box should depend on the relatedness between the primal relation and the qualifier. For example, if a qualifier (degree : bachelor) is highly related to the primal relation educated_at, the scale of the shrinking vectors should be small as it adds a weak constraint to the triple. If the qualifier is unrelated to the primal relation, e.g., (degree : bachelor) and born_in, the shrinking might even enforce an empty box.
To learn the shrinking vectors, we leverage an MLP layer that takes the primal relation and key-value qualifier as input and outputs the shrinking vectors defined by sr,k,v,Sr,k,v = MLP (concat (rθ, kθ, vθ)) where rθ, kθ, vθ are the embeddings of r, k, v, respectively.

heading: 4.3 Scoring function and learning.
text: Scoring function The score of a given hyperrelational fact is defined by
f (((h, r, t) ,Q)) = D(et,BoxQ(m,M)), (9) where BoxQ(m,M) denotes the target box that is calculated by the intersection of all shrinking boxes of the qualifier set Q. The intersection of n boxes can be calculated by taking the maximum of lower left points of all boxes and taking the minimum of upper right points of all boxes, given by
I(Box1, · · · ,Boxn) = Box (
max i∈1,··· ,n mi, min i∈1,··· ,n Mi
) .
(10)
Note that if there is no intersection between boxes, this intersection operation still works as it results in an empty box. The intersection of boxes is a permutation-invariant operation, implying that perturbing the order of qualifiers does not change the plausibility of the facts.
Learning As a standard data augmentation strategy, we add reciprocal relations ( t′, r−1, h′ ) for the primary triple in each hyper-relational fact. For each positive fact in the training set, we generate nneg negative samples by corrupting a subject/tail entity with randomly selected entities from E . We adopt the cross-entropy loss to optimize the model via the Adam optimizer, which is given by
L = − 1 N
N∑
i=1
( yi log (pi) + nneg∑
i=1
(1− yi) log (1− pi) ) ,
(11)
where N denotes the total number of facts in the training set. yi is a binary indicator denoting whether a fact is true or not. pi = σ(f(F)) is the predicted score of a fact F with σ being the sigmoid function.

heading: 5 Theoretical Analysis
text: Analyzing and modeling inference patterns is of great importance for KG embeddings because it enables generalization capability, i.e., once the patterns are learned, new facts that respect the patterns can be inferred. An inference pattern is a specification of a logical property that may exist in a KG, Formally, an inference pattern is a logical form ψ → φ with ψ and φ being the body and head, implying that if the body is satisfied then the head must also be satisfied.
In this section, we analyze the theoretical capacity of ShrinkE for modeling inference patterns. All proofs of propositions are in Appendix B.
Fact-level inference pattern (monotonicity) The following proposition shows that ShrinkE is able to model monotonicity.
Proposition 1. Given any two facts F1 = (T ,Q1) and F2 = (T ,Q2) where Q2 ⊆ Q1, i.e., F2 is a partial fact of F1, the output of the scoring function f(·) of ShrinkE satisfy the constraint f(F2) ≥ f(F1). Triple-level inference patterns Prominent triple-level inference patterns include symmetry (h, r, t) → (t, r, h), antisymmetry (h, r, t) → ¬(h, r, t), inver-
sion (h, r1, t) → (t, r2, h), composition (e1, r1, e2) ∧ (e2, r2, e3) → (e1, r3, e3), relation implication (h, r1, t) → (h, r2, t), relation intersection (h, r1, t) ∧ (h, r2, t)→ (h, r3, t), and relation mutual exclusion (h, r1, t) ∧ (h, r2, t) → ⊥. All these triple-level inference patterns also exist in hyper-relational facts when their qualifiers are the same, e.g., hyper-relational symmetry means ((h, r, t) ,Q) → ((t, r, h) ,Q). Proposition 2 states that ShrinkE is able to infer all of them.
Proposition 2. ShrinkE is able to infer hyperrelational symmetry, anti-symmetry, inversion, composition, relation implication, relation intersection, and relation exclusion.
Qualifier-level inference pattern In hyperrelational KGs, inference patterns not only exist at the triple level but also at the level of qualifiers.
Definition 4 (qualifier implication). Given two qualifiers qi and qj , qi is said to imply qj , i.e., qi → qj iff for any fact F = (T ,Q), if attaching qi to Q results in a true (resp. false) fact, then attaching qj toQ∪{qi} also results in a true (resp. false) fact. Formally, qi → qj implies
∀ T ,Q : (T ,Q∪ {qi})→ (T,Q ∪ {qi, qj}) . (12)
Definition 5 (qualifier exclusion). Two qualifiers qi, qj are said to be mutually exclusive iff for any factF = (T ,Q), by attaching qi, qj to the qualifier set of F , the new fact F ′ = (T ,Q∪ {qi, qj}) is false, meaning that they lead to a contradiction, i.e., qi ∧ qj → ⊥. Formally, qi ∧ qj → ⊥ implies
∀ T ,Q : (T,Q ∪ {qi, qj})→ ⊥ (13)
Note that if two qualifiers qi, qj are neither mutually exclusive nor forming implication pair, then qi, qj are said to be overlapping, a state between implication and mutual exclusion. Qualifier overlapping, in our case, can be captured by box intersection/overlapping. Qualifier overlapping itself
does not form any logical property in the form of ψ → φ. However, when involving three qualifiers and two of them overlap, qualifier intersection can be modeled.
Definition 6 (qualifier intersection). A qualifier qk is said to be an intersection of two qualifiers qi, qj iff for any fact F = (T ,Q), if attaching qi, qj toQ results in a true (resp. false) fact, then by replacing {qi, qj} with qk, the truth value of the fact does not change. Namely, qi ∧ qj → qk implies
∀ T ,Q : (T,Q ∪ {qi, qj})→ (T ,Q∪ {qk}) . (14)
Apparently, qualifier intersection qi ∧ qj → qk necessarily implies qualifier implications qi → qk and qj → qk. Hence, qualifier intersection can be viewed as a combination of two qualifier implications, and this can be generalized to q1∧q2∧· · · → qk. Proposition 3 shows that ShrinkE is able to infer qualifier implication, exclusion, and composition.
Proposition 3. ShrinkE is able to infer qualifier implication, mutual exclusion, and intersection.

heading: 6 Evaluation
text: In this section, we evaluate the effectiveness of ShrinkE on hyper-relational link prediction tasks.

heading: 6.1 Experimental Setup
text: Datasets. We conduct link prediction experiment on three hyper-relational KGs: JF17K (Wen et al., 2016), WikiPeople (Guan et al., 2019), and WD50k (Galkin et al., 2020). JF17K is extracted from Freebase while WikiPeople and WD50k are extracted from Wikidata. In WikiPeople and WD50k, only 11.6% and 13.6% of the facts, respectively, contain qualifiers, while the remaining facts contain only triples (after dropping statements containing literals in WikiPeople, only 2.6% facts contain qualifiers). For better comparison, we also consider
three splits of WD50K that contain a higher percentage of triples with qualifiers. The three splits are WD50K(33), WD50K(66), and WD50K(100), which contain 33%, 66%, and 100% facts with qualifiers, respectively. Statistics of the datasets are given in Table 1. We conjecture that the performance on WikiPeople and WD50k will be dominated by the scores of triple-only facts while the performance on the variants of WD50k will be dominated by the modeling of qualifiers. We conjecture that WD50K will be a more challenging benchmark than JF17K and WikiPeople. Besides, WD50K still contains only a small percentage (13.6%) of facts that contain qualifiers. Since JF17K does not provide a validation set, we split 20% of facts from the training set as the validation set. Details of the three datasets are given in Table 1.
Environments and hyperparameters We implement ShrinkE with Python 3.9 and Pytorch 1.11, and train our model on one Nvidia A100 GPU with 40GB of VRAM. We use Adam optimizer with a batch size of 128 and an initial learning rate of 0.0001. For negative sampling, we follow the strategy used in StarE (Galkin et al., 2020) by randomly corrupting the head or tail entity in the primal triple.
Different from HINGE (Rosso et al., 2020) and NeuInfer (Guan et al., 2020) that score all potential facts one by one that takes an extremely long time for evaluation, ShrinkE ranks each target answer against all candidates in a single pass and significantly reduces the evaluation time. We search the dimensionality from [50, 100, 200, 300] and the best one is 200. We set the temperature parameter to be t = 1.0. We use the label smoothing strategy and set the smoothing rate to be 0.1. We repeat all experiments for 5 times with different random seeds and report the average values, the error bars are relatively small and are omitted. Code is available at 3.
Baselines We compare ShrinkE against various models, including m-TransH (Wen et al., 2016), RAE (Zhang et al., 2018), NaLP-Fix (Rosso et al., 2020), HINGE (Rosso et al., 2020), NeuInfer (Guan et al., 2020), BoxE (Abboud et al., 2020), Transformer and StarE (Galkin et al., 2020). Note that we exclude Hy-Transformer (Yu and Yang, 2021), GRAN (Wang et al., 2021) and QUAD (Shomer et al., 2022) for comparison because 1) they are heavily based on StarE and Transformer;
3https://github.com/xiongbo010/ShrinkE
and 2) they leverage auxiliary training tasks, which can also be incorporated into our framework and we leave as one future work.
Evaluation We strictly follow the settings of Galkin et al. (2020), where the aim is to predict a missing head/tail entity in a hyper-relational fact. We consider the widely used ranking-based metrics for link prediction: mean reciprocal rank (MRR) and H@K (K=1,10). For ranking calculation, we consider the filtered setting by filtering the facts existing in the training and validation sets (Bordes et al., 2013).

heading: 6.2 Main Results and Analysis
text: Table 2 and Table 3 summarize the performances of all approaches on the six datasets. Overall, ShrinkE achieves either the best or the second-best results against all baselines, showcasing the expressivity and capability of ShrinkE on hyper-relational link prediction. In particular, We observe that ShrinkE outperforms all baselines on JF17K and the three variants of WD50K with a high ratio of facts containing qualifiers while achieving highly competitive results on WikiPeople and the original version of WD50K that contain fewer facts with qualifiers. Interestingly, we find that the performance gains increase when increasing the ratio of facts containing qualifiers. On WD50K (100) where 100% facts contain qualifiers, the performance gain of ShrinkE is most significant across all metrics (6.2%, 6.9%, and 4.7% improvements over MRR, H@1, and H@10, respectively). We believe this is because that ShrinkE is excellent at modeling qualifiers due to its explicit modeling of inference patterns.
Case analysis Table 5 shows some examples of qualifier implication pairs recovered by our learned embeddings. Note that exclusions pairs are ubiquitous (i.e., most of the random qualifiers are mutually exclusive) and hence we do not analyze them. We find that some qualifier implications happen when they are about geographic information and involve geographic inclusion such as Monte Carlo
is in Monaco. Interestingly, we find that qualifiers associated with key owned_by imply (of, voting interest), and qualifiers with key emergency phone number imply (has_use, police) or (has_use, file department), which conceptually make sense.

heading: 6.3 Ablations and Parameter Sensitivity
text: Impact of relational components To determine the importance of each component in relational modeling, we conduct an ablation study by considering three versions of ShrinkE in which one of the components (translation, rotation, and shrinking) is removed. Table 4 shows that the removal of each component of the relational transformation leads to a degradation in performance, validating the importance of each component. In particular, by removing the qualifier shrinking, which is the main contribution of our framework, the performance reduces 3% and 5% in MRR and H@10, respectively, showcasing the usefulness of modeling qualifiers as shrinking. The removals of translation and rotation both result in around 1% and 2% reduction in MRR and H@10, respectively.
Impact of dimensionality We conduct experiments on JF17K under a varied number of dimen-
sions d = [4, 8, 16, 32, 64, 128, 256]. As Fig. 3 depicts, the performance increases when increasing the number of dimensions. However, the growth trend gradually flattens with the increase of dimensions and it achieves comparable performance when the dimension is higher than 128.

heading: 6.4 Discussion
text: Comparison with neural network models Heavy neural network models such as GRAN (Wang et al., 2021) and QUAD (Shomer et al., 2022) are built on relational GNNs and/or Transformers and require a large number of parameters. In contrast, ShrinkE is a neuro-symbolic model that requires only one MLP layer and a much smaller number of parameters. The logical modelling of ShrinkE makes it more explainable than GNNbased and Transformer-based methods.
Comparison with other box embeddings in KGs ShrinkE is the first to not only represent hyper-relational facts, but also explicitly model the logical properties of these facts. SrinkE is different from previous box embedding methods (Abboud et al., 2020) of KGs in three key modules: 1) our point-to-box transform function modelling triple inference patterns; 2) a new point-to-box distance function; and 3) we introduce box shrinking to model qualifier-level inference patterns. Moreover, we provide a comprehensive theoretical analysis of ShrinkE on modelling various logical properties.

heading: 7 Conclusion
text: We present a novel hyper-relational KG embedding model ShrinkE. ShrinkE models a primal triple as a spatio-functional transformation while modeling each qualifier as a shrinking that monotonically narrows down the answer set. We proved that ShrinkE is able to spatially infer core inference patterns at different levels including triple-level, fact-level, and qualifier-level. Experimental results on three benchmarks demonstrate the advantages of ShrinkE in predicting hyper-relational links.
Limitations
Currently, the main goal of ShrinkE is to model inference patterns directly in the embedding space for hyper-relational KGs and we do not explore more advanced training strategies that have recently been proposed. For example, recent works (Yu and Yang, 2021; Wang et al., 2021; Shomer et al., 2022) have demonstrated that adding auxiliary training tasks,
e.g., the task of predicting qualifier entities, can further improve the overall performance. We believe such auxiliary training tasks can also benefit ShrinkE and we leave it as future work. Another limitation of ShrinkE, though rarely happens, is that when dealing with semantically opaque contexts, the monotonicity assumption might not hold. In that case, we need ad-hoc solutions. One simple way is to explicitly distinguish semantically transparent and semantically opaque contexts.
Ethics Statement
The authors declare that they have no conflicts of interest. This article does not contain any studies involving business data and personal information. Our experimentation does not involve any ethical concerns. However, similar to other models, when deploying our link prediction model to real-world applications such as online recommendation systems, the prediction might be biased or unfair to some ethic/gender groups. We advise researchers in the community to look into bias (Bourli and Pitoura, 2020) and fairness (Fu et al., 2020) in KGs.

heading: Acknowledgement
text: The authors thank the International Max Planck Research School for Intelligent Systems (IMPRSIS) for supporting Bo Xiong. Bo Xiong is funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No: 860801. Mojtaba Nayyeri is funded by the German Federal Ministry for Economic Affairs and Climate Action under Grant Agreement Number 01MK20008F (Service-Meister). This research was partially funded by the Ministry of Science, Research, and the Arts (MWK) Baden-Württemberg, Germany, within the Artificial Intelligence Software Academy (AISA) and the German Research Foundation (DFG) via grant agreement number STA 572/18-1 (Open Argument Mining). We acknowledge the support by the Stuttgart Center for Simulation Science (SimTech).

heading: A Supplemental Related Works
text: We survey some supplemental related work on binary relational KG embeddings and geometric relational embeddings.
Binary relational KG embeddings Most of the existing KG embedding methods consider binary relational KGs where each fact is represented in the form of triple (h, r, t). Prominent examples include the additive (or translational) family such as TransE (Bordes et al., 2013) that models each fact as a translation s + r ≈ o, and the multiplicative (or bilinear) family such as RESCAL (Nickel et al., 2011) that models the relation between two entities as a bilinear interaction < h, r, t >. Many other works have been proposed to enhance the translational and bilinear models such as modeling relational mapping properties (e.g., one-to-many and many-to-many) (Wang et al., 2014), modeling inference patterns (e.g., symmetry and composition) (Trouillon et al., 2016; Sun et al., 2019), and modeling complex graph structures (e.g., hierarchies and cycles) (Chami et al., 2020; Xiong et al., 2022d) to name a few.
Geometric relational embeddings Our work is closely related to geometric relational embeddings. See (Xiong et al., 2023) for a systematic survey. Geometric relational embeddings encode real-world relational knowledge by geometric objects such as convex regions like n-balls (Kulmanov et al., 2019), convex cones (Zhang et al., 2021; He et al., 2023), axis-parallel boxes (Vilnis et al., 2018; Xiong et al., 2022c; Ren et al., 2020) and non-Euclidean manifold components (Xiong et al., 2022a). A key advantage of these geometric embeddings is that they nicely model the set-theoretic semantics that can be used to capture logical rules of KGs (Abboud et al., 2020), ontological axioms (Kulmanov et al., 2019; Xiong et al., 2022c), transitive closure (Vilnis et al., 2018), and logical query for multi-hop reasoning (Ren et al., 2020). Different from all previous work, ShrinkE is the first geometric embedding that aims at modeling inference patterns for hyper-relational KGs.

heading: B Proof of propositions
text: Proposition B.1. Given any two facts F1 = (T ,Q1) and F2 = (T ,Q2) where Q2 ⊆ Q1, i.e., F2 is a partial fact of F1, the output of the scoring function f(·) of ShrinkE satisfy the constraint f(F2) ≥ f(F1), which implies Eq.(2).
Proof. We first prove that the resulting box of F2 subsumes the resulting box of F2. Since the primal triple of F1 and F2 are the same (let assume it is T = (h, r, t) ), the spanned boxes of the two facts are Hr(eh). Since Q2 ⊆ Q1, the final shrunken box of F1 must be a subset of the shrunken box of F2. Hence, we have,
BoxF2 ⊆ BoxF1 . (15) Given the tail entity t whose embedding is denoted by et, we consider three cases of its position.
1) If et is inside the small box BoxF2 , then et must also be inside BoxF1 since BoxF2 ⊆ BoxF1 . Note that our point-to-box function is monotonically increasing w.r.t. the increase of distance from the tail point to the center of box. Hence, we will have D(e,BoxF2) ≥ D(e,BoxF1), implying f(F2) ≥ f(F1).
2) If et is outside the small box BoxF2 but inside in the larger BoxF1 , according to the definition of the point-to-box distance function, we immediately have D(e,BoxF2) ≥ D(e,BoxF1), implying f(F2) ≥ f(F1).
3) If et is outside the larger box BoxF1„ then et must also be outside BoxF2 since BoxF2 ⊆ BoxF1 . Note that our point-to-box function is monotonically decreasing w.r.t. the increase of volume of box. Hence, we will have D(e,BoxF2) ≥ D(e,BoxF1), implying f(F2) ≥ f(F1). Proposition B.2. ShrinkE is able to infer hyperrelational symmetry, anti-symmetry, inversion, composition, hierarchy, intersection, and exclusion.
We first prove that ShrinkE is able to infer symmetry, anti-symmetry, inversion, and composition. For the sake of proof, we assume θr ∈ [−π, π). We prove them by proving Lemma B.1-4 one by one. Lemma B.1 (Symmetry). Let r be a symmetric relation such that for each triple (eh, r, et), its symmetric triple (et, r, eh) also holds. This symmetric property of r can be modeled by ShrinkE.
Proof. If r is a symmetric relation, by taking the δr = 0, br = 0, and Θr = diag ( G (θr,1) , . . . ,G ( θr, d
2
)) , where G(θ) is a
2× 2 diagonal matrix, we have eh = fr (et) = Θret, et = fr (eh) = Θreh
⇒ Θ2r = I which holds true when θr,i = 0 or θr,i = −π for i = 1, · · · , d2 .
Lemma B.2 (Anti-symmetry). Let r be an antisymmetric relation such that for each triple (eh, r, et), its symmetric triple (et, r, eh) is not true. This anti-symmetric property of r can be modeled by ShrinkE.
Proof. If r is a anti-symmetric relation, by taking the δr = 0, br = 0, and Θr = diag ( G (θr,1) , . . . ,G ( θr, d
2
)) , where G(θ) is a
2× 2 diagonal matrix, we have
eh 6= fr (et) = Θret, et = fr (eh) = Θreh ⇒ Θ2r 6= I
which holds true when θr,i 6= 0 or θr,i 6= −π for i = 1, · · · , d2 .
Lemma B.3 (Inversion). Let r1 and r2 be inverse relations such that for each triple (eh, r1, et), its inverse triple (et, r2, eh) is also true. This inverse property of r1 and r2 can be modeled by ShrinkE.
Proof. If r1 and r2 are inverse relations, by taking the δr = 0, br = 0, and Θr = diag ( G (θr,1) , . . . ,G ( θr, d
2
)) , where G(θ) is a
2× 2 diagonal matrix, we have
et = fr1 (eh) = Θr1eh, eh = fr2 (et) = Θr2eh
⇒ Θr1Θr2 = I
which holds true when for θr1,ir1 + θr2,i = 0 for i = 1, · · · , d2 .
Lemma B.4 (Composition). Let relation r1 be composed of r2 and r3 such that triple (e1, r1, e3) exists when (e1, r2, e2) and (e2, r3, e3) exist. This composition property can be modeled by ShrinkE.
Proof. If r1 is composed of r2 and r3, by taking the δr = 0, br = 0, and Θr = diag ( G (θr,1) , . . . ,G ( θr, d
2
)) , where G(θ) is a
2× 2 diagonal matrix, we have
e3 = fr1 (e1) = Θr1e1, e2 = fr2 (e1) = Θr2e1, e3 = fr3 (e2) = Θr3e2 ⇒ Θr1 = Θr2Θr3
which holds true when θr1,i = θr2,i + θr3,i or θr1,i = θr2,i+θr3,i+2π or θr1,i = θr2,i+θr3,i−2π for i = 1, · · · , d2 .
We now prove that ShrinkE is able to infer relation implication, exclusion and intersection.
Lemma B.5 (Relation implication). Let r1 → r2 form a hierarchy such that for each triple (eh, r1, et), (eh, r2, et) also holds. This hierarchy property r1 → r2 can be modeled by ShrinkE. Proof. If r1 → r2, by taking Tr1 = Tr2 , i.e., δr1 = δr2 and Θr1 = Θr2 , we have, (eh, r1, et)→ (eh, r2, et) implies that the spanning box of query (eh, r1, x?) is subsumed by the spanning box of query (eh, r2, x?). i.e., Box(Hr1(eh)− σ(δr1),Hr1(eh) + σ(δr1)) ⊆ Box(Hr1(eh) − σ(δr2),Hr1(eh)+σ(δr2)), which holds true when δr1 ≤ δr2 . Lemma B.6 (Relation exclusion). Let r1, r2 be mutually exclusive, that is, (eh, r1, et), (eh, r2, et) can not be simultaneously hold. This mutual exclusion property r1 ∧ r2 → ⊥ can be modeled by ShrinkE. Proof. If r1 ∧ r2 → ⊥, we have (eh, r1, et) ∧ (eh, r2, et) → ⊥, which implies that the spanning box of query (eh, r1, x?) and the spanning box of query (eh, r2, x?) are mutually exclusive, i.e., Box(Hr1(eh)− σ(δr1),Hr1(eh) + σ(δr1)) ∩ Box(Hr1(eh) − σ(δr2),Hr1(eh) + σ(δr2)) → ⊥ Lemma B.7 (Relation intersection). Let r3 be a intersection of r1, r2, that is, if (eh, r1, et) and (eh, r2, et) hold, then (eh, r3, et) also holds. This intersection property r1 ∧ r2 → r3 can be modeled by ShrinkE.
Proof. Note that box is closed under intersection and this property can be view as a combination of two pairs of relation implication. Hence, the proof is similar to the proof of Lemma B.
Proposition B.3. ShrinkE is able to infer qualifier implication, mutual exclusion, and intersection.
Proof. Since each qualifier is associated with a box, the implication and mutual exclusion relationships between qualifiers can be modeled by their geometric relationships, i.e., box entailment and box disjointedness, respectively, between their corresponding boxes. Qualifier intersection can be modeled by enforcing the box of one qualifier to be inside the intersection of the boxes of another two qualifiers.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitation section
3 A2. Did you discuss any potential risks of your work? In "Ethics Statement"
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? Section 6
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix C.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Section 6 and Appendix C.
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
7 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We believe such auxiliary training tasks can also benefit ShrinkE and we leave it as future work. Another limitation of ShrinkE, though rarely happens, is that when dealing with semantically opaque contexts, the monotonicity assumption might not hold. In that case, we need ad-hoc solutions. One simple way is to explicitly distinguish semantically transparent and semantically opaque contexts