Shrinking Embeddings for Hyper-Relational Knowledge Graphs (2023)

Abstract: Link prediction on knowledge graphs (KGs) has been extensively studied on binary relational KGs, wherein each fact is represented by a triple. A significant amount of important knowledge, however, is represented by hyper-relational facts where each fact is composed of a primal triple and a set of qualifiers comprising a key-value pair that allows for expressing more complicated semantics. Although some recent works have proposed to embed hyper-relational KGs, these methods fail to capture essential inference patterns of hyper-relational facts such as qualifier monotonicity, qualifier implication, and qualifier mutual exclusion, limiting their generalization capability. To unlock this, we present ShrinkE, a geometric hyper-relational KG embedding method aiming to explicitly model these patterns. ShrinkE models the primal triple as a spatial-functional transformation from the head into a relation-specific box. Each qualifier “shrinks” the box to narrow down the possible answer set and, thus, realizes qualifier monotonicity. The spatial relationships between the qualifier boxes allow for modeling core inference patterns of qualifiers such as implication and mutual exclusion. Experimental results demonstrate ShrinkE’s superiority on three benchmarks of hyper-relational KGs.

## introduction link prediction on knowledge graphs (kgs) is a central problem for many kg-based applications @xcite @xcite @xcite . existing works @xcite have mostly studied link prediction on binary relational kgs, wherein each fact is represented by a triple, e.g., (einstein, educated_at, university of zurich). in many popular kgs such as freebase @xcite , however, a lot of important knowledge is not only expressed in tripleshaped facts, but also via facts about facts, which taken together are called hyper-relational facts. for example, ((einstein, educated_at, university of zurich), {(major:physics), (degree:phd)}) is a hyper-relational fact, where the primary triple (einstein, educated_at, university of zurich) is contextualized by a set of key-value pairs {(major:physics),(degree:phd)}. like much other related work, we follow the terminology established for wikidata @xcite and use the term qualifiers to refer to the key-value pairs. 1 the qualifiers play crucial roles in avoiding ambiguity issues. for instance, einstein was edu-cated_at several universities and the qualifiers for degree and major help distinguish them. in order to predict links in hyper-relational kgs, pioneering works represent each hyperrelational fact as either an n-tuple in the form of r(e 1 , e 2 , • • • , e n ) @xcite @xcite @xcite or a set of key-value pairs in the form of {(k i : v i )} m @xmath0. however, these modelings lose key structure information and are incompatible with the rdf-star schema @xcite used by modern kgs, where both primal triples and qualifiers constitute the fundamental data structure. recent works @xcite represent each hyper-relational fact as a primary triple coupled with a set of qualifiers that are compatible with rdf-star standards @xcite . link prediction is then achieved by modeling the validity of the primary triple and its compatibility with each annotated qualifier @xcite . more complicated graph encoders and decoders @xcite @xcite are proposed to further boost the performance. however, they require a relatively huge number of parameters that make them prone to overfitting. to encourage generalization capability, kg embeddings should be able to model inference patterns, i.e., specifications of logical properties that may exist in kgs, which, if learned, empowers further principled inferences @xcite . this has been extensively studied for binary relational kg embeddings @xcite but ignored for hyper-relational kgs in which not only primal triples but also qualifiers matter. one of the most important properties is qualifier monotonicity. given a query, the answer set shrinks or at least does not expand as more qualifiers are added to the query expression. for example, a query (einstein, educated_at, ?x) with a variable ?x corresponds to two answers {university of zurich, eth zurich}, but a query ((einstein, educated_at, ?x) , {(degree : b.sc.)}) extended by a qualifier for degree will only respond with {eth zurich}. besides, different qualifiers might form logical relationships that the model must respect during inference including qualifier implication (e.g., adding a qualifier that is implicitly implied in the existing qualifiers does not change the truth of a fact) and qualifier mutual exclusion (e.g., adding any two mutually exclusive qualifiers to a fact leads to a contradiction). in light of this, we propose shrinke, a hyperrelational embedding model that allows for modeling these inference patterns. shrinke embeds each entity as a point and models a primal triple as a spatio-functional transformation from the head entity to a relation-specific box that entails the possible tails. each qualifier is modeled as a shrinking of the primal box to a qualifier box. the shrinking of boxes simulates the "monotonicity" of hyperrelational qualifiers, i.e., attaching qualifiers to a primal triple may only narrow down but never enlarges the answer set. the plausibility of a given fact is measured by a point-to-box function that judges whether the tail entity is inside the intersection of all qualifier boxes. moreover, since each qualifier is associated with a box, the spatial relationships between the qualifier boxes allow for modeling core inference patterns such as qualifier implication and mutual exclusion. we theoretically show the capability of shrinke on modeling various inference patterns including (fact-level) monotonicity, triple-level, and qualifier-level inference patterns. empirically, shrinke achieves competitive performance on three benchmarks. ## preliminaries we view a hyper-relational fact in the form of a primal triple coupled with a set of qualifiers. definition 1 (hyper-relational fact). let e and r denote the sets of entities and relations, respectively. a hyper-relational fact f is a tuple (t , q), where @xmath1, r, t), h, @xmath2, @xmath3 we call the number of involved entities in f, i.e., (m + 2), the arity of the fact. a hyper-relational fact reduces to a triple/binary fact when @xmath4. when @xmath5, each qualifier can be viewed as an auxiliary description that contextualizes or specializes the semantics of the primal triple. in typical open-world settings, facts with the same primal triple might have different numbers of qualifiers. to characterize this property, we introduce the concepts of partial fact and qualifier monotonicity in hyper-relational kgs. definition 2 (partial fact @xcite ). given two facts f 1 = (t , q 1 ) and f 2 = (t , q 2 ) that share the same primal triple. we call f 1 a partial fact of in this paper, we follow the monotonicity assumption by restricting the model to respect the monotonicity property. 2 for this purpose, we consider the monotonicity of query and inference. definition 3 (qualifier monotonicity). let qa(•) denote a query answering model taking a query and a kg as input and outputting the set of answer entities. given any pair of queries q 1 = ((h, r, x?) , q 1 ) and q 2 = ((h, r, x?) , q 2 ) that share the same primal triple and q 1 ⊆ q 2 , qualifier monotonicity is given iff, qualifier monotonicity implies that attaching any qualifiers to a query does not enlarge the answer set of the possible tail entities, and inversely, removing the qualifiers from a query can only return more possible tail entities. this implies that if a fact is true, then all its partial facts must also be true (a.k.a. weakening of inference rule), i.e., we aim to design a scoring function f (•) taking the embeddings of facts as input so that the output values respect desired logical properties. to this end, we introduce primal triple embedding and qualifier embedding, respectively. ## theoretical analysis analyzing and modeling inference patterns is of great importance for kg embeddings because it enables generalization capability, i.e., once the patterns are learned, new facts that respect the patterns can be inferred. an inference pattern is a specification of a logical property that may exist in a kg, formally, an inference pattern is a logical form ψ → φ with ψ and φ being the body and head, implying that if the body is satisfied then the head must also be satisfied. in this section, we analyze the theoretical capacity of shrinke for modeling inference patterns. all proofs of propositions are in appendix b. ## evaluation in this section, we evaluate the effectiveness of shrinke on hyper-relational link prediction tasks. ## conclusion we present a novel hyper-relational kg embedding model shrinke. shrinke models a primal triple as a spatio-functional transformation while modeling each qualifier as a shrinking that monotonically narrows down the answer set. we proved that shrinke is able to spatially infer core inference patterns at different levels including triple-level, fact-level, and qualifier-level. experimental results on three benchmarks demonstrate the advantages of shrinke in predicting hyper-relational links.