Knowledge Hypergraphs: Prediction Beyond Binary Relations

1 INTRODUCTION . Knowledge Hypergraphs are graph structured knowledge bases that store facts about the world in the form of relations between two or more entities . They can be seen as one generalization of Knowledge Graphs , in which relations are defined on exactly two entities . Since accessing and storing all the facts in the world is difficult , knowledge bases are incomplete ; the goal of link prediction ( or knowledge completion ) in knowledge ( hyper ) graphs is to predict unknown links or relationships between entities based on the existing ones . In this work we are interested in the problem of link prediction in knowledge hypergraphs . Our motivation for studying link prediction in these more sophisticated knowledge structures is based on the fact that most knowledge in the world has inherently complex composition , and that not all data can be represented as a relation between two entities without either losing a portion of the information or creating incorrect data points . Link prediction in knowledge graphs is a problem that is studied extensively , and has applications in several tasks such as searching ( Singhal , 2012 ) and automatic question answering ( Ferrucci et al. , 2010 ) . In these studies , knowledge graphs are defined as directed graphs having nodes as entities and labeled edges as relations ; edges are directed from the head entity to the tail entity . The common data structure for representing knowledge graphs is a set of triples relation ( head , tail ) that represent information as a collection of binary relations . There exist a large number of knowledge graphs that are publicly available , such as NELL ( Carlson et al. , 2010 ) and FREEBASE ( Bollacker et al. , 2008 ) . It is noteworthy to mention that FREEBASE is a complex knowledge base where 61 % of the relations are beyond binary ( defined on more than two nodes ) . However , current methods use a simplified version of FREEBASE where the non-binary relations are converted to binary ones ( defined on exactly two entities ) . Embedding-based models ( Nguyen , 2017 ) have proved to be effective for knowledge graph completion . These approaches learn embeddings for entities and relations . To find out if r ( h , t ) is a fact ( i.e . is true ) , such models define a function that embeds relation r and entities h and t , and produces the probability that r ( h , t ) is a fact . While successful , such embedding-based methods make the strong assumption that all relations are binary . In this work , we introduce two embedding-based models that perform link prediction in knowledge hypergraphs . The first is HSimplE , which is inspired from SimplE ( Kazemi & Poole , 2018 ) , origi- Michelle Obama Undergrad PhD Princeton King ’ s College Cambridge Alan Turing ( a ) DEGREE FROM UNIVERSITY defined on three facts . ( b ) Reifying non-binary relations with three additional entities . ( c ) Converting non-binary relations into cliques . Figure 1 : In this example , the three facts in the original graph ( a ) show that Turing received his PhD from Princeton and his undergraduate degree from King ’ s College Cambridge . Figures ( b ) and ( c ) show two methods of converting this ternary relation into three binary ones . nally designed to perform link prediction in knowledge graphs . For a given entity , HSimplE shifts the entity embedding by a value that depends on the position of the entity in the given relation . Our second model is HypE , which in addition to learning entity embeddings , learns positional ( convolutional ) filters ; these filters are disentangled from entity representations and are used to transform the representation of an entity based on its position in a relation . We show that both HSimplE and HypE are fully expressive . To evaluate our models , we introduce two new datasets from subsets of FREEBASE , and develop baselines by extending existing models on knowledge graphs to work with hypergraphs . We evaluate the proposed methods on standard binary and non-binary datasets . While both HSimplE and HypE outperform our baselines and the state-of-the-art , HypE is more effective with fewer parameters . It also produces much better results when predicting relations that contain at least one entity in a position never encountered during training , demonstrating the clear advantage of disentangling position representation from entity embeddings . The contributions of this paper are : ( 1 ) HypE and HSimplE , two embedding-based methods for knowledge hypergraph completion that outperform the baselines for knowledge hypergraphs , ( 2 ) a set of baselines for knowledge hypergraph completion , and ( 3 ) two new knowledge hypergraphs obtained from subsets of FREEBASE , which can serve as new evaluation benchmarks for knowledge hypergraph completion methods . 2 MOTIVATION AND RELATED WORK . Knowledge hypergraph completion is a relatively under-explored area . We motivate our work by outlining that adjusting current models to accommodate hypergraphs does not yield satisfactory results . Existing knowledge graph completion methods can be used in the beyond-binary setting by either ( 1 ) extending known models to work with non-binary relational data ( e.g. , m-TransH ( Wen et al. , 2016 ) ) , or by ( 2 ) converting non-binary relations into binary ones using methods such as reification or star-to-clique ( Wen et al. , 2016 ) , and then applying known link prediction methods . In the first case , the only example that extends a known model to work with non-binary relations is m-TransH ( Wen et al. , 2016 ) , which is an extension of TransH ( Wang et al. , 2014 ) , and which we show to be less effective than our models in Section 6 . The second case is about restructuring a knowledge hypergraph to work with current knowledge graph completion methods . One common approach to reduce a hypergraph into a graph is reification . In order to reify a fact with a relation defined on k entities , we first create a new entity e ( square nodes in Figure 1b ) and connect e to each of the k entities that are part of the given fact . Another approch is Star-to-clique , which converts a fact defined on k entities into ( k 2 ) facts with distinct relations between all pairwise entities in the fact . See Figure 1c . Both conversion approaches have their caveats when current link-prediction models are applied to the resulting graphs . The example in Figure 1a shows three facts that pertain to the relation DEGREE_FROM_UNIVERSITY . When we reify the hypergraph in this example ( Figure 1b ) , we add three reified entities . At test time , we again need to reify the test samples , which means we need a way to embed newly created entities about which we have almost no information . Applying the star-to-clique method to the hypergraph does not yield better results either : in this case , the resulting graph loses some of the information that the original hypergraph had — in Figure 1c , it is no longer clear which degree was granted by which institution . Existing methods that relate to our work in this paper can be grouped into three main categories : Knowledge graph completion . Embedding-based models such as translational ( Bordes et al. , 2013 ; Wang et al. , 2014 ) , bilinear ( Yang et al. , 2015 ; Trouillon et al. , 2016 ; Kazemi & Poole , 2018 ) , and deep models ( Nickel et al. , 2011 ; Socher et al. , 2013 ) have proved effective for knowledge graphs where all relations are binary . We extend some of the models in this category and compare them with the proposed methods . Knowledge hypergraph completion . Soft-rule models ( Richardson & Domingos , 2006 ; De Raedt et al. , 2007 ; Kazemi et al. , 2014 ) can easily handle variable arity relations and have the advantage of being interpretable . However , they have a limited learning capacity and can only learn a subset of patterns ( Nickel et al. , 2016 ) . Embedding-based methods are more powerful than soft-rule approaches . Guan et al . ( 2019 ) proposed an embedding-based method based on the star-to-clique approach which its caveats are discussed . m-TransH ( Wen et al. , 2016 ) extends TransH ( Wang et al. , 2014 ) to knowledge hypergraph completion . Kazemi & Poole ( 2018 ) prove that TransH and consequently m-TransH are not fully expressive and have restrictions in modeling relations . In contrast , our embedding-based proposed models are fully expressive and outperform m-TransH . Learning on hypergraphs . Hypergraph learning has been employed to model high-order correlations among data in many tasks , such as in video object segmentation ( Huang et al. , 2009 ) and in modeling image relationships and image ranking ( Huang et al. , 2010 ) . There is also a line of work extending graph neural networks to hypergraph neural networks ( Feng et al. , 2019 ) and hypergraph convolution networks ( Yadati et al. , 2018 ) . On the other hand , these models are designed for undirected hypergraphs , with edges that are not labeled ( no relations ) , while knowledge hypergraphs are directed and labeled . As there is no clear or easy way of extending these models to our knowledge hypergraph setting , we do not consider them as baselines for our experiments . 3 DEFINITION AND NOTATION . A world consists of a finite set of entities E , a finite set of relations R , and a set of tuples τ defined over E and R. Each tuple in τ is of the form r ( v1 , v2 , . . . , vk ) where r ∈ R is a relation and each vi ∈ E is an entity , for all i = 1 , 2 , . . . , k. Here arity |r| of a relation r is the number of arguments that the relation takes and is fixed for each relation . A world specifies what is true : all the tuples in τ are true , and the tuples that are not in τ are false . A knowledge hypergraph consists of a subset of the tuples τ ′ ⊆ τ . Link prediction in knowledge hypergraphs is the problem of predicting the missing tuples in τ ′ , that is , finding the tuples τ \ τ ′ . An embedding is a function that converts an entity or a relation into a vector ( or sometimes a higher order tensor ) over a field ( typically the real numbers ) . We use bold lower-case for vectors , that is , e ∈ Rk is an embedding of entity e , and r ∈ Rl is an embedding of a relation r. Let v1 , v2 , . . . , vk be a set of vectors . The variadic function concat ( v1 , . . . , vk ) outputs the concatenation of its input vectors . The 1D convolution operator ∗ takes as input a vector v and a convolution weight filter ω , and outputs the convolution of v with the filters ω . We define the variadic function ( ) to be the sum of the element-wise product of its input vectors , namely ( v1 , v2 , . . . , vk ) = ∑ ` i=1 v1 ( i ) v2 ( i ) . . .vk ( i ) where each vector vi has the same length , and vj ( i ) is the i-th element of vector vj . For the task of knowledge graph completion , an embedding-based model defines a function φ that takes a tuple x as input , and generates a prediction , e.g. , a probability ( or score ) of the tuple being true . A model is fully expressive if given any complete world ( full assignment of truth values to all tuples ) , there exists an assignment of values to the embeddings of the entities and relations that accurately separates the tuples that are true in the world from those that are false .

Summary: This paper proposes two new embedding strategies for the task of knowledge graph completion, with special attention on generalizations that support hypergraphs.  The first method, HSimplE, learns an embedding for entities that directly contains multiple positional representations; these are shifted depending on the relation they're used in.  The second method, HypE, disentangles entity embeddings and positional convolutional filters, allowing stronger positional generalization.  Experiments on standard benchmarks demonstrate that the approaches work well.