Regex Queries over Incomplete Knowledge Bases

1 . Introduction . Knowledge Base Completion ( KBC ) predicts unseen facts by reasoning over known facts in an incomplete KB . Embedding-based models are a popular approach for the task – they embed entities and relations in a vector space , and use a scoring function to evaluate validity of any potential fact . KBC models are typically evaluated using single-hop queries , example , “ Who founded Microsoft ? ” , which is equivalent to the query ( Microsoft→ founded by → ? ) . A natural extension [ Guu et al. , 2015 ] poses multi-hop queries , such as ( Microsoft → founded by → · → lives in→ ? ) , which represents “ Where do founders of Microsoft live ? ” . Recent works have also studied subsets of first-order logic queries with conjunction ( ∧ ) , disjunction ( ∨ ) and existential quantifiers ( ∃ ) [ Hamilton et al. , 2018 , Ren et al. , 2020 ] . We analyzed real-world query logs1 over the publicly available Wikidata KB2 and extract its distribution of queries . We found ( see Table 1 ) that while the single hop queries are indeed most frequent ( 87 % ) , multi-hop queries comprise only 1 % of all queries . A substantial fraction of queries ( 12 % ) are queries using regular expression operators ( regex ) , which use disjunction ( ∨ ) or Kleene plus ( + ) operators . In response , we pose the novel problem of answering regex queries using embedding-based KBC models . ∗Work done as a research intern at IIT Delhi 1https : //iccl.inf.tu-dresden.de/web/Wikidata_SPARQL_Logs/en 2https : //www.wikidata.org/wiki/Wikidata : Main_Page Regex queries have a potentially large set of correct answers , due to which typical embedding models that model both queries and entities as points are not very effective . We build on one such model RotatE , which models a relation as a rotation of entity vectors – it can model all relation patterns gen- erally observed in a KB except hierarchy and has not been extended to queries beyond single-hop . There also exists Query2Box [ Hamilton et al. , 2018 ] , a box embedding method which models a query as a high-dimensional axis-aligned box . This can better handle larger answer spaces , but it can not model symmetric relations ( e.g . friend of ) . To answer the full repertoire of regex queries over incomplete KBs , we propose a new baseline , RotatEBox – a RotatE model augmented with box embeddings . In our model , a regex query is an axis-aligned box . Application of a relation or regex expression rotates the center ( vector ) of the box , and enlarges/shrinks the box . Application of a regex operator is handled using two approaches : ( 1 ) learned operators over boxes , or ( 2 ) introducing separate embeddings for Kleene plus of a relation , and expanding a disjunction into multiple non-disjunctive queries . RotatE-Box can capture all typical relation patterns – symmetry , antisymmetry , inversion , composition , intersection , mutual exclusion and hierarchy , along with effectively handling a large number of correct answers . We also contribute two new datasets for our task . Our first dataset , called FB15K-Regex , is an extension of the popular FB15K dataset [ Bordes et al. , 2013 ] , and is constructed using random walks on the FB15K graph . Our second dataset is built over a dense subset of Wikidata ( which we name Wiki100 ) and is based on the real queries from existing query logs . Table 2 lists example queries in the datasets . We find that models based on RotatEBox outperform all other baselines , including strong models based on just RotatE and just Query2Box on these two datasets . In summary , our work presents a novel regex query answering task , two new datasets and a strong baseline for our task . We will release all resources for further research3 . 2 . Related Work . A KB G is a set of triples ( e1 , r , e2 ) , where e1 , e2 ∈ E ( the set of entities ) and r ∈ R ( the set of relations ) . For a KBC task , all triples are split into train/dev/test , resulting in graphs Gtrain , Gdev , and Gtest . The standard KBC task requires answering single-hop queries of the form ( e1 , r , ? ) with the answer e ∗ 2 if ( e1 , r , e ∗ 2 ) ∈ Gtest . While many kinds of solutions exist for this problem , we build on the vast literature in embedding-based approaches [ Bordes 3https : //github.com/dair-iitd/kbi-regex et al. , 2013 , Socher et al. , 2013 , Yang et al. , 2014 , Trouillon et al. , 2016 , Jain et al. , 2018 , Sun et al. , 2019 , Jain et al. , 2020 ] . These methods embed each entity and relation in a latent embedding space , and use various tensor factorization scoring functions to estimate the validity of a triple . Our work builds on RotatE , which defines relation as a rotation in a complex vector space . The distance of target entity from the query constitutes the scoring function . RotatE has shown high KBC performance and can model many relation patterns ( see Section 2.1 ) . The first work on answering complex queries over incomplete KBs extends single-hop queries to path queries ( or multi–hop queries ) of the type ( e1 , r1 , r2 , . . . , rn , ? ) . Guu et al . [ 2015 ] construct datasets for the task using random walks on FB13 and WN11 KBs . Models for the task include compositional training of existing KBC scoring functions and models based on variants of RNNs [ Yin et al. , 2018 , Das et al. , 2017 ] . Recent work has started exploring subsets of first-order logic queries . Hamilton et al . [ 2018 ] explore conjunctive queries with conjunction ( ∧ ) and existential quantification ( ∃ ) . They propose DeepSets [ Zaheer et al. , 2017 ] to model conjunction as a learned geometric operation . MPQE [ Daza and Cochez , 2020 ] and BIQE [ Kotnis et al. , 2021 ] extend this work by constructing a query representation using a graph neural network and transformer encoder respectively . Probably closest to our work is that of answering Existential Positive First-order ( EPFO ) logical queries [ Ren et al. , 2020 ] – queries that include ∧ , ∨ , and ∃ . They propose Query2Box , which represents a query as a box ( center , offset ) and each entity as point in a highdimensional space . The points closer to the center are scored higher for the query , especially if they are inside the box . The strength of Query2Box is that it can naturally handle large answer spaces , and also model many relation patterns ( see Section 2.1 ) . [ Ren and Leskovec , 2020 ] extend EPFO to complete set of first-order logical operations , by handling negation ( ¬ ) with their proposed model , BetaE . They propose high-dimensional Beta distributions as embeddings of queries and entities . The KL divergence between query and entity serves as a scoring function . There are also other works that work with probabilistic databases for answering EPFO queries [ Friedman and Van den Broeck , 2020 ] , but , to the best of our knowledge , there has been no work on answering regex queries over incomplete knowledge bases . In particular , regular expressions that contain the Kleene plus operator , are not included in path queries or first-order logic queries . 2.1 Relation Patterns in a KB . Embedding models can be critiqued based on their representation power [ Toutanova and Chen , 2015 , Guu et al. , 2015 , Trouillon et al. , 2016 , Sun et al. , 2019 ] in modeling commonly occuring relation patterns ( relationships amongst relations ) . We summarize the common patterns and model capabilities in Table 3 . Broadly , translation-based methods ( that add and subtract embedding vectors ) like TransE and Query2Box can model all patterns but can not model symmetry ( ( e1 , r1 , e2 ) ⇒ ( e2 , r1 , e1 ) ) , such as for the relation friend of . Rotation-based models such as RotatE can model all inference patterns except for hierarchy : ( e1 , r1 , e2 ) ⇒ ( e1 , r2 , e2 ) . Other models have other weaknesses , like similaritybased methods ( that use dot products between embeddings ) such as DistMult and ComplEx are incapable of handling composition and intersection . Our proposed model combines the power of Query2Box and RotatE and can express all seven common relation patterns . 3 . Task . We now formally define the task of answering regular expressions ( regex ) queries on a KB . Regular expressions are primarily used to express navigational paths on a KB in a more succinct way and to extend matching to arbitrary length paths . Any regex c in a KB is an expression over R ∪ { + , / , ∨ } , generated from the grammar : c : := r | c1 ∨ c2 | c1/c2 | c+ , where r ∈ R represents a relation in the KB and c1 and c2 represent regular expressions . / denotes followed by , ∨ denotes disjunction , and + is a Kleene Plus operator denoting one or more occurrences . We use l ( c ) to denote the set of paths ( sequence of relations ) compatible with regex c. As an example , for c = r1/r + 2 , l ( c ) will have paths like r1/r2 , r1/r2/r2 , r1/r2/r2/r2 and so on . Their path lengths will be 2 , 3 and 4 respectively . A regex query q over a KB is defined as a pair of head entity e1 and a regular expression c. For query q = ( e1 , c , ? ) , we define the answer set JqK as the union of entities reachable from e1 in G , by following the paths in l ( c ) : JqK = ⋃ p∈l ( c ) J ( e1 , p , ? ) K . The goal of the task is to , given a query q , rank entities in JqK higher than other entities . 4 . Model . We now describe RotatE-Box – a model that combines the strengths of RotatE and Query2Box . Following Query2Box , the relations and queries are modeled as boxes , and entities as points in a high-dimensional space . However , unlike Query2Box , all points ( and boxes ) are in a complex vector space Ck , instead of a real space . We first describe the model for single-hop queries . RotatE-Box embeds each entity e as a point e ∈ Ck and each relation r as a box r , represented as ( Cen ( r ) , Off ( r ) ) ∈ C2k . Here Cen ( r ) ∈ Ck is the center of the box such that its modulus in each of the k dimensions is one , i.e. , ∀j ∈ [ 1 , k ] , |Cen ( rj ) | = 1 , Cen ( rj ) ∈ C. By doing so , Cen ( rj ) can be re-written as eiθr , j using Euler ’ s formula ( here , e is Euler ’ s number ) . This corresponds to rotation by angle θr , j in the jth dimension of complex vector space . In vectorized form , Cen ( r ) = eiθr . Off ( r ) is a positive offset , i.e. , Re ( Off ( r ) ) ∈ Rk≥0 and Im ( Off ( r ) ) ∈ Rk≥0 . Any box that satisfies these constraints on its center and offset is termed as a rotation box . A point v ∈ Ck is considered inside a box if it satisfies : Re ( rmin ) ≤ Re ( v ) ≤ Re ( rmax ) and Im ( rmin ) ≤ Im ( v ) ≤ Im ( rmax ) ( 1 ) Here , rmax = Cen ( r ) + Off ( r ) ∈ Ck , and rmin = Cen ( r ) − Off ( r ) ∈ Ck . The inequalities represent element-wise inequality of vectors . A single-hop query q = ( e1 , r , ? ) has a rotation box representation q given by ( e1 Cen ( r ) , Off ( r ) ) . Here is the Hadamard product ( element-wise multiplication ) . Since Cen ( r ) = eiθr , e1 is rotated by an angle θr ∈ Rk . Specifically ej is rotated by angle θr , j in the j th dimension . Given a query q associated with a box q , RotatE-Box ranks entities e in the order of the distance of e from q , defined as : dist ( e ; q ) = distout ( e ; q ) + α · distin ( e ; q ) ( 2 ) where α ∈ ( 0 , 1 ) is a constant to downweigh the distance inside the query box , and distout ( e ; q ) = ‖Re ( Max ( e− qmax,0 ) + Max ( qmin − e,0 ) ) ‖1 + ‖Im ( Max ( e− qmax,0 ) + Max ( qmin − e,0 ) ) ‖1 distin ( e ; q ) = ‖Re ( Cen ( q ) −Min ( qmax , Max ( qmin , e ) ) ) ‖1 + ‖Im ( Cen ( q ) −Min ( qmax , Max ( qmin , e ) ) ) ‖1 Here qmax and qmin are defined analogously to rmax and rmin above . This completes the description of RotatE-Box for single-hop queries . As RotatE is a special case of RotatE-Box ( by modeling Off ( r ) = 0 , ∀r ∈ R ) , RotatE-Box can model all the relation patterns modeled by RotatE : symmetry , anti-symmetry , inversion , composition , intersection and mutual exclusion . Additionally , boxes enable modeling hierarchy . Theorem 1 . RotatE-Box can model hierarchical relation pattern . ( Proof in Appendix A.1 )

Summary: This paper combined two knowledge base completion models, RotatE and the Query2Box model, to form the proposed model RotatE-Box model. The proposed model is evaluated on a new task, knowledge base completion on regex queries. The authors also provided two new evaluation datasets for the regex queries. The proposed model achieved better performance on the regex query task compared to the two base models. But they did not compare it with enough baselines such as Beta Embeddings, and so on. And the proposed model performed much worse than the original model in the original knowledge base completion task.