A Simple Approach to Case-Based Reasoning in Knowledge Bases

that requires no training , and is reminiscent of case-based reasoning in classical artificial intelligence ( AI ) . Consider the task of finding a target entity given a source entity and a binary relation . Our non-parametric approach derives crisp logical rules for each query by finding multiple graph path patterns that connect similar source entities through the given relation . Using our method , we obtain new state-of-the-art accuracy , outperforming all previous models , on NELL-995 and FB-122 . We also demonstrate that our model is robust in low data settings , outperforming recently proposed meta-learning approaches1 . 1 . Introduction . Given a new problem , humans possess the innate ability to ‘ retrieve ’ and ‘ adapt ’ solutions to similar problems from the past . For example , an automobile mechanic might fix a car engine by recalling previous experiences where cars exhibited similar symptoms of damage . This model of reasoning has been widely studied and verified in cognitive psychology for various applications such as mathematical problem solving [ Ross , 1984 ] , diagnosis by physicians [ Schmidt et al. , 1990 ] , automobile mechanics [ Lancaster and Kolodner , 1987 ] etc . A lot of classical work in artificial intelligence ( AI ) , particularly in the field of case-based reasoning ( CBR ) has focused on incorporating such kind of reasoning in AI systems [ Schank , 1982 , Kolodner , 1983 , Rissland , 1983 , Aamodt and Plaza , 1994 , Leake , 1996 , inter-alia ] . At a high level , a case-based reasoning system is comprised of four steps [ Aamodt and Plaza , 1994 ] — ( a ) ‘ retrieve ’ , in which given a new problem , ‘ cases ’ that are similar to the given problem are retrieved . A ‘ case ’ is usually associated with a problem description ( used for matching it to a new problem ) and its corresponding solution . After the initial retrieval step , the previous solutions are ( b ) ‘ reused ’ for the problem in hand . Often times , however , the retrieved solutions can not be directly used , and hence the solutions needs to be ( c ) ‘ revised ’ . Lastly , if the revised solution is useful for solving the given problem , they are ( d ) ‘ retained ’ in a memory so that they can be used in the future . Knowledge graphs ( KGs ) [ Suchanek et al. , 2007 , Bollacker et al. , 2008 , Carlson et al. , 2010 ] contain rich facts about entities and capture relations with diverse semantics between those entities . However , KGs can be highly incomplete [ Min et al. , 2013 ] , missing important edges ( relations ) 1Code available at https : //github.com/rajarshd/CBR-AKBC between entities . For example , consider the small fragment of the KG around the entity MELINDA GATES in figure 1 . Even though a lot of facts about MELINDA is captured , it is missing the edge corresponding to works in city . A recent series of work address this problem by modeling multi-hop paths between entities in the KG , thereby able to reason along the path MELINDA→ ceo→ GATES FOUNDATION→ headquartered→ SEATTLE . However , the number of paths starting from an entity increases exponentially w.r.t the path length and therefore past work used parametric models to do approximate search using reinforcement learning ( RL ) [ Xiong et al. , 2017 , Das et al. , 2018 , Lin et al. , 2018 ] . Using RL-based methods have their own shortcomings , like hard to train and high computational requirements . Moreover , these models try to encode all the rules for reasoning into the parameters of the model which makes learning even harder . In this paper , we propose a simple non-parametric approach for reasoning in KGs ( Figure 1 ) . Given an entity and a query relation ( eq , rq ) , we first retrieve k entities in the KG that are similar to eq and for which we observe the query relation edge rq . The retrieved entities could be present anywhere in the KG and are not just restricted in the immediate neighborhood of eq . Similarity between entities is measured based on the observed relations that the entities participate in . This ensures that the retrieved entities have similar observed properties as eq ( e.g. , if eq is a CEO , then the retrieved entities are also CEO ’ s or business person ) . Next , for each of the retrieved entities , our method finds a set of reasoning paths that connect the retrieved entities to the entities that they are connected with via the query relation rq . In this way , our method removes the burden of storing the reasoning rules in the parameters of the model and rather extract it from entities similar to the query entity . Next , our method checks if similar reasoning paths exists starting from the query entity eq . If similar paths exists , then the answer to the original query is found by starting from eq and traversing the KG by following the reasoning path . In practice , we find multiple reasoning paths which end at different entities and we rank the entities based on the number of reasoning paths that lead to them , with the intuition that an entity supported by multiple reasoning paths is likely to be a better answer to the given query . Apart from being non-parametric , our proposed method has many other desirable properties . Our method is generic and even though we find very simple and symbolic methods to work very well for many KG datasets , every component of our model can be augmented by plugging in sophisticated neural models . For example , currently we use simple symbolic string matching to find if a relation path exists for the query entity . However , this component can be replaced with a neural model that matches paths which are semantically similar to each other [ Das et al. , 2017 ] . Similarly , we currently use a very simple inner product to compute similarities between entity embeddings . But that can be replaced with more sophisticated maximum inner product search [ Mussmann and Ermon , 2016 ] . The contributions of the paper are as follows — ( a ) We present a non-parametric approach for reasoning over KGs , that uses multiple paths of evidence to derive an answer . These paths are gathered from entities in different parts of the KG that are similar to the query entity . ( b ) Our approach requires no training and can be readily applied to any new KGs . ( c ) Our method achieves state-of-the-art performance on NELL-995 and the harder subset of FB-122 outperforming sophisticated neural approaches and other tensor factorization methods . We also perform competitively on the WN18RR dataset . ( d ) Lastly , we provide detailed analysis about why our method outperforms parametric rule learning approaches like MINERVA [ Das et al. , 2018 ] . 2 . Method . 2.1 Notation and Task Description . Let E denote the set of entities and R denote the set of binary relations . A knowledge base ( KB ) is a collection of facts stored as triplets ( e1 , r , e2 ) where e1 , e2 ∈ E and r ∈ R . From the KB , a knowledge graph G can be constructed where the entities e1 , e2 are represented as the nodes and relation r as labeled edge between them . Formally , a KG is a directed labeled multigraph G = ( V , E , R ) , where V and E denote the vertices and edges of the graph respectively . Note that V = E and E ⊆V ×R ×V . Also , following previous approaches [ Bordes et al. , 2013 , Xiong et al. , 2017 ] , we add the inverse relation of every edge , i.e . for an edge ( e1 , r , e2 ) ∈ E , we add the edge ( e2 , r−1 , e1 ) to the graph . ( If the set of binary relations R does not contain the inverse relation r−1 , it is added to R as well ) . A path in a KG between two entities es , et is defined as a sequence of alternating entity and relations that connect es and et . A length of a path is the number of relation ( edges ) in the path . Formally , let a path p = ( e1 , r1 , e2 , . . . , rn , en+1 ) with st ( p ) = e1 , en ( p ) = en+1 and len ( p ) = n. Let P denote the set of all paths and Pn denote the set of all paths with length up to n , i.e . Pn ⊆ P = { p ∈ P | len ( p ) ≤n } . We consider the task of query answering on KGs . Query answering seeks to answer questions of the form ( e1q , rq , ? ) ( e.g . Melinda , works in city , ? ) , where the answer is an entity in the KG . 2.2 Case-based Reasoning on Knowledge Graphs . This section describes how we apply case-based reasoning ( CBR ) in a KG . In CBR , given a new problem , similar cases are retrieved from memory [ Aamodt and Plaza , 1994 ] . In our setting , a problem is a query ( e1q , rq , ? ) for a missing edge in the KG . A case is defined as an observed fact ( e1 , r , e2 ) in a KG along with a set of paths up to length n that connect e1 and e2 . Formally , a case c = ( e1 , r , e2 , P ) ⊆ V ×R ×V × Pn is a 4 tuple where ( e1 , r , e2 ) ∈ G and P ⊆ P ( e1 , e2 ) = { p ∈ Pn | st ( p ) = e1 , en ( p ) = e2 } . In practice , it is intractable to store all paths between e1 and e2 ( P ( e1 , e2 ) ) and therefore we only store a random sample . Note , that we have a case for every observed triple in the KG and we denote the set of all such cases as C . We also assume access to a pre-computed similarity matrix S ∈ RV×V which stores the similarity between any two given entities in a KG . Intuitively , two entities which exhibit the same relations should have a high similarity ( e.g. , an athlete should have a high similarity score with another athlete ) . Lastly , we define a memory M that serves as the store for all the cases present in a KG and also the similarity matrix , i.e . M = ( C , S ) . As explained earlier , CBR broadly comprises of four steps , which we explain briefly for our setup on a KG . • Retrieve : In this step , given a query ( e1q , rq , ? ) , our method first retrieves a set of k similar entities w.r.t e1q using the pre-computed similarity matrix S , such that for each entity e′ in the retrieved set , we observe at least one fact of the form ( e′ , rq , e′′ ) in G . In other words , each retrieved entity should have at least one outgoing edge in G with the edge label as rq . Next , we gather all such facts ( e′ , rq , e′′ ) in G for each of the retrieved e′ . Note , that there could be more than one fact for a given entity and query relation ( e.g . USA , has city , New York City and USA , has city , Boston , for the entity ‘ USA ’ and query relation ‘ has city ’ ) . Finally for all the gathered facts from the k-nearest neighbors , we retrieve all the cases from the memory store M . As noted before , in our formulation , a case is a fact augmented with a sample of KG paths that connect the entities of the fact . As we will see later ( § 3 ) , the KG paths often represent reasoning rules that determine why the fact ( e′ , rq , e′′ ) hold true . The goal of CBR is to reuse these rules for the new query . • Reuse : The reasoning paths that were gathered in the retrieve step are re-used for the query entity . As described before , a path in a KG is an alternating sequence of entities and relations . The path gathered from the nearest neighbors have entities which are in the immediate neighborhood of the nearest neighbor entities . To reuse these paths , we first replace the entities from the paths with un-instantiated variables and only extract the sequence of relations in them [ Schoenmackers et al. , 2010 ] . For example , if we have a retrieved case ( a fact with a set of paths ) such as ( ( USA , has city , Boston ) , { ( USA , has state , Massachusetts , city in state , Boston ) } ) , we remove the entities from the path and extract rules such as : has state ( x , y ) , city in state ( y , z ) =⇒ has city ( x , z ) . We gather all such paths from all the cases retrieved for a query . Since the same path type can occur in different cases , we maintain a list of paths sorted w.r.t the counts ( in descending order ) . • Revise : After the paths have been gathered , we look if those paths exists for the query entity e1q . In our approach we find that simple symbolic exact-string matching for the relations works quite well . However , neural relation extraction systems [ Zeng et al. , 2014 , Verga et al. , 2016 ] can be incorporated to map a relation to other similar relations to improve recall . We keep this direction as a part of our future work . Instead if we find an exact match for the sequence of relations , we revise the rules by instantiating the variables with the entities which lie along the path in the neighborhood of e1q . • Retain : Finally , a case including the query fact and the paths that lead to the correct answer for the query can be added to the memory store M . Computing the similarity matrix : CBR approaches need access to a similarity matrix S to retrieve similar cases to the query entity . Intuitively , the similarity between entities that have similar relations should be higher ( e.g . similarity between two athletes should be higher than the similarity between an athlete and a country ) . To model this , we parameterize each entity with an m-hot vector e ∈ RR . That is , each entity is a m-hot vector with the dimension equal to the size of the number of binary relations in the KB . An entry in the vector is set to 1 , if an entity has at least one edge with that relation type , otherwise is set to 0 . Even though , this is a really simple way of parameterizing an entity , we found this to work extremely well in practice . However , as previously mentioned we propose a generic method and one could replace the entity embeddings with any pre-trained vectors from any model . As an example , we present experiments by replacing our m-hot representation with pre-trained embeddings obtained from the state-of-the-art RotatE model [ Sun et al. , 2019 ] . Lastly , the similarity between two entities is calculated by a simple inner product between the normalized embeddings . Caching ‘ cases ’ in the memory store : CBR also needs access to store containing cases . As mentioned before , in our setup a ‘ case ’ is a KG triple , along with a sample of paths that connect the two entities of the triple . Since the number of paths between entities grow exponentially w.r.t path length , it is intractable to store all paths between an entity pair . We instead consider a small subgraph around each entity in the KG spanned by 1000 randomly sampled paths of length up to 3 . Next , for each triple ( e1 , r , e2 ) in the KG , we exhaustively search the subgraph around e1 , collected in the previous step to find paths up to length 3 which lead to e2 . These paths along with the fact form a case . This process is repeated for all the triples and each case is added to the case store C . Both the similarity matrix S and the case store C are pre-computed offline and is stored in the memory M . Once that is done , our method requires no further training and can be readily used for any query in the KG .

Summary: This paper provides a very simple approach to the problem of knowledge base completion. The idea is this - given a query (subject, relation), you find other entities similar to subject, see which other paths they can take to their corresponding object if they express the relation, and check if the subject express those paths. Object reached this way are candidate answers. The one with most paths reached is marked correct.