Enriching Knowledge Bases with Interesting Negative Statements

In this paper , we make the case for explicitly stating interesting statements which are not true . Negative statements would be important to overcome current limitations of question answering , yet due to their potential abundance , any effort towards compiling them needs a tight coupling with ranking . We introduce two approaches towards automatically compiling negative statements . ( i ) In peer-based statistical inferences , we compare entities with highly related entities in order to derive potential negative statements , which we then rank using supervised and unsupervised features . ( ii ) In pattern-based query log extraction , we use a pattern-based approach for harvesting search engine query logs . Experimental results show that both approaches hold promising and complementary potential . Along with this paper , we publish the first datasets on interesting negative information , containing over 1.4M statements for 130K popular Wikidata entities . 1 . Introduction . Motivation and problem Structured knowledge is crucial in a range of applications like question answering , dialogue agents , and recommendation systems . The required knowledge is usually stored in KBs , and recent years have seen a rise of interest in KB construction , querying and maintenance , with notable projects being Wikidata [ Vrandečić and Krötzsch , 2014 ] , DBpedia [ Auer et al. , 2007 ] , Yago [ Suchanek et al. , 2007 ] , or the Google Knowledge Graph [ Singhal , 2012 ] . These KBs store positive statements such as “ Canberra is the capital of Australia ” , and are a key asset for many knowledge-intensive AI applications . A major limitation of most of these KBs is their inability to deal with negative information . At present , the major KBs contain virtually only positive statements , whereas statements such as that “ Tom Cruise did not win an Oscar ” could only be inferred with the major assumption that the KB is complete - the so-called closed-world assumption ( CWA ) . Yet as KBs are only pragmatic collections of positive statements , the CWA is not realistic to assume , and there remains uncertainty whether statements not contained in a KBs are false , or truth is merely unknown to the KB . Not being able to formally distinguish whether a statement is false or unknown poses challenges in a variety of applications . In medicine , for instance , it is important to distinguish between knowing about the absence of a biochemical reaction between substances , and not knowing about its existence at all . In corporate integrity , it is important to know whether a person was never employed by a certain competitor , while in anti-corruption investigations , absence of family relations needs to be ascertained . In data science and machine learning , on-the-spot counterexamples are important to ensure the correctness of learned extraction patterns and associations . State of the art and its limitations Current web-scale KBs contain almost only positive statements , and this is engraved in the open-world assumption ( OWA ) employed on the semantic web , which states that asserted statements are true , while the remainder is unknown ( a notable exception being Wikidata , which we discuss in details in Appendix E ) . Some formal entailment regimes like OWL [ McGuinness et al. , 2004 ] go beyond the OWA assumption , and allow to infer negation , yet are intended for use at query time , not for static materialization , and also lack ranking facilities . Similarly , data constraints [ Marx and Krötzsch , 2017 ] and association rules [ Ortona et al. , 2018 ] can in principle yield negative statements , but face the same challenges . This has consequences for usage of KBs : for instance , today ’ s question answering ( QA ) systems are well geared for positive questions , and questions where exactly one answer should be returned ( e.g. , quiz questions or reading comprehension tasks ) [ Fader et al. , 2014 , Yang et al. , 2015 ] . In contrast , for answering negative questions like “ Actors without Oscars ” , QA systems lack a data basis . Similarly , they struggle with positive questions that have no answer , like “ Children of Angela Merkel ” , too often still returning a besteffort answer even if it is incorrect . Materialized negative information would allow a better treatment of both cases . Approach and contribution In this paper , we make the case that important negative knowledge should be explicitly materialized . We motivate this selective materialization with the challenge of overseeing a near-infinite space of possibly true statements that are not asserted in KBs , and with the importance of explicit negation in search and question answering . We then develop two complementary approaches towards generating negative statements : statistical ranking methods for statements derived based on related entities , and pattern-based text extraction , applied to high-quality search engine query logs . We also present the first datasets on interesting negative information , and highlight the usefulness of negative knowledge in extrinsic use cases . Our salient contributions are : 1 . We make the first comprehensive case for materializing interesting negative statements in KBs ; 2 . We present two judiciously designed methods for collecting negative statements : peerbased statistical inference and pattern-based text extraction ; 3 . We produce two datasets containing over 1.4M interesting negative statements for 130K popular Wikidata subjects . 4 . We show the usefulness of negative knowledge in a QA use case . 2 . Problem and Design Space . Formalization For the remainder we assume that a KB is a set of statements , each being a triple ( s ; p ; o ) of subject s , property p and object o . Let Ki be an ( imaginary ) ideal KB that perfectly represents reality , i.e. , contains exactly those statements that hold in reality . Under the OWA , ( practically ) available KBs Ka contain correct statements , but may be incomplete , so the condition Ka ⊆ Ki holds , but not the converse [ Razniewski and Nutt , 2011 ] . We distinguish two forms of negative statements : Definition 1 ( Negative statements ) 1 . A grounded negative statement ¬ ( s ; p ; o ) is satisfied if ( s ; p ; o ) is not in Ki . 2 . A universally negative statement ¬∃ ( s ; p ; ) is satisfied if there exists no o such that ( s ; p ; o ) ∈ Ki . Both statements represent standard logical constructs , and could also be expressed in the OWL ontology language . Grounded negative statements could be expressed via negative property statements ( e.g. , NegativeObjectPropertyStatement ( : hasWife : Bill : Mary ) ) , while universally negative statements could be expressed via owl : complementOf and ObjectSomeValuesFrom [ Erxleben et al. , 2014 ] . For these classes of negative statements , checking that there is no conflict with a positive statement is trivial . Yet compiling negative statements faces two other challenges . First , being not in conflict with positive statements is a necessary but not a sufficient condition for correctness of negation , due to the OWA . In particular , Ki is only a virtual construct , so methods to derive correct negative statements have to rely on the limited positive information contained in Ka , or utilize external evidence , e.g. , from text . Second , the set of correct negative statements is near-infinite , especially for grounded negative statements . Thus , unlike for positive statements , negative statement construction/extraction needs a tight coupling with ranking methods . Problem 1 Given an entity e in a KB , compile a ranked list of interesting grounded negative and universally negative statements . Design space A first thought is that deletions from time-variant KBs are a natural source . For instance , in Wikidata , for subjects of type person within the last year , more than 500K triples have been deleted . Yet on careful inspection we found that most of these concern ontology restructuring , granularity refinements , or blatant typos , thus do not give rise to interesting negation . A second conceivable approach is to leverage the CWA , or its relaxed variant PCA ( Partial Completeness Assumption , aka . LCWA for Local CWA ) [ Galárraga et al. , 2015 ] , to generate negative statements . Using just the active domain of Wikidata for grounding , the CWA would give rise to about 6.4×1018 negative statements1 . Assuming that Wikidata covers 10 % of all true statements per entity , more than 99.999 % of the negative statements would be correct , but hardly interesting . For the PCA , the total would be about 3.2× 1016 negative statements ( assuming an average of 5 populated properties per entity ) , and almost all of these would be correct . But these approaches would miss the true issue : merely enumerating huge sets of negative statements is not insightful even with ( trivially ) high precision . The key challenge rather is to identify interesting statements that users find noteworthy . Instead , we propose methods that follow two major paradigms for KB construction and completion : Statistical inference and text extraction . 1 . 80 Million subjects times 1000 properties times 80 Million objects . Statistical inference methods , ranging from association rule mining such as AMIE and RuDiK [ Galárraga et al. , 2013 , Ortona et al. , 2018 ] to embedding models such as TransE and HolE [ Bordes et al. , 2013 , Nickel et al. , 2016 ] can predict positive statements and provide ranked lists of role fillers for KB relations . In Section 4 , we develop a statistical inference method for negative statements , which generates candidate sets from related entities , and uses a set of popularity and probability heuristics in order to rank these statements . Textual information extraction ( IE ) is a standard paradigm for KB construction . Common challenges in textual IE comprise noise and sparsity in observations , and canonicalization of entities and predicates . Our goal is to achieve maximal flexibility w.r.t . open predicates , and to overcome sparsity in negative statements in texts . Section 5 presents a method that combines pattern-based and open information extraction , and applies it to a particularly rich data source , search engine query logs . 3 . Related Work . The problem of compiling informative negative statements about entities is new , so there are no directly comparable methods . Nevertheless , there is prior work on rule mining over KBs [ Galárraga et al. , 2015 ] that is conceivably useful in our context . Most notably , [ Galárraga et al. , 2017 ] employed rule mining to predict the completeness of properties for given entities . This corresponds to learning whether the PCA holds in a local part of the KB , inferring that all absent values for a subject-predicate pair are false . For our task , this could be a building block , but it does not address the inference of interesting negative statements . [ Ortona et al. , 2018 ] devised a rule mining system that can learn rules with negative atoms in rule heads ( e.g. , people born in Germany can not be US president ) . This could be utilized towards predicting negative statements . Unfortunately , the mining also discovers many convoluted and exotic rules ( e.g. , people whose body weight is less than their birth year can not win the Nobel prize ) , often with a large number of atoms in the rule body , and such rules are among the top-ranked ones . Even good rules , such as “ people with birth year after 2000 do not win the Nobel prize ” , are not that useful for our task . Such rules predict way too many – correct , but uninformative – negative statements , essentially enumerating a huge set of people who are not Nobel laureates . [ Ortona et al. , 2018 ] also proposed a precision-oriented variant of PCA that assumes negation only if subject and object are connected by at least one other relation . Unfortunately , this condition is rarely met in interesting cases . For instance , none of the negative statements in Table 2 have alternative connections between subject and object in Wikidata . Another related line of work is learning which attributes are mandatory in a KB , for only non-mandatory absent predicates are candidates for universal absence . [ Lajus and Suchanek , 2018 ] exploits density differences along type hierarchies to this end . This could be an initial filter towards discovering negative statements , but does not address our key problem of inferring when a missing statement is truly negative and interesting .

Summary: The paper addresses the problem of negative statements in knowledgebase. They formalize the types of negative statements: (a) grounded statement: [s,p,o] does not exist in KB, (b) not exist [s, p, o] (there's no object that satisfy s,p]. To find negative statements, they proposes two methods (a) peer-based candidate retrieval (i.e., heuristic of finding relation that is frequently populated in nearby entities but missing in the target entity) and (b) Using search logs with meta patterns (i.e., search query logs for pattern such as "Why XXX not", and find retrieved queries such as "Why XXX never won the Oscar).