Towards Cross-checking W ord N et and SUMO Using Meronymy (2018)

Abstract: We describe the practical application of a black-box testing methodology for the validation of the knowledge encoded in WordNet, SUMO and their mapping by using automated theorem provers. In this paper,weconcentrateonthepart-whole information provided by WordNet and create a large set of tests on the basis of few question patterns. From our preliminary evaluation results, we report on some of the detected inconsistencies.

## introduction despite being created manually, knowledge resources such as wordnet @xcite and sumo @xcite are not free of errors and inconsistencies. unfortunatelly, improving, revising, and correcting such large knowledge bases is a never ending task that have been mainly carried out also manually. a few automatic approaches have been also applied focusing on checking certain structural properties on word-net (e.g. @xcite , @xcite ) or using automated theorem provers on sumo (e.g. @xcite , @xcite ). just a few more have studied automatic ways to validate the knowledge content encoded in these resources by cross-checking them. for instance, @xcite exploit the euroword-net top ontology @xcite and its mapping to wordnet for detecting many ontological conflicts and inconsistencies in the wordnet nominal hierarchy. in @xcite , we propose a method for the automatic creation of competency questions (cqs) @xcite , which enable to evaluate the competency of sumo-based ontologies. our proposal is based on several predefined question patterns (qps) that are instantiated using information from wordnet @xcite and its mapping into sumo @xcite . in addition, we also describe an application of automated theorem provers (atps) for the automatic evaluation of the proposed cqs. the main contribution of this paper is to demonstrate the practical capabilities of the method introduced in @xcite for the detection of semantic agreements and inconsistencies between wordnet and sumo thanks to their mapping. for this purpose, we propose a new set of cqs that is obtained on the basis of the part-whole data of wordnet. in our ongoing experimentations using the atps vampire @xcite ) and e @xcite , we have automatically detected some knowledge discrepancies and disagreements that were hidden in both wordnet, sumo and their mapping. outline of the paper. in the following three sections, we introduce wordnet, sumo, and their mapping. then, we describe our formal interpretation of the mapping information in section 5 and the proposed question patterns for the creation of competency questions in section 6. next, we discuss our preliminary evaluation results in section 7. finally, we report on the ongoing work in section 8 and provide some conclusions in section 9. ## meronymy relations in wordnet in wordnet, meronymy -the part-whole relation-holds between synsets like backrest 1 n and seat 1 n (i.e. parts) and chair 1 n (i.e. whole). parts are inherited from their superordinates: if a chair has a seat, then an armchair has a seat as well. but parts are not inherited "upward" as they may be characteristic only of specific kinds of things rather than the class as a whole: chairs and kinds of chairs have a seat, but not all kinds of furnitures have a seat. there exist 3 main meronymy relations in wordnet v3.0 that relate noun synsets: part, the general meronymy relation; member, which relates particulars and groups; substance, which relates physical matters and things. in total, there are 22,187 (ordered) synset pairs: 9,097 pairs using part, 12,293 pairs using member and 797 pairs using substance. for example, the synsets committee foot_0 n and committee member 1 n are related by member, while grape 1 n and wine 1 n are related by substance. ## sumo and adimen-sumo sumo 1 @xcite has its origins in the nineties, when a group of engineers from the ieee standard upper ontology working group pushed for a formal ontology standard. their goal was to develop a standard upper ontology to promote data interoperability, information search and retrieval, automated inference and natural language processing. currently, sumo consists of about 20,000 terms and about 70,000 axioms organized in several levels. in the the upper two levels -top and middle levels-one can find the concepts, relations and axioms that are meta, generic or abstract. from now on, we refer to the upper two levels of sumo as its core. on the basis of these two levels, concepts that are specific to particular domains are in the so-called domain ontologies. adimen-sumo @xcite is obtained by means of a suitable transformation of the knowledge in the core of sumo into fol, which enables its use by fol atps such as vampire @xcite ) and e @xcite . adimen-sumo inherits all the axioms in the core of sumo that can be expressed in fol (around an 88% of the axioms). the knowledge in sumo is organized around the notions of individuals and classes -the main sumo concepts. these concepts are respectively defined in adimen-sumo by means of the metapredicates @xmath0subclass. sumo individuals and classes are not disjoint, since every sumo class is defined to be instance of class and, thus, every sumo class is also a sumo individual. additionally, sumo also differentiates relations and attributes. in particular, sumo distinguishes between individual relation and attributes -that is, instances of the sumo classes relation and attribute respectively-and classes of relations and attributes -that is, subclasses of the sumo classes relation and attribute respectively. sumo provides specific predicates for dealing with relations and attributes. among others, we currently use the next ones in adimen-sumo: for simplicity, from now on we denote the nature of sumo concepts by adding as subscript the symbols o (sumo individuals that are neither classes nor relations nor attributes), c (sumo classes that are neither classes of relations nor classes of attributes), r (individual sumo relations), a (individual sumo attributes), r (classes of sumo relations) and a (classes of sumo attributes). for example: cell c , member r and larval a . ## the mapping between wordnet and sumo wordnet is linked with sumo by means of the mapping described in @xcite . this mapping connects synsets of wordnet to terms of sumo using three relations: equivalence, subsumption and instance. foot_2 equivalence denotes that the related wordnet synset and sumo concept are equivalent in meaning, whereas subsumption and instance indicate that the wordnet synset is subsumed by the sumo concept or is an instance of the sumo concept respectively. additionally, the mapping also uses the complementaries of equivalence and instance. we de- ## formal interpretations of the mapping between wordnet and sumo the automatic validation of wordnet and sumo on the basis of cqs and atps requires to translate all the information into a formal language. by means of adimen-sumo @xcite , the core information of sumo is already written in fol. however, wordnet and its mapping to sumo are not formally characterized. therefore, we next describe and compare two possible formal interpretations of the mapping between wordnet and sumo. the first possible interpretation is just to literally follow the definition of the mapping relations provided in @xcite . that is: however, the above literal interpretation of the mapping suffers from several problems. on one hand, subsumption and instance lack an obvious interpretation when referred to sumo individuals: 4 it is non-sense to assert that an individual has hyponyms or individuals and, in addition, there is only one sumo predicate for dealing with relations (i.e. subrelation r ) and attributes (subattribute r ) respectively. on the other hand, the literal interpretation of the mapping may yield to inconsistent statements when applied to synsets that are connected to several sumo concepts. for example, male horse 1 n is connected to both male a + and horse c +. thus, male horse 1 n would be interpreted of hyponym of both male a and horse c . for this purpose, we would use the sumo predicates subattribute r and @xmath1instance for sumo classes and attribute r for sumo individual attributes. 6 the interested reader is referred to @xcite for further details. for example, the synsets malacosoma americana 1 n and genus malacosoma 1 n are connected to insect c + and larval a + respectively. by associating the variables ?x and ?y to each synset, we generate the following adimen-sumo statements: on the basis of the above adimen-sumo statements that restrict the set of potential sumo individuals related to a synset, the second interpretation of the mapping information is completed according to the mapping relation that links the synset and the sumo concept: this second interpretation of the mapping information takes advantage from the fact that most of the sumo knowledge is based on the notion of individuals and that only a few of sumo predicates provide information at the level of classes. from this point of view, this interpretation enables a more precise use of the knowledge of sumo. in addition, the problem with synsets connected to several sumo concepts is overcome. going back to the example about male horse 1 n , its mapping to male a + and horse c + can be translated as (and ## competency questions based on meronymy in this section, we describe the set of cqs that is created on the basis of the part-whole data provided by wordnet. for this purpose, we consider the second interpretation of the mapping information introduced in section 5. since that interpretation does not distinguish between subsumption and instance, we only consider two linking options for wordnet synsets: synsets connected by equivalence (or its negation) and synsets connected by (the negation of) subsumption or instance. therefore, there are just 4 possible combinations of mapping relations in the 12,293 ordered synset pairs provided by wordnet and we propose a different question pattern for each of them. given an ordered synset pair, the corresponding question pattern is instantiated according to a) the wordnet meronymy relation and b) the sumo concepts to which synsets are connected. with respect to wordnet meronymy relations, we have inspected sumo in order to find the relations that are synonym or semantically similar to (exists (?x, ?y) (and first question pattern for s part, s whole meronymy pairs them. in sumo, the main meronymy relation is part r and we can find 30 different subrelations of part r in its core. among them, we have selected the sumo predicates part r , member r piece r as counterpart of the wordnet relations part, member and substance respectively. as for every sumo relation, sumo provides domain axioms that restrict the set of sumo individuals that can be related by the above predicates as follows: additionally, sumo also defines several incompatibilities between sumo individuals. among others, individuals of corpuscularobject c are not compatible with neither collection c nor substance c because corpuscularobject c and collection c (also substance c ) are defined as disjoint classes. on the basis of individual sumo incompatibilities, we can already detect some errors. for example, the synsets grape 1 n and wine 1 n are related by substance (as introduced in section 2) and respectively connected fruitorvegetable c + and wine c =. in sumo, fruitorvegetable c is defined to be subclass of corpuscularobject c . consequently, fruitorvegetable c is incompatible with substance c , which prevents the use of piece r for relating synsets pairs with individuals of fruitorvegetable c in the first place. the source of this error is discussed in section 7. after choosing the most suitable sumo predicate for a given synset pair, the instantiation of the corresponding question pattern is finished according to the sumo concepts to which synsets are figure 2 : second question pattern for s part, s whole meronymy pairs connected. more specifically, we apply the second interpretation of the mapping information in order to obtain a adimen-sumo statement for each synset. the resulting adimen-sumo statements are directly used for the instantiation of question patterns. in the next subsections, we describe the proposed question patterns. ## ongoing work currently, we are finishing our experimental evaluation of wordnet, sumo and their mapping by applying the methodology proposed in @xcite . for this purpose, we are using the atps vampire @xcite and e @xcite for checking whether the conjectures resulting from the set of cqs proposed in this paper are entailed or not by adimen-sumo. all the resources -the ontology, the set of cqs and conjectures, and the resulting execution reports-will be available at http:// adimen.si.ehu.es/web/adimensumo . by analysing our preliminary experimentation results, we can conclude that our proposal enables a sophisticated cross-checking of the information in wordnet, sumo and their mapping. in particular, by means of practical examples, we have illustrated that the proposed system enables (a) the validation of some pieces of information and (b) the detection of missing information and inconsistencies. further, our preliminary experimental results also demonstrate the suitability of the involved resources for its application to practical tasks related to natural language processing. ## concluding remarks in this work, we enlarge the set of cqs proposed in @xcite by means of part-whole data of wordnet, which illustrates the fact that our proposal can be generally applied to any data extracted from wordnet. nowadays, our complete set of cqs includes around 3,000 cqs obtained from antonymy and around 2,000 cqs obtained from morphosemantic links database of wordnet. in the last case, we exclusively concentrate on the relations event, agent, instrument and result. in the next future, we plan to extend our benchmark by considering additional wordnet relations.