Towards Effective Correction Methods Using W ord N et Meronymy Relations (2023)

Abstract: In this paper, we analyse and compare several correction methods of knowledge resources with the purpose of improving the abilities of systems that require commonsense reasoning with the least possible human-effort. To this end, we cross-check the WordNet meronymy relation member against the knowledge encoded in a SUMO-based first-order logic ontology on the basis of the mapping between WordNet and SUMO. In particular, we focus on the knowledge in WordNet regarding the taxonomy of animals and plants. Despite being created manually, these knowledge resources — WordNet, SUMO and their mapping — are not free of errors and discrepancies. Thus, we propose three correction methods by semi-automatically improving the alignment between WordNet and SUMO, by performing some few corrections in SUMO and by combining the above two strategies. The evaluation of each method includes the required human-effort and the achieved improvement on unseen data from the WebChild project, that is tested using first-order logic automated theorem provers.

## introduction the areas of commonsense knowledge representation and commonsense reasoning are of great interest for their application in many tasks related to natural language processing (nlp) e.g. recognizing textual entailment (rte) @xcite @xcite , natural language inference (nli) @xcite or interpretable semantic textual similarity (ists) @xcite . in the literature, among the knowledge resources, wordnet @xcite is one of the most frequently used semantic resources that is applied to nlp tasks. furthermore, wordnet interlinks many other semantic resources e.g. the eurowordnet top on-tology @xcite , or sumo foot_0 @xcite . when linking lexical resources such as wordnet @xcite and ontologies such as sumo @xcite , dolce @xcite or opencyc @xcite , @xcite generalised these three methodological options: restructuring, populating and aligning. but, moreover, ontologies and lexical resources can also be used to cross-check them and validate the knowledge content encoded. in order to automatically cross-check the knowledge in wordnet and sumo, @xcite introduced a general framework that enables evaluating the competency of sumo-based ontologies like adimen-sumo @xcite and proposed a method for the automatic creation of competency questions (cqs) @xcite . their proposal is based on several predefined question patterns (qps) that are instantiated using information from wordnet and its mapping into sumo @xcite . in addition, the authors described an application of first-order logic (fol) automated theorem provers (atps) for the automatic evaluation of the proposed cqs. however, a low percentage of the meronymy pairs from wordnet can be validated against sumo using the proposed framework, as reported by @xcite ; @xcite . overall, three possible causes for this low validation ratio have been identified: the first one is valid because the knowledge from wordnet, sumo and its mapping is correctly aligned: individuals (parent 1 n ) with an instance of biologicalattribute as property can be members of instances of familygroup (family 2 n ). however, the second case is invalid: n should be corrected to be subsumed by groupofanimals. in this paper, our aim is shedding light on the sources of difficulty when correcting knowledge resources, which is a mainly manual and never ending task. exactly, we want to discover which correction methods and strategies lead to maximising the improvement with the least possible humaneffort. to this end, we consider three correction approaches: i) the correction of the mapping between wordnet and sumo on the basis of the word-net hierarchy and our manual error analysis of the results reported in @xcite ; ii) the correction of the knowledge in sumo in order to its alignment to wordnet; iii) the combination of the previous two approaches. we report on a practical evaluation of the impact of each correction method on unseen data provided by the webchild project @xcite , which is a large collection of commonsense knowledge that has been automatically extracted and disambiguated from web contents. to the best of our knowledge, this is the first work dealing with the problem of correcting fol commonsense resources. outline. first, we present the related work in the next section and review the knowledge resources and evaluation framework in section 3. then, we describe the proposed correction methods in section 4 and provide the evaluation results in section 5. finally, we conclude and outline the future work in section 6. ## related work in this section, we present the works related to meronymy knowledge and its acquisition, crosschecking resources, mapping error detection and ontology debugging and repairing. meronymy is a semantic relation that connects the parts and the whole. this connection can be functional, homeomeric/homeomerous (consisting of similar parts), separable or simultaneous @xcite . in the typology of meronymy relations, the most important subrelations are constituent-object, member-collection and material-object. the importance of meronymy is pointed out by vor der brück and helbig (2010), which extract meronymy relations from wikipedia by means of a logic-oriented approach. according to them, meronymy is necessary for many nlp tasks such as question answering. following their example, if someone asks about the earthquakes in europe, then the question could be answered thanks to the meronymy relation if we had the data of each european country. both manual and automated attempts have been made to acquire meronymy knowledge. among the first ones, there are more than 22,000 meronymic pairs in wordnet @xcite , that have been manually constructed and reviewed. wordnet is a large lexical database of english where nouns, verbs, adjectives and adverbs are grouped into sets of synonyms called synsets, 2 each one denoting a distinct concept. moreover, synsets are interlinked 2 in this paper, we will refer to the synsets using the format word s p , where s is the sense number and p is the part-of-speech: n for nouns and v for verbs e.g. plant 2 n means that the word plant is a noun and that we are referring to its second sense in wordnet. by means of lexical-semantic relations. wordnet encodes three main meronymy relations that relate noun synsets: i) part, the general meronymy relation; ii) member, which relates particulars and groups; and iii) substance, which relates physical matters and things. in total, wordnet v3.0 includes 22,187 (ordered) meronymy relations (around 10 % of the relations between synset pairs in wordnet): 9,097 pairs using part, 12,293 pairs using member and 797 pairs using substance. for example, the synsets tongue 1 n and mouth 1 n are related by part, lamb 1 n and genus_ovis 1 n are related by member, and neuroglia 1 n and glioma 1 n are related by substance. furthermore, additional relations were manually added to wordnet in @xcite on the basis of featural descriptions. however, the coverage of the collected meronymy knowledge is quite restricted. this limitation is also present in some automated proposals like conceptnet @xcite , which has been obtained by crowdsourcing and contains around 20,000 meronymy relation pairs between non-disambiguated words. the coverage of the automatically acquired meronymy knowledge is larger in other works. for example, pwkb (the part-whole kb) @xcite , which has been integrated into webchild v2.0 @xcite , consists of almost 6 millions of disambiguated meronymy pairs that have been obtained from web contents and image tags by combining pattern-based information extraction methods and logical reasoning. however, this kb suffers from low salience since more pairs were obtained by expanding a small set of relations. a complementary resource is haspartkb @xcite , which contains more salient and accurate haspart relations (around 50,000) extracted from a large corpus of generic statements. finally, quasimodo @xcite and aristo tuple kb @xcite contain several thousands of non-disambiguated meronymy pairs, but their coverage is rather limited. the knowledge in all the above cited resources is restricted to relation pairs. regarding general knowledge, sumo @xcite contains both facts and axioms that encode more abstract information and properties about meronymy. in relation with cross-checking knowledge resources, @xcite exploit the euroword-net top ontology @xcite and its mapping to wordnet for detecting many ontolog-ical conflicts and inconsistencies in the wordnet nominal hierarchy. most of the works presented for error correction both in the mapping and in the ontologies have been proposed for owl ontologies. relating mapping error detection and correction, many methods have been proposed to detect mapping errors or invalid mappings between ontologies, knowledge resources, dictionaries and thesauri @xcite . similar to us, @xcite reasoning strategies for the biomedical domain. exactly, they use description logics to detect inconsistencies since they consider that ontologies are consistent, and therefore, errors come from the mappings. @xcite divided the mapping errors in four categories (from now on, wang&xu classification): redundant, imprecise, inconsistent and abnormal mappings. correction strategies are presented in @xcite for the biomedical domain, where questions are proposed to experts in order to validate the mapping and the ontology. surveys on mapping maintenance and ontology matching are respectively presented in @xcite and @xcite . relating ontology error detection, recent work on ontology debugging involves detecting hidden modelling errors: teymourlouie et al. (2018) use dbpedia during the ontology debugging process to detect contradictions in ontologies that seem coherent. unfortunately, as far as we know, no correction approach has been proposed. ## knowledge resources and evaluation framework for wordnet meronymy in this section, we describe the knowledge resources and framework that enable the automatic evaluation of the meronymy relation member of wordnet by using automated theorem provers (atps). adimen-sumo @xcite is a firstorder logic (fol) ontology obtained by means of a suitable transformation of most of the knowledge (around 88 % of the axioms) in the top and middle levels of sumo @xcite . adimen-sumo enables the application of state-ofthe-art fol atps such as vampire @xcite ) and e @xcite in order to automatically reason on the basis of the knowledge in sumo @xcite . sumo is organised around the notions of particulars (also called instances or objects) and classes by means of the meta-predicates instance and subclass. amongst them, sumo also differentiates relations and attributes, and provides specific predicates for their use that are inherited by adimen-sumo e.g. subrelation and attribute. we denote the nature of sumo concepts by adding as subscript the following symbols: o for sumo objects, c for sumo classes, r for sumo relations, a for sumo attributes and a for classes of sumo attributes. for example: waist o , groupofanimals c , material r , solid a and biologicalattribute a . wordnet and sumo are linked by means of a semantic mapping that connects wordnet synsets to sumo concepts using three relations: equivalence, subsumption and instance @xcite . the mapping relation equivalence connects wordnet synsets and sumo concepts that are semantically equivalent. subsumption (or instance) is used when the semantics of the wordnet synsets is less general than (or instance of) the semantics of the sumo concepts to which the synsets are connected. for example, the synset lamb 1 n is connected to lamb c by equivalence and neuroglia 1 n is connected to tissue c by subsumption. from now on, we denote the semantic mapping relations by concatenating the symbols '= ' (equivalence), '+' (subsumption) and '@' (instance) to the corresponding sumo concept e.g. lamb 1 for the automatic evaluation of the wordnet meronymy relations, we apply the framework introduced in @xcite , which is an adaptation of the method proposed in @xcite for the formal design and evaluation of ontologies on the basis of competency questions (cqs). this framework enables the use of atps in order to automatically classify cqs as follows: cqs are decided to be passing (if proved to be entailed by the ontology), non-passing (their negations are proved to be entailed by the ontology) and unresolved (neither the cqs nor their negations are proved to be entailed by the ontology). furthermore, we adapt the question patterns (qps) for the meronymy relation member introduced in @xcite ; @xcite , which enable the translation of its semantics into a suitable cq or yield a semantically incorrect conjecture according to the restrictions for relations provided by sumo. those qps employs the translation of the mapping information of synsets into adimen-sumo statements that is described in @xcite , which characterises the semantics of wordnet synsets in terms of sumo instances and requires the use of a new variable for each synset. there is a different qp for each possible combination of mapping relations, which states the quantification of the introduced variables and the logical connectives that enable the construction of the final cq. for example, the synsets sheep 1 n and flock 5 n are respectively connected to sheep @xmath0. thus, we use the second qp proposed in @xcite because of the use of the mapping relations equivalence and subsumption, and obtain the following conjecture: finally, the wordnet meronymy pairs on member can be classified according to the following categories depending of: a) if the member pair is translated into a cq, then it is decided to be validated, unvalidated or unknown if the cq is passing, non-passing or unresolved respectively; b) the member pairs that yield to semantically incorrect conjectures are classified as unvalidated. by using the above described framework and regarding the original versions of sumo and its mapping from wordnet, from the 12,293 member pairs provided by wordnet only 19 are validated, while 11,963 pairs are unvalidated and 311 remains unknown. moreover, from the 11,963 unvalidated pairs, only 24 yield a correct cq. that is, the direct application of the introduced evaluation framework just allows to validate a mere 1.5% of the member pairs encoded in wordnet and, apparently, most of the unvalidated pairs yield semantically incorrect sumo conjectures. this may be an indication of both misalignment in the knowledge encoded in wordnet and sumo and the existence of a large number of discrepancies in their mapping. ## knowledge correction methods in this section, we introduce the proposed correction strategies for knowledge resources. for our analysis and interventions, we have used the information contained in the multilingual central repository @xcite . ex-actly, we have consulted: the basic level concepts (blcs) @xcite , which are frequent and salient concepts in wordnet that try to represent as many concepts as possible (abstract concepts) and as many distinctive features as possible (concrete concepts); the top concept ontology (tco) @xcite ; the semantic files (sf) from wordnet and the wordnet domains (wnd) @xcite . moreover, we have also consulted sumo and its documentation. ## evaluation in this section, we evaluate the proposed knowledge correction methods on both seen and unseen data, which is extracted from the webchild project. in regarding seen data, it is easy to see that matching knowledge discrepancies outperforms mapping correction, although the improvement is low in both cases: correcting the mapping turns almost a half of the previously unvalidated pairs into unknown while matching knowledge discrepancies increases a bit the number of validated pairs. however, by combining both interventions the improvement is much higher: the amount of validated pairs is 500 times bigger and the amount of unvalidated pairs is almost 15 times smaller. with respect to the data extracted from the we-bchild project, the combined intervention heavily improves the results again, although the impact is a bit lower: many pairs still remain unknown and the ratio between validated and unvalidated pairs is lower than in the case of wordnet. for a better understanding of these results, we have manually analysed a sample of webchild pairs consisting of five randomly selected cases from each output @xcite . considering the validated pairs, 4/5 have been classified as validated for good reasons, e.g. the only error is a wrong pair in the knowledge base: the synset genus 2 n ("(biology) taxonomic group containing one or more species") is incorrectly asserted to be member of carapidae 1 n ("pearlfishes: related to the brotulidae"). from the unvalidated pairs, 2/5 pairs are wrong so they have been correctly classified as unvalidated e.g. superphylum 1 n is not a member of locative_role 1 n . however, 3/5 pairs are correct and should have been validated, but there are mapping errors e.g. auriculariaceae 1 n is member of tremellales 1 n , although the pair is classified as unvalidated because tremellales 1 n is still mapped to fungus c . finally, in relation to unknown pairs, one pair is correct -rice_weevil 1 n is member of sitophylus 1 nand 4/5 pairs are wrong, e.g. relative 1 n is not a member of ming_dynasty 1 n . however, these pairs cannot be resolved by atps because the required information is missed in the ontology or, as in the case of the correct pair, due to resource (specially time) restrictions. ## conclusions and future work in this paper we have reported on several correction methods for the knowledge about meronymy in wordnet, sumo and their mapping with the aim of improving the abilities of systems that require commonsense reasoning. to this end, we have applied fol atps on a large set of cqs automatically constructed on the basis of several predefined qps and the knowledge of the involved resources. since finding and correcting errors in knowledge resources has always been time-consuming and required quite a lot of manual work, we have focused on the human-effort required for each corby analysing our evaluation results on word-net, it seems at first glance to be worth investing effort correcting and matching the knowledge of the involved resources, since the improvement is slightly higher (see regarding the evaluation on unseen data, we have confirmed that our interventions are correct, although there is still a lot of work to do. furthermore, our detailed analysis revealed some aspects for future work. for example, the capture of metonymy, solving additional misalignments (e.g. classifying humans as animals) and the need of analysing the inheritance of relations. moreover, we plan to test if the improved knowledge resources also obtain better results in other benchmarks based on antonymy and semantic roles @xcite , and we would like to carry out similar experiments in other datasets e.g. bless 4 @xcite . additionally, we also plan to consider additional wordnet relations: for example, the remaining relations about meronymy part and substance, cause or the semantic roles described in the morphosemantic links (fellbaum 3 http://adimen.si.ehu.es/web/ adimensumo 4 https://sites.google.com/site/ geometricalmodels/shared-evaluation @xcite . longer term research includes a new mapping between wordnet and sumo on the basis of formulae instead of labels, with the aim of providing a more precise definition of the semantics of synsets in terms of the sumo language.