Comparing Sense Categorization Between E nglish P rop B ank and E nglish W ord N et (2019)

Abstract: Given the fact that verbs play a crucial role in language comprehension, this paper presents a study which compares the verb senses in English PropBank with the ones in English WordNet through manual tagging. After analyzing 1554 senses in 1453 distinct verbs, we have found out that while the majority of the senses in PropBank have their one-to-one correspondents in WordNet, a substantial amount of them are differentiated. Furthermore, by analysing the differences between our manually-tagged and an automatically-tagged resource, we claim that manual tagging can help provide better results in sense annotation.

## introduction the main challenge in lexical semantics is generated from 'polysemy', which refers to the phenomenon of a single orthographic word having multiple, interrelated senses. only with the help of the context one can pin down the particular sense in which a word is used. a further challenge in the field stems from multi-word expressions, i.e. groups of words having "a unitary meaning which does not correspond to the compositional meaning of their parts" @xcite . hence, there are two central concerns for semantic analysis, centered around compositionality: (i) at the lexical level, choosing the correct sense of a given word within a context, and (ii) at the sentence level, determining how a particular combination of words should be interpreted. having semantic analysis of annotated corpora along with the syntactic architecture enhances natural language processing (nlp) applications such as information retrieval, machine translation, information extraction, and question answering. using the added semantic layer, syntactic parser refinements can be achieved, which not only increases the efficiency but also improves application performance. propbank @xcite @xcite is one of the studies on this concept, widely accepted by computational linguistics communities. in this paper, we present a sense category evaluation between english propbank and wordnet. in order to compare the sense categories, we first manually disambiguate english verbs in the input sentences with sense tags from english wordnet. then, these annotations are compared with sense annotations in english propbank. this paper is organized as follows: since we compare senses in english propbank and word-net, we provide information about these resources in section 2 and touch upon the related work about combinations of these resources in section 3. the details of our sense-annotated corpus and how it is constructed are given in section 4. we give the comparison details and statistics in section 5, and differences between automatic vs. manual tagging in section 6. lastly, we conclude in section 7. ## resources propbank is a corpus where predicate-argument information is annotated and semantic roles or arguments each verb can take are posited @xcite . propbank uses conceptual labels for arguments from arg0 to arg5. only arg0 and arg1 indicate the same roles across different verbs, standing for agent/causer and patient/theme, respectively. the rest of the argument roles can vary across different verbs. for instance, the roles of the predicate "attack" from propbank are as follows: arg0 is "attacker", arg1 is "entity attacked", and arg2 is "attribute". wordnet is a graph data structure where the nodes are word senses with their associated word forms and edges are semantic relations between the sense pairs. the first wordnet project was princeton wordnet (pwn) which was initiated in 1995 by george miller, @xcite . over time, pwn evolved to become a comprehensive relational representation of the word senses of english @xcite . wordnet includes relations between synsets such as hypernym, instance hypernym, hyponym, instance hyponym, meronym, holonym, antonym, entailment, etc. ## related work among many previous studies similar to ours is the one by @xcite , which aims to extract frame pairs by combining the lexical database of wordnet with the syntactic and semantic information given in verbnet and semantically-annotated corpus of verbs in prop-bank. having inferred 989 frame pairs with troponymy, entailment, causation, and antinomy; they conclude that nlp applications can benefit from such repositories by making use of automatic or semi-automatic techniques to map arguments across the frames. in another study, @xcite first assigned the frame for each verb sense in word-net from framenet and then, aligned roles among framenet, wordnet, and lcs depending on their mappings. in total, 4240 senses are linked with framenet frames, 674 of which are also linked with lcs, 1250 with propbank, and 1757 with both. semlink @xcite is another project which aims to combine different information provided by various lexical resources @xcite , framenet @xcite , propbank @xcite and word-net @xcite ). with mappings among these resources, the project aims to develop an nlp resource with extended overall coverage. aiming for interoperability among the same resources used in semlink, lópez de lacalle et al. @xcite @xcite ) focus on predicates and try to develop a common semantic infrastructure, which is called the predicate matrix (pm). they define a set of methods, such as advanced graph-based word sense disambiguation algorithms and various corpus alignment methods to automatically achieve this integration. while they base their work on the central motivation of semlink, the authors criticize the limitations of the manual methods used in developing the semlink project and argue that "building large and rich enough predicate models for broad-coverage semantic processing takes a great deal of expensive manual effort" (de @xcite . lópez de lacalle et al.'s (2016b) work is definitely a major progress for nlp studies centering around predicate structure. their approach, however, does not seem to put enough emphasis on the cognitive nature of language. undoubtedly, manual tagging requires significant human effort, hence is more costly, and manually tagged corpora may be more limited in terms of systematicity and coverage. however, whether any analysis of language can be fully automatized is still a very skeptical issue. according to the approach adopted in the present study, the use of human annotators is considered worthwhile for developing corpora by focusing on semantic information. there may well be 'human errors', but overall, we believe that manual tagging still gives us better results in qualitative terms, though maybe not in quantitative terms. ## comparison details in the comparison of the senses in english prop-bank and english wordnet, what has been found out is that whereas most of the senses in word-net seem to match the ones in propbank, some of them do not. we came across these mismatches for three reasons; because of (i) the senses going under differentiation, (ii) the senses getting combined in english wordnet or (iii) the overlaps of senses in a given verb in that one sense of a verb in wordnet corresponds to more than one single sense of the same verb in propbank. in sections 5.1 and 5.2, we will review oneto-one and one-to-many sense matchings between english propbank and english wordnet, whereas in sections 5.3 and 5.4 we will review many-toone and many-to-many sense matchings between english propbank and english wordnet. ## automatic vs. manual tagging although automatic corpus alignment methods are preferred over manual tagging because of their systematicity and lower cost in many lexical resource integration studies, we argue that the manual tagging method is still highly needed. in an attempt to investigate the effectiveness of automatic and manual taggings, we compared our manually-tagged lexical resource integration with the automatically-tagged pm created by lópez de @xcite @xcite ) based on propbank senses. as a result of this comparison, we found that while the matchings of 418 wordnet senses in 413 verbs in propbank and wordnet are the same in our and their integrations (see case 1 in secondly, when we look at the number of the matches assigned for each sense, we observe that the pm has matches that do not currently exist in our integration. for example, in case 4 given in thirdly, when we analyze the senses within the same verbs in the pm, we see that while some of them were annotated by taking into account their differences, some of them were merged, which resulted in the loss of some meanings. as an example, the same sense is assigned to "appeal 01" and "appeal 03" in the pm while different senses are annotated in our work as shown in case 6 in ## conclusion in this paper, we reported our comparison results of english verb sense annotations in propbank with senses in english wordnet for the sentences from penn treebank. in opposition to the idea that automatic tagging is good enough to eliminate the necessity for manual tagging, based on our comparison of our work with the pm, we contend that manual tagging is still needed to have qualitatively-better results and that it would be quite useful to apply it, at least in combination with automatic tagging. another issue that makes our work promising is its extendibility to a larger turkish dataset. related to that, ak et al. @xcite have recently constructed a turkish proposition bank using translated sentences of english propbank. so far, 9560 translated sentences are annotated with semantic roles and framesets are created for 1914 verb senses. in spite of its limited size, their study constitutes a base for turkish proposition bank. therefore, we hope that our english propbank and english wordnet parallelization study can be used to extend many larger datasets in other languages, starting with the turkish proposition bank.