Using W ord N et to Build Lexical Sets for I talian Verbs (2016)

Abstract: We present a methodology for building lexical sets for argument slots of Italian verbs. We start from an inventory of semantically typed Italian verb frames and through a mapping to WordNet we automatically annotate the sets of fillers for the argument positions in a corpus of sentences. We evaluate both a baseline algorithm and a syntax driven algorithm and show that the latter performs significantly better in terms of precision.

## introduction in this paper we present a methodology for building lexical sets for argument slots of italian verbs. lexical sets @xcite are paradigmatic sets of words which occupy the same argument positions for a verb, as found in a corpus. for example, for the verb read, the following set can be built by observing the lexical fillers of the object position in the bnc corpus: a relevant step in our methodology is the annotation of the lexical items for argument positions in sentences. a previous work @xcite has already outlined an annotation scheme for this purpose, and highlighted its benefits for nlp applications. in that work, however, the annotation of lexical sets was intended as manual, whereas the methodology we propose here is conceived for automatic annotation, and exploits an existing external resource. under this perspective our work is related to semantic role labeling @xcite . this paper is organized as follows. section 2 introduces the t-pas resource; in section 3 the lexical set population task is defined, and in section 4 the experimental setting is presented. section 5 discusses the results and is followed by the error analysis in section 6. finally, section 7 provides some conclusions and directions for future work. ## overview of the t-pas resource t-pas, typed predicate argument structures, is a repository of verb patterns acquired from corpora by manual clustering of distributional information about italian verbs @xcite . the resource has been developed following the lexicographic procedure called corpus pattern analysis, cpa @xcite . in particular, in the resource t-pass are semantically motivated and are identified by analysing examples found in a corpus of sentences, i.e. a reduced version of itwac @xcite . after analyzing a sample of 250 concordances of the verb in the corpus, the lexicographer defines each t-pas recognising its relevant structure and identifying the semantic types (sts) for each argument slots by generalizing over the lexical sets observed in the concordances; as an example, figure1 shows the t-pas#2 of the verb divorare: [[human]] divorare [[document]] (eng. to devour), where @xcite ] stands for {libro, romanzo, saggio} (eng. {book, newspaper, essay}) (figure 2 ). sts are chosen among a list of about 230 corpus-derived semantic classes compiled by applying the cpa procedure to the analysis of concordances for about 1500 english and italian verbs @xcite foot_0 . if no generalization is possible, the lexical set is listed. finally, the lexicographer associates the instances in the corpus to the corresponding t-pas and adds a free-text description of its sense (figure 1 ). the t-pas resource thus lists the analyzed verbs 2 , the identified t-pass for each verb, the annotated instances for the t-pas in the corpus. in the next sections, we will define the lexical set population task and describe the experiment we ran and its evaluation. ## task definition the aim of our system is to automatically derive lexical sets corresponding to the sts in the t-pas resource. the task is defined as follows. the system receives as input (i) a t-pas of a certain verb and (ii) a sentence associated to that t-pas in the resource. the system should correctly mark (where present) the lexical items or the multiword expressions correspondent to the sts of each argument position specified by the t-pas (i.e. sentence annotation step). by replicating this annotation for all the sentences of a t-pas, the system will build the lexical set for a specific st in a specific t-pas (i.e. lexical set population step). for instance, example (2) shows the t-pas#1 of the verb preparare (eng. to prepare) and a sentence associated to it. ## experimental setting in order to identify possible candidate items for a st, the system uses information from multiword-net @xcite (from now on mwn); e.g. to derive that "grandmother" is a human being and associate it to the st @xcite ] and that "cake" is a type of food and associate it to the st @xcite ]. the task, thus, required an initial mapping between the t-pas resource and mwn. then, we compared a naive baseline algorithm and a more elaborated algorithm that we called lea, lexical set extraction algorithm. finally, to evaluate the performance of our methodology we also created a gold standard. st to synset mapping. for our experiment, the list of sts used in the t-pas resource was automatically mapped onto corresponding wordnet 1.6 synsets. for instance, the st @xcite ] was mapped to all the synsets for the noun human (i.e. human#n). manual inspection was limited to the case in which there is no exact match between a st and a synset (e.g. by associating "atmosphericphenomenon" to @xcite ]). the baseline algorithm. the baseline algorithm identifies possible candidate members of the lexical set corresponding to a certain st for a certain t-pas by (i) lemmatizing each sentence using textpro @xcite , (ii) checking if each lemma is in mwn and (iii) determining whether the lemma belongs to a synset that was mapped to the st, or if it is an hyponym of one such synsets. for instance, in example (2), the baseline lemmatizes the sentence and selects as possible candidates the nouns of the sentence, i.e. nonna, torta and patate. the italian lemma nonna is thus searched in mwn and the correspondent english lemmas grandma#n#1, grandmother#n#1, granny#n#1, grannie#n#1 are found. since none of these synset lemmas match with @xcite ], @xcite ] or @xcite ], the mwn hierarchy is traversed until human#n#1 is found, which is mapped to @xcite ]. the same is done for torta and patate, until [[food]] is found. thus, for (2), the baseline identifies nonna as @xcite ] and torta and patate as @xcite ] (with patate being a misclassified item, as it is not referred to the verb preparare). the lea algorithm. compared to the baseline algorithm, the lea algorithm takes into account also the dependency tree of the sentence, named entities as recognized by textpro, and multiword expressions. it starts by (i) finding the position of the verb in an example and considering as valid candidate only the chunks that are a subject, direct object or complement of the verb according to the textpro dependency tree. with respect to the baseline, this leads to a more precise identification of the items for the argument slots of just the verb we are considering. for instance, in (2) we expect the algorithm to correctly identify nonna as @xcite ] and torta as @xcite ], but not proposing patate (as the baseline does). the lea algorithm also (ii) checks if the verb allows the same st for subject and object, as in the t-pas#3 of pettinare: [[human1]] pettinare [[hu-man2]] (eng. to comb someone's hair). in the sentence "la mamma pettina il bambino" (eng. the mum combs the baby), lea will correctly propose mamma as @xcite ] and baby as @xcite ]. in this case, it also checks if the verb is in passive form and swaps the items for subject and object position as needed, improving the precision with respect to the baseline. furthermore, the algorithm (iii) checks if the chunk contains/overlaps with proper names related to persons, organizations and locations detected by textpro, and, if this is the case, checks the corresponding type of named entity against the st allowed by the t-pas frame (e.g. maria @xmath0. since the baseline recognizes only named entities that are in mwn, we expect this algorithm to identify more items. finally, lea (iv) looks for multiword expressions in a chunk by checking if the combination exists in mwn. for instance, in "la nonna prepara la conserva di frutta" (eng.: the grandmother prepares the fruit conserve), lea should identify conserva di frutta as @xcite ] (while the baseline identifies only the token frutta). the lea algorithm, thus, should recognize as valid only the items for a certain argument slot of the analyzed verb (and not for other verbs in the sentence), solve major cases of same st in different slots and identify named entities and multiword expressions. gold standard. we created a gold standard for the task by manually annotating 500 examples. we asked three annotators to mark the lexical items or the multiword expressions that correspond to the sts, without annotating pronouns or relative clauses. we selected the 500 sentences by extracting 10 sentences for 10 different sts in 5 different t-pass (for a total of 50 different t-pass belonging to 47 verbs). in particular, we chose, among all the sts within the [[inanimate]] hierarchy, 10 types that are used in at least 5 different t-pas, each of them having at least 10 (potential) sentences associated in the corpus resource. for example, we selected @xcite ] and annotated 10 sentences for t-pas#1 of mangiare " @xcite ] mangiare @xcite ]" (eng. to eat), since (i) there are at least 5 verbs with a t-pas containing @xcite ], like mangiare itself and (ii) we have at least 10 sentences available for each of these five t-pass foot_2 . this selection of few sts was intended to better compare performances of the algorithms for different lexical sets. the gold standard annotation resulted in a total of 981 annotated tokens out of 15090 (the average sentence length being 30.18 tokens). ## results for what concerns sentence annotation, we evaluate overall precision, recall and f-measure, con-sidering as a positive match when the algorithms agree with the gold standard in recognizing a token as an item (or part of the item in case of multiword expressions) instantiating a st for a precise position. compared to the baseline, the lea algorithm registers a significant higher value for precision (see automatic mapping in ## error analysis the results presented in the first part of even when the automatic mapping works, the different structure of the two resources can lead to wrong results. for instance, vehicles such as elicottero (eng.: helicopter) are frequently generalized by the st [[vehicle]] in t-pas and are hyponyms of vehicle#n in mwn. however, while in t-pas @xcite ] is a hypernym of @xcite ], the same is not true for machine#n in mwn. as a consequence, in the sentences in which vehicles are considered members of the lexical set correspondent to @xcite ], even traversing the mwn hierarchy, the algorithms can not consider these items as valid candidates for the st @xcite ]. to solve at least some of these problems, we manually inspected the 40 sts of the sentences of the gold standard, and modified the automatic mapping of 11 of those; for example, we chose to translate the st @xcite ] to construction#n, and mapped @xcite ] to both transport#n and machine#n. this led to a significant improvement of the recall for both algorithms, and a minor improvement of the precision, as shown in this improvement is also reflected on the second part of the task (i.e. the creation of the lexical set). for example, the dice value for crollare#1-sbj-[[building]] improves from 0.4 to 0.71 for the baseline and from 0.25 to 0.6 for lea. another significant aspect concerns the recognition of proper names: out of the 185 tokens that are -or are part of-proper nouns (137 are related to persons, locations or organizations), the baseline recognized correctly only 10 (mainly common nouns that are used as proper names), while the lea algorithm only 26. finally, some errors are introduced in the pos tagging and dependency parsing steps. during the former, an incorrect tag can be assigned to a word (e.g. a noun could be mis-tagged as an adjective) and hinder both algorithms, as the word would not be checked in mwn. the latter only undermines the recall of the lea algorithm instead. moreover, lea does not deal with complex syntactic structure yet (e.g. when our verb is in an infinitive phrase, which is the object of a main verb, such as "[..] e il presidente chiede agli italiani di ipotecare la casa [..]", eng.: [..] and the president asks italians to mortgage their houses [..]). ## conclusion and further work in this paper we have presented an experiment for the automatic building of lexical sets for argument positions of the italian verbs in the t-pas resource. the method is based on the use of mwn in order to match the sts with the potential fillers of each argument position. the experiment suggests that lea can be used to automatically populate the lexical sets with good precision. we believe that significantly better results could be obtained with an accurate manual mapping of the sts to synsets, possibly narrowed to specific senses (e.g. mapping @xcite ] to just the third sense of construction#n). furthermore, recognizing proper nouns proved a difficult task, and even using named entities recognition in addition to mwn was not enough. therefore a resource to map these nouns to a synset in the wordnet hierarchy is needed; ba-belnet @xcite could prove useful in this sense. further work includes the extension of the sentence annotation and lexical set population for all t-pas and the comparison of the same st in different t-pass in order to study italian verbs' selectional preferences from the perspective of verb selectional classes (for example, all verbs that se-lect @xcite ] as object).