Unsupervised Mapping of Arguments of Deverbal Nouns to Their Corresponding Verbal Labels (2023)

Abstract: Deverbal nouns are nominal forms of verbs commonly used in written English texts to describe events or actions, as well as their arguments. However, many NLP systems, and in particular pattern-based ones, neglect to handle such nominalized constructions. The solutions that do exist for handling arguments of nominalized constructions are based on semantic annotation and require semantic ontologies, making their applications restricted to a small set of nouns. We propose to adopt instead a more syntactic approach, which maps the arguments of deverbal nouns to the universal-dependency relations of the corresponding verbal construction. We present an unsupervised mechanism—based on contextualized word representations—which allows to enrich universal-dependency trees with dependency arcs denoting arguments of deverbal nouns, using the same labels as the corresponding verbal cases. By sharing the same label set as in the verbal case, patterns that were developed for verbs can be applied without modification but with high accuracy also to the nominal constructions.

## introduction systems that aim to extract and summarize information from large text collections often revolve around the concept of predicates and their arguments. such predicates are often realized as verbs (the performers interpret the music), but the same predicative concepts can also be realized as nouns (musical interpretation by the performers). this process of realizing verbal predicates as nouns is called nominalization, and it involves changing the syntactic structures around the content words participating in the construction, while keeping its semantics the same. in this work, we are interested in mapping arguments of nominal constructions that appear in text, to the corresponding ones in verbal structures (i.e., to identify the syntactic object role of music and syntactic subject role of performers, in music interpretation by the performers). nominalizations, also known as nominal predicates, are nouns derived from words of a different part of speech, such as verbs or adjectives. for example, in english foot_0 , the nominalization interpretation is derived from the verb interpret, and the nominalization precision is related to the adjective precise. the usage of nominalizations is widespread in english text, and according to @xcite , about half of all sentences in written texts contain at least one nominalization. in our work, we observed a ratio of 120k nominalizations to 180k verbs, in a random collection of 100k wikipedia sentences. thus, interpretation of nominalizations is central to many language understanding tasks. in the current work, we focus on nominalizations which are derived solely from verbs, commonly called deverbal nouns. existing attempts around identifying arguments of nominalizations either rely on a predefined semantic roles ontology (e.g., srl based roles such as those in verbnet @xcite or framenet @xcite ) as suggested by @xcite , @xcite and @xcite , or consider a limited subset of nominalized structures @xcite and @xcite ). early works approached the task in a fully supervised manner @xcite , @xcite ), hence suffering from insufficient annotated nominal data. to overcome that, @xcite and more recently zhao and titov (2020) considered a transfer scenario from verbal arguments to nominal arguments while assuming only supervised data for verbs. nevertheless, their methods were limited to specific predicates, even with extensive annotated verbal data. moreover, the previous works considered each a different set of argument types due to supervision constraints. our proposed task rather than relying on a predefined semantic roles ontology, in this work we propose to map the arguments of deverbal nouns to the syntactic arguments of the corresponding active verbal form. this allows us to define a task with a consistent and a restricted label set (syntactic subject, syntactic object, syntactic prepositional modifier with preposition x), while still maintaining expressivity: if one knows how to extract the verbal argument from the active verbal form, they will be able to also extract the nominal ones. a natural formulation is to ask "how will this verb arguments be realized in a deverbal noun construction?". however, this approach is problematic, as the same verbal structure, e.g. ibm appointed sam as manager, can be realized in many different ways around the same nominalization, including: ibm's appointment of sam as manager, sam's appointment as manger by ibm and sam's ibm appointment as manager. one solution would be to ask for all the possible nominal realizations. this is the approach taken by nominalization lexicons such as nomlex @xcite . however, this is also problematic in practice, as the different possible syntactic structures may conflict when encountering a nominalization within a sentence (ibm's appointment vs. ## deverbal nouns deverbal nouns are one type of nominalizations which are derived specifically from verbs, e.g., the deverbal noun treatment is derived from the verb treat. the events represented by deverbal nouns are described using phrases in the sentence that complement the nouns. the arguments of the deverbal noun correspond to the arguments of the matching verb; each matches a different question about the action taken. for instance, in the phrase professional treatment of illness, professional refers to the actor/subject of the verb treat (professionals), and illness refers to the object of the action treat. the deverbal nouns, as typical nouns, are most often complemented by other noun phrases (treatment of illness, his treatment and health treatment) and adjectives (professional treatment). implicit and other types of complementing arguments are not considered part of this work's scope. each deverbal noun defines a unique structure of these arguments, assigning different roles for the same typed arguments. for instance, consider the phrases time preference of the individual and individual waste of time, which match the same syntactic structure ("noun-compound of noun"). however, the first sentence matches the structure "obj noun of subj" 3 related works arguments of nominalizations were long investigated in the field of nlp. one early research explored the syntactic structure of the arguments and modeled the structure of many nominalizations, resulting in a detailed lexicon called nom-lex @xcite . the lexicon seeks to describe the allowed complements structures for a nominalization and relate the nominal complements to the arguments of the corresponding verb. following the publishing of nomlex, @xcite described how an information extraction (ie) system could exploit the linguistic information in the nomlex lexicon. yet, the suggested approach remained hardly utilized by further research, as many works only exploited the verb-noun pairs specified by the lexicon. regarding identifying and labeling nominalization's arguments, a supervised approach was suggested while considering various task settings. one preceding paper by @xcite presented a probabilistic procedure to infer whether the modifier of a nominalization (the head noun) stands in subject or object relation with it. for instance, the algorithm should predict that the modifier's role in the phrase child behavior is subject since the phrase refers to the child as the agent of the action described by the verb behave. stated differently, this procedure focuses on extracting only one specific argument of nominalizations in a noun phrase. another distinguished paper by @xcite considered framenet-based @xcite semantic arguments of nominalizations and applied a machine learning framework for eventive nominalizations in english and chinese, aiming to identify and label their arguments. finally, @xcite published a similar approach for nominalizations used in biomedical text. some related works acknowledge the shortage of labeled argument nominalizations and suggest unsupervised methods for data expansion based on labeled argument verbs. similarly to ours, these works exploited the similarity and alignment of the noun-verb arguments. for example, @xcite and zhao and titov (2020) considered the argument labeling task for nominalizations in a setup where the verbal sentences are human labeled, and with regards to semantic role labeling (srl) arguments. @xcite exploited the similarities between the argument structure of event nominalizations and corresponding verbs while utilizing common syntactic features and distributionalsemantic similarities. more recently, zhao and titov (2020) suggested a variational auto-encoder method, in which the labeler serves as an encoder, whereas the decoder generates the selectional preferences of the arguments for the predicted roles. a different approach taken by @xcite using a fully unsupervised manner while automatically extracting and labeling verbal arguments of verbs from a large parsed corpus of wikipedia. this approach resembles an intermediate stage of ours yet differs as it considers a reduced set of argument types (subject and object) and a reduced possible set of argument syntax for the nominalizations (possessive and 'of' arguments). lately, @xcite engaged with a different task with similar applications. they suggested an unsupervised method for paraphrasing clauses with nominalizations into active verbal clauses. ## task definition as discussed in the introduction, we define a task of labeling the arguments of deverbal nouns within a sentence, with labels of the arguments in the corresponding active verb constructions. here we provide a more complete and formal definition. while our aim is to label all of the deverbal nouns in a given corpus, here we focus on describing the task with relation to a single instance of a sentence and a deverbal noun within it. we consider the syntactic arguments of active verbal forms to belong to the set l consisting of the universal dependency relations nsubj, dobj and nmod:x, where x is a preposition (e.g., nmod:in, nmod:on, nmod:with). in words, the syntactic subject, syntactic object, and arguments attached as prepositional phrases where the identity of the preposition is part of the relation. while these prepositions may correspond to many different semantic roles, for a given verb they usually indicate a concrete and unique role. formally, given a sentence with words w 1 , . . . , w n , and a marked deverbal noun within the sentence (say in position w i ), we seek to find k pairs of the form (rel k , w j k ), 1 ≤ @xmath0, where rel @xmath1, dobj, nmod:x} and w j k is a word in the sentence (j k is an index of a sentence word). for simplicity, we also demand that every relation type cannot be repeated more than once in the identified set of pairs. these pairs indicate arguments of the deverbal noun and their relations to it, expressed using an active-verb label set. in figure 1 , the blue edges of the bottom tree indicate the output (nsubj, 1), (dobj, 6). note that the task includes both the identification of the arguments and their label assignment. ## methodology while we intend to handle all deverbal nouns in a given collection of sentences, here we focus on how to resolve a single deverbal noun. we identify deverbal nouns and their corresponding verbal forms based on a given lexicon of verb-noun pairs, which we consider as input. in this work, we use the nomlex lexicon @xcite , where future work can also replace this with a learned model. given a deverbal noun within a sentence, we first identify its potential arguments. this is realized by searching a set of syntactic relations in the corresponding universal dependency tree (we use the udv1 parser trained by @xcite via the spacy toolkit foot_2 ). we then label the arguments by comparing their contextualized word embeddings to those of the corresponding verb arguments, in a set of sentences containing this verb (we further motivate this comparison in appendix a). finally, based upon the labeled arguments, we construct the final output as pairs of the arguments' label (i.e. verbal ud relation) and the arguments' head word. ## evaluation data our task is to identify arguments of deverbal nouns and assign each one of them a label from the set @xmath2, dobj, nmod:x}. for evaluation, we need sentences with deverbal nouns whose arguments are labeled with these relations. for example, the deverbal noun relocation in the phrase family relocation to manchester should be labeled with the pairs (nsubj, 1) and (nmod:to, 4), as specified in section 4. we create three such evaluation datasets, the first based on a nominalization paraphrasing dataset, and the other two are based on the nomlex lexicon, while they differ by the coverage of deverbal nouns that they consider, as we further explain. moreover, to compare our method's performance to earlier works, we consider the conll-2009 dataset @xcite for evaluation, as we discuss in 7.3. the paraphrasing-derived evaluation set is derived from a manually annotated dataset for the task of paraphrasing sentences from nominal to verbal form @xcite . the original dataset includes a collection of 449 samples from 369 unique sentences representing 142 different verbs. each sample represents a paraphrasing between the original nominalization phrase (from a given sentence) and a verbal clausal phrase, for instance genetic analysis from a sample which is paraphrased as analyze genes from a sample. for every paraphrasing sample, the dataset specifies the components of the nominal phrase within the structure "adj/noun nominalization prep pobj", and the components of the active verbal phrase ("arg0 verb arg1 pp"). to construct our evaluation set based on this data, we first match each of the nominal components adj/noun and pobj with a verbal component from the set of arg0, arg1 and pp, choosing the one with the closest orthography to the nominal one. from this, we derive the verbal argument labeling for the components of the nominal phrase. then, we replace each verbal label with its matching ud relation. 6 finally, for every nominal component we determine its head word position in the given context. the word positions paired with the matching verbal relations, construct a sample in our new paraphrasing-derived evaluation set. in the course of dataset construction, we filter out some data samples. to start with, data samples that specify two nominal components that match the same verbal component were removed from our dataset, as they do not fit the constraints of the defined task. for example, in the phrase environmental assessment for the project the combined components of the noun can be understood together as the object of the matching verb (assess the environmental impact of the project), hence resulting with two nominal arguments labeled with the same verbal relation. secondly, we consider only the first single data sample for every repeated nominal phrase to ensure a single truth of labeling for every nominal phrase. following the filtering process we remain with 309 samples with 122 different verbs. the nomlex evaluation sets are constructed using the nomlex lexicon. 7 the nomlex lexicon contains a list of about 4k deverbal nouns, and for each of them specifies the various ways in which their arguments can be realized syntactically, and how they map to the corresponding verbal arguments. for example, an adapted nomlex entry for a deverbal noun like destruction would specify the related forms of the noun (i.e., the verb and other related deverbal nouns) and, most significantly, a set of dependency-tree patterns corresponding to several different realizations of the noun. each dependency-tree pattern represents a set of labeled arguments in a specific dependency tree. for instance, the entry of destruction would contain a pattern that corresponds to the dependency structure shown in the middle of figure 1 and demands the labeling of rome as subject and city as object. hence, using a parsed dependency tree of a sentence with a deverbal noun, we can extract the labeled arguments in the sentence for any specified pattern that fulfills the sentence's dependency structure. however, this method does not allow for a definitive decision in many cases, as the lexicon often contains multiple labeled contradicting patterns. in section 7 we show that relying solely on nomlex results in a significantly lower precision. we collect english wikipedia sentences from @xcite that contain a deverbal noun, and for each sentence, we identify the deverbal noun's arguments and labels based on the adapted nomlex entry as described above. we discard sentences for which the entry suggests two or more different assignments, when matching two or more dependency patterns. we then map nomlex's labels into the corresponding dependency relations of the active verbal form. to match the examples in the paraphrasing dataset, we consider only data samples with two labeled arguments each. we divide the collected samples into two evaluation sets based on the verbal form of the represented deverbal nouns. nomlex paraphrasing considers only samples which refer to verbs that appeared in the paraphrasing-derived corpus, whereas nomlex other considers samples that match 315 other verbs. in each evaluation set, we keep 25 labeled sentences for each verb. tune/test split our method is unsupervised but still requires tuning of hyperparameters. we keep a tuning subset for each origin of the evaluation set (paraphrasing-derived and nomlex), which is also used for evaluation during development. in the paraphrasing dataset, we sample 20% of the dataset to construct the tuning set while keeping aside 80% of the dataset for evaluation. out of the 122 verbs in the paraphrasing-derived evaluation set, 12 appear only in the tuning set, 83 only in the test set, and 27 appear in both sets. the split aims to ensure that the results are not verb-specific and to prevent overfitting, as we do hyperparameter optimization on the tuning set, which does not contain all the verbs that appear in the test set. to tune the method for nomlex-based data, we perform a similar tune-test split on nomlex paraphrasing based upon the same tune-test verb division made for the paraphrasing evaluation set. concretely, nomlex instances of the 12 tuning-only verbs and 83 test-only verbs were included only in the nomlex tuning set and test set, correspondingly; instances of the 27 common verbs were divided into the tune-test sets in a 20%-80% ratio. moreover, we preserve entirely nomlex other corpus for testing. evaluation metrics we use two evaluation metrics: relation-f1 is the f1 score of all the predicted word-relation pairs compared to the gold labeled pairs (without distinguishing argument labels, for comparability with zhao and titov (2020) which uses conll-2009 evaluation scorer @xcite ). exact-match scores how many noun instances had all their relations identified and labeled correctly. a predicted relation is considered correct if it matches both the same argument head word and the same label as the gold relation. ## experiments and results in this section, we consider the results of our method on the evaluation sets and experiments we conducted concerning the two stages of our method. the setup which produced the best results is discussed in 7.2, including the chosen hyperparameters, which were tuned over the tuning sets. baseline as a baseline for our approach, we considered the same process we used for generating the nomlex evaluation sets. more specifically, for a given parsed sentence with a given deverbal noun, our baseline method attempts to match the deverbal noun instance with all dependency patterns in appropriate entry within the adapted nomlex lexicon. every fulfilled pattern should result in a set of labeled arguments. the combined set of noncolliding arguments, i.e., arguments that match a single argument type, are then mapped into pairs of headwords and ud relations, which are also the output of the baseline method. ## conclusions in this work, we formulate the task of aligning arguments of deverbal nouns to the arguments of their corresponding active verbal form. we formulate the task as a ud enrichment task, aiming to enrich deverbal nouns in text with verbal ud relations for the matching nominal arguments. our formulation, compared to the ones suggested in previous works, does not rely on a predefined roles ontology. we suggest an unsupervised approach to this nominal-to-verbal argument mapping based on pretrained contextualized word representations. our method tries to match nominal identified arguments with automatically extracted arguments of the corresponding verb. the suggested method outperforms the nomlex-based baseline, which is based on an expertly constructed comprehensive lexicon. we also show the importance of contextualization, experiencing a 25% decrease in performance when using uncontextualized vectors. moreover, we further validate our hypothesis that semantics and syntactic structure are captured in the considered word representations using a dedicated experiment. we provide a standalone code for enriching universal dependency trees with nominal arguments for a given parsed corpus, which can be integrated into nlp systems that use universal dependency patterns as part of their design or features.