Supervised Disambiguation of G erman Verbal Idioms with a B i LSTM Architecture (2020)

Abstract: Supervised disambiguation of verbal idioms (VID) poses special demands on the quality and quantity of the annotated data used for learning and evaluation. In this paper, we present a new VID corpus for German and perform a series of VID disambiguation experiments on it. Our best classifier, based on a neural architecture, yields an error reduction across VIDs of 57% in terms of accuracy compared to a simple majority baseline.

## introduction figurative language is not just a momentary product of creativity and associative processes, but a vast number of metaphors, metonyms, etc. have become conventionalized and are part of every speaker's lexicon. still, in most cases, they can simultaneously be understood in a non-figurative, literal way, however implausible this reading might be. take, for example, the following sentence: he is in the bathroom and talks to huey on the big white telephone. the verbal phrase talk to huey on the big white telephone can be understood as a figurative euphemism for being physically sick. but it could also be taken literally to describe an act of remote communication with a person called huey. despite the ambiguity, a speaker of english will most probably choose the figurative reading in (1), also because of the presence of certain syntactic cues such as the adjective sequence big white or the use of telephone instead of, for example, mobile. omitting such cues generally makes the reader more hesitant at selecting the figurative meaning. there is thus a strong connection of non-literal meaning and properties pertaining to the form of the expression, which is characterstic for what @xcite call an idiom. since the figurative expression in (1) consists of a verb and its syntactic arguments, we will furthermore call it a verbal idiom (vid) adapting the terminology in @xcite . while it is safe to assume that the vid talk to huey on the big white telephone almost never occurs with a literal reading, this does not hold for all idioms. the expression break the ice for example can easily convey both a literal (the trawler broke the ice) and a non-literal meaning (the welcome speech broke the ice) depending on the subject. although recent work suggests that literal occurrences of vids generally are quite rare in comparison to the idiomatic ones @xcite , it remains a qualitatively major problem with the risk of serious errors due to wrong disambiguation. however, tackling this problem with supervised learning poses special demands on the learning and test data in order to be successful. most importantly, since the semantic and morphosyntactic properties of vid types (and idioms in general) are very diverse and idiosyncratic, the data must contain a sufficient number of tokens of both the literal and non-literal readings for each vid. in addition, each token should allow access to the context because the context can provide important hints as to the intended reading. in this paper, we investigate the supervised disambiguation of potential occurrences of german vids. for training and evaluation, we have created colf-vid (corpus of literal and figurative readings of verbal idioms), a german annotated corpus of literal and semantically idiomatic occurrences of 34 preselected vid types. altogether, we have collected 6985 sentences with candidate occurrences that have been semantically annotated by three annotators with high inter-annotator agreement. the annotations overall show a relatively low idiomaticity rate of 77.55 %, while the idiomaticity rates of the single vids vary greatly. the derived corpus is made available under the creative commons attribution-sharealike 4.0 international license. 1 to the best of our knowledge, it represents the largest available collection of german vids annotated on token-level. furthermore, we report on disambiguation experiments using colf-vid in order to establish a first baseline on this corpus. these experiments use a neural architecture with different pretrained word representations as inputs. compared to a simple majority baseline, the best classifier yields an error reduction across vids of 57% in terms of accuracy. ## annotation results the annotation was performed by three trained linguists on the whole dataset. the annotation results are summarized in what immediately stands out is that the overall idiomaticity rate is not nearly as high as the 98% reported for the german parseme dataset mentioned in section 3.1 it ranges from 19.44% (im blut haben 'be in one's blood') to 99.65% foot_7 (den nerv treffen) and is 77.55% in total. but one has to keep in mind that these two datasets are hardly comparable regarding their statistics, since colf-vid was created with the intention to maximize the number of literal occurrences by only choosing vid types with a presumably high literality count. even though there are some vid types with an unexpectedly high idiomaticity rate (auf der strecke bleiben, in eine sackgasse geraten or über die bühne gehen to name a few), the large majority of the chosen vid types is indeed represented with a relatively low idiomaticity rate. only 0.59 of the instances received the labels undecidable or both (see figure 2 ), but this is hardly surprising. we nevertheless wanted to include these tags for the sake of completeness and linguistic interest. for the three annotators we calculated the following cohen's kappa scores on the basis of the whole dataset: another feature of colf-vid is the context judgement provided by two of the annotators. these judgements can be seen in figure 1 in the last two lines (starting with a hash tag) before the beginning of the sentence. they indicate whether the annotators needed more than one sentence to determine the reading of an instance. the two zeros denote that this was not the case for this candidate expression ("1" would indicate the opposite). even if the sentence is rather short with only five words, the fact that the pulling of an emergency break requires an animate agent if used literally was enough information for both annotators to make their decisions. the context judgement feature provides the possibility of excluding candidates where none of the annotators was able to determine the reading only from a single sentence. as a result, instances where one sentence is not sufficient to make an informed decision would be prevented from entering a given system (e.g. a classifier which aims to disambiguate the candidates). ## conclusion/future work in this paper we presented colf-vid, a new corpus with annotated instances of german vids and their literal counterparts. furthermore, we experimented with vid disambiguation on the new corpus and showed that significant improvements can be gained from applying a neural architecture in comparison with a simple majority baseline. the experiments additionally demonstrated the effects of the different word representations on the resulting performance. for the future we plan on extending the annotation of colf-vid with those vids that were not in the set of pre-chosen expressions and con-sequently were not annotated. this would allow to use the corpus as a basis for an identification task and not just disambiguation. concerning the disambiguation task itself, a cornucopia of different approaches -be it supervised or unsupervised -can be imagined. we plan on conducting a survey of different approaches in an attempt to reveal which architectures, context sizes and features are best suited for the task. last but not least, crosslinguistic experiments with comparable corpora (e.g. idix) could be interesting in order to explore language-specific properties of vids.