Deep-learning the Ropes: Modeling Idiomaticity with Neural Networks
Authors: Yuri Bizzoni, Marco S.G. Senaldi, Alessandro Lenci (University of Gothenburg, Scuola Normale Superiore, Università di Pisa, 2017)
Topic: Distributional Semantics

Abstract
In this work we explore the possibility of training a neural network to classify and rank idiomatic expressions under constraints of data scarcity. We discuss our results comparing them both to other unsupervised models designed to perform idiom detection and to similar supervised classifiers trained to detect metaphoric bigrams.

Introduction
Figurative expressions like idioms (e.g. to learn the ropes `to learn how to do a job', to cut the mustard `to perform up to expectations', etc.) and metaphors (e.g. clean performance, that lawyer is a shark, etc.) are pervasive in language use. Important differences have been stressed between the two types of expressions from a theoretical AUTHOR, neurocognitive AUTHOR and corpus linguistic AUTHOR prespective. On the one hand, as stated by \citet{lakoff2008metaphors}, linguistic metaphors reflect an instantiation of conceptual metaphors, whereby abstract concepts in a target domain (e.g. the ruthlessness of a lawyer) are described by a rather transparent mapping to concrete examples taken from a source domain (e.g. the aggressiveness of a shark). On the other hand, although most idioms originate as metaphors AUTHOR, they have undergone a crystallization process in diachrony, whereby they now appear as fixed and non-compositional word combinations that belong to the wider class of Multiword Expressions (MWEs) AUTHOR and always exhibit lexical and morphosyntactic rigidity to some extent AUTHOR. It is anyway crucial to underline that idiomaticity itself is a multidimensional and gradient phenomenon AUTHOR with different idioms showing varying degrees of semantic transparency, formal versatility, proverbiality and affective valence.% All this variance within the class of idioms themselves has been demonstrated to affect the processing of such expressions in different ways AUTHOR. The aim of this work is to explore the fuzzy boundary between idiomatic and metaphorical expression, by applying a method designed to discriminate figurative vs. literal usages to the task of distinguishing idiomatic from compositional expressions. Our starting point is the work of \citet{Bizzoni:17}. The authors managed to classify adjective-noun pairs where the same adjectives were used both in a metaphorical and a literal sense (e.g. clean performance vs. clean floor) using a neural classifier trained on a composition of the words' embeddings AUTHOR. Actually, the neural network was able to detect the abstract/concrete semantic shift of nouns when used with the same adjective in figurative and literal compositions respectively, basically treating the noun as the ``context'' to discriminate the metaphoricity of the adjective. In our attempt, we will use a relatively similar approach to classify idiomatic expressions by training a three-layered neural network on a set of idiomatic and non-idiomatic expressions and we'll compare the performance of the network when trained on different syntactic patterns (Adjective-Noun and Verb-Noun expressions, AN and VN henceforth). Importantly, the abstract/concrete polarity the network was able to learn in AUTHOR will not be available this time, since none of the idiom constituents will ever appear in its literal sense inside the expressions, whatever their concreteness may be. What we want to find out is whether the sole information captured by the distributional vector of a single expression is sufficient to learn its potential idiomaticity. Differently from AUTHOR, for each idiom we collect a count-based vector AUTHOR of the expression as a whole, taken as a single token. We compare this approach with a model trained on the composition of the individual words of an expression, showing that the latter is less effective for idioms than for metaphors. In both cases we will be operating on scarce training sets (26 AN and 90 VN constructions). Traditional ways to deal with data scarcity in computational linguistics resort to a wide number of different features to annotate the training set (see for example AUTHOR) or rely on artificial bootstrapping of the training set AUTHOR. In our case we test the performance of our classifier on scarce data without bootstrapping the dataset and relying only on the information provided by the distributional semantic space, showing that the distribution of an expression in large corpora can provide enough information to learn idiomaticity from few examples with a satisfactory degree of accuracy.

Conclusion
It seems that the distribution of idiomatic and compositional expressions in large corpora can suffice for a supervised classifier to learn the difference between the two linguistic elements from small training sets and with a good level of accuracy. Unlike with metaphors AUTHOR, feeding the classifier with a composition of the individual words' vectors of such expressions performs quite scarcely and can be used to detect only some idioms. This takes us back to the core difference that while metaphors are more compositional and preserve a transparent source domain to target domain mapping, idioms are by and large non-compositional. Since our classifiers rely only on contextual features, their ability in classification must stem from a difference in distribution between idioms and non-idioms. A possible explanation is that while the literal expressions we selected, like vedere un film or ascoltare un discorso, tend to be used with animated subjects and thus to appear in more concrete contexts, most of our idioms (e.g. cadere dal cielo or lasciare il segno) allow for varying degrees of animacy or concreteness of the subject, and thus their context can easily get more diverse. At the same time, the drop in performance we observe in the joint models seems to indicate that the different parts of speech composing our elements entail a significant contextual difference between the two groups, which introduces a considerable amount of uncertainty in our model. It is also possible that other contextual elements we did not consider have played a role in the learning process of our models. We intend to deepen this aspect in future works.