ID 10 M : Idiom Identification in 10 Languages (2022)

Abstract: Idioms are phrases which present a figurative meaning that cannot be (completely) derived by looking at the meaning of their individual components. Identifying and understanding idioms in context is a crucial goal and a key challenge in a wide range of Natural Language Understanding tasks. Although efforts have been undertaken in this direction, the automatic identification and understanding of idioms is still a largely under-investigated area, especially when operating in a multilingual scenario. In this paper, we address such limitations and put forward several new contributions: we propose a novel multilingual Transformer-based system for the identification of idioms; we produce a high-quality automatically-created training dataset in 10 languages, along with a novel manually-curated evaluation benchmark; finally, we carry out a thorough performance analysis and release our evaluation suite at https://github.com/Babelscape/ID10M .

## introduction idioms pertain to a wider family of linguistic phenomena referred to as multi-word expressions (mwes). broadly speaking, an mwe can be defined as a combination of two or more words, behaving as a complex lexical unit and showing idiosyncratic properties @xcite . over the course of the last few years, several attempts have been made to classify mwes based on specific dimensions such as polylexicality, fixedness, compositionality and idiomaticity @xcite . according to @xcite , mwes can be divided into lexicalized and institutionalized phrases. while the former show syntactic or semantic idiosyncrasies, e.g. kingdom come and spill the beans, the latter are compositional from a syntactic and semantic perspective, but statistically idiosyncratic, e.g. traffic light and telephone booth. among lexicalized phrases, idioms are of particular interest in that their meaning cannot be obtained by compositionally interpreting their word constituents. these include non-compositional phrases, e.g. kick the bucket, and partiallycompositional phrases, e.g. rain cats and dogs @xcite . given their complex nature, idioms are hard to be automatically identified and pose a crucial challenge to the entire field of natural language understanding @xcite . although research in this field has recently achieved great advancements, the current formulation of many tasks tends to overlook the idiomatic usage of language. instead, idioms ought to be playing an important role in nlu as they are a frequent phenomenon which can be observed in all languages. the correct identification of idioms in context is crucial for tasks such as word sense disambiguation @xcite and entity linking @xcite , but also for many downstream applications. for instance, in question answering or dialog, a system must be able to understand "it was a piece of cake" in relation to the question "how was the test?" @xcite . similarly, if the idiom kick the bucket is identified, then a text summarization system would be able to summarize all its occurrences within a text with "die" @xcite . finally, once an idiom is identified, a machine translation system would then be able to avoid its compositional translation, and treat it as a whole @xcite . furthermore, idioms are widely studied in linguistics and psycholinguistics @xcite @xcite @xcite , hence a system capable of effectively identifying idioms in texts would significantly improve many research areas, far beyond nlu. most of the past idiom extraction strategies focused on specific domains and on a limited number of languages. in our work, we tackle these shortcomings and, taking inspiration from the named entity recognition (ner) task (yadav and bethard, 2018), we reformulate the identification of idioms as a sequence labeling task. specifically, we propose the following new contributions: we hope that this work will provide a stepping stone for further studies regarding idiomatic expressions and their applications, and encourage further work on the identification of idioms in multiple languages. we release the produced datasets and software at https://github.com/ babelscape/id10m . ## related work systems over the course of the past two decades, several approaches have been put forward to address the idiom identification task. to this end, two main properties of idioms have been leveraged, namely their syntactic and semantic idiosyncrasies. while the former refers to the peculiar syntactic behaviour of idioms, the latter indicates the linguistic property in which the meaning of an idiomatic expression cannot be completely derived from the meaning of its individual components. initial studies regarding idiom identification focused on syntactic idiosyncrasy, concentrating on verb/noun idioms, e.g. shoot the breeze @xcite @xcite , on verb/particle idioms, e.g. call off @xcite or on idioms satisfying specific restrictions, i.e. subject/verb such as tension mounted and verb/direct-object, e.g. break the ice @xcite . subsequent approaches exploited semantic idiosyncrasies. this property implies that idiomatic expressions often occur in contexts typically unrelated to the meaning of their individual constituents, thus providing a key feature to be exploited in an automatic approach. in particular, @xcite introduced new lexical and graph-based features that use wordnet foot_0 and wiktionary foot_1 , and proposed a simple yet efficient binary perceptron classifier to distinguish between idiomatic and non-idiomatic expressions by exploiting their components and dictionary definitions. a similar, but unsupervised approach was adopted by @xcite which relied on the dictionary definitions of each component of a given idiom. these latter methods have more recently been superseded by approaches making use of distributional similarity in the form of both static and contextualized word embeddings @xcite @xcite @xcite @xcite , while keeping the underlying assumption unchanged: the vector representation of the component words should be distant from the vector representation of the context or of the expression as a whole. notwithstanding the recent improvements, to the best of our knowledge, the identification of idiomatic expressions in multiple languages is largely under-investigated. ## id10m in what follows, we first describe the creation process of our training datasets (section 3.1) and the manually-curated test sets (section 3.2). then, we introduce our new task formulation and illustrate the architecture of our idiom identification system (section 3.3). ## experiments in this section, we describe our experimental setup (section 4.1), the datasets we use to train and evaluate our idiom identification system (section 4.2), and the results obtained (section 4.3). span, in this case an idiomatic expression, the i label denotes an intermediate token of a span, and o means out of a span. ## qualitative analysis together with the quantitative evaluation provided in section 4.3, we now perform a qualitative analysis of our system. more specifically, in on the other hand, we observe that the system is able to correctly identify both lemmatized and inflected forms of idiomatic expressions, for both seen and unseen ones. ## conclusions and future work in this work, we introduced id10m, an innovative framework for idiom identification consisting of i) a new multilingual transformer-based architecture, ii) a novel automatic annotation pipeline for creating high-quality silver-data in 10 languages, and iii) a challenging manually-curated benchmark in 4 languages. moreover, while the majority of current approaches to idiom identification need preidentified potentially idiomatic expressions, we, instead, dropped this requirement and proposed a new formulation for the idiom identification task that lets systems be directly applicable to raw texts. finally, our experiments showed that our system is able to generalize beyond idioms seen during training, hence achieving up to 85.4 macro f1-score on the idiom identification task. as future work, we plan to scale our system to a greater number of languages and textual sources, but, most importantly, investigate the benefits derived from our work in key tasks such as word sense disambiguation, machine translation and question answering. vikas yadav and steven bethard. 2018. a survey on recent advances in named entity recognition from deep learning models. in proceedings of the 27th international conference on computational linguistics, pages 2145-2158, santa fe, new mexico, usa. association for computational linguistics.