NER 4 ID at S em E val-2022 Task 2: Named Entity Recognition for Idiomaticity Detection (2022)

Abstract: Idioms are lexically-complex phrases whose meaning cannot be derived by compositionally interpreting their components. Although the automatic identification and understanding of idioms is essential for a wide range of Natural Language Understanding tasks, they are still largely under-investigated. This motivated the organization of the SemEval-2022 Task 2, which is divided into two multilingual subtasks: one about idiomaticity detection, and the other about sentence embeddings. In this work, we focus on the first subtask and propose a Transformer-based dual-encoder architecture to compute the semantic similarity between a potentially-idiomatic expression and its context and, based on this, predict idiomaticity. Then, we show how and to what extent Named Entity Recognition can be exploited to reduce the degree of confusion of idiom identification systems and, therefore, improve performance. Our model achieves 92.1 F1 in the one-shot setting and shows strong robustness towards unseen idioms achieving 77.4 F1 in the zero-shot setting. We release our code at https://github.com/Babelscape/ner4id .

## introduction one of the main challenges in natural language processing (nlp) is to embed the meaning of a piece of raw text (e.g. a word or a sentence) in a low-dimensional dense vector. with the advent of pretrained language models, which exploit contextual information and assume compositionality of word representations, significant improvements have been made in this direction @xcite . on the other hand, very little attention has been paid to idiomatic expressions, i.e. multi-word expressions (mwes) with an established meaning unrelated to the meanings of the individual constituents. however, since idiomaticity is a frequent phenomenon that can be observed in all languages, idiomatic expressions should play an important role in nlp. indeed, their identification and understanding is crucial not only for natural language understanding tasks such as word sense disambiguation @xcite , semantic role labeling @xcite and semantic parsing @xcite , but also for machine translation @xcite , question answering @xcite and text summarization @xcite , inter alia. in the semeval-2022 task 2: multilingual idiomaticity detection and sentence embedding @xcite , research on idioms has been promoted by adapting datasets and tasks from the work carried out by tayyar @xcite . specifically, the organizers propose two subtasks: both subtasks cover three languages: english, portuguese and galician foot_0 . in addition, the organizers provide strong baseline systems to compare with. in this paper, we present the ner4id submission to the semeval-2022 task 2 which focuses on subtask a. specifically, we successfully tackle the idiom identification task by introducing a twostep system that: i) uses named entity recognition (ner) to pre-identify non-idiomatic expressions, and ii) exploits a novel transformer-based dual-encoder architecture to compute the semantic similarities between the remaining potentiallyidiomatic expressions and their contexts and, based on these, predict idiomaticity. finally, we extensively evaluate our system on both one-shot and zero-shot settings. we release our code at https: //github.com/babelscape/ner4id . ## related work approaches to idiom identification were initially built on the notion that idiomatic expressions, like other mwes, are less syntactically and lexically flexible than non-idiomatic and compositional ones. indeed, initial studies focused on specific syntactic constructions. @xcite focused on verb/noun idioms, e.g. shoot the breeze, and used the pointwise mutual information (pmi, @xcite measure to quantify the degree of lexical, syntactic, and overall fixedness of a given verb+noun combination. @xcite and @xcite also focused on verb/noun idioms using similar strategies. other studies, instead, focused on verb/particle idioms, e.g. call off @xcite , or on idioms satisfying specific restrictions, i.e. subject/verb, such as tension mounted, and verb/direct-object, e.g. break the ice @xcite . the following generation of approaches exploited semantic idiosyncrasy, i.e. the linguistic property in which the meaning of an idiomatic expression cannot be completely derived from the meaning of its individual constituents. this property causes idioms to appear in contexts typically unrelated to the meaning of their individual components, hence it provides a key aspect to be exploited in an automatic approach. in particular, @xcite introduced new lexical and graph-based features that use wordnet foot_1 and wiktionary foot_2 , and proposed a simple yet efficient binary perceptron classifier to distinguish idiomatic and literal expressions by exploiting their components and dictionary definitions. a similar, but unsupervised approach that relied on the dictionary definitions of each component of a given idiom was adopted by @xcite . finally, these latter methods have been superseded by approaches making use of distributional similarity in the form of both static and contextualized word embeddings @xcite @xcite @xcite @xcite @xcite , while keeping the underlying assumption unchanged, that is, the vector representation of the component words should be distant from the vector representation of the context, or of the expression as a whole. although efforts have been made in this direction, most of the studies to date have focused on the english language. additionally, the low performance of current idiomaticity detection systems makes them not very reliable, and therefore such systems tend not to be included in downstream applications. in this work, instead, we propose a highperformance multilingual system for idiomaticity identification. ## ner4id we first describe our architecture for idiomaticity detection (section 3.1), and then we show how named entity recognition can be included to obtain a more robust idiom identification system (section 3.2). figure 1 provides a graphical representation of the overall idiom identification system. ## experiments in this section, we describe our experimental setup (section 4.1), the datasets we use to train and evaluate our idiom identification system (section 4.2), and the obtained results (section 4.3). ## conclusions in this paper, we presented our ner4id submission to semeval-2022 task 2 focusing on the multilingual idiomaticity detection subtask. we started by exploiting the semantic idiosyncrasy property of idiomatic expressions and introduced a novel dual-encoder transformer-based architecture that encodes both the potentially idiomatic expression (pie) and its context, and based on their similarity predicts idiomaticity. further, by manually inspecting our system's errors we discovered critical cases in which, although the individual constituents of a pie were unrelated to the context, the expressions were not idiomatic in that particular context in which they were used. in most of these cases, the pies were part of a named entity. hence, our second main contribution was devoted to the inclusion of an auxiliary ner module in the idiomaticity detection pipeline in order to avoid these errors. our experiments showed that: i) our dual-encoder architecture was able to successfully solve the idiom identification task by consistently outperforming the strong baselines provided by the task organizers, and ii) the inclusion of ner in the pipeline provided further improvements of up to 7.5 f 1 points. as future work, we plan to follow the research line proposed by @xcite , and explore the identification of idioms directly on raw texts, i.e., without pre-identified potentially idiomatic expressions, and study a broader set of languages.