M ulti C o PIE : A Multilingual Corpus of Potentially Idiomatic Expressions for Cross-lingual PIE Disambiguation (2025)

Abstract: Language models are able to handle compositionality and, to some extent, non-compositional phenomena such as semantic idiosyncrasy, a feature most prominent in the case of idioms. This work introduces the MultiCoPIE corpus that includes potentially idiomatic expressions in Catalan, Italian, and Russian, extending the language coverage of PIE corpus data. The new corpus provides additional linguistic features of idioms, such as their semantic compositionality, part-of-speech of idiom head as well as their corresponding idiomatic expressions in English. With this new resource at hand, we first fine-tune an XLM-RoBERTa model to classify figurative and literal usage of potentially idiomatic expressions in English. We then study cross-lingual transfer to the languages represented in the MultiCoPIE corpus, evaluating the model’s ability to generalize an idiom-related task to languages not seen during fine-tuning. We show the effect of ‘cross-lingual lexical overlap’: the performance of the model, fine-tuned on English idiomatic expressions and tested on the MultiCoPIE languages, increases significantly when classifying ‘shared idioms’ -idiomatic expressions that have direct counterparts in English with similar form and meaning. While this observation raises questions about the generalizability of cross-lingual learning, the results from experiments on PIEs demonstrate strong evidence of effective cross-lingual transfer, even when accounting for idioms similar across languages.

## introduction high-level language understanding is reflected in the ability to combine meaning units into larger units; this process is known as composition. natural language often departs from the principle of simple compositionality, as in the case of multiword expressions, or mwes, commonly described as combinations of words that exhibit a certain degree of lexical, morphological, syntactic and/or semantic idiosyncrasy @xcite . a particular category of mwes are idioms: this category stands out through its idiosyncratic semantics, i.e. the meaning of idiomatic mwes cannot be obtained by compositionally interpreting their components @xcite . in this work, we focus on a subset of mwes, namely, idiomatic expressions with literalidiomatic ambiguity @xcite , or expressions that can be used in a literal or figurative sense, such as blow the whistle or black sheep. idiomatic expressions with this property can be referred to as 'potentially idiomatic expressions', or pies, a term introduced by @xcite . this term is often used in the context of pie disambiguationa task that typically consists of classifying specific idiom occurrences as 'literal' or 'figurative', based on the surrounding context. in this paper, we present multicopie, a multilingual corpus of idiomatic expressions with literal and figurative occurrences in catalan, italian, and russian. 1 we fine-tune a masked language model well suited for classification-xlm-roberta @xcite -for the pie disambiguation task on available english data and investigate cross-lingual transfer to the three languages in multicopie, comparing the crosslingual model to a baseline, fine-tuned monolingually on the multicopiedata. we also measure whether the model's performance is affected by the size of provided context. the cross-lingual experiment allows us to measure whether a classifier fine-tuned for the pie disambiguation task on english data generalizes to idiomatic expressions in the multicopie languages, as these pies have not been seen by the classifier at the fine-tuning stage. however, it is important to consider that certain idiomatic expressions in the multicopie languages have idiomatic equivalents in english, i.e. cross-lingual pairs of idiomatic expressions with direct lexico-syntactic correspondence and similar semantics @xcite , such as the italian idiom rompere il ghiaccio (lit. 'to break the ice'), the catalan idiom trencar el gel (lit. 'to break the ice'), and the corresponding english idiom break the ice. since contextualized models produce similar embeddings for words with similar semantics across languages, it becomes difficult to properly interpret the classifier's performance on these cross-lingual idiom pairs and identify whether the model truly evaluates the idiomatic expression in a language outside of the fine-tuning set. to this end, we compare the performance of the classifier on two groups: idiomatic expressions in the multicopie languages that have direct equivalents in english and idiomatic expressions without such equivalents. multilingual and non-english pie corpora a pivotal role in advancing the field of multiword expressions plays the parseme project, an international research community that provides mwerelated tools and resources @xcite . the parseme corpus @xcite , a multilingual corpus annotated with mwes 2 , covers 26 2 https://parsemefr.lis-lab.fr/parseme-st-guidelines/1.3/ languages and multiple mwe categories, such as light verb constructions, verbal idioms, and more. @xcite use the parseme data to identify idiomatic, literal and coincidental foot_1 occurrences of verbal mwes in basque, german, greek, polish and portuguese; they also provide a formal definition of literal occurrences. the semeval-2022 task 2a corpus was released as the dataset for the semeval-2022 task on multilingual idiomaticity detection and sentence embedding @xcite . the corpus contains multiword expressions in english, portuguese and galician and is based on the noun compound senses dataset by @xcite as well as on the dataset by tayyar @xcite . the id10m corpus by @xcite provides a token-level annotated dataset of pies for 10 languages. pie corpora also exist for indian languages @xcite , german @xcite , swedish @xcite , russian @xcite , persian @xcite , arabic @xcite and japanese @xcite . @xcite show that bert @xcite outperforms other contextualized models in tasks related to lexical composition. the probing tasks by @xcite similarly suggests that bert is able to encode the idiomatic meaning of pies and separates the literal and idiomatic usages of pies with high precision. a word-level probing experiment by @xcite shows that bert recognizes idioms by focusing both on the idiomatic expressions themselves and on the surrounding context. @xcite use analysis of attention patterns to investigate idiom processing in pre-trained models for the task of translation; their finding gives evidence that idioms are treated differently by the encoder in comparison to literal instances. @xcite demonstrate that models such as bert, multilingual bert (mbert) @xcite and distilbert @xcite display different attention patterns when representing tokens within idioms. @xcite employ camembert @xcite , the pretrained bert-derived model for french, for a demasking task and show that the model makes better predictions for tokens within idioms, as compared to tokens within simple lexemes. despite the evidence that transformer-based pre-trained language models are able to distinguish between idiomatic and literal contexts with high accuracy, multiple studies highlight that transformer-based models struggle to represent phrase meanings in a nuanced way @xcite @xcite . hashempour and villavicencio (2020) leverage the idiom principle foot_2 and use context2vec @xcite and bert to classify literal and figurative senses of english idioms in the vnctokens dataset @xcite , with bertbased model achieving the mean f-score of 0.71. @xcite utilize contextual embeddings by bert and mbert, for supervised and unsupervised pie classification tasks in english and german, achieving the f-score of 0.93 on the semeval5b dataset @xcite , 0.90 on the vnc-tokens dataset @xcite , and 0.94 on the german data @xcite in the supervised setting. the study by @xcite proposes a novel architecture that uses contextualized and static word embeddings to detect pie occurrences based on their semantic compatibility with context. in @xcite introduced the multilingual idiomaticity detection and sentence embedding task, with subtask a dedicated to binary classification of literal and figurative idiom usage. the majority of contributions are based on the transformer architecture, including pre-trained multilingual models @xcite @xcite . in contrast to fine-tuning experiments performed jointly in several languages, @xcite take a different approach: in addition to monolingual experiments, researchers explore cross-lingual transfer for english and russian by fine-tuning several models from the bert family for binary classification of pies; the fine-tuned mbert achieves 72.4% accuracy in the englishto-russian experiment and 80.1% accuracy in the russian-to-english experiment. ## monolingual pie classification 4.1 english data to fine-tune our idiom disambiguation classifier, we use monolingual english data comprised of magpie and the english subset of the semeval-2022 task 2a dataset. both corpora were manually annotated by native speakers and include not only the target sentences containing idioms but also the surrounding context. while magpie serves as a backbone of our training data due to its size, the semeval-2022 task 2a corpus provides additional idiom types as well as interesting cases when an idiom functions as part of a named entity. from the semeval dataset, we exclude less idiomatic items, such as law firm and application form; for the selected 75 idioms, we keep all the instances. from magpie, we select 1513 phrase-level idioms, ex-cluding clauses and dependent clauses. we exclude instances with the inter-annotator agreement lower than 75% and use one preceding and one following sentence as context. the combined dataset consists of 37.9k instances of 1582 idiom types; 75.9% of the instances are labeled as figurative. ## cross-lingual lexical overlap and transfer to explore cross-lingual transfer, we use models fine-tuned for the pie disambiguation task on the english data and evaluate them on the multicopie languages, which have not been observed during fine-tuning. we employ two baselines: the majority-class base-line and the xlm-r-multicopie baseline. the majority-class assigns the figurative label (majority class) to all observations, reflecting label distribution in the multicopie for each language. for the xlm-r-multicopie baseline, we fine-tune an xlm-roberta classifier on the multicopie data, separately for each language. we fine-tune 10 models in a zero-shot setting, selecting 70% of the idioms for the training set, 15% for validation and 15% for testing. ## conclusions and future work in this paper, we introduce a new corpus, multicopie , extending language coverage of pie data. we then evaluate the performance of a classifier fine-tuned on idiom disambiguation in monolingual (english) and cross-lingual settings (catalan, italian, russian). in the monolingual setting, our classifier outperforms the majority baselines in the zero-shot, oneshot, and random settings. in the cross-lingual experiment, our classifier, fine-tuned on english data only, surpasses the majority baseline for all languages in multicopie. it also outperforms xlm-roberta models fine-tuned monolingually on the multicopie data for italian and catalan, while showing slightly lower performance on russian. this indicates that, when leveraging pre-trained models like xlm-roberta, less-resourced languages may benefit substantially from cross-lingual transfer, often outperforming fine-tuning on small monolingual datasets. in contrast, high-resource languages such as russian may achieve better re-sults when fine-tuned on even modest amounts of monolingual data, given their richer representation in the pre-training corpus. we also demonstrate that the cross-lingual model shows an increase in performance when classifying multicopie idioms that have an english equivalent with similar form and meaning present in the english training set during fine-tuning. this finding supports the idea that a pie classifier, finetuned on one language, can benefit from the lexical overlap between cross-lingual idiom pairs during evaluation on unseen languages, which may result in overly optimistic performance scores. this finding may be especially relevant for closely related languages that share a large amount of idiomatic expressions. while this result highlights limitations in crosslingual learning and cautions against overestimating cross-lingual generalization, the experiment on pie disambiguation clearly demonstrates the presence of cross-lingual transfer, even after accounting for cross-lingual overlap between languages.