Forbidden FRUIT is the Sweetest: An Annotated Tweets Corpus for F rench Unfrozen Idioms Identification (2025)

Abstract: Multiword expressions (MWEs) are a key area of interest in NLP, studied across various languages and inspiring the creation of dedicated datasets and shared tasks such as PARSEME. Puns in multiword expressions (PMWEs) can be described as MWEs that have been “unfrozen” to acquire a new meaning or create a wordplay. Unlike MWEs, they have received little attention in NLP, mainly due to the lack of resources available for their study. In this context, we introduce the French Unfrozen Idioms in Tweets (FRUIT) corpus, a dataset of tweets spanning three years and comprising 60,617 tweets containing both MWEs and PMWE candidates. We first describe the process of constructing this corpus, followed by an overview of the manual annotation task performed by three experts on 600 tweets, achieving a maximum α score of 0.83. Insights from this manual annotation process were then used to develop a Game With A Purpose (GWAP) to annotate more tweets from the FRUIT corpus. This GWAP aims to enhance players’ understanding of MWEs and PMWEs. Currently, 13 players made 2,206 annotations on 931 tweets, reaching an α score of 0.70. In total, 1,531 tweets from the FRUIT corpus have been annotated.

## introduction multiword expressions (mwes) have long posed a significant challenge in natural language processing, sometimes referred to as a "pain in the neck" @xcite . the term mwe corresponds to a large span of linguistic objects, more or less subject to variations and with a certain degree of idiomaticity at the lexical, syntactic, semantic, pragmatic and/or statistical levels @xcite . @xcite describe them as both idiosyncratic and pervasive across different languages. mwes are valuable not only for linguistic analysis but also for improving nlp tasks such as machine translation. wordplays and puns created from mwes (hereafter pmwes) can be described as mwes that have undergone lexical, syntactic, semantic and/or pragmatic changes to create a wordplay. their idiomatic status has been broken, leading to the emergence of a new meaning @xcite . in linguistics, this phenomenon is often referred to as "défigement" (fr, "unfreezing"), which is often found in french linguistic literature. @xcite claims that the underlying mwe should always remain identifiable in a pmwe. therefore, the mwe (1) is still recognisable in the pmwe (2). pmwe studies in nlp present several interests: (i) it can help to characterise mwes by their productivity in wordplay @xcite , (ii) it allows the real-time detection of wordplays and even mwes @xcite and (iii) they shed light on the cognitive processes that allow human speakers to recognise these particular mwes. we argue that such a study could also benefit mwes recognition in nlp as pmwes share the same linguistic challenges, such as idiomaticity across multiple levels, making them particularly challenging for nlp tasks like machine translation. in this paper, we introduce the french unfrozen idioms in tweets corpus (fruit), which consists of 60,617 tweets collected for the identification of french pmwes. to our knowledge, no previous effort has been made to annotate pmwes or create a dedicated corpus for them. the fruit corpus builds upon and expands an existing twitter (now x) dataset @xcite . section 3 details the corpus construction and the methodology for identifying pmwes. we then introduce two annotation tasks: manual annotation task three experts in nlp and linguistics annotated 600 tweets containing potential mwes and pmwes, highlighting challenges in the identification of these entities, which we discuss in section 6. the results of this annotation are available on github foot_0 . annotation through a gwap using insights from the manual annotation task, we designed a gwap to facilitate large-scale annotation of mwes and pmwes by a broader audience. the source code of this gwap is available on github foot_1 . through these annotation tasks, we aim to assess the difficulty of identifying both mwes and pmwes in tweets, combining expert knowledge with a gamified approach to enable non-expert contributors to participate in the annotation process. we provide the scripts used for tweet collection, along with all tweet ids, in a dedicated github repository foot_2 . gwaps gwaps (games with a purpose) correspond to games designed to let the machine learn from human inputs @xcite . they have been widely used in nlp, particularly for resource creation @xcite and annotation @xcite . gwaps offer several advantages: (i) they attract different types of players, such as the ones identified by @xcite and (ii) they provide an efficient alternative to traditional crowdsourcing methods @xcite . gwaps have been successfully applied to mwe annotation, as demonstrated by rigormortis @xcite . wordplays while wordplay has been studied to some extent in nlp -particularly through shared tasks such as joker-clef @xcite @xcite or the semeval tasks @xcite -pmwes remain largely unexplored. however, like wordplays, pmwes present unique challenges, both in terms of understanding linguistic creativity @xcite and generating computationally creative text @xcite . ## manual annotation task the annotation task was performed by 3 annotators, a 1 , a 2 , and a 3 , who are also authors of this paper. all had prior experience working with mwes and pmwes and had participated in previous annotation tasks. a 3 specializes in linguistics while a 1 and a 2 work in nlp and computer science. the participants were asked to answer two binary questions: the goal was to directly identify pmwes without requiring further analysis. after each annotation phase, adjudication sessions were conducted to review the annotations, discuss encountered issues, and resolve disagreements. ## issues encountered during the manual annotation task throughout the manual annotation process, we identified three major discrepancies between annotator a 3 and the other two annotators: (i) a 3 considered contextual influences more heavily, (ii) applied a stricter approach when annotating mwes derived from quotations, and (iii) exhibited a different stance on mwes with word order changes. these differences may stem from a 3 's linguistic background, whereas a 1 and a 2 specialise in nlp. below, we explain how we addressed these discrepancies and refined our annotation guidelines to minimise ambiguity in future pmwe-related annotation tasks. we also noticed that annotators sometimes repeated the same mistakes from previous annotation phases. to minimise this, we decided to share all consensus annotations among annotators. this way, whenever an annotator encounters a previously discussed case, they can easily refer to our established decision. moving forward, we plan to leverage our manual annotation findings to develop a gwap for annotating both mwes and pmwes in tweets. this approach will allow us to collect a larger number of annotations efficiently and is presented in the next section. ## expanding annotations with a gwap to scale up the annotations of the fruit corpus, we developed a participatory science task in the form of a game with a purpose (gwap). this initiative incorporates lessons from our manual annotation task to improve both accuracy and participant engagement. ## discussion in this paper, we introduced the fruit corpus, containing 60,617 tweets among which 1,531 have been manually annotated through (i) an expert review and (ii) a gwap. the results of the manual annotation task show that both mwe and pmwe identification tasks are challenging, even for experts with substantial experience in these two notions. we argue that the low inter-annotator score of 0.19 obtained during our pilot annotation (section 5.1), alongside the discussion presented in sec-tion 5.3, may be attributed to differences in the interpretation of mwes and pmwes between nlp experts and linguistics experts. despite these challenges, by developing clearer guidelines and organising adjudication sessions, we improved our understanding of both mwes and pmwes, which likely contributed to an increase in our inter-annotator score to 0.83. the gwap demonstrates that it is possible to teach non-expert individuals how to recognise and identify both mwes and pmwes. to achieve this, we leveraged the guidelines developed during the manual annotation task. we also allowed our players to improve their understanding of the key notions through progressive learning (section 7.2). the results exhibit a high level of agreement between players, with an inter-annotator score of 0.70. furthermore, we unveil that our players tend to agree with the reference annotation made by our three experts, with an observed mean f-score of 85.59 for every type of annotation (92.63 for pmwe identification). this result might be influenced by the fact that our players are primarily from the research area, and some of them having already basic knowledge on mwes and occasionally pmwes. despite this potential bias, the insights obtained from this annotation task will inform future improvements to the gwap and the annotation process. looking ahead, we intend to continue annotating the fruit corpus through the gwap presented here. in particular, we want to make this gwap available to a wider non-expert audience so that we can observe the quality of our progressive learning. we also plan to create a second annotation task, whose goal will be to annotate found pmwes at different levels. we plan to assemble a multilingual dataset containing mwes and pmwes from films and article titles (media and scientific). such a dataset could help us analyse differences in pmwe construction across languages. this future work could benefit from a participatory annotation task, such as the one described here. @xmath0. by seeds filtered / 3,268,394 15,381 20,244 total 3,369,636 101,242 85,861 60,617