CLIX : Cross-Lingual Explanations of Idiomatic Expressions (2025)

Abstract: Automated definition generation systems have been proposed to support vocabulary expansion for language learners. The main barrier to the success of these systems is that learners often struggle to understand definitions due to the presence of potentially unfamiliar words and grammar, particularly when non-standard language is involved. To address these challenges, we propose CLIX, the task of Cross-Lingual explanations of Idiomatic eXpressions. We explore the capabilities of current NLP models for this task, and observe that while it remains challenging, large language models show promise. Finally, we perform a detailed error analysis to highlight the key challenges that need to be addressed before we can reliably incorporate these systems into educational tools.

## introduction the use of technology has become an important part of the language learning process @xcite . one of the main areas of interest for the proponents of technology-assisted language learning is vocabulary expansion, where recent studies have demonstrated a significant impact in student engagement and increased vocabulary knowledge @xcite @xcite . to support the development of these technologies, considerable work has been devoted to the study of automated definition generation @xcite @xcite . the effective use of definition generation systems in second language vocabulary expansion is complicated by phenomena such as the coexistence of many possible meanings for a particular word or phrase @xcite , and the potential presence of unfamiliar words and grammar in the generated definitions @xcite . while we make no assertion that any particular word in a definition will be unfamiliar to a language learner, the fact remains that there might be one or more such words. however, providing the language learner with an explanation in their native language may help remove this concern in many cases. in addition, a lot of these systems ignore the complexities of non-standard language such as idiomatic expressions, which are generally composed of words that combine to represent a meaning that does not equal the sum of its parts @xcite @xcite . motivated by these challenges, we propose the task of cross-lingual explanation of idiomatic expressions (clix). given a source language idiomatic expression, we aim to generate a natural language explanation in a specified target language. as opposed to definition, we use the term explanation to allow for more flexibility in tailoring explanations to certain situations as well as potential inclusions of context. rather than expecting a static, one-to-one mapping from idiom to definition, we allow models to approximate a one-tomany function. for example, explanations can be considered correct if they include elaborations, examples of use, and etymological information. the core concern of this research is the efficacy of the explanation and the final outcome for a potential student. we additionally hypothesize that obtaining explanations of idiomatic expressions in the learner's first language removes many of the barriers to understanding introduced by traditional definition generation systems. we choose to focus on idiomatic expressions as they are an important element of language learning that is particularly challenging for learners and automated systems alike. consider the utterance, he and i don't see eye to eye on a variety of topics. the idiomatic expression contained within this sentence is not composed of particularly challenging words, yet in this instance it denotes a negative stance relative to the referent's opinions and its meaning has nothing to do with the general definitions of the words see and eye. this example helps to display the fact that idiomatic expressions often are of unpredictable form, appear in limited numbers of syntactic frames, include various types of figuration, indicate an affective stance, and can describe situations of social interest @xcite . to explore the capabilities and limitations of modern nlp systems for generating cross-lingual explanations of idiomatic expressions, we construct two datasets of english idioms and their explanations in english, spanish, and german: epie-me, an open dataset composed of 628 unique idioms with multi-lingual explanations built from the epie corpus @xcite , and oxford-me, composed of 6,218 unique idioms taken from the oxford dictionary of idioms 4th edition @xcite . we explore performance of both pre-trained sequence-to-sequence models and large language models (llms), and show that our task configuration is much more challenging than previous definition generation tasks. we further provide some discussion on why naïve translation approaches do not suffice for this task. the explicit goal of this paper is to motivate the need for and inspire approaches to designing systems that assist language learners. a summarization of our contributions is as follows: 1. we introduce a new cross-lingual idiomatic expression explanation task. 2. we release, to our knowledge, the first crosslingual dataset for this task containing english idiomatic expressions as well as explanations of their meanings in english, spanish, and german. foot_0 3. we evaluate a set of state-of-the-art models for the task and show that, while automated metrics paint a less-favorable picture, the outcome of few-shot llms is judged positively by native speakers of the target language. 4. we perform a detailed error analysis and highlight key areas that need improvement before we can confidently use automated cross-lingual explanations for non-standard language in educational applications. ## clix in this section, we define the clix task and describe the process to create a dataset of idiomatic expressions in english with their corresponding explanations in spanish and german. we present statistics for the resulting dataset and describe the models that we will use to generate predictions. ## learning and inference strategies in this section, we outline the different learning and inference strategies that we experiment with to generate cross-lingual explanations of idiomatic expressions. we experiment with fine-tuned and few-shot generation models. strategies include: whether performing the task end-to-end or decomposing it into explanation and translation, the type of contextual information used to enhance the input prompts, and the number of demonstrations used for llm-based solutions. all prompt templates can be found in app. d ## experiments we compare results of several models across a range of strategies outlined in sec. 4. we aim to investigate what makes certain models better at understanding idiomatic expressions and what types of supplemental information are beneficial for incontext learning within this domain. we discuss the addition of contextual information outlined in sec. 4.2. we also more closely examine methods of selecting good demonstrations in few-shot prompting scenarios when levels of information for the newly-observed idiomatic expression vary, as described in sec. 4.3. ## human evaluation our main goal is to motivate the need for and inspire design approaches for tools that can assist language learners. as a result, we perform manual evaluation to determine how helpful the best model's explanations might be to real people. following @xcite , we measure two qualities of this model's generated explanations, fluency and accuracy. for the highest performing model's spanish generations, we ask native speakers to rate both aspects on a likert scale ranging from 1-5. ## conclusion and future work this work represents a first step towards generating cross-lingual explanations of idiomatic expressions. we introduced two datasets, one consisting of 628 english idioms and another containing 6218 in english idioms, each with their explanations in english, spanish and german, and showed that state of the art systems perform well when generating cross-lingual explanations, but not well enough in certain aspects for our ideal target domain of education. for our learned models, we wish to devise more effective strategies that incorporate some notion of directly unwinding figuration and explicit focus on a complete explanation that delivers information so that a learner attains some requisite understanding. we also showed a disconnect in automated measurements of performance and human judgements with relation to our task. because the model only generated small samples of text, there is no perfect way to evaluate its overall fluency within a specific language. although sentence similarity is one way to evaluate accuracy, it does not necessarily focus on whether the core of the idiomatic expression is being represented in the explanation, only how the model represents certain pieces of text. finally, although outside the scope of this paper, we can see that some idiomatic expressions are more unconventional than others, but we have no obvious way of measuring or directly coping with this phenomenon. given the examples building bridges, break a leg, and kick the bucket, clearly the first is more clearly linked to its meaning by metaphor than the latter two, and might even be more easily disambiguated with less context because of the kind of words required to surround it. this is something we would like to measure in the future. specifically, assuming idiomatic expressions live on this gradient of metaphor, are expressions on one end of the spectrum easier for models to disambiguate than those on the other?