A lex UNLP - NB at S em E val-2025 Task 1: A Pipeline for Idiom Disambiguation and Visual Representation (2025)

Abstract: This paper describes our system developed for SemEval-2025 Task 1, subtask A. This sharedsubtask focuses on multilingual idiom recognition and the ranking of images based on howwell they represent the sense in which a nominal compound is used within a given contextual sentence. This study explores the use of a pipeline, where task-specific models are sequentially employed to address each problem step by step. The process involves three key steps: first, identifying whether idioms are in their literal or figurative form; second, transforming them if necessary; and finally, usingthe final form to rank the input images.

## introduction idioms are a class of multi-word expressions (mwes) that present significant challenges for current state-of-the-art models, as their meanings are often not predictable from the individual words that compose them. this unpredictability can create ambiguity between the literal, surface meaning derived from the component words and the idiomatic meaning intended by the expression. addressing the complexities of mwe handling is crucial for natural language processing (nlp) applications @xcite . semeval-2025 task 1, ad-mire: advancing multimodal idiomaticity representation @xcite , focuses on evaluating models that incorporate both visual and textual information to assess their ability to capture idiomatic representations in two languages: english and brazilian portuguese. previous datasets and tasks, such as semeval-2022 task 2 @xcite and the magpie @xcite and flute @xcite datasets, have primarily focused on idiomaticity detection. however, existing idiomaticity identification benchmarks can be exploited by models that ignore the nominal compound (nc) or its contextual usage, thus failing to develop robust semantic representations of idiomatic expressions @xcite . semeval-2025 task 1 addresses these limitations by moving beyond binary classification and introducing richer representations of meaning through visual and visual-temporal modalities. semeval-2025 task 1 consists of two subtasks. subtask a presents a set of five images alongside a contextual sentence containing a potentially idiomatic nc. the objective is to rank the images based on how well they represent the sense in which the nc is used within the given context. subtask b presents a target expression and an image sequence missing the final image, the goal is to choose the best fill from four candidates images. to tackle subtask a, we propose a simple yet effective pipeline comprising three stages. the first stage involves detecting whether the potentially idiomatic nc in the given context is used literally or idiomatically. if the nc is identified as idiomatic, a separate model generates its literal meaning. in the final stage, images are aligned with the appropriate interpretation: either the generated literal meaning for idiomatic expressions or the direct literal meaning for non-idiomatic cases. this approach effectively combines idiomaticity detection, literal meaning generation, and multimodal image alignment to ensure accurate ranking of images based on the intended sense of the compound. in our analysis, we evaluate various large language models (llms) for the first two stages of our pipeline and experiment with different zeroshot classification models in the third stage, across both english and portuguese datasets. our method achieves competitive results on semeval-2025 task 1, subtask a, reaching 3rd and 6th places on the english and portuguese benchmarks respectively. the code for our approach is publicly available at https://github.com/mbadran2000/ idiom-multimodal-representation.git . ## system overview our proposed pipeline consists of three main stages: idiom detection, literal meaning generation, and figure 1 : idiomatic case: first, we detect that it is idiomatic, then we obtain its literal meaning, and lastly, we align images according to the literal meaning. figure 2 : literal case: first, we detect that it is literal, then we align images according to the literal input. multimodal image alignment. this pipeline ensures that images are aligned with the intended meaning of a given nc based on its contextual usage. figures 1 and 2 illustrate the two possible workflows, depending on the outcome of the idiom detection stage. ## conclusion in this work, we proposed a three-stage pipeline for multimodal idiomaticity representation, consisting of idiom detection, literal meaning generation, and multimodal image alignment. we used this pipeline in our submission to subtask a of semeval 2025 task 1. our approach leverages instructiontuned llms for idiomaticity classification and literal meaning generation, followed by a zero-shot multimodal retrieval model for aligning images with the intended meaning of a nominal compound. we evaluated our system on both english and portuguese datasets, exploring translation-based and direct multilingual approaches. our results demonstrated that ensemble models improve performance by mitigating individual model errors, achieving competitive results in both languages.