PALI - NLP at S em E val 2025 Task 1: Multimodal Idiom Recognition and Alignment (2025)

Abstract: Understanding idioms in multimodal contexts poses significant challenges due to data scarcity, idiomatic ambiguity, and the need for effective alignment of visual and textual inputs. In this work, we introduce MIRA (Multimodal Idiom Recognition and Alignment), a training-free framework designed to address these challenges on the SemEval-2025 Task 1 (AdMIRe) benchmark. MIRA leverages powerful closed-source large language models (LLMs) and integrates three key innovations: bias correction via in-context learning, multi-step semantic-visual fusion, and a self-revision mechanism that iteratively refines its outputs through backward verification. By systematically processing and fusing multimodal inputs, MIRA generates high-quality, fine-grained image-text representations that enhance idiom comprehension across different languages and cultural contexts. Experimental evaluations in both English and Portuguese demonstrate that our approach achieves robust performance without the need for additional training, setting a new standard for multimodal idiom recognition.

## introduction the semeval-2025 task 1 (admire) @xcite presents a new benchmark for understanding idioms in both visual and textual forms. it poses three main challenges: 1. data scarcity: with limited data (102 samples for subtask a and 20 for subtask b), traditional training methods won't work, so we need more efficient solutions. 2. idiomatic ambiguity: phrases like "panda car" can mean different things (e.g., a police car or a toy) depending on context, making classification difficult. this is further complicated by cultural and domain-specific differences. 3. multimodal alignment: combining visual and textual information requires new ways to integrate these different types of data, beyond just merging features. ## subtask b system overview the goal is twofold: to extend a visual narrative by selecting the most appropriate candidate image from a set of four, and to classify the nc usage as either idiomatic or literal. we address this through a two-stage pipeline: multimodal story analysis followed by usage classification. similar to subtask a, we also include literal use cases to tackle idiomatic bias, as described in section 2.1.1. ## experiment in this section, we perform a series of experiments to address the following research questions: ## conclusion this work presents mira, a training-free framework for multimodal idiom comprehension. leveraging powerful closed-source language models, mira overcomes data scarcity, idiomatic ambiguity, and multimodal alignment challenges through three core components: visual-text fusion, incontext learning, and self-consistency reasoning. visual-text fusion extracts fine-grained visual details and integrates them with text to create highquality representations. in-context learning leverages in-domain knowledge to ensure semantic interpretations align with the data distribution, while self-consistency reasoning aggregates multiple reasoning paths to mitigate errors and enhance reliability. together, these components form a robust causal inference pipeline that has achieved top rankings in both english and portuguese evaluations and adapts efficiently to low-resource multilingual scenarios. future work will explore enhanced chain-of-thought reasoning and zero-shot debiasing to further expand its applicability. test-time scaling and llm reasoners the concept of test-time scaling, where increased compute at test time leads to better results, has gained traction in the context of llms @xcite . recent work using models like openai's o1 and deepseek-r1 @xcite demonstrated superior performance through scaled test-time computation. our work leverages these insights by employing self-consistency reasoning @xcite to enhance performance without fine-tuning, aligning with the goal of efficient and scalable reasoning in llms.