GenAI4ELab/papercli-papers huggingface dataset:
Zero at SemEval-2025 Task 2: Entity-Aware Machine Translation: Fine-Tuning NLLB for Improved Named Entity Translation
Authors: Revanth Gundam; Abhinav Marri; Advaith Malladi; Radhika Mamidi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Machine Translation (MT) is an essential tool for communication amongst people across different cultures, yet Named Entity (NE) translation remains a major challenge due to its rarity in occurrence and ambiguity. Traditional approaches, like using lexicons or parallel corpora, often fail to generalize to unseen entities, and hence do not perform well. To address this, we create a silver dataset using the Google Translate API and fine-tune the facebook/nllb200-distilled-600M model with LoRA (LowRank Adaptation) to enhance translation accuracy while also maintaining efficient memory use. Evaluated with metrics such as BLEU, COMET, and M-ETA, our results show that fine-tuning a specialized MT model improves NE translation without having to rely on largescale general-purpose models.
VerbaNexAI at SemEval-2025 Task 11 Track A: A RoBERTa-Based Approach for the Classification of Emotions in Text
Authors: Danileth Almanza Gonzalez; Edwin Puertas; Juan Carlos Martinez-Santos
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Emotion detection in text has become a highly relevant research area due to the growing interest in understanding emotional states from human interaction in the digital world. This study presents an approach for emotion detection in text using a RoBERTa-based model, optimized for multi-label classification of the emotions joy, sadness, fear, anger, and surprise in the context of the SemEval 2025 - Task 11: Bridging the Gap in Text-Based Emotion Detection competition. Advanced preprocessing strategies were incorporated, including the augmentation of the training dataset through automatic translation to improve the representativeness of less frequent emotions. Additionally, a loss function adjustment mechanism was implemented to mitigate class imbalance, enabling the model to enhance its detection capability for underrepresented categories. The experimental results reflect competitive performance, with a macro F1 of 0.6577 on the development set and 0.6266 on the test set. In the competition, the model ranked 47th, demonstrating solid performance against the challenge posed.
dutir914 at SemEval-2025 Task 1: An integrated approach for Multimodal Idiomaticity Representations
Authors: Yanan Wang; Dailin Li; Yicen Tian; Bo Zhang; Wang Jian; Liang Yang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: SemEval-2025 Task 1 introduces multimodal datasets for idiomatic expression representation. Subtask A focuses on ranking images based on potentially idiomatic noun compounds in given sentences. Idiom comprehension demands the fusion of visual and auditory elements with contextual semantics, yet existing datasets exhibit phrase-image discordance and culture-specific opacity, impeding cross-modal semantic alignment. To address these challenges, we propose an integrated approach that combines data augmentation and model fine-tuning in subtask A. First, we construct two idiom datasets by generating visual metaphors for idiomatic expressions to fine-tune the CLIP model. Next, We propose a three-stage multimodal chain-of-thought method, fine-tuning Qwen2.5-VL-7B-Instruct to generate rationales and perform inference, alongside zero-shot experiments with Qwen2.5-VL-72B-Instruct. Finally, we integrate the output of different models through a voting mechanism to enhance the accuracy of multimodal semantic matching. This approach achieves {textbf{0.92}} accuracy on the Portuguese test set and {textbf{0.93}} on the English test set, ranking {textbf{3rd}} and {textbf{4th}}, respectively. The implementation code is publicly available here{footnote{{url{ https://github.com/wyn1015/semeval}}}}.
Advacheck at SemEval-2025 Task 3: Combining NER and RAG to Spot Hallucinations in LLM Answers
Authors: Anastasia Voznyuk; German Gritsai; Andrey Grabovoy
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The Mu-SHROOM competition in the SemEval-2025 Task 3 aims to tackle the problem of detecting spans with hallucinations in texts, generated by Large Language Models (LLMs). Our developed system, submitted to this task, is a joint architecture that utilises Named Entity Recognition (NER), Retrieval-Augmented Generation (RAG) and LLMs to gather, compare and analyse information in the texts provided by organizers. We extract entities potentially capable of containing hallucinations with NER, aggregate relevant topics for them using RAG, then verify and provide a verdict on the extracted information using the LLMs. This approach allowed with a certain level of quality to find hallucinations not only in facts, but misspellings in names and titles, which was not always accepted by human annotators in ground truth markup. We also point out some inconsistencies within annotators spans, that perhaps affected scores of all participants.
PALI-NLP at SemEval 2025 Task 1: Multimodal Idiom Recognition and Alignment
Authors: Runyang You; Xinyue Mei; Mengyuan Zhou
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Understanding idioms in multimodal contexts poses significant challenges due to data scarcity, idiomatic ambiguity, and the need for effective alignment of visual and textual inputs. In this work, we introduce MIRA (Multimodal Idiom Recognition and Alignment), a training-free framework designed to address these challenges on the SemEval-2025 Task 1 (AdMIRe) benchmark. MIRA leverages powerful closed-source large language models (LLMs) and integrates three key innovations: bias correction via in-context learning, multi-step semantic-visual fusion, and a self-revision mechanism that iteratively refines its outputs through backward verification. By systematically processing and fusing multimodal inputs, MIRA generates high-quality, fine-grained image-text representations that enhance idiom comprehension across different languages and cultural contexts. Experimental evaluations in both English and Portuguese demonstrate that our approach achieves robust performance without the need for additional training, setting a new standard for multimodal idiom recognition.
UTBNLP at Semeval-2025 Task 11: Predicting Emotion Intensity with BERT and VAD-Informed Attention.
Authors: Melissa Moreno Novoa; Edwin Puertas; Juan Carlos Martinez-Santos
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Emotion intensity prediction plays a crucial role in affective computing, allowing for a more precise understanding of how emotions are conveyed in text. This study proposes a system that estimates emotion intensity levels by integrating contextual language representations with numerical emotion-based features derived from Valence, Arousal, and Dominance (VAD). The methodology combines BERT embeddings, predefined VAD values per emotion, and machine learning techniques to enhance emotion detection, without relying on external lexicons. The system was evaluated on the SemEval-2025 Task 11 Track B dataset, predicting five emotions (anger, fear, joy, sadness, and surprise) on an ordinal scale.The results highlight the effectiveness of integrating contextual representations with predefined VAD values, enabling a more nuanced representation of emotional intensity. However, challenges arose in distinguishing intermediate intensity levels, affecting classification accuracy for certain emotions. Despite these limitations, the study provides insights into the strengths and weaknesses of combining deep learning with numerical emotion modeling, contributing to the development of more robust emotion prediction systems. Future research will explore advanced architectures and additional linguistic features to enhance model generalization across diverse textual domains.
Samsung Research Poland at SemEval-2025 Task 8: LLM ensemble methods for QA over tabular data
Authors: Pawel Bujnowski; Tomasz Dryjanski; Christian Goltz; Bartosz Swiderski; Natalia Paszkiewicz; Bartlomiej Kuzma; Jacek Rutkowski; Jakub Stepka; Milosz Dudek; Wojciech Siemiatkowski; Weronika Plichta; Bartłomiej Paziewski; Maciej Grabowski; Katarzyna Beksa; Zuzanna Bordzicka; Filip Ostrowski; Grzegorz Sochacki
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Question answering using Large Language Models has gained significant popularity inboth everyday communication and at the workplace. However, certain tasks, such as querying tables, still pose challenges for commercial and open-source chatbots powered by advanceddeep learning models. Addressing these challenges requires specialized approaches.During the SemEval-2025 Task 8 competition focused on tabular data, our solution achieved86.21% accuracy and took 2nd place out of 100 teams. In this paper we present ten methodsthat significantly improve the baseline solution. Our code is available as open-source.
OPI-DRO-HEL at at SemEval-2025 Task 11: Few-shot prompting for Text-based Emotion Recognition
Authors: Daniel Karaś; Martyna Śpiewak
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our system, developed as our contribution to SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection task, in particular track A, Multi-label Emotion Detection subtask. Our approach relies on two distinct components: semantic search for top N most similar inputs from training set and an interface to pretrained LLM being prompted using the found examples. We examine several prompting strategies and their impact on overall performance of the proposed solution.
Saama Technologies at SemEval-2025 Task 8: Few-shot prompting with LLM-generated examples for question answering on tabular data
Authors: Kamal Raj Kanakarajan; Hwanmun Kim; Malaikannan Sankarasubbu
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: For SemEval 2025 Task 8, addressing tabular data question answering, we introduce a novel few-shot prompting system that guides large language models (LLMs) to generate Python code representing the reasoning process. Our system automatically creates a library of exemplar code snippets from training data, which are then used for few-shot prompting. Crucially, we incorporate a selection prompt to choose the best candidate code from multiple LLM-generated options, improving robustness and accuracy. Our system achieved competitive results, ranking 17th in the Open Model track and 25th overall. Ablation studies demonstrate the effectiveness of our exemplar generation and code selection strategies. We conclude with a discussion of limitations and promising avenues for future research.
Tuebingen at SemEval-2025 Task 10: Class Weighting, External Knowledge and Data Augmentation in BERT Models
Authors: Özlem Karabulut; Soudabeh Eslami; Ali Gharaee; Matthew Andrews
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The spread of disinformation and propaganda in online news presents a significant challengeto information integrity. As part of the SemEval 2025 Task-10 on Multilingual Characterization and Extraction of Narratives from Online News, this study focuses on Subtask 1: Entity Framing, which involves assigning roles to named entities within news articles across multiple languages.We investigate techniques such as data augmentation, external knowledge, and class weighting to improve classification performance. Our findings indicate that class weighting was more effective than other approaches
VerbaNexAI at SemEval-2025 Task 2: Enhancing Entity-Aware Translation with Wikidata-Enriched MarianMT
Authors: Daniel Peña Gnecco; Juan Carlos Martinez Santos; Edwin Puertas
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents the VerbaNexAi Lab system for SemEval-2025 Task 2: Entity-Aware Machine Translation (EA-MT), focusing on translating named entities from English to Spanish across categories such as musical works, foods, and landmarks. Our approach integrates detailed data preprocessing, enrichment with 240,432 Wikidata entity pairs, and fine-tuning of the MarianMT model to enhance entity translation accuracy. Official results reveal a COMET score of 87.09, indicating high fluency, an M-ETA score of 24.62, highlighting challenges in entity precision, and an Overall Score of 38.38, ranking last among 34 systems. While Wikidata improved translations for common entities like “Águila de San Juan,” our static methodology underperformed compared to dynamic LLM-based approaches.
CSECU-Learners at SemEval-2025 Task 9: Enhancing Transformer Model for Explainable Food Hazard Detection in Text
Authors: Monir Ahmad; Md. Akram Hossain; Abu Nowshed Chy
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Food contamination and associated illnesses represent significant global health challenges, leading to thousands of deaths worldwide. As the volume of food-related incident reports on web platforms continues to grow, there is a pressing demand for systems capable of detecting food hazards effectively. Furthermore, explainability in food risk detection is crucial for building trust in automated systems, allowing humans to validate predictions. SemEval-2025 Task 9 proposes a food hazard detection challenge to address this issue, utilizing content extracted from websites. This task is divided into two sub-tasks. Sub-task 1 involves classifying the type of hazard and product, while sub-task 2 focuses on identifying precise hazard and product “vectors” to offer detailed explanations for the predictions. This paper presents our participation in this task, where we introduce a transformer-based method. We fine-tune an enhanced version of the BERT transformer to process lengthy food incident reports. Additionally, we combine the transformer’s contextual embeddings to enhance its contextual representation for hazard and product “vectors” prediction. The experimental results reveal the competitive performance of our proposed method in this task. We have released our code at https://github.com/AhmadMonirCSECU/SemEval-2025_Task9.
NLP-DU at SemEval-2025 Task 11: Analyzing Multi-label Emotion Detection
Authors: Sadman Sakib; Ahaj Faiak; Abdullah Ibne Hanif Arean; Fariha Anjum Shifa
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper describes NLP-DU’s entry to SemEval-2025 Task 11 on multi-label emotion detection. We investigated the efficacy of transformer-based models and propose an ensemble approach that combines multiple models. Our experiments demonstrate that the ensemble outperforms individual models under the dataset constraints, yielding superior performance on key evaluation metrics. These findings underscore the potential of ensemble techniques in enhancing multi-label emotion detection and contribute to the broader understanding of emotion analysis in natural language processing.
WordWiz at SemEval-2025 Task 10: Optimizing Narrative Extraction in Multilingual News via Fine-Tuned Language Models
Authors: Ruhollah Ahmadi; Hossein Zeinali
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our WordWiz system for SemEval-2025 Task 10: Narrative Extraction. We employed a combination of targeted preprocessing techniques and instruction-tuned language models to generate concise, accurate narrative explanations across five languages. Our approach leverages an evidence refinement strategy that removes irrelevant sentences, improving signal-to-noise ratio in training examples. We fine-tuned Microsoft’s Phi-3.5 model using both Supervised Fine-Tuning (SFT). During inference, we implemented a multi-temperature sampling strategy that generates multiple candidate explanations and selects the optimal response using narrative relevance scoring. Notably, our smaller Phi-3.5 model consistently outperformed larger alternatives like Llama-3.1-8B across most languages. Our system achieved significant improvements over the baseline across all languages, with F1 scores ranging from 0.7486 (Portuguese) to 0.6839 (Bulgarian), demonstrating the effectiveness of evidence-guided instruction tuning for narrative extraction.
LyS at SemEval 2025 Task 8: Zero-Shot Code Generation for Tabular QA
Authors: Adrián López Gude; Roi Santos Ríos; Francisco Prado Valiño; Ana Ezquerro; Jesús Vilares
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: We developed a zero-shot pipeline that leverages an Large Language Model to generate functional code capable of extracting the relevant information from tabular data based on an input question. Our approach consists of a modular pipeline where the main code generator module is supported by additional components that identify the most relevant columns and analyze their data types to improve extraction accuracy. In the event that the generated code fails, an iterative refinement process is triggered, incorporating the error feedback into a new generation prompt to enhance robustness. Our results show that zero-shot code generation is a valid approach for Tabular QA, achieving rank 33 of 53 in the test phase despite the lack of task-specific fine-tuning.
AILS-NTUA at SemEval-2025 Task 3: Leveraging Large Language Models and Translation Strategies for Multilingual Hallucination Detection
Authors: Dimitra Karkani; Maria Lymperaiou; George Filandrianos; Nikolaos Spanos; Athanasios Voulodimos; Giorgos Stamou
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Multilingual hallucination detection stands as an underexplored challenge, which the Mu-SHROOM shared task seeks to address. In this work, we propose an efficient, training-free LLM prompting strategy that enhances detection by translating multilingual text spans into English. Our approach achieves competitive rankings across multiple languages, securing two first positions in low-resource languages. The consistency of our results highlights the effectiveness of our translation strategy for hallucination detection, demonstrating its applicability regardless of the source language.
Dataground at SemEval-2025 Task 8: Small LLMs and Preference Optimization for Tabular QA
Authors: Giuseppe Attardi; Andrea Nelson Mauro; Daniele Sartiano
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: We present our submission to SemEval 2025 Task 8: Question Answering on Tabular Data, which challenges participants to develop systems capable of answering natural language questions on real-world tabular datasets. Our approach aims at generating Pandas code that can be run on such datasets to produce the desired answer. The approach consists in fine-tuning a Small Language Model (SLM) through Preference Optimization on both positive and negative examples generated by a teacher model.A base SLM is first elicited to produce the code to compute the answer to a question through a Chain of Thought (CoT) prompt. We performed extensive testing on the DataBench development set, exploring a variety of prompts, eventually settling on a detailed instruction prompt, followed by two-shot examples. Due to hardware constraints, the base model was an SLM with ${leq}$ 8 billion parameters.We then fine-tuned the model through Odds Ratio Preference Optimization (ORPO) using as training data the code produced by a teacher model on the DataBench training set. The teacher model was GPT-4o, whose code was labeled preferred, while the code generated by the base model was rejected. This increased the accuracy on the development set from 71% to 85%.Our method demonstrated robust performance in answering complex questions across diverse datasets, highlighting the effectiveness of combining small LLMs with supervised fine-tuning and automated code execution for tabular question answering.
Core Intelligence at SemEval-2025 Task 8: Multi-hop LLM Agent for Tabular Question Answering
Authors: Maryna Chernyshevich
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper describes a multi-hop LLM agent for tabular question answering developed for SemEval-2025 Task 8 and ranked 6th with 87% accuracy. Our approach combines proprietary LLM (ChatGPT-3.5-turbo) for code generation and open source LLM (Llama-3.2-3B) for answer validation.
MALTO at SemEval-2025 Task 3: Detecting Hallucinations in LLMs via Uncertainty Quantification and Larger Model Validation
Authors: Claudio Savelli; Alkis Koudounas; Flavio Giobergia
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large language models (LLMs) often produce {textit{hallucinations}} —factually incorrect statements that appear highly persuasive. These errors pose risks in fields like healthcare, law, and journalism. This paper presents our approach to the Mu-SHROOM shared task at SemEval 2025, which challenges researchers to detect hallucination spans in LLM outputs. We introduce a new method that combines probability-based analysis with Natural Language Inference to evaluate hallucinations at the word level. Our technique aims to better align with human judgments while working independently of the underlying model. Our experimental results demonstrate the effectiveness of this method compared to existing baselines.
LCTeam at SemEval-2025 Task 3: Multilingual Detection of Hallucinations and Overgeneration Mistakes Using XLM-RoBERTa
Authors: Araya Hailemariam; Jose Maldonado Rodriguez; Ezgi Başar; Roman Kovalev; Hanna Shcharbakova
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In recent years, the tendency of large language models to produce hallucinations has become an object of academic interest. Hallucinated or overgenerated outputs created by LLMs contain factual inaccuracies which can potentially invalidate textual coherence. The Mu-SHROOM shared task sets the goal of developing strategies for detecting hallucinated parts of LLM outputs in a multilingual context. We present an approach applicable across multiple languages, which incorporates the alignment of tokens and hard labels, as well as training a multi-lingual XLM-RoBERTa model. With this approach we managed to achieve 2nd in Chinese and top-10 positions in 7 other language tracks of the competition.