GenAI4ELab/papercli-papers huggingface dataset:
Fossils at SemEval-2025 Task 9: Tasting Loss Functions for Food Hazard Detection in Text Reports
Authors: Aman Sinha; Federica Gamba
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Food hazard detection is an emerging field where NLP solutions are being explored. Despite the recent accessibility of powerful language models, one of the key challenges that still persists is the high class imbalance within datasets, often referred to in the literature as the {textit{long tail problem}}.In this work, we present a study exploring different loss functions borrowed from the field of visual recognition, to tackle long-tailed class imbalance for food hazard detection in text reports. Our submission to SemEval-2025 Task 9 on the Food Hazard Detection Challenge shows how re-weighting mechanism in loss functions prove beneficial in class imbalance scenarios. In particular, we empirically show that class-balanced and focal loss functions outperform all other loss strategies for Subtask 1 and 2 respectively.
Ustnlp16 at SemEval-2025 Task 9: Improving Model Performance through Imbalance Handling and Focal Loss
Authors: Zhuoang Cai; Zhenghao Li; Yang Liu; Liyuan Guo; Yangqiu Song
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Classification tasks often suffer from imbal- anced data distribution, which presents chal- lenges in food hazard detection due to severe class imbalances, short and unstructured text, and overlapping semantic categories. In this paper, we present our system for SemEval- 2025 Task 9: Food Hazard Detection, which ad- dresses these issues by applying data augmenta- tion techniques to improve classification perfor- mance. We utilize transformer-based models, BERT and RoBERTa, as backbone classifiers and explore various data balancing strategies, including random oversampling, Easy Data Augmentation (EDA), and focal loss. Our ex- periments show that EDA effectively mitigates class imbalance, leading to significant improve- ments in accuracy and F1 scores. Furthermore, combining focal loss with oversampling and EDA further enhances model robustness, par- ticularly for hard-to-classify examples. These findings contribute to the development of more effective NLP-based classification models for food hazard detection.
Lacuna Inc. at SemEval-2025 Task 4: LoRA-Enhanced Influence-Based Unlearning for LLMs
Authors: Aleksey Kudelya; Alexander Shirnin
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper describes LIBU (LoRA enhanced influence-based unlearning), an algorithm to solve the task of unlearning - removing specific knowledge from a large language model without retraining from scratch and compromising its overall utility (SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models). The algorithm combines classical influence functions to remove the influence of thedata from the model and second-order optimization to stabilize the overall utility. Our experiments show that this lightweight approach is well applicable for unlearning LLMs in different kinds of task.
VerbaNexAI at SemEval-2025 Task 3: Fact Retrieval with Google Snippets for LLM Context Filtering to identify Hallucinations
Authors: Anderson Morillo; Edwin Puertas; Juan Carlos Martinez Santos
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Thefirst approach leverages advanced LLMs, employing a chain-of-thought prompting strategywith one-shot learning and Google snippets forcontext retrieval, demonstrating superior performance. The second approach utilizes traditional NLP analysis techniques, including semantic ranking, token-level extraction, and rigorous data cleaning, to identify hallucinations
Team KiAmSo at SemEval-2025 Task 11: A Comparison of Classification Models for Multi-label Emotion Detection
Authors: Kimberly Sharp; Sofia Kathmann; Amelie Rüeck
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The aim of this paper is to take on the challenge of multi-label emotion detection for a variety of languages as part of Track A in SemEval 2025 Task 11: Bridging the Gap in Text-Based Emotion Detection. We fine-tune different pre-trained mono- and multilingual language models and compare their performance on multi-label emotion detection on a variety of high-resource and low-resource languages. Overall, we find that monolingual models tend to perform better, but for low-resource languages that do not have state-of-the-art pre-trained language models, multilingual models can achieve comparable results.
FiRC-NLP at SemEval-2025 Task 11: To Prompt or to Fine-Tune? Approaches for Multilingual Emotion Classification
Authors: Wondimagegnhue Tufa; Fadi Hassan; Evgenii Migaev; Yalei Fu
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In this paper, we describe our system devel-oped for participation in SemEval-2025 Task11: Bridging the Gap in Text-Based EmotionDetection. We compare three approaches formultilingual, multi-label emotion classification:XLM-R, an ensemble of models (XLM-5), anda prompt-based approach. We evaluate the per-formance of these models across a diverse setof languages, ranging from high-resource tolow-resource languages
GIL-IIMAS UNAM at SemEval-2025 Task 4: LA-Min(E): LLM Unlearning Approaches Under Function Minimizing Evaluation Constraints
Authors: Karla Salas-Jimenez; Francisco López-Ponce; Diego Hernández-Bustamante; Gemma Bel-Enguix; Helena Gómez-Adorno
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper describes Gradient Ascent and Task Vectors as LLM unlearning methodologies applied to SemEval 2025’s task 4. This task focuses on LLM unlearning on specific information under the constraints of preserving the model’s advanced text generation capabilities; meaning that our implementations of these algorithms were constrained both in the information datasets as well as the overall effect of each algorithm in the model’s general performance. Our implementation produced modified language models that ranked 7th out of 14 valid participants in the 7B parameter model, and 6th out of 24 in the 1B parameter model.
UncleLM at SemEval-2025 Task 11: RAG-Based Few-Shot Learning and Fine-Tuned Encoders for Multilingual Emotion Detection
Authors: Mobin Barfi; Sajjad Mehrpeyma; Nasser Mozayani
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our approach for SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection. We investigate multiple methodologies, including fine-tuning transformer models and few-shot learning with GPT-4o-mini, incorporating Retrieval-Augmented Generation (RAG) for emotion intensity estimation. Our approach also leverages back-translation for data augmentation and threshold optimization to improve multi-label emotion classification. The experiments evaluate performance across multiple languages, including low-resource settings, with a focus on enhancing cross-lingual emotion detection.
UoB-NLP at SemEval-2025 Task 11: Leveraging Adapters for Multilingual and Cross-Lingual Emotion Detection
Authors: Frances Adriana Laureano De Leon; Yixiao Wang; Yue Feng; Mark Lee
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Emotion detection in natural language processing is a challenging task due to the complexity of human emotions and linguistic diversity. While significant progress has been made in high-resource languages, emotion detection in low-resource languages remains underexplored. In this work, we address multilingual and cross-lingual emotion detection by leveraging adapter-based fine-tuning with multilingual pre-trained language models. Adapters introduce a small number of trainable parameters while keeping the pre-trained model weights fixed, offering a parameter-efficient approach to adaptation. We experiment with different adapter tuning strategies, including task-only adapters, target-language-ready task adapters, and language-family-based adapters. Our results show that target-language-ready task adapters achieve the best overall performance, particularly for low-resource African languages with our team ranking 7th for Tigrinya, and 8th for Kinyarwanda. In Track C, our system ranked 5th for Oromo, Tigrinya, Kinyarwanda, Amharic, and Igbo. Our approach outperforms large language models in 11 languages and matches their performance in four others, despite using significantly fewer parameters. Furthermore, we find that adapter-based models retain cross-linguistic transfer capabilities while requiring fewer computational resources compared to full fine-tuning for each language.
GIL-IIMAS UNAM at SemEval-2025 Task 3: MeSSI: A Multilmodule System to detect hallucinated Segments in trivia-like Inquiries.
Authors: Francisco López-Ponce; Karla Salas-Jimenez; Adrián Juárez-Pérez; Diego Hernández-Bustamante; Gemma Bel-Enguix; Helena Gómez-Adorno
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: We present MeSSI, a multi-module system applied to SemEval 2025’s task 3: Mu-SHROOM. Our system tags questions in order to obtain semantic relevant terms that are used as information retrieval characteristics. Said characteristics serve as extraction terms for Wikipedia pages that are in turn processed to generate gold standard texts used in a hallucination evaluation system. A PoST-based entity comparison was implemented to contrast the test dataset sentences with the corresponding generated gold standards, wich in turn was the main criteria to tag hallucinations, partitioned in soft labels and hard labels. This method was tested in Spanish and English, finishing 18th and 19th respectively on the IoU based ranking.
Howard University-AI4PC at SemEval-2025 Task 10: Ensembling LLMs for Multi-lingual Multi-Label and Multi-Class Meta-Classification
Authors: Saurav K. Aryal; Prasun Dhungana
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper describes our approach and submission to the SemEval 2025 shared task on “Multilingual Characterization and Extraction of Narratives from Online News”. The purpose of this task was to assign primary and fine-grained roles to named entities in news articles from five different languages, on the topics of Climate Change and Ukraine-Russia War. In this paper, we explain how we approached the task by utilizing multiple LLMs via Prompt Engineering and combining their results into a final task result through an ensemble meta-classification technique. Our experimental results demonstrate that this integrated approach outperforms the provided baseline in detecting bias, deception, and manipulation in news media across multiple languages.
HU at SemEval-2025 Task 9: Leveraging LLM-Based Data Augmentation for Class Imbalance
Authors: Muhammad Saad; Meesum Abbas; Sandesh Kumar; Abdul Samad
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents a solution to the food hazard detection challenge in the SemEval-2025 Task 9, focusing on overcoming class imbalance using data augmentation techniques. We employ large language models (LLMs) like GPT-4o, Gemini Flash 1.5, and T5 to generate synthetic data, alongside other methods like synonym replacement, back-translation, and paraphrasing. These augmented datasets are used to fine-tune transformer-based models such as DistilBERT, improving their performance in detecting food hazards and categorizing products. Our approach achieves notable improvements in macro-F1 scores for both subtasks, although challenges remain in detecting implicit hazards and handling extreme class imbalance. The paper also discusses various techniques, including class weighting and ensemble modeling, as part of the training process. Despite the improvements, further work is necessary to refine hazard detection, particularly for rare and implicit categories.
FunghiFunghi at SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes
Authors: Tariq Ballout; Pieter Jansma; Nander Koops; Yong Hui Zhou
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large Language Models (LLMs) often generate hallucinated content, which is factually incorrect or misleading, posing reliability challenges. The Mu-SHROOM shared task addresses hallucination detection in multilingualLLM-generated text. This study employsSpanBERT, a transformer model optimized forspan-based predictions, to identify hallucinatedspans across multiple languages. To addresslimited training data, we apply dataset augmentation through translation and synthetic generation. The model is evaluated using Intersection over Union (IoU) for span detectionand Spearman’s correlation for ranking consistency. While the model detects hallucinatedspans with moderate accuracy, it struggles withranking confidence scores. These findings highlight the need for improved probability calibration and multilingual robustness. Future workshould refine ranking methods and explore ensemble models for better performance.
CIC-IPN at SemEval-2025 Task 11: Transformer-Based Approach to Multi-Class Emotion Detection
Authors: Tolulope Abiola; Olumide Ebenezer Ojo; Grigori Sidorov; Olga Kolesnikova; Hiram Calvo
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents a multi-step approach for multi-label emotion classification as our system description paper for the SEMEVAL-2025 workshop Task A using machine learning and deep learning models. We test our methodology on English, Spanish, and low-resource Yoruba datasets, with each dataset labeled with five emotion categories: anger, fear, joy, sadness, and surprise. Our preprocessing involves text cleaning and feature extraction using bigrams and TF-IDF. We employ logistic regression for baseline classification and fine-tune Transformer models, such as BERT and XLM-RoBERTa, for improved performance. The Transformer-based models outperformed the logistic regression model, achieving micro-F1 scores of 0.7061, 0.7321, and 0.2825 for English, Spanish, and Yoruba, respectively. Notably, our Yoruba fine-tuned model outperformed the baseline model of the task organizers with micro-F1 score of 0.092, demonstrating the effectiveness of Transformer models in handling emotion classification tasks across diverse languages.
Mr. Snuffleupagus at SemEval-2025 Task 4: Unlearning Factual Knowledge from LLMs Using Adaptive RMU
Authors: Arjun Dosajh; Mihika Sanghi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, their tendency to memorize training data raises concerns regarding privacy, copyright compliance, and security, particularly in cases involving Personally Identifiable Information (PII). Effective machine unlearning techniques are essential to mitigate these risks, yet existing methods remain underdeveloped for LLMs due to their open-ended output space. In this work, we apply the Adaptive Representation Misdirection Unlearning (RMU) technique to unlearn sensitive information from LLMs. Through extensive experiments, we analyze the effects of unlearning across different decoder layers to determine the most effective regions for sensitive information removal. Our technique ranked 4th on the official leaderboard of both 1B parameter and 7B parameter models.
CAIDAS at SemEval-2025 Task 7: Enriching Sparse Datasets with LLM-Generated Content for Improved Information Retrieval
Authors: Dominik Benchert; Severin Meßlinger; Sven Goller; Jonas Kaiser; Jan Pfister; Andreas Hotho
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The focus of SemEval-2024 Task 7 is the retrieval of relevant fact-checks for social media posts across multiple languages. We approach this task with an enhanced bi-encoder retrieval setup, which is designed to match social media posts with relevant fact-checks using synthetic data from LLMs. We explored and analyzed two main approaches for generating synthetic posts. Either based on existing fact-checks or on existing posts. Our approach achieved an S@10 score of 89.53% for the monolingual task and 74.48% for the crosslingual task, ranking 16th out of 28 and 13th out of 29, respectively. Without data augmentation, scores would have been 88.69 (17th) and 72.93 (15th).
NarrativeMiners at SemEval-2025 Task 10: Combating Manipulative Narratives in Online News
Authors: Muhammad Khubaib; Muhammad Shoaib Khursheed; Muminah Khurram; Abdul Samad; Sandesh Kumar
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Our team, Narrative Miners, participated in SemEval-2025 Task 10 to tackle the challenge of detecting manipulative narratives in online news, focusing on the Ukraine-Russia war and climate change. We worked on three key subtasks: classifying entity roles, categorizing narratives and subnarratives, and generating concise narrative explanations. Using transformer-based models like BART, BERT, GPT-2, and Flan-T5, we implemented a structured pipeline and applied data augmentation to enhance performance. BART-CNN proved to be our best-performing model, significantly improving classification accuracy and explanation generation. Despite challenges like dataset limitations and class imbalance, our approach demonstrated the effectiveness of hierarchical classification and multilingual analysis in combating online disinformation. We made use of different data augmentation techniques to cover the class imbalances present in the dataset. We had different evaluation metrics set for each subtask, specifically focusing on the need of that particular outcome. With this paper, we hope to play our part in mitigating the impact of harmful disinformation.
Howard University-AI4PC at SemEval-2025 Task 11: Combining Expert Personas via Prompting for Enhanced Multilingual Emotion Analysis
Authors: Amir Ince; Saurav K. Aryal
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: For our approach to SemEval-2025 Task 11, we employ a multi-tier evaluation framework for perceived emotion analysis. Our system consists of a smaller-parameter-size large language model that independently predicts a given text’s perceived emotion while explaining the reasoning behind its decision. The initial model’s persona is varied through careful prompting, allowing it to represent multiple perspectives. These outputs, including both predictions and reasoning, are aggregated and fed into a final decision-making model that determines the ultimate emotion classification. We evaluated our approach in official SemEval Task 11 on subtasks A and C in all the languages provided.
Habib University at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection
Authors: Owais Waheed; Hammad Sajid; Kushal Chandani; Muhammad Areeb Kazmi; Sandesh Kumar; Abdul Samad
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Emotion detection in text has emerged as a pivotal challenge in Natural Language Processing (NLP), particularly in multilingual and cross-lingual contexts. This paper presents our participation in SemEval 2025 Task 11, focusing on three subtasks: Multi-label Emotion Detection, Emotion Intensity Prediction, and Cross-lingual Emotion Detection. Leveraging state-of-the-art transformer models such as BERT and XLM-RoBERTa, we implemented baseline models and ensemble techniques to enhance predictive accuracy. Additionally, innovative approaches like data augmentation and translation-based cross-lingual emotion detection were used to address linguistic and class imbalances. Our results demonstrated significant improvements in F1 scores and Pearson correlations, showcasing the effectiveness of ensemble learning and transformer-based architectures in emotion recognition. This work advances the field by providing robust methods for emotion detection, particularly in low-resource and multilingual settings.
NLP-Cimat at SemEval-2025 Task 11: Prompt Optimization for LLMs via Genetic Algorithms and Systematic Mutation applied on Emotion Detection
Authors: Guillermo Segura-Gómez; Adrian Pastor Lopez Monroy; Fernando Sanchez - Vega; Alejandro Rosales Pérez
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large Language Models (LLMs) have shown remarkable performance across diverse natural language processing tasks in recent years. However, optimizing instructions to maximize model performance remains a challenge due to the vast search space and the nonlinear relationship between input structure and output quality. This work explores an alternative prompt optimization technique based on genetic algorithms with different structured mutation processes. Unlike traditional random mutations, our method introduces variability in each generation through a guided mutation, enhancing the likelihood of obtaining better prompts for each generation. We apply this approach to emotion detection in the context of SemEval 2025 Task 11, demonstrating the potential to improve prompt efficiency, and consequently task performance. Experimental results show that our method yields competitive results compared to standard optimization techniques while maintaining interpretability and scalability.
WC Team at SemEval-2025 Task 6: PromiseEval: Multinational, Multilingual, Multi-Industry Promise Verification leveraging monolingual and multilingual BERT models
Authors: Takumi Nishi; Nicole Miu Takagi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our system developed for SemEval-2025 Task 6: PromiseEval: Multinational, Multilingual, Multi-Industry Promise Verification. The task aims at identifying “promises” made and “evidence” provided in company ESG statements for various languages. Our team participated in Subtasks 1 and 2 for the languages English, French, and Japanese. In this work, we propose using BERT and finetuning it to better address the task. We achieve competitive results, especially for English and Japanese.
Predicting Emotion Intensity in Text Using Transformer-Based Models
Authors: Temitope Oladepo; Oluwatobi Abiola; Tolulope Abiola; Abdullah -; Usman Muhammad; Babatunde Abiola
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Emotion intensity prediction in text enhances conversational AI by enabling a deeper understanding of nuanced human emotions, a crucial yet underexplored aspect of natural language processing (NLP). This study employs Transformer-based models to classify emotion intensity levels (0–3) for five emotions: anger, fear, joy, sadness, and surprise. The dataset, sourced from the SemEval shared task, was preprocessed to address class imbalance, and model training was performed using fine-tuned *bert-base-uncased*. Evaluation metrics showed that *sadness* achieved the highest accuracy (0.8017) and F1-macro (0.5916), while *fear* had the lowest accuracy (0.5690) despite a competitive F1-macro (0.5207). The results demonstrate the potential of Transformer-based models in emotion intensity prediction while highlighting the need for further improvements in class balancing and contextual representation.