GenAI4ELab/papercli-papers huggingface dataset:
Howard University-AI4PC at SemEval-2025 Task 1: Using GPT-4o and CLIP-ViLT to Decode Figurative Language Across Text and Images
Authors: Saurav K. Aryal; Lawal Abdulmujeeb
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Correctly identifying idiomatic expressions remains a major challenge in Natural Language Processing (NLP), as these expressions often have meanings that cannot be directly inferred from their individual words. The SemEval-2025 Task 1 introduces two subtasks, A and B, designed to test models’ ability to interpret idioms using multimodal data, including both text and images. This paper focuses on Subtask A, where the goal is to determine which among several images best represents the intended meaning of an idiomatic expression in a given sentence.To address this, we employed a two-stage approach. First, we used GPT-4o to analyze sentences, extracting relevant keywords and sentiments to better understand the idiomatic usage. This processed information was then passed to a CLIP-VIT model, which ranked the available images based on their relevance to the idiomatic expression. Our results showed that this approach performed significantly better than directly feeding sentences and idiomatic compounds into the models without preprocessing. Specifically, our method achieved a Top-1 accuracy of 0.67 in English, whereas performance in Portuguese was notably lower at 0.23. These findings highlight both the promise of multimodal approaches for idiom interpretation and the challenges posed by language-specific differences in model performance.
CSECU-DSG at SemEval-2025 Task 6: Exploiting Multilingual Feature Fusion-based Approach for Corporate Promise Verification
Authors: Tashin Hossain; Abu Nowshed Chy
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In SemEval-2025, we participated on the multilingual corporate promise verification task. In the task, we mainly focused on the promise and evidence identification task, and illustrated the performance for the five different languages. For all the languages, we proposed a unified state-of-the-art framework to classify the target labels. For the framework, we incorporated the pre-feature fusion approach, then integrate it with the neural network architecture. Additionally, in the dataset description and discussion section, we provide different insights of our finding through visualization of the dataset structures and explainability of the model’s performance.
Exploration Lab IITK at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection
Authors: Tafazzul Nadeem; Riyansha Singh; Suyamoon Pathak; Ashutosh Modi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our approach to SemEval-2025 Task 11 (Track A): Bridging the Gap in Text-Based Emotion Detection, with a focus on multi-label emotion classification for the English dataset. Our methodology leverages an ensemble of transformer-based models, incorporating full fine-tuning along with additional classification layers to enhance predictive performance. Through extensive experimentation, we demonstrate that fine-tuning significantly improves emotion classification accuracy compared to baseline models. Furthermore, we provide an in-depth analysis of the dataset, highlighting key patterns and challenges. The study also evaluates the impact of ensemble modeling on performance, demonstrating its effectiveness in capturing nuanced emotional expressions. Finally, we outline potential directions for further refinement and domain-specific adaptations to enhance model robustness.
Stanford MLab at SemEval-2025 Task 11: Track B–Emotion Intensity Detection
Authors: Joseph Le; Hannah Cui; James Zhang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: We outline our SemEval 2025 Track B: Emotion Intensity Prediction submission, for which the objective is to predict the intensity of six primary emotions—anger, disgust, fear, joy, sadness, and surprise—between 0 and 3, with 0 being none and 3 being very strong. We used a regression fine-tuned BERT-based model that makes use of pretrained embeddings in order to sense subtle emotional wordings in text.We include tokenization with a BERT tokenizer, training with AdamW optimization, and an ExponentialLR scheduler used for learning rate modification. Performance is monitored based on validation loss and accuracy through closeness of model outputs to gold labels.Our best-performing model is 68.97% accurate in validation and has a validation loss of 0.373, demonstrating BERT’s capability in fine-grained emotion intensity prediction. Key findings include that fine-tuning transformer models with regression loss improves prediction accuracy and that early stopping and learning rate scheduling avoid overfitting.Future improvements can include larger datasets, ensemble models, or other architectures such as RoBERTa and T5. This paper shows the potential of pretrained transformers for emotion intensity estimation and lays the groundwork for future computational emotion analysis research.
Team Anotheroption at SemEval-2025 Task 8: Bridging the Gap Between Open-Source and Proprietary LLMs in Table QA
Authors: Nikolas Evkarpidi; Elena Tutubalina
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents a system developed for SemEval 2025 Task 8: Question Answering (QA) over tabular data. Our approach integrates several key components: text-to-SQL and text-to-Code generation modules, a self-correction mechanism, and a retrieval-augmented generation (RAG). Additionally, it includes an end-to-end (E2E) module, all orchestrated by a large language model (LLM). Through ablation studies, we analyzed the effects of different parts of our pipeline and identified the challenges that are still present in this field. During the evaluation phase of the competition, our solution achieved an accuracy of 80%, resulting in a top-13 ranking among the 38 participating teams. Our pipeline demonstrates a significant improvement in accuracy for open-source models and achieves a performance comparable to proprietary LLMs in QA tasks over tables.
Howard University-AI4PC at SemEval-2025 Task 2: Improving Machine Translation With Context-Aware Entity-Only Pre-translations with GPT4o
Authors: Saurav K. Aryal; Jabez Agyemang - Prempeh
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our work on a 3-Step GPT translation system developed for SemEval-2025 Task 2 to enhance the translation of named entities within machine translation. Our approach integrates (1) entity extraction via wikidata, (2) GPT-based refinement of entity translations, and (3) final context-aware GPT translation. Results from the original dataset of six languages show significant improvements in the handling of named entities compared to direct GPT-based translation baselines. We further discuss replicability, observed challenges, and outline future research directions.
Zero_Shot at SemEval-2025 Task 11: Fine-Tuning Deep Learning and Transformer-based Models for Emotion Detection in Multi-label Classification, Intensity Estimation, and Cross-lingual Adaptation
Authors: Ashraful Islam Paran; Sabik Aftahee; Md. Refaj Hossan; Jawad Hossain; Mohammed Moshiul Hoque
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Language is a rich medium employed to convey emotions subtly and intricately, as abundant as human emotional experiences themselves. Emotion recognition in natural language processing (NLP) is now a core element in facilitating human-computer interaction and interpreting intricate human behavior via text. It has potential applications in every sector i.e., sentiment analysis, mental health surveillance. However, prior research on emotion recognition is primarily from high-resource languages while low-resource languages (LRLs) are not well represented. This disparity has been a limitation to the development of universally applicable emotion detection models. To address this, the SemEval-2025 Shared Task 11 focused on perceived emotions, aiming to identify the emotions conveyed by a text snippet. It includes three tracks: Multi-label Emotion Detection (Track A), Emotion Intensity (Track B), and Cross-lingual Emotion Detection (Track C). This paper explores various models, including machine learning (LR, SVM, RF, NB), deep learning (BiLSTM+CNN, BiLSTM+BiGRU), and transformer-based models (XLM-R, mBERT, ModernBERT). The results showed that XLM-R outperformed other models in Tracks A and B, while BiLSTM+CNN performed better for Track C across most languages.
YNU-HPCC at SemEval-2025 Task 6: Using BERT Model with R-drop for Promise Verification
Authors: Dehui Deng; You Zhang; Jin Wang; Dan Xu; Xuejie Zhang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our participation in the SemEval-2025 task 6: multinational, multilingual, multi-industry promise verification. The SemEval-2025 Task 6 aims to extract Promise Identification, Supporting Evidence, Clarity of the Promise-Evidence Pair, and Timing for Verification from the commitments made to businesses and governments. Use these data to verify whether companies and governments have fulfilled their commitments. In this task, we participated in the English task, whichincluded analysis of numbers in the text, reading comprehension of the text content and multi-label classification. Our model introduces regularization dropout based on Bert-base to focus on the stability of non-target classes, improve the robustness of the model, and ultimately improve the indicators. Our approach obtained competitive results in subtasks.
PATeam at SemEval-2025 Task 9: LLM-Augmented Fusion for AI-Driven Food Safety Hazard Detection
Authors: Xue Wan; Fengping Su; Ling Sun; Yuyang Lin; Pengfei Chen
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper introduces the approach we adopted for the SemEval-2025 “Food Hazard Detection” task, which aims to predict coarse-grained categories (such as “product category” and “hazard category”) and fine-grained vectors (such as specific products like “ice cream” or hazards like “salmonella”) from noisy, long-tailed text data.To address the issues of dirty data, as well as the severe long-tail distribution of text labels and length in the data, we proposed a pipeline system. This system combines data cleaning, LLM-based enhancement, label resampling, and ensemble learning to tackle data sparsity and label imbalance problems.The two subtasks have strong semantic relatedness. By integrating them into a unified multiturn dialogue framework, we fine-tuned five models using a bagging approach. Ultimately, we achieved good results in both subtasks, ranking 5th (with an F1 score of 80.17% for ST1 and 52.66% for ST2).
Howard University-AI4PC at SemEval-2025 Task 9: Using Open-weight BART-MNLI for Zero Shot Classification of Food Recall Documents
Authors: Saurav K. Aryal; Kritika Pant
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: We present our system for SemEval-2025 Task 9: Food Hazard Detection, a shared task focused on the explainable classification of food-incident reports. The task involves predicting hazard and product categories (ST1) and their exact vectors (ST2) from short texts. Our approach leverages zero-shot classification using the BART-large-MNLI model, which allows classification without task-specific fine-tuning. Our model achieves competitive performance, emphasizing hazard prediction accuracy, as evaluated by the macro-F1 score.
Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models
Authors: Enfa Fane; Mihai Surdeanu; Eduardo Blanco; Steven Corman
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Understanding how news narratives frame entities is crucial for studying media’s impact on societal perceptions of events. In this paper, we evaluate the zero-shot capabilities of large language models (LLMs) in classifying framing roles. Through systematic experimentation, we assess the effects of input context, prompting strategies, and task decomposition. Our findings show that a hierarchical approach of first identifying broad roles and then fine-grained roles, outperforms single-step classification. We also demonstrate that optimal input contexts and prompts vary across task levels, highlighting the need for subtask-specific strategies. We achieve a Main Role Accuracy of 89.4% and an Exact Match Ratio of 34.5%, demonstrating the effectiveness of our approach. Our findings emphasize the importance of tailored prompt design and input context optimization for improving LLM performance in entity framing.
UB_Tel-U at SemEval-2025 Task 11: Emotions Without Borders - A Unified Framework for Multilingual Classification Using Augmentation and Ensemble
Authors: Tirana Noor Fatyanosa; Putra Pandu Adikara; Rochmanu Purnomohadi Erfitra; Muhammad Rajendra Alkautsar Dikna; Sari Dewi Budiwati; Cahyana
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In this SemEval 2025 Task 11 paper, we tackled three tracks: Multi-label Emotion Detection, Emotion Intensity, and Cross-lingual Emotion Detection. Our approach harnesses diverse external corpora and robust data augmentation techniques across Spanish, English, and Arabic, enhancing both the diversity and resilience of the dataset. Instead of developing separate models for each language, we merge the data into a unified multilingual dataset, enabling our model to learn cross-lingual patterns and relationships simultaneously. Our ensemble architecture integrates the multilingual strengths of XLM-RoBERTa, a zero-shot classification capability via LLaMA 3, and a specialized pretrained model fine-tuned on English emotion classification. Notably, our system achieved strong performance, ranking 13th for Afrikaans (afr) in Track A, 13th for Amharic (amh) in Track B, and 4th for Hindi (hin) in Track C.
Deep at SemEval-2025 Task 11: A Multi-Stage Approach to Emotion Detection
Authors: Dong Shenpo
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents a novel text-based emotion detection approach for low-resource languages in SemEval-2025 Task 11. We fine-tuned Google Gemma 2 using tailored data augmentation and Chain-of-Thought prompting. Our method, incorporating supervised fine-tuning and model ensembling, significantly improved multi-label emotion recognition, intensity prediction, and cross-lingual performance. Results show strong performance in diverse low-resource settings. Challenges remain in fine-grained sentiment analysis. Future work will explore advanced data augmentation and knowledge transfer methods. This research demonstrates the potential of large language models for inclusive emotion analysis.
TartanTritons at SemEval-2025 Task 10: Multilingual Hierarchical Entity Classification and Narrative Reasoning using Instruct-Tuned LLMs
Authors: Raghav R; Adarsh Prakash Vemali; Darpan Aswal; Rahul Ramesh; Parth Tusham; Pranaya Rishi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In today’s era of abundant online news, tackling the spread of deceptive content and manipulative narratives has become crucial. This paper details our system for SemEval-2025 Task 10, focusing on Subtasks 1 (Entity Framing) and 3 (Narrative Extraction). We instruct-tuned quantized Microsoft’s Phi-4 model, incorporating prompt engineering techniques to enhance performance. Our approach involved experimenting with various LLMs, including LLaMA, Phi-4, RoBERTa, and XLM-R, utilizing both quantized large models and non-quantized small models. To improve accuracy, we employed structured prompts, iterative refinement with retry mechanisms, and integrated label taxonomy information. For subtask 1, we also fine-tuned a RoBERTa classifier to predict main entity roles before classifying the fine-grained roles with Phi-4 for the English language. For subtask 3, we instruct-tuned Phi-4 to generate structured explanations, incorporating details about the article and its dominant narrative. Our system achieves competitive results in Hindi and Russian for Subtask 1.
TreeSearch at SemEval-2025 Task 8: Monte Carlo Tree Search for Question-Answering over Tabular Data
Authors: Aakarsh Nair; Huixin Yang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large Language Models (LLMs) can answer diverse questions but often generate factually incorrect responses. SemEval-2025 Task 8 focuses on table-based question-answering, providing 65 real-world tabular datasets and 1,300 questions that require precise filtering and summarization of underlying tables.We approach this problem as a neuro-symbolic code generation task, translating natural language queries into executable Python code to ensure contextually relevant and factually accurate answers. We formulate LLM decoding as a Markov Decision Process, enabling Monte Carlo Tree Search (MCTS) as a lookahead-based planning algorithm while decoding from the underlying code-generating LLM, instead of standard beam search.Execution success on synthetic tests and real datasets serves as a reward signal, allowing MCTS to explore multiple code-generation paths, validate outcomes, assign value to partial solutions, and refine code iteratively rather than merely maximizing sequence likelihood in a single step. Our approach improves accuracy by 2.38x compared to standard decoding.
UCSC at SemEval-2025 Task 3: Context, Models and Prompt Optimization for Automated Hallucination Detection in LLM Output
Authors: Sicong Huang; Jincheng He; Shiyuan Huang; Karthik Raja Anandan; Arkajyoti Chakraborty; Ian Lane
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Hallucinations pose a significant challenge for large language models when answering knowledge-intensive queries. As LLMs become more widely adopted, it is crucial not only to detect if hallucinations occur but also to pinpoint where they arise. SemEval 2025 Task 3, Mu-SHROOM: Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes, is a recent effort in this direction. This paper describes our solution to the shared task. We propose a framework that first retrieves relevant context, next identifies false content from the answer, and finally maps them back to spans. The process is further enhanced by automatically optimizing prompts. Our system achieves the highest overall performance, ranking #1 in average position across all languages.
YNU-HPCC at SemEval-2025 Task 10: A Two-Stage Approach to Solving Multi-Label and Multi-Class Role Classification Based on DeBERTa
Authors: Ning Li; You Zhang; Jin Wang; Dan Xu; Xuejie Zhang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: A two-stage role classification model based on DeBERTa is proposed for the Entity Framework task in SemEval 2025 Task 10. The task is confronted with challenges such as multi-labeling, multi-category, and category imbalance, particularly in the semantic overlap and data sparsity of fine-grained roles. Existing methods primarily rely on rules, traditional machine learning, or deep learning, but the accurate classification of fine-grained roles is difficult to achieve. To address this, the proposed model integrates the deep semantic representation of the DeBERTa pre-trained language model through two sub-models: main role classification and sub-role classification, and utilizes Focal Loss to optimize the category imbalance issue. Experimental results indicate that the model achieves an accuracy of 75.32% in predicting the main role, while the exact matching rate for the sub-role is 8.94%. This is mainly limited by the strict matching standard and semantic overlap of fine-grained roles in the multi-label task. Compared to the baseline’s sub-role exact matching rate of 3.83%, the proposed model significantly improves this metric. The model ultimately ranked 23rd on the leaderboard. The code of this paper is available at:https://github.com/jiyuaner/YNU-HPCC-at-SemEval-2025-Task10.
Empaths at SemEval-2025 Task 11: Retrieval-Augmented Approach to Perceived Emotions Prediction
Authors: Lev Morozov; Aleksandr Mogilevskii; Alexander Shirnin
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The paper introduces EmoRAG, a retrieval-augmented emotion detection system designed for the SemEval-2025 Task 11. It uses an ensemble of models, retrieving similar examples to prompt large language models (LLMs) for emotion predictions. The retriever component fetches the most relevant examples from a database, which are then used as few-shot prompts for the models. EmoRAG achieves strong, scalable performance across languages with no training at all, demonstrating effectiveness in both high and low-resource languages.
IUST_Champs at SemEval-2025 Task 8: Structured Prompting and Retry Policy for Tabular Question Answering
Authors: Arshia Hossein Zadeh; Aysa Mayahinia; Nafiseh Ahmadi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents a novel approach to Question Answering over Tabular Data, as part of SemEval-2025 Task 8. Our system generates executable Python code to derive answers directly from structured data, leveraging open-source large language models. Key innovations include structured prompting, semantic column filtering, and a one-time retry mechanism to enhance accuracy and robustness. We evaluate our approach on the DataBench and DataBench_Lite datasets, significantly outperforming the baseline accuracy (26-27%) with our best system achieving 70.49% accuracy on the test set. Ablation studies confirm that few-shot prompting and rule-based type classification are crucial for improved performance. Despite these advancements, challenges remain in handling complex table structures and ambiguous queries. Our findings highlight the effectiveness of code-generation based methods for tabular question answering and provide insights for further research in this area.
HausaNLP at SemEval-2025 Task 11: Advancing Hausa Text-based Emotion Detection
Authors: Sani Abdullahi Sani; Salim Abubakar; Falalu Ibrahim Lawan; Abdulhamid Abubakar; Maryam Bala
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our approach to multi-label emotion detection in Hausa, a low-resource African language, as part of SemEval Track A. We fine-tuned AfriBERTa, a transformer-based model pre-trained on African languages, to classify Hausa text into six emotions: anger, disgust, fear, joy, sadness, and surprise. Our methodology involved data preprocessing, tokenization, and model fine-tuning using the Hugging Face Trainer API. The system achieved a validation accuracy of 74.00%, with an F1-score of 73.50%, demonstrating the effectiveness of transformer-based models for emotion detection in low-resource languages.