GenAI4ELab/papercli-papers huggingface dataset:
NLP_goats at SemEval-2025 Task 11: Multi-Label Emotion Classification Using Fine-Tuned Roberta-Large Tranformer
Authors: Vijay Karthick Vaidyanathan; Srihari V K; Mugilkrishna D U; Saritha Madhavan
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper serves as a solution for multi-label emotion classification and intensity for text, developed for SemEval-2025 Task 11. The method uses a fine-tuned RoBERTa-Large transformer model. The system represents a multi-label classification approach to identifying multiple emotions, and uses regression models to estimate emotion strength. The model performed with ranks of 31st and 17th place in the corresponding tracks. The findings show impressive performance and it remains possible to improve the performance of ambiguous or low-frequency emotion recognition using the state-of-the-art contextual embeddings and threshold optimization techniques.
Firefly Team at SemEval-2025 Task 8: Question-Answering over Tabular Data using SQL/Python generation with Closed-Source Large Language Models
Authors: Ho Thuy Nga; Ho Thi Thanh Tuyen; Le Minh Hung; Dang Van Thin
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In this paper, we describe our official system of the Firefly team for two main tasks in the SemEval-2025 Task 8: Question-Answering over Tabular Data. Our solution employs large language models (LLMs) to translate natural language queries into executable code, specifically Python and SQL, which are then used to generate answers categorized into five predefined types. Our empirical evaluation highlights the superiority of Python code generation over SQL for this challenge. Besides, the experimental results show that our system has achieved competitive performance in two subtasks. In Subtask I: Databench QA, where we rank the Top 9 across datasets of any size. Besides, our solution achieved competitive results and ranked 5th place in Subtask II: Databench QA Lite, where datasets are restricted to a maximum of 20 rows.
SmurfCat at SemEval-2025 Task 3: Bridging External Knowledge and Model Uncertainty for Enhanced Hallucination Detection
Authors: Elisei Rykov; Valerii Olisov; Maksim Savkin; Artem Vazhentsev; Kseniia Titova; Alexander Panchenko; Vasily Konovalov; Julia Belikova
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The Multilingual shared-task on Hallucinations and Related Observable Overgeneration Mistakes in the SemEval-2025 competition aims to detect hallucination spans in the outputs of instruction-tuned LLMs in a multilingual context. In this paper, we address the detection of span hallucinations by applying an ensemble of approaches. In particular, we synthesized a PsiloQA dataset and fine-tuned LLM to detect hallucination spans. In addition, we combined this approach with a white-box method based on uncertainty quantification techniques. Using our combined pipeline, we achieved 3rd place in detecting span hallucinations in Arabic, Catalan, Finnish, Italian, and ranked within the top ten for the rest of the languages.
NeuroReset : LLM Unlearning via Dual Phase Mixed Methodology
Authors: Dhwani Bhavankar; Het Sevalia; Shubh Agarwal; Yogesh Kulkarni; Rahee Walambe; Ketan Kotecha
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents the method for the unlearning of sensitive information from large language models as applied in the SemEval 2025 Task 4 challenge. The unlearning pipeline consists of two phases. In phase I, the model is instructed to forget specific datasets, and in phase II, the model is stabilized using a retention dataset. Unlearning with these methods secured a final score of 0.420 with the 2nd honorary mention in the 7B parameter challenge and a score of 0.36 in the 13th position for the 1B parameter challenge. The paper presents a background study, a brief literature review, and a gap analysis, as well as the methodology employed in our work titled NeuroReset. The training methodology and evaluation metrics are also presented, and the trade-offs between unlearning efficiency and model performance are discussed. The contributions of the paper are systematic unlearning, a comparative analysis of unlearning methods, and an empirical analysis of model performance post-unlearning.
Team QUST at SemEval-2025 Task 10: Evaluating Large Language Models in Multiclass Multi-label Classification of News Entity Framing
Authors: Jiyan Liu; Youzheng Liu; Taihang Wang; Xiaoman Xu; Yimin Wang; Ye Jiang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper introduces the participation of the QUST team in subtask 1 of SemEval-2025 Task 10. We evaluate various large language models (LLMs) based on instruction tuning (IT) on subtask 1. Specifically, we first analyze the data statistics, suggesting that the imbalance of label distribution made it difficult for LLMs to be fine-tuned. Subsequently, a voting mechanism is utilized on the predictions of the top-3 models to derive the final submission results. The team participated in all language tracks, achieving 1st place in Hindi (HI), 2nd in Russian (RU), 3rd in Portuguese (PT), 6th in Bulgarian (BG), and 7th in English (EN) on the official test set. We release our system code at: https://github.com/warmth27/SemEval2025_Task10
AGHNA at SemEval-2025 Task 11: Predicting Emotion and Its Intensity within a Text with EmoBERTa
Authors: Moh. Abyan
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our system that have been developed for SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection. The system is able to do two sub-tasks: Track A, related to detecting emotion(s) in a given text; Track B, related to calculate intensity of emotion(s) in a given text. The system will have EmoBERTa as the model baseline, despite some minor differences used in the system approach between these tracks. With the system designed above, Track A achieved a Macro-F1 Score of 0.7372, while Track B achieved Average Pearson r Score of 0.7618.
TUM-MiKaNi at SemEval-2025 Task 3: Towards Multilingual and Knowledge-Aware Non-factual Hallucination Identification
Authors: Miriam Anschütz; Ekaterina Gikalo; Niklas Herbster; Georg Groh
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Hallucinations are one of the major problems of LLMs, hindering their trustworthiness and deployment to wider use cases. However, most of the research on hallucinations focuses on English data, neglecting the multilingual nature of LLMs. This paper describes our submission to the "{textit{SemEval-2025 Task-3 — Mu-SHROOM, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes}}”. We propose a two-part pipeline that combines retrieval-based fact verification against Wikipedia with a BERT-based system fine-tuned to identify common hallucination patterns. Our system achieves competitive results across all languages, reaching top-10 results in eight languages, including English. Moreover, it supports multiple languages beyond the fourteen covered by the shared task. This multilingual hallucination identifier can help to improve LLM outputs and their usefulness in the future.
NITK-VITAL at SemEval-2025 Task 11: Focal-RoBERTa: Addressing Class Imbalance in Multi-Label Emotion Classification
Authors: Ashinee Kesanam; Gummuluri Venkata Ravi Ram; Chaithanya Swaroop Banoth; G Rama Mohana Reddy
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our approach to SemEval Task 11, which focuses on multi-label emotion detection in English textual data. We experimented with multiple methodologies, including traditional machine learning models, deep learning architectures, and transformer-based models. Our best-performing approach employed RoBERTa with focal loss, which effectively mitigated class imbalances and achieved a macro F1-score of 0.7563, outperforming other techniques. Comparative analyses between different embedding strategies, such as TF-IDF, BERT, and MiniLM, revealed that transformer-based models consistently provided superior performance. The results demonstrate the effectiveness of focal loss in handling highly skewed emotion distributions. Our system contributes to advancing multi-label emotion detection by leveraging robust pre-trained models and loss function optimization.
CharsiuRice at SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection
Authors: Hiu Yan Yip; Hing Man Chiu; Hai - Yin Yang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our participation in SemEval-2025 Task 11, which focuses on bridging the gap in text-based emotion detection. Our team took part in both Tracks A and B, addressing different aspects of emotion classification. We fine-tuned a RoBERTa base model on the provided dataset in Track A, achieving a Macro-F1 score of 0.7264. For Track B, we built on top of the Track A model by incorporating an additional non-linear layer, in the hope of enhancing Track A model’s understanding of emotion detection. Track B model resulted with an average Pearson’s R of 0.5658. The results demonstrate the effectiveness of fine-tuning in Track A and the potential improvements from architectural modifications in Track B for emotion intensity detection tasks.
Deloitte (Drocks) at SemEval-2025 Task 3: Fine-Grained Multi-lingual Hallucination Detection Using Internal LLM Weights
Authors: Alex Chandler; Harika Abburi; Sanmitra Bhattacharya; Edward Bowen; Nirmala Pudota
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large Language Models (LLMs) have greatly advanced the field of Natural Language Generation (NLG). Despite their remarkable capabilities, their tendency to hallucinate—producing inaccurate or misleading information-remains a barrier to wider adoption. Current hallucination detection methods mainly employ coarse-grained binary classification at the sentence or document level, overlooking the need for precise identification of the specific text spans containing hallucinations. In this paper, we proposed a methodology that generates supplementary context and processes text using an LLM to extract internal weights (features) from various layers. These extracted features serve as input for a neural network classifier designed to perform token-level binary detection of hallucinations. Subsequently, we map the resulting token-level predictions to character-level predictions, enabling the identification of spans of hallucinated text, which we refer to as hallucination spans. Our model achieved a top-ten ranking in 13 of the 14 languages and secured first place for the French language in the SemEval: Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes (Mu-SHROOM), utilizing the Mu-SHROOM dataset provided by the task organizers.
ATLANTIS at SemEval-2025 Task 3 : Detecting Hallucinated Text Spans in Question Answering
Authors: Catherine Kobus; Francois Lancelot; Marion - Cecile Martin; Nawal Ould Amer
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents the contributions of the ATLANTIS team to SemEval-2025 Task 3, focusing on detecting hallucinated text spans in question answering systems. Large Language Models (LLMs) have significantly advanced Natural Language Generation (NLG) but remain susceptible to hallucinations, generating incorrect or misleading content. To address this, we explored methods both with and without external context, utilizing few-shot prompting with a LLM, token-level classification or LLM fine-tuned on synthetic data. Notably, our approaches achieved top rankings in Spanish and competitive placements in English and German. This work highlights the importance of integrating relevant context to mitigate hallucinations and demonstrate the potential of fine-tuned models and prompt engineering.
Dianchi at SemEval-2025 Task 11: Multilabel Emotion Recognition via Orthogonal Knowledge Distillation
Authors: Zhenlan Wang; Jiaxuan Liu
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents KDBERT-MLDistill, a novel framework for multi-label emotion recognition developed for SemEval-2025 Task 11. Addressing challenges of fine-grained emotion misdetection and small-data overfitting, the method synergizes BERT-based text encoding with orthogonal knowledge distillation. Key innovations include: (1) Orthogonal regularization on classifier weights to minimize redundant feature correlations, coupled with dynamic pseudo-labeling for periodic data augmentation; (2) A hierarchical distillation mechanism where dual teacher-student models iteratively exchange parameters to balance knowledge retention and exploration.
zhouyijiang1 at SemEval-2025 Task 11: A Multi-tag Detection Method based on Pre-training Language Models
Authors: Zhou Jiang; Dengtao Zhang
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In order to effectively predict the speaker’s informing emotion from text fragments, we propose a transfer learning framework based on the BERT pre-training model through deep semantic feature extraction and cascade structure of dynamic weight linear classifier. In the speaker informing emotion prediction task, a 0.70 F1 score is achieved, illustrating the effectiveness of cross-domain emotion recognition.
DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing
Authors: Lisa Kluge; Maximilian Kähler
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper presents our system developed for the SemEval-2025 Task 5: LLMs4Subjects: LLM-based Automated Subject Tagging for a National Technical Library’s Open-Access Catalog.Our system relies on prompting a selection of LLMs with varying examples of intellectually annotated records and asking the LLMs to similarly suggest keywords for new records. This few-shot prompting technique is combined with a series of post-processing steps that map the generated keywords to the target vocabulary, aggregate the resulting subject terms to an ensemble vote and, finally, rank them as to their relevance to the record.Our system is fourth in the quantitative ranking in the all-subjects track, but achieves the best result in the qualitative ranking conducted by subject indexing experts.
NYCU-NLP at SemEval-2025 Task 11: Assembling Small Language Models for Multilabel Emotion Detection and Intensity Prediction
Authors: Zhe - Yu Xu; Yu - Hsin Wu; Lung - Hao Lee
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This study describes the design of the NYCU-NLP system for the SemEval-2025 Task 11 that focuses on multi-lingual text-based emotion analysis. We instruction-tuned three small language models: Gemma-2 (27B), Mistral-small-3 (22B), and Phi-4 (14B) and then assembled them as our main system architecture. Our NYCU-NLP system participated the English Track A for multilabel emotion detection and English Track B for emotion intensity prediction. Experimental results show our best-performing submission produced a macro-averaging F1 score of 0.8225, ranking second of 90 participating teams for Track A, and ranked second among 41 teams for Track B with a Pearson correlation coefficient of 0.8373.
PAI at SemEval-2025 Task 11: A Large Language Model Ensemble Strategy for Text-Based Emotion Detection
Authors: Zhihao Ruan; Runyang You; Kaifeng Yang; Junxin Lin; Wenwen Dai; Mengyuan Zhou; Meizhi Jin; Xinyue Mei
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: This paper describes our system used in the SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection. To address the highly subjective nature of emotion detection tasks, we propose a model ensemble strategy designed to capture the varying subjective perceptions of different users towards textual content. The base models of this ensemble strategy consist of several large language models, which are then combined using methods such as neural networks, decision trees, linear regression, and weighted voting. In Track A, out of 28 languages, our system achieved first place in 19 languages. In Track B, out of 11 languages, our system ranked first in 10 languages. Furthermore, our system attained the highest average performance across all languages in both Track A and Track B.
FENJI at SemEval-2025 Task 3: Retrieval-Augmented Generation and Hallucination Span Detection
Authors: Flor Alberts; Ivo Bruinier; Nathalie Palm; Justin Paetzelt; Erik Varecha
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Large Language Models (LLMs) have significantly advanced Natural Language Processing, however, ensuring the factual reliability of these models remains a challenge, as they are prone to hallucination - generating text that appears coherent but contains innacurate or unsupported information. SemEval-2025 Mu-SHROOM focused on character-level hallucination detection in 14 languages. In this task, participants were required to pinpoint hallucinated spans in text generated by multiple instruction-tuned LLMs. Our team created a system that leveraged a Retrieval-Augmented Generation (RAG) approach and prompting a FLAN-T5 model to identify hallucination spans. Despite contradicting prior literature, our approach yielded disappointing results, underperforming all the “mark-all” baselines and failing to achieve competitive scores. Notably, removing RAG improved performance. The findings highlight that while RAG holds potential for hallucination detection, its effectiveness is heavily influenced by the retrieval component’s context-awareness. Enhancing the RAG’s ability to capture more comprehensive contextual information could improve performance across languages, making it a more reliable tool for identifying hallucination spans.
GUIR at SemEval-2025 Task 4: Adaptive Weight Tuning with Gradual Negative Matching for LLM Unlearning
Authors: Hrishikesh Kulkarni; Nazli Goharian; Ophir Frieder
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Machine Unlearning for Large Language Models, referred to as LLM Unlearning is getting more and more attention as a result of regurgitation of sensitive and harmful content. In this paper, we present our method architecture, results, and analysis of our submission to Task4: Unlearning sensitive content from Large Language Models. This task includes three subtasks of LLM Unlearning on 1) Long Synthetic documents, 2) Short Synthetic documents, and 3) Real Training documents. Getting rid of the impact of undesirable and unauthorized responses is the core objective of unlearning. Furthermore, it is expected that unlearning should not have an adverse impact on the usability of the model. In this paper, we provide an approach for LLM unlearning that tries to make the model forget while maintaining usability of the model. We perform adaptive weight tuning with Gradient Ascent, KL minimization and Gradual Negative Matching loss functions. Our submission balances retain and forget abilities of the model while outperforming provided benchmarks.
ipezoTU at SemEval-2025 Task 7: Hybrid Ensemble Retrieval for Multilingual Fact-Checking
Authors: Iva Pezo; Allan Hanbury; Moritz Staudinger
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Fact-check retrieval plays a crucial role in combating misinformation by ensuring that claims are accurately matched with relevant fact-checks. In this work, we present a hybrid retrieval pipeline that integrates lexical and semantic retrieval models, leveraging their complementary strengths. We evaluate different retrieval and reranking strategies, demonstrating that hybrid ensembling consistently outperforms individual models, while reranking provides only marginal improvements.
Tarbiat_Modares_SemEval2025_Task11_MultiLabel_Emotion_TransferLearning
Authors: Sara Bourbour; Maryam Gheysari; Amin Saeidi Kelishami; Tahereh Talaei; Fatemeh Rahimzadeh; Erfan Moeini
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: The SemEval-2025 Task 11 addresses multi-label emotion detection, classifying perceived emotions in text. Our system targets Amharic, a morphologically complex, low-resource language. We fine-tune LaBSE with class-weighted loss for multi-label prediction.Our architecture consists of: (i) text tokenization via LaBSE, (ii) a fully connected layer with sigmoid activation for classification, and (iii) optimization using BCEWithLogitsLoss and AdamW. Ablation studies on class balancing and data augmentation showed that simple upsampling did not improve performance, highlighting the need for more sophisticated techniques.Our system ranked 14th out of 43 teams, achieving 0.4938 accuracy, 0.6931 micro-F1, and 0.6450 macro-F1, surpassing the task baseline (0.6383 macro-F1). Error analysis revealed that anger and disgust were well detected, while fear and surprise were frequently misclassified due to overlapping linguistic cues. Our findings underscore the challenges of multi-label emotion detection in low-resource languages. Future work could explore context-aware embeddings, improved data augmentation, and adaptive loss functions.
OPI-DRO-HEL at SemEval-2025 Task 9: Integrating Transformer-Based Classification with LLM-Assisted Few-Shot Learning for Food Hazard Detection
Authors: Martyna Śpiewak; Daniel Karaś
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: In this paper, we propose a hybrid approach for food hazard detection that combines a fine-tuned RoBERTa classifier with few-shot learning using an LLM model (GPT-3.5-turbo). We address challenges related to unstructured text and class imbalance by applying class weighting and keyword extraction (KeyBERT, YAKE, and Sentence-BERT). When RoBERTa’s confidence falls below a given threshold, a structured prompt which comprising the title, extracted keywords, and a few representative examples is used to re-evaluate the prediction with ChatGPT.
Zero at SemEval-2025 Task 11: Multilingual Emotion Classification with BERT Variants: A Comparative Study
Authors: Revanth Gundam; Abhinav Marri; Radhika Mamidi
Venue: ACL 2025 (Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025))

Abstract: Emotion detection in text plays a very crucial role in NLP applications such as sentiment analysis and feedback analysis. This study tackles two tasks: multi-label emotion detection, where the goal is to classify text based on six emotions (joy, sadness, fear, anger, surprise, and disgust) in a multilingual setting, and emotion intensity prediction, which assigns an ordinal intensity score to each of the above-given emotions. Using the BRIGHTER dataset, a multilingual corpus spanning 28 languages, the paper addresses issues like class imbalances by treating each emotion as an independent binary classification problem. The paper first explores strategies such as static embeddings such as GloVe with logistic regression classifiers on top of it. To capture contextual nuances more effectively, we fine-tune transformer based models, such as BERT and RoBERTa. Our approach demonstrates the benefits of fine-tuning for improved emotion prediction, while also highlighting the challenges of multilingual and multi-label classification.