H ausa NLP at S em E val-2025 Task 3: Towards a Fine-Grained Model-Aware Hallucination Detection (2025)

Abstract: This paper presents our findings of the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes, MU-SHROOM, which focuses on identifying hallucinations and related overgeneration errors in large language models (LLMs). The shared task involves detecting specific text spans that constitute hallucinations in the outputs generated by LLMs in 14 languages. To address this task, we aim to provide a nuanced, model-aware understanding of hallucination occurrences and severity in English. We used natural language inference and fine-tuned a ModernBERT model using a synthetic dataset of 400 samples, achieving an Intersection over Union (IoU) score of 0.032 and a correlation score of 0.422. These results indicate a moderately positive correlation between the model’s confidence scores and the actual presence of hallucinations. The IoU score indicates that our modelhas a relatively low overlap between the predicted hallucination span and the truth annotation. The performance is unsurprising, given the intricate nature of hallucination detection. Hallucinations often manifest subtly, relying on context, making pinpointing their exact boundaries formidable.

## introduction despite the advancements in natural language processing (nlp) and the development of natural language generation (nlg) models, their limitations and potential risks have gained increased attention. a significant issue is that nlg models often produce unfaithful text relative to the source input, a phenomenon known as "hallucination" @xcite @xcite . hallucinations in nlg models often result in outputs that, while fluent, lack accuracy. this issue arises because existing evaluation metrics prioritize fluency over correctness, ultimately diminishing system performance and failing to meet user expectations in practical applications @xcite . for example, @xcite illustrate this phenomenon by attempting to paraphrase the statement "i am not sure where my phone is," which leads to the hallucinated output: "how can i find the location of any android mobile." hallucination in nlg is concerning due to its impact on performance and safety in applications like medicine, where hallucinatory summaries or machine-translated instructions can pose risks to patient diagnosis @xcite . similarly, hallucinations can lead to privacy violations by generating sensitive information not present in the source input @xcite . to address this challenge, "semeval-2025 task 3 -mu-shroom, a shared-task on hallucinations and related observable overgeneration mistakes" @xcite was introduced. the shared task involves detecting specific text spans that constitute hallucinations in outputs generated by llms. participants compute the probability of each character being marked as a hallucination using the llm output consisting of a character string, tokens, and logits, thus enabling a fine-grained hallucination detection. ## background numerous efforts are provided to address hallucination across various nlg tasks. analyzing hallucinatory content and its relationships in different tasks could enhance our understanding and unify efforts across nlg fields @xcite . while most existing studies focus on specific tasks like abstractive summarization @xcite and machine translation @xcite , the study of @xcite offer a comprehensive analysis on the phenomenon of hallucination in abstractive summarization, dialogue generation, generative question answering, data-to-text generation, and machine translation. @xcite argue that hallucination problem occurs when there is very little divergence in dataset and encoder with a defective comprehension ability could influence the degree of hallucination. similarly, @xcite show that training and modeling choices of neural models have influence for hallucination. while large pre-trained models used for downstream nlg tasks are powerful in providing generalizability and coverage, they however prioritize parametric knowledge over the provided input and can result in hallucination of excess information in the output @xcite . recent efforts to address hallucination include the shared task on hallucinations and related observable overgeneration mistakes (shroom), which focused on binary classification for taskspecific english language models @xcite . @xcite ; @xcite ; @xcite identify cases of fluent overgeneration hallucinations in model-aware and model-agnostic settings. they detect grammatically sound outputs which contain incorrect or unsupported semantic information. building on this task instead of focusing on model-agnostic and model-aware tracks, this year's task focuses on the multilingual aspect. therefore, all data-points in this year's task are model-aware. we present our submission for the task multilingual shared-task on hallucinations and related observable overgeneration mistakes (mu-shroom) which aims to detect hallucinations in a multilingual context, providing a more nuanced understanding of their occurrence. mu-shroom is a semeval-2025 shared task that focuses on detecting hallucinations and overgeneration errors in ai-generated text, a crucial challenge in improving the reliability of llms @xcite . hallucinations occur when llms produces fluent but false or unsupported content, while overgeneration mistakes involve excessive, often misleading text @xcite . this task is important as language models become increasingly prevalent in various applications, where factual accuracy and reliability are essential for maintaining user trust and system integrity. therefore, tackling these issues is essential for ensuring ai-generated text remains trustworthy and useful across various applications @xcite . the task consists of participants detecting spans of text corresponding to hallucinations and determine which parts of the given text produced by llms constitute hallucinations. annotated dataset are provided, allowing researchers to develop and benchmark models for identifying these issues in different linguistic contexts. given the llm output as a string of characters, a list of tokens, and a list of logits, participants calculate the probability that each character is marked as a hallucination and thus provide a fine-grained hallucination detection. similarly, the task is held in multilingual and multi-model context as data are produced by a variety of public-weights llm in multiple languages which includes: arabic (modern standard), chinese (mandarin), english, finnish, french, german, hindi, italian, spanish, and swedish, along with three surprise test languages. ## system overview one of the primary challenges in this task was the lack of labeled training data. to address this, we created a synthetic training dataset using chatgpt, adhering to official annotation guidelines. for the language selection, we used english language only and this is due to time constraint. secondly, english language offers an extensive resource for nlp model development and evaluation. similarly, focusing on a single language allowed us to develop deeper linguistic pattern recognition for hallucination detection. for model selection, we fine-tuned modern-bert @xcite , an advanced variant of bert @xcite that offers significant improvements in handling long-context inputs, computational efficiency, and robustness in token classification tasks. this choice aligns with recent studies highlighting the benefits of long-context language models in detecting nuanced text errors @xcite . for a precise understanding of hallucination detection, we used the model-aware method. this method is based on analysing internal data of llm during inference. one of the possible approaches is the analysis of the outputs of the hidden layers of the transformer. using vector values of hidden layers for hallucination detection was proposed in a method called statement accuracy prediction, based on language model activations (saplma) @xcite . saplma is a probing technique that utilises a feedforward neural network trained on activation values of the hidden layers of llm. ## results at the end of training our model, the evaluation result showed that the model had a precision and recall score of 0.49 and 0.54 respectively. the f1 score of the model was at 0.43. the result indicates that the model correctly identifies hallucinations about half the time. the model is also able to detect slightly more than half of all hallucinations present in the text. on the task-based evaluation of our model, our system achieved an intersection over union (iou) score of 0.032 and a correlation score of 0.422. the iou score indicates that our model has a relatively low overlap between the predicted hallucination span and the truth annotation. with the relatively low score, the model is struggling to identify the exact boundaries of the hallucinated content. going by this result also, the model may be prone to false positives and/or false negatives. the models correlation score of 0.422 shows a moderate positive correlation between the con-fidence scores of the model and the actual presence of hallucinations. this result can translate to the model's ability to differentiate between hallucinated and non-hallucinated content. while this may not be the best performance, the results show that there is room for further improvement. our model results ranked 42nd and 24th on the iou and correlation score indices respectively. the scores are not favourable and may not be entirely unexpected given the inherent complexity of hallucination detection,. hallucinations can be subtle and context-dependent, making exact boundary detection particularly challenging. the model result is a promising starting point for further improvement. ## conclusion this paper presents our approach to the semeval shared task on llm hallucination detection, focusing on a fine-grained, model-aware analysis of hallucination occurrences in the english language. we leveraged natural language inference and fine-tuned a modernbert model using a synthetic dataset of 400 samples. our model achieved rankings of 42nd and 24th on the intersection over union (iou) and correlation score indices, respectively. these results, while modest, underscore the inherent complexity of hallucination detection and highlights the need for continued refinement and innovation in this area. our findings serve as a promising foundation for future improvements, emphasizing the importance of model-aware strategies in enhancing the reliability of llm outputs.