COGUMELO at S em E val-2025 Task 3: A Synthetic Approach to Detecting Hallucinations in Language Models based on Named Entity Recognition (2025)

Abstract: In this paper, we propose an approach to detecting hallucinations based on a Named Entity Recognition (NER) task.We focus on efficiency, aiming to develop a model that can detect hallucinations without relying on external data sources or expensive computations that involve state-of-the-art large language models with upwards of tens of billions of parameters. We utilize the SQuAD question answering dataset to generate a synthetic version that contains both correct and hallucinated responses and train encoder language models of a moderate size (RoBERTa and FLAN-T5) to predict spans of text that are highly likely to contain a hallucination. We test our models on a separate dataset of expert-annotated question-answer pairs and find that our approach achieves a Jaccard similarity of up to 0.358 and 0.227 Spearman correlation, which suggests that our models can serve as moderately accurate hallucination detectors, ideally as part of a detection pipeline involving human supervision. We also observe that larger models seem to develop an emergent ability to leverage their background knowledge to make more informed decisions, while smaller models seem to take shortcuts that can lead to a higher number of false positives.We make our data and code publicly accessible, along with an online visualizer. We also release our trained models under an open license.

## introduction hallucinations in language models (lms) are a well-known issue that has been studied in the context of text generation tasks @xcite @xcite , with some authors affirming they are inevitable @xcite . however, despite the open discussion on their avoidability, a community of authors have worked on methods to detect, prevent, or mitigate them @xcite @xcite @xcite . our work contributes to this effort by addressing hallucination detection in instructiontuned lms, a shared task proposed in the semeval 2025 task 3, mu-shroom @xcite . we approach the chanllenge by framing hallucination detection as a named entity recognition (ner) task, leveraging ner's ability to identify specific spans of text. ner extracts structured information, such as names, dates, or locations, from unstructured text @xcite . traditionally, it has been applied to sequence labeling tasks using rulebased systems or machine learning models trained on annotated datasets @xcite . however, its versatility has led to applications beyond information extraction, including social media analysis, knowledge graph construction, reinforcement learning for entity augmentation, and more @xcite @xcite . in our approach, we adapt ner to detect hallucinated spans by treating them as a specialized type of named entity, allowing us to efficiently identify incorrect or fabricated text segments without relying on external data sources or computationally expensive large-scale lms. the rest of this article is organized as follows: in section 2, we provide background information on the task setup; in section 3, we describe our system's approach; in section 4, we detail our experimental setup and present qualitative evaluations; in section 5, we report on the quantitative results of our models and discuss their performance; and in section 6, we conclude our work and suggest future directions. we also address ethical considerations in section 7 and discuss the limitations of our approach in section 8. we make our code, dataset and models publicly available at https: //github.com/acmcmc/hallucinations-ner . ## background the shared task that our work is based on, mu-shroom, involves detecting spans of text that correspond to hallucinations in the outputs of lms, with the goal of predicting where hallucinations occur in a given text. for example, given the following example from the validation dataset of the task: question. what is the population of the spanish region of galicia? tagged answer. as of 2021, the estimated population in the region is around 1.5 million people. a a the opacity of the underlines represents the probability of the character being a hallucination. the task consists of predicting the spans of text that are more associated with hallucinations, which in this case would be "2021" and "1.5 million". the authors of the task provide a dataset of expert-annotated question-answer pairs in 14 languages, but we choose to focus on english due to the complexity of generating a synthetic dataset of faithful and hallucinated responses, which we use to train our models (see section 3). our approach is thus based on training a model to predict spans of text that are likely to contain hallucinations, which we model as a named entity recognition (ner) task, under the assumption that hallucinations can be seen as a form of named entity that can be detected by a model trained to recognize them. this assumption may not cover all types of hallucinations, as we discuss in section 8, but serves as a starting point that can be later expanded upon. we implement our ner strategy using an iob @xcite tagging scheme, which is a common approach in ner tasks that assigns each token in a sequence a label indicating whether it is inside (i), outside (o), or at the beginning (b) of a named entity. in our case, however, instead of named entities, we turn the task into predicting if we are outside or inside a hallucination. also, while iob assigns an i label to single-token entities, we slightly alter this approach by assigning a b label to the first token of all entities, including single-token entities, and an i label to all subsequent tokens, often referred to as iob2 @xcite . it is important to note that ner is not limited to iob tagging; it can be performed using other approaches that may leverage graphs @xcite , neural networks (sohrab and miwa, 2018; wang and lu, 2019), constituency discriminators (finkel and @xcite , or translation to an augmented natural language form that can be easily extracted @xcite , among others. likewise, the iob tagging scheme is not exclusive to ner tasks, as it can be applied to any sequence labeling task. ## system overview our goal is to train an encoder model to predict spans of text that are likely to contain hallucinations, so we choose to model the task as a ner problem. we do not employ any of the common ner-specific labels, such as dates or verbs, but rather focus on the general applicability of the iob tagging scheme to our task. we first need to have a collection of correct and hallucinated responses to train our model, for which we use the squad dataset @xcite . squad is primarily intended for question answering based on a given context, but we repurpose it by discarding the context and using exclusively the question and suggested answers to build a synthetic dataset of correct and hallucinated responses. the first model is tasked with the transformation of the squad answers, which are an extracted span of text from the context, into a full sentence that a human could understand. then, we use the second model to generate variations of that correct response in a way that it becomes a hallucination. we assemble a synthetic dataset of questionanswer pairs, where the answers are either correct or hallucinated, and we use this dataset to train our models. figure 1 shows our approach. ## experimental setup for our training process, we abstain from using the training set provided by the shared task authors, exclusively training on our synthetic dataset, which we split on a 80/10/10 ratio for training, validation, and testing, respectively. we use the validation set to decide when to stop training. we only use the test set to evaluate our models internally; the results shown in section 5 are based on the test set provided by the shared task authors. we choose a smaller model, smollm2-360m-instruct @xcite how often is notre dame's the juggler published? figure 1: our approach. we employ two instructiontuned language models to generate synthetic questionanswer pairs from the squad dataset, including correct responses and hallucinated variants. these are used to train an encoder model, utilizing a named entity recognition framework, to identify and tag spans of text likely to contain hallucinations. to generate the correct answers since we anticipate that this is a simpler task -however, when it comes to generating hallucinated responses, we want a larger model to generate more diverse and creative hallucinations, so we select llama-3.2-1b-instruct @xcite . since we wish for the hallucinated responses to be significantly different both from the correct response and among themselves, so we select a generation configuration to encourage this. specifically, we set the number of beams and beam groups to 3, the diversity penalty to 0.5, the repetition penalty to 1.2, and the temperature to 1.3; all to force diversity in the generated hallucinations. we take the generated result of the three beams from the hallucination model and add those as hallucinated responses to our dataset. to ensure we have a balance of correct and hallucinated responses, we include the same number of correct and hallucinated responses in our synthetic generation process by upsampling the correct responses to match the number of hallucinated responses. for our choice of encoders, we selected models of the bert and t5 families, roberta-base and roberta-large @xcite and flan-t5-xl (chung et al., 2024), respectively. we opted for the flan variant of t5 as we hypothesize that its more extensive training on a diverse set of tasks may help it generalize better to our task. it is important to highlight that we only run the hallucination generation step once, which we acknowledge may lead to a decrease in the representativeness of our synthetic dataset. the test dataset, in fact, can contain more than one hallucination per question-answer pair. this could be addressed by running the hallucination generation step more than once, which we leave for future work. we only generate hard labels for evaluation (0.0 or 1.0) and do not use the probabilities assigned by the models to each token, which could be a potential improvement to our approach. ## results the results show that larger model sizes seem to correlate with better performance. model architecture also seems to play a role, with the flan-t5-xl model outperforming the roberta models in both metrics. in general, while above a baseline model, our results tend to be lower than some other participants in the shared task, which we attribute to the fact that the models we utilize may not have enough capacity to learn the task effectively; the synthetic data we generate, which is not fully aligned with the hallucinations found in the evaluation dataset; and the fact that other techniques such as rag @xcite can retrieve ground truths that greatly improve the performance of the models, since the types of questions asked in the evaluation dataset are at times very specific and we do not expect that such niche knowledge is present in the background knowledge of our models. it should also be noted that the spearman correlation is generally lower than the iou, which is very dependent on the threshold used to determine if a character is part of a hallucination or not. as seen in figure 2b , the models may sometimes tag just subparts of a hallucination, which we expect will particularly lower the spearman correlation. additionally, since we make our models generate hard labels exclusively, we expect that the spearman correlation would also be lower than if we had used soft labels. ## conclusion in this article, we present an approach to detecting hallucinations in the output of language models based on a named entity recognition task. we train moderate-size encoding models on a synthetic dataset generated from the squad questionanswering dataset, which we use to predict text segments that are likely to contain hallucinations. our models achieve a jaccard similarity of up to 0.358 and a spearman correlation of up to 0.227, suggesting that our models can serve as moderately accurate hallucination detectors, although our scores are lower than some other participants in the shared task. we also observe an interesting pattern in the behavior of our models, where larger models seem to develop an emergent ability to use their background knowledge to make more informed decisions, while smaller models seem to take shortcuts that can lead to a higher number of false positives. we publicly release a synthetic dataset, opensource code and models, along with an interactive visualizer to facilitate further research. future work could explore enhancing this ner-based approach by incorporating diverse hallucination types and multilingual data to improve detection accuracy.