B lue T oad at S em E val-2025 Task 3: Using Question-Answering-Based Language Models to Extract Hallucinations from Machine-Generated Text (2025)

Abstract: Hallucination in machine-generated text poses big risks in various domains, such as finance, medicine, and engineering. Task 3 of SemEval-2025, Mu-SHROOM, challenges participants to detect hallucinated spans in such text. Our approach uses pre-trained language models and fine-tuning strategies to enhance hallucination spam detection, focusing on the English track. Firstly, we applied GPT-4o mini to generate synthetic data by labeling unlabeled data. Then, we employed encoder-only pre-trained language models with a question-answering architecture for hallucination span detection, ultimately choosing XLM-RoBERTa for fine-tuning on multilingual data. This model appeared to be our best and ranked 18th and 22nd on the English track with 0.469 intersection-over-union and 0.441 correlation scores, respectively. It achieved promising results across multiple languages, surpassing baseline methods in 11 out of 13 languages, with Hindi having the highest scores of 0.645 intersection-over-union and 0.684 correlation coefficient. Our findings highlight the potential of a QA approach and using synthetic and multilingual data for hallucination span detection.

## introduction hallucinations can lead to dangerous and misleading information like mathematical inaccuracies in finance, programming errors in autonomous vehicles, and misunderstandings in medical diagnoses @xcite . they pose a challenge in the development of ai models. in task 3 of semeval-2025, called mu-shroom, participants were challenged to create a model that can automatically extract hallucination spans in machinegenerated text @xcite . this paper contains an overview of our approach for task 3 of semeval-2025. the organizers of this shared task define hallucinations as follows: "content that contains or describes facts that are not supported by the provided reference". @xcite in other words, hallucinations are cases where the machine-generated text is more specific than it should be or factually incorrect, given the information available in the provided context. in the task from last year @xcite , the seemingly best approach was to use pre-trained language models (plms) and fine-tuning. most teams also used unlabeled training data, resulting in promising solutions. for this reason, we have incorporated these ideas. firstly, we utilized the unlabeled data to fine-tune an open-source plm to create a model that can effectively detect the spans of hallucinations in a text. we used gpt-4o mini @xcite with prompt engineering to label the unannotated training data. then, for our span detection system, we implemented a pretrained question-answering (qa) architecture, in which we compared roberta @xcite and deberta @xcite . in our final system, a multilingual version of roberta, namely xlm-roberta @xcite , was fine-tuned on the training and validation data. this landed us the 18th and 22nd positions on the english track on the metrics of intersection-over-union and correlation, respectively. the code is available on our github foot_0 and the model on huggingface foot_1 . ## system overview our approach to the task consisted of annotating the supplied unlabeled training data and, in turn, using the data to fine-tune a pre-trained large language model for question answering. in both parts, we performed several experiments to find the optimal setup. ## experimental setup synthetic data generation. we selected gpt-4o mini (openai, 2024) as the automatic hallucination detector for its easy-to-access online interface for prompt experimentation, strong instructionfollowing capabilities, and api availability. using the various combinations of the entries from the english validation set, we constructed a few-shot prompt for the system (see appendix a.2). the included examples correspond with the case types the gpt model found the most challenging to predict, mainly involving annotation span nuances. the designed prompt generated a sample of 500 entries with automatically annotated hallucinated text segments (see an example output entry in appendix a.3). to prevent the system from incorrectly counting the hallucination spans and hallucinating on the numerical probabilities, we instructed the system to provide hallucinations in textual form only, which were then converted to corresponding hard labels using a python script. pre-trained language model experiments. we fine-tuned a roberta @xcite and de-berta @xcite model using the same hyperparameters. they were fine-tuned on the synthetically generated data and evaluated on the validation data in english. our hyperparameter tuning focused on the learning rate, batch size, weight decay, number of epochs, and the logit threshold when selecting hallucination spans. the model was then fine-tuned on the synthetic training and vali-dation english data of 550 instances in total. furthermore, a multilingual model, xlm-roberta @xcite , was fine-tuned on the synthetic training english data and all validation data (see packages. the models were loaded with huggingface @xcite and fine-tuned using the transformers and datasets python libraries. metrics. we evaluated the models using the metrics set by the organizers, which are the intersection-over-union (iou) of the characters marked as hallucinations in the predicted hallucinations and the true hallucinations and correlation (cor) between the predicted probability of a character being hallucinations and the probability assigned by human annotators. as it seemed more relevant and less ambiguous to find the hallucinations than assigning probabilities to them, we focused on achieving the highest intersection-overunion. there were three baselines introduced by the organizers of the task: the mark all, which predicted every character as hallucination, the mark none, which predicted no characters as hallucination, and the neutral, which estimates hallucinations based on probability distributions. the mark all baseline was in every language the highest scoring baseline. ## conclusion in this paper, we presented several contributions to the task of hallucination detection in machinegenerated output. we used gpt-4o mini to generate synthetic hallucination span annotations, adapted qa architecture for hallucination span extraction and finetuned an xlm-roberta model to generalize across 13 languages, outperforming the baseline in 11 languages. this approach resulted in the 18th and 22nd position in the english subtask with an intersection-over-union of 0.469 and a correlation of 0.441 respectively. further research could focus on trying different large language models. also, using humanannotated data seemed to give big improvements, so trying to add more quality data could yield better results. furthermore, we found that our method to synthesize hallucination spans from model predictions could be improved as it can detect the correct spans but assigns a low probability to them, resulting in too many characters being marked as hallucinations. finally, we used gpt-4o mini, which has the drawbacks of being closed-source and paid. attempting the automatic annotation process with an open-source model would be preferable. follow the examples and provide the hallucination spans for the following text pair: source text: <@xmath0: <@xmath1.3 synthetic training data entry {"model_input": "is the arts and humanities citation index still maintained?", "model_output_text": "as of 2021, the a&hci is no longer maintained by the u.s. government and is now maintained privately by jstor.", "hard_labels": @xcite , @xcite , @xcite ]}