R aggedy F ive at S em E val-2025 Task 3: Hallucination Span Detection Using Unverifiable Answer Detection (2025)

Abstract: Despite their broad utility, large language models (LLMs) are prone to hallucinations. The deviation from provided source inputs or disparateness with factual accuracy makes users question the reliability of LLMs. Therefore, detection systems for LLMs on hallucination are imperative. The system described in this paper detects hallucinated text spans by combining Retrieval-Augmented Generation (RAG) with Natural Language Interface (NLI). While zero-context handling of the RAG had little measurable effect, incorporating the RAG into a natural-language premise for the NLI yielded a noticeable improvement. Discrepancies can be attributed to labeling methodology and the implementation of the RAG.

## introduction large language models (llms) are specialized in generating human-like text in various styles, which lends them to many practical applications. however, even the most sophisticated models can produce hallucinations, making users question their reliability and putting the adoption of machine learning pipelines in jeopardy @xcite . hallucination refers to the generation of texts or responses that exhibit grammatical correctness, fluency, and authenticity, but deviate from the provided source inputs or do not align with factual accuracy @xcite . this is a phenomenon that established evaluation metrics struggle to detect @xcite ; as a result, it has now become imperative to develop systems that can assess the factual consistency of a claim with respect to context @xcite . the semeval shared task mu-shroom @xcite provides an opportunity to develop solutions to the problem of hallucinations in llms. the objective of the task is to classify spans, which are continuous segments of text within llm outputs, as hallucinations. for instance, in the generated sentence "marie curie won three nobel prizes for her work in physics and chemistry," the span "won three nobel prizes" would be labeled as a hallucination, since she actually won two. to this end, we only consider spans hallucinated when the llm output contradicts the relevant retrievable information. the detection of hallucination spans allows for a more fine-grained understanding of where hallucinations occur in llms, as well as giving an indication of the severity of hallucinations in llms. this is something that a binary classification is not able to provide, given its simplicity. in this paper, we present a linear composite system that employs a retrieval-augmented generation (rag) question-and-answer system to generate an answer to the question in each prompt; combines the question-answer pair into a unified premise with a generative llm; and compares this premise with the subject model output using an off-the-shelf natural language inference (nli) model. we compare the performance of this system with several baselines, and discuss its strengths and weaknesses. ## background the shared-task on hallucinations and related observable overgeneration mistakes (shroom) has been put forth by @xcite to address the issue of hallucinations in llms. the main objective of shroom is the development of systems that detect hallucinations in the generated output of llms. in the shared task, participants must detect grammatically sound outputs that nonetheless contain incorrect or unsupported semantic information compared to a source input. this task can be done with or without access to the model that produced the output: respectively, these are the model-aware and model-agnostic versions of the task @xcite . in this previous shared task, nli-based approaches achieved the best performance. @xcite . the objective of nli systems is to determine the truth value of a hypothesis, given a premise. as an example, the premise "the pedestrian walks on the zebra crossing" and the hypothesis "the pedestrian must yield" produces a contradiction and is judged false; the same premise with the hypothesis "the pedestrian is wearing a green shirt" results in a neutral judgment, though this can also be rendered as false. the remarkable similarity between nli and the shroom task lent itself to several submissions utilizing models that were (pre-)trained on nli data. the deeppavlov team of @xcite opted to directly train roberta and similar models as well as a text-to-text transfer transformer on nli data. on the other hand, the harmonee team of @xcite selected a pre-trained roberta model and fine-tuned it using data from shroom. the models produced by these teams achieved accuracies of 0.80 and higher, with har-monee's approach being the best model in the model-aware version of the task. systems that operate on a similar objective of nli, such as those for information retrieval, paraphrasing, fact verification and textual similarity, can be unified under a single model for information alignment @xcite . rag frameworks have shown promising results in regard to hallucinations. these systems combine generative models with retrieval mechanisms. this hybrid method not only improves the factual accuracy of the generated text, but also helps mitigate the risk of hallucinations by grounding the output in verifiable data @xcite . the system used in this paper will use the insights from these works to combine nli with rag to create a pipeline for hallucination detection on text spans. in this way, we hope to build upon the best-performing systems from the shroom task and test their resilience against a different method of hallucination detection. ## data in the semeval 2025 task-3 mu-shroom, the task is to detect hallucination spans in the outputs of instruction-tuned llms in a multilingual context. to this end, the data is provided in 24 different languages, with each output being produced by a variety of open-source llms. the llm output is provided in the format of a human question, a generated answer and, in the case of the validation set, labels for hallucinated spans that use string positions. the last of these is divided into soft labels, which indicate the probability of hallucinations, and hard labels, which assert spans as hallucinated if the probability exceeded 50%. ## method our approach can be distilled into three distinct steps. the pipeline for this system is shown in figure 1 . first, the rag model retrieves the relevant information from a wikipedia vector index based on the prompt question, and generates an appropriate answer to form a premise. afterwards, our model tokenizes the hallucinated answer using the treebank tokenizer from the nltk library @xcite . finally, the hallucinated answer is fed in token trigrams to the nli model as hypotheses, with the trigrams being fed in a sliding-window fashion. besides our main system, we also created a baseline classifier that uses random probabilities and allzero probabilities. the random-probability baseline classifier is the default and assigns completely random probabilities to each span, making each output unique and not reproducible. the alternative approach assigns a 0.0 value for all probabilities. these baselines are meant to gauge the effectiveness of our systems in the absence of external baseline metrics during development. ## results our main system ranked 32nd on the englishlanguage leaderboard, measured by the iou score on the test set. we applied our baseline models to the validation set only; the baseline scores reported for the test set are provided by @xcite . the full results are shown in our main model on the validation set achieved an iou score of 0.198 and a correlation score of 0.171, showing a slight improvement over the baseline models. in the test set, our model demonstrated a larger increase in performance, with an iou score of 0.275 and a correlation of 0.261. this suggests that our concatenation-based system yields an improvement in identifying hallucination spans beyond what is captured by baseline approaches; however, if this were the case, the effect size is negligible. the improved system with llama premise rewriting demonstrated the most visible gains in performance. on the validation set, the improved system achieved an iou of 0.240 and a correlation of 0.201. the results for the improved system on the test set yielded our highest scores overall, with an iou of 0.315 and a correlation of 0.304. these results indicate that the addition of a premise-rewriting step refines the hallucination detection process and leads to a more robust identification of hallucination spans. ## discussion the results indicate that our proposed system provides a noticeable improvement over our baseline models. using retrieval-augmented generation, the model ensures that the responses generated are grounded in relevant contextual information. furthermore, nli-based evaluation at the trigram level enables a more granular detection of hallucination spans, which is not possible with binary classification approaches. a key strength of our approach lies in the introduction of a premise rewriting step, which improves alignment between generated text and factual sources before the nli step. the empirical results show that this method enhances the detection performance of the hallucination range. however, this did not improve the detection of subtle hallucinations or the handling of paraphrased incorrect information. our core approach has a few systematic flaws worth addressing. for instance, we used an off-theshelf nli model without additional task-specific fine-tuning. while this allowed for rapid experimentation, it may have resulted in worse overall performance. fine-tuning or adapting the model on the hallucination detection dataset itself could have improved performance by aligning the model more closely with task-specific patterns. a similar problem exists for our rag component, whereby an unscreened set of documents may have led to lower-quality training data for the rag model. we expect a custom, curated set of documents to improve the overall efficacy of the model. in addition, our system is non-standard in ways that could affect performance. many systems either use external factual documentation to explicitly verify claims or assess internal output consistency across different runs. in contrast, our method does not rely on external factual verification beyond the initial rag retrieval, nor does it compare outputs across runs. furthermore, the detection is entirely localized, whereby the contradiction entailment is combined with the span searching. this may also have contributed to a lower leaderboard rank. we manually checked differences in span hallucination assessment for the validation set between the gold reference and our best model, in order to better understand the performance of our model. in particular, the span annotations in the gold standard are different from the spans that we created using the treebank tokenizer. for instance, the first identified span in the sentence "the elysiphale order contains 5 genera." is the word 'the' for our own system, and 'elysiphale' for the gold reference. the last identified span is '.' and 'genera.' respectively. in this example, the identified span 'genera' is assigned a probability of 1.0 by our own model, and the span 'genera.', including the period, is assigned 0.2 in the gold standard. if our nli model classifies a trigram as a hallucination, that classification is extended to all of its constituents. this makes it prone to false positives, especially at string boundaries. we highlight an example of this behavior in ## conclusion this paper presents a novel approach to hallucination span detection in machine-generated text through rag and nli. our research is conducted within the framework of the mu-shroom shared task, contributing to the broader effort of evaluating and improving hallucination detection techniques. our results demonstrate that our proposed method outperforms baseline approaches and provides a more fine-grained understanding of hallucinations in llm outputs. the introduction of a premiserewriting step within the pipeline further enhances detection accuracy. we recognize that our system has a variety of shortcomings that contributed to a lower score than most. in particular, future research could explore more selective labeling and a rag-like system with an array of outputs. nevertheless, we believe that our study contributes to the mu-shroom shared task by providing information on hallucination span detection. in this way, we hope to advance research on the factual reliability of content generated by llms by mitigating the presentation of faulty information.