MALTO at S em E val-2025 Task 3: Detecting Hallucinations in LLM s via Uncertainty Quantification and Larger Model Validation (2025)

Abstract: Large language models (LLMs) often produce {textit{hallucinations}} —factually incorrect statements that appear highly persuasive. These errors pose risks in fields like healthcare, law, and journalism. This paper presents our approach to the Mu-SHROOM shared task at SemEval 2025, which challenges researchers to detect hallucination spans in LLM outputs. We introduce a new method that combines probability-based analysis with Natural Language Inference to evaluate hallucinations at the word level. Our technique aims to better align with human judgments while working independently of the underlying model. Our experimental results demonstrate the effectiveness of this method compared to existing baselines.

## introduction large language models (llms) are widely used for various nlp tasks, such as information retrieval @xcite , medical queries @xcite , and content generation @xcite . their ability to generate coherent and contextually relevant text has led to an increasing reliance on them as primary information sources, sometimes surpassing traditional methods like search engines, expert consultations, or structured databases @xcite . this shift reflects the growing trust in llms for fast and accessible information. however, a major challenge is their tendency to produce hallucinations -factually incorrect but highly persuasive outputs @xcite . these errors can take various forms, including false claims @xcite , fabricated references @xcite , and made-up biographies @xcite , often presented in a way that makes them difficult to distinguish from accurate information. since llms generate text based on patterns in their training data rather than direct verification of facts, they may confidently assert misinformation, leading to potential risks in sensitive domains such as healthcare @xcite , law @xcite , and journalism @xcite . the problem is further amplified by the fact that hallucinations are often blended with accurate, truthful statements, making them harder to detect @xcite . a model may produce a largely correct passage with only a few inaccurate details, increasing the likelihood that users will accept the entire output as trustworthy. moreover, the possibility that models have to generate and deal with multiple languages @xcite can make evaluating these outputs even more complex. as llms become more advanced and widely deployed, addressing their tendency to hallucinate is critical to ensuring their reliability and safe integration into real-world applications. to bring more attention to this issue, the multilingual shared-task on hallucinations and related observable overgeneration mistakes (mu-shroom) has been introduced at semeval 2025 @xcite . mu-shroom invites researchers to detect hallucination spans in llm outputs across multiple languages and models. the task specifically focuses on identifying which parts of a generated text contain hallucinations. participants are provided with llm outputs in different formats, including raw text, token lists, and logit values, and are tasked with predicting hallucination probabilities at the character level. this work introduces a novel approach that operates at the word level to evaluate hallucinations 1 . by leveraging probability-based analysis and a natural language inference (nli) model, we compare each generated token to the most likely alternatives from a larger model, identifying potential hallucinations based on inconsistencies in predicted outputs. our method aims to improve alignment with human annotations while remaining model-independent. ## related works fact-checking is a common approach to mitigate hallucinations in llm outputs @xcite . however, it typically depends on external knowledge sources such as databases, search engines, or pre-verified information repositories. these sources, while useful, are often incomplete, domain-specific, and require continuous updates to remain relevant @xcite . additionally, integrating them into real-time llm applications introduces significant computational overhead, making the process inefficient and sometimes impractical at scale. to overcome these limitations, this work proposes an alternative approach based on uncertainty quantification (uq), which detects hallucinations directly from the model's own outputs without relying on external verification @xcite . our methodology provides a way to assess how confident an llm is in its generated text, offering a built-in mechanism for identifying potentially unreliable information. detecting hallucinations at the claim level is a challenging task @xcite , as a single output may contain both accurate and inaccurate information, requiring finer-grained uncertainty measurement to highlight specific false or misleading claims. our work addresses this challenge by expanding upon previous research in this direction @xcite , which introduced a new token-level uncertainty score by aggregating token uncertainties into claim-level scores. we adapt their method for word-level detection and introduce a larger model to verify the smaller model's output, leveraging its broader knowledge. additionally, we enhance the hallucination score by factoring in the uncertainty of both the nli model and the llm. word probability from model m. for each word in m's output, we provide its preceding context to m . we then extract the most likely tokens from m 's probability distribution until (i) their combined probability reaches a threshold k, or (ii) a single token has a probability lower than ρ foot_3 . therefore, the number of selected tokens n varies depending on the word. given the selected tokens, we determine the probability of the full word w by summing the probabilities of its tokens: p(w) = t @xmath0, where t j are the tokens forming w and p j are their probabilities. checking semantic consistency with nli. we assess whether each word aligns with the original sentence's meaning using an nli model. such model determines whether replacing one word with another changes the meaning of the sentence. if the sentence is too long, we truncate it around the word to fit the model's context window. the nli model assigns probabilities for three possible relationships: entailment (p + (w)), i.e., the word fits naturally in the sentence; neutral (@xmath1, i.e., the word has a different meaning but does not contradict the original sentence; contradiction (p -(w)), i.e., the word changes the meaning of the sentence. computing the hallucination score. to measure hallucination at the word level, we start from the original approach proposed by @xcite . they compute the hallucination score (hs) at the claim level as follows: we incorporate neutral entailment alongside positive entailment, unlike @xcite , as it empirically improved results on the validation set. this formulation in eq. 2 effectively captures the inverse relationship between word confidence and hallucination likelihood while accounting for semantic coherence through the nli component. the baseline methodology operates independently of human-annotated ratings, ensuring applicability in scenarios where labeled data is scarce or unavailable. however, to enhance alignment with human perception of hallucinations, we introduce a calibration mechanism through a multiplicative factor η: the parameter η serves as an alignment coefficient that is empirically determined using the validation dataset to maximize the correlation between our computed scores and the soft labels provided by human reviewers. this calibration process allows to fine-tune the sensitivity of the hallucination detection system to better match human judgment thresholds. in our experimental framework, we systematically evaluate two distinct configurations: (1) a label-agnostic variant (lav) where η = 1, which preserves the model's inherent hallucination detection capabilities without reliance on human feedback, and (2) a reviewer-aligned variant (rav) where η is optimally selected based on validation data to maximize correlation with human annotations. the former configuration is particularly valuable in zero-shot deployment scenarios or when consistent detection criteria are required across diverse domains, while the latter configuration offers enhanced performance in applications where human perception of hallucinations is the primary evaluation metric. ## results as expected, the mark-none method performs the worst. its scores are close to zero for both metrics, showing that it fails to capture hallucinated content. on the other hand, the mark-all method achieves a relatively high iou. this is likely because large portions of the text in this task are hallucinated. however, its correlation score is very low (ρ = 0.000), meaning it does not align well with human judgments. the roberta-based model performs slightly better. it improves correlation (ρ = 0.119), meaning it captures some alignment with annotator probabilities. however, it has a low iou, indicating that it struggles with precise localization. our proposed method significantly outperforms these baselines. we test it in two configurations described in section 4. both variants of our method achieve the highest overall performance as they provide the best balance between the two considered metrics. the label-agnostic model (lav) reaches a correlation of ρ = 0.300 and an iou of 0.310. this surpasses in iou the scoring method proposed in @xcite and adapted to our scenario while maintaining a similar correlation. the reviewer-aligned version (rav) further improves correlation (ρ = 0.324) while keeping a strong iou (0.311). this result shows that our approach effectively identifies hallucinated content. when properly calibrated, it also aligns well with human judgments, thus providing a strong balance between accurate hallucination detection and agreement with human perception. ## conclusion this work addresses the mu-shroom shared task at semeval 2025, focusing on detecting word-level hallucinations in llm outputs. we introduce a novel approach that uses a larger model validation without the need for external knowledge sources. our method achieves strong results compared to the proposed baselines, proving to be an excellent starting point for evaluating hallucinations when a ground truth or external sources are unavailable. one key limitation is handling multiple connected hallucinations. the model generates each word based on past text. if a hallucination appears, it becomes part of the context. the model may then continue building on this false information, making the hallucination harder to detect. this can lead to a chain of believable but incorrect statements. to address this, future work could use a broader context or develop mechanisms to review and correct past text when a hallucination is found.