NLP _ CIMAT at S em E val-2025 Task 3: Just Ask GPT or look Inside. A prompt and Neural Networks Approach to Hallucination Detection (2025)

Abstract: This paper presents NLP_CIMAT’s participation in SemEval-2025 Task 3, which focuses on hallucination detection in large language models (LLMs) at character level across multiple languages. Hallucinations—outputs that are coherent and well-formed but contain inaccurate or fabricated information—pose significant challenges in real-world NLP applications. We explore two primary approaches: (1) a prompt-based method that leverages LLMs’ own reasoning capabilities and knowledge, with and without external knowledge through a Retrieval-Augmented Generation (RAG)-like framework, and (2) a neural network approach that utilizes the hidden states of a LLM to predict hallucinated tokens. We analyze various factors in the neural approach, such as multilingual training, informing about the language, and hidden state selection. Our findings highlight that incorporating external information, like wikipedia articles, improves hallucination detection, particularly for smaller LLMs. Moreover, our best prompt-based technique secured second place in the Spanish category, demonstrating the effectiveness of in-context learning for this task.

## introduction since the introduction of the transformer architecture in 2017 @xcite , large language models have rapidly advanced, finding applications in both scientific research and everyday life. however, these models face two major challenges @xcite : they often generate false or misleading information that appears syntactically correct and current evaluation metrics prioritize fluency and grammatical accuracy over factual correctness. this combination leads to what is known as hallucination, that is, where models produce outputs that are coherent and well-formed but contain inaccurate or fabricated information-an issue that remains difficult to detect automatically. hallucinations pose a significant barrier to the practical development of llms and their mass adoption as reliable tools in everyday life. although in this work we treat hallucination detection as a task, hallucinations can appear in various domains and some works address hallucination detection in fields like machine translation @xcite , summarization @xcite , definition modeling @xcite and dialogue generation @xcite , we are still far from establishing a unified dataset and system for hallucination detection. despite some progress in hallucination detection, current works and datasets do not attack the problem with a granularity that allows one to know exactly where the hallucination is in the text. semeval-2025 task 3 introduces a test bed that allows us to tackle the problem with character level granularity and information to compare the correlation between the proposed systems and human annotations. the task is closely related to fact checking and consist of identifying the parts of the model output that are hallucinated at character level given the model input. the dataset includes 14 languages where every instance indicates the language, the ranges of hallucinated characters and the probability assigned to these hallucinations. this paper presents the participation of nlp_cimat in the shared task which consist on the development and study of two main approaches. the first one is a prompt-based method that leverages the intrinsic knowledge and capabilities of the llm, where the model is directly asked to highlight the hallucinated parts of its output. we explore two key variants of this method: without external knowledge -the model relies solely on its internal knowledge; and with external knowledge (rag-like framework) -we investigate whether incorporating retrieved external information im-proves hallucination detection performance. the second approach is a neural network that leverages the encoded information in the internal hidden state representations of a llm to predict whether a token is hallucinated. this method explores several key aspects to optimize performance: selfcheckgpt @xcite : utilizes a sample based strategy to generate multiple stochastic samples. this work proposes that a model with a good understanding and knowledge of the task or concept is less likely to generate inconsistent information and hallucinations. this work demonstrates the effectiveness of prompt based approaches to detect hallucinations, we took inspiration in their prompt based approach modifying their structure to not rely on samples. fact-checking performance of llms improves notably when they are given contextual information, as shown by @xcite ; @xcite . llms can effectively leverage external knowledge to generate responses and support claims with factual accuracy. we took inspiration from their work and provided our prompt approach with external information retrieved from wikipedia. in recent years there have been numerous studies about using hidden states of a llm to detect hallucinations. @xcite train a mlp on the hidden states of a llm to detect hallucinations at sentence level and investigate which hidden states contain relevant information to correctly classify hallucinations. @xcite analyze the changes in the internal states of a llm when it generates factual versus nonfactual claims, using these differences to determine whether a hallucination has occurred. our work draws inspiration from both studies: we adopt the mlp architecture from the first and leverage insights from the second to develop a token-level hallucination classifier. ## methodology in this section we will introduce our proposed systems for hallucination detection, dividing them in two groups: prompt based approach and hidden states neural network approach. ## results we first present the best results submitted for each language, followed by a study analyzing the impact of the different strategies we implemented.