Advacheck at S em E val-2025 Task 3: Combining NER and RAG to Spot Hallucinations in LLM Answers (2025)

Abstract: The Mu-SHROOM competition in the SemEval-2025 Task 3 aims to tackle the problem of detecting spans with hallucinations in texts, generated by Large Language Models (LLMs). Our developed system, submitted to this task, is a joint architecture that utilises Named Entity Recognition (NER), Retrieval-Augmented Generation (RAG) and LLMs to gather, compare and analyse information in the texts provided by organizers. We extract entities potentially capable of containing hallucinations with NER, aggregate relevant topics for them using RAG, then verify and provide a verdict on the extracted information using the LLMs. This approach allowed with a certain level of quality to find hallucinations not only in facts, but misspellings in names and titles, which was not always accepted by human annotators in ground truth markup. We also point out some inconsistencies within annotators spans, that perhaps affected scores of all participants.

## introduction modern advances in the field of text generation models provide artificial texts of a high quality that are hardly distinguishable from humanwritten texts at fluent reading. state-of-the-art instruction-tuned or reasoning large language models (llms) are capable of generating an answer to any user query, but despite the attractiveness and conciseness of the answer, its appropriateness and accuracy often remained a controversial issue. llms are capable of retaining massive amounts of factual knowledge and use it to answer user queries, but they are still prone to generating hallucinations -statements that appear plausible but are factually incorrect or unverifiable @xcite @xcite . hallucinations pose a critical challenge in applications that demand high factual accuracy, such as medical or legal domains @xcite . if a bit earlier we were concerned about detecting ai-generated content, nowadays with the increasing amount of the internet being flooded with texts from llms, the focus on verification and validation of information is crucial @xcite . that brings us to the point where detecting and mitigating these hallucinations is essential for enhancing the reliability and trustworthiness of llm outputs. the semeval 2025 task 3 @xcite ) poses a challenge in seeking hallucination within model answers on factual questions about famous people, locations or biological species. as most of these questions contain some sort of entity, we decided to employ named entity recognition (ner) approach to determine for which entities there might be a relevant context. for determined entities we leverage retrieval-augmented generation (rag), as all questions from the dataset could be answered with context from wikipedia. relevant retrieved context together with model answer were given to llms that were tasked to evaluate the correctness of model's answer based on provided context and this were done twice obtain multiple opinions. after that, the final model had to edit the suggestions from llm judges into initial answer. through this method, we aim to detect and fix factual hallucinations from the text and thus contribute to ongoing efforts in hallucination detection and mitigation. ## task description the objective of this task is to identify text spans that correspond to hallucinations in outputs, generated by llms. provided data includes 14 languages and outputs from various publicly available llms. participants receive an llm-generated text in three formats: we have participated with our framework presenting approach for english, but it could be extended to other languages as well. ## proposed system our system employs a combination of retrieval-augmented generation, named entity recognition, and usage of llms to semi-automatically verify factual consistency and detect hallucinations in given answers. the workflow consists of the following key components: 1. ner module extracts entities from the question. it is illustrated in figure 1 . ## experimental setup we focused only on english subset, however, our system can be used for any language as long as llms support this language and there are knowledge sources in this language. to retrieve ner entities, we use deeppavlov ner_conll2003_bert model @xcite . we decided to extract sources from wikipedia because after analysing the data provided, a significant amount of samples contain facts or references to various events. after retrieving the top-5 wikipedia pages with the langchain foot_0 loader and taking the most relevant one, we concatenate it with provided llm answer and forward it to two llm-judges: llama3 8b @xcite and deepseek-r1 8b (deepseek-ai, 2025). outputs from judges are concatenated with model answer and are given to the larger judge, llama3.1 70b-instruct. the latter judge should spot and edit the mistakes in the initial answer based on the smaller judges' opinions. then, we check the difference between the original model answer and the edited answer. the spans that differed were stored and provided as the final answer. additionally, we compared ner entities from the model answer and the question with cosine similarity, to find the inconsistencies and mistakes in the names. ## results and discussion although our system's spans do not fully overlap with spans from annotators, with 0.44425 iou score (see ## conclusion we presented a system that combines ner and rag modules and after that utilises llms to detect spans of hallucinated output and showed how it can be used to edit factual mistakes in model answers. it can be further modified to work with several knowledge sources or employ more llm judges to obtain more opinions and decrease the risk of hallucinations of the judges.