Annotating Hallucinations in Question-Answering using Rewriting (2025)

Abstract: Hallucinations pose a persistent challenge in open-ended question answering (QA). Traditional annotation methods, such as span-labelling, suffer from inconsistency and limited coverage. In this paper, we propose a rewriting-based framework as a new perspective on hallucinations in open-ended QA. We report on an experiment in which annotators are instructed to rewrite LLM-generated answers directly to ensure factual accuracy, with edits automatically recorded. Using the Chinese portion of the Mu-SHROOM dataset, we conduct a controlled rewriting experiment, comparing fact-checking tools (Google vs. GPT-4o), and analysing how tool choice, annotator background, and question openness influence rewriting behaviour. We find that rewriting leads to more hallucinations being identified, with higher inter-annotator agreement, than span-labelling.

## introduction with the rapid advancement of large language models (llms), hallucination has emerged as a central research topic, spanning annotation @xcite , detection @xcite , mitigation @xcite , and interoperability @xcite . hallucinations are typically defined as content in a model's output that is not supported by the input @xcite @xcite . to capture such phenomena, prior work @xcite @xcite proposed fine-grained annotation schemas that treat hallucination identification as a spanlabelling task: annotators are asked to highlight spans in model outputs that constitute hallucinations, sometimes followed by labelling the error types or suggesting corrections. ## the mu-shroom dataset mu-shroom @xcite ) is a multilingual hallucination dataset designed to support research on detecting hallucinations produced by llms in qa across 14 languages. the dataset comprises questions and answers to these questions that were generated by various llms. each answer is annotated with two types of token-level labels derived from human judgments: a soft label, which we were able to recruit enough native speakers for an in-lab experiment. to construct mu-shroom, vázquez et al. first manually selected 200 wikipedia pages. for each page, they wrote one question that could be answered using information contained within the page. then, they used llms to generate answers for each question. among these llm-generated answers, those that were fluent and relevant but appeared to contain hallucinations were manually selected and added to the corpus. 2 subsequently, annotators were hired to label hallucinations in the llm-generated answers as a span-labelling task. during annotation, they were allowed to consult wikipedia, not only the page associated with each question, but the entire encyclopedia. for the chinese portion, each item was annotated by up to six annotators, with each annotator labelling 20 items. despite this carefully constructed process, the aforementioned limitations of span labelling sometimes lead to annotations that appear inconsistent or unintuitive. for example, in one case, only the first two characters of "制片人" (producer) foot_2 are marked as hallucinations. one additional contributing factor is the degeneration exhibited by the llms that generate these answers @xcite , which makes it difficult for annotators to determine precisely which parts should be labelled as hallucinations. for instance, in a degenerated segment such as "(18% 白人) (moskva slezam ne verit, 1% blondynki)(10%的白人)", which is completely nonsensical in chinese, some tokens are marked as hallucinations, while others are not. in addition, the annotations exhibit inconsistencies in handling symbols: characters like '%' and ')' are sometimes labelled as hallucinations and sometimes left unmarked. ## method in this section, we elaborate on experimental settings for our rewriting experiments. ## hypotheses we aim to understand hallucinations in qa by examining how humans correct answers that contain factual errors. as an initial step, we are primarily interested in the factors that influence how people revise answers produced by llm-based qa systems. we are also interested in the effectiveness of rewriting as a method for understanding hallucinations, in comparison to span-labelling. to answer these two research questions, we form the following hypotheses. first, we gave participants access to two external tools: google and gpt-4o. as noted by @xcite , llms are effective tools for assisting in hallucination annotation. compared to search engines like google, llms have a stronger ability to interpret annotators' queries, reducing the need for careful query formulation, and to summarise the most relevant information from retrieved sources, thereby streamlining the fact-checking process. based on this, our first hypothesis (referred to as hypothesis h 1 ) was that, compared to participants using google for fact-checking, those using gpt-4o would: (1) correct answers more quickly, as measured by the time spent revising a given number of answers; (2) identify more hallucinations, as indicated by the number of tokens deleted from the original answers; and (3) add more information, as measured by the number of tokens inserted into the revised answers. second, intuitively, participants with a background in computer science or artificial intelligence may be likely have a better understanding of hallucinations and greater proficiency with tools such as google and gpt-4o. accordingly, we expected these participants to: (1) complete the task more quickly, (2) identify more hallucinations, and (3) add more information to the revised answers (referred to as hypothesis h 2 ). third, the openness of a question may influence how participants correct the answer. closedended questions (e.g., "what countries are there in the americas?") typically require concrete, factbased answers, such as lists of entities, that are relatively easy to verify. in contrast, open-ended questions (e.g., "what is the history of america?") tend to elicit narrative responses that integrate diverse types of information, making fact-checking more complex. thus, it seems plausible to hypothesise that, compared to closed-ended questions, rewritings in response to open-ended questions would involve: (1) more identified hallucinations, (2) more added information, and (3) lower inter-annotator agreement (referred to as hypothesis h 3 ). finally, as discussed in the introduction, we hoped that, compared to conventional span-labelling-based hallucination annotation, rewriting would help annotators remain more engaged and focused. based on this assumption, we anticipated that rewriting would lead annotators to: (1) identify more hallucinations, (2) add more information, and (3) achieve higher inter-annotator agreement (referred to as hypothesis h 4 ). ## data post-processing to test the hypotheses in section 4 and enable further research based on our dataset, we postprocessed the recorded rewriting operations and computed inter-annotator agreements (iaa). the processed data is available at: https://github. com/a-quei/hallucination-rewriting.git . ## results here we report on our testing of the four hypotheses we put forward in section 4. ## conclusion this paper proposes rewriting as a new lens through which to observe hallucinations in open-ended question answering. by instructing annotators to revise llm-generated answers directly, while tracking their edits, we obtained a corpus that not only highlights hallucinated content but also reveals how such content is corrected. we then analysed factors that may influence rewriting behaviours. contrary to expectations, the choice of fact-checking tool (google vs. gpt-4o) and participants' technical background had limited impact on annotation quality, while question openness primarily affected the amount of added information. compared to traditional span-labelling-based hallucination anno-tation, our analyses show that rewriting leads to the identification of more hallucinations and yields higher inter-annotator agreement. these findings suggest that rewriting encourages more engaged and consistent annotation, making it a promising alternative for the creation of datasets that can enhance the research community's understanding of hallucination in nlg, not only in question answering but potentially in other nlg tasks as well.