Unraveling Misinformation Propagation in LLM Reasoning (2025)

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, positioning them as promising tools for supporting human problem-solving. However, what happens when their performance is affected by *misinformation*, i.e., incorrect inputs introduced by users due to oversights or gaps in knowledge? Such misinformation is prevalent in real-world interactions with LLMs, yet how it propagates within LLMs’ reasoning process remains underexplored. Focusing on mathematical reasoning, we present a comprehensive analysis of how misinformation affects intermediate reasoning steps and final answers. We also examine how effectively LLMs can correct misinformation when explicitly instructed to do so. Even with explicit instructions, LLMs succeed less than half the time in rectifyingmisinformation, despite possessing correct internal knowledge, leading to significant accuracy drops (10.02% – 72.20%), and the degradation holds with thinking models (4.30% – 19.97%). Further analysis shows that applying factual corrections early in the reasoning process most effectively reduces misinformation propagation, and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality. Our work offers a practical approach to mitigating misinformation propagation.

## introduction large language models (llms) have shown remarkable progress in complex reasoning @xcite , leading to their widespread use in a variety of human-llm interactions @xcite @xcite . under such interactive reasoning scenarios, users routinely provide external information to llms (ope-nai, 2024a). however, due to human oversight or 1. we are given @xmath0, where a is the area, d is the depth, t is the top width, and b is the bottom width. 2. we are given that a is 550 m 2 , t is 14 m, and b is 8 m. 3. we can substitute these values into the formula: 550 = (14 -8) × d. 4. we can rearrange the formula: 550 / (14 -8) = d. … ... figure 1 : illustration of misinformation propagation in llm reasoning. a user provides input containing misinformation due to a lack of domain knowledge. the llm propagates this misinformation, i.e., follows user-provided erroneous equations, leading to an incorrect final answer. this propagation occurs even though llm designers have instructed the model to correct such misinformation when detected. despite its prevalence, how misinformation from user input impacts llm reasoning and how to correct such misinformation remains underexplored. existing research has worked on addressing misinformation in various llm interaction settings, such as open-domain question answering, fact verification, social media, math education, and codegeneration @xcite @xcite @xcite . however, these studies lack a systematic analysis of the fine-grained effects of misinformation on individual reasoning steps. in parallel, studies on error propagation and selfcorrection provide detailed analyses of how errors in early llm outputs influence subsequent reasoning processes @xcite @xcite , as well as strategies for correcting such errors @xcite . nonetheless, the errors examined in these works are typically generated by the model itself during reasoning, rather than introduced externally by users at the instruction level. as a result, they differ significantly in both distribution and their effects on model reasoning. to address this gap, we systematically investigate misinformation propagation in llm reasoning, including its impact and potential mitigation strategies. our study focuses on the mathematical reasoning context, where misinformation is explicit and quantitatively measurable. we simulate misinformation using synthetically generated erroneous equations (fig. 1 ), modeling common types of human mathematical misinformation, such as inappropriate operations @xcite , misapplied values @xcite , and misused operands @xcite . these misinformation patterns are sampled using designed heuristic rules that reflect documented cognitive error types. we use these patterns to guide an llm in generating plausible but incorrect questions, which are then used as inputs for llms to perform chain-of-thought (cot) reasoning @xcite or thinking before reasoning @xcite . building on the simulation design, we first analyze the impact of misinformation on llm reasoning (sec. 5), examining both final outcomes and intermediate steps. surprisingly, even when llms are capable of answering questions correctly in the absence of misinformation, they still struggle to correct misinformation with explicit instructions to do so. we observe accuracy drops ranging from 10.02% all the way up to 72.20% across different instruction llms, and the degradation holds for further post-trained thinking models (4.30% to 19.97%). moreover, llms frequently fail to correct misinformation: in 17.00% of such cases, no correction is attempted, and in 34.75% of cases, the model produces non-factual corrections-either leaving some errors unaddressed or introducing new ones. these results underscore llms' vulnerability to misinformation and their limited steerability to correct it via explicit instructions. the vulnerability originates in the instruction-tuning stage, and cannot be fully resolved through additional reasoning post-training. to reliably steer models and improve reasoning factuality, we further investigate how to correct misinformation and mitigate its propagation (sec. 6). we first conduct a controlled analysis, enforcing factual, non-factual, or no corrections at various reasoning steps and assessing their direct effects on final answers. the results indicate that early, factual corrections are critical and yield the most effective recovery from accuracy losses. building on this insight, we fine-tune llms on a synthetic dataset designed to introduce factual corrections at the first reasoning step. this approach, for example, raises gpt-4o-mini's accuracy from 85.64% to 95.68%-just 0.03% below the accuracy achieved without any misinformation. additionally, gpt-4omini attempts to correct all misinformation, with 80.37% of those corrections being accurate. in summary, we present a systematic pipeline to evaluate the impact of misinformation on llm reasoning, across instruction models and thinking models, and investigate strategies to mitigate its propagation. by analyzing both final answers and intermediate reasoning steps, we demonstrate that simply instructing models to correct misinformation is insufficient, highlighting llms' vulnerability and limited steerability in the presence of misinformation inputs. to enhance factuality and controllability, we explore correction strategies and fine-tune llms using data that applies optimal corrections. our findings underscore the effectiveness of early factual corrections and targeted fine-tuning. this study offers actionable insights for developing llms to better handle misinformed inputs. ## methodology we outline the problem formulation, testing data collection, experiment design, and evaluation framework. our pipeline is shown in fig. 2 . ## experimental setup tasks and datasets. we collect 400 math questions from canonical math datasets, including mathqa @xcite , math @xcite , gsm8k @xcite , and metamath @xcite , as our test set. data selection and processing details are in app. b.1. testing models. we select instruction models with complex reasoning abilities across various scales of sizes. specifically, we test llama-3.2 (1b, 3b, 11b, and 90b) foot_4 @xcite , mixtral (8×7b and 8×22b) @xcite , qwen-2 (72b) @xcite , and gpt-4o-mini @xcite . we also experimented on several thinking models, including deepseek-r1 distilled qwen-2.5 models (1.5b and 14b) (deepseek-ai, 2025), qwen-3 (235b-a22b-thinking-2507) @xcite , and deepseek-r1-0528 (deepseek-ai, 2025), with thinking enabled. 8 setup details are in app. e. ## impact of misinformation in this section, we investigate how misinformation impacts llm reasoning in two aspects: (i) llms' default behaviors under misinformation (sec. 5.1), and (ii) llms' steerability to correct misinformation via instructions (sec. 5.2). takeaways: (i) llms, by default, follow misinformation as instructions. (ii) even when llms are instructed to correct misinformation, they fail to do so on all questions where they possess internal knowledge. ## mitigating propagation via correction to reliably steer models to mitigate misinformation propagation via correction, we (i) investigate what factors make corrections more effective (sec. 6.1) and (ii) whether fine-tuning with effective corrections mitigates propagation and improves reasoning factuality (sec. 6.2). takeaways: (i) early factual corrections are most effective at mitigating misinformation propagation. (ii) fine-tuning models for early corrections significantly improves reasoning performance under misinformation, although still not recovering the original performance. ## conclusion we systematically study the impact of misinformation on llm reasoning and mitigation effectiveness. misinformation reduces llm reasoning accuracy and llms struggle to factually correct it, even with correct internal knowledge and explicit instructions. this reveals llms' vulnerability to misinformation and limited steerability to mitigate its propagation. we show two remedies: (i) early factual corrections in reasoning, and (ii) fine-tuning for these corrections, which sharply mitigate misinformation propagation and restore factuality, offering valuable insights for handling misinformation.