Are Reasoning Models More Prone to Hallucination?
Authors: name: Zijun Yao, name: Yantao Liu, name: Yanxu Chen, name: Jianhui Chen, name: Junfeng Fang, name: Lei Hou, name: Juanzi Li, name: Tat-Seng Chua

Recently evolved large reasoning models (LRMs) show powerful performance in
solving complex tasks with long chain-of-thought (CoT) reasoning capability. As
these LRMs are mostly developed by post-training on formal reasoning tasks,
whether they generalize the reasoning capability to help reduce hallucination
in fact-seeking tasks remains unclear and debated. For instance, DeepSeek-R1
reports increased performance on SimpleQA, a fact-seeking benchmark, while
OpenAI-o3 observes even severer hallucination. This discrepancy naturally
raises the following research question: Are reasoning models more prone to
hallucination? This paper addresses the question from three perspectives. (1)
We first conduct a holistic evaluation for the hallucination in LRMs. Our
analysis reveals that LRMs undergo a full post-training pipeline with cold
start supervised fine-tuning (SFT) and verifiable reward RL generally alleviate
their hallucination. In contrast, both distillation alone and RL training
without cold start fine-tuning introduce more nuanced hallucinations. (2) To
explore why different post-training pipelines alters the impact on
hallucination in LRMs, we conduct behavior analysis. We characterize two
critical cognitive behaviors that directly affect the factuality of a LRM: Flaw
Repetition, where the surface-level reasoning attempts repeatedly follow the
same underlying flawed logic, and Think-Answer Mismatch, where the final answer
fails to faithfully match the previous CoT process. (3) Further, we investigate
the mechanism behind the hallucination of LRMs from the perspective of model
uncertainty. We find that increased hallucination of LRMs is usually associated
with the misalignment between model uncertainty and factual accuracy. Our work
provides an initial understanding of the hallucination in LRMs.