Test-Time Scaling in Reasoning Models Is Not Effective for
  Knowledge-Intensive Tasks Yet
Authors: name: James Xu Zhao; name: Bryan Hooi; name: See-Kiong Ng
Published: 2025-09-08 16:28:25

Abstract: Test-time scaling increases inference-time computation by allowing models to
generate long reasoning chains, and has shown strong performance across many
domains. However, in this work, we show that this approach is not yet effective
for knowledge-intensive tasks, where high factual accuracy and low
hallucination rates are essential. We conduct a comprehensive evaluation of
test-time scaling using 12 reasoning models on two knowledge-intensive
benchmarks. Our results reveal that increasing test-time computation does not
consistently improve accuracy and, in many cases, it even leads to more
hallucinations. We then analyze how extended reasoning affects hallucination
behavior. We find that reduced hallucinations often result from the model
choosing to abstain after thinking more, rather than from improved factual
recall. Conversely, for some models, longer reasoning encourages attempts on
previously unanswered questions, many of which result in hallucinations. Case
studies show that extended reasoning can induce confirmation bias, leading to
overconfident hallucinations. Despite these limitations, we observe that
compared to non-thinking, enabling thinking remains beneficial. Code and data
are available at https://github.com/XuZhao0/tts-knowledge