ReaRAG: Knowledge-guided Reasoning Enhances Factuality of Large
  Reasoning Models with Iterative Retrieval Augmented Generation
Authors: name: Zhicheng Lee; name: Shulin Cao; name: Jinxin Liu; name: Jiajie Zhang; name: Weichuan Liu; name: Xiaoyin Che; name: Lei Hou; name: Juanzi Li
Published: 2025-03-27 17:44:18

Abstract: Large Reasoning Models (LRMs) exhibit remarkable reasoning abilities but rely
primarily on parametric knowledge, limiting factual accuracy. While recent
works equip reinforcement learning (RL)-based LRMs with retrieval capabilities,
they suffer from overthinking and lack robustness in reasoning, reducing their
effectiveness in question answering (QA) tasks. To address this, we propose
ReaRAG, a factuality-enhanced reasoning model that explores diverse queries
without excessive iterations. Our solution includes a novel data construction
framework with an upper bound on the reasoning chain length. Specifically, we
first leverage an LRM to generate deliberate thinking, then select an action
from a predefined action space (Search and Finish). For Search action, a query
is executed against the RAG engine, where the result is returned as observation
to guide reasoning steps later. This process iterates until a Finish action is
chosen. Benefiting from ReaRAG's strong reasoning capabilities, our approach
outperforms existing baselines on multi-hop QA. Further analysis highlights its
strong reflective ability to recognize errors and refine its reasoning
trajectory. Our study enhances LRMs' factuality while effectively integrating
robust reasoning for Retrieval-Augmented Generation (RAG).