DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for
  Dynamic Reranking in Retrieval-Augmented Generation
Authors: name: Jiashuo Sun; name: Xianrui Zhong; name: Sizhe Zhou; name: Jiawei Han
Published: 2025-05-12 05:19:01

Abstract: Retrieval-augmented generation (RAG) systems combine large language models
(LLMs) with external knowledge retrieval, making them highly effective for
knowledge-intensive tasks. A crucial but often under-explored component of
these systems is the reranker, which refines retrieved documents to enhance
generation quality and explainability. The challenge of selecting the optimal
number of documents (k) remains unsolved: too few may omit critical
information, while too many introduce noise and inefficiencies. Although recent
studies have explored LLM-based rerankers, they primarily leverage internal
model knowledge and overlook the rich supervisory signals that LLMs can
provide, such as using response quality as feedback for optimizing reranking
decisions. In this paper, we propose DynamicRAG, a novel RAG framework where
the reranker dynamically adjusts both the order and number of retrieved
documents based on the query. We model the reranker as an agent optimized
through reinforcement learning (RL), using rewards derived from LLM output
quality. Across seven knowledge-intensive datasets, DynamicRAG demonstrates
superior performance, achieving state-of-the-art results. The model, data and
code are available at https://github.com/GasolSun36/DynamicRAG