More Documents, Same Length: Isolating the Challenge of Multiple
  Documents in RAG
Authors: name: Shahar Levy; name: Nir Mazor; name: Lihi Shalmon; name: Michael Hassid; name: Gabriel Stanovsky
Published: 2025-03-06 12:38:17

Abstract: Retrieval-augmented generation (RAG) provides LLMs with relevant documents.
Although previous studies noted that retrieving many documents can degrade
performance, they did not isolate how the quantity of documents affects
performance while controlling for context length. We evaluate various language
models on custom datasets derived from a multi-hop QA task. We keep the context
length and position of relevant information constant while varying the number
of documents, and find that increasing the document count in RAG settings poses
significant challenges for LLMs. Additionally, our results indicate that
processing multiple documents is a separate challenge from handling long
contexts. We also make the datasets and code available:
https://github.com/shaharl6000/MoreDocsSameLen .