Less LLM, More Documents: Searching for Improved RAG
Authors: name: Jingjie Ning, name: Yibo Kong, name: Yunfan Long, name: Jamie Callan
Organization: Carnegie Mellon University School of Computer Science

Retrieval-Augmented Generation (RAG) couples document retrieval with large
language models (LLMs). While scaling generators improves accuracy, it also
raises cost and limits deployability. We explore an orthogonal axis: enlarging
the retriever's corpus to reduce reliance on large LLMs. Experimental results
show that corpus scaling consistently strengthens RAG and can often serve as a
substitute for increasing model size, though with diminishing returns at larger
scales. Small- and mid-sized generators paired with larger corpora often rival
much larger models with smaller corpora; mid-sized models tend to gain the
most, while tiny and large models benefit less. Our analysis shows that
improvements arise primarily from increased coverage of answer-bearing
passages, while utilization efficiency remains largely unchanged. These
findings establish a principled corpus-generator trade-off: investing in larger
corpora offers an effective path to stronger RAG, often comparable to enlarging
the LLM itself.