MIRIAD: Augmenting LLMs with millions of medical query-response pairs
Authors: name: Qinyue Zheng, name: Salman Abdullah, name: Sam Rawal, name: Cyril Zakka, name: Sophie Ostmeier, name: Maximilian Purk, name: Eduardo Reis, name: Eric J. Topol, name: Jure Leskovec, name: Michael Moor

LLMs are bound to transform healthcare with advanced decision support and
flexible chat assistants. However, LLMs are prone to generate inaccurate
medical content. To ground LLMs in high-quality medical knowledge, LLMs have
been equipped with external knowledge via RAG, where unstructured medical
knowledge is split into small text chunks that can be selectively retrieved and
integrated into the LLMs context. Yet, existing RAG pipelines rely on raw,
unstructured medical text, which can be noisy, uncurated and difficult for LLMs
to effectively leverage. Systematic approaches to organize medical knowledge to
best surface it to LLMs are generally lacking. To address these challenges, we
introduce MIRIAD, a large-scale, curated corpus of 5,821,948 medical QA pairs,
each rephrased from and grounded in a passage from peer-reviewed medical
literature using a semi-automated pipeline combining LLM generation, filtering,
grounding, and human annotation. Unlike prior medical corpora, which rely on
unstructured text, MIRIAD encapsulates web-scale medical knowledge in an
operationalized query-response format, which enables more targeted retrieval.
Experiments on challenging medical QA benchmarks show that augmenting LLMs with
MIRIAD improves accuracy up to 6.7% compared to unstructured RAG baselines with
the same source corpus and with the same amount of retrieved text. Moreover,
MIRIAD improved the ability of LLMs to detect medical hallucinations by 22.5 to
37% (increase in F1 score). We further introduce MIRIAD-Atlas, an interactive
map of MIRIAD spanning 56 medical disciplines, enabling clinical users to
visually explore, search, and refine medical knowledge. MIRIAD promises to
unlock a wealth of down-stream applications, including medical information
retrievers, enhanced RAG applications, and knowledge-grounded chat interfaces,
which ultimately enables more reliable LLM applications in healthcare.