Retrieval-Enhanced Machine Learning: Synthesis and Opportunities
Authors: name: To Eun Kim; name: Alireza Salemi; name: Andrew Drozdov; name: Fernando Diaz; name: Hamed Zamani
Published: 2024-07-17 20:01:21

Abstract: In the field of language modeling, models augmented with retrieval components
have emerged as a promising solution to address several challenges faced in the
natural language processing (NLP) field, including knowledge grounding,
interpretability, and scalability. Despite the primary focus on NLP, we posit
that the paradigm of retrieval-enhancement can be extended to a broader
spectrum of machine learning (ML) such as computer vision, time series
prediction, and computational biology. Therefore, this work introduces a formal
framework of this paradigm, Retrieval-Enhanced Machine Learning (REML), by
synthesizing the literature in various domains in ML with consistent notations
which is missing from the current literature. Also, we found that while a
number of studies employ retrieval components to augment their models, there is
a lack of integration with foundational Information Retrieval (IR) research. We
bridge this gap between the seminal IR research and contemporary REML studies
by investigating each component that comprises the REML framework. Ultimately,
the goal of this work is to equip researchers across various disciplines with a
comprehensive, formally structured framework of retrieval-enhanced models,
thereby fostering interdisciplinary future research.