Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning (2024)

Abstract: Despite the dramatic progress in Large Language Model (LLM) development, LLMs often provide seemingly plausible but not factual information, often referred to as hallucinations. Retrieval-augmented LLMs provide a non-parametric approach to solve these issues by retrieving relevant information from external data sources and augment the training process. These models help to trace evidence from an externally provided knowledge base allowing the model predictions to be better interpreted and verified. In this work, we critically evaluate these models in their ability to perform in scientific document reasoning tasks. To this end, we tuned multiple such model variants with science-focused instructions and evaluated them on a scientific document reasoning benchmark for the usefulness of the retrieved document passages. Our findings suggest that models justify predictions in science tasks with fabricated evidence and leveraging scientific corpus as pretraining data does not alleviate the risk of evidence fabrication.

## introduction large language models (llms) perform competitively in a majority of natural language processing (nlp) tasks, but tend to hallucinate with seemingly plausible but misleading predictions @xcite with no clear explanations or justifications for their predictions @xcite . recently, retrieval-augmented lms have helped address these issues by augmenting the llms with non-parametric memory by using neural retriever to extract relevant information from external knowledge resources like document corpora @xcite . retrieving external knowledge helps models update with new knowledge, inject domain specific data, and memorize long-tail knowledge. these models are also relatively small in the number of parameters and require less training and inference costs @xcite . while retrieval-augmented lms (ralm) are shown to perform well on knowledge-intensive tasks @xcite , we have very limited understanding on their ability to perform on the science-focused downstream tasks. for example, we can provide scientific documents as external knowledge at test time, and test the ability of the model to perform on science question and answering (qa) task. in this setup, the model retrieves scientific documents relevant to the question, and then generates an answer based on the retrieved documents. retrieved documents help the model predictions to be better interpreted and verified. at the same time, we can assess the trustworthiness of these models to understand whether they justify the model predictions with accurate and relevant evidence @xcite . recognizing any failure modes is essential to ensuring the safe deployment, and avoiding potential risks or negative consequences of these models, specially across multiple scientific use cases and applications. this work conducts an evaluation on ralms to improve our understanding of these models to perform on science tasks. to this end, we used the atlas model architecture @xcite as an instance of the retrieval-based language model family to drive our experiments. to adopt the models for science tasks, we provide a collection of scientific documents as external corpus during the pretraining, instruction tuning, and evaluation stages. we evaluate the model performance on scirepeval @xcite benchmark to test whether model recognizes different scientific domains and disciplines from the given scientific documents. our hypothesis is that these models will be able to retrieve relevant information from scientific documents, integrate knowledge from diverse scientific domains, and reason over complex scientific concepts. in particular, we evaluate the usefulness of 2 problem formulation and eval setup ## related work llms can be augmented with various external modules such as document corpus @xcite , vector databases foot_2 , etc. typically, the augmentation is accomplished in two ways, namely sparse (such as bag of words) and dense where neural network is used to encode documents. dense retrievers are widely used in present time mainly due to the good representation capability of neural networks. recent works suggest that the retrieval-augmented llms has significant improvement over the standard llms across various tasks especially with respect to scale @xcite . realm and rag are the initial efforts where they train the retriever and language model by representing documents as latent variable, and minimizing the language model objective @xcite . realm leverages masked-language modeling as an objective to pretrain the model in end to end fashion. however, it is computationally very expensive to train these models that requires to retrain the entire index with new knowledge. @xcite explored the concept of query-side finetuning that only refreshes the query encoder whereas document encoder remains frozen. izacard et al. @xcite proposed various ways to improve the retrieval augmented models, including novel learning objectives to align retriever with the language model @xcite . furthermore, retro @xcite shows the benefits of scaling the retrieval memory to trillions of tokens. atlas @xcite experiments with various design and training configurations for retrieval augmented models with a focus on few shot learning ability, while atlantic @xcite shows an approach to utilize heterogeneous graph information to create a structure-aware rag technique. finally, @xcite provides a word of caution, finding that while rag-based language models tend to be more confident if fine-tuned on scientific documents, they also exhibit false confidence even for incorrect predictions. ## conclusion in this study, we explored the efficacy of retrieval augmented language models on science tasks. our experiments were based on atlas model which is a state of the art retrieval augmented language model with few shot capability. we performed a systematic evaluation on the performance of different atlas model variants in two scientific document reasoning tasks. our experiments on the pretrained atlas model reveal that the model demonstrates acceptable performance in science tasks but the evidences are fabricated. we also observe that pretraining the model with scientific corpus does not alleviate evidence fabrication. we plan to develop techniques to alleviate these issues in a future work.