How Well Do Large Language Models Truly Ground? (2024)

Abstract: To reduce issues like hallucinations and lack of control in Large Language Models (LLMs), a common method is to generate responses by grounding on external contexts given as input, known as knowledge-augmented models. However, previous research often narrowly defines “grounding” as just having the correct answer, which does not ensure the reliability of the entire response. To overcome this, we propose a stricter definition of grounding: a model is truly grounded if it (1) fully utilizes the necessary knowledge from the provided context, and (2) stays within the limits of that knowledge. We introduce a new dataset and a grounding metric to evaluate model capability under the definition. We perform experiments across 25 LLMs of different sizes and training methods and provide insights into factors that influence grounding performance. Our findings contribute to a better understanding of how to improve grounding capabilities and suggest an area of improvement toward more reliable and controllable LLM applications.

## introduction large language models (llms) have shown superior performance on various tasks by leveraging the extensive world knowledge embedded in their parameters. however, these models often produce hallucinations @xcite , lack controllability @xcite , and have trouble integrating knowledge that changes over time @xcite . additionally, they may not contain specialized knowledge unique to certain entities, such as company-specific terminology, or private information not contained in the training data. although it is technically possible to inject new knowledge by further training llms on a specific corpus, this approach is generally inefficient and not practical in many scenarios @xcite @xcite . to address these issues, various systems foot_0 and work @xcite @xcite have explored methods where such dynamic, specialized, or private contexts provided by users or general world knowledge contexts retrieved from a large corpus (retrieval-augmented models) are provided to llms as additional inputs. while previous work has shown enhanced performance by allowing llms to ground their outputs on external contexts compared to solely relying on the llm's inherent knowledge @xcite @xcite , whether the model well-grounds to the contexts is usually measured by simply checking whether the generated response contains the answer @xcite @xcite or evaluating over nli model to see whether the knowledge from given context correlates with generated response @xcite . however, in some cases, this may not be sufficient and it may be more important to ensure that the entire generated response is truly grounded on the given external contexts. for example, let's consider the scenario in figure 1 , where a company's hr team is utilizing an llm to question the qualifications of candidates by providing their resumes as external contexts and prompting the llm to provide an answer to questions about the candidates based on their resumes. response 1 omits essential information about the candidate and response 2 contains misinformation about the candidate due to generating knowledge contained in its parameters; both cases do not truly represent the candidate's qualifications. it either harms the applicant by missing important information or makes the applicant overly qualified, disadvantaging other applicants. in this study, we introduce a strict definition of grounding: a model is truly grounding on given contexts when it (1) uses all essential knowledge from the contexts and (2) strictly adheres to their scope in response generation without hallucinated information foot_1 . to quantify this definition, we introduce an automatic grounding metric that extends upon @xcite for fine-grained evaluation. furthermore, we curate a new dataset incorporating crucial factors influencing llms' response (i.e., entity popularity, context length), to understand their impact on llm responses. lastly, we present a revised version of the dataset that modifies factual knowledge in external contexts to identify the knowledge sources in responses. we conduct experiments across 25 llms of different sizes and training methods to explore which model attributes significantly contribute to grounding ability and identify some important factors. ## grounding in this paper, we define that the model grounds well more strictly and share a dataset and metric to measure performance under the definition. in section 3.1, we define the grounding ability and share its importance with various use cases. in section 3.2, we share details of how we construct the dataset, and in section 3.3, we formulate an automatic metric to measure the grounding ability. ## experiments we experiment with 25 llms of various sizes and training methods (instruction-tuning, rlhf, dpo). from the results, we share interesting findings of how different factors of llms and different characteristics of input context lead to their grounding ability. section 4.1 shows brief details of the models we evaluate. section 4.2 shows how different factors of llms lead to their grounding ability and interesting findings. details of the input format, generation configurations, and others are in appendix c. ## conclusion in this paper, we introduce a strict definition of "grounding" to external contexts when given as input. to evaluate and analyze grounding performance under the definition, we propose a new dataset and grounding metric. in our extensive evaluation of 25 llms across four dataset scenarios, we observed various insights. rather than model size, various training techniques and base models tend to affect more on grounding performance. models find it challenging to utilize all necessary knowledge when generating a response. by presenting the performance of various models on different dataset settings, we provide valuable perspectives to the ongoing discourse on enhancing llm grounding abilities and practical guidance for choosing suitable models for applications that require generating response by truly grounding on a given context.