Grounding Gaps in Language Model Generations (2024)

Abstract: Effective conversation requires common ground: a shared understanding between the participants. Common ground, however, does not emerge spontaneously in conversation. Speakers and listeners work together to both identify and construct a shared basis while avoiding misunderstanding. To accomplish grounding, humans rely on a range of dialogue acts, like clarification (What do you mean?) and acknowledgment (I understand.). However, it is unclear whether large language models (LLMs) generate text that reflects human grounding. To this end, we curate a set of grounding acts and propose corresponding metrics that quantify attempted grounding. We study whether LLM generations contain grounding acts, simulating turn-taking from several dialogue datasets and comparing results to humans. We find that—compared to humans—LLMs generate language with less conversational grounding, instead generating text that appears to simply presume common ground. To understand the roots of the identified grounding gap, we examine the role of instruction tuning and preference optimization, finding that training on contemporary preference data leads to a reduction in generated grounding acts. Altogether, we highlight the need for more research investigating conversational grounding in human-AI interaction.

## introduction in dialogue, common ground refers to the mutual knowledge, beliefs, and assumptions shared by participants in a conversation. this shared understanding is essential for effective communication, as it underpins the ability of individuals to interpret, predict, and respond to each other's statements and actions accurately @xcite . through each ⋆ equal contribution. figure 1 : mental health supporters carefully employ clarification questions (one of the three grounding acts) with a seeker, taking multiple turns to ground. in contrast, simulated supporters-with the same conversational context-generate presumptive answers. failing to construct common ground in humanhuman conversation can be at best misleading and at worst harmful. consider mental-health support: if there's any indication of risk to a client (e.g., suicidal ideation or intentions of self-harm), a healthcare professional will ask clarifying questions to assess risk. failure to do so has harmful consequences (unnecessary hospitalization) which can traumatize a client and place an unjustified financial burden @xcite . in domains like education, ineffective grounding might result in misunderstanding a student, resulting in irrelevant feedback @xcite . from an nlp perspective, establishing and leveraging common ground is a complex challenge: dialogue agents must recognize and utilize implicit aspects of human communication. recent chat-based large language models (llms), however, are designed explicitly for following instructions. while humans carefully construct common ground, llms are trained to directly act on commands specified by end-users @xcite . we hypothesize that the instruction-following paradigm-a combination of supervised fine-tuning (sft) and preference optimization (po through rlhf, for example)-might result in discrepancies between how humans actually ground in dialogue, and how llms generate text similar to human grounding. we call this discrepancy the grounding gap. despite the grounding gap, llms interact regularly with humans across various applications. for a subset of interactions, however, llms should generate grounding language before completing a user's task, instead of executing literal instructions or disregarding a user's underlying goals. this is particularly crucial in llm-powered training systems, where llms simulate practice scenarios and allow individuals to rehearse and refine domain-specific skills @xcite . llmbased training already facilitates interaction in domains like education @xcite @xcite , conflict resolution @xcite , and emotional support @xcite . in these settings, effective dialog agents must coordinate to build common ground when interacting with people. given the importance of generating language for conversational grounding, we ask: do current llms generate dialogue acts that reflect grounding patterns between humans? if not, what aspect of llm training exacerbates the grounding gap? we address these questions by measuring llm generations through linguistically validated grounding acts. for example, acts that clarify or acknowledge a prior utterance offer a strong signal for measuring shared understanding @xcite . building on prior work in dialogue and conversational analysis, we curate a collection of dialogue acts used to construct common ground ( §2). then, we select datasets & domains to study human-lm grounding. we focus on settings where human-human grounding is critical, and where llms have been applied: namely, emotional support, persuasion, and teaching ( §3). after curating a set of grounding acts, we build prompted few-shot classifiers to detect them ( §4). we then use llms to simulate turn-taking in our human-human dialogue datasets and compare agreement between human and gpt-generated grounding strategies ( §5). because we use the exact same conversational context as in human conversations, we can quantify the grounding gap: off-the-shelf llm generations are, on average, 77.5% less likely to contain grounding acts than humans ( §6). even in situations where llm generations do contain a grounding act, they differ from human generations-we observe poor human-lm agreement across a range of models. to isolate potential causes of the grounding gap, we explore a range of possible interventions, from ablating training iterations on instruction following data (sft and po) to designing a simple prompting mitigation ( §7). we find that sft does not improve conversational grounding, and po erodes it. across our experiments, we generally observe significant disagreement between grounding acts in human utterances and llm generations. in summary, we contribute (1) a set of linguistically informed grounding acts, tailored towards understanding grounding in contemporary llms. using grounding acts, we (2) conduct a controlled llm simulation of conversational grounding and a characterization of grounding agreement: a comparison between grounding acts in human utterances and llm generations. finally, we (3) investigate reasons for the grounding gap, exploring different models, ablating sft and po, and testing prompting interventions. foot_0 ## grounding acts grounding acts are a set of dialogue acts used to characterize conversational grounding. while grounding failures are qualitatively well-observed in current pretrained models @xcite @xcite , measuring where or how frequently these failures occur in dialogue remains unexplored. here, we outline concrete acts to measure these failures, discuss our curation process for selecting grounding acts, and describe each act in detail. grounding as an expert to benchmark conversational grounding in llms, we focus on dialogue acts from the perspective of an expert listener. users frequently come to llms with a direct but potentially underspecified task @xcite grounding act example clarification student: so i am the leader of my class. it is like the president or mayor. tscc teacher: oh, like a head girl? ↩→ gpt: that's great to hear! being the leader of your class is a big responsibility [...] follow-up seeker: my friends, who i also work with, are upset with me because i am leaving for a esconv new position. i am sad that they are upset. support: ah, that can be awkward. are you happy about the new position? ↩→ gpt: i can understand why that would make you feel down... remember, it's okay to [...]. acknowledgment persuadee: i [donate] as much as i can, which is admittedly not as much as i would like to. persuasion persuader: i know what you mean! sometimes it is hard to find the extra time or money to help those that need it. ↩→ gpt: that's wonderful to hear! [...] would you be interested in supporting save the [...]? ## data we choose to analyze conversations in three domains where grounding is critical and where llms are already used for social skill trainingeducation, emotional support, and persuasion. our datasets are task-oriented in nature, have multiple turns, and consist of two participants. since our grounding acts are curated around an expert listener, each dataset also has one expert participant. to identify grounding gaps, we can simulate utterances from the expert using an llm. we briefly outline our datasets (in english). for emotional support, we use emotional support conversations (esconv), a corpus of one-toone online conversations between a help-seeker and a help-provider, collected via crowdsourcing @xcite . to analyze grounding acts in education, we use the teacher student chatroom corpora (tscc), a collection of written conversations captured during one-to-one lessons between teachers and learners of english @xcite . finally, for persuasion, we use persuasion for good, a dataset consisting of one-to-one online conversations between a persuader and a persuadee, where the persuader solicits donations from payouts on a crowd working website @xcite . due to resource constraints, we sample 100 conversations from each dataset and truncate them to the median length (@xmath0, @xmath1. details on our selected datasets, sampling, and truncation are in appendix c. ## classifying grounding acts to analyze disparities in grounding acts between human and llm generations, we must first classify grounding acts in our datasets. in this section, we outline the construction of test/validation sets, and our prompting setup to classify grounding acts. ## analyzing grounding acts in llms given our three grounding acts, a reasonably performing gpt-4 classifier for labeling them, and a set of target datasets & metrics, we can now measure disparities between humans and lm simulations. in this section, we outline our controlled simulation process ( §5.1) and metrics to measure conversational grounding ( §5.2). ## gaps in generating grounding acts having introduced the controlled simulation process, we now report metrics on grounding acts ( §6.1) and qualitatively analyze errors ( §6.2). ## why do grounding gaps emerge? here, we explore potential mechanisms for how grounding gaps emerge. we focus on analyzing esconv, a target domain where disagreement in grounding acts is especially consequential. first, we examine grounding acts across several openai gpt variants and observe a larger grounding gap in newer models ( §7.1). to understand the roots of this trend, we evaluate open-source models. we hypothesize that current supervised-finetuning (sft) and preference optimization (po) datasets drive human-lm disagreement. to test this, we rigorously isolate the effects of sft and po training on grounding agreement ( §7.2). ## related work background in operationalizing the concept of common ground @xcite , we build on prior work in linguistics, cognitive psychology, and communication. this includes literature on conversational structure @xcite and on subdialogues @xcite . conversational grounding in nlp to benchmark grounding abilities, existing works have operationalized tasks relevant to conversational grounding, including question answering @xcite , producing human-like acknowledgments @xcite , addressing ambiguities @xcite , providing conversational feedback @xcite , addressing repair @xcite , asking follow-up @xcite and clarification questions @xcite . correctly leveraging grounding strategies is particularly consequential in tasks that require coordination in order to achieve a goal @xcite @xcite , play games @xcite , plan ahead @xcite , retrieve data @xcite , or improvise @xcite . in such tasks, the use of grounding acts has been shown to increase success and conversation quality @xcite . furthermore, the ability to establish a common ground is a key component in efforts to design believable conversational agents @xcite @xcite and facilitate human-ai collaboration in dialogue @xcite our work synthesizes existing literature by formalizing a framework to study grounding in human-ai dialogue. llms and conversational grounding despite the ubiquity and importance of conversational grounding, previous work has identified fundamental limitations in the ways dialogue agents powered by large language models establish common ground, noting that current systems usually guess what the user intended, instead of leveraging grounding acts. 5 related to this limitation, various undesirable conversational patterns have been identified, including over-informative question answering @xcite , refusal to answer ambiguous questions @xcite @xcite , miscalibration issues @xcite , and overconfidence @xcite . similarly, llm sycophancy @xcite -mirroring the views of a user-may be related to presumptive grounding. large language models' generations have thus been criticized as not being grounded in any communicative intent, any model of the world, or any model of the reader's state of mind @xcite . in our work, we carefully examine the role of contemporary instruction following and preference optimization, analyzing their effect on conversational grounding. ## conclusion instruction-following language models are trained to "follow" instructions. thus, datasets and algorithms for finetuning llms are designed around single-step interaction. in this work, we outline a set of discourse acts-grounding acts-to measure interaction with language models beyond instruction following. we apply theory from conversational grounding to interactions with llms, finding significant differences between how humans ground in dialogue and how llms generate grounding acts. designing new datasets, models, and methods, motivated by prior work on conversational grounding, will likely be necessary to minimize the grounding gap.