Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality (2022)

Abstract: Human communication relies on common ground (CG), the mutual knowledge and beliefs shared by participants, to produce coherent and interesting conversations. In this paper, we demonstrate that current response generation (RG) models produce generic and dull responses in dialogues because they act reflexively, failing to explicitly model CG, both due to the lack of CG in training data and the standard RG training procedure. We introduce Reflect, a dataset that annotates dialogues with explicit CG (materialized as inferences approximating shared knowledge and beliefs) and solicits 9k diverse human-generated responses each following one common ground. Using Reflect, we showcase the limitations of current dialogue data and RG models: less than half of the responses in current data is rated as high quality (sensible, specific, and interesting) and models trained using this data have even lower quality, while most Reflect responses are judged high quality. Next, we analyze whether CG can help models produce better quality responses by using Reflect CG to guide RG models. Surprisingly, we find that simply prompting GPT3 to “think” about CG generates 30% more quality responses, showing promising benefits to integrating CG into the RG process.

## introduction human communication is a collaborative effort @xcite @xcite where participants strive to achieve common ground (cg), consisting of mutual beliefs and common knowledge @xcite @xcite . conversational ai systems, while able to produce fluent texts, often generate generic and dull dialogue responses @xcite , potentially because they do not explicitly model 1 link to our data and code will be provided on our project page https://inklab.usc.edu/reflect/ . oh no, i spilled the food i prepared for dinner they might be feeling bad and need help cleaning it up sorry to hear that! (okay.. they don't seem so interested ) don't worry! how about let's clean it up and order from your favorite pasta place? aww, thank you! i feel much better now! makin in erences to reach common round produces more meanin ul and en a in conversations! figure 1: a motivating example. we aim to help rg models produce more human-like responses instead of generic ones. we argue that integrating common ground by making inferences is crucial. cg in communication (as illustrated in figure 1 ). specifically, existing models mostly follow a dialogue @xmath0, skipping an important middle step that builds common ground, which naturally and universally exists in human communication, i.e., dialogue @xmath1. moreover, the same history can yield numerous responses, predicated on the cg and intent of the responder. we conjecture that the omission of modeling cg explicitly is a crucial bottleneck in rg models because they are directly trained to produce responses without learning how and why those responses are uttered. modeling common ground between speakers, however, is challenging due to its implicit and subjective nature during conversations @xcite . prior work on representing cg either mines noisy commonsense knowledge triples between dialogue history and existing re-sponses @xcite or collects human inferences after reading the whole dialogue as a bystander @xcite . such approaches provide useful augmentation, but post-hoc analysis cannot mirror the generative process and intent of diverse human dialogue. figure 2 illustrates three paradigms for rg. we argue that truly modeling this generative process requires ( 1 to this end, we formalize common ground in dialogues as inferences made by one participant to approximate potential beliefs shared by other participants, as shown in figure 1 . in this work, we instantiate inferences as question-answer (qa) pairs in natural language (nl) such as "what might happen later?" "they might need to clean the floor" to elicit others' beliefs, inspired by inquiry-based dialogic learning @xcite @xcite . another critical aspect of cg is its multi-dimensional nature, i.e., given the same dialogue context, different plausible inferences can be made, which then lead to different responses. following these principles, we create a novel dialogue resource with multiple explicitly human-annotated common ground, each of which is further substantiated as a next-turn response continuing the conversations (an example of expanded cg and responses for one context shown in figure 3 ). we design a two-stage data collection process by first asking crowdsourcing workers to answer different inference questions eliciting beliefs about cg (e.g., what is the speaker feeling right now?) answers rely on common sense, and adopt the point of view of the conversational respondent. we use these qa pairs to approximate various (non-exhaustive) inference dimensions to extend the common ground (e.g., empathy and event causality). our second step converts these cg into dialogue responses by asking different workers to write a coherent response based on the answer/inference collected in the first stage. our collected data reflect contains 9k diverse responses from 600 dialogue contexts, based on 5 inference dimensions for cg. using reflect, we first test our hypothesis that explicitly modeling cg and using cg to construct responses creates more engaging conversations. we conduct human evaluation to compare the quality of responses between reflect and "reflex" style datasets and models in terms of sensibility, specificity, and interestingness. we find that, compared to reflex-prone human-written and machinegenerated dialogues, our two-stage data collection process results in more responses that are sensible, specific, and interesting as rated by humans. this highlights limitations of existing data collection procedures and models trained on the data. next, we look to study the potential of explicitly modeling cg in dialogue systems to help build models that can create more engaging conversations. as a case study, we use the inference dimensions from reflect and test two simple ways to guide rg using cg. we surprisingly find that simple approaches such as appending an inference question to the dialogue context before the response in the few-shot (fs) in-context examples (from reflect) help gpt3-175b @xcite generate almost 30% more responses that are deemed sensible, specific, and interesting than vanilla fs learning gpt3 (no inference question). we demonstrate that, when prompted to "think" about an inference question (approximated cg), large models such as gpt-3 can create more engaging conversations. we also find that such effect is only shown in large models like gpt-3 as we find blenderbot-440m @xcite benefits from fine-tuning on reflect, but appending inference questions does not further increase response quality. in summary, our contributions are as follows: 1) we operationalize theories about common ground and formalize them for dialogue; 2) we collect the first large-scale (9k responses) dialogue dataset with diverse responses guided by cg and release this resource to facilitate training and evaluation; 3) we show important limitations of existing dialogue data and rg models that detract from engaging communication; 4) we demonstrate that cg can dramatically improve rg quality even with simple prompting, boosting quality by 30%. the resources and results from this work promise to enable the research community to create and evaluate common ground-aware rg models. ## inference-based common ground we formally introduce the notion of common ground in conversations as the implicit variable conditioned on dialogue history and provides conditions to the next-turn response. ## collecting reflect data here we describe how we collect reflect, a novel large-scale dialogue dataset with diverse humanannotated inference-based cg and grounded responses. an overview of the procedure with examples are shown in figure 3 . we first select base dialogues from a dataset that is constructed without considering cg and only has one plausible response for each context (3.1). then we aim to expand and collect multiple responses based on different inference dimensions. we introduce a two-stage process to first crowdsource potential inferences people make in conversations (3.2) and then ask a second cohort of workers to generate diverse responses based on the inferences (3.3). we designed a two-stage data collection to 1) collect multiple, diverse responses based on each cg; 2) to allow response writers to validate cg as high quality, generic common sense inferences. finally, we include discussions of data quality assurance (3.4). ## limitations of reflex-prone dialogue data and models most existing open-domain dialogue datasets are either crowdsourced by workers who do not have strong incentives to create engaging conversations @xcite or crawled from language learning websites and exams @xcite 1 ). ## a little cg goes a long way after showing that explicitly integrating inferencebased cg helps humans produce more specific and interesting dialogue responses, we now test if this ## related work we have presented discussion of previous work representing cg @xcite in section 1 and relevant communication theory and psycholinguistic literature in section 2. here we provide additional discussions. recent advances on neural rg models mainly focused on fine-tuning large pre-trained transformer models @xcite @xcite on huge number of dialogue data. however, few of the data provides explicit common grounding. modular rg @xcite aims to generate relevant knowledge first by retrieving from the web and the generate knowledge-grounded responses. compared to these work, we focus on inferences based on common sense instead of external knowledge. another closely related work by @xcite examined incorporating dialogue data with techniques from improvisational theater to teach models to implicitly build common ground. ## conclusion we introduce reflect, a dataset with diverse inference-grounded responses inspired by cg and communication theories. we carefully design our two-stage collection process and apply quality control. then we demonstrate limitations of existing dialogue data and models trained on it. finally, we present promising signs that guiding models with cg results in more engaging conversations. we hope to encourage more work on improving rg quality by looking at how humans use cg and adapt the communication process to machine learning models. future directions include providing a ranking of inference dimensions depending on dialogue context and train models to generate responses following the most suitable dimension. reflect also enables potential automated metrics to evaluate response since more responses per dialogue might help gauge the plausible response space given a context.