Towards a Progression-Aware Autonomous Dialogue Agent (2022)

Abstract: Recent advances in large-scale language modeling and generation have enabled the creation of dialogue agents that exhibit human-like responses in a wide range of conversational scenarios spanning a diverse set of tasks, from general chit-chat to focused goal-oriented discourse. While these agents excel at generating high-quality responses that are relevant to prior context, they suffer from a lack of awareness of the overall direction in which the conversation is headed, and the likelihood of task success inherent therein. Thus, we propose a framework in which dialogue agents can evaluate the progression of a conversation toward or away from desired outcomes, and use this signal to inform planning for subsequent responses. Our framework is composed of three key elements: (1) the notion of a “global” dialogue state (GDS) space, (2) a task-specific progression function (PF) computed in terms of a conversation’s trajectory through this space, and (3) a planning mechanism based on dialogue rollouts by which an agent may use progression signals to select its next response.

## introduction all human conversation serves some purpose. these may range from negotiating an agreement to explaining a topic to maintaining a social relationship. people are generally capable of forming an assessment, sometimes subconsciously, whether a conversation is going well or not and adjusting their behavior accordingly. such assessment, which underlies most human conversation, is essential in continuous awareness of the direction where the interaction is heading and whether the parties are in sync or not, e.g., @xcite . in a task-oriented interaction, the participants assess if progress towards a successful outcome is being made. in a negotiation, parties assess if an agreement is likely. even in a casual conversation, people intuitively sense when to continue, when to change the subject, or when to stop. based on such figure 1 : our framework applied to the charity solicitation task in persuasion for good @xcite . given the dialogue history (center left), the system uses rollouts @xcite to simulate the outcome of two response candidates (bottom, in red). each rollout is mapped as a path through the global dialogue state space (center right) where it can be compared with similar outcomes. the candidates are finally ranked using the progression function (top), and the best is selected. as such, we approach dialogue state tracking at a higher level, focusing instead on what we call the global dialogue state (gds). given a conversational task (e.g., negotiation), the global state of a dialogue reflects the most likely outcome (e.g., a strong agreement or a stalemate) given the history of the dialogue up to the current turn. in contrast to traditional dst, the global state remains invariant to the specific details discussed at each turn (e.g., names, dates, quantities) that are typically the concern of slot-filling models. rather, global dialogue states are influenced by the contexts in which these details occur (e.g., "i would love to donate @xmath05 to this charity"). thus, the global state of a dialogue can be measured in terms of its semantic similarity to other groups of dialogues for the same task, which can be naturally formulated as a cluster-assignment problem in the dialogue embedding space. that is, a dialogue which is assigned at the current turn to a cluster of highly successful outcomes may assume a high likelihood of success, and likewise a dialogue assigned to a cluster of unsuccessful outcomes may assume a low likelihood of success. it follows from this that the path of a dialogue through global state space can be used to derive a progression function (pf) to give turn-level estimates of task success, which can in turn be used by a dialogue agent to inform its next response. the remainder of this paper is organized as follows: in section 2 we review relevant literature pertaining to dialogue state tracking and response planning; in section 3 we formally define the global dialogue state and progression function, propose supervised and unsupervised approaches for modeling them, and describe how they can be used to assess and select dialogue response candidates; in section 4 we experimentally apply our framework to the charity solicitation task in the persuasion for good dataset @xcite , reporting results from automatic and manual evaluations; and in sections 5 and 6 we conclude with a discussion of limitations and future directions. code for our methods and experiments has been released, foot_0 and a listing of software packages we use can be found in appendix a. ## related work our work lies at the intersection of dialogue state tracking and response planning. as previously noted, we approach dialogue state at a much higher level than is typically seen in the dst literature. our concept of global dialogue state is not mutually exclusive with traditional dst approaches, which we refer to from here on as local dst. rather, an effective dialogue system might integrate local and global dst approaches to enable simultaneous tracking of user intents and slot-value pairs (needed for interfacing with external resources) and the overall likelihood of conversational success. ## methods the goal of our system is to construct a global dialogue state space for a task-specific dataset and learn a progression function to estimate how well an ongoing dialogue is progressing toward the desired outcome of the task. the quantity output by the progression function is an estimate of a dialogue-level attribute which indicates task success (e.g. satisfaction in a customer service task). in many task domains, the success of a conversation cannot be completely measured by a single attribute. for example, in the charity solicitation task we use in our experiments, donation amount is the primary success attribute. here, there are cases where the conversation appears to go very well, but ultimately no donation is made for unexpected reasons such as the solicitee not being able to afford to donate. one could reasonably expect such an outcome to be "acceptable" in the context of a solicitation task since the solicitee has engaged with the solicitor and displayed interest, and we cannot reasonably expect the solicitor to force a donation out of someone who cannot afford it. thus we introduce the "acceptability score", a synthetic attribute that measures success by considering multiple factors (e.g., donation amount and sentiment). for any dialogue dataset, the acceptability score combines multiple dialogue-level attributes in a way sensitive to their covariance with the primary success attribute: to learn the progression function, dialogue-level attribute annotations must exist for use in this purpose. however, in many settings such annotations are not available in sufficient quantity to directly learn a progression model with sufficient generalization. consequently, we propose supervised and unsupervised approaches for learning the global state and progression models. ## limitations & future direction we recognize several limitations of our study which warrant follow-up investigation. this study focuses on a single task and dataset, and thus is subject to the assumptions and biases therein. since we intend our framework to be general, it is prudent to perform additional studies to verify the efficacy of our methods on a variety of datasets spanning multiple dialogue domains and tasks. also, although we provide automatic evaluation of the ability of rollouts to improve performance on a solicitation task, we cannot assume that humans would respond in the same way as dialogpt. thus, human evaluation is needed to further validate this approach. ## conclusion in this work we introduced the concept of global dialogue state and proposed a framework with which a dialogue agent can gain awareness of where an ongoing conversation is headed, the likelihood of a successful outcome, and how its own response decisions impact the overall direction of the dialogue. we demonstrated that an unsupervised approach to modeling the gds space and progression function is feasible, which is useful in sparsely-labeled settings. however, we showed that with domainadaptation pre-training for dialogue, supervised methods are preferable when labels are available. finally, we demonstrated how using the pf as a feedback mechanism via dialogue rollouts allows an agent to improve outcomes on a solicitation task.