Dialogue Planning via Brownian Bridge Stochastic Process for Goal-directed Proactive Dialogue (2023)

Abstract: Goal-directed dialogue systems aim to proactively reach a pre-determined target through multi-turn conversations. The key to achieving this task lies in planning dialogue paths that smoothly and coherently direct conversations towards the target. However, this is a challenging and under-explored task. In this work, we propose a coherent dialogue planning approach that uses a stochastic process to model the temporal dynamics of dialogue paths. We define a latent space that captures the coherence of goal-directed behavior using a Brownian bridge process, which allows us to incorporate user feedback flexibly in dialogue planning. Based on the derived latent trajectories, we generate dialogue paths explicitly using pre-trained language models. We finally employ these paths as natural language prompts to guide dialogue generation. Our experiments show that our approach generates more coherent utterances and achieves the goal with a higher success rate.

## introduction dialogue systems have made significant progress in generating high-quality responses for open-domain chitchat @xcite and assisting users in completing specific tasks @xcite . instead of passively responding to users, dialogue systems can also take a proactive role to direct a conversation towards specific goals, such as introducing new and interesting topics @xcite or providing sociable recommendations on target items @xcite . such a proactive target-oriented or goal-directed dialogue system can guide conversations towards topics that the system knows how to discuss, making it promising to build autonomous conversational ai. figure 1 : an example from the repurposed durecdial 2.0 @xcite dataset. given a pre-determined target and current dialogue context, we expect to plan a dialogue path to direct the conversation. for goal-directed dialogue systems, the objective is to proactively direct conversations towards a designated target. previous work has primarily predetermined the targets as specific keywords @xcite , topics @xcite , and dialogue action-topic pairs @xcite . to achieve this task, effective dialogue planning is essential, which requires taking reasonable actions and smoothly directing dialogue topics to the designated one. more importantly, the whole process is expected to be coherent and natural. prior studies attempted to tackle this challenge through next-turn transition prediction @xcite , sub-goal generation @xcite , and knowledge path reasoning @xcite to control dialogue generation. however, there are still open issues worth exploring. first, previous studies adopted a greedy strategy with a single-turn topic prediction mechanism, which lacks global planning for the dialogue process @xcite . consequently, these methods are often short-sighted, resulting in sub-coherent topic threads. second, recognizing a user's engagement and willingness to follow the system is crucial for achieving coherent transitions. however, current studies often overlook the importance of modeling such user feedback. therefore, it is necessary to explore globally planned dialogue strategies while incorporating user feedback to improve the coherence of goal-directed dialogue systems. in this work, our objective is to globally plan dialogue paths that connect the current context to the target at each turn. as illustrated in figure 1 , this dialogue path should strike a balance between coherence with the ongoing dialogue context and smooth transitions towards the target. assuming that path trajectories without a target can be represented as brownian motion @xcite in latent space, we expect the embeddings of neighboring trajectory points to be similar to each other, while those of distant trajectory points to be dissimilar. drawing inspiration from @xcite , we view goal-directed dialogue behavior as a brownian bridge @xcite stochastic process conditioned on fixed start and end points. as such, we can derive latent trajectories that follow coherent temporal dynamics. based on the above intuition, we propose a coherent dialogue planning approach via brownian bridge (color) stochastic process. it involves mapping dialogue path points, such as topics or action-topic pairs, into a latent space of brownian bridge conditioned on the current context and designated target. to ensure goal-directed behavior and incorporate user feedback, we also map the latest user utterance into real-time user feedback representation using the same latent space. we leverage this feedback to perturb the density and uncertainty of the brownian bridge, simulating its impact on the dialogue planning process. our training process uses a contrastive objective, which helps retain global coherence. we then fine-tune pre-trained language models (plms) using the derived latent trajectories to plan dialogue paths explicitly. these paths provide step-by-step explanations for reaching the target and serve as natural language prompts for generating system utterances. in summary, our main contributions are: (1) we propose a novel approach called color, which effectively models global coherence and incorporates user feedback in goal-directed dialogue planning. our method utilizes the brownian bridge stochastic process, and to the best of our knowledge, this is the first work to apply this method to the goal-directed proactive dialogue task. (2) we repurpose existing dialogue datasets by automatically constructing system goals and splitting them into in-and out-ofdomain test sets. it facilitates research in the field and allows for more accurate evaluation of models. (3) extensive experiments demonstrate that our proposed approach outperforms other methods, both in automatic and human evaluations. ## preliminaries problem formulation we consider a corpus of goal-directed dialogues @xmath0, p i , c i )} n @xmath1, where n is the total number of dialogues. the domain knowledge facts relevant to the i-th dialogue are represented as given a target t consisting of an action-topic pair or a topic only, a dialogue context c, and a set of relevant domain knowledge k (and a user profile u, if any), our objective is to generate coherent utterances to reach the target t when appropriate. the problem can be decomposed into two sub-tasks: (1) dialogue planning, which involves planning suitable actions and topics to lead the dialogue proactively with coherent transitions to the target, and (2) dialogue generation, which involves generating an appropriate utterance to achieve the planned action and topic at each turn. ## method we propose a coherent dialogue planning approach via brownian bridge (color) stochastic process to steer goal-directed dialogue generation. the intuition behind color is to learn a mapping (see §3.1) in the brownian bridge latent space that captures coherent temporal dynamics for planning dialogue paths. each dialogue path consists of a sequence of topics or action-topic pairs, starting from the current context and leading to the target. we generate these paths explicitly (see §3.2) based on representations derived from the latent space, and use them to guide the generation of dialogue utterances (see §3.3). ## conclusion in this work, we explore the task of goal-directed proactive dialogue and focus on planning dialogue paths that direct conversations towards the designated target. we propose a novel approach called color, which models coherent temporal dynamics for dialogue paths in the defined latent space, and considers the impact of user feedback on the dialogue planning process. we employ the planned dialogue paths as prompts to steer dialogue generation. experiments show that our proposed method outperforms other methods significantly.