Enhancing Goal-oriented Proactive Dialogue Systems via Dynamic Multi-dimensional Consistency Optimization (2025)

Abstract: Previous work on goal-oriented proactive dialogue systems frequently failed to address the multi-dimensional consistency issue between generated responses and key contextual elements (e.g., user profile, dialogue history, domain knowledge, and subgoal). To address this issue, we propose a novel Dynamic Multi-dimensional Consistency Reinforcement Learning (DMCRL) framework, which adaptively measures the impact of each consistency dimension on overall dialogue quality and provides targeted feedback to improve response quality. Experimental results on two datasets demonstrate that our DMCRL significantly improves the consistency of generated responses.

## introduction goal-oriented proactive dialogue systems (gpds) are designed to guide user interactions toward specific objectives by dynamically generating contextually relevant responses @xcite . unlike traditional dialogue systems that passively respond to user queries @xcite , gpds actively plan and steer conversations along predefined paths, ensuring smooth and coherent progression. they are widely used in applications such as personalized recommendations @xcite @xcite , customer support @xcite , and medical consultations @xcite , where it is crucial to achieve specific results interactively. existing work on gpds has primarily focused on optimizing goal-oriented dialogue planning @xcite @xcite , such as generating optimal paths to achieve specific goals. these paths are typically composed @xmath0, @xmath1. as illustrated in figure 1 , a dialogue goal pathway might start @xmath2, progress through a series of subgoals, * corresponding author and ultimately @xmath3. however, the above methods often overlook the critical issue of multi-dimensional consistency between generated responses and key contextual elements (e.g., user profile, dialogue history, domain knowledge, and subgoal). figure 1 illustrates an example of a gpds interaction, where the system needs to generate responses that are consistent with various predefined contextual elements; otherwise, the dialogue may become disjointed and ineffective. for example, in response s 1 (systemresponse), the system fails to incorporate the user profile, resulting in a lack of personalization. in s 3 , the system deviates from the target movie crossing hennessy in its recommendation. in s 5 , the system ignores the user's previously expressed intent in the dialogue history of not wanting to watch a movie and continues to recommend one. moreover, in s 7 , the system incorrectly applies a review of the song a chinese ghost story to housten love, resulting in factual inaccuracy. these inconsistencies disrupt the coherence of the dialogue, hinder the system's ability to guide the user toward the intended goal, and ultimately degrade the user experience. although some studies on another task, personabased dialogue generation @xcite , have attempted to understand and improve consistency, they typically focus on singledimensional consistency or assume that all consistency dimensions hold equal importance throughout the dialogue. they overlook the fact that the significance of different consistency dimensions varies dynamically as the conversation progresses. as a result, they fail to effectively address the challenge of balancing multi-dimensional consistency. figure 1 illustrates an interaction case where all consistency dimensions are treated as equally important (w/static). in this case, u 1 and u 2 excessively apply the user profile, leading to an abrupt mention of the user's location and repetitive ref-since he sings so beautifully, why not listen to his song 'a chinese ghost story'? leslie cheung's rich and magnetic baritone fully brings out the historical depth of this piece. that's great! he won the hong kong film awards for best actor. you should check out his movies. then why not watch a movie to pass the time? there are plenty of great films to choose from, depending on your mood. ## task definition given a dialogue @xmath4, h, k, g, r}, where u , h, k, g, and r represent the user profile, dialogue history, domain knowledge, goal-oriented path, and set of responses at each turn, respectively, gpds consists of two primary subtasks: goal-oriented path planning and response generation. goal-oriented path planning this task aims to predict a sequence of subgoals @xmath5, g 2 , . . . , g t } that guide the conversation toward achieving the final objective. at each turn t (1 ≤ @xmath6, the model determines the next subgoal g t based on the current dialogue state: response generation given the user profile u , domain knowledge k, dialogue history @xmath7, and current subgoal g t , the response generation model at the t-th turn aims to produce a coherent and goalaligned response r t , forming part of the overall response set @xmath8, r 2 , . . . , r t }, as follows: ## dmcrl framework to address the challenge of multi-dimensional consistency in gpds, we propose the novel dynamic multi-dimensional consistency reinforcement learning (dmcrl) framework. dmcrl dynamically balances the consistency requirements across four key dimensions: user profile, dialogue history, domain knowledge, and subgoal. as illustrated in figure 2 , the framework consists of two main stages: supervised fine-tuning (sft), and dynamic consistency reinforcement learning (dcrl) with an innovative reward function that incorporates counterfactual consistency rewards. ## conclusion in this paper, we propose a dynamic multidimensional consistency reinforcement learning framework dmcrl. by introducing dynamic weight allocation and counterfactual consistency verification mechanisms, dmcrl effectively improves and balances consistency across various dimensions. extensive experiments on durecdial and durecdial 2.0 demonstrate that dmcrl significantly enhances response quality, factual accuracy, and goal success rate.