Planning Like Human: A Dual-process Framework for Dialogue Planning (2024)

Abstract: In proactive dialogue, the challenge lies not just in generating responses but in steering conversations toward predetermined goals, a task where Large Language Models (LLMs) typically struggle due to their reactive nature. Traditional approaches to enhance dialogue planning in LLMs, ranging from elaborate prompt engineering to the integration of policy networks, either face efficiency issues or deliver suboptimal performance. Inspired by the dual-process theory in psychology, which identifies two distinct modes of thinking—intuitive (fast) and analytical (slow), we propose the Dual-Process Dialogue Planning (DPDP) framework. DPDP embodies this theory through two complementary planning systems: an instinctive policy model for familiar contexts and a deliberative Monte Carlo Tree Search (MCTS) mechanism for complex, novel scenarios. This dual strategy is further coupled with a novel two-stage training regimen: offline Reinforcement Learning for robust initial policy model formation followed by MCTS-enhanced on-the-fly learning, which ensures a dynamic balance between efficiency and strategic depth. Our empirical evaluations across diverse dialogue tasks affirm DPDP’s superiority in achieving both high-quality dialogues and operational efficiency, outpacing existing methods.

## introduction large language models (llms), such as those described by @xcite , have revolutionized the field of natural language processing by demonstrating an unprecedented ability to understand context and generate coherent responses across a wide range of dialogue scenarios @xcite @xcite . despite these advances, llms predominantly operate in a reactive mode, often ## conclusion and future work in this work, we introduced a dialogue planning framework that leverages the dual-process theory of human cognition, strategically alternating between quick, intuitive responses and detailed, analytical planning to emulate human-like conversational dynamics. to bolster this framework's capabilities, we implemented a two-stage training strategy, merging offline reinforcement learning for foundational training with advanced mcts-guided self-play for refinement. the resulting empiri-cal evidence demonstrated our dual-process framework's superior performance against leading methods, showcasing significant advancements in dialogue planning. moving forward, our research will aim to refine the switching mechanism between planning modes and further optimize mcts use, reducing computational expenses and enhancing dialogue planning towards achieving more nuanced, human-like interactions.