Thoughts to Target: Enhance Planning for Target-driven Conversation (2024)

Abstract: In conversational AI, large-scale models excel in various tasks but struggle with target-driven conversation planning. Current methods, such as chain-of-thought reasoning and tree-search policy learning techniques, either neglect plan rationality or require extensive human simulation procedures. Addressing this, we propose a novel two-stage framework, named EnPL, to improve the LLMs’ capability in planning conversations towards designated targets, including (1) distilling natural language plans from target-driven conversation corpus and (2) generating new plans with demonstration-guided in-context learning. Specifically, we first propose a filter approach to distill a high-quality plan dataset, ConvPlan (Resources of this paper can be found at https://github.com/pandazzh2020/ConvPlan). With the aid of corresponding conversational data and support from relevant knowledge bases, we validate the quality and rationality of these plans. Then, these plans are leveraged to help guide LLMs to further plan for new targets. Empirical results demonstrate that our method significantly improves the planning ability of LLMs, especially in target-driven conversations. Furthermore, EnPL is demonstrated to be quite effective in collecting target-driven conversation datasets and enhancing response generation, paving the way for constructing extensive target-driven conversational models.

## introduction unlike task-oriented conversations that encompass a broader range of tasks, goal-driven conversations focus on reaching a specific goal or objective, such as recommending a target movie @xcite . the dialogue systems are required to lead the conversation to the target flexibly and coherently. due to its purpose and flexibility, target-driven dialogue agents have a broad-based demand, e.g., conversational recommendation @xcite , psychotherapy @xcite , and education @xcite . these conversations, usually characterized by defined user requirements, rely on precise planning capabilities, making it crucial to build autonomous conversational ai. in traditional target-driven conversation methods, many studies control dialogue generation through next-turn transition prediction @xcite , subgoal generation @xcite , and knowledge path reasoning @xcite . to accomplish this task, effective conversation planning is crucial @xcite , which requires reasonable actions to guide the dialogue topics to targets smoothly. different from summarizing a conversation, the process of planning requires not only capturing the key content but also ensuring logical coherence and naturalness. however, previous studies have employed greedy strategies with single-round topic prediction mechanisms that lack global planning of the conversation process @xcite . these approaches tend to be short-sighted and lead to incoherent topic cues. the generated plan is also too structured (e.g., a sequence of entities or action topic pairs) and not conducive to human understanding. this inherent rigidity prompts a shift in focus toward emergent conversational frameworks, a realm dominated by large language models (llms). recent advancements have propelled llms to the forefront of conversational ai due to their exceptional generation capabilities @xcite . however, llms fall short of proactively planning the conversation process @xcite , making it insufficient in handling target-driven conversation. this is because targetdriven conversations aim to achieve a global target that often cannot be explicitly defined as a subtask. conversation agents are required to be able to direct the conversation to the target flexibly and the process must be coherent. nevertheless, to enhance the planning and reasoning ability of llms, many researchers have investigated chain-of-thought (cot) @xcite and tree of thoughts approach (tot) @xcite , known as reasoning chains or rationales, to eventually lead to the final answer. however these works usually only apply to some welldefined tasks (such as game of 24), focusing on the evaluation of the final task and neglecting the measurement of the rationality of the plan. in addition, many works use the tree-search approach to improve planning capabilities of lms @xcite . for example, @xcite treat policy planning as a stochastic game and use prompting for every stage of an openloop tree search. however, when these methods are faced with the complexity of real-world applications, they require a lot of user simulation. in this paper, we aim to improve the constrained planning ability of llms in the task of targetdriven conversation. llms have strong comprehension and generation capability but weak planning capability @xcite . as illustrated in figure 1 , the structured plan could be difficult to understand by both human and llms. to mitigate this issue, we propose a novel twostage planning construction framework, named enhance planning framework (enpl). enpl first leverages the existing manually collected conversation dataset to distill natural language plans describing the conversation process through llms. we propose a filtering approach, which calculates the entity consistency score between the distilled plans and the conversations, to select high-quality plans for constructing a target-driven conversation plan dataset, named convplan. it consists of 12k highquality plans with targets, user settings, and plans. given a new user setting and target, the distilled plans can then serve as demonstrations for generating a new plan as thought to the target with the exceptional in-context learning capability of llms. we fully verify the rationality and intelligence of the newly generated plan and reveal that these plans can further guide conversation collection and enhance response generation, pointing out feasible directions for constructing large-scale target-driven conversation datasets and model training. our contributions are summarized as follows: (1) we propose a novel two-stage framework, named enpl, to improve the llms' capability in planning conversations towards designated targets, including distilling natural language plans from targetguided dialogue corpus and generating new plans with demonstration-guided in-context learning. @xcite we propose a filter approach to select high-quality plans distilled by llms and introduce a novel evaluation metric, named entitycov, based on entitycoverage for plan validation. (3) based on enpl, we first create a high-quality plan dataset (conv-plan) for constrained language planning. by leveraging the convplan, we validate that the generated plans play a guiding role in collecting large-scale datasets and enhancing response generation. ## evaluating step 2: generate new plan in this section, we fully verify the rationality and intelligence of the newly generated plan and reveal that generated plans can further guide the generation of target-driven conversations. ## evaluating step 3: applications we further validate the effectiveness of applying the plans generated by enpl on two applications: 1) guide conversation dataset collection, and 2) enhance response generation. guide conversation dataset collection. as shown in ## conclusion this paper introduces a novel two-stage enhanced planning framework to overcome challenges in target-driven conversation planning via llms. our method involved harnessing the generative capabilities of llm in distilling plans from existing humancurated datasets. we filter the over-generate plans and introduce comprehensive methods for plan validation. we further guide llm to generate plans according to new user scenarios and targets via incontext learning. our approach not only advances the capabilities of llms in planning target-driven conversations but also provides a scalable strategy for generating large-scale datasets. consequently, this is a significant step towards building sophisticated target-driven conversational models. future research will focus on refining the plan generation and validation process for even greater processing efficiency and accuracy.