Enhancing Proactive Dialogue Systems Through Self-Learning of Reasoning and Action-Planning (2025)

Abstract: A proactive dialogue system refers to a conversational system designed to guide the direction of a conversation in order to achieve pre-defined targets or fulfill specific goals. Recent studies have shown that Proactive Chain-of-Thought, which guides the system to explicitly think through intermediate reasoning and action-planning steps toward a conversational goal before generating a response, can significantly enhance the performance of proactive dialogue systems. However, these improvements primarily focus on prompt-based control, while the potential of fine-tuning Proactive-CoT remains largely unexplored. Furthermore, fine-tuning Proactive-CoT requires manual annotation of reasoning processes and action plans, which incurs significant time and cost. In this study, we propose a novel approach for automatically annotating reasoning processes and action plans through self-learning. This method enables fully automated annotation, significantly reducing the time and cost associated with manual annotation. Experimental results show that models trained using our proposed method outperform those trained with other fine-tuning approaches. These findings highlight the potential of self-learning approaches to advance the development of more robust and efficient proactive dialogue systems.

## introduction in recent years, dialogue agent proactivity has gained attention @xcite . proactive systems not only respond proactively but also guide interactions with a clear goal, improving user engagement and handling complex tasks such as negotiation. accordingly, recent work has explored llmbased prompting methods to clarify ambiguous queries and strategically persuade users in noncooperative task-oriented dialogues @xcite . in particular, "proactive chain-of-thought (procot)" extends conventional cot by incorporating action plans and reasoning processes to proactively achieve conversation goals @xcite . however, most studies rely on prompt design, leaving fine-tuning largely unexplored, and manual annotation of reasoning and action plans can be costly. an automatic approach to annotate reasoning and action plans from dialogue content could address these challenges and enable more effective fine-tuning. this paper proposes a self-contained framework for automatically annotating reasoning processes and action plans, then fine-tuning on the augmented data. as illustrated in figure 1 , the framework has three steps: ## method this study proposes a framework consisting of three steps, as illustrated in figure 1 . below is an explanation of each step. prompts for labeling dialogue act, negotiation, and reasoning can be found in appendix b. ## experiment in this section, we evaluate the effectiveness of the proposed method using a dataset focused on buyand-sell negotiations-an example of proactive dialogue. we employ gpt-4o-mini-2024-07-18 as the base llm and carry out annotation, fine-tuning, and inference through its api. ## conclusion this study proposes a self-contained framework for fine-tuning procot. automatic evaluations demonstrate that fine-tuning procot achieves accurate predictions of dialogue acts and negotiation strategies. additionally, human evaluations suggest the potential usefulness of procot, as it outperformed other models in some evaluation criteria. our results suggest that this labeling method can improve existing dialogue systems by automatically expanding and annotating training data. as a future work, we will carry out the experiments on the diverse datasets to validate our proposed method.