Continual Dialogue State Tracking via Reason-of-Select Distillation (2024)

Abstract: An ideal dialogue system requires continuous skill acquisition and adaptation to new tasks while retaining prior knowledge. Dialogue State Tracking (DST), vital in these systems, often involves learning new services, confronting catastrophic forgetting and a critical capability loss termed the “Value Selection Quandary”. To address these challenges, we introduce the Reason-of-Select (RoS) distillation method by enhancing smaller models with a novel “meta-reasoning” capability. Meta-reasoning, employing an enhanced multi-domain perspective, combines fragments of meta-knowledge from domain-specific dialogues during continual learning, transcending traditional single-perspective reasoning. This domain bootstrapping process enhances the model’s ability to dissect intricate dialogues from multiple possible values, and its domain-agnostic property aligns data distribution across different domains, effectively mitigating forgetting. Besides, two novel improvements, “multi-value resolution” strategy and Semantic Contrastive Reasoning Selection method, significantly enhance RoS by generating DST-specific selection chains and mitigating hallucinations in teachers’ reasoning, ensuring effective and reliable knowledge transfer. Extensive experiments validate the exceptional performance and robust generalization capabilities of our method.

## introduction practical dialogue systems require continual adaptation to new services while maintaining previous task capabilities. however, previous research in dialogue systems has focused on domain-specific offline systems, lacking adaptation abilities @xcite . with recent advances in large language models (llms), llm-based systems have hotel dst continual dialogue state tracking restaurant dst services dst taxi dst dialogues states: (@xmath0: services -@xmath1:45 am turn 3 what time do you prefer? can you make the appointment for the 5th of march instead? no, that does not work for me. as a specific task of cl, continual dst grapples with catastrophic forgetting @xcite , where sequential learning of new tasks hinders retention of prior ones. this problem is more acute in dst due to significant data distribution shifts across domains in the cl process, exemplified by transitions from hotel to restaurant domains (see figure 1 ). recent advances in continual dst, including memory replay and regularization @xcite , and refor-mulation as a question-answering task @xcite , strive to address forgetting. however, these methods face challenges like reliance on past data and high computational demands during testing, which hinder real-time applications. recent llms show impressive dst performance @xcite , but practical deployment faces hurdles like offline computational load and online data privacy concerns. moreover, enabling llms for cl demands significant resources, leading to the exploration of smaller models. after a comprehensive analysis of the current smaller dst models, we have identified a critical loss of capability brought by domain shifts in cl that models fail to output correct value when facing a set of similar candidates (named "value selection quandary" as elaborated in section 2). for example, in figure 1 , when tracking @xmath2, the model incorrectly chooses the most recent time mentioned, revealing its inability to grasp contextual subtleties and favoring direct value extraction over logical reasoning. this phenomenon is because, as the cl progresses, the model needs to understand the relevant knowledge of newly emerging domains and chooses to forget domain-irrelevant or weakly relevant knowledge, such as value selection. this paper addresses these challenges by boosting smaller models' reasoning ability, termed as meta-reasoning. meta-reasoning, guided by the insight that domain-specific dialogues represent only a fragment of the underlying meta-knowledge, can be viewed as a form of multi-view augmentation from different domains in the cl process. this domain bootstrapping strategy facilitates the transfer and broadening of meta-knowledge, transcending traditional single-perspective reasoning. meta-reasoning enriches smaller models' reasoning abilities to dissect and interpret intricate dialogue scenarios, addressing the "value selection quandary" and broadening their cognitive horizons. moreover, this domain-agnostic feature aligns data distribution shifts across different domains, effectively reducing catastrophic forgetting. inspired by the powerful reasoning abilities of llms like llama-2-70b @xcite and chatgpt foot_0 , we introduce reason-of-select (ros) distillation approach, designed to graft these advanced reasoning capabilities onto smaller models. we present a "multi-value resolution" strategy tailored to dst, prompting teacher llms to generate a "selection chain" that discerns and elaborates the best value choice from various options. then, we distill this rationale into a smaller student model to boost its meta-reasoning capabilities, making it easier to deploy to terminal systems. moreover, to enhance faithful reasoning and reduce hallucinations in the teacher model, we integrate a schema-guided prompt, strengthening source-target correlation and semantic coherence. our innovative semantic contrastive reasoning selection method uses semantic similarity to select the most accurate teacher-generated rationale, suitable even in black-box scenarios. extensive experiments with various teacher and student model sizes demonstrate our method's exceptional continual dst performance and robust cross-dataset generalization capabilities. to summarize, our main contributions are: • we propose a reason-of-select (ros) distillation framework for continual dst. through enhancing domain-agnostic meta-reasoning capabilities, ambiguous value selection and catastrophic forgetting caused by data distribution shifts across domains can be effectively mitigated. • to alleviate the hallucination outputted by a teacher model in our annotation-free reasoning framework, we present a semantic contrastive reasoning selection strategy to obtain trustworthy rationale from multiple candidates, ensuring faithful reasoning transfer. • comprehensive experiments with two teacher models and four student models of varying sizes demonstrate superior performance and robust generalization of our proposed method. ## motivation for boosting meta-reasoning capability in continual dst through an initial experiment with the same setups from prior work @xcite (details in section 4), we employed two distinct backbone models, t5-small and llama-7b, and observed a performance decline in continual dst with turn ids increasing (figure 2 (a) ). ## reason-of-select distillation problem formulation in continual dst, we train a model f : x × @xmath3, ..., t k . this model predicts the target y based on input x and task t @xmath4. within a specific task t k , a dialogue with t turns of interactions between the system and the user can be represented as x @xmath5, u 1 ) , (a 2 , u 2 ) . . . , (a t , u t )}, where a represents the system response and u represents the user input. a predefined slot set foot_1 @xmath6, . . . , s j } is provided, where j is the total number of slots for task t k . the objective of dst is to predict the dialogue state b t based on the dialogue context x t . the dialogue state, b t , is represented as a set of (slot, value) pairs, denoted as ## conclusion this study introduces the reason-of-select (ros) distillation method to address the continual dst task's catastrophic forgetting and "value selection quandary" challenges. enhancing smaller models with a novel domain-agnostic 'meta-reasoning' capability effectively broadens their reasoning horizon and aligns data distribution shifts across different domains. two innovative methods, the "multi-value resolution" strategy and semantic contrastive reasoning selection, further strengthen ros by ensuring reliable knowledge transfer. extensive experiments demonstrate our method's superior performance and robust generalization.