Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents (2023)

Abstract: Human-like chatbots necessitate the use of commonsense reasoning in order to effectively comprehend and respond to implicit information present within conversations. Achieving such coherence and informativeness in responses, however, is a non-trivial task. Even for large language models (LLMs), the task of identifying and aggregating key evidence within a single hop presents a substantial challenge. This complexity arises because such evidence is scattered across multiple turns in a conversation, thus necessitating integration over multiple hops. Hence, our focus is to facilitate such multi-hop reasoning over a dialogue context, namely dialogue chain-of-thought (CoT) reasoning. To this end, we propose a knowledge distillation framework that leverages LLMs as unreliable teachers and selectively distills consistent and helpful rationales via alignment filters. We further present DOCTOR, a DialOgue Chain-of-ThOught Reasoner that provides reliable CoT rationales for response generation. We conduct extensive experiments to show that enhancing dialogue agents with high-quality rationales from DOCTOR significantly improves the quality of their responses.

## introduction commonsense reasoning is crucial in human conversation @xcite . however, most conversational agents still lack commonsense reasoning, limiting their capability to engage in rich conversations with users @xcite . recent studies @xcite aim to tackle this issue by generating commonsense knowledge @xcite relevant to the dialogue context, but i spent the morning with a friend and then went shopping after lunch. i could't find my checkbook to make a purchase and realized it was in my handbag, which was in my car. ## dialogue chain-of-thought distillation in this section, we propose a robust knowledge distillation framework that extracts plausible cot rationales from an unreliable llm ( §3.1) and selectively distills high-quality rationales via alignment filters ( §3.2) to (re-)train a reliable cot reasoner. figure 2 presents an overview of our framework. ## donut alongside doctor, we present its training corpus, donut, a dialogue chain-of-thought dataset with annotated rationales for dialogue cot reasoning foot_6 . we choose three human-collected dialogue datasets, dailydialog @xcite , dream @xcite , and mutual @xcite , to sample source dialogues for annotation foot_7 . we also include 5% of the dialogues in soda @xcite , a million-scale social dialogue dataset, for scalability. in total, we obtain 10k dialogues for annotation. for each utterance in a dialogue (except for the first one), we instruct chatgpt to generate 10 rationale candidates. using the two alignment filters from §3.2, we filter out 122,319 candidates (24.98%) that are either inconsistent with the dialogue context or not helpful in predicting the next response. the resulting dataset, donut, consists of 10k dialogues with 367k cot rationales. donut. further analyses on the alignment filtering and generated rationales are in appendix d. ## experiments our work builds upon previous efforts @xcite @xcite to enrich dialogue models by injecting external commonsense knowledge. hence, we conduct extensive experiments on response generation to examine how dialogue cot reasoning from doctor provides commonsense knowledge to assist dialogue agents in generating high-quality responses. ## conclusion commonsense reasoning in conversations involves multi-hop reasoning, which poses challenges even for llms. to address this, we present a dialogue chain-of-thought distillation framework that selectively annotates high-quality rationales using llms. our contributions are as follows: (1) with our framework, we collect donut, a large-scale dataset for dialogue cot reasoning. (2) we present doctor, a dialogue chain-of-thought reasoner trained on donut. (3) through extensive experiments, we show the efficacy of doctor, especially in the human evaluation, where 67% of the responses generated using doctor are preferred over the responses using knowledge from llms.