CGIM : A Cycle Guided Interactive Learning Model for Consistency Identification in Task-oriented Dialogue (2022)

Abstract: Consistency identification in task-oriented dialog (CI-ToD) usually consists of three subtasks, aiming to identify inconsistency between current system response and current user response, dialog history and the corresponding knowledge base. This work aims to solve CI-ToD task by introducing an explicit interaction paradigm, Cycle Guided Interactive learning Model (CGIM), which achieves to make information exchange explicitly from all the three tasks. Specifically, CGIM relies on two core insights, referred to as guided multi-head attention module and cycle interactive mechanism, that collaborate from each other. On the one hand, each two tasks are linked with the guided multi-head attention module, aiming to explicitly model the interaction across two related tasks. On the other hand, we further introduce cycle interactive mechanism that focuses on facilitating model to exchange information among the three correlated sub-tasks via a cycle interaction manner. Experimental results on CI-ToD benchmark show that our model achieves the state-of-the-art performance, pushing the overall score to 56.3% (5.0% point absolute improvement). In addition, we find that CGIM is robust to the initial task flow order.

## introduction consistency identification task in dialogue has the potential benefits of preventing inconsistent response generation @xcite , which has attracted increasing attention. recent years have witnessed two promising research directions in consistency identification. the first focuses on consistency identification in open-domain dialogue @xcite . the second direction consider consistency identification in task-oriented dialogue system (ci-tod) @xcite . in this work, we focus on the latter. recently, @xcite intro- * email corresponding. duces a benchmark (ci-tod) for consistency identification in task-oriented dialogue to facilitate the relevant research. ci-tod introduces three subtasks including: (1) dialogue history consistency identification task (hcit) to judge whether generated response is inconsistent with dialogue history; while the idea seems promising, achieving this objective is challenging, since we need to jointly model the three sub-tasks simultaneously rather than simple two tasks setting. recent work have shown explicit joint modeling is superior to the implicit joint modeling @xcite . nevertheless, their work still limits to modeling the relationship between two tasks, it remains clear if the explicit modeling paradigm can be applied in three tasks. to this end, as shown in figure 1(b) , we propose a novel cycle guided interactive learning model (cgim) for ci-tod, which achieves to perform the three sub-tasks jointly and interactively in an explicit interaction paradigm. specifically, cgim first consists of a guided multi-head attention module (gma) that is used for two related tasks, which aims to explicitly utilize information from another task. with the help of gma, each task can not only rely on its own task information but also performed with the guidance of the corresponding correlated task explicitly. furthermore, since gma can only enable the single information flow from one task to another task, we further propose a novel cycle interactive mechanism to facilitate information flow across the three tasks in an cycle interaction fashion. with the use of cycle interactive mechanism, cgim can be stacked to form a hierarchy, which can gradually capture interaction information and better transfer knowledge. we conduct experiments on ci-tod benchmark and results show that cgim achieves the best performance, outperforming previous state-of-the-art methods by at least 5.0% (overall accuracy). besides, extensive analysis further demonstrate the superior and robustness of our approach. main contributions are summarized as follows: all codes in this work will be publicly available at https://github.com/lightchen233/cgim . ## background to make the paper self-complete, we present the definition of the task that follows @xcite in this section. ## approach the architecture of the cycle guided interactive learning model (cgim) is depicted in figure 3 . it mainly consists of three components: three taskspecific encoders for obtaining encoding representation for each task ( §3.1); a cycle interactive learning module for explicitly establishing the interaction across the three tasks ( §3.2); three separate decoders for hcit, qcit and kbcit ( §3.3), respectively. in the following sections, the details of our framework are given. ## related work increasing attention has been witnessed in consistency identification in dialogue. to this end, per-sonachat @xcite and personaldialog @xcite are introduced to implicitly consider the consistency in dialogue generation. welleck et al. (2019) model the consistency of dialogue systems by introducing a new natural language inference dataset called dialoguenli. dziri et al. (2019) propose to use state-of-the-art entailment techniques for evaluating the coherence of dialogue systems. nie et al. (2021) propose a di-alogue contradiction detection task (decode) to evaluate the ability to detect contradictory in dialogue. however, their work mainly focuses on consistency in open-domain direction. in contrast, our framework mainly considers improving consistency in task-oriented dialogues. in recent years, @xcite make the first step towards consistency identification in taskoriented dialogues and propose three sub-tasks to detect whether the system response is contradicted with the corresponding dialogue history, user query, and knowledge base. in addition, they also introduce a public benchmark ci-tod and provide some state-of-the-art pre-trained models to facilitate the research. unfortunately, their models only jointly consider the correlated three tasks in an implicit manner. compared with their model, we propose a cycle guided interactive learning model (cgim), which can explicitly model interaction across the three tasks in a cycled interaction manner. to our knowledge, we are the first to explore an explicit interaction paradigm for ci-tod. ## conclusion we studied how to explicitly model the interaction across three sub-tasks for consistency identification in task-oriented dialogue (ci-tod). to this end, we introduced a cycle interactive learning model (cgim), which facilitates the knowledge transfer across the three correlated tasks. experiments show cgim achieves state-of-the-art performance. in addition, cgim is robust to the initial task flow order and works better in a low-resource setting, which is scalable in a real-world system deployment.