FCM : A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning (2021)

Abstract: Despite the success of neural dialogue systems in achieving high performance on the leader-board, they cannot meet users’ requirements in practice, due to their poor reasoning skills. The underlying reason is that most neural dialogue models only capture the syntactic and semantic information, but fail to model the logical consistency between the dialogue history and the generated response. Recently, a new multi-turn dialogue reasoning task has been proposed, to facilitate dialogue reasoning research. However, this task is challenging, because there are only slight differences between the illogical response and the dialogue history. How to effectively solve this challenge is still worth exploring. This paper proposes a Fine-grained Comparison Model (FCM) to tackle this problem. Inspired by human’s behavior in reading comprehension, a comparison mechanism is proposed to focus on the fine-grained differences in the representation of each response candidate. Specifically, each candidate representation is compared with the whole history to obtain a history consistency representation. Furthermore, the consistency signals between each candidate and the speaker’s own history are considered to drive a model prefer a candidate that is logically consistent with the speaker’s history logic. Finally, the above consistency representations are employed to output a ranking list of the candidate responses for multi-turn dialogue reasoning. Experimental results on two public dialogue datasets show that our method obtains higher ranking scores than the baseline models.

## introduction nowadays, the neural dialogue system has achieved high performance and been widely studied in both industry @xcite and academia @xcite . however, 2. the material of this suit is imported from france. it makes the suit special. 3. but the color of our suit is very special. ## related work recently, multi-turn dialogue has gained more attention in both industry @xcite and academia (cho and may, 2020), compared with single-turn dialogue @xcite @xcite . @xcite proposes a hierarchical recurrent encoder-decoder (hred) model which uses the hierarchical encoder-decoder framework to model the relevance of the context and response. @xcite uses hred to model relationships among utterances to enhance the performance of the retrieval-based chatbot. @xcite adds the hierarchical structure and the variable memory network into a neural encoder-decoder network, which can capture both the high-level abstract variations and long-term memories during dialogue tracking. @xcite adopts dynamic and static attention to weigh the importance of each utterance in a conversation and then obtain the contextual representation. @xcite utilizes hierarchical self-attention mechanism to solve the position bias problem of dialogue models. although these models have achieved good performance on datasets like dailydialog @xcite and ecd @xcite , there is still a giant gap between high performance on the leader-board and poor practical user experience @xcite . these models frequently generate responses that are logically incorrect. one possible reason is that the previous models @xcite only solve the cases through linguistic information matching yet lack of logical reasoning. obviously, a dialogue reasoning dataset that can help the model to detect the illogical response is extremely necessary. recently, an open domain multi-turn dialogue reasoning dataset (mutual) @xcite is proposed to facilitate the reasoning capabilities of conversation models. in particular, given a context, it prepares four candidate responses and all of them are relevant to the con-text, but singly one has the correct logic. it requires fine-grained reasoning ability between context and response to make the correct choice. current multiturn dialogue models @xcite @xcite , which perform well on existing benchmarks, have declined on this dataset, which proves that the reasoning ability of these models is still insufficient. therefore, how to enhance the logical reasoning ability of dialogue models is worth discussing. logic consistency of history is essential in human-like behavior. for the multi-choice reasoning task @xcite , there are only slight differences between the illogical response and the dialogue history, so a fine-grained comparison mechanism is required to infer the logic between each candidate response and the dialogue history. previous models @xcite ignore the modeling of fine-grained comparison, which leads to the loss of reasoning ability of their models. in this paper, we overcome this challenge and propose a fine-grained comparison mechanism to perform history consistency and speaker consistency respectively. ## model in this section, we first describe the task definition and then introduce our fcm model in detail, with the architecture shown in figure 1 . our fcm model consists of three components, i.e., contextual encoding, fine-grained comparison module, and response prediction. firstly, we utilize the pre-trained language model bert to encode each token of context and response into a fixed-length vector, which carries contextual information. secondly, we utilize our comparison mechanism to obtain the fine-grained response representation and then compare such obtained representation with both the whole history and the speaker's own history. finally, we fuse consistency representation and semantic information, and then use a linear layer to obtain the candidate response score for the multi-choice prediction process. ## experiments in this section, we conduct experiments on mutual reasoning and ubuntu dialogue datasets to evaluate our proposed method. ## conclusion in this paper, we focus on multi-turn dialogue reasoning tasks and propose the fcm model. the motivation comes from the fact that the widely used dialogue models only focus on the syntactic and semantic relevance but fail to model the logical consistency between the dialogue history and the generated response. this task is challenging because there are only slight differences between the illogical response and the dialogue history. our core idea is to propose a fine-grained comparison mechanism to focus on the fine-grained differences in the representation of each response candidate, and then each candidate representation is compared with the history to obtain a consistency score. in the future, we plan to further investigate the finegrained correlation between different speakers, and utilize this information to help improve our model.