A Compare Aggregate Transformer for Understanding Document-grounded Dialogue (2020)

Abstract: Unstructured documents serving as external knowledge of the dialogues help to generate more informative responses. Previous research focused on knowledge selection (KS) in the document with dialogue. However, dialogue history that is not related to the current dialogue may introduce noise in the KS processing. In this paper, we propose a Compare Aggregate Transformer (CAT) to jointly denoise the dialogue context and aggregate the document information for response generation. We designed two different comparison mechanisms to reduce noise (before and during decoding). In addition, we propose two metrics for evaluating document utilization efficiency based on word overlap. Experimental results on the CMU_DoG dataset show that the proposed CAT model outperforms the state-of-the-art approach and strong baselines.

## introduction dialogue system (ds) attracts great attention from industry and academia because of its wide application prospects. sequence-to-sequence models (seq2seq) @xcite are verified to be an effective framework for the ds task. however, one problem of seq2seq models is that they tended to generate generic responses that provids deficient information @xcite ; @xcite . previous researchers proposed different methods to alleviate this issue. one way is to focus on models' ability to extract information from conversations. @xcite introduced maximum mutual information (mmi) as the objective function for generating diverse response. @xcite proposed a latent variable model to capture posterior information of golden response. @xcite used conditional variational autoencoders to learn discourselevel diversity for neural dialogue models. the document: movie name: the shape of water. year: 2017. director: guillermo del toro. genre: fantasy, drama.cast: sally hawkins as elisa esposito, a mute cleaner who works at a secret government laboratory. ... critical response: one of del toro's most stunningly successful works ... dialogue: s1: i thought the shape of water was one of del toro's best works. what about you? s2: yes, his style really extended the story. s1: i agree. he has a way with fantasy elements that really helped this story be truly beautiful. it has a very high rating on rotten tomatoes, too. s2: sally hawkins acting was phenomenally expressive. didn't feel her character was mentally handicapped. s1: the characterization of her as such was definitely off the mark. other way is introducing external knowledge, either unstructured knowledge texts @xcite ; @xcite ; @xcite or structured knowledge triples @xcite @xcite to help open-domain conversation generation by producing responses conditioned on selected knowledge. the document-grounded dialogue (dgd) @xcite @xcite is a new way to use external knowledge. it establishes a conversation mode in which relevant information can be obtained from the given document. one example of dgd is presented in figure 1 . two interlocutors talk about the given document and freely reference the text segment during the conversation. to address this task, two main challenges need to be considered in a dgd model: 1) determining which of the historical conversations are related to the current conversation, 2) using current conversation and the related conversation history to select proper document information and to gener-ate an informative response. previous work @xcite al. (2019); zhao et al. (2019); qin et al. (2019); tian et al. (2020); ren et al. (2019) generally focused on selecting knowledge with all the conversations. however, the relationship between historical conversations and the current conversation has not been studied enough. for example, in figure 1 , the italics utterance from user1, "yes, his style really extended the story.", is related to dialogue history. while the black fold utterance from user1, "sally hawkins acting was phenomenally expressive. didn't feel her character was mentally handicapped.", has no direct relationship with the historical utterances. when employing this sentence as the last utterance, the dialogue history is not conducive to generate a response. in this paper, we propose a novel transformerbased @xcite model for understanding the dialogues and generate informative responses in the dgd, named compare aggregate transformer (cat). previous research @xcite has shown that the last utterance is the most important guidance for the response generation in the multi-turn setting. hence we divide the dialogue into the last utterance and the dialogue history, then measure the effectiveness of the dialogue history. if the last utterance and the dialogue history are related, we need to consider all the conversations to filter the document information. otherwise, the existence of dialogue history is equal to the introduction of noise, and its impact should be eliminated conditionally. for this purpose, on one side, the cat filters the document information with the last utterance; on the other side, the cat uses the last utterance to guide the dialogue history and employs the guiding result to filter the given document. we judge the importance of the dialogue history by comparing the two parts, then aggregate the filtered document information to generate the response. experimental results show that our model can generate more relevant and informative responses than competitive baselines. when the dialogue history is less relevant to the last utterance, our model is verified to be even more effective. the main contributions of this paper are: ## related work the dgd maintains a dialogue pattern where external knowledge can be obtained from the given document. most recently, some dgd datasets @xcite al. (2018b); moghe et al. (2018); qin et al. (2019); gopalakrishnan et al. (2019 ## conclusion we propose the compare aggregate method to understand document-grounded dialogue (dgd). the dialogue is divided into the last utterance and the dialogue history. the relationship between the two parts is analyzed to denoise the dialogue context and aggregate the document information for response generation. experiments show that our model outperforms previous work in both automatic and manual evaluations. our model can better understand the dialogue context and select proper document information for response generation. we also propose knowledge utilization (ku) and quality of knowledge utilization (qku), which are used to measure the quantity and quality of the imported external knowledge, respectively. in the future, we will further study the topic transition problem and the knowledge injecting problem in the dgd.