Integrating User History into Heterogeneous Graph for Dialogue Act Recognition (2020)

Abstract: Dialogue Act Recognition (DAR) is a challenging problem in Natural Language Understanding, which aims to attach Dialogue Act (DA) labels to each utterance in a conversation. However, previous studies cannot fully recognize the specific expressions given by users due to the informality and diversity of natural language expressions. To solve this problem, we propose a Heterogeneous User History (HUH) graph convolution network, which utilizes the user’s historical answers grouped by DA labels as additional clues to recognize the DA label of utterances. To handle the noise caused by introducing the user’s historical answers, we design sets of denoising mechanisms, including a History Selection process, a Similarity Re-weighting process, and an Edge Re-weighting process. We evaluate the proposed method on two benchmark datasets MSDialog and MRDA. The experimental results verify the effectiveness of integrating user’s historical answers, and show that our proposed model outperforms the state-of-the-art methods.

## introduction dialogue acts recognition (dar) is an important but challenging task in natural language understanding (nlu), which aims to attach dialogue act (da) labels to each utterance in a conversation and recognize the speaker's intention. automatic dar can be applied to many applications such as question answering, speech recognition and dialogue systems @xcite . in this work, different from recognizing a single da, we focus on the task of recognizing multiple da in a multi-party conversation (i.e., a forum). the latter is difficult but is more common in the real world. figure 1 shows an example of multiple da recognition in a tech forum. previous studies have proposed deep learning models, which approach dar as a multi-classification problem @xcite or a sequence labeling problem @xcite @xcite . most of these approaches assume that utterances are sequentially organized, ignoring the rich interaction between multiple users in a conversation. to alleviate such a problem, some recent studies exploit graph-structured networks @xcite , which leverage speaker interaction of the interlocutors to model conversational context. however, due to the informality and diversity of natural language expressions, the same intention has a very rich form of expression. in forums where many users participate, it is more common for different users to express their intentions using personalized expressions. when encountering the unobvious intention or uncommon expression pattern, existing deep learning methods that generalize the utterance feature into a low dimensional vector may degrade the performance of da recognition. intuitively, the user's historical expressions extracted according to the da category can be used as a da-specific clue to help the encoding process recognize the user's inexplicable or uncommon expressions. for example, as shown in figure 1 , we intent to recognize three da labels for t 4 . although we can easily label t 4 as greetings/gratitude (gg) simply based on the utterance features (i.e. "thanks"), conversation label t 1 u 1 : how do i install whatsapp on my windows phone 8? oq t 2 u 2 : to install whatsapp on your phone you just download the app from here… p a t 3 u 3 : yes, i want to install it in my phone. rq t 4 u 1 : thanks for responding, how did you download the app? are you sure you can download it? gg, fq, nf h 1 [u 1 ]: there is no way to add this command to the qat. are you sure you can add this command? cq, nf h 2 [u 1 ]: thanks for the reply, but i don't quite understand "incompatible". are you sure we're talking about the same thing? gg, fq, nf h 3 [u 3 ]: i've already contacted msi but they've been fairly useless by suggesting basic things time and time again. they're honestly a waste of time. nf h 4 [u 2 ]: link not working at all. nf dialogue acts taxonomy gg greetings/gratitude p a potential answer oq original question nf negative feedback rq repeat question fq follow-up question cq clarifying question figure 1: a tech forum conversation and sampled user's historical answers labeled as nf. t i [u j ]: the i-th turn of conversation posted by user u j . h i [u j ]: the i-th historical answer of user u j . to detect the follow-up question (fq) label, additional information from t 2 need to be considered. besides, t 4 seems to be a question and there is no explicit negative feedback (nf). however, from the user's historical answers that reflect nf label such as h 1 , h 2 , we find that the phrase "are you sure" indicates a negative feedback that the user doubts about something. inspired by this, we propose a heterogeneous user history (huh) graph convolution networks, which integrates utterance, conversation, and user's historical answers into multiple da recognition. the proposed da recognition process can be divided into two stages. in the 1st-phase, we first extract the hidden features for each utterance by stacking a convolutional neural network (cnn) utterance encoder and a bi-directional long short-term memory (bilstm) utterance context encoder, and we use the hidden features to predict the initial score of da labels for each utterance. in the 2nd-phase, we use the user's historical answers as additional information to construct a heterogeneous user history (huh) graph, and we use a relation-weighted graph convolutional network (rgcn) @xcite to learn this graph and model the interaction between utterance and user's historical answers before recognizing user's intentions. despite the benefits of user's historical answers, improper use may inevitably bring a lot of noise. we thus further design sets of denoising mechanisms. firstly, we use a history selection process with similarity measures to filter out the irrelevant user's historical answers. then, we adopt the learned initial score of da labels from the 1st-phase to re-weight the similarity matrix between the user's historical answers and conversation, and re-weight the edges in the proposed huh graph, thereby reducing the noise caused by introducing supplementary information. the main contributions of our work can be summarized as follows: 1) we propose a novel heterogeneous user history (huh) graph convolution network, which models the interaction between users and integrates utterance, conversation and user's historical answers for recognizing user's intent. to the best of our knowledge, we are the first to integrate user's historical answers into heterogeneous graph on dar. 2) to alleviate the noise issue caused by introducing the user's historical answers, we design sets of denoising mechanisms, including a history selection process, a similarity re-weighting process, and an edge re-weighting process. 3) we evaluate our model on two benchmark datasets msdialog and mrda. compared to the state-of-the-art dar methods, our model achieves better performance. the experimental results verify the effectiveness of incorporating the user's historical answers. ## related work the goal of the dar is to assign da labels to each utterance in a conversation. early studies on dar are mostly based on general statistical machine learning methods and approach this task as a multi-class classification problem or a sequence labeling problem, such as hidden markov model (hmm) @xcite , support vector machines (svm) @xcite and bayesian network @xcite . recent studies on dar have proposed deep learning models and have obtained promising results. deep learning approaches typically model the interaction between adjacent utterances @xcite . some researchers capture the dependencies among both utterances and labels with conditional random field (crf) @xcite @xcite . furthermore, @xcite leverage a sequence to sequence approach to model both the conversation and the global tag dependencies. besides, some researches explore joint models to solve dar and sentiment classification simultaneously in a unified framework @xcite @xcite . however, these methods assume that utterances are sequentially organized, ignoring the rich interaction process between users in a conversation. some recent researches design graph-structured networks to model speaker interaction in a conversation. @xcite first propose a graph-structured network (gsn) to model graph-structured dialogues for response generation. @xcite leverage self and inter-speaker interaction of the interlocutors to model conversational context for emotion recognition. although these methods can play a role, they do not make full use of the user information in the conversation. the recent study @xcite encodes the user's historical answers to boost community question answering (cqa) task. but it only considers a single user and ignores the noise issue caused by introducing multiple users' historical answers. in this paper, we propose a heterogeneous user history (huh) graph convolution networks, which integrates utterance, conversation, and user's historical answers into multiple da recognition. the experimental results verify the effectiveness of integrating user's historical answers, and show that our proposed model outperforms the state-of-the-art methods. ## methodology before describing our proposed model, we first introduce the basic mathematical notions and terminologies for the problem of dar. the task of dar takes a conversation c as input, which contains a sequence of utterances {u t } n @xmath0. for each utterance u t (the t-th utterance) in a conversation, we predict a subset of da labels y @xmath1, y t 2 , ..., y t s } that describes the functionality of the utterance from a candidate set of da labels @xmath2, d 2 , ..., d s }. and y t @xmath3, 1} indicates whether the t-th utterance is labeled with da label d j . for each da label, we use the user's historical answers belonging to that da label to construct a label node. specifically, for the j-th da label d j , we retrieval a series of user's historical answers belonging to this label. we select top-k user's historical answers that are most relevant to the conversation, and sum them by similarity weight to generate a label node e j corresponding to d j . here, k is a hyperparameter. the overall architecture of our proposed model huh is shown in figure 2 . huh mainly contains two phases: 1) the 1st-phase, as shown in the left part, aims to predict the initial score of da labels for each utterance as a guide to the 2nd-phase process. 2) the 2nd-phase, as shown in the right part, constructs a heterogeneous user history graph to integrate utterance, conversation, and user's historical answers into multiple da recognition. additionally, the proposed denoising mechanisms are presented at the corresponding phase in figure 2 . in the following sections, the details of our framework are given. ## conclusion in this paper, we focus on the task of multiple da recognition in a multi-party conversation. we propose a heterogeneous user history (huh) graph convolution network model to learn utterance, conversation and user's historical answers. to handle the noise caused by introducing the user's historical answers, we design sets of denoising mechanisms, including a history selection process, a similarity re-weighting process and an edge re-weighting process. we evaluate the proposed method on two benchmark datasets msdialog and mrda. the experimental results verify the effectiveness of integrating user's historical answers, and show that our proposed model outperforms the state-of-the-art methods.