Speaker Turn Modeling for Dialogue Act Classification (2021)

Abstract: Dialogue Act (DA) classification is the task of classifying utterances with respect to the function they serve in a dialogue. Existing approaches to DA classification model utterances without incorporating the turn changes among speakers throughout the dialogue, therefore treating it no different than non-interactive written text. In this paper, we propose to integrate the turn changes in conversations among speakers when modeling DAs. Specifically, we learn conversation-invariant speaker turn embeddings to represent the speaker turns in a conversation; the learned speaker turn embeddings are then merged with the utterance embeddings for the downstream task of DA classification. With this simple yet effective mechanism, our model is able to capture the semantics from the dialogue content while accounting for different speaker turns in a conversation. Validation on three benchmark public datasets demonstrates superior performance of our model.

## introduction dialogue acts (das) are the functions of utterances in the context of a dialogue conveying the speaker's intent @xcite . in natural language understanding, da classification is of critical importance, as it underlies various tasks such as dialogue generation @xcite and intention recognition @xcite , thus providing effective means for domains like dialogue systems @xcite , talking avatars @xcite and therapy @xcite @xcite . recent studies of da classification have leveraged deep learning techniques, where promising results have been observed. generally, these methods utilize hierarchical recurrent neural networks 1 our code and data are publicly available at https://github.com/zaghe568/ speak-turn-emb-dialog-act-clf . in this regard, we aim to incorporate the speaker turns into encoding an utterance. specifically, we propose to model speaker turns in conversations and introduce two speaker turn embeddings that are combined with the utterance embeddings. given a conversation containing a sequence of utterances, we first obtain the utterance embeddings using a large pretrained language model roberta @xcite , and extracting the [cls] token embeddings from the last layer; meanwhile we use a speaker turn embedding layer to generate speaker turn embeddings given the speaker labels. the speaker turn embeddings are added to the utterance embeddings to obtain speaker turn-aware utterance representations. these representations are fed into an rnn to encode the context of the conversation, where the output hidden states are used for da classification. we evaluate the proposed method on three benchmark datasets and achieve the state-ofthe-art results, among all inductive learning meth-ods, on two of the datasets. we argue that this simple technique provides effective means for obtaining more powerful representations for dialogue. ## methods the overall framework of our model is shown in figure 1 . in this section, we will describe our model's components in detail. ## conclusion and future work in this paper, we propose a model for encoding speaker turn changes to tackle da classification. specifically, we introduce conversation-invariant speaker turn embeddings and add them to utterance embeddings produced by a pretrained language model. such a simple yet scalable module can be easily added to other models to obtain significantly better results. experiments on three datasets demonstrate the effectiveness of our method. for future work, we will explore transformer encoders @xcite instead of rnns for encod-ing context, since they have shown to be advantageous in performance and training time. our improved representations can be further utilized in other downstream tasks involving dialogue, including speaker intent classification. our findings motivate future work on encoding other interactive aspects of dialogue data into existing text representations.